tiny

# Adversarial Attacks with Time-Scale Representations

Alberto Santamaria-Pang  Jianwei Qiu  Aritra Chowdhury  James Kubricht
Peter Tu  Iyer Naresh  Nurali Virani
GE Research  1 Research Circle  Niskayuna  NY 12309
{santamar, jianwei.qiu, Aritra.Chowdhury, james.kubrich, tu, iyerna,
<EMAIL_ADDRESS>
###### Abstract

We propose a novel framework for real-time black-box universal attacks which
disrupts activations of early convolutional layers in deep learning models.
Our hypothesis is that perturbations produced in the wavelet space disrupt
early convolutional layers more effectively than perturbations performed in
the time domain. The main challenge in adversarial attacks is to preserve low
frequency image content while minimally changing the most meaningful high
frequency content. To address this, we formulate an optimization problem using
time-scale (wavelet) representations as a dual space in three steps. First, we
project original images into orthonormal sub-spaces for low and high scales
via wavelet coefficients. Second, we perturb wavelet coefficients for high
scale projection using a generator network. Third, we generate new adversarial
images by projecting back the original coefficients from the low scale and the
perturbed coefficients from the high scale sub-space. We provide a theoretical
framework that guarantees a dual mapping from time and time-scale domain
representations. We compare our results with state-of-the-art black-box
attacks from generative-based and gradient-based models. We also verify
efficacy against multiple defense methods such as JPEG compression, Guided
Denoiser and Comdefend. Our results show that wavelet-based perturbations
consistently outperform time-based attacks thus providing new insights into
vulnerabilities of deep learning models and could potentially lead to robust
architectures or new defense and attack mechanisms by leveraging time-scale
representations.

##  1 Introduction

While great progress has been made in adversarial attacks [3, 6, 16, 41], the
fundamental reason why convolutional neural networks (CNNs) are fragile to
small perturbations is still unclear [25, 33, 4, 44, 32]. State-of-the-art
models generate perturbations that seem like statistical noise that affect
both high and low image scales (similar to high and low frequency). We believe
that the inability to localize an attack becomes a key limitation of time-
domain attacks as they end up modifying unnecessary or unrelated information
in the image [20, 2]. Thus, instead of generating adversarial attacks in the
time domain (low and high frequencies), we propose to expose a new
vulnerability of deep learning models by generating fine structural-
perturbations in meaningful image content in the high scale Wavelet
decomposition. Studies have shown that early convolution layers capture low-
level edge and edge orientation information possibly similar to receptive
fields of human vision [19, 49]. We aim to induce a perturbation that will
alter the local vertical, horizontal, and diagonal edge content of images,
therefore, creating arbitrary activation spikes in early convolution layers.
We argue that decomposing and isolating localized coherent spatial
arrangements is an effective mechanism for disrupting CNN layer responses. Our
approach relies on the following steps: i) time-scale analysis is used to
provide a mechanism to decompose an image and localize high scale (frequency)
and structural image properties, ii) generative models are used to learn both
high-scale and structural patterns to perturb high scale coefficients, iii)
synthesis of image from perturbed coefficients aims disrupt early convolution
layers. Figure 1 presents a schematic of our proposed method.

Figure 1: Schematic of the proposed Time-Scale adversarial attack.

Since, state-of-the-art deep learning (DL) network designs rely on
convolutional layers that broadly target all high frequency content –
indiscriminate to structural perturbations – making them highly vulnerable to
these attacks [34, 1, 42, 48, 5]. The targeted decomposition of fine image
details accounts for a small fraction of the image energy content [24, 14, 8].
This helps with our objective to minimally perturb the image while inducing
misclassification in classifier that consist of convolution layers. The
significance of our approach is that the concept of scales (wavelet
decomposition levels) and orientations (filter banks) can be associated to one
another to investigate the fundamental principles of image/video analysis and
signal processing for a new type of black-box attack.

##  2 Literature Review

To the best of our knowledge, no work has been reported in attack-based
mechanisms from time-scale representations. All adversarial attack frameworks
reported to date [30, 21, 37, 27, 10, 7, 6, 40, 45, 18, 43], perturb an image
which is represented in the time domain using all of its frequency (Fourier)
content. A detail review can be found in Pitropakis et al. [28]. Black-box
attacks are challenging adversarial attacks as there is no access to training
data nor to the target model. These attacks can be categorized as either
gradient-based or transfer-based methods. Gradient-based methods iteratively
estimate the gradient by perturbing the input image while optimizing an
objective function. Here the major challenge is to account for robust
optimization in a high dimensional input space (see Chen et al. [6] for an
initial report). Ilyas et al. [16] subsequently proposed a natural evolution
attack with a limited number of queries. While their methods showed a
significant improvement in query efficiency and speed, it generated images
with high distortion. Performance was later improved by Cheng et al. [7] who
used a surrogate model as a prior, effectively reducing the number of gradient
averages or queries per iteration.

To alleviate computational cost, transfer-based attacks [29, 35, 36, 12] train
a generator network using white-box attacks from a surrogate or a chosen base
model. Sharif et al. [35] used adversarial generative networks to create
adversarial examples with defined objectives. The method was demonstrated in
applications for face and handwritten-digit recognition. Poursaeed et al. [29]
presented an approach that uses a generative model to design small
perturbations (high frequency) that are distributed in the overall image,
rather than following structural patterns. All of the above methods share the
following limitations: gradient-based methods are prohibitively
computationally expensive when estimating the gradient, while transfer-based
methods suffer from generalization. Our hypothesis is that accounting for
unnecessary and redundant image content in very high multidimensional space
leads to non-tractable optimization problems and poor generalization [9, 42,
46, 26, 39].

Our main contributions are the following: i) providing a novel, tractable,
fast, and black-box (surrogate-based or transfer-based) adversarial attack for
deep neural networks, ii) exploring a new direction to study vulnerability
(and hence robustness) of neural networks via time-scale perturbations that
affects early convolution layers, and iii) experimental study with state-of-
the-art classifiers and other generative or iterative attacks to illustrate
the efficacy and salient features of this attack. Our code is available online
at: https://github.com/waveletgap/wgap.

##  3 Methods

###  3.1 Background

##### Multiresolution Analysis:

We briefly review multiresolution analysis introduced by Mallat and Meyer from
[23]. Let $\varphi$ a function in $L^{2}(\mathbb{R})$ (Hilbert space of
square-integrable complex valued functions on $\mathbb{R}$), such that the
translation $\varphi(\cdot-k),k\in\mathbb{Z}$ defines an orthonormal system.
Then $\varphi$ generates the sequence of spaces $V_{j}:j\in\mathbb{N}$, if
$\varphi_{jk}(x)=2^{j/2}\varphi(2^{j}x-k)$. A multiresolution analysis (MRA)
for $L^{2}(\mathbb{R})$ is a sequence of spaces $\\{V_{j}\\}$ generated by the
orthonormal system $\varphi_{jk}$ in $L^{2}(\mathbb{R})$, if $V_{j}\subset
V_{j+1}$ and $\bigcup V_{j}$ is dense in $L^{2}(\mathbb{R})$. We say that
$\varphi$ is a father wavelet or scaling function if $\varphi$ generates a
MRA. Given $f\in L^{2}(\mathbb{R})$, it can be shown that
$f\in\bigcup_{j=0}^{\infty}V_{j}=V_{0}\oplus\bigoplus_{j=0}^{\infty}W_{j}$,
where $W_{j}$ is the orthogonal complement of $V_{j}$ in $V_{j+1}$. We refer
to $W_{j}$ as $j^{th}$ resolution level of the multiresolution analysis of
level $j$. A mother wavelet is a function $\psi\in W_{0}$ such that
$\psi_{0k}=\psi(\cdot-k)$. Then $\psi$ is an orthonormal basis in $W_{j}$ and
is orthogonal to the father wavelet. Then any $f\in L^{2}(\mathbb{R})$ can be
represented in terms of the father and mother wavelet via the following
series:

$f(x)=\sum_{k}\alpha_{k}\varphi_{0k}(x)+\sum_{j=0}^{\infty}\sum_{k}\beta_{jk}\psi_{jk}(x),$
(1)

where the wavelet coefficients are defined as: $\alpha_{k}=\int
f(x)\overline{\varphi_{0k}(x)}dx,~{}~{}\beta_{jk}=\int
f(x)\overline{\psi_{jk}(x)}dx$. The main advantage of the wavelet
representation in Eq. 1 over Fourier analysis is the ability to capture both
the spatial component and the frequency spectrum. The term
$\sum_{k}\alpha_{k}\varphi_{0k}(x)$ is the spatial component whereas the term
$\sum_{j=0}^{\infty}\sum_{k}\beta_{jk}\psi_{jk}(x)$ localizes the frequency
spectrum. The coefficients $\alpha_{k}$ inherently capture the main signal
content while $\beta_{jk}$ capture the local details. We refer to $\alpha_{k}$
and $\beta_{jk}$ as the approximation and detail coefficients, respectively.

##### 2D Discrete Wavelet Transform (DWT):

The 2D Discrete Wavelet Transform (DWT) is a direct generalization of the 1D
MRA into $L^{2}(\mathbb{Z}^{2})$. Let $\varphi$ be a scaling function and
$\psi$ its corresponding wavelet. We define three wavelets:
$\psi^{1}=\varphi\psi$, $\psi^{2}=\psi\varphi$, and $\psi^{3}=\varphi\varphi$,
where
$\psi_{j,(n_{1},n_{2})}^{k}(t_{1},t_{2})=2^{j/2}\psi^{k}((2^{j}n_{1}-t_{1})/2^{j},(2^{j}n_{2}-t_{2})/2^{j})$
with $k\in\\{1,2,3\\}$. Then the wavelet family
$\\{\psi_{j,n}^{1},\psi_{j,n}^{2},\psi_{j,n}^{3}\\}_{n\in Z^{2}}$ with
$n=(n_{1},n_{2})$ is an orthonormal basis for $W_{j}^{2}$ and
$\\{\psi_{j,n}^{1},\psi_{j,n}^{2},\psi_{j,n}^{3}\\}_{j,n\in Z^{2}}$ is an
orthonormal basis for $L^{2}(\mathbb{Z}^{2})$. The corresponding coefficients
are obtained from the inner products: $a_{j}[n]=\langle
f,\varphi_{j,n}^{2}\rangle,~{}d_{jk}[n]=\langle
f,\psi_{j,n}^{k}\rangle,\;\forall k\in\\{1,2,3\\}$. The approximation
coefficients $a_{j}$ are associated with low frequencies, whereas the detail
coefficients $d_{jk}$ are associated with high frequencies at horizontal,
vertical, and diagonal orientations. The scaling function $\varphi$ and the
mother wavelet $\psi$ can be represented as conjugate mirror filters banks
$h[n]$ and $g[n]$. We denote as $h_{d}[n]=h[-n]$ and $g_{d}[n]=[-g]$ the
mirror filters of $h[n]$ and $g[n]$, respectively. Figure 2 illustrates two-
dimensional DWT for the first decomposition level.

Figure 2: Schematic of the 2D wavelet transform for an RGB image $x$ showing
associated filter banks for analysis and synthesis. The input image is
projected into $P_{V_{0}^{2\times 3}}$ and $P_{W_{0}^{2\times 3}}$.

###  3.2 Wavelet-based Generative Adversarial Perturbation (WGAP) Modeling

We present a generic framework for black-box and transfer attacks by
perturbing the projection $P_{W_{j_{0}}}(x)$ (details) and preserving the
projection $P_{V_{j_{0}}}(x)$ (approximation) for an image $x$. The analysis
is shown for single channel images $L^{2}(\mathbb{Z}^{2})$, but is easily
generalized to three-channel color images. Since it has been observed that
early convolution layers capture low-level edge and edge orientation
information, we aim to induce a perturbation in $P_{W_{j_{0}}}(x)$ such that
it will alter the local vertical, horizontal, and diagonal edge content of
$x$, therefore, creating arbitrary activation spikes in early convolution
layers. This method systematically alters the edges, thereby influencing
shape-content information. This approach is different from denoising since
noise distribution is not explicitly modeled. Rather, object structure from
finer details is directly altered. We also deviate from intervention methods
in the high-frequency Fourier spectrum since Fourier transformations lose
spatial content coherence in contrast with time- and space-scale
representations used in this work.

####  3.2.1 General Formulation

Suppose, we have a classiﬁer $c(\cdot)$ with a loss function $\ell_{c}(X,y)$
where $X$ and $y$ is the input image and output class label, respectively.
Here $c(X)$ is the predicted label. We assume: i) an adversarial example can
be derived from an additive model as follows $X^{\prime}=X+D$, and ii) for a
fixed class label $y$, the loss $\ell_{c}$ is a continuous function. Then
adversarial attack generation can be posed as an optimization problem:

$\operatorname*{arg\,max}_{X^{\prime}:\lVert
X^{\prime}-X\rVert\leq\epsilon}\ell_{c}\big{(}X^{\prime},c(X)\big{)}.$ (2)

Please note that: i) if $c(\cdot)$ is the attacked model then it is a white-
box attack, ii) if it is a surrogate of the model to be attacked, then it is a
black-box attack, and iii) if it is a chosen base model and the model to be
attacked is unknown, then it is a transfer attack. Based on this formulation,
in this work, we are interested in transfer and black-box attacks with
perturbation in the wavelet domain. To convince that such an attack is sound,
we look at the following result.

Theorem 1: Let $(a_{j},d_{jk})$, $(a_{j},d^{\prime}_{jk})$ be wavelet
coefficients for $X$ and $X^{\prime}$, such that
$X=\text{DWT}^{-1}(a_{j},d_{jk})$ and
$X^{\prime}=\text{DWT}^{-1}(a_{j},d^{\prime}_{jk})$. For some scale $j_{0}$,
we can express problem in Equation 2 in terms of the wavelet coefficients as
follows:

$\operatorname*{arg\,max}_{\text{DWT}^{-1}(a_{j_{0}},d^{\prime}_{j_{0}k}):\lVert
d_{j_{0}k}-d^{\prime}_{j_{0}k}\rVert\leq\tilde{\epsilon}}\ell_{c}\big{(}X^{\prime},c(X)\big{)}.$
(3)

###### Proof.

We need to prove that if $\lVert X^{\prime}-X\rVert<\epsilon$, then there
exists $\tilde{\epsilon}$, such that $\lVert
d_{j_{0}k}-d^{\prime}_{j_{0}k}\rVert<\tilde{\epsilon}$ for any chosen scale
$j_{0}$. Given that $X$ and $X^{\prime}$ have the same approximation
coefficients, from Equation 1, $D=X^{\prime}-X$ is in $W_{j}^{2}$ but not in
$V_{0}$. Then using, linearity, orthonormality and the Pythagorean identity,
we can write $\lVert X-X^{\prime}\rVert$ as:

$\lVert\sum_{j=0}^{\infty}\sum_{k}d_{jk}\psi_{jk}(X)-\sum_{j=0}^{\infty}\sum_{k}d^{\prime}_{jk}\psi_{jk}(X^{\prime})\rVert=$
(4)
$\lVert\sum_{j=0}^{\infty}\sum_{k}(d_{jk}-d^{\prime}_{jk})\psi_{jk}(X-X^{\prime})\rVert<$
(5) $\sum_{j=0}^{\infty}\sum_{k}\lVert
d_{jk}-d^{\prime}_{jk}\rVert\psi_{jk}(X-X^{\prime})\leq$ (6) $\sup_{j,k}\lVert
d_{jk}-d^{\prime}_{jk}\rVert=\lVert
d_{j_{0}k}-d^{\prime}_{j_{0}k}\rVert=\tilde{\epsilon}$ (7)

Finally, the DWT is providing the dual mapping of the signal representation
from the estimated detailed coefficients $d^{\prime}$ within $W_{j}^{2}$
respect to the loss function $\ell_{c}$. ∎

The intuition behind the previous formulation is that the dual space can
create a new adversarial image by just perturbing the details coefficients
corresponding to a scale level (resolution) of the original image. The
significance of this theoretical result is that i) adversarial attacks could
be further targeted to an optimal resolution (scale and frequency) and ii)
makes no assumption on the attack type since results applies to black-box,
white-box and class-targeted attacks. Next, we will demonstrate a formulation
for black-box and generative models.

####  3.2.2 Wavelet-based Generative Adversarial Perturbations

Poursaeed et al. [29] proposed a Generative Adversarial Perturbation (GAP)
method to minimize cross entropy loss, targeting the least likely class for an
input image and surrogate model $c$. Here, noise (from uniform distribution)
is passed to a generator network to create a perturbation. Then, the output of
the generator is normalized based on a maximum magnitude of perturbation and
added to the original image to generate $X^{\prime}$.

Time Domain Formulation: We formulate a novel universal loss function for
universal black-box attacks: we do not use noise models and only use a
surrogate model $c$. We propose a conditional loss function based on a budget
parameter. This parameter controls the magnitude of the image perturbation.
Comparing to normal cross entropy loss, the key difference is that we optimize
the attack rate (fooling ratio) while keeping the amount of perturbation
within certain budget $\epsilon$. If perturbations exceeds the budget
$\epsilon$, an additional penalty term will be added to the total loss. The
same concept can be applied to gradient-based methods and targeted attacks.
For a given budget $\epsilon$, the time domain representation of the loss
function is:

$\small{\ell_{c}=\begin{cases}\mathcal{H}\big{(}p(X^{\prime}),\mathbb{I}_{p_{ll(X)}}\big{)}+l*\mathcal{D}(X^{\prime},X),&\text{if
}\lVert X-X^{\prime}\rVert>\epsilon\\\
\mathcal{H}\big{(}p(X^{\prime}),\mathbb{I}_{p_{ll(X)}}\big{)}&\text{otherwise}\end{cases}}$
(8)

The first term $\mathcal{H}$ is the cross-entropy operator, where
$\mathbb{I}_{y}$ denotes the one-hot encoding for the class label $y$,
probability $p(X^{\prime})$ is output probability for perturbed sample, and
$p_{ll(X)}=\operatorname*{arg\,min}{p(X)}$ is the least likely class given the
original image $X$ based on model $c$. For the second term, $l$ is a parameter
to regulate the loss, $\mathcal{D}$ the structural dissimilarly:

$\mathcal{D}(X^{\prime},X)=1-{\frac{(2\mu_{X^{\prime}}\mu_{X}+c_{1})(2\sigma_{X^{\prime}X}+c_{2})}{(\mu_{X^{\prime}}^{2}+\mu_{X}^{2}+c_{1})(\sigma_{X^{\prime}}^{2}+\sigma_{X}^{2}+c_{2})}},$
(9)

where $\mu_{X}^{\prime}$, $\sigma_{X}^{\prime}$ represent the mean and
covariance for $X^{\prime}$ (similarly for $X$), $\sigma_{X^{\prime}X}$ is the
covariance of $X^{\prime}$ and $X$. The values for $c_{1}$, $c_{2}$ stabilize
the division with weak denominator.

Dual Formulation: We reformulate the previous time-domain attack. From Theorem
1, given a budget $\epsilon$, there is a scale $j_{0}$ so that
$X=\text{DWT}^{-1}(a_{j_{0}},d_{j_{0}k})$ and
$X^{\prime}=\text{DWT}^{-1}(a_{j_{0}},d^{\prime}_{j_{0}k})$. The, the terms
for Equation 8 from the following terms:

$\mathcal{H}\big{(}p(\text{DWT}^{-1}(a_{j_{0}},d^{\prime}_{j_{0}k}),\mathbb{I}_{p_{ll(\text{DWT}^{-1}(a_{j_{0}},d_{j_{0}k}))}}\big{)},$
(10)

and

$\mathcal{D}(\text{DWT}^{-1}(a_{j_{0}},d^{\prime}_{j_{0}k}),\text{DWT}^{-1}(a_{j_{0}},d_{j_{0}k})),$
(11)

where, $k\in\\{1,2,3\\}$. Similarly for the budget $\epsilon$ is estimated
from $\text{DWT}^{-1}$. We show how we can use the notion of scale to
constraint the magnitude of the perturbation (budget). In our implementation,
we fixed a scale $j_{0}$, and we apply DWT to obtain the approximation and
detail coefficients $a_{j_{0}}$ and $d_{j_{0}k}$. We only pass the detail
coefficients $d_{j_{0}k}$ to the generator and we reconstruct $X^{\prime}$
using inverse DWT from the original approximation coefficients $a_{j_{0}}$ and
the new perturbed coefficients $d^{\prime}_{j_{0}k}$. Note that we are
inherently normalizing the perturbed coefficients relative to the scale
$j_{0}$ and this is possible because that approximation coefficients contain
most of the energy. Figure 3 shows a schematic of the proposed method.

Figure 3: Overview of the proposed adversarial attack.

##  4 Results

We demonstrate model transferability of our Wavelet-based Generative
Adversarial Perturbation (WGAP) method and compare performance with the
generative model GAP [29] and a gradient-based method [7]. The ILSVRC12
(ImageNet-1000) dataset [31] was used for training, and 1000 randomly selected
ImageNet images [7] were used for evaluation. The Generative model for GAP
[29] and WGAP is a ResNet generator with 6 ResNet blocks, except the size of
the input and output channels are different. The surrogate model is
Inception-v3. To implement the discrete wavelet transform (DWT), we used the
Daubechies wavelet filter bank db2 [8] at scale levels: 1, 2, 3 and budgets
$\epsilon_{1}=0.05$, $\epsilon_{2}=0.1$, $\epsilon_{3}=0.25$. The selection of
the Daubechies wavelet is due to stable numerical approximation and compact
representation. For the WGAP models, nine input and output channels were used
due to three concatenated detail coefficients. For the first convolution layer
of the generative model, generative filter kernel size was set to 7 with a
kernel number of 64. For the second and third convolution layers, the kernel
size was 3 with a kernel number of 128. For each of the ResNet block, the
kernel size as set to 3 a kernel number of 256. Batch normalization and ReLu
activation function was used along with each convolution layer. The Adam
optimization function was used with learning rate 0.0002 and beta parameters
0.5 and 0.999. The GAP and WGAP models were trained for 100 epochs with 500
iterations each. For GAP, we used magnitude value of 10. The models were then
evaluated using a conventional NVIDIA V100-32G GPU and implemented in Pytorch.
Pretrained are from the TorchVision library and https://github.com/thu-
ml/Prior-Guided-RGF.

###  4.1 Attack and defense comparison with generative-based models

To evaluate black-box attack transferability (no access to training data, nor
the target model), we implemented adversarial attacks using Inception-v3 as a
surrogate model. Testing models include ResNet-50 [13], VGG-16 [38]. To
evaluate attack and defense, we use three state-of-the-art defense approaches:
JPEG compression [11], Guided Denoiser [22], Randomization [47].

Table 1 shows fooling ratio in normal and defense performance for GAP [29], a
modified GAP formulation with budget constraint (no wavelet integration) and
WGAP. Overall, our model WGAP achieved best performance for scale level 1 and
lowest for scale level 3. Interestingly, GAP showed best performance when
using the testing networks ResNet-50, VGG-16 than the surrogate network
Inception-v3. Best performance was obtained by WGAP, scale 1, across all
budgets. Second-best performance was obtained by WGAP-J2($\epsilon_{k=1,2,3}$)
, and similar performance was obtained with GAP and
WGAP-J3($\epsilon_{k=1,2,3}$) . It is worth noting that, for a fixed budget,
there is a decrease in fooling ratio performance from
WGAP-J1($\epsilon_{k=1,2,3}$) (level 1) to WGAP-J3($\epsilon_{k=1,2,3}$)
(level 3) and GAP($\epsilon_{k=1,2,3}$) (no wavelet). This suggests that
defense approaches may not be tuned to overcome attacks from targeted
frequency perturbations.

Attacks in Normal Models Attacks in Defense Models Method Inception-v3
ResNet-50 VGG-16 JPEG Compression Randomization Guided Denoiser GAP 69.4%
91.8% 90.3% 13.0% 14.1% 1.8 % GAP($\epsilon_{1}$) 84.4% 89.5% 90.8% 10.7%
12.3% 13.4% WGAP-J1($\epsilon_{1}$) 91.1% 97.5% 98.5% 79.2% 73.5% 57.8%
WGAP-J2($\epsilon_{1}$) 81.1% 96.5% 97.3% 69.1% 68.8% 33.9%
WGAP-J3($\epsilon_{1}$) 34.2% 84.4% 88.7% 30.7% 30.3% 13.6%
GAP($\epsilon_{2}$) 92.6% 94.5% 97.1% 15.1% 17.5% 18.3%
WGAP-J1($\epsilon_{2}$) 97.6% 98.9% 99.2% 96.2% 85.5% 86.9%
WGAP-J2($\epsilon_{2}$) 85.4% 97.1% 99.3% 77.9% 76.0% 61.1%
WGAP-J3($\epsilon_{2}$) 55.4% 89.1% 95.7% 51.1% 47.0% 22.8%
GAP($\epsilon_{3}$) 95.1% 98.6% 99.6% 23.3% 25.1% 29.8%
WGAP-J1($\epsilon_{3}$) 99.4% 99.8% 99.7% 99.7% 93.6% 89.7%
WGAP-J2($\epsilon_{3}$) 92.2% 98.0% 99.3% 87.8% 83.6% 68.9%
WGAP-J3($\epsilon_{3}$) 56.3% 92.4% 95.7% 53.0% 44.5% 30.7%

Table 1: Fooling ratio performance comparison for attacks in normal and
defense models. Inception-v3 is the surrogate model and the others are testing
models. GAP is the method [29] with magnitude value of 10, WGAP for scales
J1-J3 and budgets $\epsilon_{1}$=0.05, $\epsilon_{2}$=0.1,
$\epsilon_{3}$=0.25.

###  4.2 Attack and defense comparison for iterative (query-based) models

We further evaluate the transferability of the WGAP model by comparing
performance with the Prior-guided Random Gradient-Free (P-RGF) [7]. We use the
optimal parameter $\lambda^{*}$ and ResNet-152 as surrogate model [7]. The
fooling ratio values for attack in normal models Inception-v3, ResNet-50 and
VGG-16 are 91.1%, 99.6% and 97.7% respectively. Similarly, the number of
iterations for the same models are: 649, 352 and 370. Fooling ratio for
attacks in defense models JPEG Compression, Randomization and Guided Denoiser
are 81.1%, 82.3% and 89.5% respectively. Whereas for number of iterations we
obtained 2120, 1816 and 1784. As for the number of iterations, the optimized
P-RGFD achieved 99.1% in 649 iterations and the average time required to
generate an adversarial attack was 15.15 seconds. While, P-RGFD shows very
high performance in attacks for both normal and defense models, the
computationally complexity could be highly prohibitive to use in real time
scenarios. Experimentally, we estimated an average time to produce an
adversarial example, which was 209.21 seconds per image for P-RGFD. The most
similar performance in a defense network was in the Guided Denoiser2 with a
difference of 1.4%. Here, the P-RGF required 1816 iterations, approximately
four times greater than the average number of iterations in normal models. It
should be noted that the P-RGF requires access to the target model to
iteratively estimate the gradient. While this may be useful in attacks where a
model is available, there is a risk of generalizing and over fitting. Next, we
provide an analysis to compare universal full black-box attacks and defense.

| P-RGFD ($\lambda^{*}$) | GAP | WGAP-J1($\epsilon_{1}$) | WGAP-J2($\epsilon_{1}$) | WGAP-J1($\epsilon_{2}$) | WGAP-J2($\epsilon_{2}$)
---|---|---|---|---|---|---
Comdefend | 24.5% | 28.3% | 65.3% | 77.5% | 62.1% | 69.5%
L2 Norm (%) | 1.4 % | 3.3% | 4.2% | 4.8 % | 5.1% | 5.4 %
Table 2: Performance comparison when the defense Comdefend method for fooling
ratio and L2 Norm percentage.

###  4.3 Transfer attacks and defense comparison

We compare model transferability from all approaches for black-box
attack/defense and image error approximation in terms of the L2 norm. We test
the state-of-the-art defense method Comdefend [17] against the generated
images from the surrogate models: i) Inception-v3 for GAP and WGAP and ii)
ResNet-152 for P-RGF [7]. This will provide an independent test on how general
methods could be. Please note that for the iterative method, P-RGF we do not
make any new query, we just used the generated image. Results are presented in
Table 2. The first row shows the fooling ratio from all models (higher is
better), second row shows the estimated L2 norm (lower is better). We note
that P-RGF and GAP corresponds to the poorest performance when estimating
fooling ratio: 24.5% and 28.3% respectively. However, the L2 norm values 1.4%
and 3.3% for P-RGF and GAP are the lowest. For WGAP-based fooling ratio, the
values for WGAP-J1($\epsilon_{1}$) and WGAP-J2($\epsilon_{1}$) were 65.3% and
77.5% respectively, while for WGAP-J1($\epsilon_{2}$) and
WGAP-J2($\epsilon_{2}$) the fooling ratio values were 62.1% and 69.1%
respectively. We note that while there was a significance increase in
performance from budget $\epsilon_{1}$=0.05 to $\epsilon_{2}$=0.1, there was
not a significant increase in L2 norm. For WGAP, we noticed that the L2 norm
error was close to 5% even the budget was close to 10%. This may because the
network prioritizes perturbations with no major penalty to the budget.
However, there may be significant qualitative visual differences between
budgets. We note that the fooling ratio increases as wavelet coefficients go
from details to approximations. This indicates that details (high) frequency
perturbation patterns may be most meaningful when disrupting early
convolutional layers from the network.

Figure 4 and Figure 5 depict example adversarial and residual images for each
model (we omit other models due to space constraints). Column one in Figure 4,
shows original images with corresponding class label. Column two and three
corresponds to P-RGF [7], and GAP [29] respectively. Columns four and five
corresponds to our proposed method with WGAP-J1($\epsilon_{1}$) and
WGAP-J1($\epsilon_{2}$) with scale 1 and budget $\epsilon_{1}$=0.05,
$\epsilon_{2}$=0.1.

Figures 4(b,c) and Figures 4(g,h) show an example where the defense network
Comdefend, correctly classified the adversarial image. Figures 4(d,e)
incorrectly classified the original image, with class name Thimble and Bucket.
For Figure 4(l), while the defense network incorrectly classified the original
image as oxygen mask, the prediction is semantically similar to the original
category (gasmask). Figures 4(n,o) show the predictions from our method
corresponding to Strainer and Zebra. We notice that the most semantics
difference are in relation to the budget.

(a) Lipstick

(b) Lipstick

(c) Lipstick

(d) Thimble

(e) Bucket

(f) Pizza

(g) Pizza

(h) Pizza

(i) Screw

(j) Rock python

(k) Gasmask

(l) Oxygen mask

(m) Comic book

(n) Strainer

(o) Zebra

Figure 4: Adversarial examples from adversarial attack methods. First column
((a),(f),(k)) corresponds to the original image . Second column ((b),(g),(l))
corresponds to the adversarial example generated by the P-RGF method with
optimal parameter $\lambda^{*}$. Third column ((c),(h),(m)) corresponds to the
GAP method, a parameter value of 10 was set as proposed in the original
method. Fourth ((d),(i),(n)) and fifth ((e),(j),(o)) columns corresponds to
our proposed method WGAP-J1($\epsilon_{1}$) and WGAP-J1($\epsilon_{2}$) , with
budget $\epsilon_{1}=0.05$ and $\epsilon_{1}=0.1$ for L2 norm respectively.

Images in Figures 5(a,f,k) show the original images. Images in Figures
5(b,g,l) show the residual for the P-RGF method. We can observe minimum
perturbation across all images and without sign of a defined pattern across
any image. Figures 5(c,h,m) show the result of the perturbation when GAP
method was applied. A uniform pattern can be observed across the image. This
is due to the outcome of the universal attack. Figures 5(d,i,n) and Figures
5(e,j,o) show the perturbations from our method at scale 1 and budgets 0.05
and 0.1, respectively. In contrast with the other two methods, we can observe
that the perturbation pattern is heterogenous and non-uniformly distributed
across the image. The perturbations is minimum at the background, either
uniform (Figures 5(d,e) or Figures 5(i,j)) or heterogeneous (Figures 5(n,o)).
Also, we note that the perturbations are consistently higher at the edges (as
expected) and seems to increase for larger budget. Similarly, we observe some
‘texture-like’ patterns within the object of interest distributed across the
foreground and backgrounds which seems to adapt according to the complexity of
the image texture. This may indicate adaptive changes in the fine detail
structure within the image according to the budget. For example, in Figure
5(d) there are 6 ‘texture-like’ patterns, whereas in Figure 5(e) seems there
are approximately 28 ‘texture-like’ patterns distributed in the foreground.
Similar ‘texture-like’ patterns are found in the residuals relative to the
other images. While this effect, will require more investigation, our
experimental results may suggest that the frequency spectrum is affected
differently depending on the scale, budget and object of interest.

(a) Lipstick

(b) Lipstick

(c) Lipstick

(d) Thimble

(e) Bucket

(f) Pizza

(g) Pizza

(h) Pizza

(i) Screw

(j) Rock python

(k) Gasmask

(l) Oxygen mask

(m) Comic book

(n) Strainer

(o) Zebra

Figure 5: Residual images from adversarial attack methods. First column
((a),(f),(k)) corresponds to the original image . Second column ((b),(g),(l))
corresponds to the adversarial example generated by the P-RGF method with
optimal parameter $\lambda^{*}$. Third column ((c),(h),(m)) corresponds to the
GAP method. Fourth ((d),(i),(n)) and fifth ((e),(j),(o)) columns corresponds
to our proposed method WGAP-J1($\epsilon_{1}$) and WGAP-J1($\epsilon_{2}$) ,
with budget $\epsilon_{1}=0.05$ and $\epsilon_{1}=0.1$ for L2 norm
respectively.

##  5 Conclusions

We presented a new formulation for black-box attacks via time-scale (wavelets)
analysis. Our approach performs adversarial attacks where there is no access
to model nor dataset on which the target model has been trained. We make no
assumptions about the image type or the target model and we provide extensive
validation comparing our approach with two state-of-the-art adversarial
attacks methods and four state-of-the-art defense algorithms. Our results
suggest that is it is both feasible and effective to localize coherent spatial
arrangements as a mechanism to disrupt CNN layer responses.

Our effort provides the first mathematical framework in terms of image
analysis and signal processing for creating attacks to understand underlying
vulnerability of convolution layers in neural networks. Even though time
domain attacks are capable of effectively disrupting CNNs, it is difficult to
interpret and systematically study the nature of the perturbations themselves.
Moreover, by producing attacks from the perspective of wavelets, we seek to
inspire research in academic community to look at decades of research in
signal processing on wavelet transforms to study neural network properties and
explore new architectures. This will in turn help us build up a general theory
of how and why certain scales, orientations and frequency sub-bands affect
neural networks. This will potentially lead to an understanding of CNN’s
vulnerability, providing the broader community with mathematical conditions to
avoid unfavorable architectural choices.

To the best of our knowledge this is the first time that time-scale analysis
has been used for adversarial attacks. The systematic analysis of the wavelet-
based attacks will further provide a signal processing framework for
understanding fundamental reasons for the brittleness in state-of-the-art DL
models. The current research will be extended to provide the community with
mathematical conditions to improve DL architectures and avoid choices that
beget model fragility.

## References

  * [1] Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song.  Practical Black-Box Attacks on Deep Neural Networks Using Efficient Query Mechanisms.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, volume 11216, pages 158–174. Springer International Publishing, Cham, 2018.  Series Title: Lecture Notes in Computer Science.
  * [2] Ali Borji.  Pros and cons of GAN evaluation measures.  Computer Vision and Image Understanding, 179:41–65, Feb. 2019.
  * [3] Thomas Brunner, Frederik Diehl, Michael Truong Le, and Alois Knoll.  Guessing Smart: Biased Sampling for Efficient Black-Box Adversarial Attacks.  pages 4958–4966, 2019.
  * [4] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin.  On Evaluating Adversarial Robustness.  arXiv:1902.06705 [cs, stat], Feb. 2019.  arXiv: 1902.06705.
  * [5] Nicholas Carlini and David Wagner.  Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods.  In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec ’17, pages 3–14, Dallas, Texas, USA, Nov. 2017. Association for Computing Machinery.
  * [6] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh.  ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models.  Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security - AISec ’17, pages 15–26, 2017.  arXiv: 1708.03999.
  * [7] Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu.  Improving Black-box Adversarial Attacks with a Transfer-based Prior.  arXiv:1906.06919 [cs, stat], Oct. 2019.  arXiv: 1906.06919.
  * [8] I. Daubechies.  The wavelet transform, time-frequency localization and signal analysis.  IEEE Transactions on Information Theory, 36(5):961–1005, Sept. 1990\.  Conference Name: IEEE Transactions on Information Theory.
  * [9] Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel.  On the (Statistical) Detection of Adversarial Examples.  arXiv:1702.06280 [cs, stat], Oct. 2017.  arXiv: 1702.06280.
  * [10] Chuan Guo, Jared S. Frank, and Kilian Q. Weinberger.  Low Frequency Adversarial Perturbation.  arXiv:1809.08758 [cs], July 2019.  arXiv: 1809.08758.
  * [11] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten.  Countering Adversarial Images using Input Transformations.  arXiv:1711.00117 [cs], Jan. 2018.  arXiv: 1711.00117.
  * [12] Jamie Hayes and George Danezis.  Learning Universal Adversarial Perturbations with Generative Models.  arXiv:1708.05207 [cs, stat], Jan. 2018.  arXiv: 1708.05207.
  * [13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  pages 770–778, 2016.
  * [14] Christopher Heil and Gitta Kutyniok.  The Homogeneous Approximation Property for wavelet frames.  Journal of Approximation Theory, 147(1):28–46, July 2007.
  * [15] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger.  Densely Connected Convolutional Networks.  arXiv:1608.06993 [cs], Jan. 2018.  arXiv: 1608.06993.
  * [16] Andrew Ilyas, Logan Engstrom, and Aleksander Madry.  Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors.  arXiv:1807.07978 [cs, stat], Mar. 2019.  arXiv: 1807.07978.
  * [17] Xiaojun Jia, Xingxing Wei, Xiaochun Cao, and Hassan Foroosh.  ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples.  pages 6084–6092, 2019.
  * [18] Ameya Joshi, Amitangshu Mukherjee, Soumik Sarkar, and Chinmay Hegde.  Semantic adversarial attacks: Parametric transformations that fool deep classifiers.  In Proceedings - 2019 International Conference on Computer Vision, ICCV 2019, pages 4772–4782. Institute of Electrical and Electronics Engineers Inc., Oct. 2019.
  * [19] Saeed Reza Kheradpisheh, Masoud Ghodrati, Mohammad Ganjtabesh, and Timothée Masquelier.  Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition.  Scientific Reports, 6, Sept. 2016.
  * [20] Xin Li and Fuxin Li.  Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics.  In 2017 IEEE International Conference on Computer Vision (ICCV), pages 5775–5783, Oct. 2017.  ISSN: 2380-7504.
  * [21] Yingwei Li, Song Bai, Cihang Xie, Zhenyu Liao, Xiaohui Shen, and Alan L. Yuille.  Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses.  arXiv:1904.00979 [cs], Apr. 2019.  arXiv: 1904.00979.
  * [22] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu.  Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser.  pages 1778–1787, 2018.
  * [23] Stephane Mallat.  A Wavelet Tour of Signal Processing : Stephane Mallat : 9780123743701\.
  * [24] Stéphane Mallat.  Understanding Deep Convolutional Networks.  Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065):20150203, Apr. 2016.  arXiv: 1601.04920.
  * [25] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard.  Universal adversarial perturbations.  arXiv:1610.08401 [cs, stat], Mar. 2017.  arXiv: 1610.08401.
  * [26] Konda Reddy Mopuri, Aditya Ganeshan, and R. Venkatesh Babu.  Generalizable Data-Free Objective for Crafting Universal Adversarial Perturbations.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(10):2452–2465, Oct. 2019.  Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [27] Konda Reddy Mopuri, Utsav Garg, and R. Venkatesh Babu.  Fast Feature Fool: A data independent approach to universal adversarial perturbations.  arXiv:1707.05572 [cs], July 2017.  arXiv: 1707.05572.
  * [28] Nikolaos Pitropakis, Emmanouil Panaousis, Thanassis Giannetsos, Eleftherios Anastasiadis, and George Loukas.  A taxonomy and survey of attacks against machine learning.  Computer Science Review, 34:100199, Nov. 2019.
  * [29] Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie.  Generative Adversarial Perturbations.  arXiv:1712.02328 [cs, stat], July 2018.  arXiv: 1712.02328.
  * [30] Aaditya Prakash, Nick Moran, Solomon Garber, Antonella DiLillo, and James Storer.  Deflecting Adversarial Attacks With Pixel Deflection.  pages 8571–8580, 2018.
  * [31] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  arXiv:1409.0575 [cs], Jan. 2015.  arXiv: 1409.0575.
  * [32] Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes.  ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models.  arXiv:1806.01246 [cs], Dec. 2018.  arXiv: 1806.01246.
  * [33] Pouya Samangouei, Maya Kabkab, and Rama Chellappa.  Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models.  arXiv:1805.06605 [cs, stat], May 2018.  arXiv: 1805.06605.
  * [34] Sayantan Sarkar, Ankan Bansal, Upal Mahbub, and Rama Chellappa.  UPSET and ANGRI : Breaking High Performance Image Classifiers.  arXiv:1707.01159 [cs], July 2017.  arXiv: 1707.01159.
  * [35] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein.  Adversarial training for free!  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d\textquotesingle Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 3358–3369. Curran Associates, Inc., 2019.
  * [36] Ali Shafahi, Mahyar Najibi, Zheng Xu, John Dickerson, Larry S. Davis, and Tom Goldstein.  Universal Adversarial Training.  arXiv:1811.11304 [cs], Nov. 2019.  arXiv: 1811.11304.
  * [37] Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter.  A General Framework for Adversarial Examples with Objectives.  ACM Transactions on Privacy and Security, 22(3):1–30, July 2019\.
  * [38] Karen Simonyan and Andrew Zisserman.  Very Deep Convolutional Networks for Large-Scale Image Recognition.  arXiv:1409.1556 [cs], Apr. 2015.  arXiv: 1409.1556.
  * [39] David Stutz, Matthias Hein, and Bernt Schiele.  Disentangling Adversarial Robustness and Generalization.  pages 6976–6987, 2019.
  * [40] Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai.  One Pixel Attack for Fooling Deep Neural Networks.  IEEE Transactions on Evolutionary Computation, 23(5):828–841, Oct. 2019.  Conference Name: IEEE Transactions on Evolutionary Computation.
  * [41] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry.  Robustness May Be at Odds with Accuracy.  arXiv:1805.12152 [cs, stat], Sept. 2019.  arXiv: 1805.12152 version: 2.
  * [42] Danilo Vasconcellos Vargas and Shashank Kotyan.  Robustness Assessment for Adversarial Machine Learning: Problems, Solutions and a Survey of Current Neural Networks and Defenses.  arXiv:1906.06026 [cs, stat], Nov. 2019.  arXiv: 1906.06026.
  * [43] B. S. Vivek, Konda Reddy Mopuri, and R. Venkatesh Babu.  Gray-Box Adversarial Training.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, volume 11219, pages 213–228. Springer International Publishing, Cham, 2018.  Series Title: Lecture Notes in Computer Science.
  * [44] Eric Wong and J Zico Kolter.  Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope.  page 10.
  * [45] Chaowei Xiao, Ruizhi Deng, Bo Li, Taesung Lee, Benjamin Edwards, Jinfeng Yi, Dawn Song, Mingyan Liu, and Ian Molloy.  AdvIT: Adversarial frames identifier based on temporal consistency in videos.  In Proceedings - 2019 International Conference on Computer Vision, ICCV 2019, pages 3967–3976. Institute of Electrical and Electronics Engineers Inc., Oct. 2019.
  * [46] Chaowei Xiao, Ruizhi Deng, Bo Li, Fisher Yu, Mingyan Liu, and Dawn Song.  Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, volume 11214, pages 220–237. Springer International Publishing, Cham, 2018.  Series Title: Lecture Notes in Computer Science.
  * [47] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille.  Mitigating Adversarial Effects Through Randomization.  arXiv:1711.01991 [cs], Feb. 2018.  arXiv: 1711.01991.
  * [48] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille.  Adversarial Examples for Semantic Segmentation and Object Detection.  In 2017 IEEE International Conference on Computer Vision (ICCV), pages 1378–1387, Oct. 2017.  ISSN: 2380-7504.
  * [49] Matthew D. Zeiler and Rob Fergus.  Visualizing and Understanding Convolutional Networks.  In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision – ECCV 2014, Lecture Notes in Computer Science, pages 818–833, Cham, 2014. Springer International Publishing.


# Neuronal Cell Type Classification using Deep Learning

Ofek Ophir  Faculty of Engineering, Bar-Ilan University, Ramat-Gan 5290002,
Israel  The Leslie & Susan Goldschmied (Gonda) Multidisciplinary Brain
Research Center, Bar-Ilan University, Ramat-Gan 5290002, Israel  Orit Shefi
Faculty of Engineering, Bar-Ilan University, Ramat-Gan 5290002, Israel
Corresponding author  Ofir Lindenbaum  Faculty of Engineering, Bar-Ilan
University, Ramat-Gan 5290002, Israel  Corresponding author

###### Abstract

The brain is likely the most complex organ, given the variety of functions it
controls, the number of cells it comprises, and their corresponding diversity.
Studying and identifying neurons, the brain’s primary building blocks, is a
crucial milestone and essential for understanding brain function in health and
disease. Recent developments in machine learning have provided advanced
abilities for classifying neurons. However, these methods remain black boxes
with no explainability and reasoning. This paper aims to provide a robust and
explainable deep-learning framework to classify neurons based on their
electrophysiological activity. Our analysis is performed on data provided by
the Allen Cell Types database containing a survey of biological features
derived from single-cell recordings of mice and humans. First, we classify
neuronal cell types of mice data to identify excitatory and inhibitory
neurons. Then, neurons are categorized to their broad types in humans using
domain adaptation from mice data. Lastly, neurons are classified into sub-
types based on transgenic mouse lines using deep neural networks in an
explainable fashion. We show state-of-the-art results in a dendrite-type
classification of excitatory vs. inhibitory neurons and transgenic mouse lines
classification. The model is also inherently interpretable, revealing the
correlations between neuronal types and their electrophysiological properties.

Keywords— Cell-type Classification, Deep Learning, Machine Learning, Allen
Cell Types Database

##  1 Introduction

The brain is a highly complex system with billions of neurons propagating
signals to communicate and share information. Proper functionality of the
nervous system requires mechanisms for information sharing between many
neurons in different brain regions. Understanding these mechanisms remains an
open and challenging problem in biology and requires a detailed and exact
description of all brain regions and the neurons composing them.

The task of classifying neurons, the building blocks of the nervous system,
has been an ongoing challenge in neuroscience ever since Ramon y Cajal’s
’Histology of the Nervous System of Man and Vertebrates’ [1] was published,
which was to a certain degree, an attempt to classify neurons. Neuroscientists
attempting to study the nervous system have hypothesized that the differences
in neuron morphology play a role in the neural circuit. For this reason, it is
essential to accurately classify the different types of neurons [2].

Defining a solid neuronal cell-type taxonomy is challenging and includes two
significant obstacles. First, classification studies were underpowered and
laborious, which caused highly biased results. However, in the past decade,
technological advances have made it possible to analyze hundreds of neurons
accurately and efficiently [3]. The second obstacle is determining how fine
and firm the distinctions between neuronal types should be. If the resolution
is too broad (such as the distinction between sensory and motor neurons), the
taxonomy might be too coarse and have little value for experimental purposes.
However, if the resolution is too fine, the neuronal taxonomy might have no
relevance (an extreme case would be to think of each neuron as an independent
type).

In this paper, we carry out classification using electrophysiological features
such as action potential (AP) threshold, width, height, hyperpolarization
voltage, and resting potential. These features are aimed to describe the
differences among the observed variability in neuronal activities and can be
used to define electrophysiological types of neurons [4, 5]. The axonal
morphologies also impact AP propagation and are relevant in neurons’ complex
axonal ramification patterns [6, 7]. It should be noted that
electrophysiological features are more accessible to measure than
morphological or genetic features and can be simultaneously recorded using
techniques such as optical imaging of electrical activity on hundreds of
neurons [2].

We present a deep learning framework for predicting neuronal types using
electrophysiological features in two classification tasks. The first is
classifying neurons to their broad type, excitatory vs. inhibitory. The second
task includes classifying neurons into their inhibitory subclasses and
excitatory neurons into their broad class in mice. For both classification
tasks, we use data from the Allen Cell Types Database - ACTB [8], a publicly
available brain cell database containing recordings of electrical stimulus and
response in different types of neurons from both human and mouse cells.

Since obtaining neuronal data from humans is challenging, we use additional
data from mice to develop a reliable classifier that exploits information from
humans and mice. We present a domain adaptive neural network that classifies
broad neuronal cell types from both domains. Next, we use mice data to
identify granular neuronal subclasses and implement a locally sparse network
that enables reliable classification in a low sample size regime. The model
leads to state-of-the-art results compared with leading baselines while
providing high interpretability, which is vital for future research.

##  2 Background

At the most fundamental level, cells can be classified into non-neuronal cells
and neurons, which can then be further classified into excitatory and
inhibitory neurons [9]. The significant difference between the two types is
that the excitatory neurons release neurotransmitters (most commonly glutamic
acid) that fire an action potential in the postsynaptic neuron. In contrast,
inhibitory neurons release neurotransmitters (most commonly gamma-aminobutyric
acid - GABA) which inhibit the firing of an action potential. Inhibitory
interneurons comprise only 10-20% of the total neural population in the cortex
but are essential for sensation, movement, and cognition [10].

Excitatory neurons are usually morphologically spiny, with a long apical
dendrite, and exhibit less variability in their electrophysiological features.
This makes it harder to distinguish between excitatory cell types solely using
electrophysiological characteristics. Inhibitory neurons are typically aspiny
or sparsely spiny, with a more compact dendritic structure, having a more
considerable variance in electrophysiological properties and tending to spike
faster [11, 12]. Neurons can also be classified based on their
neurotransmitter, GABAergic neurons which are mostly inhibitory cells, and
Glutamatergic neurons, which are habitually excitatory and brain-area
specific.

Many neuroscientists also consider GABAergic neurons as belonging to one of
the four subclasses based on the expression of specific principal markers;
these include:

  * •

Pvalb (parvalbumin) positive.

  * •

Vip (vasoactive intestinal peptide) positive.

  * •

Sst (somatostatin) positive.

  * •

Htr3a (5-hydroxytryptamine receptor) positive, Vip negative.

These subclasses of GABAergic interneurons account for most neurons in
specific brain regions. The classes are expressed in a non-overlapping manner,
meaning that each neuron belongs to one class in a monovalent fashion, with
distinct cell types accompanied by different physiological properties [13].

Figure 1: GABAergic neurons in the neocortex express one of four markers:
Pvalb (Parvalbumin), Sst (Somatostatin), and the ionotropic serotonin receptor
Ht3ar which can be either Vip (Vasoactive intestinal polypeptide) positive or
negative.

##  3 Related Work

In 2019, the ACTB became public [8, 14], and with recent advances in computing
capabilities and rapid development of machine and deep learning methods, the
domain of neuronal cell-type classification has leaped forward. From the ACTB,
17 electrophysiological neuron types were identified, 4 of which were
classified as excitatory subtypes, and 13 were inhibitory. The 13 inhibitory
subtypes were further mapped into the four inhibitory interneuron types based
on genetic tags: Vip, Ndnf, Sst, and Pvalb. The researchers also identified 38
morphological, and 46 morpho-electric neuron types, all of which were
classified using current clamp electrophysiological recordings and the help of
dimensionality reduction algorithms such as principal component analysis [15]
and t-distributed stochastic neighbor embedding [16].

Using the ACTB, Ghaderi et al., [17] developed a semi-supervised method in
which neuron classification occurs within three types of neurons. These types
are excitatory pyramidal cells (Pyr), parvalbumin-positive (Pvalb)
interneurons, and somatostatin-positive (Sst) interneurons from layer 2/3 of
the mouse primary visual cortex. The authors achieved accuracies of 91.59 ±
1.69, 97.47 ± 0.67, and 89.06 ± 1.99 for Pvalb, Pyr, and Sst, respectively,
which yielded an overall accuracy of 92.67 ± 0.54%.

In 2019, Seo et al. [18] used machine learning to predict transgenic markers
of neurons using electrophysiological recordings. The work evaluated three
different methods, namely: random forest - RF [19], least absolute shrinkage
and selection operator – LASSO [20], and artificial neural networks - ANN
[21]. The prediction performance of the three models was similar, with
28.57-46.93% accuracy in predicting the transgenic marker of excitatory
neurons (Ctgf, Cux2&Slc17, Nr5a1&Scnn1a, Ntsr1, Rbp4, and Rorb) and
59.03-73.49% accuracy at predicting the transgenic marker of inhibitory
neurons (Chrna2, Gad2, Htr3a, Ndnf, Nkx2, Pvalb, Sst, Vip&Chat).

In 2021, Rodríguez et al. [22] revealed a circular ordered taxonomy using a
transformation of the first two principal components and validated the
proposed taxonomy with machine learning models (linear discriminant analysis –
LDA [23], RF, gradient boosted decision tree – GBDT [24], support vector
machine – SVM [25], and ANN ensemble [26]). These models were able to
discriminate the different neuron types (4 types of inhibitory neurons –
Pvalb, Htr3a, Sst, Vip, as well as Glutamatergic excitatory cells) using
electrophysiological features with accuracy ranging between 66.1-75.2% for the
raw data, and 72.0-80.3% accuracy for a subset of the data that has been
cleaned using anomaly detectors.

It is worth noting that these studies only used mouse data, which is most
likely due to insufficient human data. One can train a machine-learning model
on data from humans and mice to overcome this limitation. However, training a
model on multiple domains may result in overfitting to that most abundant
domain, and lead to a performance gap data from other domains [27, 28]. This
is known as the domain shift problem and can be addressed using tools from
domain adaptation. Another issue with using these types of algorithms emerges
from the complexity of the neural networks, making it difficult to interpret
the model’s decisions [29]. Model interpretability is essential to bio-
medicine, where practitioners must trust the machine learning model.

This paper addresses these issues by providing a machine-learning framework
for predicting neuronal cell types in two steps. The first is classifying
excitatory vs. inhibitory neurons, and the second is classifying excitatory
Glutamatergic cells and the different subclasses (Pvalb, Htr3a, Sst, and Vip)
of inhibitory GABAergic cells. First, regarding dendrite type classification,
we use mus-musculus (house mouse) source data which we have in a larger
quantity, to learn a distribution over the homo sapiens (human) target data
using domain adaptation methods. By doing this, we improve the results and
robustness of the model on the target data. We also explain the importance of
the unique features during testing using a concept from cooperative game
theory. Then, we use a deep neural network inherently explainable model to
predict Cre-line labels from mouse data; these account for the different
GABAergic neuron subclasses and Glutamatergic neurons.

##  4 Data

###  4.1 Allen Cell Types Database

The ACTB contains electrophysiological recordings from 1920 mice and 413 human
cells. The cells from humans and mice are categorized by dendrite type: spiny,
aspiny, and sparsely spiny, as well as location and layer in the brain. The
cells from mice samples are further identified for isolation using transgenic
mouse lines harboring fluorescent reporters, with drivers that allow
enrichment for cell classes based on marker genes such as Pvalb positive, Sst
positive, Vip positive, and Htr3a (5-hydroxytryptamine receptor) positive but
Vip negative [13].

The mouse data contains whole-cell current clamp recordings from identified
fluorescent Cre-positive neurons or nearby Cre-negative neurons in acute brain
slices derived from adult mice. The human data contains whole-cell current
clamp recordings from adult human neocortical neurons in brain slices derived
from surgical specimens. Each whole-cell current clamp recording is a response
to a stimulation recorded at 200 KHz (before 2016) or 50 KHz (after 2016).
Electrophysiological features are calculated and extracted from each whole-
cell current clamp recording according to Table 3.

The stimulations types include:

  * •

Noise - Noise pulses offset with square current injections.

  * •

Ramp - a current injection of increasing intensity at a rate slower than the
time constant of the neuron.

  * •

Long square - a square pulse of duration that allows the neuron to reach a
steady state.

  * •

Short square - a square pulse brief enough to elicit a single action
potential.

Figure 2: (Top) Stimulation of noise pulses with square current injections
scaled to three amplitudes, 0.75, 1, and 1.5 times Rheobase with a coefficient
of variation (CV) equal to 0.2, as it resembles in vivo data. (Bottom) cell
response to the noise stimulation.

###  4.2 Pre-processing

The data was downloaded from the ACTB. From each whole-cell patch clamp
recording, electrophysiological features were extracted into tabular format,
then, similar Cre lines were grouped. The entire process is illustrated in
Figure 3.

Figure 3: Electrophysiological features pipeline. Data is first obtained from
the ACTB; then, features are extracted into a tabular format. Similar Cre-
lines are merged, and data is split into train, validation, and test. The data
is normalized, and classification occurs through the neural network models.

GABAergic neurons belong to four subclasses based on their expressed Cre
lines, Pvalb (Parvalbumin) positive, Vip (Vasoactive intestinal peptide)
positive, Sst (Somatostin) positive, and 5-hydroxytryptamine receptor 3A
(Htr3a) positive Vip negative. Glutamatergic neurons belong to different
subclasses according to their laminar locations and the location to which they
project their axons [13]. Using the ACTB, five transcriptomic-
electrophysiological subclasses have been defined; these include four major
GABAergic subclasses and one Glutamatergic subclass specified in Figure 4,
according to [22].

Figure 4: Cre-lines composing the defined subclasses.

The mouse data is quite balanced regarding dendrite type distribution, with
700 spiny neurons (Glutamatergic) and 724 aspiny neurons (GABAergic). The
human data contains more spiny neurons (231) than aspiny neurons (68). In
terms of Cre-line subclasses within the mouse data, Glutamatergic neurons are
the largest group with 700 neurons, Pvalb contains 231 neurons, Htr3a positive
Vip negative contains 199 neurons, Sst contains 173 neurons, and Vip positive
contains 121 neurons.

Figure 5: (Left) Distribution of dendrite type in mouse vs. human data (Right)
Distribution of the defined Cre-line subclasses in mouse data.

##  5 Models

Artificial neural networks (ANN) are computing models inspired by biological
neural networks. ANNs rely on matrix multiplications followed by nonlinear
activation functions to learn complex relations between input and output. ANNs
are comprised of artificial neurons which are connected through edges, these
edges typically have a weight value that can adjust the strength of the signal
at that connection, and the weights are ’learned’ through an optimizer such as
Stochastic Gradient Descent (SGD) [30].

Over the last decade, numerous neural network architectures have been
proposed. In this paper, we focus on fully connected neural networks, also
referred to as multi-layer perceptron (MLP), or just a ’neural network’ (NN)
[31]. We also use a new type of NN designed for tabular data, namely LSPIN.

We focus on two neuronal cell-type classification tasks:

  * •

Humans and mice dendritic cell type classification (inhibitory/excitatory)

  * •

Multi class (Pvalb, Sst, Vip, Htr3a, Glutamatergic) classificaiton in mice.

For the first task, we introduce a domain adaptation component to handle
measurements from humans and mice simultaneously. For the second task, we use
a NN with a sample-specific feature selection mechanism, namely LSPIN, to
reduce model overfitting in low-sample-size data and obtain an interpretable
model. In subsection 5.1, we describe the domain adaptation mechanism, and in
subsection 5.2, we describe the LSPIN model.

###  5.1 Domain Adversarial Neural Network

Mouse neuronal data is acquired from selected brain areas in adult mice. Cells
are identified using transgenic mouse lines harboring fluorescent reporters,
with drivers that allow enrichment for cell classes based on marker genes. In
contrast, human neuronal data is acquired from donated ex vivo brain tissues
analyzed from neurosurgical and postmortem sources and is available thanks to
the generosity of tissue donors. Thus, human neuronal data is challenging to
obtain and less abundant than data from mice (1920 mouse samples vs. 413 human
samples).

We aim to design a model that can classify human neuronal types, yet this is
difficult due to the scarcity of human samples. To deal with this issue, we
use both mouse and human data to classify human samples better. This is
possible because mouse and human neurons are similar (both come from mammalian
brain tissues). Nonetheless, standard machine learning models typically
underperform in such a setting since there may be a domain shift between
samples from both distributions. To overcome this limitation, we use a domain
adaptation scheme designed to attenuate the influence of domain shift.

We consider $X\in\mathbb{R}^{D}$ the input space, and $Y\in\\{0,1\\}$ the
output space, where 0 is an excitatory cell, and 1 is an inhibitory cell. We
define $S$ to be the source distribution over $X\times Y$, and $D_{S}$ to be
the marginal distribution such that
$S=\\{(\boldsymbol{x}_{i},y_{i})\\}_{i=1}^{n}\sim D_{S}$. We define $T$ to be
the target distribution over $X\times Y$, and $D_{T}$ to be the marginal
distribution such that $T=\\{(\boldsymbol{x}_{i},y_{i})\\}_{i={n+1}}^{N}\sim
D_{T}$. Where $n$ is the number of source samples, and $N$ is the number of
all samples. Our goal is to define a classifier $\eta:X\rightarrow{}Y$ to
which the target risk function is low:

$\displaystyle R_{D_{T}}(n)=\underset{\\{\boldsymbol{x},y\\}\sim
D_{T}}{Pr}(\eta(\boldsymbol{x})\neq y),$ (1)

while maintaining a low source risk as well. Since there may be a shift
between $D_{S}$ and $D_{T}$ training, a naive model based on Eq. 1 can be
biased towards the more abundant domain ($D_{S}$). To alleviate such bias
Ganin, Yaroslav, et al. [32] introduced a technique called ’domain-adversarial
training of neural networks (DANN), that combines both representation learning
(i.e., deep feature learning) and unsupervised domain adaptation in an end-to-
end training process. DANN jointly optimizes two adversarial losses:

  1. 1.

minimizing the loss of a label classifier.

  2. 2.

maximizing the loss of a domain classifier.

Training both losses can be considered a form of adversarial neural network
regularization. On the one hand, the network needs to classify the data into
the correct labels. On the other hand, the predictions made by the network
must be based on features that cannot discriminate between the source domain
and the target domain. In our setting, mouse cells are considered the source
distribution and are more abundant (since it is easier to obtain neurons from
the rat brain than the human brain), and the human cells serve as the target
distribution.

The prediction loss and domain loss are respectively defined as

$\displaystyle
L_{y}^{i}(\theta_{f},\theta_{y})=L_{y}(G_{y}(G_{f}(\boldsymbol{x}_{i};\theta_{f});\theta_{y}),y_{i}),$
$\displaystyle
L_{d}^{i}(\theta_{f},\theta_{d})=L_{d}(G_{d}(G_{f}(\boldsymbol{x}_{i};\theta_{f});\theta_{d}),d_{i}).$

Where $\theta_{f},\theta_{y},\theta_{d}$ are the parameters of the feature
extractor, label classifier, and domain classifier, respectively, and $d_{i}$
is the domain label of sample $i$ as illustrated in Figure 6.

Training the model consists of optimizing:

$E(\theta_{f},\theta_{y},\theta_{d})=\frac{1}{N}\sum_{n=1}^{N}L_{y}^{i}(\theta_{f},\theta_{y})-\frac{\lambda}{N}\sum_{n=1}^{N}L_{d}^{i}(\theta_{f},\theta_{d}),$

by finding the saddle point
$\hat{\theta}_{f},\hat{\theta}_{y},\hat{\theta}_{d}$ such that:

$\displaystyle(\hat{\theta}_{f},\hat{\theta}_{y})=\arg\min_{\theta_{f},\theta_{y}}E(\theta_{f},\theta_{y},\hat{\theta}_{d}),$
(2)
$\displaystyle\hat{\theta}_{d}=\arg\max_{\theta_{d}}E(\hat{\theta_{f}},\hat{\theta_{y}},\theta_{d}).$

To optimize over Eq. 2, we can use gradient descent, which relies on the
following update rules:

$\displaystyle\theta_{f}\leftarrow{}\theta_{f}-\mu(\frac{\partial
L_{y}^{i}}{\partial\theta_{f}}-\lambda\frac{\partial
L_{d}^{i}}{\partial\theta_{f}}),$
$\displaystyle\theta_{y}\leftarrow{}\theta_{y}-\mu\frac{\partial
L_{y}^{i}}{\partial\theta_{y}},$
$\displaystyle\theta_{d}\leftarrow{}\theta_{d}-\mu\lambda\frac{\partial
L_{d}^{i}}{\partial\theta_{d}}.$

Where $\mu$ is the learning rate.

Using the aforementioned NN architecture, domain adaptation is achieved by
forcing the prediction based on features that cannot discriminate between
mouse and human samples. Final classification decisions are made using
discriminative features invariant to the organism’s change. We assume that a
good representation for cross-domain transfer is one for which an algorithm
cannot identify between the two domains [28, 33].

Figure 6: The architecture of DANN. During forward propagation (solid-line),
input data may come from humans or mice. The feature extractor (gray) with
optimized weights $\theta_{f}$ outputs to both the domain classifier (red)
with optimized weights $\theta_{d}$ and label classifier (greed) with
optimized weights $\theta_{y}$. During backpropagation (dotted line), the
domain gradient is multiplied by a negative constant, while the label gradient
remains positive.

###  5.2 Locally Sparse Neural Network

Collecting whole-cell current clamp recordings is computationally challenging;
for instance, the ACTB contains only 1920 mouse cells and 413 human cells. The
low number of samples makes it challenging to train an over-parametrized NN
while avoiding overfitting. To address this obstacle, we adopt a recently
proposed method for fitting NN models to low sample size data. Specifically,
the method is designed to deal with the problem of low sample size data for
tabular heterogeneous biological data such as whole-cell current clamp
recordings of neurons in various brain areas in mice. We show that by using
the proposed method, we achieve state-of-the-art results. Furthermore, the
method proposed is an intrinsically interpretable network for biomedical data,
’Locally Sparse Interpretable Network’ – LSPIN [34]. We use LSPIN to predict
five distinct neuronal types, four from GABAergic neurons and the remainder
from Glutamatergic neurons.

The model is a locally sparse neural network in which the local sparsity is
learned to identify the subset of the most relevant features for each sample.
LSPIN includes two neural networks which are trained in tandem:

  1. 1.

The gating network - predicts the sample-specific sparsity patterns.

  2. 2.

The prediction network - classifies the neuron type using the extracted
features from Table 3.

By forcing the model to select a subset of the most informative features for
each sample, we can reduce overfitting in low-sample size data. Another
benefit of this model is that by predicting the most informative features
locally, we obtain an interpretation of the model’s predictions.

Given labeled observations $\\{\boldsymbol{x}^{(i)},y^{(i)}\\}_{i=1}^{N}$,
where $\boldsymbol{x}^{(i)}\in\mathbb{R}^{D}$, and $x_{d}^{(i)}$ represents
the dth feature of the ith sample. We want to learn a global prediction
function $\boldsymbol{f_{\theta}}$, and a set of parameters
$\\{\mu_{d}^{(i)}\\}_{d=1,i=1}^{D,n}$ such that $\mu_{d}^{(i)}$ depict the
behavior of the local stochastic gates $z_{d}^{(i)}\in[0,1]$ that sparsify
(for each instance $i$) the set of features that propagate into in the
prediction model $\boldsymbol{f_{\theta}}$. Stochastic gates [35] are
continuously relaxed Bernoulli variables highly effective for the
sparsification of NNs. They were previously used for several applications,
including feature selection [36, 37], sparse Canonical Correlation Analysis
[38], and anomaly detection [39].

Each stochastic gate (for feature $d$ and sample $i$) is defined based on the
following threshold function:

$\displaystyle
z_{d}^{(i)}=max(0,min(1,0.5+\mu_{d}^{(i)}+\epsilon_{d}^{(i)})),$

where $\epsilon_{d}^{(i)}\sim\mathcal{N}(0,\,\sigma^{2})\ $ and $\sigma$ is
fixed to a constant during training, and equals 0 during inference. The
sample-specific parameters $\boldsymbol{\mu}^{(i)}\in\mathbb{R}^{D},i=1,...,N$
are predicted based on the gating network $\boldsymbol{\psi}$ such that
$\boldsymbol{\mu}^{(i)}=\boldsymbol{\psi}(\boldsymbol{x}^{(i)}|\boldsymbol{\Omega})$,
where $\boldsymbol{\Omega}$ are the weights of the gating network. These
weights are learned simultaneously with the weights of the prediction network
by minimizing the following loss:

$\displaystyle\mathbb{E}[\mathcal{L}(\boldsymbol{f_{\theta}}(\boldsymbol{x}^{(i)}\odot\boldsymbol{z}^{(i)}),y^{(i)})+\mathcal{R}(\boldsymbol{z}^{(i)})],$

where $\mathcal{L}$ is a desired loss (e.g. cross-entropy), $\odot$ represents
the Hadamard product (element-wise multiplication), and
$\mathcal{R}(\boldsymbol{z}^{(i)})$ is a regularizer term defined as:

$\displaystyle\mathcal{R}(\boldsymbol{z}^{(i)})=\lambda_{1}||\boldsymbol{z}^{(i)}||_{0}+\lambda_{2}\sum_{j}K_{i,j}||\boldsymbol{z}^{(i)}-\boldsymbol{z}^{(j)}||_{2}^{2},$

where $K_{i,j}\geq 0$ is a user-defined kernel (e.g., radial basis function).

Figure 7: The architecture of Locally SParse Interpretable Network. The data
${\boldsymbol{x}^{(i)}=[x_{1}^{(i)},x_{2}^{(i)},...,x_{D}^{(i)}]}_{i=1}^{n}$
is fed simultaneously to a gating network $\boldsymbol{\Psi}$ and to a
prediction network $\boldsymbol{f_{\theta}}$. The gating model learns to
sparsify the set of features propagating to the prediction model, leading to
sample-specific (local) sparsification. Therefore, it can handle extreme cases
of Low-Sample-Size data and lead to interpretable predictions.

##  6 Results

###  6.1 Domain Adaptation Prediction Performance

We optimized the DANN model on 1378 training samples of cells from both humans
and mice. We split the training data to 1171 samples for training and 207
samples for validation. We also kept 60 human and 285 mouse cells for testing
the model. The performance of the domain adaptation task in which neurons are
classified based on the broad types (excitatory vs. inhibitory) is shown in
Figure 8. Using the method, we show that the model generalizes to both the
human and mouse domains while providing excellent classification results in
accuracy, F1 score, precision, and recall. Furthermore, using DANN, we show
that whole-cell current clamp recordings of neurons in mouse brains are
similar to those of the human brain; moreover, we show that it is possible to
adapt to samples from the human domain using the DANN method, providing a
gateway in neuronal cell type classification of neurons from the human brain
using samples from mouse brains, which are more abundant.

Figure 8: (left) Human classification confusion matrix. (right) Mouse
classification confusion matrix. The vertical axis represents the actual
class, while the horizontal axis represents the predicted class. The same
neural network accurately predicts data from humans and mice.

The performance of the DANN model was evaluated using the accuracy, f1 score,
precision, and recall metrics. The evaluation results are shown in Table 1. We
show that the method classifies neurons to their broad types with 95.0%
accuracy in human samples and with 97.4% accuracy in mouse samples using the
same model weights, delivering a model that generalizes to both the human and
mouse domains and classifies samples from both domains simultaneously.

| Accuracy | F1 | Precision | Recall
---|---|---|---|---
Human | 0.950 | 0.968 | 1.000 | 0.938
Mouse | 0.974 | 0.977 | 0.980 | 0.974

Table 1: DANN evaluation metrics for mouse and human data.

###  6.2 Cre Based Sub Classes Prediction Performance

We trained LSPIN on 1105 samples from mice and tested it on 277 (42 samples
with NaN values were excluded). The prediction network included two hidden
dense layers of size 40 and 20 (with an input layer of size 41 and an output
layer of size 5). The gating network was assembled from 3 layers, each
containing 50 neurons. Tanh was used as the activation function for both the
gating and prediction networks.
$\lambda_{1}=0.01047,\lambda_{2}=0,\sigma=0.5$, the network was trained for
1000 epochs with a learning rate of 0.0599 and SGD optimizer. The model
achieved an accuracy score of 0.916, an F1 score of 0.915, a precision score
of 0.917, and a recall score of 0.916. Results of the classification are shown
in Figure 10

Through the gating network and stochastic gates outputs, we can interpret the
decisions made by the prediction network, less essential features are muted
while important features are not. Figure 9 shows the evaluation of features.
The model is forced to select a subset of the most informative features
identified for each sample. By that, model overfitting is reduced in low-
sample-size data, and the selection of relevant features produces an
interpretable classification verdict in which relevant and non-relevant
features are identified for each neuronal subclass.

Figure 9: Hierarchical clustering of the gate matrix. Features that are
completely muted are displayed in black, while features that stay unchanged
are displayed in white. The vertical axis represents the test samples, and the
horizontal axis represents the electrophysiological features from Table 3 in
the appendix. Figure 10: Confusion matrix of the LSPIN model. The vertical
axis represents the true class, while the horizontal axis represents the
predicted class. Glutamatergic neurons are identified 97.8% of the time
correctly, Htr3a neurons are identified 82.9% of the time, Pvalb 86.8% of the
time, Sst 79.2% of the time, and Vip neurons are identified 95.8% of times
correctly. Interestingly, Sst neurons are most likely to be confused with
Htr3a neurons, suggesting that the two subtypes have similar
electrophysiological properties.

Furthermore, we trained a Random Forest (RF) classifier, a C-Support Vector
Classifier (SVC), and an XGBoost classifier on the same data to compare it
with the LSPIN model; the results of the comparison are shown in Table 2. The
results show that the LSPIN model outperforms all other baseline models,
achieving 91.6% accuracy, surpassing XGBoost (the second-best model) by 6.6%.

| Accuracy | F1 | Precision | Recall
---|---|---|---|---
RF | 0.824 | 0.726 | 0.774 | 0.702
SVC | 0.837 | 0.749 | 0.789 | 0.731
XGBoost | 0.850 | 0.768 | 0.779 | 0.759
LSPIN | 0.916 | 0.877 | 0.886 | 0.873

Table 2: Comparison of different machine learning models using macro averaging
on the same test data. LSPIN outperforms all other optimized models, achieving
an accuracy score of 0.916, an F1 score of 0.877, a precision score of 0.886,
and a recall score of 0.873.

##  7 Conclusions

We introduce two deep-learning frameworks to identify neuronal
electrophysiological types. The first method takes into account both mouse
samples and human samples in order to obtain more data. We overcome the domain
shift between the two distributions by using a domain adaptation method called
’Domain Adversarial training of Neural Networks.’ The second method uses a
locally sparse neural network to identify neuronal subtypes while dealing with
the issue of low sample size and producing interpretable predictions.

In summary, the main contributions of this paper can be concluded as follows:

  * •

We introduce a domain adaptive model to classify neurons into excitatory and
inhibitory types from both human and mouse samples simultaneously. We show
that data from mouse samples can be used to predict neuronal types in data
from human samples, which can have significant clinical effects, as human
samples are challenging to acquire.

  * •

We use the LSPIN model to expand the classification task to neuronal t-type
classification based on Cre lines. We show state-of-the-art results in
neuronal subtypes classification. The model can also interpret which features
are more important to the classification model for each subtype.

## References

  * [1] S Ramon Cajal et al.  Histology of the nervous system of man and vertebrates.  History of Neuroscience (Oxford Univ Press, New York), 6, 1995.
  * [2] Hongkui Zeng and Joshua R Sanes.  Neuronal cell-type classification: challenges, opportunities and the path forward.  Nature Reviews Neuroscience, 18(9):530–546, 2017.
  * [3] Y Zhao, S Inayat, DA Dikin, JH Singer, RS Ruoff, and JB Troy.  Patch clamp technique: review of the current state of the art and potential contributions from nanoengineering.  Proceedings of the Institution of Mechanical Engineers, Part N: Journal of Nanoengineering and Nanosystems, 222(1):1–11, 2008.
  * [4] Michael Beierlein, Jay R Gibson, and Barry W Connors.  Two dynamically distinct inhibitory networks in layer 4 of the neocortex.  Journal of neurophysiology, 90(5):2987–3000, 2003.
  * [5] Lionel G Nowak, Maria V Sanchez-Vives, and David A McCormick.  Lack of orientation and direction selectivity in a subgroup of fast-spiking inhibitory interneurons: cellular and synaptic mechanisms and comparison with other electrophysiological cell types.  Cerebral Cortex, 18(5):1058–1078, 2008.
  * [6] Netanel Ofer, Orit Shefi, and Gur Yaari.  Axonal tree morphology and signal propagation dynamics improve interneuron classification.  Neuroinformatics, 18(4):581–590, 2020.
  * [7] Netanel Ofer and Orit Shefi.  Axonal geometry as a tool for modulating firing patterns.  Applied Mathematical Modelling, 40(4):3175–3184, 2016.
  * [8] Allen Institute for Brain Science.  Allen cell types database, 2015.
  * [9] Sarah Melzer and Hannah Monyer.  Diversity and function of corticopetal and corticofugal gabaergic projection neurons.  Nature Reviews Neuroscience, 21(9):499–515, 2020.
  * [10] Olivia K Swanson and Arianna Maffei.  From hiring to firing: activation of inhibitory neurons and their recruitment in behavior.  Frontiers in molecular neuroscience, 12:168, 2019.
  * [11] Yasuo Kawaguchi.  Neostriatal cell subtypes and their functional roles.  Neuroscience research, 27(1):1–8, 1997.
  * [12] Carsten Strübing, Gudrun Ahnert-Hilger, Jin Shan, Bertram Wiedenmann, Jürgen Hescheler, and Anna M Wobus.  Differentiation of pluripotent embryonic stem cells into the neuronal lineage in vitro gives rise to mature inhibitory and excitatory neurons.  Mechanisms of development, 53(2):275–287, 1995.
  * [13] Robin Tremblay, Soohyun Lee, and Bernardo Rudy.  Gabaergic interneurons in the neocortex: from cellular properties to circuits.  Neuron, 91(2):260–292, 2016.
  * [14] Nathan W Gouwens, Staci A Sorensen, Jim Berg, Changkyu Lee, Tim Jarsky, Jonathan Ting, Susan M Sunkin, David Feng, Costas A Anastassiou, Eliza Barkan, et al.  Classification of electrophysiological and morphological neuron types in the mouse visual cortex.  Nature neuroscience, 22(7):1182–1195, 2019.
  * [15] Hervé Abdi and Lynne J Williams.  Principal component analysis.  Wiley interdisciplinary reviews: computational statistics, 2(4):433–459, 2010.
  * [16] Laurens Van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  Journal of machine learning research, 9(11), 2008.
  * [17] Parviz Ghaderi, Hamid Reza Marateb, and Mir-Shahram Safari.  Electrophysiological profiling of neocortical neural subtypes: a semi-supervised method applied to in vivo whole-cell patch-clamp data.  Frontiers in neuroscience, 12:823, 2018.
  * [18] Incheol Seo and Hyunsu Lee.  Predicting transgenic markers of a neuron by electrophysiological properties using machine learning.  Brain Research Bulletin, 150:102–110, 2019.
  * [19] Leo Breiman.  Random forests.  Machine learning, 45:5–32, 2001.
  * [20] Robert Tibshirani.  Regression shrinkage and selection via the lasso.  Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
  * [21] Frank Rosenblatt.  The perceptron: a probabilistic model for information storage and organization in the brain.  Psychological review, 65(6):386, 1958.
  * [22] Alejandro Rodríguez-Collado and Cristina Rueda.  Electrophysiological and transcriptomic features reveal a circular taxonomy of cortical neurons.  Frontiers in Human Neuroscience, page 410, 2021.
  * [23] Suresh Balakrishnama and Aravind Ganapathiraju.  Linear discriminant analysis-a brief tutorial.  Institute for Signal and information Processing, 18(1998):1–8, 1998\.
  * [24] Alexey Natekin and Alois Knoll.  Gradient boosting machines, a tutorial.  Frontiers in neurorobotics, 7:21, 2013.
  * [25] Corinna Cortes and Vladimir Vapnik.  Support-vector networks.  Machine learning, 20:273–297, 1995.
  * [26] Zhi-Hua Zhou and SF Chen.  Neural network ensemble.  CHINESE JOURNAL OF COMPUTERS-CHINESE EDITION-, 25(1):1–8, 2002\.
  * [27] Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein.  Sensitivity and generalization in neural networks: an empirical study.  arXiv preprint arXiv:1802.08760, 2018.
  * [28] Abolfazl Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid R Arabnia.  A brief review of domain adaptation.  Advances in data science and information engineering, pages 877–894, 2021.
  * [29] David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang.  Xai—explainable artificial intelligence.  Science robotics, 4(37):eaay7120, 2019.
  * [30] Sebastian Ruder.  An overview of gradient descent optimization algorithms.  arXiv preprint arXiv:1609.04747, 2016.
  * [31] Anders Krogh.  What are artificial neural networks?  Nature biotechnology, 26(2):195–197, 2008.
  * [32] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  The journal of machine learning research, 17(1):2096–2030, 2016\.
  * [33] Amit Rozner, Barak Battash, Lior Wolf, and Ofir Lindenbaum.  Domain-generalizable multiple-domain clustering.  arXiv preprint arXiv:2301.13530, 2023.
  * [34] Junchen Yang, Ofir Lindenbaum, and Yuval Kluger.  Locally sparse neural networks for tabular biomedical data.  In International Conference on Machine Learning, pages 25123–25153. PMLR, 2022.
  * [35] Yutaro Yamada, Ofir Lindenbaum, Sahand Negahban, and Yuval Kluger.  Feature selection using stochastic gates.  In International Conference on Machine Learning, pages 10648–10659. PMLR, 2020.
  * [36] Uri Shaham, Ofir Lindenbaum, Jonathan Svirsky, and Yuval Kluger.  Deep unsupervised feature selection by discarding nuisance and correlated features.  Neural Networks, 152:34–43, 2022.
  * [37] Soham Jana, Henry Li, Yutaro Yamada, and Ofir Lindenbaum.  Support recovery with stochastic gates: Theory and application for linear models.  arXiv preprint arXiv:2110.15960, 2021.
  * [38] Ofir Lindenbaum, Moshe Salhov, Amir Averbuch, and Yuval Kluger.  L0-sparse canonical correlation analysis.  In International Conference on Learning Representations.
  * [39] Ofir Lindenbaum, Yariv Aizenbud, and Yuval Kluger.  Probabilistic robust autoencoders for outlier detection.  arXiv preprint arXiv:2110.00494, 2021.

##  Appendix A Appendix

###  A.1 Extracted Electrophysiological Features

  1. 1.

threshold v noise

  2. 2.

threshold i noise

  3. 3.

peak v noise

  4. 4.

peak i noise

  5. 5.

trough v noise

  6. 6.

trough i noise

  7. 7.

upstroke ratio noise

  8. 8.

upstroke v noise

  9. 9.

downstroke ratio noise

  10. 10.

downstroke v noise

  11. 11.

fast trough v noise

  12. 12.

fast trough i noise

  13. 13.

width noise

  14. 14.

up-down ratio noise

  15. 15.

f-i curve slope noise

  16. 16.

fast trough v long square

  17. 17.

fast trough v ramp

  18. 18.

fast trough v short square

  19. 19.

input resistance mohm

  20. 20.

latency

  21. 21.

peak v long square

  22. 22.

peak v ramp

  23. 23.

peak v short square

  24. 24.

ri

  25. 25.

sag

  26. 26.

seal gohm

  27. 27.

tau

  28. 28.

threshold i long square

  29. 29.

threshold i ramp

  30. 30.

threshold i short square

  31. 31.

threshold v long square

  32. 32.

threshold v ramp

  33. 33.

threshold v short square

  34. 34.

trough v long square

  35. 35.

trough v ramp

  36. 36.

trough v short square

  37. 37.

up-down ratio long square

  38. 38.

up-down ratio ramp

  39. 39.

up-down ratio short square

  40. 40.

vm for sag

  41. 41.

vrest

Table 3: Summary of extracted electrophysiological features