# Wormhole MAML: Meta-Learning in Glued Parameter Space

Chih-Jung Tracy Chang, Yuan Gao11footnotemark: 1 & Beicheng Lou11footnotemark:
1
Stanford University
<EMAIL_ADDRESS>
Equal contribution. Listed alphabetically.

###### Abstract

In this paper, we introduce a novel variation of model-agnostic meta-learning,
where an extra multiplicative parameter is introduced in the inner-loop
adaptation. Our variation creates a shortcut in the parameter space for the
inner-loop adaptation and increases model expressivity in a highly
controllable manner. We show both theoretically and numerically that our
variation alleviates the problem of conflicting gradients and improves
training dynamics. We conduct experiments on 3 distinctive problems, including
a toy classification problem for threshold comparison, a regression problem
for wavelet transform, and a classification problem on MNIST. We also discuss
ways to generalize our method to a broader class of problems.

##  1 Introduction

Developing artificial intelligence that can learn to learn has been a long-
sought-after goal. On the other hand, while deep learning has achieved
impressive results on various problems in computer vision, natural language
processing, and reinforcement learning, the cost of training sophisticated
models for new tasks from scratch has also become tremendous. As a result,
multi-task and meta-learning have emerged in recent years as a promising
approach that aims to discover shared structure across tasks so that
individual ones can be solved more efficiently Finn et al. (2017). The
training process produces a meta-model that can achieve good performance on
individual tasks upon fine-tuning.

Various architectures and algorithms have been developed for multi-task and
meta-learning. One major variability lies in the choice of what to share in
the joint model. For example, in prototypical network Snell et al. (2017), all
learned parameters are shared across tasks, while the data for different tasks
are encoded into a low-dimensional representation. Similarly, in recurrent
models Duan et al. (2017), one model is shared across tasks and task-specific
information is implicitly encoded in the state vector. In contrast, in model-
agnostic meta-learning (MAML) Finn et al. (2017), the parameters for each
layer of the network are allowed to be varied during the adaptation. In fast
context adaptation for meta-learning Zintgraf et al. (2019), one explicitly
separates parameters into shared ones and task-specific ones.

One common challenge in these approaches is training efficiency. Memory-based
methods using recurrent neural networks tend to omit inductive bias and easily
lead to non-converging behavior Flennerhag et al. (2019). Gradient-based
methods are typically restricted to few-shot learning due to the need for
gradient tracking during adaptation Flennerhag et al. (2019). Otherwise, they
may be prone to overfitting Rajendran et al. (2020). In general, the extra
hierarchy in the update process requires a more delicate design of the update
rule Li et al. (2017); Zhang et al. (2020); Lou et al. (2021).

The problem with training efficiency is especially notable when data is
relatively scarce, such as in reinforcement learning settings Yu et al.
(2020). The joint training of multiple tasks tends to introduce undesired
interference between task gradients and negative transfer across tasks, where
the adaptation performance is worse than single-task learning. These
conflicting gradients could lead to a highly inefficient trajectory of
gradient updates and slow down the optimization.

One intuitive approach to tackle conflicting gradients in meta-learning is
gradient surgery Yu et al. (2020), where each task gradient is projected to
the normal plane of other conflicting gradients for updates. However, this
approach is only applicable when the gradient conflict is not too bad, so that
a significant component is projected onto the normal plane. As illustrated in
Fig. 1a, when the conflicting gradients are nearly opposite, the projection is
zero and completely dominated by noise. The model is therefore trapped in the
parameter space, as illustrated in Fig. 1b. While it is still possible for the
inner-loop adaptation to reach task-specific optima, there is hardly any
inductive bias learned in the shared parameters in this case.

Figure 1: Illustration of the problem of conflicting gradients. (a, b)
Illustration of gradient surgery and its limitation. (c) Illustration of
”wormholes” in the parameter space that resolves the problem of conflicting
gradients.

In this paper, we propose a variant architecture of MAML, with an extra
multiplicative parameter applied to the network weights that is both task-
specific and highly regularized. The extra multiplicative parameter creates a
shortcut in the parameter space and effectively glues certain parts together,
analogous to the concept of _wormhole_. Such shortcuts allow the shared
parameters to align their gradients across tasks, as illustrated in Fig. 1c,
free from the problem of conflicting gradients.

##  2 Wormhole MAML

In MAML, the meta-network parameter $\theta$ is learned through an outer loop
and the task-specific parameter $\phi_{i}$ is learned during the inner loop.
For simplicity, let $\phi$, $\theta$ directly represent the weight matrices in
the neural network, i.e. $W_{\phi}=\phi$, $W_{\theta}=\theta$. Then the inner-
loop update of MAML can be written as:

$\phi_{i}=\theta+\Delta_{i}(\theta)$ (1)

where $\Delta_{i}(\theta)$ is the aggregated steps in the gradient descent
direction. In the case of single-step update,
$\Delta_{i}(\theta)=\alpha\cdot\nabla\mathcal{L}_{{\mathcal{T}}_{i}}(\theta)$.

In Wormhole MAML, the inner-loop update is changed to:

$\phi_{i}=C_{i}\theta+\Delta_{i}(\theta)$ (2)

where $\Delta_{i}(\theta)$ is still the aggregated steps that ultimately
depend on $\theta$ only, but now it also has a partial dependence on $C_{i}$.

This extra multiplicative parameter in the inner-loop update allows some
flexibility to incorporate our prior knowledge for better training efficiency.
For example, to tackle the problem of conflicting gradients illustrated in
Fig. 1a, if $C_{i}=1$ and $C_{j}=-1$ for the two contradicting tasks $i$ and
$j$, then the gradient on $\theta$ can be highly aligned, leading to an
efficient trajectory for the outer-loop parameters.

As in MAML, the algorithm could have some variations, especially in terms of
update rules for the multiplicative parameter $C_{i}$. In the simplest case,
we can add a step between the outer-loop update and the inner-loop update, and
the algorithm is:

1 $p(\mathcal{T})$: distribution over tasks $\alpha,\beta,\gamma$: step size
hyper-parameters Randomly initialize $\theta$;

2 while  _not done_ do

3 Sample batch of tasks $\mathcal{T}_{i}\sim p(\mathcal{T})$;

4 for  _all $\mathcal{T}_{i}$_ do

5 Initialize $C_{i}$;

6 Evaluate $\nabla\mathcal{L}_{\mathcal{T}_{i}}(\theta,C_{i})$ with respect to
$K$ examples;

7 Update $C_{i}\leftarrow C_{i}-\gamma\nabla_{C_{i}}\sum_{\mathcal{T}_{i}\sim
p(\mathcal{T})}\mathcal{L}_{\mathcal{T}_{i}}(\phi_{i},C_{i})$;

8 Compute adapted parameters with gradient descent: $\phi_{i}\leftarrow
C_{i}\theta+\nabla\mathcal{L}_{\mathcal{T}_{i}}(\theta,C_{i})$;

9

10 end for

11 Update
$\theta\leftarrow\theta-\beta\nabla_{\theta}\sum_{\mathcal{T}_{i}\sim
p(\mathcal{T})}\mathcal{L}_{\mathcal{T}_{i}}(\phi_{i},C_{i})$;

12

13 end while

Algorithm 1 Wormhole MAML

where the part that differs from conventional MAML is marked in red. The
initialization of $C_{i}$ could have multiple variations depending on the
specific parameterization. For example, $C_{i}$ can be parametrized in one of
the following forms:

$\displaystyle C_{i}=c,\;c\in\mathbb{R}^{1}$ (3) $\displaystyle
C_{i}=\tanh(t),\;t\in\mathbb{R}^{1}$ (4) $\displaystyle
C_{i}=diag(C),\;C\in\mathbb{R}^{d_{\theta}}$ (5) $\displaystyle
C_{i}=diag(C),\;C\in\mathbb{R}^{L}$ (6) $\displaystyle
C_{i}=C,\;C\in\mathbb{R}^{d_{\theta}\times d_{\theta}},Rank(C)<threshold$ (7)

where $d^{\theta}$ is the dimension of the meta-network parameter and $L$ is
the number of layers. This paper mainly focuses on the case of Eq. 3, 4,6. The
initialization of $C_{i}$ is typically around $1$. Both the updates for
$C_{i}$ and $\phi_{i}$ can be more than a single step. In particular, for
$C_{i}=tanh(t)$ parametrization, we typically let $C_{i}$ converge towards
$\pm 1$.

Figure 2: (a) Illustration of vanilla MAML’s optimum. (b) Illustration of
wormhole MAML’s optimum.

The mechanism of Wormhole MAML is summarized in Fig. 2: In conventional
(vanilla) MAML, the optimum in the parameter space for the meta-network need
to be near the task-specific optima, so that all task-specific optima are
reachable within the steps allowed during inner-loop adaption, as illustrated
in Fig. 2a. In many cases, either due to restrictions on computational
resources or data availability, the inner-loop adaption is not allowed to have
long trajectories and it is highly desirable for the meta-network optimum to
be closer to the task-specific optima.

In Wormhole MAML, the choice of $C_{i}=c,\;c\in\mathbb{R}^{1}$ effectively
glues the parameter space in a way as indicated by the color and the arrows on
the boundary, where each color represents the parameter space reachable by
tuning $C_{i}$ alone. When $c=\pm 1$, $c\theta$ can be initialized at either
of the two-star locations depending on the task, which greatly reduces the
adaptation needed in $\Delta_{i}(\theta)$ for $c\theta+\Delta_{i}(\theta)$ to
reach the task-specific optima. Here we want to encourage $C_{i}$ towards $\pm
1$, potentially with $\tanh(t)$ parameterization, since we don’t want $C_{i}$
to absorb all the inner-loop tunability of $\Delta_{i}$, or $C_{i}$ near $0$
causing bad dynamics.

##  3 Related Works

The architecture in our work could interpolate various other approaches.

When the extra multiplicative parameter is shared across weights and not
binarized, our algorithm effectively encourages weights to vary in
coordination during the adaptation phase, thus maintaining a high correlation.
This is conceptually similar to the idea of correlation alignment for
unsupervised domain adaption (CORAL) Sun & Saenko (2016), where correlation in
data distribution is learned. More specifically, CORAL aims to align second-
order statistics of source and target distribution. Despite the conceptual
similarity, CORAL aims at a completely different problem of domain adaptation,
whereas our approach is promoting a certain weight correlation in this
setting.

When the extra multiplicative parameter is selectively frozen, our algorithm
is similar to fast context adaptation (CAVIA) Zintgraf et al. (2019), which
explicitly separates parameters into shared ones and task-specific ones. When
the additive part of the inner-loop adaption is highly constrained, the
multiplicative part is the major source of model expressivity. In that case,
our selection also separates weights into relatively more shared ones and
relatively more task-specific ones. The main difference is that our approach
achieves it softly and potentially leverages information during the training
process, or even in the query data batch.

In the case where the multiplicative parameter is softly binarized with
$\tanh$, potentially under some annealing scheme, the effect of the
multiplicative parameter can be viewed as introducing extra variation in the
learning rate of the network weights, which is similar to the idea of meta-SGD
Li et al. (2017), where the learning rate is also learnable. It is also
similar to MT-net Lee & Choi (2018), where task-specific adaptations are
restricted to a subspace. Despite the conceptual similarity, our approach has
a drastically different update rule and offers more flexible control over the
training dynamics.

##  4 Experiments

To verify the effectiveness of our proposed approach, we compare Wormhole MAML
with the vanilla MAML Finn et al. (2017) in three different meta-learning
tasks: Avg-Threshold, Wavelet Transform, and the MNIST dataset LeCun (1998).
The quantitative comparison results of the three tasks can be found in Table
1.

Table 1: Evaluation results on the Avg-Threshold, Wavelet Transform and MNIST
task.

| Avg-Threshold | Wavelet Transform | MNIST
---|---|---|---
|  1-step |  2-step |  5-step |  1-step |  2-step |  5-step |  1-step |  2-step |  5-step
Vanilla Finn et al. (2017) |  5.60 |  5.08 |  3.33 |  7.14 |  7.63 |  7.49 |  0.500 |  0.463 |  0.312
Wormhole 1 |  1.56 |  1.41 |  1.69 |  5.00 |  6.05 |  5.36 |  0.500 |  0.141 |  0.009
Wormhole d |  2.53 |  2.42 |  2.17 |  - |  - |  - |  - |  - |  -

###  4.1 Avg-Threshold

#### Problem Formulation

We first consider a toy meta-learning problem of telling whether the average
of an array is greater or less than a certain threshold. More specifically,
the dataset has input domain $\mathcal{X}=[0,1]^{d}$ and output domain
$\mathcal{Y}=\\{0,1\\}$. In each task, given a task-specific threshold
$\tau\in[0,1]$ and sign $s\in\\{-1,1\\}$, the goal is for the model to learn a
mapping $\phi$ such that $\phi(x)=1$ if $s*avg(x)>s*\tau$ and 0 otherwise.

#### Implementation Details

We conduct the experiments by training 150 epochs with a task batch size of
10. We set the input dimension $d$ to 5 and use a single linear layer as the
meta-model $\theta$. We use SGD with a learning rate of 0.1 as our optimizer.
The loss function is the cross-entropy error between the predicted and ground-
truth label. Two types of regularization on $C_{i}$ are considered in the
experiments: $C_{i}=\tanh(t),\;t\in\mathbb{R}^{1}$ and
$C_{i}=diag(C),\;C\in\mathbb{R}^{d_{\theta}}$. We consider 1-, 2-, and 5-step
inner loop adaptation in the experiments. The adaptation learning rate is set
to 1.0.

#### Results and Analysis

We report the cross-entropy error of each experiment in Table 1. Wormhole MAML
significantly improves over the vanilla MAML Finn et al. (2017) by reducing
$>2$ cross-entropy error in most of the settings. It is worth noting that the
advantage of using wormhole parameters is still observable even in the setting
of 5-step inner-loop adaptation. We also note that the $\mathit{tanh}$
regularization on $C_{i}$ results in lower error than the $\mathit{diag}$
regularization, indicating strong regularization is beneficial to simple meta-
learning tasks.

To further examine the training behavior under the effect of wormhole
parameters, we show the parameter and loss curve analysis in Fig. 3. Without
the adaptation of wormhole parameters (Fig. 3a), the learned meta-model
$\theta$ shrinks toward 0 for the sake of quick 1-step adaptation to +1 or -1
sign in each task. On the other hand, by adding a wormhole parameter
$C_{i}=\tanh(t),\;t\in\mathbb{R}^{1}$ (Fig. 3b), $\theta$ is able to learn the
meaningful weight that represents averaging and thresholding, with $C_{i}$
acts as a ”sign changer” in the inner-loop adaptation, leading to faster and
lower meta-training convergence.

Figure 3: Parameter and loss curve analysis on the Avg-Threshold task. (a)
Without the adaptation of the wormhole parameter, the learned meta parameter
$\theta$ shrinks toward 0. (b) The learned wormhole parameter serves as a
”sign changer” in the inner loop adaptation.

We note that the benefits in Wormhole MAML largely depend on the capability to
find correct values for the extra multiplicative parameter $C_{i}$ easily and
stably. In this case, the choice of $C_{i}$ is merely a scalar for the linear
network, and is expected to be highly stable given its regularized nature.

We can characterize the stability by analytically examining the asymptotic
behavior of $c$. Without loss of generality, assume positive sign $s=1$. The
loss over a batch of data is:

$L(\mathbf{x},\mathbf{y})=-\sum_{y_{i}=1}\log(\hat{y}_{i})-\sum_{y_{i}=0}\log(1-\hat{y}_{i})$
(8)

where $\hat{y}_{i}=\sigma(c\mathbf{W}^{T}\mathbf{x}_{i})$ is the predicted
label. For simplicity, let $z_{i}=\mathbf{W}^{T}\mathbf{x}_{i}$. The partial
gradient of $c$ is:

$\frac{\partial L(\mathbf{x},\mathbf{y})}{\partial
c}=-\sum_{y_{i}=1}(1-\sigma(cz_{i}))z_{i}-\sum_{y_{i}=0}\sigma(cz_{i})z_{i}$
(9)

To find $c^{*}$ such that $\frac{\partial L(\mathbf{x},\mathbf{y})}{\partial
c}|_{c=c^{*}}=0$, consider the $|c|\ll 1$ case, where we can perform a Taylor
expansion:

$\sigma(cz_{i})\approx cz_{i}$ (10)

Therefore the loss function gradient w.r.t. $c$ can be approximated as:

$\displaystyle\frac{\partial L(\mathbf{x},\mathbf{y})}{\partial c}$
$\displaystyle\approx-\sum_{y_{i}=1}(1-cz_{i})z_{i}-\sum_{y_{i}=0}(cz_{i})z_{i}$
(11)
$\displaystyle=-\sum_{i}I[z_{i}+\Delta\tau>0](1-cz_{i})z_{i}-I[z_{i}+\Delta\tau<0]cz_{i}^{2}$
(12)

where $\Delta\tau=\tau^{*}-\tau$ is distance to the classification boundary
and $I[\cdot]$ denotes the indicator function such that
$y_{i}=I[\mathbf{W}^{T}\mathbf{x_{i}}+\tau^{*}>0]$.

Setting $\frac{\partial L(\mathbf{x},\mathbf{y})}{\partial c}$ to $0$, we get:

$c^{*}=\frac{\sum_{i}I[z_{i}+\Delta t>0]z_{i}}{\sum_{i}z_{i}^{2}}$ (13)

which ideally should be $1$.

From Eq. 13, we see that $t$ should be near $t^{*}$, on the scale of
$std(\mathbf{x}_{i})$. If $t\ll t^{*}$, we have $c\propto t$, or if $t\gg
t^{*}$, we may get $c=0$. These two cases correspond to highly unbalanced data
batch and are both undesirable, leading to unstable convergence of $c$. In the
other cases especially when $|t-t^{*}|\ll std(\mathbf{x}_{i})$, we have $c$
close to $1$ as desired.

In conclusion, the optimality of $c$ requires a balanced data batch, which is
reasonable, as well as relatively accurate model parameters ($\tau$ and $W$ in
this case). Therefore, it may be worth contemplating a scheme where $c$ and
$W$ are co-adapted but at different rates, potentially with annealing that
encourages $c$ to converge faster towards $\pm 1$.

###  4.2 Wavelet Transform

Figure 4: Illustration of the Mexican hat wavelet transforms.

#### Problem Formulation

Another toy example to consider is to approximate wavelet functions from their
dot-product with different vectors. Specifically, we consider a set of Mexican
Hat wavelets $\mathcal{M}$ with different translations, as illustrated in Fig.
4. For each task, we sample $sign\sim\\{-1,1\\},x\sim
uniform(0,1)^{n},m\sim\mathcal{M}$, where $m$ is supported on a discrete set
of $n$ points. The desired filter for the wavelet transform is $f=sign*m$. The
true label is $y=x\cdot f$. Our goal is to learn a $w$ and a $C$ minimizing
$||x\cdot(Cw)-y||^{2}$.

#### Implementation Details

In this experiment we only consider $C_{i}=\tanh(t),\;t\in\mathbb{R}^{1}$.
Unless otherwise mentioned, we adopt 1500 epochs for training, 5 inner loops
for $C_{i}$, and 1, 2, 5 inner loops for $w$. We use 10 shots per task, and a
batch size of 10 tasks. For the learning rate, the inner loop learning rate
for $w$ is $5e-2$ and for $c$ is $1$, and the learning rate for the outer loop
is $1e-3$.

Figure 5: Parameter and loss curve analysis on the Wavelet Transform task. (a)
Without the adaptation of the wormhole parameter, the learned meta wavelet
centers toward 0. (b) With the wormhole parameter learning sign and scale, the
learned meta wavelet only needs to learn the shape.

#### Results and Analysis

Ideally, the meta $w$ would be in a bump-shape ranging in $[-0.4,0.8]$, or
$[-0.8,0.4]$ which encompasses all wavelets under a single sign, since with
$C_{i}$ as a sign changer $w$ only needs to learn the shape of the bump.
However, without introducing $C_{i}$, the meta $w$ would simply center around
0 because the conflicting gradients of each task pull the values around zero.
We further show that our results conform with the previous analysis in Fig. 5.
Here for the sake of demonstration, we initialize $w$ randomly from $-2$ to
$3$, and the inner loop adaptation for $C_{i}$ is 10 and for $w$ is $2$. As
shown in the figure, without the adaptation of the wormhole parameter, the
learned meta wavelet centers toward 0 at iteration 1500, while with the
wormhole parameter learning sign and scale, the learned meta wavelet only
needs to learn the shape, because at iteration 1500 the distribution of
$w_{i}$ clearly shifts pass zero. This leads to a significant drop in the L1
error as well.

###  4.3 MNIST Dataset

#### Problem Formulation

We consider a 2-way 1-shot classification on the MNIST dataset LeCun (1998).
Under the setting of 2-way classification, labels for the same digit class
could be the complete opposite across different tasks, contributing to
conflicting gradients during training.

#### Implementation Details

For the MNIST dataset LeCun (1998), we use a model that consists of two
$5\times 5$ convolutions followed by a ReLU nonlinearity and 2 × 2 max-
pooling, and two linear layers with a ReLU nonlinearity. We train the model
for 100 epochs with a task batch size of 16. The wormhole parameter
$C_{i}=\tanh(t),\;t\in\mathbb{R}^{1}$ is added to the last linear layer of the
model. An Adam optimizer with a learning rate of 0.005 is used in each
experiment. We adopt the cross-entropy error between the predicted and ground-
truth label as the loss function. We consider 1-, 2-, and 5-step inner loop
adaptation in the experiments. The adaptation learning rate is set to 0.01.

#### Results and Analysis

The meta-test classification error rate of each experiment is reported in
Table 1. While no advantage is observed due to training instability under the
setting of 1-step adaptation, Wormhole MAML improves the vanilla MAML Finn et
al. (2017) by a large margin in both the 2- and 5-step settings, achieving the
error rate of 0.141 and 0.009 respectively.

##  5 Limitation and Future Work

As mentioned in the result analysis, the optimality of the multiplicative
parameter $C_{i}$ found during inner-loop adaptation relies on relatively
reasonable model parameters. While in many practical scenarios, a reasonable
meta-parameter $\theta$ can lead to stable training dynamics for $C_{i}$, in
some cases, one might need to optimize over $C_{i}$ and $\Delta_{i}$ at the
same time. This potentially complicates the update rule, where some additional
design is needed. For example, an annealing scheme may be worth investigating
where $C_{i}$ converges faster than $\Delta_{i}$. In that case, the benefits
of Wormhole MAML may slightly diminish due to the hassle of hyperparameter
tuning.

It remains unclear how scalable Wormhole MAML is on more general problem sets.
In more complicated problem settings with higher-dimensional models, the
gradients are less likely to be completely contradictory, thus lessening the
benefits of Wormhole MAML.

In addition, the choice of the form of $C_{i}$ is worth optimizing. A smaller
dimension of $C_{i}$ ensures reliable and efficient optimization during each
inner loop, but in some cases the expressivity may not be enough. A natural
generalization of our approach would be to have $C_{i}$ in the same dimension
as $\theta$, but constrained to be within a low-dimensional manifold, e.g. by
a generative model. Taking one step further, one could constrain $\phi_{i}$ to
a low-dimensional manifold. One potential problem with generalizing is the
difficulty in propagating gradient through the inner-loop adaptation. More
investigation is needed to offer insights on meta-learning in such glued
parameter space.

## References

  * Duan et al. (2017) Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, and Pieter Abbeel.  RL^2: Fast reinforcement learning via slow reinforcement learning, 2017.  URL https://openreview.net/forum?id=HkLXCE9lx.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _International conference on machine learning_ , pp. 1126–1135. PMLR, 2017.
  * Flennerhag et al. (2019) Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell.  Meta-learning with warped gradient descent.  _arXiv preprint arXiv:1909.00025_ , 2019.
  * LeCun (1998) Yann LeCun.  The mnist database of handwritten digits.  _http://yann. lecun. com/exdb/mnist/_ , 1998.
  * Lee & Choi (2018) Yoonho Lee and Seungjin Choi.  Gradient-based meta-learning with learned layerwise metric and subspace.  In Jennifer Dy and Andreas Krause (eds.), _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pp. 2927–2936. PMLR, 10–15 Jul 2018.
  * Li et al. (2017) Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li.  Meta-sgd: Learning to learn quickly for few-shot learning.  _arXiv preprint arXiv:1707.09835_ , 2017.
  * Lou et al. (2021) Beicheng Lou, Nathan Zhao, and Jiahui Wang.  Meta-learning from sparse recovery.  In _Fifth Workshop on Meta-Learning at the Conference on Neural Information Processing Systems_ , 2021.
  * Rajendran et al. (2020) Janarthanan Rajendran, Alexander Irpan, and Eric Jang.  Meta-learning requires meta-augmentation.  _Advances in Neural Information Processing Systems_ , 33:5705–5715, 2020.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Sun & Saenko (2016) Baochen Sun and Kate Saenko.  Deep coral: Correlation alignment for deep domain adaptation.  In _European conference on computer vision_ , pp. 443–450. Springer, 2016.
  * Yu et al. (2020) Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.  Gradient surgery for multi-task learning.  _Advances in Neural Information Processing Systems_ , 33:5824–5836, 2020.
  * Zhang et al. (2020) Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie.  Why gradient clipping accelerates training: A theoretical justification for adaptivity.  In _International Conference on Learning Representations_ , 2020.
  * Zintgraf et al. (2019) Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson.  Fast context adaptation via meta-learning.  In _International Conference on Machine Learning_ , pp. 7693–7702. PMLR, 2019.


# Impact of Architectural Modifications on Deep Learning Adversarial
Robustness

Firuz Juraev1, Mohammed Abuhamad2, Simon S. Woo1,3,
George K Thiruvathukal2, and Tamer Abuhmed1
1 Department of Computer Science and Engineering, Sungkyunkwan University
2 Department of Computer Science, Loyola University Chicago
3 Department of Applied Data Science, Sungkyunkwan University
Emails<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Rapid advancements of deep learning are accelerating adoption in a wide
variety of applications, including safety-critical applications such as self-
driving vehicles, drones, robots, and surveillance systems. These advancements
include applying variations of sophisticated techniques that improve the
performance of models. However, such models are not immune to adversarial
manipulations, which can cause the system to misbehave and remain unnoticed by
experts. The frequency of modifications to existing deep learning models
necessitates thorough analysis to determine the impact on models’ robustness.
In this work, we present an experimental evaluation of the effects of model
modifications on deep learning model robustness using adversarial attacks. Our
methodology involves examining the robustness of variations of models against
various adversarial attacks. By conducting our experiments, we aim to shed
light on the critical issue of maintaining the reliability and safety of deep
learning models in safety- and security-critical applications. Our results
indicate the pressing demand for an in-depth assessment of the effects of
model changes on the robustness of models††footnotetext: Corresponding author:
Tamer Abuhmed (tamer@skku.edu).

###### Index Terms:

Deep Learning, Model robustness, Adversarial Attacks, Defenses, Computer
Vision

##  I Introduction

Deep Learning (DL) has proven to be exceptional at solving real-world problems
that traditional Machine Learning (ML) approaches could not address
effectively [1, 2]. The DL methods have achieved significant advances in the
traditional domains of computer vision, especially with the emergence of Deep
Neural Networks (DNNs) and the availability of high-performance hardware to
train complicated models [3]. Important issues in the DL methods have also
been disclosed simultaneously. One example is its vulnerability to
undetectable input perturbations at test time [4, 5]. For the image
classification task, Szegedy et al. [6] initially developed minor
perturbations on the input image, which tricked state-of-the-art DNNs with a
high confidence score. The perturbed images are commonly-known as adversarial
samples. This has led to a new dimension to the adversarial DL paradigm, which
entails creating DNNs that are robust to such adversarial samples. Several
factors must be addressed when designing robust networks, including what
perturbations the adversary may apply to the input and what knowledge the
adversary has about the model and the system.

There are two types of adversarial attacks: 1 training stage attacks and 2
testing stage attacks. The training stage attacks modify the training dataset
[7], the input features, or/and the data labels/classes in order to attack the
target model. Any of these types of modification meet the definition of
perturbations that must be understood in order for DNNs to be robust. The
testing stage attacks are two main types: A white-box attacks and B black-box
attacks. The white-box attack is fully aware of the DL method, including its
underlying structure and parameters. On the other hand, the black-box attacks
cannot access knowledge about the target model, but they may train a local
substitute model by querying the target model [8], exploiting the
transferability of adversarial samples, or employing a model inversion
approach.

The robustness of deep neural network models has been the subject of extensive
research, with numerous studies investigating this issue from various angles
[5]. However, with the emergence of new versions and updates of existing
models designed to improve performance, it is crucial to understand the impact
of these changes on model robustness. For instance, the Inception model was
improved by incorporating residual connections, resulting in state-of-the-art
performance in the 2015 ILSVRC challenge. Many other advancements in the field
have led to novel techniques that can be applied to various models to improve
their performance. In this study, we aim to investigate how variations of
models affect their robustness against adversarial attacks. Specifically, we
use three well-known white-box attacks (Fast Gradient Sign Method (FGSM) [9],
Projected Gradient Descent (PGD) [10], and Carlini & Wagner (C&W) [11]) to
evaluate the robustness of different versions of the VGG, Inception, and
MobileNet architectures. Our main research question is “What are the effects
of architectural modifications on models’ robustness?”

Figure 1: The accuracy of adopted models.

Contributions. Our contributions are summarized as follows:

  * 1

We investigate the impact of batch normalization on the performance and
robustness of VGG-based models under adversarial scenarios. Through a series
of experiments targeting variations of VGG models via various attacks, we
highlight the necessity for careful consideration when using batch
normalization, as its benefits in stabilizing and accelerating the training
and improving the performance might also be associated with potential
drawbacks in terms of robustness.

  * 2

We analyze variations of the Inception architectures, such as Inception V4 and
Inception ResNet V2 under various adversarial scenarios. Our analysis shows
that newer models, e.g., Inception ResNet V2 exhibit improved performance and
higher robustness to adversarial attacks. In particular, we show that these
models can withstand adversarial attacks with longer attack times and higher
noise rates, indicating a higher level of robustness.

  * 3

We also explored small and large variants of MobileNet against adversarial
scenarios. Our results suggest that MobileNet V3 outperforms MobileNet V2
while maintaining a high level of robustness. This can be contributed to the
incorporation of the squeeze-and-excitation module (which allows the model to
suppress noisy features) and hard-swish activation functions.

Organization.. The study is organized as follows. In Section II, we discuss
the dataset and studied models, Section III shows our results and
observations, and in Section IV and V we provide some discussion and
conclusion for our study.

##  II Experimental Settings

In this section, we discuss the settings of our experiments including the used
dataset, models, and evaluation metrics.

TABLE I: The parameters of selected models.

Model |  Number of parameters |  Number of Layers |  Activation functions
---|---|---|---
VGG 11 | 132,863,336 | 11 | ReLU
VGG 13 | 133,047,848 | 13 | ReLU
VGG 16 | 138,357,544 | 16 | ReLU
VGG 19 | 143,667,240 | 19 | ReLU
VGG 11 BN | 132,868,840 | 11 | ReLU
VGG 13 BN | 133,053,736 | 13 | ReLU
VGG 16 BN | 138,365,992 | 16 | ReLU
VGG 19 BN | 143,678,248 | 19 | ReLU
Inception V3 | 27,161,264 | 48 | ReLU
Inception V4 | 42,679,816 | 49 | ReLU
Inception ResNet V2 | 55,843,464 | 164 | ReLU
MobileNet V2 | 3,504,872 | 53 | ReLU
MobileNet V3 (small) | 2,542,856 | 65 | Hardswish
MobileNet V3 (large) | 5,483,032 | 157 | Hardswish

Dataset. In this work, we use the widely recognized and extensively used
ImageNet dataset, which is composed of 14 million images. Each image in the
dataset has a resolution of 224x224 pixels and belongs to one of 1000 classes.
For our experiments, we use 1000 test images for each attack. Those images are
selected based on two conditions: 1 one example from each class of ImageNet
and 2 each image must be classified correctly by all selected models.

Models. In this study, we employed a total of fourteen models from three
commonly used families of convolutional neural networks: VGG, Inception, and
MobileNet. Each family included different versions and types of models. We
utilized pre-trained models on the PyTorch framework for our experiments on
the ImageNet dataset. The characteristics and performance of the adopted pre-
trained models are shown in Table I and Figure 1, respectively. In Table I, we
show the key aspects of each model, such as the architecture, the number of
layers, and the number of parameters. Figure 1 shows the top-5 accuracy of
each model using ImageNet dataset.

Figure 2: The impact of Batch normalization on VGG models. Figure 3: Attack
success rate, attack time, and attack noise rate on VGG and VGG BN models.

Evaluation Metrics. For evaluating the robustness of the models, we utilize
the following evaluation metrics.

  * •

Attack success rate: This is calculated by dividing the number of successful
attacks by the total number of attempts.

  * •

Attack time: We measured the time it took to add a specific level of noise to
a benign image to achieve a successful attack. The results are presented as
the average attack time in seconds for each model.

  * •

Noise rate: The amount of noise in the image is determined by using a metric
called Structural Similarity Index Measure (SSIM) [12]. SSIM measures how two
images are similar to each other. For calculating the noise rate, we subtract
the SSIM value from 1 (i.e., noise rate = $1-SSIM$). SSIM is calculated as
follows.

$\text{SSIM}(x,y)=\frac{(2\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})}$

where $x$ and $y$ are the two input images being compared, $\mu_{x}$ and
$\mu_{y}$ are the mean values of $x$ and $y$, $\sigma_{x}$ and $\sigma_{y}$
are the standard deviations of $x$ and $y$, $\sigma_{xy}$ is the cross-
covariance of $x$ and $y$, and $c_{1}$ and $c_{2}$ are small constants added
to avoid division by zero.

Experiment Workstation. The experiments are conducted on a machine equipped
with an Intel Xeon(R) CPU E5-2620 v3 @ 2.40 GHz$\times$ 24 with Cuda-10.0 and
three GEFORCE GTX TITANx 12 GB GPUs, as well as Python 3.7.7 distributed in
Anaconda 4.8.3 (64-bit).

Figure 4: Attack success rate, attack time, and attack noise rate on Inception
models. Figure 5: Attack success rate, attack time, and attack noise rate on
MobileNet models.

##  III Experimental results

In our experiments, we evaluated the robustness of the models using 1,000
images in terms of attack success rate, attack time, and noise rate. The main
goal of the experiments is to examine the effects of architectural designs and
techniques of deep neural models on performance and robustness. The study
contributes to understanding the possible benefits and drawbacks of including
various techniques in model designs, and how they impact the performance and
robustness of the models. Particularly, we focus on analyzing the effects of
batch normalization and architectural variations on three popular deep
learning architectures, namely VGG, Inception, and MobileNet.

###  III-A Batch Normalization Effects on VGG models

Batch normalization (BN) is a technique used in deep neural networks to
increase training speed and stability. It works by normalizing each layer’s
inputs to have a zero mean and unit variance across a small batch of training
data. BN is needed in deep neural networks to possibly address internal
covariate shifts, improve stability, act as a form of regularization, and
allow for higher learning rates. Figure 1-(a) shows the comparison between the
accuracy of VGG models with and without BN. The figure demonstrates that
incorporating batch normalization improves the performance of VGG models.
Specifically, the accuracy of VGG 11 increased from 88.75% to 89.82% when
using BN. Similarly, in VGG 13 (increased from 89.26% to 90.49%), in VGG 16
(increased from 90.35% to 91.61%), and in VGG 19 (increased from 90.82% to
92.07%).

Although BN can enhance the stability and accuracy of the model and reduce
overfitting, our results, as depicted in Figure 2, demonstrate that
incorporating BN in VGG models can increase the success rate of adversarial
attacks. In terms of attack time and noise rate, our results (as shown in
Figure 3) indicate that BN has an unnoticeable effect. One possible reason for
this result is that BN can make the models more susceptible to gradient
masking, which occurs when the gradients of the loss function with respect to
the input become small or zero. This can contribute to decreasing the
diversity of feature representations, which can make it difficult for the
model to detect and respond to adversarial perturbations, leading to a
decrease in overall robustness [13].

While BN can add additional parameters and non-linearity to a model, which may
make it more difficult for adversarial attacks to succeed, our experiments
have shown that VGG models with BN are actually more vulnerable. Specifically,
when looking at FGSM attacks, we found that the attack success rate was 3.93
percentage points higher on VGG models with BN compared to those without BN.
Similarly, on the PGD attack, VGG models with BN had a 0.63 percentage point
higher attack success rate than models without BN. Interestingly, on the C&W
attack, we found that VGG models without BN were more robust than VGG models
with BN, with the attack success rate being 1.80 percentage points lower on
models without BN (Figure 2).

Generally, BN in deep learning models can lead to improved performance and
training speed, but our experiments suggest that it may negatively impact the
model’s robustness. Further exploration is necessary to better understand this
trade-off, including investigating the impact of BN across a broader range of
models, datasets, and attack scenarios.

In addition, we compared the performance of four different versions of the VGG
model (i.e., VGG 11, VGG 13, VGG 16, and VGG 19) to analyze how the number of
layers in the model affects its robustness to adversarial attacks in Figure 3.
the results show that as the number of layers increases, the model becomes
more robust as indicated by a decrease in attack success rate and an increase
in attack time. However, the noise rate is stable across all versions.

###  III-B Architectural Updates Effects on Inceptions

The next experiment in this study was conducted using different versions of
the Inception architecture, including Inception V3, Inception V4, and
Inception ResNet V2. Both Inception V4 and Inception ResNet V2 demonstrated a
superior performance of 95.3% (top-5 accuracy) compared to Inception V3 with
93.45% (Figure 1-(b), which was attributed to their various architectural
improvements such as residual connections, more aggressive factorization,
batch normalization, and bottleneck designs. Particularly, Inception V4
features a new “stem” module that reduces the computational cost [14].

Figure 6: Amount of noise, confidence, and time needed by the PGD attack for
Inception V3, Inception V4, and Inception ResNet V2 models. Images with a
single object and a clear background ((B) and (C)) require more added noise
and time.

In Figure 4, Inception V4 and Inception ResNet V2 demonstrate higher
robustness to adversarial attacks than Inception V3. The average attack
success rates were 48.57% and 40.23% for Inception V4 and Inception ResNet V2,
respectively, compared to 73.33% for Inception V3. Inception V4 and Inception
ResNet V2 also required longer attack times, with average times of 2.40 and
3.50 seconds, respectively, compared to 1.91 seconds for Inception V3.
Additionally, Inception V4 and Inception ResNet V2 had higher average noise
rates of 0.00961 and 0.00976, respectively, compared to 0.00907 for Inception
V3, indicating their increased robustness to attacks.

To better understand the importance of additional metrics, such as attack time
and noise rate, in evaluating model robustness, we provide examples in Figure
6. In Figure 6-(a), we show an example of a PGD attack on Inception V3,
Inception V4, and Inception ResNet V2, demonstrating that as the model becomes
more complex, it becomes more difficult to attack and requires more time to
attack. For instance, attacking Inception V3 took 1.47 seconds while attacking
Inception V4 and Inception ResNet V2 took 2.17 and 4.13 seconds, respectively.
However, the importance of noise in evaluating model robustness is difficult
to distinguish in Figure 6-(a) since it remains relatively constant across all
models. Therefore, to better illustrate its importance, we allowed the attack
to add more noise by increasing the epsilon (from 0.01 to 0.04) of the PGD
attack and tested it on challenging images with a single object and a clear
background, making it more difficult for the attack to succeed. Figure 6-(b)
and -(c) show that the attack adds more noise to the complex models, such as
Inception V4 and Inception ResNet V2, compared to Inception V3.

###  III-C Architectural Updates Effects on MobileNets

In this experiment, we analyzed the performance of different versions of
MobileNet, including MobileNet V2, MobileNet V3 (small), and MobileNet V3
(large). MobileNet V3 is an improved version of the MobileNet architecture,
with both small and large variants available. MobileNet V3 demonstrated better
performance compared to MobileNet V2 (Figure 1 c), thanks to several
enhancements made to the architecture [15], including the integration of the
squeeze-and-excitation module, the use of hard-swish activation functions, and
improved batch normalization techniques. MobileNet V2 has a top-5 accuracy of
90.29%, while MobileNet V3 (small) and (large) have an accuracy of 87.40% and
91.34%, respectively.

Our experiments, as shown in Figure 5, show that MobileNet V3 (small)
demonstrated superior robustness despite having significantly fewer parameters
(2,542,856) than MobileNet V2 (3,504,872), and even though its accuracy is not
higher than MobileNet V2. This suggests that MobileNet V3 (small) has more
robust capabilities, making it more resilient against adversarial attacks.
Specifically, MobileNet V2 had an average attack success rate of 97.63%
compared to MobileNet V3 (small) with 92.27%. Moreover, MobileNet V3 (large),
which has more parameters, also exhibited higher robustness with an attack
success rate of 91.10% compared to MobileNet V2. However, the attack time and
noise rate did not follow a clear pattern as the attack success rate.

One of the specific improvements that contributed to the increased robustness
of MobileNet V3 was the integration of the squeeze-and-excitation (SE) module,
which selectively emphasized informative features and suppressed irrelevant
ones for more accurate and robust feature extraction. Furthermore, using hard-
swish activation functions instead of traditional ReLU activation functions
helped mitigate the saturation issue that can occur when using ReLU
activations, making the model more resilient to adversarial attacks. Lastly,
the enhanced batch normalization techniques used in MobileNet V3 helped better
normalize the activations within the network, leading to improved performance
and robustness.

##  IV Discussion

Based on our experiments, we have observed several key findings that can
inform the design of deep learning models for improved robustness against
adversarial attacks. Firstly, while batch normalization can improve training
speed and stability, it may come at the cost of decreased model robustness.
Therefore, when using batch normalization, it is important to carefully
evaluate its impact on model robustness and consider alternative normalization
techniques such as layer normalization or instance normalization. In [16], Liu
et al. explored the application of normalization techniques to enhance the
training speed and the overall performance robustness of convolutional models.
Their results indicated that the models exhibit better performance outcomes by
incorporating normalization methods into the training process and demonstrate
increased robustness to variations or perturbations in the data. Furthermore,
Amini et al. [17] suggested that deep learning models’ robustness can be
enhanced through the implementation of non-smooth regularization techniques.
Secondly, we also noticed that incorporating architectural improvements such
as residual connections [18], factorization [19], bottleneck designs [18], and
squeeze-and-excitation modules [20] can lead to more robust models. These
improvements help the model better capture informative features while
suppressing irrelevant ones, leading to better feature extraction and more
resilient models. Lastly, using activation functions that avoid saturation
issues, such as hard-swish instead of traditional ReLU activations, can also
improve model robustness. As a future direction, it would be valuable to
evaluate the robustness of updated models across a broader range of
architectures, datasets, and attack scenarios [21, 22, 23].

##  V Conclusion

In conclusion, our experiments demonstrate that updates and new versions of
deep learning models can have a significant impact on their robustness to
adversarial attacks. While batch normalization can improve the training speed
and stability of models such as VGG, our results suggest that it may
compromise model robustness. On the other hand, updates to the Inception and
MobileNet architectures, such as the integration of squeeze-and-excitation
modules and improved batch normalization, can lead to improved model
robustness. These findings highlight the importance of conducting
comprehensive and in-depth research to evaluate the impact of model updates on
robustness in critical applications. Future work could explore the
generalizability of these results across a broader range of datasets and
attack scenarios, and investigate other potential techniques to improve model
performance and robustness.

## Acknowledgment

This work was supported by the National Research Foundation of Korea(NRF)
grant funded by the Korea government(MSIT)(No. 2021R1A2C1011198), (Institute
for Information & communications Technology Planning & Evaluation) (IITP)
grant funded by the Korea government (MSIT) under the ICT Creative Consilience
Program (IITP-2021-2020-0-01821), and AI Platform to Fully Adapt and Reflect
Privacy-Policy Changes (No. 2022-0-00688).

## References

  * [1] M. Abuhamad, T. Abuhmed, D. Mohaisen, and D. Nyang, “Large-scale and robust code authorship identification with deep feature learning,” _ACM Transactions on Privacy and Security (TOPS)_ , vol. 24, no. 4, pp. 1–35, 2021.
  * [2] S. Ali, O. Abusabha, F. Ali, M. Imran, and T. Abuhmed, “Effective multitask deep learning for iot malware detection and identification using behavioral traffic analysis,” _IEEE Transactions on Network and Service Management_ , 2022.
  * [3] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay, “A survey on adversarial attacks and defences,” _CAAI Transactions on Intelligence Technology_ , vol. 6, no. 1, pp. 25–45, 2021.
  * [4] E. Abdukhamidov, M. Abuhamad, S. S. Woo, E. Chan-Tin, and T. Abuhmed, “Hardening interpretable deep learning systems: Investigating adversarial threats and defenses,” _IEEE Transactions on Dependable and Secure Computing_ , 2023.
  * [5] F. Juraev, E. Abdukhamidov, M. Abuhamad, and T. Abuhmed, “Depth, breadth, and complexity: Ways to attack and defend deep learning models,” in _Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security_ , 2022, pp. 1207–1209.
  * [6] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in _International Conference on Learning Representations, ICLR 2014_ , Banff, Canada, 2014, pp. 1–10.
  * [7] M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. Tygar, “Can machine learning be secure?” in _Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security_. Association for Computing Machinery, 2006, p. 16–25. [Online]. Available: 10.1145/1128817.1128824
  * [8] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in _Proceedings of the 2017 ACM on Asia conference on computer and communications security_ , 2017, pp. 506–519.
  * [9] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA_. OpenReview.net, 2015.
  * [10] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” in _5th International Conference on Learning Representations, ICLR 2017, Toulon, France_. OpenReview.net, 2017.
  * [11] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 ieee symposium on security and privacy (sp)_. IEEE, 2017, pp. 39–57.
  * [12] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE Transactions on Image Processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [13] F. Boenisch, P. Sperl, and K. Böttinger, “Gradient masking and the underestimated robustness threats of differential privacy in deep learning,” _arXiv preprint arXiv:2105.07985_ , 2021.
  * [14] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 31, no. 1, 2017.
  * [15] A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan _et al._ , “Searching for mobilenetv3,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 1314–1324.
  * [16] S. Liu, X. Li, Y. Zhai, C. You, Z. Zhu, C. Fernandez-Granda, and Q. Qu, “Convolutional normalization: Improving deep convolutional network robustness and training,” _Advances in neural information processing systems_ , vol. 34, pp. 28 919–28 928, 2021.
  * [17] S. Amini and S. Ghaemmaghami, “Towards improving robustness of deep neural networks to adversarial perturbations,” _IEEE Transactions on Multimedia_ , vol. 22, no. 7, pp. 1889–1903, 2020.
  * [18] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [19] M. Wang, B. Liu, and H. Foroosh, “Factorized convolutional neural networks,” in _Proceedings of the IEEE international conference on computer vision workshops_ , 2017, pp. 545–553.
  * [20] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [21] E. Abdukhamidov, F. Juraev, M. Abuhamad, and T. Abuhmed, “Black-box and target-specific attack against interpretable deep learning systems,” in _Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security_ , 2022, pp. 1216–1218.
  * [22] E. Abdukhamidov, M. Abuhamad, F. Juraev, E. Chan-Tin, and T. AbuHmed, “Advedge: Optimizing adversarial perturbations against interpretable deep learning,” in _International Conference on Computational Data and Social Networks_ , 2021, pp. 93–105.
  * [23] E. Abdukhamidov, M. Abuhamad, S. S. Woo, E. Chan-Tin, and T. Abuhmed, “Interpretations cannot be trusted: Stealthy and effective adversarial perturbations against interpretable deep learning,” _arXiv preprint arXiv:2211.15926_ , 2022.