architectural defense mechanisms against adversarial examples. For example,
xie_mitigating_2018 utilized random resizing and random padding to destroy the
specific structure of adversarial perturbations; wang2018adversarial and
wang2018deep improved the robustness of neural networks by replacing the
output layers with novel interpolating functions; In xie_feature_2019, the
authors designed a feature denoising filter that can remove the perturbation’s
pattern from feature maps. In this work, we explore the intrinsic robustness
of a specific novel architecture (neural ODE), and show that the proposed
TisODE can improve the robustness of deep networks and can also work in tandem
with these state-of-the-art methods to achieve further improvements.

###  3.6 Chapter Summary

In this chapter, we first empirically study the robustness of neural ODEs. Our
studies reveal that neural ODE-based models are superior in terms of
robustness compared to CNN models. We then explore how to further boost the
robustness of vanilla neural ODEs and propose the TisODE. Finally, we show
that the proposed TisODE outperforms the vanilla neural ODE and also can work
in conjunction with other state-of-the-art techniques to further improve the
robustness of deep networks. Thus, the TisODE method is an effective drop-in
module for building robust deep models.

###  3.7 Appendices

####  3.7.1 Networks used on the MNIST, the SVHN, and the ImgNet10 datasets

Table 3.5: The architectures of the ODEnets on different datasets.

MNIST | Repetition | Layer
---|---|---
FE |  $\times$1 | Conv(1, 64, 3, 1) + GroupNorm + ReLU
$\times$1 | Conv(64, 64, 4, 2) + GroupNorm + ReLU
RM |  $\times$2 | Conv(64, 64, 3, 1) + GroupNorm + ReLU
FCC |  $\times$1 | AdaptiveAvgPool2d + Linear(64,10)
SVHN | Repetition | Layer
FE |  $\times$1 | Conv(3, 64, 3, 1) + GroupNorm + ReLU
$\times$1 | Conv(64, 64, 4, 2) + GroupNorm + ReLU
RM |  $\times$2 | Conv(64, 64, 3, 1) + GroupNorm + ReLU
FCC |  $\times$1 | AdaptiveAvgPool2d + Linear(64,10)
ImgNet10 | Repetition | Layer
FE |  $\times$1 | Conv(3, 32, 5, 2) + GroupNorm
$\times$1 | MaxPooling(2)
$\times$1 | BaiscBlock(32, 64, 2)
$\times$1 | MaxPooling(2)
RM |  $\times$3 | BaiscBlock(64, 64, 1)
FCC |  $\times$1 | AdaptiveAvgPool2d + Linear(64,10)

In Table 3.5, the four arguments of the Conv layer represent the input
channel, output channel, kernel size, and the stride. The two arguments of the
Linear layer represents the input dimension and the output dimension of this
fully-connected layer. In the network on the ImgNet10, the BasicBlock refers
to the standard architecture in [he_deep_2016], the three arguments of the
BasicBlock represent the input channel, output channel and the stride of the
Conv layers inside the block. Note that we replace the BatchNorm layers in
BasicBlocks as the GroupNorm to guarantee that the dynamics of each datum is
independent of other data in the same mini-batch.

####  3.7.2 The Construction of ImgNet10 dataset

Table 3.6: The corresponding indexes to each class in the original ImageNet
dataset

Class | Indexing
---|---
dog | n02090721, | n02091032, | n02088094
bird | n01532829, | n01558993, | n01534433
car | n02814533, | n03930630, | n03100240
fish | n01484850, | n01491361, | n01494475
monkey | n02483708, | n02484975, | n02486261
turtle | n01664065, | n01665541, | n01667114
lizard | n01677366, | n01682714, | n01685808
bridge | n03933933, | n04366367, | n04311004
cow | n02403003, | n02408429, | n02410509
crab | n01980166, | n01978455, | n01981276

####  3.7.3 Gronwall’s Inequality

We formally state the Gronwall’s Inequality here, following the version in
[howard1998gronwall].

###### Theorem 3.7.1

Let $U\subset{\mathbb{R}}^{d}$ be an open set. Let
$f:U\times[0,T]\rightarrow{\mathbb{R}}^{d}$ be a continuous function and let
${\bm{z}}_{1}$, ${\bm{z}}_{2}$: $[0,T]\rightarrow U$ satisfy the initial value
problems:

$\displaystyle\frac{\mathrm{d}{\bm{z}}_{1}(t)}{\mathrm{d}t}$
$\displaystyle=f({\bm{z}}_{1}(t),t),\quad{\bm{z}}_{1}(t)={\bm{x}}_{1}$
$\displaystyle\frac{\mathrm{d}{\bm{z}}_{2}(t)}{\mathrm{d}t}$
$\displaystyle=f({\bm{z}}_{2}(t),t),\quad{\bm{z}}_{2}(t)={\bm{x}}_{2}$

Assume there is a constant $C\geq 0$ such that, for all $t\in[0,T]$,

$\|f({\bm{z}}_{2}(t),t)-f({\bm{z}}_{1}(t),t))\|\leq
C\|{\bm{z}}_{2}(t)-{\bm{z}}_{1}(t)\|$

Then, for any $t\in[0,T]$,

$\|{\bm{z}}_{1}(t)-{\bm{z}}_{2}(t)\|\leq\|{\bm{x}}_{2}-{\bm{x}}_{1}\|\cdot
e^{Ct}.$

####  3.7.4 More experimental results

##### Comparison in the setting of Adversarial training

We implement the adversarial training of the models on the MNIST dataset, and
the adversarial examples for training are generated in real-time via the FGSM
method (epsilon=0.3) during each epoch [madry_towards_2018]. The results of
the adversarially trained models are shown in Table 3.7. We can observe that
the neural ODE-based models are consistently more robust than CNN models. The
proposed TisODE also outperforms the vanilla neural ODE.

Table 3.7: Classification accuracy (%) on perturbed images from MNIST. To
evaluate the robustness of classifiers, we use three types of perturbations,
namely zero-mean Gaussian noise with standard deviation $\sigma$, FGSM attack
and PGD attack.

| Gaussian noise | Adversarial attack
---|---|---
MNIST | $\sigma=\nicefrac{{100}}{{255}}$ | FGSM-0.3 | FGSM-0.5 | PGD-0.3
CNN | 58.0 | 98.4 | 21.1 | 5.3
ODEnet | 84.2 | 99.1 | 36.0 | 12.3
TisODE | 87.9 | 99.1 | 66.5 | 78.9

##### Experiments on the CIFAR10 dataset

We conduct experiments on CIFAR10 to compare the robustness of CNN and neural
ODE-based models. We train all the models only with original non-perturbed
images and evaluate the robustness of models against random Gaussian noise and
FGSM adversarial attacks. The results are shown in Table 3.8. We can observe
that the ONENet is more robust than the CNN model in terms of both the random
noise and the FGSM attack. Besides, our proposal, TisODE, can improve the
robustness of the vanilla neural ODE.

Table 3.8: Classification accuracy (%) on perturbed images from CIFAR10. To
evaluate the robustness of classifiers, we use two types of perturbations,
namely zero-mean Gaussian noise with standard deviation $\sigma$ and FGSM
attack.

| Gaussian noise | Adversarial attack
---|---|---
CIFAR10 | $\sigma=\nicefrac{{15}}{{255}}$ | $\sigma=\nicefrac{{20}}{{255}}$ | FGSM-8/255 | FGSM-10/255
CNN | 70.2 | 57.6 | 24.3 | 18.4
ODEnet | 72.6 | 60.6 | 31.2 | 26.0
TisODE | 74.3 | 62.0 | 33.6 | 26.8

Here, we control the number of parameters to be the same for all kinds of
models. We use a small network, which consists of five convolutional layers
and one linear layer.

Table 3.9: The architecture of the ODEnet on CIFAR10.

| Repetition | Layer
---|---|---
FE |  $\times$1 | Conv(3, 16, 3, 1) + GroupNorm + ReLU
$\times$1 | Conv(16, 32, 3, 2) + GroupNorm + ReLU
|  $\times$1 | Conv(32, 64, 3, 2) + GroupNorm + ReLU
RM |  $\times$2 | Conv(64, 64, 3, 1) + GroupNorm + ReLU
FCC |  $\times$1 | AdaptiveAvgPool2d + Linear(64,10)

##### An extension on the comparison between CNNs and ODEnets

Here, we compare CNN and neural ODE-based models by controlling both the
number of parameters and the number of function evaluations. We conduct
experiments on the MNIST dataset, and all the models are trained only with
original non-perturbed images.

For the neural ODE-based models, the time range is set from 0 to 1. We use the
Euler method, and the step size is set to be 0.05. Thus the number of
evaluations is $1/0.05=20$. For the CNN models (specifically ResNet), we
repeatedly concatenate the residual block for 20 times, and these 20 blocks
share the same weights. Our experiments show that, in this condition, the
neural ODE-based models still outperform the CNN models (FGSM-0.15: 87.5% vs.
81.9%, FGSM-0.3: 53.4% vs. 49.7%, PGD-0.2: 11.8% vs. 4.8%).

##  Chapter 4 Improving Adversarial Robustness of CNNs via Channel-wise
Importance-based Feature Selection

We investigate the adversarial robustness of CNNs from the perspective of
channel-wise activations. By comparing normally trained and adversarially
trained models, we observe that adversarial training (AT) robustifies CNNs by
aligning the channel-wise activations of adversarial data with those of their
natural counterparts. However, the channels that are negatively-relevant (NR)
to predictions are still over-activated when processing adversarial data.
Besides, we also observe that AT does not result in similar robustness for all
classes. For the robust classes, channels with larger activation magnitudes
are usually more positively-relevant (PR) to predictions, but this alignment
does not hold for the non-robust classes. Given these observations, we
hypothesize that suppressing NR channels and aligning PR ones with their
relevances further enhances the robustness of CNNs under AT. To examine this
hypothesis, we introduce a novel mechanism, i.e., Channel-wise Importance-
based Feature Selection (CIFS). The CIFS manipulates channels’ activations of
certain layers by generating non-negative multipliers to these channels based
on their relevances to predictions. Extensive experiments on benchmark
datasets including CIFAR10 and SVHN clearly verify the hypothesis and CIFS’s
effectiveness of robustifying CNNs.

###  4.1 Introduction

Convolutional neural networks (CNNs) have achieved tremendous successes in
real-world applications, such as autonomous vehicles [grigorescu2020survey;
hu2020panda] and computer-aided medical diagnoses [trebeschi2017deep;
shu2020enhancing]. However, CNNs have be shown vulnerable to well-crafted (and
even minute) adversarial perturbations to inputs [szegedy_intriguing_2014;
goodfellow_explaining_2015; ilyas_adversarial_2019]. This has become hazardous
in high-stakes applications such as medical diagnoses and autonomous vehicles.

Recently, many empirical defense methods have been proposed to secure CNNs
against these adversarial perturbations, such as adversarial training (AT)
[madry_towards_2018], input/feature denoising [xie_feature_2019; du_rain_2020]
and defensive distillation [papernot_distillation_2016]. AT
[madry_towards_2018], which generates adversarial data on the fly for training
CNNs, has emerged as one of the most successful methods. AT effectively
robustifies CNNs but leads to a clear drop in the accuracies for natural data
[tsipras_robustness_2018] and suffers from the problem of overfitting to
adversarial data used for training [rice_overfitting_2020; zhang_geometry-
aware_2020; chen2021robust]. To ameliorate these problems, researchers have
proposed variants of AT, including TRADES [zhang_theoretically_2019] and
Friendly-Adversarial-Training (FAT) [zhang_attacks_2020]. To further robustify
CNNs under AT, many works attempt to propose novel defense mechanism to
mitigate the effects of adversarial data on features [xie_feature_2019;
du_rain_2020; xu_interpreting_2019]. For example, xie_feature_2019 found that
adversarial data result in abnormal activations in the feature maps and
performed feature denoising to remove the adversarial effects. Most of these
works improved robustness by identifying and suppressing abnormalities at
certain positions across channels (commonly referred to as feature maps in
CNNs), whereas the other direction, namely, the connection between robustness
and irregular activations of certain entire channels, has received scant
attention.

Since channels of CNNs’ deeper layers are capable of extracting semantic
characteristic features [zeiler_visualizing_2014], the process of making
predictions usually relies heavily on aggregating information from various
channels [bach2015pixel]. As such, anomalous activations of certain channels
may result in incorrect predictions. Thus, it is imperative to explore which
channels are entirely irregularly activated by adversarial data and which
channels’ activations benefit or degrade robustness. By utilizing this
connection, we will be able to further enhance the robustness of CNNs via
suppressing or promoting certain vulnerable or reliable channels respectively.

In this work, we attempt to build such a connection by comparing the channel-
wise activations of non-robust (normally trained) and robustified
(adversarially trained) CNNs. The channel-wise activations are defined as the
average activation magnitudes of all features within channels
[bai_improving_2021]. To identify what types of channels appear to be abnormal
under attacks, we regard channels’ relevances to prediction results (formally
defined in Equation (4.1) as ${\bm{g}}^{l}$) as the gradients of the
corresponding logits w.r.t channel-wise activations. The channels, whose
relevances to prediction results are positive or negative
(${\bm{g}}^{l}_{[i]}>0$ or ${\bm{g}}^{l}_{[i]}\leq 0$), are called positively-
relevant (PR) or negatively-relevant (NR) channels.

On the one hand, we observe that, AT robustifies CNNs by aligning adversarial
data’s channel-wise activations with those of natural data. However, we find
that the NR channels of adversarially trained CNNs are still over-activated by
adversarial data (see Figure 4.1(c)). Thus, we wonder: If we suppress NR
channels during AT to facilitate the alignment of channels’ activations, will
it benefit CNNs’ robustness? On the other hand, we find that adversarially
trained classification models do not enjoy similar robustness across all the
classes (see Figure 4.1(c) and 4.1(d)). For classes with relatively good
robustness, channels’ activations usually align well with their relevances,
i.e., channels with larger activations are more PR to labels. Given this
phenomenon, a natural question arises: If we align channels’ activations with
their relevances during AT, will it improve the robustness of CNNs? Regarding
these two questions, we propose a unified hypothesis on robustness
enhancement, denoted as $\mathcal{H}$: Suppressing NR channels and aligning
channels’ activations with their relevances to prediction results benefit the
robustness of CNNs.

To examine this hypothesis, we propose a novel mechanism, called Channel-wise
Importance-based Feature Selection (CIFS), which adjusts channels’ activations
with an importance mask generated from channels’ relevances. For a certain
layer, CIFS first takes as input the representation of a data point at this
layer and makes a raw prediction for the data point by a probe network. The
probe serves as the surrogate for the subsequent classifier (the composition
of subsequent layers) in the backbone and is jointly trained with the backbone
under supervision of true labels. Then, CIFS computes the gradients of the sum
of the top-$k$ logits w.r.t. the channels’ activations. We can obtain the
relevance of each channel to the top-$k$ prediction results by accumulating
the gradients within the channel. Finally, CIFS generates a mask of importance
scores for each channel by mapping channels’ relevances monotonically to non-
negative values. Through extensive experiments, we answer the two questions in
the affirmative and confirm hypothesis $\mathcal{H}$. Indeed, our results show
that CIFS clearly enhances the adversarial robustness of CNNs.

We comprehensively evaluate the robustness of CIFS-modified CNNs on benchmark
datasets against various attacks. On the CIFAR10 dataset, CIFS improves the
robustness of the ResNet-18 by $4$ percentage points against the PGD-100
attack. We also observe that CIFS ameliorates the overfitting during AT. In
particular, the robustness at the last epoch is close to that at the best
epoch. Finally, we conduct an ablation study to further understand how various
elements of CIFS affect the robustness enhancement, such as the top-$k$
feedback and architectures of the probe network.

(a) Normal / “automobile”

(b) Normal / “cat”

(c) Adv. / “automobile”

(d) Adv. / “cat”

Fig. 4.1: The magnitudes of channel-wise activations (top) at the penultimate
layer, their activated frequencies (bottom), and the weights of the last
linear layer (middle) vs. channel indices. #/$*$ means the CNN is trained in
the # way and we plot the average activations of data from the class $*$. The
weights corresponding to class $*$ are sorted in the descending order and can
indicate channels’ relevances to class $*$. The activation magnitudes and the
activated frequencies of natural and adversarial (PGD-20) data are plotted
according to the indices of the sorted weights. The robust accuracies on the
whole CIFAR10 are 0% for the normally trained ResNet-18 model and 46.6% for
the adversarially trained one (69.0% for the “automobile” class and 16.7% for
the “cat” class).

###  4.2 Related Works

This section briefly reviews relevant adversarial defense methods from two
perspectives: adversarial training (AT)-based defense and robust network
architecture design.

###### AT-based Defense

Adversarial training (AT) defends against adversarial attacks by utilizing
adversarially generated data in model training [goodfellow_explaining_2015],
formulated as a minimax optimization problem. Recently variants of AT
[cai_curriculum_2018; Wang_Xingjun_MA_FOSC_DAT; wang_improving_2020;
wu2020adversarial; zhang_geometry-aware_2020] have been proposed. For example,
the Misclassification-Aware-AdveRsarial-Training [wang_improving_2020]
modifies the process of generating adversarial data by simultaneously applying
the misclassified natural data, together with the adversarial data for model
training. Recent works have shown AT robustifies CNNs but degrades the natural
accuracy [tsipras_robustness_2018; zhang_theoretically_2019;
lamb_interpolated_2019]. To achieve a better trade-off,
zhang_theoretically_2019 decomposed the adversarial prediction error into the
natural error and boundary error and proposed TRADES to control both terms at
the same time. Besides, inspired by curriculum learning [cai_curriculum_2018;
bengio_curriculum_2009], zhang_attacks_2020 proposed FAT to train models with
increasingly adversarial data, which enhances generalization without
sacrificing robustness.

###### Robust Network Design

Other than robust training strategies, some works explored robust network
architectures [yan_robustness_2020; hsieh_robustness_2019]. For instance, the
work by yan_robustness_2020 showed neural ODE-based models are inherently more
robust than conventional CNN models; guo_sparse_2018 demonstrated that
appropriately designed higher model sparsity implies better robustness of
nonlinear networks. Another line of works defended against adversarial attacks
via gradient obfuscation, such as random or non-differentiable image/feature
transformations [xie_mitigating_2018; du_rain_2020; dhillon_stochastic_2018;
xiao_enhancing_2019]. However, they have been shown to be insecure to adaptive
attacks [athalye_obfuscated_2018; tramer_adaptive_2020]. Recently, many
researchers have attempted to develop novel mechanisms for robustness
enhancement. xie_feature_2019 performed feature denoising to remove the
adversarial effects on feature maps. zoran_towards_2020 utilized the spatial
attention mechanism to identify highlight important regions of feature maps.
Most of these works manipulated CNNs’ intermediate representations in the
spatial domain, whereas our work studies the adversarial robustness from the
channel-wise activation perspective.

Channel-wise Activation Suppressing (CAS) [bai_improving_2021], the most
relevant work to ours, also studied the channel-wise activations of
adversarial data. It showed channels are activated more uniformly by
adversarial data compared to the natural ones, and AT improves the robustness
by attempting to align the distributions of channels’ activations of natural
and adversarial data. However, there are still some channels that are over-
activated by adversarial data. To suppress these channels, the authors
proposed CAS to adjust channels’ activations based on their importance.
Although CAS empirically suppresses certain channels, the authors did not show
that the suppressed channels correspond to the target ones; this means the
primary objective of CAS may not have been met. Thus, there is no guarantee
CAS can enhance the robustness of CNNs (see Section 4.4.1 for further evidence
on this). In contrast, our work first builds a connection between robustness
and channels’ activations via their relevances to predictions. Then, the
proposed CIFS can explicitly control channels’ activations based on their
relevances. Finally, experiments demonstrate the effectiveness of CIFS on
robustness enhancement.

###  4.3 Channel-wise Importance-based Feature Selection

In this section, we first study the adversarial robustness by comparing
channels’ activations of non-robust (normally trained) and robustified
(adversarially trained) CNNs. Based on our observations of AT’s effects, we
propose a hypothesis on robustness enhancement via the adjustment of channels’
activations (Section 4.3.1). To examine this hypothesis, we then develop a
novel mechanism, CIFS (Section 4.3.2), to manipulate channels’ activation
levels according to their relevances to predictions . Finally, we verify the
proposed hypothesis through extensive experiments (Section 4.3.3).

####  4.3.1 Non-robust CNNs vs. Robustified CNNs: A Channel-wise Activation
Perspective

We compare a non-robust ResNet-18 [he_deep_2016] model with an AT-robustified
one on the CIFAR10 dataset [krizhevsky2009learning]. In ResNet-18, the
representations of penultimate layer are spatially averaged for each channel,
then fed into the last linear layer for making predictions. Thus, the weights
of the last linear layer indicate channels’ relevances to predictions
(according to the definition of channels’ relevances in Introduction). We
visualize the channel-wise activation magnitudes, the activated frequencies
(counted via a threshold of 1% of the largest magnitude among all channels) in
the penultimate layer for both natural and adversarial data, as well as the
weights of the last linear layer in Figure 4.1. The details of implementation
are provided in Appendix 4.6.1.

From Figures 4.1(a) and 4.1(b), we observe, for a non-robust ResNet-18, the
activation distribution of the adversarial data is obviously mismatched with
that of the natural data: natural data activate channels that are PR to
predictions with high values and high frequency, while adversarial data tend
to amplify the NR ones. From Figures 4.1(c) and 4.1(d), we observe that AT
robustifies the model by aligning the activation distribution of adversarial
data with that of natural data. Specifically, when dealing with adversarial
data, AT boosts the activation magnitudes of PR channels while suppressing the
activations of NR ones. However, we observe that, for many NR channels (e.g.,
around 150 channels from $350^{\text{th}}$ to $512^{\text{th}}$), the
activations of adversarial data are much higher than those of natural data.
These over-activations decrease the prediction scores corresponding to their
true categories. Given this observation, we wonder (Q1): if we suppress these
NR channels to regularize the freedom of adversarial perturbations, will it
further improve the model’s robustness upon AT?

Besides, an adversarially trained model does not enjoy similar robustness for
all classes, i.e., the robust accuracy of a certain class may be much higher
than another (e.g., “automobile” with 69.0% vs. “cat” with 16.7% against
PGD-20). Comparing the activations of these two classes (Figures 4.1(c) and
4.1(d)), we observe that, for the class with strong robustness (e.g.,
“automobile”), channels’ activations align better with their relevances to
labels, i.e., the channel with a greater extent of activation usually
corresponds to a larger weight in the linear layer. In contrast, this
alignment does not hold for the class with relatively poor robustness (e.g.,
for class “cat”, the most activated channels, lying between the
$26^{\text{th}}$ and $125^{\text{th}}$, are sub-PR to predictions). Given this
phenomenon, we may ask another question (Q2): If we scale channels’
activations based on their relevances to predictions, will it improve the
model’s robustness?

Considering the two questions above, we propose the unified hypothesis
$\mathcal{H}$, as stated in the Introduction.

####  4.3.2 Importance-based Channel Adjustment

To examine the hypothesis $\mathcal{H}$, one needs a systematic approach to
manipulate the channels, viz. selecting channels via suppressing NR ones but
promoting PR ones. To this end, we introduce a mechanism, dubbed as Channel-
wise Importance-based Feature Selection (CIFS). CIFS modifies layers of CNNs
by adjusting channels’ activations with importance scores that are generated
from the channels’ relevances to predictions.

For clearly state CIFS, we first introduce some notations: For a $K$-category
classification problem, let $({\mathbf{x}},{\mathbf{y}})\sim
P_{{\mathbf{x}}{\mathbf{y}}}$ denote the pair of the random input and its
label, where ${\mathbf{x}}\in{\mathbb{X}}\subset{\mathbb{R}}^{n_{X}}$ and
${\mathbf{y}}\in{\mathbb{Y}}=\\{0,1,\dots,K-1\\}$. We design an $L$-layer CNN-
based classification model to make accurate predictions for data sampled from
$P_{{\mathbf{x}}{\mathbf{y}}}$. The $l^{\text{th}}$ layer is denoted by
$f^{l}(\cdot)$ and parametrized by $\theta^{l}\in\Theta^{l}$; the mapping from
the input to the $l^{\text{th}}$ layer’s output is denoted by
$f^{[l]}=f^{l}\circ f^{l-1}\circ\cdots\circ f^{1}$ and the combination of all
the first $l$ layers’ parameters is denoted by $\theta^{[l]}$, i.e.,
$\theta^{[l]}=(\theta^{1},\dots,\theta^{l})$. Let us examine the
$l^{\text{th}}$ layer where an input ${\bm{x}}$ is transformed into a high-
dimensional representation
${\bm{z}}^{l}=f^{[l]}({\bm{x}})\in{\mathbb{R}}^{n^{l}_{\text{C}}\times
n^{l}_{\text{F}}}$; ${\bm{z}}^{l}$ has $n^{l}_{\text{C}}$ channels and each
channel is a feature vector of length $n^{l}_{\text{F}}$. With these
notations, we elaborate the details of CIFS in three steps (as shown in Figure
4.2).

Fig. 4.2: CIFS: 1) Probe Network $A^{l}$ first makes a raw prediction
${\bm{p}}^{l}$ for ${\bm{z}}^{l}$. 2) Channels’ relevances ${\bm{g}}^{l}$ are
assessed (Relev. Ass.) based on the gradients of the top-$k$ prediction
results $y^{l,k}$. 3) The IMGF generates an importance mask ${\bm{m}}^{l}$
from ${\bm{g}}^{l}$ for channel adjustment.

1) Surrogate Raw Prediction: To assess channels’ relevances to predictions, a
naive strategy is to compute the gradients of the final prediction with
respect to ${\bm{z}}^{l}$, i.e.,
$\nabla_{{\bm{z}}^{l}}f^{[l+1:L]}({\bm{z}}^{l})$, where
$f^{[l+1:L]}=f^{L}\circ\cdots\circ f^{l+1}$. Since we need to adjust
${\bm{z}}^{l}$ with importance scores generated from
$\nabla_{{\bm{z}}^{l}}f^{[l+1:L]}({\bm{z}}^{l})$ and send the adjusted feature
$\bar{\bm{z}}^{l}$ to $f^{[l+1:L]}$ again for making the final prediction, it
will result in computing the second-order derivatives during the training
phase. Moreover, in practice, we may apply CIFS into multiple layers, the
forward pass will involve at least the second-order gradients (the latter
CIFS-modified layer is recursively called). Thus, the back-propagation has to
deal with at least the third-order gradients during training. This will
aggravate the problem of training instability.

Instead, inspired by the design of auxiliary classifiers in CAS
[bai_improving_2021], CIFS builds a probe network $A^{l}$ as the surrogate of
$f^{[l+1:L]}$ for a making raw prediction ${\bm{p}}^{l}=A^{l}({\bm{z}}^{l})$,
so that we can use the gradients of ${\bm{p}}^{l}$ to approximately assess the
channels’ relevances to the final prediction. The assessment does not involve
other CIFS-modified layers. Thus, we can avoid the problem of back-propagation
through high-order derivatives. The probe network $A^{l}$ is parameterized by
$\theta^{l}_{A}$ and ${\bm{p}}^{l}\in{\mathbb{R}}^{K}$ represents the vector
of prediction scores/logits. We can jointly optimize $A^{l}$ with the backbone
network during the training phase under the supervision of true labels.

2) Relevance Assessment: With the prediction ${\bm{p}}^{l}$, we can compute
the gradients of logits in ${\bm{p}}^{l}$ w.r.t. ${\bm{z}}^{l}$ to assess the
feature’s relevances to each class. We consider the top-$k$ prediction results
($k\geq 2$) for the assessment of channels’ relevances. As data from two
semantically similar classes (e.g., “dog” and “cat”) usually share common
features, the prediction for an input often assigns large scores to the
classes similar to the true one and the top-$k$ results may include several of
these similar classes [jia_certified_2020]. In case the top-$1$ prediction is
wrong, considering the top-$k$ results may help us reliably extract some
common relevant features (see Section 4.4.2 for more evidence).

Let $y^{l,k}$ denote indices of the $k$ largest logits of prediction
${\bm{p}}^{l}$. Let $\bm{\delta}\in{\mathbb{R}}^{n^{l}_{\text{C}}}$ be the
channel-wise perturbation added to ${\bm{z}}^{l}$, giving the perturbed
representation
${\bm{z}}^{l}_{\bm{\delta}}={\bm{z}}^{l}+\bm{\delta}\cdot\mathbf{1^{\top}}$.
Here $\mathbf{1}\in{\mathbb{R}}^{n^{l}_{\text{F}}}$ is the column vector with
all elements as one, i.e., the features in the same channel are perturbed by a
common value. We calculate the gradients of the sum of the top-$k$ logits
w.r.t. the channel-wise perturbation $\bm{\delta}$:

$\small{\bm{g}}^{l}=\left.\nabla_{\bm{\delta}}\sum_{i\in
y^{l,k}}{\bm{p}}^{l}(\bm{\delta})_{[i]}\right|_{\bm{\delta}=\mathbf{0}}=\left.\nabla_{{\bm{z}}^{l}_{\bm{\delta}}}\sum_{i\in
y^{l,k}}A^{l}({\bm{z}}^{l}_{\bm{\delta}})_{[i]}\right|_{z^{l}_{\bm{\delta}}=z^{l}}\cdot\mathbf{1},$
(4.1)

where
${\bm{g}}^{l}=({\bm{g}}^{l}_{[0]},...,{\bm{g}}^{l}_{[n^{l}_{\text{C}}-1]})$
represents the vector of channels’ relevances to the top-$k$ logits. During
the training phase, since the true label of ${\bm{z}}^{l}$ is given, we
replace the top-$1$ prediction in $y^{l,k}$ with the true label $y$ and keep
other prediction results untouched.

3) Importance Mask Generation: As we want to suppress or promote channels
based on their relevances, we need to design proper Importance Mask Generating
Functions (IMGFs), which monotonically map relevances to non-negative
importance scores; of particular importance is to map negative relevances to
values close to zero.

Here, we provide several feasible options: To answer the first question on
whether suppressing NR channels enhances robustness, one can use the sigmoid
function as the IMGF. With a large value of $\alpha$, the sigmoid function
serves as a switch by mapping negative relevances to importance scores close
to zero and the positive close to one. To answer the second question
concerning aligning channel activations with their relevances, we can use the
softplus or softmax function as the IMGF. Both of them can map negative
relevacnes to values close zero and map positive relevances monotonically to
positive values. These three functions are stated here for ease of reference:

  * •

sigmoid:
${\bm{m}}^{l}_{[i]}=\frac{1}{1+\exp(-\alpha\cdot{\bm{g}}^{l}_{[i]})},\quad\alpha>0$.

  * •

softplus:
${\bm{m}}^{l}_{[i]}=\frac{1}{\alpha}\cdot\log(1+\exp(\alpha\cdot{\bm{g}}^{l}_{[i]})),\quad\alpha>0$.

  * •

softmax:
${\bm{m}}^{l}_{[i]}=\frac{\exp({\bm{g}}^{l}_{[i]}/T)}{\sum_{j}\exp({\bm{g}}^{l}_{[j]}/T)},\quad
T>0$.

The usage of these functions will be discussed in detail in Section 4.3.3.
CIFS selects channels by multiplying the importance mask ${\bm{m}}^{l}$ with
${\bm{z}}^{l}$ as follows:

$\small\bar{\bm{z}}^{l}={\bm{z}}^{l}\otimes\text{repmat}({\bm{m}}^{l},1,n^{l}_{L}),$
(4.2)

where the “repmat” operation replicates the column vector ${\bm{m}}^{l}$ along
the second axis $n^{l}_{L}$ times.

###### Training of CIFS

In practice, we may apply the CIFS mechanism into several layers of a CNN. Let
$I$ denote the set of indices of these layers, and $\theta_{A}^{I}$ denote the
parameters of all the probes in the CIFS-modified layers. For each input
${\bm{x}}$, the modified model $\bar{f}^{[L]}$ outputs $|I|$ raw predictions
and one final prediction ${\bm{p}}=\bar{f}^{[L]}({\bm{x}})$. Given this, we
use an adaptive loss function [bai_improving_2021] for training the model:

$\small\ell_{\beta}({\bm{x}},{\bm{y}})=\frac{1}{1+\beta}\cdot\ell_{\mathrm{ce}}({\bm{p}},{\bm{y}})+\frac{\beta}{(1+\beta)|I|}\cdot{\sum\limits_{l\in
I}\ell_{\mathrm{ce}}({\bm{p}}^{l},{\bm{y}})},$ (4.3)

where $\ell_{\text{ce}}(\cdot,\cdot)$ denotes the cross-entropy loss and the
coefficient $\beta>0$ balances the accuracy of raw predictions by CIFS and the
final prediction. Since the subsequent decisions closely depend on the
channels of features selected by the previous CIFS-modified layers, we should
choose a proper value of $\beta$ to make sure that the raw predictions made by
CIFS are reliable. In practice, we set $\beta$ to be $|I|$, and the effect of
$\beta$ is discussed in the ablation study (see Appendix 4.6.5). To robustify
the CNN model against malicious attacks, we can train $\bar{f}^{[L]}$ in an
adversarial manner with a perturbation budget $\epsilon$. Namely, we solve the
following optimization problem:

$\small\min_{\theta^{[L]},\theta^{I}_{A}}\mathbb{E}_{({\mathbf{x}},{\mathbf{y}})\sim
P_{{\mathbf{x}}{\mathbf{y}}}}\left[\max_{{\mathbf{x}}^{\prime}\in\mathcal{B}({\mathbf{x}},\epsilon,l_{\infty})}\ell_{\beta}({\mathbf{x}}^{\prime},{\mathbf{y}})\right],$
(4.4)

where
$\mathcal{B}({\bm{x}},\epsilon,l_{\infty})=\\{{\bm{x}}^{\prime}~{}|~{}\|{\bm{x}}^{\prime}-{\bm{x}}\|_{l_{\infty}}\leq\epsilon\\}$.

(a) non-CIFS

(b) CIFS-sigmoid

(c) CIFS-softplus

(d) CIFS-softmax

Fig. 4.3: The magnitudes of channel-wise activations (top) at the penultimate
layer, their activated frequency (bottom), and the weights of the last linear
layer (middle) vs. channel indices. The activations of natural data and their
PGD-20 examples are averaged over all test samples in the “airplane” category.
The robust accuracies against PGD-20 (on the whole dataset) are 46.64% for
non-CIFS, 49.87% for the CIFS-sigmoid, 50.38% for the CIFS-softplus, and
51.23% for the CIFS-softmax respectively

####  4.3.3 Verification of Hypothesis $\mathcal{H}$ on Robustness Enhancement

We verify the hypothesis $\mathcal{H}$ by answering the two questions, Q1 and
Q2, in Section 4.3.1 respectively.

To answer Q1, we applied the sigmoid function to generate the mask
${\bm{m}}^{l}$ from ${\bm{g}}^{l}$. Setting $\alpha$ to be large enough (here,
$\alpha=10$), we can generate importance scores close to zeros for NR channels
and scores close to one for the PR ones, so that we approximately annihilate
the NR channels but leave the PR ones as unchanged. We adversarially trained a
ResNet-18 model and its CIFS-modified version. As shown in Figure 4.3(a), the
NR channels (Channel 300-512) of the vanilla ResNet-18 model are clearly
activated (the average activation magnitudes of these channels are larger than
0.1; the activation frequencies are over 0.4). In contrast, the ResNet-18 with
CIFS-sigmoid effectively suppresses the activation of NR channels (Channel
$100^{\text{th}}$–$512^{\text{th}}$ in Figure 4.3(b)). Most of their mean
activation magnitudes are smaller than 0.05, and their activation frequencies
have clearly decreased. The experimental results show that, under AT, the
vanilla ResNet-18 model results in a 46.64% defense rate against the PGD-20
attack while its CIFS-sigmoid modified version achieves 49.87%. More results
can be found in Appendix 4.6.1. Thus, we conclude that suppressing NR channels
enhances the robustness of CNNs.

To answer Q2, we applied the softplus and softmax functions as IMGFs to
generate the mask ${\bm{m}}^{l}$ from ${\bm{g}}^{l}$ respectively. Here the
coefficient $\alpha$ in the softplus is set to be $5$ and the temperature $T$
in the softmax is set to be $1$. From Figures 4.3(c) and 4.3(d), we observe
that, by generating importance scores positively correlated with the
relevances, the model tends to completely focus on few relevant (positive and
negative) channels. The channel of the greatest weight (most PR to
predictions) is activated with the highest magnitude. Most channels become
irrelevant (small absolute values of weights) to the predictions and are
activated at a low level. Using softplus as the IMGF (Figure 4.3(c)), the
irrelevant channels are sparsely activated, and the activation magnitudes are
smaller than 5% of the most important channel. In Figure 4.3(d), this
phenomenon is enhanced by using softmax as IMGF: most channels become
irrelevant to predictions and are usually deactivated. We evaluated the
robustness of these two CIFS-modified CNNs against PGD-20 attack. Both of them
outperformed the CIFS-sigmoid and the vanilla ResNet-18 classifiers (robust
accuracies of CIFS-softplus and CIFS-softmax are 50.38% and 51.23%
respectively, vs., 49.87% for CIFS-sigmoid and 46.64% for the vanilla
ResNet-18). We also found CIFS can ameliorate the class-wise imbalance of
adversarial robustness (e.g., CIFS-softmax increases the PGD-20 accuracy from
16.7% to 22.3% for class “cat”). More details are provided in Appendix 4.6.1.
Thus, we conclude that aligning channels activations with their relevances to
predictions can further robustify CNNs upon suppressing NR ones.

Given these empirical results, we verified the hypothesis $\mathcal{H}$ and
justified that CIFS is an effective mechanism to improve the adversarial
robustness of CNNs. In the following section, we conduct extensive experiments
to evaluate the robustness enhancement through CIFS and study CIFS in an
ablation manner.

###  4.4 Experiments

####  4.4.1 Robustness Evaluation

We utilize the CIFS to modify CNNs in different architectures to perform
classification tasks on benchmark datasets, namely a ResNet-18 and a
WideResNet-28-10 on the CIFAR10 [krizhevsky2009learning] dataset, a ResNet-18
on the SVHN [netzer2011reading] dataset and a ResNet-10 on the Fashion-MNIST
[xiao2017_online] dataset. We train the models with the standard PGD
adversarial training (AT) [madry_towards_2018] and its variants, such as FAT
[zhang_attacks_2020], to show that CIFS can work under various AT-strategies.
We compare CIFS-modified CNNs with the vanilla versions as well as the CAS-
modifications, where CAS [bai_improving_2021] also modifies CNNs by adjusting
channels’ activations. Here, we report the results on CIFAR10 and SVHN.
Results on FMNIST are presented in Appendix 4.6.4.

###### Adaptive Attacks

As mentioned in Section 4.3.2, each CIFS-modified layer of CNNs outputs a raw
prediction. To generate adversarial examples that are as strong as possible,
we follow the strategy used in CAS and attack CNNs via the adaptive loss
function $\ell_{\beta}$ in Equation (4.3) that considers all the raw and final
predictions. We let the value of $\beta$ be chosen by the attacker, i.e., the
attacker can try various values of $\beta_{\text{atk}}$ and select one that
results in the most harmful perturbations. Our setting is more challenging for
defense than CAS where the same value of $\beta$ is used for training and
attack. Here, for each adversarial attack, we evaluate the robustness by
choosing $\beta_{\text{atk}}$ from $\\{0,0.1,1,2,10,100,\infty\\}$ and report
the worst robust accuracy111Results of various values of $\beta$ are present
in Appendix 4.6.2. We observe that CAS can improve the robustness of CNNs in
most cases but fail when attackers completely focus on CAS modules.. Setting
$\beta_{\text{atk}}=\infty$ means the attacks completely focus on the CIFS-
modified layers 222For $\beta_{\text{atk}}=\infty$, we consider the cases of
attacking both CIFS-modified layers simultaneously and attacking each
separately. and only consider the second term in Equation (4.3).

##### Robustness Enhancement of CIFS under AT

We adversarially train ResNet-18 and WRN-28-10 models with PGD-10
($\epsilon=\nicefrac{{8}}{{255}}$) adversarial data. CIFS is applied to the
last two residual blocks of each model. The probes for the last and
penultimate blocks are a linear layer and a multi-layer perceptron (MLP)
respectively. Channels’ relevances are assessed based on top-$2$ results and
we use the softmax function with $T=1$ as the IMGF. Other training details are
provided in Appendix 4.6.2 and 4.6.3.

###### Defense Results

We evaluate the robustness of CNNs against four types of white-box attacks:
FGSM [szegedy_intriguing_2014], PGD-20 [madry_towards_2018], C&W
[carlini_towards_2017], and PGD-100. The $l_{\infty}$-norm of the
perturbations are bounded by the value of $\epsilon=\nicefrac{{8}}{{255}}$.
Here, we report the robustness evaluated at the last epoch for each model.
Detailed attack settings and more defense results ( AutoAttack333AutoAttack
consists of both white and black-box attacks. [croce_reliable_2020] and the
best epochs’ results), are present in Appendix 4.6.2 and 4.6.3.

The defense results on CIFAR10 are reported in Table 4.1. We observe that, for
both of the ResNet-18 and WRN-28-10 architectures, CIFS consistently
outperforms the counterparts against various types of adversarial attacks. For
example, the CIFS-modified WRN-28-10 can defend the PGD100 attack with a
success rate of $48.74\%$, which exceeds the second best by more than 4
percentage points. In contrast, under the strong adaptive attack, we see that
the baseline CAS cannot improve and even worsens the robustness of CNNs. The
defense results on SVHN are reported in Table 4.2. The results also verify the
effectiveness of CIFS on improving robustness.

Table 4.1: Robustness comparison of defense methods on CIFAR10. We report the
accuracies (%) for adversarial and natural data. For each model, the results
of the strongest attack are marked with an underline.

ResNet-18 | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---
Vanilla | 84.56 | 55.11 | 46.62 | 45.95 | 44.72
CAS | 86.73 | 55.99 | 45.29 | 44.18 | 43.22
CIFS | 83.86 | 58.86 | 51.23 | 50.16 | 48.70
WRN-28-10 | Natural | FGSM | PGD-20 | C&W | PGD-100
Vanilla | 87.29 | 58.50 | 49.17 | 48.68 | 47.08
CAS | 88.05 | 57.94 | 49.03 | 47.97 | 47.25
CIFS | 85.56 | 61.34 | 53.74 | 53.20 | 51.51

Table 4.2: Robustness comparison of defense methods on SVHN. The accuracies
(%) for natural and adversarial data are reported.

ResNet-18 | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---
Vanilla | 93.72 | 65.87 | 50.35 | 47.89 | 45.81
CAS | 94.08 | 65.24 | 48.47 | 46.15 | 43.75
CIFS | 93.94 | 66.24 | 52.02 | 50.13 | 47.49

###### Training Procedure

We train CNN classifiers in an adversarial manner for $120$ epochs and adjust
the learning rate with a multiplier $0.1$ at epoch $75$ and epoch $90$. We
summarize the training procedure by plotting the curves of training losses and
the PGD-20 accuracies w.r.t. epochs in Figure 4.4. We observe the best
adversarial robustness of the vanilla ResNet-18 ($50.64\%$ defense rate)
appears around the $75^{\text{th}}$ epoch. After epoch $75$, the model starts
to overfit to training data, i.e., the training loss continues decreasing, but
the robust accuracy drops as well. In contrast, the overfitting problem is
ameliorated by the application of CIFS. We can see that the best robust
accuracy appears around the $90^{\text{th}}$ epoch; After the best epoch, the
training loss continues decreasing, but the robustness is maintained around
the peak. This phenomenon may result from the fact that CIFS suppresses
redundant channels. The model redundancy can be controlled by selecting few
highly relevant channels and deactivating others. In this way, the overfitting
in training is ameliorated.

Fig. 4.4: Comparison on the training of a vanilla ResNet-18 and its CIFS-
modified version. Training losses and accuracies against PGD-20 attack are
plotted.

On the computation overhead, we report the training time and the evaluation
time of a ResNet-18 classifier on the CIFAR10 dataset for reference. For
PGD-10 adversarial training, the vanilla CNN takes 166s for each epoch while
the CIFS-modified model takes 172s. For the PGD-20 evaluation, the vanilla CNN
takes 53s for the CIFAR10 test set; the CIFS-modified model takes 56s instead.
In short, the proposed CIFS does not result in too much extra computation.

##### CIFS Working in Conjunction with Variants of AT

CIFS improves the adversarial robustness by adjusting channels’ activations,
which is orthogonal to defense training strategies. Here, we train the CIFS-
modified CNNs with variants of AT to examine whether CIFS can work in
conjunction with other state-of-the-art training-based defense techniques. We
consider the FAT [zhang_attacks_2020] and TRADES [zhang_theoretically_2019]
strategies. We report the defense results of FAT in Table 4.3 and provide the
results of TRADES in Appendix 4.6.2. We observe that FAT training strategy
improves the natural accuracy and robustness of CNNs (compared to the results
in Table 4.1). Under the FAT strategy, CIFS also improves the adversarial
robustness of the vanilla CNNs in both ResNet-18 and WRN-28-10 architectures.

Table 4.3: Robustness comparison of defense methods trained with FAT on
CIFAR10. Comparing the accuracies (%) against the strongest attacks, we
observe that CIFS clearly robustifies CNNs.

ResNet-18 | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---
Vanilla | 87.16 | 56.43 | 47.64 | 46.01 | 45.35
CIFS | 86.35 | 59.47 | 51.68 | 51.84 | 49.52
WRN-28-10 | Natural | FGSM | PGD-20 | C&W | PGD-100
Vanilla | 88.37 | 58.81 | 49.62 | 48.49 | 47.58
CIFS | 86.74 | 60.67 | 51.99 | 52.34 | 49.87

####  4.4.2 Ablation Study

Here, we conduct an ablation study to further understand the robustness
properties of CIFS. Specifically, we investigate the effects of the feedback
from the top-$k$ predictions. The ablation experiments are conducted on
CIFAR10 based on the ResNet-18 model. Besides, in Appendix 4.6.5, we also
study cases in which the CIFS is applied to different layers, the probe
networks are in different architectures, various values of $\beta$ are used
for training.

###### Feedback from Top-$k$ Prediction Results

As is well-known, the top-$k$ classification accuracy for $k>1$ is always not
worse than the top-$1$. For example, in Table 4.4, we can see that the top-$2$
accuracy of an adversarially trained ResNet-18 against the PGD-20 attack
exceeds the top-$1$ accuracy by $25$ percentage points. This implies that,
although adversarial data can usually fool the classifier (i.e., low top-$1$
accuracy), the prediction confidence of the true class is still high and the
corresponding score highly likely lies among the top-$2$ or $3$ logits.

Table 4.4: Top-$k$ accuracies (%) against adversarial attacks on CIFAR10 of an
adversarially trained ResNet-18.

ResNet-18 | top-$1$ | top-$2$ | top-$3$
---|---|---|---
FGSM | 55.11 | 76.22 | 85.20
PGD-20 | 46.62 | 71.71 | 81.60

CIFS generates the importance mask from the raw prediction and uses it to
suppress or promote channels at the current layer. The final prediction made
by subsequent layers strongly depends on the channels selected by CIFS. To
ensure the accuracy of final predictions, the logits used for generating
importance scores should include the true label’s logit for each input so that
the truly important channels will be highlighted. According to Table 4.4, if
we use the top-$1$ logit to assess the importance of channels for PGD-20
adversarial data, the probability of incorrect assessment is over $50$%.
Instead, if we use the feedback from top-$2$ or $3$ logits, the truly
important channels can highly likely be promoted. The following table presents
more experiments that justify this argument.

Table 4.5: Robustness comparison (%) of importance assessment based on top-$k$
results against the PGD-20 attack. The column header $*/\\#$ represents the
attack point and the output prediction. For example, CIFS/Final means the
model outputs the final prediction and the attacker solely focuses on the
CIFS’s raw prediction.

ResNet-18 | Natural/Final | CIFS/CIFS | CFIS/Final | Adap/Final
---|---|---|---|---
Vanilla | 84.56 | - | - | 46.62
top-$1$ | 87.63 | 47.24 | 47.24 | 47.24
top-$2$ | 83.86 | 48.72 | 54.96 | 51.23
top-$3$ | 83.49 | 47.59 | 55.39 | 49.91

From Table 4.5, we observe that, for the top-$1$ case, the defense rate of
‘CIFS/CIFS’ is the same as that of ‘CIFS/Final’ and that of ‘Adap/Final.’ This
implies that, once an adversarial example successfully fools the raw
prediction of CIFS, the final prediction also will be incorrect. Thus, the
attacker only needs to focus on the CIFS’s raw predictions to break the model.
In contrast, for the top-$2$ case, the defense rate of ‘CIFS/Final’ exceeds
the ‘CIFS/CIFS’ by 6 percentage points. This means that nearly 6% adversarial
data mislead the CIFS’s raw predictions. However, through the channel
adjustment via CIFS, these adversarial data are “purified”, and more relevant
characteristic features are thus transmitted to subsequent layers of CNNs. As
such, these adversarial data are finally classified correctly. In this case,
the attacker has to exhaustively search for an adaptive loss function to
generate attacks, and the CIFS-modified CNNs are safer and more reliable. More
discussion on why the top-$k$ assessment performs better and how to choose $k$
is provided in Appendix 4.6.5.

###  4.5 Chapter Summary

We developed the CIFS mechanism to verify the hypothesis that suppressing NR
channels and aligning PR ones with their relevances to predictions benefits
adversarial robustness. Empirical results demonstrate the effectiveness of
CIFS on enhancing CNNs’ robustness.

There are two limitations of our current work: 1) We empirically verify the
hypothesis $\mathcal{H}$, but it is still difficult to explicitly, not
intuitively, explain why the adjustment of channels improves robustness. 2)
Although CIFS ameliorates the overfitting during AT and improves the
robustness, it sometimes leads to a bit drop in natural accuracies on certain
datasets. In the future, we will attempt to address these two limitations.

###  4.6 Appendices

####  4.6.1 Details on Visualizing Channel-wise Activations

##### Non-robust CNNs vs. Robustified CNNs

We train ResNet-18 models to perform the classification task on the CIFAR10
dataset. The models are trained normally and adversarially. We use adversarial
data generated by PGD-10 attack ($\epsilon=\nicefrac{{8}}{{255}}$, step size
$\nicefrac{{\epsilon}}{{4}}$, and random initialization) for adversarial
training.

The ResNet-18 network consists of one convolutional layer, eight residual
blocks, and one linear fully-connected (FC) layer connected successively. Each
residual block contains two convolutional layers for the residual mapping. We
visualize the features of the penultimate layer (the output of the eighth
residual block) and the weights of the last linear layer in Figure 1.
Specifically, the weights of the last FC layer for a certain class are sorted
and plotted in descending order. We process the penultimate layer’s features
with the global average pooling operation to obtain the channel-wise
activations. For a certain class, we calculate each channel’s mean activation
magnitude over all the test samples in this category. We normalize the mean
channel-wise activations by dividing them by their absolute maximum. The mean
channel-wise activations are plotted according to the indices of the sorted
weights. We also record the activated frequency of each channel. Here, the
channel is regarded to be activated if its activation magnitude is larger than
a threshold (1% of the maximum of all channels’ activations).

##### Channel-wise Activations of CIFS-modifed CNNs

We train CIFS-modified CNNs normally and adversarially by using the adaptive
loss in Equation (3). We use the PGD-10 attack to generate adversarial data.
We illustrate the channels’ activations of CIFS-modified CNNs in Figure 3. The
implementation details are same as those in Appendix 4.6.1.

In Figure 3, we show the channels’ activations of data in class “airplane”.
Here, we plot the channels activations of data in other classes. From Figure
4.7, we see that CIFS indeed suppresses negatively-relevant (NR) channels and
promotes the positively-relevant (PR) ones.

Besides, we also observe that CIFS ameliorates the class-wise imbalance of
robustness under AT. In Figure 4.8, we can see that, for the data in class
“cat” and class “deer”, the robust accuracies of the vanilla ResNet-18 model
are 16.70% and 25.50%. Modifying the vanilla model with CIFS-softmax, we can
improve the robust accuracies by 5.6 and 3.3 percentage points, respectively.

(a) “ship”: non-CIFS

(b) “ship”: CIFS-sigmoid

(c) “ship”: CIFS-softplus

(d) “ship”: CIFS-softmax

Fig. 4.5: The magnitudes of channel-wise activations (top) at the penultimate
layer, their activated frequency (bottom), and the weights of the last linear
layer (middle) vs. channel indices. Data from class “ship” (a-d),
“automobile”(e-h), and “frog”(i-l) , are used here. The robust accuracies
against PGD-20 (on the whole dataset) are 46.64% for non-CIFS, 49.87% for the
CIFS-sigmoid, 50.38% for the CIFS-softplus, and 51.23% for the CIFS-softmax
respectively.

(a) “autmobile”: non-CIFS

(b) “autmobile”: CIFS-sigmoid

(c) “autmobile”: CIFS-softplus

(d) “autmobile”, CIFS-softmax

Fig. 4.6: The magnitudes of channel-wise activations (top) at the penultimate
layer, their activated frequency (bottom), and the weights of the last linear
layer (middle) vs. channel indices. Data from class “ship” (a-d),
“automobile”(e-h), and “frog”(i-l) , are used here. The robust accuracies
against PGD-20 (on the whole dataset) are 46.64% for non-CIFS, 49.87% for the
CIFS-sigmoid, 50.38% for the CIFS-softplus, and 51.23% for the CIFS-softmax
respectively.

(a) “frog”: non-CIFS

(b) “frog”: CIFS-sigmoid

(c) “frog”: CIFS-softplus

(d) “frog”, CIFS-softmax

Fig. 4.7: The magnitudes of channel-wise activations (top) at the penultimate
layer, their activated frequency (bottom), and the weights of the last linear
layer (middle) vs. channel indices. Data from class “ship” (a-d),
“automobile”(e-h), and “frog”(i-l) , are used here. The robust accuracies
against PGD-20 (on the whole dataset) are 46.64% for non-CIFS, 49.87% for the
CIFS-sigmoid, 50.38% for the CIFS-softplus, and 51.23% for the CIFS-softmax
respectively. Fig. 4.8: Robust accuracies (%) of PGD-20 adversarial data for
various classes of the CIFAR10 dataset.

####  4.6.2 Robustness Evaluation on CIFAR10

##### Robustness Enhancement of CIFS under AT

Training and Evaluation details: On the CIFAR10 dataset, we train ResNet-18
and WRN-28-10 models with PGD-10 adversarial examples
($\epsilon=\nicefrac{{8}}{{255}}$, step size $\nicefrac{{\epsilon}}{{4}}$ with
random initialization). The $\beta$ in CIFS is set to be $2$. For the
ResNet-18 and its CIFS-modified version, we train models for $120$ epochs with
the SGD optimizer (momentum $0.9$ and weight decay $0.0002$). The learning
rate starts from $0.1$ and is multiplied with $0.1$ at epoch $75$ and epoch
$90$. For the WRN-28-10, we train model for $110$ epochs with weight decay
$0.0005$.

In Section 4.1, we evaluate the robustness of CNNs against four white-box
attacks with a perturbation budget $\epsilon=\nicefrac{{8}}{{255}}$ in
$l_{\infty}$ norm — FGSM, PGD-20 (step size $\nicefrac{{\epsilon}}{{10}}$),
C&W (optimized by PGD for 30 steps with a step size
$\nicefrac{{\epsilon}}{{10}}$) and PGD-100 (step size
$\nicefrac{{\epsilon}}{{10}}$).

Robustness Evaluation with AutoAttack: Here, we also report the robust
accuracies of defense methods against AutoAttack [croce_reliable_2020], which
consists of both white-box and black-box attacks. AutoAttack regards models to
be robust at a certain data point only if the models correctly classify all
types of adversarial examples generated by AutoAttack of that data point. We
consider the AutoAttack including one strong white-box attack (Auto-PGD
[croce_reliable_2020]) and one black-box attack (Square-Attack
[andriushchenko2020square]). Since the Square Attack requires many queries, we
sample 2,000 images (200 per class) from the CIFAR10 for evaluation. The
attack parameters are set according to the officially released
AutoAttack444https://github.com/fra31/auto-attack. From Table 4.6, we observe
that CIFS enjoys better robustness against AutoAttack in comparison to the
vanilla ResNet-18 model and its CAS-modified version.

Table 4.6: Robustness comparison of defense methods on CIFAR10. We report the
last-epoch robust accuracies (%) against AutoAttack.

Model | Vanilla | CAS | CIFS
---|---|---|---
ResNet-18 | 44.00 | 42.70 | 46.20
WRN-28-10 | 47.20 | 46.55 | 49.75

Best-epoch robustness during training: Due to the susceptibility of
overtrained models to overfitting [rice_overfitting_2020], it seems reasonable
to compare the results at the end of the training (and not for the best
epochs) [madry_towards_2018; zhang_attacks_2020; rice_overfitting_2020]. In
Section 4.1, we report the robust accuracies of ResNet-18 and WRN-28-10 models
at the last epochs. Here, we also provide the results at the best epochs for
reference.

From Table 4.7, we see that, for the ResNet-18 architecture, the CIFS-modified
model results in the similar best-epoch robustness (PGD-100) to that of the
vanilla ResNet-18. For the WRN-28-10, the vanilla model has the better best-
epoch robustness compared to the CIFS-modified version. This may be due to the
fact that CIFS suppresses redundant channels and reduces the model capacity.

By comparing results in Table 4.7 with those in Table 1, we observe that CIFS
indeed ameliorates the overfitting of AT. Specifically, the best-epoch robust
accuracy of the vanilla WRN-28-10 (resp. ResNet-18) against PGD-100 attack is
54.17% (resp.49.47%), but the last-epoch accuracy drops to 47.08% (resp.
44.72%). In contrast, for the CIFS-modified versions, the last-epoch robust
accuracies against PGD-100 attack are maintained around the best-epoch ones
(for WRN-28-10, from 52.03% to 51.51%; for ResNet-18, from 49.76% to 48.74%).

Table 4.7: Robustness comparison of defense methods on CIFAR10. We report the
best robust accuracies (%) during training. For each model, the results of the
strongest attacks are marked with an underline.

ResNet-18 | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---
Vanilla | 83.63 | 56.73 | 50.64 | 49.51 | 49.47
CAS | 85.66 | 56.25 | 47.69 | 46.52 | 45.69
CIFS | 82.46 | 58.98 | 51.94 | 51.25 | 49.76
WRN-28-10 | Natural | FGSM | PGD-20 | C&W | PGD-100
Vanilla | 86.53 | 61.43 | 55.69 | 54.45 | 54.17
CAS | 87.51 | 58.54 | 52.06 | 51.27 | 50.69
CIFS | 84.67 | 61.03 | 54.09 | 53.76 | 52.03

Robust accuracies for various values of $\beta_{\text{atk}}$: In Section 4.1,
we evaluate the robustness of CIFS-modified models by using the adaptive loss
in Equation (3). For each type of attack, we assign various values to
$\beta_{\text{atk}}$ and report the worst robust accuracies. Here, for
reference, we provide the defense results of the ResNet-18 model on CIFAR10
for different values of $\beta_{\text{atk}}$ that are used in Section 4.1. The
results in Table 1 (ResNet-18) are collected from Table 4.8.

Table 4.8: Robust accuracies (%) for values of $\beta_{\text{atk}}$ on
CIFAR10. The value “$\infty$” means the attack only considers the second term
in Equation (3). The value “$\infty$-1” (resp. “$\infty$-2”) means the
attacker completely focuses on the first (resp. second) CIFS-modifed layer.
The bracketed numbers are those reported in Table 1 (ResNet-18).

ResNet-18 | $\beta_{\text{atk}}$ | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---|---
Vanilla | - | [84.56] | [55.11] | [46.62] | [45.95] | [44.72]
CAS | 0 | [86.73] | 83.17 | 88.45 | 88.52 | 88.24
| 0.1 | - | 58.61 | 61.36 | 85.51 | 62.40
| 1 | - | 56.36 | 52.86 | 62.34 | 56.02
| 2 | - | 56.06 | 49.76 | 54.94 | 50.62
| 10 | - | 56.03 | 47.47 | 49.35 | 47.70
| 100 | - | 56.02 | 47.04 | 48.36 | 46.74
| $\infty$ | - | 56.02 | 47.06 | 48.31 | 46.55
|  $\infty$-1 | - | [55.99] | [45.29] | [44.18] | [43.22]
|  $\infty$-2 | - | 82.68 | 87.87 | 87.79 | 87.72
CIFS | 0 | [83.86] | 60.58 | 52.64 | 51.32 | 49.94
| 0.1 | - | [58.86] | 51.40 | 50.88 | 49.42
| 1 | - | 59.20 | 51.28 | [50.16] | 48.74
| 2 | - | 59.24 | [51.23] | 50.28 | 48.79
| 10 | - | 59.35 | 51.27 | 50.70 | [48.70]
| 100 | - | 59.38 | 51.41 | 51.04 | 48.80
| $\infty$ | - | 59.43 | 51.45 | 51.08 | 48.82
|  $\infty$-1 | - | 61.06 | 54.96 | 53.83 | 52.82
|  $\infty$-2 | - | 60.03 | 52.30 | 50.92 | 50.03

##### Robustness Enhancement under TRADES

To improve the robustness of CNNs, various training-based strategies have been
proposed, including vanilla adversarial training (AT) [madry_towards_2018],
friendly-adversarial training (FAT) [zhang_attacks_2020], and TRADES
[zhang_theoretically_2019]. In Section 4.1, we show that CIFS can further
enhance the robustness of CNNs under the vanilla AT and FAT. Here, we conduct
more experiments to check whether TRADES is also suitable for CIFS.

Table 4.9: Robustness comparison of vanilla CNNs and their CIFS-modified
version under various AT-based strategies. We report the robust accuracies (%)
on various types of adversarial data.

ResNet-18 | Natural | FGSM | PGD-20 | PGD-100
---|---|---|---|---
Vanilla-AT | 84.56 | 55.11 | 46.62 | 44.72
Vanilla-TRADES | 83.96 | 57.09 | 50.27 | 48.83
Vanilla-FAT | 87.16 | 56.43 | 47.64 | 45.35
CIFS-AT | 83.86 | 58.86 | 51.23 | 48.74
CIFS-TRADES | 85.20 | 54.76 | 46.13 | 43.65
CIFS-FAT | 86.35 | 59.47 | 51.68 | 49.52

From Table 4.9, we observe that, for the vanilla ResNet-18 model, TRADES
effectively robustifies the network and outperforms its counterparts by a
large margin (e.g., 48.83% of TRADES vs. 44.72% of AT against PGD-100 attack).
However, for the CIFS-modified models, TRADES performs worse than AT and FAT.
In general, CIFS-modification in combination with the FAT training strategy
achieves the best robustness against various attacks.

####  4.6.3 Robustness Evaluation on SVHN

Training and Evaluation details: On the SVHN dataset, we train the ResNet-18
model and its CIFS-modified version with PGD-10 adversarial examples
($\epsilon=\nicefrac{{8}}{{255}}$, step size $\nicefrac{{\epsilon}}{{4}}$ with
random initialization). We train models for $120$ epochs with the SGD
optimizer (momentum $0.9$ and weight decay $0.0005$). The learning rate starts
from $0.01$ and is multiplied with $0.1$ at epoch $75$ and epoch $90$.

In Section 4.1, we evaluate the robustness of CNNs against four white-box
attacks with a perturbation budget $\epsilon=\nicefrac{{8}}{{255}}$ in
$l_{\infty}$ norm — FGSM, PGD-20 (step size $\nicefrac{{\epsilon}}{{10}}$),
C&W (optimized by PGD for 30 steps with a step size
$\nicefrac{{\epsilon}}{{10}}$) and PGD-100 (step size
$\nicefrac{{\epsilon}}{{10}}$).

Robustness Evaluation with AutoAttack: Here, we also report the robust
accuracies of defense methods against AutoAttack on SVHN (Table 4.10). The
evaluation settings of AutoAttack follows those in Appendix 4.6.2.

Table 4.10: Robustness comparison of defense methods on SVHN. We report the
robust accuracies (%) at the last epochs.

ResNet-18 | Vanilla | CAS | CIFS
---|---|---|---
AutoAttack | 40.60 | 39.30 | 42.10

Best-epoch robustness during training: In Section 4.1, we report the robust
accuracies of ResNet-18 models at the last epochs during training. Here, we
report the best-epoch robustness for reference (Table 4.11). We see that CIFS
modified version enjoys the better best-epoch robustness in comparison to the
vanilla ResNet-18 model.

Table 4.11: Robustness comparison of defense methods on SVHN. We report the
best robust accuracies (%) during training. For each model, the results of the
strongest attack are marked with an underline.

ResNet-18 | Natural | FGSM | PGD-20 | C&W | PGD-100
---|---|---|---|---|---
Vanilla | 93.88 | 66.02 | 51.71 | 48.87 | 47.59
CAS | 93.90 | 65.53 | 50.52 | 48.39 | 46.39
CIFS | 93.27 | 67.36 | 52.67 | 50.20 | 48.36

####  4.6.4 More Results on FMNIST

Training and Evaluation details: On the FMNIST dataset, we train ResNet-10
with PGD-20 adversarial examples ($\epsilon=0.3$, step size $0.02$ with random
initialization). The $\beta$ in CIFS is set to be $2$. We train models for
$120$ epochs with the SGD optimizer (momentum $0.9$ and weight decay
$0.0002$). The learning rate starts with $0.1$ and is multiplied with $0.1$ at
epochs $45$, $75$ and $90$.

We evaluate the robustness of the ResNet-10 models against FGSM, PGD-20, and
PGD-100 white-box attacks. The perturbation is bounded by $\epsilon=0.3$ in
$l_{\infty}$ norm. The step size of PGD-20 is set to be $0.01$, and that of
PGD-100 is set to be $0.02$. Here, we report both the last-epoch robust
accuracies and the best-epoch robust accuracies in Table 4.12.

Table 4.12: Robustness comparison of defense methods on FMNIST. For each
model, the robust accuracies (%) of the strongest attack are remarked with an
underline. For each type of attack, the best defense results are highlighted
in bold. Comparing the defense rates of the strongest attacks, we observe that
CIFS outperforms other defenses by a large margin.

Last | Natural | FGSM | PGD-40 | PGD-100
---|---|---|---|---
Vanilla | 85.19 | 80.52 | 66.47 | 60.99
CAS | 86.59 | 82.45 | 65.58 | 59.51
CIFS | 83.35 | 77.48 | 66.59 | 65.50
Best | Natural | FGSM | PGD-40 | PGD-100
Vanilla | 85.19 | 81.21 | 67.63 | 63.36
CAS | 86.63 | 83.59 | 68.73 | 62.65
CIFS | 83.32 | 78.55 | 69.05 | 67.21

####  4.6.5 More Results on Ablation Study

##### Effects of $\beta$ in CIFS:

Here, we train CIFS-modified ResNet-18 models on CIFAR10 with various values
of $\beta$ in Equation (3). The coefficient $\beta$ balances the accuracies of
raw predictions and the final prediction. From Table 4.13, we observe that
$\beta$ values that are too small or too large values lead to drops in the
accuracies of natural data and adversarial data. On the one hand, if the value
of $\beta$ is too small, the raw predictions made by CIFS are not reliable.
Thus, the channels selected by CIFS may not be the truly relevant ones with
respect to the ground-truth class. On the other hand, if the value of $\beta$
is too large, the optimization procedure mostly considers the raw predictions,
the final prediction (output) becomes unreliable. When $\beta=2$, we achieve
the best robustness against various types of attack.

Table 4.13: Robustness accuracies (%) on CIFAR10 for CIFS with various values
of $\beta$.

ResNet-18 | Natural | FGSM | PGD-20 | PGD-100
---|---|---|---|---
Vanilla | 84.56 | 55.11 | 46.62 | 44.72
$\beta=0.1$ | 75.22 | 53.41 | 48.10 | 46.28
$\beta=1$ | 82.34 | 58.15 | 50.50 | 48.35
$\beta=2$ | 83.86 | 58.86 | 51.23 | 48.74
$\beta=10$ | 82.97 | 57.62 | 49.34 | 47.10
$\beta=100$ | 75.41 | 52.90 | 45.00 | 43.12

##### Effects of the top-$k$ feature assessment

In general, $k$ should be larger than 1 but not too large.

If we use the top-$1$, once adv. data fool probe nets, the channels relevant
to true labels will be missed, and this will lead to wrong predictions (Table
5, line top-1). Instead, we use top-$2$ for reliable channel selection. The
efficacy is attributed to two aspects: Firstly, the top-$2$ accuracies of adv.
data are usually high (see Table 4), thus channels relevant to top-$2$ logits
include those relevant to the true class. Secondly, tian2021analysis reports
that CNNs’ predictions of adv. data usually belong to the superclass that
contains true labels. Classes (e.g., cat, dog) in the same superclass (e.g.,
animals) usually share similar semantic features. Thus, most of the top-$2$
selected channels are useful for predicting the true class.

Although the top-$2$ selected channels may contain info about the other wrong
class, the following layers (after CIFS) are capable of “purifying” features
and make better predictions. This is verified by Table 5, the results in the
line top-2 (CIFS/CIFS 48.72% vs. CIFS/Final 54.96%) mean that around 6% adv.
data, which successfully fool probes, are still finally correctly classified.
However, too large $k$ may degrade the relevance assessment due to too much
noisy info (e.g., the effect of top-3 is worse than top-2 in Table 5).

##### Layers to be modified

Positions of CIFS modules: Here, we try different combinations of the layers
to be modified by CIFS. In CNNs, the features of deep layers are usually more
characteristic in comparison to those in the shallower layers
[zeiler_visualizing_2014], and each channel of the features captures a
distinct view of the input. The predictions often depend only on the
information of a few essential views of the inputs. CIFS improves adversarial
robustness by adjusting channel-wise activations. Thus, we apply CIFS to the
deeper layers instead of the shallower ones. Specifically, we modify the
ResNet-18 by applying CIFS at the last (P1) and/or the second last (P2)
residual blocks. The experimental results are reported in Table 4.14. We
observe that simultaneously applying CIFS into P1 and P2 performs the best
against various attacks. Intuitively, because the features can be
progressively refined, applying CIFS at P1&P2 better purifies the channels
compared to applying it only at P1 or P2.

Table 4.14: Robustness (%) comparison of the positions where CIFS modules are
placed.

ResNet-18 | Natural | FGSM | PGD-20 | PGD-100
---|---|---|---|---
Vanilla | 84.56 | 55.11 | 46.62 | 44.72
P1 | 84.02 | 57.60 | 48.45 | 45.95
P2 | 82.62 | 56.55 | 47.22 | 44.81
P1-P2 | 83.86 | 58.86 | 51.23 | 48.74

Table 4.15: Robustness comparison (%) of the probe architectures in CIFS
modules (at P1-P2).

ResNet-18 | Natural | FGSM | PGD-20 | PGD-100
---|---|---|---|---
Vanilla | 84.56 | 55.11 | 46.62 | 44.72
Linear-Linear | 81.52 | 58.33 | 51.32 | 49.07
MLP-Linear | 83.86 | 58.86 | 51.23 | 48.74

##### Architecture of Probe Networks

Linear vs. Non-linear Probe: For a certain layer modified by CIFS, the probe
network in CIFS serves as the surrogate classifier of the subsequent layers in
the backbone model. Thus, the probe networks should be powerful enough to make
correct predictions based on the features of this layer. For the CIFS in the
last residual block, we use a linear layer network as the probe, while for the
CIFS in the second last residual block, we compare the cases of using a linear
layer versus using a two-layer MLP network. From Table 4.15, we observe that
the MLP-Linear combination shows a similar performance compared to the
combination of two linear layers against adversarial attacks, but enjoys a
clear advantage on the natural data ($83.86$% vs. $81.52$%). This is because
the features in the second last residual block are not as characteristic as
those in the last block and cannot be linearly separated. The MLP can thus
classify the features better than a pure linear layer.

##  Chapter 5 Conclusion

###  5.1 Summary of Contributions

The overarching theme of this thesis is building adversarially robust deep
vision systems, of which three aspects were studied, i.e., evaluating
adversarial robustness, enhancing robustness, and the connection between
adversarial robustness and adaptivity to unseen data.

In Chapter 2, we systematically investigate the robustness of deep image
denoising models. In particular, we first propose a novel PGD-based
adversarial attack method, ObsAtk, to evaluate the robustness of deep image
denoisers. ObsAtk is the first attack method for image denoising that
considers the zero-mean assumption of natural noise. Then, we develop an
adversarial training strategy, HAT, that can effectively robustify deep image
denoisers. Finally, we reveal that adversarial robustness benefits the
generalization to unseen types of noise, i.e., deep denoisers trained with HAT
by using only Gaussian synthetic data can effectively remove various types of
real-world noise that are unseen during training.

In Chapter 3 and 4, we explore new families of models that can used to
construct more robust image classifiers in comparison to vanilla convolutional
layers. Specifically, in Chapter 3, we, for the first time, study the
robustness property of neural ordinary differential equations (NODEs). We
empirically demonstrate that NODE-based classifiers outperform CNN-based ones
against input perturbations. We further boost the robustness of NODEs by
introducing the time-invariant property and imposing a steady-state constraint
on the flow of ODEs. The resultant variant, TisODE, enjoys superior robustness
compared to the vanilla version and can work in conjunction with other
robustification techniques.

In Chapter 4, we improve the robustness of vanilla CNNs by introducing a novel
channel-wise feature selection mechanism, CIFS. Inspired by the observations
that adversarial training robustifies CNNs by aligning the channels’
activations with their relevances to true labels, we propose CIFS to modify
vanilla convolutional layers by adjusting channels’ activations with
multipliers generated based on their relevances. Extensive results verify the
effectiveness of CIFS on robustification.

###  5.2 Recommendations for Future Research

The works introduced in the previous chapters naturally lead to various
avenues for further research. We mention several possible extensions in this
section.

###### Robustness of Image Restoration Algorithms

Existing research papers mainly focus on the robustness of semantic-level
tasks, such as classification and detection. The robustness of image
restoration models, including denoising, deblurring, super-resolution, and
compressive sensing, is still under-explored. Image restoration lies at the
heart of intelligent vision systems. The performance of downstream image
understanding tasks highly depends on the quality of the reconstruction from
degraded images. Thus, it is essential to design robust image restoration
models that can reliably reconstruct high-quality images against malicious
degradation.

At first, we need to develop effective and efficient attack methods to
evaluate the robustness of image restoration algorithms. These are open
problems to solve because such attack methods have to consider the constraint
about the intrinsic properties of noise/perturbations in the natural world.
For example, the noise in images is usually assumed to be zero-mean; blurring
kernels, which smoothen images, usually contain all non-negative elements and
satisfy certain types of symmetry. Then, we can use the developed attack
methods to generate adversarial examples for adversarial training. When
designing adversarial training strategies, one should consider the performance
on authentic data, the robustness issue, as well as the generalization to
unseen types of degradation sources.

###### Robustifying Deep Models without Adversarial Training

Adversarial training (AT) is a generally applicable framework for training
robust models. However, training a large, robust model with PGD-AT consumes a
massive amount of computational resources, around ten or even twenty times of
those required by normal training. That is because AT needs to perform
multiple times of back-propagation to generate adversarial examples for each
batch of data on the fly. Although some research works attempt to accelerate
AT, the computational overhead is still very high. This situation motivates us
to explore novel architectures that can build robust models without
adversarial training.

In Chapter 3, we study the robustness property of NODEs. Although the proposed
TisODE enjoys superior robustness compared to convolutional layers, the
prediction performances on adversarial data are still suboptimal. TisODE
introduces a steady-state constraint on the flow of ODEs, but this constraint
is not sufficient for the stability of states. For future research, one may
consider a stronger regularization of the flow, i.e., the Lyapunov stability
condition, to encourage the models to learn stable equilibrium points in
feature spaces. The challenges here are how to ensure the Jacobian of the
dynamics function to be symmetric and negative-definite. In practice, the
other challenge is how to perform backpropagation through the eigenvalue
decomposition of the Jacobian if one proposes to regularize the largest
eigenvalue in the loss function.