# Improving Adversarial Training using Vulnerability-Aware Perturbation Budget

Olukorede Fakorede<EMAIL_ADDRESS>
Department of Computer Science
Iowa State University Modeste Atsague<EMAIL_ADDRESS>
Iowa State University Jin Tian<EMAIL_ADDRESS>
Iowa State University

###### Abstract

Adversarial Training (AT) effectively improves the robustness of Deep Neural
Networks (DNNs) to adversarial attacks. Generally, AT involves training DNN
models with adversarial examples obtained within a pre-defined, fixed
perturbation bound. Notably, individual natural examples from which these
adversarial examples are crafted exhibit varying degrees of intrinsic
vulnerabilities, and as such, crafting adversarial examples with fixed
perturbation radius for all instances may not sufficiently unleash the potency
of AT. Motivated by this observation, we propose two simple, computationally
cheap vulnerability-aware reweighting functions for assigning perturbation
bounds to adversarial examples used for AT, named Margin-Weighted Perturbation
Budget (MWPB) and Standard-Deviation-Weighted Perturbation Budget (SDWPB). The
proposed methods assign perturbation radii to individual adversarial samples
based on the vulnerability of their corresponding natural examples.
Experimental results show that the proposed methods yield genuine improvements
in the robustness of AT algorithms against various adversarial attacks.

##  1 Introduction

In recent years, Deep Neural Networks (DNNs) have demonstrated remarkable
success across various domains, achieving impressive performance benchmarks.
However, this success has come hand in hand with a critical concern: the
vulnerability of DNNs to well-crafted adversarial perturbations Szegedy et al.
(2013); Goodfellow et al. (2014). This observed brittleness has raised
questions about the safe deployment of DNNs in safety-critical applications.

In response to the challenge of adversarial vulnerability, a multitude of
defense mechanisms have been proposed to enhance the robustness of DNNs. Among
these, adversarial training (AT) Goodfellow et al. (2014); Madry et al. (2017)
stands out as one of the most prominent and effective approaches. AT typically
involves training neural networks using adversarial examples. The
effectiveness of AT has inspired many variants such as Zhang et al. (2019);
Wang et al. (2019); Ding et al. (2019); Zhang et al. (2020); Zeng et al.
(2021), among others. Moreover, alternative methods, such as adversarial
weight perturbation Wu et al. (2020), instance re-weighting Zhang et al.
(2020); Liu et al. (2021); Fakorede et al. (2023b), and hypersphere embedding
Pang et al. (2020); Fakorede et al. (2023a), have emerged to further enhance
the performance of existing AT variants.

It has been established that the efficacy of adversarial training varies
significantly across samples of various classes Xu et al. (2021); Fakorede et
al. (2023b); Wei et al. (2023). Adversarial examples derived from natural
samples that are inherently vulnerable are significantly misclassified in
adversarially trained models Liu et al. (2021); Zhang et al. (2020).
Specifically, the robust accuracy achieved when evaluating adversarial samples
originating from natural examples closer to class boundaries is substantially
lower than adversarial samples stemming from inherently more robust natural
examples. In response to these challenges, various reweighting techniques have
been introduced to enhance the effectiveness of AT Zhang et al. (2020); Liu et
al. (2021); Fakorede et al. (2023b). These techniques operate by assigning
largerr weights to the losses of disadvantaged examples.

It’s important to highlight that current AT methods predominantly rely on
adversarial examples generated with predetermined, fixed perturbation radii.
However, the notable performance variations observed across different
adversarial examples prompt a fundamental question:  Is uniform perturbation
of adversarial examples used in AT necessary or beneficial, as is commonly
practiced in existing research? For instance, should adversarial examples
crafted from inherently vulnerable natural samples be allocated the same
perturbation budget as those derived from more robust examples?

In this work, we present a case against applying uniform perturbation. We
demonstrate that first-order adversarial attacks, such as projected gradient
descent (PGD), induce a higher increase in adversarial loss for adversarial
samples originating from vulnerable natural examples compared to those derived
from inherently robust natural examples when subjected to uniform perturbation
radii. We also argue that enlarging the perturbation radii may increase the
inner maximization loss of adversarial examples derived from inherently robust
natural examples. Consequently, we introduce reweighting methods designed to
allocate varying perturbation budgets to individual adversarial examples
employed in adversarial training. Our proposed methods assign these budgets
based on the vulnerabilities exhibited by their corresponding natural
examples. We employ two methods for estimating each natural sample’s
vulnerability: one leveraging logit margin, and the other utilizing a modified
standard deviation measure of the output logits.

Our rationale is as follows: Natural examples that are intrinsically
vulnerable may require relatively smaller perturbations to yield effective
adversarial examples suitable for training. Conversely, when crafting
adversarial examples from intrinsically robust natural examples, relatively
larger perturbations may be necessary to achieve better training impact.
Experimental results show that the proposed methods improve the performance of
existing AT methods including standard AT Madry et al. (2017), TRADES Zhang et
al. (2019), and MART Wang et al. (2019).

We summarize the contributions of our work as follows:

  1. 1.

We argue for assigning varying perturbation radii to individual adversarial
samples based on the vulnerability of their corresponding natural examples in
the inner maximization component of the min-max adversarial training
framework.

  2. 2.

Consequently, we propose two reweighting functions for assigning perturbation
radii to individual adversarial examples based on their vulnerability.

  3. 3.

We empirically demonstrate the effectiveness of the proposed strategy in
improving adversarial training and show its superiority over existing
reweighting and adaptive perturbation radii methods, especially against strong
white-box and black-box attacks.

##  2 Related Work

###  2.1 Adversarial Robustness.

Adversarial robustness is a model’s ability to withstand adversarial attacks.
Many methods Guo et al. (2018); Papernot et al. (2017); Madry et al. (2017);
Goodfellow et al. (2014); Zhang et al. (2019) have been proposed to improve
adversarial robustness of neural networks. However, some of these methods are
ineffective against stronger attacks. Adversarial training (AT) Madry et al.
(2017), which involves training the model with adversarial examples obtained
under worst-case loss, has significantly improved robustness. Formally, AT
involves solving a min-max optimization as follows:

$\displaystyle\min_{\bm{\theta}}\mathbb{E}_{(\textbf{x},y)\sim\mathcal{D}}\left[\max_{\textbf{x}^{\prime}\in
B_{\epsilon}(\textbf{x})}L(f_{\bm{\theta}}(\textbf{x}^{\prime}),y)\right]$ (1)

where $y$ is the true label of input feature x, $L()$ represents the loss
function, $\bm{\theta}$ are the model parameters, and
$B_{\epsilon}(\textbf{x}):\\{\textbf{x}^{\prime}\in\mathcal{X}:\|{\textbf{x}}^{\prime}-\textbf{x}\|_{p}\leq\epsilon\\}$
represents the $l_{p}$ norm ball centered around $\bf{x}$ constrained by
radius $\epsilon$ . In Eq. (1), the inner maximization tries to obtain a
worst-case adversarial version of the input $\bf{x}$ that increases the loss.
The outer minimization then tries to find model parameters that would minimize
this worst-case adversarial loss. The relative success of AT has inspired an
array of variants including prominent TRADES Zhang et al. (2019) and MART Wang
et al. (2019), and a few others Wu et al. (2020); Pang et al. (2020).

###  2.2 Reweighting

Recent works have advocated for assigning unequal weights to the inner
maximization loss Zeng et al. (2021) and robust losses Liu et al. (2021);
Zhang et al. (2020) to improve the performance of AT. It has been shown that
adversarially-trained models poorly classify adversarial examples crafted from
natural examples that are intrinsically harder to classify Xu et al. (2021);
Fakorede et al. (2023b). Most existing reweighting methods attempt to improve
AT by upweighting the robust losses corresponding to vulnerable adversarial
examples. While some of these methods improve model robustness to certain
attacks, they have performed quite poorly in defending against strong attacks
Fakorede et al. (2023b).

In contrast with existing reweighting approaches focusing on reweighting
losses, this paper proposes a novel approach of reweighting the perturbation
radii of adversarial examples used for AT. Unlike previous works that attempt
to improve robust accuracy by assigning larger weights to examples closer to
the decision boundary, this work improves AT by assigning larger perturbation
budgets to adversarial examples crafted from inherently robust natural
examples.

###  2.3 Adaptive Perturbation Radii.

Few works in the literature have been directed at adaptive perturbation radii
for adversarial training. Notably, Ding et al.citeding2019mma proposed MMA
that maximizes margin to achieve robustness while also adaptively selecting
the “correct“ perturbation radius for each data point. The “correct“ radius
for each data point is characterized by the “shortest successful perturbation“
to misclassify the data point. Similarly, Balaji et al. citebalaji2019instance
proposed IAAT that begins with using a perturbation size ${\epsilon}_{i}$ that
is as large as possible, then adaptively adjusts ${\epsilon}_{i}$ depending on
whether PGD succeeds in finding a misclassified label at ${\epsilon}_{i}$.

In contrast to these works that perform an exhaustive search for suitable
perturbation radii, our proposed methods use pre-defined reweighting functions
for adaptively assigning perturbation radii to individual examples at no extra
cost. In addition, MMA and IAAT are designed to keep perturbed samples on the
decision boundary without pushing farther into incorrect classes, whereas our
methods assign significantly larger perturbations to intrinsically robust
examples. Most importantly, MMA failed to improve adversarial robustness over
standard AT against strong adversarial attacks like Auto Attacks.111The
comparison results with MMA is shown in Table 5. The self-reported IAAT
robustness for PGD attack is lower compared to standard AT Madry et al.
(2017).

##  3 Preliminaries

We use bold letters to represent vectors. We denote
$\mathcal{D}=\\{\textbf{x}_{i},y_{i}\\}_{i=1}^{n}$ a data set of input feature
vectors $\textbf{x}_{i}\in\mathcal{X}\subseteq\mathbf{R}^{d}$ and labels
$y_{i}\in\mathcal{Y}$, where $\mathcal{X}$ and $\mathcal{Y}$ represent a
feature space and a label set, respectively.

Let $f_{\theta}:\mathcal{X}\to R^{C}$ denote a deep neural network (DNN)
classifier with parameters $\theta$, and $C$ represents the number of output
classes. For any $\mathbf{x}\in\mathcal{X}$, let the class label predicted by
$f_{\theta}$ be
$F_{\theta}(\mathbf{x})=\arg\max_{k}f_{\theta}(\mathbf{x})_{k}$, where
$f_{\theta}(\mathbf{x})_{k}$ denotes the $k$-th component of
$f_{\theta}(\mathbf{x})$. $f_{\theta}(x)_{y}$ is the probability of $x$ having
label $y$.

We denote $\|\cdot\|_{p}$ as the $l_{p}$\- norm over $\mathbf{R}^{d}$, that
is, for a vector
$\mathbf{x}\in\mathbf{R}^{d},\|\textbf{x}\|_{p}=(\sum^{d}_{i=1}|\textbf{x}_{i}|^{p})^{\frac{1}{p}}$.
An $\epsilon$-neighborhood for x is defined as
$B_{\epsilon}(\mathbf{x}):\\{\mathbf{x}^{\prime}\in\mathcal{X}:\|{\mathbf{x}}^{\prime}-\mathbf{x}\|_{p}\leq\epsilon\\}$.
An adversarial example corresponding to a natural input $\mathbf{x}$ is
denoted as $\mathbf{x}^{\prime}$. We often refer to the loss resulting from
the adversarial attack (inner maximization) as adversarial loss.

##  4 Proposed Approach

In this section, we motivate and present two novel perturbation budget
assignment functions for the inner maximization of the AT framework.

###  4.1 A Case Against Fixed Perturbation.

AT employs a min-max optimization procedure aimed at minimizing worst-case
losses computed over perturbed adversarial examples. While the application of
AT significantly enhances model robustness, it is essential to acknowledge
that the optimized worst-case losses represent approximations that may
sometimes be suboptimal Mao et al. (2023). This limitation arises from the
nature of the Projected Gradient Descent (PGD) algorithm, which is prone to
converging to local maxima scattered across the optimization landscape.

Furthermore, the geometric characteristics inherent to individual natural data
samples utilized in the inner maximization step of the min-max optimization
exhibit substantial variation. Some natural samples inherently fall into
misclassification regions or exhibit elevated loss values. For these
particular examples, the PGD algorithm may readily identify suitable maxima.
Conversely, naturally robust samples may not yield the worst-case loss under
similar optimization settings as their more vulnerable counterparts.

The pursuit of adversarial examples that optimally maximize losses represents
a compelling objective for achieving highly robust models. Nevertheless, it is
important to recognize that the prevailing practice involves applying uniform
optimization settings for the inner maximization step, irrespective of the
individual idiosyncrasies inherent to each original sample. Specifically, it
is commonly assumed that the maximal loss for each example can be discovered
using the same perturbation radius. However, it becomes evident that, under
this uniform perturbation radius, the losses observed for adversarial examples
derived from challenging natural samples exhibit an increased discrepancy
compared to those computed for natural samples situated farther from the
decision boundary. This assertion warrants a theoretical examination, which we
delve into below.

###### Theorem 1

Let $\mathcal{L}$ and $f_{\theta}(.)$ denote the cross-entropy loss function
and the predictions of the model respectively. Consider two natural input-
label pairs $(x_{1},y_{1})$ and $(x_{2},y_{2})$ such that
$\mathcal{L}(x_{1},y_{1})>\mathcal{L}(x_{2},y_{2})$. The following holds for
first-order adversarial examples crafted from $x_{1}$ and $x_{2}$ within the
same perturbation radius $\epsilon$:

  1. 1.

$\mathcal{L}(f_{\theta}({x}^{\prime}_{1}),y_{1})>\mathcal{L}(f_{\theta}({x}^{\prime}_{2}),y_{2})$

  2. 2.

$\mathcal{L}(f_{\theta}({x}^{\prime}_{1}),y_{1})-\mathcal{L}(f_{\theta}(x_{1}),y_{1})>\mathcal{L}(f_{\theta}({x}^{\prime}_{2}),y_{2})-\mathcal{L}(f_{\theta}(x_{2}),y_{2})$

Remark. Theorem 1 underscores that when subjected to the same perturbation
radius, adversarial examples stemming from vulnerable (with high loss values)
natural examples incur a relatively more substantial loss increase. Similarly,
adversarial examples generated from robust natural examples with lower natural
losses induce relatively smaller loss increments than adversarial examples
from vulnerable natural examples. Furthermore, it can be inferred that under
uniform perturbation, adversarial examples from various natural examples have
varying loss increments.

Finding adversarial examples with better maxima is associated with better
adversarial robustness Madry et al. (2017). Therefore, using a uniform
perturbation radius for the inner-maximization may not yield the best
robustness. The reason is due to the considerable variance in the inner
maximization losses of individual examples under uniform perturbation radius.
Additionally, considering that adversarial examples originating from
inherently robust (low-loss) natural examples tend to result in relatively
smaller increases in loss, we suggest an approach where we assign varying
perturbation radii to each instance of training example based on their natural
vulnerability. We demonstrate in Theorem 2 that enlarging the perturbation
radii around an example can increase the loss.

###### Theorem 2

The inner maximization $\max_{\textbf{x}^{\prime}\in
B_{\epsilon}(\textbf{x})}L(f_{\theta}(\textbf{x}^{\prime}),y)$ increases as
$\epsilon$ increase.

###  4.2 Vulnerability-Aware Reweighting for Perturbation Radii

In the preceding section 4.1, we presented a rationale against the use of
uniform perturbation radii for crafting adversarial examples employed in AT.
Instead, we advocate for assigning distinct perturbation radii for generating
adversarial examples based on the inherent vulnerabilities of their original
natural examples. Here, we propose two measures of estimating the intrinsic
vulnerability of individual natural examples from two perspectives: (1) the
geometric proximity of each example to the decision boundary, captured using
logit margins, and (2) the standard deviation of the DNN’s output logits.

####  4.2.1 Margin-based Vulnerability Estimation

Measuring the exact proximity of a data point to the decision boundary is not
straightforward for non-linear models like DNNs. We adopt a measure of multi-
class margin described in Koltchinskii & Panchenko (2002) to estimate the
vulnerability or robustness of natural examples. Consider the predictions of a
DNN denoted by $f_{\theta}$ and a labelled example $(x,y)$, the margin
$d_{m}(x,y;\theta)$ is given as follows:

$d_{m}(\textbf{x},y;\theta)=f_{\theta}(\textbf{x})_{y}-\max_{k,k\neq
y}f_{\theta}(\textbf{x})_{k}$ (2)

where $f_{\theta}(\textbf{x})_{y}$ is the model’s predicted probability of the
correct label $y$, and $\max_{k,k\neq y}f_{\theta}(\textbf{x})_{k}$ is the
largest prediction of the remaining classes.

We utilize the information provided by $d_{m}(\textbf{x},y;\theta)$ in
measuring the vulnerability of a natural input example x as follows:

  * •

If $d_{m}(\textbf{x},y;\theta)>0$, x is correctly classified and scored. We
consider x relatively robust.

  * •

If $d_{m}(\textbf{x},y;\theta)=0$, it implies that x has the same prediction
score as the best of the remaining classes. As such, we consider x to be
located at the class boundary.

  * •

If $d_{m}(\textbf{x},y;\theta)<0$, x is considered to be vulnerable, since it
is located in a wrong region even before x is adversarially perturbed.

  * •

In addition to identifying whether a sample x is vulnerable, we assess the
degree of vulnerability based on the magnitude of the value returned by
$d_{m}(\textbf{x},y;\theta)$. For example, if
$d_{m}(\textbf{x}1,y_{1};\theta)>d_{m}(\textbf{x}_{2},y_{2};\theta)$, we infer
that $\textbf{x}_{2}$ is relatively more vulnerable than $\textbf{x}_{1}$.

####  4.2.2 Standard-Deviation-based Vulnerability Estimation

In this section, we propose estimating the vulnerability of individual
examples using a modified standard deviation of a DNN’s model output logits.

The standard deviation serves as a metric to assess the distribution spread,
where a smaller standard deviation implies a more uniform distribution. In the
context of a model’s output logits on a given input, an evenly spread
distribution indicates a higher risk of misclassification. This is because the
model’s estimated probabilities for an input, with a more even spread among
both correct and incorrect classes, suggest potential vulnerability to
misclassification. It’s noteworthy that previous research has established
connections between various variance estimations and the difficulty or
susceptibility of input examples agarwal2022estimating as well as classes Xu
et al. (2021). In this context, we leverage the standard deviation of
predicted probabilities to gauge the vulnerability of individual samples.

Conventionally, the standard deviation is measured as the variation of random
variables around the mean of the distribution. Here, we modify the original
standard deviation formula by replacing the mean with the model’s predicted
probability of the example belonging to the true class as follows:

$\displaystyle
d_{std}(\textbf{x}_{i},y_{i},\theta)=\sqrt{\frac{\sum_{k=1}^{C}(f_{\theta}(\textbf{x}_{i})_{k}-f_{\theta}(\textbf{x}_{i})_{y_{i}})^{2}}{|C|}}$
(3)

where $x_{i}$ and $y_{i}$ represent the input of a sample and its
corresponding label, and $C$ is the number of classes. The proposed
formulation in Eq. (3) measures the spread of the model’s predicted
probabilities around the model’s predicted probability of the true class. A
low $d_{std}(x_{i},y_{i},\theta)$ implies that the model is not confident in
its estimated probability of $x_{i}$ belonging to $y_{i}$, indicating higher
risk and vulnerability of misclassification.

Contrary to the logit margin approach in Eq.(2), which calculates the
disparity between the probability of the correct class for a sample and the
probability of the nearest incorrect class, the suggested modified standard
deviation considers the model’s predictions for all classes.

###  4.3 Weight Assignment for Perturbation Radii

In Theorem 1, we establish that the inner maximization process inherently
induces a more substantial increase in loss for samples that possess
intrinsically high loss values. This holds true for misclassified natural
samples, characterized by negative margins and exhibiting high loss values.
Furthermore, considering that these misclassified examples are already
situated within regions of high loss, we advocate for a strategy in which the
inner maximization process for misclassified examples employs smaller
perturbation radii. Conversely, the inner maximization process inherently
leads to a relatively lower increase in loss for samples with low loss values,
typically associated with positive margins. Consequently, we propose the
utilization of larger perturbation radii for generating adversarial examples
from these low-loss samples.

Therefore, we introduce two vulnerability-aware radius reweighting functions
based on the two measures in Eq. (2) and (3) respectively. The first
reweighting function, termed Margin-weighted Perturbation Budget (MWPB), is
formulated as follows:

${\epsilon}_{i}=exp(\alpha\cdot d_{m}(\textbf{x}_{i},y_{i};\theta))*\epsilon$
(4)

where $\epsilon=8/255$, the commonly used fixed perturbation radius parameter,
$d_{m}(\textbf{x}_{i},y_{i};\theta)$ is given by Eq. (2), and $\alpha$ is a
hyperparameter that controls the weight of the function.

The MWPB reweighting function in Eq. (4) allocates larger perturbation radii
$({\epsilon}_{i}>8/255)$ when generating adversarial examples from natural
samples characterized by positive margins. In contrast, it assigns smaller
perturbation radii $({\epsilon}_{i}<8/255)$ when crafting adversarial examples
from samples with negative margins. Finally, adversarial examples originating
from samples situated exactly at the class boundary are crafted using the
default perturbation radii $({\epsilon}_{i}=8/255)$.

The second reweighting function, termed Standard-Deviation-Weighted
Perturbation Budget (SDWPB), is as follows:

${\epsilon}_{i}=exp(\alpha\cdot
d_{std}(\textbf{x}_{i},y_{i};\theta))*\epsilon$ (5)

where $d_{std}(\textbf{x}_{i},y_{i};\theta)$ is given by Eq. (3). The above
SDWPB reweighting function allocates comparatively larger perturbation radii
when generating adversarial examples from natural instances with relatively
larger $d_{std}(\textbf{x}_{i},y_{i};\theta)$ values. Unlike the MWPB
reweighting, SDWPB does not assign perturbation radii ${\epsilon}_{i}$ to
values less than $8/255$ because the proposed
$d_{std}(\textbf{x}_{i},y_{i};\theta)$ is non-negative. However, both MWPB and
SDWPB assign relatively larger perturbation radii to samples that are deemed
to be naturally more robust based on the metrics defined in Eq. (2) and (3)
respectively.

As seen in Theorem 1, first order adversarial examples crafted from relatively
robust natural examples incur relatively smaller increment in losses. By
assigning relatively larger perturbation radii for crafting adversarial
examples from more robust natural examples, the reweighting functions in Eq.
(4) and (5) ensure that larger inner maximization losses for these adversarial
examples. We show in Theorem 2 that increasing perturbation radii can increase
the inner maximization loss.

###  4.4 Applying the Proposed Weighted Perturbation Budget Methods

Every adversarial training method is a variant of min-max optimization. Hence,
our proposed reweighting methods may be applied to any adversarial training
variant. We re-write the min-max adversarial training objective in Eq. (1) as
follows:

$\displaystyle\min_{\bm{\theta}}\frac{1}{n}\sum_{i=1}^{n}\left[\max_{\textbf{x}_{i}^{\prime}\in
B_{{\epsilon}_{i}}(\textbf{x}_{i})}L(f_{\theta}(\textbf{x}_{i}^{\prime}),y_{i})\right]$
(6)

where each ${\epsilon}_{i}$ is computed according to Eq. (4) (MWPB) or Eq. (5)
(SDWPB) for each input-label pair $(\textbf{x}_{i},y_{i})$, and
$B_{{\epsilon}_{i}}(\mathbf{x_{i}}):\\{\textbf{x}_{i}^{\prime}\in\mathcal{X}:\|{\textbf{x}_{i}}^{\prime}-\textbf{x}_{i}\|_{p}\leq{\epsilon}_{i}\\}$.
For the purpose of our experiments, which we present in Section 5, we apply
MWPB and SDWPB to popular existing AT variants standard AT Madry et al.
(2017), TRADES Zhang et al. (2019), and MART Wang et al. (2019).

###  4.5 Challenge of Adversarial Training with Larger Perturbation Radii

The adversarial loss landscape is unfavorable to optimization under large
perturbation budgets Liu et al. (2020). It is shown that when a perturbation
size $\epsilon$ is large, the gradients become small due to decreased gradient
magnitude in the initial sub-optimal region, making it challenging for the
model to escape the sub-optimal initial region. Large perturbation size may
also encourage a model to find sharper minima. In contrast, smaller
perturbation budgets facilitate larger gradient magnitude in the initial sub-
optimal regions, which in turn help prevent the model from getting stuck in
these regions.

Given that our proposed method requires perturbing a subset of the training
data with relatively larger perturbation budgets, we use a two-phase training
approach. We train the initial epochs with adversarial examples obtained under
a smaller perturbation budget of ${\epsilon}/2$. This allows the model to
gradually adapt. Subsequently, we transition to the adversarial training
objective in Eq. (6), which employs larger perturbation budgets. We provide
the algorithms for MWPB-AT and SDWPB-AT in the Appendix C.

##  5 Experiments

In this section, we extensively evaluate the proposed method. To test the
versatility of our method, we test on various datasets including CIFAR-10
Krizhevsky et al. (2009), SVHN Netzer et al. (2011), and TinyImageNet Deng et
al. (2009). We apply simple data augmentations such as 4-pixel padding with 32
× 32 random crop and random horizontal flip on CIFAR-10 and Tiny Imagenet. We
utilize Resnet-18 He et al. (2016) and Wideresnet-34-10 He et al. (2016) as
the backbone models. For brevity, we respectively denote ResNet-18 and
Wideresnet-34-10 as RN18 and WRN34-10.

###  5.1 Experimental Setup

####  5.1.1 Training Parameters.

We trained the networks using mini-batch gradient descent for 110 epochs, with
momentum 0.9 and batch size 128. We use the weight decay of 5e-4 for training
CIFAR-10 and 3.5e-3 for SVHN and Tiny Imagenet. The initial learning rate is
set to 0.1 (0.01 for SVHN and Tiny Imagenet), and divided by 10 in the 80-th
epoch, and then at the 90-th epoch. We train the first 80 epochs with
adversarial examples obtained via PGD with a smaller perturbation budget of
$4/255$ and step size of $1/255$. Subsequently, we introduce MWPB-AT / SDWPB-
AT, MWPB-TRADES / SDWPB-TRADES, and MWPB-MART / SDWPB-MART in the 81-st epoch
to improve AT Madry et al. (2017), TRADES Zhang et al. (2019) and MART Wang et
al. (2019) respectively.

####  5.1.2 Hyperparameters.

The values of $\alpha$ in Eq. (4) and (5) are determined heuristically for
each of AT methods.

MWPB. In MWPB-AT, MWPB-TRADES, and MWPB-MART on CIFAR-10, the $\alpha$ values
are 0.58, 0.42, and 0.55, respectively. For Tiny Imagenet, the corresponding
$\alpha$ values are 0.55, 0.4, and 0.7. On SVHN, the corresponding $\alpha$
values are 0.5, 0.15, and 0.6.

SDWPB. For SDWPB-AT, SDWPB-TRADES, and SDWPB-MART on CIFAR-10, the $\alpha$
values are 0.62, 0.5, and 0.52. On Tiny Imagenet, the values are 0.5, 0.5, and
0.8. On SVHN, the $\alpha$ values are 0.6, 0.2, and 0.7.

We show how the hyperparameter $\alpha$ was selected and study the influence
of $\alpha$ on the natural and robust accuracy for MWPB-AT/SDWPB-AT on
CIFAR-10 using WRN-34-10. We provide the results and more explanations in
Appendix A.3.

####  5.1.3 Baselines.

Our baselines include Standard AT Madry et al. (2017), TRADES Zhang et al.
(2019), and MART Wang et al. (2019). Furthermore, we conduct a comparative
analysis of our approach against MMA Ding et al. (2019), which also introduces
adaptive perturbation radii to enhance adversarial robustness. Lastly, we
compare our results to other works that utilize logit-margin for improving
adversarial robustness MAIL Liu et al. (2021), WAT Zeng et al. (2021), AWP Wu
et al. (2020) and ST-AT Li et al. (2023). All the hyperparameters of the
baselines are the same as in their original papers. Nevertheless, we maintain
consistency by using the same learning rate, batch size, and weight decay
values as those utilized during the training of our proposed method.

###  5.2 Threat Models

We assess the performance of the proposed method attacks under White-box and
Black-box settings and Auto attack.

White-box attacks. These attacks have access to model parameters. To evaluate
robustness on CIFAR-10 using RN-18 and WRN34-10, we apply the PGD attack with
$\epsilon=8/255$, step size $\kappa$ = $1/255$, $K=20$; CW (CW loss Carlini &
Wagner (2017) optimized by PGD-20) attack with $\epsilon=8/255$, step size
$1/255$. On SVHN and Tiny Imagenet, we apply PGD attack with $\epsilon=8/255$,
step size $\kappa$ = $1/255$, $K=100$.

Black-box attacks. An adversary does not have access to the model parameters
under black-box settings. We tested the robust models trained on CIFAR-10
against strong black-box attacks Square Andriushchenko et al. (2020) with
5,000 queries and SPSA Uesato et al. (2018) with 100 iterations, perturbation
size of 0.001 (for gradient estimation), learning rate = 0.01, and 256 samples
for each gradient estimation. All black-box evaluations are made on trained
WRN34-10.

Auto attacks. Lastly, we evaluated the trained models on Autoattack
($l_{\infty}$ and $l_{2}$)Croce & Hein (2020b), which is a powerful ensemble
of attacks consisting of APGD-CE Croce & Hein (2020b), APGD-T Croce & Hein
(2020b), FAB-T Croce & Hein (2020a), and Square (a black-box attack)
Andriushchenko et al. (2020) attacks.

###  5.3 Performance Evaluation

We present CIFAR-10 results using RN18 and WRN34-10 in Tables 1 and 4,
respectively. Additionally, SVHN and Tiny Imagenet results using RN18 are
reported in Tables 2 and 3. Experimental outcomes are averaged over three runs
with random seeds, and standard deviations are omitted as they are deemed
insignificant $(<0.3)$.

Table 1: Comparing white-box attack robustness (accuracy %) for RN18 on CIFAR-10. Defense | Natural | PGD-20 | CW | AA
---|---|---|---|---
AT | 84.10 | 52.72 | 51.80 | 47.95
MWPB-AT | 83.78 | 56.25 | 53.02 | 49.96
SDWPB-AT | 83.56 | 56.69 | 53.51 | 50.01
MART | 80.32 | 55.15 | 49.35 | 47.63
MWPB-MART | 82.23 | 57.10 | 52.57 | 49.53
SDWPB-MART | 80.89 | 57.36 | 52.66 | 49.45
TRADES | 82.65 | 52.82 | 51.82 | 48.96
MWPB-TRADES | 82.89 | 55.53 | 53.04 | 50.73
SDWPB-TRADES | 82.69 | 55.36 | 52.92 | 50.19
Table 2: Comparing white-box attack robustness (accuracy %) for RN18 on SVHN. Defense | Natural | PGD-20 | CW | AA
---|---|---|---|---
AT | 92.27 | 55.67 | 52.92 | 45.94
MWPB-AT | 91.45 | 61.81 | 55.89 | 49.73
SDWPB-AT | 91.06 | 61.59 | 56.40 | 48.58
MART | 91.59 | 58.78 | 52.79 | 43.60
MWPB-MART | 91.51 |  61.87 | 55.11 | 48.45
SDWPB-MART | 91.33 | 61.95 | 55.48 | 49.45
TRADES | 90.85 | 57.27 | 53.59 | 46.45
MWPB-TRADES | 90.35 | 60.25 | 55.03 | 50.11
SDWPB-TRADES | 90.29 | 60.11 | 55.11 | 50.23
Table 3: Comparing white-box attack robustness (accuracy %) for RN18 on Tiny Imagenet. Defense | Natural | PGD-20 | CW | AA
---|---|---|---|---
AT | 48.83 | 23.96 | 21.85 | 17.91
MWPB-AT | 51.21 | 25.07 | 23.11 | 19.75
SDWPB-AT | 51.43 | 25.81 | 23.55 | 20.45
MART | 46.01 | 26.03 | 21.78 | 19.18
MWPB-MART | 47.39 | 27.15 | 22.89 | 20.51
SDWPB-MART | 46.71 | 27.55 | 23.01 | 20.81
TRADES | 49.11 | 22.82 | 17.79 | 16.82
MWPB-TRADES | 52.12 | 24.60 | 19.85 | 18.15
SDWPB-TRADES | 53.11 | 24.74 | 20.01 | 18.34

####  5.3.1 Performance on natural examples.

Training with adversarial examples crafted with larger perturbation budgets
($>8/255$) tends to lower natural accuracy. The warm-up period utilized in our
training before introducing MWPB and SDWPB helped ease the introduction of
larger perturbation radii in the later epochs. The warm-up approach, where
training starts normally before introducing reweighting or other adaptive
methods, is common in related work, e.g., Ding et al. (2019); Liu et al.
(2021); Fakorede et al. (2023b), to mention a few. Here, we train the first 80
epochs with $\epsilon$=4/255 and a step size 1/255 then transition to the
adaptive perturbation budgets. Experimental results show that our methods,
despite of larger perturbation radii used on some examples, yield better
natural accuracy in some cases especially on CIFAR-10 and Tiny Imagenet. We
provide more information in Appendix A.1

####  5.3.2 Comparison with vanilla baselines

We compared our proposed method with vanilla baselines standard AT Madry et
al. (2017), TRADES Zhang et al. (2019) and MART Wang et al. (2019).
Experimental results demonstrate that the introduction of MWPB and SDWPB lead
to enhancements in AT, TRADES, and MART. Moreover, our proposed methods
exhibit improvements in robust accuracy without compromising natural accuracy.
These performance gains are consistent across different datasets and
baselines. Specifically, when combined with AT, MWPB-AT showcases notable
improvements against adversarial attacks PGD-20 (+3.06), CW (+2.41),
AA($l_{\infty}$) (+2.24) on CIFAR-10 when using WRN34-10. Similarly, on
datasets SVHN and Tiny Imagenet with RN18, MWPB-AT outperforms AT against
PGD-20, CW, and Autoattack. The performance of MWPB remains consistent when
integrated with TRADES and MART. MWPB-TRADES exhibits enhancements in both
natural accuracy and robustness against attacks PGD, CW, and Autoattack.
Similarly, MWPB-MART shows considerable improvements, especially against
Autoattack, on CIFAR-10 when using WRN-34-10 These improvements extend to
datasets SVHN and Tiny Imagenet with RN-18.

Likewise, SDWPB-AT, SDWPB-TRADES, and SDWPB-MART improve over AT, TRADES, and
MART respectively. SDWPB-TRADES yields a better improvement over TRADES than
MWPB-TRADES (+0.25) on CIFAR-10 when using WRN-34-10. However, on CIFAR-10,
SDWPB appears to marginally reduce the natural accuracy on AT, TRADES, and
MART. Experimental results also suggest that SDWPB generally performs better
than MWPB on Tiny Imagenet, as may be observed in Table 3.

Lastly, experimental results also show that our method improves performance on
strong black-box attacks Square and SPSA, summarized in the last two columns
of Table 4.

Table 4: Comparing white-box and black-box attack robustness (accuracy %) for WRN34-10 on CIFAR-10. Defense | Natural | PGD-20 | CW |  AA ($L_{\infty}$) |  AA ($L_{2}$) | SQUARE | SPSA |
---|---|---|---|---|---|---|---|---
|  |  $\epsilon$ = 8/255 |  $\epsilon$ = 8/255 |  $\epsilon$ = 8/255 |  $\epsilon$ = 128/255 |  |  |
AT | 86.21 | 56.12 | 54.95 | 51.92 | 58.52 | 60.12 | 61.05 |
MWPB-AT | 86.82 | 59.18 | 57.36 | 54.16 | 61.09 | 61.15 | 63.07 |
SDWPB-AT | 86.09 | 59.36 | 57.04 | 54.08 | 60.83 | 60.41 | 63.14 |
MART | 84.17 | 58.10 | 54.51 | 51.11 | 57.75 | 58.74 | 58.91 |
MWPB-MART | 85.70 | 60.65 | 56.78 | 53.80 | 60.41 | 60.83 | 62.02 |
SDWPB-MART | 85.31 | 60.71 | 56.82 | 53.88 | 60.09 | 60.35 | 61.89 |
TRADES | 84.70 | 56.30 | 54.51 | 53.06 | 58.05 | 59.16 | 61.15 |
MWPB-TRADES | 86.09 | 59.10 | 57.04 | 54.38 | 60.03 | 60.77 | 62.19 |
SDWPB-TRADES | 85.62 | 58.99 | 57.01 | 54.49 | 59.89 | 60.69 | 62.31 |
Table 5: Comparing white-box and black-box attack robustness (accuracy %) of
various margin-based approaches for WRN34-10 on CIFAR-10, and other prominent
baselines

.

Defense | Natural | PGD-20 | CW | AA | SPSA |
---|---|---|---|---|---|---
MMA (Ding et al. (2019)) | 86.29 | 57.12 | 57.59 | 44.52 | 59.87 |
WAT (Zeng et al. (2021)) | 85.13 | 56.63 | 53.97 | 50.01 | 60.75 |
MAIL (Liu et al. (2021)) | 86 .81 | 60.49 | 51.45 | 47.11 | 59.25 |
GAIRAT (Zhang et al. (2020)) | 85.41 | 60.76 | 45.02 | 42.29 | 52.32 |
ST-AT (Li et al. (2023)) | 84.91 | 57.52 | 55.11 | 53.54 | 61.34 |
AWP (Wu et al. (2020)) | 85.36 | 58.04 | 55.92 | 53.92 | 62.57 |
MWPB-AT (ours) | 86.85 | 59.18 | 57.36 | 54.16 | 63.07 |
SDWPB-AT(ours) | 86.09 | 59.36 | 57.04 | 54.08 | 63.14 |
MWPB-AWP (ours + awp) | 87.61 | 61.56 | 58.54 | 56.02 | 64.11 |
SDWPB-AWP (ours + awp) | 87.59 | 61.89 | 59.09 | 56.22 | 64.22 |

####  5.3.3 Comparison with other adaptive radii, margin-based, and recent
methods

We compare our proposed method to MMA Ding et al. (2019), which also aims to
improve adversarial training by enabling the adaptive selection of the
“correct“ perturbation radii. MMA minimizes adversarial losses at the
“shortest possible perturbation“ for individual examples. Experimental results
displayed in Table 5 show that the proposed MWPB-AT outperforms MMA on natural
accuracy (+0.46), PGD-20 (+2.06), AA (+9.64), and SPSA (+3.2), albeit MMA
slightly performs better on CW attack (-0.33). Similarly, SDWPB-AT performs
better than MMA on PDG-20 (+2.24), AA (+ 9.56).

It’s important to highlight that MMA employs a bisection search algorithm to
determine optimal perturbation radii for adversarial examples, whereas our
approach involves a simpler reweighting of the commonly used perturbation
radius. This difference in methodology is worth noting, as the bisection
search employed by MMA can be computationally more expensive.

Prior works WAT Zeng et al. (2021) and MAIL Liu et al. (2021) have
incorporated the idea of multi-class margin specified in Eq.(2) to improve
adversarial robustness. Specifically, these methods utilize multi-class logit
margins to reweight adversarial losses, assigning larger weights to losses
corresponding to easily misclassified adversarial examples. Experimental
results in Table 5 show that our methods perform better than these methods on
stronger attacks CW and Auto attack. Also, these methods have been argued to
show signs of gradient obfuscation Fakorede et al. (2023b), given their low
performance on strong black-box attacks.

The proposed methods also significantly outperform a prominent reweighting
approach GAIRAT Zhang et al. (2020) on stronger attacks CW and AA. MWPB-AT and
SDWPB-AT improve over GAIRAT on AA by 11.87% and 11.79% , respectively.
Similarly, the proposed methods outperform recent work ST-AT Li et al. (2023)
on natural accuracy and all other attacks evaluated. Combining AWP Wu et al.
(2020) with MWPB-AT and SDWPB-AT respectively improve over AWP on all attacks
as observed in Table 5.

####  5.3.4 Distribution of perturbation radii

We show in Fig. 1 in the Appendix the perturbation radii distributions for
MWPR-AT/SDWPB-AT, MWPB-TRADES/SDWPB-TRADES, and MWPB-MART/SDWPB-MART for RN-18
over 50,000 training samples of CIFAR-10. The perturbation radii distribution
is computed on the best-performing epoch in each case.

In addition, we added more experimental results showing the robustness of the
proposed methods under various perturbation radii and strong adaptive attack
FAB Croce & Hein (2020a) in Appendix A.1.

##  6 Conclusion

In this paper, we argue that natural examples, from which adversarial examples
are generated, exhibit differing levels of inherent vulnerabilities. As a
result, we advocate against the use of uniform perturbations in the inner
maximization step of the adversarial training framework. Rather, we propose
instance-specific weighting functions for determining the perturbation budgets
when crafting adversarial examples for adversarial training. The weighting
function assesses the vulnerability of each natural example and utilizes this
information for determining perturbation radii when generating adversarial
examples. Experimental results show that our proposed approach consistently
enhances the performance of popular adversarial training methods across
various datasets and under different attacks.

## References

  * Andriushchenko et al. (2020) Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein.  Square attack: a query-efficient black-box adversarial attack via random search.  In _European Conference on Computer Vision_ , pp. 484–501. Springer, 2020.
  * Carlini & Wagner (2017) Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In _2017 ieee symposium on security and privacy (sp)_ , pp. 39–57. Ieee, 2017.
  * Croce & Hein (2020a) Francesco Croce and Matthias Hein.  Minimally distorted adversarial examples with a fast adaptive boundary attack.  In _International Conference on Machine Learning_ , pp. 2196–2205. PMLR, 2020a.
  * Croce & Hein (2020b) Francesco Croce and Matthias Hein.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In _International conference on machine learning_ , pp. 2206–2216. PMLR, 2020b.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Ding et al. (2019) Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang.  Mma training: Direct input space margin maximization through adversarial training.  In _International Conference on Learning Representations_ , 2019.
  * Fakorede et al. (2023a) Olukorede Fakorede, Ashutosh Nirala, Modeste Atsague, and Jin Tian.  Improving adversarial robustness with hypersphere embedding and angular-based regularizations.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 1–5. IEEE, 2023a.
  * Fakorede et al. (2023b) Olukorede Fakorede, Ashutosh Kumar Nirala, Modeste Atsague, and Jin Tian.  Vulnerability-aware instance reweighting for adversarial training.  _Transactions on Machine Learning Research_ , 2023b.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ , 2014.
  * Guo et al. (2018) Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten.  Countering adversarial images using input transformations.  In _International Conference on Learning Representations_ , 2018.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Katharopoulos & Fleuret (2017) Angelos Katharopoulos and François Fleuret.  Biased importance sampling for deep neural network training.  _arXiv preprint arXiv:1706.00043_ , 2017.
  * Koltchinskii & Panchenko (2002) Vladimir Koltchinskii and Dmitry Panchenko.  Empirical margin distributions and bounding the generalization error of combined classifiers.  _The Annals of Statistics_ , 30(1):1–50, 2002.
  * Krizhevsky et al. (2009) Alex Krizhevsky et al.  Learning multiple layers of features from tiny images.  2009\.
  * Li et al. (2023) Qizhang Li, Yiwen Guo, Wangmeng Zuo, and Hao Chen.  Squeeze training for adversarial robustness.  2023\.
  * Liu et al. (2020) Chen Liu, Mathieu Salzmann, Tao Lin, Ryota Tomioka, and Sabine Süsstrunk.  On the loss landscape of adversarial training: Identifying challenges and how to overcome them.  _Advances in Neural Information Processing Systems_ , 33:21476–21487, 2020.
  * Liu et al. (2021) Feng Liu, Bo Han, Tongliang Liu, Chen Gong, Gang Niu, Mingyuan Zhou, Masashi Sugiyama, et al.  Probabilistic margins for instance reweighting in adversarial training.  _Advances in Neural Information Processing Systems_ , 34:23258–23269, 2021.
  * Madry et al. (2017) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  _arXiv preprint arXiv:1706.06083_ , 2017.
  * Mao et al. (2023) Yuhao Mao, Mark Niklas Müller, Marc Fischer, and Martin Vechev.  Taps: Connecting certified and adversarial training.  _arXiv preprint arXiv:2305.04574_ , 2023.
  * Netzer et al. (2011) Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * Pang et al. (2020) Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su.  Boosting adversarial training with hypersphere embedding.  _Advances in Neural Information Processing Systems_ , 33:7779–7792, 2020.
  * Papernot et al. (2017) Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In _Proceedings of the 2017 ACM on Asia conference on computer and communications security_ , pp. 506–519, 2017.
  * Simon-Gabriel et al. (2019) Carl-Johann Simon-Gabriel, Yann Ollivier, Leon Bottou, Bernhard Schölkopf, and David Lopez-Paz.  First-order adversarial vulnerability of neural networks and input dimension.  In _International conference on machine learning_ , pp. 5809–5817. PMLR, 2019.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ , 2013.
  * Uesato et al. (2018) Jonathan Uesato, Brendan O’donoghue, Pushmeet Kohli, and Aaron Oord.  Adversarial risk and the dangers of evaluating against weak attacks.  In _International Conference on Machine Learning_ , pp. 5025–5034. PMLR, 2018.
  * Wang et al. (2019) Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu.  Improving adversarial robustness requires revisiting misclassified examples.  In _International conference on learning representations_ , 2019.
  * Wei et al. (2023) Zeming Wei, Yifei Wang, Yiwen Guo, and Yisen Wang.  Cfa: Class-wise calibrated fair adversarial training.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 8193–8201, 2023.
  * Wu et al. (2020) Dongxian Wu, Shu-Tao Xia, and Yisen Wang.  Adversarial weight perturbation helps robust generalization.  _Advances in Neural Information Processing Systems_ , 33:2958–2969, 2020.
  * Xu et al. (2021) Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang.  To be robust or to be fair: Towards fairness in adversarial training.  In _International conference on machine learning_ , pp. 11492–11501. PMLR, 2021.
  * Zeng et al. (2021) Huimin Zeng, Chen Zhu, Tom Goldstein, and Furong Huang.  Are adversarial examples created equal? a learnable weighted minimax risk for robustness under non-uniform attacks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pp. 10815–10823, 2021.
  * Zhang et al. (2019) Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan.  Theoretically principled trade-off between robustness and accuracy.  In _International conference on machine learning_ , pp. 7472–7482. PMLR, 2019.
  * Zhang et al. (2020) Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli.  Geometry-aware instance-reweighted adversarial training.  In _International Conference on Learning Representations_ , 2020.

##  Appendix A Appendix

###  A.1 Further Experiments

####  A.1.1 Effect of warm-up period on natural accuracy.

In an ideal scenario, larger perturbation budgets typically lead to a
reduction in natural accuracy. However, we addressed this by initiating
training with a smaller perturbation budget ($\epsilon=4/255$) and a step size
of $1/255$ until the 80th epoch. Our experimental results illustrate how this
training approach effectively mitigates the impact on natural accuracy in the
proposed MWPB. Specifically, the experiments presented below were conducted on
WRN-34-10 on CIFAR-10.

Table 6: Studies on AT, MWPB-AT and SDWPB-AT showing the influence of the warming up training before applying reweighting.  Method | Natural | PGD-20 | AA
---|---|---|---
AT | 86.21 | 56.21 | 51.92
MWPB-AT (no warm-up) | 84.39 | 58.71 | 53.87
SDWPB-AT (no warm-up) | 83.09 | 59.19 | 53.84
MWPB-AT (with warm-up) | 86.85 | 59.18 | 54.16
SDWPB-AT (with warm-up) | 86.02 | 59.36 | 54.08

####  A.1.2 Robustness against PGD-20 under various perturbation sizes and FAB
attack

Given that our approach rely on training models with adversarial samples
crafted using adaptive perturbation budget, we study the robustness of the
proposed methods under various $l_{\infty}$-bounded perturbation sizes.
Furthermore, we tested against a strong attack, FAB Croce & Hein (2020a),
which adaptively tracks the decision boundary with the aim of changing the
class of a given input. Our experimental results displayed in Table 7, show
that combining MWPB and SDWPB with AT, TRADES and MART consistently yield
improved robustness to PGD attacks crafted under varied adversarial
perturbations. In addition, our methods improve the robustness of AT, TRADES
and MART respectively to FAB attack.

Table 7: Comparing white-box attack robustness (accuracy %) for WRN-34-10 on CIFAR-10 under various $l_{\infty}$-bounded perturbation sizes for PGD-20 and FAB attack. Defense | PGD-20 | PGD-20 | PGD-20 | FAB |
---|---|---|---|---|---
|  $\epsilon$ = 6/255 |  $\epsilon$ = 10/255 |  $\epsilon$ = 12/255 |  $\epsilon$ = 128/255 |
AT | 64.49 | 49.58 | 45.29 | 52.02 |
MWPB-AT | 67.12 | 52.08 | 48.81 | 54.52 |
MART | 65.41 | 52.31 | 48.84 | 52.18 |
MWPB-MART | 67.19 | 55.63 | 51.96 | 54.25 |
TRADES | 64.19 | 51.92 | 48.23 | 53.59 |
MWPB-TRADES | 65.82 | 53.58 | 50.11 | 55.17 |

###  A.2 Distribution of perturbation radii

We show in Fig. 1, the perturbation radii distribution for MWPB-AT, MWPB-
TRADES, and MWPB-MART for RN-18 over the 50,000 training samples of CIFAR-10.
The perturbation radii distribution is computed on the best-performing epoch
in each case. Experimental results show that MWPB-AT has utilized the minimum
perturbation radii of 0.018 and the maximum perturbation radii of 0.0552.
MWPB-MART and MWPB-TRADES have minimum perturbation radii of 0.0200 and
0.02253, respectively. The maximum perturbation radii of MWPB-MART and MWPB-
TRADES are respectively 0.0554 and 0.0471.

Similarly, we show in Fig. (2) the perturbation radii distribution for SDWPB-
AT, SDWPB-TRADES, and SDWPB-MART, respectively. Our experimental results show
that the adversarial example with the minimum perturbation has radii of 0.0317
and maximum perturbation radii of 0.057 for SDWPB-AT. The adversarial examples
used for training SDWPB-TRADES and SDWPB-MART have minimum perturbations of
0.0313 and 0.0312. The maximum perturbation radii of SDWPB-MART and SDWPB-
TRADES are respectively 0.0485 and 0.0498.

###  A.3 Impact of $\alpha$ hyperparameter.

In this analysis, we investigate the influence of the hyperparameter $\alpha$
in the reweighting functions introduced in Equations (4) and (5). $\alpha$
plays a crucial role in controlling the strength of the reweighting functions.
Our experiments across various combinations of MWPB and SDWPB with AT, TRADES,
and MART reveal a trade-off. When $\alpha$ is set to a low value, natural
accuracies tend to be relatively higher, but robust accuracies are relatively
lower. Conversely, as the value of $\alpha$ increases, robust accuracy on
PGD-20 and Auto-attack improves, but at the expense of natural accuracy. Our
selection of the optimal $\alpha$ values is guided by finding a balance that
ensures a reasonable trade-off between natural and robust accuracy,
particularly in the case of Autoattack robust accuracy. We presents the
experimental results in Tables (8) - (13).

Table 8: Studies on MWPB-AT showing the impact of the $\alpha$ hyperparameter.  $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.10 | 88.93 | 56.25 | 51.67 |
0.20 | 88.56 | 56.85 | 51.88 |
0.30 | 88.29 | 57.80 | 52.58 |
0.40 | 87.72 | 58.11 | 52.85 |
0.50 | 87.40 | 58.78 | 53.64 |
0.58 | 86.82 | 59.18 | 54.16 |
0.70 | 85.55 | 59.12 | 54.14 |
0.85 | 85.11 | 59.39 | 54.02 |
1.2 | 83.75 | 59.69 | 53.50 |
Table 9: Studies on MWPB-TRADES showing the impact of the $\alpha$ hyperparameter.  $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.1 | 86.82 | 58.51 | 53.76 |
0.2 | 86.21 | 58.63 | 54.01 |
0.3 | 86.11 | 58.82 | 54.27 |
0.42 | 86.09 | 59.10 | 54.38 |
0.5 | 85.63 | 58.91 | 54.31 |
0.6 | 85.41 | 58.87 | 54.22 |
0.85 | 84.43 | 58.63 | 54.05 |
Table 10: Studies on MWPB-MART showing the impact of the $\alpha$ hyperparameter.  $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.1 | 88.21 | 59.17 | 52.16 |
0.2 | 87.79 | 59.69 | 52.63 |
0.3 | 87.18 | 60.04 | 52.81 |
0.4 | 86.76 | 60.19 | 53.32 |
0.5 | 86.13 | 60.48 | 53.71 |
0.55 | 85.70 | 60.65 | 53.80 |
0.65 | 85.11 | 60.82 | 53.68 |
Table 11: Studies on SDWPB-AT showing the impact of the $\alpha$ hyperparameter. $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.10 | 88.85 | 56.32 | 51.51 |
0.20 | 88.76 | 56.91 | 51.75 |
0.30 | 88.16 | 57.93 | 52.29 |
0.40 | 87.23 | 58.52 | 52.83 |
0.50 | 86.58 | 58.92 | 53.51 |
0.62 | 86.02 | 59.36 | 54.08 |
0.70 | 85.39 | 59.35 | 53.94 |
0.85 | 84.65 | 59.71 | 53.90 |
1.2 | 82.13 | 59.88 | 53.19 |
Table 12: Studies on SDWPB-TRADES showing the impact of the $\alpha$ hyperparameter.  $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.1 | 86.64 | 58.57 | 53.71 |
0.2 | 86.10 | 58.64 | 53.89 |
0.3 | 85.91 | 58.71 | 54.09 |
0.4 | 85.73 | 58.81 | 54.22 |
0.52 | 85.62 | 58.99 | 54.49 |
0.6 | 84.91 | 58.75 | 54.31 |
0.85 | 84.67 | 58.49 | 54.13 |
Table 13: Studies on SDWPB-MART showing the impact of the $\alpha$ hyperparameter.  $\alpha$ | Natural | PGD-20 | AA |
---|---|---|---|---
0.1 | 87.67 | 59.33 | 52.21 |
0.2 | 86.95 | 59.97 | 52.72 |
0.3 | 86.34 | 60.11 | 52.91 |
0.4 | 86.17 | 60.49 | 53.26 |
0.52 | 85.31 | 60.77 | 53.88 |
0.6 | 84.76 | 60.72 | 53.74 |
Figure 1: Plots showing the distribution of perturbation radii for MWPB-AT,
MWPB-TRADES and MWPB-MART respectively.  Figure 2: Plots showing the
distribution of perturbation radii for SDWPB-AT, SDWPB-TRADES and SDWPB-MART
respectively.

##  Appendix B Appendix

Proof of Theorem 1

The PGD attack utilized for inner maximization is a first-order adversary
Madry et al. (2017); Simon-Gabriel et al. (2019).

######  Lemma 1 (Pang et al. (2020))

Given a loss function $\mathcal{L}$ and under the first-order Taylor
expansion, the solution to the inner maximization:

$\max_{\textbf{x}^{\prime}\in
B_{\epsilon}(\textbf{x})}L(f_{\theta}(\textbf{x}^{\prime}),y)$

is $x^{*}=x+\epsilon\mathbb{U}_{p}({\nabla}_{x}\mathcal{L}(x))$. Furthermore,
$\mathbf{L}(f(x^{*}),y)=\mathcal{L}(f(x),y)+\epsilon{\|{\nabla}_{x}\mathcal{L}(x)\|}_{q}$,
where ${\|.\|}_{q}$ is the dual norm of ${\|.\|}_{p}$

######  Lemma 2 (Katharopoulos & Fleuret (2017))

Let $\mathcal{L}$ be either a negative log-likelihood or square error loss
function. Then,
$\mathcal{L}(f_{\theta}(x_{1}),y_{1})>\mathcal{L}(f_{\theta}(x_{2}),y_{2})$
$\iff$
$\|{\nabla}_{x}\mathcal{L}(f_{\theta}(x_{1}),y_{1})\|>\|{\nabla}_{x}\mathcal{L}(f_{\theta}(x_{2}),y_{2})\|$.

###### Proof 1

We anchor the proof of Theorem 1 on Lemma 1 and Lemma 2.

Given two input-label pairs $(x_{1},y_{1})$ and $(x_{2},y_{2})$, such that
$\mathcal{L}(f_{\theta}(x_{1}),y)>\mathcal{L}(f_{\theta}(x_{2}),y)$. Then
according to Lemma 1, the inner maximization of
$\mathcal{L}(f_{\theta}(x_{1}),y_{1})$ and
$\mathcal{L}(f_{\theta}(x_{2}),y_{2})$ under the same perturbation bound
$\epsilon$ are given as:

$\mathcal{L}(f_{\theta}(x^{*}_{1}),y_{1})=\mathcal{L}(f_{\theta}(x_{1}),y_{1})+\epsilon{\|{\nabla}_{x_{1}}\mathcal{L}f_{\theta}(x_{1},y_{1})\|}_{q}$
$\mathcal{L}(f_{\theta}(x^{*}_{2}),y_{2})=\mathcal{L}(f_{\theta}(x_{2}),y_{2})+\epsilon{\|{\nabla}_{x_{2}}\mathcal{L}f_{\theta}(x_{2},y_{2})\|}_{q}$

It can seen that the increment in loss resulting from the inner maximization
depend on
$\epsilon{\|{\nabla}_{x_{1}}\mathcal{L}(f_{\theta}(x_{1}),y_{2})\|}_{q}$ and
$\epsilon{\|{\nabla}_{x_{2}}\mathcal{L}(f_{\theta}(x_{2}),y_{2})\|}_{q}$.
Since $\epsilon$ is constant then the loss increment depend on norm of the
gradient. Also according to Lemma 2,
${\|{\nabla}_{x}L(f_{\theta}(x_{1}),y_{1})\|}_{q}>{\|{\nabla}_{x}L(f_{\theta}(x_{2}),y_{2})\|}_{q}$.
Therefore under a fixed perturbation radius $\epsilon$,

  1. 1.

$\mathcal{L}(f_{\theta}(x^{*}_{1}),y_{1})$ $>$
$\mathcal{L}(f_{\theta}(x^{*}_{2}),y_{2})$

  2. 2.

$\mathcal{L}(f_{\theta}(x^{*}_{1}),y_{1})-\mathcal{L}(f_{\theta}(x_{1}),y_{1})$
$>$
$\mathcal{L}(f_{\theta}(x^{*}_{2}),y_{2})-\mathcal{L}(f_{\theta}(x_{2}),y_{2})$

Proof of Theorem 2

###### Proof 2

Consider an input-label pair $(x,y)$ and a loss function $\mathcal{L}$ with
the inner maximization

$\max_{\textbf{x}^{\prime}\in
B_{\epsilon}(\textbf{x})}L(f_{\theta}(\textbf{x}^{\prime}),y)$

and the solution

$L(f(x^{*}),y)=L(f(x),y)+\epsilon{\|{\nabla}_{x}L(f_{\theta}(\textbf{x}),y)\|}_{q}$

. Then, given a perturbation radius ${\epsilon}_{g}$ $>$ $\epsilon$, we have:

$L(f(x^{*}),y)=L(f(x),y)+{\epsilon}_{g}{\|{\nabla}_{x}L(f_{\theta}(\textbf{x}),y)\|}_{q}$

. Since $L(f(x),y)$ and
${\|{\nabla}_{x}L(f_{\theta}(\textbf{x}^{\prime}),y)\|}_{q}$ remain fixed, and
${\epsilon}_{g}$ $>$ $\epsilon$, then $L(f(x^{*}),y)$ computed within
${\epsilon}_{g}$ is greater than $L(f(x^{*}),y)$ within $\epsilon$.

##  Appendix C Appendix

Here we present the proposed algorithm for MWPB-AT

Algorithm 1 MWPB-AT Algorithm.

Input: a neural network model with the parameters $\theta$, step sizes
$\kappa_{i}$ and $\kappa$, and a training dataset $\mathcal{D}$ of size n.
Output: a robust model with parameters $\theta^{*}$

1:set $\epsilon=8/255$

2:for $epoch=1$ to num_epochs do

3: for $batch=1$ to num_batchs do

4: sample a mini-batch $\\{(x_{i},y_{i})\\}_{i=1}^{M}$ from
$\mathcal{D}$;$\triangleright$ mini-batch of size $M$.

5: for $i=1$ to M do

6:
$d_{m}(\textbf{x}_{i},y_{i};\theta)=f_{\theta}(\textbf{x}_{i})_{y_{i}}-\max_{k,k\neq{y_{i}}}f_{\theta}(\textbf{x}_{i})_{k}$

7: ${\epsilon}_{i}=exp(\alpha\cdot
d_{m}(\textbf{x}_{i},y_{i};\theta))*\epsilon$

8: ${\kappa}_{i}={\epsilon}_{i}/4$

9: $\mathbf{x}_{i}^{{}^{\prime}}$ $\leftarrow$ $\mathbf{x}_{i}$ \+ 0.001
$\cdot$ $\mathcal{N}(0,1)$; $\triangleright$ $\mathcal{N}(0,I)$ is a Gaussian
distribution with zero mean and identity variance.

10: for $k=1$ to $K$ do

11: if $epoch\leq 80$ then

12:
$\mathbf{x}_{i}^{\prime}\leftarrow\prod_{B_{{\epsilon}/2}(\mathbf{x}_{i})}(x_{i}+{\kappa}/2\cdot
sign(\nabla_{\mathbf{x}_{i}^{\prime}}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i}))$;
$\triangleright$ $\prod$ is a projection operator.

13: else

14:
$\mathbf{x}_{i}^{\prime}\leftarrow\prod_{B_{{\epsilon}_{i}}(\mathbf{x}_{i})}(x_{i}+{\kappa}_{i}\cdot
sign(\nabla_{\mathbf{x}_{i}^{\prime}}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i}))$

15: end if

16: end for

17: end for

18: $\theta$ $\leftarrow$
$\theta-\kappa\nabla_{\theta}\sum_{i=1}^{M}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i})$

19: end for

20:end for

Algorithm 2 SDWPB-AT Algorithm.

Input: a neural network model with the parameters $\theta$, step sizes
$\kappa_{i}$ and $\kappa$,a training dataset $\mathcal{D}$ of size n and $|C|$
is the number of classes.
Output: a robust model with parameters $\theta^{*}$

1:set $\epsilon=8/255$

2:for $epoch=1$ to num_epochs do

3: for $batch=1$ to num_batchs do

4: sample a mini-batch $\\{(x_{i},y_{i})\\}_{i=1}^{M}$ from
$\mathcal{D}$;$\triangleright$ mini-batch of size $M$.

5: for $i=1$ to M do

6:
$d_{std}(\textbf{x}_{i},y_{i};\theta)=\\{\sum_{k=1}^{C}\frac{(f_{\theta}(\textbf{x}_{i})_{k}-f_{\theta}(\textbf{x}_{i})_{y_{i}})^{2})}{|C|}\\}^{0.5}$

7: ${\epsilon}_{i}=exp(\alpha\cdot
d_{std}(\textbf{x}_{i},y_{i};\theta))*\epsilon$

8: ${\kappa}_{i}={\epsilon}_{i}/4$

9: $\mathbf{x}_{i}^{{}^{\prime}}$ $\leftarrow$ $\mathbf{x}_{i}$ \+ 0.001
$\cdot$ $\mathcal{N}(0,1)$; $\triangleright$ $\mathcal{N}(0,I)$ is a Gaussian
distribution with zero mean and identity variance.

10: for $k=1$ to $K$ do

11: if $epoch\leq 80$ then

12:
$\mathbf{x}_{i}^{\prime}\leftarrow\prod_{B_{{\epsilon}/2}(\mathbf{x}_{i})}(x_{i}+{\kappa}/2\cdot
sign(\nabla_{\mathbf{x}_{i}^{\prime}}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i}))$;
$\triangleright$ $\prod$ is a projection operator.

13: else

14:
$\mathbf{x}_{i}^{\prime}\leftarrow\prod_{B_{{\epsilon}_{i}}(\mathbf{x}_{i})}(x_{i}+{\kappa}_{i}\cdot
sign(\nabla_{\mathbf{x}_{i}^{\prime}}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i}))$

15: end if

16: end for

17: end for

18: $\theta$ $\leftarrow$
$\theta-\kappa\nabla_{\theta}\sum_{i=1}^{M}L(f_{\theta}(\mathbf{x}_{i}^{\prime}),y_{i})$

19: end for

20:end for