In this study, we introduce a novel, probabilistic viewpoint on adversarial examples, achieved through box-constrained Langevin Monte Carlo (LMC). Proceeding from this perspective, we develop an innovative approach for generating semantics-aware adversarial examples in a principled manner. This methodology transcends the restriction imposed by geometric distance, instead opting for semantic constraints. Our approach empowers individuals to incorporate their personal comprehension of semantics into the model. Through human evaluation, we validate that our semantics-aware adversarial examples maintain their inherent meaning. Experimental findings on the MNIST and SVHN datasets demonstrate that our semantics-aware adversarial examples can effectively circumvent robust adversarial training methods tailored for traditional adversarial attacks.

§ INTRODUCTION

The purpose of generating adversarial examples is to deceive a classifier by making minimal changes to the original data's meaning. In image classification, most existing adversarial techniques ensure the preservation of adversarial example semantics by limiting their geometric distance from the original image [Szegedy et al., 2013, Goodfellow et al., 2014, Carlini and Wagner, 2017, Madry et al., 2017]. These methods are able to deceive classifiers with a very small geometric based perturbation. However, when targeting robust classifiers trained using adversarial methods, an attack involving a relatively large geometric distance may be necessary. Unfortunately, these considerable distances can be so vast that they ultimately undermine the original image's semantics, going against the core objective of creating adversarial examples. As illustrated in the left portion of Figure <ref>, when applying the PGD attack [Madry et al., 2017] constrained by $L_2$ norm on a robust classifier, the attacked images that successfully deceive the classifier consistently lose their original meaning, which is undesirable.

To counter this problem, we propose an innovative approach for generating semantics-aware adversarial examples. Instead of being limited by geometric distance, our approach hinges on a proposed semantic divergence. Specifically, we treat generating adversarial examples as a box-constrained non-convex optimization problem. We employ box-constrained Langevin Monte Carlo (LMC) to find near-optimal solutions for this complex problem. As LMC samples converge to a stationary distribution, we gain a probabilistic understanding of the adversarial attack. Within this probabilistic perspective, the geometric constraint of the adversarial attack can be viewed as a distribution. By replacing this geometric-based distribution with a semantic-based distribution, we can define a semantics-aware adversarial attack in a principled manner. The corresponding divergence induced by the semantic-based distribution is called semantic divergence. Our semantics-aware adversarial attack is capable of deceiving robust classifiers while preserving most of the original image's semantics, as demonstrated in the right section of Figure <ref>.

Top left: Targeted attack on an adversarially trained MadryNet [Madry et al., 2017] for MNIST using Projected Gradient Descent (PGD) with $L_2$ norm. To ensure successful targeted attacks in most cases, we increased the $\epsilon$ to $5$. Bottom left: Targeted attack on an adversarially trained ResNet18 [He et al., 2016] for SVHN using PGD with $L_2$ norm and $\epsilon=5$. Top right & Bottom right: Our proposed method applied to targeted attacks on the same MadryNet and ResNet18 for MNIST and SVHN, respectively. A green border signifies a successful deception of the victim classifier, while a red border indicates failure. Notably, with PGD, a successful attack often results in the alteration of the source image's semantics, which is undesirable. Additional PGD attack examples are provided in Appendix <ref>.

§ PRELIMINARIES

§.§ Adversarial examples

The notion of adversarial examples was first introduced by Szegedy et al., 2013. Let's assume we have a classifier $C: [0, 1]^n \rightarrow \mathcal{Y}$, where $n$ represents the dimension of the input space and $\mathcal{Y}$ denotes the label space. Given an image $\xori \in [0, 1]^n$ and a target label $\ytar\in\mathcal{Y}$, the optimization problem for finding an adversarial instance for $\xori$ can be formulated as follows:

\begin{equation*}
\text{minimize } \mathcal{D}(\xori, \xadv) \quad \text{ such that } C(\xadv) = \ytar \text{ and } \xadv \in [0, 1]^n
\end{equation*}

Here, $\mathcal{D}$ is a distance metric employed to assess the difference between the original and perturbed images. This distance metric typically relies on geometric distance, which can be represented by $L_0$, $L_2$, or $L_\infty$ norms.

However, solving this problem is challenging. As a result,  Szegedy et al., 2013 propose a relaxation of the problem:

\begin{equation}
\label{eq:finalopti}
\text{minimize } \mathcal{L}(\xadv, \ytar):=c_1\cdot \mathcal{D}(\xori, \xadv) + c_2\cdot f(\xadv, \ytar)\quad \text{ such that } \xadv \in [0,1]^n
\end{equation}

where $c_1$, $c_2$ are constants, and $f$ is an objective function closely tied to the classifier's prediction. For example, in [Szegedy et al., 2013], $f$ is the cross-entropy loss function, while Carlini and Wagner, 2017 suggest several different choices for $f$. Szegedy et al., 2013 recommend solving (<ref>) using box-constrained L-BFGS.

§.§ Adversarial training

Adversarial training, a widely acknowledged method for boosting adversarial robustness in deep learning models, has been extensively studied [Szegedy et al., 2013, Goodfellow et al., 2014, Huang et al., 2015, Madry et al., 2017]. This technique uses adversarial samples as (part of) the training data, originating from Szegedy et al., 2013, and has evolved into numerous variations. In this paper, we apply the min-max problem formulation by Madry et al., 2017 to determine neural network weights, denoted as $\theta$. They propose choosing $\theta$ to solve:

\begin{equation}
\label{eq:advtrain}
\min_\theta \mathbb{E}_{(\x,y)\sim \pdata}\left[\max_{\lVert \delta \rVert_p\leq \epsilon } \mathcal{L}_\text{CE} (\theta, \x + \delta, y) \right]
\end{equation}

where $\pdata$ represents the data distribution, $\mathcal{L}_\text{CE}$ is the cross-entropy loss, $\lVert \cdot \rVert_p$ denotes the $L_p$ norm, and $\epsilon$ specifies the radius of the corresponding $L_p$ ball. In what follows, we will use the term “robust classifier” to refer to classifiers that have undergone adversarial training.

§.§ Energy-based models (EBMs)

An Energy-based Model (EBM) [Hinton, 2002, Du and Mordatch, 2019] involves a non-linear regression function, represented by $E_\theta$, with a parameter $\theta$. This function is known as the energy function. Given a data point, $\mathbf{x}$, the probability density function (PDF) is given by:

\begin{equation}
\label{eqn:ebmdensity}
p_\theta(\mathbf{x}) = \frac{\exp(-E_\theta(\mathbf{x}))}{Z_\theta}
\end{equation}

where $Z_\theta = \int \exp (-E_\theta(\mathbf{x})) \mathrm{d}\mathbf{x}$ is the normalizing constant that ensures the PDF integrates to $1$.

§.§ Langevin Monte Carlo (LMC)

Langevin Monte Carlo (also known as Langevin dynamics) is an iterative method that could be used to find near-minimal points of a non-convex function $g$ [Raginsky et al., 2017, Zhang et al., 2017, Tzen et al., 2018, Roberts and Tweedie, 1996]. It involves updating the function as follows:

\begin{equation}
\label{eq:ld}
\mathbf{x}_0\sim p_0,\quad \xtpp = \xt - \frac{\epsilon^2}{2}\nabla_x g(\xt) + \epsilon \mathbf{z}_t,\quad \mathbf{z}_t\sim \mathcal{N}(0, I)
\end{equation}

where $p_0$ could be a uniform distribution. Under certain conditions on the drift coefficient $\nabla_x g$, it has been demonstrated that the distribution of $\xt$ in (<ref>) converges to its stationary distribution [Chiang et al., 1987, Roberts and Tweedie, 1996], also referred to as the Gibbs distribution $p(\mathbf{x})\propto \exp (g(\mathbf{x}))$. This distribution concentrates around the global minimum of $g$[Gelfand and Mitter, 1991, Xu et al., 2018, Roberts and Tweedie, 1996]. If we choose $g$ to be $-E_\theta$, then the stationary distribution corresponds exactly to the EBM's distribution defined in (<ref>). As a result, we can draw samples from the EBM using LMC. By replacing the exact gradient with a stochastic gradient, we obtain Stochastic Gradient Langevin Dynamics (SGLD) [Welling and Teh, 2011, Teh et al., 2016].

§.§ Training EBM

To train an EBM, we aim to minimize the minus expected log-likelihood of the data , represented by

\begin{equation*}
\mathcal{L}_\text{EBM} = \mathbb{E}_{X\sim p_d}[-\log p_\theta (X)] = \mathbb{E}_{X\sim p_d}[E_\theta(X)] - \log Z_\theta
\end{equation*}

where $p_d$ is the data distribution. The gradient is

\begin{equation}
\label{eq:trainebm}
\nabla_\theta \mathcal{L}_\text{EBM} = \mathbb{E}_{X\sim p_d}[ \nabla_\theta E_\theta(X)] - \nabla_\theta \log Z_\theta = \mathbb{E}_{X\sim p_d}[\nabla_\theta E_\theta(X)] - \mathbb{E}_{X\sim p_\theta}[\nabla_\theta E_\theta(X)]
\end{equation}

(see [Song and Kingma, 2021] for derivation). The first term of $\nabla_\theta \mathcal{L}_\text{EBM}$ can be easily calculated as $p_d$ is the distribution of the training set. For the second term, we can use LMC to sample from $p_\theta$ [Hinton, 2002].

Effective training of an energy-based model (EBM) typically requires the use of techniques such as sample buffering and regularization. For more information, refer to the work of Du and Mordatch, 2019.

§ GENERATING SEMANTICS-AWARE ADVERSARIAL EXAMPLES

In this section, we introduce a probabilistic approach to understanding adversarial examples. Through this lens, we establish the concept of semantic divergence, offering an alternative to conventional geometric distance. This concept of semantic divergence enables individuals to integrate their unique understanding of semantics into the model, thereby facilitating the creation of semantics-aware adversarial examples.

(a) and (b) display samples drawn from $\pvic(\cdot; \ytar)$ with the victim classifier being non-adversarially trained and adversarially trained, respectively. (c) showcases samples from $\pdis(\cdot; \xori)$ when $\mathcal{D}$ is the square of $L_2$ norm. (d) illustrates $t(\xori)$ for $t \sim \mathcal{T}$, where $\mathcal{T}$ represents a distribution of transformations, including TPS (see Section <ref>), scaling, rotation, and cropping. The $\xori$s in (c) and (d) consist of the first 36 images from the MNIST test set.

§.§ A probabilistic perspective on adversarial examples

LMC and SGLD are not directly applicable to the optimization problem presented in (<ref>) due to their incompatibility with box-constrained optimization problems. To overcome this limitation, Lamperski, 2021 proposed Projected Stochastic Gradient Langevin Algorithms (PSGLA). By employing PSGLA to generate samples near the solution of the optimization problem specified in (<ref>), we obtain the subsequent update rule:

\begin{equation}
\label{eq:advlange}
\mathbf{x}_0\sim p_0,\quad \xtpp = \Pi_{[0,1]^n}\left(\xt - \frac{\epsilon^2}{2}\nabla_x \mathcal{L}(\xt, \ytar) + \epsilon\mathbf{z}_t \right),\quad \mathbf{z}_t\sim \mathcal{N}(0, I)
\end{equation}

where $\Pi{[0,1]^n}$ is a clamp projection that enforces the constraints within the $[0,1]^n$ interval. We refer to the stationary distribution of PSGLA as the adversarial distribution $\padv(\mathbf{x}; \ytar)\propto \exp (-\mathcal{L}(\mathbf{x}, \ytar))$, since samples drawn from this distribution are in close proximity to the optimal value of the optimization problem presented in (<ref>).

Then by definition of $\mathcal{L}$, the adversarial distribution can be represented as a product of expert distributions [Hinton, 2002]:

\begin{equation}
\label{eq:padv}
\padv(\xadv; \xori, \ytar) \propto \pvic(\xadv; \ytar)\pdis(\xadv;\xori)
\end{equation}

where $\pvic(\xadv; \ytar)\propto \exp(-c_2\cdot f(\xadv, \ytar))$ denote the victim distribution and $\pdis(\xadv;\xori)\propto \exp(-c_1\cdot\mathcal{D}(\xori, \xadv))$ represent the distance distribution.

The victim distribution $\pvic$ is dependent on the victim classifier. As suggested by Szegedy et al., 2013, $f$ could be the cross-entropy loss of the classifier. We can sample from this distribution using Langevin dynamics. Figure <ref>(a) presents samples drawn from $\pvic$ when the victim classifier is subjected to standard training, exhibiting somewhat indistinct shapes of the digits. This implies that the classifier has learned the semantics of the digits to a certain degree, but not thoroughly. In contrast, Figure <ref>(b) displays samples drawn from $\pvic$ when the victim classifier undergoes adversarial training. In this scenario, the shapes of the digits are clearly discernible. This observation suggests that we can obtain meaningful samples from adversarially trained classifiers, indicating that such classifiers depend more on semantics, which corresponds to the fact that an adversarially trained classifier is more difficult to attack. A similar observation concerning the generation of images from an adversarially trained classifier has been reported by Santurkar et al., 2019.

The distance distribution $\pdis$ relies on $\mathcal{D}(\xori, \xadv)$, representing the distance between $\xadv$ and $\xori$. By its nature, samples that are closer to $\xori$ may yield a higher $\padv$, which is consistent with the objective of generating adversarial samples. Moreover, if $\mathcal{D}$ represents the square of the $L_2$ norm, then $\pdis$ becomes a Gaussian distribution with a mean of $\xori$ and a variance determined by $c_1$. Figure <ref>(c) portrays samples drawn from $\pdis$ when $\mathcal{D}$ is the square of the $L_2$ distance. The samples closely resemble the original images, $\xori$s, from the MNIST testset, because each sample is positioned near an optimal point, and these optimal points are the original images, $\xori$s.

§.§ From Geometric Distance to Semantic Divergence

Based on the probabilistic perspective, we propose a semantic divergence, denoted by a non-symmetric divergence $\mathcal{D}_\text{sem}(\xadv, \xori):=E(\xadv; \xori)$, where $E(\cdot; \xori)$ represents the energy of an energy-based model trained on a dataset consisting of $\{t_1(\xori), t_2(\xori), \dots \}$. Here, $t_i\sim \mathcal{T}$, and $\mathcal{T}$ is a distribution of transformations that do not alter the original image's semantics. In practice, the choice of $\mathcal{T}$ depends on human subjectivity related to the dataset. Individuals are able to incorporate their personal comprehension of semantics into the model by designing their own $\mathcal{T}$. For instance, in the case of the MNIST dataset, the transformations could include scaling, rotation, distortion, and cropping, as illustrated in Figure <ref>(d). We assume that such transformations do not affect the semantics of the digits in the MNIST dataset. Consequently, our proposed semantic divergence induces the corresponding distance distribution $\pdis (\xadv; \xori)\propto \exp(-c_1\cdot E(\xadv; \xori))$.

We claim that, given an appropriate $\mathcal{T}$, semantic divergence can surpass geometric distance. Empirically, maintaining the semantics of the original image by limiting the geometric distance between the adversarial image and the original image when deceiving a robust classifier is challenging: as shown in Figure <ref> and Figure <ref>, it is difficult to preserve the semantics of the original images. The attacked images either display a `shadow' of the target digits or reveal conspicuous tampering traces, such as in Figure <ref>(c), where the attacked digit turns gray. This phenomenon was empirically observed and tested by Song et al., 2018 through an A/B test. Conversely, as depicted in Figure <ref>, the samples from $\padv$ neither exhibit the `shadow' of the target digits nor any obvious traces indicating adversarial attack. While semantic divergence can't entirely prevent the generation of a sample resembling the target class, as shown in Figure <ref>(a), we discuss certain techniques to mitigate this issue in Section <ref>.

A plausible explanation for this is that the utilization of geometric distance causes $\pdis(\cdot, \xori)$ to overly focus on $\xori$. However, when applying semantic divergence induced by a suitable $\mathcal{T}$, the density of the distance distribution $\pdis(\cdot, \xori)$ spreads out relatively more, resulting in a higher overlap between $\pdis(\cdot, \xori)$ and $\pvic$. This, in turn, provides more opportunities for their product $\padv$ to reach a higher value.

(a): Samples from $\padv(\cdot; \xori, \ytar)\propto \exp(-c_1\cdot\mathcal{D}(\xori, \xadv))\exp(-c_2\cdot f(\xadv, \ytar))$, where $\mathcal{D}$ is the $L_2$ norm, $f$ is the cross-entropy $f_\text{CE}$, $\xori$ are the first 36 images from the MNIST test set, $\ytar$ are set to 1, $c_1$ is $10^{-3}$, and $c_2$ is $10^{-2}$. (c): Similar to (a), but with $f$ replaced by $f_\text{CW}$, as introduced in section <ref>. Essentially, this case applies the $L_2$ CW attack [Carlini and Wagner, 2017] using LMC instead of Adam optimization. A green border indicates successful deception of the victim classifier, while a red border signifies failure. (b) & (d): the predictive probability (softmax probability) of the target class, corresponding to each digit of Figures (a) and (c) on a one-to-one basis.

§ DECEIVING ROBUST CLASSIFIERS

In this section, we present several techniques that enhance the performance of our proposed method in generating high-quality adversarial examples.

§.§ Victim distributions

The victim distribution $\pvic\propto \exp(c_2\cdot f(\xadv, \ytar))$ is influenced by the choice of function $f$. Let $g_\phi:[0,1]^n\rightarrow \mathbb{R}^{|\mathcal{Y}|}$ be a classifier that produces logits as output with $\phi$ representing the neural network parameters, $n$ denoting the dimensions of the input, and $\mathcal{Y}$ being the set of labels (the output of $g_\phi$ are logits). Szegedy et al., 2013 suggested using cross-entropy as the function $f$, which can be expressed as

\begin{equation*}
f_\text{CE}(\x, \ytar):=-g_\phi(\x)[\ytar] + \log\sum_y\exp(g_\phi(\x)[y])=-\log \sigma(g_\phi(\x))[\ytar]
\end{equation*}

where $\sigma$ denotes the softmax function.

Carlini and Wagner, 2017 explored and compared multiple options for $f$. They found that, empirically, the most efficient choice of their proposed $f$s is:

\begin{equation*}
f_\text{CW}(\x, \ytar):=\max(\max_{y\neq \ytar}g_\phi(\x)[y] - g_\phi(\x)[\ytar], 0).
\end{equation*}

From Figure <ref> and Figure <ref>, we observe that $f_\text{CW}$ outperforms $f_\text{CE}$ when the $\pdis$ depends on either geometric distance or semantic divergence. A potential explanation for this phenomenon is that, according to its definition, $f_\text{CW}$ becomes $0$ if the classifier is successfully deceived during the iteration process. This setting ensures that the generator does not strive for a relatively high softmax probability for the target class; it simply needs to reach a point where the victim classifier perceives the image as belonging to the target class. Consequently, after the iteration, the victim classifier assigns a relatively low predictive probability to the target class $\sigma(g_\phi(\xadv))[\ytar]$, as demonstrated in Figure <ref>(d) and Figure <ref>(d).

In this study, we introduce two additional choices for the function $f$. Although these alternatives are not as effective as $f_\text{CW}$, we present them in Appendix <ref> for further exploration.

(a) & (c): Samples from $\padv(\cdot; \xori, \ytar)\propto \exp(-c_1\cdot\mathcal{D}(\xori, \xadv))\exp(-c_2\cdot f(\xadv, \ytar))$, where $\xori$ refers to the original image of digit “7” shown in Figure <ref> and $\ytar$ refers to class 9. $\mathcal{D}$ represents our proposed semantic divergence. In (a), $f$ is the cross-entropy $f_\text{CE}$, while in (c), $f$ is $f_\text{CW}$. Constants are set as $c_1=1.0$ and $c_2=10^{-2}$. A green border indicates successful deception of the victim classifier, whereas a red border denotes failure. (b) & (d): The predictive probability (softmax probability) of the target class, corresponding to each digit in Figures (a) and (c) on a one-to-one basis.

TPS as a data augmentation. Left: The original image $\xori$ superimposed with a $5\times 5$ grid of source control points $\mathcal{P}_\text{sou}$. Right: The transformed image overlaid with a grid of target control points $\mathcal{P}_\text{tar}$.

§.§ Data Augmentation by Thin Plate Splines (TPS) Deformation

Thin-plate-spline (TPS) [Bookstein, 1989] is a commonly used image deforming method. Given a pair of control points and target points, TPS computes a smooth transformation that maps the control points to the target points, minimizing the bending energy of the transformation. This process results in localized deformations while preserving the overall structure of the image, making TPS a valuable tool for data augmentation.

As introduced in Section <ref>, we aim to train an energy-based model on transformations of a single image $\xori$. In practice, if the diversity of the augmentations of $\xori$, represented as $t(\xori)$, is insufficient, the training of the probabilistic generative model is prone to overfitting. To address this issue, we use TPS as a data augmentation method to increase the diversity of $t(\xori)$. For each $\xori$, we set a $5\times 5$ grid of source control points, $\mathcal{P}_\text{sou} = \{(x^{(i)}, y^{(i)})\}_{i=1}^{5\times 5}$, and defining the target points as $\mathcal{P}_\text{tar} = \{(x^{(i)} + \epsilon_x^{(i)}, y^{(i)} + \epsilon_y^{(i)})\}_{i=1}^{5\times 5}$, where $\epsilon_x^{(i)}, \epsilon_y^{(i)}\sim \mathcal{N}(0, \sigma^2)$ are random noise added to the source control points. We then apply TPS transformation to $\xori$ with $\mathcal{P}_\text{sou}$ and $\mathcal{P}_\text{tar}$ as its parameters. This procedure is depicted in Figure <ref>. By setting an appropriate $\sigma$, we can substantially increase the diversity of the one-image dataset while maintaining its semantic content.

§.§ Rejection Sampling

Directly sampling from $\padv(\cdot; \xori, \ytar)$ does not guarantee the generation of samples capable of effectively deceiving the classifier. To overcome this issue, we adopt rejection sampling [Von Neumann, 1951], which eliminates unsuccessful samples and ultimately yields samples from $\padv(\xadv|\arg\max_y g_\phi(\xadv)[y] = \ytar; \xori, \ytar)$.

§.§ Sample Refinement

After rejection sampling, the samples are confirmed to successfully deceive the classifier. However, not all of them possess high visual quality, as demonstrated in Figure <ref>(c). To automatically obtain $N$ semantically valid samples[In practice, we could select adversarial samples by hand, but we focus on automatic selection here.], we first generate $M$ samples from the adversarial distribution. Following rejection sampling, we sort the remaining samples and select the top $\kappa$ percent based on the softmax probability of the original image's class, as determined by an auxiliary classifier. Finally, we choose the top $N$ samples with the lowest energy $E$, meaning they have the highest likelihood according to the energy-based model.

The auxiliary classifier is trained on the data-augmented training set. We do not use the energy of the samples as the sole criterion for selection because some low-visual quality samples may also have a high likelihood. This occurrence is further explained and examined in Appendix <ref>. The entire process of rejection sampling and sample refinement is portrayed in Algorithm <ref>.

Rejection Sampling and Sample Refinement

A trained energy based model $E(\cdot; \xori)$ based on the original image $\xori$, the victim classifier $g_\phi$, an auxiliary classifier $g_\psi$, number of initial samples $M$, number of final samples $N$, the percentage $\kappa$.
$N$ adversarial samples $\x$.
$\x = \emptyset$
$0\leq i< M$
$\xadv\sim \padv(\cdot; \xori, \ytar)$ Sample from the adversarial distribution.
$\arg\max_y g_\phi (\xadv)[y] = \ytar$ Accept if $\xadv$ deceive the classifier.
$\x = \x\cup \{\xadv\}$
Sort $\x$ by $\sigma(g_\psi (\x_i))[\yori]$ for $i\in\{1,\dots, |\x|\}$ in descent order
$\x = (\x_i)_{i=1}^{\lfloor \kappa |\x| \rfloor}$ Select the first $\kappa$ percent elements from $\x$.
Sort $\x$ by $E(\x_i; \xori)$ for $i\in\{1,\dots, |\x|\}$ in ascent order
$\x = (\x_i)_{i=1}^{N}$ Select the first $N$ elements from $\x$.

§ EXPERIMENT

§.§ Implementation

We implemented our proposed semantics-aware adversarial attack on two datasets: MNIST and SVHN. For the MNIST dataset, the victim classifier we used was an adversarially trained MadryNet [Madry et al., 2017]. For the SVHN dataset, we utilized an adversarially trained ResNet18, in accordance with the methodology outlined by Song et al., 2018. On the distance distribution side, for every original image denoted as $\xori$, we trained an energy-based model on the training set, which is represented as $\{t_1(\xori), t_2(\xori), \dots\}$. In this case, $t_i$ follows a distribution of transformations, $\mathcal{T}$, that do not change the semantics of $\xori$. For the MNIST dataset, we characterized $\mathcal{T}_\text{MNIST}$ as including Thin Plate Spline (TPS) transformations, scaling, and rotation. For the SVHN dataset, we defined $\mathcal{T}_\text{SVHN}$ as comprising Thin Plate Spline (TPS) transformations and alterations in brightness and hue. Detailed specifics related to our implementation can be found in Appendix <ref>.

The success rates (%) of our targeted unrestricted adversarial attack. Corresponding sample examples for each grid are depicted in the top right and bottom right sections of Figure <ref>. Refer to Table <ref> for overall success rate.

§.§ Evaluation

Our method generates adversarial samples that can deceive classifiers, but it does not guarantee the preservation of the original label's semantic meaning. As such, we consider an adversarial example successful if human annotators perceive it as having the same meaning as the original label, in line with the approach by Song et al., 2018. To enhance the signal-to-noise ratio, we assign the same image to five different annotators and use the majority vote as the human decision, as done in [Song et al., 2018]. The screenshot of the annotator's interface is in Appendix <ref>.

In detail, we begin with an original image $\xori$, its label $\yori$, and a target class $\ytar$. We draw $M=2000$ samples from $\padv(\cdot; \xori, \ytar)$, rejecting those that fail to deceive the victim classifier. After sample refinement, we obtain $N=100$ adversarial examples, $\xadv^{(i)}$ for $i\in\{1,\dots, N\}$. We express the human annotators' decision as function $h$ and derive the human decision $\yhum^{(i)} = h(\xadv^{(i)})$. As previously mentioned, an adversarial example $\xadv^{(i)}$ is considered successful if $\yhum^{(i)}$ is equal to $\yori$. We then compute the success rate $s$ as follows:

\begin{equation*}
s = \frac{\sum_{i=1}^{N} \mathbbm{1}(\yhum^{(i)} = \yori)}{N}
\end{equation*}

where $\mathbbm{1}$ represents the indicator function.

We randomly select 10 digits, each representing a different class, from the MNIST/SVHN test set to serve as the original image $\xori$. These are depicted on the left side of Figure <ref>. For each $\xori$, we iterate through the target class $\ytar$ ranging from 0 to 9, excluding the class $\yori$ that signifies the ground-truth label of $\xori$. As previously described, for every pair of $\xori$ and $\ytar$, we generate $N=100$ adversarial examples post sample refinement. The result of each pair is illustrated in Figure <ref>. The overall success rate is illustrated in Figure <ref>.

§.§ Results

As depicted in Figure <ref> and Table <ref>, our proposed method often succeeds in fooling robust classifiers, all the while preserving the original semantics of the input. It should be noted, however, that this does not occur in every instance.

Success rate comparison between the method proposed by Song et al., 2018 and ours. The results presented in this table are for reference only, as Song's results are taken directly from their paper, and we did not use the same group of annotators for our evaluation.

2*Robust Classifier      Success Rate of      2*Our Success Rate

Song et al., 2018

MadryNet [Madry et al., 2017] on MNIST     85.2     96.2

ResNet18 [He et al., 2016] (adv-trained) on SVHN     84.2     86.3

§ RELATED WORK

Unrestricted adversarial examples
Song et al., 2018 proposed generating unrestricted adversarial examples from scratch using conditional generative models. In their work, the term “unrestricted” indicates that the generated adversarial samples, $\xadv$, are not restricted by a geometric distance such as the $L_2$ norm or $L_\infty$ norm. The key difference between their approach and ours is that their adversarial examples $\xadv$ are independent of any specific $\xori$, while our model generates $\xadv$ based on a given $\xori$. By slightly modifying (<ref>), we can easily incorporate Song's “unrestricted adversarial examples” into our probabilistic perspective:

\begin{equation}
\label{eq:padvunres}
\padv(\xadv; \ysou, \ytar) := \pvic(\xadv; \ytar)\pdis(\xadv;\ysou)
\end{equation}

where $\ysou$ is the source class. It becomes evident that the adversarial examples generated by our $\padv(\cdot; \xori, \ytar)$ adhere to Song's definition when $\xori$ is labeled as $\ysou$.

TPS as a Data Augmentation Technique To the best of our knowledge, Vinker et al., 2021 were the first to employ TPS as a data augmentation method. They utilized TPS as a data augmentation strategy in their generative model for conditional image manipulation based on a single image.

§ LIMITATION

This work's foremost limitation pertains to the inherent difficulties in training energy-based models (EBMs), as underscored in the earlier studies by Du and Mordatch, 2019 and Grathwohl et al., 2019. The EBM training process is notoriously challenging, and a notable gap persists between the generation quality of EBMs and that of other widely-used probabilistic generative models, such as variational autoencoders and diffusion models. Consequently, we are currently unable to generate adversarial samples for images with higher resolution.

§ CONCLUSION

In this work, we present a probabilistic perspective on adversarial examples by employing Langevin Monte Carlo. Building on this probabilistic perspective, we introduce semantic divergence as an alternative to the commonly used geometric distance. We also propose corresponding techniques for generating semantically-aware adversarial examples. Human participation experiments indicate that our proposed method can often deceive robust classifiers while maintaining the original semantics of the input, although not in all cases.

[Bookstein, 1989]
F. L. Bookstein.
Principal warps: Thin-plate splines and the decomposition of
IEEE Transactions on pattern analysis and machine
intelligence, 110 (6):0 567–585, 1989.

[Carlini and Wagner, 2017]
N. Carlini and D. Wagner.
Towards evaluating the robustness of neural networks.
In 2017 ieee symposium on security and privacy (sp), pages
39–57. Ieee, 2017.

[Chiang et al., 1987]
T.-S. Chiang, C.-R. Hwang, and S. J. Sheu.
Diffusion for global optimization in rn.
SIAM Journal on Control and Optimization, 250
(3):0 737–753, 1987.

[Du and Mordatch, 2019]
Y. Du and I. Mordatch.
Implicit generation and generalization in energy-based models.
arXiv preprint arXiv:1903.08689, 2019.

[Gelfand and Mitter, 1991]
S. B. Gelfand and S. K. Mitter.
Recursive stochastic algorithms for global optimization in rd.
SIAM Journal on Control and Optimization, 290
(5):0 999–1018, 1991.

[Goodfellow et al., 2014]
I. J. Goodfellow, J. Shlens, and C. Szegedy.
Explaining and harnessing adversarial examples.
arXiv preprint arXiv:1412.6572, 2014.

[Grathwohl et al., 2019]
W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and
K. Swersky.
Your classifier is secretly an energy based model and you should
treat it like one.
arXiv preprint arXiv:1912.03263, 2019.

[He et al., 2016]
K. He, X. Zhang, S. Ren, and J. Sun.
Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 770–778, 2016.

[Hinton, 2002]
G. E. Hinton.
Training products of experts by minimizing contrastive divergence.
Neural computation, 140 (8):0 1771–1800,

[Huang et al., 2015]
R. Huang, B. Xu, D. Schuurmans, and C. Szepesvári.
Learning with a strong adversary.
arXiv preprint arXiv:1511.03034, 2015.

[Lamperski, 2021]
A. Lamperski.
Projected stochastic gradient langevin algorithms for constrained
sampling and non-convex learning.
In Conference on Learning Theory, pages 2891–2937. PMLR,

[Madry et al., 2017]
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu.
Towards deep learning models resistant to adversarial attacks.
arXiv preprint arXiv:1706.06083, 2017.

[Raginsky et al., 2017]
M. Raginsky, A. Rakhlin, and M. Telgarsky.
Non-convex learning via stochastic gradient langevin dynamics: a
nonasymptotic analysis.
In Conference on Learning Theory, pages 1674–1703. PMLR,

[Roberts and Tweedie, 1996]
G. O. Roberts and R. L. Tweedie.
Exponential convergence of langevin distributions and their discrete
Bernoulli, pages 341–363, 1996.

[Santurkar et al., 2019]
S. Santurkar, A. Ilyas, D. Tsipras, L. Engstrom, B. Tran, and A. Madry.
Image synthesis with a single (robust) classifier.
Advances in Neural Information Processing Systems, 32, 2019.

[Song and Kingma, 2021]
Y. Song and D. P. Kingma.
How to train your energy-based models.
arXiv preprint arXiv:2101.03288, 2021.

[Song et al., 2018]
Y. Song, R. Shu, N. Kushman, and S. Ermon.
Constructing unrestricted adversarial examples with generative
Advances in Neural Information Processing Systems, 31, 2018.

[Szegedy et al., 2013]
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and
R. Fergus.
Intriguing properties of neural networks.
arXiv preprint arXiv:1312.6199, 2013.

[Teh et al., 2016]
Y. W. Teh, A. H. Thiery, and S. J. Vollmer.
Consistency and fluctuations for stochastic gradient langevin
Journal of Machine Learning Research, 17, 2016.

[Tzen et al., 2018]
B. Tzen, T. Liang, and M. Raginsky.
Local optimality and generalization guarantees for the langevin
algorithm via empirical metastability.
In Conference On Learning Theory, pages 857–875. PMLR, 2018.

[Vinker et al., 2021]
Y. Vinker, E. Horwitz, N. Zabari, and Y. Hoshen.
Image shape manipulation from a single augmented training sample.
In Proceedings of the IEEE/CVF International Conference on
Computer Vision, pages 13769–13778, 2021.

[Von Neumann, 1951]
J. Von Neumann.
13. various techniques used in connection with random digits.
Appl. Math Ser, 120 (36-38):0 3, 1951.

[Welling and Teh, 2011]
M. Welling and Y. W. Teh.
Bayesian learning via stochastic gradient langevin dynamics.
In Proceedings of the 28th international conference on machine
learning (ICML-11), pages 681–688, 2011.

[Xu et al., 2018]
P. Xu, J. Chen, D. Zou, and Q. Gu.
Global convergence of langevin dynamics based algorithms for
nonconvex optimization.
Advances in Neural Information Processing Systems, 31, 2018.

[Zhang et al., 2017]
Y. Zhang, P. Liang, and M. Charikar.
A hitting time analysis of stochastic gradient langevin dynamics.
In Conference on Learning Theory, pages 1980–2022. PMLR,

§ IMPLEMENTATION DETAILS

In this section, we delve into the specifics of this study's implementation.

§.§ Victim Classifiers

We utilize two distinct classifiers as the victim classifiers, each with a specific dataset: MadryNet [Madry et al., 2017] with MNIST, and ResNet18 [He et al., 2016] with the SVHN dataset.

§.§.§ MadryNet

We utilize MadryNet [Madry et al., 2017], a convolutional neural network (CNN), as the victim classifier for the MNIST dataset. The MadryNet model is trained using the Adam optimizer, with a learning rate set at $10^{-4}$. Our training regimen comprises $14$ epochs, with a batch size of $64$. Any additional hyperparameters are retained at their default settings as prescribed by PyTorch.

During adversarial training on MadryNet, we implement a Projected Gradient Descent (PGD) untargeted attack on the training data, using the parameters $\epsilon=0.3, \alpha=0.036$, and 10 steps.

§.§.§ ResNet18

In the case of the SVHN dataset, we employ ResNet18 [He et al., 2016], another well-known architecture. The ResNet18 model is trained using the Adam optimizer, with a learning rate set at $10^{-4}$. Our training regimen comprises $14$ epochs, with a batch size of $64$. Any additional hyperparameters are retained at their default settings as prescribed by PyTorch. Similarly to the MadryNet, adversarial training on ResNet18 also involves a PGD untargeted attack on the training data, but with different parameters: $\epsilon=0.03, \alpha=0.01$, and 10 steps.

§.§ Energy-based models

§.§.§ Neural network structure

For the MNIST dataset, we utilize a specialized convolutional neural network with the undermentioned architecture:

* The model commences with a 2D convolutional layer employing 64 filters of 5x5 kernel size, a stride of 2, and a larger padding of 4, effectively augmenting the input image size to 32x32. A `Swish' activation function is then invoked to incorporate non-linearity.

* The second layer consists of another convolutional layer using 128 filters of 3x3 size, with a stride of 2 and padding of 1, followed by the 'Swish' activation function.

* The third layer is a replica of the previous one but escalates the filter count to 256 while preserving the filter size, stride, and padding, followed by a `Swish' activation.

* The fourth convolutional layer utilizes 256 filters, similar to the third layer, with a 3x3 kernel size, stride of 2, and padding of 1. This is succeeded by a `Swish' activation function.

* Post convolution, the output undergoes flattening to eliminate spatial dimensions.

* The flattened output is then passed through a fully connected layer with 256 units, followed by the `Swish' activation function.

* The architecture culminates with a second fully connected layer mapping the 256 units to a determined output size. This size usually correlates to the number of classes in a classification task or the desired output size in regression tasks.

The neural network training employs the Adam optimizer with a learning rate of $10^{-4}$, batch size of 128, and 200 epochs.

For the SVHN dataset, we utilize the WideResNet structure, specifically, the WRN-28-10 variant. Training the WRN-28-10 also involves the Adam optimizer with a learning rate of $10^{-4}$, batch size of 128, and spans across 50 epochs.

§.§ Hardware Specifications

All of our experiments were conducted on a machine equipped with an Intel E5-2680 v3 CPU and an NVIDIA RTX 3090 GPU.

§ ANNOTATOR INTERFACE

Annotator Interface

§ CHOOSING A PROPER F

In this section, we propose two additional $f$ functions, where the first is based on predictive entropy and the second is rooted in joint-energy.

The Predictive Entropy based $f$, denoted as $f_\text{PE}$, is formulated as follows:

\begin{equation*}
f_\text{PE}(\x, \ytar) := -c_\text{PE}\sum_y\sigma(g_\phi(\x))[y]\log\sigma(g_\phi(\x))[y] + f_\text{CE}(\x,\ytar)
\end{equation*}

Here, $c_\text{PE}$ is a constant that determines the weight of the predictive entropy.

On the other hand, the Joint-Energy based $f$, denoted as $f_\text{JE}$, is given by:

\begin{equation*}
f_\text{JE}(\x, \ytar) := -g_\phi(\x)[\ytar] + c_\text{JE}\log\sum_y\exp(g_\phi(\x)[y])
\end{equation*}

In this case, $c_\text{JE}$ is a constant controlling the weight of the logsumexp term. It is worth noting that when $c_\text{JE}=1$, $f_\text{JE}$ simplifies to $f_\text{CE}$.

As shown in Figure <ref>, when $\pdis$ is fixed, choosing $f_\text{CW}$ results in better generation compared to $f_\text{CE}$, $f_\text{PE}$, and $f_\text{JE}$.

An interesting visual interpretation of this phenomenon can be found in Figure <ref>. Here, we draw samples from $\pvic(\cdot; \ytar)$, observing that the samples drawn from the $\pvic$ induced by $f_\text{CW}$ contain the least semantic information.

Targeted Attack on a Single Image: The source image belongs to class 7, and the target class is 9. The first row displays samples drawn from $\padv(\cdot, \xori, \ytar)$, where $\xori$ is an image from class 7 and $\ytar = 9$. All cases share the same random seed and the same $\pdis(\cdot;\xori)$ trained on augmentations of $\xori$. The key distinction among the plots is the function $f$ used, in this order: $f_\text{CE}$, $f_\text{CW}$, $f_\text{PE}$, $f_\text{JE}$. The second row of plots showcases the predictive probability (softmax probability) of the target class, corresponding to each digit in the first row on a one-to-one basis. A green border signifies a successful deception of the victim classifier, while a red border indicates failure.

Samples drawn from the victim distribution $\pvic(\cdot; \ytar)$ with randomly sampled $\ytar$. All four cases share the same random seed. The parameters are consistent with those in Figure <ref>.

§ LOW-VISUAL QUALITY SAMPLES WITH HIGH LIKELIHOOD

During our experiments, we observed that high likelihood samples do not invariably exhibit high visual quality. This phenomenon is showcased in Figure <ref>, where, despite being sorted by energy (a parameter proportional to likelihood), the earliest samples do not always deliver high visual quality. Further, we empirically identified a pattern: high likelihood samples that possess low visual quality often correspond to a low softmax probability for class $\yori$ (the label of the original image). Leveraging this observation, we decided to retain only the top few percentile of samples that have the highest softmax probability for class $\yori$ within an auxiliary classifier, and then sort the remaining samples by energy.

In this instance, the source image, denoted as $\xori$, represents the digit `0' as displayed in Figure <ref>, while the target is class 1. We derived 4641 samples from $\padv(\cdot; 0, 1)$ via rejection sampling. The Left portion of the figure shows the initial 100 samples, ordered by energy. The Right section, on the other hand, depicts the same initial 100 samples, also sorted by energy, but only after retaining the top 10 percent of samples with the highest softmax probability of class 0 in the auxiliary classifier.

§ PGD ATTACKS ON ADVERSARIALLY TRAINED VICTIM CLASSIFIERS

This section is dedicated to showcasing the application of Projected Gradient Descent (PGD) attacks on robustly trained classifiers, employing a variety of parameters.

Targeted attacks on an adversarially trained MadryNet [Madry et al., 2017] using Projected Gradient Descent (PGD) with $L_\infty$ norm, $\alpha=0.04$, and 100 steps. Left: $\epsilon = 0.3$. Right: $\epsilon = 0.4$.

Targeted attacks on an adversarially trained MadryNet [Madry et al., 2017] using Projected Gradient Descent (PGD) with $L_2$ norm, $\alpha=0.2$, and 100 steps. Left: $\epsilon = 3$. Right: $\epsilon = 4$.

Targeted attacks on an adversarially trained ResNet18 [He et al., 2016] using Projected Gradient Descent (PGD) with $L_\infty$ norm, $\alpha=0.005$, and 100 steps. Left: $\epsilon = 0.1$. Right: $\epsilon = 0.3$.

Targeted attacks on an adversarially trained MadryNet [Madry et al., 2017] using Projected Gradient Descent (PGD) with $L_2$ norm, $\alpha=0.1$, and 100 steps. Left: $\epsilon = 1$. Right: $\epsilon = 3$.

§ BROADER IMPACTS OF THIS WORK

The present study introduces a novel approach: the semantics-aware adversarial attack. This method provides significant insights into the resilience and vulnerability of sophisticated classifiers.

From an advantageous perspective, it highlights the inherent risks associated with robust classifiers. By exposing potential weak points in such systems, the study underscores the necessity for further improvements in classifier security. This can pave the way for building more resilient artificial intelligence systems in the future.

Conversely, the work also presents potential pitfalls. There is a risk that malicious entities might exploit the concepts discussed here for nefarious purposes. It is crucial to take into account the potential misuse of this semantics-aware adversarial attack and accordingly develop preventive measures to deter its utilization for unethical ends.