# Stress overshoot, hysteresis and Bauschinger effect in sheared dense
colloidal suspensions

Dmytro Kushnir  IPCMS/CNRS, 23 rue du Loess 67034 Strasbourg, FRANCE  Céline
Ruscher  ICS/CNRS 23 rue du Loess 67034 Strasbourg, FRANCE  Eckhard Bartsch
Institut für Physikalische Chemie and Institut für Makromolekulare Chemie,
Albert-Ludwigs-Universität, D-79104 Freiburg, GERMANY  Fabrice Thalmann  ICS/
CNRS 23 rue du Loess 67034 Strasbourg, FRANCE  Pascal Hébraud  IPCMS/ CNRS, 23
rue du Loess 67034 Strasbourg, FRANCE

###### Abstract

The mechanical non-linear response of dense Brownian suspensions of polymer
gel particles is studied experimentally and by means of numerical simulations.
It is shown that the response to the application of a constant shear rate
depends on the previous history of the suspension. When the flow starts from a
suspension at rest, it exhibits an elastic response followed by a stress
overshoot and then a plastic flow regime. Conversely, after flow reversal, the
stress overshoot does not occur, and the apparent elastic modulus is reduced
while numerical simulations reveal that the anisotropy of the local
microstructure is delayed relative to the macroscopic stress.

††preprint: APS/123-QED

##  I Introduction

Materials deform or flow under the application of stress. Many exhibit both
behaviors and their response depends on the amplitude of the applied stress,
and very often it depends also on the past history of the materials. A common
feature of crystalline materials and amorphous systems, such as foams,
emulsions or concentrated suspensions is the existence of a transition between
an elastic response at small deformation amplitudes to a plastic response,
where the system flows, at larger amplitudes. In general, under an increasing
applied deformation, the stress first linearly increases, and reaches a steady
state value at large deformations, but goes through a maximum value before it
reaches its steady-state value. This stress overshoot is a common feature of
many amorphous materials, such as emulsions [1], foam [2] suspensions [3] and
its amplitude depends on the increase rate of the deformation [4].

Numerical simulations of binary Lennard-Jones glasses suggest that the
overshoot value, $\sigma_{over}$, scales with the applied shear rate
$\dot{\gamma}$ as $\sigma_{over}\propto\ln\dot{\gamma}$ [5, 6]. Conversely,
numerical simulations of attractive spherical particules exhibit a power law
behavior, $\sigma\propto\dot{\gamma}^{0.5}$ [7] and neutralized carbopol
microgel suspensions exhibit a weaker dependency
$\sigma_{over}\propto\dot{\gamma}^{0.13}$ [8]. Moreover, experiments have
shown that attractive colloidal suspensions exhibit two distinct yield points,
due to the breaking of inter-cluster bonds and to collisions between clusters
respectively [9]. The second yield point is characterized by a stress
overshoot whose value depends on the applied shear rate, as
$\sigma_{over}\propto\dot{\gamma}^{0.5}$, the exponent being almost
independent of the concentration.
The yielding mechanisms of hard sphere suspensions are a priori different as
they do not involve the breaking of attractive bonds between particules, but
the escape from cages is the main mechanism of yielding. Yield is related to
the fact that the Brownian motion is not sufficient to relax the structure
formed under stress and that the suspension is able to store stress before
yielding [10, 11].

Moreover, at large deformation amplitudes, materials very often exhibit a
response that depends on their deformation history. In a typical experiment, a
pre-shear is applied, consisting in a constant or oscillatory deformation at
high shear rate or high amplitude, and the mechanical property of the system
is measured at some time $t_{w}$ after the end of this so-called rejuvenating
protocol. For instance, the stress overshoot increases with the time elapsed
since the rejuvenation, $t_{w}$. Past history dependence of the mechanical
response of a system is a common feature that is well known in the engineering
literature. Thus, metals exhibit a smaller resistance to compression when they
have been previously submitted to tension. This effect has been discovered by
Bauschinger when studying steels [12, 13]. A similar effect has been observed
in atomistic simulations of amorphous systems [14, 15], and discussed in the
athermal limit, in terms of the elementary irreversible deformations that
occur in such systems, in Shear Transformation Zone( STZ) [14]. In that
context, Patinet et al. [16] have shown the importance of the role of the
local stress barrier anisotropy to understand the origin of the Bauschinger
effect in the deformation of athermal glasses.

More generally, the Bauschinger effect is related to hysteresis behavior of
the stress when the applied deformation is cyclic and depends on the nature of
the materials. It may, moreover, present some other specificities, such as a
drastic reduction of the elastic response after the first cycle in filled
elastomers (the so-called Mullins effect). As far as concentrated suspensions
are concerned, they exhibit a reduction of their viscosity after the
application of a high shear rate, a phenomenon called thixotropy. The behavior
may be seen as a hysteresis response if one considers the stress vs shear rate
behavior : the flow curve is measured upon increasing the applied shear rate
and then upon decreasing the shear rate. These two flow curves do not
superimpose and the area enclosed by the hysteresis loops decreases when the
applied shear rate decreases. Experiments in carbopol suspensions [17] have
shown that this behavior is also related to the appearance of shear banding.

Here, we study experimentally the nonlinear flow behavior of a concentrated
suspension of spheres and which mimic hard sphere behavior reasonably well. We
consider mainly the two behaviors described above : stress overshoot and non-
reversibility of the flow, that manifests itself by a hysteresis behavior and
a change of the apparent elastic modulus upon flow reversal. Moreover, we use
numerical simulations to compute the anisotropy of the structure of the
suspension submitted to the same deformation histories and we study how the
response is related to the local microstructure.

##  II System

###  II.1 Experimental system

We use spherical particles of cross-linked polymer chains, called microgels,
that are swollen in a good solvent. Several such systems have been developed,
in particular N-isopropylacrylamide particles cross-linked with
N,N’-methylenbisacrylamide, whose swelling ratio in water may be controlled by
the temperature [18, 19]. In the present study, non-thermosensitive PS
microgel particles are used [20, 21]. Their structure and dynamical properties
[22, 23] have been studied in detail and closely match that of hard sphere
suspensions [24]. They have been used to study glass transition behavior [25]
and crystallization kinetics [26, 27, 28]. We use bidisperse suspensions of
particules with crosslinl ratio $1:50$ (i.e. there is on average 1 crosslink
per 50 monomer units) and radii of $175$ nm and $150$ nm with number ratio
$N=3.1$ ($3.1$ small particles per $1$ large particle). The system exhibits a
glass transition at $\phi=0.58$ [29].

###  II.2 Rheological measurements

An Anton-Paar stress controlled rheometer MCR-301 is used in order to perform
rheological measurements. All measurements are done in a cone/plate geometry
of radius $12.5$ mm and angle $2^{\circ}$. In a typical experiment, the sample
is first rejuvenated by the application of an oscillatory deformation of
amplitude $\gamma=400\%$ and frequency $1$ Hz, for $300$ s. Then, the system
is let at rest for $t_{w}=600$ s and the measurements are started. During all
the experiments we follow a deformation protocol with a quasi-constant
absolute value of the shear rate. In order to avoid inertia effects when the
direction of flow is reversed, we apply a deformation protocol that follows a
series of steps. The strain is incremented or decremented by a constant value,
$\delta\gamma=5\cdot 10^{-4}$, separated by a constant time $\delta t$. The
time of the deformation increase is negligibly small in comparison with
$\delta t$ which is itself much smaller than the typical relaxation time of
the system under study. We then define the effective shear rate as
$\dot{\gamma}=\delta\gamma/\delta t$ whose absolute value is kept constant
during an experiment. The maximal deformation amplitude $\gamma_{max}$ is
varied between $0.03$ and $0.2$.

###  II.3 Simulations

In order to get the suspension microstructure, we perform molecular dynamic
(MD) simulations using LAMMPS. We consider a binary mixture of particles,
denoted $A$ and $B$, that interact through an inverse power law (IPL),
potential. Experimenal measurements of the evolution of the elastic modulus
with the volume fraction allows to infer the interaction potential between the
particles, that is well described by [24] :

$U_{\alpha\beta}(r_{ij})=k_{B}T\left(\frac{\sigma_{\alpha\beta}}{r_{ij}}\right)^{42}$
(1)

where ${\alpha,\beta}={A,B}$ and $r_{ij}$ s the distance between two
particles. We choose $\sigma_{AA}=1$ and $\sigma_{BB}=0.876$. Assuming that
the interactions are additive, the cross characteristic length scale is equal
to $\sigma_{AB}=0.938$. Particles are subjected to an overdamped Langevin
dynamics at $T=1.0$ with a damping parameter $\xi=4.0$. The time step is fixed
to $\delta t=10^{-3}$. The shear rate value is determined so that the
experimental and numerical Péclet numbers are identical. We have :

$Pe=\dot{\gamma}\tau_{B}=\dot{\gamma}\frac{R^{2}}{D}=\dot{\gamma}\frac{R\xi}{k_{B}T}$
(2)

The experimental Péclet number is equal to $3\cdot 10^{-3}$, and the numerical
shear rate is chosen in order to match the Péclet number obtained with Eq. 2,
taking into account the largest particles, of radius $R_{AA}=0.5$. This leads
to the numerical shear rate value : $\dot{\gamma}=1.5\cdot 10^{-3}$. Simple
shear deformation is obtained by the application of a succession of strain
increments, $\delta\gamma$ during times $\delta t$, to the cubic box of size
$L$. At each time step, the positions of the particles are updated according
to : $x^{\prime}=x+\delta\gamma y$, $y^{\prime}=y$ and $z^{\prime}=z$. Then,
the system is allowed to relax through viscous dissipation during time
interval $\Delta t$. The applied shear rate is defined as
$\dot{\gamma}=\frac{\delta\gamma}{\Delta t}$.
Cyclic shear experiments are performed, during which the simulation box is
strained up to a value $\gamma=\gamma_{max}$. Then the shear is reversed, i.e
the strain increment is $-\delta\gamma$ and the system is deformed up to
$\gamma=-\gamma_{max}$. Finally the cycle is completed by the application of
shear in the positive direction back to $\gamma=0$.

To compare the numerical simulations with the experiments, we have to consider
equivalent packing fractions. To this aim, we use predictions of the mode
coupling theory to estimate the packing fraction $\phi_{c}=0.585$ at which the
breakdown of ergodicity occurs. We then determine the relevant packing
fractions $\phi$ from the separation parameter
$\epsilon=\frac{\phi-\phi_{c}}{\phi_{c}}$. As a result, we numerically
consider $\phi\in\\{0.607,0.617,0.627\\}$, corresponding to the experimental
volume fractions $\\{0.60,0.61,0.62\\}$. Throughout this work, we consider for
each packing fraction $50$ samples of size $L=60$.

To characterize the anisotropy of the suspension, we consider the fabric
tensor $F$ which has been used successfully for instance in granular media and
in silica or metallic glasses to track residual anisotropy during cyclic
shearing. Following [30], we define the local fabric tensor as :

$\overline{\overline{F}}(i)=\frac{1}{N_{nn}}\sum_{i,j\in
n.n}\frac{\mathbf{r}_{ij}}{r_{ij}}\otimes\frac{\mathbf{r}_{ij}}{r_{ij}}$ (3)

where $\sum_{i,j\in n.n}$ denotes the sum taken over the nearest neighbors of
particle $i$ and $N_{nn}$ is the number of nearest neighbors. The global
fabric tensor is consequently expressed as

$\overline{\overline{F}}=\frac{1}{N}\sum_{i=1}^{N}\overline{\overline{F}}(i)$
(4)

The global fabric tensor $\overline{\overline{F}}$ can be diagonalized and its
eigenvalues $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ measure the degree
of isotropy in the $x,y,z$ directions. For a fully isotropic system one
expects $\lambda_{1}=\lambda_{2}=\lambda_{3}=\frac{1}{3}$. We define the
scalar quantity $\alpha$ which quantifies the degree of anisotropy [30] :

$\alpha=\frac{3}{2}\sqrt{\sum_{k=1}^{3}\left(\lambda_{k}-\frac{1}{3}\right)^{2}}$
(5)

For a fully isotropic (resp. anisotropic) system, $\alpha=0$ (resp.
$\alpha=1$).

###  II.4 The Generalized Maxwell Model

We use the Generalized Maxwell Model, proposed by Voigtmann et al [31, 32] in
order to fit our experimental data. This model is based on the Integration
Through Transients [33] description of the flow of glassy systems and allows
for an analytic expression of the $\sigma(\gamma)$ flow curve. It is first
assumed that the stress may be expressed as an integral over the past applied
shear rates, that involves a memory function, $G(t,t^{\prime})$ :

$\sigma(t)=\int_{-\infty}^{t}\dot{\gamma}(t^{\prime})G(t,t^{\prime})$ (6)

where $G(t,t^{\prime})$ depends on the applied deformation history. It is
written as :

$\nu_{0}\left(1-\left(\frac{\gamma(t,t^{\prime})}{\gamma^{*}}\right)^{2}\right)e^{-\left(\frac{\gamma(t,t^{\prime})}{\gamma^{**}}\right)^{2}}e^{-\frac{(t-t^{\prime})|\dot{\gamma}|}{\gamma_{c}}}$
(7)

The model thus involves four parameters. $\nu_{0}$ is the elastic modulus at
start-up flow when $\gamma\longrightarrow 0$, $\gamma_{c}/|\dot{\gamma}|$ is
the characteristic shear induced relaxation rate value, and $\gamma^{*}$ and
$\gamma^{**}$ capture the distribution of the shear induced relaxation rates
over different length scales. In particular, it should be noted that this
expression of the memory function allows $G(t,t^{\prime})$ to take negative
values, depending on the relative values of $\gamma^{*}$ and $\gamma^{**}$.

This model has been shown to exhibit a well-defined stress overshoot, and
yields a hysteresis behavior when the deformation is reversed.

##  III Startup flow and elasto-plastic transition

Let us consider the start-up flow curve of suspensions of volume fractions
$0.60$, $0.61$ and $0.62$ under the application of a constant shear rate (Fig.
1 (a)). At low deformations, the response is linear and the elastic modulus is
obtained and plotted in the inset of Fig. 1 (a) . The elastic modulus
$G^{\prime}$ at low deformation may be compared to the modulus obtained for
suspensions of PMMA particles. We have, at $\phi=0.60$, $G^{\prime}_{0}=9.8$
Pa $=11.6\frac{k_{B}T}{\langle R\rangle^{3}}$ Pa, close to the value
$7.5\frac{k_{B}T}{R^{3}}$ obtained for PMMA particules suspensions of
identical concentration [34]. Moreover, we have $G^{\prime}\propto\phi^{m}$
with $m=14$ (Fig. 1 (a) inset), in agreement with measurements of Schneider
[24] for similar system, where $m=14-17$ was reported.

Then after yielding, characterized by stress overshoot, a plateau value of the
stress is reached, corresponding to the flowing state. The numerical results
obtained for the corresponding conditions are also given in Fig. 1 (b). The
evolution of the maximum stress value, $\sigma_{over}$ both from experiments
and numerical simulations are given in Fig. 2 (a). The amplitude of the stress
overshoot increases when the shear rate increases : during a given amount of
time, more stress is relaxed under a lower shear rate. The overall set of flow
curves may be adjusted by the Generalized Maxwell Model. In particular the
memory function exhibits negative values, that allow for the description of
stress overshoot. The amplitude and the position of this negative value is
controlled by the parameters $\gamma^{*}$ and $\gamma^{**}$. These parameters
are slightly varied between adjustment curves at different shear rates and are
respectively equal to $0.074\pm 0.013$ and $0.092\pm 0.012$.
The evolution of the stress overshoot amplitude as a function of the shear
rate is well described by $\sigma_{over}\propto\dot{\gamma}^{\alpha}$ with
$\alpha=0.14$ for the experimental data and $\alpha=0.16$ for the simulations.
This value is close to the exponent obtained for attractive carbopol gel
particules where shear banding is observed [8]. Although we do not have access
to experimental measures of the local particle velocity, the numerical
simulations do not exhibit any signature of shear banding, and the computed
shear rate is constant through the gap of the cell. One may also consider the
amount of excess stress stored at yielding, relative to the stress
corresponding to the maximal sheared state, under plastic flow, that is
$\delta\sigma_{excess}=(\sigma_{over}-\sigma_{\infty})/\sigma_{\infty}$.
Experimental and numerical results are given in Fig. 2 (b). This measures the
ability of the system to store stress relative to the stress value of the
suspension under a fully developed flow. At low shear rates,
$\delta\sigma_{excess}$ increases with the shear rate, reflecting the fact
that the suspension is able to store more stress, but at higher shear rates
the suspension cannot deform more than in the fully developed flow and the
stress excess saturates. Our data are consistent with the results obtained in
[35], where the evolution of the stress also exhibits a saturation of excess
of stress at high shear rates.

##  IV Flow reversal behavior

Let us now consider the flow behavior after flow reversal. The suspension is
first submitted to a constant shear rate $\dot{\gamma}=5\cdot 10^{-3}$ s-1 up
to a deformation value $\gamma_{max}$ and then to a reverse flow of same shear
rate. The evolution of the stress is given in Fig. 3, for $\phi=0.62$, both
experimental results (a) and numerical results(b) are given. A strong
hysteresis behavior is observed, already reported in athermal systems [36].
The stress/strain relationship after flow reversal is markedly different from
the startup flow curve. One gets a strong reduction of the stress overshoot.
Adjustment of the reverse flow curves may be obtained from the GMM model using
the parameters determined from the adjustment of the startup flow curve, and
reproduce the reversal flow behavior. In order to understand this reduction,
we consider the memory function $G$ of the GMM model. As already discussed,
during the startup flow, the memory function exhibits negative values
responsible for the stress overshoot; upon flow reversal, the memory is erased
by the $e^{-\gamma/\gamma_{c}}$ factor and the overshoot is reduced.

Experimental and numerical results exhibit a softening of the suspensions upon
flow reversal (Fig. 4 (a) and (b)). We compute the effective shear modulus
$G_{back}$ as the slope of the stress vs strain curve when the total stress
gets back to zero (the slope is plotted in Fig. 3 (a)). This effective modulus
upon shear reversal is smaller than the elastic modulus at rest and decreases
when the maximum applied amplitude increases, reaching a plateau value at
large $\gamma_{max}$. This trend is well reproduced by the Generalized Maxwell
Model. Nevertheless, at the highest studied concentration, the amplitude of
the decay is strongly overestimated by the model. More precisely, the GMM
cannot describe quantitatively both the value of the overshoot and the
amplitude of the reduction of the apparent elasticity after flow reversal and
a slight overestimation is also visible for the other volume fractions. This
indicates that the model does not capture the overall relaxation mechanisms
that play a role in the non-linear response of the suspension, in particular
at the largest volume fractions.
Another characteristic feature of the non-linear behavior of the response is
its hysteretic behavior. The area enclosed by the hysteresis curves is
represented in Fig. 4 (c) and (d). At low maximal deformation amplitudes, the
area enclosed by the hysteresis curves is close to $0$, and the response is
close to purely elastic. Then, with increasing value of $\gamma_{max}$, the
amplitude enclosed by the curves increases. Moreover the experimental
evolution of the area as a function of $\gamma_{max}$ exhibits two behaviors,
corresponding to $\gamma_{max}$ larger or smaller than the deformation at
which the overshoot is observed, $\gamma_{over}$ : at large maximum
deformation values, the enclosed area increases slower than at smaller
deformations, which may be understood as the fact that the stress reaches a
constant value at high $\gamma_{max}$. This behavior is not recovered in
simulations, where the slope of the area as a function of $\gamma_{max}$ does
not decrease at $\gamma_{over}$. This is due a slight difference in the width
of the hysteretic response at small deformation amplitudes.

These results show that, even though flow non-linearities are qualitatively
explained by a simple model of the relaxation mechanisms and may be reproduced
by numerical simulations, the exact dissipation mechanisms require a more
precise description of the interaction potential between the particles and a
more refined relaxation spectra.

##  V Microscopic interpretation

To understand the microscopic origin of the Bauschinger effect in the
colloidal suspensions, we follow the anisotropy during the deformation and, in
particular, we characterize the possible differences between states when no
constraint is applied on the systems, i.e $\sigma=0$. As we perform simple
shear in the $xy$ plane, we first look at the evolution of $F_{xy}$, the $xy$
component of the fabric tensors with strain. The hysteresis cycles of the
stress and of the $xy$ component of the fabric tensor, $F_{xy}$ are plotted in
Fig. 5 (a), for a series of shear maximum amplitudes, $\gamma_{max}$. $F_{xy}$
follows a similar hysteretic cycle as the stress : at low amplitudes, the
anisotropy increases linearly with the deformation and takes the same values
when the shear is reversed. When the amplitude increases, stress overshoot
develops, and corresponds to an overshoot of the anisotropy of the
microstructure. Nevertheless, the evolution of $F_{xy}$ as a function of the
deformation is delayed, when compared to the evolution of the stress. In
particular, after flow reversal, whereas the overall stress comes back to a
null value, the microscopic structure still exhibits an anisotropy
characterized by a positive value of $F_{xy}$. The suspension has not relaxed
back to its equilibrium isotropic structure and it might be expected that a
different linear response is obtained. By plotting the evolution of $F_{xy}$
with the stress (Fig. 5 (b)), one may quantify the delay of the microstructure
anisotropy relative to the stress. The retardation of the anisotropy of the
microstructure may be quantified by the ellipticity value obtained by singular
value decomposition of the data. The results are given in Fig. 5 (c) : the
phase lag between the stress and the anisotropy is plotted as a function of
the maximum applied deformation. It increases during the initial startup flow
phase, until the deformation corresponding to the stress overshoot and then
remains constant as a function of the maximum applied deformation. This
corresponds to the observation of the decay of the effective modulus, that
decays at low maximum shear amplitudes and reaches a plateau at $\gamma\approx
0.1$. This indicates that the Bauschinger effect is associated to a phase lag
between the stress and the deformation that builds up during the initial
startup flow and remains constant when under the established plastic flow
regime.

##  VI Conclusion

Concentrated colloidal suspensions exhibit a rich flow behavior, and their
mechanical properties depend on the previously applied stress. In particular,
after flow reversal, the deformation is not reversible. This leads to a
hysteresis curve in the $\sigma(\gamma)$ plane whose area is the energy
dissipated during a cycle. Moreover, the shape of the hysteresis curve does
not possess a central symmetry relative to the non deformed state at zero
stress, and shearing in an opposite direction leads to a different response
from the deformation in the direction from a rest state : the stress overshoot
disappears and the apparent elastic modulus is reduced after flow reversal.
These observations are in agreement with numerical simulations [32] and our
own simulations show that the anisotropy of the suspension microstructure is
retarded relative to the macroscopic stress after flow reversal.

Figure 1: (a) Evolution of the stress as a function of the deformation, during
the start-up flow. (a) Experimental data for $\phi=0.62$ and for different
shear rate values, increasing from bottom. to top : $\dot{\gamma}=2.5\cdot
10^{-4},5\cdot 10^{-4},10^{-3},2.5\cdot 10^{-3},5\cdot 10^{-3},10^{-2},5\cdot
10^{-2}$ s-1. Continuous curves are adjustment with the Generalized Maxwell
Model. b) Numerical data for $\phi=0.627$ and $\dot{\gamma}=3\cdot
10^{-4},6\cdot 10^{-4},1.5\cdot 10^{-3},3\cdot 10^{-3},6\cdot 10^{-3},1.5\cdot
10^{-2},3\cdot 10^{-2},6\cdot 10^{-2},1.5\cdot 10^{-1}$ and $3\cdot 10^{-1}$
s-1. Inset of (a) and (b) Evolution of the elastic modulus as a function of
the volume fraction. The dashed lines are power-law adjustments, leading to
$G_{0}\propto\phi^{m}$, with $m=14$ for the experimental data and $m=16$ for
the numerical data.

Figure 2:  Evolution of the stress maximum at the overshoot (a) and of the
overshoot stress relative to the stress at high shear rate (b) as function of
the applied shear rate for $\phi=0.62$ (experimental data, $\bullet$) and for
$\phi=0.627$ (numerical data, $\blacksquare$). (a) The lines are power law
adjustment of the data, leading to $\sigma_{over}\propto\dot{\gamma}^{\alpha}$
with $\alpha=0.14$ (continuous line, experimental data) and $0.16$ (dashed
line, numerical data).

Figure 3: Stress as a function of the deformation during the first cycle of
deformation : the deformation is increased from $0$ to $\gamma_{max}$; then
decreased from $\gamma_{max}$ to $-\gamma_{max}$ and increased back from
$-\gamma_{max}$ to $0$. at a constant absolute value of the shear rate. (a)
Experimental data for $\phi=0.62$ and $\gamma_{max}=0.03$, $0.05$, $0.07$,
$0.09$, $0.11$, $0.13$, $0.15$, $0.17$ and $0.20$. The continuous black lines
are adjustment of the data with the Generalized Maxwell Model using a single
set of parameter values for all cycles. The thick segments are linear
adjustment of the $\sigma(\gamma)$ curves around zero stress, after flow
reversal, from with the effective modulus, $G_{back}$ is computed. (b)
Numerical data for $\phi=0.627$ and for the corresponding set of
$\gamma_{max}$ values as experimental data.

Figure 4: (a) and (b) Evolution of the effective modulus, measured as the
slope of the $\sigma$ vs $\gamma$ curve when the stress is null, after flow
reversal (see Fig. 3, for experimental systems (a) and numerical systems (b),
as a function of the maximum deformation applied, $\gamma_{max}$. In (a)
dashed lines are the evolutions of $G_{back}$ obtained from the adjustment of
the experimental data with the Generalized Maxwell Model (see also Fig. 3).
The horizontal thick lines correspond to the elastic modulus at rest, defined
as the slope of the startup flow curves when $\gamma\longrightarrow 0$ (in
(a), continuous lines : experimental data, dotted lines : GMM data). (c) and
d) Evolution of the area of the hysteresis cycles as a function of their
maximum amplitude $\gamma_{max}$. Vertical dashed line indicates the $\gamma$
value at the stress overshoot, $\gamma_{over}$. Continuous lines are linear
adjustments of the data for $\gamma$ values larger and smaller than the
deformation at stress overshoot. Three volume fractions are studied,
increasing from black to grey : $\phi=0.60$, $0.61$ and $0.62$ for the
experimental system and $\phi=0.607$, $0.617$ and $0.627$ for the numerical
system.

Figure 5: (a) Evolution of the stress (left axis, continuous lines) and of the
$xy$ component of the fabric tensor(right axis, dashed line) for a suspensions
of volume fraction $0.627$ submitted to cycles of maximum amplitudes
$\gamma_{max}=0.03$, $0.05$, $0.07$, $0.09$, $0.11$, $0.13$, $0.15$, $0.17$
and $0.20$. (b) Stress as a function of the $xy$ component of the fabric
tensor for $\gamma_{max}=0.2$. The continuous line is an elliptic adjustment
of the data obtained by Principal Component Analysis. (c) Evolution of the
phase shift $\phi$ between the stress and $F_{xy}$, deduced from the
elliptical adjustement of the $\sigma(F_{xy})$ curve.

## References

  * Papenhuijzen [1972] J. M. P. Papenhuijzen, The role of particle interactions in the rheology of dispersed systems, Rheologica Acta 11, 73 (1972).
  * Khan  _et al._ [1988] S. A. Khan, C. A. Schnepper, and R. C. Armstrong, Foam Rheology: III. Measurement of Shear Flow Properties, Journal of Rheology 32, 69 (1988).
  * Nagase and Okada [1986] Y. Nagase and K. Okada, Heterogeneous Behavior after Yielding of Solid Suspensions, Journal of Rheology 30, 1123 (1986).
  * Dzuy and Boger [1983] N. Q. Dzuy and D. V. Boger, Yield Stress Measurement for Concentrated Suspensions, Journal of Rheology 27, 321 (1983).
  * Varnik  _et al._ [2004] F. Varnik, L. Bocquet, and J.-L. Barrat, A study of the static yield stress in a binary lennard-jones glass, The Journal of Chemical Physics 120, 2788 (2004).
  * Rottler and Robbins [2005] J. Rottler and M. Robbins, Unified description of aging and rate effects in yield of glassy solids, Physical Review Letters 95, 225504 (2005).
  * Whittle and Dickinson [2004] M. Whittle and E. Dickinson, Stress overshoot in a model particle gel, The Journal of Chemical Physics 107, 10191 (2004).
  * Divoux  _et al._ [2011] T. Divoux, C. Barentin, and S. Manneville, Stress overshoot in a simple yield stress fluid: An extensive study combining rheology and velocimetry, Soft Matter 7, 9335 (2011).
  * Koumakis and Petekidis [2011] N. Koumakis and G. Petekidis, Two step yielding in attractive colloids: Transition from gels to attractive glasses, Soft Matter 7, 2456 (2011).
  * Koumakis  _et al._ [2012] N. Koumakis, M. Laurati, S. U. Egelhaaf, J. F. Brady, and G. Petekidis, Yielding of Hard-Sphere Glasses during Start-Up Shear, Physical Review Letters 108, 098303 (2012).
  * Zausch  _et al._ [2008] J. Zausch, J. Horbach, L. M., S. Egelhaaf, J. Brader, T. Voigtmann, and M. Fuchs, From equilibrium to steady state: the transient dynamics of colloidal liquids under shear, Journal of Physics: Condensed Matter 20, 404210 (2008).
  * Bauschinger [1881] J. Bauschinger, Uber die veränderung der elasticitätsgrenze und des elasticitätsmoduls verschiedener metalle, Zivil Ingenieur 21, 289 (1881).
  * Sowerby  _et al._ [1979] R. Sowerby, D. Uko, and Y. Tomita, A review of certain aspects of the bauschinger effect in metals, Materials Science and Engineering 41, 43 (1979).
  * Falk and Langer [1998] M. L. Falk and J. S. Langer, Dynamics of viscoplastic deformation in amorphous solids, Physical Review E 57, 7192 (1998).
  * Karmakar  _et al._ [2010] S. Karmakar, E. Lerner, I. Procaccia, and J. Zylberg, Statistical physics of elastoplastic steady states in amorphous solids: Finite temperatures and strain rates, Physical Review E 82, 031301 (2010).
  * Patinet  _et al._ [2020] S. Patinet, A. Barbot, M. Lerbinger, D. Vandembroucq, and A. Lemaître, Origin of the Bauschinger Effect in Amorphous Solids, Physical Review Letters 124, 205503 (2020).
  * Divoux  _et al._ [2013] T. Divoux, V. Grenard, and S. Manneville, Rheological hysteresis in soft glassy materials, Physical Review Letters 110, 018304 (2013).
  * Schild [1992] H. Schild, Poly(N-isopropylacrylamide): Experiment, theory and application, Progress in Polymer Science 17, 163 (1992).
  * Senff and Richtering [1999] H. Senff and W. Richtering, Temperature sensitive microgel suspensions: Colloidal phase behavior and rheology of soft spheres, The Journal of Chemical Physics 111, 1705 (1999).
  * Beyer  _et al._ [2015] R. Beyer, M. Franke, H. J. Schöpe, E. Bartsch, and T. Palberg, From nuclei to micro-structure in colloidal crystallization: Investigating intermediate length scales by small angle laser light scattering, The Journal of Chemical Physics 143, 064903 (2015).
  * Eckert and Bartsch [2004] T. Eckert and E. Bartsch, Glass transition dynamics of hard sphere like microgel colloids with short-ranged attractions, Journal of Physics: Condensed Matter 16, S4937 (2004).
  * Eckert and Bartsch [2003] T. Eckert and E. Bartsch, The effect of free polymer on the interactions and the glass transition dynamics of microgel colloids, Faraday Discussions 123, 51 (2003).
  * Kegel and van Blaaderen [2000] M. Kegel and A. van Blaaderen, Space-resolved dynamic light scattering probing inhomogeneous dynamics in soft matter, Science 287, 290 (2000).
  * Schneider  _et al._ [2017] J. Schneider, M. Wiemann, A. Rabe, and E. Bartsch, On tuning microgel character and softness of cross-linked polystyrene particles, Soft Matter 13, 445 (2017).
  * Eckert and Bartsch [2002] T. Eckert and E. Bartsch, Re-entrant Glass Transition in a Colloid-Polymer Mixture with Depletion Attractions, Physical Review Letters 89, 125701 (2002).
  * Palberg  _et al._ [2009] T. Palberg, A. Stipp, and E. Bartsch, Unusual Crystallization Kinetics in a Hard Sphere Colloid-Polymer Mixture, Physical Review Letters 102, 038302 (2009).
  * Iacopini  _et al._ [2009] S. Iacopini, T. Palberg, and H. J. Schöpe, Crystallization kinetics of polydisperse hard-sphere-like microgel colloids: Ripening dominated crystal growth above melting, The Journal of Chemical Physics 130, 084502 (2009).
  * Franke  _et al._ [2014] M. Franke, S. Golde, and H. J. Schöpe, Solidification of a colloidal hard sphere like model system approaching and crossing the glass transition, Soft Matter 10, 5380 (2014).
  * Burger and Bartsch [2014] S. Burger and E. Bartsch, Influence of the polymer size on depletion attraction-induced gel and glass transitions of microgel colloids, Colloids and Surfaces A 442, 6)15 (2014).
  * Rountree  _et al._ [2009] D. Rountree, C.L. andVandembroucq, M. Talamali, E. Bouchaud, and S. Roux, Plasticity-induced structural anisotropy of silica glass, Physical Review Letters 102, 195501 (2009).
  * Voigtmann  _et al._ [2020] T. Voigtmann, M. Siebenbürger, C. Amann, S. Egelhaaf, S. Fritschi, M. Krüger, M. Laurati, K. Mutch, and K. Samwer, Rheology of colloidal and metallic glass formers, Colloid and Polymer Science 298, 681 (2020).
  * Frahsa  _et al._ [2013] F. Frahsa, B. A.K., J. Horbach, M. Fuchs, and T. Voigtmann, On the bauschinger effect in supercooled melts under shear: Results from mode coupling theory and molecular dynamics simulations, Jounral of Chemical Physics 138, 12A513 (2013).
  * Fuchs and Cates [2002] M. Fuchs and M. Cates, Theory of nonlinear rheology and yielding of dense colloidal suspensions, Physical Review Letters 89, 248304 (2002).
  * Amann  _et al._ [2013] C. P. Amann, M. Siebenbürger, M. Krüger, F. Weysser, M. Ballauff, and M. Fuchs, Overshoots in stress-strain curves: Colloid experiments and schematic mode coupling theory, Journal of Rheology 57, 149 (2013).
  * Laurati  _et al._ [2012] M. Laurati, K. Mutch, N. Koumakis, J. Zausch, C. Amann, A. Schofield, G. Petekidis, J. Brady, J. Horbach, M. Fuchs, and S. Egelhaaf, Transient dynamics in dense colloidal suspensions under shear: shear rate dependence, Journal of Physics : Condensed Matter 24, 464104 (2012).
  * Fiocco  _et al._ [2013] D. Fiocco, G. Foffi, and S. Sastry, Oscillatory athermal quasistatic deformation of a model glass, Physical Review E 88, 020301 (2013).


# Beware the Noise: In-depth Investigation of the Behavior and Defenses of
Black-box Attacks

###### Abstract

Deep learning has grown into a powerful and effective tool that contributes to
solving a wide range of complicated learning tasks that were previously
impossible to solve using conventional machine learning approaches. Deep
learning is rapidly maturing to the point that it can be used in crucial
safety and security applications, including self-driving vehicles,
surveillance, drones, and robots. However, employing adversarial samples,
which are undetectable to the human eye but cause the model to misclassify the
output, might compromise the efficiency of such applications. Addressing the
robustness of deep learning models has become vital to understanding and
defending against adversarial attacks. In this study, a series of experiments
are performed to examine the effect of adversarial attacks and defenses on
various model architectures across well-known datasets. Our research focuses
on black-box attacks such as SimBA, HopSkipJump, and boundary attack, as well
as preprocessor-based defensive mechanisms, including bits squeezing, median
smoothing, and JPEG filter. In the case of ResNet, VGG, and DenseNet, the
experimental results demonstrate that the level of noise needed for the attack
increases as the number of layers increases. On the other hand, the attack
success rate decreases as the number of layers increases. This research
indicates that model complexity and robustness have a significant
relationship. In addition, we investigate the diversity and robustness
correlation for various DNN families, including Inception, Xceptoin,
GoogLeNet, and MobileNet. Our experiments show that having a large number of
parameters does not mean that the model is robust since networks with fewer
parameters are subjected to greater adversarial noise than networks with a
large number of parameters. In the further experiment, various datasets such
as ImageNet-1000, CIFAR-100, and CIFAR-10 are used to evaluate the black-box
attacks. Based on the findings, there are two key observations: 1) the number
of classes in a dataset has no effect on the success rate of black-box
attacks, and 2) the input image size has a significant effect on the attack
success. Moreover, Median and JPEG filters outperform bit squeezing as defense
techniques for the low-resolution image, i.e., CIFAR-100 and CIFAR-10. This
research provides in-depth analysis and insight into the robustness of deep
learning models against various attacks, and the possible defenses. The code,
trained models, and detailed settings and results are anonymously available
at: https://github.com/adv-panda/black-box-attacks.

###### Index Terms:

Deep neural network, black-box attack, adversarial perturbation, defensive
techniques.

##  I Introduction

Deep Learning (DL) is currently extensively used in a wide variety of modern
day-to-day applications since it has made tremendous progress in terms of
equal or beyond human-level performance. Consequently, it is now being
employed on a massive scale to solve difficult scientific challenges such as
image classification, voice recognition, and language translation, thanks to
the advances in the field and the availability of high-performance resources
to train complicated models. Furthermore, in safety and security-critical
situations such as malware detection, self-driving vehicles, drones, and
robots, DL-based solutions are widely used [chakraborty2021survey].

Despite the success of DL, using models to analyze data raises issues
regarding efficiency, energy consumption, data and model privacy, and
susceptibility to adversarial examples [long2021complexity]. A distributed
mechanism is commonly used to address some of these challenges. For example,
due to restricted data accessibility and computing resources, many learning
techniques are distributed, including the Internet of Things, Federated
Learning, and multi-agent networks [su2020defending]. DL models may run more
quickly and have a shorter training period if the models are run on a
distributed system.

As DNN models have progressed from the experiment settings to the real world,
the applications’ security and integrity have become a significant issue. For
instance, vulnerabilities of automatic speech recognition [carlini2016hidden],
Voice controllable systems [zhang2017dolphinattack], and autonomous vehicles
[kurakin2016adversarial] has been demonstrated by adversarial training
examples. Adversaries may modify valid inputs in a way that is undetectable to
the human eyes yet causes a trained model to generate inaccurate outputs
[biggio2013evasion, xie2019intriguing]. After the findings of Szegedy et al.
[szegedy2014intriguing], several intriguing studies involving adversarial
attacks on DL have appeared in the research community.

Existing adversarial attacks may be classified into three categories: white-
box, gray-box, and black-box attacks. The adversaries’ knowledge is what
distinguishes the three attacks [chakraborty2021survey]. The adversary is
considered to have complete knowledge of the target model, including the
architecture and model parameters, in the threat model of white-box attacks.
Therefore, the white-box attack may use any method to generate adversarial
samples on the target model. The adversaries’ information is constrained to
the target model’s structure in the gray-box attacks. On the other hand,
adversaries can only use query access to produce adversarial samples in black-
box attacks. Black-box attacks assume a practical and realistic scenario,
where DL-powered systems are targeted with no knowledge of the systems’ inner
processes. Such attacks are relatively harder to conduct and alarming-to-
devastating when they succeed. Figure 1 shows a taxonomy of black-box attacks.
This work sheds light on the characteristics, effects across multiple
dimensions, and defenses against black-box attacks.

Attacking a classifier may become a search issue inside a narrow region around
a target image if adversarial instances exist. For example, gradient descent
may successfully direct the search in the white-box attack [madry2018towards,
szegedy2014intriguing]. However, since DL models are of black-box nature, the
black-box attacks are more realistic in many cases and are closer to real-
world challenges. This study concentrates on the most frequently accessible
methodologies among the black-box attack techniques, such as Simple Black Box
Attack (SimBA) [guo2019simple], HopSkipJump [chen2020hopskipjumpattack], and
boundary attacks [brendel2018decision] (in Fig 1).

As defensive strategies against adversarial examples, there have been a
plethora of preprocessor, trainer, and detector defenses. Shaham et al.
[shaham2018defending] evaluated several defensive approaches, such as low-pass
filtering, PCA, soft thresholding, and JPEG, by applying a preprocessing step
on an adversarial input, and found that preprocessing improves the performance
of all defenses. As a consequence, we focus on commonly used preprocessor type
defenses, including bits squeezing [xu2018feature], median smoothing
[osadchy2017no], and JPEG filter [dziugaite2016study].

Figure 1: A taxonomy of black-box attacks. The stars mark the attacks
considered in this work (i.e., SimBA, HopSkipJump, and Boundary Attack).

Contributions. In this work, we investigate the various aspects of the attacks
and defenses on the state-of-the-art models, including VGG, ResNet, DenseNet
families using the benchmark datasets such as ImageNet, CIFAR-10, and
CIFAR-100. Furthermore, the different model architecture Xeption, InceptionV3,
MobileNetV2, and GoogLeNet for diversity are examined. In order to address the
following research questions, a large-scale experiments are performed and
analyzed:

  * 1

Due to the risk of making incorrect decisions when faced with an adversarial
attack, the use of DL models may be limited. It is unclear, however, which
attack is the most powerful with reference to the model complexity and how it
affects various models. Is the complexity of the models a factor in the
attack’s success/failure rate? To answer this question, we tested the
relationship between model complexity and robustness to different black-box
attacks using the ImageNet1000, CIFAR10, and CIFAR100 datasets.

  * 2

VGG, ResNet, and DenseNet all have distinct architectures than GoogleNet,
Inception, Xception, and mobileNet. When different network architectures face
adversarial examples, is there a relationship between model diversity and
robustness? To answer this question, we investigate various model
architectures under adversarial settings to observe their behavior and
susceptibility to attacks.

  * 3

It is critical to investigate the variables on which an attack is predicated.
It would be advantageous if studies could determine under what circumstances
the attacks fail or succeed. In this context, an experiment is conducted to
analyze how adversarial attacks work on various datasets in terms of image
size and the number of classes.

  * 4

The defense to adversarial input may be done using a variety of
countermeasures. When adversarial examples are provided to the model, how do
the models effectively address the attack problem while implementing defenses?
The most commonly used preprocessor defense strategies are adopted and
analyzed to address this question.

Organization. The remainder of the paper is organized as follows: The basis of
attacks and defensive strategies is provided in Section II. Section III
highlights the essential principles for black-box attacks and defensive
strategies, then it describes the experimental settings, i.e., the datasets,
models, and evaluation metrics employed in this work. Section IV summarizes
the experimental findings and discussions. The paper is concluded in Section
VI.

##  II Preliminaries and Background

The primary concept behind distributed DL is to divide a DNN or coordinate
several shallow neural networks. A DNN may be divided into many separate
branches, each with its own set of nodes, resulting in a massive quantity of
intermediate data being transmitted [long2021complexity]. However, by using
parallelized optimization techniques to distribute DNN over numerous nodes,
excellent performance may be achieved [zinkevich2010parallelized,
dean2012large, li2014scaling, jia2018highly, li2019hpdl].

Attacks and Datasets. Although the sensitivity of DNNs to adversarial
instances in the image context was primarily assessed using two datasets,
CIFAR [krizhevsky2009learning] and MNIST [lecun1998gradient], Carlini et al.
[carlini2017adversarial] discovered that approaches created using one of these
datasets may not always extend to other datasets. Either of the abovementioned
datasets, in comparison to ImageNet [russakovsky2015imagenet], include images
with lower resolution and fewer categories. As a consequence, in recent years,
the vast majority of research efforts have favored ImageNet over CIFAR and
MNIST [croce2019sparse, guo2019simple, su2018empirical]. Nonetheless, ImageNet
has a key drawback when it comes to analyzing adversarial assaults,
particularly in pre-trained networks, according to Ozbulak et al.
[ozbulak2021evaluating]: ImageNet has a huge number of classes that are
semantically very similar to one another.

Attacks and Defenses. New defensive mechanisms are created to countermeasure
the attacks. The intuitive strategy of adversarial re-training was among the
first methods to be attempted to avoid the attacks
[Goodfellow2015ExplainingAH]. On ImageNet, Kurakin et al.
[kurakin2016adversarial] used adversarial re-training and discovered that it
could defend against specific attacks. Papernot et al.
[papernot2016distillation] proposed yet another approach to avoid adversarial
examples, demonstrating that network distillation gives some protection when
created using Jacobian-based Saliency Maps (JSMA) [papernot2016limitations].
Carlini et al. [carlini2017adversarial] published the most influential work on
adversarial defense mechanisms to date, demonstrating that neither of the
suggested defenses are efficient against strong attacks and nor the properties
of available defense mechanisms scale well from lower to higher resolution
images, such as MNIST to ImageNet datasets.

Unlike previous research efforts, which have primarily focused on creating
more efficacious adversarial perturbations as well as assessing adversarial
defenses, this paper looks into a topic that has yet to be addressed:
specifically, the effect of adversaries and defenses across datasets, the most
important factors that influence the adversarial success rate, and the
relationship between complexity, diversity, and robustness. As a result, this
work provides an in-depth analysis of the required defensive mechanisms for
adversarial attacks against various models using a variety of input data. We
show that a considerable proportion of black-box adversarial examples may be
identified simply by studying the prediction logits.

##  III Methods and Settings

This section highlights the terminology and notations used in this work,
describes the studied attacks and defenses, and outlines the experiment
settings.

General notations. Let the parameters of a model $\mathcal{F}$ be $\Theta$,
the input $x$, and the label $y$ associated with input. The produced model
output may be $y^{\prime}$. After an attack, $x^{*}$ represents the perturbed
input sample. To train the model, a cost function $\mathcal{J}(.)$ is
utilized.

###  III-A Black-box Attack

The development of adversarial examples in a black-box context may be found in
the literature [akhtar2018threat]. The research examines the most predominant
black-box attacks, which are summarized in this subsection.

SimBA. SimBA aims to detect a modest disturbance $\sigma$ that causes the
prediction $\mathcal{F}(x+\sigma)$ to differ from the actual value $y$. It may
be defined as a modified restricted optimization problem [guo2019simple], with
a small misapplication of notation:

$\begin{split}\min_{(x+\sigma)}&\hskip
5.69046pt\mathcal{M}\big{[}(x+\sigma),~{}~{}x\big{]}\\\ subject\hskip
2.84544ptto&\hskip 5.69046pt||\sigma||_{2}<\omega,\quad
queries\leq\mathcal{B}\\\ \end{split}$

Where the distance metric $\mathcal{M}$ is minimized, using a perturbation
$\sigma$ with $\ell_{2}$ below a pre-defined threshold $\omega$ when the
adversary is allowed to query the model less than $\mathcal{B}$ times.

HopSkipJump. HopSkipJump aims to generate the adversarial examples by looking
at the output labels $y^{\prime}$ of the targeted model. The suggested attack
is divided into two modes: Targeted (T) and Untargeted (UT) attacks
[chen2020hopskipjumpattack]. The objectives of both settings are as follows:

$\begin{split}&\min_{x^{*}}\mathcal{M}(x^{*},x)\\\ \textbf{T:}&\hskip
5.69046ptsubject\hskip 2.84544ptto\hskip
2.84544pt\mathcal{F}(x^{*})=y^{\prime}\\\ \textbf{UT:}&\hskip
5.69046ptsubject\hskip 2.84544ptto\hskip
2.84544pt\mathcal{F}(x^{*})\neq\mathcal{F}(x)\neq y\\\ \end{split}$

Where $x^{*}=(x+\sigma)$ and $y^{\prime}$ is the target class. The attack
suggests the following boundary function $\mathcal{S}(x^{*})$:

$\begin{split}\textbf{T:}&\hskip 5.69046pt\hskip
2.84544pt\mathcal{F}(x^{*})_{y}-\max_{y^{\prime}\neq
y}\mathcal{F}(x^{*})_{y^{\prime}}\\\ \textbf{UT:}&\hskip 5.69046pt\hskip
2.84544pt\max_{y^{\prime}\neq\mathcal{F}(x)}\mathcal{F}(x^{*})_{y^{\prime}}-\mathcal{F}(x^{*})_{\mathcal{F}(X)}\\\
\end{split}$

Boundary Attacks. Boundary attacks emphasize the significance of attacks that
are exclusively based on the model’s end decision. The input image is
constrained to a range of [0,255] resampled to a uniform distribution
$\mathcal{U}[0,1]$ [brendel2018decision]. The attack effectiveness is
determined by the proposal distribution. The following is the concept
underlying this proposal distribution:

$\begin{split}\mathcal{F}(x,x^{*})-\mathcal{F}(x,x^{*}+\delta^{k})=\hskip
2.84544pt\eta.\mathcal{F}(x,x^{*})\end{split}$

Where $\delta^{k}$ is the perturbation from a maximum entropy distribution
must be selected for the $k^{th}$ stage. The perturbation decreases the
distance between the perturbed image and the original image by $\eta$.

Figure 2: On ResNet, VGG, and DenseNet model architectures, the SimBa attack
is described. (a) Shows how as the number of layers rises, the amount of noise
injected ascents continuously. (b) Shows that as the number of layers grows,
the attack success rate falls. TABLE I: The rate of success of black-box
attacks across datasets and models. The following is the representation: total
instances successfully attacked out of the 200 instances testing data, and
average MC score $\pm$ standard deviation in parentheses.

Attack | Target Model | ImageNet-1000 | CIFAR-100 | CIFAR-10
---|---|---|---|---
SimBA | ResNet-152 | 77/200 (0.254±0.127) | 193/200 (0.314±0.119) | 167/200 (0.482±0.040)
VGG-19 | 124/200 (0.235±0.125) | 181/200 (0.425±0.077) | 157/200 (0.486±0.033)
DenseNet-201 | 75/200 (0.199±0.128) | 192/200 (0.419±0.075) | 183/200 (0.488±0.028)
HopSkipJump | ResNet-152 | 200/200 (0.656±0.174) | 199/200 (0.629±0.163) | 180/200 (0.892±0.110)
VGG-19 | 200/200 (0.504±0.160) | 200/200 (0.781±0.141) | 197/200 (0.925±0.105)
DenseNet-201 | 200/200 (0.673±0.162) | 200/200 (0.783±0.151) | 181/200 (0.917±0.091)
Boundary Attack | ResNet-152 | 200/200 (0.420±0.085) | 198/200 (0.420±0.076) | 177/200 (0.496±0.019)
VGG-19 | 198/200 (0.374±0.103) | 197/200 (0.453±0.052) | 192/200 (0.491±0.020)
DenseNet-201 | 197/200 (0.428±0.074) | 197/200 (0.443±0.057) | 179/200 (0.492±0.020)

###  III-B Defense Approaches

There are numerous studies on defense mechanisms against adversarial examples
[Papernot2018SoKSA, yuan2019adversarial]. The most prevalent strategies
utilized in the study are summarized in this subsection.

Bits Squeezing. Bits squeezing approach is used to remove redundant pixels in
the color bit depths [xu2018feature]. An RGB pixel has a 24-bit color depth,
which means it can represent $2^{24}\approx 16$ million distinct colors. The
bit squeezing defense technique reduces the input image to i-bit depth $(1\leq
i\leq 7)$ by first multiplying the input with round$(2^{i}-1)$. The output is
normalized to $[0,1]$, by dividing $2^{i}-1$. The information capacity of the
representation is reduced from 24-bit to i-bit with the integer-rounding
operation [xu2018feature].

Median Smoothing. This approach is particularly successful in the setting of
the adversarial examples at reducing noise by smoothing each pixel by
averaging the neighboring pixels [osadchy2017no]. A sliding window is applied
on each pixel, replacing the center pixel with the median value of the nearby
pixels inside the window. It is especially efficient at eliminating
sporadically occurring monochromatic pixels from an image while keeping the
borders of objects intact [xu2018feature].

JPEG filtering. This defense may be used to reverse small adversarial
perturbations induced by black-box attacks [dziugaite2016study]. Consider the
weight of a DNN be $\Theta$, and the perturbation is calculation as follows:

$\Gamma(x)=\frac{\lambda}{255}\hskip
5.69046pt\textbf{JPEG}\Big{(}\nabla_{x^{*}}\mathcal{J}(x^{*},\Theta,y)\Big{|}_{x^{*}=x,y=l_{x}}\Big{)}$

Where the $\textbf{JPEG}(.)$ denotes the JPEG filtering. The class label
represented by $l_{x}=\arg\max\mathcal{P}_{\Theta}(y|x)$. The gradient
$\nabla_{x^{*}}\mathcal{J}(.)$ may be back propagated to compute efficiently.

###  III-C Experimental Settings

Dataset. ImageNet [russakovsky2015imagenet], CIFAR-10
[krizhevsky2009learning], and CIFAR-100 [krizhevsky2009learning] are three
common datasets used in all of the experiments. The ImageNet dataset, with a
224x224 image size and 14 million samples, is organized into 1000 classes. The
CIFAR-10 dataset contains 60,000 images with a 32x32 image size for 10
classes. The CIFAR-100 dataset, with a 32x32 image size and 60,000 samples, is
divided into 100 classes. Due to the time complexity of the black box attacks,
all of the stated results are centered on test data that includes 200
occurrences.

Models. To examine the correlation between model complexity and robustness, 13
models from the ResNet [he2016deep], VGG [simonyan2014very], and DenseNet
families are employed in experiment 1. In experiment 2, seven distinct models
are trained on each dataset to explore the relation between model diversity
and robustness, including GoogLeNet, Inception V3, Xception, and MobileNet V2.
Pre-trained models for ImageNet dataset based on the Pytorch framework were
employed, and for CIFAR-10 and CIFAR-100 datasets, we trained all models on
our servers.

Experiment Workstation. The experiments are run on a machine equipped with an
Intel Xeon(R) CPU E5-2620 v3 @ 2.40 GHz$\times$ 24 with Cuda-10.0 and three
GEFORCE GTX TITANx 12 GB GPUs, as well as Python 3.7.7 distributed in Anaconda
4.8.3 (64-bit).

Evaluation Metrics. We use the following evaluation metrics.

  * 1

Misclassification confidence. In this metrics, we calculated the confidence
for misclassified image. 2 Attack time. We calculated an average attack time
for each models. We measured it in seconds. 3 Noise rate. We used structural
similarity (SSIM) for measuring amount of noise. SSIM measures how two images
are similar to each other. For calculating the noise rate, we subtracted the
SSIM value from 1 (i.e., noise rate = 1 - SSIM). 4 L1 Distance. L1 distance is
a way to measure distance between vectors. It calculates the sum of absolute
difference of the components of the vectors. To obtain the results, all values
are normalized to [0, 1].

##  IV Results and Discussion

Adversarial perturbations are intriguing because, whereas humans perceive them
undetectable, the adversaries are quite observable to DNNs over a broad range
of training regimens. Large-scale research is performed across the diverse
families of DNN architectures and compared them under the various datasets to
assess the performance and efficacy of the black-box attacks and preprocessing
type defensive mechanisms. In this part, the outcomes of the experiments are
discussed.

###  IV-A Experiment 1: Model Complexity and Robustness

The ImageNet dataset is used in Experiment 1 to explore the relationship
between complexity and robustness of the model. For a valid comparison, the
same family model architecture is utilized with varying numbers of layers,
such as ResNet, VGG, and DenseNet. The conception is that a significant
quantity of noise is introduced into the most complicated model. Figure 2 (a)
depicts how the quantity of noise introduced steadily grows as the number of
layers increases. Because the divergence between the layers is larger than in
VGG, the variation of noise is easily noticeable in ResNet and DenseNet. As
the number of layers expands, the model gets more intricate, and the adversary
attempts to mislead the model by adding more noise and requiring an extended
time.

Observation 1:  Number of layers affects robustness The model becomes more
robust as the number of layers on the same model increases (See Figure 2). The
black-box attacks cause more noise as the model (i.e., from the same family)
grows more intricate.

For instance and as illustrated in Figure 3, the true label of a benign image
is an airship. The HopSkipJump attack adds 8.4% noise to ResNet-18 in 51
seconds, whereas ResNet-152 receives 28.6% noisy image in 78 seconds. The
ResNet-18 Misclassification Confidence (MC) score of 53.94% is lower than the
ResNet-152 since the input is less noisy.

Figure 3: The example illustrates the amount of noise perturbed and the time
needed by the HopSkipJump attack for ResNet-18 and ResNet-152 models. Figure
4: It represents a wide range of DNNs for determining the relationship between
the number of large parameters and the model’s robustness against black box
attacks. Figure 5: On ImageNet, CIFAR-100, and CIFAR-10 datasets, the noise
rate of the HopSkipJump black-box attack is reported. VGG outperforms ResNet
and DenseNet when dealing with low resolution input images. Even with fewer
layers, VGG is more robust than ResNet and DenseNet on low resolution images.
TABLE II: Across datasets and models, the success rate of bits squeezing
defense against black-box attacks is reported. The following is the
representation: total instances successfully defended out of the total
successful adversarial examples, and average MC score $\pm$ standard deviation
in parentheses. Bit squeezing fails only to defend against HopSkipJump for
CIFAR 100 and CIFAR 10.

Attack | Target Model | Bit Squeezing
---|---|---
ImageNet | CIFAR-100 | CIFAR-10
SimBA | ResNet-152 | 73/77 (0.585±0.183) | 146/193 (0.388±0.154) | 128/167 (0.583±0.092)
VGG-19 | 100/124 (0.518±0.234) | 137/181 (0.576±0.134) | 124/157 (0.629±0.114)
DenseNet-201 | 72/75 (0.517±0.25) | 154/192 (0.641±0.178) | 162/183 (0.682±0.145)
HopSkipJump | ResNet-152 | 170/200 (0.734±0.18) | 12/199 (0.512±0.138) | 11/180 (0.619±0.093)
VGG-19 | 157/200 (0.609±0.195) | 23/200 (0.577±0.110) | 12/197 (0.598±0.119)
DenseNet-201 | 154/200 (0.655±0.184) | 65/200 (0.664±0.165) | 29/181 (0.772±0.166)
Boundary Attack | ResNet-152 | 198/199 (0.844±0.139) | 168/198 (0.552±0.125) | 153/177 (0.696±0.139)
VGG-19 | 189/198 (0.682±0.192) | 166/197 (0.657±0.139) | 166/192 (0.729±0.135)
DenseNet-201 | 196/197 (0.786±0.158) | 181/197 (0.733±0.186) | 155/179 (0.827±0.141)

The attack success rate, on the contrary, decreases as the number of layers
increases, as illustrated in Figure 2 (b). As a result, when the model
encompasses a high number of layers, it becomes more robust to black-box
attacks.

###  IV-B Experiment 2: Model Diversity and Robustness

According to Goodfellow et al. [Goodfellow2016], adding many hidden layers
makes the model more complicated, but it does not ensure the model’s general
robustness. The model overfits because to the large number of parameters,
which means it may learn effectively from training data but not from test
data. Therefore, in Experiment 2, models with less number of parameters such
as Inception V3, Xception, GoogLeNet, and MobileNet V2 were incorporated from
a diverse family to evaluate the complexity in relation to robustness against
black-box attacks. As seen in Figure 4, a substantial quantity of noise is
introduced to the model with fewer model parameters. For example, although
VGG-19 has the most parameters, it is not the most robust. In other words,
black-box attacks may fool the model by introducing a modest amount of noise.
Except in the case of MobileNet V2, as moving to the right along x-axis, the
number of parameters decreases, but the quantity of noise steadily rises. We
observed that the large number of parameters do not assure robustness.

Observation 2:  Model size and robustness Model architectures and the number
of parameters do not guarantee robustness. The robustness of DNNs may vary
across architectures, and the number of parameters is not an indicator of
robustness when multiple architectures are involved.

###  IV-C Experiment 3: Attacks across Different Dataset

According to the literature, there is several factors to be considered in the
performance evaluation of DNNs, such as input image quality
[dodge2016understanding], dataset size [linjordet2019impact], bias and
variance [yang2020rethinking], and batch size [hoffer2017train]. This
experiment examines black-box attacks on ImageNet-1000, CIFAR-100, and
CIFAR-10 datasets. The evaluation of the black-box attacks yielded two
significant conclusions. 1 The number of classes in the dataset has no
consequence on the success rate of the black-box attack. As shown in Table I,
the attack success rate is consistent with CIFAR-100 and CIFAR-10, indicating
that the number of classes does not affect the attack success rate. Moreover,
except for the CIFAR-10 dataset, it is observed that the black-box attack
success rate is roughly 100% in the case of HopSkipJump and boundary attack.
Regarding SimBA attacks, ImageNet-1000 has the lowest attack success rate
compared to CIFAR-100 and CIFAR-10.

Observation 3:  Number of classes and robustness The number of classes in a
dataset has little influence on the success rate of a black-box attack (Table
I).

2

The size of the input image heavily influences the attack success rate.
Compared to CIFAR-100 and CIFAR-10, ImageNet-1000 has a substantially higher
input size. As can be seen from Table I, the attack deceived the model with
small MC score for larger input size, whereas higher MC score for smaller
input size.

Observation 4:  Size of input and robustness The input image size has a
significant impact on the success rate of black-box attacks.

Despite more number of layers, ResNet is less complex, the architecture of VGG
is more complex while having fewer layers. Each layer of the VGG is more
densely complicated than ResNet, with a far larger number of parameters
[he2016deep]. The experiment findings showed that VGG is more robust for lower
input image sizes, as shown in Figure 5. In the case of ImageNet, less noise
is needed to attack VGG; however, for CIFAR-100 and CIFAR-10, more noise is
introduced to attack VGG as compared to ResNet and DenseNet.

Figure 6: It demonstrates how the bit squeezing, median, and JPEG defensive
techniques increase and decrease the amount of noise needed to defend the
model effectively. Under the HopSkipJump attack, bit squeezing seems to add
more noise than median and JPEG filters, reducing its confidence score.

###  IV-D Experiment 4: Defenses against Black-box Attacks

Arguably, defending an attempted attack is as crucial as predicting an
accurate result. The purpose of this experiment is to identify hostile inputs
and harden DNNs against adversarial attacks. The defenses techniques in this
study modify the input image but do not change the model. We observed that the
confidence score of the benign image drops, maximum of 2%, when the defenses
are used against black-box attacks, as seen in Table II. Consequently,
defenses with extremely weak parameters are used to maintain a high benign
confidence score. The findings revealed that black-box attacks are ineffective
with using minimal parameters for defenders. The bit depth for bit squeezing,
for example, is 4 that is very weak. The primary reasoning underlying the
attacks is to add or delete a given amount of noise to push the target image
nearer the border of another class. Most adversarial attacks, such as boundary
attacks and SimBA, cease perturbing noise when the image passes the boundary
and is transferred to another class boundary. Therefore, such attacks result
in a low MC score, and the adversarial images revert to the original class
border with minimal defender settings. For instance, ResNet-152 misclassifies
the airship as a sea cucumber with an MC score of 63.02% after adding 28.6%
noise, as illustrated in Figure 6. With a confidence score of 68.13%, the bit
squeezing defensive technique adds 3.1% additional noise to effectively defend
the model. Whereas the median and JPEG defensive filter techniques remove the
noise by smoothing the pixels, the confidence score is roughly 100%.

Observation 5:  Weakness of black-box attacks Even with weak defensive
parameters, black-box attacks are vulnerable to countermeasures.

The bit squeezing defense technique, on the other hand, fails miserably in the
CIFAR-100 and CIFAR-10 datasets under the HopSkipJump attack, as seen in Table
II. On a small input image size with a high MC score, the HopSkipJump attack
misled the model.

As a consequence of the previous finding, we hypothesize that bit squeezing
increases noise, lowering confidence scores. On the other hand, with a high
confidence level, both the median and JPEG filters outperformed the bit
squeezing. For the CIFAR-100 and CIFAR-10, as shown in Table III, the median
smoothing outperformed bit squeezing and JPEG filter under the HopSkipJump
attack. Using the example for the CIFAR-100 dataset with ResNet-152, bit
squeezing successfully defended 12 input images out of 199 adversarial
samples, while median smoothing effectively defended 188 images. Similarly,
VGG and DenseNet seem to have a high level of consistency across datasets.

TABLE III: The performance of median smoothing against HipSkipJump black-box
attacks is addressed.

Attack | Target Model | CIFAR-100 | CIFAR-10
---|---|---|---
HopSkipJump | ResNet-152 | 188/200 (0.914±0.165) | 168/180 (0.987±0.056)
VGG-19 | 185/200 (0.960±0.115) | 185/197 (0.992±0.033)
DenseNet-201 | 187/200 (0.963±0.103) | 171/181 (0.992±0.031)

JPEG filter outperformed bit squeezing for CIFAR-10 and CIFAR-100 across
ResNet-152, VGG-19, and DenseNet-201, but was ineffective compared to median
smoothing. In the CIFAR-100 dataset using ResNet-152, the JPEG filter
outperforms VGG and DenseNet by successfully defending 105 occurrences, as
shown in Table IV. For the CIFAR-10 dataset, however, the JPEG filter
consistently performs with a maximum accuracy of 78.33% with ResNet-152
network. The authors further examined why bit squeezing and the JPEG filter
failed while using HopSkipJump attack.

TABLE IV: The performance of JPEG filter against HipSkipJump black-box attacks
is addressed.

Attack | Target Model | CIFAR-100 | CIFAR-10
---|---|---|---
HopSkipJump | ResNet-152 | 105/200 (0.806±0.225) | 141/180 (0.964±0.096)
VGG-19 | 82/200 (0.922±0.135) | 136/197 (0.963±0.089)
DenseNet-201 | 97/200 (0.867±0.204) | 128/181 (0.978±0.058)

###  IV-E Experiment 5: The Behavior of HopSkipJump Attack

Based on the findings of the previous experiment, bit squeezing fails to
defend against the HopSkipJump attack; however, medium smoothing and JPEG
filtering worked effectively. The HopSkipJump attack is shown to require extra
noise to the input image in order to make it an adversarial example. Another
experimental analysis is conducted to determine why this extra noise was
needed for the HopSkipJump attack and why bit squeezing failed to defend
against it. In this subsection, a thorough analysis is presented to have
insight into the HopSkipJump attack’s behavior.

TABLE V: Under the HopSkipJump attack with ResNet-152 on CIFAR-10 dataset, the
number of misclassified samples with the average noise rate in the
parenthesis. The yellow cells represent the misclassified class that is
closest to the actual class. The gray cells indicate the relationship between
the same classes. The pink cells denote the assumed target class by the
HopSkipJump attack.

| Plane | Car | Bird | Cat | Deer | Dog | Frog | House | Ship | Truck
---|---|---|---|---|---|---|---|---|---|---
Plane | 0 | 1 (0.0001) | 19 (0.0185) | 8 (0.0090) | 20 (0.0163) | 0 | 34 (0.0342) | 0 | 17 (0.0133) | 4 (0.0045)
Car | 6 (0.0146) | 0 | 2 (0.0019) | 3 (0.0142) | 0 | 2 (0.0016) | 42 (0.0149) | 0 | 7 (0.0085) | 49 (0.0058)
Bird | 9 (0.0072) | 2 (0.0113) | 0 | 7 (0.0073) | 17 (0.0035) | 7 (0.0113) | 43 (0.0299) | 1 (0.0002) | 0 | 1 (0.0007)
Cat | 0 | 0 | 11 (0.0011) | 0 | 13 (0.0040) | 11 (0.0006) | 37 (0.0070) | 1 (0.0011) | 2 (0.0006) | 0
Deer | 0 | 0 | 26 (0.0047) | 6 (0.0027) | 0 | 2 (0.0039) | 63 (0.0099) | 3 (0.0003) | 0 | 1 (0.0019)
Dog | 1 (0.0000) | 0 | 12 (0.0037) | 37 (0.0033) | 10 (0.0072) | 0 | 28 (0.0108) | 0 | 1 (0.0184) | 0
Frog | 0 | 0 | 7 (0.0110) | 1 (0.0015) | 2 (0.0025) | 0 | 96 (0.0000) | 0 | 0 | 0
House | 2 (0.0043) | 0 | 13 (0.0043) | 9 (0.0038) | 50 (0.0080) | 14 (0.0052) | 16 (0.0164) | 0 | 2 (0.0055) | 2 (0.0046)
Ship | 16 (0.0126) | 2 (0.0126) | 11 (0.0130) | 5 (0.0105) | 15 (0.0137) | 0 | 54 (0.0166) | 0 | 0 | 5 (0.0091)
Truck | 8 (0.0034) | 11 (0.0029) | 7 (0.0041) | 8 (0.0108) | 5 (0.0112) | 4 (0.0051) | 44 (0.0113) | 0 | 25 (0.0035) | 0

Figure 7: On CIFAR-10 datasets, the class decision boundaries are reported
among ResNet-152, VGG-19, and DenseNet-201 networks.

Observation 6:  Undefined behavior of attacks The boundary attack and
HopSkipJump attack select a target class and alter the input class images
toward the selected target class, even when the setting is for an untargeted
attack scenario.

Even for untargeted scenarios, the HopSkipJump is an iterative-based decision
boundary attack that is initiated with a target class. For the CIFAR-10
dataset with ResNet-152, the frog is considered as the target class, as shown
in Table V (light pink cells). In the majority of the instances, the input
classes are incorrectly categorized as a frog. A boundary plot is used to
analyze the explanation of why the frog is selected as the target class.
Figure 7 depicted the class decision boundary graph, which demonstrated that
the frog class is extremely far from all other classes. Based on statistical
distance measurements, it is determined that the HopSkipJump attack picks a
target class that is the furthest away from the other classes. As a result,
the frog is selected as the target class. The HopSkipJump technique fails to
mislead the model when a frog class is an input to the network, as shown in
Table V, since no noise is introduced to the frog class. Other classes, on the
other hand, have a significant amount of noise added to them in order to
misclassify them as frog (target) classes.

In addition, a substantial number of cases have been misclassified to classes
other than the targeted class. For example, a plane class is incorrectly
labeled 17, 19, and 20 times as a ship, bird, and deer, respectively. Another
example, a ship class is misclassified 11, 15, and 16 times as a bird, deer,
and aircraft, respectively. Other class instances also suffer from the uneven
pattern of misclassification. The incorrectly labeled are the nearby classes
of the input class, as can be seen in Figure 7. It has been observed that when
an input class is misclassified to the neighboring class, less noise is added.

Observation 7:  Targeting a neighbor class The black-box attacks perturbed
less noise when an input image class is misclassified to an adjacent class.

However, when evaluated across the models, it is revealed that the target
class is not the same. For instance, frog class is reported as the target
class for ResNet-152 and VGG-19, whereas bird class is selected as the target
class for DenseNet-201. It also demonstrated that, in addition to the class
distance measurement statistic, the target class is determined by the model
internal structure. Furthermore, the authors argue that the HopSkipJump attack
and the boundary attack have analogous attacking mechanisms. A comparison
between both attacks is presented in the discussion section (Section V).

TABLE VI: Under the Boundary attack with ResNet-152, the number of
misclassified instances with the average noise rate in the parenthesis. The
yellow cells represent the misclassified class that is closest to the actual
class. The gray cells indicate the relationship between the same classes. The
pink cells denote the assumed target class by the Boundary attack.

| Plane | Car | Bird | Cat | Deer | Dog | Frog | Horse | Ship | Truck
---|---|---|---|---|---|---|---|---|---|---
Plane | 0 | 2 (0.0007) | 23 (0.0214) | 7 (0.0086) | 19 (0.0092) | 0 | 30 (0.0255) | 0 | 17 (0.0080) | 5 (0.0052)
Car | 6 (0.0135) | 0 | 4 (0.0028) | 1 (0.0058) | 1 (0.0101) | 2 (0.0013) | 35 (0.0126) | 0 | 12 (0.0073) | 50 (0.0055)
Bird | 7 (0.0045) | 1 (0.0115) | 1 (0.0000) | 11 (0.0134) | 17 (0.0032) | 4 (0.0047) | 43 (0.0204) | 1 (0.0002) | 1 (0.1233) | 1 (0.0005)
Cat | 0 | 0 | 12 (0.0027) | 1 (0.0015) | 17 (0.0025) | 15 (0.0007) | 28 (0.0058) | 1 (0.0013) | 1 (0.0001) | 0
Deer | 0 | 0 | 31 (0.0044) | 8 (0.0040) | 4 (0.0004) | 0 | 52 (0.0077) | 5 (0.0028) | 0 | 1 (0.0018)
Dog | 1 (0.0000) | 0 | 12 (0.0035) | 36 (0.0021) | 13 (0.0063) | 1 (0.0033) | 25 (0.0109) | 0 | 1 (0.0056) | 0
Frog | 0 | 0 | 6 (0.0065) | 1 (0.0012) | 4 (0.0047) | 0 | 95 (0.0000) | 0 | 0 | 0
Horse | 1 (0.0037) | 1 (0.0039) | 15 (0.0052) | 8 (0.0032) | 47 (0.0065) | 15 (0.0037) | 15 (0.0127) | 1 (0.0000) | 1 (0.0045) | 4 (0.0031)
Ship | 20 (0.0100) | 2 (0.0046) | 14 (0.0083) | 4 (0.0104) | 13 (0.0130) | 1 (0.0076) | 47 (0.0152) | 0 | 0 | 7 (0.0052)
Truck | 10 (0.0057) | 14 (0.0017) | 11 (0.0074) | 9 (0.0065) | 2 (0.0032) | 2 (0.0043) | 38 (0.0107) | 1 (0.0022) | 24 (0.0044) | 1 (0.0009)

###  IV-F Experiment 6: Image Background and Attack Behavior

After a comprehensive examination, another finding associated with the visual
background is described in this subsection. In comparison to regular images,
there are a few images where the attacks took a long time and caused a lot of
noise. SimBA, HopskipJump, and Boundary attacks on ResNet-152 had average
noise levels of 0.005, 0.061, and 0.072, respectively. According to the
experimental findings, attacks have a hard time misclassifying images with a
clear background, as can be seen in Figure 8. In comparison to the plain
background (middle and bottom row), the attack takes less time and has a lower
noise rate (top row). The SimBA attack, for example, takes 58 seconds to
deceive the model, while a clean backdrop image takes 240 seconds. Moreover,
the SimBA attack failed to categorize the second example (bottom row)
correctly. In HipSkipJump and boundary attack, we observed a similar pattern.
The amount of noise introduced to clean-background images is much higher than
average.

Observation 8:  Image background Black-box attacks have a great deal of
difficulty in misclassifying images with a clear background.

Figure 8: It depicts a contrast between images with a clear background and
those with a complicated background. The images with intricate background are
shown in the top row. The images with a clean background are shown in the
middle and bottom rows. Figure 9: It demonstrates the amount of noise that
were perturbed by three black-box attacks in the variety of models across
different dataset.

##  V Discussion

Various black-box attacks responded differently in terms of attack success
rate, misclassification confidence, and quantity of noise across various
datasets, according to the observation of the conducted experiments in this
work. Each attack’s behavior is examined and discussed in depth in this
section.

On the decision boundary, the basic notion of SimBA attack is to shift the
actual class toward the adjacent classes. As a consequence, SimBA perturbs
significantly less noise than HopSkipJump and boundary attacks, suggesting
that a tiny quantity of noise is sufficient to push a class to its neighboring
classes (Fig 9). Table II shows the misclassification confidence of SimBA is
low as a result of the aforementioned characteristic, and therefore defending
the model against SimBA attack is straightforward. Another observation, the
amount of noise that perturbed by SimBA increased as the number of classes
decreases across different dataset. As mentioned before, SimBa shifts the true
class of the image to its near class. As the number of classes decreases, the
distance between classes increases, for that reason, SimBA needs to add more
noise as the number of classes decreases.

In the context of boundary attack, an intriguing observation is made. Despite
the fact that the boundary attack is untargeted, it selects a target class
based on the statistical analysis and model parameters, and attempts to push
instances of other classes toward that target class. The boundary attack is
evaluated on ResNet-152 using the CIFAR-10 dataset, and the attack selected
the frog class as a target class, as shown in Table VI. The amount of noise
introduced to deceive the model for a specific class instance is significantly
large. For example, when a plane object image is provided to the model, the
attack attempts to add a large quantity of noise in order to shift the plane
class to frog (target) class. However, the attack eventually fails to deceive
the model when it is given a frog class input since the input and target
classes are the same.

HopSkipJump uses a decision-based boundary approach as well. The sole
difference is that it is query efficient and highly tuned, allowing it to
outperform boundary attacks. The HopSkipJump attack, on the other hand, adopts
the same strategy as the boundary attack, which is to find a suitable target
class even if the targeted setting is disabled. The HopSkipJump attack is
assessed on ResNet-152 with the CIFAR-10 dataset, and it selects the frog
class as a target class, similar to the boundary attack (refer Table V). The
HopSkipJump attack also forces all other classes towards the designated target
(frog) class in order to deceive the model. Due to a similar characteristic,
when a frog class instance is provided to the model, the attack fails to
mislead. Another key observation is that when a class is misclassified to its
adjacent classes, the amount of the noise is very little (refer Table VI and
Table V, yellow cells). In essence, the boundary and HopSkipJump attack both
picked one target, despite the fact that the attacks were not configured as
target attacks.

##  VI Conclusion and Future Scope

This work presents the results of large-scale experiments on the effect of
adversarial attacks and countermeasures on distinct models utilizing three
state-of-the-art datasets. The results of the experiments revealed that there
is a relationship between model complexity and robustness, and that having a
large number of parameters does not imply that the model is more robust.
Moreover, the attack success rate is unaffected by the number of classes in
the dataset. When a large size of input is provided to the model, the model
misclassifies the adversarial image with a low MC score. When the input size
is modest, VGG is proven to be more robust. For SimBa and boundary black-box
attacks, the defensive technique with minimal parameters settings is
effective. In comparison to the median and JPEG filters, the bit squeezing
defensive technique generated more noise, lowering the confidence score. Bit
squeezing was shown to be less accurate than the median and JPEG defensive
approaches for low-resolution inputs.

Future Work. In this research, we addressed the behavior of well-known black-
box attacks and countermeasures. A promising future direction can be to
explore the behavior of the white-box and gray-box attacks, including recent
advanced attacks such as [abdukhamidov2021advedge, zhang2020interpretable]
with other defense strategies. Moreover, studying the complexity and
robustness of various architectures other than CNNs, such as Vision
Transformers, is an interesting future direction.