11institutetext: Amirarsalan Rajabi 22institutetext: Complex Adaptive Systems
Lab (CASL), University of Central Florida, Orlando, 32816
22email<EMAIL_ADDRESS>33institutetext: Seyyedmilad
Talebzadehhosseini 44institutetext: Complex Adaptive Systems Lab (CASL),
University of Central Florida, Orlando, 32816
44email<EMAIL_ADDRESS>55institutetext: Ivan Garibay
66institutetext: Complex Adaptive Systems Lab (CASL), University of Central
Florida, Orlando, 32816
66email<EMAIL_ADDRESS>
# Resistance of communities against disinformation

Amirarsalan Rajabi  Seyyedmilad Talebzadehhosseini  and Ivan Garibay

###### Abstract

The spread of disinformation is considered a big threat to societies and has
recently received unprecedented attention. In this paper we propose an agent-
based model to simulate dissemination of a conspiracy in a population. The
model is able to compare the resistance of different network structures
against the activity of conspirators. Results show that connectedness of
network structure and centrality of conspirators are of crucial importance in
preventing conspiracies from becoming widespread.

##  1 Introduction

We define conspiratorial thinking as a belief held by an individual or a group
of individuals, while there is enough evidence and information to undermine or
totally refute the belief. If some conspiracy theories get popular support, it
may cause serious concerns.This is particularly true of conspiracies over
scientific and medical issues where conspiracy theories can result in
rejection of the scientific method lewandowsky2013role .

Several underlying reasons have been proposed to explain the existence of
conspiracy theories. According to Barkun, conspiratorial thinking exhibits
three characteristics: Firstly, nothing happens by accident. Secondly, things
are not as they seems on surface. And thirdly, things are highly connected
barkun2013culture . All three characteristics mentioned by Barkun, refer to a
special cognitive function of the conspiracy theorist. Indeed, a great deal of
literature on conspiracy theory associates conspiratorial thinking with a
special and different heuristic of the conspiracy theorist. On the other hand,
Sunstein and Vermeule claim that many of those who hold conspiracy theories do
so not as a result of a mental illness of any kind, or of simple
irrationality, but as a result of crippled epistemology (knowing very few
things, which are indeed wrong) sunstein2009conspiracy . For most of what they
believe and know, human beings lack direct information; they must rely on what
others say and think. Hardin argues many people suffer from crippled
epistemology, meaning they only get their information from a few incorrect
sources breton2002political . Crippled epistemology usually takes place in
echo chambers. Echo chambers are communities in which individuals merely
communicate with each other and rarely seek information from entities outside
the community. The advent of social media platforms has resulted in the rise
of echo-chambers bakshy2015exposure . Echo chambers play an important role in
political and social polarization barbera2015tweeting . The negative effects
of polarization in social networks is studied in garibay2019polarization .
Bauman states that individuals who are embedded in isolated groups or small,
self-enclosed networks who are thus exposed only to skewed information, will
more often hold conspiracy theories that are justified, relative to their
limited informational environment bauman2013liquid . The study of the dynamics
by which the echo chambers form, can therefore shed light on the mechanisms by
which conspiratorial thinking forms and thrives in a community.

Traditional studies of conspiratorial thinking assume that conspiratorial
thinking extinguishes in large network structure grimes2016viability , and
that conspiratorial ideation is because of flawed reasoning and biased
heuristics barkun2013culture . This study challenges both of these claims.
Contrary to a large body of literature on conspiracy theory that studies the
cognitive function of isolated individuals [22], this paper takes into account
systemic belief dissemination as a result of interactions between individuals.

##  2 Opinion Dynamics

Opinion formation is a complex process which is formed by the interaction of
multiple underlying elements. People tend to form their opinion on a wide
variety of subjects through the process of learning. “Social learning” is a
term referring to the process of learning through the communication of
individuals with each other, their own experience and their observations of
others’ experiences, media sources, propaganda and indoctrination from
political leaders and the state acemoglu2011opinion . In this paper, we refer
to models of opinion dynamics as mathematical models that aim at capturing the
dynamics of social learning, opinion spreading, collective decision making,
and so on from a mathematical point of view.

Models of opinion dynamics can be divided into two categories : Bayesian
models of opinion dynamics and non-Bayesian models of opinion dynamics.
Bayesian models rely on Bayes rule bayes1991essay . These kinds of models
assume that an individual (agent) is Bayesian rational and update their belief
optimally with respect to Bayes rule, given an underlying model of the world.
One problem with Bayesian models of opinion dynamics is that these models make
a lot of assumptions. One demanding assumption of these models is that an
agent must have a reliable prior assumption about the world, an assumption
that might be unrealistic in many cases. Additionally, it is assumed that they
then go on and update their prior beliefs based on the new information that
they get from others. Bayesian models also put too much structure on updating
by ruling out “zero probability events” acemoglu2011opinion . The
aforementioned problems make Bayesian models unfavorable to be incorporated in
our study of dissemination of conspiracy theories.

Non-Bayesian approaches and models on the other hand try to avoid some of
these problems. Non-Bayesian approaches are believed to be more effective in
modeling belief manipulation and the spread of disinformation. The simplest of
these models start by specifying rules of thumb acemoglu2011opinion . Several
different non-Bayesian models exist. Classical models of interacting particle
systems which in inspired by statistical mechanics inspires many of these
models (see for example see for example clifford1973model ; galam2002minority
; latane1981psychology ; castellano2009nonlinear ; hegselmann2002opinion ).

One noteworthy non-Bayesian model is DeGroot (1974) degroot1974reaching . In
this model, a set of interacting agents start by an initial beliefs about an
underlying state held by each agent, and exchange information about their
beliefs with their neighbors and update their beliefs at discrete time
instances, with respect to a weight matrix that represents the social network
structure of interactions. This model captures the “imitation” aspect of non-
Bayesian models. While notably innovative, this model suffers from duplication
of information acemoglu2010spread , meaning that agents in this model might
interact endlessly with their neighbors that hold unchanging opinions in each
timestep.

##  3 Model of Conspirators

Our agent-based model is inspired by the variation of Acemoglu et. al. on
DeGroot model acemoglu2010spread . In our model, two types of agents exist.
Majority of agents are _susceptible_ agents and a minority of agents exist
that are called _conspirators_. Conspirators deliberately disseminate false
information to susceptibles.

Let’s consider a conceptual underlying state of the world and call it
$\Theta$. We assume that the true value of $\Theta$ is 1, and the discussion
between agents in the model is on the true value of $\Theta$. $x_{i}^{s}(k)$
and $x_{i}^{c}(k)$ represent the opinion of susceptible agent $i$ at time $k$,
and the opinion of conspirator agent $i$ at time $k$, respectively. At first,
each susceptible agent holds an initial belief about the underlying state. N
shows the total number of agents in the model. The initial belief of each
susceptible agent is a randomly generated float number between zero and two,
the initial belief of each conspirator agent is 0, and the average of initial
beliefs of all susceptible agents is very close to one:

$x_{i}^{s}(0)\in[0,2]\qquad
x_{i}^{c}(0)=0\qquad\frac{1}{N}\sum_{i}x_{i}^{s}(0)\approx 1$ (1)

Hence although at first each susceptible agent has their own initial belief on
the underlying state $\Theta$, there is a consensus on this underlying state
between the susceptible agents. On the other hand, the initial belief of
conspirator agents is 0 and remains 0 during the simulation, irrespective of
their interactions. These could be thought of as individuals, entities, media
or propaganda outlets that deliberately and constantly disseminate false
information throughout the population.

(a) Scale Free (Barabasi-Albert)

(b) Small World (Watts-Strogatz)

Figure 1: NetLogo environment. A) shows a network generated with Barabasi-
Albert algorithm. B) shows a network generated by Watts-Strogat algorithm

There are two main differences of between our proposed model and the work of
acemoglu2010spread : 1) Agents are not able to freely communicate directly
with any other agents. Rather, each agent is only able to communicate with the
agents that she is connected to by a link. This means that a network structure
regulates the communication in the population. 2) In the variation of
acemoglu2010spread  on DeGroot model, each agent meets and communicates with
other agents at instances defined by a rate one Poisson process independent of
other agents. In our model agents are connected to each other by undirected
links, and together, they form a network. in each timestep of the simulation,
an agent will choose another agent to which it is connected and may or may not
communicate with them. In this model, the capacity of an agent for
communication in each timestep is the number of her links. For example an
agent with 5 links will communicate with 5 or less other agents in each
timestep.

Following the interaction of two agents i and j, there is a potential exchange
of information between them with probability $\textrm{p}_{interaction}$.
Agents update their beliefs according to one of the following possibilities:

If i and j are both susceptibles:
$\displaystyle\begin{cases}x_{i}(k+1)=x_{j}(k+1)=\frac{1}{2}[x_{i}(k)+x_{j}(k)]&\text{with
probability}\;\textrm{p}\\\
x_{i}(k+1)=x_{i}(k)\;\&\;x_{j}(k+1)=x_{j}(k)&\text{with
probability}\;1-\textrm{p}\end{cases}$ If i is susceptible and j is
conspirator:
$\displaystyle\begin{cases}x_{i}(k+1)=\frac{x_{i}(k)+0}{2}&\text{with
probability}\;\textrm{p}\\\ x_{i}(k+1)=x_{i}(k)&\text{with
probability}\;1-\textrm{p}\\\ x_{j}(k+1)=x_{j}(k)=0&\text{with probability
1}\end{cases}$ If i and j are both conspirators: no opinion sharing.

The underlying network structure of the model plays an important role in
determining the behavior of the model, and it is itself determined by the
algorithm by which the model forms the network. It determines which agents are
connected to each other, how easily the information would propagate throughout
the network, and so forth.

Watts and Strogatz proposed a model to capture the small world, high
clustering, and low average path properties of real complex networks
watts1998collective . On the other hand, Barabasi and Albert showed that
power-law degree distribution is the property of many real world networks and
proposed an algorithm that can capture this phenomenon barabasi2003scale .
Indeed, it is believed that majority of complex networks exhibit small world
and scale free properties wang2003complex . In order to capture all of these
essential properties, the model generates the network of agents using both
Watts-Strogatz and Barabasi-Albert algorithms.

##  4 Results

We developed our agent-based model in NetLogo sunstein2009conspiracy . Figure
1 shows two instances of the model with watts-strogatz and Barabasi-Albert
model. For our purposes we defined a variable named collective-thought. This
variable is simply the average of the belief of all susceptible agents and
represents the collective belief of the population about the underlying state
$\Theta$. Collective-thought starts with a value close to one, and always
converges to zero(Figure 2).

A desired population is one that resists the activity of conspirators.
Numerous individuals and groups deliberately disseminate false information in
a society. As we know the collective-thought of the model always converges to
zero after some timesteps. We call this state the convergence. In order to
quantify the resistance of a network against the activity of conspirators, we
record the number of timesteps required for collective-thought of the network
to converge. This number can be thought of as symbolically representing how
long a society resist the effort of conspirators, before it is nearly deceived
into believing that the underlying state $\Theta$ is zero.

We argue that contrary to conventional belief that conspiratorial beliefs are
untenable with larger network structure (e.g. grimes2016viability ), a large
network cannot ensure eradication of conspiratorial beliefs. Indeed, the
important aspect of a network structure that can ensure resistance against
conspiracies is the _connectedness_ of the network;that is, how easily an
agent can send an information through the network to any other agent. A
network with high connectedness means that information propagates more easily
and eco-chambers are less likely to form. We needed to come up with a
benchmark that enables us to quantitatively compare different networks by our
objective of connectedness. For this purpose, the model records the mean path
length of the network in each instance of the simulation as follows
fronczak2004average :

mean path length = average shortest path between all distincrt pairs of nodes
in network (2)

The model was run 1000 times for each of the Watts-Strogatz and Barabasi-
Albert networks. In each run, the required timsteps for the collective-thought
to converge and the mean path length of the network were recorded. Figure 3
shows that there is a positive relationship between mean path length of the
network, and required timesteps for the collective-thought to reach
convergence. With Barabasi-Albert algorithm, the Pearson correlation
coefficient between the mean path length and the required timesteps was 0.18,
with a p-value less than 0.00001 which shows the correlation is statistically
significant. The corresponding coefficient for Watts-Strogatz networks was
0.39 with a p-value less than 0.00001, which shows this correlation is also
significant. These results prove our hypothesis that the connectedness of a
network improves its resistance against conspirators.

(a) Scale Free (Barabasi-Albert)

(b) Small World (Watts-Strogatz)

Figure 2: Figure show the required timesteps required for the network’s
collective-thought to converge. Small world network tends to be deceived more
quickly than a scale free network.

(a) Scale Free (Barabasi-Albert)

(b) Small World (Watts-Strogatz)

Figure 3: The relationship between mean path length and the required timesteps
for a population’s collective-thought to converge in A)scale free and B)small
world networks

(a) Scale Free (Barabasi-Albert)

(b) Small World (Watts-Strogatz)

Figure 4: The relationship between sum of the eigenvector of all 4
conspirators and the required timesteps for a population’s collective-thought
to converge in A)scale free and B)small world networks

Besides the connectedness of a network, the structural position of
conspirators in a network determines the reach of their disinformation and
enhances their ability to disseminate their message through the whole
population more effectively. To show that our model captures this phenomenon,
we assigned to each agent in the network their eigenvector centrality.
Eigenvector centrality is a measure of centrality in a network
newman2008mathematics . The more central a node is, the closer it is to all
other nodes.

The model was run 1000 times for each of the Watts-Strogatz and Barabasi-
Albert networks. For each run, the required timsteps for the collective belief
to reach convergence and the sum of the eigenvector centrality of conspirator
agents were recorded. Figure 4 shows the negative relationship between
required timesteps for collective-thought to reach convergence and the sum of
the eigenvector centrality of conspirators. With Barabasi-Albert algorithm,
the Pearson correlation coefficient between the mean path length and the
required timesteps was -0.19, with a p-value less than 0.00001 which shows the
correlation is statistically significant. The corresponding coefficient for
Watts-Strogatz networks was -0.42 with a p-value less than 0.00001, which
shows this correlation is also statistically significant.

##  5 Discussion

The model shows promising results in that its vulnerability is correlated with
the connectedness of the network and the importance (eigenvector) of
conspirators. Considering the minimum required time, the resistance of a
population against conspiracies was imitated. The results show that a network
that is built by the Watts-Strogatz algorithm was slightly more vulnerable to
conspiracies than a network which is built by Barabasi-Albert algorithm (scale
free). The reason for this difference might be that a Watts-Strogatz network
has a high local clustering, while holding a short average path length like
random networks, therefore the propagation of (dis)information is easier and
faster in them. On the other hand, while noting that we used only 4
conspirators in our model, and the total number of agents were 100, the
probability that conspirators become a hub was low, and therefore their
ability to propagate their conspiracy was slightly smaller than in Watts-
Strogatz network. A probable misinterpretation here might be that a less
connected network is better against conspiracy theories. This is not correct
because in our model, only conspirators were acting to deceive the population
and were trying to propagate their disinformation in the whole network. In
fact, the results of this study can be interpreted in a content-agnostic
manner. In reality, both sides of the discussion try to influence the network.
Nevertheless, both networks show that the connectedness and the eigenvector
centrality of conspirators is a network is highly correlated with the
network’s vulnerability to conspiracies.

This was a first step to make a framework for computational study of
conspiracy propagation. Further research must be conducted using other network
formation algorithms. Other ranges of conspirators’ ratio and population sizes
should also be experimented. Another extension to this research might be to
test the effect of multi-dimensional opinion-space in agents. Finally, other
non-Bayesian models of opinion dynamics could be tested.

## References

  * (1) Acemoglu, D., and Ozdaglar, A. Opinion dynamics and learning in social networks.  Dynamic Games and Applications 1, 1 (2011), 3–49.
  * (2) Acemoglu, D., Ozdaglar, A., and ParandehGheibi, A. Spread of (mis) information in social networks.  Games and Economic Behavior 70, 2 (2010), 194–227.
  * (3) Bakshy, E., Messing, S., and Adamic, L. A. Exposure to ideologically diverse news and opinion on facebook.  Science 348, 6239 (2015), 1130–1132.
  * (4) Barabási, A.-L., and Bonabeau, E. Scale-free networks.  Scientific american 288, 5 (2003), 60–69.
  * (5) Barberá, P., Jost, J. T., Nagler, J., Tucker, J. A., and Bonneau, R. Tweeting from left to right: Is online political communication more than an echo chamber?  Psychological science 26, 10 (2015), 1531–1542.
  * (6) Barkun, M. A culture of conspiracy: Apocalyptic visions in contemporary America, vol. 15.  Univ of California Press, 2013.
  * (7) Bauman, Z. Liquid modernity.  John Wiley & Sons, 2013.
  * (8) Bavelas, A. Communication patterns in task-oriented groups.  The Journal of the Acoustical Society of America 22, 6 (1950), 725–730.
  * (9) Bayes, T. An essay towards solving a problem in the doctrine of chances. 1763.  MD computing: computers in medical practice 8, 3 (1991), 157.
  * (10) Breton, A., Galeotti, G., Salmon, P., and Wintrobe, R. Political extremism and rationality.  Cambridge University Press, 2002.
  * (11) Castellano, C., Muñoz, M. A., and Pastor-Satorras, R. Nonlinear q-voter model.  Physical Review E 80, 4 (2009), 041129.
  * (12) Clifford, P., and Sudbury, A. A model for spatial conflict.  Biometrika 60, 3 (1973), 581–588.
  * (13) DeGroot, M. H. Reaching a consensus.  Journal of the American Statistical Association 69, 345 (1974), 118–121.
  * (14) Fronczak, A., Fronczak, P., and Hołyst, J. A. Average path length in random networks.  Physical Review E 70, 5 (2004), 056110.
  * (15) Galam, S. Minority opinion spreading in random geometry.  The European Physical Journal B-Condensed Matter and Complex Systems 25, 4 (2002), 403–406.
  * (16) Garibay, I., Mantzaris, A. V., Rajabi, A., and Taylor, C. E. Polarization in social media assists influencers to become more influential: analysis and two inoculation strategies.  Scientific Reports 9, 1 (2019), 1–9.
  * (17) Grimes, D. R. On the viability of conspiratorial beliefs.  PloS one 11, 1 (2016), e0147905.
  * (18) Hegselmann, R., Krause, U., et al. Opinion dynamics and bounded confidence models, analysis, and simulation.  Journal of artificial societies and social simulation 5, 3 (2002).
  * (19) Latané, B. The psychology of social impact.  American psychologist 36, 4 (1981), 343.
  * (20) Lewandowsky, S., Gignac, G. E., and Oberauer, K. The role of conspiracist ideation and worldviews in predicting rejection of science.  PloS one 8, 10 (2013), e75637.
  * (21) Newman, M. E. The mathematics of networks.  The new palgrave encyclopedia of economics 2, 2008 (2008), 1–12.
  * (22) Sunstein, C. R., and Vermeule, A. Conspiracy theories: Causes and cures.  Journal of Political Philosophy 17, 2 (2009), 202–227.
  * (23) Sunstein, C. R., and Vermeule, A. Conspiracy theories: Causes and cures.  Journal of Political Philosophy 17, 2 (2009), 202–227.
  * (24) Wang, X. F., and Chen, G. Complex networks: small-world, scale-free and beyond.  IEEE circuits and systems magazine 3, 1 (2003), 6–20.
  * (25) Watts, D. J., and Strogatz, S. H. Collective dynamics of ‘small-world’networks.  nature 393, 6684 (1998), 440.


# Sparse vs Contiguous Adversarial Pixel Perturbations in Multimodal Models:
An Empirical Analysis

Cristian-Alexandru Botocan<EMAIL_ADDRESS>EPFLLausanneSwitzerland armasuisse S+TThunSwitzerland ,  Raphael Meier
<EMAIL_ADDRESS>armasuisse S+TThunSwitzerland and  Ljiljana
Dolamic<EMAIL_ADDRESS>armasuisse S+TThunSwitzerland

(2024)

###### Abstract.

Assessing the robustness of multimodal models against adversarial examples is
an important aspect for the safety of its users. We craft $L_{0}$-norm
perturbation attacks on the preprocessed input images. We launch them in a
black-box setup against four multimodal models and two unimodal DNNs,
considering both targeted and untargeted misclassification. Our attacks target
less than 0.04% of perturbed image area and integrate different spatial
positioning of perturbed pixels: sparse positioning and pixels arranged in
different contiguous shapes (row, column, diagonal, and patch). To the best of
our knowledge, we are the first to assess the robustness of three state-of-
the-art multimodal models (ALIGN, AltCLIP, GroupViT) against different sparse
and contiguous pixel distribution perturbations. The obtained results indicate
that unimodal DNNs are more robust than multimodal models. Furthermore, models
using CNN-based Image Encoder are more vulnerable than models with ViT—for
untargeted attacks, we obtain a 99% success rate by perturbing less than 0.02%
of the image area.

AI security, Multimodal models, Contiguous Attacks, Sparse Attacks, Pixel-
perturbations

††copyright: cc

##  1\. Introduction

In computer vision, the researchers try to find new methods for improving
image classification and other tasks beyond that (e.g. given a set of textual
descriptions and an image, the model chooses the most likely image
description). For that, the research community developed multimodal models
(Radford et al., 2021; Haas et al., 2023; Jia et al., 2021; Chen et al., 2022;
Li et al., 2022; Xu et al., 2022; Pham et al., 2023; Zhong et al., 2022),
which are used as a basis for further research (CLIP (Radford et al., 2021)
led e.g. to various modifications such as RegionCLIP (Zhong et al., 2022)) and
are increasingly also considered for commercial usage 111Deloitte report:
Novel Design Classification with CLIP. Thus, it is essential to use such
models, knowing their robustness and security implications. While prior
studies have demonstrated the vulnerability of multimodal models to
perturbations of the whole image (Mao et al., 2022; Qiu et al., 2022), their
vulnerability to pixel-level perturbations remains largely unexplored. In
particular, the impact of the number of perturbed pixels and their spatial
distribution on the attack performance has not been explored so far. Both of
these basic aspects must be considered when constructing adversarial attacks
using pixel perturbations. Empirical analysis of their impact on attack
performance would enable more informed design choices when crafting novel
adversarial pixel perturbations and ultimately lead to a better understanding
of the overall robustness of multimodal models.

In this work, we develop methods and experiments to address this gap. We focus
on the black-box scenario because we want to confront the security problem of
an attacker who does not have any prior information about the model. We rely
on $L_{0}$-norm perturbations to have control over the number of pixels being
perturbed. We extend _Sparse_ pixel distribution attack presented in work by
Su et al. (Su et al., 2019) to incorporate different spatial encodings
resulting in five different _Contiguous Attacks_ (details are described in
Section 4.2). Depending on the number and distribution of perturbed pixels,
our method exploits potential vulnerabilities hidden in the model
architectures, such as the kernel sizes in the case of CNN-based Image Encoder
of multimodal models. Pixel perturbations are usually performed in the
original image. This assumes a threat model with minimal information available
to the adversary (i.e., does not need to know the preprocessing routine).
However, if you perturb the preprocessed image, you can be certain that your
experimental results are not confounded by the different preprocessing
pipelines, which we prioritized for this study. We evaluate the effectiveness
of these attacks with images from ImageNet on four open-source models (ALIGN
(Jia et al., 2021), AltCLIP(Chen et al., 2022), CLIP-B/32 (Radford et al.,
2021), and GroupViT (Xu et al., 2022)) representative of the domain.
Additionally, we evaluate our approach for two state-of-the-art Deep Neural
Networks (DNNs), which are trained on the ImageNet dataset (Krizhevsky et al.,
2012). Finally, we also release the code for reproducing the study:
https://github.com/ChristianB024/SparseVsContiguityRepo

Our empirical analysis led to several key insights:

  * •

We found that three out of four multimodal models and both unimodal DNN models
are most vulnerable to the _Sparse Attack_ for all examined attack scenarios
(targeted and untargeted misclassification).

  * •

The _Patch Attack_ is most effective against the CNN-based multimodal model
(ALIGN (Jia et al., 2021)). In an untargeted scenario, we reach a 99% success
rate only by perturbing 0.01915% of the image (16 pixels).

  * •

Overall, we observe that the multimodal models are more vulnerable to pixel
perturbation attacks than the state-of-the-art DNNs, which we suspect to be
linked to the way multimodal models and unimodal DNNs are trained.

##  2\. Related Works

###  2.1. Advesarial Attacks on multimodal models

In recent years, multimodal models (Radford et al., 2021; Haas et al., 2023;
Jia et al., 2021; Chen et al., 2022; Li et al., 2022; Xu et al., 2022; Pham et
al., 2023; Zhong et al., 2022), which combine information from different
modalities such as text, image, and audio, have emerged rapidly as powerful
tools across various applications ranging from natural language processing and
computer vision to speech recognition. Previous research studied attacking
techniques on multimodal models and tried to identify certain vulnerabilities
in those models.

An adversarial perturbation attack can be defined by the approach used for
perturbing the input. For instance, the works (Cao et al., 2023; Fort, 2021)
explored the CLIP (Radford et al., 2021) multimodal model’s robustness against
typographical attacks—the attacker introduced a textual sticker containing a
word on the image to misclassify the image to the class written on the textual
sticker.

The paper by Qiu et al. (Qiu et al., 2022) analyzed the robustness of
multimodal models by adding noise/blur to the whole image to create the image
perturbations, while in our work, we focus on perturbing only a limited number
of pixels. Similarly, the study by Mao et al. (Mao et al., 2022) quantified
the robustness of the CLIP model by creating adversarial images using white-
box PGD attack (Madry et al., 2017). They also presented two defense methods
for mitigating that kind of attack. Moreover, some works, such as Yang et al.
(Yang et al., 2021), focus on adversarial perturbation against multimodal
models using only the PGD white-box scenario. Similarly, in the work by
Schlarmann et al. (Schlarmann and Hein, 2023), they launch imperceptible
attacks using the same PGD white-box technique on a single open-source model
(OpenFlamingo (Awadalla et al., 2023)). In contrast, in our study, we limit
the adversary’s power to considering the black-box scenarios only. A more
different attack is presented in the work by Freiberger et al. (Freiberger et
al., 2023) where it exploited the vulnerability of the modality gap in a
contrastive learning approach by generating adversarial examples using the
evolution strategy for searching generative models’ latent space.

###  2.2. Adversarial attacks using Genetic Algorithms (GA)

Certain adversarial perturbation attacks on Deep Learning models focused only
on the white-box scenarios (Szegedy et al., 2013; Goodfellow et al., 2014;
Kurakin et al., 2016; Madry et al., 2017; Papernot et al., 2016; Carlini and
Wagner, 2017; Moosavi-Dezfooli et al., 2016; Athalye et al., 2018), while
others evolved in direction of black-box attacks (Su et al., 2019; Chen et
al., 2017; Brendel et al., 2017; Tu et al., 2019). Moreover, for improving
their attacks, there are studies where genetic algorithms have been introduced
(Su et al., 2019; Wu et al., 2021). For instance, the study by Su et al. (Su
et al., 2019) generated the adversarial examples against the DNNs by
perturbing a specific number of pixels, thus using an $L_{0}$ norm.
Furthermore, the paper by Jere et al. (Jere et al., 2019) used evolutionary
strategies (including Differential Evolution) for generating _scratch_
perturbations on the image against DNNs only. Similarly, in terms of applying
the evolutionary algorithms but using sparse adversarial perturbations through
simultaneous minimization of $L_{0}$ and $L_{2}$ norms, the research by
Williams et al. (Williams and Li, 2023) restricted the space of potential
perturbations quite aggressively and focused on studying only sparsely
distributed pixel perturbations. So there is no exploration regarding the
impact of contiguous shapes of the perturbation on attack performance.

In addition, both studies (Su et al., 2019; Williams and Li, 2023) performed
the pixel perturbations on the original images. In this work, we perturb the
preprocessed image such that our attack is neither influenced by the dimension
of the original image nor the particular preprocessing routines. Therefore, we
investigate the attack based solely on the different models. Also, the work by
Qiu et al. (Qiu et al., 2021) included an analysis of creating the image
perturbations with respect to the $L_{\infty}$-norm in a black-box setup,
using different evolutionary strategies and characterizing which evolutionary
algorithm is more suitable for this kind of attack. More recently, the paper
by Nam et al. (Nam and Kil, 2023) explored the one-pixel idea but using an
exhaustive search procedure, and the perturbations were identified directly on
the original image while we are attacking the preprocessed images. In the
study by Ghosh et al. (Ghosh et al., 2022), the differential evolution (DE)
(Storn and Price, 1997) technique generated the pixel-perturbations in a
black-box setup. The attack was launched against relatively old CNN models
(VGG16 (Simonyan and Zisserman, 2014), GoogleNet (Szegedy et al., 2015),
InceptionV (Szegedy et al., 2016), ResNet-50 (He et al., 2016)), while we
attack more recently proposed multimodal models. Since we want to compare
their robustness with established unimodal deep neural networks, we also
compare them against the ResNet-50 (He et al., 2016) and the VAN (Visual
Attention Network (Guo et al., 2023), which used Dilated Convolution Layers
(Yu and Koltun, 2015)).

###  2.3. Adversarial Patch Attacks

The idea of perturbing pixels in the form of a patch is not new, and it was
already tested before in the white-box (Brown et al., 2017; Karmon et al.,
2018) and black-box setups (Brown et al., 2017; Zhou et al., 2021). For
instance, the work by Alrasheedi et al. (Alrasheedi and Zhong, 2023) explored
the problem of making the image perturbations imperceptible and also found
certain boundaries in the width limit. However, the entire attack was launched
in a white-box setup, while we focus on black-box attacks since they are much
closer to real-world scenarios. The study by Wei et al. (Wei et al., 2022)
performed the attacks in a black-box manner by applying reinforcement learning
as an optimization technique to simultaneously find the best position and
perturbation values for the patch. This process is similar to what we try to
achieve in our experiments, but we use the DE approach to evolve our
perturbations. Different methods were used in the context of patch attacks by
even hiding the patch as a QR code (Chindaudom et al., 2020, 2022), such that
the pixel perturbations became less suspicious to the human eye. Moreover,
these patch adversarial attacks were also examined concerning object detection
models using aerial images (Tang et al., 2023) and deep hashing models (Hu et
al., 2021b).

Lastly, the research community relied on GANs to generate adversarial patches
for image classification (Bai et al., 2021; Demir and Unal, 2018; Liu et al.,
2019) or object-detection models (Hu et al., 2021a; Pavlitskaya et al., 2022).
With this technique, the recent paper by Zhou et al. (Zhou et al., 2023)
showed a non-targeted attack against the multimodal model CLIP (Radford et
al., 2021). However, we do not train GAN models since we try to limit the
attacker’s power (including the computational one).

##  3\. Attack model

Model

Figure 1. Threat Model Visualization

We analyze the black-box scenario of an adversary with access to Artificial
Intelligence (AI) models during inference, so the attacker has access only to
the model’s input and output. Figure 1 presents a visualization of the
described threat model. To formalize the attack, let the function
$f:\\{0,...,255\\}^{w\times h\times 3}\rightarrow\mathbb{[}0,1]^{C}$ represent
the model classification which takes as an input $x\in\\{0,...,255\\}^{w\times
h\times 3}$ representing an RGB image with dimension $w$ and $h$ for width and
height respectively. The output corresponds to a probability distribution for
the $C$ classes. The maximum probability from the list of all given classes
$C$ yields the label (or textual description)
$y_{original}=\arg\max_{i\in\\{1,\ldots,C\\}}f_{i}(\mathbf{x})$. We consider
two possible pixel-perturbation attacks. In the first one, the goal of the
attacker is to find a perturbation ($\Delta\mathbf{x}$) such that the new
image is classified as a specific target label (targeted attack noted as
_tar_). The attack is presented in the Equation 1 where the adversary is
maximizing the target label ($p_{tar}$) probability by perturbing at most $d$
pixels.

(1) $\displaystyle\text{find}\;\Delta\mathbf{x}\;\text{such
that}\begin{cases}p_{\text{tar}}=\underset{\Delta\mathbf{x}}{\max}\;f_{i:=\text{tar}}(\mathbf{x}+\Delta\mathbf{x}),\\\
\text{and}\;||\Delta\mathbf{x}||_{0}\leq d\end{cases}$

The second one, untargeted attack (noted as _untar_), has as the goal to do
random misclassification (maximizing any other class probability) as it is
described similarly by the Equation 2.

(2) $\displaystyle\text{find}\;\Delta\mathbf{x}\;\text{such
that}\begin{cases}p_{untar}=\underset{\Delta\mathbf{x}}{\max}\;f_{i\neq
original}(\mathbf{x}+\Delta\mathbf{x}),\\\
\text{and}\;||\Delta\mathbf{x}||_{0}\leq d\end{cases}$

The perturbation of our attack is created for the preprocessed image instead
of the original image. In fact, if the adversary perturbs one pixel in a
$32\times 32$ image (dimension of the original image), then it perturbed
10.24% of the image (approach presented in the paper by Su et al. (Su et al.,
2019)), while one pixel in a preprocessed image of dimension $224\times 224$
represents less than 0.002% of the image area. Table 1 shows the exact
percentages of the perturbed area for the target models. Furthermore,
attacking this small area of the image makes the attack less perceptible for
the human eye and hard to detect by the systems, hence having potentially
catastrophic consequences in a sensitive application. Moreover, this setup is
closer to the real-world scenario, where companies usually release new AI
models via an API, which takes input images of a specific dimension (the
company also need to release the Image Processor of the model) and outputs the
probabilities for specific classes/textual descriptions.

Lastly, the attack is limited to $Q$ number of queries to the model. Hence,
the attacker creates the perturbations using an evolutionary algorithm
(differential evolution—DE (Storn and Price, 1997)) and makes a maximum of $Q$
queries.

Table 1. Percentage of the perturbed area in the attacked image Preprocessed image dimension \Pixels | 4 pixels | 9 pixels | 16 pixels
---|---|---|---
ALIGN - 289 x 289 | 0.00478% | 0.01077% | 0.01915%
AltCLIP, CLIP-ViT-B/32, GroupViT - 224 x 224 | 0.00797% | 0.01793% | 0.03188%
VAN-base, ResNet-50 - 224 x 224 | 0.00797% | 0.01793% | 0.03188%

##  4\. Evolutionary Attacks

This section presents the encoding for each type of attack used (_Sparse_ and
_Contiguous Attacks_), followed by how we initialize the genetic algorithm
setup. The evolutionary process is then explained in depth, covering mutation,
crossover, and fitness functions.

###  4.1. Sparse Attack

Figure 2. Agent encoding for the Sparse Attack

As mentioned in Section 3, we use Su et al. (Su et al., 2019)’s approach, the
differential evolution (DE) (Storn and Price, 1997) genetic algorithm, as an
optimization method for finding the best perturbations for misclassification
tasks. Encodings of the perturbations in specific vectors (noted as agents or
population members) become accessible to the evolutionary approach, which
tries to iteratively yield better perturbations over time. For the Sparse
Attack, we can use five values,
$[x_{coord},y_{coord},r_{value},g_{value},b_{value}]$, to describe a pixel,
namely its positioning by the coordinates and its RGB values. As a result, an
agent is represented by $N$ flattened and concatenated vectors, where $N$ is
the number of pixels for which the perturbation is being performed. Section 2
illustrates the encoded perturbation vector we use for this type of attack. An
important property of the _Sparse Attack_ is that there are no constraints on
the spatial distribution of the pixels used for the attack.

###  4.2. Contiguous Attacks

In contrast to the _Sparse Attack_ , we can impose constraints on the spatial
distribution of the pixels used for an attack. In particular, we choose to
force pixels to form spatially contiguous regions in the image (i.e., anti-
diagonal, diagonal, column, row, patch). Since we also address the question of
how different shapes of contiguous pixels can affect the model robustness, we
create appropriate encodings for the _Contiguous Attack_. We decrease the
number of the values that must be mutated for other shapes and store only the
coordinates of the first pixel from that shape. Thus, we have the generic
encoding pattern
$[x_{coord},y_{coord},r_{value_{1}},g_{value_{1}},b_{value_{1}},...,r_{value_{n}},g_{value_{n}},b_{value_{n}}]$,
where the $r$, $g$, $b$ values represent the colors for each perturbed pixel
in the specific shape. The first two values are the x and y coordinates,
representing different starting pixels:

Anti-Diag Attack:

the x and y coordinates of the lowest pixel of the shape.

Diagonal Attack:

the x and y coordinates of the up-most pixel of the diagonal.

Column Attack:

the x and y coordinates of the up-most pixel of the column.

Row Attack:

the x and y coordinates of the left-most pixel of the row.

Patch Attack:

the x and y coordinates of the up-left corner pixel of the patch.

As an example on how the pixels are positioned and what is their order in the
encoding vector depending on the attack type, we illustrate the case where we
perturb four pixels in Figure 3.

(a) Anti-Diagonal

(b) Diagonal

(c) Column

(d) Row

(e) Patch

Figure 3. Spatial arrangement of pixels in Contiguous Attacks for exemplary
case of four perturbed pixels

The idea of _Contiguous Attacks_ is to actually try to perturb multiple visual
patches (depending on the length of the shape) from the ViT model. So, suppose
an attacker chooses the row shape. In that case, it can perturb a number of
consecutive visual patches (or tokens) differently depending on their
alignment with the row of perturbed pixels. Figure 4 shows an example of
perturbing 6 pixels in different shapes. We have most perturbations in the
middle visual patch for the _Row Attack_ and less for the beginning and final
patch. Hence, we want to test if it is better to perturb less information but
from many very different visual patches (_Sparse Attack_), perturb less
information to some visual patches and more in some others (_Row Attack_), or
perturb more in a smaller number of visual patches (_Patch Attack_). Thus, we
do not focus on the evolutionary method for improving the position of those
pixel shapes. However, we emphasize that if we use an evolutionary approach to
find the best perturbation based on the provided shape, we can still exploit
how ViT creates the image tokens.

(a) Sparse Attack

(b) Contiguous - Row Attack

(c) Patch Attack

Figure 4. Pixel perturbations for different attacks on patch-based models
(e.g. Vision Transformers, ViTs)

###  4.3. Initialization

Similarly with the study by Su et al. (Su et al., 2019), for the
initialization process of the population, we do the following:

  * •

for $x_{coord}$ and $y_{coord}$ we assign uniformly random integer values
based on the dimension of the attacking image (i.e., $U(0,max_{dim})$)

  * •

for RGB values, since their values are between 0 and 255, we assign them as a
random value from a normal distribution $N(\mu=128,\sigma=127)$

###  4.4. Evolutionary process

After initializing the first population (generating $\mathcal{P}$
perturbations, also called agents), we calculate the fitness value for each
one. This is computed by applying a fitness function. As mentioned in Section
3, we have two possible attack scenarios. Hence, we use two hinge loss
functions, inspired by the paper by Chen et al. (Chen et al., 2017) and
exemplified in the Equations 3 and 4—for targeted attacks $F_{tar}$ and
$F_{untar}$ for the non-targeted attacks. Furthermore, we maximize the fitness
function of every new generation. Since we assume a black-box attack scenario,
we need to consider that we can only access the probabilities for each class.
For instance, in our case, the $p[x]$ means we obtain the softmax
(probability) value from the model for class $x$. Also, as these functions
ascend, sorting the agents by their fitness values, at position zero in the
population yields the best perturbation.

(3) $\displaystyle F_{tar}$
$\displaystyle=p_{tar}-\max_{i\neq\text{tar}}\,p_{i}$ (4) $\displaystyle
F_{untar}$
$\displaystyle=\max_{i\neq\text{original}}\,p_{i}-p_{\text{original}}$

(5) $\displaystyle
agent_{cand}=population_{0}+mutation_{rate}\cdot(population_{r1}-population_{r2})$
where $r1$ and $r2$ are random indices that are within the population size

For the next iteration, we create $\mathcal{P}$ other new candidates
($agent_{cand}$). Thus, for each member in the population, we apply the
mutation function from Equation 5 and the crossover operation, using the
exponential strategy approach (see Appendix A). A list with the parameter
values used in the DE is presented in the Section 5.1.

After obtaining the candidates, we perform $\mathcal{P}$ queries to the model
and compute the fitness values based on the probabilities obtained. Therefore,
we decide which $\mathcal{P}$ agents out of $2\cdot\mathcal{P}$ (old and new
agents) survive for the next generation, based on their fitness values—the
$\mathcal{P}$ agents with the highest fitness values are selected. We repeat
this process $\mathcal{G}$ times, as we are limited to the maximum number of
queries we can do. The idea of the evolutionary approach is generating
increasingly better perturbations over time and, as a result, have at least
one perturbation that categorizes the attack as successful. Further details
about how we quantify the success of an attack are explained in Section 5.2.

##  5\. Experiments

This section presents the images used for the attack and the values for the
parameters used. Moreover, we describe the evaluation metrics and baselines.
Lastly, we present our numerical results.

###  5.1. Experiment setup

We evaluate our attacks on samples from ImageNet (Krizhevsky et al., 2012). It
contains images with a variable resolution length and, thus, a different
number of pixels. As explained in Section 3, we attack the preprocessed image
before it is fetched into the model; we do not apply any preprocessing
operations to the original images from the dataset to scale all of them to a
fixed resolution, $256\times 256$ resolution, as it is done in the original
paper (Krizhevsky et al., 2012). The number of labels for this dataset is
1000.

Table 2. Performance comparison between Multimodal Models and DNNs Model | Accuracy | Model Type
---|---|---
ALIGN (Jia et al., 2021) | 0.5442 | Multimodal
CLIP_ViT-B32 (Radford et al., 2021) | 0.5601 | Multimodal
AltCLIP (Chen et al., 2022) | 0.6993 | Multimodal
GroupViT (Xu et al., 2022) | 0.3187 | Multimodal
ResNet-50 (He et al., 2016) | 0.8087 | DNN
VAN-base (Guo et al., 2023) | 0.8022 | DNN

The models used can be split into two branches based on their architecture. In
the first category, we have the state-of-the-art multimodal models; in the
other, we have the state-of-the-art DNNs, trained on the ImageNet dataset.
Table 2 shows the list of all the models and their measured accuracy for
10,000 images from the testing dataset of ImageNet. The motivation behind this
assessment is given by the connection between robustness, the architecture
used, and the versatility of the models (multimodal models can be applied to
different datasets without the need for extensive retraining due to the zero-
shot property, while DNNs usually require training on a specific dataset and
are optimized for that one only).

In order to use the multimodal models for image classification, we rely on
appropriate input captions. In particular, we append the string _”a photo of”_
in front of the original label of the image to enable zero-shot image
classification.

As observed in Table 2, the models perform very differently, and we want an
extensive analysis of their robustness. Thus, to avoid any possible bias, we
extract 100 random correctly classified images for each model. Those are part
of the testing dataset; consequently, the models did not see those images
during their training phase. Then, after extracting those images, we store the
preprocessed version based on each model. Hence, we test different attack
setups on the same images to quantify the attack’s effectiveness.

Based on empirical observations, we observe that the best parameters for the
Differential Evolution are $mutation_{rate}=0.55$ and $crossover_{rate}=0.8$.
Moreover, the population of $\mathcal{P}=300$ gives the best results when we
limit the number of iterations to $\mathcal{G}=100$. As a remark from Section
3, the total number of queries an attacker needs to do to perturb an image is
at most $Q$, which is translated to $\mathcal{P}\cdot\mathcal{G}\leq Q$, which
means $300\cdot 100\leq 30,000\leq Q$.

###  5.2. Evaluation Metrics and Baselines

Attack performance is quantified by computing the _SR_ (success rate) as an
empirical measurement. Depending on the type of attack, we define the success
of the attack as follows: targeted attacks involves how many images managed to
be misclassified as the target label, while for untargeted scenarios, we count
the number of images that are misclassified as a different label than the
original one. As mentioned Section 5.1, for our setup, it translates to using
100 images and computes how many of them we manage to attack successfully.
Then, since the SR is a value between 0 and 1, we divide it by 100 (the number
of images). Thus, an SR value of 1 means we create successful perturbations
for all images.

Lastly, to investigate the effectiveness of the evolutionary attacks, we
create random versions for each type of attack and use them as baselines.
These _Random Attacks_ use the same parameters as the evolutionary one
($\mathcal{P}=300$ and $\mathcal{G}=100$), but the difference is in generating
those random agents without an evolutionary approach. At each iteration, we
generate $\mathcal{P}$ agents according to their respective encodings
presented in Sections 4.1 and 4.2, and the values are random and set based on
the initialization values presented in Section 4.3. However, some corrections
are applied if the values are outside the ranges (especially for the RGB
values).

###  5.3. Results

Figure 5. Targeted Attacks

Figure 6. Untargeted Attacks

We evaluate the performances of attacks based on the SR metric and against the
baselines described in Section 5.2. For instance, Figure 5 presents the SR for
the targeted attacks for each selected model and the shape of the attacks,
depending on the number of pixels perturbed. At first glance, we observe that
for the ALIGN model, the most successful attack is the _Patch Attack_ , while
for the others, it is the _Sparse Attack_. Furthermore, the best attacks
against the multimodal models achieve an SR higher than 0.35, while for the
DNNs, the SR is generally lower except for ResNet-50 with the _Sparse Attack_
and perturbing 16 pixels. Overall, the progression of SR values with
increasing perturbed pixels for the DE attacks is steeper than for the random
baseline attacks. Thus demonstrating that the SR of the proposed attacks is a
function of the number of perturbed pixels with an increased number of pixels
yielding higher SR.

For ALIGN, Figure 5 shows that the _Patch Attack_ performs the best (0.74),
but the value against the _Sparse Attack_ for 4 pixels is relatively close
(0.58 for _Patch_ and 0.57 for _Sparse_). However, the transition from 4 to 6
pixels is steeper for the _Patch Attack_ , starting from 0.57 and reaching
0.71, while for the _Sparse Attack_ , it starts from 0.56 and reaches 0.64 for
the 9 pixels. Nevertheless, for the transition from 9 to 16 pixels, the slopes
for both attacks seem similar. Moreover, all the _Evolutionary Attacks_ manage
to beat the _Random Attacks_ , with one exception: _Random Patch Attack_ ,
which has an SR close to the ones of the _Contiguous Attacks_. Besides this
one, the rest of the _Random_ ones only achieve an SR below 40%, even after
increasing the number of pixels.

Figure 5 highlights that for AltCLIP, the _Sparse Attack_ is the best attack
for 4 and 9 pixels with SR of 0.35 and 0.46, respectively. For 16 pixels, the
most successful attack is the _Patch Attack_ , with the SR almost 0.5. All the
_Contiguous attacks_ outperform the _Random_ ones, with the gap between those
becoming more prominent with more pixels. Initially, the difference is
negligible, while for 16 pixels, the gap between the _Row Attack_ (worst
_Contiguous Attack_) and _Random Patch_ (the best _Random Attack_) is
approximately 0.7. However, _Random Patch_ is still the best _Random attack_ ,
usually above 3-5% from the second-best _Random Attack_. Additionally, there
is a big difference between the _Patch_ and _Sparse Attacks_ and the remaining
_Contiguous Attacks_ (10-25%).

Based on Figure 5, CLIP is more robust to our attacks than ALIGN but less
robust when compared against the other multimodal models (GroupViT, AltCLIP).
The best attack remains the _Sparse Attack_ with values from 0.5 for 4 pixels
to 0.68 for 16 pixels, followed by the _Patch Attack_ with values from 0.33 to
0.50. The third best attack is _Row_ one, and for the 16 pixels, it is
followed behind with 2% by _Column_ one.

Similar to CLIP, Figure 5 shows that for GroupViT, the top two best attacks
are the _Sparse Attack_ , reaching an SR of more than 55% for all pixels, and
the _Patch Attack_ , achieving a maximum SR of 47%. Nonetheless, there is a
gap between those two attacks and the others: for 16 pixels, the _Row Attack_
reaches a value of 0.45 and below, with 0.02 less than the _Random Sparse
Attack_. The best _Random Attack_ is the _Sparse_ one, which outperforms the
_Patch_ with approximately 10% for 16 pixels. Similarly, all the other
_Contiguous Attacks_ are better than the other _Random_ ones, except for
_Random Sparse_.

From Figure 5 we observe that both DNNs have similar behavior: they are more
robust than any multimodal model. With ResNet-50, the SR value does not exceed
0.45 for the best attack (_Sparse Attack_), while for the VAN-base, the best
one stops at 0.25. Moreover, the trend regarding the best two attacks remains
the same as for CLIP and GroupViT, with a significant difference regarding the
gaps between those two attacks and the rest—the discrepancy is much smaller
(approx. 5% for ResNet-50 and less than 2% for the VAN-base). Moreover, the
_Random Attacks_ behave in the same manner as the rest of the _Contiguous
Attacks_ with values fluctuating within the limit of 5% for both models.

Results for the untargeted attacks are presented in Figure 6. Overall, the
trends remain the same as the targeted attacks, but the SR value is increased
by approximately 25% for ALIGN, 15% for AltCLIP, and 10% for CLIP and
GroupViT. Interestingly, a much smaller increase is observable for both DNNs
(approximately 5%). ALIGN appears to have the worst security against the
_Patch Attack_ for a more significant number of pixels since, using 16 pixels,
we obtain a result of 99% SR. Moreover, the _Random Patch Attack_ is in the
third position, with an SR close to the _Sparse Attack_ for 16 and achieving a
score of 0.85. For AltCLIP, the difference between the _Random Attacks_ and
the _Evolutionary_ ones increases proportionally with the number of pixels. In
addition, similar to the targeted attack, the _Patch Attack_ is slightly
better than the _Sparse_ one for the 16 pixels, while for 4 and 9, the winning
one is the _Sparse Attack_. For CLIP and GroupViTs, the plots for ranking the
effectiveness of the attacks remain the same as for targeted attacks, as is
the case for the DNNs.

##  6\. Discussion

At first glance, based on the results presented in Section 5.3, we can see
that the targeted attack is more challenging than the untargeted one, which is
indeed expected. However, the average difference between those two attacks for
multimodal models differs from the DNNs. Also, the DNNs are more robust to our
attacks than any multimodal models. An important observation is that for all
setups we have at least 2-3 attacks which are better than any of the
baselines.

Our results suggest that multimodal models with ViT (Visual Transformer)
(Dosovitskiy et al., 2020) as an Image Encoder are more prone to the _Sparse_
than the _Patch Attack_. To provide a potential explanation, we describe what
happens when an image is fed to a ViT. Initially, ViT split the images into
visual patches and then considers those as tokens. After this stage, those
tokens are processed similarly to a normal Transformer (Vaswani et al., 2017).
Suppose the adversary perturbs pixels on different visual patches (as you have
in _Sparse Attack_), then it can alter more tokens. Hence, in the self-
attention layer (Vaswani et al., 2017) (core component of a Transformer that
also provides good results in terms of robustness), there is a high
possibility of breaking the semantical meaning information from different
visual patches. However, using any _Contiguous Attack_ , the adversary cannot
perturb as many visual patches as would be possible using the _Sparse Attack_.
Additionally, the _Patch Attack_ outperforms any _Contiguous Attack_ because,
with this shape, we can alter more information in a single visual patch than
other contiguous pixel distributions.

We also question why CNN-based Image Encoders (ALIGN model) are more prone to
_Patch Attacks_. Based on the idea from the previous paragraph, if one piece
of information in the kernel is altered for those types of architectures, it
is a complete disaster for the upcoming max-pooling layers. The altered
information propagates to deeper layers, which try to extract correct
information that can be used as features for classification.

Based on our results, ALIGN (Jia et al., 2021) is the most vulnerable model to
the implemented $L_{0}$-norm perturbations. The best method to attack the
ALIGN model is by using the _Patch Attack_ and not the _Sparse_ one, as it is
for most of the cases in the other models. A potential explanation for this
observation could be because ALIGN uses the EfficientNet (Tan and Le, 2021)
(CNN-based architecture that uses 3x3 Convolutional Layers) as an Image
Encoder. This is why we have a steep increase in the SR for _Patch Attack_
from 4 to 9 pixels, from a patch of 2x2 to 3x3 (representing the same size as
the Convolutional Layer). However, the transition from 9 to 16 pixels is
smoother because we already manage to create a patch with the same dimension
as the Convolutional layer, so the attack does not increase a lot in
effectivness from the previous size (it increased only by 4%, while before we
have an increasing of approximate 10%). Based on this information, the
adversary can deduce some information regarding the architecture used (the
size of the kernel in initial Convolutional layers) in the model and therefore
here we could have a potential privacy issue. Moreover, this vulnerability of
the ALIGN model against the _Patch Attack_ can have serious consequences
since, for the untargeted attack, a patch of 16 pixels obtains a 99% success
rate. We see that the shape of the attack (_Patch_) additionally confirms this
vulnerability since the _Random Patch Attacks_ achieve high scores in both
cases, especially in the untargeted setting, being the third most powerful
attack.

We observe that AltCLIP is more robust than the original implementation of
CLIP. This could be explained because of the architecture used in the Image
Encoder. More precisely, AltCLIP (Chen et al., 2022) uses the ViT-L14, which
has the _dimension of the embedding_ equals to 768, 24 _layers_ , _width_
equals to 1024, and 16 _heads_ , while CLIP (Radford et al., 2021) uses the
ViT-B/32 which has _dimension of the embedding_ equals to 512, 12 _layers_ ,
_width_ equals to 768, and 16 _heads_. Hence, since the embeddings from
AltCLIP are larger, they capture more information, and therefore, it is harder
to fool those systems with the $L_{0}$-norm perturbations. Moreover, since the
AltCLIP Image Encoder is deeper (based on the number of layers), it can
capture more complex patterns and hierarchical features. It may exhibit better
robustness and generalization—this is also proved by the Table 2, where
AltCLIP obtained the best performance compared to the other multimodal models.
However, it seems that there exists a connection between the depth and the
width configuration of the model with larger patches since for both non-
targeted and targeted attacks in the case of 16 pixels, the _Patch Attack_ is
slightly better, and there exists a stagnation in the _Sparse Attack_.
Furthermore, AltCLIP uses the XLM-R (Conneau et al., 2019) as a Text Encoder,
while CLIP uses the masked self-attention transformer (Dosovitskiy et al.,
2020). Although this aspect might influence the robustness in favor of
AltCLIP, the core component that improves the security aspect is the Image
Encoder since we target it with our adversarial examples and do not give
adversarial textual input.

For the second most robust multimodal model, GroupViT (Xu et al., 2022), we
consider that the good results in terms of robustness are determined mainly by
the grouping mechanism used in the Image Encoder. This new architecture uses a
hierarchical group ViT—in short, besides the classical ViT approach, where the
image is initially split into visual patches and projected into tokens (using
Linear Projection), it also contains the Group Learnable Tokens. Those group
tokens describe a different semantic notion; thus, the model also groups more
semantic concepts. Since those tokens are learned during the training phase,
during the grouping semantic phase, the model tries to avoid perturbed visual
tokens; therefore, this mechanism increases the robustness of the model.

In addition, we observe significant differences in the SR between the DNNs and
the rest of the multimodal models. A potential cause for this behavior could
be related to the fact that we evaluate the DNNs on the images with specific
class labels for which they were originally trained. Consequently, these DNNs
contain class-specific features extracted during training on ImageNet
(Krizhevsky et al., 2012) which requires an attacker to perturb more pixels.
In contrast to DNNs, multimodal models were trained on large corpuses of image
and text data without explicit definition of target classes during training
(contrastive learning). Hence, they can be used on any dataset other than the
ones they were trained on. However, we argue that this flexibility comes with
an increased vulnerability to pixel perturbations, which is evident from our
experiments.

Lastly, a potential explanation why VAN-base (Guo et al., 2023) is the most
resilient model is based on its use of Dilated Convolutional Layers (Yu and
Koltun, 2015). Attacks based on contiguous pixels may be thwarted by expanding
the filter’s receptive field when it is widened through introducing gaps
between successive elements—the main feature of dilated convolutions. As a
result, the _Sparse Attack_ appears to be the most effective attack on VAN-
base, while the _Contiguous_ ones behave similarly to _Random Attacks_.
Moreover, the robustness of the VAN-base model surpasses the ResNet-50 (He et
al., 2016), which employs regular convolutional layers, providing further
evidence for the robustness of dilated convolutions.

##  7\. Conclusion and Future Work

Our work analyzes the robustness of four multimodal models and two state-of-
the-art unimodal DNNs against Sparse and Contiguous adversarial examples
defined by the $L_{0}$-norm in a black-box attack scenario. For both types of
attack (targeted and untargeted) and all models, we have at least two
evolutionary attacks that are more effective than the rest—all multimodal
models that use a ViT as an Image Encoder are most vulnerable to the _Sparse
Attack_. In contrast, in the CNN-based multimodal model (ALIGN), the most
effective attack is represented by a contiguous pixel perturbation in the form
of a _Patch_. This attack prove to be so powerful against ALIGN that with an
increased patch dimension of 4x4 (less than 0.02% of the image area), in a
non-targeted scenario, we achieved a 99% SR. Furthermore, based on the overall
SR, we rank the most secure models as follows: VAN-base, ResNet-50, AltCLIP,
GroupViT, CLIP-B/32, and ALIGN. Our results also point towards different
characteristics of AI model architectures (e.g., tokenization, convolutions,
dilated convolutions) that may be responsible for different levels of
robustness against our attacks. When contrasting the results for multimodal
and unimodal models, we observe that there may be an essential trade-off
between the robustness of the model and its adaptability to be used in diverse
tasks (i.e., zero-shot capability). Both of these aspects should be further
investigated in a follow-up study.

For future work, we propose to study the attacks on the $L_{0}$-norm for other
types of multimodal models that focus on object detection and text generation,
such as IDEFICS (Laurençon et al., 2023), Kosmos-2 (Peng et al., 2023), and
Gemini (Team et al., 2023). Moreover, we want to do more investigation on the
contiguous attacks and focus on finding the best hyperparameters for the
evolution algorithm used for those specific patterns, such that we can
construct a systematic search setup, where we can infer different information
regarding the model architecture from the observed attack performance.

## References

  * (1)
  * Alrasheedi and Zhong (2023) Fahad Alrasheedi and Xin Zhong. 2023.  Imperceptible Adversarial Attack on Deep Neural Networks from Image Boundary.  _arXiv preprint arXiv:2308.15344_ (2023).
  * Athalye et al. (2018) Anish Athalye, Nicholas Carlini, and David Wagner. 2018.  Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In _International conference on machine learning_. PMLR, 274–283.
  * Awadalla et al. (2023) Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. 2023\.  Openflamingo: An open-source framework for training large autoregressive vision-language models.  _arXiv preprint arXiv:2308.01390_ (2023).
  * Bai et al. (2021) Tao Bai, Jinqi Luo, and Jun Zhao. 2021.  Inconspicuous adversarial patches for fooling image-recognition systems on mobile devices.  _IEEE Internet of Things Journal_ 9, 12 (2021), 9515–9524.
  * Brendel et al. (2017) Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2017.  Decision-based adversarial attacks: Reliable attacks against black-box machine learning models.  _arXiv preprint arXiv:1712.04248_ (2017).
  * Brown et al. (2017) Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. 2017.  Adversarial patch.  _arXiv preprint arXiv:1712.09665_ (2017).
  * Cao et al. (2023) Liangliang Cao, Bowen Zhang, Chen Chen, Yinfei Yang, Xianzhi Du, Wencong Zhang, Zhiyun Lu, and Yantao Zheng. 2023.  Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness.  _arXiv preprint arXiv:2305.05095_ (2023).
  * Carlini and Wagner (2017) Nicholas Carlini and David Wagner. 2017.  Towards evaluating the robustness of neural networks. In _2017 ieee symposium on security and privacy (sp)_. Ieee, 39–57.
  * Chen et al. (2017) Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 2017.  Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In _Proceedings of the 10th ACM workshop on artificial intelligence and security_. 15–26.
  * Chen et al. (2022) Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, and Ledell Wu. 2022.  Altclip: Altering the language encoder in clip for extended language capabilities.  _arXiv preprint arXiv:2211.06679_ (2022).
  * Chindaudom et al. (2020) Aran Chindaudom, Prarinya Siritanawan, Karin Sumongkayothin, and Kazunori Kotani. 2020.  AdversarialQR: An adversarial patch in QR code format. In _2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR)_. IEEE, 1–6.
  * Chindaudom et al. (2022) Aran Chindaudom, Prarinya Siritanawan, Karin Sumongkayothin, and Kazunori Kotani. 2022.  Surreptitious Adversarial Examples through Functioning QR Code.  _Journal of Imaging_ 8, 5 (2022), 122.
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  _arXiv preprint arXiv:1911.02116_ (2019).
  * Demir and Unal (2018) Ugur Demir and Gozde Unal. 2018.  Patch-based image inpainting with generative adversarial networks.  _arXiv preprint arXiv:1803.07422_ (2018).
  * Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020\.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ (2020).
  * Fort (2021) Stanislav Fort. 2021.  Pixels still beat text: attacking the OpenAI CLIP model with text patches and adversarial pixel perturbations.  _Stanislav Fort [Internet]_ 5 (2021).
  * Freiberger et al. (2023) Matthias Freiberger, Peter Kun, Anders Sundnes Løvlie, and Sebastian Risi. 2023.  CLIPMasterPrints: Fooling Contrastive Language-Image Pre-training Using Latent Variable Evolution.  _arXiv preprint arXiv:2307.03798_ (2023).
  * Ghosh et al. (2022) Arka Ghosh, Sankha Subhra Mullick, Shounak Datta, Swagatam Das, Asit Kr Das, and Rammohan Mallipeddi. 2022.  A black-box adversarial attack strategy with adjustable sparsity and generalizability for deep image classifiers.  _Pattern Recognition_ 122 (2022), 108279.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ (2014).
  * Guo et al. (2023) Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming-Ming Cheng, and Shi-Min Hu. 2023.  Visual attention network.  _Computational Visual Media_ 9, 4 (2023), 733–752.
  * Haas et al. (2023) Lukas Haas, Silas Alberti, and Michal Skreta. 2023.  Learning Generalized Zero-Shot Learners for Open-Domain Image Geolocalization.  arXiv:2302.00275 [cs.CV]
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 770–778.
  * Hu et al. (2021b) Shengshan Hu, Yechao Zhang, Xiaogeng Liu, Leo Yu Zhang, Minghui Li, and Hai Jin. 2021b.  Advhash: Set-to-set targeted attack on deep hashing with one single adversarial patch. In _Proceedings of the 29th ACM International Conference on Multimedia_. 2335–2343.
  * Hu et al. (2021a) Yu-Chih-Tuan Hu, Bo-Han Kung, Daniel Stanley Tan, Jun-Cheng Chen, Kai-Lung Hua, and Wen-Huang Cheng. 2021a.  Naturalistic physical adversarial patch for object detectors. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 7848–7857.
  * Jere et al. (2019) Malhar Jere, Loris Rossi, Briland Hitaj, Gabriela Ciocarlie, Giacomo Boracchi, and Farinaz Koushanfar. 2019.  Scratch that! An evolution-based adversarial attack against neural networks.  _arXiv preprint arXiv:1912.02316_ (2019).
  * Jia et al. (2021) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021.  Scaling up visual and vision-language representation learning with noisy text supervision. In _International conference on machine learning_. PMLR, 4904–4916.
  * Karmon et al. (2018) Danny Karmon, Daniel Zoran, and Yoav Goldberg. 2018.  Lavan: Localized and visible adversarial noise. In _International Conference on Machine Learning_. PMLR, 2507–2515.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012.  Imagenet classification with deep convolutional neural networks.  _Advances in neural information processing systems_ 25 (2012).
  * Kurakin et al. (2016) Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016.  Adversarial machine learning at scale.  _arXiv preprint arXiv:1611.01236_ (2016).
  * Laurençon et al. (2023) Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M Rush, Douwe Kiela, et al. 2023\.  Obelisc: An open web-scale filtered dataset of interleaved image-text documents.  _arXiv preprint arXiv:2306.16527_ (2023).
  * Li et al. (2022) Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In _International Conference on Machine Learning_. PMLR, 12888–12900.
  * Liu et al. (2019) Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. 2019.  Perceptual-sensitive gan for generating adversarial patches. In _Proceedings of the AAAI conference on artificial intelligence_ , Vol. 33. 1028–1035.
  * Madry et al. (2017) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017.  Towards deep learning models resistant to adversarial attacks.  _arXiv preprint arXiv:1706.06083_ (2017).
  * Mao et al. (2022) Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. 2022.  Understanding zero-shot adversarial robustness for large-scale models.  _arXiv preprint arXiv:2212.07016_ (2022).
  * Moosavi-Dezfooli et al. (2016) Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016.  Deepfool: a simple and accurate method to fool deep neural networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2574–2582.
  * Nam and Kil (2023) Wonhong Nam and Hyunyoung Kil. 2023.  AESOP: Adjustable Exhaustive Search for One-Pixel Attacks in Deep Neural Networks.  _Applied Sciences_ 13, 8 (2023), 5092.
  * Papernot et al. (2016) Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. 2016.  The limitations of deep learning in adversarial settings. In _2016 IEEE European symposium on security and privacy (EuroS &P)_. IEEE, 372–387.
  * Pavlitskaya et al. (2022) Svetlana Pavlitskaya, Bianca-Marina Codău, and J Marius Zöllner. 2022.  Feasibility of inconspicuous GAN-generated adversarial patches against object detection.  _arXiv preprint arXiv:2207.07347_ (2022).
  * Peng et al. (2023) Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023.  Kosmos-2: Grounding Multimodal Large Language Models to the World.  _arXiv preprint arXiv:2306.14824_ (2023).
  * Pham et al. (2023) Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, et al. 2023\.  Combined scaling for zero-shot transfer learning.  _Neurocomputing_ 555 (2023), 126658.
  * Qiu et al. (2021) Hao Qiu, Leonardo Lucio Custode, and Giovanni Iacca. 2021.  Black-box adversarial attacks using evolution strategies. In _Proceedings of the Genetic and Evolutionary Computation Conference Companion_. 1827–1833.
  * Qiu et al. (2022) Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, and Mu Li. 2022.  Are Multimodal Models Robust to Image and Text Perturbations?  _arXiv preprint arXiv:2212.08044_ (2022).
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International conference on machine learning_. PMLR, 8748–8763.
  * Schlarmann and Hein (2023) Christian Schlarmann and Matthias Hein. 2023.  On the adversarial robustness of multi-modal foundation models. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 3677–3685.
  * SciPy Contributors (2024) SciPy Contributors. Accessed: 2024.  _scipy.optimize.differential_evolution_.
  * Simonyan and Zisserman (2014) Karen Simonyan and Andrew Zisserman. 2014.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ (2014).
  * Storn and Price (1997) Rainer Storn and Kenneth Price. 1997.  Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.  _Journal of global optimization_ 11 (1997), 341–359.
  * Su et al. (2019) Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019.  One pixel attack for fooling deep neural networks.  _IEEE Transactions on Evolutionary Computation_ 23, 5 (2019), 828–841.
  * Szegedy et al. (2015) Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015.  Going deeper with convolutions. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1–9.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2818–2826.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ (2013).
  * Tan and Le (2021) Mingxing Tan and Quoc Le. 2021.  Efficientnetv2: Smaller models and faster training. In _International conference on machine learning_. PMLR, 10096–10106.
  * Tang et al. (2023) Guijian Tang, Tingsong Jiang, Weien Zhou, Chao Li, Wen Yao, and Yong Zhao. 2023.  Adversarial patch attacks against aerial imagery object detectors.  _Neurocomputing_ 537 (2023), 128–140.
  * Team et al. (2023) Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023\.  Gemini: a family of highly capable multimodal models.  _arXiv preprint arXiv:2312.11805_ (2023).
  * Tu et al. (2019) Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, and Shin-Ming Cheng. 2019.  Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 742–749.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wei et al. (2022) Xingxing Wei, Ying Guo, Jie Yu, and Bo Zhang. 2022.  Simultaneously optimizing perturbations and positions for black-box adversarial patch attacks.  _IEEE transactions on pattern analysis and machine intelligence_ (2022).
  * Williams and Li (2023) Phoenix Neale Williams and Ke Li. 2023.  Black-Box Sparse Adversarial Attack via Multi-Objective Optimisation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 12291–12301.
  * Wu et al. (2021) Chenwang Wu, Wenjian Luo, Nan Zhou, Peilan Xu, and Tao Zhu. 2021.  Genetic algorithm with multiple fitness functions for generating adversarial examples. In _2021 IEEE Congress on Evolutionary Computation (CEC)_. IEEE, 1792–1799.
  * Xu et al. (2022) Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. 2022.  Groupvit: Semantic segmentation emerges from text supervision. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 18134–18144.
  * Yang et al. (2021) Karren Yang, Wan-Yi Lin, Manash Barman, Filipe Condessa, and Zico Kolter. 2021.  Defending multimodal fusion models against single-source adversaries. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 3340–3349.
  * Yu and Koltun (2015) Fisher Yu and Vladlen Koltun. 2015.  Multi-scale context aggregation by dilated convolutions.  _arXiv preprint arXiv:1511.07122_ (2015).
  * Zhong et al. (2022) Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. 2022\.  Regionclip: Region-based language-image pretraining. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 16793–16803.
  * Zhou et al. (2021) Xingyu Zhou, Zhisong Pan, Yexin Duan, Jin Zhang, and Shuaihui Wang. 2021.  A data independent approach to generate adversarial patches.  _Machine Vision and Applications_ 32 (2021), 1–9.
  * Zhou et al. (2023) Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, and Hai Jin. 2023.  Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning. In _Proceedings of the 31st ACM International Conference on Multimedia_. 6311–6320.

##  Appendix A Exponential Strategy in Differential Evolution

Algorithm 1 was used for the Crossover operation - Exponential Strategy in the
Differential Evolution. The idea of this algorithm is to recombine the agents
based on some $crossover_{rate}$—how much information from the old agents
($agent_{old}$) should also be preserved in the new ones ($agent_{cand}$).
Thus, we directly modify the new agents in the $agent_{cand}$ vector and
return it for further processing. This code was adapted from the Differential
Evolution implemented in Scipy (SciPy Contributors, 2024).

Algorithm 1 Exponential Strategy (SciPy Contributors, 2024)

1:function Crossover Exp($agent_{old},agent_{cand},crossover_{rate}$)

2: $big_{r}\leftarrow\text{random index value of the }agent_{cand}$

3: $i\leftarrow 0$

4: while $i<\text{size of }agent_{cand}$ do

5: $r\leftarrow$ random number between 0 and 1

6: if $crossover_{rate}<r$ then

7: break

8: $agent_{cand}[big_{r}]\leftarrow agent_{old}[big_{r}]$

9: $big_{r}\leftarrow(big_{r}+1)\mod agent_{cand}\\_size$

10: $i\leftarrow i+1$

11: return $agent_{cand}$