# Differential Evolution based Dual Adversarial Camouflage: Fooling Human Eyes
and Object Detectors

Jialiang Sun  Tingsong Jiang  Wen Yao<EMAIL_ADDRESS>Donghua Wang
Xiaoqian Chen  Defense Innovation Institute, Chinese Academy of Military
Science, No. 53, Fengtai East Street, Beijing 100071, China  College of
Computer Science and Technology, Zhejiang University

###### Abstract

Recent studies reveal that deep neural network based object detectors are
vulnerable to adversarial examples. Exsiting works to fool object detectors
can be categorized into the patch-based and camouflage-based. The camouflage-
based method is more often adopted due to its adaptation to multi-view
scenarios. However, most of them can still be easily observed by human eyes,
which limits their application in the real world. To fool human eyes and
object detectors simultaneously, we propose a differential evolution based
dual adversarial camouflage (DE$\\_$DAC) method. Specifically, we try to
obtain the camouflage texture by the two stage training, which can be rendered
over the surface of the object. In the first stage, we optimize the global
texture to minimize the discrepancy between the rendered object and the scene
images, making human eyes difficult to distinguish. In the second stage, we
design three loss functions to optimize the local texture, which is selected
from the global texture, making object detectors ineffective. In addition, we
introduce the differential evolution algorithm to search for the near-optimal
areas of the object to attack, improving the adversarial performance under
certain attack area limitations. Experimental results show that our proposed
method can obtain a good trade-off between fooling human eyes and object
detectors under multiple specific scenes and objects.

###### keywords:

Object detection , Adversarial attack , Camouflage , Differential evolution

††journal: Journal of Elsevier

##  1 Introduction

Deep neural network (DNN) have made great progresses in many computer vision
tasks such as the image classification [1, 2, 3] and the object detection [4,
5, 6]. However, DNN models proved to be vulnerable to adversarial examples
(AEs) that are intentionally crafted with the perturbations. The phenomenon of
AEs causes a significant threat to the real-world application of DNN models
such as self-driving or facial recognition systems. To explore the robustness
of DNN models and guide their further practical application, adversarial
attack algorithms have been extensively exploited for the majority of tasks,
which aim to study how to generate AEs to fool DNN models.

According to the magnitude of the perturbation, the adversarial attack can be
categorized into $l_{p}$ norm based adversarial attack and unrestricted
adversarial attack. In $l_{p}$ norm based adversarial attack, the $l_{p}$ norm
of the generated perturbation can not exceed a predefined threshold, which can
ensure that the perturbations are imperceptible. Currently, extensive works
about the $l_{p}$ norm based adversarial attack are developed such as fast
gradient sign method (FGSM) [7], projection gradient descent (PGD) [8] attack
and Carlini $\&$ Wagner (C$\&$W) attack [9]. However, the AEs generated by the
$l_{p}$ norm based attacks can not be printed in the real physical-world,
which limits the further application in complicated tasks such as object
detection. Thus in the physical-world attacks, unrestricted adversarial
attacks are more adopted, which do not limit the magnitude or area of the
generated perturbations. However, in unrestricted adversarial attacks, how to
balance the adversarial performance and naturalness towards human eyes is a
challenging problem.

To realize the balance between adversarial performance and naturalness in
unrestricted perturbations, some works have been devoted to image
classification and object detection tasks. In the classification task, some
works tried to change the color or lightness of each image [10, 11]. Another
way to generate AEs is by simulating the natural weather, such as fog and snow
[12]. Duan et al. [13] further combined style transfer technology with the
adversarial attack to make the generated AEs maintain the specific style. In
the object detection task, existing methods can be divided into patch-based
and camouflage-based. The patch-based method tried to optimize a small patch
pasted on the object, while the camouflage-based method utilized the optimized
camouflage to paint on the full surface of the object. In the patch-based
methods, the optimized areas are like stickers that can be pasted on the stop
sign [14], the patch that can be hung on a person [15] or placed on the
background of images [16]. In the camouflage-based method, Hu et al. [17]
proposed the adversarial camouflage that can be worn to fool person detectors.
Wang et al [18]. proposed dual attention suppression (DAS) attack, which
optimizes a smile camouflage that can be pasted on the surface of cars. To
improve the adversarial performance, Wang et al. [19] proposed a full
camouflage attack (FCA), which optimizes a camouflage that can be pasted on
the full surface of cars to fool detectors. In general, the camouflage-based
method is more practical in the real world under multi-view scenarios.

Though these works about balancing the adversarial performance and naturalness
of adversarial examples have been exploited, it is still challenging to
generate the AEs that can fool human eyes and object detectors simultaneously.
First, most of the works in the classification task can not be transferred to
the object detection task directly. It is because that AEs of these methods in
classification tasks are stationary and are designed for single images. For
instance, the adversarial camouflage in [13] has a good visual effect, but it
is not suitable for moving objects like soldiers and cars. In their work, it
is hard to select a suitable image style to match the constant changing
backgrounds. Second, existing camouflages towards object detectors are usually
required to ensure their own naturalness, like the simile face, which can not
be adaptive to the changing environments, resulting in the problem that AEs
are easily observed by human eyes. Third, Wang et al. [20] made some attempts
to generate the AEs that can fool human eyes and object detectors
simultaneously. They generate individual camouflage by extracting the dominant
color features and optimizing a patch that looks close to the dominant color.
However, the generation of individual digital camouflage by extracting
dominant color features is relatively cumbersome, and the optimized patch is
not practical enough for real-world scenarios such as military environments.

Figure 1: (a) is the person without camouflage. (b) is the camouflaged person
by FCA. (c) is the camouflaged person using the pure color constraint. (d) is
the camouflage produced by DAC. (e) shows the final detection performance of
(a) and (d) in the specific scene.

To bridge the gap, given the specific scene, we propose an end-to-end
differentiable evolution based dual adversarial camouflage (DE$\\_$DAC)
pipeline, which can fool human eyes and object detection simultaneously.
Specifically, treating the generation of camouflage as the optimization of the
texture that can be painted on the surface of objects, we design a two stage
method based on the neural render. In the first stage, given the training
dataset, we select several background images as reference scenes. We train the
global texture to make the rendered image close to the scene images. After the
first stage ends, the rendered object can be well integrated with the scene,
which makes the human eyes uneasy to distinguish. In the second stage, we
select the local texture from the optimized texture, the number of which can
be set flexibly. We design three loss functions to optimize the local texture.
The adversarial loss is to enhance the adversarial performance. The smooth
loss can be utilized to ensure the printness of the final texture, while the
color loss is developed to minimize the MSE between the local texture and the
global texture. The final texture is the combination of the global texture and
local texture. The whole process is denoted as dual adversarial camouflage
(DAC). To further improve the performance of our proposed method, under the
same number of faces of the model, we introduce differential evolution (DE)
algorithm to search for the near-optimal attack area. In addition, motivated
by the recently developed robot that can change color in real-time according
to the background [21], we further study the performance of adaptive
DE$\\_$DAC (A-DE$\\_$DAC). One example of our proposed method is presented in
Figure 1. Figure 1 (c) is the optimized camouflage adapted to the specific
scenes to fool human eyes. Figure 1 (d) is the final camouflage when
adversarial performance is considered in the second optimization stage. Figure
1 (e) presents the detection comparison of the clean image and the optimized
camouflage by us. From Figure 1 (c), we can see that our generated camouflage
can fool object detectors and human eyes well.

Our contributions can be summarized as follows:

  * 1.

We propose differential evolution based dual adversarial camouflage to
generate the camouflage that obtains a good trade-off between fooling the
human eyes and object detectors, which can also be easily combined with
different scenes and objects.

  * 2.

We introduce the two-stage training strategy to obtain the camouflage. In the
first stage, we optimize the global texture to make the rendered object look
close to the specific scenes. In the second stage, we optimize the local
texture to fool the object detectors.

  * 3.

We introduce the DE algorithm to search for the near-optimal area to be
attacked, improving the attack success rate with the limitation of areas.

  * 4.

Consider improving the adaptability to the environment, we further study the
performance of adaptive DE$\\_$DAC. Under the same selected local texture,
compared with DE$\\_$DAC, adaptive DE$\\_$DAC can further improve the
camouflage performance.

The remainder of this paper is organized as follows. Section 2 introduces the
preliminary knowledge about adversarial attack, natural adversarial examples
and neural render. Section 3 introduces our proposed method. Section 4
elaborates the experimental settings and results. Section 5 makes the
conclusions.

##  2 Background

###  2.1 Adversarial attack

Recently, the majority of works have been devoted to adversarial attacks in
the digital and physical world. In the digital world, the classical $l_{p}$
norm bounded attacks are extensively exploited, including FGSM [7], PGD [8]
and CW [9]. By bounding the perturbation within the certain thresholds, the
generated AEs are imperceptible towards human eyes, which can also make the
classifier or detectors output wrong results. These AEs can be effective in
the digital world such as facial recognition systems. To ensure the certain
printness, in the physical world, the magnitude of the perturbation is usually
not bounded, which can be categorized into patch-based attack and camouflage-
based attack. In patch-based attack, Eykholt et al. [14] utilized the stickers
to attack the Faster RCNN detector, which can be pasted on the stop sign. Thys
et al. [15] optimized the patch towards person detection, which can be hung on
a person. Liu et al. [16] proposed Dpatch, which verifies that the optimized
patch on the background can also fool the detectors. In camouflage-based
attack, Hu et al. [17] proposed the adversarial camouflage to fool person
detectors. Wang et al. [19] proposed full camouflage attack (FCA), which
optimizes a full camouflage that can be pasted on the surface to fool
detectors.

###  2.2 Natural adversarial example

Due to the unlimitation of the magnitude in perturbation, how to realize the
naturalness of AEs towards human eyes remains a challenging problem. On the
one hand, many works focused on the goal that the optimized AEs possess their
own naturalness. Tan et al. [22] proposed the legitimate adversarial patches
(LAP) to fool object detectors, which constrains the edge and texture close to
the pre-selected natural image in the process of training the patch. Duan et
al. [13] proposed AdvCam, which combines the style loss, smooth loss and
adversarial loss together to train the adversarial example, which is verified
in the classification task. Huang et al. [23] propose universal physical
camouflage (UPC) towards object detectors. The final AEs look like the dogs.
Wang et al. [18] proposed a dual attention suppression attack, which optimizes
a smile texture pasted on the surface of the cars. Zhang et al. [24] proposed
Camou, which learns a 3D camouflage by repeating the images on the surface of
objects. On the other hand, Wang et al. [20] hoped to obtain the digital
camouflage that can fool detectors and human eyes simultaneously. They
proposed dual attribute adversarial camouflage (DAAC). Some of the works
mentioned above are visualized in Figure 2. In general, most of the existing
works focus on the methodology of constraining the optimized camouflage toward
the specific shape or texture. Though they can obtain the trade-off between
the adversarial performance and the naturalness of the optimized camouflage
itself, the final adversarial example can still be easily observed by human
eyes. DAAC can address the above challenge to some extent, while the
generation of the camouflage towards different scenes is complicated, and the
optimized patch limits the further application in the real world towards
multi-view cameras.

(a) LAP

(b) AdvCam

(c) UPC

(d) Camou

(e) DAAC

Figure 2: The visualization of the optimized camouflage using different
methods.

###  2.3 Neural render

The goal of the traditional renderer is to realize the 2D to 3D
transformation. One of the applications is to wrap the texture image to the 3D
model, which is then rendered to the 2D image. To make the process of
rendering differentiable, Kato et al. [25] proposed the neural render, which
utilized the approximate gradient for rasterization to integrate the rendering
into neural networks. By initializing the texture with different camera
parameters, such as the rotation and location, one could render the 3D object
model consisting of mesh and texture under different view angles. Due to the
ability to approach the physical conditions, neural render has been utilized
the CARLA [26] simulator to render the adversarial patch onto the 3D object.
Xiao et al. [27] also used the neural renderer to modify the shape and texture
of 3D objects. Following Wang et al. [18], we utilize the neural renderer to
paint our adversarial camouflage onto the surface of 3D object.

##  3 The proposed DE based Dual Adversarial Camouflage

In this section, the proposed DE based dual adversarial camouflage is
introduced in detail. In section 3.1, the overview of dual adversarial
camouflage generation framework is presented. In section 3.1.1 and section
3.1.2, the specific strategies of two-stage in DAC are elaborated
respectively. In section 3.2, the DE based search for near-optimal attack
areas is described. In section 3.3, depending on whether the camouflage can be
adapted to the scene, A-DE$\\_$DAC is further introduced.

###  3.1 Overview of DAC

Our goal is to generate the camouflage to fool the human eyes and object
detectors simultaneously. Our proposed DAC method includes two stages. In the
first stage, we minimize the mean square error (MSE) between the scene images
and the rendered images to obtain the texture, which could fool human eyes
easily. In the second stage, on the given training dataset, we optimize the
local texture to enhance the performance of adversarial attacks. The overview
of our proposed DAC method is presented in Figure 3.

Figure 3: The overview of the proposed DAC. The whole process can be divided
into two stages. In the fist stage, we optimize the global texture to obtain
the camouflage to fool human eyes. Under different camera parameters, the MSE
between the rendered image and the scene image is as the loss function. In the
second stage, we optimize the local texture to fool object detector.

####  3.1.1 First stage: generate the global texture to fool human eyes

In this first stage, the goal of the optimization process is to obtain the
camouflage that fools human eyes under the specific scene. Taking the person
model as an example, by controlling the camera parameters $\theta_{c}$, we can
obtain different rendered person images with multi-views. Each rendered person
image just includes the person and does not have the background. Given some
scene images selected from the specific scene, we aim to minimize the gap
between the rendered person images and the scene images. Thus the loss
function of optimizing the global texture is expressed as follows:

$\begin{array}[]{l}\mathcal{L}_{first}=\frac{\sum\limits_{i=1}\sum\limits_{j=1}{\text{MSE}(\mathbf{O}_{i},\textbf{I}_{j})}}{n}\end{array}$
(1)

where $\textbf{I}_{j}$ denotes the $j_{th}$ scene image selected from the
training dataset. $\mathbf{O}_{i}$ stands for the $i_{th}$ rendered image
using the texture $\textbf{T}_{g}$. $n$ denotes the total number of final
images that consist of rendered person images and scene images. We can obtain
the trained $\textbf{T}_{g}$ by minimizing the MSE loss between the rendered
image and scene image. In this stage, the camera parameters are manually-
controlled to ensure that the generated camouflage is natural enough. For
instance, near camera distance and 360-degree rotation perspective can be
better for training the final texture. Detailed operations can also be seen in
neural render introduced by Kennedym et al. [28].

####  3.1.2 Second stage:generate the local texture to fool object detectors

In the second stage, the goal of the optimization process is to obtain the
camouflage that fools the object detector with high naturalness. To realize
the purpose, we retrain the selected part in the global texture by the first
stage. The 3D model is rendered by multiple polyhedrons, which can be
described as faces. Assuming the number of faces of the model is $n_{m}$, the
continuous sequence of the face [1,2,…,$n_{m}$] can stand for the full
surface. If we need to select the part of the surface such as $n_{f}$ faces,
we can generate the sequence consisting of $n_{f}$ non-repeatable values from
[1,$n_{m}$]. The optimized local texture is denoted as $\textbf{T}_{l}$. By
introducing the mask $\textbf{M}_{l}$ consisting of zero and one determined by
the faces, we could merely optimize the wanted positions. The one in
$\textbf{M}_{l}$ stands for the positions of texture $\textbf{T}_{l}$, and
other parameters could be obtained from the $\textbf{T}_{g}$. Thus, the final
texture $\textbf{T}_{adv}$ could be obtained as follows.

$\begin{array}[]{l}\textbf{T}_{adv}=\textbf{T}_{g}\cdot(1-${M}$_{l})+${M}$_{l}\cdot(\textbf{T}_{l})\end{array}$
(2)

After obtaining $\textbf{T}_{l}$, we can obtain the final texture
$\textbf{T}_{adv}$ according to Eq. 2. Then we can obtain the rendered person
image $\mathbf{O}$.

$\mathbf{O}=\mathcal{R}((\textbf{M},\textbf{T}_{adv});\theta_{c})$ (3)

The segment network is utilized to obtain the mask $m$ of $\mathbf{O}$, where
one stands for the object while zeros represents the background. The final AEs
can be composed of $\mathbf{O}$ and $\mathbf{I}$ by $m$, as presented in Eq.
4.

$\mathbf{I}_{adv}=\Phi(\mathbf{O})=m\cdot\mathbf{O}+(1-m)\cdot\mathbf{I}$ (4)

To obtain the near-optimal $\textbf{T}_{l}$, we design the combination of
three loss functions, including the adversarial loss $\mathcal{L}_{adv}$,
color loss $\mathcal{L}_{color}$ and smooth loss $\mathcal{L}_{smooth}$. These
three loss functions are described as follows.

  * 1.

Adversarial Loss

In our work, the classical Yolo-V3 object detector is adopted as the white-box
target model. The adversarial loss as follow.

$\begin{array}[]{l}\mathcal{L}_{adv}=\mathcal{L}_{obj}(\mathbf{I}_{adv})\end{array}$
(5)

where $\mathcal{L}_{obj}$ represents the confidence score whether the
detection box contains an object. For other object detectors, the specific
$\mathcal{L}_{adv}$ could be designed adaptively.

  * 1.

Color Loss

To ensure the naturalness of the generated camouflage by local texture
$\textbf{T}_{l}$, in the second stage, the color loss is introduced as a
constraint. We aim to make the discrepancy between the local texture and the
obtained global texture $\textbf{T}_{g}$ close, named $\mathcal{L}_{color}$,
which is expressed as follows.

$\begin{array}[]{l}\mathcal{L}_{color}=\sum[(\textbf{T}_{g}-\textbf{T}_{l})\cdot${M}$_{l}]^{2}\end{array}$
(6)

  * 1.

Smooth Loss

To ensure the printness of the generated adversarial camouflage, we follow
Wang et al. [19] to utilize the smooth loss that introduced by Mahendran et
al. [29] to reduce the inconsistent among adjacent pixels. For a rendered
person image painted with adversarial camouflage, the calculation of smooth
loss can be written as

$\begin{array}[]{l}\mathcal{L}_{smooth}=\sum_{i,j}\left(x_{i,j}-x_{i+1,j}\right)^{2}+\left(x_{i,j}-x_{i,j+1}\right)^{2}\end{array}$
(7)

where $x_{i,j}$ is the pixel value of $\textbf{I}_{adv}$ at coordinate
$(i,j)$.

Combining three above loss functions, we could obtain the local texture:

$\begin{array}[]{l}\mathcal{L}_{second}=\mathcal{L}_{adv}+\lambda_{1}\mathcal{L}_{color}+\lambda_{2}\mathcal{L}_{smooth}\end{array}$
(8)

where $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ are the coefficients to
control the weight of loss values. Larger $\lambda_{2}$ means higher attack
success rate, while smaller one would make the optimization process focus on
the naturalness.

To sum up, the whole process of generating the camouflage is presented in the
following DAC algorithm.

Input: Training set ($\textbf{X},\theta_{c}$), 3D model (M,T),neural render
$\mathcal{R}$, object detector $\mathcal{F}$, segmentation network
$\mathcal{U}$, the mask $\textbf{M}_{l}$ determined by the selected faces of
model.

Output: Adversarial texture $\textbf{T}_{adv}$.

1

2Initialize the $\textbf{T}_{g}$ with the random noise;

3 for  _the maximum epoch of the first stage_ do

4 Select the mini-batch sample from the training set
($\textbf{X},\theta_{c}$);

5 Extract the background of the selected sample using $\mathcal{U}$;

6 Generate the rendered image $O$ $\leftarrow$
$\mathcal{R}((\textbf{M},\textbf{T}_{g});\theta_{c})$;

7 Calculate the loss function of the first stage $\mathcal{L}_{first}$ by Eq.
1;

8 Update the $\textbf{T}_{g}$ by gradient back-propagation;

9

10 end for

11Initialize the $\textbf{T}_{l}$ with the random noise;

12 for  _the maximum epoch of the second stage_ do

13 Select the mini-batch sample from the training set
($\textbf{X},\theta_{c}$);

14 Extract the mask of the render image $m$ $\leftarrow$
$\mathcal{U}(\textbf{X})$;

15 Obtain the final texture $\textbf{T}_{adv}$
$\leftarrow$$\textbf{T}_{g}\cdot(1-$M${}_{l})+$M${}_{l}\cdot(\textbf{T}_{l})$;

16 Generate the rendered image $\mathbf{O}$ $\leftarrow$
$\mathcal{R}((\textbf{M},\textbf{T}_{adv});\theta_{c})$;

17 Extract the background $\mathbf{I}$ of the selected sample using
$\mathcal{U}$;

18 Obtain the image composed of the reder image and background
$\mathbf{I}_{adv}$ $\leftarrow$ $m\cdot\mathbf{O}+(1-m)\cdot\mathbf{I}$;

19 Calculate the $\mathcal{L}_{adv}$ using $\mathbf{I}_{adv}$ by Eq. 5 for the
object detector $\mathcal{F}$;

20 Calculate the $\mathcal{L}_{color}$ using $\textbf{T}_{g}$ and
$\textbf{T}_{adv}$ by Eq. 6;

21 Calculate the $\mathcal{L}_{smooth}$ using $\mathbf{O}$ by Eq. 7;

22 Obtain the total loss $\mathcal{L}_{second}$ by Eq. 8;

23 Update the $\textbf{T}_{l}$ by gradient backpropagation;

24

25 end for

26

Algorithm 1 Dual Adversarial Camouflage (DAC)

###  3.2 DE based search for near-optimal attack area

In above-mentioned DAC method, the optimized local texture is randomly
selected from the global texture. In this section, we focus on improving the
DAC method by searching for the near-optimal attack area when the number of
faces of the models is settled. The mathematical model of searching for the
near optimal attack area can be described as follows.

$\begin{array}[]{ll}\min\limits_{x}&f(\textbf{x})\\\ \text{ s.t.
}&1\leq{\textbf{x}_{i}}\leq{{n}_{m}},\forall i\in\\{1,2,\cdots,{{n}_{f}}\\}\\\
&{\textbf{x}_{i}}\neq{{x}_{j}},\forall i,j\in\\{1,2,\cdots,{{n}_{f}}\\}\\\
\end{array}$ (9)

where x denotes the selected face of the model, ${n}_{f}$ is the preset
maximum number of selected faces, ${n}_{m}$ stands for the maximum number of
faces the model with full surface, $f(\textbf{x})$ is the<EMAIL_ADDRESS>value of x
solved by the proposed DAC method in the dataset.

Input: The population size $n$, the number of faces $n_{f}$, the maximum
number of faces of the model $n_{m}$, the maximum iteration number $T$,
crossover rate $r_{c}$, mutation rate $r_{m}$.

Output: The optimal face $\textbf{x}_{best}$.

1 Iteration counter number $i=1$;

2 Population Initialization P $\leftarrow$ generate $n$ individuals randomly,
each of which is one sequence consisting of $n_{f}$ non-repeatable values from
[1,$n_{m}$];

3 $fitness$(P) $\leftarrow$ Evaluate the<EMAIL_ADDRESS>value of each individual in
population P using DAC algorithm;

4 for  _$i <=T$_ do

5 $\textbf{P}_{m}$$\leftarrow$ Population mutation based on P;

6 $\textbf{P}_{c}$ $\leftarrow$ Population crossover based on
$\textbf{P}_{m}$;

7 // Population selection

8 Individual counter number $j=1$;

9 for  _$j <=n$_ do

10 if  _fitness $(\textbf{P}_{c})[j]$ $<$ fitness$(\textbf{P})[j]$_ then

11 $\textbf{P}[j]$ = $\textbf{P}_{c}[j]$;

12 end if

13 $j=j+1$;

14 end for

15

16 Obtain the best individual $\textbf{x}_{best}$ $\leftarrow$
$argmax$($fitness$(P));

17 Update P by removing the duplicate numbers;

18 $i=i+1$;

19

20 end for

Algorithm 2 DE based Dual Adversarial Camouflage (DE$\\_$DAC)

Then we introduce the DE algorithm to search for the near-optimal attack area
in the second stage of DAC. The whole framework is denoted as DE$\\_$DAC. DE
is one classical gradient-free evolutionary algorithm to solve the black-box
optimization problem, which is composed of population initialization,
mutation, crossover, and selection operations. To realize the purpose of
searching for the near-optimal attack area, as the description in section 3.3,
we generate the population consisting of different individuals. Each
individual is a sequence, consisting of $n_{f}$ non-repeatable values from
[1,$n_{m}$], where $n_{f}$ is the pre-settled number of faces. During the
optimization process, the<EMAIL_ADDRESS>value by DAC is adopted as the evaluation
metric when the individual is determined. By the subsequent operations,
including population crossover, mutation, and selection, the near-optimal
sequence can be obtained, which corresponds to the near-optimal attack area in
DE$\\_$DAC. In this process, we need to remove the duplicate numbers in the
searched sequence and randomly generate that again at each iteration. The
pseudo-code of DE$\\_$DAC is shown in the DE$\\_$DAC algorithm.

###  3.3 Adaptive DE$\\_$DAC

Based on the formulations of above mentioned DE$\\_$DAC, depending on whether
the camouflage can be adapted to the scene, we further propose A-DE$\\_$DAC.
As described in DE$\\_$DAC above mentioned, both the textures optimized in the
first and second stages are universal. In A-DE$\\_$DAC, we try to optimize the
global texture on each scene image in the first stage. That is, each scene
image would generate one camouflage. After that, we try to optimize the local
texture on all images. That is to say, the global texture optimized by
A-DE$\\_$DAC in the first stage is sample-independent, while the local texture
in the second stage is universal. In this way, the trained local texture is
universal to fool detectors, while other areas of objects adopt the global
texture that is adaptive to the background environment to improve the
camouflage performance of fooling human eyes.

##  4 Experiments

In this section, extensive experiments are performed to illustrate the
effectiveness of our proposed DE$\\_$DAC method. In section 4.1, we describe
our experimental setting and details. Section 4.2 shows the experiment
results.

###  4.1 Experiment Protocol

####  4.1.1 Dataset

To approach the physical adversarial camouflage in the real world, the photo-
realistic datasets are performed in our experiments. The data preparation is
similar to Wang et al. [19]. In our experiments, we select the person as the
attack object, which is provided in UPC [23]. To this end, we select Unreal
Engine 4 (UE 4) [30], a prevalent open-source simulator for game development
research, as the tools to provide various scenes. The UE4 provides a variety
of high-fidelity digital scenarios (e.g., modern urban). The rendered object
can be easily combined with the sampled scenes from UE4 by the masks,
generating images consisting of objects and background scenes. To compare with
previous works, we use UE4 to construct our training and test dataset
directly. The training dataset includes 5000 high-resolution images consisting
of the rendered person model and specific scenes, while the testing set has
500 high-resolution images. The datasets contain the rendered images that are
sampled from different views angles and distances. In the process of preparing
the training dataset, the camera distance of rendering the objects is randomly
sampled from [2, 7]. The elevation is randomly sampled from [0, 45], and the
azimuth is sampled from [0, 360]. The number of render images is 500, and the
number of scene images is set to 10, which is randomly selected from UE 4.
Thus we can obtain 5000 images consisting of different render images and scene
images. In the test dataset, the camera distance, elevation, and azimuth are
randomly selected. The same operation above-mentioned is performed on each
specific type of scene.

####  4.1.2 Evaluation Metrics

In this work, we adopt three metrics to evaluate the performance of our
proposed camouflage generation method. Two metrics, including attack success
rate (ASR) and<EMAIL_ADDRESS>accuracy, are utilized to evaluate the adversarial attack
performance. ASR is defined as the percentage of the incorrectly detected
images on all the correctly detected images<EMAIL_ADDRESS>denotes the percentage of
the correct detected images on all the images when the detection IoU threshold
is set to 0.5. Another metric, MSE, is utilized to evaluate the naturalness of
the generated camouflage, which is defined as the MSE between the rendered
object and the scene images. The lower MSE is, the better the generated
camouflage towards fooling human eyes is.

####  4.1.3 Implementation details

We chose the widely used detector, Yolo-V3 [31], as our white-box model to
train the adversarial camouflage texture. Then we evaluate the transferring
attack performances on the following prevalence object detection models:
Yolo-V2 [32], Yolo-V5 [33], Faster R-CNN [34], and Mask R-CNN [35]. These
models are all pretrained on the COCO dataset. In our experiments, these
models are the official implementation version provided by PyTorch. The
detection minimum threshold is set as 0.5 for all detectors by default. The
adversarial camouflage texture is initialized as the random noise, and the
Adam with default parameter is adopted as the optimizer. The learning rate is
0.01. The segmentation network used to extract the background from the photo-
realistic image is U2-Net [36]. In our proposed DE$\\_$DAC method, the maximum
epochs of the first and second stages are set to 1 and 10, respectively. The
number of images in the training datasets in the first and second stages is
set to 300 and 5000, respectively. Both batch sizes are set to 1. The
$\lambda_{1}$ and $\lambda_{2}$ in $\mathcal{L}_{second}$ loss of the second
stage are set to 5$\times e^{-4}$ and 1$\times e^{-7}$ for all the scenes
respectively. In the DE algorithm, the parameters setting is as follows. The
population size is set to 20. The maximum iteration number $T$ is set to 10.
The rate of crossover and mutation is set to 0.6. We conduct all the
experiments on a NVIDIA RTX 3090 24GB GPU cluster.

###  4.2 Experimental results

In this section, we conduct a series of experiments to illustrate the
effectiveness of our proposed method on the object detection task. The person
is selected as our attack object. We conduct the experiments on three types of
scenes provided by UE 4, including WinterValley, Forest, and Desert, which are
also frequently encountered scenes in real military applications. We study the
performance of the white box attack on the Yolo-V3 detector, the black box
attack on other detectors, adaptive DE$\\_$DAC, and different coefficients of
DE$\\_$DAC. In addition, we also study the performance of attacking other
objects, such as cars and trucks, to further verify the generality of our
proposed method.

####  4.2.1 White-box attack on object detection

Raw stands for the original images ,also called clean images. Camou is the
camouflage using the method provided by Zhang et al. [24]. AdvCam was first
proposed by Dual et al. [13] to ensure the naturalness and adversarial
performance of the generated camouflage using the style transfer technique in
the classification task. We transfer the method to the detection task. In
addition, to further illustrate the effectiveness of our proposed two-stage
method, we also make a comparison with the one-stage method, denoted as one-
stage DAC. One stage means that the camouflage is trained jointly by the
adversarial loss, smooth loss, and the MSE between the rendered person images
with scene images. The typical one-stage method has been adopted in the
classification task, such as Dual et al. [13]. Pure camouflage is the
optimized texture after the first stage in our proposed method, denoted as
camouflage. DE$\\_$DACfull denotes the optimized texture by our proposed whole
method when the number of the local texture is the same as the global texture.
The statistical results of<EMAIL_ADDRESS>ASR, and MSE under different scenes using
the Yolo-V3 detection model are listed in Table 1.

Table 1: Comparison of the<EMAIL_ADDRESS>ASR and MSE of different attack methods on three person datasets with specific scenes. Methods | WinterValley | Forest | Desert
---|---|---|---
|  P@0.5($\downarrow$) |  ASR($\uparrow$) |  MSE($\downarrow$) |  P@0.5($\downarrow$) |  ASR($\uparrow$) |  MSE($\downarrow$) |<EMAIL_ADDRESS>($\downarrow$) |  ASR($\uparrow$) |  MSE($\downarrow$)
Raw |  100.0$\%$ |  0.00$\%$ | 857.62 |  100.0$\%$ |  0.00$\%$ | 729.41 |  100.0$\%$ |  0.00$\%$ | 1377.10
Camou |  64.8$\%$ |  36.1$\%$ | 1527.58 |  78.4$\%$ |  21.6$\%$ | 1393.22 |  57.0$\%$ |  43.0$\%$ | 2019.04
AdvCam |  53.2$\%$ |  46.8$\%$ | 735.98 |  24.8$\%$* |  75.2$\%$* | 881.13 |  53.2$\%$ |  46.8$\%$ | 1225.15
One-stage DAC |  30.6$\%$ |  69.4$\%$ | 407.85 |  30.6$\%$ |  69.4$\%$ | 388.74 |  15.6$\%$* |  84.4$\%$* | 832.86
Camouflage(Ours) |  72.6$\%$ |  27.4$\%$ | 252.22 |  68.8$\%$ |  31.2$\%$ | 168.98 |  29.8$\%$ |  70.2$\%$ | 118.23
DE$\\_$DACfull(Ours) | 25.4$\%$ | 74.6$\%$ | 327.00* | 23.8$\%$ | 76.2$\%$ | 298.61* | 10.0$\%$ | 90.0$\%$ | 524.63*

(a) Clean

(b) Camou

(c) AdvCam

(d) One-stage DAC

(e) Camouflage

(f) DE$\\_$DAC

(g) Clean

(h) Camou

(i) AdvCam

(j) One-stage DAC

(k) Camouflage

(l) DE$\\_$DAC

(m) Clean

(n) Camou

(o) AdvCam

(p) One-stage DAC

(q) Camouflage

(r) DE$\\_$DAC

Figure 4: The visualization of different adversarial attack methods on
WinterValley, Forest, and Desert scenes.

From Table 1, we can see that camouflage optimized by DE$\\_$DACfull can
obtain almost the lowest MSE among all methods on three datasets and possess
certain adversarial performance simultaneously, causing the significant drop
in<EMAIL_ADDRESS>and increase in ASR than clean images. Camouflage directly optimizes
the global texture to fool human eyes, achieving a certain adversarial
performance and making the accuracy of detectors decrease from 100$\%$ to
about 70$\%$. In addition, on three datasets, all the<EMAIL_ADDRESS>and ASR values of
DE$\\_$DACfull are the best among the comparison methods. Meanwhile, the MSE
of DE$\\_$DACfull is close to the pure camouflage optimized by the first
stage. These experimental results show that our proposed method strikes a
better balance between fooling the detector and human eyes than other methods.
It indicates that our proposed method can provide a better initialization
after the first stage, further improving the final adversarial performance of
DE$\\_$DACfull. That is, our proposed two stage dual adversarial camouflage
can collaboratively boost performances, where the first stage makes up for the
disadvantage of not being natural in the second stage, and the second stage
bridges the gap of fooling object detectors in the first stage. In contrast,
the MSE value of generated camouflage by Camou is relatively too large,
causing the conspicuous appearance towards human eyes, which also can be seen
in other methods. From the results of AdvCam and one-stage DAC, we can see
that compared with the classification task, it is more difficult to directly
train the combination of three loss functions using the universal dataset for
the detection task. The trained camouflage texture can not obtain a good
trade-off between fooling human eyes and object detectors. The camouflages
generated by us and existing methods towards different scenes are presented in
Figure 4. From the aspect of fooling human eyes, pure camouflage and
DE$\\_$DAC can obtain better performance than other methods.

####  4.2.2 Black-box attack on object detection

To further illustrate the effectiveness of our proposed method, we conduct the
transfer experiment on other object detectors, including Yolo-V2, Yolo-V5,
FasterRCNN, and MaskRCNN. The performance of the generated camouflage by
Yolo-V3 is evaluated on the above-mentioned models. Table 2 shows the ASR and
<EMAIL_ADDRESS>values of detection results by different detectors on three datasets,
respectively. From Table 2, we can see that among four detectors, the<EMAIL_ADDRESS>value of Yolo-V2 is about 85$\%$, and that of other detectors is 100$\%$,
which shows superior detection performance. However, these superior detectors
can still be fooled by our proposed method. On all object detectors, compared
with existing methods, DE$\\_$DAC obtains the highest ASR and lowest<EMAIL_ADDRESS>values. For example, the ASR of attacking Yolo-V5 on the WinterValley scene
using DE$\\_$DAC can reach 93.2$\%$, while that of Camou, AdvCam, One-stage
DAC, and Camouflage are just 71.6$\%$, 32.2$\%$, 46.2$\%$, 47.0$\%$,
respectively. In most detection cases, the ASR of DE$\\_$DAC can also be over
50.0$\%$. It is worth noting that the pure camouflage generated by us can also
possess a certain transfer adversarial performance, which verifies that dual
adversarial camouflage also does a good effect on the improvement of
adversarial performance collaboratively. In the two-stage detectors such as
FasterRCNN and MaskRCNN, DE$\\_$DAC has a slight drop in the performance of
ASR than pure camouflage. It may be because these detectors can not recognize
the camouflaged object well. Some examples generated by different methods are
visualized in Figure 5, which are detected by the Yolo-V5 model. Under the
same training and test settings, we can see that our proposed method can
obtain the lowest detection performance from different views. Thus our
proposed DE$\\_$DAC method can also possess competitive transfer adversarial
performance.

(a) Clean

(b) Clean

(c) Clean

(d) Clean

(e) Clean

(f) Clean

(g) Camou

(h) Camou

(i) Camou

(j) Camou

(k) Camou

(l) Camou

(m) AdvCam

(n) AdvCam

(o) AdvCam

(p) AdvCam

(q) AdvCam

(r) AdvCam

(s) Camouflage

(t) Camouflage

(u) Camouflage

(v) Camouflage

(w) Camouflage

(x) Camouflage

(y) DE$\\_$DAC

(z) DE$\\_$DAC

(aa) DE$\\_$DAC

(ab) DE$\\_$DAC

(ac) DE$\\_$DAC

(ad) DE$\\_$DAC

Figure 5: The visualization of different adversarial attack methods on Forest scene. Table 2: Comparison of the<EMAIL_ADDRESS>ASR and MSE of different attack methods on other object detectors in three specific scenes. Scene | Methods | Yolo-V2 | Yolo-V5 | FasterRCNN | MaskRCNN
---|---|---|---|---|---
|  |  Accuracy($\downarrow$) |  ASR ($\uparrow$) |  Accuracy($\downarrow$) |  ASR ($\uparrow$) |  Accuracy($\downarrow$) |  ASR($\uparrow$) |  Accuracy($\downarrow$) |  ASR($\uparrow$)
WinterValley | Raw |  84.4$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$
| Camou |  44.2$\%$ |  47.6$\%$ |  28.4$\%$ |  71.6$\%$ |  84.8$\%$ |  15.2$\%$ |  94.8$\%$ |  5.2$\%$
| AdvCam |  19.4$\%$ |  77.0$\%$ |  67.8$\%$ |  32.2$\%$ |  69.0$\%$ |  31.0$\%$ |  77.2$\%$ |  22.8$\%$
| One-stage DAC |  18.4$\%$ |  78.2$\%$ |  53.8$\%$ |  46.2$\%$ |  72.8$\%$ |  27.2$\%$ |  76.8$\%$ |  23.2$\%$
| Camouflage(Ours) |  46.0$\%$ |  45.5$\%$ |  53.0$\%$ |  47.0$\%$ | 44.8$\%$ | 55.2$\%$ | 47.4$\%$ | 52.6$\%$
|  DE$\\_$DACfull(Ours) | 17.8$\%$ | 78.9$\%$ | 6.8$\%$ | 93.2$\%$ |  58.2$\%$ |  41.8$\%$ |  62.6$\%$ |  37.4$\%$
Forest | Raw |  86.6$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$
| Camou |  53.8$\%$ |  37.9$\%$ |  16.8$\%$ |  83.2$\%$ |  71.4$\%$ |  28.6$\%$ |  80.6$\%$ |  19.4$\%$
| AdvCam |  36.6$\%$ |  57.7$\%$ |  77.0$\%$ |  23.0$\%$ |  57.4$\%$ |  42.6$\%$ |  64.4$\%$ |  35.6$\%$
| One-stage DAC |  33.0$\%$ |  61.9$\%$ |  63.2$\%$ |  36.8$\%$ |  64.6$\%$ |  35.4$\%$ |  69.0$\%$ |  31.0$\%$
| Camouflage(Ours) |  55.8$\%$ |  35.6$\%$ |  46.2$\%$ |  53.8$\%$ | 42.2$\%$ | 57.8$\%$ |  47.8$\%$ |  52.2$\%$
|  DE$\\_$DACfull(Ours) | 30.2$\%$ | 65.1$\%$ | 11.4$\%$ | 88.6$\%$ |  51.4$\%$ |  48.6$\%$ | 46.0$\%$ | 54.0$\%$
Desert | Raw |  83.8$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$ |  100$\%$ |  0.00$\%$
| Camou |  62.6$\%$ |  33.3$\%$ |  17.6$\%$ |  30.6$\%$ |  90.6$\%$ |  9.40$\%$ |  89.4$\%$ |  10.6$\%$
| AdvCam |  29.6$\%$ |  23.1$\%$ |  44.8$\%$ |  55.2$\%$ |  67.0$\%$ |  33.0$\%$ |  64.6$\%$ |  35.4$\%$
| One-stage DAC |  45.2$\%$ |  69.9$\%$ |  32.2$\%$ |  67.8$\%$ |  72.8$\%$ |  27.2$\%$ |  76.8$\%$ |  23.2$\%$
| Camouflage(Ours) |  43.8$\%$ |  47.7$\%$ |  44.4$\%$ |  55.6$\%$ | 39.4$\%$ | 60.6$\%$ | 42.0$\%$ | 58.0$\%$
|  DE$\\_$DACfull(Ours) | 36.0$\%$ | 57.0$\%$ | 12.8$\%$ | 87.2$\%$ |  77.4$\%$ |  22.6$\%$ |  52.2$\%$ |  47.8$\%$

(a) WinterValley (b) Forest (c) Desert

Figure 6: The performance of ASR and MSE of random search and DE with
different number of face in different scenes.

####  4.2.3 The performance of DE$\\_$DAC

In our experiment, the number of faces of the person model is 1840. Thus the
full surface can be represented by 1840 inter values from 1 to 1840. We can
select unduplicated $n_{f}$ values from 1 to 1840 to obtain the corresponding
area in the person model. If $n_{f}$ does not reach 1840, our proposed
DE$\\_$DAC can search for the near-optimal attack area under the settled
number of faces. To illustrate the effectiveness of our proposed method, we
set the number of faces as 400, 800, 1200, 1600, and 1840, respectively, and
utilize the random selection strategy and DE algorithm to select the faces of
the optimized local texture. The comparisons of the<EMAIL_ADDRESS>and ASR of the
generated camouflage by random selection and DE are presented in Figure 6,
respectively.

From Figure 6, we can see that with the increasing number of faces, the attack
performance, including<EMAIL_ADDRESS>and ASR, also increases. The smaller number of
faces means less perturbation and a more natural appearance. Given the settled
number of faces, DE can find a better attack area compared with random
selection, which has a higher ASR and a lower<EMAIL_ADDRESS>From Figure 6 (b), we can
see that the ASR of camouflage by random selection is only 59.4$\%$ when the
settled number of faces is 800, while the ASR of camouflage by DE can reach
67.2$\%$. Particularly, the ASR of camouflage by random search with the number
of faces 1600 is just 65.1$\%$. It means that the adversarial performance of
camouflages using 800 faces searched by DE can outperform that using 1600
faces by random search. On the one hand, it shows that our proposed DE can
effectively find the important area to improve adversarial performance. On the
other hand, it shows the effectiveness of our proposed DE$\\_$DAC method,
possessing competitive adversarial performance even though a small number of
faces are given.

(a) 400

(b) 800

(c) 1200

(d) 1600

(e) 1840

Figure 7: The visualization of the searched attack area with different number
of faces in the second stage.

(a) WinterValley (b) Forest (c) Desert

Figure 8: The performance of ASR and MSE of DE$\\_$DAC and A-DE$\\_$DAC with
different number of faces in different scenes.

####  4.2.4 The performance of Adaptive DE$\\_$DAC

In this section, we study the performance of the proposed adaptive DE$\\_$DAC
method. Specifically, taking the forest scene as an example, we first train
the texture according to each scene image belonging to the forest scene in the
first stage of DE$\\_$DAC, obtaining ten trained global textures. In the
second stage, we select the local texture to optimize. As for each input
image, the global texture is selected from the corresponding one. Thus in the
whole training process, the optimized local texture is universal while the
global texture is adaptive. The procedure is the same in the evaluation of the
final textures. We also set the different numbers of faces in the selection of
local textures, including 400, 800, 1200, 1600, and 1840. The performance of
A-DE$\\_$DAC and DE$\\_$DAC under three datasets is presented in Figure 8.

From Figure 8, we can see that compared with DE$\\_$DAC, A-DE$\\_$DAC can
obtain better naturalness with the scenes. On three datasets, all of the MSE
of optimized textures are lower than DE$\\_$DAC when the number of faces is
the same, dropping about 10. It can be predictable that A-DE$\\_$DAC can
fastly generate the global texture adaptively according to the background of
each image. In general, the MSE value with the increasing number of faces
becomes larger in A-DE$\\_$DAC, which also corresponds to the previous
conclusions. In contrast, in most cases, the performance of ASR of
A-DE$\\_$DAC is slightly worse than DE$\\_$DAC. It may be because the
optimization problem of A-DE$\\_$DAC is more difficult than DE$\\_$DAC under
our settings. On the whole, A-DE$\\_$DAC can obtain better camouflage towards
human eyes with a slight sacrifice of ASR. Figure 9 shows the visualization of
generated AEs by A-DE$\\_$DAC. The generated global camouflage is adaptive to
the background, while the trained local camouflage is universal.

Figure 9: The visualization of adaptive DE$\\_$DAC.

####  4.2.5 The performance of the coefficient $\lambda_{1}$

The coefficient $\lambda_{1}$ in the total loss plays a key role in the final
performance of our optimized camouflage. To study the effect of $\lambda_{1}$
on the performance of ASR and MSE of our proposed method, we conduct the
experiment on three datasets under different $\lambda_{1}$ settings. In our
experiments, $\lambda_{1}$ is set to 0.0001, 0.0005, 0.01, 0.02 and 0.03
respectively, while $\lambda_{2}$ remains unchanged. The selected number of
faces in the second stage is set to 1840. We can train five different
camouflages under five parameter settings by attacking the Yolo-V3 detector.
All the generated test AEs are also detected by Yolo-V3. The statistical
result of ASR and MSE of five camouflages is presented in Figure 10. From
Figure 10, we can see that the selection of $\lambda_{1}$ has a great
influence on the performance of ASR and MSE. With the increase of
$\lambda_{1}$, both the ASR and MSE decrease, which means the adversarial
performance and the naturalness of the generated camouflage are contradictory.
In addition, as the $\lambda_{1}$ increases, the change in ASR becomes
progressively greater, while the change in MSE tends to flatten out. Thus, we
can select a relatively better value like 0.0005 to keep a good balance
between the adversarial performance and naturalness towards human eyes. In
general, the ASR using different $\lambda_{1}$ can be maintained at a high
level. For instance, even though the $\lambda_{1}$ is set to 0.003, the ASR on
Forest and Desert datasets can still be over 65$\%$ and 82$\%$, which verifies
the effectiveness of our proposed two-stage DAC method.

(a) ASR (b) MSE

Figure 10: The effect of $\lambda_{1}$ on ASR and MSE on three datasets.

####  4.2.6 Other objects and scenes

To illustrate the generality of our proposed method, we also conduct
experiments on attacking other objects, including the car, trucks and planes.
The selected scenes include the city streets, sky, residential buildings and
wild road. The visualization of AEs by attacking these objects is presented in
Figure 11. Figure 11 (a) to Figure 11 (f) are the camouflage optimized by the
first stage, while Figure 11 (g) and Figure 11 (l) are the final camouflage
optimized by DE$\\_$DAC. All of them can cause the dropping of the accuracy of
the classifier or detector models. From Figure 11, we can see that the
camouflage by the first stage in our proposed method can possess a superior
ability to adapt well to the surrounding environment. Our proposed method can
learn well camouflage to fool human eyes and object detectors not only in
relatively monochrome scenes like Figure 11 (b) and Figure 11 (c) but also in
complicated scenes such as Figure 11 (a). It means that by the neural render,
our method can learn the camouflage with rich background environmental
features. Besides, our proposed method can possess better flexibility in the
real world. For instance, Figure 11 (a) and Figure 11 (e) present the
visualization of practical camouflage in the person, which is more practical
in application. In addition, our method can also be easily transferred to
other objects.

(a) Camouflage

(b) Camouflage

(c) Camouflage

(d) Camouflage

(e) Camouflage

(f) Camouflage

(g) DE$\\_$DAC

(h) DE$\\_$DAC

(i) DE$\\_$DAC

(j) DE$\\_$DAC

(k) DE$\\_$DAC

(l) DE$\\_$DAC

Figure 11: The visualization of attacking other objects by our method .

##  5 Conclusion

In this paper, we propose a method called differential evolution based dual
adversarial camouflage to fool human eyes and object detectors simultaneously.
In this method, we first propose a two-stage training strategy to train the
texture that can be pasted on the surface of objects. In the first stage, we
train the global texture to minimize the discrepancy between the objects and
the background environments. In the second stage, we train the partial
texture, denoted as the local texture, to fool the object detectors. In
addition, given a 3D object and the maximum number of faces area to optimize,
we introduce the DE algorithm to search for the near-optimal area. In digital-
world experiments including three datasets, our proposed method can obtain the
best performance among the comparison methods. Compared with pure camouflage,
with a slight drop in MSE between the render object and background, our method
can improve the adversarial performance, such as ASR largely, from 30$\%$ to
about 80$\%$. Under the same number of faces to be optimized in the second
stage, our proposed method can also improve the ASR by about 5$\%$. In
addition, experiments show that our proposed end-to-end method can also be
easily combined to other scenes and objects.

## Acknowledgments

This work was supported by the National Natural Science Foundation of China
(Nos.11725211 and 52005505).

## References

  * Chen et al. [2022] Chen, S., Wei, Q., Wang, W., Tang, J., Luo, B., Wang, Z..  Remote sensing scene classification via multi-branch local attention network.  IEEE Trans Image Process 2022;31:99–109.  URL: https://doi.org/10.1109/TIP.2021.3127851. doi:10.1109/TIP.2021.3127851.
  * Wang et al. [2017] Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., et al.  Residual attention network for image classification.  In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, p. 3156–3164.
  * Lu and Weng [2007] Lu, D., Weng, Q..  A survey of image classification methods and techniques for improving classification performance.  International journal of Remote sensing 2007;28(5):823–870.
  * Zhou et al. [2021] Zhou, Q., Yu, C., Wang, Z., Qian, Q., Li, H..  Instant-teaching: An end-to-end semi-supervised object detection framework.  In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021. Computer Vision Foundation / IEEE; 2021, p. 4081–4090.
  * Zhao et al. [2019] Zhao, Z.Q., Zheng, P., Xu, S.t., Wu, X..  Object detection with deep learning: A review.  IEEE transactions on neural networks and learning systems 2019;30(11):3212–3232.
  * Szegedy et al. [2013] Szegedy, C., Toshev, A., Erhan, D..  Deep neural networks for object detection.  Advances in neural information processing systems 2013;26.
  * Goodfellow et al. [2014] Goodfellow, I.J., Shlens, J., Szegedy, C..  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:14126572 2014;.
  * Madry et al. [2018] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A..  Towards deep learning models resistant to adversarial attacks.  In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net; 2018,.
  * Carlini and Wagner [2017] Carlini, N., Wagner, D.A..  Towards evaluating the robustness of neural networks.  In: 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017. IEEE Computer Society; 2017, p. 39–57.  doi:10.1109/SP.2017.49.
  * Hosseini and Poovendran [2018] Hosseini, H., Poovendran, R..  Semantic adversarial examples.  In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018, p. 1614–1619.
  * Laidlaw and Feizi [2019] Laidlaw, C., Feizi, S..  Functional adversarial attacks.  Advances in neural information processing systems 2019;32.
  * Hendrycks and Dietterich [2019] Hendrycks, D., Dietterich, T..  Benchmarking neural network robustness to common corruptions and perturbations.  arXiv preprint arXiv:190312261 2019;.
  * Duan et al. [2020] Duan, R., Ma, X., Wang, Y., Bailey, J., Qin, A.K., Yang, Y..  Adversarial camouflage: Hiding physical-world attacks with natural styles.  In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020, p. 1000–1008.
  * Eykholt et al. [2018] Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., et al.  Robust physical-world attacks on deep learning visual classification.  In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, p. 1625–1634.
  * Thys et al. [2019] Thys, S., Van Ranst, W., Goedemé, T..  Fooling automated surveillance cameras: adversarial patches to attack person detection.  In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2019, p. 0–0.
  * Liu et al. [2018] Liu, X., Yang, H., Liu, Z., Song, L., Li, H., Chen, Y..  Dpatch: An adversarial patch attack on object detectors.  arXiv preprint arXiv:180602299 2018;.
  * Hu et al. [2022] Hu, Z., Huang, S., Zhu, X., Sun, F., Zhang, B., Hu, X..  Adversarial texture for fooling person detectors in the physical world.  In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, p. 13307–13316.
  * Wang et al. [2021a] Wang, J., Liu, A., Yin, Z., Liu, S., Tang, S., Liu, X..  Dual attention suppression attack: Generate adversarial camouflage in physical world.  In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021a, p. 8565–8574.
  * Wang et al. [2022] Wang, D., Jiang, T., Sun, J., Zhou, W., Gong, Z., Zhang, X., et al.  Fca: Learning a 3d full-coverage vehicle camouflage for multi-view physical adversarial attack.  In: Proceedings of the AAAI Conference on Artificial Intelligence; vol. 36. 2022, p. 2414–2422.
  * Wang et al. [2021b] Wang, Y., Fang, Z., Zheng, Y.f., Yang, Z., Tong, W., Cao, T.y..  Dual attribute adversarial camouflage toward camouflaged object detection.  Defence Technology 2021b;.
  * Kim et al. [2021] Kim, H., Choi, J., Kim, K.K., Won, P., Hong, S., Ko, S.H..  Biomimetic chameleon soft robot with artificial crypsis and disruptive coloration skin.  Nature communications 2021;12(1):1–11.
  * Tan et al. [2021] Tan, J., Ji, N., Xie, H., Xiang, X..  Legitimate adversarial patches: Evading human eyes and detection models in the physical world.  In: Proceedings of the 29th ACM International Conference on Multimedia. 2021, p. 5307–5315.
  * Huang et al. [2020] Huang, L., Gao, C., Zhou, Y., Xie, C., Yuille, A.L., Zou, C., et al.  Universal physical camouflage attacks on object detectors.  In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, p. 720–729.
  * Zhang et al. [2018] Zhang, Y., Foroosh, H., David, P., Gong, B..  Camou: Learning physical vehicle camouflages to adversarially attack detectors in the wild.  In: International Conference on Learning Representations. 2018,.
  * Kato et al. [2018] Kato, H., Ushiku, Y., Harada, T..  Neural 3d mesh renderer.  In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, p. 3907–3916.
  * Dosovitskiy et al. [2017] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V..  Carla: An open urban driving simulator.  In: Conference on robot learning. PMLR; 2017, p. 1–16.
  * Xiao et al. [2019] Xiao, C., Yang, D., Li, B., Deng, J., Liu, M..  Meshadv: Adversarial meshes for visual recognition.  In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, p. 6898–6907.
  * Kennedym and Eberhart [2002] Kennedym, J., Eberhart, R..  Particle swarm optimization.  In: Proceedings of ICNN’95 - International Conference on Neural Networks. 2002,.
  * Mahendran and Vedaldi [2015] Mahendran, A., Vedaldi, A..  Understanding deep image representations by inverting them.  In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015, p. 5188–5196.
  * Qiu and Yuille [2016] Qiu, W., Yuille, A..  Unrealcv: Connecting computer vision to unreal engine.  In: European Conference on Computer Vision. Springer; 2016, p. 909–916.
  * Redmon and Farhadi [2018] Redmon, J., Farhadi, A..  Yolov3: An incremental improvement.  arXiv preprint arXiv:180402767 2018;.
  * Redmon and Farhadi [2017] Redmon, J., Farhadi, A..  Yolo9000: better, faster, stronger.  In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, p. 7263–7271.
  * Zhu et al. [2021] Zhu, X., Lyu, S., Wang, X., Zhao, Q..  Tph-yolov5: Improved yolov5 based on transformer prediction head for object detection on drone-captured scenarios.  In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, p. 2778–2788.
  * Ren et al. [2015] Ren, S., He, K., Girshick, R., Sun, J..  Faster r-cnn: Towards real-time object detection with region proposal networks.  Advances in neural information processing systems 2015;28.
  * He et al. [2017] He, K., Gkioxari, G., Dollár, P., Girshick, R..  Mask r-cnn.  In: Proceedings of the IEEE international conference on computer vision. 2017, p. 2961–2969.
  * Qin et al. [2020] Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O.R., Jagersand, M..  U2-net: Going deeper with nested u-structure for salient object detection.  Pattern recognition 2020;106:107404.