# AdvFAS: a robust face anti-spoofing framework against adversarial examples

Jiawei Chen Xiao Yang Heng Yin Mingzhi Ma Bihui Chen Jianteng Peng Yandong Guo
Zhaoxia Yin Hang Su<EMAIL_ADDRESS>Anhui Provincial Key
Laboratory of Multimodal Cognitive Computation, Anhui University, Hefei
230601, China.  Department of Computer Science and Technology, Institute for
AI, THBI Lab, Tsinghua University, Beijing 100084, China.  Zhongguancun
Laboratory, Beijing, 100080, China  OPPO Research Institute, Beijing, China
Shanghai Key Laboratory of Multidimensional Information Processing, East China
Normal University, Shanghai 200241, China.

###### Abstract

Ensuring the reliability of face recognition systems against presentation
attacks necessitates the deployment of face anti-spoofing techniques. Despite
considerable advancements in this domain, the ability of even the most state-
of-the-art methods to defend against adversarial examples remains elusive.
While several adversarial defense strategies have been proposed, they
typically suffer from constrained practicability due to inevitable trade-offs
between universality, effectiveness, and efficiency. To overcome these
challenges, we thoroughly delve into the coupled relationship between
adversarial detection and face anti-spoofing. Based on this, we propose a
robust face anti-spoofing framework, namely AdvFAS, that leverages two coupled
scores to accurately distinguish between correctly detected and wrongly
detected face images. Extensive experiments demonstrate the effectiveness of
our framework in a variety of settings, including different attacks, datasets,
and backbones, meanwhile enjoying high accuracy on clean examples. Moreover,
we successfully apply the proposed method to detect real-world adversarial
examples.

###### keywords:

###### MSC:

41A05, 41A10, 65D05, 65D17 Keyword1, Keyword2, Keyword3

††journal: Computer Vision and Image Understanding

##  1 Introdution

The proliferation of face recognition systems has underscored the paramount
importance of face anti-spoofing technology(Yang et al., 2014; Wang et al.,
2022c; Yu et al., 2022, 2021; Yang et al., 2019). This technology aims to
distinguish between real human faces and various presentation attacks (face
substitutes), such as print, video replay, 3D mask attacks, $etc$. As
highlighted by (Szegedy et al., 2013), deep neural networks are vulnerable to
adversarial examples, which incorporate concealed messages into images.
Furthermore, face recognition systems (Yang et al., 2021a; Yin et al., 2021;
Yang et al., 2021b) based on deep neural networks also have exhibited their
vulnerability against adversarial examples. In terms of face anti-spoofing,
adversarial examples can fool face anti-spoofing systems by introducing
adversarial perturbations to the face substitutes, resulting in the paralysis
of face recognition systems. Therefore, the creation of more realistic
spoofing faces while encompassing a wide range of attack types remains a
pressing concern in the domain of presentation attacks.

Moreover, the study of adversarial attacks is crucial to maximize the effect
of presentation attacks 111we do not consider real-to-fake attacks because
they pose an ill-suited threat to face recognition systems, where the breadth
of attacks is a pivotal consideration. Adversarial attacks can be superimposed
on existing face substitutes, endowing more examples with the ability to
deceive face anti-spoofing technology.

Therefore, in security-sensitive applications, face anti-spoofing must exhibit
enough robustness against both presentation attacks and adversarial attacks.

However, few studies have been concerned about defending adversarial examples
in face anti-spoofing. (Bousnina et al., 2021) propose to defend differential
evolution-based adversarial attacks using eight data augmentation techniques,
which only concern the specific attack. UniFAD(Deb et al., 2021) is proposed
in an attempt to merge digital attacks and presentation attacks under one
framework. However, UniFAD requires a large number of images to be prepared in
advance to train an additional network.

Fig. 1: Left: Conventional detector methods demonstrate very poor performance
in defending adversarial examples. Right: Our AdvFAS can distinguish wrongly
detected examples from correctly detected ones through two coupled scores:
$f_{\theta}(x)$ and ES$(x)$. ES($x$) refers to the expected score.
$g_{\kappa}(x)$ builds the bridge between $f_{\theta}(x)$ and ES$(x)$.

Another limitation is that both of them focus on detecting black-box attacks,
which ignore the more challenging white-box attack detection. Compared to
black-box attacks, it can only observe the output of the targeted model. An
attacker has complete access to the target model in white-box settings,
including the architecture and parameters of the model. Since black-box
attacks are a more realistic assumption in the real world, evaluating models
against white-box attacks is crucial to measure the performance of the model
in the worst-case scenarios (Wiyatno et al., ). Moreover, defenses against
white-box attacks resist insiders, who could access the model by maliciously
creating adversarial examples to jeopardize the performance of face
recognition systems.

Adversarial training is recognized as an effective approach to defending
against white-box attacks(Madry et al., 2017; Zhang et al., 2019). However,
this approach typically leads to degraded detection performance on clean
examples. This phenomenon is unacceptable in face anti-spoofing. A recent
work(Pang et al., 2022) proposes that adversarial examples could be provably
told apart by two coupled rejection metrics. This approach performs
excellently against white-box adversarial attacks and is compatible with
different models for improving robustness. It inspired us to explore the
coupled relationship between face anti-spoofing and adversarial detection.

In this paper, we explore the coupled relationship between the two tasks, as
illustrated in Fig. 1, and propose a novel robust framework (AdvFAS) for face
anti-spoofing to detect white-box adversarial examples. In Sec. 3.2, we
present the theoretical analysis of this coupled relationship in detecting
adversarial examples. Specifically, two coupled scores, $f_{\theta}(x)$ and
ES$(x)$, are defined to explicitly represent the coupled relationship.
$f_{\theta}(x)$ is the output score of the conventional detector, and ES$(x)$
is the expected score. In the case of correctly detected examples, ES$(x)$ is
equal to $f_{\theta}(x)$. ES$(x)$ is equal to the ground truth label (0 or 1)
for wrongly detected examples. However, due to the absence of the true label
$y$, we are unable to obtain ES$(x)$ during testing. Therefore, a correction
score $g_{\kappa}(x)$ is introduced to assist to predict ES$(x)$. We realize
that the conventional detector structure is not sufficient to solve this
problem with a single score $f_{\theta}(x)$, $i.e.$, an additional structure
(named corrector) is required to calculate $g_{\kappa}(x)$. To save memory
costs, we adopt a shared block with a two-head structure to model the detector
and the corrector. The accuracy of clean examples is vulnerable to over-
involvement of the adversarial examples during training. Thus, some strategies
(such as stopping gradients and adding a mask) are adopted to eliminate the
negative effects, which are illustrated in detail in Sec. 3.3.

To demonstrate the effectiveness of the framework, we conduct experiments on
three widely used backbones (Depthnet (Liu et al., 2018), Resnet-18 (He et
al., 2016), and Resnet-50), two publicly available datasets (WMCA (George et
al., 2019) and CASIA-SURF 3DMask (Yu et al., 2020)), and two adversarial
attack methods (PGD (Madry et al., 2017) and patch attack (Tong et al.,
2021)). In addition, we apply the proposed framework to CMFL (George and
Marcel, 2021a) to verify its effectiveness on multi-modal inputs.

Extensive experiments demonstrate that the proposed framework is suitable for
various scenarios and obtains state-of-the-art performance. Moreover, adaptive
attacks (Tramer et al., 2020) are designed to evade our corrector module, for
the purpose of fully validating the effectiveness of the proposed framework.
We further apply our framework to detect real-world adversarial examples, in
order to show its practical applicability.

Our main contributions are summarized as follows:

  * 1.

We thoroughly explore the coupled relationship between face anti-spoofing and
adversarial detection and propose a novel face anti-spoofing framework against
adversarial examples.

  * 2.

We present a theoretical analysis that two coupled scores can distinguish
wrongly detected face images from correctly detected ones.

  * 3.

Extensive experiments show that the proposed framework is universal and
effective in various scenarios. The highest accuracy on adversarial examples
increased by 93.79%.

##  2 Related Work

Face Anti-Spoofing. There has been increasing research on face anti-spoofing
in the last decade (Yu et al., 2022). During the first few years, traditional
handcrafted features (Pan et al., 2007; Li et al., 2016; Freitas Pereira et
al., 2012; Komulainen et al., 2013; Patel et al., 2016) were utilized to
detect print attacks. These include liveness cues, eye-blinking (Pan et al.,
2007; Jee et al., 2006; Li, 2008), head, face motion, and gaze tracking.
However, these handcrafted features cannot be elegantly applied because they
require rich prior knowledge and are vulnerable to being fooled by attacks
with dynamic characteristics ($eg$, replay attacks). With the development of
deep learning, many researchers have turned to deep learning to achieve face
anti-spoofing. Some research (Zhang et al., 2021; Li et al., 2021; Ming et
al., 2022) focuses on using the whole RGB image to extract features. As
illustrated by (Wang et al., 2022a; Grinchuk et al., 2021a), the network
sometimes loses local information when training with whole faces. These
approaches crop the face images into patches, trying to capture more local
information. As presentation attacks become more realistic, RGB images of real
faces and face substitutes become more difficult to distinguish. (George and
Marcel, 2021b; Jia et al., 2021a; Wang et al., 2022b) utilize multi-modal
information ($eg$, deep and infrared) or other sensors. These methods impose
additional demands on sensors or deep estimation methods. Some studies (Jia et
al., 2020, 2021a; Liu et al., 2022) demonstrated that the generalizability of
the face anti-spoofing models is poor. These studies mainly investigate
generalization to unseen domains and unknown attack types. In general, the
research on face anti-spoofing tends to be diverse and practical. However,
most face anti-spoofing research ignores the threat to face recognition
systems from adversarial attacks.

Adversarial Examples Defense. The methods of defending against adversarial
examples can be broadly categorized into two categories: adversarial training
(Madry et al., 2017; Zhang et al., 2019; Wu et al., 2020) and adversarial
detection (Metzen et al., 2017; Gong et al., 2017; Carrara et al., 2018; Zhang
et al., 2018; Ma et al., 2018; Feinman et al., 2017). On the one hand,
adversarial training is widely accepted as the most effective method to
improve the adversarial robustness of deep learning models (Yu et al., 2022).
The main idea is to make the network robust by adding adversarial examples to
the training process through a min-max formulation. (Madry et al., 2017)
proposed the PGD-AT, which signiﬁcantly increased the adversarial robustness
of deep learning models against a wide range of attacks. Based on PGD-AT, some
derivative work has followed their designs and proposed settings such as AWP
(Wu et al., 2020), TRADES (Zhang et al., 2019), and Free-AT (Shafahi et al.,
2019), $etc$. However, adversarial training generally reduces the detection
accuracy on clean examples (Madry et al., 2017), which is particularly evident
in face anti-spoofing according to our experiments. On the other hand,
adversarial detection bypasses the impact on the accuracy on clean examples by
training an additional detector (Metzen et al., 2017; Gong et al., 2017;
Carrara et al., 2018; Zhang et al., 2018) or designing statistics (Ma et al.,
2018; Feinman et al., 2017). The pipeline of (Deb et al., 2021) is similar to
adversarial detection. But training an additional network increases
consumption, and the defense system will be destroyed when this network is
attacked.

Due to such open challenges, we propose a novel robust face anti-spoofing
framework against adversarial examples (AdvFAS). Specifically, we find a
coupled relationship between adversarial detection and face anti-spoofing, and
use this relationship to merge the two tasks into a single framework.

##  3 Method

We propose a new robust framework for face anti-spoofing to detect adversarial
examples by a corrector to correct the score of face anti-spoofing models, as
illustrated in Fig. 2. Specifically, a two-head structure is adopted to
implement the framework. One is a detector that does the conventional face
anti-spoofing task; the other is a corrector that corrects the errors of the
detector through a correction score. The combination of the two can be used to
calculate an expected score as the final score for judging whether a face is
real. In this section, we discuss the proposed framework in detail.

###  3.1 Problem Formulation

Defending against white-box adversarial attacks can always be formulated as a
min-max problem and has also been verified to be effective. For a standard
face anti-spoofing network (detector) $f_{\theta}$, consider a data pair
$(x,y)$, with $x\in\mathbb{R}^{d}$ as the input and $y$ as the true label.
Generally, the problem is formulated as a saddle point problem as follows:

$\underset{\theta}{\min}~{}\mathbb{E}_{p(x,y)}\left[\underset{\xi\in\mathcal{E}}{\max}\mathcal{L}(f_{\theta}(x+\xi),y)\right],\vspace{2ex}$
(1)

where the $x+\xi$ is the adversarial example generated by adding the
perturbation $\xi$ from set $\mathcal{E}=\\{\xi:||\xi||\leq\epsilon\\}$ to the
clean example $x$. In this paper, the coupled relationship of the two tasks
can be similarly formulated as a min-max problem.

What to Correct? Some previous approaches for adversarial examples detection
have associated adversarial characters with error detection. However, face
anti-spoofing for clean examples still has a few errors despite impressive
detection. And, for adversarially trained face anti-spoofing models, there are
still many adversarial examples that can be correctly detected. Therefore, in
contrast to adversarial-based correction, it would be more reasonable to
correct output based on whether the $f_{\theta}(x)$ will be wrong.

Fig. 2: Construction of the objective $\mathcal{L}_{cor}$ in Eq. 7 for
training the corrector, which is the binary cross-entropy (BCE) loss between
ES$(x)$ and $E_{label}$. The corrector shares a main backbone with the
detector, introducing little extra memory cost.

Couple Detector and Corrector. The problem scenario is that the detector has
excellent performance on clean examples with few errors, but has poor ability
to detect adversarial examples. This inspires us not to wholly overturn
current face anti-spoofing technologies but to correct some of the
$f_{\theta}(x)$. Therefore, the corrector is created to tackle the problem and
we construct the expected score (ES) as:

$\vspace{2ex}{\rm ES}(x)=f_{\theta}(x)\cdot g_{\kappa}(x),$ (2)

where $g_{\kappa}(x)\in[0,1]$ is the corrector output and parameterized by
${\kappa}$. Through the ES, we expect the corrector to be aware of whether or
not the $f_{\theta}(x)$ is correct and to correct the $f_{\theta}(x)$ when it
is wrong. In other words, the ES, instead of $f_{\theta}(x)$, is treated as
the final decision. In addition, the corrector and the detector are coupled:
for a wrong $f_{\theta}(x)$, if the $f_{\theta}(x)$ is close to 1, the ES$(x)$
must be close to 0 if the ES works (and vice versa). Because of this property,
attackers cannot achieve successful attacks by attacking the detector or the
corrector.

Optimization Goal. Assume that clean and adversarial examples are detected
incorrectly as $x_{c}^{\prime}$ and $x_{a}^{\prime}$ respectively, where
$x_{a}^{\prime}=\underset{x^{\prime}\in
S(x)}{\arg\max}~{}\mathcal{L}_{spoof}(x^{\prime},y;\theta)$. $S(x)$ is a set
of allowed points around $x$ ($e.g.$, a ball of
$||x^{\prime}-x||_{p}\leq\epsilon$) and $x$ is spoofing. $\mathcal{L}_{spoof}$
means the loss of the detector. Suppose
$x^{*}\in\left\\{x_{c}^{\prime},x_{a}^{\prime}\right\\}$ is the input that is
detected wrongly by the detector. The problem is formulated as:

$\vspace{2ex}\underset{\theta,\kappa}{\min}~{}\mathbb{E}_{p(x,y)}\left[\mathcal{L}_{spoof}(x,y;\theta)+\mathcal{L}_{cor}(x^{*},y;\theta,\kappa)\right],$
(3)

where the $\mathcal{L}_{cor}$ is utilized to optimize ES. Moreover,
$f_{\theta}(x)$ and $g_{\kappa}(x)$ conform in distribution to the above
coupled property by concurrently optimizing $\theta$ and $\kappa$. Eq. 3 can
be considered a min-max problem. For the inner maximization step, the most
aggressive perturbation is found for the face anti-spoofing network under the
$\ell_{p}$ norm constraint to generate adversarial examples, $x_{a}^{\prime}$.
The outer minimization objective is to detect clean examples correctly and to
correct the wrongly detected examples. More details are given in Sec. 3.2 and
Sec. 3.3.

###  3.2 Theoretical Analysis

The problem can be summarized into a set of cases $\mathcal{C}$. $\mathcal{C}$
denotes all possible results for which the input $x$ is detected by the
detector. It can be split into $FP$:
$\\{x|x=spoofing||adv,x\in\mathbb{R}^{d},f_{\theta}(x)\geq\frac{1}{2}\\}$,
$FN$ : $\\{x|x=real,x\in\mathbb{R}^{d},f_{\theta}(x)<\frac{1}{2}\\}$, $TN$ :
$\\{x|x=spoofing||adv,x\in\mathbb{R}^{d},f_{\theta}(x)<\frac{1}{2}\\}$, and
$TP$ : $\\{x|x=real,x\in\mathbb{R}^{d}$, $f_{\theta}(x)\geq\frac{1}{2}\\}$.
Where $adv$ are generated from spoofing faces. Clearly, $FP$ and $FN$ are the
wrong cases, while $TN$ and $TP$ are the correct cases.

At training time, we optimize ES by minimizing the binary cross-entropy (BCE)
loss. Due to the diversity of cases and different optimization strategies of
ES for different cases, an adaptive loss $\mathcal{L}_{cor}$ was designed:

$\displaystyle FP:\mathcal{L}_{cor}(x;\theta,\kappa)$
$\displaystyle=BCE(f_{\theta}(x)\cdot g_{\kappa}(x),0),$ (4) $\displaystyle
FN:\mathcal{L}_{cor}(x;\theta,\kappa)$ $\displaystyle=BCE(f_{\theta}(x)\cdot
g_{\kappa}(x),1),$ (5) $\displaystyle
TN||TP:\mathcal{L}_{cor}(x;\theta,\kappa)$
$\displaystyle=BCE(f_{\theta}(x)\cdot g_{\kappa}(x),f_{\theta}(x)).$ (6)

Defining $E_{label}\in\\{0,1,f_{\theta}(x)\\}$, the adaptive loss can thereby
be rewritten as:

$\mathcal{L}_{cor}(x;\theta,\kappa)=BCE(f_{\theta}(x)\cdot
g_{\kappa}(x),E_{label}).\vspace{2ex}$ (7)

In this case, the optimal solution of minimizing $\mathcal{L}_{cor}$ is
$g_{\kappa}^{*}(x)=\frac{E_{label}}{f_{\theta}(x)}$. The corrector and the
detector can be jointly optimized by minimizing:

$\mathcal{L}_{cs}=\mathcal{L}_{spoof}(x;\theta)+\lambda\cdot\mathcal{L}_{cor}(x^{*};\theta,\kappa).\vspace{2ex}$
(8)

Here, $\lambda$ is a hyper-parameter that defaults to 1.0. The Eq. 8 has the
same optimization objective as Eq. 3. In the training phase, the network
simultaneously optimizes parameters for the detector and the corrector by
$\mathcal{L}_{cs}$. The whole process is described in detail in Algorithm 1 .

Algorithm 1 the optimization strategy of AdvFAS

Input:A detector $f_{\theta}$ with loss function $\mathcal{L}_{spoof}$ and a
corrector $g_{\kappa}$ with a coupled loss $\mathcal{L}_{cor}$.

Input:A clean example $x$ and ground-truth label $y$ (spoofing: 0, real: 1)
and epochs $E$.

Output:$f_{\theta}(x),~{}g_{\kappa}(x).$

1: for e = 0 to $E$-1 do

2: if y = 0 then

3: Generate perturbation $\xi$ to $x$;

4: $x^{*}=x+\xi$;

5: end if

6: Put $x$ to the detector $f_{\theta}$;

7: Put $x$ and $x^{*}$ to the corrector $g_{\kappa}$;

8: Minimize the $\mathcal{L}_{spoof}$;

9: if $f_{\theta}(x)$ or $f_{\theta}(x^{*})$ is correct then

10: Stop the gradient of $f_{\theta}(x)$ or $f_{\theta}(x^{*})$;

11: end if

12: Minimize the $\mathcal{L}_{cor}$.

13: end for

####  3.2.1 Theoretically Coupling ES and $\boldsymbol{f_{\theta}}$

Based on the above design of $\mathcal{L}_{cor}$, we state the optimization
objectives of ES on different cases respectively. We observe that properly
coupling $f_{\theta}$ and ES can provably separate wrongly and correctly
detected inputs, as stated below:

######  Lemma 1.

(Separability) Given the detector $f_{\theta}$, $\forall x_{1},x_{2}$ with
spoofing scores larger than $\frac{1}{2}$, $i.e.$,

$f_{\theta}(x_{1})\geq\frac{1}{2},and~{}f_{\theta}(x_{2})\geq\frac{1}{2}.\vspace{2ex}$
(9)

If $x_{1}$ is detected correctly, while $x_{2}$ is detected wrongly, then
$\rm{ES}$ $(x_{1})$ $>\frac{1}{2}>$ $\rm{ES}$ $(x_{2})$.

Proof. $x_{1}$ conforms to Eq. 6 because it is correctly detected, thus the
ES$(x_{1})$=$f_{\theta}(x_{1})\geq\frac{1}{2}$. On the other side, $x_{2}$ is
wrongly detected and $f_{\theta}(x_{2})\geq\frac{1}{2}$, so it conforms to Eq.
4 and we have ES$(x_{2})=0<\frac{1}{2}$. As for other cases, we have a similar
conclusion.

######  Lemma 2.

(Separability) Given the detector $f_{\theta}$, $\forall x_{1},x_{2}$ with
spoofing scores less than $\frac{1}{2}$, $ie$,

$f_{\theta}(x_{1})<\frac{1}{2},and~{}f_{\theta}(x_{2})<\frac{1}{2}.\vspace{2ex}$
(10)

If $x_{1}$ is detected correctly, while $x_{2}$ is detected wrongly, then
$\rm{ES}$ $(x_{1})$ $<\frac{1}{2}<$ $\rm{ES}$ $(x_{2})$.

Proof. $x_{1}$ conforms to Eq. 6 because it is correctly detected, thus the
ES$(x_{1})$=$f_{\theta}(x_{1})<\frac{1}{2}$. On the other side, $x_{2}$ is
wrongly detected and $f_{\theta}(x_{2})<\frac{1}{2}$, so it conforms to Eq. 5
and we have ES$(x_{2})=1\geq\frac{1}{2}$.

According to Lemma 1 (Lemma 2), when the spoofing scores of the detector are
larger (less) than $\frac{1}{2}$, then for any $x$ that passes the detector,
ES$(x)\geq\frac{1}{2}$ (ES$(x)<\frac{1}{2}$) as long as $x$ is correctly
detected; conversely, ES$(x)<\frac{1}{2}$ (ES$(x)\geq\frac{1}{2}$). The
property of the two coupled scores can be utilized to distinguish wrongly
detected examples from correctly detected ones.

####  3.2.2 Coupling in Practice ES and $\boldsymbol{f_{\theta}}$

Sec. 3.2.1 shows that coupling ES and $f_{\theta}$ is properly separable for
wrongly and correctly detected inputs. However, during actual training, there
is necessarily an error between $g_{\kappa}(x)$ and $g_{\kappa}^{*}(x)$. We
introduce a definition on the point-wise error between $g_{\kappa}(x)$ and
$g_{\kappa}^{*}(x)$:

######  Definition 1.

(Point-wisely $\delta$-error). If a point x satisfies the following range:

$|g_{\kappa}(x)-g_{\kappa}^{*}(x)|\leq\frac{\delta}{2},\vspace{2ex}$ (11)

where $\delta\in[0,1)$, then $g_{\kappa}$ is called $\delta$-error at the
input $x$.

For any $g_{\kappa}(x)$ trained to be better than a random guess at $x$, the
$\delta$ is always found to satisfy Definition 1. In particular, assume that
$g_{\kappa}(x)$ performs a random guess on $x$, meaning
$g_{\kappa}(x)=\frac{1}{2}$. Because $g_{\kappa}^{*}(x)\in[0,1]$, there is
$|\frac{1}{2}-g_{\kappa}^{*}(x)|\leq\frac{1}{2}$, which means that even a
random guess $g_{\kappa}(x)$ can satisfy Definition 1 with $\delta=1$.

We prove the separability of wrongly and correctly detected inputs at the
optimal threshold equal to $\frac{1}{2}$ in Lemma 1. However, in practice we
cannot access the optimal threshold, which is not always equal to
$\frac{1}{2}$ due to training error. To this end, we demonstrate that the
detector with $\frac{1}{2-\delta}$ and the corrector with $\delta$-error
$g_{\kappa}(x)$ can be coupled to achieve separability, similarly to the
property shown in Lemma 1.

######  Theorem 1.

(Separability) Given the detector $f_{\theta}$, for any $x_{1},x_{2}$ with
spoofing scores larger than $\frac{1}{2-\delta}$, $i.e.$,

$f_{\theta}(x_{1})>\frac{1}{2-\delta},~{}and~{}f_{\theta}(x_{2})>\frac{1}{2-\delta},\vspace{2ex}$
(12)

where $\delta\in[0,1)$, $x_{1}$ and $x_{2}$ are detected correctly and wrongly
respectively, and $g_{\kappa}$ is $\delta$-error at $x_{1}$, $x_{2}$, then
there must be $\rm{ES}$ $(x_{1})>\frac{1}{2}>$ $\rm{ES}$ $(x_{2})$.

Specifically, after we reset the threshold of the corrector to
$\frac{1}{2-\delta}$ and obtain the remaining examples, any wrongly detected
examples will be corrected correctly by the ES, as long as $g_{\kappa}$ is
trained to be $\delta$-error for these examples.

Proof. The conditions in 1 can be written as $x_{1}$ is correctly detected,
$f_{\theta}(x_{1})>\frac{1}{2-\delta}$ and $x_{2}$ is wrongly detected,
$f_{\theta}(x_{2})>\frac{1}{2-\delta}$, where $\delta\in[0,1)$. Since
$g_{\kappa}(x)$ is $\delta$-error at $x_{1}$ and $x_{2}$, according to 1, we
have the following range:

$|g_{\kappa}(x)-g_{\kappa}^{*}(x)|\leq\frac{\delta}{2}.\vspace{2ex}$ (13)

For $x_{1}$, there is $g_{\kappa}^{*}(x_{1})=1$. And we can obtain

$\displaystyle{\rm ES}(x_{1})$ $\displaystyle=f_{\theta}(x_{1})\cdot
g_{\alpha}(x_{1})>f_{\theta}(x_{1})\cdot(\frac{2-\delta}{2})$
$\displaystyle>(\frac{1}{2-\delta})\cdot(\frac{2-\delta}{2})=\frac{1}{2}.\vspace{2ex}$
(14)

Similarly for $x_{2}$, there is $g_{\kappa}^{*}(x_{2})=0$. We can obtain:

$\displaystyle{\rm ES}(x_{2})$ $\displaystyle=f_{\theta}(x_{2})\cdot
g_{\alpha}(x_{2})<f_{\theta}(x_{2})\cdot\frac{\delta}{2}$
$\displaystyle<\frac{1}{2-\delta}\cdot\frac{\delta}{2}<\frac{1}{2}.\vspace{2ex}$
(15)

Thus we have proven ES$(x_{1})>\frac{1}{2}>$ ES$(x_{2})$ and the 1 is
established.

###  3.3 Framework Details

Coupling Loss Design. The $\mathcal{L}_{cs}$ consists of $\mathcal{L}_{spoof}$
and $\mathcal{L}_{cor}$. The former is the same as the conventional face anti-
spoofing methods and the latter is utilized to optimize the corrector.
Therefore, the conventional loss of face anti-spoofing is not altered during
training. In other words, the model degenerates to the regular face anti-
spoofing model when $\lambda$ is equal to 0. The particularity ensures that
the proposed framework is compatible with different face anti-spoofing
frameworks. (In the case of the loss function, it is sufficient to add
$\mathcal{L}_{cor}$).

Shared Block. There are three types of possible inputs: adversarial examples
generated from spoofing faces, spoofing faces, and real faces. During the
training period, all three types of images are shuffled and then input into
the shared block. The shared block simultaneously optimizes parameters for the
detector and the corrector. Compared to training on an additional network, the
design of a shared block saves computation and memory costs.

Stop Gradients and Add a Mask. When the images are detected correctly,
stopping gradients on the spoof scores of $\mathcal{L}_{cor}$ can avoid
focusing on easy examples and keep the optimal solution of the detector
unbiased. In particular, we use $p(x,y)$ to represent data distribution. For
$\mathcal{L}_{spoof}$, the optimal solution of minimizing
$\mathcal{L}_{spoof}(f_{\theta}(x),y)$ is $f_{\theta}(x)[y]=p(y|x)$. However,
if we do not stop gradients of $f_{\theta}(x)[y]$ in the corrector, then the
optimal $\theta$ of the entire framework objective no longer satisfies
$f_{\theta}(x)[y]=p(y|x)$. In this case, the corrector will introduce bias in
the optimal solution of the detector. A mask is utilized to eliminate the loss
of adversarial examples to ensure that the corrector works as conventionally
as possible during training. Specifically, the mask, which is an image-level
$(0,1)$ matrix, is implemented to separate adversarial examples from clean
examples (real and spoofing). The zeros in the matrix correspond to the
adversarial examples. For a mask $\mathcal{M}=\\{m_{i}\\}_{i=1}^{n}$, where
$n$ represents the number of images and $m_{i}\in\\{0,1\\}$ refers to the
$i$-th image. Define $x_{i}$ as i-th examples. We can formulate as:

$\mathcal{L}_{spoof}(x_{i},\theta)=m_{i}\cdot{L}_{spoof}(x_{i},\theta).\vspace{2ex}$
(16)

Without the mask, there may be problems caused by data imbalance. During the
training time, too much participation of the adversarial examples will affect
the detection of clean examples.

##  4 Experiments

Table 1: The success rates (%) of adversarial attacks against three models we study. The adversarial examples are crafted for HRFP, CMFL and DBMnet respectively using C&W, PGD, patch, DeepFool and AutoAttack on WMCA and CASIA-SURF 3DMask. Method | WMCA | CASIA-SURF 3DMask
---|---|---
HRFP | CMFL | DBMnet | HRFP | CMFL | DBMnet
C&W | 90.10 | 100.0 | 100.0 | 71.80 | 100.0 | 100.0
PGD | 87.40 | 97.55 | 100.0 | 80.00 | 99.82 | 99.65
Patch | 81.21 | 86.86 | 100.0 | 73.60 | 98.10 | 100.0
DeepFool | 90.00 | 100.0 | 100.0 | 74.20 | 100.0 | 100.0
AutoAttack | 97.00 | 100.0 | 99.78 | 88.40 | 100.0 | 99.78
Table 2: Performance of the baseline systems and the proposed framework
against PGD on WMCA. The performance is tested on three models Depthnet,
Resnet18, and Resnet50 respectively. Note that only color channels are used on
WMCA. The ACC (w/o adv) (%) represents the accuracy of models in detecting
clean examples. The ACC (adv) (%) is the accuracy of models in detecting
adversarial examples. The ACC(avg) refers to the average test accuracy on
clean and adversarial examples.

Method | Depthnet | Resnet18 | Resnet50
---|---|---|---
ACC(w/o adv) | ACC(adv) | ACC(avg) | ACC(w/o adv) | ACC(adv) | ACC(avg) | ACC(w/o adv) | ACC(adv) | ACC(avg)
Clean | 91.10 | 22.15 | 56.63 | 87.58 | 5.59 | 46.59 | 89.90 | 20.33 | 55.12
PGD-AT(eps=16/255) | 25.43 | 99.87 | 62.65 | 22.25 | 100.0 | 61.13 | 25.07 | 94.69 | 59.88
PGD-AWP(eps=16/255) | 34.94 | 78.68 | 56.81 | 29.31 | 95.84 | 62.58 | 22.14 | 100.0 | 61.07
PGD-AT(eps=8/255) | 22.29 | 100.0 | 61.15 | 27.4 | 90.23 | 58.82 | 20.99 | 99.94 | 60.47
PGD-AWP(eps=8/255) | 24.00 | 98.20 | 66.10 | 79.27 | 97.26 | 88.27 | 30.06 | 100.0 | 65.03
Ours | 87.32 | 92.01 | 89.67 | 90.26 | 98.03 | 94.15 | 88.55 | 97.80 | 93.18

Table 3: Performance of the baseline systems and the proposed framework against PGD on WMCA. The performance is tested on CMFL. Method | CMFL
---|---
ACC(w/o adv) | ACC(adv) | ACC(avg)
Clean | 86.57 | 2.11 | 44.34
PGD-AT(eps=16/255) | 21.83 | 99.89 | 60.86
PGD-AT(eps=8/255) | 79.27 | 100.0 | 89.64
PGD-AWP(eps=16/255) | 23.50 | 100.0 | 61.75
PGD-AWP(eps=8/255) | 24.89 | 99.81 | 62.35
Ours | 86.10 | 95.90 | 91.00
Table 4: The AUC of the baseline systems and the proposed framework against
PGD on CASIA-SURF 3DMask.

Model | Method | AUC(%)
---|---|---
Depthnet |  | PGD-AT(eps=16/255)
---
PGD-AT(eps=8/255)
PGD-AWP(eps=16/255)
PGD-AWP(eps=8/255)
Our
| 73.21
---
78.34
68.00
77.22
99.12
CMFL |  | PGD-AT(eps=16/255)
---
PGD-AT(eps=8/255)
PGD-AWP(eps=16/255)
PGD-AWP(eps=8/255)
Our
| 75.19
---
80.63
72.71
75.50
90.94

Table 5: The performance of the baseline systems and the proposed framework aganist other adversarial attacks (MI-FGSM and DI-FGSM) on CASIA-SURF 3DMask. And the performance is tested on Depthnet. Method | Depthnet
---|---
ACC(w/o adv) | ACC(adv) | ACC(avg)
MIFGSM-AT | 68.22 | 99.24 | 83.73
Ours | 85.15 | 93.59 | 89.37
DIFGSM-AT | 71.18 | 98.74 | 84.96
Ours | 94.39 | 87.64 | 91.02
Table 6: Performance of the baseline systems and the proposed framework
against PGD on WMCA. The performance is tested on FAS-wrapper and Patchnet.

Method | FAS-wrapper | Patchnet
---|---|---
ACC(w/o adv) | ACC(adv) | ACC(avg) | ACC(w/o adv) | ACC(adv) | ACC(avg)
Clean | 92.31 | 2.36 | 47.34 | 91.64 | 4.54 | 48.09
PGD-AT(eps=16/255) | 52.51 | 96.18 | 74.35 | 53.68 | 98.39 | 76.04
PGD-AT(eps=8/255) | 73.83 | 99.04 | 86.44 | 74.16 | 97.46 | 85.81
PGD-AWP(eps=16/255) | 51.38 | 98.36 | 74.87 | 58.39 | 98.78 | 78.59
PGD-AWP(eps=8/255) | 72.34 | 96.39 | 84.37 | 73.81 | 98.56 | 86.19
Ours | 87.57 | 94.17 | 90.97 | 88.13 | 95.18 | 91.66

###  4.1 Experimental Settings

Data Preparation. The WMCA and CASIA-SURF 3DMask are used in our experiments
as evaluation datasets. WMCA is a publicly available face image dataset that
contains a variety of presentation attacks in 2D and 3D, such as paper print,
video replay, and 3D mask. We use the grandtest protocol of WMCA to divide it
into the training set, validation set, and test set. Compared with WMCA, face
data on CASIA-SURF 3DMask is in video format. Therefore, data pre-processing
is required for the CASIA-SURF 3DMask. To obtain this, we extract one frame
out of every three frames of each video and adopt MTCNN(Zhang et al., 2016)
for face detection. Then, we crop and resize each extracted frame to the size
of 128${\times}128$. Finally, we utilize an appropriate proportion to divide
these pre-processing frames into the training set, the validation set, and the
test set.

Network Setting. The detector and corrector of the two-head structure adopt
the same neural network, such as Depthnet, Resnet-18, and Resnet-50. Moreover,
deep pixel-wise binary supervision (George and Marcel, 2019) is utilized to
improve the performance of these neural networks for face anti-spoofing.

Attack Setting. Five publicly available adversarial attacks are used in our
experiments, including C&W(Carlini and Wagner, 2017), PGD, patch attack,
DeepFool(Moosavi-Dezfooli et al., 2016) and AutoAttack(Croce and Hein, 2020).
C&W, PGD, AutoAttack, and DeepFool are the most popular and powerful
adversarial attacks in the digital world. In contrast, patch attack is easily
realized in the physical world and has excellent attack capability.

Training Setting. The batch size is set to 100. The $\mathcal{L}_{cs}$ is
optimized with the $\lambda$=1.0. An Adam optimizer with a learning rate (lr)
of 1e-3 and a weight decay of 5e-5 is applied to our experiments. We first
base testing on a validation set to measure the optimal threshold. Then, the
optimal threshold is utilized to calculate the final detection score on a test
set.

###  4.2 Vulnerability Evaluation of Existing Detectors

To evaluate the vulnerability of existing SOTA face-spoofing methods, High
Resolution Face Parts (HRFP)(Grinchuk et al., 2021b), CMFL, and DBMnet (Jia et
al., 2021b). are tested in adversarial attacks. HRFP achieved first place in
the 3D High-Fidelity Mask Face Presentation Attack Detection Challenge based
on the CASIA-SURF HiFiMask(Liu et al., 2021). To ensure fairness, the
checkpoint shared by the original authors is adopted in the evaluation.

As shown in Tab. 1, we utilize adversarial attacks against these three methods
mentioned above on WMCA and CASIA-SURF 3DMask respectively. Any success rate
of adversarial attacks can simply be above 70%, which shows that existing
face-spoofing methods are vulnerable to adversarial attacks. Especially under
the attack configuration of C&W, DeepFool, PGD and AutoAttack in CMFL and
DBMnet, success rates of adversarial attacks can approach 100% no matter on
WMCA or CASIA-SURF 3DMask. Although HRFP segments each face data into
different face features as input to a corresponding convolution network,
success rates of adversarial attacks also reach above 80% on WMCA and 70% on
CASIA-SURF 3DMask. Hence, the existing high-performance face-spoofing
detectors can be easily broken by existing adversarial attack technologies,
$i.e.$, it is necessary for face anti-spoofing methods to defend against
adversarial examples.

Table 7: Performance of the baseline systems and the proposed framework
against patch attack on WMCA. The performance is tested on three models:
Depthnet, Resnet18, and Resnet50. Note that only color channels are used on
WMCA.

Method | Depthnet | Resnet18 | Resnet50
---|---|---|---
ACC(w/o adv) | ACC(adv) | ACC(avg) | ACC(w/o adv) | ACC(adv) | ACC(avg) | ACC(w/o adv) | ACC(adv) | ACC(avg)
Clean | 91.10 | 3.90 | 47.93 | 87.58 | 10.26 | 51.07 | 89.90 | 28.91 | 58.74
patch-AT(eps=255/255) | 21.04 | 99.99 | 60.52 | 21.80 | 99.98 | 60.89 | 20.94 | 99.93 | 60.44
patch-AWP(eps=255/255) | 20.77 | 99.83 | 60.30 | 20.87 | 99.94 | 60.41 | 21.90 | 100.0 | 60.95
patch-AT(eps=125/255) | 30.97 | 99.98 | 65.48 | 21.32 | 99.85 | 60.59 | 79.27 | 100.0 | 89.64
patch-AWP(eps=125/255) | 79.17 | 100.0 | 89.58 | 26.90 | 98.70 | 62.80 | 24.57 | 100.0 | 62.28
Ours | 92.16 | 74.78 | 83.47 | 93.31 | 90.04 | 91.68 | 89.01 | 87.8 | 88.41

Fig. 3: The adversarial examples are crafted on WMCA and CASIA-SURF 3DMask by
patch attack.

###  4.3 Performance Against Different Settings

AdvFAS vs. PGD. The performance of the proposed framework and baselines
against PGD in the grandtest protocol of WMCA are shown in Tab. 2, Tab. 3 and
Tab. 6. In order to fully evaluate, we also use the AUC to evaluate our method
and the baseline in Tab. 4. We test baselines and AdvFAS on four models:
depthnet, resnet18, resnet50 and CMFL. For CMFL, we use the four channels of
RGBD on WMCA. To provide a comprehensive comparison, we also conduct a
comparison with FAS-wrapper(Guo et al., 2022) and Patchnet(Wang et al.,
2022a). Due to their poor robustness, when training with clean examples, they
have no ability to distinguish between adversarial examples and clean
examples, although it has nice accuracy on the clean examples. The accuracy on
adversarial examples is only around 20%. On the contrary, PGD-AT and PGD-AWP
can recognize adversarial examples with higher accuracy, but these methods
push the classification boundary of the training network away from the clean
examples, so the accuracy on the clean examples dramatically reduces. Our
AdvFAS not only performs well on clean examples but also has higher accuracy
on adversarial examples comparable to the above methods.

AdvFAS vs. Patch Attack. Patch attack is different from a slight noise
perturbation adversarial attacks such as PGD, C&W, DeepFool and AutoAttack.
The adversarial examples perturbed by patch attack have a specific shape and
are much easier to implement in the physical world. In Fig. 3, we show some
demos of patch attacks.

|
---|---
(a) adversarial 3D face | (b) adversarial 2D glasses
Fig. 4: Examples of test data in real-world scenarios; a) adversarial 3D face
crafted for a face recognition system, b) adversarial 2D glasses crafted for a
face anti-spoofing system.

We conduct similar experiments in the configuration for patch attacks. The
result is shown in Tab. 7 (We also try to test the methods on MI-FGSM and DI-
FGSM adversarial attacks in Tab. 5). According to the experimental results,
when training with the clean examples, although it has excellent accuracy on
the clean examples, the accuracy on face images with patch attack is only
about 14% on average. Furthermore, different adversarial training strategies
such as patch-AT and patch-AWP can recognize adversarial examples with
excellent accuracy, but the accuracy of the clean examples dramatically
reduce. Our method significantly improves the metric for detecting adversarial
examples while maintaining high performance on clean examples.

Table 8: Performance of the baseline systems and the proposed framework against PGD on CASIA-SURF 3DMask. Method | Depthnet
---|---
ACC(w/o adv) | ACC(adv) | ACC(avg)
Clean | 99.59 | 31.30 | 65.45
PGD-AT(eps=16/255) | 31.88 | 99.71 | 65.80
PGD-AT(eps=8/255) | 68.45 | 100.0 | 84.23
PGD-AWP(eps=16/255) | 31.40 | 99.46 | 65.43
PGD-AWP(eps=8/255) | 31.44 | 100.0 | 65.72
Ours | 99.75 | 99.88 | 99.82

On CASIA-SURF 3DMask. To investigate the generality of the proposed framework,
we compare with baselines on the second dataset, namely CASIA-SURF 3DMask. We
adopt Depthnet as the basic detector, and design experiments to compare AdvFAS
with the baseline method, as shown in Tab. 8. Both the ACC(w/o adv) and
ACC(avg) of the proposed framework provide the best performance compared to
the baseline methods. In fact, the ACC(adv) of our framework is close to 100%.

We attribute the excellent performance of AdvFAS to the coupled relationship
between adversarial detection and face anti-spoofing and the training
strategy. At training time, the coupled relationship, which can provably
distinguish wrongly detected face images from correctly detected ones, can
greatly improve the performance of detecting adversarial examples. Our
training strategy and the two-head structure bring a small modification to the
conventional detector. Therefore, the performance of clean examples fluctuates
in a small range. Moreover, because all wrongly detected examples are
corrected (not just the adversarial examples), the accuracy of clean examples
will also improve if the corrector is well optimized. The experimental results
of AdvFAS on different adversarial attacks, two publicly available datasets,
and widely used backbones are similar. Therefore, our proposed method is
efficient in improving the robustness of face anti-spoofing and is compatible
with various face anti-spoofing methods.

###  4.4 In Real-world Scenarios

In this section, we apply our framework to detect adversarial examples in
real-world scenarios, as illustrated in Fig. 4. We craft adversarial examples
for a face recognition system ArcFace(Deng et al., 2019) and a face anti-
spoofing system, respectively, to validate the effectiveness of our framework
against adversarial attacks in real-world scenarios.

For the Face Recognition System. Adversarial 3D masks and adversarial 3D
patches are crafted to fool face recognition systems. The adversarial 3D
patches are crafted through (Yang et al., 2022), which simulates the complex
transformations of faces in the physical world via 3D-face modeling. We employ
some volunteers and they are required to wear the adversarial 3D face to
record face videos for about 10 seconds. Then, we extract one frame out of
every ten frames of each video. Finally, we adopt MTCNN to get 1555 face
images as the input images.

For the Face Anti-Spoofing System. We conduct a similar experimental
configuration of Sec. 4.3 for patch attack. First, we use patch attacks to
generate adversarial 2D glasses for the CMFL. Then, the adversarial examples
are printed and cropped to the adversarial 2D glasses. Next, the adversarial
2D glasses are put on the 3D mask to record videos, extract frames, and crop
to face images. Finally, the cropped face images are used as the input images
of our proposed framework.

Table 9: Performance in real-world scenarios

Method | Depthnet
---|---
ACC(w/o adv) | ACC(adv 3D face) | ACC(adv 2D glasses) | ACC(avg)
Clean | 85.96 | 93.21 | 20.37 | 66.51
Patch-AT(eps=16/255) | 8.70 | 92.55 | 89.34 | 63.53
Ours | 85.08 | 78.09 | 88.48 | 83.58

The experimental results are shown in Tab. 9. Overall, our proposed framework
outperforms the baseline methods. Although, due to the influence of lighting,
position, and background, the proposed framework and baseline seem to have a
performance gap compared to the results in the digital world. These
experimental results show that the proposed framework still detects
adversarial examples successfully in real-world scenarios. Furthermore, the
results on the adversarial 3D face show that the face anti-spoofing systems
have a certain defense against adversarial examples that can fool the face
recognition systems.

Fig. 5: Performances under adaptive attacks on WMCA. We design four adaptive
objectives to evade both the detector and corrector.

###  4.5 Performance Against Adaptive Attacks

We design adaptive attacks to simultaneously evade the detector and corrector.
Fig. 5 shows the performance of the proposed framework under adaptive
adversarial attacks on WMCA. We explore four different adaptive objectives,
including $\mathcal{L}_{spoof}+\eta\cdot\mathcal{L}_{ens}$,
$\mathcal{L}_{spoof}+\eta\cdot\mathcal{L}_{cor}$,
$\mathcal{L}_{det}+\eta\cdot\mathcal{L}_{ens}$, and
$\mathcal{L}_{det}+\eta\cdot\mathcal{L}_{cor}$, where $\mathcal{L}_{det}$ and
$\mathcal{L}_{cor}$ is used to directly optimize the spoofing score and
correction score respectively. Test Accuracy in each sub-graph represents the
detection accuracy, and the horizontal axis represents the value of the
coefficient $\eta$ of $\mathcal{L}_{ens}$. ACC(adv) is the test accuracy on
adversarial examples crafted on WMCA, and ACC(avg) represents the average test
accuracy on clean and adversarial examples. When $\mathcal{L}_{spoof}$ and
$\mathcal{L}_{ens}$, or $\mathcal{L}_{spoof}$ and $\mathcal{L}_{cor}$ are
combined as the adaptive attack objective, although ACC(adv) and ACC(avg)
decrease at first, the latter trend tends to stabilize with the increase of
$\eta$. When we combined $\mathcal{L}_{det}$ and $\mathcal{L}_{ens}$, or
$\mathcal{L}_{det}$ and $\mathcal{L}_{cor}$, even though $\eta$ is increasing,
the values of ACC(adv) and ACC(avg) are relatively stable.

Although there is some decrease in the ACC(adv) and ACC(avg), the ACC(avg) can
still reach 70% or more and maintain high performance in detecting clean
examples, which means the proposed framework still significantly outperforms
baselines. This can be attributed to the coupled mechanism. Specially, when
adaptive attacks simultaneously evade the detector and corrector, the accuracy
on clean and adversarial examples does not decrease significantly because of
the coupled mechanism.

##  5 Conclusion

In this paper, we first explore the coupled relationship between face anti-
spoofing and adversarial detection and then propose a novel face anti-spoofing
framework (AdvFAS) against adversarial examples. The effectiveness of the
coupled relationship in detecting adversarial examples is proved from both
theoretical and practical aspects. By introducing two coupled scores to
distinguish incorrectly detected face images from correctly detected ones, our
framework significantly improves the robustness of face anti-spoofing
technology. We conduct extensive experiments on three backbones and a multi-
modal method, two publicly available datasets, and different adversarial
attacks to validate the effectiveness our framework. Compared to results
without our framework, our best result shows that the accuracy on adversarial
examples increased by 93.79%. Experimental results demonstrate that our
framework is compatible with various face anti-spoofing methods and maintains
excellent performance. Finally, we detect real-world adversarial examples by
the proposed framework and demonstrate its practical applicability.

## Acknowledgement

This work was supported by the NSFC Projects (Nos. 62076147, 62071292,
62172001, 61771303, U19B2034, U1811461, U19A2081), Tsinghua Institute for Guo
Qiang, Tsinghua-OPPO Joint Research Center for Future Terminal Technology, and
the High Performance Computing Center, Tsinghua University.

## References

  * Bousnina et al. (2021) Bousnina, N., Zheng, L., Mikram, M., Ghouzali, S., Minaoui, K., 2021.  Unraveling robustness of deep face anti-spoofing models against pixel attacks.  Multimedia Tools and Applications 80, 7229–7246.
  * Carlini and Wagner (2017) Carlini, N., Wagner, D., 2017\.  Towards evaluating the robustness of neural networks, in: 2017 ieee symposium on security and privacy (sp), Ieee. pp. 39–57.
  * Carrara et al. (2018) Carrara, F., Becarelli, R., Caldelli, R., Falchi, F., Amato, G., 2018.  Adversarial examples detection in features distance spaces, in: Proceedings of the European conference on computer vision (ECCV) workshops, pp. 0–0.
  * Croce and Hein (2020) Croce, F., Hein, M., 2020.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks, in: International conference on machine learning, PMLR. pp. 2206–2216.
  * Deb et al. (2021) Deb, D., Liu, X., Jain, A.K., 2021.  Unified detection of digital and physical face attacks.  arXiv preprint arXiv:2104.02156 .
  * Deng et al. (2019) Deng, J., Guo, J., Xue, N., Zafeiriou, S., 2019.  Arcface: Additive angular margin loss for deep face recognition, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690–4699.
  * Feinman et al. (2017) Feinman, R., Curtin, R.R., Shintre, S., Gardner, A.B., 2017\.  Detecting adversarial samples from artifacts.  arXiv preprint arXiv:1703.00410 .
  * Freitas Pereira et al. (2012) Freitas Pereira, T.d., Anjos, A., Martino, J.M.D., Marcel, S., 2012\.  Lbp- top based countermeasure against face spoofing attacks, in: Asian Conference on Computer Vision, Springer. pp. 121–132.
  * George and Marcel (2019) George, A., Marcel, S., 2019\.  Deep pixel-wise binary supervision for face presentation attack detection, in: 2019 International Conference on Biometrics (ICB), IEEE. pp. 1–8.
  * George and Marcel (2021a) George, A., Marcel, S., 2021a.  Cross modal focal loss for rgbd face anti-spoofing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7882–7891.
  * George and Marcel (2021b) George, A., Marcel, S., 2021b.  Cross modal focal loss for rgbd face anti-spoofing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7882–7891.
  * George et al. (2019) George, A., Mostaani, Z., Geissenbuhler, D., Nikisins, O., Anjos, A., Marcel, S., 2019\.  Biometric face presentation attack detection with multi-channel convolutional neural network.  IEEE Transactions on Information Forensics and Security 15, 42–55.
  * Gong et al. (2017) Gong, Z., Wang, W., Ku, W.S., 2017.  Adversarial and clean data are not twins.  arXiv preprint arXiv:1704.04960 .
  * Grinchuk et al. (2021a) Grinchuk, O., Parkin, A., Glazistova, E., 2021a.  3d mask presentation attack detection via high resolution face parts, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 846–853.
  * Grinchuk et al. (2021b) Grinchuk, O., Parkin, A., Glazistova, E., 2021b.  3d mask presentation attack detection via high resolution face parts, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 846–853.
  * Guo et al. (2022) Guo, X., Liu, Y., Jain, A., Liu, X., 2022.  Multi-domain learning for updating face anti-spoofing models, in: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIII, Springer. pp. 230–249.
  * He et al. (2016) He, K., Zhang, X., Ren, S., Sun, J., 2016.  Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
  * Jee et al. (2006) Jee, H.K., Jung, S.U., Yoo, J.H., 2006.  Liveness detection for embedded face recognition system.  International Journal of Biological and Medical Sciences 1, 235–238.
  * Jia et al. (2021a) Jia, Y., Zhang, J., Shan, S., 2021a.  Dual-branch meta-learning network with distribution alignment for face anti-spoofing.  IEEE Transactions on Information Forensics and Security 17, 138–151.
  * Jia et al. (2021b) Jia, Y., Zhang, J., Shan, S., 2021b.  Dual-branch meta-learning network with distribution alignment for face anti-spoofing.  IEEE Transactions on Information Forensics and Security 17, 138–151.
  * Jia et al. (2020) Jia, Y., Zhang, J., Shan, S., Chen, X., 2020.  Single-side domain generalization for face anti-spoofing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8484–8493.
  * Komulainen et al. (2013) Komulainen, J., Hadid, A., Pietikäinen, M., 2013.  Context based face anti-spoofing, in: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), IEEE. pp. 1–8.
  * Li (2008) Li, J.W., 2008.  Eye blink detection based on multiple gabor response waves, in: 2008 International Conference on Machine Learning and Cybernetics, IEEE. pp. 2852–2856.
  * Li et al. (2021) Li, S., Zhu, X., Feng, G., Zhang, X., Qian, Z., 2021\.  Diffusing the liveness cues for face anti-spoofing, in: Proceedings of the 29th ACM International Conference on Multimedia, pp. 1636–1644.
  * Li et al. (2016) Li, X., Komulainen, J., Zhao, G., Yuen, P.C., Pietikäinen, M., 2016.  Generalized face anti-spoofing by detecting pulse from face videos, in: 2016 23rd International Conference on Pattern Recognition (ICPR), IEEE. pp. 4244–4249.
  * Liu et al. (2021) Liu, A., Zhao, C., Yu, Z., Su, A., Liu, X., Kong, Z., Wan, J., Escalera, S., Escalante, H.J., Lei, Z., et al., 2021.  3d high-fidelity mask face presentation attack detection challenge, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 814–823.
  * Liu et al. (2022) Liu, Y., Chen, Y., Dai, W., Li, C., Zou, J., Xiong, H., 2022.  Causal intervention for generalizable face anti-spoofing, in: 2022 IEEE International Conference on Multimedia and Expo (ICME), IEEE. pp. 01–06.
  * Liu et al. (2018) Liu, Y., Jourabloo, A., Liu, X., 2018.  Learning deep models for face anti-spoofing: Binary or auxiliary supervision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 389–398.
  * Ma et al. (2018) Ma, X., Li, B., Wang, Y., Erfani, S.M., Wijewickrema, S., Schoenebeck, G., Song, D., Houle, M.E., Bailey, J., 2018\.  Characterizing adversarial subspaces using local intrinsic dimensionality.  arXiv preprint arXiv:1801.02613 .
  * Madry et al. (2017) Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A., 2017.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083 .
  * Metzen et al. (2017) Metzen, J.H., Genewein, T., Fischer, V., Bischoff, B., 2017\.  On detecting adversarial perturbations.  arXiv preprint arXiv:1702.04267 .
  * Ming et al. (2022) Ming, Z., Yu, Z., Al-Ghadi, M., Visani, M., Luqman, M.M., Burie, J.C., 2022\.  Vitranspad: Video transformer using convolution and self-attention for face presentation attack detection, in: 2022 IEEE International Conference on Image Processing (ICIP), IEEE. pp. 4248–4252.
  * Moosavi-Dezfooli et al. (2016) Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P., 2016.  Deepfool: a simple and accurate method to fool deep neural networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574–2582.
  * Pan et al. (2007) Pan, G., Sun, L., Wu, Z., Lao, S., 2007.  Eyeblink-based anti-spoofing in face recognition from a generic webcamera, in: 2007 IEEE 11th international conference on computer vision, IEEE. pp. 1–8.
  * Pang et al. (2022) Pang, T., Zhang, H., He, D., Dong, Y., Su, H., Chen, W., Zhu, J., Liu, T.Y., 2022.  Two coupled rejection metrics can tell adversarial examples apart, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15223–15233.
  * Patel et al. (2016) Patel, K., Han, H., Jain, A.K., 2016.  Secure face unlock: Spoof detection on smartphones.  IEEE transactions on information forensics and security 11, 2268–2283.
  * Shafahi et al. (2019) Shafahi, A., Najibi, M., Ghiasi, M.A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., Goldstein, T., 2019.  Adversarial training for free!  Advances in Neural Information Processing Systems 32\.
  * Szegedy et al. (2013) Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R., 2013.  Intriguing properties of neural networks.  arXiv preprint arXiv:1312.6199 .
  * Tong et al. (2021) Tong, L., Chen, Z., Ni, J., Cheng, W., Song, D., Chen, H., Vorobeychik, Y., 2021\.  Facesec: A fine-grained robustness evaluation framework for face recognition systems, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13254–13263.
  * Tramer et al. (2020) Tramer, F., Carlini, N., Brendel, W., Madry, A., 2020\.  On adaptive attacks to adversarial example defenses.  Advances in Neural Information Processing Systems 33, 1633–1645.
  * Wang et al. (2022a) Wang, C.Y., Lu, Y.D., Yang, S.T., Lai, S.H., 2022a.  Patchnet: A simple face anti-spoofing framework via fine-grained patch recognition, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20281–20290.
  * Wang et al. (2022b) Wang, W., Wen, F., Zheng, H., Ying, R., Liu, P., 2022b.  Conv-mlp: A convolution and mlp mixed model for multimodal face anti-spoofing.  IEEE Transactions on Information Forensics and Security 17, 2284–2297.
  * Wang et al. (2022c) Wang, Z., Wang, Q., Deng, W., Guo, G., 2022c.  Learning multi-granularity temporal characteristics for face anti-spoofing.  IEEE Transactions on Information Forensics and Security 17, 1254–1269.
  * (44) Wiyatno, R., Xu, A., Dia, O., de Berker, A., .  Adversarial examples in modern machine learning: A review. arxiv 2019.  arXiv preprint arXiv:1911.05268 .
  * Wu et al. (2020) Wu, D., Xia, S.T., Wang, Y., 2020.  Adversarial weight perturbation helps robust generalization.  Advances in Neural Information Processing Systems 33, 2958–2969.
  * Yang et al. (2014) Yang, J., Lei, Z., Li, S.Z., 2014.  Learn convolutional neural network for face anti-spoofing.  arXiv preprint arXiv:1408.5601 .
  * Yang et al. (2021a) Yang, J., Liu, J., Han, R., Wu, J., 2021a.  Generating and restoring private face images for internet of vehicles based on semantic features and adversarial examples.  IEEE Transactions on Intelligent Transportation Systems .
  * Yang et al. (2021b) Yang, X., Dong, Y., Pang, T., Su, H., Zhu, J., Chen, Y., Xue, H., 2021b.  Towards face encryption by generating adversarial identity masks, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3897–3907.
  * Yang et al. (2022) Yang, X., Dong, Y., Pang, T., Xiao, Z., Su, H., Zhu, J., 2022.  Controllable evaluation and generation of physical adversarial patch on face recognition.  arXiv e-prints , arXiv–2203.
  * Yang et al. (2019) Yang, X., Luo, W., Bao, L., Gao, Y., Gong, D., Zheng, S., Li, Z., Liu, W., 2019.  Face anti-spoofing: Model matters, so does data, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3507–3516.
  * Yin et al. (2021) Yin, B., Wang, W., Yao, T., Guo, J., Kong, Z., Ding, S., Li, J., Liu, C., 2021.  Adv-makeup: A new imperceptible and transferable attack on face recognition.  arXiv preprint arXiv:2105.03162 .
  * Yu et al. (2022) Yu, Z., Qin, Y., Li, X., Zhao, C., Lei, Z., Zhao, G., 2022.  Deep learning for face anti-spoofing: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence .
  * Yu et al. (2021) Yu, Z., Qin, Y., Zhao, H., Li, X., Zhao, G., 2021\.  Dual-cross central difference network for face anti-spoofing.  arXiv preprint arXiv:2105.01290 .
  * Yu et al. (2020) Yu, Z., Wan, J., Qin, Y., Li, X., Li, S.Z., Zhao, G., 2020.  Nas-fas: Static-dynamic central difference network search for face anti-spoofing.  IEEE transactions on pattern analysis and machine intelligence 43, 3005–3023.
  * Zhang et al. (2018) Zhang, C., Ye, Z., Wang, Y., Yang, Z., 2018.  Detecting adversarial perturbations with saliency, in: 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP), IEEE. pp. 271–275.
  * Zhang et al. (2019) Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M., 2019.  Theoretically principled trade-off between robustness and accuracy, in: International conference on machine learning, PMLR. pp. 7472–7482.
  * Zhang et al. (2016) Zhang, K., Zhang, Z., Li, Z., Qiao, Y., 2016.  Joint face detection and alignment using multitask cascaded convolutional networks.  IEEE signal processing letters 23, 1499–1503.
  * Zhang et al. (2021) Zhang, W., Liu, H., Ramachandra, R., Liu, F., Shen, L., Busch, C., 2021\.  Face presentation attack detection using taskonomy feature.  arXiv preprint arXiv:2111.11046 .