# Semantic Contextualization of Face Forgery: A New Definition, Dataset, and
Detection Method

Mian Zou, Baosheng Yu, Yibing Zhan, Siwei Lyu, and Kede Ma Mian Zou and Kede
Ma are with the Department of Computer Science, City University of Hong Kong,
Kowloon, Hong Kong (e-mail<EMAIL_ADDRESS>kede.ma@cityu.edu.hk).Baosheng Yu is with the School of Computer Science, The
University of Sydney, Darlington, NSW, Australia (e-mail:
baosheng.yu@sydney.edu.au).Yibing Zhan is with the JD Explore Academy,
Beijing, China (e-mail: zhanyibing@jd.com).Siwei Lyu is with the Department of
Computer Science and Engineering, University at Buffalo, State University of
New York, Buffalo, NY USA (e-mail: siweilyu@buffalo.edu)._Corresponding
author: Kede Ma_.

###### Abstract

In recent years, deep learning has greatly streamlined the process of
generating realistic fake face images. Aware of the dangers, researchers have
developed various tools to spot these counterfeits. Yet none asked the
fundamental question: What digital manipulations make a real photographic face
image fake, while others do not? In this paper, we put face forgery in a
semantic context and define that computational methods that alter semantic
face attributes to exceed human discrimination thresholds are sources of face
forgery. Guided by our new definition, we construct a large face forgery image
dataset, where each image is associated with a set of labels organized in a
hierarchical graph. Our dataset enables two new testing protocols to probe the
generalization of face forgery detectors. Moreover, we propose a semantics-
oriented face forgery detection method that captures label relations and
prioritizes the primary task (_i.e._ , real or fake face detection). We show
that the proposed dataset successfully exposes the weaknesses of current
detectors as the test set and consistently improves their generalizability as
the training set. Additionally, we demonstrate the superiority of our
semantics-oriented method over traditional binary and multi-class
classification-based detectors.

###### Index Terms:

Face forgery detection, face semantics, datasets.

##  I Introduction

The recent strides in deep learning [1, 2, 3] have significantly facilitated
face forgery [4]. Alongside its entertaining applications, face forgery sparks
widespread public anxieties due to reported misuses. Instances include the
generation of nonconsensual pornography, biometric fraud, service disruption,
and political manipulation.

While several face forgery detection tools have been created, their
generalizability to novel face manipulations remains limited. Early detectors
rely primarily on three types of knowledge. The first is knowledge about
statistical regularities of real face images [5, 6]. The second is knowledge
about the digital photography pipeline, in which various visual artifacts may
arise and can be characterized to expose forgeries [7, 8, 9, 10, 11]. The
third is knowledge about the 3D world we are living in, particularly the
physical laws that govern the interactions of light, optics, and objects [12,
13, 14, 15]. However, designing computational structures manually that exploit
domain knowledge is a highly challenging task. Consequently, existing
knowledge-driven methods are often tailored to specific forgery scenarios,
resulting in limited generalizability. With the advent of deep learning, data-
driven detectors [16, 17, 18, 19, 20, 21, 22, 23] have come to the forefront,
whose effectiveness is attributed to the quality of training data and the
formulation of face forgery detection.

Typically, face forgery detection is formulated as a standard binary
classification problem. A natural extension is multi-class (_i.e._ , $C$-way)
classification [24, 25], in which $C-1$ classes are designated for $C-1$
different types of fake manipulations, while the remaining class is dedicated
to encompassing all real manipulations. Despite the demonstrated success, a
fundamental question in this field has been treated superficially:

What digital manipulations make a real photographic face image fake, while
others do not?

If we are unable to draw a boundary between real and fake manipulations, it is
not possible to discuss the generalizability of face forgery detectors.
Previous generation tests involve training detectors on some “deemed fake”
manipulations (_e.g._ , Deepfakes [1]) and measuring the performance on
another set of “deemed fake” manipulations (_e.g._ , Face2Face [26]). Clearly,
this setting does not accurately reflect the complexities of real-world
scenarios, which are far less constrained.

In this paper, we rethink face forgery from conceptual and computational
perspectives. We first define face forgery in a semantic context:

Computational methods that alter semantic face attributes to exceed human
discrimination thresholds are sources of face forgery.

Here, the description “to exceed human discrimination thresholds” means that
the alteration of semantic face attributes relative to the initial real
photographic image is discernable to the human eye. Fig. 1 presents an example
of diffusion autoencoders [27] on age manipulation. By moving the image latent
along the age direction111Such a direction can be identified as the weight
vector of a linear classifier trained on latent codes of positive and negative
images of the age attribute [27]., _i.e._ , tuning the age parameter
$d_{\mathrm{age}}$, we can generate a sequence of manipulated images, of which
some are easily distinguishable from the original image222The original real
photographic face image is chosen from the FaceForensics++ dataset [28] for
illustration purposes only. (_e.g._ , $d_{\mathrm{age}}\leq-0.20$). In
contrast, images with an age parameter greater than $-0.10$ are innocuous in
real-world applications because they retain nearly all semantic face
attributes.

(a) Original face

(b) $d_{\mathrm{age}}=-0.05$

(c) $d_{\mathrm{age}}=-0.10$

(d) $d_{\mathrm{age}}=-0.15$

(e) $d_{\mathrm{age}}=-0.20$

(f) $d_{\mathrm{age}}=-0.25$

(g) $d_{\mathrm{age}}=-0.30$

Figure 1: Illustrating of diffusion autoencoders [27] on age manipulation. By
varying the age parameter $d_{\mathrm{age}}$, which controls the movement of
the image latent along the age direction, we create a set of age-manipulated
images, only a subset of which are considered fake according to our definition
(_e.g._ , those with $d_{\mathrm{age}}\leq-0.20$). A more negative number
indicates an older age.

Guided by our definition, we create a new Face Forgery in the Semantic Context
(FFSC) dataset, where each image in FFSC is associated with a set of semantic
labels organized in a hierarchical acyclic graph, as shown in Fig. 2. We
contextualize twelve popular face manipulations [27, 29, 30, 31, 32, 33, 20,
34, 35, 36, 37, 38] into five global face attributes (_i.e._ , age,
expression, gender, identity, and pose). Notably, a single manipulation method
can modify multiple attributes, while multiple different methods can modify
the same face attribute. In FFSC, the manipulation degree that exceeds human
discrimination thresholds and the connectivity to the local face regions
(_i.e._ , eye, eyebrow, lip, mouth, nose, and skin) are determined using
formal psychophysical testing. FFSC supports two fine-grained semantics-
oriented testing protocols: 1) generalization to novel manipulation methods
for the same face attribute, and 2) generalization to novel face attributes.

(a)

(b)

Figure 2:  (a) Hierarchical graph for label relation encoding in FFSC. We
partition the root node, denoted as face, into five global face attribute
nodes, age, expression, gender, identity, and pose, each of which is further
connected to a set of leaf nodes, representing local face regions. (b) Parsing
of the face in Fig. 1 (a) into non-overlapping local face regions.

Moreover, we introduce a new semantics-oriented (SO) face forgery detection
method. We first compute the joint probability distribution over the label
hierarchy as a way of encoding label relations. Next, we derive marginal
probabilities for all labels, each corresponding to a standard binary
classification task. This leads to a multi-task learning setting, in which we
prioritize the primary task—detecting whether a face image is real or
fake—through bi-level optimization [39]. Our SO-detection method offers two
significant advantages. First, it encourages learning transferable features
across manipulations that alter the same face attribute, rather than relying
solely on manipulation-specific cues. Second, it enables integration of
features at the semantic level (by detecting manipulations of global face
attributes) with those at the signal level (by detecting manipulations in
specific face regions) through end-to-end optimization.

Extensive experiments show that the proposed FFSC dataset poses a challenge to
current face forgery detectors as a test set and is more effective in inducing
more generalizable SO-detectors as a training set, surpassing those trained on
FF++ [28], DFDC [40], and ForgeryNet [24]. Additionally, we demonstrate the
superiority of the proposed SO-detection method over the binary and multi-
class classification-based counterparts. In summary, our contributions include

  * •

a new definition of face forgery that emphasizes the importance of face
semantics,

  * •

a new dataset of face forgery that includes a semantic label hierarchy for
each image, and

  * •

a new face forgery detection method that is oriented to rely on face
semantics.

##  II Related Work

In this section, we provide a concise review of representative face forgery
datasets and detection methods.

###  II-A Face Forgery Datasets

Four major face manipulation techniques are adopted in the construction of
existing face forgery datasets. The first is face editing supplied by Adobe®
Photoshop® (_e.g._ , Face-Aware Liquify), which provides high-level semantic
abstractions for face manipulations [41]. The second is face swapping based on
autoencoders [1], which replaces the face from a target image/video with that
from the source. The third is the application of conditional generative models
[2, 42], based on generative adversarial networks (GANs) [43] and denoising
diffusion models [2], for controllable face editing. The fourth is image-based
face rendering [26], which estimates (morphable) 3D face models from input
images/videos [44], followed by alignment and re-rendering. Representative
face forgery datasets include UADFV [45], FaceForensics++ (FF++) [28], Celeb-
DF [46], DeepFakeDetection (Google-DFD) [47], DeeperForensics-1.0 (DF-1.0)
[48], DeepFake Detection Challenge (DFDC) [40], FFIW [49], ForgeryNet [24],
and OW-DFA [25] with consistent improvements in dataset size, sample
complexity, identity diversity, visual quality, task complexity, and ethical
approval. Table I presents a detailed summary of these datasets, following the
binary and $C$-way classification formulations. In stark contrast, the
proposed FFSC dataset contextualizes face manipulations from the perspective
of face semantics.

TABLE I: Summary of face forgery datasets

Dataset | # Real Samples | # Fake Samples | # Manipulations | Formulation
---|---|---|---|---
UADFV [45] | 49 | 49 | 1 | Binary
FF++ [28] | 1,000 | 4,000 | 4 | Binary
Google-DFD [47] | 363 | 3,068 | 5 | Binary
Celeb-DF [46] | 590 | 5,639 | 1 | Binary
DF-1.0 [48] | 50,000 | 10,000 | 1 | Binary
DFDC [40] | 23,564 | 104,500 | 8 | Binary
FFIW [49] | 10,000 | 10,000 | 3 | Binary
ForgeryNet [24] | 99,630 | 121,617 | 15 |  Binary/$C$-way
OW-DFA [25] | 25,000 | 34,000 | 20 |  $C$-way
FFSC (Ours) | 63,344 | 83,840 | 12 | Semantics-oriented

###  II-B Face Forgery Detectors

Traditional forensics tools are designed to detect image forgeries by spotting
statistical irregularities [5, 6], visual artifacts (such as demosaicking [7],
chromatic aberration [8], vignetting [9], and noise [11]), and physical and
geometric inconsistencies [12, 13, 14, 15] in the image. Face forgery
detection has been significantly influenced by these methods as a subfield of
forensic science. Generally, these knowledge-driven methods are limited by the
expressiveness of handcrafted (and manipulation-specific) features.

With the advancements of deep learning, many methods learn to expose face
forgeries from physiological signals, including eye blinking [16], head pose
[45], pupil shape [50], corneal specularity [51], and behavioral patterns
[52]. Pure data-driven approaches in the spatial [28, 17, 53, 54, 55, 20, 56,
57] and frequency [58, 59, 60] domain have also been proposed, coupling with
advanced learning strategies, such as attention learning [21, 61, 62],
adversarial learning [63], meta-learning [64], graph learning [65, 66],
contrastive learning [67, 68], and multi-task learning [19, 25, 69, 70].
Empirically, data-driven methods tend to overfit training manipulations and
struggle to generalize to novel manipulations.

Researchers have recently begun detecting fake faces by examining face
semantics [71, 22]. Haliassos _et al._ [71] leveraged lipreading features,
while Dong _et al._ [22] exploited face identity features. In this paper, we
further explore this direction, and conduct a systematic reexamination of face
forgery at the semantic level. The resulting SO-detectors can be loosely seen
as a generalization of the above two methods, with significantly improved
detection performance.

(a) Age by diffusion autoencoders (b) Age by StyleRes (c) Expression by first-
order motion (d) Expression by StyleRes

(e) Expression by first-order motion (f) Gender by StyleGAN2 distillation (g)
Gender by diffusion autoencoders (h) Identity by SimSwap

(i) Identity by FSGAN (j) Identity by BlendFace (k) Pose by first-order motion
(l) Pose by thin-plate spline motion

Figure 3: Face manipulations adopted in the FFSC dataset. Each subfigure
displays the original and manipulated face images on the left and right,
respectively.

##  III FFSC Dataset

In this section, we summarize our efforts to create the FFSC dataset,
including data collection, face manipulation, and data annotation.

###  III-A Data Collection

We start by collecting real photographic videos from two popular datasets,
AVSpeech [72] and Celeb-DF [46] YouTube-real, which contain a diverse set of
video clips of different ages, expressions, genders, identities, poses,
ethnicities, and shooting conditions. We randomly select $700$ and $300$ high-
resolution videos from the AVSpeech test set and Celeb-DF YouTube-real,
respectively. As most face manipulation methods are image-based, we first
uniformly sample $128$ frames from each video and detect the face regions in
each frame by RetinaFace [73]. We only retain the largest face and extend it
to $1.3^{2}$ times the area of the tight crop produced by the face detector,
as suggested in [28]. We adjust each face image to a fixed size of $317\times
317$. Importantly, we exclude face images with low visual quality, closed
eyes, extreme poses, and occlusions, which takes roughly a week by the first
author and a research assistant at City University of Hong Kong. In total, we
collect $63,344$ real face images, corresponding to $1,000$ face identities.

###  III-B Face Manipulation

We construct the FFSC dataset by instantiating five global face attribute
nodes: age, expression, gender, identity, and pose, through twelve face
manipulation methods. We leverage eight face manipulation methods [27, 29, 30,
31, 32, 33, 20, 34] to build the main FFSC dataset, including the well-split
training, validation, and test sets with the ratio of $7.8:1.1:1.1$. The
independence of face identity is ensured during splitting. To create an
additional test set, we reserve four face manipulation algorithms [38, 35, 36,
37]. This is designed to assess the generalizability of face forgery detectors
when exposed to new manipulations that alter the same face attributes seen in
the training set. To keep a data balance between real and fake face images in
the main FFSC dataset, we randomly select one-eighth of real images for face
manipulation, totaling $75,176$ fake images. As for the additional FFSC test
set, we manipulate the real images from the validation and test sets of the
main FFSC dataset to obtain $8,664$ fake images.

Age Manipulation. To alter the age attribute, we use two distinct methods,
diffusion autoencoders [27] and StyleRes [29], which are based on denoising
diffusion models [74] and GANs for style transfer [3], respectively. These
techniques are capable of adjusting the perceived age, either to appear more
youthful (see Fig. 3 (a)) or more aged (see Fig. 3 (b)).

Expression Manipulation. To alter the expression attribute, we consider two
target expressions - smile and surprise (from the neutral face), and adopt two
different methods: the first-order motion model [30] and StyleRes [29]. Unlike
StyleRes, which takes a single face image as input, the first-order motion
model requires an additional driving video to transform the source face image
into a video clip that imitates the face expression in the driving video.

Gender Manipulation. To flip the gender attribute, we adopt diffusion
autoencoders [27] and StyleGAN2 distillation [31], _i.e._ , converting males
to females (see Fig. 3 (f)) and vice versa (see Fig. 3 (g)). Note that
StyleGAN2 distillation is an image-to-image translation method that does not
offer adjustable parameters to control the degree of manipulation.

Identity Manipulation. To alter the identity attribute, we follow the practice
in existing face forgery datasets [28, 46, 40, 24], which swap two faces with
different identities by SimSwap [32], FSGAN [33], and BlendFace [20]. The
former two are data-driven methods based on deep learning, while the latter is
a knowledge-driven algorithm. For each target face, we first search for the
best source face by minimizing the Euclidean distance between detected face
landmarks [20], while excluding faces with the same identity and different
gender. Figs. 3 (h)-(j) show the visual examples, which involve no
manipulation degree tuning.

Pose Manipulation. To alter the pose attribute, particularly the head posture
in the horizontal plane, we adopt two methods: the first-order motion model
[30] and the thin-plate spline motion model [34]. Similar to the former, the
thin-plate spline motion model also requires a driving video as input to
supply the horizontal head rotation movement, and allows more complex
nonlinear motion transfer. The visual examples are shown in Figs. 3 (k) and
(l).

As for the additional test set in FFSC, we employ four different face
manipulation algorithms: HFGI [38] (for both age and expression attributes),
StyleCLIP [35] (for the gender attribute), InfoSwap [36] (for the identity
attribute), and FNeVR [37] (for the pose attribute). HFGI, StyleCLIP, and
InfoSwap are GAN-based and only need a single face image as input, whereas
FNeVR is based on motion transfer, which needs an additional driving video as
input.

###  III-C Data Annotation

####  III-C1 Specification of Manipulation Degree Parameters

For the face manipulation under consideration, we tune its manipulation degree
parameter (if any) to generate a series of manipulated images. We then
estimate human discrimination thresholds using a yes-no task. On each trial,
three subjects are shown a real face image and a corresponding manipulated
image (for one second and in randomized spatial order), and then asked to
indicate whether the two images are perceptually different. This procedure is
repeated for $300$ trials for each manipulation degree value and over $50$
image pairs, with ordering determined by a standard psychophysical staircase
procedure [75]. The distribution of human responses, as a function of the
manipulation degree parameter, is fitted using a Gaussian cumulative
distribution function, and the human discrimination threshold is set to the
parameter value such that the subjects can clearly perceive the visual
differences between the two images $75\%$ of the time. Finally, we transfer
the fitted degree parameter of each manipulation method (on the $50$ real
images) to all remaining real images for fake face image generation.

(a) Age by diffusion autoencoders (b) Age by StyleRes (c) Expression by first-
order motion (d) Expression by StyleRes

(e) Expression by first-order motion (f) Gender by StyleGAN2 (g) Gender by
diffusion autoencoders (h) Identity by SimSwap

(i) Identity by FSGAN (j) Identity by BlendFace (k) Pose by first-order motion
(l) Pose by thin-plate spline motion

Figure 4: Human discrimination distributions of face attributes/regions. Zoom
in for improved visibility.

####  III-C2 Specification of Label Hierarchy

To link each fake image to a label hierarchy (refer to Fig. 2 (a)), it is
essential to determine 1) whether altering one global face attribute impacts
other non-targeted attributes and 2) which specific changes in local face
regions result in the modification of the global face attribute (_i.e._ , the
connectivity between the global face attribute to the local face regions).
Towards this goal, we invite a group of $12$ human subjects to participate in
another formal psychophysical experiment. Similarly, a pair of real and
manipulated images are shown to at least three subjects (for unlimited time
and in randomized spatial order), who are asked to indicate whether the
modification of non-targeted face attributes/regions is visually
discriminable, each corresponding to a yes-no task. This process is repeated
over $200$ image pairs for each face manipulation method, with human
discrimination distributions shown in Fig. 4. We find that subjects are
generally confident in perceiving non-targeted face attributes/regions as
clearly discriminable (or indiscriminable) with a probability higher than
$70\%$ (or lower than $30\%$), which is associated with a binary label of one
(or zero). As for the non-targeted attributes where the probabilities of
discrimination fall within the range of $30\%$ to $70\%$—for instance, the age
attribute when adjusting for gender and identity—we refrain from assigning
definitive binary labels. In such cases, we consider the underlying label of
the non-targeted attribute to be unobserved. Additionally, the clear
discrimination of targeted face attributes, each with a probability over
$75\%$, verifies the specification of manipulation degree parameters in the
preceding psychophysical experiment.

The results of the psychophysical experiment suggest a hierarchical acyclic
graph instantiation to encode the semantic labels as shown in Fig. 2 (a).
Starting from the root node face, we split it into five global face attribute
nodes: age, expression, gender, identity, and pose, and connect them to a set
of leaf nodes representing non-overlapping local face regions: eye, eyebrow,
lip, mouth, nose, and skin. We share the label hierarchy,
$\bm{y}\in\\{0,1\\}^{N}$, for images with the targeted face attribute
manipulated by the same method. $y_{0}=1$ indicates that the test face image
is fake, while $y_{0}=0$ signifies that it is real. Likewise, $y_{i}=1$, for
$i>1$, indicates the $i$-th face attribute/region is fake, and $y_{i}=0$
otherwise.

##  IV Semantics-Oriented Face Forgery Detection

In this section, we describe a new SO-detection method, drawing inspiration
from the proposed definition of face forgery.

###  IV-A Probabilistic Formulation

Conditioning on an input face image $\bm{x}\in\mathbb{R}^{H\times W\times 3}$,
we specify an unnormalized probability distribution over the label hierarchy
[76]:

$\displaystyle\begin{split}\tilde{p}(\bm{y}|\bm{x})=&\prod_{i}e^{f_{i}(\bm{x})\mathbb{I}[y_{i}=1]}\prod_{j,i\in\mathcal{P}_{j}}\mathbb{I}\left[\left(\sum_{i}y_{i},y_{j}\right)\neq(0,1)\right]\\\
&\prod_{i,j\in\mathcal{C}_{i}}\mathbb{I}\left[\left(y_{i},\sum_{j}y_{j}\right)\neq(1,0)\right],\end{split}$
(1)

where $f_{i}(\bm{x})$ can be regarded as the raw score (_i.e._ , confidence)
of the detector $\bm{f}$ on the $i$-th label. $\mathbb{I}[\cdot]$ is an
indication function that excludes two illegal label relations. If the $j$-th
node is manipulated (_i.e._ , $y_{j}=1$), it necessitates that at least one of
its parents, with indices in $\mathcal{P}_{j}$, is also manipulated, ensuring
that $\sum_{i}y_{i}\neq 0$. Conversely, if the $i$-th node is manipulated, at
least one of its children, with indices in $\mathcal{C}_{i}$, should also be
manipulated, leading to $\sum_{j}y_{j}\neq 0$.

We then normalize Eq. (1) to obtain the joint probability

$\displaystyle
p(\bm{y}|\bm{x})=\frac{\tilde{p}(\bm{y}|\bm{x})}{Z(\bm{x})},\,\mathrm{where}\,Z(\bm{x})=\sum_{\bm{y}^{\prime}}\tilde{p}(\bm{y}^{\prime}|\bm{x}).$
(2)

Given that our label hierarchy is small, $Z(\bm{x})$ can be computed
exhaustively via matrix multiplication.

###  IV-B Bi-level Optimization

Assume a training minibatch
$\mathcal{B}_{\mathrm{tr}}=\\{\bm{x}^{(k)},\bm{y}^{(k)},\mathcal{I}^{(k)}\\}_{k=1}^{K}$,
where $\bm{y}^{(k)}=\\{0,1\\}^{N}$ is the complete ground truth vector and
$\mathcal{I}^{(k)}\subseteq\\{0,1,\ldots,N\\}$ is the index set of observed
labels. Our goal is to optimize a differentiable function
$\bm{f}(\bm{x};\bm{\theta})$, parameterized by a vector $\bm{\theta}$, that
outputs the raw scores of all $N$ labels, based on which the joint probability
$p(\bm{y}|\bm{x};\bm{\theta})$ in Eq. (2) can be computed. A straightforward
optimization objective is to minimize the negative log joint likelihood of the
observed labels:

$\displaystyle\ell_{\mathrm{JL}}(\mathcal{B}_{\mathrm{tr}};\bm{\theta})=$
$\displaystyle-\frac{1}{K}\sum_{k=1}^{K}\log
p(\bm{y}^{(k)}_{\mathcal{I}^{(k)}}|\bm{x}^{(k)};\bm{\theta})$ $\displaystyle=$
$\displaystyle-\frac{1}{K}\sum_{k=1}^{K}\log\sum_{\bm{y}:\bm{y}_{\mathcal{I}^{(k)}}=\bm{y}^{(k)}_{\mathcal{I}^{(k)}}}p(\bm{y}|\bm{x}^{(k)};\bm{\theta}),$
(3)

where we marginalize the unobserved labels (_i.e._ , the second summation in
the second equation) in computing the joint likelihood [76]. Despite
mathematical elegance, Eq. (IV-B) does not facilitate prioritization of the
primary task, namely detecting whether a face image is real or fake, resulting
in suboptimal performance (as will be clear in Sec. V-C). Thus, we opt for the
multi-task learning setting, and minimize an alternative loss function that
represents a linear weighted sum of the negative log marginal likelihoods of
the observed labels:

$\displaystyle\ell_{\mathrm{ML}}(\mathcal{B}_{\mathrm{tr}};\bm{\theta})=$
$\displaystyle-\frac{1}{K}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}^{(k)}}\lambda_{i}\log
p(y^{(k)}_{i}|\bm{x}^{(k)};\bm{\theta})$ $\displaystyle=$
$\displaystyle-\frac{1}{K}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}^{(k)}}\lambda_{i}\log\sum_{\bm{y}:y_{i}=y^{(k)}_{i}}p(\bm{y}|\bm{x}^{(k)};\bm{\theta}),$
(4)

where
$\bm{\lambda}=[\lambda_{0},\lambda_{1},\ldots,\lambda_{N-1}]^{\intercal}$ is
the loss weight vector, trading off the $N$ tasks. To prioritize the primary
task and to automate the loss weight adjustment, we resort to bi-level
optimization [39], in particular the Auto-$\lambda$ algorithm [77]. At the
upper level, we only minimize the loss of the primary task with respect to
$\bm{\lambda}$ on the validation minibatch $\mathcal{B}_{\mathrm{val}}$. At
the lower level, we minimize the overall loss defined in Eq. (IV-B) with
respect to the model parameters $\bm{\theta}$ on the training minibatch
$\mathcal{B}_{\mathrm{tr}}$. This leads to the following bi-level optimization
problem:

$\displaystyle\min_{\bm{\lambda}}-$
$\displaystyle\frac{1}{|\mathcal{B}_{\mathrm{val}}|}\sum_{\bm{x}\in\mathcal{B}_{\mathrm{val}}}\log
p(y_{0}|\bm{x};\bm{\theta}^{\star})$
$\displaystyle\text{s.t.}\quad\bm{\theta}^{\star}$
$\displaystyle=\operatorname*{arg\,min}_{\bm{\theta}}\ell_{\mathrm{ML}}(\mathcal{B}_{\mathrm{tr}};\bm{\theta}),$
(5)

where $|\cdot|$ denotes the cardinality of a set. During optimization, we
sample training and validation data to be different minibatches in the same
training dataset. Solving Problem (IV-B) necessitates second-order
derivatives, which can be heavy on computing and memory usage. As suggested in
[77], we employ the finite difference method as the efficient approximator.

TABLE II: Results of thirteen face forgery detectors under the testing
Protocol-1 and Protocol-2 on the combined main and additional test sets of
FFSC. All methods are trained on the training set of FF++ (or its respective
augmented versions). “✗” indicates that no such evaluations can be properly
done. The top two results are highlighted in bold

Method | Intra-dataset | Protocol-1 | Protocol-2
---|---|---|---
Expression | Identity | Age | Gender | Pose
Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC
Rossler19 [28] | 98.61 | 99.73 | 52.02 | 64.01 | 68.13 | 72.65 | 67.95 | 78.54 | 73.21 | 73.99 | 61.65 | 74.50
CNND [78] | 98.13 | 99.56 | 52.36 | 60.12 | 68.52 | 74.41 | 64.77 | 75.46 | 68.75 | 80.65 | 58.04 | 72.03
F3-Net [58] | 97.87 | 99.70 | 57.78 | 71.94 | 66.99 | 75.12 | 60.65 | 76.31 | 69.66 | 82.48 | 71.99 | 85.66
FFD [79] | 97.54 | 98.58 | 52.16 | 66.92 | 60.00 | 67.83 | 62.39 | 74.10 | 72.90 | 76.55 | 70.71 | 90.49
Patch-Forensics [55] | 98.14 | 99.95 | 57.07 | 77.31 | 60.80 | 81.36 | 54.23 | 68.03 | 60.85 | 67.81 | 53.07 | 70.96
Face X-ray [20] | 93.48 | 98.37 | 62.90 | 92.86 | 70.38 | 93.11 | 81.99 | 97.96 | 65.17 | 81.52 | 66.99 | 82.03
MADD [21] | 97.68 | 99.51 | 74.94 | 82.22 | 84.20 | 91.63 | 67.05 | 75.04 | 83.13 | 91.48 | 71.59 | 79.72
FRDM [60] | 98.32 | 99.41 | 50.68 | 60.15 | 72.16 | 87.71 | 60.77 | 68.31 | 64.74 | 71.04 | 61.53 | 71.45
Lip-Forensics [71] | 97.81 | 98.83 | ✗ | ✗ | 60.00 | 76.76 | 52.16 | 60.75 | 55.96 | 74.26 | ✗ | ✗
RECCE [69] | 97.06 | 99.32 | 57.78 | 75.33 | 61.08 | 80.77 | 55.11 | 67.46 | 60.26 | 81.76 | 55.74 | 73.61
SBI [56] | 97.33 | 99.64 | 73.52 | 79.81 | 85.52 | 96.37 | 85.94 | 92.64 | 70.82 | 75.80 | 61.56 | 68.82
ICT [22] | 88.20 | 94.39 | ✗ | ✗ | 76.89 | 84.20 | 66.23 | 72.18 | 51.65 | 52.83 | 70.08 | 76.12
CADDM [23] | 98.77 | 99.70 | 65.85 | 73.61 | 78.24 | 87.14 | 78.98 | 85.31 | 78.86 | 87.14 | 78.66 | 88.17

##  V Experiments

In this section, we first employ the proposed FFSC dataset as the test set for
evaluating current face forgery detectors predicated on various “intended”
features. We next compare FFSC with current face forgery datasets [28, 40, 24]
in fostering generalization when utilized as training sets. Last, we highlight
the superiority of our SO-detection method over the traditional binary and
multi-class classification-based methods.

###  V-A FFSC as the Test Set

We introduce two new testing protocols with the goal of facilitating a fine-
grained assessment of face forgery detectors, yielding valuable insights into
their relative strengths and weaknesses.

  * •

Protocol-1: Generalization to novel manipulation methods for the same face
attribute.

  * •

Protocol-2: Generalization to novel face attributes.

TABLE III: AUC results of various base models retrained on different datasets.
Intra-dataset results are omitted, as denoted by “–”

Base Model | Training | FF++ | DFDC | DF-1.0 | Celeb-DF
---|---|---|---|---|---
Rossler19 [28] | FF++ | – | 62.19 | 85.41 | 73.70
DFDC | 71.34 | – | 79.60 | 71.19
ForgeryNet | 85.06 | 71.08 | 90.09 | 73.06
FFSC (Ours) | 92.99 | 77.29 | 89.94 | 82.23
CNND [78] | FF++ | – | 72.10 | 74.40 | 75.60
DFDC | 73.34 | – | 69.32 | 64.79
ForgeryNet | 66.81 | 75.55 | 76.49 | 69.82
FFSC (Ours) | 88.01 | 72.37 | 92.26 | 83.30
MADD [21] | FF++ | – | 67.94 | 66.58 | 77.44
DFDC | 74.22 | – | 74.32 | 65.53
ForgeryNet | 82.74 | 74.53 | 82.10 | 73.53
FFSC (Ours) | 92.70 | 78.90 | 88.67 | 88.61
FRDM [60] | FF++ | – | 79.70 | 73.80 | 79.40
DFDC | 67.33 | – | 74.11 | 63.80
ForgeryNet | 71.88 | 74.97 | 78.07 | 74.38
FFSC (Ours) | 93.54 | 76.64 | 86.48 | 83.88
RECCE [69] | FF++ | – | 69.06 | 63.05 | 68.71
DFDC | 61.63 | – | 69.86 | 63.42
ForgeryNet | 68.47 | 75.68 | 72.82 | 74.98
FFSC (Ours) | 97.11 | 71.78 | 81.36 | 80.12
CADDM [23] | FF++ | – | 59.22 | 64.27 | 68.28
DFDC | 61.69 | – | 73.48 | 63.63
ForgeryNet | 72.98 | 75.16 | 73.75 | 67.44
FFSC (Ours) | 87.10 | 71.81 | 88.08 | 83.63

We examine thirteen face forgery detectors, including Rossler19 [28], CNND
[78], F3-Net [58], FFD [79], Patch-Forensics [55], Face X-ray [20], MADD [21],
FRDM [60], Lip-Forensics [71], RECCE [69], SBI [56], ICT [22], and CADDM [23].
Rossler19, CNND, and Patch-Forensics are commonly compared baseline models.
F3-Net and FRDM expose face forgeries by high-frequency analysis. Face X-ray
and SBI learn to detect blending boundaries. FFD and MADD employ attention
mechanisms to extract and focus on the most relevant features. RECCE and
CADDM, respectively, adopt face reconstruction and manipulation localization
as auxiliary tasks. Notably, Lip-Forensics and ICT rely primarily on high-
level lipreading and face identity features, respectively, rather than signal-
level analysis.

We adopt the prediction accuracy (_i.e._ , Acc (%)) and the area under the
receiver operating characteristic curve (_i.e._ , AUC (%)) as the evaluation
metrics. The results are shown in Table II. Despite the superior intra-dataset
performance achieved by nearly all detectors, they struggle to generalize to
novel manipulations that alter the same face attribute. This provides a strong
indication that existing detectors rely heavily on manipulation-specific
features that are less generalizable. Two exceptions are Face X-ray [20] and
SBI [56] that deliver satisfactory results under Protocol-1, largely due to
their ability to detect blending boundaries commonly found in expression and
identity manipulations. However, their effectiveness diminishes under
Protocol-2, especially for the gender and pose attributes, where blending
techniques are not involved during alteration. Engineered to identify local
visual artifacts, CADDM [23] performs remarkably under Protocol-2, suggesting
that current techniques for manipulating age, gender, and pose still have room
for improvement in terms of visual fidelity. Correspondingly, CADDM achieves
subpar performance for the expression attribute, which focuses on local
semantic changes with fewer noticeable artifacts across the entire face.

The performance of the semantics-based detectors Lip-Forensics [71] and ICT
[22] is not exceptional under either Protocol-1 or Protocol-2. This could be
attributed to the fact that these detectors only exploit a single semantic
face attribute, and do not capture the intrinsic interactions of multiple face
semantics and their relationship with local face regions.

In summary, the assessment results on the proposed FFSC dataset underscore the
ongoing challenge in constructing generalizable face forgery detectors across
different manipulation methods and face attributes. Presently, detectors are
predominantly dependent on cues specific to training manipulations,
particularly those accompanied by visual distortions. Furthermore, our new
testing protocols have revealed certain deficiencies in current semantics-
based detectors, inspiring us to re-examine the role of face semantics in
developing face forgery detectors.

###  V-B FFSC as the Training Set

TABLE IV: Protocol-1 test results. All detectors are trained on the training
set of FFSC and tested on the additional test set of FFSC

Base Model | Protocol-1
---|---
Age | Expression | Gender | Identity | Pose
Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC
Rossler19 | 86.60 | 98.34 | 88.04 | 99.13 | 88.24 | 98.70 | 83.48 | 95.49 | 80.31 | 89.14
SO-Rossler19 | 95.73 | 98.87 | 85.01 | 98.81 | 95.63 | 98.90 | 94.78 | 97.77 | 84.24 | 92.11
CNND | 81.29 | 97.17 | 81.25 | 98.59 | 80.86 | 98.22 | 77.65 | 92.15 | 70.91 | 80.58
SO-CNND | 91.61 | 97.24 | 91.70 | 97.62 | 93.21 | 98.90 | 80.51 | 86.87 | 79.18 | 84.42
MADD | 94.06 | 99.20 | 95.64 | 99.71 | 95.45 | 99.58 | 89.94 | 96.52 | 88.44 | 94.93
SO-MADD | 93.70 | 99.30 | 92.89 | 98.93 | 94.18 | 99.59 | 93.09 | 98.14 | 87.80 | 95.20
FRDM | 83.99 | 93.59 | 84.94 | 93.82 | 85.27 | 95.53 | 71.52 | 80.72 | 77.79 | 85.61
SO-FRDM | 86.60 | 96.41 | 86.78 | 96.99 | 87.07 | 98.07 | 80.35 | 88.22 | 78.30 | 87.95
RECCE | 94.06 | 99.34 | 97.48 | 99.76 | 97.16 | 99.75 | 91.98 | 97.94 | 86.87 | 93.99
SO-RECCE | 95.94 | 99.21 | 98.08 | 99.20 | 99.27 | 99.92 | 97.45 | 99.09 | 90.12 | 96.87
CADDM | 80.44 | 96.67 | 79.72 | 97.67 | 81.53 | 97.72 | 74.56 | 89.11 | 74.35 | 85.77
SO-CADDM | 86.65 | 96.09 | 89.25 | 97.46 | 86.98 | 98.06 | 85.43 | 90.99 | 79.43 | 88.20

TABLE V: Protocol-2 test results. All detectors are trained on four out of
five face attributes in the training set of FFSC and tested on the remaining
face attribute in the main test set of FFSC

Base Model | Protocol-2
---|---
Age | Expression | Gender | Identity | Pose
Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC | Acc | AUC
Rossler19 | 77.90 | 88.77 | 74.43 | 82.80 | 79.23 | 90.42 | 56.87 | 77.33 | 77.36 | 90.30
SO-Rossler19 | 83.92 | 92.66 | 78.66 | 88.74 | 84.29 | 93.30 | 64.66 | 81.26 | 83.30 | 92.09
CNND | 70.37 | 78.04 | 68.15 | 77.78 | 70.48 | 81.41 | 57.95 | 63.12 | 70.39 | 79.15
SO-CNND | 73.86 | 82.31 | 76.48 | 85.44 | 78.23 | 89.39 | 60.34 | 70.45 | 73.21 | 82.36
MADD | 66.36 | 80.53 | 63.61 | 74.78 | 77.95 | 89.99 | 53.30 | 75.55 | 73.66 | 88.48
SO-MADD | 77.65 | 87.84 | 78.67 | 88.69 | 79.48 | 86.89 | 57.68 | 78.60 | 76.78 | 88.08
FRDM | 75.11 | 81.59 | 74.63 | 84.28 | 74.20 | 84.38 | 55.57 | 76.57 | 74.20 | 86.21
SO-FRDM | 83.92 | 92.37 | 77.59 | 85.71 | 93.04 | 89.73 | 66.31 | 82.83 | 78.52 | 88.09
RECCE | 79.66 | 89.33 | 77.87 | 87.94 | 79.55 | 91.65 | 53.92 | 70.63 | 67.24 | 83.85
SO-RECCE | 81.52 | 94.77 | 85.12 | 91.83 | 85.85 | 95.10 | 70.92 | 78.41 | 83.58 | 92.32
CADDM | 74.83 | 84.50 | 73.89 | 80.84 | 72.98 | 83.22 | 56.93 | 74.59 | 70.57 | 80.01
SO-CADDM | 78.49 | 87.96 | 74.60 | 81.71 | 78.18 | 86.41 | 60.57 | 78.25 | 75.88 | 84.09

####  V-B1 Training Dataset Comparison

We compare the proposed FFSC dataset with three established face forgery
datasets: FF++ [28], DFDC [40], and ForgeryNet [24] for retraining various
face forgery detectors as base models. It is important to note that each
dataset may admit a different training strategy. Specifically, FF++ and DFDC
treat face forgery detection as a standard binary classification task, while
ForgeryNet approaches it as a multi-class classification task. The FFSC
dataset is used to train SO-detectors. To assess the cross-dataset
generalization, we include two more datasets: DF-1.0 [48] and Celeb-DF [46] as
suggested in [71, 60, 63, 23, 70].

Six representative face forgery detectors are selected as base models:
Rossler19 [28], CNND [78], MADD [21], FRDM [60], RECCE [69], and CADDM [23].
It is important to highlight that some detectors were initially trained with
auxiliary tasks, such as image reconstruction [69] and manipulation
localization [23]. To adhere to the original implementation of each detector
when retraining on FFSC, we incorporate the auxiliary loss (if any) into the
objective of the bi-level optimization problem, as denoted by Eq. (IV-B).
Other training procedures and hyperparameter settings strictly follow the
original publication. One exception is CADDM, where we omit the multi-scale
face swap module because it depends on a reference image that is not
accessible during retraining.

The AUC results are shown in Table III, from which we find that SO-detectors
retrained on FFSC significantly surpass those retrained on the other three
datasets. We believe such superior cross-dataset generalization arises because
of the label hierarchy of FFSC, which facilitates SO-detection of face
forgeries. This approach encourages the detectors to disregard features
specific to manipulation methods and instead focus on more transferable
features related to face attributes. Nevertheless, the detectors retrained
using FFSC marginally fall short of those using ForgeryNet [24] on the DFDC
dataset. This discrepancy may stem from the domain shift attributable to
divergent data augmentation techniques and varying photography conditions
between FFSC and DFDC. ForgeryNet addresses this gap by employing a broader
spectrum of data augmentations, including several that coincide with those
used in DFDC. Nonetheless, some augmentations (_e.g._ , highly random
brightness adjustment, excessive blurring, and extreme compression) tend to
damage major face semantics and are thus not label-preserving.

TABLE VI: AUC results of different face forgery detection methods, in which base models are trained on the training set of FFSC and tested on the other four datasets. The prefixes “B-” and “M-” denote the binary and multi-class classification, respectively Base Model | FF++ | DFDC | DF-1.0 | Celeb-DF
---|---|---|---|---
B-Rossler19 | 87.06 | 74.78 | 90.39 | 69.12
M-Rossler19 | 85.10 | 72.18 | 86.56 | 75.91
SO-Rossler19 | 92.99 | 77.29 | 89.94 | 82.23
B-CNND | 80.57 | 72.99 | 85.98 | 83.93
M-CNND | 78.14 | 69.84 | 79.31 | 81.32
SO-CNND | 88.01 | 72.37 | 92.26 | 83.30
B-MADD | 87.63 | 77.02 | 83.96 | 87.46
M-MADD | 86.35 | 76.79 | 83.29 | 84.72
SO-MADD | 92.70 | 78.90 | 88.67 | 88.61
B-FRDM | 89.07 | 73.58 | 86.46 | 80.39
M-FRDM | 87.65 | 74.18 | 83.96 | 80.87
SO-FRDM | 93.54 | 76.64 | 86.48 | 83.88
B-RECCE | 95.56 | 67.20 | 69.73 | 78.51
M-RECCE | 94.55 | 69.23 | 54.81 | 75.94
SO-RECCE | 97.11 | 71.78 | 81.36 | 80.12
B-CADDM | 83.29 | 65.23 | 87.04 | 81.10
M-CADDM | 85.96 | 69.65 | 84.17 | 79.61
SO-CADDM | 87.10 | 71.81 | 88.08 | 83.63

####  V-B2 Training Method Comparison

To single out the role of our SO-detection method, we compare it against the
corresponding base model with the original training strategy by fixing the
training set to FFSC.

Table IV shows the Protocol-1 results on the additional test set of FFSC (see
the descriptions in Sec. III-B), which examines the generalization to novel
manipulations that alter the same attributes in the training set. The primary
observation is that our SO-detection method significantly improves the
efficacy of all base models for almost every face attribute. RECCE [69]
performs satisfactorily on its own, without leveraging the label hierarchy in
FFSC. This can likely be credited to the integration of a face reconstruction
auxiliary task within RECCE, which promotes extracting semantic face features,
thus enhancing generalizability across the same face attributes. Frequency-
based FRDM [60] and distortion-based CADDM [23] both experience noticeable
boosts in Acc and AUC, suggesting that the features derived from the SO-
detection are either superior to or at least provide a beneficial complement
to the signal-level features used in these methods.

Table V presents the Protocol-2 results, where we train all base models on
four out of five face attributes in the training set of FFSC and test them on
the remaining face attribute in the test set of FFSC. Similarly, we see
consistent improvements in performance under the more challenging Protocol-2.
Even though the test face attribute is unobserved, our SO-detection method
successfully captures the relationship between face attributes by modeling the
joint probability distribution in Eq. (1), therefore enabling the SO-detectors
to generalize across different face attributes.

Moreover, we demonstrate the advantages of our SO-detection method by
comparing it to the standard binary and multi-class classification
counterparts in Table VI, where we train all base models on the training set
of FFSC and test them on FF++ [28], DFDC [40], DF-1.0 [48], and Celeb-DF [46].
It is evident that our proposed SO-detection method is consistently better
across nearly all base models and test datasets. This indicates that
exploiting label hierarchy can steer SO-detectors towards learning more
generalizable and semantically rich features, enhancing their generalization
capabilities. Interestingly, multi-class classification tends to hinder the
generalizability of the detectors, compared to the binary classification
baseline. This suggests that multi-class classification appears to promote
dependence on specific manipulation cues, which act as “shortcuts” for
detecting face forgeries and lead to potential overfitting.

###  V-C Ablation Studies

TABLE VII: AUC results of different labeling formulations. “Global” denotes
the five global face attributes, while “Local” stands for the six local face
regions. “Independent” indicates the equally weighted independent logistic
regressions

Global | Local | Formulation | FF++ | DFDC | DF-1.0 | Celeb-DF | Avg
---|---|---|---|---|---|---|---
✓ |  | Independent | 89.90 | 75.78 | 91.97 | 75.74 | 83.35
✓ |  | Hierarchical | 91.66 | 77.02 | 87.52 | 80.86 | 84.27
✓ | ✓ | Independent | 89.31 | 71.00 | 91.29 | 71.50 | 80.77
✓ | ✓ | Hierarchical | 92.99 | 77.29 | 89.94 | 82.23 | 85.61

Figure 5: Weight dynamics during bi-level optimization.

(a) Age Manipulation (b) Expression Manipulation (Smile) (c) Expression
Manipulation (Surprise)

(d) Gender Manipulation (e) Identity Manipulation (f) Pose Manipulation

Figure 6: Illustration of the SO-detector (Rossler19 [28] as the base model)
in making predictions at the face attribute and region levels.

We carry out a series of ablation experiments to validate the effectiveness of
our SO-detection method using the base model Rossler19 [28] trained on FFSC.
We first compare it with the independent logistic regression formulation,
where we discard all label relations and weight each task equally. Table VII
shows the results, where we find that the proposed label hierarchy
significantly enhances detection capabilities. Remarkably, even when we remove
the leaf nodes for local face regions, our method still demonstrates
substantial efficacy by leveraging the interplay among the five global face
attributes.

We next contrast different optimization strategies for leveraging the label
hierarchy: 1) minimizing the negative log joint likelihood in Eq. (IV-B), 2)
training with the fixed equal weights in Eq. (IV-B), 3) training with the
fixed and learned optimal weights (produced by the bi-level optimization), and
4) training with dynamic weighting average [80] without prioritizing the
primary task. As presented in Table VIII, direct optimization of the joint
likelihood of all observed labels results in a performance comparable to fixed
equal weighting, with neither method giving prominence to the primary task.
The application of dynamic weighting average appears ineffective in learning
to emphasize the primary task, yielding only a marginal improvement over fixed
equal weighting. Furthermore, our findings indicate that employing fixed
optimal weights determined through bi-level optimization falls short of
matching the performance of the default bi-level optimization procedure. This
underscores the significance of dynamic weighting adjustment in promoting the
primary task during optimization (refer to Fig. 5).

TABLE VIII: AUC results of different optimization strategies

Optimization strategy | FF++ | DFDC | DF-1.0 | Celeb-DF | Avg
---|---|---|---|---|---
Joint likelihood | 88.86 | 75.28 | 87.48 | 81.67 | 83.32
Fixed equal weights | 86.35 | 76.40 | 88.32 | 81.27 | 83.09
Fixed learned weights | 91.86 | 76.94 | 89.73 | 81.01 | 84.88
Dynamic weighting average | 90.20 | 75.70 | 90.31 | 79.11 | 83.83
Bi-level optimization (Ours) | 92.99 | 77.29 | 89.94 | 82.23 | 85.61

Benefiting from the proposed SO-detection method, we can compute the marginal
probability of each face attribute/region being manipulated, which endows the
SO-detectors with some degree of interpretability. For example, in Fig. 6 (b),
the SO-detector discerns the face as fake, and meanwhile indicates a likely
alteration of expression, pinpointing the mouth and lips as manipulated
regions.

##  VI Conclusion and Discussion

In this work, we have given face forgery a new definition from the perspective
of face semantics. Working with this definition, we constructed a new face
forgery image dataset, FFSC, and proposed an SO-detection method. Extensive
experiments have validated the value of the proposed FFSC dataset as training
and test sets, and the superiority of our method in improving the
generalizability of face forgery detectors.

It is important to note that the current definition of face forgery is
imperfect. An excellent example is that reducing the frame rate in a speech
video of House Speaker Nancy Pelosi makes her sound sluggish and slurred.
Despite the obvious falseness of the video, it would still be classified as
real under our definition as it does not involve any modifications of semantic
face attributes. To address this issue, a possible solution could be adding a
psychological_response node at the global face attribute level, connected to a
behavior node at the local face region level. Additionally, it is natural to
expand the label hierarchy in Fig. 2 by adding other attribute nodes (_e.g._ ,
race and attractiveness).

Moreover, our current definition of face forgery is limited to face
manipulation methods. It would be beneficial to expand this definition to
include face creation methods, such as those utilizing GANs [3], diffusion
models [2, 81] and computer graphics [82]. However, this expansion presents
significant challenges. First, technology has advanced to the point where we
can render an existing face presented in a real physical scene with such
realism that it is indistinguishable from one captured by a digital camera
(see Fig. 7). Second, there is evidence that generative models can memorize
training data [83], and techniques exist to retrieve these memorized images
(see Fig. 8). In both cases, determining the authenticity of a face image
requires heightened awareness or insight to perceive its true nature. Taking a
step back, a more pragmatic extension of our definition is to include digital
manipulation methods for face images to images of natural scenes that do not
necessarily contain faces.

(a)  (b)

Figure 7: Face rendering results by the method in [84]. Within each subfigure,
the real and rendered faces are displayed on the left and right, respectively.
Figure 8:  Memorization results of a diffusion model trained on a small-sized
CelebA dataset [85], as presented in [83]. The top row depicts training face
images downsampled to $80\times 80$, and the bottom row displays face images
generated by the diffusion model, which closely resemble those from the
training set.

## Acknowledgement

The authors would like to thank Mr. Weiran Zhao for his kind help in data
collection.

## References

  * [1] “Deepfakes,” https://github.com/deepfakes/faceswap, accessed: May 9, 2024.
  * [2] Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” in _Conference on Neural Information Processing Systems_ , 2019, pp. 11918–11930.
  * [3] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of StyleGAN,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8110–8119.
  * [4] H. Farid, _Photo Forensics_. MIT Press, 2016.
  * [5] A. C. Popescu and H. Farid, “Exposing digital forgeries by detecting traces of resampling,” _IEEE Transactions on Signal Processing_ , vol. 53, no. 2, pp. 758–767, 2005.
  * [6] A. C. Popescu and H. Farid, “Statistical tools for digital forensics,” in _International Workshop on Information Hiding_ , 2004, pp. 128–147.
  * [7] A. C. Popescu and H. Farid, “Exposing digital forgeries in color filter array interpolated images,” _IEEE Transactions on Signal Processing_ , vol. 53, no. 10, pp. 3948–3959, 2005.
  * [8] M. K. Johnson and H. Farid, “Exposing digital forgeries through chromatic aberration,” in _ACM Workshop on Multimedia and Security_ , 2006, pp. 48–55.
  * [9] S. Lyu, “Estimating vignetting function from a single image for image authentication,” in _ACM Workshop on Multimedia and Security_ , 2010, pp. 3–12.
  * [10] Z. Lin, R. Wang, X. Tang, and H.-Y. Shum, “Detecting doctored images using camera response normality and consistency,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2005, pp. 1087–1092.
  * [11] S. Lyu, X. Pan, and X. Zhang, “Exposing region splicing forgeries with blind local noise estimation,” _International Journal of Computer Vision_ , vol. 110, no. 2, pp. 202–221, 2014.
  * [12] M. K. Johnson and H. Farid, “Exposing digital forgeries in complex lighting environments,” _IEEE Transactions on Information Forensics and Security_ , vol. 2, no. 3, pp. 450–461, 2007.
  * [13] E. Kee, J. F. O’brien, and H. Farid, “Exposing photo manipulation from shading and shadows.” _ACM Transactions on Graphics_ , vol. 33, no. 5, pp. 165:1–165:21, 2014.
  * [14] J. F. O’brien and H. Farid, “Exposing photo manipulation with inconsistent reflections.” _ACM Transactions on Graphics_ , vol. 31, no. 1, pp. 4:1–4:11, 2012.
  * [15] V. Conotter, J. F. O’Brien, and H. Farid, “Exposing digital forgeries in ballistic motion,” _IEEE Transactions on Information Forensics and Security_ , vol. 7, no. 1, pp. 283–296, 2011.
  * [16] Y. Li, M.-C. Chang, and S. Lyu, “In Ictu Oculi: Exposing AI created fake videos by detecting eye blinking,” in _IEEE International Workshop on Information Forensics and Security_ , 2018, pp. 1–7.
  * [17] D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “MesoNet: A compact facial video forgery detection network,” in _IEEE International Workshop on Information Forensics and Security_ , 2018, pp. 1–7.
  * [18] P. Zhou, X. Han, V. I. Morariu, and L. S. Davis, “Two-stream neural networks for tampered face detection,” in _IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2017, pp. 1831–1839.
  * [19] H. H. Nguyen, F. Fang, J. Yamagishi, and I. Echizen, “Multi-task learning for detecting and segmenting manipulated facial images and videos,” in _IEEE International Conference on Biometrics: Theory, Applications and Systems_ , 2019, pp. 1–8.
  * [20] L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen, and B. Guo, “Face X-ray for more general face forgery detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 5001–5010.
  * [21] H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang, and N. Yu, “Multi-attentional deepfake detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 2185–2194.
  * [22] X. Dong, J. Bao, D. Chen, T. Zhang, W. Zhang, N. Yu, D. Chen, F. Wen, and B. Guo, “Protecting celebrities from DeepFake with identity consistency transformer,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 9468–9478.
  * [23] S. Dong, J. Wang, R. Ji, J. Liang, H. Fan, and Z. Ge, “Implicit identity leakage: The stumbling block to improving deepfake detection generalization,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 3994–4004.
  * [24] Y. He, B. Gan, S. Chen, Y. Zhou, G. Yin, L. Song, L. Sheng, J. Shao, and Z. Liu, “ForgeryNet: A versatile benchmark for comprehensive forgery analysis,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 4360–4369.
  * [25] Z. Sun, S. Chen, T. Yao, B. Yin, R. Yi, S. Ding, and L. Ma, “Contrastive pseudo learning for open-world DeepFake attribution,” in _International Conference on Computer Vision_ , 2023, pp. 20882–20892.
  * [26] J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2Face: Real-time face capture and reenactment of RGB videos,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 2387–2395.
  * [27] K. Preechakul, N. Chatthee, S. Wizadwongsa, and S. Suwajanakorn, “Diffusion autoencoders: Toward a meaningful and decodable representation,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10619–10629.
  * [28] A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “FaceForensics++: Learning to detect manipulated facial images,” in _International Conference on Computer Vision_ , 2019, pp. 1–11.
  * [29] H. Pehlivan, Y. Dalva, and A. Dundar, “StyleRes: Transforming the residuals for real image editing with StyleGAN,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 1828–1837.
  * [30] A. Siarohin, S. Lathuilière, S. Tulyakov, E. Ricci, and N. Sebe, “First order motion model for image animation,” in _Conference on Neural Information Processing Systems_ , 2019, pp. 1–11.
  * [31] Y. Viazovetskyi, V. Ivashkin, and E. Kashin, “StyleGAN2 distillation for feed-forward image manipulation,” in _European Conference on Computer Vision_ , 2020, pp. 170–186.
  * [32] R. Chen, X. Chen, B. Ni, and Y. Ge, “SimSwap: An efficient framework for high fidelity face swapping,” in _ACM International Conference on Multimedia_ , 2020, pp. 2003–2011.
  * [33] Y. Nirkin, Y. Keller, and T. Hassner, “FSGAN: Subject agnostic face swapping and reenactment,” in _International Conference on Computer Vision_ , 2019, pp. 7184–7193.
  * [34] J. Zhao and H. Zhang, “Thin-plate spline motion model for image animation,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 3657–3666.
  * [35] O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski, “StyleCLIP: Text-driven manipulation of stylegan imagery,” in _International Conference on Computer Vision_ , 2021, pp. 2085–2094.
  * [36] G. Gao, H. Huang, C. Fu, Z. Li, and R. He, “Information bottleneck disentanglement for identity swapping,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 3404–3413.
  * [37] B. Zeng, B. Liu, H. Li, X. Liu, J. Liu, D. Chen, W. Peng, and B. Zhang, “FNeVR: Neural volume rendering for face animation,” in _Conference on Neural Information Processing Systems_ , 2022, pp. 22451–22462.
  * [38] T. Wang, Y. Zhang, Y. Fan, J. Wang, and Q. Chen, “High-fidelity GAN inversion for image attribute editing,” _arXiv preprint arXiv:2109.06590_ , 2021.
  * [39] S. Dempe, _Foundations of Bilevel Programming_. Springer Science & Business Media, 2002.
  * [40] B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, and C. C. Ferrer, “The DeepFake detection challenge (DFDC) dataset,” _arXiv preprint arXiv:2006.07397_ , 2020.
  * [41] S.-Y. Wang, O. Wang, A. Owens, R. Zhang, and A. A. Efros, “Detecting Photoshopped faces by scripting Photoshop,” in _International Conference on Computer Vision_ , 2019, pp. 10072–10081.
  * [42] E. Richardson and Y. Weiss, “On GANs and GMMs,” in _Conference on Neural Information Processing Systems_ , 2018, pp. 5852–5863.
  * [43] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in _Conference on Neural Information Processing Systems_ , 2014, pp. 2672–2680.
  * [44] D. Vlasic, M. Brand, H. Pfister, and J. Popović, “Face transfer with multilinear models,” _ACM Transactions on Graphics_ , vol. 24, no. 3, pp. 426–433, 2005.
  * [45] X. Yang, Y. Li, and S. Lyu, “Exposing Deep Fakes using inconsistent head poses,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2019, pp. 8261–8265.
  * [46] Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-DF: A large-scale challenging dataset for DeepFake forensics,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 3207–3216.
  * [47] N. Dufour and A. Gully, “Contributing data to deepfake detection research,” https://research.google/blog/contributing-data-to-deepfake-detection-research/, 2019, accessed: May 9, 2024.
  * [48] L. Jiang, R. Li, W. Wu, C. Qian, and C. C. Loy, “DeeperForensics-1.0: A large-scale dataset for real-world face forgery detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 2889–2898.
  * [49] T. Zhou, W. Wang, Z. Liang, and J. Shen, “Face forensics in the wild,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 5778–5788.
  * [50] H. Guo, S. Hu, X. Wang, M.-C. Chang, and S. Lyu, “Eyes tell all: Irregular pupil shapes reveal GAN-generated faces,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2022, pp. 2904–2908.
  * [51] S. Hu, Y. Li, and S. Lyu, “Exposing GAN-generated faces using inconsistent corneal specular highlights,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2021, pp. 2500–2504.
  * [52] S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, and H. Li, “Protecting world leaders against deep fakes.” in _IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 38–45.
  * [53] Y. Li and S. Lyu, “Exposing DeepFake videos by detecting face warping artifacts,” in _IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 46–52.
  * [54] H. H. Nguyen, J. Yamagishi, and I. Echizen, “Capsule-Forensics: Using capsule networks to detect forged images and videos,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2019, pp. 2307–2311.
  * [55] L. Chai, D. Bau, S.-N. Lim, and P. Isola, “What makes fake images detectable? Understanding properties that generalize,” in _European Conference on Computer Vision_ , 2020, pp. 103–120.
  * [56] K. Shiohara and T. Yamasaki, “Detecting deepfakes with self-blended images,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 18720–18729.
  * [57] Y. Xu, J. Liang, G. Jia, Z. Yang, Y. Zhang, and R. He, “TALL: Thumbnail layout for deepfake video detection,” in _International Conference on Computer Vision_ , 2023, pp. 22658–22668.
  * [58] Y. Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao, “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in _European Conference on Computer Vision_ , 2020, pp. 86–103.
  * [59] H. Liu, X. Li, W. Zhou, Y. Chen, Y. He, H. Xue, W. Zhang, and N. Yu, “Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 772–781.
  * [60] Y. Luo, Y. Zhang, J. Yan, and W. Liu, “Generalizing face forgery detection with high-frequency features,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 16317–16326.
  * [61] W. Lu, L. Liu, B. Zhang, J. Luo, X. Zhao, Y. Zhou, and J. Huang, “Detection of deepfake videos using long-distance attention,” _IEEE Transactions on Neural Networks and Learning Systems_ , to appear, 2023.
  * [62] Q. Yin, W. Lu, B. Li, and J. Huang, “Dynamic difference learning with spatio-temporal correlation for deepfake video detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 18, pp. 4046–4058, 2023.
  * [63] L. Chen, Y. Zhang, Y. Song, L. Liu, and J. Wang, “Self-supervised learning of adversarial example: Towards good generalizations for DeepFake detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 18710–18719.
  * [64] L. Chen, Y. Zhang, Y. Song, J. Wang, and L. Liu, “OST: Improving generalization of DeepFake detection via one-shot test-time training,” in _Conference on Neural Information Processing Systems_ , 2022, pp. 1–14.
  * [65] Y. Wang, K. Yu, C. Chen, X. Hu, and S. Peng, “Dynamic graph learning with content-guided spatial-frequency relation reasoning for deepfake detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 7278–7287.
  * [66] Z. Yang, J. Liang, Y. Xu, X.-Y. Zhang, and R. He, “Masked relation learning for DeepFake detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 18, pp. 1696–1708, 2023.
  * [67] K. Sun, T. Yao, S. Chen, S. Ding, J. Li, and R. Ji, “Dual contrastive learning for general face forgery detection,” in _AAAI Conference on Artificial Intelligence_ , 2022, pp. 2316–2324.
  * [68] A. Luo, C. Kong, J. Huang, Y. Hu, X. Kang, and A. C. Kot, “Beyond the prior forgery knowledge: Mining critical clues for general face forgery detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 19, pp. 1168–1182, 2024.
  * [69] J. Cao, C. Ma, T. Yao, S. Chen, S. Ding, and X. Yang, “End-to-end reconstruction-classification learning for face forgery detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 4113–4122.
  * [70] Z. Yan, Y. Zhang, Y. Fan, and B. Wu, “UCF: Uncovering common features for generalizable deepfake detection,” in _International Conference on Computer Vision_ , 2023, pp. 22412–22423.
  * [71] A. Haliassos, K. Vougioukas, S. Petridis, and M. Pantic, “Lips don’t lie: A generalisable and robust approach to face forgery detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 5039–5049.
  * [72] A. Ephrat, I. Mosseri, O. Lang, T. Dekel, K. Wilson, A. Hassidim, W. T. Freeman, and M. Rubinstein, “Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separation,” _arXiv preprint arXiv:1804.03619_ , 2018.
  * [73] J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, “RetinaFace: Single-shot multi-level face localisation in the wild,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 5203–5212.
  * [74] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” _arXiv preprint arXiv:2010.02502_ , 2020.
  * [75] T. N. Cornsweet, “The staircase-method in psychophysics,” _The American Journal of Psychology_ , vol. 75, no. 3, pp. 485–491, 1962.
  * [76] J. Deng, N. Ding, Y. Jia, A. Frome, K. Murphy, S. Bengio, Y. Li, H. Neven, and H. Adam, “Large-scale object classification using label relation graphs,” in _European Conference on Computer Vision_ , 2014, pp. 48–64.
  * [77] S. Liu, S. James, A. J. Davison, and E. Johns, “Auto-Lambda: Disentangling dynamic task relationships,” _arXiv preprint arXiv:2202.03091_ , 2022.
  * [78] S.-Y. Wang, O. Wang, R. Zhang, A. Owens, and A. A. Efros, “CNN-generated images are surprisingly easy to spot…for now,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8695–8704.
  * [79] H. Dang, F. Liu, J. Stehouwer, X. Liu, and A. Jain, “On the detection of digital face manipulation,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 5781–5790.
  * [80] S. Liu, E. Johns, and A. J. Davison, “End-to-end multi-task learning with attention,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1871–1880.
  * [81] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10684–10695.
  * [82] P. Chandran, S. Winberg, G. Zoss, J. Riviere, M. Gross, P. Gotardo, and D. Bradley, “Rendering with style: Combining traditional and neural approaches for high-quality face rendering,” _ACM Transactions on Graphics_ , vol. 40, no. 6, pp. 1–14, 2021.
  * [83] Z. Kadkhodaie, F. Guth, E. P. Simoncelli, and S. Mallat, “Generalization in diffusion models arises from geometry-adaptive harmonic representation,” in _International Conference on Learning Representations_ , 2024, pp. 1–25.
  * [84] L. Chen, C. Cao, F. De la Torre, J. Saragih, C. Xu, and Y. Sheikh, “High-fidelity face tracking for AR/VR via deep lighting adaptation,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 13059–13069.
  * [85] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in _International Conference on Computer Vision_ , 2015, pp. 3730–3738.