# Learning to Recognize Patch-Wise Consistency for Deepfake Detection

Tianchen Zhao Xiang Xu Mingze Xu Hui Ding Yuanjun Xiong Wei Xia
Amazon/AWS AI
<EMAIL_ADDRESS><EMAIL_ADDRESS>Currently
at University of Michigan, Ann Arbor. The work was conducted during internship
at AWS.

###### Abstract

We propose to detect Deepfake generated by face manipulation based on one of
their fundamental features: images are blended by patches from multiple
sources, carrying distinct and persistent source features. In particular, we
propose a novel representation learning approach for this task, called patch-
wise consistency learning (PCL). It learns by measuring the consistency of
image source features, resulting to representation with good interpretability
and robustness to multiple forgery methods. We develop an inconsistency image
generator (I2G) to generate training data for PCL and boost its robustness. We
evaluate our approach on seven popular Deepfake detection datasets. Our model
achieves superior detection accuracy and generalizes well to unseen generation
methods. On average, our model outperforms the state-of-the-art in terms of
AUC by 2% and 8% in the in- and cross-dataset evaluation, respectively.

##  1 Introduction

Face manipulation [43] has been emerging as a prominent stream of deep face
forgery. It replaces the identity or expression of a target subject with that
of another subject, by stitching image patches from different sources
together. Extensive work has been done to detect manipulated facial images and
videos [63, 28, 40, 55, 50, 7, 25, 52, 24, 8], with the attempt to spot
various faking cues, such as eye blinking [28], face warping [29], blending
boundaries [25], fake prototypes [52], . However, most of the work focuses on
implicitly or explicitly capturing holistic representations to identify
suspicious artifacts from specific synthetic methods. As a result, the learned
models are able to detect artifacts seen during their training, but have
trouble keeping up with unseen Deepfake generation methods and more advanced
blending techniques.

Figure 1:  An overview of our patch-wise consistency learning. An “self-
inconsistent” image is generated by stitching the target and source faces,
where the mask indicates the region of modification and separates the patches
into two groups. For each patch, we compute its source features consistencies
with the rest, and their corresponding consistency scores are shown on the
right.

We tackle the above issue by utilizing the fundamental feature of Deepfake
generated by face manipulation: blending images from multiple sources. Images
from different sources carry distinctive source features, such as in-camera
features (, PRNU noise [32], specifications [17]), out-camera features (, JPEG
compression patterns [3], compression rates, frame rates), and forgery
features (, copy-paste artifacts [2], blending artifacts [25], GAN
fingerprints [62]). And these cues are still preserved after being stitched
into Deepfake images, even with state-of-the-art generation and blending
methods. Therefore, Deepfake images are poised to contain different source
features in different locations, whereas the source features of a pristine
image must be self-consistent across all of its patches despite the variation
of their semantic contents [17]. By building representation models that
separate the different source features in different image patches, we may be
able to effectively detect manipulated face images, even for those from unseen
generation techniques.

In this paper, we explore this paradigm of detecting Deepfake by source
features consistency. We propose a novel representation learning method,
called Patch-Wise Consistency Learning (PCL), and present experimental results
showing that predicting the self-consistency of source features indeed
improves the performance of Deepfake detection. In particular, classification
is built on the learned consistency features and the consistency map is
predicted to provide an intuitive visual explanation of the manipulated
regions. Fig. 1 illustrates an example of our consistency learning pipeline.
We split the input frame into small patches and cluster them into groups
according to their pairwise similarity score of source features. A non-linear
classifier is built upon the learned similarity representation for binary
Deepfake prediction.

Learning with PCL requires patch-level annotation of the manipulated regions
which is not widely available. We develop the Inconsistency Image Generator
(I2G), which synthesizes “inconsistent” samples by face manipulation using the
pristine images along with random “forgery masks” and thus also produces
ground truth for PCL supervision. Different from other data generators [17,
29], in I2G, we introduce random blending and strong data augmentation to
enhance the robustness of PCL learned representation against attempts to hide
source features in Deepfake.

The contributions of this paper are three-fold. First, we introduce Patch-Wise
Consistency Learning (PCL) to Deepfake detection, which is more robust to
unseen generation methods. Second, we propose an end-to-end learning
architecture to realize PCL in practice, which can work with various backbones
as a plugin module and produces interpretable visualization. Third, we
evaluate the performance of our model on seven public-available datasets. To
evaluate the generalization ability, we also conduct the cross-dataset
experiment, where we train and test PCL on separate datasets with different
generation methods. Experimental results show that, on average, our approach
outperforms the state-of-the-art in terms of AUC by $2\%$ and $8\%$ in the in-
and cross-dataset evaluation, respectively. We believe that our work provides
a comprehensive benchmark for future work on Deepfake detection.

##  2 Related Work

Figure 2:  Visualization of PCL architecture. The consistency branch focuses
on measuring the consistency of image patches according to their source
features, and outputs a visualization map that indicates the manipulated
region. A classification branch is applied after the learned feature map and
predicts the binary score for Deepfake detection.

Deepfake Generation. In general, there are four common types of Deepfake [51]:
whole image synthesis, modification of facial attribute or expression, and
face identity swap. The 3D models [48], AutoEncoders [47, 53], or Generative
Adversarial Networks [19, 20, 31, 6] are used to generate the image and blend
it back to the original images.

Deepfake Detection. To detect the whole image synthesis, recent research [62,
39, 16, 56] observes that generated images contain specific cues that can be
easily detected, and the trained models have exhibited good generalization
ability across synthesis methods. To support the research on detecting the
other types of face manipulations, several Deepfake datasets are released [61,
22, 45, 12, 30, 10, 18, 64] and countermeasures have been introduced.
FakeSpotter [55] is proposed with a layer-wise neuron behavior for fake face
detection. Tolosana et al. [50] split the facial image into pieces and make
binary predictions on these local regions. Kumar et al. [24] train a triplet
network with positive and negative samples. Recurrent neural networks [46] and
various types of 3D ConvNets [8] are utilized to detect the manipulation
artifacts across the video frames. However, binary classifiers are criticized
for their interpretability, and several localization methods are introduced
through either multi-task learning [40] or attention-based mechanisms [7]. To
improve the generalization ability, DSP-FWA [29] and Face X-ray [25] also make
their own data generation pipeline and the latter focuses on predicting the
blending boundaries in fake video frames. Our approach also lies in this line
but has several main differences. First, from a methodology perspective, our
approach focuses on learning similarity representations by measuring local
patch consistency for patch-wise consistency recognition rather than spotting
face wrapping or blending artifacts; Second, from a network design
perspective, our approach is more simple, which can use any common network as
the backbone. The designed inconsistency predictor only contains a few
parameters and can serve as a plugin module.

Consistency Learning. The concept of inconsistency has been studied in the
image forensic literature [37], where image patches are fed as inputs to the
model to predict the similarity scores [37, 36, 17, 35, 63, 4]. Zhou et al.
[63] propose a two-stream network to detect both tampered faces and low-level
inconsistencies among image patches, but the training is not end-to-end and
requires additional steganalysis feature extraction. Huh et al. [17] use a
Siamese network to predict the image metadata inconsistency on the local
patches cropped from the different images. We introduce consistency learning
to Deepfake detection and propose a practical end-to-end consistency learning
architecture.

##  3 Our Approach

Given an input video frame that contains one or more faces, our goal is to
detect if the identity or expression of a target subject is swapped with that
of a source subject. Observing that Deepfake images are blended by patches
from multiple sources with distinct source features, we explore to learn
effective and robust representations for Deepfake detection by measuring the
consistency of image source features. More specifically, we propose a multi-
task learning architecture, as shown in Fig. 2. The consistency branch is
optimized to predict a consistency map for each image patch, indicating if its
source features are consistent with all others. We integrate the consistency
maps together as a visualization heatmap, which ideally identifies the
manipulated regions and provides a better interpretation for human analysis.
The classification branch is applied to the learned features and outputs
binary scores for forgery detection.

###  3.1 Patch-Wise Consistency Learning (PCL)

The consistency branch computes the pairwise similarity scores of all possible
pairs of local patches in an image and predicts a 4D consistency volume
$\widehat{\textbf{V}}$. Given a pre-processed video frame $X$ of size $H\times
W\times 3$ as input, we first feed it into the backbone and extract the
feature $F$ of size $H^{\prime}\times W^{\prime}\times C$ from an intermediate
convolution layer, where $H^{\prime}$, $W^{\prime}$, and $C$ are height,
width, and channel size, respectively. For each patch $\mathcal{P}_{h,w}$ in
the extracted feature map, we compare it against all the rest to measure their
feature similarities, and obtain a 2D consistency map
$\widehat{M}^{\mathcal{P}_{h,w}}$ of size $H^{\prime}\times W^{\prime}$ of
consistency scores in the range of $[0,1]$, where the superscript indicates
the position of the base patch. To be specific, for any pair of patches
$\mathcal{P}_{i}$ and $\mathcal{P}_{j}$, we compute their dot-product
similarity [57] using their extracted feature vector $f_{i}$ and $f_{j}$, both
of size $C$, to estimate their consistency score:

$\displaystyle
s(f_{i},f_{j})=\sigma\left(\frac{\theta(f_{i})\phi(f_{j})}{\sqrt{C^{\prime}}}\right),$
(1)

where $\theta$ and $\phi$ are the embedding functions, realized by $1\times 1$
convolutions, $C^{\prime}$ is the embedding dimension, and $\sigma$ is the
Sigmoid function. We iterate this process over all patches
$\\{\mathcal{P}_{h,w}|1\leq h\leq H^{\prime},1\leq w\leq W^{\prime}\\}$ in the
extracted feature map, and finally get the 4D consistency volume
$\widehat{\textbf{V}}$ of size $H^{\prime}\times W^{\prime}\times
H^{\prime}\times W^{\prime}$. To easier interpret and provide visualization
clues about the region of modification, we fuse the 4D consistency volume
$\widehat{\textbf{V}}$ over all patches and generate a 2D global heatmap
$\widehat{\mathcal{M}}$ of size $H^{\prime}\times W^{\prime}$. We up-sample
$\widehat{\mathcal{M}}$ to $\widehat{\mathscr{M}}$ of size $H\times W$ to
match the input size for better visualization, as shown in Fig. 5.

Figure 3:  An example of PCL architecture, where ResNet-34 is adopted as
backbone. The features are shown as the shape of their tensors, and proper
reshaping is performed. $\bigotimes$ denotes matrix multiplication, and
$\theta$ and $\phi$ are two $1\times 1\times 1$ convolutions.

The optimization of consistency branch requires the 4D “ground truth”
consistency volume V. Given the mask $\mathscr{M}$ of size $H\times W$
indicating the manipulated region of input $X$, we first create its coarse
version $\mathcal{M}$ matching the size of $H^{\prime}\times W^{\prime}$
through bi-linear down-sampling. We obtain the ground truth 2D consistency map
$M^{\mathcal{P}_{h,w}}$ for the $(h,w)$-th patch by computing the element-wise
difference between its own value $\mathcal{M}_{h,w}$ and all others,

$\displaystyle
M^{\mathcal{P}_{h,w}}=1-\left|\mathcal{M}_{h,w}-\mathcal{M}\right|,$ (2)

where $\mathcal{M}_{h,w}$ is the scalar value in position $(h,w)$, and
$M^{\mathcal{P}_{h,w}}$ is in size of $H^{\prime}\times W^{\prime}$. For each
entry of $M^{\mathcal{P}_{h,w}}$, a value close to $1$ denotes the two patches
are consistent, and close to $0$ otherwise. To obtain the ground truth 4D
global map V, we compute $M^{\mathcal{P}_{h,w}}$ for all patches. Note that V
of a pristine image should be 1, a 4D volume in which all values are equal or
close to one.

We use the binary cross-entropy (BCE) loss to supervise the consistency
prediction over the 4D consistency volume $\widehat{\textbf{V}}$, and more
formally,

$\displaystyle\mathcal{L}_{\textit{pcl}}=\frac{1}{N}\sum_{h,w,h^{\prime},w^{\prime}}\text{BCE}(\textbf{V}_{h,w,h^{\prime},w^{\prime}},\widehat{\textbf{V}}_{h,w,h^{\prime},w^{\prime}}),$
(3)

where $h$ and $h^{\prime}$ $\in[1,H^{\prime}]$, $w$ and $w^{\prime}$
$\in[1,W^{\prime}]$, and $N$ equals to $H^{\prime}\times W^{\prime}\times
H^{\prime}\times W^{\prime}$.

The consistency branch learns the representations that cluster image patches
into groups according to their source features, which by our claim could
significantly benefit the Deepfake detection in both effectiveness and
robustness. Nevertheless, these features cannot directly make inferences for
detection purposes.

The classification branch is applied after the learned feature map to predict
whether the input is real or fake. More specifically, the intermediate
extracted feature $F$ is fed into another convolution operation. A global
average pooling and fully-connected layer are built after that as the
classifier, which outputs the probability score for the input of being fake or
real. We use the two-class cross-entropy (CE) loss
$\mathcal{L}_{\textit{cls}}$ to supervise the training in the classification
branch.

Fig. 3 illustrates an example of the PCL architecture, and the overall loss
function of our model is defined as follows:

$\displaystyle\mathcal{L}=\lambda\mathcal{L}_{\textit{pcl}}+\mathcal{L}_{\textit{cls}},$
(4)

with hyper-parameter $\lambda$. The ablation study in Section 4.5 suggests
that a choice of large $\lambda$ value significantly improves the performance.
This observation demonstrates that the representations learned from the
consistency branch play a dominant role in the success.

###  3.2 Inconsistency Image Generator (I2G)

Training PCL requires data with patch-level annotation of the manipulated
regions, which is not always available in existing datasets. To provide this
training data, we propose Inconsistency Image Generator (I2G) to generate
“self-inconsistent” images from the pristine ones, along with the ground truth
masks $\mathscr{M}$ discussed in Section 3.1. Similar self-supervised
approaches [29, 25, 38, 10] have been studied in other tasks or methods for
Deepfake detection.

I2G particularly addresses several challenges to fit PCL better. First,
because face images have some strong structural bias, the localized pair
sampling scheme in PCL could lead to overfitting to face region information
instead of learning to distinguish source features. I2G uses elastic
deformation [44] to remove the bias in pairs-sampling for the loss computation
in Eq. 3. In practice, we found that this step is particularly crucial in
ensuring that the learned representation can distinguish source features, upon
which an effective classifier can be learned that generalizes well. Second,
because attackers will intentionally try to remove source features to make
Deepfake images more realistic, PCL needs to make use of source features that
are not vulnerable to these approaches. I2G randomly selects one from an
exhaustive set of blending methods in data generation so the representation
learned by PCL can be robust to source feature removal attempts. Third, we
expect the learned representation to generalize to a wide range of sources,
even unseen during training. I2G adds image augmentation to the generation
process to achieve this goal. The augmentation methods include JPEG
compression, Gaussian noise/blur, brightness contrast, random erasing, and
color jittering.

Figure 4:  Illustration of the workflow of I2G. For each source and target
frame pair, a morphed mask is generated by taking a convex hull of the
landmarks followed by elastic deformation and Gaussian blur. The masked region
of the target image is replaced by that of the source with blending techniques
[42, 38].  Algorithm 1 Inconsistency Image Generator (I2G)

Input: Target video frame $X^{t}$ of size $(H,W,3)$.
Output: Generated video frame $X^{g}$ and mask $\mathscr{M}$.
Landmark Detector $\mathcal{K}:\mathbb{R}^{H\times W\times
3}\rightarrow\mathbb{R}^{68\times 2}$.

1:Get the video frame $X^{t}$ and its landmarks $\mathcal{K}(X^{t})$.

2:Find a random source frame $X^{s}$ of different ID, which satisfies
$\lVert\mathcal{K}(X^{t})-\mathcal{K}(X^{s})\rVert_{2}<\epsilon$ for threshold
$\epsilon>0$.

3:Align $X^{s}$ to $X^{t}$ using landmarks.

4:Compute convex hull $\mathcal{H}$ of $\mathcal{K}(X^{t})$.

5:Get mask $\mathscr{M}$ by elastic deforming and blurring $\mathcal{H}$.

6:Get $X^{g}$ by blending $X^{t}$ with $X^{s}$ with $\mathscr{M}$.

The workflow of I2G is summarized in Alg. 1 and illustrated in Fig. 4. Given a
target video frame $X^{t}$, we take its $68$-point facial landmarks and
retrieve another frame from different videos with different identities. We
make sure that the faces in these two frames have similar landmarks measured
in $\ell_{2}$ norm. For a pair of images, we first align their faces with the
pre-computed landmarks and then detect the facial region by taking the convex
hull of the landmarks. Elastic deformation [44] is also employed to morph the
convex hull: we generate the smooth deformations using random displacement
vectors sampled from a Gaussian distribution with a standard deviation of $6$
to $12$ pixels on a coarse $4$ by $4$ grid, and compute per-pixel
displacements using bi-cubic interpolation. The deformed mask is further
blurred by a Gaussian kernel of size $16$. Finally, the facial region of the
source frame within the mask is stitched to the target frame using various
blending methods [42, 38]. I2G outputs a forged video frame and the
corresponding mask $\mathscr{M}$.

I2G could also be utilized as a part of the data augmentation for Deepfake
detection. It has a low computational cost and can be used for online data
generation on CPU during training. For each epoch, the source video frame that
satisfies the condition (Alg. 1 line 2) is randomly chosen to increase the
amount and diversity of the dataset.

##  4 Experiments

We evaluated the performance of our approach (PCL + I2G) against multiple
state-of-the-art methods on seven publicly-available datasets. First, we
showed that our model is able to achieve convincing performance under the in-
dataset setting, where training and testing are conducted on the same dataset.
To demonstrate that PCL has superior generalization ability, we conducted
cross-dataset evaluation by training the model with only I2G-augmented real
videos and testing on unseen datasets. The ablation studies explored the
contribution of each component in our model, such as the effect of PCL and
I2G.

Method | Backbone | Train Set | Test Set (AUC (%))
---|---|---|---
DF | F2F | FS | NT | FF++
MIL [58] | Xception | FF++ | 99.51 | 98.59 | 94.86 | 97.96 | 97.73
Fakespotter [55] | ResNet-50 | FF++, CD2, DFDC | - | - | - | - | 98.50
XN-avg [45] | Xception | FF++ | 99.38 | 99.53 | 99.36 | 97.29 | 98.89
Face X-ray [25] | HRNet | FF++ | 99.12 | 99.31 | 99.09 | 99.27 | 99.20
S-MIL-T [27] | Xception | FF++ | 99.84 | 99.34 | 99.61 | 98.85 | 99.41
PCL + I2G | ResNet-34 | FF++ | 100.00 | 99.57 | 100.00 | 99.58 | 99.79
Table 1: In-dataset evaluation results on FF++. Our method performs better on
all manipulation types with fewer network parameters.

Method Backbone Train Set Test Set (AUC (%)) CD2 Fakespotter [55] ResNet-50
CD2 66.80 Tolosana et al. [50] Xception CD2 83.60 S-MIL-T [27] Xception CD2
98.84 PCL + I2G ResNet-34 CD2 99.98

Table 2: In-dataset evaluation results on CD2. We achieve saturated
performance in terms of AUC.

Method Backbone Train Set Test Set (AUC (%)) DFDC-P Tolosana et al. [50]
Xception DFDC-P 91.10 S-MIL-T [27] Xception DFDC-P 85.11 PCL + I2G ResNet-34
DFDC-P 94.38

Table 3: In-dataset evaluation results on DFDC-P. Our method improves the best
existing result by 3.28% in terms of AUC.

###  4.1 Implementation Details

Pre-processing. For each raw video frame, face crops are detected and tracked
by using [26] and landmarks are detected by public toolbox [5]. We normalize
all face crops with ImageNet mean $[0.485,0.456,0.406]$ and standard deviation
$[0.229,0.224,0.225]$, and resize them to $256\times 256$. We also use
standard data augmentations, including flipping, rotation, JPEG compression,
and Gaussian noise/blur.

Network Architecture. We adopt ResNet-34 [15] with an extra $3\times 3$ max
pooling operation after the conv1 layer as backbone and initialized with
pretrained weights on ImageNet [9]. Given a pre-processed video frame $X$ of
size $H\times W\times 3$, we first feed it into the backbone and extract the
features $F$ after the conv3 layer of size $H^{\prime}\times W^{\prime}\times
256$, where $H^{\prime}=H/16$ and $W^{\prime}=W/16$. Here each patch
corresponds to a $16\times 16$ region in the original image.

Training. For each epoch, we randomly sample $32$ frames from every video, and
the total number of training samples from $K$ videos is $32\times K$. The
model is trained for $150$ epochs using Adam optimizer [21] with batch size
$128$, betas $0.9$ and $0.999$, and epsilon $10^{-8}$. The learning rate is
linearly increased from $0$ to $5\times 10^{-5}$ in the first quarter of the
training iterations and is decayed to zero in the last quarter. The hyper-
parameter $\lambda$ is set to be $10$ by default.

###  4.2 Settings

Training Data. Each of our training samples is of the form $(X,\textbf{V},y)$,
where $X$ is the input video frame, V is the ground truth 4D consistency
volume, and $y$ is the binary label. For a real frame, V is a 4D tensor of
ones, indicating the image is self-consistent, and $y$ is zero. There are two
types of forged samples in our settings. The first type is the Deepfake video
frame from the existing Deepfake datasets, for which we find the corresponding
real video frame and compute the structural dissimilarity (DSSIM) [59] between
them. The mask $\mathscr{M}$ is then generated by taking a Gaussian blur of
DSSIM followed by thresholding. V is computed from $\mathscr{M}$ by Eq. 2 and
$y$ is one. The second type is the fake image augmented by I2G on real images,
where V is computed with the mask from I2G and $y$ is one. By training with
I2G-augmented datasets, whenever a real data $(X^{t},\bm{1},0)$ is sampled
during the training, there is a $50\%$ chance that it is dynamically
transformed by I2G into a fake data $(X^{g},\textbf{V},1)$ where $X^{g}$ and
$\mathscr{M}$ are the outputs of I2G as described in Alg. 1 and V is computed
with $\mathscr{M}$ by Eq. 2. A proper strategy is to balance the total number
of real and fake data when both types of forgeries are involved in the
training. Moreover, we observe that the performance of our model is not
sensitive to the exact real and fake ratio in the train set, as long as it is
not extremely imbalanced.

More specifically, in the in-dataset experiments, our train set includes both
the pristine and Deepfake videos from the train split of the dataset (to be
evaluated). Unless otherwise noted, we also utilize the fake samples augmented
by I2G as data augmentation. In the cross-dataset experiments, we follow prior
work [25] and train with only real videos from the raw version of
FaceForensics++ (FF++) [45], augmented by I2G.

Test Data. FaceForensics++ (FF++) [45] is by far the most popular dataset in
the Deepfake detection research community, with plentiful benchmark results.
The raw version of its test set contains $700$ videos, including $140$
pristine videos and $560$ manipulated counterparts from $4$ different
algorithms, which are Deepfakes (DF) [13], Face2Face (F2F) [48], FaceSwap (FS)
[23] and NeuralTextures (NT) [49], respectively. DeepfakeDetection (DFD) [12]
dataset is released by Google & JigSaw incorporated with the FaceForensics++
team, supporting the Deepfake detection research. Celeb-DF-v1 (CD1) & -v2
(CD2) [30] are datasets consisting of high-quality Deepfake celebrity videos
using advanced synthesis process. Deepfake detection Challenge (DFDC) [10]
public test set is a large-scale dataset released for the Deepfake Detection
Challenge, and DFDC Preview (DFDC-P) [11] is its preliminary version. We
observe that both DFDC and DFDC-P contain many extremely low-quality videos,
which make them exceptionally challenging. DeeperForensics-1.0 (DFR) [18] is a
recently released dataset, which modifies the pristine videos in FF++ with new
face IDs and more advanced techniques. More detailed statistics for test sets
is summarized in the Appendix.

Evaluation Metrics. We report the Deepfake detection results with the most
commonly used metrics in the literature, including area under the ROC curve
(AUC) and Average precision (AP). A higher AUC or AP value indicates better
performance. To provide a comprehensive benchmark for future work, we report
our performance on all datasets in terms of AUC, AP, as well as Equal Error
Rate (EER) in the Appendix. Unless otherwise noted, the evaluation results in
the experiments are at video-level, computed by averaging the classification
scores of the video frames.

###  4.3 In-Dataset Evaluation

In-dataset evaluation is abundantly adopted in the literature, where the focus
is on specialization but not generalization. To compare against the existing
work, we consider three of the most popular datasets, which are FF++, CD2, and
DFDC-P. Given a dataset, our model is trained on both real and Deepfake data
from train split, and performance is evaluated with the corresponding test
set.

The results for FF++, CD2, DFDC-P are shown in Table 1, Table 2, Table 3,
respectively. In particular, our models achieve the state-of-the-art with
almost perfect performance on CD2 ($99.98\%$) and all four manipulations of
FF++ ($100.00\%$ on DF, $99.57\%$ on F2F, $100.00\%$ FS, $99.58\%$ on NT,
respectively), surpassing all existing work. For DFDC-P, our model outperforms
the state-of-the-art result by $3.28\%$ in terms of AUC score. Note that the
results reported for DFDC-P are comparably lower, due to the fact that a non-
negligible portion of the dataset is of extremely low quality, , human faces
in some videos are hardly recognizable. In addition to video-level evaluation,
we also compare with prior work in terms of frame-level AUC on CD2, and
outperform the state-of-the-art [33] by $8\%$. (see Appendix for more
details).

Figure 5: Visualization of the consistency heatmaps, which localize the manipulated regions. The predictions are obtained in the cross-dataset experiments, where our model is trained with only real videos of FF++, augmented by I2G. The ground truth masks are generated by DSSIM, as discussed in Section 4.2. Method | Backbone | Train Set | Test Set (AUC (%))
---|---|---|---
DF | F2F | FS | NT | FF++ | DFD
Face X-ray [25] | HRNet | FF++ (real videos) | 99.17 | 98.57 | 98.21 | 98.13 | 98.52 | 93.47
PCL + I2G | ResNet-34 | FF++ (real videos) | 100.00 | 98.97 | 99.86 | 97.63 | 99.11 | 99.07
Table 4: Cross-dataset evaluation results on FF++ and DFD. Our model is on par with Face X-ray [25] on FF++, but has better performance on DFD by 5.67% in terms of AUC, with fewer network parameters. Method | Backbone | Train Set | Test Set (AUC (%))
---|---|---|---
DFR | CD1 | CD2 | DFDC | DFDC-P
Dang et al. [7] | Xception + Reg. |  UADFV [61], DFFD [7] | - | 71.20 | - | - | -
DSP-FWA [29] | ResNet-50 | FF++ | - | - | 69.30 | - | -
Xception [45] | Xception | FF++ | - | - | 73.04 | - | -
Masi et al. [33] | LSTM | FF++ | - | - | 76.65 | - | -
Face X-ray [25] | HRNet | FF++ | - | 80.58 | - | - | 80.92
PCL + I2G | ResNet-34 | FF++ (real data) | 99.41 | 98.30 | 90.03 | 67.52 | 74.37
Table 5: Cross-dataset evaluation results on DFR, CD1, CD2, DFDC and DFDC-P
datasets. Our model out-performs the state-of-the-arts on CD1 and CD2, by
about $18.00\%$ and $13.00\%$ in terms of AUC, and provides pioneering cross-
dataset baselines on DFR (99.51%) and DFDC (67.52%). For DFDC-P, we have a
lower AUC score but a higher AP score in comparison with the state-of-the-arts
(see Table 6).

Method Backbone Train Set Test Set (AP (%)) CD1 DFDC-P Face X-ray [25] HRNet
FF++ 73.33 72.65 PCL + I2G ResNet-34 FF++ (real data) 98.97 82.94

Table 6: Cross-dataset evaluation results on CD1 and DFDC-P datasets. Our
models can identify the attack video more precisely.

###  4.4 Cross-Dataset Evaluation

The generalization of Deepfake detection is a serious concern even for large-
scale training. It is reported that the ranking of top-performing models
changes significantly from DFDC public test set to private one [14]. Whether
or not a method that achieves good results on its own test set can still
perform well on other benchmarks remains unclear.

Table 4 presents the cross-dataset evaluation results on FF++ and DFD
datasets, where we only used the real videos of FF++ for training. The
performance of our model is on par with Face X-ray [25] on FF++, achieving
convincing results with more than $99.00\%$ in terms of AUC. Interestingly,
the performance gap between our model and Face X-ray [25] is much larger
(99.07% vs. 93.47%) when evaluated on DFD. It is possible that the test data
of FF++ is highly correlated with its training data, since they are very
likely to be collected from the same source, whereas the correlation
disappears in DFD. The results demonstrate that predicting the source features
consistency can effectively generalize across different source cues, without
overfitting to any spurious correlation among data from the same generation
method.

We further evaluate our model on five more advanced datasets, as shown in
Table 5. In particular, our model outperforms the state-of-the-art on CD1 and
CD2, by about $18.00\%$ and $13.00\%$ in terms of AUC, and provide pioneering
cross-dataset baselines on DFR (99.51%) and DFDC (67.52%). On DFDC-P, our
performance is comparable with Face X-ray [25], where we are lower AUC but
have a higher AP score, as shown in Table 6. We observe that both our model
and state-of-art-art methods cannot achieve appealing results on DFDC/DFDC-P
datasets, which motivates us to do more failure analysis in Section 4.6.

The consistency predictions from our model provide interpretable visualization
clues about the region of face modification. Qualitative results generated by
our cross-dataset models on different datasets are shown in Fig. 5.

###  4.5 Ablation Studies

Effect of PCL. We impose a weight $\lambda$ to balance between the consistency
and classification losses, as shown in Eq. 4. By setting $\lambda=0$, we
disable PCL and get a network architecture equivalent to vanilla ResNet-34
with binary classification loss. To exhibit the advantage of our consistency
loss, we train models with increasing $\lambda$s and evaluate their cross-
dataset generalizations with four test sets. As shown in Table 7, we follow
the cross-dataset setting and train all models on real data of FF++ augmented
by I2G and report the AUC score for performance comparison. We observe that
training with $\lambda>0$ significantly outperforms the training with
$\lambda=0$. Especially, the performance on DFDC is improved by $15.8\%$ in
terms of AUC. The results demonstrate that it is beneficial to use large
$\lambda$ for the training, which also suggests that PCL plays a dominant role
in the success.

Effect of I2G as Joint-Training. We have been training with our proposed
consistency loss on datasets augmented by I2G. I2G generates fake data
dynamically during the training, enhancing the training data variety, thereby
improving the performance and generalization of our model. To demonstrate the
effect of I2G experimentally, we conduct ablation studies by training on
either DF or DFDC-P and benchmarking on DFR, CD2, DFDC, and DFDC-P test sets.
Results in Table 8 show that models trained on datasets augmented by I2G are
able to achieve a significant performance boost. In particular, we train
models on the DF with or without I2G, whereas the performance of the latter is
improved by an average AUC score of $7.18\%$ over four test sets. The model
trained with the DFDC-P train set has noticeably better performance on DFDC
and DFDC-P test sets in comparison with the previous models, but generalizes
poorly on other datasets such as DFR. I2G significantly improves its
generalization, , performance on DFR is raised from $51.61\%$ to $92.25\%$ in
terms of AUC, with a minor sacrifice to the performance on DFDC-P.

Method Hyper- Parameter Test Set (AUC (%)) Avg DFR CD2 DFDC DFDC-P I2G
$\lambda=0$ 95.12 78.18 51.72 69.93 73.74 PCL + I2G $\lambda=1$ 99.1 86.52
60.65 74.13 80.10 PCL + I2G $\lambda=10$ 99.41 90.03 67.52 74.37 82.83 PCL +
I2G $\lambda=100$ 99.78 90.98 63.22 74.36 82.09

Table 7: Ablation study on the effect of PCL on DFR, CD2, DFDC, and DFDC-P
datasets. The use of large $\lambda$ significantly improves the cross-dataset
performance, especially on DFDC.

Method Train Set Test Set (AUC (%)) Avg DFR CD2 DFDC DFDC-P PCL DF 90.42 84.59
66.26 75.49 79.19 PCL + I2G DF 99.64 91.92 73.08 80.83 86.37 PCL DFDC-P 51.61
82.82 69.14 95.53 74.78 PCL + I2G DFDC-P 92.25 87.65 71.12 94.38 86.35

Table 8: Ablation study on the effect of I2G as joint-training on DF and
DFDC-P datasets. I2G can enhance the variety of training data, thereby
improving the generalization of our model. Figure 6: Visualization of false
negative (FN) and false positive (FP) predictions on DFDC dataset.

Effect of I2G as Pre-Training. Given a dataset for which even computing the
DSSIM masks of Deepfake data is not feasible, one can always use our
consistency loss to train a pretrained model with any real data augmented by
I2G. After that, any standard training for Deepfake detection or related tasks
can be conducted with the whole dataset. In particular, we conducted an
experiment where we first pretrained a ResNet-34 on the real data of DF
augmented by I2G for consistency prediction, then finetuned with both real and
Deepfake data of DF for classification. We report the evaluation results on
DFR, CD2, DFDC, DFDC-P test sets with $99.57\%$, $91.88\%$, $68.95\%$,
$79.17\%$ ($84.89\%$ on average) in terms of AUC, respectively. These results
are unsurprisingly lower than those of joint-training, but they still
significantly outperform the baselines. On the other hand, our pretrained
models may not necessarily be trained from the established Deepfake datasets.
I2G can be potentially applied to any face image or video datasets, such as
IMDb-Face [54] and YouTube Faces [60], providing new and strong pretrained
models for Deepfake-related research.

###  4.6 Failure Cases & Limitations

Although our results are encouraging, our approach has limitations and raise
opportunities for future work. First, we only focus on facial image forgeries
that are self-inconsistent. It is unknown whether it is applicable to entire
face synthesis, where the consistency is supposed to be high across all
regions. Second, we investigate the failure cases in DFDC, as shown in Fig. 6.
From false negative samples, we observed that the quality of the samples is
low in the dataset due to the high compression, high/low exposure that some of
the samples cannot be recognized. Therefore, the cost of Deepfake generation
is highly increased to bypass our system. From false positive samples, we
observed the inconsistency is caught by our model in some cases, which might
be caused by the lighting and unusual texture.

##  5 Conclusion

In this paper, we proposed to use patch-wise consistency learning for
detecting face manipulations and localizing the Deepfake forged regions. To
provide the pixel-level annotations, Inconsistency Image Generator (I2G) is
developed to generate “self-inconsistent” images from the pristine ones. The
designed consistency comparison module only requires minor modification upon
the standard backbones and provides a visual explanation of face manipulation
without extra deconvolutions. We demonstrate that our method has superior
generalization ability, since the classification based on consistency features
instead of forgery features is more generalizable to unseen generation
techniques. Our model can also be adopted as pretrained initialization for
downstream datasets from which the consistency labels are not available. On
the other hand, the proposed I2G pipeline is also effective in improving the
cross-domain generalization of our models. Extensive experiments are conducted
to exhibit that our approach is competitive in comparison with the state-of-
the-art on seven widely used datasets, providing strong baselines for future
research.

## References

  * [1] Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen.  Mesonet: a compact facial video forgery detection network.  In Proc. Workshop on Information Forensics and Security, 2018.
  * [2] Edoardo Ardizzone, Alessandro Bruno, and Giuseppe Mazzola.  Copy-move forgery detection by matching triangles of keypoints.  IEEE Transactions on Information Forensics and Security, 10(10):2084–2094, 2015.
  * [3] Mauro Barni, Luca Bondi, Nicolò Bonettini, Paolo Bestagini, Andrea Costanzo, Marco Maggini, Benedetta Tondi, and Stefano Tubaro.  Aligned and non-aligned double jpeg detection using convolutional neural networks.  Journal of Visual Communication and Image Representation, 49:153–163, 2017.
  * [4] Luca Bondi, Luca Baroffio, David Güera, Paolo Bestagini, Edward J Delp, and Stefano Tubaro.  First steps toward camera model identification with convolutional neural networks.  IEEE Signal Processing Letters, 24(3):259–263, 2016.
  * [5] Adrian Bulat and Georgios Tzimiropoulos.  How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks).  In Proc. IEEE/CVF International Conference on Computer Vision, pages 1021–1030, Oct. 22–29 2017.
  * [6] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha.  StarGAN v2: Diverse image synthesis for multiple domains.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8188–8197, Jun. 14–19 2020.
  * [7] Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil Jain.  On the detection of digital face manipulation.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5781–5790, Jun 14–19 2020.
  * [8] Oscar de Lima, Sean Franklin, Shreshtha Basu, Blake Karwoski, and Annet George.  Deepfake detection using spatiotemporal convolutional networks.  arXiv:2006.14749, 2020.
  * [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A large-scale hierarchical image database.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 248–255, Jun. 20–25 2009.
  * [10] Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer.  The deepfake detection challenge dataset.  arXiv:2006.07397, 2020.
  * [11] Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton Ferrer.  The deepfake detection challenge (dfdc) preview dataset.  arXiv:1910.08854, 2019.
  * [12] Nick Dufour and Andrew Gully.  Contributing data to deepfake detection research, 2019.  https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html.
  * [13] FaceSwapDevs.  Deepfakes, 2019.  https://github.com/deepfakes/faceswap.
  * [14] Chistian Conton Ferrer, Brian Colhansky, Ben Pflaum, Joanna Bitton, Jacqueline Pan, and Jikuo Lu.  Deepfake detection challenge results: An open initiative to advance ai, 2020.  https://ai.facebook.com/blog/deepfake-detection-challenge-results-an-open-initiative-to-advance-ai/.
  * [15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–778, Jun. 26–Jul. 1 2016.
  * [16] Yihao Huang, Felix Juefei-Xu, Run Wang, Xiaofei Xie, Lei Ma, Jianwen Li, Weikai Miao, Yang Liu, and Geguang Pu.  FakeLocator: Robust localization of GAN-based face manipulations via semantic segmentation networks with bells and whistles.  arXiv:2001.09598, 2020.
  * [17] Minyoung Huh, Andrew Liu, Andrew Owens, and Alexei A. Efros.  Fighting fake news: Image splice detection via learned self-consistency.  In Proc. European Conference on Computer Vision, pages 101–117, Sep 8–14 2018.
  * [18] Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy.  DeeperForensics-1.0: A large-scale dataset for real-world face forgery detection.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2886–2895, Jun. 14–19 2020.
  * [19] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, Jun. 16–20 2019.
  * [20] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of StyleGAN.  arXiv:1912.04958, 2019.
  * [21] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In Proc. International Conference on Learning Representations, 2015\.
  * [22] Pavel Korshunov and Sebastien Marcel.  DeepFakes: a new threat to face recognition? assessment and detection.  arXiv:1812.08685, 2018.
  * [23] Marek Kowalski.  FaceSwap, 2018.  https://github.com/MarekKowalski/FaceSwap.
  * [24] Akash Kumar and Arnav Bhavsar.  Detecting deepfakes with metric learning.  arXiv:2003.08645, 2020.
  * [25] Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo.  Face X-ray for more general face forgery detection.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5001–5010, Jun. 14–19 2020.
  * [26] Wei Li, Yuanjun Xiong, Shuo Yang, Siqi Deng, and Wei Xia.  SMOT: Single-shot multi object tracking.  arXiv:2010.16031, 2020.
  * [27] Xiaodan Li, Yining Lang, Yuefeng Chen, Xiaofeng Mao, Yuan He, Shuhui Wang, Hui Xue, and Quan Lu.  Sharp multiple instance learning for DeepFake video detection.  In Proc. ACM Multimedia, Oct. 12–16 2020.
  * [28] Yuezun Li, Ming-Ching Chang, and Siwei Lyu.  In ictu oculi: Exposing ai created fake videos by detecting eye blinking.  In Proc. International Workshop on Information Forensics and Security, 2018.
  * [29] Yuezun Li and Siwei Lyu.  Exposing deepfake videos by detecting face warping artifacts.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Jun. 16–20 2019.
  * [30] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu.  Celeb-DF: A large-scale challenging dataset for deepfake forensics.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3207–3216, Jun. 14–19 2020.
  * [31] Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen.  STGAN: A unified selective transfer network for arbitrary image attribute editing.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3673–3682, Jun. 16–20 2019.
  * [32] Jan Lukas, Jessica Fridrich, and Miroslav Goljan.  Determining digital image origin using sensor imperfections.  In Proc. Image and Video Communications and Processing, 2005.
  * [33] Iacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed.  Two-branch recurrent network for isolating deepfakes in videos.  In Proc. European Conference on Computer Vision, Aug. 23–28 2020\.
  * [34] Falko Matern, Christian Riess, and Marc Stamminger.  Exploiting visual artifacts to expose deepfakes and face manipulations.  In Proc. Winter Applications of Computer Vision Workshops, 2019\.
  * [35] Owen Mayer and Matthew C Stamm.  Learned forensic source similarity for unknown camera models.  In International Conference on Acoustics, Speech and Signal Processing, pages 2012–2016, 2018.
  * [36] Owen Mayer and Matthew C Stamm.  Forensic similarity for digital images.  IEEE Transactions on Information Forensics and Security, 15:1331–1346, 2019.
  * [37] Owen Mayer and Matthew C Stamm.  Exposing fake images with forensic similarity graphs.  IEEE Journal of Selected Topics in Signal Processing, 14(5):1049–1064, 2020.
  * [38] Jacek Naruniec, Leonhard Helminger, Christopher Schroers, and Romann M. Weber.  High-resolution neural face swapping for visual effects.  In Proc. Eurographics Symposium on Rendering, 2020.
  * [39] João C. Neves, Ruben Tolosana, Ruben Vera-Rodriguez, Vasco Lopes, Hugo Proença, and Julian Fierrez.  GANprintR: Improved fakes and evaluation of the state-of-the-art in face manipulation detection.  Journal of Selected Topics in Signal Processing, 14(5):1038–1048, 2020.
  * [40] Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen.  Multi-task learning for detecting and segmenting manipulated facial images and videos.  In Proc. Biometrics Theory, Applications and Systems, Sep. 23–26 2019.
  * [41] Huy H Nguyen, Junichi Yamagishi, and Isao Echizen.  Capsule-forensics: Using capsule networks to detect forged images and videos.  In Proc. International Conference on Acoustics, Speech and Signal Processing, 2019.
  * [42] Patrick Pérez, Michel Gangnet, and Andrew Blake.  Poisson image editing.  ACM Transactions on Graphics, 2003.
  * [43] Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Mr. Dpfks, Carl Shift Facenheim, Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, and Weiming Zhang.  DeepFaceLab: A simple, flexible and extensible face swapping framework.  arXiv:2005.05535, 2020.
  * [44] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional networks for biomedical image segmentation.  In Proc. Medical Image Computing and Computer-assisted Intervention, 2015.
  * [45] Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner.  FaceForensics++: Learning to detect manipulated facial images.  In Proc. IEEE/CVF International Conference on Computer Vision, pages 1–11, Oct. 27–Nov. 2 2019.
  * [46] Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael AbdAlmageed, Iacopo Masi, and Prem Natarajan.  Recurrent convolutional strategies for face manipulation detection in videos.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop, Jun. 16–20 2019.
  * [47] Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt.  MoFA: Model-based deep convolutional face autoencoder for unsupervised monocular reconstruction.  In Proc. IEEE/CVF International Conference on Computer Vision Workshops, pages 1274–1283, Oct. 22–29 2017.
  * [48] J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner.  Face2Face: Real-time Face Capture and Reenactment of RGB Videos.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2387–2395, Jun. 26–Jul. 1 2016.
  * [49] Justus Thies, Michael Zollhöfer, and Matthias Nießner.  Deferred neural rendering: Image synthesis using neural textures.  In ACM Transactions on Graphics, volume 38, pages 1–12, 2019.
  * [50] Ruben Tolosana, Sergio Romero-Tapiador, Julian Fierrez, and Ruben Vera-Rodriguez.  DeepFakes evolution: Analysis of facial regions and fake detection performance.  arXiv:2004.07532, 2020.
  * [51] Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia.  Deepfakes and beyond: A survey of face manipulation and fake detection.  arXiv preprint arXiv:2001.00179, 2020.
  * [52] Loc Trinh, Michael Tsang, Sirisha Rambhatla, and Yan Liu.  Interpretable deepfake detection via dynamic prototypes.  arXiv:2006.15473, 2020.
  * [53] Arash Vahdat and Jan Kautz.  NVAE: A deep hierarchical variational autoencoder.  In Proc. Advances in Neural Information Processing Systems, Dec. 6–12 2020.
  * [54] Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, and Chen Change Loy.  The devil of face recognition is in the noise.  arXiv:1807.11649, 2018.
  * [55] Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, and Yang Liu.  FakeSpotter: A simple yet robust baseline for spotting ai-synthesized fake faces.  arXiv:1909.06122, 2019.
  * [56] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros.  CNN-generated images are surprisingly easy to spot… for now.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8695–8704, Jun 14–19 2020.
  * [57] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He.  Non-local neural networks.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7794–7803, Jun. 18–22 2018.
  * [58] Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu.  Revisiting multiple instance neural networks.  Pattern Recognition, 74:15–24, 2018.
  * [59] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  * [60] Lior Wolf, Tal Hassner, and Itay Maoz.  Face recognition in unconstrained videos with matched background similarity.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 529–534, Jun. 20–25 2011.
  * [61] Xin Yang, Yuezun Li, and Siwei Lyu.  Exposing deep fakes using inconsistent head poses.  In International Conference on Acoustics, Speech, and Signal Processing, 2018.
  * [62] Ning Yu, Larry Davis, and Mario Fritz.  Attributing fake images to GANs: Analyzing fingerprints in generated images.  In Proc. IEEE/CVF International Conference on Computer Vision, pages 7556–7566, Oct. 27–Nov. 2 2019.
  * [63] P. Zhou, X. Han, V. I. Morariu, and L. S. Davis.  Two-stream neural networks for tampered face detection.  In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Jun. 21–26 2017.
  * [64] Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang.  WildDeepfake: A challenging real-world dataset for deepfake detection.  In Proc. ACM International Conference on Multimedia, pages 2382–2390, Oct. 2020.

##  Appendix A Video-Level Comprehensive Results

To provide a comprehensive benchmark for future work on Deepfake detection, we
report the performance of our model on seven popular datasets in terms of AUC,
AP, and EER. The results are shown in Table 9. Note that the evaluation
results are at video-level, computed by averaging the classification scores of
the video frames. All of these experiments are conducted under the cross-
dataset setting, where we train our model with only real videos from the raw
version of FF++ [45] and the fake/positive samples are generated by I2G (see
Sec. 4.2 for more details).

Additional qualitative results are shown in Fig. 7. These images are randomly
chosen from Celeb-DF-v2 [30] and DFDC Public test set [10], which are among
the most challenging datasets in Deepfake detection. In particular, we
visualize the consistency maps from true positive, false negative, true
negative, and false positive predictions to help fully interpret our model.

Method | Test Set | Evaluation Metrics (%)
---|---|---
AUC | AP | EER
PCL + I2G | DF [13] | 100.00 | 100.00 | 0.00
F2F [48] | 98.97 | 99.32 | 3.57
FS [23] | 99.86 | 99.86 | 1.43
NT [49] | 97.63 | 98.20 | 6.43
FF++ [45] | 99.11 | 99.80 | 3.57
DFD [12] | 99.07 | 99.89 | 4.42
DFR [18] | 99.41 | 99.51 | 3.48
CD1 [30] | 98.30 | 98.97 | 7.89
CD2 [30] | 90.03 | 94.45 | 17.98
DFDC [10] | 67.52 | 69.99 | 37.18
DFDC-P [11] | 74.37 | 82.94 | 31.87
Table 9: Comprehensive evaluation of our model in terms of video-level AUC,
AP, and EER on seven datasets.

##  Appendix B Frame-Level Results on Celeb-DF-v2

Besides the video-level evaluation, we also compare our model with state-of-
the-art methods in terms of frame-level AUC on CD2 [30], where our model is
trained under the cross-dataset setting (see Sec. 4.2 for more details). As
shown in Table 10, our model outperforms the state-of-the-art method [33] by
over $8\%$.

Method CD2 (Frame-Level AUC (%)) Two-stream [63] 53.8 Meso4 [1] 54.8
MesoInception4 53.6 HeadPose [61] 54.6 FWA [29] 56.9 VA-MLP [34] 55.0 VA-
LogReg 55.1 Xception-raw [45] 48.2 Xception-c23 65.3 Xception-c40 65.5 Multi-
task [40] 54.3 Capsule [41] 57.5 DSP-FWA [29] 64.6 Masi et al. [33] 73.4 PCL +
I2G 81.8

Table 10:  Cross-dataset evaluation results of our model in terms of frame-
level AUC on CD2 dataset. The performance of existing methods are cited for
comparison.

##  Appendix C Test Set Statistics

Test Set | # Real / Fake Videos
---|---
FaceForensics++ (FF++) [45] | 140 / 560
FF++ - Deepfakes (DF) [13] | 140 / 140
FF++ - Face2Face (F2F) [48] | 140 / 140
FF++ - FaceSwap (FS) [23] | 140 / 140
FF++ - NeuralTextures (NT) [49] | 140 / 140
DeepfakeDetection (DFD) [12] | 363 / 3431
Celeb-DF-v1 (CD1) [30] | 38 / 62
Celeb-DF-v2 (CD2) [30] | 178 / 340
DFDC Public (DFDC) [10] | 2000 / 2000
DFDC Preview (DFDC-P) [11] | 276 / 504
DeeperForensics-1.0 (DFR) [18] | 201 / 201
Table 11: Statistics of real and fake videos in the test sets. Figure 7:
Visualization of the consistency maps, which localize the manipulated regions.
The predictions are obtained in the cross-dataset experiments, where our model
is trained with only real videos of FF++, augmented by I2G. The ground truth
masks are generated by DSSIM, as discussed in Section 4.2.