# Block shuffling learning for Deepfake Detection

Sitong Liu İD , Zhichao Lian İD , , Siqi Gu İD , Liang Xiao İD Manuscript
received September 20, 2022. This work was supported in part by the National
Natural Science Foundation of China under Grant 61871226, Grant 61571230,
Grant 61802190, and Grant 61906093, in part by the Jiangsu Provincial Social
Developing Project under Grant BE2018727, in part by the Open Research Fund in
2021 of Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense under
Grant JSGP202101, and Grant JSGP202204, in part by the National Key R&D
Program of China under Grant 2021YFF0602104-2.

###### Abstract

Deepfake detection methods based on convolutional neural networks (CNN) have
demonstrated high accuracy. However, these methods often suffer from decreased
performance when faced with unknown forgery methods and common transformations
such as resizing and blurring, resulting in deviations between training and
testing domains. This phenomenon, known as overfitting, poses a significant
challenge. To address this issue, we propose a novel block shuffling
regularization method. Firstly, our approach involves dividing the images into
blocks and applying both intra-block and inter-block shuffling techniques.
This process indirectly achieves weight-sharing across different dimensions.
Secondly, we introduce an adversarial loss algorithm to mitigate the
overfitting problem induced by the shuffling noise. Finally, we restore the
spatial layout of the blocks to capture the semantic associations among them.
Extensive experiments validate the effectiveness of our proposed method, which
surpasses existing approaches in forgery face detection. Notably, our method
exhibits excellent generalization capabilities, demonstrating robustness
against cross-dataset evaluations and common image transformations. Especially
our method can be easily integrated with various CNN models. Source code is
available at Github.

###### Index Terms:

Deepfake detection, overfitting, regularization, block shuffling.

##  I Introduction

In recent years, the field of deep learning has witnessed significant
advancements, particularly in the area of face forgery techniques. Notably,
these techniques, which rely on deep learning, have demonstrated substantial
progress. As reported by Sensity in a comprehensive survey[1], the
proliferation of harmful deepfake videos crafted by skilled creators on the
Internet has been alarmingly escalating, with the number of such videos
approximately doubling every 6 months. These face forgery techniques can be
categorized based on their intended targets, namely face swapping[2], facial
attribute editing[3, 4], and face generation[5], as depicted in Fig. 1. Of
particular influence are face swapping methods, with DeepFakes[6] serving as a
prominent example. These techniques possess the capability to manipulate the
facial identity information within an image. Consequently, the widespread
dissemination of fabricated images on the Internet has engendered significant
security risks, thus necessitating the advancement of deepfake detection
technologies. Presently, the majority of detection algorithms[7] approach the
problem as a classification task, leveraging data-driven training of
convolutional neural networks (CNNs) to identify fake faces and achieve good
performance. However, the continuous update of forgery methods and the
propagation of images often leads to varying degrees of degradation, including
resizing and blurring, thereby challenging the generalization capability of
detection methods. In severe cases, deep models may exhibit satisfactory
performance on the training data but encounter a considerable decline in
accuracy when confronted with new data. An essential contributing factor to
this phenomenon is the overfitting problem inherent in the detection model’s
local context.

Figure 1: An illustration of various real and fake faces. Both real faces and
swapped faces are taken from CelebDF(v2) [8], the faces edited by the
expression adopt GANimation [9], and the generated faces are taken from
StyleGAN [10].

The issue of overfitting significantly affects the task of forgery detection,
as the decision-making process for identifying fake images primarily relies on
localized regions. Face forgery entails altering specific attributes, such as
identity and expression, of a face in an image to resemble those of another
face. The forgery techniques aim to bridge the visual and distribution gap
between the two images. Various forgery methods introduce distinct artifacts-
based features in different facial regions[11]. Deepfake detection methods[12,
13, 14] heavily rely on the accurate perception of these local regions to
identify subtle cues, such as mismatched colors, distorted shapes, and
incorrect textures, caused by imperfect facial manipulation techniques.
However, these traces of forgery are inherently unstable, primarily due to
differences in forgery methods and image scaling, compression, and cropping
during their dissemination across the Internet. Consequently, the relatively
small and unstable local regions, combined with the vast number of parameters
within the network, give rise to the overfitting problem, thus impeding the
model’s generalization ability. Motivated by the effectiveness of
regularization techniques in addressing overfitting, we propose a novel
regularization method that specifically focuses on the local regions, aiming
to enhance the forgery detection performance of the model.

This paper introduces a weight-sharing regularization-based framework for
deepfake detection, referred to as Block Shuffling Learning (BSL). The
proposed framework focuses on positional transformations of image pixels,
maintaining their original values. It comprises two main operations: intra-
block shuffling and inter-block shuffling, which facilitate indirect weight-
sharing across different dimensions. In the intra-block shuffling operation,
local representations are enhanced through the introduction of positional
transformations. Additionally, an adversarial loss is designed to filter out
noisy patterns resulting from intra-block shuffling. In the inter-block
shuffling operation, the global structure is disrupted to encourage feature
extraction from local regions. The semantic correlations among blocks are then
modeled by restoring the original regional arrangement. It is important to
emphasize that our framework represents more than a simple integration or
modification of existing methods. It fundamentally differs from existing
approaches in terms of usage scenarios, compositional principles, and
operational logic. To the best of our knowledge, we are the first to address
the issue of overfitting caused by local regions in the context of deepfake
detection and propose regularization as a solution. Existing scramble-based
regularization methods [15, 16] are not specifically designed for deepfake
detection tasks and fail to account for the noise introduced by random
shuffling operations, rendering them relatively ineffective.

Our work makes the following main contributions:

  * •

We propose an efficient weight-sharing regularization method to address the
overfitting problem in local regions, which consists of an intra-block
shuffling and an inter-block shuffling. Our method can be easily adopted in a
variety of CNN models.

  * •

We propose an adversarial loss to overcome the shuffling noise and model the
semantic relevance of local features through the position restoration of the
blocks.

  * •

Our method has undergone extensive experimentation, which substantiates its
superiority in terms of detection performance across various datasets.
Consistently surpassing existing methods, our approach stands out,
particularly excelling in cross-dataset and cross-image quality experiments.
These results serve as compelling evidence that our regularization method
effectively ensures the network’s generalization capabilities in the context
of forgery detection.

##  II Related works

In this section, we briefly review the current representative face forgery
detection and regularisation methods relevant to our work.

Deepfake Detection. To cope with the security problems caused by Deepfakes,
people propose deepfake detection methods from multiple insights. Early work
focused on designing handcrafted features [17, 18, 19, 20]. The mainstream
deep learning-based methods focus on how to extract features from the spatial
domain [21, 22, 23, 24]and frequency domain information [25, 26, 27]. Nguyen
et al. [23] propose a detection method based on the capsule network [28]. Li
et al. [24] propose that the artefacts in affine face warping as the
distinctive feature to forgery detect, and achieve state-of-the-art
performance based on SSPNet [29]. Li et al. [14] propose the Face X-ray that
focuses on the blending step of forgery and achieves state-of-the-art
transferability performance. However, it cannot be used for fully synthesized
images, and the performance on low-resolution images drops sharply. Durallet
et al. [25] first propose using Discrete Fourier Transform (DFT) to mine
abnormal information in forged face images. Luo et al. [26] propose utilising
the high-frequency noises for face forgery detection. Qian et al. [27] use DFT
to extract frequency-aware image decomposition and local frequency statistics
to improve the forgery detection performance on high-compression videos.
However, the cross-data set performance dropped significantly.

Since the differences between real and fake images are primarily subtle and
local, researchers currently find solutions from local areas. Du et al. [7]
focus on local spatial features and learn the internal representation of local
forged regions through auto-encoders to bridge the generalization differences.
Zhao et al. [13] propose a multi-attentional network to focus on different
local parts and subtle artefacts. Wang et al. [30] propose an attention-based
data enhancement framework, which encourages the network to mine diverse
features by occluding sensitive areas. Wang et al. [31] propose a multi-modal
multi-scale transformer to capture the subtle manipulation artefacts at
different scales. We focus on local regions to solve the forgery detection
problem. The difference is that we pay more attention to the overfitting
problem of the network.

Regularization. Generalization is an essential topic for forgery detection
algorithms that rely on CNN. Regularization is an effective method to improve
generalization and reduce the impact of overfitting in the training of CNN
models. Two kinds of regularisation are common today: model ensemble and
weight-sharing. Model ensemble refers to multiple separately trained models
voting to decide the output. The voting procedure is robust to prediction
errors made by individual classifiers. Using a model ensemble requires a huge
computational cost, so many methods implement it implicitly. Dropout[32]
achieves the average of architectures by temporarily dropping the neurons in
the network with a certain probability during training. Stochastic Pooling[33]
randomly selects activation from a multinomial distribution during training to
average the hidden layer inputs. In another way, weight-sharing requires a set
of weights in the network to be equal to achieve invariant transformation
properties[34] and gains generalizability to permutations of the input[35].
PatchShuffle[15] randomly shuffles the pixels within each local patch while
maintaining nearly the same global structures as the original ones. It yields
rich local variations for the training of CNN. Shen et al. [16] propose
dividing the feature maps into non-overlapping patches and reordering the
patches to enhance the rotation and translation invariance. It is widely used
in neural architecture search (NAS) because it can improve the search
efficiency and reduce instability[36].

Most of the latest CNN models[37, 12, 38] integrate multiple regularization
methods. However, these methods cannot efficiently solve the overfitting
problem of forgery detection tasks because most of them do not pay attention
to the local area. Their works [15, 16] are similar to our proposed method. We
all design shuffling methods that act on blocks. However, compared with
PatchShuffle[15], we randomly select blocks for intra-block shuffling, not all
blocks. And we notice the noise patterns introduced by intra-block shuffling
and design an adversarial loss to solve it. PatchReordering [16] reorder
blocks according to specific rules while we shuffle randomly. And we propose a
position restoration method to build correlations between blocks.

##  III Proposed Method

In this section, we commence by providing an overview of the deepfake
detection problem, highlighting key observations that guide the development of
our approach. Subsequently, we present a Block Shuffling Learning (BSL)
framework, comprising four integral modules: an intra-block shuffling module,
an inter-block shuffling module, an adversarial loss module, and an position
restoration module. An illustration of the framework can be found in Fig. 2.
In Sec. III-B, we delve into the specifics of the intra-block shuffling module
and adversarial module, while Sec. III-C details the inter-block shuffling
module and position restoration module.

Figure 2: The framework of our method. Our method consists of four important
components: 1) an intra-block shuffling module for rich local representations;
2) an inter-block shuffling module that emphasizes local features; 3) an
adversarial loss module for overcoming the shuffling noise; 4) a position
restoration module for modeling the semantic relevance of local features.

###  III-A Motivation

The deepfake algorithms for face swapping can be roughly divided into three
categories: methods based on graphics [39], methods based on autoencoder [6]
and methods based on generative adversarial network (GAN) [40]. These
techniques only require a few face images to forge realistic fake images in a
short time, which is pretty deceptive. It is difficult for even human eyes to
distinguish whether the images have been manipulated [41]. As shown in Fig. 3,
face forgery technology mainly consists of three s.png to manipulate the
target face and create a fake image: 1) face detection; 2) face synthesis; 3)
image blending. It is noted that a fake image $I_{F}$ is obtained by combining
two images $I_{T}$ and $I_{S}$:

$I_{F}=F\left(I_{T},S\left(I_{T},I_{S}\right)\right)$ (1)

where $I_{T}$ is the image of the replaced face identity and $I_{S}$ is the
face image used for replacement. $S$ is a method of synthesizing two faces,
such as making the expression in $I_{S}$ similar to that in $I_{T}$. $F$ is
the method of blending the synthesized face into $I_{T}$, which usually
requires some graphics and colour transformations.

Figure 3: Process of typical face replacement.

Face forgery detection methods generally consider the traces produced by the
second and third stages in local areas of fake images [14, 13]. However, these
traces are small, varying and not easily captured by CNN. Therefore, we
propose a regularization method to help the network extract local features
more effectively in deepfake detection tasks. For the characteristics of
focusing on local areas, we divide the image into blocks of the same size and
perform random intra-block and inter-block shuffling respectively.

The utilization of intra-block shuffling introduces rich local variations
within the image. Firstly, each pixel instance within a specific image
position can be regarded as being uniformly sampled from the corresponding
image block. Consequently, weight-sharing is achieved within each block across
multiple iterations. Secondly, the random shuffling technique effectively
applies perceptual encryption to the affected regions, prompting the network
to extract distributional features while discouraging the extraction of non-
distributed features. This mechanism improves the generalization of the
network by simulating the occlusion of local areas[42, 30]. Lastly, local
changes induced by intra-block shuffling do not adversely impact the global
information of the image. The distinct representations captured within the
local and global contexts allow intra-block shuffling to play a beneficial
role in the forgery detection task.

Through the implementation of inter-block shuffling, the global structure of
the image is intentionally disrupted. The reordering of image patches enhances
the network’s translation invariance capabilities[16], thereby preventing the
forgery detection task from devolving into a mere face recognition task. Each
block instance can be viewed as being sampled uniformly from the blocks into
which the image is divided. Given that specific regions of the face image are
more likely to exhibit forgery traces, we have devised a position restoration
method to establish correlations between the blocks.

###  III-B Intra-block shuffling

Detecting one or several specific defects in a fake image can often be an easy
task. However, the challenge arises when deepfake detection methods encounter
scenarios where training and test data are generated using different deepfake
techniques. In such cases, the performance of these methods tends to
significantly deteriorate with the addition of simple transformations. To
address this limitation and enhance the generalization capabilities of the
network, we propose a novel approach involving the shuffling of the internal
structure of image blocks. By shuffling the internal structure of image
blocks, we aim to enable the network to learn more generalized features that
can effectively discriminate between genuine and fake images, even when
subjected to various transformations. However, it is important to note that
the process of intra-block shuffling introduces noise patterns that are
potentially detrimental to the detection performance. Consequently, we propose
the utilization of an adversarial loss to identify and eliminate patterns
unrelated to deepfake detection, thereby enhancing the overall effectiveness
of our approach.

####  III-B1 Pixel shuffling transformation

Intra-block shuffling methods serve to enhance the local representations of
the image while facilitating weight-sharing within each block. By applying
perceptual image encryption [43, 44, 45] through intra-block shuffling, the
network faces challenges in extracting non-distributed features from the
shuffled local regions. This observation highlights a similar role of intra-
block shuffling to occluding local areas [42], thereby promoting the network’s
generalization capabilities. Importantly, our method ensures that the local
distribution of the image remains unchanged, thereby encouraging the neural
network to extract both local statistical features and other structure-
independent features. In order to achieve this objective, we have devised a
transformation method, which is outlined as follows.

Specifically, given an input image I with a dimension of $x\times y$, we first
divide the image into blocks with a size of $s\times s$, where $x$ and $y$
should be divisible by $s$. Thus, we get $m\times n$ blocks denoted by
$B_{i,j}$, where $m$ and $n$ are the numbers of horizontal and vertical
blocks, $i$ and $j$ are the horizontal and vertical indices, respectively.
Aprilpyone et al. [46] demonstrate the ability of the neural network to learn
specific shuffling rules. Thus, our proposed method uses a random
transformation to shuffle the pixels in the block. The $k^{th}$ pixel value is
denoted by $p\left(k\right)$ in each block, where $1\leq k\leq s\times s$. Let
$\alpha$ be a random permutation vector of the integers from 1 to $s\times s$.
The new $k^{th}$ pixel value $p^{\prime}\left(k\right)$ after shuffling is
given by $p^{\prime}\left(k\right)=p\left(\alpha_{k}\right)$. Fig. 4
illustrates the process of pixel shuffling transformation.

Figure 4: Process of the pixel shuffling transformation.

This transformation is randomly applied to each image block in image $I$, and
the permutation vector $\alpha$ is regenerated each time. Then we can get a
shuffled image $I_{P}$. We can only shuffle part of blocks instead of all by
adjusting the ratio parameter $q$. The reason why we regenerate the vector
$\alpha$ every time the blocks are shuffled is that we find that if the same
vector $\alpha$ is used for shuffling transformation, the neural network can
restore the local spatial information destroyed by a single transformation
operation through learning. But our goal is to occlude local areas by
shuffling, so we discourage the network from restoring spatial information.

For deepfake detection tasks, the neural network predicts whether the input
image $I$ is real or fake. Thus we label each image using a scalar $y$ as 1 if
the input image is fake, otherwise it is 0. The classification network $C$
maps the input image $I$ to a probability variable $C_{\Theta}\left(I\right)$,
where $\Theta$ are learnable parameters in the neural network. Using the
shuffled image $I_{P}$ for training, the cross-entropy loss
$\ell_{cls}\left(\Theta\right)$ can be written as:

$\ell_{cls}\left(\Theta\right)=-\sum{y\bullet\log{\left[C_{\Theta}\left(I_{P}\right)\right]}}$
(2)

####  III-B2 Adversarial Loss

Neural networks also describe random noise and error while learning the
underlying data distribution [47]. As shown in Fig. 5, the shuffling
transformation introduces noise patterns. The features learned by the neural
network from these noise patterns mislead the detection task, which increases
the false positive rate in the results. Adversarial loss is efficient for
preserving domain-invariant patterns and rejecting domain-specific patterns
[48, 49, 50]. Thus, we propose an adversarial loss to reject patterns
unrelated to deepfake detection to prevent the network from being affected by
noise patterns introduced by shuffling.

Figure 5: Examples for original images (left), intra-block shuffled images
(right) and marked matrixs (middle).

Given image $I$ and its corresponding shuffled version $I_{P}$, we get a
$m\times n$ matrix $P$. $P_{i,j}$ represents the element in the $i^{th}$ row
and $j^{th}$ column corresponding to the block $B_{i,j}$ in the image $I_{P}$.
$P_{i,j}$ is 1 if the image block $B_{i,j}$ is shuffled, otherwise it is 0. An
adversarial network $A$ can be added as a new branch in the framework to
output a matrix with a size of $m\times n$ to judge whether image blocks are
shuffled or not. The intra-block adversarial loss
$\ell_{adv}\left(\Theta^{\left[1,\ u\right]},\Psi\right)$ can be computed as:

$\ell_{adv}\left(\Theta^{\left[1,\
u\right]},\Psi\right)=-\sum{P\bullet\log{\left[A_{\Psi}\left(C_{\Theta^{\left[1,\
u\right]}}\left(I_{P}\right)\right)\right]}}$ (3)

where $\Theta^{\left[1,\ u\right]}$ is the learnable parameters from the
$1^{st}$ layer to $u^{th}$ layer in the neural network, $C_{\Theta^{\left[1,\
u\right]}}\left(I_{P}\right)$ is the output features of the $u^{th}$ layer in
the neural network, $A$ represents a region alignment network work on the
output features, and $\Psi$ is its parameters. As shown in Fig. 6, we use a
PixleShuffle[51] instead of upscaling to align the input features with the
mark matrix $P$. The features are processed by $1\times 1$ convolution to
obtain outputs with one channel. Then the outputs are handled by a ReLU to get
a matrix with the size of $m\times n$.

Figure 6: The structure of adversarial branch.

###  III-C Inter-block shuffling

In deepfake detection tasks, the focus is primarily directed towards local
features rather than global information. This emphasis is attributed to the
fact that facial images tend to exhibit a consistent global structure, while
the significant differences between real and fake facial images typically
manifest in local regions. Therefore, the accurate identification of specific
differences within local areas plays a important role in effective deepfake
detection. Moreover, an undue emphasis on global information by neural
networks may lead to a degradation of the deepfake detection process,
converging it towards face recognition. To address this problem, we propose a
novel approach that disrupts the global structure through inter-block
shuffling, so that the network can focus on the extraction of features from
local areas. By inducing changes in the locations of image patches, we
establish a function that indirectly facilitates weight-sharing among local
regions and enhances the translation invariance capabilities of the network.
Additionally, we introduce a position restoration method to preserve the
correlation of local features without requiring labels. By implementing these
strategies, our proposed method aims to optimize deepfake detection by
augmenting the focus on local features while mitigating the potential for
misclassification stemming from an overemphasis on global information.

####  III-C1 Block shuffling transformation

Algorithm 1 Block Shuffling Procedure.

Input: Input image $I$; Block size $s$; The numbers of horizontal blocks $m$;
The numbers of horizontal blocks $n$; Blocks of image $B$; intra-shuffling
ratio $q_{a}$; inter-shuffling ratio $q_{b}$.

Output: Shuffled image $I^{*}$; Matrix $P$ marks the intra-shuffling of the
blocks; Matrix $M$ marks the inter-shuffling of the blocks.

$q_{1}\leftarrow Rand(0,1)$;

if $q_{1}>q_{b}$ then

generate the matrix $M$;

inter-shuffle image $I$;

end if

for $i\leftarrow 1$ to $m$ do

for $j\leftarrow 1$ to $n$ do

$q_{2}\leftarrow Rand(0,1)$;

if $q_{2}>q_{a}$ then

intra-shuffle block $B_{i,j}$;

$P_{i,j}\leftarrow 1$;

else

$P_{i,j}\leftarrow 0$;

end if

end for

end for

In Sec. III-B1 we have divided image $I$ into blocks with a size of $m\times
n$. As shown in Fig. 7, we use a similar method to arrange and shuffle the
image blocks to obtain a new image block arrangement. Then we can get a
shuffled image $I_{B}$. The new image block $B_{i,j}^{\prime}$ after shuffling
is given by $B_{i,j}^{\prime}=B_{\beta\left(i,j\right)}$, where
$\beta\left(i,j\right)$ represents the original coordinates in image $I$
before being shuffled. Using two shuffling methods for training, the shuffled
algorithm flow is shown in Algorithm 1. To conveniently obtain the label
matrix $P$ of intra-block shuffling, we first perform inter-block shuffling
and then intra-block shuffling to get the shuffled image $I^{*}$. The cross-
entropy loss $\ell_{cls}\left(\Theta\right)$ can be written as:

$\ell_{cls}\left(\Theta\right)=-\sum{y\bullet\log{\left[C_{\Theta}\left(I^{*}\right)\right]}}$
(4)

Figure 7: Example images for original images (left), inter-block shuffled
images (right) and the coordinates of image blocks (bottom).

####  III-C2 Block position restoration

Some discernible signs of forgery consistently emerge in specific regions of
manipulated images, such as inconsistencies in eye color, the absence of
nostril shadows, or misalignments in teeth structures. These localized
imperfections represent important cues that neural networks utilize to
identify fake images. Therefore, we propose the implementation of a position
restoration algorithm to preserve the correlation of these distinctive
attributes[52, 53]. For the human observer, even when presented with only a
small portion of a facial image, it is often possible to ascertain its origin
within the face, such as the eye or nose region, and subsequently infer its
approximate position within the complete facial image. In a similar vein, our
objective is to enable the neural network to acquire this intuitive
understanding and extract more effective features for the detection of
deepfake images. By this approach, we aim to imbue the neural network with a
human-like understanding, allowing it to discern and localize significant
facial components, thus enhancing its ability to detect deepfake images more
accurately.

Given image $I$ and its corresponding inter-block shuffled version $I^{*}$, we
get a $2\times m\times n$ matrix $M$. $M_{0,i,j}$ and $M_{1,i,j}$ are the row
coordinates and column coordinates of the coordinates $\beta\left(i,j\right)$,
respectively. An inter-block position restoration branch $R$ extracts the
output features of the $v^{th}$ layer of the classification network and
outputs the original coordinates corresponding to each image block. The inter-
block restoration loss $\ell_{loc}\left(\Theta^{\left[1,v\right]},\Phi\right)$
can be expressed as:

$\ell_{loc}\left(\Theta^{\left[1,v\right]},\Phi\right)=-\sum\left|M-R_{\Phi}\left(C_{\Theta^{\left[1,v\right]}}\left(I^{*}\right)\right)\right|_{1}$
(5)

where $\Phi$ are the parameters in the branch $R$. As shown in Fig. 8, the
branch $R$ consists of a PixleShuffle, a $1\times 1$ convolution and a
Hardtanh to get a map with the size of $2\times m\times n$.

Figure 8: The structure of position restoration branch.

###  III-D Loss function

Our framework completes the deepfake detection task through an end-to-end
framework. As shown in Algorithm 2, firstly, the input image $I$ enters the
shuffling module to get a shuffled image $I^{*}$, a mark matrix $P$ records
the intra-shuffling information, and a coordinate matrix $M$ records the
inter-shuffling information. The shuffled image $I^{*}$ input classification
network $C$ and get the result of forgery detection. The $u^{t}h$ and $v^{t}h$
feature layers of network $C$ are input to the adversarial branch $A$ and the
restoration branch $R$, respectively. These branches help to suppress random
noise and learn unique features based on the correlation of various blocks in
the face image. Therefore, the neural network can extract a set of image
features with good discrimination and generalization. Our method can work on
most CNNs with almost no additional computational cost. Specifically, we want
to minimize the following objective:

$\begin{split}\ell\left(\Theta,\Psi,\Phi\right)=\ell_{cls}\left(\Theta\right)+\alpha\ell_{adv}\left(\Theta^{\left[1,\
u\right]},\Psi\right)\\\
+\beta\ell_{loc}\left(\Theta^{\left[1,v\right]},\Phi\right)\end{split}$ (6)

where $\alpha$ and $\beta$ are the balancing hyperparameters.

Algorithm 2 Block Shuffling learning framework Procedure.

Input: Training set
${\\{\left(I^{\left(n\right)},y^{\left(n\right)}\right)\\}}^{N}_{n=1}$; the
learning rate $\gamma$; the maximum number of iterations $T$.

Initialization: $\mathbf{w}\leftarrow 0$; $\mathbf{u}\leftarrow 0$;
$\mathbf{v}\leftarrow 0$; $t\leftarrow 0$.

repeat

for $n\leftarrow 1$ to $N$ do

generate shuffled image $I^{*}$, matrix $P$ and $M$ based on Algorithm 1

$\mathbf{w}\leftarrow\mathbf{w}+y^{\left(n\right)}\bullet\log\left[C_{\Theta}\left(I^{*}\right)\right]$

$\mathbf{u}\leftarrow\mathbf{u}+P\bullet\log{\left[A_{\Psi}\left(C_{\Theta^{\left[1,\
u\right]}}\left(I^{*}\right)\right)\right]}$

$\mathbf{v}\leftarrow\mathbf{v}+\left|M-R_{\Phi}\left(C_{\Theta^{\left[1,v\right]}}\left(I^{*}\right)\right)\right|_{1}$

end for

$\Theta\leftarrow\Theta-\gamma\nabla\left(\mathbf{w}+\alpha\mathbf{u}+\beta\mathbf{v}\right)$

$\Psi\leftarrow\Psi-\gamma\nabla\alpha\mathbf{u}$

$\Phi\leftarrow\Phi-\gamma\nabla\beta\mathbf{v}$

$t\leftarrow t+1$

until $t=T$

##  IV Experiments

In this section, we first introduce the experimental setups and then use a
large number of experimental results to show the superiority of our method.

###  IV-A Experimental settings

Datasets: We conduct experiments on FaceForensics++
(FF++)111https://github.com/ondyari/FaceForensics [54], Celeb-
DF(v2)222https://github.com/yuezunli/celeb-deepfakeforensics [8] and
DeeperForensics-1.0
(DF1.0)333https://github.com/EndlessSora/DeeperForensics-1.0 [55]. FF++ is a
large-scale video dataset consisting of 1,000 original videos with real faces
that have been manipulated by four deepfake methods [6, 56, 39, 57]. On this
basis, FF++ has two subsets FaceShifter [58] and Deep Fake Detection Dataset
provided by Google & JigSaw444https://ai.googleblog.com/2019/09/contributing-
data-to-deepfake-detection.html. The use of these two subsets in this paper is
described separately. Celeb-DF(v2) includes 590 original videos collected from
YouTube and 5639 deepfake videos generated by an improved deepfake synthesis
algorithm, which brings better visual quality. DF1.0 contains 60000 videos,
where fake videos are generated by their DF-VAE method. The dataset performs
more challenging benchmarks with larger scale and higher diversity by applying
a wide range of real-world perturbations on videos.

Evaluation Metrics: We mainly apply the Accuracy score (ACC) and the Area
Under the RoC Curve (AUC) as our evaluation metrics, which are commonly used
in deepfake detection tasks [21, 24, 31, 7, 30, 13].

Implementation details: For all real and fake videos, we use a state-of-the-
art efficient face detector BlazeFace [59] to detect the faces of each frame
and resize the facial images as inputs with a size of $224\times 224$. Random
horizontal flip is applied for data augmentation. For intra-block shuffling,
we set the block size $s$ to 16. We set the ratio parameter $q\sim
U\left(0.4,0.6\right)$ of intra-block shuffling. For inter-block shuffling, we
set the block size $s$ to 32. We apply the inter-block shuffling to all blocks
of the image. We set the hyperparameters $\alpha=1$ and $\beta=1$ by default.
We adopt Xception [12] as the backbone network of our BSL framework. We use
Adam [60] for optimization with a learning rate of $1e^{-4}$ and weight decay
of $1e^{-6}$. We train our models on an Nvidia A10 GPU with a batch size of
64.

###  IV-B Evaluation of Intra-datasets

Detecting fake images using the same forgery method as the training set is the
basic ability of the deepfake detection model. FF++ is the most widely used
dataset to test the basic capabilities of deepfake detection algorithms. FF++
contains multiple video quality versions. We train and test the HQ (c23) and
the LQ (c40) versions. As the results shown in Table I, our method achieves
state-of-the-art performance on the HQ version. However, since M2TR [31]
incorporates a multi-model and multi-scale feature extractor for the
generalization of image compression, our method performs 0.18% lower than M2TR
in the LQ version. In contrast, our method achieves similar results with
almost no additional computation. Moreover, our method does not conflict with
M2TR and can work together with the network. As shown in Table II, our
proposed method achieves good performance on all experimental datasets, which
demonstrates the effectiveness of the method in the face of various forgery
methods.

TABLE I: Quantitative frame-level detection results (AUC (%)) on FF++ dataset with the LQ version and the HQ version, respectively. Results of CFFs are cited directly from [61] and other methods are cited directly from [13]. The best results are marked as bold. Methods | LQ | HQ
---|---|---
Xception555https://github.com/tstandley/Xception-PyTorch [12] | 89.30 | 96.30
DSP-FWA666https://github.com/yuezunli/DSP-FWA [24] | 92.49 | 99.28
M2TR777https://github.com/wangjk666/M2TR-Multi-modal-Multi-scale-Transformers-for-Deepfake-Detection [31] | 94.25 | 99.05
SRM888https://github.com/crywang/face-forgery-detection [26] | 91.36 | 98.70
Multi-att999https://github.com/yoctta/multiple-attention [13] | 87.26 | 98.97
CFFs101010https://github.com/botianzhe/CFFExtractor [61] | 90.35 | 97.63
Ours(Xception) | 93.13 | 99.95
TABLE II: Intra-datasets results(AUC (%)) on six subsets of FF++, Celeb-DF(v2) and DF1.0 Methods | ACC | AUC
---|---|---
Deepfakes | 99.09 | 99.95
Face2Face | 99.90 | 99.99
FaceSwap | 99.19 | 99.89
NeuralTextures | 95.63 | 97.59
FaceShifter | 98.12 | 99.54
Google & JigSaw | 95.15 | 98.64
Celeb-DF | 98.26 | 99.72
DF1.0 | 99.99 | 99.99

###  IV-C Cross-dataset evaluation

It is possible for a deepfake detection model to encounter fake methods that
it did not encounter during the training phase. To evaluate the performance of
the detection method in this case, we trained the model on the HQ version of
FF++ and tested the model on Celeb-DF(v2). The results are shown in Table III.
Our method outperforms most of the existing methods, especially better than
our model’s backbone network Xception. Although Two Branch [62] is slightly
higher than ours in cross-dataset performance, its performance on the original
dataset FF++ is far inferior to ours.

TABLE III: Cross-dataset evaluation(AUC(%)) on Celeb-DF(v2) by training on the HQ version of FF++. Results of some other methods are cited directly from [13]. The best results are marked as bold. Methods | FF++ | Celeb-DF(v2)
---|---|---
Two-stream [21] | 70.10 | 53.80
MesoNet111111https://github.com/HongguLiu/MesoNet-Pytorch [22] | 84.70 | 54.80
Mesolnception4 [22] | 83.00 | 53.60
FWA [24] | 80.10 | 56.90
Xception-raw [12] | 99.70 | 48.20
Xception-c23 [12] | 99.70 | 65.30
Xception-c40 [12] | 95.50 | 65.50
Capsule121212https://github.com/nii-yamagishilab/Capsule-Forensics [23] | 96.60 | 57.50
DSP-FWA [24] | 93.00 | 64.60
Two Branch [62] | 93.18 | 73.41
Multi-att [13] | 99.80 | 67.44
Ours(Xception) | 99.72 | 69.10

In order to more intuitively demonstrate the generalization of our method
under different datasets, we train the model on each of the four methods on
the HQ version of FF++ dataset and evaluate its performance on CelebDF(v2) and
DF1.0. This experimental setting is very challenging. Each model is trained
with only one type of fake sample, and the real samples and fake methods in
the test sets are not repeated with the training sets. The results are shown
in Table IV. Our method achieves the best results in most experiments. Among
them, Face X-ray [14] and SLADD [63] both add additional training data, and
SRM [26] and F3Net [27] rely on high-frequency components of images to detect
deepfake. These settings can effectively improve generalization performance,
so these methods have achieved better results than ours in some experiments.
But our proposed method can also be combined with the above methods to achieve
better performance.

TABLE IV: Cross-dataset evaluation(AUC(%)) on CelebDF(v2) and DF1.0 by training on each of the four methods on the HQ version of FF++. The first row denotes the training data, and the second row denotes the corresponding test data. Results of some other methods are cited directly from [63]. The best results are marked as bold. Methods | DF | F2F | FS | NT
---|---|---|---|---
CelebDF | DF1.0 | CelebDF | DF1.0 | CelebDF | DF1.0 | CelebDF | DF1.0
Xception [12] | 68.1 | 61.7 | 59.8 | 74.5 | 60.1 | 60.5 | 62.5 | 83.8
Face X-ray131313https://github.com/Daisy-Zhang/Face-X-ray [14] | 55.4 | 66.8 | 68.4 | 76.6 | 69.7 | 79.5 | 70.3 | 86.6
F3Net141414https://github.com/Leminhbinh0209/F3Net [27] | 66.4 | 65.8 | 65.4 | 76.1 | 63.6 | 65.1 | 68.9 | 93.2
RFM151515https://github.com/crywang/RFM [30] | 72.3 | 71.7 | 66.3 | 73.2 | 59.1 | 71.4 | 60.0 | 84.6
SRM [26] | 65.0 | 72.0 | 69.3 | 77.5 | 64.3 | 77.1 | 65.1 | 93.6
SLADD161616https://github.com/liangchen527/SLADD [63] | 73.0 | 74.2 | 78.1 | 78.6 | 80.0 | 69.5 | 75.9 | 88.9
Ours(Xception) | 73.7 | 82.0 | 67.1 | 92.7 | 62.6 | 76.9 | 69.8 | 87.0

###  IV-D Cross video quality evaluation

The quality of the video will be affected when it spreads in the network, and
this quality degradation may affect the performance of the deepfake detection
model. FF++ provides video versions compressed under H264 video encoding by
ffmpeg software. The HQ version and the LQ version correspond to the c23 and
c40 compression settings respectively. We simulate the video quality
degradation experienced by deepfake detection models by training the HQ
version and testing on the LQ version. The results are listed in Table VI. We
reproduce three existing methods and compare with our method. Although the
results illustrate that the metrics of all models have dropped significantly
and are lower than the results trained with the LQ version. Our method shows
better generalization than SRM [26] using frequency domain information and
DSP-FWA [24] using multi-scale design in the case of video compression.

TABLE V:  Cross video quality evaluation(ACC(%) and AUC(%)) on the LQ version of FF++ by training on the HQ version. The best results are marked as bold. Methods | ACC | AUC
---|---|---
Xception [12] | 83.45 | 93.13
SRM [26] | 84.05 | 91.33
DSP-FWA [24] | 85.98 | 92.84
Ours(Xception) | 87.41 | 95.89
TABLE VI:  Cross video quality evaluation(ACC(%) and AUC(%)) on the LQ version of FF++ by training on the HQ version. The best results are marked as bold. Methods | Attack | Random | PD | WD+SR | ComDefend | Con | Ours
---|---|---|---|---|---|---|---
Clean | 100 | 99.68 | 83.50 | 86.34 | 93.80 | 98.74 | 97.70
FGSM | 33.84 | 53.52 | 53.86 | 60.98 | 56.53 | 97.90 | 97.52
BIM | 0.06 | 17.72 | 60.76 | 69.22 | 44.15 | 92.00 | 95.35
C&W | 1.90 | 93.32 | 84.81 | 87.50 | 90.39 | 96.50 | 97.00
DeepFool | 0.68 | 90.94 | 84.32 | 87.36 | 89.09 | 96.48 | 96.21
PGD | 0.02 | 18.50 | 66.44 | 79.42 | 50.91 | 93.14 | 94.82
MI-FGSM | 0.04 | 8.96 | 39.44 | 65.08 | 34.17 | 87.00 | 90.46
DI-FGSM | 1.30 | 2.80 | 60.68 | 69.00 | 41.77 | 65.44 | 70.34
MD-FGSM | 1.18 | 2.28 | 21.10 | 29.62 | 20.53 | 75.26 | 81.26
APGD-CE | 0.00 | 18.14 | 69.13 | 70.68 | 41.40 | 96.94 | 97.18
APGD-DLR | 0.00 | 66.38 | 77.47 | 77.64 | 45.48 | 96.78 | 97.02

###  IV-E Resizing and blurring evaluation

We evaluate the performance of the forgery detection method under image
transformations such as resizing and blurring, which are the most common
transformations that images spread across the Internet. We chose Celeb-DF(v2)
as the experimental data set, because the high-quality images provided by
Celeb-DF(v2) are more suitable for testing the effect of the forgery detection
model after the image quality is compromised. We reproduce three existing
methods and compare with our method. We train models on Celeb-DF(v2) with an
input image size of 224. Fig. 9 shows the AUC score compared with current
methods under various input image sizes of 160, 112, 80 and 56, respectively.
Fig. 10 shows the AUC score under various Gaussian kernel sizes of 3, 5, 7 and
9, respectively. Our method has achieved the best results as a whole, and as
the intensity of transformation increases, we achieve more accurate results
than other methods. DSP-FWA [24] uses a dual spatial pyramid strategy on both
image and feature level to solve multi-scale problems, so it achieves good
results in this experiments, while our method uses a simple network structure
to achieve slightly higher results.

Figure 9: Resizing evaluation (AUC(%)) on Celeb-DF v2 dataset with five
different input image sizes. Figure 10: Blurring evaluation (AUC(%)) on Celeb-
DF v2 dataset with five different Gaussian blur kernel sizes.

##  V Ablation Study

###  V-A Effectiveness of block shuffling

To evaluate the effectiveness of different modules, we combine them and use
evaluations, respectively. All models are trained on the HQ version of FF++
dataset and tested on the HQ version and the LQ version of FF++ dataset. This
setting separately evaluates the basic detection ability and generalization
ability in the face of video quality degradation. The results are listed in
Table VII.

Compared with model 1 (baseline Xception), model 2 (Xception with Intra-block
shuffling) achieves a 1.69% improvement on the HQ version but a 0.63% drop on
the LQ version. This is because although intra-block shuffling enhances
detection, the introduced noise patterns limit the generalization ability.
Model 3 (addition of adversarial learning) dramatically enhances performance
and surpasses model 1 because it overcomes the overfitting caused by noise
patterns due to the adversarial loss. Model 4 (addition of the Inter-block
shuffling) saw a 0.77% drop in performance on the HQ version because relying
solely on the features extracted within the block significantly limits the
detection ability of the model. We enhance local features by position
restoration to build correlations in local regions. Model 5 (addition of the
Block position restoration) achieves the best performance, and the ACC score
increases by 2.91% on the HQ version and 3.96% on the LQ version.

TABLE VII:  Ablation study of the proposed BSL. We compare BSL by gradually removing each module. ID | Intra | Adv | Inter | Restore | HQ | LQ
---|---|---|---|---|---|---
1 |  |  |  |  | 95.73 | 83.45
2 | ✓ |  |  |  | 97.42 | 82.82
3 | ✓ | ✓ |  |  | 97.69 | 85.29
4 | ✓ | ✓ | ✓ |  | 96.92 | 85.64
5 | ✓ | ✓ | ✓ | ✓ | 98.64 | 87.41
Figure 11: Examples for intact images (top), inter-block shuffling images
(middle) and restore images (bottom) from ff++ dataset.

###  V-B Effectiveness of block position restoration

The Block position restoration is designed to help the network model the
correlations between local features. The distance between blocks before and
after restoration represents the effect of correlation modelling. We count the
restoration results of the model trained on the HQ version of FF++ dataset. As
shown in the Fig. 12, the experimental results show that the distance before
and after restoration of more than 80% of blocks is less than or equal to 1.
This means that most blocks are back in their original positions after
restoration. Fig. 11 intuitively illustrates the effect of inter-block
shuffling and restoration.

Figure 12: Evaluation of Block position restoration on the FF++ dataset. The
distance between blocks before and after restoration is counted.

###  V-C Effectiveness of block size for intra-block shuffling

To evaluate the impact of different block sizes, we count the forgery
detection results and the results of the adversarial branch at different sizes
on the HQ version of the ff dataset. Since the adversarial branch predicts
whether each block is shuffled, we adopt ACC to measure how well the
adversarial branch works. The results are shown in Fig. 13. For fake detection
results, the detection performance is the best when the block size equals 16.
But compared with other block sizes, the detection performance drops
significantly when the block size equals 32. The choice of block size for
intra-block shuffling is subtle. Too large a block causes intra-block shuffles
to behave like global shuffles, which is disastrous for tasks such as forgery
detection that value local regions. For the adversarial branch, the
adversarial effect decreases slightly as the block size decreases. Because the
smaller the blocks are, the more likely they are completely in the solid
colour area (e.g. hair, forehead), and it is difficult to judge whether these
blocks are shuffled or not.

Figure 13: Evaluation of different block sizes for intra-block shuffling on
the FF++ dataset with the HQ version. The blue line shows performance metrics
for forgery detection, the red line shows performance metrics for adversarial
effects.

###  V-D Effectiveness of block size for inter-block shuffling

The block size of inter-block shuffling directly affects the granularity of
local features and the difficulty of position restoration. The smaller block
size, the less information it contains, and it is more challenging to build
correlations between blocks. We count the forgery detection and restoration
results of the models at different block sizes on the HQ version of the ff
dataset. We use ACC to evaluate the detection performance and use the average
distance difference before and after restoration to evaluate the restoration
performance. To intuitively represent the restoration performance at different
block sizes, we divide the distance difference by the block size to get the
relative distance difference. The smaller the value, the better the
restoration.

As shown in Fig. 14, the experimental results show that the detection and
reduction performance are the best when the block size equals 32. The
detection performance and reduction performance decrease significantly as the
block size decreases. Because the small block contains very little high-level
feature information, almost only low-level colour information can be
extracted, which significantly increases the difficulty of restoration.
Meanwhile, the result of restoration affects the detection performance, and
the detection performance also decreases.

Figure 14: Evaluation of different block sizes for inter-block shuffling on
the FF++ dataset with the HQ version. The blue line shows performance metrics
for forgery detection, the red line shows performance metrics for restoration
effects.

###  V-E Effectiveness of hyperparameters

To evaluate the impact of hyperparameters $\alpha$ and $\beta$, we train on
the HQ version of the ff++ dataset and test the HQ version and the LQ version
under different parameter values. Fig. 15 and Fig. 16 show the ACC results
tested on the HQ and LQ version datasets, respectively, and the best results
are achieved when $\alpha=1,\beta=1$. When $\alpha$ and $\beta$ take large
values, the centre of gravity of the network tends to adversarial learning and
position restoration learning, two auxiliary tasks, so the forgery detection
effect is not good. When $\alpha$ and $\beta$ take small values, the network
degenerates into the backbone network, and the result of forgery detection is
also close to the performance of the backbone network. When the
hyperparameters take values 1 and 0.1, the model shows similar performance,
with the highest performance when both hyperparameters are 1.

Figure 15: Evaluation (ACC) on the HQ version of FF++ dataset with different
hyperparameters. Figure 16: Cross video quality evaluation (ACC) on the LQ
version of FF++ dataset by training on the HQ version with different
hyperparameters.

###  V-F Universality of BSL with various backbones

In this section, we evaluate the versatility of our method. The above
experiments are all based on Xception, and we also perform the same experiment
on ResNet50171717https://github.com/Lornatang/ResNet-PyTorch [37] and
EfficientNet181818https://github.com/lukemelas/EfficientNet-PyTorch [38]. The
results listed in Table VIII show that our proposed method plays an active
role in various CNNs. The performance of the networks trained based on our
method is significantly improved.

TABLE VIII:  Quantitative frame-level detection results (AUC (%)) on FF++ dataset. The best results are marked as bold. Methods | HQ | LQ
---|---|---
ResNet50 [37] | 96.04 | 80.90
Ours(ResNet50) | 97.60 | 86.79
EfficientNet-B4 [38] | 96.54 | 76.93
Ours(EfficientNet-B4) | 97.92 | 85.61

##  VI Conclusion

This paper discusses the significance of addressing the overfitting issue that
arises within local regions during deepfake detection tasks, while also
exploring its potential causes. Our focus has been on weight-sharing
regularization as a means of investigating deepfake detection, with the
introduction of the random shuffling method. To achieve weight-sharing across
different dimensions indirectly, we propose a block shuffling learning
framework that comprises an intra-block shuffling module and an inter-block
shuffling module. The former module enhances network generalization by
randomly occluding local areas and employs adversarial loss to suppress noise
modes. On the other hand, the latter module aims to disrupt the global
structure, encouraging the network to learn local features. Additionally, it
employs position restoration algorithms to model the semantic correlation
between local features. Our proposed method has demonstrated substantial
improvements across a range of extensive metrics, as validated by
comprehensive experimental results. It is important to note that our current
approach primarily focuses on the spatial location of pixels within the image
and does not involve alterations to pixel values. Moving forward, future work
will consider augmenting network generalization by incorporating pixel-level
random perturbations and exploring methods for reconstructing the original
images during the training phase. These avenues of investigation hold promise
for further enhancing the effectiveness of our proposed methodology.

## References

  * [1] Sensity, “The state of deepfakes 2020: Updates on statistics and trends,” https://sensity.ai/reports/, 2021.
  * [2] Y. Nirkin, Y. Keller, and T. Hassner, “Fsgan: Subject agnostic face swapping and reenactment,” _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 7183–7192, 2019.
  * [3] A. Siarohin, S. Lathuilière, S. Tulyakov, E. Ricci, and N. Sebe, “First order motion model for image animation,” _ArXiv_ , vol. abs/2003.00196, 2019\.
  * [4] Y. Choi, M.-J. Choi, M. S. Kim, J.-W. Ha, S. Kim, and J. Choo, “Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,” _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 8789–8797, 2018.
  * [5] T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 4396–4405, 2019.
  * [6] DeepFakes, “Deepfakes github,” http://www.github.com/deepfakes/faceswap, 2017.
  * [7] M. Du, S. K. Pentyala, Y. Li, and X. Hu, “Towards generalizable deepfake detection with locality-aware autoencoder,” in _CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020_ , M. d’Aquin, S. Dietze, C. Hauff, E. Curry, and P. Cudré-Mauroux, Eds. ACM, 2020, pp. 325–334.
  * [8] Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-df: A large-scale challenging dataset for deepfake forensics,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_. IEEE, 2020, pp. 3204–3213.
  * [9] A. Pumarola, A. Agudo, A. M. Martinez, A. Sanfeliu, and F. Moreno-Noguer, “Ganimation: Anatomically-aware facial animation from a single image,” _Computer vision - ECCV … : … European Conference on Computer Vision : proceedings. European Conference on Computer Vision_ , vol. 11214, pp. 835–851, 2018.
  * [10] T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 4396–4405.
  * [11] M. S. Rana, M. N. Nobi, B. Murali, and A. H. Sung, “Deepfake detection: A systematic literature review,” _IEEE Access_ , vol. 10, pp. 25 494–25 513, 2022.
  * [12] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017_. IEEE Computer Society, 2017, pp. 1800–1807.
  * [13] H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang, and N. Yu, “Multi-attentional deepfake detection,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2185–2194, 2021.
  * [14] L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen, and B. Guo, “Face x-ray for more general face forgery detection,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_. IEEE, 2020, pp. 5000–5009.
  * [15] G. Kang, X. Dong, L. Zheng, and Y. Yang, “Patchshuffle regularization,” _CoRR_ , vol. abs/1707.07103, 2017.
  * [16] X. Shen, X. Tian, S. Sun, and D. Tao, “Patch reordering: A novelway to achieve rotation and translation invariance in convolutional neural networks,” in _AAAI_ , 2017.
  * [17] J. J. Fridrich and J. Kodovský, “Rich models for steganalysis of digital images,” _IEEE Transactions on Information Forensics and Security_ , vol. 7, pp. 868–882, 2012.
  * [18] G. Chierchia, S. Parrilli, G. Poggi, L. Verdoliva, and C. Sansone, “Prnu-based detection of small-size image forgeries,” _2011 17th International Conference on Digital Signal Processing (DSP)_ , pp. 1–6, 2011.
  * [19] C. Chen, S. McCloskey, and J. Yu, “Focus manipulation detection via photometric histogram analysis,” in _2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018_. IEEE Computer Society, 2018, pp. 1674–1682.
  * [20] J. Lukás, J. J. Fridrich, and M. Goljan, “Detecting digital image forgeries using sensor pattern noise,” in _Electronic Imaging_ , 2006.
  * [21] P. Zhou, X. Han, V. I. Morariu, and L. S. Davis, “Two-stream neural networks for tampered face detection,” _2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , pp. 1831–1839, 2017.
  * [22] D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “Mesonet: a compact facial video forgery detection network,” _2018 IEEE International Workshop on Information Forensics and Security (WIFS)_ , pp. 1–7, 2018.
  * [23] H. H. Nguyen, J. Yamagishi, and I. Echizen, “Capsule-forensics: Using capsule networks to detect forged images and videos,” in _IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019_. IEEE, 2019, pp. 2307–2311.
  * [24] Y. Li and S. Lyu, “Exposing deepfake videos by detecting face warping artifacts,” _2019 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , 2019.
  * [25] R. Durall, M. Keuper, F.-J. Pfreundt, and J. Keuper, “Unmasking deepfakes with simple features,” _ArXiv_ , vol. abs/1911.00686, 2019.
  * [26] Y. Luo, Y. Zhang, J. Yan, and W. Liu, “Generalizing face forgery detection with high-frequency features,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 16 312–16 321, 2021.
  * [27] Y. Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao, “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in _ECCV_ , 2020.
  * [28] S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” in _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, Eds., 2017, pp. 3856–3866.
  * [29] M. Hong, S. Li, Y. Yang, F. Zhu, Q. Zhao, and L. Lu, “Sspnet: Scale selection pyramid network for tiny person detection from uav images,” _IEEE Geoscience and Remote Sensing Letters_ , pp. 1–5, 2021.
  * [30] C. Wang and W. Deng, “Representative forgery mining for fake face detection,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 14 918–14 927, 2021.
  * [31] J. Wang, Z. Wu, J. Chen, and Y. Jiang, “M2tr: Multi-modal multi-scale transformers for deepfake detection,” _ArXiv_ , vol. abs/2104.09770, 2021\.
  * [32] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” _J. Mach. Learn. Res._ , vol. 15, pp. 1929–1958, 2014.
  * [33] M. D. Zeiler and R. Fergus, “Stochastic pooling for regularization of deep convolutional neural networks,” _CoRR_ , vol. abs/1301.3557, 2013.
  * [34] S. J. Nowlan and G. E. Hinton, “Simplifying neural networks by soft weight-sharing,” _Neural Computation_ , vol. 4, pp. 473–493, 1992.
  * [35] S. Ravanbakhsh, J. G. Schneider, and B. Póczos, “Deep learning with sets and point clouds,” _ArXiv_ , vol. abs/1611.04500, 2017.
  * [36] L. Xie, X. Chen, K. Bi, L. Wei, Y. Xu, Z. Chen, L. Wang, A. Xiao, J. Chang, X. Zhang, and Q. Tian, “Weight-sharing neural architecture search: A battle to shrink the optimization gap,” _ACM Computing Surveys (CSUR)_ , vol. 54, pp. 1 – 37, 2022.
  * [37] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_. IEEE Computer Society, 2016, pp. 770–778.
  * [38] M. Tan and Q. V. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” _ArXiv_ , vol. abs/1905.11946, 2019.
  * [39] FaceSwap, “Faceswap github,” http://www.github.com/MarekKowalski/FaceSwap, 2016.
  * [40] Faceswap-GAN, “Faceswap-gan github,” https://github.com/shaoanlu/faceswap-GAN, 2020.
  * [41] Nvidia, “This person does not exist,” https://thispersondoesnotexist.com, 2019.
  * [42] Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, pp. 13 001–13 008, Apr. 2020.
  * [43] T. Chuman, W. Sirichotedumrong, and H. Kiya, “Encryption-then-compression systems using grayscale-based image encryption for jpeg images,” _IEEE Transactions on Information Forensics and Security_ , vol. 14, pp. 1515–1525, 2019\.
  * [44] W. Sirichotedumrong and H. Kiya, “Grayscale-based block scrambling image encryption using ycbcr color space for encryption-then-compression systems,” _APSIPA Transactions on Signal and Information Processing_ , vol. 8, 2019\.
  * [45] W. Sirichotedumrong, Y. Kinoshita, and H. Kiya, “Pixel-based image encryption without key management for privacy-preserving deep neural networks,” _IEEE Access_ , vol. 7, pp. 177 844–177 855, 2019.
  * [46] M. Aprilpyone and H. Kiya, “Encryption inspired adversarial defense for visual classification,” _2020 IEEE International Conference on Image Processing (ICIP)_ , pp. 1681–1685, 2020.
  * [47] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning (still) requires rethinking generalization,” _Communications of the ACM_ , vol. 64, pp. 107 – 115, 2021.
  * [48] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2962–2971, 2017.
  * [49] H. Liu, M. Long, J. Wang, and M. I. Jordan, “Transferable adversarial training: A general approach to adapting deep classifiers,” in _ICML_ , 2019\.
  * [50] X. Chen, S. Wang, M. Long, and J. Wang, “Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation,” in _ICML_ , 2019.
  * [51] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 1874–1883, 2016.
  * [52] M. Noroozi and P. Favaro, “Unsupervised learning of visual representations by solving jigsaw puzzles,” in _ECCV_ , 2016.
  * [53] Y. Chen, Y. Bai, W. Zhang, and T. Mei, “Destruction and construction learning for fine-grained image recognition,” in _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_. Computer Vision Foundation / IEEE, 2019, pp. 5157–5166.
  * [54] A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics++: Learning to detect manipulated facial images,” in _2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019_. IEEE, 2019, pp. 1–11.
  * [55] L. Jiang, W. Wu, R. Li, C. Qian, and C. C. Loy, “Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection,” _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2886–2895, 2020.
  * [56] J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2face: Real-time face capture and reenactment of RGB videos,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_. IEEE Computer Society, 2016, pp. 2387–2395.
  * [57] J. Thies, M. Zollhöfer, and M. Nießner, “Deferred neural rendering: Image synthesis using neural textures,” _ACM Trans. Graph._ , vol. 38, no. 4, jul 2019.
  * [58] L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, “Faceshifter: Towards high fidelity and occlusion aware face swapping,” _ArXiv_ , vol. abs/1912.13457, 2019.
  * [59] V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, “Blazeface: Sub-millisecond neural face detection on mobile gpus,” _Third Workshop on Computer Vision for AR/VR_ , Jun 2019.
  * [60] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2015.
  * [61] P. Yu, J. Fei, Z. Xia, Z. Zhou, and J. Weng, “Improving generalization by commonality learning in face forgery detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 17, pp. 547–558, 2022.
  * [62] I. Masi, A. Killekar, R. M. Mascarenhas, S. P. Gurudatt, and W. AbdAlmageed, “Two-branch recurrent network for isolating deepfakes in videos,” in _ECCV_ , 2020.
  * [63] L. Chen, Y. Zhang, Y. Song, L. Liu, and J. Wang, “Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection,” _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 18 689–18 698, 2022.

|  Sitong Liu received the B.S. degree in information and computing science
from Nanjing University of Science and Technology in 2018, where he is
currently pursuing the doctoral degree with the School of Computer Science.
His reach interests include artificial intelligence security.
---|---
|  Zhichao Lian (Member, IEEE) received the Ph.D. degree in electrical and
electronic engineering from Nanyang Technological University, Singapore, in
2013. From 2012 to 2014, he was a Postdoctoral Associate with the Department
of Statistics, Yale University. He is currently an Associate Professor with
the School of Computer Science and Engineering, Nanjing University of Science
and Technology. His research interests include image processing, pattern
recognition, and the IoT.
---|---
|  Siqi Gu received the B.S. degree in the School of Information Science and
Technology, Beijing University of Chemical Technology in 2019 and the M.S.
degree in computer science and engineering from Nanjing University of Science
and Technology in 2022. Now she is currently working toward the doctoral
degree in the School of Software Engineering from Nanjing University. Her
research interests include intelligent software testing.
---|---
|  Liang Xiao (Member, IEEE) received the B.S. degree in applied mathematics
and the Ph.D. degree in computer science from the Nanjing University of
Science and Technology, Nanjing, China, in 1999 and 2004, respectively.
From 2009 to 2010, he was a Post-Doctoral Fellow at the Rensselaer Polytechnic
Institute, Troy, NY, USA. He is currently a Professor with the School of
Computer Science, Nanjing University of Science and Technology. His main
research areas include inverse problems in image processing, scientific
computing, data mining, and pattern recognition.
---|---