# The Tug-of-War Between Deepfake Generation and Detection

Hannah Lee  Changyeon Lee  Kevin Farhat  Lin Qiu  Steve Geluso  Aerin Kim
Oren Etzioni

###### Abstract

Multimodal generative models are rapidly evolving, leading to a surge in the
generation of realistic video and audio that offers exciting possibilities but
also serious risks. Deepfake videos, which can convincingly impersonate
individuals, have particularly garnered attention due to their potential
misuse in spreading misinformation and creating fraudulent content. This
survey paper examines the dual landscape of deepfake video generation and
detection, emphasizing the need for effective countermeasures against
potential abuses. We provide a comprehensive overview of current deepfake
generation techniques, including face swapping, reenactment, and audio-driven
animation, which leverage cutting-edge technologies like generative
adversarial networks and diffusion models to produce highly realistic fake
videos. Additionally, we analyze various detection approaches designed to
differentiate authentic from altered videos, from detecting visual artifacts
to deploying advanced algorithms that pinpoint inconsistencies across video
and audio signals.

The effectiveness of these detection methods heavily relies on the diversity
and quality of datasets used for training and evaluation. We discuss the
evolution of deepfake datasets, highlighting the importance of robust,
diverse, and frequently updated collections to enhance the detection accuracy
and generalizability. As deepfakes become increasingly indistinguishable from
authentic content, developing advanced detection techniques that can keep pace
with generation technologies is crucial. We advocate for a proactive approach
in the “tug-of-war” between deepfake creators and detectors, emphasizing the
need for continuous research collaboration, standardization of evaluation
metrics, and the creation of comprehensive benchmarks.

machine learning, deepfakes, deepfake detection, deep learning, generative
models, misinformation, survey, generative adversarial networks, diffusion,
datasets

##  1 Introduction

Recent advances in multimodal generative models have made manipulated media
increasingly more realistic and accessible. Although synthetically generated
audio, images, and videos can have creative and beneficial applications such
as improved dubbing or translation of films (Yang et al., 2020; Hu et al.,
2021), deepfake videos that impersonate humans highlight the potential harms
of media manipulation and synthetic generation. For example, deepfakes that
blend celebrities’ faces onto bodies in pornographic videos (Nguyen et al.,
2022; Hsu, 2024) and alter politicians’ messages (Suwajanakorn et al., 2017)
can spread misinformation, threaten individuals, and damage reputations (Zhou
& Zafarani, 2020), disrupting election campaigns and financial markets.
Recently, deepfakes have also become integral to fraudulent schemes, resulting
in scams of up to $25 million (Chen & Magramo, 2024). Given the rise of social
media and online media consumption, it is unsurprising that deepfake videos
are increasingly interfering with people’s lives.

The first modern deepfake videos surfaced in 2017 when users on Reddit posted
computer-generated pornographic videos of actresses (Nguyen et al., 2022).
Since then, many deepfake generation tools have become available for public
use. FaceSwap (Fac, 2024), FaceSwapGAN (Lu, 2024), StyleGAN (Karras et al.,
2019), FSGAN (Nirkin et al., 2019) and other applications allow anyone with
basic programming skills to generate their own deepfakes, and as text-to-video
models such as Imagen Video (Ho et al., 2022), CogVideo (Hong et al., 2022),
and Sora (Brooks et al., 2024) improve and become widespread, the barrier to
entry will continue to be lowered.

As deepfake video generation has become increasingly democratized and capable
of producing realistic results, emphasis on countermeasures has also grown.
Generally, addressing misinformation and the harms related to deepfakes
follows two patterns: detection and prevention. While prevention through
techniques like watermarking (Lv, 2021) and blockchain frameworks (Rashid et
al., 2021) as well as through technology policy (Reisach, 2021) is critical
for mitigating deepfake harms, it is out of scope for this paper and we
instead focus on deepfake detection, the currently more common strategy.

Deepfake video detection is comprised of fake image detection, fake audio
detection, and techniques specific to video sequences. These techniques are
generally leveraged for the binary classification task where classifiers learn
to distinguish authentic and manipulated videos. Initial detection tools
focused primarily on visual artifacts such as blended face edges (Li et al.,
2020a) and image forgery techniques that have preceded deepfakes (Chaitra &
Reddy, 2022). More recently, deep learning techniques have capitalized on the
large amounts of real and fake data available online. However, since training
detection algorithms depends on fake data created by generation tools,
deepfake detectors lag behind generators. Consequently, the development of
detection algorithms provides direct feedback to generation algorithms on what
makes deepfakes detectable and can encourage adversarial generation to bypass
detection. We refer to this relationship as the ongoing tug-of-war between
deepfake generation and detection.

There are existing survey papers that explore deepfake generation and
detection (Rana et al., 2022; Yu et al., 2021; Yi et al., 2023b; Mirsky & Lee,
2021; Nguyen et al., 2022; Swathi & Sk, 2021). However, these often only focus
on detection (Rana et al., 2022; Yu et al., 2021; Yi et al., 2023b); do not
include modern methods that have gained popularity (Mirsky & Lee, 2021; Swathi
& Sk, 2021); or do not consider in depth the impact of the datasets used to
train generation and detection models. In our approach, we focus on video
generation and detection while discussing the current landscape of curating
relevant training and evaluation datasets. Our main contributions include an
updated survey of deepfake generation and detection techniques; identification
of the successes, challenges, and limitations of current deepfake detection
practices; and suggestions for future deepfake detection research, focusing on
the importance of quality datasets in this tug-of-war.

##  2 Deepfake Video Generation

Generating deepfake video media consists of generating both visual and audio
content. Here, we briefly describe common generation processes for both
modalities.

###  2.1 Face Swapping

Face swapping is one method used to create deepfake images and videos by
replacing one person’s face with another’s face. This technique is now widely
available through well-developed packages like Face Fusion (Ruhs, 2024) and
Faceswap (Fac, 2024). Early works could only handle faces in the same pose,
but later developments have incorporated 3D-based methods to construct faces
with variations (Dale et al., 2011; Lin et al., 2012). With the development of
neural networks, face swapping can now involve an encoder-decoder network
(Perov et al., 2020). The encoder extracts latent features from the faces in
the source image while the decoder reconstructs the target image using these
features. More recently, GAN-based methods have also been applied; FSGAN
(Nirkin et al., 2019) and FSGAN2 (Nirkin et al., 2022) provide a two-stage
pipeline that supports both face swapping and reenactment (further discussed
in Section 2.2) simultaneously. These methods train a generator to learn the
latent representation of the source image and then inpaint the segmented area
in the target image. These GAN-based methods are able to generate more
realistic images with higher resolution and fidelity. However, training GANs
can be unstable, restricting their actual application to lower resolution
images (Xu et al., 2022b; Liu et al., 2023b).

###  2.2 Reenactment

Reenactment focuses on making one person’s face in a video mimic the facial
expressions and head movements of another person. Unlike face swapping, the
identity of the face remains the same while its expressions and movements are
altered (Nguyen et al., 2022). Reenactment can drive the expression, gaze,
mouth, pose, identity, and body of the target subject based on the source
subject (Mirsky & Lee, 2021). Face2Face (Thies et al., 2016) proposed the
first method that achieves real-time RGB-only reenactment without the need for
a teeth proxy (Garrido et al., 2016; Thies et al., 2015) or direct source-to-
target copying (Vlasic et al., 2005). FSGAN can be applied to unseen pairs of
faces and adjusts significant pose and expression variations that can be
applied to a single image or a video sequence (Nirkin et al., 2019). Later,
FSGAN2 extended the model by introducing preprocessing and additional post-
processing steps for reducing flickering and saturation artifacts (Nirkin et
al., 2022). Recently, facial reenactment efforts have not only focused on
updating the expressions on the target image itself but have also involved a
joint effort in creating both audio and visual components for talking face
generation (Section 2.4).

###  2.3 Diffusion-based Deepfakes

Diffusion-based deepfake generation represents a significant advancement over
traditional methods like GANs and autoencoders, offering more realistic and
efficient capabilities for generating deepfakes. Recent developments in
diffusion models (DMs), such as Denoising Diffusion Probabilistic Models
(DDPM) (Ho et al., 2020), have further refined this approach. DDPM utilizes a
series of learned noise adjustments to reverse the addition of noise,
effectively reconstructing the data distribution of original images.
Subsequently, the Latent Diffusion Model (LDM) (Rombach et al., 2021) enhances
this process by generating in the latent space, speeding up the diffusion
process and handling complex data distributions.

Image Generation. Diffusion-generated images are categorized into two types:
text-to-image, where users provide text prompts to specify image content to
models such as DALL-E (Ramesh et al., 2021), DreamBooth (Ruiz et al., 2022)
and Imagen (Saharia et al., 2022); and text guided image editing or
generation, which transforms existing images based on text prompts using
Imagic (Kawar et al., 2022), InstructPix2Pix (Brooks et al., 2022), or other
tools. Chen et al. (2023) proposes a method to specifically generate high
quality deepfakes by leveraging diffusion, using celebrity images as a guide
for the image’s latent initialization.

Video Generation. DMs are also capable of generating videos. Typically, this
is done by inflating 2D convolutional layers into pseudo-3D convolutional
layers in the DM and incorporating temporal self-attention layers in each
transformer block to manage the temporal consistency across frames. As one
example, Diffusion Heads (Stypułkowski et al., 2024) leverages an
autoregressive DM that takes one identity image and an audio sequence to
generate a talking head, which can also be applied to the deepfake generation
with a custom identity image.

###  2.4 Audio-Driven Facial Animation

The field of audio-driven facial animation has been a fascinating area of
exploration for the computer vision and graphics community. Many studies have
been carried out on both digital 3D human faces (Karras et al., 2017a; Richard
et al., 2021; Zhou et al., 2018; Fan et al., 2022) and realistic human head
generation (Fan et al., 2015; Suwajanakorn et al., 2017; Jamaludin et al.,
2019).

Talking Head Generation. Early research on lip-syncing mostly focused on
creating the entire head of a speaking person (Suwajanakorn et al., 2017;
Jamaludin et al., 2019; Wang et al., 2020a; Zhang et al., 2023). Several
methods used structural details like 2D (Chen et al., 2019) landmarks, 3D
landmarks (Zhou et al., 2020) and 3D meshes (Chen et al., 2020). However, the
resulting output often exhibits inconsistencies between connecting parts and
suffers from a lack of precision. Many methods that are specific to
individuals (Suwajanakorn et al., 2017; Lu et al., 2021; Ji et al., 2021)
depend on the subject and may struggle with generalization. Furthermore, the
recent application of NeRF for person-specific modeling, as cited in several
works (Guo et al., 2021; Liu et al., 2022; Shen et al., 2022), often exhibits
sub-optimal performance, including low visual quality, restricted
expressiveness, and inconsistency across frames, when the available training
data is limited.

Lip-Syncing on Faces. Other research has focused on lip-syncing the mouth in
videos while leaving other elements unchanged (Prajwal et al., 2020; Park et
al., 2022; Thies et al., 2020). Specifically, Wav2Lip (Prajwal et al., 2020)
is capable of generating lip-sync results that are not person-specific.
However, the generative model used is built on low-resolution images,
resulting in blurry outputs. It also falls short in capturing the unique
identity of a target including the shape of teeth and lips when provided with
a target template video.

###  2.5 Style-based Generator for Faces

StyleGAN models (Karras et al., 2019, 2020, 2021) have shown great success on
image generation tasks, particularly on facial image generation and editing
(Abdal et al., 2019, 2020; Tov et al., 2021; Richardson et al., 2021). They
have also been leveraged for face restorations (Wang et al., 2021; Yang et
al., 2021) and face swapping (Xu et al., 2022c, a), which require the
preservation of the original facial emotion and expressions. Concurrently,
style-based generators have been effectively utilized in the creation of
facial animations (Burkov et al., 2020; Liang et al., 2022). However, the
generators rely on the style vectors in the $W$ or $W^{+}$ latent spaces for
controlling both the appearances and motion dynamics. A significant limitation
of this approach is the $W^{+}$ space’s inability to maintain the spatial
consistency of backgrounds. This often results in non-realistic outcomes or
noticeable artifacts.

###  2.6 Audio Generation

Deepfake audio generation involves creating realistic synthetic audio, often
mimicking a specific person’s voice. We describe common techniques used to
produce highly convincing audio deepfakes.

Text-to-Speech (TTS). Text-to-speech systems convert written text into spoken
words. Modern TTS systems use deep learning models to produce natural-sounding
speech. Key approaches include:

  * •

Concatenative TTS: A traditional method involving concatenating pre-recorded
speech segments to generate fluent speech. While it can produce high-quality
and natural-sounding results by leveraging real audio, it often struggles with
capturing the dynamic variations in human speech (Hande, 2014). This can lead
to noticeable discontinuities and a lack of flexibility when synthesizing
speech with varying intonation, emphasis, or speaking styles (Cohn & Zellou,
2020).

  * •

Parametric TTS: Generating speech by predicting a set of parameters that
describe aspects of the speech signal such as pitch, duration, and spectral
features by using statistical models like Hidden Markov Models (Tokuda et al.,
2000). This approach offers flexibility in creating various voices and
speaking styles, but the synthesized speech may sound less natural due to the
oversimplification of speech variations.

  * •

Neural TTS: Using deep learning models to generate more natural and expressive
speech (Tan et al., 2021). Examples include Tacotron 2 (Shen et al., 2017) and
WaveNet (van den Oord et al., 2016), where Tacotron 2 converts text to a mel-
spectrogram then uses a modified WaveNet as a vocoder to generate audio
waveforms from the spectrogram.

Voice Conversion (VC). VC aims to modify a source speaker’s voice to sound
like a target speaker without changing the linguistic content. This process
typically involves extracting features from the source speech, such as pitch,
spectral envelope, and timbre, then transforming these features to match the
target speaker’s characteristics. Statistical models like Gaussian Mixture
Models (GMMs) (Kain & Macon, 1998) and deep learning approaches such as GANs
and variational autoencoders (VAEs) are employed to learn the mapping between
the source and target features. Recent advancements have significantly
improved the naturalness and intelligibility of converted speech (Sisman et
al., 2020).

Emotion Fake. Emotion fake, or emotional VC, involves altering the emotional
tone of a speaker’s voice. This technique is essential for creating more
expressive and engaging synthetic speech. It typically involves extracting
prosodic features such as pitch, duration, and intensity before modifying
these features to reflect the desired emotion. Deep learning methods,
particularly Convolutional Neural Networks (CNNs), GANs and Recurrent Neural
Networks (RNNs), have shown promising results in this area (Trinh Van et al.,
2022; Liu et al., 2020a). These models can learn complex mappings between
neutral and emotional speech, resulting in more natural and convincing
emotional expressions.

Scene Fake. Scene fake, also known as environmental sound synthesis, involves
generating background sounds or environmental effects to accompany synthetic
speech to enhance the overall experience. This technique aims to create a more
realistic auditory scene by simulating various ambient sounds such as street
noise, office chatter, or natural environments like piano or birdsong (Donahue
et al., 2019; Huzaifah & Wyse, 2020). GANs and autoencoder-based architectures
have been applied to synthesize scene fake audio.

Partially Fake. Partially altered audio involves modifying specific words or
segments within a spoken recording by replacing them with either authentic or
artificially generated audio snippets. The speaker’s voice remains consistent
throughout the original speech and the altered segments, making the modified
audio sound authentic despite the introduced changes. This technique allows
for counterfeit versions of the initial speech to be created while maintaining
the speaker’s identity and vocal characteristics (Wu et al., 2022).

##  3 Deepfake Video Detection

Deepfake video generation techniques necessitate a broad arsenal of detection
tools. Fake video detection is generally broken down into three main
categories; fake image detection where individual frames are analyzed; fake
audio detection to analyze the audio of a video; and fake video detection,
which may utilize both images and audio as well as temporal data.

###  3.1 Fake Image Detection

Detecting fake images precedes deepfakes and deep learning. We discuss methods
ranging from artifact detection to modern methods that have developed with
advancements in deep learning.

General Visual Artifact Detection. Deepfake images may introduce subtle
artifacts that are not present in real images. These can include artifacts
introduced by face blending or face warping when swapping faces, as well as
inconsistencies in the overall image. For example, Li et al. (2020a) propose
the “face X-ray” image representation to detect anomalies in the blending
boundaries of faces in blended images. Other work has focused on the differing
image textures between generated and real content (Liu et al., 2020b);
inconsistencies in head positions (Yang et al., 2019); the resolution
variability that arises from warping faces (Li & Lyu, 2018); and missing
reflections or details in the teeth and eyes (Matern et al., 2019). Many other
visual artifact detection methods have been proposed, but as deepfake
generation has improved and fewer artifacts remain, these techniques have
become overshadowed by methods focused on detecting more subtle identifiers.

Detecting GAN-generated Images. A subset of image deepfake detection methods
have focused on detecting artifacts unique to popular GAN models. Some
detection methods still rely on visible differences such as irregular pupil
shapes (Guo et al., 2022), while many others exploit lower level
abnormalities. For example, the upsampling operations involved in GAN
generation introduce model specific artifacts into the images’ spatial and
frequency domains (Zhang et al., 2019; Marra et al., 2019; Yu et al., 2021).
Wang et al. (2020b) trained a ResNet-50 model (He et al., 2016) that detects
these artifacts and found that GAN-generated images are easily detected.
Similarly, FakeSpotter (Wang et al., 2019) can effectively detect AI-
synthesized fake faces generated by GANs. Building upon these detectors,
PatchForensics (Chai et al., 2020) introduced a detector that analyzes smaller
patches of images to determine if there are AI-generated or manipulated areas.

Diffusion Detection. Generalization of GAN-based detection methods to newer,
diffusion based image generation techniques is difficult (Corvi et al., 2023;
Ojha et al., 2023). The artifacts introduced by GANs in images are no longer
present with DMs. Recently, Wang et al. (2023) proposed DIRE, a new image
representation that uses reconstructions of images using DMs as a method for
detecting DM generated images. They hypothesize that DM generated images
consist of features that are better reconstructed by other pretrained DMs,
compared to reconstructions of real images. Furthermore, Lim et al.
(lim2024distildire) introduced DistilDIRE, a diffusion-generated image
detection framework that significantly reduces the computational demands of
the original DIRE method. Ojha et al. (2023) instead utilize the learned
feature space of a pretrained vision-language model to determine if an image
was AI-generated. And Lorenz et al. (2023) rely on multi Local Intrinsic
Dimensionality to detect diffusion. These novel methods highlight how
detection algorithms continue to adapt to the newer generative models.

###  3.2 Fake Audio Detection

Audio deepfake detection can utilize traditional classifiers like Support
Vector Machines (SVM) after feature extraction using methods such as STFT
spectrograms or Mel-frequency Cepstral coefficients (MFCC). However, deep
learning models outperform traditional classifiers by learning complex
patterns, resulting in better accuracy; these approaches are now widely
preferred (Zaman et al., 2023).

Typically, deep learning-based audio deepfake detection involves extracting
image features like STFT spectrogram feature images, followed by employing a
CNN-based architecture to extract embeddings. A binary classification layer is
then added to classify as real or fake. However, other techniques that are
often employed in traditional audio classification can also be applied. RNN-
based models can capture temporal patterns of audio signals due to their
sequential nature (Zaman et al., 2023), enabling classification into different
categories (Scarpiniti et al., 2021; Gimeno et al., 2020).

More recently, transformer-based models have been introduced. These models,
when compared to CNN-based ones, can handle input-length variance due to their
multi-head self-attention mechanisms (Zaman et al., 2023). This allows them to
effectively capture useful global-context information, regardless of the audio
length (Gong et al., 2021; Ghosh et al., 2023; Liu et al., 2023a).

###  3.3 Fake Video Detection

Deepfake detection techniques for videos transcend detection techniques for
single images by comparing images temporally frame after frame. Deepfake video
detection techniques may also combine information from images and audio to
make inferences based on coherence, synchronization, and physiology.

Frame by Frame Analysis. Evidence of deepfake video generation may be apparent
analyzing videos frame by frame. Determining movement via optical flow may
distinguish real videos from fake videos (Amerini et al., 2019). Guera and
Delp (2018) use a CNN and RNN to extract features in frames and detect
inconsistencies from the overall frame sequence, acknowledging deepfake
manipulations often only appear briefly in overall videos. Taking one frame,
and predicting the next frame, and measuring the error between the prediction
and the actual next frame provides a basis for prediction error energy
analysis (Amerini & Caldelli, 2020). Frame analysis techniques can be improved
by pre-processing video data to focus on area where faces occur (Sabir et al.,
2019).

Recent frame by frame methods include GenConViT (Wodajo & Atnafu, 2023), which
generalizes deepfake video detection by extracting latent spaces of video
frames using two networks with an independently trained autoencoder and VAE.
The VAE reconstructs images and compares the reconstruction to the sample
image. The autoencoder and VAE each feed into a ConvNeXt layer, and a Swin
Transformer forming a hybrid model ConvNeXt-Swin which learns the
relationships among the latent features. Sun et al. (2023) also analyzes
frames, tracking the horizontal and vertical displacement trajectories of
virtual anchor points on faces over time. They found real videos produce
smoother trajectories than anchor points on fake faces and developed a fake
trajectory detection network to classify real and fake videos.

Physiological Features. Generative techniques often lack artifacts of
biological processes. Deepfake video detection techniques have been able to
identify deepfakes by analyzing how people in the video blink (Li et al.,
2018), whether the shape of their mouth is in synchronization with the sounds
their mouth is making (Agarwal et al., 2020), and measuring heart rate via
photoplethysmography (PPG) signals from hemoglobin content in the blood
changing how skin reflects light (Umur Aybars Ciftci, 2020).

Audio-visual Analysis. Some work has observed inconsistencies between images
and audio. Chugh and Subramanian (2020) observe that deepfake videos often
have dissonance between the audio and video and create a metric called the
Modality Dissonance Score. Computing audio-visual dissimilarity over 1-second
video clips, they then aggregate these scores to perform classification.

###  3.4 Adversarial Attacks and Evading Detectors

Despite numerous detection methods, studies have found that evasion efforts
can be effective at rendering some detectors useless. Carlini and Farid (2020)
found multiple small perturbation attacks that can be applied to deepfake
images, resulting in the performance of the classifier by Wang et al. (2020b)
to be reduced to worse than chance while minimizing distortions visible to
humans. Hou et al. (2023) and Neekhara et al. (2021) build on this work to
extend the attacks and apply to other detectors. While these works do not
include evasions of more modern detection methods, they speak to the tug-of-
war nature between generation and detection.

##  4 Detection Challenges, Competitions, and Datasets

Each of the methods described in Sections 2 and 3 have been trained on
datasets of curated real and fake images. Here, we discuss the current
landscape of available datasets and the progression of modern deepfake video
detection algorithms. Table 1 summarizes some commonly used, publicly
available datasets.

Table 1: Summary of common datasets used for training and evaluating deepfake
detection models.
Dataset | Modality | Identities | Real Samples | Generated Samples | Generation Methods | Year
---|---|---|---|---|---|---
CNNDetect | Image | / | 72,400 | 72,400 | Multiple CNNs | 2020
CIFAKE | Image | / | 60,000 | 60,000 | Diffusion | 2024
FoR | Audio | / | >111,000 | >87,000 | TTS | 2019
ASVspoof (LA) | Audio | 107 | 12,483 | 108,978 | TTS, VC | 2019
H-Voice | Audio | / | 3,268 | 3,404 | Multiple | 2020
WaveFake | Audio | / | / | 117,985 | Multiple | 2021
In-the-Wild | Audio | 58 | 20.7 hours | 17.2 hours | Multiple | 2022
EmoFake | Audio | 10 | 17,500 | 36,400 | Multiple EVC Models | 2022
SceneFake | Audio | 107 | 19,838 | 64,642 | Multiple | 2022
DEEP-VOICE | Audio | 8 | 62 min 22 sec | 62 min 22 sec | RVC model | 2023
ADD | Audio | / | 243,194 | 273,874 | Multiple | 2023
DeepfakeTIMIT | Video | 32 | 320 | 640 | GAN (face swap) | 2018
FaceForensics++ | Video | / | 1,000 | 4,000 | Multiple | 2019
Celeb-DF | Video | 59 | 590 | 5,639 | GAN (face swap) | 2019
WildDeepfake | Video | 707 | 3,805 | 3,509 | Multiple | 2020
DFDC | Video | 960 | 23,654 | 104,500 | Multiple | 2020
DeeperForensics-1.0 | Video | 100 | 50,000 | 10,000 | DF-VAE (face swap) | 2020
AV-Deepfake1M | Video | 2,068 | 286,721 | 860,039 | Multiple | 2023

###  4.1 Deepfake Image Datasets

Many datasets of real faces already exist for facial recognition tasks. In
2007, the Labeled Faces in the Wild (LFW) database curated over 13,000 images
of faces from the internet (Huang et al., 2007). Though now retracted, the MS-
Celeb-1M dataset consisted of one million real images of 100,000 different
identities (Guo et al., 2016). And IMDB-WIKI (Rothe et al., 2018) and IMDB-
Clean (Lin et al., 2021) include 524,230 and 287,683 images respectively of
faces, designed to train age-estimation algorithms. These datasets also aid
the training of face image generation models.

For deepfake image generation techniques that rely on face swapping or
reenactment, the deepfake creator must have sufficient training data focused
on both subjects (Fac, 2024; Ruhs, 2024). This has limited deepfakes to be of
celebrities with large amounts of publicly available content, but as the
quantity of public data increases and generation techniques improve, the
accessibility to creating deepfakes continues to rise. The CelebA-HQ dataset
(Karras et al., 2017b) is one early dataset of 30,000 images of the faces of
celebrities, based on the larger CelebA dataset (Liu et al., 2015), that has
been used to train GANs to generate higher resolution images of human faces.
Karras et al. (2019) went on to create Flickr-Faces-HQ (FFHQ), a larger and
more diverse dataset of 70,000 images no longer limited to celebrities,
inclusive of broader ranges of ages, ethnicities, and image backgrounds.

In contrast to deepfake generation models, detection models are typically
trained on large datasets of both real and fake images. Often, these are
custom datasets depending on the detection focus and currently available image
generators. Wang et al. (2020b) trained an image classifier on the LSUN (Yu et
al., 2015) dataset and ProGAN (Karras et al., 2019) generated images. To test
their image classifier, Wang et al. generated over 72,000 images using 11
different CNN-based models including StyleGAN (Karras et al., 2019), BigGAN
(Brock et al., 2018), StarGAN (Choi et al., 2018), and CycleGAN (Zhu et al.,
2017). This collection of real and fake samples has been used by subsequent
detection methods to train and evaluate models (Ojha et al., 2023; Corvi et
al., 2023), but since it lacks newer generation methods, new custom
evaluations appear often (Wang et al., 2023; Lorenz et al., 2023; Lu &
Ebrahimi, 2024). Standalone efforts have also been made to curate synthetic
image datasets (Wang et al., 2022; Bird & Lotfi, 2024), which can be used to
train detectors. However, while there have been attempts to standardize
deepfake video detection benchmarks (see Section 4.3), there are no commonly
established evaluations for manipulated image detection.

###  4.2 Deepfake Audio Datasets and Detection Challenges

As synthetic audio becomes increasingly accessible, interest in synthetic
speech detection has grown (Yi et al., 2023b; Almutairi & Elgibreen, 2022;
Hamza et al., 2022). This trend has spurred the creation of competitions like
the ASVspoof Challenge (Wang et al., 2020c; Yamagishi et al., 2021), which
aims to reduce reliance on specific knowledge of speaker verification systems
and spoofing attacks, encouraging a more realistic evaluation scenario
reflecting real-world examples, and the Audio Deepfake Detection (ADD)
Challenge (Yi et al., 2022, 2023a), which aims to identify and analyze
deepfake speech utterances, tackling the growing threats from advancements in
speech synthesis and VC technologies.

Beyond standarized challenges, research on audio deepfake detection has been
energized by the introduction of various datasets. For example, the “In-the-
Wild” dataset (Müller et al., 2022) contains audio deepfakes featuring
politicians and public figures gathered from the internet. The ASVspoof DF
dataset (Wang et al., 2020c; Yamagishi et al., 2021) comprises genuine and
spoofed speech recordings that have been subjected to various lossy codecs
such as m4a that are commonly employed in media storage. The DEEP-VOICE
dataset (Bird & Lotfi, 2023) curates authentic human speech recordings from
eight notable individuals, along with voices altered to mimic each other
through Retrieval-based VC techniques. Similarly, datasets like the Fake or
Real (FoR) dataset (Reimao & Tzerpos, 2019), WaveFake (Frank & Schönherr,
2021), EmoFake (Zhao et al., 2023), SceneFake (Yi et al., 2024), and H-Voice
(Ballesteros et al., 2020) offer specialized perspectives, each contributing
uniquely to the field. The FoR Dataset comprises over 198,000 utterances from
deep-learning speech synthesizers and real speech, serving as a cornerstone
for research in speech synthesis and synthetic speech detection. WaveFake
provides samples from various network architectures and languages; EmoFake
explores the impact of altering audio emotions; and SceneFake aims to detect
manipulated audio scenes through modifications of the acoustic environment.
Lastly, H-Voice consists of 6,672 histograms derived from authentic and
synthetic voice recordings.

###  4.3 Deepfake Video Datasets and Detection Challenges

One of the earliest and most influential datasets in the domain of deepfake
video detection is the FaceForensics++ dataset (Rössler et al., 2019). It
includes over 1,000 real video sequences collected from YouTube and
corresponding deepfakes created using four different manipulation methods:
Deepfakes (Fac, 2024), Face2Face (Thies et al., 2016), FaceSwap (Marek, 2024),
and NeuralTextures (Thies et al., 2019). The dataset is widely used for
training and benchmarking detection algorithms due to its diversity in
manipulation techniques, but remains limited by the small number of unique
identities present in the videos. Other early works include the WildDeepfake
dataset (Zi et al., 2020), which contains a set of 7314 sequences of deepfake
content collected from the internet, and the DeepfakeTIMIT dataset (Korshunov
& Marcel, 2018), a collection of public real videos each with a GAN generated
deepfake counterpart.

Building on the foundational work of datasets like FaceForensics++, the
DeepFake Detection Challenge (DFDC) dataset was introduced by Facebook AI to
spur advancements in detection technologies (Dolhansky et al., 2020). The DFDC
dataset is one of the most comprehensive public datasets, containing over
100,000 face swap video clips of both real and manipulated content. It is
notable for having faces of more than 3,000 subjects and using 8 facial
modification algorithms. It uses several deepfake generation techniques
including GAN-based and non-learned methods to produce the clips.

Another significant contribution is the Celeb-DF dataset (Li et al., 2020b),
which addresses some of the limitations found in earlier datasets such as
visual artifacts. Celeb-DF contains 590 YouTube videos of celebrities of
varying gender, age, and ethnicity, and 5,639 high-quality deepfake videos of
these subjects generated using improved deepfake algorithms that reduce common
visual artifacts, thereby posing a greater challenge for detection systems.

In addition to these datasets, other efforts have focused on generating more
complex and varied datasets. For instance, the DeeperForensics-1.0 dataset
(Jiang et al., 2020) introduces perturbations such as compression and noise to
its video sequences to create a more rigorous and comprehensive benchmark. It
contains over 17 million frames and 11,000 manipulated deepfake videos
generated using DeepFake VAE swapping. This dataset is subjected to a wide
range of real-world distortions, resulting in a larger and more diverse
collection of face swap videos that better reflect the complexities of real-
world scenarios.

AV-Deepfake1M (Cai et al., 2023) is one of the largest deepfake datasets to
date, containing over 2,000 subjects and 1 million videos with various types
of manipulations, including video, audio, and audio-visual deepfakes. Before
this dataset, there had been limited datasets including small segments of
audio-visual manipulations embedded within real videos, leaving detectors
susceptible to these types of attacks.

##  5 Discussion

The development of deepfake technologies has significantly advanced, leading
to a continuous tug-of-war between generation techniques and the corresponding
detection methods. This dynamic interplay shapes the landscape of both fields.

###  5.1 Challenges in Current Detection Approaches

Data Scarcity and Bias. One major issue is the lack of comprehensive and
diverse datasets that reflect the full range of manipulations generation
models can produce. This scarcity leads to detection models that may perform
well on specific types of deepfakes but fail to generalize to out-of-
distribution media, including new or slightly different deepfake approaches.

Detection datasets are often custom-made, focusing on outputs from specific
generation models. This approach allows detectors to identify unique artifacts
but limits their ability to generalize. Additionally, relying on custom
datasets complicates the direct comparison of detection approaches. Recent
efforts such as DeepfakeBench (Yan et al., 2023) have made strides towards
standardizing comparisons between models by creating a benchmark incorporating
many different datasets and streamlining the evaluation of models through a
series of analysis tools. However, the rapid advancements in generators make
detection datasets become quickly outdated.

Evolving Generation Techniques. As generation methods evolve, they often
develop capabilities to bypass specific detection mechanisms, especially those
relying on detecting artifacts or inconsistencies that newer models no longer
produce: the frequency space artifacts characteristic of GAN-generated images
are noticeably absent from diffusion-generated images (Corvi et al., 2023) and
adversarial perturbations can evade detectors (Carlini & Farid, 2020).
Relatedly, the visibly increased resolution and realism of deepfakes make it
challenging for human experts and automated systems to distinguish between
genuine and manipulated content. This raises concerns about the effectiveness
of current detection technologies and how to best curate datasets if the
authenticity of media cannot be easily verified.

###  5.2 Future Directions

Curate Robust Datasets and Design Competitions for Detection. To effectively
combat deepfakes, there is a critical need for creating and maintaining
robust, diverse, and representative datasets that are publicly available to
the research community. These datasets should include a wide variety of
deepfake types and techniques to better train and test detection models.
Diversity in modality should also be present; these datasets should include
different types of media such as videos, audio, and images from various
demographics and in multiple languages to ensure comprehensive coverage. The
datasets should be continuously updated at a regular cadence to include the
latest deepfake techniques and real-world examples, ensuring that detection
models can learn to counter new threats. By updating datasets in this way,
deepfake detection methods can be trained more closely to detect the in-the-
wild examples that pose the greatest threats to mislead individuals. Creating
competitions for model evaluations can also establish standardized comparisons
that test detectors on these pertinent examples as well.

Focused Efforts on Representation and Consent. The use of web-scraped data to
train both generation and detection models also raises concerns about
representation and consent. Deepfake subjects are often public figures like
celebrities since ample data is available for them online. However, as the
barrier to create deepfakes lowers, issues around non-consensual deepfake
pornography and identity misuse will likely become more prevalent. While such
content should be moderated and prevented from being created through policy
and moderation considerations, detection methods must also be proactive in
handling such cases.

Harness Capabilities of Foundation Models for Deepfake Detection. One avenue
parallel to curating robust datasets for training detection models is to
harness the capabilities of foundation models that have been pretrained on
large amounts of data, likely already inclusive of deepfake datasets. Some
detection methods have started to utilize pretrained CLIP (Radford et al.,
2021) models (Ojha et al., 2023; Pianese et al., 2024; Tariang et al., 2024)
to detect both fake images and audio, but these methods are relatively new and
still face the same difficulties when it comes to standardized evaluations.

##  6 Conclusion

We provide a short survey of the current landscape of deepfake video
generation and detection, as well as of the datasets used to train and
evaluate these methods. Although great progress has been made in deepfake
video detection, there is still much room for improvement in curating datasets
that enable the development of robust, generalizable, and responsible
detectors. Closer collaboration between the generation and detection
communities could help anticipate future directions and ensure that detectors
are well-equipped to handle the latest deepfake techniques. Regular detection
competitions using shared and frequently updated datasets are a promising
avenue to align research efforts to minimize deepfake misinformation spread
and related consequences.

## References

  * Fac (2024) Faceswap, May 2024.  URL https://github.com/deepfakes/faceswap.  original-date: 2017-12-19T09:44:13Z.
  * Abdal et al. (2019) Abdal, R., Qin, Y., and Wonka, P.  Image2stylegan: How to embed images into the stylegan latent space?  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 4432–4441, 2019.
  * Abdal et al. (2020) Abdal, R., Qin, Y., and Wonka, P.  Image2stylegan++: How to edit the embedded images?  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 8296–8305, 2020.
  * Agarwal et al. (2020) Agarwal, S., Farid, H., Fried, O., and Agrawala, M.  Detecting deep-fake videos from phoneme-viseme mismatches.  pp. 660–661, 2020.
  * Almutairi & Elgibreen (2022) Almutairi, Z. and Elgibreen, H.  A review of modern audio deepfake detection methods: Challenges and future directions.  _Algorithms_ , 15(5):155, 2022.
  * Amerini & Caldelli (2020) Amerini, I. and Caldelli, R.  Exploiting prediction error inconsistencies through lstm-based classifiers to detect deepfake videos.  _IH &MMSec 2020_, 2020.
  * Amerini et al. (2019) Amerini, I., Galteri, L., Caldelli, R., and Del Bimbo, A.  Deepfake video detection through optical flow based cnn.  pp. 1205–1207, 2019.  doi: 10.1109/ICCVW.2019.00152.
  * Ballesteros et al. (2020) Ballesteros, D. M., Rodriguez, Y., and Renza, D.  A dataset of histograms of original and fake voice recordings (h-voice).  _Data in Brief_ , 29:105331, 2020.  ISSN 2352-3409.  doi: https://doi.org/10.1016/j.dib.2020.105331.  URL https://www.sciencedirect.com/science/article/pii/S2352340920302250.
  * Bird & Lotfi (2023) Bird, J. J. and Lotfi, A.  Real-time detection of ai-generated speech for deepfake voice conversion, 2023.
  * Bird & Lotfi (2024) Bird, J. J. and Lotfi, A.  Cifake: Image classification and explainable identification of ai-generated synthetic images.  _IEEE Access_ , 2024.
  * Brock et al. (2018) Brock, A., Donahue, J., and Simonyan, K.  Large scale gan training for high fidelity natural image synthesis.  _arXiv preprint arXiv:1809.11096_ , 2018.
  * Brooks et al. (2022) Brooks, T., Holynski, A., and Efros, A. A.  Instructpix2pix: Learning to follow image editing instructions.  _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 18392–18402, 2022.  URL https://api.semanticscholar.org/CorpusID:253581213.
  * Brooks et al. (2024) Brooks, T., Peebles, B., Holmes, C., DePue, W., Guo, Y., Jing, L., Schnurr, D., Taylor, J., Luhman, T., Luhman, E., Ng, C., Wang, R., and Ramesh, A.  Video generation models as world simulators.  2024\.  URL https://openai.com/research/video-generation-models-as-world-simulators.
  * Burkov et al. (2020) Burkov, E., Pasechnik, I., Grigorev, A., and Lempitsky, V.  Neural head reenactment with latent pose descriptors.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 13786–13795, 2020.
  * Cai et al. (2023) Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., and Stefanov, K.  Av-deepfake1m: A large-scale llm-driven audio-visual deepfake dataset, 2023.
  * Carlini & Farid (2020) Carlini, N. and Farid, H.  Evading deepfake-image detectors with white-and black-box attacks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops_ , pp. 658–659, 2020.
  * Chai et al. (2020) Chai, L., Bau, D., Lim, S.-N., and Isola, P.  What makes fake images detectable? understanding properties that generalize, 2020.
  * Chaitra & Reddy (2022) Chaitra, B. and Reddy, P. B.  Digital image forgery: taxonomy, techniques, and tools–a comprehensive study.  _International Journal of System Assurance Engineering and Management_ , 14:18–33, 2022.  URL https://api.semanticscholar.org/CorpusID:255039112.
  * Chen & Magramo (2024) Chen, H. and Magramo, K.  Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’, 2024.
  * Chen et al. (2019) Chen, L., Maddox, R. K., Duan, Z., and Xu, C.  Hierarchical cross-modal talking face generation with dynamic pixel-wise loss.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 7832–7841, 2019.
  * Chen et al. (2020) Chen, L., Cui, G., Kou, Z., Zheng, H., and Xu, C.  What comprises a good talking-head video generation?: A survey and benchmark.  _arXiv preprint arXiv:2005.03201_ , 2020.
  * Chen et al. (2023) Chen, Y., Haldar, N. A. H., Akhtar, N., and Mian, A.  Text-image guided diffusion model for generating deepfake celebrity interactions.  _2023 International Conference on Digital Image Computing: Techniques and Applications (DICTA)_ , pp. 348–355, 2023.  URL https://api.semanticscholar.org/CorpusID:262825503.
  * Choi et al. (2018) Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., and Choo, J.  Stargan: Unified generative adversarial networks for multi-domain image-to-image translation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 8789–8797, 2018.
  * Chugh et al. (2020) Chugh, K., Gupta, P., Dhall, A., and Subramanian, R.  Not made for each other-audio-visual dissonance-based deepfake detection and localization.  pp. 439–447, 2020.
  * Cohn & Zellou (2020) Cohn, M. and Zellou, G.  Perception of Concatenative vs. Neural Text-To-Speech (TTS): Differences in Intelligibility in Noise and Language Attitudes.  In _Proc. Interspeech 2020_ , pp. 1733–1737, 2020.  doi: 10.21437/Interspeech.2020-1336.
  * Corvi et al. (2023) Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., and Verdoliva, L.  On the detection of synthetic images generated by diffusion models.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 1–5. IEEE, 2023.
  * Dale et al. (2011) Dale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., and Pfister, H.  Video face replacement.  _Proceedings of the 2011 SIGGRAPH Asia Conference_ , 2011.  URL https://api.semanticscholar.org/CorpusID:8692593.
  * Dolhansky et al. (2020) Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C. C.  The deepfake detection challenge (dfdc) dataset, 2020.
  * Donahue et al. (2019) Donahue, C., McAuley, J., and Puckette, M.  Adversarial audio synthesis, 2019.
  * Fan et al. (2015) Fan, B., Wang, L., Soong, F. K., and Xie, L.  Photo-real talking head with deep bidirectional lstm.  In _2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 4884–4888. IEEE, 2015.
  * Fan et al. (2022) Fan, Y., Lin, Z., Saito, J., Wang, W., and Komura, T.  Faceformer: Speech-driven 3d facial animation with transformers.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 18770–18780, 2022.
  * Frank & Schönherr (2021) Frank, J. and Schönherr, L.  Wavefake: A data set to facilitate audio deepfake detection, 2021.
  * Garrido et al. (2016) Garrido, P., Valgaerts, L., Rehmsen, O., Thormählen, T., Pérez, P., and Theobalt, C.  Automatic face reenactment.  _CoRR_ , abs/1602.02651, 2016.  URL http://arxiv.org/abs/1602.02651.
  * Ghosh et al. (2023) Ghosh, S., Seth, A., Umesh, S., and Manocha, D.  Mast: Multiscale audio spectrogram transformers, 2023.
  * Gimeno et al. (2020) Gimeno, P., Viñals, I., Ortega, A., Miguel, A., and Lleida, E.  Multiclass audio segmentation based on recurrent neural networks for broadcast domain data.  _EURASIP Journal on Audio, Speech, and Music Processing_ , 2020(1), March 2020.  ISSN 1687-4722.  doi: 10.1186/s13636-020-00172-6.  URL http://dx.doi.org/10.1186/s13636-020-00172-6.
  * Gong et al. (2021) Gong, Y., Chung, Y.-A., and Glass, J.  Ast: Audio spectrogram transformer, 2021.
  * Guera (2018) Guera, David; Delp, E. J.  Deepfake video detection using recurrent neural networks.  _IEEE 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_ , 2018.
  * Guo et al. (2022) Guo, H., Hu, S., Wang, X., Chang, M.-C., and Lyu, S.  Eyes tell all: Irregular pupil shapes reveal gan-generated faces.  In _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 2904–2908. IEEE, 2022.
  * Guo et al. (2016) Guo, Y., Zhang, L., Hu, Y., He, X., and Gao, J.  Ms-celeb-1m: A dataset and benchmark for large-scale face recognition.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14_ , pp. 87–102. Springer, 2016.
  * Guo et al. (2021) Guo, Y., Chen, K., Liang, S., Liu, Y., Bao, H., and Zhang, J.  Ad-nerf: Audio driven neural radiance fields for talking head synthesis.  In _IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2021.
  * Hamza et al. (2022) Hamza, A., Javed, A. R. R., Iqbal, F., Kryvinska, N., Almadhor, A. S., Jalil, Z., and Borghol, R.  Deepfake audio detection via mfcc features using machine learning.  _IEEE Access_ , 10:134018–134028, 2022.  ISSN 2169-3536.  doi: 10.1109/access.2022.3231480.  URL http://dx.doi.org/10.1109/ACCESS.2022.3231480.
  * Hande (2014) Hande, S. S.  A review of concatenative text to speech synthesis.  _International Journal of Latest Trends in Engineering and Technology (IJLTEMAS)_ , 3(9):12–15, 2014.  URL https://www.ijltemas.in/DigitalLibrary/Vol.3Issue9/12-15.pdf.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Ho et al. (2020) Ho, J., Jain, A., and Abbeel, P.  Denoising diffusion probabilistic models.  _ArXiv_ , abs/2006.11239, 2020.  URL https://api.semanticscholar.org/CorpusID:219955663.
  * Ho et al. (2022) Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al.  Imagen video: High definition video generation with diffusion models.  _arXiv preprint arXiv:2210.02303_ , 2022.
  * Hong et al. (2022) Hong, W., Ding, M., Zheng, W., Liu, X., and Tang, J.  Cogvideo: Large-scale pretraining for text-to-video generation via transformers.  _arXiv preprint arXiv:2205.15868_ , 2022.
  * Hou et al. (2023) Hou, Y., Guo, Q., Huang, Y., Xie, X., Ma, L., and Zhao, J.  Evading deepfake detectors via adversarial statistical consistency.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 12271–12280, 2023.
  * Hsu (2024) Hsu, T.  Fake and explicit images of taylor swift started on 4chan, study says, 2024.
  * Hu et al. (2021) Hu, C., Tian, Q., Li, T., Yuping, W., Wang, Y., and Zhao, H.  Neural dubber: Dubbing for videos according to scripts.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 16582–16595. Curran Associates, Inc., 2021.  URL https://proceedings.neurips.cc/paper_files/paper/2021/file/8a9c8ac001d3ef9e4ce39b1177295e03-Paper.pdf.
  * Huang et al. (2007) Huang, G. B., Ramesh, M., Berg, T., and Learned-Miller, E.  Labeled faces in the wild: A database for studying face recognition in unconstrained environments.  Technical Report 07-49, University of Massachusetts, Amherst, October 2007.
  * Huzaifah & Wyse (2020) Huzaifah, M. and Wyse, L.  Deep generative models for musical audio synthesis, 2020.
  * Jamaludin et al. (2019) Jamaludin, A., Chung, J. S., and Zisserman, A.  You said that?: Synthesising talking faces from audio.  _International Journal of Computer Vision_ , 127(11):1767–1779, 2019.
  * Ji et al. (2021) Ji, X., Zhou, H., Wang, K., Wu, W., Loy, C. C., Cao, X., and Xu, F.  Audio-driven emotional video portraits.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 14080–14089, 2021.
  * Jiang et al. (2020) Jiang, L., Li, R., Wu, W., Qian, C., and Loy, C. C.  Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection, 2020.
  * Kain & Macon (1998) Kain, A. and Macon, M. W.  Spectral voice conversion for text-to-speech synthesis.  _Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP ’98 (Cat. No.98CH36181)_ , 1:285–288 vol.1, 1998.  URL https://api.semanticscholar.org/CorpusID:1332456.
  * Karras et al. (2017a) Karras, T., Aila, T., Laine, S., Herva, A., and Lehtinen, J.  Audio-driven facial animation by joint end-to-end learning of pose and emotion.  _ACM Transactions on Graphics (TOG)_ , 36(4):1–12, 2017a.
  * Karras et al. (2017b) Karras, T., Aila, T., Laine, S., and Lehtinen, J.  Progressive growing of gans for improved quality, stability, and variation.  _arXiv preprint arXiv:1710.10196_ , 2017b.
  * Karras et al. (2019) Karras, T., Laine, S., and Aila, T.  A style-based generator architecture for generative adversarial networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 4401–4410, 2019.
  * Karras et al. (2020) Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T.  Analyzing and improving the image quality of StyleGAN.  In _Proc. CVPR_ , 2020.
  * Karras et al. (2021) Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., and Aila, T.  Alias-free generative adversarial networks.  In _Proc. NeurIPS_ , 2021.
  * Kawar et al. (2022) Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H.-T., Dekel, T., Mosseri, I., and Irani, M.  Imagic: Text-based real image editing with diffusion models.  _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 6007–6017, 2022.  URL https://api.semanticscholar.org/CorpusID:252918469.
  * Korshunov & Marcel (2018) Korshunov, P. and Marcel, S.  Deepfakes: A new threat to face recognition? assessment and detection.  _arXiv preprint arXiv:1812.08685_ , 2018.
  * Li et al. (2020a) Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., and Guo, B.  Face x-ray for more general face forgery detection.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 5001–5010, 2020a.
  * Li & Lyu (2018) Li, Y. and Lyu, S.  Exposing deepfake videos by detecting face warping artifacts.  _arXiv preprint arXiv:1811.00656_ , 2018.
  * Li et al. (2018) Li, Y., Chang, M.-C., and Lyu, S.  In ictu oculi: Exposing ai created fake videos by detecting eye blinking.  pp. 1–7, 2018.
  * Li et al. (2020b) Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S.  Celeb-df: A large-scale challenging dataset for deepfake forensics, 2020b.
  * Liang et al. (2022) Liang, B., Pan, Y., Guo, Z., Zhou, H., Hong, Z., Han, X., Han, J., Liu, J., Ding, E., and Wang, J.  Expressive talking head generation with granular audio-visual control.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 3387–3396, 2022.
  * Lin et al. (2012) Lin, Y., Wang, S., Lin, Q., and Tang, F.  Face swapping under large pose variations: A 3d model based approach.  _2012 IEEE International Conference on Multimedia and Expo_ , pp. 333–338, 2012.  URL https://api.semanticscholar.org/CorpusID:30759641.
  * Lin et al. (2021) Lin, Y., Shen, J., Wang, Y., and Pantic, M.  Fp-age: Leveraging face parsing attention for facial age estimation in the wild.  _arXiv_ , 2021.
  * Liu et al. (2020a) Liu, S., Cao, Y., and Meng, H. M.  Emotional voice conversion with cycle-consistent adversarial network.  _arXiv: Audio and Speech Processing_ , 2020a.  URL https://api.semanticscholar.org/CorpusID:215416116.
  * Liu et al. (2022) Liu, X., Xu, Y., Wu, Q., Zhou, H., Wu, W., and Zhou, B.  Semantic-aware implicit neural audio-driven video portrait generation.  _ECCV_ , 2022.
  * Liu et al. (2023a) Liu, X., Lu, H., Yuan, J., and Li, X.  Cat: Causal audio transformer for audio classification, 2023a.
  * Liu et al. (2015) Liu, Z., Luo, P., Wang, X., and Tang, X.  Deep learning face attributes in the wild.  In _Proceedings of International Conference on Computer Vision (ICCV)_ , December 2015.
  * Liu et al. (2020b) Liu, Z., Qi, X., and Torr, P. H.  Global texture enhancement for fake face detection in the wild.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 8060–8069, 2020b.
  * Liu et al. (2023b) Liu, Z., Li, M., Zhang, Y., Wang, C., Zhang, Q., Wang, J., and Nie, Y.  Fine-grained face swapping via regional gan inversion, 2023b.
  * Lorenz et al. (2023) Lorenz, P., Durall, R. L., and Keuper, J.  Detecting images generated by deep diffusion models using their local intrinsic dimensionality.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 448–459, 2023.
  * Lu (2024) Lu, S.-A.  shaoanlu/faceswap-GAN, May 2024.  URL https://github.com/shaoanlu/faceswap-GAN.  original-date: 2017-12-23T08:40:32Z.
  * Lu & Ebrahimi (2024) Lu, Y. and Ebrahimi, T.  Towards the detection of ai-synthesized human face images.  _arXiv preprint arXiv:2402.08750_ , 2024.
  * Lu et al. (2021) Lu, Y., Chai, J., and Cao, X.  Live speech portraits: real-time photorealistic talking-head animation.  _ACM Transactions on Graphics (TOG)_ , 40(6):1–17, 2021.
  * Lv (2021) Lv, L.  Smart watermark to defend against deepfake image manipulation.  In _2021 IEEE 6th international conference on computer and communication systems (ICCCS)_ , pp. 380–384. IEEE, 2021.
  * Marek (2024) Marek.  MarekKowalski/FaceSwap, May 2024.  URL https://github.com/MarekKowalski/FaceSwap.  original-date: 2016-06-19T00:09:07Z.
  * Marra et al. (2019) Marra, F., Gragnaniello, D., Verdoliva, L., and Poggi, G.  Do gans leave artificial fingerprints?  In _2019 IEEE conference on multimedia information processing and retrieval (MIPR)_ , pp. 506–511. IEEE, 2019.
  * Matern et al. (2019) Matern, F., Riess, C., and Stamminger, M.  Exploiting visual artifacts to expose deepfakes and face manipulations.  In _2019 IEEE Winter Applications of Computer Vision Workshops (WACVW)_ , pp. 83–92. IEEE, 2019.
  * Mirsky & Lee (2021) Mirsky, Y. and Lee, W.  The creation and detection of deepfakes: A survey.  _ACM Computing Surveys_ , 54(1):1–41, January 2021.  ISSN 1557-7341.  doi: 10.1145/3425780.  URL http://dx.doi.org/10.1145/3425780.
  * Müller et al. (2022) Müller, N. M., Czempin, P., Dieckmann, F., Froghyar, A., and Böttinger, K.  Does audio deepfake detection generalize?, 2022.
  * Neekhara et al. (2021) Neekhara, P., Dolhansky, B., Bitton, J., and Ferrer, C. C.  Adversarial threats to deepfake detection: A practical perspective.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 923–932, 2021.
  * Nguyen et al. (2022) Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., and Nguyen, C. M.  Deep learning for deepfakes creation and detection: A survey.  _Computer Vision and Image Understanding_ , 223:103525, October 2022.  ISSN 1077-3142.  doi: 10.1016/j.cviu.2022.103525.  URL http://dx.doi.org/10.1016/j.cviu.2022.103525.
  * Nirkin et al. (2019) Nirkin, Y., Keller, Y., and Hassner, T.  FSGAN: subject agnostic face swapping and reenactment.  _CoRR_ , abs/1908.05932, 2019.  URL http://arxiv.org/abs/1908.05932.
  * Nirkin et al. (2022) Nirkin, Y., Keller, Y., and Hassner, T.  Fsganv2: Improved subject agnostic face swapping and reenactment.  2022\.
  * Ojha et al. (2023) Ojha, U., Li, Y., and Lee, Y. J.  Towards universal fake image detectors that generalize across generative models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 24480–24489, 2023.
  * Park et al. (2022) Park, S. J., Kim, M., Hong, J., Choi, J., and Ro, Y. M.  Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory.  In _AAAI Conference on Artificial Intelligence_. Association for the Advancement of Artificial Intelligence, 2022.
  * Perov et al. (2020) Perov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Umé, C., Dpfks, M., Facenheim, C. S., RP, L., Jiang, J., Zhang, S., Wu, P., Zhou, B., and Zhang, W.  Deepfacelab: A simple, flexible and extensible face swapping framework.  _CoRR_ , abs/2005.05535, 2020.  URL https://arxiv.org/abs/2005.05535.
  * Pianese et al. (2024) Pianese, A., Cozzolino, D., Poggi, G., and Verdoliva, L.  Training-free deepfake voice recognition by leveraging large-scale pre-trained models, 2024.
  * Prajwal et al. (2020) Prajwal, K., Mukhopadhyay, R., Namboodiri, V. P., and Jawahar, C.  A lip sync expert is all you need for speech to lip generation in the wild.  In _Proceedings of the 28th ACM International Conference on Multimedia_ , pp. 484–492, 2020.
  * Radford et al. (2021) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pp. 8748–8763. PMLR, 2021.
  * Ramesh et al. (2021) Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., and Sutskever, I.  Zero-shot text-to-image generation.  _ArXiv_ , abs/2102.12092, 2021.  URL https://api.semanticscholar.org/CorpusID:232035663.
  * Rana et al. (2022) Rana, M. S., Nobi, M. N., Murali, B., and Sung, A. H.  Deepfake detection: A systematic literature review.  _IEEE Access_ , 10:25494–25513, 2022.  doi: 10.1109/ACCESS.2022.3154404.
  * Rashid et al. (2021) Rashid, M. M., Lee, S.-H., and Kwon, K.-R.  Blockchain technology for combating deepfake and protect video/image integrity.  _Journal of Korea Multimedia Society_ , 24(8):1044–1058, 2021.
  * Reimao & Tzerpos (2019) Reimao, R. and Tzerpos, V.  For: A dataset for synthetic speech detection.  In _2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)_ , pp. 1–10, 2019.  doi: 10.1109/SPED.2019.8906599.
  * Reisach (2021) Reisach, U.  The responsibility of social media in times of societal and political manipulation.  _European journal of operational research_ , 291(3):906–917, 2021.
  * Richard et al. (2021) Richard, A., Zollhöfer, M., Wen, Y., de la Torre, F., and Sheikh, Y.  Meshtalk: 3d face animation from speech using cross-modality disentanglement.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2021.
  * Richardson et al. (2021) Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., and Cohen-Or, D.  Encoding in style: a stylegan encoder for image-to-image translation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 2287–2296, 2021.
  * Rombach et al. (2021) Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B.  High-resolution image synthesis with latent diffusion models.  _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 10674–10685, 2021.  URL https://api.semanticscholar.org/CorpusID:245335280.
  * Rössler et al. (2019) Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M.  FaceForensics++: Learning to detect manipulated facial images.  In _International Conference on Computer Vision (ICCV)_ , 2019.
  * Rothe et al. (2018) Rothe, R., Timofte, R., and Gool, L. V.  Deep expectation of real and apparent age from a single image without facial landmarks.  _International Journal of Computer Vision_ , 126(2-4):144–157, 2018.
  * Ruhs (2024) Ruhs, H.  Facefusion, May 2024.  URL https://github.com/facefusion/facefusion.  original-date: 2023-08-17T19:59:55Z.
  * Ruiz et al. (2022) Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K.  Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.  _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 22500–22510, 2022.  URL https://api.semanticscholar.org/CorpusID:251800180.
  * Sabir et al. (2019) Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., and Natarajan, P.  Recurrent convolutional strategies for face manipulation detection in videos.  _Interfaces (GUI)_ , 3(1):80–87, 2019.
  * Saharia et al. (2022) Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., Salimans, T., Ho, J., Fleet, D. J., and Norouzi, M.  Photorealistic text-to-image diffusion models with deep language understanding.  _ArXiv_ , abs/2205.11487, 2022.  URL https://api.semanticscholar.org/CorpusID:248986576.
  * Scarpiniti et al. (2021) Scarpiniti, M., Comminiello, D., Uncini, A., and Lee, Y.-C.  Deep recurrent neural networks for audio classification in construction sites.  In _2020 28th European Signal Processing Conference (EUSIPCO)_ , pp. 810–814, 2021.  doi: 10.23919/Eusipco47968.2020.9287802.
  * Shen et al. (2017) Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerry-Ryan, R. J., Saurous, R. A., Agiomyrgiannakis, Y., and Wu, Y.  Natural tts synthesis by conditioning wavenet on mel spectrogram predictions.  _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 4779–4783, 2017.  URL https://api.semanticscholar.org/CorpusID:206742911.
  * Shen et al. (2022) Shen, S., Li, W., Zhu, Z., Duan, Y., Zhou, J., and Lu, J.  Learning dynamic facial radiance fields for few-shot talking head synthesis.  In _ECCV_ , 2022.
  * Sisman et al. (2020) Sisman, B., Yamagishi, J., King, S., and Li, H.  An overview of voice conversion and its challenges: From statistical modeling to deep learning.  August 2020.
  * Stypułkowski et al. (2024) Stypułkowski, M., Vougioukas, K., He, S., Zieba, M., Petridis, S., and Pantic, M.  Diffused heads: Diffusion models beat gans on talking-face generation.  pp. 5091–5100, 2024.
  * Sun et al. (2023) Sun, Y., Zhang, Z., Echizen, I., Nguyen, H. H., Qiu, C., and Sun, L.  Face forgery detection based on facial region displacement trajectory series.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pp. 633–642, 2023.
  * Suwajanakorn et al. (2017) Suwajanakorn, S., Seitz, S. M., and Kemelmacher-Shlizerman, I.  Synthesizing obama: learning lip sync from audio.  _ACM Transactions on Graphics (ToG)_ , 36(4):1–13, 2017.
  * Swathi & Sk (2021) Swathi, P. and Sk, S.  Deepfake creation and detection: A survey.  In _2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA)_ , pp. 584–588. IEEE, 2021.
  * Tan et al. (2021) Tan, X., Qin, T., Soong, F., and Liu, T.-Y.  A survey on neural speech synthesis, 2021.
  * Tariang et al. (2024) Tariang, D., Corvi, R., Cozzolino, D., Poggi, G., Nagano, K., and Verdoliva, L.  Synthetic image verification in the era of generative ai: What works and what isn’t there yet.  _arXiv preprint arXiv:2405.00196_ , 2024.
  * Thies et al. (2015) Thies, J., Zollhöfer, M., Nießner, M., Valgaerts, L., Stamminger, M., and Theobalt, C.  Real-time expression transfer for facial reenactment.  _ACM Transactions on Graphics (TOG)_ , 34(6), 2015.
  * Thies et al. (2016) Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., and Nießner, M.  Face2face: Real-time face capture and reenactment of rgb videos.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2387–2395, 2016.
  * Thies et al. (2019) Thies, J., Zollhöfer, M., and Nießner, M.  Deferred neural rendering: Image synthesis using neural textures.  _ACM Transactions on Graphics (TOG)_ , 38(4):1–12, 2019.
  * Thies et al. (2020) Thies, J., Elgharib, M., Tewari, A., Theobalt, C., and Nießner, M.  Neural voice puppetry: Audio-driven facial reenactment.  In _European Conference on Computer Vision_ , pp. 716–731. Springer, 2020.
  * Tokuda et al. (2000) Tokuda, K., Yoshimura, T., Masuko, T., Kobayashi, T., and Kitamura, T.  Speech parameter generation algorithms for hmm-based speech synthesis.  _2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100)_ , 3:1315–1318 vol.3, 2000.  URL https://api.semanticscholar.org/CorpusID:9372431.
  * Tov et al. (2021) Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., and Cohen-Or, D.  Designing an encoder for stylegan image manipulation.  _ACM Transactions on Graphics (TOG)_ , 2021.
  * Trinh Van et al. (2022) Trinh Van, L., Dao, T., Le, T., and Castelli, E.  Emotional speech recognition using deep neural networks.  _Sensors_ , 22:1414, 02 2022.  doi: 10.3390/s22041414.
  * Umur Aybars Ciftci (2020) Umur Aybars Ciftci, Ilke Demir, L. Y.  How do the hearts of deep fakes beat? deep fake source detection via interpreting residuals with biological signals.  2020\.
  * van den Oord et al. (2016) van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A. W., and Kavukcuoglu, K.  Wavenet: A generative model for raw audio.  _ArXiv_ , abs/1609.03499, 2016.  URL https://api.semanticscholar.org/CorpusID:6254678.
  * Vlasic et al. (2005) Vlasic, D., Brand, M., Pfister, H., and Popović, J.  Face transfer with multilinear models.  _ACM Trans. Graph._ , 24(3):426–433, jul 2005.  ISSN 0730-0301.  doi: 10.1145/1073204.1073209.  URL https://doi.org/10.1145/1073204.1073209.
  * Wang et al. (2020a) Wang, K., Wu, Q., Song, L., Yang, Z., Wu, W., Qian, C., He, R., Qiao, Y., and Loy, C. C.  Mead: A large-scale audio-visual dataset for emotional talking-face generation.  In _ECCV_ , August 2020a.
  * Wang et al. (2019) Wang, R., Juefei-Xu, F., Ma, L., Xie, X., Huang, Y., Wang, J., and Liu, Y.  Fakespotter: A simple yet robust baseline for spotting ai-synthesized fake faces.  _arXiv preprint arXiv:1909.06122_ , 2019.
  * Wang et al. (2020b) Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros, A. A.  Cnn-generated images are surprisingly easy to spot…for now.  In _CVPR_ , 2020b.
  * Wang et al. (2020c) Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., Juvela, L., Alku, P., Peng, Y.-H., Hwang, H.-T., Tsao, Y., Wang, H.-M., Maguer, S. L., Becker, M., Henderson, F., Clark, R., Zhang, Y., Wang, Q., Jia, Y., Onuma, K., Mushika, K., Kaneda, T., Jiang, Y., Liu, L.-J., Wu, Y.-C., Huang, W.-C., Toda, T., Tanaka, K., Kameoka, H., Steiner, I., Matrouf, D., Bonastre, J.-F., Govender, A., Ronanki, S., Zhang, J.-X., and Ling, Z.-H.  Asvspoof 2019: A large-scale public database of synthesized, converted and replayed speech, 2020c.
  * Wang et al. (2021) Wang, X., Li, Y., Zhang, H., and Shan, Y.  Towards real-world blind face restoration with generative facial prior.  In _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021.
  * Wang et al. (2023) Wang, Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., and Li, H.  Dire for diffusion-generated image detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 22445–22455, 2023.
  * Wang et al. (2022) Wang, Z. J., Montoya, E., Munechika, D., Yang, H., Hoover, B., and Chau, D. H.  Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models.  _arXiv:2210.14896 [cs]_ , 2022.  URL https://arxiv.org/abs/2210.14896.
  * Wodajo & Atnafu (2023) Wodajo, D. and Atnafu, S.  Deepfake video detection using generative convolutional vision transformer.  2023\.
  * Wu et al. (2022) Wu, H., Kuo, H.-C., Zheng, N., Hung, K.-H., yi Lee, H., Tsao, Y., Wang, H.-M., and Meng, H. M.  Partially fake audio detection by self-attention-based fake span discovery.  _ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 9236–9240, 2022.  URL https://api.semanticscholar.org/CorpusID:246823684.
  * Xu et al. (2022a) Xu, C., Zhang, J., Hua, M., He, Q., Yi, Z., and Liu, Y.  Region-aware face swapping.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 7632–7641, 2022a.
  * Xu et al. (2022b) Xu, Y., Deng, B., Wang, J., Jing, Y., Pan, J., and He, S.  High-resolution face swapping via latent semantics disentanglement, 2022b.
  * Xu et al. (2022c) Xu, Z., Zhou, H., Hong, Z., Liu, Z., Liu, J., Guo, Z., Han, J., Liu, J., Ding, E., and Wang, J.  Styleswap: Style-based generator empowers robust face swapping.  In _European Conference on Computer Vision_ , pp. 661–677. Springer, 2022c.
  * Yamagishi et al. (2021) Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., and Delgado, H.  Asvspoof 2021: accelerating progress in spoofed and deepfake speech detection, 2021.
  * Yan et al. (2023) Yan, Z., Zhang, Y., Yuan, X., Lyu, S., and Wu, B.  Deepfakebench: A comprehensive benchmark of deepfake detection.  In _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2023.  URL https://openreview.net/forum?id=hizSx8pf0U.
  * Yang et al. (2021) Yang, T., Ren, P., Xie, X., and Zhang, L.  Gan prior embedded network for blind face restoration in the wild.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 672–681, 2021.
  * Yang et al. (2019) Yang, X., Li, Y., and Lyu, S.  Exposing deep fakes using inconsistent head poses.  In _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 8261–8265. IEEE, 2019.
  * Yang et al. (2020) Yang, Y., Shillingford, B., Assael, Y., Wang, M., Liu, W., Chen, Y., Zhang, Y., Sezener, E., Cobo, L. C., Denil, M., et al.  Large-scale multilingual audio visual dubbing.  _arXiv preprint arXiv:2011.03530_ , 2020.
  * Yi et al. (2022) Yi, J., Fu, R., Tao, J., Nie, S., Ma, H., Wang, C., Wang, T., Tian, Z., Bai, Y., Fan, C., Liang, S., Wang, S., Zhang, S., Yan, X., Xu, L., Wen, Z., Li, H., Lian, Z., and Liu, B.  Add 2022: the first audio deep synthesis detection challenge, 2022.
  * Yi et al. (2023a) Yi, J., Tao, J., Fu, R., Yan, X., Wang, C., Wang, T., Zhang, C. Y., Zhang, X., Zhao, Y., Ren, Y., Xu, L., Zhou, J., Gu, H., Wen, Z., Liang, S., Lian, Z., Nie, S., and Li, H.  Add 2023: the second audio deepfake detection challenge, 2023a.
  * Yi et al. (2023b) Yi, J., Wang, C., Tao, J., Zhang, X., Zhang, C. Y., and Zhao, Y.  Audio deepfake detection: A survey, 2023b.
  * Yi et al. (2024) Yi, J., Wang, C., Tao, J., Zhang, C. Y., Fan, C., Tian, Z., Ma, H., and Fu, R.  Scenefake: An initial dataset and benchmarks for scene fake audio detection, 2024.
  * Yu et al. (2015) Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., and Xiao, J.  Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.  _arXiv preprint arXiv:1506.03365_ , 2015.
  * Yu et al. (2021) Yu, P., Xia, Z., Fei, J., and Lu, Y.  A survey on deepfake video detection.  _IET Biometrics_ , 10(6):607–624, April 2021.  ISSN 2047-4946.  doi: 10.1049/bme2.12031.  URL http://dx.doi.org/10.1049/bme2.12031.
  * Zaman et al. (2023) Zaman, K., Sah, M., Direkoglu, C., and Unoki, M.  A survey of audio classification using deep learning.  _IEEE Access_ , 11:106620–106649, 2023.  doi: 10.1109/ACCESS.2023.3318015.
  * Zhang et al. (2019) Zhang, X., Karaman, S., and Chang, S.-F.  Detecting and simulating artifacts in gan fake images.  In _2019 IEEE international workshop on information forensics and security (WIFS)_ , pp. 1–6. IEEE, 2019.
  * Zhang et al. (2023) Zhang, Z., Hu, Z., Deng, W., Fan, C., Lv, T., and Ding, Y.  Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video.  _AAAI_ , 2023.
  * Zhao et al. (2023) Zhao, Y., Yi, J., Tao, J., Wang, C., Zhang, X., and Dong, Y.  Emofake: An initial dataset for emotion fake audio detection, 2023.
  * Zhou & Zafarani (2020) Zhou, X. and Zafarani, R.  A survey of fake news: Fundamental theories, detection methods, and opportunities, September 2020.  ISSN 1557-7341.  URL http://dx.doi.org/10.1145/3395046.
  * Zhou et al. (2018) Zhou, Y., Xu, Z., Landreth, C., Kalogerakis, E., Maji, S., and Singh, K.  Visemenet: Audio-driven animator-centric speech animation.  _ACM Transactions on Graphics (TOG)_ , 37(4):1–10, 2018.
  * Zhou et al. (2020) Zhou, Y., Han, X., Shechtman, E., Echevarria, J., Kalogerakis, E., and Li, D.  Makelttalk: speaker-aware talking-head animation.  _ACM Transactions on Graphics (TOG)_ , 39(6):1–15, 2020.
  * Zhu et al. (2017) Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 2223–2232, 2017.
  * Zi et al. (2020) Zi, B., Chang, M., Chen, J., Ma, X., and Jiang, Y.-G.  Wilddeepfake: A challenging real-world dataset for deepfake detection.  In _Proceedings of the 28th ACM international conference on multimedia_ , pp. 2382–2390, 2020.