# AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit
Multimodal Inconsistency for Video Deepfake Detection

Sahibzada Adil Shahzad, Ammarah Hashmi, Yan-Tsung Peng, Yu Tsao, , Hsin-Min
Wang Sahibzada Adil Shahzad is with the Social Networks and Human-Centered
Computing Program, Taiwan International Graduate Program, Academia Sinica,
Taipei 11529, Taiwan, and also with the Department of Computer Science,
National Chengchi University, Taipei 11605, Taiwan. (e-mail:
adilshah275@iis.sinica.edu.tw). Ammarah Hashmi is with the Social Networks and
Human-Centered Computing Program, Taiwan International Graduate Program,
Academia Sinica, Taipei 11529, Taiwan, and also with the Institute of
Information Systems and Applications, National Tsing Hua University, Hsinchu
30013, Taiwan. (e-mail: hashmiammarah0@gmail.com). Yan-Tsung Peng is with the
Department of Computer Science, National Chengchi University, Taipei, Taiwan.
(e-mail<EMAIL_ADDRESS>Yu Tsao is with the Research Center for
Information Technology Innovation, Academia Sinica, Taipei 11529, Taiwan.
(e-mail: yu.tsao@citi.sinica.edu.tw). Hsin-Min Wang is with the Institute of
Information Science, Academia Sinica, Taipei 11529, Taiwan. (e-mail:
<EMAIL_ADDRESS>
###### Abstract

Multimodal manipulations (also known as audio-visual deepfakes) make it
difficult for unimodal deepfake detectors to detect forgeries in multimedia
content. To avoid the spread of false propaganda and fake news, timely
detection is crucial. The damage to either modality (i.e., visual or audio)
can only be discovered through multi-modal models that can exploit both pieces
of information simultaneously. Previous methods mainly adopt uni-modal video
forensics and use supervised pre-training for forgery detection. This study
proposes a new method based on a multi-modal self-supervised-learning (SSL)
feature extractor to exploit inconsistency between audio and visual modalities
for multi-modal video forgery detection. We use the transformer-based SSL pre-
trained Audio-Visual HuBERT (AV-HuBERT) model as a visual and acoustic feature
extractor and a multi-scale temporal convolutional neural network to capture
the temporal correlation between the audio and visual modalities. Since AV-
HuBERT only extracts visual features from the lip region, we also adopt
another transformer-based video model to exploit facial features and capture
spatial and temporal artifacts caused during the deepfake generation process.
Experimental results show that our model outperforms all existing models and
achieves new state-of-the-art performance on the FakeAVCeleb and DeepfakeTIMIT
datasets.

###### Index Terms:

Deepfakes, Deepfake detection, Audio-Visual, Lip Syn, Inconsistency, Video
Forgery, Audio-Visual Deepfake Detection

Figure 1: Example video frames produced by Faceswap, Faceswap-wav2lip, Fsgan,
Fsgan-wav2lip, RTVC (Real-Time Voice Clone), and Wav2lip manipulation methods
and the corresponding real video frames. Those video frames and audio
waveforms framed in red are fake, while those framed in green are real.

##  I Introduction

With smartphones, social networks, and high-speed internet, it is now at one’s
fingertips to capture, upload, and share content without any delays or fees.
However, this convenience makes the spread of deepfakes a heavy social cost.
The term “deepfake” encompasses synthetic media such as images, videos, and
audio. While there are many benefits to generating content through artificial
intelligence (AI) technology, it also has a dark side. AI-generated content is
often used for unethical objectives and malicious purposes, including
pornography, fraudulent activities, and political defamation. Social media
platforms are conduits for this type of manipulated content, and they often
have no filtering mechanisms to prevent its rapid spread.

Face forgery detection is challenging due to advancements in deepfake
generation methods. The realistic nature of deepfake videos makes it difficult
for forgery detection methods to identify video deepfakes based on artifacts.
The image and video forensics community has developed specific methods and
algorithms to detect forgery in image and video content. Similarly, the audio
forensics research community has developed synthetic speech detectors for
detecting spoofed audio. Due to their unimodal nature, these detectors fail
when the manipulated modality remains unseen during training. Spoofed audio
can evade visual deepfake detectors, while audio deepfake detectors cannot
catch visual deepfakes. Recently, the multimedia forensics community has begun
to pay attention to more realistic deepfake audio-visual content. To further
facilitate this audio-visual deepfake detection technology, various models
pre-trained through self-supervised-learning (SSL) have been fine-tuned on
deepfake video datasets to detect manipulation.

Pre-trained SSL models have recently emerged and achieved success in various
downstream tasks. Audio-Visual HuBERT (AV-HuBERT) [1] is an SSL-based audio-
visual representation learning model that achieves state-of-the-art
performance in lip reading, audio-visual speech enhancement [2], audio-visual
speech separation [2], and audio-visual speech recognition [3]. Motivated by
its state-of-the-art performance in multiple downstream tasks, we leverage AV-
HuBERT for feature extraction to capture the inconsistency between the mouth
region of interest and the corresponding audio modality for the downstream
task of audio-visual deepfake detection. Since this model detects deepfake
videos by examining speech-lip synchronization and is an extension of previous
work in [4], it is called AV-Lip-Sync+. Considering that lip feature-based
deepfake detectors may fail when the lip region is not manipulated or only
slightly manipulated, to enhance our audio-visual forgery detection model, we
employ a transformer-based visual encoder to exploit whole-face features to
assist our proposed audio-visual deepfake detector. By integrating powerful
audio-visual representations, speech-lip synchronization features, and
spatiotemporal facial features, the proposed system achieves state-of-the-art
performance on two multimodal deepfake datasets. Our main contributions are as
follows.

  * •

We adopt a transformer-based audio-visual feature extraction method instead of
the convolutional neural network (CNN)-based feature extraction method widely
used in existing video forgery detection systems.

  * •

The proposed system can effectively capture audio-visual correlation and
synchronization between audio and video modalities using the pre-trained SSL
model.

  * •

The proposed system achieves state-of-the-art performance on two multimodal
deepfake datasets and exhibits robustness to various deepfake manipulation
techniques.

##  II Related Work

In this section, we first briefly review common deepfake video generation
techniques, and then introduce state-of-the-art deepfake detection methods.

###  II-A Deepfake Generation

Figure 1 shows examples of real and various fake videos in the FakeAVCeleb
dataset [5]. Deepfake content can be categorized into four main types: visual,
textual, audio, and audio-visual (multimodal) deepfakes. Visual forgery [6, 7,
8, 9, 10, 11] involves the manipulation of the entire face, facial
expressions, or lip movements using deepfake generation models such as
Faceswap [6], Fsgan [12], and wav2lip [13]. The manipulation in visual
deepfakes is limited to the visual modality of the video, while the audio
content is not affected. Visual manipulation can also involve the integration
of two or more deepfake generation techniques, such as Faceswap-wav2lip and
Fsgan-wav2lip, to simultaneously manipulate face and lip movements and align
the latter with the audio track. Text deepfake refers to seemingly real text
information manipulated or generated by AI. Examples include fake news
articles [14], fake online reviews [15], spam email generation [16], and
synthetic text that incites violence [17]. Audio deepfake involves the
acoustic manipulation of audio for malicious purposes using voice conversion
and text-to-speech techniques (such as WaveNet [18], WaveGlow [19], MelNet
[20], and Tacotron [21]). SV2TTS [22] is a real-time voice cloning tool that
synthesizes fabricated audio content without altering visual content such as
video frames.

The most challenging and realistic deepfakes are audio-visual deepfakes, also
known as multimodal deepfakes. In these instances, attackers use video and
audio deepfake generation algorithms to manipulate both modalities. For
example, the fake video and fake audio (FVFA) category of deepfake videos in
the FakeAVCeleb dataset contains visual manipulations in the entire face or
lip region using Faceswap, Fsgan, or wav2lip deepfake generation techniques,
while the audio counterpart is cloned or synthetic speech generated by
acoustic manipulation techniques. Our study focuses on detecting audio-visual
deepfakes.

###  II-B Deepfake Detection

High-quality AI-generated content is useful in many ways, but it also comes at
a cost, and timely detection is crucial to avoid any harm to society. To avoid
the spread of misinformation and disinformation and to protect the reputations
and privacy of individuals, we need automated methods to promptly detect
deepfake content in our widespread digital world. Academia and industry have
made considerable progress in using deep learning-based methods to detect
forged multimedia content. These deep learning-based deepfake detection
methods can be roughly divided into two major categories: unimodal and
multimodal methods. Unimodal forgery detection models are specifically
designed to detect forgery in one modality: video or audio. Video forgery
detectors can be divided into three types [23]: physiological, visual
artifact-based, and high-level feature-based methods. In physiological
methods, researchers have exploited abnormal eye blinks [24] and incoherent
head poses [25]. Visual artifact-based methods analyze anomalies and
irregularities such as unnatural facial movements, illumination variation,
blended face boundaries, and misalignment of video content. High-level-
feature-based deepfake detectors extract high-level features that are immune
to video processing (e.g., compression). Lipforensics [26] is an example of a
high-level feature-based deepfake detector that leverages a lip-reading-based
model to detect abnormal lip movements for video deepfake detection. Video
forgery detection models have achieved excellent results, thanks to various
rich datasets available for model training, such as DeepfakeTIMIT [27], UADVF
[25], FaceForensics++ (FF++) [28], Celeb-DF [29], Google DFD [28], DFDC [30],
DeeperForensics [31], KoDF [32], and the recently released multimodal
FakeAVCeleb dataset [5]. Well-known unimodal fake video detection models
include Capsule Forensics [33], HeadPose [34], Xception [35], LipForensics
[26], Meso-4 [36], and MesoInception-4 [36]. On the other hand, to trick
automatic speaker verification systems, attackers can develop audio spoofing
attacks or replay attacks using only a few minutes of a person’s recorded
speech, which makes these systems vulnerable. Similar to video deepfakes,
audio spoofing attacks are a major challenge that must be solved. In response
to audio spoofing attacks on automatic speaker verification systems, the audio
forensics community has proposed various traditional and deep learning-based
methods [37]. Most traditional systems use short-term power spectrum, short-
term phase spectrum, and long-term spectral features as front-end features.
Backend classifiers are based on traditional machine learning models or
ensemble models. Deep learning-based models can be roughly divided into multi-
pass, end-to-end, and ensemble models. Using different hand-crafted acoustic
features or raw waveforms, audio spoofing detection methods [38, 39, 40] can
differentiate between genuine speech and spoofed speech.

Recently, deepfake technology has expanded from unimodal manipulation to
multimodal manipulation (such as audio-visual manipulation). Unimodal deepfake
video and spoofed audio detectors are insufficient to detect these audio-
visual manipulations. To solve this problem, the multimedia forensics
community has proposed several deep learning-based methods to capture unimodal
or multimodal manipulations. Some multimodal methods exploit faces as visual
features and mel-frequency cepstral coefficients (MFCCs) as acoustic features
[41] or detect forged videos based on facial and speech emotions [42]. To
capture the intrinsic synchronization between visual and acoustic modalities,
Facebook AI [43] proposed a novel and general CNN-based method to jointly
learn audio and video features. AVFakeNet audio-visual forgery detection
method [44] uses Swin Transformer as the feature extraction module. Instead of
learning audio-visual artifacts, Cozzolino et al. [45] used contrastive
learning to learn discriminative audio-visual identity features. Localized
Audio Visual DeepFake (LAV-DF) [46] introduces a content-driven deepfake
detector for learning temporal forgery localization. In [47], a multimodal
deepfake detector based on ensemble learning is introduced to leverage
multiple learners and make decisions based on hard voting. The main
disadvantage of ensemble learning is its time-consuming process due to
involving training multiple models. In [4], lip synchronization features based
on the difference between the extracted lip sequence and the synthetic lip
sequence are used to detect audio-visual deepfakes. This model requires the
wav2lip module to generate lip sequences from audio, which increases model
complexity, training, and inference time. The field of multimodal forgery
detection is still less explored, and further research is needed to determine
how to effectively use audio-visual information to detect forgery in any
modality of multimedia content.

Figure 2:  The proposed AV-Lip-Sync+ architecture for multimodal forgery
detection. The lip image sequence is extracted from the input video, while the
log filterbank energies are extracted from the audio track. The SSL pre-
trained model consists of ResNet-18 for visual feature extraction, FFN for
acoustic feature extraction, and a transformer encoder to extract
spatiotemporal information from the visual and acoustic features. The
extracted audio-visual features are further mapped through multi-scale
temporal convolution network (MS-TCN), temporal pooling, and linear layer for
classification.  Figure 3: Embedding space representation of lip embedding
${\vec{F}_{v}}$ and speech embedding ${\vec{F}_{a}}$ for a real video (top)
and a deepfake video (bottom).

##  III Methodology

As shown in Fig. 2, our proposed AV-Lip-Sync+ model consist of three feature
extractors, namely a lip image feature extractor, an acoustic feature
extractor, and an audio-visual feature extractor. These feature extraction
modules are followed by the Sync-Check Module, Feature Fusion Module, and
Temporal Convolutional Network to capture the temporal correlation between
visual and audio features. Finally, a temporal pooling layer and a linear
layer are used for classification. Our feature extractors are based on the AV-
HuBERT model [1] pre-trained on the LRS3 dataset [48]. AV-HuBERT will be fine-
tuned when training the detection model on the multimodal deepfake datasets.

###  III-A Audio-Visual Feature Extractor

As shown in Fig. 2, the audio-visual feature extractor consists of a
Resnet-18, a light-weight Feed Forward Network (FFN), and a transformer
encoder. The 2-D Resnet-18 with front-end 3-D convolutional layers is used to
extract lip-based visual features from each input lip image frame. The FFN is
used to extract frame-level acoustic features from the input log filterbank
energies of the audio waveform. These frame-level visual and acoustic features
are concatenated along the feature dimension and fed to the transformer
decoder, which generates a 768-D audio-visual embedding sequence
$\vec{F}_{av}$ via

$\vec{F}_{av}={F_{\theta_{e}}(v,a)},$ (1)

where $v$ and $a$ represent frame-level visual features and acoustic features,
respectively.

###  III-B Lip Image Feature Extractor

In the lip image feature extractor, the output of Resnet-18 is fed to the
transformer encoder, which generates the lip image embedding sequence
$\vec{F}_{v}$ via

$\vec{F}_{v}={F_{\theta_{e}}(v,a_{dropout})},$ (2)

where $a_{dropout}$ indicates audio dropout, i.e., the audio information is
not used.

###  III-C Acoustic Feature Extractor

In the acoustic feature extractor, the output of FFN is fed to the transformer
encoder, which generates the acoustic feature embedding sequence $\vec{F}_{a}$
via

$\vec{F}_{a}={F_{\theta_{e}}(v_{dropout},a)},$ (3)

where $v_{dropout}$ indicates video dropout, i.e., the visual information is
not used.

###  III-D Sync-Check Module

Because deepfake technology is not yet mature enough to generate synchronized
and perfect audio-visual deepfakes, there is often a disharmony between the
visual and audio modalities of deepfake videos. As shown in Fig. 3, in a low-
dimensional embedding space, the embeddings of lip movements in a real video
should be close to the embeddings of the audio in this video. In contrast, for
a fake video, we expect the video and audio embeddings to be relatively
separated. This idea motivates us to exploit synchronization between lip
movements and speech to detect forgeries in deepfake videos. Therefore, the
input of the sync-check module includes the output representation
${\vec{F}_{v}}$ of the lip image feature extractor and the output
representation ${\vec{F}_{a}}$ of the acoustic feature extractor. For each
time frame $i$, we calculate the absolute difference between the corresponding
lip embedding ${\vec{F}_{vi}}$ and audio embedding ${\vec{F}_{ai}}$ to capture
the frame-level difference between the visual and acoustic modalities.
Consequently, the output of the sync-check module is the sync-based feature
vector sequence ${\vec{F}_{sync}}$ calculated as

$\vec{F}_{sync}=\\{|{\vec{F}_{vi}}-{\vec{F}_{ai}}|\\}_{i=1}^{T},$ (4)

where $T$ is the number of frames in the input video.

###  III-E Feature Fusion Module

In addition to the sync-based feature vector sequence ${\vec{F}_{sync}}$, the
robust audio-visual feature vector sequence $F_{av}$ obtained from the audio-
visual feature extractor also captures the correlation between the two
modalities. Therefore, we combine these two audio-visual representations for
multimodal forgery detection. In the feature fusion module, $F_{av}$ and
$F_{sync}$ are concatenated along the feature dimension to form a fusion
representation sequence $\vec{F}_{fusion}$ as

$\vec{F}_{fusion}={\vec{F}_{av}\oplus\vec{F}_{sync}},$ (5)

where $\oplus$ denotes the concatenation operation.

###  III-F Temporal Convolutional Network and Classifier

The temporal dynamics across the audio and visual frames contain important
information about the video content. To capture inter-modal and intra-modal
temporal correlations, we adopt the multi-scale temporal convolutional network
(MS-TCN) in [49]. Temporal convolution takes a sequence of frame-level feature
vectors and maps them into another sequence of the same dimension using one-
dimensional temporal convolution. The temporal convolutional network acts as a
sequence encoder, capturing short-term and long-term information by providing
the network with visibility across multiple time scales. MS-TCN is followed by
a temporal pooling layer and a linear layer for outputting the Real/Fake
probability given $\vec{F}_{fusion}$ as

$\hat{y}={F_{{\theta}_{m}}(\vec{F}_{fusion})},$ (6)

where $\hat{y}$ represents the probability of the target class.

###  III-G Model Training

The entire model is trained with the cross-entropy loss, defined as

${L(y,\hat{y})=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}\log\hat{y}_{i}+\left(1-y_{i}\right)\log\left(1-\hat{y}_{i}\right)]},$
(7)

where $N$ is the number of training samples, $y_{i}$ represents the ground
truth label of the $i$-th sample (0 or 1), and $\hat{y}_{i}$ represents the
class prediction probability of the $i$-th sample. During training, the pre-
trained front-end feature extractors and transformer encoder are fine-tuned,
while MS-TCN and the linear classifier are trained from scratch.

TABLE I: Statistics of Multimodal Fogery Datasets for Deepfake Detection. Datasets | Real Videos | Fake Videos | Manipulation Methods | No of Subjects | Visual Manipulation | Audio Manipulation
---|---|---|---|---|---|---
| FakeAVCeleb [5]
---
500 | 20000 |  | Faceswap, Fsgan,
---
wav2lip, RTVC
500 | Yes | Yes
| DeepfakeTIMIT [27]
---
320 | 320 | Faceswap | 32 | Yes | No

###  III-H AV-Lip-Sync+ with FE

As reported in [4], speech-lip synchronization based methods may not be good
at detecting fake videos generated by some visual manipulation models such as
Faceswap and FS-GAN. In these types of fake videos, the video contains only
visual manipulation, while the audio is genuine. Furthermore, the manipulation
does not necessarily occur in the lip region, but artifacts can be observed in
other regions or throughout the face, including face boundaries. Deepfake
detectors based on lip features may fail when the lip region is not
manipulated, or when there is little manipulation in the mouth region. To
address these issues, we use a face encoder to utilize the entire face
features to enhance our proposed deepfake detector and make it more robust and
generalizable to deep face manipulation techniques. To this end, we employ the
pre-trained Video Vision Transformer (ViViT) [50] as the face encoder to
extract the spatiotemporal face features. Using tubelet embeddings and spatial
and temporal transformers, the face encoder extracts inter- and intra-frame
information from video content. The output of the face encoder is a single-
vector representation, which is fed to a linear layer for classification. The
face-based deepfake detection model is trained on multimodal deepfake dataset
using cross-entropy loss. During training, the pre-trained face encoder is
fine-tuned, while the linear classifier is trained from scratch.

The model that combines AV-Lip-Sync+ and the face encoder is called AV-Lip-
Sync+ with FE. The extracted single-vector representation of the face encoder
and the representation obtained from the AV-Lip-Sync+ model are concatenated
and fed to a two-layer linear classifier. During training, the pre-trained
face encoder and AV-Lip-Sync+ are fixed, and only the classifier is trained
using cross-entropy loss.

##  IV Experiments

We conducted experiments on two datasets: FakeAVCeleb [5] and DeepfakeTIMIT
[27]. Unlike other unimodal audio or video deepfake datasets, these two
datasets contain both audio and visual modalities, and their fake samples
contain audio and/or visual manipulations. Furthermore, the faces in the
videos in both datasets are frontal, which makes them suitable for lip frame
extraction as visual input to the proposed model. The statistics of the two
dataset are shown in Table I.

###  IV-A Datasets

####  IV-A1 FakeAVCeleb

The FakeAVCeleb dataset is an audio-visual dataset released in 2021
specifically designed for the deepfake detection task. It is based on a
collection of 500 YouTube videos featuring 500 celebrities from diverse ethnic
regions including South Asia, East Asia, Africa, Europe and America. Fake
videos are generated from these 500 real videos using the Faceswap [6], Fsgan
[12], wav2lip [13], and real-time voice cloning (SV2TTS) [22] manipulation
methods and their combinations. Several examples are shown in Fig. 1. In the
case of Faceswap-wav2lip, the video is manipulated using both Faceswap and
wav2lip manipulation methods. Similarly, in the case of Fsgan-wav2lip, the
video is generated using a combination of Fsgan and wav2lip. The videos
manipulated by wav2lip include two types, namely Fake-Video-Real-Audio (FVRA)
and Fake-Video-Fake-Audio (FVFA). Wav2lip FVRA videos contain manipulated lips
and real audio. In the case of wav2lip FVFA, in addition to lip manipulation,
a real-time voice cloning method is also used to manipulate the audio. In
total, the dataset contains 500 real videos and more than 20000 forged videos.

Following [4, 47], we used multiple test sets, namely Faceswap, Fsgan, RTVC,
wav2lip, Faceswap-wav2lip, and Fsgan-wav2lip. Furthermore, two other major and
diverse test sets are Test-set-1 and Test-set-2. In Test-set-1, the number of
samples is the same for all manipulation methods, while in Test-set-2, the
number of samples of RVFA (Real-Video-Fake-Audio), FVRA (Fake-Video-Real-
Audio), and FVFA (Fake-Video-Fake-Audio) in the fake class is the same. The
training-test spit is based on the number of subjects in the dataset. The
training set and test set contain real and fake videos corresponding to 430
subjects and 70 subjects respectively. Furthermore, all the test sets are
balanced in terms of real and fake videos and contain 70 videos per class
(real and fake). The training set contains only 430 real videos, which is
significantly less than the number of fake videos. If the imbalance problem is
not properly resolved, the experimental results will be biased. To eliminate
the imbalance problem, we borrowed real videos from the VoxCeleb1 dataset [51]
to make the training data of the real and fake classes more balanced.

####  IV-A2 DeepfakeTIMIT

The DeepfakeTIMIT dataset contains 320 audio-visual human speech recordings
from 32 subjects and is a subset of the VidTIMIT dataset [52]. Each subject
has 10 videos. For each video, the corresponding fake video is generated by
the Faceswap manipulation method. The audio in both real and fake videos is
always real. Since the dataset is small, we performed 5-fold cross-validation
on it and evaluated the average performance.

###  IV-B Preprocessing

Our model mainly utilizes lip and audio features for multimodal forgery
detection. For this purpose, the initial step is to extract the lip region
from the frontal face using facial landmarks. We leveraged a pre-trained CNN-
based face detector from the Dlib toolkit [53]. The lip image sequence
extracted from the input video is $96\times 96$ RGB pixels. Before the lip
image sequences are fed into the model, they are converted to grayscale. The
input shape of the extracted lip features is $C\times F\times H\times W$,
where $C$ represents the number of channels, $F$ denotes the number of frames,
and $H$ and $W$ represent the height and width of each frame, respectively. In
addition, for the audio modality, the waveform is extracted from the video and
then converted to the log filterbank energies as the acoustic input of the
model.

For facial feature extraction, among the different variants of ViViT in [50],
we selected the best performing factorized encoder model as the face encoder.
The visual encoder has two transformer blocks, namely a spatial transformer
and a temporal transformer. The input to the model is short fixed-length video
clips from the entire video. The number of frames in a clip is $16$, the input
frame size is $224\times 224$, the patch size is $16$, the number of input
channels is $3$, and the embedding dimension is $768$. We used the tubelet
embedding method. The ViViT model is pre-trained on the kinetics dataset [54].

###  IV-C Hyper-parameters in Training

Our model was trained by the Adam optimizer with a learning rate of 0.00001.
We trained the model for $30$ epochs using an early stopping strategy. We
added 3570 real videos from the VoxCeleb1 dataset [51] to the real class and
applied oversampling during training to deal with the data imbalance problem.

TABLE II: Evaluation results of AV-Lip-Sync+ on the FakeAVCeleb dataset.

Test set | Class | Precision | Recall | F1-score | Accuracy
---|---|---|---|---|---
Faceswap | Real | 0.85 | 0.99 | 0.91 | 0.91
Fake | 0.98 | 0.83 | 0.90
Faceswap_wav2lip | Real | 1.00 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 1.00 | 0.99
Fsgan | Real | 0.86 | 0.99 | 0.92 | 0.91
Fake | 0.98 | 0.84 | 0.91
Fsgan_wav2lip | Real | 1.00 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 1.00 | 0.99
RTVC | Real | 0.96 | 0.99 | 0.97 | 0.97
Fake | 0.99 | 0.96 | 0.97
Wav2lip | Real | 1.00 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 1.00 | 0.99
Test-set-1 | Real | 0.93 | 0.99 | 0.96 | 0.96
Fake | 0.98 | 0.93 | 0.96
Test-set-2 | Real | 0.99 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 0.99 | 0.99

###  IV-D Evaluation Metrics

We evaluated our proposed multimodal forgery detector using Precision, Recall,
F1-score, and Accuracy, which are calculated as follows:

${Accuracy}={\frac{TP+TN}{TP+TN+FP+FN}},$ (8)
${Precision}={\frac{TP}{TP+FP}},$ (9) ${Recall}={\frac{TP}{TP+FN}},$ (10)
${F1}={\frac{2\times Precision\times Recall}{Precision+Recall}},$ (11)

where ${TP}$, ${TN}$, ${FP}$, and ${FN}$ stand for True Positive, True
Negative, False Positive, and False Negative, respectively. In addition, the
receiver operating characteristics (ROC) curve and the area under the curve
(AUC) were also used. For a fair comparison, We report video-level performance
rather than frame-level performance.

Figure 4: ROC curves and AUC scores of the proposed AV-Lip-Sync+ method on various test sets of the FakeAVCeleb dataset. Figure 5: t-SNE visualization on the embedding space of audio-visual representations from AV-Lip-Sync+. TABLE III: Evaluation results of different audio-visual deepfake video detection models on the FakeAVCeleb dataset. Model | Faceswap | Faceswap_wav2lip | Fsgan | Fsgan_wav2lip | RTVC | Wav2lip | Test-set-1 | Test-set-2
---|---|---|---|---|---|---|---|---
| Lipforensics [26]
---
0.91 | 0.96 | 0.89 | 0.96 | 0.49 | 0.96 | 0.86 | 0.76
| Multimodal Ensemble-Learning [47]
---
0.72 | 0.99 | 0.88 | 0.99 | 0.89 | 0.93 | 0.89 | 0.89
| AV-Lip-Sync [4]
---
0.86 | 0.98 | 0.82 | 0.97 | 0.89 | 0.96 | 0.94 | 0.94
| AVH-TCN
---
0.85 | 0.98 | 0.93 | 0.98 | 0.96 | 0.98 | 0.94 | 0.98
| AVH-TCN-Sync
---
0.91 | 0.98 | 0.92 | 0.98 | 0.96 | 0.98 | 0.95 | 0.95
| AV-Lip-Sync+
---
0.91 | 0.99 | 0.91 | 0.99 | 0.97 | 0.99 | 0.96 | 0.99
| AV-Lip-Sync+ with FE
---
0.98 | 0.99 | 0.96 | 0.99 | 0.96 | 0.99 | 0.96 | 0.99

###  IV-E Results

####  IV-E1 Evaluation of the proposed AV-Lip-Sync+ model

Table II shows the performance of the proposed AV-Lip-Sync+ model evaluated on
the FakeAVCeleb dataset. The manipulation methods of Faceswap, Faceswap-
wav2lip, Fsgan, Fsgan-wav2lip, RTVC, and wav2lip are all seen during the model
training process. It is obvious that AV-Lip-Sync+ performed well on almost all
test sets except Faceswap and Fsgan. The main reason may be that the AV-HuBERT
feature extractor only extracts visual information from lip images and ignores
information outside the lip region. Figure 4 shows the ROC curves and
corresponding AUC scores for all test sets. The AUC scores under different
manipulation conditions are all above 0.96. Additionally, we visualize the
features of the AV-Lip-Sync+ model as shown in Fig. 5. The t-SNE feature
visualization confirms the descriminative ability of our proposed model. Only
very few Faceswap and Fsgan test samples were mixed with real samples, and all
test samples of other manipulation methods were perfectly classified as
“Fake”.

Facial features are crucial for detecting deepfake videos, as local or entire
facial regions may contain artifacts resulting from video manipulation. To
address the limitations of the proposed AV-Lip-Sync+ model on Faceswap and
Fsgan test samples, we used a ViViT-based face encoder to inject face
embeddings into an ensemble model (called Lip-Sync+ with FE). As shown in
Table III, the accuracy of Faceswap and Fsgan test sets increased from 0.91 to
0.98 and 0.96, respectively. The results show that providing supplementary
information to the proposed model can improve the detection of videos of faces
manipulated by Faceswap and Fsgan techniques.

We also compared our models (AV-Lip-Sync+ and AV-Lip-Sync+ with FE) with other
audio-visual deepfake video detection models. In Table III, AVH-TCN refers to
the model with AV-HuBERT feature extraction followed by MS-TCN, temporal
pooling and linear layer (i.e., AVH-TCN detects fake videos based on
$\vec{F}_{av}$), while AVH-TCN-Sync refers to the counterpart model for
detection based on the speech-lip synchronization feature $\vec{F}_{sync}$.
Both AVH-TCN and AVH-TCN-Sync are considered simplified versions of AV-Lip-
Sync+. As shown in Table III, both AVH-TCN and AVH-TCN-Sync generally
performed better than previous methods but worse than AV-Lip-Sync+ and AV-Lip-
Sync+ with FE. Our AV-Lip-Sync+ and AV-Lip-Sync+ with FE models achieve state-
of-the-art results in all test sets.

####  IV-E2 Comparison of different models

In this experiment, we compared our models with various existing unimodal,
multi-modal, fusion, and ensemble deepfake detection models on Test-set-2 of
the FakeAVCeleb dataset. The results are shown in Table IV. Several uni-modal,
multi-modal and ensemble models have been evaluated in [55], but the
performance of most models is unsatisfactorily. Unimodal video models were
trained using visual features only, unimodal audio models were trained with
MFCC features of the audio modality, and multi-modal and ensemble models were
trained with audio (MFCC features) and video (visual frames). Several recent
multi-modal, ensemble, and fusion models have achieved much better performance
than before, including Multimodaltrace [56], AVFakeNet [44], PVASS-MDD [57],
MIS-AVioDD [58], AVTENet [59] and our previous speech-lip synchronization
based model AV-Lip-Sync [4]. Clearly, our previous AV-Lip-Sync model is one of
the recent leading models, confirming the effectiveness of using speech-lip
synchronization for deepfake video detection. In order to check the
synchronization between speech and lip movements, AV-Lip-Sync requires
generating a synthesized lip sequence using the wav2lip generator and then
comparing it to the extracted lip sequence. AV-Lip-Sync+ not only eliminates
the need for the wav2lip synthetic lip generator, but also delivers
significant performance gains. AV-Lip-Sync+ and AV-Lip-Sync+ with FE both
achieved an accuracy of 0.99, which is a new state-of-the-art performance on
Test-set-2 of the FakeAVCeleb dataset. The results confirm the effectiveness
of using the pre-trained SSL AV-HuBERT model for audio-visual feature
extraction in deepfake video detection.

####  IV-E3 Ablation study

There are various strategies to train the proposed AV-Lip-Sync+ model. For
example, we can fix/freeze or fine-tune a pre-trained feature extractor, or
train all modules from scratch. To see the effect of different settings, we
first freeze the weights of the front-end audio-visual feature extractor and
transformer encoder, and train the temporal network and linear classifier. The
second strategy is to freeze the front-end audio-visual feature extraction
part and fine-tune the transformer encoder while training the temporal network
and linear classifier. Third, we fine-tune the front-end feature extraction
part and transformer encoder while training the temporal network and linear
classifier. Table V shows the performance comparison using different training
strategies. Some observations can be made. First, the results show that for
AVH-TCN, AVH-TCN-Sync, and AV-Lip-Sync+, fine-tuning the entire AV-HuBERT
model yields better results than fixing any specific module in multi-modal
deepfake detection. Second, the model that combines the audio-visual feature
$\vec{F}_{av}$ and the speech-lip synchronization feature $\vec{F}_{sync}$
(i.e., AV-LIP-Sync+) outperforms the models using $\vec{F}_{av}$ and
$\vec{F}_{sync}$ alone (i.e., AVH-TCN and AVH-TCN-Sync). For the AV-Lip-Sync+
with FE model, we compared two training strategies. First, we fine-tuned AV-
HuBERT and ViViT when training the temporal network and linear classifier (see
Jointly trained with linear layers). Second, we first train AV-HuBERT-based
and ViViT-based detectors separately, and then use them as fixed feature
extractors to train a linear classifier (see Fixed FE, trained only linear
layers). The results show that fine-tuning the pre-trained AV-HuBERT and ViViT
models individually can lead to better performance.

TABLE IV: Evaluation results of different deepfake video detection models on the FakeAVCeleb dataset. Type | Model | Modality | Class | Precision | Recall | F1-score | Accuracy
---|---|---|---|---|---|---|---
Uni-modal[55] | VGG16 | V | Real | 0.6935 | 0.8966 | 0.7821 | 0.8103
Fake | 0.8724 | 0.7750 | 0.8208
Uni-modal [55] | Xception | A | Real | 0.8750 | 0.6087 | 0.7179 | 0.7626
Fake | 0.7033 | 0.9143 | 0.7950
Uni-modal [26] | LipForensics | V | Real | 0.7000 | 0.9100 | 0.8000 | 0.7600
Fake | 0.8800 | 0.6100 | 0.7200
Ensemble (Soft-Voting)[55] | VGG16 | AV | Real | 0.6935 | 0.8966 | 0.7821 | 0.7804
Fake | 0.8948 | 0.6894 | 0.7788
Ensemble (Hard-Voting)[55] | VGG16 | AV | Real | 0.6935 | 0.8966 | 0.7821 | 0.7804
Fake | 0.8948 | 0.6894 | 0.7788
Multi-modal-1[55] | Multi-modal-1 | AV | Real | 0.000 | 0.000 | 0.000 | 0.5000
Fake | 0.496 | 1.000 | 0.663
Multi-modal-2[55] | Multi-modal-2 | AV | Real | 0.7100 | 0.5870 | 0.6430 | 0.6740
Fake | 0.6480 | 0.7600 | 0.7000
Multi-modal-3[55] | CDCN | AV | Real | 0.5000 | 0.0680 | 0.1200 | 0.5150
Fake | 0.5000 | 0.9400 | 0.6510
Multi-modal-4[41] | Not-made-for-each-other | AV | Real | 0.6200 | 0.9900 | 0.7600 | 0.6900
Fake | 0.9400 | 0.4000 | 0.5700
Multi-modal-Ensemble[47] | Ensemble Model | AV | Real | 0.83 | 0.99 | 0.90 | 0.89
Fake | 0.98 | 0.80 | 0.88
Multi-modal[56] | Multimodaltrace | AV | Real | - | - | - | 0.929
Fake | - | - | -
Ensemble[44] | AVFakeNet | AV | Real | - | - | - | 0.934
Fake | - | - | -
Fusion[60] | AVoiD-DF | AV | Real | - | - | - | 0.837
Fake | - | - | -
Fusion[57] | PVASS-MDD | AV | Real | - | - | - | 0.957
Fake | - | - | -
Fusion[58] | MIS-AVioDD | AV | Real | - | - | - | 0.962
Fake | - | - | -
Multi-modal-Ensemble[59] | AVTENet | AV | Real | 1.00 | 0.97 | 0.99 | 0.99
Fake | 0.97 | 1.00 | 0.99
Lip Sync Matters[4] | AV-Lip-Sync | AV | Real | 0.93 | 0.96 | 0.94 | 0.94
Fake | 0.96 | 0.93 | 0.94
Multi-modal (ours) | AV-Lip-Sync+ | AV | Real | 0.99 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 0.99 | 0.99
Multi-modal (ours) | AV-Lip-Sync+ with FE | AV | Real | 1.00 | 0.99 | 0.99 | 0.99
Fake | 0.99 | 1.00 | 0.99
TABLE V: Evaluation results of AV-Lip-Sync+ using different training/fine-
tuning strategies on the FakeAVCeleb dataset. Front-end FE and TE represent
the front-end feature extraction part and transformer encoder of the pre-
trained AV-HuBERT model, respectively.

Model | Faceswap | Faceswap_wav2lip | Fsgan | Fsgan_wav2lip | RTVC | Wav2lip | Test-set-1 | Test-set-2
---|---|---|---|---|---|---|---|---
| AVH-TCN
---
(Fixed Front-end FE and TE)
0.45 | 0.81 | 0.52 | 0.88 | 0.79 | 0.76 | 0.71 | 0.76
| AVH-TCN
---
(Fixed Front-end FE only)
0.71 | 0.94 | 0.76 | 0.96 | 0.90 | 0.94 | 0.87 | 0.93
| AVH-TCN
---
(Whole fine tune)
0.85 | 0.98 | 0.93 | 0.98 | 0.96 | 0.98 | 0.94 | 0.98
| AVH-TCN-Sync
---
(Fixed Front-end FE and TE)
0.50 | 0.90 | 0.55 | 0.89 | 0.84 | 0.74 | 0.76 | 0.78
| AVH-TCN-Sync
---
(Fixed Front-end FE only)
0.56 | 0.98 | 0.64 | 0.98 | 0.90 | 0.96 | 0.85 | 0.92
| AVH-TCN-Sync
---
(Whole fine tune)
0.91 | 0.98 | 0.92 | 0.98 | 0.96 | 0.98 | 0.95 | 0.95
| AV-Lip-Sync+
---
(Fixed Front-end FE and TE)
0.46 | 0.84 | 0.55 | 0.87 | 0.79 | 0.75 | 0.71 | 0.76
| AV-Lip-Sync+
---
(Fixed Front-end FE only)
0.70 | 0.95 | 0.74 | 0.95 | 0.93 | 0.93 | 0.87 | 0.94
| AV-Lip-Sync+
---
(Whole fine tune)
0.91 | 0.99 | 0.91 | 0.99 | 0.97 | 0.99 | 0.96 | 0.99
| AV-Lip-Sync+ with FE
---
(Jointly trained with linear layers)
0.96 | 0.98 | 0.97 | 0.98 | 0.91 | 0.96 | 0.96 | 0.95
| AV-Lip-Sync+ with FE
---
(Fixed FE, trained only linear layers)
0.98 | 0.99 | 0.96 | 0.99 | 0.96 | 0.99 | 0.96 | 0.99

TABLE VI: Evaluation results of AV-Lip-Sync+ on the DeepfakeTIMIT dataset.

Type | Model | Modality | Quality | AUC
---|---|---|---|---
Uni-modal [33] | Capsule | V | LQ | 0.7840
HQ | 0.7440
Uni-modal [61] | Multi-task | V | LQ | 0.6220
HQ | 0.5530
Uni-modal [25] | HeadPose | V | LQ | 0.5510
HQ | 0.5320
Uni-modal [62] | Two-stream | V | LQ | 0.8350
HQ | 0.7350
Uni-modal [63] | VA-MLP | V | LQ | 0.6140
HQ | 0.6210
Uni-modal [63] | VA-LogReg | V | LQ | 0.7700
HQ | 0.7730
Uni-modal [36] | Meso-4 | V | LQ | 0.8780
HQ | 0.6840
Uni-modal [28] | Xception-raw | V | LQ | 0.5670
HQ | 0.5400
Uni-modal [28] | Xception-c40 | V | LQ | 0.7580
HQ | 0.7050
Uni-modal [28] | Xception-c23 | V | LQ | 0.9590
HQ | 0.9440
Uni-modal [64] | FWA | V | LQ | 0.9990
HQ | 0.9320
Uni-modal [64] | DSP-FWA | V | LQ | 0.9990
HQ | 0.9970
Multi-modal [42] | Emotions Don’t lie | AV | LQ | 0.9630
HQ | 0.9490
Multi-modal [41] |  Not-made-for-each-other | AV | LQ | 0.9790
HQ | 0.9680
Multi-modal [4] | AV-Lip-Sync | AV | LQ | 0.9790
HQ | 0.9680
Multi-modal (ours) | AV-Lip-Sync+ | AV | LQ | 0.9500
HQ | 0.9880
Multi-modal (ours) |  AV-Lip-Sync+ with FE | AV | LQ | 0.9996
HQ | 0.9998

####  IV-E4 Evaluation on the DeepfakeTIMIT dataset

The DeepfakeTIMIT dataset is primarily used for training and evaluating visual
deepfake detectors, as it only contains visual manipulation. It comes in two
versions, Low-Quality (LQ) and High-Quality (HQ). For a fair comparison, we
compared the proposed model with uni-modal visual and multi-modal audio-visual
deepfake detectors. Additionally, we performed 5-fold cross-validation and
reported the average AUC score. As can been seen from Table VI, among uni-
modal visual detectors, FWA and DSP-FWA [64] achieved the best AUC of 0.9990
under the LQ condition, and DSP-FWA achieved the best AUC of 0.9970 under the
HQ condition. Although the three existing multi-modal audio-visual detectors
(Emotions Don’t lie [42], Not-made-for-each-other [41], and AV-Lip-Sync [4])
are overall better than most uni-modal visual detectors, their performance is
worse than that of the best performing uni-modal visual detector DSP-FWA.
However, our multi-modal AV-Lip-Sync+ with FE outperformed all models compared
in the table. The AUC score is 0.9996 for LQ and 0.9998 for HQ. By
appropriately integrating pre-trained AV-HuBERT and ViViT models for audio-
visual feature extraction, our proposed model achieves state-of-the-art
results on the DeepfakeTIMIT dataset.

##  V Conclusions

In this study, we have leveraged AV-HuBERT and ViViT to perform downstream
tasks of audio-visual video forgery detection. We exploit the inconsistency
between visual and audio modalities by using powerful audio-visual
representations provided by AV-HuBERT, which is pre-trained using Self-
Supervised learning. Since AV-HuBERT only extracts visual features from the
lip region, which may not be enough to detect artifacts outside the lip
region, we also adopt another transformer-based ViViT model to exploit facial
features. Overall, our model jointly exploits SSL audio/visual/audio-visual
representations, synchronization features, temporal correlation between lip
image frames and audio, and inter- and intra-video frames features to detects
deepfakes. Experimental results on the FakeAVCeleb and DeepfakeTIMIT datasets
show that our model outperforms all existing models and achieves new state-of-
the-art performance on both datasets. In future work, we will work to further
improve the generalization of deepfake content detection technology in multi-
modal settings.

## References

  * [1] B. Shi, W.-N. Hsu, K. Lakhotia, A. Mohamed, Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction, in: Proceedings of the International Conference on Learning Representations, 2021\.
  * [2] I.-C. Chern, K.-H. Hung, Y.-T. Chen, T. Hussain, M. Gogate, A. Hussain, Y. Tsao, J.-C. Hou, Audio-Visual Speech Enhancement and Separation by Utilizing Multi-modal Self-Supervised Embeddings, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023, pp. 1–5.
  * [3] B. Shi, W.-N. Hsu, A. Mohamed, Robust Self-Supervised Audio-Visual Speech Recognition, in: Proceedings of the Interspeech Conference, 2022, pp. 2118–2122.
  * [4] S. A. Shahzad, A. Hashmi, S. Khan, Y.-T. Peng, Y. Tsao, H.-M. Wang, Lip Sync Matters: A Novel Multimodal Forgery Detector, in: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2022, pp. 1885–1892.
  * [5] H. Khalid, S. Tariq, M. Kim, S. S. Woo, FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset, in: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
  * [6] I. Korshunova, W. Shi, J. Dambre, L. Theis, Fast Face-swap Using Convolutional Neural Networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3677–3685.
  * [7] R. Chen, X. Chen, B. Ni, Y. Ge, SimSwap: An Efficient Framework For High Fidelity Face Swapping, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 2003–2011.
  * [8] G. Gao, H. Huang, C. Fu, Z. Li, R. He, Information bottleneck disentanglement for identity swapping, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3404–3413.
  * [9] J. Thies, M. Zollhöfer, M. Nießner, Deferred Neural Rendering: Image Synthesis using Neural Textures, ACM Transactions on Graphics (TOG) 38 (4) (2019) 1–12.
  * [10] L. Li, J. Bao, H. Yang, D. Chen, F. Wen, FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, p. 5074–5083.
  * [11] J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, M. Nießner, Face2Face: Real-time Face Capture and Reenactment of RGB Videos, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2387–2395.
  * [12] Y. Nirkin, Y. Keller, T. Hassner, FSGAN: Subject Agnostic Face Swapping and Reenactment, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7184–7193.
  * [13] K. Prajwal, R. Mukhopadhyay, V. P. Namboodiri, C. Jawahar, A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 484–492.
  * [14] R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, Y. Choi, Defending Against Neural Fake News, Advances in Neural Information Processing Systems 32 (2019).
  * [15] Y. Yao, B. Viswanath, J. Cryan, H. Zheng, B. Y. Zhao, Automated Crowdturfing Attacks and Defenses in Online Review Systems, in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1143–1158.
  * [16] A. Das, R. Verma, Automated email Generation for Targeted Attacks using Natural Language, in: Proceedings of the TA-COS Workshop on Text Analytics for Cybersecurity and Online Safety, 2018, p. 23.
  * [17] K. McGuffie, A. Newhouse, The Radicalization Risks of GPT-3 and Advanced Neural Language Models, arXiv preprint arXiv:2009.06807 (2020).
  * [18] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, K. Kavukcuoglu, WaveNet: A Generative Model for Raw Audio, in: Proceedings of the ISCA Workshop on Speech Synthesis Workshop (SSW 9), 2016, p. 125.
  * [19] R. Prenger, R. Valle, B. Catanzaro, WaveGlow: A Flow-Based Generative Network for Speech Synthesis, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 3617–3621.
  * [20] S. Vasquez, M. Lewis, MelNet: A Generative Model for Audio in the Frequency Domain, arXiv preprint arXiv:1906.01083 (2019).
  * [21] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, et al., Tacotron: Towards End-to-End Speech Synthesis, Proceedings of the Interspeech Conference (2017).
  * [22] Y. Jia, Y. Zhang, R. Weiss, Q. Wang, J. Shen, F. Ren, P. Nguyen, R. Pang, I. Lopez Moreno, Y. Wu, et al., Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis, Advances in Neural Information Processing Systems 31 (2018).
  * [23] S. Lyu, Deepfake Detection: Current Challenges and Next Steps, in: Proceedings of the IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2020, pp. 1–6.
  * [24] Y. Li, M.-C. Chang, S. Lyu, In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking, in: Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS), 2018, pp. 1–7.
  * [25] X. Yang, Y. Li, S. Lyu, Exposing Deep Fakes Using Inconsistent Head Poses, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8261–8265.
  * [26] A. Haliassos, K. Vougioukas, S. Petridis, M. Pantic, Lips Don’t Lie: A Generalisable and Robust Approach to Face Forgery Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5039–5049.
  * [27] P. Korshunov, S. Marcel, DeepFakes: a New Threat to Face Recognition? Assessment and Detection, arXiv preprint arXiv:1812.08685 (2018).
  * [28] A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, M. Nießner, FaceForensics++: Learning to Detect Manipulated Facial Images, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 1–11.
  * [29] Y. Li, X. Yang, P. Sun, H. Qi, S. Lyu, Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3207–3216.
  * [30] B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, C. C. Ferrer, The DeepFake Detection Challenge (DFDC) Dataset, arXiv preprint arXiv:2006.07397 (2020).
  * [31] L. Jiang, R. Li, W. Wu, C. Qian, C. C. Loy, DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2889–2898.
  * [32] P. Kwon, J. You, G. Nam, S. Park, G. Chae, KoDF: A Large-scale Korean DeepFake Detection Dataset, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10744–10753.
  * [33] H. H. Nguyen, J. Yamagishi, I. Echizen, Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 2307–2311.
  * [34] K. Lutz, R. Bassett, DeepFake Detection with Inconsistent Head Poses: Reproducibility and Analysis, arXiv preprint arXiv:2108.12715 (2021).
  * [35] F. Chollet, Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251–1258.
  * [36] D. Afchar, V. Nozick, J. Yamagishi, I. Echizen, Mesonet: a Compact Facial Video Forgery Detection Network, in: IEEE International Workshop on Information Forensics and Security (WIFS), 2018, pp. 1–7.
  * [37] A. Khan, K. M. Malik, J. Ryan, M. Saravanan, Voice Spoofing Countermeasures: Taxonomy, State-of-the-art, experimental analysis of generalizability, open challenges, and the way forward, arXiv preprint arXiv:2210.00417 (2022).
  * [38] L. Wang, Y. Yoshida, Y. Kawakami, S. Nakagawa, Relative Phase Information for Detecting Human Speech and Spoofed Speech, in: Proceedings of the International Speech Communication Association, 2015.
  * [39] M. Todisco, H. Delgado, N. W. Evans, A New Feature for Automatic Speaker Verification Anti-Spoofing: Constant Q Cepstral Coefficients., in: Proceedings of the Odyssey, Vol. 2016, 2016, pp. 283–290.
  * [40] T. B. Patel, H. A. Patil, Combining Evidences from Mel Cepstral, Cochlear Filter Cepstral and Instantaneous Frequency Features for Detection of Natural vs. Spoofed Speech, in: Proceedings of the International Speech Communication Association, 2015.
  * [41] K. Chugh, P. Gupta, A. Dhall, R. Subramanian, Not made for each other-Audio-Visual Dissonance-based Deepfake Detection and Localization, in: Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 439–447.
  * [42] T. Mittal, U. Bhattacharya, R. Chandra, A. Bera, D. Manocha, Emotions Don’t Lie: An Audio-Visual Deepfake Detection Method using Affective Cues, in: Proceedings of the ACM International Conference on Multimedia, 2020, pp. 2823–2832.
  * [43] Y. Zhou, S.-N. Lim, Joint Audio-Visual Deepfake Detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14800–14809.
  * [44] H. Ilyas, A. Javed, K. M. Malik, AVFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio-visual deepfakes detection, Applied Soft Computing 136 (2023) 110124.
  * [45] D. Cozzolino, A. Pianese, M. Nießner, L. Verdoliva, Audio-Visual Person-of-Interest DeepFake Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 943–952.
  * [46] Z. Cai, K. Stefanov, A. Dhall, M. Hayat, Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization, in: Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA), 2022, pp. 1–10.
  * [47] A. Hashmi, S. A. Shahzad, W. Ahmad, C. W. Lin, Y. Tsao, H.-M. Wang, Multimodal Forgery Detection Using Ensemble Learning, in: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2022, pp. 1524–1532.
  * [48] T. Afouras, J. S. Chung, A. Zisserman, LRS3-TED: a large-scale dataset for visual speech recognition, arXiv preprint arXiv:1809.00496 (2018).
  * [49] B. Martinez, P. Ma, S. Petridis, M. Pantic, Lipreading Using Temporal Convolutional Networks, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 6319–6323.
  * [50] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, C. Schmid, ViViT: A Video Vision Transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6836–6846.
  * [51] A. Nagrani, J. S. Chung, A. Zisserman, VoxCeleb: a large-scale speaker identification dataset, in: Proceedings of the Interspeech Conference, 2017.
  * [52] C. Sanderson, The VIDTIMIT Database, Tech. rep. (2002).
  * [53] D. E. King, Dlib-ml: A Machine Learning Toolkit, The Journal of Machine Learning Research 10 (2009) 1755–1758.
  * [54] W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijayanarasimhan, F. Viola, T. Green, T. Back, P. Natsev, et al., The Kinetics Human Action Video Dataset, arXiv preprint arXiv:1705.06950 (2017).
  * [55] H. Khalid, M. Kim, S. Tariq, S. S. Woo, Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors, in: Proceedings of the Workshop on Synthetic Multimedia-Audiovisual Deepfake Generation and Detection, 2021, pp. 7–15.
  * [56] M. A. Raza, K. M. Malik, Multimodaltrace: Deepfake Detection using Audiovisual Representation Learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 993–1000.
  * [57] Y. Yu, X. Liu, R. Ni, S. Yang, Y. Zhao, A. C. Kot, PVASS-MDD: Predictive Visual-audio Alignment Self-supervision for Multimodal Deepfake Detection, IEEE Transactions on Circuits and Systems for Video Technology (2023).
  * [58] V. S. Katamneni, A. Rattani, MIS-AVioDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection, arXiv preprint arXiv:2310.02234 (2023).
  * [59] A. Hashmi, S. A. Shahzad, C.-W. Lin, Y. Tsao, H.-M. Wang, Avtenet: Audio-visual transformer-based ensemble network exploiting multiple experts for video deepfake detection, arXiv preprint arXiv:2310.13103 (2023).
  * [60] W. Yang, X. Zhou, Z. Chen, B. Guo, Z. Ba, Z. Xia, X. Cao, K. Ren, AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake, IEEE Transactions on Information Forensics and Security 18 (2023) 2015–2029.
  * [61] H. H. Nguyen, F. Fang, J. Yamagishi, I. Echizen, Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos, in: Proceedings of the IEEE International Conference on Biometrics Theory, Applications and Systems (BTAS), 2019, pp. 1–8.
  * [62] X. Han, V. Morariu, P. I. Larry Davis, et al., Two-Stream Neural Networks for Tampered Face Detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 19–27.
  * [63] F. Matern, C. Riess, M. Stamminger, Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations, in: Proceedings of the IEEE Winter Applications of Computer Vision Workshops (WACVW), 2019, pp. 83–92.
  * [64] Y. Li, S. Lyu, Exposing DeepFake Videos By Detecting Face Warping Artifacts, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 46–52.