# Hybrid Transformer Network for Deepfake Detection

Sohail Ahmed Khan<EMAIL_ADDRESS>MediaFuturesBergenNorway and  Duc-Tien
Dang-Nguyen<EMAIL_ADDRESS>MediaFuturesBergenNorway

(2022)

###### Abstract.

Deepfake media is becoming widespread nowadays because of the easily available
tools and mobile apps which can generate realistic looking deepfake
videos/images without requiring any technical knowledge. With further advances
in this field of technology in the near future, the quantity and quality of
deepfake media is also expected to flourish, while making deepfake media a
likely new practical tool to spread mis/disinformation. Because of these
concerns, the deepfake media detection tools are becoming a necessity. In this
study, we propose a novel hybrid transformer network utilizing early feature
fusion strategy for deepfake video detection. Our model employs two different
CNN networks, i.e., (1) XceptionNet and (2) EfficientNet-B4 as feature
extractors. We train both feature extractors along with the transformer in an
end-to-end manner on FaceForensics++, DFDC benchmarks. Our model, while having
relatively straightforward architecture, achieves comparable results to other
more advanced state-of-the-art approaches when evaluated on FaceForensics++
and DFDC benchmarks. Besides this, we also propose novel face cut-out
augmentations, as well as random cut-out augmentations. We show that the
proposed augmentations improve the detection performance of our model and
reduce overfitting. In addition to that, we show that our model is capable of
learning from considerably small amount of data.

deepfake detection, face forensics, attention mechanisms, image analysis,
feature fusion, misinformation detection, transformers

††copyright: acmcopyright††journalyear: 2022††conference: ; ; Graz,
Austria††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Security and
privacy Social aspects of security and privacy

##  1\. Introduction

The availability of huge image/video datasets and affordable compute
resources, has resulted in swift progress in the field of deep learning
research, specifically in the subarea of Generative Adversarial Networks
(GANs) (Goodfellow et al., 2014). This progress has made it almost effortless
to generate realistic synthetic content even for non-technical computer users.
The synthetic content generated using deep learning models (i.e., GANs) is
called Deepfake media. Deepfake media can be in the form of images, videos,
text and audios. However, out of all the different categories of deepfake
media, the visual deepfake media is the most common form of fake/synthetic
content we encounter nowadays. The number of deepfake media generation
techniques is growing exponentially. The newer generation techniques are able
to generate extremely plausible synthetic content, and it is becoming more and
more challenging to detect the generated fake media.

Figure 1. Our face pre-processing module is responsible for applying 3
different types of image augmentations, (1) Affine Transformations, (2) Random
cut-out augmentations, and (3) Face cut-out augmentations. More details about
the augmentations we use are given in upcoming sections. We show that the
employed image augmentations improve detection performance while reducing
overfitting. Photos acquired from Flickr Faces HQ dataset (Karras et al.,
2019).

The most popular form of facial deepfake media we encounter at present is
generated using face swapping method, in which the face of a person (target)
is swapped with the face of another person (source). There are 4 different
types of facial deepfake media, i.e., (1) Face Swapping, (2) Face Re-
enactment, (3) Face Editing and (4) Face Synthesis (Mirsky and Lee, 2021). In
this paper we focus on detecting facial deepfake media, specifically the media
generated using face swapping and face re-enactment techniques.

Using ensembled or fusion based models tend to achieve exceptional results as
compared to single models (Dolhansky et al., 2020a; Qi et al., 2020; Zhu et
al., 2021; Dolhansky et al., 2020b). We therefore propose to employ two
different CNN models as feature extractors along with a transformer
architecture (Vision Transformer (Dosovitskiy et al., 2021)). We expect that
by fusing features extracted using different feature extractors will result in
diverse feature spaces, which will help the transformer to learn diverse set
of features. Transformer architectures are capable of simultaneously learning
meaningful associations from long input sequences. We therefore choose
transformer (Dosovitskiy et al., 2021) to learn joint feature space, instead
of the classical way of using a fully connected layer to combine different
feature sets. Besides this, hybrid (having CNN as feature extractor instead of
using simple patch embeddings) transformer models tend to achieve even better
results.

We choose XceptionNet as one of the feature extractors as it has been widely
employed in deepfake detection domain (Rössler et al., 2019). The second CNN
which we choose is the EfficientNet B4 model, which also achieved exceptional
results on ImageNet benchmark. We do not freeze the feature extractors during
training, i.e., we train both feature extractors, as well as the transformer
architecture in an end-to-end manner using a single loss function.

The contributions of this paper are three fold, (1) we propose a novel hybrid
transformer architecture which learns from joint feature sets extracted by two
different CNN feature extractors, (2) we show that image augmentations we
generate using our face pre-processing module, combined with other affine
transformation based augmentations, improve the performance of the detection
models while reducing overfitting, and (3) we show that while having a simple
and easy to implement architecture, our model achieves comparable results to
other more complex state-of-the-art approaches while being trained on
comparably smaller number of data samples.

This paper is structured as follows, in section 2 we present a brief
literature review, in section 3 we describe the augmentations we employ, the
proposed face pre-processing module, our model architecture, the datasets used
to train our models, and implementation details, in section 4 we compare the
achieved results with other deepfake detection baselines, and in section 5 we
conclude our study and propose future research directions.

##  2\. Related Work

Recent works on deepfake media detection mostly employ CNN based architectures
along with other strategies (e.g., multimodal features, recurrent networks,
transformer models etc) to detect deepfake images/videos. Unlike the previous
research studies, in this paper we propose a novel strategy to simultaneously
learn from joint feature spaces extracted using two different CNN feature
extractors using a transformer architecture while employing heavy image
augmentations.

Rossler et al. in (Rössler et al., 2019) proposed a simple deepfake detection
technique based on the XceptionNet (Chollet, 2017) CNN model pre-trained on
the imagenet dataset. Authors fine-tune the generic XceptionNet on their
FaceForensics++ dataset while reporting excellent performance scores the four
subsets of the FaceForensics++ dataset, namely, (1) FaceSwap, (2) Face2Face,
(3) DeepFakes, and (4) NeuralTextures (Rössler et al., 2019). The proposed
model achieved excellent results on uncompressed videos, however, lost
performance when tested on compressed videos.

In (Zhu et al., 2021) Zhu _et al._ propose to utilize 3D facial details to
detect deepfakes. Authors find that merging the 3D identity texture and direct
light is significantly helpful in detecting deepfakes. They employ the
XceptionNet CNN model for feature extraction. A face cropped image and its 3D
detail is used to train the detection model. They also perform a detailed
analysis of a number of different feature fusion strategies. The proposed
technique was trained on FaceForensics++ dataset and evaluated on (1)
FaceForensics++, (2) Google Deepfake Detection Dataset, and (3) DFDC datasets.
Authors report promising results on all of the three datasets along with
better generalization capability than the previously proposed techniques.

Qi _et al._ in (Qi et al., 2020) propose a novel deepfake detection technique
which they call, DeepRhythm. The proposed technique works by analyzing the
heartbeat rhythms of a person in a given video. They employ
photoplethysmography (PPG) to analyze minute changes in the skin tone inherent
with the blood pumping visible on the human faces. Authors evaluate the
proposed technique on FaceForensics++, and DFDC datasets and report excellent
performance results.

In (Xuan et al., 2019) Xuan _et al._ proposed a more general deepfake media
detection technique by employing image augmentations, for example, gaussian
blur and gaussian noise to preprocess images in order to remove low level high
frequency GAN artifacts present inside the generated images. They then trained
a forensic convolutional neural network model on the preporcessed images. They
established that by using image augmentations on both real and fake images,
destroy the low level noise cues, while forcing the forensics model to learn
more meaningful features. Through experimentation authors establish the
effectiveness of their proposed technique on deepfake media detection.

Sabir _et al._ in (Sabir et al., 2019) proposed a recurrent convolutional
network for deepfake media detection. The DenseNet convolutional neural
network was employed along with a gated recurrent neural network to exploit
temporal inconsistencies present between frames of a deepfake video. The
proposed technique was evaluated on the FaceForensics++ (Rössler et al., 2019)
dataset showing promising performance.

Figure 2. Our hybrid transformer network comprising of two different CNNs, (1)
XceptionNet, (2) EfficientNet-B4 and a transformer. Our model is trained on
the cropped face images. The face image is fed to the face pre-processing
module which applies face cut-out/random cut-out and affine transformation
augmentations randomly. The augmented face image is then fed to the CNN
feature extractors. The extracted features are then fused together by
concatenation. A BERT style $[class]$ token and learnable positional embedding
are added to the concatenated features which are then fed to the transformer.
Both feature extractors and the transformer are trained in an end-to-end
manner using a single loss function.

Following a similar path, Guera and Delp in (Güera and Delp, 2018) proposed to
employ a convolutional neural network along with a long short term memory
(LSTM) network for deepfake video detection. Authors tried to exploit inter-
frame discrepancies inherent to most deepfake videos. The CNN was tasked with
extracting frame-level features, which were then fed to the LSTM network to
learn the temporal features. Authors evaluated the model on their own dataset
showing exceptional performance.

Nguyen _et al._ in (Nguyen et al., 2019b) proposed a capsule network based
model to detect deepfake media. The proposed model was evaluated on four
different deepfake detection datasets containing a wide range of synthetic
images and videos. The proposed method achieved excellent results as compared
to other methods on all datasets.

Again in (Nguyen et al., 2019a), Nguyen _et al._ proposed a different strategy
employing a Y-shaped encoder-decoder model. Authors trained the model by
following a multi-task learning based technique which was able to classify and
generate a segmentation mask of the tampered regions within manipulated
images/videos. The proposed model was evaluated on the FaceForensics and
FaceForensics++ (Rössler et al., 2019) datasets achieving promising results
even when finetuned on a small number of images.

In (Ciftci et al., 2020) Ciftci _et al._ proposed a novel deepfake detection
technique employing biological signals (i.e., photoplethysmography or PPG
signals) to train a CNN and a support vector machine (SVM). The predictions
from the CNN and SVM were fused to get final classification label. The
proposed model achieved promising results when evaluated on several different
deepfake detection datasets including, Face Forensics, Face Forensics++
(Rössler et al., 2019), and CelebDF (Li et al., 2020) datasets.

In (Afchar et al., 2018) Afchar _et al._ proposed two different CNN models,
which they called, (1) Meso-4 and (2) MesoInception-4, both containing a small
number of layers focusing on mesoscopic image features. The proposed model was
tested on an existing deepfake detection dataset, as well as, a custom dataset
collected by the authors. The model achieved excellent results on both
datasets.

In (Khan and Dai, 2021) proposed a novel video transformer network for
deepfake media detection, capable of learning from new data in an incremental
manner. The proposed video transformer model was trained on multimodal data
(i.e., face cropped images, and UV texture maps). The proposed models achieved
excellent results on a number of different deepfake detection benchmarks.

##  3\. Methodology

In this section, we briefly introduce, (1) image augmentations we employ to
train our models, (2) face pre-processing module which is responsible for
applying the image aumentations and normalizing the image, (3) the two CNN
backbones we employ to extract image features, and (4) the transformer model.

###  3.1. Augmentations

We use two different image cropping (face cut-out, random cut-out) based
augmentations, which are applied along with three different affine
transformation (rotation, translation, scaling) based augmentations.

In face cut-out augmentations, we crop out specific face part from the image
in a random order using facial landmarks, as shown in figure. Whereas, in case
of random cut-out augmentations, we crop out random square shaped region from
a given face image. Both of these augmentations are separately applied to
train two different models. However, the affine transformation based
augmentations are applied along with both of these cut-out based
augmentations. The pipeline shown in figure 2, applies face cut-out
augmentation along with affine transformation based augmentations to a given
face image.

###  3.2. Face Pre-processing Module

The face pre-processing module is responsible for applying augmentations and
normalizing a given input face image. In case of face cut-out augmentations,
the face pre-processing module takes a face cropped image as input, predicts
81 face landmarks using OpenCV’s facial landmarks predictor. It then selects a
number of different landmarks randomly to cut a specific face part out from
the image.

The face pre-processing module in case of random cut-out augmentations, takes
a face cropped image, randomly crop outs two squared shaped regions from the
image. The purpose of using heavy cut out augmentations is that we want to
prevent our model from overfitting. If we do not use augmentations, the model
will memorize the training data and will not be able generalize well on the
test set, as can be seen from the results in table 2.

###  3.3. XceptionNet

XceptionNet architecture with depth-wise separable convolutions was proposed
by François Chollet in (Chollet, 2017). For deepfake detection, Rossler _et
al._ employed XceptionNet in (Rössler et al., 2019). They showed that the
XceptionNet architecture achieved exceptional results on FaceForensics++
dataset. Because of the excellent performance of XceptionNet on deepfake
detection, in this paper we use XceptionNet as one of the feature extractors.

###  3.4. EfficientNet-B4

EfficientNets are a new set of state-of-the-art convolutional neural network
models for image classification proposed in (Tan and Le, 2019). For deepfake
detection EfficientNet architectures achieved promising results on DFDC
dataset. In-fact, the winning model of the DFDC was an ensemble of 5
EfficientNet CNNs (Dolhansky et al., 2020a). Furthermore, because of the
memory constraints we needed a smaller but effective model, making
EfficientNet-B4 the best choice for our study.

###  3.5. Hybrid Image Transformer

Table 1. Performance (accuracy) comparison of a number of different deepfake detection baseline models on FaceForensics++ dataset. Each of the mentioned model was trained on all subsets of the FaceForensics++ dataset at once. Best results are highlighted. Approach | Deepfakes | Face2Face | FaceSwap | NeuralTextures | Pristine | Cumulative
---|---|---|---|---|---|---
Steg. Features + SVM (Fridrich and Kodovský, 2012) | 68.80% | 67.69% | 70.12% | 69.21% | 72.98% | 70.97%
Cozzolino _et al._ (Cozzolino et al., 2017) | 75.51% | 86.34% | 76.81% | 75.34% | 78.41% | 78.45%
Bayar and Stamm (Bayar and Stamm, 2016) | 90.25% | 93.96% | 87.74% | 83.69% | 77.02% | 82.97%
Afchar _et al._ (Afchar et al., 2018) | 89.55% | 88.60% | 81.24% | 76.62% | 82.19% | 83.10%
Rossler _et al._ (Rössler et al., 2019) | 97.49% | 97.69% | 96.79% | 92.19% | 95.41% | 95.73%
Qi _et al._ (Qi et al., 2020) | 99.70% | 98.90% | 97.80% | - | - | -
Ours (Face cut-out) | 97.85% | 97.85% | 96.42% | 90.71% | 95.00% | 95.57%
Ours (Random cut-out) | 98.57% | 98.57% | 97.85% | 92.14% | 97.85% | 97.00%
Table 2. Performance comparison of our models trained with and without image augmentations. In this table, DF refers to Deepfakes, F2F refers to Face2Face, FS refers to FaceSwap, NT refers to NeuralTextures, and P refers to Pristine. Model | DF | F2F | FS | NT | P | Agg.
---|---|---|---|---|---|---
No Augs | 95.71% | 93.57% | 92.85% | 85.00% | 96.42% | 92.71%
Face cut-out | 97.85% | 97.85% | 96.42% | 90.71% | 95.00% | 95.57%
Random cut-out | 98.57% | 98.57% | 97.85% | 92.14% | 97.85% | 97.00%

Table 2 presents a performance comparison of our model trained under different
augmentation strategies. We use heavy image augmentations for example,
rotation, horizontal flipping, translation, scaling, face cut-outs and random
cut-outs. During experimentation we found that the model overfits severely
when no image augmentations are employed. Random cut-out augmentations along
with affine transformation based augmentations give best results.

Table 3. Number of frames used to train models on different datasets: Pristine, FaceSwap, Deepfakes, Face2Face, Neural Textures. Dataset | Training | Validation | Test
---|---|---|---
Pristine | 100K | 20K | 2.24K
FaceSwap | 25K | 5K | 2.24K
Deepfakes | 25K | 5K | 2.24K
Face2Face | 25K | 5K | 2.24k
Neural Textures | 25K | 5K | 2.24K
Table 4. We compare the performance (accuracy) of our model with the XceptionNet proposed by Rossler _et al._ in (Rössler et al., 2019). In this table, DF refers to Deepfakes, F2F refers to Face2Face, FS refers to FaceSwap, NT refers to NeuralTextures. Model | DF | F2F | FS | NT | Agg.
---|---|---|---|---|---
Rossler _et al._ (Rössler et al., 2019) | 92.48% | 91.33% | 92.63% | 85.98% | 90.60%
Ours (Random cut-out) | 95.00% | 95.00% | 95.71% | 90.00% | 93.92%
Table 5. Number of training and validation images used by different deepfake detection techniques. The number of train, validation, and test sets of other studies in this table are rough estimates, as the the authors do not specify exact number of images they used to train their models. Approach | Train | Validation | Test
---|---|---|---
Rossler _et al._ (Rössler et al., 2019) | 388K | 70K | 70K
Zhu _et al._ (Zhu et al., 2021) | 360K | 70K | 70K
Ours | 200K | 40K | 11.2K

We employ BERT (Devlin et al., 2019; Dosovitskiy et al., 2021) styled
transformer to learn the joint features extracted using the two CNNs. Since
Transformers have proven to be excellent in simultaneously learning meaningful
properties from long sequences because of their bidirectional representation
learning capability, we employ these models to learn the joint features
extracted by two CNNs. We believe that rather than utilizing the early feature
fusion followed by a fully connected layer to learn from the joint feature
space, as it is widely done while using multiple CNNs; using a transformer to
learn the joint feature space will yield better results.

In addition to this, ensembled/fusion based models are proven to achieve
better results when compared with single models specifically in deepfake
detection, as we can infer from Facebook’s Deepfake Detection Challenge, in
which most of the top ranked models employed ensembled/fusion based networks
(Dolhansky et al., 2020a).

We utilize Transformer architecture pretrained on
ImageNet111https://github.com/lukemelas/PyTorch-Pretrained-ViT) on features
extracted using an early fusion based strategy in which two different CNN
models as introduced above. The extracted features are concatenated, a
classification token (similar to BERT’s [class] token) is added at the start,
and assigned a positional embedding before being fed to the transformer
architecture. Our model comprises of 12 encoder blocks and 12 attention heads
(following the architecture of ViT-Base-16 (Dosovitskiy et al., 2021)). The
weights for the pretrained CNN models are obtained from Ross
Wightman222https://github.com/rwightman/pytorch-image-models. Both the feature
extractors, and the transformer are trained in an end-to-end manner using a
single loss function i.e., we do not freeze the weights of feature extractors
while training.

###  3.6. Dataset

We train and evaluate our models on FaceForensics++ (Rössler et al., 2019) and
DFDC (Dolhansky et al., 2020b) datasets.

####  3.6.1. FaceForensics++

This dataset comprises of four subsets namely, (1) FaceSwap, (2) Face2Face,
(3) Deepfakes, and (4) Neural Textures. The videos we used to train our models
are high quality (c23). We create train, validation, and test sets according
to the instructions in FaceForensics++ dataset paper (Rössler et al., 2019).
There are 720 train videos, 140 validation videos, and 140 test videos. We
train our models on 200K images (100k real and 100k fake), which is less than
half of the size of training set Rossler _et al._ used to train their
XceptionNet model in (Rössler et al., 2019). Out of the 200K training images,
160K images are used for training and the remaining 40K images are used for
validation.

For fake videos, we extract 50 face frames starting from the beginning of each
video, whereas, for the real videos, we extract 150 frames from each video. We
do this to balance the real and fake test sets. The exact amount of images
from each dataset used to train, validate and test our models as given in
table 3.

To evaluate our models we extract 16 face frames from 140 test videos which
results in a total of 2100 images as mentioned in the table 3. To assign a
classification label to any test video, we take 16 frames and feed to our
model one by one. The final prediction is made after averaging the predictions
obtained from the model for each frame.

####  3.6.2. Deepfake Detection Challenge (DFDC)

This dataset comprises of around 124K videos. To train our models we use only
around 8K videos. The amount of fake videos in DFDC dataset is more than the
real videos, and thus to balance the real and fake image samples, we extract
50 frames from each fake video, whereas, from each real video we extract 150
frames. This results in around 265K real and fake frames, from which we only
use 48K images to train our model and 12K images for validation purposes. So
in total, we only used 60K real and fake face frames from DFDC dataset to
train and validate our models.

We use 400 test videos provided with the DFDC dataset to evaluate our model.
For evaluation, we use the same strategy we used for evaluating the models on
FaceForensics++ dataset, i.e., we extract 16 face frames from each test video
and feed to our model one by one. The final prediction about a video is made
after averaging the individual frame predictions.

###  3.7. Implementation Details

For face detection and cropping, we use OpenCV333https://opencv.org/. For
custom image augmentations such as, rotation, flipping, translation and
cutouts we employ ImgAug444https://imgaug.readthedocs.io/en/latest/ library.
We use Ross Wightman’s github repoitory to download CNN weights pretrained on
ImageNet. We borrow code and weights of vision transformer pretrained on
ImageNet from Luke Melas 555https://github.com/lukemelas/PyTorch-Pretrained-
ViT.

To train our models we use SGD with a momentum ranging from 0.6 to 0.9, with a
learning rate of $3\times 10^{-3}$. We stop training when the validation loss
keeps on increasing for 3 consecutive epochs, or the training accuracy
approaches to 100% (to prevent severe overfitting). We train the two CNNs and
the transformer in an end-to-end manner, and optimize them through a single
binary cross entropy loss function.

We resize images to $[3,224,224]$ dimensions. Having higher resolution images
yield better results but because of memory constraints we choose to use this
image resolution to train and evaluate our models. The input image is fed to
the CNN feature extractors, which after extracting features, each of the
feature extractors return features of dimension $[1,162,768]$. The obtained
features are then concatenated to get final features of dimension
$[1,324,768]$. We append a BERT style $[class]$ token at the start of the
extracted features resulting making the dimension $[1,325,768]$. A learnable
positional embedding is added to these features through element wise addition.
The resulting features are then fed to the transformer.

##  4\. Results

In this section we will present and compare the results our model achieved on
the FaceForensics++ and DFDC dataset. We trained our model on FaceForensics++
dataset under three different settings e.g., (1) without image augmentations,
(2) with face cut-out augmentations, and (3) with random cut-out
augmentations. We found that the model trained using random cut-out
augmentations outperformed the other two variants, and thus we further trained
this model on DFDC dataset as well. Performance comparison of our model under
three augmentation strategies can be seen in table 2. As we understand, the
reason behind the excellent performance of random cut-out augmentation is
because it cuts out most parts of the image in a random order preventing the
model from memorizing face images.

It should also be noted that we trained our models on smaller number of
samples from the FaceForensics++ dataset (given in table 3) as compared to
other approaches, e.g., Rossler _et al._ (Rössler et al., 2019) train their
model on around $388K$ images, Zhu _et al._ (Zhu et al., 2021) train their
models on around $360K$ images. We compare our models with the baseline
results provided in the original FaceForensics++ paper (Rössler et al., 2019),
since they provide performance scores of each model on every subset of the
FaceForensics++ dataset. We also present the results achieved by our models on
DFDC dataset in table 6.

Table 1 presents a detailed comparison of the obtained accuracy scores on all
of the subsets of FaceForensics++ dataset. Our model trained using heavy image
augmentations achieves comparable results to the baseline techniques listed in
Table 1 and new state-of-the-art deepfake detection techniques(Zhu et al.,
2021; Qi et al., 2020). The reason for choosing the techniques in table 1 for
comparison is that these techniques are also evaluated in the original
FaceForensics++ dataset paper, and carry out the training in the same manner
as we do in this paper. Testing is done in a different manner than these
approaches i.e., we test our models on 16 face frames per video, whereas these
approaches test there models on 100 face frames per video. For validation, the
approaches in FaceForensics++ paper use 100 face frames per video, whereas, we
split the train and validation set in 80:20 ratio i.e., 200K images for
training and 40K images for validation.

Some of the other deepfake detection state-of-the-art techniques (Zhu et al.,
2021; Qi et al., 2020; Ciftci et al., 2020) achieve better results than our
model, however, those techniques are quite complex to implement, and in most
cases use more data than we use to train the models.

In table 4 we present a comparison of our model with XceptionNet (Rössler et
al., 2019) model proposed by Rossler _et al_. We train and evaluate our model
on each of the four subsets of FaceForensics++ dataset separately. For
training we only use 5000 fake images (from each subset) and 5000 real images.
While Rossler _et al._ trained their models on 50 videos or 13500 images since
they use 270 images from each video for training. We show that our model even
being trained on less data, improves detection performance. In table 5 we
compare the size of training set Rossler _et al._ (Rössler et al., 2019) and
Zhu _et al._ (Zhu et al., 2021) use to train their respective models.

Table 6. Performance (accuracy) comparison of a number of different deepfake detection baseline models on DFDC dataset. Best results are highlighted. Approach | Dataset | Train Size | Accuracy
---|---|---|---
Mittal _et al._ (Mittal et al., 2020) | DFDC | - | 84.40%
Wodajo _et al._ (Wodajo and Atnafu, 2021) | DFDC | 112K | 91.50%
Bondi _et al._ (Bondi et al., 2020) | DFDC | - | 92.20%
Ours (Random cut-out) | DFDC | 48K | 98.24%

We present a comparison of results our model achieved on the DFDC dataset in
table 6. We show that while being trained on smaller training set, our model
still achieves exceptional performance scores as compared to other relevant
works proposed in the past.

##  5\. Conclusion and Future Work

Detecting deepfake media is crucial as well as challenging. Besides other
challenges of deepfake media detection, for example, poor generalization
capability of the detection models, deepfake media is also adversarial in
nature and continues to evolve rapidly. In this study we presented an early
fusion based hybrid transformer network for deepfake media detection. Our
model achieved comparable results to most of the state-of-the-art deepfake
detection techniques (Zhu et al., 2021; Qi et al., 2020) . After this, we plan
to train and evaluate our model on other prominent deepfake detection
datasets, such as, Celeb-DF (Li et al., 2020), ForgeryNet (He et al., 2021)
and others. We also plan to analyze the generalization capability of our model
on unseen data in future studies, while trying to visually interpret our model
to know what kind of features it utilizes more while making decisions and how
it differentiates between different categories of deepfake media, i.e., face
swapping, face re-enactment etc.

In future we will focus our work mainly on improving the generalization
capability of the deepfake detection models, and further work on improving the
adversarial robustness of the detection models.

##  6\. Acknowledgments

This research was supported by industry partners and the Research Council of
Norway with funding to MediaFutures: Research Centre for Responsible Media
Technology and Innovation, through the Centres for Research-based Innovation
scheme, project number 309339.

## References

  * (1)
  * Afchar et al. (2018) Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018.  MesoNet: a Compact Facial Video Forgery Detection Network.  In _Proceedings of IEEE International Workshop on Information Forensics and Security (WIFS)_. IEEE.  {https://arxiv.org/abs/1809.00888}
  * Bayar and Stamm (2016) Belhassen Bayar and Matthew C. Stamm. 2016.  A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer.  _Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security_ (2016).
  * Bondi et al. (2020) L. Bondi, Edoardo Daniele Cannas, Paolo Bestagini, and Stefano Tubaro. 2020.  Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection.  _Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS)_ (2020), 1–6.
  * Chollet (2017) François Chollet. 2017\.  Xception: Deep Learning with Depthwise Separable Convolutions.  In _Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE.  {https://arxiv.org/abs/1610.02357}
  * Ciftci et al. (2020) Umur Aybars Ciftci, ˙Ilke Demir, and Lijun Yin. 2020.  FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals.  In _IEEE Transactions on Pattern Analysis and Machine Intelligence_. IEEE.  {https://arxiv.org/abs/1901.02212}
  * Cozzolino et al. (2017) Davide Cozzolino, Giovanni Poggi, and Luisa Verdoliva. 2017\.  Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection. In _Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  _ArXiv_ abs/1810.04805 (2019).
  * Dolhansky et al. (2020a) Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. 2020a.  The DeepFake Detection Challenge (DFDC) Dataset.  Available: https://arxiv.org/abs/2006.07397.
  * Dolhansky et al. (2020b) Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. 2020b.  The DeepFake Detection Challenge (DFDC) Dataset.  _arXiv: Computer Vision and Pattern Recognition_ (2020).
  * Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021\.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.  _ArXiv_ abs/2010.11929 (2021).
  * Fridrich and Kodovský (2012) Jessica J. Fridrich and Jan Kodovský. 2012.  Rich Models for Steganalysis of Digital Images.  _IEEE Transactions on Information Forensics and Security_ 7 (2012), 868–882.
  * Goodfellow et al. (2014) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative Adversarial Nets.  In _Advances in Neural Information Processing Systems, (NIPS)_. Curran Associates, Inc., 2672–2680.  http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
  * Güera and Delp (2018) David Güera and Edward J. Delp. 2018.  Deepfake Video Detection Using Recurrent Neural Networks.  In _Proceedings of the 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_. IEEE.  {https://ieeexplore.ieee.org/document/8639163}
  * He et al. (2021) Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. 2021.  ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis.  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ (2021), 4358–4367.
  * Karras et al. (2019) Tero Karras, Samuli Laine, and Timo Aila. 2019.  A Style-Based Generator Architecture for Generative Adversarial Networks. In _Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 4396–4405.
  * Khan and Dai (2021) Sohail Ahmed Khan and Hang Dai. 2021.  Video Transformer for Deepfake Detection with Incremental Learning. In _Proceedings of the 29th ACM International Conference on Multimedia_.
  * Li et al. (2020) Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020.  Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. In _Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 3204–3213.
  * Mirsky and Lee (2021) Yisroel Mirsky and Wenke Lee. 2021.  The Creation and Detection of Deepfakes: A Survey.  _ACM Computing Surveys (CSUR)_ (2021).
  * Mittal et al. (2020) Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha. 2020\.  Emotions Don’t Lie: An Audio-Visual Deepfake Detection Method using Affective Cues.  In _Proceedings of the 28th ACM International Conference on Multimedia_. ACM, 2823–2832.  {https://arxiv.org/abs/2003.06711}
  * Nguyen et al. (2019a) Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019a.  Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos.  In _IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS)_. IEEE.  {https://arxiv.org/abs/1906.06876}
  * Nguyen et al. (2019b) Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. 2019b.  Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos.  In _Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE.  {https://ieeexplore.ieee.org/document/8682602}
  * Qi et al. (2020) Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, L. Ma, Wei Feng, Yang Liu, and Jianjun Zhao. 2020.  DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms.  In _Proceedings of the 28th ACM International Conference on Multimedia_.
  * Rössler et al. (2019) Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019.  FaceForensics++: Learning to Detect Manipulated Facial Images.  _Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV)_ (2019), 1–11.
  * Sabir et al. (2019) Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael AbdAlmageed, Iacopo Masi, and Prem Natarajan. 2019\.  Recurrent Convolutional Strategies for Face Manipulation Detection in Videos.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_. IEEE, 80–87.  {https://openaccess.thecvf.com/content_CVPRW_2019/html/Media_Forensics/Sabir_Recurrent_Convolutional_Strategies_for_Face_Manipulation_Detection_in_Videos_CVPRW_2019_paper.html}
  * Tan and Le (2019) Mingxing Tan and Quoc V. Le. 2019.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks.  _ArXiv_ abs/1905.11946 (2019).
  * Wodajo and Atnafu (2021) Deressa Wodajo and Solomon Atnafu. 2021.  Deepfake Video Detection Using Convolutional Vision Transformer.  _ArXiv_ abs/2102.11126 (2021).
  * Xuan et al. (2019) Xinsheng Xuan, Bo Peng, Wei Wang, and Jing Dong. 2019\.  On the generalization of GAN image forensics.  In _In Sun Z., He R., Feng J., Shan S., Guo Z. (eds) Biometric Recognition. CCBR 2019_. Springer, Cham.  {https://doi.org/10.1007/978-3-030-31456-9_15}
  * Zhu et al. (2021) Xiangyu Zhu, Hao Wang, Hongyan Fei, Zhen Lei, and S. Li. 2021.  Face Forgery Detection by 3D Decomposition. In _Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 2928–2938.


# Thermodynamical description of the interacting new agegraphic dark energy

A. Sheykhi<EMAIL_ADDRESS>and M.R. Setare<EMAIL_ADDRESS>1Department of Physics, Shahid Bahonar University, Kerman 76175, Iran
2Research Institute for Astronomy and Astrophysics of Maragha (RIAAM),
Maragha, Iran
3 Department of Science, Payame Noor University, Bijar, Iran

###### Abstract

We describe the thermodynamical interpretation of the interaction between new
agegraphic dark energy and dark matter in a non-flat universe. When new
agegraphic dark energy and dark matter evolve separately, each of them remains
in thermodynamic equilibrium. As soon as an interaction between them is taken
into account, their thermodynamical interpretation changes by a stable thermal
fluctuation. We obtain a relation between the interaction term of the dark
components and this thermal fluctuation.

Keywords: dark energy; thermodynamics; entropy.

##  I Introduction

The dark energy puzzle is one of the biggest challenges of the modern
cosmology in the past decade. There is an ample evidences on the observational
side that our universe is currently experiencing a phase of accelerated
expansion Rie1 ; Rie2 ; Rie3 ; Rie4 . These observations suggest that nearly
three quarters of our universe consists of a mysterious energy component (dark
energy) which is responsible for this expansion, and the remaining part
consists of pressureless dark matter. Nevertheless, despite the mounting
observational evidences, the nature of such dark energy remains elusive and it
has become a source of much debate except for the fact that it has negative
pressure. Most discussions on dark energy rely on the assumption that it
evolves independently of dark matter. Given the unknown nature of both dark
energy and dark matter there is nothing in principle against their mutual
interaction and it seems very special that these two major components in the
universe are entirely independent. Indeed, this possibility has received a lot
of attention recently (see Ame1 ; Ame2 ; Ame3 ; Ame4 ; Ame5 ; Zim1 ; Zim2 ;
Zim3 ; Seta1 ; Set2 ; Set3 ; Set4 ; Set5 ; wang1 ; wang11 ; Shey0  and
references therein). In particular, it has been shown that the coupling can
alleviate the coincidence problem Pav1 . Furthermore, it was argued that the
appropriate coupling between dark components can influence the perturbation
dynamics and the cosmic microwave background (CMB) spectrum and account for
the observed CMB low $l$ suppression wang2 . It was shown that in a model with
interaction the structure formation has a different fate as compared with the
non-interacting case wang2 . It was also discussed that with strong coupling
between dark energy and dark matter, the matter density perturbation is
stronger during the universe evolution till today, which shows that the
interaction between dark energy and dark matter enhances the clustering of
dark matter perturbation compared to the noninteracting case in the past.
Therefore, the coupling between dark components could be a major issue to be
confronted in studying the physics of dark energy. However, so long as the
nature of these two components remain unknown it will not be possible to
derive the precise form of the interaction from first principles. Therefore,
one has to assume a specific coupling from the outset Ads ; Amen1 ; Amen2  or
determine it from phenomenological requirements Chim1 ; Chim2 .
Thermodynamical description of the interaction (coupling) between holographic
dark energy and dark matter has been studied in wang3 ; SetVag .

Among the various candidates to explain the accelerated expansion, the
agegraphic and new agegraphic dark energy (NADE) models condensate in a class
of quantum gravity may have interesting cosmological consequences. These
models take into account the Heisenberg uncertainty relation of quantum
mechanics together with the gravitational effect in general relativity. The
agegraphic dark energy models assume that the observed dark energy comes from
the spacetime and matter field fluctuations in the universe Cai1 ; Wei2 ; Wei1
. Since in agegraphic dark energy model the age of the universe is chosen as
the length measure, instead of the horizon distance, the causality problem in
the holographic dark energy is avoided. The agegraphic models of dark energy
have been examined and constrained by various astronomical observations age1 ;
age2 ; age3 ; age4 ; age5 ; age6 ; age7 ; sheykhi1 ; sheykhi2 ; sheykhi3 ;
sheykhi4 ; sheykhi5 ; Setare2 ; Setare22 . Although going along a fundamental
theory such as quantum gravity may provide a hopeful way towards understanding
the nature of dark energy, it is hard to believe that the physical foundation
of agegraphic dark energy is convincing enough. Indeed, it is fair to say that
almost all dynamical dark energy models are settled at the phenomenological
level, neither holographic dark energy model nor agegraphic dark energy model
is exception. Though, under such circumstances, the models of holographic and
agegraphic dark energy, to some extent, still have some advantage comparing to
other dynamical dark energy models because at least they originate from some
fundamental principles in quantum gravity.

The main purpose of this Letter is to study thermodynamical interpretation of
the interaction between dark matter and NADE model for a universe enveloped by
the apparent horizon. It was shown that for an accelerating universe the
apparent horizon is a physical boundary from the thermodynamical point of view
Jia . In particular, it was argued that for an accelerating universe inside
the event horizon the generalized second law does not satisfy, while the
accelerating universe enveloped by the apparent horizon satisfies the
generalized second law of thermodynamics Jia ; sheywang1 ; sheywang2 ;
sheywang3 . Therefore, the event horizon in an accelerating universe might not
be a physical boundary from the thermodynamical point of view. This Letter is
outlined as follows. In the next section we consider the thermodynamical
picture of the non-interacting NADE in a non-flat universe. In section III, we
extend the thermodynamical description in the case where there is an
interaction term between the dark components. We also present an expression
for the interaction term in terms of a thermal fluctuation. The last section
is devoted to summary and discussion.

##  II Thermodynamical description of the non-interacting NADE

We consider the Friedmann-Robertson-Walker (FRW) universe which is described
by the line element

$\displaystyle
ds^{2}=dt^{2}-a^{2}(t)\left(\frac{dr^{2}}{1-kr^{2}}+r^{2}d\Omega^{2}\right),$
(1)

where $a(t)$ is the scale factor, and $k$ is the curvature parameter with
$k=-1,0,1$ corresponding to open, flat, and closed universes, respectively. A
closed universe with a small positive curvature ($\Omega_{k}\simeq 0.01$) is
compatible with observations spe1 ; spe2 ; spe3 ; spe4 . The first Friedmann
equation takes the form

$\displaystyle
H^{2}+\frac{k}{a^{2}}=\frac{1}{3m_{p}^{2}}\left(\rho_{m}+\rho_{D}\right),$ (2)

where $H=\dot{a}/a$ is the Hubble parameter, $\rho_{m}$ and $\rho_{D}$ are the
energy density of dark matter and dark energy, respectively. We define, as
usual, the fractional energy densities such as

$\displaystyle\Omega_{m}=\frac{\rho_{m}}{3m_{p}^{2}H^{2}},\hskip
14.22636pt\Omega_{D}=\frac{\rho_{D}}{3m_{p}^{2}H^{2}},\hskip
14.22636pt\Omega_{k}=\frac{k}{H^{2}a^{2}}.$ (3)

Thus, the Friedmann equation can be written

$\displaystyle\Omega_{m}+\Omega_{D}=1+\Omega_{k}.$ (4)

Let us first review the origin of the agegraphic dark energy model. Following
the line of quantum fluctuations of spacetime, Karolyhazy et al. Kar1 ; Kar2 ;
Kar3  argued that the distance $t$ in Minkowski spacetime cannot be known to a
better accuracy than $\delta{t}=\beta t_{p}^{2/3}t^{1/3}$ where $\beta$ is a
dimensionless constant of order unity. Based on Karolyhazy relation, Sasakura
discussed that the energy density of metric fluctuations of the Minkowski
spacetime is given by Sas  (see also Maz1 ; Maz2 )

$\rho_{D}\sim\frac{1}{t_{p}^{2}t^{2}}\sim\frac{m^{2}_{p}}{t^{2}},$ (5)

where $t_{p}$ is the reduced Planck time and $t$ is a proper time scale. On
these basis, Cai wrote down the energy density of the original agegraphic dark
energy as Cai1

$\rho_{D}=\frac{3n^{2}m_{p}^{2}}{T_{A}^{2}},$ (6)

where $T_{A}$ is the age of the universe,

$T_{A}=\int_{0}^{a}{\frac{da}{Ha}},$ (7)

and the numerical factor $3n^{2}$ is introduced to parameterize some
uncertainties, such as the species of quantum fields in the universe, the
effect of curved space-time, and so on. However, to avoid some internal
inconsistencies in the original agegraphic dark energy model, the so-called
“new agegraphic dark energy” was proposed, where the time scale is chosen to
be the conformal time $\eta$ instead of the age of the universe Wei2 . The
NADE contains some new features different from the original agegraphic dark
energy and overcome some unsatisfactory points. For instance, the original
agegraphic dark energy suffers from the difficulty to describe the matter-
dominated epoch while the NADE resolved this issue Wei2 . The energy density
of the NADE can be written

$\rho_{D}=\frac{3n^{2}m_{p}^{2}}{\eta^{2}},$ (8)

where the conformal time $\eta$ is given by

$\eta=\int{\frac{dt}{a}}=\int_{0}^{a}{\frac{da}{Ha^{2}}}.$ (9)

Consider the FRW universe filled with dark energy and dust (dark matter) which
evolve according to their conservation laws

$\displaystyle\dot{\rho}_{D}+3H\rho_{D}(1+w_{D}^{0})=0,$ (10)
$\displaystyle\dot{\rho}_{m}+3H\rho_{m}=0,$ (11)

where $w_{D}=p_{D}/\rho_{D}$ is the equation of state parameter of NADE. The
superscript above the equation of state parameter, $w_{D}$, denotes that there
is no interaction between the dark components. The fractional energy density
of the NADE is given by

$\displaystyle\Omega_{D}=\frac{n^{2}}{H^{2}\eta^{2}},$ (12)

where its evolution behavior is governed by sheykhi1

$\displaystyle{\Omega^{\prime}_{D}}$ $\displaystyle=$
$\displaystyle\Omega_{D}\left[(1-\Omega_{D})\left(3-\frac{2}{n}\sqrt{\Omega_{D}}\right)+\Omega_{k}\right].$
(13)

Here the prime stands for the derivative with respect to $x=\ln{a}$. Taking
the derivative with respect to the cosmic time of Eq. (8) and using Eq. (12)
we get

$\displaystyle\dot{\rho}_{D}=-2H\frac{\sqrt{\Omega_{D}^{0}}}{na}\rho_{D}.$
(14)

Inserting this relation into Eq. (10) we obtain the equation of state
parameter of the NADE

$\displaystyle 1+w_{D}^{0}=\frac{2}{3na}\sqrt{\Omega_{D}^{0}}.$ (15)

We also limit ourselves to the assumption that the thermal system bounded by
the apparent horizon remains in equilibrium so that the temperature of the
system must be uniform and the same as the temperature of its boundary. This
requires that the temperature $T$ of the energy content inside the apparent
horizon should be in equilibrium with the temperature $T_{h}$ associated with
the apparent horizon, so we have $T=T_{h}$. This expression holds in the local
equilibrium hypothesis. If the temperature of the fluid differs much from that
of the horizon, there will be spontaneous heat flow between the horizon and
the fluid and the local equilibrium hypothesis will no longer hold. This is
also at variance with the FRW geometry. Thus, when we consider the thermal
equilibrium state of the universe, the temperature of the universe is
associated with the horizon temperature. In this picture the equilibrium
entropy of the NADE is connected with its energy and pressure through the
first law of thermodynamics

$TdS_{D}=dE_{D}+p_{D}dV,$ (16)

where the volume enveloped by the apparent horizon is given by

$V=\frac{4\pi}{3}r_{A}^{3},$ (17)

and $r_{A}$ is the apparent horizon radius. The apparent horizon was argued as
a causal horizon for a dynamical spacetime and is associated with
gravitational entropy and surface gravity Hay1 ; Hay2 ; Bak . For the FRW
universe the apparent horizon radius reads sheyahmad1 ; sheyahmad2

${r}_{A}=\frac{1}{\sqrt{H^{2}+k/a^{2}}}.$ (18)

The total energy of the NADE inside the apparent horizon is

$E_{D}=\rho_{D}V=\frac{4\pi n^{2}m_{p}^{2}r_{A}^{3}}{\eta^{2}}.$ (19)

Taking the differential form of Eq. (19) and using Eq. (12), we find

$dE_{D}=4\pi
m_{p}^{2}({{r}^{0}_{A}})^{2}H_{0}^{2}\Omega_{D}^{0}\left[3d{r}^{0}_{A}-2\frac{{r}^{0}_{A}}{n}H_{0}\sqrt{\Omega_{D}^{0}}d\eta^{0}\right].$
(20)

The associated temperature on the apparent horizon can be written as

$T=\frac{1}{2\pi r_{A}}.$ (21)

Inserting Eqs. (17), (20) and (21) into (16), we obtain

$dS_{D}^{(0)}=8\pi^{2}m_{p}^{2}({{r}^{0}_{A}})^{3}H_{0}^{2}\Omega_{D}^{0}\left[3(1+w_{D}^{0})d{r}^{0}_{A}-2\frac{{r}^{0}_{A}}{n}H_{0}\sqrt{\Omega_{D}^{0}}d\eta^{0}\right],$
(22)

Using Eq. (15) as well as relation $H_{0}d\eta^{0}=dx^{0}/a_{0}$, we find

$dS_{D}^{(0)}=16\pi^{2}m_{p}^{2}({{r}^{0}_{A}})^{3}H_{0}^{2}\Omega_{D}^{0}\frac{\sqrt{\Omega_{D}^{0}}}{na_{0}}\left[d{r}^{0}_{A}-{r}^{0}_{A}dx^{0}\right].$
(23)

Here the superscript/subscript $(0)$ denotes that in this picture our universe
is in a thermodynamical stable equilibrium.

##  III Thermodynamical description of the interacting NADE

In this section we study the case where the pressureless dark matter and the
NADE interact with each other. In this case $\rho_{m}$ and $\rho_{D}$ do not
conserve separately; they must rather enter the energy balances

$\displaystyle\dot{\rho}_{m}+3H\rho_{m}=Q,$ (24)
$\displaystyle\dot{\rho}_{D}+3H\rho_{D}(1+w_{D})=-Q.$ (25)

Here $Q$ denotes the interaction term and can be taken as
$Q=3b^{2}H(\rho_{m}+\rho_{D})$ with $b^{2}$ being a coupling constant Pav1 .
Inserting Eq. (14) into (25), we obtain the equation of state parameter of the
interacting NADE

$\displaystyle
1+w_{D}=\frac{2}{3na}\sqrt{\Omega_{D}}-\frac{Q}{9m_{p}^{2}H^{3}\Omega_{D}}.$
(26)

The evolution behavior of the NADE is now given by sheykhi1

$\displaystyle{\Omega^{\prime}_{D}}$ $\displaystyle=$
$\displaystyle\Omega_{D}\left[(1-\Omega_{D})\left(3-\frac{2}{na}\sqrt{\Omega_{D}}\right)-3b^{2}(1+\Omega_{k})+\Omega_{k}\right].$
(27)

Comparing Eq. (26) with Eq. (15), we see that the presence of the interaction
term $Q$ has provoked a change in the equation of state parameter and
consequently in the dimensionless density parameter of the dark energy
component and thus now there is no subscript above the aforesaid quantities to
denote the absence of interaction. The interacting NADE model in the non-flat
universe as described above is not anymore thermodynamically interpreted as a
state in thermodynamical equilibrium. Indeed, as soon as an interaction
between dark components is taken into account, they cannot remain in their
respective equilibrium states. The effect of interaction between the dark
components is thermodynamically interpreted as a small fluctuation around the
thermal equilibrium. It was shown Das  that due to the fluctuation, there is a
leading logarithmic correction $S^{(1)}_{D}=-\frac{1}{2}\ln(CT^{2})$ to the
thermodynamic entropy around equilibrium in all thermodynamical systems.
Therefore, the entropy of the NADE is connected with its energy and pressure
through the first law of thermodynamics

$TdS_{D}=dE_{D}+p_{D}dV,$ (28)

where now the entropy has been assigned an extra logarithmic correction Das

$S_{D}=S^{(0)}_{D}+S^{(1)}_{D},$ (29)

where the leading logarithmic correction is

$S^{(1)}_{D}=-\frac{1}{2}\ln(CT^{2}),$ (30)

and $C$ is the heat capacity defined by

$C=T\frac{\partial S^{(0)}_{D}}{\partial T}.$ (31)

It is a matter of calculation to show that

$C=-16\pi^{2}m_{p}^{2}({{r}^{0}_{A}})^{4}H_{0}^{2}\Omega_{D}^{0}\frac{\sqrt{\Omega_{D}^{0}}}{na_{0}},$
(32)

and therefore

$S^{(1)}_{D}=-\frac{1}{2}\ln\left(-4m_{p}^{2}({{r}^{0}_{A}})^{2}H_{0}^{2}\Omega_{D}^{0}\frac{\sqrt{\Omega_{D}^{0}}}{na_{0}}\right).$
(33)

Substituting the expressions for the volume, energy, and temperature in Eq.
(28) for the interacting case, we get

$dS_{D}=8\pi^{2}m_{p}^{2}{{r}^{3}_{A}}H^{2}\Omega_{D}\left[3(1+w_{D})d{r}_{A}-\frac{2{r}_{A}}{n}H\sqrt{\Omega_{D}}d\eta\right],$
(34)

or in another way

$dS_{D}=8\pi^{2}m_{p}^{2}{{r}^{3}_{A}}H^{2}\Omega_{D}\left[3(1+w_{D})d{r}_{A}-\frac{2{r}_{A}}{na}\sqrt{\Omega_{D}}dx\right],$
(35)

and thus one gets

$\displaystyle 1+w_{D}$ $\displaystyle=$
$\displaystyle\frac{1}{24\pi^{2}m_{p}^{2}{{r}^{3}_{A}}H^{2}\Omega_{D}}\frac{dS_{D}}{d{r}_{A}}+\frac{2{r}_{A}}{3na}\sqrt{\Omega_{D}}\frac{dx}{d{r}_{A}},$
(36)
$\displaystyle=\frac{1}{24\pi^{2}m_{p}^{2}{{r}^{3}_{A}}H^{2}\Omega_{D}}\left[\frac{dS^{(0)}_{D}}{d{r}_{A}}+\frac{dS^{(1)}_{D}}{d{r}_{A}}\right]+\frac{2{r}_{A}}{3na}\sqrt{\Omega_{D}}\frac{dx}{d{r}_{A}}.$

Employing Eqs. (23), (30)-(33), we can easily find

$\displaystyle\frac{dS^{(0)}_{D}}{d{r}_{A}}$ $\displaystyle=$
$\displaystyle\frac{\partial
S^{(0)}_{D}}{\partial{r}^{0}_{A}}\frac{d{r}^{0}_{A}}{d{r}_{A}}+\frac{\partial
S^{(0)}_{D}}{\partial{x}^{0}}\frac{d{x}^{0}}{d{r}_{A}}=16\pi^{2}m_{p}^{2}{({r_{A}^{0}})^{3}}H_{0}^{2}\frac{({\Omega_{D}^{0}})^{3/2}}{na_{0}}\left(\frac{d{r}^{0}_{A}}{d{r}_{A}}-{r}^{0}_{A}\frac{d{x}^{0}}{d{r}_{A}}\right),$
(37) $\displaystyle\frac{dS^{(1)}_{D}}{d{r}_{A}}$ $\displaystyle=$
$\displaystyle\frac{\partial
S^{(1)}_{D}}{\partial{r}^{0}_{A}}\frac{d{r}^{0}_{A}}{d{r}_{A}}=-\frac{1}{{r}^{0}_{A}}\frac{d{r}^{0}_{A}}{d{r}_{A}}.$
(38)

Finally, by equating expressions (26) and (36) for the equation of state
parameter of the interacting NADE evaluated on cosmological and
thermodynamical sides, respectively, one gets an expression for the
interaction term

$\displaystyle\frac{Q}{9m_{p}^{2}H^{3}}$ $\displaystyle=$
$\displaystyle\frac{2\sqrt{\Omega_{D}}}{3na}{\Omega_{D}}\left(1-r_{A}\frac{dx}{dr_{A}}\right)-\frac{1}{24\pi^{2}m_{p}^{2}{{r_{A}}^{3}}H^{2}}\left[\frac{dS^{(0)}_{D}}{dr_{A}}+\frac{dS^{(1)}_{D}}{dr_{A}}\right]$
(39) $\displaystyle=$
$\displaystyle\frac{2\sqrt{\Omega_{D}}}{3na}{\Omega_{D}}\left(1-r_{A}\frac{dx}{dr_{A}}\right)-\frac{2}{3na_{0}}\frac{H_{0}^{2}}{H^{2}}({\Omega_{D}^{0}})^{3/2}\left(\frac{{r}^{0}_{A}}{{r}_{A}}\right)^{3}\left(\frac{d{r}^{0}_{A}}{d{r}_{A}}-{r}^{0}_{A}\frac{d{x}^{0}}{d{r}_{A}}\right)$
$\displaystyle+\frac{1}{24\pi^{2}m_{p}^{2}{{r_{A}}^{3}}{r}^{0}_{A}H^{2}}\frac{d{r}^{0}_{A}}{d{r}_{A}}.$

In this way we provide the relation between the interaction term of the dark
components and the thermal fluctuation.

##  IV Summary and discusion

One of the important questions concerns the thermodynamical behavior of the
accelerated expanding universe driven by dark energy. It is interesting to ask
whether thermodynamics in an accelerating universe can reveal some properties
of dark energy. It was first pointed out in Jac  that the hyperbolic second
order partial differential Einstein equation has a predisposition to the first
law of thermodynamics. The profound connection between the thermodynamics and
the gravitational field equations has also been observed in the cosmological
situations Cai2 ; Cai3 ; CaiKim ; Wang1 ; Wang2 ; Wang3 ; Cai4 ; sheyahmad3 .
This connection implies that the thermodynamical properties can help
understand the dark energy, which gives strong motivation to study
thermodynamics in the accelerating universe.

Although at this point the interaction between dark energy and dark matter
looks purely phenomenological, but in the absence of a symmetry that forbids
the interaction there is nothing, in principle, against it. Further, the
interacting dark mater dark energy (the latter in the form of a quintessence
scalar field and the former as fermions whose mass depends on the scalar
field) has been investigated at one quantum loop with the result that the
coupling leaves the dark energy potential stable if the former is of
exponential type but it renders it unstable otherwise. Thus, microphysics
seems to allow enough room for the coupling; however, this point is not fully
settled and should be further investigated. Recently evidence was provided by
the Abell Cluster A$586$ in support of the interaction between dark energy and
dark matter Ber1 ; Ber2 .

In this Letter, we provided a thermodynamical description for the NADE model
in a universe with spacial curvature. It was shown that for an accelerating
universe the apparent horizon is a physical boundary from the thermodynamical
point of view. We explored the thermodynamical picture of the interacting NADE
model for a FRW universe enveloped by the apparent horizon. The NADE contains
some new features different from the original agegraphic dark energy and
overcome some unsatisfactory points. For instance, the original agegraphic
dark energy suffers from the difficulty to describe the matter-dominated epoch
while the NADE resolved this issue. We assumed that in the absence of a
coupling, the two dark components remain in separate thermal equilibrium and
that the presence of a small coupling between them can be described as stable
fluctuations around equilibrium. Finally, resorting to the logarithmic
correction to the equilibrium entropy we derived an expression for the
interaction term in terms of a thermal fluctuation.

###### Acknowledgements.

This work has been supported by Research Institute for Astronomy and
Astrophysics of Maragha.

## References

  * (1) A.G. Riess, et al., Astron. J. 116 (1998) 1009\.
  * (2) S. Perlmutter, et al., Astrophys. J. 517 (1999) 565\.
  * (3) S. Perlmutter, et al., Astrophys. J. 598 (2003) 102\.
  * (4) P. de Bernardis, et al., Nature 404 (2000) 955.
  * (5) L. Amendola, Phys. Rev. D 60 (1999) 043501.
  * (6) L. Amendola, Phys. Rev. D 62 (2000) 043511.
  * (7) L. Amendola and C. Quercellini, Phys. Rev. D 68 (2003) 023514.
  * (8) L. Amendola and D. Tocchini-Valentini, Phys. Rev. D 64 (2001) 043509.
  * (9) L. Amendola and D. T. Valentini, Phys. Rev. D 66 (2002) 043528.
  * (10) W. Zimdahl, D. Pavon, L.P. Chimento, Phys. Lett. B 521 (2001) 133.
  * (11) W. Zimdahl and D. Pavon, Gen. Rel. Grav. 35 (2003) 413.
  * (12) L. P. Chimento, A. S. Jakubi, D. Pavon and W. Zimdahl, Phys. Rev. D 67 (2003) 083513.
  * (13) M. R. Setare, Eur. Phys. J. C 50 (2007) 991.
  * (14) M. R. Setare, JCAP 0701 (2007) 023.
  * (15) M. R. Setare, Phys. Lett. B 654 (2007) 1.
  * (16) M. R. Setare, Phys. Lett. B 642 (2006) 421.
  * (17) M. R. Setare, Phys. Lett. B 642 (2006)1.
  * (18) B. Wang, Y. Gong and E. Abdalla, Phys. Lett. B 624 (2005) 141.
  * (19) B. Wang, C. Y. Lin and E. Abdalla, Phys. Lett. B 637 (2006) 357.
  * (20) A. Sheykhi, Phys Lett B 681 (2009) 205.
  * (21) D. Pavon, W. Zimdahl, Phys. Lett. B 628 (2005) 206.
  * (22) B. Wang, J. Zang, Ch.Y. Lin, E. Abdalla, S. Micheletti, Nucl. Phys. B 778 (2007) 69.
  * (23) S. Das, P.S. Corasaniti, J. Khoury, Phys. Rev. D 73 (2006) 083509.
  * (24) L. Amendola, S. Tsujikawa, M. Sami, Phys. Lett. B 632 (2006) 155.
  * (25) L. Amendola, C. Quercellini, Phys. Rev. D 68 (2003) 023514;
  * (26) L.P. Chimento, A.S. Jakubi, D. Pavon,W. Zimdahl, Phys. Rev. D 67 (2003) 083513.
  * (27) S. del Campo, R. Herrera, D. Pav n, Phys. Rev. D 70 (2004) 043540.
  * (28) B. Wang, C.Y Lin, D. Pavon, E. Abdalla, Phys. Lett. B 662 (2008) 1\.
  * (29) M.R. Setare, E.C. Vagenas, Phys. Lett. B 666 (2008) 111.
  * (30) R. G. Cai, Phys. Lett. B 657 (2007) 228.
  * (31) H. Wei and R. G. Cai, Phys. Lett. B 660 (2008) 113.
  * (32) H. Wei and R. G. Cai, Eur. Phys. J. C 59 (2009) 99.
  * (33) H. Wei and R. G. Cai, Phys. Lett. B 663 (2008) 1.
  * (34) Y. W. Kim, et al., Mod. Phys. Lett. A 23 (2008) 3049\.
  * (35) Y. Zhang, et al. arXiv:0708.1214.
  * (36) K. Y. Kim, H. W. Lee, Y. S. Myung, Phys.Lett. B 660 (2008) 118\.
  * (37) I. P. Neupane, Phys. Lett. B 673 (2009) 111.
  * (38) J. Zhang, X. Zhang, H. Liu, Eur. Phys. J. C 54 (2008) 303.
  * (39) J .P Wu, D. Z. Ma, Y. Ling, Phys. Lett. B 663, (2008) 152.
  * (40) A. Sheykhi, Phys. Lett. B 680 (2009) 113.
  * (41) A. Sheykhi, Phys. Lett. B 682 (2010) 329.
  * (42) A. Sheykhi, Int. J. Mod. Phys. D 18, No. 13 (2009) 2023.
  * (43) A. Sheykhi, Phys. Rev. D 81 (2010) 023525.
  * (44) A. Sheykhi, Int. J. Mod. Phys. D Vol. 19, No. 3 (2010) 305.
  * (45) M. R. Setare, arXiv:0907.4910.
  * (46) M. R. Setare, arXiv:0908.0196.
  * (47) J. Zhou, B. Wang, Y. Gong, E. Abdalla, Phys. Lett. B 652 (2007) 86.
  * (48) A. Sheykhi, Class. Quantum Grav. 27 (2010) 025007\.
  * (49) A. Sheykhi, B. Wang, Phys. Lett. B 678 (2009) 434.
  * (50) A. Sheykhi, B. Wang, Mod. Phys. Lett. A, Vol. 25, No. 14 (2010) 1199\.
  * (51) D. N. Spergel, Astrophys. J. Suppl. 148 (2003) 175.
  * (52) C. L. Bennett, et al., Astrophys. J. Suppl. 148 (2003) 1.
  * (53) U. Seljak, A. Slosar, P. McDonald, JCAP 0610 (2006) 014\.
  * (54) D. N. Spergel, et al., Astrophys. J. Suppl. 170 (2007) 377.
  * (55) F. Karolyhazy, Nuovo.Cim. A 42 (1966) 390.
  * (56) F. Karolyhazy, A. Frenkel and B. Lukacs, in Physics as natural Philosophy edited by A. Shimony and H. Feschbach, MIT Press, Cambridge, MA, (1982).
  * (57) F. Karolyhazy, A. Frenkel and B. Lukacs, in Quantum Concepts in Space and Time edited by R. Penrose and C.J. Isham, Clarendon Press, Oxford, (1986).
  * (58) N. Sasakura, Prog. Theor. Phys., Vol. 102, No. 1 (1999) 169.
  * (59) M. Maziashvili Int. J. Mod. Phys. D 16 (2007) 1531.
  * (60) M. Maziashvili, Phys. Lett. B 652 (2007) 165.
  * (61) S. A. Hayward, S.Mukohyana, andM. C. Ashworth, Phys. Lett. A 256, 347 (1999).
  * (62) S. A. Hayward, Class. Quantum Grav. 15, 3147 (1998).
  * (63) D. Bak and S. J. Rey, Class. Quantum Grav. 17, L83 (2000).
  * (64) A. Sheykhi, B. Wang and R. G. Cai, Nucl. Phys. B 779 (2007)1.
  * (65) A. Sheykhi, B. Wang and R. G. Cai, Phys. Rev. D 76 (2007) 023515.
  * (66) S. Das, P. Majumdar, R.K. Bhaduri, Class. Quantum Grav. 19 (2002) 2355.
  * (67) T. Jacobson, Phys. Rev. Lett. 75, 1260 (1995).
  * (68) M. Akbar and R. G. Cai, Phys. Rev. D 75, 084003 (2007).
  * (69) R. G. Cai and L. M. Cao, Phys.Rev. D 75, 064008 (2007).
  * (70) R. G. Cai and S. P. Kim, JHEP 0502, 050 (2005).
  * (71) B. Wang, E. Abdalla and R. K. Su, Phys.Lett. B 503, 394 (2001)
  * (72) B. Wang, E. Abdalla and R. K. Su, Mod. Phys. Lett. A 17, 23 (2002).
  * (73) R. G. Cai and Y. S. Myung, Phys. Rev. D 67, 124021 (2003).
  * (74) R. G. Cai and L. M. Cao, Nucl. Phys. B 785 (2007) 135.
  * (75) A. Sheykhi, JCAP 05 (2009) 019.
  * (76) O. Bertolami, F. Gil Pedro, M. Le Delliou, Phys. Lett. B 654 (2007) 165.
  * (77) M. Le Delliou, O. Bertolami, F. Gil Pedro, AIP Conf. Proc. 957 (2007) 421.