# Sentence-Level BERT and Multi-Task Learning of Age and Gender in Social
Media

Muhammad Abdul-Mageed1 Chiyu Zhang1 Arun Rajendran1
AbdelRahim Elmadany1 Michael Przystupa1 Lyle Ungar2
1Natural Language Processing Lab, University of British Columbia
2Computer and Information Science, University of Pennsylvania
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Social media currently provide a window on our lives, making it possible to
learn how people from different places, with different backgrounds, ages, and
genders use language. In this work we exploit a newly-created Arabic dataset
with ground truth age and gender labels to learn these attributes both
individually and in a multi-task setting at the sentence level. Our models are
based on variations of deep bidirectional neural networks. More specifically,
we build models with gated recurrent units and bidirectional encoder
representations from transformers (BERT). We show the utility of multi-task
learning (MTL) on the two tasks and identify task-specific attention as a
superior choice in this context. We also find that a single-task BERT model
outperform our best MTL models on the two tasks. We report tweet-level
accuracy of 51.43% for the age task (three-way) and 65.30% on the gender task
(binary), both of which outperforms our baselines with a large margin. Our
models are language-agnostic, and so can be applied to other languages.

utf8

##  1 Introduction

Wide use of social media has made it possible to collect large amounts of data
from people belonging to various types of backgrounds. These data can then be
used for developing natural language processing models capable of predicting
user attributes such as location, gender, age, and personality type. While
there has been multiple works on learning these tasks, most of them focus on
single-task learning rather than trying to learn more than one task
simultaneously. This is sub-optimal and different from how humans learn
Caruana (1997).

Figure 1: Illustration of our MTL network with shared hiddent layers and a
task-specific attention layer.

In this work, we investigate the utility of multi-task learning (MTL), where
more than one task is learned simultaneously in a single model with an overall
loss that is the sum of each independent task’s loss. We focus on learning the
two tasks of age and gender at the sentence level, exploring a deep neural
network paradigm where the model’s hidden layers are shared in the MTL setting
with (1) a shared attention layer but also (2) with task-specific attention
layers. An illustration of our MTL network with task-specific attention is
presented in Figure 1.

In a related vein, recent progress in representation learning of natural
language has resulted in a proliferation of methods. These include methods
aimed at improving input to the neural network by learning better embeddings
either exploiting parallel data in machine translation tasks (as in CoVe
McCann et al. (2017)) or by enhancing these embedding by exploiting
bidirectional language models (as in ELMo Peters et al. (2018)). Other
approaches such as the Transformer model Vaswani et al. (2017), which
dispenses entirely with both the recurrence and convolution approaches
dominant in neural networks, and its extensions such as the work of Radford et
al. (2018) and Devlin et al. (2018) that leverage Transformers for pre-
training. These methods have hardly been used in Arabic NLP and hence it
remains unclear what their utilities are and how it is that they can be best
deployed for the language. Toward bridging this gap, we explore the benefit of
using a specific type of Transformers based on bidirectional, masked language
models (BERT) Devlin et al. (2018).

Finally, we note that in spite of the Arabic world being a large region with a
huge population ( 300M) and a wide use of social media, tools for
understanding attributes of Arabic users online are still lacking. In spite of
the important and worthy privacy concerns related to user online behavior,
such tools can be valuable for making discoveries about whole communities and
as means to facilitate humanitarian (as in disaster), educational (as in
targeted mediated pedagogy), and safety causes (as in detecting hate speech
and extremist ideology), to list a few. Our models offer a step forward toward
building tools for online Arabic user modeling.

Overall, we make the following contributions: (1) We develop novel models
targeting the two tasks of age and gender; (2) we investigate and establish
the utility of multi-task learning on the these two tasks, showing the
advantage of task-specific attention (vs. shared attention, see Section 4.3);
and (3) we carry out BERT-based modeling on the two tasks. The rest of the
paper is organized as follows: In Section 2 we describe the dataset. In
Section 3 we introduce our various methods. We present our experimental setup
and our models in Section 4, and related works in Section 6. We conclude in
Section 7.

##  2 Data

Figure 2: Arabic countries covered in the dataset.

Data split | Under 25 | 25 until 34 | 35 and up | #tweets
---|---|---|---|---
| Female | Male | Female | Male | Female | Male |
TRAIN | 215,950 | 213,249 | 207,184 | 248,769 | 174,511 | 226,132 | 1,285,795
DEV | 27,076 | 26,551 | 25,750 | 31,111 | 21,942 | 28,294 | 160,724
TEST | 26,878 | 26,422 | 25,905 | 31,211 | 21,991 | 28,318 | 160,725
ALL | 269,904 | 266,222 | 258,839 | 311,091 | 218,444 | 282,744 | 1,607,244

Table 1: Distribution of age and gender classes in our data splits

We make use of Arap-Tweet Zaghouani and Charfi (2018), which we will refer to
as Arab-Tweet. Arab-tweet is a dataset of tweets covering 11 Arabic regions
from 17 different countries. 111Please note Zaghouani and Charfi (2018)
mention the dataset covers 16 countries, but in the distribution we received
we found the data cover 17 countries. For each region, data from 100 Twitter
users were crawled. Users needed to have posted at least 2,000 and were
selected based on an initial list of seed words characteristic of each region.
The seed list included words such as ¡برشة¿ /barsha/ ‘many’ for Tunisian
Arabic and ¡وايد¿ /wayed/ ‘many’ for Gulf Arabic. Zaghouani and Charfi (2018)
employed human annotators to verify that users do belong to each respective
region. Annotators also assigned gender labels from the set male, female and
age group labels from the set under-25, 25-to34, above-35 at the user-level,
which in turn is assigned at tweet level. Tweets with less than 3 words and
re-tweets were removed. Refer to Zaghouani and Charfi (2018) for details about
how annotation was carried out. We provide a description of the data in Table
1. (Table 1 also provides class breakdown across our splits, as we explain in
more detail in Section 4.1). 222We provide randomly sampled examples from the
various age and gender categories in the supplementary material. We note that
Zaghouani and Charfi (2018) do not report classification models exploiting the
data.

##  3 Methods

###  3.1 GRU

For comparison, we employ gated recurrent units (GRU) Cho et al. (2014), a
simplification of long-short term memory networks (LSTMs) Hochreiter and
Schmidhuber (1997). Like LSTMs, GRUs can address the problem of long-range
dependencies in recurrent neural networks. Unlike an LSTM, a GRU does not have
a memory cell but makes use of an update gate $\textbf{{z}}^{(t)}$ and a reset
gate $\textbf{{r}}^{(t)}$. The activation of GRU at time step $t$ is a linear
interpolation of the previous activation hidden state $\textbf{{h}}^{(t-1)}$
and the candidate activation hidden state $\textbf{{$\widetilde{h}$}}^{(t)}$.
The update state $\text{{z}}^{(t)}$ decides how much the unit updates its
content. Essentially, the forget gate of an LSTM and an update gate of a GRU
each acts as a manager of information flow where they allow the network to
update its state by adding content from the current cell to that of a previous
cell, rather than overwriting previous cell content (as is the case in vanilla
RNNs).

###  3.2 Multi-Task Learning

In addition to attempting to learn each of the two tasks independently, we
explore the utility of learning them together in a multi-task learning (MTL)
framework. The main intuition behind MTL is that most real-world tasks are
multi-task problems that humans learn together. Hence, when we treat problems
as single-tasks, we might be sacrificing performance Caruana (1993). For
related tasks, MTL helps achieve inductive transfer between these tasks by
leveraging additional sources of information (from other tasks) to improve
performance on a given task (current task). Caruana (1997). MTL also helps
regularize models. The framework has been successfully employed to several NLP
tasks such as machine translation and syntactic parsing Luong et al. (2015)
and syntactic chunking, POS tagging, and CCG supertagging Søgaard and Goldberg
(2016).

In the single task scenario, an independent network is designed for each task.
Each network is trained in isolation, with its own backpropagation. For
example, Figure 3 is an illustration of our network for learning gender and a
similar, independent network is devoted for learning age. In MTL, these tasks
would be learned together in a single network with 2 different outputs, one
for each task. The MTL network would have one or more hidden layers that are
shared between all the tasks. Backpropagation is then applied in parallel on
the 2 outputs.Figure 1 presented earlier is an illustration of an MTL network
for learning age and gender, with 1 shared hidden layer.

In our current work, each of the two independent networks has its own loss
function. Formally, let $\\{x^{(i)},y^{(i)}\\}_{i=1}^{N}$ denote the training
data, where $x^{(i)}\in R^{d}$ is the input vector,
$y^{(i)}\in\\{1,\dots,C^{(y)}\\}$ is the class label of the i-th data point, d
is the dimensionality of the input vector, $C^{(y)}$ is the number of classes,
and N is the number of data points. The cross-entropy function is calculated
as:

$\mathcal{L}(\theta)\\\ =-\sum_{i}logP\left(y^{(i)}|x^{(i)};\theta\right)\\\ $
(1)

For gender, we calculate binary cross-entropy with
$y^{(i)}\in\\{0=male,1=female\\}$, and in age we compute a categorical cross-
entropy as $y^{(i)}\in\\{0=under-25,1=25-to-34,2=above-34\\}$. Our MTL loss
can then be computed as:

$\mathcal{L}(\theta_{MTL})=\left(\mathcal{L}(\theta_{age})+\mathcal{L}(\theta_{gender})\right)/2\\\
$ (2)

We now introduce the Transformer Vaswani et al. (2017), on which our attention
mechanism is based. The Transformer is also the backbone of the BERT model
Devlin et al. (2018) that we employ as we will explain in Section 3.5.

###  3.3 Transformer

Vaswani et al. (2017) introduced the Transformer, which is based solely on
attention, thus allowing for parallelizing the network and dispensing of
recurrence and convolution. Similar to sequence transduction models such as
Bahdanau et al. (2014); Cho et al. (2014); Sutskever et al. (2014), the
Transformer is an encoder-decoder architecture. The encoder takes a sequence
of symbol representations $x^{(i)}\dots x^{(n)}$, maps them into a sequence of
continuous representations $z^{(i)}\dots x^{(n)}$ that are then used by the
decoder to generate an output sequence $y^{(i)}\dots y^{(n)}$, one symbol at a
time. This is performed using self-attention, where different positions of a
single sequence are related to one another. The Transformer employs an
attention mechanism based on a function that operates on queries, keys, and
values. The attention function maps a query and a set of key-value pairs to an
output, where the output is a weighted sum of the values. For each value, a
weight is computed as a compatibility function of the query with the
corresponding key.

We implement this attention method in our BiGRU models as an attention layer.
As such, we still use a BiGRU architecture that is a variation of RNNs but
that has the Transformer’s self-attention mechanism. In our BiGRUs, the query
is the hidden state vector from the BiGRU and both the key and value are the
same output vector of the network. The query, key, and value have the same
dimension $d_{k}=500$. This particular version of attention is a scaled dot
product of queries and keys (each of $d_{k}$) that is scaled by a factor of
$\frac{1}{\sqrt{d_{k}}}$ on which a softmax is applied to acquire the weights
on the values. The scaled dot product attention is computed as as a set of
queries, keys, and values in three matrices Q, K, and V, respectively,
follows:

$Attention\left(\textit{Q, K,
V}\right)=softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V\\\ $ (3)

Encoder of the Transformer in Vaswani et al. (2017) has 6 attention layers,
each of which is composed of two sub-layers. The first sub-layer is a multi-
head attention where, rather than performing a single attention function with
queries, keys, and values, these are are projected h times into linear,
learned projections and ultimately concatenated. The second sub-layer is a
simple, fully-connected feed-forward network (FFN), with two linear layers and
a ReLU activation function in-between, that is applied to each position
separately and identically. Decoder of the Transformer also employs 6
identical layers, similar to the encoder, yet with an extra/third sub-layer
that performs multi-head attention over the encoder stack. More information
about the Transformer can be found in Vaswani et al. (2017). In addition to
Transformer self-attention, we also experiment with supervised attention based
on mutual information, which we now introduce.

###  3.4 Supervised Attention.

We propose a novel method of supervised attention that uses implicit
vocabulary list obtained from mututal information (MI). MI is a measure of the
similarity between two labels (e.g., “male” and “female” in the case of
gender) of the same N objects (discrete features, or words in our case). Where
$|U^{i}|$ is the number of the samples in class $U^{i}$ and $|V^{j}|$ is the
number of the samples in class $V^{j}$, the MI between clusterings $U$ and $V$
is given (in set cardinality formulation) by: 333We use scikit-learn to
acquire MI over words. For more information, see: https://scikit-
learn.org/stable/modules/clustering.html#mutual-info-score.

$MI(U,V)=\sum_{i=1}^{|U|}\sum_{j=1}^{|V|}\frac{|U^{i}\cap
V^{j}|}{N}\log\frac{N|U^{i}\cap V^{j}|}{|U^{i}||V^{j}|}$ (4)

We take the top 500 words based on mutual information score for each label and
we use this list of words to perform the supervised attention. We employ the
supervised attention for each of the labels using their respective lists. The
gold attention weights are calculated based on the presence of the words from
the list. For example, if we have four words in the sentence from the list,
then our gold attention weights vector will have 0.25 (1/4) at the
corresponding indexes. In absence of all words from the gold list, we follow
Liu et al. (2017) in using the regular attention mechanism, thus allowing our
models to perform our multi-head attention. As mentioned earlier, we also use
BERT Devlin et al. (2018) to learn age and gender independently. We now
introduce BERT.

###  3.5 BERT

BERT Devlin et al. (2018) stands for Bidirectional Encoder Representations
from Transformers. BERT is an approach for pre-training language
representations. BERT involves two unsupervised learning tasks, (1) masked
language models (Masked LM) and (2) next sentence prediction. Since BERT uses
bidirectional conditioning, a given percentage of random input tokens are
masked and the model attempts to predict these masked tokens. This is the
Masked LM task, where masked tokens are simply replaced by a string [MASK] .
Devlin et al. (2018) mask 15% of the tokens (the authors use WordPieces) and
feed the final hidden vectors of these masked tokens to an output softmax over
the vocabulary. The next sentence prediction task of BERT is also
straightforward. The authors simply cast the task as binary classification.
For a given sentence S, two sentences A and B are generated where A (positive
class) is an actual sentence from the corpus and B is a randomly chosen
sentence (negative class). Once trained on an unlabeled dataset, BERT can then
be fine-tuned with supervised data for a downstream task (e.g., text
classification, question answering). We now introduce our experiments.

##  4 Experiments

We detail how we split and pre-process our data and list our baselines and
evaluation metrics here. We then present (i) our single-task models (Section
4.2), (ii) multi-task models (Section 4.3), and (iii) the BERT model (Section
4.4).

###  4.1 Data Splits, Baselines, and Evaluation

We shuffle the dataset and split it into 80% training (TRAIN), 10% development
(DEV), and 10% test (TEST). The distribution of classes in our splits is in
Table 1. For pre-processing, we reduce 2 or more consecutive repetitions of
the same character into only 2 and remove diacritics. We have two baselines,
(1) (maj-base), is the majority class in our TRAIN and (2) a small
unidirectional GRU (small-GRU) with a single 500-units hidden layer. We train
the small GRU with the same batch size = 8 and dropout =0.5 as our bigger
BiGRUs (see Section 4.2), but with no attention. We report our results in
accuracy.

###  4.2 Single-Task BiGRUs

Figure 3: Illustration of our network architecture for gender classification
(single task network)

Figure 4: Illustration of our 3-Layered MTL networks. Left: Network with
shared attention; Right: Network with task-specific attention. We visualize 2
shared BiGRU layers, to emphasize the idea of sharing hidden layers. Our best
performance, however, is acquired with only 1 shared BiGRU layer at the bottom
of each network, followed by another BiGRU layer (either shared or task-
specific) on which attention is applied.

We first design an independent network for each of the two tasks. We use the
same network architecture across the 2 tasks. For each network, we experiment
with 2 and 3 hidden BiGRU layers, each with 1,000 units (500 units in each
direction). 444We found the networks with 3 hidden layers to overfit and so we
report only networks with 2 hidden layers in all our experiments. We add
Transformer-style attention only to the last hidden layer. We trim each
sequence at 30 words, and use a batch size of 8. Each word in the input
sequence is represented as a vector of 300 dimensions learned directly from
the data. Word vectors weights $W$ are initialized with a standard normal
distribution, with $\mu=0$, and $\sigma=1$, i.e., $W\sim N(0,1)$. Figure 3
offers an illustration of the single-task network for gender. The network for
age is similar, with the exception that its output layer has the 3 units (for
the 3 age classes in the data). For optimization, we use Adam Kingma and Ba
(2014) with a fixed learning rate of $1e-3$. For regularization, we use
dropout Srivastava et al. (2014) with a value of $0.5$ on each of the hidden
layers. We run each network for 10 epochs, with early stopping, choosing the
model that performs highest on DEV as our best model. We then run each best
model on TEST, and report accuracy. Our best results on age are acquired with
3 epochs, and for gender with 2 epochs. Tables 6 and 3 presents our results on
DEV and TEST respectively. (The tables also show results for our subsequent
experiments with MTL and BERT on DEV and TEST). On TEST, Attn-BiGRU is 10.78%
better than small-GRU for age and 8.30% higher than the majority class
baseline for gender. We note that results on DEV are very close to those on
TEST. We now describe our MTL experiments.

###  4.3 MTL BiGRUs for Age and Gender

Setting | Age | Gender
---|---|---
maj-base | 35.53 | 53.47
small-GRU | 36.13 | 53.39
Attn-BiGRU | 47.03 | 61.64
MTL-common-attn | 47.85 | 62.50
MTL-spec-attn | 47.92 | 63.09
MTL-sprvsd-spec-attn | 48.23 | 62.99
MTL-MI-spec-attn | 47.90 | 63.15
BERT | 50.95 | 65.31
Table 2: Model performance in accuracy on DEV. Baseline 1, maj-base, is the majority class in our TRAIN. Baseline 2, small-GRU is a unidirectional, 500-unit, one-layered GRU. For each block, best results that are higher than the baselines and the best results in preceding block(s) are underlined. Best result in each task is in bold. For single task models, best results for age are with 3 epochs and for gender with 2 epochs. For multi-task models, best accuracy is acquired with epochs between 3 and 6. 666We provide a table in the supplementary material with information about best epoch per model. Setting | Age | Gender
---|---|---
maj-base | 35.53 | 53.47
small-GRU | 36.29 | 53.37
Attn-BiGRU | 47.07 | 61.77
MTL-common-attn | 47.82 | 62.36
MTL-spec-attn | 47.99 | 63.04
MTL-sprvsd-spec-attn | 48.25 | 62.81
MTL-MI-spec-attn | 47.81 | 63.13
BERT | 51.42 | 65.30
Table 3: Model performance in accuracy on TEST.

To learn the two tasks simultaneously, we use a single network with 2 hidden
layers each with the same dimensionality as the single-task networks (1,000
units in each layer). 777We also experiment with a network with 3 hidden
layers, but we find it sub-optimal and so we do not discuss it further. For
comparison, we also use the same hyper-parameters as the single-task models
for optimization with Adam, dropout regularization, and run for 10 epochs. We
share the parameters of the first (bottom) hidden BiGRU layer across the three
tasks. For the attention BiGRU layer (2nd layer), we have three settings:

Word | Eng | MTL-S
---|---|---
¡يسالونني¿ | they ask me | 0.169779
¡ياغالين¿ | you dear (+plural) | 0.164816
¡منيحاا¿ | beautiful (+fem) | 0.164619
¡مسساء¿ | evening | 0.161141
¡سيصادفك¿ | you’ll meet by chance | 0.153956
¡بونسوار¿ | good evening (French) | 0.147900
¡سيصيبك¿ | it’ll befall you | 0.147765
¡شلوونكم¿ | how are you (+plural) | 0.146776
¡امبيه¿ | I want it | 0.146680
¡ياعيون¿ | you, darling | 0.142256
¡ياصدفه¿ | what a surprise | 0.142011
¡انزيين¿ | we beautify | 0.141639
¡صبحهم¿ | good morning | 0.139647
¡ياسااتر¿ | God protect | 0.138652
¡سابتسم¿ | I’ll smile | 0.138609
Table 4: Top 15 most highly weighted words based on average attention weights
from our MTL-spec-attn (MTL-S) network for gender.

(i) Shared Attention: In this setting, we share the attention BiGRU layer
across the two tasks. As such, this network has all its layers shared across
the two tasks. We refer to this network as MTL-common-attn. This architecture
is aimed at investigating the utility of allowing the network to attend
simultaneously to each of the two tasks.

(ii) Task-Specific Attention: For this network, while the first hidden
layer(s) are shared, the last layer (attention layer) is task-specific (i.e.,
independent for each task). We will call this model MTL-spec-attn. The goal of
this architecture is to investigate the feasibility of allowing each task to
specialize its own attention, while also sharing information (i.e., the bottom
layer) with the other task.

(iii) Supervised Task-Specific Attention: We perform supervised attention
based on mutual information (MI) with a list of 500 weighted words, as
explained in Section 3.4. As Table 3 shows, for the two tasks, MTL is
consistently better than single task learning (and our baselines). We observe
that task-specific attention is superior to shared attention across the two
tasks, in both DEV and TEST. We also observe that supervised attention
improves over regular self-attention for the age task (0.31% acc on DEV and
0.26% on TEST), but not for gender. This shows that while the MTL is useful,
it is better to specialize the attention layer. It seems this specialized
architecture allows the network to identify representations that are harder to
disentangle in the shared attention setting.

MI-Based MTL: Rather than using frequency as a basis for choosing our input
vocabulary, we experiment with using a vocabulary weighted by MI. More
specifically, we use the top 100K MI-weighted words as our vocabulary in the
embedding layer. For this setting, we only run our best MTL network from
previous experiments (MTL with task-specific attention). We refer to this
network as MTL-MI-spec-attn. We note that this network improves performance
slightly over all our previous models on the gender task.

####  4.3.1 Interpreting Attention in MTL

We have established the effectiveness of attention in the context of our MTL
networks. To further understand model behavior, we extract words most highly
attended to by our MTL-spec-attn network for the gender task. Table 4 provides
the top 15 examples among these words. Intuitively, these words seem relevant
for distinguishing gender. We provide more samples of words attended to by our
single and MTL models in the supplementary material, along with corresponding
MI scores. We observe these words are assigned very low MI scores and are not
top ranked by MI 888MI scores and neural model weights are not directly
comparable, but we find words with higher weights from the MTL-spec-attn
network to be more intuitive, thereby highlighting how the neural network is
capable of capturing nuances not necessarily identified by the statistical
approach. We hypothesize this is partially because MI does not operate over
distributed representations of words, but rather treat them as atomic symbols.
999We provide more samples of words attended to by our single and MTL models
in the supplementary material, along with corresponding MI scores. For further
probing into our model behavior, we provide an attention-based visualization
of a number of hand-picked samples along with their English translations in
the supplementary material.

###  4.4 BERT-Based Modeling

We use the BERT-Base, Multilingual Cased model released by the authors
101010https://github.com/google-research/bert/blob/master/multilingual.md. to
model each of our two tasks. The model is trained on 104 languages, including
Arabic, with 12 layer, 768 hidden units each, 12 attention heads, and has 110M
parameters. The model has 119,547 word pieces shared across all the languages.
For fine-tuning, we use a maximum sequence size of 30 words and a batch size
of 32. We set the learning rate to 2e-5 and train for 15 epochs. From Tables 6
and 3, we can see that BERT performs consistently better than the two
baselines on the two tasks, irrespective of the data split. On TEST, it
improves with 15.13% acc. (for age) and 11.83% acc. (for gender) over the best
baseline. BERT also outperforms our best MTL models. Also, BERT outperforms
MTL-suprvsd-spec-attn on age with 3.17% acc and MTL-spec-attn on gender with
2.17% acc on TEST. These results demonstrate (i) the effectiveness of BERT on
our two tasks, and (ii) the relative utility of our MTL models, which still
perform well even though they do not need pre-training on a huge amount of
data like BERT.

##  5 Model Evaluation on External Data

Tweet-Level Gender Evaluation: In order to test how our best model (BERT) will
generalize to external data, we develop a new, additional dataset EXTERNAL
only for the gender task. To acquire EXTERNAL, we manually label an in-house
dataset of $\sim$ 2,124 users from 20 Arab countries (including all countries
in Arab-Tweet) 111111Users are manually assigned country tags in the data.
with gender tags. EXTERNAL has data from 550 female users, 1,335 male users,
and 239 users with unknown gender. For a balanced dataset, we only keep each
gender at 550 users, for a total of 1,100 users contributing $\sim$1,320K
tweets. We split EXTERNAL into DEV-EXT and TEST-EXT. We run BERT on each of
these splits. We acquire 61.31% acc on DEV-EXT and 61.46% acc on TEST-EXT. We
note these results are 4% and 3.84% acc less than our performance on Arab-
Tweet DEV and TEST, respectively. We hypothesize this could be due to (i)
diachronic degradation of language models (Jaidka et al., 2018), since
EXTERNAL covers only up until the year 2017; but also (ii) EXTERNAL being
harder and more diverse since, unlike Arab-Tweet, it was not collected with a
seed word approach.

##  6 Related Works

###  6.1 Arabic

Arabic is a collection of language varieties, including a standard variety
that is referred to as modern standard Arabic (MSA). MSA is more popular in
pan-Arabic media and formal communication (such as in educational settings)
and literary and religious venues. Dialectal Arabic is used widely by Arabs in
their daily lives. Arabic varieties differ at various linguistic levels,
including phonological, morphological, lexical, and syntactic differences
Holes (2004); Bassiouney (2009); Palva (2006). Traditionally, Arabic dialects
have been categorized based on coarse-grained geographical divisions that
cross country boundaries, e.g., Habash (2010); Versteegh (2014). More
recently, there has been interest in more fine-grained differences, e.g.,
Mubarak and Darwish (2014); Sadat et al. (2014); Abdul-Mageed et al. (2018);
Salameh and Bouamor (2018); Qwaider et al. (2018).

###  6.2 Author Profiling

Detecting sociological and psychological attributes of a user is usually
referred to as author profiling. This typically involves tasks such as
detecting author gender and age (see Section 6.3), personality (Schwartz et
al., 2013; Matz et al., 2017; Bleidorn and Hopwood, 2018; Hinds and Joinson,
2019), political orientation (Colleoni et al., 2014; Preoţiuc-Pietro et al.,
2017), and moral traits (Johnson and Goldwasser, 2018; Pang et al., 2019).
Related terms include author attribution and forensic authorship analysis
(Wright, 2013), where efforts are focused on identifying the specific author
of a given text. Text-based approaches to author profiling (e.g., (Gamon,
2004; Argamon, 2019)) are rooted in computational stylometry (Goswami et al.,
2009; Verhoeven, 2018) and can be traced back to the work of Holmes (1998).
From a linguistic perspective, this line of work is based on the concept of
idiolect, the tenet that language is a property of the individual Hudson
(1996); Barlow (2013, 2018) as contrasted to the idea of language being the
property of a speech community (Labov, 1989). Network-based approaches involve
making use of online relations such as friendships, followings, mentions, and
comments (Mitrou et al., 2014) for author profiling. Notably, the PAN author
profiling shared task Rangel et al. (2013, 2014); Rangel Pardo et al. (2015);
Rangel et al. (2016, 2017) was launched to progress work in this area. A
summary of the PAN work can be found in (Potthast et al., 2019)

###  6.3 Age and Gender

Several works have been conducted on English-based age and gender detection.
These include Rao et al. (2010); Flekova et al. (2016); Burger et al. (2011);
Verhoeven et al. (2016); Daneshvar and Inkpen (2018). Several of these works
make use of text n-gram and topic models features (Schwartz et al., 2013; Sap
et al., 2014). In addition, for age, many studies cast the as multi-class
classification, with several age group categories (e.g., ages 10-19, 20-29,
30-39). Other works cast age as a regression (Nguyen et al., 2011) or it with
classification Chen et al. (2019). There are also works that target age and
gender prediction from facial data (Levi and Hassner, 2015) or a mixture of
text and image data (Vijayaraghavan et al., 2017). We also note that most of
the literature is on English, with only few focusing on cross-lingual settings
for gender (van der Goot et al., 2018), and age and personality (Verhoeven et
al., 2016). Minimal works exist on other languages (Verhoeven, 2018). For
Arabic, with the exception of some recent treatments Rangel et al. (2017);
Alrifai et al. (2017), there is not much work we know of that is focused on
learning these tasks.

##  7 Conclusion

In this work, we reported our models targeting age and gender detection in
Arabic social media. We have shown the utility of MTL, especially with task-
specific attention, on the two learning tasks. Even though our models do not
need to be pre-trained on huge amounts of data, as is the case with BERT, they
perform well on both tasks. We have also shown the utility of the newly-
released BERT model. Our models can be deployed for real-world social media
analysis at scale, since they can cover 17 Arabic countries. The models are
also language-agnostic, and so can be applied to other languages. This is one
of our future directions.

## References

  * Abdul-Mageed et al. (2018) Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018.  You tweet what you speak: A city-level dataset of arabic dialects.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018)_.
  * Alrifai et al. (2017) Khaled Alrifai, Ghaida Rebdawi, and Nada Ghneim. 2017.  Arabic tweeps gender and dialect prediction.  In _CLEF (Working Notes)_.
  * Argamon (2019) Shlomo Engelson Argamon. 2019.  Register in computational language research.  _Register Studies_ , 1(1):100–135.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_.
  * Barlow (2013) Michael Barlow. 2013.  Individual differences and usage-based grammar.  _International Journal of Corpus Linguistics_ , 18(4):443–478.
  * Barlow (2018) Michael Barlow. 2018.  The individual and the group from a corpus perspective.  _The Corpus Linguistics Discourse: In honour of Wolfgang Teubert_ , 87:163.
  * Bassiouney (2009) Reem Bassiouney. 2009.  _Arabic sociolinguistics_.  Edinburgh University Press.
  * Bleidorn and Hopwood (2018) Wiebke Bleidorn and Christopher James Hopwood. 2018.  Using machine learning to advance personality assessment and theory.  _Personality and Social Psychology Review_ , page 1088868318772990\.
  * Burger et al. (2011) John D Burger, John Henderson, George Kim, and Guido Zarrella. 2011.  Discriminating gender on twitter.  In _Proceedings of the conference on empirical methods in natural language processing_ , pages 1301–1309. Association for Computational Linguistics.
  * Caruana (1993) R. Caruana. 1993.  Multitask learning: A knowledge-based source of inductive bias.  In _Proceedings of the 10th International Conference on Machine Learning_.
  * Caruana (1997) Rich Caruana. 1997.  Multitask learning.  _Machine learning_ , 28(1):41–75.
  * Chen et al. (2019) Jing Chen, Long Cheng, Xi Yang, Jun Liang, Bing Quan, and Shoushan Li. 2019.  Joint learning with both classification and regression models for age prediction.  In _Journal of Physics: Conference Series_ , volume 1168, page 032016\. IOP Publishing.
  * Cho et al. (2014) Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  _arXiv preprint arXiv:1406.1078_.
  * Colleoni et al. (2014) Elanor Colleoni, Alessandro Rozza, and Adam Arvidsson. 2014.  Echo chamber or public sphere? predicting political orientation and measuring political homophily in twitter using big data.  _Journal of communication_ , 64(2):317–332.
  * Daneshvar and Inkpen (2018) Saman Daneshvar and Diana Inkpen. 2018.  Gender identification in twitter using n-grams and lsa.  In _Proceedings of the Ninth International Conference of the CLEF Association (CLEF 2018)_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Flekova et al. (2016) Lucie Flekova, Daniel Preoţiuc-Pietro, and Lyle Ungar. 2016.  Exploring stylistic variation with age and income on twitter.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , volume 2, pages 313–319.
  * Gamon (2004) Michael Gamon. 2004.  Linguistic correlates of style: authorship classification with deep linguistic analysis features.  In _Proceedings of the 20th international conference on Computational Linguistics_ , page 611. Association for Computational Linguistics.
  * van der Goot et al. (2018) Rob van der Goot, Nikola Ljubešić, Ian Matroos, Malvina Nissim, and Barbara Plank. 2018.  Bleaching text: Abstract features for cross-lingual gender prediction.  _arXiv preprint arXiv:1805.03122_.
  * Goswami et al. (2009) Sumit Goswami, Sudeshna Sarkar, and Mayur Rustagi. 2009.  Stylometric analysis of bloggers’ age and gender.  In _Third international AAAI conference on weblogs and social media_.
  * Habash (2010) Nizar Y Habash. 2010.  Introduction to arabic natural language processing.  _Synthesis Lectures on Human Language Technologies_ , 3(1):1–187.
  * Hinds and Joinson (2019) Joanne Hinds and Adam Joinson. 2019.  Human and computer personality prediction from digital footprints.  _Current Directions in Psychological Science_ , page 0963721419827849\.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780.
  * Holes (2004) Clive Holes. 2004.  _Modern Arabic: Structures, functions, and varieties_.  Georgetown University Press.
  * Holmes (1998) David I Holmes. 1998.  The evolution of stylometry in humanities scholarship.  _Literary and linguistic computing_ , 13(3):111–117.
  * Hudson (1996) Richard Anthony Hudson. 1996.  _Sociolinguistics_.  Cambridge university press.
  * Jaidka et al. (2018) Kokil Jaidka, Niyati Chhaya, and Lyle Ungar. 2018.  Diachronic degradation of language models: Insights from social media.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 195–200.
  * Johnson and Goldwasser (2018) Kristen Johnson and Dan Goldwasser. 2018.  Classification of moral foundations in microblog political discourse.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 720–730.
  * Kingma and Ba (2014) Diederik Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Labov (1989) William Labov. 1989.  Exact description of the speech community: Short a in philadelphia.  _Language change and variation_ , pages 1–57.
  * Levi and Hassner (2015) Gil Levi and Tal Hassner. 2015.  Age and gender classification using convolutional neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , pages 34–42.
  * Liu et al. (2017) Chenxi Liu, Junhua Mao, Fei Sha, and Alan Yuille. 2017.  Attention correctness in neural image captioning.  In _Thirty-First AAAI Conference on Artificial Intelligence_.
  * Luong et al. (2015) Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015\.  Multi-task sequence to sequence learning.  _arXiv preprint arXiv:1511.06114_.
  * Matz et al. (2017) Sandra C Matz, Michal Kosinski, Gideon Nave, and David J Stillwell. 2017.  Psychological targeting as an effective approach to digital mass persuasion.  _Proceedings of the national academy of sciences_ , 114(48):12714–12719.
  * McCann et al. (2017) Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017.  Learned in translation: Contextualized word vectors.  In _Advances in Neural Information Processing Systems_ , pages 6294–6305.
  * Mitrou et al. (2014) Lilian Mitrou, Miltiadis Kandias, Vasilis Stavrou, and Dimitris Gritzalis. 2014\.  Social media profiling: A panopticon or omniopticon tool?  In _Proc. of the 6th Conference of the Surveillance Studies Network_. Barcelona, Spain.
  * Mubarak and Darwish (2014) Hamdy Mubarak and Kareem Darwish. 2014.  Using twitter to collect a multi-dialectal corpus of arabic.  In _Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)_ , pages 1–7.
  * Nguyen et al. (2011) Dong Nguyen, Noah A Smith, and Carolyn P Rosé. 2011.  Author age prediction from text using linear regression.  In _Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities_ , pages 115–123. Association for Computational Linguistics.
  * Palva (2006) Heikki Palva. 2006.  Dialects: classification.  _Encyclopedia of Arabic Language and Linguistics_ , 1:604–613.
  * Pang et al. (2019) Dandan Pang, Johannes C Eichstaedt, Anneke Buffone, Barry Slaff, Willibald Ruch, and Lyle H Ungar. 2019.  The language of character strengths: Predicting morally valued traits on social media.  _Journal of personality_.
  * Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , volume 1, pages 2227–2237.
  * Potthast et al. (2019) Martin Potthast, Paolo Rosso, Efstathios Stamatatos, and Benno Stein. 2019.  A decade of shared tasks in digital text forensics at pan.  In _European Conference on Information Retrieval_ , pages 291–300. Springer.
  * Preoţiuc-Pietro et al. (2017) Daniel Preoţiuc-Pietro, Ye Liu, Daniel Hopkins, and Lyle Ungar. 2017.  Beyond binary labels: political ideology prediction of twitter users.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 729–740.
  * Qwaider et al. (2018) Chatrine Qwaider, Motaz Saad, Stergios Chatzikyriakidis, and Simon Dobnik. 2018\.  Shami: A corpus of levantine arabic dialects.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018)_.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.  _URL https://s3-us-west-2. amazonaws. com/openai-assets/research-covers/languageunsupervised/language understanding paper. pdf_.
  * Rangel et al. (2014) Francisco Rangel, Paolo Rosso, Irina Chugur, Martin Potthast, Martin Trenkmann, Benno Stein, Ben Verhoeven, and Walter Daelemans. 2014.  Overview of the 2nd author profiling task at pan 2014.  In _CLEF 2014 Evaluation Labs and Workshop Working Notes Papers, Sheffield, UK, 2014_ , pages 1–30.
  * Rangel et al. (2013) Francisco Rangel, Paolo Rosso, Moshe Koppel, Efstathios Stamatatos, and Giacomo Inches. 2013.  Overview of the author profiling task at pan 2013.  In _CLEF Conference on Multilingual and Multimodal Information Access Evaluation_ , pages 352–365. CELCT.
  * Rangel et al. (2017) Francisco Rangel, Paolo Rosso, Martin Potthast, and Benno Stein. 2017.  Overview of the 5th author profiling task at pan 2017: Gender and language variety identification in twitter.  _Working Notes Papers of the CLEF_.
  * Rangel et al. (2016) Francisco Rangel, Paolo Rosso, Ben Verhoeven, Walter Daelemans, Martin Potthast, and Benno Stein. 2016.  Overview of the 4th author profiling task at pan 2016: cross-genre evaluations.  In _Working Notes Papers of the CLEF 2016 Evaluation Labs. CEUR Workshop Proceedings/Balog, Krisztian [edit.]; et al._ , pages 750–784.
  * Rangel Pardo et al. (2015) Francisco Manuel Rangel Pardo, Fabio Celli, Paolo Rosso, Martin Potthast, Benno Stein, and Walter Daelemans. 2015.  Overview of the 3rd author profiling task at pan 2015.  In _CLEF 2015 Evaluation Labs and Workshop Working Notes Papers_ , pages 1–8.
  * Rao et al. (2010) Delip Rao, David Yarowsky, Abhishek Shreevats, and Manaswi Gupta. 2010.  Classifying latent user attributes in twitter.  In _Proceedings of the 2nd international workshop on Search and mining user-generated contents_ , pages 37–44. ACM.
  * Sadat et al. (2014) Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. 2014.  Automatic identification of arabic language varieties and dialects in social media.  _Proceedings of SocialNLP_ , page 22.
  * Salameh and Bouamor (2018) Mohammad Salameh and Houda Bouamor. 2018.  Fine-grained arabic dialect identification.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1332–1344.
  * Sap et al. (2014) Maarten Sap, Gregory Park, Johannes Eichstaedt, Margaret Kern, David Stillwell, Michal Kosinski, Lyle Ungar, and Hansen Andrew Schwartz. 2014.  Developing age and gender predictive lexica over social media.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1146–1151.
  * Schwartz et al. (2013) H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Stephanie M Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin EP Seligman, et al. 2013.  Personality, gender, and age in the language of social media: The open-vocabulary approach.  _PloS one_ , 8(9):e73791.
  * Søgaard and Goldberg (2016) Anders Søgaard and Yoav Goldberg. 2016.  Deep multi-task learning with low level tasks supervised at lower layers.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , volume 2, pages 231–235.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014.  Dropout: a simple way to prevent neural networks from overfitting.  _The Journal of Machine Learning Research_ , 15(1):1929–1958.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.  Sequence to sequence learning with neural networks.  In _NIPS_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , pages 6000–6010.
  * Verhoeven (2018) Ben Verhoeven. 2018.  _Two authors walk into a bar: studies in author profiling_.  Ph.D. thesis, University of Antwerp.
  * Verhoeven et al. (2016) Ben Verhoeven, Walter Daelemans, and Barbara Plank. 2016.  Twisty: a multilingual twitter stylometry corpus for gender and personality profiling.  In _Proceedings of the 10th Annual Conference on Language Resources and Evaluation (LREC 2016)/Calzolari, Nicoletta [edit.]; et al._ , pages 1–6.
  * Versteegh (2014) Kees Versteegh. 2014.  _The arabic language_.  Edinburgh University Press.
  * Vijayaraghavan et al. (2017) Prashanth Vijayaraghavan, Soroush Vosoughi, and Deb Roy. 2017.  Twitter demographic classification using deep multi-modal multi-task learning.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 478–483.
  * Wright (2013) David Wright. 2013.  Using corpora in forensic authorship analysis: Investigating idiolect in enron emails.  _Corpus Linguistics 2013_ , page 296.
  * Zaghouani and Charfi (2018) Wajdi Zaghouani and Anis Charfi. 2018.  Arap-tweet: A large multi-dialect twitter corpus for gender, age and language variety identification.  _arXiv preprint arXiv:1808.07674_.

##  Appendix A Summary of Supplementary Material

We provide the following supplementary items:

  1. 1.

Table 5 shows top 20 words attended to by our MTL-spec-attn model, along with
attention weights by other (MTL) models and MI.

  2. 2.

Table 6 shows top 20 words based on MI scores, along with attention weights by
other models.

  3. 3.

Figures 5 and 6 provides examples of attention-based visualization of our MTL-
spec-attn model on the gender task.

  4. 4.

Table 7 provides information about the epoch acquiring best performance across
all our models on DEV.

Figure 5: Attention-based visualization based on our MTL-spec-attn of hand-
picked, correctly predicted samples from our TRAIN data.

Figure 6: Attention-based visualization based on our MTL-spec-attn of hand-picked, incorrectly predicted samples from our TRAIN data. Word | Eng | MI | Attn-BiGRU | MTL-C | MTL-S
---|---|---|---|---|---
¡يسالونني¿ | they ask me | 1.00E-06 | 0.038055 | 0.119423 | 0.169779
¡ياغالين¿ | you dear (+plural) | 9.40E-05 | 0.052512 | 0.078695 | 0.164816
¡منيحاا¿ | beautiful (+fem) | 5.00E-06 | 0.031567 | 0.117903 | 0.164619
¡مسساء¿ | evening | 4.00E-06 | 0.027612 | 0.142467 | 0.161141
¡سيصادفك¿ | you’ll meet by chance | 0.00E+00 | 0.040002 | 0.103273 | 0.153956
¡بونسوار¿ | good evening (French) | 3.00E-05 | 0.044578 | 0.10257 | 0.1479
¡سيصيبك¿ | it’ll befall you | 2.00E-06 | 0.031588 | 0.097372 | 0.147765
¡شلوونكم¿ | how are you (+plural) | 5.00E-06 | 0.0352 | 0.125699 | 0.146776
¡امبيه¿ | I want it | 1.50E-05 | 0.041253 | 0.095131 | 0.14668
¡ياعيون¿ | you, darling | 0.00E+00 | 0.032135 | 0.123004 | 0.142256
¡ياصدفه¿ | what a surprise | 1.00E-06 | 0.025398 | 0.124698 | 0.142011
¡انزيين¿ | we beautify | 1.00E-06 | 0.025684 | 0.096841 | 0.141639
¡صبحهم¿ | good morning | 0.00E+00 | 0.03443 | 0.118689 | 0.139647
¡ياسااتر¿ | God protect | 0.00E+00 | 0.036266 | 0.104663 | 0.138652
¡سابتسم¿ | I’ll smile | 1.00E-06 | 0.029951 | 0.097428 | 0.138609
¡تقييمكم¿ | what’s your evaluation | 2.00E-06 | 0.034339 | 0.091574 | 0.137949
¡مسلخير¿ | good evening | 7.00E-06 | 0.036409 | 0.114726 | 0.137911
¡االخيرر¿ | good (morning/evening) | 0.000141 | 0.058663 | 0.094799 | 0.137106
¡ليلهه¿ | night | 9.00E-06 | 0.026944 | 0.104899 | 0.136817
¡نبئ¿ | we want | 0.00E+00 | 0.036886 | 0.098703 | 0.136785
Table 5: Top 20 most highly weighted words based on average attention weights from our MTL-spec-attn (MTL-S) network for gender. We also add corresponding average attention weights allocated to the words by our single-task attention BiGRU model (Attn-BiGRU) and MTL-common-attn (MTL-C) model. In addition, we add the mutual information (MI) weights for these words. Note the MI values are very small.  Word | Eng | MI | Attn-BiGRU | MTL-C | MTL-S
---|---|---|---|---|---
¡حبيبتي¿ | my darling (+fem.) | 0.002155 | 0.050888 | 0.077016 | 0.094471
¡في¿ | in | 0.001078 | 0.033693 | 0.047777 | 0.047659
¡يارب¿ | Oh, Lord | 0.001053 | 0.035691 | 0.069558 | 0.0758
¡عارفه¿ | I know (+fem.) | 0.000664 | 0.043761 | 0.06062 | 0.068043
¡انا¿ | I | 0.000559 | 0.034185 | 0.066936 | 0.072569
¡قلبي¿ | my heart | 0.00047 | 0.03583 | 0.058327 | 0.060456
¡الدوري¿ | soccer season | 0.000457 | 0.039459 | 0.049224 | 0.051559
¡القصاص¿ | retributive justice | 0.000444 | 0.052651 | 0.046728 | 0.068582
¡مباراه¿ | game | 0.000434 | 0.037312 | 0.057625 | 0.060602
¡بن¿ | son of | 0.000431 | 0.034937 | 0.04922 | 0.048669
¡لاعب¿ | player | 0.000407 | 0.038375 | 0.055207 | 0.055142
¡انتي¿ | you (+fem.) | 0.000399 | 0.034844 | 0.065708 | 0.068472
¡رئيس¿ | president | 0.00039 | 0.034811 | 0.057263 | 0.059484
¡علي¿ | Ali/on | 0.000382 | 0.033676 | 0.049267 | 0.04817
¡يالغالي¿ | dear (+you) | 0.000358 | 0.043446 | 0.069519 | 0.072201
¡ميسي¿ | Messi | 0.000325 | 0.038833 | 0.061201 | 0.067188
¡الهلال¿ | Hilal (soccer team) | 0.000319 | 0.037964 | 0.053231 | 0.057722
¡يابو¿ | father of (+you) | 0.000302 | 0.040432 | 0.064222 | 0.070818
¡برشلونه¿ | Barcelona | 0.000299 | 0.03753 | 0.055462 | 0.060405
¡كاس¿ | cup (soccer) | 0.000279 | 0.036868 | 0.04998 | 0.048994
Table 6: Top 20 highest weighted words based on mutual information (MI) for gender. We also add corresponding average attention weights allocated to the words by our single-task attention BiGRU model (Attn-BiGRU), MTL-common-attn (MTL-C), and MTL-spec-attn (MTL-S) networks. Setting | Age | Epoch | Gender | Epoch
---|---|---|---|---
small-GRU | 36.13 | 1 | 53.39 | 1
Attn-BiGRU | 47.03 | 3 | 61.64 | 2
MTL-common-attn | 47.85 | 6 | 62.50 | 4
MTL-spec-attn | 47.92 | 4 | 63.09 | 4
MTL-sprvsd-spec-attn | 48.23 | 5 | 62.99 | 3
MTL-MI-spec-attn | 47.90 | 3 | 63.15 | 5
BERT | 50.95 | 4 | 64.96 | 6
Table 7: Epoch information for best model performance on our DEV.


Holographic Dual of Crosscap Conformal Field Theory

Zixia Wei

Jefferson Physical Laboratory, Harvard University, Cambridge, MA 02138, USA

We propose a holographic dual for 2D CFT defined on closed non-orientable
manifolds, such as the real projective plane $\mathbb{RP}^{2}$ and the Klein
bottle $\mathbb{K}^{2}$. Such CFT can be constructed by introducing
antipodally identified cuttings, i.e. crosscaps, to a sphere and hence called
crosscap CFT (XCFT). The gravity dual is AdS3 spacetime with dS2 end-of-the-
world branes. In particular, the Lorentzian spacetime with a global dS2 brane
is dual to the unitary time evolution of a crosscap state in CFT, post-
selected on the CFT ground state. We compute the holographic $\mathbb{RP}^{2}$
partition function (or the $p$-function), one-point function, and
$\mathbb{K}^{2}$ partition function, and see that they successfully reproduce
the XCFT results. We also show a holographic $p$-theorem as an application.

###### Contents

  1. 1 Introduction
  2. 2 General Construction and Gravity Dual of $\mathbb{RP}^{2}$ XCFT
  3. 3 Partition Function, Crosscap Entropy and One-point Function
  4. 4 Lorentzian Signature from Wick Rotation
  5. 5 Holographic XCFT on $\mathbb{K}^{2}$
  6. 6 Holographic $p$-theorem
  7. 7 Conclusions and Discussions

##  Introduction

As the most well-studied example of the holographic principle [1, 2], the
AdS/CFT correspondence has played a central role in our understanding of AdS
quantum gravity in a nonperturbative way [3]. In this paper, we construct a
minimal model for describing the AdS3 dual of 2D CFT defined on non-orientable
manifolds and find that a structure of dS2 vacuum naturally appears in this
construction. Being a standard holographic CFT, the boundary theory is unitary
and local and has a standard Hilbert space and a canonical time evolution.

The most simple 2D non-orientable manifold is the real projective plane
$\mathbb{RP}^{2}$, which is defined by identifying the antipodal points on a
sphere $S^{2}$. Topologically, this is equivalent to starting from a
hemisphere or a disk and then identifying the antipodal points on the boundary
as sketched in Fig.1. The structure obtained by identifying the antipodal
points on the boundary is called a crosscap. Since any 2D non-orientable
manifold can be constructed by introducing crosscaps to an orientable
manifold, we may call the CFT defined on such manifolds the crosscap CFT
(XCFT). Historically, XCFT has been extensively studied in the context of
string world sheet theory with orientifolds [4, 5, 6]. We will be mainly
focusing on holographic XCFT defined on $\mathbb{RP}^{2}$ and the Klein bottle
$\mathbb{K}^{2}$, which are topologically equivalent to inserting one and two
crosscaps into a sphere, respectively.

In our construction of holographic duals of XCFT2, a dynamical end-of-the-
world brane which is locally dS2 is introduced and plays the most central role
throughout the results presented in this paper. This makes a major distinction
from previous discussions on the holographic dual of crosscaps and non-
orientable CFTs including [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17].

In the following, we start by presenting the holographic dual of XCFT defined
on $\mathbb{RP}^{2}$ and writing down the action of our minimal construction.
We will compute the holographic partition function and 1-point functions and
see their matching with standard $\mathbb{RP}^{2}$ XCFT results. We then move
on to the Lorentzian counterpart and explain how a dS2 vacuum is described by
a conformal crosscap state under unitary time evolution and post-selected on
the CFT ground state. We will also point out an intriguing relation between
the conformal crosscap state and the Type II1 von Neumann algebra. As a
slightly more complicated example, we will analyze the holographic dual of
$\mathbb{K}^{2}$ and show that the holographic partition function matches
known universal results in the XCFT and shows a first order phase transition
analogous to the Hawking-Page transition. We will end up with an application
of our AdS/XCFT correspondence by showing a holographic $p$-theorem, which
states that there exists a holographic $p$-function which monotonically
decreases under the holographic renormalization group (RG) flow.

Figure 1: The real projective plane $\mathbb{RP}^{2}$ is defined by imposing
an antipodal identification to a sphere $S^{2}$. This is equivalent to
identifying the antipodal points on the boundary of a hemisphere, i.e.
inserting a crosscap denoted as $\otimes$. The state introduced by the
crosscap is denoted as $\ket{C}$.

##  General Construction and Gravity Dual of $\mathbb{RP}^{2}$ XCFT

Let us start by considering a holographic CFT on a unit $S^{2}$ parameterized
by $\theta\in[-\frac{\pi}{2},\frac{\pi}{2}]$ and $\phi\in[0,2\pi)$ with
$\phi\sim\phi+2\pi$. The gravity dual is given by the global AdS3 with metric

$\displaystyle
ds^{2}=d\eta^{2}+{\mathrm{sinh}}^{2}\eta\left(d\theta^{2}+\cos^{2}\theta
d\phi^{2}\right).$ (1)

For simplicity, we set the AdS radius to $1$ throughout this paper. Let us
then impose the antipodal identification $(\theta,\phi)\sim(-\theta,\phi+\pi)$
to squash the boundary manifold $S^{2}$ into an $\mathbb{RP}^{2}$. If we were
to directly extend this identification into the bulk, then a fixed point would
appear at $\eta=0$ and turn out to be a singularity. Although this bulk
geometry can recover certain features of XCFT on $\mathbb{RP}^{2}$ [10, 12,
14], it is globally not a solution of the bulk Einstein equation and hence
incompatible with the standard dictionary of AdS/CFT [18, 19, 20, 21].

In order to resolve this singularity, we propose a minimal model by
introducing an end-of-the-world brane $Q$ in the bulk $\mathcal{M}$. For a
holographic CFT defined on a closed manifold $\Sigma$, we consider the dual
bulk (Euclidean) action given by

$\displaystyle I=-\frac{1}{16\pi G_{N}}\int_{{\cal
M}}\sqrt{g}\left(R-2\Lambda\right)-\frac{1}{8\pi
G_{N}}\int_{Q}\sqrt{h}(K-T)-\frac{1}{8\pi G_{N}}\int_{\Sigma}\sqrt{\gamma}B,$
(2)

where the three terms are the Einstein-Hilbert term in $\mathcal{M}$, the
Gibbons-Hawking term on the brane $Q$, and the Gibbons-Hawking term on the
asymptotic boundary $\Sigma$, respectively. $G_{N}$, $g_{\mu\nu}$, $R$ and
$\Lambda$ are the Newton constant, the metric, the scalar curvature and the
cosmological constant in $\mathcal{M}$. $h_{ab}$, $K_{ab}$, $T$ are the
induced metric, the extrinsic curvature and the tension of $Q$. $\gamma_{ij}$
and $B_{ij}$ are the induced metric and the extrinsic curvature on the
asymptotic boundary $\Sigma$. Note that we have defined the extrinsic
curvatures of $\Sigma$ and $Q$ such that the normal vectors are pointing
outwards from the bulk $\mathcal{M}$. While imposing the standard Dirichlet
boundary condition on $\Sigma$, we choose the Newmann boundary condition

$\displaystyle K_{ab}-Kh_{ab}=-Th_{ab},$ (3)

on the brane $Q$ to make it dynamical. In order to get a sensible solution, we
restrict the tension to $T<-1$.

Solving the Einstein equation in $\mathcal{M}$ with the asymptotic boundary
given by $\mathbb{RP}^{2}$, we get a $\mathbb{Z}_{2}$ quotient of global AdS3
obtained by $(\theta,\phi)\sim(-\theta,\phi+\pi)$, with an end-of-the-world
brane $Q$ located at

$\displaystyle\eta={\rm arccoth}(-T)\equiv\eta_{*}.$ (4)

The geometry of $Q$ is a $\mathbb{Z}_{2}$ quotient of $S^{2}$ (or Euclidean
dS2). The bulk $\mathcal{M}$ is defined at $\eta\geq\eta_{*}$ and the
$\eta<\eta_{*}$ region is excluded by introducing the brane $Q$. Therefore,
there is no singularity in the bulk any more. See Fig.2 for a sketch.

Figure 2: (Left) The holographic dual of XCFT on $\mathbb{RP}^{2}$ is given by
the $\eta\geq\eta_{*}$ region of the $\mathbb{Z}_{2}$ quotient
$(\theta,\phi)\sim(-\theta,\phi+\pi)$ of AdS3 with a Euclidean
dS${}_{2}/\mathbb{Z}_{2}$ end-of-the-world brane at $\eta=\eta_{*}$. If $Q$
were not introduced, there would be a problematic singularity at $\eta=0$.
(Right) The $(\rho,\tau,\phi)$-coordinate is useful for considering
holographic XCFT defined on a half-cylinder with a crosscap.

Let us check how the symmetries match between the XCFT side and the gravity
side in terms of the global conformal group. On the XCFT side, realizing the
$\mathbb{RP}^{2}$ by identifying the antipodal points on the Riemann sphere
$S^{2}$, it is easy to see that the isometry group of $\mathbb{RP}^{2}$ is
$PO(3)=PSO(3)\cong SO(3)$. This $SO(3)$ is inherited from the $SO(3)$ isometry
of $S^{2}$, which is a subgroup of its conformal group $SO(3,1)$. On the other
hand, the other half of the conformal symmetries are broken, since they do not
map great circles to great circles on $S^{2}$ and hence break antipodal
relations. This $SO(3)$ symmetry manifests in the Euclidean gravity dual. The
existence of the dS end-of-the-world brane, which essentially introduce a
center in the bulk spacetime, breaks the three translational symmetries but
preserve three rotational symmetries, which form the $SO(3)$.

It is useful to introduce another coordinate system by performing

$\displaystyle\cos\theta=\frac{{\mathrm{sinh}}\rho}{{\mathrm{sinh}}\eta},~{}~{}\sin\theta=\frac{{\mathrm{tanh}}\tau}{{\mathrm{tanh}}\eta}.$
(5)

In the new coordinate system, the metric is

$\displaystyle
ds^{2}=d\rho^{2}+{\mathrm{cosh}}^{2}\rho~{}d\tau^{2}+{\mathrm{sinh}}^{2}\rho~{}d\phi^{2},$
(6)

the identification is $(\tau,\phi)\sim(-\tau,\phi+\pi)$, and the brane profile
is

$\displaystyle
T^{2}=\frac{{\mathrm{cosh}}^{2}\rho}{{\mathrm{sinh}}^{2}\rho+{\mathrm{tanh}}^{2}\tau},$
(7)

as shown in Fig.2. If we introduce a cutoff at $\rho=\rho_{\infty}$, then the
corresponding XCFT is defined on a half-infinite cylinder with a crosscap
inserted at $\tau=0$.

Before proceeding, we would like to note that there are two possible ways to
interpret the brane term introduced in the action Eq.(2). One is to introduce
the brane only when there are crosscaps on the CFT side. This way of thinking
is similar to that in the AdS/BCFT constructions [22, 23], where AdS end-of-
the-world branes are introduced only when there are boundaries on the CFT
side. Another way of thinking is to include the brane term regardless of
whether there are crosscaps on the CFT side or not, and allow all the
solutions in the gravitational path integral including brane nucleations. Both
ways of thinking give the same results as long as we only look at the most
dominating bulk configurations. However, it would be an interesting future
question to discuss their differences when subdominating contributions are
included. Especially, the second way of thinking may mediate a Coleman-de
Luccia instability [24] in the bulk.

##  Partition Function, Crosscap Entropy and One-point Function

The partition function of an XCFT defined on a unit $\mathbb{RP}^{2}$ can be
written as the inner product between the CFT ground state $\ket{0}$ and a
crosscap state $\ket{C}$

$\displaystyle Z_{\mathbb{RP}^{2}}=\braket{0}{C},$ (8)

and is sometimes called the $p$-function.111Here we adopt the terminology
introduced in [16] where $p$ stands for “parity”.  One natural way to see this
is to consider the fundamental domain $\theta\in[-\frac{\pi}{2},0]$ of the
$\mathbb{RP}^{2}$. Then the path integral over $\theta\in[-\frac{\pi}{2},0)$
gives the CFT ground state $\ket{0}$ and the crosscap state $\ket{C}$ is the
“boundary” condition imposed on $\theta=0$. (Refer to Fig.1.) In order for the
energy-momentum tensor to match between identified antipodal points, $\ket{C}$
must satisfy the following crosscap condition [25],

$\displaystyle\left(L_{n}-(-1)^{n}\bar{L}_{-n}\right)\ket{C}=0,\qquad
n\in\mathbb{Z},$ (9)

where $L_{n}$’s ($\bar{L}_{n}$’s) are the chiral (anti-chiral) Virasoro
generators. Note that the crosscap condition Eq.(9) preserves exactly a half
of the Virasoro symmetry. Besides, the crosscap entropy222The crosscap entropy
is also called the Klein bottle entropy in some literature [26] since it
appears in a limit of the Klein bottle partition function as we will also see
later. Here we adopt the more general name crosscap entropy. defined as

$\displaystyle S_{\rm xc}=\log|\langle 0|C\rangle|,$ (10)

is a quantity closely related to the $p$-function. The $p$-function and the
crosscap entropy in XCFT are analogous to the $g$-function and the boundary
entropy in BCFT [27], respectively.

In our AdS/XCFT for $\mathbb{RP}^{2}$, we have

$\displaystyle
R=-6,~{}~{}\Lambda=-1,~{}~{}\sqrt{g}={\mathrm{sinh}}^{2}\eta|\cos\theta|,$
(11) $\displaystyle
K=-2\frac{{\mathrm{cosh}}\eta_{*}}{{\mathrm{sinh}}\eta_{*}},~{}~{}T=-\frac{{\mathrm{cosh}}\eta_{*}}{{\mathrm{sinh}}\eta_{*}},~{}~{}\sqrt{h}={\mathrm{sinh}}^{2}\eta_{*}|\cos\theta|.$
(12)

Plugging these into the action Eq.(2), integrating over the bulk dual and
subtracting divergent terms, we obtain the holographic $p$-function

$\displaystyle p\equiv
Z_{\mathbb{RP}^{2}}=\exp\left(-I^{\rm(ren)}\right)=\exp\left(-\frac{\eta_{*}}{4G_{N}}\right)=\exp\left(-\frac{c}{6}{\rm
arccoth}(-T)\right),$ (13)

where $I^{\rm(ren)}$ is the renormalized bulk action and $c$ is the central
charge of the XCFT. Note that we have used Eq.(4) and the Brown-Henneaux
relation $c=\frac{3}{2G_{N}}$ [28] in the last line. Accordingly, the crosscap
entropy is

$\displaystyle S_{\rm xc}=\log|p|=-\frac{\eta_{*}}{4G_{N}}=-\frac{{\rm
arccoth}(-T)}{4G_{N}}=-\frac{c}{6}{\rm arccoth}(-T).$ (14)

Given the similarity between the $p$-function in XCFT and the $g$-function in
BCFT, we may expect that there is a “$p$-theorem” stating that the
$p$-function monotonically decreases under some RG flow. While how to
formulate such a “$p$-theorem” is not yet known in XCFT, we will later come
back to the holographic $p$-function and show a holographic $p$-theorem.

It is also straightforward to check that holographic 1-point functions for
scalar primaries match the XCFT result. Consider an XCFT defined on
$\mathbb{RP}^{2}$ with radius $r$ and insert a scalar primary $\mathcal{O}$
with scaling dimension $\Delta$ at $(\theta,\phi)$. The 1-point function is
determined by the symmetries except for the coefficient, and has the form

$\displaystyle\langle\mathcal{O}(\theta,\phi)\rangle_{\mathbb{RP}^{2}}=\frac{a_{\mathcal{O}}}{(2r)^{\Delta}}.$
(15)

On the gravity side, the most simple regime is given by $\Delta$ that is
$O(c^{0})$ but at the same time large $\Delta\gg 1$. In this regime, the
primary operator corresponds to a free scalar field with mass
$m=\sqrt{\Delta(\Delta-2)}$. With the presence of the brane $Q$, the
holographic 1-point function can be approximated by $\lambda e^{-\Delta L}$,
where $L$ is the distance between the boundary operator insertion and the
brane $Q$ (with proper renormalization performed) and $\lambda$ is a constant
depending on the details of the boundary condition imposed on the brane $Q$
for the scalar field [23, 29]. Since we are considering a holographic CFT
defined on $\mathbb{RP}^{2}$ with radius $r$, we can regard our XCFT defined
at $\eta=\log(2r/\epsilon)$ where $\epsilon$ is a UV cutoff corresponding to
the lattice distance. The distance between $(\log(2r/\epsilon),\theta,\phi)$
and the brane $Q$, after subtracting the divergent term is $L=\log
2r-\eta_{*}$. Accordingly, the holographic 1-point function is given by
$\lambda e^{\eta_{*}\Delta}/(2r)^{\Delta}$, which has the same form as the
XCFT result Eq.(15). It is straightforward (though a little bit more
complicated) to check the holographic 1-point functions for scalar primaries
with $\Delta\sim 1$ and $\Delta=O(c)$ also have the form Eq.(15), following
the same strategy as in [23, 29, 30].

##  Lorentzian Signature from Wick Rotation

Let us then consider the Wick rotation to the Lorentzian signature in the
setups discussed above. First, if we take $\theta\rightarrow i\theta_{L}$ in
Eq.(1), the metric turns into

$\displaystyle
ds^{2}=d\eta^{2}+{\mathrm{sinh}}^{2}\eta\left(-d\theta_{L}^{2}+{\mathrm{cosh}}^{2}\theta_{L}~{}d\phi^{2}\right),$
(16)

with identification $(\theta_{L},\phi)\sim(-\theta_{L},\phi+\pi)$. The brane
$Q$ is still at $\eta=\eta_{*}$. The bulk geometry is manifestly a warped
product between dS${}_{2}/\mathbb{Z}_{2}$ and a half infinite line
$\eta\geq\eta_{*}$. The boundary geometry is also dS${}_{2}/\mathbb{Z}_{2}$.
See Fig.3 for a sketch.

To track the boundary time evolution in the CFT Hilbert space, it is more
convenient to perform $\tau\rightarrow it$ in Eq.(6) and Eq.(7). The metric
turns into

$\displaystyle
ds^{2}=d\rho^{2}-{\mathrm{cosh}}^{2}\rho~{}dt^{2}+{\mathrm{sinh}}^{2}\rho~{}d\phi^{2},$
(17)

with identification $(t,\phi)\sim(-t,\phi+\pi)$, and the brane profile is

$\displaystyle
T^{2}=\frac{{\mathrm{cosh}}^{2}\rho}{{\mathrm{sinh}}^{2}\rho-\tan^{2}t}.$ (18)

Note that the $(\eta,\theta_{L},\phi)$ coordinate only covers the
$t\in(-\pi/2,\pi/2)$ region while the $(\rho,t,\phi)$ coordinate can be
extended to $t\in(-\infty,\infty)$ as shown in Fig.3. To satisfy
$(t,\phi)\sim(2n\pi-t,\phi+\pi)$ for any $n\in\mathbb{Z}$, it is necessary and
sufficient to impose $t\sim t+2\pi$. As a result, the maximally extended
geometry can be regarded as a $\mathbb{Z}_{2}$ quotient of the global AdS3
(but not its universal cover) with dS2 end-of-the-world branes.

Considering the symmetries, let us focus on the $t\in[0,\pi/2]$ region in the
following. It is straightforward to see that at $t=\pi/2$, the dS brane
approaches the asymptotic boundary $\rho\rightarrow\infty$, as sketched in
Fig.3.

Figure 3: The analytic continuations to the Lorentzian signature of the
configurations in Fig.2. The end-of-the-world brane $Q$ hits the asymptotic
boundary at $t=\pi/2$, reflecting the features of the boundary state
$\ket{B}=e^{-i\pi H/2}\ket{C}$.

This behavior can be understood as a consequence of the unitary time evolution
in the dual CFT Hilbert space defined on a circle with length $2\pi$ as
follows. At $t=0$, the CFT state is given by the transition matrix

$\displaystyle\mathcal{T}_{t=0}\propto|C\rangle\langle 0|,$ (19)

prepared by the Euclidean path integral over $\mathbb{RP}^{2}$. Since
$\ket{C}$ satisfies the crosscap condition Eq.(9), the transition matrix
$\mathcal{T}_{t=0}$ is invariant under the following three global symmetry
generators: $L_{0}-\bar{L}_{0},L_{1}+\bar{L}_{-1},L_{-1}+\bar{L}_{1}$ which
satisfies

$\displaystyle[L_{0}-\bar{L}_{0},L_{1}+\bar{L}_{-1}]=-(L_{1}+\bar{L}_{-1}),$
$\displaystyle[L_{0}-\bar{L}_{0},L_{-1}+\bar{L}_{1}]=(L_{-1}+\bar{L}_{1}),$
(20)
$\displaystyle[L_{1}+\bar{L}_{-1},L_{-1}+\bar{L}_{1}]=2(L_{0}+\bar{L}_{0}),$

and hence form an $SL(2,\mathbb{R})\cong SO(2,1)$. On the gravity side, the
domain of dependence of the $t=0$ time slice has two boost symmetries and one
rotation symmetry, which generate an $SO(2,1)$. On the other hand, the three
translations (two spacelike and one timelike) are broken. This is also
precisely the isometry group of dS2.

On the other hand, at $t=\pi/2$, the CFT state is given by

$\displaystyle\mathcal{T}_{t=\frac{\pi}{2}}\propto e^{-i\pi
H/2}|C\rangle\langle 0|e^{i\pi H/2}\propto|B\rangle\langle 0|,$ (21)

where $H=L_{0}+\bar{L}_{0}-\frac{c}{12}$ is the CFT Hamiltonian, and
$\ket{B}\equiv e^{-i\pi H/2}\ket{C}$ satisfies

$\displaystyle\left(L_{n}-\bar{L}_{-n}\right)\ket{B}=0,\qquad n\in\mathbb{Z}.$
(22)

This can be shown by noticing that

$\displaystyle L_{n}H^{k}=(n+H)^{k}L_{n},$ (23)

and hence

$\displaystyle L_{n}e^{itH}=e^{it(n+H)}L_{n}.$ (24)

Accordingly,

$\displaystyle(L_{n}-\bar{L}_{-n})e^{-i\left(\frac{1}{2}+k\right)\pi
H}\ket{C}=0,$ (25) $\displaystyle(L_{n}-(-1)^{n}\bar{L}_{-n})e^{-ik\pi
H}\ket{C}=0.$ (26)

Eq.(22) is nothing but the condition satisfied by conformal boundary states
[25, 31], which gurantees there is no energy flux across the boundary. Since a
conformal boundary state $|B\rangle$ does not contain real space entanglement
[32], the transition matrix
$\mathcal{T}_{t=\frac{\pi}{2}}\propto|B\rangle\langle 0|$ has vanishing
entanglement pseudo entropy [33] and is expected to be dual to a trivial
spacetime. This expectation is indeed realized in the proposed gravity dual,
where the entire $t=\pi/2$ time slice and its causal diamond are nothing but
void. From the point of view of symmetries, $\ket{B}$ is invariant under the
following three global symmetry generators
$L_{0}-\bar{L}_{0},L_{1}-\bar{L}_{-1},L_{-1}-\bar{L}_{1},$ which again form an
$SL(2,\mathbb{R})\cong SO(2,1)$. This is, however, a different $SO(2,1)$ from
the one appearing in the crosscap state Eq.(4). The void bulk time slice at
$t=\pi/2$ clearly satisfies the two spatial translation symmetries and the
rotation symmetry, which again forms an $SO(2,1)$. On the other hand, it is
not invariant under the time translation and the two boosts. This matching
between the CFT transition matrix $|B\rangle\langle 0|$ and the void spacetime
at $t=\pi/2$ provides another strong and independent consistency check of our
AdS/XCFT construction with dS branes.

As a summary, we have established a correspondence between a bouncing dS2
brane in AdS3/$\mathbb{Z}_{2}$ and the unitary time evolution of the crosscap
state, post-selected on the CFT ground state. This suggests that XCFT may be
used to describe dS gravity. As a hint from an alternative point of view, it
is intriguing to note that the crosscap state $\ket{C}$ is profoundly related
to the Type II1 von Neumann algebra, which is believed to describe the
observables on the static patch of the dS spacetime [34], in the following
way. Consider a chain of $2N$ spins aligned in a circle, and then produce EPR
pairs between antipodal points, one gets

$\displaystyle\ket{\Psi}=\frac{1}{2^{N/2}}\bigotimes_{n=1}^{N}\left(\ket{\uparrow}_{i}\otimes\ket{\uparrow}_{i+N}+\ket{\downarrow}_{i}\otimes\ket{\downarrow}_{i+N}\right).$
(27)

While it is hard to write down the crosscap state $\ket{C}$ explicitly in
general, it is known that $\ket{\Psi}$ serves as a generalization of crosscap
states in a class of integrable spin chains [16], in the sense that it
satisfies the matching between the energy-momentum tensor at the antipodal
points and preserves infinitely many conserved charges.333Another interesting
property of $\ket{\Psi}$ is that, when restricted to subsystem $A$ which
contains the first $N$ spins, $\ket{\Psi}$ is indistinguishable from the
canonical Gibbs state with infinite temperature in any 1D spin system [35]. We
note that, at $N\rightarrow\infty$, $\ket{\Psi}$ can be regarded as the cyclic
separating vector used to construct a Type II1 von Neumann algebra in [36, 37,
38, 39]. The Type II1 von Neumann algebra is supported on exactly a half of
the whole system, which indeed corresponds to a static patch in our setup.

##  Holographic XCFT on $\mathbb{K}^{2}$

Figure 4: The standard definition of a Klein bottle (leftmost) and how it is
related to $\langle C|e^{-\beta H/2}|C\rangle$ (rightmost) by sewing and
gluing.

As a slightly more complicated example, let us consider a holographic XCFT
defined on a Klein bottle $\mathbb{K}^{2}$. A standard definition of the Klein
bottle and how it is equivalent to a cylinder with two crosscaps on the two
ends is skeched in Fig.4. Let us consider the Klein bottle in Fig.4. Its
partition function computes the inner product between two regulated crosscap
states defined on the Hilbert space associated to a circle with length $2\pi$,

$\displaystyle Z_{\mathbb{K}^{2}}=\langle C|e^{-\beta H/2}|C\rangle.$ (28)

It is also convenient to notice that such a Klein bottle can be obtained by
starting from a torus $T^{2}$ parameterized by $\tau\in[-\beta/2,\beta/2)$,
$\phi\in[0,2\pi)$ and then imposing the identification
$(\tau,\phi)\sim(\beta/2-\tau,\phi+\pi)$. Let us restrict to the fundamental
domain $\tau\in[-\beta/4,\beta/4]$.

It is straightforward to find that there exist two bulk solutions. One has the
metric

$\displaystyle
ds^{2}=d\rho^{2}+{\mathrm{cosh}}^{2}\rho~{}d\tau^{2}+{\mathrm{sinh}}^{2}\rho~{}d\phi^{2},$
(29)

and two disconnected pieces of branes located at

$\displaystyle
T^{2}=\frac{{\mathrm{cosh}}^{2}\rho}{{\mathrm{sinh}}^{2}\rho+{\mathrm{tanh}}^{2}\left(\tau\pm\frac{\beta}{4}\right)},$
(30)

respectively, with the identification $\phi\sim\phi+\pi$ at
$\tau=\pm\frac{\beta}{4}$. We call this the brane configuration. The bulk
action $I^{\rm(brane)}$ computed from the brane configuration is

$\displaystyle I^{\rm(brane)}=-\frac{\beta}{16G_{N}}+\frac{{\rm
arccoth}(-T)}{2G_{N}}=-\frac{c\beta}{24}-2\log
Z_{\mathbb{RP}^{2}}=-\frac{c\beta}{24}-2S_{\rm xc},$ (31)

where we have used Eq.(13)(14). The second term is due to the existence of the
two disconnected pieces of the end-of-the-world branes.

Another solution has the metric

$\displaystyle
ds^{2}=d\rho^{2}+{\mathrm{cosh}}^{2}\rho~{}d\phi^{2}+{\mathrm{sinh}}^{2}\rho~{}d\tau^{2},$
(32)

with the identification $\phi\sim\phi+\pi$ at $\tau=\pm\frac{\beta}{4}$ and no
branes. There are no singularities in the bulk. This is the well-known
Euclidean BTZ geon geometry. The bulk action $I^{\rm(geon)}$ computed from the
geon configuration is

$\displaystyle
I^{\rm(geon)}=-\frac{\pi^{2}}{4G_{N}\beta}=-\frac{\pi^{2}c}{6\beta}.$ (33)

Note that $S_{\rm xc}=Z_{\mathbb{RP}^{2}}$ does not appear since there are no
branes in the bulk.

The holographic $\mathbb{K}^{2}$ partition function is given by

$\displaystyle
Z_{\mathbb{K}^{2}}=\exp\left(-\min\\{I^{\rm(brane)},I^{\rm(geon)}\\}\right),$
(34)

and shows a first order phase transition at

$\displaystyle\beta=\sqrt{\left(\frac{24}{c}S_{\rm
xc}\right)^{2}+4\pi^{2}}-\frac{24}{c}S_{\rm xc}\equiv\beta_{*}.$ (35)

The high temperature phase at $\beta<\beta_{*}$ is dominated by the geon
geometry, and the low temperature phase at $\beta>\beta_{*}$ is dominated by
the brane geometry.

Similarly to the Cardy formula for the torus partition function, the Klein
bottle partition function has a universal behavior at the high temperature
limit $\beta\rightarrow 0$ and the low temperature limit
$\beta\rightarrow\infty$ [10, 26, 15]. The holographic partition functions
computed from Eq.(31) and (33) precisely reproduce the universal behavior.
While the $\beta$-dependent part is easy to recover just by taking a
$\mathbb{Z}_{2}$ quotient of the thermal AdS geometry and the BTZ geometry,
the end-of-the-world brane $Q$ plays an essential role in reproducing the
$Z_{\mathbb{RP}^{2}}$-dependent part.

##  Holographic $p$-theorem

So far we have established a correspondence between XCFT2 and AdS3 with dS2
end-of-the-world branes, and checked that holographic computations match
universal results in CFT. Let us then see an application of this AdS/XCFT
correspondence.

In 2D CFT, the central charge $c$ plays a central role and characterizes the
degrees of freedom. It is known as the $c$-theorem that there exists a
function (often called a $c$-function) which monotonically decreases under the
RG flow and coincides the central charge $c$ at the critical points [40].
Similarly, in 2D BCFT, the disk partition function (also called the
$g$-function) characterizes the degrees of freedom of the boundary [27] and
monotonically decreases under the boundary RG flow, known as the $g$-theorem
[41]. Given the similarity between the XCFT and the BCFT, it is natural to
conjecture that there exists a $p$-function which monotonically decreases
under some RG flow and coincides Eq.(8) at critical points. However, there is
no proof of this “$p$-conjecture” so far. In fact, different from the
boundary, since the crosscap is not a local concept, it is not even clear
where the RG flow should be implemented. Here, we will show that, in AdS/XCFT,
there is a natural choice of the holographic RG flow. We prove a holographic
$p$-theorem, following a similar strategy to the proof of the holographic
$g$-theorem [22, 23]. We expect that this holographic $p$-theorem should
provide valuable hints to the $p$-conjecture on the XCFT side.

Let us start by considering a holographic XCFT on $\mathbb{RP}^{2}$. Instead
of its Euclidean gravity dual shown in Fig.2, let us consider its Lorentzian
continuation shown in Fig.3. It turns out to be useful to consider a Poincaré
coordinate $(t_{P},x_{P},z_{P})$ whose $t_{P}=0$ time slice is identified with
the global $t=\frac{\pi}{2}$ time slice. Performing the coordinate
transformation

$\displaystyle\sqrt{1+{\mathrm{sinh}}^{2}\rho}~{}\cos\left(t-\frac{\pi}{2}\right)=\frac{z_{P}}{2}\left(1+\frac{1-t_{P}^{2}+x_{P}^{2}}{z_{P}^{2}}\right),$
(36)
$\displaystyle\sqrt{1+{\mathrm{sinh}}^{2}\rho}~{}\sin\left(t-\frac{\pi}{2}\right)=\frac{t_{P}}{z_{P}},$
(37)

we get the standard Poincaré metric

$\displaystyle
ds^{2}=\frac{(dz_{P})^{2}-(dt_{P})^{2}+(dx_{P})^{2}}{(z_{P})^{2}}.$ (38)

Let us again focus on the $0<t\leq\pi/2$ region, which is partially covered by
$t_{P}\leq 0$. The brane profile in the Poincaré coordinate is given by

$\displaystyle t_{P}=-\sqrt{\frac{T^{2}}{T^{2}-1}}z_{P}.$ (39)

Let us consider the holographic RG flow on the $z_{P}$ direction. To probe
this RG flow without changing the bulk gravity (and hence without changing the
central charge $c$), let us put a brane-localized matter field on the brane
$Q$ and conisder the case where the matter configuration is translational
invariant on the $x_{P}$ direction. In order for the brane-localized matter
field to be senseible, we impose the null energy condition (NEC) on it:

$\displaystyle T_{ab}^{Q}u^{a}u^{b}\geq 0,$ (40)

where $T_{ab}^{Q}$ is the energy-stress tensor on the brane and $u^{a}$ can be
any null vector. Due to the back reaction, the brane now has a new profile and
let us denote it as

$\displaystyle t_{P}=f(z_{P}).$ (41)

We propose a $p$-function as

$\displaystyle p(z_{P})=\exp\left(-\frac{{\rm
arccoth}\left(-\frac{f^{\prime}(z_{P})}{\sqrt{\left(f^{\prime}(z_{P})\right)^{2}-1}}\right)}{4G_{N}}\right).$
(42)

First of all, this matches Eq.(13) when there is no brane-localized matter and
the AdS3 gravity describes a holographic XCFT. Secondly, if we choose
$(u^{t},u^{x},u^{z})=(f^{\prime},\sqrt{(f^{\prime})^{2}-1},1)$, then one can
derive that $f^{\prime\prime}(z_{P})\leq 0$ from Eq.(40), and this further
leads to $p^{\prime}(z_{P})\leq 0$. Another quicker but less straightforward
way to see this is by noticing that when the brane-localized matter satisfies
NEC, two points on the brane that are connected by a casual curve on the brane
should be always connected by a causal geodesic in the bulk [42, 43]. Then
$f^{\prime\prime}(z_{P})\leq 0$ must be satisfied otherwise one would find
counterexamples to the statement above.

As a summary, we have constructed $p(z_{P})$ which monotonically decreases
under the holographic RG flow on the $z_{P}$ direction. Based on the arguments
above, we have derived a holographic $p$-theorem.

##  Conclusions and Discussions

We have presented a minimal construction of gravity duals for 2D XCFTs, which
is AdS3 with dS2 end-of-the-world branes.444The bulk geometries we constructed
inherit the non-orientability from the boundary. It has been recently argued
that non-orientable bulk saddle points should be taken into account even when
the boundary CFT is orientable [44, 45, 46]. The construction is free from
singularities thanks to the branes. We first investigated 2D XCFT on
$\mathbb{RP}^{2}$, computed the partition function (or equivalently
$p$-function or crosscap entropy) and 1-point functions, saw the relation
between the boundary state and the crosscap state
$\ket{B}=e^{-i\frac{\pi}{2}H}\ket{C}$ naturally appears from the Wick rotation
to the Lorentzian signature, and investigated 2D XCFT on $\mathbb{K}^{2}$. We
have also proven a holographic $p$-theorem as an application of our AdS/XCFT
construction. From an alternative point of view, the searching of CFT dual to
AdS3 with negative tension dS2 branes was initiated in [47] and further
explored in [48, 49, 50]. It was proposed in [47, 48, 49, 50] that the the
dual should be a Lorentzian CFT with spacelike boundaries which is however not
well-defined due to the non-normalizability of the boundary state. We conclude
that a natural dual is a 2D XCFT, where a $\mathbb{Z}_{2}$ quotient is taken
compared to the setup in [47, 48, 49, 50], and it is indeed related to the
boundary state via $\ket{B}=e^{-i\frac{\pi}{2}H}\ket{C}$.

Another interesting point we have made is that a dS2 structure naturally
appears in the bulk AdS3, which is a warped product between dS2 and a half-
infinite line. Besides, the conformal crosscap state $\ket{C}$ can be
naturally related to a Type II1 von Neumann algebra, which is believed to
describe observables in a static patch of dS [34]. It would be interesting to
explore the possibility of using AdS/XCFT to describe dS holography, in the
light of double holography [51]. We would like to emphasize that the UV-
complete description here is the XCFT, which is local and unitary, and has a
standard Hilbert space and time direction by construction. At least one of
these features is often missed in dS holography proposals [52, 53, 54, 55, 56,
57, 58, 59, 60]. One may worry that, different from the standard double
holography with positive tension AdS branes, there is no locally localized
graviton on our negative tension dS branes via the standard Karch-Randall
mechanism [61, 62, 63] in the higher dimensional analogy of our setup. This
problem may be got around by introducing a holographic CFT on the end-of-the-
world brane, in a similar manner to [64].

## Acknowledgements

I would like to in particular thank Andy Strominger and Tadashi Takayanagi for
extensive discussions and insightful comments. I am also grateful to Levent
Alpöge, Stefano Antonini, Raphael Bousso, Xujia Chen, Katie Ebner-Landy,
Matthew Heydeman, Daniel Jafferis, Yikun Jiang, Alex May, Jake McNamara,
Hidetoshi Omiya, Haoyu Sun, Marija Tomašević, Herman Verlinde, Diandian Wang
and Yasushi Yoneta for stimulating conversations. Finally, I would like to
thank Shan-Ming Ruan, Tadashi Takayanagi and Diandian Wang for helpful
comments on a draft of this paper. I am supported by the Society of Fellows at
Harvard University.

## References

  * [1] G. ’t Hooft, _Dimensional reduction in quantum gravity_ , _Conf. Proc._ C930308 (1993) 284 [gr-qc/9310026].
  * [2] L. Susskind, _The World as a hologram_ , _J. Math. Phys._ 36 (1995) 6377 [hep-th/9409089].
  * [3] J. M. Maldacena, _The Large N limit of superconformal field theories and supergravity_ , _Int. J. Theor. Phys._ 38 (1999) 1113 [hep-th/9711200].
  * [4] D. Fioravanti, G. Pradisi and A. Sagnotti, _Sewing constraints and nonorientable open strings_ , _Phys. Lett. B_ 321 (1994) 349 [hep-th/9311183].
  * [5] G. Pradisi, A. Sagnotti and Y. S. Stanev, _The Open descendants of nondiagonal SU(2) WZW models_ , _Phys. Lett. B_ 356 (1995) 230 [hep-th/9506014].
  * [6] R. Blumenhagen and E. Plauschinn, _Introduction to Conformal Field Theory: With Applications to String Theory_ , Lecture Notes in Physics. Springer Berlin Heidelberg, 2009.
  * [7] M. Miyaji, T. Numasawa, N. Shiba, T. Takayanagi and K. Watanabe, _Continuous Multiscale Entanglement Renormalization Ansatz as Holographic Surface-State Correspondence_ , _Phys. Rev. Lett._ 115 (2015) 171602 [1506.01353].
  * [8] H. Verlinde, _Poking Holes in AdS/CFT: Bulk Fields from Boundary States_ , 1505.05069.
  * [9] Y. Nakayama and H. Ooguri, _Bulk Locality and Boundary Creating Operators_ , _JHEP_ 10 (2015) 114 [1507.04130].
  * [10] A. Maloney and S. F. Ross, _Holography on Non-Orientable Surfaces_ , _Class. Quant. Grav._ 33 (2016) 185006 [1603.04426].
  * [11] K. Goto, M. Miyaji and T. Takayanagi, _Causal Evolutions of Bulk Local Excitations from CFT_ , _JHEP_ 09 (2016) 130 [1605.02835].
  * [12] Y. Nakayama and H. Ooguri, _Bulk Local States and Crosscaps in Holographic CFT_ , _JHEP_ 10 (2016) 085 [1605.00334].
  * [13] A. Lewkowycz, G. J. Turiaci and H. Verlinde, _A CFT Perspective on Gravitational Dressing and Bulk Locality_ , _JHEP_ 01 (2017) 004 [1608.08977].
  * [14] S. Giombi, H. Khanchandani and X. Zhou, _Aspects of CFTs on Real Projective Space_ , _J. Phys. A_ 54 (2021) 024003 [2009.03290].
  * [15] I. Tsiares, _Universal Dynamics in Non-Orientable CFT 2_, 2011.09250.
  * [16] J. Caetano and S. Komatsu, _Crosscap States in Integrable Field Theories and Spin Chains_ , _J. Statist. Phys._ 187 (2022) 30 [2111.09901].
  * [17] J. Caetano and L. Rastelli, _Holography for $\mathcal{N}$ = 4 on $\mathbb{RP}^{4}$_, _JHEP_ 02 (2023) 106 [2206.06375].
  * [18] S. S. Gubser, I. R. Klebanov and A. M. Polyakov, _Gauge theory correlators from noncritical string theory_ , _Phys. Lett._ B428 (1998) 105 [hep-th/9802109].
  * [19] E. Witten, _Anti-de Sitter space and holography_ , _Adv. Theor. Math. Phys._ 2 (1998) 253 [hep-th/9802150].
  * [20] E. Witten, _Three-Dimensional Gravity Revisited_ , 0706.3359.
  * [21] A. Maloney and E. Witten, _Quantum Gravity Partition Functions in Three Dimensions_ , _JHEP_ 02 (2010) 029 [0712.0155].
  * [22] T. Takayanagi, _Holographic Dual of BCFT_ , _Phys. Rev. Lett._ 107 (2011) 101602 [1105.5165].
  * [23] M. Fujita, T. Takayanagi and E. Tonni, _Aspects of AdS/BCFT_ , _JHEP_ 11 (2011) 043 [1108.5152].
  * [24] S. R. Coleman and F. De Luccia, _Gravitational Effects on and of Vacuum Decay_ , _Phys. Rev. D_ 21 (1980) 3305.
  * [25] N. Ishibashi, _The Boundary and Crosscap States in Conformal Field Theories_ , _Mod. Phys. Lett._ A4 (1989) 251.
  * [26] H.-H. Tu, _Universal entropy of conformal critical theories on a Klein bottle_ , _Phys. Rev. Lett._ 119 (2017) 261603 [1707.05812].
  * [27] I. Affleck and A. W. Ludwig, _Universal noninteger “ground-state degeneracy”in critical quantum systems_ , _Physical Review Letters_ 67 (1991) 161.
  * [28] J. D. Brown and M. Henneaux, _Central Charges in the Canonical Realization of Asymptotic Symmetries: An Example from Three-Dimensional Gravity_ , _Commun. Math. Phys._ 104 (1986) 207.
  * [29] J. Kastikainen and S. Shashi, _Structure of holographic BCFT correlators from geodesics_ , _Phys. Rev. D_ 105 (2022) 046007 [2109.00079].
  * [30] Y. Kusuki and Z. Wei, _AdS/BCFT from conformal bootstrap: construction of gravity with branes and particles_ , _JHEP_ 01 (2023) 108 [2210.03107].
  * [31] J. L. Cardy, _Boundary Conditions, Fusion Rules and the Verlinde Formula_ , _Nucl. Phys._ B324 (1989) 581.
  * [32] M. Miyaji, S. Ryu, T. Takayanagi and X. Wen, _Boundary States as Holographic Duals of Trivial Spacetimes_ , _JHEP_ 05 (2015) 152 [1412.6226].
  * [33] Y. Nakata, T. Takayanagi, Y. Taki, K. Tamaoka and Z. Wei, _New holographic generalization of entanglement entropy_ , _Phys. Rev. D_ 103 (2021) 026005 [2005.13801].
  * [34] V. Chandrasekaran, R. Longo, G. Penington and E. Witten, _An algebra of observables for de Sitter space_ , _JHEP_ 02 (2023) 082 [2206.10780].
  * [35] Y. Chiba and Y. Yoneta, _Exact thermal eigenstates of nonintegrable spin chains at infinite temperature_ , 2403.12330.
  * [36] J. von Neumann, _On infinite direct products_ , _Compositio Mathematica_ 6 (1939) 1.
  * [37] R. T. Powers, _Representations of uniformly hyperfinite algebras and their associated von neumann rings_ , _Annals of Mathematics_ (1967) 138.
  * [38] H. Araki and E. J. Woods, _A classification of factors_ , _Publications of the Research Institute for Mathematical Sciences_ 4 (1968) 51.
  * [39] E. Witten, _Why Does Quantum Field Theory In Curved Spacetime Make Sense? And What Happens To The Algebra of Observables In The Thermodynamic Limit?_ , 2112.11614.
  * [40] A. B. Zamolodchikov, _Irreversibility of the Flux of the Renormalization Group in a 2D Field Theory_ , _JETP Lett._ 43 (1986) 730.
  * [41] D. Friedan and A. Konechny, _On the boundary entropy of one-dimensional quantum systems at low temperature_ , _Phys. Rev. Lett._ 93 (2004) 030402 [hep-th/0312197].
  * [42] H. Ishihara, _Causality of the brane universe_ , _Phys. Rev. Lett._ 86 (2001) 381 [gr-qc/0007070].
  * [43] H. Omiya and Z. Wei, _Causal structures and nonlocality in double holography_ , _JHEP_ 07 (2022) 128 [2107.01219].
  * [44] C. Yan, _Crosscap contribution to late-time two-point correlators_ , _JHEP_ 12 (2023) 051 [2203.14436].
  * [45] C. Yan, _More on torus wormholes in 3d gravity_ , _JHEP_ 11 (2023) 039 [2305.10494].
  * [46] D. Harlow and T. Numasawa, _Gauging spacetime inversions in quantum gravity_ , 2311.09978.
  * [47] I. Akal, Y. Kusuki, T. Takayanagi and Z. Wei, _Codimension two holography for wedges_ , _Phys. Rev. D_ 102 (2020) 126007 [2007.06800].
  * [48] I. Akal, T. Kawamoto, S.-M. Ruan, T. Takayanagi and Z. Wei, _Page curve under final state projection_ , _Phys. Rev. D_ 105 (2022) 126026 [2112.08433].
  * [49] I. Akal, T. Kawamoto, S.-M. Ruan, T. Takayanagi and Z. Wei, _Zoo of holographic moving mirrors_ , _JHEP_ 08 (2022) 296 [2205.02663].
  * [50] T. Kawamoto, S.-M. Ruan, Y.-k. Suzuki and T. Takayanagi, _A half de Sitter holography_ , _JHEP_ 10 (2023) 137 [2306.07575].
  * [51] A. Almheiri, R. Mahajan, J. Maldacena and Y. Zhao, _The Page curve of Hawking radiation from semiclassical geometry_ , _JHEP_ 03 (2020) 149 [1908.10996].
  * [52] A. Strominger, _The dS / CFT correspondence_ , _JHEP_ 10 (2001) 034 [hep-th/0106113].
  * [53] E. Witten, _Quantum gravity in de Sitter space_ , in _Strings 2001: International Conference_ , 6, 2001, hep-th/0106109.
  * [54] J. M. Maldacena, _Non-Gaussian features of primordial fluctuations in single field inflationary models_ , _JHEP_ 05 (2003) 013 [astro-ph/0210603].
  * [55] M. Alishahiha, A. Karch, E. Silverstein and D. Tong, _The dS/dS correspondence_ , _AIP Conf. Proc._ 743 (2004) 393 [hep-th/0407125].
  * [56] D. Anninos, T. Hartman and A. Strominger, _Higher Spin Realization of the dS/CFT Correspondence_ , _Class. Quant. Grav._ 34 (2017) 015009 [1108.5735].
  * [57] X. Dong, E. Silverstein and G. Torroba, _De Sitter Holography and Entanglement Entropy_ , _JHEP_ 07 (2018) 050 [1804.08623].
  * [58] L. Susskind, _Black Holes Hint towards De Sitter Matrix Theory_ , _Universe_ 9 (2023) 368 [2109.01322].
  * [59] L. Susskind, _Entanglement and Chaos in De Sitter Space Holography: An SYK Example_ , _JHAP_ 1 (2021) 1 [2109.14104].
  * [60] Y. Hikida, T. Nishioka, T. Takayanagi and Y. Taki, _Holography in de Sitter Space via Chern-Simons Gauge Theory_ , _Phys. Rev. Lett._ 129 (2022) 041601 [2110.03197].
  * [61] A. Karch and L. Randall, _Locally localized gravity_ , _JHEP_ 05 (2001) 008 [hep-th/0011156].
  * [62] L. Randall and R. Sundrum, _A Large mass hierarchy from a small extra dimension_ , _Phys. Rev. Lett._ 83 (1999) 3370 [hep-ph/9905221].
  * [63] L. Randall and R. Sundrum, _An Alternative to compactification_ , _Phys. Rev. Lett._ 83 (1999) 4690 [hep-th/9906064].
  * [64] J. Maldacena, _Vacuum decay into Anti de Sitter space_ , 1012.0274.