# Types of Out-of-Distribution Texts and How to Detect Them

Udit Arora♠ William Huang♣ He He♠
♠New York University
♣Capital One
<EMAIL_ADDRESS><EMAIL_ADDRESS>Work done while at New
York University.

###### Abstract

Despite agreement on the importance of detecting out-of-distribution (OOD)
examples, there is little consensus on the formal definition of OOD examples
and how to best detect them. We categorize these examples by whether they
exhibit a _background_ shift or a _semantic_ shift, and find that the two
major approaches to OOD detection, model calibration and density estimation
(language modeling for text), have distinct behavior on these types of OOD
data. Across 14 pairs of in-distribution and OOD English natural language
understanding datasets, we find that density estimation methods consistently
beat calibration methods in background shift settings, while performing worse
in semantic shift settings. In addition, we find that both methods generally
fail to detect examples from challenge data, highlighting a weak spot for
current methods. Since no single method works well across all settings, our
results call for an explicit definition of OOD examples when evaluating
different detection methods.

##  1 Introduction

Current NLP models work well when the training and test distributions are the
same (e.g. from the same benchmark dataset). However, it is common to
encounter out-of-distribution (OOD) examples that diverge from the training
data once the model is deployed to real settings. When training and test
distributions differ, current models tend to produce unreliable or even
catastrophic predictions that hurt user trust (Ribeiro et al., 2020).
Therefore, it is important to identify OOD inputs so that we can modify
models’ inference-time behavior by abstaining, asking for human feedback, or
gathering additional information Amodei et al. (2016).

Figure 1: Illustration of semantic shift and background shift in
$\mathbb{R}^{2}$. Each point consists of semantic features ($x$-axis) and
background features ($y$-axis). OOD examples (red points) can shift in either
direction. The background color indicates regions of ID (light) and OOD (dark)
given by the density estimation method (left) and the calibration method
(right). The calibration method fails to detect OOD examples due to background
shift.

Current work in NLP either focuses on specific tasks like intent
classification in task-oriented dialogue Zheng et al. (2020), or arbitrary in-
distribution (ID) and OOD dataset pairs Hendrycks et al. (2020b, 2019); Zhou
and Chen (2021), e.g. taking a sentiment classification dataset as ID and a
natural language inference dataset as OOD. However, getting inputs intended
for a different task is rare in realistic settings as users typically know the
intended task. In practice, an example is considered OOD due to various
reasons, e.g. being rare Sagawa et al. (2020), out-of-domain Daumé III (2007),
or adversarial Carlini and Wagner (2017). This broad range of distribution
shifts makes it unreasonable to expect a detection algorithm to work well for
arbitrary OOD examples without assumptions on the test distribution Ahmed and
Courville (2020).

In this paper, we categorize OOD examples by common types of distribution
shifts in NLP problems inspired by Ren et al. (2019) and Hsu et al. (2020).
Specifically, we assume an input (e.g. a movie review) can be represented as
background features (e.g. genre) that are invariant across different labels,
and semantic features (e.g. sentiment words) that are discriminative for the
prediction task. Correspondingly, at test time we consider two types of OOD
examples characterized by a major shift in the distribution of background and
semantic features, respectively. While the two types of shifts often happen
simultaneously, we note that there are realistic settings where distribution
shift is dominated by one or the other. For example, background shift
dominates when the domain or the style of the text changes Pavlick and
Tetreault (2016), e.g. from news to tweets, and semantic shift dominates when
unseen classes occur at test time, as in open-set classification Scheirer et
al. (2013).111 We exclude _task_ shift where the OOD examples are from a
different task, e.g. textual entailment inputs for a text classification
model, because it is less likely to happen in realistic settings where users
are often aware of the intended use of the model.

We use this categorization to evaluate two major approaches to OOD detection,
namely calibration methods that use the model’s prediction confidence
(Hendrycks and Gimpel, 2017; Liang et al., 2018) and density estimation
methods that fit a distribution of the training inputs (Nalisnick et al.,
2019a; Winkens et al., 2020; Kirichenko et al., 2020). We show that the two
approaches make implicit assumptions on the type of distribution shift, and
result in behavioral differences under each type of shift. By studying ID/OOD
pairs constructed from both simulations and real datasets, we find that the
density estimation method better accounts for shifts in background features,
consistently outperforming the calibration method on background shift pairs.
We further see the opposite in semantic shift pairs, with the calibration
method consistently yielding higher performance.

In addition, we analyze the detection performance on challenge datasets McCoy
et al. (2019a); Naik et al. (2018b) through the lens of background/semantic
shift. We find that these challenge datasets provide interesting failure cases
for both methods. Calibration methods completely fail when the model is over-
confident due to spurious semantic features. While density estimation methods
are slightly more robust, language models are easily fooled by repetitions
that significantly increase the probability of a piece of text. Together, our
findings suggest that better definitions of OOD and corresponding evaluation
datasets are required for both model development and fair comparison of OOD
detection methods.

##  2 Categorization of OOD Examples

###  2.1 Problem Statement

Consider classification tasks where each example consists of an input
$x\in\mathcal{X}$ and its label $y\in\mathcal{Y}$. In the task of OOD
detection, we are given a training dataset $\mathcal{D}_{\text{train}}$ of
$(x,y)$ pairs sampled from the training data distribution $p(x,y)$. At
inference time, given an input $x^{\prime}\in\mathcal{X}$ the goal of OOD
detection is to identify whether $x^{\prime}$ is a sample drawn from $p(x,y)$.

###  2.2 Types of Distribution Shifts

As in (Ren et al., 2019), we assume that any representation of the input $x$,
$\phi(x)$, can be decomposed into two independent and disjoint components: the
background features $\phi_{b}(x)\in\mathbb{R}^{m}$ and the semantic features
$\phi_{s}(x)\in\mathbb{R}^{n}$. Formally, we have

$\displaystyle\phi(x)$ $\displaystyle=[\phi_{s}(x);\phi_{b}(x)],$ (1)
$\displaystyle p(x)$ $\displaystyle=p(\phi_{s}(x))p(\phi_{b}(x))$ (2)

Further, we assume that $\phi_{b}(x)$ is independent of the label while
$\phi_{s}(x)$ is not. Formally, $\forall y\in\mathcal{Y}$,

$\displaystyle p(\phi_{b}(x)\mid y)=p(\phi_{b}(x)),$ (3) $\displaystyle
p(\phi_{s}(x)\mid y)\neq p(\phi_{s}(x))$ (4)

Note that $p$ refers to the ground truth distribution, as opposed to one
learned by a model.

Intuitively, the background features consist of population-level statistics
that do not depend on the label, whereas the semantic features have a strong
correlation with the label. A similar decomposition is also used in previous
work on style transfer (Fu et al., 2018), where a sentence is decomposed into
the content (semantic) and style (background) representations in the embedding
space.

Based on this decomposition, we classify the types of OOD data as either
semantic or background shift based on whether the distribution shift is driven
by changes in $\phi_{s}(x)$ or $\phi_{b}(x)$, respectively. An example of
background shift is a sentiment classification corpus with reviews from IMDB
versus GoodReads where phrases indicating positive reviews (e.g. “best”,
“beautifully”) are roughly the same while the background phrases change
significantly (e.g. “movie” vs “book”). On the other hand, semantic shift
happens when we encounter unseen classes at test time, e.g. a dialogue system
for booking flight tickets receiving a request for meal vouchers Zheng et al.
(2020), or a question-answering system handling unanswerable questions
Rajpurkar et al. (2018). We note that the two types of shifts may happen
simultaneously in the real world, and our categorization is based on the most
prominent type of shift.

##  3 OOD Detection Methods

To classify an input $x\in\mathcal{X}$ as ID or OOD, we produce a score $s(x)$
and classify it as OOD if $s(x)<\gamma$, where $\gamma$ is a pre-defined
threshold. Most methods differ by how they define $s(x)$. Below we describe
two types of methods commonly used for OOD detection.

#### Calibration methods.

These methods use the model’s prediction confidence as the score. A well-
calibrated model’s confidence score reflects the likelihood of the predicted
label being correct. Since the performance on OOD data is usually lower than
on ID data, lower confidence suggests that the input is more likely to be OOD.
The simplest method to obtain the confidence score is to directly use the
conditional probability produced by a probabilistic classifier
$p_{\text{model}}$, referred to as maximum softmax probability (MSP; Hendrycks
and Gimpel, 2017). Formally,

$\displaystyle
s_{\textit{MSP}}(x)=\max_{k\in\mathcal{Y}}p_{\text{model}}(y=k\mid x).$ (5)

While there exist more sophisticated methods that take additional calibration
steps Liang et al. (2018); Lee et al. (2018), MSP proves to be a strong
baseline, especially when $p_{\text{model}}$ is fine-tuned from pre-trained
transformers Hendrycks et al. (2020b); Desai and Durrett (2020).

#### Density estimation methods.

These methods use the likelihood of the input given by a density estimator as
the score. For text or sequence data, a language model $p_{\text{LM}}$ is
typically used to estimate $p(x)$ Ren et al. (2019). To avoid bias due to the
length of the sequence (see analysis in Appendix A), we use the token
perplexity (PPL) as the score. Formally, given a sequence
$x=(x_{1},\ldots,x_{T})$,

$\displaystyle s_{\textit{PPL}}(x)=\exp\left\\{\frac{1}{T}\sum_{t=1}^{T}\log
p_{\text{LM}}(x_{t}\mid x_{1:t-1})\right\\}$ (6)

While there are many works on density estimation methods using flow-based
models in computer vision (e.g. Nalisnick et al., 2019a; Zhang et al., 2020a),
there is limited work experimenting with density estimation methods for OOD
detection on text Lee et al. (2020).

#### Implicit assumptions on OOD.

One key question in OOD detection is how the distribution shifts at test time,
i.e. what characterizes the difference between ID and OOD examples. Without
access to OOD data during training, the knowledge must be incorporated into
the detector through some inductive bias. Calibration methods rely on $p(y\mid
x)$ estimated by a classifier, thus they are more influenced by the semantic
features which are correlated with the label. We can see this formally by

$\displaystyle p(y\mid x)$ $\displaystyle\propto p(x\mid y)p(y)$ (7)
$\displaystyle=p(\phi_{b}(x)\mid y)p(\phi_{s}(x)\mid y)p(y)$ (8)
$\displaystyle\propto p(\phi_{s}(x)\mid y)p(y).$ (9)

In contrast, density estimation methods are sensitive to all components of the
input, including both background and semantic features, even in situations
where distribution shifts are predominately driven by one particular type. In
the following sections, we examine how these implicit assumptions impact
performance on different ID/OOD pairs.

##  4 Simulation of Distribution Shifts

As an illustrative example, we construct a toy OOD detection problem using a
binary classification setting similar to the one depicted in Figure 1. This
allows us to remove estimation errors and study optimal calibration and
density estimation detectors under controlled semantic and background shifts.

###  4.1 Data Generation

We generate the ID examples from a Gaussian Mixture Model (GMM):

$\displaystyle y$ $\displaystyle=\begin{cases}0&\text{w.p. }0.5\\\
1&\text{otherwise}\end{cases},$ (10) $\displaystyle x\mid y=i$
$\displaystyle\sim\mathcal{N}(\mu^{i},\Sigma).$ (11)

The centroids are sets of semantic and background features such that
$\mu^{1}=[\mu_{s},\mu_{b}]$ and $\mu^{0}=[-\mu_{s},\mu_{b}]$, where
$\mu_{s}\in\mathbb{R}^{n}$ and $\mu_{b}\in\mathbb{R}^{m}$. In the 2D case in
Figure 1, this corresponds to the two Gaussian clusters where the first
component is the semantic feature and the second is the background feature.

In this case, we know the true calibrated score $p(y\mid x)$ and the true
density $p(x)$ given any inputs. Specifically, the optimal classifier is given
by the Linear Discriminant Analysis (LDA) predictor. By setting $\Sigma$ to
the identity matrix, it corresponds to a linear classifier with weights
$[2\mu_{s},\mathbf{0}_{b}]$, where $\mathbf{0}_{b}\in\mathbb{R}^{m}$ is a
vector of all $0$s. For simplicity, we set $\mu_{s}=\mathbf{1}_{s}$ and
$\mu_{b}=\mathbf{0}_{b}$, where
$\mathbf{1}_{s}\in\mathbb{R}^{n},\mathbf{0}_{b}\in\mathbb{R}^{m}$ are vectors
of all $0$s.

###  4.2 Semantic Shift

We generate sets of OOD examples using a semantic shift by varying the overlap
of ID and OOD semantic features. Formally, we vary the overlap rate $r$ such
that

$\displaystyle r$
$\displaystyle=\frac{|\mu_{s}\cap\mu_{s}^{\text{Shift}}|}{|\mu_{s}|}$ (12)

where $\mu_{s},\mu_{s}^{\text{Shift}}\in\mathbb{R}^{n}$ are the set of
semantic features for ID and OOD, respectively,
$\mu_{s}\cap\mu_{s}^{\text{Shift}}$ represents the common features between the
two, and $|\cdot|$ denotes the number of elements.

We fix the total dimensions to $n+m=200$ and set $n=40$ (semantic features)
and $m=160$ (background features). Further, we vary $r$ by increments of
$10\%$. Larger $r$ indicates stronger semantic shift. For each $r$, we
randomly sample ID and OOD semantic features and report the mean over $20$
trials with $95\%$ confidence bands in Figure 2.

###  4.3 Background Shift

We generate sets of OOD examples using a background shift by applying a
displacement vector $z=[\mathbf{0}_{s},z_{b}]$ to the two means. Formally,

$\displaystyle\mu^{i,\text{ Shift}}=\mu^{i}+z$ (13)

where $\mathbf{0}_{s}\in\mathbb{R}^{n}$ is a vector of all $0$s.

We set $z=\alpha[\mathbf{0}_{s},\mathbf{1}_{b}]$, where
$\mathbf{1}_{b}\in\mathbb{R}^{m}$ is a vector of $1$s. Note that this shift
corresponds to a translation of the ID distribution along the direction of
$\mu_{b}$. We set the total dimensions to $n+m=200$ while varying the split
between semantic ($n$) and background ($m$) components by increments of $20$.

###  4.4 Simulation Results

Figure 2: Area Under Receiver Operating Characteristics (AUROC) of calibration
(blue) and density estimation (orange) methods for OOD detection using our toy
binary classification problem. The calibration method outperforms the density
estimation method under larger semantic shifts while the opposite is true
under larger background shifts.

Figure 2 shows the OOD detection performance of our simulated experiment. We
use Area Under the Receiver Operating Characteristics (AUROC) as our
performance metric.

We see that the calibration method generally outperforms density estimation.
Further, the performance gap between the two methods decreases as both methods
approach near-perfect performance under large semantic shifts with no overlap
in semantic features, and approach chance under no semantic shift with
completely overlapping semantic features. However, the calibration method is
unable to improve performance under background shifts in either regime because
the background features do not contribute to $p(y\mid x)$ as the LDA weights
are $0$ for these components (Section 4.1). We find these results in line with
our expectations and use them to drive our intuition when evaluating both
types of OOD detection methods for real text data.

##  5 Experiments and Analysis

We perform head-to-head comparisons of calibration and density estimation
methods on 14 ID/OOD pairs categorized as either background shift or semantic
shift, as well as 8 pairs from challenge datasets.

###  5.1 Setup

#### OOD detectors.

Recall that the calibration method MSP relies on a classifier trained on the
ID data. We fine-tune the RoBERTa (Liu et al., 2019) model on the ID data and
compute its prediction probabilities (see Equation (5)). For the density
estimation method PPL, we fine-tune GPT-2 (Radford et al., 2019) on the ID
data and use perplexity as the OOD score (see Equation (6)).222 We also use
the sentence probability ($p(x)$) as the score, but find it highly sensitive
to sequence lengths (Appendix A).  To control for model size of the two
methods, we choose RoBERTaBase and GPT-2Small, which have 110M and 117M
parameters, respectively. We also experiment with two larger models,
RoBERTaLarge and GPT-2Medium with 355M and 345M parameters, respectively.

We evaluate the OOD detectors by AUROC and the False Alarm Rate at 95% Recall
(FAR95), which measures the misclassification rate of ID examples at 95% OOD
recall. Both metrics show similar trends (see Appendix B for FAR95 results).

#### Training details.

For RoBERTa, we fine-tune the model for 3 epochs on the training split of ID
data with a learning rate of 1e-5 and a batch size of 16. For GPT-2, we fine-
tune the model for 1 epoch on the training split of ID data for the language
modeling task, using a learning rate of 5e-5 and a batch size of 8. 333Our
code can be found at https://github.com/uditarora/ood-text-emnlp.

#### Oracle detectors.

To get an estimate of the upper bound of OOD detection performance, we
consider the situation where we have access to the OOD data and can directly
learn an OOD classifier. Specifically, we train a logistic regression model
with bag-of-words features using 80% of the test data and report results on
the remaining 20%.

###  5.2 Semantic Shift

Recall that the distribution of discriminative features changes in the
semantic shift setting, i.e. $p_{\text{train}}(\phi_{s}(x))\neq
p_{\text{test}}(\phi_{s}(x))$ (Section 2). We create semantic shift pairs by
including test examples from classes unseen during training. Thus, semantic
features useful for classifying the training data are not representative in
the test set.

We use the News Category (Misra, 2018) and DBPedia Ontology Classification
(Zhang et al., 2015) multiclass classification datasets to create two ID/OOD
pairs. The News Category dataset consists of HuffPost news data. We use the
examples from the five most frequent classes as ID (News Top-5) and the data
from the remaining 36 classes as OOD (News Rest). The DBPedia Ontology
Classification dataset consists of data from Wikipedia extracted from 14 non-
overlapping classes of DBPedia 2014 (Lehmann et al., 2015). We use examples
from the first four classes by class number as ID (DBPedia Top-4) and the rest
as OOD (DBPedia Rest).

#### Results.

ID | OOD | AUROC
---|---|---
PPL | MSP | Oracle
News Top-5 | News Rest | 60.2 | 78.9 | 72.0
DBPedia Top-4 | DBPedia Rest | 75.4 | 88.8 | 99.6

Table 1: Performance on semantic shifts, with higher score (among PPL/MSP) in
bold. We can see that the calibration method using MSP significantly
outperforms the density estimation methods.

Table 1 shows the results for our semantic shift pairs. The calibration method
consistently outperforms the density estimation method, indicating that
calibration methods are better suited for scenarios with large semantic
shifts, which is in line with our simulation results (Section 4).

###  5.3 Background Shift

Recall that background features (e.g. formality) do not depend on the label.
Therefore, we consider domain shift in sentiment classification and natural
language inference (NLI) datasets.

For our analysis, we use the SST-2 Socher et al. (2013), IMDB (Maas et al.,
2011), and Yelp Polarity (Zhang et al., 2015) binary sentiment classification
datasets. The SST-2 and IMDB datasets consist of movie reviews with different
lengths. Meanwhile, the Yelp polarity dataset contains reviews for different
businesses, representing a domain shift from SST-2 and IMDB. Each of these
datasets is used as ID/OOD, using the validation split of SST-2 and test split
of IMDB and Yelp Polarity for evaluation.

We also use the SNLI Bowman et al. (2015), MNLI Williams et al. (2018) and RTE
(from GLUE, Wang et al., 2018a) datasets. SNLI and MNLI consist of NLI
examples sourced from different genres. RTE comprises of examples sourced from
a different domain. Where there is some change in semantic information since
the task has the two labels (entailment and non-entailment) as opposed to
three (entailment, neutral and contradiction) in SNLI and MNLI,444Both neutral
and contradiction are considered as non-entailment when evaluating accuracy
with RTE vs SNLI/MNLI or vice-versa. domain/background shift is more prominent
since the semantic features for the NLI task are similar. Each of these
datasets is used as either ID or OOD, and we use the validation set of the OOD
data for evaluation.

#### Results.

ID | OOD | AUROC | Accuracy
---|---|---|---
PPL | MSP | Oracle | OOD ($\Delta$) | ID
SST-2 | IMDB | 97.9 | 66.2 | 100.0 | 92.0 (-1.8) | 93.8
Yelp | 98.7 | 57.5 | 99.8 | 94.4 (+0.6)
IMDB | SST-2 | 96.9 | 82.6 | 100.0 | 89.2 (-6.3) | 95.5
Yelp | 77.9 | 67.1 | 100.0 | 95.4 (-0.1)
Yelp | SST-2 | 98.9 | 85.9 | 99.8 | 88.9 (-9.3) | 98.2
IMDB | 86.6 | 61.8 | 100.0 | 93.2 (-5.0)
SNLI | RTE | 94.6 | 78.7 | 99.8 | 67.5 (-22.6) | 90.1
MNLI | 96.7 | 75.6 | 99.7 | 77.9 (-12.2)
RTE | SNLI | 81.2 | 45.1 | 99.7 | 82.0 (+6.9) | 75.1
MNLI | 81.4 | 55.5 | 97.0 | 77.3 (+2.2)
MNLI | SNLI | 75.7 | 56.1 | 99.7 | 80.4 (-4.4) | 84.8
RTE | 68.0 | 76.5 | 96.7 | 76.5 (-8.3)

Table 2: Performance on background shifts caused by shift in domain. For each
pair, higher score obtained (by PPL or MSP) is in bold. The density estimation
method using PPL outperforms the calibration method.

Table 2 shows the results for binary sentiment classification and NLI domain
shifts. The density estimation method consistently outperforms the calibration
method (for all pairs except MNLI vs RTE), indicating that PPL is more
sensitive to changes in background features. Further, in cases where the
discriminative model generalizes well (as evident by the small difference in
ID and OOD accuracy numbers), we find that the calibration method performance
is close to random (50) because a well-calibrated model also has higher
confidence on its correct OOD predictions.

We note that the discriminative models tend to generalize well here, hence it
might be better to focus on domain adaptation instead of OOD detection when
the shift is predominantly a background shift. We discuss this further in
Section 6.

###  5.4 Analysis

#### Controlled distribution shifts.

We use two controlled distribution shift experiments on real text data to
further study the framework of semantic and background shifts. For background
shift, we append different amounts of text from Wikitext (Merity et al., 2017)
and Civil Comments (Borkan et al., 2019a) to SST-2 examples to create
synthetic ID and OOD examples, respectively. We append the unrelated texts
with lengths $\in(25,50,100,150,200)$ words. For semantic shift, we use the
News Category dataset and move classes from ID to OOD. We start with the top
40 ID classes by frequency and move classes in increments of 10. The ID
coverage of semantic information decreases as more classes move to the OOD
subset, resulting in a larger semantic shift.

#### Results.

Figure 3: AUROC of PPL (orange) and MSP (blue) for controlled background and
semantic shift experiments. The density estimation method performance improves
as we increase the amount of background shift by appending longer texts, and
the calibration method performance increases as we increase the amount of
semantic shift by moving more classes to OOD.

Figure 3 shows the AUROC score obtained from both methods for our controlled
distribution shift experiments. We see that the density estimation method is
more sensitive to the amount of synthetic background text than calibration
methods, and that the calibration method is more sensitive to the number of
ID/OOD classes. This is in line with our intuition about the shifts and the
results we obtain from simulated data (Section 4).

ID | OOD | Base | Large |
---|---|---|---|---
PPL | MSP | PPL | MSP | Oracle
IMDB | Yelp | 77.9 | 67.1 | 75.5 | 74.5 | 100.0
News Top-5 | News Rest | 60.2 | 78.9 | 61.7 | 79.1 | 72.0

Table 3: Performance of Base and Large models for a background shift pair and
semantic shift pair each, with higher score in bold. The larger discriminative
model helps close the performance gap between the calibration method and
density estimation method for background shift.

#### Larger models.

Table 3 shows the results using larger models for OOD detection. We observe
that the larger discriminative model achieves a much higher score for the
background shift pair, closing the gap with the language model performance. We
speculate that the larger model is able to learn some of the background
features in its representation. The performance for the semantic shift pair is
largely unchanged when using the larger models.

###  5.5 Challenge Data

Challenge datasets are designed to target either superficial heuristics
adopted by a model (e.g. premise-hypothesis overlap) or model deficiencies
(e.g. numerical reasoning in NLI), which creates significant challenges for
deployed models Ribeiro et al. (2020). It is therefore desirable to abstain on
detected OOD examples. We consider the following challenge datasets.

#### Human-generated challenge data.

Kaushik et al. (2020) crowdsourced a set of counterfactually-augmented IMDB
examples (c-IMDB) by instructing annotators to minimally edit examples to
yield counterfactual labels. This changes the distribution of semantic
features with high correlation to labels such that
$p_{\text{train}}(\phi_{s}(x))\neq p_{\text{test}}(\phi_{s}(x))$, creating a
semantic shift. We consider IMDB as ID and c-IMDB as OOD, combining the
training, validation, and test splits of c-IMDB for evaluation.

#### Rule-based challenge data.

HANS (McCoy et al., 2019b) consists of template-based examples that have high
premise-hypothesis overlap but are non-entailment, which mainly results in
background shift due to the specific templates/syntax. Similarly, the Stress
Test dataset (Naik et al., 2018a) is a set of automatically generated examples
designed to evaluate common errors from NLI models. We categorize the type of
distribution shifts from these test categories with respect to MNLI (ID)
depending on whether they append “background” phrases to the ID examples or
replace discriminative phrases (Table 4).

Antonym (changing premise to obtain an antonymous hypothesis resulting in
contradiction despite high overlap) and Numerical Reasoning (different
semantic information than MNLI training set) constitute semantic shifts, as
the set of semantic features now focus on specific types of entailment
reasoning (e.g. antonymy and numerical representation). Negation (appending
“and false is not true” to hypothesis), Spelling Errors (randomly introducing
spelling errors in one premise word), Word Overlap (appending “and true is
true” to each hypothesis), and Length Mismatch (appending a repetitive phrase
“and true is true” five times to the premise) constitute background shifts
because they introduce population level changes (e.g. appending “and true is
true” to each hypothesis) that are unrelated to the entailment conditions of
each example.

We consider the matched Negation, Spelling Errors, Word Overlap and Length
Mismatch examples from the Stress Test as background shifts, and the Numerical
Reasoning and Antonym examples as semantic shifts. We consider MNLI as ID for
these challenge examples and use the validation split of HANS and MNLI for
evaluation.

ID | OOD | Shift | AUROC
---|---|---|---
PPL | MSP | Oracle
IMDB | c-IMDB | Semantic | 53.5 | 63.7 | 77.5
MNLI | HANS | Background | 98.3 | 55.0 | 100.0
Negation | Background | 44.5 | 60.5 | 99.9
Len. Mismatch | Background | 19.6 | 51.6 | 100.0
Spell. Error | Background | 43.9 | 57.7 | 98.4
Word Overlap | Background | 42.4 | 61.7 | 99.8
Antonym | Semantic | 4.5 | 55.3 | 97.3
Num. Reason. | Semantic | 27.5 | 75.8 | 99.7

Table 4: AUROC scores obtained using PPL, MSP and Oracle for challenge data.
The primary type of shift observed is indicated in the ‘Shift’ column. Higher
performance (among MSP/PPL) for each pair is in bold. We can see that both
methods struggle with most types of challenge data.

#### Failure case 1: spurious semantic features.

Challenge data is often constructed to target _spurious features_ (e.g.
premise-hypothesis overlap for NLI) that are useful on the training set but do
not correlate with the label in general, e.g. on the test set. Therefore, a
discriminative model would be _over-confident_ on the OOD examples because the
spurious semantic features that were discriminative during training, while
still prominent, are no longer predictive of the label. As a result, in Table
4, MSP struggles with most challenge data, achieving an AUROC score close to
random (50). On the other hand, the density estimation method achieves almost
perfect performance on HANS.

#### Failure case 2: small shifts.

While density estimation methods perform better in background shift settings,
our simulation results show that they still struggle to detect small shifts
when the ID and OOD distributions largely overlap. Table 4 shows similar
findings for Negation and Word Overlap Stress Test categories that append
short phrases (e.g. “and true is true”) to each ID hypothesis.

#### Failure case 3: repetition.

For Antonym, Numerical Reasoning, and Length Mismatch, PPL performance is
_significantly worse than random_ , indicating that our language model assigns
higher likelihoods to OOD than ID examples. These challenge examples contain
highly repetitive phrases (e.g. appending “and true is true” five times in
Length Mismatch, or high overlap between premise and hypothesis in Numerical
Reasoning and Antonym), which is known to yield high likelihood under
recursive language models Holtzman et al. (2020). Thus repetition may be used
as an attack to language model-based OOD detectors.

Overall, the performance of both methods drops significantly on the challenge
datasets. Among these, human-generated counterfactual data is the most
difficult to detect, and rule-based challenge data can contain unnatural
patterns that cause unexpected behavior.

###  5.6 Discussion

The performance of calibration and density estimation methods on OOD examples
categorized along the lines of semantic and background shift provides us with
insights that can be useful in improving OOD detection. This framework can be
used to build better evaluation benchmarks that focus on different challenges
in OOD detection. A choice between the two methods can also be made based on
the anticipated distribution shift at test time, i.e, using calibration
methods when detecting semantic shift is more important, and using density
estimation methods to detect background shifts. However, we observe failure
cases from challenge examples, with density estimation methods failing to
detect OOD examples with repetition and small shifts, and calibration methods
failing to detect most challenge examples. This indicates that these challenge
examples constitute a type of OOD that target the weaknesses of both
approaches. This highlights the room for a more explicit definition of OOD to
progress the development of OOD detection methods and create benchmarks that
reflect realistic distribution shifts.

##  6 Related Work

#### Distribution shift in the wild.

Most early works on OOD detection make no distinctions on the type of
distribution shift observed at test time, and create synthetic ID/OOD pairs
using different datasets based on the setup in Hendrycks and Gimpel (2017).
Recently, there is an increasing interest in studying real-world distribution
shifts Ahmed and Courville (2020); Hsu et al. (2020); Hendrycks et al.
(2020a); Koh et al. (2020a). On these benchmarks with a diverse set of
distribution shifts, no single detection method wins across the board. We
explore the framework of characterization of distribution shifts along the two
axes of semantic shift and background (or non-semantic) shift, shedding light
on the performance of current methods.

#### OOD detection in NLP.

Even though OOD detection is crucial in production (e.g. dialogue systems Ryu
et al. (2018)) and high-stake applications (e.g. healthcare Borjali et al.
(2020)), it has received relatively less attention in NLP until recently.
Recent works evaluated/improved the calibration of pretrained transformer
models Hendrycks et al. (2020b); Goyal and Durrett (2020); Kong et al. (2020);
Zhou and Chen (2021). They show that while pretrained transformers are better
calibrated, making them better at detecting OOD data than previous models,
there is scope for improvement. Our analysis reveals one limitation of
calibration-based detection when faced with a background shift. Other works
focus on specific tasks, including prototypical network for low-resource text
classification Tan et al. (2019) and data augmentation for intent
classification Zheng et al. (2020).

#### Inductive bias in OOD detection.

Our work shows that the effectiveness of a method largely depends on whether
its assumption on the distribution shift matches the test data. One
straightforward way to incorporate prior knowledge on the type of distribution
shift is through augmenting similar OOD data during training, i.e., the so-
called outlier exposure method Hendrycks et al. (2019), which has been shown
to be effective on question answering Kamath et al. (2020). Given that the
right type of OOD data can be difficult to obtain, another line of work uses a
hybrid of calibration and density estimation methods to achieve a balance
between capturing semantic features and background features. These models are
usually trained with both a discriminative loss and a generative (or self-
supervised) loss Winkens et al. (2020); Zhang et al. (2020a); Nalisnick et al.
(2019b).

#### Domain adaptation versus OOD detection.

There are two ways of handling the effect of OOD data: 1) build models that
perform well across domains (i.e., background shifts), i.e., domain adaptation
Chu and Wang (2018); Kashyap et al. (2021) or 2) allow models to detect a
shift in data distribution, and potentially abstain from making a prediction.
In our setting (2), we want to guard against all types of OOD data without any
access to it, unlike domain adaptation which usually relies on access to OOD
data. This setting can be more important than (1) for safety-critical
applications, such as those in healthcare, because the potential cost of an
incorrect prediction is greater, motivating a more conservative approach to
handling OOD data by abstaining. This could also help improve performance in
selective prediction Kamath et al. (2020); Xin et al. (2021).

##  7 Conclusion

Despite the extensive literature on outlier and OOD detection, previous work
in NLP tends to lack consensus on a rigorous definition of OOD examples,
instead relying on arbitrary dataset pairs from different tasks. In our work,
we approach this problem in natural text and simulated data by categorizing
OOD examples as either background or semantic shifts and study the performance
of two common OOD detection methods—calibration and density estimation. For
both types of data, we find that density estimation methods outperform
calibration methods under background shifts while the opposite is true under
semantic shifts. However, we find several failure cases from challenge
examples that target model shortcomings.

As explained in Section 2, we assume that $\phi_{s}$ and $\phi_{b}$ map $x$ to
two disjoint sets of components for simplicity. This assumption helps us
simplify the framework and compare the two types of detection methods in
relation to the two types of shifts. While this simplified framework explains
much of the differences between the two methods, failure cases from challenge
examples highlight the room for better frameworks and a more explicit
definition of OOD to progress the development of OOD detection methods. Such a
definition can inform the creation of benchmarks on OOD detection that reflect
realistic distribution shifts.

Defining (or at least explicitly stating) the types of OOD examples that
predictors are designed to target can also guide future modeling decisions
between using calibration and density estimation methods, and help improve
detection. Some promising directions include test-time fine-tuning Sun et al.
(2020) and data augmentation Chen et al. (2020), which can be guided towards a
specific type of distribution shift for improved detection performance against
it. Finally, the methods we studied work well for one type of shift, which
motivates the use of hybrid models Zhang et al. (2020b); Liu and Abbeel (2020)
that use both calibration and density estimation when both types of shift
occur at the same time.

## Ethical Considerations

As society continues to rely on automated machine learning systems to make
important decisions that affect human lives, OOD detection becomes
increasingly vital to ensure that these systems can detect natural shifts in
domain and semantics. If medical chat-bots cannot recognize that new disease
variants or rare co-morbidities are OOD while diagnosing patients, they will
likely provide faulty and potentially harmful recommendations
555https://www.nabla.com/blog/gpt-3/ if they don’t contextualize their
uncertainty. We believe that implementing OOD detection, especially for more
challenging but commonly occurring semantic shifts should be part of any long-
lasting production model.

In addition, OOD detection can be used to identify and alter model behavior
when encountering data related to minority groups. For example, Koh et al.
(2020b) present a modified version of the CivilComments dataset Borkan et al.
(2019b), with the task of identifying toxic user comments on online platforms.
They consider domain annotations for each comment based on whether the comment
mentions each of 8 demographic identities - male, female, LGBTQ, Christian,
Muslim, other religions, Black and White. They note that a standard BERT-based
model trained using ERM performs poorly on the worst group, with a 34.2 % drop
in accuracy as compared to the average. Such models may lead to unintended
consequences like flagging a comment as toxic just because it mentions some
demographic identities, or in other words, belongs to some domains. Our work
can be useful in altering the inference-time behavior of such models upon
detection of such domains which constitute a larger degree of background
shift. Of course, nefarious agents could use the same pipeline to alter model
behavior to identify and discriminate against demographics that display such
background shifts.

## Acknowledgements

We thank Samsung Advanced Institute of Technology (Next Generation Deep
Learning: From Pattern Recognition to AI) for their support. Further, we thank
the anonymous reviewers, Richard Pang, Ethan Perez, Angelica Chen and other
members of the Machine Learning for Language Lab at New York University for
their thoughtful suggestions on improving the paper. We also want to thank
Diksha Meghwal, Vaibhav Gadodia and Ambuj Ojha for their help with an initial
version of the project and experimentation setup.

## References

  * Ahmed and Courville (2020) Faruk Ahmed and Aaron C. Courville. 2020.  Detecting semantic anomalies.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 3154–3162. AAAI Press.
  * Amodei et al. (2016) Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané. 2016.  Concrete problems in AI safety.  _CoRR_ , abs/1606.06565.
  * Borjali et al. (2020) Alireza Borjali, Martin Magneli, David Shin, Henrik Malchau, Orhun K. Muratoglu, and Kartik M. Varadarajan. 2020.  Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: A case study of detecting total hip replacement dislocation.  _CoRR_ , abs/2004.08333.
  * Borkan et al. (2019a) Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019a.  Nuanced metrics for measuring unintended bias with real data for text classification.  In _Companion of The 2019 World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019_ , pages 491–500. ACM.
  * Borkan et al. (2019b) Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019b.  Nuanced metrics for measuring unintended bias with real data for text classification.  In _Companion Proceedings of The 2019 World Wide Web Conference_ , WWW ’19, page 491–500, New York, NY, USA. Association for Computing Machinery.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015_ , pages 632–642. The Association for Computational Linguistics.
  * Carlini and Wagner (2017) Nicholas Carlini and David A. Wagner. 2017.  Adversarial examples are not easily detected: Bypassing ten detection methods.  In _Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec@CCS 2017, Dallas, TX, USA, November 3, 2017_ , pages 3–14. ACM.
  * Chen et al. (2020) Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. 2020.  Robust out-of-distribution detection via informative outlier mining.  _CoRR_ , abs/2006.15207.
  * Chu and Wang (2018) Chenhui Chu and Rui Wang. 2018.  A survey of domain adaptation for neural machine translation.  In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_ , pages 1304–1319. Association for Computational Linguistics.
  * Daumé III (2007) Hal Daumé III. 2007.  Frustratingly easy domain adaptation.  In _ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic_. The Association for Computational Linguistics.
  * Desai and Durrett (2020) Shrey Desai and Greg Durrett. 2020.  Calibration of pre-trained transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 295–302, Online. Association for Computational Linguistics.
  * Fu et al. (2018) Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018.  Style transfer in text: Exploration and evaluation.  In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ , pages 663–670. AAAI Press.
  * Goyal and Durrett (2020) Tanya Goyal and Greg Durrett. 2020.  Evaluating factuality in generation with dependency-level entailment.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, EMNLP 2020, Online Event, 16-20 November 2020_ , pages 3592–3603. Association for Computational Linguistics.
  * Hendrycks et al. (2020a) Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. 2020a.  Scaling out-of-distribution detection for real-world settings.  _CoRR_ , abs/1911.11132.
  * Hendrycks and Gimpel (2017) Dan Hendrycks and Kevin Gimpel. 2017.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Hendrycks et al. (2020b) Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020b.  Pretrained transformers improve out-of-distribution robustness.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 2744–2751. Association for Computational Linguistics.
  * Hendrycks et al. (2019) Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. 2019.  Deep anomaly detection with outlier exposure.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Hsu et al. (2020) Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. 2020.  Generalized ODIN: detecting out-of-distribution image without learning from out-of-distribution data.  In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_ , pages 10948–10957. IEEE.
  * Kamath et al. (2020) Amita Kamath, Robin Jia, and Percy Liang. 2020.  Selective question answering under domain shift.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 5684–5696. Association for Computational Linguistics.
  * Kashyap et al. (2021) Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, and Roger Zimmermann. 2021.  Domain divergences: A survey and empirical analysis.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021_ , pages 1830–1849. Association for Computational Linguistics.
  * Kaushik et al. (2020) Divyansh Kaushik, Eduard H. Hovy, and Zachary Chase Lipton. 2020.  Learning the difference that makes A difference with counterfactually-augmented data.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Kirichenko et al. (2020) Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. 2020.  Why normalizing flows fail to detect out-of-distribution data.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Koh et al. (2020a) Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2020a.  WILDS: A benchmark of in-the-wild distribution shifts.  _CoRR_ , abs/2012.07421.
  * Koh et al. (2020b) Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2020b.  WILDS: A benchmark of in-the-wild distribution shifts.  _CoRR_ , abs/2012.07421.
  * Kong et al. (2020) Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. 2020\.  Calibrated language model fine-tuning for in- and out-of-distribution data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_ , pages 1326–1340. Association for Computational Linguistics.
  * Lee et al. (2018) Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada_ , pages 7167–7177.
  * Lee et al. (2020) Nayeon Lee, Yejin Bang, Andrea Madotto, and Pascale Fung. 2020.  Misinformation has high perplexity.  _CoRR_ , abs/2006.04666.
  * Lehmann et al. (2015) Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015.  Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia.  _Semantic web_ , 6(2):167–195.
  * Liang et al. (2018) Shiyu Liang, Yixuan Li, and R. Srikant. 2018.  Enhancing the reliability of out-of-distribution image detection in neural networks.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Liu and Abbeel (2020) Hao Liu and Pieter Abbeel. 2020.  Hybrid discriminative-generative training via contrastive learning.  _CoRR_ , abs/2007.09070.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A robustly optimized BERT pretraining approach.  _CoRR_ , abs/1907.11692.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * McCoy et al. (2019a) Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019a.  Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 3428–3448. Association for Computational Linguistics.
  * McCoy et al. (2019b) Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019b.  Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3428–3448, Florence, Italy. Association for Computational Linguistics.
  * Merity et al. (2017) Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017.  Pointer sentinel mixture models.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Misra (2018) Rishabh Misra. 2018.  News category dataset.
  * Naik et al. (2018a) Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018a.  Stress test evaluation for natural language inference.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Naik et al. (2018b) Aakanksha Naik, Abhilasha Ravichander, Norman M. Sadeh, Carolyn Penstein Rosé, and Graham Neubig. 2018b.  Stress test evaluation for natural language inference.  In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_ , pages 2340–2353. Association for Computational Linguistics.
  * Nalisnick et al. (2019a) Eric T. Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Görür, and Balaji Lakshminarayanan. 2019a.  Do deep generative models know what they don’t know? In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Nalisnick et al. (2019b) Eric T. Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Görür, and Balaji Lakshminarayanan. 2019b.  Hybrid models with deep and invertible features.  In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 4723–4732. PMLR.
  * Pavlick and Tetreault (2016) Ellie Pavlick and Joel R. Tetreault. 2016.  An empirical analysis of formality in online communication.  _Trans. Assoc. Comput. Linguistics_ , 4:61–74.
  * Radford et al. (2019) A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Rajpurkar et al. (2018) P. Rajpurkar, R. Jia, and P. Liang. 2018.  Know what you don’t know: Unanswerable questions for SQuAD.  In _Association for Computational Linguistics (ACL)_.
  * Ren et al. (2019) Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. 2019.  Likelihood ratios for out-of-distribution detection.  In _Advances in Neural Information Processing Systems_ , volume 32, pages 14707–14718. Curran Associates, Inc.
  * Ribeiro et al. (2020) Marco Túlio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020\.  Beyond accuracy: Behavioral testing of NLP models with checklist.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 4902–4912. Association for Computational Linguistics.
  * Ryu et al. (2018) Seonghan Ryu, Sangjun Koo, Hwanjo Yu, and Gary Geunbae Lee. 2018.  Out-of-domain detection based on generative adversarial network.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018_ , pages 714–718. Association for Computational Linguistics.
  * Sagawa et al. (2020) Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. 2020.  Distributionally robust neural networks.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Scheirer et al. (2013) Walter J. Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E. Boult. 2013.  Toward open set recognition.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 35(7):1757–1772.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL_ , pages 1631–1642. ACL.
  * Sun et al. (2020) Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. 2020.  Test-time training with self-supervision for generalization under distribution shifts.  In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 9229–9248. PMLR.
  * Tan et al. (2019) Ming Tan, Yang Yu, Haoyu Wang, Dakuo Wang, Saloni Potdar, Shiyu Chang, and Mo Yu. 2019.  Out-of-domain detection for low-resource text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , pages 3564–3570. Association for Computational Linguistics.
  * Wang et al. (2018a) A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. 2018a.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_.
  * Wang et al. (2018b) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018b.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the Workshop: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2018, Brussels, Belgium, November 1, 2018_ , pages 353–355. Association for Computational Linguistics.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122. Association for Computational Linguistics.
  * Winkens et al. (2020) Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, A. Taylan Cemgil, S. M. Ali Eslami, and Olaf Ronneberger. 2020.  Contrastive training for improved out-of-distribution detection.  _CoRR_ , abs/2007.05566.
  * Xin et al. (2021) Ji Xin, Raphael Tang, Yaoliang Yu, and Jimmy Lin. 2021.  The art of abstention: Selective prediction and error regularization for natural language processing.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 1040–1051. Association for Computational Linguistics.
  * Zhang et al. (2020a) Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. 2020a.  Hybrid models for open set recognition.  In _Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III_ , volume 12348 of _Lecture Notes in Computer Science_ , pages 102–117. Springer.
  * Zhang et al. (2020b) Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. 2020b.  Hybrid models for open set recognition.  In _Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III_ , volume 12348 of _Lecture Notes in Computer Science_ , pages 102–117. Springer.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In _Advances in Neural Information Processing Systems_ , volume 28, pages 649–657. Curran Associates, Inc.
  * Zheng et al. (2020) Yinhe Zheng, Guanyi Chen, and Minlie Huang. 2020.  Out-of-domain detection for natural language understanding in dialog systems.  _IEEE ACM Trans. Audio Speech Lang. Process._ , 28:1198–1209.
  * Zhou and Chen (2021) Wenxuan Zhou and Muhao Chen. 2021.  Contrastive out-of-distribution detection for pretrained transformers.  _CoRR_ , abs/2104.08812.

##  Appendix A Example Probability

We additionally evaluate our density estimation methods using $\log p(x)$ as a
detection measure. In the case of text, $\log p(x)$ is defined as
$\sum_{i=1}^{t}\log p(x_{i}\mid x_{<i})$.

Figure 4: OOD detection performance as measured by AUROC using different
measures for binary sentiment classification based background shift, using
IMDB as ID data. We can see that using $\log p(x)$ as a measure is highly
noisy due to its dependency on sequence lengths.

While PPL accounts for varying sequence lengths by averaging word likelihoods
over the input sequence, $\log p(x)$ does not. Figure 4 shows that this
difference significantly impacts performance. With IMDB as the ID data, using
$\log p(x)$ fails for SST-2, achieving close to 100 FAR95 and near 0 AUROC. We
suspect this because IMDB examples are a full paragraph while SST-2 examples
are one to two sentences. $\log p(x)$ would naturally be smaller for IMDB
examples than these OOD examples, resulting in complete failure for simple
thresholding methods measured by AUROC.

##  Appendix B FAR95 Results

We additionally evaluate the performance for all experiments using FAR95,
which measures the false positive rate at 95% recall. In the context of OOD
detection, this measure gives the misclassification rate of ID data at 95%
recall of OOD classification, hence a lower value indicates better
performance.

ID | OOD | FAR95 ($\downarrow$)
---|---|---
PPL | MSP | Oracle
News Top-5 | News Rest | 88.5 | 75.7 | 80.4
DBPedia Top-4 | DBPedia Rest | 78.3 | 86.3 | 1.3
Table 5: FAR95 scores obtained using PPL, MSP and Oracle for semantic shifts, with lower score (among PPL/MSP) in bold. ID | OOD | FAR95 ($\downarrow$)
---|---|---
PPL | MSP | Oracle
SST-2 | IMDB | 8.6 | 76.5 | 0.0
Yelp | 5.2 | 83.0 | 0.0
IMDB | SST-2 | 17.0 | 47.7 | 0.2
Yelp | 70.2 | 82.6 | 0.0
Yelp | SST-2 | 3.1 | 45.4 | 1.1
IMDB | 36.2 | 90.4 | 0.0
SNLI | RTE | 19.1 | 61.4 | 0.7
MNLI | 14.7 | 62.5 | 0.3
RTE | SNLI | 62.5 | 95.3 | 0.0
MNLI | 64.3 | 93.9 | 10.3
MNLI | SNLI | 70.9 | 84.6 | 1.2
RTE | 93.2 | 69.8 | 6.2
Table 6: FAR95 scores obtained using PPL, MSP and Oracle for background shift
caused by shift in domain. For each pair, lower score obtained (by PPL or MSP)
is in bold.

ID | OOD | FAR95 ($\downarrow$)
---|---|---
PPL | MSP | Oracle
Fiction | Government | 57.4 | 95.0 | 9.7
Slate | 66.0 | 92.7 | 37.7
Telephone | 29.1 | 93.3 | 36.0
Travel | 58.0 | 93.3 | 10.0
Government | Fiction | 74.7 | 92.6 | 6.4
Slate | 70.7 | 92.1 | 13.7
Telephone | 35.2 | 95.5 | 6.2
Travel | 52.8 | 92.4 | 6.2
Slate | Fiction | 90.6 | 96.2 | 32.2
Government | 90.0 | 96.1 | 12.6
Telephone | 57.4 | 96.0 | 22.7
Travel | 83.3 | 95.8 | 16.8
Telephone | Fiction | 54.2 | 93.3 | 32.5
Government | 50.9 | 93.7 | 8.5
Slate | 49.6 | 91.1 | 36.3
Travel | 44.6 | 91.4 | 10.7
Travel | Fiction | 74.5 | 95.5 | 10.2
Government | 69.0 | 94.4 | 7.8
Slate | 75.9 | 93.8 | 16.8
Telephone | 30.3 | 93.7 | 9.5

Table 7: FAR95 scores obtained using PPL, MSP and Oracle for background shift
caused by shift in MNLI genre. For each pair, lower score obtained (by PPL or
MSP) is in bold.

Tables 5, 6, 7 and 8 show the results obtained using FAR95 as a metric for the
corresponding ID/OOD pairs used earlier. We observe that FAR95 results are in
line with AUROC results except for DBPedia, in which case density estimation
methods yield a better result. The difference may be a result of the
accumulative nature of AUROC in contrast to FAR95, which is a point
measurement.

ID | OOD | Shift | FAR95 ($\downarrow$)
---|---|---|---
PPL | MSP | Oracle
IMDB | c-IMDB | Semantic | 93.1 | 82.8 | 69.3
MNLI | HANS | Background | 4.2 | 73.1 | 0.0
Negation | Background | 94.9 | 93.5 | 0.1
Len. Mismatch | Background | 98.3 | 95.0 | 0.1
Spell. Error | Background | 96.9 | 92.4 | 3.0
Word Overlap | Background | 96.0 | 94.4 | 1.1
Antonym | Semantic | 100.0 | 90.8 | 6.3
Num. Reas. | Semantic | 99.5 | 77.6 | 0.7

Table 8: FAR95 scores obtained using PPL, MSP and Oracle for challenge data.
The primary type of shift observed is indicated in the ’Shift’ column. Lower
score (among MSP/PPL) for each pair is in bold.

##  Appendix C Background shift in MNLI Genres

MNLI is a crowd-sourced collection of sentence pairs for textual entailment
sourced from 10 genres including Fiction, Government, Slate, Telephone, and
Travel. We use examples from these five MNLI genres and separately consider
each genre as ID and OOD, using the validation splits for evaluation.

Table 9 shows the results for MNLI genres. The discriminative model
generalizes well to other genres and we find that the OOD detection
performance of calibration method is close to random (50) because of the
higher confidence on correct OOD predictions by a well-calibrated model.

ID | OOD | AUROC | Accuracy
---|---|---|---
PPL | MSP | Oracle | OOD | ID
Fiction | Govt. | 83.3 | 48.5 | 98.4 | 87.0 | 86.1
Slate | 81.6 | 54.1 | 92.7 | 82.2
Tel. | 92.3 | 51.0 | 94.6 | 84.0
Travel | 82.2 | 49.9 | 98.3 | 84.3
Govt. | Fiction | 75.2 | 57.4 | 98.9 | 82.8 | 88.4
Slate | 77.1 | 58.3 | 97.7 | 82.0
Tel. | 89.9 | 57.6 | 98.5 | 82.8
Travel | 82.6 | 57.1 | 99.4 | 84.1
Slate | Fiction | 60.6 | 48.2 | 94.2 | 84.3 | 82.5
Govt. | 61.3 | 45.3 | 97.7 | 87.8
Tel. | 83.0 | 49.8 | 95.3 | 84.1
Travel | 63.7 | 46.8 | 97.6 | 84.6
Tel. | Fiction | 85.7 | 55.9 | 95.2 | 82.5 | 85.7
Govt. | 86.0 | 52.5 | 98.3 | 85.9
Slate | 86.8 | 59.2 | 94.2 | 80.6
Travel | 87.8 | 56.8 | 98.6 | 82.5
Travel | Fiction | 76.4 | 54.8 | 98.0 | 81.3 | 86.7
Govt. | 78.8 | 49.0 | 98.7 | 87.4
Slate | 77.2 | 55.8 | 96.3 | 80.8
Tel. | 92.7 | 56.0 | 98.1 | 82.2

Table 9: Performance on background shifts caused by shift in MNLI genre. For
each pair, higher score obtained (by PPL or MSP) is in bold. We can see that
the density estimation method using PPL significantly outperforms the
calibration method.


Cosmic acceleration in Regge-Teitelboim gravity

S<EMAIL_ADDRESS>A<EMAIL_ADDRESS>and Chuang
<EMAIL_ADDRESS>

Department of Physics, University of Alabama,

Tuscaloosa, Alabama 35487, USA

###### Abstract

The Regge-Teitelboim formulation of gravity, which utilizes dynamical
embeddings in a background space, effectively introduces source terms in the
standard Einstein equations that are not attributable to the energy-momentum
tensor. We show that for a simple class of embeddings of the Robertson-Walker
metric, these source terms naturally generate cosmic acceleration.

##  1 Introduction

Long ago, Regge and Teitelboim wrote down a formulation of gravity which
relies on embedding the space-time manifold in a higher dimensional
background, and regards the embedding coordinates, rather than the metric
tensor, as dynamical degrees of freedom, in the spirit of string theory.[1].
The theory was further examined in [2],[3],[4],[5],[6],[7]. In addition to its
relation with strings and membranes, the theory shares some similarities to
certain matrix models in the continuum limit, where the matrices reduce to
continuous variables which define embeddings of continuous manifolds in a
background space.[8] Dynamics in that case is governed by matrix actions (see
[9]), while for the Regge-Teitelboim (RT) formulation of gravity the dynamics
is governed by the Einstein-Hilbert action, expressed in terms of embedding
coordinates.

Many articles have been devoted to cosmological-type solutions to matrix
models (see for example [10], [11],[12],[13]), while not much attention has
been previously given to cosmological solutions to RT gravity. (For examples,
see[14],[15].) Further examination of the role of cosmological solutions to RT
gravity appears to be warranted. The RT formulation of gravity effectively
introduces source terms in the standard Einstein equations that are not
attributable to the energy-momentum tensor, suggesting that it is well suited
to as a candidate for a source for cosmic acceleration. These additional
contributions $T_{{}_{\rm RT}}^{\mu\nu}$ to the Einstein equations are due
solely to the embedding of the space-time manifold in the background geometry.
Moreover, $T_{{}_{\rm RT}}^{\mu\nu}$ is strongly dependent on the type of
embedding, and a number of constraints restrict its form. In fact, for some
embeddings, the constraints are too restrictive to allow for a nonvanishing
$T_{{}_{\rm RT}}^{\mu\nu}$. The example we consider in this article is the
embedding of the Robertson-Walker metric in a five-dimensional flat
background. (As indicated above, the embedding is not unique. See for example
[16].) The embedding we choose leads to modifications of the standard
Friedmann equations for $k=1$ and $-1$.§§§Embeddings of the Robertson-Walker
metric into higher dimensional spaces are known to induce corrections to the
Friedmann equations.[17] However, we get no such modification for the case of
$k=0$. Our embedding is distinct from the one used previously by Davidson[14],
and leads to a different modification of the standard Friedmann equations. The
system presented in this article has the advantage that it allows for exact
solutions in the case where the usual energy-momentum tensor vanishes.
Numerical solutions are found for the case where nonrelativistic matter and
radiation, as well as from $T_{{}_{\rm RT}}^{\mu\nu}$, are present. The $k=-1$
solutions describe an accelerating expansion at large times. Moreover,
solutions exist which exhibit a transition from a decelerating universe to an
accelerating one.

We review the RT formulation of gravity in section 2, and apply it to an
embedding of the Robertson-Walker metric (with $k=\pm 1$) in a five-
dimensional flat background in section 3. A modification of the standard
Friedmann equations is obtained, which is solved for various special cases.
Concluding remarks are given in section 4.

##  2 Regge-Teitelboim formulation of gravity

In the Regge-Teitelboim formulation of gravity the four-dimensional space-time
manifold $M$ is embedded in a flat background $\mathbb{R}^{m,n}$, and the
metric tensor $g_{\mu\nu}\,,\;\mu,\nu\cdots=0,...,3,$ on $M$ is induced from
the background metric $\eta_{ab}\,,\;a,b=0,...,m+n$. Upon denoting the
embedding functions by $Y^{a}(x)$, where $x^{\mu}$ are coordinates on $M$,

$g_{\mu\nu}=\eta_{ab}\partial_{\mu}Y^{a}\partial_{\nu}Y^{b}\;,$ (2.1)

$\partial_{\mu}$ denoting differentiation with respect to $x^{\mu}$. From
metric compatibility, $\nabla_{\lambda}g_{\mu\nu}=0$, one gets the
conditions[1]

$\eta_{ab}\partial_{\mu}Y^{a}\nabla_{\lambda}\partial_{\nu}Y^{b}=0$ (2.2)

The usual Einstein-Hilbert action

$S=S_{EH}+S_{source},\qquad S_{EH}=\frac{1}{16\pi
G}\int_{M}d^{4}x\sqrt{|g|}R\;,$ (2.3)

governs the dynamics of RT gravity, with the caveat that the dynamical degrees
of freedom in this case are the embedding functions, rather than the metric
tensor. Thus $R$ in (2.3) is the Riemann curvature scalar expressed as a
function of $Y^{a}$. The field equations resulting from extremizing the action
with respect to $Y^{a}$ are

$\nabla_{\mu}\Bigl{(}\left(G^{\mu\nu}-8\pi
G\,T^{\mu\nu}\right)\partial_{\nu}Y^{a}\Bigr{)}=0\;,$ (2.4)

where $G^{\mu\nu}$ is the Einstein tensor, and $T^{\mu\nu}$ is the energy-
momentum tensor that arises from the variations of $S_{source}$. $T^{\mu\nu}$
is covariantly conserved, provided that the metric tensor is nonsingular. This
is a consequence of the conditions (2.2). Applying the Bianchi identities to
(2.4) gives

$(G^{\mu\nu}-8\pi GT^{\mu\nu})\nabla_{\mu}\,\partial_{\nu}Y^{a}=8\pi
G\nabla_{\mu}T^{\mu\nu}\,\partial_{\nu}Y^{a}$ (2.5)

Then after contracting the left- and right-hand sides with
$\partial_{\lambda}Y_{a}$, and applying (2.2), one gets

$\nabla_{\mu}T^{\mu\nu}g_{\lambda\nu}=0$ (2.6)

The equations of motion (2.4) can be expressed as current conservation laws on
$M$

$\partial_{\mu}{\cal J}^{\mu\,a}=0\;,\qquad{\cal
J}^{\mu\,a}=\sqrt{|g|}(G^{\mu\nu}-8\pi G\,T^{\mu\nu})\partial_{\nu}Y^{a}\;,$
(2.7)

The currents effectively introduce additional source terms in the Einstein
equations

$G^{\mu\nu}=8\pi G\,(T^{\mu\nu}+T_{{}_{\rm RT}}^{\mu\nu})\;,$ (2.8)
$T_{{}_{\rm RT}}^{\mu\nu}=\frac{1}{16\pi G\sqrt{|g|}}\eta_{ab}\,({\cal
J}^{\mu\,a}\partial^{\nu}Y^{b}+{\cal J}^{\nu\,a}\partial^{\mu}Y^{b})$ (2.9)

Obviously, $T_{{}_{\rm RT}}^{\mu\nu}$ is covariantly conserved since
$T^{\mu\nu}$ is, the latter following from the equation of motion (2.7). On
the other hand, the converse is not necessarily true. Given
$\nabla_{\mu}T_{{}_{\rm RT}}^{\mu\nu}=0$, we only have

$\partial_{\mu}{\cal J}^{\mu\,a}=8\pi G\,\sqrt{|g|}\,T_{{}_{\rm
RT}}^{\mu\nu}\nabla_{\mu}\partial_{\nu}Y^{a}\;,$ (2.10)

The right hand side of (2.10) is required to vanish according to the RT
equations of motion, thus constraining $T_{{}_{\rm RT}}^{\mu\nu}$. Symmetry
considerations put further restrictions on $T_{{}_{\rm RT}}^{\mu\nu}$, which
can severely restrict the form of these source terms. In fact, for many
embeddings the constraints do not allow for a nonvanishing $T_{{}_{\rm
RT}}^{\mu\nu}$. In the example which follows we examine an embedding of the
Robertson-Walker metric, and as a result of the above constraints, the
expression for $T_{{}_{\rm RT}}^{\mu\nu}$ is uniquely determined up to an
overall constant.

##  3 Application to Cosmology

Starting with the Robertson-Walker metric

$ds^{2}=-dt^{2}+a(t)^{2}\Big{(}\frac{dr^{2}}{1-kr^{2}}+r^{2}(d\theta^{2}+\sin^{2}\theta
d\phi^{2})\Bigr{)}\;,$ (3.1)

describing maximally isometric $t$-slices, we introduce an energy-momentum
tensor $T^{\mu\nu}$, as well as the additional source terms $T_{{}_{\rm
RT}}^{\mu\nu}$. Before writing down the embedding, we write down the expected
modification of the Friedmann equations, and mention the constraints which
restrict the form of $T_{{}_{\rm RT}}^{\mu\nu}$. As usual, we assume that
$T^{\mu\nu}$ is associated with a perfect fluid. It is natural to extend this
assumption to the additional source term $T_{{}_{\rm RT}}^{\mu\nu}$ as well,
and further that both $T^{\mu\nu}$ and $T_{{}_{\rm RT}}^{\mu\nu}$ have a
common co-moving frame, as this will preserve the maximal isometry. Thus

$\displaystyle T^{00}=\rho(t)$ $\displaystyle T^{ij}=p(t)g^{ij}$ (3.2)
$\displaystyle T_{{}_{\rm RT}}^{00}=\rho_{{}_{\rm RT}}(t)$ $\displaystyle
T_{{}_{\rm RT}}^{ij}=p_{{}_{\rm RT}}(t)g^{ij}\;,$ (3.4)

with all other components vanishing. $i,j,...$ denote spatial indices, and $0$
is the time index. As these tensors are required to be covariantly conserved,

$\displaystyle\dot{\rho}+3\frac{\dot{a}}{a}(\rho+p)$ $\displaystyle=$
$\displaystyle 0$ (3.5) $\displaystyle\dot{\rho}_{{}_{\rm
RT}}+3\frac{\dot{a}}{a}(\rho_{{}_{\rm RT}}+p_{{}_{\rm RT}})$ $\displaystyle=$
$\displaystyle 0\;,$ (3.7)

the dot denoting a $t-$derivative. Substituting (3.2) and (3.4) into (2.8)
leads to the following Friedmann equations for $a(t)$

$\Bigl{(}\frac{\dot{a}}{a}\Bigr{)}^{2}=\frac{8\pi G}{3}(\rho+\rho_{{}_{\rm
RT}})-\frac{k}{a^{2}}$ (3.8) $\frac{\ddot{a}}{a}=-\frac{4\pi
G}{3}\Bigl{(}\rho+\rho_{{}_{\rm RT}}+3(p+p_{{}_{\rm RT}})\Bigr{)}\;,$ (3.9)

Given some embedding $Y^{a}(t,x^{i})$, $a=0,1,...{D-1}$ ($x^{i}$ denoting
spatial coordinates) of the space-time manifold in a $D-$dimensional
background, the currents ${\cal J}^{\mu\,a}$ in (2.7) have the form

$\displaystyle{\cal J}^{0\,a}$ $\displaystyle=$ $\displaystyle 8\pi
G\,\sqrt{|g|}\,\rho_{{}_{\rm RT}}(t)\partial_{0}Y^{a}$ (3.10)
$\displaystyle{\cal J}^{i\,a}$ $\displaystyle=$ $\displaystyle 8\pi
G\,\sqrt{|g|}\,p_{{}_{\rm RT}}(t)g^{ij}\partial_{j}Y^{a}\;,$ (3.12)

The requirement of current conservation leads to the following constraints on
$\rho_{{}_{\rm RT}}(t)$ and $p_{{}_{\rm RT}}(t)$

$\rho_{{}_{\rm RT}}(t)\nabla_{0}\partial_{0}Y^{a}+p_{{}_{\rm
RT}}(t)g^{ij}\nabla_{i}\partial_{j}Y^{a}=0\;,$ (3.13)

which depends on the choice of the embedding.

Next we shall pick an embedding and this will lead to a particular form for
$\rho_{{}_{\rm RT}}(t)$ and $p_{{}_{\rm RT}}(t)$. We consider the embedding of
the Robertson-Walker metric with $k=\pm 1$ in a five-dimensional flat
background, and make the following choice[18]¶¶¶The choice is not unique.
Another embedding in five dimensions is applied in [14], which leads to a
different modification of the Friedmann equations.

$\begin{pmatrix}Y^{0}\\\ Y^{1}\\\ Y^{2}\\\ Y^{3}\\\
Y^{4}\end{pmatrix}=\begin{pmatrix}b_{k}(t)\\\ a(t)r\sin\theta\sin\phi\\\
a(t)r\sin\theta\cos\phi\\\ a(t)r\cos\theta\\\
a(t)\sqrt{1-kr^{2}}\end{pmatrix}\;,$ (3.14)

where $0<r<1$ for $k=1$ and $0<r<\infty$ for $k=-1$, along with
$0\leq\theta\leq\pi$, $0\leq\phi<\pi$. For the background metric take
$(-,+,+,+,k)$. The background space is therefore $\mathbb{R}^{4,1}$ for $k=1$,
and $\mathbb{R}^{3,2}$ for $k=-1$. In order to recover the Robertson-Walker
induced metric one needs to impose that $\dot{b}_{k}=\sqrt{1+k\dot{a}^{2}}$,
making $b_{k}(t)$ nonlocal in time. Upon substituting directly into the
equations of motion (2.4), and assuming that the stress-energy tensor has the
form (3.2), one gets

$k\dot{a}\dot{b}_{k}^{4}+3a\dot{a}\ddot{a}\dot{b}_{k}^{2}=\frac{8\pi
G}{3}a^{2}\left(ka\dot{a}\ddot{a}\rho+a\dot{b}_{k}^{2}\dot{\rho}+3\dot{a}\dot{b}_{k}^{2}\rho\right)\;,$
(3.15)

along with the conservation equation (3.5). Integration of (3.15) gives

$ka\dot{b}_{k}^{3}=\frac{8\pi G}{3}(a^{3}\dot{b}_{k}\rho+c_{0})\;,$ (3.16)

where $c_{0}$ is an integration constant. It can be rewritten as

$\frac{\dot{a}^{2}}{a^{2}}=\frac{8\pi
G}{3}\Bigl{(}\rho+\frac{c_{0}}{a^{3}\sqrt{1+k\dot{a}^{2}}}\Bigr{)}-\frac{k}{a^{2}}\;,$
(3.17)

We thus recover the standard Friedmann dynamics when $c_{0}=0$. More
generally, upon comparing (3.17) with (3.8) we get that

$\rho_{{}_{\rm RT}}=\frac{c_{0}}{a^{3}\sqrt{1+k\dot{a}^{2}}}$ (3.18)

$p_{{}_{\rm RT}}$ can then be obtained from the conservation equation (3.7)

$p_{{}_{\rm RT}}=-\frac{a}{3\dot{a}}\dot{\rho}_{{}_{\rm RT}}-\rho_{{}_{\rm
RT}}=\frac{kc_{0}\ddot{a}}{3a^{2}\left(1+k\dot{a}^{2}\right)^{3/2}}$ (3.19)

Combining (3.18) and (3.19) gives the analogue of an equation of state for
$T_{{}_{\rm RT}}^{\mu\nu}$:

$p_{{}_{\rm RT}}=\frac{ka\ddot{a}}{3(1+k\dot{a}^{2})}\rho_{{}_{\rm RT}}$
(3.20)

The quantity in parenthesis in (3.17), times $a^{2}$, can be regarded as minus
an effective potential, $V_{eff}(a)$. For this to make sense, the second term
in parenthesis should be evaluated on-shell, thereby making $\dot{a}$ a
function of $a$. It gives the following contribution to the effective force
$-\frac{d}{da}V_{eff}$:

$-\frac{8\pi
Gc_{0}}{3a^{2}(1+k\dot{a}^{2})^{3/2}}\Big{[}1+k\dot{a}^{2}+ka\dot{a}\frac{d\dot{a}}{da}\Big{]}$
(3.21)

This can be considered to be a ‘dark force’, in the sense that it does not
originate from the energy momentum tensor. Provided that the quantity in
brackets is positive, it follows that this is an attractive force when
$c_{0}>0$, and a repulsive force when $c_{0}<0$.

It is usual to relate the spatial geometry, i.e., $k$, to the density
parameter $\Omega=\rho/\rho_{c}$, $\rho_{c}=3H^{2}/8\pi G$ being the critical
density. From eq. (3.17) we get

$\frac{k}{a^{2}}=H^{2}\Bigl{(}\Omega-1+\frac{{\tilde{c}_{0}}}{H^{2}a^{3}\sqrt{1+ka^{2}H^{2}}}\Bigr{)}\;,$
(3.22)

where $H=\dot{a}/a$ is the Hubble parameter and ${\tilde{c}_{0}}=\frac{8\pi
Gc_{0}}{3}$. When ${\tilde{c}_{0}}$ is zero, we recover the usual result, that
$k=-1,1$ corresponds to $\Omega<1,\Omega>1$, respectively. [Recall, $k=0$ is
not valid for the embedding (3.14).] We can make the same argument when
${\tilde{c}_{0}}\neq 0$, provided we replace the density parameter $\Omega$ by

$\Omega^{\prime}=\Omega+\frac{{\tilde{c}_{0}}}{H^{2}a^{3}\sqrt{1+ka^{2}H^{2}}}$
(3.23)

Let us assume the standard equation of state relation for $p$ and $\rho$

$p=w\rho\;$ (3.24)

Then from (3.5) it follows that $\frac{d}{da}(a^{3}\rho)=-3wa^{2}\rho$, and
one obtains the usual relation between the density and the scale factor

$\rho=c_{1}a^{-3(w+1)}\;,$ (3.25)

$c_{1}$ being another integration constant. (3.17) then becomes

${ka}(1+k{\dot{a}^{2}})=\frac{\tilde{c}_{1}}{a^{3w}}+\frac{\tilde{c}_{0}}{\sqrt{1+k\dot{a}^{2}}}\;,$
(3.26)

where ${\tilde{c}_{m}}=\frac{8\pi G}{3}c_{m}$, $m=0,1$, from which we obtain
the evolution of the scale factor. The first term on the right hand side
represents the usual matter/energy contribution, while the second term is a
result of the embedding. For $k=-1$, the second term in (3.26) is significant
when $\dot{a}$ approaches $1$. Below we examine the $k=-1$ solutions for three
cases: the absence of a normal matter/energy contribution discussed in 3.1, a
nonrelativistic matter component and a radiation component, discussed,
respectively, in 3.2 and 3.3. Exact solutions are found for 3.1, while
numerical solutions can be obtained for 3.2 and 3.3. Due to the square root in
the second term on the right hand side of (3.26), we get the restriction that
$\dot{a}^{2}<1$ when $k=-1$ (at least, if we restrict to real values of
$\tilde{c}_{0}$), and so an inflationary phase scenario for $k=-1$ appears to
be problematic in this model.

###  3.1 $\tilde{c}_{1}=0$

In the absence of a matter/energy source, (3.26) reduces to

$k{\dot{a}^{2}}=\Bigl{(}\frac{k\tilde{c}_{0}}{a}\Bigr{)}^{2/3}-1\;,$ (3.27)

where we need to restrict $\tilde{c}_{0}>0$ for $k=1$, and $\tilde{c}_{0}<0$
for $k=-1$ in order for $a(t)$ to be real. The acceleration of the scale
factor in this case is simply

$\ddot{a}=-\frac{k(k\tilde{c}_{0})^{2/3}}{3a^{5/3}}\;,$ (3.28)

which tends to zero for large $a$. It is negative for $k=1$ and positive for
$k=-1$. Substitution of (3.27) and (3.28) into (3.20) leads to a fixed
equation of state for $T_{{}_{\rm RT}}^{\mu\nu}$: $p_{{}_{\rm
RT}}=-\frac{1}{9}\rho_{{}_{\rm RT}}$. (This result was recovered in a certain
limit in [14].) Note that for $k=-1$, $p_{{}_{\rm RT}}>0$ and $\rho_{{}_{\rm
RT}}<0$.

Integration of (3.27) gives

$\displaystyle t$ $\displaystyle=$ $\displaystyle
2\tilde{c}_{0}-a\Biggl{(}2\Bigl{(}\frac{\tilde{c}_{0}}{a}\Bigr{)}^{2/3}+1\Biggr{)}\sqrt{\Bigl{(}\frac{\tilde{c}_{0}}{a}\Bigr{)}^{2/3}-1}\;,\qquad\;0\leq
t\leq 2\tilde{c}_{0}\;,\qquad k=1$ (3.29) $\displaystyle t$ $\displaystyle=$
$\displaystyle
a\Biggl{(}2\Bigl{(}\frac{-\tilde{c}_{0}}{a}\Bigr{)}^{2/3}+1\Biggr{)}\sqrt{1-\Bigl{(}\frac{-\tilde{c}_{0}}{a}\Bigr{)}^{2/3}}\;,\qquad\quad\;\,t\geq
0\;,\qquad\qquad k=-1\;,$ (3.31)

where we fixed the time origin in both cases in order for $a$ to be a minimum
at $t=0$. The results are plotted in figure 1. As usual, $k=1$ and $k=-1$
correspond, respectively, to closed and open universes. The $k=1$ solution
yields a singularity at $t=0$, and $a$ reaches its maximum value of
$|\tilde{c}_{0}|$ at $t=2|\tilde{c}_{0}|$. For the $k=-1$ solution, $a$ has a
nonzero minimum value equal to $|\tilde{c}_{0}|$ at $t=0$, and its
continuation to $t<0$ describes a bouncing cosmology. In this case, $\dot{a}$
is zero at $t=0$, while it approaches a constant value at large $t$, which is
consistent with an accelerating cosmology. We will focus on accelerating
cosmologies in the following two cases and hence set $k=-1$.

Figure 1: Plot of $t$ as a function of $|a/\tilde{c}_{0}|$ when
$\tilde{c}_{1}=0$ for the cases $k=1$ and $k=-1$.

###  3.2 $w=0$ and $k=-1$

This case includes a nonrelativistic matter component for $k=-1$, in addition
to the RT source. Perturbations around the previous solution with
$\tilde{c}_{1}=0,\;k=-1$ are obtained for $c_{0}$ real and negative.
Furthermore, eq. (3.26) allows for the initial conditions $\dot{a}=0$ and
$\ddot{a}>0$ at $t=0$ provided that $-\tilde{c}_{0}-\tilde{c}_{1}=a(0)\geq 0$.
The minimum value of the scale parameter, $a(0)$, can therefore be nonzero
when $-\tilde{c}_{0}>\tilde{c}_{1}$, again describing a bouncing cosmology. An
example, $\tilde{c}_{0}=-\frac{2}{3},\tilde{c}_{1}=\frac{1}{3}$, is plotted in
figure 2, and it has $\ddot{a}>0$. The ratio $p_{{}_{\rm RT}}/\rho_{{}_{\rm
RT}}$, evaluated on-shell, is not a constant for this case.

The previous solution, $\tilde{c}_{0}=-1\,,\tilde{c}_{1}=0$, as well as
$\tilde{c}_{0}=0,\tilde{c}_{1}=1$ (corresponding to the absence of the RT
source term), are shown in figure 2 for comparison purposes. The
$\tilde{c}_{0}=0,\tilde{c}_{1}=1$ solution has $\dot{a}>1$, and so, from eq.
(3.26), in order to perturb around that solution one would need that
$\tilde{c}_{0}$ is purely imaginary. We find that these perturbations do not
alter the $\tilde{c}_{0}=0,\tilde{c}_{1}=1$ solution significantly, and
moreover do not appear to exhibit a positive acceleration.

Figure 2: Plot of $t$ as a function of $a$ for $k=-1$, $w=0$ for the cases:
$\;i)\;\tilde{c}_{0}=-1,\tilde{c}_{1}=0\;$,
$\;ii)\;\tilde{c}_{0}=-\frac{2}{3},\tilde{c}_{1}=\frac{1}{3}\;$ and
$\;iii)\;\tilde{c}_{0}=0,\tilde{c}_{1}=1$.

###  3.3 $w=\frac{1}{3}$ and $k=-1$

This case includes a radiation component for $k=-1$, along with the Regge-
Teitelboim source. Once again, perturbations around the solution with
$\tilde{c}_{1}=0,\;k=-1$ are obtained for $c_{0}$ real and negative. Two kinds
of initial conditions can be chosen: 1) $\dot{a}=0$ and $\ddot{a}>0$ at $t=0$
or 2) $a(0)=0$. From eq. (3.26), 1) is possible when
$\frac{1}{2}(-\tilde{c}_{0}\pm\sqrt{\tilde{c}_{0}^{2}-4\tilde{c}_{1}})=a(0)\geq
0$. $a(0)$ then corresponds to a minimum, leading to a bouncing cosmology. 2)
describes a singularity. Two nontrivial solutions are plotted in figure 3
$\;i)\;\tilde{c}_{0}=-.85,\tilde{c}_{1}=.15\;$, and
$\;ii)\;\tilde{c}_{0}=-.8,\tilde{c}_{1}=.2\;$ $a$ has a nonzero minimum value
for the former, while $a(0)=0$ for the latter. $\ddot{a}$ is positive for all
$t$ for $\;i)$. On the other hand, for case $\;ii)$, starting from a value of
$\dot{a}\approx 1$ in the limit $t\rightarrow 0$, the system initially
decelerates, and then makes a transition to an accelerating phase
($\ddot{a}>0$) at a finite time $t$. More generally, from (3.26), such a
transition occurs when $\dot{a}^{2}+\frac{\tilde{c}_{1}}{a^{2}}=1$.

Plots for $\;iii)\;\tilde{c}_{0}=-1,\tilde{c}_{1}=0\;$, and
$\;iv)\;\tilde{c}_{0}=0,\tilde{c}_{1}=1$ are included in figure 3 for
comparison purposes. As in the previous subsection, the
$\tilde{c}_{0}=0,\tilde{c}_{1}=1$ solution has $\dot{a}>1$, and so, from eq.
(3.26), perturbations around that solution are not possible for real values of
$\tilde{c}_{0}$.

Figure 3: Plot of $t$ as a function of $a$ for $k=-1$, $w=\frac{1}{3}$ for the
cases:
$\;i)\;\tilde{c}_{0}=-.85,\tilde{c}_{1}=.15\;$,
$\;ii)\;\tilde{c}_{0}=-.8,\tilde{c}_{1}=.2\;,$
$\;iii)\;\tilde{c}_{0}=-1,\tilde{c}_{1}=0\;$, and
$\;iv)\;\tilde{c}_{0}=0,\tilde{c}_{1}=1$.

##  4  Concluding remarks

The introduction of the RT source term in the Einstein equations leads to the
modification of the Friedmann equations (3.26). The modification can be
interpreted as a ‘dark force’, as it does not originate from the energy-
momentum tensor. For the choice of embedding used in this paper, this force
was given by (3.21). The modification of the Friedmann equations also gives an
additional contribution to the density parameter (3.23). Exact solutions were
found for the case where the usual energy-momentum tensor vanishes, which
describe an accelerating universe when $k=-1$. This result persisted when a
nonrelativistic matter component or radiation component was included.
Moreover, for the case of a radiation component, there are $k=-1$ solutions
which undergo a transition from a decelerating universe to an accelerating
one.

Our results rely on a particular choice for the embedding[18]. It remains to
be seen how robust the results are with respect to other choices of
embeddings. Different ‘dark’ source terms $T_{{}_{\rm RT}}^{\mu\nu}$ will
result from different choices of the embedding. These source terms are highly
constrained by the current conservation conditions in (2.10), and by symmetry.
For the case of the Robertson-Walker metric, with sources having the form in
(3.4), the constraints are given by (3.13). For some embeddings, the
conditions are too strong to allow for any modification of the Einstein
equations. This is the case for the embedding of the $k=0$ Robertson-Walker
metric in [18]. As $k=0$ is of physical relevance, it is worthwhile to do an
extensive search for embeddings that produce analogous ‘dark’ contributions to
the Friedmann equations. Another possible extension of this work is to relax
the requirement that the background space is geometrically flat, and perhaps
only demanding that it be Ricci flat.[17],[19] Of course, it is of interest to
work towards constructing a more realistic cosmological model, one for
example, that includes different matter-energy components, and combines the
radiation and nonrelativistic matter eras. We plan to address these and other
issues in the future.

Acknowledgment

A.S. is grateful for discussions with A. Pinzul.

## References

  * [1] T. Regge and C. Teitelboim, “General Relativity \‘a la string: a progress report,” in Proceedings of the First Marcel Grossmann Meeting (Trieste, Italy,1975), 77-87, ed. by R. Ruffini, North-Holland, Amsterdam, 1977; [arXiv:1612.05256 [hep-th]].
  * [2] S. Deser, F. A. E. Pirani and D. C. Robinson, “Imbedding the G-String,” Phys. Rev. D 14, 3301-3303 (1976).
  * [3] M. Pavsic, “On the Quantization of Gravity by Embedding Space-time in a Higher Dimensional Space,” Class. Quant. Grav. 2, 869 (1985).
  * [4] V. Tapia, “GRAVITATION A LA STRING,” Class. Quant. Grav. 6, L49 (1989).
  * [5] M. D. Maia, “On the Integrability Conditions for Extended Objects,” Class. Quant. Grav. 6, 173-183 (1989).
  * [6] I. A. Bandos, “String - like description of gravity and possible applications for F theory,” Mod. Phys. Lett. A 12, 799-810 (1997).
  * [7] R. Capovilla, A. Escalante, J. Guven and E. Rojas, “Hamiltonian dynamics of extended objects: Regge-Teitelboim model,” [arXiv:gr-qc/0603126 [gr-qc]].
  * [8] H. Steinacker, “Emergent Geometry and Gravity from Matrix Models: an Introduction,” Class. Quant. Grav. 27, 133001 (2010).
  * [9] D. N. Blaschke and H. Steinacker, “Curvature and Gravity Actions for Matrix Models,” Class. Quant. Grav. 27, 165010 (2010).
  * [10] D. Z. Freedman, G. W. Gibbons and M. Schnabl, “Matrix cosmology,” AIP Conf. Proc. 743, no.1, 286-297 (2004) doi:10.1063/1.1848334 [arXiv:hep-th/0411119 [hep-th]].
  * [11] D. Klammer and H. Steinacker, “Cosmological solutions of emergent noncommutative gravity,” Phys. Rev. Lett. 102, 221301 (2009).
  * [12] S. W. Kim, J. Nishimura and A. Tsuchiya, “Expanding (3+1)-dimensional universe from a Lorentzian matrix model for superstring theory in (9+1)-dimensions,” Phys. Rev. Lett. 108, 011601 (2012);S. W. Kim, J. Nishimura and A. Tsuchiya, “Expanding universe as a classical solution in the Lorentzian matrix model for nonperturbative superstring theory,” Phys. Rev. D 86, 027901 (2012).
  * [13] A. Stern, “Matrix Model Cosmology in Two Space-time Dimensions,” Phys. Rev. D 90, no.12, 124056 (2014); A. Chaney, L. Lu and A. Stern, “Matrix Model Approach to Cosmology,” Phys. Rev. D 93, no.6, 064074 (2016); A. Stern and C. Xu, “Signature change in matrix model solutions,” Phys. Rev. D 98, no.8, 086015 (2018).
  * [14] A. Davidson, “Lambda = 0 cosmology of a brane - like universe,” Class. Quant. Grav. 16, 653-659 (1999).
  * [15] R. Cordero, M. Cruz, A. Molgado and E. Rojas, “Quantum modified Regge-Teitelboim cosmology,” Gen. Rel. Grav. 46, 1761 (2014) [erratum: Gen. Rel. Grav. 46, 1770 (2014)].
  * [16] S. A. Paston and A. A. Sheykin, “Embeddings for solutions of Einstein equations,” Theor. Math. Phys. 175, 806-815 (2013).
  * [17] P. S. Wesson, “A Physical Interpretation of Kaluza-Klein Cosmology,” Astrophys. J. 394, 19-24 (1992).
  * [18] J. Rosen, “Embedding of Various Relativistic Riemannian Spaces in Pseudo-Euclidean Spaces,” Rev. Mod. Phys. 37, 204 (1965).
  * [19] C. Romero, R. K. Tavakol and R. Zalaletdinov, “The embedding of general relativity in five-dimensions,” Gen. Rel. Grav. 28, 365-376 (1996).