* [1] V.I. Arnol’d, Mathematical Methods of Classical Mechanics, Graduate Texts in Mathematics, 2nd ed. (Springer-Verlag, New York, 1989); L.D. Landau & E.M. Lifshitz, Mechanics (Volume 1 of Course of Theoretical Physics, Elsevier, Oxford 3rd Ed, 2005); H. Goldstein, C.P. Poole Jr. & J.L. Safko, Classical Mechanics (Addison Wesley, London, 3rd Ed., 2000); N.A. Lemos, Analytical Mechanics, (Oxford University Press, Cambridge, 2018).
  * [2] V.I. Arnol’d, Ordinary Differential Equations, (Springer-Verlag, New York, 1992).
  * [3] W. Paul, Electromagnetic traps for charged and neutral particles, Reviews of Modern Physics 62, 531 (1990); P.K. Ghosh, Ion Traps (Oxford University Press, New York, 1995).
  * [4] D. Leibfried & R. Blatt, C. Monroe, and D. Wineland, Quantum dynamics of single trapped ions, Rev. Mod. Phys. 75, 281 (2003).
  * [5] J.I. Cirac & P. Zoller, Quantum Computations with Cold Trapped Ions, Physical Review Letters 74, 4091 (1995).
  * [6] F. Nicacio, K. Furuya, & F.L. Semião, Motional entanglement with trapped ions and a nanomechanical resonator, Physical Review A 88, 022330 (2013); arXiv:1212.0711 [quant-ph] (2013).
  * [7] If $\det{\bf U}\neq 0$, the solution for $q_{\star}$ is unique and given by $q_{\star}=q_{0}-\tfrac{C}{d}{\bf U}\left(\begin{smallmatrix}1\\\ -1\end{smallmatrix}\right)$; otherwise, there can be multiple equilibrium points $q_{\star}$.
  * [8] Implementing the point transformation $q^{\prime}=q-q_{\star}$ in (SM-43), this Lagragian attains (SM-25) with ${\bf T}=m{\bf I}_{2}$. The potential matrix in (SM-26) is thus $\tilde{\bf U}=m^{-1}\bf U$ and $\tilde{\bf O}$ is the diagonalizing matrix of $\bf U$.
  * [9] Squeezing is a property related to the variances of measurements, see Eq.(81). Suppose that there is a quantum state such that $\langle\Delta\hat{q}_{j}^{2}\rangle\langle\Delta\hat{p}_{j}^{2}\rangle=c$, where of course $c\geq\hbar^{2}/4$, see Eq.(84). A new state is said squeezed with respect to the former if one of the variances is increased while the other is decreased by the same factor, that is, the new variances are such that $\langle\Delta\hat{q}_{j}^{\prime 2}\rangle=s^{2}\langle\Delta\hat{q}_{j}^{2}\rangle$ and $\langle\Delta\hat{p}_{j}^{\prime 2}\rangle=s^{-2}\langle\Delta\hat{p}_{j}^{2}\rangle$. In a squeezed state, measurements of one variable will have a sharper distribution, while the one for the conjugate variable will be broader, however their product is left unchanged, $\langle\Delta\hat{q}_{j}^{\prime 2}\rangle\langle\Delta\hat{p}_{j}^{\prime 2}\rangle=c$. For the quantum electromagnetic field, position and momentum are called quadrature10 and are identified by relations (59) and (61). The mentioned squeezing effect is generated by a Hamiltonian like $\hat{H}_{1}$ in Eq.(SM-55), while $\hat{H}_{2}$ generates the same effect but taking into account different fields 10.
  * [10] M. Scully & M. Zubairy, Quantum Optics (Cambridge University Press, Cambridge, 1997); D.F. Walls & G.J. Milburn, Quantum Optics (Springer-Verlag, Berlin, 2nd ed. 2008); W.P. Schleich, Quantum Optics in Phase Space (Wiley-VCH Verlag, Berlin, 2001); G. Grynberg, A. Aspect & C. Fabre, Introduction to Quantum Optics — From the Semi-classical Approach to Quantized Light (Cambridge University Press, Cambridge, 2010).
  * [11] F. Nicacio & F.L. Semião, Coupled harmonic systems as quantum buses in thermal environments, Journal of Physics A 49, 375303 (2016); arXiv:1601.07528 [quant-ph](2016).
  * [12] J.J. Sakurai & J. Napolitano, Modern Quantum Mechanics (Addison-Wesley, Boston, 2nd Ed. 2011). L.E. Ballentine, Quantum Mechanics – A Modern Development (World Scientific, Singapore 2000); C.C.-Tannoudji, B. Diu & F. Laloë, Quantum Mechanics (Wiley-VCH, Singapore, 2005).
  * [13] L.D. Landau & E.M. Lifshitz, Statistical Physics Part 1, (Volume 5 of Course of Theoretical Physics, Pergamon Press, Oxford 3rd Ed, 1980); K. Huang, Statistical Mechanics (John Wiley & Sons, 2${}^{\text{nd}}$ Ed. 1987); R.K. Pathria & P.D. Beale, Statistical Mechanics (Elsevier Science, 1996).
  * [14] The transformation (SM-70) is not a congruence between $\bf V$ and $\tilde{\bf V}$ due to the matrices $\bf W$. By the same reason, it is not possible to ensure that ${\bf W}{\bf\Delta}{\bf W}$ is a positive-definite matrix, which forbids the statement ${\bf\Delta}\geq 0\Longleftrightarrow{\bf W}{\bf\Delta}{\bf W}\geq 0$, about (89).


#  SelfOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank

Dheeraj Mekala♠,♢ Adithya Samavedhi♠,♢ Chengyu Dong♢ Jingbo Shang♢,♡,∗
♢University of California San Diego
♡ Halıcıoğlu Data Science Institute, University of California San Diego
{dmekala, asamavedhi, cdong<EMAIL_ADDRESS>
###### Abstract

††$\spadesuit$ Equal Contribution††$*$ Jingbo Shang is the corresponding
author.

Deep neural classifiers trained with cross-entropy loss (CE loss) often suffer
from poor calibration, necessitating the task of out-of-distribution (OOD)
detection. Traditional supervised OOD detection methods require expensive
manual annotation of in-distribution and OOD samples. To address the
annotation bottleneck, we introduce SelfOOD, a self-supervised OOD detection
method that requires only in-distribution samples as supervision. We cast OOD
detection as an inter-document intra-label (IDIL) ranking problem and train
the classifier with our pairwise ranking loss, referred to as IDIL loss.
Specifically, given a set of in-distribution documents and their labels, for
each label, we train the classifier to rank the softmax scores of documents
belonging to that label to be higher than the scores of documents that belong
to other labels. Unlike CE loss, our IDIL loss function reaches zero when the
desired confidence ranking is achieved and gradients are backpropagated to
decrease probabilities associated with incorrect labels rather than
continuously increasing the probability of the correct label. Extensive
experiments with several classifiers on multiple classification datasets
demonstrate the effectiveness of our method in both coarse- and fine-grained
settings.

##  1 Introduction

Deep neural networks (DNN) are ubiquitously used for Text classification Liu
et al. (2019); Devlin et al. (2019); Yang et al. (2019); Brown et al. (2020).
However, they are generally poorly calibrated, resulting in erroneously high-
confidence scores for both in-distribution and out-of-distribution (OOD)
samples Szegedy et al. (2013); Nguyen et al. (2015); Guo et al. (2017); Mekala
et al. (2022). Such poor calibration makes DNNs unreliable and OOD detection
task vital for the safe deployment of deep learning models in safety-critical
applications Moon et al. (2020).

(a) CE Loss

(b) SelfOOD

Figure 1: CE Loss and SelfOOD optimization for two documents $D_{1},D_{2}$
belonging to Sports and Arts. CE loss increases the scores corresponding to
the Sports class for $D_{1}$ and Arts class for $D_{2}$, implying an intra-
document comparison. Instead, SelfOOD compares the softmax scores in an inter-
document intra-label fashion where it reduces the scores corresponding to
Sports class for $D_{2}$ to be less than that of $D_{1}$. Figure 2: SelfOOD is
a self-supervised framework that requires only annotated in-distribution data
to train the OOD classifier. Firstly, we bucket documents based on their
annotated label (in dotted lines). Then, we compare each document in a bucket
with all documents in other buckets to compute IDIL loss (in solid lines).
Finally, we backpropagate gradients to decrease scores associated with
incorrect labels during the training of the OOD classifier.

Traditional supervised OOD detection methods Hendrycks et al. (2018); Larson
et al. (2019); Kamath et al. (2020); Zeng et al. (2021b) assume access to
high-quality manually annotated in-distribution and OOD samples. However, this
requires extensive annotation of OOD samples belonging to diverse
distributions, which is expensive to obtain. Moreover, text classifiers are
ideally desired to be more confident on in-distribution samples than OOD
samples. However, the poor calibration of DNN precludes this phenomenon.

To address these problems, we propose SelfOOD, a self-supervised OOD detection
framework that requires only in-distribution samples as supervision. Deep
learning models are desired to be more confident in in-distribution samples
than OOD samples. To adhere to this constraint, we formulate OOD detection as
an inter-document intra-label (IDIL) ranking problem and train the classifier
using our pairwise ranking loss, referred to as IDIL loss. As shown in
Figure-1(a), text classifiers are generally trained using cross-entropy loss
(CE loss) Good (1952) in an intra-document fashion where for each document,
the classifier is trained to distinguish between different labels by
maximizing the score corresponding to the correct label. Instead, in our
method, as shown in Figure 1(b), we propose to train in an inter-document,
intra-label fashion where for each label, we train the model to rank the
considered label probability score in documents belonging to the label to be
higher compared to those not belonging to it. As OOD documents generally don’t
belong to any label, we hypothesize such explicit training to rank, translates
to accurately distinguishing OOD from in-distribution documents.

Moreover, minimizing CE loss involves continuous optimization to increase the
probability of the correct label over the other labels, making the classifier
overconfident Wei et al. (2022). Instead, in our method, our IDIL loss
function becomes zero once the desired ranking is achieved, and during
training, we backpropagate gradients to decrease probabilities associated with
incorrect labels rather than increasing the probability of the correct label.
Theoretically, the perfect model trained using the CE loss is a solution to
our ranking problem, however, empirically, we observe that our ranking
objective leads to a different solution, demonstrating the importance of the
optimization procedure. Finally, it is important to note that our ranking
formulation, loss function, and the self-supervised training strategy have
been specifically designed to improve the performance of OOD detection, rather
than classification accuracy.

We present our framework in Figure 2. Given a set of in-distribution documents
and corresponding labels as input, we bucket documents belonging to each label
and train the classifier to rank the probabilities using our IDIL loss
function. Specifically, for each document in a label bucket, we pair up with
all documents in every other bucket and compute IDIL loss.

Our contributions are summarized as follows:

  * •

We propose SelfOOD, a novel self-supervised method to train an OOD detection
model without any OOD samples as supervision.

  * •

We formulate OOD detection as an inter-document intra-label ranking problem
and optimize it using our IDIL ranking loss.

  * •

We perform extensive experiments on multiple text classification datasets to
demonstrate the effectiveness of our method in OOD detection.

  * •

We release the code on Github111https://github.com/dheeraj7596/SELFOOD.

##  2 Related Work

Traditional supervised methods cast the OOD detection as classification with
binary labels Kamath et al. (2020), one additional label for unseen classes
Fei and Liu (2016). Recent works have deviated from requiring extensive
labeled OOD data to leverage various distance metrics to detect OOD samples.
Leys et al. (2018); Xu et al. (2020) use Mahalanobis distance as a post-
processing technique to identify OOD samples. These methods use distance-based
scoring functions along with the intermediate model layer features to
determine an OOD score. Lee et al. (2018); Hsu et al. (2020) use a similar
distance metric called ODIN to detect OOD images. An alternate approach to
compensate for the lack of OOD training data involves generating pseudo-OOD
data for training. Ouyang et al. (2021) propose a framework to generate high-
quality OOD utterances and importance weights by selectively replacing phrases
in in-domain samples. Zhan et al. (2021) generate pseudo-OOD samples for the
task of intent detection using self-supervision. Zhou et al. (2021); Zeng et
al. (2021a) introduce self-supervised approaches to ood detection using a
contrastive learning framework. They suggest fine-tuning transformers using a
margin-based contrastive loss to learn text representations for OOD
classification. Vyas et al. (2018); Li et al. (2021) treat a part of in-domain
data as OOD samples as an alternate self-supervised approach. Further, Wu et
al. (2022) use a Reassigned Contrastive Loss (RCL) along with an adaptive
class-dependent threshold mechanism to separate in-domain and OOD intents. Ren
et al. (2019); Gangal et al. (2020) leverage likelihood ratios crafted by
generative models to classify OOD samples. Wei et al. (2022) observe that the
norm of the logits keeps increasing during training, leading to overconfident
outputs, and propose LogitNorm as a solution to decoupling the output norm
during training optimization. Moon et al. (2020) introduce a novel Correctness
Ranking Loss function in order to regularize output probabilities to produce
well-ranked confidence estimates. Other calibration techniques include "Top-
label" calibration which is used to regularize the reported probability for
the predicted class Gupta and Ramdas (2021).

##  3 SelfOOD: Self-Supervised OOD Detection

In this section, we first present the problem statement, then briefly discuss
the motivation of our ranking formulation for OOD detection, and finally
describe our method including the loss function and its optimization strategy.

###  3.1 Problem Statement

In this paper, we work on the Out-of-distribution detection task with only in-
distribution samples as supervision. Specifically, given a labeled dataset
$\mathcal{D}^{InD}=\\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots(x_{n},y_{n})\\}$
sampled from a distribution space $(\mathcal{X},\mathcal{C})$ where documents
$x_{i}\in\mathcal{X}$ and labels $y_{i}\in\mathcal{C}$ as input, our goal is
to train an OOD detector $\mathcal{M}$ that accurately distinguishes in-
distribution documents $\mathcal{D}^{InD}$ and OOD documents
$\mathcal{D}^{OOD}\notin(\mathcal{X},\mathcal{C})$ without any OOD documents
required for training.

###  3.2 Motivation

Numerous neural text classifiers have been proposed, incorporating multiple
hidden layers Rosenblatt (1957), convolutional layers Kim (2014), and various
types of attention mechanisms Devlin et al. (2019); Liu et al. (2019); Radford
et al. (2019). All these models culminate in a softmax head, which produces
probabilities corresponding to each class. These classifiers are generally
trained with CE loss in an intra-document fashion i.e. each document is
considered independently and the softmax score of the true label is maximized.
Such training of neural text classifiers is known to increase the magnitude of
logit vectors even when most training examples are correctly classified Wei et
al. (2022), making them poorly calibrated that produce unreasonably high
probabilities even for incorrect predictions Szegedy et al. (2013); Nguyen et
al. (2015); Guo et al. (2017). This diminishes their ability to maintain the
desired attribute of ordinal ranking for predictions based on confidence
levels, wherein a prediction exhibiting a higher confidence value should be
considered more likely to be accurate than one with a lower confidence value
Moon et al. (2020). Intuitively, a text classifier possessing such quality
would be a perfect OOD detector.

###  3.3 OOD Detection as Inter-Document Intra-Label Ranking

In order to align with the aforementioned characteristic, we propose
formulating the OOD detection as an inter-document intra-label ranking
problem. Specifically, given a set of in-distribution documents, we compare
across documents but within the same label and train our model to generate
higher probability score for documents belonging to the label than for
documents not belonging to the label. We consider the same model architecture
as any text classifier with a softmax head that generates scores corresponding
to each label, however, we train it using our IDIL loss instead of CE loss.
Our assumption is that an OOD document does not fall under any specific label
in the in-distribution space. Hence, we anticipate that the trained model
would produce lower scores for OOD documents compared to in-distribution
documents. This distinction in scores is expected to facilitate easy
separation between OOD and in-distribution documents.

IDIL Loss is a pairwise-ranking loss that enforces desired ordinal ranking of
confidence estimates. This loss function reaches its minimum value for a
particular label when the probability of that label being the annotated label
is greater than its probability when it is not the annotated label.
Specifically, for documents $x_{1},x_{2}\in\mathcal{D}^{InD}$ and their
corresponding annotated labels $y_{1},y_{2}$ where $y_{1}\neq y_{2}$, IDIL
loss corresponding to label $y_{1}$ is mathematically computed as follows:

$\mathcal{L}_{IDIL}(y_{1}|x_{1},x_{2})=SiLU(p(y_{1}|x_{2})-p(y_{1}|x_{1}))$

(1)

where $SiLU(x)=x\sigma(x)$ is the Sigmoid Linear Unit (SiLU) function Elfwing
et al. (2018). To ensure stable training and enhance performance, we
incorporate the SiLU function, a continuous variant of the Rectified Linear
Unit (ReLU) Hahnloser et al. (2000), in conjunction with the ranking loss. The
SiLU function introduces smooth gradients around zero, effectively mitigating
potential instability issues during training. We observe that this inclusion
contributes to the overall stability in training and improved performance of
the model as shown in Section 4.5. Note that, in contrast to CE loss, IDIL
loss becomes zero once the desired ranking is achieved, addressing the
overconfidence issue.

###  3.4 Implementation

Ideally, the loss has to be computed over all possible pairs of documents for
each model update. However, it is computationally expensive. Therefore,
following Toneva et al. (2019); Moon et al. (2020), we approximate the
computation by considering only documents in each mini-batch. Specifically, we
bucket the documents in the mini-batch based on their annotated label and pair
each document in a bucket with all documents in other buckets and compute the
loss. Mathematically, the loss for a mini-batch $b$ is computed as follows:

$\mathcal{L}=\sum_{l\in\mathcal{C}}\sum_{x_{1}\in b_{l}}\sum_{x_{2}\in b_{\neg
l}}\mathcal{L}_{IDIL}(l|x_{1},x_{2})$

where $b_{l}$ denotes the set of training data points $x$ in this batch $b$
whose label are $l$, and $b_{\neg l}$ denotes the set of training data points
$x$ in this batch $b$ whose label is _not_ $l$.

In contrast to CE loss, where the optimization involves increasing the score
corresponding to the correct label, we backpropagate gradients to decrease
scores associated with incorrect labels. Specifically, during the
backpropagation of gradients, we detach the gradients for the subtrahend of
the difference and exclusively propagate the gradients through the minuend. In
Equation 1, for instance, we detach the gradients for $p(y_{1}|x_{1})$ and
solely backpropagate the gradients through $p(y_{1}|x_{2})$. This detachment
allows for a more controlled and selective gradient flow, aiding in the
optimization process, and improvement in performance as shown in Section 4.5.

It is important to note that our optimization focuses solely on the inter-
document ranking loss. Consequently, while the trained model would serve as a
reliable OOD detector, it may not perform as effectively as a classifier.

##  4 Experiments

Dataset | Domain | Criteria | # Docs | # labels
---|---|---|---|---
NYT | News | Topic | 13081 | 26
Yelp | Reviews | Sentiment | 70000 | 5
Clima | Climate | Question Type | 17175 | 8
TREC | General | Question Type | 5952 | 6

Table 1: Dataset statistics.

|  |  | BERT | RoBERTa
---|---|---|---|---
In-dist | OOD | Method | FPR95($\downarrow$) | ERR($\downarrow$) | AUROC($\uparrow$) | AUPR($\uparrow$) | FPR95($\downarrow$) | ERR($\downarrow$) | AUROC($\uparrow$) | AUPR($\uparrow$)
Yelp | NYT | CE Loss | 82.1 | 34.9 | 63.4 | 50.7 | 79.8 | 32.9 | 66.8 | 58.0
SelfOOD | 63.2 | 19.6 | 79.4 | 82.7 | 69.5 | 21.6 | 72.6 | 78.8
| CRL | 99.9 | 36.9 | 43.9 | 51.1 | 99.8 | 28.1 | 64.8 | 60.5
Clima | CE Loss | 83.2 | 27.8 | 48.6 | 37.0 | 76.4 | 22.0 | 78.2 | 68.4
SelfOOD | 17.6 | 4.8 | 97.9 | 96.4 | 65.6 | 16.6 | 73.2 | 77.2
| CRL | 99.6 | 29.0 | 49.1 | 38.8 | 99.9 | 20.6 | 63.0 | 56.9
TREC | CE Loss | 74.2 | 33.7 | 52.8 | 62.7 | 58.5 | 27.5 | 75.3 | 78.3
SelfOOD | 0.0 | 0.0 | 100.0 | 100.0 | 55.1 | 20.6 | 80.2 | 90.3
| CRL | 33.3 | 15.5 | 66.7 | 77.2 | 100.0 | 37.3 | 56.7 | 73.9
NYT | Clima | CE Loss | 22.1 | 1.0 | 96.9 | 92.5 | 15.7 | 1.0 | 97.3 | 92.8
SelfOOD | 10.5 | 1.6 | 98.4 | 90.5 | 4.6 | 1.0 | 99.3 | 95.4
| CRL | 32.3 | 1.7 | 95.4 | 86.5 | 79.3 | 3.2 | 79.8 | 67.0
TREC | CE Loss | 14.5 | 5.8 | 97.0 | 87.1 | 21.9 | 6.7 | 95.5 | 83.5
SelfOOD | 0.0 | 0.0 | 100.0 | 100.0 | 0.0 | 0.0 | 100.0 | 100.0
| CRL | 67.9 | 6.5 | 89.9 | 79.1 | 85.6 | 9.6 | 71.4 | 61.4
Yelp | CE Loss | 3.4 | 0.7 | 99.2 | 82.3 | 1.8 | 0.6 | 99.3 | 85.4
SelfOOD | 22.5 | 1.1 | 95.7 | 57.8 | 0.2 | 0.2 | 99.9 | 97.5
| CRL | 69.5 | 0.5 | 89.3 | 77.8 | 73.3 | 1.0 | 83.8 | 58.3
Clima | TREC | CE Loss | 34.9 | 5.6 | 94.9 | 92.2 | 60.2 | 9.1 | 90.4 | 83.8
SelfOOD | 0.3 | 1.1 | 99.4 | 99.4 | 38.9 | 8.8 | 90.1 | 84.7
| CRL | 83.0 | 12.7 | 82.0 | 72.2 | 60.9 | 12.6 | 79.2 | 63.2
Yelp | CE Loss | 46.3 | 0.9 | 93.8 | 80.1 | 79.5 | 1.9 | 86.5 | 47.2
SelfOOD | 14.7 | 0.6 | 98.0 | 90.8 | 37.6 | 1.5 | 72.3 | 54.9
| CRL | 98.7 | 1.4 | 67.3 | 53.3 | 72.9 | 2.1 | 50.9 | 19.0
NYT | CE Loss | 44.9 | 4.8 | 93.2 | 84.4 | 62.5 | 6.7 | 89.6 | 75.2
SelfOOD | 17.2 | 2.6 | 97.8 | 94.5 | 55.3 | 6.7 | 68.6 | 61.5
| CRL | 85.6 | 7.6 | 78.0 | 62.5 | 61.3 | 9.4 | 61.8 | 39.9
TREC | Yelp | CE Loss | 27.2 | 0.3 | 94.9 | 71.2 | 11.3 | 0.4 | 96.9 | 62.3
SelfOOD | 0.0 | 0.0 | 100.0 | 100.0 | 0.0 | 0.0 | 100.0 | 100.0
| CRL | 96.6 | 0.7 | 70.9 | 16.5 | 99.8 | 0.7 | 24.3 | 9.7
NYT | CE Loss | 8.4 | 1.3 | 97.6 | 87.5 | 6.8 | 1.4 | 98.1 | 87.1
SelfOOD | 0.0 | 0.0 | 100.0 | 100.0 | 0.0 | 0.0 | 100.0 | 100.0
| CRL | 97.2 | 3.3 | 73.0 | 35.2 | 99.9 | 3.8 | 22.9 | 12.2
Clima | CE Loss | 15.1 | 1.7 | 96.2 | 69.5 | 12.2 | 1.8 | 96.4 | 67.8
SelfOOD | 0.0 | 0.0 | 100.0 | 100.0 | 0.0 | 0.0 | 100.0 | 100.0
| CRL | 86.3 | 2.2 | 80.7 | 39.5 | 98.9 | 2.7 | 30.1 | 13.0

Table 2: OOD detection results with BERT & RoBERTa classifiers. Each
experiment is repeated with three random seeds and the mean scores are
reported. The false-positive-rate at 95% true-positive-rate (FPR95), minimum
detection error over all thresholds (ERR), the area under the risk-coverage
curve (AURC), and the area under the precision-recall curve (AUPR) using in-
distribution samples as the positives are used as evaluation metrics.

We evaluate our OOD detection method against state-of-the-art baselines with
two classifiers on multiple datasets belonging to different domains. In this
section, we present our experimental settings, compared methods, and
performance.

###  4.1 Datasets

We evaluate our method and baselines on four publicly available English text
classification datasets belonging to different domains. In particular, we
consider the news topic classification dataset New York Times
(NYT)222http://developer.nytimes.com/, restaurant review sentiment
classification dataset Yelp333https://www.yelp.com/dataset/, and question-type
classification datasets related to climate: Clima-Insurance+ (Clima) Laud et
al. (2023), and a general domain: TREC Li and Roth (2002); Hovy et al. (2001).
The documents within the New York Times dataset are labeled with both coarse
and fine-grained labels. For our training and testing process, we utilize
fine-grained labels. The dataset statistics are provided in Table 1.

###  4.2 Compared Methods

We compare with several OOD detection methods mentioned below:

  * •

Cross Entropy Loss (CELoss) trains a classifier using cross-entropy loss on
in-distribution documents. The predicted probabilities from the classifier are
used as confidence estimates for OOD detection.

  * •

Correctness Ranking Loss (CRL) Moon et al. (2020) is a regularization term
added to the CE-loss to make class probabilities better confidence estimates.
It estimates the true class probability to be proportional to the number of
times a sample is classified correctly during training.

###  4.3 Experimental Settings

We experiment with BERT Devlin et al. (2019) and RoBERTa Liu et al. (2019) as
text classifiers. For SelfOOD, we train the classifier for 5 epochs with a
batch size of 16 using an AdamW optimizer. We use a learning rate of 5e-5
using a linear scheduler with no warm-up steps. For all baselines, we train
the classifier for the same number of steps.

In our evaluation, for each dataset as in-distribution, we treat all other
datasets as OOD and compute the performance. Our evaluation follows a standard
approach for each in-distribution dataset where we begin by splitting the in-
distribution dataset into three subsets: 80% for training, 10% for validation,
and 10% for testing. The model is trained using the training split, and its
performance is evaluated on both the test split of the in-distribution dataset
and the entire OOD dataset.

Evaluation Metrics. We utilize evaluation metrics from Hendrycks and Gimpel
(2017); DeVries and Taylor (2018); Moon et al. (2020) such as the false
positive rate at 95% true positive rate (FPR95), minimum detection error over
all possible thresholds (ERR), the area under the risk-coverage curve (AURC),
and the area under the precision-recall curve (AUPR) using in-distribution
samples as the positives.

###  4.4 Results

We summarize the evaluation results with BERT and RoBERTa as classifiers on
four datasets in Table 2. All experiments are run on three random seeds and
the mean performance scores are reported. As shown in Table 2, we observe that
SelfOOD performs better than the baselines on most of the in-distribution, OOD
dataset pairs for both classifiers. A low FPR95 value indicates that the top
95% confident samples, selected based on their probability scores,
predominantly belong to the in-distribution class and SelfOOD achieves
improvements of up to 82 points in FPR95 with Yelp as in-distribution and
Clima as OOD datasets when compared to CRL with BERT classifier. SelfOOD also
exhibits substantial improvements of up to 33 points in Detection Error, 48
points in AUROC, and 58 points in AUPR when compared to CE-Loss with BERT
classifier. SelfOOD achieves a perfect OOD detection score for some settings
such as TREC as OOD and NYT as In-distribution datasets for both BERT and
RoBERTa classifiers. These results highlight the effectiveness of our ranking
formulation with self-supervised training using IDIL loss.

Method | FPR95 | ERR | AUROC | AUPR
---|---|---|---|---
|  |  | In Dist: Clima
---
OOD: NYT
|
SelfOOD | 17.2 | 2.6 | 97.8 | 94.5
\+ Grad Sub | 67.9 | 8.9 | 82.4 | 61.1
\+ Grad Sub & Min | 98.4 | 10.7 | 49.2 | 32.3
\+  | Intra-Doc
---
92.7 | 10.6 | 67.3 | 41.6
\- SILu | 14.6 | 3.9 | 96.7 | 89.3
|  |  | In Dist: NYT
---
OOD: Clima
|
SelfOOD | 10.5 | 1.7 | 98.4 | 90.5
\+ Grad Sub only | 83.5 | 5.1 | 77.4 | 44.8
\+ Grad Sub & Min | 96.5 | 5.2 | 60.3 | 36.0
\+  | Intra-Doc
---
48.3 | 2.6 | 93.2 | 80.2
\- SiLU | 19.0 | 2.3 | 97.4 | 86.0

Table 3: Ablation Study. All experiments are repeated with three random seeds
and the mean score is reported.

###  4.5 Ablation Study

To understand the impact of each component in our IDIL loss design and
implementation, we compare our method with four ablated versions with BERT
classifier in Table 3: (1) SelfOOD \+ Grad Sub represents our method with
backpropagating gradients through the subtrahend instead of the minuend, (2)
SelfOOD \+ Grad Sub & Min represents our method with gradients backpropagating
through both minuend and subtrahend, (3) SelfOOD \+ Intra-Doc considers intra-
document comparison similar to CE loss, in addition to inter-document intra-
label comparison for loss computation, and (4) SelfOOD\- SiLU excludes SiLU
function from the IDIL loss formulation. We also present the performance of
SelfOOD for reference. SelfOOD performs better than SelfOOD \+ Grad Sub & Min
demonstrating that backpropagating through one part of the difference is more
beneficial than both, and the comparison between SelfOOD and SelfOOD \+ Grad
Sub indicates that backpropagating through minuend is better than subtrahend.
We observe that incorporating intra-document comparison into the loss
formulation leads to a decrement in the ranking ability of the model. Finally,
we observe that removing the SiLU function from the IDIL loss leads to a
decrease in most of the metrics.

(a) CE Loss (b) CRL (c) SelfOOD

Figure 3: Maximum softmax score vs percentile in-distribution & OOD data
associated with that score. By using the least in-distribution maximum softmax
score as the threshold (dotted line) for OOD classification, CE loss considers
more than 50% and CRL considers almost 100% of the OOD data as in-distribution
(red region). However, in the case of SelfOOD, we observe a clear margin in
maximum softmax scores that separate OOD and in-distribution.

##  5 Analysis & Case Studies

In this section, we present a comprehensive analysis of our proposed method
from different perspectives to understand its effectiveness.

Figure 4:  We train a BERT OOD classifier on the Clima dataset and plot
validation FPR95 with Yelp as the OOD dataset during training. We observe a
continuous decrease in FPR95 for SelfOOD.

###  5.1 Maximum softmax score Analysis

We plot the maximum softmax score with the percentile in-distribution and OOD
data associated with that score for CE loss, CRL, and SelfOOD with BERT
classifier on Clima as in-distribution and NYT as OOD datasets in Figure 3.
When comparing the maximum softmax score with the percentile data associated
with that score, interesting observations can be made regarding its
distribution. Specifically, when using the least in-distribution maximum
softmax score as the threshold for OOD classification, we find that the CE
loss considers over 50% of the OOD data and CRL considers almost 100% of the
OOD data as in-distribution. However, in the case of SelfOOD, we observe a
clear margin in the maximum softmax scores that effectively separates OOD and
in-distribution data. This suggests that the threshold needs to be carefully
tuned for CE loss and CRL, requiring more effort and annotated data, whereas
in the case of SelfOOD, we do not require such tuning. This demonstrates that
SelfOOD is capable of accurately classifying between OOD and in-distribution
samples based on their respective maximum softmax scores, resulting in
superior performance.

###  5.2 Validation FPR95 vs Training steps

We train a BERT OOD classifier on Clima as the in-distribution dataset and
plot validation FPR95 vs training steps with Yelp as the OOD dataset. As shown
in Figure 4, we observe that FPR95 continuously decreases and plateaus for
SelfOOD whereas for other methods it fluctuates and stagnates at a higher
value. This demonstrates that the minimization of IDIL loss solely on in-
distribution data successfully translates to the OOD detection task.

###  5.3 Batch size Analysis

(a) NYT (b) Yelp

Figure 5: Performance of SelfOOD w.r.t. batch size. We consider BERT
classifier, Clima as in-distribution, and NYT, Yelp as OOD datasets. Each
experiment is repeated with three random seeds and the mean is plotted. We
observe an increase in performance with an increase in batch size.

We would like to note that in our methodology, we compare documents within the
mini-batch when computing the loss. Consequently, the number of document pairs
over which the loss is computed depends on the batch size used during
training. To investigate the impact of batch size on performance, we vary the
batch size and evaluate the corresponding performance in Figure 5 with BERT
classifier. We consider Clima as in-distribution, and NYT, Yelp as OOD
datasets. As the batch size increases, the performance of the model also
improves, up until a batch size of 16, where the performance reaches a
plateau. This observation aligns with our intuition that a larger batch size
allows for more in-batch document pairs to be compared, leading to more
accurate loss computation. Based on these findings, we recommend utilizing a
batch size of 16 or higher to achieve optimal performance.

###  5.4 Fine-grained OOD Detection

To explore the limits of our proposed method, we consider in-distribution and
OOD datasets from the same domain. The objective is to assess the
discriminative capabilities of SelfOOD and evaluate its ability to
differentiate between samples that are closely related but still OOD, which is
a challenging task. Specifically, we consider the news domain and choose NYT
as the in-distribution, and AGNews Zhang et al. (2015),
20News444http://qwone.com/~jason/20Newsgroups/ as OOD datasets and train BERT
classifier. As shown in Table 4, SelfOOD performs significantly better than CE
Loss and CRL on most of the metrics. Moreover, it also achieves near-perfect
scores for the AGNews as an OOD dataset, highlighting its ability to
accurately identify OOD samples even when they belong to the same domain as
the in-distribution dataset.

OOD | Method | FPR95 | ERR | AUROC | AUPR
---|---|---|---|---|---
AGNews | CE-Loss | 45.5 | 0.9 | 86.5 | 16.6
SelfOOD | 0.0 | 0.1 | 100.0 | 99.4
CRL | 77.2 | 0.9 | 80.6 | 9.5
20News | CE-Loss | 25.4 | 6.2 | 91.9 | 39.6
SelfOOD | 28.2 | 4.9 | 94.2 | 61.5
CRL | 59.9 | 6.4 | 87.4 | 36.9

Table 4: Fine-grained OOD detection results with NYT as the in-distribution
dataset with BERT classifier. We choose two datasets AGNews, 20News from the
news domain, the same as NYT, and consider them as OOD datasets. The results
show that SelfOOD can accurately detect OOD samples within the same domain.

###  5.5 SelfOOD \+ Mahalanobis distance

Mahalanobis distance-based estimation of OOD scores is an effective post-
processing method used on trained OOD classifiers Leys et al. (2018); Xu et
al. (2020). We investigate whether this post-processing further improves the
performance of our method. OOD scores are estimated as the distance between a
test data point and the distribution of in-distribution samples using the
Mahalanobis distance metric. Following Xu et al. (2020), we consider the
intermediate layer encodings of the OOD classifier trained on in-distribution
samples as its representative distribution. We apply this post-processing on
top of SelfOOD and experiment with two in-distribution, OOD pairs with BERT
classifier. As shown in Table 5, we observe further improvement in
performance, demonstrating the quality of learned representative distribution
using SelfOOD.

InD | OOD | Method | FPR95 | ERR | AUROC | AUPR
---|---|---|---|---|---|---
Clima | Yelp | SelfOOD | 14.7 | 0.6 | 98.0 | 90.8
SelfOOD \+ Maha | 0.8 | 0.2 | 99.2 | 97.2
Yelp | NYT | SelfOOD | 63.2 | 19.6 | 79.4 | 82.7
SelfOOD \+ Maha | 12.4 | 4.4 | 97.4 | 97.5

Table 5: OOD detection results with Mahalanobis Distance post-processing
technique. We choose two in-distribution, OOD pairs with BERT classifier. The
results show that SelfOOD’s OOD detection capabilities are enhanced with the
post-processing technique.

###  5.6 Classification performance of SelfOOD

We present the text classification performance of SelfOOD and CE-Loss with
BERT classifier on in-distribution datasets in Table 6. The findings highlight
that while SelfOOD serves as a reliable OOD detector, its performance as a
text classifier is subpar. This observation can be attributed to our IDIL loss
formulation, which focuses on comparing confidence levels across documents for
each label rather than across labels for each document. As a result, the IDIL
loss primarily promotes the ordinal ranking of confidence levels across
documents, which enhances the model’s OOD detection capabilities. However,
this emphasis on inter-document ranking comes at the expense of inter-label
ranking, resulting in limited classification capabilities. Moreover, when we
introduce the intra-document comparison to the IDIL loss, as discussed in
Section 4.5, we observe a decline in the model’s ranking ability. This further
supports the notion that balancing the inter-document and intra-document
comparisons is crucial for achieving optimal performance in both OOD detection
and text classification tasks.

Dataset | Method | Accuracy
---|---|---
NYT | CE-Loss | 97.4
SelfOOD | 0.3
Clima | CE-Loss | 85.1
SelfOOD | 9.5
Yelp | CE-Loss | 69.5
SelfOOD | 20.0
TREC | CE-Loss | 97.1
SelfOOD | 1.8

Table 6: Classification Performance of SelfOOD and CE-Loss with BERT
classifier on two datasets. The results show that SelfOOD is a poor text
classifier.

##  6 Conclusion

In this paper, we present SelfOOD, a novel framework for OOD detection that
leverages only in-distribution samples as supervision. Building upon the
insight that OOD samples typically do not belong to any in-distribution
labels, we formulate the OOD detection problem as an inter-document intra-
label ranking task. To address this challenge, we propose a novel IDIL loss,
which guides the training process. Through extensive experiments on multiple
datasets, we demonstrate the effectiveness of SelfOOD on OOD detection task.
However, we also acknowledge that it comes at the expense of text
classification performance. Future research can focus on developing techniques
that effectively balance inter-document and intra-document comparisons,
enabling improved performance in both OOD detection and text classification
tasks.

## References

  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _ArXiv_ , abs/1810.04805.
  * DeVries and Taylor (2018) Terrance DeVries and Graham W Taylor. 2018.  Learning confidence for out-of-distribution detection in neural networks.  _arXiv preprint arXiv:1802.04865_.
  * Elfwing et al. (2018) Stefan Elfwing, Eiji Uchibe, and Kenji Doya. 2018.  Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.  _Neural Networks_ , 107:3–11.
  * Fei and Liu (2016) Geli Fei and Bing Liu. 2016.  Breaking the closed world assumption in text classification.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 506–514.
  * Gangal et al. (2020) Varun Gangal, Abhinav Arora, Arash Einolghozati, and Sonal Gupta. 2020.  Likelihood ratios and generative classifiers for unsupervised out-of-domain detection in task-oriented dialog.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 7764–7771.
  * Good (1952) Irving John Good. 1952.  Rational decisions.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 14(1):107–114.
  * Guo et al. (2017) Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017.  On calibration of modern neural networks.  In _International conference on machine learning_ , pages 1321–1330. PMLR.
  * Gupta and Ramdas (2021) Chirag Gupta and Aaditya Ramdas. 2021.  Top-label calibration and multiclass-to-binary reductions.  _arXiv preprint arXiv:2107.08353_.
  * Hahnloser et al. (2000) Richard HR Hahnloser, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian Seung. 2000.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit.  _nature_ , 405(6789):947–951.
  * Hendrycks and Gimpel (2017) Dan Hendrycks and Kevin Gimpel. 2017.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In _International Conference on Learning Representations_.
  * Hendrycks et al. (2018) Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. 2018.  Deep anomaly detection with outlier exposure.  _arXiv preprint arXiv:1812.04606_.
  * Hovy et al. (2001) Eduard Hovy, Laurie Gerber, Ulf Hermjakob, Chin-Yew Lin, and Deepak Ravichandran. 2001.  Toward semantics-based answer pinpointing.  In _Proceedings of the First International Conference on Human Language Technology Research_.
  * Hsu et al. (2020) Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. 2020.  Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10951–10960.
  * Kamath et al. (2020) Amita Kamath, Robin Jia, and Percy Liang. 2020.  Selective question answering under domain shift.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5684–5696.
  * Kim (2014) Yoon Kim. 2014.  Convolutional neural networks for sentence classification.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1746–1751, Doha, Qatar. Association for Computational Linguistics.
  * Larson et al. (2019) Stefan Larson, Anish Mahendran, Joseph J Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K Kummerfeld, Kevin Leach, Michael A Laurenzano, Lingjia Tang, and et al. 2019.  An evaluation dataset for intent classification and out-of-scope prediction.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1311–1316.
  * Laud et al. (2023) Tanmay Laud, Daniel Spokoyny, Tom Corringham, and Taylor Berg-Kirkpatrick. 2023\.  Climabench: A benchmark dataset for climate change text understanding in english.  _arXiv preprint arXiv:2301.04253_.
  * Lee et al. (2018) Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  _Advances in neural information processing systems_ , 31.
  * Leys et al. (2018) Christophe Leys, Olivier Klein, Yves Dominicy, and Christophe Ley. 2018.  Detecting multivariate outliers: Use a robust variant of the mahalanobis distance.  _Journal of experimental social psychology_ , 74:150–156.
  * Li et al. (2021) Xiaoya Li, Jiwei Li, Xiaofei Sun, Chun Fan, Tianwei Zhang, Fei Wu, Yuxian Meng, and Jun Zhang. 2021.  $k$ folden: $k$-fold ensemble for out-of-distribution detection.  _arXiv preprint arXiv:2108.12731_.
  * Li and Roth (2002) Xin Li and Dan Roth. 2002.  Learning question classifiers.  In _COLING 2002: The 19th International Conference on Computational Linguistics_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Mekala et al. (2022) Dheeraj Mekala, Chengyu Dong, and Jingbo Shang. 2022.  LOPS: Learning order inspired pseudo-label selection for weakly supervised text classification.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 4894–4908, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Moon et al. (2020) Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. 2020.  Confidence-aware learning for deep neural networks.  In _international conference on machine learning_ , pages 7034–7044. PMLR.
  * Nguyen et al. (2015) Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 427–436.
  * Ouyang et al. (2021) Yawen Ouyang, Jiasheng Ye, Yu Chen, Xinyu Dai, Shujian Huang, and Jiajun Chen. 2021\.  Energy-based unknown intent detection with data manipulation.  _arXiv preprint arXiv:2107.12542_.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Ren et al. (2019) Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. 2019.  Likelihood ratios for out-of-distribution detection.  _Advances in neural information processing systems_ , 32.
  * Rosenblatt (1957) Frank Rosenblatt. 1957.  _The perceptron, a perceiving and recognizing automaton Project Para_.  Cornell Aeronautical Laboratory.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_.
  * Toneva et al. (2019) Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. 2019.  An empirical study of example forgetting during deep neural network learning.  In _International Conference on Learning Representations_.
  * Vyas et al. (2018) Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L Willke. 2018.  Out-of-distribution detection using an ensemble of self supervised leave-out classifiers.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 550–564.
  * Wei et al. (2022) Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. 2022.  Mitigating neural network overconfidence with logit normalization.  In _International Conference on Machine Learning_ , pages 23631–23644. PMLR.
  * Wu et al. (2022) Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, and Weiran Xu. 2022.  Revisit overconfidence for ood detection: Reassigned contrastive learning with adaptive class-dependent threshold.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4165–4179.
  * Xu et al. (2020) Hong Xu, Keqing He, Yuanmeng Yan, Sihong Liu, Zijun Liu, and Weiran Xu. 2020.  A deep generative distance-based classifier for out-of-domain detection with mahalanobis space.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 1452–1460.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  In _Neural Information Processing Systems_.
  * Zeng et al. (2021a) Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang, and Weiran Xu. 2021a.  Modeling discriminative representations for out-of-domain detection with supervised contrastive learning.  _arXiv preprint arXiv:2105.14289_.
  * Zeng et al. (2021b) Zhiyuan Zeng, Hong Xu, Keqing He, Yuanmeng Yan, Sihong Liu, Zijun Liu, and Weiran Xu. 2021b.  Adversarial generative distance-based classifier for robust out-of-domain detection.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 7658–7662. IEEE.
  * Zhan et al. (2021) Li-Ming Zhan, Haowen Liang, Bo Liu, Lu Fan, Xiao-Ming Wu, and Albert Lam. 2021.  Out-of-scope intent detection with self-supervision and discriminative training.  _arXiv preprint arXiv:2106.08616_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, _Advances in Neural Information Processing Systems 28_ , pages 649–657. Curran Associates, Inc.
  * Zhou et al. (2021) Wenxuan Zhou, Fangyu Liu, and Muhao Chen. 2021.  Contrastive out-of-distribution detection for pretrained transformers.  _arXiv preprint arXiv:2104.08812_.


# A note on background independence in $\text{AdS}_{3}$ string theory

Bob Knighton<EMAIL_ADDRESS>
###### Abstract

In this note, we comment on the path integral formulation of string theory on
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ where $\mathcal{M}$ is any
hyperbolic 3-manifold. In the special case of $k=1$ NS-NS flux, we provide a
covariant description of the worldsheet theory and argue that the path
integral depends only on the details of the conformal boundary
$\partial\mathcal{M}$, making the background independence of this theory
manifest. We provide a simple path integral argument that the path integral
localizes onto holomorphic covering maps from the worldsheet to the boundary.
For closed manifolds $\mathcal{M}$, the gravitational path integral is argued
to be trivial. This implies that the bulk gravitational theory has precisely
one state in its Hilbert space. Finally, we comment on the effect of
continuous deformations of the worldsheet theory which introduce non-minimal
string tension.

††institutetext: Department of Applied Mathematics & Theoretical Physics,
University of Cambridge,
Wilberforce Road, Cambridge CB3 0WA, United Kingdom

##  1 Introduction

For the last three decades, the AdS/CFT correspondence Maldacena:1997re ;
Witten:1998qj  has served as a guiding principle for the study of quantum
gravity. The most basic statement of the holographic dictionary is that the
quantum gravity path integral on an asymptotically $\text{AdS}_{d+1}$ manifold
$\mathcal{M}$ should be equal to the path integral of a corresponding
$\text{CFT}_{d}$ on the conformal boundary $X$. Mathematically111In the
quantum gravity literature it is common to express the left-hand-side as a sum
over all suitable geometries with boundary $X$. In such cases, we consider the
sum over geometries as a part of the definition of $Z_{\text{grav}}$ and
therefore do not write the sum out explicitly.

$Z_{\text{grav}}(\mathcal{M})=Z_{\text{CFT}}(X)\,.$ (1)

Perhaps the most striking, and poorly understood, feature of this equation is
the independence of the right-hand-side on the details of the bulk manifold
$\mathcal{M}$. The holographic principle seems to imply that the gravitational
path integral, however that may be defined, depends only on the data of the
asymptotic boundary of the bulk manifold, such as the induced conformal class
of metrics and the induced spin structure on $X$.

The independence of the right-hand-side of (1) has two immediate consequences.
The first and most obvious is that if $\mathcal{M}_{1}$ and $\mathcal{M}_{2}$
are bulk manifolds which both have boundary $X$, then

$Z_{\text{grav}}(\mathcal{M}_{1})=Z_{\text{grav}}(\mathcal{M}_{2})\,.$ (2)

This property should be thought of as a version of the background independence
of quantum gravity. Put simply, the gravitational path integral on a
background $\mathcal{M}$ with boundary $X$ should somehow include a sum over
all other spacetimes with the same boundary. This sum over geometries is often
put into the gravitational path integral by hand. An immediate corollary of
(2) is that if the boundary $X$ is disconnected, then the quantum gravity
partition function should factorize, despite the existence of connected on-
shell spacetimes with disconnected boundaries Maldacena:2004rf .

The second property pertains to spacetimes $\mathcal{M}$ whose boundaries are
empty. Such spacetimes are sometimes called ‘baby universes’ in the quantum
gravity literature. The AdS/CFT correspondence predicts that the gravitational
partition function of such spacetimes must satisfy

$Z_{\text{grav}}(\mathcal{M})=1\,.$ (3)

This follows immediately from the fact that a CFT on the empty set can only
have one state, namely the ground state. Equation (3) implies that the Hilbert
space of quantum gravity in a baby universe has precisely one state. The idea
that the Hilbert space of quantum gravity in a closed universe is one-
dimensional dates back to Coleman Coleman:1988cy  and has recently gained
traction in the context of solvable models of gravity Marolf:2020xie  and in
the swampland program McNamara:2020uza .

Understanding the properties (2) and (3) in the context of concrete UV-
complete theories of gravity is a difficult problem which in large part due to
the ambiguity in the definition of the quantum gravity path integral. For
example, in the case of 2D dilaton gravity, the naive definition of the
gravitational path integral does not lead to factorizing partition functions.
However, it has been shown that there is a consistent modification of the path
integral which does lead to factorizing answers by including nonperturbative
extended objects in the sum over topologies Blommaert:2021fob . A similar
construction leads to a trivial Hilbert space in closed universes
Usatyuk:2024mzs ; Usatyuk:2024isz .

Another example of a UV-complete gravitational theory which is under analytic
control is the worldsheet theory dual to the symmetric orbifold
$\text{Sym}^{N}(\mathbb{T}^{4})$. The bulk theory is described by type IIB
string theory on the background
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$, supported by $k=1$ units
of pure NS-NS flux. Here, $\mathcal{M}$ is any locally-$\text{AdS}_{3}$
spacetime, i.e. a hyperbolic three-manifold. The duality between this ‘minimal
tension’ string theory and the symmetric orbifold
$\text{Sym}^{N}(\mathbb{T}^{4})$ has been essentially proven in the framework
of perturbative string theory Eberhardt:2018ouy ; Eberhardt:2019ywk ;
Eberhardt:2020akk ; Hikida:2020kil ; Eberhardt:2020bgq ; Dei:2020zui ;
Knighton:2020kuh ; Eberhardt:2021jvj ; Dei:2023ivl .

Although the worldsheet theory is a sigma model on $\mathcal{M}$, and is thus
naively sensitive to the precise structure of the bulk, it was argued in
Eberhardt:2021jvj  that the worldsheet path integral is indeed background
independent. The proof of this statement used the classical fact that any
hyperbolic three-manifold is a quotient $\mathbb{H}^{3}/\Gamma$ of hyperbolic
three-space by a discrete subgroup $\Gamma\subset\text{SL}(2,\mathbb{C})$.
This allows one to treat the worldsheet theory as an orbifold of the theory on
$\mathbb{H}^{3}\times\text{S}^{3}\times\mathbb{T}^{4}$, which is well-
understood. The proof of background independence then boils down to showing
that the worldsheet path integral gives the same answer for two subgroups
$\Gamma_{1},\Gamma_{2}$ so long as

$\partial(\mathbb{H}^{3}/\Gamma_{1})\cong\partial(\mathbb{H}^{3}/\Gamma_{2})\,,$
(4)

where $\cong$ denotes conformal equivalence.

In this note, we revisit the arguments of Eberhardt:2021jvj  using the
recently-discovered free field realization of $\mathfrak{psu}(1,1|2)$ explored
in Beem:2023dub ; Dei:2023ivl . We review the worldsheet CFT on
$\text{AdS}_{3}\times\text{S}^{3}\times\mathbb{T}^{4}$, paying specific
attention to the free field realization of Beem:2023dub ; Dei:2023ivl , and
explain how to define the path integral on
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ for any hyperbolic
3-manifold $\mathcal{M}$. We find that the resulting path integral depends
only on the conformal data of the asymptotic boundary $X$ of $\mathcal{M}$,
and is completely insensitive to the details of the bulk. We also show that
the worldsheet path integral localizes to holomorphic maps $\gamma:\Sigma\to
X$, in a fashion analogous to the topological A-model.

Since the path integral depends only on the conformal data of the boundary
$X$, we conclude immediately that the worldsheet theory is background
independent, verifying the result of Eberhardt:2021jvj  in this new language.
We also comment briefly on the case of spacetimes with empty boundary, and
argue that the path integral is ‘empty’. This implies that the string theory
free energy on such spacetimes vanishes, and so the gravitational partition
function is

$Z_{\text{grav}}(\mathcal{M})=\exp\left(\mathcal{F}_{\text{st}}(\mathcal{M})\right)=1\,,$
(5)

as expected from the holographic principle. Finally, we explain that these
properties (background independence and triviality in closed universes)
continue to hold true as one deforms continuously away from the tensionless
limit by turning on infinitesimal Ramond-Ramond flux.

##  2 The minimal tension string

Type IIB superstrings on
$\text{AdS}_{3}\times\text{S}^{3}\times\mathbb{T}^{4}$ can be treated in the
RNS formalism, for which the worldsheet CFT is described by an $\mathcal{N}=1$
supersymmetric WZW model of the form

$\mathfrak{sl}(2,\mathbb{R})_{k}^{(1)}\oplus\mathfrak{su}(2)_{k}^{(1)}\oplus\mathbb{T}^{4}\,,$
(6)

and well as the usual superconformal ghosts. While enabling a straightforward
covariant quantization scheme, the RNS description has the disadvantage of
obscuring the target space (and thus dual CFT) supersymmetries, and is thus
not well-suited for analyzing the bulk/boundary dictionary in AdS/CFT.

An alternative quantization scheme involves the hybrid formalism of Berkovits-
Vafa-Witten Berkovits:1999im . This formulation trades the $\mathcal{N}=1$ WZW
models with a WZW model on the supergroup $\text{PSU}(1,1|2)$, which is the
group of left super-isometries of $\text{AdS}_{3}\times\text{S}^{3}$. As
payment for the gift of manifest target-space supersymmetry, the compact
$\mathbb{T}^{4}$ becomes topologically twisted, and the superconformal ghost
system gets traded for the usual $b,c$ ghosts, as well as an exotic new $c=28$
scalar, known as the $\rho$ ‘ghost’. All told, the worldsheet theory on
$\text{AdS}_{3}\times\text{S}^{3}\times\mathbb{T}^{4}$ takes the schematic
form

$\mathfrak{psu}(1,1|2)_{k}\oplus(\text{topologically twisted
}\mathbb{T}^{4})\oplus(b,c,\rho)\,.$ (7)

Or, in terms of actions,

$S=S_{\mathfrak{psu}}+S_{\mathbb{T}^{4}}+S_{b,c}+S_{\rho}\,,$ (8)

where $S_{\mathfrak{psu}}$ is the usual WZW model action on the supergroup
$\text{PSU}(1,1|2)$. The $\mathbb{T}^{4}$ action is identical to the usual
one, except the fermions are now topologically twisted (i.e. half have
conformal weight $h=1$, while the other half have $h=0$). The topologically
twisted $\mathbb{T}^{4}$ has central charge $c=0$, and so the total central
charge of this theory is

$c(\mathfrak{psu}(1,1|2)_{k})+c(b,c)+c(\rho)=-2-26+28=0\,,$ (9)

and thus describes a critical string theory. Here, we used the fact that the
$\text{PSU}(1,1|2)$ WZW model has central charge $c=-2$, independent of the
level.

When the amount $k$ of NS-NS flux is taken to its minimal value ($k=1$), the
$\text{PSU}(1,1|2)$ WZW model admits a description in terms of a free field
theory. There are several equivalent descriptions of this free field theory,
but the one which is of most direct use is that of Beem:2023dub ; Dei:2023ivl
. There, it was argued that the $\text{PSU}(1,1|2)$ WZW model is equivalent to
a free CFT consisting of

  * •

A bosonic first-order system $(\beta,\gamma)$ with $h(\beta)=1$ and
$h(\gamma)=0$.

  * •

Two fermionic first-order systems $(p_{a},\theta^{a})$ with $h(p_{a})=1$ and
$h(\theta^{a})=0$ ($a=1,2$).

Each of these fields also has a right-moving counterpart. This system has
action

$S_{\mathfrak{psu}}=\frac{1}{2\pi}\int_{\Sigma}\left(\beta\overline{\partial}\gamma+p_{a}\overline{\partial}\theta^{a}+\bar{\beta}\partial\bar{\gamma}+\bar{p}_{a}\partial\bar{\theta}^{a}\right)\,,$
(10)

and has central charge $c=2-4=-2$, which is the central charge of the
$\text{PSU}(1,1|2)$ model.

To get an intuition for the target space interpretation of the free fields, we
note that the global $\text{PSU}(1,1|2)$ isometry group of the superspace
$\text{Super}(\text{AdS}_{3}\times\text{S}^{3})$ is a classical symmetry of
the action $S_{\mathfrak{psu}}$. The action on the free fields is described as
follows. Consider the supergroup element222We will always work with the
complexified group $\text{PSU}(1,1|2)_{\mathbb{C}}$. The choice of real form
will correspond to a choice of signature in $\text{AdS}_{3}$.

$g=\begin{pmatrix}a&b&h_{11}&h_{12}\\\ c&d&h_{21}&h_{22}\\\
f_{11}&f_{12}&A&B\\\ f_{21}&f_{22}&C&D\end{pmatrix}\in\text{PSU}(1,1|2)\,,$
(11)

where $a,b,c,d$ are entries in an $\text{SL}(2,\mathbb{C})$ matrix, $A,B,C,D$
are entries in an $\text{SU}(2,\mathbb{C})$ matrix, and $h_{ij},f_{ij}$ are
Grassman variables generating the $8$ supercharges. The action of $g$ on the
free fields $(\gamma,\theta^{1},\theta^{2})$ is by the fractional linear
transformations

$\begin{split}g\cdot\gamma&=\frac{a\gamma+b+h_{11}\theta^{1}+h_{12}\theta^{2}}{c\gamma+d+h_{21}\theta^{1}+h_{22}\theta^{2}}\,,\\\
g\cdot\theta^{1}&=\frac{f_{11}\gamma+f_{12}+A\theta^{1}+B\theta^{2}}{c\gamma+d+h_{21}\theta^{1}+h_{22}\theta^{2}}\,\\\
g\cdot\theta^{2}&=\frac{f_{21}\gamma+f_{22}+C\theta^{1}+D\theta^{2}}{c\gamma+d+h_{21}\theta^{1}+h_{22}\theta^{2}}\end{split}$
(12)

For purely bosonic transformations (i.e. with $h_{ij}=f_{ij}=0$), we see that
$\text{PSU}(1,1|2)$ acts on $\gamma$ via Möbius transformations, while the
fermions $\theta^{a}$ live in the defining representation of $\text{SU}(2)$,
while transforming like a field of weight $h=-\frac{1}{2}$ under
$\text{SL}(2,\mathbb{C})$.

Geometrically, the above discussion suggests a concrete meaning of the free
fields $\gamma,\theta^{1},\theta^{2}$, namely that they represent the
holomorphic $\mathcal{N}=2$ superspace coordinates on the boundary of
$\text{AdS}_{3}$. Indeed, the above fractional linear transformations are the
expected global supersymmetry transformations of a 2D CFT with $\mathcal{N}=4$
supersymmetry. As $\gamma$ is taken to be a complex scalar, the worldsheet
theory describes global $\text{AdS}_{3}$, which in Euclidean signature is the
global hyperbolic space $\mathbb{H}^{3}$, i.e. the solid Poincaré ball. Some
care is needed to take into consideration the north pole of the boundary
sphere, see for example the respective discussions in Frenkel:2005ku ;
Dei:2023ivl .

Once the $\text{PSU}(1,1|2)$ WZW model is defined, the worldsheet path
integral is defined in terms of a twisted $\mathcal{N}=4$ algebra on the
worldsheet. The details are somewhat complicated and we will not need the
specifics, see Berkovits:1999im ; Berkovits:1994vy ; Dei:2020zui ; Dei:2023ivl
for a detailed discussion. The upshot is that the worldsheet free energy is
then defined to be a loop expansion:

$\mathcal{F}_{\text{st}}=\sum_{g=0}^{\infty}g_{s}^{2g-2}\int_{\mathcal{M}_{g}}\left\langle\prod_{\alpha=1}^{g-1}|(G^{-},\mu_{\alpha})|^{2}\prod_{\alpha=g}^{3g-3}|\tilde{G}^{-},\mu_{\alpha}|^{2}\left(\int_{\Sigma}|G^{+}|^{2}\right)^{g-1}\int_{\Sigma}|J|^{2}\right\rangle\,,$
(13)

where $G^{\pm},\tilde{G}^{\pm}$ are the $\mathcal{N}=4$ generators on the
worldsheet, and $J$ is the Cartan of the worldsheet $\text{SU}(2)$
$R$-symmetry. The expectation value is defined formally by the path integral
over the fields $(\beta,\gamma,p_{a},\theta^{a})$, as well as the fields of
the topologically-twisted $\mathbb{T}^{4}$ and the $(b,c,\rho)$ ghost system.

The string free energy $\mathcal{F}_{\text{st}}$ represents the connected
piece of the full second quantized string theory partition function, which we
will denote as $Z_{\text{grav}}$, as it will play the role of a
‘gravitational’ partition function in our theory. The relationship between the
two is simply

$Z_{\text{grav}}=\exp\left(\mathcal{F}_{\text{st}}\right)\,.$ (14)

##  3 Defining the path integral

The above definition of the minimal tension string on
$\text{AdS}_{3}\times\text{S}^{3}\times\mathbb{T}^{4}$ was exactly that – a
definition of the worldsheet theory on global (Euclidean) $\text{AdS}_{3}$.
However, the AdS/CFT dictionary relates string theories on asymptotically-AdS
spacetimes with arbitrary boundary topologies to CFTs living on the boundary.
Thus, in order to say things about more interesting CFTs (i.e. those not
defined on the 2-sphere), we need to formulate the worldsheet theory on more
interesting bulk manifolds.

Any bulk manifold $\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ which
has a hope of having a CFT dual has to be at least asymptotically hyperbolic,
in the sense that $\mathcal{M}$ has an asymptotic boundary near which one can
take the metric tensor to have constant negative curvature. If we take
$\mathcal{M}$ to be globally hyperbolic (in the sense that $\mathcal{M}$
admits a metric of constant negative curvature everywhere), then a classical
theorem in the geometry of 3-manifolds guarantees that $\mathcal{M}$ is the
quotient of global hyperbolic space $\mathbb{H}^{3}$ by a discrete group
$\Gamma\subset\text{SL}(2,\mathbb{C})$. Thus, one way to describe string
theory on a hyperbolic 3-manifold $\mathcal{M}$ is to consider the orbifold of
string theory on global $\text{AdS}_{3}$ by the discrete group $\Gamma$.

This prescription, while in principle correct, is more subtle than it might
originally seem. The reason is that while the discrete group $\Gamma$ acts on
$\mathbb{H}^{3}$ nicely, it doesn’t necessarily act nicely on the sphere
$\text{S}^{2}\cong\partial\mathbb{H}^{3}$. In most cases, there will be a
nonempty subset $\Omega$ of $\text{S}^{2}$ for which $\Gamma$ acts ‘badly’,
while the complement $U=\text{S}^{2}\setminus\Omega$ admits a free action
under $\Gamma$. The boundary of $\mathbb{H}^{3}/\Gamma$, then, is not
$\text{S}^{2}/\Gamma$ (which is in general a very singular object), but rather
$U/\Gamma$. The set $\Omega$ is guaranteed to be a closed measure-zero set,
and is usually extremely complicated and fractal.

Let us nevertheless proceed. In order to define the path integral on
$\mathbb{H}^{3}/\Gamma$, we need to know how a discrete subgroup of
$\text{SL}(2,\mathbb{C})$ acts on the worldsheet fields. The generators of the
global $\mathfrak{sl}(2,\mathbb{C})$ symmetry on the worldsheet are the
charges Dei:2023ivl

$J^{+}_{0}=\oint\beta\,,\quad
J^{3}_{0}=\oint\left((\beta\gamma)+\frac{1}{2}(p_{a}\theta^{a})\right)\,,\quad
J^{-}_{0}=\oint\left((\beta\gamma^{2})+\gamma(p_{a}\theta^{a})\right)\,,$ (15)

where $(\cdots)$ denotes normal ordering. By acting on the worldsheet fields
with these charges, one can read off the finite transformation laws

$\begin{gathered}\gamma\to\frac{a\gamma+b}{c\gamma+d}\,,\quad\beta\to(c\gamma+d)^{2}\beta-c(c\gamma+d)(p_{a}\theta^{a})\,,\\\
p_{a}\to(c\gamma+d)p_{a}\,,\quad\theta^{a}\to(c\gamma+d)^{-1}\theta^{a}\end{gathered}$
(16)

for some matrix

$\begin{pmatrix}a&b\\\ c&d\end{pmatrix}\in\text{SL}(2,\mathbb{C})\,.$ (17)

Defining the worldsheet orbifold $\mathbb{H}^{3}/\Gamma$ is now a matter of
choosing a discrete subgroup $\Gamma\subset\text{SL}(2,\mathbb{C})$ and
treating the transformation laws (16) as equivalence relations in the path
integral. As discussed above, this process is poorly defined for $\gamma$,
since there will be a measure zero subset $\Omega$ of points on $\text{S}^{2}$
for which $\Gamma$ acts poorly. To remide this, we define $\gamma$ to live in
the set $U=\text{S}^{2}\setminus\Omega$.333If $\Omega$ is non-empty, then we
can WLOG pick $\infty\in\Omega$, so that $\gamma\neq\infty$ in the path
integral. If $\Omega$ is empty, we have to add in the point at infinity by
hand. This case is discussed at length in Section 4 of Dei:2023ivl .

While in principle this orbifold is well-defined, the non-homogeneous term in
the transformation law of $\beta$ in equation (16) means that there will be
non-trivial mixing between $\beta$ and the fermions $p_{a},\theta^{a}$, making
the computation of the path integral somewhat complicated. To fix this, follow
the trick of Frenkel:2008vz . Namely, we define a new field

$\beta^{\prime}=\beta+\omega\,(p_{a}\theta^{a})\,,$ (18)

where $\omega$ is a scalar on the worldsheet which transforms under
$\text{SL}(2,\mathbb{R})$ transformations as

$\omega\to\omega+c(c\gamma+d)\,.$ (19)

This modification in turn cancels the non-homogeneous term in (16), so that
$\beta^{\prime}$ transforms as

$\beta^{\prime}\to(c\gamma+d)^{2}\beta^{\prime}\,.$ (20)

The redefinition of $\beta$ also has the effect of changing the free-field
action (10) to

$S=\frac{1}{2\pi}\int_{\Sigma}(\beta^{\prime}\overline{\partial}\gamma+p_{a}\overline{\nabla}\theta^{a}+\bar{\beta}^{\prime}\partial\bar{\gamma}+\bar{p}_{a}\nabla\bar{\theta}^{a})\,,$
(21)

where we have defined the covariant derivative

$\overline{\nabla}=\overline{\partial}-\omega\overline{\partial}\gamma\,.$
(22)

This covariant derivative is defined in such a way that
$p_{a}\overline{\nabla}\theta^{a}$ is invariant under
$\text{SL}(2,\mathbb{R})$ transformations. In this way, we should think of
$\omega\overline{\partial}\gamma$ as a connection of the $\Gamma$-bundle on
the worldsheet of which $\theta^{a}$ is a section. We can explicitly construct
$\omega$ by picking a metric
$\rho(\gamma,\bar{\gamma})\mathrm{d}\gamma\,\mathrm{d}\bar{\gamma}$ in the
conformal gauge on the boundary of $\mathbb{H}^{3}$. Given the transformation
law

$\rho(\gamma,\bar{\gamma})\to|c\gamma+d|^{4}\rho(\gamma,\bar{\gamma})$ (23)

of the boundary metric under $\text{SL}(2,\mathbb{R})$ transformations, we see
that the combination

$\omega\overline{\partial}\gamma=\frac{1}{2}\overline{\partial}\log\rho(\gamma,\bar{\gamma})$
(24)

transforms in the appropriate way, and thus can be used to define $\omega$. We
note that with this identification, the connection
$\omega\overline{\partial}\gamma$ is the pullback of the spin connection on
the boundary.

From now on, we drop the prime on $\beta^{\prime}$ and work with the action

$S=\frac{1}{2\pi}\int_{\Sigma}(\beta\overline{\partial}\gamma+p_{a}\overline{\nabla}\theta^{a}+\bar{\beta}\partial\bar{\gamma}+\bar{p}_{a}\nabla\bar{\theta}^{a})$
(25)

as defining the free-field realization of $\mathfrak{psu}(1,1|2)$. Under
$\text{SL}(2,\mathbb{C})$ transformations, the fields now transform
homogeneously as

$\begin{gathered}\gamma\to\frac{a\gamma+b}{c\gamma+d}\,,\quad\beta\to(c\gamma+d)^{2}\beta\,,\\\
p_{a}\to(c\gamma+d)p_{a}\,,\quad\theta^{a}\to(c\gamma+d)^{-1}\theta^{a}\,.\end{gathered}$
(26)

Defining the path integral is now straightforward. We consider a discrete
subgroup $\Gamma\subset\text{SL}(2,\mathbb{C})$ and treat the transformation
laws (26) as identifications. Alternatively, we can view the identifications
(26) as defining the spaces to which our worldsheet fields
$\gamma,\beta,p_{a},\theta^{a}$ belong. Specifically, the above orbifold
identifies $\gamma$ as the holomorphic coordinates of a map from the
worldsheet into $X=\partial\mathcal{M}$. On the other hand, $\beta$ transforms
as a $(1,0)$-form on $X$. Thus, we can identify $\gamma,\beta$ as living in
the spaces

$\gamma\in\text{Map}(\Sigma\to
X)\,,\quad\beta\in\Gamma(K_{\Sigma}\otimes\gamma^{*}K_{X})\,,$ (27)

where $K_{\Sigma}$ and $K_{X}$ are the canonical bundles on the worldsheet and
the conformal boundary of $X$, respectively. This is nothing more than the
statement that $\beta$ transforms as a $(1,0)$-form under conformal
transformation on both $\Sigma$ and $X$. Here, we use the standard notation
that $\Gamma(\mathscr{L})$ denotes the space of sections of a line bundle
$\mathcal{L}$.

Next, the transformation laws (26) for the fermions $p_{a},\theta^{a}$ define
the data of the spinor bundle on $X$. That is, we can identify

$\theta^{a}\in\Gamma(\gamma^{*}S_{X}^{-1})\,,\quad
p_{a}\in\Gamma(K_{\Sigma}\otimes\gamma^{*}S_{X})\,,$ (28)

where $S_{X}$ is the holomorphic spin bundle on $X$. The right-moving fields
live in the right-moving analogues of these spaces.

As further evidence for this proposal, we can check the transformation laws of
the worldsheet fields under local conformal transformations on the boundary
$X$. These transformations take the form

$\begin{split}\gamma\to\gamma-\varepsilon\gamma^{n+1}\,,&\quad\beta\to(1+\varepsilon(n+1)\gamma^{n})\beta\,,\\\
\theta^{a}\to\left(1-\varepsilon\frac{n+1}{2}\gamma^{n}\right)\theta^{a}\,,&\quad
p_{a}\to\left(1-\varepsilon\frac{n+1}{2}\gamma^{n}\right)p_{a}\,.\end{split}$
(29)

These transformations leave the action (25) invariant and are generated by an
infinite set of conserved charges $\mathcal{L}_{n}$ generated by the
asymptotic symmetries of $\text{AdS}_{3}$, and which are dual to the Virasoro
algebra in the boundary CFT Brown:1986nw . These transformation laws are the
infinitesimal version of the finite local conformal transformations

$\begin{split}\gamma\to
f(\gamma)\,,&\quad\beta\to(f^{\prime}(\gamma))^{-1}\beta\,,\\\
\theta^{a}\to\sqrt{f^{\prime}(\gamma)}\,\theta^{a}\,,&\quad
p_{a}\to\sqrt{f^{\prime}(\gamma)^{-1}}\,p_{a}\,.\end{split}$ (30)

These transformation laws correspond precisely to the transformation laws of
sections of the bundles (27) and (28). The ambiguity of defining the square
roots in the above transformation laws is captured in the choice of spin
structure on the boundary $X$, which we take as input.444This ambiguity can be
traced back to the fact that it is actually
$\text{PSL}(2,\mathbb{C})=\text{SL}(2,\mathbb{C})/\\{\textbf{1}\sim-\textbf{1}\\}$
which acts faithfully on $\mathbb{H}^{3}$. The ambiguity in the choice of sign
in the transformation law for $p_{a},\theta^{a}$ in (16) is reflected in the
choice of spin structure on $X$.

The result of this discussion is that we can bypass the usual orbifolding
procedure of defining the worldsheet theory on
$\mathcal{M}=\mathbb{H}^{3}/\Gamma$ by simply identifying the free fields as
belonging to the spaces (27) and (28) in the path integral. Note that this
construction is formally very similar to the definition of topological sigma
models in two dimensions Witten:1988xj , except that there are twice as many
fermions, and the fermions transform as spinors (not one-forms) on the target
space $X$.

In order to define the path integral measure, we note that the spaces (27) and
(28) to which $\gamma,\beta,\theta^{a},p_{a}$ all belong are well-defined and
admit canonical metrics. Specifically, if we choose a metric $h$ on $\Sigma$
and $g$ on $X$, we can define the path integral measure implicitly by
specifying norms of variations $\delta\gamma,\delta\beta,\delta
p_{a},\delta\theta^{a}$. These norms are given by

$\begin{split}||\delta\gamma||^{2}=\int_{\Sigma}\mathrm{d}^{2}z\sqrt{h}\,g_{x\bar{x}}\delta\gamma\delta\bar{\gamma}\,,&\quad||\delta\beta||^{2}=\int_{\Sigma}\mathrm{d}^{2}z\sqrt{h}\,h^{z\bar{z}}g^{x\bar{x}}\delta\beta\,\delta\bar{\beta}\,,\\\
||\delta\theta^{a}||^{2}=\int_{\Sigma}\mathrm{d}^{2}z\sqrt{h}\sqrt{g_{x\bar{x}}}\,\delta\theta^{a}\delta\bar{\theta}^{a}\,,&\quad||\delta
p_{a}||^{2}=\int_{\Sigma}\mathrm{d}^{2}z\sqrt{h}\,h^{z\bar{z}}\sqrt{g^{x\bar{x}}}\,\delta
p_{a}\delta\bar{p}_{a}\,.\end{split}$ (31)

Note that the definitions of these norms are not Weyl-invariant on the
worldsheet, and so the path integral measure will have a Weyl anomaly. This
will be canceled by the central charge of the $c=2$ ghost system, so that the
full path integral is Weyl-invariant.

##  4 The path integral localizes

A key feature of the minimal tension string is that its path integral
localizes to a discrete sum of holomorphic covering spaces of $X$. This was
shown in Eberhardt:2019ywk ; Dei:2020zui ; Knighton:2020kuh  for
$X\cong\text{S}^{2}$ and extended in Eberhardt:2020bgq ; Eberhardt:2021jvj  to
boundaries of higher genus. These proofs used a somewhat complicated analysis
based on a different free-field realization than the one presented here. In
the free field representation with action (25), this argument can be
simplified by a great deal. Let us briefly explain how this works.

Assuming that none of the vertex operators in a correlation function depend on
$\beta$, we can integrate it out. The result is that the path integral
includes a delta-functional $\delta^{(2)}(\overline{\partial}\gamma)$ which
imposes that the map $\gamma:\Sigma\to X$ is holomorphic. The space
$\mathcal{H}(\Sigma\to X)$ of holomorphic maps is disconnected, and breaks up
into a union of connected spaces which depend on the degree
$\text{deg}(\gamma)$ of the map (i.e. the number of preimages of a generic
point). This component of $\mathcal{H}(\Sigma\to X)$ has dimension

$\text{deg}(\gamma)(2-2G)+1-g\,,$ (32)

so that the dimension of the full path integral (including the integration
over $\mathcal{M}_{g}$) is

$\text{deg}(\gamma)(2-2G)+2g-2\,.$ (33)

Now, the degree of the map $\gamma$ in the path integral is not allowed to
take any random value, but is constrained by the zero mode condition of the
fermions $p_{a},\theta^{a}$. The supercharges $\tilde{G}^{-}$ appearing in the
definition of the free energy (13) are proportional to $p_{1}p_{2}$
Dei:2023ivl  and so the number of, say, $p_{1}$’s in the correlator defining
$\mathcal{F}_{\text{st}}$ is $2g-2$. However, the number of zero modes of
$\theta_{1}$ minus the number of zero modes of $p^{1}$ is given by the index
of the covariant derivative $\overline{\nabla}$, which is readily calculated
by the Riemann-Roch theorem:

$\begin{split}\text{dim}\,\text{ker}(\overline{\nabla})-\text{dim}\,\text{coker}(\overline{\nabla})&=\deg(\gamma^{*}(S_{X}))+1-g\\\
&=\text{deg}(\gamma)(1-G)+1-g\,.\end{split}$ (34)

Demanding that this equal $2g-2$, so that the path integral (13) is nonzero,
gives the selection rule

$\text{deg}(\gamma)(1-G)=1-g$ (35)

which completely determines the degree of the map $\gamma$ in terms of the
worldsheet and boundary genera.

By the Riemann-Hurwitz formula, the selection rule (35) implies that the path
integral can only include holomorphic maps $\gamma$ which are topologically
covering maps (that is, $\gamma$ has no branch points). Plugging the selection
rule (35) into the dimension (33) of the path integral immediately yields the
result that the worldsheet path integral, after integrating out $\beta$ and
the fermions $p_{a},\theta^{a}$, localizes to a zero-dimensional integral,
i.e. a discrete sum. This sum is over the space of (connected) unbranched
holomorphic maps $\gamma$.

It is worth pointing out that the localization principle described above is
essentially the same as the analogous property of the topological A-model
Witten:1988xj ; Witten:1991zz , and indeed, as mentioned in the previous
section, the path integral of the $k=1$ theory is structurally similar to the
A-model at the level of the free fields (25). It has been suggested in the
literature that the minimal tension string should, in some sense, be related
to a topological string theory Eberhardt:2018ouy ; Dei:2020zui . The above
analysis sheds light on the sense in which this is true.

##  5 Background independence

In the previous section we argued that the minimal tension string theory on
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ is given by a worldsheet
CFT whose action is simply the free field action in (25) (plus the action of
the $\mathbb{T}^{4}$ and ghosts), but for which the fields are taken to lie in
sections of various line bundles. Importantly, the only ingredients going into
defining this worldsheet CFT are

  * •

The complex structure on $\partial\mathcal{M}$, which is induced by the
asymptotic form of the metric on $\mathcal{M}$ near the boundary.

  * •

The spin structure on $\partial\mathcal{M}$, which simply tells us the
periodicity of the fermions $\theta^{a}$ as we traverse a noncontractible
cycle on the boundary.

The key observation is that this information is completely independent of the
details of the bulk manifold $\mathcal{M}$ which are not visible near the
asymptotic boundary. This includes, for example, whether a non-contractible
cycle on $\partial\mathcal{M}$ becomes contractible in the bulk.

The structure of the worldsheet CFT seems to imply then that string theory on
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ is completely independent
of the bulk details of the 3-manifold $\mathcal{M}$, as the fundamental fields
are completely specified by asymptotic data on the boundary. In particular,
this implies that if $\mathcal{M}$ and $\mathcal{M}^{\prime}$ are two
different bulk manifolds with the same asymptotic boundary data, then any
string theory observable computed on
$\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ is equal to the same
observable computed on
$\mathcal{M}^{\prime}\times\text{S}^{3}\times\mathbb{T}^{4}$.555We emphasize
that we had to use the special property of string theory at $k=1$ units of NS-
NS flux that it is described by the free-field action (10). For other values
of the string tension, this argument would not go through. This statement is a
concrete realization of the idea of background independence of string theory.

An immediate consequence of this statement is that if $\mathcal{M}$ is a
connected 3-manifold with a disconnected boundary $X_{1}\sqcup\cdots\sqcup
X_{\ell}$, then the string free energy (13) satisfies

$\mathcal{F}_{\text{st}}(\mathcal{M})=\sum_{i=1}^{\ell}\mathcal{F}_{\text{st}}(\mathcal{M}_{i})\,,$
(36)

where $\mathcal{M}_{i}$ are $\ell$ 3-manifolds with
$\partial\mathcal{M}_{i}=X_{i}$. This can be seen directly from the string
path integral: the path integral over $\mathcal{M}$ decomposes decomposes into
a sum over free field theories of the form (10) with

$\begin{split}(\gamma,\bar{\gamma})\in\text{Map}(\Sigma\to
X_{i})\,,&\quad\beta\in\Gamma(K_{\Sigma}\otimes\gamma^{*}(K_{X}))\,,\\\
\theta^{a}\in\Gamma(\gamma^{*}(S_{X}^{-1}))\,,&\quad
p_{a}\in\Gamma(K_{\Sigma}\otimes\gamma^{*}(S_{X}))\,.\end{split}$ (37)

Exponentiating $\mathcal{F}_{\text{st}}(\mathcal{M})$ to get the gravitational
partition function gives

$Z_{\text{grav}}(\mathcal{M})=\prod_{i=1}^{\ell}Z_{\text{grav}}(\mathcal{M}_{i})\,.$
(38)

That is, the quantum gravity partition function in the $k=1$ string theory
factorizes when the bulk manifold has a disconnected boundary.

Another consequence is that if $\mathcal{M}$ is a 3-manifold with empty
boundary, then the string partition function precisely vanishes. Indeed, if
$\mathcal{N}$ is a 3-manifold with boundary $X$, then the disjoint union
$\mathcal{N}\sqcup\mathcal{M}$ also has boundary $X$. We thus conclude that

$\mathcal{F}_{\text{st}}(\mathcal{N}\sqcup\mathcal{M})=\mathcal{F}_{\text{st}}(\mathcal{N})\,,$
(39)

so long as $\mathcal{M}$ is compact. However, since
$\mathcal{F}_{\text{st}}(\mathcal{N}\sqcup\mathcal{M})=\mathcal{F}_{\text{st}}(\mathcal{N})+\mathcal{F}_{\text{st}}(\mathcal{M})$
(which must be true of any string theory), we conclude that666This can also be
seen as an immediate consequence of the fact that
$\gamma\in\text{Map}(\Sigma\to\emptyset)=\emptyset$, so that the path integral
is empty.

$\mathcal{F}_{\text{st}}(\mathcal{M})=0\text{ if
}\partial\mathcal{M}=\emptyset\,.$ (40)

In terms of the gravitational partition function, which includes disconnected
worldsheets, we conclude

$Z_{\text{grav}}(\mathcal{M})=1\text{ if }\partial\mathcal{M}=\emptyset\,.$
(41)

The implication, then, is that the bulk theory of quantum gravity described by
$k=1$ string theory on $\text{AdS}_{3}\times\text{S}^{3}\times\mathbb{T}^{4}$
has precisely one state – the ground state – when the bulk manifold is closed.

##  6 Beyond minimal tension

The discussion of the previous sections relied heavily on the fact that the
string theory is described by the $k=1$ limit of the $\text{PSU}(1,1|2)$ WZW
model, which is a worldsheet CFT that only depends on the boundary data of the
target space. However, one may argue that this is a miracle of minimal
tension, and this feature of the string theory may be ‘lifted’ as one deforms
away from this critical value of the string tension. Here, we will argue that
the background independence is at least preserved for a set of continuous
perturbations of the worldsheet theory corresponding to turning on Ramond-
Ramond flux.

From the point of view of the dual CFT, the appropriate deformation is by the
supersymmetric descendant of a twist-2 field in the symmetric orbifold theory.
Specifically, if $\Psi^{\alpha\dot{\alpha}}_{2}$ is the BPS field in the $w=2$
twisted sector of the symmetric orbifold, where $\alpha,\dot{\alpha}$ are
spinor indices of the $\mathfrak{su}(2)_{L}\oplus\mathfrak{su}(2)_{R}$
R-symmetry, then there is a four-dimensional family of exactly marginal
deformations of $\text{Sym}^{N}(\mathbb{T}^{4})$ given by

$\Phi^{A\dot{A}}_{2}(x,\bar{x})=\varepsilon_{\alpha\beta}\varepsilon_{\dot{\alpha}\dot{\beta}}\mathcal{G}^{\beta
A}_{-\frac{1}{2}}\mathcal{G}^{\dot{\beta}\dot{A}}_{-\frac{1}{2}}\Psi^{\alpha\dot{\alpha}}_{2}(x,\bar{x})\,.$
(42)

Here, $\mathcal{G}^{\alpha A},\overline{\mathcal{G}}^{\dot{\alpha}\dot{A}}$
are the $\mathcal{N}=(4,4)$ supersymmetry generators, and the capital roman
indices label the representation with respect to the outer
$\mathfrak{su}(2)_{L}\oplus\mathfrak{su}(2)_{R}$ symmetry of the small
$\mathcal{N}=(4,4)$ supersymmetry algebra. These marginal operators, along
with the $4\times 4=16$ Narain moduli of the $\mathbb{T}^{4}$ lattice,
parametrize the 20 dimensional moduli space which contains
$\text{Sym}^{N}(\mathbb{T}^{4})$.

On the worldsheet, the operators (42) are dual to exactly marginal operators
in the worldsheet sigma model which take the form

$\mathcal{O}^{A\dot{A}}_{2}(z,\bar{z})=\tilde{G}\,^{-}_{-1}\tilde{\overline{G}}\,^{-}_{-1}\int_{X}\varepsilon_{\alpha\beta}\varepsilon_{\dot{\alpha}\dot{\beta}}\mathcal{G}^{\beta
A}_{-\frac{1}{2}}\mathcal{G}^{\dot{\beta}\dot{A}}_{-\frac{1}{2}}\mathcal{V}_{2}^{\alpha\dot{\alpha}}(z,\bar{z},x,\bar{x})\,,$
(43)

where $\mathcal{V}_{2}^{\alpha\dot{\alpha}}$ is the worldsheet dual of
$\Psi_{2}^{\alpha\dot{\alpha}}$ and the operators $\mathcal{G}$ should be
thought of as the corresponding DDF operators on the worldsheet which were
constructed in Naderi:2022bus . It was shown in Fiset:2022erp  that the
worldsheet deformation by $\mathcal{O}_{2}^{A\dot{A}}$ is equivalent to
deforming by $\Phi_{2}^{A\dot{A}}$ in the boundary CFT to second order in
conformal perturbation theory.777This class of deformations have also been
recently shown to admit an integrable structure in the boundary CFT, see
Gaberdiel:2023lco .

Let us briefly argue that how this result structurally holds at all orders in
perturbation theory. Firstly, the BPS operators
$\mathcal{V}_{2}^{\alpha\dot{\alpha}}$ can be written in closed form as
Dei:2023ivl

$\mathcal{V}_{2}^{\alpha\dot{\alpha}}=e^{2\rho+i\sigma+iH}\partial\theta^{\alpha}\overline{\partial}\bar{\theta}^{\dot{\alpha}}\overline{\theta}^{1}\overline{\theta}^{2}\theta^{1}\theta^{2}\delta^{(2)}_{2}(\gamma(z),x)\,,$
(44)

where $\alpha,\dot{\alpha}\in\\{1,2\\}$. Here, we have defined the delta
function

$\delta^{(2)}_{2}(\gamma(z),x):=\delta(\gamma(z),x)\delta(\partial\gamma(z))\delta(\bar{\gamma}(\bar{z}),\bar{x})\delta(\overline{\partial}\bar{\gamma}(\bar{z}))\,,$
(45)

which is a $(-1,-1)$-form valued distribution in the path integral that
demands the criticality condition

$\gamma(y)\sim x+\mathcal{O}((y-z)^{2})\,,\quad y\to z\,.$ (46)

Now, let us ignore the various supercharges appearing in equation (43) as well
as the various indices. At $m^{\text{th}}$ order in worldsheet perturbation
theory, we consider the correlator

$\int_{\mathcal{M}_{g,n}}\Braket{\left(\int\mathcal{O}_{2}\right)^{m}\cdots}\,,$
(47)

where $\cdots$ represents the fields involved in the undeformed correlation
function. The effect of inserting $\mathcal{O}_{2}$ in the correlation
function is to introduce $m$ critical points $y_{i}$ on the worldsheet such
that

$\partial\gamma(z)\sim\mathcal{O}(z-y_{i})\,,\quad z\to y_{i}\,,$ (48)

where the image $\gamma(y_{i})$ is left arbitrary, i.e. is integrated over in
the path integral. Put in other words, if the undeformed correlation function
is a path integral over holomorphic maps $\gamma$ with prescribed behavior,
the deformed path integral at $m^{\text{th}}$ order in perturbation theory
will be an integral over holomorphic maps $\gamma$ with $m$ ‘extra’ simple
branch points. This is precisely the prescription for computing perturbation
theory with respect to twist-2 fields in the symmetric orbifold Lunin:2000yv .

Crucially, every ingredient in the worldsheet path integral is a function only
of worldsheet data. The worldsheet vertex operator dual to the spacetime BPS
state lives in the space

$\mathcal{V}_{2}^{\alpha\dot{\alpha}}\in\Gamma(\gamma^{*}(S_{X}\otimes\overline{S}_{X}))\,.$
(49)

The deformation operator is built out of
$\mathcal{V}_{2}^{\alpha\dot{\alpha}}$ and the DDF operators

$\mathcal{G}_{-\frac{1}{2}}^{\alpha
A}\in\Gamma(\gamma^{*}(S_{X}))\,,\quad\mathcal{G}_{-\frac{1}{2}}^{\alpha
A}\in\Gamma(\gamma^{*}(\overline{S}_{X}))\,,$ (50)

as well as an integral over $X$. Since none of these ingredients depend in any
way on the details of the three-dimensional bulk, we conclude that background
independence is preserved even as we deform away from the orbifold point. Of
course, this is to be expected if one believes that the AdS/CFT correspondence
holds in the full moduli space containing $\text{Sym}^{N}(\mathbb{T}^{4})$.

##  7 Discussion

In this note, we argued that the gravitational partition function of minimal
tension string on $\mathcal{M}\times\text{S}^{3}\times\mathbb{T}^{4}$ depends
only on asymptotic information encoded on the boundary of $\mathcal{M}$ such
as the induced conformal class of metrics and the induced spin structure. This
implies that the second-quantized string theory is background independent. As
a corollary, we argued that the quantum gravity Hilbert space on a closed
hyperbolic three-manifold $\mathcal{M}$ contains exactly one state.

The arguments in this paper, however, were extremely specific to the case of
$k=1$ units of NS-NS flux, where the worldsheet theory admits an extremely
powerful free field realization. While we argued that the background
independence should remain upon continuous deformations of the worldsheet
theory (e.g. when switching on infinitesimal RR flux), it is unclear how the
arguments in this paper would extend to $k>1$. For larger values of $k$, the
definition of the worldsheet sigma model depends intimately on the details of
the bulk manifold away from the boundary. Thus, while we expect background
independence to remain true for larger values of NS-NS flux, the mechanism
whereby background independence appears will likely be much more complicated
and will most likely be invisible from string perturbation theory alone.

### Acknowledgements

I thank Andrea Dei, Matthias Gaberdiel, Kiarash Naderi, Vit Sriprachyakul,
Jakub Vošmera and especially Mykhaylo Usatyuk for stimulating discussions. I
would also like to thank thank Matthias Gaberdiel and Mykhaylo Usatyuk for
helpful comments on a draft of this manuscript, as well as the oranizers of
the KITP program “What is string theory? Weaving together perspectives.” for
their hospitality during the writing of this work. This research was supported
by STFC consolidated grants ST/T000694/1 and ST/X000664/1, and was supported
in part by grant NSF PHY-2309135 to the Kavli Institute for Theoretical
Physics (KITP).

## References

  * (1) J. M. Maldacena, “The Large $N$ Limit of Superconformal Field Theories and Supergravity,” Int. J. Theor. Phys. 38 (1999) 1113–1133, hep-th/9711200. [Adv. Theor. Math. Phys.2,231(1998)].
  * (2) E. Witten, “Anti-de Sitter space and holography,” Adv. Theor. Math. Phys. 2 (1998) 253–291, hep-th/9802150.
  * (3) J. M. Maldacena and L. Maoz, “Wormholes in AdS,” JHEP 02 (2004) 053, hep-th/0401024.
  * (4) S. R. Coleman, “Black holes as red herrings: Topological fluctuations and the loss of quantum coherence,” Nucl. Phys. B 307 (1988) 867–882.
  * (5) D. Marolf and H. Maxfield, “Transcending the ensemble: baby universes, spacetime wormholes, and the order and disorder of black hole information,” JHEP 08 (2020) 044, 2002.08950.
  * (6) J. McNamara and C. Vafa, “Baby Universes, Holography, and the Swampland,” 2004.06738.
  * (7) A. Blommaert, L. V. Iliesiu, and J. Kruthoff, “Gravity factorized,” JHEP 09 (2022) 080, 2111.07863.
  * (8) M. Usatyuk, Z.-Y. Wang, and Y. Zhao, “Closed universes in two dimensional gravity,” 2402.00098.
  * (9) M. Usatyuk and Y. Zhao, “Closed universes, factorization, and ensemble averaging,” 2403.13047.
  * (10) L. Eberhardt, M. R. Gaberdiel, and R. Gopakumar, “The Worldsheet Dual of the Symmetric Product CFT,” JHEP 04 (2019) 103, 1812.01007.
  * (11) L. Eberhardt, M. R. Gaberdiel, and R. Gopakumar, “Deriving the AdS3/CFT2 correspondence,” JHEP 02 (2020) 136, 1911.00378.
  * (12) L. Eberhardt, “$\mathrm{AdS}_{3}/\mathrm{CFT}_{2}$ at higher genus,” JHEP 05 (2020) 150, 2002.11729.
  * (13) Y. Hikida and T. Liu, “Correlation functions of symmetric orbifold from $\mathrm{AdS}_{3}$ string theory,” JHEP 09 (2020) 157, 2005.12511.
  * (14) L. Eberhardt, “Partition functions of the tensionless string,” JHEP 03 (2021) 176, 2008.07533.
  * (15) A. Dei, M. R. Gaberdiel, R. Gopakumar, and B. Knighton, “Free field world-sheet correlators for ${\rm AdS}_{3}$,” JHEP 02 (2021) 081, 2009.11306.
  * (16) B. Knighton, “Higher genus correlators for tensionless AdS3 strings,” JHEP 04 (2021) 211, 2012.01445.
  * (17) L. Eberhardt, “Summing over Geometries in String Theory,” JHEP 05 (2021) 233, 2102.12355.
  * (18) A. Dei, B. Knighton, and K. Naderi, “Solving AdS3 string theory at minimal tension: tree-level correlators,” 2312.04622.
  * (19) C. Beem and A. E. V. Ferrari, “Free field realisation of boundary vertex algebras for Abelian gauge theories in three dimensions,” 2304.11055.
  * (20) N. Berkovits, C. Vafa, and E. Witten, “Conformal field theory of AdS background with Ramond-Ramond flux,” JHEP 03 (1999) 018, hep-th/9902098.
  * (21) E. Frenkel and A. Losev, “Mirror symmetry in two steps: A-I-B,” Commun. Math. Phys. 269 (2006) 39–86, hep-th/0505131.
  * (22) N. Berkovits and C. Vafa, “N=4 topological strings,” Nucl. Phys. B 433 (1995) 123–180, hep-th/9407190.
  * (23) E. Frenkel, A. Losev, and N. Nekrasov, “Instantons beyond topological theory II,” 0803.3302.
  * (24) J. D. Brown and M. Henneaux, “Central Charges in the Canonical Realization of Asymptotic Symmetries: An Example from Three-Dimensional Gravity,” Commun. Math. Phys. 104 (1986) 207–226.
  * (25) E. Witten, “Topological Sigma Models,” Commun. Math. Phys. 118 (1988) 411.
  * (26) E. Witten, “Mirror manifolds and topological field theory,” AMS/IP Stud. Adv. Math. 9 (1998) 121–160, hep-th/9112056.
  * (27) K. Naderi, “DDF operators in the hybrid formalism,” JHEP 12 (2022) 043, 2208.01617.
  * (28) M.-A. Fiset, M. R. Gaberdiel, K. Naderi, and V. Sriprachyakul, “Perturbing the symmetric orbifold from the worldsheet,” JHEP 07 (2023) 093, 2212.12342.
  * (29) M. R. Gaberdiel, R. Gopakumar, and B. Nairz, “Beyond the Tensionless Limit: Integrability in the Symmetric Orbifold,” 2312.13288.
  * (30) O. Lunin and S. D. Mathur, “Correlation Functions for $M^{N}/S^{N}$ Orbifolds,” Commun. Math. Phys. 219 (2001) 399–442, hep-th/0006196.