# Uncertainty in Natural Language Processing: Sources, Quantification, and
Applications

Mengting Hu, Zhen Zhang, Shiwan Zhao, Minlie Huang and Bingzhe Wu  Mengting Hu
and Zhen Zhang are with the College of Software, Nankai University. E-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Shiwan Zhao is an independent
researcher. E-mail<EMAIL_ADDRESS>Minlie Huang is with the Department of
Computer Science, Tsinghua University. E-mail<EMAIL_ADDRESS>Bingzhe
Wu is with Tencent AI Lab. E-mail<EMAIL_ADDRESS>This work has been
submitted to the IEEE for possible publication. Copyright may be transferred
without notice, after which this version may no longer be accessible.

###### Abstract

As a main field of artificial intelligence, natural language processing (NLP)
has achieved remarkable success via deep neural networks. Plenty of NLP tasks
have been addressed in a unified manner, with various tasks being associated
with each other through sharing the same paradigm. However, neural networks
are black boxes and rely on probability computation. Making mistakes is
inevitable. Therefore, estimating the reliability and trustworthiness (in
other words, _uncertainty_) of neural networks becomes a key research
direction, which plays a crucial role in reducing models’ risks and making
better decisions. Therefore, in this survey, we provide a comprehensive review
of uncertainty-relevant works in the NLP field. Considering the data and
paradigms characteristics, we first categorize the sources of uncertainty in
natural language into three types, including input, system, and output. Then,
we systemically review uncertainty quantification approaches and the main
applications. Finally, we discuss the challenges of uncertainty estimation in
NLP and discuss potential future directions, taking into account recent trends
in the field. Though there have been a few surveys about uncertainty
estimation, our work is the first to review uncertainty from the NLP
perspective.

###### Index Terms:

Natural Language Processing, Uncertainty Estimation, Pre-trained Language
Models

##  1 Introduction

Natural Language Processing (NLP) is a multidisciplinary field that
encompasses computer science, artificial intelligence, and linguistics. Its
aim is to develop machines that understand natural language and allow humans
to interact with it using natural language. Benefiting from the development of
deep neural networks (DNNs), NLP technology has a wide range of applications,
including sentiment analysis (SA), machine translation (MT), question-
answering (QA) systems, etc. With the development of pre-trained language
models (PLMs) [1], plenty of NLP tasks can be tackled in a similar manner by
sharing the same paradigm. That is to say, various tasks can be simply
formulated into a classification, regression, generation, etc., problem. A
unified trend is being observed in the development of NLP field, which enables
the efficient utilization of PLMs and promotes knowledge transfer across
tasks.

Though NLP field has achieved great success, it relies heavily on neural
networks. Such a technique is a black box and depends on probability
computation. There must be probable to make mistakes, which is inevitable.
Thus, estimating the uncertainty of neural networks becomes a crucial research
direction, which measures the reliability and trustworthiness of models.
Especially in safety-critical applications like autonomous systems or medical
diagnosis, uncertainties in predictions can have severe consequences if not
appropriately addressed. Here, we demonstrate the importance of uncertainty
through an example in the medical domain. As depicted in Fig. 1, a user
inquires about the potential use of the addictive drug _“Xanax”_ for anxiety,
the system fails to differentiate the intensity of anxiety symptoms, resulting
in a direct affirmation of its usability. This is clearly deemed inappropriate
for patients, considering the grave side effects associated with the drug
_“Xanax”_. However, incorporating a probability of uncertainty would aid in
determining whether to abstain from the response or defer to the expertise of
professionals for a final decision. Some studies have shown that accurately
quantifying uncertainty can help identify situations where the model is
uncertain, thereby improving the reliability and interpretability of the
output [2, 3, 4].

Figure 1: An illustration of NLP systems applied in the medical domain.

Considering the unified trends of NLP and the essentiality of uncertainty, a
systematic review of uncertainty-relevant NLP and the corresponding solutions
is still lacking, which we aim to fill in this survey. Initially, we arise a
research question: what are the sources of uncertainty in the NLP field? Here,
we naturally connect uncertainty with multiple stages of an NLP system and
summarize the sources into _input_ , _system_ , and _output_. More concretely,
natural language input to NLP system is inherently ambiguous and context-
dependent, making it difficult to achieve perfect performance and reliability
in many NLP tasks [5, 6, 7]. Then, the system, such as the network
initialization, architecture, and interior computation, etc., could introduce
randomness, leading to uncertainty. Lastly, the uncertainties of outputs are
due to complicating factors, such as a lack of broader knowledge, unreasonable
model parameters, or specification of task output.

In light of the in-depth analysis of uncertainty sources, we further review
the literature about the uncertainty quantification. Uncertainty estimation is
ubiquitous in DNNs, and a common example is represented by confidence of the
network output. Specifically, the Softmax scores obtained from these networks
provide a direct means of estimating confidence values that are easily
converted to uncertainty scores, e.g. subtracting the confidence from 1.
However, the predictions are typically represented as point estimates [8, 9],
and mistakes are inevitable. Guo et al. [10] point out that DNNs tend to
exhibit excessive confidence, making their confidence scores inaccurate. This
is problematic because under-/over-confident in the network can lead to wrong
decisions and actions based on overconfident predictions. To address this
challenge, researchers have developed various uncertainty estimation methods
for DNNs. This survey categorizes uncertainty estimation methods into three
groups based on different modeling approaches: (1) calibration confidence-
based methods, (2) sampling-based methods, and (3) distribution-based methods.
These methods go beyond traditional confidence level measurements to quantify
the uncertainty in model predictions, identify situations where NLP systems
are uncertain about outputs, and offer insights into model behavior.
Throughout the survey, we discuss the features and challenges associated with
each of these estimation methods.

Recently, the applications of uncertainty estimation in NLP are diverse and
increasing. In this survey, we mainly divide the applications into three
categories, including (1) data filtering and action guidance, which is to
select the required data according to the uncertainty and take the next
action. In typical active learning, filter the data through the uncertain
estimation of the teacher model, and guide the student network to train;
another example is the detection of out-of-distribution data. (2) Improve the
performance and efficiency of the system based on uncertainty, (3) Output
quality assessment, such as the quality assessment of machine translation and
understanding whether the answer of the QA system is trustworthy.

The challenges of uncertainty estimation in applications depend on the problem
setting, including the paradigm of the NLP model and the type of task (e.g.,
classification, generation, regression, etc.). When observing the challenges
faced by the three applications mentioned above, we considered different
paradigms for classification in deterministic estimation methods in this
survey. Recently, with the emergence of large pre-trained models such as T5
[11] and the GPT family [12, 13, 14], which can capture linguistic patterns
and knowledge from massive text data. The capabilities of NLP models in
various paradigms have greatly improved. However, the complexity of large
models makes it difficult to understand the models themselves, which include
high-dimensional spaces, variable lengths of generated text, and expressions
of uncertainty for interpretation of results. These will be described in
detail in the survey [15, 4].

Overall, the main purpose of this survey paper is to systematically review the
advances and challenges of uncertainty in the NLP field. To the best of our
knowledge, our paper is the first to summarize this topic. Concretely,
considering recent trends and paradigms in NLP, we provide a categorization of
uncertainty sources according to the stages of NLP systems. Besides, a
comprehensive review of the uncertainty estimation (a.k.a. quantification)
technique in recent years is further provided. We also summarize the
applications of uncertainty in NLP. Finally, we conclude the challenges based
on the characteristics of text data and the NLP paradigms, highlighting future
challenges and potential trends.

###  1.1 Organization of The Survey

The rest of the survey is organized as follows: In Sec 2, we introduce two
kinds of uncertainty backgrounds in deep learning and illustrate the main
sources of uncertainty in NLP systems in Fig. 2. In Sec 3, we classify
uncertainty estimation techniques based on their modeling approach. We also
abstract the insights behind each category of technology and provide a
comparison of different technologies. Following this, we present commonly used
uncertainty evaluation metrics in NLP. Then, in Sec 4, we summarize the
application directions of uncertainty in NLP under the three major categories
and sort out the detailed literature of specific applications under each
category. We then discuss the current challenges of uncertainty estimation and
potential future research directions in Sec 5 and conclude this survey in Sec
6.

##  2 Uncertainty Sources

In this section, we aim to analyze the uncertainty sources of NLP from the
_input_ , _system_ , and _output_ stages, respectively. In advance, we review
the theoretical background of uncertainty definition.

Figure 2: Illustration of sources of uncertainty. The figure includes the
sources of uncertainty in the interaction of the NLP system. We start from the
three processes of Input, System, and Output to analyze the possible causes of
each uncertainty. It is worth noting that these three parts are interrelated.
As the query passes through the NLP system, due to the combination of neural
networks and different task specifications in complex ways, the type of
uncertainty in the output prediction becomes complicated, including both the
aleatoric and epistemic uncertainty, which are hard to decompose. Thus, we
refer to it as _combined uncertainty_.

###  2.1 Theory Background of Uncertainty

In the field of machine learning, uncertainty is commonly divided into
aleatoric uncertainty and epistemic uncertainty ([3, 16, 17, 6]). In general,
these two types of uncertainty can be summarized as follows:

  * •

Aleatoric Uncertainty It is also known as data uncertainty, which refers to
the uncertainty inherent in data due to its randomness or noise. This type of
uncertainty is irreducible, meaning it cannot be eliminated through model
improvements or tuning. It can arise from a variety of sources, such as noisy
observations, overlapping classes, ground truth errors, inherent randomness,
or other factors that are not entirely predictable.

  * •

Epistemic Uncertainty It is also known as model uncertainty, which is
reducible uncertainty that arises from a lack of knowledge or understanding
about the model itself. This can include uncertainty about the model’s
structure choice, and parameters, which can be reduced by increasing the
amount and quality of training data. Epistemic uncertainty can also result
from out-of-distribution examples, such as new languages or domains, and may
be caused by model structure errors or training procedure errors.

It is worth noting that there is no clear distinction between epistemic
uncertainty and aleatoric uncertainty, and these types of uncertainty can even
be mutual [17]. In particular, it may not always be clear which type of
uncertainty is dominant in a given NLP system, and the two types of
uncertainty can interact in complex ways, thus making combinations complex and
difficult to decouple. Nonetheless, it is important to understand the source
and classification of uncertainty, e.g., in MT, decoupled uncertainty can be
used to infer whether predictions originate from noisy and ambiguous
references, or out-of-distribution examples or noisy annotations [18].

To better understand the sources of uncertainty in NLP systems, it is
necessary to incorporate NLP usage scenarios. To this end, we consider human-
machine interactions in NLP tasks to mine factors that may contribute to
uncertainty. Ultimately, we have determined three main sources of uncertainty:
(1) Input text entered by a user, (2) System composed of models, and (3)
Output returned to the user. In the following sub-parts, we will discuss these
three sources. Meanwhile, in Fig. 2, we provide a detailed source overview.
It’s worth emphasizing that three sources tend to be interconnected, and
addressing one may help to mitigate uncertainty in the others.

###  2.2 Uncertainty Source from Input

During the usage of an NLP system, a user would feed input to it, expecting to
obtain processed results or interactive feedback. Due to the language’s
intrinsic ambiguity and unknown query, the input itself contains uncertainty.
Formally, assuming that the model parameters $\mathbf{W}$ have been trained on
a dataset $D=\\{(\mathbf{x}_{1},y_{i})\\}^{N}_{i=1}$, and there exists a
correct and specific hypothesis space $\mathcal{H}$, if an input text falls
within the hypothesis space $\mathcal{H}$, then the input adheres to the data
distribution. However, if an input contains inherent uncertainty or noise of
the language, it may fall on the boundary of the hypothesis space or even
outsides [19]. In such cases, the system is prone to making incorrect
judgments. Consequently, it is necessary to capture and quantify this
resulting uncertainty. As depicted in the upper-left of Fig. 2, we distinguish
common sources of uncertainty in inputs from two perspectives, including the
language intrinsic and unknown query, respectively.

####  2.2.1 Language Intrinsic

Language inherently contains ambiguity that leads to uncertainty [20]. That is
to say, ambiguity is a natural feature of language that arises from
inconsistencies and contextual factors in the text [21]. For instance, the
word _“apple”_ has various meanings when appearing in different contexts, i.e.
_“I like to eat apple”_ and _“Steven Jobs built up Apple“_. Inconsistencies
can take the form of incomplete sentences or expressions [21], while
contextual semantics encompass the completeness and accuracy of information
expression, as well as social and cultural aspects and differences between the
unique context of text production (including features of space, time,
authorship, etc.) and multiple interpretation contexts [22, 20, 23].

Natural language noise contains out-of-vocabulary (OOV) and distraction. OOV
indicates spelling errors or emerging words that are unseen tokens for a
running NLP system. These would result in unsureness. Besides, distracting
refers to the presence of some words in a sentence that are not related to the
meaning of the sentence. This kind of noise is ubiquitous in informal texts
[24, 25, 26]. For example, Liu et al. [25] in entity recognition find that a
part of the original sentence retains enough words to express the
relationship, and the rest has many irrelevant words that can be regarded as
noise that may hinder the performance of the extractor. Kadavath et al. [26]
set different prompt templates, such as helpful, incorrect, and distracting
hints, and find that distracting hints in natural language generation (NLG)
also lead to deviations in the generated answers.

As mentioned above, natural language is inherently complex, ambiguous, and
context-dependent. It can be expressed in different ways to convey the same
ideas or concepts. Therefore, the mapping relationship in NLP is also a source
of uncertainty [23]. This complexity can result in multiple input-output
mapping scenarios, such as one-to-one and many-to-one mapping. In such cases,
the model must select an output from several possible ones, which can be
challenging and uncertain. For instance, in MT, the same source sentence can
have multiple semantic equivalent translations, resulting in multiple nature
of machine translation learning tasks [22, 23, 27]. Similarly, Text-to-SQL
needs to deal with many-to-one cases, indicating that multiple input texts
correspond to the same SQL query [28]. This is also caused by the uncertainty
of natural language.

####  2.2.2 System Unknown Query

In the application of the actual world, systems usually encounter an unknown
query differently distributed from training data [16, 29]. Systems tend to
produce unreliable or even catastrophic predictions, thereby harming the trust
of users. The uncertainty of the field is derived from the model that the
model cannot explain the out-of-distribution (OOD) sample due to the lack of
external knowledge. The source of this uncertainty is the input data extracted
with the unknown subspace. Although DNNs can extract the knowledge in the
domain from the domain migration sample, they cannot extract the knowledge
samples in the domain from the external sample [16]. For example, QA systems
deployed in search engines or individual assistants require elegantly
processing OOD input, because users usually propose a problem that is not
within the scope of system training distribution [26, 30]. Another example is
to make entailment judgments for breaking news articles in search engines
[31]. If an input is not acceptable, the model needs to acknowledge its
uncertainty and turn to humans or better (but more costly) models [32].

###  2.3 Uncertainty Source from System

The sources of uncertainty induced by NLP systems mainly originate from model
blocks, which are usually introduced in the design and training of neural
networks. The design of DNNs involves explicit modeling of the network
architecture and its stochastic training process. The assumptions made about
the problem structure, based on the network design and training, are referred
to as inductive biases [33].

####  2.3.1 Model Structure

Since neural networks are manually designed, their structures are also a
source of uncertainty. In other words, changing the structure of networks
might alter decision-making, bringing randomness to model training and
inference stages. Here we discuss from the views of _architecture_ and
_parameter scale_.

In an end-to-end model, many factors are relevant to model architecture, such
as the dimension of hidden states, and interior neuron connections, directly
affect the performance of the network and thus introduce uncertainty [34, 33,
16] When studying complex AI systems, it needs to take into account the
predictions of several independent models in downstream tasks [35] due to the
effect of pipeline propagation. The cost and risk of any decision-making
process that produces different errors need to be balanced. This creates
uncertainty when confidence in one module’s predictions is allowed to have an
impact on the next module. For this reason, if the uncertainty and confidence
scores can be reliably interpreted as probabilities, the rules of
probabilistic calculus can be applied, allowing one system to abort the
decision if its predictions are not confident enough [36, 37, 38]. This is a
useful property in many situations. For example, in the prediction of an
information extraction system [6], there are locations where the information
extraction system extracts useful information and uses confidence to check
whether the extracted information is meaningful. After this, another model
makes a final prediction of the extracted value of the field.

The parameter scale is a critical factor that contributes to uncertainty in a
fixed model structure [39]. The size of a model can have a significant impact
on the level of uncertainty. In particular, larger models tend to possess a
greater number of parameters and increased complexity. This enables them to
better capture the intricacies of the training data, leading to reduced error
on the training set. However, the larger parameter count also renders these
models more susceptible to overfitting [10], potentially causing poor
performance when presented with unseen data. On the other hand, smaller models
have fewer parameters and lower complexity, making them more prone to
underfitting the training data. Consequently, these models may exhibit higher
error rates on the training set but have the potential to generalize better on
previously unseen data, thereby potentially reducing uncertainty. It is
important to recognize that the relationship between model size and
uncertainty is multifaceted, and a comprehensive evaluation should consider
various factors such as model complexity, dataset size and characteristics,
and the regularization techniques employed during training.

####  2.3.2 Model Training

During the model training, we distinguish uncertainty sources into the dataset
and hyperparameter views, respectively. Initially, to learn an intelligent
system, high-quality training data is essential. Yet due to the different
language abilities of annotators, texts in the dataset pose various challenges
for them [40]. As a result, this directly affects the correlation between
samples and corresponding labels. Uncertainty arises whenever there are
multiple possible interpretations of the data, but the knowledge or
information to definitively choose one of them is not available [41]. Then,
uncertainty may be part of the pre-processing stage. For example, NLP tools
used for pre-processing can introduce uncertainty [42]. More concretely, Part-
of-speech taggers are often trained on data from historical periods. Yet,
recent data is often not available. Making results for recent data is possibly
unreliable, leading to uncertainty, and this uncertainty is usually
irreducible. Furthermore, Bender et al. [43] discuss ethical issues raised by
PLMs, including issues of bias and fairness. They argue that language models
encode biases and assumptions in the training data, which perpetuate social
inequalities. This highlights sources of uncertainty related to the ethical
implications of PLMs, and the need to consider them carefully for
transparency.

The setting of hyperparameters, such as batch size, learning rate, and epoch
number, is also random, resulting in different local optimal solutions and
different final models [16]. Dodge et al. [44] study the fine-tuning process
of PLMs and find that hyperparameters and tuning such as weight
initializations, data orders, and early stopping are critical to achieving
high performance on downstream tasks. This highlights that under the large
model, fine-tuning the model requires more strategies and hyperparameter
requirements and adjustments, which will also introduce new uncertainty in the
model framework.

###  2.4 Uncertainty Source from Output

Since making mistakes is inevitable for an NLP system, its output is not fully
trusted, i.e. presenting uncertainty. Here we summarize the uncertainty of the
output according to the paradigms and the nature of the task, namely
_classification_ , _sequence labeling_ , _generation_ , and _regression_
paradigms. It is worth noting that although the task outputs are quite
different, their sources of uncertainty are often combined and complex. That
is to say, uncertainties sourced from the input text and model itself will
affect the output. Therefore, in the right part of Fig. 2, we do not
specifically define whether the uncertainty of output is epistemic or
aleatoric uncertainty.

####  2.4.1 Classification Paradigm

The output of the classification paradigm is usually connected with the number
and granularity of categories. Firstly, as the number of categories increases,
the classification task becomes more difficult. There may be more errors or
misclassification space, which enlarges uncertainty. This is because models
may have difficulty distinguishing between similar categories, or be more
error-prone due to the inherent ambiguity of the classification task.
Secondly, the classifying granularity (e.g., classifying at words or sentences
level, defining hierarchical or multi-label categories) may introduce
uncertainty. More concretely, word-level classification may have variability.
For example, a word like _“Washington”_ could be classified as a person or
place depending on the context, and this classification itself may be vague or
indeterminate. Classification at a coarser level (e.g. classifying entire
sentences or documents) depends on more factors (e.g. contribution of each
token, contextual semantics, length of sentence/document, etc.).

####  2.4.2 Sequence Labeling Paradigm

Sequence labeling aims to assign a tagger to each token in the given input
text. It needs to consider contexts to determine the dependencies of label
sequences. Specifically, for sequence dependencies, the labels assigned to
elements can depend on the labels assigned to adjacent elements in the
sequence. This can lead to non-determinism because (1) due to the inherent
non-determinism of natural language itself, the label assigned to an element
may be ambiguous, which needs to be determined according to the context and
neighboring elements in the sequence [45, 46]. (2) The labels assigned to
adjacent elements in the sequence should be consistent and follow a certain
pattern. However, this can be challenging as the model may struggle to capture
and model complex dependencies and relationships between adjacent elements
[47].

In addition, the boundary, e.g. the start and end positions of entities in
named entity recognition (NER) tasks, is also crucial. Correct entity
boundaries are effective in mitigating error propagation in entities that are
linked to knowledge bases [48]. Some studies consider entity boundary
detection as a subtask in NER [49, 50]. In summary, analyzing the prediction
uncertainty in sequence labeling can be roughly divided into boundary
uncertainty and boundary entity classification uncertainty. Both of these are
inseparable from boundary detection and are dependent on the sequence. This
analysis increases the rationality and reliability of the output results.

####  2.4.3 Generation Paradigm

A more challenging output is the generation paradigm, where the uncertainty of
the prediction is affected by uncontrollable factors. The output space of
generating natural language has $\mathcal{O}(|\mathcal{T}|^{N}$) dimensions.
For example, MT models have hundreds of millions of parameters, and the search
space is exponentially large, which poses great challenges for search
algorithm. We usually observe only one reference for a given source sentence,
leading to uncertainty in the final output. At present, tools and strategies
can be borrowed and combined from machine learning and statistics [51, 52, 10,
23]. Beam search is an effective search strategy, but some external
uncertainty (such as the quality of training data) will lead to large beam
performance degradation [23].

One of the goals of NLG is to ensure that the generated text conveys the
intended semantic content of the sentence. However, in decision-making tasks
that rely on NLG, e.g. QA system [4], it is crucial to ensure invariance to
the output space, but this is often not explicitly specified in the model. The
meaning of the generated text is especially important in terms of its beliefs.
While a system may be reliable even when generating output using many
different expressions, it may be less reliable when answering questions with
inconsistent meanings.

Another key purpose of NLG is to ensure the language quality. which is
typically evaluated based on several factors, including adequacy, fluency,
readability, and variation [53]. A recent example of an effective strategy is
reinforcement learning from human feedback (RLHF) [54]. But it has been found
to be wrongly calibrated [26]. This is not surprising because RL fine-tuning
tends to collapse language model predictions to the behavior that yields the
highest reward, which raises the inherent problem of whether the model’s
output is biased or unfair. Furthermore, the length of the input text can vary
widely [55, 56], leading to uncertainty in the output. This particularly
exists for longer sequences, where the joint likelihood decreases due to the
conditional independence of labeling probabilities [4]. Rare words also
contribute to this problem because they have small probabilities and therefore
occur less frequently in the sequence [57, 23].

####  2.4.4 Regression Paradigm

The uncertainty source in the classification output can be easily explained. A
discrete probability distribution over the classifier’s output is naturally
provided through a softmax layer. However, in the regression paradigm, the
output is a single numerical value in continuous target spaces [58]. The
representation of the output becomes important, specifically the range of the
output. For example, FLORES [59] divides 0-100 points into ten intervals, each
representing different translation quality. In semantic text similarity
assessment [60], the objective is to predict a similarity score for a sentence
pair $({S}_{1},{S}_{2})$ within the range $[0,5]$. A score of 0 indicates
complete dissimilarity, while a score of 5 signifies the sentences are equal
in meaning. Calibrated regression [61] extends the calibration method of
classification to regression. It applies the obtained algorithm to the
Bayesian deep learning model and calibrates the confidence score in the
$[0,1]$ interval.

Figure 3: Rationale visualization of three types of uncertainty modeling. For
a given input sample $\mathbf{x}$, each method provides a prediction y whose
uncertainty is quantified as ${u}$. (a) Calibration-based uncertainty
representation, (b) Sampling-based uncertainty estimation method, (c)
Distribution-based uncertainty estimation method. $\Xi$ denotes a specific
distribution. The mean $E$ and variance $Var$ are only used to keep the
visualization simple, there are other quantifications in practice.

##  3 Uncertainty Estimation

Summarizing the sources of uncertainty in NLP systems can help reduce
uncertainty, however, accurately distinguishing uncertainty types is
challenging [16]. In this section, We first introduce the background of
uncertainty estimation techniques, Then, As shown in Fig. 3, according to the
modeling characteristics, we propose a taxonomy of uncertainty estimation in
NLP, and the specificity of linguistic uncertainty modeling.

###  3.1 Uncertainty Estimation Background

Modern neural networks are parameterized by a set of model weights
$\boldsymbol{W}$, providing a formalization of uncertainty in the BNN case,
and these are sufficient statistics
$\boldsymbol{\omega}=({{\mathbf{W}}_{i}})^{L}_{i}$. For a given dataset
$\mathcal{D}=\\{x_{i},y_{i}\\}^{N}_{i=1}$ with distribution
${{p}}(\mathbf{x},\mathbf{y})$, for a classification model
${{P}}({y}^{\ast}=\omega_{c}|{x}^{\ast},\mathcal{D})$ trained on
$\mathcal{D}$, the distribution on prediction ${y}^{\ast}$ is described as:

${{P}}({y}^{\ast}=\omega_{c}|\mathbf{x}^{\ast},\mathcal{D})=\int\underbrace{{P}({y}^{\ast}=\omega_{c}|\mathbf{x}^{\ast},\boldsymbol{\theta})}_{\text{(a)
Data}}\,\underbrace{{p}(\boldsymbol{\theta}|\mathcal{D})}_{\text{(b)
Model}}\,d\boldsymbol{\theta},$ (1)

where the aleatoric uncertainty is formalized as the posterior probability
${{P}}({y}^{\ast}=\omega_{c}|\mathbf{x}^{\ast},\boldsymbol{\theta})$ of over
class labels for a given parameter, while
${{p}}(\boldsymbol{\theta}|\mathcal{D})$ is used as the posterior distribution
of the model parameters, describing the uncertainty of the model parameters,
given a dataset $\mathcal{D}$.

However, ${{p}}(\boldsymbol{\theta}|\mathcal{D})$ is intractable using Bayes’
rule posterior distributions, and it is necessary to use variational inference
methods to achieve ${{p}}(\mathbf{\boldsymbol{\theta}}|\mathcal{D})\approx
q(\boldsymbol{\theta})$ [62, 63, 64]. In the uncertainty estimation method,
three ways are mainly considered, which will be placed in the next subsequent
section.

###  3.2 Uncertainty Estimation Methods

After reviewing the uncertainty estimation methods commonly used in recent NLP
tasks, we generally divide them into three types, namely, calibration
confidence-based methods, sampling-based methods, and distribution-based
methods. We provide the usage and hierarchical classification of these methods
in Fig. 4, and below we summarize from each of these three types.

Figure 4: An overview of the taxonomy of uncertainty estimation techniques.
The inner circle represents uncertainty modeling methods, the outer circle
represents actual uncertainty methods, and some methods are represented by
abbreviations.

####  3.2.1 Calibration Confidence-based Methods

Calibration methods aim to correct the reliability of the uncertainty
estimates provided by a model. The basic idea is to measure the accuracy of
the predicted probabilities against the true probabilities. As shown in Fig.
5, if the predicted probabilities are well calibrated, then they will
accurately reflect probabilities, and the model will be considered reliable.
It is worth noting that the probability distribution can be obtained by a
single forward pass of the network, also including some post-processing
methods. We introduce below two commonly used methods of expressing
uncertainty, as well as calibration methods in NLP research.

Softmax Response. Confidence method is a widely used uncertainty estimation
method. It is based on the idea that the predicted category with the highest
probability is the most likely category, and the uncertainty can be
represented by Softmax Response (SR). There are also methods based on the
difference (SD) between the top two values of the Softmax output, with a
slight difference indicating uncertainty and a significant difference
indicating confidence. Given an input ${\mathbf{x}^{(i)}}$, the uncertainty
describe as:

$u_{SR}(\mathbf{x}^{(i)})=1-{\mathop{\mathrm{max}}_{c\in
C}\;p(y^{(i)}=c|\mathbf{x}^{(i)})}.$ (2)
$u_{SD}(\mathbf{x}^{(i)})=u_{SR}(\mathbf{x}^{(i)})-\mathop{\mathop{\mathrm{max}}_{c_{2}\in
C}}_{c_{2}\neq c}\,p(y^{(i)}=c_{2}|\mathbf{x}^{(i)}).$ (3)

Entropy-based. Prediction entropy, which can be understood as the total
uncertainty of the output distribution. When the certainty of the output
distribution is higher, the entropy value is lower. Maximum entropy is reached
when all outcomes have equal probability. In other words, entropy is highest
value when the model is unable to distinguish between different outcomes. The
predictive entropy at point $\mathbf{x}^{(i)}$ is equal to the conditional
entropy of the output random variable $Y$:

$PE(\mathbf{x}^{(i)})=-\int
p(y|\mathbf{x}^{(i)})\,{\mathrm{ln}}\,p(y|\mathbf{x}^{(i)})dy.$ (4)

In NLP research, the entropy value can represent a variety of uncertain
information, such as the N-grams word entropy value as a heuristic feature to
select domain adaptation data [65], the entropy value at the sentence level
can represent semantic uncertainty [4], The composition of information entropy
in NLP is diverse and complex, and these issues need to be considered when
using entropy values.

Recently, the problem of overconfidence can arise in DNNs [10], often
requiring calibration to obtain reliable confidence. A simple but effective
calibration method is Temperature scaling (TS), which was originally
introduced into the machine learning community as a tool for knowledge
distillation [66]. Specifically, TS helps introduces a temperature parameter
$T>0$ and generates a calibrated prediction vector by mapping:

$\mathbf{q}^{(i)}=\sigma_{SM}(\frac{\mathbf{z}^{(i)}}{T}),$ (5)

where $\sigma_{SM}(\mathbf{z})=e^{\mathbf{z}}/\sum_{k=1}^{K}e^{z_{k}}$ refers
to the logits $\mathbf{z}$ pass the Softmax function. With
$T=1,\mathbf{q}^{(i)}=\hat{\mathbf{p}}^{(i)}$, when $T>1$, the entropy of
$\mathbf{q}^{(i)}$ increases, which helps reduce confidence $s^{(i)}$ and
combat overconfidence, and larger $T$ will make the distribution flatter.
Similarly, $T<1$ reduces entropy and increases confidence, making the
distribution sharper to help with underconfidence predictions. The temperature
parameter $T$ is trained with NLL on the validation set. TS acts as a post-
processing calibration method, treating each prediction independently without
explicitly taking contextual information into account. Therefore, it’s
important to consider whether token-level or sequence-level calibration is
more appropriate for the specific NLP task at hand.

Figure 5: Illustration of uncertainty representation method based on
calibration confidence. a) Calibration curve, the closer to the perfect curve,
the better the confidence calibration. b) Reliability diagram combined with
ECE calibration indicators.

In addition, there are some implicit calibration methods to combat
overconfidence, such as Label smoothing and Re-weighting techniques [67, 68,
69]. Label smoothing is a regularization technique that replaces the one-hot
target vector $\mathbf{y}^{(i)}$ with a weighted combination of targets that
have a uniform distribution. The amount of smoothing is controlled by an
$\alpha$ parameter. In the $K\\!-\\!class$ setting, the true class represented
by the value 1 is replaced by $1-\alpha$, and the other classes are assigned
the value of $\frac{\alpha}{K-1}$. Label smoothing has been shown to have
several benefits for the model predictions. It implicitly calibrates model
predictions similar to TS and does not require post-operation quantification.
Similarly, reweighting techniques such as Focal loss can also reasonably
reduce the sharpness, and thus improve the calibration effect and performance
[70, 67]. By increasing the entropy of the target distribution, label
smoothing and Focal loss prevent the model from producing extremely low-
entropy predictions.

Conformal Prediction (CP) [71] is a framework for constructing calibrated
predictive models that provide a measure of confidence for each prediction.
When CP is formulated in terms of a set of predictions $\mathcal{C}(X_{n+1})$,
it provides finite-sample, distribution-free guarantees for events in which
$\mathcal{C}$ contains $Y_{n+1}$. CP is based on hypothesis testing, where for
a given input $x$ and possible output $y$, a statistical test is performed to
accept or reject the null hypothesis that the pairing $(x,y)$ is correct. It
can be seen as a post-hoc calibration method, which means that it does not
require access to the training data during the calibration process. Instead,
it uses the model’s predictions on a validation set or a hold-out set to
construct a set of prediction regions or intervals.

Challenging: How to solve the trade-off between accuracy and calibration? As
increasing the accuracy may result in overfitting and poor calibration, while
increasing the calibration may result in underfitting and poor accuracy.
Finding the optimal balance requires careful consideration of the model
complexity and the application requirements. Furthermore, NLP tasks often
involve making predictions at different levels of granularity, such as
sentence-level, document-level, or token-level predictions. Uncertainty
quantification of texts with different granularities is also worth thinking
about.

####  3.2.2 Sampling-based Methods

Sampling methods provide a representation of the target random variable, which
can be divided into parametric and predictive sampling, from which implicit
conditional distributions of the posterior model parameters can be derived.
Bayesian variational inference approximates the posterior distribution of
model parameters. It optimizes a variational objective function. This method
estimates uncertainty in model predictions by sampling from the approximate
posterior distribution. For instance, variational inference methods
approximate intractable posterior distributions [72, 62] by optimizing a
family of tractable distributions. A commonly used method is MC dropout [73],
which involves randomly dropping out some neurons in the network during
testing. This technique results in a different prediction for each dropout
configuration. These methods are based on Markov Chain Monte Carlo and further
extensions [74]. Furthermore, sampling-based methods also include ensemble
methods. These methods involve training multiple models with different
initializations or architectures and combining their predictions to make a
final prediction.

Monte Carlo dropout (MCD) is the typical sampling method. Specifically, the
MCD-based method performs $M$ stochastic forward passes with activated
dropout, explicit ensembling to approximate integral Eq. 1:

${{P}}({y}^{(i)}|\mathbf{x}^{(i)},\mathcal{D})\approx\frac{1}{M}\sum_{i=1}^{M}{{P}}({y}^{(i)}|\mathbf{x}^{(i)},\hat{\boldsymbol{\theta}}^{(i)}),\quad{\hat{\theta}}^{(i)}\sim{{q}}(\boldsymbol{\theta})$
(6)

Each ${{P}}({y}^{(i)}|\mathbf{x}^{(i)},\hat{\boldsymbol{\theta}}^{(i)})$
sampled from the distribution ${{q}}$ is a categorical distribution of class
labels on a given input x. It visualizes as a point on the simplex [8].

Therefore, these methods construct an implicit conditional distribution
simplex by sampling, and the characteristic is that whether a certain answer
can be determined through the sharpness of the simplex, that is, the
expression containing uncertainty. Given such a collection of distributions,
it is expected that the entropy of the distribution $P$ will indicate the
uncertainty of the prediction. It should be noted that although the
uncertainty expressed by entropy is effective, it cannot be directly
distinguished as aleatoric uncertainty or epistemic uncertainty. To address
this, measures such as mutual information can be used to estimate prediction
uncertainty due to epistemic uncertainty [75, 76].

In addition, Bayesian active learning by disagreement (BALD; [77]) can be seen
as a way to quantify uncertainty based on MCD:

$u_{\mathrm{BALD}}=-\sum_{k=1}^{K}\overline{p}_{c}\;{\mathrm{log}}\;\overline{p}_{c}+\frac{1}{M}\sum_{c,m}p_{c}^{t}\;{\mathrm{log}}\;p_{c}^{t}$
(7)

From a Bayesian perspective, MCD is a way of approximating Bayesian inference
in neural networks [73]. It can be understood as placing a prior distribution
on the weights of the neural network, and then using dropout to sample from
the posterior distribution.

Ensemble Methods. An alternative approach to Bayesian approximation involves
creating ensembles of multiple independent deterministic neural networks. The
technique involves training $M$ neural network classifiers, with different
models randomly initialized and optimized individually, and combining their
networks outputs to form a single classification function $f(x):X\rightarrow
Y$, For example, this can be achieved by simply averaging the predictions of
the members:

$f(x):=\frac{1}{M}\sum_{i=j}^{M}f_{j}(x).$ (8)

The premise behind this is to leverage diverse perspectives to improve the
generalization of the model, on the basis that a group of decision-makers is
typically more effective than a solitary one. Ensembles have been found to be
particularly effective in uncertainty estimation networks in neural systems.
Moreover,to measure different sources of uncertainty, mutual information (MI;
[77, 75]) between ensemble predictions and their parameters is often used in
NLP models [78, 79, 80]. Furthermore, MCD can be thought of as an ensemble
technique that simulates different models using dropout [34], wherein a set of
independently trained networks with their own weights are generated. By
aggregating the predictions from these networks, we can obtain more accurate
and robust predictions. In addition, recent work has achieved _“better”_
uncertainty by combining the expressive power of ensemble methods with MCD
[81]. It has been demonstrated that sampling-based methods exhibit inherent
calibration and robustness to unknown data representations [82].

Challenging: How to design methods to reduce time and computational cost? A
set of models with varying initializations has the potential to explore
multiple modes of the parameter space [83]. Reducing sampling time and
computational cost is a challenging task in sampling-based uncertainty
estimation models. It is important to maintain the diversity of individual
models while achieving these goals.

####  3.2.3 Distribution-based Methods.

As shown in Fig. 3, a distribution-based model is also a deterministic model,
but requires a specific distribution for uncertainty modeling. In general,
distribution-based models need to pass some prior/posterior assumptions to
allow the model to learn distribution information during training. The
following are two commonly used examples of distribution-based uncertainty
modeling.

Dirichlet-based Uncertainty Models (DBU) utilize logits and construct a
Dirichlet distribution. Unlike standard (Softmax) neural networks, this model
predicts the parameters of the Dirichlet distribution instead of point
estimates. The density of the Dirichlet distribution is defined as

$\displaystyle{\rm{Dir}}(\mathbf{p}^{(i)}|\boldsymbol{\alpha}^{(i)})=\frac{1}{B(\boldsymbol{\alpha}^{(i)})}\prod_{c=1}^{C}p_{c}^{(\alpha^{(i)}_{c}-1)},$
(9)

where $B(\boldsymbol{\alpha}^{(i)})$ is the $C$-dimensional multinomial beta
function.

Specifically, the $P(y^{(i)}|\mathbf{x}^{(i)},\boldsymbol{\theta})$ obtained
by the neural network is a categorical distribution
${\mathrm{Cat}}(\mathbf{p}^{(i)})$, which is a point on the simplex, and the
Dirichlet distribution is a prior distribution over categorical distribution,
which is parameterized by its concentration parameters, and the epistemic
uncertainty on $\mathbf{x}^{(i)}$ can be expressed by
${q}^{(i)}={\mathrm{Dir}(\boldsymbol{\alpha}^{(i)})}$, so it can be
interpreted as the distribution of the categorical distribution.

${{p}^{(i)}_{c}}\\!=\\!\frac{{\alpha}^{(i)}_{c}}{\alpha^{(i)}_{0}},\;\quad
y^{(i)}\\!=\\!\mathop{\mathrm{arg\,max}}\limits_{c\in
C}\left[{{p}^{(i)}_{c}}\right].$ (10)

There are currently some studies using the Dirichlet distribution to
quantitative uncertainty [8, 84, 85, 86]. The advantage is that the network
can obtain uncertainty estimates once forward. In addition, this
parameterization allows the calculation of closed-form classical uncertainty
measures [9], such as the differential entropy of Dirichlet distribution
$m^{(i)}_{\mathrm{diffE}}=H({\mathrm{Dir}(\boldsymbol{\alpha}^{(i)})})$ or
mutual information $m^{(i)}_{\mathrm{MI}}=I({y^{(i)},(\boldsymbol{p}^{(i)})})$
[8].

Moreover, constructing distributions by modeling networks is also a
deterministic approach. For example, modeling a Dirichlet distribution on
class probabilities rather than the point estimate of a Softmax output [8, 84,
85].

Gaussian-based Uncertainty Models (GBU). Parameterizing uncertainty
information using the mean and variance of a Gaussian distribution is a common
approach used in various fields. Specifically, referring to $\mu(\mathbf{x})$
and $\sigma(\mathbf{x})$ as functions parameterized by $W$, the output mean
and standard deviation are computed for input $\mathbf{x}$, assuming
$y\sim\mathcal{N}(\mu(\mathbf{x}),\sigma(\mathbf{x})^{2})$ for the data
generating process in the regression setting, and for the logits vector
$\mathbf{z}$ in the classification setting samples are then converted to
probabilities using a Softmax operation. This process can be described as The
process is described as:

$\displaystyle\mathbf{u}\sim\mathcal{N}(\boldsymbol{\mu}(\mathbf{x}),\boldsymbol{\sigma}(\mathbf{x})^{2}),$
(11) $\displaystyle\boldsymbol{p}={\mathrm{Softmax}}(\mathbf{u}),$
$\displaystyle y\sim{\mathrm{Categorical}}(\boldsymbol{p}).$

This method models uncertainty as a Gaussian distribution, where the mean
represents the most likely value or prediction, and the variance represents
the level of uncertainty or variability. By embedding words as Gaussian
distributional potential functions in an infinite dimensional function space,
Vilnis et al. [87] not only maps word types to vectors, but also to soft
regions in space, allowing uncertainty modeling of meaning and metaphor and
providing a rich geometry of the latent space.

In NLP systems, Gaussian distributions and Mahalanobis distance can be used
together in uncertainty estimation techniques [88, 89]. Gaussian distributions
are used to model variables, while Mahalanobis distance are used to compare
simulated distributions with actual observed data and quantify the distance
between them. For example, measure the Mahalanobis distance between a test
instance and the closest class conditional Gaussian distribution to estimate
uncertainty:

$u_{MD}={\min\limits_{c\in
C}}(h^{(i)}-\mu_{c})^{T}\Sigma^{-1}(h^{(i)}-\mu_{c}),$ (12)

where $h^{(i)}$ denotes the hidden representation of the $i-th$ sample.

Furthermore, the Bayesian neural network can approximate the distribution of
the function by learning the parameter distribution of some models, and the
Gaussian processes (GPs) non-parametric Bayesian model directly uses the
function $f$ to approximate the distribution of the function [90], which can
measure the uncertainty of the model. Formally, a Gaussian process can be
defined as a set of random variables in which any limited number of variables
follow a joint Gaussian distribution. To fully specify GP, two functions are
needed, the mean function $m(\mathbf{x})$ and the covariance function
$k(\mathbf{x},\mathbf{x}^{\prime})$. If a function $f$ is based on GPs
distribution of these two functions, it can be defined as:

$\displaystyle m(\mathbf{x})=\mathbb{E}[f(\mathbf{x})]$ (13) $\displaystyle
k(\mathbf{x},\mathbf{x}^{\prime})=\mathbb{E}[(f(\mathbf{x})-m(\mathbf{x}))(f(\mathbf{x}^{\prime})-m(\mathbf{x}^{\prime}))]$
$\displaystyle
f(\mathbf{x})\sim\mathcal{GP}(m(\mathbf{x}),k(\mathbf{x},\mathbf{x}^{\prime})),$

where $m(\mathbf{x})$ is the mean function, which is usually the 0 constant,
and $k(\mathbf{x},\mathbf{x}^{\prime})$ is the kernel or covariance function,
which describes the covariance between values of $f$ at the different
locations of $k(\mathbf{x}$ and $\mathbf{x}^{\prime})$.

GPs are an alternative kernel-based framework that provides competitive
results for point estimation [91, 92, 93], and they explicitly epistemic
uncertainty in data and predictions. This makes GPs ideal when well-calibrated
uncertainty estimates are required. A commonly used approach to uncertainty
estimation using GPs is the posterior predictive distribution. The posterior
predictive distribution is the distribution of the predicted value of the
output variable at any given input point, given the observed data. The mean of
the posterior predictive distribution gives the predicted value, while the
variance provides a measure of uncertainty. The larger the variance, the more
uncertain the forecast. However, the time complexity of GPs increases with the
size of the data, which makes it intractable in many practical applications.

Challenging: How to make reasonable assumptions about different distributions
in distribution-based uncertainty estimation modeling? Distribution-based
uncertainty estimation modeling often requires making assumptions about the
underlying distribution of the data, which can be challenging when dealing
with complex or multi-modal data. Developing techniques for selecting
appropriate distributions and assessing their validity is necessary to ensure
the accuracy of the model and the reliability of the uncertainty estimates.

###  3.3 Uncertainty Estimation Metrics

####  3.3.1 Calibration Metrics

The calibration metric assesses the consistency of the classifier in producing
reliable predictions. Fig. 5 can be intuitively visualized by (a) Calibration
curves and (b) Reliability diagram. This part focuses on the following metrics
for evaluating the calibration of predictions produced by deep learning
classifiers using uncertainty estimation techniques.

Expected Calibration Error (ECE) [10]. It denotes the expected calibration
error, which aims to evaluate the expected difference between model prediction
confidence and accuracy. The concrete formulation is as follows:

$\displaystyle
ECE=\sum_{i=1}^{|B|}\frac{N_{i}}{N}\lvert{\mathrm{acc}(b_{i})-\mathrm{conf}(b_{i})}\rvert,$
(14)

where $b_{i}$ represents the $i$-th bin and $|B|$ represents the total number
of bins. $N$ denotes the number of total samples. $N_{i}$ represents the
number of samples in the $i$-th bin. $\mathrm{acc}(b_{i})$ denotes the
accuracy and $\mathrm{conf}(b_{i})$ denotes the average of confidences in the
$i$-th bin.

Maximal Calibration Error (MCE) [10]. MCE represents the maximum deviation
between model accuracy and prediction confidence. In programs that require a
reliable measure of confidence, it is desirable to minimize the worst-case
deviation between confidence and accuracy. MCE is similar to ECE in that this
approximation involves bins:

$MCE=\mathop{\mathrm{max}}_{m\in\\{1,...,M\\}}|{\mathrm{acc}(b_{m})-\mathrm{conf}(b_{m})}|.$
(15)

Brier Score (BS). The BS is a metric that evaluates the proximity of a model’s
predicted probability to the actual class probability, which is always 1. A
desirable BS is near zero, as this indicates minimal deviation from the true
class probabilities due to the squared differences [94]. BS is often written
as the mean square prediction error (MSE) in regression tasks, defined as:

$BS=\frac{1}{N}\sum_{i=1}^{n}(y^{(i)}-\hat{y}^{(i)})^{2}.$ (16)

Both BS and mean squared error (MSE) involve calculating the squared
difference between predicted and actual values. BS is often used to evaluate
probabilistic predictions, while MSE is more commonly used to evaluate
regression models.

Negative Log-likelihood (NLL). The NLL is a widely accepted metric for
evaluating the effectiveness of a probabilistic model [95], and in the realm
of deep learning it is commonly referred to as cross entropy loss. When
presented with a probabilistic model and a set of $n$ samples, this measure
can be used to gauge the model’s calibration:

$NLL=-\sum_{i=1}^{n}{\mathrm{log}(\hat{\pi}(y^{(i)}|\mathbf{x}^{(i)}))}.$ (17)

Both NLL and Negative Log Predictive Density (NLPD) are calculated based on
the negative logarithm of probability, and NLPD calculates the negative
logarithm of the predicted density of new data given the training data and
model.

Furthermore, some NLP research proposes to calibrate evaluation metrics to
make them more suitable for evaluating specific tasks. For instance, Lin et
al. [96] proposes a new metric called Compositional Expected Calibration Error
(CECE) in his Seq2Seq graph parsing work to measure the behavior of the model
in predicting the structure of the combined graph, which evaluates the
performance of the model on graph elements. This metric calibrates the case to
better explain the behavior of the model in the combined graph structure and
the distributional behavior of predicted graph structures under offset.

####  3.3.2 Uncertainty Indication

Apart from assessing confidence calibration, there are several other
evaluation indicators for uncertainty. These metrics are primarily focused on
constructing classification metrics by utilizing either uncertainty or
confidence to determine the model’s capability to differentiate between
misclassified and outlier samples.

The Area Under the Curve (AUC). AUC is a commonly used measure the area under
the receiver-operator curve, the formulation is as follows:

${AUC}=\frac{\sum_{t_{0}\in\mathcal{D}^{0}}\sum_{t_{1}\in\mathcal{D}^{1}}\mathbbm{1}[g(t_{0})<g(t_{1})]}{|\mathcal{D}^{0}|\cdot|\mathcal{D}^{1}|}$
(18)

where $\mathcal{D}^{0}$ is the set of negative examples, and $\mathcal{D}^{1}$
is the set of positive examples. $\mathbf{1}[g(t_{0})<g(t_{1})]$ denotes an
indicator function which returns 1 if $g(t_{0})<g(t_{1})$ otherwise return 0.

Area Under the Receiver-Operator Characteristic Curve (AUROC), and the AUROC
metric measures the probability that a randomly selected accurate response has
a greater uncertainty score than a randomly selected inaccurate response.
Higher values indicate superior performance, with an ideal uncertainty score
of 1 and a value of 0.5 for a random uncertainty measure. Therefore, AUROC
scores high when the model is confident about correct predictions but
uncertain about incorrect ones. Depending on the axes and decision information
used, AUC can produce varying score indicators, such as AUCPR (Area Under the
Precision-Recall Curve) can be seen as an extension of the AUC score. However,
AUC is based on the True Positive Rate (TPR) and False Positive Rate (FPR),
while AUCPR is based on Precision and Recall.

Figure 6: An overview of the application of uncertainty estimation to NLP
systems. The red box indicates how the uncertainty is applied, and the blue
box is the action taken using the uncertainty.

The Area Under the Risk Coverage Curve (AUC-RCC) [97]. AUC-RCC evaluates the
quality of uncertainty estimation in terms of its ability to reject
predictions with high uncertainty and avoid misclassification. AUC-RCC is
based on the risk coverage curve, which plots the cumulative risk (or loss) as
a function of the uncertainty level used for rejecting predictions.

The Reversed Pair Proportion (RPP) [98]. Given a labeled dataset $D$ of size
$n$, RPP is used to evaluate the distance of the uncertainty estimator
$\tilde{u}$ from the ideal value. In order to minimize the AUC of RCC, the
selective classifier should be intuitive for the correct classification output
$\tilde{u}=0$ for the example of correct classification output, and
$\tilde{u}=1$ for the wrong example. Therefore, the formula is defined as
follows:

$RPP=\frac{1}{n^{2}}{\sum_{1\leq i,j\leq
n}^{n}\mathbbm{1}[\tilde{u}(x_{i})>\tilde{u}(x_{j}),l_{i}<l_{j}]},$ (19)

where $n^{2}$ in the denominator is used to normalize the values.

RPP measures the proportion of pairs of examples with an inverse confidence-
error relationship, and an ideal confidence estimator would have an RPP value
of 0. In contrast to AUROC, which measures overall discrimination, the RPP
incorporates consistency of ranking.

Figure 7: An overview of the applied classification of uncertainty. The inner
circle represents the application, the middle circle represents the
uncertainty modeling method used, and the outer circle refers to the specific
uncertainty estimation method.

##  4 Applications of Uncertainty in NLP

In this section, we review the applications of uncertainty estimation to NLP
systems. As shown in Fig. 6, the high-level perspective is mainly divided into
three dimensions: _data_ , _system_ , and _assessment_. The proportions of
each type and its sub-types are further counted, depicted in Fig. 7.

###  4.1 Data Filter and Action Guidance

In the _data_ dimension, we summarize the application of uncertainty as
filtering and guidance. It mainly involves three primary aspects, discussed
separately below.

TABLE I: Application of Uncertainty-Based Data Correlation in NLP, and some task/method names are represented as abbreviations. Application | Paradigm | Tasks Types | UE_Methods | Metrics
---|---|---|---|---
Active Learning | Class. |  TC[99, 100, 101]; [102, 103, 104, 105]; NLI[106]; SA[103]. |  Confidence[100, 101, 102]; Entropy[99, 104]; KL[103]; MCD[101, 102, 105]; Ensemble[101]; Evidence[106]. |  Accuracy [99, 100, 101]; [102, 103, 105, 106]; F1[100, 101, 106].
Active Learning | SeqLab |  NER[2, 107, 108], [109, 104]. |  Confidence[109]; Entropy[108, 104]; MCD[2, 107]. |  F1[2, 108, 107, 104]; Accuracy[2, 109].
Active Learning | Generative |  QA[110]; AS[111, 15]. |  Confidence[15]; MCD[110, 111]. |  ROUGE[15]; $\rm{BLEU_{Var}}$[110, 15, 111].
OOD detection | Class |  SA[112, 76, 88]; SLU[113, 88]; NLI[114]; TC[115, 114, 116], [103, 117, 89]. |  Confidence[115, 114, 112, 88]; Entropy[76, 113]; TS[113]; KL[103]; Mahalanobis[88, 89]; Evidence[117]; MCD[116]; Ensemble[116]; Gaussian[113]. |  Accuracy[114, 116, 112]; [103, 76, 113]; F1,[115, 116, 89]; ECE[114]; AUROC[117, 88, 113]; AUPR[117, 113, 89]; FAR95[112, 88, 113]; AUCRCC[89], FPR90[117].
OOD detection | SeqLab | NER[89, 76, 86]. |  Entropy[76]; Evidence[86]. |  F1[89, 76, 86]; AUROC [86].
OOD detection | Generative |  LM[76]; MT[118, 119]; QA[78]. |  MCD[118, 119]; Entropy[76, 113]; Ensemble[78]. |  Perplexity[76]; $\rm{BLEU_{Var}}$[118, 119]; Accuracy[78].
Selective prediction | Generative |  QA[30, 120, 121]. |  Confidence[120, 121]; Smoothing[121]; MCD[121, 30]; |  AUROC[121, 30]. Accuracy[30]; Precision/Recall [120].
Selective prediction | Class |  Duplicate Detection[120, 122]; NLI[120, 122]; SA[123]; TC[80]. |  Confidence[123, 122]; MCD[80]; Ensemble[80]; Bayes by Backprob[80]. |  Accuracy[122, 80]; F1[80]; AUROC[123, 122, 80]; AUPR[123].

####  4.1.1 Active Learning

Active learning (AL) is a machine learning technique that reduces the amount
of labeled data required for training by selecting the most informative
unlabeled data for annotation. This is based on the idea that not all data
points are equally useful during learning. Uncertainty-based AL is a popular
approach that assumes that data with higher uncertainty is more informative
and likely to enhance model performance. As shown in Table I, uncertainty-
based AL has been applied to various NLP tasks, and performance metrics (e.g.,
Accuracy, F1) are mainly used for AL evaluation. As shown in Fig. 7, AL mainly
uses the MCD based on sampling, and the confidence and entropy method based on
calibration, focusing more on the quality of data filtered by the teacher
model and the overall performance of the student system.

Benefiting from the development of NLP’s pre-training model, uncertainty-based
AL can effectively reduce the labeling cost and improve model performance. For
example, self-supervised language modeling and uncertainty filtering data can
be used for AL in the cold-start setting, effectively reducing labeling costs
and improving model performance [100]. Ein-Dor et al. [101] combine BERT with
the uncertainty estimation in the study of text classification AL to achieve
good results in data screening of minority classes. In addition, pre-trained
models allow us to improve data filtering capabilities in few-shot learning.
Mukherjee et al. [105] consider fine-tuning DNNs with limited labeled data.
This work develops two strategies for AL data filtering: filtering hard data
with high uncertainty and filtering soft data with low uncertainty improves
the performance of PLMs in few-shot learning scenarios. Combining pre-trained
models with uncertainty-based active learning improves model performance by
leveraging pre-existing knowledge and selecting informative examples for fine-
tuning.

Moreover, data can be filtered using uncertainty and additional information.
For example, ACTUNE [102] proposes uncertainty-based data filtering and
guidance, allowing switching between data filtering and model training. This
approach chooses highly deterministic unlabeled samples as actively labeled,
while low-uncertainty samples are selected for model self-training. AL usually
needs to traverse all unlabeled data to find informative unlabeled samples,
which are always close to the decision boundary with large uncertainty. AUSDS
[104] annotates unlabeled text samples with high uncertainty through
adversarial attacks, which significantly compresses the search space and
protects the decision boundary from drastic changes.

Since AL can select samples with large uncertainty, it is natural to imagine
applying uncertainty-based AL to multiple domains in the NLP. For example, Lyu
et al. [110] identify language as a key factor in the difference between data
inside and outside the distribution in QA. They further propose a data
selection strategy for AL based on an uncertainty measure developed using
deterministic sequence probability and MCD sequence probability. This approach
was shown to be more effective in selecting uncertain data for multilingual
QA. Information in real-world scenarios is often multi-domain and rich in
unlabeled data. Siddhant et al. [2] point out that classical uncertainty
estimation can only account for arbitrary uncertainty (e.g., entropy or
confidence). In contrast, using MCD+ BALD and Backprop+BALD as two uncertainty
estimation methods can achieve excellent results in the three NLP tasks of
emotion classification, NER, and semantic role labeling.

However, there are also some challenges and limitations of uncertainty-based
AL, such as how to measure uncertainty effectively, how to trade off
exploration and exploitation, and how to deal with the class imbalance and
data diversity. This section provides a comprehensive review of existing work
on uncertainty-based AL in NLP, covering data selection strategies, NLP issues
involved, and uncertainty estimation methods.

####  4.1.2 OOD/Outlier Detection

Out-of-distribution (OOD)/outlier detection is an important application of
uncertainty estimation in NLP. It helps to distinguish in-distribution
examples from OOD/outlier examples. As shown in Fig. 7, detecting OOD data in
NLP covers three paradigms of uncertainty estimation. The goal is to identify
examples that are different from the training data and the model has not been
seen before. Metrics used to detect system performance include task
performance metrics and uncertainty classification metrics. The task
performance metric evaluates the model’s ability to correctly handle the
corresponding task, while the uncertainty classification metric measures the
uncertainty quality of the model’s predictions. We summarize the types of
detection data into three ones: unknown domain data, misinformation data, and
misclassification data.

Unknown Domain Data Detection. Domain shift in NLP can be of different types
such as background shift (same task, but style/domain change) and semantic
shift (unseen labels) [124]. For example, traditional data selection based on
testing domain knowledge often fails in unknown domains such as patents and
tweets. Thus, unknown domain data detection is an important field of outlier
data, including multi-domain intersection, open domain, etc. Uncertainty-based
detection techniques can help solve these problems to improve the reliability
and usability of NLP models in real scenarios. The corresponding NLP tasks
have applications. For instance, MULTIUAT [119] addresses the challenge of
learning multilingual and multi-domain translation models due to heterogeneous
and imbalanced data, which makes the model converge inconsistently over
different corpora. Li et al. [78] perform multi-task-intensive retrieval in
open-domain QA, exploiting epistemic uncertainty to deal with corpus
inconsistencies. Yu et al. [76] show that maximizing the uncertainty of
training data using entropy can enhance prediction accuracy on unseen domains
and outperform CNN, BERT, and Transformer baselines, even without knowledge of
unknown domains.

Different domains have various distributions. Therefore, some previous works
apply distribution-based uncertainty estimation for OOD detection. Zhou et al.
[88] propose an unsupervised unknown domain detection method using a
contrastive learning framework. They fine-tune Transformers with a contrastive
loss to improve representation compactness. They further use Mahalanobis
distance and Gaussian distribution in the penultimate layer to accurately
detect OOD instances. Hu et al. [117] exploit uncertainty in OOD detection for
text classification tasks using evidential neural networks based on Dirichlet
distribution. They propose a framework that adopts auxiliary outliers and
pseudo off-manifold samples to train the model with prior knowledge of a
certain class, which has high vacuity for OOD samples. In addition, PLMs may
affect OOD detection performance. Hendrycks et al. [112] systematically study
the OOD robustness of pre-trained Transformers on various models. To measure
OOD detection performance, they use the negative prediction confidence as the
outlier supervision score and show that the pre-trained Transformer performs
better than previous models both in generalizing to OOD examples and in
detecting OOD examples.

Misinformation Data Detection. The proliferation of misinformation on social
media platforms is a major concern for society. One approach to developing
uncertainty-based methods and preventing misinformation is to use uncertainty-
based methods to detect and prevent misinformation and rumor spreading. To
name a few, Zhang et al. [125] conduct research on uncertainty estimation
methods, specifically Bayesian deep learning and Evidence Lower Bound (ELBO)
objective function for rumor detection. Kochkina et al. [126] research
error/outlier detection for automatic rumor verification. They propose two
uncertainty-based instance rejection methods using aleatoric and epistemic
uncertainty estimates obtained through confidence-based and MCD-based methods.
This research aims to solve the problem of resolving rumors circulating online
by prioritizing difficult instances for human fact-checkers and interpreting
model performance during rumor unfolding. Furthermore, Feng et al. [127] study
_“none of the above”_ (NOTA) detection tasks in dialogue systems, i.e. the
case where no correct response (i.e. ground truth) exists in the candidate
set. To this end, the paper utilizes MCD as an uncertainty estimation method
to measure the ability to capture uncertainty for end-to-end retrieval models.
Overall, the speed at which misinformation spreads on social media makes
manual verification difficult, and these tools can help detect and mitigate
the impact of misinformation, thereby reducing its potential negative impact
on the public.

Misclassification Data Detection. To detect OOD, epistemic uncertainty is
caused by limited training data or model structure. However, misclassification
detection also requires modeling aleatoric uncertainty caused by noise and
ambiguity in data [128]. Hendrycks et al. [123] study uncertainty estimation
such as confidence-based and Softmax prediction probability for sentiment
classification. The goal is to indicate when classifiers are likely to make
mistakes to increase their adoption and prevent accidents. The authors provide
a misclassification detection baseline for neural networks that outperforms
MaxProb for selective prediction. Vazhentsev et al. [89] investigate the usage
of uncertainty estimation for Transformer-based NER and text
misclassification, and add an experimental setup given OOD data. The research
proposes two computationally efficient modifications, including Mahalanobis
Distance with a spectral-normalized network, which approach or outperform
computationally intensive approaches.

However, how to determine the OOD boundary? One of the main reasons for OOD
boundary is the limited input information available to NLP models, as they
typically rely on a limited set of features extracted from the input text.
This can lead to many models that are well-calibrated or achieve excellent
uncertainty estimation in IDs, but poor results in OOD samples [26]. One
option for finding the boundary between ID and OOD data is to train a
supervised classifier on both ID and OOD data [8, 124]. Nevertheless,
collecting a representative set of OOD data may not be practical due to the
infinite compositionality of languages, and selecting an arbitrary subset may
introduce selection bias and limit the generalizability of the model to unseen
OOD data. Alternatively, Ryu et al. [129, 130] propose using generative
models, such as autoencoders and GANs [131], to capture the ID data
distribution and differentiate between ID and OOD data based on reconstruction
error or likelihood. Other approaches, such as meta-learning for OOD detection
and generating pseudo-OOD data [132], also provide decision boundary between
ID and OOD [133], but all require additional data or training procedures
beyond the task, which may result in significant data collection work or
inference overhead.

TABLE II: Uncertainty-based applications related to efficiency and performance improvement in NLP systems.  Application | Paradigm | Tasks Types | UE_Methods | Metrics
---|---|---|---|---
Efficacy | Class |  GLUE[134, 135]; SA[136, 137, 138]; NLI[137]; Fact verification[138]; Topic classification[137, 138]; Sentence classification[36]; Shopping review[36]; Sentences-matching[36]. |  Confidence[138]; Entropy[134]; TS[137, 138]; CP[138]. |  Accuaracy[134, 36]; [136, 137, 138]; Consistency[138]; Layers[138]; Time%[134]; Speedup[36, 136, 138].
Efficacy | Regression |  GLUE[136]; Similarity[135, 138]. |  Confidence[135, 138]; TS[138]; CP[138]. |  Speedup[136]; Relative scores[135].
Efficacy | Generative |  Text generation[38]. |  Confidence[38]. |  BLEU[38]; ROUGE[38]; F1[38]; Layers[38].
Performance | Class |  TC[7]; Fake News Detection[139]. |  Gumbel[7]; MCD[7]; Ensemble[7]; Gaussian[139]. |  F1[139]; Accuaracy[7, 139].
Performance | Structure | KGs [140, 141, 142]. |  Gumbel[141]; Confidence[140, 142]. |  BS[140, 141, 142]; MAE[140, 141]; Accuaracy[140]; F1[140].
Performance | SeqLab | NER[37, 68, 143]. |  MCD[37]; Entropy[143]; Smoothing[68]. |  F1[37, 68, 143]; ECE[68].
Performance | Generative |  MT[144, 145, 27]; QA[146]. |  Confidence[146, 27]; KL[27]; Norma_Confidence[145]; MCD[144, 145]. |  BLEU[144, 145, 27]; Accuaracy[146]; AUROC[146].

####  4.1.3 Selective Prediction

Selective prediction refers to the ability of AI systems to abstain from
making predictions when faced with novel inputs that differ from their
training data distribution. This is essential for improving the reliability of
NLP systems in real-world safety-critical domains like biomedical and
autonomous robots, where incorrect predictions can have serious consequences.
Typically, $c$ contains a prediction confidence estimator $\tilde{c}$ and a
threshold $\tau$ that controls the level of abstention:

$c(x)=\mathbbm{1}[\tilde{c}(x)>\tau]$ (20)

For dataset $\mathcal{D}$, the coverage of the threshold $\tau$ corresponds to
the fraction of answered instances (where $\tilde{c}(x)>\tau$), and the risk
is the error of these answered instances.

Selective predictive systems make a trade-off between coverage and risk. As
shown in Table I, QA systems for medical domains require high precision and
discard questions that will not be answered by the QA system. This presents a
cost-saving opportunity [120]. In addition, the example of a medical diagnosis
model usually provides high-confidence classifications even when requires
human intervention. The failure of classifiers indicates when they are likely
mistaken, limiting their adoption or causing serious accidents. The
application of selective prediction in NLP is crucial for enabling the
reliable deployment of NLP systems in real-world applications.

Recently, selective prediction in NLP has received attention. Andersen et al.
[80] study MCD for selectivity prediction in text classification. A semi-
automatic text classification framework that minimizes unreliable and error-
prone classifications by explicitly modeling uncertainty. Varshney et al.
[122] handle NLI tasks based on ID and OOD datasets, allowing systems to avoid
making predictions when they might go wrong, improving their reliability in
safety-critical domains. Furthermore, Varshney et al. [121] investigate
selective prediction in NLP using 17 datasets covering NLI, duplicate
detection, and QA tasks. In BERT-based experiments, the results show that
these methods outperform the MaxProb method when comparing whether the MCD and
label smoothing can improve the performance of selectivity prediction. Garg et
al. [120] conduct research on filtering out unanswered questions in medical QA
systems. They focus on Transformer-based question models to improve answer
models and estimate uncertainty. This research shows that confidence scores
for answers can be approximated from question text alone without requiring
answers. By filtering out error-prone problems, the improved QA model improves
the reliability of real-world NLP applications. Kamath et al. [30] explore
selectivity prediction in QA tasks under domain shift. Here TS is used to
calibrate confidence scores, which enables the model to discard answers when
necessary to account for distributions different from the training data.

Selective prediction in NLP involves using uncertainty to determine which data
to process, resulting in higher accuracy on the answered subset. However,
determining how to rank examples according to confidence metrics and balancing
the trade-offs between risk and coverage are still considerations in selective
prediction.

###  4.2 NLP System Efficiency and Performance

The framework of an NLP system is usually multi-layer or multi-module.
Uncertainty estimation can be exploited for each component to improve the
computational efficiency or performance of the network. Although performances
could be also improved by better dealing with data in Sec 4.1, here in this
section, we review from the perspective to leverage uncertainty in the
interior computation of NLP systems and models. As presented in Fig. 6,
uncertainty is introduced for deciding the necessity of calculation. And it
promotes performance by providing more information or guidance.

Figure 8: Illustration of the early exit method based on uncertainty. Common
layers/modules such as Transformer, $C_{i}$ are usually a classifier,
determining whether the early exit is determined by its confidence.

####  4.2.1 Uncertainty and Efficiency

PLMs have impressive performance but require substantial computational
resources, which limits their use in real-world applications [137, 138, 135].
In particular, autoregressive decoding using a full stack of Transformer
layers for each output token is computationally expensive, and this approach
is commonly used in NLP [38]. This limitation makes it difficult where fast
inference and low computational cost are essential. Additionally, over-
parameterization of PLMs can lead to overthinking in decision-making, where
the model may focus on complex or irrelevant features in later layers instead
of relying on simpler features from earlier layers that generalize better
[136]. As demonstrated in Fig. 7, the calibration-based uncertainty estimation
approach is more commonly leveraged to improve efficiency. Its calculation is
convenient and sufficient. Accurate confidence information allows the model to
improve efficiency in a more optimal way.

An uncertainty-based early exit in NLP is a technique to improve the
efficiency of neural models. As shown in Fig. 8, early exit allows the model
to exit the sequence being processed early based on uncertainty about the
predicted output. This strategy works by introducing a threshold value that
determines when the model should stop computation in advance via the
uncertainty of the predicted output. The threshold value can be set through
the desired trade-off between accuracy and efficiency. If the uncertainty
score of the prediction is below the threshold, the model continues processing
the sequence. Yet if the uncertainty score exceeds the threshold, the model
exits early and returns the prediction. This approach can significantly reduce
the computation required to process the sequence and improve the overall
efficiency of the model.

To name a few, Schuster et al. [38] propose confidence adaptive language
modeling to improve the efficiency of autoregressive decoding in PLMs.
Confidence-based uncertainty estimation allows PLMs to generate new tokens
with intermediate network layers rather than the full model. Thereby it
reduces the amount of computation. The approach builds on distribution-free
uncertainty quantification and provides a principled approach to improving
model efficiency while maintaining predictive quality. Liu et al. [36] design
a pre-trained model called FastBERT that applies self-distillation and
knowledge distillation to achieve faster and more accurate results in Chinese
and English sentence classification tasks. To further improve efficiency,
FastBERT utilizes normalized entropy for uncertainty estimation, removing
cases with low uncertainty from the batch and sending cases with high
uncertainty to the next layer for further reasoning. Xin et al. [135] propose
BERxiT, which combines _BERT_ and _exit_ for regression tasks in NLP.
Furthermore, Table II shows the application is mainly general performance
metrics and efficiency improvement comparison metrics. By using uncertainty-
based early exit, NLP models can achieve faster processing times and reduce
computational costs without sacrificing accuracy.

Overall, uncertainty estimation has the potential to enhance the performance
and efficiency of NLP systems. However, one challenge is how to incorporate
uncertainty estimates into the decision-making process of NLP systems without
introducing bias or overconfidence. Additionally, it is essential to trade off
the computational cost of uncertainty estimation with its potential benefits
in terms of performance and efficiency.

####  4.2.2 Uncertainty and Performance

Uncertainty information can also be used to guide the model and improve its
performance. For instance, Xiao et al. [3] show that applying uncertainty
estimation to NLP tasks such as sentiment analysis, NER, and language modeling
models can learn accurate mappings. Additionally, uncertainty can provide
richer embedding information. By incorporating uncertainty into the model,
knowledge graph embeddings can improve the accuracy and robustness of models.
Uncertain-aware embeddings can incorporate probabilistic uncertainty into the
embedding process, which allows for more precise representations [140, 147,
142, 141]. It can be seen from Table II that, in addition to performance
indicators, uncertainty estimation is mainly exploited for uncertainty
calibration indicators.

Uncertainty can also be incorporated into the model’s training and inference
stages to improve model performance. Gui et al. [37] propose to improve NER
task performers with MCD, which predicts error-prone draft labels in the first
stage. A two-stream self-attention model in the second stage aims to refine
these draft predictions. This approach helps to prevent error propagation and
improves the accuracy of NLP models. In addition, Zhou et al. [145] assume
that language features alone cannot completely solve the difficulties
encountered by the MT model. They propose an adaptive curriculum learning
strategy to evaluate the uncertain information of the model. This strategy
enhances the performance of neural machine translation, taking into account
the complexity and rarity of information in translation pairs. In MT tasks, a
source-language sentence may have multiple semantically similar expressions.
And a source-language sentence may also have multiple valid counterparts in
another language. Thus such tasks are many-to-many problems, which inherently
contain uncertainty. Wei et al. [27] leverage confidence and KL distributions
to capture the relationship between multiple semantically equivalent source
sentences and use general semantic information to enhance hidden
representations for better translation results. Accurate uncertainty refers to
the process of adjusting a model’s uncertainty estimate to better reflect its
true performance. This helps ensure the model makes reliable predictions and
improves its accuracy.

TABLE III: Uncertainty-based applications related to NLP system evaluation. Tasks Types | UE_Methods | Metrics
---|---|---
MT [18, 93] [148]; QA[149, 150] [26, 151]. |  Confidence[149], [150, 26, 148]; TS[149, 26]; GPs[93]; MCD[148, 18]; Ensemble[148]; Heteroscedastic[18]. |  Accuracy[26] [149, 150]; MSE[150]; NLPD[93]; NLL[93, 148] [18]; MAE[93]; ECE[149, 148] [26, 18]; AUROC[26]; BS[26]; Sharpness[148, 18]; Pearson [148, 18].
Document Quality Assessment[152]. | GPs[152]. |  NLPD[152]; RMSE[152]; Pearson[152].

###  4.3 Reliability and Trustworthy Assessment

In this section, we review the applications of uncertainty in the output stage
of NLP system. Here the main insight is that _can we fully trust the output of
a neural network?_ The answer is obviously _no_. The main reason is that its
output relies on probability and thereby mistakes are inevitable. The question
can be converted to _how much can we trust the output?_ In other words, it
means reliability or trustworthy extents. Since uncertainty is reflected as a
numerical value, which naturally helps us to assess the quality of the model
or output.

By quantifying and analyzing uncertainty, we can gain valuable insights into
the strengths and weaknesses of our NLP models, and better understand the
risks and limitations of the decisions we make based on them. For instance, if
an NLP model has high uncertainty in its predictions for certain data points
or scenarios, we expect to investigate further or use additional information
to improve the performance of the model. Uncertainty can be used to evaluate
models’ performance. Confidence scores can be compared to the actual
correctness of decisions made by the model. This approach enables us to
determine how reliable and trustworthy the model is in terms of making
accurate decisions. In particular, some metrics given in Sec 3.3 can be
exploited to evaluate the confidence calibration and uncertainty
classification ability of the model, and then reflect the advantages and
disadvantages of the model.

Furthermore, assessing the reliability of generated content is crucial.
Especially in the exponential search space of the generative model, the use of
uncertainty for model and output analysis is valuable. Table III shows that in
recent years, research on uncertainty estimation in MT and QA has gradually
increased, and various evaluation indicators have increased to explain the
model or output results. Typically, multiple candidate translations can be
generated in MT, making it difficult to determine the best translation.
Estimating the uncertainty of each candidate translation can help researchers
choose the translation with the lowest uncertainty, leading to better
translation quality [93, 148, 18]. Recently, as PLMs have become more powerful
and widely used, it has become increasingly important to be able to explain
how they arrive at answers and provide some degree of uncertainty or
confidence in predictions. There are various techniques for evaluating the
performance and credibility of PLMs, including self-evaluation techniques,
calibration scores, and other methods [149, 150, 26, 151]. These techniques
can help to ensure that NLP systems are trustworthy and reliable, and can be
used with confidence by users in a range of different applications.

Last but not least, an intuitive question is how to express uncertainty to
improve model evaluation and obtain valuable information? This is a natural
challenge because language expression itself is inherently uncertain.
Quantifying the uncertainty expressed by the model, making effective
evaluations, and reducing the interaction bias caused by expression are all
important considerations. We provide a more detailed discussion of this
challenge in Sec 5.3.

##  5 Challenges And Future Directions

###  5.1 Extremely High-Dimensional Language-space

The field of NLP is challenged by the extremely high-dimensional language
space, as highlighted by the need to estimate predictive entropy, which
requires taking an expectation in output space. This output space has a
dimensionality of $\mathcal{O}(|\mathcal{T}|^{N})$, which presents significant
computational challenges. Additionally, the lack of a normalized probability
density function over sentences necessitates approximating the expectation
using Monte Carlo integration, which involves averaging the likelihoods of a
finite set of sampled sentences. However, Monte Carlo integration becomes
difficult for entropy as it is often dominated by low-probability sentences
that have large and negative logs. MT models, for example, can contain
hundreds of millions of parameters, which exponentially increases the search
space. Additionally, with only a single reference for each source sentence, it
becomes difficult to measure the fitness of MT models to data distributions.
As a result, researchers face significant scientific challenges in adapting
tools from both the machine learning and statistics fields to effectively
tackle this problem.

In addition, the complexity of uncertainty in NLP systems is also constantly
increasing due to the evolving models and the demand for human-AI interaction.
Although the current uncertainty estimation techniques are rich, further
exploration is still needed to address issues such as time cost and inference
accuracy, and how to consider the impact of natural language features on
uncertainty estimation. Some uncertainty estimation techniques can also be
limited in their scalability. Firstly, TS is a convenient calibration
technique, but the performance of TS is very sensitive to the choice of the
scaling factor. Choosing the optimal scaling factor may require extensive
hyperparameter tuning, which can be time-consuming and computationally
expensive. Secondly, ensembling multiple models or MCD can be effective for
uncertainty estimation but can also be resource-intensive and difficult to
scale to large models. Finally, distribution-based uncertainty estimation can
provide valuable insights into the uncertainty of DNNs, but it requires
distributional assumptions, such as Gaussian or a mixture of Gaussian
distributions, which may not apply in all situations and limit its
applicability. In the era of large PLMs such as ChatGPT and GPT-4, researchers
are not accessible to the model. Even if a PLM is open-sourced, retraining is
often necessary, which can be computationally expensive and time-consuming.

###  5.2 Variable Length Generation

The challenge of variable text length in NLP cannot be ignored. One major
obstacle is the significant variation in the length of sentences [55]. As
Malinin et al. [8] point out, variable-length text generation is particularly
challenging in NLG, where longer sequences tend to have lower joint
likelihoods due to the conditional independence of token probabilities. In
other words, as the length of a sequence increases, its joint likelihood
decreases exponentially, leading to a linear increase in its negative log
probability. This means that longer sentences contribute more to entropy,
which exacerbates the difficulty of dealing with variable-length texts. Text
length normalization is commonly adopted to combat variable text length, which
assumes that the uncertainty in a sentence is independent of its length [153,
154, 55]. Specifically, for estimating the probability $\mathbf{e}=e_{1:l}$ of
generating the output sequence, length normalization divides the score
$s(\mathbf{e})$ by the length $l$ of the survival text to get
$s^{\prime}(\mathbf{e})=s(\mathbf{e})/l$. For example, Kuhn et al. [4] apply
length normalization of the log probability when estimating the semantic
entropy of a QA system. However, this technique may not be useful in all
cases, especially when dealing with long sentences, as it fails to capture the
increased complexity and difficulty of longer text sequences.

###  5.3 Expression of Uncertainty in Language Model

In real life, information is rarely black and white, which is why expressing
uncertainty is necessary to support the decision-making process. While highly
certain expressions are commonly leveraged, uncertain expressions tell us
about the confidence, source, and limitations of the information. In the
current research, we find two main forms of uncertainty expression:
probabilistic expression and natural expression, as demonstrated in Fig. 9.
The former one generally has relevant values available for analysis, but the
community has found different results on the calibration of neural models. For
example, Desai and Durrett [114] show that pre-trained Transformers are
relatively well calibrated, while Wang et al. [155] find severe miscalibration
in MT. Jiang et al. [149] study calibration in generative QA and observe only
a weak correlation between the log-likelihood model assigned to their answers
and the correctness of the answers. In general, PLMs output a possibility for
a given tag sequence, but do not output the entire meaning [4].

On the one hand, whether this uncertainty expression is suitable for
generative NLP systems remains questionable. We may need stronger correlation
indicators to focus on the uncertainty expression of probability. On the other
hand, naturalistic representations of uncertainty cover a wide range of
discursive behaviors, such as signaling hesitancy, attributing information, or
acknowledging limitations, and such representations are intuitive to humans.
As the examples displayed in Fig. 9, QA prompts are able to express
uncertainty. Through different prefixes, suffixes, or confidence levels of
self-assessment outputs, we can obtain numerical uncertainty representations
to further analyze the calibration of the model.

Different Templates for Uncertainty RepresentationÄpgjy Answer Prefix:
1\. I’m 90% sure it’s …
2\. I vaguely remember it’s …
Answer Suffix:
1\. …But I would need to double-check.
2\. …With 100% confidence.
Answer Self-evaluation:
1\. …With what confidence could you answer this question? and output an answer
like 0%, 10%, 20%, …, 100%. Figure 9: Illustration of a problem template for
uncertainty estimation.

Kadavath et al. [26] expect to observe large benefits of few-shot evaluation
with natural language methods, but instead. Yet no major gains are observed in
early QA experiments. Zhou et al. [151] find that non-deterministic
expressions can affect language production, and that changes in these
expressions can have a substantial impact on overall accuracy, especially when
using high-certainty expressions, including in accuracy and calibration. The
authors further hypothesize that this may be due to the usage of hyperbolic or
exaggerated language in the training set, where numbers are used non-
literally. Then, the model somehow recognizes idiomatic, non-literal usage of
these extreme values, resulting in lower performance of the task when hints
are introduced.

In summary, effective understanding and expression of uncertainty are crucial
for NLP applications to ensure better decision-making. While there is a wealth
of literature on methods for estimating uncertainty, there is little
understanding of how linguistic uncertainty interacts with natural language.
This lack of attention has resulted in a poor understanding of how models
interact with natural language, leading to challenges in formulating and
evaluating uncertainty estimates.

###  5.4 NLP and Social Security

It is imperative to analyze uncertainty information in NLP systems when
integrating social and ethical demands. Therefore, in the following
discussion, we will examine future directions from these three perspectives.
As an important part of general AI, NLP systems need to account for moral
uncertainty, that is, try to explain what decisions should be made, given
various details of the circumstances of their decisions, including the choices
they face and their theoretical rationale. Newberry et al. [156] discuss the
idea of using an automated _“moral parliament”_ as a way to mitigate the risk
of value erosion in AI systems. By incorporating a variety of values from
different stakeholders, the AI could be directed by a simulated group
representing different values. This approach would better reflect the moral
uncertainty of humans and avoid committing AI to one value system. Perhaps the
AI system will become a superior version of the current chatbot, surpassing
humans in multiple fields [157]. As AI systems become more powerful and have a
greater impact on society, there is an increasing need to ensure that they are
designed with appropriate ethical and moral considerations, despite
uncertainty surrounding implementation details across multiple paradigms. This
includes developing AI systems that can incorporate ethical uncertainty and
allow for multi-stage scrutiny to ensure they are safe and beneficial for
society.

Overall, in addition to the limitations and challenges of existing uncertainty
estimation methods, there is a need for more effective methods that can
provide interpretable and reliable uncertainty estimates. This is an active
area of research, and developing better uncertainty estimation methods can
help improve the reliability, robustness, and safety of NLP systems.

##  6 Conclusion

Considering the semantic ambiguity of natural language, uncertainty is an
important attribute of text information. For the first time, this survey aims
to provide a comprehensive review of uncertainty estimation in the NLP field,
including its sources, quantification approaches, and applications. We first
depict the backgrounds of uncertainty types and summarize the uncertainty
sources of an NLP system by its multiple stages. Then, uncertainty estimation
methods are systematically reviewed, with pointing out the technical
advantages and application difficulties respectively. Meanwhile, evaluation
metrics for uncertainty estimation are also discussed. We further investigate
relevant literature on uncertainty-aware applications in the NLP field,
dividing it into three directions and providing detailed analyses. Finally, we
highlight four challenges in the combination between uncertainty estimation
and the development of PLMs. Promising research areas are also explored,
including the discussion on the scalability of uncertainty technology, the
expression of uncertainty, and AI security.

## References

  * [1] T.-X. Sun, X.-Y. Liu, X.-P. Qiu, and X.-J. Huang, “Paradigm shift in natural language processing,” _Machine Intelligence Research_ , vol. 19, no. 3, pp. 169–183, 2022.
  * [2] A. Siddhant and Z. C. Lipton, “Deep Bayesian active learning for natural language processing: Results of a large-scale empirical study,” in _EMNLP_ , 2018, pp. 2904–2909.
  * [3] Y. Xiao and W. Y. Wang, “Quantifying uncertainties in natural language processing tasks,” in _AAAI_ , 2019, pp. 7322–7329.
  * [4] L. Kuhn, Y. Gal, and S. Farquhar, “Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation,” in _ICLR_ , 2023.
  * [5] K. Ethayarajh, “Is your classifier actually biased? measuring fairness under uncertainty with bernstein bounds,” in _ACL_ , 2020, pp. 2914–2919.
  * [6] J. Kivimäki _et al._ , “Uncertainty estimation with calibrated confidence scores,” 2022.
  * [7] J. Pei, C. Wang, and G. Szarvas, “Transformer uncertainty estimation with hierarchical stochastic attention,” in _AAAI_ , 2022, pp. 11 147–11 155.
  * [8] A. Malinin and M. Gales, “Predictive uncertainty estimation via prior networks,” _NeurIPS_ , 2018.
  * [9] A.-K. Kopetzki, B. Charpentier, D. Zügner, S. Giri, and S. Günnemann, “Evaluating robustness of predictive uncertainty estimation: Are dirichlet-based models reliable?” in _ICML_ , 2021, pp. 5707–5718.
  * [10] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in _ICML_ , 2017, pp. 1321–1330.
  * [11] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” _JMLR_ , pp. 5485–5551, 2020.
  * [12] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever _et al._ , “Improving language understanding by generative pre-training,” 2018.
  * [13] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever _et al._ , “Language models are unsupervised multitask learners.”
  * [14] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _NeurIPS_ , pp. 1877–1901, 2020.
  * [15] J. Xu, S. Desai, and G. Durrett, “Understanding neural abstractive summarization models via uncertainty,” in _EMNLP_ , 2020, pp. 6275–6281.
  * [16] J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscher _et al._ , “A survey of uncertainty in deep neural networks,” _arXiv preprint arXiv:2107.03342_ , 2021.
  * [17] E. Hüllermeier and W. Waegeman, “Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods,” _Machine Learning_ , pp. 457–506, 2021.
  * [18] C. Zerva, T. Glushkova, R. Rei, and A. F. Martins, “Disentangling uncertainty in machine translation evaluation,” in _EMNLP_ , 2022, pp. 8622–8641.
  * [19] A. Delaforge, J. Azé, S. Bringay, C. Mollevi, A. Sallaberry, and M. Servajean, “Ebbe-text: Explaining neural networks by exploring text classification decision boundaries,” _IEEE Transactions on Visualization and Computer Graphics_ , 2022.
  * [20] V. Dragos, “An ontological analysis of uncertainty in soft data,” in _FUSION_ , 2013, pp. 1566–1573.
  * [21] S. L. Blodgett, S. Barocas, H. Daumé III, and H. Wallach, “Language (technology) is power: A critical survey of “bias” in NLP,” in _ACL_ , 2020, pp. 5454–5476.
  * [22] L. VENUTI, “The translator’s invisibility,” _Criticism_ , pp. 179–212, 1986\.
  * [23] M. Ott, M. Auli, D. Grangier, and M. Ranzato, “Analyzing uncertainty in neural machine translation,” in _ICML_ , 2018, pp. 3956–3965.
  * [24] R. Levy, “A noisy-channel model of human sentence comprehension under uncertain input,” in _EMNLP_ , 2008, pp. 234–243.
  * [25] T. Liu, X. Zhang, W. Zhou, and W. Jia, “Neural relation extraction via inner-sentence noise reduction and transfer learning,” in _EMNLP_ , 2018, pp. 2195–2204.
  * [26] S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. H. Dodds, N. DasSarma, E. Tran-Johnson _et al._ , “Language models (mostly) know what they know,” _arXiv preprint arXiv:2207.05221_ , 2022.
  * [27] X. Wei, H. Yu, Y. Hu, R. Weng, L. Xing, and W. Luo, “Uncertainty-aware semantic augmentation for neural machine translation,” in _EMNLP_ , 2020, pp. 2724–2735.
  * [28] B. Qin, L. Wang, B. Hui, B. Li, X. Wei, B. Li, F. Huang, L. Si, M. Yang, and Y. Li, “SUN: Exploring intrinsic uncertainties in text-to-SQL parsers,” in _COLING_ , 2022, pp. 5298–5308.
  * [29] U. Arora, W. Huang, and H. He, “Types of out-of-distribution texts and how to detect them,” in _EMNLP_ , 2021, pp. 10 687–10 701.
  * [30] A. Kamath, R. Jia, and P. Liang, “Selective question answering under domain shift,” in _ACL_ , 2020, pp. 5684–5696.
  * [31] M. Carlebach, R. Cheruvu, B. Walker, C. Ilharco Magalhaes, and S. Jaume, “News aggregation with diverse viewpoint identification using neural embeddings and semantic understanding models,” in _Argument Mining_ , 2020, pp. 59–66.
  * [32] J. Xin, R. Tang, Y. Yu, and J. Lin, “The art of abstention: Selective prediction and error regularization for natural language processing,” in _ACL-IJCNLP_ , 2021, pp. 1040–1051.
  * [33] P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner _et al._ , “Relational inductive biases, deep learning, and graph networks,” _arXiv preprint arXiv:1806.01261_ , 2018.
  * [34] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” _NeurIPS_ , 2017\.
  * [35] A. Amini, W. Schwarting, A. Soleimany, and D. Rus, “Deep evidential regression,” _NeurIPS_ , pp. 14 927–14 937, 2020.
  * [36] W. Liu, P. Zhou, Z. Wang, Z. Zhao, H. Deng, and Q. Ju, “FastBERT: a self-distilling BERT with adaptive inference time,” in _ACL_ , 2020, pp. 6035–6044.
  * [37] T. Gui, J. Ye, Q. Zhang, Z. Li, Z. Fei, Y. Gong, and X. Huang, “Uncertainty-aware label refinement for sequence labeling,” in _EMNLP_ , 2020, pp. 2316–2326.
  * [38] T. Schuster, A. Fisch, J. Gupta, M. Dehghani, D. Bahri, V. Tran, Y. Tay, and D. Metzler, “Confident adaptive language modeling,” _NeurIPS_ , pp. 17 456–17 472, 2022.
  * [39] W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” _The Journal of Machine Learning Research_ , vol. 23, no. 1, pp. 5232–5270, 2022.
  * [40] C. Beck, H. Booth, M. El-Assady, and M. Butt, “Representation problems in linguistic annotations: Ambiguity, variation, uncertainty, error and bias,” in _14th Linguistic Annotation Workshop_ , 2020, pp. 60–73.
  * [41] G.-P. Bonneau, H.-C. Hege, C. R. Johnson, M. M. Oliveira, K. Potter, P. Rheingans, and T. Schultz, “Overview and state-of-the-art of uncertainty visualization,” _Scientific Visualization: Uncertainty, Multifield, Biomedical, and Scalable Visualization_ , pp. 3–27, 2014.
  * [42] M. John, “Uncertainty in visual text analysis in the context of the digital humanities,” 2017.
  * [43] E. M. Bender and B. Friedman, “Data statements for natural language processing: Toward mitigating system bias and enabling better science,” _TACL_ , pp. 587–604, 2018.
  * [44] J. Dodge, G. Ilharco, R. Schwartz, A. Farhadi, H. Hajishirzi, and N. Smith, “Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping,” _arXiv preprint arXiv:2002.06305_ , 2020.
  * [45] C. Liang, Y. Yu, H. Jiang, S. Er, R. Wang, T. Zhao, and C. Zhang, “Bond: Bert-assisted open-domain named entity recognition with distant supervision,” 2020.
  * [46] A. Pandey, S. Daw, N. Unnam, and V. Pudi, “Multilinguals at SemEval-2022 task 11: Complex NER in semantically ambiguous settings for low resource languages,” in _SemEval_ , 2022, pp. 1469–1476.
  * [47] R. Panchendrarajan and A. Amaresan, “Bidirectional lstm-crf for named entity recognition,” in _Proceedings of the 32nd Pacific Asia conference on language, information and computation_ , 2018.
  * [48] J. Li, A. Sun, J. Han, and C. Li, “A survey on deep learning for named entity recognition,” _IEEE Trans. on Knowl. and Data Eng._ , p. 50–70, 2022.
  * [49] F. Zhai, S. Potdar, B. Xiang, and B. Zhou, “Neural models for sequence chunking,” in _AAAI_ , 2017.
  * [50] I. Partalas, C. Lopez, N. Derbas, and R. Kalitvianski, “Learning to search for recognizing named entities in Twitter,” in _WNUT_ , 2016, pp. 171–177.
  * [51] O. Levy and Y. Goldberg, “Linguistic regularities in sparse and explicit word representations,” in _CoNLL_ , 2014, pp. 171–180.
  * [52] V. Kuleshov and P. S. Liang, “Calibrated structured prediction,” _NeurIPS_ , 2015.
  * [53] A. Stent, M. Marge, and M. Singhai, “Evaluating evaluation methods for generation in the presence of variation.” in _CICLing_ , 2005, pp. 341–351.
  * [54] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan _et al._ , “Training a helpful and harmless assistant with reinforcement learning from human feedback,” _arXiv preprint arXiv:2204.05862_ , 2022.
  * [55] K. Murray and D. Chiang, “Correcting length bias in neural machine translation,” in _Proceedings of the Third Conference on Machine Translation: Research Papers_ , 2018, pp. 212–223.
  * [56] A. Malinin and M. Gales, “Uncertainty estimation in autoregressive structured prediction,” in _ICLR_ , 2022.
  * [57] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” _NeurIPS_ , 2013.
  * [58] Y. Wang, D. Beck, T. Baldwin, and K. Verspoor, “Uncertainty estimation and reduction of pre-trained models for text regression,” _Transactions of the Association for Computational Linguistics_ , vol. 10, pp. 680–696, 2022. [Online]. Available: https://aclanthology.org/2022.tacl-1.39
  * [59] F. Guzmán, P.-J. Chen, M. Ott, J. Pino, G. Lample, P. Koehn, V. Chaudhary, and M. Ranzato, “The FLORES evaluation datasets for low-resource machine translation: Nepali–English and Sinhala–English,” in _EMNLP-IJCNLP_ , Hong Kong, China, Nov. 2019, pp. 6098–6111. [Online]. Available: https://aclanthology.org/D19-1632
  * [60] C. Corley and R. Mihalcea, “Measuring the semantic similarity of texts,” in _Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment_. Ann Arbor, Michigan: Association for Computational Linguistics, Jun. 2005, pp. 13–18. [Online]. Available: https://aclanthology.org/W05-1203
  * [61] V. Kuleshov, N. Fenner, and S. Ermon, “Accurate uncertainties for deep learning using calibrated regression,” in _International conference on machine learning_. PMLR, 2018, pp. 2796–2804.
  * [62] C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra, “Weight uncertainty in neural network,” in _ICML_ , 2015, pp. 1613–1622.
  * [63] D. P. Kingma, T. Salimans, and M. Welling, “Variational dropout and the local reparameterization trick,” _NeurIPS_ , 2015.
  * [64] C. Louizos and M. Welling, “Structured and efficient variational deep learning with matrix gaussian posteriors,” in _ICML_ , 2016, pp. 1708–1716.
  * [65] S. Ruder and B. Plank, “Learning to select data for transfer learning with Bayesian optimization,” in _EMNLP_ , Sep. 2017, pp. 372–382.
  * [66] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” _arXiv preprint arXiv:1503.02531_ , 2015.
  * [67] J. Mukhoti, V. Kulharia, A. Sanyal, S. Golodetz, P. Torr, and P. Dokania, “Calibrating deep neural networks using focal loss,” _NeurIPS_ , pp. 15 288–15 299, 2020.
  * [68] E. Zhu and J. Li, “Boundary smoothing for named entity recognition,” in _ACL_ , 2022, pp. 7096–7108.
  * [69] Y. Xiao, P. P. Liang, U. Bhatt, W. Neiswanger, R. Salakhutdinov, and L.-P. Morency, “Uncertainty quantification with pre-trained language models: A large-scale empirical analysis,” in _EMNLP Findings_ , Dec. 2022, pp. 7273–7284.
  * [70] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2980–2988.
  * [71] V. Vovk, A. Gammerman, and G. Shafer, _Algorithmic learning in a random world_.
  * [72] R. M. Neal, “Bayesian training of backpropagation networks by the hybrid monte carlo method,” Citeseer, Tech. Rep., 1992.
  * [73] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in _ICML_ , 2016, pp. 1050–1059.
  * [74] G. E. Hinton and D. Van Camp, “Keeping the neural networks simple by minimizing the description length of the weights,” in _6COLT93_ , 1993, pp. 5–13.
  * [75] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker, “On variational bounds of mutual information,” in _ICML_ , 2019, pp. 5171–5180.
  * [76] Y. Yu, H. Sajjad, and J. Xu, “Learning uncertainty for unknown domains with zero-target-assumption,” in _ICLR_ , 2023.
  * [77] N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel, “Bayesian active learning for classification and preference learning,” _arXiv preprint arXiv:1112.5745_ , 2011.
  * [78] M. Li, M. Li, K. Xiong, and J. Lin, “Multi-task dense retrieval via model uncertainty fusion for open-domain question answering,” in _EMNLP-Findings_ , 2021, pp. 274–287.
  * [79] V. Raina and M. Gales, “Answer uncertainty and unanswerability in multiple-choice machine reading comprehension,” in _ACL-Findings_ , 2022, pp. 1020–1034.
  * [80] J. S. Andersen and W. Maalej, “Efficient, uncertainty-based moderation of neural networks text classifiers,” in _ACL-Findings_ , 2022, pp. 1536–1546.
  * [81] R. Pop and P. Fulop, “Deep ensemble bayesian active learning: Addressing the mode collapse issue in monte carlo dropout via ensembles,” _arXiv preprint arXiv:1811.03897_ , 2018.
  * [82] Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and J. Snoek, “Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift,” _NeurIPS_ , 2019\.
  * [83] S. Fort, H. Hu, and B. Lakshminarayanan, “Deep ensembles: A loss landscape perspective,” _arXiv preprint arXiv:1912.02757_ , 2019.
  * [84] M. Sensoy, L. Kaplan, and M. Kandemir, “Evidential deep learning to quantify classification uncertainty,” _NeurIPS_ , 2018.
  * [85] B. Charpentier, D. Zügner, and S. Günnemann, “Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts,” _NeurIPS_ , pp. 1356–1367, 2020.
  * [86] Z. Zhang, M. Hu, S. Zhao, M. Huang, H. Wang, L. Liu, Z. Zhang, Z. Liu, and B. Wu, “E-ner: Evidential deep learning for trustworthy named entity recognition,” _arXiv preprint arXiv:2305.17854_ , 2023.
  * [87] L. Vilnis and A. McCallum, “Word representations via gaussian embedding,” _arXiv preprint arXiv:1412.6623_ , 2014.
  * [88] W. Zhou, F. Liu, and M. Chen, “Contrastive out-of-distribution detection for pretrained transformers,” in _EMNLP_ , 2021, pp. 1100–1111.
  * [89] A. Vazhentsev, G. Kuzmin, A. Shelmanov, A. Tsvigun, E. Tsymbalov, K. Fedyanin, M. Panov, A. Panchenko, G. Gusev, M. Burtsev, M. Avetisian, and L. Zhukov, “Uncertainty estimation of transformer predictions for misclassification detection,” in _ACL_ , 2022, pp. 8237–8252.
  * [90] C. K. Williams and C. E. Rasmussen, _Gaussian processes for machine learning_ , 2006.
  * [91] K. Shah, T. Conn, and L. Specia, “An investigation on the effectiveness of features for translation quality estimation,” in _MTSummit_ , 2013.
  * [92] D. Beck, T. Cohn, and L. Specia, “Joint emotion analysis via multi-task gaussian processes,” in _EMNLP_ , 2014, pp. 1798–1803.
  * [93] D. Beck, L. Specia, and T. Cohn, “Exploring prediction uncertainty in machine translation quality estimation,” in _SIGNLL_ , 2016, pp. 208–218.
  * [94] M. Assel, D. D. Sjoberg, and A. J. Vickers, “The brier score does not evaluate the clinical utility of diagnostic tests or prediction models,” _Diagnostic and prognostic research_ , pp. 1–7, 2017.
  * [95] J. Quinonero-Candela, C. E. Rasmussen, F. Sinz, O. Bousquet, and B. Scholkopf, “Evaluating predictive uncertainty challenge,” _Lecture Notes in Computer Science_ , pp. 1–27, 2006.
  * [96] Z. Lin, D. Phan, P. Pasupat, J. Z. Liu, and J. Shang, “On compositional uncertainty quantification for seq2seq graph parsing,” in _ICLR_ , 2023.
  * [97] R. El-Yaniv and Y. Wiener, “On the foundations of noise-free selective classification,” _J. Mach. Learn. Res._ , p. 1605–1641, 2010.
  * [98] J. Xin, R. Tang, Y. Yu, and J. Lin, “The art of abstention: Selective prediction and error regularization for natural language processing,” in _ACL-IJCNLP_ , 2021, pp. 1040–1051.
  * [99] J. Zhu, H. Wang, T. Yao, and B. K. Tsou, “Active learning with sampling by uncertainty and density for word sense disambiguation and text classification,” in _COLING_ , 2008, pp. 1137–1144.
  * [100] M. Yuan, H.-T. Lin, and J. Boyd-Graber, “Cold-start active learning through self-supervised language modeling,” in _EMNLP_ , 2020, pp. 7935–7948.
  * [101] L. Ein-Dor, A. Halfon, A. Gera, E. Shnarch, L. Dankin, L. Choshen, M. Danilevsky, R. Aharonov, Y. Katz, and N. Slonim, “Active Learning for BERT: An Empirical Study,” in _EMNLP_ , 2020, pp. 7949–7962.
  * [102] Y. Yu, L. Kong, J. Zhang, R. Zhang, and C. Zhang, “AcTune: Uncertainty-based active self-training for active fine-tuning of pretrained language models,” in _NAACL_ , 2022, pp. 1422–1436.
  * [103] K. Margatina, G. Vernikos, L. Barrault, and N. Aletras, “Active learning by acquiring contrastive examples,” in _EMNLP_ , 2021, pp. 650–663.
  * [104] D. Ru, J. Feng, L. Qiu, H. Zhou, M. Wang, W. Zhang, Y. Yu, and L. Li, “Active sentence learning by adversarial uncertainty sampling in discrete space,” in _EMNLP_ , 2020, pp. 4908–4917.
  * [105] S. Mukherjee and A. Awadallah, “Uncertainty-aware self-training for few-shot text classification,” in _NeurIPS_ , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., 2020, pp. 21 199–21 212.
  * [106] S. Lei, X. Zhang, J. He, F. Chen, and C.-T. Lu, “Uncertainty-aware cross-lingual transfer with pseudo partial labels,” in _NAACL-Findings_ , 2022, pp. 1987–1997.
  * [107] A. Shelmanov, D. Puzyrev, L. Kupriyanova, D. Belyakov, D. Larionov, N. Khromov, O. Kozlova, E. Artemova, D. V. Dylov, and A. Panchenko, “Active learning for sequence tagging with deep pre-trained models and Bayesian uncertainty estimates,” in _EACL_ , 2021, pp. 1698–1712.
  * [108] A. Chaudhary, J. Xie, Z. Sheikh, G. Neubig, and J. Carbonell, “A little annotation does a lot of good: A study in bootstrapping low-resource named entity recognizers,” in _EMNLP-IJCNLP_ , 2019, pp. 5164–5174.
  * [109] M. Liu, Z. Tu, T. Zhang, T. Su, X. Xu, and Z. Wang, “Ltp: a new active learning strategy for crf-based named entity recognition,” _Neural Processing Letters_ , pp. 2433–2454, 2022.
  * [110] Z. Lyu, D. Duolikun, B. Dai, Y. Yao, P. Minervini, T. Z. Xiao, and Y. Gal, “You need only uncertain answers: Data efficient multilingual question answering,” _TWorkshop on Uncertainty and Ro-Bustness in Deep Learning_ , 2020.
  * [111] A. Gidiotis and G. Tsoumakas, “Should we trust this summary? Bayesian abstractive summarization to the rescue,” in _ACL-Findings_ , 2022, pp. 4119–4131.
  * [112] D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, and D. Song, “Pretrained transformers improve out-of-distribution robustness,” in _ACL_ , 2020, pp. 2744–2751.
  * [113] Y. Shen, Y.-C. Hsu, A. Ray, and H. Jin, “Enhancing the generalization for intent classification and out-of-domain detection in slu,” in _COLING_ , 2021, pp. 2443–2453.
  * [114] S. Desai and G. Durrett, “Calibration of pre-trained transformers,” in _EMNLP_ , 2020, pp. 295–302.
  * [115] X. Zhang, F. Chen, C.-T. Lu, and N. Ramakrishnan, “Mitigating uncertainty in document classification,” in _NAACL_ , 2019, pp. 3126–3136.
  * [116] J. He, X. Zhang, S. Lei, Z. Chen, F. Chen, A. Alhamadani, B. Xiao, and C. Lu, “Towards more accurate uncertainty estimation in text classification,” in _EMNLP_ , 2020, pp. 8362–8372.
  * [117] Y. Hu and L. Khan, “Uncertainty-aware reliable text classification,” in _SIGKDD_ , 2021, p. 628–636.
  * [118] T. Z. Xiao, A. N. Gomez, and Y. Gal, “Wat zei je? detecting out-of-distribution translations with variational transformers,” _arXiv preprint arXiv:2006.08344_ , 2020.
  * [119] M. Wu, Y. Li, M. Zhang, L. Li, G. Haffari, and Q. Liu, “Uncertainty-aware balancing for multilingual and multi-domain neural machine translation training,” in _EMNLP_ , 2021, pp. 7291–7305.
  * [120] S. Garg and A. Moschitti, “Will this question be answered? question filtering via answer model distillation for efficient question answering,” in _EMNLP_ , 2021, pp. 7329–7346.
  * [121] N. Varshney, S. Mishra, and C. Baral, “Investigating selective prediction approaches across several tasks in IID, OOD, and adversarial settings,” in _ACL-Findings_ , 2022, pp. 1995–2002.
  * [122] N. Varshney, S. Mishra, and C. Baral, “Towards improving selective prediction ability of nlp systems,” in _RepL4NLP_ , 2022, pp. 221–226.
  * [123] D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” in _ICLR_ , 2017.
  * [124] D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” in _ICLR_ , 2019.
  * [125] Q. Zhang, A. Lipani, S. Liang, and E. Yilmaz, “Reply-aided detection of misinformation via bayesian deep learning,” in _WWW_ , 2019, p. 2333–2343.
  * [126] E. Kochkina and M. Liakata, “Estimating predictive uncertainty for rumour verification models,” in _ACL_ , 2020, pp. 6964–6981.
  * [127] Y. Feng, S. Mehri, M. Eskenazi, and T. Zhao, ““none of the above”: Measure uncertainty in dialog response retrieval,” in _ACL_ , 2020, pp. 2013–2020.
  * [128] J. Mukhoti, A. Kirsch, J. van Amersfoort, P. H. Torr, and Y. Gal, “Deterministic neural networks with inductive biases capture epistemic and aleatoric uncertainty,” _arXiv preprint arXiv:2102.11582_ , p. 13, 2021.
  * [129] S. Ryu, S. Kim, J. Choi, H. Yu, and G. G. Lee, “Neural sentence embedding using only in-domain sentences for out-of-domain sentence detection in dialog systems,” _Pattern Recognition Letters_ , pp. 26–32, 2017.
  * [130] S. Ryu, S. Koo, H. Yu, and G. G. Lee, “Out-of-domain detection based on generative adversarial network,” in _EMNLP_ , 2018, pp. 714–718.
  * [131] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _stat_ , p. 10, 2014.
  * [132] M. Tan, Y. Yu, H. Wang, D. Wang, S. Potdar, S. Chang, and M. Yu, “Out-of-domain detection for low-resource text classification tasks,” in _EMNLP-IJCNLP_ , 2019, pp. 3566–3572.
  * [133] K. Lee, H. Lee, K. Lee, and J. Shin, “Training confidence-calibrated classifiers for detecting out-of-distribution samples,” in _ICLR_ , 2018\.
  * [134] S. Geng, P. Gao, Z. Fu, and Y. Zhang, “Romebert: Robust training of multi-exit bert,” _arXiv preprint arXiv:2101.09755_ , 2021.
  * [135] J. Xin, R. Tang, Y. Yu, and J. Lin, “BERxiT: Early exiting for BERT with better fine-tuning and extension to regression,” in _EACL_ , 2021, pp. 91–104.
  * [136] W. Zhou, C. Xu, T. Ge, J. McAuley, K. Xu, and F. Wei, “Bert loses patience: Fast and robust inference with early exit,” _NeurIPS_ , pp. 18 330–18 341, 2020.
  * [137] R. Schwartz, G. Stanovsky, S. Swayamdipta, J. Dodge, and N. A. Smith, “The right tool for the job: Matching model and instance complexities,” in _ACL_ , 2020, pp. 6640–6651.
  * [138] T. Schuster, A. Fisch, T. Jaakkola, and R. Barzilay, “Consistent accelerated inference via confident adaptive transformers,” in _EMNLP_ , 2021, pp. 4962–4979.
  * [139] L. Wei, D. Hu, W. Zhou, and S. Hu, “Uncertainty-aware propagation structure reconstruction for fake news detection,” in _COLING_ , 2022, pp. 2759–2768.
  * [140] X. Chen, M. Chen, W. Shi, Y. Sun, and C. Zaniolo, “Embedding uncertain knowledge graphs,” in _AAAI_ , 2019, pp. 3363–3370.
  * [141] X. Chen, M. Boratko, M. Chen, S. S. Dasgupta, X. L. Li, and A. McCallum, “Probabilistic box embeddings for uncertain knowledge graph reasoning,” in _NAACL_ , 2021, pp. 882–893.
  * [142] K. Boutouhami, J. Zhang, G. Qi, and H. Gao, “Uncertain ontology-aware knowledge graph embeddings,” in _JIST_ , 2020, pp. 129–136.
  * [143] Y. Li, L. Liu, and S. Shi, “Rethinking negative sampling for handling missing entity annotations,” in _ACL_ , 2022, pp. 7188–7197.
  * [144] S. Wang, Y. Liu, C. Wang, H. Luan, and M. Sun, “Improving back-translation with uncertainty-based confidence estimation,” in _EMNLP-IJCNLP_ , 2019, pp. 791–802.
  * [145] Y. Zhou, B. Yang, D. F. Wong, Y. Wan, and L. S. Chao, “Uncertainty-aware curriculum learning for neural machine translation,” in _ACL_ , 2020, pp. 6934–6944.
  * [146] S. Zhang, C. Gong, and E. Choi, “Knowing more about questions can help: Improving calibration in question answering,” in _ACL-IJCNLP_ , 2021, pp. 1958–1970.
  * [147] N. Kertkeidkachorn, X. Liu, and R. Ichise, “Gtranse: generalizing translation-based model on uncertain knowledge graph embedding,” in _JSAI_ , 2020, pp. 170–178.
  * [148] T. Glushkova, C. Zerva, R. Rei, and A. F. Martins, “Uncertainty-aware machine translation evaluation,” in _EMNLP-Findings_ , 2021, pp. 3920–3938.
  * [149] Z. Jiang, J. Araki, H. Ding, and G. Neubig, “How can we know when language models know? on the calibration of language models for question answering,” _TACL_ , pp. 962–977, 2021.
  * [150] S. Lin, J. Hilton, and O. Evans, “Teaching models to express their uncertainty in words,” 2022.
  * [151] K. Zhou, D. Jurafsky, and T. Hashimoto, “Navigating the grey area: Expressions of overconfidence and uncertainty in language models,” _arXiv preprint arXiv:2302.13439_ , 2023.
  * [152] A. Shen, D. Beck, B. Salehi, J. Qi, and T. Baldwin, “Modelling uncertainty in collaborative document quality assessment,” in _Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)_ , 2019, pp. 191–201.
  * [153] S. Jean, O. Firat, K. Cho, R. Memisevic, and Y. Bengio, “Montreal neural machine translation systems for WMT’15,” in _Proceedings of the Tenth Workshop on Statistical Machine Translation_ , 2015, pp. 134–140.
  * [154] P. Koehn and R. Knowles, “Six challenges for neural machine translation,” in _Proceedings of the First Workshop on Neural Machine Translation_ , 2017, pp. 28–39.
  * [155] S. Wang, Z. Tu, S. Shi, and Y. Liu, “On the inference calibration of neural machine translation,” in _ACL_ , 2020, pp. 3070–3079.
  * [156] T. Newberry and T. Ord, “The parliamentary approach to moral uncertainty.”
  * [157] D. Hendrycks, “Natural selection favors ais over humans,” 2023.