# Beyond Confidence: Reliable Models Should Also Consider Atypicality

Mert Yuksekgonul
Stanford University
<EMAIL_ADDRESS>
&Linjun Zhang
Rutgers University
<EMAIL_ADDRESS>
James Zou333Joint Advisors.
Stanford University
<EMAIL_ADDRESS>
&Carlos Ernesto Guestrin333Joint Advisors.
Stanford University, CZ Biohub
<EMAIL_ADDRESS>

###### Abstract

While most machine learning models can provide confidence in their
predictions, confidence is insufficient to understand a prediction’s
reliability. For instance, the model may have a low confidence prediction if
the input is not well-represented in the training dataset or if the input is
inherently ambiguous. In this work, we investigate the relationship between
how atypical (rare) a sample or a class is and the reliability of a model’s
predictions. We first demonstrate that atypicality is strongly related to
miscalibration and accuracy. In particular, we empirically show that
predictions for atypical inputs or atypical classes are more overconfident and
have lower accuracy. Using these insights, we show incorporating atypicality
improves uncertainty quantification and model performance for discriminative
neural networks and large language models. In a case study, we show that using
atypicality improves the performance of a skin lesion classifier across
different skin tone groups without having access to the group attributes.
Overall, _we propose that models should use not only confidence but also
atypicality to improve uncertainty quantification and performance_. Our
results demonstrate that simple post-hoc atypicality estimators can provide
significant value.111Our code is available at
https://github.com/mertyg/beyond-confidence-atypicality

##  1 Introduction

_Typicality_ is an item’s resemblance to other category members [59]. For
example, while a dove and a sparrow are typical birds, a penguin is an
atypical bird. Many works from cognitive science (e.g., [58, 61, 47]) suggest
that typicality plays a crucial role in category understanding. For instance,
humans have been shown to learn, remember, and refer to typical items faster
[49]. Similarly, the representativeness heuristic is the tendency of humans to
use the typicality of an event as a basis for decisions [66]. This cognitive
bias is effective for making swift decisions, but it can lead to poor
judgments of uncertainty. For instance, the likelihood of typical events can
be overestimated [66] or uncertainty judgments can be inferior for atypical
events [67].

While it is hard to quantify the uncertainty of human judgments, machine
learning models provide confidence in their predictions. However, confidence
alone can be insufficient to understand the reliability of a prediction. For
instance, a low-confidence prediction could arise from an ambiguity that is
easily communicated, or due to the sample being underrepresented in the
training distribution. Similarly, a high-confidence prediction could be
reliable or miscalibrated. Our main proposal is that _models should quantify
not only the confidence but also the atypicality_ to understand the
reliability of predictions or the coverage of the training distribution.
However, many machine learning applications rely on pretrained models that
solely provide confidence levels, devoid of any measure of atypicality.

Contributions: To support our position, we use a simple formalization of
atypicality estimation. With the following studies, we show that by using
simple atypicality estimators, we can:

1\. Understand Prediction Quality: Calibration is a measure that assesses the
alignment between predicted probabilities of a model and the true likelihoods
of outcomes [21]. Neural networks [20] or even logistic regression [7] can be
miscalibrated out-of-the-box. Here, we argue that using atypicality can give
insights into when a model’s confidence is reliable. Through theoretical
analysis and extensive experimentation, we demonstrate that atypicality
results in lower-quality predictions. Specifically, _we show that predictions
for atypical inputs and samples from atypical classes are more overconfident
and have lower accuracy._

2\. Improve Calibration and Accuracy: _Recalibration_ methods offer some
mitigation to miscalibration [20] by adjusting a probabilistic model. We show
that models need different adjustments according to the atypicality of inputs
and classes, and atypicality is a key factor in recalibration. In light of
these findings, we propose a simple method: _Atypicality-Aware Recalibration_.
Our recalibration algorithm takes into account the atypicality of the inputs
and classes and is simple to implement. We show that complementing
recalibration methods with atypicality improves uncertainty quantification and
the accuracy of predictors. Further, in a case study for skin lesion
classification, we show that atypicality awareness can improve performance
across different skin-tone subgroups without access to group annotations.

3\. Improve Prediction sets: An alternative approach to quantify uncertainty
is to provide prediction sets that contain the label with high probability
[2]. Here, we investigate existing methods with atypicality and show that
prediction sets could underperform for atypical or low-confidence samples. By
using atypicality, we demonstrate the potential for improving prediction sets.

Overall, we propose that models should also consider atypicality, and we show
simple- and easy-to-implement atypicality estimators can provide significant
value.

Figure 1: Atypicality in Uncertainty. Left: We show examples from the
ImageNet-R dataset with our atypicality framework. Right: We provide a
conceptualization of the quadrants. Using atypicality, we can understand
prediction quality (§3), improve predictions (§4), and prediction sets (§5).

##  2 Interpreting Uncertainty with Atypicality

Motivation: In many machine learning applications, we have access to a model’s
confidence, which aims to quantify the likelihood that a prediction will be
accurate. In classification, model output is a probability distribution over
classes and confidence is the predicted probability of the top class, i.e.
$\max_{y}~{}\mathbb{\hat{P}}(Y=y|X=x)$. In practical scenarios, confidence is
the primary tool used to evaluate the reliability of a prediction where higher
confidence is associated with better predictions. However, the uncertainty in
confidence can stem from different sources that require different treatment
[46].

Here, we call a prediction _reliable_ if it is high-confidence and well-
calibrated. High confidence could be reliable or miscalibrated, and low
confidence could be due to ambiguity or rare inputs. We propose that
_atypicality_ provides a natural way to understand reliability when combined
with confidence. A sample is called typical if it is well-represented in the
previously observed samples, e.g., an image of a dog that is similar to other
dogs in the training data. However, if the image is unlike any other seen
during training, it is atypical. We argue that atypicality can help us
interpret a prediction’s reliability. Below we categorize samples and
predictions according to atypicality and confidence in four quadrants (Figure
1).

High-confidence and representative: Reliable predictions often fall within the
Reliable Quadrant, which includes _typical, high-confidence_ samples. These
samples are well-represented in the training dataset (typical), thus we expect
the high-confidence prediction to be reliable. For instance, the first image
on the top left (Figure 1) is a typical golden retriever and the model makes a
reliable prediction.

High-confidence yet far from the support: Having high-confidence does not
always indicate reliability. If the sample does not have support in the
training distribution, the confidence could be miscalibrated. Such samples lie
in the  Extrapolation Quadrant which contains _atypical, high-confidence_
samples. For instance, the second image in the top right of Figure 1 is a
_toy_ hog and the model has not seen similar ones during training.

Low confidence due to ambiguity: In contrast, low confidence could also be
reliable when it correctly reflects an ambiguity. Such samples are in the
Ambiguous Quadrant that contains _typical, low-confidence_ samples. These are
typical since they may represent multiple classes; yet, due to ambiguity, the
model’s confidence is low. For instance, the second image in the bottom left
of Figure 1 can both be a hog and a comic book.

Low confidence and rare: For samples that are not well-represented in training
data, we expect to have low-quality predictions.  Untrustworthy Quadrant
comprises _atypical, low-confidence_ samples that can include extremely rare
subgroups, for which we expect miscalibration and lower accuracy. For example,
the image in Figure 1 bottom right is an origami hog that was not seen in
training.

These examples suggest that relying solely on confidence does not provide a
complete understanding of the reliability of the predictions, and we can use
atypicality to interpret and improve reliability.

Formalizing Atypicality: Atypicality here is defined with respect to the
training distribution. Informally, an input or a class is atypical if it is
not _well-represented_ in the training distribution. For instance, if there
are no or limited similar examples to an input, it can be called atypical.
Note that this notion is not restricted to being ‘out-of-distribution’ [24],
since in-distribution groups could also be atypical or rare, and our goal is
to perform reliably for the entire spectrum.

Formally, let $X\in\mathbb{R}^{d}$ be the random variable denoting features
and $Y\in\mathcal{Y}=\\{1,2,...,C\\}$ denote the class, where we focus on
classification.

######  Definition 2.1 (Input Atypicality).

We define the atypicality of the input $x$ as222Here atypicality differs from
’typical sets’ in information theory that refers to a sequence of variables
[65].

$a_{X}(x)=-\max_{y}\log\mathbb{P}(X=x|Y=y).$

We use the logarithm of the class-conditional densities due to high
dimensionality and density values being close to zero. Intuitively, for a dog
image $x$, if $\mathbb{P}(X=x|Y=\textrm{dog})$ has a low value, we call $x$ an
atypical dog image. Overall, if $a(x)$ is high, then we call $x$ an atypical
input. Specifically, if an input is not typical for any class, then it is
atypical with respect to the training distribution. Similarly, we can also use
marginal density, $\mathbb{P}(X=x)$, or distance333For an input $x$, if the
nearest neighbor (NN) distance is large, then we call $x$ atypical as all
inputs in the training set are far from $x$. Density and distance are
connected through non-parametric density estimation and [29] shows that NN
distance can recover high-density regions. to quantify atypicality.

Similarly, the notion of atypical (rare) classes is prevalent in imbalanced
classification [13, 71]. Ensuring reliable performance for atypical classes
can be safety-critical, e.g., for a rare presence of dangerous melanoma [16].
We define class atypicality in the following:

######  Definition 2.2 (Class Atypicality).

For a class $y$, atypicality of a class is defined as

$a_{Y}(y)=-\log\mathbb{P}(Y=y).$

Estimating Atypicality for Discriminative Models: Quantifying input
atypicality requires access to the class-conditional / marginal distributions.
In practice, for neural networks trained for classification, these
distributions are unavailable and we need to perform the estimation. This
estimation can be challenging if the dimensionality is large, or the data is
unstructured, requiring assumptions about the distributions. Prior works [46,
36] showed that Gaussian Mixture Models (GMMs) in the embedding space of
neural networks can be used to model these distributions.

In experiments, we use Gaussians with shared covariance, i.e.
$\hat{\mathbb{P}}(X=x|Y=c)\sim~{}N(\hat{\mu}_{c},\hat{\Sigma})$, to estimate
input atypicality. We perform the estimation in the penultimate layer of
neural networks used to make predictions, using maximum-likelihood estimation
with samples from the training data. We explore other metrics, such as
$k$-Nearest Neighbors distance. We give implementation details and results
with different metrics in Appendix A.1. With these estimators, atypicality
estimation is cheap and can run on a CPU. Our goal is to show that simple
estimators can already reap large benefits. Our framework is flexible and
exploring more sophisticated estimators is a topic for future work.

Atypicality for LLMs: LLMs are increasingly used for classification [6].
Modern LLMs are autoregressive models that compute a marginal distribution,
$\hat{\mathbb{P}}_{\textrm{LLM}}(X)$. We compute the negative log-likelihood
of a prompt or a label and use this as an atypicality metric, i.e.
$a_{X}(x)=-\log\hat{\mathbb{P}}_{\textrm{LLM}}(x)$,
$a_{Y}(y)=-\log\hat{\mathbb{P}}_{\textrm{LLM}}(y)$. Similar to the
discriminative setting, atypicality here aims to quantify whether a prompt is
well-represented in the training data. A larger value for $a(X)$ would imply
that a prompt is more atypical. Below, we present typical and atypical prompts
for the AGNews dataset:

Classify the news articles into the categories of World, Sports, Business, and
Technology. Article: Safin tallest obstacle to host #39;s patriotic games hope
AS tennis fans go, Houston #39;s Jim #39;Mattress Mack #39; McIngvale is very
rich, extremely forthright, exceedingly patriotic and unflinchingly
Republican. Answer: Atypicality: $353.45$, Percentile: $\%94.5$ Classify the
news articles into the categories of World, Sports, Business, and Technology.
Article: Delta Air Lines Prepares Chapter 11 Filing Delta Air Lines Inc. could
file for Chapter 11 bankruptcy protection as soon as next week, a source
familiar with the matter said yesterday. Answer: Atypicality: $171.50$.
Percentile: $\%0.9$

##  3 Understanding the Prediction Quality with Atypicality

In this section, we show how our framework can be applied to understand the
quality of predictions. Experimental Setup: We investigate three
classification settings across a range of datasets:

  1. 1.

Balanced Supervised Classification: We use ResNet18-50-152 [28], WideResNet28
[73], RoBERTa [39] trained on ImageNet [15], CIFAR10,100 [32], MNLI [70]
respectively.

  2. 2.

Imbalanced Supervised Classification: We use ResNet18, ResNext50, ResNet152
trained on CIFAR-LT, ImageNet-LT and Places365-LT where models and data are
mostly from [71, 45]. Note that all of the test and validation sets have
balanced class distributions.

  3. 3.

Classification with LLMs: We use open-source Alpaca7B [64] on IMDB [43], TREC
[40], and AG News [76] datasets with the prompts from [75].

(a)

(b)

(c)

(d)

Figure 2: Atypical Samples Have Low-Quality Predictions. (a) Here, samples are
grouped according to the Input Atypicality (x-axis) and Confidence (y-axis),
to the right meaning more atypical. Values show the difference between
confidence and accuracy, lighter color indicates more overconfidence. Within
the same confidence range, atypical groups have more miscalibration and are
more overconfident. (b,c,d) Predictions for atypical samples are less accurate
and more miscalibrated in balanced and imbalanced supervised classification
and classification with LLMs.

Details on datasets, models, and prompts are in Appendix B. Our experiments
were run on a single NVIDIA A100-80GB GPU. We report error bars over 10 random
calibration/test splits.

###  3.1 Atypicality is Correlated with Miscalibration

We first explore the importance of atypicality to understand model
calibration. Calibration quantifies the quality of a probabilistic model [21].
Informally, a model is considered perfectly calibrated if all events that are
predicted to occur $P\%$ of the time occur $P\%$ of the time for any
$P\in[0,100]$.

For the sake of simplicity, consider a binary classification problem where the
predictor is $\hat{\mathbb{P}}:\mathcal{X}\to[0,1]$. We quantify
miscalibration with Calibration Error (CE):

$\textrm{CE}[\hat{\mathbb{P}}]=\mathbb{E}[|\mathbb{P}(Y|\hat{\mathbb{P}}(X)=p)-p|].$

It is computationally infeasible to calculate the above expectation with the
conditional probability $\mathbb{P}(Y|\hat{\mathbb{P}}(X)=p)$. In practice, we
use a binned version of this quantity, Expected Calibration Error (ECE) [50,
20], to estimate CE. See Appendix C.1 for a formal definition.

Here, we aim to examine the relationship between model calibration and
atypicality. Given any $K>1$, we consider the quantiles of $a(X)$,
$a_{1},a_{2},\ldots,a_{K+1}$ such that
$\mathbb{P}(a(X)\in(a_{k},a_{k+1}])=1/K$ for $k\in[K]$. For imbalanced
classification problems, we compute the quantiles using the class atypicality.
Specifically, we investigate the atypicality-conditional calibration error
$\textrm{ECE}[\hat{\mathbb{P}}\mid a(X)\in(a_{k},a_{k+1}]]$, i.e., the
expected calibration error of an input that falls within the atypicality
quantile $k$.

Atypical Examples are Poorly Calibrated: In Figure 2a, we show the
distribution of miscalibration where each bin within the grid contains the
intersection of the corresponding confidence and atypicality quantiles. We
observe that within the same confidence range, predictions for atypical points
have lower accuracies and are more overconfident. In other words, predictions
in the  Extrapolation or  Untrustworthy regions are more miscalibrated than
the ones in the typical regions.

In Figure 2b, we split inputs into quantiles according to atypicality and
compute the ECE and Accuracy for each group. Results show a monotonic
relationship between atypicality and ECE or Accuracy across the three
settings. Specifically, we see that predictions for atypical inputs or samples
from rare classes are more miscalibrated and have lower accuracy. For samples
from rare classes, the model overpredicts the probabilities of the typical
class, hence we have overconfidence and low accuracy. Appendix C.3, and §4
present figures and tables for all model and dataset pairs.

###  3.2 Theoretical Analysis: Characterizing Calibration Error with
Atypicality

We characterize how calibration error varies with atypicality in a tractable
model that is commonly used in machine learning theory [7, 8, 72, 12]. Our
theoretical analysis further supports our empirical findings.

Data Generative Model: We consider the well-specified logistic model for
binary classification with Gaussian data, where $Y\in\\{-1,1\\}$ and the
$\mathbb{P}(Y=1|X)$ is defined by the sigmoid function:

$\mathbb{P}(Y=1\mid X)=\sigma(\langle\beta^{*},X\rangle),\quad X\sim
N(0,I_{d}).$

Where $I_{d}$ denotes the $d$-dimensional identity matrix, $\beta^{*}$ is the
ground truth coefficient vector, $\sigma(x)=1/(1+e^{-x})$, and we have
$i.i.d.$ observations $\\{(x_{i},y_{i})\\}_{i=1}^{n}$ sampled from the above
distribution.

The Estimator:  We focus on studying the solution produced by minimizing the
logistic loss

$\hat{\beta}=\arg\min_{\beta}\frac{1}{n}\sum_{i=1}^{n}[\log(1+\exp(\beta^{\top}x_{i}))-y_{i}\cdot\beta^{\top}x_{i}].$

For $k\in\\{-1,1\\}$, $\hat{\mathbb{P}}_{k}(x)$ is an estimator of
$\mathbb{P}(y=k|x)$, with the form
$\hat{\mathbb{P}}_{k}(x)=\frac{1}{e^{-k\cdot\hat{\beta}^{\top}x}+1}$.

Calibration: We consider all $x$ where $\mathbb{P}_{1}(x)>1/2$, as
$\mathbb{P}_{1}(x)\leq 1/2$ can be analyzed similarly by symmetry (see
Appendix G). For $u\in(1/2,1)$, the signed calibration error at a confidence
level $u$ is

$u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u).$

We want to show that when $X$ is atypical, i.e., when $a(X):=\|X\|^{2}/2$ is
larger555The definition of atypicality follows from the marginal likelihood of
the data model: density for the Gaussian with zero mean and identity
covariance. , the accuracy $\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)$
would be generally smaller than the confidence $u$ (over-confidence).

######  Theorem 3.1.

Consider the data generative model and the learning setting above. For any
$K>1$, suppose we consider the quantiles of $a(X)$,
$a_{1},a_{2},...,a_{K},a_{K+1}$ such that
$\mathbb{P}(a(X)\in(a_{k},a_{k+1}])=1/K$ for $k\in[K]$. We assume
$\|\beta^{*}\|\leq c_{0}$, and $d/n=\kappa$, for some sufficiently small
$c_{0}$. Then, for sufficiently large $n$, for $k=2,\ldots,K$, we have

$\displaystyle\mathbb{E}_{u\sim\hat{\mathbb{P}}_{1}(X)}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in(a_{k},a_{k+1}]]>$
$\displaystyle\mathbb{E}_{u\sim\hat{\mathbb{P}}_{1}(X)}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in(a_{k-1},a_{k}]]\geq 0.$

That is, the resulting classifier is over-confident, and the level of over-
confidence becomes larger when the data is more atypical (with larger $a(X)$).
Further, the gap becomes larger for smaller sample sizes $n$. The proof of the
theorem is in Appendix G.2 and builds on the results from [7, 62].

##  4 Using Atypicality to Improve Recalibration

Here, we show how atypicality can complement and improve post-hoc calibration.
In §2, we observed that predictions for atypical inputs and samples from
atypical classes are more overconfident with lower accuracy. We next show that
taking input and class atypicality into account improves calibration.

###  4.1 Parametric Recalibration: Different Groups Need Different
Temperatures

Temperature scaling (TS), a single parameter variant of Platt Scaling [52], is
a simple recalibration method that calibrates the model using a single
parameter. The predictor is of the form

$\log\hat{\mathbb{P}}_{\textrm{TS}}(Y|X)\propto\log\hat{\mathbb{P}}(Y|X)/\tau,$
(1)

where $\hat{\mathbb{P}}(Y|X)$ is the model that takes an input and outputs
scores/logits, and $\tau$ is the temperature parameter. In practice, $\tau$ is
optimized using a calibration set to minimize a proper scoring rule [21, 9]
such as the cross-entropy loss.

To understand the behavior of TS with respect to atypicality, we separately
perform TS on points grouped according to the atypicality quantiles. Let us
denote the temperature fitted to the quantile covering
$a(X)\in(a_{k-1},a_{k}]$ by $\tau_{a_{k}}$. In Appendix Figure 10 we observe
an increasing relationship between $a_{k}$ and $\tau_{a_{k}}$. Different
atypicality groups need different adjustments, and more atypical groups need
larger temperatures. _This suggests that being atypicality-aware can improve
calibration. While a single temperature value improves average calibration, it
may hurt certain groups._

(a)

(b)

(c)

Figure 3: Post-hoc Recalibration for Classification. (a) Balanced Supervised
Classification: Atypicality-Aware Recalibration improves the calibration of
models trained with balanced datasets, across atypicality groups. (b)
Imbalanced Supervised Classification: Atypicality-Aware Recalibration improves
both the calibration across groups and the overall accuracy of models trained
with imbalanced datasets. (c) Classification with LLMs: Atypicality-Aware
Recalibration improves both the calibration across groups and the overall
accuracy of LLMs performing classification.

###  4.2 Atypicality-Aware Recalibration

We showed that predictions are more reliable when the input is typical.
However, predictions are less reliable for atypical inputs, and we may need
further revision. An analogy can be drawn to decision-making literature where
opinions of individuals are combined with geometric averaging weighted by
their expertise [17, 3]. Analogously, we propose _Atypicality-Aware
Recalibration (AAR)_ a method designed to address the reliability issues
identified in dealing with atypical inputs:

$\hat{\mathbb{P}}_{\textrm{AAR}}(Y|X)=\frac{\hat{\mathbb{P}}(Y|X)^{\psi(a(X))}\exp({S_{Y}})^{1-\psi(a(X))}}{Z(X)},$
(2)

where $\psi(a(X))$ is a function of input atypicality, $S_{Y}$ is a tunable
score for class $Y$, $Z(X)$ is the normalization term. Intuitively, when the
input is typical, we trust the model confidence; otherwise, we use a score for
the given class estimated from the calibration set. Note that this form
simplifies to

$\log{\hat{\mathbb{P}}_{\textrm{AAR}}(Y|X)}\propto{\phi(a(X))}\log{\hat{\mathbb{P}}(Y|X)}+S_{Y},$
(3)

where we subsume $(1-\psi(a(X))$ into $\phi(a(X))$. We give a simple
interpretation of this form: the multiplicative term is an atypicality-
dependent temperature, and the additive term is a class-dependent correction
where $\exp{(S_{Y})}$ can be considered to induce a correction distribution
over classes estimated from the calibration set.

Intuitively, when $\psi(a(X))=0$, the output reduces to a fixed distribution
over classes that was estimated using the calibration set. This distribution
can be seen to induce a prior probability over classes, and $\psi$ controls
the tradeoff between this prior and the model’s predictive distribution. As
the point becomes more typical, this distribution is closer to the model’s
predictive distribution. In Appendix Figure 11, we show how these values
behave with class atypicality. We find that rare classes require larger
positive corrections with larger $S_{Y}$.

Implementation Details:  Following TS, we minimize the cross-entropy loss on a
calibration set. With the temperature-atypicality relationship observed in
Figure 10 we choose to instantiate the multiplicative factor as a quadratic
function, where $\phi(a(X))=c_{2}a(X)^{2}+c_{1}a(X)+c_{0}$ and in total we
have $|\\{S_{1},..,S_{|\mathcal{Y}|},c_{0},c_{1},c_{2}\\}|=|\mathcal{Y}|+3$
interpretable parameters. Once the embeddings and logits are computed, AAR
runs on a CPU in under 1 minute for all experimented settings.

Similar to our adaptive interpretation, a concurrent work, Adaptive
Temperature Scaling (AdaTS) [30], uses temperature scaling where the
temperature is parameterized by a Variational Autoencoder(VAE) [34] and a
multi-layer perceptron on top of the VAE embeddings. In the below experiments,
we give results with AdaTS as a baseline when applicable.

For Balanced Supervised Classification, in Figure 3a, we observe that being
atypicality aware improves recalibration across all groups. We perform
comparably to AdaTS, where the temperature function has in the order of
millions of parameters, whereas AAR has only $|\mathcal{Y}|+3$ parameters.

In Imbalanced Supervised Classification (Figure 3b), our algorithm not only
provides better calibration rates across all classes but also improves overall
accuracy. Note that only our method can change accuracy (due to the additive
term), and it performs better than other baselines in terms of ECE across all
classes. Further, the second column shows using Progressive Balancing [75] in
training, showing that our post-hoc method can complement methods that modify
training procedures.

For Classification with LLMs, we add an LLM calibration baseline Content-Free
Calibration (CF) [75]. We cannot use AdaTS as the embeddings are not fixed in
size. In Figure 3c, we see AAR has better calibration and accuracy across the
three datasets. Namely, by adjusting the LLM output using the LLM atypicality,
we can adjust the probabilities to increase the prediction quality.

###  4.3 Case Study: Fairness through Atypicality-Awareness

Machine learning models reportedly have performance disparity across subgroups
[5] due to factors such as varying sample size or noise levels [11]. For
instance, skin lesion classifiers can exhibit performance disparity across
different skin tones [16]. Fitzpatrick17k [18] is a dataset of clinical images
with Fitzpatrick skin tone annotations between 1-to-6, where a larger number
means darker skin tones, and when annotators do not agree, it is labeled as
‘Unknown’. We explore the classification problem with 9 classes indicating the
malignancy and the type of skin condition, using a ResNet18/34 pretrained on
ImageNet and finetuned on this task (See Appendix F).

When the goal is to improve performance across groups, one can use group
annotations and optimize performance within each group [26, 31]. Here, we
investigate how complementing recalibration with atypicality can improve
prediction quality across all groups _without group annotations_. For
comparison, we perform 3 recalibration methods: TS, AAR, and Skin-Tone
Conditional TS which calibrates the model individually for each skin-tone
group with TS. Since the skin-tone conditional calibration uses group
attributes, ideally it should act as an oracle.

In Figure 4, we give the Accuracy and ECE analyses where AAR improves
performance across all groups. For instance, the worst-group Accuracy (0.69)
or ECE (0.072) with AAR is close to the best-group Accuracy (0.63) or ECE
(0.062) with the other two methods. Overall, _our findings suggest that
Atypicality-Awareness can complement fairness-enforcing methods, and improve
performance even when the group annotations are unavailable._ We hypothesize
that with AAR, we can perform better than using supervised group attributes
since groups may not have sufficient sample size in the calibration set (131,
1950, 1509, 555 samples for Unknown, 1&2, 3&4, and 5&6 respectively), and we
can leverage atypicality to offer some mitigation. Further investigating how
to leverage atypicality to improve fairness and factors affecting performance
disparities is a promising direction for future work [11].

Figure 4: Improving Group Performance through Atypicality-Awareness. Here we
show that AAR improves the calibration and accuracy of models across different
skin tone groups. With AAR, we can improve both the worst group performance
and overall performance significantly without using group attributes. TS curve
is less visible since it significantly overlaps with Skin Tone Conditional.

##  5 Improving Prediction Sets with Atypicality

Conformal Prediction [63, 1] is a framework that assigns a calibrated
prediction set to each instance. The goal is to find a function
$\mathcal{C}:\mathcal{X}\rightarrow 2^{\mathcal{Y}}$ that returns a subset of
the label space such that $Y\in\mathcal{C}(X)$ with high probability. The
framework aims to guarantee _marginal coverage_ , i.e.,
$\mathbb{P}(Y\in\mathcal{C}(X))\geq 1-\alpha$, for a choice of $\alpha$. We
investigate two conformal calibration methods, Adaptive Prediction Sets (APS)
[60] and Regularized APS (RAPS) [2]. Let $\pi(X)$ be the permutation of the
label set that sorts $\hat{\mathbb{P}}(Y=c|X)$, i.e. the predicted
probabilities for each class $c$ after TS. The prediction sets are produced by
the function $\mathcal{C}(x)=\\{y:s(x,y)\leq\hat{q}\\}$, and these methods fit
the threshold $\hat{q}$ for a choice of the scoring function. APS uses the
cumulative sum of the predicted probabilities
$s(x,y)=\sum_{j=1}^{c}\hat{\mathbb{P}}(Y=j|X)$, where $y=\pi_{c}(X)$.
Intuitively, if the model was perfectly calibrated, we would have expected to
have $\hat{q}=1-\alpha$. Similarly, RAPS builds on the idea that tail
probabilities are noisy and regularizes the number of samples in the
prediction set.

Building on our ideas in the previous sections we implement Atypicality-Aware
prediction sets, namely _AA-APS_ and _AA-RAPS_ in the following way: We first
group points according to their confidence and atypicality quantiles. A group
$G$ here is defined using 4 thresholds, namely $G={x:(l_{a}^{(G)}<q_{a}(x)\leq
h_{a}^{(G)})\land(l_{c}^{(G)}<q_{c}(x)\leq h_{c}^{(G)})}$ where $q_{a}(x)$ and
$q_{c}(x)$ denote the atypicality and confidence quantiles for the sample $x$,
$l_{a}^{(G)}$ and $h_{a}^{(G)}$ denote the atypicality lower and upper bounds
for group $G$, and $l_{c}^{(G)}$ and $h_{c}^{(G)}$ denote the confidence lower
and upper bounds for group $G$. Using a calibration set, these bounds are
simply determined by the quantiles of confidence and atypicality statistics.
Then, we fit separate thresholds $\hat{q}_{G}$ for each group’s prediction
sets with APS or RAPS as subroutines. This allows us to have an adaptive
threshold depending only on the atypicality and confidence of predictions.

Figure 5: Improving Conformal Calibration with Atypicality for ResNet50 on
ImageNet. Here we show that atypicality awareness improves conformal
calibration performance across different groups. Methods are fitted to satisfy
$95\%$ coverage. We observe that APS and RAPS do not satisfy conditional
coverage for high atypicality regions or low confidence regions.

In Figure 5, we provide the coverage plots for APS and RAPS in the first and
third columns. Even though marginal coverage is satisfied, models do not
satisfy conditional coverage for atypical inputs or low-confidence
predictions. We observe that being Atypicality-Aware improves coverage across
otherwise underperforming groups. Further, AA-APS has lower set sizes on
average than APS ($15.6$ vs $21.3$). While RAPS has a lower average set size
than AA-RAPS ($4.2$ vs $9.1$) AA-RAPS has smaller set sizes for high-
confidence samples, whereas a larger set size for low-confidence samples where
the coverage is not met for RAPS. In Appendix D.3, we provide the same
analysis for ResNet18,50,152 at different coverage levels along with analyzing
the performance in the Confidence and Atypicality dimensions individually. For
instance in Figure 8, we observe that RAPS and APS do not satisfy coverage for
high atypicality regions, even when averaged across different confidence
levels.

##  6 Additional Related Work

Uncertainty and Atypicality:  [46, 53] use density estimation to disentangle
epistemic and aleatoric uncertainty. Following this, they show improvements in
active learning and OOD detection [36]. We note that our goal is not this
disentanglement (e.g.  Untrustworthy quadrant can have both aleatoric or
epistemic uncertainty), or  Ambiguity could be due to a lack of features or
noise. [37] propose the related notion of distance awareness, and that it
leads to better uncertainty quantification. They offer architecture and
training modifications whereas we analyze existing models using our framework
including imbalanced and LLM settings, and propose simple and post-hoc
approaches. ‘OOD’ [25] or ‘anomaly’ [27] notions are tied to atypicality, yet
our goal is not to make a binary distinction between ‘in’ or ‘out’. We argue
that in-distribution samples could also be atypical (e.g. rare groups), and
the goal is to perform reliably in the entire spectrum. Other works with an
atypicality notion include bounding calibration of groups by the excess risk
[41], miscalibration under distribution shifts [51], uncertainty in Gaussian
Processes [56], forgetting time for rare examples [44], the poor performance
of groups with lower sample sizes [11], energy-based models improving
calibration [22], relating perplexity to zero-shot classification performance
for LLMs [19], grouping loss and local definitions of miscalibration [55], the
relationship between active learning and atypicality [23], sample size as a
factor for subgroup performance disparity [11]. [54] provide insightful
discussion around the nature of softmax confidence, and here we show that its
reliability depends on the atypicality of the input. Our new findings include
showing that predictions for atypical samples are more miscalibrated and
overconfident, and atypicality awareness improves prediction quality.
_Overall, while there are other relevant notions in the literature, our
distinct goal is to show that post-hoc atypicality estimation and
recalibration is a simple yet useful framework to understand and improve
uncertainty quantification that complements existing methods._

Recalibration and Conformal Prediction: There is a rich literature on
recalibration methods and prediction sets: TS [20], Platt Scaling [52],
conformal calibration [63, 2] among many. [35, 57, 10, 4] make a relevant
observation, showing that the coverage of conformal prediction is not equal
across all groups. They propose group conformal calibration, which requires
group labels whereas our proposal is unsupervised and does not depend on any
attribute information. Concurrent work [30] explores AdaTS, where they train a
separate VAE and MLP to produce an adaptive temperature. However, our
parameterization of temperature has 3 parameters and is interpretable.

##  7 Conclusion

Atypicality offers a simple yet flexible framework to better understand and
improve model reliability and uncertainty. We propose that pretrained models
should be released not only with confidence but also with an atypicality
estimator. While there are other relevant notions in the literature, our main
goal is to show that atypicality can provide a unifying perspective to discuss
uncertainty, understand individual data points, and improve fairness. Here we
focus on classification problems; it would be interesting to extend
atypicality to regression and generation settings. Furthermore, we would like
to extend the theoretical analysis to more general settings, as our empirical
results demonstrate that the observed phenomena hold more broadly.

## Acknowledgments

We would like to thank Adarsh Jeewajee, Bryan He, Edward Chen, Federico
Bianchi, Kyle Swanson, Natalie Dullerud, Ransalu Senanayake, Sabri Eyuboglu,
Shirley Wu, Weixin Liang, Xuechen Li, Yongchan Kwon, Yu Sun, and Zach Izzo for
their comments and suggestions on the manuscript. Linjun Zhang’s research is
partially supported by NSF DMS-2015378. Carlos Ernesto Guestrin is a Chan
Zuckerberg Biohub – San Francisco Investigator.

## References

  * AB [21] Anastasios N Angelopoulos and Stephen Bates.  A gentle introduction to conformal prediction and distribution-free uncertainty quantification.  arXiv preprint arXiv:2107.07511, 2021.
  * ABMJ [20] Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael I Jordan.  Uncertainty sets for image classifiers using conformal prediction.  arXiv preprint arXiv:2009.14193, 2020.
  * AR [89] János Aczél and Fred S Roberts.  On the possible merging functions.  Mathematical Social Sciences, 17(3):205–243, 1989.
  * BGJ+ [22] Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth.  Practical adversarial multivalid conformal prediction.  arXiv preprint arXiv:2206.01067, 2022.
  * BHN [17] Solon Barocas, Moritz Hardt, and Arvind Narayanan.  Fairness in machine learning.  Nips tutorial, 1:2, 2017.
  * BMR+ [20] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [7] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong.  Don’t just blame over-parametrization for over-confidence: Theoretical analysis of calibration in binary classification.  In International Conference on Machine Learning, pages 566–576. PMLR, 2021.
  * [8] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong.  Understanding the under-coverage bias in uncertainty estimation.  Advances in Neural Information Processing Systems, 34:18307–18319, 2021.
  * BW [19] David Bolin and Jonas Wallin.  Local scale invariance and robustness of proper scoring rules.  arXiv preprint arXiv:1912.05642, 2019.
  * BYR+ [21] Noam Barda, Gal Yona, Guy N Rothblum, Philip Greenland, Morton Leibowitz, Ran Balicer, Eitan Bachmat, and Noa Dagan.  Addressing bias in prediction models by improving subpopulation calibration.  Journal of the American Medical Informatics Association, 28(3):549–558, 2021.
  * CJS [18] Irene Chen, Fredrik D Johansson, and David Sontag.  Why is my classifier discriminatory?  Advances in neural information processing systems, 31, 2018.
  * CLKZ [22] Lucas Clarté, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová.  Theoretical characterization of uncertainty in high-dimensional linear classification.  arXiv preprint arXiv:2202.03295, 2022.
  * CWG+ [19] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma.  Learning imbalanced datasets with label-distribution-aware margin loss.  Advances in neural information processing systems, 32, 2019.
  * DCLT [19] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  ArXiv, abs/1810.04805, 2019.
  * DDS+ [09] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * DVN+ [22] Roxana Daneshjou, Kailas Vodrahalli, Roberto A Novoa, Melissa Jenkins, Weixin Liang, Veronica Rotemberg, Justin Ko, Susan M Swetter, Elizabeth E Bailey, Olivier Gevaert, et al.  Disparities in dermatology ai performance on a diverse, curated clinical image set.  Science advances, 8(31):eabq6147, 2022.
  * FP [98] Ernest Forman and Kirti Peniwati.  Aggregating individual judgments and priorities with the analytic hierarchy process.  European journal of operational research, 108(1):165–169, 1998.
  * GHS+ [21] Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri.  Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1820–1828, 2021.
  * GIB+ [22] Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer.  Demystifying prompts in language models via perplexity estimation.  arXiv preprint arXiv:2212.04037, 2022.
  * GPSW [17] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  In International conference on machine learning, pages 1321–1330. PMLR, 2017.
  * GR [07] Tilmann Gneiting and Adrian E Raftery.  Strictly proper scoring rules, prediction, and estimation.  Journal of the American statistical Association, 102(477):359–378, 2007.
  * GWJ+ [20] Will Grathwohl, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky.  Your classifier is secretly an energy based model and you should treat it like one.  In International Conference on Learning Representations, 2020.
  * HDW [22] Guy Hacohen, Avihu Dekel, and Daphna Weinshall.  Active learning on a budget: Opposite strategies suit high and low budgets.  arXiv preprint arXiv:2202.02794, 2022.
  * [24] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  arXiv preprint arXiv:1610.02136, 2016.
  * [25] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  ArXiv, abs/1610.02136, 2016.
  * HJKRR [18] Ursula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum.  Multicalibration: Calibration for the (computationally-identifiable) masses.  In International Conference on Machine Learning, pages 1939–1948. PMLR, 2018.
  * HMD [18] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich.  Deep anomaly detection with outlier exposure.  arXiv preprint arXiv:1812.04606, 2018.
  * HZRS [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * JKGG [18] Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta.  To trust or not to trust a classifier.  Advances in neural information processing systems, 31, 2018.
  * JPL+ [23] Tom Joy, Francesco Pinto, Ser-Nam Lim, Philip HS Torr, and Puneet K Dokania.  Sample-dependent adaptive temperature scaling for improved calibration.  AAAI, 2023.
  * KGZ [19] Michael P Kim, Amirata Ghorbani, and James Zou.  Multiaccuracy: Black-box post-processing for fairness in classification.  In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 247–254, 2019.
  * Kri [09] Alex Krizhevsky.  Learning multiple layers of features from tiny images.  2009\.
  * Kum [22] Sawan Kumar.  Answer-level calibration for free-form multiple choice question answering.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 665–679, 2022.
  * KW [13] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * LLC+ [22] Charles Lu, Andréanne Lemay, Ken Chang, Katharina Höbel, and Jayashree Kalpathy-Cramer.  Fair conformal predictors for applications in medical imaging.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12008–12016, 2022.
  * LLLS [18] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  Advances in neural information processing systems, 31, 2018.
  * LLP+ [20] Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshminarayanan.  Simple and principled uncertainty estimation with deterministic deep learning via distance awareness.  Advances in Neural Information Processing Systems, 33:7498–7512, 2020.
  * LN [89] Dong C Liu and Jorge Nocedal.  On the limited memory bfgs method for large scale optimization.  Mathematical programming, 45(1):503–528, 1989.
  * LOG+ [19] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * LR [02] Xin Li and Dan Roth.  Learning question classifiers.  In COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
  * LSH [19] Lydia T Liu, Max Simchowitz, and Moritz Hardt.  The implicit fairness criterion of unconstrained learning.  In International Conference on Machine Learning, pages 4051–4060. PMLR, 2019.
  * LVdMJ+ [21] Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf.  Datasets: A community library for natural language processing.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, 2021.
  * MDP+ [11] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts.  Learning word vectors for sentiment analysis.  In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
  * MGLK [22] Pratyush Maini, Saurabh Garg, Zachary C Lipton, and J Zico Kolter.  Characterizing datapoints via second-split forgetting.  arXiv preprint arXiv:2210.15031, 2022.
  * MKS+ [20] Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip HS Torr, and Puneet K Dokania.  Calibrating deep neural networks using focal loss.  2020\.
  * MKvA+ [21] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal.  Deep deterministic uncertainty: A simple baseline.  arXiv e-prints, pages arXiv–2102, 2021.
  * MP [80] Carolyn B Mervis and John R Pani.  Acquisition of basic object categories.  Cognitive Psychology, 12(4):496–522, 1980.
  * MR [10] Sébastien Marcel and Yann Rodriguez.  Torchvision the machine-vision package of torch.  In Proceedings of the 18th ACM international conference on Multimedia, pages 1485–1488, 2010.
  * Mur [04] Gregory Murphy.  The big book of concepts.  MIT press, 2004.
  * NCH [15] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht.  Obtaining well calibrated probabilities using bayesian binning.  In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
  * OFR+ [19] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek.  Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift.  Advances in neural information processing systems, 32, 2019.
  * P+ [99] John Platt et al.  Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods.  Advances in large margin classifiers, 10(3):61–74, 1999.
  * PBC+ [20] Janis Postels, Hermann Blum, Cesar Cadena, Roland Siegwart, Luc Van Gool, and Federico Tombari.  Quantifying aleatoric and epistemic uncertainty using density estimation in latent space.  arXiv preprint arXiv:2012.03082, 2020.
  * PBZ [21] Tim Pearce, Alexandra Brintrup, and Jun Zhu.  Understanding softmax confidence and uncertainty.  arXiv preprint arXiv:2106.04972, 2021.
  * PLMV [23] Alexandre Perez-Lebel, Marine Le Morvan, and Gael Varoquaux.  Beyond calibration: estimating the grouping loss of modern neural networks.  In The Eleventh International Conference on Learning Representations, 2023.
  * Ras [04] Carl Edward Rasmussen.  Gaussian processes in machine learning.  In Summer school on machine learning, pages 63–71. Springer, 2004.
  * RBSC [19] Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel J Candès.  With malice towards none: Assessing uncertainty via equalized coverage.  arXiv preprint arXiv:1908.05428, 2019.
  * Rip [89] Lance J . Rips.  Similarity, typicality, and categorization, page 21–59.  Cambridge University Press, 1989.
  * RM [75] Eleanor Rosch and Carolyn B Mervis.  Family resemblances: Studies in the internal structure of categories.  Cognitive psychology, 7(4):573–605, 1975.
  * RSC [20] Yaniv Romano, Matteo Sesia, and Emmanuel Candes.  Classification with valid and adaptive coverage.  Advances in Neural Information Processing Systems, 33:3581–3591, 2020.
  * RSS [73] Lance J Rips, Edward J Shoben, and Edward E Smith.  Semantic distance and the verification of semantic relations.  Journal of verbal learning and verbal behavior, 12(1):1–20, 1973.
  * SC [19] Pragya Sur and Emmanuel J Candès.  A modern maximum-likelihood theory for high-dimensional logistic regression.  Proceedings of the National Academy of Sciences, 116(29):14516–14525, 2019.
  * SV [08] Glenn Shafer and Vladimir Vovk.  A tutorial on conformal prediction.  Journal of Machine Learning Research, 9(3), 2008.
  * TGZ+ [23] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca, 2023.
  * TJ [06] MTCAJ Thomas and A Thomas Joy.  Elements of information theory.  Wiley-Interscience, 2006.
  * TK [74] Amos Tversky and Daniel Kahneman.  Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty.  science, 185(4157):1124–1131, 1974.
  * TK [92] Amos Tversky and Daniel Kahneman.  Advances in prospect theory: Cumulative representation of uncertainty.  Journal of Risk and uncertainty, 5(4):297–323, 1992.
  * VGS [05] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer.  Algorithmic learning in a random world.  Springer Science & Business Media, 2005.
  * WDS+ [20] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Transformers: State-of-the-art natural language processing.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, 2020.
  * WNB [18] Adina Williams, Nikita Nangia, and Samuel Bowman.  A broad-coverage challenge corpus for sentence understanding through inference.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 2018.
  * ZCLJ [21] Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia.  Improving calibration for long-tailed recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16489–16498, 2021.
  * ZDKZ [22] Linjun Zhang, Zhun Deng, Kenji Kawaguchi, and James Zou.  When and how mixup improves calibration.  In International Conference on Machine Learning, pages 26135–26160. PMLR, 2022.
  * ZK [16] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  arXiv preprint arXiv:1605.07146, 2016.
  * ZKL+ [16] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, and Aude Oliva.  Places: An image database for deep scene understanding.  arXiv preprint arXiv:1610.02055, 2016.
  * ZWF+ [21] Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh.  Calibrate before use: Improving few-shot performance of language models.  In International Conference on Machine Learning, pages 12697–12706. PMLR, 2021.
  * ZZL [15] Xiang Zhang, Junbo Jake Zhao, and Yann LeCun.  Character-level convolutional networks for text classification.  In NIPS, 2015.

##  Appendix A Atypicality Estimation

###  A.1 Input Atypicality Estimation

To estimate input atypicality, we use two ways to estimate the likelihood of a
point under the training distribution. First, we give methods for the
discriminative models.

##### Fitting individual Gaussians to class conditionals

Here, we follow a similar approach to [46]. Namely, we model the class
conditionals with a Gaussian distribution, where the covariance matrix is tied
across classes:

$\hat{\mathbb{P}}(X|Y=y)\sim N(X;\mu_{y},\Sigma)$ (4)

We fit the parameters $\mu_{y}$ and $\Sigma$ with maximum likelihood
estimation. The reason to tie the covariance matrix is due to the number of
samples required to fit the density. Namely, for a $d$-dimensional problem,
the total number of parameters to fit individual matrices becomes $O(yd^{2})$,
which results in low-quality estimates. Then, the atypicality becomes

$a_{X}(x)=-\max_{y\in\mathcal{Y}}\log\hat{\mathbb{P}}(X=x|Y=y)$ (5)

##### Computing distance with k-Nearest Neighbors

k-Nearest Neighbors:  Similarly, we can use the nearest neighbor distance.
Concretely, we use the nearest neighbor distance,
$a_{X}(x)=d_{\min}(x,\mathcal{D}_{\textup{train}})=\min_{x^{\prime}\in\mathcal{D}_{\textup{train}}}|x^{\prime}-x|$
, as the atypicality metric. Alternatively, we can use different notions such
as the average of k-nearest neighbors, or the distance to the kth neighbor.
Below, we report the results by using the average distance to 5-nearest
neighbors.

##### Fitting the estimators

For all of the atypicality estimators, we fit the estimators using samples
from the training sets and make inferences for the calibration and test sets.
For instance, we use the training split of ImageNet to fit the corresponding
density estimator and compute the atypicality for the samples from the
validation/test split of ImageNet. All of our results using atypicality are
reported for the test splits of the below datasets.

##### Atypicality Estimation with LLMs:

For language models we simply compute the negative log-likelihood of each
prompt as the atypicality metric:
$a_{X}(x)=-\log\hat{\mathbb{P}}_{\textrm{LLM}}(x)$. To define confidence, we
use the logits of the language model, conditioned on the prompt. We use the
logit of the first token of each class label and compute the predicted
probabilities by applying softmax to the logits of each class.

###  A.2 Class Atypicality

To estimate class atypicality, we simply count the fraction of examples from a
particular label in the training dataset. Let us have a training dataset
$\mathcal{D}_{\textrm{train}}=\\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots,(x_{N},y_{N})\\}$.
Then, we estimate class atypicality with

$a_{Y}(y)=-\log\frac{\sum_{i\in[N]}\mathbf{1}[y_{i}=y]}{N}.$ (6)

###  A.3 Atypicality and Confidence

Figure 6: Input Atypicality and Confidence. Here, the x-axis reflects the
input atypicality quantile, and the y-axis indicates confidence. The coloring
for the figure on the left indicates the accuracy within a bin, and the figure
on the right has the difference between confidence and accuracy within a bin.
We observe that even within the same confidence range, atypical examples tend
to be more miscalibrated and overconfident compared to typical examples.

Are atypicality and confidence equally informative? Beyond the data
perspective given in Figure 1, here we provide quantitative results to
demonstrate the difference. In Figure 6, we give a grid plot where the x-axis
indicates the typicality quantile of a point, and the y-axis indicates the
confidence of a point. The coloring on the left indicates the accuracy within
a bin split according to accuracy, and the right has the difference between
average confidence and accuracy. Observe that for a specific confidence
interval, larger values of typicality mean better quality probabilistic
estimates and larger atypicality means more miscalibration.

##  Appendix B Experimental Details

###  B.1 Balanced Supervised Classification

####  B.1.1 Datasets

Below is a full list of datasets for balanced classification:

  1. 1.

ImageNet [15] from Torchvision [48] is an object recognition dataset with 1000
classes. We use the ImageNet-1k version.

  2. 2.

CIFAR10/100 [32] from Torchvision [48] are object recognition datasets with
10/100 classes.

  3. 3.

MNLI [70] from Huggingface Datasets [42] is a natural language inference
dataset with 3 classes, indicating entailment, neutral, and contradiction
outcomes.

####  B.1.2 Models

Most of the models are public models, e.g., obtained from the Transformers
Library [69] or Torchvision [48]. Below we give the full model details and how
one can access them:

  1. 1.

RoBERTa(HuggingFace roberta-large-mnli) trained on the MNLI dataset. One can
use the id given here on HuggingFace to download the model.

  2. 2.

ResNet18, ResNet50, ResNet152 from (Torchvision [48]) trained on ImageNet.

  3. 3.

WideResNet28 trained on CIFAR10,100 obtained from [45].

For all BERT [14] style models we use the [CLS] token embeddings in the final
layer to perform classification. For all vision models, we use the penultimate
layer embeddings to fit the density estimators and perform the analyses. In
the experiments, we randomly split the test sets into two equal halves to have
a calibration split and a test split, and repeat the experiments over 10
random seeds.

###  B.2 Imbalanced Supervised Classification

####  B.2.1 Datasets

All of our imbalanced classification datasets are previous benchmarks obtained
from the GitHub repository666https://github.com/dvlab-research/MiSLAS of [71]
with corresponding training, validation, and test splits. All of these
datasets have an exponential class imbalance.

  1. 1.

ImageNet-LT is the long-tailed variant of ImageNet with 1000 classes.

  2. 2.

CIFAR10/100-LT is the long-tailed variant of CIFAR10/100 with 10/100 classes.

  3. 3.

Places365-LT is the long-tailed variant of Places365 [74] with 365 classes.

####  B.2.2 Models

Similarly, most of these models are obtained from [71].

  1. 1.

ResNeXt50 trained on ImageNet-LT with and without Progressive Balancing, which
is a strategy to address class imbalance during training.

  2. 2.

ResNet152 trained on Places365-LT

  3. 3.

ResNet18 trained on CIFAR100-LT trained by us. This model is pretrained on
ImageNet and finetuned on CIFAR100-LT.

We use the validation splits of these datasets as the calibration set, and
report the results on the test set.

###  B.3 Classification with LLMs

####  B.3.1 Model

We use Alpaca-7B [64] in a zero-shot setting, where we simply prompt the model
with the classification question. We use the prompting strategy from Content-
Free Calibration [75].

Below, we show examples of each dataset and prompt.

####  B.3.2 Datasets

IMDB is a binary classification dataset of movie reviews, where the goal is to
classify the sentiment in a review. The example prompt has the form ‘The
following review was written for a movie: [Review].\n What is the sentiment,
Positive or Negative? Answer: ’. Below is an example:

The following review was written for a movie: I and a friend rented this
movie. We both found the movie soundtrack and production techniques to be
lagging. The movie’s plot appeared to drag on throughout with little surprise
in the ending. We both agreed that the movie could have been compressed into
roughly an hour giving it more suspense and moving plot.
What is the sentiment, Positive or Negative? Answer:

where the correct answer should be ‘Negative’. We filter out the examples that
exceed the context length limit of Alpaca7B ($512$). We noticed that the
‘validation’ split of IMDB leads to significantly worse calibration compared
to splitting the test set. Thus, for all experiments, we use the test split of
IMDB and split it into 2 sets (instead of using the validation split as a
calibration set as in the other two datasets).

TREC is a 6-class question classification dataset where the goal is to predict
whether a question will have an answer that is an ‘Abbreviation’, ‘Entity’,
‘Description’, ‘Human’, ‘Location’, or a ‘Number’. We format the prompts with
‘Classify the questions based on their Answer Type. Potential Answer Types
are: Number, Location, Person, Description, Entity, or
Abbreviation.\n\nQuestion: [question]\n\nAnswer Type: ’. Below is an example
prompt:

Classify the questions based on their Answer Type. Potential Answer Types are:
Number, Location, Person, Description, Entity, or Abbreviation.
Question: What county is Modesto , California in ? Answer Type:

where the correct answer should be ‘Location’.

AG News is a news classification dataset. The goal is to classify a given news
into 4 potential classes: ‘World’, ‘Sports’, ‘Business’, or ‘Science and
Technology’. We format the prompts with ‘Classify the news articles into the
categories of World, Sports, Business, and Technology.\n\n Article:
[article]\n\nAnswer: ’. Below is an example prompt:

Classify the news articles into the categories of World, Sports, Business, and
Technology.
Article: Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-
sellers, Wall Street’s dwindling band of ultra-cynics, are seeing green again.
Answer:

where the correct answer should be ‘Business’.

Furthermore, we also use [75] as another calibration baseline. Following their
paper, we use N/A, [MASK], and the empty string as content-free. Concretely,
we follow their paper to first obtain the average predicted probabilities for
each label token for the content-free input, denoted by $p_{cf}$. We then let

$W=\textrm{diag}(p_{cf})^{-1}.$

When making test-time predictions, we compute $\textrm{Softmax}(W^{T}p)$ as
the new predicted probabilities. In our experiments, we observe that it does
not perform as well in this setting, as was previously suggested by [33].

##  Appendix C Calibration

We run all our experiments with 10 different random seeds, where the seeds are
$\\{0,1,2,\ldots,9\\}$. Randomness is over fitting the atypicality estimators,
and calibration-test splits (we use the same splits with the recalibration
experiments for the sake of consistency).

###  C.1 Expected Calibration Error

To compute ECE, we generate $\mathbb{B}=\\{B_{1},B_{2},...,B_{M}\\}$, $M$
equally-spaced bins where samples are sorted and grouped according to their
confidence. $B_{m}$ here denotes the set of the data indices where the
confidence of the prediction for the sample falls into the interval
$(\frac{m-1}{M},\frac{m}{M}]$. We compute ECEby

$\textrm{ECE}[\hat{\mathbb{P}}]=\sum_{m=1}^{M}\frac{|B_{m}|}{N}\lvert\textup{acc}(B_{m})-\textup{conf}(B_{m})\rvert,$
(7)

where
$\textup{acc}(B_{m})=\frac{1}{|B_{m}|}\sum_{i=1}^{|B_{m}|}\mathbf{1}[\hat{y}_{i}=y_{i}]$
is the accuracy for the bin $m$, and
$\textup{conf}(B_{m})=\frac{1}{|B_{m}|}\sum_{i=1}^{|B_{m}|}\hat{\mathbb{P}}(Y=\hat{y}_{i}|X=x_{i})$
gives the average confidence within the bin. $|B_{m}|$ is the size of the bin
$m$, $N$ is the total number of samples, and $\mathbf{1}[\cdot]$ is the
indicator function.

Throughout our experiments, we let the number of bins $|\mathbb{B}|=10$ by
default when computing ECE.

Similarly, below we report results with RMSCE (Root Mean Squared Error) [27]
as another calibration metric, which is formulated as the following:

$\textrm{RMSCE}[\hat{\mathbb{P}}]=\sqrt{\sum_{m=1}^{M}\frac{|B_{m}|}{N}(\textup{acc}(B_{m})-\textup{conf}(B_{m}))^{2}}$
(8)

###  C.2 ECE and Atypicality Results with Different Atypicality Metrics

We further experiment with different atypicality metrics, such as the average
distance to the 5-nearest neighbors (Figure 7). We broadly observe that while
there are slight differences in the quantitative results between different
atypicality metrics, the qualitative phenomena remain intact. In Tables 1, 2,
and 3 we give all the results in the tabular form.

Figure 7: Atypicality with 5-nearest neighbors and Uncertainty. Here, we
report the results of the same experiments as Figure 3 with the average of the
distance to the 10-nearest neighbors as the atypicality metric. See Tables 1
for the results in tabular format.

###  C.3 Results in the Tabular Format

Table 1: Recalibration Results for Balanced Classification. For each dataset
and atypicality quantile, the best results are marked in bold. We provide the
standard errors next to the means over 10 random seeds.

Uncalibrated TS AdaTS Atypicality-Aware Atypicality ECE RMSE ECE RMSE ECE RMSE
ECE RMSE GMM ResNet152 ImageNet 0.2 $0.029\pm 0.001$ $0.077\pm 0.001$
$0.014\pm 0.001$ $0.066\pm 0.002$ $0.014\pm 0.001$ $0.064\pm 0.002$ $0.016\pm
0.001$ $0.065\pm 0.001$ 0.4 $0.039\pm 0.001$ $0.086\pm 0.001$ $0.020\pm 0.001$
$0.074\pm 0.001$ $0.020\pm 0.002$ $0.075\pm 0.002$ $0.019\pm 0.001$ $0.070\pm
0.001$ 0.6 $0.050\pm 0.001$ $0.097\pm 0.001$ $0.026\pm 0.001$ $0.079\pm 0.002$
$0.025\pm 0.002$ $0.081\pm 0.002$ $0.026\pm 0.000$ $0.078\pm 0.001$ 0.8
$0.062\pm 0.001$ $0.106\pm 0.001$ $0.024\pm 0.001$ $0.076\pm 0.001$ $0.023\pm
0.002$ $0.078\pm 0.002$ $0.024\pm 0.001$ $0.077\pm 0.001$ 1.0 $0.084\pm 0.001$
$0.123\pm 0.001$ $0.037\pm 0.001$ $0.088\pm 0.001$ $0.033\pm 0.004$ $0.085\pm
0.003$ $0.026\pm 0.001$ $0.080\pm 0.001$ ResNet18 ImageNet 0.2 $0.017\pm
0.001$ $0.074\pm 0.001$ $0.031\pm 0.001$ $0.084\pm 0.001$ $0.028\pm 0.004$
$0.081\pm 0.004$ $0.016\pm 0.001$ $0.069\pm 0.001$ 0.4 $0.016\pm 0.001$
$0.074\pm 0.001$ $0.020\pm 0.001$ $0.079\pm 0.001$ $0.023\pm 0.003$ $0.079\pm
0.003$ $0.015\pm 0.001$ $0.072\pm 0.002$ 0.6 $0.025\pm 0.001$ $0.080\pm 0.001$
$0.023\pm 0.001$ $0.077\pm 0.001$ $0.024\pm 0.003$ $0.080\pm 0.002$ $0.019\pm
0.001$ $0.074\pm 0.001$ 0.8 $0.040\pm 0.001$ $0.092\pm 0.001$ $0.024\pm 0.001$
$0.078\pm 0.001$ $0.029\pm 0.002$ $0.082\pm 0.003$ $0.018\pm 0.001$ $0.074\pm
0.001$ 1.0 $0.054\pm 0.001$ $0.100\pm 0.001$ $0.033\pm 0.001$ $0.085\pm 0.001$
$0.032\pm 0.003$ $0.083\pm 0.003$ $0.017\pm 0.002$ $0.073\pm 0.002$ ResNet50
ImageNet 0.2 $0.021\pm 0.001$ $0.069\pm 0.002$ $0.018\pm 0.001$ $0.071\pm
0.001$ $0.017\pm 0.001$ $0.068\pm 0.002$ $0.018\pm 0.001$ $0.067\pm 0.002$ 0.4
$0.029\pm 0.001$ $0.079\pm 0.001$ $0.023\pm 0.001$ $0.077\pm 0.001$ $0.020\pm
0.002$ $0.074\pm 0.001$ $0.022\pm 0.001$ $0.074\pm 0.001$ 0.6 $0.041\pm 0.001$
$0.091\pm 0.001$ $0.023\pm 0.001$ $0.078\pm 0.001$ $0.026\pm 0.002$ $0.079\pm
0.002$ $0.026\pm 0.001$ $0.078\pm 0.001$ 0.8 $0.042\pm 0.001$ $0.092\pm 0.001$
$0.024\pm 0.001$ $0.078\pm 0.001$ $0.024\pm 0.001$ $0.080\pm 0.002$ $0.020\pm
0.001$ $0.075\pm 0.001$ 1.0 $0.078\pm 0.000$ $0.118\pm 0.000$ $0.049\pm 0.001$
$0.095\pm 0.001$ $0.047\pm 0.004$ $0.096\pm 0.003$ $0.031\pm 0.001$ $0.082\pm
0.001$ RoBERTa MNLI 0.2 $0.005\pm 0.001$ $0.062\pm 0.004$ $0.013\pm 0.001$
$0.077\pm 0.002$ $0.006\pm 0.002$ $0.063\pm 0.005$ $0.006\pm 0.001$ $0.063\pm
0.002$ 0.4 $0.016\pm 0.001$ $0.091\pm 0.004$ $0.013\pm 0.001$ $0.083\pm 0.002$
$0.010\pm 0.002$ $0.075\pm 0.005$ $0.008\pm 0.001$ $0.072\pm 0.002$ 0.6
$0.034\pm 0.002$ $0.116\pm 0.002$ $0.015\pm 0.001$ $0.092\pm 0.004$ $0.016\pm
0.001$ $0.093\pm 0.005$ $0.017\pm 0.002$ $0.090\pm 0.004$ 0.8 $0.061\pm 0.002$
$0.153\pm 0.003$ $0.031\pm 0.002$ $0.113\pm 0.004$ $0.024\pm 0.002$ $0.104\pm
0.004$ $0.028\pm 0.002$ $0.114\pm 0.004$ 1.0 $0.065\pm 0.002$ $0.156\pm 0.002$
$0.028\pm 0.002$ $0.112\pm 0.003$ $0.021\pm 0.002$ $0.106\pm 0.004$ $0.023\pm
0.003$ $0.107\pm 0.004$ WideResNet28 CIFAR10 0.2 $0.003\pm 0.000$ $0.041\pm
0.001$ $0.009\pm 0.000$ $0.057\pm 0.000$ $0.003\pm 0.000$ $0.051\pm 0.002$
$0.001\pm 0.000$ $0.041\pm 0.001$ 0.4 $0.005\pm 0.001$ $0.051\pm 0.002$
$0.007\pm 0.001$ $0.055\pm 0.001$ $0.002\pm 0.001$ $0.047\pm 0.002$ $0.002\pm
0.000$ $0.048\pm 0.002$ 0.6 $0.004\pm 0.001$ $0.050\pm 0.003$ $0.008\pm 0.001$
$0.058\pm 0.001$ $0.004\pm 0.001$ $0.049\pm 0.004$ $0.002\pm 0.000$ $0.044\pm
0.002$ 0.8 $0.009\pm 0.001$ $0.066\pm 0.003$ $0.004\pm 0.001$ $0.056\pm 0.002$
$0.003\pm 0.001$ $0.058\pm 0.003$ $0.002\pm 0.001$ $0.055\pm 0.002$ 1.0
$0.142\pm 0.002$ $0.240\pm 0.002$ $0.073\pm 0.002$ $0.168\pm 0.003$ $0.046\pm
0.002$ $0.129\pm 0.004$ $0.041\pm 0.002$ $0.123\pm 0.002$ WideResNet28
CIFAR100 0.2 $0.007\pm 0.001$ $0.059\pm 0.003$ $0.018\pm 0.001$ $0.084\pm
0.001$ $0.021\pm 0.003$ $0.089\pm 0.005$ $0.021\pm 0.001$ $0.094\pm 0.002$ 0.4
$0.029\pm 0.001$ $0.110\pm 0.002$ $0.005\pm 0.001$ $0.070\pm 0.003$ $0.008\pm
0.002$ $0.073\pm 0.003$ $0.005\pm 0.001$ $0.078\pm 0.003$ 0.6 $0.101\pm 0.002$
$0.201\pm 0.002$ $0.052\pm 0.001$ $0.136\pm 0.002$ $0.044\pm 0.004$ $0.124\pm
0.007$ $0.049\pm 0.001$ $0.132\pm 0.003$ 0.8 $0.257\pm 0.004$ $0.297\pm 0.002$
$0.106\pm 0.003$ $0.191\pm 0.003$ $0.101\pm 0.003$ $0.187\pm 0.002$ $0.105\pm
0.003$ $0.190\pm 0.003$ 1.0 $0.371\pm 0.004$ $0.351\pm 0.002$ $0.105\pm 0.004$
$0.200\pm 0.003$ $0.101\pm 0.004$ $0.199\pm 0.003$ $0.109\pm 0.005$ $0.207\pm
0.005$ KNN ResNet152 ImageNet 0.2 $0.035\pm 0.001$ $0.085\pm 0.001$ $0.014\pm
0.001$ $0.062\pm 0.002$ $0.016\pm 0.002$ $0.068\pm 0.001$ $0.011\pm 0.001$
$0.061\pm 0.001$ 0.4 $0.039\pm 0.001$ $0.085\pm 0.001$ $0.015\pm 0.001$
$0.067\pm 0.001$ $0.018\pm 0.001$ $0.069\pm 0.002$ $0.017\pm 0.001$ $0.071\pm
0.001$ 0.6 $0.035\pm 0.001$ $0.083\pm 0.001$ $0.021\pm 0.001$ $0.076\pm 0.002$
$0.024\pm 0.003$ $0.079\pm 0.004$ $0.019\pm 0.001$ $0.072\pm 0.001$ 0.8
$0.057\pm 0.002$ $0.102\pm 0.001$ $0.031\pm 0.001$ $0.084\pm 0.001$ $0.033\pm
0.003$ $0.085\pm 0.003$ $0.032\pm 0.001$ $0.082\pm 0.001$ 1.0 $0.099\pm 0.001$
$0.129\pm 0.001$ $0.036\pm 0.001$ $0.090\pm 0.001$ $0.042\pm 0.006$ $0.093\pm
0.004$ $0.037\pm 0.001$ $0.091\pm 0.001$ ResNet18 ImageNet 0.2 $0.025\pm
0.001$ $0.077\pm 0.002$ $0.020\pm 0.001$ $0.074\pm 0.001$ $0.037\pm 0.021$
$0.115\pm 0.042$ $0.019\pm 0.001$ $0.072\pm 0.001$ 0.4 $0.017\pm 0.001$
$0.072\pm 0.001$ $0.021\pm 0.001$ $0.076\pm 0.001$ $0.045\pm 0.025$ $0.120\pm
0.046$ $0.019\pm 0.001$ $0.074\pm 0.001$ 0.6 $0.021\pm 0.001$ $0.074\pm 0.001$
$0.027\pm 0.001$ $0.083\pm 0.001$ $0.052\pm 0.029$ $0.130\pm 0.050$ $0.020\pm
0.001$ $0.074\pm 0.002$ 0.8 $0.033\pm 0.002$ $0.085\pm 0.001$ $0.022\pm 0.001$
$0.079\pm 0.001$ $0.061\pm 0.038$ $0.136\pm 0.058$ $0.021\pm 0.001$ $0.077\pm
0.001$ 1.0 $0.054\pm 0.001$ $0.102\pm 0.001$ $0.030\pm 0.001$ $0.084\pm 0.001$
$0.079\pm 0.045$ $0.152\pm 0.064$ $0.027\pm 0.001$ $0.081\pm 0.002$ ResNet50
ImageNet 0.2 $0.029\pm 0.001$ $0.078\pm 0.001$ $0.016\pm 0.001$ $0.066\pm
0.001$ $0.015\pm 0.002$ $0.068\pm 0.002$ $0.016\pm 0.001$ $0.067\pm 0.001$ 0.4
$0.029\pm 0.001$ $0.078\pm 0.001$ $0.018\pm 0.001$ $0.072\pm 0.001$ $0.019\pm
0.001$ $0.074\pm 0.002$ $0.017\pm 0.001$ $0.070\pm 0.002$ 0.6 $0.029\pm 0.001$
$0.078\pm 0.001$ $0.021\pm 0.001$ $0.077\pm 0.001$ $0.020\pm 0.002$ $0.077\pm
0.002$ $0.020\pm 0.001$ $0.075\pm 0.001$ 0.8 $0.046\pm 0.001$ $0.096\pm 0.001$
$0.028\pm 0.001$ $0.079\pm 0.001$ $0.026\pm 0.001$ $0.079\pm 0.001$ $0.031\pm
0.001$ $0.082\pm 0.001$ 1.0 $0.076\pm 0.001$ $0.116\pm 0.001$ $0.039\pm 0.002$
$0.092\pm 0.001$ $0.040\pm 0.004$ $0.091\pm 0.003$ $0.038\pm 0.002$ $0.090\pm
0.001$ RoBERTa MNLI 0.2 $0.003\pm 0.000$ $0.056\pm 0.003$ $0.011\pm 0.001$
$0.079\pm 0.002$ $0.011\pm 0.002$ $0.067\pm 0.005$ $0.003\pm 0.001$ $0.061\pm
0.003$ 0.4 $0.014\pm 0.001$ $0.080\pm 0.003$ $0.015\pm 0.002$ $0.091\pm 0.003$
$0.016\pm 0.004$ $0.092\pm 0.007$ $0.008\pm 0.001$ $0.074\pm 0.002$ 0.6
$0.029\pm 0.002$ $0.111\pm 0.003$ $0.014\pm 0.001$ $0.077\pm 0.002$ $0.023\pm
0.005$ $0.095\pm 0.008$ $0.015\pm 0.001$ $0.081\pm 0.005$ 0.8 $0.067\pm 0.002$
$0.156\pm 0.003$ $0.034\pm 0.002$ $0.108\pm 0.004$ $0.030\pm 0.003$ $0.107\pm
0.004$ $0.026\pm 0.003$ $0.101\pm 0.005$ 1.0 $0.064\pm 0.002$ $0.153\pm 0.003$
$0.027\pm 0.002$ $0.110\pm 0.003$ $0.033\pm 0.007$ $0.119\pm 0.009$ $0.026\pm
0.003$ $0.107\pm 0.005$ WideResNet28 CIFAR10 0.2 $0.001\pm 0.000$ $0.031\pm
0.003$ $0.011\pm 0.000$ $0.061\pm 0.001$ $0.005\pm 0.001$ $0.049\pm 0.004$
$0.003\pm 0.000$ $0.037\pm 0.001$ 0.4 $0.000\pm 0.000$ $0.010\pm 0.005$
$0.012\pm 0.000$ $0.062\pm 0.001$ $0.007\pm 0.001$ $0.054\pm 0.004$ $0.004\pm
0.000$ $0.037\pm 0.001$ 0.6 $0.005\pm 0.000$ $0.049\pm 0.002$ $0.007\pm 0.001$
$0.055\pm 0.002$ $0.003\pm 0.001$ $0.046\pm 0.004$ $0.001\pm 0.000$ $0.048\pm
0.002$ 0.8 $0.009\pm 0.001$ $0.061\pm 0.001$ $0.004\pm 0.001$ $0.058\pm 0.001$
$0.004\pm 0.001$ $0.062\pm 0.003$ $0.003\pm 0.000$ $0.052\pm 0.002$ 1.0
$0.148\pm 0.002$ $0.241\pm 0.002$ $0.079\pm 0.003$ $0.170\pm 0.003$ $0.053\pm
0.004$ $0.136\pm 0.005$ $0.039\pm 0.003$ $0.121\pm 0.003$ WideResNet28
CIFAR100 0.2 $0.006\pm 0.000$ $0.054\pm 0.002$ $0.020\pm 0.001$ $0.089\pm
0.001$ $0.021\pm 0.002$ $0.089\pm 0.003$ $0.014\pm 0.001$ $0.083\pm 0.001$ 0.4
$0.022\pm 0.001$ $0.092\pm 0.002$ $0.004\pm 0.001$ $0.070\pm 0.002$ $0.006\pm
0.001$ $0.067\pm 0.004$ $0.005\pm 0.001$ $0.076\pm 0.002$ 0.6 $0.090\pm 0.001$
$0.183\pm 0.002$ $0.054\pm 0.001$ $0.140\pm 0.002$ $0.049\pm 0.003$ $0.132\pm
0.004$ $0.041\pm 0.002$ $0.120\pm 0.002$ 0.8 $0.266\pm 0.005$ $0.295\pm 0.003$
$0.124\pm 0.004$ $0.205\pm 0.003$ $0.118\pm 0.004$ $0.202\pm 0.003$ $0.091\pm
0.003$ $0.185\pm 0.003$ 1.0 $0.380\pm 0.003$ $0.356\pm 0.002$ $0.086\pm 0.003$
$0.187\pm 0.004$ $0.087\pm 0.004$ $0.185\pm 0.005$ $0.117\pm 0.005$ $0.216\pm
0.004$

Table 2: Recalibration Results for Imbalanced Classification. For each dataset
and atypicality quantile, the best results are marked in bold. We provide the
standard errors next to the means over 10 random seeds.

Uncalibrated TS AdaTS Atypicality-Aware Atypicality ECE RMSE Accuracy ECE RMSE
Accuracy ECE RMSE Accuracy ECE RMSE Accuracy GMM ResNet152 Places365-LT 0.2
$0.159\pm 0.000$ $0.144\pm 0.000$ $0.492\pm 0.000$ $0.064\pm 0.000$ $0.099\pm
0.000$ $0.492\pm 0.000$ $0.044\pm 0.004$ $0.088\pm 0.002$ $0.492\pm 0.000$
$0.065\pm 0.000$ $0.101\pm 0.000$ $0.412\pm 0.000$ 0.4 $0.208\pm 0.000$
$0.166\pm 0.000$ $0.387\pm 0.000$ $0.061\pm 0.000$ $0.095\pm 0.000$ $0.387\pm
0.000$ $0.053\pm 0.001$ $0.099\pm 0.001$ $0.387\pm 0.000$ $0.042\pm 0.000$
$0.090\pm 0.000$ $0.415\pm 0.000$ 0.6 $0.270\pm 0.000$ $0.186\pm 0.000$
$0.317\pm 0.000$ $0.073\pm 0.000$ $0.111\pm 0.000$ $0.317\pm 0.000$ $0.076\pm
0.005$ $0.111\pm 0.003$ $0.317\pm 0.000$ $0.054\pm 0.000$ $0.097\pm 0.000$
$0.398\pm 0.000$ 0.8 $0.334\pm 0.000$ $0.214\pm 0.000$ $0.218\pm 0.000$
$0.120\pm 0.000$ $0.150\pm 0.000$ $0.218\pm 0.000$ $0.127\pm 0.006$ $0.152\pm
0.002$ $0.218\pm 0.000$ $0.058\pm 0.000$ $0.103\pm 0.000$ $0.371\pm 0.000$ 1.0
$0.437\pm 0.000$ $0.243\pm 0.000$ $0.081\pm 0.000$ $0.211\pm 0.000$ $0.185\pm
0.000$ $0.081\pm 0.000$ $0.212\pm 0.007$ $0.187\pm 0.002$ $0.081\pm 0.000$
$0.103\pm 0.000$ $0.137\pm 0.000$ $0.283\pm 0.000$ ResNet18 CIFAR10-LT 0.2
$0.017\pm 0.000$ $0.077\pm 0.000$ $0.927\pm 0.000$ $0.060\pm 0.000$ $0.164\pm
0.000$ $0.927\pm 0.000$ $0.084\pm 0.008$ $0.190\pm 0.008$ $0.927\pm 0.000$
$0.030\pm 0.000$ $0.104\pm 0.000$ $0.874\pm 0.000$ 0.4 $0.054\pm 0.000$
$0.145\pm 0.000$ $0.825\pm 0.000$ $0.082\pm 0.000$ $0.182\pm 0.000$ $0.825\pm
0.000$ $0.102\pm 0.008$ $0.201\pm 0.006$ $0.825\pm 0.000$ $0.027\pm 0.000$
$0.104\pm 0.000$ $0.741\pm 0.000$ 0.6 $0.152\pm 0.000$ $0.236\pm 0.000$
$0.672\pm 0.000$ $0.049\pm 0.000$ $0.133\pm 0.000$ $0.672\pm 0.000$ $0.069\pm
0.005$ $0.174\pm 0.007$ $0.672\pm 0.000$ $0.044\pm 0.000$ $0.127\pm 0.000$
$0.770\pm 0.000$ 0.8 $0.098\pm 0.000$ $0.188\pm 0.000$ $0.779\pm 0.000$
$0.022\pm 0.000$ $0.107\pm 0.000$ $0.779\pm 0.000$ $0.032\pm 0.005$ $0.113\pm
0.007$ $0.779\pm 0.000$ $0.034\pm 0.000$ $0.104\pm 0.000$ $0.810\pm 0.000$ 1.0
$0.244\pm 0.000$ $0.286\pm 0.000$ $0.584\pm 0.000$ $0.128\pm 0.000$ $0.206\pm
0.000$ $0.584\pm 0.000$ $0.157\pm 0.011$ $0.248\pm 0.009$ $0.584\pm 0.000$
$0.034\pm 0.000$ $0.109\pm 0.000$ $0.829\pm 0.000$ CIFAR100-LT 0.2 $0.138\pm
0.000$ $0.231\pm 0.000$ $0.660\pm 0.000$ $0.106\pm 0.000$ $0.203\pm 0.000$
$0.660\pm 0.000$ $0.105\pm 0.012$ $0.198\pm 0.010$ $0.660\pm 0.000$ $0.031\pm
0.000$ $0.112\pm 0.000$ $0.594\pm 0.000$ 0.4 $0.235\pm 0.000$ $0.291\pm 0.000$
$0.523\pm 0.000$ $0.056\pm 0.000$ $0.137\pm 0.000$ $0.523\pm 0.000$ $0.058\pm
0.006$ $0.145\pm 0.007$ $0.523\pm 0.000$ $0.040\pm 0.000$ $0.109\pm 0.000$
$0.556\pm 0.000$ 0.6 $0.295\pm 0.000$ $0.320\pm 0.000$ $0.431\pm 0.000$
$0.063\pm 0.000$ $0.166\pm 0.000$ $0.431\pm 0.000$ $0.067\pm 0.009$ $0.160\pm
0.010$ $0.431\pm 0.000$ $0.043\pm 0.000$ $0.142\pm 0.000$ $0.508\pm 0.000$ 0.8
$0.381\pm 0.000$ $0.360\pm 0.000$ $0.344\pm 0.000$ $0.121\pm 0.000$ $0.221\pm
0.000$ $0.344\pm 0.000$ $0.100\pm 0.016$ $0.193\pm 0.013$ $0.344\pm 0.000$
$0.080\pm 0.000$ $0.179\pm 0.000$ $0.432\pm 0.000$ 1.0 $0.403\pm 0.000$
$0.382\pm 0.000$ $0.269\pm 0.000$ $0.116\pm 0.000$ $0.234\pm 0.000$ $0.269\pm
0.000$ $0.129\pm 0.015$ $0.233\pm 0.010$ $0.269\pm 0.000$ $0.051\pm 0.000$
$0.165\pm 0.000$ $0.423\pm 0.000$ ResNext50 ImageNet-LT 0.2 $0.019\pm 0.000$
$0.061\pm 0.000$ $0.716\pm 0.000$ $0.073\pm 0.000$ $0.097\pm 0.000$ $0.716\pm
0.000$ $0.069\pm 0.006$ $0.095\pm 0.003$ $0.716\pm 0.000$ $0.013\pm 0.000$
$0.062\pm 0.000$ $0.627\pm 0.000$ 0.4 $0.032\pm 0.000$ $0.075\pm 0.000$
$0.592\pm 0.000$ $0.059\pm 0.000$ $0.089\pm 0.000$ $0.592\pm 0.000$ $0.056\pm
0.006$ $0.087\pm 0.003$ $0.592\pm 0.000$ $0.022\pm 0.000$ $0.068\pm 0.000$
$0.576\pm 0.000$ 0.6 $0.081\pm 0.000$ $0.103\pm 0.000$ $0.481\pm 0.000$
$0.045\pm 0.000$ $0.079\pm 0.000$ $0.481\pm 0.000$ $0.046\pm 0.002$ $0.081\pm
0.001$ $0.481\pm 0.000$ $0.020\pm 0.000$ $0.063\pm 0.000$ $0.525\pm 0.000$ 0.8
$0.171\pm 0.000$ $0.144\pm 0.000$ $0.309\pm 0.000$ $0.079\pm 0.000$ $0.115\pm
0.000$ $0.309\pm 0.000$ $0.086\pm 0.007$ $0.116\pm 0.002$ $0.309\pm 0.000$
$0.032\pm 0.000$ $0.080\pm 0.000$ $0.446\pm 0.000$ 1.0 $0.324\pm 0.000$
$0.198\pm 0.000$ $0.113\pm 0.000$ $0.230\pm 0.000$ $0.174\pm 0.000$ $0.113\pm
0.000$ $0.235\pm 0.008$ $0.175\pm 0.002$ $0.113\pm 0.000$ $0.082\pm 0.000$
$0.113\pm 0.000$ $0.318\pm 0.000$ ResNext50(+P.B.) ImageNet-LT 0.2 $0.017\pm
0.000$ $0.064\pm 0.000$ $0.653\pm 0.000$ $0.081\pm 0.000$ $0.100\pm 0.000$
$0.653\pm 0.000$ $0.084\pm 0.004$ $0.101\pm 0.002$ $0.653\pm 0.000$ $0.020\pm
0.000$ $0.070\pm 0.000$ $0.579\pm 0.000$ 0.4 $0.023\pm 0.000$ $0.068\pm 0.000$
$0.579\pm 0.000$ $0.062\pm 0.000$ $0.090\pm 0.000$ $0.579\pm 0.000$ $0.063\pm
0.004$ $0.089\pm 0.002$ $0.579\pm 0.000$ $0.013\pm 0.000$ $0.064\pm 0.000$
$0.535\pm 0.000$ 0.6 $0.065\pm 0.000$ $0.090\pm 0.000$ $0.508\pm 0.000$
$0.027\pm 0.000$ $0.074\pm 0.000$ $0.508\pm 0.000$ $0.027\pm 0.003$ $0.070\pm
0.002$ $0.508\pm 0.000$ $0.018\pm 0.000$ $0.067\pm 0.000$ $0.508\pm 0.000$ 0.8
$0.129\pm 0.000$ $0.121\pm 0.000$ $0.388\pm 0.000$ $0.042\pm 0.000$ $0.082\pm
0.000$ $0.388\pm 0.000$ $0.043\pm 0.003$ $0.083\pm 0.002$ $0.388\pm 0.000$
$0.039\pm 0.000$ $0.076\pm 0.000$ $0.444\pm 0.000$ 1.0 $0.236\pm 0.000$
$0.164\pm 0.000$ $0.224\pm 0.000$ $0.147\pm 0.000$ $0.134\pm 0.000$ $0.224\pm
0.000$ $0.147\pm 0.004$ $0.133\pm 0.002$ $0.224\pm 0.000$ $0.078\pm 0.000$
$0.100\pm 0.000$ $0.359\pm 0.000$ KNN ResNet152 Places365-LT 0.2 $0.159\pm
0.000$ $0.144\pm 0.000$ $0.492\pm 0.000$ $0.064\pm 0.000$ $0.099\pm 0.000$
$0.492\pm 0.000$ $0.047\pm 0.004$ $0.089\pm 0.002$ $0.492\pm 0.000$ $0.091\pm
0.000$ $0.116\pm 0.000$ $0.387\pm 0.000$ 0.4 $0.208\pm 0.000$ $0.166\pm 0.000$
$0.387\pm 0.000$ $0.061\pm 0.000$ $0.095\pm 0.000$ $0.387\pm 0.000$ $0.054\pm
0.001$ $0.097\pm 0.001$ $0.387\pm 0.000$ $0.048\pm 0.000$ $0.090\pm 0.000$
$0.448\pm 0.000$ 0.6 $0.270\pm 0.000$ $0.186\pm 0.000$ $0.317\pm 0.000$
$0.073\pm 0.000$ $0.111\pm 0.000$ $0.317\pm 0.000$ $0.068\pm 0.004$ $0.107\pm
0.002$ $0.317\pm 0.000$ $0.057\pm 0.000$ $0.098\pm 0.000$ $0.416\pm 0.000$ 0.8
$0.334\pm 0.000$ $0.214\pm 0.000$ $0.218\pm 0.000$ $0.120\pm 0.000$ $0.150\pm
0.000$ $0.218\pm 0.000$ $0.117\pm 0.004$ $0.149\pm 0.002$ $0.218\pm 0.000$
$0.065\pm 0.000$ $0.106\pm 0.000$ $0.367\pm 0.000$ 1.0 $0.437\pm 0.000$
$0.243\pm 0.000$ $0.081\pm 0.000$ $0.211\pm 0.000$ $0.185\pm 0.000$ $0.081\pm
0.000$ $0.202\pm 0.004$ $0.184\pm 0.001$ $0.081\pm 0.000$ $0.111\pm 0.000$
$0.152\pm 0.000$ $0.207\pm 0.000$ ResNet18 CIFAR10-LT 0.2 $0.017\pm 0.000$
$0.077\pm 0.000$ $0.927\pm 0.000$ $0.060\pm 0.000$ $0.164\pm 0.000$ $0.927\pm
0.000$ $0.098\pm 0.009$ $0.204\pm 0.008$ $0.927\pm 0.000$ $0.025\pm 0.000$
$0.085\pm 0.000$ $0.873\pm 0.000$ 0.4 $0.054\pm 0.000$ $0.145\pm 0.000$
$0.825\pm 0.000$ $0.082\pm 0.000$ $0.182\pm 0.000$ $0.825\pm 0.000$ $0.127\pm
0.008$ $0.220\pm 0.005$ $0.825\pm 0.000$ $0.024\pm 0.000$ $0.106\pm 0.000$
$0.742\pm 0.000$ 0.6 $0.152\pm 0.000$ $0.236\pm 0.000$ $0.672\pm 0.000$
$0.049\pm 0.000$ $0.133\pm 0.000$ $0.672\pm 0.000$ $0.054\pm 0.005$ $0.157\pm
0.005$ $0.672\pm 0.000$ $0.039\pm 0.000$ $0.124\pm 0.000$ $0.768\pm 0.000$ 0.8
$0.098\pm 0.000$ $0.188\pm 0.000$ $0.779\pm 0.000$ $0.022\pm 0.000$ $0.107\pm
0.000$ $0.779\pm 0.000$ $0.028\pm 0.002$ $0.106\pm 0.005$ $0.779\pm 0.000$
$0.029\pm 0.000$ $0.102\pm 0.000$ $0.810\pm 0.000$ 1.0 $0.244\pm 0.000$
$0.286\pm 0.000$ $0.584\pm 0.000$ $0.128\pm 0.000$ $0.206\pm 0.000$ $0.584\pm
0.000$ $0.153\pm 0.009$ $0.248\pm 0.007$ $0.584\pm 0.000$ $0.037\pm 0.000$
$0.123\pm 0.000$ $0.832\pm 0.000$ CIFAR100-LT 0.2 $0.138\pm 0.000$ $0.231\pm
0.000$ $0.660\pm 0.000$ $0.106\pm 0.000$ $0.203\pm 0.000$ $0.660\pm 0.000$
$0.088\pm 0.008$ $0.184\pm 0.008$ $0.660\pm 0.000$ $0.029\pm 0.000$ $0.108\pm
0.000$ $0.595\pm 0.000$ 0.4 $0.235\pm 0.000$ $0.291\pm 0.000$ $0.523\pm 0.000$
$0.056\pm 0.000$ $0.137\pm 0.000$ $0.523\pm 0.000$ $0.049\pm 0.003$ $0.135\pm
0.004$ $0.523\pm 0.000$ $0.049\pm 0.000$ $0.123\pm 0.000$ $0.554\pm 0.000$ 0.6
$0.295\pm 0.000$ $0.320\pm 0.000$ $0.431\pm 0.000$ $0.063\pm 0.000$ $0.166\pm
0.000$ $0.431\pm 0.000$ $0.073\pm 0.008$ $0.171\pm 0.008$ $0.431\pm 0.000$
$0.030\pm 0.000$ $0.113\pm 0.000$ $0.509\pm 0.000$ 0.8 $0.381\pm 0.000$
$0.360\pm 0.000$ $0.344\pm 0.000$ $0.121\pm 0.000$ $0.221\pm 0.000$ $0.344\pm
0.000$ $0.119\pm 0.012$ $0.210\pm 0.009$ $0.344\pm 0.000$ $0.078\pm 0.000$
$0.175\pm 0.000$ $0.435\pm 0.000$ 1.0 $0.403\pm 0.000$ $0.382\pm 0.000$
$0.269\pm 0.000$ $0.116\pm 0.000$ $0.234\pm 0.000$ $0.269\pm 0.000$ $0.145\pm
0.011$ $0.244\pm 0.008$ $0.269\pm 0.000$ $0.059\pm 0.000$ $0.163\pm 0.000$
$0.424\pm 0.000$ ResNext50 ImageNet-LT 0.2 $0.019\pm 0.000$ $0.061\pm 0.000$
$0.716\pm 0.000$ $0.073\pm 0.000$ $0.097\pm 0.000$ $0.716\pm 0.000$ $0.067\pm
0.004$ $0.094\pm 0.002$ $0.716\pm 0.000$ $0.014\pm 0.000$ $0.065\pm 0.000$
$0.626\pm 0.000$ 0.4 $0.032\pm 0.000$ $0.075\pm 0.000$ $0.592\pm 0.000$
$0.059\pm 0.000$ $0.089\pm 0.000$ $0.592\pm 0.000$ $0.052\pm 0.004$ $0.086\pm
0.002$ $0.592\pm 0.000$ $0.023\pm 0.000$ $0.068\pm 0.000$ $0.575\pm 0.000$ 0.6
$0.081\pm 0.000$ $0.103\pm 0.000$ $0.481\pm 0.000$ $0.045\pm 0.000$ $0.079\pm
0.000$ $0.481\pm 0.000$ $0.042\pm 0.001$ $0.080\pm 0.001$ $0.481\pm 0.000$
$0.023\pm 0.000$ $0.068\pm 0.000$ $0.524\pm 0.000$ 0.8 $0.171\pm 0.000$
$0.144\pm 0.000$ $0.309\pm 0.000$ $0.079\pm 0.000$ $0.115\pm 0.000$ $0.309\pm
0.000$ $0.088\pm 0.005$ $0.116\pm 0.002$ $0.309\pm 0.000$ $0.037\pm 0.000$
$0.079\pm 0.000$ $0.447\pm 0.000$ 1.0 $0.324\pm 0.000$ $0.198\pm 0.000$
$0.113\pm 0.000$ $0.230\pm 0.000$ $0.174\pm 0.000$ $0.113\pm 0.000$ $0.238\pm
0.005$ $0.176\pm 0.001$ $0.113\pm 0.000$ $0.086\pm 0.000$ $0.116\pm 0.000$
$0.318\pm 0.000$ ResNext50(+P.B.) ImageNet-LT 0.2 $0.017\pm 0.000$ $0.064\pm
0.000$ $0.653\pm 0.000$ $0.081\pm 0.000$ $0.100\pm 0.000$ $0.653\pm 0.000$
$0.083\pm 0.005$ $0.100\pm 0.003$ $0.653\pm 0.000$ $0.018\pm 0.000$ $0.069\pm
0.000$ $0.578\pm 0.000$ 0.4 $0.023\pm 0.000$ $0.068\pm 0.000$ $0.579\pm 0.000$
$0.062\pm 0.000$ $0.090\pm 0.000$ $0.579\pm 0.000$ $0.062\pm 0.005$ $0.089\pm
0.003$ $0.579\pm 0.000$ $0.015\pm 0.000$ $0.062\pm 0.000$ $0.535\pm 0.000$ 0.6
$0.065\pm 0.000$ $0.090\pm 0.000$ $0.508\pm 0.000$ $0.027\pm 0.000$ $0.074\pm
0.000$ $0.508\pm 0.000$ $0.028\pm 0.003$ $0.073\pm 0.001$ $0.508\pm 0.000$
$0.024\pm 0.000$ $0.069\pm 0.000$ $0.507\pm 0.000$ 0.8 $0.129\pm 0.000$
$0.121\pm 0.000$ $0.388\pm 0.000$ $0.042\pm 0.000$ $0.082\pm 0.000$ $0.388\pm
0.000$ $0.043\pm 0.005$ $0.083\pm 0.002$ $0.388\pm 0.000$ $0.038\pm 0.000$
$0.080\pm 0.000$ $0.444\pm 0.000$ 1.0 $0.236\pm 0.000$ $0.164\pm 0.000$
$0.224\pm 0.000$ $0.147\pm 0.000$ $0.134\pm 0.000$ $0.224\pm 0.000$ $0.148\pm
0.005$ $0.134\pm 0.002$ $0.224\pm 0.000$ $0.076\pm 0.000$ $0.101\pm 0.000$
$0.358\pm 0.000$

Table 3: Recalibration Results for LLM Classification. For each dataset and
atypicality quantile, the best results are marked in bold. We provide the
standard errors next to the means over 10 random seeds.

Uncalibrated Content-Free Atypicality-Aware ECE RMSE Accuracy ECE RMSE
Accuracy ECE RMSE Accuracy LLM Alpaca7B AG News 0.25 $0.180\pm 0.000$
$0.219\pm 0.000$ $0.681\pm 0.000$ $0.452\pm 0.000$ $0.411\pm 0.000$ $0.527\pm
0.000$ $0.070\pm 0.000$ $0.142\pm 0.000$ $0.760\pm 0.000$ 0.50 $0.165\pm
0.000$ $0.202\pm 0.000$ $0.671\pm 0.000$ $0.413\pm 0.000$ $0.370\pm 0.000$
$0.571\pm 0.000$ $0.027\pm 0.000$ $0.101\pm 0.000$ $0.775\pm 0.000$ 0.75
$0.169\pm 0.000$ $0.204\pm 0.000$ $0.657\pm 0.000$ $0.429\pm 0.000$ $0.364\pm
0.000$ $0.549\pm 0.000$ $0.030\pm 0.000$ $0.099\pm 0.000$ $0.752\pm 0.000$
1.00 $0.202\pm 0.000$ $0.222\pm 0.000$ $0.580\pm 0.000$ $0.448\pm 0.000$
$0.350\pm 0.000$ $0.524\pm 0.000$ $0.028\pm 0.000$ $0.110\pm 0.000$ $0.715\pm
0.000$ IMDB 0.25 $0.023\pm 0.001$ $0.087\pm 0.002$ $0.887\pm 0.001$ $0.141\pm
0.001$ $0.194\pm 0.001$ $0.883\pm 0.001$ $0.011\pm 0.001$ $0.068\pm 0.002$
$0.927\pm 0.001$ 0.50 $0.024\pm 0.001$ $0.095\pm 0.002$ $0.851\pm 0.001$
$0.117\pm 0.002$ $0.185\pm 0.002$ $0.838\pm 0.002$ $0.014\pm 0.001$ $0.078\pm
0.002$ $0.920\pm 0.001$ 0.75 $0.051\pm 0.002$ $0.124\pm 0.002$ $0.795\pm
0.002$ $0.110\pm 0.001$ $0.181\pm 0.002$ $0.823\pm 0.002$ $0.009\pm 0.001$
$0.069\pm 0.001$ $0.895\pm 0.001$ 1.00 $0.063\pm 0.001$ $0.136\pm 0.001$
$0.756\pm 0.001$ $0.122\pm 0.001$ $0.201\pm 0.001$ $0.752\pm 0.001$ $0.021\pm
0.001$ $0.088\pm 0.002$ $0.883\pm 0.001$ TREC 0.25 $0.139\pm 0.000$ $0.355\pm
0.000$ $0.744\pm 0.000$ $0.286\pm 0.000$ $0.265\pm 0.000$ $0.776\pm 0.000$
$0.109\pm 0.000$ $0.266\pm 0.000$ $0.896\pm 0.000$ 0.50 $0.217\pm 0.000$
$0.444\pm 0.000$ $0.680\pm 0.000$ $0.314\pm 0.000$ $0.503\pm 0.000$ $0.672\pm
0.000$ $0.068\pm 0.000$ $0.148\pm 0.000$ $0.816\pm 0.000$ 0.75 $0.238\pm
0.000$ $0.482\pm 0.000$ $0.592\pm 0.000$ $0.464\pm 0.000$ $0.622\pm 0.000$
$0.520\pm 0.000$ $0.067\pm 0.000$ $0.198\pm 0.000$ $0.784\pm 0.000$ 1.00
$0.207\pm 0.000$ $0.451\pm 0.000$ $0.544\pm 0.000$ $0.528\pm 0.000$ $0.685\pm
0.000$ $0.432\pm 0.000$ $0.150\pm 0.000$ $0.199\pm 0.000$ $0.696\pm 0.000$

##  Appendix D Recalibration

Through all our recalibration results, we first split the test set into two
equally sized calibration and test splits. Then, we fit the recalibration
method using the calibration split and compute the performance on the test
split. We run all our experiments with 10 different random seeds.

###  D.1 Temperature Scaling

To perform temperature scaling [20], we use the calibration set to fit the
temperature parameter. To perform the optimization, we use the LBFGS [38]
algorithm from PyTorch with strong Wolfe line search, following [20]. Namely,
we optimize the parameter $\tau$ with

$\hat{\mathbb{P}}_{\textup{TS}}(X)=\textup{Softmax}(f({\bm{X}})/\tau)$ (9)

and then use it during inference to rescale the logits produced by $f$. We use
$0.1$ learning rate and $3000$ maximum iterations across all experiments and
initialize the temperature value as $1$, although find that TS is pretty
robust to the choice of hyperparameters.

###  D.2 Atypicality-Aware Recalibration

Here we describe the implementation details For Atypicality-Aware
Recalibration (AAR). We formulate AAR with:

$\log{\hat{\mathbb{P}}_{\textrm{AAR}}(Y|X)}\propto{\phi(a(X))}\log{\hat{\mathbb{P}}(Y|X)}+S_{Y},$
(10)

In total, this gives us $|\mathcal{Y}|+3$ parameters. Using exactly the same
setting as TS, we use LBFGS with a strong Wolfe search to optimize the three
parameters, with the same splits as temperature scaling. We normalize the
atypicality values (subtract the mean and divide by the standard deviation of
the calibration set) for numerical stability. We use the same hyperparameters
as TS (with $0.1$ learning rate and $3000$ maximum iterations) without any
modification across all experiments, initialize $c_{0},c_{1},c_{2}$ as $0$ and
$S_{Y}$ parameters as $1$. We run the recalibration procedure on a CPU with
precomputed logits.

####  D.2.1 Adaptive Temperature Scaling

For AdaTS [30] we use the implementation provided with the paper
777https://github.com/thwjoy/adats. We identically use the hyperparameters and
the architecture provided in the paper and their repository. They use an
encoder and decoder architecture with $[1024,512,512]$ hidden units each, and
a temperature predictor network with $[128,128]$ hidden units. They use an
Adam Optimizer with a learning rate of $5e-4$ with $128$ batch size.

###  D.3 Conformal Prediction

We follow the presentation in [1, 2]. Let $\pi(X)$ be the permutation of
$\mathcal{Y}=\\{1,\ldots,C\\}$ that sorts $\hat{\mathbb{P}}(Y=c|X)$, i.e. the
predicted probabilities for each class $c$. We define a score function

$s(x,y)=\sum_{j=1}^{c}\hat{\mathbb{P}}(Y=j|X),\textup{where }y=\pi_{c}.$ (11)

This means greedily including classes until the set contains the true label,
and using the cumulative sum of the probabilities as the score function. We
compute all of the scores for the calibration set,
$S_{\textrm{calib}}=\\{s(x_{1},y_{1}),...,s(x_{N},y_{N})\\}$, we the
$\frac{\lceil(N+1)(1-\alpha)\rceil}{N}$th quantile of the scores, $\hat{q}$.
Then, the prediction set is defined as

$\mathcal{C}(x)=\\{y:s(x,y)\leq\hat{q}\\}$ (12)

We can further add randomization to the procedure where we have the prediction
set function to be $\mathcal{C}(x,u):\mathcal{X}\times[0,1]$ for randomization
purposes to satisfy exact coverage. We refer to [68, 2, 1] for a more thorough
presentation.

RAPS is a variant of APS that regularizes the set sizes. They modify the
scoring function to add a regularization term. This is controlled by the test
size offset $k_{reg}$ that controls the value beyond which the regularization
is applied, and the $\lambda_{reg}$ gives the strength of the regularization.
To fit the $k_{reg},\lambda_{reg}$ parameters in RAPS, we follow the procedure
in [2] to fit both parameters. Namely, we fit $k_{reg}$ by Algorithm 4 in
their paper that leverages the set sizes in the calibration set, and we fit
$\lambda_{reg}$ by the largest regularization parameter that achieves the
smallest set sizes, searched over a grid of $\\{0.001,0.01,0.1,0.2,0.5\\}$
following their presentation.

###  D.4 Atypicality-Aware Conformal Prediction

We have a simple discrete grouping scheme to make conformal prediction
atypicality aware. Namely, we group points using their atypicality and
confidence percentiles and fit individual thresholds. Concretely, we construct
a dataset of
$\mathcal{D}_{AA}=\\{(c_{i},c_{i+1}],(a_{j},a_{j+1}],\hat{q}_{i,j}\\}_{i,j\in[N]}$
using the calibration set where $(c_{i},c_{i+1}]$ denotes the confidence range
for quantile $i$, $(a_{j},a_{j+1}]$ denotes the atypicality range for quantile
$j$ and $\hat{q}_{i,j}$ denotes the threshold fitted to the group specified by
these intervals. We let $N=6$ as the number of groups, and in total, we end up
with $36$ thresholds. At test time, we check the quantile of the confidence
and atypicality of a point and use the corresponding temperature. For AA-RAPS,
we use the same $k_{reg}$ and $\lambda_{reg}$ values as was found with the
RAPS procedure. For practical purposes, we do not allow zero sets (prediction
sets at least include the top prediction).

We would like to make the remark that sometimes the marginal coverage can
exceed the desired value (e.g. Figure 9). This is often because the underlying
model is already very confident for a majority of data points (e.g. More than
half of the data points have 92% confidence). The gains we provide are often
for points with lower confidence regions, as the coverage is not satisfied in
those regions.

Figure 8: Atypicality-Aware Conformal Prediction for ResNet18 and ImageNet.
Target coverage rate is $95\%$. Figure 9: Atypicality-Aware Conformal
Prediction for ResNet152 and ImageNet. Target coverage rate is $95\%$. Figure
10: Fitted Temperature vs Atypicality. We observe a monotonically increasing
relationship between the atypicality of a group and the temperature parameter
fitted to that group with TS.  Figure 11: Fitted Additive Correction Factor vs
Class Atypicality. We observe a monotonically increasing relationship between
the atypicality of a class and the additive correction parameter fitted to
that class with AAR.  Figure 12: Atypicality-Aware Conformal Prediction for
ResNet50 and ImageNet. Target coverage rate is $95\%$.

##  Appendix E Tables for Results

Here, we present the table version of the results in Figure 3. Tables 1,2
contain the ECE analysis.

##  Appendix F Fitzpatrick17k and Skin Lesion Classification

We use the training script from [18] to finetune models on the Fitzpatrick17k
dataset. We train the models for $50$ epochs, fixing the backbone and training
only the probe on top of the penultimate layer. The probe consists of $2$
layers, one layer of $256$ units followed by ReLU and Dropout with probability
$0.4$, followed by the classifier layer with an output dimensionality of $9$.
We use an Adam optimizer with a $0.0001$ learning rate.

The entire dataset consists of $16,577$ images, where the potential labels
are: $10,886$ inflammatory, $1,352$ malignant epidermal, $1,194$
genodermatoses, $1,067$ benign dermal, $931$ benign epidermal, $573$ malignant
melanoma, $236$ benign melanocyte, $182$ malignant cutaneous lymphoma, and
$156$ malignant dermal. We split the dataset into 3 sets (Training ($0.5$),
Validation ($0.25$), and Test ($0.25$)). We use the validation set as the
calibration set and perform the experiments with 10 random splits.

##  Appendix G Proofs

###  G.1 Detailed derivation of the claim on Page 5

When $\mathbb{P}_{1}(X)\leq 1/2$, the signed calibration error at level
$u\in(1/2,1)$ becomes
$u-\mathbb{P}(Y=-1\mid\hat{\mathbb{P}}_{-1}(X)=u)=u-\mathbb{P}(Y=-1\mid\hat{\mathbb{P}}_{1}(-X)=u)=u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)$.

The last inequality is due to symmetry. More specifically, we claim
$(X,Y)\stackrel{{\scriptstyle d}}{{=}}(-X,-Y)$, where the notation
$\stackrel{{\scriptstyle d}}{{=}}$ denotes equal in distribution. In fact, as
$X\stackrel{{\scriptstyle d}}{{=}}-X$, it suffices to show that for any
$y\in\\{-1,1\\}$, and $x\in\mathbb{R}^{d}$, we have

$\mathbb{P}(Y=y\mid X=x)=\mathbb{P}(-Y=y\mid-X=x).$

When $y=-1$, the right hand side

$\displaystyle\mathbb{P}(-Y=-1\mid-X=x)=\mathbb{P}(Y=1\mid
X=-x)=\sigma(\langle\beta^{*},-x\rangle)$ $\displaystyle=$ $\displaystyle
1-\sigma(\langle\beta^{*},x\rangle)=1-\mathbb{P}(Y=1\mid
X=x)=\mathbb{P}(Y=-1\mid X=x).$

Similarly, when $y=1$,

$\displaystyle\mathbb{P}(-Y=1\mid-X=x)=\mathbb{P}(Y=-1\mid
X=-x)=1-\sigma(\langle\beta^{*},-x\rangle)$ $\displaystyle=$
$\displaystyle\sigma(\langle\beta^{*},x\rangle)=\mathbb{P}(Y=1\mid X=x).$

We complete the proof.

###  G.2 Proof of Theorem 3.1

######  Theorem G.1 (Restatement of Theorem 3.1).

Consider the data generative model with the algorithm described in Section
3.2. For any $K>1$, suppose we consider the quantiles of $a(X)$,
$a_{1},a_{2},...,a_{K},a_{K+1}$ such that
$\mathbb{P}(a(X)\in(a_{k},a_{k+1}])=1/K$ for $k\in[K]$. In addition, we assume
$\|\beta^{*}\|\leq c_{0}$, and $d/n=\kappa$ for some sufficiently small
$c_{0},\kappa>0$. Then for sufficiently large $n$, we have

$\displaystyle\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in[a_{k-1},a_{k}]]>$
$\displaystyle\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in(a_{k},a_{k+1}]],$

for $k=2,..,K$.

###### Proof.

Following [7], we have

$u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)=u-\mathbb{E}_{Z}[\sigma(\frac{\|\beta^{*}\|}{\|\hat{\beta}\|}\cos\hat{\theta}\cdot\sigma^{-1}(u))+\sin\hat{\theta}\cdot\|\beta^{*}\|Z],$

where
$\cos\hat{\theta}=\frac{\hat{\beta}^{\top}\beta^{*}}{\|\hat{\beta}\|\cdot\|\beta^{*}\|}$
and $Z\sim N(0,1)$.

According to the results in Section 2.2 of [62], we have $\|\hat{\beta}\|\to
R^{*}=R^{*}(\kappa,\beta^{*})$ and $\cos\hat{\theta}\to
c^{*}=c^{*}(\kappa,\beta^{*})$, for two quantities $R^{*}$ and $c^{*}$ that
depend on $\kappa$ and $\beta^{*}$. We then have

$u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\to
u-\mathbb{E}_{Z}[\sigma(\frac{\|\beta^{*}\|}{R^{*}}c^{*}\cdot\sigma^{-1}(u))+\sqrt{1-c^{*2}}\cdot\|\beta^{*}\|Z].$

Using the proof of Theorem 3 in [7], we have that

$u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)=C_{\kappa}(u)\cdot\kappa+o(\kappa),$

where

$C_{\kappa}(u)=c_{1}\sigma^{\prime}(\sigma^{-1}(u))\cdot\sigma^{-1}(u)-c_{2}\sigma^{\prime\prime}(\sigma^{-1}(u)),$

for two positive constants $c_{1},c_{2}$.

As a result, we have

$\displaystyle u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\geq 0$ (13)

In addition, since when $z\in[-1,1]$, $z\cdot\sigma^{\prime}(z)$ and
$-\sigma^{\prime\prime}(z)$ are both increasing, we then have $C_{\kappa}(u)$
increasing for $\hat{\beta}^{\top}x=\sigma^{-1}(u)\in(-1,1)$.

##### Proving the result for $\\{k=2,\ldots,K-1\\}$

In addition, by our model assumption $x\sim N(0,I_{d})$, we have that $\|x\|$
and $\frac{x}{\|x\|}$ are independent, and $\frac{x}{\|x\|}\sim S$ where $S$
is a uniform distribution on the sphere in the $d$-dimensional space. As the
monotonic transformations will not change the events defined by quantiles, and
$\exp(-\|x\|^{2}/2)$ is a monotonic function in $\|x\|$, for the simplicity of
presentation we use $a(X)=\|X\|$ in the rest of this proof. As a result, given
$\|x\|=a$, we have

$\hat{\beta}^{\top}x\mid\|x\|=a\stackrel{{\scriptstyle
d}}{{=}}a\cdot\hat{\beta}^{\top}S=a\cdot\|\hat{\beta}\|\cdot S_{1},$

where $S_{1}$ is the first coordinate of $S$.

Consequently, if we further condition on the event where
$\hat{\beta}^{\top}x>0$ (as we assume $u>0$ throughout Section 3.2), we have

$\hat{\beta}^{\top}x\stackrel{{\scriptstyle d}}{{=}}a\cdot\|\hat{\beta}\|\cdot
S_{1}\mid S_{1}>0\stackrel{{\scriptstyle
d}}{{=}}a\cdot\|\hat{\beta}\|\cdot\frac{Z_{1}}{\sqrt{Z_{1}^{2}+Q}}\to a\cdot
R^{*}\cdot\frac{Z_{1}}{\sqrt{Z_{1}^{2}+Q}},$

where $Q\sim\chi_{p-1}^{2}$, $Z_{1}\sim N(0,1)$ and they are independent.

Due to the monotonicity of $C_{\kappa}(u)$ on $u$, we have that for any
$a_{1}>a_{2}$,

$C_{\kappa}(u)\mid\|x\|=a_{1}\stackrel{{\scriptstyle
d}}{{>}}C_{\kappa}(u)\mid\|x\|=a_{2},$

where the notation $\stackrel{{\scriptstyle d}}{{>}}$ denotes stochastic
dominance.

Consequently, we have

$\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in[a_{k-1},a_{k}]]<\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in(a_{k},a_{k+1}]],$

for $k=2,..,K-1$.

##### Proving the result for $k=K$

To complete the proof, it suffices to show that the inequality is also true
for $K$th quantile:

$\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in[a_{K-1},a_{K}]]<\mathbb{E}_{u}[u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u)\mid
a(X)\in(a_{k},a_{k+1}]],$

which is equivalent to

$\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K-1},a_{K}]\\}]<\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in(a_{k},a_{k+1}]\\}].$

In the above inequality, the right hand side can be decomposed into

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in(a_{k},a_{k+1}]\\}]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K},2p]\\}]$
$\displaystyle+\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[2p,a_{K+1}]\\}].$

Denote the $\alpha$ quantile of $\chi_{p}^{2}$ by $\chi_{\alpha,p}^{2}$. We
then have $a_{k}=\chi_{\frac{k}{K+1},p}^{2}$. We further decompose the
equation into

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K},2p]\\}]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K},\chi_{\frac{K+\delta}{K+1},p}^{2}]\\}]$
$\displaystyle+\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]\\}].$

In the following, we proceed to prove

$\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]\\}]>\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K-1},a_{K}\\}].$
(14)

We now use the approximation of the chi-square quantile: when $p\to\infty$, we
have

$a_{K}=\frac{1}{2}(z_{\frac{K}{K+1}}+\sqrt{2p})^{2}+o(1),\text{ and
}\chi_{\frac{K+\delta}{K+1},p}^{2}=\frac{1}{2}(z_{\frac{K+\delta}{K+1}}+\sqrt{2p})^{2}+o(1),$

where $z_{\alpha}$ denotes the $\alpha$-quantile of a standard normal random
variable.

Then

$\chi_{\frac{K+\delta}{K+1},p}^{2}-a_{K}=\frac{1}{2}(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}})(z_{\frac{K+\delta}{K+1}}+z_{\frac{K}{K+1}}+2\sqrt{2p}).$

Using the fact that $z_{1-\frac{1}{K}}=\sqrt{2\log K}+o(1)$ for $K\to\infty$,
then we have

$z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}}=\frac{-\log(1-\delta)}{\sqrt{2\log
K}}+o(1).$

In addition, for any $a\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]$ and
$a^{\prime}\in[a_{K-1},a_{K}]$, we have

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)=a]-\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)=a^{\prime}]$ $\displaystyle\geq$ $\displaystyle
C(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}}),$

for some universal constant $C$.

Therefore

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]]-\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]]$ $\displaystyle\geq$ $\displaystyle
C(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}}).$

Then

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]\\}]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]]\cdot\mathbb{P}(a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p])$
$\displaystyle\geq$
$\displaystyle\left(\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]]+C(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}})\right)\cdot(\frac{1}{K}-\frac{\delta}{K+1}+o(\frac{\delta}{K+1}))$
$\displaystyle=$
$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K-1},a_{K}\\}]$
$\displaystyle+C(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}})-(1+o(1))\frac{\delta}{K+1}\cdot\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]].$

The last equality uses the fact that $\mathbb{P}(a(X)\in[a_{K-1},a_{K}])=1/K,$
and therefore

$\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]\cdot\frac{1}{K}=\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K-1},a_{K}\\}]$

Then use the fact that
$|\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]]|=O(1)$ and we choose $\delta=o(1/\log K)$ so

$\frac{\delta}{K}=o(|\frac{\log(1-\delta)}{\sqrt{\log K}}|).$

Consequently,

$C(z_{\frac{K+\delta}{K+1}}-z_{\frac{K}{K+1}})-(1+o(1))\frac{\delta}{K+1}\cdot\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\mid
a(X)\in[a_{K-1},a_{K}]]>0,$

which implies

$\displaystyle\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[\chi_{\frac{K+\delta}{K+1},p}^{2},2p]\\}]\geq\mathbb{E}_{u}[(u-\mathbb{P}(Y=1\mid\hat{\mathbb{P}}_{1}(X)=u))\cdot\bm{1}\\{a(X)\in[a_{K-1},a_{K}\\}].$

Combining with equation 13, we prove equation 14 and complete the proof. ∎

###  G.3 Theoretical Justification of the calibration improvement using the
atypicality score

In this section, we provide the theoretical justification to understand why
incorporating the atypicality score will improve calibration. In particular,
we consider the binary classification problem with prediction $f:X\in[0,1]$
indicating the predicted probability of $Y=1$ given $X=x$.

For a predictor $f$, let us denote its conditional calibration error at an
atypicality level $\gamma$ by
$\textrm{CE}_{\gamma}(f)=\mathbb{E}[(f(X)-\mathbb{E}[Y|f(X)])^{2}|a(X)=\gamma]$.

######  Theorem G.2.

Consider the same setting as Theorem 3.1. Suppose the temperature function
$\hat{\tau}(a(X))=\operatorname*{arg\,min}_{\tau}\mathbb{E}[l(Y,\textup{Softmax}(f(X)/\tau(a(X))))]$
with $l$ being the cross entropy loss, and let
$\hat{\mathbb{P}}_{\textup{AAR}}(X)=\textup{Softmax}(f(X)/\hat{\tau}(a(X)))$.
Then

$\textup{CE}_{\gamma}(\hat{\mathbb{P}}_{\textup{AAR}})\leq\min\\{\textup{CE}_{\gamma}(\hat{\mathbb{P}}_{\textup{TS}}),\textup{CE}_{\gamma}(f)\\}.$
(15)

Proof: For a prediction function $f$, we first define the conditional mean
squared error of $f$ at an atypicality level $\gamma$ by
$\textup{MSE}_{\gamma}(f)=\mathbb{E}[(f(X)-Y)^{2}\mid a(X)=\gamma]$, then we
have

$\displaystyle\textup{MSE}_{\gamma}(f)-CE_{\gamma}(f)=$
$\displaystyle\mathbb{E}[(f(X)-Y)^{2}\mid
a(X)=\gamma]-\mathbb{E}[(f(X)-\mathbb{E}[Y\mid f(X),a(X)=\gamma])^{2}\mid
a(X)=\gamma]$ $\displaystyle=$ $\displaystyle\mathbb{E}[(\mathbb{E}[Y\mid
f(X),a(X)=\gamma]-Y)\cdot(2f(X)-\mathbb{E}[Y\mid f(X),a(X)=\gamma]-Y)\mid
a(X)=\gamma]$ $\displaystyle=$ $\displaystyle\mathbb{E}[(\mathbb{E}[Y\mid
f(X),a(X)=\gamma]-Y)\cdot(\mathbb{E}[Y\mid f(X),a(X)=\gamma]-Y)\mid
a(X)=\gamma]$ $\displaystyle+2\mathbb{E}[(\mathbb{E}[Y\mid
f(X),a(X)=\gamma]-Y)\cdot(f(X)-\mathbb{E}[Y\mid f(X),a(X)=\gamma]))\mid
a(X)=\gamma]$

Since

$\displaystyle\mathbb{E}[Y\mathbb{E}[Y\mid f(X),a(X)=\gamma]\mid a(X)=\gamma]$
$\displaystyle=$ $\displaystyle\mathbb{E}_{f(X)\mid
a(X)=\gamma}\mathbb{E}[Y\mathbb{E}[Y\mid f(X),a(X)=\gamma]\mid
f(X),a(X)=\gamma]]$ $\displaystyle=$
$\displaystyle\mathbb{E}[(\mathbb{E}[Y\mid f(X),a(X)=\gamma])^{2}\mid
a(X)=\gamma],$

we have

$\mathbb{E}[(\mathbb{E}[Y\mid f(X),a(X)=\gamma]-Y)\cdot(f(X)-\mathbb{E}[Y\mid
f(X),a(X)=\gamma]))\mid a(X)=\gamma]=0,$

and therefore

$\textup{MSE}_{\gamma}(f)-CE_{\gamma}(f)=\mathbb{E}[(\mathbb{E}[Y\mid
f(X),a(X)=\gamma]-Y)^{2}\mid a(X)=\gamma]$

Now that $\hat{\mathbb{P}}_{AAR}(f(x),a(x))$ is monotonic on the
$\hat{\mathbb{P}}(x)$, we have

$\mathbb{E}[Y\mid
f(x),a(X)=\gamma]=\mathbb{E}[Y\mid\hat{\mathbb{P}}_{AAR}(f(x),a(X)),a(X)=\gamma],$

implying

$\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{AAR})-CE_{\gamma}(\hat{\mathbb{P}}_{AAR})=\textup{MSE}_{\gamma}(\hat{\mathbb{P}})-CE_{\gamma}(\hat{\mathbb{P}}).$
(16)

Similarly, we have

$\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{TS})-CE_{\gamma}(\hat{\mathbb{P}}_{TS})=\textup{MSE}_{\gamma}(\hat{\mathbb{P}})-CE_{\gamma}(\hat{\mathbb{P}}).$
(17)

In the following, we will show that

$\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{AAR})<\min\\{\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{TS}),\textup{MSE}_{\gamma}(\hat{\mathbb{P}})\\}.$
(18)

First, as we consider the binary classification setting, with $l$ being the
cross-entropy loss, we have

$l(Y,\textup{Softmax}(f(X)/\tau(a(X))))=Y\log(\sigma(f(X)/\tau(a(X)))+(1-Y)\log(1-\sigma(f(X)/\tau(a(X))),$

where $\sigma(x)=1/(1+e^{x}).$

Then, by the definition of $\hat{\tau}(a(X))$, we have that

$\displaystyle\hat{\tau}(a(X))=$
$\displaystyle\operatorname*{arg\,min}_{\tau}\mathbb{E}[Y\log(\sigma(f(X)/\tau(a(X)))+(1-Y)\log(1-\sigma(f(X)/\tau(a(X)))]$
$\displaystyle=$
$\displaystyle\operatorname*{arg\,min}_{\tau}\mathbb{E}[\mathbb{E}[Y\log(\sigma(f(X)/\tau(a(X)))+(1-Y)\log(1-\sigma(f(X)/\tau(a(X)))\mid
a(X)]].$

Taking the derivative on the last line and setting it to zero, we have

$\mathbb{E}[\frac{Y}{\sigma(f(X)/\hat{\tau}(a(X))}-\frac{1-Y}{1-\sigma(f(X)/\hat{\tau}(a(X))}\mid
a(X)]=0,$

implying

$\mathbb{E}[\sigma(f(X)/\hat{\tau}(a(X))\mid a(X)]=\mathbb{E}[Y\mid a(X)].$

This makes the derivative of $\mathbb{E}[(Y-\sigma(f(X)/\tau(a(X))))^{2}\mid
a(X)]$ zero and therefore $\hat{\tau}(a(X))$ is also a minimizer of
$\mathbb{E}[(Y-\sigma(f(X)/\tau(a(X))))^{2}\mid a(X)]$:

$\hat{\tau}(a(X))=\operatorname*{arg\,min}_{\tau}\mathbb{E}[Y\log(\sigma(f(X)/\tau(a(X)))+(1-Y)\log(1-\sigma(f(X)/\tau(a(X)))]=\operatorname*{arg\,min}_{\tau}\mathbb{E}[(Y-\sigma(f(X)/\tau(a(X))))^{2}].$

Letting
$g(\gamma)=\operatorname*{arg\,min}_{c}\mathbb{E}[(Y-\sigma(f(X)/c))^{2}\mid
a(X)=\gamma]$, we have that

$g(a(X))=\operatorname*{arg\,min}_{\tau}\mathbb{E}[(Y-\sigma(f(X)/\tau(a(X))))^{2}\mid
a(X)],$

and therefore

$g(a(X))=\operatorname*{arg\,min}_{\tau}\mathbb{E}[\mathbb{E}[(Y-\sigma(f(X)/\tau(a(X))))^{2}\mid
a(X)]=\hat{\tau}(a(X)).$

As a result,

$\displaystyle\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{AAR})=$
$\displaystyle\mathbb{E}[(\hat{\mathbb{P}}_{AAR}(X)-Y)^{2}\mid a(X)=\gamma]$
$\displaystyle=$
$\displaystyle\mathbb{E}[(\hat{\mathbb{P}}_{AAR}(X)-Y)^{2}\mid a(X)=\gamma]$
$\displaystyle=$
$\displaystyle\mathbb{E}[(\sigma(f(X)/\hat{\tau}(a(X))-Y)^{2}\mid
a(X)=\gamma]$ $\displaystyle=$
$\displaystyle\mathbb{E}[(\textup{Softmax}(\hat{\mathbb{P}}(X)/g(a(X))-Y)^{2}\mid
a(X)=\gamma]$ $\displaystyle=$
$\displaystyle\operatorname*{arg\,min}_{c}\mathbb{E}[(\sigma(f(X)/c-Y)^{2}\mid
a(X)=\gamma]$ $\displaystyle\leq$
$\displaystyle\mathbb{E}[(\sigma(f(X)-Y)^{2}\mid a(X)=\gamma]$
$\displaystyle=$ $\displaystyle\textup{MSE}_{\gamma}(\hat{\mathbb{P}}).$

Similarly, we have
$\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{AAR})\leq\textup{MSE}_{\gamma}(\hat{\mathbb{P}}_{TS})$,
and therefore equation 18 holds.

Combining with equation 16 and equation 17, we have

$CE_{\gamma}(\hat{\mathbb{P}}_{AAR})\leq\min\\{CE_{\gamma}(\hat{\mathbb{P}}_{TS}),CE_{\gamma}(\hat{\mathbb{P}})\\}.$

Table 4: Conformal Calibration with Atypicality-Awareness.

Model Dataset Input Atypicality Group Confidence Group APS AA-APS RAPS AA-RAPS
Coverage SetSize Coverage SetSize Coverage SetSize Coverage SetSize ResNet152
ImageNet 1 1 $0.982\pm 0.002$ $51.192\pm 1.570$ $0.970\pm 0.003$ $32.694\pm
3.517$ $0.951\pm 0.004$ $7.168\pm 0.127$ $0.963\pm 0.004$ $14.408\pm 2.275$ 2
$0.984\pm 0.001$ $80.425\pm 2.506$ $0.961\pm 0.002$ $39.752\pm 1.670$
$0.940\pm 0.003$ $7.771\pm 0.180$ $0.966\pm 0.004$ $15.713\pm 0.977$ 3
$0.980\pm 0.001$ $114.526\pm 1.293$ $0.955\pm 0.004$ $55.887\pm 2.830$
$0.885\pm 0.005$ $8.485\pm 0.217$ $0.952\pm 0.004$ $25.191\pm 1.349$ 4
$0.979\pm 0.001$ $118.819\pm 1.787$ $0.949\pm 0.002$ $60.998\pm 1.293$
$0.833\pm 0.005$ $8.677\pm 0.235$ $0.949\pm 0.002$ $33.256\pm 0.812$ 5
$0.975\pm 0.001$ $111.696\pm 1.090$ $0.953\pm 0.002$ $66.884\pm 1.347$
$0.807\pm 0.006$ $8.801\pm 0.242$ $0.948\pm 0.002$ $42.109\pm 0.829$ 6
$0.958\pm 0.002$ $90.283\pm 1.460$ $0.949\pm 0.002$ $77.404\pm 1.682$
$0.777\pm 0.005$ $8.565\pm 0.229$ $0.946\pm 0.003$ $52.214\pm 1.401$ 2 1
$0.961\pm 0.004$ $16.767\pm 0.389$ $0.953\pm 0.004$ $14.533\pm 1.036$
$0.977\pm 0.002$ $5.279\pm 0.035$ $0.954\pm 0.005$ $3.888\pm 0.102$ 2
$0.964\pm 0.002$ $18.359\pm 0.530$ $0.951\pm 0.003$ $12.902\pm 0.814$
$0.974\pm 0.001$ $5.801\pm 0.046$ $0.950\pm 0.003$ $4.484\pm 0.079$ 3
$0.961\pm 0.001$ $21.409\pm 0.413$ $0.948\pm 0.002$ $14.636\pm 0.318$
$0.966\pm 0.002$ $5.964\pm 0.051$ $0.952\pm 0.003$ $5.172\pm 0.112$ 4
$0.964\pm 0.002$ $23.162\pm 0.321$ $0.956\pm 0.003$ $20.246\pm 0.634$
$0.966\pm 0.002$ $6.253\pm 0.081$ $0.962\pm 0.004$ $6.485\pm 0.112$ 5
$0.953\pm 0.002$ $24.833\pm 0.486$ $0.952\pm 0.003$ $25.470\pm 0.829$
$0.938\pm 0.002$ $6.331\pm 0.092$ $0.953\pm 0.003$ $9.446\pm 0.332$ 6
$0.929\pm 0.003$ $21.124\pm 0.290$ $0.943\pm 0.004$ $32.341\pm 1.343$
$0.909\pm 0.003$ $6.409\pm 0.106$ $0.952\pm 0.002$ $16.970\pm 0.828$ 3 1
$0.957\pm 0.002$ $6.843\pm 0.249$ $0.954\pm 0.004$ $5.860\pm 0.354$ $0.984\pm
0.001$ $4.839\pm 0.046$ $0.951\pm 0.004$ $3.015\pm 0.098$ 2 $0.963\pm 0.003$
$6.654\pm 0.202$ $0.965\pm 0.003$ $6.689\pm 0.406$ $0.986\pm 0.001$ $5.274\pm
0.044$ $0.966\pm 0.003$ $3.494\pm 0.071$ 3 $0.951\pm 0.002$ $7.609\pm 0.195$
$0.955\pm 0.003$ $8.542\pm 0.472$ $0.968\pm 0.001$ $5.435\pm 0.042$ $0.951\pm
0.003$ $4.052\pm 0.097$ 4 $0.938\pm 0.002$ $8.415\pm 0.205$ $0.955\pm 0.002$
$12.476\pm 0.349$ $0.961\pm 0.002$ $5.728\pm 0.048$ $0.951\pm 0.002$ $5.085\pm
0.085$ 5 $0.945\pm 0.002$ $8.500\pm 0.204$ $0.955\pm 0.003$ $12.539\pm 0.815$
$0.964\pm 0.002$ $5.724\pm 0.050$ $0.956\pm 0.003$ $5.531\pm 0.154$ 6
$0.932\pm 0.004$ $8.009\pm 0.199$ $0.953\pm 0.004$ $13.942\pm 0.657$ $0.944\pm
0.003$ $5.744\pm 0.048$ $0.953\pm 0.003$ $7.718\pm 0.745$ 4 1 $0.958\pm 0.001$
$1.480\pm 0.034$ $0.962\pm 0.003$ $1.828\pm 0.131$ $0.985\pm 0.001$ $3.810\pm
0.132$ $0.962\pm 0.003$ $1.613\pm 0.078$ 2 $0.953\pm 0.002$ $1.388\pm 0.021$
$0.958\pm 0.002$ $1.913\pm 0.118$ $0.982\pm 0.001$ $3.816\pm 0.141$ $0.957\pm
0.002$ $1.692\pm 0.070$ 3 $0.931\pm 0.002$ $1.451\pm 0.028$ $0.947\pm 0.003$
$2.699\pm 0.146$ $0.967\pm 0.002$ $3.920\pm 0.148$ $0.951\pm 0.003$ $2.237\pm
0.071$ 4 $0.935\pm 0.001$ $1.450\pm 0.020$ $0.958\pm 0.003$ $3.488\pm 0.224$
$0.976\pm 0.002$ $3.880\pm 0.128$ $0.957\pm 0.003$ $2.577\pm 0.117$ 5
$0.928\pm 0.002$ $1.424\pm 0.018$ $0.951\pm 0.003$ $5.173\pm 0.391$ $0.963\pm
0.002$ $4.160\pm 0.127$ $0.952\pm 0.002$ $3.185\pm 0.077$ 6 $0.902\pm 0.003$
$1.476\pm 0.028$ $0.958\pm 0.005$ $8.900\pm 0.726$ $0.955\pm 0.004$ $3.925\pm
0.130$ $0.959\pm 0.004$ $4.235\pm 0.163$ 5 1 $0.988\pm 0.001$ $1.000\pm 0.000$
$0.988\pm 0.001$ $1.000\pm 0.000$ $0.989\pm 0.001$ $1.344\pm 0.116$ $0.988\pm
0.001$ $1.000\pm 0.000$ 2 $0.982\pm 0.001$ $1.000\pm 0.000$ $0.982\pm 0.001$
$1.000\pm 0.000$ $0.984\pm 0.001$ $1.363\pm 0.122$ $0.982\pm 0.001$ $1.000\pm
0.000$ 3 $0.976\pm 0.001$ $1.000\pm 0.000$ $0.976\pm 0.001$ $1.001\pm 0.001$
$0.980\pm 0.002$ $1.351\pm 0.122$ $0.976\pm 0.001$ $1.001\pm 0.001$ 4
$0.977\pm 0.001$ $1.000\pm 0.000$ $0.977\pm 0.001$ $1.020\pm 0.009$ $0.980\pm
0.001$ $1.363\pm 0.109$ $0.977\pm 0.001$ $1.004\pm 0.003$ 5 $0.965\pm 0.001$
$1.000\pm 0.000$ $0.965\pm 0.001$ $1.137\pm 0.055$ $0.968\pm 0.002$ $1.342\pm
0.115$ $0.965\pm 0.001$ $1.063\pm 0.028$ 6 $0.970\pm 0.002$ $1.000\pm 0.000$
$0.972\pm 0.003$ $1.607\pm 0.125$ $0.973\pm 0.003$ $1.339\pm 0.107$ $0.973\pm
0.003$ $1.539\pm 0.080$ 6 1 $0.992\pm 0.000$ $1.000\pm 0.000$ $0.992\pm 0.000$
$1.000\pm 0.000$ $0.992\pm 0.000$ $1.000\pm 0.000$ $0.992\pm 0.000$ $1.000\pm
0.000$ 2 $0.989\pm 0.001$ $1.000\pm 0.000$ $0.989\pm 0.001$ $1.000\pm 0.000$
$0.989\pm 0.001$ $1.000\pm 0.000$ $0.989\pm 0.001$ $1.000\pm 0.000$ 3
$0.992\pm 0.000$ $1.000\pm 0.000$ $0.992\pm 0.000$ $1.000\pm 0.000$ $0.992\pm
0.000$ $1.000\pm 0.000$ $0.992\pm 0.000$ $1.000\pm 0.000$ 4 $0.995\pm 0.001$
$1.000\pm 0.000$ $0.995\pm 0.001$ $1.000\pm 0.000$ $0.995\pm 0.001$ $1.000\pm
0.000$ $0.995\pm 0.001$ $1.000\pm 0.000$ 5 $0.989\pm 0.001$ $1.000\pm 0.000$
$0.989\pm 0.001$ $1.000\pm 0.000$ $0.989\pm 0.001$ $1.000\pm 0.000$ $0.989\pm
0.001$ $1.000\pm 0.000$ 6 $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$
$1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm
0.000$ ResNet18 ImageNet 1 1 $0.982\pm 0.001$ $167.265\pm 2.058$ $0.952\pm
0.004$ $79.307\pm 3.180$ $0.876\pm 0.008$ $20.623\pm 1.724$ $0.952\pm 0.003$
$67.925\pm 2.552$ 2 $0.986\pm 0.001$ $157.710\pm 1.132$ $0.956\pm 0.002$
$85.303\pm 1.015$ $0.848\pm 0.009$ $20.740\pm 1.745$ $0.958\pm 0.003$
$80.333\pm 3.054$ 3 $0.970\pm 0.001$ $147.730\pm 1.267$ $0.943\pm 0.003$
$95.805\pm 2.283$ $0.810\pm 0.009$ $20.689\pm 1.725$ $0.938\pm 0.003$
$91.786\pm 1.213$ 4 $0.963\pm 0.002$ $129.188\pm 1.035$ $0.954\pm 0.003$
$109.238\pm 1.837$ $0.815\pm 0.009$ $20.328\pm 1.592$ $0.953\pm 0.002$
$102.926\pm 2.651$ 5 $0.957\pm 0.002$ $111.271\pm 1.097$ $0.952\pm 0.002$
$103.457\pm 2.047$ $0.809\pm 0.011$ $19.932\pm 1.441$ $0.942\pm 0.004$
$91.695\pm 3.655$ 6 $0.941\pm 0.004$ $82.127\pm 1.167$ $0.951\pm 0.005$
$96.879\pm 4.165$ $0.828\pm 0.008$ $19.108\pm 1.138$ $0.950\pm 0.004$
$87.490\pm 3.277$ 2 1 $0.972\pm 0.001$ $37.918\pm 1.031$ $0.958\pm 0.001$
$25.001\pm 0.687$ $0.978\pm 0.002$ $15.096\pm 0.135$ $0.959\pm 0.003$
$10.650\pm 0.322$ 2 $0.959\pm 0.002$ $42.117\pm 0.743$ $0.951\pm 0.004$
$31.357\pm 1.294$ $0.954\pm 0.002$ $15.879\pm 0.162$ $0.951\pm 0.004$
$16.748\pm 0.617$ 3 $0.964\pm 0.003$ $43.312\pm 0.868$ $0.957\pm 0.003$
$39.329\pm 1.425$ $0.951\pm 0.003$ $16.138\pm 0.225$ $0.960\pm 0.002$
$22.619\pm 1.125$ 4 $0.951\pm 0.003$ $41.803\pm 0.701$ $0.955\pm 0.003$
$45.617\pm 1.420$ $0.930\pm 0.002$ $16.357\pm 0.295$ $0.953\pm 0.003$
$31.513\pm 1.447$ 5 $0.945\pm 0.002$ $36.697\pm 0.482$ $0.957\pm 0.002$
$45.379\pm 1.101$ $0.935\pm 0.002$ $16.094\pm 0.249$ $0.961\pm 0.001$
$32.569\pm 0.974$ 6 $0.930\pm 0.003$ $27.148\pm 0.321$ $0.949\pm 0.005$
$43.307\pm 2.384$ $0.924\pm 0.003$ $15.730\pm 0.144$ $0.948\pm 0.005$
$28.152\pm 1.657$ 3 1 $0.964\pm 0.001$ $15.809\pm 0.414$ $0.954\pm 0.002$
$13.080\pm 0.586$ $0.985\pm 0.002$ $13.208\pm 0.471$ $0.955\pm 0.002$
$6.864\pm 0.146$ 2 $0.960\pm 0.002$ $18.034\pm 0.515$ $0.956\pm 0.003$
$16.515\pm 0.618$ $0.978\pm 0.001$ $13.659\pm 0.394$ $0.955\pm 0.002$
$8.275\pm 0.221$ 3 $0.964\pm 0.002$ $19.231\pm 0.344$ $0.957\pm 0.003$
$17.463\pm 0.722$ $0.979\pm 0.001$ $13.901\pm 0.310$ $0.957\pm 0.003$
$8.794\pm 0.163$ 4 $0.950\pm 0.001$ $19.479\pm 0.207$ $0.952\pm 0.003$
$21.470\pm 0.744$ $0.974\pm 0.001$ $13.899\pm 0.304$ $0.955\pm 0.003$
$11.390\pm 0.313$ 5 $0.940\pm 0.002$ $17.801\pm 0.160$ $0.953\pm 0.003$
$22.066\pm 0.545$ $0.957\pm 0.002$ $14.036\pm 0.296$ $0.951\pm 0.003$
$12.743\pm 0.271$ 6 $0.931\pm 0.002$ $13.096\pm 0.153$ $0.949\pm 0.004$
$21.352\pm 1.080$ $0.956\pm 0.002$ $13.169\pm 0.394$ $0.949\pm 0.005$
$12.924\pm 0.663$ 4 1 $0.964\pm 0.002$ $5.377\pm 0.082$ $0.957\pm 0.003$
$4.595\pm 0.226$ $0.990\pm 0.002$ $11.670\pm 0.870$ $0.957\pm 0.004$ $3.823\pm
0.144$ 2 $0.958\pm 0.002$ $6.371\pm 0.130$ $0.954\pm 0.003$ $5.887\pm 0.275$
$0.986\pm 0.003$ $11.954\pm 0.767$ $0.953\pm 0.003$ $4.648\pm 0.106$ 3
$0.946\pm 0.002$ $5.810\pm 0.109$ $0.953\pm 0.002$ $7.575\pm 0.321$ $0.984\pm
0.003$ $12.145\pm 0.751$ $0.954\pm 0.003$ $5.364\pm 0.157$ 4 $0.940\pm 0.001$
$6.698\pm 0.159$ $0.950\pm 0.003$ $9.025\pm 0.498$ $0.980\pm 0.002$ $12.432\pm
0.700$ $0.950\pm 0.004$ $6.110\pm 0.225$ 5 $0.926\pm 0.003$ $5.817\pm 0.102$
$0.949\pm 0.004$ $10.234\pm 0.682$ $0.977\pm 0.003$ $12.325\pm 0.750$
$0.950\pm 0.004$ $6.925\pm 0.198$ 6 $0.915\pm 0.002$ $4.948\pm 0.098$
$0.949\pm 0.003$ $11.054\pm 0.601$ $0.968\pm 0.004$ $11.478\pm 0.813$
$0.948\pm 0.003$ $7.368\pm 0.183$ 5 1 $0.963\pm 0.002$ $1.193\pm 0.008$
$0.968\pm 0.002$ $1.497\pm 0.052$ $0.989\pm 0.002$ $8.559\pm 1.391$ $0.967\pm
0.002$ $1.413\pm 0.041$ 2 $0.959\pm 0.002$ $1.232\pm 0.014$ $0.963\pm 0.002$
$1.456\pm 0.063$ $0.984\pm 0.003$ $8.755\pm 1.352$ $0.962\pm 0.002$ $1.485\pm
0.057$ 3 $0.957\pm 0.001$ $1.243\pm 0.012$ $0.961\pm 0.002$ $1.693\pm 0.085$
$0.986\pm 0.003$ $8.947\pm 1.344$ $0.961\pm 0.002$ $1.637\pm 0.071$ 4
$0.945\pm 0.002$ $1.264\pm 0.013$ $0.959\pm 0.002$ $2.492\pm 0.131$ $0.985\pm
0.003$ $8.933\pm 1.325$ $0.959\pm 0.002$ $2.309\pm 0.082$ 5 $0.943\pm 0.002$
$1.234\pm 0.013$ $0.953\pm 0.002$ $2.770\pm 0.345$ $0.976\pm 0.004$ $8.913\pm
1.365$ $0.954\pm 0.002$ $2.540\pm 0.195$ 6 $0.932\pm 0.002$ $1.214\pm 0.010$
$0.961\pm 0.002$ $3.875\pm 0.249$ $0.976\pm 0.004$ $8.459\pm 1.351$ $0.963\pm
0.003$ $3.635\pm 0.192$ 6 1 $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$
$1.000\pm 0.000$ $0.993\pm 0.001$ $4.540\pm 1.729$ $0.991\pm 0.001$ $1.000\pm
0.000$ 2 $0.986\pm 0.001$ $1.000\pm 0.000$ $0.986\pm 0.001$ $1.000\pm 0.000$
$0.990\pm 0.002$ $4.395\pm 1.738$ $0.986\pm 0.001$ $1.000\pm 0.000$ 3
$0.988\pm 0.001$ $1.000\pm 0.000$ $0.988\pm 0.001$ $1.000\pm 0.000$ $0.992\pm
0.001$ $4.449\pm 1.739$ $0.988\pm 0.001$ $1.000\pm 0.000$ 4 $0.988\pm 0.001$
$1.000\pm 0.000$ $0.988\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.002$ $4.481\pm
1.729$ $0.988\pm 0.001$ $1.000\pm 0.000$ 5 $0.986\pm 0.001$ $1.000\pm 0.000$
$0.986\pm 0.001$ $1.000\pm 0.000$ $0.990\pm 0.002$ $4.390\pm 1.741$ $0.986\pm
0.001$ $1.000\pm 0.000$ 6 $0.981\pm 0.001$ $1.000\pm 0.000$ $0.981\pm 0.001$
$1.003\pm 0.003$ $0.987\pm 0.002$ $4.146\pm 1.767$ $0.981\pm 0.001$ $1.009\pm
0.007$ ResNet50 ImageNet 1 1 $0.980\pm 0.002$ $80.190\pm 1.594$ $0.955\pm
0.004$ $50.203\pm 2.778$ $0.945\pm 0.002$ $9.874\pm 0.169$ $0.964\pm 0.002$
$16.099\pm 0.593$ 2 $0.986\pm 0.002$ $118.839\pm 1.737$ $0.955\pm 0.003$
$52.066\pm 2.094$ $0.896\pm 0.003$ $10.681\pm 0.209$ $0.950\pm 0.003$
$25.545\pm 0.845$ 3 $0.979\pm 0.001$ $134.702\pm 0.697$ $0.951\pm 0.003$
$60.743\pm 1.012$ $0.848\pm 0.004$ $11.056\pm 0.228$ $0.940\pm 0.002$
$35.518\pm 0.870$ 4 $0.976\pm 0.001$ $126.892\pm 0.939$ $0.952\pm 0.003$
$79.627\pm 2.106$ $0.835\pm 0.004$ $11.079\pm 0.230$ $0.953\pm 0.004$
$51.339\pm 1.445$ 5 $0.964\pm 0.001$ $114.559\pm 0.975$ $0.954\pm 0.003$
$93.609\pm 3.277$ $0.797\pm 0.005$ $11.011\pm 0.232$ $0.942\pm 0.003$
$63.441\pm 1.422$ 6 $0.949\pm 0.002$ $88.034\pm 0.799$ $0.950\pm 0.003$
$88.418\pm 1.898$ $0.806\pm 0.005$ $10.744\pm 0.221$ $0.957\pm 0.002$
$62.049\pm 1.862$ 2 1 $0.961\pm 0.002$ $17.461\pm 0.533$ $0.952\pm 0.004$
$14.757\pm 0.785$ $0.976\pm 0.002$ $7.080\pm 0.116$ $0.956\pm 0.004$ $5.205\pm
0.127$ 2 $0.967\pm 0.002$ $24.896\pm 0.732$ $0.958\pm 0.002$ $20.892\pm 1.441$
$0.965\pm 0.002$ $8.013\pm 0.106$ $0.958\pm 0.002$ $6.879\pm 0.191$ 3
$0.961\pm 0.001$ $29.101\pm 0.759$ $0.947\pm 0.003$ $21.754\pm 1.123$
$0.963\pm 0.001$ $8.134\pm 0.126$ $0.956\pm 0.003$ $8.773\pm 0.321$ 4
$0.958\pm 0.002$ $30.078\pm 0.379$ $0.956\pm 0.003$ $28.199\pm 0.951$
$0.946\pm 0.002$ $8.398\pm 0.129$ $0.951\pm 0.003$ $11.264\pm 0.347$ 5
$0.944\pm 0.001$ $29.401\pm 0.357$ $0.946\pm 0.002$ $32.362\pm 1.180$
$0.937\pm 0.003$ $8.559\pm 0.143$ $0.952\pm 0.002$ $15.040\pm 0.547$ 6
$0.939\pm 0.002$ $22.100\pm 0.488$ $0.954\pm 0.004$ $33.527\pm 1.955$
$0.922\pm 0.003$ $8.269\pm 0.135$ $0.955\pm 0.001$ $18.645\pm 0.283$ 3 1
$0.958\pm 0.002$ $9.385\pm 0.202$ $0.953\pm 0.002$ $7.633\pm 0.346$ $0.988\pm
0.001$ $6.471\pm 0.129$ $0.955\pm 0.002$ $3.820\pm 0.076$ 2 $0.951\pm 0.003$
$9.619\pm 0.194$ $0.949\pm 0.005$ $8.739\pm 0.516$ $0.977\pm 0.002$ $6.919\pm
0.118$ $0.950\pm 0.004$ $4.648\pm 0.082$ 3 $0.957\pm 0.002$ $11.449\pm 0.274$
$0.957\pm 0.003$ $10.853\pm 0.446$ $0.975\pm 0.001$ $7.163\pm 0.112$ $0.954\pm
0.004$ $5.130\pm 0.102$ 4 $0.953\pm 0.001$ $10.809\pm 0.223$ $0.951\pm 0.002$
$10.546\pm 0.396$ $0.972\pm 0.001$ $7.600\pm 0.113$ $0.952\pm 0.003$ $5.872\pm
0.235$ 5 $0.947\pm 0.002$ $11.507\pm 0.228$ $0.951\pm 0.003$ $14.754\pm 0.650$
$0.966\pm 0.003$ $7.414\pm 0.114$ $0.953\pm 0.004$ $6.874\pm 0.157$ 6
$0.920\pm 0.003$ $9.150\pm 0.188$ $0.951\pm 0.005$ $20.041\pm 1.624$ $0.944\pm
0.002$ $7.143\pm 0.101$ $0.950\pm 0.002$ $9.505\pm 0.576$ 4 1 $0.955\pm 0.002$
$1.739\pm 0.025$ $0.953\pm 0.002$ $1.667\pm 0.070$ $0.985\pm 0.002$ $5.088\pm
0.196$ $0.954\pm 0.002$ $1.574\pm 0.038$ 2 $0.948\pm 0.002$ $1.958\pm 0.016$
$0.950\pm 0.002$ $1.964\pm 0.083$ $0.983\pm 0.001$ $5.285\pm 0.189$ $0.951\pm
0.003$ $1.845\pm 0.061$ 3 $0.936\pm 0.003$ $2.082\pm 0.034$ $0.955\pm 0.002$
$4.255\pm 0.202$ $0.976\pm 0.002$ $5.495\pm 0.164$ $0.957\pm 0.003$ $3.423\pm
0.123$ 4 $0.936\pm 0.003$ $2.173\pm 0.045$ $0.955\pm 0.003$ $4.508\pm 0.303$
$0.973\pm 0.002$ $5.573\pm 0.170$ $0.952\pm 0.002$ $3.013\pm 0.110$ 5
$0.928\pm 0.002$ $2.071\pm 0.041$ $0.953\pm 0.003$ $8.483\pm 0.789$ $0.968\pm
0.002$ $5.720\pm 0.167$ $0.951\pm 0.002$ $4.338\pm 0.138$ 6 $0.895\pm 0.004$
$1.932\pm 0.026$ $0.950\pm 0.005$ $10.274\pm 1.147$ $0.950\pm 0.003$ $5.379\pm
0.151$ $0.948\pm 0.004$ $5.270\pm 0.220$ 5 1 $0.979\pm 0.001$ $1.000\pm 0.000$
$0.979\pm 0.001$ $1.001\pm 0.001$ $0.985\pm 0.001$ $1.918\pm 0.222$ $0.979\pm
0.001$ $1.000\pm 0.000$ 2 $0.982\pm 0.001$ $1.000\pm 0.000$ $0.982\pm 0.001$
$1.015\pm 0.008$ $0.985\pm 0.001$ $1.880\pm 0.221$ $0.982\pm 0.001$ $1.019\pm
0.007$ 3 $0.964\pm 0.002$ $1.000\pm 0.000$ $0.964\pm 0.002$ $1.126\pm 0.028$
$0.971\pm 0.002$ $1.977\pm 0.222$ $0.965\pm 0.002$ $1.178\pm 0.045$ 4
$0.972\pm 0.002$ $1.000\pm 0.000$ $0.973\pm 0.002$ $1.186\pm 0.047$ $0.978\pm
0.002$ $1.958\pm 0.205$ $0.973\pm 0.002$ $1.156\pm 0.050$ 5 $0.968\pm 0.002$
$1.000\pm 0.000$ $0.971\pm 0.002$ $1.417\pm 0.102$ $0.974\pm 0.002$ $1.911\pm
0.222$ $0.971\pm 0.002$ $1.293\pm 0.067$ 6 $0.946\pm 0.002$ $1.000\pm 0.000$
$0.957\pm 0.002$ $1.842\pm 0.143$ $0.956\pm 0.003$ $1.833\pm 0.193$ $0.957\pm
0.002$ $1.763\pm 0.112$ 6 1 $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$
$1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm
0.000$ 2 $0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm 0.000$
$0.991\pm 0.001$ $1.000\pm 0.000$ $0.991\pm 0.001$ $1.000\pm 0.000$ 3
$0.994\pm 0.001$ $1.000\pm 0.000$ $0.994\pm 0.001$ $1.000\pm 0.000$ $0.994\pm
0.001$ $1.000\pm 0.000$ $0.994\pm 0.001$ $1.000\pm 0.000$ 4 $0.992\pm 0.001$
$1.000\pm 0.000$ $0.992\pm 0.001$ $1.000\pm 0.000$ $0.992\pm 0.001$ $1.000\pm
0.000$ $0.992\pm 0.001$ $1.000\pm 0.000$ 5 $0.992\pm 0.001$ $1.000\pm 0.000$
$0.992\pm 0.001$ $1.000\pm 0.000$ $0.992\pm 0.001$ $1.000\pm 0.000$ $0.992\pm
0.001$ $1.000\pm 0.000$ 6 $0.987\pm 0.001$ $1.000\pm 0.000$ $0.987\pm 0.001$
$1.000\pm 0.000$ $0.987\pm 0.001$ $1.000\pm 0.000$ $0.987\pm 0.001$ $1.000\pm
0.000$

(a)

(b)

Figure 13: Post-hoc Recalibration for Classification with 5-NN distance as an
Atypicality Metric. (a) Balanced Supervised Classification: Atypicality-Aware
Recalibration improves the calibration of models trained with balanced
datasets, across atypicality groups. (b) Imbalanced Supervised Classification:
Atypicality-Aware Recalibration improves both the calibration across groups
and the overall accuracy of models trained with imbalanced datasets with
5-nearest neighbors distance as an atypicality metric.

##  Appendix H Limitations

###  H.1 Quantifying Atypicality

Since we do not have access to the true distribution of $\mathbb{P}(X)$, we
estimate it through the model, e.g. using the embeddings. This means we are
capturing the atypicality not solely with respect to the training distribution
but also the model. It is possible that a model that does not fit the data
well and produces low-quality atypicality estimates. _We would like to stress
that our goal here is to show that even simple estimators can demonstrate
significant benefits._ In general, we observe that our findings hold for large
datasets and widely used models, and atypicality gives a semantically