11institutetext: Computer Science Department, Vrije Universiteit Amsterdam, NL

# A Framework for Imbalanced Time-series Forecasting

Luis P. Silvestrin 11 0000-0002-5759-1986 Leonardos Pantiskas 11
0000-0002-4898-5334 Mark Hoogendoorn 11 0000-0003-3356-3574

###### Abstract

Time-series forecasting plays an important role in many domains. Boosted by
the advances in Deep Learning algorithms, it has for instance been used to
predict wind power for eolic energy production, stock market fluctuations, or
motor overheating. In some of these tasks, we are interested in predicting
accurately some particular moments which often are underrepresented in the
dataset, resulting in a problem known as _imbalanced regression_. In the
literature, while recognized as a challenging problem, limited attention has
been devoted on how to handle the problem in a practical setting. In this
paper, we put forward a general approach to analyze time-series forecasting
problems focusing on those underrepresented moments to reduce imbalances. Our
approach has been developed based on a case study in a large industrial
company, which we use to exemplify the approach.

###### Keywords:

Imbalanced regression Deep Learning Time-series forecasting Multivariate time-
series.

##  1 Introduction

Due to the recent advances in artificial intelligence research, the task of
time-series forecasting is being increasingly tackled with machine learning
and deep learning techniques. There has been a large number of approaches
suggested, ranging from relatively simple machine learning models[1] to a
variety of deep learning models[12]. Those approaches have been utilized in a
broad spectrum of forecasting tasks, such as wind power forecasting, stock
market prediction and motor temperature prediction[12]. In the above examples
of tasks, as well as in multiple other applied cases, some samples are more
crucial from the point of view of the user and thus would require a more
accurate prediction from a model compared to its average performance. At the
same time, those data points may be scarce in the training data. Hence, if
left unattended performance might be worse than average for that data, which
is highly undesirable. This issue is characterized as imbalanced regression,
and so far has been addressed with data pre-processing or ensemble model
methods[3].

Despite the existing methods to tackle imbalanced regression problems, it is
still a non-trivial task to data scientists and machine learning practitioners
to identify and solve them in real-life time-series forecasting contexts. In
the effort of developing the best performing machine learning model and
minimizing the error across all data points, some important artifacts in the
data might be overlooked. Moreover, data sampling methods or ensemble model
approaches[4][3] up until now focus on minimizing the prediction error in the
underrepresented data samples and assume that the remaining data is
negligible. That assumption is inaccurate, as in some applications, for
example in stock market prediction, the cost of larger forecasting error in
the more frequent cases could offset in the long run the potential benefit of
a smaller error in a rare case. In order to tackle real-world applications,
there is a need for a broader, balanced, flexible and iterative approach,
honed through interaction with domain experts and integrating the latest
research in predictive models.

In this paper we propose such an approach that has been designed based on a
case study in a large industrial company, targeted to forecast the temperature
of a core component in a large production line. The approach involves three
steps: first selecting a weight function which quantifies the sample
importance; then applying one or more sampling methods to the data; and
finally training and evaluating the model with and without sampling. In the
last step, we also analyze the input importance learnt by the model using
SHAP[7] to gain insights about the effect of the imbalance. To exemplify our
approach, we show how it is used for the aforementioned industrial task. We
study the impact of choices in each of the steps, comparing different sampling
techniques and deep learning models. In the end, we also combine the sampling
with attention mechanisms [13] to extract insights of what is learned by a
deep learning model.

##  2 Related Work

The advancements in data availability from a plethora of sources, the
increasing computational capacity and the progress in artificial intelligence
research has led to the usage of machine and deep learning models across a
multitude of applications of time-series forecasting. Previous work [12]
surveys several use-cases including wind power forecasting, stock price
prediction and estimation of remaining useful life of motors. Despite this
widespread use of models, the majority of works present specific architectures
for specific datasets, while works focusing on integrated frameworks in the
sense of structured approaches to a more generalized problem are more scarce.

Although there has been an extensive amount of work in handling imbalanced
datasets in classification tasks [3], regression with imbalanced data in the
area of machine and deep learning has not been largely covered. In [4], Branco
et al. study the effect of three proposed sampling methods on the predictive
performance of machine learning models. In an applied example, in [11], the
objective is that high water flows are predicted in a timely and accurate
manner, and the problem is addressed with various sampling and ensemble
methods, with an artificial neural network as a base. However, there is still
a need for research into a structured approach towards real-world imbalanced
regression problems, especially in the context of state of the art deep
learning models.

##  3 Methodology

It is common in time-series forecasting that certain data samples have more
importance than others, but are also underrepresented in the dataset,
resulting in an imbalanced regression problem. In this section we present a
new approach for identifying and tackling this discrepancy with respect to
imbalances in the context of regression tasks. This approach uses a weight
function that quantifies the importance of each sample which is combined with
under-sampling methods to create a more balanced dataset. The new sampled
dataset is then evaluated first visually, by making density plots of the data
and then numerically, by using it for training and testing a predictive model.

###  3.1 Steps for Identifying and Treating Imbalanced Regression

We propose a set of general steps for approaching the imbalance on time-series
forecasting problems, which has been defined based on experiences we have
collected in applying machine learning in a large scale industrial company. It
consists of three steps illustrated in Figure 1. The first step is to select
or define a weight function $w_{i}$ to quantify the sample importance, which
allows us to identify and compare the different regions of interest in the
data. The second step consists of selecting one or more sampling methods based
on the weight function, applying them to the data and comparing the resulting
distribution of $w_{i}$ against the original data using density plots.
Finally, in the third step, a predictive model is trained and evaluated using
both the sampled and the original versions of the dataset. A feedback loop can
take the user from step 3 back to step 2 if the current combination of the
selected sampling methods and models does not provide a satisfying performance
after evaluation. We provide more details about each step in the following
sections.

Figure 1: Flowchart of the framework showing the three steps and how they
interact with each other.

####  3.1.1 Step 1: Weight Function Definition

In the example of forecasting the temperature of a motor, there can be several
days where the temperature is stable, with only small fluctuations, and only a
few days where the temperature increases or decreases largely. Let us assume a
use-case where the user is interested in building a model to predict
accurately these rarer moments when the temperature changes more than usual.
In such example, the daily temperature variation can be computed as a function
of the data, which we refer in our framework as the weight function. In
addition, we say that the data points mapped to a high variation by the weight
function belong to a region of interest.

In general, the weight function can depend either only on the target variable
(or a transformation of it), only on a subset of the input variables (e.g. to
signify working points of interest) or on a combination of input and output
variables, to express complex regions of interest. It can be written as
$w_{i}=f(x_{i},y_{i})$, where $x_{i}$ and $y_{i}$ refer to the input (a vector
for multivariate input) and the prediction target, respectively.

Equation 1 gives an example of such a function for the target variation in the
context of time-series forecasting. The weight $w_{t}$ models the variation of
the forecast target $y$ over the forecast horizon $\Delta$ at time step $t$.

$w_{t}=\lvert y(t+\Delta)-y(t)\rvert$ (1)

####  3.1.2 Step 2: Application of Sampling Method

At this step, a sampling method is applied to the dataset and its effects are
analyzed. The sampling is based on the weight function previously selected and
the relative proportions that the user wants to keep for the different regions
of interest. We identify three scenarios for this step and we propose an
under-sampling method for each one of them.

  * •

Threshold under-sampling: when the user identifies which region of the weight
function is not important for the forecast task and can be removed;

  * •

Stochastic under-sampling: when some regions are more important than others,
but none can be entirely discarded;

  * •

Inverse histogram under-sampling: when all regions are equally important and
the user wants to have a balanced distribution of data over all of them.

Threshold under-sampling (TUS) consists of removing all samples that lie below
a given threshold of the weight function, and all the remaining samples have
the same chance of being selected. This method is suitable for the cases where
the user knows exactly what is the region of interest is in order to be able
to select the best threshold, and it assumes that the samples below the
threshold aren’t interesting to the prediction task. Equation 2 expresses the
chance of sampling data point $t$ given its weight $w_{t}$.

$\displaystyle\text{TUS}(w_{t})$ $\displaystyle=\begin{cases}1,&\text{if
}w_{t}>\tau\\\ 0,&\text{otherwise}\end{cases}$ (2)
$\displaystyle\text{SUS}(w_{t})$ $\displaystyle=w_{t}^{f}$ (3)
$\displaystyle\text{IHS}(w_{t})$ $\displaystyle=h^{-1}(w_{t})$ (4)

Stochastic under-sampling (SUS) uses the weight $w_{t}$ computed for each data
point as the probability of sampling it. Different from the TUS, SUS allows
every sample, even the ones with lower weight value, to be sampled to avoid
the creation of a new imbalance against those samples. Equation 3 models the
relative probability of sampling a window from the dataset at time $t$ by
using SUS. The factor $f$ is used to increase (or decrease) the effect of the
weights, thus emphasizing the more interesting moments which might be
underrepresented in the data.

Inverse histogram under-sampling (IHS) is an automatic method to obtain a
sample where data is approximately uniformly distributed across the selected
weight function $w_{t}$. It consists of building a histogram of the values of
$w_{t}$ in the dataset and taking the inverse of the frequency of each value
as the chance of sampling it. It ensures that each $w_{t}$ will be under-
sampled proportionally to its original frequency, so the most common values
will have lower chance, while the rarer values will have higher chance. In Eq.
4 we can see a formalization of the method, where $h(w_{t})$ represents the
frequency of $w_{t}$ in the data histogram.

A good approach to gain insight about the data and the result of the sampling
method is to compare the density plot of the weight function with and without
using the sampling. Such a plot can show the regions which are over or
underrepresented in the data, and can also give insights about how to tune the
parameters of the selected method or which method should be selected.

In some real-life cases, it might not be easy to infer directly which of the
three scenarios fits the problem better. In those cases, a subset of these
methods can be selected for the next step, where we provide a heuristic to
select the final method.

####  3.1.3 Step 3: Predictive Model Training and Evaluation

Finally, at this step, we can assess how much a predictive model improves by
using the selected sampling methods. For that, we train and evaluate the model
with and without using the sampling method on a separate evaluation set, so we
end up with different combinations of training and evaluation sets which we
will use to contrast the obtained evaluation errors and determine if the model
benefits from the sampling. For the cases where the goal is to have a model
that performs well on the samples of higher weight without sacrificing the
performance on the rest of the data, we propose a heuristic for selecting the
final sampling method to train the predictive model based on the results of
the different evaluation sets. It is defined as:

  1. 1.

For each sampling method, sample a training set and train a model with it;

  2. 2.

For each sampling method, sample a separate evaluation set and evaluate the
trained model on it;

  3. 3.

Make a list of highest error over all the evaluation sets of each trained
model to get an upper bound on its RMSE error;

  4. 4.

Select the model with the lowest error in the list.

Next to studying the impact of the sampling on the performance of the models,
we also propose to study how the models themselves change by using SHAP[7],
which is a model agnostic technique. SHAP gives the relative importance of
each input feature to the output of the model which can be compared when the
model is trained with and without the sampling.

In addition, we take advantage of deep learning models with attention
mechanisms [13] to gain extra insights of what is learnt. As an example of an
attention-based model, TACN[8] is a deep neural network model that provides
the importance of the input time-series across time steps through an attention
mechanism. The change of the patterns shown by the mechanism also provides
insights into the sampling effects on the model.

##  4 Experimental Setup

To give a real-life example of our approach, in this section we present a case
to evaluate it based on a motor temperature prediction dataset. We also
explain the techniques used at each step of the experiments and why they were
chosen.

###  4.1 Motor Temperature Dataset

The dataset used in this experiment is made of sensor measurements extracted
from a steel processing conveyor belt. The prediction target is the
temperature of a bridle motor, which should be forecasted 5 minutes in advance
to allow the operators to take preventive actions before a possible overheat.
The rest of the data consist of properties of the steel strip (i.e. width,
thickness and yield), the speed of the line, the tension applied by the
bridles, the current temperature measurements of the motor, among others. The
sensors are sampled every 10 seconds, and there are in total about 2 million
samples.

In this dataset, we identify the temperature variation as a special property
regarding the prediction target. We analyze the dataset based on this property
and follow the steps of our framework: selecting a weight function, then
selecting the sampling methods, and visualizing the sampling result.

###  4.2 Instantiation of the Framework

Here we describe the choices made at each step of the framework for analyzing
the imbalance of the temperature variation.

####  4.2.1 Step 1 - Temperature Variation Weight Function

The temperature variation is an important property to this forecasting task
since the predictive model must predict accurately when the temperature will
rise. Even if, on average, the model has a satisfying performance, it may
still be inaccurate when predicting higher variation if the dataset is
imbalanced. So for step 1 of the framework, we select the temperature
variation as the weight function, which is modelled by equation 1, using
$\Delta$ as 30 time steps (5 minutes), which is the forecast horizon.

####  4.2.2 Step 2 - Sampling Method Choice

For step 2, we experiment with three sampling methods: SUS with factor 1, SUS
with factor 3 and IHS. Each one under-samples a different amount of low
temperature variation data, creating a different balance, as shown by Figure
2a. SUS with factor 1 and 3 are chosen to compare the effect of the factor in
the proportion of data samples with low and high variation. For the IHS
method, we use the Freedman Diaconis estimator[5] to compute the bin width of
the histogram. 10.000 training data samples are extracted using each method.

####  4.2.3 Step 3 - Predictive Model Choices

In our experiments, we choose a multilayer preceptron[9] as a deep neural
network baseline which has been used in time-series forecasting[1] and three
deep neural networks specialized in temporal data. These specialized
architectures are the long short-term memory (LSTM)[6], a popular recurrent
neural network, the temporal convolutional network (TCN)[2], a sequence-to-
sequence model which has shown promise when trained on a large amount of
data[10] and the temporal attention convolutional network (TACN)[8].

The TACN is an architecture which combines a TCN with an attention
mechanism[13] to achieve interpretable and accurate forecasting. The per-
instance interpretability comes in the form of a vector, equal to the input
window size, which shows the importance of each input step to the forecasting
output. The higher the value of the vector at a specific step, the higher the
contribution of the input value at that step to the final output. By scaling
the vector to the 0-1 range, we can estimate the relative importance among the
input steps. Although this vector is produced per instance, we can draw
conclusions about the generic learned behavior of the model by collecting and
analyzing the vectors from a large number of instances.

For data pre-processing, we extract a window of 5 minutes (or 30 time steps)
for each sample, which is the input for the TCN, LSTM, and TACN models. For
the MLP model, we extract basic features of each sensor such as the mean,
standard deviation, minimal and maximal values for each window. We also keep
the last time step as an additional feature and for later analysis of the
temperature variation case. All the models are evaluated using the root-mean-
square error metric (RMSE).

##  5 Results

In this section we describe the results obtained after applying our framework
starting from step 2. Step 1 is already defined in Section 4.

###  5.1 Step 2 - Comparison of the Sampling Methods

Figure 2a shows the variation distribution after applying the sampling
methods. Without any sampling, the dataset has a strong bias towards samples
with variation close to zero, meaning that the temperature is stable, or
varies very slowly most of the time. SUS with factor 3 give more emphasis to
samples with higher variation, while significantly reducing the number of
samples with lower variation. The sampling using SUS with factor 1, on the
other hand, is more conservative and preserves a considerable amount of
samples with low variation. Finally, IHS gives the best balance across all
values, and is the one which gives the highest proportion of samples in the
extreme of the temperature variation spectrum (above 6 degrees in Figure 2a).

###  5.2 Step 3 - Analysis of the Results

The results of the four models trained and tested with the selected sampling
methods based on temperature variation can be seen in Table 1, with the lower
error per evaluation set highlighted. The effect of the imbalance of the
original data distribution is clearly shown in the ”None” rows, where the
models were trained without sampling. For those lines, the RMSE is much higher
in the SUS 3 column, where there is a smaller number of samples of low
variation, suggesting that the models are biased towards low variation samples
if trained without sampling methods.

On the other hand, these results show that there is a trade-off between
favoring samples with and without temperature variation. Models trained with a
more aggressive kind of sampling, such as SUS with factor 3, have a much
higher error when evaluated on the unsampled data than the models trained with
SUS factor 1, for example. This can be explained by the density difference
between samples with low variation (below 2.5 degrees in Figure 2a), the same
samples that are more common in the ”no sampling” dataset. With our approach,
this trade-off which exhibits non-linear behavior can be estimated, taking
into account the end-user preferences, and it can lead to a re-evaluation of
the sampling method in step 2. Also, together with these metrics, using
insights about the model as described later in this subsection can indicate
the sampling method that leads to the most encompassing, generalizable
patterns learned by the models, thus creating a balance for the performance
across data samples.

Table 1: RMSE results for different sampling methods based on temperature variation. ”None” means that the model was trained or evaluated without using a sampling method. Model | Trained on | Evaluated on
---|---|---
|  | None | SUS - 1 | SUS - 3 | IHS
MLP | None | 1.401 $\pm$ 0.181 | 2.270 $\pm$ 0.177 | 3.611 $\pm$ 0.322 | 2.984 $\pm$ 0.241
| SUS - 1 | 1.704 $\pm$ 0.289 | 2.085 $\pm$ 0.239 | 2.886 $\pm$ 0.269 | 2.495 $\pm$ 0.234
| SUS - 3 | 4.150 $\pm$ 0.635 | 3.089 $\pm$ 0.390 | 2.430 $\pm$ 0.305 | 2.836 $\pm$ 0.32
| IHS | 3.066 $\pm$ 0.401 | 2.728 $\pm$ 0.228 | 2.539 $\pm$ 0.248 | 2.663 $\pm$ 0.217
LSTM | None | 1.032 $\pm$ 0.091 | 1.857 $\pm$ 0.112 | 3.275 $\pm$ 0.129 | 3.275 $\pm$ 0.13
| SUS - 1 | 1.283 $\pm$ 0.153 | 1.469 $\pm$ 0.123 | 2.769 $\pm$ 0.130 | 2.731 $\pm$ 0.094
| SUS - 3 | 3.595 $\pm$ 0.268 | 2.549 $\pm$ 0.128 | 1.464 $\pm$ 0.24 | 2.415 $\pm$ 0.095
| IHS | 2.728 $\pm$ 0.174 | 2.131 $\pm$ 0.093 | 1.863 $\pm$ 0.106 | 2.27 $\pm$ 0.093
TCN | None | 0.871 $\pm$ 0.021 | 1.684 $\pm$ 0.037 | 3.060 $\pm$ 0.068 | 3.142 $\pm$ 0.05
| SUS - 1 | 1.007 $\pm$ 0.079 | 1.462 $\pm$ 0.041 | 2.686 $\pm$ 0.066 | 2.703 $\pm$ 0.063
| SUS - 3 | 3.41 $\pm$ 0.213 | 2.4 $\pm$ 0.091 | 1.592 $\pm$ 0.124 | 2.283 $\pm$ 0.008
| IHS | 2.579 $\pm$ 0.231 | 2.016 $\pm$ 0.091 | 1.845 $\pm$ 0.062 | 2.145 $\pm$ 0.039
TACN | None | 1.171 $\pm$ 0.016 | 2.637 $\pm$ 0.051 | 5.167 $\pm$ 0.1 | 5.064 $\pm$ 0.101
| SUS - 1 | 1.334 $\pm$ 0.242 | 2.077 $\pm$ 0.355 | 3.878 $\pm$ 0.675 | 3.183 $\pm$ 0.694
| SUS - 3 | 3.802 $\pm$ 0.538 | 2.786 $\pm$ 0.428 | 2.36 $\pm$ 0.758 | 2.883 $\pm$ 0.547
| IHS | 3.093 $\pm$ 0.918 | 2.424 $\pm$ 0.608 | 2.430 $\pm$ 0.748 | 2.752 $\pm$ 0.623
Table 2: The maximum error obtained by the TCN model over all the evaluation sets. Each line corresponds to a training set obtained from a different sampling method. Trained on | Max. error | Measured on
---|---|---
No sampling | 3.142 $\pm$ 0.05 | IHS
SUS - 1 | 2.703 $\pm$ 0.063 | IHS
SUS - 3 | 3.41 $\pm$ 0.213 | No sampling
IHS | 2.579 $\pm$ 0.231 | No sampling

Since the results show that the TCN achieves a relatively lower error in all
the evaluation sets, we select it as the best model and follow the heuristic
described in section 3.1. Table 2 shows the maximum error obtained by it over
all the evaluation samples. The two lowest RMSE values reported in that table
are from the TCN model trained with SUS factor 1 and IHS, and the evaluation
sets where they have the highest error are _No sampling_ and _IHS_. By
comparing the performance of the TCN trained with both methods on the
evaluation set without sampling, we can clearly see that the model trained
with SUS with factor 1 has lower spreading of the error (Fig. 2b). On the
other hand, the same comparison on the evaluation set with higher variation
(Fig. 2c) shows that both models have similar error spreading, and the small
advantage of using IHS in this case does not compensate for the increase in
error in the low variation samples. Therefore, we can conclude that SUS with
factor 1 is the sampling method with best performance across samples with low
and high temperature variation.

(a)

(b)

(c)

Figure 2: (a) Comparison of different sampling methods using the temperature
variation as weight function. (b) Prediction error for TCN model trained using
IHS and SUS with factor 1 on ”no sampling” evaluation set and (c) on SUS with
factor 3 evaluation set.

####  5.2.1 Imbalance Effects in DL Models

In the third step of the framework, we also verify how the imbalances affect
the performance and learned patterns of the predictive models, when they are
trained on data with sampling versus unsampled data. To do that, we focus on
the temperature variation property of the dataset, and we measure the SHAP
values of the MLP model, as well as the attention importance values of the
TACN.

To assess how the sampling methods influence the MLP model, we extract its
SHAP values using the SUS 3 evaluation set. We compare both the MLP trained
with SUS 3 and without sampling. Figure 3 shows that both models rely mostly
on the last temperature measurement to make the forecast. This could be
explained by the fact that the last temperature is relatively close to the
predicted temperature, even when there is high variation. One hypothesis for
such fact is that the MLP does not handle the time dependency of the inputs
and, thus has a disadvantage in comparison to other temporal models such as
the TCN or the LSTM.

Figure 3: Comparison of the absolute SHAP values estimated for the MLP model
trained with SUS factor 3 and without sampling. The values are computed based
on inputs from the evaluation set sampled using SUS factor 3. The input
features are computed over a window of 5 minutes. _mt1_ and _mt2_ correspond
to the first and second motors of the bridle set.

To gain insights about the differences in the behavior of the trained TACN
models using the interpretability mechanism, we run inference on the SUS with
factor 3 evaluation set for the models trained on (a) unsampled data and (b)
on the SUS factor 3 train set, and we study the resulting attention pattern
variance.

Figure 4: Unique attention values per input step from sample TACN models
trained on data with (a) no sampling and (b) SUS with factor 3

In order to quantify this variance, we enumerate for both models the unique
learned attention values for each input time step across all test samples,
rounded to the second decimal, and present the result on Fig. 4. For the model
trained on unsampled data, the unique values for each position are at most 3,
while for the SUS model they are between 30 and 50. The above observations
lead us to the following conclusions: The model trained on the unsampled data
has learned a high reliance on the last value and limited number of patterns,
which serves well in minimizing the error for the majority of the samples but
results in low performance on the large variation samples. In contrast, the
model trained on the SUS data is forced to learn a larger variety of patterns
to accommodate for this target variation.

##  6 Conclusion

We presented a framework to analyze imbalanced time-series forecasting
problems and to train and evaluate ML models taking into account important
properties. To our knowledge, this is the first framework which provides clear
steps to help practitioners to select and compare different sampling methods
and predictive models for such problems. It is put into practice to forecast
the temperature of a motor in a steel processing conveyor belt, based on data
extracted from a real-world industrial process and is validated in cooperation
with domain experts. The problem analysis is made through the lens of the
temperature variation property. We study the dataset using three different
sampling methods and we train four different DL models to evaluate and compare
the effectiveness of each combination of sampling and model. We also show the
imbalance of the temperature variation and how it changes the models’
predictions when they are trained with different proportions of samples with
high temperature variation. Finally, we use SHAP values and the TACN model’s
attention mechanism to show the effect of low temperature variation in the
dataset on the forecast models, inducing them to rely mostly on the last
observed temperature.

As future work, our framework could be put into practice to analyze new time-
series prediction tasks, combining with more sampling techniques. The
framework can be extended with new weight functions, making it suitable for an
even wider range of tasks. In addition, our results point out to a possible
relationship between the prediction error and the distribution of the training
data which might be worth investigating.

## Acknowledgements

This work has been conducted as part of the Just in Time Maintenance project
funded by the European Fund for Regional Development. We also thank Tata Steel
Europe for providing the data and technical expertise required for our
experiments.

## References

  * [1] Ahmed, N.K., Atiya, A.F., Gayar, N.E., El-Shishiny, H.: An empirical comparison of machine learning models for time series forecasting. Econometric Reviews 29(5-6), 594–621 (2010)
  * [2] Bai, S., Kolter, J.Z., Koltun, V.: An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv:1803.01271 [cs] (2018)
  * [3] Branco, P., Torgo, L., Ribeiro, R.P.: A Survey of Predictive Modeling on Imbalanced Domains. ACM Computing Surveys 49(2), 1–50 (2016)
  * [4] Branco, P., Torgo, L., Ribeiro, R.P.: Pre-processing approaches for imbalanced distributions in regression. Neurocomputing 343, 76–99 (2019). https://doi.org/10.1016/j.neucom.2018.11.100
  * [5] Freedman, D., Diaconis, P.: On the histogram as a density estimator: L 2 theory. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 57(4), 453–476 (1981)
  * [6] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
  * [7] Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30, pp. 4765–4774. Curran Associates, Inc. (2017)
  * [8] Pantiskas, L., Verstoep, K., Bal, H.: Interpretable multivariate time series forecasting with temporal attention convolutional neural networks. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI). pp. 1687–1694 (2020). https://doi.org/10.1109/SSCI47803.2020.9308570
  * [9] Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review 65(6), 386 (1958)
  * [10] Silvestrin, L.P., Hoogendoorn, M., Koole, G.: A Comparative Study of State-of-the-Art Machine Learning Algorithms for Predictive Maintenance. In: 2019 IEEE Symposium Series on Computational Intelligence (SSCI). pp. 760–767. IEEE (2019)
  * [11] Snieder, E., Abogadil, K., Khan, U.T.: Resampling and ensemble techniques for improving ann-based high streamflow forecast accuracy. Hydrology and Earth System Sciences Discussions 2020, 1–35 (2020)
  * [12] Torres, J.F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., Troncoso, A.: Deep learning for time series forecasting: A survey. Big Data 9(1), 3–21 (2021). https://doi.org/10.1089/big.2020.0159
  * [13] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems Systems (NIPS). pp. 5998–6008 (2017)


# On the Importance of Feature Separability
in Predicting Out-Of-Distribution Error

Renchunzi Xie  Hongxin Wei  Yuzhou Cao  Lei Feng  Bo An

###### Abstract

Estimating the generalization performance is practically challenging on out-
of-distribution (OOD) data without ground truth labels. While previous methods
emphasize the connection between distribution difference and OOD accuracy, we
show that a large domain gap not necessarily leads to a low test accuracy. In
this paper, we investigate this problem from the perspective of feature
separability, and propose a dataset-level score based upon feature dispersion
to estimate the test accuracy under distribution shift. Our method is inspired
by desirable properties of features in representation learning: high inter-
class dispersion and high intra-class compactness. Our analysis shows that
inter-class dispersion is strongly correlated with the model accuracy, while
intra-class compactness does not reflect the generalization performance on OOD
data. Extensive experiments demonstrate the superiority of our method in both
prediction performance and computational efficiency.

Machine Learning, ICML

##  1 Introduction

Machine learning techniques deployed in the open world often struggle with
distribution shifts, where the test data are not drawn from the training
distribution. Such issues can substantially degrade the test accuracy, and the
generalization performance of a trained model may vary significantly on
different shifted datasets (Quinonero-Candela et al., 2008; Koh et al., 2021).
This gives rise to the importance of estimating out-of-distribution (OOD)
errors for AI Safety (Deng & Zheng, 2021). However, it is prohibitively
expensive or unrealistic to collect large-scale labeled examples for each
shifted testing distribution encountered in the wild. Subsequently, predicting
OOD error becomes a challenging task without access to ground truth labels.

In the literature, a popular direction in predicting OOD error is to utilize
the model output on the shifted dataset (Jiang et al., 2021; Guillory et al.,
2021; Garg et al., 2022), which heavily relies on the model calibration. Yet,
machine learning models generally suffer from the overconfidence issue even
for OOD inputs (Wei et al., 2022), leading to suboptimal performance in
estimating OOD performance. Many prior works turned to measuring the
distribution difference between training and OOD test set, due to the
conventional wisdom that a higher distribution shift normally leads to lower
OOD accuracy. AutoEval (Deng & Zheng, 2021) applies Fréchet Distance to
calculate the distribution distance for model evaluation under distribution
shift. A recent work (Yu et al., 2022) introduces Projection Norm (ProjNorm)
metric to measure the distribution discrepancy in network parameters. However,
we find that the connection between distribution distance and generalization
performance does not always hold, making these surrogate methods to be
questionable. This motivates our method, which directly estimates OOD accuracy
based on the feature properties of test instances.

In this work, we show that feature separability maintains a strong connection
with test accuracy even when it encounters distribution shifts. To quantify
the feature separability, we introduce a simple dataset-level statistic,
Dispersion Score, which gauges the inter-class divergence from feature
representations, i.e., outputs of feature extractor. Our method is motivated
by the desirable properties of embeddings in representation learning (Bengio
et al., 2013). To achieve high accuracy, we generally desire embeddings where
different classes are relatively far apart (i.e., high inter-class
dispersion), and samples in each class form a compact cluster (i.e., high
intra-class compactness). Surprisingly, our analysis shows that intra-class
compactness does not reflect the generalization performance, while inter-class
dispersion is strongly correlated with the model accuracy on OOD data.

Extensive experiments demonstrate the superiority of Dispersion Score over
existing methods for estimating OOD error. First, our method dramatically
outperforms existing training-free methods in evaluating model performance on
OOD data. For example, our method leads to an increase of the $R^{2}$ from
0.847 to 0.970 on TinyImageNet-C – a 14.5$\%$ of relative improvement.
Compared to the recent ProjNorm method (Yu et al., 2022), our method not only
achieves superior performance by a meaningful margin, but also maintains huge
advantages in computational efficiency and sample efficiency. For example,
using CIFAR-10C dataset as OOD data, Dispersion Score achieves an $R^{2}$ of
0.972, outperforming that of ProjNorm (i.e., 0.947), while our approach only
takes around 3% of the time consumed by ProjNorm.

Overall, using Dispersion Score achieves strong performance in OOD error
estimation with high computational efficiency. Our method can be easily
adopted in practice. It is straightforward to implement with deep learning
models and does not require access to training data. Thus our method is
compatible with large-scale models that are trained on billions of images.

We summarize our main contribution as follows:

  1. 1.

We find that the correlation between distribution distance and generalization
performance does not always hold, downgrading the reliability of existing
distance-based methods.

  2. 2.

We propose a simple dataset-level score that gauges the inter-class dispersion
from feature representations, i.e., outputs of feature extractor. Besides, we
show that intra-class compactness does not reflect the generalization
performance under distribution shifts (Section 5).

  3. 3.

We conduct extensive evaluations to show the superiority of the Dispersion
Score in both prediction performance and computational efficiency. Our
analysis shows that Dispersion Score is more robust to various data conditions
in OOD data, such as limited sample size and class imbalance.

##  2 Problem Setup and Motivation

###  2.1 Preliminaries: OOD performance estimation

#### Setup

In this work, we consider multi-class classification task with $k$ classes. We
denote the input space as $\mathcal{X}$ and the label space with $k$ classes
as $\mathcal{Y}=\\{1,\ldots,k\\}$. We assume there is a training dataset
$\mathcal{D}=\\{\boldsymbol{x}_{i},y_{i}\\}^{n}_{i=1}$, where the $n$ data
points are sampled _i.i.d._ from a joint data distribution
$\mathcal{P}_{\mathcal{X}\mathcal{Y}}$. In the training stage, we learn a
neural network $f:\mathcal{X}\rightarrow\mathbb{R}^{k}$ with trainable
parameter ${\theta}\in\mathbb{R}^{p}$ on $\mathcal{D}$.

In particular, the neural network can be viewed as a combination of a feature
extractor $f_{g}$ and a classifier $f_{\omega}$, where $g$ and $\omega$ denote
the parameters of the corresponding parts, respectively. The feature extractor
$f_{g}$ is a function that maps instances to features
$f_{g}:\mathcal{X}\rightarrow\mathcal{Z}$, where $\mathcal{Z}$ denotes the
feature space. We denote by $\boldsymbol{z}_{i}$ the learned feature of
instance $\boldsymbol{x}_{i}$: $\boldsymbol{z}_{i}=f_{g}(\boldsymbol{x}_{i})$.
The classifier $f_{\omega}$ is a function from the feature space $\mathcal{Z}$
to $\mathbb{R}^{k}$, which outputs the final predictions. A trained model can
be obtained by minimizing the following expected risk:

$\displaystyle\mathcal{R}_{\mathcal{L}}(f)$
$\displaystyle=\mathbb{E}_{(\boldsymbol{x},y)\sim\mathcal{P}_{\mathcal{X}\mathcal{Y}}}\left[\mathcal{L}\left(f(\boldsymbol{x};{\theta}),y\right)\right]$
$\displaystyle=\mathbb{E}_{(\boldsymbol{x},y)\sim\mathcal{P}_{\mathcal{X}\mathcal{Y}}}\left[\mathcal{L}\left(f_{\omega}(f_{g}(\boldsymbol{x})),y\right)\right]$

(a) Fréchet

(b) MMD

Figure 1: Distribution Gap Vs. OOD accuracy on CIFAR10-C via (a) Fréchet
distance (Dowson & Landau, 1982) and (b) MMD (Gretton et al., 2006). All
colors indicate 14 different types of corruption. The test accuracy varies
significantly on shifted datasets with the same distribution distances.

#### Problem statement

At test time, we generally expect that the test data are drawn from the same
distribution as the training dataset. However, distribution shifts usually
happen in reality and even simple shifts can lead to large drops in
performance, which makes it unreliable in safety-critical applications. Thus,
our goal is to estimate how a trained model might perform on the shifted data
without labels, i.e., unlabeled out-of-distribution (OOD) data.

Assume that $\tilde{\mathcal{D}}=\\{\tilde{\boldsymbol{x}}_{i}\\}^{m}_{i=1}$
be the OOD test dataset and $\\{\tilde{y}_{i}\\}^{M}_{i=1}$ be the
corresponding unobserved labels. For a certain test instance
$\tilde{\boldsymbol{x}}_{i}$, we obtain the predicted labels of a trained
model by $\tilde{y}^{\prime}_{i}=\arg\max
f_{\omega}(\tilde{\boldsymbol{x}}_{i})$. Then the ground-truth test error on
OOD data can be formally defined as:

$\operatorname{Err}(\tilde{\mathcal{D}})=\frac{1}{m}\sum_{i=1}^{m}\mathds{1}(\tilde{y}^{\prime}_{i}\neq\widetilde{y}_{i}),$
(1)

To estimate the real OOD error, the key challenge is to formulate a score
$S(\tilde{\mathcal{D}})$ that is strongly correlated with the test error
across diverse distribution shifts without utilization of corresponding test
labels. With such scores, a simple linear regression model can be learned to
estimate the test error on shifted datasets, following the commonly-used
scheme (Deng & Zheng, 2021; Yu et al., 2022).

While those output-based approaches suffer from the overconfidence issue
(Hendrycks & Gimpel, 2016; Guillory et al., 2021), other methods primarily
rely on the distribution distance between training data $\mathcal{D}$ and test
data $\tilde{\mathcal{D}}$, with the intuition that the distribution gap
impacts classification accuracy. In the following, we motivate our method by
analyzing the failure of those methods based on distribution distance.

###  2.2 The failure of distribution distance

In the literature, distribution discrepancy has been considered as a key
metric to predict the generalization performance of the model on unseen
datasets (Deng & Zheng, 2021; Tzeng et al., 2017; Gao & Kleywegt, 2022; Sinha
et al., 2017; Yu et al., 2022). AutoEval (Deng & Zheng, 2021) estimates model
performance by quantifying the domain gap in the feature space:

$S(\mathcal{D},\tilde{\mathcal{D}})=d(\mathcal{D},\tilde{\mathcal{D}}),$

where $d(\cdot)$ denotes the distance function, such as Fréchet Distance
(Dowson & Landau, 1982) or maximum mean discrepancy (MMD) (Gretton et al.,
2006).

ProjNorm measures the distribution gap with the Euclidean distance in the
parameter space:

$S(\mathcal{D},\tilde{\mathcal{D}})=\|\theta-\tilde{\theta}\|_{2},$

where $\theta$ and $\tilde{\theta}$ denote the model parameters fine-tuned on
training data $\mathcal{D}$ and OOD data $\tilde{\mathcal{D}}$, respectively.

The underlying assumption is inherited from the conventional wisdom in domain
adaptation, where a representation function that minimizes domain difference
leads to higher accuracy in target domain (Tzeng et al., 2014; Ganin &
Lempitsky, 2015; Tzeng et al., 2017). Yet, the relationship between
distribution distance and test accuracy remains controversial. In other words,
given a fixed model, does a larger distribution distance always lead to a
lower test accuracy?

To verify the correlation between distribution distance and model performance,
we compare the model performance on different OOD test sets of CIFAR-10C,
using the ResNet-50 model trained on CIFAR-10. For each OOD test set, we
calculate the distribution distances in the feature space $\mathcal{Z}$ via
Fréchet distance (Dowson & Landau, 1982) and MMD (Gretton et al., 2006),
respectively. Figure 1 presents the classification accuracy versus the
distribution distance. The results show that, on different test datasets with
similar distances to the train data in the feature space. the performance of
the trained model varies significantly with both the two distance metrics. For
example, calculated by Fréchet distance, the distribution gap varies only a
small margin from 223.36 to 224.04, but the true accuracy experiences a large
drop from 61.06% to 46.99%. Similar to MMD distance, when the distribution gap
changes from 87.63 to 88.35, the OOD accuracy drops from 66.53% to 47.73%. The
high variability in the model performance reveals that, the distribution
difference is not a reliable surrogate for generalization performance under
distribution shifts.

Figure 2: A toy example for illustrating the failure of distribution distance.
Blue and Orange denote the two classes. The test accuracy will not change
regardless of the shift distance of test data along the y-axis.

#### A toy example.

To provide a straightforward view, we show in Figure 2 a toy example for
illustrating the failure of distribution distance. Consider a binary
classification with the data generating from a mixture of two Gaussian
distributions $\mathcal{P}_{\mathcal{X}\mathcal{Y}}$. In particular, the label
Y is either positive ($+1$) or negative ($-1$) with equal probability (i.e.,
0.5). Without loss of generality, we assume there is a trained model
$f(\boldsymbol{x})=\operatorname{sign}(\boldsymbol{x}-\kappa)$, where $\kappa$
denotes a constant between class centers. At test time, we consider a specific
OOD dataset that shifts along the y-axis. Naturally, we find that the test
accuracy will not change regardless of the shift distance of test data. It
implies that, distribution distance may fail to indicate the model
performance. We proceed by introducing our method, exploring a new perspective
in the feature space.

##  3 Propose Method

Figure 3: t-SNE visualization of feature representation on training and OOD
test datasets (CIFAR-10C) with contrast corruption under different severity
levels. The first left figure shows feature representation of the training
dataset, while the rest of three figures illustrate those of the OOD datasets.
From this figure, we observe that different clusters tends to be more
separated as the corruption severity level gets smaller.

In this section, we first introduce the intuition of our method with a natural
example. Next, we characterize and provide quantitative measure on the the
desirable properties of feature representations for predicting OOD error. We
then present the advantages of our proposed method.

###  3.1 An empirical study of feature separability

In representation learning, it is desirable to learn a separable feature
space, where different classes are far away from each other and samples in
each class form a compact cluster. Therefore, separability is viewed as an
important characteristic to measure the feature quality. For example, one may
train a nearest neighbor classifier over the learned representation using
labeled data and regard its performance as the indicator. In this paper, we
investigate how to utilize the characteristic of separability for estimating
the model performance under distribution shifts, without access to ground-
truth labels.

In Figure 3, we present a t-SNE visualization of the representation for
training and test data. In particular, we compare the separability of the
learned representation on shifted datasets with various severity. While the
feature representation of training data exhibits well-separated clusters,
those of the shifted datasets are more difficult to differentiate and the
cluster gaps are well correlated with the corruption severity, as well as the
ground-truth accuracy. It implies that, a model with high classification
accuracy is usually along with a well-separated feature distribution, and vice
versa.

###  3.2 Dispersion score

In our previous analysis, we show a strong correlation between the test
accuracy and feature separability under different distribution shifts. To
quantify the separability in the feature space $\mathcal{Z}$, we introduce
_Dispersion score_ that measures the inter-class margin without annotation
information.

First, we allocate OOD instances $\\{\tilde{\boldsymbol{x}}\\}_{i=1}^{m}$ into
different clusters $j$ based on the model predictions, i.e., their pseudo
labels from the trained classifier $f_{\omega}$:

$j=\tilde{y}^{\prime}_{i}=\arg\max f_{\omega}(\boldsymbol{z}_{i}).$

With these clusters, we compute the _Dispersion score_ by the average
distances between each cluster centroid $\tilde{\boldsymbol{\mu}}_{j}$ and the
center of all features $\bar{\boldsymbol{\mu}}$, weighted by the sample size
of each cluster $m_{j}$. Formally, the _Dispersion score_ is defined as:

$S(\tilde{\mathcal{D}})=\frac{1}{k-1}\sum_{j=1}^{k}m_{j}\cdot\varphi(\bar{\boldsymbol{\mu}},\tilde{\boldsymbol{\mu}}_{j})$

where $k-1$ is the degree of freedom and $\varphi$ denotes the distance
function. With the weight $m_{j}$, the induced score receives a stronger
influence from those larger clusters, i.e., the majority classes. This enables
our method to be more robust to the long-tailed issue, which naturally arises
in the unlabeled OOD data. In subsection 4.3, we explicitly show the advantage
of our method in the class-imbalanced case and analyze the importance of the
weight in Appendix A.

In particular, we use square Euclidean distances to calculate the distance in
the feature space $\mathcal{Z}$. So it converts to:

$S(\tilde{\mathcal{D}})=\log\frac{\sum_{j=1}^{k}m_{j}\cdot\|\bar{\boldsymbol{\mu}}-\tilde{\boldsymbol{\mu}}_{j}\|_{2}^{2}}{k-1}$
(2)

Following the common practice (Jiang et al., 2019), we adopt a log transform
on the final score, which corresponds to multiplicative combination of class
statistics.

Algorithm 1 OOD Error Estimation via Dispersion Score

Input: OOD test dataset $\tilde{\mathcal{D}}=\\{\tilde{x}_{i}\\}_{i=1}^{m}$, a
trained model $f$ composed of a feature extractor $f_{g}$ and a classifier
$f_{\omega}$

Output: The dispersion score

for each OOD instance $\tilde{x}_{i}$ do

Obtain feature representation via
$\tilde{\boldsymbol{z}}_{i}=f_{g}(\tilde{\boldsymbol{x}_{i}})$.

Obtain pseudo labels via $\tilde{y}^{\prime}_{i}=\arg\max
f_{\omega}(\boldsymbol{z}_{i})$

end for

Calculate cluster centroids $\\{\tilde{\boldsymbol{\mu}}_{j}\\}_{j=1}^{k}$
according to pseudo labels $\tilde{y}^{\prime}_{i}$, with
$\tilde{\boldsymbol{\mu}}_{j}=\frac{1}{m_{j}}\sum_{i=1}^{m_{j}}\boldsymbol{z}_{i}\cdot\mathbbm{1}\\{\tilde{y}^{\prime}_{i}=j\\}$

Calculate the feature center of all instances by
$\boldsymbol{\mu}=\frac{1}{m}\sum_{i=1}^{m}\boldsymbol{z}_{i}$

Calculate Dispersion Score $S(\tilde{\mathcal{D}})$ via Equation (2)

Table 1: Performance comparison of training free approaches on CIFAR-10,
CIFAR-100 and TinyImageNet, where $R^{2}$ refers to coefficients of
determination, and $\rho$ refers to Spearson correlation coefficients (higher
is better). The best results are highlighted in bold.

Dataset | Network | Rotation | ConfScore | Entropy | AgreeScore | ATC | Fréchet | Ours
---|---|---|---|---|---|---|---|---
$R^{2}$ | $\rho$ | $R^{2}$ | $\rho$ | $R^{2}$ | $\rho$ | $R^{2}$ | $\rho$ | $R^{2}$ | $\rho$ | $R^{2}$ | $\rho$ | $R^{2}$ | $\rho$
CIFAR 10 | ResNet18 | 0.822 | 0.951 | 0.869 | 0.985 | 0.899 | 0.987 | 0.663 | 0.929 | 0.884 | 0.985 | 0.950 | 0.971 | 0.968 | 0.990
ResNet50 | 0.835 | 0.961 | 0.935 | 0.993 | 0.945 | 0.994 | 0.835 | 0.985 | 0.946 | 0.994 | 0.858 | 0.964 | 0.987 | 0.990
WRN-50-2 | 0.862 | 0.976 | 0.943 | 0.994 | 0.942 | 0.994 | 0.856 | 0.986 | 0.947 | 0.994 | 0.814 | 0.973 | 0.962 | 0.988
Average | 0.840 | 0.963 | 0.916 | 0.991 | 0.930 | 0.992 | 0.785 | 0.967 | 0.926 | 0.991 | 0.874 | 0.970 | 0.972 | 0.990
CIFAR 100 | ResNet18 | 0.860 | 0.936 | 0.916 | 0.985 | 0.891 | 0.979 | 0.902 | 0.973 | 0.938 | 0.986 | 0.888 | 0.968 | 0.952 | 0.988
ResNet50 | 0.908 | 0.962 | 0.919 | 0.984 | 0.884 | 0.977 | 0.922 | 0.982 | 0.921 | 0.984 | 0.837 | 0.972 | 0.951 | 0.985
WRN-50-2 | 0.924 | 0.970 | 0.971 | 0.984 | 0.968 | 0.981 | 0.955 | 0.977 | 0.978 | 0.993 | 0.865 | 0.987 | 0.980 | 0.991
Average | 0.898 | 0.956 | 0.936 | 0.987 | 0.915 | 0.983 | 0.927 | 0.982 | 0.946 | 0.988 | 0.864 | 0.976 | 0.962 | 0.988
TinyImageNet | ResNet18 | 0.786 | 0.946 | 0.670 | 0.869 | 0.592 | 0.842 | 0.561 | 0.853 | 0.751 | 0.945 | 0.826 | 0.970 | 0.966 | 0.986
ResNet50 | 0.786 | 0.947 | 0.670 | 0.869 | 0.651 | 0.892 | 0.560 | 0.853 | 0.751 | 0.945 | 0.826 | 0.971 | 0.977 | 0.986
WRNt-50-2 | 0.878 | 0.967 | 0.757 | 0.951 | 0.704 | 0.935 | 0.654 | 0.904 | 0.635 | 0.897 | 0.884 | 0.984 | 0.968 | 0.986
Average | 0.805 | 0.959 | 0.727 | 0.920 | 0.650 | 0.890 | 0.599 | 0.878 | 0.693 | 0.921 | 0.847 | 0.976 | 0.970 | 0.987

We summarize our approach in Algorithm 1. Notably, the Dispersion score
derived from the feature separability offers several compelling advantages:

  1. 1.

Training data free. The calculation procedure does not rely on the information
of training data. Thus, our method is compatible with large-scale models that
are trained on billions of images.

  2. 2.

Easy-to-use. The computation of the Dispersion score only does forward
propagation for each test instance once and does not require extra
hyperparameter, training a new model, or updating the model parameters.
Therefore, our method is easy to implement in real-world tasks and
computational efficient, as demonstrated in Tables 2 and 3.

  3. 3.

Strong flexibility in OOD data. Previous state-of-the-art methods, like
ProjNorm (Yu et al., 2022), usually requires a large amount of OOD data for
predicting the prediction performance. Besides, the class distribution of
unlabeled OOD datasets might be severely imbalanced, which makes it
challenging to estimate the desired test accuracy on balanced data. In
Subsection 4.3, we will show the Dispersion score derived from the feature
separability exhibits stronger flexibility and generality in sample size and
class distribution.

##  4 Experiments

In this subsection, we first compare the proposed score to existing training-
free methods. Then, we provide an extensive comparison between our method and
recent state-of-the-art method – ProjNorm, in both prediction performance and
computational efficiency. We then robustness of our method under more general
conditions of OOD data.

###  4.1 Experimental Setup

Train datasets. During training, we train models on the CIFAR-10, CIFAR-100
(Krizhevsky et al., 2009) and TinyImageNet (Le & Yang, 2015) datasets.
Specifically, the train data of CIFAR-10 and CIFAR-100 contain 50,000 training
images, which are allocated to 10 and 100 classes, respectively. The
TinyImageNet dataset contains 100,000 64 $\times$ 64 training images, with 200
classes.

Out-of-distribution (OOD) datasets. To evaluate effectiveness of the proposed
method on predicting OOD error at test time, we use CIFAR-10C, CIFAR-100C,
which span 19 types of corruption with 5 severity levels. For the testing of
TinyImageNet, we use TinyImageNet-C that spans 15 types of corruption with 5
severity levels (Hendrycks & Dietterich, 2019) as OOD dataset. All the
datasets with certain corruption and severity contain 10,000 images, which are
evenly distributed in the classes.

Figure 4: OOD error prediction versus True OOD error on CIFAR-10 with
ResNet50. We compare the performance of Dispersion Score with that of ProjNorm
and Fréchet via scatter plots. Each point represents one dataset under certain
corruption and certain severity, where different shapes represent different
types of corruption, and darker color represents the higher severity level.

Evaluation metrics. To measure the linear relationship between OOD error and
designed scores, we use coefficients of determination ($R^{2}$) and Spearson
correlation coefficients ($\rho$) as the evaluation metrics. For the
comparison of computational efficiency, we calculate the average evaluation
time ($T$) for each test set with certain corruption and severity.

Training details. During the training process, we train ResNet18, ResNet50 (He
et al., 2016) and WRN-50-2 (Zagoruyko & Komodakis, 2016) on CIFAR-10,
CIFAR-100 (Krizhevsky et al., 2009) and TinyImageNet (Le & Yang, 2015) with
20, 50 and 50 epochs, respectively. We use SGD with the learning rate of
$10^{-3}$, cosine learning rate decay (Loshchilov & Hutter, 2016), a momentum
of 0.9 and a batch size of 128 to train the model.

The compared methods. We consider 7 existing methods as benchmarks for
predicting OOD error: Rotation Prediction (Rotation) (Deng et al., 2021),
Averaged Confidence (ConfScore) (Hendrycks & Gimpel, 2016), Entropy (Guillory
et al., 2021), Agreement Score (AgreeScore) (Jiang et al., 2021), Averaged
Threshold Confidence (ATC) (Garg et al., 2022), AutoEval (Fréchet) (Deng &
Zheng, 2021), and ProjNorm (Yu et al., 2022). Rotation and AgreeScore predict
OOD error from the view of unsupervised loss constructed by generating
rotating instances and measuring output agreement of two independent models,
respectively. ConfScore, Entropy, and ATC formulate scores by model
predictions. ProjNorm measures the parameter-level difference between the
models fine-tuned on train and OOD test data respectively, while Fréchet
quantifies the distribution difference in the feature space between the
training and test datasets.

Table 2: Performance comparison between ProjNorm (Yu et al., 2022) and our
Dispersion score on CIFAR-10, CIFAR-100 and TinyImageNet, where $R^{2}$ refers
to coefficients of determination, $\rho$ refers to Spearson correlation
coefficients (higher is better), and $T$ refers to average evaluation time
(lower is better). The best results are highlighted in bold.

Dataset | Network | ProjNorm | Ours
---|---|---|---
$R^{2}$ | $\rho$ | $T$ | $R^{2}$ | $\rho$ | $T$
CIFAR 10 | ResNet18 | 0.936 | 0.982 | 179.616 | 0.968 | 0.990 | 10.980
ResNet50 | 0.944 | 0.989 | 266.099 | 0.987 | 0.990 | 11.259
WRN-50-2 | 0.961 | 0.989 | 575.888 | 0.962 | 0.988 | 11.017
Average | 0.947 | 0.987 | 326.201 | 0.972 | 0.990 | 11.085
CIFAR 100 | ResNet18 | 0.979 | 0.980 | 180.453 | 0.952 | 0.988 | 6.997
ResNet50 | 0.988 | 0.991 | 262.831 | 0.953 | 0.985 | 11.138
WRN-50-2 | 0.990 | 0.991 | 605.616 | 0.980 | 0.991 | 12.353
Average | 0.985 | 0.987 | 349.63 | 0.962 | 0.988 | 10.163
TinyImageNet | ResNet18 | 0.970 | 0.981 | 182.127 | 0.966 | 0.986 | 13.938
ResNet50 | 0.979 | 0.987 | 264.651 | 0.977 | 0.990 | 7.039
WRN-50-2 | 0.965 | 0.983 | 590.597 | 0.968 | 0.986 | 11.235
Average | 0.972 | 0.984 | 345.792 | 0.970 | 0.987 | 10.737

###  4.2 Results

#### Can Dispersion score outperform existing training-free approaches?

In Table 1, we present the performance of OOD error estimation on three model
architectures and three datasets. We find that Dispersion Score dramatically
outperforms existing training-free methods. For example, averaged across three
architectures on TinyImageNet, our method leads to an increase of the $R^{2}$
from 0.847 to 0.970 – a 14.5$\%$ of relative improvement. In addition,
Dispersion Score achieves consistently high performance over the three
datasets with a $R^{2}$ higher than 0.950, while scores of other approaches
such as Rotation varying from 0.787 to 0.924 are not stable. We observe a
similar phenomenon on $\rho$, where the Entropy method achieves performance
that is ranging from 0.842 to 0.994, while the performance of our method
fluctuates around 0.988.

Table 3: Summary of prediction performance on Imbalanced CIFAR-10C and
CIFAR-10C0, where $R^{2}$ refers to coefficients of determination, $\rho$
refers to Spearson correlation coefficients (higher is better), and $T$ refers
to average evaluation time (lower is better). The best results are highlighted
in bold.

Dataset | Network | ProjNorm | Dispersion
---|---|---|---
$R^{2}$ | $\rho$ | $T$ | $R^{2}$ | $\rho$ | $T$
CIFAR 10 | ResNet18 | 0.799 | 0.968 | 204.900 | 0.959 | 0.982 | 2.076
ResNet50 | 0.897 | 0.980 | 430.406 | 0.968 | 0.982 | 3.295
WRN-50-2 | 0.922 | 0.978 | 561.611 | 0.932 | 0.978 | 2.970
Average | 0.873 | 0.973 | 398.972 | 0.953 | 0.980 | 2.780
CIFAR 100 | ResNet18 | 0.886 | 0.968 | 210.05 | 0.941 | 0.982 | 1.864
ResNet50 | 0.980 | 0.988 | 433.860 | 0.956 | 0.982 | 2.974
WRN-50-2 | 0.978 | 0.982 | 768.883 | 0.986 | 0.994 | 3.242
Average | 0.948 | 0.980 | 470.931 | 0.961 | 0.986 | 2.693

#### Dispersion score is superior to ProjNorm.

In Table 2, we compare our method to the recent state-of-the-art method –
ProjNrom (Yu et al., 2022) in both prediction performance and computational
efficiency. The results illustrate that Dispersion Score could improve the
prediction performance over ProjNorm with a meaningful margin. For example,
with trained models on CIFAR-10 dataset, using Dispersion score achieves a
$R^{2}$ of 0.953, much higher than the average performance of ProjNorm as
$0.873$. On CIFAR-100, our method also achieves comparable (slightly better)
performance with ProjNorm (0.961 vs. 0.948). Besides, Dispersion score obtains
huge advantages to ProjNorm in computational efficiency. Using WRN-50-2,
ProjNorm takes an average of 575.888 seconds for estimating the performance of
each OOD test dataset, while our method only requires 11.085 seconds. Since
the computation of Dispersion score does not needs to update model parameters
or utilize the training data, our method enables to predict OOD error with
large-scale models trained on billions of images.

To further analyze the advantage of our method, we present in Figure 4 the
scatter plots for Fréchet, ProjNorm and Dispersion Score on CIFAR-10C with
ResNet50. From the figure, we find that Dispersion Score estimates OOD errors
linearly w.r.t. true OOD errors in all cases. In contrast, we observe that
those methods based on distribution difference tends to fail when the
classification error is high. This phenomenon clearly demonstrates the
reliable and superior performance of the Dispersion score in predicting
generalization performance under distribution shifts.

###  4.3 Flexibility in OOD data

In previous analysis, we show that Dispersion score can outperform existing
methods on standard benchmarks, where the OOD test datasets contain sufficient
instances and have a balanced class distribution. In reality, the unlabeled
OOD data are naturally imperfect, which may limit the performance of
predicting OOD error. In this section, we verify the effectiveness of our
method with long-tailed data and small data, compared to ProjNorm (Yu et al.,
2022).

(a) ResNet18

(b) ResNet-50

Figure 5: Prediction performance $R^{2}$ vs. sample size of OOD data (subsets
of CIFAR-10C) with (a) ResNet18 and (b) ResNet50.

#### Class imbalance.

We first evaluate the prediction performance under class imbalanced setting.
In particular, given a trained model, we aim to estimate its balanced accuracy
under distribution shift while we only have access to a long-tailed test data,
where a few classes (majority classes) occupy most of the data and most
classes (minority classes) are under-represented. We conduct experiments on
CIFAR-10C and CIFAR-100C with imbalance rate 100.

Our results in Table 3 show that Dispersion Score achieves better performance
than ProjNorm (Yu et al., 2022) under class imbalance. For example, our
approach achieves an average $R^{2}$ of 0.953 on CIFAR-10C with 2.780 seconds,
while ProjNorm obtains an $R^{2}$ of 0.873 and takes 398.972 seconds. In
addition, the performance of our method is more stable across different model
architectures and datasets with the $R^{2}$ ranging from 0.932 to 0.986 than
ProjNorm (0.799 - 0.980). Overall, Dispersion score maintains reliable and
superior performance even when the OOD test set are class imbalanced.

Sampling efficiency. During deployment, it can be challenging to collect
instances under a specific distribution shift. However, existing state-of-the-
art methods (Yu et al., 2022) normally require a sufficiently large test
dataset to achieve meaningful results in predicting OOD error. Here, we
further validate the sampling efficiency of Dispersion score, compared with
ProjNorm.

Figure 5 presents the performance comparison between Dispersion Score and
ProjNorm on subsets of CIFAR10C with various sample sizes. The results show
that our method can achieve excellent performance even when only 50 examples
are available in the OOD data. In contrast, the performance of ProjNorm
decreases sharply with the decrease of sample size. The phenomenon shows that
Dispersion score is more efficient in exploiting the information of OOD
instances.

(a) Compare with K-means

(b) Compactness vs. test error

Figure 6: (a) Comparison of Dispersion score and the K-means variant on three
datasets. (b) Compactness vs. test error on CIFAR-10C with ResNet50. Each
point represents a dataset with certain corruption, where shapes represent
corruption types and a darker color represents a higher severity level.

##  5 Discussion

#### K-means vs. Pseudo labels.

While our Dispersion score derived from pseudo labels has demonstrated strong
promise, a question arises: _can a similar effect be achieved by alternative
clustering methods?_ In this ablation, we show that labels obtained by K-means
(Lloyd, 1982; MacQueen, 1967) does not achieve comparable performance with
pseudo labels obtained from the trained classifier. In particular, we allocate
instances into clusters by using K-means instead of pseudo labels from
classifier. We provide an analysis of using ground-truth labels in Appendix B.

We present the performance comparison of our method and the variant of K-means
in Figure 6(a). The results show that our Dispersion score performs better
than the K-means variant and the gap is enlarged with more classes. On the
other hand, the variant of K-means does not require a classifier, i.e., the
linear layer in the trained model, which enables to evaluate the OOD
performance of representation learning methods, e.g., self-supervised
learning.

#### Compactness.

In Section 3, we empirically show that feature separability is naturally tied
with the final accuracy and propose an effective score based on inter-class
dispersion. However, the connection between intra-class compactness and
generalization performance is still a mystery. In this analysis, we show that
compactness is not a good indicator of OOD accuracy. Specifically, we define
the compactness score:

$S(\tilde{\mathcal{D}})=-\log{\frac{\sum_{j=1}^{k}\sum_{i=1}^{m_{j}}||\boldsymbol{z}_{i}-\tilde{\boldsymbol{\mu}}_{j}||^{2}}{n-k}}$

where $\tilde{\boldsymbol{\mu}}_{j}$ denotes the centroid of cluster $j$ that
the instance $\boldsymbol{z}_{i}$ belongs to. Therefore, the compactness score
can measure the clusterability of the learned features, i.e., the average
distance between each instance and its cluster centroid. Intuitively, a high
compactness score may corresponds to a well-separated feature distribution,
which leads to high test accuracy. Surprisingly, in Figure 6(b), we find that
the compactness score is largely irrelevant to the final OOD error, showing
that it is not an effective indicator for predicting generalization
performance under distribution shifts.

##  6 Related work

Predicting generalization. Since the generalization capability of deep
networks under distribution shifts is a mysterious desideratum, a surge of
researches pay attention to estimate the generalization capability from two
directions.

1) Some works aim to measure generalization gap between training and test
accuracy with only training data (Corneanu et al., 2020; Jiang et al., 2019;
Neyshabur et al., 2017; Unterthiner et al., 2020; Yak et al., 2019; Martin &
Mahoney, 2020). For example, the model-architecture-based method (Corneanu et
al., 2020) summarizes the persistent topological map of a trained model to
formulate its inner-working function, which represents the generalization gap.
Margin distribution (Jiang et al., 2019) measures the gap by gauging the
distance between training examples and the decision boundary. However, those
methods are designed for the identical distribution between the training and
test dataset, being vulnerable to distribution shift.

2) Some studies try to estimate generalization performance on a specific OOD
test dataset without annotation during evaluation. Many of them utilize
softmax outputs of the shifted test dataset to form a quantitative indicator
of OOD error (Guillory et al., 2021; Jiang et al., 2021; Guillory et al.,
2021; Garg et al., 2022). However, those methods are unreliable across diverse
distribution shifts due to the overconfidence problem (Wei et al., 2022).
Another popular direction considers the negative correlation between
distribution difference and model’s performance in the space of features (Deng
& Zheng, 2021) or parameters (Yu et al., 2022). Nevertheless, common
distribution distances practically fail to induce stable error estimation
under distribution shift (Guillory et al., 2021), and those methods are
usually computationally expensive. Unsupervised loss such as agreement among
multiple classifiers (Jiang et al., 2021; Madani et al., 2004; Platanios et
al., 2016, 2017) and data augmentation (Deng et al., 2021) is also employed
for OOD error prediction, which requires specific model structures during
training. In this work, we focus on exploring the connection between feature
separability and generalization performance under distribution shift, which is
training-free and does not have extra requirements for datasets and model
architectures.

Exploring Feature distribution in deep learning. In the literature, feature
distribution has been widely studied in domain adaptation (Ben-David et al.,
2006; Pan et al., 2010; Zhuang et al., 2015; Tzeng et al., 2017),
representation learning (Bengio et al., 2013; HaoChen et al., 2021; Ming et
al., 2023; Huang et al., 2021), OOD generalization (Li et al., 2018; Chen et
al., 2021; Wang et al., 2021), and noisy-label learning (Zhu et al., 2021,
2022). Domain adaptation methods usually learn a domain-invariant feature
representation by narrowing the distribution distance between the two domains
with certain criteria, such as maxinum mean discrepancy (MMD) (Pan et al.,
2010), Kullback-Leibler (KL) divergence (Zhuang et al., 2015), central moment
discrepancy (CMD) (Zellinger et al., 2017), and Wasserstein distance (Lee &
Raginsky, 2017). InfoNCE (Huang et al., 2021) shows a key factor of
contrastive learning that the distance between class centers should be large
enough. In learning with noisy labels, it has been shown that the feature
clusterability can be used to estimate the transition matrix (Zhu et al.,
2021). To the best of our knowledge, we are the first to analyze the
connection between feature separability and the final accuracy on OOD data.

##  7 Conclusion

In this paper, we introduce Dispersion score, a simple yet effective indicator
for predicting the generalization performance on OOD data without labels. We
show that Dispersion score is strongly correlated to the OOD error and
achieves consistently better performance than previous methods under various
distribution shifts. Even when the OOD datasets are class imbalanced or have
limit number of instances, our method maintains a high prediction performance,
which demonstrates the strong flexibility of dispersion score. This method can
be easily adopted in practical settings. It is straightforward to implement
with trained models with various architectures, and does not require access to
the training data. Thus, our method is compatible with large-scale models that
are trained on billions of images. We hope that our insights inspire future
research to further explore feature separability for predicting OOD error.

## References

  * Ben-David et al. (2006) Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F.  Analysis of representations for domain adaptation.  _Advances in Neural Information Processing Systems_ , 19, 2006.
  * Bengio et al. (2013) Bengio, Y., Courville, A., and Vincent, P.  Representation learning: A review and new perspectives.  _IEEE transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828, 2013.
  * Chen et al. (2021) Chen, Y., Wang, Y., Pan, Y., Yao, T., Tian, X., and Mei, T.  A style and semantic memory mechanism for domain generalization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 9164–9173, 2021.
  * Corneanu et al. (2020) Corneanu, C. A., Escalera, S., and Martinez, A. M.  Computing the testing error without a testing set.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 2677–2685, 2020.
  * Deng & Zheng (2021) Deng, W. and Zheng, L.  Are labels always necessary for classifier accuracy evaluation?  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 15069–15078, 2021.
  * Deng et al. (2021) Deng, W., Gould, S., and Zheng, L.  What does rotation prediction tell us about classifier accuracy under varying testing environments?  In _International Conference on Machine Learning_ , pp. 2579–2589. PMLR, 2021.
  * Dowson & Landau (1982) Dowson, D. and Landau, B.  The fréchet distance between multivariate normal distributions.  _Journal of Multivariate Analysis_ , 12(3):450–455, 1982.
  * Ganin & Lempitsky (2015) Ganin, Y. and Lempitsky, V.  Unsupervised domain adaptation by backpropagation.  In _International Conference on Machine Learning_ , pp. 1180–1189. PMLR, 2015.
  * Gao & Kleywegt (2022) Gao, R. and Kleywegt, A.  Distributionally robust stochastic optimization with wasserstein distance.  _Mathematics of Operations Research_ , 2022.
  * Garg et al. (2022) Garg, S., Balakrishnan, S., Lipton, Z. C., Neyshabur, B., and Sedghi, H.  Leveraging unlabeled data to predict out-of-distribution performance.  _arXiv preprint arXiv:2201.04234_ , 2022.
  * Gretton et al. (2006) Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., and Smola, A.  A kernel method for the two-sample-problem.  _Advances in Neural Information Processing Systems_ , 19, 2006.
  * Guillory et al. (2021) Guillory, D., Shankar, V., Ebrahimi, S., Darrell, T., and Schmidt, L.  Predicting with confidence on unseen distributions.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 1134–1144, 2021.
  * HaoChen et al. (2021) HaoChen, J. Z., Wei, C., Gaidon, A., and Ma, T.  Provable guarantees for self-supervised deep learning with spectral contrastive loss.  _Advances in Neural Information Processing Systems_ , 34:5000–5011, 2021.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , pp. 770–778, 2016.
  * Hendrycks & Dietterich (2019) Hendrycks, D. and Dietterich, T.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hendrycks & Gimpel (2016) Hendrycks, D. and Gimpel, K.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  _arXiv preprint arXiv:1610.02136_ , 2016.
  * Huang et al. (2021) Huang, W., Yi, M., and Zhao, X.  Towards the generalization of contrastive self-supervised learning.  _arXiv preprint arXiv:2111.00743_ , 2021.
  * Jiang et al. (2019) Jiang, Y., Krishnan, D., Mobahi, H., and Bengio, S.  Predicting the generalization gap in deep networks with margin distributions.  In _International Conference on Learning Representations_ , 2019.
  * Jiang et al. (2021) Jiang, Y., Nagarajan, V., Baek, C., and Kolter, J. Z.  Assessing generalization of sgd via disagreement.  _arXiv preprint arXiv:2106.13799_ , 2021.
  * Koh et al. (2021) Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al.  Wilds: A benchmark of in-the-wild distribution shifts.  In _International Conference on Machine Learning_ , pp. 5637–5664. PMLR, 2021.
  * Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G., et al.  Learning multiple layers of features from tiny images.  2009\.
  * Le & Yang (2015) Le, Y. and Yang, X.  Tiny imagenet visual recognition challenge.  _CS 231N_ , 7(7):3, 2015.
  * Lee & Raginsky (2017) Lee, J. and Raginsky, M.  Minimax statistical learning with wasserstein distances.  _arXiv preprint arXiv:1705.07815_ , 2017.
  * Li et al. (2018) Li, Y., Tian, X., Gong, M., Liu, Y., Liu, T., Zhang, K., and Tao, D.  Deep domain generalization via conditional invariant adversarial networks.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 624–639, 2018.
  * Lloyd (1982) Lloyd, S.  Least squares quantization in pcm.  _IEEE Transactions on Information Theory_ , 28(2):129–137, 1982.
  * Loshchilov & Hutter (2016) Loshchilov, I. and Hutter, F.  Sgdr: Stochastic gradient descent with warm restarts.  _arXiv preprint arXiv:1608.03983_ , 2016.
  * MacQueen (1967) MacQueen, J.  Classification and analysis of multivariate observations.  In _5th Berkeley Symp. Math. Statist. Probability_ , pp. 281–297, 1967.
  * Madani et al. (2004) Madani, O., Pennock, D., and Flake, G.  Co-validation: Using model disagreement on unlabeled data to validate classification algorithms.  _Advances in Neural Information Processing Systems_ , 17, 2004.
  * Martin & Mahoney (2020) Martin, C. H. and Mahoney, M. W.  Heavy-tailed universality predicts trends in test accuracies for very large pre-trained deep neural networks.  In _Proceedings of the 2020 SIAM International Conference on Data Mining_ , pp. 505–513. SIAM, 2020.
  * Ming et al. (2023) Ming, Y., Sun, Y., Dia, O., and Li, Y.  How to exploit hyperspherical embeddings for out-of-distribution detection.  In _The Eleventh International Conference on Learning Representations (ICLR)_ , 2023.
  * Neyshabur et al. (2017) Neyshabur, B., Bhojanapalli, S., McAllester, D., and Srebro, N.  Exploring generalization in deep learning.  _Advances in Neural Information Processing Systems_ , 30, 2017.
  * Pan et al. (2010) Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q.  Domain adaptation via transfer component analysis.  _IEEE Transactions on Neural Networks_ , 22(2):199–210, 2010.
  * Platanios et al. (2017) Platanios, E., Poon, H., Mitchell, T. M., and Horvitz, E. J.  Estimating accuracy from unlabeled data: A probabilistic logic approach.  _Advances in Neural Information Processing Systems_ , 30, 2017.
  * Platanios et al. (2016) Platanios, E. A., Dubey, A., and Mitchell, T.  Estimating accuracy from unlabeled data: A bayesian approach.  In _International Conference on Machine Learning_ , pp. 1416–1425. PMLR, 2016.
  * Quinonero-Candela et al. (2008) Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D.  _Dataset shift in machine learning_.  Mit Press, 2008.
  * Sinha et al. (2017) Sinha, A., Namkoong, H., Volpi, R., and Duchi, J.  Certifying some distributional robustness with principled adversarial training.  _arXiv preprint arXiv:1710.10571_ , 2017.
  * Tzeng et al. (2014) Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., and Darrell, T.  Deep domain confusion: Maximizing for domain invariance.  _arXiv preprint arXiv:1412.3474_ , 2014.
  * Tzeng et al. (2017) Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T.  Adversarial discriminative domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 7167–7176, 2017.
  * Unterthiner et al. (2020) Unterthiner, T., Keysers, D., Gelly, S., Bousquet, O., and Tolstikhin, I.  Predicting neural network accuracy from weights.  _arXiv preprint arXiv:2002.11448_ , 2020.
  * Wang et al. (2021) Wang, Z., Luo, Y., Qiu, R., Huang, Z., and Baktashmotlagh, M.  Learning to diversify for single domain generalization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 834–843, 2021.
  * Wei et al. (2022) Wei, H., Xie, R., Cheng, H., Feng, L., An, B., and Li, Y.  Mitigating neural network overconfidence with logit normalization.  _arXiv preprint arXiv:2205.09310_ , 2022.
  * Yak et al. (2019) Yak, S., Gonzalvo, J., and Mazzawi, H.  Towards task and architecture-independent generalization gap predictors.  _arXiv preprint arXiv:1906.01550_ , 2019.
  * Yu et al. (2022) Yu, Y., Yang, Z., Wei, A., Ma, Y., and Steinhardt, J.  Predicting out-of-distribution error with the projection norm.  _arXiv preprint arXiv:2202.05834_ , 2022.
  * Zagoruyko & Komodakis (2016) Zagoruyko, S. and Komodakis, N.  Wide residual networks.  _arXiv preprint arXiv:1605.07146_ , 2016.
  * Zellinger et al. (2017) Zellinger, W., Grubinger, T., Lughofer, E., Natschläger, T., and Saminger-Platz, S.  Central moment discrepancy (cmd) for domain-invariant representation learning.  _arXiv preprint arXiv:1702.08811_ , 2017.
  * Zhu et al. (2021) Zhu, Z., Song, Y., and Liu, Y.  Clusterability as an alternative to anchor points when learning with noisy labels.  In _International Conference on Machine Learning_ , pp. 12912–12923. PMLR, 2021.
  * Zhu et al. (2022) Zhu, Z., Dong, Z., and Liu, Y.  Detecting corrupted labels without training a model to predict.  In _International Conference on Machine Learning_ , pp. 27412–27427. PMLR, 2022.
  * Zhuang et al. (2015) Zhuang, F., Cheng, X., Luo, P., Pan, S. J., and He, Q.  Supervised representation learning: Transfer learning with deep autoencoders.  In _Twenty-Fourth International Joint Conference on Artificial Intelligence_ , pp. 4119–4125, 2015.

##  Appendix A The importance of the weight

As introduced in Section 3, Dispersion Score can be viewed as a weighted
arithmetic mean of the distance from centers of each class to the center of
the whole samples in the feature space, where the weight is the total number
of samples in the corresponding class. To verify the importance of the weight
in long-tailed setting, we consider a variant that removes the weight:

$S(\tilde{\mathcal{D}})=\frac{1}{k-1}\sum_{j=0}^{k}\varphi(\bar{\boldsymbol{\mu}},\tilde{\boldsymbol{\mu}}_{j}).$

The results are shown in Table 4, where we compare performances of Dispersion
score and the variant without weight respectively under imbalanced CIFAR-10C
and CIFAR-100C with ResNet10 and ResNet50. From this table, we could observe
that the weight enhance the robustness significantly in long-tail conditions.

Table 4: Summary of prediction performance on Imbalanced CIFAR-10C and
CIFAR-100C, where $R^{2}$ refers to coefficients of determination, and $\rho$
refers to Spearson correlation coefficients (higher is better). The best
results are highlighted in bold.

Dataset | Network | w/o Weights | w/ Weights
---|---|---|---
$R^{2}$ | $\rho$ | $R^{2}$ | $\rho$
CIFAR 10 | ResNet18 | 0.675 | 0.930 | 0.959 | 0.982
ResNet50 | 0.748 | 0.948 | 0.968 | 0.982
Average | 0.712 | 0.939 | 0.978 | 0.990
CIFAR 100 | ResNet18 | 0.595 | 0.838 | 0.941 | 0.982
ResNet50 | 0.395 | 0.733 | 0.956 | 0.982
Average | 0.494 | 0.785 | 0.952 | 0.986

##  Appendix B Sensitivity analysis: pseudo labels

Here, we conduct a sensitivity analysis by using ground-truth labels in our
method. Table 5 illustrates that the performance with pseudo labels is
comparable with the performance using ground-truth labels, showing that our
method is not sensitive to noises from classification error.

Table 5: Comparison of Dispersion Score with pseudo labels and true labels on
CIFAR10, CIFAR100 and TinyImageNet, where $R^{2}$ refers to coefficients of
determination and $\rho$ refers to Spearson correlation coefficients (higher
is better). The best results are highlighted in bold.

Dataset | Network | Pseudo labels | True labels
---|---|---|---
$R^{2}$ | $\rho$ | $R^{2}$ | $\rho$
CIFAR 10 | ResNet18 | 0.968 | 0.990 | 0.979 | 0.989
ResNet50 | 0.987 | 0.990 | 0.985 | 0.991
WRN-50-2 | 0.961 | 0.988 | 0.945 | 0.987
Average | 0.972 | 0.990 | 0.970 | 0.989
CIFAR 100 | ResNet18 | 0.952 | 0.988 | 0.915 | 0.989
ResNet50 | 0.953 | 0.985 | 0.959 | 0.989
WRN-50-2 | 0.980 | 0.991 | 0.978 | 0.995
Average | 0.962 | 0.988 | 0.950 | 0.991
TinyImageNet | ResNet18 | 0.966 | 0.986 | 0.937 | 0.985
ResNet50 | 0.977 | 0.990 | 0.954 | 0.995
WRN-50-2 | 0.968 | 0.986 | 0.977 | 0.994
Average | 0.970 | 0.987 | 0.956 | 0.991