capbtabboxtable[][]

# An Effective Baseline for Robustness to Distributional Shift

Sunil Thulasidasan Los Alamos National Laboratory
Los Alamos, NM, USA  Dept. of Electrical & Computer Engineering
University of Washington
Seattle, USA  Sushil Thapa  Dept. of Computer Science and Engineering
New Mexico Tech
Socorro, NM, USA  Sayera Dhaubhadel  Los Alamos National Laboratory
Los Alamos, NM, USA  Gopinath Chennupati  Los Alamos National Laboratory
Los Alamos, NM, USA  Tanmoy Bhattacharya  Los Alamos National Laboratory
Los Alamos, NM, USA  Jeff Bilmes  Dept. of Electrical & Computer Engineering
University of Washington
Seattle, USA

###### Abstract

Refraining from confidently predicting when faced with categories of inputs
different from those seen during training is an important requirement for the
safe deployment of deep learning systems. While simple to state, this has been
a particularly challenging problem in deep learning, where models often end up
making overconfident predictions in such situations. In this work we present a
simple, but highly effective approach to deal with out-of-distribution
detection that uses the principle of abstention: when encountering a sample
from an unseen class, the desired behavior is to abstain from predicting. Our
approach uses a network with an extra abstention class and is trained on a
dataset that is augmented with an uncurated set that consists of a large
number of out-of-distribution (OoD) samples that are assigned the label of the
abstention class; the model is then trained to learn an effective
discriminator between in and out-of-distribution samples. We compare this
relatively simple approach against a wide variety of more complex methods that
have been proposed both for out-of-distribution detection as well as
uncertainty modeling in deep learning, and empirically demonstrate its
effectiveness on a wide variety of of benchmarks and deep architectures for
image recognition and text classification, often outperforming existing
approaches by significant margins. Given the simplicity and effectiveness of
this method, we propose that this approach be used as a new additional
baseline for future work in this domain.

##  1 Introduction and Related Work

Most of supervised machine learning has been developed with the assumption
that the distribution of classes seen at train and test time are the same.
However, the real-world is unpredictable and open-ended, and making machine
learning systems robust to the presence of unknown categories and out-of-
distribution samples has become increasingly essential for their safe
deployment. While refraining from predicting when uncertain should be
intuitively obvious to humans, the peculiarities of DNNs makes them
overconfident to unknown inputs Nguyen et al. [2015] and makes this a
challenging problem to solve in deep learning.

A very active sub-field of deep learning, known as out-of-distribution (OoD)
detection, has emerged in recent years that attempts to impart to deep neural
networks the quality of "knowing when it doesn’t know". The most straight-
forward approach in this regard is based on using the DNNs output as a proxy
for predictive confidence. For example, a simple baseline for detecting OoD
samples using thresholded softmax scores was presented in Hendrycks and Gimpel
[2016]. where the authors provided empirical evidence that for DNN
classifiers, in-distribution predictions do tend to have higher winning scores
than OoD samples, thus empirically justifying the use of softmax thresholding
as a useful baseline. However this approach is vulnerable to the pathologies
discussed in Nguyen et al. [2015]. Subsequently, increasingly sophisticated
methods have been developed to attack the OoD problem. Liang et al. [2018]
introduced a detection technique that involves perturbing the inputs in the
direction of increasing the confidence of the network’s predictions on a given
input, based on the observation that the magnitude of gradients on in-
distribution data tend to be larger than for OoD data. The method proposed in
Lee et al. [2018] also involves input perturbation, but confidence in this
case was measured by the Mahalanobis distance score using the computed mean
and co-variance of the pre-softmax scores. A drawback of such methods,
however, is that it introduces a number of hyperparameters that need to be
tuned on the OoD dataset, which is infeasible in many real-world scenarios as
one does not often know in advance the properties of unknown classes. A
modified version of the perturbation approach was recently proposed in in Hsu
et al. [2020] that circumvents some of these issues, though one still needs to
ascertain an ideal perturbation magnitude, which might not generalize from one
OoD set to the other.

Given that one might expect a classifier to be more uncertain when faced with
OoD data, many methods developed for estimating uncertainty for DNN
predictions have also been used for OoD detection. A useful baseline in this
regard is the temperature scaling method of Guo et al. [2017] that was was
proposed for calibrating DNN predictions on in-distribution data and has been
observed to also serve as a useful OoD detector in some scenarios. Further,
label smoothing techniques like mixup Zhang et al. [2017] have also been shown
to be able to improve OoD detection performance in DNNs Thulasidasan et al.
[2019]. An ensemble-of-deep models approach, that is also augmented with
adversarial examples during training, described in Lakshminarayanan et al.
[2017] was also shown to improve predictive uncertainty and successfully
applied to OoD detection.

In the Bayesian realm, methods such as Maddox et al. [2019] and Osawa et al.
[2019] have also been used for OoD detection, though at increased
computational cost. However, it has been argued that for OoD detection,
Bayesian priors on the data are not completely justified since one does not
have access to the prior of the open-set Boult et al. [2019]. Nevertheless,
simple approaches like dropout – which have been shown to be equivalent to
deep Gaussian processes Gal and Ghahramani [2016] – have been used as
baselines for OoD detection.

Training the model to recognize unknown classes by using data from categories
that do not overlap with classes of interest has been shown to be quite
effective for out-of-distribution detection and a slew of methods that use
additional data for discriminating between ID and OD data have been proposed.
DeVries and Taylor [2018] describes a method that uses a separate confidence
branch and misclassified training data samples that serve as a proxy for OoD
samples. In the outlier exposure technique described in Hendrycks et al.
[2018], the predictions on natural outlier images used in training are
regularized against the uniform distribution to encourage high-entropy
posteriors on outlier samples. An approach that uses an extra-class for
outlier samples is described in Neal et al. [2018], where instead of natural
outliers, counterfactual images that lie just outside the class boundaries of
known classes are generated using a GAN and assigned the extra class label. A
similar approach using generative samples for the extra class, but using a
conditional Variational Auto-Encoders Kingma and Welling [2013] for
generation, is described in Vernekar et al. [2019]. A method to force a DNN to
produce high-entropy (i.e., low confidence) predictions and suppress the
magnitude of feature activations for OoD samples was discussed in Dhamija et
al. [2018], where, arguing that methods that use an extra background class for
OoD samples force all such samples to lie in one region of the feature space,
the work also forces separation by suppressing the activation magnitudes of
samples from unknown classes

The above works have shown that the use of known OoD samples (or known
unknowns) often generalizes well to unknown unknown samples. Indeed, even
though the space of unknown classes is potentially infinite, and one can never
know in advance the myriad of inputs that can occur during test time,
empirically this approach has been shown to work. The abstention method that
we describe in the next section borrows ideas from many of the above methods:
as in Hendrycks et al. [2018], we uses additional samples of real images and
text from non-overlapping categories to train the model to abstain, but
instead of entropy regularization over OoD samples, out method uses an extra
abstention class. While it has been sometimes argued in the literature that
that using an additional abstention (or rejection) class is not an effective
approach for OoD detection Dhamija et al. [2018], Lee et al. [2017],
comprehensive experiments we conduct in this work demonstrate that this is not
the case. Indeed, we find that such an approach is not only simple but also
highly effective for OoD detection, often outperforming existing methods that
are more complicated and involve tuning of multiple hyperparameters. The main
contributions of this work are as follows:

  * •

To the best of our knowledge, this is the first work to comprehensively
demonstrate the efficacy of using an extra abstention (or rejection class) in
combination with outlier training data for effective OoD detection.

  * •

In addition to being effective, our method is also simple: we introduce no
additional hyperparameters in the loss function, and train with regular cross
entropy. From a practical standpoint, this is especially useful for deep
learning practitioners who might not wish to make modifications to the loss
function while training deep models. In addition, since outlier data is simply
an additional training class, no architectural modifications to existing
networks are needed.

  * •

Due to the simplicity and effectiveness of this method, we argue that this
approach be considered a strong baseline for comparing new methods in the
field of OoD detection.

##  2 Out-of-Distribution Detection with an Abstaining Classifier (DAC)

Our approach uses a DNN trained with an extra abstention class for detecting
out-of-distribution and novel samples; from here on, we will refer to this as
the deep abstaining classifier (DAC). We augment our training set of in-
distribution samples ($\mathcal{D}_{in}$) with an auxiliary dataset of known
out-of-distribution samples ($\mathcal{\tilde{D}}_{out}$), that are known to
be mostly disjoint from the main training set (we will use $\mathcal{D}_{out}$
to denote unknown out-of-distribution samples that we use for testing). We
assign the training label of $K+1$ to all the outlier samples in
$\mathcal{\tilde{D}}_{out}$ (where $K$ is the number of known classes) and
train with cross-entropy; the minimization problem then becomes:

$\displaystyle\mathcal{L}$
$\displaystyle=\min_{\theta}\\{\mathbb{E}_{(\mathbf{x},y)\sim\mathcal{D}_{in}}\left[-\log
P_{\theta}(y=\hat{y}|{\mathbf{x}})\right]$
$\displaystyle+\mathbb{E}_{\mathbf{x,y}\sim\mathcal{\tilde{D}}_{out}}\left[-\log
P_{\theta}(y=K+1|\mathbf{x})\right]\\}$

where $\theta$ are the weights of the neural network. This is somewhat similar
to the approaches described in Hendrycks et al. [2018] as well as in Lee et
al. [2017], with the main difference being that in those methods, an extra
class is not used; instead predictions on outliers are regularized against the
uniform distribution. Further the loss on the outlier samples is weighted by a
hyperparameter $\lambda$ which has to be tuned; in contrast, our approach does
not introduce any additional hyperparameters.

Figure 1: An illustration of the separability of scores on in and out-of-
distribution data for a regular DNN (left) and the DAC (right).

In our experiments, we find that the presence of an abstention class that is
used to capture the mass in $\mathcal{\tilde{D}}_{out}$ significantly
increases the ability to detect $\mathcal{D}_{out}$ during testing. For
example, in Figure 1, we show the distribution of the winning logits (pre-
softmax activations) in a regular DNN (left). For the same experimental setup,
the abstention logit of the DAC produces near-perfect separation of the in and
out-of-distribution logits indicating that using an abstention class for
mapping outliers can be a very effective approach to OoD detection.
Theoretically, it might be argued that the abstention class might only capture
data that is aligned with the weight vector of that class, and thus this
approach might fail to detect the myriad of OoD inputs that might span the
entire input region. Comprehensive experiments over a wide variety of
benchmarks described in the subsequent section, however, empirically
demonstrate that while the detection is not perfect, it performs very well,
and indeed, much better than more complicated approaches.

Once the model is trained, we use a simple thresholding mechanism for
detection. Concretely, the detector, $g(x):X\to{0,1}$ assigns label 1 (OoD) if
the softmax score of the abstention class, i.e., $p_{K+1}(x)$ is above some
threshold $\delta$, and label $0$, otherwise:

$g(\mathbf{x})=\left\\{\begin{array}[]{ll}1&\text{ if }p_{K+1}(x)\geq\delta\\\
0&\text{ otherwise }\end{array}\right.$

Like in other methods, the threshold $\delta$ has to be determined based on
acceptable risk that might be specific to the application. However, using
performance metrics like area under the ROC curve (AUROC), we can determine
threshold-independent performance of various methods, and we use this as one
of our evaluation metrics in all our experiments. We note here that recent
work in Mohseni et al. [2020] attacks the OoD problem along similar lines by
using a rejection class and training on outlier data; however the results in
this paper are much more comprehensive since we compare not only to a wide
variety of OoD methods, but also with techniques that focus on predictive
uncertainty in deep learning. Further, we also demonstrate the effectiveness
of an abstention class using only a small fraction of the outlier data that
was used in Mohseni et al. [2020].

##  3 Experiments

The experiments we describe here can be divided into two sets: in the first
set, we compare against methods that are explicitly designed for OoD
detection, while in the second category, we compare against methods that are
known to improve predictive uncertainty in deep learning. In both cases, we
report results over a variety of architectures to demonstrate the efficacy of
our method.

###  3.1 Datasets

For all computer vision experiments, we use CIFAR-10 and CIFAR-100 Krizhevsky
and Hinton [2009] as the in-distribution datasets, in addition to augmenting
our training set with $100$K unlabeled samples from the Tiny Images dataset
Torralba et al. [2008]. For the out-of-distribution datasets, we test on the
following:

  * •

SVHN Netzer et al. [2011], a large set of $32\times 32$ color images of house
numbers, comprising of ten classes of digits $0-9$. We use a subset of the 26K
images in the test set.

  * •

LSUN Yu et al. [2015], the Large-scale Scene Understanding dataset, comprising
of 10 different types of scenes.

  * •

Places365 Zhou et al. [2017], a large collection of pictures of scenes that
fall into one of 365 classes.

  * •

Tiny ImageNet tin [2017] (not to be confused with Tiny Images) which consists
of images belonging to 200 categories that are a subset of ImageNet
categories. The images are $64\times 64$ color, which we scale down to
$32\times 32$ when testing.

  * •

Gaussian A synthetically generated dataset consisting of $32\times 32$ random
Gaussian noise images, where each pixel is sampled from an i.i.d Gaussian
distribution.

For the NLP experiments, we use 20 Newsgroup Lang [1995], TREC Sherman , and
SST Socher et al. [2013] datasets as our in-distribution datasets, which are
the same as those used by Hendrycks et al. [2018] to facilitate direct
comparison. We use the 50-category version of TREC, and for SST, we use
binarized labels where neutral samples are removed. For out OoD training data,
we use unlabeled samples from Wikitext2 by assigning them to the abstention
class. We test our model on the following OoD datasets:

  * •

SNLI Bowman et al. [2015] is a dataset of predicates and hypotheses for
natural language inference. We use the hypotheses for testing .

  * •

IMDB Maas et al. [2011] is a sentiment classification dataset of movie
reviews, with similar statistics to those of SST.

  * •

Multi30K Barrault et al. [2018] is a dataset of English-German image
descriptions, of which we use the English descriptions.

  * •

WMT16 Bojar et al. [2016] is a dataset of English-German language pairs
designed for machine translation task. We use the English portion of the test
set from WMT16.

  * •

Yelp Zhang et al. [2015] is a dataset of restaurant reviews.

###  3.2 Comparison against OoD Methods

In this section, we compare against a slew of recent state-of-the-art methods
that have been explicitly designed for OoD detection. For the image
experiments, we compare against the following:

  * •

Deep Outlier Exposure, as described in Hendrycks et al. [2018] and discussed
in Section 1

  * •

Ensemble of Leave-out Classifiers Vyas et al. [2018] where each classifier is
trained by leaving out a random subset of training data (which is treated as
OoD data), and the rest is treated as ID data.

  * •

ODIN, as described in Liang et al. [2018] and discussed in Section 1. ODIN
uses input perturbation and temperature scaling to differentiate between ID
and OoD samples.

  * •

Deep Mahalanobis Detector, proposed in Lee et al. [2018] which estimates the
class-conditional distribution over hidden layer features of a deep model
using Gaussian discriminant analysis and a Mahalanobis distance based
confidence-score for thresholding, and further, similar to ODIN, uses input
perturbation while testing.

  * •

OpenMax, as described in Bendale and Boult [2016] for novel category
detection. This method uses mean activation vectors of ID classes observed
during training followed by Weibull fitting to determine if a given sample is
novel or out-of-distribution.

For all of the above methods, we use published results when available, keeping
the architecture and datasets the same as in the experiments described in the
respective papers. For the NLP experiments, we only compare against the
published results in Hendrycks et al. [2018]. For OpenMax, we re-implement the
authors’ published algorithm using the PyTorch framework Paszke et al. [2019].

####  3.2.1 Metrics

Following established practices in the literature, we use the following
metrics to measure detection performance of our method:

  * •

AUROC or Area Under the Receiver Operating Characteristic curve depicts the
relationship between the True Positive Rate (TPR) (also known as Recall)and
the False Positive Rate (FPR) and can be interpreted as the probability that a
positive example is assigned a higher detection score than a negative example
Fawcett [2006]. Unlike 0/1 accuracy, the AUROC has the desirable property that
it is not affected by class imbalance111An alternate area-under-the-curve
metric, known as Area under Precision Recall Curve, or AUPRC, is used when the
size of the negative class is high compared to the positive class. We do not
report AUPRC here, as we keep our in-distribution and out-of-distribution sets
balanced in these experiments..

  * •

FPR at 95% TPR which is the probability that a negative sample is
misclassified as a positive sample when the TPR (or recall) on the positive
samples is 95%.

In work that we compare against, the out-of-distribution samples are treated
as the positive class, so we do the same here, and treat the in-distribution
samples as the negative class.

####  3.2.2 Results

|  | vs. Outlier Exposure (OE) Hendrycks et al. [2018]
---
(Model: Wide ResNet 40x2)
|  $\mathcal{D}_{in}$: CIFAR-10 |  $\mathcal{D}_{in}$: CIFAR-100
| FPR95 $\downarrow$ | AUROC $\uparrow$ | FPR95 $\downarrow$ | AUROC $\uparrow$
$\mathcal{D}_{out}$ | OE | Ours | OE | Ours | OE | Ours | OE | Ours
SVHN | 4.8 | 2.00.69 | 98.4 | 99.460.16 | 42.9 | 40.4611.96 | 86.9 | 85.446.25
LSUN | 12.1 | 0.10.06 | 97.6 | 99.960.02 | 57.5 | 9.274.62 | 83.4 | 97.671.40
Places365 | 17.3 | 0.220.12 | 96.2 | 99.920.05 | 49.8 | 23.375.30 | 86.5 | 93.981.67
Gaussian | 0.7 | 0.130.14 | 99.6 | 99.930.09 | 12.1 | 13.2614.74 | 95.7 | 90.0311.81

|  | vs. Ensemble of Leave-out Classifiers (ELOC) Vyas et al. [2018]
---
(Model: Wide ResNet 28x10)
|  $\mathcal{D}_{in}$: CIFAR-10 |  $\mathcal{D}_{in}$: CIFAR-100
| FPR95 $\downarrow$ | AUROC $\uparrow$ | FPR95 $\downarrow$ | AUROC $\uparrow$
$\mathcal{D}_{out}$ | ELOC | Ours | ELOC | Ours | ELOC | Ours | ELOC | Ours
Tiny ImageNet | 2.94 | 1.912.24 | 99.36 | 99.450.67 | 24.53 | 18.686.31 | 95.18 | 94.881.75
LSUN | 0.88 | 1.51.80 | 99.7 | 99.610.47 | 16.53 | 9.231.87 | 96.77 | 97.890.48
Gaussian | 0.0 | 0.130.20 | 99.58 | 99.950.08 | 98.26 | 0.720.79 | 93.04 | 99.650.39

|  | vs. ODIN Liang et al. [2018]
---
(Model: Wide ResNet 28x10)
|  $\mathcal{D}_{in}$: CIFAR-10 |  $\mathcal{D}_{in}$: CIFAR-100
| FPR95 $\downarrow$ | AUROC $\uparrow$ | FPR95 $\downarrow$ | AUROC $\uparrow$
$\mathcal{D}_{out}$ | ODIN | Ours | ODIN | Ours | ODIN | Ours | ODIN | Ours
Tiny ImageNet | 25.5 | 1.912.24 | 92.1 | 99.450.67 | 55.9 | 18.686.31 | 84.0 | 94.881.75
LSUN | 17.6 | 1.51.80 | 95.4 | 99.610.47 | 56.5 | 9.231.87 | 86.0 | 97.890.48
Gaussian | 0.0 | 0.130.20 | 100.0 | 99.950.08 | 1.0 | 0.720.79 | 98.5 | 99.650.39

|  | vs. Deep Mahalanobis Detector (MAH) Lee et al. [2018]
---
(Model: ResNet 34)
|  $\mathcal{D}_{in}$: CIFAR-10 |  $\mathcal{D}_{in}$: CIFAR-100
| FPR95 $\downarrow$ | AUROC $\uparrow$ | FPR95 $\downarrow$ | AUROC $\uparrow$
$\mathcal{D}_{out}$ | MAH | Ours | MAH | Ours | MAH | Ours | MAH | Ours
SVHN | 24.2 | 1.890.78 | 95.5 | 99.490.17 | 58.1 | 41.318.01 | 84.4 | 86.852.38
Tiny ImageNet | 4.5 | 0.360.15 | 99.0 | 99.880.04 | 29.7 | 12.101.22 | 87.9 | 97.140.29
LSUN | 1.9 | 0.300.12 | 99.5 | 99.910.03 | 43.4 | 7.140.66 | 82.3 | 98.450.13

|  | vs. OpenMax Bendale and Boult [2016]
---
(Model: ResNet 34)
|  $\mathcal{D}_{in}$: CIFAR-10 |  $\mathcal{D}_{in}$: CIFAR-100
| FPR95 $\downarrow$ |  | AUROC
---
$\uparrow$
| FPR95
---
$\downarrow$
| AUROC
---
$\uparrow$
$\mathcal{D}_{out}$ | OpenMax | Ours | OpenMax | Ours | OpenMax | Ours | OpenMax | Ours
SVHN | 23.672.06 | 1.890.78 | 90.720.90 | 99.490.17 | 53.227.52 | 41.318.01 | 80.881.08 | 86.852.38
Tiny ImageNet | 24.209.11 | 0.360.15 | 93.390.75 | 99.880.04 | 32.675.21 | 12.101.22 | 81.222.21 | 97.140.29
LSUN | 18.681.24 | 0.300.12 | 92.161.82 | 99.910.03 | 30.212.71 | 7.140.66 | 83.082.16 | 98.450.13
Places365 | 27.272.77 | 0.840.28 | 90.720.85 | 99.670.07 | 50.711.25 | 30.542.26 | 81.130.30 | 92.690.65
Gaussian | 40.5822.18 | 0.040.02 | 84.7410.19 | 99.980.01 | 21.5011.73 | 1.661.76 | 89.375.46 | 99.480.47

Table 1: Comparison of the extra class method (ours) with various other out-
of-distribution detection methods when trained on CIFAR-10 and CIFAR-100 and
tested on other datasets. All numbers from comparison methods are sourced from
their respective original publications. For our method, we also report the
standard deviation over five runs (indicated by the subscript), and treat the
performance of other methods within one standard deviations as equivalent to
ours. For fair comparison with the Mahalanobis detector (MAH) Lee et al.
[2018], we use results when their method was not tuned separately on each OoD
test set (Table 6 in Lee et al. [2018]. The OpenMax implementation was based
on code available at https://github.com/abhijitbendale/OSDN and re-implemented
by us in PyTorch Paszke et al. [2019].

| vs. Outlier Exposure for NLP Classification
---
(Model: 2 Layered GRU)
$\mathcal{D}_{in}$ | $\mathcal{D}_{out}$ | FPR95 $\downarrow$ |  | AUROC
---
$\uparrow$
|  | OE | Ours | OE | Ours
20 Newsgroup | SNLI | 12.5 | 3.90.54 | 95.1 | 98.32.490.1
IMDB | 18.6 | 1.780.12 | 93.5 | 99.170.0
Multi30K | 3.2 | 0.800.13 | 97.3 | 99.520.04
WMT16 | 2.0 | 1.40.16 | 98.8 | 99.330.02
Yelp | 3.9 | 0.760.08 | 97.8 | 99.610.02
TREC | SNLI | 4.2 | 12.08.1 | 98.1 | 97.031.08
IMDB | 0.6 | 0.00.0 | 99.4 | 99.990.0
Multi30K | 0.3 | 8.33.4 | 99.7 | 97.560.5
WMT16 | 0.2 | 4.673.26 | 99.8 | 99.940.6
Yelp | 0.4 | 0.00.0 | 99.7 | 99.00.0
SST | SNLI | 33.4 | 20.92.3 | 86.8 | 92.240.77
IMDB | 32.6 | 0.70.46 | 85.9 | 99.370.1
Multi30K | 33.0 | 70.97.7 | 88.3 | 70.654.9
WMT16 | 17.1 | 31.65.9 | 92.9 | 90.641.7
Yelp | 11.3 | 0.060.0 | 92.7 | 99.670.08

Table 2: DAC vs OE for NLP Classification task. OE implementation was based on
code available at https://github.com/hendrycks/outlier-exposure

.

Detailed results against the various OoD methods are shown in Tables 1 through
2 for vision and language respectively, where we have a clear trend: in almost
all cases, the DAC outperforms the other methods, often by significant margins
especially when the in-distribution data is more complex, as is the case with
CIFAR-100. While the Outlier Exposure method Hendrycks et al. [2018] (shown at
the top in Table 1) is conceptually similar to ours, the presence of an extra
abstention class in our model often bestows significant performance
advantages. Further, we do not need to tune a separate hyperparameter which
determines the weight of the outlier loss, as done in Hendrycks et al. [2018].

In fact, the simplicity of our method is one of its striking features: we do
not introduce any additional hyperparameters in our approach, which makes it
significantly easier to implement than methods such as ODIN and the
Mahalanobis detector; these methods need to be tuned separately on each OoD
dataset, which is usually not possible as one does not have access to the
distribution of unseen classes in advance. Indeed, when performance of these
methods is tested without tuning on the OoD test set, the DAC significantly
outperforms methods such as the Mahalanobis detector (shown at the bottom of
Table 1). We also show the performance against the OpenMax approach of Bendale
and Boult [2016] and in every case, the DAC outperforms OpenMax by significant
margins.

While the abstention approach uses an extra class and OoD samples while
training, and thus does incur some training overhead, it is significantly less
expensive during test time, as the forward pass is no different from that of a
regular DNN. In contrast, methods like ODIN and the Mahalanobis detector
require gradient calculation with respect to the input in order to apply the
input perturbation; the DAC approach thus offers a computationally simpler
alternative. Also, even though the DAC approach introduces additional network
parameters in the final linear layers (due to the presence of an extra
abstention class), and thus might be more prone to overfitting, we find that
this to be not the case as evidenced by the generalization of OoD performance
to different types of test datasets.

###  3.3 Comparison against Uncertainty-based Methods

Next we perform experiments to compare the OoD detection performance of the
DAC against various methods that have been proposed for improving predictive
uncertainty in deep learning. In these cases, one expects that such methods
will cause the DNN to predict with less confidence when presented with inputs
from a different distribution or from novel categories; we compare against the
following methods:

  * •

Softmax Thresholding This is the simplest baseline, where OoD samples are
detected by thresholding on the winning softmax score; scores falling below a
threshold are rejected.

  * •

Entropy Thresholding Another simple baseline, where OoD samples are rejected
if the Shannon entropy calculated over the softmax posteriors is above a
certain threshold.

  * •

MonteCarlo Dropout A Bayesian inspired approach proposed in Gal and Ghahramani
[2016] for improving the predictive uncertainty for deep learning. We found a
dropout probability of $p=0.5$ to perform well, and use $100$ forward passes
per sample during the prediction.

  * •

Temperature Scaling, which improves DNN calibration as described in Guo et al.
[2017]. The scaling temperature $T$ is tuned on a held-out subset of the
validation set of the in-distribution data.

  * •

Mixup As shown in Thulasidasan et al. [2019], Mixup can be an effective OoD
detector, so we also use this as one of our baselines.

  * •

Deep Ensembles which was introduced in Lakshminarayanan et al. [2017] for
improving uncertainty estimates for both classification and regression. In
this approach, multiple versions of the same model are trained using different
random initializations, and while training, adversarial samples are generated
to improve model robustness. We use an ensemble size of $5$ as suggested in
their paper.

  * •

SWAG, as described in Maddox et al. [2019], which is a Bayesian approach to
deep learning and exploits the fact that SGD itself can be viewed as
approximate Bayesian inference Mandt et al. [2017]. We use an ensemble size of
$30$ as proposed in the original paper.

####  3.3.1 Results

|  |  $\mathcal{D}_{in}$: CIFAR-100
---
AUROC $\uparrow$
$\mathcal{D}_{out}$ | Softmax | Entropy |  | Monte Carlo
---
Dropout
| Temp
---
Scaling
Mixup |  | Deep
---
Ensemble
SWAG |  | DAC
---
(Ours)
LSUN | 87.860.54 | 89.530.68 | 84.352.99 | 87.750.70 | 86.031.65 | 89.360.23 | 83.872.59 | 98.730.10
Places-365 | 79.770.73 | 80.340.83 | 80.211.06 | 79.531.08 | 82.090.98 | 82.690.27 | 83.370.42 | 93.390.59
Gaussian | 80.766.52 | 80.468.50 | 68.4023.29 | 80.526.51 | 92.136.08 | 80.275.01 | 91.806.94 | 99.690.28
SVHN | 78.453.97 | 79.434.30 | 81.042.46 | 78.253.99 | 79.152.52 | 82.420.22 | 80.451.67 | 87.742.34
Tiny ImageNet | 86.630.92 | 88.001.22 | 82.074.44 | 86.491.14 | 84.571.61 | 86.640.23 | 80.902.37 | 97.600.25
|  |  $\mathcal{D}_{in}$: CIFAR-100
---
FPR95 $\downarrow$
$\mathcal{D}_{out}$ | Softmax | Entropy |  | Monte Carlo
---
Dropout
| Temp
---
Scaling
Mixup |  | Deep
---
Ensemble
SWAG |  | DAC
---
(Ours)
LSUN | 35.641.65 | 34.521.82 | 42.934.60 | 35.641.65 | 34.521.82 | 32.480.37 | 40.504.09 | 5.410.58
Places-365 | 55.191.36 | 55.811.40 | 56.501.33 | 55.191.36 | 55.811.40 | 46.550.81 | 47.760.91 | 29.592.48
Gaussian | 33.708.14 | 32.8210.19 | 41.3423.76 | 33.708.14 | 32.8210.19 | 23.855.07 | 13.6110.38 | 0.890.96
SVHN | 55.2810.30 | 55.1610.67 | 50.084.49 | 55.2810.30 | 55.1610.67 | 47.570.53 | 49.433.63 | 40.448.31
Tiny ImageNet | 37.271.04 | 36.651.25 | 47.106.05 | 37.271.04 | 36.651.25 | 38.580.51 | 45.054.08 | 10.151.09

|  |  $\mathcal{D}_{in}$: CIFAR-10
---
AUROC $\uparrow$
$\mathcal{D}_{out}$ | Softmax | Entropy |  | Monte Carlo
---
Dropout
| Temp
---
Scaling
Mixup |  | Deep
---
Ensemble
SWAG | DAC
LSUN | 93.450.36 | 93.840.37 | 91.561.77 | 93.990.89 | 96.581.38 | 93.440.19 | 96.280.06 | 99.920.02
Places-365 | 91.700.56 | 92.080.58 | 89.290.72 | 92.391.00 | 96.380.87 | 94.780.23 | 96.070.63 | 100.000.00
Gaussian | 79.3913.06 | 79.2213.24 | 94.262.74 | 80.409.22 | 95.623.69 | 95.821.30 | 96.510.52 | 100.000.00
SVHN | 91.381.03 | 91.701.11 | 83.563.85 | 91.990.34 | 92.622.96 | 90.580.20 | 96.121.19 | 99.500.16
Tiny ImageNet | 91.981.85 | 92.311.88 | 88.845.19 | 92.591.47 | 95.672.02 | 94.070.29 | 95.650.13 | 99.880.04
|  |  $\mathcal{D}_{in}$: CIFAR-10
---
FPR95 $\downarrow$
$\mathcal{D}_{out}$ | Softmax | Entropy |  | Monte Carlo
---
Dropout
| Temp
---
Scaling
Mixup |  | Deep
---
Ensemble
SWAG | DAC
LSUN | 19.151.38 | 18.931.30 | 25.864.46 | 18.762.65 | 12.474.88 | 19.360.54 | 12.710.02 | 0.270.12
Places-365 | 26.493.65 | 26.423.79 | 33.652.31 | 26.376.27 | 14.774.62 | 15.790.39 | 12.801.67 | 0.000.00
Gaussian | 52.8422.15 | 53.5221.98 | 12.634.24 | 55.9118.40 | 10.9611.42 | 9.151.36 | 9.550.52 | 0.000.00
SVHN | 24.642.41 | 24.672.49 | 53.3214.76 | 25.063.46 | 29.7112.57 | 24.990.65 | 13.044.05 | 1.780.76
Tiny ImageNet | 26.2110.40 | 26.2410.66 | 33.2515.67 | 24.965.58 | 14.846.12 | 17.030.58 | 12.070.14 | 0.330.14
|  |  $\mathcal{D}_{in}$: SST
---
FPR95 $\downarrow$
$\mathcal{D}_{out}$ | Monte Carlo Dropout | Entropy |  | Monte Carlo
---
Dropout
| Temp
---
Scaling
Mixup |  | Deep
---
Ensemble
SWAG | DAC
SNLI | 19.151.38 | 18.931.30 | 25.864.46 | 18.762.65 | 12.474.88 | 19.360.54 | 12.710.02 | 0.270.12
IMDB | 26.493.65 | 26.423.79 | 33.652.31 | 26.376.27 | 14.774.62 | 15.790.39 | 12.801.67 | 0.000.00
Multi30K | 52.8422.15 | 53.5221.98 | 12.634.24 | 55.9118.40 | 10.9611.42 | 9.151.36 | 9.550.52 | 0.000.00
WMT16 | 24.642.41 | 24.672.49 | 53.3214.76 | 25.063.46 | 29.7112.57 | 24.990.65 | 13.044.05 | 1.780.76
Yelp Reviews | 26.2110.40 | 26.2410.66 | 33.2515.67 | 24.965.58 | 14.846.12 | 17.030.58 | 12.070.14 | 0.330.14

Table 3: The performance of DAC as an OoD detector, evaluated on various
metrics and compared against competing baselines. All experiments used the
ResNet-34 architecture, except for MC Dropout, in which case we used the
WideResNet 28x10 network. $\uparrow$ and $\downarrow$ indicate that higher and
lower values are better, respectively. Best performing methods (ignoring
statistically insignificant differences)on each metric are in bold.

Detailed results are shown in Table 3, where the best performing method for
each metric is shown in bold. The DAC is the only method in this set of
experiments that uses an augmented dataset, and as is clearly evident from the
results, this confers a significant advantage over the other methods in most
cases. Calibration methods like temperature scaling, while producing well
calibrated scores on in-distribution data, end up reducing the confidence on
in-distribution data as well, and thus losing discriminative power between the
two types of data. We also note here that many of the methods listed in the
table, like temperature scaling and deep ensembles, can be combined with the
abstention approach. Indeed, the addition of an extra abstention class and
training with OoD data is compatible with most uncertainty modeling techniques
in deep learning; we leave the exploration of such combination approaches for
future work.

##  4 Conclusion

We presented a simple, but highly effective method for open set and out-of-
distribution detection that clearly demonstrated the efficacy of using an
extra abstention class and augmenting the training set with outliers. While
previous work has shown the efficacy of outlier exposure Hendrycks et al.
[2018], here we demonstrated an alternative approach for exploiting outlier
data that further improves upon existing methods, while also being simpler to
implement compared to many of the other methods. The ease of implementation,
absence of additional hyperparameter tuning and computational efficiency
during testing makes this a very viable approach for improving out-of-
distribution and novel category detection in real-world deployments; we hope
that this will also serve as an effective baseline for comparing future work
in this domain.

## References

  * tin [2017] Tiny imagenet visual recognition challenge, 2017.  URL https://tiny-imagenet.herokuapp.com/.
  * Barrault et al. [2018] Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, and Stella Frank.  Findings of the third shared task on multimodal machine translation.  In _Proceedings of the Third Conference on Machine Translation: Shared Task Papers_ , pages 304–323, 2018.
  * Bendale and Boult [2016] Abhijit Bendale and Terrance E Boult.  Towards open set deep networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1563–1572, 2016.
  * Bojar et al. [2016] Ond rej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri.  Findings of the 2016 conference on machine translation.  In _Proceedings of the First Conference on Machine Translation_ , pages 131–198, Berlin, Germany, August 2016. Association for Computational Linguistics.  URL http://www.aclweb.org/anthology/W/W16/W16-2301.
  * Boult et al. [2019] TE Boult, S Cruz, AR Dhamija, M Gunther, J Henrydoss, and WJ Scheirer.  Learning and the unknown: Surveying steps toward open world recognition.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 9801–9807, 2019.
  * Bowman et al. [2015] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, 2015.
  * DeVries and Taylor [2018] Terrance DeVries and Graham W Taylor.  Learning confidence for out-of-distribution detection in neural networks.  _arXiv preprint arXiv:1802.04865_ , 2018.
  * Dhamija et al. [2018] Akshay Raj Dhamija, Manuel Günther, and Terrance Boult.  Reducing network agnostophobia.  In _Advances in Neural Information Processing Systems_ , pages 9157–9168, 2018.
  * Fawcett [2006] Tom Fawcett.  An introduction to roc analysis.  _Pattern recognition letters_ , 27(8):861–874, 2006.
  * Gal and Ghahramani [2016] Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _international conference on machine learning_ , pages 1050–1059, 2016.
  * Guo et al. [2017] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  _arXiv preprint arXiv:1706.04599_ , 2017.
  * Hendrycks and Gimpel [2016] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  _arXiv preprint arXiv:1610.02136_ , 2016.
  * Hendrycks et al. [2018] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich.  Deep anomaly detection with outlier exposure.  _arXiv preprint arXiv:1812.04606_ , 2018.
  * Hsu et al. [2020] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira.  Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data.  _arXiv preprint arXiv:2002.11297_ , 2020.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Krizhevsky and Hinton [2009] Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  Technical report, Citeseer, 2009.
  * Lakshminarayanan et al. [2017] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell.  Simple and scalable predictive uncertainty estimation using deep ensembles.  In _Advances in Neural Information Processing Systems_ , pages 6402–6413, 2017.
  * Lang [1995] Ken Lang.  Newsweeder: Learning to filter netnews.  In _Proceedings of the Twelfth International Conference on Machine Learning_ , pages 331–339, 1995.
  * Lee et al. [2017] Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin.  Training confidence-calibrated classifiers for detecting out-of-distribution samples.  _arXiv preprint arXiv:1711.09325_ , 2017.
  * Lee et al. [2018] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  In _Advances in Neural Information Processing Systems_ , pages 7167–7177, 2018.
  * Liang et al. [2018] Shiyu Liang, Yixuan Li, and R Srikant.  Enhancing the reliability of out-of-distribution image detection in neural networks.  2018\.  URL http://arxiv.org/abs/1706.02690.
  * Maas et al. [2011] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.  URL http://www.aclweb.org/anthology/P11-1015.
  * Maddox et al. [2019] Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson.  A simple baseline for bayesian uncertainty in deep learning.  In _Advances in Neural Information Processing Systems_ , pages 13132–13143, 2019.
  * Mandt et al. [2017] Stephan Mandt, Matthew D Hoffman, and David M Blei.  Stochastic gradient descent as approximate bayesian inference.  _The Journal of Machine Learning Research_ , 18(1):4873–4907, 2017.
  * Mohseni et al. [2020] Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang.  Self-supervised learning for generalizable out-of-distribution detection.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 5216–5223, 2020.
  * Neal et al. [2018] Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li.  Open set learning with counterfactual images.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 613–628, 2018.
  * Netzer et al. [2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * Nguyen et al. [2015] Anh Nguyen, Jason Yosinski, and Jeff Clune.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 427–436, 2015.
  * Osawa et al. [2019] Kazuki Osawa, Siddharth Swaroop, Mohammad Emtiyaz E Khan, Anirudh Jain, Runa Eschenhagen, Richard E Turner, and Rio Yokota.  Practical deep learning with bayesian principles.  In _Advances in Neural Information Processing Systems_ , pages 4289–4301, 2019.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  In _Advances in neural information processing systems_ , pages 8026–8037, 2019.
  * [31] Garrick Sherman.  Trec document topic annotations.  URL http://qwone.com/~jason/20Newsgroups/.
  * Socher et al. [2013] Richard Socher, A. Perelygin, J.Y. Wu, J. Chuang, C.D. Manning, A.Y. Ng, and C. Potts.  Recursive deep models for semantic compositionality over a sentiment treebank.  _EMNLP_ , 1631:1631–1642, 01 2013.
  * Thulasidasan et al. [2019] Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak.  On mixup training: Improved calibration and predictive uncertainty for deep neural networks.  In _Advances in Neural Information Processing Systems_ , pages 13888–13899, 2019.
  * Torralba et al. [2008] Antonio Torralba, Rob Fergus, and William T Freeman.  80 million tiny images: A large data set for nonparametric object and scene recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 30(11):1958–1970, 2008.
  * Vernekar et al. [2019] Sachin Vernekar, Ashish Gaurav, Vahdat Abdelzad, Taylor Denouden, Rick Salay, and Krzysztof Czarnecki.  Out-of-distribution detection in classifiers via generation.  _arXiv preprint arXiv:1910.04241_ , 2019.
  * Vyas et al. [2018] Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L Willke.  Out-of-distribution detection using an ensemble of self supervised leave-out classifiers.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 550–564, 2018.
  * Yu et al. [2015] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao.  Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.  _arXiv preprint arXiv:1506.03365_ , 2015.
  * Zhang et al. [2017] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz.  mixup: Beyond empirical risk minimization.  _arXiv preprint arXiv:1710.09412_ , 2017.
  * Zhang et al. [2015] Xiang Zhang, Junbo Zhao, and Yann LeCun.  Character-level Convolutional Networks for Text Classification .  _arXiv:1509.01626 [cs]_ , September 2015.  URL https://www.yelp.com/dataset.
  * Zhou et al. [2017] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(6):1452–1464, 2017.