University of Illinois at Urbana-Champaign
Department of Mathematics
Department of Computer Science

UNDERGRADUATE THESIS
Submitted as part of an undergraduate research program

#  DOODLER: Determining Out-Of-Distribution
Likelihood from Encoder Reconstructions

By: Jonathan S. Kent
<EMAIL_ADDRESS>Advisor: Professor Bo Li

###### Contents

  1. 1 Abstract
  2. 2 Introduction
  3. 3 Related Work
  4. 4 Methodology
    1. 4.1 Variational Auto-Encoder
    2. 4.2 Reconstruction Error
  5. 5 Error Distribution Analyses
    1. 5.1 Out-Of-Distribution Sample Detection
    2. 5.2 Out-Of-Distribution Pixel Detection
    3. 5.3 Out-Of-Distribution Stream Detection
  6. 6 Experimentation
  7. 7 Results and Comparisons
  8. 8 Conclusions and Future Work
  9. 9 Example Figures

## Acknowledgements

I would like to begin by thanking my mom and dad for their unending and
essential support, and all the other obvious reasons why anyone would thank
their parents. But I must also thank them, Elliot N. Kent and Marsha N. Hahn,
as individuals, unique and especial, because I get to be nobody else but their
son. They taught me to be me. They taught me to learn. It is from them that I
inherited my appetite for knowledge, and my desire to bend that knowledge to
the world as it exists. What I have written here, being both original
Scientific research and working towards real-world applications, is dedicated
to them.

I would also like to thank my friend, Moira E. Iten, and her family, for their
ceaseless moral support and cheering-on. It would be impossible to determine
the precise extent of their contributions, but that is meaningless in the face
of my appreciation for them.

I will finish by thanking my colleagues: Charles C. Wamsley, Davin Flateau,
and Amber Ferguson, and all the other folks at Ball Aerospace & Technologies
Corp., for the excellent and interesting technical conversations, the insight
into the problems faced in the real-world practice of Machine Learning, and
the opportunity to develop as a Researcher and Engineer.

“A little song, a little dance, a little seltzer down your pants.” -Tim

Figure 1: Training and inference steps for using DOODLER

##  1 Abstract

Deep Learning models possess two key traits that, in combination, make their
use in the real world a risky prospect. One, they do not typically generalize
well outside of the distribution for which they were trained, and two, they
tend to exhibit confident behavior regardless of whether or not they are
producing meaningful outputs. While Deep Learning possesses immense power to
solve realistic, high-dimensional problems, these traits in concert make it
difficult to have confidence in their real-world applications. To overcome
this difficulty, the task of Out-Of-Distribution (OOD) Detection has been
defined, to determine when a model has received an input from outside of the
distribution for which it is trained to operate.

This paper introduces and examines a novel methodology, DOODLER, for OOD
Detection, which directly leverages the traits which result in its necessity.
By training a Variational Auto-Encoder (VAE) on the same data as another Deep
Learning model, the VAE learns to accurately reconstruct In-Distribution (ID)
inputs, but not to reconstruct OOD inputs, meaning that its failure state can
be used to perform OOD Detection. Unlike other work in the area, DOODLER
requires only very weak assumptions about the existence of an OOD dataset,
allowing for more realistic application. DOODLER also enables pixel-wise
segmentations of input images by OOD likelihood, and experimental results show
that it matches or outperforms methodologies that operate under the same
constraints.

##  2 Introduction

Perhaps the most fundamental of all questions in both the academic study and
applied practice of Machine Learning is that of Out-Of-Distribution
Generalization, or how to produce a model that is capable of operating in
settings for which it has not been expressly prepared [23]. This is an
extraordinarily difficult problem, and while inroads have been made in some
areas like Online Learning and Meta-Learning [19, 22], it is still generally
true that

  1. 1.

Machine Learning models are not good at generalizing outside of the domain
they were trained for, and

  2. 2.

Machine Learning models will still express their inferences with high
confidence when outside their domain [16].

The confluence of these traits means that, despite the extraordinary power of
Machine and Deep Learning to crack complex, real-world, hyperdimensional
problems that are intractable with any other methodology [18], actually using
these models in a production environment becomes a fraught decision. This is
true to an extreme degree in areas like Vehicle Autonomy, Medical Diagnostics,
and Defense, where a wrong decision made confidently may result in human
casualties.

Therefore, robust Out-Of-Distribution Detection, the ability to preempt the
differences between laboratory and real-world conditions, to determine when a
model is receiving data for which it is not prepared, is essential. By making
these determinations accurately, it allows for autonomous systems to activate
a backup protocol or Human-in-the-Loop system, greatly increasing their
trustworthiness for applications.

##  3 Related Work

Much of the OOD Detection research literature available makes a number of
strange assumptions; among these, the existence of a dataset consisting of
Out-Of-Distribution samples to train a binary classifier [15]. To
comprehensively cover what is OOD, this dataset would have to contain samples
from the entirety of the space outside of the In-Distribution. If the ID
contained pictures of dogs, for example, this OOD dataset would have to
contain pictures of literally everything that isn’t a dog: cats, foxes,
centaurs, Gaussian noise, fruit, supernovae, cartographic maps of Poland, so
on and so forth. If any class of data is left out, then it is outside of the
distribution for which the detector is prepared, making it necessary to
perform OOD Detection in order to perform OOD Detection, leaving us back at
square one. Thus, any true OOD Detector must be trained in an Unsupervised
manner, without training on an OOD dataset.

Other methods, like ODIN [13], make significant progress towards achieving
this. ODIN operates by comparing a classification model’s output on an input
to its output when the input has a small adversarial perturbation, with the
output changing less for Out-Of-Distribution inputs than In-Distribution ones.
While this performs very well, it’s also much more computationally expensive,
requiring two forward passes and a gradient calculation, than single-shot
inference, so it’s unsuited to real-time applications. Also worth noting is
that, although the model may be trained without OOD samples, the analysis of
the output distributions required to differentiate between ID and OOD data
requires at least a few samples. But this is a much weaker requirement than a
comprehensive OOD dataset, as it’s only needed in order to determine a handful
of parameters, rather than to train a model that takes them as input. As a
result, ODIN can be assumed to generalize much more reasonably.

Another method, DAGMM [24], involves using a Gaussian Mixture Model on the
latent space of a Variational Auto-Encoder [7]. While a classical model like a
Gaussian Mixture would be unable to deal with the high dimensionality of
images, the low number of dimensions of the VAE’s encodings means that it can
handle much more abstract data than might otherwise be possible. This still
makes a number of assumptions about the underlying distribution, including
that the probability density of the latent space can be represented
meaningfully by a Gaussian Mixture Model. It also imposes both a limit on the
number of dimensions that can be used in the latent space, and a requirement
that this number of dimensions stays constant between training, testing, and
real-world inference. This is disappointing, as ordinarily fully convolutional
models can be trained on smaller chips than they will be used for, making
training easier. However, using VAEs for OOD Detection does get at the heart
of the matter, because the underlying problem involves getting a handle on
patterns in the In-Distribution, which is fundamental to how VAEs operate.

| ODIN | DAGMM | DOODLER
---|---|---|---
Unsupervised | $\checkmark$ | $\checkmark$ | $\checkmark$
Chip Size Invariant | $\checkmark$ |  | $\checkmark$
One-Shot Inference |  | $\checkmark$ | $\checkmark$
Segmentation |  |  | $\checkmark$
Table 1: Feature Comparison between ODIN, DAGMM, and DOODLER

Something else worth considering is the possibility of performing segmented
Out-Of-Distribution Detection on images, to help with explainability; an OOD
Detector would be much more trustworthy, and provide helpful information to a
Human-in-the-Loop counterpart, if it could point out exactly where in an image
it was seeing something that set it off. However, the work in this area has
relied extensively on the existence of OOD datasets [2, 5, 20], so it could
still benefit greatly from moving to an Unsupervised setting.

But on a fundamental level, much of the available Unsupervised Out-Of-
Distribution Detection research comes at it from an analytic rather than a
functional perspective. Broadly, the work on the topic has thought of the
defining trait of OOD data to be that they literally fall outside of the high-
density region of a probabilistic In-Distribution. However, when looking at it
from a functional perspective, the primary trait of OOD data is that they
cause models to fail and fail confidently, this being the reason why it is
valuable to detect them in the first place. By shifting over to this
framework, a new methodology immediately presents itself: find some task for
which a failure state can be detected without a ground truth label for
comparison, and then use that failure state as a detection signal. Thus,
DOODLER: Determining Out-Of-Distribution Likelihood from Encoder
Reconstructions.

##  4 Methodology

The DOODLER methodology we propose, illustrated in Figure 1, involves training
a Variational Auto-Encoder on the same training data as the Deep Learning
model you want to perform Out-Of-Distribution Detection for, and then setting
conditions on the reconstruction error under which you declare inputs to be
OOD. Compared to its closest relatives, ODIN and DAGMM, DOODLER is similarly
unsupervised, but improves on DAGMM by being invariant to chip size, improves
on ODIN by only requiring one forward pass per input, and improves on them
both by enabling image segmentation, illustrating which regions specifically
are OOD within an image.

###  4.1 Variational Auto-Encoder

Per [7], let’s assume that a given sample $x_{i}$ is generated from a process
involving a lower-dimensional latent representation $z_{i}$. Where $x_{i}$
belongs to an high-dimensional field $\mathcal{X}$, $z_{i}$ belongs to a low-
dimensional field $\mathcal{Z}$, such that
$\dim(\mathcal{X})>>\dim(\mathcal{Z})$. If $x_{i}$ is, say, a three-channel
RGB $H\times W$ picture of a dog, then
$x_{i}\in\mathcal{X}=\mathbb{R}^{3\times H\times W}$, while
$z_{i}\in\mathcal{Z}=\mathbb{R}^{m}$ is a vector in “Dog-Space,” with $m$
dimensions describing higher-order concepts like fur length, appearance,
posture, and breed.

This allows us to define the process $f:\mathbb{Z}\to\mathbb{X}$ by which
$z_{i}$ becomes $x_{i}$, by saying that $f(z_{i})\coloneqq x_{i}$. In the
context of dog photos, $f$ would involve taking the vector description $z_{i}$
of a dog, and using it to produce a picture of the dog described. Furthermore,
we might hypothesize that $f$ is invertible, with inverse $g$, such that
$f\big{(}g(x_{i})\big{)}=x_{i}$. Here, $g$ would start by taking a picture of
a dog, and then producing the vector description of that dog, with the
condition that this vector can be used by $f$ to reconstruct the original
image of the dog.

However, while $\mathcal{X}$, containing photographs, can be reasonably
understood by humans, the field $\mathcal{Z}$, containing the perfect
description of dogs, and the process $f$, by which a dog is brought into
existence and photographed, are almost certainly inscrutable. What higher-
order concepts might a human overlook when discussing a dog or its appearance?
How could they precisely describe the process by which a dog might be drawn?
So, $\mathcal{Z}$ and $f$ will instead be approximated via a learning
algorithm, using $\hat{\mathcal{Z}}=\mathbb{R}^{m}$ with some manually
selected $m$ as a stand-in for $\mathcal{Z}$, and a neural network $\hat{f}$
parameterized with $\theta_{\hat{f}}$ of the form
$\hat{f}(\theta_{\hat{f}};\hat{z}_{i})=\hat{x}_{i}$. This will allow for the
model $\hat{f}(\theta_{\hat{f}};\cdot)$ to be created automatically by
training it on data, and together with a second neural network of the form
$\hat{g}(\theta_{\hat{g}};x_{i})=\hat{z}_{i}$ to invent a meaningful version
of $\hat{\mathcal{Z}}$. In a sense, by training these two neural networks on
photographs of dogs, the requirement that
$\dim(\mathcal{X})>>\dim(\hat{\mathcal{Z}})$ means that
$\hat{g}(\theta_{\hat{g}};\cdot)$ and $\hat{f}(\theta_{\hat{f}};\cdot)$ will
have to learn to produce a meaningful description of a dog’s fur or posture in
a higher-order, lower-dimensional setting than a picture, and to then re-draw
that picture based on that description, respectively.

By composing $\hat{f}(\theta_{\hat{f}};\cdot)$ and
$\hat{g}(\theta_{\hat{g}};\cdot)$, they can be referred to as a single model,
the Variational Auto-Encoder
$h(\theta;\cdot)\coloneqq\hat{f}\big{(}\theta_{\hat{f}};\hat{g}(\theta_{\hat{g}};\cdot)\big{)}$,
with $\theta=\theta_{\hat{f}}\cup\theta_{\hat{g}}$. In order to train this
combined model $h(\theta;\cdot)$, we will use a training dataset
$\mathcal{D}_{IDtrain}\coloneqq\\{x_{i}\\}\sim\mathscr{D}_{ID}$, which
consists of samples from a distribution over the input field $\mathcal{X}$,
and then attempt to minimize
$\mathbb{E}\Big{[}\mathcal{L}\big{(}h(\theta;x_{i}),x_{i}\big{)}\Big{]}$ for
some loss or reconstruction error function $\mathcal{L}(\hat{x}_{i},x_{i})$.
To use DOODLER to detect Out-Of-Distribution inputs for an application,
$\mathcal{D}_{IDtrain}$ would be the same dataset that the application model
was trained on - here, it would contain pictures of dogs - and $\mathcal{L}$
might be something like pixel-wise Mean Squared Error, which we will justify
using in the next section. To optimize the parameters of the VAE, an algorithm
like Adam Optimizer might be used, denoted as $\mathcal{A}lg$. This training
procedure is given by Algorithm 1.

Result: A Variational Auto-Encoder $h(\theta;\cdot)$

Initialize $\theta_{0}$ randomly, $t=1$;

while  _$||\theta_{t}-\theta_{t-1}|| >\epsilon$_ do

$X_{t}\leftarrow\\{x_{0},x_{1},\dots,x_{n}\\}\sim\mathcal{D}_{IDtrain}$;

$\hat{X}_{t}\leftarrow\\{\hat{x}_{0},\hat{x}_{1},\dots,\hat{x}_{n}\\}=\\{h(\theta_{t};x_{i})|x_{i}\in
X_{t}\\}$;

$G_{t}=\nabla_{\theta}\frac{1}{n}\sum_{i=0}^{n}\mathcal{L}(\hat{x}_{i},x_{i})$;

$\theta_{t+1}\leftarrow\mathcal{A}lg(\theta_{t},G_{t})$;

$t\leftarrow t+1$;

end while

Algorithm 1 Variational Auto-Encoder Training Procedure

This produces a model $h(\theta;\cdot)$ that, in some sense, has an
understanding of the structure of $\mathscr{D}_{ID}$. If this understanding
was not present, then $\hat{g}(\theta_{\hat{g}};\cdot)$ would not be able to
encode an input from $\mathscr{D}_{ID}$ into a lower dimension and then have
$\hat{f}(\theta_{\hat{f}};\cdot)$ successfully reconstruct it, so they must be
leveraging context and patterns within the data to compress and recover
information. Because the lower dimension of the compressed state means that
some information will be lost, the information that is preserved must be
capable of being understood in such a way as to fill in most of the missing
information. If trained on pictures of dogs, by understanding what dogs look
like $\hat{g}(\theta_{\hat{g}};\cdot)$ might encode information like “the tail
is at this location, and is upright with long golden-brown fur” in order for
$\hat{f}(\theta_{\hat{f}};\cdot)$ to reconstruct the pixel information of the
region containing the tail.

However, since this model operates entirely on the basis of understanding the
structure of $\mathscr{D}_{ID}$, if it was provided with $x_{i}$ sampled from
somewhere else, possessing structures and patterns that differ greatly from
$\mathscr{D}_{ID}$, then it’s reasonable to assume that
$\mathcal{L}\big{(}h(\theta;x_{i}),x_{i}\big{)}$ would be relatively large.
For example, if a photograph of an airplane was put into the Variational Auto-
Encoder trained on pictures of dogs, we could guess that
$\hat{g}(\theta_{\hat{g}};\cdot)$ would not be able to successfully encode the
appearance of a jet engine into dimensions like “fur length” and “ear size,”
instead producing numerical gibberish. From there,
$\hat{f}(\theta_{\hat{f}};\cdot)$ would attempt to turn that gibberish into a
picture of some kind of weird dog, resulting in significant error when
compared to the original airplane.

###  4.2 Reconstruction Error

Within the framework of [7], a “canonical” Variational Auto-Encoder consists
of two parts, $p(\theta_{p};\cdot|\cdot)$ and $q(\theta_{q};\cdot|\cdot)$,
which are conditional probability estimators, with
$q(\theta_{q};\hat{z}_{i}|x_{i})$ being the estimated probability that
$\hat{z}_{i}$ is the true encoding of $x_{i}$, and
$p(\theta_{p};x_{i}|\hat{z}_{i})$ being the estimated probability of receiving
$x_{i}$ given the encoding $\hat{z}_{i}$. This means that training
$p(\theta_{p};\cdot|\cdot)$ and $q(\theta_{q};\cdot|\cdot)$ involves reducing
the degree to which they disagree for any given sample $x_{i}$. For example,
if $q(\theta_{q};\hat{z}_{i}|x_{i})$ is high, meaning
$q(\theta_{q};\cdot|\cdot)$ estimates that $\hat{z}_{i}$ is likely the true
encoding of $x_{i}$, then $p(\theta_{p};x_{i}|\cdot)$ should be low everywhere
besides $\hat{z}_{i}$. This disagreement over $\hat{z}_{i}$ when considering
$x_{i}$ is expressed as the Kullback-Leibler Divergence:

$D_{KL}(\theta;x_{i})=\mathbb{E}_{q(\theta_{q};\hat{z}_{i}|x_{i})}\big{[}\ln
p(\theta_{p};x_{i}|\hat{z}_{i})-\ln q(\theta_{q};\hat{z}_{i}|x_{i})\big{]}$

However, actually estimating $D_{KL}(x_{i})$ and its gradients is
computationally infeasible. Going forward, we’ll be abbreviating the notation
using $p=p(\theta_{p};x_{i}|\hat{z}_{i})$ and
$q=q(\theta_{q};\hat{z}_{i}|x_{i})$, to make further algebraic work more
legible. By using the same assumption as [7] that the distributions over
$\hat{z}_{i}$ represented by $p$ and $q$ approximate multivariate Gaussian
distributions, we get the closed form:111Here, as in standard Statistical
notation, $\Sigma$ is the covariance matrix and $\mu$ is the vector-valued
mean of a multivariate distribution. Additionally, ‘$\operatorname{tr}$’ and
‘$\det$’ are the trace and determinant of a matrix, and ‘$\dim$’ is the
dimension of a field.

$D_{KL}(\theta;x_{i})=\frac{1}{2}\Bigg{(}\operatorname{tr}\big{(}\Sigma_{p}^{-1}\Sigma_{q}\big{)}+(\mu_{p}-\mu_{q})^{T}\Sigma_{p}^{-1}(\mu_{p}-\mu_{q})-\dim(\mathcal{\hat{Z}})+\ln\Big{(}\frac{\det\Sigma_{p}}{\det\Sigma_{q}}\Big{)}\Bigg{)}$

Next, it’s worth noting that a nonzero covariance between different dimensions
in $\mathcal{\hat{Z}}$ in either distribution would mean that the same
information was being stored in multiple entries in $\hat{z}_{i}$. Because the
number of entries is very limited, this repetition in storage is expected to
disappear, and the covariance matrices will become approximately diagonal. By
replacing the covariance matrices $\Sigma$ with just their diagonal entries in
the form of variance vectors $\sigma^{2}$, and by removing the constant due to
the dimension of $\mathcal{\hat{Z}}$, we can approximate and simplify this
expression without seriously affecting the future gradients with respect to
$\theta$:

$\widetilde{D}_{KL}(\theta;x_{i})=\frac{1}{2}\Big{(}\sigma_{p}^{-2}\cdot\sigma_{q}^{2}+\sigma_{p}^{-2}\cdot(\mu_{p}-\mu_{q})^{2}+||\ln\sigma_{p}^{2}||_{1}-||\ln\sigma_{q}^{2}||_{1}\Big{)}$

Lastly, we can insert a normalization step into the modeling process itself,
forcibly setting the variances along each dimension to 1. This guarantee
reduces the entire expression to:

$\widetilde{D}_{KL}(\theta;x_{i})=\frac{1}{2}||\mu_{p}-\mu_{q}||_{2}^{2}$

What this means is that, effectively, when given $x_{i}$, the gradient
behavior of the Kullback-Leibler Divergence between
$p(\theta_{p};x_{i}|\cdot)$ and $q(\theta_{q};\cdot|x_{i})$ is similar to that
of the $L2$ distance between their means in $\mathcal{\hat{Z}}$. By moving
back into thinking in terms of functions, with
$\hat{f}(\theta_{\hat{f}};\hat{z}_{i})=\hat{x}_{i}$ and
$\hat{g}(\theta_{\hat{g}};x_{i})=\hat{z}_{i}$, we can use the local continuity
of $\hat{f}(\theta_{\hat{f}};\cdot)$ and $\hat{g}(\theta_{\hat{g}};\cdot)$ to
get:

$||\hat{f}^{-1}(\theta_{\hat{f}};x_{i})-\hat{g}(\theta_{\hat{g}};x_{i})||_{2}^{2}\propto||\hat{f}(\theta_{\hat{f}};\hat{g}(\theta_{\hat{g}};x_{i}))-x_{i}||_{2}^{2}$

Meaning that it’s possible to relate the Kullback-Leibler Divergence over
$\mathcal{\hat{Z}}$ to the Mean Squared Error in $\mathcal{X}$, reducing to a
loss function given by:

$\mathcal{L}(\hat{x}_{i},x_{i})=\frac{||\hat{x}_{i}-x_{i}||^{2}_{2}}{\dim(\mathcal{X})}$
(1)

Which satisfies the condition that the value of $\hat{x}_{i}$ which minimizes
$\mathcal{L}(\hat{x}_{i},x_{i})$ is the same as the mean of
$p(\theta_{p};\cdot)$ given $q(\theta_{q};\hat{z}_{i}|x_{i})$ with optimal
$\theta$, or

$\operatorname*{argmin}_{\hat{x}_{i}\in\mathcal{X}}\mathcal{L}(\hat{x}_{i},x_{i})=\mu\bigg{(}p\Big{(}\theta_{p};x_{i}|q(\theta_{q};\hat{z}_{i}|x_{i})\Big{)}\bigg{|}\theta=\operatorname*{argmin}_{\theta\in\Theta}D_{KL}(\theta,x_{i})\bigg{)}$

Therefore, the mean of the squared differences between $x_{i}$ and
$\hat{x}_{i}=h(\theta;x_{i})$ has gradient behavior that is approximately
equivalent to that of the Kullback–Leibler formulation, and can be used both
as a loss function for the direct optimization of the Variational Auto-Encoder
for Algorithm 1, and as the reconstruction error during inference.

##  5 Error Distribution Analyses

Figure 2: Distribution of Pixel-Wise Differences between example $x_{i}$ and
$\hat{x}_{i}$

In order to find the best rules to use to declare samples to be Out-Of-
Distribution, we have to know the probability distributions of the
reconstruction errors. As we see in Figure 2222This histogram was produced
with 80 samples each from the testing sets of CIFAR-100 and TinyImagenet,
using a VAE trained on CIFAR-100. This behavior, with the pixel-wise
differences approximating Gaussian distributions, is general between different
datasets., the pixel-wise differences between $x_{i}$ and $\hat{x}_{i}$
approximate a Gaussian distribution. As a result, the reconstruction error of
an image, given by
$l=\mathcal{L}(\hat{x}_{i},x_{i})=\frac{1}{\dim(\mathcal{X})}||\hat{x}_{i}-x_{i}||^{2}_{2}$
from Equation 1, being the mean of the squares of values sampled from what is
approximately Gaussian, approximates a Gamma distribution.333Formally, a Gamma
distribution represents the probability of receiving a given sum when adding
the squares of a known number of values sampled from a Gaussian distribution.
But, because a mean is the sum over a set divided by its size, and a Gaussian
divided by a constant is another Gaussian, the mean of the squares follows a
Gamma distribution, the same as their sum. This aligns well with the Gamma
distribution-esque appearances of the histograms of image-wise reconstruction
errors, as in Figure 6.

Taking the $\alpha,\beta$ formulation of the Gamma distribution, we can give
the probability density function as:

$p(l)=\frac{\beta^{\alpha}}{\Gamma(\alpha)}l^{\alpha-1}e^{-\beta l}$

With $\Gamma$ being the Gamma function. Because the mean and the variance of
the Gamma distribution are known to be $\frac{\alpha}{\beta}$ and
$\frac{\alpha}{\beta^{2}}$ respectively, and because the mean and variance of
the distribution of $l$ can be approximated as $\mu_{l}=\mathbb{E}[l]$ and
$\sigma_{l}^{2}=\mathbb{E}[l^{2}]-\mathbb{E}[l]^{2}$ to a high degree of
precision with a large enough number of samples, $\alpha$ and $\beta$ can be
solved for using the Method of Moments:

$\mu_{l}=\frac{\alpha}{\beta},\
\sigma_{l}^{2}=\frac{\alpha}{\beta^{2}}\rightarrow\alpha=\frac{\mu_{l}^{2}}{\sigma_{l}^{2}},\
\beta=\frac{\mu_{l}}{\sigma_{l}^{2}}$

Given a VAE $h(\theta;\cdot)$ trained on the training set
$\mathcal{D}_{IDtrain}$ sampled from $\mathscr{D}_{ID}$, it is now possible to
produce a distribution $p(l|x_{i}\sim\mathscr{D}_{ID})$, the probability
density of receiving $l$ as the reconstruction error if we know that $x_{i}$
was sampled from the In-Distribution. We do this by taking a test set
$\mathcal{D}_{IDtest}$ that has also been sampled from $\mathscr{D}_{ID}$,
calculating the mean and variance of $l$ over $\mathcal{D}_{IDtest}$, and then
using them to solve for $\alpha_{ID}$ and $\beta_{ID}$. Using the same method,
but a test set $\mathcal{D}_{OODtest}$ that has been sampled from
$\mathscr{D}_{OOD}$, we can solve for $\alpha_{OOD}$ and $\beta_{OOD}$, which
parameterize $p(l|x_{i}\sim\mathscr{D}_{OOD})$, the probability density of $l$
when $x_{i}$ was sampled from Out-Of-Distribution.

###  5.1 Out-Of-Distribution Sample Detection

True Positive |  $l>T_{v}$ and $x_{i}\not\sim\mathscr{D}_{ID}$
---|---
False Positive |  $l>T_{v}$ and $x_{i}\sim\mathscr{D}_{ID}$
False Negative |  $l<T_{v}$ and $x_{i}\not\sim\mathscr{D}_{ID}$
True Negative |  $l<T_{v}$ and $x_{i}\sim\mathscr{D}_{ID}$
Table 2: Conditions under which a sample $x_{i}$ with reconstruction error $l$ is put in each category. $\mathcal{TP}(T_{v})$ | $=$ | $p(x_{i}\not\sim\mathscr{D}_{ID})\int_{l>T_{v}}p(l|x_{i}\not\sim\mathscr{D}_{ID})\partial l$
---|---|---
$\mathcal{FP}(T_{v})$ | $=$ | $p(x_{i}\sim\mathscr{D}_{ID})\int_{l>T_{v}}p(l|x_{i}\sim\mathscr{D}_{ID})\partial l$
$\mathcal{FN}(T_{v})$ | $=$ | $p(x_{i}\not\sim\mathscr{D}_{ID})\int_{l<T_{v}}p(l|x_{i}\not\sim\mathscr{D}_{ID})\partial l$
$\mathcal{TN}(T_{v})$ | $=$ | $p(x_{i}\sim\mathscr{D}_{ID})\int_{l<T_{v}}p(l|x_{i}\sim\mathscr{D}_{ID})\partial l$
Table 3: Calculating proportions of the data $Sensitivity(T_{v})$ | $=$ | $\frac{\mathcal{TP}(T_{v})}{\mathcal{TP}(T_{v})+\mathcal{FN}(T_{v})}$
---|---|---
$Specificity(T_{v})$ | $=$ | $\frac{\mathcal{TN}(T_{v})}{\mathcal{TN}(T_{v})+\mathcal{FP}(T_{v})}$
$\mathcal{PPV}(T_{v})$ | $=$ | $\frac{\mathcal{TP}(T_{v})}{\mathcal{TP}(T_{v})+\mathcal{FP}(T_{v})}$
$\mathcal{NPV}(T_{v})$ | $=$ | $\frac{\mathcal{TN}(T_{v})}{\mathcal{TN}(T_{v})+\mathcal{FN}(T_{v})}$
Table 4: The values derived from the proportions of the data.

If some prior probabilities
$p(x_{i}\sim\mathscr{D}_{ID})+p(x_{i}\not\sim\mathscr{D}_{ID})=1$ are assumed,
as well as the existence of a relatively small Out-Of-Distribution dataset
$\mathcal{D}_{OODtest}$ sampled from $\mathscr{D}_{OOD}$ to determine
$p(l|x_{i}\sim\mathscr{D}_{OOD})$, it is possible to use Bayes’ Theorem to
compute $p(x_{i}\sim\mathscr{D}_{ID}|l)$ with

$p(x_{i}\sim\mathscr{D}_{ID}|l)=\frac{p(l|x_{i}\sim\mathscr{D}_{ID})\cdot
p(x_{i}\sim\mathscr{D}_{ID})}{p(l|x_{i}\sim\mathscr{D}_{ID})\cdot
p(x_{i}\sim\mathscr{D}_{ID})+p(l|x_{i}\sim\mathscr{D}_{OOD})\cdot
p(x_{i}\not\sim\mathscr{D}_{ID})}$

At which point $p(x_{i}\sim\mathscr{D}_{ID}|l)$ going below some threshold
probability $T_{p}$ would be used to declare a positive Out-Of-Distribution
Detection. However, it must be noted that this does rely on both the existence
of $\mathcal{D}_{OODtest}$, and on the approximate equivalence between
$p(l|x_{i}\sim\mathscr{D}_{OOD})$ and $p(l|x_{i}\not\sim\mathscr{D}_{ID})$.
But, these are much weaker assumptions than those made by works like [16], as
we’re not using a dataset sampled from $\mathscr{D}_{OOD}$ for training
$h(\theta;\cdot)$, reducing the quantity of data needed. We only ever use
$\mathcal{D}_{OODtest}$ to approximate the parameters of Statistical
distributions, not to capture the actual underlying behavior of
$x_{i}\not\sim\mathscr{D}_{ID}$.

For a given threshold probability $T_{p}$, it’s possible for us to calculate
the expected True Positive, False Positive, False Negative, and True Negative
proportions of the inferences made. With “positive” meaning the presence of an
Out-Of-Distribution sample, these are the proportions of incoming data that
were sampled Out-Of-Distribution and correctly detected, that were sampled In-
Distribution and incorrectly detected as Out-Of-Distribution, that were
sampled Out-Of-Distribution and incorrectly ignored as In-Distribution, and
that were sampled In-Distribution and were correctly ignored, respectively.

Under the assumption that $p(l|x_{i}\sim\mathscr{D}_{OOD})$, the distribution
of reconstruction errors from the Out-Of-Distribution, is shifted to the right
of $p(l|x_{i}\sim\mathscr{D}_{ID})$, the distribution from the In-
Distribution, this begins by finding the matching threshold value $T_{v}$,
using:

$T_{v}=l|[p(x_{i}\sim\mathscr{D}_{ID}|l)=T_{p}]$

And then using the conditions from Table 2 to define when a sample will fall
into each category. These conditions lead to the formulae listed in Table 3,
which can be used to calculate the proportions of the data that will fall into
each category. Because those formulae, when applied to this situation, involve
integrating the probability density function of the Gamma distribution, it may
be useful to replace the integral with the cumulative density function:

$\int_{a}^{b}\frac{\beta^{\alpha}}{\Gamma(\alpha)}l^{\alpha-1}e^{-\beta
l}\partial l=\frac{1}{\Gamma(\alpha)}\gamma(\alpha,\beta
l)\Bigg{|}_{l=a}^{l=b}$

But this comes with the caveat that $\gamma$ is itself defined as the integral
that satisfies this condition, rather than it being possible to reduce the
integral to elementary functions. Next, having determined those proportions,
we can calculate the Sensitivity, Specificity, Positive Predictive Value, and
Negative Predictive Value, using the relationships from Table 4. Those values
in turn can be used to produce the Receiver Operating Characteristic ($ROC$)
curve. This curve is generated by pairs of the False Positive Rate, or “Fall-
Out,” and the True Positive Rate, or “Recall,” taken at the same threshold
value. These are not the False Positive Proportion and True Positive
Proportion from Tables 2 and 3, but rather the rates at which negative cases
are declared positive and positive cases are declared positive, respectively.
The False Positive Rate/Fall-Out, here $f$, is the probabilistic inverse of
Specificity, and the True Positive Rate/Recall, here $r$, is equivalent to
Specificity. These are used to produce the $ROC$ curve:

$ROC=\\{(f,r)=\big{(}1-Specificity(T_{v}),Sensitivity(T_{v})\big{)}|T_{v}\in\mathbb{R}\\}$

Which may be turned into a functional form:

$ROC(f)=r|(f,r)\in ROC$

And then integrated, to retrieve the Area Under the ROC Curve (AUROC):

$AUROC=\int_{0}^{1}ROC(f)\partial f$

This integral itself being possible to approximate computationally by using an
interpolation method on a finite subset of $ROC$ to yield
$\widetilde{ROC}(f)$. Getting this number, the AUROC, is useful, as it
represents the model’s trade-off between the True Positive Rate and the False
Positive Rate at different thresholds. A high AUROC means that the model can
have both a high True Positive Rate and a low False Positive Rate at the same
time, indicating that the model performs well. A similar method can be used to
calculate the Area Under the Precision-Recall Curve, which measures how the
model trades off between minimizing false positives, and minizing false
negatives.

###  5.2 Out-Of-Distribution Pixel Detection

One of the most important usecases of Out-Of-Distribution Detection involves
alerting a human operator that an OOD input has been detected, so that the
human can make a more sensible decision than a model would be able to. What
might be helpful for a human analyst is to understand what, specifically,
about the input caused it to declared OOD. Helpfully, DOODLER can be used to
produce a segmentation over an image, based on how likely it is that each
individual pixel belongs to an OOD input.

Determining the probability of a given pixel having been sampled OOD based on
the reconstruction error falls along similar lines to doing for an entire
sample. Collect pixel-wise reconstruction errors from both In-Distribution and
Out-Of-Distribution samples, determine the parameters of the $\chi^{2}$
distributions that produce them, and then use Bayes’ Theorem again. For a
given pixel reconstruction error $t$ produced by a pixel $v_{i}$:

$p(v_{i}\sim\mathscr{D}_{ID}|t)=\frac{p(t|v_{i}\sim\mathscr{D}_{ID})\cdot
p(v_{i}\sim\mathscr{D}_{ID})}{p(t|v_{i}\sim\mathscr{D}_{ID})\cdot
p(v_{i}\sim\mathscr{D}_{ID})+p(t|v_{i}\sim\mathscr{D}_{OOD})\cdot
p(v_{i}\not\sim\mathscr{D}_{ID})}$

We can use $\chi^{2}$ distributions to approximate
$p(t|v_{i}\sim\mathscr{D}_{ID})$ and $p(t|v_{i}\sim\mathscr{D}_{OOD})$,
because they’re the distributions of the squares of values from a single
Gaussian. The $\chi^{2}$ distribution only has a single parameter, $\alpha$,
which is the same as its mean, and a probability density function given by:

$p(t)=\frac{1}{2^{\alpha/2}\Gamma(\alpha/2)}t^{\alpha/2-1}e^{-t/2}$

###  5.3 Out-Of-Distribution Stream Detection

$H_{0}:$ | $\mathbb{E}[l|x_{i}\sim S]\leq\mathbb{E}[l|x_{i}\sim\mathscr{D}_{ID}]$
---|---
$H_{a}:$ | $\mathbb{E}[l|x_{i}\sim S]>\mathbb{E}[l|x_{i}\sim\mathscr{D}_{ID}]$
Table 5: Hypotheses for testing if a given stream is sampling Out-Of-
Distribution.

Compared to sample detection, stream detection doesn’t even require a dataset
$\mathcal{D}_{OODtest}$ or overly restrictive assumptions about
$x_{i}\not\sim\mathscr{D}_{ID}$, as we can do it entirely through the use of
Hypothesis Testing.

For this setting, let’s say that there’s a stream of data $S$ from which
$x_{i}$ is being sampled, like a sensor in a new location. Then the
reconstruction error $l$ for a given sample is the result of comparing the
output of the trained model $\hat{x}_{i}=h(\theta;x_{i})$ to the sample,
$l=\mathcal{L}(\hat{x}_{i},x_{i})$. By assuming that the expected error for
Out-Of-Distribution samples is greater than it is for In-Distribution samples,
the Null Hypothesis, $H_{0}$, and the Alternative Hypothesis, $H_{a}$, can be
taken according to Table 5. Essentially, we start by assuming $H_{0}$, that
$S$ is producing ID data, with low reconstruction errors. Under this
assumption, we can calculate the probability that we’d see the mean
reconstruction error that $S$ has given us so far. If this probability is too
low, we can reject $H_{0}$, saying that it’s an unreasonable assumption in
light of the evidence, and accept $H_{a}$, declaring that $S$ is sampling from
Out-Of-Distribution, because it’s producing higher reconstruction errors.

The Central Limit Theorem gives us that, if we repeatedly take a number of
samples from a population and calculate their mean, the distribution of the
means over the repetitions approaches a normal distribution, allowing us to
use Gosset’s/Student’s $t$-test.444This Statistical test, involving using a
test statistic $t$ based on the number of standard errors that a sample mean
differs from its expectated value if it was produced by a known population,
was originally developed by Brewer and Statistician William Sealy Gosset,
while working at Guinness. At the time, this new test allowed brewers to draw
meaningful conclusions about new strains of barley or Chemical processes from
smaller sample sizes. Because use of this new test for brewing constituted a
trade secret, and Guinness required their researchers to write under
pseudonyms, Gosset published his work under the name “Student,” after which
his $t$-test and $t$-distribution are named To start, take a set of $n$
samples:

$\psi\coloneqq\\{x_{1},x_{2},\dots,x_{n}\\}\sim S$

And take the difference between the mean reconstruction error from this set
$\psi$, and the expected reconstruction error $\mu_{l}$ from the In-
Distribution $\mathscr{D}_{ID}$:

$Z\coloneqq\Bigg{(}\frac{1}{n}\sum_{x_{i}\in\psi}\mathcal{L}\big{(}h(\theta;x_{i}),x_{i}\big{)}\Bigg{)}-\mu_{l}$

From there, take the standard error of the mean over $n$ samples, using the
standard deviation $\sqrt{\sigma_{l}^{2}}=\sigma_{l}$ of the In-Distribution
$\mathscr{D}_{ID}$:

$s\coloneqq\frac{\sigma_{l}}{\sqrt{n}}$

Which lets us calculate the test statistic:

$t=\frac{Z}{s}$

And then we can choose to reject the Null Hypothesis depending on whether or
not $t$ exceeds the $Z$-score, the number of standard errors required to
produce this result if the expected mean of the samples was equal to the mean
of the population, corresponding to the significance level, or $P$-value.

Under the hypothesis that $S$ really was sampling from $\mathscr{D}_{ID}$, we
would expect the mean reconstruction error yielded from $n$ samples from $S$
would follow a normal distribution centered around the expected reconstruction
error $\mu_{l}$ from $\mathscr{D}_{ID}$. This test statistic $t$ then tells us
how many standard deviations above the expected reconstruction error the mean
reconstruction error of your samples is. By using that number of standard
deviations, together with the cumulative density of the normal distribution,
we can determine the likelihood of receiving a value for $t$ as or more
extreme than the one that appeared. If this likelihood is too low, say ¡1%, we
reject the Null Hypothesis, and say that something must be increasing the
reconstruction errors of the data from $S$, namely, that they’re OOD.

##  6 Experimentation

FPR at Detection AUPR AUPR ID OOD 95% TPR $\downarrow$ Error $\downarrow$
AUROC $\uparrow$ Out $\uparrow$ In $\uparrow$ ODIN/DAGMM/DOODLER MNIST F-MNIST
0.49/0.35/0.19 0.27/0.2/0.12 0.89/0.92/0.96 0.89/0.92/0.96 0.88/0.92/0.96

Table 6: Example Experimental Result

Following other papers on the subject [1, 12], our experiments were done by
selecting two available Computer Vision datasets, and declaring one to be the
In-Distribution, and the other Out-Of-Distribution. Then a VAE was trained on
the training portion of the ID dataset, and used to classify samples from the
testing portions of both the ID and OOD datasets. For example, Table 6 shows
an experiment where a VAE was trained on the training portion of the MNIST
dataset, and then was tasked with classifying testing MNIST samples as In-
Distribution, versus Fashion-MNIST samples as Out-Of-Distribution. The metrics
shown will be explained in the Results and Comparisons section.

The datasets we selected include MNIST [11], Fashion-MNIST [21], Omniglot [9],
CIFAR-100 [8], TinyImagenet [3, 10], SVHN [17], and CelebA [14]. These
datasets were collected together into “simple” and “complex” groups, with
MNIST, F-MNIST, and Omniglot in the former, and CIFAR-100, TinyImagenet, SVHN,
and CelebA in the latter. Each dataset had its own VAE trained for it, which
was then tested with DOODLER for its ability to detect samples from every
other dataset in the group as Out-Of-Distribution. Additionally, in both
groups, random Gaussian noise clipped to $[0,1]$ and Uniform noise from
$[0,1]$ were added as purely OOD samples.

Implementations of ODIN and DAGMM were used as comparisons, with models
similar in capability and with equivalent training regimens to the VAEs used
for DOODLER.

##  7 Results and Comparisons

FPR at Detection AUPR AUPR ID OOD 95% TPR $\downarrow$ Error $\downarrow$
AUROC $\uparrow$ Out $\uparrow$ In $\uparrow$ ODIN/DAGMM/DOODLER MNIST F-MNIST
0.49/0.35/0.19 0.27/0.2/0.12 0.89/0.92/0.96 0.89/0.92/0.96 0.88/0.92/0.96
Omniglot 0.33/0.21/0.07 0.19/0.13/0.06 0.93/0.96/0.98 0.93/0.97/0.98
0.93/0.96/0.98 Gaussian 0.12/0.17/0.01 0.09/0.11/0.03 0.98/0.97/1.0
0.98/0.97/1.0 0.98/0.97/1.0 Uniform 0.12/0.17/0.02 0.08/0.11/0.03
0.98/0.96/1.0 0.98/0.96/1.0 0.98/0.96/0.99 F-MNIST MNIST 0.49/0.23/0.01
0.27/0.14/0.03 0.88/0.95/1.0 0.88/0.95/1.0 0.88/0.95/1.0 Omniglot
0.59/0.18/0.0 0.32/0.12/0.03 0.85/0.97/1.0 0.86/0.97/1.0 0.85/0.96/1.0
Gaussian 0.02/0.02/0.02 0.03/0.04/0.03 0.99/1.0/0.99 0.99/0.99/0.99
0.99/0.99/0.99 Uniform 0.02/0.01/0.03 0.04/0.03/0.04 0.99/1.0/0.99
0.99/1.0/0.99 0.99/1.0/0.99 Omniglot MNIST 0.22/0.43/0.04 0.14/0.24/0.04
0.96/0.89/0.99 0.95/0.9/0.99 0.96/0.89/0.99 F-MNIST 0.26/0.56/0.05
0.16/0.3/0.05 0.94/0.87/0.99 0.95/0.88/0.99 0.94/0.86/0.99 Gaussian
0.03/0.08/0.0 0.04/0.07/0.03 0.99/0.98/1.0 0.99/0.98/1.0 0.99/0.98/1.0 Uniform
0.03/0.06/0.0 0.04/0.05/0.03 0.99/0.99/1.0 0.99/0.99/1.0 0.99/0.99/1.0

Table 7: Result comparisons on visually simple datasets

FPR at Detection AUPR AUPR ID OOD 95% TPR $\downarrow$ Error $\downarrow$
AUROC $\uparrow$ Out $\uparrow$ In $\uparrow$ ODIN/DAGMM/DOODLER CIFAR-100
TinyImagenet 0.81/0.63/0.5 0.43/0.34/0.28 0.73/0.81/0.86 0.73/0.79/0.86
0.71/0.81/0.86 SVHN 0.84/0.93/0.8 0.44/0.49/0.42 0.68/0.57/0.71 0.66/0.55/0.69
0.67/0.55/0.7 CelebA 0.84/0.82/0.47 0.45/0.44/0.26 0.67/0.72/0.88
0.66/0.7/0.88 0.66/0.71/0.88 Gaussian 0.03/0.02/0.01 0.04/0.04/0.03
0.99/1.0/1.0 0.99/0.99/1.0 0.99/0.99/1.0 Uniform 0.02/0.02/0.0 0.04/0.04/0.03
0.99/1.0/1.0 0.99/0.99/1.0 0.99/0.99/1.0 TinyImagenet CIFAR-100 0.89/0.75/0.64
0.47/0.4/0.34 0.62/0.73/0.82 0.61/0.71/0.82 0.61/0.72/0.82 SVHN 0.78/0.79/0.67
0.41/0.42/0.36 0.74/0.74/0.78 0.73/0.74/0.78 0.72/0.73/0.77 CelebA
0.61/0.53/0.78 0.33/0.29/0.41 0.81/0.86/0.72 0.81/0.86/0.71 0.81/0.86/0.72
Gaussian 0.03/0.04/0.01 0.04/0.05/0.03 0.99/0.99/1.0 0.99/0.99/1.0
0.99/0.99/1.0 Uniform 0.03/0.03/0.01 0.04/0.04/0.03 0.99/0.99/1.0
0.99/0.99/1.0 0.99/0.99/1.0 SVHN CIFAR-100 0.58/0.45/0.32 0.32/0.25/0.19
0.85/0.9/0.93 0.86/0.9/0.92 0.84/0.9/0.93 TinyImagenet 0.23/0.41/0.14
0.14/0.23/0.1 0.95/0.91/0.97 0.95/0.91/0.97 0.95/0.91/0.97 CelebA
0.35/0.23/0.06 0.2/0.14/0.05 0.91/0.96/0.99 0.91/0.96/0.99 0.91/0.96/0.99
Gaussian 0.01/0.01/0.01 0.03/0.03/0.03 1.0/1.0/1.0 0.99/0.99/1.0 0.99/0.99/1.0
Uniform 0.02/0.02/0.01 0.03/0.04/0.03 1.0/0.99/1.0 1.0/0.99/1.0 1.0/0.99/1.0
CelebA CIFAR-100 0.86/0.78/0.9 0.46/0.41/0.47 0.67/0.73/0.6 0.67/0.72/0.6
0.65/0.72/0.59 TinyImagenet 0.64/0.54/0.73 0.35/0.29/0.39 0.81/0.85/0.75
0.8/0.84/0.75 0.8/0.85/0.75 SVHN 0.61/0.82/0.89 0.33/0.44/0.47 0.81/0.7/0.61
0.8/0.69/0.6 0.81/0.68/0.61 Gaussian 0.02/0.02/0.01 0.03/0.04/0.03 1.0/1.0/1.0
1.0/0.99/1.0 1.0/0.99/1.0 Uniform 0.02/0.02/0.01 0.03/0.03/0.03 1.0/1.0/1.0
1.0/0.99/1.0 1.0/0.99/1.0

Table 8: Results comparisons on more visually complex datasets

When doing analyses of the results, we assumed the prior probabilities
$p(x_{i}\sim\mathscr{D}_{ID}),\ p(x_{i}\not\sim\mathscr{D}_{ID})$ of receiving
an In-Distribution and an Out-Of-Distribution sample, respectively, to both be
$\frac{1}{2}$.

The metrics used are taken from [6, 12, 1]. They include:

  * •

The FPR at 95% TPR, which is the probability of an ID sample being
accidentally declared OOD when a real OOD sample has a 95% chance of being
detected. This should be minimized.

  * •

Detection Error is the over-all probability of any given sample being
misclassified when at a 95% True Positive Rate. This should be minimized.

  * •

Area Under the Receiver Operating Characteristic curve is the over-all
probability that an OOD sample receives a higher metric than an ID sample, in
the case of DOODLER a higher reconstruction error. This should be maximized.

  * •

Area Under the Precision-Recall curve represents balancing a low rate of false
positives with a high rate of collecting true positives. Both the AUPR
calculated with OOD samples being positive and ID samples being positive are
provided. Both should be maximized.

Looking at Tables 7 and 8, we can see that DOODLER competes very well against
ODIN and DAGMM, often superseding them. An example of this high performance is
seen in the graph of reconstruction errors on MNIST and Fashion-MNIST with a
VAE trained for MNIST, given in Figure 3, which highlights the differences
between the reconstruction error distributions. In that figure, we see very
little overlap, meaning that determining which inputs belong to which dataset
based on the reconstruction error yields good results. Similarly, Figure 4
shows the differences in the reconstruction errors between CIFAR-100 and
TinyImagenet with a VAE trained on CIFAR-100. While the differences are less
stark, they are still very significant, allowing DOODLER to reach an AUROC of
0.86.

Looking at Figures 5 and 6, which contain the $\chi^{2}$ distributions
regressed from the actual error contributions of individual pixels, shows us
that the actual difference between how accurate the VAE is on any given pixel
of an In-Distribution versus an Out-Of-Distribution input is relatively
subtle. Rather, the large differences in over-all image reconstruction error
are due to the small per-pixel differences adding up over a very large number
of pixels, producing very different per-image distributions.

Additionally, Figures 7 and 8 provide examples of inputs, reconstructions, and
the estimated per-pixel Out-Of-Distribution likelihoods from both In-
Distribution and OOD samples. Even though the reconstructions on the visually
complex datasets are quite cloudy, they do align relatively well with the
inputs in shape and color on the ID samples, while failing on the OOD samples,
producing significantly greater error. Looking at the pixel-wise OOD
likelihood estimation, it becomes clear to see that the bright spots align
themselves closely with features in the image, representing the objects that
the VAE was unfamiliar with from training.

This means that the intuitions developed earlier about the functionality of
DOODLER being based on a failure to reconstruct unseen data bear out
experimentally, because on a fundamental level, what is happening is that the
VAEs trained on the In-Distribution are failing to reconstruct objects they
haven’t seen before in the Out-Of-Distribution inputs. In a very real sense,
this means that DOODLER is performing Out-Of-Distribution Detection to the
letter, because it highlights objects that don’t exist within the In-
Distribution.

##  8 Conclusions and Future Work

While DOODLER represents a step forward for Out-Of-Distribution detection,
work remains to be done. This includes placing theoretical bounds on its error
rate, understanding the reconstruction error distributions more wholly than is
possible with Gamma distributions, and increasing the saliency of pixel-wise
segmentations. Additionally, it might be worthwhile to try and study the
connection between how the Variational Auto-Encoder builds its latent space,
and the resulting reconstruction errors, to help establish what exactly in the
sample has resulted in detection as an OOD input. It might also be worthwhile
to pair the VAE with some kind of Generative Adversarial framework [4], to try
and improve the sharpness of the reconstructions. It may also be possible to
further improve on OOD Detection by combining elements from multiple
methodologies, including DOODLER, or by using a an ensemble of them to cover
each other’s weaknesses.

There is also work to be done to study how DOODLER behaves when provided with
adversarial inputs. For example, can an adversarial attack actually decrease
the reconstruction error of an input? This could be a potential avenue of
attack, if an adversary wanted to provide an anomalous input without a human
operator being notified.

But, to conclude, DOODLER represents a novel framework for Out-Of-Distribution
Detection, introducing a functional perspective to the problem, improving on
aspects of earlier methods, and providing a basis on which to pursue
functionality like image segmentation. This constitutes a step forwards in the
pursuit of making Machine Learning robust and trustworthy for real-world
applications.

## References

  * [1] Vahdat Abdelzad, Krzysztof Czarnecki, Rick Salay, Taylor Denouden, Sachin Vernekar, and Buu Phan.  Detecting out-of-distribution inputs in deep neural networks using an early-layer output.  CoRR, abs/1910.10307, 2019.
  * [2] Petra Bevandić, Ivan Krešo, Marin Oršić, and Siniša Šegvić.  Discriminative out-of-distribution detection for semantic segmentation.  arXiv preprint arXiv:1808.07703, 2018.
  * [3] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [4] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  Advances in neural information processing systems, 27, 2014.
  * [5] Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song.  A benchmark for anomaly segmentation.  arXiv preprint arXiv:1911.11132, 2019.
  * [6] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  CoRR, abs/1610.02136, 2016.
  * [7] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes, 2014.
  * [8] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Cifar-100 (canadian institute for advanced research).
  * [9] Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum.  Human-level concept learning through probabilistic program induction.  Science, 350(6266):1332–1338, 2015.
  * [10] Ya Le and X. Yang.  Tiny imagenet visual recognition challenge.  2015\.
  * [11] Yann LeCun and Corinna Cortes.  MNIST handwritten digit database.  2010\.
  * [12] Shiyu Liang, Yixuan Li, and R. Srikant.  Principled detection of out-of-distribution examples in neural networks.  CoRR, abs/1706.02690, 2017.
  * [13] Shiyu Liang, Yixuan Li, and R. Srikant.  Enhancing the reliability of out-of-distribution image detection in neural networks, 2020.
  * [14] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [15] Alexander Meinke, Julian Bitterwolf, and Matthias Hein.  Provably robust detection of out-of-distribution data (almost) for free.  arXiv preprint arXiv:2106.04260, 2021.
  * [16] Alexander Meinke and Matthias Hein.  Towards neural networks that provably know when they don’t know.  arXiv preprint arXiv:1909.12180, 2019.
  * [17] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng.  Reading digits in natural images with unsupervised feature learning.  2011\.
  * [18] Tomaso Poggio, Hrushikesh Mhaskar, Lorenzo Rosasco, Brando Miranda, and Qianli Liao.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review.  International Journal of Automation and Computing, 14(5):503–519, 2017.
  * [19] Doyen Sahoo, Quang Pham, Jing Lu, and Steven CH Hoi.  Online deep learning: Learning deep neural networks on the fly.  arXiv preprint arXiv:1711.03705, 2017.
  * [20] David Williams, Matthew Gadd, Daniele De Martini, and Paul Newman.  Fool me once: Robust selective segmentation via out-of-distribution detection with contrastive learning.  arXiv preprint arXiv:2103.00869, 2021.
  * [21] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  CoRR, abs/1708.07747, 2017.
  * [22] Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn.  Adaptive risk minimization: A meta-learning approach for tackling group distribution shift.  arXiv preprint arXiv:2007.02931, 2020.
  * [23] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy.  Domain generalization: A survey.  arXiv preprint arXiv:2103.02503, 2021.
  * [24] Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen.  Deep autoencoding gaussian mixture model for unsupervised anomaly detection.  In International Conference on Learning Representations, 2018.

##  9 Example Figures

Figure 3: Reconstruction errors compared between the MNIST and Fashion-MNIST
datasets with a VAE trained on MNIST Figure 4: Reconstruction errors compared
between the CIFAR-100 and TinyImagenet datasets with a VAE trained on
CIFAR-100 Figure 5: Pixel-wise errors compared between the MNIST and Fashion-
MNIST datasets with a VAE trained on MNIST Figure 6: Pixel-wise errors
compared between the CIFAR-100 and TinyImagenet datasets with a VAE trained on
CIFAR-100

Figure 7: Comparison of reconstructions and pixel-wise OOD probabilities
between MNIST and Fashion-MNIST samples using a VAE trained on MNIST

Figure 8: Comparison of reconstructions and pixel-wise OOD probabilities
between CIFAR-100 and TinyImagenet samples using a VAE trained on CIFAR-100


Globerson and personRicardo Silva (Eds.).
publisherAUAI Press, addressCalifornia,

[Whitehead et al(2022)]
authorpersonSpencer Whitehead, personSuzanne
Petryk, personVedaad Shakib, personJoseph Gonzalez,
personTrevor Darrell, personAnna Rohrbach, and
personMarcus Rohrbach. year2022.
titleReliable Visual Question Answering: Abstain Rather
Than Answer Incorrectly.

[Wu and Suen(2001)]
authorpersonXuejing Wu and personC. Y.
Suen. year2001.
Injection of human knowledge into the rejection
criterion of a neural network classifier. In booktitleJoint
9th IFSA World Congress and 20th NAFIPS International Conference, July 25,
2001 - July 28, 2001 (seriesAnnual Conference of the North
American Fuzzy Information Processing Society - NAFIPS,
Vol. volume1). publisherInstitute of Electrical and
Electronics Engineers Inc., address-, pages499–505.

[Xia and Bouganis(2022)]
authorpersonGuoxuan Xia and
personChristos-Savvas Bouganis. year2022.
titleAugmenting Softmax Information for Selective
Classification with Out-of-Distribution Data.

[Yamaguchi and Murakawa(2021)]
authorpersonTakumi Yamaguchi and
personMasahiro Murakawa. year2021.
Mixup Gamblers: Learning to Abstain with
Auto-Calibrated Reward for Mixed Samples. In
booktitleArtificial Neural Networks and Machine Learning –
ICANN 2021, editorpersonIgor Farkaš,
personPaolo Masulli, personSebastian Otte, and
personStefan Wermter (Eds.). publisherSpringer
International Publishing, addressCham,

[Yang et al(2016)]
authorpersonFan Yang, personWongun Choi,
and personYuanqing Lin. year2016.
Exploit All the Layers: Fast and Accurate CNN
Object Detector with Scale Dependent Pooling and Cascaded Rejection
Classifiers. In booktitle29th IEEE Conference on Computer
Vision and Pattern Recognition, CVPR 2016, June 26, 2016 - July 1, 2016
(seriesProceedings of the IEEE Computer Society Conference
on Computer Vision and Pattern Recognition,
Vol. volume2016-December). publisherIEEE Computer
Society, address-, pages2129–2137.

[Yang et al(2021)]
authorpersonJingkang Yang, personKaiyang
Zhou, personYixuan Li, and personZiwei Liu.
titleGeneralized Out-of-Distribution Detection: A Survey.

[Ye et al(2022)]
authorpersonWeicheng Ye, personDangxing
Chen, and personIlqar Ramazanli.
Learning Algorithm in Two-Stage Selective
Prediction. In booktitle2022 Asia Conference on Algorithms,
Computing and Machine Learning (CACML). publisher-,
address-, pages512–521.

[Yildirim et al(2019)]
authorpersonMehmet Yigit Yildirim,
personMert Ozer, and personHasan Davulcu.
titleLeveraging uncertainty in deep learning for selective
classification.  (year2019).
Compilation and indexing terms, Copyright 2022 Elsevier Inc.
20200203012 Bayesian Classifcation Decisions makings Google+ Learning methods
Mixed-Integer Programming Programming framework Recognition algorithm
Uncertainty Unintended consequences.

[Yu and Gao(2022)]
authorpersonLijia Yu and personXiao-Shan
Gao. year2022.
Improve Robustness and Accuracy of Deep Neural
Network with $L_{2,\infty}$ Normalization.
journalJournal of Systems Science and Complexity
volume- (date15 Sep year2022),

[Zadeh et al(2018)]
authorpersonPourya Habib Zadeh, personReshad
Hosseini, and personSuvrit Sra.
titleDeep-RBF networks revisited: Robust classification
with rejection.  (year2018).
Compilation and indexing terms, Copyright 2022 Elsevier Inc.
20200470984 Cost-function High confidence Input datas Input space Metric
learning Network likes Non-linear transformations Output layer Robust
classification Vanishing gradient.

[Zhang et al(2021a)]
authorpersonChiyuan Zhang, personSamy
Bengio, personMoritz Hardt, personBenjamin Recht,
and personOriol Vinyals. year2021a.
Understanding deep learning (still) requires
rethinking generalization.
journalCommun. ACM volume64,
number3 (year2021), pages107–115.

[Zhang et al(2021b)]
authorpersonLu Zhang, personSangarapillai
Lambotharan, personGan Zheng, and personFabio
Roli. year2021b.
A Neural Rejection System Against Universal
Adversarial Perturbations in Radio Signal Classification. In
booktitle2021 IEEE Global Communications Conference,
GLOBECOM 2021, December 7, 2021 - December 11, 2021
(series2021 IEEE Global Communications Conference, GLOBECOM
2021 - Proceedings). publisherInstitute of Electrical and
Electronics Engineers Inc., address-, pageset al.;
Huawei; Intel; Qualcomm; Vodafone; ZTE.

[Zhang et al(2022)]
authorpersonWenrui Zhang, personXinxin Di,
personGuodong Wei, personShijia Geng,
personZhaoji Fu, and personShenda Hong.
titleA Deep Bayesian Neural Network for Cardiac Arrhythmia
Classification with Rejection from ECG Recordings.  (year2022).
Compilation and indexing terms, Copyright 2022 Elsevier Inc.
20220053316 Arrhythmia classification Bayesian neural networks Cardiac
arrhythmia Case-studies Electrocardiogram recordings Interpretability
Learning-based methods Performance Prediction with rejection Uncertainty.

[Zheng et al(2016)]
authorpersonStephan Zheng, personYang Song,
personThomas Leung, and personIan J. Goodfellow.
Improving the Robustness of Deep Neural Networks
via Stability Training.
journalCoRR  volumeabs/1604.04326
(year2016), pages–.

[Zhong et al(2020)]
authorpersonZiyuan Zhong, personYuchi Tian,
and personBaishakhi Ray. year2020.
Understanding Spatial Robustness of Deep Neural
journalCoRR  volumeabs/2010.04821
(year2020), pages–.

[Ziyin et al(2019)]
authorpersonLiu Ziyin, personZhikang T.
Wang, personPaul Pu Liang, personRuslan
Salakhutdinov, personLouis-Philippe Morency, and
personMasahito Ueda. year2019.
titleDeep gamblers: Learning to abstain with portfolio
theory.  (year2019).
Compilation and indexing terms, Copyright 2022 Elsevier Inc.
20200282522 Doublings End to end Inference models Loss functions Model
inference Neural-networks Performance Portfolio theories Problem-based
Supervised learning problems.