# Self-Trained One-class Classification
for Unsupervised Anomaly Detection

Jinsung Yoon, Kihyuk Sohn, Chun-Liang Li, Sercan Ö. Arık, Chen-Yu Lee, Tomas
Pfister
Google Cloud AI
Sunnyvale, CA
<EMAIL_ADDRESS>

###### Abstract

Anomaly detection (AD), separating anomalies from normal data, has various
applications across domains, from manufacturing to healthcare. While most
previous works have shown to be effective for cases with fully or partially
labeled data, they are less practical for AD applications due to tedious data
labeling processes. In this work, we focus on _unsupervised_ AD problems whose
entire training data are unlabeled and may contain both normal and anomalous
samples. To tackle this problem, we build a robust one-class classification
framework via data refinement. To refine the data accurately, we propose an
ensemble of one-class classifiers, each of which is trained on a disjoint
subset of training data. Moreover, we propose a self-training of deep
representation one-class classifiers (STOC) that iteratively refines the data
and deep representations. In experiments, we show the efficacy of our method
for unsupervised anomaly detection on benchmarks from image and tabular data
domains. For example, with a 10% anomaly ratio on CIFAR-10 data, the proposed
method outperforms state-of-the-art one-class classification method by 6.3 AUC
and 12.5 average precision.

##  1 Introduction

Anomaly detection (AD), distinguishing anomalies from normal data, is an
important problem with many applications, such as detecting faulty products
using visual sensors in manufacturing, fraudulent behaviors at credit card
transactions, or adversarial outcomes at intensive care units.

Given a wide range of application scenarios, AD has been studied under various
settings with their own challenges based on the availability of negative
(normal) and positive (anomalous) data and their labels at training. For
example, when entire positive and negative data are available with their
labels (Fig. 1(a)), the challenge comes from the imbalance in label
distributions [13, 17, 24, 4, 27, 30]. When only negative labeled data is
available (Fig. 1(b)), we often cast it as a one-class classification problem
[38, 43, 36, 23, 20, 41, 28]. Their formulations have been thereby extended to
accommodate unlabeled data (Fig. 1(c,d,e)) [15, 46, 8, 21, 37]. We refer to
Section 2 for comprehensive review.

While there exist many prior works on these settings, they require some
labeled data, which could be tedious to obtain. Despite their practical
importance, unsupervised anomaly detection problem (Fig. 1(f)), whose training
data is entirely unlabeled and may contain both negative and positive, has
been under-explored. Some recent studies [48, 5] have evaluated their one-
class classifiers for unsupervised AD settings, but their performance has been
suboptimal. To illustrate this, we evaluate the unsupervised AD performance of
state-of-the-art deep one-class classifier [41] with different anomaly ratios
in unlabeled training data. As shown in Fig. 2, the average precision
significantly drops even when a small portion (2%) of training data is
contaminated with anomalies.

Figure 1: Anomaly detection problem settings. Blue and red are for labeled
negative (normal) and positive (anomalous) samples, respectively. Grey dots
denote unlabeled samples. While previous works mostly focus on supervised (a,
b) or semi-supervised (c, d, e) settings, we tackle an anomaly detection using
only unlabeled data (f) that may contain both negative and positive samples.

In this paper, we focus on _unsupervised_ anomaly detection problems, where
only unlabeled data that may include both negative and positive samples are
available during training. Given the impressive performance of one-class
classifiers trained on negative samples [20, 23, 5, 41, 28], our goal is to
utilize them, and improve their performance by refining the unlabeled training
data so as to adopt learning strategies of one-class classification for
unsupervised anomaly detection. Inspired by the recent success of self-
training in learning without labels [7, 45], we propose a novel self-trained
one-class classification (STOC) framework. We illustrate STOC built on a two-
stage framework of deep one-class classifiers [41, 28] in Fig. 3. STOC
iteratively trains deep representations using refined data while improving
refinement of unlabeled data by excluding potential positive (anomalous)
samples. For the data refinement process, we employ an ensemble of one-class
classifiers, each of which is trained on a different subset of unlabeled
training data, and dictate samples as normal with the consensus of all the
classifiers. The refined training data are used to train the final one-class
classifier to generate the anomaly scores in the unsupervised setting. Note
that the representation learning part is optional in the STOC framework; in
other words, any shallow [38] or deep representation one-class classifiers
[41, 28] can benefit from the proposed framework (see Fig. 7 in Section
4.3.2). Unlike previous works [48, 36, 20, 23, 5, 41, 28], STOC works under
realistic assumptions that majority of the training data is negative – human
labeling can be completely removed from the entire process.

Figure 2: Performance of our proposed model and baseline one-class classifier
using contrastive representation [41] on CIFAR-10 with various anomaly ratios
in the training data.

We conduct extensive experiments across various datasets from different
domains, including semantic AD (CIFAR-10 [25], Dog-vs-Cat [16]), manufacturing
AD (MVTec AD [6]), and tabular AD. We study methods from shallow ([38, 31]) to
deep ([41, 28, 5]) models. By evaluating at different anomaly ratios of
unlabeled training data, we show that STOC significantly boosts the
performance of one-class classifiers. For example, in Fig. 2, STOC improves
more than 15 average precision (AP) with the 10% anomaly ratio upon one-class
contrastive representation [41] on CIFAR-10. Similarly, on MVTec AD, STOC
retains a strong performance, dropping less than 1 AUC with 10% anomaly ratio,
while the best existing one-class classifiers [28] drops more than 6 AUC. We
further investigate the efficacy of our design choices, such as numbers of
ensemble classifiers, thresholds, or refinement of deep representations via
ablation study.

##  2 Related Work

We review existing works under different settings as described in Fig. 1 and
discussed their challenges.

Positive + Negative setting is often considered as a supervised binary
classification problem. The challenge arises due to the imbalance in label
distributions as positive (anomalous) samples are rare. As summarized in [10],
to address this, over-/under-sampling [13, 17], weighted optimization [24, 4],
synthesizing data of minority classes [27, 30], and hybrid methods [18, 2]
have been studied.

Negative setting is often converted to a one-class classification problem,
with the goal of finding a decision boundary that includes as many one-class
samples as possible. Shallow models for this setting include one-class support
vector machines [38] (OC-SVM), support vector data description [43] (SVDD),
kernel density estimation (KDE) [26], and Gaussian density estimation (GDE)
[35]. There are also auto-encoder based models [47] that treat the
reconstruction error as the anomaly score. In recent years, deep learning
based one-class classifiers have been developed, such as deep one-class
classifier [36], geometric transformation [20], or outlier exposure [23].
Noting the degeneracy or inconsistency of learning objectives of existing end-
to-end trainable deep one-class classifiers, [41] has proposed a deep
representation one-class classifier, a two-stage framework that learns self-
supervised representations [19, 14] followed by shallow one-class classifiers,
and has been extended for texture anomaly localization with CutPaste [28]. The
robustness of these methods under unsupervised setting has also been shown in
part [48, 5] when the anomaly ratio is very low.

Semi-supervised setting is defined as utilizing small labeled samples and
large unlabeled samples to distinguish anomalies from normal data. Depending
on which labeled samples are given, we categorize this setting into three sub-
categories. If only some positive/negative labeled samples are provided, we
denote that as a PU/NU setting. Most previous works in semi-supervised AD
settings focused on NU setting where only some of the normal labeled samples
are given [34, 42, 1]. PNU setting is a more general semi-supervised setting
where subsets of both positive and negative labeled samples are given. Deep
SAD [37] and SU-IDS [33] are included in this category. Note that in
comparison to Deep SAD, which utilized some labeled data, the proposed
framework significantly outperforms under multiple benchmark datasets without
using any labeled data (see Section 4.2).

Unlabeled setting has received relatively less attention despite its
significance in automating machine learning. Two of the most popular methods
for this setting are isolation forest [31] and local outlier factor [11], but
they are hard to scale and less compatible with recent advances in
representation learning. While one-class classifiers, such as OC-SVM, SVDD, or
their deep counterparts, apply to unlabeled settings by assuming the data is
all negative, and the robustness of those methods has also been demonstrated
in part [48, 5], in practice we observe a significant performance drop with a
high anomaly ratio, as shown in Fig. 2. Our proposed framework mostly recovers
the lost performance of state-of-the-art shallow and deep one-class
classifiers trained on unlabeled data.

Self-training [39, 32] is an iterative training mechanism using machine-
generated pseudo labels as targets to train machine learning models. It has
regained popularity recently with its successful applications to semi-
supervised image classification [7, 40, 45]. Improving the quality of pseudo
labels using an ensemble of classifiers is also studied. [12] trains an
ensemble of classifiers with different classification algorithms to make a
consensus for noisy label verification. Co-training [9] trains multiple
classifiers, each of which is trained on the distinct views, to supervised
other classifiers. Co-teaching [22] and DivideMix [29] share a similar idea in
that they both train multiple deep networks on separate data batches to learn
different decision boundaries, thus become useful for noisy label
verification. While sharing a similarity, to our knowledge, we are the first
to apply self-training to _unsupervised_ learning of an anomaly detector, with
no human-generated labels.

##  3 Self-Trained One-class Classifier (STOC)

In this section, we describe our proposed framework for unsupervised anomaly
detection. Similarly to the self-training framework for semi-supervised
learning [7], our Self-Trained One-class Classifier (STOC) is an iterative
training method, where we refine the data (Section 3.1) and update the
representation with refined data (Section 3.2), followed by building a one-
class classifier using refined data and representations, as described in
Algorithm 1 and Fig. 3. Note that the framework is amenable to use fixed
(e.g., raw or pretrained) representations – in this case, the framework would
be composed of a single step of data refinement followed by building an one-
class classifier using refined data.

Notation. We denote the training data as
$\mathcal{D}\,{=}\,\\{\textbf{x}_{i}\\}_{i=1}^{N}$ where
$\textbf{x}_{i}\,{\in}\,\mathcal{X}$ and $N$ is the number of training
samples. $y_{i}\,{\in}\,\\{0,1\\}$ is the corresponding label to
$\textbf{x}_{i}$, where $0$ denotes normal (negative) and $1$ denotes anomaly
(positive). Note that labels are not provided in the unsupervised setting.

Let us denote a feature extractor as $g:\mathcal{X}\rightarrow\mathcal{Z}$.
$g$ may include any data preprocessing functions, an identity function (if raw
data is directly used for one-class classification), and learned or learnable
representation extractors such as deep neural networks. Let us define a one-
class classifier as $f:\mathcal{Z}\rightarrow[-\infty,\infty]$ that outputs
anomaly scores of the input feature ($g(\textbf{x})$). Higher the score
$f(g(\textbf{x}))$, more anomalous the sample $\mathbf{x}$ is, and the binary
prediction is made by thresholding:
$\mathbbm{1}\big{(}f(g(\textbf{x}))\geq\eta\big{)}$.

Figure 3: Block diagram of STOC composed of representation learner (Section
3.2), data refinement (Section 3.1), and the final one-class classifier
blocks. Representation learner updates the deep models using refined data from
the data refinement block, but is optional if using fixed representations.
Data refinement is done by ensemble one-class classifiers, each of which is
trained on $K$ disjoint subsets of unlabeled training data. Samples predicted
as normal by all classifiers are retained in the refined data.

###  3.1 Data Refinement

A naive way to generate pseudo labels of unlabeled data is to construct a one-
class classifier on raw or learned representations (e.g., [41]) and threshold
the anomaly score to obtain a binary label of normal and anomalous. As we
update the model with refined data that excludes samples predicted as
positive, it is important to generate pseudo labels of training data as
accurately as possible.

To this end, instead of training a single classifier, we train an ensemble of
$K$ one-class classifiers and aggregate their predictions to generate pseudo
labels. We illustrate the data refinement block in Fig. 3 and as RefineData in
Algorithm 1. Specifically, we randomly divide the unlabeled training data
$\mathcal{D}$ into $K$ disjoint subsets $\mathcal{D}_{1},...,\mathcal{D}_{K}$,
and train $K$ different one-class classifiers $(f_{1},...,f_{K})$ on
corresponding subsets $(\mathcal{D}_{1},...,\mathcal{D}_{K})$. Then, we
predict a binary label of the data $\mathbf{x}_{i}\,{\in}\,\mathcal{D}$ as
follows:

$\displaystyle\hat{y}_{i}$
$\displaystyle=1-\prod_{k=1}^{K}\Big{[}1-\mathbbm{1}\big{(}f_{k}(g(\mathbf{x}_{i}))\geq\eta_{k}\big{)}\Big{]}$
(1) $\displaystyle\eta_{k}$ $\displaystyle=\max\eta\,\mbox{ s.t.
}\,\frac{1}{N}\sum_{i=1}^{N}\mathbbm{1}\big{(}f_{k}(g(\textbf{x}_{i}))\geq\eta\big{)}\geq\gamma$
(2)

where $\mathbbm{1}(\cdot)$ is the indicator function that outputs 1/0 if the
input is True/False. $f_{k}(g(\mathbf{x}_{i}))$ represents an anomaly score of
$\mathbf{x}_{i}$ for a one-class classifier $f_{k}$. $\eta_{k}$ in Eq. (2) is
a threshold determined as a $\gamma$ percentile of the anomaly score
distribution $\\{f_{k}(g(\mathbf{x}_{i}))\\}_{i=1}^{N}$.

To interpret Eq. (1), $\mathbf{x}_{i}$ is predicted as normal, i.e.,
$\hat{y}_{i}\,{=}\,0$, if all $K$ one-class classifiers predict it as normal.
While this may be too strict and potentially reject many true normal samples
in the training set, we find that in practice it is more critical to be able
to exclude true anomalous samples from the training set. The effectiveness of
the ensemble classifier is empirically shown in Section 4.3.

Algorithm 1 STOC: Self-Trained One-class Classifier.

Input: Training data $\mathcal{D}=\\{\textbf{x}_{i}\\}_{i=1}^{N}$, Ensemble
count ($K$), threshold ($\gamma$)

Output: Refined data ($\hat{\mathcal{D}}$), trained one-class classifier
($f$), feature extractor ($g$)

1:function RefineData($\mathcal{D},g,K,\gamma$)

2: Train OCC models $\\{f_{k}\\}_{k=1}^{K}$ on
$\\{\mathcal{D}_{k}\\}_{k=1}^{K}$, $K$ disjoint subsets of the training data
$\mathcal{D}$.

3: Compute thresholds $\eta_{k}$’s for $\gamma$ percentile of anomaly
distributions (Eq. (2)).

4: Predict binary labels $\hat{y}_{i}$ (Eq. (1)).

5: Return
$\hat{\mathcal{D}}=\\{\mathbf{x}_{i}:\hat{y}_{i}=0,\mathbf{x}_{i}\,{\in}\,\mathcal{D}\\}$.

6:end function

7:function STOC($\mathcal{D},K,\gamma$)

8: Initialize the feature extractor $g$.

9: while $g$ not converged do

10: $\hat{\mathcal{D}}$ = RefineData($\mathcal{D},g,K,\gamma$).

11: Update $g$ using $\hat{\mathcal{D}}$ with self-supervised learning
objectives.

12: end while

13: $\hat{\mathcal{D}}$ = RefineData($\mathcal{D},g,K,\gamma$).

14: Train an OCC model ($f$) on refined data $(\hat{\mathcal{D}})$.

15:end function

###  3.2 Representation Update

STOC follows the idea of deep representation one-class classifiers [41], where
in the first stage deep neural network is trained with self-supervision, such
as rotation prediction [20], contrastive [41], or CutPaste [28], to provide
meaningful representations of the data, and in the second stage off-the-shelf
one-class classifiers are trained on these learned representations. Such a
two-stage framework is shown to be beneficial as it prevents the degeneracy of
the deep one-class classifiers [36].

Here, we propose to conduct self-supervised representation learning jointly
with the data refinement. More precisely, we train a trainable feature
extractor $g$ using
$\hat{\mathcal{D}}\,{=}\,\\{\mathbf{x}_{i}\,{:}\,\hat{y}_{i}\,{=}\,0\\}$, a
subset of unlabeled data $\mathcal{D}$ that includes samples whose predicted
labels with an ensemble one-class classifier from Section 3.1 are negative. We
also update $\hat{\mathcal{D}}$ as we proceed the representation learning. The
proposed method is illustrated in Algorithm 1 as STOC. In contrast to previous
work [41, 28] that use the entire training data for learning self-supervised
representation, we find it necessary to refine the training data even for
learning deep representations. Without representation refinement, the
performance improvements of STOC are limited, as shown in Section 4.3.2.

Following representation learning, we train a one-class classifier on refined
data $\hat{\mathcal{D}}$ using updated representation by $g$ for test time
prediction, as in line 13, 14 in Algorithm 1.

###  3.3 Selecting hyperparameters

As STOC is designed for unsupervised anomaly detection, labeled validation
data for hyperparameter tuning is typically not available. In this section, we
provide insights on how to select important hyperparameters. Later in Section
4.3.1, we also provide sensitivity analyses on these hyperparameters.

Our data refinement block introduces two hyperparameters: the number of one-
class classifiers ($K$) and the percentile threshold ($\gamma$). First, there
is a trade-off between the number of classifiers for ensemble and the size of
disjoint subsets for training each classifier. For example, with large $K$, we
aggregate prediction from many classifiers, each of which may contain more
randomness. However, it comes at a cost of reduced performance per classifier
as we use smaller subsets to train them. In practice, we find $K\,{=}\,5$
works well across different datasets and anomaly ratios in unlabeled training
data. Second, $\gamma$ controls the purity and coverage of refined data. If
$\gamma$ is large and thus classifiers reject too many samples, the refined
data could be more pure and contain mostly the normal samples; however, the
coverage of the normal samples would be limited. On the other hand, with small
$\gamma$, the refined data may still contain many anomalies and the
performance improvement with STOC would be limited. As such, the performance
of STOC could be suboptimal if the true anomaly ratio is not available.
Nevertheless, in our experiments, we observe that STOC is robust to selection
of $\gamma$ when chosen between the true anomaly ratio and the twice of that.
This suggests that it is safer to use $\gamma$ with an over-estimate of true
anomaly ratio.

##  4 Experiments

In this section, we test the efficacy of our proposed self-training framework
for unsupervised anomaly detection tasks on tabular (Section 4.1) and image
(Section 4.2) domains. We evaluate the methods with varying ratios of anomaly
samples in unlabeled training data and with different combinations of
representation learning and one-class classifiers. In Section 4.3, we provide
an ablation study to better understand sources of gain, as well as sensitivity
analyses to multiple hyperparameters.

Implementation details: to reduce the computational complexity of data
refinement block, we utilize simple one-class classifier such as GDE as the
OCC in the data refinement block. In two-stage model, we only update data
refinement block at 1st, 2nd, 5th, 10th, 20th, 50th, 100th, 500th epochs
instead of every epoch. After 500 epochs, we update the data refinement block
per each 500th epoch. Each run of experiments requires a single V100 GPU.

###  4.1 Experiments on Tabular Domain

Datasets. Following [48, 5], we test the efficacy of STOC on a variety of
tabular datasets, including KDDCup, Thyroid, or Arrhythmia from UCI repository
[3]. We also use KDDCup-Rev, where the labels of KDDCup is reversed so that
attack represents anomaly [48]. To construct data splits, we use 50% of normal
samples for training. In addition, we take anomaly samples worthy of the 10%
of the normal samples in the training set. This allows to simulate
unsupervised settings with anomaly ratio of up to 10% of entire training set.
The rest of the data is used for testing.111Note that the experimental
settings with contaminated training data in GOAD [5] and DAGMM [48] are
slightly different from ours. Our contamination ratio is defined as anomaly
ratio over the entire training data, while their contamination ratio is the
anomaly ratio over the entire anomalies in the dataset. We conduct experiments
using 5 random splits and 5 random seeds. We report the average and standard
deviation of 25 F1-scores (with scale 0-100) for the performance metric.

Models. We re-implement GOAD [5], a type of augmentation prediction methods
[19, 41] with random projections for learning representations, for the
baseline with a few modifications. First, instead of using embeddings to
compute the loss, we use a parametric classifier, similarly to the
augmentation prediction [41]. Second, we follow the two-stage framework [41]
to construct deep one-class classifiers. For N setting, our implementation
achieves 98.0 for KDD, 95.0 for KDD-Rev, 75.1 for Thyroid, and 54.8 for
Arrhythmia F1-scores, which are comparable to those reported in [5]. Please
see Appendix A.3 for formulation and implementation details.

Figure 4: Unsupervised anomaly detection performance (F1-score) using GOAD [5]
and that with STOC on various tabular datasets. Shaded areas represent the
standard deviation.

Results. We show results of GOAD and GOAD with STOC in Fig. 4. The ranges of
the noise ratio are set as 0% to the anomaly ratios in the original dataset
(if anomaly ratios are larger than 10%, we set the maximum anomaly ratio as
10%). For KDD-Rev, we set the maximum anomaly ratio as 2.5% because the
performances of GOAD (without STOC) drop significantly even with a small
amount of anomalies in the training data.

As can be seen in Fig. 4, integrating STOC significantly outperforms GOAD, the
state-of-the-art method for tabular one-class classifier. The improvements are
more significant especially at higher anomaly ratios. These also show the
applicability of STOC to both small (Thyroid & Arrhythmia) and large datasets
(KDD & KDD-Rev).

###  4.2 Experiments on Image Domain

Datasets. We evaluate STOC on visual anomaly detection benchmarks, including
CIFAR-10 [25], f-MNIST [44], Dog-vs-Cat [16], and MVTec anomaly detection (AD)
[6]. For CIFAR-10, f-MNIST, and Dog-vs-Cat datasets, samples from one class
are set to be normal and the remainders from other classes are set to be
anomaly. Similarly to experiments on tabular data in Section. 4.1, we swap a
certain amount of the normal training data with anomalies so that the anomaly
ratio reaches the target percentage for unsupervised settings. For MVTec AD
experiments, since there is no anomalous data available for training, we
borrow 10% of the anomalies from the test set and swap them with normal
samples in the training set. Note that the 10% of samples borrowed from the
test set are excluded from evaluation. For all datasets, we experiment with
varying anomaly ratios from 0% to 10%.

We use area under ROC curve (AUC) and average precision (AP) metrics to
quantify the performance of visual anomaly detection (with scale 0-100). When
computing AP, we set the minority class of the test set as label 1 and
majority as label 0 (e.g., normal samples are set as label 1 for CIFAR-10
experiments as there are more anomaly samples that are from 9 classes in the
test set). We run all experiments with 5 random seeds, and report the average
performance for each dataset across all classes in Fig. 5 and Fig. 6. Per-
class AUC and AP are reported in Appendix A.4.

Models. For semantic anomaly detection benchmarks, CIFAR-10, f-MNIST, and Dog-
vs-Cat, we compare the STOC with two-stage one-class classifiers [41] using
various representation learning methods, such as distribution-augmented
contrastive learning [41], rotation prediction [19, 20] and its improved
version [41], or denoising autoencoder. For MVTec AD benchmarks, we use
CutPaste [28] as a baseline and compare to its version with STOC integration.
For both experiments, we use ResNet-18 model, trained from random
initialization, using the hyperparameters from [41] and [28]. The same model
and hyperparameter configurations are used for STOC with $K\,{=}\,5$, the
count of classifier ensembles. We set $\gamma$ as twice the anomaly ratios of
the training data. For 0% anomaly ratios, we set $\gamma$ as 0.5. Finally,
Gaussian Density Estimator (GDE) on learned representations is used for one-
class classifiers.

Figure 5: Unsupervised anomaly detection performance with various one-class
classifiers on CIFAR-10 dataset. For STOC we adopt distribution-augmented
contrastive representation learning [41]. (Left) AUC, (Right) Average
Precision (AP).

Results. Fig. 5 shows a significant performance drop with increased anomaly
ratios, regardless of the representation learning algorithms. For example, AUC
of distribution-augmented contrastive representation [41] drops from 92.1 to
83.6 when anomaly ratio becomes 10%. Similarly, improved rotation prediction
representation [19], though not as significant, drops from 90.0 to 84.1. On
the other hand, STOC effectively handles the contamination in the training
data and achieves 89.9 AUC with 10% anomaly ratio, reducing the performance
drop by 74.1%. Note that the oracle performance of STOC would be perfectly
removing the anomalies from the training data (which is the same as the
performances at 0% anomaly ratio for the same size of the data). As can be
seen in Fig. 5, the performances of STOC are similar (less than 2.5 AUC
difference) with the oracle performances of STOC even with high anomaly ratios
(10%). Similar trends have been observed from other metrics, such as AP in
Fig. 5 or Recall at Precision of 70 and 90 in Fig. 9 in the Appendix.

We repeat experiments on 3 more visual anomaly detection datasets and report
results in Fig. 6. We observe consistent and significant improvements over the
baseline across different datasets and different one-class classification
methods. Note that the improvement is more significant at higher anomaly
ratios. For instance, on MVTec AD dataset, STOC improves AUC by 4.9 and AP by
7.1 upon state-of-the-art CutPaste one-class classifier with anomaly ratio of
10%. In the Appendix (Section A.4), we also illustrate per-class performances
of STOC compared with the state-of-the-art.

(a) MVTec AD

(b) f-MNIST

(c) Dog-vs-Cat

Figure 6: Unsupervised anomaly detection performance on (a) MVTec AD (b)
f-MNIST, and (c) Dog-vs-Cat datasets with varying anomaly ratios. We use
state-of-the-art one-class classification models for baselines, such as
distribution-augmented contrastive representations [41] for f-MNIST and Dog-
vs-Cat, or CutPaste [28] for MVTec AD, and build STOC on top of them.

###  4.3 Ablation studies

In this section, we conduct sensitivity analyses on two hyperparameters of
STOC, namely, the number $K$ for ensemble one-class classifiers and the
percentile threshold $\gamma$ that determines normal and anomalies in the data
refinement module. In addition, we show the importance of updating
representation with refined data. Ablation studies are conducted on two visual
anomaly detection benchmarks, CIFAR-10 and MVTec AD. More results can be found
in the Appendix.

####  4.3.1 Sensitivity Analyses for Hyperparameters

Since our method is designed for solving unsupervised anomaly detection
problem, it is important to ensure robust performance against changes in the
hyperparameters. In Fig. 7, we present the sensitivity analyses of our method
with respect to various hyperparameters.

In Fig. 7(a), we observe the performance improvement as we increase the number
of classifiers for ensemble. This is particularly effective on CIFAR dataset
(top in Fig. 7(a)) where the number of samples in the training data is large
enough that even with large $K$, the number of samples to train each OCC
($N/K$) would be sufficient.

In Fig. 7(b), we observe that STOC performs robustly when $\gamma$ is set to
be larger than the actual anomaly ratio (10%). When $\gamma$ is less than 10%,
however, we see a significant drop in performance, all the way to the baseline
($\gamma\,{=}\,0$). Our results show that STOC improves upon baseline
regardless of the threshold. In addition, it suggests that $\gamma$ could be
set to be within the true anomaly ratio and the twice of it to maximize its
effectiveness.

####  4.3.2 Importance of Representation Update with Refined Data

While iterative updates of representations with refined data is important for
STOC, one can easily decompose representation learning and data refinement of
STOC. This would result in a three-stage framework, where we learn
representations until convergence without data refinement, followed by the
data refinement and learning one-class classifier. As in Fig. 7(c), a STOC
using a pretrained then fixed representation (i.e., data refinement is used
for OCC only) already improves the performance upon the baseline with no data
refinement at any stage of learning representation or classifier, in AUC by
4.9, AP by 8.0 on CIFAR-10, and AUC by 1.3, AP by 1.3 on MVTec AD. This
confirms the effectiveness of our data refinement module. More results on
applying STOC on raw tabular features or learned (and fixed) visual
representations are provided in Appendix A.2. Moreover, we find that learning
representation with refined data plays a crucial role, resulting in another
improvement in AUC by 1.4, AP by 4.5 on CIFAR-10, and AUC by 3.2, AP by 4.8 on
MVTec AD, upon a STOC using a fixed representation.

(a) Ensemble count $K$

(b) Percentile threshold $\gamma$

(c) Representation update

Figure 7: Ablation studies on (top) CIFAR-10 and (bottom) MVTec AD datasets
under 10% anomaly ratio setting with respect to (a) ensemble count $K$, (b)
percentile threshold $\gamma$, and (c) data refinement with or without
representation update.

###  4.4 Quantifying the Refinement Efficacy

In this section, we evaluate how many normal and anomalies are excluded via
the proposed data refinement block. As in Fig. 8(a, b), with data refinement,
we can exclude more than 80% of anomalies in the training set without removing
too many normal samples. For instance, among 4% anomalies in CIFAR-10 data,
STOC is able to exclude 80% anomalies while removing less than 20% normal
samples. Such a high recall of anomalies of STOC not only is useful for
unsupervised anomaly detection, but also could be useful to improve the
annotation efficiency when a budget for active learning is available. Fig.
8(c, d) demonstrates the removed normal and abnormal samples by the data
refinement module over training epochs. It shows that better representation
learning (as training epochs increase) consistently improves the efficacy of
the data refinement.

(a) CIFAR-10

(b) MVTec AD

(c) CIFAR-10

(d) MVTec AD

Figure 8: Percentage of excluded anomalous and normal samples by data
refinement (a, b) with different anomaly ratios in training data and (c, d)
over training epochs for 10% anomaly ratio.

##  5 Conclusion

Anomaly detection has wide range of practical use cases. A challenging and
costly aspect of building an anomaly detection system is labeling as anomalies
are rare and are not easy to detect by humans. To enable high performance
anomaly detection without any labels, we propose a novel anomaly detection
framework, STOC. STOC can be flexibly integrated with any one-class
classifier, applied on raw data or on trainable representations. STOC employs
an ensemble of multiple one-class classifiers to propose candidate anomaly
samples that are refined from training, which allows more robust fitting of
the anomaly decision boundaries as well as better learning of data
representations. We demonstrate the state-of-the-art anomaly detection
performance of STOC on multiple tabular and image data.

STOC has the potential to make huge positive impact in real-world AD
applications where detecting anomalies is crucial, such as for financial crime
elimination, cybersecurity advances, or improving manufacturing quality. We
note the potential risk associated with using STOC that when representations
are not sufficiently good, there will be a negative cycle of refinement and
representation updates/one-class classifier. While we rely on the existence of
good representations this may not be true for other domains (e.g.,
cybersecurity). This paper focuses on unsupervised setting, and demonstrates
strong anomaly detection performance. This opens new horizons for human-in-
the-loop anomaly detection systems that are low cost and robust. We leave
these explorations to future work.

##  6 Acknowledgements

Discussions with Jeremy Kubica, Madeleine Udell, Mike Dusenberry, Clayton
Mellina and Ting Yu are gratefully acknowledged.

## References

  * [1] S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon.  Ganomaly: Semi-supervised anomaly detection via adversarial training.  In Asian conference on computer vision, pages 622–637. Springer, 2018.
  * [2] R. Alejo, R. M. Valdovinos, V. García, and J. H. Pacheco-Sanchez.  A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios.  Pattern Recognition Letters, 34(4):380–388, 2013.
  * [3] A. Asuncion and D. Newman.  Uci machine learning repository, 2007.
  * [4] S. Barua, M. M. Islam, X. Yao, and K. Murase.  Mwmote–majority weighted minority oversampling technique for imbalanced data set learning.  IEEE Transactions on knowledge and data engineering, 26(2):405–425, 2012.
  * [5] L. Bergman and Y. Hoshen.  Classification-based anomaly detection for general data.  arXiv preprint arXiv:2005.02359, 2020.
  * [6] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger.  Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9592–9600, 2019.
  * [7] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. Raffel.  Mixmatch: A holistic approach to semi-supervised learning.  arXiv preprint arXiv:1905.02249, 2019.
  * [8] G. Blanchard, G. Lee, and C. Scott.  Semi-supervised novelty detection.  The Journal of Machine Learning Research, 11:2973–3009, 2010.
  * [9] A. Blum and T. Mitchell.  Combining labeled and unlabeled data with co-training.  In Proceedings of the eleventh annual conference on Computational learning theory, pages 92–100, 1998.
  * [10] P. Branco, L. Torgo, and R. Ribeiro.  A survey of predictive modelling under imbalanced distributions.  arXiv preprint arXiv:1505.01658, 2015.
  * [11] M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander.  Lof: identifying density-based local outliers.  In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000.
  * [12] C. E. Brodley, M. A. Friedl, et al.  Identifying and eliminating mislabeled training instances.  In Proceedings of the National Conference on Artificial Intelligence, pages 799–805. Citeseer, 1996.
  * [13] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer.  Smote: synthetic minority over-sampling technique.  Journal of artificial intelligence research, 16:321–357, 2002.
  * [14] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton.  A simple framework for contrastive learning of visual representations.  In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  * [15] F. Denis.  Pac learning from positive statistical queries.  In International Conference on Algorithmic Learning Theory, pages 112–126. Springer, 1998.
  * [16] J. Elson, J. R. Douceur, J. Howell, and J. Saul.  Asirra: a captcha that exploits interest-aligned manual image categorization.  In ACM Conference on Computer and Communications Security, volume 7, pages 366–374, 2007.
  * [17] A. Estabrooks, T. Jo, and N. Japkowicz.  A multiple resampling method for learning from imbalanced data sets.  Computational intelligence, 20(1):18–36, 2004.
  * [18] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera.  A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches.  IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):463–484, 2011.
  * [19] S. Gidaris, P. Singh, and N. Komodakis.  Unsupervised representation learning by predicting image rotations.  arXiv preprint arXiv:1803.07728, 2018.
  * [20] I. Golan and R. El-Yaniv.  Deep anomaly detection using geometric transformations.  arXiv preprint arXiv:1805.10917, 2018.
  * [21] N. Görnitz, M. Kloft, K. Rieck, and U. Brefeld.  Toward supervised anomaly detection.  Journal of Artificial Intelligence Research, 46:235–262, 2013.
  * [22] B. Han, Q. Yao, X. Yu, G. Niu, M. Xu, W. Hu, I. W. Tsang, and M. Sugiyama.  Co-teaching: robust training of deep neural networks with extremely noisy labels.  In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 8536–8546, 2018.
  * [23] D. Hendrycks, M. Mazeika, and T. Dietterich.  Deep anomaly detection with outlier exposure.  arXiv preprint arXiv:1812.04606, 2018.
  * [24] J. P. Hwang, S. Park, and E. Kim.  A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function.  Expert Systems with Applications, 38(7):8580–8585, 2011.
  * [25] A. Krizhevsky, G. Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [26] L. J. Latecki, A. Lazarevic, and D. Pokrajac.  Outlier detection with kernel density functions.  In International Workshop on Machine Learning and Data Mining in Pattern Recognition, pages 61–75. Springer, 2007.
  * [27] S. S. Lee.  Noisy replication in skewed binary classification.  Computational statistics & data analysis, 34(2):165–191, 2000\.
  * [28] C.-L. Li, K. Sohn, J. Yoon, and T. Pfister.  Cutpaste: Self-supervised learning for anomaly detection and localization.  arXiv preprint arXiv:2104.04015, 2021.
  * [29] J. Li, R. Socher, and S. C. Hoi.  Dividemix: Learning with noisy labels as semi-supervised learning.  In International Conference on Learning Representations, 2019.
  * [30] A. Liu, J. Ghosh, and C. E. Martin.  Generative oversampling for mining imbalanced datasets.  In DMIN, pages 66–72, 2007.
  * [31] F. T. Liu, K. M. Ting, and Z.-H. Zhou.  Isolation forest.  In 2008 eighth ieee international conference on data mining, pages 413–422. IEEE, 2008.
  * [32] G. J. McLachlan.  Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis.  Journal of the American Statistical Association, 70(350):365–369, 1975.
  * [33] E. Min, J. Long, Q. Liu, J. Cui, Z. Cai, and J. Ma.  Su-ids: A semi-supervised and unsupervised framework for network intrusion detection.  In International Conference on Cloud Computing and Security, pages 322–334. Springer, 2018.
  * [34] J. Mũnoz-Marí, F. Bovolo, L. Gómez-Chova, L. Bruzzone, and G. Camp-Valls.  Semisupervised one-class support vector machines for classification of remote sensing data.  IEEE transactions on geoscience and remote sensing, 48(8):3188–3197, 2010.
  * [35] D. A. Reynolds.  Gaussian mixture models.  Encyclopedia of biometrics, 741:659–663, 2009.
  * [36] L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft.  Deep one-class classification.  In International conference on machine learning, pages 4393–4402. PMLR, 2018.
  * [37] L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft.  Deep semi-supervised anomaly detection.  In International Conference on Learning Representations, 2020.
  * [38] B. Schölkopf, R. C. Williamson, A. J. Smola, J. Shawe-Taylor, J. C. Platt, et al.  Support vector method for novelty detection.  In NIPS, volume 12, pages 582–588. Citeseer, 1999.
  * [39] H. Scudder.  Probability of error of some adaptive pattern-recognition machines.  IEEE Transactions on Information Theory, 11(3):363–371, 1965.
  * [40] K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  Advances in Neural Information Processing Systems, 33, 2020.
  * [41] K. Sohn, C.-L. Li, J. Yoon, M. Jin, and T. Pfister.  Learning and evaluating representations for deep one-class classification.  arXiv preprint arXiv:2011.02578, 2020.
  * [42] H. Song, Z. Jiang, A. Men, and B. Yang.  A hybrid semi-supervised anomaly detection model for high-dimensional data.  Computational intelligence and neuroscience, 2017.
  * [43] D. M. Tax and R. P. Duin.  Support vector data description.  Machine learning, 54(1):45–66, 2004.
  * [44] H. Xiao, K. Rasul, and R. Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  arXiv preprint arXiv:1708.07747, 2017.
  * [45] Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le.  Self-training with noisy student improves imagenet classification.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687–10698, 2020.
  * [46] B. Zhang and W. Zuo.  Learning from positive and unlabeled examples: A survey.  In 2008 International Symposiums on Information Processing, pages 650–654. IEEE, 2008.
  * [47] C. Zhou and R. C. Paffenroth.  Anomaly detection with robust deep autoencoders.  In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665–674, 2017.
  * [48] B. Zong, Q. Song, M. R. Min, W. Cheng, C. Lumezanu, D. Cho, and H. Chen.  Deep autoencoding gaussian mixture model for unsupervised anomaly detection.  In International Conference on Learning Representations, 2018.

##  Appendix A Appendix

###  A.1 Additional results with different metrics

There are various performance metrics of one-class classifiers. In the main
manuscript, we mainly use AUC, Average Precision (AP) and F1-score as the
evaluation metrics. In this subsection, we report the performances of the
proposed model (STOC) and baselines in terms on Recall at Precision 70 and 90
as the additional metrics on CIFAR-10 dataset.

Figure 9: Performance of various one-class classifiers on CIFAR-10 dataset.
STOC is applied on top of Contrastive [41]. (Left) Recall at Precision 70,
(Right) Recall at Precision 90.

As in Fig. 9, we observe a similar trends with these two additional metrics as
well. For example, the performances of STOC are robust across various anomaly
ratios. On the other hand, all the other one-class classifiers show consistent
and significant performance degradation as the anomaly ratio increases. STOC
performs 15.8 and 10.9 better than the state-of-the-art one-class classifier
[41] in terms of recall at precision 70 and 90, respectively.

###  A.2 STOC on raw tabular features / learned image representations

(a) KDD-Rev

(b) Thyroid

(c) Arrhythmia

(d) RotNet [20]

(e) Rotation [19]

(f) Contrastive [41]

Figure 10: Performance of STOC on (top) raw tabular features and (lower)
learned image representations. STOC consistently outperforms baseline and in
some cases (e.g., Thyroid, and Contrastive [41]), the performance improvements
are significant.

STOC is also applicable on raw tabular features or learned image
representation without representation update using data refinement. In this
section, we demonstrate the performance improvements by STOC without
representation update to verify the effectiveness of data refinement block of
STOC for shallow one-class classifiers.

Fig. 10 (upper) demonstrates consistent and significant performance
improvements when we apply STOC on top of raw tabular features. Specifically,
the Average Precision (AP) improvements are 10.2, 29.0, and 4.1 with KDD-Rev,
Thyroid, and Arrhythmia tabular datasets, respectively.

We also apply STOC on top of various learned image representations. As can be
seen in Fig. 10 (lower), the performance improvements of STOC are consistent
across various different learned image representations (without representation
update). For instance, the AP improvements are 9.2, 1.0, and 12.1 with learned
image representations using RotNet [20], Rotation [19], and Contrastive [41],
respectively.

###  A.3 Implementation Details on GOAD [5] for Tabular Data Experiments

A classification-based anomaly detection method, GOAD [5], has demonstrated
strong anomaly detection performance on tabular datasets. Unlike previous
works [20, 23] that formulate a parametric classifier for multiple
transformation classification, GOAD employs distance-based classification of
multiple transformations. For the set of transformations
$T_{m}:\mathcal{X}\,{\rightarrow}\,\mathcal{D}$, $m\,{=}\,1,...,M$, the loss
function of GOAD is written as in Equation (3) with the probability defined in
Eq. (4).

$\displaystyle\mathcal{L}=-\mathbb{E}_{m,x}\big{[}\log P(m|T_{m}(x))\big{]},$
(3) $\displaystyle
P(\hat{m}|T_{m}(x))=\frac{\exp(-\|f(T_{m}(x))-c_{\hat{m}}\|^{2})}{\sum_{n}\exp(-\|f(T_{m}(x))-c_{n}\|^{2})},$
(4)

where the centers $c_{m}$’s are updated by the average feature over the
training set. While it is shown to perform well [5], we find that the
distance-based formulation is not necessary, and we achieve the similar
performance, if not worse, to [5] using a parametric classifier when computing
the probability:

$P(\hat{m}|T_{m}(x))=\frac{\exp\left(w_{\hat{m}}^{\top}f(T_{m}(x))+b_{\hat{m}}\right)}{\sum_{n}\exp\left(w_{n}^{\top}f(T_{m}(x))+b_{n}\right)}$
(5)

The formulation in Eq. (5) is easier to optimize than its original form in Eq.
(4) as it can be fully optimized with backpropagation without alternating
updates of feature extractor $f$ and centers $c_{m}$. Once we learn a
representation by optimizing the loss in Equation (3) using Equation (5), we
follow a two-stage one-class classification framework of [41] to construct a
set of Gaussian density estimation one-class classifiers for each
transformation. Finally, we aggregate a maximum normality scores from a set of
classifiers as the normality score.

In Table 1, we summarize the implementation details, such as network
architecture or hyperparameters, and anomaly detection performance under N
setting that reproduces the results in [5].

Datasets | KDD | KDD-Rev | Thyroid | Arrhythmia
---|---|---|---|---
F-score [5] | 98.4$\pm$0.2 | 98.9$\pm$0.3 | 74.5$\pm$1.1 | 52.0$\pm$2.3
F-score (ours) | 98.0$\pm$0.2 | 95.0$\pm$0.2 | 75.1$\pm$2.4 | 54.8$\pm$3.2
$f$ (feature) |  $\big{[}$Linear(8), LeakyReLU(0.2)$\big{]}$ $\times$5
Optimizer | Momentum SGD (momentum${=}\,0.9$)
Learning rate | 0.001
L2 weight regularization | 0.00003
Batch size | 64$\times M$
Random projection dimension | 32
$M$ (number of transformations) | 32 | 256
Train steps | $2^{10}$ | $2^{16}$
Table 1: The anomaly detection performance under N setting of GOAD in [5] and
our implementation. Our implementation demonstrates comparable, if not worse,
performance to those reported in [5]. Our implementation also shares most
hyperparameters across datasets except the $M$, the number of transformations,
and the train steps, which are closely related to the size of training data.

###  A.4 Per-class AUC and AP

In the main manuscript, we report the mean and standard deviation of AUC and
AP across all classes in each dataset. In this section, we report the mean and
standard deviation of AUC and AP for each class in each dataset (including
CIFAR-10 (Table 2), MVTec AD (Table 3), fMNIST (Table 4), and Dog-vs-Cat
(Table 5) datasets).

####  A.4.1 Per-class AUC and AP on CIFAR-10 dataset

anomaly ratios | 0% | 10%
---|---|---
Classes / Methods | Baseline [41] | STOC + [41] | Baseline [41] | STOC + [41]
AUC
Airplane | 90.5$\pm$0.6 | 90.4$\pm$0.4 | 85.3$\pm$0.8 | 87.6$\pm$0.7
Automobile | 98.8$\pm$0.1 | 98.7$\pm$0.1 | 95.2$\pm$0.5 | 97.9$\pm$0.3
Bird | 87.5$\pm$0.6 | 87.1$\pm$0.8 | 75.6$\pm$1.1 | 84.7$\pm$1.2
Cat | 81.5$\pm$0.5 | 80.3$\pm$0.7 | 68.9$\pm$1.0 | 78.5$\pm$1.5
Deer | 90.3$\pm$0.4 | 90.2$\pm$0.5 | 79.9$\pm$0.8 | 87.4$\pm$0.5
Dog | 90.8$\pm$0.7 | 90.8$\pm$0.6 | 79.2$\pm$2.5 | 89.6$\pm$0.6
Frog | 92.0$\pm$0.6 | 91.4$\pm$0.9 | 78.5$\pm$2.1 | 87.5$\pm$1.3
Horse | 97.8$\pm$0.1 | 97.9$\pm$0.1 | 93.3$\pm$0.5 | 96.0$\pm$0.6
Ship | 96.5$\pm$0.2 | 96.4$\pm$0.1 | 91.5$\pm$0.9 | 95.2$\pm$0.5
Truck | 95.1$\pm$0.3 | 95.8$\pm$0.1 | 88.9$\pm$0.4 | 94.7$\pm$0.4
Average Precision
Airplane | 68.3$\pm$1.7 | 67.9$\pm$1.1 | 61.3$\pm$1.9 | 61.8$\pm$1.1
Automobile | 93.9$\pm$0.2 | 94.1$\pm$0.1 | 85.9$\pm$3.4 | 91.5$\pm$0.7
Bird | 62.2$\pm$2.1 | 61.7$\pm$2.4 | 42.4$\pm$3.0 | 57.5$\pm$2.3
Cat | 44.0$\pm$2.4 | 42.9$\pm$1.9 | 30.3$\pm$3.8 | 44.9$\pm$0.9
Deer | 59.5$\pm$0.8 | 58.8$\pm$1.2 | 42.9$\pm$3.5 | 53.6$\pm$1.5
Dog | 59.1$\pm$6.5 | 59.8$\pm$4.7 | 39.7$\pm$9.1 | 64.5$\pm$2.0
Frog | 72.7$\pm$2.0 | 70.4$\pm$6.9 | 38.5$\pm$11.0 | 73.6$\pm$3.6
Horse | 89.0$\pm$0.7 | 89.2$\pm$0.6 | 80.6$\pm$1.9 | 86.2$\pm$1.1
Ship | 83.9$\pm$0.8 | 83.7$\pm$0.6 | 74.9$\pm$2.4 | 79.8$\pm$0.7
Truck | 79.0$\pm$1.6 | 81.3$\pm$0.4 | 67.0$\pm$1.9 | 75.8$\pm$1.2
Table 2: The anomaly detection performance per class on CIFAR-10 dataset in
terms of AUC and AP. Class represents the label of normal samples.

####  A.4.2 Per-class AUC and AP on MVTec AD dataset

anomaly ratios | 0% | 10%
---|---|---
Classes / Methods | Baseline [28] | STOC + [28] | Baseline [28] | STOC + [28]
AUC
Bottle | 98.0$\pm$1.0 | 98.8$\pm$0.7 | 94.3$\pm$0.8 | 98.7$\pm$0.7
Cable | 77.7$\pm$0.7 | 77.9$\pm$2.6 | 72.9$\pm$3.3 | 80.4$\pm$0.9
Capsule | 96.1$\pm$1.1 | 97.2$\pm$1.3 | 92.4$\pm$1.7 | 96.8$\pm$0.7
Hazelnut | 97.2$\pm$0.7 | 97.5$\pm$0.4 | 83.5$\pm$3.6 | 96.7$\pm$0.6
Metal Nut | 98.5$\pm$1.0 | 98.6$\pm$0.4 | 91.2$\pm$2.8 | 98.7$\pm$0.3
Pill | 92.1$\pm$3.1 | 91.1$\pm$2.5 | 85.1$\pm$1.5 | 92.1$\pm$0.5
Screw | 87.4$\pm$1.6 | 86.8$\pm$2.5 | 79.9$\pm$1.0 | 86.4$\pm$1.4
Toothbrush | 98.5$\pm$1.5 | 98.3$\pm$1.1 | 98.6$\pm$1.3 | 99.3$\pm$1.2
Transistor | 96.4$\pm$2.0 | 96.7$\pm$1.6 | 93.9$\pm$2.8 | 95.7$\pm$0.7
Zipper | 99.4$\pm$0.4 | 98.0$\pm$0.6 | 91.1$\pm$2.9 | 99.2$\pm$0.7
Carpet | 90.0$\pm$3.9 | 90.8$\pm$4.6 | 80.4$\pm$6.1 | 91.2$\pm$2.7
Grid | 99.2$\pm$0.5 | 99.7$\pm$0.3 | 92.9$\pm$7.6 | 99.7$\pm$0.2
Leather | 99.8$\pm$0.3 | 99.7$\pm$0.4 | 98.0$\pm$1.1 | 100.0$\pm$0.0
Tile | 93.1$\pm$1.5 | 93.3$\pm$2.7 | 86.4$\pm$3.4 | 92.4$\pm$1.8
Wood | 98.8$\pm$0.9 | 98.5$\pm$1.4 | 92.1$\pm$2.2 | 99.0$\pm$0.5
Average Precision
Bottle | 98.8$\pm$0.7 | 99.3$\pm$0.5 | 94.3$\pm$1.6 | 98.0$\pm$0.6
Cable | 81.8$\pm$0.9 | 82.3$\pm$1.7 | 70.8$\pm$4.3 | 74.1$\pm$3.0
Capsule | 99.0$\pm$0.3 | 99.3$\pm$0.3 | 79.5$\pm$5.3 | 89.8$\pm$2.7
Hazelnut | 97.1$\pm$0.6 | 97.0$\pm$0.4 | 88.8$\pm$3.6 | 93.6$\pm$1.9
Metal Nut | 99.6$\pm$0.3 | 99.6$\pm$0.1 | 76.1$\pm$8.4 | 92.1$\pm$3.1
Pill | 98.2$\pm$0.7 | 97.9$\pm$0.6 | 56.6$\pm$2.6 | 67.3$\pm$2.2
Screw | 93.8$\pm$1.1 | 93.6$\pm$1.3 | 60.1$\pm$2.8 | 63.7$\pm$1.8
Toothbrush | 99.2$\pm$0.9 | 99.2$\pm$0.6 | 97.4$\pm$2.2 | 98.6$\pm$2.0
Transistor | 93.0$\pm$3.0 | 93.7$\pm$2.4 | 97.8$\pm$1.6 | 97.1$\pm$0.5
Zipper | 99.8$\pm$0.2 | 99.2$\pm$0.3 | 86.1$\pm$2.8 | 98.0$\pm$1.5
Carpet | 94.4$\pm$3.5 | 95.7$\pm$2.8 | 64.2$\pm$8.2 | 71.6$\pm$4.2
Grid | 99.5$\pm$0.3 | 99.8$\pm$0.2 | 90.9$\pm$6.1 | 99.4$\pm$0.4
Leather | 99.9$\pm$0.2 | 99.9$\pm$0.2 | 97.2$\pm$1.3 | 100.0$\pm$0.1
Tile | 96.1$\pm$0.9 | 96.4$\pm$1.4 | 82.4$\pm$4.5 | 86.4$\pm$2.8
Wood | 99.4$\pm$0.5 | 99.4$\pm$0.6 | 88.1$\pm$3.2 | 97.2$\pm$1.7
Table 3: The anomaly detection performance per class on MVTec AD dataset in
terms of AUC and AP. Class represents the label of normal samples.

####  A.4.3 Per-class AUC and AP on fMNIST dataset

anomaly ratios | 0% | 10%
---|---|---
Classes / Methods | Baseline [41] | STOC + [41] | Baseline [41] | STOC + [41]
AUC
T-shirt/top | 93.3$\pm$1.2 | 93.2$\pm$1.0 | 79.2$\pm$1.3 | 84.7$\pm$1.0
Trouser | 99.2$\pm$0.3 | 99.2$\pm$0.2 | 92.9$\pm$0.8 | 94.5$\pm$1.9
Pullover | 93.8$\pm$0.2 | 93.8$\pm$0.2 | 78.4$\pm$2.7 | 90.1$\pm$1.1
Dress | 94.4$\pm$0.5 | 94.0$\pm$0.5 | 78.4$\pm$3.5 | 85.8$\pm$1.6
Coat | 95.1$\pm$0.1 | 94.9$\pm$0.2 | 80.4$\pm$3.1 | 88.7$\pm$1.7
Sandal | 95.4$\pm$0.3 | 95.5$\pm$0.4 | 67.6$\pm$4.3 | 81.7$\pm$1.8
Shirt | 87.9$\pm$0.2 | 87.9$\pm$0.2 | 73.9$\pm$1.2 | 81.0$\pm$2.5
Sneaker | 99.1$\pm$0.0 | 99.2$\pm$0.1 | 80.2$\pm$4.9 | 94.7$\pm$3.6
Bag | 97.8$\pm$0.2 | 97.6$\pm$0.3 | 58.2$\pm$3.3 | 68.6$\pm$2.3
Ankle boot | 98.6$\pm$0.2 | 98.5$\pm$0.2 | 76.6$\pm$2.6 | 87.1$\pm$3.6
Average Precision
T-shirt/top | 68.7$\pm$12.1 | 67.9$\pm$8.9 | 43.0$\pm$4.7 | 56.8$\pm$3.7
Trouser | 96.5$\pm$2.9 | 98.2$\pm$0.3 | 78.2$\pm$5.2 | 87.1$\pm$3.7
Pullover | 62.7$\pm$1.2 | 63.6$\pm$1.3 | 27.2$\pm$6.9 | 57.1$\pm$1.5
Dress | 72.3$\pm$3.7 | 71.6$\pm$2.7 | 36.1$\pm$8.8 | 57.3$\pm$2.6
Coat | 66.2$\pm$1.1 | 65.6$\pm$1.0 | 24.3$\pm$3.9 | 50.7$\pm$9.5
Sandal | 70.9$\pm$2.8 | 70.8$\pm$2.4 | 17.8$\pm$5.9 | 40.1$\pm$5.4
Shirt | 49.6$\pm$0.9 | 49.7$\pm$0.9 | 27.2$\pm$5.8 | 37.6$\pm$4.7
Sneaker | 91.9$\pm$0.4 | 92.9$\pm$0.9 | 21.0$\pm$3.8 | 81.6$\pm$11.4
Bag | 92.2$\pm$0.4 | 91.8$\pm$1.0 | 13.2$\pm$2.0 | 20.1$\pm$3.7
Ankle boot | 87.9$\pm$1.4 | 86.5$\pm$1.5 | 19.6$\pm$3.1 | 48.0$\pm$16.5
Table 4: The anomaly detection performance per class on fMNIST dataset in
terms of AUC and AP. Class represents the label of normal samples.

####  A.4.4 Per-class AUC and AP on Dog-vs-Cat dataset

anomaly ratios | 0% | 10%
---|---|---
Classes / Methods | Baseline [41] | STOC + [41] | Baseline [41] | STOC + [41]
AUC
Dog | 89.7$\pm$0.6 | 89.2$\pm$0.4 | 74.7$\pm$2.1 | 79.1$\pm$1.3
Cat | 88.2$\pm$0.4 | 88.0$\pm$0.7 | 70.1$\pm$2.4 | 70.7$\pm$1.8
Average Precision
Dog | 91.6$\pm$0.7 | 91.0$\pm$0.6 | 80.0$\pm$2.2 | 84.0$\pm$1.2
Cat | 90.2$\pm$0.3 | 89.8$\pm$1.0 | 71.6$\pm$3.8 | 71.5$\pm$4.8
Table 5: The anomaly detection performance per class on Dog-vs-Cat dataset in
terms of AUC and AP. Class represents the label of normal samples.