11institutetext: University of Limerick, Castletroy, Limerick, V94 T9PX,
Ireland
11email<EMAIL_ADDRESS>
# On the detection of anomalous or out-of-distribution data in vision models
using statistical techniques

Laura O’Mahony 11 David JP O’Sullivan 11 Nikola S. Nikolov 11

###### Abstract

Out-of-distribution data and anomalous inputs are vulnerabilities of machine
learning systems today, often causing systems to make incorrect predictions.
The diverse range of data on which these models are used makes detecting
atypical inputs a difficult and important task. We assess a tool, Benford’s
law, as a method used to quantify the difference between real and corrupted
inputs. We believe that in many settings, it could function as a filter for
anomalous data points and for signalling out-of-distribution data. We hope to
open a discussion on these applications and further areas where this technique
is underexplored.

###### Keywords:

Anomaly detection Out-of-distribution data Computer vision Benford’s law Image
corruption.

##  1 Introduction

Deploying machine learning systems in the real world brings with it multiple
AI safety challenges, for example, issues related to model robustness and
monitoring. Research on the detection of adversarial examples, anomalous
inputs, and flagging when the data a model is deployed on is out-of-
distribution attempts to tackle such issues. Out-of-distribution means there
is a difference in distributional properties between the test, training, and
real world data. For machine learning models, even simple shifts in data
distribution can lead to a large drop in performance [20]. An example of a
failure caused by data anomalies is the discovery that classifiers often fail
by giving high-confidence predictions on anomalous inputs when their
predictions are incorrect [25]. The failure of such classifiers to indicate
where they are likely wrong is problematic [4]. In this work, we study how a
range of image corruptions appreciably deviate in distribution from natural
images by leveraging a dataset containing various image corruption types. Our
work trials the broad applicability of a method that could act as a filter for
anomalous inputs and indicate distribution shifts. We find the method reflects
some corruption types in the experiments performed and thus may be practical
as an accompaniment to other methods for out-of-distribution data and
anomalous input detection.

A vast amount of literature exists aiming to detect out-of-distribution and
anomalous data [37]. Some of these methods focus on analysing the activations
of hidden layers [35, 18, 14, 26]. Many works formulate the problem as a model
selection between an in-distribution and an out-of-distribution density model
and use a likelihood-ratio based approach to detect out-of-distribution data
[7, 33, 38]. Another approach relates to the softmax distribution [16]. Other
authors utilise early-layer outputs [1] or gradient space in out-of-
distribution data detection [19]. A technique known as outlier exposure
tackles classifier overconfidence by training anomaly detectors against an
auxiliary outlier dataset [17]. There also exists previous literature
concerning the properties, such as the mean, of a dataset computed in the
presence of outliers [36]. Other work on the low-order statistics of data
approaches the problem of out-of-distribution data by analysing the low-order
statistics and framing it as a statistical hypothesis testing problem [13].

Our approach differs in that it uses a less-known property of image
distributions relating to Benford’s law (BFL), which has a proven track record
in image forensics, as is discussed in Sec. 2. To our knowledge, the use of
Benford’s law is novel in the context of out-of-distribution and anomaly
detection in the context of machine learning classification. We note that our
approach is compatible with other differing out-of-distribution heuristics and
anomaly detection techniques, meaning it could complement other methods in the
literature. Furthermore, our approach is surprisingly simple and
computationally inexpensive as it does not modify the model parameters or
architecture, making it scalable.

The paper is structured as follows: In Sec. 2, we outline the theory and
current applications for the distributional summaries of images. In Sec. 3, we
outline the pipeline used to measure an image’s deviation from its theoretical
distribution before introducing the dataset used to evaluate this pipeline. We
then present the experimental results. We also discuss possible extensions to
this work and its applications before concluding in Sec. 4.

##  2 Background

In this section, we introduce the Benford equation as a benchmark that can be
used to compare the statistical distributional properties of images. In 1938,
Benford [6] introduced BFL for numbers. This paper predicts the statistical
frequency of the leading digits (LDs) for a set of measurements of a broad
range of “natural” quantities [29]. Here, natural data is understood to be
data collected by observing natural phenomena. It is worth noting that the
cases where the rule applies are often determined experimentally rather than
mathematically [10]. Benford’s law has been empirically observed for many
natural quantities, such as the lengths of rivers, population numbers, stock
market prices, etc. The predicted distribution, $p(d)$ is given by Eq. 1,
where $d$ is the LD of the number (e.g., ‘245326’ has the LD ‘2’. The
remaining digits ‘45326’ are discarded as Benford’s law only concerns the
distribution of the LDs.),

$p(d)=log_{b}\left(1+\frac{1}{d}\right),\hskip 8.53581ptd=1,2,...,b-1.$ (1)

As described in the equation, the nature of probabilities of the first digits
is logarithmic to the base $b$, for invariant $b$. The distribution given by
the Benford equation, i.e., Eq. 1, in base 10, is shown in Fig. 1. As shown in
this figure, the occurrence of the smaller first digits is more frequent than
larger first digits.

Figure 1: The theoretical distribution of first digits in data that obeys
Benford’s law. The height of the bar is the percentage of numbers that start
with the digit.

It has been observed that this rule is not well-fitted by the LD statistics of
altered data [11]. As a result, it has been useful for purposes such as fraud
detection in taxing and accounting [12, 5]. It has also been used to detect
anomalies that require investigation for fraud in customs declarations [9].

BFL can also be applied to image data, provided an appropriate transform is
made before the distribution is extracted from images. The discrete cosine
transform (DCT) [3] is a widely used transformation technique in signal
processing and data compression [30]. It expresses a finite sequence of data
points as a sum of cosine functions oscillating at various frequencies. It has
been shown that the DCT coefficients of “natural” images can typically be
modelled by a Laplacian-like distribution, which follows BFL [22]. Therefore,
images mapped into the DCT domain typically follow BFL closely [28], giving us
a transform that can be used to extract a distribution that should follow BFL.
Previous work has exploited this for applications such as detecting JPEG
compression [27, 24], synthetic imagery [2], estimating the amount of
processing that has been applied to an image, and detecting image tampering
[34]. It has been applied to detect if an image is generated by a generative
adversarial network (GAN) [8]. It has also been shown that images that reflect
only a small part of a scene do not fit BFL well [2]. A connection between a
neural network’s generalisation capability and a BFL related metric has been
studied [32].

The extensive analysis of BFL in image forensics and otherwise lends the
question of whether the rule could be employed in AI safety applications, such
as detecting anomalies and indicating if data is out-of-distribution. We ask
if this theoretical structure of natural images shifts as images become less
legible, allowing us to detect anomalies and reflect when data is out-of-
distribution.

##  3 Analysis of image discrete cosine transform coefficients

###  3.1 Methodology for extracting a distributional comparison measure from
images

In this section, we provide the details of how images are processed to analyse
the DCT coefficient distribution of images and how we measure the divergence
of the empirical distribution of images from the BFL theoretical distribution.

We first need to obtain a distributional representation of an image so that we
can compare it to the theoretical distribution. The process of obtaining this
distribution and measuring the divergence from BFL is depicted in Fig. 2.
Given an RGB image $I$, the image is initially converted to greyscale. The
image is then divided into 8x8 pixel blocks that do not overlap. Each block is
transformed using the DCT. The empirical distribution of the LDs of the DCT
coefficients from each block is calculated with respect to a base, e.g., base
10. This outputs a probability mass function (pmf), $\hat{p}(d)$, for the
image in question, $I$. This computed pmf is compared to the theoretical
distribution, given by the Benford equation, i.e., Eq. 1. A statistical
metric, the Jensen-Shannon divergence [23], is employed to investigate the
difference between these distributions. This gives us $D_{JS}$, a measure of
the distributional difference between the empirical and theoretical
distribution of $I$.

This process provides a measure of how far the distribution of an image is
from BFL. We can then use this as a comparison between clean images and images
with different corruption types applied at varying severity levels, which we
could think of as distributional shifts in the data. To test this process, we
use the Image-C dataset [15], which we describe in the next section.

Figure 2: The procedure of quantifying the difference between the distribution
of the image discrete cosine transform (DCT) coefficients and the theoretical
distribution given by Benford’s law consists of (1) applying a discrete cosine
transform to the image $I$, followed by (2) calculating the leading digit (LD)
distribution of the transform coefficients, and finally (3) a statistical
comparison between the theoretical distribution $p$ and observed distribution
$\hat{p}$ of the transformed coefficients is made. The raw image $I$ is taken
from the ImageNet-C dataset [15].

###  3.2 Data

To test our pipeline described in Sec. 3.1, the ImageNet-C dataset was
repurposed to assess whether varying levels of image corruption are reflected
by deviations of images from BFL.

ImageNet-C 111License Creative Commons Attribution 4.0 Intern, DOI:
10.5281/zenodo.2235448ational [15] is a dataset created by Hendrycks and
Dietterich, which is designed to establish benchmarks for image classifier
robustness. The dataset contains 15 distinct types of image corruptions
(noise, blur, weather, and digital corruptions). Each of these corruption
types has severity levels 1 - 5, where higher levels of severity indicate a
higher level of corruption being applied. The dataset is algorithmically
generated from the ImageNet validation data [31] and intended to evaluate the
performance of a model on common image corruptions. Therefore, the data is the
standard ImageNet size and consists of 1000 ImageNet classes. Both the type
and severity of corruption give us a unique dataset to test if we can detect
differences in the distribution of images using our method. See Fig. 3 for an
example of an image with a subset of the corruption types applied at
corruption severity levels 1, 3 and 5 respectively. Four different corruption
types are shown, corresponding to a common image from the ImageNet validation
data.

Figure 3: An example of images taken from ImageNet-C [15] representing
different corruption types.

###  3.3 Results of the image comparison metric and discussion

The pipeline detailed in Sec. 3.1 was applied to the 50,000 ImageNet
validation dataset [31] images. Due to computational constraints, it was
applied to the ImageNet-C images of corruption severity levels 1, 3 and 5,
corruption types glass blur, contrast, Gaussian noise, and fog 222Code
available at https:/github.com/lomahony/first-digits.. We first look at the
distribution of these divergence statistics. From Fig. 4, we can see that,
typically, across all corruption types, with the exception of Gaussian noise,
corrupted images show an increase in the median divergence from the Benford’s
law (BFL) benchmark compared to uncorrupted images. Higher severity levels of
corruption types glass blur and contrast lead to a larger median divergence
from the BFL benchmark compared to lower severity levels of corruption. A
different pattern is seen for the Gaussian noise corrupted images at severity
levels 1 and 3. A possible explanation is that Gaussian noise may interfere
with the Laplacian-like distribution followed by the DCT coefficients of
images.

Further experiments assessed whether the proposed approach can capture model
degradation on out-of-distribution data. A popular pretrained image
classification model called AlexNet [21] was used as a benchmark model to
compare the test accuracy of the top-1 prediction accuracy at the selected
corruption levels. Fig. 5 shows that, as expected, the baseline ImageNet
validation accuracy of AlexNet of 0.633 decreased as the level of corruption
increased for each corruption category. In some cases, the model performance
drop did not reflect a higher median divergence from the BFL benchmark, such
as the median Jensen-Shannon divergence for Gaussian noise corrupted images at
severity levels 1 and 3 compared with the median divergence for the clean
ImageNet images. In other cases, a considerable jump in the median divergence
from the BFL benchmark reflected a large model performance degradation. An
example of this is the jump in Jensen-Shannon divergence between contrast and
Gaussian noise corrupted images at severity levels 3 and 5. This was reflected
by a very low top-1 accuracy. From these results, a clear limitation of this
technique is that the distributional shift may impact model performance
significantly more than what is reflected by the divergence statistic. This
suggests this approach may be useful as a complementary approach but should
not replace other anomaly or out-of-distribution detection methods. Another
limitation of this approach is that it may be useful for sensory anomaly
detection (detecting changes in the input space e.g., indicating where samples
may be from a different domain), but not semantic anomaly detection (detecting
changes in the label space e.g., detecting the occurrence of new classes)
[37].

A generalisation of BFL has been shown to better fit natural images due to the
properties of the DCT transform [28]. This suggests an extension of this work
could involve investigating if a generalised Benford’s equation improves on
the results presented here for Gaussian noise as well as the other corruption
types. In addition, further extensions of this work could involve calculating
the empirical distribution of training images. This could be used as the
benchmark instead of Benford’s equation to test if this technique could be
used outside of natural images. As BFL is applicable to various data types,
another potential future direction could involve testing the methodology on
other data types. Further experiments could also involve testing to what
extent model performance drop correlates with the level of distribution shift
detected using our pipeline.

Figure 4: The Jensen-Shannon divergence of the statistics of the discrete
cosine transform coefficients for clean ImageNet images (blue), images of
corruption severity 1 (orange), corruption severity 3 (green), and corruption
severity 5 (red). Figure 5: The top-1 predictive accuracy evaluated using
AlexNet [21] is shown for clean ImageNet images, images of corruption severity
1, corruption severity 3, and corruption severity 5.

##  4 Conclusion

In this paper, we investigated if the divergence of image statistics from
Benford’s Law is reflected when images are corrupted at various levels.
Importantly, we do not claim that this phenomenon is universal, but we found
there to be a difference in image distribution across many corruption types
for natural images taken from ImageNet. Results show that for many corruption
types, images that are corrupted to a higher level typically deviate from the
expected distribution more. This positive result motivates further testing,
and we believe extensions of the methods employed in this paper are likely to
give positive results. Future work might involve testing the extent that the
proposed approach reflects the model degradation on out-of-distribution data.
This technique could be added to the toolkit as a low computational filter for
anomalous or out-of-distribution data or to assess the reliability of images
before attempting to classify them. It could even be investigated as a coarse
filter to detect some types of adversarial examples. Furthermore, wider
ramifications of the successful differentiation in distribution between clean
and corrupted images that the authors are not aware of may lead to further
interesting research questions.

####  4.0.1 Acknowledgements

This publication has emanated from research conducted with the financial
support of Science Foundation Ireland under Grant Number 18/CRT/6049. For the
purpose of Open Access, the authors have applied a CC BY public copyright
licence to any Author Accepted Manuscript version arising from this
submission.

## References

  * [1] Abdelzad, V., Czarnecki, K., Salay, R., Denounden, T., Vernekar, S., Phan, B.: Detecting out-of-distribution inputs in deep neural networks using an early-layer output. arXiv preprint arXiv:1910.10307 (2019)
  * [2] Acebo, E., Sbert, M.: Benford’s law for natural and synthetic images. In: Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imaging. pp. 169–176 (2005)
  * [3] Ahmed, N., Natarajan, T., Rao, K.R.: Discrete cosine transform. IEEE transactions on Computers 100(1), 90–93 (1974)
  * [4] Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016)
  * [5] Asllani, A., Naco, M., et al.: Using benford’s law for fraud detection in accounting practices. Journal of Social Science Studies 2(1), 129–143 (2015)
  * [6] Benford, F.: The law of anomalous numbers. Proceedings of the American philosophical society pp. 551–572 (1938)
  * [7] Bishop, C.M.: Novelty detection and neural network validation. IEE Proceedings-Vision, Image and Signal processing 141(4), 217–222 (1994)
  * [8] Bonettini, N., Bestagini, P., Milani, S., Tubaro, S.: On the use of benford’s law to detect gan-generated images. In: 2020 25th international conference on pattern recognition (ICPR). pp. 5495–5502. IEEE (2021)
  * [9] Cerioli, A., Barabesi, L., Cerasa, A., Menegatti, M., Perrotta, D.: Newcomb–benford law and the detection of frauds in international trade. Proceedings of the National Academy of Sciences 116(1), 106–115 (2019)
  * [10] Crisan, D., Irimia, A., Gota, D., Miclea, L., Puscasiu, A., Stan, O., Valean, H.: Analyzing benford’s law’s powerful applications in image forensics. Applied Sciences 11(23), 11482 (2021)
  * [11] Diekmann, A.: Not the first digit! using benford’s law to detect fraudulent scientif ic data. Journal of Applied Statistics 34(3), 321–329 (2007)
  * [12] Durtschi, C., Hillison, W., Pacini, C.: The effective use of benford’s law to assist in detecting fraud in accounting data. Journal of forensic accounting 5(1), 17–34 (2004)
  * [13] Haroush, M., Frostig, T., Heller, R., Soudry, D.: A statistical framework for efficient out of distribution detection in deep neural networks. In: International Conference on Learning Representations (2021)
  * [14] Hashemi, V., Křetínskỳ, J., Mohr, S., Seferis, E.: Gaussian-based runtime detection of out-of-distribution inputs for neural networks. In: International Conference on Runtime Verification. pp. 254–264. Springer (2021)
  * [15] Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. In: International Conference on Learning Representations (2018)
  * [16] Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
  * [17] Hendrycks, D., Mazeika, M., Dietterich, T.: Deep anomaly detection with outlier exposure. arXiv preprint arXiv:1812.04606 (2018)
  * [18] Henzinger, T.A., Lukina, A., Schilling, C.: Outside the box: Abstraction-based monitoring of neural networks. arXiv preprint arXiv:1911.09032 (2019)
  * [19] Huang, R., Geng, A., Li, Y.: On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems 34, 677–689 (2021)
  * [20] Koh, P.W., Sagawa, S., Marklund, H., Xie, S.M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R.L., Gao, I., et al.: Wilds: A benchmark of in-the-wild distribution shifts. In: International Conference on Machine Learning. pp. 5637–5664. PMLR (2021)
  * [21] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Communications of the ACM 60(6), 84–90 (2017)
  * [22] Lam, E.Y., Goodman, J.W.: A mathematical analysis of the dct coefficient distributions for images. IEEE transactions on image processing 9(10), 1661–1666 (2000)
  * [23] Lin, J.: Divergence measures based on the shannon entropy. IEEE Transactions on Information theory 37(1), 145–151 (1991)
  * [24] Milani, S., Tagliasacchi, M., Tubaro, S.: Discriminating multiple jpeg compressions using first digit features. APSIPA Transactions on Signal and Information Processing 3 (2014)
  * [25] Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 427–436 (2015)
  * [26] Olber, B., Radlak, K., Popowicz, A., Szczepankiewicz, M., Chachula, K.: Detection of out-of-distribution samples using binary neuron activation patterns. arXiv preprint arXiv:2212.14268 (2022)
  * [27] Pasquini, C., Boato, G., Pérez-González, F.: Multiple jpeg compression detection by means of benford-fourier coefficients. In: 2014 IEEE International Workshop on Information Forensics and Security (WIFS). pp. 113–118. IEEE (2014)
  * [28] Pérez-González, F., Heileman, G.L., Abdallah, C.T.: Benford’s law in image processing. In: 2007 IEEE International Conference on Image Processing. vol. 1, pp. I–405. IEEE (2007)
  * [29] Raimi, R.A.: The first digit problem. The American Mathematical Monthly 83(7), 521–538 (1976)
  * [30] Rao, K.R., Yip, P.: Discrete cosine transform: algorithms, advantages, applications. Academic press, San Diego, CA, United States (2014)
  * [31] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
  * [32] Sahu, S.K., Java, A., Shaikh, A., Kilcher, Y.: Rethinking neural networks with benford’s law. arXiv preprint arXiv:2102.03313 (2021)
  * [33] Schirrmeister, R., Zhou, Y., Ball, T., Zhang, D.: Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. Advances in Neural Information Processing Systems 33, 21038–21049 (2020)
  * [34] Singh, N., Bansal, R.: Analysis of benford’s law in digital image forensics. In: 2015 International Conference on Signal Processing and Communication (ICSC). pp. 413–418. IEEE (2015)
  * [35] Stano, M., Benesova, W., Martak, L.S.: Explaining predictions of deep neural classifier via activation analysis. arXiv preprint arXiv:2012.02248 (2020)
  * [36] Steinhardt, J., Charikar, M., Valiant, G.: Resilience: A criterion for learning in the presence of arbitrary outliers. arXiv preprint arXiv:1703.04940 (2017)
  * [37] Yang, J., Zhou, K., Li, Y., Liu, Z.: Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 (2021)
  * [38] Zhang, M., Zhang, A., Xiao, T.Z., Sun, Y., McDonagh, S.: Out-of-distribution detection with class ratio estimation. arXiv preprint arXiv:2206.03955 (2022)