# Models Out Of Line:
A Fourier Lens On Distribution Shift Robustness

Sara Fridovich-Keil†, Brian R. Bartoldson$\ddagger$, James
Diffenderfer$\ddagger$,
Bhavya Kailkhura$\ddagger$, Peer-Timo Bremer$\ddagger$
†UC Berkeley, $\ddagger$Lawrence Livermore National Laboratory Corresponding
author<EMAIL_ADDRESS>
###### Abstract

Improving the accuracy of deep neural networks (DNNs) on out-of-distribution
(OOD) data is critical to an acceptance of deep learning (DL) in real world
applications. It has been observed that accuracies on in-distribution (ID)
versus OOD data follow a linear trend and models that outperform this baseline
are exceptionally rare (and referred to as “effectively robust”). Recently,
some promising approaches have been developed to improve OOD robustness: model
pruning, data augmentation, and ensembling or zero-shot evaluating large
pretrained models. However, there still is no clear understanding of the
conditions on OOD data and model properties that are required to observe
effective robustness. We approach this issue by conducting a comprehensive
empirical study of diverse approaches that are known to impact OOD robustness
on a broad range of natural and synthetic distribution shifts of CIFAR-10 and
ImageNet. In particular, we view the “effective robustness puzzle" through a
Fourier lens and ask how spectral properties of both models and OOD data
influence the corresponding effective robustness. We find this Fourier lens
offers some insight into why certain robust models, particularly those from
the CLIP family, achieve OOD robustness. However, our analysis also makes
clear that no known metric is consistently the best explanation (or even a
strong explanation) of OOD robustness. Thus, to aid future research into the
OOD puzzle, we address the gap in publicly-available models with effective
robustness by introducing a set of pretrained models— _RobustNets_ —with
varying levels of OOD robustness.

##  1 Introduction

Deep learning (DL) holds great promise for solving difficult real-world
problems in domains such as healthcare, autonomous driving, and cyber-physical
systems. The major roadblock in adopting DL for real-world applications is
that real-world data often deviates from the training data due to noise,
corruptions, or other changes in distribution that may have temporal or
spatial causes [3, 14, 29, 31]. DL models are known to be highly brittle under
such distribution shifts, which raises the risk of making incorrect
predictions with harmful consequences. Designing approaches to learn models
that achieve both high accuracy on in-distribution data and high robustness to
distribution shifts is paramount to the safe adoption of DL in real-world
applications.

Although a performance drop under distribution shifts is expected, several
recent works [1, 23] have noticed an intriguing phenomenon: accuracy on in-
distribution (ID) versus accuracy on distribution-shifted data obeys a linear
trend across a wide range of models, corruption benchmarks, and prediction
tasks. Unfortunately, as nearly all current DL models bear a substantial loss
under distribution shifts, this linear relationship implies that our current
training strategies are insufficient to bridge the gap between ID and out-of-
distribution (OOD) performance. However, despite being exceedingly rare,
models that break this linear trend to obtain effective robustness [1]
exist—such models defy explanation by the wisdom that OOD accuracy and ID
accuracy should correlate strongly, they are “out of line”. Identifying when
and why such models arise is key to overcoming OOD brittleness in DL models.

Related research efforts have focused on improving OOD robustness via new
techniques such as model pruning [7], data augmentation [15], and ensembling
large pre-trained models [25, 34]. While these efforts have created
significant excitement in the ML robustness community, it remains unclear (a)
under which conditions on OOD data these methods can improve effective
robustness (ER), and (b) which underlying model properties make a DNN
effectively robust. A more in-depth understanding of these phenomena is likely
to help in bridging the gap between ID and OOD performance.

Towards achieving this goal, we carry out a comprehensive empirical study
leveraging models trained using each of the aforementioned state-of-the-art
approaches to improving OOD robustness. We analyze the impact of these
robustness improvement methods on various model architectures, two clean
datasets (CIFAR-10 and ImageNet), and a broad range of natural and synthetic
distribution shifts. In particular, we view the OOD robustness puzzle through
a Fourier lens and ask how spectral properties of OOD data and models
influence their effective robustness. In this process, we design new metrics
that capture Fourier sensitivity of models as test data moves farther away
from the training data manifold. We make the following contributions:

  * •

_A new perspective on the state of OOD robustness:_ across diverse models,
metrics, and distribution shifts, we observe that no one single metric rules
them all, not even in-distribution accuracy. This accents the potential need
for a multi-faceted approach to understanding OOD robustness, in contrast to
ID generalization where single properties like model flatness can enjoy
predictive success across an array of models [18].

  * •

_Design of Fourier sensitivity metrics:_ our new metrics quantify various
model properties, including spectral ones that can explain the effective
robustness of ensembled CLIP models [34] better than ID accuracy.

  * •

_The first collection of publicly-available effectively robust models:_ to
help crack the OOD robustness puzzle, we make available (coming soon) a
dataset of CIFAR-10 models with effective robustness that stems from training
under a variety of data augmentation and pruning schemes.

##  2 Related work

Distribution shift fragility is a well-documented phenomenon among neural
networks [28, 29, 14, 13, 22] in which a model trained on an ID dataset
performs markedly worse when evaluated on OOD data, even when humans find the
OOD data just as easy to classify. These OOD accuracy gaps often follow a
linear relationship between ID and OOD accuracy, but the slope and intercept
of the linear trendline varies depending on the specific ID and OOD datasets
[22, 10]. In our work, we focus on two popular image classification benchmark
ID datasets, CIFAR-10 [20] and ImageNet [5], and two OOD test datasets for
each, CIFAR-10.1 [28], CIFAR-10-C [14], ImageNetV2 [29], and ImageNet-C [13].

While most models follow the linear ID-OOD trendlines of Miller et al. [22],
some models are “effectively robust" and appear above the trendline, with OOD
accuracy higher than expected given ID accuracy [1]. These robust models can
arise through various methods: by pruning a model as a sort of regularization
[7], by training with data augmentation intended to mimic the expected OOD
test data [15], or by pretraining on a larger and more diverse dataset and
evaluating zero-shot [25], partially finetuning [1], or interpolating the
weights of a zero-shot and a finetuned model [34].

Although as a community we are beginning to uncover robust training methods,
it remains a mystery why these particular methods achieve robustness on
particular distribution shifts. We study this mystery via a Fourier lens based
on prior work involving both image frequency and function frequency analyses.
Ortiz-Jimenez et al. [24], Yin et al. [36], and Sun et al. [33] study model
sensitivity to Fourier perturbations of the input images and analyze how
different data augmentations produce different Fourier sensitivities (and
correspondingly robustness to image perturbations of different frequencies).
Another line of research focuses on spectral bias [9, 2, 35, 26], wherein
models prioritize learning low frequency functions over the input space. This
bias towards low function frequencies can also affect robustness and is
influenced by training hyperparameters like data augmentation and weight decay
[9].

Among the model properties we study as potential predictors of OOD accuracy
are ID accuracy [22], model Jacobian norm [16], the linear pixel-space
interpolation metrics of Fridovich-Keil et al. [9], and our own Fourier
amplitude and phase interpolation metrics. We study these metrics over a wide
range of robust and nonrobust models, including sparse models from
Diffenderfer et al. [7] on CIFAR-10 and pretrained models from CLIP [25, 34]
and RobustBench [4] on ImageNet.

##  3 Methods

#### Datasets.

We consider two standard image classification tasks: CIFAR-10 [20] and
ImageNet [5]. CIFAR-10 consists of low-resolution (32 by 32 pixels) color
images from 10 animal and object classes, divided into 50000 training and
10000 test images; ImageNet consists of higher-resolution (cropped to 224 by
224 pixels) color images from 1000 animal and object classes, with more than a
million training images and 50000 validation images. For each of these "in-
distribution" (ID) datasets we consider two "out of distribution" (OOD)
datasets: one defined by a set of synthetic corruptions of the original
test/validation images (CIFAR-10-C [14] and ImageNet-C [13], respectively) and
another defined by a re-collection of new test images following as closely as
possible the original dataset creation procedure (CIFAR-10.1 [28] and
ImageNetV2 [29]).

#### Models.

In our CIFAR-10 experiments we use 3 different model architectures: Conv8 [8],
ResNet18 [12], and VGG16 [32]. For each model architecture, we consider a
range of model pruning strategies that have been studied for their effect on
OOD robustness [7, 17] as some of these pruned models were demonstrated to
have higher OOD robustness than dense models. Namely, lottery-ticket style
pruning methods [8, 30, 27, 6] were able to provide robustness gains on
CIFAR-10-C [7]. We outline the various pruning techniques and sparsity levels
we utilized in detail in Section 4.2.

In our ImageNet experiments we use 28 standard (nonrobust) pretrained models
from Torchvision [21], 5 pretrained models from the ImageNet-C leaderboard on
RobustBench [4], and 33 models obtained by robustness-enhancing weight-space
interpolation [34] between zero-shot and ImageNet-finetuned CLIP ViT-B/16,
ViT-B/32, and ViT-L/14 [25].

#### Previously-proposed metrics: ID accuracy, Jacobian norm, pixel
interpolation.

Because OOD robustness is a complex and multi-faceted problem, we make use of
multiple previously-proposed model measurements and apply them to study
effectively robust models. To the best of our knowledge, our work is the first
application of most of these metrics (except for in-distribution accuracy) to
the study of effective robustness. Specifically, we evaluate four previously-
proposed metrics that have been observed to correlate with some notion of
generalization: accuracy on unseen in-distribution data ("ID accuracy") [22],
Jacobian norm [16], and within-class and between-class pixel interpolation
high frequency fraction [9]. These latter metrics are analogous to the high
frequency fractions we compute for amplitude and phase interpolation
(described below), except that interpolating paths simply interpolate in pixel
space between two images, where the two images are from either the same class
("within-class") or different classes ("between-class") [9]. Prior work
observed that in-distribution generalization typically improves as within-
class high frequency fraction decreases and between-class high frequency
fraction increases [9]. Reducing the norm of the Jacobian of model outputs
with respect to input data can push the decision boundary away from the
training points, providing robustness to random perturbations of the input
data [16]. Thus, models with smaller Jacobian norms may perform better on
corrupted/shifted data. We estimate the norm of the Jacobian using a random-
projection-based approach [16]; further details can be found in the
supplement.

#### Fourier interpolation metrics.

We also introduce novel Fourier interpolation metrics to capture more
information about model behavior that may help answer the robustness puzzle.
Inspired by the pixel-space interpolation procedure of Fridovich-Keil et al.
[9], we propose evaluating the smoothness of a model’s predictions along image
paths that perturb the Fourier amplitude or phase information of one test
image towards another, while preserving the Fourier phase or amplitude
(respectively) information of the original image. The intuition for this type
of interpolation is that most of the semantic content of an image is contained
within its Fourier phases, which encode structural information like edges.
Accordingly, we can perturb the Fourier amplitude of an image without
destroying its semantic meaning, and we can reliably destroy semantic content
by perturbing Fourier phase. Example Fourier amplitude and phase interpolating
paths on ImageNet are shown in Figure 1; example paths on CIFAR-10 are
included in the appendix.

Figure 1: Example Fourier amplitude (top) and phase (bottom) interpolating
paths from ImageNet. Each path includes 100 images; every 15th image is
visualized here. The first image along each path is an unaltered image from
the original test set; the last image has the same Fourier phases (top) or
amplitudes (bottom) as the original but has some of the Fourier amplitudes
(top) or phases (bottom) of a random other image from the validation set.
Amplitude interpolation produces a corruption that preserves semantic content,
whereas phase interpolation destroys semantic content.

More precisely, we compute an interpolating path by first randomly selecting
two images $\textbf{x}_{0}$ and $\textbf{x}_{1}$ from the ID test/validation
set. We perform standard preprocessing on each image, which involves cropping
to a standard size (for ImageNet) and normalizing each pixel by the training
mean and standard deviation (for both CIFAR-10 and ImageNet). We then compute
the two-dimensional discrete Fourier transform (DFT) of each image and
separate the complex-valued results into amplitude and phase components
$\textbf{a}_{i}$ and $\textbf{p}_{i}$ for $i\in\\{0,1\\}$. To construct an
image along an amplitude interpolating path, we retain the original phase
$\textbf{p}_{0}$ and interpolate the low-frequency amplitude as
$(1-\lambda)\textbf{a}_{0}+\lambda\textbf{a}_{1}$ (while preserving the high
frequency amplitude), where $\lambda\in[0,1]$, and produce an image via the
inverse discrete Fourier transform. Phase interpolation is defined
analogously; we retain the original amplitude and interpolate the low-
frequency phase. We choose a frequency threshold for what is considered low
frequency for interpolation based on visual inspection to ensure that
amplitude interpolation preserves semantic meaning and phase interpolation
destroys it. We interpolate the lowest 40% of image frequencies for both
amplitude and phase on CIFAR-10, the lowest 20% of image frequencies for phase
on ImageNet, and all image frequencies for amplitude on ImageNet. We choose
100 evenly-spaced $\lambda$ values to define each path, and choose 5000 random
paths for CIFAR-10 and 7000 for ImageNet. The images in Figure 1 are rescaled
for visualization (essentially undoing the initial pixel-wise normalization).

We evaluate each model on each path, producing a probability vector over the
available classes for each image along the path. We compute two metrics based
on these path predictions: high frequency fraction (HFF) and consistent
distance (CD). For high frequency fraction, we take the one-dimensional DFT of
the 100 model predictions along the path, average the Fourier amplitudes among
all the classes, and compute the fraction of the total amplitude above a
frequency threshold. This metric is always between 0 and 1 (usually between
0.1 and 0.3). The higher the average high frequency fraction, the more
sensitive a model is to the given Fourier perturbation of its input images.

We also consider consistent distance, which is defined as the index of the
first image along the path that the model classifies differently (by highest
softmax prediction) than the original image. This metric is always between 1
and 100, and is often at least 40. Consistent distance is intended as a more
intuitive metric to capture the same notion of robustness to corruptions of
image Fourier content. In particular, Fourier amplitude corruptions change
image statistics but do not hamper human semantic identification; a more
robust model in this sense should have a lower high frequency fraction and a
higher consistent distance on average compared to a less robust model.

##  4 Results

We are interested in understanding _when_ (under what conditions on the
distribution shift) and _why_ (via model properties) neural networks can
exhibit effective robustness. We perform a systematic study of three tunable
"knobs" that are available to the neural network practitioner and have been
shown to impact OOD robustness: pruning [7], data augmentation [15], and
weight ensembling [34]. For each knob, we evaluate a benchmark set of
pretrained CIFAR-10 or ImageNet models on the original in-distribution test
set, several OOD test sets, and a suite of model property metrics capturing
local smoothness (via model Jacobian norm), frequency response to pixel-space
interpolation within and between classes [9], and Fourier amplitude and phase
sensitivity (via high frequency fraction and consistent distance).
Representative results from our analysis of these three knobs are presented in
Sections 4.2, 4.3, and 4.4. For each dataset we consider one natural
distribution shift (CIFAR-10.1 or ImageNetV2) as well as six synthetic
corruptions (from CIFAR-10-C or ImageNet-C) comprised of two low-frequency
corruptions, two mid-frequency corruptions, and two high-frequency corruptions
(in that order). Full results on all the corruptions in CIFAR-10-C and
ImageNet-C are deferred to the supplement. In all figures, we show 95%
confidence intervals around each measurement (Clopper-Pearson for accuracy
measurements and Gaussian bounds for averages of other metrics).

We begin by comparing the spectral properties of different distribution shifts
in Section 4.1. In the following sections we find that the relationship
between model properties and OOD robustness is dependent upon these spectral
statistics of the specific distribution shift. This analysis highlights the
difficulty of using a single model property to understand OOD performance but
also indicates that our proposed metrics may play an important role in solving
the OOD puzzle.

###  4.1 Spectral characterization of distribution shifts

To explore the behavior of models on OOD data, it is beneficial to
characterize the nature of the OOD data with respect to the ID test data.
Specifically, considering the difference in power spectral density (PSD)
between each of the OOD test sets and the original ID test set allows us to
categorize each OOD test set as being concentrated on low, mid, or high
frequency perturbations with respect to the ID test set. We perform this
characterization for CIFAR-10.1 and each of the 15 corruptions in CIFAR-10-C;
computational details for PSD are provided in the appendix. Figure 2 shows the
distribution shift PSDs for CIFAR-10.1 together with two low (brightness,
contrast), mid (defocus blur, pixelate), and high (gaussian noise, impulse
noise) frequency CIFAR-10-C corruptions. Of note, this PSD characterization
illustrates that the OOD shift encountered on CIFAR-10.1 data is composed of
low-frequency information, much like the brightness and contrast CIFAR-10-C
corruptions.

CIFAR-10.1

Brightness

Contrast

Defocus Blur

Pixelate

Gaussian Noise

Impulse Noise

Figure 2: Power spectral densities for a selection of low (CIFAR-10.1,
brightness, contrast), mid (defocus blur, pixelate), and high (gaussian noise,
impulse noise) frequency shifts w.r.t. CIFAR-10.

###  4.2 When and why does model pruning confer effective robustness?

Diffenderfer et al. [7] demonstrated the nuanced behavior that model pruning
has on OOD robustness, specifically with respect to the distribution shift
from CIFAR-10 to CIFAR-10-C. Notably, pruned models are capable of providing
superior OOD robustness compared to dense, or unpruned, models. Furthermore,
the effect, either positive or negative, and degree of robustness arising from
model pruning is dependent on both the model architecture and the pruning
algorithm. It remains unknown which properties of these pruned models
contribute to their robustness on OOD data.

In an effort to better understand these findings, we further investigate this
behavior by considering the three categories of pruning used in Diffenderfer
et al. [7]: _traditional_ (fine-tuning [11], gradual magnitude pruning [37]),
_rewinding lottery-tickets_ (weight-rewinding [8], learning-rate rewinding
[30]), and _initialization lottery-tickets_ (edgepopup [27], biprop [6]). Note
that we often abbreviate "lottery tickets" as LT. We provide details on each
of the pruning techniques in the supplement. For each pruning strategy and
each architecture, we prune models to 50%, 60%, 80%, 90%, and 95% sparsity.
For all pruning methods, pruning is performed in an unstructured (i.e.,
individual weights are pruned) and global manner (i.e., prune to a given
sparsity across the entire network). Additionally, for traditional and
initialization LTs, pruning was performed in a layerwise fashion (i.e., where
each network layer is pruned to the given sparsity level). In Figure 3 we
investigate _when_ (on which types of distribution shifts) and in Table 1 we
study _why_ (via corresponding model property measurements) these differently-
pruned CIFAR-10 models achieve their varying degrees of robustness.

Figure 3: When does pruning affect OOD robustness?

| Traditional | Rewinding LT | Initialization LT
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
CIFAR-10.1 | ID accuracy | 0.85 | 0.85 | 0.9 | 0.59 | 0.79 | 0.87
Jacobian | -0.0 | 0.16 | -0.01 | 0.36 | 0.0 | 0.19
Pixel - within | -6.42 | 0.79 | -8.54 | 0.66 | -6.7 | 0.78
Pixel - between | 8.45 | 0.48 | -1.29 | 0.25 | 7.45 | 0.45
Amp - HFF | -6.11 | 0.72 | -5.97 | 0.31 | -4.73 | 0.75
Amp - CD | 0.04 | 0.6 | 0.05 | 0.52 | 0.04 | 0.75
Phase - HFF | -8.09 | 0.26 | -5.66 | 0.24 | -6.78 | 0.41
Phase - CD | 0.03 | 0.33 | 0.01 | 0.26 | 0.02 | 0.4

Brightness | ID accuracy | 1.0 | 0.96 | 0.9 | 0.89 | 0.97 | 0.97
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.2 | -0.01 | 0.49 | 0.0 | 0.29
Pixel - within | -7.41 | 0.88 | -7.47 | 0.79 | -8.09 | 0.85
Pixel - between | 9.73 | 0.51 | 0.6 | 0.34 | 9.32 | 0.57
Amp - HFF | -6.73 | 0.7 | -6.47 | 0.53 | -5.9 | 0.88
Amp - CD | 0.04 | 0.63 | 0.04 | 0.67 | 0.05 | 0.84
Phase - HFF | -7.7 | 0.18 | -8.02 | 0.31 | -6.66 | 0.35
Phase - CD | 0.03 | 0.27 | 0.02 | 0.23 | 0.02 | 0.4

Contrast | ID accuracy | 0.91 | 0.85 | 0.89 | 0.66 | 0.87 | 0.91
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.15 | -0.01 | 0.4 | 0.0 | 0.25
Pixel - within | -6.59 | 0.74 | -8.13 | 0.64 | -7.4 | 0.81
Pixel - between | 8.08 | 0.39 | 3.92 | 0.3 | 8.46 | 0.51
Amp - HFF | -6.13 | 0.63 | -5.9 | 0.4 | -5.45 | 0.86
Amp - CD | 0.04 | 0.65 | 0.04 | 0.54 | 0.05 | 0.83
Phase - HFF | -8.83 | 0.26 | -9.17 | 0.3 | -7.18 | 0.4
Phase - CD | 0.03 | 0.29 | 0.02 | 0.23 | 0.02 | 0.42

Defocus Blur | ID accuracy | 0.55 | 0.51 | 0.86 | 0.63 | 0.42 | 0.39
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.17 | -0.01 | 0.64 | 0.01 | 0.45
Pixel - within | -4.54 | 0.55 | -7.82 | 0.61 | -4.49 | 0.41
Pixel - between | 6.35 | 0.36 | 8.82 | 0.26 | 7.84 | 0.52
Amp - HFF | -4.01 | 0.42 | -5.96 | 0.43 | -2.79 | 0.45
Amp - CD | 0.03 | 0.37 | 0.04 | 0.55 | 0.02 | 0.37
Phase - HFF | -4.78 | 0.16 | -6.51 | 0.34 | 1.13 | 0.18
Phase - CD | 0.02 | 0.19 | 0.01 | 0.37 | -0.0 | 0.12

Pixelate | ID accuracy | 0.73 | 0.44 | 0.85 | 0.61 | -0.31 | 0.33
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.11 | -0.01 | 0.75 | 0.0 | 0.13
Pixel - within | -6.8 | 0.46 | -8.09 | 0.6 | 2.67 | 0.22
Pixel - between | 10.92 | 0.45 | 14.0 | 0.25 | 0.01 | 0.23
Amp - HFF | -3.92 | 0.3 | -7.48 | 0.57 | 2.11 | 0.44
Amp - CD | 0.02 | 0.31 | 0.05 | 0.62 | -0.02 | 0.48
Phase - HFF | -0.17 | 0.26 | -8.55 | 0.31 | 11.15 | 0.61
Phase - CD | 0.01 | 0.25 | 0.02 | 0.39 | -0.03 | 0.5

Gaussian Noise | ID accuracy | 0.55 | 0.16 | 0.24 | 0.1 | -1.16 | 0.47
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.05 | -0.0 | 0.15 | -0.0 | 0.05
Pixel - within | -6.17 | 0.24 | -2.89 | 0.14 | 10.65 | 0.38
Pixel - between | 7.48 | 0.11 | 9.49 | 0.14 | -10.94 | 0.31
Amp - HFF | -2.28 | 0.09 | -5.56 | 0.11 | 7.09 | 0.47
Amp - CD | 0.01 | 0.12 | 0.03 | 0.17 | -0.06 | 0.47
Phase - HFF | -0.52 | 0.18 | -11.92 | 0.25 | 16.98 | 0.49
Phase - CD | 0.0 | 0.13 | 0.03 | 0.18 | -0.04 | 0.42

Impulse Noise | ID accuracy | 0.04 | 0.0 | -0.17 | 0.5 | -0.58 | 0.39
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.05 | 0.01 | 0.5 | -0.0 | 0.05
Pixel - within | -1.26 | 0.03 | 1.31 | 0.54 | 5.47 | 0.33
Pixel - between | 3.21 | 0.07 | -17.81 | 0.35 | -5.25 | 0.28
Amp - HFF | 0.13 | 0.01 | 0.48 | 0.36 | 3.43 | 0.38
Amp - CD | -0.01 | 0.04 | -0.0 | 0.47 | -0.03 | 0.38
Phase - HFF | 4.05 | 0.1 | -7.43 | 0.41 | 8.61 | 0.43
Phase - CD | -0.01 | 0.06 | 0.02 | 0.26 | -0.02 | 0.38

Table 1: Can model metrics explain why pruning affects OOD robustness?

Because different model architectures respond differently to different pruning
methods, in each subplot of Figure 3 we color-code models by their
architecture and fit a separate probit-domain OOD vs. ID accuracy regression
line to each architecture, so that each regression captures a range of
different pruning amounts (what fraction of weights are pruned). We display
these regressions with opacity proportional to the fit quality $R^{2}$, so
that regression lines that fit the data well are dark and easily seen, while
lower-quality linear fits are more transparent. In the corresponding Tables 1,
each reported slope ($m$) and $R^{2}$ value is the average of the
corresponding values among the three architectures we consider. For ease of
reading, we bold average $R^{2}$ values above 0.5, and among these entries we
highlight the best (bright green) and second-best (faded green) $R^{2}$ values
for each pruning method on each OOD dataset. Although these results are
nuanced and raise additional open questions, we summarize a few key takeaways
from our CIFAR-10 model pruning experiments:

  * •

As reported in Miller et al. [22], ID accuracy is often a strong predictor of
OOD accuracy; this trend holds across different pruning methods. However, the
linear trend breaks down for mid and high frequency image corruptions like
pixelate, Gaussian noise, and impulse noise.

  * •

Even for low-frequency corruptions, different model architectures might lie on
either the same or different robustness lines (as evidenced by comparing
brightness and contrast).

  * •

Initialization LT pruning confers robustness to high-frequency corruptions and
can induce a negative correlation between ID and OOD accuracy for these
corruptions.

  * •

Jacobian norm is rarely a good predictor of OOD accuracy, but it does
correlate negatively (lower norm is better) for models pruned by rewinding
methods on mid-frequency corruptions.

  * •

As reported in [9], lower high frequency fraction HFF (i.e. smoother
predictions) for within-class pixel interpolation is a good predictor of OOD
accuracy, at least for low-frequency distribution shifts where ID accuracy is
also predictive.

  * •

Amplitude and phase interpolation are sometimes predictive of OOD accuracy,
particularly for initialization LT pruning, and in the expected direction of
correlation (a better model should have lower HFF and higher CD). However,
even these metrics fail to explain most of the robustness behavior for high
(and some mid) frequency distribution shifts on CIFAR-10.

###  4.3 When and why does data augmentation confer effective robustness?

A common practice to protect against distribution shift is to train with data
augmentation [15]. Accordingly, we revisit each of the CIFAR-10 models
considered in Section 4.2 with three different variations of data augmentation
used during training: no augmentation (clean), AugMix [15], and Gaussian noise
augmentation [19]. In Figure 4 we investigate _when_ (on which types of
distribution shifts) and in Table 2 we study _why_ (via corresponding model
property measurements) these differently-augmented CIFAR-10 models achieve
varying degrees of robustness. As in Figure 3 and Table 1, we fit a separate
regression line to each model architecture and average the resulting slope and
$R^{2}$ values. The models that contribute to each regression line therefore
share an architecture and an augmentation strategy, but vary by pruning method
and amount. Our findings in this experiment echo those in Section 4.2, but we
make the following additional observations:

  * •

The closer the match between the training data and the OOD evaluation data in
terms of their spectral profile, the better ID accuracy, within-class pixel
interpolation, and amplitude interpolation HFF and CD are at predicting OOD
accuracy. This is evident from the high $R^{2}$ values for these metrics when
predicting defocus blur accuracy for models trained with AugMix, and when
predicting Gaussian noise or impulse noise accuracy for models trained with
Gaussian noise.

  * •

The closer the match (in terms of PSD) between the training data and the OOD
data, the better the OOD accuracy. This unsurprising result is evident from
the higher accuracies of AugMix-trained models on contrast and defocus blur
corruptions and the higher accuracies of Gaussian-noise-trained models on
Gaussian noise and impulse noise corruptions.

###  4.4 CLIP: Weight ensembling a robust model on ImageNet

Perhaps the most stunning example of OOD robustness for image classification
is CLIP [25], which showed that a large model pretrained on a massive dataset
and evaluated zero-shot on ImageNet achieves effective robustness on many OOD
test sets simultaneously. Andreassen et al. [1] found that finetuning these
robust models towards ImageNet improves ID accuracy on ImageNet but erodes
effective robustness, and Wortsman et al. [34] offered a "best of both worlds"
solution by interpolating in weight space between a pretrained zero-shot model
and its finetuned counterpart, improving ID accuracy on ImageNet without
sacrificing robustness. Accordingly, we study a set of 33 CLIP models obtained
by interpolating the pretrained and finetuned versions of CLIP ViT-B/16,
ViT-B/32, and ViT-L/14 models [25, 34] with various relative weights on the
two checkpoints. We compare this CLIP model set to a benchmark set of 28
standard (nonrobust) ImageNet pretrained models from Torchvision [21] as well
as 5 pretrained models from the RobustBench [4] ImageNet-C leaderboard; these
models typically use data augmentation to improve corruption performance but
are not nearly as effectively robust as the CLIP models. We make the following
observations in Figure 5 and Table 3:

  * •

Although in-distribution accuracy is a reasonable predictor of OOD accuracy
for most models and distribution shifts, it performs markedly worse when
predicting OOD accuracy for CLIP models (and a couple of effectively robust
RobustBench models on some OOD shifts).

  * •

Fourier interpolation metrics, particularly HFF and CD for amplitude
interpolation, are strong predictors (often $R^{2}\geq 0.9$) of OOD
performance for CLIP models across OOD shifts. This observation helps answer
_why_ CLIP models are more robust: they are less sensitive to perturbations of
Fourier amplitude, which preserve image semantics for humans but confuse
nonrobust models.

Figure 4: When does augmentation affect OOD robustness?

| Clean | AugMix | Gaussian
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
CIFAR-10.1 | ID accuracy | 0.91 | 0.92 | 0.91 | 0.93 | 0.89 | 0.86
Jacobian | 0.01 | 0.14 | 0.01 | 0.35 | -0.0 | 0.05
Pixel - within | -8.45 | 0.79 | -6.88 | 0.86 | -7.13 | 0.73
Pixel - between | 10.68 | 0.57 | 8.55 | 0.67 | 6.95 | 0.45
Amp - HFF | -5.43 | 0.79 | -5.93 | 0.83 | -5.64 | 0.71
Amp - CD | 0.04 | 0.74 | 0.06 | 0.66 | 0.04 | 0.68
Phase - HFF | -7.36 | 0.36 | 2.63 | 0.06 | 4.18 | 0.09
Phase - CD | 0.02 | 0.45 | 0.02 | 0.19 | 0.03 | 0.29

Brightness | ID accuracy | 0.99 | 0.98 | 1.02 | 0.99 | 1.01 | 0.97
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.12 | 0.01 | 0.37 | 0.0 | 0.06
Pixel - within | -9.36 | 0.87 | -7.7 | 0.91 | -7.51 | 0.74
Pixel - between | 11.52 | 0.61 | 9.73 | 0.73 | 8.54 | 0.6
Amp - HFF | -5.94 | 0.84 | -6.68 | 0.88 | -6.32 | 0.78
Amp - CD | 0.04 | 0.78 | 0.06 | 0.7 | 0.04 | 0.69
Phase - HFF | -7.38 | 0.33 | 3.32 | 0.07 | 7.44 | 0.18
Phase - CD | 0.02 | 0.44 | 0.02 | 0.19 | 0.03 | 0.24

Contrast | ID accuracy | 1.01 | 0.87 | 1.25 | 0.93 | 0.55 | 0.54
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.08 | 0.02 | 0.44 | -0.0 | 0.05
Pixel - within | -9.88 | 0.79 | -9.79 | 0.9 | -5.22 | 0.61
Pixel - between | 11.55 | 0.5 | 12.76 | 0.78 | 3.12 | 0.16
Amp - HFF | -6.55 | 0.86 | -8.23 | 0.84 | -3.64 | 0.44
Amp - CD | 0.05 | 0.83 | 0.07 | 0.6 | 0.02 | 0.47
Phase - HFF | -9.91 | 0.44 | 8.98 | 0.11 | 0.25 | 0.03
Phase - CD | 0.03 | 0.53 | 0.02 | 0.12 | 0.02 | 0.22

Defocus Blur | ID accuracy | 0.47 | 0.34 | 0.98 | 0.98 | 0.7 | 0.6
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.04 | 0.01 | 0.39 | -0.0 | 0.12
Pixel - within | -5.33 | 0.45 | -7.44 | 0.94 | -6.77 | 0.72
Pixel - between | 5.81 | 0.24 | 9.54 | 0.76 | 5.42 | 0.31
Amp - HFF | -3.27 | 0.42 | -6.39 | 0.88 | -5.19 | 0.64
Amp - CD | 0.02 | 0.36 | 0.06 | 0.67 | 0.03 | 0.6
Phase - HFF | -2.38 | 0.11 | 4.59 | 0.08 | 3.9 | 0.08
Phase - CD | 0.01 | 0.13 | 0.02 | 0.18 | 0.03 | 0.28

Pixelate | ID accuracy | 0.11 | 0.13 | 0.42 | 0.45 | 0.68 | 0.55
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.13 | 0.0 | 0.04 | -0.01 | 0.14
Pixel - within | -2.12 | 0.14 | -3.27 | 0.49 | -6.33 | 0.64
Pixel - between | 2.03 | 0.16 | 3.7 | 0.29 | 5.31 | 0.3
Amp - HFF | -0.4 | 0.18 | -3.0 | 0.49 | -5.1 | 0.57
Amp - CD | -0.0 | 0.13 | 0.03 | 0.47 | 0.03 | 0.54
Phase - HFF | 5.4 | 0.2 | 2.31 | 0.05 | 5.28 | 0.1
Phase - CD | -0.01 | 0.15 | 0.01 | 0.08 | 0.03 | 0.25

Gaussian Noise | ID accuracy | -1.11 | 0.23 | 0.08 | 0.11 | 0.88 | 0.94
---|---|---|---|---|---|---|---
Jacobian | -0.03 | 0.31 | -0.01 | 0.11 | -0.0 | 0.08
Pixel - within | 6.83 | 0.09 | -0.81 | 0.12 | -7.08 | 0.85
Pixel - between | -17.36 | 0.23 | -0.34 | 0.07 | 7.32 | 0.57
Amp - HFF | 5.63 | 0.25 | -1.26 | 0.08 | -5.74 | 0.79
Amp - CD | -0.04 | 0.23 | 0.03 | 0.15 | 0.03 | 0.7
Phase - HFF | 10.47 | 0.26 | 0.8 | 0.05 | 6.39 | 0.16
Phase - CD | -0.03 | 0.27 | 0.02 | 0.15 | 0.03 | 0.25

Impulse Noise | ID accuracy | -0.64 | 0.32 | 0.33 | 0.26 | 0.75 | 0.67
---|---|---|---|---|---|---|---
Jacobian | -0.01 | 0.27 | 0.0 | 0.13 | -0.0 | 0.1
Pixel - within | 4.67 | 0.16 | -2.64 | 0.26 | -6.54 | 0.71
Pixel - between | -9.44 | 0.26 | 2.62 | 0.23 | 5.93 | 0.42
Amp - HFF | 3.08 | 0.3 | -2.23 | 0.27 | -5.39 | 0.61
Amp - CD | -0.02 | 0.31 | 0.03 | 0.25 | 0.03 | 0.53
Phase - HFF | 5.28 | 0.25 | 0.32 | 0.03 | 4.36 | 0.16
Phase - CD | -0.01 | 0.27 | 0.02 | 0.2 | 0.03 | 0.2

Table 2: Can model metrics explain why augmentation affects OOD robustness?

##  5 Discussion

Our work is only the beginning of a true understanding of what makes models
effectively robust to distribution shifts. For example, Section 4.2 showed
different types of model pruning can confer robustness to different
distribution shifts, and while some of this robustness can be explained by
improved behavior in response to amplitude and phase perturbations, much
remains an open mystery.

As we saw in Section 4.3 (and has been noted by prior work [36, 15]), models
can achieve robustness to some distribution shifts by training with data
augmentations designed to imitate the expected OOD data. This approach can be
a powerful way to mitigate expected test-time distribution shifts but leaves
open the question of how to prepare for the unexpected.

We learned in Section 4.4 that one way in which CLIP models achieve OOD
robustness is by reducing their sensitivity to perturbations of Fourier
amplitude, perhaps making them more attuned to human perceptions of semantic
meaning. This finding provides an exciting opportunity for future work to both
confirm our results across additional robust models and incorporate these
Fourier interpolation metrics into training paradigms, for instance as
regularizers, to explicitly encode robustness in future models.

Figure 5: When and why does CLIP ensembling affect OOD robustness?

| All | CLIP | Non-CLIP
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
ImageNet V2 | ID accuracy | 0.98 | 0.94 | 0.64 | 0.96 | 0.93 | 1.0
Jacobian | 0.0 | 0.14 | -0.0 | 0.96 | 0.0 | 0.0
Pixel - within | -10.2 | 0.88 | -6.21 | 1.0 | -9.8 | 0.92
Pixel - between | 8.39 | 0.74 | 6.46 | 0.78 | 14.29 | 0.73
Amp - HFF | -12.27 | 0.36 | -11.56 | 0.57 | -0.66 | 0.0
Amp - CD | 0.02 | 0.77 | 0.03 | 0.9 | 0.03 | 0.67
Phase - HFF | -4.68 | 0.05 | -5.05 | 0.08 | 19.65 | 0.42
Phase - CD | 0.02 | 0.64 | 0.02 | 0.42 | 0.04 | 0.57

Brightness | ID accuracy | 1.06 | 0.91 | 0.66 | 0.79 | 1.12 | 0.92
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.06 | -0.0 | 0.81 | -0.0 | 0.0
Pixel - within | -10.9 | 0.81 | -6.8 | 0.91 | -11.4 | 0.79
Pixel - between | 8.36 | 0.59 | 6.03 | 0.51 | 18.99 | 0.81
Amp - HFF | -11.2 | 0.25 | -15.99 | 0.83 | -8.87 | 0.01
Amp - CD | 0.02 | 0.71 | 0.03 | 0.99 | 0.04 | 0.82
Phase - HFF | -2.09 | 0.01 | -11.16 | 0.29 | 23.56 | 0.39
Phase - CD | 0.02 | 0.51 | 0.03 | 0.7 | 0.05 | 0.65

Contrast | ID accuracy | 1.38 | 0.6 | 0.25 | 0.16 | 1.28 | 0.6
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.2 | -0.0 | 0.2 | -0.0 | 0.01
Pixel - within | -13.46 | 0.48 | -3.2 | 0.3 | -12.12 | 0.45
Pixel - between | 14.19 | 0.67 | 0.83 | 0.01 | 24.8 | 0.7
Amp - HFF | -23.97 | 0.44 | -13.62 | 0.91 | -26.29 | 0.06
Amp - CD | 0.04 | 0.86 | 0.02 | 0.64 | 0.05 | 0.85
Phase - HFF | -12.78 | 0.11 | -15.9 | 0.88 | 25.25 | 0.22
Phase - CD | 0.04 | 0.69 | 0.03 | 0.98 | 0.07 | 0.59

Defocus Blur | ID accuracy | 1.36 | 0.75 | 0.27 | 0.23 | 1.39 | 0.81
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.15 | -0.0 | 0.27 | -0.0 | 0.01
Pixel - within | -13.75 | 0.66 | -3.38 | 0.38 | -14.13 | 0.7
Pixel - between | 12.1 | 0.64 | 1.29 | 0.04 | 24.5 | 0.78
Amp - HFF | -18.59 | 0.35 | -13.03 | 0.94 | -17.62 | 0.03
Amp - CD | 0.03 | 0.79 | 0.02 | 0.71 | 0.05 | 0.82
Phase - HFF | -6.15 | 0.03 | -14.51 | 0.83 | 32.63 | 0.43
Phase - CD | 0.03 | 0.62 | 0.03 | 0.99 | 0.07 | 0.61

Pixelate | ID accuracy | 1.26 | 0.6 | 0.46 | 0.5 | 1.24 | 0.55
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.08 | -0.0 | 0.55 | -0.0 | 0.06
Pixel - within | -12.06 | 0.47 | -5.12 | 0.67 | -11.47 | 0.4
Pixel - between | 12.07 | 0.59 | 3.47 | 0.22 | 25.27 | 0.71
Amp - HFF | -18.74 | 0.33 | -15.18 | 0.97 | -29.89 | 0.08
Amp - CD | 0.03 | 0.76 | 0.03 | 0.93 | 0.05 | 0.83
Phase - HFF | -5.38 | 0.02 | -14.03 | 0.59 | 32.24 | 0.36
Phase - CD | 0.03 | 0.55 | 0.03 | 0.93 | 0.07 | 0.57

Gaussian Noise | ID accuracy | 1.6 | 0.65 | 0.44 | 0.46 | 1.73 | 0.65
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.05 | -0.0 | 0.5 | -0.0 | 0.04
Pixel - within | -15.68 | 0.54 | -4.93 | 0.62 | -16.75 | 0.51
Pixel - between | 13.82 | 0.52 | 3.18 | 0.19 | 33.06 | 0.74
Amp - HFF | -20.7 | 0.27 | -15.19 | 0.98 | -50.62 | 0.14
Amp - CD | 0.04 | 0.73 | 0.03 | 0.9 | 0.07 | 0.93
Phase - HFF | -4.93 | 0.01 | -14.38 | 0.62 | 35.48 | 0.26
Phase - CD | 0.04 | 0.51 | 0.03 | 0.94 | 0.1 | 0.72

Impulse Noise | ID accuracy | 1.86 | 0.66 | 0.54 | 0.53 | 1.94 | 0.64
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.07 | -0.0 | 0.57 | -0.0 | 0.04
Pixel - within | -18.45 | 0.55 | -5.88 | 0.69 | -19.1 | 0.52
Pixel - between | 16.68 | 0.56 | 4.15 | 0.25 | 37.04 | 0.72
Amp - HFF | -25.88 | 0.31 | -17.02 | 0.96 | -58.31 | 0.14
Amp - CD | 0.05 | 0.76 | 0.03 | 0.94 | 0.08 | 0.9
Phase - HFF | -7.67 | 0.02 | -15.28 | 0.55 | 40.22 | 0.26
Phase - CD | 0.05 | 0.55 | 0.04 | 0.91 | 0.11 | 0.69

Table 3: Can model metrics explain why CLIP ensembling affects OOD robustness?

Our results provide a new perspective on the effective robustness problem by
showing that no one single metric rules them all. This dictates the need for a
multi-faceted analysis to understand OOD robustness, which is in sharp
contrast to the in-distribution setting where a single property such as model
flatness can reasonably explain generalization performance.

####  RobustNets benchmark.

To aid further research into understanding, modeling, and designing for OOD
robustness, we will publish a database of our pretrained models on CIFAR-10
(with diverse effective robustness) with the full range of model
architectures, data augmentations, and pruning types we study. The models we
study on ImageNet are already available publicly. We will also publish the ID
and OOD accuracy values for each model, and our code used to evaluate Jacobian
norm and high frequency fraction and consistent distance for Fourier amplitude
and phase interpolation, so that future researchers can apply it to study new
and more robust models as they become available.

## Acknowledgements

We would like to thank Jonathan Frankle, Mitchell Wortsman, and Ludwig Schmidt
for helpful discussion and pointers to resources. SFK acknowledges support
from NSF GRFP. This work was performed under the auspices of the U.S.
Department of Energy by Lawrence Livermore National Laboratory under Contract
DE-AC52-07NA27344 and was supported by the LLNL-LDRD Program under Project No.
2022-SI-004 (LLNL-CONF-835574).

## References

  * Andreassen et al. [2021] Anders Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs.  The evolution of out-of-distribution robustness throughout fine-tuning, 2021.  URL https://arxiv.org/abs/2106.15831.
  * Basri et al. [2019] Ronen Basri, David Jacobs, Yoni Kasten, and Shira Kritchman.  The convergence rate of neural networks for learned functions of different frequencies.  2019\.  doi: 10.48550/ARXIV.1906.00425.  URL https://arxiv.org/abs/1906.00425.
  * Bulusu et al. [2020] Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song.  Anomalous example detection in deep learning: A survey.  _IEEE Access_ , 8:132330–132347, 2020.
  * Croce et al. [2020] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein.  Robustbench: a standardized adversarial robustness benchmark, 2020.  URL https://arxiv.org/abs/2010.09670.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Diffenderfer and Kailkhura [2021] James Diffenderfer and Bhavya Kailkhura.  Multi-prize lottery ticket hypothesis: Finding accurate binary neural networks by pruning a randomly weighted network.  _International Conference on Learning Representations_ , 2021.
  * Diffenderfer et al. [2021] James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura.  A winning hand: Compressing deep networks can improve out-of-distribution robustness.  In _Advances in Neural Information Processing Systems_ , 2021.  URL https://proceedings.neurips.cc/paper/2021/file/0607f4c705595b911a4f3e7a127b44e0-Paper.pdf.
  * Frankle and Carbin [2018] Jonathan Frankle and Michael Carbin.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  _arXiv preprint arXiv:1803.03635_ , 2018.
  * Fridovich-Keil et al. [2021] Sara Fridovich-Keil, Raphael Gontijo Lopes, and Rebecca Roelofs.  Spectral bias in practice: The role of function frequency in generalization.  _CoRR_ , abs/2110.02424, 2021.  URL https://arxiv.org/abs/2110.02424.
  * Guillory et al. [2021] Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt.  Predicting with confidence on unseen distributions, 2021.  URL https://arxiv.org/abs/2107.03315.
  * Han et al. [2015] Song Han, Jeff Pool, John Tran, and William J Dally.  Learning both weights and connections for efficient neural networks.  _arXiv preprint arXiv:1506.02626_ , 2015.
  * He et al. [2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition, 2015.
  * Hendrycks and Dietterich [2019a] Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations, 2019a.  URL https://arxiv.org/abs/1903.12261.
  * Hendrycks and Dietterich [2019b] Dan Hendrycks and Thomas G. Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net, 2019b.  URL https://openreview.net/forum?id=HJz6tiCqYm.
  * Hendrycks et al. [2019] Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan.  Augmix: A simple data processing method to improve robustness and uncertainty.  In _International Conference on Learning Representations_ , 2019.
  * Hoffman et al. [2019] Judy Hoffman, Daniel A Roberts, and Sho Yaida.  Robust learning with jacobian regularization.  _arXiv preprint arXiv:1908.02729_ , 2019.
  * Hooker et al. [2019] Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome.  What do compressed deep neural networks forget?  _arXiv preprint arXiv:1911.05248_ , 2019.
  * Jiang et al. [2019] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio.  Fantastic generalization measures and where to find them.  _arXiv preprint arXiv:1912.02178_ , 2019.
  * Kireev et al. [2021] Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion.  On the effectiveness of adversarial training against common corruptions.  _arXiv preprint arXiv:2103.02325_ , 2021.
  * Krizhevsky [2009] Alex Krizhevsky.  Learning multiple layers of features from tiny images.  Technical report, 2009.
  * Marcel and Rodriguez [2010] Sébastien Marcel and Yann Rodriguez.  Torchvision the machine-vision package of torch.  In _Proceedings of the 18th ACM International Conference on Multimedia_ , MM ’10, page 1485–1488, New York, NY, USA, 2010. Association for Computing Machinery.  ISBN 9781605589336.  doi: 10.1145/1873951.1874254.  URL https://doi.org/10.1145/1873951.1874254.
  * Miller et al. [2021a] John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt.  Accuracy on the line: On the strong correlation between out-of-distribution and in-distribution generalization, 2021a.  URL https://arxiv.org/abs/2107.04649.
  * Miller et al. [2021b] John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt.  Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization.  In _International Conference on Machine Learning_ , pages 7721–7735. PMLR, 2021b.
  * Ortiz-Jimenez et al. [2020] Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard.  Hold me tight! influence of discriminative features on deep network boundaries, 2020.  URL https://arxiv.org/abs/2002.06349.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision, 2021.  URL https://arxiv.org/abs/2103.00020.
  * Rahaman et al. [2018] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville.  On the spectral bias of neural networks.  2018\.  doi: 10.48550/ARXIV.1806.08734.  URL https://arxiv.org/abs/1806.08734.
  * Ramanujan et al. [2020] Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari.  What’s hidden in a randomly weighted neural network?  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11893–11902, 2020.
  * Recht et al. [2018] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar.  Do cifar-10 classifiers generalize to cifar-10?, 2018.  URL https://arxiv.org/abs/1806.00451.
  * Recht et al. [2019] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar.  Do imagenet classifiers generalize to imagenet?, 2019.  URL https://arxiv.org/abs/1902.10811.
  * Renda et al. [2020] Alex Renda, Jonathan Frankle, and Michael Carbin.  Comparing rewinding and fine-tuning in neural network pruning, 2020.
  * Shankar et al. [2019] Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt.  Do image classifiers generalize across time?, 2019.  URL https://arxiv.org/abs/1906.02168.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv 1409.1556_ , 09 2014.
  * Sun et al. [2021] Jiachen Sun, Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, Dan Hendrycks, Jihun Hamm, and Z Morley Mao.  Certified adversarial defenses meet out-of-distribution corruptions: Benchmarking robustness and simple baselines.  _arXiv preprint arXiv:2112.00659_ , 2021.
  * Wortsman et al. [2021] Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt.  Robust fine-tuning of zero-shot models, 2021.  URL https://arxiv.org/abs/2109.01903.
  * Xu et al. [2018] Zhi-Qin John Xu, Yaoyu Zhang, and Yanyang Xiao.  Training behavior of deep neural network in frequency domain, 2018.  URL https://arxiv.org/abs/1807.01251.
  * Yin et al. [2019] Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, and Justin Gilmer.  A fourier perspective on model robustness in computer vision, 2019.  URL https://arxiv.org/abs/1906.08988.
  * Zhu and Gupta [2017] Michael Zhu and Suyog Gupta.  To prune, or not to prune: exploring the efficacy of pruning for model compression, 2017.

##  Appendix A Appendix

###  A.1 Pruning methods

In Section 4.2 we present results considering three categories of model
pruning used in Diffenderfer et al. [7]: _traditional_ (fine-tuning [11],
gradual magnitude pruning [37]), _rewinding lottery-tickets_ (weight-rewinding
[8], learning-rate rewinding [30]), and _initialization lottery-tickets_
(edgepopup [27], biprop [6]). Here we provide a brief description of each
pruning method.

#### Traditional.

As the name suggests, fine-tuning prunes a model at the end of the regular
training period by removing $p\%$ of the weights with the smallest magnitude,
then fine tunes the remaining weights using the learning rate at the end of
the regular training period. Gradual magnitude pruning progressively removes
weights during training in accordance with a sparsity scheduler (see Fig. 1 in
Zhu and Gupta [37]) until the target sparsity is reached. Typically, some
training takes place before and after pruning in gradual magnitude pruning.

#### Rewinding lottery tickets.

Rewinding lottery tickets perform repeated training-then-pruning steps in
which some percentage of the remaining weights are pruned at each pruning step
(traditionally 20% of the remaining weights are pruned at each step). After
each training-then-pruning step, either the weights and learning rate are
rewound to their values at initialization (i.e. weight-rewinding) or the
learning rate is rewound to a value earlier in the training process (learning-
rate rewinding). The first rewinding lottery ticket method supported the
Lottery Ticket Hypothesis [8] which suggests that sparse subnetworks exist
within randomly initialized dense networks that can be trained to the same
accuracy as dense networks.

#### Initialization lottery tickets.

Stronger versions of this hypothesis, such as the Strong [27] and Multi-Prize
Lottery Ticket Hypotheses [6] suggest that such subnetworks exist at
initialization that achieve comparable accuracy to the dense network without
requiring any training and, further, that these subnetworks’ weights can be
binarized. Edgepopup identifies such subnetworks using surrogate scores to
identify the most important weights at initialization. Biprop integrates
weight and/or activation binarization into the search process for such
subnetworks resulting in binary-weight or binary-weight and activation
networks (BNNs). Note that biprop networks in this paper only make use of
binarized weights.

###  A.2 Fourier interpolation on CIFAR-10

Figure 1 in the main paper shows example Fourier amplitude and phase
interpolating paths on ImageNet. Figure 6 shows example Fourier amplitude and
phase interpolating paths on CIFAR-10.

Figure 6: Example Fourier amplitude (top) and phase (bottom) interpolating
paths from CIFAR-10. Each path includes 100 images; every 15th image is
visualized here. The first image along each path is an unaltered image from
the original test set; the last image has the same Fourier phases (top) or
amplitudes (bottom) as the original but has some of the Fourier amplitudes
(top) or phases (bottom) of a random other image from the validation set.
Amplitude interpolation produces a corruption that preserves semantic content,
whereas phase interpolation destroys semantic content.

###  A.3 Jacobian norm computational details

We estimate the norm of the Jacobian using a random-projection-based approach
that estimates the Jacobian norm as a function of a batch of samples of size
$B$ and $n_{proj}$ projection vectors [16]. For CIFAR-10 and ImageNet, we set
$n_{proj}$ to $10$ and $20$, respectively; we use $B=400$ for both datasets.
Error bars for our estimates are 95% confidence intervals constructed under
the assumption that the $n_{proj}B$ unique Jacobian norm estimates made by the
projection method are i.i.d., which is consistent with the error order of
$(n_{proj}B)^{-\frac{1}{2}}$ expected by Hoffman et al. [16] when $B\gg 1$.
Under this assumption, which may lead to underestimation of the true error if
violated, our choice of $n_{proj}$ and $B$ ensures that the average error bar
is less than 2% of the size of the estimated Jacobian norm (and all error bars
are less than 8% of the size of the estimated Jacobian norm). Our estimated
Jacobian norms are between 60 and 500 for ImageNet models and between 2 and 50
for CIFAR-10 models.

###  A.4 Power spectral density of distribution shifts

The PSDs shown in Figure 2 reflect the distribution _shift_ between each OOD
dataset and the original CIFAR-10 test set. As each of the CIFAR-10-C
corruptions are transformations applied to the original CIFAR-10 test images,
the PSD for each CIFAR-10-C corruption is computed by taking the difference
between each corrupted and original test image, computing the PSD of the
resulting difference image, then averaging the PSDs for all images of the
given corruption. As CIFAR-10.1 images are not corruptions of the original
CIFAR-10 test images, the PSD characterization must be computed in a different
manner. First, for each of the ten CIFAR-10 classes we compute the average PSD
for CIFAR-10 test images and CIFAR-10.1 images and take the difference of
these class PSDs. The final approximated PSD representing CIFAR-10.1 shift is
taken to be the average of the difference PSDs for the ten classes.

In Figure 7, we show the PSDs for all perturbations to CIFAR-10 test data we
considered including the nine CIFAR-10-C corruptions not provided in the main
text (fog, snow, frost, elastic transform, zoom blur, motion blur, glass blur,
JPEG compression, and shot noise). As in the main text, we sort the
corruptions roughly in order of increasing frequency.

CIFAR-10.1

Brightness

Contrast

Fog

Frost

Snow

Elastic

Motion Blur

Zoom Blur

Glass Blur

Defocus Blur

Pixelate

JPEG

Gaussian Noise

Impulse Noise

Shot Noise

Figure 7: Power spectral densities for low (CIFAR-10.1, brightness, contrast,
fog, frost, snow), mid (elastic transform, motion blur, zoom blur, glass blur,
defocus blur, pixelate, JPEG compression), and high (Gaussian noise, impulse
noise, shot noise) frequency shifts with respect to CIFAR-10.

###  A.5 Pruning results on additional corruptions from CIFAR-10-C

Figure 8 and Table 4 extend Figure 3 and Table 1 to the nine remaining
corruptions from CIFAR-10-C.

Figure 8: When does pruning affect OOD robustness?

| Traditional | Rewinding LT | Initialization LT
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
Fog | ID accuracy | 0.96 | 0.83 | 0.63 | 0.66 | 0.98 | 0.89
Jacobian | -0.0 | 0.19 | -0.01 | 0.43 | 0.01 | 0.36
Pixel - within | -6.74 | 0.72 | -5.6 | 0.68 | -8.65 | 0.84
Pixel - between | 8.77 | 0.41 | 3.75 | 0.24 | 9.88 | 0.59
Amp - HFF | -6.91 | 0.68 | -4.54 | 0.38 | -6.1 | 0.84
Amp - CD | 0.05 | 0.67 | 0.03 | 0.53 | 0.05 | 0.78
Phase - HFF | -9.69 | 0.27 | -4.88 | 0.29 | -6.36 | 0.33
Phase - CD | 0.03 | 0.32 | 0.01 | 0.21 | 0.02 | 0.36

Snow | ID accuracy | 0.82 | 0.71 | 1.04 | 0.84 | 0.4 | 0.4
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.14 | -0.01 | 0.68 | 0.0 | 0.34
Pixel - within | -6.97 | 0.76 | -9.12 | 0.8 | -3.48 | 0.46
Pixel - between | 9.75 | 0.55 | 9.66 | 0.4 | 5.5 | 0.6
Amp - HFF | -4.94 | 0.44 | -8.36 | 0.66 | -2.46 | 0.4
Amp - CD | 0.03 | 0.35 | 0.05 | 0.76 | 0.02 | 0.26
Phase - HFF | -3.8 | 0.12 | -10.37 | 0.4 | 0.55 | 0.04
Phase - CD | 0.01 | 0.14 | 0.02 | 0.28 | -0.0 | 0.0

Frost | ID accuracy | 0.95 | 0.62 | 1.04 | 0.64 | 0.18 | 0.21
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.16 | -0.01 | 0.49 | 0.0 | 0.31
Pixel - within | -8.3 | 0.69 | -8.82 | 0.62 | -1.67 | 0.17
Pixel - between | 12.22 | 0.57 | 11.92 | 0.3 | 3.42 | 0.29
Amp - HFF | -5.81 | 0.4 | -8.44 | 0.47 | -0.92 | 0.26
Amp - CD | 0.03 | 0.31 | 0.05 | 0.61 | 0.01 | 0.23
Phase - HFF | -4.79 | 0.16 | -12.66 | 0.43 | 4.57 | 0.19
Phase - CD | 0.02 | 0.16 | 0.02 | 0.22 | -0.01 | 0.15

Elastic Transform | ID accuracy | 0.73 | 0.78 | 0.89 | 0.81 | 0.49 | 0.51
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.22 | -0.01 | 0.7 | 0.01 | 0.48
Pixel - within | -6.09 | 0.86 | -7.82 | 0.84 | -4.67 | 0.64
Pixel - between | 8.83 | 0.6 | 6.19 | 0.34 | 6.34 | 0.64
Amp - HFF | -5.19 | 0.6 | -6.83 | 0.51 | -3.05 | 0.52
Amp - CD | 0.03 | 0.43 | 0.04 | 0.69 | 0.03 | 0.35
Phase - HFF | -4.96 | 0.13 | -6.3 | 0.33 | -0.49 | 0.05
Phase - CD | 0.02 | 0.2 | 0.01 | 0.25 | 0.0 | 0.06

Zoom Blur | ID accuracy | 0.67 | 0.45 | 1.05 | 0.68 | 0.35 | 0.31
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.19 | -0.01 | 0.6 | 0.01 | 0.44
Pixel - within | -5.87 | 0.51 | -9.33 | 0.67 | -4.44 | 0.33
Pixel - between | 8.37 | 0.35 | 2.55 | 0.3 | 8.85 | 0.47
Amp - HFF | -4.58 | 0.34 | -7.43 | 0.39 | -2.48 | 0.38
Amp - CD | 0.03 | 0.3 | 0.05 | 0.54 | 0.02 | 0.32
Phase - HFF | -4.65 | 0.18 | -7.2 | 0.29 | 3.66 | 0.22
Phase - CD | 0.02 | 0.2 | 0.01 | 0.23 | -0.01 | 0.18

Motion Blur | ID accuracy | 0.62 | 0.54 | 1.13 | 0.72 | 0.43 | 0.31
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.18 | -0.02 | 0.69 | 0.01 | 0.52
Pixel - within | -4.93 | 0.54 | -10.02 | 0.69 | -4.59 | 0.36
Pixel - between | 6.87 | 0.33 | 6.94 | 0.36 | 7.72 | 0.5
Amp - HFF | -4.41 | 0.42 | -9.25 | 0.56 | -2.78 | 0.35
Amp - CD | 0.03 | 0.38 | 0.06 | 0.63 | 0.02 | 0.27
Phase - HFF | -5.86 | 0.18 | -10.9 | 0.41 | 1.7 | 0.09
Phase - CD | 0.02 | 0.23 | 0.02 | 0.28 | -0.0 | 0.05

Glass Blur | ID accuracy | 0.17 | 0.26 | 0.99 | 0.55 | -0.85 | 0.43
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.05 | -0.01 | 0.57 | 0.0 | 0.01
Pixel - within | -2.97 | 0.34 | -8.77 | 0.52 | 7.07 | 0.32
Pixel - between | 5.93 | 0.21 | 9.81 | 0.28 | -5.23 | 0.2
Amp - HFF | -0.31 | 0.29 | -8.75 | 0.54 | 5.26 | 0.44
Amp - CD | -0.0 | 0.36 | 0.05 | 0.56 | -0.05 | 0.49
Phase - HFF | 2.57 | 0.32 | -13.16 | 0.48 | 14.49 | 0.59
Phase - CD | -0.01 | 0.32 | 0.03 | 0.45 | -0.04 | 0.54

JPEG Compression | ID accuracy | 0.58 | 0.58 | 0.36 | 0.36 | -0.06 | 0.51
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.2 | -0.01 | 0.49 | 0.0 | 0.26
Pixel - within | -5.48 | 0.66 | -3.19 | 0.43 | 1.36 | 0.48
Pixel - between | 9.92 | 0.72 | -0.09 | 0.21 | -1.74 | 0.39
Amp - HFF | -4.26 | 0.48 | -4.26 | 0.37 | 0.53 | 0.52
Amp - CD | 0.02 | 0.24 | 0.02 | 0.38 | -0.0 | 0.48
Phase - HFF | -0.89 | 0.06 | -4.44 | 0.28 | 3.26 | 0.24
Phase - CD | 0.01 | 0.09 | 0.01 | 0.21 | -0.0 | 0.15

Shot Noise | ID accuracy | 0.67 | 0.22 | 0.34 | 0.11 | -0.77 | 0.46
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.08 | -0.0 | 0.19 | 0.0 | 0.06
Pixel - within | -6.76 | 0.29 | -3.82 | 0.19 | 7.06 | 0.37
Pixel - between | 7.69 | 0.16 | 9.73 | 0.16 | -6.9 | 0.3
Amp - HFF | -2.99 | 0.1 | -5.63 | 0.14 | 4.66 | 0.46
Amp - CD | 0.02 | 0.12 | 0.04 | 0.22 | -0.04 | 0.46
Phase - HFF | -1.88 | 0.16 | -11.59 | 0.3 | 12.66 | 0.45
Phase - CD | 0.01 | 0.09 | 0.03 | 0.2 | -0.03 | 0.37

Table 4: Can model metrics explain why pruning affects OOD robustness?

###  A.6 Augmentation results on additional corruptions from CIFAR-10-C

Figure 9 and Table 5 extend Figure 4 and Table 2 to the nine remaining
corruptions from CIFAR-10-C.

Figure 9: When does augmentation affect OOD robustness?

| Clean | AugMix | Gaussian
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
Fog | ID accuracy | 1.03 | 0.87 | 1.08 | 0.97 | 0.62 | 0.65
Jacobian | 0.01 | 0.13 | 0.01 | 0.39 | -0.0 | 0.06
Pixel - within | -9.85 | 0.8 | -8.17 | 0.91 | -5.75 | 0.72
Pixel - between | 12.02 | 0.56 | 10.58 | 0.77 | 3.78 | 0.22
Amp - HFF | -6.59 | 0.87 | -7.11 | 0.89 | -4.29 | 0.59
Amp - CD | 0.05 | 0.81 | 0.07 | 0.69 | 0.03 | 0.58
Phase - HFF | -9.64 | 0.42 | 4.88 | 0.08 | -0.32 | 0.04
Phase - CD | 0.03 | 0.51 | 0.02 | 0.17 | 0.03 | 0.34

Snow | ID accuracy | 0.59 | 0.5 | 0.86 | 0.92 | 0.84 | 0.8
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.06 | 0.01 | 0.31 | -0.0 | 0.08
Pixel - within | -6.52 | 0.58 | -6.68 | 0.88 | -7.24 | 0.79
Pixel - between | 7.08 | 0.29 | 8.48 | 0.71 | 6.52 | 0.43
Amp - HFF | -3.59 | 0.42 | -5.67 | 0.83 | -5.85 | 0.78
Amp - CD | 0.03 | 0.4 | 0.05 | 0.66 | 0.04 | 0.75
Phase - HFF | -2.4 | 0.06 | 4.59 | 0.08 | 4.88 | 0.12
Phase - CD | 0.01 | 0.08 | 0.02 | 0.18 | 0.03 | 0.31

Frost | ID accuracy | 0.39 | 0.2 | 0.82 | 0.88 | 0.94 | 0.85
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.14 | 0.01 | 0.25 | -0.0 | 0.06
Pixel - within | -5.24 | 0.31 | -6.38 | 0.85 | -7.64 | 0.76
Pixel - between | 4.29 | 0.13 | 7.97 | 0.66 | 7.48 | 0.48
Amp - HFF | -2.38 | 0.24 | -5.52 | 0.83 | -6.4 | 0.76
Amp - CD | 0.02 | 0.21 | 0.05 | 0.68 | 0.04 | 0.7
Phase - HFF | -0.47 | 0.08 | 4.42 | 0.07 | 6.13 | 0.15
Phase - CD | 0.0 | 0.08 | 0.02 | 0.16 | 0.03 | 0.26

Elastic Transform | ID accuracy | 0.59 | 0.66 | 0.73 | 0.96 | 0.77 | 0.83
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.06 | 0.01 | 0.34 | -0.0 | 0.08
Pixel - within | -6.11 | 0.77 | -5.61 | 0.94 | -6.82 | 0.86
Pixel - between | 7.01 | 0.45 | 7.08 | 0.74 | 6.55 | 0.49
Amp - HFF | -3.59 | 0.58 | -4.83 | 0.87 | -5.3 | 0.77
Amp - CD | 0.03 | 0.49 | 0.05 | 0.67 | 0.03 | 0.7
Phase - HFF | -2.83 | 0.09 | 3.28 | 0.08 | 5.16 | 0.12
Phase - CD | 0.01 | 0.16 | 0.01 | 0.19 | 0.03 | 0.29

Zoom Blur | ID accuracy | 0.48 | 0.28 | 0.95 | 0.96 | 0.69 | 0.57
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.06 | 0.01 | 0.37 | -0.0 | 0.12
Pixel - within | -5.71 | 0.39 | -7.3 | 0.93 | -6.79 | 0.69
Pixel - between | 6.6 | 0.24 | 9.38 | 0.76 | 5.5 | 0.3
Amp - HFF | -3.19 | 0.32 | -6.27 | 0.87 | -5.23 | 0.62
Amp - CD | 0.02 | 0.26 | 0.06 | 0.66 | 0.03 | 0.58
Phase - HFF | -0.9 | 0.05 | 4.92 | 0.09 | 4.11 | 0.08
Phase - CD | 0.01 | 0.06 | 0.02 | 0.17 | 0.03 | 0.26

Motion Blur | ID accuracy | 0.56 | 0.41 | 0.99 | 0.95 | 0.67 | 0.53
---|---|---|---|---|---|---|---
Jacobian | -0.0 | 0.04 | 0.01 | 0.37 | -0.0 | 0.11
Pixel - within | -6.25 | 0.5 | -7.62 | 0.93 | -6.65 | 0.67
Pixel - between | 6.62 | 0.25 | 9.89 | 0.77 | 5.21 | 0.27
Amp - HFF | -3.7 | 0.43 | -6.56 | 0.88 | -5.17 | 0.62
Amp - CD | 0.03 | 0.38 | 0.06 | 0.66 | 0.03 | 0.6
Phase - HFF | -2.75 | 0.09 | 5.9 | 0.1 | 2.9 | 0.06
Phase - CD | 0.01 | 0.12 | 0.02 | 0.16 | 0.03 | 0.31

Glass Blur | ID accuracy | -0.43 | 0.13 | 0.31 | 0.27 | 0.39 | 0.34
---|---|---|---|---|---|---|---
Jacobian | -0.01 | 0.23 | -0.0 | 0.14 | -0.01 | 0.22
Pixel - within | 1.83 | 0.08 | -2.82 | 0.31 | -4.99 | 0.47
Pixel - between | -5.7 | 0.13 | 3.29 | 0.26 | 1.75 | 0.24
Amp - HFF | 2.4 | 0.2 | -2.23 | 0.26 | -3.46 | 0.38
Amp - CD | -0.02 | 0.17 | 0.02 | 0.16 | 0.02 | 0.32
Phase - HFF | 8.08 | 0.28 | 4.34 | 0.08 | 0.08 | 0.09
Phase - CD | -0.02 | 0.26 | 0.0 | 0.05 | 0.02 | 0.19

JPEG Compression | ID accuracy | -0.03 | 0.27 | 0.28 | 0.35 | 0.75 | 0.85
---|---|---|---|---|---|---|---
Jacobian | -0.01 | 0.21 | -0.0 | 0.06 | -0.0 | 0.09
Pixel - within | -0.52 | 0.22 | -2.02 | 0.36 | -6.4 | 0.85
Pixel - between | -1.5 | 0.17 | 1.92 | 0.13 | 6.08 | 0.48
Amp - HFF | -0.04 | 0.33 | -2.07 | 0.4 | -5.09 | 0.74
Amp - CD | -0.0 | 0.28 | 0.03 | 0.53 | 0.03 | 0.68
Phase - HFF | 1.7 | 0.18 | -0.57 | 0.08 | 5.39 | 0.13
Phase - CD | -0.0 | 0.22 | 0.01 | 0.23 | 0.03 | 0.27

Shot Noise | ID accuracy | -0.73 | 0.2 | 0.26 | 0.18 | 0.9 | 0.96
---|---|---|---|---|---|---|---
Jacobian | -0.02 | 0.3 | -0.0 | 0.06 | -0.0 | 0.07
Pixel - within | 3.92 | 0.09 | -2.18 | 0.22 | -7.23 | 0.84
Pixel - between | -12.26 | 0.2 | 1.73 | 0.09 | 7.45 | 0.57
Amp - HFF | 3.66 | 0.25 | -2.3 | 0.21 | -5.87 | 0.79
Amp - CD | -0.03 | 0.23 | 0.03 | 0.3 | 0.04 | 0.71
Phase - HFF | 7.47 | 0.25 | 1.22 | 0.05 | 6.31 | 0.16
Phase - CD | -0.02 | 0.26 | 0.02 | 0.15 | 0.03 | 0.25

Table 5: Can model metrics explain why augmentation affects OOD robustness?

###  A.7 CLIP results on additional corruptions from ImageNet-C

Figure 10 and Table 6 extend Figure 5 and Table 3 to the nine remaining
corruptions from ImageNet-C.

Figure 10: When and why does CLIP ensembling affect OOD robustness?

| All | CLIP | Non-CLIP
---|---|---|---
$m$ | $R^{2}$ | $m$ | $R^{2}$ | $m$ | $R^{2}$
Fog | ID accuracy | 1.41 | 0.7 | 0.94 | 0.71 | 1.26 | 0.7
Jacobian | 0.0 | 0.48 | 0.0 | 0.66 | -0.0 | 0.0
Pixel - within | -14.49 | 0.57 | -9.87 | 0.66 | -12.4 | 0.57
Pixel - between | 13.29 | 0.76 | 9.59 | 0.39 | 24.15 | 0.8
Amp - HFF | -11.58 | 0.62 | -7.99 | 0.93 | -21.57 | 0.05
Amp - CD | 0.03 | 0.9 | 0.02 | 0.97 | 0.05 | 0.85
Phase - HFF | -14.72 | 0.43 | -11.73 | 0.91 | 25.81 | 0.28
Phase - CD | 0.03 | 0.75 | 0.02 | 0.95 | 0.07 | 0.62

Snow | ID accuracy | 1.66 | 0.72 | 1.44 | 0.75 | 1.21 | 0.64
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.57 | 0.0 | 0.67 | -0.0 | 0.01
Pixel - within | -17.05 | 0.58 | -14.89 | 0.67 | -11.69 | 0.5
Pixel - between | 15.22 | 0.75 | 15.5 | 0.45 | 24.06 | 0.78
Amp - HFF | -14.64 | 0.74 | -12.02 | 0.93 | -27.11 | 0.08
Amp - CD | 0.04 | 0.96 | 0.04 | 0.99 | 0.05 | 0.89
Phase - HFF | -19.15 | 0.54 | -17.5 | 0.9 | 26.64 | 0.3
Phase - CD | 0.03 | 0.86 | 0.03 | 0.98 | 0.07 | 0.63

Frost | ID accuracy | 1.38 | 0.77 | 1.09 | 0.8 | 1.23 | 0.71
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.49 | 0.0 | 0.64 | -0.0 | 0.01
Pixel - within | -14.3 | 0.63 | -11.29 | 0.72 | -12.17 | 0.58
Pixel - between | 12.2 | 0.74 | 12.16 | 0.51 | 23.58 | 0.81
Amp - HFF | -11.0 | 0.64 | -8.8 | 0.93 | -24.59 | 0.07
Amp - CD | 0.03 | 0.92 | 0.03 | 0.98 | 0.05 | 0.9
Phase - HFF | -13.7 | 0.43 | -12.47 | 0.85 | 26.54 | 0.32
Phase - CD | 0.03 | 0.78 | 0.02 | 0.94 | 0.07 | 0.66

Elastic Transform | ID accuracy | 0.97 | 0.76 | 0.82 | 0.92 | 0.75 | 0.59
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.41 | 0.0 | 0.42 | -0.0 | 0.01
Pixel - within | -9.99 | 0.62 | -8.56 | 0.84 | -7.31 | 0.46
Pixel - between | 9.05 | 0.82 | 10.21 | 0.74 | 15.5 | 0.77
Amp - HFF | -7.5 | 0.6 | -5.52 | 0.75 | -13.64 | 0.05
Amp - CD | 0.02 | 0.85 | 0.02 | 0.81 | 0.03 | 0.74
Phase - HFF | -8.65 | 0.34 | -7.06 | 0.55 | 21.1 | 0.44
Phase - CD | 0.02 | 0.68 | 0.01 | 0.68 | 0.04 | 0.46

Zoom Blur | ID accuracy | 1.08 | 0.79 | 0.99 | 0.68 | 0.99 | 0.83
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.58 | 0.0 | 0.78 | -0.0 | 0.01
Pixel - within | -11.47 | 0.69 | -10.02 | 0.58 | -10.43 | 0.76
Pixel - between | 8.33 | 0.58 | 10.61 | 0.4 | 16.94 | 0.74
Amp - HFF | -8.78 | 0.69 | -8.88 | 0.98 | -3.79 | 0.0
Amp - CD | 0.02 | 0.85 | 0.02 | 0.94 | 0.03 | 0.75
Phase - HFF | -11.12 | 0.47 | -12.57 | 0.89 | 23.37 | 0.44
Phase - CD | 0.02 | 0.78 | 0.02 | 0.95 | 0.04 | 0.52

Motion Blur | ID accuracy | 1.37 | 0.74 | 1.03 | 0.76 | 1.15 | 0.72
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.53 | 0.0 | 0.7 | -0.0 | 0.01
Pixel - within | -14.19 | 0.6 | -10.58 | 0.67 | -11.66 | 0.62
Pixel - between | 12.65 | 0.77 | 11.25 | 0.47 | 21.15 | 0.76
Amp - HFF | -11.45 | 0.67 | -8.67 | 0.96 | -13.66 | 0.03
Amp - CD | 0.03 | 0.91 | 0.02 | 0.96 | 0.04 | 0.79
Phase - HFF | -14.18 | 0.44 | -12.2 | 0.86 | 28.92 | 0.44
Phase - CD | 0.03 | 0.78 | 0.02 | 0.94 | 0.06 | 0.52

Glass Blur | ID accuracy | 1.25 | 0.67 | 0.93 | 0.9 | 0.85 | 0.55
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.45 | 0.0 | 0.48 | -0.0 | 0.03
Pixel - within | -12.62 | 0.52 | -9.56 | 0.79 | -8.15 | 0.43
Pixel - between | 13.03 | 0.88 | 11.74 | 0.74 | 17.35 | 0.72
Amp - HFF | -10.69 | 0.63 | -6.52 | 0.79 | -22.87 | 0.1
Amp - CD | 0.03 | 0.88 | 0.02 | 0.83 | 0.03 | 0.82
Phase - HFF | -12.77 | 0.39 | -8.27 | 0.57 | 21.5 | 0.34
Phase - CD | 0.02 | 0.72 | 0.01 | 0.72 | 0.05 | 0.55

JPEG Compression | ID accuracy | 1.16 | 0.85 | 0.92 | 0.89 | 1.23 | 0.81
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.36 | 0.0 | 0.5 | -0.0 | 0.02
Pixel - within | -12.29 | 0.73 | -9.5 | 0.8 | -12.42 | 0.69
Pixel - between | 9.42 | 0.69 | 11.19 | 0.68 | 21.94 | 0.8
Amp - HFF | -7.82 | 0.51 | -6.59 | 0.82 | -9.03 | 0.01
Amp - CD | 0.02 | 0.78 | 0.02 | 0.86 | 0.04 | 0.77
Phase - HFF | -8.7 | 0.27 | -8.57 | 0.63 | 29.26 | 0.44
Phase - CD | 0.02 | 0.61 | 0.01 | 0.75 | 0.06 | 0.54

Shot Noise | ID accuracy | 1.52 | 0.65 | 0.83 | 0.67 | 1.71 | 0.64
---|---|---|---|---|---|---|---
Jacobian | 0.0 | 0.35 | 0.0 | 0.7 | -0.0 | 0.05
Pixel - within | -15.38 | 0.51 | -8.45 | 0.58 | -16.57 | 0.5
Pixel - between | 13.81 | 0.66 | 9.22 | 0.43 | 32.77 | 0.73
Amp - HFF | -11.21 | 0.47 | -7.28 | 0.92 | -51.38 | 0.14
Amp - CD | 0.03 | 0.78 | 0.02 | 0.84 | 0.07 | 0.93
Phase - HFF | -12.72 | 0.26 | -9.81 | 0.76 | 35.09 | 0.26
Phase - CD | 0.03 | 0.59 | 0.02 | 0.82 | 0.1 | 0.73

Table 6: Can model metrics explain why CLIP ensembling affects OOD robustness?