# United We Stand: Using Epoch-wise Agreement of Ensembles to Combat Overfit

Uri Stern, Daniel Shwartz, Daphna Weinshall

###### Abstract

Deep neural networks have become the method of choice for solving many image
classification tasks, largely because they can fit very complex functions
defined over raw images. The downside of such powerful learners is the danger
of overfitting the training set, leading to poor generalization, which is
usually avoided by regularization and ”early stopping” of the training. In
this paper, we propose a new deep network ensemble classifier that is very
effective against overfit. We begin with the theoretical analysis of a
regression model, whose predictions – that the variance among classifiers
increases when overfit occurs – is demonstrated empirically in deep networks
in common use. Guided by these results, we construct a new ensemble-based
prediction method designed to combat overfit, where the prediction is
determined by the most consensual prediction throughout the training. On
multiple image and text classification datasets, we show that when regular
ensembles suffer from overfit, our method eliminates the harmful reduction in
generalization due to overfit, and often even surpasses the performance
obtained by early stopping. Our method is easy to implement, and can be
integrated with any training scheme and architecture, without additional prior
knowledge beyond the training set. Accordingly, it is a practical and useful
tool to overcome overfit.

##  1 Introduction

Deep supervised learning has achieved exceptional results in various image
recognition tasks in recent years. This impressive success is largely
attributed to two factors – ready access to very large annotated datasets, and
powerful architectures involving a huge number of parameters, thus capable of
learning very complex functions from large datasets. However, when training
very large models, we face the risk of _overfit_ – when models are fine-tuned
to irrelevant details in the training set. It is defined here as the co-
occurrence of _increase_ in training accuracy and _decrease_ in test accuracy,
which implies poorer generalization as training proceeds.

One important cause of overfit in deep learning is the existence of false
labels in the dataset (also termed ”noisy labels”), which the network
nevertheless is able to memorize. Empirically, such noisy labels are known to
be memorized late by deep networks (Arpit et al. 2017), resulting in overfit -
errors that occur later in training. As deep learning requires large annotated
datasets, the problem of false labels often arises in real applications,
medical data being an important example (Karimi et al. 2020), where the
reliable labeling of data is expensive. Moreover, cheap alternatives such as
crowd sourcing or automatic labeling typically introduce noisy labels to the
dataset (Nicholson, Sheng, and Zhang 2016). It is thus important to develop
additional tools for combating overfit, especially heavy overfit caused by
noisy labels, which is our goal in this paper.

Not surprisingly, given the gravity of this issue, a variety of measures have
been developed through the years to combat overfit. This previous work can be
largely divided into two groups, which are not mutually exclusive. The first
group, which includes the majority of works, focuses on attempting to modify
_the learning process_ in order to prevent or reduce overfit. These include
various forms of data augmentation (Shorten and Khoshgoftaar 2019),
regularization techniques such as dropout (Srivastava et al. 2014), weight
decay (Krogh and Hertz 1991), batch normalization (Ioffe and Szegedy 2015) and
”early stopping” - terminating the training before overfit occurs.

Another line of relevant work in the first group falls under the category of
dealing with label noise. This may be accomplished by various means,
including:  (i) Changing the loss function (Ghosh, Kumar, and Sastry 2017;
Zhang and Sabuncu 2018; Wang et al. 2019; Xu et al. 2019), estimating the
noise transition matrix (Goldberger and Ben-Reuven 2017; Patrini et al. 2017)
or employing further regularization (Tanno et al. 2019; Jenni and Favaro 2018;
Bai et al. 2021).  (ii) Filtering out noisy labels and retraining (Pleiss et
al. 2020; Han et al. 2018; Arazo et al. 2019; Chen et al. 2019; Jiang et al.
2018).  (iii) Augmenting supervised learning with unsupervised or semi-
supervised methods (Li, Socher, and Hoi 2020; Liu et al. 2020; Zheltonozhskii
et al. 2022; Yao et al. 2021; Karim et al. 2022).

The second group, to which our method belongs, comes into play after learning
is concluded. Such methods involve the design of post-processing algorithms to
be invoked during _inference time_ (e.g., (Lee et al. 2019)). Relevant work,
when dealing with label noise, includes (Zhang, Lee, and Agarwal 2021; Bae et
al. 2022). An ensemble classifier is used in (Salman and Liu 2019) to combat
overfit due to insufficient training data, using network confidence to filter
out erroneous predictions. Our method differs primarily in that it provides
labels for all points, implicitly correcting false labels of regular
ensembles, which is achieved by relying on a novel analysis of the ensemble’s
predictions.

One benefit of the second approach is that it doesn’t require additional time
for training, e.g., by way of re-training or data-specific hyper-parameter
tuning, which is a major drawback of the first approach. Additionally, methods
from the second group can be incorporated with any training scheme, which is
useful when methods from the first group fail to completely eliminate overfit.

(a) Illustrative example with 3 models

(b) Final predictions

Figure 1: Illustrative example, showing the hypothetical behavior of an
ensemble of size 3. (a) Visualization of each model’s prediction over time,
given as input a car image falsely labeled as banana. (b) The final prediction
of the 3 models (top row), regular ensemble (bottom left), and our method
(bottom right).

Our proposed method belongs to the family of deep ensemble classifiers.
Engaging a set of classifiers, rather than a single one, is an established
methodology to deal with errors (Hansen and Salamon 1990). Ensembles of deep
neural networks are typically aggregated by simple yet useful methods. The two
most widely used aggregation methods are the majority vote or class
probabilities average (Lakshminarayanan, Pritzel, and Blundell 2017) of the
ensemble.

This paradigm is effective when the members of the ensemble generate different
predictions, namely, there is diversity (or variability) among the
classifiers. To generate such variability, members of the ensemble may be
trained with different training sets, as in the popular methods of bagging
(Breiman 1996) and boosting (Freund and Schapire 1997). In the context of deep
learning, ensemble methods attempt to increase the diversity between the
networks in a variety of ways, as can be seen in recent reviews (Li, Wang, and
Ding 2018; Ganaie, Hu et al. 2021). In (Wenzel et al. 2020), for example, a
pool of models with different weights initialization is trained with different
hyper-parameters for each weight, from which the best models for ensemble are
chosen. As ensembles are expensive, some papers focus on minimizing their cost
rather than maximizing their performance, see for example (Wen, Tran, and Ba
2020).

### Our contribution

Differently from most methods, our new ensemble classifier does not only
consider the final predictions of the networks in the ensemble, but also
tracks the networks’ responses as learning proceeds. In this context we recall
recent empirical findings (Hacohen, Choshen, and Weinshall 2020; Pliushch et
al. 2021), which show that when the predictions of networks are tracked
through time, almost all the networks will either predict correctly the
correct label, or almost all will fail. This is true in all the epochs, for
both the training and the test sets, irrespective of architecture or learning
parameters. These findings imply that deep networks demonstrate high agreement
per epoch in their predictions of correct labels. But while all the networks
are shown to succeed together, they may still fail in different ways, as
visualized in Fig. 1. Here we pursue the hypothesis that such failures can be
exploited to distinguish between false predictions and correct ones, most
effectively so when overfit occurs.

Our work begins by investigating the hypothesis mentioned above, where we
study networks’ agreement over false prediction both theoretically and
empirically. We start with the theoretical analysis of a regression model
(Section 3.1), which reinforces the intuition that _overfit should increase
variance in prediction between linear models_. Accordingly, we make two
conjectures:  (i) ensembles become more effective when overfit occurs;  (ii)
the agreement between networks is reduced when overfit occurs . These
conjectures are verified empirically in Section 3.2, while evaluating deep
networks in common use. We see that ensemble classifiers indeed improve
performance when overfit occurs. However, overfit is not eliminated.
Conjecture (ii) is also shown to be valid, and we report a _new empirical
result_ : the variance of correct predictions during inference is much smaller
than the variance of false predictions.

The new empirical result implies that correct predictions at inference time
can be distinguished from false ones by looking at the _variance in
predictions between multiple networks throughout the training epochs_. This is
used to construct a new algorithm for ensemble-based prediction, which is much
more resistant to overfit (see Section 4.1). This method is then tested on
different image classification and text classification datasets with label
noise where the networks manifest significant overfit (see Section 4.2). In
these experiments we see two positive effects of our method: overfit is
eliminated in almost all the experiments (except in extreme cases of label
noise), showing that the method is _robust against overfit_ ; and the
performance is better than that of the ensemble, even in its best epoch
(obtained by early stopping). Our method also shows a clear significant
advantage compared to other methods from the second group described above, and
performs well when training is done using the ensemble training methods
described earlier.

To conclude, our main contribution is a new method for ensemble-based
prediction, which is resistant to overfit, and which does not require any
prior knowledge on the data or changes to the training protocol. It is
applicable with any network and dataset, easy to implement, and is effective
_as a practical tool against overfit_. Most importantly, when the models are
still able to learn useful patterns from the data after the occurrence of
overfit (e.g., when the test error shows ”double descent”, see (Stephenson and
Lee 2021; Nakkiran et al. 2021)), our methods allows the user to use this
forgotten knowledge while reducing overfit, ourperforming the popular ”early
stopping” method.

Moreover, our network can be easily incorporated with any method from the
first group, or with other ensemble methods: in our experiments we use some of
these methods in the training, in which case significant _additional_
improvement in the final performance is shown.

##  2 Methodology

#### Notations

Let $N$ denote the number of networks in an ensemble of deep neural networks
trained independently on a training dataset with $C$ classes for $E$ epochs
using Stochastic Gradient Descent (SGD), and tested after each epoch of
training on test set $\mathbb{T}$. Let $f_{i}^{e}(x)$ denote the prediction of
network $i$ for ${\mathbf{x}}\in\mathbb{T}$ in epoch $e$. $N^{e}(x)$ denotes
the ensemble prediction in each epoch $e$, defined as:

$\displaystyle N^{e}({\mathbf{x}})=\operatorname*{argmax}_{c\in
C}\sum_{i=1}^{N}\mathds{1}_{[f_{i}^{e}({\mathbf{x}})=c]}$ (1)

#### Label noise generates overfit

As deep models are able to memorize any data distribution, false labels in the
training set (also termed ”noisy labels”) lead networks to learn irrelevant
features while memorizing them, which may lead to a large decrease in their
test accuracy. As noisy labels are memorized towards the end of training, this
means that training with noisy labels is usually accompanied by overfit. We
thus expose our models to data with label noise in order to evaluate
robustness to heavy overfit. We evaluate our method on:  (i) datasets with
synthetic label noise,  (ii) real-world datasets with label noise crafted from
the internet by web-labeling (Clothing1M, Webvision), and  (iii) datasets with
inherent label noise since they by design confusing to annotators (Animal10N).

#### Injecting label noise

To generate data with synthetic label noise, we use two standard noise models
(Patrini et al. 2017):

  1. 1.

Symmetric noise: a fraction $p\in\\{0.2,0.4,0.6\\}$ of the labels is selected
randomly. Each selected label is switched to any other label with equal
probability.

  2. 2.

Asymmetric noise: a fraction $p$ of the labels is selected randomly. Each
selected label is switched to another label using a deterministic permutation
function.

#### Inter-Model Agreement

In order to measure agreement between different models during inference time,
we define below in (2) an Agreement score. This score measures the average
fraction of networks in the ensemble that predict class $c$ at point
${\mathbf{x}}$ in a set of training epochs $\mathcal{E}$.

$\displaystyle
Agr({\mathbf{x}},c)=\frac{1}{N|\mathcal{E}|}\sum_{e\in\mathcal{E}}\sum_{i=1}^{N}\mathds{1}_{[f_{i}^{e}({\mathbf{x}})=c]}$
(2)

##  3 Overfit and Ensemble Classifiers

In this section we study, theoretically and empirically, ensembles’ dynamics
when overfit actually occurs (for some motivation, we refer the reader to a
motivating $2D$ example in Appendix A). We begin with a theoretical analysis
of a regression model of linear classifiers (Section 3.1), showing that the
agreement between such classifiers decreases when overfit occurs. In Section
3.2 we empirically study the relevance of this result to deep learning,
showing that the same phenomenon occurs also with deep neural networks. We
conclude in Section 3.3 with the analysis of the onset time of correct and
false predictions. This empirical analysis shows that false predictions caused
by overfit are associated with lower agreement score as compared to the
correct predictions, implying that false predictions are less common
throughout the training than the corresponding correct predictions. Later in
Section 4.1, these results guide the construction of a new ensemble-based
prediction algorithm, which is resistant to overfit.

###  3.1 Overfit in linear models: theoretical result

Since deep learning models are difficult to analyze theoretically, common
practice invokes simple models (such as linear regression) that can be
theoretically analyzed, whose analysis may shed light on the behaviour of
actual deep models. Accordingly, it is shown in (Sollich and Krogh 1995) that
using a large ensemble of linear networks, trained on subsets of the training
data, can improve performance. We take a different approach, theoretically
analyzing an ensemble of linear regression models trained using gradient
descent via the perspective of their agreement.

Our analysis culminates in a theorem, which states that the agreement between
linear regression models decreases when overfit occurs in all the models,
namely, when the generalization error in all the models increases.

######  Theorem.

Assume that all models see the same training set, and all suffer from overfit
at the same time. Then, under reasonable assumptions, the disagreement between
the models increases as overfit occurs.

Here is a brief sketch of the proof (see App. B):

  1. 1.

We measure _Disagreement_ by the empirical variance over models of the error
vector at each test point, averaged over the test examples.

  2. 2.

We prove the following (intuitive) Lemma 1: _Overfit occurs in a model iff the
gradient step of the model, which is computed from the training set, is
negatively correlated with a vector unknown to the learner - the gradient step
defined by the test set._

  3. 3.

We show that under certain asymptotic assumptions, the disagreement is
approximately minus the sum of the correlation between each network’s gradient
step and its ”test gradient step”. It now readily follows from Lemma 1 that if
overfit occurs in all the models then the aforementioned disagreement score
increases.

###  3.2 Overfit in deep networks: empirical result

We discuss in the introduction known findings concerning ensembles of deep
models, which imply that all networks are likely to succeed or fail together
in their prediction. However, it is still possible that they fail in different
ways, where each network predicts a different false label. With overfit in
particular, they may favor different false labels at different epochs (see
illustration in Fig. 1).

We therefore start our empirical investigation by analyzing the variance of
false predictions (or errors) in such ensembles. We use datasets injected with
label noise in this study, in order to amplify the prevalence of overfit. The
diversity in the ensembles has 2 sources: (i) different random initialization,
and (ii) different random mini-batches within each epoch, where all networks
train on the full datasets.

More specifically, we begin by training an ensemble of $N$ DNNs on noisy
labeled datasets, and then isolate the test points with erroneous prediction
in the last epoch. Using this set of points, we compute the _Error Consensus
Score (ECS)_ , which measures the number of networks that agree on each
erroneous prediction (from 1 to $N$). In Fig. 2 we plot some empirical
histograms of _ECS_.

(a) cifar10 40% SN

(b) cifar10 20% AsN

(c) cifar100 60% SN

(d) cifar100 10% AsN

Figure 2: Histograms of _ECS_ \- the number of networks that agree on each
erroneous prediction, for an ensemble of 5 networks. ’SN’ denotes symmetric
noise, and ’AsN’ asymetric noise.

The results in Fig. 2 indicate that when erroneous predictions are concerned,
their distribution seems to have a large variance. In fact, in many of our
study cases, the minority value ($\emph{ECS}<\frac{N}{2}$) dominates the
distribution. We also see that in more difficult settings (e.g., cifar100),
more mistakes are done by the majority, making the ensemble unable to correct
them. We therefore conjecture that ensemble classifiers will be effective in
correcting erroneous predictions in cases of overfit. In addition, we
conjecture that with minor overfit, ensembles will only make a little
difference.

(a) cifar10, 60% symmetric noise

(b) cifar100, 40% sym noise

Figure 3: Test accuracy over the epochs of a single network (green) and
ensemble (red) trained on a dataset with label noise. For comparison, we also
show the test accuracy of a single network (blue) and ensemble (orange)
trained on the clean data only..

These conjectures are supported by our empirical results, in experiments
involving various datasets and noise settings, as shown in Fig. 3 and Table 1.
In particular, Fig. 3 clearly shows that the benefit of the ensemble
classifier is much larger when there is significant overfit, which is seen
when the training data has significant label noise. However, Fig. 3 also shows
that while ensembles improve accuracy when there is overfit, they do not
eliminate the phenomenon \- there is still performance deterioration as
training proceeds. The ”rebound” in test accuracy of the single network is due
to the ”epoch-wise double descent” phenomenon, see details in (Stephenson and
Lee 2021; Nakkiran et al. 2021).

(a) cifar10, 60% symmetric noise

(b) cifar100, 20% asymmetric noise

(c) TinyImg, 40% symmetric noise

(d) Imagenet100, 40% symm. noise

Figure 4: Histograms of the margin score (3), where blue indicates correct
predictions, and orange erroneous predictions.

###  3.3 Remember your history and avoid overfit

Ensembles are most effective when the predictions of the different members of
the ensemble show large variance, as discussed above. In this section we
inspect another potential source of variability - a prediction’s persistence.
Following our theoretical and empirical analysis and the agreement score
definition in (2), we hypothesize that since erroneous predictions on the test
examples have large variance, correct predictions will have larger persistence
(agreement) than erroneous ones, amplifying the difference between the two.

In order to test this conjecture, we use the agreement score (2) defined in
Section 2. More specifically, we compare the agreement score of the ensemble’s
final prediction in epoch $E$ $N^{E}(x)$ with the agreement score of the most
agreed upon label, which is different from the final prediction:

$\begin{split}AgrMargin({\mathbf{x}},y)&=Agr({\mathbf{x}},y)-\max_{c\neq
y}Agr({\mathbf{x}},c)\cr y&=N^{E}({\mathbf{x}})\end{split}$ (3)

Fig. 4 shows histograms of score (3), separated to correct and erroneous
predictions. The phenomenon is rather general, shown in a variety of
conditions in which overfit is enhanced by injecting label noise. Clearly,
most of the predictions with negative margin score are false, while the vast
majority of the correct predictions have a positive score.

##  4 Proposed method (MAP)

We present a new ensemble classifier algorithm called Max Agreement Prediction
(MAP) in Section 4.1. Extensive experiments, demonstrating its superior
performance, are described in Section 4.3.

###  4.1 Algorithm

Our empirical results, as seen in Fig. 4, show that the agreement margin score
in (3) can be reliably used to identify false predictions. We take this result
one step further, and propose to use the ensemble statistics of agreement
score in order to select the label prediction, instead of the usual practice
of selecting $N^{E}({\mathbf{x}})$ from (1). Specifically, we propose the
following prediction selection rule:

$y({\mathbf{x}})=\operatorname*{argmax}_{c}Agr({\mathbf{x}},c)$ (4)

We use this score in Alg. 1 and test it in Section 4.2, where its boost in
performance is shown.

Algorithm 1 max Agreement Prediction (MAP)

Input:$f_{i}^{e}(x)$ \- the prediction of network $i$ for test example
${\mathbf{x}}\in\mathbb{T}$, $~{}~{}\forall{\mathbf{x}},i,e\in\mathcal{E}$

Output: final prediction $\forall{\mathbf{x}}\in\mathbb{T}$

for ${\mathbf{x}}$ in $\mathbb{T}$ do

final_prediction_arr[x]
$\leftarrow\operatorname*{argmax}\limits_{c}Agr({\mathbf{x}},c)$

end for

Return final_prediction_arr;

###  4.2 Empirical setup

To evaluate our method with different levels of overfit we use image and nlp
classification datasets with injected noisy labels (using cifar10/100,
TinyImagenet, Imagenet100, MNLI, QNLI, QQP) and with native label noise
(Webvision50, Clothing1M, Animal10N). We defer full discussion of the
implementation details to App. E.

We compare our method’s performance (with 5 networks) to the obvious
baselines:  (i) A single network of the same architecture and similarly
trained.  (ii) The same ensemble while using the common prediction rule
$N^{E}({\mathbf{x}})$ of ‘Majority Vote’ as defined in (1).  (iii) The same
ensemble while using the popular ‘Class probabilities average’ rule defined
e.g. in (Lakshminarayanan, Pritzel, and Blundell 2017).  The last 2
classifiers are versions of the ‘regular ensemble’, which incurs the same
training cost as our method. Note that the ‘regular ensemble’ is chosen as it
is a fair comparison in terms of _training time_. Most results are reported in
Table 1, and the rest appear in Table 6 in Appendix C. A comparison with
comparable _inference time_ , using a much larger ensemble that is _almost
infeasible to train_ , is reported in Table 5.

In addition to final performance, we check the different methods for overfit
by looking for deterioration in performance as learning proceeds. All networks
are trained with common methods to avoid overfit - data augmentation, batch
normalization and weight decay. This way we are able to test our method’s
_added value_ as compared to these methods.

Additionally, we compare our method to alternative post-processing methods,
which also aim to improve classification at inference time. To assure a fair
comparison, all the results are obtained using the same network models in the
ensemble, which is most relevant to the methods that make use of an ensemble
as we do (Salman and Liu 2019; Lee et al. 2019). The method describe in (Bae
et al. 2022) is executed several times with random seeds, the results of which
are processed by the majority vote as in a regular ensemble. To the best of
our knowledge, no other method uses the training history of an ensemble to
improve classification at inference time, to be used as a baseline.

Lastly, we test our method on two distinct methods for training ensembles:
batch ensemble (Wen, Tran, and Ba 2020) and hyper ensemble (Wenzel et al.
2020). The first method aims to reduce costs, whereas the second aims to
increase the ensemble’s diversity by using different hyper-parameters as well
as different initialization.

###  4.3 Empirical results

Fig. 5 and Table 1 summarize the test accuracy results. When the performance
drop becomes more severe (due to increased label noise), our method
significantly outperforms the regular ensemble (both majority vote and class
probabilities average) at the end of the training. It often even outperforms
early stopping (i.e. the ensemble before the overfit, see Fig. 5). Lastly, our
results in Table 3 show that our method is complementary to the two
specialized ensemble training method that aim to increase diversity or
decrease their cost.

Importantly, note that MAP eliminates the overfit in almost all cases: when
inspecting the case studies shown in Fig. 5, we clearly see that the test
accuracy does not deteriorate when MAP is used, unlike a single network and
the regular ensemble, making it a superior alternative to an early stopping of
the training. Only in severe, unrealistic settings of label noise (such as 40%
asymmetric noise) do we see a deterioration in our method’s performance in the
late stages of the training (though MAP still outperforms the regular
ensemble), but such cases are not common.

(a) cifar10, 40% symmetric noise

(b) cifar100, 20% asymmetric noise

(c) cifar100, 20% symmetric noise

(d) TinyImg, 20% symmetric noise

Figure 5: Mean test accuracy of our method (MAP, in red) in each training
epoch (X-axis), compared with the following baselines:  (i) A single network
trained on the noisy dataset (orange).  (ii) An ensemble trained on the noisy
dataset (green).  (iii) An oracle of a _single network_ trained on the clean
subset of the data (dotted blue) . Our method shows significant _and stable_
improvement, and largely eliminates the overfit caused by the noisy labels. It
even outperforms the single network oracle in some cases. Note that even
though in some cases the test accuracy of the network/ensemble improves at the
late stages of the training, it is often inferior compared to its past
performance, an is _always_ inferior to our method’s performance.

Method/Dataset | CIFAR-10 sym | CIFAR-100 sym | Clothing1M
---|---|---|---
% label noise | 20% | 40% | 60% | 20% | 40% | 60% | 38% (est)
_single network_ | $85.4\pm.1$ | $67.7\pm.6$ | $43.5\pm.5$ | $67.0\pm.2$ | $51.1\pm.2$ | $32.2\pm.4$ | $65.1\pm.1$
_majority vote ensemble_ | $90.1\pm.2$ | $78.5\pm.7$ | $54.0\pm.4$ | $73.5\pm.2$ | $61.2\pm.4$ | $42.8\pm.2$ | $66.0\pm.2$
_probability average ensemble_ | 90.5 | 79.5 | 56.6 | 74.9 | 64.2 | 46.3 | 67.1
_MAP_ | $\mathbf{93.2\pm.1}$ | $\mathbf{90.0\pm.1}$ | $\mathbf{83.8\pm.7}$ | $\mathbf{76.7\pm.1}$ | $\mathbf{69.7\pm.5}$ | $\mathbf{60.0\pm.4}$ | $\mathbf{71.7\pm.1}$

Method/Dataset | TinyImagenet sym | Imagenet100 sym
---|---|---
% label noise | 20% | 40% | 20% | 40%
_single network_ | $54.5\pm.7$ | $40.3\pm.2$ | $76.4\pm.5$ | $64.9\pm.1$
_majority vote ensemble_ | $62.0\pm.1$ | $50.3\pm.4$ | $82.8\pm.2$ | $75.5\pm.5$
_probability average ensemble_ | 64.0 | 53.2 | 83.9 | 77.9
_MAP_ | $\mathbf{66.0\pm.1}$ | $\mathbf{60.1\pm.1}$ | $\mathbf{83.9\pm.1}$ | $\mathbf{80.9\pm.3}$

Method/Dataset | CIFAR-10 asym | CIFAR-100 asym | Animal10N
---|---|---|---
% label noise | 10% | 20% | 40% | 10% | 20% | 40% | 8% (est)
_single network_ | $90.8\pm.1$ | $83.1\pm.2$ | $59.7\pm.3$ | $74.1\pm.2$ | $67.5\pm.2$ | $47.5\pm.1$ | 86.0
_majority vote ensemble_ | $93.5\pm.1$ | $88.4\pm.2$ | $64.0\pm.4$ | $79.0\pm.1$ | $73.7\pm.1$ | $53.4\pm.2$ | $87.3$
_probability average ensemble_ | 93.4 | 88.6 | 63.3 | 79.8 | 74.5 | 52.9 | 87.9
_MAP_ | $\mathbf{95.2\pm.1}$ | $\mathbf{94.4\pm.2}$ | $\mathbf{85.6\pm.2}$ | $\mathbf{80.6\pm.08}$ | $\mathbf{77.7\pm.1}$ | $\mathbf{57.5\pm.2}$ | $87.4$

Table 1: Mean test accuracy (in percent) and standard error, comparing our method (MAP) and the baselines. In the top 2 tables we show results with 4 image datasets with injected label noise, and 2 image datasets with presumed label noise marked by (est). In the bottom table we show results with 3 text datasets with injected label noise and 1 image dataset with presumed label noise marked by (est).  Method/Dataset | CIFAR-10 sym | CIFAR-100 sym | TinyImagenet sym | Clothing1M
---|---|---|---|---
% label noise | 20% | 40% | 20% | 40% | 20% | 40% | 38% (est)
_RoG_ | $87.4$ | $81.8$ | $64.3$ | $55.6$ | - | - | 68.0
_consensus_ | $87.4\pm.3$ | $74.7\pm.3$ | $71.0\pm.4$ | $41.9\pm.1$ | $23.7\pm.2$ | $18.5\pm.3$ | 66.9
_NPC_ | 89.8 | $77.5$ | $73.7$ | $61.5$ | 62.1 | 50.3 | $70.8$
_MAP_ | $\mathbf{93.2\pm.07}$ | $\mathbf{90.0\pm.1}$ | $\mathbf{76.7\pm.1}$ | $\mathbf{69.7\pm.5}$ | $\mathbf{66.0\pm.1}$ | $\mathbf{60.1\pm.06}$ | $\mathbf{71.7\pm.2}$
Table 2: Mean test accuracy (in percent) and standard error, comparing our
method (MAP) to some of the post processing methods discussed above. We report
3 image datasets with injected label noise, and a single dataset with presumed
annotation errors marked by (est). Source of alternative methods results: RoG
from (Lee et al. 2019) and consensus from (Salman and Liu 2019). NPC (Bae et
al. 2022) results are recomputed using our settings and our own
implementation.

Method/method | BatchEnsemble | Hyper-ensemble
---|---|---
% label noise (sym) | 20% | 40% | 20% | 40%
_original accuracy_ | 69.5 | 52.8 | 72.1 | 61.3
_MAP_ | 72.5 | 60.1 | 74.1 | 67.6

Method/method | BatchEnsemble | Hyper-ensemble
---|---|---
% label noise (asym) | 10% | 20% | 10% | 20%
_original accuracy_ | 75.7 | 67.8 | 76.9 | 71.2
_MAP_ | 77.8 | 71.1 | 77.5 | 75.3

Table 3: testing our method on unique ensembles designed to decrease
cost/increase diversity, trained on CIFAR100 with injected label noise. Our
method is superior compared to those ensembles, both in the unique training
setting and in the general one.

###  4.4 Limitations

Our method has two main limitations: (i) the need for multiple network
training, and (ii) the need to save multiple checkpoints of the network during
training to be used at inference time. Importantly, using multiple GPUs can
mitigate these limitations, as the networks are trained independently and the
checkpoints are inspected independently at inference time. Not less
importantly, in Sections. 5.1-5.2 and Table 3 we show that training a few
networks, and saving only a few checkpoints for each one (equally spaced
through time), suffice for optimal or almost optimal results; and that
ensemble methods that minimize the training cost can go along well with our
method, making our method much more practical.

##  5 Ablation study

Our method requires the training of a few networks, which is computationally
costly. This is justified by the improvement achieved by ensemble classifiers
over a single classifier, as shown above. In order to evaluate the practical
implication of this added complexity, we investigated the effect of the number
of networks in the ensemble on the final outcome of a regular ensemble and of
our method, see Section 5.1.

When compared with alternative ensemble classifiers, our method incurs an
additional cost as it requires that we save multiple checkpoints during the
training, used to generate a time series of predictions. This is justified by
the additional improvement achieved by our method as shown above. To evaluate
the practical significance of this factor, we investigated the effect of using
only a small subset of the training checkpoints, see Section 5.2. Additional
results are described in App. D.1 (effect of architecture and training
choices) and App. D.2 (performance without label noise).

Summarizing the ablation results, even with _a few networks_ and _a few
checkpoints for each_ , our method achieves optimal or near optimal
performance, making it practical and useful with limited computational
resources. Interestingly, our method can be combined with methods for learning
with noisy labels, improving performance when such methods fail to eliminate
overfit (see Table 6.) Finally, the network architecture does not change the
overall picture - our method maintains its benefit over alternative deep
ensemble.

(a) cifar10, 20% asymmetric noise

(b) cifar100, 40% symmetric noise

Figure 6: test accuracy of a regular ensemble and MAP when varying the size of the ensemble, where the x-axis denotes the number of networks in the ensemble. Method/Dataset | TinyImagenet
---|---
% label noise (sym) | 20% | 40% | 60%
_elr_ | 57.5 | 47.8 | 25.9
_elr ensemble_ | 61.6 | 54.0 | 33.3
_elr + MAP_ | $\mathbf{62.3}$ | $\mathbf{57.4}$ | $\mathbf{47.2}$
Table 4: A competitive method for label noise - elr (Liu et al. 2020), and its
performance with and without MAP.

Method/Dataset | CIFAR-10 sym | CIFAR-100 sym
---|---|---
% label noise | 20% | 40% | 20% | 40%
_MAP (5)_ | 93.3 | 90.0 | 76.6 | 70.1
_majority vote ( $opt$)_ | 91.8 | 84.2 | 77.3 | 69.1
_MAP ( $opt$)_ | 93.5 | 90.7 | 77.4 | 71.3

Method/Dataset | CIFAR-10 asym | CIFAR-100 asym
---|---|---
% label noise | 20% | 40% | 20% | 40%
_MAP (5)_ | 94.4 | 85.7 | 77.9 | 57.3
_majority vote ( $opt$)_ | 90.5 | 66.5 | 76.6 | 56.1
_MAP ( $opt$)_ | 94.5 | 86.9 | 78.4 | 59.1

Table 5: Comparing ensembles with optimal size, denoted ($opt$), beyond which
adding more networks does not improve the ensemble performance, to our method
with 5 and $opt$ networks.

###  5.1 The effect of ensemble size

In the results shown in Table 1, all ensemble classifiers use the same
ensemble size (5), which shows the usefulness of adding the networks’ history
to the prediction at inference. However, to achieve a fair comparison between
a regular ensemble and our method, we repeat the study without limiting the
number of networks in the ensemble, reporting accuracy at the point where the
addition of models does not improve performance, see results in Table 5.
Notably, our method maintains its superiority. Moreover, we see that with MAP
- even a few networks achieve optimal or close to optimal results, making it
practical for use (see Figs. 6(a),6(b)).

###  5.2 How many checkpoints are needed?

The agreement score in (2) can be estimated from any subset of epochs. Fig. 7
shows test accuracy when using a subset of the training epochs, equally spaced
through time. These results demonstrate that only a few checkpoints are
sufficient to achieve the same accuracy (or close to it) as when using all the
epochs. The method is quite robust to the selection of epochs: we considered
eliminating early epochs, but observed that this had no significant effect on
performance, and only increased the number of hyperparameters.

(a) cifar10 40% AsN

(b) cifar100 60% SN

(c) TinyImg 20% SN

(d) cifar100 20% AsN

Figure 7: Test accuracy of MAP when using only a fraction of the epochs
(equally spaced) per number of epochs. ’SN’ denotes symmetric noise, and ’AsN’
asymetric noise.

##  6 Conclusions

Overfit is a notorious problem, which afflicts deep learning especially in the
context of real life data. In this paper we propose a new ensemble classifier
based on a collection of deep networks, termed MAP, which tracks the evolution
of predictions in each model in addition to its final prediction, and then
generates predictions that in most cases do not show any overfit. This
empirical result is consistently shown using various datasets, including both
textual and image data, and a variety of settings that exhibit heavy overfit.
The method is based on an empirical result, which shows that the agreement
among deep networks decreases with the occurrence of overfit. Further support
is provided by the theoretical analysis of a linear regression model.

Our empirical study focused for the most part, though not entirely, on
datasets with noisy labels, a real-world condition that creates heavy
overfitting, where our method is most useful. In almost all cases, the
method’s resistance to overfit is maintained even when the level of noise is
very high. In such cases, our method significantly outperforms regular
ensembles, as well as other post-processing methods designed specifically to
handle noisy labels. Its advantage over early stopping, shown in Fig. 5, is
particularly interesting, as it hints towards the possibility that the
networks learn new _useful_ patterns in the data even while they are
overfitting, in accordance with the double descent phenomenon. This hypothesis
is an interesting topic for future research, and serves as an additional
motivation to using our method: it could, in some cases, allow the user to
harness the new ”knowledge” learned by the model at the late stages of
training, while minimizing the damages of overfit.

Our method has some practical advantages. Notably, it is easy to implement,
and can be readily used at post-processing with almost any method, network and
dataset. This is accomplished _without any change to the training process,
without prior assumptions on the existence of label noise in the training data
and its properties, and without any additional hyper-parameter tuning_. Since
our method is only employed at post-processing, when no overfit is suspected
it can be completely avoided. Lastly, unless extreme overfit occurs, MAP also
eliminates the need for a clean validation set, as it does not introduce new
hyper-parameters and does not require ”early stopping”, often even surpassing
the early stopping performance. Thus, it serves as a practical tool to combat
overfit (with or without label noise in the training set), especially when
large amounts of correctly labeled data are difficult to acquire.

## References

  * Arazo et al. (2019) Arazo, E.; Ortego, D.; Albert, P.; O’Connor, N.; and McGuinness, K. 2019.  Unsupervised label noise modeling and loss correction.  In _International conference on machine learning_ , 312–321. PMLR.
  * Arpit et al. (2017) Arpit, D.; Jastrzebski, S.; Ballas, N.; Krueger, D.; Bengio, E.; Kanwal, M. S.; Maharaj, T.; Fischer, A.; Courville, A.; Bengio, Y.; et al. 2017.  A closer look at memorization in deep networks.  In _International conference on machine learning_ , 233–242. PMLR.
  * Bae et al. (2022) Bae, H.; Shin, S.; Na, B.; Jang, J.; Song, K.; and Moon, I.-C. 2022.  From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Model.  In _International Conference on Machine Learning_ , 1277–1297. PMLR.
  * Bai et al. (2021) Bai, Y.; Yang, E.; Han, B.; Yang, Y.; Li, J.; Mao, Y.; Niu, G.; and Liu, T. 2021\.  Understanding and improving early stopping for learning with noisy labels.  _Advances in Neural Information Processing Systems_ , 34: 24392–24403.
  * Breiman (1996) Breiman, L. 1996.  Bagging predictors.  _Machine learning_ , 24(2): 123–140.
  * Chen et al. (2019) Chen, P.; Liao, B. B.; Chen, G.; and Zhang, S. 2019.  Understanding and utilizing deep neural networks trained with noisy labels.  In _International Conference on Machine Learning_ , 1062–1070. PMLR.
  * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , 248–255. Ieee.
  * Devlin et al. (2018) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Freund and Schapire (1997) Freund, Y.; and Schapire, R. E. 1997.  A decision-theoretic generalization of on-line learning and an application to boosting.  _Journal of computer and system sciences_ , 55(1): 119–139.
  * Ganaie, Hu et al. (2021) Ganaie, M. A.; Hu, M.; et al. 2021.  Ensemble deep learning: A review.  _arXiv preprint arXiv:2104.02395_.
  * Ghosh, Kumar, and Sastry (2017) Ghosh, A.; Kumar, H.; and Sastry, P. S. 2017.  Robust loss functions under label noise for deep neural networks.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 31.
  * Goldberger and Ben-Reuven (2017) Goldberger, J.; and Ben-Reuven, E. 2017.  Training deep neural-networks using a noise adaptation layer.  In _International Conference on Learning Representations_.
  * Hacohen, Choshen, and Weinshall (2020) Hacohen, G.; Choshen, L.; and Weinshall, D. 2020.  Let’s Agree to Agree: Neural Networks Share Classification Order on Real Datasets.  In _Int. Conf. Machine Learning ICML_ , 3950–3960.
  * Han et al. (2018) Han, B.; Yao, Q.; Yu, X.; Niu, G.; Xu, M.; Hu, W.; Tsang, I.; and Sugiyama, M. 2018\.  Co-teaching: Robust training of deep neural networks with extremely noisy labels.  _Advances in neural information processing systems_ , 31.
  * Hansen and Salamon (1990) Hansen, L. K.; and Salamon, P. 1990.  Neural network ensembles.  _IEEE transactions on pattern analysis and machine intelligence_ , 12(10): 993–1001.
  * Ioffe and Szegedy (2015) Ioffe, S.; and Szegedy, C. 2015.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International conference on machine learning_ , 448–456. PMLR.
  * Jenni and Favaro (2018) Jenni, S.; and Favaro, P. 2018.  Deep bilevel learning.  In _Proceedings of the European conference on computer vision (ECCV)_ , 618–633.
  * Jiang et al. (2018) Jiang, L.; Zhou, Z.; Leung, T.; Li, L.-J.; and Fei-Fei, L. 2018.  Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels.  In _International Conference on Machine Learning_ , 2304–2313. PMLR.
  * Karim et al. (2022) Karim, N.; Rizve, M. N.; Rahnavard, N.; Mian, A.; and Shah, M. 2022.  UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 9676–9686.
  * Karimi et al. (2020) Karimi, D.; Dou, H.; Warfield, S. K.; and Gholipour, A. 2020.  Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis.  _Medical Image Analysis_ , 65: 101759.
  * Krizhevsky, Hinton et al. (2009) Krizhevsky, A.; Hinton, G.; et al. 2009.  Learning multiple layers of features from tiny images.  _Online_.
  * Krogh and Hertz (1991) Krogh, A.; and Hertz, J. 1991.  A simple weight decay can improve generalization.  _Advances in neural information processing systems_ , 4.
  * Lakshminarayanan, Pritzel, and Blundell (2017) Lakshminarayanan, B.; Pritzel, A.; and Blundell, C. 2017.  Simple and scalable predictive uncertainty estimation using deep ensembles.  _Advances in neural information processing systems_ , 30.
  * Le and Yang (2015) Le, Y.; and Yang, X. 2015.  Tiny imagenet visual recognition challenge.  _CS 231N_ , 7(7): 3.
  * Lee et al. (2019) Lee, K.; Yun, S.; Lee, K.; Lee, H.; Li, B.; and Shin, J. 2019.  Robust inference via generative classifiers for handling noisy labels.  In _International Conference on Machine Learning_ , 3763–3772. PMLR.
  * Li, Wang, and Ding (2018) Li, H.; Wang, X.; and Ding, S. 2018.  Research and development of neural network ensembles: a survey.  _Artificial Intelligence Review_ , 49(4): 455–479.
  * Li, Socher, and Hoi (2020) Li, J.; Socher, R.; and Hoi, S. C. 2020.  Dividemix: Learning with noisy labels as semi-supervised learning.  _arXiv preprint arXiv:2002.07394_.
  * Li et al. (2017) Li, W.; Wang, L.; Li, W.; Agustsson, E.; and Van Gool, L. 2017.  Webvision database: Visual learning and understanding from web data.  _arXiv preprint arXiv:1708.02862_.
  * Liu et al. (2020) Liu, S.; Niles-Weed, J.; Razavian, N.; and Fernandez-Granda, C. 2020.  Early-learning regularization prevents memorization of noisy labels.  _Advances in neural information processing systems_ , 33: 20331–20342.
  * Nakkiran et al. (2021) Nakkiran, P.; Kaplun, G.; Bansal, Y.; Yang, T.; Barak, B.; and Sutskever, I. 2021\.  Deep double descent: Where bigger models and more data hurt.  _Journal of Statistical Mechanics: Theory and Experiment_ , 2021(12): 124003.
  * Nicholson, Sheng, and Zhang (2016) Nicholson, B.; Sheng, V. S.; and Zhang, J. 2016.  Label noise correction and application in crowdsourcing.  _Expert Systems with Applications_ , 66: 149–162.
  * Patrini et al. (2017) Patrini, G.; Rozza, A.; Krishna Menon, A.; Nock, R.; and Qu, L. 2017.  Making deep neural networks robust to label noise: A loss correction approach.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 1944–1952.
  * Pleiss et al. (2020) Pleiss, G.; Zhang, T.; Elenberg, E.; and Weinberger, K. Q. 2020.  Identifying mislabeled data using the area under the margin ranking.  _Advances in Neural Information Processing Systems_ , 33: 17044–17056.
  * Pliushch et al. (2021) Pliushch, I.; Mundt, M.; Lupp, N.; and Ramesh, V. 2021.  When Deep Classifiers Agree: Analyzing Correlations between Learning Order and Image Statistics.  _arXiv preprint arXiv:2105.08997_.
  * Salman and Liu (2019) Salman, S.; and Liu, X. 2019.  Overfitting mechanism and avoidance in deep neural networks.  _arXiv preprint arXiv:1901.06566_.
  * Sanh et al. (2021) Sanh, V.; Webson, A.; Raffel, C.; Bach, S. H.; Sutawika, L.; Alyafeai, Z.; Chaffin, A.; Stiegler, A.; Scao, T. L.; Raja, A.; Dey, M.; Bari, M. S.; Xu, C.; Thakker, U.; Sharma, S. S.; Szczechla, E.; Kim, T.; Chhablani, G.; Nayak, N.; Datta, D.; Chang, J.; Jiang, M. T.-J.; Wang, H.; Manica, M.; Shen, S.; Yong, Z. X.; Pandey, H.; Bawden, R.; Wang, T.; Neeraj, T.; Rozen, J.; Sharma, A.; Santilli, A.; Fevry, T.; Fries, J. A.; Teehan, R.; Biderman, S.; Gao, L.; Bers, T.; Wolf, T.; and Rush, A. M. 2021.  Multitask Prompted Training Enables Zero-Shot Task Generalization.  arXiv:2110.08207.
  * Shorten and Khoshgoftaar (2019) Shorten, C.; and Khoshgoftaar, T. M. 2019.  A survey on image data augmentation for deep learning.  _Journal of big data_ , 6(1): 1–48.
  * Sollich and Krogh (1995) Sollich, P.; and Krogh, A. 1995.  Learning with ensembles: How overfitting can be useful.  _Advances in neural information processing systems_ , 8.
  * Song, Kim, and Lee (2019) Song, H.; Kim, M.; and Lee, J.-G. 2019.  SELFIE: Refurbishing Unclean Samples for Robust Deep Learning.  In _ICML_.
  * Srivastava et al. (2014) Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014.  Dropout: a simple way to prevent neural networks from overfitting.  _The journal of machine learning research_ , 15(1): 1929–1958.
  * Stephenson and Lee (2021) Stephenson, C.; and Lee, T. 2021.  When and how epochwise double descent happens.  _arXiv preprint arXiv:2108.12006_.
  * Tanno et al. (2019) Tanno, R.; Saeedi, A.; Sankaranarayanan, S.; Alexander, D. C.; and Silberman, N. 2019.  Learning from noisy labels by regularized estimation of annotator confusion.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 11244–11253.
  * Van Gansbeke et al. (2020) Van Gansbeke, W.; Vandenhende, S.; Georgoulis, S.; Proesmans, M.; and Van Gool, L. 2020.  Scan: Learning to classify images without labels.  In _European Conference on Computer Vision_ , 268–285. Springer.
  * Wang et al. (2018) Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2018.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.  _CoRR_ , abs/1804.07461.
  * Wang et al. (2019) Wang, Y.; Ma, X.; Chen, Z.; Luo, Y.; Yi, J.; and Bailey, J. 2019.  Symmetric cross entropy for robust learning with noisy labels.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 322–330.
  * Wen, Tran, and Ba (2020) Wen, Y.; Tran, D.; and Ba, J. 2020.  Batchensemble: an alternative approach to efficient ensemble and lifelong learning.  _arXiv preprint arXiv:2002.06715_.
  * Wenzel et al. (2020) Wenzel, F.; Snoek, J.; Tran, D.; and Jenatton, R. 2020.  Hyperparameter ensembles for robustness and uncertainty quantification.  _Advances in Neural Information Processing Systems_ , 33: 6514–6527.
  * Xiao et al. (2015) Xiao, T.; Xia, T.; Yang, Y.; Huang, C.; and Wang, X. 2015.  Learning from massive noisy labeled data for image classification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2691–2699.
  * Xu et al. (2019) Xu, Y.; Cao, P.; Kong, Y.; and Wang, Y. 2019.  L_dmi: A novel information-theoretic loss function for training deep nets robust to label noise.  _Advances in neural information processing systems_ , 32.
  * Yao et al. (2021) Yao, Y.; Sun, Z.; Zhang, C.; Shen, F.; Wu, Q.; Zhang, J.; and Tang, Z. 2021.  Jo-src: A contrastive approach for combating noisy labels.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 5192–5201.
  * Zhang, Lee, and Agarwal (2021) Zhang, M.; Lee, J.; and Agarwal, S. 2021.  Learning from noisy labels with no change to the training process.  In _International Conference on Machine Learning_ , 12468–12478. PMLR.
  * Zhang and Sabuncu (2018) Zhang, Z.; and Sabuncu, M. 2018.  Generalized cross entropy loss for training deep neural networks with noisy labels.  _Advances in neural information processing systems_ , 31.
  * Zheltonozhskii et al. (2022) Zheltonozhskii, E.; Baskin, C.; Mendelson, A.; Bronstein, A. M.; and Litany, O. 2022\.  Contrast to divide: Self-supervised pre-training for learning with noisy labels.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 1657–1667.

## Appendix

##  Appendix A Ensembles improve robustness to errors

(a) 3 decision boundaries

(b) Ensemble classifier

Figure 8: (a) A $2D$ classification example, with a single false label (in
red) and an ensemble of 3 models, each trained to overfit the false label. (b)
The black line marks the correct decision boundary of the data’s empirical
distribution. The decision boundary of an ensemble classifier, based on the
majority vote of the 3 instances in (a), is shown in pink. It is clearly
closer to the correct decision boundary than any of the 3 members of the
ensemble.

Ensemble-based methods deliver some statistics of the distribution of
predictions, seeking a better estimator of the prediction of the correct
classifier, and are expected to be effective in reducing error when there is
variance in the classifiers’ predictions. In Fig. 8 we demonstrate this
intuition with a simple toy problem, a $2D$ example with two classes and label
noise. In Fig. 8(a) we show three possible decision boundaries (models),
obtained since the fitting procedure is assumed to be stochastic. Because the
training set has an erroneous label, the models deviate largely from the true
separator. In Fig. 8(b) we plot the decision boundary of the ensemble
constructed using these models (in pink), which is clearly superior, being
closer to the true separator (in black).

##  Appendix B Overfit and inter-model correlation

In this section we formally analyze the relation between two type of scores,
which measure either overfit or inter-model agreement. _Overfit_ is a
condition that can occur during the training of deep neural networks. It is
characterized by the co-occurring decrease of train error or loss, which is
continuously minimized during the training of a deep model, and the increase
of test error or loss, which is the ideal measure one would have liked to
minimize and which determines the network’s generalization error. An
_agreement_ score measures how similar the models are in their predictions.

We start by introducing the model and some notations in Section B.1. In
Section B.2 we prove the main result (Prop. Theorem): the occurrence of
overfit at time s in all the models of the ensemble implies that the agreement
between the models decreases.

###  B.1 Model and notations

Model. We analyze the agreement between an ensemble of $Q$ models, computed by
solving the linear regression problem with Gradient Descent (GD) and random
initialization. In this problem, the learner estimates a linear function
$f({\mathbf{x}}):\mathbb{R}^{d}\to\mathbb{R}$, where
${\mathbf{x}}\in\mathbb{R}^{d}$ denotes an input vector and $y\in\mathbb{R}$
the desired output. Given a training set of $M$ pairs
$\\{{\mathbf{x}}_{m},y_{m}\\}_{m=1}^{M}$, let $X\in\mathbb{R}^{d\times M}$
denote the training input - a matrix whose $m^{\mathrm{th}}$ column is
${\mathbf{x}}_{m}\in\mathbb{R}^{d}$, and let row vector
$\bm{y}\in\mathbb{R}^{M}$ denote the output vector whose $m^{\mathrm{th}}$
element is $y_{m}$. When solving a linear regression problem, we seek a row
vector $\hat{\bm{w}}\in\mathbb{R}^{D}$ that satisfies

$\hat{\bm{w}}=\operatorname{argmin}\limits_{\bm{w}}L({\bm{w}}),\quad
L({\bm{w}})=\frac{1}{2}\|{\bm{w}}X-\bm{y}\|_{F}^{2}$ (5)

To solve (5) with GD, we perform at each iterative step $s\geq 1$ the
following computation:

$\begin{split}{\bm{w}}^{s+1}=&{\bm{w}}^{s}-\mu\Delta\bm{w}^{s}\\\
\Delta\bm{w}^{s}=&\frac{\partial
L(\mathbb{X})}{\partial{\bm{w}}}\bigg{|}_{{\bm{w}}={\bm{w}}^{s}}={\bm{w}}^{s}\Sigma_{\scriptscriptstyle
XX}-\Sigma_{\scriptscriptstyle YX}\\\ &\Sigma_{\scriptscriptstyle
XX}=XX^{\top},~{}\Sigma_{\scriptscriptstyle YX}=\bm{y}X^{\top}\end{split}$ (6)

for some random initialization vector ${\bm{w}}_{0}\in\mathbb{R}^{M}$ where
usually $\mathbb{E}[{\bm{w}}_{0}]=0$, and learning rate $\mu$. Henceforth we
omit the index $s$ when self evident from context.

As a final remark, when we use the notation $\|A\|$ below, it denotes the
operator norm of the symmetric matrix $A$, namely, its largest singular value.

Additional notations

  * •

Index $i\in[Q]$ denotes a network instance, and $t$ denotes the test data. For
simplicity and with some risk of notation abuse, let $Q$ and $Q^{\prime}$ also
denote sets of indices, either training or test. Specifically,
$Q=[1,\ldots,Q]$ and $Q^{\prime}=[1,\ldots,Q,t]$.

  * •

We use function notation, where $\\{X(i),y(i)\\}$ is the training set of
network $i$ and $\\{X(t),y(t)\\}$ is the test set. Thus

$\Sigma_{\scriptscriptstyle
XX}(j)=\mathbb{X}(j)\mathbb{X}(j)^{\top},\Sigma_{\scriptscriptstyle
YX}(j)=\bm{y}(j)\mathbb{X}(j)^{\top}\quad j\in Q^{\prime}$

  * •

Similarly, ${\bm{w}}(i)\in\mathbb{R}^{d}$ is the model learned by network $i$,
and $\Delta\bm{w}(i)$ is the gradient step of ${\bm{w}}(i)$, where

$\Delta\bm{w}(i)={\bm{w}}(i)\Sigma_{\scriptscriptstyle
XX}(i)-\Sigma_{\scriptscriptstyle YX}(i)\qquad i\in Q$

  * •

$\bm{e}(i,j)$ denotes a function, which maps indices $i\in Q,j\in Q^{\prime}$
to the cross error of model $i$ on data $j$ \- the classification error vector
when using model ${\bm{w}}(i)$ to estimate $y(j)$. Let $M^{\prime}=M$ if $j\in
Q$ is a training index, and $M^{\prime}=N$ if $j\in\\{t\\}$ .Then we can write

$\begin{split}&\bm{e}(i,j):Q\times
Q^{\prime}\to\mathbb{R}^{M^{\prime}}\quad\bm{e}(i,j)={\bm{w}}(i)\mathbb{X}(j)-\bm{y}(j)\\\
&\implies\quad\Delta\bm{w}(i)=\bm{e}(i,i)\mathbb{X}(i)^{\top}\end{split}$

Note that in this notation, $e(i,t)$ is the classification error vector when
using model $i$, which is trained on data $\mathbb{X}(i)$, to estimate the
desired outcome on the test data - $y(t)$. $\|e(i,t)\|_{F}$ is the test error,
estimate of the generalization error, of classifier $i$.

  * •

Let $\Delta(i,j)$ denote the cross gradient:

$\begin{split}\Delta(i,j)=\bm{e}(i,j)\mathbb{X}(j)^{\top}={\bm{w}}(i)\Sigma_{\scriptscriptstyle
XX}(j)-\Sigma_{\scriptscriptstyle YX}(j)\quad\\\
\implies\quad\Delta\bm{w}(i)=\Delta(i,i)\end{split}$ (7)

After each GD step, the model and the error are updated as follows:

$\begin{split}\tilde{\bm{w}}(i)&={\bm{w}}(i)-\mu\Delta\bm{w}(i)\\\
\tilde{\bm{e}}(i,j)&=\tilde{\bm{w}}(i)\mathbb{X}(j)-\bm{y}(j)=\bm{e}(i,j)-\mu\Delta(i,i)\mathbb{X}(j)\end{split}$

We note that at step $s$ and $\forall i,j\in Q$, $\tilde{\bm{w}}(i)$ is a
random vector in $\mathbb{R}^{d}$, and $\tilde{\bm{e}}(i,j)$ is a random
vector in $\mathbb{R}^{M}$. If $j\in\\{t\\}$, then
$\tilde{\bm{e}}(i,j)=\tilde{\bm{e}}(i,t)$ is a random vector in
$\mathbb{R}^{N}$.

Test error random variable. Let $N$ denote the number of test examples. Note
that $\\{\bm{e}(i,t)\\}_{i=1}^{Q}$ is a set of $Q$ test errors vectors in
$\mathbb{R}^{N}$, where the $n^{\mathrm{th}}$ component of the
$i^{\mathrm{th}}$ vector $\bm{e}(i,t)_{n}$ captures the test error of model
$i$ on test example $n$. In effect, it is a sample of size $Q$ from the random
variable $\bm{e}(*,t)_{n}$. This random variable captures the error over test
point $n$ of a model computed from a random sample of $M$ training examples.
The empirical variance of this random variable will be used to estimate the
agreement between the models.

Overfit. Overfit occurs at step $s$ if

$\|\tilde{\bm{e}}(i,t)\|_{F}^{2}>\|\bm{e}(i,t)\|_{F}^{2}$ (8)

Measuring inter-model agreement. For our analysis, we need a score to measure
agreement between the predictions of $Q$ linear functions. This measure is
chosen to be the variance of the test error among models. Accordingly, we will
measure _disagreement_ by the empirical variance of the test error random
variable $\tilde{\bm{e}}(*,t)_{n}$, average over all test examples $n\in[N]$.

More specifically, consider an ensemble of linear models
$\\{w(i)\\}_{i=1}^{Q}$ trained on set $\mathbb{X}$ to minimize (5) with $s$
gradient steps, where $i$ denotes the index of a network instance and $Q$ the
number of network instances. Using the test error vectors of these models
$e(i,t)$, we compute the empirical variance of each element
$\mathrm{var}[\bm{e}(*,t)_{n}]$, and sum over the test examples $n\in[N]$:

$\begin{split}\sum_{n=1}^{N}\sigma^{2}[e(*,t)_{n}]&=\sum_{n=1}^{N}\frac{1}{2Q^{2}}\sum_{i=1}^{Q}\sum_{j=1}^{Q}|e(i,t)_{n}-e(j,t)_{n}|^{2}\\\
&=\frac{1}{2Q^{2}}\sum_{i=1}^{Q}\sum_{j=1}^{Q}\|e(i,t)-e(j,t)\|^{2}\end{split}$

######  Definition 1 (Inter-model DisAgreement.).

The disagreement among a set of $Q$ linear models $\\{w(i)\\}_{i=1}^{Q}$ at
step $s$ is defined as follows

$DisAg(s)=\frac{1}{2Q^{2}}\sum_{i=1}^{Q}\sum_{j=1}^{Q}\|e(i,t)-e(j,t)\|^{2}$
(9)

###  B.2 Overfit and Inter-Network Agreement

We first prove Lemma 1, which has the following intuitive interpretation:
overfit occurs in model $i$ iff the gradient step of model $i$ (denoted
$\Delta\bm{w}(i)$), which is computed using the training set, is negatively
correlated with the ’correct’ gradient step - the one we would have obtained
had we known the test set (this unattainable vector is denoted $\Delta(i,t)$).

######  Lemma 1.

Assume that the learning rate $\mu$ is small enough so that we can neglect
terms that are $O(\mu^{2})$. Then in each gradient descent step $s$, overfit
occurs iff the gradient step $\Delta\bm{w}(i)$ of network $i$ is negatively
correlated with the cross gradient $\Delta(i,t)$.

###### Proof.

Starting from (8)

$\begin{split}\quad\mathrm{{\color[rgb]{0.36,0.54,0.66}(overfit)}}\iff&\|\tilde{\bm{e}}(i,t)\|_{F}^{2}>\|\bm{e}(i,t)\|_{F}^{2}\\\
\iff&\|\tilde{\bm{e}}(i,t)\|_{F}^{2}-\|\bm{e}(i,t)\|_{F}^{2}>0\\\
\iff&\|\bm{e}(i,t)-\mu\Delta(i,i)X(t)\|_{F}^{2}-\|\bm{e}(i,t)\|_{F}^{2}>0\\\
\iff&-2\mu\Delta(i,i)X(t)\bm{e}(i,t)^{\top}+O(\mu^{2})>0\\\
\iff&\Delta(i,i)\cdot\Delta(i,t)<0\\\
\iff&\Delta\bm{w}(i)\cdot\Delta(i,t)<0\end{split}$

∎

Lemma 2 claims that if the magnitude of the gradient step $\mu$ is small
enough, then the operator norm of matrix $I-\mu\Sigma_{\scriptscriptstyle XX}$
is smaller than 1. The implication is that a geometric sum of this matrix
converges, a technical result which will be used later.

######  Lemma 2.

For any invertible covariance matrix $\Sigma_{\scriptscriptstyle XX}$ there
exists $\hat{\mu}>0$, such that
$\mu<\hat{\mu}\implies\|I-\mu\Sigma_{\scriptscriptstyle XX}\|<1$.

###### Proof.

Since $\Sigma_{\scriptscriptstyle XX}$ is positive-definite, we can write
$\Sigma_{\scriptscriptstyle XX}=USU^{\top}$ for orthogonal matrix $U$ and the
diagonal matrix of singular values $S=diag\\{s_{i}\\}$. It follows that
$I-\mu\Sigma_{\scriptscriptstyle XX}=Udiag\\{1-\mu s_{i}\\}U^{\top}$, a matrix
whose largest singular value is $1-\mu s_{d}$. Since by assumption $s_{d}>0$,
the lemma follows. ∎

Our last Lemma 3 claims that eventually, after sufficiently many gradient
steps, the expected value of the solution is exactly the closed-form solution
of the vector that minimizes the loss.

######  Lemma 3.

Assume that $\|I-\mu\Sigma_{\scriptscriptstyle XX}\|<1$ and
$\Sigma_{\scriptscriptstyle XX}$ is invertible. If the number of gradient
steps $s$ is large enough so that $\|I-\mu\Sigma_{\scriptscriptstyle
XX}\|^{s}$ can be neglected, then

$\mathbb{E}[{\bm{w}}^{s}]\approx\Sigma_{\scriptscriptstyle
YX}\Sigma_{\scriptscriptstyle XX}^{-1}$ (10)

###### Proof.

Starting from (6), we can show that

${\bm{w}}^{s}={\bm{w}}^{0}(I-\mu\Sigma_{\scriptscriptstyle
XX})^{s-1}+\mu\Sigma_{\scriptscriptstyle
YX}\sum_{k=1}^{s-1}(I-\mu\Sigma_{\scriptscriptstyle XX})^{k-1}$

Since $\mathbb{E}({\bm{w}}^{0})=0$

$\begin{split}\mathbb{E}({\bm{w}}^{s})&=\mathbb{E}({\bm{w}}^{0})(I-\mu\Sigma_{\scriptscriptstyle
XX})^{s-1}+\mu\Sigma_{\scriptscriptstyle
YX}\sum_{k=1}^{s-1}(I-\mu\Sigma_{\scriptscriptstyle XX})^{k-1}\\\
&=\mu\Sigma_{\scriptscriptstyle
YX}\sum_{k=1}^{s-1}(I-\mu\Sigma_{\scriptscriptstyle XX})^{k-1}\end{split}$

Given the lemma’s assumptions, this expression can be evaluated and
simplified:

$\begin{split}\mathbb{E}({\bm{w}}^{s})&=\mu\Sigma_{\scriptscriptstyle
YX}[I-(I-\mu\Sigma_{\scriptscriptstyle
XX})]^{-1}[I-(I-\mu\Sigma_{\scriptscriptstyle XX})^{s-1}]\\\
&=\Sigma_{\scriptscriptstyle YX}\Sigma_{\scriptscriptstyle
XX}^{-1}-\Sigma_{\scriptscriptstyle YX}\Sigma_{\scriptscriptstyle
XX}^{-1}(I-\mu\Sigma_{\scriptscriptstyle XX})^{s-1}\\\
&\approx\Sigma_{\scriptscriptstyle YX}\Sigma_{\scriptscriptstyle
XX}^{-1}\end{split}$

∎

From (9) it follows that a decrease in inter-model agreement at step $s$,
which is implied by increased test variance among models, is indicated by the
following inequality:

$\begin{split}\mathbb{C}=&DisAg(s)-DisAg(s-1)\\\
=&\frac{1}{2Q^{2}}\sum_{i,j=1}^{Q}\|\tilde{\bm{e}}(i,t)-\tilde{\bm{e}}(j,t)\|^{2}-\\\
&\quad\frac{1}{2Q^{2}}\sum_{i,j=1}^{Q}\|\bm{e}(i,t)-\bm{e}(j,t)\|^{2}~{}>~{}0\end{split}$
(11)

######  Theorem.

Assume that all models see the same training set, denoted as $X(i)=X~{}\forall
i\in[Q]$, and that the training data covariance matrix
$\Sigma_{\scriptscriptstyle XX}$ is full rank.

We make the following asymptotic assumptions, which are loosely phrased but
can be rigorously defined with additional notations:

  1. 1.

The learning rate $\mu$ is small enough so that
$\|I-\mu\Sigma_{\scriptscriptstyle XX}\|<1$ (from Lemma 2), and additionally
we can neglect terms that are $O(\mu^{2})$.

  2. 2.

The number of gradient steps $s$ is large enough so that
$\|I-\mu\Sigma_{\scriptscriptstyle XX}\|^{s}$ can be neglected.

  3. 3.

The number of models $Q$ is large enough so that using the law of large
numbers, we get
$\frac{1}{Q}\sum_{i=1}^{Q}{\bm{w}}(i)\approx\mathbb{E}[{\bm{w}}]$.

Finally, we assume that overfit occurs at time $s$ in all the models of the
ensemble. In other words, at time $s$ the generalization error does not
decrease in all the models.

When these assumptions hold, the agreement between the models decreases.

###### Proof.

(11) can be rearranged as follows

$\begin{split}\mathbb{C}=&\frac{1}{2Q^{2}}\sum_{i,j=1}^{Q}\|[\bm{e}(i,t)-\mu\Delta(i,i)X(t)]-[\bm{e}(j,t)-\\\
&\qquad\mu\Delta(j,j)X(t)]\|^{2}-\frac{1}{2Q^{2}}\sum_{i,j=1}^{Q}\|\bm{e}(i,t)-\bm{e}(j,t)\|^{2}\\\
=&\frac{1}{Q^{2}}\sum_{i,j=1}^{Q}-\mu[\bm{e}(i,t)-\bm{e}(j,t)]\cdot[\Delta(i,i)X(t)]-\\\
&\qquad\Delta(j,j)X(t)]+O(\mu^{2})\\\
=&\frac{\mu}{Q^{2}}\sum_{i,j=1}^{Q}\left[\Delta(i,i)\cdot\Delta(j,t)+\Delta(j,j)\cdot\Delta(i,t)\right]-\\\
&\qquad\left[\Delta(i,i)\cdot\Delta(i,t)+\Delta(j,j)\cdot\Delta(j,t)\right]+O(\mu^{2})\\\
\end{split}$

where the last transition follows from
$\bm{e}(i,t)X(t)^{\top}\\!=\Delta(i,t)$. Using assumption 2

$\mathbb{C}=\mu(\mathbb{C}^{\prime}-\mathbb{C}^{\prime\prime})+O(\mu^{2})\approx\mu(\mathbb{C}^{\prime}-\mathbb{C}^{\prime\prime})$
(12)

where

$\begin{split}\mathbb{C}^{\prime\prime}&=\frac{1}{Q^{2}}\sum_{i,j=1}^{Q}\left[\Delta(i,i)\cdot\Delta(i,t)+\Delta(j,j)\cdot\Delta(j,t)\right]\\\
&=\frac{2}{Q}\sum_{i=1}^{Q}\Delta(i,i)\cdot\Delta(i,t)\end{split}$ (13)

and

$\begin{split}\mathbb{C}^{\prime}=&\frac{1}{Q^{2}}\sum_{i,j=1}^{Q}\left[\Delta(i,i)\cdot\Delta(j,t)+\Delta(j,j)\cdot\Delta(i,t)\right]\\\
=&\frac{1}{Q}\sum_{i=1}^{Q}\Delta(i,i)\cdot\frac{1}{Q}\sum_{j=1}^{Q}\Delta(j,t)+\\\
&\qquad\frac{1}{Q}\sum_{j=1}^{Q}\Delta(j,j)\cdot\frac{1}{Q}\sum_{i=1}^{Q}\Delta(i,t)\\\
=&\frac{1}{Q}\sum_{i=1}^{Q}\Delta(i,i)\cdot\frac{2}{Q}\sum_{j=1}^{Q}\Delta(j,t)\end{split}$
(14)

Next, we prove that $\mathbb{C}^{\prime}$ is approximately 0. We first deduce
from assumptions 1 and 4 that

$\begin{split}\frac{1}{Q}&\sum_{i=1}^{Q}\Delta(i,i)=\frac{1}{Q}\sum_{i=1}^{Q}{\bm{w}}(i)\Sigma_{\scriptscriptstyle
XX}(i)-\Sigma_{\scriptscriptstyle YX}(i)\\\
&=\left(\frac{1}{Q}\sum_{i=1}^{Q}{\bm{w}}(i)\right)\Sigma_{\scriptscriptstyle
XX}-\Sigma_{\scriptscriptstyle
YX}\approx\mathbb{E}[{\bm{w}}]\Sigma_{\scriptscriptstyle
XX}-\Sigma_{\scriptscriptstyle YX}\end{split}$

From assumption 3 and Lemma 3, we have that
$\mathbb{E}[{\bm{w}}]\approx\Sigma_{\scriptscriptstyle
YX}\Sigma_{\scriptscriptstyle XX}^{-1}$. Thus

$\begin{split}\frac{1}{Q}\sum_{i=1}^{Q}\Delta(i,i)&\approx\mathbb{E}[{\bm{w}}]\Sigma_{\scriptscriptstyle
XX}-\Sigma_{\scriptscriptstyle YX}\\\ &\approx\Sigma_{\scriptscriptstyle
YX}\Sigma_{\scriptscriptstyle XX}^{-1}\Sigma_{\scriptscriptstyle
XX}-\Sigma_{\scriptscriptstyle YX}=0\end{split}$

From this derivation and (14) we may conclude that $\mathbb{C}^{\prime}\approx
0$. Thus

$\mathbb{C}\approx-\mu\mathbb{C}^{\prime\prime}=-\mu\frac{2}{Q}\sum_{i=1}^{Q}\Delta(i,i)\cdot\Delta(i,t)$
(15)

If overfit occurs at time $s$ in all the models of the ensemble, then
$\mathbb{C}>0$ from Lemma 1 and (15). From (11) we may conclude that the
inter-model agreement decreases, which concludes the proof.

∎

##  Appendix C Additional evaluations

In Table 6 we present additional results on another image classification
dataset with natural label noise and three text classification datasets with
injected label noise, which demostrates our method’s improvement.

Method/Dataset | mnli | qnli | qqp | Webvision50
---|---|---|---|---
% label noise | 20% | 40% | 20% | 40% | 20% | 40% | 20% (est)
_single network_ | $79.3\pm.07$ | $74.5\pm.06$ | $86.0\pm.05$ | $74.5\pm.05$ | $85.7\pm.04$ | $75.6\pm.04$ | $78.0\pm.2$
_majority vote ensemble_ | $81.6\pm.2$ | $76.7\pm.2$ | $87.3\pm.1$ | $74.6\pm.4$ | $88.1\pm.1$ | $76.3\pm.1$ | $85.3\pm.1$
_probability average ensemble_ | 82.1 | 77.3 | 87.5 | 74.3 | 88.2 | 77.0 | -
_MAP_ | $\mathbf{82.6\pm.09}$ | $\mathbf{79.3\pm.1}$ | $\mathbf{89.0\pm.1}$ | $\mathbf{82.5\pm.08}$ | $\mathbf{88.8\pm.05}$ | $\mathbf{82.2\pm.1}$ | $\mathbf{86.5\pm.03}$
Table 6: Mean test accuracy (in percent) and standard error, comparing our
method (MAP) and the baselines. In the top 2 tables we show results with 4
image datasets with injected label noise, and 2 image datasets with presumed
label noise marked by (est). In the bottom table we show results with 3 text
datasets with injected label noise and 1 image dataset with presumed label
noise marked by (est).

##  Appendix D Additional ablation results

We present in this appendix additional ablation results, demonstrating that
(i) our method is not dependent on specific architecture, optimizer and hyper-
parameters, and is even robust to the presence of an adversarial network in
the ensemble (ii) our ensemble can be constructed with diverse networks, or
ones with pretrained weights on a different dataset (such as Imagenet) (iii)
our method can be integrated with methods designed for learning under the
presence of noisy labels, when they fail to eliminate all overfit (iiii) our
method can also improve performance on datasets with only clean data, when
there is small amount of training data per class (when overfit is likely to
increase).

###  D.1 The role of architecture and hyperparameters

Fig. 9 shows results from experiments with different network architecture or
learning rate scheduler. In all cases, our method still successfully
eliminates the overfit.

We have also tested our method on a few other settings of interest:

  * •

We tested our method on ensembles of different networks architectures/sizes
trained in the same manner, and saw that the method is still effective in its
performance improvement, showing comparable improvement to the one presented
in Table 1.

  * •

Transfer learning - we tested our method when initial weights are taken from
pretraining (Imagenet), and saw our method still provides large improvement in
its final performance.

  * •

Using different optimizer - we also tested our method when using a different
optimizer (adamw) - our method still shows large improvement in performance.

  * •

Having an adversarial network in the ensemble - interestingly, we also saw
that due to the usage of multiple networks, our method is robust to the usage
of a network that is poorly trained, or even provides random predictions,
which is another advantage of our method.

The full results appear in Table 7. All experiments were done with densenet121
trained on cifar100 with 40% symmetric noise, except transfer learning, where
our network choice was resnet18 pre-trained on Imagenet.

Method/Test | Adversarial network | Transfer learning | Different architectures | Different optimizer
---|---|---|---|---
_single network_ | 52.0 | 54.9 | 52.2 | 29.4
_ensemble_ | 59.9 | 63.9 | 61.4 | 42.6
_MAP_ | $\mathbf{68.7}$ | $\mathbf{70.1}$ | $\mathbf{68.9}$ | $\mathbf{58.5}$
Table 7: Additional ablation results. In the adversarial network setting, one
network provides random predictions. In general, the single network shows the
best network in the ensemble.

(a) cifar100, 40% noise, const lr

(b) cifar100, 40% noise, resnet32

Figure 9: Test accuracy of MAP, different lr (a) and architecture (b).

###  D.2 Performance evaluation on clean datasets

We evaluate our method on clean datasets of different sizes, where overfit can
be much smaller or nonexistent. Results are shown in Fig. 10. Compared to a
regular ensemble, MAP is mostly better or comparable. We believe the
improvement in performance in smaller datasets is caused by the overfit caused
by the small amounts of data (another known source of overfit), making our
method useful in such cases as well.

(a) TinyImagenet

(b) cifar10

(c) cifar100

(d) Imagenet with 50 classes

Figure 10: x-axis: number of datapoints per class. y-axis: fraction of
predictions of a regular ensemble, which MAP changes from true to false
(orange) and from false to true (blue). MAP outperforms the ensemble whenever
the blue curve lies above the orange curve.

##  Appendix E Implementation details

#### Implementation details

We used the following datasets in our experiments: cifar10/100 (Krizhevsky,
Hinton et al. 2009) with $50,000$ training images of $10/100$ classes
respectively; TinyImagenet (Le and Yang 2015) with 100,000 training images of
200 classes; and Imagenet100 (cf. (Van Gansbeke et al. 2020)), which is a
subset of 100 classes from Imagenet (Deng et al. 2009) with 1300 training
images per class. In these datasets, in order to amplify overfit, we used
standard methods for injecting label noise (see above). As in all empirical
studies and in order to evaluate the generalization error correctly, the test
set is kept clean. We used three additional datasets, Clothing1M (Xiao et al.
2015), Webvision (Li et al. 2017) and Animal10N (Song, Kim, and Lee 2019)),
which are known to contain real label noise caused by automatic labeling
(Clothing1M and Webvision) or human annotation mistakes (Animal10N). In
Webvision, following (Pleiss et al. 2020), we used only the first 50 classes
from Flickr and Google - around 100,000 images. For the NLP classification
tasks, we used tasks from the GLUE benchmark (Wang et al. 2018), including
MNLI, QNLI and QQP.

In our experiments the ensemble contained 5 networks and was trained for 200
epochs (except clothing, on which it was trained for 80 epochs). We used
batch-size of 32, learning rate of 0.01, SGD optimizer with momentum of 0.9
and weight decay of 5e-4, cosine annealing scheduler, and standard
augmentations (horizontal flip, random crops). For cifar10/100, TinyImagenet
and Imagenet100 we used DenseNet with width of 32 and batch normalization
layers. For Clothing1M we used Resnet50 pretrained on Imagenet, while for
Animal10N and Webvision we trained resnet50 from scratch. For elr (Liu et al.
2020) we used their implementation and hyperparameters. For HyperEnsembles we
used different values of learning rate, ranging from 0.01 to 0.1 at equal
distances, using ensembles of 4 networks. For Batchensembles, we used 4
networks of wide-resnet, at the recommended settings. For the NLP
classification tasks we used bert-base-cased (Devlin et al. 2018) with hugging
face (Sanh et al. 2021) implementation as our model, with batch size of 32,
learning rate of 2e-5, max sequence length of 128, and 4 training epochs. We
tested the networks every $\frac{1}{10}$ epoch. Experiments were conducted on
a cluster of GPU type AmpereA10. Each experiment reports the mean and standard
error (ste) results over 3 repetitions.