11institutetext: School of Physics and Center of Excellence in High Energy
Physics $\&$ Astrophysics, Suranaree University of Technology, Nakhon
Ratchasima 30000, Thailand 22institutetext: Frankfurt Institute for Advanced
Studies, Ruth-Moufang-Str. 1, 60438 Frankfurt am Main, Germany
33institutetext: Institut für Theoretische Physik, Goethe Universität
Frankfurt, Max-von-Laue-Strasse 1, D-60438 Frankfurt am Main, Germany
44institutetext: GSI Helmholtzzentrum für Schwerionenforschung GmbH,
Planckstr. 1, 64291 Darmstadt , Germany 55institutetext: John von Neumann-
Institut für Computing, Forschungzentrum Jülich,52425 Jülich, Germany
55email<EMAIL_ADDRESS>
# Influence of centrality definition and detector efficiency on the net-proton
kurtosis

Sukanya Sombun 11 Jan Steinheimer 22 Christoph Herold 11 Ayut Limphirat 11
Yupeng Yan 11 Marcus Bleicher 22334455

###### Abstract

We study the influence of the centrality definition and detector efficiency on
the net-proton kurtosis for minimum bias Au+Au collisions at a beam energy of
$\sqrt{s_{\mathrm{NN}}}=7.7$ GeV by using the UrQMD model. We find that
different ways of defining the centrality lead to different cumulant ratios.
Moreover, we demonstrate that the kurtosis is suppressed for central
collisions when a wider transverse momentum acceptance is used. Finally, the
influence of a detector efficiency on the measured cumulant ratios is
estimated.

###### Keywords:

Centrality definition Cumulant ratios Net-proton number fluctuations.

##  1 Introduction

One aim of heavy ion collisions is to study the phase structure of the strong
interaction or quantum chromodynamics (QCD) which has been widely investigated
both in experiment and theory. QCD based models predict a first-order phase
transition at large $\mu_{B}$ which ends in a QCD critical point (CP) [1]. The
QCD phase structure can be disclosed from the study of event-by-event
fluctuations of conserved quantities which can be defined in the form of
cumulants. Especially cumulants of sthe net-proton number are predicted to be
sensitive to the presence of a QCD phase transition [2, 3] and CP [4, 5, 6,
7]. Higher order cumulants are particularly sensitive to a divergent
correlation length close to the CP. The higher order susceptibilities of
baryon number, electric charge and strangeness have been calculated
theoretically in [8, 9, 10, 11, 12, 13, 14, 15, 16]. Experimentally, event-by-
event fluctuations of higher order cumulants of net-proton, net-pion and net-
kaon number were measured by RHIC [17, 18, 19, 20, 21, 22] and LHC [23, 24,
25]. Hereby, a non-monotonic behaviour of higher order cumulant ratios as a
function of beam energy might disclose critical behaviour. Although critical
fluctuations have been widely studied, there are still difficulties in
understanding the interplay of different effects and their impact on the
measured observables. Some uncertainties that we are going to address in this
paper using the UrQMD model are the centrality determination, the importance
of volume fluctuations, the transverse momentum $(p_{T})$ acceptance, and
efficiency corrections.

###  1.1 The UrQMD model

For the present study, we use the Ultrarelativistic Quantum Molecular Dynamics
(UrQMD) model to study cumulant ratios of the net-proton number distribution.
The UrQMD model is a microscopic transport model which is able to simulate
p+p, p+A and A+A collisions. It is based on the covariant propagation of
constituent quarks and anti-quarks accompanied by mesonic and baryonic degrees
of freedom, binary elastic and inelastic scattering of hadrons, resonance
excitations as well as string dynamics and strangeness exchange reactions [26,
27, 28]. The elementary cross-sections are interpreted geometrically and are
taken from experimental data [29].

##  2 Method

###  2.1 Calculation of cumulants

The fluctuations of the net-proton number distribution can be characterized by
the corresponding cumulants. These are calculated as

$\displaystyle C_{1}$ $\displaystyle=$ $\displaystyle
M=\left\langle\mathrm{N}\right\rangle$ (1) $\displaystyle C_{2}$
$\displaystyle=$
$\displaystyle\sigma^{2}=\left\langle(\delta\mathrm{N})^{2}\right\rangle$ (2)
$\displaystyle C_{3}$ $\displaystyle=$ $\displaystyle
S\sigma^{3}=\left\langle(\delta\mathrm{N})^{3}\right\rangle$ (3)
$\displaystyle C_{4}$ $\displaystyle=$
$\displaystyle\kappa\sigma^{4}=\left\langle(\delta\mathrm{N})^{4}\right\rangle-3\left\langle(\delta\mathrm{N})^{2}\right\rangle^{2}$
(4)

where $(\delta\mathrm{N})=\mathrm{N}-\left\langle\mathrm{N}\right\rangle$, and
$\left\langle\mathrm{N}\right\rangle$ is the event-averaged value of the net-
proton number $\mathrm{N}$. We consider the following ratios of these
cumulants,

$\displaystyle\frac{{{C_{2}}}}{{{C_{1}}}}$ $\displaystyle=$
$\displaystyle\frac{{\left\langle{\delta{N^{2}}}\right\rangle}}{{\left\langle
N\right\rangle}}=\frac{{{\sigma^{2}}}}{M}$ (5)
$\displaystyle\frac{{{C_{3}}}}{{{C_{2}}}}$ $\displaystyle=$
$\displaystyle\frac{{\left\langle{\delta{N^{3}}}\right\rangle}}{{\left\langle{\delta{N^{2}}}\right\rangle}}=S\sigma$
(6) $\displaystyle\frac{{{C_{4}}}}{{{C_{2}}}}$ $\displaystyle=$
$\displaystyle\frac{{\left\langle{\delta{N^{4}}}\right\rangle}}{{\left\langle{\delta{N^{2}}}\right\rangle}}-3\left\langle{\delta{N^{2}}}\right\rangle=\kappa{\sigma^{2}}$
(7)

with combinations of mean (M), variance ($\sigma^{2}$), skewness ($S$) and
kurtosis ($\kappa$) of measured event-by-event fluctuations. For the
statistical error calculations, we use the the delta theorem [30] which states
that the error of cumulant ratios is proportional to a certain power of the
standard deviation as:

$error(\frac{C_{r}}{C_{2}})\propto\frac{\sigma^{r-2}}{\sqrt{n}}~{}.$ (8)

Here, $r$ denotes the order of the cumulant, and $n$ the number of events.

###  2.2 Centrality definition

In heavy-ion experiments, there is no unique definition of centrality. As the
impact parameter ($b$) is experimentally not accessible, observables like the
number of participants ($N_{\mathrm{part}}$) or the number of charged
particles ($N_{\mathrm{charge}}$) are used. These can be related to the impact
parameter e.g. by a Glauber model. Therefore, the centrality is practically
determined by particle multiplicities. In our work, the centrality will be
defined by $N_{\mathrm{part}}$, $N_{\mathrm{charge}}$ or the number of
participants in the projectile, $N_{\mathrm{part-projectile}}$. Calculation of
cumulants as a function of $N_{\mathrm{part}}$ and $N_{\mathrm{charge}}$ is
believed to minimize volume fluctuations which can have an influence on the
extracted ratio of cumulants. We study the dependence of the net-proton number
on the different centrality definitions. For this, we simulate Au+Au collision
at a beam energy of $\sqrt{s_{\mathrm{NN}}}=7.7$ GeV with impact parameters in
the range $0\leq b\leq 15$ fm. We define the following quantities,

  * •

$N_{\rm charge}$: The number of all charged particles with $|\eta|\leq 1$ and
$0.15<p_{T}<2.0$ GeV minus the number of protons and anti-protons in this
specific acceptance range.

  * •

$N_{\rm part}$: 394 minus all spectator protons and neutrons defined by
$|y|>1.5$ and $p_{T}<0.3$ GeV.

  * •

$N_{\rm part-projectile}$: 197 minus all projectile spectator protons and
neutrons defined by $y>1.5$ and $p_{T}<0.3$ GeV.

We determine the distribution of these three different quantities as shown in
figure 1. The three different methods give different distributions. We find
that the participant distribution of $N_{\rm part}$ shows a sharp cutoff at
the maximum number of participants, whereas the distribution of $N_{\rm
charge}$ shows a much smoother drop. The centrality is classified into 10
centrality bins of 0-10$\%$, 10-20$\%$, 20-30$\%$, 30-40$\%$, 40-50$\%$,
50-60$\%$, 60-70$\%$, 70-80$\%$, 80-90$\%$ and 90-100$\%$. We calculate the
cumulants $C_{n}$ for a fixed $N_{\mathrm{part}}$ ($N_{\mathrm{charge}}$,
$N_{\mathrm{part-projectile}}$) and then average those cumulants over all
$N_{\mathrm{part}}$ in a given centrality bin. The ratios of cumulants are
then determined as ratios of averages.

Figure 1: [Color online] Distributions of $N_{charge}$, $N_{part}$ and $N_{\rm
part-projectile}$ in Au+Au collision at a beam energy of
$\sqrt{s_{\mathrm{NN}}}=7.7$ GeV with impact parameter $0\leq b\leq 15$ fm.
Figure from [31].

##  3 Results

###  3.1 Dependence on centrality definition

We first assume that the efficiency of the detector and its acceptance are
perfect (100$\%$) for all particles. Figure 2 shows the result of the net-
proton number kurtosis as a function of centrality for the three centrality
definitions, at mid-rapidity $\left|y\right|<0.5$ and within transverse
momentum $0.4<{p_{T}}<0.8$ GeV. For the most central collisions, we find that
the value of the kurtosis does not depend on the centrality definition. On the
other hand, for mid-central collision, the difference between the values of
$\kappa\sigma^{2}$ becomes larger. Moreover, the centrality defined by
$N_{\mathrm{charge}}$ yields only a mild dependence of the kurtosis on the
centrality because it not subject to large volume fluctuations. Thus, we use
$N_{\mathrm{charge}}$ in what follows to define the centrality to investigate
other effects.

Figure 2: [Color online] The kurtosis of the net-proton number as a function
of centrality which is defined by three different quantities, the number of
charged particles, the number of participants and the number of participants
in the projectile. Figure from [31].

###  3.2 Effect of transverse momentum range

We now study the kurtosis of the net-proton number as a function of centrality
which is defined by $N_{\mathrm{charge}}$ for two different transverse
momentum ranges, namely ($0.4<{p_{T}}<0.8$ GeV) and ($0.4<{p_{T}}<2.0$ GeV).
The result is shown in figure 3. It can be seen that for the most central
collisions, the value of the kurtosis is strongly suppressed for the larger
acceptance range due to baryon number conservation. At mid-central collisions,
the value of the kurtosis is larger for the wider transverse momentum range
which indicates that volume fluctuations affect the extracted kurtosis.

Figure 3: [Color online] The kurtosis of the net-proton number as a function
of centrality (defined by $N_{\rm charge}$) for two different transverse
momentum $p_{T}$ ranges. Figure from [31].

###  3.3 Effect of efficiency

In practice, particle detectors are not ideal systems but suffer from a finite
particle detection efficiency. The efficiency is the number of produced
particles that are recorded in the detector divided by the overall yield, see
[32]. Therefore, detectors measuring particle number fluctuations will never
perform perfectly. We now show how an efficiency below $100\%$ influences the
measurement of cumulants. We begin by studying the effect of efficiency on the
fluctuations. In figure 4, we show the result of the net-proton number
kurtosis in two different $p_{T}$ ranges as a function of centrality at
$100\%$ and $70\%$ $N_{\rm charge}$ efficiency. It is found that the reduced
(realistic) efficiency leads to an overall increase of the kurtosis.

Figure 4: [Color online] The kurtosis of the net-proton number in two
different $p_{T}$ acceptance bins (black squares and red circles) as a
function of centrality with $100\%$ (open symbols) and $70\%$ (full symbols)
$N_{\rm charge}$ efficiency. Figure from [31].

Second, we study the effect of a proton efficiency. The result of the
variance, skewness and kurtosis of the net-proton number as a function of
centrality defined by $70\%$ $N_{\rm charge}$ efficiency is shown in figure 5.
We compare the results for $100\%$ and $75\%$ proton efficiency. The circles
represent calculations with $100\%$ proton efficiency, the squares are $75\%$
proton efficiency. We find that the cumulant ratios of the net-proton number
for the $75\%$ proton efficiency is smaller than for the 100$\%$ proton
efficiency.

Figure 5: [Color online] The result of variance, skewness and kurtosis of the
net-proton number in two different $p_{T}$ acceptance bins for a 70$\%$
$N_{\rm charge}$ efficiency. We compare the result between 100$\%$ (red circle
symbol) and 75$\%$ (black squares symbol) proton efficiency. Figure from [31].

##  4 Conclusions

We have studied the effect of different methods for centrality determination
on the measured net-proton kurtosis. We have found that different centrality
definitions give different results for the kurtosis. Using a centrality
defined by $N_{\rm charge}$ reduces the effect of volume fluctuations.
Moreover, we have studied the effect of two different transverse momentum
ranges accepting net-protons in the measurement. We have seen that the wider
transverse momentum range leads to a strongly suppressed kurtosis at central
collisions. Finally, we have observed the effect of centrality determination,
finding a clear impact on the value of the kurtosis.

##  5 Acknowledgments

The computational resources have been provided by the LOEWE Frankfurt Center
for Scientific Computing (LOEWE-CSC) and the Center for Computer Services at
SUT. This work is supported by the German Academic Exchange Service (DAAD),
HIC for FAIR and the Thailand Research Fund (TRF). SS and AL acknowledge
support from TRF-RGJ (PHD/0185/2558). CH, AL and YY acknowledge support from
Suranaree University of Technology and the Office of the Higher Education
Commission under NRU project of Thailand.

## References

  * [1] M. Stephanov, K. Rajagopal, and E. Shuryak, “Signatures of the tricritical point in qcd,” Physical Review Letters, vol. 81, no. 22, p. 4816, 1998\.
  * [2] J. Randrup, “Spinodal decomposition during the hadronization stage at rhic?,” Physical review letters, vol. 92, no. 12, p. 122301, 2004.
  * [3] C. Sasaki, B. Friman, and K. Redlich, “Density fluctuations in the presence of spinodal instabilities,” Physical review letters, vol. 99, no. 23, p. 232301, 2007.
  * [4] M. Stephanov, “Non-gaussian fluctuations near the qcd critical point,” Physical review letters, vol. 102, no. 3, p. 032301, 2009.
  * [5] C. Herold, M. Bleicher, M. Nahrgang, J. Steinheimer, A. Limphirat, C. Kobdaj, and Y. Yan, “Broadening of the chiral critical region in a hydrodynamically expanding medium,” The European Physical Journal A, vol. 54, no. 2, p. 19, 2018.
  * [6] M. Nahrgang, C. Herold, and M. Bleicher, “Influence of an inhomogeneous and expanding medium on signals of the qcd phase transition,” Nuclear Physics A, vol. 904, pp. 899c–902c, 2013.
  * [7] M. Szymański, M. Bluhm, K. Redlich, and C. Sasaki, “Net-proton number fluctuations in the presence of the qcd critical point,” Journal of Physics G: Nuclear and Particle Physics, vol. 47, no. 4, p. 045102, 2020.
  * [8] M. Cheng, P. Hegde, C. Jung, F. Karsch, O. Kaczmarek, E. Laermann, R. Mawhinney, C. Miao, P. Petreczky, C. Schmidt, et al., “Baryon number, strangeness, and electric charge fluctuations in qcd at high temperature,” Physical Review D, vol. 79, no. 7, p. 074505, 2009.
  * [9] W.-j. Fu, Y.-x. Liu, and Y.-L. Wu, “Fluctuations and correlations of conserved charges in qcd at finite temperature with effective models,” Physical Review D, vol. 81, no. 1, p. 014028, 2010.
  * [10] L. Chen, X. Pan, F. Xiong, L. Li, N. Li, Z. Li, G. Wang, and Y. Wu, “Statistical and dynamical fluctuations in the ratios of higher net-proton cumulants in relativistic heavy-ion collisions,” Journal of Physics G: Nuclear and Particle Physics, vol. 38, no. 11, p. 115004, 2011.
  * [11] F. Karsch and K. Redlich, “Has t c been measured by heavy ion experiments?,” Physical Review D, vol. 84, no. 5, p. 051504, 2011.
  * [12] B.-J. Schaefer and M. Wagner, “Qcd critical region and higher moments for three-flavor models,” Physical Review D, vol. 85, no. 3, p. 034027, 2012\.
  * [13] X. Wang and C. Yang, “Energy and centrality dependence of higher order moments of net-proton distributions in relativistic heavy ion collisions,” Physical Review C, vol. 85, no. 4, p. 044905, 2012.
  * [14] D.-M. Zhou, A. Limphirat, Y.-l. Yan, C. Yun, Y.-p. Yan, X. Cai, L. P. Csernai, and B.-H. Sa, “Higher-moment singularities explored by net-proton nonstatistical fluctuations,” Physical Review C, vol. 85, no. 6, p. 064916, 2012.
  * [15] P. Rau, J. Steinheimer, S. Schramm, and H. Stöcker, “Conserved charge fluctuations in a chiral hadronic model including hadrons and quarks,” Physics Letters B, vol. 733, pp. 176–182, 2014.
  * [16] W. Fan, X. Luo, and H. Zong, “Probing the qcd phase structure with higher order baryon number susceptibilities within the njl model,” Chinese Physics C, vol. 43, no. 3, p. 033103, 2019.
  * [17] M. Aggarwal, Z. Ahammed, A. Alakhverdyants, I. Alekseev, J. Alford, B. Anderson, D. Arkhipkin, G. Averichev, J. Balewski, L. Barnby, et al., “Higher moments of net proton multiplicity distributions at rhic,” Physical review letters, vol. 105, no. 2, p. 022302, 2010.
  * [18] X. Luo, S. Collaboration, et al., “Probing the qcd critical point with higher moments of net-proton multiplicity distributions,” in Journal of Physics: Conference Series, vol. 316, p. 012003, IOP Publishing, 2011.
  * [19] L. Adamczyk, J. Adkins, G. Agakishiev, M. Aggarwal, Z. Ahammed, I. Alekseev, J. Alford, C. Anson, A. Aparin, D. Arkhipkin, et al., “Energy dependence of moments of net-proton multiplicity distributions at rhic,” Physical review letters, vol. 112, no. 3, p. 032302, 2014.
  * [20] L. Adamczyk, J. Adkins, G. Agakishiev, M. Aggarwal, Z. Ahammed, I. Alekseev, J. Alford, C. Anson, A. Aparin, D. Arkhipkin, et al., “Beam energy dependence of moments of the net-charge multiplicity distributions in au+ au collisions at rhic,” Physical review letters, vol. 113, no. 9, p. 092301, 2014.
  * [21] A. Adare, S. Afanasiev, C. Aidala, N. Ajitanand, Y. Akiba, R. Akimoto, H. Al-Bataineh, J. Alexander, H. Al-Ta’ani, A. Angerami, et al., “Measurement of higher cumulants of net-charge multiplicity distributions in au+ au collisions at s n n= 7.7–200 gev,” Physical Review C, vol. 93, no. 1, p. 011901, 2016.
  * [22] J. Xu, “Energy dependence of moments of net-proton, net-kaon, and net-charge multiplicity distributions at star,” in Journal of Physics: Conference Series, vol. 736, p. 012002, IOP Publishing, 2016.
  * [23] B. Abelev, J. Adam, D. Adamová, A. Adare, M. Aggarwal, G. A. Rinella, A. Agocs, A. Agostinelli, S. A. Salazar, Z. Ahammed, et al., “Net-charge fluctuations in pb-pb collisions at s nn= 2.76 tev,” Physical Review Letters, vol. 110, no. 15, p. 152301, 2013.
  * [24] A. Rustamov, A. Collaboration, et al., “Net-baryon fluctuations measured with alice at the cern lhc,” Nuclear Physics A, vol. 967, pp. 453–456, 2017.
  * [25] A. Ohlson, “Measurements of the fluctuations of identified particles in alice at the lhc,” arXiv preprint arXiv:1901.00709, 2019.
  * [26] S. A. Bass, M. Belkacem, M. Bleicher, M. Brandstetter, L. Bravina, C. Ernst, L. Gerland, M. Hofmann, S. Hofmann, J. Konopka, et al., “Microscopic models for ultrarelativistic heavy ion collisions,” Progress in Particle and Nuclear Physics, vol. 41, pp. 255–369, 1998.
  * [27] M. Bleicher, E. Zabrodin, C. Spieles, S. A. Bass, C. Ernst, S. Soff, L. Bravina, M. Belkacem, H. Weber, H. Stöcker, et al., “Relativistic hadron-hadron collisions in the ultra-relativistic quantum molecular dynamics model,” Journal of Physics G: Nuclear and Particle Physics, vol. 25, no. 9, p. 1859, 1999.
  * [28] G. Graef, J. Steinheimer, F. Li, and M. Bleicher, “Deep sub-threshold $\xi$ and $\lambda$ production in nuclear collisions with the urqmd transport model,” Physical Review C, vol. 90, no. 6, p. 064909, 2014.
  * [29] C. Patrignani, D. Weinberg, C. Woody, R. Chivukula, O. Buchmueller, Y. V. Kuyanov, E. Blucher, S. Willocq, A. Höcker, C. Lippmann, et al., “Review of particle physics,” Chin. Phys., vol. 40, p. 100001, 2016.
  * [30] X. Luo, “Error estimation for moment analysis in heavy-ion collision experiment,” Journal of Physics G: Nuclear and Particle Physics, vol. 39, no. 2, p. 025008, 2012.
  * [31] S. Sombun, J. Steinheimer, C. Herold, A. Limphirat, Y. Yan, and M. Bleicher, “Higher order net-proton number cumulants dependence on the centrality definition and other spurious effects,” Journal of Physics G: Nuclear and Particle Physics, vol. 45, no. 2, p. 025101, 2017.
  * [32] B. Abelev, M. Aggarwal, Z. Ahammed, B. Anderson, D. Arkhipkin, G. Averichev, Y. Bai, J. Balewski, O. Barannikova, L. Barnby, et al., “Systematic measurements of identified particle spectra in pp, d+ au, and au+ au collisions at the star detector,” Physical Review C, vol. 79, no. 3, p. 034909, 2009.


# An Investigation of Critical Issues in Bias Mitigation Techniques

Robik Shrestha1 Kushal Kafle2 Christopher Kanan1,3,4
Rochester Institute of Technology1 Adobe Research2 Paige3 Cornell Tech4
{rss9369, kk6055<EMAIL_ADDRESS>
###### Abstract

A critical problem in deep learning is that systems learn inappropriate
biases, resulting in their inability to perform well on minority groups. This
has led to the creation of multiple algorithms that endeavor to mitigate bias.
However, it is not clear how effective these methods are. This is because
study protocols differ among papers, systems are tested on datasets that fail
to test many forms of bias, and systems have access to hidden knowledge or are
tuned specifically to the test set. To address this, we introduce an improved
evaluation protocol, sensible metrics, and a new dataset, which enables us to
ask and answer critical questions about bias mitigation algorithms. We
evaluate seven state-of-the-art algorithms using the same network architecture
and hyperparameter selection policy across three benchmark datasets. We
introduce a new dataset called Biased MNIST that enables assessment of
robustness to multiple bias sources. We use Biased MNIST and a visual question
answering (VQA) benchmark to assess robustness to hidden biases. Rather than
only tuning to the test set distribution, we study robustness across different
tuning distributions, which is critical because for many applications the test
distribution may not be known during development. We find that algorithms
exploit hidden biases, are unable to scale to multiple forms of bias, and are
highly sensitive to the choice of tuning set. Based on our findings, we
implore the community to adopt more rigorous assessment of future bias
mitigation methods. All data, code, and results are publicly
available111https://github.com/erobic/bias-mitigators.

##  1 Introduction

Deep learning systems are trained to minimize their loss on a training
dataset. However, datasets often contain spurious correlations and hidden
biases which result in systems that have low loss on the training data
distribution, but then fail to work appropriately on minority groups because
they exploit and even amplify these spurious correlations [71, 35].

Figure 1: Current bias mitigation systems are tested on simple datasets that
are easy to analyze, but do not offer challenges present in realistic cases.
Addressing this, we propose the Biased MNIST dataset which is easy to analyze,
yet is reflective of real world challenges since it contains multiple sources
of biases. We find that methods fail on Biased MNIST even when all the biases
are explicitly labeled. We also test on GQA-OOD, where the sources of biases
are not very obvious and, thus the methods fail to generalize.

For example, in systems trained to infer hair color on the CelebA dataset
[43], the majority group of non-blond males occurs $50$ times more than the
minority group of blond males, resulting in systems incorrectly predicting
non-blond as hair color for the minority group.

While this is a toy problem, in the real world, hidden minority patterns are
common and failing on them can have dire consequences. Systems designed to aid
human resources, help with medical diagnosis, determine probation, or loan
qualification could be biased against minority groups based on age, gender,
religion, sexual orientation, ethnicity, or race [54, 8, 16, 13, 48]. Systems
can exploit correlated variables even if they are not directly a part of the
input e.g., through inferred zip codes [21], failing to work effectively on
minority groups.

Recently, many methods have been proposed to make neural networks bias
resistant. These methods can be grouped into two types: 1) those that assume
the bias variables e.g., the gender label in CelebA, are explicitly annotated
and can be accessed during training [55, 55, 69, 37] and, 2) those that do not
require explicit access [46, 50]. Assuming explicit access requires extra
annotations in addition to the actual target, and for many tasks it may not be
immediately clear what the bias variables are e.g., biases may only be
discovered years later [51, 50]. Methods that do not assume access to these
bias variables have only recently been proposed [46, 65, 50].

So far, there is no study comparing methods from either group comprehensively.
Often papers fail to compare against recent methods and vary widely in the
protocols, datasets, architectures, and optimizers used. For instance, the
widely used Colored MNIST dataset, where colors and digits are spuriously
correlated with each other, is setup differently across papers. Some use it as
a binary classification task (class 0: digits 0-4, class 1: digits: 5-9) [5,
50], whereas others use a multi-class setting (10 classes) [37, 40]. For
CelebA, [46] uses ResNet-18 whereas [50] uses ResNet-50, but the comparison
was done without taking this architectural change into account. These
discrepancies make it difficult to judge the methods on an even ground.

Methods are typically highly sensitive to hyperparameter choices, and papers
report numbers on systems in which the hyperparameters were tuned using the
test set distribution [18, 50, 64]. In the real world, biases may stem from
multiple factors and may change in different environments, making this setup
unrealistic. Furthermore, tuning on the test distribution can lead to methods
that are right for the wrong reasons. When this is done, systems can perform
well just by exploiting the biases they are supposed to overcome [62, 64], and
they will then fail once deployed because they have not really learned to
solve the task.

In addition, we posit that the commonly used benchmarks are not challenging
enough to test generalization to realistic scenarios. For example CelebA and
Colored MNIST, two of the most widely used benchmarks, contain a single bias
variable to mitigate: gender and color respectively. It is unclear how well
methods would fare in presence of multiple types of bias, e.g., position or
co-occurring objects/patterns, which are commonly present in real-world
datasets. For some tasks it can be impossible to exhaustively enumerate all
bias variables. For example, in visual question answering (VQA), where a
system answers questions about images, biases can stem from: object-context
co-occurrences, visual concept/language correlations, question type/answer
distributions, and more. Annotating all such sources of bias is unrealistic.
Even when the bias variables are explicitly labeled, it is still unclear if
the methods can remain robust to all of the bias sources, since this entails
generalization to a large number of dataset groups e.g., hundred thousand
groups for GQA-OOD [36].

#### We address the above issues via these contributions:

  1. 1.

We describe our new Biased MNIST dataset and corresponding evaluation protocol
for measuring resistance to multiple forms of bias. It measures resistance to
spuriously correlated background/foreground color, texture, co-occurring
distractors, position, and more.

  2. 2.

We compare seven state-of-the-art bias mitigation methods on classification
tasks using Biased MNIST and CelebA, measuring generalization to minority
patterns, scalability to multiple sources of biases, sensitivity to
hyperparameters, etc. We ensure fair comparisons by using the same
architecture, optimizer, and performing grid searches for hyperparameters.

  3. 3.

To go beyond image classification, we measure the performance of these methods
on the biased GQA benchmark for VQA.

  4. 4.

We provide concrete recommendations for future studies, so that comparisons
among algorithms are meaningful and reflective of real-world challenges.

##  2 Problem Statement

To properly study bias mitigation, it is necessary to provide a definition of
biased data and biased behavior in a model. We study bias in supervised
classification i.e., the goal is to learn a function $f:X\rightarrow Y$ which
outputs a categorical target $y\in Y$ given $x\in X$. Each $x$ is itself a
mixture of a signal $s$ that we wish the system to use for inference and bias
$b$ that is spuriously correlated with $y$. Since the spurious correlations
between $y$ and $b$ do not always hold, systems exploiting $b$ to infer $y$
fail to generalize.

We can measure the robustness to such tendencies by intentionally introducing
covariate shift e.g., with a test dataset distribution that differs from
training or a metric that balances performance across groups. For our study,
we use the mean per group accuracy/unbiased accuracy, which weighs all the
groups equally. Furthermore, we focus on the cross-bias setting defined by [6]
where the same set of bias variables are present in both train and test sets.

##  3 Bias Mitigation Strategies

Without bias mitigation mechanisms, standard models (StdM) often use spurious
bias variables for inference, rather than developing invariance to them, which
often results in their inability to perform well on minority patterns [27, 11,
3, 61]. To address this, several bias mitigation mechanisms have been
proposed, and they can be categorized into two groups: 1) methods that access
explicit bias labels during training, and 2) methods that do not assume such
access. We briefly review methods from these categories, with an emphasis on
the methods assessed in our studies.

###  3.1 Explicit Bias Mitigation Methods

Explicit bias mitigation techniques directly access the bias variables:
$b_{expl.}$ during training to develop invariance to them. Based on the way
these variables are utilized during training, we choose five different
explicit methods for our study. We refer to them as explicit methods for
conciseness.

Re-sampling/Re-weighting: These approaches balance out the spurious
correlations. The classical approach is to re-balance the class distribution
by adjusting the sampling probability/ loss weight for majority/minority
samples [14, 26, 41, 72, 20]. This includes synthesizing minority instances
too [14, 26]. Moving beyond class imbalances, REPAIR [40] proposed learning
dynamic weights to mitigate representation bias [39]. However, [55] have shown
promising results by using static weights to upweight minority patterns. We
choose this method due to its simplicity.

Group Upweighting (Up Wt) [55] attempts to mitigate the correlations between
$y$ and $b_{expl.}$ by upweighting the minority patterns. Specifically, each
sample ($x,y$) is assigned to a group: $g=(y,b_{1},b_{2},..,b_{E})$, where $E$
is the total number of variables contained in $b_{expl.}$ and the loss is
scaled by $\frac{1}{N_{g}}$, where $N_{g}$ is the number of instances in group
$g$. Up Wt requires the models to be sufficiently regularized, i.e., be
trained with low learning rates and/or high weight decays to be robust to the
minority groups.

Distributionally Robust Optimization (DRO): DRO [22] minimizes the worst-case
expected loss over potential test distributions. Often, such distributions are
approximated by sampling from a uniform divergence ball around the train
distribution [10, 23, 47]. However, this lacks structured priors about the
potential shifts, and instead hurts generalization [32].

Group DRO (GDRO) [55] provides DRO with the necessary prior that it must
generalize to all groups. Similar to Up Wt, GDRO also uses $y$ and $b_{expl.}$
to create groups and has been shown to work well with sufficiently regularized
models. However, unlike Up Wt, it performs weighted sampling from each group
and has an optimization procedure to minimize the loss over the worst-case
group.

Ensembling Approaches: Ensembling approaches [28, 17, 12] have a two-branch
setup: a) a bias-only branch $f_{b}$ that predicts $y$ from $b$ alone to
identify the bias-prone samples, and b) a de-biased branch $f_{d}(.)$ that is
trained to focus on samples that $f_{b}$ finds difficult so that it learns
richer features that work on difficult samples too. The two branches can be
ensembled in different ways. DRiFt [28] uses product-of-experts [31] and
LearnedMixin [17] extends this through learned weights and entropy constraints
that control $f_{b}$.

Reduction of Unimodal Biases (RUBi) [12] multiplies the outputs from
$f_{d}(.)$ with sigmoided outputs from $f_{b}(.)$, thereby assigning higher
loss weights to samples that cannot be predicted through biases alone. RUBi
was the previous state-of-the-art on VQA-CP [3], a testbed for measuring
robustness to biases in VQA. For the bimodal problem of VQA, the original
implementation focused on linguistic biases, training $f_{b}$ on question
features only. For our studies, we instead train $f_{b}$ on $b_{expl.}$
directly, to control the type of biases captured by $f_{b}$. We assess RUBi
[12] over others since it performed better in the preliminary studies.

Adversarial Debiasing: These techniques impair the ability of the
representation learner to encode biases [69, 1, 52, 25]. Like ensembling
methods, they also employ a two-branch setup, with the representation encoder
in the main branch being penalized if the bias-only branch: $f_{b}()$ is
successful at predicting biases from them [69]. Alternately, $f_{b}()$ may be
trained to predict the class label from the biased features [52, 25], but in
either case, the gradient from $f_{b}()$ is reversed during backpropagation
for debiasing.

Learning Not to Learn (LNL) [37] uses an adversarial setup derived from
minimization of mutual information between representation and bias. In
addition to the gradient reversal, the mutual information formulation
introduces an entropy regularization on the bias predictions.

Invariant Risk Minimization (IRM): The goal of IRM is to extract
representations that are invariant across environments:
$\mathcal{E}=\\{e_{1},e_{2},...e_{E}\\}$, each encoding different spurious
correlations [5, 63, 15]. Such representations enable the same classifier to
be simultaneously optimal over all $\mathcal{E}$. For this, [5] propose to
regularize the gradient norm of a fixed linear classifier. More recent
variants include regularization of variance of risks [38, 67]. However, [53]
have shown that such objectives can fail to recover the invariant features in
practice. Despite this negative result, we still compare against IRM since it
is a promising research direction.

IRMv1 [5] is an efficient approximation of an otherwise computationally
expensive bi-level IRM objective. It consists of a regularization constraint
on the gradient norm with respect to a fixed scalar $\theta_{c}=1.0$:

$\displaystyle\underset{\theta}{min}\sum_{e\in\mathcal{E}_{tr}}l^{e}(\hat{y})+\lambda||\nabla_{\theta_{c}|\theta_{c}=1.0}l^{e}(\theta_{c}.\hat{y})||^{2},$

where, $l^{e}$ is loss on environment $e$, $\hat{y}$ is the logit vector
yielded by the model parameterized by $\theta$ and $\lambda$ balances between
the empirical risk and invariance. In our experiments, we use the previously
defined explicit data groups as the training environments for IRM.

###  3.2 Implicit Bias Mitigation Methods

Since explicit access to bias variables is an undesirable requirement in
practice, some recent methods have proposed to mitigate biases without such
assumptions. We call them implicit methods for conciseness and describe them
below.

Limited Capacity Models: Most implicit methods assume that easy-to-learn
biases can be captured by limiting the capacity of the models [65, 57, 46].
The capacity of such bias-prone models: $f_{b}()$ can be limited by using a
small subset of train instances for a few epochs [65], using fewer model
parameters [57], attaching a classifier to intermediate layers, instead of
using the final representation layers [18], using bias-prone architectures [7]
or amplifying biases [46]. Main network: $f_{d}$ is then debiased by assigning
higher weights to the harder samples, so that it generalizes to samples that
cannot be predicted through biases alone.

Learning From Failure (LFF) [46] amplifies the bias in $f_{b}$ using the
generalized cross entropy loss [70]:

$\displaystyle GCE(s(x;\theta),y)=\frac{1-s_{y}(x;\theta)^{\gamma}}{\gamma},$

where $s_{y}$ is the softmax score for the ground truth class and $\gamma$
determines the degree of bias amplification. Samples with high $f_{b}$ loss
are then assigned higher weights while training $f_{d}$. While $\gamma$ seems
critical, the original paper does not discuss a way to tune it and instead
fixes it to a default value of $\gamma=0.7$. In fact, the paper does not
provide details on model selection at all; however, this is an important
question, which we discuss in Sec. 6.4.

Gradient Starvation Mitigation: Different from the limited capacity methods,
spectral decoupling [50] aims to overcome the issue of gradient starvation
[19], which is the tendency to only rely on statistically dominant features.
This is related to the simplicity bias exploitation, where models exploit the
simplest features despite having access to more predictive features [59, 30],
which are more complex.

Spectral Decoupling (SD) [50] aims to decouple the learning dynamics between
features. The authors show that regularizing the network outputs ($\hat{y}$)
as:

$\displaystyle\frac{\lambda}{2}||\hat{y}-\gamma||_{2}^{2},$

where, $\lambda$ and $\gamma$ are hyperparameters, provably decouples the
learning dynamics, enabling learning of better features.

##  4 Datasets

We use datasets that enable probing existing methods with critical questions
regarding their robustness. We test on datasets with varying scales and types
of biases, allowing us to perform highly controlled studies that analyze
scalability to a large number of hidden groups.

###  4.1 Biased MNIST

Figure 2: Biased MNIST requires the methods to classify the target digit while
remaining invariant to multiple sources of biases.

Existing datasets for assessing bias mitigation methods do not enable analysis
of multiple bias sources, e.g., Colored MNIST only tests for color versus
class bias. To address this, we created the Biased MNIST dataset, which
requires recognizing digits while remaining robust to multiple sources of
biases i.e., other factors which are also correlated with the target variable.
Specifically, the dataset consists of $3\times 3$ grids of cells, where the
target digit is placed at one of the grid cells and is correlated with
multiple bias variables $\\{b_{j}\\}$ including: a) background color, b) digit
color, c) digit position in the grid, d) distracting shapes present in other
cells, e) color of the distracting shapes, f) texture type and g) texture
color (see Fig. 2). Each variable can take one of ten discrete values: each
variable takes the majority/biased value corresponding to the target digit
with a probability of $p_{bias}$, otherwise takes one of the remaining nine
values with uniform probability. For instance, if $p_{bias}=0.7$, then $70\%$
of 1’s are colored as green, $70\%$ of 2’s are colored red and so on. This
applies to all the variables e.g., $70\%$ of 1’s placed on a purple
background, while $70\%$ of 2’s are placed on a green background. The bias
level: $p_{bias}$ can be specified for each variable to control the types and
degrees of biases. For convenience, $p_{bias}$ is set to $0.7$ in the train
set, unless otherwise specified. The test set is unbiased i.e., has
$p_{bias}=0.1$.

###  4.2 CelebA

The CelebA dataset [43] of celebrity faces is widely used to assess bias
mitigation techniques [55, 56, 46, 50]. Following earlier work, it is used for
binary hair color classification (blond or non-blond), which is correlated
with gender. There are two major bias sources: a) class imbalance, with non-
blond occurring 5.7 times more than blond hair color, and b) presence of a
rare group, i.e., blond male celebrities only account for 0.86% of the
training instances.

###  4.3 GQA-OOD

We use the GQA visual question answering dataset [33] to highlight the
challenges of using bias mitigation methods on real-world tasks. It has
multiple sources of biases including imbalances in answer distribution, visual
concept co-occurrences, question word correlations, and question type/answer
distribution. It is unclear how the explicit bias variables should be defined
so that the methods can generalize to all minority groups. GQA-OOD [36]
divides the evaluation and test sets into majority (head) and minority (tail)
groups based on the answer frequency within each ‘local group’ (e.g., colors
of bags), which is a unique combination of ‘global group’ or answer type
(e.g., objects or colors) and the main concept asked in the question (e.g.,
‘bag’, ‘chair’, etc.). The head/tail categorization makes analysis easier;
however, it is unclear how one should specify the explicit biases so that the
models generalize even to the rarest of local groups. Therefore, we explore
multiple ways of defining the explicit bias variable in separate experiments:
a) majority/minority group label (2 groups), b) answer class (1833 groups), c)
global group (115 groups) and d) local group (133328 groups). It is unknown if
bias mitigation methods can scale to hundreds and thousands of groups in GQA,
yet natural tasks require such an ability.

##  5 Network Architecture & Tuning Procedure

For each dataset, we assess all bias mitigation methods with the same neural
network architecture. For CelebA, we use ResNet-18 [29]. For Biased MNIST, we
use a convolutional neural network with four ReLU layers consisting of a max
pooling layer attached after the first convolutional layer. For GQA-OOD, we
employ the UpDn architecture [4], which is widely used for VQA [58, 36, 66].

For each dataset, we use the class label $y$ and the explicit bias variables
$b_{expl.}$ to define explicit groups for Up Wt, GDRO and IRMv1. For instance,
for CelebA, hair color and gender result in four explicit groups while for
Biased MNIST, the number of groups: $|G|$ depends on the number of explicit
bias variables: $|b_{expl.}|$, with $|G|=10^{|b_{expl.}|}$. We will specify
the exact $b_{expl.}$ for each experiment in Sec. 6. For GQA, we use
head/tail, answer class, global and local groups as explicit variables. For
all datasets, RUBi uses $b_{expl.}$ to predict $y$, whereas LNL trains the
adversarial branch to predict $b_{expl.}$ from representations. Of course the
implicit methods: StdM, LFF and SD are invariant to the choice of the explicit
biases during training. Unless otherwise specified, results from Biased MNIST
are averaged across 3 random seeds, but due to computational constraints, we
ran models on CelebA and GQA-OOD only once.

Hyperparameters for each method were chosen using a grid search with unbiased
accuracy on each dataset’s validation set. To make this tractable, we first
ran a grid search for the learning rate over $\\{10^{-3},10^{-4},10^{-5}\\}$
and weight decay over $\\{0.1,10^{-3},10^{-5},0\\}$. After the best values
were chosen, we searched for method-specific hyperparameters. Due to the size
of GQA-OOD, hyperparameter search was performed by training on only 10% of
instances, and then the best selected hyperparameters were used with the full
training dataset. The exact values for the hyperparameters are specified in
the Appendix.

##  6 Questions Posed and Answered

In this section, we probe the existing methods with critical questions
regarding their robustness. For each question, we first describe the empirical
setup to explore the question, and then present the results.

###  6.1 Head-to-Head Comparisons

Question 1: Are there clear winners in a head-to-head comparisons across
datasets?

We first present the mean per group accuracy for all eight methods on all
three datasets in Table. 1 to see if any method does consistently well across
benchmarks. For this, we used class and gender labels as explicit biases for
CelebA. For Biased MNIST, there are multiple ways to define explicit biases,
but for this section, we simply use each of the seven variables as explicit
biases in different runs and average across the runs. We study combinations of
multiple explicit variables in Sec. 6.3. We set $p_{bias}=0.7$ for this
section, and present results across different $p_{bias}$ in the Appendix.
Similarly for GQA, we consider each of the four variables described in Sec.
4.3 as explicit bias in separate runs and present the average.

Table 1: Unbiased accuracies $Acc(\alpha=0)$ on all datasets for all methods. We format the first, second and $\underline{third}$ best results. Methods that do not access explicit biases have gray background. | Methods/
---
Datasets
CelebA |  | Biased MNIST
---
GQA
StdM | 80.3 | 42.0 | 44.8
Up Wt [56] | 87.4 | 30.1 | 30.0
GDRO [55] | 88.5 | 27.2 | 26.4
RUBi [12] | 87.2 | 38.9 | 24.1
LNL [37] | 79.2 | 40.6 | 28.6
IRMv1 [5] | 79.8 | 38.7 | 39.3
LFF [46] | 77.8 | 56.6 | 45.1
SD [50] | 88.6 | 41.3 | 46.9

Results. As shown in Table. 1, no method performs universally well across
datasets; however, the implicit methods LFF and SD obtain high unbiased
accuracies on most datasets. This shows that implicit methods can deal with
multiple bias sources without explicit access. Explicit methods work well on
CelebA but fail on Biased MNIST and GQA. Specifically, Up Wt, GDRO and RUBi
obtain 7-8% improvements over StdM on CelebA, which requires generalization to
only 4 groups. However, all explicit methods perform worse than StdM on Biased
MNIST and GQA, signifying their inability to deal with multiple bias sources.
LNL and IRMv1 were comparable to StdM even on CelebA, demonstrating lack of
generalization even on simple settings. Despite being a simpler method, Up Wt
outperformed GDRO on both Biased MNIST and GQA, but both were worse than StdM.
These results show that implicit methods can outperform explicit methods.

###  6.2 Bias Exploitation

Question 2: Do methods show robustness to both explicit and implicit biases?

In this set of experiments, we compare the resistance to explicit and implicit
biases. We primarily focus on the Biased MNIST dataset, reserving each
individual variable as the explicit bias in separate runs of the explicit
methods, while treating the remaining variables as implicit biases. To ease
analysis, we compute the accuracy gap between the majority and minority groups
i.e., majority/minority difference (MMD). Majority/minority groups are defined
per variable e.g., for foreground color, green 1’s, red 2’s etc are placed in
the majority group and the rest in the minority group and MMD simply computes
the accuracy difference between the two groups for each variable. High MMDs
indicate that the methods rely heavily on spurious patterns favoring the
majority groups and thus fail on the minority groups.

(a) (b)

Figure 3: Boxplots of majority/minority difference (MMD) on Biased MNIST over:
a) bias variables, and b) different methods.

Results. In Fig. 3(a), we present the MMD boxplots for all bias variables,
comparing cases when the label of the variable is either explicitly specified
(explicit bias), or kept hidden (implicit bias) from the methods. Barring
digit position, we observe that the MMD values are higher when the variables
are not explicitly labeled for the methods, indicating that the explicit
methods in general fail to mitigate implicit biases. Fig. 3(b) breaks down
exploitation of explicit and implicit biases for each method. UpWt, GDRO and
RUBi have low MMD values for explicit biases, but high MMD values for implicit
biases, showing that they mitigate the explicit biases to some extent, but are
not robust to the implicit biases. LNL and IRMv1 seem to be equanimously
affected by both explicit and implicit biases, and thus fail to improve upon
the baseline as previously shown in Table 1. LFF has a relatively low range of
MMDs and as shown by the improvements in Table 1, the method outperforms
others on Biased MNIST.

Interestingly, MMD was low for digit position. We hypothesize this is because
CNNs are unable to use position information for inference [42]. To confirm
this, we add CoordConv layers [42] before and after the maxpooling layer in
CNN to enable usage of position information. This resulted in methods
exploiting digit position too, showing larger MMD values of 11.1%-25.6% as
compared to the 2.2%-8.7% without the CoordConv layers. Such inductive biases
affect whether or not methods exploit certain dataset biases, and we discuss
this in Sec. 7.

###  6.3 Scalability of Methods

Figure 4: Unbiased accuracy as a function of number of explicit bias variables. StdM, LFF and SD are invariant to the choice of explicit bias variables. Table 2: Mean of head and tail accuracies on GQA, considering different variables as explicit biases. Methods |  | Head/Tail
---
(2
groups)
| Answer
---
Class
(1833
groups)
| Global
---
Group
(115
groups)
| Local
---
Group
(133328
groups)
StdM | 44.8
Up Wt [56] | 43.3 | 26.0 | 26.4 | 24.2
GDRO [55] | 46.9 | 28.6 | 10.8 | 19.4
RUBi [12] | 44.1 | N/A | 5.6 | 22.6
LNL [37] | 42.9 | N/A | 32.4 | 10.7
IRMv1 [5] | 47.2 | 35.8 | 40.4 | 33.8
LFF [46] | 45.1
SD [50] | 46.9

Question 3: Do methods scale up to multiple types of biases and a large number
of dataset groups?

It is unknown how well the methods scale up to multiple sources of biases and
large number of groups, even when they are explicitly annotated. To study
this, we train the explicit methods with multiple explicit variables for
Biased MNIST and individual variables that lead to hundreds and thousands of
groups for GQA and compare them with the implicit methods. For Biased MNIST,
we first sort the seven total variables in the descending order of MMD
(obtained by StdM) and then conduct a series of experiments. In the first
experiment, the most exploited variable, distractor shape, is used as the
explicit bias. In the second experiment, the two most exploited variables,
distractor shape and texture, are used as explicit biases. This is repeated
until all seven variables are used222The exact order is given in the
Appendix.. Note that conducting the seventh experiment entails annotating each
instance with every possible source of bias. While this may not be realistic
in practice, such a controlled setup will reveal if the explicit methods can
generalize when they have complete information about every bias source.

To test scalability on a natural dataset, we conduct four experiments per
explicit method on GQA-OOD with the explicit bias variables: a) head/tail (2
groups), b) answer class (1833 groups), c) global group (115 groups), and d)
local group (133328 groups). Unlike Biased MNIST, we do not test with
combinations of these variables since the last three variables already entail
generalization to many groups.

Results. We find that implicit methods either improve or are comparable with
StdM, but most explicit methods fail when asked to generalize to multiple bias
variables and a large number of groups, even when the bias variables are
explicitly provided. As shown in Fig. 4, all explicit methods are below StdM
on Biased MNIST. Barring LNL and Up Wt, other explicit methods exhibit
degraded accuracy as the number of explicit bias variables increases. Because
the implicit methods do not rely on the choice of explicit biases, we simply
repeat the same accuracy across x-axis. Among the implicit methods, LFF
obtains the highest improvement, whereas SD is close to StdM.

Results for GQA-OOD are similar, with explicit methods failing to scale up to
a large number of groups, while implicit methods showing some improvements
over StdM. As shown in Table 2, when the number of groups is small, i.e., when
using a head/tail binary indicator as the explicit bias, explicit methods
remain comparable or even outperform StdM, but when the number of groups grow
to hundreds and thousands, they fail. IRMv1 and GDRO obtain the highest
improvements of 2.4% and 2.1% over StdM, respectively, with the binary
head/tail bias, but they show large drops when using answer class, global
group or local group as explicit bias variables. Some drops are extreme, e.g.,
RUBi drops 39% when using global group as the explicit bias variable.

Figure 5: Improvement Over the Standard Model (IOSM) for each group of CelebA.

Results on a simpler setting. We further study bias exploitation on CelebA.
For this, we plot improvement over the standard model ($IOSM$) in Fig. 5,
which is the accuracy gain over the standard model on each dataset group. The
improvements in blond (minority group) incur degradation in non-blond
(majority group). The methods tilt predictions either in the favor of minority
or majority groups, which shows the inability to learn the signal even on
simple settings.

###  6.4 Robustness to Model Selection Criteria

Question 4: Can the methods generalize to the test set without being tuned on
the test distribution? Do they exhibit robustness across a wide range of
hyperparameters?

Assuming access to the test distribution for model selection is unrealistic
and can result in models being right for the wrong reasons [64]. Rather, it is
ideal if the methods can generalize without being tuned on the test
distribution and we study this ability by comparing models selected through
varying tuning distributions. To control the tuning distribution, we define a
generalization of the mean per group accuracy (MPG) metric, that can
interpolate within as well as extrapolate beyond the train and test
distributions:

$\displaystyle
Acc(\alpha)=\frac{\sum_{g=1}^{|G|}p_{g}^{\alpha}Acc_{g}}{\sum_{g=1}^{|G|}p_{g}^{\alpha}}.$

Here, $p_{g}$ denotes the ratio of samples present in group $g$, $|G|$ is the
total number of groups and $\alpha$ is used to control the group prior. When
$\alpha=0$, $Acc(\alpha=0)$ yields the MPG i.e., a balanced distribution where
all groups are weighed equally. When $\alpha=1$, then the weights reflect the
train priors/biases. When $0<\alpha<1$, it interpolates between biased (train)
priors and unbiased group weights. When $\alpha<0$, minority groups are
weighed more and when $\alpha>1$, majority groups are weighed more i.e., it
amplifies the train bias.

Ideally, methods should yield robust models regardless of the tuning
distribution i.e., the value of $\alpha$. To test this ability, we train a set
of candidate models for model selection, with different sets of
hyperparameters. Specifically, we train nine different models with
$\textit{learning rate}\in\\{10^{-3},10^{-4},10^{-5}\\}$ and $\textit{weight
decay}\in\\{10^{-1},10^{-3},10^{-5}\\}$ for CelebA and Biased MNIST and, then
perform model selection by computing $Acc(\alpha)$ at
$\alpha\in\\{-1.0,-0.5,0.0,0.5,1.0,1.5,2.0\\}$ on the validation sets.

Figure 6: Range of minority (blond haired males) and majority (mean over rest
of the groups) test accuracies on CelebA when varying the validation
distribution from $\alpha=-1$ (inverted train bias) to $\alpha=2$ (increased
train bias).

Results. For CelebA, methods generally show large variance on the minority
patterns (blond haired male celebrities), and lower variance on the majority
patterns (mean over rest of the groups), whereas for Biased MNIST, we find
that methods only work for certain set of hyperparameters and show degraded
results on both majority/minority patterns if the hyperparameters change. As
illustrated by the violin plots of CelebA’s unbiased test accuracies in Fig.
6, LNL and IRMv1 have the lowest variance, but neither improves over StdM. Up
Wt, GDRO and RUBi show the largest variances for the minority group,
indicating they are highly sensitive to the choice of tuning distribution. For
LFF, we found high variance for both majority and minority patterns. In fact,
we were unable to replicate the published LFF results, with $\gamma=0.7$
yielding a high accuracy ($86\%$) on the rarest group, but low accuracies on
the rest. After tuning it, we found that $\gamma=0.1$ gave the best unbiased
test accuracy.

Interestingly, for Biased MNIST we found that $\textit{learning rate}=10^{-3}$
and $\textit{weight decay}=10^{-5}$ worked best for all methods. Even though
Up Wt and GDRO are known to generalize to minority groups when using a low
learning rate and high weight decay [56], we did not observe this for Biased
MNIST. We hypothesize that when multiple sources of bias are present, as in
Biased MNIST, methods have multiple ways of predicting the classes, some of
which maybe easier to learn than the others. When the hyperparameters are
suitable to exploit these biases, methods obtain their best accuracies, which
are still lower than StdM.

##  7 Discussion

Our study demonstrates that systems are highly sensitive to the tuning
distribution, that explicit methods cannot handle multiple bias sources, and
that more rigorous analysis is critical for bias mitigation algorithms for
future progress. Based on our results, we argue that the community should
focus on implicit methods, rather than explicit, not only because explicit
methods require additional annotations, but also because they perform worse on
realistic settings.

#### We make the following recommendations:

  1. 1.

Compare against multiple state-of-the-art methods under fair settings.

  2. 2.

Test on datasets that enable control over the number and degrees of biases,
including realistic datasets.

  3. 3.

Analyze generalization to both explicit and implicit sources of bias.

  4. 4.

Be forthcoming about whether test distribution was used for model selection
and compare robustness to tuning distributions that differ from the test.

If these guidelines are adopted, we believe significant progress can be made
so that bias mitigation algorithms can have real-world benefit for deployed
systems.

An interesting observation was that a weaker architecture, CNNs, were able to
ignore position bias, whereas a more powerful architecture, CoordConv,
resorted to exploiting this bias resulting in worse performance. While the
community has largely focused on training procedures for bias mitigation, an
exciting avenue for future work is to incorporate appropriate inductive biases
into the architectures, perhaps endowing them with the ability to choose the
the minimal computational power to do a task so that they are less sensitive
to unwanted biases. This will essentially enable the algorithms to use Occam’s
razor to determine the minimal capabilities required to do a task to reduce
their ability to utilize biases.

We have pointed to issues with the existing bias mitigation approaches, which
alter the loss or use resampling. An orthogonal avenue for attacking bias
mitigation is to use alternative architectures. Neuro-symbolic and graph-based
systems could be created that focus on learning and grounding predictions on
structured concepts, which have shown promising generalization capabilities
[68, 44, 34, 24, 60]. Causality is another relevant line of research, where
the goal is to uncover the underlying causal mechanisms [49, 45, 9, 2].
Discovery and usage of causal concepts is a promising direction for building
robust systems. These areas have not been explicitly studied for their ability
to overcome dataset bias.

#### Acknowledgements.

This work was supported in part by the DARPA/SRI Lifelong Learning Machines
program [HR0011-18-C-0051], AFOSR grant [FA9550-18-1-0121], and NSF award
#1909696. The views and conclusions contained herein are those of the authors
and should not be interpreted as representing the official policies or
endorsements of any sponsor.

## References

  * [1] E. Adeli, Qingyu Zhao, A. Pfefferbaum, E. Sullivan, Li Fei-Fei, Juan Carlos Niebles, and K. Pohl.  Bias-resilient neural network.  ArXiv, abs/1910.03676, 2019.
  * [2] Vedika Agarwal, Rakshith Shetty, and Mario Fritz.  Towards causal VQA: revealing and reducing spurious correlations by invariant and covariant semantic editing.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 9687–9695. IEEE, 2020.
  * [3] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi.  Don’t just assume; look and answer: Overcoming priors for visual question answering.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 4971–4980. IEEE Computer Society, 2018.
  * [4] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.  Bottom-up and top-down attention for image captioning and visual question answering.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 6077–6086. IEEE Computer Society, 2018.
  * [5] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz.  Invariant risk minimization.  arXiv preprint arXiv:1907.02893, 2019.
  * [6] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh.  Learning de-biased representations with biased representations.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 528–539. PMLR, 2020.
  * [7] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh.  Learning de-biased representations with biased representations.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 528–539. PMLR, 2020.
  * [8] Solon Barocas and Andrew D Selbst.  Big data’s disparate impact.  Calif. L. Rev., 104:671, 2016.
  * [9] Alexis Bellot and Mihaela van der Schaar.  Accounting for unobserved confounding in domain generalization.  arXiv preprint arXiv:2007.10653, 2020.
  * [10] Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen.  Robust solutions of optimization problems affected by uncertain probabilities.  Management Science, 59(2):341–357, 2013.
  * [11] Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 4349–4357, 2016.
  * [12] Rémi Cadène, Corentin Dancette, Hedi Ben-younes, Matthieu Cord, and Devi Parikh.  RUBi: Reducing Unimodal Biases for Visual Question Answering.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 839–850, 2019.
  * [13] Robert Challen, Joshua Denny, Martin Pitt, Luke Gompels, Tom Edwards, and Krasimira Tsaneva-Atanasova.  Artificial intelligence, bias and clinical safety.  BMJ Quality & Safety, 28(3):231–237, 2019.
  * [14] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer.  Smote: synthetic minority over-sampling technique.  Journal of artificial intelligence research, 16:321–357, 2002.
  * [15] Yo Joong Choe, Jiyeon Ham, and Kyubyong Park.  An empirical study of invariant risk minimization.  ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning, 2020.
  * [16] Alexandra Chouldechova.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.  Big data, 5(2):153–163, 2017.
  * [17] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer.  Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4069–4082, Hong Kong, China, 2019. Association for Computational Linguistics.
  * [18] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer.  Learning to model and ignore dataset bias with mixed capacity ensembles.  In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3031–3045, Online, 2020. Association for Computational Linguistics.
  * [19] Remi Tachet des Combes, Mohammad Pezeshki, Samira Shabanian, Aaron Courville, and Yoshua Bengio.  On the learning dynamics of deep neural networks.  arXiv preprint arXiv:1809.06848, 2018.
  * [20] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge J. Belongie.  Class-balanced loss based on effective number of samples.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 9268–9277. Computer Vision Foundation / IEEE, 2019.
  * [21] Anupam Datta, Matthew Fredrikson, Gihyuk Ko, Piotr Mardziel, and Shayak Sen.  Use privacy in data-driven systems: Theory and experiments with machine learnt programs.  In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1193–1210, 2017.
  * [22] Erick Delage and Yinyu Ye.  Distributionally robust optimization under moment uncertainty with application to data-driven problems.  Operations research, 58(3):595–612, 2010.
  * [23] John C Duchi, Peter W Glynn, and Hongseok Namkoong.  Statistics of robust optimization: A generalized empirical likelihood approach.  Mathematics of Operations Research, 2021.
  * [24] Difei Gao, Ke Li, Ruiping Wang, Shiguang Shan, and Xilin Chen.  Multi-modal graph neural network for joint reasoning on vision and scene text.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 12743–12753. IEEE, 2020.
  * [25] Gabriel Grand and Yonatan Belinkov.  Adversarial regularization for visual question answering: Strengths, shortcomings, and side effects.  In Proceedings of the Second Workshop on Shortcomings in Vision and Language, pages 1–13, Minneapolis, Minnesota, 2019. Association for Computational Linguistics.
  * [26] Haibo He, Yang Bai, Edwardo A Garcia, and Shutao Li.  Adasyn: Adaptive synthetic sampling approach for imbalanced learning.  In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pages 1322–1328. IEEE, 2008\.
  * [27] Haibo He and Edwardo A Garcia.  Learning from imbalanced data.  IEEE Transactions on knowledge and data engineering, 21(9):1263–1284, 2009.
  * [28] He He, Sheng Zha, and Haohan Wang.  Unlearn dataset bias in natural language inference by fitting the residual.  In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 132–142, Hong Kong, China, 2019. Association for Computational Linguistics.
  * [29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016.
  * [30] Katherine L Hermann and Andrew K Lampinen.  What shapes feature representations? exploring datasets, architectures, and training.  NeurIPS, 2020.
  * [31] Geoffrey E Hinton.  Training products of experts by minimizing contrastive divergence.  Neural computation, 14(8):1771–1800, 2002.
  * [32] Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama.  Does distributionally robust supervised learning give robust classifiers?  In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2034–2042. PMLR, 2018\.
  * [33] Drew A. Hudson and Christopher D. Manning.  GQA: A new dataset for real-world visual reasoning and compositional question answering.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6700–6709. Computer Vision Foundation / IEEE, 2019.
  * [34] Drew A. Hudson and Christopher D. Manning.  Learning by abstraction: The neural state machine.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 5901–5914, 2019.
  * [35] Kushal Kafle, Robik Shrestha, and Christopher Kanan.  Challenges and prospects in vision and language research.  Frontiers in Artificial Intelligence, 2:28, 2019.
  * [36] Corentin Kervadec, Grigory Antipov, Moez Baccouche, and Christian Wolf.  Roses are red, violets are blue… but should vqa expect them to?  arXiv preprint arXiv:2006.05121, 2020.
  * [37] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim.  Learning not to learn: Training deep neural networks with biased data.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 9012–9020. Computer Vision Foundation / IEEE, 2019.
  * [38] David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville.  Out-of-distribution generalization via risk extrapolation (rex).  arXiv preprint arXiv:2003.00688, 2020.
  * [39] Yingwei Li, Yi Li, and Nuno Vasconcelos.  Resound: Towards action recognition without representation bias.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 513–528, 2018.
  * [40] Yi Li and Nuno Vasconcelos.  REPAIR: removing representation bias by dataset resampling.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 9572–9581. Computer Vision Foundation / IEEE, 2019.
  * [41] Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 2999–3007. IEEE Computer Society, 2017.
  * [42] Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski.  An intriguing failing of convolutional neural networks and the coordconv solution.  In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 9628–9639, 2018.
  * [43] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Large-scale celebfaces attributes (celeba) dataset.  Retrieved August, 15:2018, 2018.
  * [44] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu.  The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [45] Nicolai Meinshausen.  Causality from a distributional robustness point of view.  In 2018 IEEE Data Science Workshop (DSW), pages 6–10. IEEE, 2018\.
  * [46] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.  Learning from failure: Training debiased classifier from biased classifier.  In Advances in Neural Information Processing Systems, 2020.
  * [47] Hongseok Namkoong and John C. Duchi.  Stochastic gradient methods for distributionally robust optimization with f-divergences.  In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 2208–2216, 2016.
  * [48] Peter A Noseworthy, Zachi I Attia, LaPrincess C Brewer, Sharonne N Hayes, Xiaoxi Yao, Suraj Kapa, Paul A Friedman, and Francisco Lopez-Jimenez.  Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ecg analysis.  Circulation: Arrhythmia and Electrophysiology, 13(3):e007988, 2020\.
  * [49] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen.  Causal inference by using invariant prediction: identification and confidence intervals.  Journal of the Royal Statistical Society. Series B (Statistical Methodology), pages 947–1012, 2016.
  * [50] Mohammad Pezeshki, Sékou-Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, and Guillaume Lajoie.  Gradient starvation: A learning proclivity in neural networks.  arXiv preprint arXiv:2011.09468, 2020.
  * [51] Oskar Pfungst.  Clever Hans:(the horse of Mr. Von Osten.) a contribution to experimental animal and human psychology.  Holt, Rinehart and Winston, 1911.
  * [52] Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee.  Overcoming language priors in visual question answering with adversarial regularization.  In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 1548–1558, 2018.
  * [53] Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski.  The risks of invariant risk minimization.  arXiv preprint arXiv:2010.05761, 2020.
  * [54] Swati Sachan, Jian-Bo Yang, Dong-Ling Xu, David Eraso Benavides, and Yang Li.  An explainable ai decision-support-system to automate loan underwriting.  Expert Systems with Applications, 144:113100, 2020.
  * [55] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang.  Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.  arXiv preprint arXiv:1911.08731, 2019.
  * [56] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang.  An investigation of why overparameterization exacerbates spurious correlations.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 8346–8356. PMLR, 2020.
  * [57] Victor Sanh, Thomas Wolf, Yonatan Belinkov, and Alexander M Rush.  Learning from others’ mistakes: Avoiding dataset biases without modeling them.  arXiv preprint arXiv:2012.01300, 2020.
  * [58] Ramprasaath Ramasamy Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry P. Heck, Dhruv Batra, and Devi Parikh.  Taking a HINT: leveraging explanations to make vision and language models more grounded.  In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 2591–2600. IEEE, 2019.
  * [59] Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli.  The pitfalls of simplicity bias in neural networks.  arXiv preprint arXiv:2006.07710, 2020.
  * [60] Jiaxin Shi, Hanwang Zhang, and Juanzi Li.  Explainable and explicit visual reasoning over scene graphs.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 8376–8384. Computer Vision Foundation / IEEE, 2019.
  * [61] Robik Shrestha, Kushal Kafle, and Christopher Kanan.  Answer them all! toward universal visual question answering models.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 10472–10481. Computer Vision Foundation / IEEE, 2019.
  * [62] Robik Shrestha, Kushal Kafle, and Christopher Kanan.  A negative case analysis of visual grounding methods for VQA.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8172–8181, Online, July 2020. Association for Computational Linguistics.
  * [63] Damien Teney, Ehsan Abbasnejad, and Anton van den Hengel.  Unshuffling data for improved generalization.  arXiv preprint arXiv:2002.11894, 2020.
  * [64] Damien Teney, Kushal Kafle, Robik Shrestha, Ehsan Abbasnejad, Christopher Kanan, and Anton van den Hengel.  On the value of out-of-distribution testing: An example of goodhart’s law.  2020\.
  * [65] Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych.  Towards debiasing NLU models from unknown biases.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7597–7610, Online, 2020. Association for Computational Linguistics.
  * [66] Jialin Wu, Liyan Chen, and Raymond J Mooney.  Improving vqa and its explanations by comparing competing explanations.  arXiv preprint arXiv:2006.15631, 2020.
  * [67] Chuanlong Xie, Fei Chen, Yue Liu, and Zhenguo Li.  Risk variance penalization: From distributional robustness to causality.  arXiv preprint arXiv:2006.07544, 2020.
  * [68] Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum.  Neural-symbolic VQA: disentangling reasoning from vision and language understanding.  In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 1039–1050, 2018.
  * [69] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell.  Mitigating unwanted biases with adversarial learning.  In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340, 2018.
  * [70] Zhilu Zhang and Mert R. Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 8792–8802, 2018.
  * [71] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang.  Men also like shopping: Reducing gender bias amplification using corpus-level constraints.  In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989, Copenhagen, Denmark, 2017. Association for Computational Linguistics.
  * [72] Yang Zou, Zhiding Yu, BVK Vijaya Kumar, and Jinsong Wang.  Unsupervised domain adaptation for semantic segmentation via class-balanced self-training.  In Proceedings of the European conference on computer vision (ECCV), pages 289–305, 2018.

##  A Appendix

Figure A7: Unbiased accuracy on Biased MNIST as a function $p_{bias}$.

Figure A8: Unbiased test accuracy (mean per group) as a function of learning
rate and weight decay (using single random seed).

###  A.1 Additional Results

We now provide additional results for all datasets, that provide further
details for the results presented in Sec. 6.

Biased MNIST. First, we show the unbiased accuracy as a function of $p_{bias}$
in Fig. A7. None of the methods exhibit robustness across the range of
$p_{bias}$ values, with all methods showing drops in the unbiased accuracy as
$p_{bias}$ increases from 0.5 to 0.9. Second, in Fig. A8, we show the unbiased
accuracies as functions of learning rate and weight decay, when $p_{bias}$ is
set to 0.7. The figure on the left shows unbiased accuracies as a function of
learning rate with a fixed weight decay of $10^{-5}$ and the figure on the
right shows unbiased accuracies as a function of weight decay, with learning
rate set to $10^{-3}$. Barring Up Wt, all methods show highest unbiased
accuracies at learning rate = $10^{-3}$ and weight decay = $10^{-5}$, and show
large drops with other values, indicating high sensitivities to
hyperparameters.

Third, in Table. A4, we compare the explicit methods with each other, when
considering each of the seven variables as explicit bias variables
($b_{expl.}$) in separate experiments. For the explicit methods, variables
apart from $b_{expl.}$ act as implicit biases, whereas implicit methods are
not affected by the choice of the explicit variables. As previously discussed
in Sec. 6.2, Up Wt, GDRO and RUBi show higher majority/minority difference for
$b_{expl.}$ as compared to other variables, and all explicit methods are below
StdM in terms of unbiased accuracies. Finally, in Table. A5, we show the per
variable majority/minority accuracies when considering combinations of
variables as explicit biases. As discussed in Sec. 6.3, explicit methods
barring Up Wt fail when multiple variables are specified as explicit bias
variables. The results have been averaged across three runs for both tables.
We provide a ten samples for each digit in Fig. A10.

Table A3: Per group accuracies on CelebA. Groups/ Methods | Male | Non-Male |  Unbiased Accuracy
---|---|---|---
| Non-
---
Blond
Blond |  | Non-
---
Blond
Blond
StdM | 99.3 | 42.8 | 95.9 | 83.3 | 80.3
Up Wt | 89.3 | 77.8 | 87.4 | 95.0 | 87.4
GDRO | 91.2 | 80.0 | 90.9 | 92.1 | 88.5
RUBi | 96.6 | 68.9 | 94.3 | 89.1 | 87.2
LNL | 99.1 | 39.4 | 95.4 | 82.8 | 79.2
IRMv1 | 99.1 | 40.6 | 95.9 | 83.6 | 79.8
LFF | 96.0 | 46.1 | 90.4 | 80.0 | 77.8
SD | 96.4 | 73.3 | 88.0 | 96.6 | 88.6

CelebA. We show accuracy for each group of CelebA in Table. A3. SD and GDRO
obtain the highest accuracies. As discussed previously, we observe trade-offs
between blond and non-blond classes with the improvements in the rare blond
class incurring degradations in the non-blond class.

GQA-OOD. GQA-OOD [36] defines the tail accuracy (Acc-tail) metric which is
computed on the samples of the tail of the answer class distribution.
Specifically, an answer class $a_{i}$ is considered to be a tail class for a
local group if:

$\displaystyle|a_{i}|\leq(1+\beta)\mu(a),$

where, $|a_{i}|$ is the number of instances for answer $a_{i}$ in the given
group, $\mu(a)$ is the mean number of answers in the group and $\beta$ can be
used to control the tail size. In Fig. A9, we plot the tail accuracies at
different tail sizes, considering different explicit bias variables for the
explicit methods. For implicit methods: StdM, LFF and SD, same tail accuracies
are repeated on all four charts since they are not affected by the choice of
explicit variables during training. Explicit methods fail when the explicit
variables entail generalization to large number of groups, whereas implicit
methods are close to or above StdM.

###  A.2 Hyperparameters and Other Details

We select hyperparameters based on the best unbiased validation set accuracy
on each dataset, which is reflective of the unbiased test distribution. For
all datasets and methods, we first perform a grid search over the learning
rates $\in$ {1e-3, 1e-4, 1e-5} and weight decays $\in$ {0, 0.1, 1e-3, 1e-5},
and then tune the method-specific hyperparameters. For Biased MNIST, the
hyperparameters were selected using single run, considering ‘distractor shape’
as the explicit bias variable for explicit methods. For CelebA, they were
selected based on the unbiased accuracy/mean per group on the validation set
and for GQA-OOD, they were selected based on the best mean head/tail accuracy
when setting $\beta=0.2$ (the default value in the original paper).

Next we specify the ranges considered for method-specific hyperparameters. For
GDRO, we search the group weight step size between $\\{0.001,0.01,0.1\\}$. For
LNL, we perform a grid search over gradient reversal weights
$\lambda_{adv}\in\\{-1.0,-0.1,-0.01\\}$ and entropy loss weights
$\lambda_{ent}\in\\{1.0,0.1,0.01,0\\}$. For IRMv1, we search for
$\lambda_{grad}$ values from {1, 10, 100, 1000, 10000}. The original
implementation of IRMv1 samples from all environments in a single mini-batch
during training. While this is feasible for small scale problems with few
environments e.g., four environments/explicit groups for CelebA, it is
computationally infeasible for Biased MNIST and GQA, where the number of
environments is larger than the batch size itself. So, for Biased MNIST and
GQA, we sample from 16 randomly selected groups or environments within each
mini-batch during training. This implies that our implementation of IRMv1
samples uniformly from all environments.

Even though the original paper proposing LFF uses a fixed $\gamma=0.7$ for all
datasets, we search over $\gamma\in\\{0.1,0.3,0.5,0.7,0.9\\}$ and find that
LFF is indeed sensitive to $\gamma$. The default value: $\gamma=0.7$ is not
the optimal value for all cases. For SD, we consider
$\gamma,\lambda\in\\{10^{-3},10^{-2},0.1,1.0,10.0,100.0\\}$. For CelebA, we
use the class-specific $\gamma$ values specified in the paper and only tune
$\lambda$. The procedure for obtaining class-specific hyperparameters when
there are large number of classes e.g., in GQA however remains unclear for SD.
The complete specification of hyperparameters is provided in Table. A6.

Table A4: Majority (Maj.) and Minority (Min.) group accuracies for each
variable in Biased MNIST when using one of the seven variables as explicit
bias for the explicit methods.

Variables/ Methods |  |  | Background
---
Color
| Digit
---
Color
| Digit
---
Position
| Distractor
---
Shape
| Distractor
---
Color
Texture |  | Texture
---
Color
| Unbiased
---
Accuracy
Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min.
StdM | 42.0 | 70.3 | 39.1 | 74.6 | 38.7 | 48.3 | 41.6 | 69.2 | 39.2 | 68.9 | 39.4 | 69.4 | 39.1 | 65.2 | 39.7
$b_{expl.}$ = Background Color
Up Wt | 39.0 | 42.0 | 38.9 | 72.7 | 35.4 | 43.3 | 38.7 | 71.2 | 35.6 | 70.3 | 35.8 | 67.6 | 35.9 | 65.4 | 36.2
GDRO | 32.8 | 52.2 | 30.8 | 66.8 | 29.2 | 37.0 | 32.5 | 62.4 | 29.6 | 64.1 | 29.6 | 64.5 | 29.3 | 62.6 | 29.6
RUBi | 47.9 | 51.9 | 47.7 | 78.8 | 44.6 | 52.4 | 47.6 | 77.4 | 44.8 | 77.8 | 44.9 | 76.5 | 44.8 | 72.9 | 45.2
LNL | 44.0 | 55.8 | 43.5 | 59.7 | 43.0 | 46.8 | 44.5 | 57.5 | 43.3 | 57.6 | 43.3 | 56.6 | 43.3 | 53.2 | 43.7
IRMv1 | 38.0 | 66.0 | 35.0 | 70.5 | 34.5 | 41.2 | 37.8 | 68.1 | 34.8 | 68.3 | 34.9 | 64.5 | 35.1 | 65.6 | 35.0
$b_{expl.}$ = Digit Color
Up Wt | 19.4 | 48.6 | 16.1 | 22.8 | 19.0 | 21.5 | 19.2 | 52.0 | 15.7 | 47.7 | 16.3 | 50.5 | 15.7 | 44.9 | 16.5
GDRO | 22.6 | 50.1 | 19.6 | 45.7 | 20.1 | 25.7 | 22.3 | 52.1 | 19.3 | 52.3 | 19.4 | 52.0 | 19.2 | 51.5 | 19.4
RUBi | 24.0 | 54.5 | 20.6 | 27.9 | 23.6 | 25.0 | 23.9 | 57.3 | 20.2 | 55.4 | 20.6 | 55.4 | 20.3 | 52.6 | 20.7
LNL | 40.2 | 53.7 | 39.5 | 57.9 | 39.0 | 46.2 | 40.3 | 56.1 | 39.2 | 55.1 | 39.4 | 56.1 | 39.2 | 50.9 | 39.8
IRMv1 | 40.9 | 66.8 | 38.4 | 72.3 | 37.8 | 43.8 | 41.0 | 71.1 | 37.9 | 68.9 | 38.3 | 67.8 | 38.2 | 66.5 | 38.4
$b_{expl.}$ = Digit Position
Up Wt | 21.4 | 46.7 | 18.6 | 50.4 | 18.2 | 21.9 | 21.4 | 53.4 | 17.8 | 50.4 | 18.3 | 49.0 | 18.2 | 45.8 | 18.6
GDRO | 26.8 | 54.9 | 23.9 | 55.4 | 23.9 | 27.6 | 27.0 | 58.2 | 23.5 | 54.9 | 24.0 | 54.5 | 23.9 | 51.6 | 24.3
RUBi | 26.6 | 54.9 | 23.5 | 59.1 | 23.0 | 28.3 | 26.5 | 56.8 | 23.2 | 56.9 | 23.4 | 56.0 | 23.2 | 53.5 | 23.6
LNL | 41.4 | 51.8 | 40.9 | 56.6 | 40.4 | 46.4 | 41.5 | 54.4 | 40.6 | 53.7 | 40.7 | 52.7 | 40.7 | 49.4 | 41.1
IRMv1 | 38.1 | 67.8 | 35.0 | 70.8 | 34.7 | 42.9 | 37.8 | 69.3 | 34.8 | 68.0 | 35.1 | 67.0 | 34.9 | 66.0 | 35.1
$b_{expl.}$ = Distractor Shape
Up Wt | 21.5 | 52.0 | 18.2 | 56.0 | 17.8 | 24.2 | 21.4 | 25.6 | 21.2 | 53.0 | 18.2 | 54.0 | 17.9 | 51.9 | 18.2
GDRO | 25.2 | 54.1 | 22.2 | 56.9 | 21.9 | 27.9 | 25.2 | 48.9 | 22.8 | 56.0 | 22.1 | 56.0 | 21.9 | 52.3 | 22.4
RUBi | 34.5 | 66.9 | 31.0 | 70.9 | 30.5 | 35.2 | 34.5 | 40.4 | 33.9 | 67.7 | 31.0 | 67.5 | 30.7 | 64.6 | 31.1
LNL | 44.1 | 55.8 | 43.1 | 59.6 | 42.6 | 50.5 | 43.7 | 57.9 | 42.8 | 57.4 | 42.9 | 58.8 | 42.7 | 55.0 | 43.1
IRMv1 | 38.3 | 67.0 | 35.5 | 70.5 | 35.1 | 41.5 | 38.3 | 69.3 | 35.2 | 65.9 | 35.7 | 67.9 | 35.2 | 64.6 | 35.7
$b_{expl.}$ = Distractor Color
Up Wt | 40.9 | 70.1 | 38.0 | 70.8 | 38.0 | 46.6 | 40.7 | 73.0 | 37.7 | 44.8 | 40.9 | 73.0 | 37.6 | 68.5 | 38.2
GDRO | 28.1 | 58.6 | 24.8 | 57.4 | 24.9 | 30.4 | 28.0 | 62.1 | 24.4 | 49.1 | 25.9 | 59.1 | 24.6 | 55.8 | 25.1
RUBi | 49.5 | 77.3 | 46.6 | 80.1 | 46.3 | 54.5 | 49.1 | 78.3 | 46.4 | 53.8 | 49.2 | 77.6 | 46.4 | 76.4 | 46.6
LNL | 38.9 | 52.4 | 38.2 | 55.9 | 37.8 | 46.0 | 38.9 | 56.6 | 37.7 | 52.8 | 38.2 | 56.0 | 37.7 | 51.5 | 38.3
IRMv1 | 39.7 | 66.1 | 36.8 | 72.3 | 36.1 | 45.1 | 39.2 | 68.8 | 36.5 | 69.1 | 36.6 | 66.4 | 36.6 | 64.8 | 36.9
$b_{expl.}$ = Texture
Up Wt | 29.3 | 60.8 | 25.9 | 66.1 | 25.4 | 32.9 | 29.1 | 64.6 | 25.5 | 62.3 | 25.9 | 32.9 | 29.1 | 60.7 | 25.9
GDRO | 24.5 | 54.9 | 21.3 | 58.0 | 21.0 | 27.3 | 24.4 | 56.8 | 21.1 | 54.8 | 21.4 | 42.7 | 22.6 | 51.6 | 21.6
RUBi | 43.1 | 73.7 | 39.8 | 78.5 | 39.3 | 44.9 | 43.1 | 74.2 | 39.7 | 74.8 | 39.8 | 48.1 | 42.7 | 71.7 | 40.0
LNL | 41.7 | 55.9 | 40.8 | 59.1 | 40.5 | 48.4 | 41.7 | 56.3 | 40.8 | 56.5 | 40.8 | 57.1 | 40.6 | 53.7 | 41.0
IRMv1 | 37.5 | 64.4 | 34.6 | 67.1 | 34.3 | 39.5 | 37.4 | 68.0 | 34.2 | 65.8 | 34.5 | 66.2 | 34.3 | 62.5 | 34.8
$b_{expl.}$ = Texture Color
Up Wt | 39.5 | 71.6 | 36.1 | 71.4 | 36.2 | 43.7 | 39.3 | 70.3 | 36.3 | 67.6 | 36.7 | 69.4 | 36.2 | 42.4 | 39.4
GDRO | 30.4 | 59.5 | 27.4 | 60.7 | 27.2 | 33.5 | 30.3 | 61.6 | 27.1 | 60.9 | 27.3 | 61.9 | 26.9 | 45.1 | 28.9
RUBi | 46.8 | 74.0 | 43.8 | 76.6 | 43.5 | 48.9 | 46.6 | 73.8 | 43.8 | 74.0 | 43.9 | 75.7 | 43.5 | 49.2 | 46.6
LNL | 34.3 | 49.2 | 33.4 | 47.9 | 33.5 | 41.8 | 34.2 | 49.9 | 33.3 | 47.8 | 33.6 | 48.6 | 33.4 | 43.3 | 34.0
IRMv1 | 38.5 | 66.1 | 35.7 | 71.0 | 35.2 | 43.7 | 38.3 | 69.2 | 35.4 | 67.6 | 35.7 | 68.2 | 35.4 | 63.6 | 36.0
Implicit Methods
LFF | 56.6 | 71.2 | 55.4 | 82.6 | 54.2 | 63.8 | 56.3 | 77.0 | 54.8 | 74.6 | 55.1 | 76.2 | 54.8 | 75.4 | 54.9
SD | 41.3 | 69.5 | 38.3 | 71.2 | 38.1 | 46.3 | 40.9 | 72.2 | 38.0 | 72.1 | 38.1 | 71.1 | 38.0 | 70.5 | 38.2

Table A5: Method accuracies with increasing number of explicit bias variables.

Variables/ Methods |  |  | Background
---
Color
| Digit
---
Color
| Digit
---
Position
| Distractor
---
Shape
| Distractor
---
Color
| Texture
---
Pattern
| Texture
---
Color
| Unbiased
---
Accuracy
Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min. | Maj. | Min.
StdM | 42.0 | 70.3 | 39.1 | 74.6 | 38.7 | 48.3 | 41.6 | 69.2 | 39.2 | 68.9 | 39.4 | 69.4 | 39.1 | 65.2 | 39.7
$b_{expl.}$ = Distractor Shape
Up Wt | 21.5 | 52.0 | 18.2 | 56.0 | 17.8 | 24.2 | 21.4 | 25.6 | 21.2 | 53.0 | 18.2 | 54.0 | 17.9 | 51.9 | 18.2
GDRO | 25.2 | 54.1 | 22.2 | 56.9 | 21.9 | 27.9 | 25.2 | 48.9 | 22.8 | 56.0 | 22.1 | 56.0 | 21.9 | 52.3 | 22.4
RUBi | 34.5 | 66.9 | 31.0 | 70.9 | 30.5 | 35.2 | 34.5 | 40.4 | 33.9 | 67.7 | 31.0 | 67.5 | 30.7 | 64.6 | 31.1
LNL | 44.1 | 55.8 | 43.1 | 59.6 | 42.6 | 50.5 | 43.7 | 57.9 | 42.8 | 57.4 | 42.9 | 58.8 | 42.7 | 55.0 | 43.1
IRMv1 | 38.3 | 67.0 | 35.5 | 70.5 | 35.1 | 41.5 | 38.3 | 69.3 | 35.2 | 65.9 | 35.7 | 67.9 | 35.2 | 64.6 | 35.7
$b_{expl.}$ = Distractor Shape+Texture Pattern
Up Wt | 16.1 | 49.0 | 12.6 | 51.6 | 12.3 | 18.3 | 16.0 | 20.6 | 15.8 | 50.7 | 12.5 | 18.5 | 16.0 | 45.3 | 12.9
GDRO | 17.4 | 43.7 | 14.6 | 44.3 | 14.6 | 20.1 | 17.3 | 42.5 | 14.8 | 46.4 | 14.4 | 42.6 | 14.6 | 41.8 | 14.8
RUBi | 27.9 | 64.3 | 24.0 | 70.9 | 23.3 | 32.1 | 27.6 | 36.5 | 27.1 | 65.9 | 24.0 | 34.1 | 27.4 | 60.6 | 24.3
LNL | 10.0 | 14.7 | 10.5 | 11.0 | 10.9 | 11.3 | 10.9 | 12.3 | 10.8 | 11.8 | 10.9 | 12.9 | 10.7 | 11.7 | 10.9
IRMv1 | 34.3 | 64.4 | 31.2 | 68.5 | 30.8 | 38.5 | 34.2 | 63.9 | 31.3 | 63.7 | 31.4 | 61.3 | 31.4 | 62.3 | 31.4
$b_{expl.}$ = Distractor Shape+Texture Pattern+Digit Color
Up Wt | 12.6 | 47.7 | 8.7 | 20.1 | 11.8 | 13.5 | 12.5 | 22.9 | 11.5 | 45.8 | 9.0 | 22.7 | 11.4 | 42.3 | 9.2
GDRO | 18.4 | 44.1 | 15.6 | 42.6 | 15.8 | 20.0 | 18.3 | 44.8 | 15.5 | 45.7 | 15.6 | 44.2 | 15.5 | 43.6 | 15.7
RUBi | 11.0 | 48.9 | 6.8 | 12.9 | 10.9 | 11.8 | 11.0 | 11.1 | 11.1 | 46.8 | 7.2 | 11.0 | 11.1 | 41.0 | 7.6
LNL | 10.2 | 15.2 | 10.6 | 11.5 | 11.1 | 11.7 | 11.0 | 11.4 | 11.1 | 12.2 | 11.0 | 18.4 | 10.2 | 15.3 | 10.6
IRMv1 | 24.9 | 57.0 | 21.4 | 53.8 | 21.8 | 27.6 | 24.8 | 52.4 | 21.9 | 56.0 | 21.7 | 50.7 | 22.0 | 55.3 | 21.6
$b_{expl.}$ = Distractor Shape+Texture Pattern+Digit Color+Background Color
Up Wt | 30.1 | 54.5 | 27.6 | 59.3 | 27.1 | 35.0 | 29.8 | 56.0 | 27.4 | 64.0 | 26.7 | 56.3 | 27.3 | 63.4 | 26.5
GDRO | 19.9 | 48.2 | 17.0 | 45.3 | 17.4 | 22.8 | 19.9 | 50.9 | 16.7 | 47.4 | 17.2 | 46.7 | 17.1 | 43.8 | 17.5
RUBi | 10.0 | 10.3 | 9.9 | 11.9 | 9.7 | 10.0 | 10.0 | 9.2 | 10.0 | 59.4 | 4.6 | 10.4 | 9.9 | 48.0 | 5.6
LNL | 10.0 | 10.0 | 11.5 | 10.6 | 11.4 | 11.2 | 11.4 | 11.9 | 11.3 | 13.0 | 11.2 | 11.3 | 11.4 | 10.6 | 11.4
IRMv1 | 25.6 | 52.9 | 22.7 | 58.1 | 22.1 | 30.1 | 25.2 | 52.3 | 22.7 | 57.5 | 22.3 | 52.4 | 22.6 | 56.0 | 22.3
$b_{expl.}$ = Distractor Shape+Texture Pattern+Digit Color+Background
Color+Texture Color
Up Wt | 38.8 | 65.3 | 36.0 | 71.4 | 35.3 | 42.3 | 38.5 | 68.2 | 35.6 | 71.0 | 35.4 | 67.1 | 35.6 | 61.5 | 36.3
GDRO | 21.9 | 49.9 | 19.1 | 47.2 | 19.4 | 25.1 | 21.8 | 53.2 | 18.7 | 50.8 | 19.1 | 51.6 | 18.7 | 45.6 | 19.5
RUBi | 9.8 | 10.2 | 9.7 | 9.6 | 9.8 | 9.8 | 9.7 | 8.8 | 9.8 | 88.2 | 1.2 | 9.7 | 9.7 | 9.3 | 9.8
LNL | 10.0 | 10.0 | 11.5 | 10.6 | 11.4 | 11.2 | 11.4 | 11.9 | 11.3 | 13.0 | 11.2 | 11.3 | 11.4 | 10.6 | 11.4
IRMv1 | 28.4 | 58.7 | 25.0 | 61.5 | 24.7 | 32.2 | 28.0 | 56.6 | 25.2 | 59.3 | 25.1 | 57.8 | 25.0 | 53.5 | 25.6
$b_{expl.}$ = Distractor Shape+Texture Pattern+Digit Color+Background
Color+Texture Color+Distractor Color
Up Wt | 43.2 | 71.0 | 40.4 | 75.3 | 39.9 | 47.3 | 43.1 | 72.1 | 40.2 | 69.7 | 40.6 | 72.8 | 40.0 | 68.5 | 40.6
GDRO | 19.7 | 47.3 | 16.7 | 42.2 | 17.3 | 22.2 | 19.5 | 50.7 | 16.3 | 48.5 | 16.7 | 48.1 | 16.5 | 44.4 | 17.0
RUBi | 9.9 | 9.0 | 10.1 | 9.8 | 10.0 | 11.4 | 9.8 | 10.2 | 10.0 | 12.3 | 9.7 | 10.9 | 9.9 | 9.2 | 10.1
LNL | 10.4 | 22.4 | 9.6 | 12.1 | 10.8 | 13.5 | 10.6 | 11.3 | 10.9 | 11.5 | 10.8 | 30.8 | 8.6 | 24.8 | 9.3
IRMv1 | 24.8 | 52.6 | 21.8 | 58.3 | 21.2 | 26.0 | 24.8 | 57.4 | 21.3 | 53.2 | 21.9 | 53.3 | 21.6 | 49.5 | 22.1
|  $b_{expl.}$ = Distractor Shape+Texture Pattern+Digit Color+Background
Color+Texture Color+Distractor Color
---
+Digit Position
Up Wt | 44.0 | 70.5 | 41.4 | 73.9 | 41.0 | 48.8 | 43.9 | 71.8 | 41.2 | 71.8 | 41.4 | 71.0 | 41.2 | 68.2 | 41.6
GDRO | 16.6 | 42.6 | 13.7 | 34.6 | 14.7 | 19.0 | 16.4 | 46.8 | 13.3 | 46.5 | 13.4 | 44.1 | 13.4 | 38.9 | 14.1
RUBi | 10.0 | 11.9 | 10.6 | 10.2 | 10.8 | 10.7 | 10.7 | 10.2 | 10.8 | 12.6 | 10.5 | 11.2 | 10.7 | 9.7 | 10.8
LNL | 10.0 | 10.1 | 11.5 | 10.5 | 11.4 | 11.1 | 11.4 | 12.2 | 11.2 | 12.8 | 11.2 | 11.4 | 11.3 | 10.7 | 11.4
IRMv1 | 21.8 | 47.7 | 19.0 | 51.6 | 18.6 | 24.6 | 21.6 | 49.8 | 18.7 | 52.2 | 18.6 | 49.3 | 18.7 | 46.5 | 19.1
Implicit Methods
LFF | 56.6 | 71.2 | 55.4 | 82.6 | 54.2 | 63.8 | 56.3 | 77.0 | 54.8 | 74.6 | 55.1 | 76.2 | 54.8 | 75.4 | 54.9
SD | 41.3 | 69.5 | 38.3 | 71.2 | 38.1 | 46.3 | 40.9 | 72.2 | 38.0 | 72.1 | 38.1 | 71.1 | 38.0 | 70.5 | 38.2

(a) Explicit Bias = Head vs Tail (# groups = 2) (b) Explicit Bias = Global
Group (# groups = 115) (c) Explicit Bias = Answer Class (# groups = 1833) (d)
Explicit Bias = Local Group (# groups = 133,328)

Figure A9: Tail accuracies on GQA-OOD, when considering 4 different explicit
biases. Table A6: Hyperparameters for all datasets for all the comparison
methods selected using unbiased accuracies of the validation sets after
performing grid search over hyperparameters.

Methods/Datasets | Parameters | CelebA | Biased MNIST | GQA-OOD
---|---|---|---|---
Common to all methods unless specified otherwise | Optimizer | SGD | Adam | Adam
Batch Size | 128 | 128 | 128
Epochs | 50 | 30 | 30
Standard Model (StdM) | Learning Rate | $10^{-3}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | 0 | $10^{-5}$ | 0
Group Upweighting (Up Wt) [56] | Learning Rate | $10^{-5}$ | $10^{-3}$ | $10^{-3}$
Weight Decay | 0.1 | $10^{-5}$ | 0
Group DRO (GDRO) [55] | Learning Rate | $10^{-5}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | 0.1 | $10^{-5}$ | 0
Step Size | 0.01 | $10^{-3}$ | 0.01
Reduction of Unimodal Biases (RUBi) [12] | Learning Rate | $10^{-4}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | $10^{-5}$ | $10^{-5}$ | 0
Adversarial Regularization (LNL) [37] | Learning Rate | $10^{-4}$ | $10^{-3}$ | $10^{-3}$
Weight Decay | $10^{-4}$ | $10^{-5}$ | 0
Gradient Reversal Weight ($\lambda_{grad.}$) | -0.1 | -0.1 | -0.1
Entropy Loss Weight ($\lambda_{ent.}$) | 0 | 0.01 | 0.01
Invariant Risk Minimization (IRMv1) [5] | Learning Rate | $10^{-4}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | 0 | $10^{-5}$ | 0
Gradient Regularization Weight ($\lambda_{grad.}$) | 1.0 | 0.01 | 0.01
| Number of environments per mini-batch | 4 | 16 | 16
Learning From Failure (LFF) [46] | Optimizer | Adam | Adam | Adam
Learning Rate | $10^{-4}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | 0 | $10^{-5}$ | 0
| Amplification Factor ($\gamma$) | 0.1 | 0.5 | 0.7
Spectral Decoupling (SD) [50] | Learning Rate | $10^{-4}$ | $10^{-3}$ | $10^{-4}$
Weight Decay | $10^{-5}$ | $10^{-5}$ | 0
Output Decay ($\lambda$) |  |  $\lambda_{0}$ = 10
---
$\lambda_{1}$ = 10
$10^{-3}$ | $10^{-3}$
Output Shift ($\gamma$) |  |  $\gamma_{0}$ = 0.44
---
$\gamma_{1}$ = 2.5
$10^{-3}$ | $10^{-3}$

Figure A10: Biased MNIST requires the methods to classify the target digit
while remaining invariant to biases.