Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

ISWC’22: The 21st International Semantic Web Conference, October 23–27, 2022,
Hangzhou, China

[orcid=0000-0001-8402-2853<EMAIL_ADDRESS>]

[orcid=0000-0003-4805-1467<EMAIL_ADDRESS>]

[orcid=0000-0003-1047-6339<EMAIL_ADDRESS>]

# SignalKG: Towards Reasoning about the Underlying Causes of Sensor
Observations

Anj Simmons  Rajesh Vasa  Antonio Giardina  Applied Artificial Intelligence
Institute, Deakin University, Geelong, Australia

(2022)

###### Abstract

This paper demonstrates our vision for knowledge graphs that assist machines
to reason about the cause of signals observed by sensors. We show how the
approach allows for constructing smarter surveillance systems that reason
about the most likely cause (e.g., an attacker breaking a window) of a signal
rather than acting directly on the received signal without consideration for
how it was produced.

###### keywords:

knowledge graph ontology sensor surveillance

##  1 Introduction

Standards such as the Semantic Sensor Network (SSN/SOSA) ontology [1] allow
capturing the semantics of sensor observations, and emerging standards for
smart buildings [2, 3] and smart cities allow capturing the semantics of the
environments in which sensors operate. However, reasoning about the underlying
cause of sensor observations requires not only knowledge of the sensors and
their environment, but also an understanding of the signals they detect and
the possible causes of these signals. For example, inferring that the sound of
breaking glass may be due to a broken window requires knowledge of the fact
that glass windows produce a distinct sound when broken and that this sound
propagates as sound waves through the air to a sensor such as a human ear or
microphone. This paper proposes a signal knowledge graph (SignalKG) to support
machines to reason about the underlying cause of sensor observations. Sensors
and their environment are represented using existing standards, and then
linked to SignalKG. To reason about the cause of sensor observations, we
automatically generate a Bayesian network based on information in the
knowledge graph, and use this to infer the posterior probability of causes
given the sensor data.

Figure 1: High level overview of SignalKG ontology

Figure 1 presents an ontology describing the high level concepts. A category
of entities (e.g., humans) perform actions (e.g., walking) that act on a type
of object or place (e.g., in hallways), which in turn create a type of signal
(e.g., the sound of footsteps). A sensor observes a particular type of signal,
which usually reduces in strength with distance and can be distorted by
surrounding objects (e.g., a wall between a sound source and the receiver
attenuates the sound signal). The sensor may implement a classifier to detect
the presence of the signal (e.g., an acoustic detector may make use of a
binary machine learning classifier to detect if the sound of footsteps is
present or not). A formal RDF/OWL specification of the SignalKG ontology is
available online111SignalKG ontology: https://signalkg.visualmodel.org/skg as
well as an interactive demonstration222Interactive demonstration available at:
https://signalkg.visualmodel.org of how SignalKG can be applied and used to
reason about the underlying causes of sensor observations.

##  2 Related Work

Past research has utilised ontologies for the purpose of threat and situation
awareness using description logic [4] and rule based reasoning [5]. However,
these approaches assume that threats can be classified into classes according
to deterministic rules, whereas in reality threats may be probabilistic in
nature and there may be multiple possible explanations for a given set of
observations. To overcome this limitation, reasoning methods have been
proposed that combine deterministic rules with a Bayesian network for
reasoning probabilistically [6]. However, the structure of the Bayesian
network and associated probabilities need to be manually specified. In
contrast, we seek to express this knowledge in a reusable and extensible form.
There have been attempts to extend OWL to support representing/reasoning about
uncertainty [7]. However, general approaches do not directly specify concepts
for reasoning about sensor signals.

##  3 Demonstration

Figure 2: Knowledge graph of audio signals constructed using SignalKG ontology
showing link to building rooms/assets and sensors

Figure 2 demonstrates how concepts in the SignalKG ontology can be applied to
construct a knowledge graph of audio signals. Attackers and employees are
entities that are capable of producing an action, such as breaking a window or
dropping a tray of glasses in a dining room. Actions create signals, in this
case, the sound of breaking glass or the sound of dropped glass. The building
asset/room type on/in which an action can occur is represented using the
RealEstateCore ontology [2]. To group similar signals together, we use the
Simple Knowledge Organization System (SKOS) [8] ‘broader’ property to
represent a signal hierarchy, for example, the sound of breaking glass is
similar to the sound of dropped glass and thus are grouped under the same
category. The knowledge graph also includes information about how signals
propagate, for example, that sound intensity reduces with distance (according
to an inverse square law) and is attenuated by walls. To model the case in
which signals are classified on the sensing device, we allow for describing
different classification models, such as YAMNet, an audio classifier that
recognises 521 classes of sounds (e.g., glass).

Figure 3: Bayesian network before (left) and after (right) conditioning on
sensor observations. Green bars indicate the probability of each node value.
(Key: a=action, s=signal emitted due to action, r=received signal strength at
location of sensor, d=detected signal after classification). Visualisation
created with jsbayes-viz [9].

Sensor observations can be linked to the signal knowledge graph via the
property (signal) that the sensor observes. Our interactive demonstration
includes a simulator to generate sample sensor observations (represented using
the SSN/SOSA ontology) for hypothetical scenarios that could occur. The goal
is to infer what took place given only the sensor observations, knowledge of
the building and sensor placement, and our understanding of possible
underlying causes of observed signals (specified using SignalKG).

To support reasoning probabilistically about causes of sensor observations,
the knowledge graph also includes probabilities, such as the prior probability
that an entity will be present, and the probability that an entity (if
present) will perform an action. As our goal is to infer a cause given
observations, it lends itself to Bayesian reasoning. Rather than manually
specifying a Bayesian network for a particular scenario, we automatically
generate one based on the information in the knowledge graph. Encoding all the
information needed to reason about signals in the knowledge graph itself helps
facilite reuse and extension.

Nodes in the Bayesian network are generated for each entity, action at a
location, signal emitted/received, and sensor. While our simple example
results in a 1:1 mapping (shown in Figure 3), in more complex scenarios there
may be vastly more nodes due to all the possible permutations of entity,
action, location, signal and sensor. If multiple types of signals are present
(audio, vision, social, etc.) then these all appear as part of the same
generated Bayesian network, e.g., an attacker breaking a window will create
both audio (sound of breaking glass) and vision (suspicious behaviour in a
video feed) based signals. Prior probabilities for entities and actions need
to be specified in the knowledge graph. The probability that a signal emitted
by an action will be detected by a sensor is calculated based on the distance
from the location of the action to the location of the sensor, how the signal
intensity reduces with distance (e.g., the knowledge graph may specify an
inverse square law for sound signals), any barriers between the source and the
sensor that may attenuate/block the signal (e.g., the knowledge graph may
specify sound is attenuated by walls), and the sensitivity of the classifier
used by the sensor to detect presence of a signal.

Once we have generated a Bayesian network, we can condition it on sensor
observations to infer the posterior probability of the underlying cause. For
the demonstration, we estimate the posterior probability via likelihood weight
sampling, implemented by [9], drawing 20,000 samples (the number of samples to
draw is a trade-off between accuracy and computation time). In the example,
prior to conditioning on observations, there is a 50% chance of an attacker
being present. After conditioning on the observation that the microphone has
detected the sound of glass, the posterior probability of an attacker
increases to 97%.

##  4 Next Steps

Even for the simple example of detecting building intrusions, the space of
possible causes is large (e.g., an attacker could impersonate an employee then
ask someone to let them in, or suspicious sounds could be due to a movie
playing in the background). Furthermore, signals are not independent (as
assumed by our preliminary prototype), but rather occur in sequences (e.g.,
the sound of footsteps, followed by a weapon detected in video footage,
followed by a scream) that could help more reliably distinguish between
possible causes. Also, more realistic models of signal propagation are needed,
which may require continuous probability distributions rather than a
conditional probability table over a discrete set of values as in our example
Bayesian network. To support efficient reasoning about these more complex
scenarios, we plan to explore generation of probabilistic programs in place of
the discrete Bayesian networks used in this paper.

A practical barrier to uptake of our approach is the need to specify prior
probabilities for each action that an entity can perform. Ordinary behaviour
could potentially be learned from data. However, modelling prior probabilities
of actions intruders perform is more difficult, as attacks are rare events
(limited data to learn from) and an adversary will adjust their actions to
avoid detection. In future work, we plan to include the goals of the intruder
as part of the knowledge graph, then use a game theoretic approach to
determine probable actions they will take rather than manually specifying
prior probabilities for each action.

###### Acknowledgements.

This research was funded by National Intelligence Postdoctoral Grant
NIPG-2021-006.

## References

  * Haller et al. [2018] A. Haller, K. Janowicz, S. J. Cox, M. Lefrançois, K. Taylor, D. Le Phuoc, J. Lieberman, R. García-Castro, R. Atkinson, C. Stadler,  The modular SSN ontology: A joint W3C and OGC standard specifying the semantics of sensors, observations, sampling, and actuation,  Semantic Web 10 (2018) 9–32. doi:10.3233/SW-180320.
  * Hammar et al. [2019] K. Hammar, E. O. Wallin, P. Karlberg, D. Hälleberg,  The RealEstateCore Ontology,  in: The Semantic Web – ISWC 2019, 2019, pp. 130–145. doi:10.1007/978-3-030-30796-7_9.
  * Rasmussen et al. [2021] M. H. Rasmussen, M. Lefrançois, G. F. Schneider, P. Pauwels,  BOT: the building topology ontology of the W3C linked building data group,  Semantic Web 12 (2021) 143–161. doi:10.3233/SW-200385.
  * Roy and Guyard [2012] J. Roy, A. B. Guyard,  Supporting threat analysis through description logic reasoning,  in: 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support, IEEE, 2012, pp. 308–315. doi:10.1109/CogSIMA.2012.6188401.
  * Pai et al. [2017] F. P. Pai, L. J. Yang, Y. C. Chung,  Multi-layer ontology based information fusion for situation awareness,  Applied Intelligence 46 (2017) 285–307. doi:10.1007/s10489-016-0834-7.
  * Yao et al. [2022] H. Yao, C. Han, F. Xu,  Reasoning Methods of Unmanned Underwater Vehicle Situation Awareness Based on Ontology and Bayesian Network,  Complexity 2022 (2022) 7143974. doi:10.1155/2022/7143974.
  * Carvalho et al. [2017] R. N. Carvalho, K. B. Laskey, P. C. Costa,  PR-OWL – a language for defining probabilistic ontologies,  International Journal of Approximate Reasoning 91 (2017) 56–79. doi:10.1016/j.ijar.2017.08.011.
  * Miles and Bechhofer [2009] A. Miles, S. Bechhofer,  SKOS simple knowledge organization system reference,  W3C recommendation (2009). URL: https://www.w3.org/TR/skos-reference/.
  * Vang [2016] J. Vang, jsbayes-viz, 2016. URL: https://github.com/vangj/jsbayes-viz/.


# Enhancing Intrinsic Features for Debiasing via Investigating
Class-Discerning Common Attributes in Bias-Contrastive Pair

Jeonghoon Park*1, Chaeyeon Chung*1, Juyoung Lee2, Jaegul Choo1
1Korea Advanced Institute of Science and Technology, South Korea, 2Kakao
Brain, South Korea.
1{jeonghoon_park, cy_chung<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

In the image classification task, deep neural networks frequently rely on bias
attributes that are spuriously correlated with a target class in the presence
of dataset bias, resulting in degraded performance when applied to data
without bias attributes. The task of debiasing aims to compel classifiers to
learn intrinsic attributes that inherently define a target class rather than
focusing on bias attributes. While recent approaches mainly focus on
emphasizing the learning of data samples without bias attributes (i.e., bias-
conflicting samples) compared to samples with bias attributes (i.e., bias-
aligned samples), they fall short of directly guiding models where to focus
for learning intrinsic features. To address this limitation, this paper
proposes a method that provides the model with explicit spatial guidance that
indicates the region of intrinsic features. We first identify the intrinsic
features by investigating the class-discerning common features between a bias-
aligned (BA) sample and a bias-conflicting (BC) sample (i.e., bias-contrastive
pair). Next, we enhance the intrinsic features in the BA sample that are
relatively under-exploited for prediction compared to the BC sample. To
construct the bias-contrastive pair without using bias information, we
introduce a bias-negative score that distinguishes BC samples from BA samples
employing a biased model. The experiments demonstrate that our method achieves
state-of-the-art performance on synthetic and real-world datasets with various
levels of bias severity.

††* indicates equal contribution.

##  1 Introduction

Deep neural networks in image classification [21, 5, 20, 26] are known to be
vulnerable to the dataset bias [23], which refers to a spurious correlation
between the target classes and the peripheral attributes. Basically, image
classification aims to learn intrinsic attributes — the visual features that
inherently define a target class — that generally appear across the samples in
the class. However, when the dataset bias exists in the training data, the
models tend to use the frequently appearing peripheral attribute (_i.e_., bias
attribute) to predict the class unintentionally. For instance, if airplanes in
the training images are mostly in the sky, a model can heavily rely on the sky
to predict an image as an airplane class due to its high correlation with the
airplane class. This indicates that the model is biased towards the bias
attribute (_e.g_., sky) rather than focusing on intrinsic features (_e.g_.,
the shape of wings or the body) when making decisions. As a result, even
though the biased model achieves high accuracy on the samples including bias
attributes (_e.g_., airplanes in the sky), termed as bias-aligned (BA)
samples, it may fail to accurately predict samples devoid of such bias
attributes (_e.g_., airplanes on the runway), referred to as bias-conflicting
(BC) samples.

In this regard, debiasing aims to encourage the model to focus on intrinsic
attributes rather than bias attributes when dataset bias exists. One
straightforward approach is utilizing prior knowledge regarding bias (_e.g_.,
labels for bias attribute) to inform the model which attributes to focus on or
not to focus on [9, 25, 2, 22]. However, acquiring such bias information is
often infeasible in real-world scenarios. Therefore, recent studies [15, 14,
13, 12, 8] have proposed debiasing methods that do not require bias
information. They identify and emphasize BC samples during the training using
an additional biased classifier that mainly learns the bias attributes.
However, such a training strategy fails to directly indicate where the model
should focus to learn the intrinsic features.

To address this issue, we present a debiasing approach that explicitly informs
the model of the region of the intrinsic features during the training while
not using bias labels. While the intrinsic features in the unbiased dataset
can simply be identified in generally appearing features in the training
samples, generally appearing features in the biased dataset inevitably include
bias features. Therefore, we identify the intrinsic features in the biased
dataset by investigating the common features between a BA and a BC sample
(i.e., a bias-contrastive pair). Here, the common features also need to be
class-discerning since the common features might include irrelevant
environmental features. For example, in the above scenario, the common feature
between an airplane in the sky (BA sample) and an airplane on the runway (BC
sample) might include the features of wings, the body, and trees. In this
case, the intrinsic features are the shape of the wings and the body that can
distinguish the airplane class from the others.

Specifically, we introduce an intrinsic feature enhancement (IE) weight that
identifies the spatial regions of intrinsic features commonly appearing in a
bias-contrastive pair. We leverage an auxiliary sample in addition to the
original input to construct the bias-contrastive pair. Since the majority of
the original input from training samples are BA samples, we mainly adopt the
BC samples as the auxiliary sample. To achieve this without bias information,
we present a bias-negative (BN) score that identifies BC samples by employing
a classification loss of a biased model. Our IE weight investigates common
features in the bias-contrastive pair and identifies the class-discerning
features among the common features. Within the identified intrinsic features,
we enhance the features that are relatively under-exploited in the BA samples
compared to the BC samples. In this way, we can explicitly provide our model
with spatial guidance for intrinsic attributes while not using bias labels.

We verify the effectiveness of our method on both synthetic and real-world
datasets with various levels of bias severity. Furthermore, the in-depth
analysis demonstrates that our method successfully guides the model to make
predictions based on the intrinsic features.

##  2 Related work

Debiasing with bias information.  Previous approaches [9, 22, 18, 25, 4, 2]
utilize bias labels or predefined bias types to encourage the model to learn
intrinsic attributes for debiasing. Kim _et al_. [9], Tartaglione _et al_.
[22], and Sagawa _et al_. [18] employ bias labels to encourage the model not
to learn specific bias features. Wang _et al_. [25] and Bahng _et al_. [2]
predefine the bias type (e.g., color, texture, etc.) and utilize such prior
knowledge to supervise models to be robust against such predefined bias type.
However, obtaining bias information requires additional cost, which is often
infeasible in the real world.

Debiasing without bias information.  Recent studies [3, 7, 15, 12, 13, 14, 8,
1, 28, 16, 11] propose debiasing strategies that do not require bias
information. Nam _et al_. [15] present an approach that encourages the model
to concentrate on BC samples during the training process considering that the
bias attributes are easier to learn than intrinsic attributes. Instead of
using bias information, they additionally train a biased model that mainly
learns bias attributes and regard the samples that are not easily trained by
the biased model as BC samples. Lee _et al_. [13] reveal that BC samples serve
as noisy samples when training the additional biased model and propose a
method to eliminate such BC samples using multiple biased models. Liu _et al_.
[14] regard the samples misclassified by the model trained with empirical risk
minimization as BC samples and emphasize them during training of a debiased
model. Also, MaskTune [1] expects the model to learn intrinsic features by
fine-tuning the model with the data whose already-explored area is masked out
using Grad-CAM [19].

Another stream of approaches [10, 12, 8] synthesize samples having similar
characteristics with BC samples and employ them to train a debiased model. Kim
_et al_. [10] synthesize images without bias attributes leveraging an image-
to-image translation model [17]. Lee _et al_. [12] and Hwang _et al_. [8]
augment BC samples in the feature space by employing the disentangled
representations and mixup [27], respectively. A recent pair-wise debiasing
method $\mathcal{X}^{2}$-model [28] encourages the model to retain intra-class
compactness using samples generated via feature-level interpolation between BC
and BA samples. However, such approaches lack explicit supervision about which
features to focus on to learn intrinsic features. To address this issue, we
present a debiasing method that provides spatial guidance to encourage a model
to learn intrinsic features during the training while not using bias labels.
We design our model architecture using bias-contrastive pairs referring to the
previous studies [8, 28].

Figure 1: Overview of our method. We provide explicit spatial guidance
$g(\mathbf{z})$ for a debiased model $f_{d}$, which is described with
$f_{d}^{\text{emb}}$ and $f_{d}^{\text{cls}}$, to learn intrinsic features. To
achieve this, we leverage a bias-contrastive pair, $\mathbf{x}$ and
$\mathbf{x}^{\text{BN}}$ from the same target class $y$. $g(\mathbf{z})$
highlights intrinsic features that are relatively under-exploited in
$\mathbf{z}$ compared to $\mathbf{z}^{\text{BN}}$, calculated by common
feature score $c$ and relative-exploitation score $r$. Here, we mainly adopt
BC samples from $\mathcal{D}^{\text{BN}}_{\text{cand}}$ to construct
$\mathcal{D}^{\text{BN}}$, where we sample $\mathbf{x}^{\text{BN}}$.
$\mathcal{D}^{\text{BN}}$ is updated every iteration using the BN score $S$,
which is also updated every iteration. At the inference, we only use $f_{d}$
in the gray-colored area.

##  3 Methodology

###  3.1 Overview

As shown in Fig. 1, our framework consists of a biased model $f_{b}$ that
focuses on bias attributes and a debiased model $f_{d}$ that learns debiased
representations. We use BiasEnsemble (BE) [13] as a backbone, where $f_{b}$ is
trained with bias-amplified dataset $\mathcal{D}^{\text{A}}$ which mainly
consists of BA samples, while $f_{d}$ concentrates on the samples that $f_{b}$
fails to learn. Our method provides $f_{d}$ with spatial guidance for
intrinsic features using a bias-contrastive pair: an input $\mathbf{x}$ and an
auxiliary input $\mathbf{x}^{\text{BN}}$. We denote the auxiliary input
$\mathbf{x}^{\text{BN}}$ as a bias-negative (BN) sample because we primarily
adopt samples devoid of bias attributes. We sample an image $\mathbf{x}$ from
the original training data $\mathcal{D}$, and $\mathbf{x}^{\text{BN}}$ from a
BN dataset $\mathcal{D}^{\text{BN}}$ which mainly consists of BC samples.
$\mathcal{D}^{\text{BN}}$ is updated every iteration to mainly include BC
samples using the BN score $S$ that employs $f_{b}$ to identify BC samples.
The BN score is also updated every iteration. Given the intermediate features
$\mathbf{z}$ and $\mathbf{z}^{\text{BN}}$, we first extract the common
features between the bias-contrastive pair ($c(\mathbf{z})$ in Fig. 1). Also,
we identify the class-discerning features that are relatively under-exploited
in $\mathbf{z}$ compared to $\mathbf{z}^{\text{BN}}$ ($r(\mathbf{z})$ in Fig.
1). Next, we calculate the IE weight that indicates relatively under-exploited
intrinsic features in $\mathbf{z}$ based on $c(\mathbf{z})$ and
$r(\mathbf{z})$ ($\text{IE}(\mathbf{z})$ in Fig. 1). Finally, we obtain the
guidance $g(\mathbf{z})$ that emphasizes the region of intrinsic feature in
$\mathbf{z}$ during the training. At the inference, we utilize $f_{d}$ without
$\mathbf{x}^{\text{BN}}$, as in a gray-colored area of Fig. 1.

###  3.2 Constructing bias-negative dataset

We construct a BN dataset $\mathcal{D}^{\text{BN}}$, where we sample
$\mathbf{x}^{\text{BN}}$ during the training. As the majority of the training
dataset is BA samples, we aim to mainly adopt BC samples as
$\mathbf{x}^{\text{BN}}$ to construct bias-contrastive pairs. To achieve this,
we first construct $\mathcal{D}^{\text{BN}}_{\text{cand}}$, a candidate
dataset for $\mathcal{D}^{\text{BN}}$, that contains roughly identified BC
samples. During the training, we dynamically update $\mathcal{D}^{\text{BN}}$
every iteration to mainly adopt BC samples from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ using our newly proposed BN score.

Constructing candidate dataset $\mathcal{D}^{\text{BN}}_{\text{cand}}$.  To
roughly identify BC samples in $\mathcal{D}$, we filter out easily learned BA
samples from $\mathcal{D}$ using multiple biased models, following BE [13].
Since the bias features are easier to learn than the intrinsic features [15],
each biased model is trained only for a few iterations so that BC samples can
be distinguished from the easily learned BA samples. Inspired by JTT [14], we
regard the samples that are incorrectly predicted by the majority of the
biased models as BC samples. Finally, we construct
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ with the roughly identified BC
samples.

Adopting BC samples with BN score.  We introduce a BN score to update
$\mathcal{D}^{\text{BN}}$ to primarily exploit BC samples as
$\mathbf{x}^{\text{BN}}$ from $\mathcal{D}^{\text{BN}}_{\text{cand}}$ during
training $f_{d}$. Considering the unavailability of bias labels, the BN score
employs $f_{b}$ to further exclude BA samples from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$. As training proceeds, $f_{b}$ is
overfitted to the bias attributes in $\mathcal{D}^{\text{A}}$ and outputs a
high probability on the ground-truth label for the samples that have similar
bias features with samples in $\mathcal{D}^{\text{A}}$. This indicates that
samples whose $f_{b}$ loss decreases as training proceeds are likely to have
bias attributes learned from $\mathcal{D}^{\text{A}}$. Such samples disturb
the extraction of intrinsic features when selected as
$\mathbf{x}^{\text{BN}}$. To validate this, we investigate the samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ whose $f_{b}$ loss at the later stage
of training (50K-th iteration) decreases compared to the early stage of
training (1K-th iteration). The result shows that 95.63% of them are BA
samples. We use the BFFHQ dataset [10] with a bias severity of 1% for the
analysis. Further details of the dataset are described in Sec. 4.1.

In this regard, we design a BN score to exclude the samples with decreasing
$f_{b}$ loss from the $\mathcal{D}^{\text{BN}}_{\text{cand}}$ to construct
$\mathcal{D}^{\text{BN}}$ by tracking the $f_{b}$ loss during training
$f_{d}$. First, the $f_{b}$ loss of $\mathbf{x}$ at the $t$-th iteration is
calculated as follows:

$l_{t}(\mathbf{x})=\alpha_{l}\cdot\mathcal{L}_{\text{CE}}(f_{b}(\mathbf{x}),y)+(1-\alpha_{l})\cdot
l_{t-1}(\mathbf{x}),$ (1)

where $\mathcal{L}_{\text{CE}}(f_{b}(\mathbf{x}),y)$ indicates the cross-
entropy (CE) loss of $\mathbf{x}$ on its ground-truth label $y$ and
$\alpha_{l}$ is a hyperparameter for the exponential moving average (EMA). We
employ EMA to enable a stable tracking of the classification losses. Note that
$l_{t}$ is updated only for the samples in a mini-batch at the $t$-th
iteration.

The BN score tracks $l_{t}(\mathbf{x})$ compared to the loss recorded at the
early stage of training. The BN score at the $t$-th iteration is formulated as
follows:

$s_{t}(\mathbf{x})=\alpha_{s}\cdot(l_{t}(\mathbf{x})-l_{\text{ref}}(\mathbf{x}))+(1-\alpha_{s})\cdot
s_{t-1}(\mathbf{x}),$ (2)

where $\alpha_{s}$ is a hyperparameter for the EMA and
$l_{\text{ref}}(\mathbf{x})$ denotes the reference loss of $\mathbf{x}$ that
is first recorded after a few iterations of training. We exploit EMA to
stabilize the tracking. Note that we update $s_{t}$ only for the samples in a
mini-batch at the $t$-th iteration. The negative value of $s_{t}(\mathbf{x})$
indicates that the loss of $\mathbf{x}$ decreased compared to the early stage
of training, which means that the sample is likely to contain bias attributes.

Updating $\mathcal{D}^{\text{BN}}$ with BN score.  At every iteration, we
update $\mathcal{D}^{\text{BN}}$ to exclude the newly detected BA samples
whose BN score $s_{t}(\mathbf{x})$ is smaller than zero as follows:

$\mathcal{D}^{\text{BN}}_{t}=\\{\mathbf{x}\mid
s_{t}(\mathbf{x})>0,\mathbf{x}\sim\mathcal{D}^{\text{BN}}_{\text{cand}}\\},$
(3)

where $\mathcal{D}^{\text{BN}}_{t}$ indicates $\mathcal{D}^{\text{BN}}$ at the
$t$-th iteration. We employ
$\mathbf{x}^{\text{BN}}\sim\mathcal{D}^{\text{BN}}_{t}$ as an auxiliary input
at the $t$-th iteration. In this way, we can construct a bias-contrastive pair
that encourages the intrinsic attributes to be extracted as their common
features. We abbreviate $\mathcal{D}^{\text{BN}}_{t}$ as
$\mathcal{D}^{\text{BN}}$ for brevity in the rest of the paper.

###  3.3 Intrinsic feature enhancement

To emphasize the intrinsic features in $f_{d}$, we introduce the intrinsic
feature enhancement (IE) weight that imposes a high value on the intrinsic
features. The IE weight identifies the region of intrinsic features from bias-
contrastive pairs by investigating 1) their common features with common
feature score $c$ and 2) class-discerning features that are relatively under-
exploited in the input with relative-exploitation score $r$. For the
explanation, we split $f_{d}$ into two parts.
$f_{d}^{\text{emb}}:\mathbb{R}^{H\times W\times 3}\to\mathbb{R}^{h\times
w\times c}$ maps an input to the intermediate feature, and
$f_{d}^{\text{cls}}:\mathbb{R}^{h\times w\times c}\to\mathbb{R}^{C}$ is
composed of the average pooling and the linear classifier and outputs the
classification logits, where
$f_{d}(\mathbf{x})=f_{d}^{\text{cls}}\left(f_{d}^{\text{emb}}(\mathbf{x})\right)$.

First, given the input $\mathbf{x}$, common feature score $c$ identifies the
features that are similar to the features in $\mathbf{x}^{\text{BN}}$ that has
the same class label as $\mathbf{x}$ while not having bias attributes.
Specifically, we extract the intermediate features
$\mathbf{z}=f_{d}^{\text{emb}}(\mathbf{x})$ and
$\mathbf{z}^{\text{BN}}=f_{d}^{\text{emb}}(\mathbf{x}^{\text{BN}})$,
respectively. Next, we obtain the common feature score of $\mathbf{z}$
(_i.e_., $c(\mathbf{z})\in\mathbb{R}^{h\times w}$). Given the $n$-th feature
of $\mathbf{z}$ (_i.e_., $\mathbf{z}_{n}\in\mathbb{R}^{c}$), let $i^{*}$-th
feature of $\mathbf{z}^{\text{BN}}$ (_i.e_.,
$\mathbf{z}^{\text{BN}}_{i^{*}}\in\mathbb{R}^{c}$) be the most similar feature
to $\mathbf{z}_{n}$, where
$i^{*}=\underset{i}{\arg\max}\left(\mathbf{z}^{\text{BN}}_{i}\cdot\mathbf{z}_{n}\right)$.
Then, the $n$-th element of $c(\mathbf{z})$ denotes the similarity score
between $\mathbf{z}_{n}$ and $\mathbf{z}^{\text{BN}}_{i^{*}}$, which is
formulated as follows:

$c(\mathbf{z})_{n}=\frac{\mathbf{z}_{i^{*}}^{\text{BN}}\cdot\mathbf{z}_{n}}{\max_{i,j}(\mathbf{z}_{i}^{\text{BN}}\cdot\mathbf{z}_{j})},$
(4)

where $\cdot$ indicates a dot product operation. We adopt the dot product for
the similarity metric to consider both the scale and the direction of the
features. The max normalization is employed to limit the score to less than
one. We consider the features with a high common feature score $c$ in
$\mathbf{z}$ as features that have a high likelihood of being intrinsic
features.

Next, the relative-exploitation score $r$ identifies class-discerning features
that are relatively under-exploited in $\mathbf{x}$ compared to
$\mathbf{x}^{\text{BN}}$. Since most of the $\mathbf{x}^{\text{BN}}$ does not
contain bias attributes, we identify class-discerning intrinsic features by
investigating the features that are mainly used to predict
$\mathbf{x}^{\text{BN}}$ as its target label. At the same time, we identify
the features that are under-exploited in the $\mathbf{x}$ compared to the
$\mathbf{x}^{\text{BN}}$. To achieve this, we use a visual explanation map of
Grad-CAM [19] that imposes a higher value on the features that have more
contribution to predicting a specific label. We calculate the explanation map
$\text{E}(\mathbf{z})$ and $\text{E}(\mathbf{z}^{\text{BN}})$ with respect to
their ground-truth labels. We apply max normalization to the explanation maps
to compare the relative importance of the features in prediction. We compare
the $n$-th value of $\text{E}(\mathbf{z})$ (_i.e_.,
$\text{E}(\mathbf{z})_{n}$) with the $i^{*}$-th value of
$\text{E}(\mathbf{z}^{\text{BN}})$, where $i^{*}$ is the index of the feature
in $\mathbf{z}^{\text{BN}}$ that is the most similar with $\mathbf{z}_{n}$.
Accordingly, the $n$-th element of $r(\mathbf{z})\in\mathbb{R}^{h\times w}$ is
calculated as:

$r(\mathbf{z})_{n}=\left(\frac{2\text{E}(\mathbf{z}^{\text{BN}})_{i^{*}}}{\text{E}(\mathbf{z}^{\text{BN}})_{i^{*}}+\text{E}(\mathbf{z})_{n}}\right)^{\tau},$
(5)

where $\tau$ is the amplification factor. The score becomes larger than one
when the $\mathbf{z}_{n}$ is relatively under-exploited than
$\mathbf{z}^{\text{BN}}_{i^{*}}$ for prediction. When
$\mathbf{z}^{\text{BN}}_{i^{*}}$ is not used for discerning the class,
$\text{E}(\mathbf{z}^{\text{BN}})_{i^{*}}$ becomes close to zero and the score
converges to zero.

Finally, $n$-th element of the $\text{IE}(\mathbf{z})$ is defined as:

$\text{IE}(\mathbf{z})_{n}=\max(c(\mathbf{z})_{n}\odot r(\mathbf{z})_{n},1),$
(6)

where $\odot$ indicates the element-wise multiplication. The IE weight has a
large value on the features of $\mathbf{x}$ that commonly appear in
$\mathbf{x}^{\text{BN}}$ but has not exploited enough for the prediction of
$\mathbf{x}$. We clip the values to be larger than one to enhance only the
relatively under-exploited features in $\mathbf{z}$ while preserving the other
features.

Using the IE weight, we obtain the guidance $g(\mathbf{z})$ that emphasizes
the intrinsic features in $\mathbf{z}$ as follows:

$g(\mathbf{z})=\mathbf{z}\odot\text{IE}(\mathbf{z}).$ (7)

We broadcast $\text{IE}(\mathbf{z})$ to match its shape with $\mathbf{z}$
before multiplication. During the training, this spatial guidance informs the
model of where to focus to learn intrinsic features from training samples.

Algorithm 1 Debiasing with the intrinsic feature guidance

Input: pretrained biased models,
training dataset $\mathcal{D}$, biased model $f_{b}$, debiased model
$f_{d}$, reference iteration $T_{1}$ for BN score, starting
iteration $T_{2}(\geq T_{1})$ to apply intrinsic feature guidance
Output: trained debiased model $f_{d}$

1:Construct $\mathcal{D}^{\text{A}},\mathcal{D}^{\text{BN}}_{\text{cand}}$
from $\mathcal{D}$ using pretrained biased models

2:for every iteration $t$ do

3: Sample $\mathbf{x}\sim\mathcal{D}$

4: if $t\geq T_{1}$ and $l_{\text{ref}}(\mathbf{x})$ is not initialized then

5: $l_{\text{ref}}(\mathbf{x})\leftarrow l(\mathbf{x})$

6: end if

7: if $\mathbf{x}\in\mathcal{D}^{\text{A}}$ then

8: Train $f_{b}(\mathbf{x})$ with $\mathcal{L}_{\text{CE}}$

9: end if

10: if $t<$ $T_{2}$ then $\triangleright$ Train w/o guidance

11: Train $f_{d}(\mathbf{x})$ with $\mathcal{L}_{\text{main}}$

12: else if $t\geq$ $T_{2}$ then $\triangleright$ Train w/ guidance

13: Update $\mathcal{D}^{\text{BN}}$

14: Sample $\mathbf{x}^{\text{BN}}\sim\mathcal{D}^{\text{BN}}$

15: Train $f_{d}(\mathbf{x},\mathbf{x}^{\text{BN}})$ with
$\mathcal{L}_{\text{total}}$

16: end if

17:end for

###  3.4 Training with intrinsic feature guidance

We basically train $f_{d}$ with the CE loss as follows:

$\mathcal{L}_{\text{main}}=w(\mathbf{x})\mathcal{L}_{\text{CE}}(f_{d}(\mathbf{x}),y)$
(8)

where $w(\mathbf{x})$ is the sample reweighting value of $\mathbf{x}$ [15].
$w(\mathbf{x})$ emphasizes the samples that $f_{b}$ fails to learn, which are
mostly BC samples. The detailed description of $w(\mathbf{x})$ is included in
the Supplementary.

In addition, we guide the model to focus on the region of intrinsic features
through a guidance loss and a BN loss. We observe that the BN score has a
higher value on the BC samples compared to the BA samples as training $f_{b}$
proceeds (See Sec. 4.3). In this respect, we employ the BN score of
$\mathbf{x}^{\text{BN}}$ (_i.e_., $s(\mathbf{x}^{\text{BN}})$) to upweight the
loss when BC samples are adopted as $\mathbf{x}^{\text{BN}}$. Here, we clip
the value of loss weight $s(\mathbf{x}^{\text{BN}})$ to be larger than zero.

Guidance loss.  To guide the model to exploit the intrinsic features from
$\mathbf{x}$, we minimize the L1 distance between $g(\mathbf{z})$ and
$\mathbf{z}$ as follows:

$\mathcal{L}_{\text{guide\\_sim}}=s(\mathbf{x}^{\text{BN}})\lVert\text{GAP}(\mathbf{z})-\text{GAP}\left({g(\mathbf{z})}\right)\rVert_{1},$
(9)

where GAP represents the global average pooling. $s(\mathbf{x}^{\text{BN}})$
is multiplied as a loss weight to impose a high weight on the loss when BC
samples are selected as $\mathbf{x}^{\text{BN}}$.

Also, we apply the CE loss to the guidance $g(\mathbf{z})$ to encourage it to
include the intrinsic features that contribute to the correct prediction as
follows:

$\mathcal{L}_{\text{guide\\_cls}}=w(\mathbf{x})\mathcal{L}_{\text{CE}}\left(f_{d}^{\text{cls}}(g(\mathbf{z})),y\right),$
(10)

where $w(\mathbf{x})$ is the reweighting value as in
$\mathcal{L}_{\text{main}}$.

Finally, our guidance loss $\mathcal{L}_{\text{guide}}$ is calculated as
follows:

$\mathcal{L}_{\text{guide}}=\lambda_{\text{sim}}\mathcal{L}_{\text{guide\\_sim}}+\mathcal{L}_{\text{guide\\_cls}},$
(11)

where $\lambda_{\text{sim}}$ is a hyperparameter to control the relative
significance between the losses. We set $\lambda_{\text{sim}}$ set as 0.1.

BN loss.  We also employ the CE loss on $\mathbf{x}^{\text{BN}}$ to encourage
the model to learn class-discerning features. This enables the IE weight to
find intrinsic features among the common features. The BN loss is defined as:

$\mathcal{L}_{\text{BN}}=s(\mathbf{x}^{\text{BN}})\mathcal{L}_{\text{CE}}(f_{d}(\mathbf{x}^{\text{BN}}),y).$
(12)

Here, we also exploit $s(\mathbf{x}^{\text{BN}})$ to impose high weight on the
loss when $\mathbf{x}^{\text{BN}}$ is a BC sample.

Overall objective function.  In summary, the overall objective function is
defined as follows:

$\mathcal{L}_{\text{total}}=\lambda_{\text{main}}\mathcal{L}_{\text{main}}+\mathcal{L}_{\text{guide}}+\mathcal{L}_{\text{BN}},$
(13)

$\lambda_{\text{main}}$ is the constant value that linearly increases from
zero to one during training $f_{d}$ with the guidance. This prevents the model
from focusing on bias features in $\mathbf{x}$ in the early phase. The overall
process of our method is provided in Algorithm 1. Here, we set $T_{1}$ and
$T_{2}$ as 1K and 10K, respectively. Note that all the hyperparameters are
identically applied across different datasets and bias severities. We provide
further details of the training and the implementation in the Supplementary.

Method | Waterbirds | BFFHQ | BAR
---|---|---|---
0.5 | 1.0 | 2.0 | 5.0 | 0.5 | 1.0 | 2.0 | 5.0 | 1.0 | 5.0
Vanilla [5] | 57.41 | 58.07 | 61.04 | 64.13 | 55.64 | 60.96 | 69.00 | 82.88 | 70.55 | 82.53
HEX [25] | 57.88 | 58.28 | 61.02 | 64.32 | 56.96 | 62.32 | 70.72 | 83.40 | 70.48 | 81.20
LNL [9] | 58.49 | 59.68 | 62.27 | 66.07 | 56.88 | 62.64 | 69.80 | 83.08 | - | -
EnD [22] | 58.47 | 57.81 | 61.26 | 64.11 | 55.96 | 60.88 | 69.72 | 82.88 | - | -
ReBias [2] | 55.44 | 55.93 | 58.53 | 62.14 | 55.76 | 60.68 | 69.60 | 82.64 | 73.04 | 83.90
LfF [15] | 60.66 | 61.78 | 58.92 | 61.43 | 65.19 | 69.24 | 73.08 | 79.80 | 70.16 | 82.95
DisEnt [12] | 59.59 | 60.05 | 59.76 | 64.01 | 62.08 | 66.00 | 69.92 | 80.68 | 70.33 | 83.13
LfF+BE [13] | 61.22 | 62.58 | 63.00 | 63.48 | 67.36 | 75.08 | 80.32 | 85.48 | 73.36 | 83.87
DisEnt+BE [13] | 51.65 | 54.10 | 53.43 | 54.21 | 67.56 | 73.48 | 79.48 | 84.84 | 73.29 | 84.96
Ours | 63.64 | 65.22 | 65.23 | 66.33 | 71.68 | 77.56 | 83.08 | 87.60 | 75.14 | 85.03

Table 1: Comparison to the baselines. We measure the classification accuracy
on test sets with different bias severities. The best accuracy values are in
bold. The hyphen mark ‘-’ means it is not applicable. Results with standard
deviations are provided in the Supplementary.

|  $\mathcal{D}^{\text{BN}}_{\text{cand}}$ \- $\mathcal{D}^{\text{BN}}$ |  $\mathcal{D}^{\text{BN}}$/$\mathcal{D}$ (%)
---|---|---
Dataset | BA | BC | BA | BC
Waterbirds | 26.50 $\pm$5.32 | 0.75 $\pm$0.83 | 2.75 $\pm$0.31 | 79.69 $\pm$3.72
BFFHQ | 199.80 $\pm$40.14 | 8.00 $\pm$2.76 | 0.46 $\pm$0.09 | 50.00 $\pm$1.04
BAR | 30.60 $\pm$3.83 | 3.20 $\pm$1.60 | 3.58 $\pm$0.14 | 47.14 $\pm$5.71

Table 2: Effectiveness of BN score on excluding BA samples.
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ \- $\mathcal{D}^{\text{BN}}$ presents
the number of excluded samples when constructing $\mathcal{D}^{\text{BN}}$
from $\mathcal{D}^{\text{BN}}_{\text{cand}}$.
$\mathcal{D}^{\text{BN}}$/$\mathcal{D}$ indicates that the ratio of samples in
$\mathcal{D}^{\text{BN}}$ to the samples in $\mathcal{D}$.

##  4 Experiments

###  4.1 Experimental settings

Dataset.  We utilize Waterbirds [18], biased FFHQ (BFFHQ) [10], and BAR [15]
for the experiments. Each dataset contains different types of target class and
bias attributes: Waterbirds - {bird type, background}, BFFHQ - {age, gender},
and BAR - {action, background}. The former and the latter in the bracket
indicate the target class and the bias attribute, respectively. To be
specific, the Waterbirds dataset has two bird classes: waterbirds and
landbirds. Most of the waterbirds are in the water background, and most of the
landbirds are in the land background. In the training dataset of BFFHQ, most
young people are female while most old people are male. The word ‘young’
indicates an age ranging between 10 and 29, and ‘old’ indicates an age ranging
between 40 and 59. Lastly, the BAR dataset consists of six classes of action
(e.g., fishing), where the background (e.g., water surface) is highly
correlated with each class. Following the previous studies [15, 12, 13], we
validate our model’s effectiveness under different levels of bias severity,
_i.e_., a ratio of BC samples to the total training samples: 0.5%, 1%, 2%, and
5%. In the test sets, the spurious correlations found in the training set do
not exist. More details are provided in Supplementary.

Evaluation.  We report the best accuracy of the test set averaged over five
independent trials with different random seeds. The Waterbirds dataset has an
extremely skewed test dataset composed of 4,600 landbirds and 1,194
waterbirds. This can mislead the debiasing performance as the model may
achieve high classification accuracy by simply predicting most images as
landbirds. We measure the classification accuracy for each class and report
their average value to obtain an accurate understanding of the effectiveness
of methods, regardless of class frequencies. Also, for the BFFHQ, we report
the best accuracy of BC samples in the test set, following the previous works
[12, 13]. For the analyses, we utilize the datasets with 1% bias severity.

###  4.2 Comparison to previous works

We compare the classification accuracy on the test sets between the baselines
and ours in Table 1. For baselines, we employ a vanilla model trained with the
CE loss, the methods using explicit bias label (_i.e_., LNL [9], EnD [22]),
presuming the type of the bias (_i.e_., HEX [25], ReBias [2]), and assuming
the bias information is unknown (_i.e_., LfF [15], DisEnt [12], LfF+BE [13],
DisEnt+BE [13]). Our approach achieves state-of-the-art performance in
comparison to the previous methods including those utilizing explicit bias
labels. The results exhibit that our method improves performance robustly
across various levels of bias severity, even under the constraints of extreme
bias severity (_e.g_., 0.5 %). This shows that providing explicit spatial
guidance for intrinsic features effectively encourages the model to learn
debiased representations, leading to performance improvement.

Figure 2: Visualization of BN scores of the samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ during the training. The red lines and
the blue lines indicate the BN scores of BA and BC samples, respectively.
Figure 3: Visualization of the spatial guidance using (a) Waterbirds and (b)
BAR dataset. Given bias-contrastive pairs, $\mathbf{x}$ and
$\mathbf{x}^{\text{BN}}$, $\text{E}(\mathbf{z})$ indicates the regions
originally focused on by $f_{d}$ and $\text{IE}(\mathbf{z})$ shows the regions
highlighted by our IE weight. Figure 4: Comparison of the region focused by a
debiased model trained with and without our method. We compare Grad-CAM
results on the test set of (a) Waterbirds and (b) BAR.

###  4.3 Analysis of BN score

We analyze our BN score that identifies and emphasizes BC samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ during training $f_{d}$. In this
section, we assess the effectiveness of the BN score on excluding BA samples
from $\mathcal{D}^{\text{BN}}_{\text{cand}}$. Also, we evaluate the efficacy
of the BN score as a loss weight by investigating the BN scores of the samples
in $\mathcal{D}^{\text{BN}}_{\text{cand}}$.

Effectiveness of BN score on excluding BA samples.  Table 2 presents how the
BN score effectively filters out BA samples from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ while preserving BC samples when
constructing $\mathcal{D}^{\text{BN}}$. The first two columns present the
number of the BA and BC samples excluded from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ to construct
$\mathcal{D}^{\text{BN}}$, respectively. Also, the last two columns represent
the ratio of the number of the BA and BC samples in $\mathcal{D}^{\text{BN}}$
to that in $\mathcal{D}$, respectively. For the analysis, we use
$\mathcal{D}^{\text{BN}}$ at the 50K-th iteration and report the mean value of
the five independent trials. Here, we expect $\mathcal{D}^{\text{BN}}$ to
contain a maximal number of BC samples while including a minimal number of BA
samples. As shown in the first two columns of Table 2, our BN score excludes a
large number of BA samples while minimizing the loss of BC samples. As a
result, $\mathcal{D}^{\text{BN}}$ preserves around 50-80% of BC samples, while
containing a minimal number of BA samples compared to $\mathcal{D}$, as
presented in the last two columns.

Efficacy of BN score as loss weight.  We utilize the BN score to upweight the
training loss when BC samples are chosen as $\mathbf{x}^{\text{BN}}$. To
verify its effectiveness as a loss weight, we compare the BN scores of BC
samples and BA samples in $\mathcal{D}^{\text{BN}}_{\text{cand}}$ during the
training for the Waterbirds, BFFHQ, and BAR dataset. In Fig. 2, we present the
average BN scores of BA (red line) and BC samples (blue line) in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ at every 500 iterations. Since BN
scores are recorded after the 1K-th iteration, the BN scores until the 1K-th
iteration are reported as zero. The BN scores of BC samples in the BFFHQ
dataset mostly range from 0.4 to 0.5 while the scores of BA samples are close
to 0. This indicates that the BN score as a loss weight in the BFFHQ imposes a
much larger value on the BC samples, while approximately zero values on the BA
samples. Also, the BN scores of BC samples in the Waterbirds and the BAR
dataset become twice larger than the scores of BA samples. The result shows
that the BN score effectively emphasizes BC samples compared to the BA samples
in $\mathcal{D}^{\text{BN}}_{\text{cand}}$ during the training. Further
analysis of the BN score is included in the Supplementary.

###  4.4 Analysis of intrinsic feature guidance

We conduct a qualitative analysis of the regions emphasized by our intrinsic
feature guidance during the training and the features learned by $f_{d}$ after
training. We use the Waterbirds and BAR datasets with 1% of bias severity for
the analysis.

Visualization of the guidance during training.  In Fig. 3, we visualize the
features emphasized by our IE weight $\text{IE}(\mathbf{z})$. For comparison,
we also visualize $\text{E}(\mathbf{z})$, the features focused by the model
before applying $\text{IE}(\mathbf{z})$ for guidance. We select a BA sample as
$\mathbf{x}$ and a BC sample as $\mathbf{x}^{\text{BN}}$ from the training
data for the analysis. For the Waterbirds dataset in Fig. 3(a),
$\text{IE}(\mathbf{z})$ highlights the wings or the beak of the bird compared
to $\text{E}(\mathbf{z})$, where the forest or the water (_i.e_., bias
attributes) is highlighted. Also, in Fig. 3(b), $\text{E}(\mathbf{z})$ focuses
more on the bias attributes such as rocks or the water than the intrinsic
attributes. In contrast, $\text{IE}(\mathbf{z})$ emphasizes the action of the
human that is less exploited compared to the bias features in
$\text{E}(\mathbf{z})$. The results demonstrate that our guidance successfully
identifies and enhances under-exploited intrinsic features during the
training.

Effect of intrinsic feature guidance on debiasing.  We qualitatively evaluate
the effectiveness of the intrinsic feature guidance by investigating the
visual explanation maps of the test samples. We compare the Grad-CAM [19]
results of the model trained with and without our method in Fig 4. The Grad-
CAM results highlight the features that the model employs to predict the input
as its ground-truth label. In Fig. 4 (a), while the model trained without
guidance focuses on the forest or the sea, ours focuses on the tail or a
curved shape of the bird’s body. Additionally, Fig. 4 (b) shows that ours
focuses on the motion of the human rather than the backgrounds that are
concentrated on by the model trained without our guidance. The results verify
that our method successfully encourages the model to learn intrinsic features
from the training dataset, improving the robustness of the model against
dataset bias.

###  4.5 Ablation study

As shown in Table 3, we perform ablation studies to verify the effectiveness
of the individual components in our method. The results of ours are reported
in the last row.

Importance of $\mathbf{x}^{\text{BN}}$ selection and BN score as loss weight.
We demonstrate the efficacy of adopting BC samples as
$\mathbf{x}^{\text{BN}}$. We train the model by sampling
$\mathbf{x}^{\text{BN}}$ from three different datasets: $\mathcal{D}$,
$\mathcal{D}^{\text{BN}}_{\text{cand}}$, and $\mathcal{D}^{\text{BN}}$. In the
first row of Table 3, we randomly sample $\mathbf{x}^{\text{BN}}$ from the
training dataset $\mathcal{D}$ without using the BN score. In the second row,
we train the model with $\mathbf{x}^{\text{BN}}$ sampled from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$, where BA samples are roughly filtered
out using the early-stopped biased models. The model in the third row is
trained with $\mathbf{x}^{\text{BN}}$ from $\mathcal{D}^{\text{BN}}$ that
mainly includes BC samples using our BN score. The results show a gradual
improvement in debiasing performance as more BC samples are selected as
$\mathbf{x}^{\text{BN}}$. This is because auxiliary samples without bias
attributes prevent the common features from including the bias features,
composing a bias-contrastive pair with the input. Therefore, our guidance
effectively enhances the intrinsic features during the training. Finally, the
last row in Table 3 presents that employing the BN score
$s(\mathbf{x}^{\text{BN}})$ to reweight the losses further enhances
performance by emphasizing the usage of BC samples as
$\mathbf{x}^{\text{BN}}$.

Training objectives.  We examine the impact of each training objective,
$\mathcal{L}_{\text{guide}}$ and $\mathcal{L}_{\text{BN}}$, in our method. We
report the performance of the model trained without
$\mathcal{L}_{\text{guide}}$ (the fourth row) and without
$\mathcal{L}_{\text{BN}}$ (the fifth row) in Table 3. The model trained
without $\mathcal{L}_{\text{guide}}$ exhibits degraded performance, facing
difficulties in identifying where to focus to learn intrinsic features.
Similarly, training the model without $\mathcal{L}_{\text{BN}}$ also results
in a performance decrease. The results verify that $\mathcal{L}_{\text{BN}}$
successfully supports the IE weight to identify intrinsic features among the
common features by learning class-discerning features from
$\mathbf{x}^{\text{BN}}$. The model that incorporates both
$\mathcal{L}_{\text{guide}}$ and $\mathcal{L}_{\text{BN}}$ demonstrates the
best performance (the last row of Table 3).

$\mathcal{L}_{\text{guide}}$ | $\mathcal{L}_{\text{BN}}$ | $\mathbf{x}^{\text{BN}}$ |  |  $s(\mathbf{x}^{\text{BN}})$ as
---
loss weight
Waterbirds | BFFHQ | BAR
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | $\mathcal{D}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✗ | 62.79 $\pm$1.21 | 71.04 $\pm$2.55 | 73.36 $\pm$1.40
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | $\mathcal{D}^{\text{BN}}_{\text{cand}}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✗ | 64.65 $\pm$1.23 | 75.64 $\pm$1.87 | 74.27 $\pm$0.66
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | $\mathcal{D}^{\text{BN}}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✗ | 65.10 $\pm$0.87 | 77.08 $\pm$2.05 | 74.62 $\pm$1.07
TextRenderingMode=FillStroke, LineWidth=.5pt, ✗ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | $\mathcal{D}^{\text{BN}}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | 63.81 $\pm$1.24 | 76.92 $\pm$1.03 | 74.03 $\pm$1.13
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✗ | $\mathcal{D}^{\text{BN}}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | 62.10 $\pm$3.35 | 74.84 $\pm$2.00 | 74.87 $\pm$1.51
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ | $\mathcal{D}^{\text{BN}}$ |  TextRenderingMode=FillStroke, LineWidth=.5pt, ✓ |  65.22 $\pm$0.95 |  77.56 $\pm$1.24 |  75.14 $\pm$0.82

Table 3: Ablation study on the proposed training objectives, the dataset that
$\mathbf{x}^{\text{BN}}$ is sampled from, and the BN score of
$\mathbf{x}^{\text{BN}}$ as a loss weight. The check mark (
TextRenderingMode=FillStroke, LineWidth=.5pt, ✓) denotes the inclusion of the
corresponding method, while the cross mark ( TextRenderingMode=FillStroke,
LineWidth=.5pt, ✗) indicates the exclusion of the component in the experiment.

##  5 Conclusion

In this paper, we propose a debiasing method that explicitly provides the
model with spatial guidance for intrinsic features. Leveraging an auxiliary
sample, we first identify intrinsic features by investigating the class-
discerning features commonly appearing in a bias-contrastive pair. Our IE
weight enhances the intrinsic features that have not been focused on yet in
the input by a debiased model. To construct the bias-contrastive pair without
bias labels, we introduce a bias-negative (BN) score that tracks the
classification loss of a biased model to distinguish BC samples from BA
samples during the training. The effectiveness of our method is demonstrated
through experiments on synthetic and real-world datasets with varying levels
of bias severity. We believe this work sheds light on the significance of
providing explicit guidance on the intrinsic attributes for debiasing.

## Acknowledgments

This work was supported by the Institute for Information & communications
Technology Promotion(IITP) grant funded by the Korea government(MSIT)
(No.2019-0-00075, Artificial Intelligence Graduate School Program(KAIST), the
National Research Foundation of Korea (NRF) grant funded by the Korea
government (MSIT) (No. NRF-2022R1A2B5B02001913 & No. 2022R1A5A7083908) and
partly by Kakao Brain Corporation.

## References

  * Asgari et al. [2022] Saeid Asgari, Aliasghar Khani, Fereshte Khani, Ali Gholami, Linh Tran, Ali Mahdavi Amiri, and Ghassan Hamarneh.  Masktune: Mitigating spurious correlations by forcing to explore.  In _Proc. the Advances in Neural Information Processing Systems (NeurIPS)_ , pages 23284–23296, 2022.
  * Bahng et al. [2020] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh.  Learning de-biased representations with biased representations.  In _Proc. the International Conference on Machine Learning (ICML)_ , 2020.
  * Darlow et al. [2020] Luke Darlow, Stanislaw Jastrzebski, and Amos Storkey.  Latent adversarial debiasing: Mitigating collider bias in deep neural networks.  _arXiv preprint arXiv:2011.11486_ , 2020.
  * Geirhos et al. [2019] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel.  Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.  In _Proc. the International Conference on Learning Representations (ICLR)_ , 2019.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , 2016.
  * Hu et al. [2018] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , 2018.
  * Huang et al. [2020] Zeyi Huang, Haohan Wang, Eric P. Xing, and Dong Huang.  Self-challenging improves cross-domain generalization.  In _Proc. of the European Conference on Computer Vision (ECCV)_ , 2020.
  * Hwang et al. [2022] Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, and Byoung-Tak Zhang.  Selecmix: Debiased learning by contradicting-pair sampling.  In _Proc. the Advances in Neural Information Processing Systems (NeurIPS)_ , 2022.
  * Kim et al. [2019] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim.  Learning not to learn: Training deep neural networks with biased data.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , 2019.
  * Kim et al. [2021] Eungyeup Kim, Jihyeon Lee, and Jaegul Choo.  Biaswap: Removing dataset bias with bias-tailored swapping augmentation.  In _Proc. of the IEEE international conference on computer vision (ICCV)_ , pages 14992–15001, 2021.
  * Kwon et al. [2022] Bum Chul Kwon, Jungsoo Lee, Chaeyeon Chung, Nyoungwoo Lee, Ho-Jin Choi, and Jaegul Choo.  DASH: Visual Analytics for Debiasing Image Classification via User-Driven Synthetic Data Augmentation.  _Eurographics Conference on Visualization (EuroVis)-Short Papers. The Eurographics Association_ , 2022.
  * Lee et al. [2021] Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo.  Learning debiased representation via disentangled feature augmentation.  In _Proc. the Advances in Neural Information Processing Systems (NeurIPS)_ , 2021.
  * Lee et al. [2023] Jungsoo Lee, Jeonghoon Park, Daeyoung Kim, Juyoung Lee, Edward Choi, and Jaegul Choo.  Revisiting the importance of amplifying bias for debiasing.  In _Proc. the AAAI Conference on Artificial Intelligence (AAAI)_ , pages 14974–14981, 2023.
  * Liu et al. [2021] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn.  Just train twice: Improving group robustness without training group information.  In _Proc. the International Conference on Machine Learning (ICML)_ , pages 6781–6792, 2021.
  * Nam et al. [2020] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.  Learning from failure: Training debiased classifier from biased classifier.  In _Proc. the Advances in Neural Information Processing Systems (NeurIPS)_ , 2020.
  * Park et al. [2023] Geon Yeong Park, Sangmin Lee, Sang Wan Lee, and Jong Chul Ye.  Training debiased subnetworks with contrastive weight pruning.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 7929–7938, 2023.
  * Park et al. [2020] Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, and Richard Zhang.  Swapping autoencoder for deep image manipulation.  In _Proc. the Advances in Neural Information Processing Systems (NeurIPS)_ , 2020.
  * Sagawa et al. [2020] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang.  Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.  _Proc. the International Conference on Learning Representations (ICLR)_ , 2020.
  * Selvaraju et al. [2017] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proc. of the IEEE international conference on computer vision (ICCV)_ , 2017.
  * Simonyan and Zisserman [2015] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _Proc. the International Conference on Learning Representations (ICLR)_ , 2015.
  * Szegedy et al. [2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 1–9, 2015.
  * Tartaglione et al. [2021] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto.  End: Entangling and disentangling deep representations for bias correction.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 13508–13517, 2021.
  * Torralba and Efros [2011] Antonio Torralba and Alexei A. Efros.  Unbiased look at dataset bias.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 1521–1528, 2011.
  * Wah et al. [2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * Wang et al. [2019] Haohan Wang, Zexue He, Zachary L. Lipton, and Eric P. Xing.  Learning robust representations by projecting superficial statistics out.  In _Proc. the International Conference on Learning Representations (ICLR)_ , 2019.
  * Zagoruyko and Komodakis [2016] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  _Proc. of the British Machine Vision Conference (BMVC)_ , 2016.
  * Zhang et al. [2018] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz.  mixup: Beyond empirical risk minimization.  In _Proc. the International Conference on Learning Representations (ICLR)_ , 2018.
  * Zhang et al. [2023] Yi-Kai Zhang, Qi-Wei Wang, De-Chuan Zhan, and Han-Jia Ye.  Learning debiased representations via conditional attribute interpolation.  In _Proc. of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 7599–7608, 2023.
  * Zhang and Sabuncu [2018] Zhilu Zhang and Mert R Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  _arXiv preprint arXiv:1805.07836_ , 2018.
  * Zhou et al. [2017] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  _The IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)_ , 40(6):1452–1464, 2017.

Supplementary Material

This supplementary material offers further analysis of our approach,
additional experimental results, the details of the datasets and
implementation, limitations, and future work. Appendix A and Appendix B
provide the analysis of the bias-negative (BN) score as a loss weight and
samples with negative BN score, respectively. Appendix C analyzes the effect
of BC samples in $\mathcal{D}^{\text{BN}}$ on debiasing performance. Also,
Appendix D compares the recent sample selection methods with ours. Moreover,
Appendix E and Appendix F present additional qualitative results regarding the
guidance and additional quantitative results, respectively. Appendix G and
Appendix H provide the details about the dataset and implementation. Lastly,
Appendix I discusses the limitations and future work.

##  Appendix A Additional analysis of the BN score as a loss weight

Figure 5: The distributions of $f_{b}$’s classification loss of samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$. The red and blue lines denote the
losses of BA and BC samples, respectively. The dotted and solid lines indicate
the losses at the early and later stages of the training, respectively. Best
viewed in color.

As described in Sec. 3.4 in the main paper, we utilize the BN score of
$\mathbf{x}^{\text{BN}}$ (_i.e_., $s(\mathbf{x}^{\text{BN}})$) to reweight the
guidance loss $\mathcal{L}_{\text{guide\\_sim}}$ and the BN loss
$\mathcal{L}_{\text{BN}}$. The BN score as a loss weight is designed to
upweight the losses when bias-conflicting (BC) samples are selected as
$\mathbf{x}^{\text{BN}}$, which further encourages our IE weight to enhance
the intrinsic features. For verification, we present that the BN score has a
much larger value on the BC samples compared to bias-aligned (BA) samples
during the training in Fig. 2 in the main paper.

Since $s(\mathbf{x}^{\text{BN}})$ has a larger value when the current $f_{b}$
loss of $\mathbf{x}^{\text{BN}}$ is larger than that of the early stage of
training, the results imply that the $f_{b}$ loss of BC samples largely
increases as training proceeds compared to BA samples.

To further verify this, we present $f_{b}$’s classification loss of samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$ during the training in Fig. 5. The
BFFHQ dataset [10] with a bias severity of 1% is used for the analysis. In
Fig. 5, the dotted lines denote the distribution of $f_{b}$’s classification
loss at the early stage of training (1K-th iteration), and the solid lines
indicate that of the later stage of training (50K-th iteration). The results
show that the $f_{b}$ loss of BC samples (blue lines) largely increases at the
later stage of training compared to the early stage, unlike BA samples (red
lines). This demonstrates that the BN score as a loss weight can effectively
upweight the training losses when BC samples are chosen as
$\mathbf{x}^{\text{BN}}$.

##  Appendix B Samples having negative BN score

Figure 6: The examples of samples that have negative BN scores at the later
stage of training. Figure 7:  Additional visualization results of the spatial
guidance using (a) Waterbirds and (b) BAR dataset. Given bias-contrastive
pairs, $\mathbf{x}$ and $\mathbf{x}^{\text{BN}}$, $\text{E}(\mathbf{z})$
indicates the regions originally focused on by $f_{d}$ and
$\text{IE}(\mathbf{z})$ shows the regions highlighted by our IE weight.

As mentioned in Sec. 3.2 in the main paper, we further filter out the samples
with negative BN scores from $\mathcal{D}_{\text{cand}}^{\text{BN}}$ to mainly
exploit the BC samples as $\mathbf{x}^{\text{BN}}$. Here, we expect that the
samples with negative BN scores are mostly BA samples. To investigate the
samples with negative BN scores, we chose the samples that were erroneously
incorporated into $\mathcal{D}_{\text{cand}}^{\text{BN}}$ initially but
excluded at the later stage of training (_i.e_., 50K-th iteration), exhibiting
negative BN scores. This process is repeated five times, and we visualize the
samples chosen more than three times in Fig. 6. We use the BFFHQ dataset with
a 1% bias severity for the experiment.

We observe that the samples with negative BN scores in
$\mathcal{D}_{\text{cand}}^{\text{BN}}$ are mostly BA samples. As shown in the
figure, while the samples obviously contain bias attributes (_i.e_., features
representing female or male), the samples mostly have extreme shade, blur,
saturation, or unusual makeup, exhibiting non-typical appearance. Although the
bias attributes are known to be easy to learn, the non-typical appearance
prevents $f_{b}$ from detecting such bias attributes in the early stage of
training. In Sec. 4.5 of the main paper, we verify that employing such BA
samples as $\mathbf{x}^{\text{BN}}$ largely degrades the debiasing performance
by allowing the bias attributes to be included in the common features between
$\mathbf{x}$ and $\mathbf{x}^{\text{BN}}$. Our BN score effectively alleviates
this issue by filtering out such BA samples from
$\mathcal{D}_{\text{cand}}^{\text{BN}}$.

Figure 8: Additional comparison of the region focused by a debiased model
trained with and without our method. We compare Grad-CAM results on the test
set of (a) Waterbirds and (b) BAR.

##  Appendix C Importance of BN sample selection

We analyze the effect of the BC sample ratio in $\mathcal{D}^{\text{BN}}$ on
debiasing performance. We measure the accuracy using the BFFHQ dataset with a
1% bias severity by varying the number of BA and BC samples in
$\mathcal{D}^{\text{BN}}$. Table 4 shows that higher accuracy is achieved for
more BC samples and a lower ratio of BA to BC samples in
$\mathcal{D}^{\text{BN}}$. Overall, our method constantly shows performance
gain, except for the last column ({${\text{\\#BC in
}\mathcal{D}^{\text{BN}}}/{\text{\\#BC in }\mathcal{D}}$, ${\text{\\#BA in
}\mathcal{D}^{\text{BN}}}/{\text{\\#BC in }\mathcal{D}^{\text{BN}}}$}-{1.0,
10.0}). It is crucial not to select too many BA samples as
$\mathbf{x}^{\text{BN}}$.

${\text{\\#BC in }\mathcal{D}^{\text{BN}}}/{\text{\\#BC in }\mathcal{D}}$ | 0.1 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0
---|---|---|---|---|---|---|---
${\text{\\#BA in }\mathcal{D}^{\text{BN}}}/{\text{\\#BC in }\mathcal{D}^{\text{BN}}}$ | 0.0 | 0.0 | 0.0 | 0.1 | 1.0 | 2.0 | 10.0
Accuracy | 75.84 | 78.12 | 81.40 | 80.24 | 77.48 | 75.48 | 70.90

Table 4: Importance of $\mathbf{x}^{\text{BN}}$ selection.

##  Appendix D Comparison to recent sample selection methods

Our BN score is designed to further filter out BA samples in
$\mathcal{D}^{\text{BN}}_{\text{cand}}$, improving debiasing performance (Sec.
4.5 in the main paper). We compare BC sample selection in recent methods with
ours using BFFHQ with a 1% bias severity. Let $\mathcal{S}$ be a set of
samples identified as BC samples from training data $\mathcal{D}$.
$\\{\text{\\#BC in }\mathcal{S}/\text{\\#BC in }\mathcal{D},\text{\\#BA in
}\mathcal{S}/\text{\\#BC in }\mathcal{S}\\}$ is {75.63, 10.18}-BE [13],
{27.29, 4.32}-DCWP [16], and {50.0, 0.89}-Ours, respectively. Our method has
the least number of BA compared to BC samples in $\mathcal{S}$ while
preserving half of the total BC samples.

##  Appendix E Additional qualitative results

###  E.1 Visualization of the guidance during training

In addition to Fig. 3 of the main paper, we provide supplementary qualitative
results that present the features that the current model $f_{d}$ focuses on
(_i.e_., $\text{E}(\mathbf{z})$) and the features emphasized by the guidance
(_i.e_., $\text{IE}(\mathbf{z})$) during the training in Fig. 7. We use the
Waterbirds and BAR datasets with a bias severity of 1% for the analysis. We
train $f_{d}$ during 10K iterations and obtain the visual explanation map
$\text{E}(\mathbf{z})$ for the ground-truth label using Grad-CAM [19]. The
min-max normalization is applied to the values of $\text{E}(\mathbf{z})$ and
$\text{IE}(\mathbf{z})$ for visualization. We intentionally select the BA
sample and BC sample as $\mathbf{x}$ and $\mathbf{x}^{\text{BN}}$,
respectively, to compose a bias-negative pair.

As shown in Fig. 7, our IE weight (_i.e_., $\text{IE}(\mathbf{z})$)
appropriately emphasizes the regions of the intrinsic features while
$\text{E}(\mathbf{z})$ shows that the current model $f_{d}$ relies on the bias
attributes for prediction. For example, in the Waterbirds dataset,
$\text{IE}(\mathbf{z})$ properly enhances the intrinsic features of a bird
such as wings, a body, or a neck, while $\text{E}(\mathbf{z})$ highlights the
background features such as the land, the forest or the water. Also, in the
BAR dataset, $\text{IE}(\mathbf{z})$ emphasizes the arm throwing the javelin,
the motion of a person vaulting, climbing, or diving, while the current
$f_{d}$ mainly focuses on the biased features such as the playing field, the
sky, the mountain, or the water. These results verify the validity of our IE
weight $\text{IE}(\mathbf{z})$ as guidance for emphasizing the intrinsic
features in $\mathbf{x}$ that are under-exploited yet.

###  E.2 Effect of intrinsic feature guidance on debiasing

We present an additional qualitative analysis regarding the effectiveness of
the intrinsic feature guidance to supplement Fig. 4 in the main paper. Fig. 8
illustrates the Grad-CAM [19] results of the model trained with and without
our method. Here, the model trained without our method is the same as LfF+BE
[13]. We train the models with the Waterbids and the BAR datasets with a bias
severity of 1% and apply the Grad-CAM to the test samples for visualization.
The highlighted regions indicate the features that the model mainly employs
for prediction.

Fig. 8 (a) shows that our approach properly focuses on the intrinsic features
of the bird (_e.g_., wings, a beak, or feet), while the model trained without
our guidance mostly concentrates on the bias features (_e.g_., the water or
trees). For the BAR dataset in Fig. 8 (b), our model principally exploits the
action of a person (_e.g_., fishing, vaulting, or throwing) or the racing car
for prediction, while the model without our guidance focuses on the
backgrounds (_e.g_., the playing field or the water). The results demonstrate
the effectiveness of our method in guiding the model to learn intrinsic
features.

Method |  Waterbirds
---|---
0.5 |  1.0 |  2.0 |  5.0
Vanilla [5] |  57.41 $\pm$0.74 |  58.07 $\pm$1.00 |  61.04 $\pm$0.55 |  64.13 $\pm$0.14
HEX [25] |  57.88 $\pm$0.83 |  58.28 $\pm$0.67 |  61.02 $\pm$0.48 |  64.32 $\pm$0.62
LNL [9] |  58.49 $\pm$0.81 |  59.68 $\pm$0.78 |  62.27 $\pm$0.91 |  66.07 $\pm$1.15
EnD [22] |  58.47 $\pm$0.97 |  57.81 $\pm$1.04 |  61.26 $\pm$0.54 |  64.11 $\pm$0.52
ReBias [2] |  55.44 $\pm$0.24 |  55.93 $\pm$0.66 |  58.53 $\pm$0.52 |  62.14 $\pm$1.03
LfF [15] |  60.66 $\pm$0.77 |  61.78 $\pm$1.53 |  58.92 $\pm$2.93 |  61.43 $\pm$1.92
DisEnt [12] |  59.59 $\pm$1.67 |  60.05 $\pm$0.82 |  59.76 $\pm$1.26 |  64.01 $\pm$1.36
LfF+BE [13] |  61.22 $\pm$2.54 |  62.58 $\pm$1.12 |  63.00 $\pm$1.18 |  63.48 $\pm$0.56
DisEnt+BE [13] |  51.65 $\pm$1.45 |  54.10 $\pm$1.04 |  53.43 $\pm$1.42 |  54.21 $\pm$1.36
Ours |  63.64 $\pm$1.63 |  65.22 $\pm$0.95 |  65.23 $\pm$1.06 |  66.33 $\pm$1.42

Table 5: Comparison to the baselines. We measure the classification accuracy
on test sets of the Waterbirds dataset with different bias severities. The
best accuracy values are in bold. Results with standard deviations are
provided in the Supplementary.

Method |  BFFHQ |  BAR
---|---|---
0.5 |  1.0 |  2.0 |  5.0 |  1.0 |  5.0
Vanilla [5] |  55.64 $\pm$0.44 |  60.96 $\pm$1.00 |  69.00 $\pm$0.50 |  82.88 $\pm$0.49 |  70.55 $\pm$0.87 |  82.53 $\pm$1.08
HEX [25] |  56.96 $\pm$0.62 |  62.32 $\pm$1.21 |  70.72 $\pm$0.89 |  83.40 $\pm$0.34 |  70.48 $\pm$1.74 |  81.20 $\pm$0.68
LNL [9] |  56.88 $\pm$1.13 |  62.64 $\pm$0.99 |  69.80 $\pm$1.03 |  83.08 $\pm$0.93 |  - |  -
EnD [22] |  55.96 $\pm$0.91 |  60.88 $\pm$1.17 |  69.72 $\pm$1.14 |  82.88 $\pm$0.74 |  - |  -
ReBias [2] |  55.76 $\pm$1.50 |  60.68 $\pm$1.24 |  69.60 $\pm$1.33 |  82.64 $\pm$0.64 |  73.04 $\pm$1.04 |  83.90 $\pm$0.82
LfF [15] |  65.19 $\pm$3.23 |  69.24 $\pm$2.07 |  73.08 $\pm$2.70 |  79.80 $\pm$1.09 |  70.16 $\pm$0.77 |  82.95 $\pm$0.27
DisEnt [12] |  62.08 $\pm$3.89 |  66.00 $\pm$1.33 |  69.92 $\pm$2.72 |  80.68 $\pm$0.25 |  70.33 $\pm$0.19 |  83.13 $\pm$0.46
LfF+BE [13] |  67.36 $\pm$3.10 |  75.08 $\pm$2.29 |  80.32 $\pm$2.07 |  85.48 $\pm$2.88 |  73.36 $\pm$0.97 |  83.87 $\pm$0.82
DisEnt+BE [13] |  67.56 $\pm$2.11 |  73.48 $\pm$2.12 |  79.48 $\pm$1.80 |  84.84 $\pm$2.11 |  73.29 $\pm$0.41 |  84.96 $\pm$0.69
DCWP [16] |  64.08 $\pm$1.08 |  67.44 $\pm$2.87 |  75.24 $\pm$1.73 |  85.00 $\pm$0.94 |  69.63 $\pm$0.85 |  81.89 $\pm$0.68
Ours |  71.68 $\pm$1.74 |  77.56 $\pm$1.24 |  83.08 $\pm$1.69 |  87.60 $\pm$1.68 |  75.14 $\pm$0.82 |  85.03 $\pm$0.64

Table 6: Comparison to the baselines. We measure the classification accuracy
on test sets of the BFFHQ and BAR datasets with different bias severities. The
best accuracy values are in bold. The hyphen mark (-) means it is not
applicable. Results with standard deviations are provided in the
Supplementary.

##  Appendix F Additional quantitative results

###  F.1 Quantitative results with standard deviations

In Table 1 of our main paper, we report the quantitative comparison results
with classification accuracies on the test set which are averaged across five
independent experiments with different random seeds. We additionally provide
the standard deviations of the classification accuracies in Table 5 and Table
6. Each table shows the results of the synthetic dataset (_i.e_., Waterbirds)
and the real-world dataset (_i.e_., BFFHQ and BAR), respectively. Since the
BAR dataset lacks explicit bias labels, approaches such as LNL and EnD that
necessitate explicit bias labels are not applicable to the BAR dataset. The
baseline results for the BFFHQ and the BAR dataset are from the results
reported in BE [13] except for DCWP [16].

###  F.2 Comparison to recent baseline

Our primary contribution lies in providing the model with explicit spatial
guidance for intrinsic features by examining features that commonly appear in
bias-contrastive pairs. The intrinsic feature exists in generally appearing
features within a class, however, this property has not been tackled to
provide intrinsic feature guidance in prior studies to the best of our
knowledge. While recent debiasing approaches aim to encourage the model to
learn intrinsic features, they fail to directly indicate where the model
should focus to learn the features.

For instance, MaskTune [1] expects the model to learn intrinsic features by
fine-tuning the model with the data whose already-explored area is masked out
using GradCAM. However, simply exploring the unmasked area cannot inform the
model where exactly the intrinsic features are located. In this case, the
model may rather focus on non-intrinsic features during the fine-tuning. We
experiment on real-world datasets with a 1% bias severity: {58.00,
69.42}-MaskTune and {77.56, 75.14}-Ours for {BFFHQ, BAR}. Ours achieves better
debiasing performance by providing explicit spatial guidance for intrinsic
features based on common features in bias-contrastive pairs.

A recent pair-wise debiasing method $\mathcal{X}^{2}$-model [28] encourages
the model to retain intra-class compactness using samples generated via
feature-level interpolation between BC and BA samples. However,
$\mathcal{X}^{2}$-model does not inform the model where the intrinsic features
are located in the interpolated features. Simply making samples closer to the
interpolated samples does not assure the model to focus on the intrinsic
features. In contrast, our method directly encourages the model to focus on
the area of the intrinsic features.

Also, we conduct a quantitative comparison to the recently proposed debiasing
approach, DCWP [16], in Table 6. We use real-world datasets, BFFHQ and BAR,
with various bias severity. For a fair comparison, we utilize ResNet18, which
is the same architecture as ours. The ImageNet pretrained weight is employed
only for the BAR dataset. The results demonstrate the superiority of our
method over the DCWP, where ours provides the model with explicit guidance for
intrinsic features for debiasing, unlike DCWP.

###  F.3 Worst accuracy between the accuracy of BA and BC samples in
Waterbirds

BS |  Vanilla [5] |  HEX [25] |  LNL [9] |  EnD [22] |  ReBias [2] |  LfF [15] |  DisEnt [12] |  LfF+BE [13] |  DisEnt+BE [13] | Ours
---|---|---|---|---|---|---|---|---|---|---
0.5 |  24.08 $\pm$1.56 |  28.20 $\pm$3.07 |  26.08 $\pm$1.64 |  28.29 $\pm$3.53 |  27.00 $\pm$1.10 |  56.22 $\pm$6.07 |  38.07 $\pm$11.01 |  55.15 $\pm$2.78 |  36.60 $\pm$10.88 |  59.12 $\pm$3.67
1.0 |  24.78 $\pm$2.45 |  26.32 $\pm$2.90 |  29.72 $\pm$3.45 |  25.69 $\pm$2.41 |  27.95 $\pm$1.56 |  59.07 $\pm$3.40 |  47.02 $\pm$7.26 |  55.53 $\pm$1.60 |  28.35 $\pm$4.17 |  63.05 $\pm$1.97
2.0 |  34.39 $\pm$2.24 |  32.12 $\pm$2.89 |  33.92 $\pm$1.94 |  32.94 $\pm$1.48 |  32.16 $\pm$0.76 |  53.07 $\pm$6.74 |  44.93 $\pm$8.54 |  52.91 $\pm$2.62 |  31.08 $\pm$6.01 |  61.71 $\pm$4.94
5.0 |  38.34 $\pm$1.05 |  39.08 $\pm$0.92 |  43.22 $\pm$1.94 |  40.91 $\pm$1.11 |  39.72 $\pm$1.11 |  58.05 $\pm$2.37 |  52.96 $\pm$6.33 |  48.48 $\pm$3.72 |  37.92 $\pm$6.47 |  58.60 $\pm$3.32

Table 7: The worst accuracy between the accuracy of BA and BC samples in the
Waterbirds dataset. BS is bias severity.

To further analyze our model’s performance on the Waterbirds dataset, we
measure the accuracy of BA and BC samples separately, where the class accuracy
values are averaged. Then, we report the worst accuracy between them in Table
7. The results show that ours achieves the highest worst accuracy compared to
other baselines.

##  Appendix G Detailed description of datasets

Figure 9: Visualization of datasets used in the experiments. A group of three
columns represents each class for (a) Waterbirds-{Landbird, Waterbird} and (b)
BFFHQ-{Young, Old}, and each column of (c) BAR-{Climbing, Diving, Fishing,
Vaulting, Racing, Throwing} represents a distinct class. The samples above the
dashed line are bias-aligned samples and the below ones are bias-conflicting
samples.

We utilize Waterbirds [18], BFFHQ [10], and BAR [15] dataset. First, the
Waterbirds dataset is composed of two classes of bird images and has
background bias. In the training set, the waterbirds are mostly with the water
background and the landbirds are with the land background. The number of BA
samples and that of BC samples are balanced in the test set. By following
Sagawa et al. [18], we utilize two datsets, the Caltech-UCSD Birds-200-2011
(CUB) dataset [24] and the Places111CC BY dataset [30], to construct the
Waterbirds dataset. The bird images are segmented from the CUB dataset, and
the segmented birds are combined with the background images from the Place
dataset. We employ the code released by Sagawa et al.
[18]222https://github.com/kohpangwei/group_DRO for constructing the dataset.
As mentioned in the repository, a few landbirds (Eastern Towhees, Western
Meadowlarks, and Western Wood Pewees) in the original dataset are incorrectly
labeled as waterbirds. Therefore, we correct their labels to landbirds for the
experiments.

The BFFHQ is initially presented by Kim et al. [10] and constructed by
modifying the FFHQ dataset 333BY-NC-SA 4.0. In the BFFHQ, the bias attribute
is the gender and the intrinsic attribute is the age. Specifically, most of
the young people are female, and most of the old people are male in the
training dataset.

Lastly, the BAR dataset is introduced by Nam et al. [15]. The dataset contains
six action classes (_i.e_., Climbing, Diving, Fishing, Vaulting, Racing,
Throwing) and each class is biased to a certain background (_i.e_., RockWall,
Underwater, WaterSurface, Sky, APavedTrack, PlayingField). In the test set,
such correlations do not exist. For the experiments, we use the BFFHQ dataset
and BAR dataset released by Lee et al.
[13]444https://github.com/kakaoenterprise/BiasEnsemble. The examples of the BA
samples and BC samples in each dataset are shown in Fig. 9.

##  Appendix H Implementation details

Following the previous studies [15, 12, 13], we utilize ResNet18 [5] for the
biased model $f_{b}$ and the debiased model $f_{d}$. Also,
$f_{d}^{\text{emb}}$ indicates the subnetwork before the average pooling
layer, and $f_{d}^{\text{cls}}$ consists of an average pooling layer and a
linear classifier that outputs logits, where
$f_{d}(\mathbf{x})=f_{d}^{\text{cls}}\left(f_{d}^{\text{emb}}(\mathbf{x})\right)$.
Before training, $f_{b}$ and $f_{d}$ are initialized with the ImageNet
pretrained weight for the BAR dataset, while we randomly initialize the models
for the other datasets. This is because the size of the BAR dataset is
extremely small compared to the others [13].

During training $f_{d}$, we employ the sample reweighting value
$w(\mathbf{x})$ termed as relative difficulty score [15], as mentioned in Sec.
3.4 in the main paper. $w(\mathbf{x})$ is calculated as follows:

$w(\mathbf{x})=\frac{\mathcal{L}_{\text{CE}}(f_{b}(\mathbf{x}),y)}{\mathcal{L}_{\text{CE}}(f_{b}(\mathbf{x}),y)+\mathcal{L}_{\text{CE}}(f_{d}(\mathbf{x}),y)}.$
(14)

This score assigns a high weight to the BC samples and a low weight to the BA
samples. This encourages $f_{d}$ to mainly learn intrinsic features by
emphasizing BC samples with $w(\mathbf{x})$.

The models are trained for 50K iterations with a batch size of 64. The
horizontal flip and a random crop with a size of 224 are used for data
augmentation during the training. All the models are trained with the Adam
optimizer. The learning rate is set as 1e-4 for the Waterbirds and the BFFHQ
dataset, and 1e-5 for the BAR dataset. The hyperparameters of $\alpha_{l}$,
$\alpha_{s},$ and $\tau$ are set as $0.1$, $0.9$, and $2$, respectively, for
all the datasets. We apply class-wise max normalization to our BN score to
consider the different ranges of the scores across the classes for stability
of training.

During the training, we aim to select an auxiliary sample that has no bias
attribute but has the same class label with $\mathbf{x}$ as
$\mathbf{x}^{\text{BN}}$ from $\mathcal{D}^{\text{BN}}$. If no sample in
$\mathcal{D}^{\text{BN}}$ has the same label as $\mathbf{x}$, we select the
sample that has the same label with $\mathbf{x}$ from
$\mathcal{D}^{\text{BN}}_{\text{cand}}$. In a case where there’s no sample
with the same label as $\mathbf{x}$ in both $\mathcal{D}^{\text{BN}}$ and
$\mathcal{D}^{\text{BN}}_{\text{cand}}$, we sample $\mathbf{x}^{\text{BN}}$
that has the same label with $\mathbf{x}$ from $\mathcal{D}$.

As described in Sec. 3.1 in the main paper, we utilize the pretrained biased
models to construct $\mathcal{D}^{\text{A}}$, following the previous work
[13]. We utilize ResNet18 [5] for the pretrained biased models, and all the
pretrained biased models are randomly initialized before training. The models
are trained for 1K iterations with the generalized cross entropy (GCE) loss
[29]. Within each model, the samples with a high ground-truth probability
(i.e., $\geq$ 0.99) are considered as BA samples. Based on majority voting, we
collect the samples that are considered as the BA sample by the majority of
the models and construct the bias-amplified dataset $\mathcal{D}^{\text{A}}$.
We use five pretrained biased models following the study of Lee _et al_. [13].
Lee _et al_. demonstrate that adopting the additional biased models requires a
negligible amount of additional computational costs and memory space. Note
that the same biased models are utilized when constructing
$\mathcal{D}_{\text{cand}}^{\text{BN}}$.

##  Appendix I Limitations and future work

Although our BN score effectively encourages BC samples to be mainly adopted
as auxiliary inputs, the auxiliary inputs still can include a few BA samples,
as shown in Table 2 of Sec. 4.3 in the main paper. Accordingly, such BA
samples may interfere with the model to capture the intrinsic features.
Identifying intrinsic attributes without relying on auxiliary inputs can be
one promising future work.

In addition, since our IE weight is designed to enhance intrinsic features by
imposing spatially different values on the features, our method might be more
effective especially when bias attributes are located in different regions
from the intrinsic attributes. In this regard, applying channel-wise re-
weighting [6] to our approach will further improve the general applicability
of our method.

Despite the limitations above, we believe that our work poses the importance
of enhancing intrinsic attributes for debiasing.