# FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class
Associations

Ioannis Sarridis
Centre for Research and Technology Hellas
Thessaloniki, Greece
<EMAIL_ADDRESS>
&Christos Koutlis
Centre for Research and Technology Hellas
Thessaloniki, Greece
<EMAIL_ADDRESS>
&Symeon Papadopoulos
Centre for Research and Technology Hellas
Thessaloniki, Greece
<EMAIL_ADDRESS>
&Christos Diou
Harokopio University of Athens
Athens, Greece
<EMAIL_ADDRESS>

###### Abstract

Bias in computer vision systems can perpetuate or even amplify discrimination
against certain populations. Considering that bias is often introduced by
biased visual datasets, many recent research efforts focus on training fair
models using such data. However, most of them heavily rely on the availability
of protected attribute labels in the dataset, which limits their
applicability, while label-unaware approaches, i.e., approaches operating
without such labels, exhibit considerably lower performance. To overcome these
limitations, this work introduces FLAC, a methodology that minimizes mutual
information between the features extracted by the model and a protected
attribute, without the use of attribute labels. To do that, FLAC proposes a
sampling strategy that highlights underrepresented samples in the dataset, and
casts the problem of learning fair representations as a probability matching
problem that leverages representations extracted by a bias-capturing
classifier. It is theoretically shown that FLAC can indeed lead to fair
representations, that are independent of the protected attributes. FLAC
surpasses the current state-of-the-art on Biased MNIST, CelebA, and UTKFace,
by 29.1%, 18.1%, and 21.9%, respectively. Additionally, FLAC exhibits 2.2%
increased accuracy on ImageNet-A consisting of the most challenging samples of
ImageNet. Finally, in most experiments, FLAC even outperforms the bias label-
aware state-of-the-art methods.

_K_ eywords fairness $\cdot$ bias mitigation $\cdot$ mutual information

##  1 Introduction

Figure 1:  Distances between the central sample and the rest of the samples
belonging to Biased MNIST, a dataset that demonstrates a strong association
between the target labels (i.e., digits) and the protected attributes (i.e.,
colors). Subfigure on the left shows that using the standard task-specific
loss introduces bias to the representations which FLAC, the proposed approach,
can effectively mitigate as shown on the right. In particular, FLAC focuses on
situations where task-specific losses are susceptible to data bias, namely
pairs with the same target label but different protected attribute values
(i.e., $\mathcal{A}_{10}$) and pairs with different target labels and the same
protected attribute value (i.e., $\mathcal{A}_{01}$).

During the past decade, AI models have demonstrated exceptional performance in
a growing number of application areas; however, there have also been numerous
incidents where failures in the AI systems have disproportionately affected
certain individuals or groups of people [1]. Bias in Artificial Intelligence
(AI) typically refers to AI systems that demonstrate discriminatory behavior
(e.g., high errors) against groups or populations w.r.t. certain protected or
sensitive attributes that in several jurisdictions are legally protected from
discrimination, e.g., race, gender, age, and religion. This makes fairness and
non-discrimination an important ethical and often legal requirement for AI
models, and a prerequisite for their wide adoption and use in many real-world
applications [2, 3, 4, 5].

Bias can emerge in several Computer Vision (CV) systems, often in applications
involving the processing of face images but also in other domains where visual
data is involved [6]. Many research efforts focusing on face verification
fairness [7, 8, 9] highlight that AI models tend to be inaccurate for black
people, especially for black women [10]. In addition, similar biased behavior
against under-represented groups has been noticed in AI systems for face
recognition [10], facial expression recognition [11], and facial attribute
analysis [12].

Taking into account the importance of fairness in AI, several approaches for
learning fair representations have been proposed in recent years. The
underlying idea of such approaches is that if the representations of samples
belonging to different groups w.r.t. a protected attribute are similar, then
the classifier built on top of them will inevitably make decisions
irrespective of the group to which the samples belong. This serves as a means
for achieving demographic parity (statistical parity) between the groups of
different protected attribute values [13]. Many approaches make use of the
protected attribute labels (also known as bias labels in the literature)
provided in the training set [14, 15, 16, 17]. We refer to those as Bias
Label-Aware (BL-A) methods. Despite the effectiveness of such methods in terms
of bias mitigation, they can only be applied in a narrow range of problems and
datasets, due to their reliance on the availability of protected attribute
labels. To overcome this limitation, several recent efforts employ the
representations derived by a bias-capturing classifier for mitigating the bias
without being aware of the protected attribute labels [18, 19, 20, 21]. We
refer to these methods as Bias Label-Unaware (BL-U). The bias-capturing
classifier is a model that attempts to encapsulate information for the
protected attribute. In addition, BL-U approaches can be applied in cases
where bias is not categorical. For instance, representations with texture bias
can be derived by a CNN with small receptive fields [20]. However, the
performance of BL-U approaches has been shown to lag considerably behind the
one of BL-A methods [17, 16].

Mutual Information (MI) can be used to measure the dependence between the
representation provided by the main model and the protected attribute values.
By minimizing MI during representation learning, one can deter the main model
from using a given protected attribute to predict the label. In this way,
fairness can be achieved through a group-invariant representation, which is
known to guarantee accuracy parity between the groups defined by the protected
attribute [13]. However, solving the above MI minimization problem would
require access to the protected attributes (i.e., would need a BL-A method).
In this paper we introduce FLAC, a BL-U approach that leverages the
representations of a bias-capturing classifier to force an initial potentially
biased model to learn fairer representations. In particular, we cast the
reduction of the MI minimization problem into a simpler probability matching
problem between the similarity distributions of the main model and the bias-
capturing classifier. This turns out to be an effective means of
disassociating the target representation from the bias-capturing model and, as
a result, from the protected attributes. To this end, the proposed method
leverages the pairs of samples for which a typical task-specific loss is prone
to bias, namely samples sharing either only targets or only protected
attributes (see Figure 1). This is not the case for previous BL-U methods that
ignore the importance of a proper selection process, thereby impeding the bias
mitigation effectiveness. Furthermore, it is theoretically justified that FLAC
can minimize the mutual information between the main model representation and
protected attribute. In summary, the main contributions of this paper are the
following:

  * •

A fairness-aware representation learning approach that enables a model to
learn fair representations w.r.t. a protected attribute by leveraging the
representations of a bias-capturing classifier, without using the protected
attribute labels.

  * •

A condition for selecting the under-represented sample pairs that can
contribute most effectively to the bias mitigation procedure.

  * •

A wide comparative analysis involving 9 state-of-the-art approaches, 4
benchmark datasets, namely Biased MNIST [20], CelebA [22], UTK-Face [23], and
9-Class ImageNet [20]. In all the conducted experiments, FLAC surpasses the
current BL-U state-of-the-art by achieving +29.1%, +18.1%, +21.9%, and +2.2%
accuracy on Biased MNIST, CelebA, UTKFace, and 9-Class ImageNet, respectively.
Moreover, in most experiments, FLAC even outperforms the BL-A state-of-the-
art. Furthermore, we report the performance of FLAC in terms of three fairness
metrics measuring disparate impact and mistreatment (see Appendix B), which
are typically ignored by the related fairness-aware works in the literature.

The code is available at https://github.com/gsarridis/FLAC.

##  2 Related Work

In recent years, many approaches have emerged for mitigating bias in neural
networks. Some approaches focus on debiasing the training data by balancing
the available data across the different groups of interest [24], using more
data sources [25], applying data augmentation [26], and collecting more data
[27, 28]. However, such data-oriented approaches do not provide the
flexibility required in cases where fairness requirements are not static
(e.g., the need for considering a new protected attribute), while data
availability may also be limited. To address these shortcomings, many efforts
focus on developing approaches to mitigate bias while training a model using
biased data (i.e., in-processing approaches [29]). To achieve this, many works
propose ensembling [18, 30] or adversarial frameworks [31, 32, 14, 33, 34,
35], contrastive learning-based approaches [16, 17], and regularization
terms[15, 19, 16]. These in-processing bias mitigation approaches can be
divided into two primary categories that significantly impact their range of
applicability and performance: methods that necessitate access to the
protected attribute labels, i.e., BL-A, and methods that do not require such
labels, i.e., BL-U.

#### Bias mitigation with protected attribute labels

In [14] an adversarial approach is proposed, namely Learning Not to Learn
(LNL), based on the MI between feature embeddings and protected attribute
labels, that penalizes the model if it is capable of predicting the protected
attributes. Domain-Independent (DI) [30] suggests multiple classifier heads,
one per domain (i.e., bias type), for mitigating the bias in the feature
space. The approach of Entangling and Disentangling deep representations (EnD)
[15] proposes a regularization process that tries to entangle the feature
vectors of samples with the same target class and disentangle the features
representations of samples that share the same protected attribute label.
Given the representations derived by the main model, FairKL [17] aims to match
the distances of positive (negative) bias-conflicting and bias-aligned samples
from the anchor, where bias-conflicting and bias-aligned denote the under-
represented and the over-represented groups, respectively. When protected
attribute labels are not available, the similarities between the biased
features are utilized to assign weights to the FairKL regularizer. By contrast
to FairKL, FLAC solves a probability matching problem between the
representations of a bias-capturing classifier and the main model. Finally,
the Bias-Contrastive and Bias-Balance (BC-BB) method [16] constitutes a
contrastive learning-based approach for bias mitigation. Bias-Contrastive (BC)
encourages the main model to pull the samples with the same target but
different protected attributes closer in the feature space while ignoring the
bias-aligned samples with different target labels. Bias-Balance (BB) can be
used in combination with BC to further mitigate the bias by optimizing the
model toward the data distribution. Although these methods demonstrate state-
of-the-art performance in bias mitigation, they require the protected
attribute labels to work. This constitutes a crucial limitation, as they can
not be applied in many real-world settings, where protected attribute labels
are not available, or even worse, in cases where the bias is introduced by
non-categorical visual attributes (e.g., texture).

#### Bias mitigation without protected attribute labels

The significant limitation described above, led the research community to
explore methods that can address bias without being aware of the protected
attribute labels. In [18], the Learned-Mixin (LM) approach is introduced,
which proposes training the main model in an ensemble with the bias-capturing
model in order to discourage the main model to encode the information that has
been already captured by the bias-capturing model. Furthermore, Rubi [19]
suggests a regularization term for adjusting the weighting of logits, thereby
reducing the influence of biases during the training process. ReBias [20] is a
framework that aims at the independence between the representations of the
main model and the bias-capturing model using the Hilbert-Schmidt Independence
Criterion [36]. The authors of [21] suggest an approach, namely Learning from
Failure (LfF), that uses the generalized Cross-Entropy (CE) loss for training
the bias-capturing model to focus on simple samples that are likely to be
aligned with the bias and in parallel, the training procedure followed for the
main model encourages it to focus on the samples that the bias-capturing model
fails to learn, which are expected to be bias-conflicting. Finally, the Soft-
Contrastive (SoftCon) method [16] constitutes an extension of the BiasCon [16]
that makes use of the similarities of the representations extracted by the
bias-capturing model in order to enable BiasCon to be employed in scenarios
where the protected attribute labels are not available. Although these methods
successfully address the lack of protected attribute labels, they exhibit
considerably lower performance compared to BL-A methods, as they do not
consider appropriate mechanisms for focusing only on the samples that can
effectively contribute to the bias mitigation, thus adding noise and
preventing the model from learning fairer representations. In contrast, FLAC
is a BL-U method that minimizes the MI between the learned model and the
protected attributes. This is achieved through the use of a representation
derived by a bias-capturing classifier (instead of the attribute labels) and a
sampling strategy with optimality guarantees.

#### Mutual Information

MI has been used in various tasks, such as knowledge distillation for
preserving the teacher model’s knowledge [37, 38, 39], feature selection for
selecting the features that are most related to the desired outcome [40],
clustering for assessing the clustering quality [41], and generative models
for guiding generators in learning different manifolds [42]. LnL [14] proposed
using MI in an adversarial framework for bias mitigation, but its performance
is considerably lower than most of the bias mitigation approaches, due to the
instability that adversarial approaches exhibit. Moreover, [43] introduces a
statistical framework that penalizes the MI between the target labels and the
protected attribute labels. An effort of converting the Quadratic Mutual
Information (QMI) [44] problem to a probability matching problem for the
knowledge distillation task is presented in [38]. Inspired by this idea, we
propose in this paper a method that leverages the capabilities of MI in order
to effectively mitigate the bias, even in extreme data bias scenarios.

##  3 Methodology

###  3.1 Problem formulation

The problem of mitigating bias in network representations can be formulated as
follows. Let $(\mathbf{X}_{i},y_{i})$ be the $i$-th training sample of the
dataset $\mathcal{D}$, where $\mathbf{X}_{i}$ is the input image,
$y_{i}\in\mathcal{Y}$ the target, $h(\cdot)$ the model that we are interested
to improve in terms of bias, and $b(\cdot)$ the bias-capturing model. Model
$h(\cdot)$ is trained on the main task with targets $\mathcal{Y}$, while the
bias-capturing model, $b(\cdot)$, is trained to predict protected attributes
$t_{i}\in\mathcal{T}$ towards which the data (and consequently the model) is
biased. Note that $t_{i}$ is not provided by $\mathcal{D}$, thus $b(\cdot)$ is
trained on a different dataset (cf. Section 3.2). The representations
extracted by the penultimate layer of the network are denoted as
$\mathbf{h}_{i}$, while the corresponding bias-capturing classifier
representations are denoted as $\mathbf{b}_{i}$. If the dataset $\mathcal{D}$
consists of samples that exhibit strong dependence between $\mathcal{Y}$ and
$\mathcal{T}$, then training the model $h(\cdot)$ using only the task-specific
loss function (e.g. CE for the multi-class classification task or a
contrastive loss) will introduce bias into the model since it will rely on
features that encapsulate information about $t_{i}$ for predicting $y_{i}$.
Therefore, the goal is to eliminate the dependencies between representations
$\mathbf{h}$ and $\mathbf{b}$.

###  3.2 Bias-Capturing Classifier

In many real-world scenarios, the attributes introducing bias are not
categorical or the corresponding labels are not available. To address this
issue, we make use of a bias-capturing classifier, as in [20, 16, 17]. This
acts as a feature extractor that encapsulates information related to the
attributes that introduce bias to $h(\cdot)$.

If the protected attributes are categorical, the $b(\cdot)$ is trained on a
dataset $\mathcal{D}^{\prime}=\\{(\mathbf{X}^{\prime}_{i},t_{i})\\}$ in which
the protected attribute $t_{i}\in\mathcal{T}$ is provided (unlike
$\mathcal{D}$). For instance, if gender is the attribute related to bias, the
bias-capturing classifier is trained to predict genders. In some cases, the
model is biased towards attributes that are not categorical or explicit, e.g.,
texture or background. Following previous literature [20, 16, 17], we opt for
models trained on $\mathcal{D}$ that predict the task-specific targets
$\mathcal{Y}$, but using network architectures such as BagNets. For these
networks, the penultimate layer is expected to adequately capture the
attributes of interest [20].

###  3.3 FLAC

Discouraging the main model from learning to predict the protected attribute
labels can be achieved by minimizing $\mathcal{I}(\mathbf{h}_{i},t_{i})$,
where $\mathcal{I}(\cdot,\cdot)$ denotes MI. However, this is a task of high
complexity and additionally it would require access to the protected attribute
labels $\mathcal{T}$. Instead, we propose to use the pairwise similarity
between the model representation and the representation derived by a bias-
capturing classifier, i.e., a predictor of $t_{i}$, to minimize
$\mathcal{I}(\mathbf{h}_{i},t_{i})$. More specifically, the target is to match
the probability distributions of distances derived by the bias-capturing
classifier features with the distributions of similarities derived by the
representations of the main model for a certain subset of sample pairs. By
doing so, we bring the samples with different protected attribute labels
(i.e., $t_{i}\neq t_{j}$) and the same target label (i.e., $y_{i}=y_{j}$)
closer to each other while increasing the distance between samples with the
same protected attribute label (i.e., $t_{i}=t_{j}$) and different target
labels (i.e., $y_{i}\neq y_{j}$). For all possible pairs of batch samples’
indices $(i,j)$ we define the following sets:

$\displaystyle\mathcal{A}_{10}$ $\displaystyle=\\{(i,j)\mid y_{i}=y_{j}\land
t_{i}\neq t_{j}\\}$ $\displaystyle\mathcal{A}_{01}$
$\displaystyle=\\{(i,j)\mid y_{i}\neq y_{j}\land t_{i}=t_{j}\\}$
$\displaystyle\mathcal{A}_{11}$ $\displaystyle=\\{(i,j)\mid y_{i}=y_{j}\land
t_{i}=t_{j},i\neq j\\}$ $\displaystyle\mathcal{A}_{00}$
$\displaystyle=\\{(i,j)\mid y_{i}\neq y_{j}\land t_{i}\neq t_{j}\\}$
$\displaystyle\mathcal{A}$
$\displaystyle=\mathcal{A}_{10}\cup\mathcal{A}_{01}\cup\mathcal{A}_{11}\cup\mathcal{A}_{00}$

Then, involved pairs of samples should satisfy the following condition:

$(y_{i}=y_{j}\land t_{i}\neq t_{j})\lor(y_{i}\neq y_{j}\land t_{i}=t_{j})$ (1)

forming the set $\mathcal{S}=\mathcal{A}_{10}\cup\mathcal{A}_{01}$. Note that
involving all the possible sample pairs could have an adverse effect on the
main model as it would lead to reducing (increasing) the similarity of samples
with the same (different) target labels (see Appendix A). However,
$\mathcal{T}$ labels are not available, thus a criterion needs to be defined
for inferring whether two samples share the same protected attribute label.
Let $K(\cdot)$ be a kernel function and $\mathcal{A}$ the set of all the
possible pairs of samples, then the protected attribute equality is determined
by:

$K(\mathbf{b}_{i},\mathbf{b}_{j})>\frac{\max\limits_{u,v\in\mathcal{A}}{K(\mathbf{b}_{u},\mathbf{b}_{v})}+\min\limits_{u,v\in\mathcal{A}}{K(\mathbf{b}_{u},\mathbf{b}_{v})}}{2}\Rightarrow
t_{i}=t_{j},~{}(i,j)\in\mathcal{A}$ (2)

otherwise $t_{i}\neq t_{j}$. It should be noted that the proposed methodology
will not be negatively impacted even if $K(\mathbf{b}_{i},\mathbf{b}_{j})$
falls below or exceeds this threshold for $(i,j)$ with $t_{i}=t_{j}$ or
$t_{i}\neq t_{j}$, respectively. This is because the objective remains to
enhance the similarity of $(i,j)$ where $y_{i}=y_{j}$ or reduce it when
$y_{i}\neq y_{j}$.

Then, the task-specific network’s, $h(\cdot)$, probability distributions of
the pairwise similarities can be defined as follows:

$p_{i\mid
j}^{(h)}=\frac{K(\mathbf{h}_{i},\mathbf{h}_{j})}{\sum_{k:(k,j)\in\mathcal{S}}K(\mathbf{h}_{k},\mathbf{h}_{j})}\in[0,1],~{}(i,j)\in\mathcal{S}.$
(3)

Accordingly, for the bias-capturing model, we calculate the probability
distributions of dissimilarities (i.e., ($1-K(\cdot)$):

$p_{i\mid
j}^{(b)}=\frac{1-K(\mathbf{b}_{i},\mathbf{b}_{j})}{\sum_{k:(k,j)\in\mathcal{S}}1-K(\mathbf{b}_{i},\mathbf{b}_{j})}\in[0,1],~{}(i,j)\in\mathcal{S}.$
(4)

As regards the similarity kernel function, in this work we opted for the
student’s t kernel that demonstrates good performance on classification tasks
[37]:

$K(\mathbf{a},\mathbf{b})=\frac{1}{1+||\mathbf{a}-\mathbf{b}||_{2}}.$ (5)

Having calculated the probability distributions of Equations (3) and (4) for
the pairs of samples in $\mathcal{S}$, our goal is to train a model that
demonstrates high (low) similarity for the sample pairs that the bias-
capturing classifier exhibit low (high) similarity. The divergence between
these distributions can be calculated using Jeffreys divergence [45], a
symmetric version of Kullback-Leibler divergence:

$\mathcal{L}_{FLAC}=\sum_{(i,j)\in\mathcal{S}}(p_{i\mid j}^{(b)}-p_{i\mid
j}^{(h)})\cdot(\log p_{i\mid j}^{(b)}-\log p_{i\mid j}^{(h)}).$ (6)

Then, the final loss can be defined as follows:

$\mathcal{L}=\mathcal{L}_{task}+\alpha\cdot\mathcal{L}_{FLAC},$ (7)

where $\alpha$ is a hyperparameter and $\mathcal{L}_{task}$ is the cost
function of the task (e.g., the CE or a contrastive loss). Note that FLAC fits
better with pairwise losses as $\mathcal{L}_{task}$, due to its dependency on
pairwise similarities. This is described in detail in Appendix A where, we
provide a theoretical analysis for justifying that FLAC results in minimizing
$\mathcal{I}(\mathbf{h}_{i},t_{i})$. Figure 2 presents the proposed framework.

Figure 2: Illustration of the proposed framework.

##  4 Experimental setup

###  4.1 Datasets

#### Datasets with Artificially Injected Bias

Biased-MNIST constitutes a biased version of the original MNIST dataset [46],
introduced in [20] as a standard benchmark for evaluating the effectiveness of
bias mitigation methods. It consists of digits with colored backgrounds (10
colors in total). The bias is introduced through the association of each digit
with a certain color. The probability of a digit having biased background is
denoted as $q$, while a random background color is assigned with probability
$1-q$. This way, higher $q$ values lead to a stronger association between
digits and colors and thus more biased data. Following previous works using
this dataset, here we consider four variations of the Biased-MNIST w.r.t.
values of $q$, namely 0.99, 0.995, 0.997, and 0.999.

#### Established computer vision datasets

We also evaluate the proposed approach on three established computer vision
datasets, namely CelebA [22], UTK-Face [23], and 9-Class ImageNet [20]. As
regards the CelebA dataset that consists of more than 200,000 facial images
annotated with 40 binary attributes, the gender is the protected attribute,
while the HeavyMakeup and the BlondHair constitute the two target labels, as
in [16]. For the UTKFace, which is a dataset consisting of over 20,000 facial
images with gender, race, and age labels, we consider the gender as the target
label, while race and age as the protected attributes. For UTKFace, we enforce
a dependence of 90% between the target and the protected attribute following
the experimental setup of [16]. The 9-Class ImageNet, a subset of the ImageNet
dataset [47] consisting of 9 super-classes, is employed to evaluate the
proposed method in settings where bias is not explicitly associated with a
given attribute (e.g., texture bias). In addition, the ImageNet-A [48], which
comprises ImageNet samples that Vanilla models often fail to classify, is also
involved in the experiments as a test set.

###  4.2 Model architectures

For the controlled experiments conducted for Biased-MNIST, we employ the CNN
architecture proposed in [20], namely SimpleConvNet that consists of four
convolutional layers with 7$\times$7 kernels and a fully connected layer with
128 neurons. For the CelebA, UTKFace, and 9-Class ImageNet, we employ the
Resnet-18 [49] architecture for both the main and the bias-capturing models,
except for the 9-Class ImageNet, where we use the BagNet-18 [50] as the bias-
capturing model following the practice of [20, 16, 17]. BagNets mainly consist
of 1$\times$1 convolutions and they are found to be more prone to texture bias
[20]. Note that all model selections are based on the previous literature to
ensure comparability.

###  4.3 Baseline methods

The proposed method is compared to 9 state-of-the-art approaches. Of those,
BL-U approaches are directly comparable to FLAC and include the LM [18], Rubi
[19], ReBias [20], LfF [21], and SoftCon [16]. However, the exceptional
performance of FLAC led us to also include BL-A approaches in our comparative
study, namely LNL [14], EnD [15], BC-BB [16], and FairKL [17], even if this
comparison can be considered as unfair against our proposed method. More
details about the competing methods can be found in Section 2.

###  4.4 Implementation details and evaluation protocol

We employ the Adam optimizer for all the experiments with an initial learning
rate of 0.001 that decays by a factor of 0.1 at 1/3 and 2/3 of the total
training epochs, the weight decay is equal to $10^{-4}$ and the batch size is
set to 128, unless stated otherwise. For the Biased-MNIST, models are trained
for 80 epochs. The values of $\alpha$ for each BiasedMNIST variant, 0.99,
0.995, 0.997, and 0.999 are 110, 1500, 2000, and 10,000 respectively. Note
that such large values of $\alpha$ were selected due to the large discrepancy
between the values of $\mathcal{L}_{FLAC}$ and $\mathcal{L}_{task}$. The
augmentations applied for the established computer vision datasets (i.e.,
CelebA, UTKFace, and 9-Class ImageNet) are the random resized crop and random
horizontal flip. For the CelebA, models are trained for 40 epochs with
$\alpha$ equal to 20,000 for the HeavyMakeup classification task, while 10
training epochs with $\alpha$ equal to 30,000 are used for the BlondHair
classification task. The images are resized to 224$\times$224\. For UTKFace,
models are trained for 20 epochs, $\alpha$ is set to 1,000 and image size to
64$\times$64\. For 9-Class ImageNet, the cosine annealing learning rate
scheduler is used, the number of training epochs is equal to 250, $\alpha$ is
set to 1,000, and image size to 224$\times$224\. As $\mathcal{L}_{task}$ we
consider both the Supervised Contrastive (SupCon) [51] and the CE loss across
all the experiments in order to provide a fair comparison between the
competing bias mitigating methods. All the experiments were conducted on a
single NVIDIA RTX-3090 GPU and repeated for 5 different random seeds (all
results reported in Section 5 refer to the corresponding mean scores).

The test set used for BiasedMNIST evaluation has $q=0.1$ in order to be
unbiased (i.e., no association between digits and colors). For CelebA and
UTKFace, we use the unbiased and bias-conflict test sets provided by [16] for
the evaluation. Finally, for the 9-Class ImageNet experiments, two test sets
are involved, (a) the official 9-Class ImageNet test set and (b) the
ImageNet-A. For all the above, accuracy is employed as an evaluation metric.
In addition, we measure the unbiased accuracy on 9-Class ImageNet test set by
using the texture bias annotations provided by [20] and averaging the accuracy
calculated for each one of the texture groups. Furthermore, apart from the
typical performance metric (i.e, accuracy) we also report the performance of
FLAC in terms of three fairness metrics for disparate impact and disparate
mistreatment, namely p% rule [52], Difference in False Positive Rates (DFPR),
and Difference in False Negative Rates (DFNR) [53, 54] (see Appendix B).

##  5 Results

###  5.1 Controlled experiments

Table 1 presents the performance comparison of the proposed approach against 5
BL-U and 4 BL-A methods. Vanilla refers to the model performance using only
the CE loss without considering any bias mitigation algorithm. According to
Table 1, the proposed method consistently outperforms all competing methods,
even the BL-A ones. In particular, FLAC enhances the accuracy by 0.1%-0.6% for
the different $q$ values compared to the best performing BL-A method and by
3.5%-29.1% compared to the best performing BL-U approach. Furthermore, it is
worth noting that while $q$ increases, FLAC is capable of maintaining very
high accuracy, which is not the case for most of the compared methods. For
instance, for the extreme $q$ value of 0.999, FLAC achieves 94.1% accuracy,
while the best-performing BL-U method achieves 65%. In addition, note that
FLAC combined with CE loss significantly outperforms the corresponding
competing methods that use CE as $\mathcal{L}_{task}$. As regards the
selection of $\mathcal{L}_{task}$, as expected opting for a pairwise loss
(i.e., SupCon) results in improved accuracy as justified in Appendix A.

As shown in Figure 3 (a), which presents the distributions (with $q=0.99$)
derived by Equation (3) for Vanilla and FLAC the proposed method increases
(decreases) the similarities of samples with the same (different) target
labels and different (same) protected attribute labels compared to Vanilla.
Furthermore, for $q=0.999$, where Vanilla results in a model that learned the
background colors instead of the digits, FLAC is capable of learning the
proper distributions as depicted in Figure 3 (b).

Table 1: Evaluation on Biased MNIST for different bias levels. Underlined and dotted-underlined values refer to the second best performing BL-U and the best performing BL-A methods, respectively. Con. refer to Contrastive losses. methods | $L_{task}$ | BL-U | $q$
---|---|---|---
0.99 | 0.995 | 0.997 | 0.999
Vanilla | CE | ✓ | 88.9 | 74.5 | 57.2 | 11.8
LNL | CE | ✗ | 86.0 | 72.5 | 57.2 | 18.2
EnD | CE | ✗ | 96.0 | 93.9 | 83.7 | 52.3
BC-BB | Con. | ✗ | 98.1 | 97.7 | 97.3 | 94.0
FairKL | CE | ✗ | 95.9 | 94.8 | 93.9 | 79.9
FairKL | Con. | ✗ | 97.9 | 97.0 | 96.2 | 90.5
LM | CE | ✓ | 91.5 | 80.9 | 56.0 | 10.5
Rubi | CE | ✓ | 85.9 | 71.8 | 49.6 | 10.6
ReBias | CE | ✓ | 88.4 | 75.4 | 65.8 | 26.5
LfF | CE | ✓ | 95.1 | 90.3 | 63.7 | 15.3
SoftCon | Con. | ✓ | 95.2 | 93.1 | 88.6 | 65.0
FLAC | CE | ✓ | 96.9 | 94.9 | 94.1 | 89.3
FLAC | Con. | ✓ | 98.7 | 98.2 | 97.8 | 94.1

(a)  Distributions of Equation (3) for Vanilla and FLAC trained on Biased
MNIST with $q=0.99$ and achieves 88.9% and 96.9% accuracy, respectively. Using
FLAC, the similarity between samples with the same or different label is
increased or decreased respectively, compared to Vanilla.

(b) Distributions of Equation (3) for Vanilla (11.8% acc.) and FLAC (89.3%
acc.) trained on Biased MNIST with $q=0.999$. In this evaluation setup,
Vanilla results in an extremely biased model that exhibits high (low)
similarities for samples with the same (different) protected attribute label
and different (same) target. Here, the FLAC’s impact is significant as it
manages to flip the distributions.

Figure 3: The distributions of similarities or/and distances for the samples
that satisfy Equation (1). Red color represents the sample pairs with the same
target, but different protected attribute label, while blue depicts the pairs
with different target, but the same protected attribute labels.

###  5.2 Evaluation on established datasets

Table 2 compares the methods for the two tasks of CelebA, namely predicting
BlondHair and HeavyMakeup. The unbiased and the bias-conflict refer to test
sets that have balance bias-aligned and bias-conflict samples and only bias-
conflict samples, respectively. As regards the BlondHair prediction task, the
proposed method outperforms the best competing BL-U method by 7% and 7.5% on
the unbiased and the bias-conflict test sets, respectively. In addition, FLAC
manages to outperform even the best competing BL-A method by 1.5% on the bias-
conflict test set, while demonstrating competitive performance on the unbiased
test set (i.e., -0.2%). As regards the most challenging task, namely
HeavyMakeup, the proposed method significantly outperforms both the BL-A and
the BL-U approaches. In particular, FLAC achieves 85.4% and 79.1% accuracy on
the unbiased and the bias-conflict test sets, while the best performing BL-A
method only manages to get 78.6% and 63.5% in terms of accuracy, respectively.
Here, FLAC with CE demonstrates higher performance compared to FLAC combined
with SupCon loss.

Table 2: Evaluation of the proposed method on CelebA for two different target attributes, namely HeavyMakeup and BlondHair, with gender as the protected attribute. methods | $L_{task}$ | BL-U | target
---|---|---|---
BlondHair | HeavyMakeup
unbiased | bias-conflict | unbiased | bias-conflict
Vanilla | CE | ✓ | 79.0 | 59.0 | 76.0 | 55.2
LNL | CE | ✗ | 80.1 | 61.2 | 76.4 | 57.2
DI | CE | ✗ | 90.9 | 86.3 | 74.3 | 53.8
EnD | CE | ✗ | 86.9 | 76.4 | 74.8 | 53.3
BC-BB | Con. | ✗ | 91.4 | 87.2 | 78.6 | 63.5
LfF | CE | ✓ | 84.2 | 81.2 | 66.2 | 45.5
SoftCon | Con. | ✓ | 84.1 | 74.4 | 77.4 | 61.0
FLAC | CE | ✓ | 90.1 | 87.6 | 85.4 | 79.1
FLAC | Con. | ✓ | 91.2 | 88.7 | 84.7 | 78.8

Table 3 presents the results of FLAC compared to other methods on the UTKFace
dataset with race and age as protected attributes. For both protected
attributes, the proposed method surpasses the best performing competing
methods on both the unbiased (i.e., +1% and +1.5%) and bias-conflict test sets
(i.e., 3% and 9.4%), while it outperforms the state-of-the-art BL-U methods by
5%-6% and 12%-21.9% on the unbiased and bias-conflict test sets, respectively.
Here, it is noteworthy that FLAC exhibits very high accuracy (i.e., 81.1%) on
the most challenging test set (i.e., bias-conflict test set with age as the
protected attribute), whereas the best BL-U method only achieves 59.2%.
Furthermore, Figure 4 visualizes the 8 most similar samples to one query image
for each minority group (i.e., non-white males and white females) for Vanilla
and FLAC. By inspecting Figures 4(a) and 4(b), we may observe that given a
query depicting a non-white male, Vanilla’s similar representations correspond
mostly to people of color (both males and females) while retrieved images
based on the FLAC’s representations depict males of various races, i.e., they
do not encode racial information. Similarly, Figures 4(c) and 4(d) present the
retrieved images given a white female query for Vanilla and FLAC,
respectively. Here, the Vanilla similar images consist of mostly non-white
males, while the images retrieved using FLAC depict females of several races.
The visualizations provided in Figure 4 indicate the effectiveness of FLAC in
(i) disassociating the representations from the race attribute and (ii)
bringing the representations with the same gender closer to each other.

(a) Method: Vanilla, Query image: non-white male

(b) Method: FLAC, Query image: non-white male

(c) Method: Vanilla, Query image: white female

(d) Method: FLAC, Query image: white female

Figure 4: The top 8 images that are most similar to queries representing the minority groups of UTKFace (i.e., non-white males and white females) based on the representations derived by Vanilla and FLAC approaches. Images with green borders denote the query samples. Table 3: Evaluation of the proposed method on UTKFace for 2 different protected attributes, namely race and age, with gender as the target attribute. methods | $L_{task}$ | BL-U | bias
---|---|---|---
race | age
unbiased | bias-conflict | unbiased | bias-conflict
Vanilla | CE | ✓ | 87.4 | 79.1 | 72.3 | 46.5
LNL | CE | ✗ | 87.3 | 78.8 | 72.9 | 47.0
DI | CE | ✗ | 88.9 | 89.1 | 75.6 | 60.0
EnD | CE | ✗ | 88.4 | 81.6 | 73.2 | 47.9
BC-BB | Con. | ✗ | 91.0 | 89.2 | 79.1 | 71.7
SoftCon | Con. | ✓ | 87.0 | 80.2 | 74.6 | 59.2
FLAC | CE | ✓ | 90.9 | 90.2 | 78.7 | 81.1
FLAC | Con. | ✓ | 92.0 | 92.2 | 80.6 | 71.6

The last evaluation scenario does not refer to societal biases, but biases
that are introduced from the image’s background or texture. Table 4 presents
the results of the experiments conducted on the 9-Class ImageNet dataset. In
this scenario, the bias is not categorical, thus only BL-U methods can be
applied. As presented in Table 4, FLAC outperforms the state-of-the-art on all
three test sets. As in previous experiments, FLAC demonstrates 2.2%
improvements in terms of accuracy on the most challenging test set (i.e.,
ImageNet-A).

Table 4: Evaluation of the proposed method on the biased and the unbiased 9-Class ImageNet test sets and the ImageNet-A. FairKL-lu refers to the BL-U version of FairKL. methods | $\mathcal{L}_{task}$ | biased | unbiased | ImageNet-A
---|---|---|---|---
Vanilla | CE | 94.0 | 92.7 | 30.5
LM | CE | 79.2 | 76.6 | 19.0
Rubi | CE | 93.9 | 92.5 | 31.0
ReBias | CE | 94.0 | 92.7 | 30.5
LfF | CE | 91.2 | 89.6 | 29.4
SoftCon | Con. | 95.3 | 94.1 | 34.1
FairKL-lu | Con. | 95.1 | 94.8 | 35.7
FLAC | CE | 95.5 | 95.2 | 37.6
FLAC | Con. | 95.7 | 95.2 | 37.9

##  6 Ablation study

Figure 5: The frequency of the number of sample pairs belonging to
$\mathcal{S}$ per batch ($N=128$) for Biased MNIST training set with
$p\in\\{0.99,0.995,0.997,0.999\\}$.

FLAC is designed to be applied only on the meaningful pairs of samples (see
Appendix A), which is not the case for other BL-U approaches. Particularly,
other works [16, 17] involve all the possible sample pairs by assigning
weights based on their similarities. However, the meaningful pairs of samples
are only a very small subset of the set of all the possible pairs , especially
in extremely biased data, such as the Biased MNIST with $q=0.999$ or the
CelebA with HeavyMakeup as target. As presented in Figure 5, the maximum
number of sample pairs in the Biased MNIST with $q=0.99$ that satisfy Equation
(1) is 366 (i.e., only 0.022% of all pairs), while the number of all the
possible sample pairs is $128^{2}$. Involving all sample pairs in the bias
mitigation procedure makes competing BL-U methods much more sensitive to
scenarios where only few sample pairs in the dataset are responsible for the
emergence of bias. Table 5 compares the performance of FLAC and two state-of-
the-art BL-U approaches, namely SoftCon [16] and FairKL-lu [17], w/ and w/o
leveraging the sampling based on Equation (1). It becomes obvious that using
Equation (1) can significantly improve the performance of other BL-U methods,
while FLAC w/ Equation (1) still exhibits the best performance. Furthermore,
the importance of involving both terms of Equation (1) as analyzed in Section
A is experimentally confirmed in Table 6, which presents the performance of
FLAC using only the first (i.e., pairs with the same target and different
protected attribute label) or the second term (i.e., pairs with different
target and same protected attribute label) of Equation (1).

Table 5: Performance of state-of-the-art BL-U methods w/o or w/ adopting Equation (1). Results pertain to BiasedMNIST with $q=0.999$. method | $\mathcal{L}_{task}$ | accuracy w/o Eq. (1) | accuracy w/ Eq. (1)
---|---|---|---
SoftCon | Con. | 65.0 | 84.0
FairKL-lu | Con. | 13.7 | 73.9
FLAC | CE | 24.1 | 89.3
FLAC | Con. | 28.2 | 94.1
Table 6: The impact of the two terms of Equation (1) on the performance of FLAC. Results pertain to BiasedMNIST with $q=0.999$. Equation (1) variants | acc. (CE) | acc. (Con.)
---|---|---
$y_{i}\neq y_{j}\land t_{i}=t_{j}$ | 10.1 | 29.1
$y_{i}=y_{j}\land t_{i}\neq t_{j}$ | 10.5 | 20.6
$(y_{i}=y_{j}\land t_{i}\neq t_{j})\lor(y_{i}\neq y_{j}\land t_{i}=t_{j})$ | 89.3 | 94.1
Table 7: The impact of kernel function on the performance of FLAC. Results pertain to BiasedMNIST with $q=0.999$. kernel | accuracy (CE) | accuracy (Con.)
---|---|---
cosine | 60.4 | 82.8
RBF | 83.4 | 90.7
student’s t | 89.3 | 94.1
Table 8: Performance of FLAC with MSE, Kullback-Leibler, and Jeffreys divergence as loss functions. Results pertain to BiasedMNIST with $q=0.999$. loss | accuracy (CE) | accuracy (Con.)
---|---|---
MSE | 12.1 | 85.6
Kullback-Leibler divergence | 85.2 | 89.2
Jeffreys divergence (Eq. (6)) | 89.3 | 94.1
Table 9: The impact of FLAC when the training data is unbiased. Results for Biased MNIST with $q=0.1$. method | accuracy (CE) | accuracy (Con.)
---|---|---
Vanilla | 99.3 | -
FLAC@$\alpha$=1 | 99.4 | 99.3
FLAC@$\alpha$=100 | 99.1 | 99.1
FLAC@$\alpha$=1000 | 98.2 | 98.9

Table 7 presents the impact of several kernel functions on the performance of
the proposed method. The kernel function should be carefully selected as it
can significantly affect the model’s performance. For instance, although the
cosine kernel fits well in retrieval tasks, it is not effective enough on
classification tasks as previous studies have noticed [37], which is also
reflected in Table 7. Furthermore, Radial Basis Function (RBF) kernels
constitute a common kernel option, but tuning them is often difficult.
Finally, the student’s t kernel, which is considered as a good option for
classification tasks, is shown to achieve the best performance compared to
other kernels in the conducted experiments.

Table 8 presents the evaluation of FLAC using different loss functions, namely
the MSE, Kullback-Leibler divergence, and Jeffreys divergence. As expected,
using a common regression loss, such as MSE, is not a good choice to mitigate
bias, while Kullback-Leibler and Jeffreys divergences lead to highly accurate
models, with Jeffreys divergence leading to the best results.

In order to investigate whether the proposed method can have a negative impact
on the model’s performance when the training data is unbiased, we further
evaluate the proposed method on a dataset that does not suffer from any
attribute-label associations (i.e., Biased MNIST with $q=0.1$). Table 9
presents the performance of FLAC for $\alpha\in\\{1,100,1000\\}$: for
$\alpha=1$ and $\alpha=100$ FLAC does not affect the model’s performance,
while for the extreme value $\alpha=1000$ only a small drop in accuracy is
noticed.

##  7 Conclusion

In this paper, we introduce FLAC, a bias mitigation approach that leverages
the representations of a bias-capturing classifier for enabling the main model
to learn fair representations without being aware of the protected attribute
labels. In particular, FLAC aims to minimize the MI between the main model’s
representations and the protected attribute without taking advantage of the
protected attribute labels. To this end, FLAC only leverages the under-
represented samples that can effectively contribute to minimizing MI. By doing
so, the derived fair representations do not capture any information related to
the protected attributes. The proposed approach demonstrates superior
performance compared to state-of-the-art in a wide range of experiments on
both datasets with artificially injected bias and established computer vision
datasets. Considering the importance of AI systems to be fair in terms of
multiple protected attributes, as a future work, we consider enabling FLAC to
be applied in a multi-attribute fairness scenario. Furthermore, taking into
account that bias is uniformly distributed across classes in the existing
fairness benchmarks, another future work could be the exploration of fairness-
aware approaches performance on datasets with non-uniform bias distribution.
Finally, using FLAC for tasks beyond classification ( e.g., retrieval), is a
potential subject for future work.

## Acknowledgments

This research was supported by the EU Horizon Europe project MAMMOth (Grant
Agreement 101070285).

##  Appendix A Theoretical Analysis

Let us denote with $\mathcal{B}=\\{\mathbf{X}_{i}\\}_{i=1}^{N}$ a batch with
$N$ samples. Following the notation of Section 3.3, we additionally consider
the set $\Omega$ as a placeholder variable that upon declaration equals either
$\mathcal{S}$ or $\mathcal{A}$. Finally, we assume that for a well-trained
bias capturing classifier $\exists\epsilon>0$, such that
$\forall(i,j)\in\Omega$:

$t_{i}\neq t_{j}\Rightarrow K(\mathbf{b}_{i},\mathbf{b}_{j})\in[0,\epsilon)$
(8) $t_{i}=t_{j}\Rightarrow K(\mathbf{b}_{i},\mathbf{b}_{j})\in(1-\epsilon,1]$
(9)

When the contrastive loss, denoted by $\mathcal{L}_{task}$, is minimized after
training, then
$\exists\delta_{10},\delta_{01},\delta_{11},\delta_{00},\epsilon^{\prime}>0$
with $\epsilon^{\prime}\leq\delta_{10},\delta_{01},\delta_{11},\delta_{00}$,
such that111 It has been shown that optimizing the standard cross-entropy
loss: (i) corresponds to an approximate bound-optimizer of an underlying
pairwise loss [55] and (ii) produces a feature space with inter-class
distances being greater than intra-class ones [56]. Nevertheless,
accomplishing low $\delta_{10}$, $\delta_{01}$, $\delta_{11}$, $\delta_{00}$
applying to all $i,j$ is hardly possible. Hence, adopting the cross-entropy
will only approximate the optimality derived by the contrastive loss.:

$K(\mathbf{h}_{i},\mathbf{h}_{j})\in\left\\{\begin{array}[]{ll}(1-\delta_{10}-\epsilon^{\prime},1-\delta_{10}+\epsilon^{\prime}]\text{,
if }(i,j)\in\mathcal{A}_{10}\\\
[\delta_{01}-\epsilon^{\prime},\delta_{01}+\epsilon^{\prime})\text{, if
}(i,j)\in\mathcal{A}_{01}\\\
(1-\delta_{11}-\epsilon^{\prime},1-\delta_{11}+\epsilon^{\prime}]\text{, if
}(i,j)\in\mathcal{A}_{11}\\\
[\delta_{00}-\epsilon^{\prime},\delta_{00}+\epsilon^{\prime})\text{, if
}(i,j)\in\mathcal{A}_{00}\\\ \end{array}\right.$ (10)

However, in cases of existing bias in which the model has learned unwanted
shortcuts (i.e., $\mathbf{h}_{i}\not\\!\perp\\!\\!\\!\perp t_{i}$), the
magnitude of $\delta_{10},\delta_{01}$ monotonically depends on the amount of
shared information between $\mathbf{h}_{i}$ and $t_{i}$222If the shared
information level is low, then there exist very small
$\delta_{10},\delta_{01}$. If the shared information level is high, then the
$\delta_{10},\delta_{01}$ can indeed be high but the cardinality of
$\mathcal{A}_{10}$ and $\mathcal{A}_{01}$ is very small, much smaller than the
cardinality of $\mathcal{A}_{11}$ and $\mathcal{A}_{00}$ (cf. Section 6 &
Figure 5)., while the magnitude of $\delta_{11},\delta_{00}$ not. This
indicates the need to control $\delta_{10},\delta_{01}$ with an additional
loss term that acts at least on $(i,j)\in\mathcal{S}$333An analysis showing
that acting only on $\mathcal{S}$ is the best option in comparison to acting
on the whole space $\mathcal{A}$, is considered below.. To this end, in this
work we propose the consideration of an additional loss term, namely
$\mathcal{L}_{FLAC}$, defined by Equation (6), which will be minimized jointly
with $\mathcal{L}_{task}$. This, then implies the following:

$\begin{split}\mathcal{L}_{FLAC}&=0\Rightarrow\\\ \Rightarrow(p_{i\mid
j}^{(b)}-p_{i\mid j}^{(h)}=0)&\lor(\log p_{i\mid j}^{(b)}-\log p_{i\mid
j}^{(h)}=0)\Rightarrow\\\ \Rightarrow p_{i\mid j}^{(b)}&=p_{i\mid
j}^{(h)}\Rightarrow\\\
\Rightarrow\frac{K(\mathbf{h}_{i},\mathbf{h}_{j})}{\sum_{k:(k,j)\in\Omega}K(\mathbf{h}_{k},\mathbf{h}_{j})}&=\frac{1-K(\mathbf{b}_{i},\mathbf{b}_{j})}{\sum_{k:(k,j)\in\Omega}1-K(\mathbf{b}_{k},\mathbf{b}_{j})}\Rightarrow\\\
\Rightarrow
K(\mathbf{h}_{i},\mathbf{h}_{j})&=\alpha(j)\cdot(1-K(\mathbf{b}_{i},\mathbf{b}_{j}))\end{split}$
(11)

where $(i,j)\in\Omega$. Considering for simplicity
$\epsilon\approx\epsilon^{\prime}\approx\delta_{11}\approx\delta_{00}\approx
0$ and $\delta_{10}\approx\delta_{01}\overset{\text{def}}{=}\delta$, based on
Equation (10) we get:

$\begin{split}\alpha(j)&=\frac{\sum_{k:(k,j)\in\Omega}K(\mathbf{h}_{k},\mathbf{h}_{j})}{\sum_{k:(k,j)\in\Omega}1-K(\mathbf{b}_{k},\mathbf{b}_{j})}=\frac{\sum_{k:(k,j)\in\mathcal{A}_{10}\cup\mathcal{A}_{01}\cup\mathcal{A}_{11}}K(\mathbf{h}_{k},\mathbf{h}_{j})}{\sum_{k:(k,j)\in\mathcal{A}_{10}\cup\mathcal{A}_{00}}1-K(\mathbf{b}_{k},\mathbf{b}_{j})}=\\\
&=\frac{(1-\delta)\cdot\mid\mathcal{A}_{10}^{j}\mid+\delta\cdot\mid\mathcal{A}_{01}^{j}\mid+\mid\mathcal{A}_{11}^{j}\mid}{\mid\mathcal{A}_{10}^{j}\mid+\mid\mathcal{A}_{00}^{j}\mid}=\frac{\mid\mathcal{A}_{10}^{j}\mid+\mid\mathcal{A}_{11}^{j}\mid+\delta\cdot\big{(}\mid\mathcal{A}_{01}^{j}\mid-\mid\mathcal{A}_{10}^{j}\mid\big{)}}{\mid\mathcal{A}_{10}^{j}\mid+\mid\mathcal{A}_{00}^{j}\mid}\end{split}$
(12)

where $\mathcal{A}_{uv}^{j}=\\{k\mid(k,j)\in\mathcal{A}_{uv}\\}$ and
$u,v\in\\{0,1\\}$.

Additionally, based on the Equations (8), (9) and (11), minimizing Equation
(6) results in:

$K(\mathbf{h}_{i},\mathbf{h}_{j})=\left\\{\begin{array}[]{ll}\alpha(j)&\text{,
if }(i,j)\in\mathcal{A}_{10}\\\ 0&\text{, if }(i,j)\in\mathcal{A}_{01}\\\
0&\text{, if }(i,j)\in\mathcal{A}_{11}\\\ \alpha(j)&\text{, if
}(i,j)\in\mathcal{A}_{00}\\\ \end{array}\right.$ (13)

So, if $\Omega=\mathcal{S}$, then
$\mid\mathcal{A}_{11}^{j}\mid=\mid\mathcal{A}_{00}^{j}\mid=0$ implying
$\alpha(j)=1$ (cf. Equation 12)444Note that when bias is uniformly distributed
across classes $\mid\mathcal{A}_{01}^{j}\mid$ =
$\mid\mathcal{A}_{10}^{j}\mid$, which is the case in all experiments of this
work. We consider exploring atypical cases of non-uniform bias distribution
across classes as out of this works scope and leave it for future work., which
results in an ideal form of Equation (13), namely:

$K(\mathbf{h}_{i},\mathbf{h}_{j})=\left\\{\begin{array}[]{ll}1&\text{, if
}(i,j)\in\mathcal{A}_{10}\\\ 0&\text{, if }(i,j)\in\mathcal{A}_{01}\\\
\end{array}\right.$ (14)

accomplishing the very small and independent $\delta_{10},\delta_{01}$
existence. In contrast, if $\Omega=\mathcal{A}$, then:

$\mid\mathcal{A}_{11}^{j}\mid\geq 0,~{}\mid\mathcal{A}_{00}^{j}\mid\geq 0$

so depending on the sizes of the above sets $\alpha(j)$ can greatly vary.
Moreover, not only the 3rd branch of Equation 13 forces pairs
$(i,j)\in\mathcal{A}_{11}$ to have dissimilar embeddings $\mathbf{h}_{i}$ and
$\mathbf{h}_{j}$ -thus lowering the accuracy-, but also the training becomes
highly unstable with each batch imposing different $\alpha(j)$ (and
consequently $K(\mathbf{h}_{i},\mathbf{h}_{j})$) values. Thus, we opt for
considering $\mathcal{L}_{FLAC}$ acting on $\Omega=\mathcal{S}$ to address
bias and achieve higher accuracy levels.

By doing so, we argue that minimizing the total loss (i.e., Equation 7)
minimizes mutual information between the main model’s representations
$\mathbf{h}$ and the corresponding protected attributes $t$. In order to prove
this, we consider the definition of Quadratic Mutual Information (QMI) as
defined in [44], namely Equation (15):

$\mathcal{I}(\mathbf{h},t)=\sum_{t}\int_{\mathbf{h}}p(\mathbf{h},t)^{2}d\mathbf{h}+\sum_{t}\int_{\mathbf{h}}(p(\mathbf{h})P(t))^{2}d\mathbf{h}-2\sum_{t}\int_{\mathbf{h}}p(\mathbf{h},t)p(\mathbf{h})P(t)d\mathbf{h},$
(15)

whose 3 terms are called information potentials and are denoted by $V_{IN}$,
$V_{ALL}$, and $V_{BTW}$, respectively. Then, the information potentials can
be estimated as follows:

$V_{IN}=\frac{1}{N^{2}}\sum_{p=1}^{N_{c}}\sum_{k=1}^{J_{p}}\sum_{l=1}^{J_{p}}K(\mathbf{h}_{pk},\mathbf{h}_{pl}),$
(16)
$V_{ALL}=\frac{1}{N^{2}}\sum_{p=1}^{N_{c}}(\frac{J_{p}}{N})^{2}\sum_{k=1}^{N}\sum_{l=1}^{N}K(\mathbf{h}_{k},\mathbf{h}_{l}),$
(17)
$V_{BTW}=\frac{1}{N^{2}}\sum_{p=1}^{N_{c}}\frac{J_{p}}{N}\sum_{j=1}^{J_{p}}\sum_{k=1}^{N}K(\mathbf{h}_{pj},\mathbf{h}_{k}),$
(18)

where $N$, $N_{c}$, and $J_{p}$ are the number of samples, classes, and
samples belonging to the class $p$, respectively. Note that $J_{p}$ is equal
to $\frac{N}{N_{c}}$ as QMI should be calculated on balanced data. $V_{IN}$
consists of the interactions between pairs within each protected attribute
class, $V_{ALL}$ consists of the interactions between all pairs, and $V_{BTW}$
consists of interactions between samples of each protected attribute class
against all samples. Considering the optimal scenario with
$\mathcal{L}_{task}=0$ and $\mathcal{L}_{FLAC}=0$, the Equation (10) holds for
very small $\delta_{10}$, $\delta_{10}$, $\delta_{11}$, $\delta_{00}$ (for
simplicity we consider
$\delta_{10}\approx\delta_{10}\approx\delta_{11}\approx\delta_{00}\approx\epsilon^{\prime}\approx
0$) and the information potentials, $V_{IN}$, $V_{ALL}$, and $V_{BTW}$, result
in the following:

$V_{IN}=\frac{1}{N^{2}}\mid\mathcal{A}_{11}\mid,$ (19)
$V_{ALL}=\frac{1}{N^{2}}\sum_{p=1}^{N_{c}}\left(\frac{J_{p}}{N}\right)^{2}\left(\mid\mathcal{A}_{11}\mid+\mid\mathcal{A}_{10}\mid\right)\overset{J_{p}=\frac{N}{N_{c}}}{=}\frac{1}{N^{2}}\frac{1}{N_{c}}\left(\mid\mathcal{A}_{11}\mid+\mid\mathcal{A}_{10}\mid\right),$
(20)
$V_{BTW}=\frac{1}{N^{2}}\sum_{p=1}^{N_{c}}\frac{J_{p}}{N}\frac{\mid\mathcal{A}_{11}\mid+\mid\mathcal{A}_{10}\mid}{N_{c}}\overset{J_{p}=\frac{N}{N_{c}}}{=}\frac{1}{N^{2}}\frac{1}{N_{c}}(\mid\mathcal{A}_{11}\mid+\mid\mathcal{A}_{10}\mid).$
(21)

Finally, given that
$\mid\mathcal{A}_{10}\mid=(N_{c}-1)\cdot\mid\mathcal{A}_{11}\mid$, in the
optimal scenario of $\mathcal{L}=0$ the QMI between the representations
$\mathbf{h}$ and the corresponding protected attributes $t$, is eliminated:

$\begin{split}\mathcal{I}(\mathbf{h},t)&=V_{IN}+V_{ALL}-2V_{BTW}=V_{IN}-V_{BTW}=\frac{1}{N^{2}}\left(\mid\
\mathcal{A}_{11}\mid-\frac{\mid\ \mathcal{A}_{11}\mid+\mid\
\mathcal{A}_{10}\mid}{N_{c}}\right)=\\\ &=\frac{1}{N^{2}}\left(\mid\
\mathcal{A}_{11}\mid-\frac{\mid\ \mathcal{A}_{11}\mid+(N_{c}-1)\mid\
\mathcal{A}_{11}\mid}{N_{c}}\right)=\frac{1}{N^{2}}\left(\mid\
\mathcal{A}_{11}\mid-\frac{\mid\
\mathcal{A}_{11}\mid}{N_{c}}-\frac{(N_{c}-1)\mid\
\mathcal{A}_{11}\mid}{N_{c}}\right)=\\\
&=\frac{1}{N^{2}}\left(\left(1-\frac{1}{N_{c}}\right)\mid\
\mathcal{A}_{11}\mid-\frac{(N_{c}-1)}{N_{c}}\mid\
\mathcal{A}_{11}\mid\right)=0\end{split}$ (22)

##  Appendix B Fairness Metrics

In this appendix we report the performance of FLAC in terms of three fairness
metrics, namely p% rule [52], DFPR, and DFNR [53, 54]. p% rule is defined as:

$\min\left\\{\frac{P(\hat{y_{i}}=1\mid t_{i}=0)}{P(\hat{y_{i}}=1\mid
t_{i}=1)},\frac{P(\hat{y_{i}}=1\mid t_{i}=1)}{P(\hat{y_{i}}=1\mid
t_{i}=0)}\right\\}\geq\frac{p}{100},$ (23)

where $p$% should be at least 80% to consider the model’s outcomes fair [57].
DFPR and DFNR are defined as follows:

$D_{FPR}=P\left(\hat{y}_{i}\neq y_{i}\mid
y_{i}=1,t_{i}=1\right)-P\left(\hat{y}_{i}\neq y_{i}\mid
y_{i}=1,t_{i}=0\right),$ (24) $D_{FNR}=P\left(\hat{y}_{i}\neq y_{i}\mid
y_{i}=1,t_{i}=0\right)-P\left(\hat{y}_{i}\neq y_{i}\mid
y_{i}=0,t_{i}=0\right).$ (25)

Typically, these two metrics are combined into the following quantity to
measure the overall disparate mistreatment [54]:

$\mid D_{FPR}\mid+\mid D_{FNR}\mid.$ (26)

In Table 10, we compare the performance of Vanilla and FLAC in terms of these
fairness metrics. As it can be noticed, not only FLAC demonstrates higher p%
values, but it also satisfies the lower bound of 80% in all cases. This
indicates that FLAC results in fair representations for both UTKFace (with
race and gender as protected attributes) and CelebA (with BlondHair and
HeavyMakeup as targets). Furthermore, FLAC demonstrates reduced overall
disparate mistreatment compared to Vanilla in all experiments except for
CelebA (HeavyMakeup). Note that occasionally higher $\mid D_{FPR}\mid$ and
$\mid D_{FNR}\mid$ values can be attributed to the frequently encountered
conflict between disparate mistreatment (as captured by these two metrics) and
disparate impact ( measured by p% rule) mitigation.

Table 10: Performance of FLAC compared to Vanilla in terms of p% rule and $\mid D_{FPR}\mid+\mid D_{FNR}\mid$ on UTKFace and CelabA datasets. dataset | method | $\mathcal{L}_{task}$ | p% rule | $\mid D_{FPR}\mid+\mid D_{FNR}\mid$
---|---|---|---|---
UTKFace (race) | Vanilla | CE | 0.736 | 0.255
FLAC | CE | 0.985 | 0.013
FLAC | Con. | 0.998 | 0.063
UTKFace (age) | Vanilla | CE | 0.450 | 0.447
FLAC | CE | 0.803 | 0.424
FLAC | Con. | 0.759 | 0.419
CelebA (BlondHair) | Vanilla | CE | 0.324 | 0.526
FLAC | CE | 0.897 | 0.078
| FLAC | Con. | 0.890 | 0.093
CelebA (HeavyMakeup) | Vanilla | CE | 0.250 | 0.113
FLAC | CE | 0.800 | 0.225
| FLAC | Con. | 0.889 | 0.344

## References

  * [1] Solon Barocas, Moritz Hardt, and Arvind Narayanan.  Fairness and Machine Learning: Limitations and Opportunities.  fairmlbook.org, 2019.  http://www.fairmlbook.org.
  * [2] Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf.  Deepface: Closing the gap to human-level performance in face verification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701–1708, 2014.
  * [3] Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath.  Generative adversarial networks: An overview.  IEEE signal processing magazine, 35(1):53–65, 2018.
  * [4] Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez.  Recommender systems survey.  Knowledge-based systems, 46:109–132, 2013.
  * [5] Mingxing Tan, Ruoming Pang, and Quoc V Le.  Efficientdet: Scalable and efficient object detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10781–10790, 2020.
  * [6] Simone Fabbrizzi, Symeon Papadopoulos, Eirini Ntoutsi, and Ioannis Kompatsiaris.  A survey on bias in visual datasets.  Computer Vision and Image Understanding, 223:103552, 2022.
  * [7] Megan Frisella, Pooya Khorrami, Jason Matterer, Kendra Kratkiewicz, and Pedro Torres-Carrasquillo.  Quantifying bias in a face verification system.  In Computer Sciences & Mathematics Forum, volume 3, page 6. MDPI, 2022.
  * [8] Stefan Glüge, Mohammadreza Amirian, Dandolo Flumini, and Thilo Stadelmann.  How (not) to measure bias in face recognition networks.  In IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pages 125–137. Springer, 2020.
  * [9] Jacqueline G Cavazos, P Jonathon Phillips, Carlos D Castillo, and Alice J O’Toole.  Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?  IEEE transactions on biometrics, behavior, and identity science, 3(1):101–111, 2020.
  * [10] Tomáš Sixta, Julio Jacques Junior, Pau Buch-Cardona, Eduard Vazquez, and Sergio Escalera.  Fairface challenge at eccv 2020: Analyzing bias in face recognition.  In European conference on computer vision, pages 463–481. Springer, 2020.
  * [11] Tian Xu, Jennifer White, Sinan Kalkan, and Hatice Gunes.  Investigating bias and fairness in facial expression recognition.  In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 506–523, 2020.
  * [12] Xin Zheng, Yanqing Guo, Huaibo Huang, Yi Li, and Ran He.  A survey of deep facial attribute analysis.  International Journal of Computer Vision, 128:2002–2034, 2020.
  * [13] Han Zhao and Geoffrey J Gordon.  Inherent tradeoffs in learning fair representations.  Journal of Machine Learning Research, 23:1–26, 2022.
  * [14] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim.  Learning not to learn: Training deep neural networks with biased data.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9012–9020, 2019.
  * [15] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto.  End: Entangling and disentangling deep representations for bias correction.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13508–13517, 2021.
  * [16] Youngkyu Hong and Eunho Yang.  Unbiased classification through bias-contrastive and bias-balanced learning.  Advances in Neural Information Processing Systems, 34:26449–26461, 2021.
  * [17] Carlo Alberto Barbano, Benoit Dufumier, Enzo Tartaglione, Marco Grangetto, and Pietro Gori.  Unbiased supervised contrastive learning.  arXiv preprint arXiv:2211.05568, 2022.
  * [18] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer.  Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4069–4082, 2019.
  * [19] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al.  Rubi: Reducing unimodal biases for visual question answering.  Advances in neural information processing systems, 32, 2019.
  * [20] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh.  Learning de-biased representations with biased representations.  In International Conference on Machine Learning, pages 528–539. PMLR, 2020.
  * [21] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.  Learning from failure: De-biasing classifier from biased classifier.  Advances in Neural Information Processing Systems, 33:20673–20684, 2020.
  * [22] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [23] Zhang Zhifei, Song Yang, and Qi Hairong.  Age progression/regression by conditional adversarial autoencoder.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.
  * [24] Shen Yan, Di Huang, and Mohammad Soleymani.  Mitigating biases in multimodal personality assessment.  In Proceedings of the 2020 International Conference on Multimodal Interaction, pages 361–369, 2020.
  * [25] Abhinav Gupta, Adithyavairavan Murali, Dhiraj Prakashchand Gandhi, and Lerrel Pinto.  Robot learning in homes: Improving generalization and reducing dataset bias.  Advances in neural information processing systems, 31, 2018.
  * [26] Daizong Liu, Xiaoye Qu, and Wei Hu.  Reducing the vision and language bias for temporal sentence grounding.  In Proceedings of the 30th ACM International Conference on Multimedia, pages 4092–4101, 2022.
  * [27] Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshan, and Kush R Varshney.  Fairness gan.  stat, 1050:24, 2018.
  * [28] Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu.  Fairgan: Fairness-aware generative adversarial networks.  In 2018 IEEE International Conference on Big Data (Big Data), pages 570–575. IEEE, 2018.
  * [29] Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, et al.  Bias in data-driven artificial intelligence systems—an introductory survey.  Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3):e1356, 2020.
  * [30] Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky.  Towards fairness in visual recognition: Effective strategies for bias mitigation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8919–8928, 2020.
  * [31] Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker.  Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings.  In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0–0, 2018.
  * [32] Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig.  Controllable invariance through adversarial feature learning.  Advances in neural information processing systems, 30, 2017.
  * [33] Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez.  Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5310–5319, 2019.
  * [34] Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon.  Learning controllable fair representations.  In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2164–2173. PMLR, 2019.
  * [35] Tameem Adel, Isabel Valera, Zoubin Ghahramani, and Adrian Weller.  One-network adversarial fairness.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2412–2420, 2019.
  * [36] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf.  Measuring statistical dependence with hilbert-schmidt norms.  In Algorithmic Learning Theory: 16th International Conference, ALT 2005, Singapore, October 8-11, 2005. Proceedings 16, pages 63–77. Springer, 2005.
  * [37] Nikolaos Passalis, Maria Tzelepi, and Anastasios Tefas.  Heterogeneous knowledge distillation using information flow modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2339–2348, 2020.
  * [38] Nikolaos Passalis and Anastasios Tefas.  Learning deep representations with probabilistic knowledge transfer.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 268–284, 2018.
  * [39] Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Ioannis Kompatsiaris.  Indistill: Transferring knowledge from pruned intermediate layers.  arXiv preprint arXiv:2205.10003, 2022.
  * [40] Jorge R Vergara and Pablo A Estévez.  A review of feature selection methods based on mutual information.  Neural computing and applications, 24:175–186, 2014.
  * [41] Alexander Kraskov, Harald Stögbauer, Ralph G Andrzejak, and Peter Grassberger.  Hierarchical clustering using mutual information.  Europhysics Letters, 70(2):278, 2005.
  * [42] Wei Li, Zhixuan Liang, Julian Neuman, Jinlin Chen, and Xiaohui Cui.  Multi-generator gan learning disconnected manifolds with mutual information.  Knowledge-Based Systems, 212:106513, 2021.
  * [43] Kristian Lum and James Johndrow.  A statistical framework for fair predictive algorithms.  arXiv preprint arXiv:1610.08077, 2016.
  * [44] Kari Torkkola.  Feature extraction by non-parametric mutual information maximization.  Journal of machine learning research, 3(Mar):1415–1438, 2003.
  * [45] Syed Mumtaz Ali and Samuel D Silvey.  A general class of coefficients of divergence of one distribution from another.  Journal of the Royal Statistical Society: Series B (Methodological), 28(1):131–142, 1966.
  * [46] Yann LeCun.  The mnist database of handwritten digits.  http://yann. lecun. com/exdb/mnist/, 1998.
  * [47] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009.
  * [48] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song.  Natural adversarial examples.  CVPR, 2021.
  * [49] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [50] Wieland Brendel and Matthias Bethge.  Approximating cnns with bag-of-local-features models works surprisingly well on imagenet.  arXiv preprint arXiv:1904.00760, 2019.
  * [51] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan.  Supervised contrastive learning.  Advances in neural information processing systems, 33:18661–18673, 2020.
  * [52] Paul Meier, Jerome Sacks, and Sandy L Zabell.  What happened in hazelwood: Statistics, employment discrimination, and the 80% rule.  American Bar Foundation Research Journal, 9(1):139–186, 1984.
  * [53] Patrick Janssen and Bert M Sadowski.  Bias in algorithms: On the trade-off between accuracy and fairness.  2021\.
  * [54] Emmanouil Krasanakis, Eleftherios Spyromitros-Xioufis, Symeon Papadopoulos, and Yiannis Kompatsiaris.  Adaptive sensitive reweighting to mitigate bias in fairness-aware classification.  In Proceedings of the 2018 world wide web conference, pages 853–862, 2018.
  * [55] Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed.  A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI, pages 548–564. Springer, 2020.
  * [56] Rudrajit Das and Subhasis Chaudhuri.  On the separability of classes with the cross-entropy loss function.  arXiv preprint arXiv:1909.06930, 2019.
  * [57] Dan Biddle.  Adverse impact and test validation: A practitioner’s guide to valid and defensible employment testing.  Gower Publishing, Ltd., 2006.