# On Disentangled and Locally Fair Representations

Yaron Gurovich, Sagie Benaim, Lior Wolf

###### Abstract

We study the problem of performing classification in a manner that is fair for
sensitive groups, such as race and gender. This problem is tackled through the
lens of disentangled and locally fair representations. We learn a locally fair
representation, such that, under the learned representation, the neighborhood
of each sample is balanced in terms of the sensitive attribute. For instance,
when a decision is made to hire an individual, we ensure that the $K$ most
similar hired individuals are racially balanced. Crucially, we ensure that
similar individuals are found based on attributes not correlated to their
race. To this end, we disentangle the embedding space into two
representations. The first of which is correlated with the sensitive attribute
while the second is not. We apply our local fairness objective only to the
second, uncorrelated, representation. Through a set of experiments, we
demonstrate the necessity of both disentangled and local fairness for
obtaining fair and accurate representations. We evaluate our method on real-
world settings such as predicting income and re-incarceration rate and
demonstrate the advantage of our method.

## Introduction

Machine learning technology in domains with high social impact such as health,
law, and finance, should be handled with particular care. Beyond test
accuracy, one must make sure that additional social constraints are met. For
instance, when deciding if to hire an individual, it may be the case that, for
the collected data, a particular racial group exhibits higher positive
outcomes resulting in an unfair model (Barocas and Selbst 2016; Feldman et al.
2015; Primus 2010). One must also ensure that the false positive and false
negative rates are roughly similar across ethnic groups (Zafar et al. 2017a).

In this paper, we tackle this problem through the lens of locally fair
representations. Consider, for example, the task of learning a classifier for
hiring an individual for a job. Our key idea is based on a simple principle:
for every individual that is hired (resp. not hired), on average, the set of
most similar individuals that are also hired (resp. not hired) must be
racially unbiased, proportionally to their total population ratio. This gives
rise to our local fairness loss. For either the hired or non-hired
individuals, we consider the neighborhood of the $K$ nearest-neighbors in the
feature space of our classifier. We encourage our model to make predictions
such that, for example, the probability of African-American or Caucasian in
this neighborhood, matches this probability in the entire population.

One must take particular care when measuring the similarity of individuals. To
give a concrete example, we want to ensure that individuals are not compared
to each other based on attributes related to race, but rather on attributes
such as job experience or education, which may be predictive of employability.
To this end, we consider the setting of class-based disentanglement.

Class-based disentanglement methods (Lample et al. 2017; Hadad, Wolf, and
Shahar 2018; Benaim et al. 2019), focus on producing two representations of
the data $x$. The first contains the information relevant to the class, and
the second contains the rest of the information. This second part is a
representation of the data that contains all the information that is not
correlated with the sensitive attribute, such as race or gender, and it is
produced without the knowledge of the downstream task and labels. In our
setting, we can consider the similarity of individuals only in the
representation which is not correlated to the sensitive attribute.

Since real-world scenarios often involve training labels that are biased with
an unknown rate, learning a representation that is mostly independent on the
downstream task is essential to achieve high fairness rates. Therefore,
unsupervised disentanglement has a major impact on the obtained fairness. This
is evident from our experiments, which repeatedly show that the tradeoff
between accuracy and fairness can be directly controlled through balancing the
label-independent disentanglement loss-term and the classification loss.

Overall, our method provides the following contributions: (1) We introduce a
novel local fairness loss, which ensures that, under the learned
representation, the neighborhood of each sample is balanced in terms of the
sensitive attribute. (2) We show how the framework of disentangled
representation can be used in the context of fairness. In this framework, two
representations are learned: one that is correlated with the sensitive
attribute and one that is uncorrelated. (3) We demonstrate that the
disentangled representation is necessary for benefiting from learning locally
fair representations. (4) We demonstrate the ability to control the tradeoff
between accuracy and fairness on various real-world datasets. (5) Our method
improves fairness, both in terms of disparate impact and equality of
opportunity, while maintaining high accuracy. It excels especially in the
regime of high accuracy levels.

## Related Work

Previous work on fairness can be divided into three categories: pre-
processing, in-processing and post-processing techniques. Pre-processing
techniques, such as (Calders, Kamiran, and Pechenizkiy 2009; Edwards and
Storkey 2015; Feldman et al. 2015), manipulate the training data, without
modifying the training procedure. The advantage of the pre-processing
techniques is that they require no knowledge of the downstream task. A basic
version of our method, which uses disentanglement, but not local fairness,
falls into this category. However, we further improve it to make use of
labels. Post-processing techniques intervene after a model is trained.
Previous work either modify the input before it enters the model (Adler et al.
2018; Hardt, Price, and Srebro 2016; Pleiss et al. 2017) or modify the model’s
prediction (Kamiran, Calders, and Pechenizkiy 2010; Menon and Williamson
2018).

In-processing techniques modify the learning algorithm, for example, by
modifying the loss function (Bechavod and Ligett 2017; Kamishima et al. 2012).
Related to the disentanglement part of our algorithm are adversarial
approaches (Adel et al. 2019; Beutel et al. 2017; Wadsworth, Vera, and Piech
2018). (Adel et al. 2019), for instance, generate an embedding by applying
both an adversarial objective and a classification loss. Since the labels
might be correlated with the sensitive attribute, such classification loss
could reduce fairness to increase accuracy. Our disentanglement approach
applies a similar adversarial objective, where the classification loss is only
optionally applied to adjust the tradeoff between accuracy and fairness.

Our work crucially relies on a disentangled representation to induce a space
where the similarity of individuals is based on attributes that are not
correlated to the sensitive attribute. In this space, (Locatello et al. 2019)
investigate the usefulness of unsupervised disentanglement approaches for
making fair predictions. They show that both the optimal and empirical
predictions may be unfair. Since the sensitive attribute is not known at
training, an assumption is made that the sensitive attribute and the label are
independent, which is not true in many real-world situations. Our approach
does not make such an assumption and instead builds on the disentanglement
approach of (Hadad, Wolf, and Shahar 2018), which uses the certain attribute
$a$ to create a representation that is independent of $a$. The authors of
(Creager et al. 2019) propose a disentanglement approach which does not make
use of labels $y$ at training. Similarly to our method, two representations
are learned from $x$, one relevant to $a$, and one irrelevant to it. Compared
to (Creager et al. 2019), our method directly optimizes the fair
representation with respect to the sensitive attribute and target
classification task and thus allows the control of the tradeoff between
fairness and accuracy.

Other approaches also investigate this setting proposing an additional
correlation metric (Tan et al. 2020). Unlike these works, in our setting, the
labels $y$ can be incorporated at training to adjust the accuracy-fairness
tradeoff, and this can be done to a varying degree by adjusting the weight of
corresponding classification loss. More crucially, we introduce a novel local
fairness loss which improves the fairness of produced representation.

Individual fairness is a fairness criterion in which two similar individuals
should have the same classification (Dwork et al. 2012). It was presented
without an algorithmic way to compute similarity and only as means to evaluate
a set of given labels. A follow-up work assumes access to an oracle which can
evaluate the distance between two individuals according to an unknown but
fixed fair metric and show how to learn a fair classifier with a limited
number of oracle queries (Kim, Reingold, and Rothblum 2018). In our work we
consider $K$ neighbors and not the nearest one, and learn the appropriate
metric by employing disentanglement. Equally important, our notion of
locality, which considers the nearest neighbors out of the closest samples
with a the same label is different, and so is our loss, which is based on the
distribution of the protected attribute in the obtained set of neighbors.

## Method

We begin by describing the problem formulation. We then define our
disentangled model that does not use labels. We then introduce our novel local
fairness loss which operates on the non-sensitive representation learned
through this disentangled model. We also describe a classification loss, which
can be used, when labels $y$ are available.

### Problem Formulation

We consider a set of data points $x\in\mathbb{R}^{n}$, each having an
associated binary label $y\in\\{-1,1\\}$ and a sensitive attribute
$a\in\\{0,1\\}$, where $0$ is the disadvantaged group and $1$ is the
advantaged group. We are interested in training a classifier $C$ such that
$\bar{y}=C(x)$, $\bar{y}$ is accurate with respect to $y$ and also fair with
respect to $a$. To this end, beyond classification accuracy, we consider
multiple fairness measures. Demographic parity ensures that positive and
negative outcomes of $C$ occur at the same rate for both sensitive attributes:

$\displaystyle P(\bar{y}=c|a=0)=P(\bar{y}=c|a=1)~{}~{}~{}~{}~{}\forall
c\in\\{-1,1\\}\ $ (1)

where $\bar{y}$ is a the prediction made by $C$ for a random data point $x$,
and $a$ is its sensitive attribute.

We measure demographic parity for $c=1$, by the Disparate Impact (DI) measure
(Barocas and Selbst 2016; Feldman et al. 2015; Primus 2010):

$\displaystyle|P(\bar{y}=1|a=0)-P(\bar{y}=1|a=1)|$ (2)

Demographic parity alone, however, is limited, in the case where the ground
truth rates differ. That is, when $P(y=c|a=0)\neq P(y=c|a=1)$, where $y$ is
the ground truth label of a random data point $x$. Equality of Odds addresses
disparities in the ground truth rates, and is defined as:

$\displaystyle P(\bar{y}=1|a=0,y=y_{gt})=P(\bar{y}=1|a=1,y=y_{gt})$ (3)

where $y_{gt}\in\\{-1,1\\}$. Equality of Odds requires both equal false
positive and true positive rates between these groups. The Equality of
Opportunity (EO)  measure is obtained by considering the equality of odds
focuses on the true positive rates only :

$\displaystyle|P(\bar{y}=1|a=0,y=1)-P(\bar{y}=1|a=1,y=1)|$ (4)

It ensures that the classification rates are equal across groups (having
different sensitive attributes). We note that there is often a tradeoff
between accuracy and fairness (Zafar et al. 2017b), since labels $y$ and
sensitive attribute $a$ may be correlated. Our framework enables the control
of this tradeoff.

---
Figure 1: An illustration of our method. In step I, $F_{a}$ is trained jointly
with $M_{a}$ using loss $\mathcal{L}_{a}$, to produce representation $z_{a}$,
which is correlated to $a$. In step II, $F_{a}$ and $M_{a}$ are frozen. We
then train $F_{z}$ to produce a representation $z_{x/a}$ which contains the
information in $x$ not correlated with $a$. To this end, $F_{z}$ trains, in an
adversarial manner, to fool a discriminator $d$ using $\mathcal{L}_{adv}$
loss. We also apply a reconstruction loss ($\mathcal{L}_{recon}$) using a
generator $G$. When labels $y$ are available, we apply the ‘local fairness’
loss ($\mathcal{L}_{local}$) on $F_{z}$’s embeddings. We also train $F_{z}$
jointly with $M_{y}$ to classify label $y$. Before evaluation, we carry a
final step, where $F_{z}$ is fixed, and $M_{y}$ is finetuned to minimize
$L_{y}$. $M_{y}$ is then used to evaluate test samples encoded using $F_{z}$.

### Disentangled Model

Our method is illustrated in Fig. 1. To ensure demographic parity, it extracts
a representation of $x$ that is independent of $a$ but captures all other
information about $x$. To do so, we adopt the class-based disentanglement
framework of (Hadad, Wolf, and Shahar 2018). Specifically, we consider two
feature extractors, $F_{a}$ and $F_{z}$, where $F_{a}$ encodes only the
information relevant for the classification of $a$, and $F_{z}$ contains all
other information in $x$. To this end, we train $F_{a}$ and $F_{z}$ in two
steps.

##### Step I.

We first train an MLP $M_{a}$ jointly with $F_{a}$ for classifying the correct
attribute of input $x$. The following loss is optimized over $M_{a}$ and
$F_{a}$:

$\displaystyle\mathcal{L}_{a}=\frac{1}{m}\sum_{x\in
X}\ell_{BCE}(M_{a}(F_{a}(x)),a)\,,$ (5)

where $X$ is the set of training points $x\in\mathbb{R}^{n}$ and $a$ is the
associated sensitive attribute of $x$. $m=|X|$ and $\ell_{BCE}$ stands for the
binary cross-entropy loss. We choose the network architectures, such that
$F_{a}$ maps $x$ to a low dimensional vector and $M_{a}$ has a low capacity,
thus creating a bottleneck (Tishby and Zaslavsky 2015) that ensures that
$F_{a}$ encodes mostly the information that is relevant to the classification
of $a$.

##### Step II.

In the second step, we train $F_{z}$ to capture all the information in $x$
which is independent of $a$. First, we enforce that $F_{z}$ does not capture
information about $a$. To do so, we use a domain confusion loss (Tzeng et al.
2014) that ensures that the distribution of encoding $F_{z}(X_{a=0})$ equals
$F_{z}(X_{a=1})$, where $X_{a=0}$ (resp. $X_{a=1}$) denotes the data points
whose sensitive value is 0 (resp. 1). The loss used is:

$\displaystyle\mathcal{L}_{adv}=$ $\displaystyle\frac{1}{m_{0}}\sum_{x_{0}\in
X_{a=0}}\ell_{BCE}(d(F_{z}(x_{0})),1)$ $\displaystyle+$
$\displaystyle\frac{1}{m_{1}}\sum_{x_{1}\in
X_{a=1}}\ell_{BCE}(d(F_{z}(x_{1})),1)$ (6)

where $m_{0}=|X_{a=0}|$, $m_{1}=|X_{a=1}|$, and $d$ is a discriminator that
minimizes:

$\displaystyle\mathcal{L}_{d}=$ $\displaystyle\frac{1}{m_{0}}\sum_{x_{0}\in
X_{a=0}}\ell_{BCE}(d(F_{z}(x_{0})),0)$ $\displaystyle+$
$\displaystyle\frac{1}{m_{1}}\sum_{x_{1}\in
X_{a=1}}\ell_{BCE}(d(F_{z}(x_{1})),1)$ (7)

The discriminator $d$ attempts to separate between the distributions
$F_{z}(X_{a=0})$ and $F_{z}(X_{a=1})$, by classifying samples of the former as
$0$ and the samples of the latter as $1$, whereas $F_{z}$ tries to fool the
discriminator, by trying to force the discriminator to label the embeddings
$F_{z}(X_{a=0})$ and $F_{z}(X_{a=1})$ as $1$. Second, we use a loss term to
ensure that $F_{z}$ captures the information in $x$ that is not captured by
$F_{a}$. To do so, we freeze $F_{a}$ and require that $F_{a}$’s encoding of
$x$ (the factors dependent on $a$) and $F_{z}$’s encoding of $x$ (the factors
independent of $a$) together would suffice to reconstruct $x$. That is:

$\displaystyle\mathcal{L}_{rec}=\frac{1}{m}\sum_{i=1}^{m}\|G(F_{a}(x_{i}),F_{z}(x_{i}))-x_{i}\|_{1}\,,$
(8)

where $G$ is a decoder that accepts a concatenation of $F_{a}$’s and $F_{z}$’s
embeddings.

The disentanglement model alone does not make use of the labels $y$. This is
useful in the case where the labels are unknown at first, and as shown in our
experiments, fairness is improved through the use of these label-independent
embeddings. This is a direct result of the labels being biased with respect to
the sensitive group.

### Local Fairness

Given our disentangled representation, we can now consider our notion of
‘local fairness’. As a motivating example, consider the following setting: $x$
denotes a person’s employment profile (such as education and experience). $y$
denotes a recommendation to either hire ($y=1$) or not hire ($y=-1$) this
person for a given job and $a$ denotes the person’s race – African-American
for $a=0$ and Caucasian for $a=1$. Assume that a person $x$ is not hired
($y=-1$). We would then like that the group of most similar persons to $x$ (in
the embedding space), who are also not hired, to be attribute-balanced. That
is, the proportion of Caucasian to African-American persons in this group
should be close to their true proportion in the population. If this is not the
case, then locally in the data, there is a region in which the advantageous
label indicates the sensitive attribute. A similar reasoning also holds for
the case of $y=1$. To obtain local neighborhoods that are apriori (before
considering $y$) balanced with respect to the sensitive information, we want
to ensure that similarity to other persons is done only based on information
that is independent of the sensitive attribute.

To make the embedding $F_{z}$’s locally fair, we consider the embeddings
$F_{z}(X_{y=-1})$, where $X_{y=-1}$ is the set of training points for which
$y=-1$ (we denote by an application of a function to the set obtained by
applying this function to all set elements). Using the framework introduced
earlier, $F_{z}(X_{y=-1})$ are learned to be non-correlated with the sensitive
information and to capture the rest of the information. For each embedding
point $z_{i}\in F_{z}(X_{y=-1})$, we take the $K$ nearest neighbors in this
embedding space, denoted by $z^{1}_{i},\dots,z^{K}_{i}$.

Assume the ratio of the protected group ($a=1$) to the non-protected group
($a=0$) in the population is $1\mathrel{\mathop{\mathchar 58\relax}}r$. We
want that, on average, $1/(r+1)$ of the $z^{1}_{i},\dots,z^{K}_{i}$ to have
$a=0$ and $a=1$ otherwise. Let $a^{\prime j}_{i}$ be $-1$ if its associated
sensitive attribute ($a$) is $0$, and otherwise set $a^{\prime j}_{i}$ to be
$r$. We enforce this requirement by setting that $\|a^{\prime 1}_{i}\cdot
z^{1}_{i}+\dots+a^{\prime K}_{i}\cdot z^{K}_{i}\|_{2}=0$.

The same loss is applied also in the case where $y=1$, in this case
considering nearest neighbors from the group $F_{z}(X_{y=1})$. Let
$m^{\prime}=|X_{y=y_{gt}}|$, the local loss is defined as:

$\displaystyle\mathcal{L}^{y_{gt}}_{local}$
$\displaystyle=\frac{1}{m^{\prime}}\sum_{z_{i}\in
F_{z}(X_{y=y_{gt}})}\|a^{\prime 1}_{i}\cdot z^{1}_{i}+\dots+a^{\prime
K}_{i}\cdot z^{K}_{i}\|_{2}$ (9)

We then have
$\mathcal{L}_{local}=\mathcal{L}^{0}_{local}+\mathcal{L}^{1}_{local}$.

As an intuition, consider the case where the dimensionality of the $z_{i}$’s
is $1$ and $r=1$, that is, the ratio of the sizes of the protected to the non-
protected groups in the population is $1$. Here, the constraint is satisfied
($\mathcal{L}_{local}=0$) if the group of $z_{i}$’s with $a=0$ have the same
average value as the group of $z_{i}$’s with $a=1$. If the number of
individuals in both groups is the same, then the individuals should be, on
average, similar.

On the other hand, if the non-protected group consists of many individuals,
i.e., there are $M>>1$ individuals $z_{i}^{\prime}$ for which $a=0$, and there
is a single individual $z_{i}^{\prime\prime}$ for which $a=1$, then in order
for $\mathcal{L}_{local}$ to become zero, the embedding value
$z_{i}^{\prime\prime}$ has to be very far from the average $z_{i}^{\prime}$
(for instance $z_{i}^{\prime\prime}=M$ and $z_{i}^{\prime}=1$).

In our method, the local fairness is added to the loss terms that are
minimized as part of step II training the disentangled model. This makes the
disentanglement supervised since label information is used. Furthermore, we
add to this step the training of an MLP classifier $M_{y}$, which is trained
in conjugation with $F_{z},G$ (and $d$). The classification loss used to train
$M_{y}$ is given by

$\displaystyle\mathcal{L}_{y}=\frac{1}{m}\sum_{x\in
X}\ell_{BCE}(M_{y}(F_{z}(x)),y)\,,$ (10)

and the full objective of step II that is minimized over $M_{y},F_{z},G$ is
given by:

$\displaystyle\mathcal{L}_{full}=\mathcal{L}_{rec}+\lambda_{1}\cdot\mathcal{L}_{adv}+\lambda_{2}\cdot\mathcal{L}_{local}+\lambda_{3}\cdot\mathcal{L}_{y}$
(11)

Where $\lambda_{1},\lambda_{2},\lambda_{3}$ are hyperparameters. $d$ is
trained simultaneously to minimize $\mathcal{L}_{d}$. As shown in the
experiments, adjusting $\lambda_{3}$ results in controlling the tradeoff
between accuracy and fairness. Using $\lambda_{3}=0$ leads to the maximal
fairness, but the classifier, in this case, has the lowest accuracy since the
representation resulting from $F_{z}$ is not optimized for classification
accuracy.

##### The final classifier

To allow the same framework for different values of $\lambda_{3}$, and to
allow the classifier to converge after the adversarial training has been
stopped, the final classifier is trained after $F_{z}$ has been learned.
$M_{y}$ is finetuned to minimize $\mathcal{L}_{y}$ of Eq. 10 where $F_{z}$ is
fixed and is no longer trained. To evaluate our method, test samples are first
encoded using $F_{z}$, and then classified using $M_{y}$.

### Training and Implementation Details

$F_{a}$ and $F_{z}$ are MLPs with 2 hidden layers with $(10,20)$ hidden units,
where each layer is followed by a Leaky-Relu (slope $0.2$) and Dropout
(probability $0.5$). The input dimension is data-dependent and the output
embedding dimension is $20$. Decoder $G$ has the reverse architecture. MLP’s
$M_{a}$, $M_{y}$ and $d$ have similar architecture with an output to which a
sigmoid activation is applied. Input $x$ is pre-processed and normalized as in
CTGAN (Xu et al. 2019), where discrete values are represented as one-hot
vectors and continuous columns are normalized by a distribution representation
and scale to $[0,1]$. We use Adam optimizer with parameters $\beta_{1}=0.5$
and $\beta_{2}=0.999$. We start with a learning rate of $0.001$ and multiply
it by 0.1 every 30 epochs, with a batch size of $64$ and run for $100$ epochs.
For every iteration of $F_{z},M_{y}$ and $G$, we train $d$ for $20$
iterations. Data splits were made as in the dataset description (if test and
train were available), or by a random split into train/test sets. All random
sampling experiments run in 5 folds.

The Local Fairness hyperparameters are set to $K=4$. Hyperparameters for the
loss aggregation are set to $\lambda_{1}=1$, $\lambda_{2}=1$ and
$\lambda_{3}=0.1$. These parameters are set for all datasets and were found
based on the validation set of the synthetic dataset as described in the
experiments section. The various experiments were either run on the CPU (small
datasets) or a single NVIDIA GTX1080 GPU (bigger datasets).

## Experiments

We begin by evaluating our method on the real world datasets of Adult (Dua and
Graff 2017b) and COMPAS (Larson et al. 2016) that have been previously used
for studying fairness. We then demonstrate the tradeoff between accuracy and
fairness for two additional datasets of Bank (Dua and Graff 2017c) and
Communities and Crime (Dua and Graff 2017a). We then study the necessity of
both the disentangled representation and local fairness on a synthetic
dataset. We show that the maximal advantage occurs when learning the
representation using both disentanglement and local fairness, as well as the
standard classification loss. Lastly we evaluate the effect of our losses on
accuracy and fairness when varying $K$, the local neighborhood size, as well
as the bias level of the training set.

### Real World Datasets

|
---|---
(a) | (b)
|
(c) | (d)
Figure 2:  Accuracy vs. fairness tradeoff, obtained by varying $\lambda_{3}$.
The blue points correspond to different values of $\lambda_{3}$ of our method:
$0,0.1,0.2,0.5,0.75,1$, and appear from left to right in increasing accuracy
(error bars indicate standard error). Other colors correspond to baselines
with their corresponding tradeoffs. For accuracy, higher is better and for
fairness lower is better. (a) Adult Disparate Impact, (b) Adult Euqality of
Oppotunity. (c,d) same for COMPAS.
---
Figure 3:  Accuracy vs fairness (Disparate Impact) tradeoff for the datasets
of Bank (Dua and Graff 2017c) (blue) and Communities and Crime (Dua and Graff
2017a) dataset (green+red). In the green plot, race is used as the sensitive
attribute and in the red plot, the capital attribute is used.
$\lambda_{3}=0,0.1,0.2,0.5,0.75,1$, and appear from left to right in
increasing accuracy in each plot. A triangle indicates a vanilla classifier
trained with $\mathcal{L}_{y}$ only. Error bars indicate the standard error.

The Adult dataset (Asuncion and Newman 2007) contains $32,561$ individuals
with $14$ attributes, and we are asked to classify whether a given person
makes more or less than $50,000 a year; gender is the sensitive attribute. The
COMPAS dataset (Larson et al. 2016) contains 5,278 instances with 12 features,
and the associated task is to predict if a criminal defendant will be re-
incarcerated within two years. Here, race is the sensitive attribute. We
compared against a set of recent baseline methods: 1. (Agarwal et al. 2018),
2. (Zafar et al. 2017b), 3. (Zhang, Lemoine, and Mitchell 2018), 4. (Cho,
Hwang, and Suh 2020), 5. (Kamishima et al. 2012), 6. (Perrone et al. 2021). We
consider the results reported in these papers (consistent with our definition
of Disparate Impact and Equal Opportunity). We note that some baselines only
report values on the Adult dataset for Disparate Impact.

Fig. 2 reports accuracy and fairness values for different weights of
$\mathcal{L}_{y}$ in comparison to baselines for Adult and COMPAS. We consider
both Disparate Impact and Equal Opportunity with respect to accuracy. For the
Adult dataset, our method achieves the best tradeoff between accuracy and
Disparate Impact. Further, for maximal accuracy, it achieves the best fairness
measures, both in terms of Disparate Impact and Equality of Opportunity. Note
how in this dataset, accuracy benefits significantly from the addition of the
target labels to the training and most of the gain achieved with
$\lambda_{3}=0.1$.

For the COMPAS dataset our method also achieves the best tradeoff between
accuracy and fairness. For the highest accuracy value, the best Disparate
Impact is achieved. With regards to Equal Opportunity, while our method does
not achieve the overall best accuracy among baselines, it provides better
fairness results for a range of accuracy values.

### Accuracy vs Fairness Tradeoff

We now demonstrate that the tradeoff between accuracy and fairness can be
controlled using our method. To this end we consider two additional datasets.
The Bank dataset (Dua and Graff 2017c) contains a total of 41,188 subjects,
each with 20 attributes (such as marital status) and a binary label, which
indicates whether the client has subscribed (positive class) or not (negative
class) to a term deposit. We consider age as a sensitive attribute, and
binarize the age attribute by assigning it to 1 if a client’s age is between
25 and 60, and 0 otherwise. The Communities and Crime (Dua and Graff 2017a)
dataset contains 1,994 instances and 128 attributes gathered from different
communities in the United States. The attributes are related to several
factors that can highly influence some common crimes such as robberies,
murders, or rapes. The classification task is to predict violent communities.
We take the $70th$ percentile of the total number of violent crimes per 100K
population communities, as a threshold for the violence label. We consider two
sensitive attributes: ’racePctBlack’ (% neighborhood population which is
Black) and ’blackPerCap’ (avg per capita income of Black residents).

Fig. 3 reports the tradeoff between accuracy and Disparate Impact on these
datasets. This tradeoff is controlled through varying $\lambda_{3}$. As can be
seen, increasing the value $\lambda_{3}$ results in a better accuracy and
worse fairness (Disparate Impact). For the Communities and Crime dataset (Fig.
3 (green+red)), we see a large variation in Disparate Impact and accuracy,
while for the Bank dataset (Fig.3 (blue)), we see a small accuracy variation,
with significant variation in Disparate Impact. Interestingly, on the
Communities and Crime dataset, high values of $\lambda_{3}$ provide accuracy
that can reach, or even surpass, the accuracy of the vanilla classifier that
employs only the $\mathcal{L}_{y}$ loss, while still having somewhat lower DI.

### Disentangled and Locally Fair Representations

To demonstrate the impact of our disentangled framework and local fairness we
use a synthetic dataset, which follows (Zhang, Lemoine, and Mitchell 2018).
Samples $(x_{i},y_{i},z_{i})^{n}_{i=1}$ are generated as follows: For each
sample $i$, we first generated two random seeds, $r_{i},yo_{i}\in\\{0,1\\}$.
We then sample $v_{i}\sim\mathcal{N}(r_{i},1)$ and
$u_{r_{1}},\dots,u_{r_{d}}\sim\mathcal{N}(v_{i},1)$ (dependently on $v_{i}$).
We also sample $w_{i}\sim\mathcal{N}(v_{i},1)$ dependently on $v_{i}$. Next we
sample $vo_{i}\sim\mathcal{N}(yo_{i},1)$ and
$u_{yo_{1}},\dots,u_{yo_{d}}\sim\mathcal{N}(vo_{i},1)$ dependently on
$vo_{i}$. Each sample $i$ is then defined as follows:
$x^{(i)}=(r_{i},u_{r_{1}},\dots,u_{r_{d}},u_{yo_{1}},\dots,u_{yo_{d}})$,
$y_{r}^{(i)}=\mathbbm{1}[w_{i}>0]$, $y_{o}^{(i)}=yo_{i}$ and $z^{(i)}=r$
(where $\mathbbm{1}$ denotes an indicator function). To construct the final
dataset, we define $p_{z}$ to be the portion of biased samples in the dataset
and choose the final labels $Y$ such that $p_{y}$ portion of the $n$ samples
comes from the biased distribution $y_{r}^{(i)}$ and the rest from
$y_{o}^{(i)}$. For this experiment, we set $p_{z}=0.5$ for train data and
$p_{z}=0$ for test data. This is done to demonstrate a case where the training
phase is exposed to biased data ($50\%$ of samples) and our goal is to learn a
fair predictor. In our setting, $d=25$. We use $1000$ (resp. 500) train (resp.
test) samples.

#### Variants of Our Method

In Tab. 1, we consider different variants of our method as well as of a
baseline method with no disentanglement. We evaluate the ability of the
resulting representation $F_{z}$ to predict the unbiased labels $y$ and the
sensitive attribute $a$. We would like to get a high accuracy for the labels
prediction and low accuracy (close to the random $0.5$) for the sensitive
attribute.

Settings (1-4) do not contain the two-step disentanglement architecture and
are similar to our 2nd step of training but without the reconstruction loss.
Setting (1) uses a standard baseline classifier with only $\mathcal{L}_{y}$ as
a loss. Setting (2) is a classifier trained with $\mathcal{L}_{y}$ and an
adversarial loss $\mathcal{L}_{adv}$. This is similar to (Adel et al. 2019).
In (3) and (4) we consider these variants with our local fairness loss
($\mathcal{L}_{local}$). As can be seen, classifier (1) is very biased,
archiving $67\%$ accuracy on the sensitive variable, and only $57\%$ accuracy
on the true labels $y$ of the test set (recall that the train and test
distributions differ). Adding an adversarial loss ($\mathcal{L}_{adv}$) in (2)
improves the bias of the representation by reducing the accuracy to sensitive
variable to $55\%$, while also improving the prediction accuracy to the true
test labels $y$. Adding local fairness ($\mathcal{L}_{local}$) in (3) and (4)
reduces both results towards random chance. This indicates that the classifier
is unable to learn and gives a random output, showing that in the standard
setting, without disentanglement, local fairness does not improve fairness.

Settings (5-8) consider the cases where the reconstruction loss,
$\mathcal{L}_{recon}$, is used, but where $\mathcal{L}_{adv}$ is not used. In
these settings, the accuracy of the sensitive attribute is very high
($67-70\%$). Accuracy of the true test labels $y$ set are still not optimal
($56-59\%$).

Settings (9-16) are based on the disentanglement framework we present. In
these settings ,$\mathcal{L}_{recon}$ and $\mathcal{L}_{adv}$ are used
throughout. Using only the reconstruction ($\mathcal{L}_{recon}$) loss and
adversarial loss ($\mathcal{L}_{adv}$) in (9), which is the baseline method of
(Hadad, Wolf, and Shahar 2018), the sensitive accuracy is $58\%$ and the label
prediction is $60\%$, which shows that disentanglement improves upon the
baseline classifier. Compared to the adversarial setting of (2), it classifies
the sensitive attributes more accurately (worse) but also classifies the true
test labels $y$ more accurately (better). Adding a small portion of
$\mathcal{L}_{y}$ in (10), by using $\lambda_{3}=0.1$, leaves the results the
same. However, the significant gain is achieved by adding to (9) the local
fairness loss ($\mathcal{L}_{local}$) as shown in (11). Here, the accuracy of
the sensitive variable is reduced by $5\%$ to $53\%$, while leaving the label
prediction rate the same. Finally, considering, the full loss
$\mathcal{L}_{full}$ by using $\lambda_{1}=1$, $\lambda_{2}=1$,
$\lambda_{3}=0.1$ in (12), achieves the best results with sensitive accuracy
of $53\%$ and true labels $y$ accuracy of $61\%$. Overall, we observe that
local fairness improves accuracy, but only in the case where a disentangled
representation is used.

Next, we explore the impact of $\lambda_{3}$ on results (classifiers (12) -
(16)). Note that as $p_{z}$ is $0.5$ for train and $0$ for test, the
proportion of biased samples is different at train and test, and so increasing
$\lambda_{3}$ does not imply an increase in the test label $y$ accuracy. As
$\lambda_{3}$ increases, the prediction rate of the sensitive attribute also
increases, as a result of a bigger exposure of the learning scheme to the
inherent bias within the training labels. We also see that the impact on the
accuracy of the true labels $y$ is reaching a plateau at $\lambda_{3}=0.1$,
which indicates the limit of the relevant information in the training labels
for the true (unbiased) test labels. In real-world datasets where the test set
is biased, this increase in accuracy does not plateau, as our real dataset
experiments show.

| $\mathcal{L}_{rec}$ | $\mathcal{L}_{y}$ | $\mathcal{L}_{adv}$ | $\mathcal{L}_{local}$ | Acc | Acc
---|---|---|---|---|---|---
|  |  |  |  |  $a$ $\downarrow$ |  $y$ $\uparrow$
(1) | No | Yes ($1$) | No | No | $67\%$ | $57\%$
(2) | No | Yes ($1$) | Yes | No | $55\%$ | $58\%$
(3) | No | Yes ($1$) | Yes | Yes | $50\%$ | $50\%$
(4) | No | Yes ($1$) | No | Yes | $50\%$ | $50\%$
(5) | Yes | No | No | No | $67\%$ | $58\%$
(6) | Yes | Yes ($0.1$) | No | No | $66\%$ | $59\%$
(7) | Yes | No | No | Yes | $66\%$ | $56\%$
(8) | Yes | Yes ($0.1$) | No | Yes | $70\%$ | $58\%$
(9) | Yes | No | Yes | No | $58\%$ | $60\%$
(10) | Yes | Yes (0.1) | Yes | No | $58\%$ | $60\%$
(11) | Yes | No | Yes | Yes | $53\%$ | $60\%$
(12) | Yes | Yes ($0.1$) | Yes | Yes | $53\%$ | $61\%$
(13) | Yes | Yes ($0.2$) | Yes | Yes | $54\%$ | $61\%$
(14) | Yes | Yes ($0.5$) | Yes | Yes | $55\%$ | $61\%$
(15) | Yes | Yes ($0.75$) | Yes | Yes | $56\%$ | $60\%$
(16) | Yes | Yes ($1$) | Yes | Yes | $56\%$ | $60\%$
Table 1: Exploring the contribution of various loss terms. Row (12)
corresponds to our complete method. Row (2) corresponds to the method of (Adel
et al. 2019) (no disentanglement). All four variants of (1-4) include
classification loss $\mathcal{L}_{y}$. Row (1) shows a regular classifier, (2)
adds the adversarial loss $\mathcal{L}_{adv}$, rows (3) and (4) add our local
fairness $\mathcal{L}_{local}$. Rows (5-8) consider the setting where
$\mathcal{L}_{rec}$ is used, but where $\mathcal{L}_{adv}$ is not. In the
disentangled setting, rows (9-16) all rows include reconstruction loss
$\mathcal{L}_{rec}$ and the adversarial loss $\mathcal{L}_{adv}$. (9) applies
the method of (Hadad, Wolf, and Shahar 2018), focusing only on the
disentanglement component. (10) adds the classification loss $\mathcal{L}_{y}$
with $\lambda_{3}=0.1$ and row (11) shows results with local fairness
$\mathcal{L}_{local}$ without $\mathcal{L}_{y}$. Rows (12- 16) show the impact
of $\lambda_{3}$ together with $\mathcal{L}_{rec}$, $\mathcal{L}_{adv}$ and
$\mathcal{L}_{local}$, by assigning $\lambda_{3}$ to the values
$\\{0.1,0.2,0.5,0.75,1\\}$. The value of $\lambda_{3}$ is indicted in
brackets.

#### Varying the local neighborhood size

Next we consider the effect of varying the local neighborhood size, $K$ as
defined in Eq. 9. We use the synthetic dataset defined earlier, and consider
three different setting: (1) $\lambda_{3}=0$, only disentanglement and local
fairness is used, (2) $\lambda_{3}=0.1$, our default setting, (3)
$\lambda_{3}=1$. As shown in Tab. 2, a value of $K=4$ results in the lowest
(best) accuracy of the sensitive attribute $a$. A large value of $K$ (16)
results in a significantly worse accuracy value of $a$, as the effect of our
loss diminishes. The test label $y$ accuracy values are very similar, varying
by at most $1\%$ for each setting.

$\lambda_{3}$ ($\mathcal{L}_{y}$) | $K$ | Accuracy $a$ $\downarrow$ | Accuracy $y$ $\uparrow$
---|---|---|---
$0$ | $2$ | $53\%$ | $59\%$
$0$ | $4$ | $52\%$ | $58\%$
$0$ | $8$ | $53\%$ | $58\%$
$0$ | $16$ | $54\%$ | $59\%$
$0.1$ | $2$ | $52\%$ | $60\%$
$0.1$ | $4$ | $52\%$ | $60\%$
$0.1$ | $8$ | $52\%$ | $60\%$
$0.1$ | $16$ | $57\%$ | $60\%$
$1$ | $2$ | $56\%$ | $59\%$
$1$ | $4$ | $55\%$ | $59\%$
$1$ | $8$ | $55\%$ | $60\%$
$1$ | $16$ | $58\%$ | $60\%$
Table 2: Effect of varying $K$, the local neighborhood of samples considered as part of the local fairness loss. We consider three different settings with $\lambda_{3}=0$ (only disentanglement and local fairness), $\lambda_{3}=0.1$ (our default setting) and $\lambda_{3}=1$.  |
---|---
(a) | (b)
Figure 4: Effect of varying the training set bias $p_{z}$ (synthetic dataset)
on the accuracy of the sensitive attribute (a), where lower is better, and
accuracy of the test labels $y$ (b), where higher is better. Note the labels
prediction $y$ is measured on test data for which $p_{z}=0$, that is no biased
exists. Error bars indicate the standard error.

#### Varying the bias

We consider the effect of varying the bias, $p_{z}$, of the training set. We
consider the synthetic dataset as defined earlier, but where $p_{z}=0.25$,
$p_{z}=0.5$ (default) and $p_{z}=0.75$. As can be seen in Fig. 4(a),
increasing the value of $\lambda_{3}$ beyond $0.1$, increases the accuracy
value of the sensitive attribute a. The increase is most significant for a
large bias value of $0.75$, where the accuracy reaches a value of $61\%$. With
regards the the test label $y$ accuracy in Fig. 4(b), increasing the value of
$\lambda_{3}$ beyond $0.1$, results in either the same or lower accuracy. A
value of $\lambda_{3}=0.1$ results in maximal accuracy for $p_{z}=0.5$
(default) and $p_{z}=0.25$.

#### Limitations

Our method has some limitations, first with respect to the ability to support
multiple sensitive attributes at once. One may expand the loss terms to serve
as individual heads. However, this also opens the sub-group fairness question,
not addressed in this study We believe our fairness loss can support sub-
groups, but this is left for followup research.

## Conclusion

In this work, we considered the role of disentangled and locally fair
representations in making fair predictions. We proposed to learn a locally
fair representation that ensures the neighborhood of each sample is balanced
in terms of the sensitive attribute. To find this neighbourhood, we embedded
samples into a feature space where samples are not correlated to the sensitive
attribute. This was done by learning a disentangled representation which
separates each sample into features that are correlated to the sensitive
attribute and those that are not. We considered several real-world datasets,
where we analyzed the trade-off between accuracy and fairness. For both the
COMPAS and adult dataset, our method achieves the overall best tradeoff
between accuracy and fairness. In particular, where high level of accuracy is
required, best fairness is achieved. On a synthetic dataset, we demonstrated
that both disentanglement and local fairness are required for fairness
improvement and that a classifier further helps to control the tradeoff
between accuracy and fairness. As future work, one could consider our notion
of local fairness in the context of causal based fairness and individual
fairness.

## Acknowledgments

This project has received funding from the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation programme
(grant ERC CoG 725974).

## References

  * Adel et al. (2019) Adel, T.; Valera, I.; Ghahramani, Z.; and Weller, A. 2019.  One-network adversarial fairness.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, 2412–2420.
  * Adler et al. (2018) Adler, P.; Falk, C.; Friedler, S. A.; Nix, T.; Rybeck, G.; Scheidegger, C.; Smith, B.; and Venkatasubramanian, S. 2018.  Auditing black-box models for indirect influence.  _Knowledge and Information Systems_ , 54(1): 95–122.
  * Agarwal et al. (2018) Agarwal, A.; Beygelzimer, A.; Dudík, M.; Langford, J.; and Wallach, H. 2018\.  A reductions approach to fair classification.  In _International Conference on Machine Learning_ , 60–69. PMLR.
  * Asuncion and Newman (2007) Asuncion, A.; and Newman, D. 2007.  UCI machine learning repository.
  * Barocas and Selbst (2016) Barocas, S.; and Selbst, A. 2016.  Big data’s disparate impact. 104 California Law Review 671.  _Available online at SSRN: www. ssrn. com/abstract_ , 2477899.
  * Bechavod and Ligett (2017) Bechavod, Y.; and Ligett, K. 2017.  Learning fair classifiers: A regularization-inspired approach.  _arXiv preprint arXiv:1707.00044_ , 1733–1782.
  * Benaim et al. (2019) Benaim, S.; Khaitov, M.; Galanti, T.; and Wolf, L. 2019.  Domain intersection and domain difference.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 3445–3453.
  * Beutel et al. (2017) Beutel, A.; Chen, J.; Zhao, Z.; and Chi, E. H. 2017.  Data decisions and theoretical implications when adversarially learning fair representations.  _arXiv preprint arXiv:1707.00075_.
  * Calders, Kamiran, and Pechenizkiy (2009) Calders, T.; Kamiran, F.; and Pechenizkiy, M. 2009.  Building classifiers with independency constraints.  In _2009 IEEE International Conference on Data Mining Workshops_ , 13–18. IEEE.
  * Cho, Hwang, and Suh (2020) Cho, J.; Hwang, G.; and Suh, C. 2020.  A fair classifier using kernel density estimation.  _Advances in Neural Information Processing Systems_ , 33: 15088–15099.
  * Creager et al. (2019) Creager, E.; Madras, D.; Jacobsen, J.-H.; Weis, M.; Swersky, K.; Pitassi, T.; and Zemel, R. 2019.  Flexibly fair representation learning by disentanglement.  In _International Conference on Machine Learning_ , 1436–1445. PMLR.
  * Dua and Graff (2017a) Dua, D.; and Graff, C. 2017a.  UCI Machine Learning Repository, http://archive.ics.uci.edu/ml//datasets/Communities+and+Crime.
  * Dua and Graff (2017b) Dua, D.; and Graff, C. 2017b.  UCI Machine Learning Repository, https://archive.ics.uci.edu/ml/datasets/adult.
  * Dua and Graff (2017c) Dua, D.; and Graff, C. 2017c.  UCI Machine Learning Repository, https://archive.ics.uci.edu/ml/datasets/Bank+Marketing.
  * Dwork et al. (2012) Dwork, C.; Hardt, M.; Pitassi, T.; Reingold, O.; and Zemel, R. 2012.  Fairness through awareness.  In _Proceedings of the 3rd innovations in theoretical computer science conference_ , 214–226.
  * Edwards and Storkey (2015) Edwards, H.; and Storkey, A. 2015.  Censoring representations with an adversary.  _arXiv preprint arXiv:1511.05897_.
  * Feldman et al. (2015) Feldman, M.; Friedler, S. A.; Moeller, J.; Scheidegger, C.; and Venkatasubramanian, S. 2015.  Certifying and removing disparate impact.  In _proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining_ , 259–268.
  * Hadad, Wolf, and Shahar (2018) Hadad, N.; Wolf, L.; and Shahar, M. 2018.  A two-step disentanglement method.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 772–780.
  * Hardt, Price, and Srebro (2016) Hardt, M.; Price, E.; and Srebro, N. 2016.  Equality of opportunity in supervised learning.  _arXiv preprint arXiv:1610.02413_.
  * Kamiran, Calders, and Pechenizkiy (2010) Kamiran, F.; Calders, T.; and Pechenizkiy, M. 2010.  Discrimination aware decision tree learning.  In _2010 IEEE International Conference on Data Mining_ , 869–874. IEEE.
  * Kamishima et al. (2012) Kamishima, T.; Akaho, S.; Asoh, H.; and Sakuma, J. 2012.  Fairness-aware classifier with prejudice remover regularizer.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , 35–50. Springer.
  * Kim, Reingold, and Rothblum (2018) Kim, M. P.; Reingold, O.; and Rothblum, G. N. 2018.  Fairness through computationally-bounded awareness.  _arXiv preprint arXiv:1803.03239_.
  * Lample et al. (2017) Lample, G.; Zeghidour, N.; Usunier, N.; Bordes, A.; Denoyer, L.; and Ranzato, M. 2017.  Fader networks: Manipulating images by sliding attributes.  _arXiv preprint arXiv:1706.00409_.
  * Larson et al. (2016) Larson, J.; Mattu, S.; Kirchner, L.; ; and Angwin, J. 2016.  ProPublica COMPAS dataset.  _https://github.com/propublica/compas-analysis._
  * Locatello et al. (2019) Locatello, F.; Abbati, G.; Rainforth, T.; Bauer, S.; Schölkopf, B.; and Bachem, O. 2019.  On the fairness of disentangled representations.  _arXiv preprint arXiv:1905.13662_.
  * Menon and Williamson (2018) Menon, A. K.; and Williamson, R. C. 2018.  The cost of fairness in binary classification.  In _Conference on Fairness, Accountability and Transparency_ , 107–118. PMLR.
  * Perrone et al. (2021) Perrone, V.; Donini, M.; Zafar, M. B.; Schmucker, R.; Kenthapadi, K.; and Archambeau, C. 2021.  Fair bayesian optimization.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , 854–863.
  * Pleiss et al. (2017) Pleiss, G.; Raghavan, M.; Wu, F.; Kleinberg, J.; and Weinberger, K. Q. 2017.  On fairness and calibration.  _arXiv preprint arXiv:1709.02012_.
  * Primus (2010) Primus, R. 2010.  The future of disparate impact.  _Michigan Law Review_ , 1341–1387.
  * Tan et al. (2020) Tan, Z.; Yeom, S.; Fredrikson, M.; and Talwalkar, A. 2020.  Learning fair representations for kernel models.  In _International Conference on Artificial Intelligence and Statistics_ , 155–166. PMLR.
  * Tishby and Zaslavsky (2015) Tishby, N.; and Zaslavsky, N. 2015.  Deep learning and the information bottleneck principle.  In _2015 IEEE Information Theory Workshop (ITW)_ , 1–5. IEEE.
  * Tzeng et al. (2014) Tzeng, E.; Hoffman, J.; Zhang, N.; Saenko, K.; and Darrell, T. 2014.  Deep domain confusion: Maximizing for domain invariance.  _arXiv preprint arXiv:1412.3474_.
  * Wadsworth, Vera, and Piech (2018) Wadsworth, C.; Vera, F.; and Piech, C. 2018.  Achieving fairness through adversarial learning: an application to recidivism prediction.  _arXiv preprint arXiv:1807.00199_.
  * Xu et al. (2019) Xu, L.; Skoularidou, M.; Cuesta-Infante, A.; and Veeramachaneni, K. 2019.  Modeling tabular data using conditional gan.  _arXiv preprint arXiv:1907.00503_.
  * Zafar et al. (2017a) Zafar, M. B.; Valera, I.; Gomez Rodriguez, M.; and Gummadi, K. P. 2017a.  Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment.  In _Proceedings of the 26th international conference on world wide web_ , 1171–1180.
  * Zafar et al. (2017b) Zafar, M. B.; Valera, I.; Rogriguez, M. G.; and Gummadi, K. P. 2017b.  Fairness constraints: Mechanisms for fair classification.  In _Artificial Intelligence and Statistics_ , 962–970. PMLR.
  * Zhang, Lemoine, and Mitchell (2018) Zhang, B. H.; Lemoine, B.; and Mitchell, M. 2018.  Mitigating unwanted biases with adversarial learning.  In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ , 335–340.