11institutetext: Machine Learning and Knowledge Representation Lab,
Innopolis University, Innopolis, Russia
11email<EMAIL_ADDRESS><EMAIL_ADDRESS>
22institutetext: Departments of Informatics, University of Oslo, Oslo, Norway
22email<EMAIL_ADDRESS>

# RepFair-GAN: Mitigating Representation Bias in GANs Using Gradient Clipping

Patrik Joslin Kenfack 11 Kamil Sabbagh 11 Adín Ramírez Rivera 22 Adil Khan 11

###### Abstract

Fairness has become an essential problem in many domains of Machine Learning
(ML), such as classification, natural language processing, and Generative
Adversarial Networks (GANs). In this research effort, we study the unfairness
of GANs. We formally define a new fairness notion for generative models in
terms of the distribution of generated samples sharing the same protected
attributes (gender, race, etc.). The defined fairness notion (representational
fairness) requires the distribution of the sensitive attributes at the test
time to be uniform, and, in particular for GAN model, we show that this
fairness notion is violated even when the dataset contains equally represented
groups, i.e., the generator favors generating one group of samples over the
others at the test time. In this work, we shed light on the source of this
representation bias in GANs along with a straightforward method to overcome
this problem. We first show on two widely used datasets (MNIST, SVHN) that
when the norm of the gradient of one group is more important than the other
during the discriminator’s training, the generator favors sampling data from
one group more than the other at test time. We then show that controlling the
groups’ gradient norm by performing group-wise gradient norm clipping in the
discriminator during the training leads to a more fair data generation in
terms of representational fairness compared to existing models while
preserving the quality of generated samples.

###### Keywords:

Fairness Representation bias GANs

##  1 Introduction

The increasing use of Machine Learning (ML) models to make high-stake
decisions in areas like hiring, college admission and criminal justice has
raised concerns about the fairness of decisions made by these models [3]. In
fact, the data collected to train these models may also incorporate biases or
historical discrimination from the real world. As ML models are good at
discovering patterns in the data, they can also learn and perpetuate these
biases, leading to discriminatory outcomes against subgroups in the population
(minorities) [22, 5, 17]. Several efforts have been made to define and
quantify fairness mathematically [9, 11, 7, 4]. The proposed definitions of
fairness aim to equalize models’ performances or error rates across groups
present in the dataset. Algorithms to achieve these fairness notions can be
classified into three main categories: pre-processing [29, 12, 20, 16], in-
processing [30], and post-processing [11], depending on whether biases are
mitigated at the data level, during model training or after model training,
respectively. Biases can originate from different sources such as the real
world, user interactions, and algorithmic bias [22]. The most common type of
bias in the real world is the representation bias, which occurs when certain
groups or subgroups are under-represented in the dataset. The model will tend
to be biased in favor of the well-represented groups. One way to mitigate
representation bias is to collect more data from under-represented groups, but
these data are difficult to collect or are not available in most cases. To
overcome this limitation, one can rely on generative models to augment the
dataset with synthetic data [26, 21].

Generative Adversarial Network (GAN) is considered as the most prominent
technique for learning the underlying distribution of data [10]. The model
consists of two competing networks: the generator $G$, which captures the
distribution by sampling as realistic data as possible from the noize vector,
and the discriminator $D$, which distinguishes between samples from $G$ (fake
sample) and the data. The goal of $G$ is to defeat the discriminator to
distinguish between fake and real samples. Once this goal is achieved, $G$ can
be used to generate synthetic data. GAN models and their variants have shown
incredible results in synthetic image generation [13, 14] and music generation
[8]. As GAN models are increasingly used as data augmentation tools, it is
crucial to ensure that the sampling process is fair and free from any type of
bias. Previous works have shown that GANs can also generate biased data [15,
27, 25] by encoding disparities in the data and generating imbalance data,
i.e., synthetic data close to realistic data but are less representative of
the underlying population. In essence, work done in [15, 25] has shown that
when certain groups are under-represented (minorities) in the data, the
generator tends to favor well-represented groups at testing time, leading to
skewed data that can result in unfairness in downstream tasks [22].

This paper addresses the issue of representation bias in GAN; we provide
insights into why the generator favors one group at the test time. We observe
through experiments that when there are disparities in the magnitude of the
gradients of different groups in discriminator during training, the generator
tends to favor the groups with larger magnitude at test time, i.e., the
generate samples points from one group more often than the other. Based on the
previous observation, we propose a training process that allows the generator
to uniformly generate data from different groups at the testing time by
controlling the gradient of each group during the training. We achieve this by
clipping the norm of the gradient of groups during the training of the
discriminator; This avoids the discriminator to be more powerful in certain
groups than the other, and as such, it enforces the generator to not mainly
converges towards the group with which it can easily defeat the discriminator.
Our method can be incorporated easily in many GAN variants as the
discriminator’s gradient is readily available in general.

The contributions of this paper are as follows:

  * •

We formalize a fairness notion for generative models (representational
fairness) that promotes the uniformity of the sensitive attributes in the
samples generated at the test time. In the context of GAN, this notion is
achieved if, at the test time, the generator uniformly samples points from
different groups present in the training dataset. In other words, a well-
trained generator violates this notion if it is more likely to sample points
from one group than the others.

  * •

We conducted empirical studies to reveal the reason why GANs generator may
favor one group over the others at the test time. We showed that the magnitude
of the gradient of groups on the discriminator side during the training plays
an important role in guiding the generator to sample groups uniformly at the
test time. We considered the cases where group representations in the training
dataset are balanced and imbalanced. Our results also showed that even when we
train GAN on balanced datasets, the generator can still favor one group over
the others at testing time.

  * •

We proposed Representational Fair-GAN (RepFair-GAN) an adaptation of GAN that
clips the gradient norm group-wise to avoid one group dominating the training
and to provide a more uniform (fair) data generation at the testing time.

  * •

We demonstrated through intensive experiments that our method–yet simple– is
effective and can generate a more balanced dataset than the existing GANs
models, even when the dataset is imbalanced.

The rest of this paper is organized as follows. In Section 2, we discuss
related work and the position of our work. In Sections 3 and 4, we present
empirical evidence of the effect of disparity in groups’ gradient norms on the
fairness of the generator and experimental results respectively. Section 5 is
dedicated to the conclusion and future works.

##  2 Related Work

The approaches closest in spirit to our work involve fair data generation
using GANs. These approaches differ mainly in the fairness notion they aim to
achieve or the type of bias they aim to mitigate. FairGAN [27] uses one
generator and two discriminators. As in the classic GAN, the goal of the first
generator is to distinguish between fake and real samples while the goal of
the second discriminator is to predict the sensitive attribute (ethnicity,
gender) from fake and real samples. In a mini-max game optimization, the goal
of the generator in FairGAN is to defeat the two discriminators in their
respective objectives. Thus, FairGAN generates samples that are independent of
the sensitive attribute to achieve statistical parity in a downstream
classifier. Statistical parity [9] is a fairness notion mainly used in
classification tasks that promotes the independence between the classifier’s
positive outcome and the sensitive attribute, i.e.,
$P(Y=1\>\lvert\>S=1)=P(Y=1\>\lvert\>S=0)$ where $S$ is the random variable
describing the sensitive attribute and $Y$ the classifier’s output. Xu et al.
[28] take a similar approach but add a classifier and consider three
discriminators to improve the predictive performance of the generated samples
while satisfying other notions of fairness such as equalized odds and equal
opportunity [11]. In contrast to previous work, our work does not target any
specific downstream task or fairness notion but focuses on fair sampling. As
GANs are primarily used as a data augmentation tool, our goal is to mitigate
the representation bias exhibited by the generator [15], i.e., to improve the
ability of GAN models to uniformly generate samples from different groups,
even when these groups are not equally represented in the training data.

More closely related to our work, Choi et al. [6] use instance reweighing
during the training of GANs such that over-represented data points are down-
weighted while under-represented are up-weighted. Tan et al. [25] propose a
post-training approach that considers already well-trained GANs, and shifts
the distribution of the latent space in a way to provide a more fair sampling.
In our work, we control the gradient flowing from the discriminator, in a way
that it does not provide a bigger signal for one group than others. We clip
the gradient norm group-wise, in a way that the magnitude of the gradient of
one group will not enlarge during the training. Our empirical results
demonstrate that our training process results in fairer vanilla GANs in terms
of uniformity of the sampled data points, i.e., all groups are equally likely
to be sampled by the generator at test time. The advantages of our method are
as follows: (1) Gradient clipping is readily available and general, and as
such, can be incorporated easily for many GAN variants. In our experiment, we
use the vanilla GANs [10] and the conditional GAN (CGAN) [23]. (2) It
preserves high-quality data generation with a carefully chosen clipping
factor. (3) When the groups are not equally represented in the training data,
our training process leads to uniform data generation while preserving the
data quality of both groups.

##  3 RepFair-GAN

In this section, we provide more details on the problem statement with the
evidence of biased data generation of the classic GANs model. We also
demonstrate the influence of disparity in groups’ gradient magnitude over the
generation process and describe the training of our method (RepFair-GAN).

###  3.1 Representational Fairness

We consider a dataset $\mathcal{D}=\\{\mathcal{X},\mathcal{S}\\}$ of size $N$
where $\mathcal{X}=\\{x_{i}\\}_{i=1}^{N}$ ; $x_{i}\in\mathbb{R}^{d}$
represents data points sampled from an unknown distribution
$P_{\operatorname{data}}(\mathcal{X})$, and
$\mathcal{S}=\\{s_{i}\\}_{i=1}^{N}$ the binary sensitive attribute associated
with each point. Each data point $x_{i}$ can be an image and we assume the
image contains sensitive attributes, like the gender of the face in the image.
The goal of a GANs is to estimate the underlying distribution
$P_{\theta}(\mathcal{X})$ of the data, such that
$P_{\theta}(\mathcal{X})=P_{\operatorname{data}}(\mathcal{X})$. In the classic
GAN, the optimization aims to minimize the Jensen-Shannon divergence between
the two distribution. Given a well trained GAN on the observed data
$\mathcal{D}$, its generator
$g_{\theta}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{d}$ can generate realistic
data $D_{\theta}=\\{g_{\theta}(\mathcal{Z})\\}$ where
$\mathcal{Z}=\\{z_{i}\\}_{i=1}^{N}$ ; $z_{i}\in\mathbb{R}^{n}$ represents
latent space codes drawn from a given distribution $P(\mathcal{Z})$, generally
assumed Gaussian.

In addition, we consider a function $h:\mathcal{X}\rightarrow\mathcal{S}$ that
predicts the sensitive attribute of the given image, e.g., predicting the
gender of the face in an image or predicting the color of a digit on an image.
We assume without loss of generality that the classifier $h$ is very accurate.
Using $h$, we can analyze the distribution of sensitive attributes of the
generated data points, i.e., $P(h(g_{\theta}(\mathcal{Z}))$. Our goal is to
enforce fair sampling by making the distribution of the sensitive attributes
over the generator $g_{\theta}$ close to uniform distribution, i.e.,
$\operatorname{KL}(P(h(g_{\theta}(\mathcal{Z}))\;\|\;\mathcal{U}(\mathcal{S}))=0$
where $\mathcal{U}$ is the uniform distribution, we refer to this fairness
notion as representational fairness. In essence, a biased generator has a
higher probability of sampling data points from one group than the other.
Formally, we define the ”representational fairness” notion for generative
models as follows:

###### Definition 1

A well-trained generator ($g_{\theta}$) is $\epsilon$-representationaly fair
with respect to the sensitive attributes $S$ if the distribution of $S$ over a
set point randomly sampled from $g_{\theta}$,
$D_{\theta}=\\{g_{\theta}(\mathcal{Z})\\}$, is closed to the uniform
distribution, i.e., for an $\epsilon\geq 0$,
$\;\operatorname{d}(P(h(g_{\theta}(\mathcal{Z}))\;\|\;\mathcal{U}(\mathcal{S}))\leq\epsilon$.
Where $\operatorname{d}$ is a distance over probability distributions, e.g.,
the KL-divergence; $\mathcal{U}$ the uniform distribution; and $\epsilon$ the
level of group balance achieved by $g_{\theta}$, i.e., the maximum distance
from the uniform distribution obtained across multiples random sampling from
$g_{\theta}$.

When $\epsilon=0$, the generative model achieves uniformity over the sensitive
attributes. The lower the value of $\epsilon$, the fairer the generator, while
a high value of $\epsilon$ indicates an imbalanced representation of the
sensitive attribute in $D_{\theta}$.

###  3.2 Biased data generation

Previous works have studied biases in GANs [25, 15, 6]. These works have
provided evidence that the distribution of generated data from different GAN
models do not follow the uniform distribution over the sensitive attribute.
They also provide evidence on how the biased data generated can lead to
unfairness on downstream tasks such as classification, i.e., a classifier
trained on synthetic data is more accurate on one group than another
(minorities). The main reason for this behaviour stems from the fact that
minority groups are underrepresented in the training data obtained by the
GANs. However, most of these works considered settings where the training data
contains imbalanced group distributions. In addition, we show in the
experiment section that even when the training contains a balanced group
representation, the generator still outputs non-uniform distribution at the
test time. We observe through the experiments that there is a disparity in the
magnitude of the gradient of each group during the training of the
discriminator, and we hypothesize that this is the source of bias generation
at the test time.

###  3.3 The influence of the magnitude of the gradient vectors

####  3.3.1 Why disparity in groups’ gradient norms causes unfair generation?

A question that naturally arises is why the magnitude of the gradient between
groups is different during training and how this lead to a biased generator.

$0$$20$$40$$60$$80$$100$$0.42$$0.44$$0.46$$0.48$$0.5$EpochGradient norm(a)
Gradient magnitude over epochNormal imagesShaded
imagesNormalShaded$0$$20$$40$$60$$80$Frequency (%)(b) Group distribution
Figure 1: Average gradient norms of different groups across epochs during the
training of the vanilla GANs in the SVHN dataset. The difference in the
magnitude of the gradients of each group enlarges during the training (a).
After the training, the generator samples more data points from the group with
a larger gradient during the training (b).

In the mini-max training of GANs, the discriminator is simply a classifier
that distinguishes samples provided by the generator, from real samples
obtained from the training data. However, as described by Goodfellow et al.
[10], if the discriminator is too good, it may not provide enough information
in the gradients (vanishing) to improve the generator, resulting in a
convergence failure [10]. We intuitively hypothesize that if the discriminator
is stronger at distinguishing samples in one group better than the others, it
can provide more information in the gradient for the other group, leading this
generator to converge more toward the group for which the discriminator
provided more information. In the SVHN dataset, we created two groups by
adding a black opacity layer over certain images (see section 4). We performed
experiments to analyze the amount of signal the discriminator provides for
each group in the training dataset. Figure 1 showcases the average gradient
norms of two groups present in the data during the training. We observe the
disparities in the magnitude of their gradients is close to each other at the
beginning of the training, and then enlarge over epochs. At the test time, we
observed that the generator samples imbalanced distribution in which the group
with higher gradient magnitude (shaded images) was over-represented (60% vs
40%). Note that in training data both groups were equally represented.
Experiment with groups imbalance is presented in the section 4.

####  3.3.2 Algorithm

Based on the previous empirical analysis, we can see the influence of the
gradient norm of groups over the biased sampling at the test time. We designed
a straightforward training process of GANs that controls the gradient of each
group during training, such that the disparities in the magnitude of the
gradient will not enlarges during the training. The key idea of our method is
to modify the training of the process of the discriminator, by alternating
groups used to optimize the discrimination across batches, and clipping the
norm of their gradients when it exceeds a defined threshold (maximum gradient
norm).

Algorithm 1 Training of the discriminator in RepFair-GAN (outline)

$t$ the batch size, $M$ training epochs, $C$ maximum gradient norm, and
$\mathcal{L}_{D}(\Theta)$ the discriminator’s loss function with network
parameter $\Theta$.

for $n\leftarrow 1,M$ do

$U\leftarrow True$ $\triangleright$ Boolean indicating whether to update with
samples from group 0 or group 1

for each batch of size $t$ do

if U then

$X_{t}\leftarrow\\{D|S=0\\}$ $\triangleright$ random mini-batch of real
samples from group 0

else

$X_{t}\leftarrow\\{D|S=1\\}$ $\triangleright$ random mini-batch of real
samples from group 1

end if

$g_{t}\leftarrow\nabla_{\Theta}\mathcal{L}_{D}(\Theta;X_{t})$ $\triangleright$
Compute the gradient using the discriminator’s loss

$\bar{g_{t}}\leftarrow\operatorname{clip}(g_{t},C)$ $\triangleright$ Clip the
gradient norm with max norm $C$

$\Theta\leftarrow\Theta-\alpha\bar{g_{t}}$ $\triangleright$ Update network
param.

$U\leftarrow\operatorname{not}U$

Generate samples from the generator $\bar{X_{t}}$ feed to the discriminator
and take a gradient step.

end for

end for

Algorithm 1 summarizes the training of the discriminator in our approach. We
note that gradient clipping is an ingredient used in stochastic gradient
descent (SGD) to solve different purposes. For instance, the gradient clipping
is used to mitigate the gradient exploding problem, which arises when the
weights of the model become very large so that it creates an overflow or
underflow problem [19]. The clipping can be done on weights of models (value
clipping) or on the norm of the gradient (norm clipping). Abadi et al. [1]
used norm clipping to bound the influence of each training example to provide
a privacy guarantee in SGD. In the same spirit, we apply of the norm clipping
in our method to bound the gradient norm of different groups in the data such
that none of these will provide an increasingly larger signal throughout the
training of GAN. In our method, there is no a priori bound on the size of the
gradients, thus, the maximum gradient norm parameter ($C$) must be carefully
chosen. A too small value of $C$ will avoid the generator to learn, due to the
vanishing gradient problem, while a too-large value of $C$ will not change
anything to training leading to the classic GAN’s training with bias data
generation. In the next section, we show the effectiveness of our method on
two datasets.

##  4 Experiments

To demonstrate the effectiveness of our approach, we experiment on two
datasets in which we created two different groups with different
representation rates. We trained GAN models using the classical training
procedure and compared them to our training process (algorithm 1) in terms of
the distribution of the generated groups at the time of test time over several
training runs.

###  4.1 Experiment settings

Figure 2: Example of two groups created from MNIST (left) and SVHN (right)
datasets. In MNIST, one group of digits has a black background (two first
rows) and another group has a white background (two last rows). In SVHN, one
with normal images (first two rows), and for the second group we added a black
overlay (shaded)

####  4.1.1 Datasets

We used MNIST [18] and SVHN [24] datasets to run the experiments. MNIST
dataset consists of 60,000 samples of labelled handwritten digits, for which
we created two groups, one with white background color and the other with a
black background color. SVHN contains 73,256 samples of home street numbers,
which adopted a similar process to create groups, with the difference that the
labels were not taken into consideration. As in [15], the dataset was divided
into two groups: the shaded and the normal SVHN images. For the shaded images,
we added a shade filter on the photos and the other group contains the normal
SVHN. Figure 2 showcases examples of samples from each group for both
datasets. We performed the experiments with balanced and imbalanced group
representation.

####  4.1.2 Models

For GAN models, we considered both the Vanilla and the CGAN. We trained
Vanilla GAN on the SVHN dataset with different group representations and
reported group distribution after sampling 10,000 points from the trained
generator. We trained CGAN conditioned with class label on MNIST dataset and
reported the results class-wise, i.e., group distribution for every type of
digit. The objective was to examine whether the imbalance in group
distribution affects all classes of digits or whether it is just an overall
imbalance. All the models were trained with the classic training process and
our proposed method. All generators were trained seven times, and the group
distribution is averaged over all training runs.

For the model’s architecture, for MNIST, we built a discriminator model whose
input is a grey-scale $28\times 28$ image and an integer representing the
class label. The discriminator output is a binary prediction deciding if the
generated image is real or fake. For our purpose, we implemented the
discriminator neural network that follows GAN design common best practices
such as choosing LeakyRelU as the activation function with slope 0.2 and
$2\times 2$ stride to downsample [2]. For CGAN, the discriminator takes as
input the image and the class label. The label will pass through a layer of
embedding of size 50. Then the embedded layer is connected to the dense layers
then is concatenated with the image before it goes through two convolutional
layers with the same padding and stride of $2\times 2$, each convolutional
layer is followed by LeakyRelU with a slope 0.2. We used the Adam optimizer
for SGD optimization with a learning rate of 0.0002 and a momentum of 0.5.

The generator model handles the conditional architecture as well by having in
the inputs the class label along with the noise vector sampled from Gaussian
distribution. The label is handled as in the discriminator by passing it
through the embedding layer of size 50, followed by a dense layer then
concatenated with the noise vector before going through two transposed
convolutional layers with the same padding and a stride of $2\times 2$,
followed by LeakyRelu with slope 0.2. the output of the generator model is a
single $28\times 28$ image generated by a convolutional layer with activation
function $tanh$

Finally, the GAN model was simply built by sequentially providing the
generator’s output to the discriminator. The CGAN model also uses an Adam
optimizer with a learning rate of 0.002 and a momentum of 0.5 and binary
cross-entropy as a loss function.

####  4.1.3 Metrics

To evaluate the group distributions of the generated data, we trained
classifiers on the real training data to predict the sensitive attribute. For
the MNIST dataset, the classifier is trained to predict the background color
of the digits, while for SVHN, it predicts whether the image is shaded or not.
The classifier was trained on 70% of the real images and achieved 100% and 98%
accuracy respectively for MNIST and SVHN on the testing dataset in SVHN. For
MNIST the group was defined by checking the pixel intensity of the corner of
the digit, i.e., if the intensity is greater than 0.5 we consider the digit as
a black background and vice versa. We note that the classifier used to obtain
the group distributions may also be biased, but for simplicity, we assume that
this is not the case here. Furthermore, to quantify the unfairness, one can
consider measuring the representation bias using the KL-divergence between the
distribution of the generated samples and the uniform distribution (definition
1). In the results, we reported the frequency distribution of each group. Note
that all experiments are based on 10 training runs, and frequencies are
averaged across runs.

$50$$100$$150$$200$$0.42$$0.42$$0.43$$0.43$EpochAverage gradient(a) Vanilla
GANWhite background digitsBlack background
digits$50$$100$$150$$200$$0.51$$0.52$$0.53$Epoch(b) RepFair-GAN Figure 3:
Effect of the gradient clipping over the disparity in gradient norms of each
group. Average gradient for white background digits and black background
digits vanilla GAN. Clipping enforces the gradient norm of each group to
remain closer during the training of the discriminator (b).

###  4.2 Results

####  4.2.1 Effect of gradient clipping

During the training of CGAN, with reported the average gradient norm of each
group across epochs. Figure 3 shows that in the unfair CGAN, the difference
between the gradient magnitude of digits with white and black backgrounds
tends to widen over epoch. In contrast, we observe that when using our
fairness mechanism, the magnitude of the gradient of each group remains closer
over epochs. We observed the same pattern on SVHN.

####  4.2.2 Frequency distributions

We performed the experiments to analyze the representation bias in CGAN on
MNIST dataset conditioned on the digits label. The model was trained for 200
epochs and the frequency of generating white background digits and black
background digits the results are reported in figure 4(a) shows that even when
groups are equally represented in the training set, the generator does not
uniformly sample them at the test time. We perform the same experiment using
the RepFair-GAN, the figure 4(b) shows the improvement in the uniform
generation of all groups.

We also analysed if the clipping can affect the quality of the generated
images. Figure 7 shows that our method can uniformly generate all groups while
preserving image quality. For each class, we observed that all colors were
equally represented while on CGAN without our fairness mechanism, digits with
black backgrounds over-represented. Our method reduces the difference in group
representation from about 18% to 3%. Similar results were observed on SVHN
datasets Cf. figures 5 and 6. This shows that by controlling the magnitude of
the gradient of each group, the discriminator is no more competent at
distinguishing real samples from one group than from the other, and thus,
pushes the generator to not converge primarily to the distribution of one
group while neglecting the other.

$0$$1$$2$$3$$4$$5$$6$$7$$8$$9$$20.00$$40.00$$60.00$Frequency(a) CGANWhite
background digitsBlack background
digits$0$$1$$2$$3$$4$$5$$6$$7$$8$$9$$20.00$$30.00$$40.00$$50.00$$60.00$DigitFrequency(b)
RepFair-GAN Figure 4: Class-wise background color distribution of digits
generated with CGAN trained on MNIST dataset. (a) Imbalanced digit color
distributions for each class produced by CGAN. Digit colors were balanced in
the training data and the generator favors generating digits with a black
background at the test time. (b) Balanced digit color distributions for each
class produced by RepFair-GAN. Our fairness mechanism provides a more uniform
group representation at the test time.
NormalShaded$0$$20$$40$$60$$80$percentages(a) Distribution Vanilla
GANNormalShaded$0$$20$$40$(b) Distribution RepFair-GAN Figure 5: Group
distributions for vanilla GAN (a) and our method (b) on SVHN dataset trained
with balanced group representation. Our method provides uniform generation
compared to the vanilla GAN.

Figure 6: Qualitative results on SVHN dataset. Examples of images generated by
GAN (left) and RepFair-GAN (right). We observe uniform generation of our
method along with similar images quality

Figure 7: Qualitative results on MNIST dataset. Examples of digits generated
by CGAN (left) and RepFair-GAN (right). We observe that for each type of
digit, group are equally represented with our method while preserving the same
quality as the classic GAN

####  4.2.3 Effect of maximum gradient norm

We trained CGAN on MNIST with our fairness mechanism using different values of
maximum gradient norms, $C\in\\{0.1,0.5,1,2,10,50,100\\}$. We reported the
overall group distribution, i.e., for all classes. In figure 8, we observed
that for $C=$ 0.1, 0.5, and 1 the clipping was too high and the model was
unable to converge properly, and the generator could not produce realistic
digits leading to almost random results in the distribution, i.e., images were
a mixture of white and black backgrounds. For norm $C$ between 2 and 10 the
clipping was good enough to let the model converge and to provide fair
sampling at the test time. For bigger values of $C$ ($\geq 20$ ), we observed
that the representation bias started occurring, suggesting that the maximum
gradient value was too big and did not influence the training. In all other
experiments, on MNIST and SVHN with $C=2$. The best value $C$ can be chosen
via hyper-parameter by turning on a validation set.

00.512102050100200$20.00$$40.00$$60.00$$80.00$Maximum gradient norm
($C$)FrequencyDigit background distributionsWhite background digitsBlack
background digits Figure 8: Group distributions with different maximum
gradient norms in RepFair-GAN on MNIST data. For max norm less than 2 the
training does not converge due to the vanishing gradient. For max gradient
norms between 2 and 10 groups are sampled uniformly and the sampling becomes
biased for values bigger than 20.

####  4.2.4 Effect of group imbalance

We trained vanilla GANs and RepFair-GAN on SVHN with different group imbalance
rates: 20%, 30%, 40%. As shown in the figure 9, vanilla GAN generates data
highly biased towards the well-represented group (shaded image) in all cases.
While distributions provided by our method are less biased. Surprisingly the
distribution at 20% is less skewed than that of 30%. In all cases, a default
value $C=2$ was used. Tuning this value on a validation dataset for each case
would result in better fairness performance.

NormalShaded$0$$20$$40$$60$percentages(a) 20% vs 80%
(GAN)NormalShaded$0$$20$$40$$60$(b) 20% vs 80% (RepFair-
GAN)NormalShaded$0$$20$$40$$60$percentages30% vs 70%NormalShaded$0$$20$$40$30%
vs 70%NormalShaded$0$$20$$40$$60$percentages40% vs
60%NormalShaded$0$$20$$40$40% vs 60% Figure 9: Group distributions on SVHN
with different group representation ratios 20%, 30%, and 40% respectively on
each row. Distribution on the left column is obtained from a classic vanilla
GAN and distribution on the right are obtained from the Vanilla GAN trained
with our fairness mechanism (RepFair-GAN).

s

##  5 Conclusion and future work

In this work, we shed light on the reason for unfairness in GANs. Using
empirical evidence, we attributed this phenomenon to the disparity in groups’
gradient norms during the discriminator training. We proposed a new training
for GAN that mitigates the representation bias of the generator at the test
time. In a nutshell, our method controls the gradient of each group during the
training by alternating their gradient step within batches and clipping their
norms if it exceeds a defined maximum value. Our experiments demonstrated the
effectiveness of this process in generating groups present in data uniformly
even when groups are imbalanced in the dataset. In future work, we will test
our method against existing techniques for fair data generation while
considering more complex and high-dimensional data. Furthermore, this work
laid the foundation for bias analysis and mitigation in ML from the
perspective of gradient disparity. It is therefore important to examine the
applicability of the proposed fairness mechanism to other gradient-based
models and classification tasks. Also, the relationship between fair data
sampling and diversity in GANs remains a research direction that needs to be
explored.

## References

  * [1] Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. pp. 308–318 (2016)
  * [2] Arjovsky, M., Bottou, L.: Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862 (2017)
  * [3] Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. L. Rev. 104, 671 (2016)
  * [4] Berk, R.: Criminal justice forecasts of risk: A machine learning approach. Springer Science & Business Media (2012)
  * [5] Caton, S., Haas, C.: Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053 (2020)
  * [6] Choi, K., Grover, A., Singh, T., Shu, R., Ermon, S.: Fair generative modeling via weak supervision. In: International Conference on Machine Learning. pp. 1887–1898. PMLR (2020)
  * [7] Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. pp. 797–806 (2017)
  * [8] Dong, H.W., Hsiao, W.Y., Yang, L.C., Yang, Y.H.: Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32 (2018)
  * [9] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference. pp. 214–226 (2012)
  * [10] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural information processing systems 27 (2014)
  * [11] Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016)
  * [12] Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowledge and information systems 33(1), 1–33 (2012)
  * [13] Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017)
  * [14] Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4401–4410 (2019)
  * [15] Kenfack, P.J., Arapov, D.D., Hussain, R., Kazmi, S.A., Khan, A.: On the fairness of generative adversarial networks (gans). In: 2021 International Conference” Nonlinearity, Information and Robotics”(NIR). pp. 1–7. IEEE (2021)
  * [16] Kenfack, P.J., Khan, A.M., Hussain, R., Kazmi, S.: Adversarial stacked auto-encoders for fair representation learning. arXiv preprint arXiv:2107.12826 (2021)
  * [17] Kenfack, P.J., Khan, A.M., Kazmi, S.A., Hussain, R., Oracevic, A., Khattak, A.M.: Impact of model ensemble on the fairness of classifiers in machine learning. In: 2021 International Conference on Applied Artificial Intelligence (ICAPAI). pp. 1–6. IEEE (2021)
  * [18] LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
  * [19] Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 (2015)
  * [20] Madras, D., Creager, E., Pitassi, T., Zemel, R.: Learning adversarially fair and transferable representations. In: International Conference on Machine Learning. pp. 3384–3393. PMLR (2018)
  * [21] Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., Malossi, C.: Bagan: Data augmentation with balancing gan. arXiv preprint arXiv:1803.09655 (2018)
  * [22] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) 54(6), 1–35 (2021)
  * [23] Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  * [24] Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
  * [25] Tan, S., Shen, Y., Zhou, B.: Improving the fairness of deep generative models without retraining. arXiv preprint arXiv:2012.04842 (2020)
  * [26] Tanaka, F.H.K.d.S., Aranha, C.: Data augmentation using gans. arXiv preprint arXiv:1904.09135 (2019)
  * [27] Xu, D., Yuan, S., Zhang, L., Wu, X.: Fairgan: Fairness-aware generative adversarial networks. In: 2018 IEEE International Conference on Big Data (Big Data). pp. 570–575. IEEE (2018)
  * [28] Xu, D., Yuan, S., Zhang, L., Wu, X.: Fairgan+: Achieving fair data generation and classification through generative adversarial nets. In: 2019 IEEE International Conference on Big Data (Big Data). pp. 1401–1406. IEEE (2019)
  * [29] Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. In: International conference on machine learning. pp. 325–333. PMLR (2013)
  * [30] Zhang, B.H., Lemoine, B., Mitchell, M.: Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. pp. 335–340 (2018)