# Learning Fair Classifiers via Min-Max F-divergence Regularization
††thanks: This work was supported by NSF grants CCF 2100013, CAREER 1651492,
CNS 2209951 and CNS 2317192.

Meiyu Zhong Ravi Tandon  Department of Electrical and Computer Engineering
University of Arizona, Tucson, USA
E-mail: {meiyuzhong<EMAIL_ADDRESS>
###### Abstract

As machine learning (ML) based systems are adopted in domains such as law
enforcement, criminal justice, finance, hiring and admissions, ensuring the
fairness of ML aided decision-making is becoming increasingly important. In
this paper, we focus on the problem of fair classification, and introduce a
novel min-max F-divergence regularization framework for learning fair
classification models while preserving high accuracy.

Our framework consists of two trainable networks, namely, a classifier network
and a bias/fairness estimator network, where the fairness is measured using
the statistical notion of F-divergence. We show that F-divergence measures
possess convexity and differentiability properties, and their variational
representation make them widely applicable in practical gradient based
training methods. The proposed framework can be readily adapted to multiple
sensitive attributes and for high dimensional datasets. We study the
F-divergence based training paradigm for two types of group fairness
constraints, namely, demographic parity and equalized odds. We present a
comprehensive set of experiments for several real-world data sets arising in
multiple domains (including COMPAS, Law Admissions, Adult Income, and CelebA
datasets).

To quantify the fairness-accuracy tradeoff, we introduce the notion of
fairness-accuracy receiver operating characteristic (FA-ROC) and a
corresponding low-bias FA-ROC, which we argue is an appropriate measure to
evaluate different classifiers. In comparison to several existing approaches
for learning fair classifiers (including pre-processing, post-processing and
other regularization methods), we show that the proposed F-divergence based
framework achieves state-of-the-art performance with respect to the trade-off
between accuracy and fairness.

###### Index Terms:

Fair Machine Learning, Regularization.

##  I Introduction

Machine learning based solutions are being increasingly deployed and adopted
in various sectors of society, such as criminal justice, law enforcement,
hiring and admissions. Despite their impressive predictive performance, there
is a large body of recent evidence [1, 2, 3] which shows a flip side of using
data driven solutions: bias in decision making, which is often attributed to
the inherent bias present in training data. For instance, in criminal justice,
risk assessment algorithms are often used to assess the risk of recidivism
(reoffence), which is then used together with human input for decision making
[4]. A classic example is that of Correctional Offender Management Profiling
for Alternative Sanctions (COMPAS) algorithm [4] which measures the recidivism
risk and is used by judges for pretrial detention and release decisions. It
was shown in [4] that COMPAS often falsely predicts a higher risk for some
racial groups (specifically, african americans) compared to others. Another
prominent example of bias with respect to gender is when a job advertisement
system tends to show less STEM related job advertisements to women [5]. With
the proliferation of data driven algorithms, ensuring fairness becomes crucial
in the process of designing ML based decision making systems.

Notions of Fairness. There is a vast literature on the notions of fairness [6,
7, 8] which mainly falls into three categories: (1) Group Fairness [9, 10, 11]
which requires that the subjects in the protected and unprotected groups have
equal probability of being assigned to the positive predicted class. (2)
Individual Fairness [6, 8, 12] which requires that similar individuals
(measured by a domain specific similarity metric) should be treated similarly.
(3) Causality-based Fairness [13, 8]: which requires that using causality-
based tools to design fair algorithms. In this paper, we consider group
fairness, and focus on the notions of demographic parity (DP) and equalized
odds (EO). The techniques for achieving group fairness can be mainly divided
into (1) Pre-processing methods [2], which reduce bias by processing the
training data before being used for training; (2) In-processing methods [3,
14], which add fairness constraints via regularization in the training
process. (3) Post-processing methods [15], which appropriately modify the
model parameters post training.

Related Work on Learning Fair Classifiers. The prominent in-processing method
for learning fair classifiers is via regularization methods, where the key
idea is to train a classifier which minimizes the classification error
regularized by a bias penalty (which measures the discrepancy of the
classifier across population sub groups). We remark that there have been
several other approaches on regularization based training for fair
classification, which include: a) using a covariance proxy [3] to measure the
bias between predictions and sensitive attributes. Unfortunately, ensuring
small correlation does not necessarily satisfy the stronger requirement of
statistical independence.; b) kernel density estimation (KDE) based methods
which first estimate conditional probability distribution of classifier
predictions for each population group and use these as a fairness regularizer
[16]. KDE methods are appropriate when the data dimensionality is relatively
small but are not scalable for high dimensional problems; c) another approach
is to balance the TPR and FPR (true- and false-positive rates) across
population sub-groups while training [17]; and d) measuring bias by the mean
of Hirschfeld-Gebelein-Rényi (HGR) Maximum Correlation Coefficient [18] or
mutual information [14] between predictions and sensitive attributes.

Admittedly, there are numerous choices for adding fairness constraints, and
this opens up the following key questions: a) What is the optimal choice of a
fairness regularization for a given notion of fairness, as well as the dataset
and sensitive attributes? b) How does the regularization procedure impact the
resulting tradeoff between accuracy and bias? c) How should one design a
flexible procedure for learning fair classifiers which can work for high-
dimensional datasets and is compatible with gradient based optimization?

Main Contributions. To deal with the above challenges, we propose a general
min-max $F$-divergence regularization framework for learning fair classifiers.
$F$-divergence, denoted by $D_{f}(P||Q)$ measures the difference between two
probability distributions $P,Q$ and different choices of the function
$f(\cdot)$ lead to well-known divergence measures, such as KL divergence,
Hellinger distance and Total Variation (TV) distance. Specifically, we propose
to measure the bias using $F$-divergence between the classifier probability
distributions across protected and unprotected groups. We next summarize the
main contributions of this paper.

  * •

By leveraging the variational representation of $F$-divergence, we cast the
training process as a min-max optimization problem, which is suitable for
commonly used gradient based optimization methods. The flexibility of the
framework is two-fold: a) it can be readily applied in high-dimensional
datasets, and b) by varying the choice of $f$, one can test and validate
different proxies of achieving fairness within a single rubric.

  * •

To quantify the fairness-accuracy tradeoff, we introduce the notion of
fairness-accuracy receiver operating characteristic (FA-ROC) and also provide
some interesting theoretical properties when using Total Variation distance as
the measure of bias. Within this context, we also introduce the notion of low-
bias FA-ROC, which we argue is an appropriate measure to evaluate different
classifiers.

  * •

We present a comprehensive set of results on multiple real world datasets
(namely, COMPAS, Adult Census, Law School admissions and CelebA), and show the
superiority of the proposed approach versus existing regularization, pre- and
post-processing methods as discussed above. As an example, for the Adult
census dataset, $F$-divergence regularization leads to $\approx 13\%$ increase
in FA-AUC (area under the curve) compared to the state-of-the-art
regularization, pre- and post-processing methods for Demographic parity (we
achieve a gain of $\approx 10\%$ in FA-AUC for Equalized odds). For the high
dimensional dataset (CelebA), our method consistently achieves better
performance and receives a gain of $6\%$ w.r.t EO constraints in the Low-bias
region.

##  II Preliminaries on Fair Classification

We consider a supervised classification problem, where we are given a dataset
of $N$ users: ${\\{X_{n},\;Y_{n},\;Z_{n}\\}}_{n=1}^{N}$, where $X_{n}$ denotes
the set of features of user $n$; $Y_{n}$ represents the true label of user
$n$; $Z_{n}$ denotes the set of sensitive attributes of user $n$, which
depends on the dataset and underlying context. For instance, in predicting
recidivism risk in criminal justice, $X_{n}$ includes features such as prior
criminal history, demographic information, charge (type of crime); $Z_{n}$
represents sensitive attributes, for instance, race or gender111More
generally, $Z_{n}$ can take values from a discrete set, i.e.,
$Z_{n}\in\mathcal{G}$ and our formulation allows for the number of sensitive
groups, i,e,. $|\mathcal{G}|\geq 2$ to be any finite number.; $Y_{n}$ denotes
ground truth like whether a user will re-offend in two years. Our goal is to
build a fair binary classifier222We note that while our discussion in the
paper is for binary classification, the proposed framework can be readily
adopted for multi-class settings, as we discuss later in this section., which
yields the estimate of the probability of the true label defined as follows:

$\displaystyle\pi(\hat{Y}|X)\triangleq\begin{cases}\pi(0|x)=P(\hat{Y}=0|X=x)\\\
\pi(1|x)=P(\hat{Y}=1|X=x).\end{cases}$ (1)

Note that we do not use sensitive attributes $Z$ as an input to the
classifier. However, it is well known [8] that excluding the sensitive
attributes alone does not necessarily lead to a fair classifier due to
possible correlation between the sensitives attributes and features. For the
scope of this paper, we focus on two statistical notions of group fairness:
demographic parity (DP) and equalized odds (EO). Before we present group
fairness notions, we first introduce the definition of F-divergence:

######  Definition 1.

(F-divergence) Let function $f:{\mathbb{R}}_{+}\rightarrow\mathbb{R}$ be a
convex, lower-semicontinuous function satisfying $f(1)=0$. Given two
probability distributions P and Q on a measurable
space$(\mathcal{X},\mathcal{F})$, F-divergence is defined as:

$D_{f}(P\parallel Q)=E_{Q}\left[f\left(\frac{dP}{dQ}\right)\right]$

For instance, when $f(x)=x\log(x)$, this reduces to the KL divergence;
$f(x)=(x-1)^{2}$ corresponds to $\chi^{2}$ divergence; $f(x)=(1-\sqrt{x})^{2}$
corresponds to Squared Hellinger (SH) distance. Next, we now define the notion
of Demographic Parity (DP) as follows:

######  Definition 2.

(Demographic Parity) A binary classifier $\pi$ satisfies Demographic Parity
(DP) if its prediction $\hat{Y}$ is independent of the sensitive attribute
$Z$:

$\displaystyle\pi(1|Z=i)=\pi(1|Z=j)),~{}~{}\forall i\neq j.$

Following from previous works [16, 15, 7], the standard measurement of DP is
the difference between the conditional output probability distribution of the
classifier given sensitive group $i$ and $j$:

$\displaystyle\Delta_{DP}:=\sum_{i\neq j}|\pi(1|Z=i)-\pi(1|Z=j)|.$ (2)

Figure 1: Schematic of the Min-Max $F$-divergence regularization framework for
fair training. The classifier (parameterized by $w$) is trained to minimize
the regularized objective (containing classification loss $+$ F-divergence
regularization term). The estimator (parameterized by $\theta$) estimates the
F-divergence between distribution $\pi_{i}$ and distribution $\pi_{j}$,
measuring the bias in classification across groups $(i,j)$. For DP, the
distribution $\pi_{i}$ for a group $i$ is given by
$\pi_{i}\sim\pi(\hat{Y}\left|Z=i\right.)$, whereas, for EO,
$\pi_{i}\sim\pi(\hat{Y}\left|Z=i,Y=1\right)$. The two networks are trained
alternatively, where for a fixed classifier, the $F$-divergence estimation is
performed using a maximization problem leveraging the variational
representation of $F$-divergence.

Note that our notion of $\Delta_{DP}$ is the same as Total Variation distance.
For the special case when $\Delta_{DP}=0$, this reduces to the notion of
perfect demographic parity [9]. We note that for the case of multi-class
classification, the above notion generalizes by considering
$\sum_{\hat{y}}\sum_{i\neq
j}|\pi(\hat{Y}=\hat{y}|Z=i)-\pi(\hat{Y}=\hat{y}|Z=j)|$. In a similar manner,
we can define the notion of equalized odds as follows:

######  Definition 3.

(Equalized Odds) A binary classifier $\pi$ satisfies Equalized Odds (EO) if
its prediction $\hat{Y}$ is conditionally independent of its sensitive
attribute $Z$ given the label $Y$.

$\displaystyle\pi(1|Z=i,Y=y)=\pi(1|Z=j,Y=y)),$

where $\forall i\neq j~{}\text{and}~{}y\in\\{0,1\\}.$ Same as above, we define
the standard measurement of EO as follows:

$\displaystyle\Delta_{EO}:=\sum_{y}\sum_{i\neq
j}|\pi(1|Z=i,Y=y)-\pi(1|Z=j,Y=y)|.$ (3)

Motivation for $F$-divergence based Regularization– One prominent approach for
learning fair classifiers is that of fairness regularization, i.e., adding a
penalty term in the training loss function, which acts as a proxy to capture
the fairness constraints (either DP or EO). In this work, we propose to use
$F$-Divergence (as defined above) as the fairness regularization term in the
loss function. $F$-divergence family has natural benefits like convexity and
differentiability, which makes it an ideal candidate for gradient based
optimization algorithms. In addition, as we discussed in the introduction,
compared to other approaches, such as correlation/covariance between sensitive
attributes and classifier outputs, the $F$-divergence notions are stronger
notions to capture dependence and provide stronger fairness guarantees.
Furthermore, $F$-divergence notions often also have an operational
interpretation; for instance, the Kullback-Leibler (KL) and Chernoff
divergences control the decay rates of error probabilities [19]. As a member
of the $F$-divergence family, prior work which uses mutual information [14]
between classifier predictions and sensitive attributes is therefore a special
case of our framework.

##  III $F$-divergence Regularized Fair Training

We consider a classifier described by trainable parameters $w$, where
$L_{w}(\hat{y}_{n};y_{n})$ is the loss function333For our experiments, we use
binary cross-entropy loss function for training. between the output of the
classifier ($w$) and the ground truth of user $n$; and the fairness
constrained learning can be formulated as the following optimization problem
(denoted by OPT):

$\displaystyle\underset{w}{min\;}\frac{1}{N}\sum_{n=1}^{N}L_{w}(\hat{y}_{n};y_{n})\;+\lambda\sum_{i\neq
j}D_{f}^{(w)}(\pi_{i}\parallel\pi_{j}),$ (4)

where $D_{f}^{(w)}(\pi_{i}\parallel\pi_{j})$ is the $F$-divergence between
group $i$ and group $j$ for the classifier parameterized by $w$; $\lambda$ is
a hyperparameter that can be tuned to balance the trade-off between accuracy
and fairness. Solving OPT requires the estimation of $F$-divergence for a
classifier. To this end, we propose three $F$-divergence estimators to compare
their performances on fair classification problem. We first leverage the
variational representation of $F$-divergence which allows us to estimate
F-divergence in an efficient manner. We show a schematic of the $F$-divergence
based framework in Fig 1.

###  III-A Variational Representation of $F$-divergences (NN)

It is well known [19] that $F$-divergence between two distributions admits a
variational representation, given as

$\displaystyle D_{f}(P\parallel Q)=\underset{T(\cdot)}{sup}~{}E_{X\sim
P}\left[T(X)\right]-E_{X\sim Q}\left[f^{*}(T(X))\right],$ (5)

where the function $f^{\ast}(t)=\underset{x\in dom_{f}}{sup}\\{xt-f(x)\\}$
denotes the convex conjugate (also known as the Fenchel conjugate) of the
function $f$. The above variational representation involves a supremum over
all possible functions $T(\cdot)$. We can obtain an estimate for
$F$-divergence by replacing the supremum over a restricted class of functions.
Specifically, if we use a parametric model $T_{\theta}$, (e.g., a neural
network) with parameters $\theta$, then taking the supremum over the
parameters $\theta$ yields a lower bound on $F$-divergence in (5) as stated
next:

$\displaystyle D_{f}(P\parallel Q)\geq\underset{\theta}{sup}\hskip
1.0ptE_{X\sim P}\left[T_{\theta}(X)\right]-E_{X\sim
Q}\left[f^{*}(T_{\theta}(X))\right].$ (6)

We use the above variational lower bound to estimate the $F$-divergence for
fair classification as described next. For enforcing fairness constraints
(DP/EO), we need to estimate $F$-divergence between joint distributions in
different groups, i.e., $D_{f}(\pi_{i}\parallel\pi_{j})~{}~{}\forall~{}i\neq
j,~{}~{}i,j\in\mathcal{G}$ where for Demographic parity (DP),
$\pi_{i}\sim\pi(\hat{Y}\left|Z=i\right.)$, and for Equalized Odds (EO),
$\pi_{i}\sim\pi(\hat{Y}\left|Z=i,Y=1\right)$. The variational lower bound on
$F$-divergence in (6) can then be estimated as:

$\displaystyle\underset{\theta}{\max}\frac{1}{M}\left(\sum_{m=1}^{M}T_{\theta}\left(x_{i}^{(m)}\right)-\sum_{m=1}^{M}f^{\ast}\left(T_{\theta}\left(x_{j}^{(m)})\right)\right)\right),$
(7)

where in (7), we have replaced the expectation operators with their empirical
estimates, and $\\{x_{i}^{(m)}\\}$ denote i.i.d. samples drawn from the
distribution $\pi_{i}$. Together with the above estimate, from OPT we arrive
at the following min-max optimization problem (denoted by MIN-MAX-OPT):

$\displaystyle\underset{w}{\min\;}\underset{\theta}{\max\;}\frac{1}{N}\sum_{n=1}^{N}\mathcal{L}_{w}(\hat{y}_{n};y_{n})~{}$
(8) $\displaystyle+\;\lambda\sum_{i\neq
j}\underbrace{\left(\frac{1}{M}\sum_{m=1}^{M}\left(T_{\theta}\left(x_{i}^{(m)}\right)-f^{\ast}\left(T_{\theta}\left(x_{j}^{(m)}\right)\right)\right)\right)}_{\triangleq\hat{D}^{(w,\theta)}_{f}(\pi_{i},\pi_{j})}$

MIN-MAX-OPT can be solved by alternatively training the classifier $w$ and
$F$-divergence estimator $\theta$. Specifically, we update the classifier
weight $w$ while fixing the $F$-divergence estimator parameter $\theta^{*}$,
and then we update $F$-divergence estimator parameter $\theta$ fixing the
classifier weight $w^{*}$. The $F$-divergence estimator updates each time as
the classifier changes. We show the training details in Algorithm 1.

Algorithm 1 $F$-divergence based Fair Training

1: Input: Sample $\\{x_{i}^{(1)},...,x_{i}^{(m)}\\}$ from distribution
$\pi_{i}$ and Sample $\\{x_{j}^{(1)},...,x_{j}^{(m)}\\}$ from distribution
$\pi_{j}$; base classifier $w_{t}$; $F$-divergence estimator $\hat{D}_{f}$.

2: Output: Fair classifier ($w^{*}$) and corresponding $F$-divergence
estimator ($\hat{D}_{f}^{*}$)

3: for training iterations $(t=1,2,\ldots,T_{1})$ do

4: Divergence estimation for a fixed classifier $w_{t}$: Update divergence
estimator $\theta$ for each pair ($i,j$) of groups for $T_{2}$ steps (or until
convergence) to maximize  $\hat{D}^{(w_{t},\theta)}_{f}(\pi_{i},\pi_{j})$

5: Classifier update: Update classifier $w_{t}$ to minimize the regularized
loss:
$\frac{1}{N}\sum_{n=1}^{N}\mathcal{L}_{w_{t}}(\hat{y}_{n};y_{n})+\lambda\sum_{i\neq
j}\hat{D}^{(w_{t},\theta_{t})}_{f}(\pi_{i},\pi_{j})$

6: end forFor demographic parity (DP):
$\pi_{i}\sim\pi(\hat{Y}\left|Z_{i}\right.)$; for equalized odds (EO):
$\pi_{i}\sim\pi(\hat{Y}\left|Z_{i},Y=1\right)$.

Figure 2: (a) Quantification of Fairness-Accuracy (FA) tradeoffs via notions
of FA-AUC and Low-Bias FA-AUC. The red curve (areas $A$ and $B$) represents
the convex hull of the achievable fairness-accuracy pairs, i.e.,
$(\epsilon,a)$ obtained by varying the fairness regularization parameter. The
low-bias FA-AUC measures the AUC when the bias is less than a prescribed
threshold, $\Delta_{DP}\leq\zeta$. We pick this low-bias threshold as the bias
in the classifier when it is optimized without any fairness constraint. (b)
Low-bias Threshold(s) w.r.t DP/EO and unconstrained test accuracy for four
real world datasets.

##  IV Evaluation Metrics, Results and Discussion

In this Section, we first discuss evaluation metrics (Section IV-A) to
evaluate the tradeoffs between fairness and predictive test accuracy.
Speficically, we propose the fairness-accuracy receiver operating
characteristic (FA-ROC) and a corresponding low-bias version of FA-ROC as a
quantitative measure of this tradeoff. We next present the details of our
experimental setup in Section IV-B, specifically, the datasets, details of
classifier architectures and $F$-divergence estimators, training
hyperparameters, as well as other existing approaches for learning fair
classifiers which we compare against the proposed framework. Finally, in
Section IV-C, we present and discuss the results and show the improved
performance of the proposed $F$-divergence against various other existing
approaches, including regularization, pre- and post-processing methods for
four real world datasets (COMPAS dataset [4] for recidivism prediction, Adult
Census dataset [20] for income level prediction, Law School admissions dataset
[21] for law students’ scores prediction, CelebA dataset [22] for facial image
classification) and one synthetic dataset (Moon dataset [23]), which are
widely used for the assessment of fairness in classification problems. The
comprehensive additional experiments are provided in the Appendix (including
multiple sensitive attributes, groupwise tradeoff(s) between fairness and
accuracy, and comparison of different F-divergence estimators).

###  IV-A Fairness-Accuracy Tradeoff & Evaluation Metrics

In addition to imposing approximate fairness constraints, one is
simultaneously interested in learning classifiers with high predictive
accuracy. To this end, we next define the notion of fairness-vs-accuracy
receiver operating characteristic, i.e., FA-ROC. For a classifier
$\pi(\hat{Y}|X)$, we denote it’s accuracy as
$\text{Acc}(\pi)=\text{Pr}(\hat{Y}=Y)$.

######  Definition 4.

A fairness-accuracy pair ($\epsilon,a$) w.r.t. demographic parity is
achievable if there exists a classifier $\pi(\cdot)$ with $\text{Acc}(\pi)\geq
a$ and $\Delta_{\text{DP}}(\pi)\leq\epsilon$. Similarly, one can define the
achievability of $(\epsilon,a)$ w.r.t. equalized odds.

We next state a Proposition which reveals an interesting property of the
achievable fairness-accuracy tuples.

######  Proposition 1.

Suppose that the fairness-accuracy tuples
$(\epsilon_{1},a_{1}),(\epsilon_{2},a_{2}),\ldots,(\epsilon_{k},a_{k})$ are
achievable. Then any convex combination
$(\sum_{i=1}^{k}\beta_{i}\epsilon_{i},\sum_{i=1}^{k}\beta_{i}a_{i})$, for
$\beta_{i}\in[0,1],\sum_{i}\beta_{i}=1$ of these pairs is also achievable.
Consequently, the convex hull of these pairs is also achievable.

###### Proof.

Suppose that the fairness-accuracy pairs $(\epsilon_{i},a_{i}),i=1,\ldots,k$
are achievable, i.e., there exists classifiers $\pi_{i},i=1,\ldots,k$ for
which $a_{i}=\pi_{i}(\hat{Y}=Y)$ and

$\displaystyle\epsilon_{i}=\Delta_{DP}(p_{i},q_{i})=|p_{i}-q_{i}|=|\pi_{i}(\hat{Y}|Z=0)-\pi_{i}(\hat{Y}|Z=1)|$

We need to show that any convex combination of these tuples, i.e.,
$(\epsilon,a)=(\sum_{i}\beta_{i}\epsilon_{i},\sum_{i}\beta_{i}a_{i})$ is
achievable. To this end, we can construct a new classifier as follows:
$\pi=\pi_{i}$ with prob. $\beta_{i}$, $i=1,2,\dots,k$, for
$\beta_{i}\in[0,1],\sum_{i}\beta_{i}=1$. We now prove that the above
classifier achieves the desired $(\epsilon,a)$. The accuracy of $\pi$ can be
lower bounded as

$\displaystyle\pi(\hat{Y}=Y)$
$\displaystyle=\sum_{i=1}^{k}\beta_{i}\pi_{i}(\hat{Y}=Y)\geq\sum_{i=1}^{k}\beta_{i}a_{i}\triangleq
a.$ (9)

We next show the bound on the fairness constraint for the classifier $\pi$,
where $\Delta_{DP}(p,q)$ can be upper bounded by:

$\displaystyle\left|\sum_{i}\beta_{i}p_{i}-\sum_{i}\beta_{i}q_{i}\right|\overset{(a)}{\leq}\sum_{i=1}^{k}\beta_{i}|p_{i}-q_{i}|\leq\sum_{i=1}^{k}\beta_{i}\epsilon_{i}\triangleq\epsilon$

where (a) follows from the fact that the norm $\Delta_{DP}(p,q)$ is convex in
the pair $(p,q)$ followed by Jensen’s inequality. This completes the proof of
Proposition 1. ∎

Datasets | Moon | COMPAS | Adult | Law School | CelebA
---|---|---|---|---|---
Learning Rate | 2e-3 / 2e-3 | 6e-4 / 6e-4 | 1e-2 / 1e-2 | 1e-4 / 1e-4 | 1e-3 / 1e-3
Batch Size | 2048 / 2048 | 2048 / 2048 | 2048 / 2048 | 2048 / 2048 | 256/256
Range of $\lambda$ (Regularization parameter) | 0-9 / 0-9 | 0-9 / 0-9 | 0-9 / 0-9 | 0-9 / 0-9 | 0-500 / 0-500
Number of Epochs | 200 / 200 | 200 / 200 | 200 / 200 | 200 / 200 | 10/10
Number of seeds | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 | 5/5
Optimizer | Adam / Adam | Adam / Adam | Adam / Adam | Adam / Adam | Adam / Adam
Original (unconstrained) Accuracy | 0.9728 / 0.9728 | 0.6802 / 0.6802 | 0.8427 / 0.8427 | 0.7641 / 0.7641 | 0.7983 / 0.7983
Original Bias (i.e., Low-bias Threshold) | 0.2706 / 0.0105 | 0.1257 / 0.1015 | 0.0874 / 0.0165 | 0.4380 / 0.2839 | 0.4429 / 0.2376
Threshold for classifier prediction | 0.5 / 0.5 | 0.5 / 0.5 | 0.5 / 0.5 | 0.5 / 0.5 | 0.5 / 0.5
Steps of F-divergence estimator (per classifier update) | 100 / 100 | 100 / 100 | 10 / 1 | 100 / 100 | 3/3

TABLE I: Hyperparameters for training process. Each entry represents the
hyperparameter w.r.t DP / EO.

The above Proposition uses the convexity property of $\Delta_{DP}(p,q)$. An
important consequence of this Proposition is the following: as the fairness
constraint (quantified by $\epsilon$) is varied, one achieves different
(fairness, accuracy) operating points. The above Proposition shows that the
convex hull of these pairs is also achievable (essentially, in our proof we
show that to achieve the convex combination of the original tuples, one can
construct a new classifier by combining these classifiers). We now define the
fairness-accuracy receiver operating characteristic (FA-ROC) with respect to
demographic parity (resp. equalized odds) as follows:

$\displaystyle\text{FA-ROC}_{\text{DP}}$
$\displaystyle=\\{(\epsilon,a):(\epsilon,a)\text{ is achievable w.r.t.
DP}\\}.$ $\displaystyle\text{FA-ROC}_{\text{EO}}$
$\displaystyle=\\{(\epsilon,a):(\epsilon,a)\text{ is achievable w.r.t.
EO}\\}.$

The performance of different regularization techniques can be compared by
calculating the area under FA-ROC curve (denoted by FA-AUC) as shown in Fig
2(a).

Low-bias FA-ROC– One shortcoming of the FA-ROC is that it will mask the
performance of the fair classifier when the original bias of a classifier is
small as we explain next. Suppose that we do not impose any fairness
constraint, and let $\zeta$ denote the bias of the resulting unconstrained
classifier. Then, the low-bias FA-ROC consists of all achievable
$(\epsilon,a)$ pairs such that $\epsilon\leq\zeta$. Intuitively, since $\zeta$
is the natural bias one would obtain when not imposing any fairness penalty,
the non-trivial portion of the trade-off is the one corresponding to
$\epsilon\leq\zeta$. Therefore, we introduce the notion of low-bias FA-ROC,
and argue that this is a more intuitive and justifiable measure of comparing
the performance of different regularization techniques. The low-bias region
and corresponding AUC are shown in Fig. 2(a). In Fig. 2(b), we also show the
low-bias threshold(s) ($\zeta$) as well as the unconstrained classification
accuracy for the four real-world datasets used in our experiments (COMPAS,
Adult Income, Law School admissions and CelebA datasets).

###  IV-B Experimental Setup

####  IV-B1 Datasets

We consider four real-world datasets and one synthetic dataset in our
experiments as described next: a) COMPAS Dataset: This dataset consists of
data from $N=7,214$ users ($N_{train}=5,049$, $N_{test}=2,165$), with 10
features (including age, prior criminal history, charge degree etc.) which are
used for predicting the risk of recidivism in the next two years. b) Adult
Census Dataset (Adult dataset): This dataset includes income related data with
14 features (i.e., age, work class, occupation, education etc.) of $N=45,222$
users ($N_{train}=32,561$, $N_{test}=12,661$) to predict whether the income of
a person exceeds a threshold (e.g., $50k) in a year. c) Law School Admissions
Dataset (Law School dataset): This dataset includes the admission related data
with 7 features (LSAT score, gender, undergraduate GPA etc.) of $N=4,862$
applicants ($N_{train}=3,403$, $N_{test}=1,459$) to predict the likelihood of
passing the bar. d) CelebA dataset: This high-dimensional image dataset
contains $N=202,599$ ($N_{train}=162,770$, $N_{validation}=19,867$,
$N_{test}=19,962$) face images of celebrities with 40 binary attributes, which
are cropped and resized to 64 $\times$ 64 pixels images. We use
attractive/not-attractive as the binary classification label and gender as the
sensitive attribute. The original CelebA contains training, validation and
testing data. We use the CelebA testing data to report test accuracy and
fairness measurements. e) Moon Dataset: This synthetic dataset contains
$N=15,000$ examples ($N_{train}=10,000$, $N_{test}=5,000$) with two features
and one sensitive attribute.

For all the above datasets, our goal will be to build a fair binary classifier
for two scenarios: a) $|Z|=2$, when the sensitive attribute is race
(COMPAS/Law School/Adult datasets), i.e., $Z\in\\{C,O\\}$, where
$C=\text{``Caucasian"}$ or $O=\text{``Other race"}$, corresponding to two
groups; or when the sensitive attribute is gender (CelebA dataset). We also
study another scenario, when $|Z|=4$, when the sensitive attribute(s) are both
race and gender (COMPAS/Law School/Adult datasets), i.e.,
$Z\in\\{(C,M),(C,F),(O,M),(O,F),(C,O),(M,F)\\}$. Experimental results on
multiple sensitive attributes are presented in the Appendix.

####  IV-B2 Other Methods on Learning Fair Classifiers

We compare and demonstrate the superiority of our proposed $F$-divergence
framework with several existing regularization methods, pre-processing, and
post-processing techniques as described next: (1) KDE based Regularization:
Cho et al.[16]: A kernel density estimation (KDE) based fair training, which
directly estimates the conditional distribution $p(\hat{Y}|Z)$ (or
$p(\hat{Y}|Z,Y)$) using KDE and uses it for fairness regularization. (2)
Correlation based Regularization: Mary et al. [18]: A correlation coefficient
based fair training, which use correlation coefficient
$\rho(\hat{Y},Z)~{}(or~{}\rho(\hat{Y},Z~{}|Y))$ as the regularization term
added in the loss function. (3) Covariance based Regularization: Zafar et al.
[3, 7]: For demographic parity (DP), [3] uses the covariance between the
sensitive attributes and the signed distance from the feature vectors to the
decision boundary. For equalized odds (EO), [7] uses the covariance between
sensitive attributes and the signed distance between the feature vectors of
misclassified data and the classifier decision boundary as the regularization
term. (4) FNR-FPR based Regularization: Yahav et al. [17], which proposes the
difference of FNR across different groups and the difference of FPR across
different groups as the regularization term. (5) Pre-processing Method (LFR):
Zemel et al. [2], which proposes a pre-processing method to learn a fair
representation of the data and then use the fair representation to train the
model. (6) Post-processing Method (ROC) : Hardt et al. [11], which proposes a
post-processing method to learn a fair predictor from a discriminatory binary
predictor by finding the optimal threshold between TPR and FPR for equalized
odds (EO) constraints. For all comparison methods, we use authors’ original
codes or codes for these papers adapted from AI Fairness
360444https://aif360.mybluemix.net/.

Figure 3: (a) FA-AUC w.r.t $\Delta_{DP}$ (top) / $\Delta_{EO}$ (bottom) and
Accuracy; (b) FA-AUC in the low-bias region w.r.t $\Delta_{DP}$ (top) /
$\Delta_{EO}$ (bottom) and Accuracy: We compare our best results in Table 5
with regularization based methods (correlation [18], KDE [16], covariance [3,
7], FNR-FPR [17]), the pre-processing method (LFR [2]) and post-processing
method (ROC [11]) method. We show the overall Area-under-the-curve (AUCs)
(mean AUC $\pm$ standard deviation) for different techniques under both DP and
EO fairness notions. Bracket All represents that all KL, SH, $\chi^{2}$
divergence can achieve the value. Bracket SH represents that only SH
divergence can achieve the value. Figure 4: (a) Trade-off between fairness and
accuracy (FA-ROC) on the Law School Admission dataset w.r.t DP constraints
(left) and corresponding low-bias region FA-ROC (right); (b) Trade-off between
fairness and accuracy (FA-ROC) on the COMPAS dataset w.r.t EO constraints
(left) and corresponding low-bias region FA-ROC (right). As shown in the
figure, for both DP and EO constraints, our method outperforms other
regularization based methods.

####  IV-B3 Model Architectures, Training Methodology and Hyperparameters

: For low/medium dimensional datasets, namely Moon, COMPAS, Adult and Law
School, we consider neural network classifiers with three fully connected
layers, where each hidden layer has $200$ nodes and is followed by an $SeLU$
[24] non-linear layer. According to the experimental results, three fully
connected layers are sufficient to obtain state-of-art accuracy; we report the
original accuracy (without fairness constraints) in Table I. We obtained the
best accuracy with SeLU activation (in comparison to ReLU, Sigmoid
activations). For the proposed F-divergence based fair training, the
F-divergence estimator is modeled using two-layer neural networks, each with
$5$ hidden nodes and a sigmoid non-linearity (see additional discussion on the
choice of the model architecture in the Appendix). For high dimensional image
classification dataset (CelebA), we use ResNet 18 [25] as the classifier for
predicting attractive/un-attractive. For the F-divergence estimator, we use
three linear layers, each with $10$ hidden nodes and a sigmoid non-linearity.
Note that this architecture is sufficient for CelebA dataset since we only
take outputs of the classifier and sensitive attributes as the input of the
F-divergence estimator.

For all regularization based training methods, we use the following training
loss function:
$\mathcal{L}_{\text{Error}}+\lambda\mathcal{R}_{\text{Fairness}}$, where
$\mathcal{L}_{\text{Error}}$ denotes the binary cross-entropy loss (for the
classification error) together with the fairness related regularization term
$\mathcal{R}_{\text{Fairness}}$. The trade-off parameter $\lambda$ is used to
adjust the proportion of classification loss and fairness constraints; varying
$\lambda$ give us the set of points reflecting the trade-off between fairness
and accuracy. To compare with pre / post - processing methods, we vary the
classification threshold(s) of the corresponding algorithms to form a FA-ROC.
By constructing the convex hull w.r.t these points, we form the FA-ROC as
shown in Fig 2. For each $\lambda$, we train over 5 runs with different random
seeds. To obtain the Low-bias FA-ROC, we set $\lambda=0$ (i.e., no fairness
constraint), and find the corresponding bias of the learned classifier. This
yields the low-bias threshold $\zeta$, and the low-bias FA-ROC is the convex
hull of all points for which the bias is no more than $\zeta$ ($\zeta$ w.r.t
DP / EO are shown in Table I). For all methods, the models were trained using
the Adam optimizer. For fair comparison, we use same training steps and the
same classification model for other mechanisms. We report the accuracy and
fair measurements on the test dataset. All the training hyperparameters are
summarized in Table I. For a fair comparison of different regularization
techniques (including the $F$-divergence techniques and prior works) pre- and
post-processing methodologies, we compare all of the techniques by measuring
fairness using the well accepted notion of TV distance ($\Delta_{DP}$ as
defined in (2) for DP or $\Delta_{EO}$ (3) for EO), which are the same metrics
as defined in previous works.

###  IV-C Comparison of $F$-divergence Regularization with other approaches
for learning fair classifiers

In Fig 3 (a), we report the overall FA-AUC for both DP and EO fairness notions
for the six other existing methods and the proposed $F$-divergence approach.
The numbers in the table are the mean AUC $\pm$ standard deviation over $5$
independent trials. We notice that our proposed method achieves better trade-
off than other compared methods for all the real world datasets. We also
report the low-bias FA-AUC for all the datasets and the different
regularization techniques in Fig 3 (b). In contrast to the overall FA-AUC
(where the performance improvement given by $F$-divergence is modest), when we
zoom in the low-bias region, we see a significant performance improvement in
the fairness-accuracy tradeoffs. As an example, for the Adult dataset with the
DP fairness objective, our proposed method achieves 12.89% higher low-bias FA-
AUC than other compared state-of-art mechanisms (the gain for EO fairness
objective is 10.14%). Our method also performs well on high dimensional
dataset (CelebA) where we achieve a gain of 2.67% / 5.67% w.r.t the EO
constraint for FA-AUC / Low-bias FA-AUC. We show the plots of FA-ROC in Fig 4
on Law School and COMPAS datasets (similar figures for other datasets are
provided in the Appendix). Another interesting observation is that the optimal
choice of $F$-divergence regularization is dependent on both the dataset as
well as the notion of fairness. For instance, Pearson $\chi^{2}$ divergence
regularization yields the best trade-off than other $F$-divergence based
methods w.r.t DP on all real world datasets. For the equalized odds (EO)
notion, SH divergence regularization shows better performance on both COMPAS
and Law School datasets while $\chi^{2}$ divergence achieves better results on
Adult dataset.

##  V Concluding Remarks

In this work, we introduced a general min-max $F$-divergence regularization
framework for fair classifiers, which is readily adaptable for high-
dimensional problems and compatible with gradient based optimization methods.
In contrast to existing regularization methods such as
correlation/covariance/TPR/FPR based methods, $F$-divergence notions for
quantifying fairness/bias are stronger notions to capture dependence between
classifier decisions and sensitive attributes and provide stronger fairness
guarantees.

We also proposed the notion of Fairness-Accuracy ROC (FA-ROC) and a
corresponding low-bias FA-ROC, which we argue as the correct approach to
compare different mechanisms for learning fair classifiers and quantifing the
tradeoff between fairness and accuracy. Through an extensive set of
experiments on four real-world datasets (COMPAS, Adult Income, Law School
Admissions and CelebA), we demonstrated the improvement in the fairness-
accuracy tradeoffs compared to prior works on regularization as well as pre-
and post-processing methods.

## References

  * [1] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  ACM Computing Surveys (CSUR), 54(6):1–35, 2021.
  * [2] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork.  Learning fair representations.  In International conference on machine learning, pages 325–333. PMLR, 2013.
  * [3] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi.  Fairness constraints: Mechanisms for fair classification.  In Artificial Intelligence and Statistics, pages 962–970. PMLR, 2017.
  * [4] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner.  Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks.  ProPublica, May 2016.
  * [5] Anja Lambrecht and Catherine Tucker.  Algorithmic bias? an empirical study of apparent gender-based discrimination in the display of stem career ads.  Management science, 65(7):2966–2981, 2019.
  * [6] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel.  Fairness through awareness.  In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226, 2012.
  * [7] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi.  Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment.  In Proceedings of the 26th international conference on world wide web, pages 1171–1180, 2017.
  * [8] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva.  Counterfactual fairness.  In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [9] Solon Barocas and Andrew D Selbst.  Big data’s disparate impact.  Calif. L. Rev., 104:671, 2016.
  * [10] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian.  Certifying and removing disparate impact.  In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 259–268, 2015.
  * [11] Moritz Hardt, Eric Price, and Nati Srebro.  Equality of opportunity in supervised learning.  Advances in neural information processing systems, 29, 2016.
  * [12] Mikhail Yurochkin, Amanda Bower, and Yuekai Sun.  Training individually fair ml models with sensitive subspace robustness.  In International Conference on Learning Representations, 2020.
  * [13] Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf.  Avoiding discrimination through causal reasoning.  Advances in neural information processing systems, 30, 2017.
  * [14] Jaewoong Cho, Gyeongjo Hwang, and Changho Suh.  A fair classifier using mutual information.  In 2020 IEEE International Symposium on Information Theory (ISIT), pages 2521–2526. IEEE, 2020.
  * [15] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger.  On fairness and calibration.  Advances in neural information processing systems, 30, 2017.
  * [16] Jaewoong Cho, Gyeongjo Hwang, and Changho Suh.  A fair classifier using kernel density estimation.  Advances in Neural Information Processing Systems, 33:15088–15099, 2020.
  * [17] Yahav Bechavod and Katrina Ligett.  Learning fair classifiers: A regularization-inspired approach.  Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML), 2017.
  * [18] Jérémie Mary, Clément Calauzenes, and Noureddine El Karoui.  Fairness-aware learning for continuous attributes and treatments.  In International Conference on Machine Learning, pages 4382–4391. PMLR, 2019.
  * [19] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan.  Estimating divergence functionals and the likelihood ratio by convex risk minimization.  IEEE Transactions on Information Theory, 56(11):5847–5861, 2010\.
  * [20] Dheeru Dua and Casey Graff.  Adult data set.  UCI Machine Learning Repository, 2017.
  * [21] Linda F. Wightman.  LSAC National Longitudinal Bar Passage Study. LSAC research report series.  1998\.
  * [22] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [23] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al.  Scikit-learn: Machine learning in python.  the Journal of machine Learning research, 12:2825–2830, 2011.
  * [24] Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter.  Self-normalizing neural networks.  Advances in neural information processing systems, 30, 2017.
  * [25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [26] Show-Jane Yen and Yue-Shi Lee.  Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset.  In Intelligent Control and Automation: International Conference on Intelligent Computing, ICIC 2006 Kunming, China, August 16–19, 2006, pages 731–740. Springer, 2006.
  * [27] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori.  Density ratio estimation in machine learning.  Cambridge University Press, 2012.

##  Appendix A Implementation details

###  A-A Additional Description of Datasets and Pre-processing

In this Section, we present additional details regarding the synthetic (Moon)
dataset, as well as some pre-processing steps we followed for the Law
Admissions dataset.

Synthetic Moon Dataset: For the synthetic Moon dataset, we use scikit-learn
[23] – $make\\_moons$ function to generate two interleaving half circles with
$P(Y=1|Z=1)\approx 0.35$ and $P(Y=1|Z=0)\approx 0.65$ while $P(Y=1)\approx
0.5$, which follows the same procedure in KDE based mechanism [16].

Law Admissions Dataset: The law school dataset contains $21,791$ numbers of
tabular data from law students, where $19,360$ data have positive label and
$2,431$ users contain negative label, which is an extremely unbalanced
dataset. For this kind of extremely imbalanced dataset, a classifier will
attend to predict more positive label leading to a skewed classification [26].
To overcome this situation, we balance the dataset by under-sampling the
majority group w.r.t values of label (Y = 0 / Y = 1). Specifically, we
randomly sample $2,431$ data points from majority group (positive label) and
combine with the minority group to form a balanced law school admission
dataset.

###  A-B Choice of activation functions and Architecture for F-divergence
estimator

Our binary classifier architecture consists of three fully connected layers,
where each hidden layer has $200$ nodes and is followed by a Scaled
exponential Linear Unit (SeLU) non linearity, defined as
$\text{SeLU}(x)=\text{scale}\times\max(0,x)+\min(0,\alpha\times(e^{x}-1)$,
where $\text{scale}=1.0507$ and $\alpha=1.6732$. The last layer is a sigmoid
non-linear layer which outputs the probabilities for the two classes. We
compare other activation methods with SeLU activation function on COMPAS
dataset without fairness constraints as shown in Table II. We find that SeLU
activation function yields higher accuracy ($0.68$) while preserving fairness
constraints (bias $\approx 0.12$).

| Unconstrained Accuracy | $\Delta_{DP}$
---|---|---
ReLU | 0.6660 $\pm$ 0.0041 | 0.0991 $\pm$ 0.0038
Sigmoid | 0.6769 $\pm$ 0.0030 | 0.1302 $\pm$ 0.0053
SeLU | 0.6802 $\pm$ 0.0056 | 0.1257 $\pm$ 0.0083
TABLE II: Comparison of activation functions used for the classifier on the COMPAS dataset. We compare the SeLU activation function with ReLU and Sigmoid activation function. We can observe that SeLU achieves the best trade-off than other two activation functions. | Constrained Accuracy | $\Delta_{DP}$
---|---|---
ReLU | 0.6666 $\pm$ 0.0370 | 0.1077 $\pm$ 0.0426
SeLU | 0.6008 $\pm$ 0.0119 | 0.0238 $\pm$ 0.0164
Sigmoid | 0.6056 $\pm$ 0.0111 | 0.0176 $\pm$ 0.0245
TABLE III: Comparison of activation functions for the $F$-divergence estimator
on the COMPAS dataset, where we enforce largest fairness constraints
($\lambda=9$) to the classifier during the training process and output
corresponding test accuracy and $\Delta_{DP}$. We compare the Sigmoid
activation function with ReLU and SeLU activation function. We can observe
that Sigmoid gives the best trade-off than other two activation functions.

We next present the details regarding the architecture of the $F$-divergence
estimator. We train the NN based $F$-divergence estimator with two hidden
layers each followed by a sigmoid non-linearity. Similar as above, in Table
III, we compare sigmoid function with SeLU and ReLU activation functions on
COMPAS dataset while imposing the largest fairness constraints ($\lambda=9$)
with respect to demographic parity. We find that sigmoid function gives the
best trade-off between fairness and accuracy.

###  A-C Other Approaches for Estimating $F$-divergence

In this section, we discuss other possible alternatives to estimate
$F$-divergence. We also perform a comprehensive study on the choice of
$F$-divergence estimator in our experiments.

a) Conventional representation of $F$-divergence (CON) Another alternative is
to directly estimate the probability distributions $\hat{\pi}_{i}$ and
$\hat{\pi}_{j}$, and plug them to get an estimate of $F$-divergence term as
follows:

$\displaystyle
D_{f}(\pi_{i}\parallel\pi_{j})\approx\sum_{u\in\\{0,1\\}}\hat{\pi}_{j}(u)\times
f\left(\frac{\hat{\pi}_{i}(u)}{\hat{\pi}_{j}(u)}\right)$ (10)

which can be used as the fairness regularization term.

b) Density ratio based estimation (DRE) of $F$-divergence Another approach is
to estimate the density ratio (in our case, a probability ratio)
$r(u)=\pi_{i}(u)/\pi_{j}(u)$ from the i.i.d. samples from the groups $i,j$
borrowing techniques from density ratio estimation (DRE) [27]. With an
estimate $\hat{r}(u)$ for the probability ratio, one can then estimate
$F$-divergence as follows:

$\displaystyle
D_{f}(\pi_{i}\parallel\pi_{j})=E_{X\sim\pi_{j}}f(r(X))\approx\frac{1}{M}\sum_{m=1}^{M}f(\hat{r}(x_{j}^{(m)}))$
(11)

where $\\{x_{j}^{(m)}\\}$ denote i.i.d. samples drawn from the distribution
$\pi_{j}$. In Section IV-C, we perform a comprehensive comparison of various
methods for estimating $F$-divergence and their associated impact on the
tradeoffs between accuracy and fairness.

##  Appendix B ADDITIONAL EXPERIMENTAL RESULTS

In this Section, we first provide additional experimental results (FA-ROC and
Low-bias FA-ROC) about the tradeoff between accuracy and fairness on Moon /
Adult / CelebA datasets. Next, we compare our proposed method with other
regularization based methods (FNR-FPR [17], Correlation [18] and Covariance
[3, 7]) w.r.t the groupwise AUC for DP and EO constraints.

Figure 5: Evaluation of the $F$-divergence based framework (via FA-AUC) w.r.t
various $F$-divergence Estimators (NN, CON, DRE) and different $F$-divergences
(KL, SH, $\chi^{2}$) for the DP (left) / EO (right) constraint. We can observe
that NN based $F$-divergence estimator consistently leads to higher area under
the FA-ROC curve. Figure 6: Trade-off between fairness-accuracy (FA-ROC) for
DP and EO on the Moon dataset (a) / Adult dataset (b) / CelebA dataset (c). We
also show the low-bias region (Low-bias FA-ROC) in these figures. For both DP
and EO constraints, we observe that our proposed method consistently
outperforms other regularization based methods (Correlation [18], KDE [16],
Covariance [3, 7], FNR-FPR [17]), the pre-processing method (LFR [2]) and
post-processing method (ROC [11]).

###  B-A Evaluation of different $F$-divergence Estimators

To validate the $F$-divergence regularization framework, we first conduct
experiments comparing different $F$-divergence estimation techniques.
Specifically, we test the three methods discussed in Section III (variational
representation (NN), convention representation (CON) and DRE method (DRE)) for
three types of $F$-divergence measures: KL divergence (KL), Squared Hellinger
distance (SH) and Pearson $\chi^{2}$ divergence ($\chi^{2}$). In Fig 5, we
show the performance of trade-off between fairness and accuracy w.r.t.
different $F$-divergence measures under different estimation techniques for
the five datasets. We first observe that the $F$-divergence estimator leads to
consistent results for all five datasets. We also observe that NN and CON
methods for $F$-divergence estimation can be slightly better than the DRE
method. Therefore, we will primarily use NN based $F$-divergence estimator in
the following experiments unless specified otherwise. Our second interesting
observation is that the optimal choice of the $F$-divergence measures (KL vs.
SH vs. $\chi^{2}$) is dependent on the dataset. For instance, for demographic
parity, $\chi^{2}$-divergence regularization outperforms other two
$F$-divergences on the Law School dataset.

###  B-B Fairness-accuracy tradeoffs on Moon, Adult and CelebA datasets

In this Section, we show the plots for Moon, Adult and CelebA datasets, which
are shown in Fig. 6. For Moon dataset, our proposed method with KL divergence
is close to the state-of-the-art compared methods w.r.t DP fairness
objectives. For EO, three of our proposed technique has better trade-off
between fairness and accuracy than others. For the Adult dataset, we have a
similar observation that our proposed methods outperform other compared
mechanisms (the gain for DP is 12.89%; the gain for EO is 10.14%). We can
observe similar trends on CelebA dataset, where our proposed method
consistently achieves better results than others (achieves a gain of 2.67% /
5.67% w.r.t the EO constraint for FA-AUC / Low-bias FA-AUC . ).

Figure 7: Groupwise ROC w.r.t. TPR and FPR on the COMPAS dataset for DP
constraints (left) and for EO constraints (right). For COMPAS dataset, the
majority group is ”Caucasian” and the minority group corresponds to ”Other
Race”.

Figure 8: Comparison of groupwise AUC w.r.t TPR and FPR with other
regularization based methods (FNR-FPR [17], Correlation [18] and Covariance
[3, 7] based method ) for each population group w.r.t DP (a) / EO (b). As
shown above, $F$-divergence framework has mild reductions on each group while
ensuring fairness constraints for all three real world datasets.

.

###  B-C Impact of Fairness Constraints on Groupwise TPR/FPR

We now study the impact of fairness constraints on the groupwise ROC and AUC.
Specifically, for each population sub-group (e.g., for COMPAS/Adult/Law-School
datasets, the population was divided into ”C” (Caucasian) and ”O” (other
race)), we calculate the true positive rate (TPR) and the false positive rate
(FPR) of the fair classifier and show the resulting ROC in Fig 7. To explore
the impact of fairness constraints, we compare the ROC of a classifier without
fairness constraints ($\lambda=0$) to the classifier trained with largest
fairness constraint (i.e., highest regularization penalty, e.g., $\lambda=9$
for COMPAS/Adult/Law-School datasets) in our experiments. We then calculate
the area under ROC to get the quantitative results, which are shown in Table
IV. We observe that for both minority and majority population groups, the
$F$-divergence framework only leads to a mild reduction in the AUC with
fairness constraints, when compared with the AUC without fairness constraints.

| Majority Group | Minority Group | Majority Group | Minority Group
---|---|---|---|---
| w/o DP | w/ DP | w/o DP | w/ DP | w/o EO | w/ EO | w/o EO | w/ EO
COMPAS | 0.72 | 0.67 | 0.71 | 0.65 | 0.72 | 0.66 | 0.71 | 0.66
Adult | 0.90 | 0.88 | 0.88 | 0.85 | 0.90 | 0.86 | 0.88 | 0.85
Law School | 0.81 | 0.80 | 0.81 | 0.80 | 0.81 | 0.76 | 0.81 | 0.75

TABLE IV: Groupwise AUC w.r.t DP (left) / EO (right) for each population group
on real world datasets with (or without fairness constraints). As shown in the
Table, groupwise AUC perserves high utility while adding fair constraints (DP
or EO).

Next, We study the impact of our proposed methods on the groupwise ROC and AUC
compared with other mechanisms (FNR-FPR [17], Correlation [18] and Corvariance
[3, 7] based method). For each population group (C or O), we construct the ROC
with respect to TPR and FPR of the classifier (with or without fairness
constraints) and then calculate the AUC in Fig 8. $F$-divergence based
framework consistently leads to a slightly reduction in the AUC with fairness
objectives compared with the classifier without fairness constraints. As an
example, for Law School dataset, the reduction of enforcing DP fairness
constraints for each population group is only 0.01. Similar as Law school
dataset, the maximum reduction among each group in Adult dataset is 0.03 for
DP and 0.04 for EO.

FA-AUC
---
| COMPAS | Adult | Law School
Correlation [18] | 0.59 $\pm$ 0.04 | 0.83 $\pm$ 0.03 | 0.63 $\pm$ 0.03
KDE[16] | 0.67 $\pm$ 0.00 | 0.81 $\pm$ 0.02 | 0.72 $\pm$ 0.01
Covariance[3] | 0.67 $\pm$ 0.01 | 0.83 $\pm$ 0.01 | 0.72 $\pm$ 0.01
FNR-FPR [17] | 0.67 $\pm$ 0.00 | 0.80 $\pm$ 0.02 | 0.72 $\pm$ 0.03
Proposed method ($\chi^{2}$) | 0.67 $\pm$ 0.00 | 0.84 $\pm$ 0.01 | 0.73 $\pm$ 0.01
Low-bias FA-AUC
| COMPAS | Adult | Law School
Correlation [18] | 0.5961 $\pm$ 0.0299 | 0.6688 $\pm$ 0.0349 | 0.6182 $\pm$ 0.0396
KDE[16] | 0.6120 $\pm$ 0.0449 | 0.7652 $\pm$ 0.0380 | 0.6730 $\pm$ 0.0168
Covariance[3] | 0.6192 $\pm$ 0.0410 | 0.8092 $\pm$ 0.0759 | 0.6863 $\pm$ 0.0374
FNR-FPR [17] | 0.5974 $\pm$ 0.0073 | 0.7071 $\pm$ 0.0048 | 0.6863 $\pm$ 0.0192
Proposed method ($\chi^{2}$) | 0.6328 $\pm$ 0.0086 | 0.8067 $\pm$ 0.0226 | 0.7000 $\pm$ 0.0170

TABLE V: FA-AUC and Low-bias FA-AUC w.r.t multiple sensitive attributes
(gender and race). We can observe that $F$-divergence regularization
consistently yields the better AUC in most scenarios.

###  B-D Results for Multiple Sensitive Attributes.

We next present results for the case when more than one attributes can be
sensitive (specifically, both race and gender) for COMPAS, Adult and Law
School datasets, which leads to four population groups. According to Eq (4),
we calculate the sum of $\Delta_{DP}$ on each pair of groups and then add them
in the classification loss function. For proposed method, we use $\chi^{2}$
divergence based NN model for COMPAS and Adult datasets. For Law School
dataset, we apply $\chi^{2}$ divergence based regularization which is
estimated using the conventional (CON) method. As shown in Table V, our
proposed method achieves better performance than other existing regularization
techniques, i.e., for the COMPAS dataset with DP fairness constraints, our
proposed method achieves 2.20% higher low-bias FA-AUC than other compared
state-of-art techniques (the gain for Law School dataset is 2.00%).