# The Fairness Field Guide: Perspectives from Social and Formal Sciences

Alycia N. Carey, Xintao Wu
Department of Computer Science and Computer Engineering
University of Arkansas
Fayetteville, Arkansas
{ancarey<EMAIL_ADDRESS>

###### Abstract

Over the past several years, a slew of different methods to measure the
fairness of a machine learning model have been proposed. However, despite the
growing number of publications and implementations, there is still a critical
lack of literature that explains the interplay of fair machine learning with
the social sciences of philosophy, sociology, and law. We hope to remedy this
issue by accumulating and expounding upon the thoughts and discussions of fair
machine learning produced by both social and formal (specifically machine
learning and statistics) sciences in this field guide. Specifically, in
addition to giving the mathematical and algorithmic backgrounds of several
popular statistical and causal-based fair machine learning methods, we explain
the underlying philosophical and legal thoughts that support them. Further, we
explore several criticisms of the current approaches to fair machine learning
from sociological and philosophical viewpoints. It is our hope that this field
guide will help fair machine learning practitioners better understand how
their algorithms align with important humanistic values (such as fairness) and
how we can, as a field, design methods and metrics to better serve oppressed
and marginalized populaces.

_K_ eywords fair machine learning, philosophy, sociology, law, fair
classification, fair causal models

##  1 Introduction

In 2019, Obermeyer et al. published a disconcerting article that illustrated
how a widely used health risk-prediction tool, that is applied to roughly 200
million individuals in the U.S. per year, exhibited significant racial bias
[1]. They found that this bias was due to the algorithm equating the predicted
health-care cost of a patient with how sick they were likely to be. But, due
to systemic racism issues, such as distrust of the health-care system and
direct racial discrimination, Black individuals usually have more barriers to
receiving proper health care. Therefore, they generally spend less on health-
care costs per year [2]. Obermeyer et al. found that only 17.7% of patients
that the algorithm assigned to receive extra care were Black, but if the bias
in the system was corrected for, the percentage would increase drastically to
46.5% [3].

Numerous examples like the one above exist which depict machine learning
algorithms as being biased to a particular marginalization class such as race
or gender. For instance, women are less likely to be shown ads for high paid
positions like CEO or CFO [4], and facial recognition systems are more likely
to be wrong when given a picture of a Black woman [5]. Finding deficiencies
such as these has spurred research into creating machine learning models that
achieve fair outcomes.

Defining, implementing, and enforcing fairness in machine learning is, above
all else, a sociotechnical111The field of Science and Technology Studies (STS)
describes systems that consist of a combination of technical and social
components as “sociotechnical systems” [6]. challenge. Machine learning
systems may behave unfairly, or in a biased manner, for a multitude of
reasons: e.g., due to social biases reflected in the datasets used in
training, because of societal biases that are reflected (explicitly or
implicitly) in the design process, and/or as a result of interaction with
particular stakeholders or clients during run-time [7].

Without viewing fairness metrics from lenses of philosophy, sociology, and
law, choosing and implementing a metric stays firmly technical, and does not
consider societal impacts that could arise after deployment. In this instance,
we are blindly hoping to align to our wanted societal principles. To solve
this problem, and to help practitioners choose correct fairness metrics in an
informed, societal-aware, manner, we develop the following field guide that
depicts popular binary statistical- and causal-based fairness metrics through
lenses of philosophy, sociology, and the law. We note that this field guide
does not cover bias mitigation strategies and we refer interested readers to
[8, 9, 10] for more in depth discussions on this topic.

The rest of the field guide is as follows. We begin by explaining commonly
(mis)used definitions in fair machine learning literature and introduce
mathematical notions that will be used throughout the rest of the field guide
in Section 2. In Section 3, we introduce important philosophical perspectives,
such as Rawl’s Equality of Opportunity (EOP), that serve as a foundation for
many of the proposed fairness metrics. Next, in Section 4, we depict popular
legal ideals that have a strong connection to fairness in machine learning.
Sections 5 and 6 contain our analysis and discussion on popular statistical-
and causal-based fairness metrics. In Section 7, we give several critiques
from philosophy and sociology of the fair machine learning field as a whole.
Finally, in Sections 8 and 9 we detail closely related work and our major
conclusions, respectively.

##  2 Background

Being a sociotechnical issue, the terminology used in fair machine learning
literature can take on different meanings depending on the context in which
the word is being used. This can potentially lead to confusion when selecting
the right fairness metric to deploy in specific situations. For this reason,
in Table 1 we concretely define the meaning of commonly used terms in fair
machine learning publications in lenses of both philosophy and computer
science. Additionally, in this section we provide definitions for important
statistical terms as well as three overarching statistical categories that are
commonly used to classify machine learning fairness metrics. Further, we lay
the groundwork for the discussion on causal models in Section 6 by stating
several important frameworks and definitions.

###  2.1 Statistical Definitions

Most proposed machine learning fairness metrics have groundings in statistical
measures [11]. For example, statistical parity (otherwise known as group
fairness) depends on the measurement of raw positive classification rates;
equalized odds depends on false positive and false negative rates; and
predictive parity depends on true positive rates. The use of statistical
measures is attractive because they are relatively simple to measure and
definitions built using statistical measures can usually be achieved without
having to make any assumptions on the underlying data distributions. Many of
the common statistical measures used in machine learning fairness metrics are
listed in Table 2.

It is important to note, however, that statistical groundings do not provide
for individual level fairness, or even sub-group fairness for a marginalized
class [12]. Instead, they provide meaningful guarantees to the “average"
member of a marginalized group. Additionally, many statistical measures
directly oppose one another. For instance, it is impossible to satisfy both
false positive rates, false negative rates, and positive predictive value
across marginalized groups. This creates the direct consequence that many
definitions of machine learning fairness cannot be satisfied at the same time.

This fact was firmly cemented in the work completed by Barocas, Hardt, and
Narayanan [13]. In this work, they propose three representative fairness
criteria – independence, separation, and sufficiency – that relate to many of
the fair machine learning metrics that have been published. They capitalize on
the fact that most proposed fairness criteria are properties of the joint
distribution of a marginalization222What we call the marginalization attribute
(or marginalized class) is often called the sensitive or protected
attribute/class in fair machine learning literature. See Section 7.2 for why
we choose to use marginalized instead of protected or sensitive. attribute $S$
(e.g., race or gender), a target variable $Y$, and the classification (or in
some cases probability score) $\hat{Y}$, which allowed them to create three
distinct categories by forming conditional independence statements between the
three random variables.

####  2.1.1 Independence

The first formal category, independence, only requires that the
marginalization attribute, $S$ ($S=0$ non-marginalized, $S=1$ marginalized),
is statistically independent of the classification outcome $\hat{Y}$,
$\hat{Y}\perp S$. For the binary classification case, they note two different
formulations:

Exact: | $P[\hat{Y}=1\;|\;S=0]=P[\hat{Y}=1\;|\;S=1]$
---|---
Relaxed: | $\frac{P[\hat{Y}=1\;|\;S=0]}{P[\hat{Y}=1\;|\;S=1]}\geq 1-\epsilon$
Table 1: Definitions of popular terms from the view point of philosophy and
computer science.

Term | Philosophy | Computer Science
---|---|---
Bias |  | Prejudice in favor of or against one thing,
---
person, or group compared with another,
usually in a way considered to be unfair.
| Occurs when an algorithm produces results
---
that are systemically prejudiced due to erroneous
assumptions in the machine learning process. Normally
due to data collection, sampling, and/or measurement
procedures.
Bias vector: an additional set of weights in a neural
network that require no input, giving importance to
some of the features in order to generalize better.
Oppression |  | A social system of barriers that operate institutionally
---
and interpersonally to disempower people because
of their gender, race, class, sexuality, ethnicity, religion
body size, ability, and/or nationality.
-
Ethics |  | Involves systematizing, defending, and
---
recommending concepts of right and wrong
behavior.
| Area of artificial intelligence research concerned
---
with adding or ensuring moral behaviors of man-made
machines that use artificial intelligence.
Justice |  | Adherence to the standards agreed upon
---
in society (based on laws).
-
Fairness |  | A subjective principle of judgement of
---
whether a decision is morally right or wrong.
| Recently established area of machine learning that
---
studies how to ensure that biases in the data, and
model inaccuracies, do not lead to models that treat
individuals unfavorably on the basis of sensitive
characteristics.
Equality |  | Signifies correspondence between a group
---
of different objects, persons, processes or
circumstances that have the same qualities
in at least one respect, but not all.
-
Discrimination |  | Unfavorable treatment of people due to the
---
membership in certain demographic groups
that are distinguished by attributes
(supposedly) protected by law.
| A source for unfairness in machine learning due
---
to (intentional or unintentional) human prejudice
and stereotyping based on the sensitive attributes.
GAN Discriminator: tries to distinguish real data
from the data created by the generation.
| Marginalization
---
(Sensitive /Protected)
Characteristic
| Those characteristics commonly referenced and reflected in non-
discrimination law. E.g., race, ethnicity,
---
gender, religion, age, disability, sexual orientation, etc.

When considering the event $\hat{Y}=1$ to be the positive outcome, this
condition requires the acceptance rates to be the same across all groups. The
relaxed version notes that the ratio between the acceptance rates of different
groups needs to be greater than a threshold that is determined by a predefined
slack term $\epsilon$. In many cases $\epsilon=.2$ in order to align with the
four-fifths rule in disparate impact law (see Section 4.1).

Barocas, Hardt, and Narayanan also note that while independence aligns well
with how humans reason about fairness, several draw-backs exists for fairness
metrics that fall into this category (e.g., group fairness, disparate impact,
conditional statistical parity, and overall accuracy equality) [13]. Namely,
that they ignore any correlation between the sensitive attributes and the
target variable $Y$ which constrains the construction of a perfect prediction
model. Additionally, it enables laziness. In other words, it allows situations
where qualified people are carefully selected for one group (e.g., non-
marginalized), while random people are selected for the other (marginalized).
Further, it allows the trade of false negatives for false positives, meaning
that neither of these rates are considered more important, which is false in
many circumstances [14].

Table 2: Definitions for common statistical measures. $Y$ = actual, $\hat{Y}$
= predicted, 1 = positive, 0 = negative.

Measure | Abrriv. | Formula | Definition
---|---|---|---
True Positive | TP | $Y=1\cap\hat{Y}=1$ |  | A case when the predicted and actual outcome
---
are both in the positive class.
True Negative | TN | $Y=0\cap\hat{Y}=0$ |  | A case when the predicted and actual outcome
---
are both in the negative class.
False Positive | FP | $Y=0\cap\hat{Y}=1$ |  | A case predicted to be in the positive class when
---
the actual outcome belongs to the negative class.
False Negative | FN | $Y=1\cap\hat{Y}=0$ |  | A case predicted to be in the negative class when
---
the actual outcome belongs to the positive class.
| Positive Predictive Value
---
(Precision)
PPV | $\frac{TP}{TP+FP}$ |  | Fraction of positive cases correctly predicted to be
---
in the positive class out of all predicted positive cases.
False Discovery Rate | FDR | $\frac{FP}{TP+FP}$ |  | Fraction of negative cases incorrectly predicted to be
---
in the positive class out of all predicted positive cases.
False Omission Rate | FOR | $\frac{FN}{TN+FN}$ |  | Fraction of positive cases incorrectly predicted to be
---
in the negative class out of all predicted negative cases.
| Negative Predictive
---
Value
NPV | $\frac{TN}{TN+FN}$ |  | Fraction of negative cases correctly predicted to be
---
in the negative class out of all negative cases.
| True Positive Rate
---
(Sensitivity/ Recall)
TPR | $\frac{TP}{TP+FN}$ |  | Fraction of positive cases correctly predicted to be
---
in the positive class out of all actual positive cases.
False Positive Rate | FPR | $\frac{FP}{TN+FP}$ |  | Fraction of negative cases incorrectly predicted to be
---
in the positive class out of all actual negative cases.
False Negative Rate | FNR | $\frac{FN}{TP+FN}$ |  | Fraction of positive cases incorrectly predicted to be
---
in the negative class out of all actual positive cases.
True Negative Rate | TNR | $\frac{TN}{TN+FP}$ |  | Fraction of negative cases correctly predicted to be
---
in the negative class out of all actual negative cases.

####  2.1.2 Separation

The second category Barocas, Hardt, and Narayanan propose is separation, which
captures the idea that in many scenarios, the sensitive characteristic may be
correlated with the target variable [13]. Specifically, the random variables
satisfy separation if $\hat{Y}\perp S\;|\;Y$, or in other words, if $\hat{Y}$
is conditionally independent of $S$ when given $Y$. In the binary
classification case, it is equivalent to requiring that all groups achieve the
same true and false positive rates.

$TP\;:\;P[\hat{Y}=1\;|\;Y=1\cap S=0]=P[\hat{Y}=1\;|\;Y=1\cap S=1]$
$FP\;:\;P[\hat{Y}=1\;|\;Y=0\cap S=0]=P[\hat{Y}=1\;|\;Y=0\cap S=1]$

Additionally, this requirement can be relaxed to only require the same true
positive rates or the same false positive rates. Fairness metrics that fall
under both the equivalent and relaxed version of separation include: false
positive error rate balance, false negative error rate balance, equalized
odds, treatment equality, balance for the positive class, and balance for the
negative class.

####  2.1.3 Sufficiency

The final category, sufficiency, makes use of the idea that for the purpose of
predicting $Y$, the value of $S$ doesn’t need to be used when you are given
$\hat{Y}$ [14]. In other words, the classification $\hat{Y}$ already has made
use of $S$ to predict the target $Y$. Specifically, the random variables
satisfy sufficiency if $Y\perp S\;|\;\hat{Y}$. I.e., $Y$ is conditionally
independent of $S$ given $\hat{Y}$. In the binary classification case, this is
the same as requiring a parity of positive or negative predictive values
across all groups $\hat{y}\in\hat{Y}=\\{0,1\\}$:

$P[Y=1\;|\;\hat{Y}=\hat{y}\cap S=0]=P[Y=1\;|\;\hat{Y}=\hat{y}\cap S=1]$

They note that it is common to assume that $\hat{Y}$ satisfies sufficiency if
the marginalization attribute $S$ and the target variable $Y$ are clearly
understood from the problem context. Some examples of fairness metrics that
satisfy sufficiency include: predictive parity, conditional use accuracy, test
fairness, and well calibration.

####  2.1.4 Impossibility Results of Statistics-Based Metrics

Although each of the statistics-based fairness metrics we introduce in Section
5 formalize an intuitive notion of fairness, the definitions are not
mathematically compatible in general. In other words, some definitions of
fairness cannot be enforced at the same time. These incompatibilities between
the fairness definitions were first explored during public debate over a
recidivism333Recidivism: the tendency of a convicted criminal to re-offend.
tool called COMPAS (Correctional Offender Management Profiling for Alternative
Sanctions) [15]. While ProPublica proved that COMPAS does not satisfy false
positive error rate balance (see Section 5.2.2) [16], other researchers found
that it did satisfy metrics such as predictive parity (see Section 5.2.1) and
test fairness (see Section 5.3.1) [17, 18].

The tension experienced here is due to impossibility results that govern the
underlying statistics of the different fairness measures. This notion is
backed by several research publications including [13] who explained how
independence, separation, and sufficiency are mutually exclusive, and [19, 20]
who both showed that if a model satisfies balance for the negative class,
balance for the positive class, and test fairness among marginalized and non-
marginalized groups, then there must be equal base rates (which implies that
the actual classification was independent of the group) or the model was 100%
accurate [15].

###  2.2 Causal Frameworks

Here, we briefly introduce two fundamental causal frameworks which are
required for causal-based fairness notions: the structural causal model (SCM)
[21] and the potential outcome framework [22]. We refer interested readers to
recent surveys, such as [23] and [24], in the area of causal inference and
causal discovery for a more in depth discussion. We emphasize that the two
causal frameworks are logically equivalent, although they have different
assumptions. In SCMs, we are able to study the causal effect of any variable
along different paths as the complete causal graph is often assumed. On the
other hand, in the potential outcome framework, we do not assume the
availability of the causal graph and focus on estimating causal effects of the
treatment variables.

Throughout the following discussion, and in Section 6 (which details the
causal-based fairness notions), we use the following notation conventions. An
uppercase letter denotes a variable, e.g., $X$; a bold uppercase letter
denotes a set of variables, e.g., $\mathbf{X}$; and a lowercase letter denotes
a value or a set of values of the corresponding variables, e.g., $x$ and
$\mathbf{x}$.

####  2.2.1 Structural Causal Model

The structural causal model (SCM) was first proposed by Judea Pearl in [21].
Pearl believed that by understanding the logic behind causal thinking, we
would be able to emulate it on a computer to form more realistic artificial
intelligence [25]. He proposed that causal models would give the ability to
“anchor the elusive notions of science, knowledge, and data in a concrete and
meaningful setting, and will enable us to see how the three work together to
produce answers to difficult scientific questions,” [25]. We recount the
important details of SCMs below.

######  Definition 2.1 (Structural Causal Model [21]).

A structural causal model $\mathcal{M}$ is represented by a quadruple
$\langle\mathbf{U},\mathbf{V},\mathbf{F},P(\mathbf{U})\rangle$ where

  1. 1.

$\mathbf{U}$ is a set of exogenous (external) variables that are determined by
factors outside the model.

  2. 2.

$\mathbf{V}$ is a set of endogenous (internal) variables that are determined
by variables in $\mathbf{U}\cup\mathbf{V}$, i.e., $\mathbf{V}$’s values are
determined by factors within the model.

  3. 3.

$\mathbf{F}$ is a set of structural equations from
$\mathbf{U}\cup\mathbf{V}\to\mathbf{V}$. Specifically, for each
$V\in\mathbf{V}$, there is a function $f_{V}\in\mathbf{F}$ mapping from
$\mathbf{U}\cup(\mathbf{V}\backslash V)\to V$, i.e.,
$v=f_{V}(\mathsf{pa}_{V},u_{V})$, where $\mathsf{pa}_{V}$ is a realization of
a set of endogenous variables $\mathsf{PA}_{V}\in\mathbf{V}\setminus V$ that
directly determines $V$, and $u_{V}$ is a realization of a set of exogenous
variables that directly determines $V$. In other words, $f_{V}(\cdot)$ is a
structural equation that expresses the value of each endogenous variable as a
function of the values of the other variables in $\mathbf{U}$ and
$\mathbf{V}$.

  4. 4.

$P(\mathbf{U})$ is a joint probability distribution defined over $\mathbf{U}$.

In general, $f_{V}(\cdot)$ can be any type of equation. In some cases,
$f_{V}(\cdot)$ is assumed as a specific type, for example, the nonlinear
additive function: $v=f_{V}(\mathsf{pa}_{V})+\mathbf{u}_{V}$. If all exogenous
variables in $\mathbf{U}$ are assumed to be mutually independent, then the
causal model is called a _Markovian model_ ; otherwise, it is called a _semi-
Markovian model_.

The causal model $\mathcal{M}$ is associated with a causal graph
$\mathcal{G}=\langle\mathcal{V},\mathcal{E}\rangle$ where $\mathcal{V}$ is a
set of nodes (otherwise known as vertices) and $\mathcal{E}$ is a set of
edges. Each node of $\mathcal{V}$ corresponds to a variable of $\mathbf{V}$ in
$\mathcal{M}$. Each edge in $\mathcal{E}$, denoted by a directed arrow
$\rightarrow$, points from a node $X\in\mathbf{U}\cup\mathbf{V}$ to a
different node $Y\in\mathbf{V}$ if $f_{Y}$ uses values of $X$ as input. A
_causal path_ from $X$ to $Y$ is a directed path which traces arrows directed
from $X$ to $Y$. The causal graph is usually simplified by removing all
exogenous variables from the graph. In a Markovian model, exogenous variables
can be directly removed without loss of information. In a semi-Markovian
model, after removing exogenous variables, we also need to add dashed bi-
directional edges between the children of correlated exogenous variables to
indicate the existence of unobserved common cause factors, i.e., hidden
confounders. Fig. 1 depicts an example of a structural causal model.

Figure 1: Example of a structural causal model depicting the relationships
between variables that determine the survival of some disease.

Quantitatively measuring causal effects in a causal model is facilitated with
the $do$-operator [21] which forces some variable $X$ to take on a certain
value $x$, which can be formally denoted by $do(X=x)$ or $do(x)$. In a causal
model $\mathcal{M}$, the intervention $do(x)$ is defined as the substitution
of the structural equation $X=f_{X}(\mathsf{PA}_{X},U_{X})$ with $X=x$, which
corresponds to a modified causal graph that has removed all edges into $X$ and
in turn sets $X$ to $x$. For an observed variable $Y$ which is affected by the
intervention, its interventional variant is denoted by $Y_{x}$. The
distribution of $Y_{x}$, also referred to as the post-intervention
distribution of $Y$ under $do(x)$, is denoted by $P(Y_{x}=y)$ or simply
$P(y_{x})$.

Similarly, the intervention that sets the value of a set of variables
$\mathbf{X}$ to $\mathbf{x}$ is denoted by $do(\mathbf{X}=\mathbf{x})$. The
post-intervention distribution of all other attributes
$\mathbf{Y}=\mathbf{V}\backslash\mathbf{X}$, i.e.,
$P(\mathbf{Y}=\mathbf{y}|do(\mathbf{X}=\mathbf{x}))$ or simply
$P(\mathbf{y}|do(\mathbf{x}))$, can be computed by the truncated factorization
formula [21],

$P(\mathbf{y}|do(\mathbf{x}))=\prod_{Y\in\mathbf{Y}}P(y|PA(Y))\delta_{\mathbf{X}=\mathbf{x}},$
(1)

where $\delta_{\mathbf{X}=\mathbf{x}}$ means assigning attributes in
$\mathbf{X}$ involved in the next term with the corresponding values in
$\mathbf{x}$. Specifically, the post-intervention distribution of a single
attribute $Y$ given an intervention on a single attribute $X$ is given by

$P(y|do(x))=\sum_{\mathbf{V}\backslash\\{X,Y\\},Y=y}\prod_{V\in\mathbf{V}\backslash\\{X\\}}P(v|PA(V))\delta_{X=x},$
(2)

where the summation is a marginalization that traverses all value combinations
of $\mathbf{V}\backslash\\{X,Y\\}$. We emphasize that $P(y|do(x))$ and
$P(y|x)$ are not the same as the probabilistic distribution representing the
statistical association ($P(y|x)$) is not equivalent to the interventional
distribution ($P(y|do(x))$). We refer the interested readers to [23] for a
discussion of this difference in relation to confounding bias, back-door
criterion, and causal identification.

By using the $do$-operator, the total causal effect is defined as follows.

######  Definition 2.2 (Total Causal Effect [21]).

The total causal effect (TCE) of the value change of $X$ from $x_{0}$ to
$x_{1}$ on $Y=y$ is given by

$\mathrm{TCE}(x_{1},x_{0})=P(y_{x_{1}})-P(y_{x_{0}}).$

The total causal effect is defined as the effect of $X$ on $Y$ where the
intervention is transferred along all causal paths from $X$ to $Y$. Different
from the total causal effect, the controlled direct effect (CDE) measures the
effect of $X$ on $Y$ while holding all the other variables fixed.

######  Definition 2.3 (Controlled Direct Effect).

The controlled direct effect (CDE) of the value change of $X$ from $x_{0}$ to
$x_{1}$ on $Y=y$ is given by

$\mathrm{CDE}(x_{1},x_{0})=P(y_{x_{1},\mathbf{Z}})-P(y_{x_{0},\mathbf{Z}})$

where $\mathbf{Z}$ is the set of all other variables.

In [26], Pearl introduced the causal mediation formula to decompose total
causal effect into natural direct effect (NDE) and natural indirect effect
(NIE).

######  Definition 2.4 (Natural Direct Effect).

The natural direct effect (NDE) of the value change of $X$ from $x_{0}$ to
$x_{1}$ on $Y=y$ is given by:

$\mathrm{NDE}(x_{1},x_{0})=P(y_{x_{1},\mathbf{Z}_{x_{0}}})-P(y_{x_{0}})$

where $\mathbf{Z}$ is the set of mediator variables and
$P(y_{x_{1},\mathbf{Z}_{x_{0}}})$ is the probability of $Y=y$ had $X$ been
$x_{1}$ and had $\mathbf{Z}$ been the value it would naturally take if
$X=x_{0}$. In the causal graph, $X$ is set to $x_{1}$ in the direct path
$X\rightarrow Y$ and is set to $x_{0}$ in all other indirect paths.

######  Definition 2.5 (Natural Indirect Effect).

The natural indirect effect (NIE) of the value change of $X$ from $x_{0}$ to
$x_{1}$ on $Y=y$ is given by:

$\mathrm{NIE}(x_{1},x_{0})=P(y_{x_{0},\mathbf{Z}_{x_{1}}})-P(y_{x_{0}}).$

NDE measures the direct effect of $X$ on $Y$ while NIE measures the indirect
effect of $X$ on $Y$. NDE differs from CDE since the mediators $\mathbf{Z}$
are set to $\mathbf{Z}_{x_{0}}$. In other words, the mediators are set to the
level that they would have naturally attained under the reference condition
$X=x_{0}$.

The problem with NIE is that it does not separate the fair (explainable) and
unfair (indirect discrimination) effects. Path-specific effect [21] is an
extension of total causal effect in the sense that the effect of the
intervention is transmitted only along a subset of causal paths from $X$ to
$Y$. Let $\pi$ denote a subset of causal paths. The $\pi$-specific effect
considers a counterfactual situation where the effect of $X$ on $Y$ with the
intervention is transmitted along $\pi$, while the effect of $X$ on $Y$
without the intervention is transmitted along paths not in $\pi$.

######  Definition 2.6 (Path-specific Effect [27]).

Given a causal path set $\pi$, the $\pi$-specific effect ($\mathrm{PE}_{\pi}$)
of the value change of $X$ from $x_{0}$ to $x_{1}$ on $Y=y$ through $\pi$
(with reference $x_{0}$) is given by

$\mathrm{PE}_{\pi}(x_{1},x_{0})=P(y_{x_{1}|\pi,x_{0}|\bar{\pi}})-P(y_{x_{0}}),$

where $P(Y_{x_{1}|\pi,x_{0}|\bar{\pi}})$ represents the post-intervention
distribution of $Y$ where the effect of intervention $do(x_{1})$ is
transmitted only along $\pi$ while the effect of reference intervention
$do(x_{0})$ is transmitted along the other paths.

It can be directly obtained from the definitions above that the total causal
effect and path-specific effect have the following connections:

  1. 1.

if $\pi$ contains all causal paths from $X$ to $Y$, then
$\mathrm{PE}_{\pi}(x_{1},x_{0})=\mathrm{TCE}(x_{1},x_{0})$

  2. 2.

for any $\pi$, we have
$\mathrm{PE}_{\pi}(x_{1},x_{0})+(-\mathrm{PE}_{\bar{\pi}}(x_{0},x_{1}))=\mathrm{TCE}(x_{1},x_{0})$

Definition 2.2 and 2.6 consider the average causal effect over the entire
population without any prior observations. In contrast, the effect of
treatment on the treated considers the sub-population of the treated group.

######  Definition 2.7 (Effect of Treatment on the Treated).

The effect of treatment on the treated (ETT) of intervention $X=x_{1}$ on
$Y=y$ (with baseline $x_{0}$) is given by

$\mathrm{ETT}_{x_{1},x_{0}}=P(y_{x_{1}|x_{0}})-P(y|x_{0}),$

where $P(y_{x_{1}|x_{0}})$ represents the counterfactual quantity that read as
“the probability of $Y$ would be $y$ had $X$ been $x_{1}$, given that in the
actual world, $X=x_{0}$.”

Generally, if we have certain observations about a subset of attributes
$\mathbf{O}=\mathbf{o}$ and use them as conditions when inferring the causal
effect, then the causal inference problem becomes a _counterfactual inference_
problem. This means that the causal inference is performed on the sub-
population specified by $\mathbf{O}=\mathbf{o}$ only. Symbolically,
conditioning the distribution of $Y_{x}$ on factual observation
$\mathbf{O}=\mathbf{o}$ is denoted by $P(y_{x}|\mathbf{o})$. The
counterfactual effect is defined as follows.

######  Definition 2.8 (Counterfactual Effect [28]).

Given a factual condition $\mathbf{O}=\mathbf{o}$, the counterfactual effect
(CE) of the value change of $X$ from $x_{0}$ to $x_{1}$ on $Y=y$ is given by

$\mathrm{CE}(x_{1},x_{0}|\mathbf{o})=P(y_{x_{1}}|\mathbf{o})-P(y_{x_{0}}|\mathbf{o}).$

In [29], the authors present a general representation of causal effects,
called path-specific counterfactual effect, which considers an intervention on
$X$ transmitted along a subset of causal paths $\pi$ to $Y$, conditioning on
observation $\mathbf{O}=\mathbf{o}$.

######  Definition 2.9 (Path-specific Counterfactual Effect).

Given a factual condition $\mathbf{O}=\mathbf{o}$ and a causal path set $\pi$,
the path-specific counterfactual effect (PCE) of the value change of $X$ from
$x_{0}$ to $x_{1}$ on $Y=y$ through $\pi$ (with reference $x_{0}$) is given by

$\mathrm{PCE}_{\pi}(x_{1},x_{0}|\mathbf{o})=P(y_{x_{1}|\pi,x_{0}|\bar{\pi}}|\mathbf{o})-P(y_{x_{0}}|\mathbf{o}).$

In [30], the conditional path-specific effect is different from our notion in
that, for the former, the condition is on the post-intervention distribution,
and for the latter, the condition is on the pre-intervention distribution.

####  2.2.2 Potential Outcome Framework

The potential outcome framework [22], also known as Neyman-Rubin potential
outcomes or the Rubin causal model, has been widely used in many research
areas to perform causal inference. It refers to the outcomes one would see
under each treatment option. Let $Y$ be the outcome variable, $T$ be the
binary or multiple valued ordinal treatment variable, and $\mathbf{X}$ be the
pre-treatment variables (covariates). Note that pre-treatment variables are
the ones that are not affected by the treatment. On the other hand, the post-
treatment variables, such as the intermediate outcome, are affected by the
treatment.

######  Definition 2.10 (Potential Outcome).

Given the treatment $T=t$ and outcome $Y=y$, the potential outcome of the
individual $i$, $Y_{i}(t)$, represents the outcome that would have been
observed if the individual $i$ had received treatment $t$.

In practice, only one potential outcome can be observed for each individual.
The observed outcome is called the factual outcome and the remaining
unobserved potential outcomes are the counterfactual outcomes. The potential
outcome framework aims to estimate potential outcomes under different
treatment options and then calculate the treatment effect. The treatment
effect can be measured at the population, treated group, subgroup, and
individual levels. Without the loss of generality, we discuss the treatment
effect under binary treatment.

######  Definition 2.11 (Average Treatment Effect).

Given the treatment $T=t$ and outcome $Y=y$, the average treatment effect
(ATE) is defined as

$\mathrm{ATE}=\mathbb{E}[Y(t^{\prime})-Y(t)]$

where $Y(t^{\prime})$ and $Y(t)$ are the potential outcome and the observed
control outcome of the whole population, respectively.

######  Definition 2.12 (Average Treatment Effect on the Treated).

Given the treatment $T=t$ and outcome $Y=y$, the average treatment effect on
the treated group ($ATT$) is defined as

$\mathrm{ATT}=\mathbb{E}[Y(t^{\prime})-Y(t)|T=t].$

The average treatment effect answers the question of how, on average, the
outcome of interest $Y$ would change if everyone in the population of interest
had been assigned to a particular treatment $t^{\prime}$ relative to if they
had received another treatment $t$. The average treatment effect on the
treated on the other hand details the average outcome would change if everyone
who received one particular treatment $t$ had instead received another
treatment $t^{\prime}$.

######  Definition 2.13 (Conditional Average Treatment Effect).

Given the treatment $T=t$ and outcome $Y=y$, the conditional average treatment
effect ($CATE$) is defined as

$\mathrm{CATE}=\mathbb{E}[Y(t^{\prime})-Y(t)|\mathbf{W}={w}]$

where $\mathbf{W}$ is a subset of variables defining the subgroup.

######  Definition 2.14 (Individual Treatment Effect).

Given the treatment $T=t$ and outcome $Y=y$, the individual treatment effect
(ITE) is defined as

$\mathrm{ITE}=\mathbb{E}[Y_{i}(t^{\prime})-Y_{i}(t)]$

where $Y_{i}(t^{\prime})$ and $Y_{i}(t)$ are the potential outcome and the
observed control outcome of individual $i$, respectively.

The potential outcome framework relies on three assumptions:

  1. 1.

Stable Unit Treatment Value Assumption (SUTVA): requires the potential outcome
observation on one unit be unaffected by the particular assignment of
treatments to other units.

  2. 2.

Consistency Assumption: requires that the value of the potential outcomes
would not change no matter how the treatment is observed or assigned through
an intervention.

  3. 3.

Strong Ignorability (unconfoundedness) Assumption: is equal to the assumption
that there are no unobserved confounders444A confounder is a pre-treatment
variable that affects both treatment and outcome variables..

Under these assumptions, causal inference methods can be applied to estimate
the potential outcome and treatment effect given the information of treatment
variable and pre-treatment variables. We refer the interested readers to the
survey [24] for various causal inference methods, including re-weighting,
stratification, matching based, and representation based methods.

##  3 Philosophical Perspectives

This section is devoted to explaining the philosophical underpinnings of both
statistical and causal-based fair machine learning metrics. Many of the
statistical-based fairness metrics in the fair machine learning literature
correspond to the notions of distributive justice from social science
literature [31]. Here, we introduce the philosophical ideal of equality of
opportunity (EOP) and its three main frames: formal EOP, substantive EOP, and
luck-egalitarian EOP. We note that other definitions and ideals of
egalitarianism exist, and are relevant to the discussion on fair machine
learning [32, 33, 34], but we limit ourselves to the discussion below as it
directly aligns with our classification of popular fairness metrics in Section
5. We direct interested readers to these mentioned works, along with
corresponding surveys [35, 36], for additional reading. Additionally, we
introduce the philosophical notions of causality as proposed by historical
philosophers Aristotle, Hume, and Kant as well as through the modern day
descriptions by American philosopher David Lewis.

###  3.1 Equality of Opportunity (EOP)

In [37], they propose grounding current (and future) proposed fair machine
learning metrics in the moral framework of equality of opportunity (EOP) [38].
EOP is a political ideal that is opposed to assigned-at-birth (caste)
hierarchy, but not to hierarchy itself. In a caste hierarchy, a child normally
acquires the social status of their parents. Social mobility may be possible,
but the process to rise through the hierarchy is open to only specific
individuals depending on their initial social status. In contrast to a caste
hierarchy, EOP demands that the social hierarchy is determined by a form of
equal competition among all members of the society.

From a philosophical perspective, EOP is a principle that dictates how
desirable positions, or opportunities, should be distributed among members of
a society. As a moral framework, EOP allows machine learning practitioners to
see fairness notions’ motivations, strengths, and shortcomings in an organized
and comparative fashion. Additionally, it presents moral questions that
machine learning practitioners must answer to guide the construction of a
system that has fairness ideals that satisfy their desired values [37].
Further, it allows practitioners to understand and appreciate why there may be
disagreement when it comes to choosing a specific fairness metric, as
different people will have different moral beliefs about what fairness and
equality mean. Different conceptions of EOP, such as formal, substantive, and
luck-egalitarian EOP, interpret the idea of competing on equal terms in
different ways.

Figure 2: Definitions and warnings for popular equality of opportunity modes.

####  3.1.1 Formal EOP

Formal EOP emphasizes that any desirable position (in a society, or more
concretely, a job opening) is available and open to everyone. The distribution
of these desirable positions follows according to the individual’s relevant
qualifications, and in this setting, the most qualified always wins. Formal
EOP takes a step in the direction of making decisions based on relevant
criteria rather than making them in a blatant discriminatory fashion [37]. In
fair machine learning settings, formal EOP has often been implemented as
fairness through blindness or fairness through unawareness [39]. In other
words, formal EOP-based metrics strip away any irrelevant marginalization
attributes, such as race or gender, before training is performed.

However, while formal EOP has the benefit of awarding positions based on
actual qualifications of an individual, and in excluding irrelevant
marginalization information, it makes no attempt to correct for arbitrary
privileges. This includes unequal access to opportunities that can lead to
disparities between individuals’ qualifications. Examples of this situation
can be seen in the task of predicting prospective students’ academic
performance for use in college admission decisions [40]. Individuals belonging
to marginalized or non-conforming groups, such as Black or LGBTQIA+555Lesbian,
Gay, Bisexual, Transgender, Queer, Intersex, and Asexual students, are
disproportionately impacted by the challenges of poverty, racism, bullying,
and discrimination. An accurate predictor for a student’s success therefore
may not correspond to a fair decision-making procedure as the impact of these
challenges create a tax on the “cognitive-bandwidth” of non-majority students,
which in turn, affects their academic performance [41, 42].

The problem of not accounting for arbitrary privileges can be broken down into
two main problems: the Before and After problems. In the Before problem,
arbitrary and morally irrelevant privileges weigh heavily on the outcomes of
formally fair competitions as people with more privilege are often in a better
position to build relevant qualifications. This can be seen in the problem
described above of predicting students’ performance for admissions. The After
problem is an effect of formally fair, but not arbitrary-privilege aware,
competition in that the winner of a competition (e.g., getting hired or
admitted to a top-tier college) is then in the position to gain even more
success and qualifications. It introduces a compounding snow-ball effect in
that winners win faster, but losers also lose faster [37]. Overall, formal EOP
compounds both privilege and disprivilege, a term referred to as
discrimination laundering [43].

####  3.1.2 Substantive (Rawls) EOP

Substantive EOP goes one step beyond formal EOP, and address the
discrimination laundering problem in that it requires all individuals have the
same opportunity to gain qualifications. It aims to give everyone a fair
chance at success in a competition. For example, making all extra curricular
activities and high-school opportunities equally available to all students
regardless of wealth or social status. Substantive EOP is often equated with
Rawls’ fair EOP which states that all individuals, regardless of how rich or
poor they are born, should have the same opportunities to develop their
talents in order to allow people with the same talents and motivation to have
the same opportunities [44]. In fair machine learning, substantive EOP is
often implemented through metrics such as statistical parity and equalized
odds, which assume that talent and motivation are equally distributed among
sub-populations.

But, the assumption that these talents are equally distributed, often does not
hold in practice. By the time a machine learning system is being used to make
a decision, it is normally too late to provide individuals with the
opportunity to develop qualifications. In this lens, fair machine learning has
re-written Rawls’ EOP to say that a competition must only measure a candidate
on the basis of their talent, while ignoring qualifications that reflect the
candidates unequal developmental opportunities prior to the point of the
competition [37].

####  3.1.3 Luck-Egalitarian EOP

Furthering the ideals of substantive EOP, luck-egalitarian EOP enforces that a
person’s outcome should be affected only by their choices, not their
circumstances [45]. For example, a student with rich parents who did not try
hard in their studies should not have an advantage over a student with poor
parents who did work hard in being admitted to a university. Overall, luck-
egalitarian EOP is an attractive idea, but the difficulty of separating choice
from circumstance is non-trivial, and in practice, quite difficult to
implement.

A few solutions for separation have been proposed. Economist and political
scientist John Roemer proposed instead of trying to separate an individuals
qualifications into effects of consequence and choice, we should instead
control for certain matters of consequence (such as, race, gender, and
disability) that will impact a person’s access to opportunities to develop
qualifications [36]. While this solution solves the separation problem,
another issue of sub-group comparison emerges. We can compare apples to
apples, and oranges to oranges, but we are now unable to compare apples to
oranges [37]. Unfortunately, the EOP frameworks offer no solution to this
problem of overall ranking.

Another problem in luck-egalitarian EOP is the separation of efforts in
addition to circumstance [37]. It may be the case that a wealthy student works
hard at their studies, i.e., the circumstance of being wealthy interacts with
the effort of the student. This effect of entanglement is nearly impossible to
separate. But, fortunately, this separation is only required when the
circumstance gives access to a broad range of advantages. For instance, if a
student’s family wealth status allows them to gain an advantage over all other
students in almost every competition (not just university admission, but also
job hiring or access to other opportunities), then there is a fairness
problem. This is because there is an indication that the arbitrary privilege
or circumstance, and not the relevant skill, is being used as the basis for
decision. On the other hand, if the student only has the advantage in the
admissions process, then it could be due to their effort rather than their
circumstance, and we may or may not have a matter of unfairness where we need
to separate effort from circumstance.

###  3.2 Causality and Counterfactuals

Figure 3: Pearl’s Ladder of Causation. The first rung, associations, only
allows predictions based on passive observations. The second rung,
interventions, not only relies on seeing, but also changing what is. Rung
three, counterfactuals, deals with the imaginary, or what might have been. See
Section 6 for a more in-depth view of the Ladder of Causation.

“Felix, qui potuit rerum cognoscere causas," Latin poet Virgil wrote in 29 BC,
“fortunate is he, who is able to know the causes of things." Several hundreds
of years later in 1953, Albert Einstein reaffirmed this statement, proclaiming
that “the development of Western science is based on two great achievements:
the invention of the formal logical system (in Euclidean geometry) by the
Greek philosophers, and the discovery of the possibility to find out causal
relationships by systematic experiment (during the Renaissance).” But even
thousands of years before Virgil and Einstein, dating back to 3,000-2,000 BC,
the Sumerian civilization often wrote of causal relationships (termed as omens
in their day) that had the form of “if $x$ happens, then $y$ will happen”
[46]. Causal and counterfactual reasoning is “part of the mental apparatus
that makes us human", so it is not surprising that we can find causal and
counterfactual statements “as far back as we want to go in human history
[25]."

The first formal investigation into causality was done by the Greek
philosopher Aristotle. In 350 BC, Aristotle published his two famous treatise,
Physics and Metaphysics. In these treatise, Aristotle not only opposed the
previously proposed notions of causality for not being grounded in any solid
theory [47], but he also constructed a taxonomy of causation which he termed
“the four causes." In order to have proper knowledge, he deemed that we must
have grasped its cause, and that giving a relevant cause is necessary and
sufficient in offering a scientific explanation. His four causes can be seen
as the four types of answers possible when asked a question of “why."

  1. 1.

The material cause: “that out of which” (something is made). E.g, the marble
of a statue.

  2. 2.

The formal cause: “the form”, “the account of what-it-is-to-be.” E.g, the
shape of the statue.

  3. 3.

The efficient cause: “the primary source of the change or rest.” E.g, the
artist/sculptor of the marble statue.

  4. 4.

The final cause: “the end, that for the sake of which a thing is done.” E.g,
the creation of a work of art.

Despite giving four causes, Aristotle was not committed to the idea that
everything explanation had to have all four. Rather, he reasoned that any
scientific explanation required up to four kinds of cause [47]. It is
important to note, however, that Aristotle made no mention of the theory of
counterfactuals.

To find a philosopher who centered causation around counterfactual reasoning,
we look to 18th century Scottish philosopher David Hume. He rejected
Aristotle’s taxonomy and instead insisted on a single definition of cause.
This is despite the fact that he himself could not choose between two
different, and later found to be incompatible, definitions [25]. In his
Treatise of Human Nature, Hume states that “several occasions of everyday
life, as well as the observations carried out for scientific purposes, in
which we speak of a condition A as a cause and a condition B as its effect,
bear no justification on the facts, but are simply based on our habit of
observing B after having observed A" [46]. In other words, Hume believed that
the cause-effect relationship was a sole product of our memory and experience
[25]. Later, in 1739, Hume published An Enquiry Concerning Human Understanding
in which he framed causation as a type of correlation: “we may define a cause
to be an object followed by another, and where all the objects, similar to the
first, are followed by objects similar to the second. Or in other words,
where, if the first object had not been, the second never had existed." While
he tried to pass these two definitions off as one by using “in other words,"
David Lewis pointed out that the second statement is contradictory to the
first as it explicitly invokes the notion of a counterfactual which, cannot be
observed, only imagined [25].

The incorporation of counterfactuals into causal theory went largely ignored
by philosophers through the 19th and 20th centuries who instead tried to
justify Hume’s first definition through the theory of probabilistic
causation666See The Book of Why: The New Science of Cause and Effect by Judea
Pearl and Dana Mackenzie for the entangled history of causation,
counterfactuals, and statistics. [25]. It was not until the 1973 publication
of Counterfactuals by David Lewis that this trend would change. He called for
abandoning the first definition all together, and to instead interpret “A has
caused B" as “B would not have occurred if not for A" as it more closely
aligned with human intuition [48]. He reasoned that we as humans make
counterfactual judgements easily, often, and that we assign these
counterfactual situations truth values and probabilities with the same
confidence that we have for factual statements by envisioning “possible
worlds" in which the counterfactual statements are actually true [25]. The
idea of other possible worlds was met with mass criticism, and it would take
the work of Judea Pearl to defend Lewis’s claims.

Much of modern day philosophy and theory on causality can be attributed to
Judea Pearl. Pearl, an Israeli-American computer scientist and philosopher, is
credited with the development of causal and counterfactual inference theory
based on structural models as well as the Ladder of Causation, which is shown
in Fig. 3. He believed that Lewis’s critics (as well as Lewis himself) were
thinking the wrong way about the “existence of other worlds." He argued that
we do not need to debate about if these world exist or not, but rather we are
simply hypothesizing that people are capable of imagining other worlds in
their heads, which relates to the “amazing uniformity of the architecture of
the human mind" [25]. He believed that we humans experience the same world,
which in turn allows us to experience the same mental model of its causal
structure and binds us together into communities [25] (see Section 7.4 for
critique of this idea). Additionally, he believed that while most philosophers
thought the structural causal model to be just one possible implementation of
Lewis’s logic, SCM were actually extremely similar to the way that our brains
represent counterfactuals [25].

Beyond Lewis and Pearl, several other modern day philosophers have proposed
notions of causality with vary degrees of “realness”. For instance, Bertrand
Russell and Karl Pearson denied any scientific status to the cause-effect
relationship [46]. Other statisticians and philosophers of science opposed
this line of thinking. One example is Wesley C. Salmon, who wrote: “to many
scientists uncorrupted by philosophical training, it is evident that causality
plays a central role in scientific explanation" [49].

##  4 Legal Perspectives

The aspects of legality-based fairness will change depending on what country
hosts the party who is wanting to create a fair machine learning algorithm.
What is considered legally fair in one country may contradict or oppose that
of another. For this reason, we specify that this article is framed from the
viewpoint of Western law, specifically that of U.S. anti-discrimination and
criminal law. We chose this frame of reference since most fair machine
learning literature focuses on terminology and legal aspects proposed in this
body of law [50].

Figure 4: Graphical depiction of disparate impact vs. disparate treatment. For
the disparate impact (left) side, while both people were given boxes to stand
on in order to see over the fence, the person on the left is
disproportionately impacted by being shorter. For disparate treatment, the
person on the left if directly being discriminated against by not being given
a box to stand on.

###  4.1 Disparate Impact

Disparate impact occurs when members of a marginalized class are negatively
affected more than others when using a formally neutral policy or rule [51].
In other words, it is unintentional or indirect discrimination. This statutory
standard was first formalized in the 1971 landmark U.S. Supreme Court case of
Griggs vs. Duke Power Co., where Duke Power Co. required employees to have a
high school diploma in order to be considered for a promotion [52]. The
Supreme Court noted that Title VII of the 1964 Civil Rights Act was violated
in this situation as Duke Power Co.’s diploma requirement, which had little
relation to the overall job performance, prevented a disproportionate number
of Black employees from being promoted, and thus, deemed it to have an illegal
(unjustified) disparate impact. In fair machine learning literature, disparate
impact is often framed as occurring when the outcomes of classification differ
across different subgroups, even if this outcome disparity is not intentional
[53].

However, disparate impact in of itself is not illegal. In U.S. anti-
discrimination law, indirect discrimination in employment is not unlawful if
it can be justified by a “legitimate aim" or a genuine occupational
requirement and/or business necessity [54]. For example, in the Griggs vs.
Duke Power Co. case, if the high school diploma requirement was shown to be
necessary for overall job success, then the resulting disparities would have
been legal [12].

The most common measure of disparate impact is the four-fifths rule proposed
in the Uniform Guidelines on Employee Selection Procedures in the Code of
Federal Regulation [55]. This rule states that if the selection rate for a
certain group is less than 80% of the group with the highest selection rate,
then there is a disparate impact (or adverse impact) on that group. For a
concrete example, in the Griggs vs. Duke Power Co. case, the selection rate of
Black individuals for promotion was 6% while the selection rate of White
individuals was 58%. By taking the ratio of the two selection rates we get
$\frac{6}{58}=10.3\%$ which is well below the 80% threshold. In Section 5.1 we
discuss a fairness metric that aligns well with the disparate impact four-
fifths rule.

###  4.2 Disparate Treatment

In direct contrast with disparate impact, disparate treatment occurs when an
individual is intentionally treated different based on their membership of a
marginalized class. The Uniform Guidelines on Employee Selection Procedures
states that: “disparate treatment occurs where members of a race, gender, or
ethnic group have been denied the same employment, promotion, membership, or
other employment opportunities as have been available to other employees or
applicants” [56]. The key legal question behind disparate treatment is whether
the alleged discriminator’s actions were motivated by discriminatory intent
[55]. In fair machine learning literature, a metric is often described as
adhering to disparate treatment law if it does not use any marginalization
attributes in the decision making process [57].

###  4.3 Anti-Classification and Anti-Subordination

Two main principles that were produced by the U.S. anti-discrimination
legislation are anti-classification and anti-subordination [50, 58]. The anti-
classification principle notes that the U.S. government may not classify
people (either overtly or surreptitiously) by use of a marginalization
attribute such as race or gender. Anti-subordination, on the other hand, is
the idea that anti-discrimination laws should serve a grander purpose and
should actively be aiming to tackle societal hierarchies between different
groups. In other words, it argues that it is inappropriate for any group to be
“subordinated” in society, and that empowering marginalized groups, even at
the sake of the non-marginalized group, should be the main focus of anti-
discrimination law [59]. Anti-classification is often seen in fair machine
learning as fairness through unawareness [60], although, many note that the
exclusion of the marginalization attributes can lead to discriminatory
solutions [12]. Anti-subordination is less popular in the fair machine
learning literature, but some work using debiasing techniques could
potentially have the effect of anti-subordination [50, 53].

###  4.4 Critique of Current Fair ML Metric Alignment

Several works exist that critique the alignment of current fairness metrics
with disparate impact and disparate treatment. Xiang and Raji note that both
disparate impact and disparate treatment were developed with human
discriminators in mind, and simply replacing human decision makers with
algorithmic ones is often not appropriate [50]. They state that “intent is an
inherently human characteristic”, and the common fair machine learning
characterization of disparate treatment as not using marginalization class
variables in an algorithm should be contested. They also note that simply
accounting for disproportionate outcomes is not enough to prove disparate
impact. It is only the first step of a disparate impact case, and there is
only liability if the defendant cannot justify the outcomes by using non-
discriminatory rationals.

Additionally, [61] notes that while most work on fair machine learning has
focused on achieving a fair distribution of decision outcomes, little to no
attention has been paid to the overall decision process used to generate the
outcome. They note that this is at the determent of not incorporating human
moral sense for whether or not it is fair to use a feature in a decision
making scenario. To this end, they propose notions of procedural fairness that
utilize several considerations that are overlooked in distributive fairness
cases, such as feature volitionality, feature reliability, feature privacy,
and feature relevance.

Table 3: Definitions of legal causation terminology. Term | Definition
---|---
Causation | The causing or producing of an effect.
| Factual
---
(“but for”)
Causation
| An act or circumstance that causes an event, where the event
---
would not have happened had the act or circumstance not occurred.
| Proximate
---
Causation
A cause that is legally sufficient to result in liability.
| Intervening
---
Factor
| An event that comes between the initial event (in a sequence of
---
events) and the end result, thereby altering the natural course of
events that might have connected a wrongful act to an injury.
| Superceding
---
Factor
| An intervening act that the law considers sufficient to override the
---
cause for which the original actor is responsible, thereby relieving
the original actor of liability for the result.

###  4.5 Procedural Fairness

Machine learning fairness researchers are beginning to recognize that most of
the proposed fairness metrics have focused solely on distributive fairness
(i.e., disparate impact or disparate treatment) [62, 63, 64]. But, over the
last few years, commentary on utilizing procedural fairness as a grounding for
fair machine learning metrics have been published [62, 61]. Procedural
fairness refers to the fairness of the decision making process that lead to
the outcome. This contrasts with distributive fairness which refers to making
sure that the outcomes of a process (or classification task) are fair [61].

However, there has been some push-back on this view of procedural fair machine
learning, though. Xiang and Raji note that the term “procedural fairness” as
described in the fair machine learning literature is a narrow and misguided
view of what procedural fairness means from a legal lens [50]. Procedural
justice aims to arrive at a just outcome through an iterative process as well
as through a close examination of the set of governing laws in place that
guide the decision-maker to a specific decision [50, 65]. They pose that the
overall goal of procedural fairness in machine learning should be re-aligned
with the aim of procedural justice by instead analyzing the system surrounding
the algorithm, as well as its use, rather than simply looking at it from the
specifics of the algorithm itself.

###  4.6 Causality and the Law

When proving guilt in a criminal court case, the prosecution is required to
prove that the defendant’s action was the legal cause of the result.
Establishing this causal relationship is a two-step process in which the
prosecution first establishes factual (“but for”) causation and then
determines if there is proximate causation [66]. Different to criminal law,
anti-discrimination law is motivated by “because of” causality, i.e., decision
cannot be made “because of” an individual’s marginalization attribute [50]. In
this section, we will focus on criminal law as it is the most intuitive
implementation of causality in the legal field.

To prove factual causation, the prosecutor must depict that “but for” the
defendant’s specific actions, the result or injury would not have happened as
it did, when it did, or at all. However, this does not mean that the
prosecution has to prove that the defendant’s actions were the sole cause of
the result, as their actions may have been combined with those of another
person, or another circumstance, that all contributed to the final result. An
exception to factual causation is when the chain of events caused by the
defendant’s actions is effectively broken. These intervening factors must be
unforseeable. For instance, if the defendant’s actions put the victim in the
hospital (in a non-critical condition), but by the effect of gross medical
malpractice (just negligent medical care is foreseeable), they die. Then, the
defendant would most likely be charged for assault, but not homicide.

After proving factual causation, the prosecution must then prove proximate
causation. Typically, proximate cause issues arise when the final result
occurs through an unexpected manner. For instance, if the defendant shot the
victim in the arm, who then while running away from the defendant, tripped,
fell, and cracked their skull, which resulted in their death a few moments
later, then the defendant’s actions were the proximate cause of the victims
death. The general rule is that the defendant’s actions will be regarded as
the proximate cause of a result if the result occurred as a “natural and
probable consequence” of the acts, and there was no intervening factor
sufficient enough to break the chain of causation [67, 68].

Figure 5: Graphical model displaying the causal relationships between the
defendant’s actions, the victim’s result, along with intervening and
superceding Factors.

Using causal graphical modeling, we can display the relationship of causation
in the law as shown in Fig. 5. When relating to casual-based fairness metrics
(see Section 6), the legal notion of causality closely aligns with the idea of
path-specific causal effect. In this case, instead of computing the direct and
indirect effects, path-specific causal effect isolates the contribution of the
effect along a specific group of paths [69]. This is similar to how lawyers
and judges make decisions on if a certain action caused a certain effect. For
instance, they reason if the intervening factor (if there is one) played a
role in the victim’s result and if this intervening factor “broke the chain”
of the defendants actions in a way that no longer holds them liable. This
would result in turning Defendant’s Actions $\to$ Intervening Factor $\to$
Victim’s Result to simply be Intervening Factor $\to$ Victim’s Result as shown
in Figure 5.

While judges and policy makers are apt at determining cause in the real-world,
they often do not understand what indicates causality in the use of
algorithms. Xiang and Raji give the following example [50]:

> “The U.S. Department of Housing and Urban Development recently proposed a
> rule that would create a safe harbor from disparate impact claims for
> housing algorithms that do not use [marginalization] class variables or
> close proxies. The rule seeks to implement the Supreme Court decision in the
> Texas Department of Housing and Community Affairs v. Inclusive Communities
> Project, Inc. case, which required housing discrimination plaintiffs to
> ‘show a causal connection between the Department’s policy and a disparate
> impact.’ That said, from a statistical perspective, the presence or absence
> of [marginalization] class variables (or close proxies) in the algorithm
> does not indicate the presence or absence of a causal connection.”

With this confusion, it is important that the fair machine learning community
makes efforts to communicate with the legal field about what causality means
from a technical perspective, and help them to translate what the
corresponding nature of causality would then be in legal terms.

##  5 Statistics-Based Fairness Metrics

In this section, we organize several popular statistics-based fair machine
learning metrics into categories based on what attributes of the system they
use (e.g., the predicted outcome, the predicted and actual outcomes, or the
predicted probability and actual outcome), which formal statistical criterion
(independence, separation, or sufficiency) it aligns with as proposed in [13],
as well as which philosophical ideal serves as its foundation (e.g., Rawls’
(substantive) EOP or Luck-Egalitarian EOP) by using the classification
procedure explained in [70, 71]. Fig. 6 shows our classification through an
extended confusion matrix and Table 4 summarizes our main conclusions.
Additionally, at the end of this section, we devote space to discussing
individual fairness and the (apparent) differences between individual and
group fairness metrics.

Figure 6: Classification of each fairness metric into our extended confusion
matrix. Metrics were classified by which statistics they used. For instance,
since equalized odds uses positive predictive rates, it is on the row where
the predicted value is positive. Similarly, since test fairness says that all
groups should have equal probability of belonging to the positive class, it is
classified in the top left square related to the actual label of positive.

For clarity, we note that this section uses the following variables: $S=1$ is
the marginalized or minority group, $S=0$ is the non-marginalized or majority
group, $\hat{Y}$ is the predicted value or class (i.e., label), and $Y$ is the
true or actual label. Additionally, to provide a background for our future
explanations, we state a relaxed, binary, version of two important definitions
for Rawls’ EOP and Luck-Egalitarian EOP for supervised learning proposed by
Heidari et al [70]:

######  Definition 5.1 (Rawls’ EOP for Supervised Learning).

Predictive model $h$ satisfies Rawlsian EOP if for all $s\in S=\\{0,1\\}$ and
all $y\in Y=\\{0,1\\}$:

$F^{h}(U\leq u\;|\;S=0\cap Y=y)=F^{h}(U\leq u\;|\;S=1\cap Y=y)$

where $F^{h}(U\leq u)$ specifies the distribution of utility across
individuals under the predictive model $h$. I.e., difference between the
individuals’ actual effort, $A$, and their circumstance, $D$, $U=A-D$. In this
relaxed case, utility is the difference between the individuals predicted and
actual class.

######  Definition 5.2 (Luck-Egalitarian EOP for Supervised Learning).

Predictive model $h$ satisfies luck-egalitarian EOP if for all
$\hat{y}\in\hat{Y}=\\{0,1\\}$ and all $s\in S=\\{0,1\\}$:

$F^{h}(U\leq u\;|\;S=0\cap\hat{Y}=\hat{y})=F^{h}(U\leq
u\;|\;S=1\cap\hat{Y}=\hat{y})$

where $F^{h}(U\leq u)$ specifies the distribution of utility across
individuals under the predictive model $h$. I.e., difference between the
individuals’ actual effort, $A$, and their circumstance, $D$, $U=A-D$. In this
relaxed case, utility is the difference between the individuals predicted and
actual class.

###  5.1 Predicted Outcome

The predicted outcome family of fairness metrics are the simplest, and most
intuitive, notions of fairness. This is because the metrics in this category
only use the model’s final predictions and do not rely on the original class
labels. More explicitly, the predicted outcome class of metrics focuses on
using the predicted outcome of various different demographic distributions of
the data, and models only satisfy this definition if both the marginalized and
non-marginalized groups have equal probability of being assigned to the
positive predictive class [72].

Many different metrics fall into the predicted outcome category, such as group
fairness and conditional statistical parity. Additionally, each metric in this
group satisfies Rawls’ definition of EOP as well as satisfies the statistical
constraint of independence.

####  5.1.1 Group Fairness

Group fairness is also often called statistical parity, statistical fairness,
equal acceptance rate, demographic parity, or benchmarking. As the name
implies, it requires that there is an equal probability for both individuals
in the marginalized and non-marginalized groups to be assigned to the positive
class [60, 39]. Notationally, group fairness can be written as:

$P[\hat{Y}=1\;|\;S=0]=P[\hat{Y}=1\;|\;S=1]$

Heidari et. al [70] derive the proof that group fairness falls into Rawls’
definition of EOP and we recall their proof here.

######  Proposition 5.1 (Group Fairness as Rawls’ EOP [70]).

Consider the binary classification task where $Y,\hat{Y}=\\{0,1\\}$. Suppose
$U=A-D$, $A=\hat{Y}$, and $D=Y=1$ (i.e., the effort-based utility of all
individuals is assumed to be the same). Then, the conditions of Rawls’ EOP is
equivalent to group fairness when $\hat{Y}=1$.

###### Proof.

Recall that Rawls’ EOP requires that $s\in S=\\{0,1\\}$, $y\in Y=\\{0,1\\}$,
$u=a-d\in\\{-1,0\\}$:

$P[U\leq u\;|\;S=0\cap Y=y]=P[U\leq u\;|\;S=1\cap Y=y]$

Replacing $U$ with $(A-D)$, $D$ with 1, and $A$ with $\hat{Y}$, the above is
equivalent to:

$P[A-D\leq u\;|\;S=0\cap Y=1]=P[A-D\leq u\;|\;S=1\cap Y=1]$ $P[\hat{Y}-1\leq
u\;|\;S=0\cap Y=1]=P[\hat{Y}-1\leq u\;|\;S=1\cap Y=1]$ $P[\hat{Y}\leq
u+1\;|\;S=0]=P[\hat{Y}\leq u+1\;|\;S=1]$
$P[\hat{Y}=\hat{y}\;|\;S=0]=P[\hat{Y}=\hat{y}\;|\;S=1]$

because of the facts that $u=\hat{y}-y$ and $y=1$ produce the result
$\hat{y}=u+1$. This is equal to the definition for statistical parity when
$\hat{Y}=1$, therefore, the conditions of Rawls’ EOP is equivalent to
statistical parity. ∎

One important derivative of group fairness is disparate impact, which we
discuss below.

#####  $\square$ Disparate Impact

It is no coincidence that the fairness metric of disparate impact shares a
name with the legal term of disparate impact. This fairness metric, discussed
extensively in [73, 74], was designed to be the mathematical counterpart to
the legal notion. It is defined as:

$\frac{P[\hat{Y}=1\;|\;S=0]}{P[\hat{Y}=1\;|\;S=1]}\geq 1-\epsilon$

where $\epsilon$ is the allowed slack of the metric and is usually set to
$0.2$ to achieve the $80\%$ rule of disparate impact law. This equation says
that the proportion of positive predictions for both the marginalized and non-
marginalized groups must be similar (around threshold $1-\epsilon$).

Like group fairness, it also aligns with Rawls’ EOP and the statistical notion
of independence. To see how it aligns with Rawls’ EOP, consider the last line
from the proof above (when substituting $\hat{Y}$ with 1):

$P[\hat{Y}=1\;|\;S=0]=P[\hat{Y}=1\;|\;S=1]$

Dividing both sides by $P[\hat{Y}=1\;|\;S=1]$ we achieve

$\displaystyle\frac{P[\hat{Y}=1\;|\;S=0]}{P[\hat{Y}=1\;|\;S=1]}$
$\displaystyle=1$
$\displaystyle\frac{P[\hat{Y}=1\;|\;S=0]}{P[\hat{Y}=1\;|\;S=1]}$
$\displaystyle\geq 1-\epsilon$

which is the same equation for disparate impact. Additionally, it matches
exactly with the relaxed notion of independence mentioned in Section 2.1.1.

####  5.1.2 Conditional Statistical Parity

Conditional statistical parity, or conditional group fairness, is an extension
of group fairness which allows a certain set of legitimate attributes to be
factored into the outcome [75]. Factors are considered “legitimate” if they
can be justified by ethics, by the law, or by a combination of both. This
notion of fairness was first defined by Kamiran et al. in 2013 who wanted to
quantify explainable and illegal discrimination in automated decision making
where one or more attributes could contribute to the explanation [76].
Conditional statistical parity is satisfied if both marginalized and non-
marginalized groups have an equal probability of being assigned to the
positive predicted class when there is a set of legitimate factors that are
being controlled for. Notationally, it can be written as:

$P[\hat{Y}=1\;|\;L_{1}=a\cap L_{2}=b\cap S=0]=P[\hat{Y}=1\;|\;L_{1}=a\cap
L_{2}=b\cap S=1]$

where $L_{1},L_{2}$ are legitimate features that are being conditioned on. For
example, if the task was to predict if a certain person makes over $50,000 a
year777This is the proposed task of a famous toy dataset in fair machine
learning called the Adult Income dataset [77]., then $L_{1}$ could represent
work status and $L_{2}$ could be the individuals relationship status. Another,
simplified way to write this can be seen as:

$P[\hat{Y}=1\;|\;L=\ell\cap S=1]=P[\hat{Y}=1\;|\;L=\ell\cap S=0]$

where $\ell\in L$ is the set of legitimate features being conditioned on.

Like group fairness, conditional statistical parity also falls under the
definition of Rawls’ EOP and adheres to a relaxed notion of statistical
independence. The proof for conditional statistical parity falling into Rawls’
EOP can be seen when we let $D$ be equal to $L$. In other words, instead of
having the effort-based utility be equal to the true label of the individual,
let the effort-based utility be equal to the set of legitimate factors that
affect the result. Then, the proof reduces to the proof shown for group
fairness. This substitution is allowed since the legitimate factors provide
reasoning for the actual label $Y$.

Furthermore, conditional statistical parity has a theoretical grounding of
Simpson’s paradox as it tries to incorporate extra conditioning information
beyond the original class label. Simpson’s paradox says that if a correlation
occurs in several different groups, it may disappear, or even reverse, when
the groups are aggregated [78]. This event can be seen in Fig. 7. Put
mathematically, Simpson’s paradox can be written as:

$P[A\;|\;B\cap C]<P[A\;|\;B^{c}\cap C]\textrm{ and }P[A\;|\;B\cap
C^{c}]<P[A\;|\;B^{c}\cap C^{c}]$ but $P[A\;|\;B]>P[A\;|\;B^{c}]$

An analysis that does not consider all of the relevant statistics might
suggest that unfairness and discrimination is at play, when in reality, the
situation may be morally and legally acceptable if all of the information was
known.

Figure 7: Two different examples of Simpson’s paradox. In the image on the
left, while both the blue and red groups are positively correlated among
themselves, when taken as a whole, there is a negative correlation. The
opposite can be seen in the second image where the groups are negatively
correlated when taken alone, but when aggregated, a positive correlation is
produced.

###  5.2 Predicted and Actual Outcome

The predicted and actual outcome class of metrics goes one step beyond the
predicted outcome class since it uses both the model’s predictions as well as
the true labels of the data. This class of fairness metrics includes:
predictive parity, false positive error rate balance, false negative error
rate balance, equalized odds, conditional use accuracy, overall accuracy
equality, and treatment equality.

####  5.2.1 Predictive Parity

Predictive parity, otherwise known by the name outcome test, is a fairness
metric that requires the positive predictive values to be similar across both
marginalized and non-marginalized groups [79]. Mathematically, it can be seen
in two ways:

$P[Y=1\;|\;\hat{Y}=1\cap S=0]=P[Y=1\;|\;\hat{Y}=1\cap S=1]$
$P[Y=0\;|\;\hat{Y}=1\cap S=0]=P[Y=0\;|\;\hat{Y}=1\cap S=1]$

since if a classifier has equal positive predictive values for both groups, it
will also have equal false discovery rates. Since predictive parity is simply
conditional use accuracy when $\hat{Y}=1$, it falls into the same
philosophical category as conditional use accuracy, which is luck-egalitarian
EOP. This proof can be seen in Section 5.2.5. Further, [13] states that
predictive parity aligns with the relaxed version of sufficiency.

####  5.2.2 False Positive Error Rate Balance

False positive error rate balance, otherwise known as predictive equality,
requires that false positive rates are similar across different groups [79].
It can be seen mathematically through the following two equations

$P[\hat{Y}=1\;|\;Y=0\cap S=0]=P[\hat{Y}=1\;|\;Y=0\cap S=1]$
$P[\hat{Y}=0\;|\;Y=0\cap S=0]=P[\hat{Y}=0\;|\;Y=0\cap S=1]$

It can be written in two different forms because if a classifier has equal
false positive rates for both groups, it will also have equal true negative
rates. As opposed to predictive parity, false positive error rate balance
falls into Rawls’ EOP for its philosophical classification. This is because it
is a constrained version of equalized odds, which is classified as Rawls’ EOP
as seen in Section 5.2.4. Additionally, false positive error rate balance
aligns with the relaxed version of separation [13].

####  5.2.3 False Negative Error Rate Balance

False negative error rate balance, also called equal opportunity, is the
direct opposite of the above fairness metric of false positive error rate
balance in that it requires false negative rates to be similar across
different groups [79]. This metric can also be framed two different ways:

$P[\hat{Y}=0\;|\;Y=1\cap S=0]=P[\hat{Y}=0\;|\;Y=1\cap S=1]$
$P[\hat{Y}=1\;|\;Y=1\cap S=0]=P[\hat{Y}=1\;|\;Y=1\cap S=1]$

since a classifier that has equal false negative rates across the two groups
will also have equal true positive rates. Similar to false positive error rate
balance, false negative error rate balance can be classified into Rawls’ EOP
as well since it is also a constrained version of equalized odds.
Additionally, like false positive error rate balance, false negative error
rate balance aligns with the relaxed version of separation [13].

####  5.2.4 Equalized Odds

The fairness metric of equalized odds is also known as conditional procedure
accuracy equality and disparate mistreatment. It can be seen as the
combination of false positive error rate balance and false negative error rate
balance since it requires that true and false positive rates are similar
across different groups [80].

$P[\hat{Y}=1\;|\;Y=y\cap S=0]=P[\hat{Y}=1\;|\;Y=y\cap S=1]\;\;\textrm{ for
}\;\;y\in\\{0,1\\}$

As mentioned in the above two metrics of false negative/positive error rate
balance, equalized odds aligns with Rawls’ EOP. [70] provides a proof for this
classification, which we recall here:

######  Proposition 5.2 (Equalized Odds as Rawls’ EOP [70]).

Consider the binary classification task where $Y,\hat{Y}=\\{0,1\\}$. Suppose
$U=A-D$, $A=\hat{Y}$ (i.e., the actual utility is equal to the predicted
label) and $D=Y$ (i.e., effort-based utility of an individual is assumed to be
the same as their true label). Then the conditions of Rawls’ EOP is equivalent
to those of equalized odds.

###### Proof.

Recall that Rawls’ EOP requires that for $s\in S=\\{0,1\\}$, $y\in
Y=\\{0,1\\}$, and $u\in\\{-1,1\\}$:

$P[U\leq u\;|\;Y=y\cap S=0]=P[U\leq u\;|\;Y=y\cap S=1]$

By replacing $U$ with $A-D$, $D$ with $Y$, and $A$ with $\hat{Y}$, the above
is equivalent to:

$P[A-D\leq u\;|\;Y=y\cap S=0]=P[A-D\leq u\;|\;Y=y\cap S=1]$ $P[\hat{Y}-Y\leq
u\;|\;Y=y\cap S=0]=P[\hat{Y}-Y\leq u\;|\;Y=y\cap S=1]$ $P[\hat{Y}\leq
u+y\;|\;Y=y\cap S=0]=P[\hat{Y}\leq u+y\;|\;Y=y\cap S=1]$
$P[\hat{Y}=\hat{y}\;|\;Y=y\cap S=0]=P[\hat{Y}=\hat{y}\;|\;Y=y\cap S=1]$

since $u=a-d=\hat{y}-y$ produces $\hat{y}=u+y$. Then, the last line is
identical to the conditions of equalized odds for binary classification when
$\hat{Y}=1$. ∎

Further, since equalized odds is a combination of false negative error rate
balance and false positive error rate balance, it also aligns with separation.
But, instead of being the relaxed version, equalized odds aligns exactly with
separation.

####  5.2.5 Conditional Use Accuracy

Conditional use accuracy, also termed as predictive value parity, requires
that positive and negative predicted values are similar across different
groups [81]. Statistically, it aligns exactly with the requirement for
sufficiency [13]. Mathematically, it can be written as follows:

$P[Y=y\;|\;\hat{Y}=y\cap S=0]=P[Y=y\;|\;\hat{Y}=y\cap S=1]\;\;\textrm{ for
}\;\;y\in\\{0,1\\}$

In [70], they provide a proof that conditional use accuracy falls into the
Luck-Egalitarian EOP criterion and we recall their proof here:

######  Proposition 5.3 (Conditional Use Accuracy as Luck-Egalitarian EOP
[70]).

Consider the binary classification task where $y\in Y=\\{0,1\\}$. Suppose that
$U=A-D$, $A=Y$, and $D=\hat{Y}$ (i.e., the effort-based utility of an
individual under model a $h$ is assumed to be the same as their predicted
label). Then the conditions of Luck-Egalitarian EOP are equivalent to those of
conditional use accuracy (otherwise known as predictive value parity).

###### Proof.

Recall that Luck-Egalitarian EOP requires that for $s\in S=\\{0,1\\}$,
$\hat{y}\in\hat{Y}=\\{0,1\\}$, and $u\in\\{-1,1\\}$:

$P[U\leq u\;|\;\hat{Y}=\hat{y}\cap S=0]=P[U\leq u\;|\;\hat{Y}=\hat{y}\cap
S=1]$

Replacing $U$ with $A-D$, $D$ with $\hat{Y}$, and $A$ with $Y$, we obtain the
following:

$P[A-D\leq u\;|\;\hat{Y}=\hat{y}\cap S=0]=P[A-D\leq u\;|\;\hat{Y}=\hat{y}\cap
S=1]$ $P[Y-\hat{Y}\leq u\;|\;\hat{Y}=\hat{y}\cap S=0]=P[Y-\hat{Y}\leq
u\;|\;\hat{Y}=\hat{y}\cap S=1]$ $P[Y\leq u+\hat{y}\;|\;\hat{Y}=\hat{y}\cap
S=0]=P[Y\leq u+\hat{y}\;|\;\hat{Y}=\hat{y}\cap S=1]$
$P[Y=y\;|\;\hat{Y}=\hat{y}\cap S=0]=P[Y=y\;|\;\hat{Y}=\hat{y}\cap S=1]$

since $u=a-d=y-\hat{y}$ produces the result that $y=u+\hat{y}$. The last line
is then equal to the statement for conditional use accuracy when
$y=\hat{y}=\\{0,1\\}$. ∎

####  5.2.6 Overall Accuracy Equality

As the name implies, overall accuracy equality requires similar prediction
accuracy across different groups. In this case, we are assuming that obtaining
a true negative is as desirable as obtaining a true positive [81]. According
to [13], it matches exactly with the statistical measure of independence.
Mathematically, it can be written as:

$P[\hat{Y}=y\;|\;Y=y\cap S=0]=P[\hat{Y}=y\;|\;Y=y\cap S=1]\;\;\textrm{ for
}\;\;y\in\\{0,1\\}$

Overall accuracy equality is the third fairness metric that Heidari et al.
proves belongs to the Rawls’ EOP category of fairness metrics [70]. We recall
their proof here:

######  Proposition 5.4 (Overall Accuracy Equality as Rawls’ EOP [70]).

Consider the binary classification task where $Y,\hat{Y}=\\{0,1\\}$. Suppose
$U=A-D$ where $A=(\hat{Y}-Y)^{2}$ and $D=0$ (i.e., effort-based utility of all
individuals are assumed to be the same and equal to 0). Then the conditions of
Rawl’s EOP is equivalent to overall accuracy equality.

###### Proof.

Recall that Rawls’ EOP requires that for $s\in S=\\{0,1\\}$, $y\in
Y=\\{0,1\\}$, and $u\in\\{0,1\\}$:

$P[U\leq u\;|\;Y=y\cap S=0]=P[U\leq u\;|\;Y=y\cap S=1]$

By replacing $U$ with $A-D$, $D$ with $0$, and $A$ with $(\hat{Y}-Y)^{2}$, the
above is equivalent to:

$P[A-D\leq u\;|\;Y=y\cap S=0]=P[A-D\leq u\;|\;Y=y\cap S=1]$
$P[(\hat{Y}-Y)^{2}\leq u\;|\;Y=y\cap S=0]=P[(\hat{Y}-Y)^{2}\leq u\;|\;Y=y\cap
S=1]$ $P[\hat{Y}^{2}-2\hat{Y}Y+Y^{2}\leq u\;|\;Y=y\cap
S=0]=P[\hat{Y}^{2}-2\hat{Y}Y+Y^{2}\leq u\;|\;Y=y\cap S=1]$ $P[\hat{Y}^{2}\leq
u+2\hat{y}y-y^{2}\;|\;Y=y\cap S=0]=P[\hat{Y}^{2}\leq
u+2\hat{y}y-y^{2}\;|\;Y=y\cap S=1]$ $P[\hat{Y}^{2}=\hat{y}^{2}\;|\;Y=y\cap
S=0]=P[\hat{Y}^{2}=\hat{y}^{2}\;|\;Y=y\cap S=1]$
$P[\hat{Y}=\hat{y}\;|\;Y=y\cap S=0]=P[\hat{Y}=\hat{y}\;|\;Y=y\cap S=1]$

since $u=\hat{y}^{2}-2\hat{y}y+y^{2}$ and $\hat{y}^{2}=u+2\hat{y}y-y^{2}$.
This is equal to the statement for overall accuracy equality when
$Y=\hat{Y}=\\{0,1\\}$. ∎

####  5.2.7 Treatment Equality

Treatment equality analyzes fairness by looking at how many errors were
obtained rather than through the lens of accuracy. It requires an equal ratio
of false negative and false positive values for all groups [81]. Further, it
agrees exactly with the statistical measure of separation [13].

$\frac{FN_{S=0}}{FP_{S=0}}=\frac{FN_{S=1}}{FP_{S=1}}$

Treatment equality can be considered as the ratio of false positive error rate
balance and false negative error rate balance. Since both of these metrics
fall into the Rawls’ EOP category, treatment equality does as well.

###  5.3 Predicted Probability and Actual Outcome

The predicted probability and actual outcome category of fairness metrics is
similar to the above category of metrics that use the predicted and actual
outcomes. But, instead of using the predictions themselves, this category uses
the probability of being predicted to a certain class. This category of
metrics includes: test fairness, well calibration, balance for the positive
class, and balance for the negative class. The first two metrics fall in line
with the statistical measure of sufficiency while the last two align with the
relaxed notion of separation.

####  5.3.1 Test Fairness

Test fairness is satisfied if, for any predicted probability score
$p\in\mathcal{P}$, subjects in both the marginalized and non-marginalized
groups have equal probability of actually belonging to the positive class.
Test fairness has also been referenced by the terms calibration, equal
calibration, and matching conditional frequencies [79]. Mathematically, it can
be written as follows:

$P[Y=1\;|\;\mathcal{P}=p\cap S=0]=P[Y=1\;|\;\mathcal{P}=p\cap S=1]$

Test fairness falls under the luck-egalitarian EOP category. To see why this
is the case, we recall the last line of the proof for conditional use
accuracy:

$P[Y=y\;|\;S=0\cap\hat{Y}=\hat{y}]=P[Y=y\;|\;S=1\cap\hat{Y}=\hat{y}]$

By letting $Y=1$ and replacing the predicted class $\hat{y}\in\hat{Y}$ with
the probability to be assigned to a class $p\in\mathcal{P}$, we receive the
following:

$P[Y=1\;|\;S=0\cap\mathcal{P}=p]=P[Y=1\;|\;S=1\cap\mathcal{P}=p]$

which is the same as the condition for test fairness.

####  5.3.2 Well Calibration

Well calibration is very similar to the metric of test fairness, but it goes
one step further and it requires that for any predicted probability score
$p\in\mathcal{P}$, not only should the majority and minority classes have
equal probability of belonging to the positive class, but this probability
should be $p$ [19].

$P[Y=1\;|\;\mathcal{P}=p\cap S=0]=P[Y=1\;|\;\mathcal{P}=p\cap S=1]=p$

Since well calibration is an extension of test fairness, it in turn also falls
under the luck-egalitarian EOP category.

####  5.3.3 Balance for the Positive Class

As the name suggests, the balance for the positive class metric requires that
individuals who experience a positive outcome, regardless of which group they
belong to, should have an equal mean predicted probability of being in the
positive class [19]. It can be seen as being similar to the metric of equal
opportunity, which says that a classifier should give equivalent treatment to
all groups.

$\mathbb{E}[\mathcal{P}\;|\;Y=1\cap S=0]=\mathbb{E}[\mathcal{P}\;|\;Y=1\cap
S=1]$

Like false positive error rate balance, the balance for the positive class
metric can be seen as a derivative of the equalized odds metric when $Y=1$.
Additionally, instead of taking into account the predicted label
$\hat{y}\in\hat{Y}$, it concerns itself with the predicted probability
$\mathcal{P}$. Since equalized odds falls into Rawls’ EOP category of metrics,
the balance for the positive class metric does as well.

####  5.3.4 Balance for the Negative Class

The metric of balance for the negative class is opposite of the balance for
the positive class metric. Instead of requiring balance in the predictive mean
of the positive class, it requires balance in the predicted mean of the
negative class [19]. It is similar to the measure of false positive error rate
balance.

$\mathbb{E}[\mathcal{P}\;|\;Y=0\cap S=0]=\mathbb{E}[\mathcal{P}\;|\;Y=0\cap
S=1]$

Same as the argument for balance for the positive class, the balance for the
negative class metric is a derivative of equalized odds when $Y=0$ and we
approximate $\hat{y}\in\hat{Y}$ with probability score $\mathcal{P}$.
Therefore, the balance for the negative class metric also falls under Rawls’
EOP category.

Table 4: Definitions and classifications for popular fair machine learning
classification metrics. IND = Independent, SUF = Sufficiency, SEP =
Separation, R = Relaxed, E = Equivalent, LE = Luck-Egalitarian

Metric | Definition | Formula | Closest | R/E | Rawls’ / LE
---|---|---|---|---|---
|  |  |  |  |
1\. Predicted Outcome
| Group Fairness
---
(Statistical Parity)
| All groups have equal probability of being
---
assigned to the positive class.
$P[\hat{Y}=1\;|\;S=0]=P[\hat{Y}=1\;|\;S=1]$ | IND | E | Rawls’
– Disparate Impact |  | Proportion of positive predictions
---
of all groups must be similar.
$\frac{P[\hat{Y}=1\;|\;S=0]}{P[\hat{Y}=1\;|\;S=1]}\geq 1-\epsilon$ | IND | R | Rawls’
Conditional Statistical Parity |  | Requires statistics for all groups to
---
be equal, allowing for a set of legitimate
factors $L=\ell$.
$P[\hat{Y}=1\;|\;L=\ell\cap S=0]=P[\hat{Y}=1\;|\;L=\ell\cap S=1]$ | IND | R | Rawls’
|  |  |  |  |
2\. Predicted and Actual Outcome
Predictive Parity |  | Similar positive predictive values
---
(or FDR) across groups.
| PPV: $P[Y=1\;|\;\hat{Y}=1\cap S=0]=P[Y=1\;|\;\hat{Y}=1\cap S=1]$
---
FDR:$P[Y=0\;|\;\hat{Y}=1\cap S=0]=P[Y=0\;|\;\hat{Y}=1\cap S=1]$
SUF | R | LE
False Positive Error Rate Balance |  | Similar false positive rates
---
(or TNR) across groups.
| FPR:$P[\hat{Y}=1\;|\;Y=0\cap S=0]=P[\hat{Y}=1\;|\;Y=0\cap S=1]$
---
TNR:$P[\hat{Y}=0\;|\;Y=0\cap S=0]=P[\hat{Y}=0\;|\;Y=0\cap S=1]$
SEP | R | Rawls’
False Negative Error Rate Balance |  | Similar false negative rates
---
(or TPR) across groups.
| FNR:$P[\hat{Y}=0\;|\;Y=1\cap S=0]=P[\hat{Y}=0\;|\;Y=1\cap S=1]$
---
TPR:$P[\hat{Y}=1\;|\;Y=1\cap S=0]=P[\hat{Y}=1\;|\;Y=1\cap S=1]$
SEP | R | Rawls’
Equalized Odds |  | Similar false positive and true positive
---
rates across groups.
| $P[\hat{Y}=1\;|\;Y=y\cap S=0]=P[\hat{Y}=1\;|\;Y=y\cap S=1]$
---
for $y\in\\{0,1\\}$
SEP | E | Rawls’
Conditional Use Accuracy |  | Similar positive and negative predictive
---
values across groups.
| $P[Y=y\;|\;\hat{Y}=y\cap S=0]=P[Y=y\;|\;\hat{Y}=y\cap S=1]$
---
for $y\in\\{0,1\\}$
SUF | E | LE
Overall Accuracy Equality | Requires similar accuracy across groups. |  | $P[\hat{Y}=y\;|\;Y=y\cap S=0]=P[\hat{Y}=y\;|\;Y=y\cap S=1]$
---
for $y\in\\{0,1\\}$
IND | E | Rawls’
Treatment Equality |  | Equal ratio of false negatives and false
---
positive between groups.
$\frac{FN_{S=0}}{FP_{S=0}}=\frac{FN_{S=1}}{FP_{S=1}}$ | SEP | E | Rawls’
|  |  |  |  |
3\. Predicted Probabilities and Actual Outcomes
Test Fairness |  | All groups have equal probability to belong to
---
the positive class.
| $P[Y=1\;|\;\mathcal{P}=p\cap S=0]=P[Y=1\;|\;\mathcal{P}=p\cap S=1]$
---
for any predicted probability score $p\in\mathcal{P}$
SUF | R | LE
Well Calibration |  | The probability of all groups to belong to the
---
positive class is the predicted probability
score $p\in\mathcal{P}$ .
$P[Y=1\;|\;\mathcal{P}=p\cap S=0]=P[Y=1\;|\;\mathcal{P}=p\cap S=1]=p$ | SUF | E | LE
Balance for Positive Class |  | Equal mean predicted probabilities for all people
---
in the positive class, regardless of group.
$\mathbb{E}[\mathcal{P}\;|\;Y=1\cap S=0]=\mathbb{E}[\mathcal{P}\;|\;Y=1\cap S=1]$ | SEP | R | Rawls’
Balance for Negative Class |  | Equal mean predicted probabilities for all subjects
---
in the negative class, regardless of group.
$\mathbb{E}[\mathcal{P}\;|\;Y=0\cap S=0]=\mathbb{E}[\mathcal{P}\;|\;Y=0\cap S=1]$ | SEP | R | Rawls’

###  5.4 Individual Fairness Notions

Up until this point, the metrics we have discussed all focus on the notion of
group fairness. In other words, these metrics ensure some kind of statistical
parity for members of different groups and not a specific individual [82].
Another set of fairness metrics that consider the fairness as it relates to a
specific individual is called individual fairness. Individual fairness ensures
that people who are similar in the eyes of the classification task are treated
similarly (i.e., obtain the same prediction) [82]. In this section, we recount
the first (and most famous) notion of individual fairness - fairness through
awareness. We note that other individual fairness metrics exist, such as [83,
84, 85, 86], and we direct interested readers to these publications, as well
as a survey over them [82], for more detail.

####  5.4.1 Fairness through Awareness

Fairness through awareness, most commonly called individual fairness, was
first proposed by Dwork et al. in 2012 [60]. The motivation in creating
fairness through awareness was that there was concerns that simply using
statistical parity between different groups could result in unfair outcomes at
the individual level. To solve this issue, Dwork et al. proposed to use a
distance metric that measured how similar an individual was to another. Two
individuals were considered alike if their combination of task-relevant
attributes were nearby each other, and the overall process was deemed fair if
the two individuals (who were alike) received the same outcome from the model
[82]. This process can be seen as being similar to the legal practice of
situation testing. Situation testing is an experimental method that aims to
establish discrimination on the spot [87]. It takes pairs of individuals who
are similar, but do not necessarily belong to the same group, and puts them in
the same situation. If the individual who is part of the marginalized group is
treated differently than the individual in the non-marginalized group, then
there is a viable case of discrimination that can be brought to court. Several
research works [88, 89] studied the use of kNN and causal Bayesian networks to
facilitate the similarity measurements in situation testing based
discrimination detection. Additionally, fairness through awareness aligns with
Aristotle’s conception of “justice as consistency” [82, 90].

A downfall of this metric is that it does not allow for comparison of all
individuals since it only compares similar individuals. So in the hiring
example, the applicants who have similar background experiences can be
compared to each other, but they cannot be compared to those who have
different prior work experience. This makes it impossible to construct a total
ranking of all the candidates. Additionally, fairness through awareness can be
difficult to implement as it requires explicitly defining what similarity
means in a certain context and what is considered similar in one case may not
be considered similar in another. Further, specifically for fairness through
awareness, it requires the defining of a distance metric by the people who set
the policy, which is not a simple task to do [82].

####  5.4.2 Group Fairness vs. Individual Fairness

Many technical research papers assume that both group and individual fairness
are important, but conflicting, measures [85, 86]. But, Binns argues that this
conflict is based on a misconception, and when we look at the philosophical
underpinnings of group and individual fairness, they are not actually trying
to achieve different things [82]. Specifically, in [82] he wrote:

> “… while there may be natural affinities between the usual formulations of
> individual fairness and [Aristotle’s] consistency, and the usual motivations
> for group fairness and egalitarianism, these are only surface deep.
> Consistency and egalitarianism themselves do not conflict at the level of
> principle. In fact they can even be seen as mutually implied in so far as
> luck egalitarianism aims to remove luck from allocation, it implies
> consistency. And in so far as the process for defining task-relevant
> similarity can already be an exercise in normative judgement, it makes sense
> to incorporate egalitarian concerns into it. It is therefore both possible
> and indeed coherent to adopt consistency-respecting formulations of group
> fairness, as well as egalitarian formulations of individual fairness.”

All this being to say that the philosophical ideas of individual and group
fairness are actually the same. Binns goes on to reiterate that: “the
appearance of conflict between the two is an artifact of the failure to fully
articulate assumptions behind them, and the reasons for applying them in a
particular context” [82].

##  6 Causal-Based Fairness Notions

In Section 5, we presented fairness notions in machine learning that are based
on various statistical discrimination criteria. These criteria are
observational as they depend only on the joint distribution of predictor
(algorithm), marginalization attribute, features, and the final outcome.
Observational criteria have severe inherent limitations that prevent them from
resolving matters of fairness in a conclusive manner, mainly because fairness
cannot be well assessed based only on mere correlation or association (rung
one of Pearl’s Ladder of Causation). Additionally, statistics-based fairness
metrics do not align well with the legal process of discrimination handling
since discrimination claims usually require plaintiffs to demonstrate a causal
connection between the challenged decision and the sensitive feature. Most
recent fairness notions are causal-based and reflect the now widely accepted
idea that using causality is necessary to appropriately address the problem of
fairness. In this section, we present fairness in the language of causal
reasoning. Causal-based fairness notions differ from the previously explained
ones in that they are not totally based on data888“Data is profoundly dumb.
Data can tell you that people who took a medicine recovered faster than those
who did not take it, but they can’t tell you why.” - Judea Pearl [25], but
consider additional knowledge about the structure of the world, in the form of
a causal model.

Causal-based fairness notions are developed mainly under the two causal
frameworks: the structural causal model (SCMs) and the potential outcome. SCMs
assume that we know the complete causal graph, and hence, we are able to study
the causal effect of any variable along many different paths. The potential
outcome framework does not assume the availability of the causal graph and
instead focuses on estimating the causal effects of treatment variables. In
Table 5, we present the causal framework to which each causal-based fairness
notion discussed in this section belongs.

In [91], Pearl presented the causal hierarchy through the Ladder of Causation,
as shown in Fig. 3. The Ladder of Causation has the 3 rungs: association,
intervention, and counterfactual. On the first rung, associations can be
inferred directly from the observed data using conditional probabilities and
conditional expectations. The intervention rung involves not only seeing what
is, but also changing what we see. Interventional questions deal with
$P(y|do(x),z)$ which stands for “the probability of $Y=y$, given that we
intervene and set the values of $X$ to $x$ and subsequently observe event
$Z=z$.” Interventional questions cannot be answered from purely observational
data alone. They can be estimated experimentally from randomized trials or
analytically using causal Bayesian networks. The top rung invokes
counterfactuals and deals with $P(y_{x}|x^{\prime},y^{\prime})$ which stands
for “the probability that event $Y=y$ would be observed had $X$ been $x$,
given that we actually observed $X$ to be $x^{\prime}$ and $Y$ to be
$y^{\prime}$.” Such questions can be computed only when the model is based on
functional relations or is structural. In Table 5, we also show the causal
hierarchical level that each causal-based fairness notion aligns with.

Table 5: Classification of causal-based fairness notions. SCM = structure
causal model, PO = potential outcome. The last column describes whether the
fairness notion involves both $Y$ and $\hat{Y}$ in their counterfactual
quantity.

Notion | Association | SCM | PO | Intervention | Counterfactual |  $Y$ and $\hat{Y}$
---|---|---|---|---|---|---
Total Variation | $\checkmark$ |  |  |  |  |
Total Causal Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Natural Direct Effect |  | $\checkmark$ |  | $\checkmark$ |  |
Natural Indirect Effect |  | $\checkmark$ |  | $\checkmark$ |  |
Path-specific Causal Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Direct Causal Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Indirect Causal Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Counterfactual Fairness |  | $\checkmark$ |  |  | $\checkmark$ |
Counterfactual Direct Effect |  | $\checkmark$ |  |  | $\checkmark$ |
Counterfactual Indirect Effect |  | $\checkmark$ |  |  | $\checkmark$ |
Path-specific Counterfactual Fairness |  | $\checkmark$ |  |  | $\checkmark$ |
Proxy Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Justifiable Fairness |  | $\checkmark$ |  | $\checkmark$ |  |
Counterfactual Direct Error Rate |  | $\checkmark$ |  |  | $\checkmark$ | $\checkmark$
Counterfactual Indirect Error Rate |  | $\checkmark$ |  |  | $\checkmark$ | $\checkmark$
Individual Equalized Counterfactual Odds |  | $\checkmark$ |  |  | $\checkmark$ | $\checkmark$
Fair on Average Causal Effect |  |  | $\checkmark$ | $\checkmark$ |  |
Fair on Average Causal Effect on the Treated |  |  | $\checkmark$ |  | $\checkmark$ |
Equal Effort Fairness |  |  | $\checkmark$ |  | $\checkmark$ |

In the context of fair machine learning, we use $S\in\\{s^{+},s^{-}\\}$ to
denote the marginalization attribute, $Y\in\\{y^{+},y^{-}\\}$ to denote the
decision, and $\mathbf{X}$ to denote a set of non-marginalization attributes.
The underlying mechanism of the population over the space
$S\times\mathbf{X}\times Y$ is represented by a causal model $\mathcal{M}$,
which is associated with a causal graph $\mathcal{G}$. Fig. 8 shows a causal
graph that will be used to illustrate fairness notions throughout this
section. With $\mathcal{M}$, we want to reason about counterfactual queries,
e.g., “what would the prediction have been for this individual if their
marginalization attribute value changed?” A historical dataset $\mathcal{D}$
is drawn from the population, which is used to construct a predictor
$h:\mathbf{X},S\rightarrow\hat{Y}$. Note that the input of the predictor can
be a subset of $\mathbf{X},S$ and we use $\widehat{PA}$ to denote the set of
input features of the predictor when introducing counterfactual error rate in
Section 6.8. The causal model for the population over space
$S\times\mathbf{X}\times\hat{Y}$ can be considered the same as $\mathcal{M}$,
except that function $f_{Y}$ is replaced with a predictor $h$. Most fairness
notions involve either $Y$ or $\hat{Y}$ in their counterfactual quantity and,
roughly speaking, they correspond to statistical parity. A few fairness
notions, e.g., counterfactual direct error rate [92], correspond to the
concept of equalized odds and involve both $Y$ and $\hat{Y}$ in their
counterfactual quantity. We also mark them in Table 5.

For each fairness notion, we have two versions: strict and relaxed. The strict
version means there is absolutely no discrimination effect, whereas the
relaxed version often compares the causal effect with $\tau$, a use-defined
threshold for discrimination. In our discussions, we adhere to the strict
version when introducing each fairness notion.

Since the majority of causal-based fairness notions are defined in terms of
the non-observable quantities of interventions and counterfactuals, their
applicability depends heavily on the identifiability of those quantities from
observational data. We refer readers who are interested in learning the
specifics of identifiability theory and criteria, and how they can be used to
decide the applicability of causal-based fairness metrics to [93]. In this
section, we simply present causal-based fairness notions and discuss their
relationships.

Figure 8: Causal graph of the college admission example used throughout this
section.

###  6.1 Total, Natural Direct, and Natural Indirect Causal Fairness

Discrimination can be viewed as the causal effect of $S$ on $Y$. Total causal
fairness answers the question of if the marginalization attribute $S$ changed
(e.g., changing from marginalized group $s^{-}$ to non-marginalized group
$s^{+}$), how would the outcome $Y$ change on average? A straightforward
strategy to answer this question is to measure the average causal effect of
$S$ on $Y$ when $S$ changes from $s^{-}$ to $s^{+}$, an approach called total
causal fairness.

######  Definition 6.1 (Total Causal Fairness).

Given the marginalization attribute $S$ and decision $Y$, we achieve total
causal fairness if

$\mathrm{TCE}(s_{1},s_{0})=P(y_{s_{1}})-P(y_{s_{0}})=0$

where $s_{1},s_{0}\in\\{s^{+},s^{-}\\}$.

In fact, the causal effect of $S$ on $Y$ includes the direct discriminatory
effect, the indirect discriminatory effect, and the explainable effect. In
[26], Pearl proposed the use of NDE and NIE to measure the direct and indirect
discrimination. Recall that
$\mathrm{NDE}(s_{1},s_{0})=P(y_{s_{1},\mathbf{Z}_{s_{0}}})-P(y_{s_{0}})$ and
$\mathrm{NIE}(s_{1},s_{0})=P(y_{s_{0},\mathbf{Z}_{s_{1}}})-P(y_{s_{0}})$ where
$\mathbf{Z}$ is the set of mediator variables. When applied to the example in
Figure 8, the mediator variable could be the major.
$P(y_{s_{1},\mathbf{Z}_{s_{0}}})$ in NDE is the probability of $Y=y$ had $S$
been $s_{1}$ and had $\mathbf{Z}$ been the value it would naturally take if
$S=s_{0}$. Similarly, NIE measures the indirect effect of $S$ on $Y$. However,
NIE does not distinguish between explainable and indirect discrimination.

###  6.2 Path-Specific Causal Fairness

In [94], Zhang et al. introduced path-specific causal fairness based on path-
specific causal effect [21]. Different from total, natural direct, and natural
indirect causal effects, the path-specific causal effect is based on graph
properties of the causal graph, and characterizes the causal effect in term of
specific paths.

######  Definition 6.2 (Path-Specific Causal Fairness).

Given the marginalization attribute $S$, decision $Y$, and redlining
attributes $\mathbf{R}$, define $\pi_{d}$ as the path set that contains some
paths from $S$ to $Y$. We achieve path-specific causal fairness if

$\mathrm{PE}_{\pi}(s_{1},s_{0})=P(y_{s_{1}|\pi,s_{0}|\bar{\pi}})-P(s_{x_{0}})=0$

where $s_{1},s_{0}\in\\{s^{+},s^{-}\\}$. Specifically, define $\pi_{d}$ as the
path set that contains only $S\rightarrow Y$ and define $\pi_{i}$ as the path
set that contains all the causal paths from $S$ to $Y$ which pass through some
redlining attributes of $\mathbf{R}$. We achieve direct causal fairness if
$\mathrm{PE}_{\pi_{d}}(s_{1},s_{0})=0$, and indirect causal fairness if
$\mathrm{PE}_{\pi_{i}}(s_{1},s_{0})=0$.

Direct discrimination considers the causal effect transmitted along the direct
path from $S$ to $Y$, i.e., $S\rightarrow Y$. The physical meaning of
$\mathit{PE}_{\pi_{d}}(s^{+},s^{-})$ can be explained as the expected change
in decisions of individuals from marginalized group $s^{-}$, if the decision
makers are told that these individuals were from the non-marginalized group
$s^{+}$. When applied to the example in Fig. 8, it means that the expected
change in admission of applications is actually from the marginalized group
(e.g., female), when the admission office is instructed to treat the
applicants as from the non-marginalized group (e.g., male).

Indirect discrimination considers the causal effect transmitted along all the
indirect paths from $S$ to $Y$ that contain the redlining attributes. The
physical meaning of $\mathit{PE}_{\pi_{i}}(s^{+},s^{-})$ is the expected
change in decisions of individuals from marginalized group $s^{-}$, if the
values of the redlining attributes in the profiles of these individuals were
changed as if they were from the non-marginalized group $s^{+}$. When applied
to the example in Fig. 8, it means the expected change in admission of the
marginalized group if they had the same gender makeups shown in the major as
the non-marginalized group.

The following propositions [94] further show two properties of the path-
specific effect metrics.

######  Proposition 6.1.

If path set $\pi$ contains all causal paths from $S$ to $Y$ and $S$ has no
parent in $\mathcal{G}$, then we have

$\mathrm{PE}_{\pi}(s_{1},s_{0})=\mathrm{TE}(s_{1},s_{0})=P(y^{+}|s_{1})-P(y^{+}|s_{0}).$

$P(y^{+}|s^{+})-P(y^{+}|s^{-})$ is known as the _risk difference_. Therefore,
the path-specific effect metrics can be considered as an extension to the risk
difference for explicitly distinguishing the discriminatory effects of direct
and indirect discrimination from the total causal effect.

######  Proposition 6.2.

For any path sets $\pi_{d}$ and $\pi_{i}$, we do not necessarily have

$\mathrm{PE}_{\pi_{d}}(s_{1},s_{0})+\mathrm{PE}_{\pi_{i}}(s_{1},s_{0})=\mathrm{PE}_{\pi_{d}\cup\pi_{i}}(s_{1},s_{0}).$

This implies that there might not be a linear connection between direct and
indirect discrimination.

###  6.3 Counterfactual Fairness

In Section 6.1 and 6.2, the intervention is performed on the whole population.
These metrics deal with effects on an entire population, or on the average
individual from a population. But, up to this point we are still missing the
ability to talk about “personalized causation" at the level of particular
events of individuals [25]. Counterfactuals provide such a tool. If we infer
the post-intervention distribution while conditioning on certain individuals,
or groups specified by a subset of observed variables, the inferred quantity
will involve two worlds simultaneously: the real world represented by causal
model $\mathcal{M},$ as well as the counterfactual world $\mathcal{M}_{x}$.
Such causal inference problems are called counterfactual inference, and the
distribution of $Y_{x}$ conditioning on the real world observation
$\mathbf{O}=\mathbf{o}$ is denoted by $P(y_{x}|\mathbf{o})$.

In [95], Kusner et al. defined counterfactual fairness to be the case where
the outcome would have remained the same had the marginalization attribute of
an individual or a group been different, and all other attributes been equal.

######  Definition 6.3 (Counterfactual Fairness).

Given a factual condition $\mathbf{O}=\mathbf{o}$ where
$\mathbf{O}\subseteq\\{S,\mathbf{X},Y\\}$, we achieve counterfactual fairness
if

$\mathrm{CE}(s_{1},s_{0}|\mathbf{o})=P(y_{s_{1}}|\mathbf{o})-P(y_{s_{0}}|\mathbf{o})=0$

where $s_{1},s_{0}\in\\{s^{+},s^{-}\\}$.

Note that we can simply define a classifier as counterfactually fair by
replacing outcome $Y$ with the predictor $\hat{Y}$ in the above equation. The
meaning of counterfactual fairness can be interpreted as follows when applied
to the example in Figure 8. Applicants are applying for admission and a
predictive model is used to make the decision $\hat{Y}$. We concern ourselves
with an individual from marginalized group $s^{-}$ who is specified by a
profile $\mathbf{o}$. The probability of the individual to get a positive
decision is $P(\hat{y}|s^{-},\mathbf{o})$, which is equivalent to
$P(\hat{y}_{s^{-}}|s^{-},\mathbf{o})$ since the intervention makes no change
to $S$’s value of that individual. Now assume the value of $S$ for the
individual had been changed from $s^{-}$ to $s^{+}$. The probability of the
individual to get a positive decision after the hypothetical change is given
by $P(\hat{y}_{s^{+}}|s^{-},\mathbf{o})$. Therefore, if the two probabilities
$P(\hat{y}_{s^{-}}|s^{-},\mathbf{o})$ and
$P(\hat{y}_{s^{+}}|s^{-},\mathbf{o})$ are identical, we can claim the
individual is treated fairly as if they had been from the other group.

###  6.4 Counterfactual Effects

In [96], Zhang and Bareinboim introduced three fine-grained measures of the
transmission of change from stimulus to effect called the counterfactual
direct, indirect, and spurious effects. Throughout Section 6.4, we use
$\mathbf{W}$ to denote all the observed intermediate variables between $S$ and
$Y$ and use the group with $S=s_{0}$ as the baseline to measure changes of the
outcome.

######  Definition 6.4 (Counterfactual Direct Effect).

Given a SCM, the counterfactual direct effect (Ctf-DE) of intervention
$S=s_{1}$ on $Y$ (with baseline $s_{0}$) conditioned on $S=s$ is defined as

$\textrm{Ctf-
DE}_{s_{0},s_{1}}(y|s)=P(y_{s_{1},\mathbf{W_{s_{0}}}}|s)-P(y_{s_{0}}|s).$

$Y_{s_{1},\mathbf{W}_{s_{0}}}=y|S=s$ is a more involved counterfactual
compared to NDE and can be read as “the value of $Y$ would be had $S$ been
$s_{1}$, while $\mathbf{W}$ is kept at the same value that it would have
attained had $S$ been $s_{0}$, given that $S$ was actually equal to $s$.”

######  Definition 6.5 (Counterfactual Indirect Effect).

Given a SCM, the counterfactual indirect effect (Ctf-IE) of intervention
$S=s_{1}$ on $Y$ (with baseline $s_{0}$) conditioned on $S=s$ is defined as

$\textrm{Ctf-
IE}_{s_{0},s_{1}}(y|s)=P(y_{s_{0},\mathbf{W}_{s_{1}}}|s)-P(y_{s_{0}}|s).$

Ctf-IE measures changes in the probability of the outcome $Y$ would be $y$ had
$S$ been $s_{0}$, while changing $\mathbf{W}$ to whatever level it would have
naturally obtained had $S$ been $s_{1}$, in particular, for the individuals
that $S=s_{0}$.

######  Definition 6.6 (Counterfactual Spurious Effect ).

Given a SCM, the counterfactual spurious effect (Ctf-SE) of $S=s_{1}$ on $Y=y$
(with baseline $s_{0}$) is defined as

$\textrm{Ctf-SE}_{s_{0},s_{1}}(y)=P(y_{s_{0}}|s_{1})-P(y|{s_{0}}).$

$\text{Ctf-SE}_{s_{0},s_{1}}(y)$ measures the difference in the outcome $Y=y$
had $S$ been $s_{0}$ for the individuals that would naturally choose $S$ to be
$s_{0}$ versus $s_{1}$.

######  Proposition 6.3.

For a SCM, if $S$ has no direct (indirect) causal path connecting $Y$ in the
causal graph, then $\textrm{Ctf-DE}_{s_{0},s_{1}}(y|s)=0$ ($\textrm{Ctf-
IE}_{s_{0},s_{1}}(y|s)=0$) for any $s$, $y$; if $S$ has no back-door path
connecting $Y$ in the causal graph, then $\textrm{Ctf-SE}_{s_{0},s_{1}}(y)=0$
for any $y$.

Building on these measures, the Zhang and Bareinboim derived the causal
explanation formula for the disparities observed in total variation. Recall
that the total variation is simply the difference between the conditional
distributions of $Y$ when observing $S$ changing from $s_{0}$ to $s_{1}$.

######  Definition 6.7 (Total Variation).

The total variation (TV) of $S=s_{1}$ on $Y=y$ (with baseline $s_{0}$) is
given by

$\mathrm{TV}_{s_{0},s_{1}}(y)=P(y|s_{1})-P(y|s_{0}).$

######  Theorem 6.1 (Causal Explanation Formula).

For any $s_{0}$, $s_{1}$, $y$, the total variation, counterfactual spurious,
direct, and indirect effects obey the following relationship:

$\mathrm{TV}_{s_{0},s_{1}}(y)=\textrm{Ctf-SE}_{s_{0},s_{1}}(y)+\textrm{Ctf-
IE}_{s_{0},s_{1}}(y|s_{1})-\textrm{Ctf-DE}_{s_{1},s_{0}}(y|s_{1}),$
$\mathrm{TV}_{s_{0},s_{1}}(y)=\textrm{Ctf-
DE}_{s_{0},s_{1}}(y|s_{0})-\textrm{Ctf-SE}_{s_{1},s_{0}}(y)-\textrm{Ctf-
IE}_{s_{1},s_{0}}(y|s_{0}).$

Theorem 6.1 allows the machine learning designer to quantitatively evaluate
fairness and explain the total observed disparity of decisions through
different discriminatory mechanisms. For example, the first formula shows that
the total disparity experienced by the individuals who have naturally attained
$s_{1}$ (relative to $s_{0}$) is equal to the disparity associated with
spurious discrimination, plus the advantage it lost due to indirect
discrimination, minus the advantage it would have gained without direct
discrimination.

###  6.5 Path-Specific Counterfactual Fairness

In [29], Wu et al. proposed path-specific counterfactual fairness (PC
fairness) that covers the previously mentioned causality-based fairness
notions. Letting $\Pi$ be all causal paths from $S$ to $Y$ in the causal
graph, the path-specific counterfactual fairness metric is defined as follows.

######  Definition 6.8 (Path-specific Counterfactual Fairness (PC Fairness)).

Given a factual condition $\mathbf{O}=\mathbf{o}$ where
$\mathbf{O}\subseteq\\{S,\mathbf{X},Y\\}$ and a causal path set $\pi$, we
achieve the PC fairness if

$\mathrm{PCE}_{\pi}(s_{1},s_{0}|\mathbf{o})=P(y_{s_{1}|\pi,s_{0}|\bar{\pi}}|\mathbf{o})-P(y_{s_{0}}|\mathbf{o})=0$

where $s_{1},s_{0}\in\\{s^{+},s^{-}\\}$.

Table 6: Connection between path-specific counterfactual fairness (PC Fairness) and other fairness notions. Description | Relating to PC Fairness
---|---
Total Causal Fairness |  $\mathbf{O}=\emptyset$ and $\pi=\Pi$
Direct Causal Fairness |  $\mathbf{O}=\emptyset$ and $\pi=\pi_{d}=\\{S\rightarrow\hat{Y}\\}$
Indirect Causal Fairness |  $\mathbf{O}=\emptyset$ and $\pi=\pi_{i}\subset\Pi$
Counterfactual Fairness |  $\mathbf{O}=\\{S,\mathbf{X}\\}$ and $\pi=\Pi$
Counterfactual Direct Effect (Ctf-DE) |  $\mathbf{O}=\\{S,Y\\}$ and $\pi=\pi_{d}$
Counterfactual Indirect Effect (Ctf-IE) |  $\mathbf{O}=\\{S,Y\\}$ and $\pi_{i}$

We point out that we can simply define the PC Fairness on a classifier by
replacing outcome $Y$ with the predictor $\hat{Y}$ in the above equation.
Previous causality-based fairness notions can be expressed as special cases of
the PC fairness based on the value of $\mathbf{O}$ (e.g., $\emptyset$ or
$S,{\mathbf{X}}$) and the value of $\pi$ (e.g., $\Pi$ or $\pi_{d}$). Their
connections are summarised in Table 6, where $\pi_{d}$ contains the direct
edge from $S$ to $\hat{Y}$, and $\pi_{i}$ is a path set that contains all
causal paths passing through any redlining attributes (i.e., a set of
attributes in $\mathbf{X}$ that cannot be legally justified if used in
decision-making). The notion of PC fairness also resolves new types of
fairness, e.g., individual indirect fairness, which means discrimination along
the indirect paths for a particular individual. Formally, the individual
indirect fairness can be directly defined and analyzed using PC fairness by
letting $\mathbf{O}=\\{S,\mathbf{X}\\}$ and $\pi=\pi_{i}$.

###  6.6 Proxy Fairness

In [97], Kilbertus et al. proposed proxy fairness. A proxy is a descendant of
$S$ in the causal graph whose observable quantity is significantly correlated
with $S$, but should not affect the prediction.

######  Definition 6.9 (Proxy Discrimination).

A predictor $\hat{Y}$ exhibits no proxy discrimination based on a proxy $P$ if
for all $p,p^{\prime}\in\text{Dom}(P,P(\hat{Y}_{p})=P(\hat{Y}_{p^{\prime}}))$.

Note that $P(\hat{Y}_{p})$ is equivalent to $P(\hat{Y}|do(P=p)$. Intuitively,
a predictor satisfies proxy fairness if the distribution of $\hat{Y}$ under
two interventional regimes in which $P$ set to $p$ and $p^{\prime}$ is the
same. The authors presented the conditions and developed procedures to remove
proxy discrimination given the structural equation model.

###  6.7 Justifiable Fairness

In [98], Salimi et al. presented a pre-processing approach for removing the
effect of any discriminatory causal relationship between the marginalization
attribute and classifier predictions by manipulating the training data to be
non-discriminatory. The repaired training data can be seen as a sample from a
hypothetical fair world.

######  Definition 6.10 ($\mathbf{K}$-fair).

For a give set of variables $\mathbf{K}$, a decision function is said to be
$\mathbf{K}$-fair with regarding to $S$ if, for any context
$\mathbf{K}=\mathbf{k}$ and any outcome $Y=y$,
$P(y_{s_{0},\mathbf{k}})=P(y_{s_{1},\mathbf{k}})$.

Note that the notion of $\mathbf{K}$-fair intervenes on both the sensitive
attribute $S$ and marginalization attributes $\mathbf{K}$. It is more fine-
grained than proxy fairness, but it does not attempt to capture fairness at
the individual level. The authors further introduced justifiable fairness for
applications where the user can specify admissible variables through which it
is permissible for the marginalization attribute to influence the outcome.

######  Definition 6.11 (Justifiable Fairness).

A fairness application is justifiable fair if it is $\mathbf{K}$-fair with
regarding to all supersets $\mathbf{K}\supseteq\mathbf{A}$ where $\mathbf{A}$
is the set of admissible variables.

Different from previous causal-based fairness notions, which require the
presence of the underlying causal model, the justifiable fairness notion is
based solely on the notion of intervention. The user only requires
specification of a set of admissible variables and does not need to have a
causal graph. The authors also introduced a sufficient condition for testing
justifiable fairness that does not require access to the causal graph.
However, with the presence of the causal graph, if all directed paths from $S$
to $Y$ go through an admissible attribute in $\mathbf{A}$, then the algorithm
is justifiably fair. If the probability distribution is faithful to the causal
graph, the converse also holds.

###  6.8 Counterfactual Error Rate

Zhang and Bareinboim [92] developed a causal framework to link the disparities
realized through equalized odds (EO) and the causal mechanisms by which the
marginalization attribute $S$ affects change in the prediction $\hat{Y}$. EO,
also referred to as error rate balance, considers both the ground truth
outcome $Y$ and predicted outcome $\hat{Y}$. EO achieves fairness through the
balance of the misclassification rates (false positive and negative) across
different demographic groups. They introduced a family of counterfactual
measures that allows one to explain the misclassification disparities in terms
of the direct, indirect, and spurious paths from $S$ to $\hat{Y}$ on a
structural causal model. Different from all previously discussed causal-based
fairness notions, counterfactual error rate considers both $Y$ and $\hat{Y}$
in their counterfactual quantity.

######  Definition 6.12 (Counterfactual Direct Error Rate).

Given a SCM and a classifier $\hat{y}=f(\widehat{pa})$ where $\widehat{PA}$ is
a set of input features of the predictor, the counterfactual direct error rate
($\mathrm{ER}^{d}$) for a sub-population $s,y$ (with prediction $\hat{y}\neq
y)$ is defined as

$\mathrm{ER}^{d}_{s_{0},s_{1}}(\hat{y}|s,y)=P(\hat{y}_{s_{1},y,(\widehat{PA}\backslash
S)_{s_{0},y}}|s,y)-P(\hat{y}_{s_{0},y}|s,y).$

For an individual with the marginalization attribute $S=s$ and the true
outcome $Y=y$, the counterfactual direct error rate calculates the difference
of two terms. The first term is the prediction $\hat{Y}$ had $S$ been $s_{1}$,
while keeping all the other features $\widehat{PA}\backslash S$ at the level
that they would attain had $S=s_{0}$ and $Y=y$, whereas the second term is the
prediction $\hat{Y}$ the individual would receive had $S$ been $s_{0}$ and $Y$
been $y$.

######  Definition 6.13 (Counterfactual Indirect Error Rate).

Given a SCM and a classifier $\hat{y}=f(\widehat{pa})$, the counterfactual
indirect error rate ($\mathrm{ER}^{i}$) for a sub-population $s,y$ (with
prediction $\hat{y}\neq y)$ is defined as

$\mathrm{ER}^{i}_{s_{0},s_{1}}(\hat{y}|s,y)=P(\hat{y}_{s_{0},y,(\hat{PA}\backslash
S)_{s_{1},y}}|s,y)-P(\hat{y}_{s_{0},y}|s,y).$

######  Definition 6.14 (Counterfactual Spurious Error Rate).

Given a SCM and a classifier $\hat{y}=f(\hat{pa})$, the counterfactual
spurious error rate ($\mathrm{ER}^{s}$) for a sub-population $s,y$ (with
prediction $\hat{y}\neq y)$ is defined as

$\mathrm{ER}^{s}_{s_{0},s_{1}}(\hat{y}|y)=P(\hat{y}_{s_{0},y}|s_{1},y)-P(\hat{y}_{s_{0},y}|s_{0},y).$

The counterfactual spurious error rate can be read as “for two demographics
$s_{0}$, $s_{1}$ with the same true outcome $Y=y$, how would the prediction
$\hat{Y}$ differ had they both been $s_{0}$, $y$?”

Building on these measures, Zhang and Bareinboim [92] derived the causal
explanation formula for the error rate balance. The equalized odds notion
constrains the classification algorithm such that its disparate error rate is
equal to zero across different demographics.

######  Definition 6.15 (Error Rate Balance).

The error rate (ER) balance is given by

$\mathrm{ER}_{s_{0},s_{1}}(\hat{y}|y)=P(\hat{y}|s_{1},y)-P(\hat{y}|s_{0},y).$

######  Theorem 6.2 (Causal Explanation Formula of Equalized Odds).

For any $s_{0}$, $s_{1}$, $\hat{y}$, $y$, we have the following relationship:

$\mathrm{ER}_{s_{0},s_{1}}(\hat{y}|y)=\mathrm{ER}^{d}_{s_{0},s_{1}}(\hat{y}|s_{0},y)-\mathrm{ER}^{i}_{s_{1},s_{0}}(\hat{y}|s_{0},y)-\mathrm{ER}^{s}_{s_{1},s_{0}}(\hat{y}|y).$

The above theorem shows that the total disparate error rate can be decomposed
into terms, each of which estimates the adverse impact of its corresponding
discriminatory mechanism.

###  6.9 Individual Equalized Counterfactual Odds

In [99], Pfohl et al. proposed the notion of individual equalized
counterfactual odds that is an extension of counterfactual fairness and
equalized odds. The notion is motivated by clinical risk prediction and aims
to achieve equal benefit across different demographic groups.

######  Definition 6.16 (Individual Equalized Counterfactual Odds).

Given a factual condition $\mathbf{O}=\mathbf{o}$ where
$\mathbf{O}\subseteq\\{\mathbf{X},Y\\}$, predictor $\hat{Y}$ achieves the
individual equalized counterfactual odds if

$P(\hat{y}_{s_{1}}|\mathbf{o},y_{s_{1}},s_{0})-P(\hat{y}_{s_{0}}|\mathbf{o},y_{s_{0}},s_{0})=0$

where $s_{1},s_{0}\in\\{s^{+},s^{-}\\}$.

The notion implies that the predictor must be counterfactually fair given the
outcome $Y$ matching the counterfactual outcome $y_{s_{0}}$. Therefore, in
addition to requiring predictions to be the same across factual/counterfactual
samples, those samples must also share the same value of the actual outcome
$Y$. In other words, it considers the desiderata from both counterfactual
fairness and equalized odds.

###  6.10 Fair on Average Causal Effect

In [100], Khademi et al. introduced two definitions of group fairness: fair on
average causal effect (FACE), and fair on average causal effect on the treated
(FACT) based on the Rubin-Neyman potential outcomes framework. Let $Y_{i}(s)$
be the potential outcome of an individual data point $i$ had $S$ been $s$.

######  Definition 6.17 (Fair on Average Causal Effect (FACE)).

A decision function is said to be fair, on average over all individuals in the
population, with respect to $S$, if $\mathbb{E}[Y_{i}(s_{1})-Y_{i}(s_{0})]=0$.

FACE considers the average causal effect of the marginalization attribute $S$
on the outcome $Y$ at the population level and is equivalent to the expected