# A Comprehensive Empirical Study of Bias Mitigation Methods for Software
Fairness

Zhenpeng Chen  University College LondonLondonUnited Kingdom ,  Jie M. Zhang
King’s College LondonLondonUnited Kingdom ,  Federica Sarro  University
College LondonLondonUnited Kingdom and  Mark Harman  University College
LondonLondonUnited Kingdom

###### Abstract.

Software bias is an increasingly important operational concern for software
engineers. We present a large-scale, comprehensive empirical evaluation of 17
representative bias mitigation methods, evaluated with 12 Machine Learning
(ML) performance metrics, 4 fairness metrics, and 24 types of fairness-
performance trade-off assessment, applied to 8 widely-adopted benchmark
software decision/prediction tasks. The empirical coverage is comprehensive,
covering the largest numbers of bias mitigation methods, evaluation metrics,
and fairness-performance trade-off measures compared to previous work on this
important operational software characteristic. We find that (1) the bias
mitigation methods significantly decrease the values reported by all ML
performance metrics (including those not considered in previous work) in a
large proportion of the scenarios studied (42%$\sim$75% according to different
ML performance metrics); (2) the bias mitigation methods achieve fairness
improvement in only approximately 50% over all scenarios and metrics (ranging
between 29%$\sim$59% according to the metric used to asses bias/fairness); (3)
the bias mitigation methods have a poor fairness-performance trade-off or even
lead to decreases in both fairness and ML performance in 37% of the scenarios;
(4) the effectiveness of the bias mitigation methods depends on tasks, models,
and fairness and ML performance metrics, and there is no ‘silver bullet’ bias
mitigation method demonstrated to be effective for all scenarios studied. The
best bias mitigation method that we find outperforms other methods in only 29%
of the scenarios. We have made publicly available the scripts and data used in
this study in order to allow for future replication and extension of our work.

Software fairness, machine learning performance, bias mitigation methods,
fairness-performance trade-off

††copyright: none††ccs: Software and its engineering Software creation and
management††ccs: Computing methodologies Machine learning

##  1\. Introduction

Machine Learning (ML)-enabled software (in short as _ML software_) has made
its way into a wide range of critical decision-making applications, such as
hiring, criminal justice, credit risk prediction, and admissions (Mehrabi et
al., 2021). There are several widely-known examples of software exhibiting
unfair behaviours, relating to protected attributes such as gender (fai,
[n.d.]f, [n.d.]d) and race (fai, [n.d.]c, [n.d.]e). Unfair software behaviour
may result in unacceptable and unethical consequences that adversely affect
users in minority and/or historically disadvantaged groups. Moreover, when
software falls within legal or regulatory frameworks, unfair behaviour also
incurs legal risks to software engineers.

The issue of fairness has been studied for some time in Software Engineering
(SE) research (Finkelstein et al., 2008), pre-dating the recent upsurge in ML
applications. However, the SE research community has increasingly focused on
fairness since then. This increased focus is induced by increasing software
systems’ reliance on ML as a powerful generic technique to tackle complex
decision and prediction problems, bringing with it the potential for
unfairness as a result of software bias.111We use “bias” to refer to the
opposite of “fairness” and treat “unfairness” and “bias” as synonyms. As a
result, the software engineering literature has witnessed a large number of
recent results on software bias and fairness (Hort and Sarro, 2021; Brun and
Meliou, 2018; Hort et al., 2021; Zhang and Harman, 2021; Biswas and Rajan,
2020; Chakraborty et al., 2020a, 2021; Galhotra et al., 2017; Udeshi et al.,
2018; Aggarwal et al., 2019; Zhang et al., 2020; Biswas and Rajan, 2021;
Chakraborty et al., 2020b). Software engineers tend to regard fairness as a
non-functional property (typically most pertinent to ML software (Zhang et
al., 2019), although also applicable more generally in SE (Finkelstein et al.,
2008)).

In SE nomenclature, unfairness can be thought of ‘fairness bugs’, thereby
motivating software engineers to tackle the problem using software testing
(Galhotra et al., 2017; Udeshi et al., 2018; Aggarwal et al., 2019; Zhang et
al., 2020). Alternatively, software engineers can regard fairness as a non-
functional property (Hort et al., 2021; Hort and Sarro, 2021; Chakraborty et
al., 2020a, 2021; Biswas and Rajan, 2020) seeking techniques for bias
mitigation, i.e., fixing fairness bugs to reduce software bias.

With the emergence of various bias mitigation methods, SE researchers have
started to evaluate and compare these methods. Such empirical evaluations
enable the development of scientific knowledge about how useful different bias
mitigation methods are in different application scenarios. Since there is a
widespread belief in the research community that bias mitigation methods often
improve fairness at the cost of ML performance (i.e., fairness-performance
trade-off) (Berk et al., 2021; Wick et al., 2019), to evaluate the
effectiveness of a bias mitigation method, researchers not only measure the
improvement of fairness achieved by it, but also consider its effect on ML
performance (e.g., accuracy).

However, existing evaluations of bias mitigation methods have yet to achieve
full coverage and completeness to give a comprehensive picture. In particular,
researchers often use only one or two metrics to measure the effects of
existing bias mitigation methods on ML performance, overlooking other metrics
that are widely used in industry and academia. For example, Zhang and Harman
(Zhang and Harman, 2021) and Hort et al. (Hort et al., 2021) measured ML
performance in terms of only accuracy; Chakraborty et al. (Chakraborty et al.,
2020a) used recall on favorable and unfavorable classes as only ML performance
metrics; Biswas and Rajan (Biswas and Rajan, 2020, 2021) measured only
accuracy as well as F1-score on the favorable class.

Nevertheless, different situations require different metrics. For example, in
an online advertising task targeting users with lower incomes, developers
would more likely care about precision for this category of users, because it
is common practice in online advertising to provide specific treatments to
users for whom the algorithm is confident about the attributes (Li et al.,
2015). However, existing work does not consider the corresponding metric
(i.e., precision on the unfavorable class) when evaluating bias mitigation
methods on benchmark tasks like the income prediction task (Zhang and Harman,
2021; Hort et al., 2021; Chakraborty et al., 2021, 2020a). Therefore, based on
the results of existing work, developers will have no findings on which to
base their decisions for such scenarios.

This gap motivates us to evaluate existing bias mitigation methods in terms of
comprehensive metrics. Since different metrics measure the functional or non-
functional properties of ML software from different aspects, we believe that
the results of this study can provide insightful implications for real-world
applications as well as a foundational baseline for further research and
follow-on studies.

We present a comprehensive study, evaluating 17 representative bias mitigation
methods in 8 widely adopted benchmark tasks with 12 ML performance metrics, 4
fairness metrics, and 24 types of fairness-performance trade-offs (i.e., the
fairness-performance trade-off in terms of different ML performance metrics
and fairness metrics). Our study covers the largest number of bias mitigation
methods, evaluation metrics, and fairness-performance trade-off measures in
software fairness research literature. Specifically, our study yields the
following implications for fairness research and practice:

  1. (1)

The values of all ML performance metrics that we use (including those not
considered in previous work) decrease in a large proportion (42%$\sim$75%) of
scenarios after applying bias mitigation methods. Therefore, it is important
for researchers to use a comprehensive set of ML performance metrics in their
evaluations, to be sure to capture any decreases in performance caused by bias
mitigation methods.

  2. (2)

Existing bias mitigation methods improve fairness values in only approximately
50% of the scenarios we studied, and have a poor fairness-performance trade-
off, or even lead to decreases in both fairness and ML performance in 37% of
the scenarios. Therefore, a community effort is required to bring software
fairness improvements to a level where it becomes more effective and usable in
practice.

  3. (3)

The effectiveness of the bias mitigation methods depends on tasks, models, and
fairness and ML performance metrics; there is no ‘silver bullet’ bias
mitigation method demonstrated to be effective for all scenarios. Therefore,
researchers and practitioners need to choose the bias mitigation method best
suited to their intended application scenario(s). Our results provide
empirical guidance for such choices.

  4. (4)

We have made available all scripts and data we used for this study (Chen et
al., [n.d.]) as an additional contribution to the research community for other
researchers to replicate and extend this work.

The rest of this paper is organized as follows. Section 2 describes the
background knowledge and related work about software fairness. Section 3
presents the research questions and methodology of this study. Section 4
reports and analyzes the results. Section 5 discusses threats to the validity,
followed by concluding remarks in Section 6.

##  2\. Background

In this section, we briefly provide background knowledge on software fairness
and introduce the literature on fairness research.

###  2.1. Definition of Fairness

As indicated in previous work (Speicher et al., 2018; Zhang and Harman, 2021;
Hort et al., 2021), there are two primary types of fairness that researchers
pursue, i.e., individual fairness and group fairness. Individual fairness
requires an ML model to produce similar predictive outcomes for similar
individuals, while group fairness requires an ML model to treat different
groups equally. Since existing bias mitigation methods (Wick et al., 2019;
Zhang and Harman, 2021; Hort et al., 2021; Chakraborty et al., 2020a, 2021;
Biswas and Rajan, 2021) focus on group fairness, in this paper, we also focus
on group fairness.

In the context of group fairness, a population is partitioned into the
privileged and unprivileged groups based on the values of protected
attributes, which refer to the sensitive characteristics (e.g., sex and race)
that need to be protected against unfairness. Usually, an unfair ML model
tends to favor the privileged group (i.e., inclined to produce the favorable
class for its members), thereby putting the unprivileged group at
disadvantage. For example, in the recidivism assessment task, race is a
protected attribute yet existing recidivism assessment systems have been
demonstrated to recommend favourable decisions for white defendants compared
to otherwise equivalent black defendants (fai, [n.d.]c).

###  2.2. Related Work

Fairness has attracted increasing interest from both the ML research community
(Mehrabi et al., 2021) and the SE research community (Hort et al., 2021) and
also from practitioners as well as researchers. For example, Microsoft
recently published the ethical principles of AI (mic, [n.d.]), stating that ML
software must be fair in real-life applications, creating a research group
named FATE (fat, [n.d.]) to promote software fairness. In addition, recently,
researchers, Brun and Meliou (Brun and Meliou, 2018) set out a vision for SE
research approaches to tackle fairness problems, and a recent survey on ML
testing (Zhang et al., 2019) classified fairness as a non-functional software
property, surveying SE approaches to tackling it.

The rising attention on fairness inspires the emergence of a series of
fairness testing techniques. Themis (Galhotra et al., 2017) generated test
suites to measure causal discrimination in software. Aeqitas (Udeshi et al.,
2018) exploited the inherent robustness property of ML models for directing
fairness test generation. SG (Aggarwal et al., 2019) combined symbolic
execution and local explainability to generate test inputs for detecting
individual discrimination. ADF (Zhang et al., 2020) used gradient computation
and clustering to generate individual discriminatory instances for DNNs.

In addition to testing software fairness, researchers also attempt to improve
software fairness. Zhang and Harman (Zhang and Harman, 2021) explored the
factors that affect software fairness, and found that enlarging feature set is
a possible way to improve fairness. Chakraborty et al. (Chakraborty et al.,
2020a) removed ambiguous data points in training data and then applied multi-
objective optimization to train fair ML models. To better improve software
fairness, the follow-up work of Chakraborty et al. (Chakraborty et al., 2021)
not only removed ambiguous data points, but also balanced the internal
distribution of training data. Moreover, IBM launched a software toolkit
called AI Fairness 360 (abbreviated as IBM AIF360) (aif, [n.d.]), which
integrated popular fairness improvement methods (i.e., bias mitigation
methods) proposed in the ML community, including Adversarial Debiasing (Zhang
et al., 2018), Reweighting (Kamiran and Calders, 2011), Reject Option
Classification (Kamiran et al., 2012), Learning Fair Representation (Zemel et
al., 2013), etc.

Furthermore, there is some work that empirically evaluates the effectiveness
of different bias mitigation methods. For example, Biswas and Rajan (Biswas
and Rajan, 2020, 2021) evaluated seven bias mitigation methods on real-world
ML models from a crowd-sourced platform and explored the impact of popular
pre-processing procedures on ML performance and fairness. Chakraborty et al.
(Chakraborty et al., 2020a, 2021) compared the bias mitigation methods
proposed by them with several methods proposed in the ML community. In these
work, the changes in ML performance and fairness caused by bias mitigation
methods are measured and visualised separately. Therefore, it is difficult to
judge whether the improved fairness is simply the consequence of ML
performance loss. To tackle this problem, Hort et al. (Hort et al., 2021)
proposed a model behavior mutation method to quantitatively benchmark and
evaluate the fairness-performance trade-off of different bias mitigation
methods.

Nevertheless, existing evaluations of bias mitigation methods are conducted in
terms of limited metrics and measures. In particular, researchers often use
only one or two ML performance metrics in the evaluations. For example, most
of fairness work (Zhang and Harman, 2021; Hort et al., 2021; Calders et al.,
2009; Calders and Verwer, 2010; Celis et al., 2019; Feldman et al., 2015a;
Kamiran and Calders, 2011; Kamiran et al., 2010, 2018; Zafar et al., 2017)
measured ML performance in terms of only accuracy. Chakraborty et al.
(Chakraborty et al., 2020a) used recall on the favorable class and false alarm
(i.e., 1 minus recall on the unfavorable class) to compare different methods.
Biswas and Rajan (Biswas and Rajan, 2020, 2021) measured accuracy as well as
F1-score on the favorable class. In contrast, Chakraborty et al. (Chakraborty
et al., 2021) employed the most ML performance metrics, including accuracy and
precision/recall/F1-score on the favorable class, but they still ignored other
metrics that measure ML performance on the unfavorable class and those that
measure overall performance on favorable and unfavorable classes. Since
different ML performance metrics indicate different functional properties of
ML software, it is important to evaluate bias mitigation methods on various ML
performance metrics to provide comprehensive and insightful implications for
real-world applications. To fill the knowledge gap, in this paper, we aim to
evaluate existing bias mitigation methods in terms of comprehensive metrics
and fairness-performance trade-off measures.

In addition, existing work focuses on limited bias mitigation methods. For
example, Bias and Rajan (Biswas and Rajan, 2020) evaluated seven of our used
methods; Chakraborty et al. (Chakraborty et al., 2020a) evaluated five;
Chakraborty et al. (Chakraborty et al., 2021) evaluated three. In constrast,
Hort et al. (Hort et al., 2021) evaluated the most bias mitigation methods, a
total of twelve. However, they focused on only methods proposed in the ML
community. In this paper, we aim to consider 17 representative bias mitigation
methods from both the ML and SE communities. This large-scale study allows us
to get a big picture of the literature as well as future research challenges
and opportunities.

##  3\. Experimental Setup

In this section, we describe the research questions and experimental design of
this study.

Figure 1. Overview of experimental design.

###  3.1. Overview of Experimental Design

Fig. 1 illustrates our experimental design in a nutshell. First, we use five
widely-adopted benchmark datasets (covering eight different bias mitigation
tasks) to train ML models with three traditional ML algorithms. Second, we
apply 17 representative bias mitigation methods from the ML and SE communities
to these ML models. Third, we adopt 12 ML performance metrics and 4 fairness
metrics to evaluate different methods in terms of ML performance and fairness,
separately. Finally, we consider performance and fairness together, and
evaluate the fairness-performance trade-off of the 17 bias mitigation methods
using 24 fairness-performance metric pairs. Specifically, we aim to answer the
following research questions (RQs):

RQ1 (Influence on ML performance): _How do existing bias mitigation methods
affect ML software in terms of performance changes?_ ML performance (e.g.,
accuracy and precision) represents important functional requirements of ML
software, but bias mitigation methods may improve fairness at the cost of ML
performance. Therefore, we first investigate how existing methods change ML
performance in terms of various metrics.

RQ2 (Influence on fairness): _How do existing bias mitigation methods affect
ML software in terms of fairness improvement?_ As the main goal of bias
mitigation methods is to reduce bias in ML software, we explore how well
existing bias mitigation methods achieve this goal in terms of various
fairness metrics.

RQ3 (Influence on fairness-performance trade-off): _How do existing bias
mitigation methods affect ML software in terms of fairness-performance trade-
off?_ We finally consider fairness and ML performance together, and evaluate
existing bias mitigation methods in terms of different types of fairness-
performance trade-offs, i.e., combinations of different fairness metrics and
ML performance metrics.

In the following, we introduce the bias mitigation methods (Section 3.2),
benchmark datasets (Section 3.3), metrics and measures (Section 3.4), and
experimental settings (Section 3.5).

###  3.2. Bias Mitigation Methods

We focus our analysis on 17 representative bias mitigation methods proposed in
the ML and SE communities. To the best of our knowledge, this work covers the
most bias mitigation methods in software fairness research. As for the methods
proposed in the ML community, we follow previous work (Biswas and Rajan, 2020;
Zhang and Harman, 2021; Hort et al., 2021; Chakraborty et al., 2020a) to
employ the state-of-the-art ones implemented in the IBM AIF360 framework (aif,
[n.d.]). Specifically, we employ all the ten methods listed on its homepage
(aif, [n.d.]), covering three types, i.e., pre-processing, in-processing, and
post-processing. Pre-processing methods aim to process the training data to
mitigate data bias; in-precessing methods aim to improve group fairness during
the training process; post-processing methods aim to modify the prediction
outcomes of ML models to improve fairness. Next, we briefly introduce each
method by type.

_Pre-processing:_ Optimized Pre-processing (OP) (du Pin Calmon et al., 2017)
learns a probabilistic transformation to modify data features and labels.
Learning Fair Representation (LFR) (Zemel et al., 2013) learns fair
representations by obfuscating information about protected attributes.
Reweighting (REW) (Kamiran and Calders, 2011) generates different weights for
the training samples in each (group, label) combination. Disparate Impact
Remover (DIR) (Feldman et al., 2015b) modifies feature values to improve group
fairness while preserving rank-ordering within groups.

_In-processing:_ Prejudice Remover (PR) (Kamishima et al., 2012) adds a
discrimination-aware regularization term to the learning objective.
Adversarial Debiasing (AD) (Zhang et al., 2018) uses adversarial techniques to
maximize accuracy and reduce evidence of protected attributes in the
predictions simultaneously. Meta Fair Classifier (MFC) (Celis et al., 2019)
takes the fairness metric as part of the input and returns a classifier
optimized for that metric.

_Post-processing:_ Reject Option Classification (ROC) (Kamiran et al., 2012)
targets predictions with high uncertainty and tends to assign favorable
outcomes to the unprivileged group and unfavorable outcomes to the privileged
group. Calibrated Equalized Odds Post-processing (CEO) (Pleiss et al., 2017)
optimizes over calibrated classifier score outputs to find probabilities with
which to change output labels with an equalized odds objective. Equalized Odds
Post-processing (EO) (Hardt et al., 2016) solves a linear program to find
probabilities with which to change output labels to optimize equalized odds.

As for the methods proposed in the SE community, we use two methods recently
published on top SE venues, including Fairway (Chakraborty et al., 2020a) at
ESEC/FSE 2020 and Fair-SMOTE (Chakraborty et al., 2021) at ESEC/FSE 2021.

Fairway (Chakraborty et al., 2020a) combines pre-processing and in-processing
techniques to improve fairness. First, it evaluates the original labels of the
training data and removes ambiguous data points that can eventually make the
classifier biased. Then, it employs multi-objective optimization to maximize
the model performance while making it fair.

Fair-SMOTE (Chakraborty et al., 2021) is a pre-processing method that employs
two strategies. First, it generates new data points to make the numbers of
training data in different subgroups (i.e., combinations of different outcomes
and protected attribute values) equal. Second, it uses the same method as
Fairway to remove ambiguous data points from the training data.

In the IBM AIF360, MFC, ROC, and CEO are implemented with two, three, and
three different metrics to guide the bias mitigation process, respectively.
Specifically, MFC offers a choice between Disparate Impact (DI) and FDR (False
Discovery Rate); ROC offers a choice among Statistical Parity Difference
(SPD), Average Odds Difference (AOD), and Equal Opportunity Difference (EOD);
CEO offers a choice among False Negative Rate (FNR), False Positive Rate
(FPR), and a weighted metric to combine both. We implement and evaluate each
of the settings. Therefore, we have a total of 17 bias mitigation methods for
our study.

###  3.3. Benchmark Datasets

We follow previous work (Zhang and Harman, 2021) to use five most widely
studied benchmark datasets implemented in the IBM AIF360 (as listed in Table
1) for this study. The five datasets cover social, financial, and medical
domains, and are the most widely adopted in fairness literature of the ML
(Mehrabi et al., 2021) and SE (Zhang and Harman, 2021; Hort et al., 2021;
Chakraborty et al., 2020a, 2021; Biswas and Rajan, 2021) communities. The
number of datasets used in this study aligns with the fairness literature, as
previous work (Hort et al., 2021) points out that 90% of fairness papers use
no more than three datasets. Next, we briefly introduce each dataset that we
use.

Table 1. Benchmark datasets used for bias mitigation. Name | Size | #Features | Protected attribute(s) | Favorable label | Majority label
---|---|---|---|---|---
Adult | 45,222 | 14 | Sex, Race | 1 (income $\textgreater$ 50K) | 0 (75.2%)
Compas | 6,167 | 10 | Sex, Race | 0 (no recidivism) | 0 (54.5%)
German | 1,000 | 20 | Sex, Age | 1 (good credit) | 1 (70.0%)
Bank | 30,488 | 20 | Age | 1 (subscriber) | 0 (87.3%)
Mep | 15,830 | 41 | Race | 1 (utilizer) | 0 (82.8%)

Audult Income dataset (adu, [n.d.]) (a.k.a., Adult dataset) contains
demographic and financial information about individuals extracted from the
1994 U.S. census data, and is used to predict whether the income of a person
exceeds $50K a year or not.

ProPublica Recidivism dataset (com, [n.d.]) (a.k.a., Compas dataset) contains
demographic information and criminal histories of defendants from Broward
County, and is used to predict whether a defendant will be re-offended within
two years.

German Credit dataset (ger, [n.d.]) (a.k.a., German dataset) contains
demographic and credit information of 1,000 individuals, and is used to
predict people’s credit risk levels.

Bank Marketing dataset (ban, [n.d.]) (a.k.a., Bank dataset) contains
demographic, social, and financial information of clients of a Portuguese
banking institution, and is used to predict whether a client will subscribe a
term deposit.

Medical Survey 2015 dataset (mep, [n.d.]) (a.k.a., Mep dataset) contains data
measuring how Americans use and pay for medical care, health insurance, and
out-of-pocket spending, and is used to predict health care utilization of
individuals.

As shown in Table 1, each dataset has its protected attribute(s) determined by
its provider based on its task. In line with previous work (Zhang and Harman,
2021; Hort et al., 2021; Chakraborty et al., 2020a; Biswas and Rajan, 2021),
we consider one protected attribute each time and thus have eight dataset-
attribute pairs (e.g., Adult-Sex and Adult-Race). We use the eight pairs as
the eight bias mitigation tasks of this study and take each task as a binary
classification problem.

###  3.4. Evaluation Metrics and Measures

We investigate 12 ML performance metrics, 4 fairness metrics, and 24 types of
fairness-performance trade-offs for a comprehensive evaluation of existing
bias mitigation methods. To the best of our knowledge, this work covers the
most evaluation metrics and trade-off measures in software fairness research.
In the following, we briefly introduce the metrics and measures that we use.

Given a bias mitigation task, let the protected attribute be $A$, with 0 as
the unprivileged group and 1 the privileged group; let the real classification
label be $Y$ and the predicted label $\hat{Y}$, with 0 as the unfavorable
class and 1 the favorable class. In addition, we use $Pr$ to denote
probability.

####  3.4.1. ML Performance Metrics

For each bias mitigation method, we measure the ML performance changes caused
by it on favorable and unfavorable classes in terms of precision, recall, and
F1-score, which are widely employed in classification work in SE (Novielli et
al., 2018; Chen et al., 2021, 2019; Novielli et al., 2020).

Precision measures the exactness of a method. The precision for a given class
c (i.e., 0 or 1) is calculated as:

(1<EMAIL_ADDRESS>
Recall measures the sensitivity of a method. The recall for a given class c is
calculated as:

(2<EMAIL_ADDRESS>
F1-score measures a harmonic mean of precision and recall. The F1-score for a
given class c is calculated as:

(3) $F1@c=\frac{2*Precision@c*Recall@c}{Precision<EMAIL_ADDRESS>
In addition, to measure the overall performance, we follow previous
classification work in SE to use Area Under Curve (AUC) (Rahman et al., 2012;
Nam and Kim, 2015; Agrawal and Menzies, 2018) and accuracy (Acc) (Hort et al.,
2021; Zhang and Harman, 2021; Chakraborty et al., 2021; Biswas and Rajan,
2020, 2021). AUC measures the area under the receiver operating characteristic
curve, which is plotted using the true positive rate and the false positive
rate by changing different prediction thresholds, while Acc measures how often
a method makes the correct prediction and is calculated as:

(4) $Acc=Pr[\hat{Y}=Y].$

Acc is often criticized as not being suitable for the imbalanced class
distribution, because it is easy for an ML model to obtain a high accuracy
just by predicting all samples as the majority class in such a distribution.
Considering that some datasets (e.g., the Bank dataset) that we use have an
imbalanced class distribution, we follow previous SE work to use three
additional macro-average metrics (Novielli et al., 2018; Novielli et al.,
2020; Chen et al., 2021) and the Matthews Correlation Coefficient (MCC) metric
(Yao and Shepperd, 2020; Rodríguez et al., 2014), which are all demonstrated
to be suitable for dealing with imbalanced scenarios (Sebastiani, 2002; Chicco
and Jurman, 2020). As for the macro-average metrics, we use macro-precision
(Mac-P), macro-recall (Mac-R), and macro-F1 (Mac-F1), which take the average
of precision, recall, and F1-score on the favorable and unfavorable classes,
respectively. As for MCC, it is calculated as:

(5) $MCC=\frac{TP*TN-FP*FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}},$

where TP, TN, FP, and FN denote the numbers of favorable samples predicted as
favorable, unfavorable samples predicted as unfavorable, unfavorable samples
predicted as favorable, and favorable samples predicted as unfavorable,
respectively.

To summarize, we use 12 ML performance metrics, including F-P (precision for
the favorable), F-R (recall for the favorable), F-F1 (F1-score for the
favorable), UnF-P (precision for the unfavorable), UnF-R (recall for the
unfavorable), UnF-F1 (F1-score for the unfavorable), AUC, Acc, Mac-P, Mac-R,
Mac-F1, and MCC. The values of F-P, F-R, F-F1, UnF-P, UnF-R, UnF-F1, AUC, Acc,
Mac-P, Mac-R, and Mac-F1 are between 0 and 1. The value of MCC is between -1
and 1, where 1 represents a perfect prediction, 0 no better than random
prediction, and -1 total disagreement between prediction and observation. For
all the metrics, larger values indicate better ML performance.

####  3.4.2. Fairness Metrics

Based on different definitions of fairness, various fairness metrics have been
proposed to measure the difference in classification between the privileged
and unprivileged groups. In this work, we use the group fairness metrics that
are most widely adopted in fairness research (Biswas and Rajan, 2020, 2021;
Zhang and Harman, 2021; Hort et al., 2021; Chakraborty et al., 2020a, 2021).

Statistical Parity Difference (SPD) measures the difference in the acceptance
rate of the favorable class between the privileged and unprivileged groups:

(6) $\displaystyle SPD=Pr[\hat{Y}=1|A=0]-Pr[\hat{Y}=1|A=1].$

Average Odds Difference (AOD) measures the average of differences in the true
positive rate and the false positive rate between the privileged and
unprivileged groups:

(7) $\displaystyle AOD=$
$\displaystyle\frac{1}{2}(|Pr[\hat{Y}=1|A=0,Y=0]-Pr[\hat{Y}=1|A=1,Y=0]|$
$\displaystyle+|Pr[\hat{Y}=1|A=0,Y=1]-Pr[\hat{Y}=1|A=1,Y=1]|).$

Equal Opportunity Difference (EOD) measures the difference in the true
positive rate between the privileged and unprivileged groups:

(8) $\displaystyle EOD$
$\displaystyle=Pr[\hat{Y}=1|A=0,Y=1]-Pr[\hat{Y}=1|A=1,Y=1].$

Error Rate Difference (ERD) measures the difference in the error rate between
the privileged and unprivileged groups:

(9) $\displaystyle ERD$ $\displaystyle=Pr[\hat{Y}\neq Y|A=0]-Pr[\hat{Y}\neq
Y|A=1].$

There is another fairness metric called Disparate Impact (DI), which is also
widely adopted in the fairness literature. DI and SPD both compare the
probabilities of classifying samples as favorable in the privileged and
unprivileged groups. Specifically, DI computes the ratio of the two
probabilities, while SPD computes the difference of the two probabilities.
When computing DI, we may encounter the divided-by-zero error. Therefore,
between SPD and DI, we follow previous work (Biswas and Rajan, 2021; Hort et
al., 2021) to use only SPD in our evaluation.

For all the fairness metrics, we use their absolute values. In this way,
values equal to 0 indicate the greatest fairness; larger values indicate more
bias.

####  3.4.3. Fairness-performance Trade-off Measures

It is difficult to evaluate which bias mitigation method is better based on
fairness alone, since it is unclear whether the improved fairness is simply
the consequence of ML performance loss. Therefore, here, we consider fairness
and ML performance together and measure the fairness-performance trade-off of
different methods.

To this end, we employ Fairea (Hort et al., 2021), a model behavior mutation
method proposed at ESEC/FSE 2021, to benchmark and quantify the fairness-
performance trade-off achieved by existing bias mitigation methods. Next, we
briefly introduce Fairea, which is illustrated in Fig. 2.222The figure is
taken from the original paper (Hort et al., 2021). Specifically, Fairea
includes three steps as follows:

Figure 2. Illustration of Fairea (Hort et al., 2021). (a) presents the
fairness-performance trade-off baseline, where $P_{OM}$ represents the
original model, BM the model after bias mitigation, and $P_{10},...,P{100}$
the points obtained by model behavior mutation. (b) presents the effectiveness
regions of bias mitigation methods based on the changes in ML performance and
bias.

_Step 1: Create trade-off baseline._ Fairea presents the ML performance and
fairness achieved by bias mitigation methods in a two-dimensional coordinate
system as shown in Fig. 2(a). It constructs the fairness-performance trade-off
baseline by connecting the original model without applying bias mitigation
(i.e., $P_{OM}$) and a series of pseudo mitigation models generated by model
behavior mutation (i.e., $P_{10}$, …, $P_{100}$). The pseudo mitigation models
are mutated based on the original model by sacrificing ML performance to
reduce bias in a naive way. Specifically, Fairea randomly chooses a subset of
the predictions made by the original model and replaces them with the majority
class of the data. It considers different mutation degrees (i.e., the fraction
of chosen predictions) from 10% to 100%, with a step size of 10%, to obtain
$P_{10}$, …, $P_{100}$. The core insight of Fairea is that when the original
model is mutated into a model that always predicts the same class, the
fairness will be greatly improved as the predictive performance is equally
worse in the privileged and unprivileged groups. The fairness-performance
trade-offs of these naive mutated models are expected to be surpassed by any
reasonable bias mitigation method, so we use these models as the baseline.

_Step 2: Divide effectiveness regions._ The obtained baseline categorizes bias
mitigation methods into five regions that represent different effectiveness
levels. As illustrated in Fig. 2(b), the win-win region contains bias
mitigation methods that improve ML performance and decrease bias, while the
lose-lose region contains methods that decrease ML performance and increase
bias. Methods that improve both ML performance and bias fall in the inverted
trade-off region. The remaining two regions contain methods that reduce bias
but decrease ML performance. Specifically, if a method achieves a better
trade-off than the baseline constructed by Fairea, it falls within the good
trade-off region. Otherwise, it belongs to the poor trade-off region. The
region division of Fairea provides an overview of the overall effectiveness of
a bias mitigation method.

_Step 3: Quantify trade-off effectiveness._ Fairea also provides a solution to
measure the effectiveness of a bias mitigation method in a quantitative way.
Fig. 2(a) shows the area (indicated in grey) obtained by connecting the point
of the model after applying a bias mitigation method (i.e., BM) and the Fairea
baseline vertically and horizontally. Fairea calculates the size of the area
as a quantitative measure of the fairness-performance trade-off achieved by a
bias mitigation method. A larger area indicates a better trade-off. Using the
area as a measure of the trade-off enables a convenient comparison among
different bias mitigation methods.

In the original paper (Hort et al., 2021), Fairea evaluated only two types of
fairness-performance trade-offs (i.e., SPD&Acc and AOD&Acc) on 12 bias
mitigation methods proposed in the ML community. In this study, we aim to
extend our evaluation to the trade-off between more fairness and performance
metric pairs on 17 bias mitigation methods proposed in the ML and SE
communities. Since we employ 12 ML performance metrics and 4 fairness metrics
in this study, there are a total of 48 fairness-performance metric pairs.
However, as Fairea replaces predictions with the majority class, in terms of
ML performance metrics for a certain class (e.g., recall on the majority
class), we may observe that ML performance and fairness are both improved with
the increased mutation degrees. For such metrics, we cannot obtain the trade-
off baselines as in Fig. 2(a). Therefore, here we choose only 6 ML performance
metrics that measure performance on both the favorable and unfavorable
classes, i.e., AUC, Acc, Mac-P, Mac-R, Mac-F1, and MCC, to reflect the overall
performance of each bias mitigation method. As a result, we have 24 types of
fairness-performance trade-offs, i.e., combinations of 6 ML performance
metrics and 4 fairness metrics. To the best of our knowledge, this work covers
the most types of fairness-performance trade-offs in software fairness
research.

###  3.5. Experimental Settings

To ensure the verifiability and transparency of our study, in this section, we
describe the experimental settings in details.

_Implementation of datasets:_ We use the five benchmark datasets implemented
in the IBM AIF360 via directly invoking off-the-shelf APIs (aif, [n.d.]).
Moreover, we follow previous work (Chakraborty et al., 2021, 2020a; Zhang and
Harman, 2021; Hort et al., 2021) to normalize all feature values to be between
0 and 1.

_Implementation of bias mitigation methods:_ For each bias mitigation task, we
train original models for bias mitigation using three traditional
classification algorithms that have been widely adopted in previous work:
Logistic Regression (LR) (Hort et al., 2021; Chakraborty et al., 2020a; Biswas
and Rajan, 2020; Zhang and Harman, 2021), Support Vector Machine (SVM) (Hort
et al., 2021; Biswas and Rajan, 2020), and Random Forest (RF) (Chakraborty et
al., 2021; Biswas and Rajan, 2020; Zhang and Harman, 2021). In line with
previous work (Hort et al., 2021; Zhang and Harman, 2021; Chakraborty et al.,
2021), we use the default configuration provided by the scikit-learn library
(skl, [n.d.]) to implement each classification algorithm. We apply 17 bias
mitigation methods based on the original models, respectively. Specifically,
pre- and post-processing methods are applied before and after model training,
while in-processing methods are applied during the training process. We
implement the 15 bias mitigation methods proposed in the ML community based on
the IBM AIF360 framework (aif, [n.d.]), and implement the two methods proposed
in the SE community based on the code released by their authors (fai, [n.d.]b,
[n.d.]a). Since the IBM AIF360 does not support the OP method for the Bank and
Mep datasets, we apply OP only for six tasks. For the remaining 16 methods, we
apply each of them for all the eight bias mitigation tasks. Each application
is repeated 50 times. In each of the 50 runs, the dataset is shuffled and
randomly splitted into 70% training data and 30% test data. We treat each
single run as an individual _mitigation case_. As a result, for each task,
each bias mitigation method has 3*50 = 150 mitigation cases.

_Implementation of Fairea:_ For each _(task, classification algorithm,
fairness-performance metric pair)_ combination, to create the fairness-
performance trade-off baseline, we train the original model 50 times. Each
time, based on the original model, we repeat the mutation procedure 50 times
for each mutation degree, i.e., 10%, 20%, …, and 100%. Finally, as suggested
by Fairea (Hort et al., 2021), we construct the baseline using the mean value
of the multiple runs.

_Statistical analysis:_ To test whether the difference between two bias
mitigation methods is statistically significant on a metric, we employ the
non-parametric Mann Whitney U-test (Mann and Whitney, 1947). This test suits
our purpose well as it does not assume normality. The difference is considered
significant, only if the $p$-value of the computed statistic is lower than a
pre-specified level (usually 0.05). For example, when we compare two sets of
accuracy values achieved by the 50 runs of methods _A_ and _B_ via the Mann
Whitney U-test, the null hypothesis is that the accuracy of _A_ is similar to
_B_. If we find that $p$-value $\textless$ 0.05, we can conclude with 95%
confidence that our alternative hypothesis is true, which indicates that _A_
achieves a significantly different accuracy than _B_. Furthermore, we compute
the effect size with the Cohen’s $d$ (Cohen, 2013), to check whether the
difference has a meaningful effect. We consider the difference with 0
$\textless$ $d$ $\textless$ 0.5 a small effect, 0.5 $\leq$ $d$ $\textless$ 0.8
a medium effect, and $d$ $\geq$ 0.8 a large effect (Sawilowsky, 2009).

In addition, we employ the Spearman’s rank correlation coefficient $\rho$
(Myers et al., 2013) to investigate whether the value changes of different ML
performance metrics or fairness metrics caused by bias mitigation methods are
similar. Spearman’s rank correlation coefficient does not assume normality,
and thus suits our purpose. The value of $\rho$ is between -1 to 1, where -1
indicates perfectly negative correlation, 0 no correlation, 1 perfectly
positive correlation. Moreover, the $p$-value is reported together with the
correlation coefficient. The correlation is considered statistically
significant, only if the $p$-value is lower than a pre-specified level
(usually 0.05).

_Experimental environment:_ All experiments are implemented with Python 3.7.11
and TensorFlow 2.6.0, and executed on a Ubuntu 16.04 LTS with 128GB RAM,
having 2.3 GHz Intel Xeon E5-2653 v3 Dual CPU and two NVidia Tesla M40 GPUs.

##  4\. Results

In this section, we first analyze the evaluation results for ML performance
(Section 4.1) and fairness (Section 4.2), separately. Then, we present the
measurement results for different types of fairness-performance trade-offs
(Section 4.3).

Figure 3. RQ1.1: Effects of bias mitigation methods on different ML
performance metrics. After applying existing bias mitigation methods, the
values of the 12 ML performance metrics significantly decrease among an
average of 58% of the applications.

###  4.1. RQ1: Influence on ML Performance

This section presents the results for 12 ML performance metrics. Based on the
results, we dive into RQ1 by answering two specific questions:

RQ1.1 (Effects on ML performance metric values): _How do the values of ML
performance metrics change after applying bias mitigation methods?_ First, we
investigate whether the values of ML performance metrics are significantly
changed after applying existing bias mitigation methods. Additionally, we
explore whether the value changes in different ML performance metrics are
significantly correlated. If so, researchers and practitioners may employ the
ML performance metrics used in previous work as the proxy of the unconsidered
ones.

RQ1.2 (ML performance comparison among methods): _How do different bias
mitigation methods affect ML performance?_ Second, we compare existing bias
mitigation methods in terms of different ML performance metrics. The results
provide implications for the choice of bias mitigation methods in application
scenarios where ML performance is critical.

####  4.1.1. RQ1.1: Effects on ML Performance Metric Values

To answer RQ1.1, we use the original models that do not apply any bias
mitigation method as the baselines. Specifically, for each bias mitigation
task, we use the LR, SVM, and RF algorithms to train the original models, with
each algorithm repeated 50 times. Then, for each task-algorithm pair, we use
the average level (i.e., mean values of ML performance metrics) of the
corresponding 50 original models as the baseline. Since we apply each bias
mitigation method in each task-algorithm pair, we have a total of 16*8*3+1*6*3
= 402 applications.333Note that we apply the OP method in six tasks and the
other 16 methods in eight tasks, as described in Section 3.5. We repeat each
application 50 times. For each application, we calculate the difference
between the mean values achieved by the 50 models after applying the bias
mitigation method and the corresponding 50 original models, for each ML
performance metric. Then we analyze the significance and effect size of the
difference using Mann Whitney U-test and Cohen’s $d$. In Fig. 3, for each ML
performance metric, we present the proportions of applications that fall into
different effects, i.e., decreasing ML performance significantly ($p$-value
$\textless$ 0.05) with a large effect ($d$ $\geq$ 0.8), decreasing ML
performance significantly with a medium effect (0.5 $\leq$ $d$ $\textless$
0.8), decreasing ML performance significantly with a small effect (0
$\textless$ $d$ $\textless$ 0.5) , and increasing ML performance or decreasing
ML performance insignificantly ($p$-value $\geq$ 0.05).

From Fig. 3, we observe that the values of the 12 ML performance metrics
decrease significantly in an average of 58% of applications (ranging from 42%
to 75% according to different metrics). In particular, the values of the 12
metrics significantly decrease with a large effect in an average of 46% of
applications (ranging from 38% to 58%). However, as described in Section 1,
existing studies often evaluate bias mitigation methods in terms of only one
or two ML performance metrics, ignoring some important metrics such as UnF-P
and UnF-F1. As a result, researchers and practitioners may be unaware of the
significantly decreased performance caused by bias mitigation methods in these
unconsidered metrics and thus choose inappropriate methods, making the
functional properties of ML software not up to expectations.

Table 2. RQ1.1: Correlation between ML performance metrics. In the table, * indicates a significant correlation ($p$-value $\textless$ 0.05) overall, and numbers in parentheses indicate that in how many task-algorithm pairs the correlation shares the consistent pattern with the overall correlation. We observe that the effects of bias mitigation methods on the ML performance metrics that are not considered in previous work do not have a consistent correlation with any previously employed metric across all the task-algorithm pairs. | F-R | F-F1 | UnF-P | UnF-R | UnF-F1 | AUC | Acc | Mac-P | Mac-R | Mac-F1 | MCC
---|---|---|---|---|---|---|---|---|---|---|---
F-P | -0.614*(18/24) | -0.131*(1/24) | -0.383*(8/24) | 0.776*(23/24) | 0.782*(22/24) | 0.085(2/24) | 0.579*(12/24) | 0.697*(13/24) | 0.085(2/24) | 0.414*(13/24) | 0.331*(8/24)
F-R | - | 0.689*(22/24) | 0.763*(17/24) | -0.851*(23/24) | -0.501*(13/24) | 0.373*(14/24) | -0.103*(2/24) | -0.179*(5/24) | 0.373*(14/24) | 0.101*(7/24) | 0.217*(8/24)
F-F1 | - | - | 0.653*(23/24) | -0.401*(12/24) | 0.016(13/24) | 0.587*(15/24) | 0.447*(13/24) | 0.294*(10/24) | 0.587*(15/24) | 0.610*(15/24) | 0.677*(16/24)
UnF-P | - | - | - | -0.532*(12/24) | -0.171*(3/24) | 0.514*(14/24) | 0.201*(11/24) | 0.195*(12/24) | 0.514*(14/24) | 0.292*(7/24) | 0.424*(11/24)
UnF-R | - | - | - | - | 0.818*(22/24) | 0.046(1/24) | 0.367*(6/24) | 0.368*(8/24) | 0.046(1/24) | 0.315*(10/24) | 0.202*(9/24)
UnF-F1 | - | - | - | - | - | 0.439*(12/24) | 0.682*(15/24) | 0.562*(14/24) | 0.439*(12/24) | 0.715*(19/24) | 0.620*(15/24)
AUC | - | - | - | - | - | - | 0.287*(8/24) | 0.124*(7/24) | 1.000*(24/24) | 0.821*(21/24) | 0.868*(23/24)
Acc | - | - | - | - | - | - | - | 0.841*(23/24) | 0.287*(8/24) | 0.652*(16/24) | 0.600*(14/24)
Mac-P | - | - | - | - | - | - | - | - | 0.124*(7/24) | 0.447*(11/24) | 0.445*(11/24)
Mac-R | - | - | - | - | - | - | - | - | - | 0.821*(21/24) | 0.868*(23/24)
Mac-F1 | - | - | - | - | - | - | - | - | - | - | 0.959*(24/24)

Next, we calculate the Spearman’s rank correlation coefficient $\rho$ between
every pair of ML performance metrics. Specifically, we calculate the value
differences of each ML performance metric before and after bias mitigation in
the 402 applications. Then, for each two ML performance metrics, we calculate
the overall correlation between the 402 value differences of the two metrics.
In addition, we also calculate the correlation in the 8*3 = 24 task-algorithm
pairs, separately. Table 2 shows the correlation results. For each metric
pair, we present the overall $\rho$, with * indicating a significant
correlation (i.e., $p$-value $\textless$ 0.05). Additionally, we list the
number of task-algorithm pairs where the correlation shares the consistent
pattern with the overall correlation in parentheses. For example, the
correlation result of F-P and MCC is 0.331*(8/24), indicating that the value
differences of F-R and MCC have a significantly positive correlation overall,
but that the significantly positive correlation holds in only 8 out of 24
task-algorithm pairs. Overall, among the 66 metric pairs in Table 2, 51 show a
significantly positive correlation, 10 a significantly negative correlation, 5
no significant correlation. Furthermore, we find that 64 metric pairs do not
present a consistent correlation pattern across all the task-algorithm pairs.
Specifically, only the (AUC, Mac-R) and (Mac-F1, MCC) pairs present a
consistent correlation pattern. However, the four metrics are all not
considered in previous software fairness work (Hort et al., 2021; Zhang and
Harman, 2021; Chakraborty et al., 2021, 2020a; Biswas and Rajan, 2020, 2021).
Since all the metrics not considered in previous work do not have a consistent
correlation with any previously employed metric, we may not use the latter as
their proxy. This finding suggests that we take comprehensive ML performance
metrics into account during the evaluation of bias mitigation methods,
especially considering that different ML performance metrics measure the
functional properties of ML software from different aspects and thus may
provide different guidelines for real-world application scenarios.

Finding 1: The values of all the 12 ML performance metrics (including those
not considered in previous work) decrease significantly in a notable
proportion of applications (42%$\sim$75% according to different metrics) after
applying existing bias mitigation methods. Moreover, the effects of bias
mitigation methods on the ML performance metrics unconsidered in previous work
do not have a consistent correlation with previously employed metrics, and
therefore the latter cannot be used as a proxy.

Figure 4. RQ1.2: Effect distribution of different bias mitigation methods in ML performance. We find that only 7 out of 17 (41%) bias mitigation methods significantly decrease ML performance in less than 50% of scenarios. Table 3. RQ1.2: Mean rank of each bias mitigation method in terms of different ML performance metrics. In terms of any ML performance metric, none of the 17 bias mitigation methods can consistently achieve better results than other methods in all the task-algorithm pairs. Metric | OP | LFR | RW | DIR | PR | AD |  | MFC
---
-FDR
| MFC
---
-SR
| ROC
---
-SPD
| ROC
---
-AOD
| ROC
---
-EOD
| CEO
---
-FNR
| CEO
---
-FPR
| CEO
---
-W
EOP |  | Fair
---
-way
| Fair
---
-SMOTE
F-P | 12 | 15 | 5 | 7 | 5 | 6 | 12 | 11 | 7 | 7 | 8 | 9 | 8 | 7 | 9 | 7 | 6
F-R | 9 | 11 | 9 | 7 | 11 | 10 | 5 | 5 | 11 | 9 | 8 | 7 | 8 | 9 | 8 | 8 | 9
F-F1 | 11 | 15 | 6 | 5 | 10 | 9 | 7 | 10 | 11 | 8 | 7 | 6 | 8 | 7 | 8 | 8 | 7
UnF-P | 12 | 16 | 8 | 7 | 11 | 9 | 3 | 4 | 10 | 8 | 7 | 7 | 10 | 8 | 9 | 8 | 8
UnF-R | 10 | 9 | 7 | 9 | 6 | 6 | 12 | 11 | 7 | 8 | 9 | 9 | 8 | 7 | 9 | 9 | 7
UnF-F1 | 10 | 14 | 5 | 7 | 8 | 5 | 11 | 11 | 9 | 7 | 8 | 9 | 10 | 8 | 9 | 8 | 5
AUC | 11 | 16 | 7 | 8 | 11 | 8 | 5 | 6 | 7 | 4 | 3 | 11 | 12 | 10 | 10 | 9 | 6
Acc | 10 | 14 | 3 | 4 | 8 | 7 | 10 | 11 | 12 | 10 | 11 | 7 | 8 | 6 | 8 | 7 | 5
Mac-P | 11 | 16 | 4 | 5 | 6 | 7 | 10 | 11 | 11 | 9 | 10 | 7 | 8 | 6 | 10 | 6 | 7
Mac-R | 11 | 16 | 7 | 8 | 11 | 8 | 5 | 6 | 7 | 4 | 3 | 11 | 12 | 10 | 10 | 9 | 6
Mac-F1 | 11 | 16 | 6 | 7 | 10 | 7 | 7 | 10 | 9 | 5 | 5 | 10 | 11 | 9 | 9 | 8 | 5
MCC | 12 | 16 | 6 | 6 | 10 | 8 | 7 | 9 | 9 | 4 | 4 | 10 | 11 | 9 | 10 | 8 | 5

####  4.1.2. RQ1.2: ML Performance Comparison among Methods

To compare different bias mitigation methods in terms of ML performance
degradation, we first calculate the proportions of scenarios, i.e., _(task-
algorithm pair, ML performance metric)_ combinations, that fall into different
effects after applying each bias mitigation method. Fig. 4 show the results.
We observe that only 7 (i.e., RW, DIR, AD, MFC-FDR, ROC-AOD, ROC-EOD, and
Fair-SMOTE) out of the 17 bias mitigation methods decrease ML performance
significantly in less than 50% of scenarios. In particular, RW significantly
decreases ML performance in only 33% of scenarios, with a large effect in only
13%. Meanwhile, we observe that some bias mitigation methods are at a
considerable cost of ML performance. For instance, LFR decreases ML
performance significantly with a large effect in 92% of scenarios.

Furthermore, for each ML performance metric, we follow previous work (Biswas
and Rajan, 2020) to compute the average rank of each bias mitigation method in
the 24 task-algorithm pairs. The smaller the rank value, the less the
corresponding method reduces the ML performance metric value. Table 3 shows
the results and highlights the top-ranked method for each metric. We observe
that for any ML performance metric, none of the bias mitigation methods can
consistently achieve better results than other methods in all the task-
algorithm pairs. For example, although RW and PR are the top-ranked methods
for F-P, the mean rank of them is only fifth. In addition, for different ML
performance metrics, the rank results vary a lot. This indicates that a bias
mitigation method may largely decrease ML performance in certain metrics. For
example, although MFC-FDR is the top-ranked method for F-R, it ranks 12th
among the 17 methods for F-P. This finding urges researchers and practitioners
to determine which ML performance metrics they should focus on according to
their intended goals. For instance, if one needs to perform bias mitigation in
a scenario where precision for the favorable class is important, they may pay
more attention to the F-P values achieved by different methods.

Finding 2: In terms of the 12 ML performance metrics, only 7 out of 17 (41%)
bias mitigation methods significantly decrease ML performance in less than 50%
of scenarios. Moreover, for any ML performance metric, none of the methods can
consistently achieve better results than other methods in all the scenarios.
Additionally, a bias mitigation method may largely decrease ML performance in
certain metrics. For example, MFC-FDR ranks fifth for F-R among the 17
methods, but it ranks 12th for F-P.

###  4.2. RQ2: Influence on Fairness

Similar to Section 4.1, this section dives into the evaluation results for
four fairness metrics (i.e., RQ2) by answering two specific questions:

RQ2.1 (Effects on fairness metric values): _How do the values of fairness
metrics change after applying bias mitigation methods?_ First, we investigate
whether the values of fairness metrics are significantly changed after
applying bias mitigation methods, and whether the changes in different
fairness metrics are significantly correlated.

RQ2.2 (Fairness comparison among methods): _How do different bias mitigation
methods affect ML software fairness?_ Second, we compare existing bias
mitigation methods in terms of different fairness metrics. The results provide
implications for the choice of bias mitigation methods in application
scenarios where fairness is critical.

Figure 5. RQ2.1: Effects of bias mitigation methods on different fairness
metrics. After applying existing bias mitigation methods, the values of the
four fairness metrics significantly decrease in an average of only 50% of
applications.

####  4.2.1. RQ2.1: Effects on Fairness Metric Values

We follow the procedures in Section 4.1.1 to analyze the value changes of the
four fairness metrics caused by bias mitigation methods, and present the
results in Fig. 5. We observe that after applying existing bias mitigation
methods, the values of the four fairness metrics (i.e., SPD, AOD, EOD, and
ERD) significantly decrease in only 59%, 55%, 57%, and 29% of applications,
with an average of 50%. Considering that the main goal of bias mitigation
methods is to improve software fairness (i.e, decrease the values of fairness
metrics), our results warn the research community about the limitations of
existing bias mitigation methods, especially in reducing ERD.

Furthermore, similar to Section 4.1.1, we calculate the correlation between
each two fairness metrics in terms of their value differences before and after
bias mitigation. Table 4 shows the results. We observe that the correlation
coefficient between AOD and EOD is 0.888 at a significant level, and the
significantly positive correlation holds in all the 24 task-algorithm pairs.
This suggests that bias mitigation methods that perform well in AOD may also
perform well in EOD, and that the evaluation results of bias mitigation
methods using AOD may also provide guidance for method selection in
application scenarios that pursue a low EOD. Therefore, it is not surprising
to find that some previous work (Hort et al., 2021) employ only one of them
for evaluation.

In other fairness metric pairs, we find that the correlations vary in
different task-algorithm pairs, which is consistent with the finding in
previous work (Biswas and Rajan, 2020) that analyzes the metric correlation in
only two bias mitigation tasks. For instance, we find that SPD and EOD have a
significantly positive correlation in only 15 out of the 24 task-algorithm
pairs.

Additionally, we observe that ERD has a negative correlation with other
fairness metrics overall. This indicates that bias mitigation methods that
decrease the ERD value often increase bias in terms of other three fairness
metrics. Moreover, ERD changes do not have a consistent correlation with
changes of any other metric across the task-algorithm pairs. This indicates
that other metrics may not act as a proxy of ERD. However, some existing work
(Zhang et al., 2019; Hort et al., 2021; Chakraborty et al., 2021, 2020a) does
not take ERD into account. To perform a comprehensive evaluation, we suggest
that researchers follow Biswas and Rajan (Biswas and Rajan, 2020, 2021) to
consider this metric in future work.

Finding 3: Existing bias mitigation methods improve fairness significantly (in
terms of SPD, AOD, EOD, and ERD) in an average of only 50% of applications,
according to the evaluation results of 17 methods and 8 bias mitigation tasks.
Additionally, fairness improvement measured by different metrics are not
necessarily correlated. In particular, ERD changes do not have a consistent
correlation with the changes of any other fairness metric.

Table 4. RQ2.1: Correlation between fairness metrics. In the table, * indicates a significant correlation ($p$-value $\textless$ 0.05) overall, and numbers in parentheses indicate the number of task-algorithm pairs the correlation shares the same pattern with the overall correlation. We find that AOD and EOD have a consistently positive correlation across all the task-algorithm pairs, and that ERD does not have a consistent correlation with any other metric. | AOD | EOD | ERD
---|---|---|---
SPD | 0.861*(23/24) | 0.651*(15/24) | -0.037(18/24)
AOD | - | 0.888*(24/24) | -0.101*(0/24)
EOD | - | - | -0.159*(4/24)
Figure 6. RQ2.2: Effect distribution of different bias mitigation methods in
fairness. We observe that 10 out of 17 (59%) bias mitigation methods
significantly reduce bias in more than 50% of scenarios.

####  4.2.2. RQ2.2: Fairness Comparison among Methods

Similar to Section 4.1.2, we calculate the proportions of scenarios, i.e.,
_(task-algorithm pair, fairness metric)_ combinations, that fall into
different effects after applying each bias mitigation method. Fig. 6 shows the
results. We find that only 10 out of the 17 bias mitigation methods can
significantly improve fairness in more than 50% of scenarios. In particular,
RW significantly improves fairness in the most scenarios (72%). The
superiority of RW in reducing bias is also observed in previous work (Biswas
and Rajan, 2020). As we find that LFR significantly decreases ML performance
with a large effect in the most scenarios in Section 4.1.2, it is not
surprising to observe that it also significantly improves fairness with a
large effect in the most scenarios (59%).

Table 5. RQ2.2: Mean rank of each bias mitigation method in terms of different fairness metrics. It is difficult for bias mitigation methods to achieve fairness with respect to all the metrics. Metric | OP | LFR | RW | DIR | PR | AD |  | MFC
---
-FDR
| MFC
---
-SR
| ROC
---
-SPD
| ROC
---
-AOD
| ROC
---
-EOD
| CEO
---
-FNR
| CEO
---
-FPR
| CEO
---
-W
EOP |  | Fair
---
-way
| Fair
---
-SMOTE
SPD | 7 | 2 | 3 | 8 | 5 | 9 | 13 | 11 | 4 | 8 | 11 | 10 | 11 | 14 | 4 | 10 | 12
AOD | 6 | 3 | 4 | 6 | 8 | 12 | 11 | 11 | 6 | 7 | 8 | 10 | 13 | 15 | 1 | 11 | 11
EOD | 6 | 4 | 6 | 7 | 8 | 13 | 10 | 8 | 9 | 5 | 6 | 11 | 11 | 15 | 3 | 11 | 10
ERD | 10 | 15 | 6 | 7 | 12 | 8 | 10 | 9 | 5 | 5 | 7 | 9 | 9 | 5 | 8 | 12 | 6

Next, we calculate the mean rank of each bias mitigation method in the 24
task-algorithm pairs for each fairness metric. Table 5 shows the results. For
SPD, the top-ranked method is LFR. However, we find that LFR achieves poor
fairness for ERD, ranking 15th among the 17 methods. Similarly, we observe the
different top-ranked bias mitigation methods for ERD and AOD/EOD. This
observation is consistent with the finding in previous work (Biswas and Rajan,
2020; Berk et al., 2021; Chouldechova, 2017) that it is difficult for bias
mitigation methods to achieve fairness with respect to all the metrics, and
for some pairs of fairness metrics, mathematically impossible.

Finding 4: 10 out of 17 (59%) bias mitigation methods significantly improve
fairness in more than 50% of scenarios. Moreover, it is difficult for bias
mitigation methods to achieve fairness with respect to all metrics that we
consider. For example, LFR is the top-ranked method for SPD, but it ranks 15th
among the 17 methods for ERD.

###  4.3. RQ3: Influence on Fairness-performance Trade-off

In this section, we present the measurement results of 24 types of fairness-
performance trade-offs, i.e., combinations of four fairness metrics (SPD, AOD,
EOD, and ERD) and six ML performance metrics (AUC, Acc, Mac-P, Macro-R,
Mac-F1, and MCC), achieved by different bias mitigation methods using Fairea
(Hort et al., 2021).

Figure 7. RQ3: Fairness-performance trade-off baselines for LR model in Adult-
Sex task. ERD shows a different trade-off pattern from other fairness metrics
in the baselines construted by Fairea.

As described in Section 3.4.3, the first step of Fairea is to construct the
trade-off baseline using a series of pseudo models generated via model
behavior mutation. For each _(task-algorithm pair, fairness-performance metric
pair)_ combination, we construct the baseline separately. As a result, we
construct a total of 8*3*4*6 = 576 baselines. Based on the 576 baselines, we
observe that the pseudo models show a fairness-performance trade-off for SPD,
AOD, and EOD, i.e., the higher the value of SPD, AOD, or EOD, the higher the
value of each ML performance metric. However, we fail to construct such trade-
off baselines for ERD. We take the baselines constructed for the LR model in
the Adult-Sex task (shown in Fig. 7) as an example. Based on the generated
pseudo models, ERD shows a different trade-off pattern from SPD, AOD, and EOD.
The different trade-off patterns shown by ERD and other fairness metrics can
be explained by their different definitions (i.e., calculation methods). Based
on their calculation methods presented in Section 3.4.2, we can find that when
all samples are predicted as the same label (i.e., the model achieves the
worst ML performance), SPD, AOD, and EOD all equal to 0 (their minimum), but
ERD equals to the difference of favorable rates between the privileged and
unprivileged groups (not its minimum). This means that ERD does not meet the
hypothesis behind Fairea, i.e., the bias measured by fairness metrics
monotonically decrease with the increased mutation degrees. Therefore, it is
reasonable that we fail to observe the trade-off between ERD and ML
performance based on the baselines constructed by Fairea.

Since Fairea fails to construct the trade-off baseline between ERD and ML
performance, the effectiveness region division method and the trade-off
quantification method provided by Fairea are also not applicable to ERD.
Therefore, we consider 18 types of fairness-performance trade-offs, i.e.,
combinations of three fairness metrics (SPD, AOD, and EOD) and six ML
performance metrics (AUC, Acc, Mac-P, Macro-R, Mac-F1, and MCC), and the
corresponding 8*3*3*6 = 432 baselines, in the rest of the paper.

Based on the measurement results of the 18 types of trade-offs, we take a deep
dive into RQ3 by answering two specific questions:

RQ3.1 (Effectiveness region distribution): _What effectiveness regions do
existing bias mitigation methods fall into according to Fairea?_ This research
question evaluates the overall effectiveness of existing bias mitigation
methods by analyzing how the mitigation cases achieved by them are matched
into the five effectiveness regions in Fig. 2(b).

RQ3.2 (Quantitative assessment of trade-off): _What fairness-performance
trade-off do existing bias mitigation methods achieved based on Fairea?_ This
research question evaluates existing bias mitigation methods in terms of
quantitative assessment of fairness-performance trade-off. Specifically, we
calculate the area shown in Fig. 2(a) for each bias mitigation method and then
compare these methods quantitatively.

Figure 8. RQ3.1: Proportion of mitigation cases that fall into each mitigation
region organized by (a) bias mitigation methods, (b) classification
algorithms, (c) bias mitigation tasks, and (d) fairness-performance metric
pairs. We observe that a notable proportion (37%) of mitigation cases fall
into a lose-lose or poor trade-off region, and that the effectiveness of bias
mitigation methods depends on models, tasks, and fairness and ML performance
metrics.

####  4.3.1. RQ3.1: Effectiveness Region Distribution

We use the 432 baselines to evaluate the effectiveness of 17 bias mitigation
methods in 24 task-algorithm pairs. As described in Section 3.5, we apply each
bias mitigation method to each task-algorithm pair 50 times and treat each
individual run as a mitigation case. Therefore, we have 8*3*50 = 1,200
mitigation cases for each bias mitigation method.444For the OP method, we have
6*3*50 = 900 mitigation cases, as we apply it in only six tasks. In Fig. 8(a),
we present the overall results of different bias mitigation methods for all
the task-algorithm pairs and fairness-performance metric pairs.

In Fig. 8(a), we observe that only 9 of the 17 bias mitigation methods achieve
a win-win or good trade-off in more than 50% of mitigation cases, including OP
(51%), RW (80%), DIR (74%), PR (62%), ROC-SPD (58%), ROC-AOD (63%), ROC-EOD
(57%), EOP (75%), and Fairway (62%). From this perspective, RW makes the best
trade-off in general, achieving a good trade-off in 54% of cases (ranging from
41% to 65% according to different fairness-performance metric pairs) and a
win-win trade-off in 26% (ranging from 21% to 38%). One possible reason behind
the superiority of RW is that it modifies the weights of different training
samples so that it tackles the training data imbalance problem, which has been
recognized as a root cause of bias in ML software (Chakraborty et al., 2021;
Mehrabi et al., 2021). Similarly, previous work (Hort et al., 2021) covering
12 bias mitigation methods and two types of trade-offs (i.e., SPD&Acc and
AOD&Acc) reports that only 6 out of the 12 methods can achieve a win-win or
good trade-off in more than 50% of cases, and that RW achieves such trade-offs
in 77% of cases.

Nevertheless, we still observe that 15% of mitigation cases fall into a lose-
lose or poor trade-off after applying RW, and the situation is even worse for
other bias mitigation methods. For example, CEO-W achieves a lose-lose trade-
off in 72% of cases; LFR achieves a poor trade-off in 40% of cases. Overall,
the average proportion of cases that fall into the lose-lose or poor trade-off
region obtained by each method is 26% and 11%, respectively. The notable
proportion of the lose-lose and poor trade-offs is also observed in previous
work (Hort et al., 2021), and our results increase the confidence of this
finding with more bias mitigation methods and fairness-performance metric
pairs. One possible reason behind the high percentage of the lose-lose and
poor trade-offs is that bias mitigation methods are often designed to optimize
one fairness metric, which may affect other fairness metrics. For instance,
ROC-SPD achieves a lose-lose or poor trade-off between AOD and Acc in 53% of
mitigation cases. However, we find that the percentage of lose-lose and poor
trade-offs can still be high even for the optimized metric. For example, after
applying ROC-AOD, 49% of mitigation cases fall into a lose-lose or poor trade-
off between AOD and Acc.

Furthermore, we organize the results by different classification algorithms,
bias mitigation tasks, and fairness-performance metric pairs, to analyze
whether the region distribution is influenced by these factors. The results
are presented in Figs. 8(b), (c), and (d), respectively, and the analysis is
as follows.

_Comparison among classification algorithms:_ From Fig. 8(b), we observe that
the region distribution is model-dependent. Although LR, SVM, and RF share
similar proportions of mitigation cases that fall into the lose-lose, poor,
and win-win trade-off regions, they have obvious differences in the
proportions of the inverted and good trade-offs. Specifically, LR and SVM
achieve a relatively higher percentage of the good trade-off (a lower
percentage of the inverted trade-off) than RF. For example, the difference
between LR and RF in the proportion of the good trade-off is 12% (i.e., 39%
v.s. 27%). Therefore, we suggest that researchers consider different
classification algorithms when evaluating the effectiveness of bias mitigation
methods.

_Comparison among tasks:_ From Fig. 8(c), we find that the region distribution
is task-dependent. For example, in the Compas-Sex task, the proportion of the
lose-lose trade-off is 12%, but in the Adult-Sex task, the corresponding
proportion is more than three times the value (i.e., 37%). Furthermore, we
observe that even for the same dataset, the selection of different protected
attributes also affects the region distribution. For example, the proportions
of the lose-lose trade-off in the Compas-Sex and Compas-Race tasks are 12% and
26%, with a difference of 14%. This finding suggests that (1) researchers
evaluate bias mitigation methods in diverse tasks to improve the
generalizability of the results, and (2) practitioners need to be careful when
choosing bias mitigation methods for their own tasks based on existing
evaluation results on other tasks.

_Comparison among fairness-performance metric pairs:_ From Fig. 8(d), we find
that different fairness-performance metric pairs result in different region
distributions. In particular, the mitigation cases that obtain a win-win or
good trade-off between AOD and Mac-P account for 39%, the lowest among all the
fairness-performance metric pairs. In contrast, the corresponding proportion
between EOD and AUC is 17% higher (i.e., 56%). This finding further
demonstrates the necessity of this study that measures the trade-off in terms
of various fairness-performance metric pairs to obtain more general and
comprehensive results.

Finding 5: Based on the results in terms of 18 types of fairness-performance
trade-offs, 9 out of 17 (53%) bias mitigation methods achieve a win-win or
good trade-off in more than 50% of mitigation cases according to Fairea.
Furthermore, a notable proportion of cases fall into a lose-lose trade-off
(26%) or a poor trade-off (11%) on average for each bias mitigation method.
The effectiveness of bias mitigation methods depends on models, tasks, and the
metrics of fairness and ML performance.

Table 6. RQ3.2: Numbers of scenarios where each method achieves the best quantitative result, organized by different bias mitigation tasks. Overall, none of existing bias mitigation methods can achieve a better trade-off than other methods in all the scenarios, as even the best method that we find outperforms other methods in only 29% of scenarios. Dataset-Attr | OP | LFR | RW | DIR | PR | AD |  | MFC
---
-FDR
| MFC
---
-SR
| ROC
---
-SPD
| ROC
---
-AOD
| ROC
---
-EOD
| CEO
---
-FNR
| CEO
---
-FPR
| CEO
---
-W
EOP |  | Fair
---
-way
| Fair
---
-SMOTE
Adult-Sex | 0% | 0% | 0% | 11% | 16% | 16% | 0% | 0% | 11% | 18% | 11% | 0% | 0% | 0% | 3% | 16% | 48%
Adult-Race | 0% | 0% | 0% | 5% | 0% | 14% | 7% | 7% | 25% | 31% | 18% | 0% | 0% | 0% | 3% | 37% | 44%
Compas-Sex | 50% | 0% | 9% | 27% | 33% | 0% | 11% | 0% | 9% | 3% | 0% | 0% | 0% | 0% | 0% | 33% | 33%
Compas-Race | 33% | 0% | 29% | 5% | 33% | 22% | 0% | 0% | 11% | 20% | 0% | 0% | 0% | 0% | 0% | 0% | 0%
German-Sex | 0% | 0% | 3% | 12% | 0% | 0% | 7% | 33% | 37% | 61% | 61% | 0% | 9% | 0% | 3% | 12% | 16%
German-Age | 0% | 0% | 11% | 20% | 0% | 0% | 0% | 38% | 33% | 50% | 44% | 0% | 5% | 0% | 0% | 0% | 18%
Bank-Age | - | 0% | 74% | 0% | 3% | 0% | 7% | 14% | 14% | 0% | 12% | 61% | 0% | 0% | 29% | 27% | 0%
Mep-Race | - | 0% | 24% | 16% | 33% | 16% | 14% | 0% | 25% | 48% | 40% | 16% | 0% | 0% | 16% | 16% | 27%
Overall | 13% | 0% | 18% | 12% | 15% | 8% | 6% | 11% | 21% | 29% | 23% | 9% | 1% | 0% | 7% | 18% | 23%

####  4.3.2. RQ3.2: Quantitative Assessment of Trade-off

We finally use Fairea to quantify the fairness-performance trade-off of
different bias mitigation methods. Fairea quantifies only the cases that fall
into the good trade-off region, as the other regions are either dominating the
original model (the win-win region), dominated by the Fairea baseline (the
poor trade-off region), or do not improve fairness (the inverted and lose-lose
trade-off regions) (Hort et al., 2021).

For each mitigation scenario, i.e., _(task-algorithm pair, fairness-
performance metric pair)_ combination, we calculate the mean ML performance
and fairness results of the 50 runs of each bias mitigation method to indicate
its average effectiveness. Then we quantify the trade-off effectiveness of
each method with the help of the trade-off baselines. The trade-off
quantification enables a direct and convenient comparison among different bias
mitigation methods. For example, in terms of the SPD&Acc trade-off for the RF
model in the Compas-Race task, the quantitative results of DIR, PR, ROC-SPD,
and Fairway are 0.065, the win-win trade-off, 0.176, and the inverted trade-
off, respectively. In this scenario, the four methods in descending order of
the quantitative results are PR, ROC-SPD, DIR, and Fairway. According to this
rule, for each bias mitigation task, we calculate the proportion of scenarios
where each bias mitigation method achieves the best quantitative result. Table
6 shows the result for each task and the overall result in all the tasks. We
highlight the highest proportion for each row (i.e., each bias mitigation
task). Note that the sum of the numbers in a row is larger than 100%, since
there may be multiple methods to achieve the best quantitative result in a
scenario.

At a glance of Table 6, we find that the distributions of the best results
vary in different bias mitigation tasks, further demonstrating the
aforementioned finding in Section 4.3.1 that the effectiveness of bias
mitigation methods is task-dependent. For instance, RW achieves the best
quantitative result in 74% of scenarios in the Bank-Age task, but the
corresponding proportion is 0% in the Adult-Sex and Adult-Race tasks.

Furthermore, we compare the overall quantitative results achieved by different
bias mitigation methods. From this perspective, the top five methods are ROC-
AOD (achieving the best quantitative result in 29% of all the scenarios),
Fair-SMOTE and ROC-EOD (both 23%), ROC-SPD (21%), Fairway and RW (both 18%).
Given that the highest proportion is only 29%, we conclude that there is no
‘silver bullet’ method that can achieve superior results than other methods in
most of scenarios.

Finding 6: According to the quantitative assessment results using Fairea for
18 types of fairness-performance trade-offs, there is no ‘silver bullet’ bias
mitigation method that can achieve the best trade-off in all the scenarios we
studied, as even the best method that we observe outperforms other methods in
only 29% of the scenarios.

##  5\. Threats to Validity

The primary _threat to internal validity_ lies in the implementation of the
code used in this study. To mitigate this threat, we conduct the experiments
based on the widely-adopted IBM AIF360 framework and the released code of
Fairway, Fair-SMOTE, and Fairea. We also make all scripts and data publicly
available to allow for reproductions and replications.

The _threat to external validity_ concerns the generalizability of our
experimental results. To alleviate this threat, we adopt eight bias mitigation
tasks, which cover social, financial, and medical domains, and are well
adopted in the fairness literature of ML and SE. Moreover, we employ 17
representative bias mitigation methods proposed in the ML and SE communities.
For each method, we use three traditional ML algorithms for implementation,
reducing the influence of algorithm selection on the generalization of
results. In terms of evaluation measures, we use 12 ML performance metrics, 4
fairness metrics, and 24 types of fairness-performance trade-offs to obtain
comprehensive measurement results. The 12 ML performance metrics measure not
only the performance on individual class, but also the overall performance;
the 4 fairness metrics are the most widely adopted in previous fairness work;
the 24 types of fairness-performance trade-offs are considerable given that
previous work uses only two types. Nevertheless, with increasing attention on
software fairness, researchers are proposing more and more bias mitigation
methods, fairness metrics, and datasets. In the future, one could replicate
this study with more methods, metrics, measures, and datasets. In addition, we
follow previous work (Hort et al., 2021; Biswas and Rajan, 2020, 2021) to
treat each protected attribute individually for each bias mitigation task,
since most of existing bias mitigation methods do not support dealing with
multiple protected attributes at the same time. In the future, when more
methods that support this functionality are proposed, one could evaluate them
with multiple protected attributes considered at the same time.

##  6\. Conclusion

This paper presents a large-scale empirical study evaluating 17 representative
bias mitigation methods with 12 ML performance metrics, 4 fairness metrics,
and 24 types of fairness-performance trade-offs for 8 widely adopted benchmark
tasks. The results of our comprehensive study reveal a series of findings. In
particular, we find that (1) the bias mitigation methods significantly
decrease the values of all ML performance metrics (including those not
considered in previous work) in a notable proportion of scenarios
(42%$\sim$75% according to different metrics); (2) the bias mitigation methods
achieve fairness improvement in only 50% of scenarios (29%$\sim$59% according
to different fairness metrics); (3) the bias mitigation methods have a poor
fairness-performance trade-off, or even lead to decreases in both fairness and
ML performance in 37% of scenarios; (4) the effectiveness of the bias
mitigation methods depends on tasks, models, and fairness and ML performance
metrics, and there is no ‘silver bullet’ bias mitigation method that works for
all scenarios. The best bias mitigation method that we find outperforms other
methods in only 29% of scenarios. We have made publicly available the scripts
and data used in this study for other researchers to replicate and extend this
work.

## References

  * (1)
  * adu ([n.d.]) [n.d.].  The Adult Census Income dataset.  https://archive.ics.uci.edu/ml/datasets/adult.  Retrieved on September 20, 2021.
  * ban ([n.d.]) [n.d.].  The Bank dataset.  https://archive.ics.uci.edu/ml/datasets/Bank+Marketing.  Retrieved on September 20, 2021.
  * com ([n.d.]) [n.d.].  The Compas dataset.  https://github.com/propublica/compas-analysis.  Retrieved on September 20, 2021.
  * fat ([n.d.]) [n.d.].  FATE: Fairness, Accountability, Transparency, and Ethics in AI.  https://www.microsoft.com/en-us/research/theme/fate/.  Retrieved on September 20, 2021.
  * ger ([n.d.]) [n.d.].  The German Credit dataset.  https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29.  Retrieved on September 20, 2021.
  * fai ([n.d.]a) [n.d.]a.  The GitHub repository of Fair-SMOTE.  https://github.com/joymallyac/Fair-SMOTE/tree/master/Fair-SMOTE.  Retrieved on September 20, 2021.
  * fai ([n.d.]b) [n.d.]b.  The GitHub repository of Fairway.  https://github.com/joymallyac/Fairway.  Retrieved on September 20, 2021.
  * aif ([n.d.]) [n.d.].  IBM AI Fairness 360.  https://aif360.mybluemix.net.  Retrieved on September 20, 2021.
  * fai ([n.d.]c) [n.d.]c.  Machine Bias.  https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.  Retrieved on September 20, 2021.
  * mep ([n.d.]) [n.d.].  The Mep dataset.  https://meps.ahrq.gov/mepsweb/data_stats/download_data_files_detail.jsp?cboPufNumber=HC-181.  Retrieved on September 20, 2021.
  * mic ([n.d.]) [n.d.].  Microsoft AI principles.  https://www.microsoft.com/en-us/ai/responsible-ai?activetab=pivot1%3aprimaryr6.  Retrieved on September 20, 2021.
  * skl ([n.d.]) [n.d.].  Scikit-learn.  https://scikit-learn.org.  Retrieved on September 20, 2021.
  * fai ([n.d.]d) [n.d.]d.  Semantics derived automatically from language corpora contain human-like biases.  https://www.science.org/doi/10.1126/science.aal4230.  Retrieved on September 20, 2021.
  * fai ([n.d.]e) [n.d.]e.  Study finds gender and skin-type bias in commercial artificial-intelligence systems.  https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212.  Retrieved on September 20, 2021.
  * fai ([n.d.]f) [n.d.]f.  When good algorithms go sexist: why and how to advance AI gender equity.  https://ssir.org/articles/entry/when_good_algorithms_go_sexist_why_and_how_to_advance_ai_gender_equity.  Retrieved on September 20, 2021.
  * Aggarwal et al. (2019) Aniya Aggarwal, Pranay Lohia, Seema Nagar, Kuntal Dey, and Diptikalyan Saha. 2019.  Black box fairness testing of machine learning models. In _Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019_. 625–635.
  * Agrawal and Menzies (2018) Amritanshu Agrawal and Tim Menzies. 2018.  Is ”better data” better than ”better data miners”?: on the benefits of tuning SMOTE for defect prediction. In _Proceedings of the 40th International Conference on Software Engineering, ICSE 2018_. 1050–1061.
  * Berk et al. (2021) Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2021.  Fairness in criminal justice risk assessments: The state of the art.  _Sociological Methods & Research_ 50, 1 (2021), 3–44.
  * Biswas and Rajan (2020) Sumon Biswas and Hridesh Rajan. 2020.  Do the machine learning models on a crowd sourced platform exhibit bias? An empirical study on model fairness. In _Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020_. 642–653.
  * Biswas and Rajan (2021) Sumon Biswas and Hridesh Rajan. 2021.  Fair preprocessing: towards understanding compositional fairness of data transformers in machine learning pipeline. In _Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021_. 981–993.
  * Brun and Meliou (2018) Yuriy Brun and Alexandra Meliou. 2018.  Software fairness. In _Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018_. 754–759.
  * Calders et al. (2009) Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. 2009\.  Building classifiers with independency constraints. In _Proceedings of the 2009, IEEE International Conference on Data Mining_. 13–18.
  * Calders and Verwer (2010) Toon Calders and Sicco Verwer. 2010.  Three naive Bayes approaches for discrimination-free classification.  _Data Mining and Knowledge Discovery_ 21, 2 (2010), 277–292.
  * Celis et al. (2019) L. Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nisheeth K. Vishnoi. 2019.  Classification with fairness constraints: a meta-algorithm wit provable guarantees. In _Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019_. 319–328.
  * Chakraborty et al. (2021) Joymallya Chakraborty, Suvodeep Majumder, and Tim Menzies. 2021\.  Bias in machine learning software: why? how? what to do?. In _Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021_. 429–440.
  * Chakraborty et al. (2020a) Joymallya Chakraborty, Suvodeep Majumder, Zhe Yu, and Tim Menzies. 2020a.  Fairway: a way to build fair ML software. In _Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020_. 654–665.
  * Chakraborty et al. (2020b) Joymallya Chakraborty, Kewen Peng, and Tim Menzies. 2020b.  Making fair ML software using trustworthy explanation. In _Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020_. 1229–1233.
  * Chen et al. (2019) Zhenpeng Chen, Yanbin Cao, Xuan Lu, Qiaozhu Mei, and Xuanzhe Liu. 2019.  SEntiMoji: an emoji-powered learning approach for sentiment analysis in software engineering. In _Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESECFSE 2019_. 841–852.
  * Chen et al. (2021) Zhenpeng Chen, Yanbin Cao, Huihan Yao, Xuan Lu, Xin Peng, Hong Mei, and Xuanzhe Liu. 2021.  Emoji-powered sentiment and emotion detection from software developers’ communication data.  _ACM Transactions on Software Engineering and Methodology_ 30, 2 (2021), 18:1–18:48.
  * Chen et al. ([n.d.]) Zhenpeng Chen, Jie M. Zhang, Federica Sarro, and Mark Harman. [n.d.].  Replication package for this paper.  https://github.com/chenzhenpeng18/BiasMitigationStudy.  Retrieved on December 2, 2021.
  * Chicco and Jurman (2020) Davide Chicco and Giuseppe Jurman. 2020.  The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation.  _BMC genomics_ 21, 1 (2020), 1–13.
  * Chouldechova (2017) Alexandra Chouldechova. 2017\.  Fair prediction with disparate impact: a study of bias in recidivism prediction instruments.  _Big data_ 5, 2 (2017), 153–163.
  * Cohen (2013) Jacob Cohen. 2013\.  _Statistical power analysis for the behavioral sciences_.  Academic press.
  * du Pin Calmon et al. (2017) Flávio du Pin Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R. Varshney. 2017.  Optimized pre-processing for discrimination prevention. In _Proceedings of the Annual Conference on Neural Information Processing Systems 2017, NIPS 2017_. 3992–4001.
  * Feldman et al. (2015a) Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015a.  Certifying and removing disparate impact. In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 259–268.
  * Feldman et al. (2015b) Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015b.  Certifying and removing disparate impact. In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2015_. 259–268.
  * Finkelstein et al. (2008) Anthony Finkelstein, Mark Harman, Afshin Mansouri, Jian Ren, and Yuanyuan Zhang. 2008.  Fairness analysis in requirements assignments. In _Proceedings of the 16th IEEE International Requirements Engineering Conference_. 115–124.
  * Galhotra et al. (2017) Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017\.  Fairness testing: testing software for discrimination. In _Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017_. 498–510.
  * Hardt et al. (2016) Moritz Hardt, Eric Price, and Nati Srebro. 2016.  Equality of opportunity in supervised learning. In _Proceedings of the Annual Conference on Neural Information Processing Systems 2016, NIPS 2016_. 3315–3323.
  * Hort and Sarro (2021) Max Hort and Federica Sarro. 2021.  Did you do your homework? Raising awareness on software fairness and discrimination. In _Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021_.
  * Hort et al. (2021) Max Hort, Jie M. Zhang, Federica Sarro, and Mark Harman. 2021\.  Fairea: a model behaviour mutation approach to benchmarking bias mitigation methods. In _Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, ESEC/FSE 2021_. 994–1006.
  * Kamiran and Calders (2011) Faisal Kamiran and Toon Calders. 2011.  Data preprocessing techniques for classification without discrimination.  _Knowledge and Information Systems_ 33, 1 (2011), 1–33.
  * Kamiran et al. (2010) Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. 2010\.  Discrimination Aware Decision Tree Learning. In _Proceedings of the 10th IEEE International Conference on Data Mining, ICDM 2010_. 869–874.
  * Kamiran et al. (2012) Faisal Kamiran, Asim Karim, and Xiangliang Zhang. 2012\.  Decision theory for discrimination-aware classification. In _Proceedings of the 12th IEEE International Conference on Data Mining, ICDM 2012_. 924–929.
  * Kamiran et al. (2018) Faisal Kamiran, Sameen Mansha, Asim Karim, and Xiangliang Zhang. 2018.  Exploiting reject option in classification for social discrimination control.  _Information Science_ 425 (2018), 18–33.
  * Kamishima et al. (2012) Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012.  Fairness-aware classifier with prejudice remover regularizer. In _Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML/PKDD 2012_. 35–50.
  * Li et al. (2015) Cheng Li, Yue Lu, Qiaozhu Mei, Dong Wang, and Sandeep Pandey. 2015.  Click-through Prediction for Advertising in Twitter Timeline. In _Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2015_. 1959–1968.
  * Mann and Whitney (1947) Henry B Mann and Donald R Whitney. 1947.  On a test of whether one of two random variables is stochastically larger than the other.  _The Annals of Mathematical Statistics_ (1947), 50–60.
  * Mehrabi et al. (2021) Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021\.  A survey on bias and fairness in machine learning.  _Comput. Surveys_ 54, 6 (2021), 115:1–115:35.
  * Myers et al. (2013) Jerome L Myers, Arnold D Well, and Robert F Lorch Jr. 2013\.  _Research design and statistical analysis_.  Routledge.
  * Nam and Kim (2015) Jaechang Nam and Sunghun Kim. 2015.  CLAMI: Defect Prediction on Unlabeled Datasets (T). In _Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015_. 452–463.
  * Novielli et al. (2020) Nicole Novielli, Fabio Calefato, Davide Dongiovanni, Daniela Girardi, and Filippo Lanubile. 2020\.  Can we use SE-specific sentiment analysis tools in a cross-platform setting?. In _Proceedings of the 17th International Conference on Mining Software Repositories, MSR 2020_. 158–168.
  * Novielli et al. (2018) Nicole Novielli, Daniela Girardi, and Filippo Lanubile. 2018\.  A benchmark study on sentiment analysis for software engineering research. In _Proceedings of the 15th International Conference on Mining Software Repositories, MSR 2018_. 364–375.
  * Pleiss et al. (2017) Geoff Pleiss, Manish Raghavan, Felix Wu, Jon M. Kleinberg, and Kilian Q. Weinberger. 2017\.  On fairness and calibration. In _Proceedings of the Annual Conference on Neural Information Processing Systems 2017, NIPS 2017_. 5680–5689.
  * Rahman et al. (2012) Foyzur Rahman, Daryl Posnett, and Premkumar T. Devanbu. 2012\.  Recalling the ”imprecision” of cross-project defect prediction. In _Proceedings of the 20th ACM SIGSOFT Symposium on the Foundations of Software Engineering, SIGSOFT/FSE 2012_. 61.
  * Rodríguez et al. (2014) Daniel Rodríguez, Israel Herraiz, Rachel Harrison, José Javier Dolado, and José C. Riquelme. 2014.  Preliminary comparison of techniques for dealing with imbalance in software defect prediction. In _Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE 2014_. 43:1–43:10.
  * Sawilowsky (2009) Shlomo S Sawilowsky. 2009\.  New effect size rules of thumb.  _Journal of Modern Applied Statistical Methods_ 8, 2 (2009), 26\.
  * Sebastiani (2002) Fabrizio Sebastiani. 2002\.  Machine learning in automated text categorization.  _Comput. Surveys_ 34, 1 (2002), 1–47.
  * Speicher et al. (2018) Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P. Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018\.  A unified approach to quantifying algorithmic unfairness: measuring individual &group unfairness via inequality indices. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018_. 2239–2248.
  * Udeshi et al. (2018) Sakshi Udeshi, Pryanshu Arora, and Sudipta Chattopadhyay. 2018\.  Automated directed fairness testing. In _Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018_. 98–108.
  * Wick et al. (2019) Michael L. Wick, Swetasudha Panda, and Jean-Baptiste Tristan. 2019.  Unlocking fairness: a trade-off revisited. In _Proceedings of the Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019_. 8780–8789.
  * Yao and Shepperd (2020) Jingxiu Yao and Martin J. Shepperd. 2020.  Assessing software defection prediction performance: why using the Matthews correlation coefficient matters. In _Proceedings of Evaluation and Assessment in Software Engineering, EASE 2020_. 120–129.
  * Zafar et al. (2017) Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. 2017.  Fairness constraints: mechanisms for fair classification. In _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017_. 962–970.
  * Zemel et al. (2013) Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. 2013.  Learning fair representations. In _Proceedings of the 30th International Conference on Machine Learning, ICML 2013_. 325–333.
  * Zhang et al. (2018) Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018\.  Mitigating unwanted biases with adversarial learning. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2018_. 335–340.
  * Zhang and Harman (2021) Jie M. Zhang and Mark Harman. 2021.  Ignorance and prejudice in software fairness. In _Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021_. 1436–1447.
  * Zhang et al. (2019) Jie M. Zhang, Mark Harman, Lei Ma, and Yang Liu. 2019\.  Machine learning testing: survey, landscapes and horizons.  _IEEE Transactions on Software Engineering_ (2019).
  * Zhang et al. (2020) Peixin Zhang, Jingyi Wang, Jun Sun, Guoliang Dong, Xinyu Wang, Xingen Wang, Jin Song Dong, and Ting Dai. 2020\.  White-box fairness testing through adversarial sampling. In _Proceedings of the 42nd International Conference on Software Engineering, ICSE 2020_. 949–960.