backgroundcolor=, commentstyle=, breakatwhitespace=false, keywordstyle=,
language=Java, frame=single, stringstyle=, basicstyle=, captionpos=b,
showstringspaces=false  backgroundcolor=, commentstyle=,
breakatwhitespace=false, keywordstyle=, language=Python, stringstyle=,
basicstyle=, frame=single, captionpos=t, showstringspaces=false

11institutetext: Giordano d’Aloisio 22institutetext: University of L’Aquila,
Italy
22email<EMAIL_ADDRESS>33institutetext: Claudio Di
Sipio 44institutetext: University of L’Aquila, Italy
44email<EMAIL_ADDRESS>55institutetext: Antinisca Di Marco
66institutetext: University of L’Aquila, Italy
66email<EMAIL_ADDRESS>77institutetext: Davide Di Ruscio
88institutetext: University of L’Aquila, Italy
88email<EMAIL_ADDRESS>
# How fair are we? From conceptualization to automated assessment of fairness
definitions

Giordano d’Aloisio  Claudio Di Sipio  Antinisca Di Marco  Davide Di Ruscio

###### Abstract

Fairness is a critical concept in ethics and social domains, but it is also a
challenging property to engineer in software systems. With the increasing use
of machine learning in software systems, researchers have been developing
techniques to automatically assess the fairness of software systems.
Nonetheless, a significant proportion of these techniques rely upon pre-
established fairness definitions, metrics, and criteria, which may fail to
encompass the wide-ranging needs and preferences of users and stakeholders. To
overcome this limitation, we propose a novel approach, called MODNESS, that
enables users to customize and define their fairness concepts using a
dedicated modeling environment. Our approach guides the user through the
definition of new fairness concepts also in emerging domains, and the
specification and composition of metrics for its evaluation. Ultimately,
MODNESS generates the source code to implement fair assessment based on these
custom definitions. In addition, we elucidate the process we followed to
collect and analyze relevant literature on fairness assessment in software
engineering (SE). We compare MODNESS with the selected approaches and evaluate
how they support the distinguishing features identified by our study. Our
findings reveal that i) most of the current approaches do not support user-
defined fairness concepts; ii) our approach can cover two additional
application domains not addressed by currently available tools, i.e.,
mitigating bias in recommender systems for software engineering and Arduino
software component recommendations; iii) MODNESS demonstrates the capability
to overcome the limitations of the only two other Model-Driven Engineering-
based approaches for fairness assessment.

###### Keywords:

Fairness assessment, Model-Driven engineering, Bias and Fairness definition

##  1 Introduction

In recent years, machine learning (ML) has become increasingly widespread and
popular. ML is the backbone of many systems we use daily, such as intelligent
speakers and e-commerce systems that provide personalized recommendations.
However, creating data-driven decision-making systems using ML can be complex
due to the needed vast amount of training data, the complexity of the
information relationships, and continuous learning activities that aim to
improve the accuracy of such systems giray_software_2021 ; nahar_meta-
summary_2023 . Unfortunately, these factors make it challenging for developers
to ensure that these systems are free from bias. To address this issue, the
scientific community has begun evaluating ML-intensive systems in terms of
quality attributes such as explainability, privacy, and fairness
giray_software_2021 ; villamizar_requirements_2021 ; muccini_software_2021 .
Among these quality attributes, fairness gained considerable relevance in the
SE community giray_software_2021 ; villamizar_requirements_2021 ;
muccini_software_2021 ; kumeno_sofware_2020 ; zhang_machine_2020 . The
relevance of fairness is also highlighted by the AI Act proposal of the
European Commission, where it is described as a key quality property for high-
risk AI-enabled systems ai-act .

Fairness (and relative unbias) can be defined as ”the absence of any prejudice
or favouritism toward an individual or group based on their inherent or
acquired characteristics” mehrabi_survey_2021 . Although the concept of
fairness has been introduced primarily in the context of machine learning (ML)
systems (mainly for legal reasons mehrabi_survey_2021 ; caton_fairness_2020 ),
the concept of bias originates in the ethical domain (referring to the general
concept of discrimination), and has been adapted to the AI and ML domain
olteanu2019social . The relevance of software fairness in ML-intensive systems
has also been made popular by infamous incidents in the recruitment instrument
employed by Amazon111https://shorturl.at/jptx2 and the criminal recidivism
predictions made by the commercial risk assessment software COMPAS
angwin_machine_2016 .

In general, an ML system is said to be biased (or unfair) if the output of the
ML model is directly correlated to the value of a set of so-called sensitive
variables, like the race or gender of a person. Starting from a definition of
bias for a particular domain, several fairness analyses can be computed by
selecting a set of metrics compliant with a particular scope. The process of
analyzing if a given system produces fair outcomes involves four steps
mehrabi_survey_2021 : _bias definition_ , _fairness analysis specification_ ,
_analysis implementation_ , and _fairness assessment_. However, performing
these steps manually can be challenging and error-prone, requiring users with
different expertise levels. To address this issue, several toolkits and
frameworks have been developed to automate the fairness analysis of ML-based
systems by i) identifying the sources of bias and ii) suggesting appropriate
interventions. These tools are helpful, but they have some limitations: they
rely on predefined fairness definitions that may not suit every application
domain, and they do not allow users to customize the concepts of bias,
fairness, and the corresponding metrics. Moreover, as recent studies and
surveys have shown mehrabi_survey_2021 ; caton_fairness_2020 , there is no
consensus on defining and measuring fairness across different domains.
Therefore, there is a need for dedicated languages that enable users to
specify their fairness criteria and metrics lee_landscape_2021 . Furthermore,
while different types of bias have been studied in specific domains (such as
recommender systems deldjoo2022fairness ; wang_survey_2023 ), ethical, legal,
or social factors still influence the notion of bias.

In this paper, we introduce MODNESS, a versatile tool that helps to
conceptualize, personalize, and assess fairness definitions across different
domains. It is based on the Model-Driven Engineering (MDE) paradigm and
features a specialized metamodel that caters to two levels of abstraction: the
conceptual and the development level. With the former, users can define
critical fairness concepts such as bias and metrics to detect it independently
of the application domain. The latter utilizes such a definition to
automatically evaluate and generate code modules using real-world datasets.

The main contributions of the paper are the following:

  * •

We carefully review existing tools and approaches that automatically detect
group and individual bias;

  * •

We propose a tailored metamodel222The full source code, as well as the
replication package of the performed experiments, are available at
https://bit.ly/3RhH4y7 that is capable of representing existing fairness
definitions and providing an extensible mechanism to conceptualize new ones;

  * •

We evaluate the proposed approach by modeling a set of use cases belonging to
different application domains, i.e., social, financial, software development,
and IoT;

  * •

We compare the proposed approach with the only two other MDE-based baselines,
highlighting how MODNESS can overcome their current limitations.

This paper is organized as follows: Section 2 provides an overview of
terminology and the peculiar phases of fairness assessment workflows. Section
3 reviews existing approaches and tools for fairness analysis presented in
premiere SE venues. The MODNESS approach and supporting tools are presented in
Section 4. In Section 5, we show applications of MODNESS on different use
cases. Treats to the validity of the performed evaluation are discussed in
Section 6. Section 7 concludes the paper and describes future directions.

##  2 Background and Motivation

In this section, we recall the main concepts of bias and fairness and describe
a general process for fairness assessment. As previously stated, an ML system
is said to be biased if its results are closely associated with certain
sensitive variables. To explain this concept more clearly, we will refer to a
case presented in austin_will_2016  throughout this section. The case involves
a university that uses a machine learning system to determine student
admissions and wants to ensure that the system is fair. Specifically, the
concern is whether or not the system is biased against women.

Figure 1: General fairness assessment workflow

Figure 1 shows a general fairness assessment workflow highlighting, for each
step, the main concepts and the main involved actors. The workflow has been
derived by analyzing the behavior of some of the most adopted fairness
toolkits and libraries (i.e., Aequitas 10.1145/3238147.3238165 , IBM AIF360
bellamy_ai_2019 , and Fairlearn bird2020fairlearn ) and by reviewing
foundational papers on bias and fairness mehrabi_survey_2021 ;
brun_software_2018 ; chen_fairness_2022 .

When defining bias and conducting fairness analysis, it is important to
consider the specific domain being studied, as highlighted in recent research
mehrabi_survey_2021 ; caton_fairness_2020 . The fairness assessment process
may involve multiple parties, with three key actors typically identified:

  * –

Domain expert provides domain-specific knowledge (e.g., in our example, the
person in charge of managing admissions to the university);

  * –

Data scientist provides knowledge about fairness metrics and their
implementation;

  * –

Legal expert provides knowledge about specific regulations if needed in the
fairness assessment process.

The first step in the workflow depicted in Fig. 1 is the bias definition,
which involves both the domain and legal experts. In general, bias can be of
two types: mehrabi_survey_2021 ; caton_fairness_2020 :

  * –

Group bias: if the discrimination is performed at the group level.

  * –

Individual bias: if the discrimination is performed for a singular individual.

To establish a definition of bias within a specific domain, it is crucial to
pinpoint the following key concepts, as outlined in mehrabi_survey_2021 :

  * –

Sensitive variables: these are the variables that have the potential to lead
to discrimination;

  * –

Positive outcome: this refers to a specific prediction generated by the ML
system that could potentially result in discriminatory consequences;

  * –

Privileged and Unprivileged groups: These are the groups or entities
identified based on specific values of the sensitive variables. Privileged
groups are favoured by the system, while unprivileged groups face
discrimination from the system.

In the explanatory scenario, domain and legal experts want to assess if the ML
system under analysis discriminates against women. Thus, the type of examined
bias is group bias, with the sensitive variable being gender. A positive
outcome would be cheerful admission to the university, and the privileged
group is men while the unprivileged one is women.

From an abstract bias definition, several fairness analyses can be depicted.
In general, a fairness analysis has a scope (i.e., a fairness definition), a
set of metrics compliant to the given scope (which can be fairness metrics
known in the literature or can be custom ones), and has a dataset, which
contains all the information needed to perform the given analysis. In the
considered scenario, the domain and legal experts decide that the ML system is
fair if men and women have the same probability of being admitted to the
university. This is the scope of the analysis. Hence, the data scientist
suggests using the Statistical Parity fairness metric
kusner_counterfactual_2017 , a metric compliant with the given fairness
definition. Finally, the domain expert and the data scientist collect all the
needed information (i.e., the predictions of the ML model and the gender of
each person) inside a dataset that will be used for the analysis.

It is important to mention that other analyses can be conducted. For example,
domain and legal experts may want to examine whether someone is only admitted
to a university based on their high school grades, regardless of gender.
Hence, other metrics (e.g., Equalized Odds hardt_equality_2016 ) can be used
to cover this fairness definition, and further information has to be provided
in the dataset.

The next step in the fairness assessment process is the implementation of the
defined fairness analyses. Concretely, this means implementing an automatic
procedure that, given a specific dataset as input, computes all the selected
metrics and returns the calculated values. In our example, the data scientist
has to implement a script using a programming language (e.g., Python) that
takes as input the dataset and computes the Statistical Parity fairness
metric.

Once the results are computed, all parties must evaluate them to determine the
system’s fairness. For example, in our scenario, the data scientist notes that
a Statistical Parity score of zero indicates fairness. Next, legal and domain
experts analyze the results to assess whether the ML system is fair based on
this definition.

The process depicted in Fig. 1 and described earlier can be challenging and
prone to mistakes, mainly because it involves several stakeholders and
interconnected tasks. To address these issues, we introduce a tool-based
approach that automates the assessment of fairness. This approach enables
users to define their own notions of bias and fairness, enlarging the
applicability of fairness assessment to multiple application domains. We
present the details of this approach in the following sections.

##  3 Existing approaches for fairness assessment

This section presents the semi-automated procedure exploited to gather
pertinent literature within the realm of fairness assessment. First, we
describe the adopted procedure in Section 3.1, including the search string and
the inclusion and exclusion criteria. Subsequently, Section 3.2 delves into an
exploration of key features and sub-features constituting the fairness
assessment workflow, while Section 3.3 provides an overview of the collected
approaches.

###  3.1 Methodology

In this section, we provide an overview of prominent approaches within the
domain of bias and fairness assessment in machine learning-based systems.
Please note that our aim is not to present a comprehensive survey of the
entire field, as it goes beyond the scope of this paper. Instead, we have
adopted a semi-automated approach inspired by the well-established ”four
W-question strategy” 10.1016/j.infsof.2010.12.010  to select existing
approaches that perform bias detection and mitigation using automated or semi-
automated methods. The four W-questions guiding our approach are as follows:

  * •

_Which?_ We conducted a comprehensive search, combining both automated and
manual methods, to gather relevant papers from a variety of sources, including
conferences and journals.

  * •

_Where?_ Our literature analysis focused on prominent software engineering
venues, encompassing nine conferences: ASE, ESEC/FSE, ESEM, ICSE, ICSME, ICST,
ISSTA, MSR, and SANER, as well as five journals: EMSE, IST, JSS, TOSEM, and
TSE. To automate this process, we utilized the Scopus database
333https://scopus.com and employed advanced search and export functions to
retrieve all papers published in specific venues within the temporal range we
decided.

  * •

_What?_ For each article, we extracted information from the title and abstract
by applying predefined keywords to ensure relevance to our research focus.

  * •

_When?_ Given that automated fairness assessment is a relatively recent
research area, our search was limited to the most recent five years, spanning
from 2017 to 2022. This temporal constraint allowed us to capture the latest
developments and trends in the field. It is worth noticing how the query was
executed in April of 2023, hence 2023 has not been considered.

⬇

TITLE-ABS((’’fairness’’ OR ’’bias’’) AND

(’’data science’’ OR ’’machine learning’’) AND

(‘‘toolkit’’ OR ’’definition’’ OR ’’audit’’ OR ’’testing’’) AND

(LIMIT-TO (PUBYEAR, 2022) OR LIMIT-TO (PUBYEAR, 2017))

Listing 1: The Scopus search string. Table 1: Number of papers for the related topics. | FAIR | ML | TOOL
---|---|---|---
FAIR | 201 |  |
ML | 31 | 690 |
TOOL | 37 | 123 | 1,898

Listing 1 represents the search string we executed on the Scopus database by
using three sets of keywords as follows: _(i)_ FAIR: “ _fairness_ ” or “
_bias_ ”; _(ii)_ ML: “ _data science_ ” or “ _machine learning_ ; _(iii)_
TOOL: “ _toolkit_ ,” or “ _definition_ ”, or “ _audit_ ”, or “ _testing_ ”.
Table 1 reports the number of papers that contain such keywords in the
corresponding column and row. Our ideal targets are papers containing at least
one keyword for all the sets of terms, i.e., FAIR, ML, and TOOL. By running
such a combination, we obtain only 15 works considering the abovementioned
criteria. Therefore, we enlarged the set of eligible papers to two additional
combinations highlighted in green, _(i)_ FAIR and ML; or _(ii)_ FAIR and TOOL.
In the end, we obtained a total number of 68 scientific papers, including
journal and conference publications.

Starting from this initial set of works, we manually inspected the title and
abstract to scale down the search to meet our requirements. In particular, we
defined the following inclusion and exclusion criteria:

✔ Inclusion criteria: We included all the approaches that use traditional bias
definitions, i.e., group or individual. Furthermore, we consider toolkits or
frameworks that provide an automatic or semi-automatic strategy to assess it
on a set of use cases. Some of them have been published as extensions of
initial works. Therefore, we consider the most recent version of the tool in
such cases.

✖ Exclusion criteria: The study we conducted intentionally excludes
foundational papers that primarily provide a high-level abstract definition of
fairness, such as surveys, empirical studies, or position papers.
Additionally, we have omitted short papers from our comparison that only offer
preliminary results and do not comprehensively cover bias definitions and
metrics.

As mentioned previously, our analysis is confined to peer-reviewed scientific
works that make significant contributions in two key areas: i) defining or
addressing fairness concerns within software systems, and ii) employing
automated methods to mitigate identified biases while considering notable
datasets. To ensure an unbiased selection process, we employed a rigorous
approach. Two different authors independently evaluated all the papers, and
the two senior co-authors thoroughly reviewed the entire selection process.
Finally, we also included two papers from MODELS 2022 10.1145/3550355.3552401
and FASE 2023 manila2023 , respectively, as they presented approaches that are
close to the proposed one. Ultimately, this meticulous process yielded a total
of 23 works.444The complete list of selected and excluded papers is available
at https://bit.ly/3R1Cjsq

###  3.2 Elicited features

Table 2 summarises the list of papers we collected by means of the previously
described process. For each approach, we list the name of the tool, the venue,
the year, and the underpinning mechanism used to define and assess fairness
(if any). Furthermore, starting from the four steps of the general fairness
assessment workflow described in Section 2, we elicit six different features
to evaluate the degree of automation and customization. These features are
grouped in Table 2 by the primary step of the workflow described in Figure 1
they belong to, i.e., bias definition, fairness analysis specification,
analysis implementation & fairness assessment. The selected features are as
follows:

➤ F1 - Bias definition: The approach models and assesses individual bias
definitions, group bias definitions, or both;

➤ F2 - Custom bias definition: The approach provides extension mechanisms to
provide a tailored bias definition apart from the existing ones;

➤ F3 - Custom metric definition: Similar to the previous one, the approach
allows the definition of additional metrics to detect bias considering widely
adopted datasets;

➤ F4 - Metric composition: It is possible to combine defined metrics to create
new ones;

➤ F5 - Automated fairness assessment: The underlying system assesses the
fairness by automatically generating the corresponding source code;

➤ F6 - Tool availability: The paper is supported by a publicity available
tool;

For each tool, these features are marked in Table 2 as supported (✔) or
unsupported (✗). Concerning the feature fairness definition, the symbols I and
G are used for _individual_ and _group_ fairness, respectively.

Table 2: Comparison of the existing fairness toolkit and approaches

| Bias Definition |  Fairness Analysis Specification |  Analysis Implementation & Fairness Assessment
---|---|---|---
Approach | Venue & Year | Base strategy | F1 - Bias def. |  F2 - Custom bias def. |  F3 - Custom metric def. |  F4 - Metric Comp. |  F5 - Automated fairness assess. | F6 - Tool avail.
Aequitas 10.1145/3238147.3238165  | ASE (2018) | Search-based | G | ✗ | ✗ | ✗ | ✔ | ✔
Themis angell_themis_2018  | ESEC/FSE (2018) | Search-based | G | ✗ | ✔ | ✗ | ✔ | ✔
TILE sharma_testing_2019  | ICST (2019) | Metamorphic testing | I | ✗ | ✗ | ✗ | ✔ | ✗
ADF 10.1145/3377811.3380331  | ICSE (2020) | Adversarial DL | I | ✗ | ✗ | ✗ | ✗ | ✔
Fairway chakraborty_fairway_2020  | ESEC/FSE (2020) | Search-based | G | ✗ | ✔ | ✗ | ✔ | ✔
DeepInspect 10.1145/3377811.3380400  | ICSE (2020) | Deep learning | G | ✗ | ✗ | ✗ | ✔ | ✔
AITEST aggarwal_testing_2021  | ICSE (2021) | Search-based | I | ✗ | ✗ | ✗ | ✔ | ✗
EIDG zhang2021efficient  | ISSTA (2021) | Search-based | I | ✗ | ✗ | ✗ | ✗ | ✔
Fair-SMOTE 10.1145/3468264.3468537  | ESEC/FSE (2021) | Situation testing | I, G | ✗ | ✗ | ✗ | ✔ | ✔
Fairea 10.1145/3468264.3468565  | ESEC/FSE (2021) | Mutation testing | G | ✗ | ✗ | ✗ | ✔ | ✔
BiasFinder 9653830  | TSE (2021) | Mutation testing | I | ✗ | ✗ | ✗ | ✔ | ✔
FairKit-learn 9793775  | ICSE (2022) | Search-based | I, G | ✗ | ✗ | ✗ | ✔ | ✔
PAIRFAIT-ML10.1145/3510003.3510202  | ICSE (2022) | Search-based | G | ✗ | ✗ | ✗ | ✗ | ✔
MAATchen_maat_2022  | ESEC/FSE (2022) | Ensemble learning | G | ✗ | ✗ | ✗ | ✔ | ✔
FairMaskpeng_fairmask_2023  | TSE (2022) | Hybrid | G | ✗ | ✗ | ✗ | ✔ | ✔
ExpGA 10.1145/3510003.3510137  | ICSE (2022) | Genetic algorithm | I | ✗ | ✗ | ✗ | ✔ | ✔
Astraea soremekun_astraea_2022  | TSE (2022) | Grammar-based gen. | I, G | ✔ | ✗ | ✗ | ✔ | ✗
SBFT perera_search-based_2022  | EMSE (2022) | Genetic algorithm | I | ✗ | ✗ | ✗ | ✗ | ✔
NeuronFair 9793943  | ICSE (2022) | Adversarial DL | I | ✗ | ✗ | ✗ | ✔ | ✔
LTDD 10.1145/3510003.3510091  | ICSE (2022) | Linear regression | G | ✗ | ✗ | ✗ | ✔ | ✔
iRec2.0 wang_context-_2022  | TOSEM (2022) | Optimization problem | I | ✗ | ✗ | ✗ | ✗ | ✔
FairML 10.1145/3550355.3552401  | MODELS (2022) | MDE-based | I, G | ✗ | ✗ | ✗ | ✔ | ✔
MANILA manila2023  | FASE (2023) | MDE-based | I, G | ✗ | ✗ | ✔ | ✔ | ✔

###  3.3 Selected approaches

Aequitas 10.1145/3238147.3238165  exploits three different search-based
strategies to assess the group fairness of benchmarking ML-based classifiers,
i.e., random, semi-directed, and fully directed. The results of the conducted
evaluation show that Aequitas can reduce the unfairness of the examined ML
models.

Similarly, Themis angell_themis_2018  provides a GUI to specify the schema of
the dataset on which a user wants to assess fairness and generates a test case
for it. Furthermore, it generates a report showing the relative group fairness
for each variation in the value of the variables.

Sharma et al.sharma_testing_2019  investigate fairness in the learning phase
of an ML algorithm. The proposed tool, called TILE, relies on a metamorphic
testing approach to analyze the so-called balanced data usage, i.e., the
learner should treat all data in the training set equally. TILE assesses
fairness regarding this metric by being tested on several scikit-learn ML
models.

A scalable gradient-based algorithm called Adversarial Discrimination Finder
(ADF) has been proposed to assess individual bias by injecting individual
discriminatory instances into a given dataset 10.1145/3377811.3380331 . The
global generation phase generates discriminatory entities by combining
generative models and clustering techniques. Such data are refined by the
local generation phase using underpinning gradients. As stated in the
evaluation, the ADF algorithm overcomes two state-of-the-art tools regarding
effectiveness and efficiency.

Fairway is a tool proposed by Chakraborty et al.that covers both bias
detection and mitigation chakraborty_fairway_2020 . This tool works under the
Equal Opportunity (EO) hardt_equality_2016  and Average Odds (AO)
berk_fairness_2018  group definitions of fairness by identifying ambiguous
data points. Next, it removes the bias learned by the ML algorithm through an
optimization approach. Fairway succeeds in improving fairness under the EO and
AO definitions.

Fairness in image classification has been investigated in
10.1145/3377811.3380400 . The authors propose DeepInspect, a deep learning
approach to mitigate two types of discrimination, i.e., confusion and bias.
The underpinning network uses the neuron activation probability (NAP) matrix
to predict the abovementioned discriminations. The results show that
DeepInspect performs better than existing approaches in terms of accuracy,
thus detecting misclassification correctly.

AITEST aggarwal_testing_2021  is a tool that combines constraint-based linear
optimization with the local interpretable model-agnostic explanation (LIME)
techniques to perform individual fairness assessment. The proposed hybrid
search strategy outperforms two notable fairness toolkits, i.e., Themis and
Aequitas.

The same authors of 10.1145/3377811.3380331  extend their former work by
proposing an Efficient Individual Discriminatory Instances Generator (EIDIG)
to generate individual fairness test cases for DNN models systematically
zhang2021efficient . The evaluation demonstrates that considering the gradient
of the model output instead of the gradient of the loss improves the ADF’s
overall accuracy and F1 scores.

Fair-SMOTE 10.1145/3468264.3468537  has been conceived to remove bias by
exploiting a relabeling strategy. After the bias detection phase, it
rebalances sensitive groups using the K-nearest neighbour algorithm. The
results show that Fair-SMOTE solved biases before training the models compared
to existing approaches.

Fairea 10.1145/3468264.3468565  mitigates group biases based on bias-
mitigation models generated using a mutation engine. The tool identifies and
tests five bias mitigation strategies to measure the trade-off between
accuracy and fairness. Fairea has been evaluated using two different metrics,
i.e., statistical parity difference and average odds difference.

Similarly, BiasFinder 9653830  adopts mutation testing to assess fairness in
sentiment analysis (SA) systems. The mutant engine produces actual instances
by relying on bias-targeting templates extracted from the textual content. The
tool was evaluated quantitatively and qualitatively by considering two large
SA datasets and human annotators.

Being built on top of sklearn and AIF360 frameworks, the Fairkit-learn toolkit
9793775  provides a comprehensive platform to train, test, and compare ML
models by considering fairness aspects. The tool retrieves fairer models than
the two abovementioned libraries by relying on a set of Pareto-optimal
strategies.

PAIRFAIT-ML 10.1145/3510003.3510202  exploits three different dynamic search
algorithms to support hyper-parameters tuning by providing a set of bias-free
configurations. The evaluation shows that the retrieved items reduce the bias
by considering two metrics, i.e., equal opportunity and average odd
difference.

Chen et al.chen_maat_2022  propose MAAT, an ensemble approach to optimize the
bias removal by combining two different models, i.e., fairness and performance
models. The former relies on the undersampling strategy to mitigate the group
bias. The latter is combined with the fairness model to enhance the mitigation
process regarding execution time. The empirical evaluation shows that MAAT
outperforms the existing approaches, thus mitigating the detected biases in
less time.

FairMASK peng_fairmask_2023  is a hybrid approach that exploits the
explanation bias technique to infer possible biases before the training phase.
In particular, the underpinning model is trained on non-protected attributes
to use as the dependent feature in the classification task. SUbsequently, the
approach performs the prediction phase by using a masking strategy to assess
the overall performance. FairMASK has been compared with benchmarking tools,
demonstrating that the adopted technique is more effective in mitigating group
biases.

Fan et al.propose a model-agnostic individual fairness testing approach,
namely ExpGA, based on genetic algorithms 10.1145/3510003.3510137 . The
proposed strategy can handle black-box models by feeding the underlying model
with the prediction probabilities, thus optimizing the fitness value. The
approach is evaluated by considering i) the overall performance, ii) the
execution time, and iii) improvement through retraining.

Conceived explicitly for NLP systems, ASTRAEA soremekun_astraea_2022  is a
grammar-based instance-generation tool that identifies features causing
fairness violations given an input model and mitigates them. To this end, it
extracts sensitive attributes from the input grammar to cover individual and
group fairness metrics.

Perera et al.propose the Search-based Fairness Testing (SBFT) tool to evaluate
the individual fairness of ML regression systems perera_search-based_2022
based on the fairness degree metric. The tool generates a set of unfair
instances using a genetic algorithm approach. SBTF outperforms Aequitas and
Themis in discovering individual unfairness.

Similar to 10.1145/3377811.3380331 , NeuronFair 9793943  exploits the
adversarial strategy to generate individual discrimination instances (IDIs)
and produce interpretable test cases for DNNs. The findings show that
NeuronFair outperforms four baselines in terms of four different aspects,
i.e., effectiveness, efficiency, interpretability, and generalization, by
considering seven different datasets.

Li et al.10.1145/3510003.3510091  propose a logistic-regression-based training
data debugging (LTDD) strategy to remove group bias from training feature
values. In this respect, the approach predicts the biased part of the features
and removes them from the training samples to predict the final label. The
proposed strategy outperforms state-of-the-art methodologies in terms of
notable fairness indicators.

iRecSys2.0 is a fairness-aware in-process crowdworker recommendation system
proposed by Wang et al.wang_context-_2022 . The proposed approach is optimized
to overcome popularity bias by means of a multi-objective optimization-based
re-ranking component. The authors evaluated their approach in terms of the
effectiveness of the predictions and fairness.

The most relevant work to our approach is FairML 10.1145/3550355.3552401 , an
MDE-based approach specifically conceived to conceptualize fairness by relying
on a tailored metamodel. The dedicated DSL covers the definition of bias and
the actual assessment using predefined metrics. FairML eventually generates a
YAML specification of the system compliant with the metamodel that the user
can fine-tune.

d’Aloisio et al.propose MANILA manila2023 , a low-code tool to develop and
execute fairness evaluation experiments by combining Software Product Lines
and Extended Feature Models formalisms. After the feature selection phase,
MANILA generates a Python implementation of the given experiment that
evaluates each ML and fairness-enhancing method combination using the provided
metrics. The authors evaluate their approach by replicating a case study,
allowing the composition of different metrics.

In summary, the majority of the reviewed tools primarily concentrate on
applying pre-existing fairness definitions and metrics, ultimately conducting
the final assessment within the social domain. Consequently, we recognize a
compelling need to offer users a comprehensive and domain-agnostic framework
that empowers them to define and evaluate their own bias and fairness
criteria.

##  4 Proposed approach

In this section, we introduce MODNESS, a model-driven framework designed to
conceptualize, design, implement, and execute the fairness assessment workflow
illustrated in Figure 1. Our approach can be characterized as a two-layered
framework. At its core lies an abstract bias definition, upon which multiple
fairness analyses can be defined and built.

Figure 2: MODNESS high-level view

Figure 2 provides a high-level overview of MODNESS: round boxes represent the
four primary steps of the fairness assessment workflow outlined in Section 2,
while square boxes depict the artifacts that are either developed or
automatically generated. Adjacent to each artifact, we also indicate the
technologies employed for its implementation. The fairness assessment workflow
within MODNESS can be divided into two main phases: a Human Driven phase,
which involves user interaction (as described in Section 2), and an Automated
phase, which operates without direct user involvement.

To initiate the fairness assessment process with MODNESS, the initial step
involves defining bias by specifying the sensitive variables, privileged and
unprivileged groups, and positive outcome. Subsequently, multiple analyses can
be constructed based on this bias definition. A fairness analysis tailors a
specific bias definition to a particular dataset, with a defined scope and
associated fairness metrics. These fairness metrics can either be established
in existing literature or custom-defined by the user. These specification
steps are implemented in MODNESS as a model-driven approach, utilizing the EMF
ecosystem steinberg2008emf . In this phase, the output is a model (referred to
as the Bias and Fairness Model in the figure) that encapsulates both the bias
and the corresponding fairness analysis definitions. This model adheres to the
Bias and Fairness Metamodel, which serves as the foundational structure of
MODNESS.

Starting from a model describing the bias and the related fairness analyses
definitions, MODNESS automatically generates an implementation of them through
a code generator based on the Acceleo
technology.555https://www.eclipse.org/acceleo/ In particular, MODNESS
generates a Python code that automatically checks the fairness of a given
dataset using the information provided in the fairness analysis specification.
The analyses implementation and the fairness assessment steps comprise the
automated phase of MODNESS and do not require direct human intervention.

In the following, we detail MODNESS by first describing the implemented
metamodel in Section 4.1 and, next, describing the code generation and
fairness assessment processes in Section 4.2.666The full source code of
MODNESS is available online at https://bit.ly/3RhH4y7

###  4.1 Bias and Fairness Metamodel

The bias and fairness metamodel is the foundation of MODNESS. It allows users
to define the bias related to a particular domain and to specify fairness
analyses starting from that definition. Hence, the metamodel can be divided
into three related packages providing the modeling constructs for bias
definitions (see Fig. 3), fairness analyses specification (see Fig. 4), and
for the definition of metrics (see Fig. 5).777The whole metamodel is available
online at https://bit.ly/3PkNNoc

Figure 3: Bias Definition

The root of the metamodel in Figure 3 is the abstract class Bias representing
the concept of discrimination at a higher level of abstraction. Bias has a
domain and one or more sources (i.e., what generated bias). The possible
sources of bias have been selected from mehrabi_survey_2021  e.g., human
discrimination, wrong sampling of data, among others. Then, bias is composed
of a PositiveOutcome and one or more SensitiveVariable instances, which have
one or more SensitiveVarValue each. Finally, in a definition of bias, both the
privileged and unprivileged groups must be specified. The SensitiveGroup
metaclass models these groups. In particular, each
SensitiveGroup is identified by one or more
SensitiveVarValue. The Bias metaclass is then specialized by two sub-
metaclasses representing GroupBias and IndividualBias. Both group and
individual biases are composed of one or more Analysis with a particular
Scope. In particular, GroupBias has one or more GroupAnalysis, while
IndividualBias has one or more IndividualAnalysis.

Figure 4: Fairness Analysis

Figure 4 depicts the portion of the metamodel dedicated to the specification
of fairness analysis, which is represented by the Analysis abstract metaclass.
An analysis may have a scope, i.e., a textual description of the analysis, and
is composed by one or more Datasets. A Dataset has an attribute to specify the
name of the column containing the groundTruthLabel (if any), an attribute to
specify the name of the column containing the predictedLabel (if available),
and an attribute to specify its filePath. Then, a mapping of each general
concept defined in the bias definition must be identified in the dataset. In
particular, a Dataset is composed of one DatasetPositiveOutcome metaclass
mapping the value of the positive outcome in the dataset, one or more
DatasetSensitiveVar metaclasses (which are in turn composed of one or more
DatasetSensitiveVarValue metaclasses) mapping the sensitive variables, and, if
needed, one or more OtherVariable metaclasses representing other values
encoded in the dataset. Then the sensitive groups must also be mapped in the
dataset through the DatasetSensitiveGroup metaclass. All the metaclasses
representing values extend a VariableValue metaclass. It is worth noting that
all the values can be absolute or relative to the dataset. Finally, an
analysis comprises one or more Metric.

Figure 5: Metric Definition

Figure 5 reports the portion of the metamodel dedicated to the metric
definition. A Metric is composed of an EqualityOperator representing the
threshold and has an attribute representing the toleranceValue (i.e., the
level of bias tolerated by the system). The
EqualityOperator can be a SingleOperator (e.g., ”$=0$” or ”$\leq 1$”) or a
RangeOperator (e.g., ”the metric must be $<1$ and $>0$”). Next, a Metric has a
Function representing the actual metric implementation. Currently, the
following functions are available: Operation (representing a generic
arithmetical operation),
Logarithm, Summation, ExpectedValue, GroupSize, and Probability. A matric can
also be based on
ExistingGroupFairnessMetric and
ExistingIndividualFairnessMetric. Such metaclasses are used to represent the
set of fairness metrics known in the literature and already implemented for
group and individual fairness definitions, respectively.888To select the set
of metrics, we referred to the ones implemented in the AIF360 library
bellamy_ai_2019 . For each function, we included a set of attributes needed
for their implementation. In the future, new functions can be added by
extending the Function metaclass.

###  4.2 Code generation and fairness assessment

After defining fairness and its corresponding metrics, MODNESS generates the
implementation using a source code generator developed with Acceleo. This
generator uses specific templates to create both static and dynamic parts of
the target code by incorporating queries on the source models. Acceleo
templates use a defined syntax to specify conditions or iterations over
elements in the input models. Listing 2 depicts an excerpt of the developed
Acceleo template.999The developed Acceleo-based code generator is available at
https://bit.ly/3ECHQxZ

Listing 2: Fragment of an explanatory MODNESS Acceleo template.

⬇

file_path = ’[biasModel.dataset.filePath/]’

predicted_label_name = ’[biasModel.dataset.predictedLabelName/]’

ground_truth_label_name = ’[biasModel.dataset.groundTruthLabelName/]’

...

[if biasModel.metric -> size()>0]

[for (metric: Metric | biasModel.metric)]

[if (metric.operator.oclIsTypeOf(SingleOperator))]

threshold = [metric.operator.oclAsType(SingleOperator).value/]

[/if]

[/for]

[/if]

To support the generation phase, we rely on Pandas mckinney-proc-scipy-2010
and AI360 Python bellamy_ai_2019  libraries to preprocess and support fairness
analysis, respectively. In particular, MODNESS exploits three different
Acceleo templates developed to support each phase of the process, i.e., bias
definition, fairness analysis specification, and metric definition. To cover
the first phase, the generated code supports the high-level specification of
the analyzed fairness scenario, including the sensitive variables, the
expected positive outcome, and the parameters needed to feed the selected
metric. In such a way, the configuration is compliant with the user-defined
specification. Afterwards, the fairness analysis phase could be conducted by
means of a set of predefined implementations of the state-of-the-art fairness
metrics (taken from the AIF360 library bellamy_ai_2019 ). Alternatively,
MODNESS can generate operator specifications that define and compose different
metrics.

##  5 Evaluation

In this section, we present the evaluation of the proposed approach. We
rigorously assess the expressiveness and correctness of MODNESS, demonstrating
how it effectively addresses the limitations of contemporary state-of-the-art
methods in assessing fairness. Thus, we answer the following three research
questions (RQ):

  * •

RQ1: _How do state-of-the-art toolkits allow users to explicitly specify
customized fairness definitions?_

  * •

RQ2: _Can MODNESS support the whole fairness analysis process, including the
automated assessment phase?_

  * •

RQ3: _Is MODNESS able to overcome the limitations of current MDE-based
approaches for fairness assessment?_

To address RQ1, we conduct a literature review by carefully inspecting
existing tools and approaches collected from premiere SE venues according to
the inclusion and exclusion criteria presented in Sec. 3.1. Plus two extra
papers due to their relevance and relation with MODNESS. Then, the elicited
approaches have been characterized in terms of recurrent operation in the
fairness assessment workflow, i.e., bias definition, fairness analysis
specification, and implementation and assessment. Our outcomes demonstrate
that the examined tools prevent users from composing or defining their custom
metric definitions.

To tackle RQ2, we evaluate the expressiveness and accuracy of MODNESS. We
gauge expressiveness by demonstrating MODNESS’s capability to model and
implement a diverse range of use cases concerning bias and fairness analysis.
Conversely, we assess accuracy by verifying that the code generated by MODNESS
can reliably identify biases in the chosen use cases. In particular, we use
MODNESS to model and generate the implementation of the most notable use cases
in the fairness domain (i.e., social angwin_machine_2016 ;
kohavi_scaling_nodate  and financial hofmann2013uci ; moro2014data ). In
addition, we analyze the bias of two datasets from use cases not yet studied
in the fairness domain, i.e., Java and Arduino software projects. The former
is composed of third-party libraries (TPLs) that have been used in a recent
study to evaluate popularity bias in recommender systems for software
engineering (RSSE) nguyen2023dealing . The latter includes Arduino project
metadata such as description, views, and source code that have been used to
feed a collaborative filtering recommender system dirocco2023resyduo .

To address RQ3, we compare MODNESS with two baselines elicited from the
examined ones as discussed in Section 3 i.e., FairML 10.1145/3550355.3552401
and MANILA manila2023 . The rationale behind this choice is that they i) have
been developed using the MDE paradigm and ii) allow the specification of the
fairness assessment workflow. Assessing the quality of MDE-based tools is
daunting since they usually rely on tailored metamodels conceived for a
specific application domain. Prior works have defined a set of quality metrics
that investigate several aspects such as expressiveness, completeness, or
portabilityBertoa2010QualityAF . Within the scope of our paper, we establish
two dimensions for facilitating comparison: expressiveness and automation. We
frame these aspects by referencing the set of features detailed in Section
3.2:

  * –

Expressiveness: This dimension measures the extent to which the tool enables
the modelling of bias definitions and relative fairness analysis (encompassing
features F1-F2-F3).

  * –

Automation: This dimension evaluates the degree to which the tool streamlines
the entire fairness assessment process (encompassing features F4-F5).

These dimensions are assessed on a scale ranging from 1 to 3. For
expressiveness, a rating of ”1” signifies that the tool only permits the
specification of fairness analyses without accommodating high-level bias
specifications (referring to the steps of the fairness assessment workflow
described in Section 2). A rating of ”2” indicates that the tool facilitates
the specification of both fairness analyses and high-level fairness
definitions. Meanwhile, a rating of ”3” implies that the tool not only
supports the specification of fairness analyses and high-level fairness
definitions but also allows for creating custom metrics for analysis.
Regarding automation, a rating of ”1” suggests that the tool does not generate
implementation code for fairness assessment. A rating of ”2” signifies that
the tool generates the implementation code for fairness assessment. Lastly, a
rating of ”3” indicates that the tool not only generates implementation code
but also offers guidance to the user in experiment specification.

We model with the two baselines the use cases used to answer RQ2, and we use
them to evaluate these features.

###  5.1 Examined use cases

By carefully analyzing the approaches described in Table 2, we have identified
pertinent use cases utilized across the literature to evaluate fairness within
various domains. Given the vast array of potential scenarios, it is
impractical to examine all of them within this section. Therefore, we focus on
widely embraced case studies, specifically those covered by at least three
distinct approaches. Furthermore, a recent survey on recommender systems
highlighted an existing imbalance in the different fairness notions that might
lead to inconsistencies during the assessment phase deldjoo_fairness_2023 . To
address this concern, we introduce two additional case studies into our
comparative analysis. These case studies pertain to the evaluation of
popularity bias in recommender systems encompassing: a curated dataset of
third-party Java libraries, and a curated dataset of Arduino hardware and
software components, respectively. In the following, we provide detailed
descriptions of the selected use cases and the associated datasets. These
descriptions are also summarized in Table 3, which includes a reference to the
approaches that have addressed each case.

➤ ProPublica Recidivism (COMPAS) angwin_machine_2016  - The Correctional
Offender Management Profiling for Alternative Sanctions (COMPAS) was an ML
system used by judges in the US to predict if a condemned person would have
been a repeating offender in the two years after their release. An
investigation of this software showed that this system had a bias against non-
white women. In this case, the sensitive variables are race and gender, the
favourable outcome is non-recidiv, and the unprivileged group is non-white
women.

➤ Adult Census Income (ADULT) kohavi_scaling_nodate  - This use case is about
predicting whether a person’s income is above 50,000$ a year based on their
personal information. This system was biased against non-white women. Hence,
the sensitive variables are gender and race and the positive outcome is income
higher than 50,000$ a year.

➤ German Credit (GERMAN) hofmann2013uci  - This use case is about the adoption
by a German bank of an ML system to predict the granting of credit. This
system has been proven to be biased against women, i.e., women have a lower
probability of getting credit from the bank. In this case, the sensitive
variable is gender, and the unprivileged group is women. The positive outcome
is having a credit granted.

➤ Bank Marketing (BANK) moro2014data  - The Bank Marketing system was an ML
system developed for a phone call company to predict whether the client would
subscribe to a term deposit. This system was shown to be biased against people
more than 25 years old. Hence, in this case, the sensitive variable is age,
and the unprivileged group is people with more than 25 years. The positive
outcome is will subscribe.

➤ Software third-party libraries (TPL) nguyen2023dealing  - This dataset has
been used in nguyen2023dealing  to measure the popularity bias in the context
of recommender systems for software engineering (RSSE). Being built on top of
a previous work NGUYEN2019110460 , it contains 5,200 Java projects and 31,817
software libraries. In the scope of our paper, we consider frequency as a
sensitive variable capable of measuring the popularity bias in the context of
recommender systems DBLP:conf/recsys/AbdollahpouriMB19 . In this respect, the
unprivileged group and the positive outcome are unpopular library and the list
of recommended libraries, respectively.

➤ Resyduo dataset (RESYDUO) - This dataset comprises all the Arduino projects
collected from the ProjectHub101010https://projecthub.arduino.cc/ open-source
repository. In particular, it includes 5,547 projects, 3,137 tags, 11,645
hardware components, and 1,802 libraries. It is worth mentioning that this
data has been used to feed a collaborative filtering-based recommender system
supporting Arduino project development dirocco2023resyduo . Similar to the TPL
dataset, the scope of the fairness analysis is to measure how popularity
impacts the recommended items by considering two different sensitive variables
for each project, i.e., views and respects. The former quantifies project
popularity based on the number of users who view it. The latter represents
explicit feedback on project quality. Consequently, fairness assessment can be
conducted on two distinct disadvantaged groups, i.e., low viewed and low
respected projects.

Table 3: The examined use cases

Name | Domain | Sensitive attribute(s) | Positive outcome | Existing approaches
---|---|---|---|---
COMPAS | Social | gender, race | Non-recidiv |  9793775 ,10.1145/3510003.3510202 ,9793943 , 10.1145/3468264.3468537 , chen_maat_2022 , peng_fairmask_2023 , 10.1145/3550355.3552401 , 10.1145/3468264.3468565 , 10.1145/3510003.3510091 , chakraborty_fairway_2020
ADULT | Social | race, gender | Income $>$ 50.000 $ |  10.1145/3377811.3380331 , 10.1145/3238147.3238165 , 10.1145/3510003.3510137 , aggarwal_testing_2021 , 10.1145/3510003.3510202 , 9793943  , chen_maat_2022 , peng_fairmask_2023 , 10.1145/3468264.3468537 , 10.1145/3550355.3552401 , 10.1145/3468264.3468565 ,10.1145/3510003.3510091 ,chakraborty_fairway_2020 , sharma_testing_2019 ,aggarwal_testing_2021 ,angell_themis_2018
GERMAN | Financial | gender | Credit granted |  10.1145/3377811.3380331 ,aggarwal_testing_2021 ,10.1145/3510003.3510137 , 10.1145/3510003.3510202 ,9793943 , 10.1145/3468264.3468537 , 10.1145/3468264.3468537 ,10.1145/3550355.3552401 , chen_maat_2022 , peng_fairmask_2023 , 10.1145/3468264.3468565 , 10.1145/3510003.3510091 , chen_maat_2022 , chakraborty_fairway_2020 ,sharma_testing_2019 , angell_themis_2018
BANK | Financial | age | Client subscribed |  10.1145/3510003.3510137 , 10.1145/3510003.3510202 ,9793943 , chen_maat_2022 , peng_fairmask_2023 , 10.1145/3468264.3468537 , 10.1145/3377811.3380331 , 10.1145/3510003.3510091
TPL | RSSE | frequency | Library recommended | nguyen2023dealing
RESYDUO | IoT | view, respect | Item recommended | dirocco2023resyduo

###  5.2 Answer to RQ1

To answer RQ1, we evaluate the existing approaches in terms of the elicited
features (F1-F6) introduced in Section 3. Furthermore, we discuss our approach
by highlighting the contribution compared to the examined works.

F1 support - Based on our investigations, it is evident that only five of the
analyzed approaches can address both individual and group fairness. These
approaches are Fair-SMOTE, FairKit-learn, Astraea, FairML, and MANILA. In
contrast, the other tools are designed to focus exclusively on either
individual or group fairness. Notably, MODNESS stands out by offering modeling
constructs that comprehensively cover both individual and group bias
definitions. This flexibility is achieved by drawing upon essential concepts
extracted from surveys and empirical studies mehrabi_survey_2021 ;
caton_fairness_2020 ; wang_survey_2023 .

F2 support - While tools like Fair-SMOTE, Fairkit-learn, FairML, or MANILA
provide methods to automatically assess several canonical individual and group
bias definitions, only ASTRAEA offers a dedicated grammar that allows the user
to define any biases besides the traditional ones. MODNESS goes a step forward
compared to ASTRAEA by providing a tailored metamodel conceived to define
fairness at two different levels of abstractions, i.e., domain-level and
dataset-level (e.g., refer to the evaluation of the RESYDUO use case in
Section 5.3, where, starting from the same domain, two different fairness
analyses are depicted selecting two different sensitive attributes of the
dataset).

F3 support - It is worth noting that only three approaches within our analysis
allow for the customization of metric definitions: Themis, Fairway, and
FairML. MODNESS provides the Function metaclass (see Section 4.1) to specify a
dedicated operation on sensitive variables defined in the specification phase.
Additionally, the metamodel incorporates a selection of significant metrics
that have already been established in the literature, serving as valuable
tools for assessing fairness in various contexts beyond the original ones. In
essence, MODNESS can be used to evaluate the statistical parity of two
sensitive groups that are not strictly bounded by the social domain.
Furthermore, it empowers users to define novel bias metrics (e.g., coverage
NGUYEN2019110460 ) necessary for emerging domains, such as recommender
systems, thereby adapting to evolving research needs and applications.

F4 support - Among the examined strategies, only MANILA supports this feature
by modeling each metric as a feature and allowing their compositions by means
of aggregation functions. Meanwhile, MODNESS relies on the metamodel to
compose the defined metrics, thus pursuing the generalizability of the whole
process in terms of entities and their combinations.

F5 support - Although all the investigated tools offer automatic fairness
assessment, these approaches are generally confined to established use cases
and state-of-the-art metrics. In contrast, MODNESS is extendible and very
flexible since it allows the conceptualization of fairness in domains known in
the literature and domains not yet covered (i.e., recommender systems and
novel use cases). Furthermore, it allows for the creation of novel bias
metrics without sacrificing automation, making it a powerful and adaptable
tool for addressing the evolving landscape of fairness assessment.

F6 support - Regarding the tool availability, it is important to note that all
the scrutinized approaches offer a replication package, with the exceptions
being TILE, AITEST, and ASTRAEA. The majority of these approaches utilize
Python libraries and frameworks, primarily due to the fact that the tested
models are machine learning-based. Furthermore, both FairML and MANILA go a
step further by offering code generation functionalities that automate the
deployment and testing of the system as defined during the setup phase.
MODNESS adopts a comparable approach by utilizing dedicated Acceleo templates,
which are fed with models adhering to the specified metamodel.

Answer to RQ1: Although offering a good degree of automation, existing
approaches lack in supporting the customization of bias and fairness
definitions. MODNESS fills this gap by covering all the elicited features for
bias definition, fairness analysis specification, analysis implementation and
fairness assessment.

###  5.3 Answer to RQ2

To address RQ2, we have selected four distinct use cases from those previously
discussed in Section 5.1, encompassing various application domains including
social, financial, recommender systems in software engineering (RSSE), and the
Internet of Things (IoT). The details of the four use cases, implemented using
MODNESS, are provided in Table 4. Specifically, for each use case, we specify
the chosen metric for assessment, the number of sensitive variables
considered, and provide links to the developed artifacts, including the models
and automatically generated assessment code.

Table 4: MODNESS implementation of the use cases.

Use case | Domain | Metric |  Number of sensitive vars | Use case model | Generated code |  Outcome (Expected) |  MODNESS Assessment result
---|---|---|---|---|---|---|---
COMPAS | Social | SP | 2 (sex,race) | https://bit.ly/46ebwgT | https://bit.ly/3EEea3y | 0.3 ($\leq|0.2|$) | Biased
GERMAN (BIASED) | Financial | EO | 1 (sex) | https://bit.ly/3Lndt2h | https://bit.ly/48ijbN1 | -0.25 ($\leq|0.2|$) | Biased
GERMAN (DEBIASED) | https://bit.ly/45FWQan | -0.05 ($\leq|0.2|$) | Fair
TPL | RSSE | COV | 1 (frequency) | https://bit.ly/3LV2HkF | https://bit.ly/3Zfc184 | 0.29 ($\geq|1.2|$) | Biased
RESYDUO | IoT | GEI | 1 (views) | https://bit.ly/44ifovW | https://bit.ly/3PDljHP | 0.31 ($\geq|0.8|$) | Biased
1 (respects) | https://bit.ly/3PGE3X2 | 0.28 ($\geq|0.8|$) | Biased

Notably, in the GERMAN use case, we conduct two separate analyses: the first
utilizing the original biased dataset and the second involving the same
dataset after applying a preprocessing algorithm designed to mitigate bias
(specifically, the Debiaser for Multiple Variables daloisio_debiaser_2023 ).
This analysis scenario exemplifies a typical scenario for MODNESS, where a
user initially assesses the fairness of the original dataset and subsequently
verifies if the bias has been reduced after employing a debiasing
method111111It is important to clarify that the mitigation of bias is beyond
the scope of this approach, as MODNESS primarily focuses on designing and
implementing the fairness assessment workflow, as outlined in Section 2.

Moreover, for the RESYDUO use case, we perform two distinct analyses, one
considering the views sensitive variable and the other focusing on the
respects sensitive variable. This approach showcases MODNESS’s versatility and
ability to handle different sensitive variables, further highlighting its
capabilities.

#### The COMPAS use case

Concerning the Social domain, we replicated the COMPAS use case. Figure 6
shows an excerpt of the implemented model reporting also the attributes of the
Dataset and SingleOperator classes. As previously described, this use case is
about discrimination of non-white men in the prediction of recidivism. Hence,
we modeled the bias definition in MODNESS, specifying gender and race as
sensitive variables, non-recidiv as the positive outcome, non-white men as the
unprivileged group, and white women as the privileged group.121212Both
privileged and unprivileged groups are modeled through two compositions of the
SensitiveGroup metaclass

Figure 6: An excerpt of the MODNESS model for the COMPASS use case

Next, we defined our fairness analysis by specifying the dataset containing
all the related information. In particular, we specified in the attributes of
the Dataset class that the ground truth labels are encoded in the
two_years_recid column. In contrast, the model predictions are encoded in the
prediction column. Then, we modeled that the positive outcome equals 1.0.
Finally, we specified that the sensitive variables are encoded in the race and
sex131313We refer to the original column names of the dataset reported in
angwin_machine_2016  columns where non-white women have a value of 1 for both
columns. After defining the dataset, we specified the metrics for analysis.
For this use case, we adopted the Statistical Parity (SP)dwork_fairness_2012
fairness definition, which is included in the ExistingFairnessMetric class.
Finally, we specified that this metric should be equal to 0 to have fairness,
with a tolerance value of 0.2.

From such a model, MODNESS generates the code shown in Listing 3. In
particular, all the metrics defined in the metamodel (i.e., all the
metaclasses extending the Function metaclass) have been implemented as
functions in the FairnessMetric Python class, which is imported at the
beginning of the script. Next, there is a set of variables specifying
attributes defined in the model (i.e., file path, predicted and ground truth
label names, privileged and unprivileged groups, positive outcome, threshold,
and tolerance value). Finally, the FairnessMetric class is instantiated, and
the function related to the specified metric (i.e., statistical parity) is
called. If the returned metric’s absolute value is different from the
threshold plus the tolerance value, then the dataset is biased; otherwise, it
is fair.

Listing 3: Generated code for the COMPAS use case.

⬇

from fairnessMetric import FairnessMetric

import pandas as pd

file_path = "data/compas.csv"

predicted_label_name = "prediction"

ground_truth_label_name = "two_year_recid"

data = pd.read_csv(file_path)

indexes = ["sex" "race"]

dataset_unprivileged_group = {"race": 0, "sex": 0}

dataset_privileged_group = {"race": 1, "sex": 1}

dataset_positive_outcome = 1

threshold = 0.0

tolerance_value = 0.2

metrics =
FairnessMetric(data,dataset_unprivileged_group,dataset_privileged_group,ground_truth_label_name,predicted_label_name,dataset_positive_outcome)

print(metrics.statistical_parity_difference())

if abs(metrics.statistical_parity_difference()) > (threshold +
tolerance_value):

print("Biased")

else:

print("Fair")

#### The GERMAN use case

Concerning the Financial domain, we implemented the GERMAN use case, which is
about the discrimination of women in credit granting. Similarly to the COMPAS
use case, we first specified in the bias definition the sensitive variable
(gender), the positive outcome (credit grant) and the privileged (men) and
unprivileged (women) groups. Next, we specified two different fairness
analyses, one involving the original biased dataset and another involving the
debiased one. In both analyses, we modeled that the sex column encodes the
gender sensitive variable where women have a value equal to 1. In contrast,
the positive outcome is encoded in the credit column with a value equal to 1.
In both analyses, we selected the Equal Opportunity (EO) hardt_equality_2016
fairness definitions, specifying a threshold of 0 and a tolerance value of
0.2. The generated code follows the same structure of Listing 3 and is linked
in Table 4, as well as the specified model.

#### The TPL use case

We implemented the TPL use case for the Software Engineering domain, which is
about popularity bias in recommender systems for third-party libraries. In the
bias definition, we specified that the sensitive variable is frequency, the
positive outcome is high ranking, the unprivileged group is non-frequent
items, and the privileged group is frequent items. Next, in the fairness
analysis, we specified that the predicted rank is encoded in the ranking
column and that we consider a rank high if it is greater than 80% of the
predicted ranks (this has been done by flagging the relativeToDatasetSize
attribute in the
DatasetPositiveColumn class to true). Similarly, we specified that the
frequency is encoded in a column named frequency and that an item is highly
frequent if it appears more than 80% of other items. Concerning the adopted
metric, following the work of nguyen2023dealing , we modeled a custom metric
named coverage NGUYEN2019110460 . This metric calculates the portion of non-
frequent items with a high ranking over all the items.

Figure 7: The MODNESS implementation of the coverage metric

Figure 7 reports the implementation of this metric in MODNESS. We modeled this
metric as a ratio operation where the nominator is the number of items with
both a low frequency and a high ranking. At the same time, the denominator is
the number of items with only a high ranking. Finally, we specified that this
metric should be equal to 1 to have fairness, with a tolerance value of 0.2.
Listing 4 reports the portion of the generated code implementing the coverage
metric, while the rest of the code follows the same structure as Listing 3.
The generated code as well as the implemented model, are linked in Table 4.

Listing 4: Portion of the generated code implementing the coverage metric

⬇

threshold = 1.0

tolerance_value = 0.2

metrics = FairnessMetric(data, dataset_unprivileged_group,
dataset_privileged_group, ground_truth_label_name,

predicted_label_name, dataset_positive_outcome)

coverage = metrics.group_size(

"frequency == 0 and ranking == 1") / metrics.group_size(

"ranking == 1")

print(coverage)

if abs(coverage) < threshold + tolerance_value:

print("Biased")

else:

print("Fair")

#### The RESYDUO use case

Finally, for the IoT domain, we implemented the RESYDUO use case, which is
about popularity bias in recommending software and hardware Arduino components
dirocco2023resyduo . In the bias definition, we specified views and respect as
sensitive variables and high ranking as the positive outcome. Next, we defined
two sensitive groups: one identified by the views sensitive variable (i.e.,
the privileged group is high-viewed items, while the unprivileged group is
low-viewed items), and one identified by the respect sensitive variable (i.e.,
the privileged group is high-respected items, while the unprivileged group is
low-respected items). Further, we defined two different fairness analyses. The
first one aims at assessing the amount of popularity bias with respect to the
number of views, while the second aims at assessing the popularity bias with
respect to the level of respects. In both analyses, we specified that the
predicted rank is encoded in the tot_recommendations column and that we
consider a rank high if it is greater than the 80% of the predicted ranks
(like done in the previous use case). Next, in the first analysis, we
specified that the number of views is encoded in the views column and that an
item is highly viewed if its number of views is higher than 80% of the other
items. Instead, in the second analysis, we specified that the level of respect
is encoded in the respects column and that an item is highly respects if it
has a respect level higher than 80% of the other items. In both analyses, we
modeled a custom metric used in the RecSys literature named Generalized Cross
Entropy (GEI)deldjoo2019recommender . This metric measures how the probability
distribution of having an item of the privileged group recommended is
different from the probability of having an item of the unprivileged group
recommended. Following the metric definition, we specified that this metric
should be equal to or greater than 0.8 to have fairness. As for the other
cases, the generated code follows the same structure of Listing 3 and is
linked in Table 4. The model is also reported in Table 4.

Altogether, the performed fairness assessment confirms that the bias is
correctly detected in all the considered use cases, meaning that MODNESS is
capable of detecting the biases defined at the model level.

Answer to RQ2: MODNESS has a level of expressiveness and correctness able to
model and successfully evaluate use cases from various domains, including
social, financial, RSSE, and IoT. Our experiments demonstrate the extensive
range of MODNESS’s ability to define bias and fairness in different domains
and its capability to automatically generate the relative experiments and
hence assess fairness in the considered use cases.

###  5.4 Answer to RQ3

To address RQ3, we conducted a comparative analysis between MODNESS and two
MDE-based baselines for fairness assessment, namely FairML
10.1145/3550355.3552401  and MANILA manila2023 . The comparison results,
focusing on Expressiveness and Automation, are presented in Figure 5.4.

Figure 8: Baseline comparison

Expressiveness pertains to the extent to which tools offer abstraction
capabilities to model a variety of heterogeneous use cases. To assess this
aspect, we implemented each of the use cases detailed in Section 5.3 with
every baseline tool. The list of implemented use cases is presented in Table
5. It is evident that all the tools can successfully handle traditional use
cases within the fairness domain (indicated by ✔), such as COMPAS and GERMAN.
However, neither FairML nor MANILA can accommodate less common use cases like
TPL or RESYDUO (indicated by ✗). This limitation stems from the fact that both
these tools lack the capability to define custom fairness metrics. Instead, as
demonstrated in Section 5.3, MODNESS can model all the selected use cases,
including the less common ones. Furthermore, we observe that both FairML and
MANILA do not distinguish between the concepts of bias definitions and
fairness analyses, only allowing for the modeling of fairness analyses. On the
contrary, MODNESS clearly distinguishes between a general bias definition and
the related fairness analyses. As exemplified in the RESYDUO use case,
multiple fairness analyses can be conducted based on a general bias
definition, each considering different datasets, sensitive variables, or
metrics. Consequently, we assigned a score of 1 to the customization aspect of
FairML and MANILA, while MODNESS received a score of 3, as illustrated in Fig.
5.4.

Table 5: List of implemented use cases

Tool Use Case | COMPAS | GERMAN | TPL | RESYDUO
---|---|---|---|---
FairML 10.1145/3550355.3552401  | ✔ | ✔ | ✗ | ✗
MANILA manila2023  | ✔ | ✔ | ✗ | ✗
MODNESS | ✔ | ✔ | ✔ | ✔

Regarding the degree of automation, both FairML and MANILA offer user guidance
when defining fairness assessments. FairML, for instance, employs a decision
tree to assist users in selecting the appropriate metric based on the analysis
scope they intend to pursue. On the other hand, MANILA employs the Feature
Model formalism to guide users in selecting a set of features that invariably
results in a correct (i.e., executable) fairness experiment. As of the current
development stage, MODNESS does not provide this level of user guidance,
although it is worth noting that we have plans to expand the tool’s
capabilities in this regard (see Section 7). Consequently, we assigned a score
of 3 for the automation level of FairML and MANILA, while MODNESS received a
score of 2 for its current automation capabilities.

From this analysis, we have seen how all the selected baselines provide a high
level of automation in the definition and implementation of a fairness
evaluation. However, both baselines do not have a level of expressiveness
fairness analyses definition in terms of domains (e.g., RSSE or IoT) and
metrics (e.g., coverage or GEI).

Answer to RQ3: While all the examined baseline tools exhibit a high degree of
automation throughout the fairness assessment process, both of them share a
common limitation, i.e., they lack the ability to express fairness in
different domains. In contrast, MODNESS overcomes these limitations by
offering a versatile framework for modeling high-level bias definitions and
specifying and implementing custom fairness metrics tailored to specific
application domains.

##  6 Threats to validity

This section discusses possible threats that can hamper the results of the
performed evaluation.

Internal validity concerns factors that can influence the results of our
evaluation. It is possible that the metamodel we have created and the
supporting tools are not extensive enough to cover all fairness assessment
scenarios. However, we purposely considered different application domains,
including the one related to the popularity bias of recommender systems in
software engineering. Another potential threat of our study concerns the
macro-sources of bias we cover. In particular, we address algorithmic bias and
unbalanced group bias mehrabi_survey_2021 ; daloisio_debiaser_2023 , despite
various other macro-sources of bias have been identified over the years, such
as confounding variables bias daloisio_debiaser_2023 . However, we believe
that our approach covers most of the bias case studies documented in the
literature, as they originate from macro-sources of bias that we address. In
addition, our proposed metamodel is also designed to be extendable to model
sources of bias that are not currently addressed. A final potential threat to
our study is that the literature review, presented in Section 3, only includes
software engineering sources. This limits our consideration of other possible
approaches. However, we addressed this issue by clearly stating our inclusion
and exclusion criteria and by including two additional papers from MODELS and
FASE proposing two approaches close to ours.

_External validity_ threats concern the generalizability of our approach. In
this respect, the results obtained in this paper may be valid only for the
considered datasets. To mitigate this threat, we diversified the datasets,
which have been collected from different sources and domains. Furthermore, we
demonstrated that the proposed approach could cover fairness conceptualization
and assessment also in the software engineering domain by considering
popularity bias in recommender systems. Another threat that may hamper the
obtained results is the choice of the baselines, i.e., we cannot conduct a
quantitative comparison in terms of metrics. To mitigate this, we conduct a
qualitative analysis by reimplementing the examined use cases using the
selected approaches.

##  7 Conclusion and future work

In this paper, we introduced MODNESS, an innovative model-driven engineering
(MDE) approach designed to facilitate the customization of fairness
definitions. Leveraging a dedicated metamodel, MODNESS offers a two-tier
abstraction framework capable of modeling and validating the entire fairness
workflow, spanning from conceptualization to assessment. Additionally, MODNESS
incorporates a generation module that automates the fairness assessment
process.

Our research commenced with a comprehensive survey of fairness toolkits and
strategies published in prominent software engineering venues. Surprisingly,
none of the existing tools supported the critical features of custom fairness
and metric definitions. To showcase the capabilities of MODNESS, we conducted
a qualitative evaluation demonstrating its efficacy in supporting fairness
assessments across various domains, including recommender systems and the
Internet of Things, in addition to traditional applications. While we covered
a limited number of use cases, our results suggest that our methodology
exhibits a high degree of both expressiveness and correctness, effectively
facilitating the entire fairness assessment workflow. Furthermore, we compared
MODNESS with two MDE-based baseline approaches for fairness assessment and
highlighted how MODNESS overcomes their limitations.

In future work, we plan to develop a textual domain-specific language (DSL)
using the proposed metamodel. Additionally, we aim to enhance the underlying
code generator by integrating additional target frameworks and toolkits to
support a broader range of programming languages and fairness metrics.
Furthermore, we intend to implement a recommendation system that guides users
through the specification of bias definitions and fairness analyses by
providing suggestions for sensitive variables and appropriate fairness metrics
based on user inputs. Finally, we aspire to conduct an evaluation involving
users from both industry and academia to assess the usability of MODNESS and
identify areas for further improvement.

###### Acknowledgements.

This work has been partially supported by the EMELIOT national research
project, which has been funded by the MUR under the PRIN 2020 program
(Contract 2020W3A5FY) and by the European Union - NextGenerationEU - National
Recovery and Resilience Plan (Piano Nazionale di Ripresa e Resilienza, PNRR) -
Project: “SoBigData.it - Strengthening the Italian RI for Social Mining and
Big Data Analytics” - Prot. IR0000013 - Avviso n. 3264 del 28/12/2021

## References

  * (1) Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B.: The unfairness of popularity bias in recommendation.  In: R. Burke, H. Abdollahpouri, E.C. Malthouse, K.P. Thai, Y. Zhang (eds.) Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), Copenhagen, Denmark, September 20, 2019, _CEUR Workshop Proceedings_ , vol. 2440. CEUR-WS.org (2019).  URL https://ceur-ws.org/Vol-2440/paper4.pdf
  * (2) Aggarwal, A., Shaikh, S., Hans, S., Haldar, S., Ananthanarayanan, R., Saha, D.: Testing Framework for Black-box AI Models.  In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 81–84 (2021).  DOI 10.1109/ICSE-Companion52605.2021.00041.  ISSN: 2574-1926
  * (3) Angell, R., Johnson, B., Brun, Y., Meliou, A.: Themis: automatically testing software for discrimination.  In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018, pp. 871–875. Association for Computing Machinery, New York, NY, USA (2018).  DOI 10.1145/3236024.3264590
  * (4) Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine bias.  ProPublica, May 23(2016), 139–159 (2016)
  * (5) Asyrofi, M.H., Yang, Z., Yusuf, I.N.B., Kang, H.J., Thung, F., Lo, D.: Biasfinder: Metamorphic test generation to uncover bias for sentiment analysis systems.  IEEE Transactions on Software Engineering 48(12), 5087–5101 (2022).  DOI 10.1109/TSE.2021.3136169
  * (6) Austin, K.A., Christopher, C.M., Dickerson, D.: Will I Pass the Bar Exam: Predicting Student Success Using LSAT Scores and Law School Performance.  HofstrA l. rev. 45, 753 (2016).  Publisher: HeinOnline
  * (7) Bellamy, R.K.E., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., Nagar, S., Ramamurthy, K.N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K.R., Zhang, Y.: AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias.  IBM Journal of Research and Development 63(4/5), 4:1–4:15 (2019).  DOI 10.1147/JRD.2019.2942287.  Conference Name: IBM Journal of Research and Development
  * (8) Berk, R., Heidari, H., Jabbari, S., Kearns, M., Roth, A.: Fairness in Criminal Justice Risk Assessments: The State of the Art 50(1), 3–44 (2018).  DOI 10.1177/0049124118782533.  Publisher: SAGE PublicationsSage CA: Los Angeles, CA
  * (9) Bertoa, M.F., Vallecillo, A.: Quality attributes for software metamodels (2010).  URL https://api.semanticscholar.org/CorpusID:15268921
  * (10) Bird, S., Dudík, M., Edgar, R., Horn, B., Lutz, R., Milan, V., Sameki, M., Wallach, H., Walker, K.: Fairlearn: A toolkit for assessing and improving fairness in AI.  Tech. Rep. MSR-TR-2020-32, Microsoft (2020).  URL https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/
  * (11) Brun, Y., Meliou, A.: Software fairness.  In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 754–759. ACM, Lake Buena Vista FL USA (2018).  DOI 10.1145/3236024.3264838
  * (12) Caton, S., Haas, C.: Fairness in Machine Learning: A Survey (2020).  DOI 10.48550/arXiv.2010.04053.  ArXiv:2010.04053 [cs, stat]
  * (13) Chakraborty, J., Majumder, S., Menzies, T.: Bias in machine learning software: Why? how? what to do?  In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021, p. 429–440. Association for Computing Machinery, New York, NY, USA (2021).  DOI 10.1145/3468264.3468537.  URL https://doi.org/10.1145/3468264.3468537
  * (14) Chakraborty, J., Majumder, S., Yu, Z., Menzies, T.: Fairway: a way to build fair ML software.  In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, pp. 654–665. Association for Computing Machinery, New York, NY, USA (2020).  DOI 10.1145/3368089.3409697
  * (15) Chen, Z., Zhang, J.M., Hort, M., Sarro, F., Harman, M.: Fairness Testing: A Comprehensive Survey and Analysis of Trends (2022).  URL http://arxiv.org/abs/2207.10223.  ArXiv:2207.10223 [cs]
  * (16) Chen, Z., Zhang, J.M., Sarro, F., Harman, M.: MAAT: a novel ensemble approach to addressing fairness and performance bugs for machine learning software.  In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022, pp. 1122–1134. Association for Computing Machinery, New York, NY, USA (2022).  DOI 10.1145/3540250.3549093.  URL https://dl.acm.org/doi/10.1145/3540250.3549093
  * (17) d’Aloisio, G., Di Marco, A., Stilo, G.: Democratizing quality-based machine learning development through extended feature models.  In: L. Lambers, S. Uchitel (eds.) Fundamental Approaches to Software Engineering, pp. 88–110. Springer Nature Switzerland, Cham (2023)
  * (18) Deldjoo, Y., Anelli, V.W., Zamani, H., Bellogín, A., Di Noia, T.: Recommender systems fairness evaluation via generalized cross entropy.  arXiv preprint arXiv:1908.06708 (2019)
  * (19) Deldjoo, Y., Jannach, D., Bellogin, A., Difonzo, A., Zanzonelli, D.: Fairness in recommender systems: Research landscape and future directions (2022)
  * (20) Deldjoo, Y., Jannach, D., Bellogin, A., Difonzo, A., Zanzonelli, D.: Fairness in recommender systems: research landscape and future directions.  User Modeling and User-Adapted Interaction (2023).  DOI 10.1007/s11257-023-09364-z.  URL https://doi.org/10.1007/s11257-023-09364-z
  * (21) Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness.  In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS ’12, pp. 214–226. Association for Computing Machinery, New York, NY, USA (2012).  DOI 10.1145/2090236.2090255
  * (22) d’Aloisio, G., D’Angelo, A., Di Marco, A., Stilo, G.: Debiaser for Multiple Variables to enhance fairness in classification tasks.  Information Processing & Management 60(2), 103226 (2023).  DOI 10.1016/j.ipm.2022.103226.  URL https://www.sciencedirect.com/science/article/pii/S0306457322003272
  * (23) European Commission: Regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts (2021).  URL https://eur-lex.europa.eu/resource.html?uri=cellar:e0649735-a372-11eb-9585-01aa75ed71a1.0001.02/DOC_1&format=PDF
  * (24) Fan, M., Wei, W., Jin, W., Yang, Z., Liu, T.: Explanation-guided fairness testing through genetic algorithm.  In: Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, p. 871–882. Association for Computing Machinery, New York, NY, USA (2022).  DOI 10.1145/3510003.3510137.  URL https://doi.org/10.1145/3510003.3510137
  * (25) Giray, G.: A software engineering perspective on engineering machine learning systems: State of the art and challenges.  Journal of Systems and Software 180, 111031 (2021).  DOI 10.1016/j.jss.2021.111031
  * (26) Hardt, M., Price, E., Price, E., Srebro, N.: Equality of Opportunity in Supervised Learning.  In: Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016).  URL https://proceedings.neurips.cc/paper/2016/hash/9d2682367c3935defcb1f9e247a97c0d-Abstract.html
  * (27) Hofmann, H.: Uci machine learning repository: Statlog (german credit data) data set (2013)
  * (28) Hort, M., Zhang, J.M., Sarro, F., Harman, M.: Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods.  In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021, p. 994–1006. Association for Computing Machinery, New York, NY, USA (2021).  DOI 10.1145/3468264.3468565.  URL https://doi.org/10.1145/3468264.3468565
  * (29) Johnson, B., Brun, Y.: Fairkit-learn: A fairness evaluation and comparison toolkit.  In: 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 70–74 (2022).  DOI 10.1145/3510454.3516830
  * (30) Kohavi, R.: Scaling Up the Accuracy of Naive-Bayes Classifiers: a Decision-Tree Hybrid p. 6
  * (31) Kumeno, F.: Sofware engneering challenges for machine learning applications: A literature review.  Intelligent Decision Technologies 13(4), 463–476 (2020).  DOI 10.3233/IDT-190160.  URL https://www.medra.org/servlet/aliasResolver?alias=iospress&doi=10.3233/IDT-190160
  * (32) Kusner, M.J., Loftus, J., Russell, C., Silva, R.: Counterfactual Fairness.  In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017).  URL https://proceedings.neurips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html
  * (33) Lee, M.S.A., Singh, J.: The Landscape and Gaps in Open Source Fairness Toolkits.  In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, pp. 1–13. Association for Computing Machinery, New York, NY, USA (2021).  DOI 10.1145/3411764.3445261
  * (34) Li, Y., Meng, L., Chen, L., Yu, L., Wu, D., Zhou, Y., Xu, B.: Training data debugging for the fairness of machine learning software.  In: Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, p. 2215–2227. Association for Computing Machinery, New York, NY, USA (2022).  DOI 10.1145/3510003.3510091.  URL https://doi.org/10.1145/3510003.3510091
  * (35) Wes McKinney: Data Structures for Statistical Computing in Python.  In: Stéfan van der Walt, Jarrod Millman (eds.) Proceedings of the 9th Python in Science Conference, pp. 56 – 61 (2010).  DOI 10.25080/Majora-92bf1922-00a
  * (36) Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A Survey on Bias and Fairness in Machine Learning.  ACM Computing Surveys 54(6), 1–35 (2021).  DOI 10.1145/3457607
  * (37) Moro, S., Cortez, P., Rita, P.: A data-driven approach to predict the success of bank telemarketing.  Decision Support Systems 62, 22–31 (2014)
  * (38) Muccini, H., Vaidhyanathan, K.: Software Architecture for ML-based Systems: What Exists and What Lies Ahead.  In: 2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN), pp. 121–128 (2021).  DOI 10.1109/WAIN52551.2021.00026
  * (39) Nahar, N., Zhang, H., Lewis, G., Zhou, S., Kästner, C.: A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners (2023).  DOI 10.48550/arXiv.2304.00078.  ArXiv:2304.00078 [cs]
  * (40) Nguyen, P.T., Di Rocco, J., Di Ruscio, D., Di Penta, M.: CrossRec: Supporting Software Developers by Recommending Third-party Libraries.  Journal of Systems and Software p. 110460 (2019).  DOI https://doi.org/10.1016/j.jss.2019.110460.  URL http://www.sciencedirect.com/science/article/pii/S0164121219302341
  * (41) Nguyen, P.T., Rubei, R., Rocco, J.D., Sipio, C.D., Ruscio, D.D., Penta, M.D.: Dealing with popularity bias in recommender systems for third-party libraries: How far are we? (2023)
  * (42) Olteanu, A., Castillo, C., Diaz, F., Kıcıman, E.: Social data: Biases, methodological pitfalls, and ethical boundaries.  Frontiers in big data 2, 13 (2019)
  * (43) Peng, K., Chakraborty, J., Menzies, T.: FairMask: Better Fairness via Model-Based Rebalancing of Protected Attributes.  IEEE Transactions on Software Engineering 49(4), 2426–2439 (2023).  DOI 10.1109/TSE.2022.3220713.  Conference Name: IEEE Transactions on Software Engineering
  * (44) Perera, A., Aleti, A., Tantithamthavorn, C., Jiarpakdee, J., Turhan, B., Kuhn, L., Walker, K.: Search-based fairness testing for regression-based machine learning systems.  Empirical Software Engineering 27(3), 79 (2022).  DOI 10.1007/s10664-022-10116-7
  * (45) Rocco, J.D., Sipio, C.D.: Resyduo: Combining data models and cf-based recommender systems to develop arduino projects (2023)
  * (46) Sharma, A., Wehrheim, H.: Testing Machine Learning Algorithms for Balanced Data Usage.  In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 125–135 (2019).  DOI 10.1109/ICST.2019.00022.  ISSN: 2159-4848
  * (47) Soremekun, E., Udeshi, S., Chattopadhyay, S.: Astraea: Grammar-Based Fairness Testing.  IEEE Transactions on Software Engineering 48(12), 5188–5211 (2022).  DOI 10.1109/TSE.2022.3141758.  Conference Name: IEEE Transactions on Software Engineering
  * (48) Steinberg, D., Budinsky, F., Merks, E., Paternostro, M.: EMF: eclipse modeling framework.  Pearson Education (2008)
  * (49) Tian, Y., Zhong, Z., Ordonez, V., Kaiser, G., Ray, B.: Testing dnn image classifiers for confusion & bias errors.  In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, ICSE ’20, p. 1122–1134. Association for Computing Machinery, New York, NY, USA (2020).  DOI 10.1145/3377811.3380400.  URL https://doi.org/10.1145/3377811.3380400
  * (50) Tizpaz-Niari, S., Kumar, A., Tan, G., Trivedi, A.: Fairness-aware configuration of machine learning libraries.  In: Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, p. 909–920. Association for Computing Machinery, New York, NY, USA (2022).  DOI 10.1145/3510003.3510202.  URL https://doi.org/10.1145/3510003.3510202
  * (51) Udeshi, S., Arora, P., Chattopadhyay, S.: Automated directed fairness testing.  In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE ’18, p. 98–108. Association for Computing Machinery, New York, NY, USA (2018).  DOI 10.1145/3238147.3238165.  URL https://doi.org/10.1145/3238147.3238165
  * (52) Villamizar, H., Escovedo, T., Kalinowski, M.: Requirements Engineering for Machine Learning: A Systematic Mapping Study.  In: 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 29–36. IEEE, Palermo, Italy (2021).  DOI 10.1109/SEAA53835.2021.00013
  * (53) Wang, J., Yang, Y., Wang, S., Hu, J., Wang, Q.: Context- and Fairness-Aware In-Process Crowdworker Recommendation.  ACM Transactions on Software Engineering and Methodology 31(3), 35:1–35:31 (2022).  DOI 10.1145/3487571
  * (54) Wang, Y., Ma, W., Zhang, M., Liu, Y., Ma, S.: A Survey on the Fairness of Recommender Systems.  ACM Transactions on Information Systems 41(3), 52:1–52:43 (2023).  DOI 10.1145/3547333.  URL https://dl.acm.org/doi/10.1145/3547333
  * (55) Yohannis, A., Kolovos, D.: Towards model-based bias mitigation in machine learning.  In: Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems, MODELS ’22, p. 143–153. Association for Computing Machinery, New York, NY, USA (2022).  DOI 10.1145/3550355.3552401.  URL https://doi.org/10.1145/3550355.3552401
  * (56) Zhang, H., Babar, M.A., Tell, P.: Identifying relevant studies in software engineering.  Inf. Softw. Technol. 53(6), 625–637 (2011).  DOI 10.1016/j.infsof.2010.12.010.  URL https://doi.org/10.1016/j.infsof.2010.12.010
  * (57) Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine Learning Testing: Survey, Landscapes and Horizons.  IEEE Transactions on Software Engineering pp. 1–1 (2020).  DOI 10.1109/TSE.2019.2962027.  URL https://ieeexplore.ieee.org/document/9000651/
  * (58) Zhang, L., Zhang, Y., Zhang, M.: Efficient white-box fairness testing through gradient search.  In: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 103–114 (2021)
  * (59) Zhang, P., Wang, J., Sun, J., Wang, X., Dong, G., Wang, X., Dai, T., Dong, J.S.: Automatic Fairness Testing of Neural Classifiers Through Adversarial Sampling.  IEEE Transactions on Software Engineering 48(9), 3593–3612 (2022).  DOI 10.1109/TSE.2021.3101478.  Conference Name: IEEE Transactions on Software Engineering
  * (60) Zheng, H., Chen, Z., Du, T., Zhang, X., Cheng, Y., Ti, S., Wang, J., Yu, Y., Chen, J.: Neuronfair: Interpretable white-box fairness testing through biased neuron identification.  In: 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pp. 1519–1531 (2022).  DOI 10.1145/3510003.3510123