# Fair Mixed Effects Support Vector Machine

Jan Pablo Burgard, João Vitor Pamplona  Trier University, Department of
Economic and Social Statistics, Universitätsring 15, 54296 Trier, Germany
<EMAIL_ADDRESS><EMAIL_ADDRESS>
(Date: August 28, 2024)

###### Abstract.

To ensure unbiased and ethical automated predictions, fairness must be a core
principle in machine learning applications. Fairness in machine learning aims
to mitigate biases present in the training data and model imperfections that
could lead to discriminatory outcomes. This is achieved by preventing the
model from making decisions based on sensitive characteristics like ethnicity
or sexual orientation. A fundamental assumption in machine learning is the
independence of observations. However, this assumption often does not hold
true for data describing social phenomena, where data points are often
clustered based. Hence, if the machine learning models do not account for the
cluster correlations, the results may be biased. Especially high is the bias
in cases where the cluster assignment is correlated to the variable of
interest. We present a fair mixed effects support vector machine algorithm
that can handle both problems simultaneously. With a reproducible simulation
study we demonstrate the impact of clustered data on the quality of fair
machine learning predictions.

###### Key words and phrases:

Support Vector Machine, Fair Machine Learning, Mixed Models.

###### 2020 Mathematics Subject Classification:

90C90, 90-08, 68T99

##  1\. Introduction

The rise of automated decision-making systems calls for the development of
fair algorithms. These algorithms must be constrained by societal values,
particularly to avoid discrimination against any population group. While
machine learning offers efficiency gains, it can also unintentionally
perpetuate biases in critical areas like loan approvals ([11]) and criminal
justice ([15]). In loan applications, factors like marital status can lead to
unfair disadvantages for single individuals, while in criminal justice,
algorithms might associate race with recidivism risk, leading to
discriminatory sentencing despite individual circumstances. This highlights
the need for fair and unbiased AI frameworks to ensure equal opportunities and
outcomes for all.

A key challenge in machine learning for automated decision-making is the
training data. Often sourced from surveys, this data may not perfectly align
with the assumption common in machine learning that all elements are sampled
independently with equal probability of inclusion. Survey methods, including
various sampling strategies, are inherently designed to target specific
populations, potentially biasing the data. For a detailed discussion on survey
methods, see [19].

Driven by the need to mitigate bias in algorithms, fair machine learning is
experiencing a surge in research activity. Numerous research articles are
exploring innovative approaches to achieve fairer outcomes. Notable examples
include fair versions of Logistic and Linear Regression ([3]), Support Vector
Machines ([21]), Random Forests ([28]), Decision Trees ([1]), and Generalized
Linear Models (GLMs) ([12]). These methods aim to address potential
discrimination arising from historical data or algorithmic design, ensuring
fairer outcomes for all individuals.

In this paper we propose a Mixed Effects Support Vector Machine for fair
predictions. We show how to estimate the model and evaluate its performance
against the current model that does not take the possible clustering of the
data into account. Similar approaches for longitudinal data can be found in
[20, 18], for Least-squares support vector machine in [8] and applications in
agricultural activities in [24].

The paper is organized as follows: In Section 2 we will explore the theory and
metrics behind fairness in machine learning and how it applies to support
vector machines. In Section 3 we establish the theoretical underpinnings of
fair mixed effects support vector machine and propose a strategy for solving
them. In Section 4, we conduct a comprehensive evaluation of our proposed
method’s effectiveness through various tests. Finally, in Section 5, we
demonstrate the practical applicability of our algorithm by solving a real-
world problem using the Adult dataset ([2]). Our key findings and potential
future directions are presented in Section 6.

##  2\. Fair Support Vector Machine

In a binary classification, identifying a function that accurately predicts a
label $\hat{y}\in\\{0,1\\}$ from a feature vector $x\in\mathbb{R}^{p}$ is
paramount. This objective is achieved by training the model on a dataset
$D={(x^{\ell},y_{\ell})}^{N}_{\ell=1}$.

In the realm of fair binary classification, each observation $x^{\ell}$
possesses a corresponding sensitive attribute $s_{\ell}$ a binary value of
either $0$ or $1$. The goal becomes identifying a solution that balances
accuracy with fairness. Fairness in machine learning can be assessed using
various metrics, but this discussion focuses on disparate impact (DI).
Alternative fairness metrics for binary classifiers can be found in [27].

A binary classifier is deemed to be free from disparate impact if the
probability of it classifying an instance as either $0$ or $1$ is identical
for both values of the sensitive feature being considered. In other terms,
disparate impact is absent when the classifier treats all groups equally
regardless of their sensitive feature values. Mathematically, we can write
this as:

$P(\hat{y}_{\ell}=1|s_{\ell}=0)=P(\hat{y}_{\ell}=1|s_{\ell}=1).$

An algorithm exhibits disparate impact when it produces significantly more
favorable outcomes, for certain groups defined by sensitive features. In the
following we present how to add the fairness in the context of Support Vector
Machine (SVM).

In SVM, proposed by [25] and [16], a hyperplane $\beta^{\top}x$ splits the
data based on its classification. Note that in our formulation the first
column of X is equal to 1 everywhere and, hence, the bias parameter is the
first entry in $\beta$ ([17]). By adding the fairness constraint, our fixed
effect $\beta$ can be found by solving the following quadratic optimization
problem.

$\displaystyle\underset{\left(\beta,\xi\right)}{\min}$
$\displaystyle\;\;\;\;\frac{1}{2}\|\beta\|^{2}+K\sum_{\ell=1}^{N}\xi_{\ell}$
(1) $\displaystyle\rm{s.t}$
$\displaystyle\;\;\;\;y_{\ell}(\beta^{\top}x_{\ell})\geq 1-\xi_{\ell},\text{
}\ell=1,\dots,N,$ (2)
$\displaystyle\;\;\;\;\frac{1}{N}\sum\limits_{\ell=1}^{N}(s_{\ell}-\bar{s})(\beta^{\top}x_{\ell})\leq
c,$ (3)
$\displaystyle\;\;\;\;\frac{1}{N}\sum\limits_{\ell=1}^{N}(s_{\ell}-\bar{s})(\beta^{\top}x_{\ell})\geq-c,$
(4) $\displaystyle\;\;\;\;\xi_{\ell}\geq 0,\text{ }\ell=1,\dots,N,$ (5)

with

$\bar{s}=\dfrac{\displaystyle\sum_{\ell=1}^{N}s_{\ell}}{N},$ (6)

and $s$ being the sensitive feature. Constraints (1), (2) and (5) are from the
Support Vector Machine problem. Constraints (3) and (4) guarantees the
fairness in the prediction. The construction and justification of these
constraints can be found in [26]. The parameter $c\in\mathbb{R}^{+}$ is a
threshold that controls the importance of fairness. We refer to the problem
above as Fair Support Vector Machine (SVMF).

##  3\. Fair Mixed Effects Support Vector Machine

Often, data exhibits unexplained variability within groups or at different
levels. While these “random effects” remain unobserved directly, statistical
methods can estimate their impact. They represent deviations from the overall
trend specific to individuals or groups as we can see in clustered or
stratified data. To account for this variability, we include them into our
optimization problem. For that, we incorporate both fixed effects (average
effect) and random effects in our SVM model.

First we need incorporate, into the Objective function (1), the random effects
$g_{i}$, with $i$ being the random effect group, where $i\leavevmode\nobreak\
\in\leavevmode\nobreak\ [1,n]\leavevmode\nobreak\ :=\leavevmode\nobreak\
\\{1,\dots,n\\}$ and $n$ the number of groups. We can add random effects to
the objective function by adding a term that penalizes the variance of the
random effects. This helps to ensure that the parameters of the model are
estimated in a way that minimizes bias and provides accurate and reliable
results. Since the variance of $g$ is:

$\sigma^{2}=\frac{\sum_{i=1}^{n}(g_{i}-\bar{g})^{2}}{n-1},$

and the random effect $g$ follows a normal distribution with mean zero, then:

$\displaystyle\sigma^{2}$
$\displaystyle=\frac{\sum_{i=1}^{n}(g_{i}-0)^{2}}{n-1}=\frac{\sum_{i=1}^{n}g_{i}^{2}}{n-1}.$

This means that our regularization term is
$\frac{\sum_{i=1}^{n}g_{i}^{2}}{n-1}$. Since we have a minimization problem
and $n-1$ is fixed, we can consider w.l.o.g a parameter $\lambda$ that
controls the importance of the random effects.

In the Constraint (2) and in the fairness constraints (3) and (4), we also add
the random effect. This allows the fixed effects $\beta$’s to find the
hyperplane that separates the data, while the random effects allow this
hyperplane to vary in position depending on the group. So each group has the
same fixed effect but different random effect.

Consequently, we formulate the following constrained optimization problem:

$\displaystyle\begin{split}\underset{\left(\beta,b,\xi\right)}{\min}&\;\;\;\;\frac{1}{2}\|\beta\|^{2}+K\sum_{i=1}^{n}\sum_{j=1}^{T_{i}}\xi_{i_{j}}+\lambda\sum_{i}^{n}g_{i}^{2}\\\
\rm{s.t}&\;\;\;\;y_{i_{j}}(\beta^{\top}x_{i_{j}}+g_{i})\geq
1-\xi_{i_{j}},\text{ }i=1,\dots,n,\text{ }j=1,\dots,T_{i},\\\
&\;\;\;\;\frac{1}{N}\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{T_{i}}(s_{i_{j}}-\bar{s})(\beta^{\top}x_{i_{j}}+g_{i})\leq
c,\\\
\;&\;\;\;\;\frac{1}{N}\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{T_{i}}(s_{i_{j}}-\bar{s})(\beta^{\top}x_{i_{j}}+g_{i})\geq-c,\\\
&\;\;\;\;\xi_{i_{j}}\geq 0,\text{ }i=1,\dots,n,\text{
}j=1,\dots,T_{i},\end{split}$ (7)

with, $y_{i_{j}}$ being the label in the observation $j$ in group $i$ and
$j\leavevmode\nobreak\ \in\leavevmode\nobreak\ [1,T_{i}]$, and $T_{i}$ the
size of the group $i$. We refer to the problem above as Fair Mixed Effects
Support Vector Machine (FMESVM) and without the fairness constraints as Mixed
Effects Support Vector Machine (MESVM).

One alternative approach to incorporating random effects into grouped data
analysis involves the use of dummy variables, also known as one-hot encoding
as can be seen in [14]. This strategy involves creating a separate binary
variable for each group, taking a value of $1$ for observations within that
group and $0$ otherwise. While this method can effectively capture group-level
variation, preliminary numerical tests have demonstrated that our proposed
approach (7) offers significant advantages in terms of both computational
efficiency and memory usage, making it a more resource-friendly option for
large datasets and complex models in both situations where the datasets
present unfairness and when they do not.

The following preliminary numerical experiment employed a dataset consisting
of $100000$ points with the training set having 3 to 5 points per group and
systematically varying the number of groups within the data, considering
configurations with $2$, $10$, $50$, $500$, $1000$, $1250$, $2000$, $2500$,
$3125$, $4000$, and $5000$ groups.

The Figures 1 and 2 shows two different analyses of the data. The top line
visualizes a second-order polynomial fit. The bottom line, on the other hand,
creates a Performance Profile proposed by [13] of both approaches.

Figure 1. Memory comparison in Bytes. Figure 2. Time comparison in
Microsecond.

The next section dives into numerical tests, demonstrating the efficacy of the
proposed methodology.

##  4\. Simulation Study

First we present the step-by-step strategy used to create the datasets and to
conduct the numerical experiments. Using the Julia 1.9 ([5]) language with the
packages Distributions ([4]), DataFrames ([7]), MLJ ([6]) and MKL ([10]), we
generate the following parameters:

  * •

_Number of points_ : Number of points in the dataset;

  * •

_$\beta^{\prime}s$_ : The fixed effects;

  * •

_$g^{\prime}s$_ : The random effects with distribution $N(0,2)$;

  * •

_Data points_ : The covariate vector associated with fixed effects with
distribution $N(0,1)$;

  * •

$c$: Threshold from Fair Support Vector Machine;

  * •

_seed_ : Random seed used in the generation of data;

  * •

_Train-Test split_ : Approximately 0.4% of the dataset was used for the
training set, and 99.6% for the test set. This percentage was due to the fact
that we randomly selected 3 to 5 points from each cluster for the training
set.

then the predictions of the synthetic dataset were computed using

$m=\beta^{T}x+g$ (8)

in tests where the dataset has random effects, and

$m=\beta^{T}x$ (9)

in the tests where the dataset has just fixed effects. Since
$m\in[-\infty,\infty]$, we project the value to $[0,1]$ using the function:

$M=\frac{exp(m)}{1+exp(m)}.$ (10)

Finally,

$y=Binomial(1,M).$ (11)

Thus, our datasets are ready.

Regarding the tests, the comparisons will be made between the following
optimizations problems:

  1. (1)

Mixed Effects Support Vector Machine (MESVM);

  2. (2)

Fair Mixed Effects Support Vector Machine (FMESVM);

  3. (3)

Support Vector Machine (SVM);

  4. (4)

Fair Support Vector Machine (SVMF).

comparing the accuracy and the disparate impact between them. The test were
conducted on a computer with an Intel Core i9-13900HX processor with a clock
speed of 5.40 GHz, 64 GB of RAM, and Windows 11 operating system, with 64-bit
architecture.

To compute accuracy, first we need compute the predictions so we use equations
(8) - (11) with,

$\hat{y}=\begin{cases}1\hskip 11.38092pt\text{if }\hskip 1.42271ptm\geq 0.5\\\
-1\hskip 11.38092pt\text{if }\hskip 1.42271ptm<0.5\end{cases}\,,$

and given the true label of all points, we can distinguish them into four
categories: true positive (TP) or true negative (TN) if the point is
classified correctly in the positive or negative class, respectively, and
false positive (FP) or false negative (FN) if the point is misclassified in
the positive or negative class, respectively. Based on this, we can compute
the accuracy, where a higher value indicates a better classification, as
follows,

$AC=\dfrac{TP+TN}{TP+TN+FP+FN}\in[0,1].$

To compute the Disparate Impact of a specific sensitive feature $s$ we use the
following equation as can be seen in [22],

$DI=\min\Big{(}\dfrac{p(\hat{y}|s=1)}{p(\hat{y}|s=0)},\dfrac{p(\hat{y}|s=0)}{p(\hat{y}|s=1)}\Big{)}\in[0,1].$

Disparate impact, as a measure, should be equal to 1. This indicates that
discrimination does not exist. Values greater or lower than 1 suggest that
unwanted discrimination exists. However, $DI=2$ and $DI=0.5$ represent the
same level of discrimination, although in the first case, the difference
between the perfect value is 1, and in the latter case, it is 0.5. To avoid
such situations, we use the minimum of two possible values.

The parameters for the Unfair cases are:

  * •

$\beta$’s $=[-1.5,0.4,0.8,0.5,1.5]$;

  * •

$g$’s: 100 clusters with $b_{i}\sim N(0,Q)$, with $i\in[1,100]$;

  * •

$c=10^{-3}$;

  * •

$K=1$;

  * •

$\lambda=1$.

And for the fair cases are:

  * •

$\beta$’s $=[-1,1,2,1,0.1]$;

  * •

$g$’s: 100 clusters with $b_{i}\sim N(0,Q)$, with $i\in[1,100]$;

  * •

$c=10^{-3}$;

  * •

$K=1$;

  * •

$\lambda=1$.

with $Q$ being $2$ for cases with random effect and $0$ for cases without
random effect.

The following figures present experimental results on four different datasets
with 1000 samples each. The datasets are presented individually, and the
changes between samples occur only in the training set. Each figure contains
four corresponding images representing accuracy and disparate impact for all
possible datas. All figures were created using the Plots and PlotlyJS
packages, developed by [9].

Figure 3. Accuracy. Figure 4. Disparate Impact.

Figure 3 demonstrates that our proposed approach consistently outperformed
alternative methods in scenarios with random effects. In settings without
random effects, both methods achieved comparable accuracy, as anticipated.
However, a slight decrease in accuracy was observed when introducing
unfairness in the data. This is understandable, as our approach must balance
addressing both random effects and unfairness simultaneously.

Figure 4 demonstrates that our approach consistently yields improved disparate
impact metrics when applied to datasets containing inherent biases.
Conversely, in scenarios where the underlying data exhibits no inherent bias,
our approach produces equivalent outcomes, as there is no inherent disparity
to mitigate.

##  5\. Application

In this set of experiments, we will test the Adult dataset. To test the
method’s efficiency, we created groups based on individuals age and marital
status, and a sensitive feature, gender ([23]). The Adult dataset is a famous
tool in machine learning, where the goal is predicting whether the individuals
earn more ($y=1$) or less ($y=0$) than $50000$ USD annually. Were conducted
1000 samples with 0.5% of the data as the training set and the remaining data
as the test set.

The features used in the prediction process are the follows:

  * •

Age: The age of the individual in years;

  * •

Capital_gain: Capital gain in the previous year;

  * •

Capital_loss: Capital loss in the previous year;

  * •

Education: Highest level of education achieved by the individual;

  * •

Education_num: A numeric form of the highest level of education achieved;

  * •

Fnlwgt: An estimate of the number of individuals in the population with the
same demographics as this individual;

  * •

Hours_per_week: Hours worked per week;

  * •

Marital_status: The marital status of the individual;

  * •

Native_country: The native country of the individual;

  * •

Occupation: The occupation of the individual;

  * •

Race: The individual’s race;

  * •

Relationship: The individual’s relationship status;

  * •

Gender: The individual’s gender;

  * •

Workclass: The sector that the individual works in.

Figure 5. Accuracy. Figure 6. Disparate Impact.

As can be seen in Figure 5, in this set of experiments, we obtained a better
accuracy in MESVM and FMESVM in comparison to regular SVM since this one not
account for random effects.

Figure 6 show that we also obtained an improvement in the disparate impact on
the Fair algorithms. Note that make sense since we have a unfair population.

Upon examining all the tests, we were able to observe an improvement in
Disparate Impact in ($100\%$) of the cases.

##  6\. Conclusion

This study investigated a novel approach to mitigating disparate impact, a
fairness issue in machine learning models, while simultaneously addressing
mixed effects. We introduce a novel Fair Mixed Effects Support Vector Machine
(FMESVM) algorithm that tackles both concerns cohesively, overcoming the
limitations of existing methods often dedicated to separate problem solving.
This integrated approach tailors treatments to the specific demands of each
issue, ensuring optimal performance.

Employing the widely respected Support Vector Machine (SVM) for binary
classification, the FMESVM framework incorporates mixed effects within the SVM
setting and deploys novel regularization techniques to effectively mitigate
disparate impact. Extensive evaluation across diverse datasets and metrics
demonstrates the success of our proposed method in demonstrably reducing
disparate impact while maintaining or minimally compromising overall accuracy.

For comprehensive experimentation, we systematically explored all possible
scenarios involving the two concerns: datasets exhibiting both, only
unfairness with random effects, only fairness with random effects, and neither
issue. This approach yielded expected results, with each combination directly
impacting accuracy or disparate impact as predicted.

The FMESVM presents a significant advancement in fairness-aware machine
learning by comprehensively addressing disparate impact and mixed effects
through a unified framework. This paves the way for the development of more
robust and ethical machine learning models with broader applicability.

## Acknowledgements

The authors are grateful for the support of the German Federal Ministry of
Education and Research (BMBF) for this research project, as well as for the
“OptimAgent Project”.

We would also like to express our sincere appreciation for the generous
support provided by the German Research Foundation (DFG) within Research
Training Group 2126 “Algorithmic Optimization”.

## References

  * [1] S Aghaei, M J Azizi and P Vayanos  “Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making”  In _CoRR_ abs/1903.10598, 2019  arXiv: http://arxiv.org/abs/1903.10598
  * [2] B Becker and R Kohavi  “Adult” DOI: https://doi.org/10.24432/C5XW20, UCI Machine Learning Repository, 1996
  * [3] R Berk et al.  “A convex framework for fair regression”  In _arXiv preprint arXiv:1706.02409_ , 2017
  * [4] M Besançon et al.  “Distributions.jl: Definition and Modeling of Probability Distributions in the JuliaStats Ecosystem”  In _Journal of Statistical Software_ 98.16, 2021, pp. 1–30  DOI: 10.18637/jss.v098.i16
  * [5] J Bezanson, A Edelman, S Karpinski and V B Shah  “Julia: A fresh approach to numerical computing”  In _SIAM review_ 59.1, 2017, pp. 65–98  DOI: 10.1137/14100067
  * [6] A Blaom, F Kiraly, T Lienart and S Vollmer  “alan-turing-institute/MLJ.jl: v0.5.3”  Zenodo, 2019  DOI: 10.5281/zenodo.3541506
  * [7] M Bouchet-Valat and B Kamiński  “DataFrames.jl: Flexible and Fast Tabular Data in Julia”  In _Journal of Statistical Software_ 107.4, 2023, pp. 1–32  DOI: 10.18637/jss.v107.i04
  * [8] Q Cheng, J Tezcan and J Cheng  “Confidence and prediction intervals for semiparametric mixed-effect least squares support vector machine”  In _Pattern Recognition Letters_ 40, 2014, pp. 88–95  DOI: https://doi.org/10.1016/j.patrec.2013.12.010
  * [9] S Christ et al.  “Plots.jl – a user extendable plotting API for the julia programming language”  Journal of Open Research Software, 2023  DOI: https://doi.org/10.5334/jors.431
  * [10] Intel Corporation  “Intel Math Kernel Library”  Intel Corporation, 2023
  * [11] S Das et al.  “Fairness measures for machine learning in finance”  In _The Journal of Financial Data Science_ Institutional Investor Journals Umbrella, 2021
  * [12] H Do et al.  “Fair generalized linear models with a convex penalty”  In _International Conference on Machine Learning_ , 2022, pp. 5286–5308  PMLR
  * [13] E D Dolan and J J Moré  “Benchmarking Optimization Software with Performance Profiles”  In _Math. Program._ 91, 2002, pp. 201–213  DOI: 10.1007/s101070100263
  * [14] A Fernandes, Solimun and Nurjannah  “Computational Statistics with Dummy Variables”, 2022  DOI: 10.5772/intechopen.101460
  * [15] B Green  “Fair risk assessments: A precarious approach for criminal justice reform”  In _5th Workshop on fairness, accountability, and transparency in machine learning_ , 2018, pp. 1–5
  * [16] M A Hearst et al.  “Support vector machines”  In _IEEE Intelligent Systems and their applications_ 13.4 IEEE, 1998, pp. 18–28
  * [17] C J Hsieh, K Chang and C J Lin  “A dual coordinate descent method for large-scale linear SVM”  In _Proceedings of the Twenty-fifth International Conference on Machine Learning_ , 2008, pp. 1369–1398
  * [18] S Hu, Y Wang, C Drovandi and T Cao  “Predictions of machine learning with mixed-effects in analyzing longitudinal data under model misspecification”  In _Statistical Methods & Applications_ 32, 2022, pp. 1–31  DOI: 10.1007/s10260-022-00658-x
  * [19] Sharon L Lohr  “Sampling : Design and Analysis”  Florence, KY: Brooks/Cole, 2009
  * [20] J Luts et al.  “A mixed effects least squares support vector machine model for classification of longitudinal data”  In _Computational Statistics & Data Analysis_ 56, 2012, pp. 611–628  DOI: 10.1016/j.csda.2011.09.008
  * [21] M Olfat and A Aswani  “Spectral Algorithms for Computing Fair Support Vector Machines”  In _CoRR_ abs/1710.05895, 2017  arXiv: http://arxiv.org/abs/1710.05895
  * [22] S Radovanović, A Petrović, B Delibašić and M Suknović  “Enforcing fairness in logistic regression algorithm”  In _2020 International Conference on INnovations in Intelligent SysTems and Applications (INISTA)_ , 2020, pp. 1–7  DOI: 10.1109/INISTA49547.2020.9194676
  * [23] T Speicher et al.  “A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices”  In _CoRR_ abs/1807.00787, 2018  arXiv:1807.00787
  * [24] P Srivastava et al.  “Support vector machines and generalized linear models for quantifying soil dehydrogenase activity in agro-forestry system of mid altitude central Himalaya”  In _Environmental Earth Sciences_ 75, 2016  DOI: 10.1007/s12665-015-5074-3
  * [25] V Vapnik and A Y Chervonenkis  “A class of algorithms for pattern recognition learning”  In _Avtomat. i Telemekh_ 25.6, 1964, pp. 937–945
  * [26] M B Zafar, I Valera, M Gomez-Rodriguez and K P Gummadi  “Fairness Constraints: Mechanisms for Fair Classification”  In _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ 54, Proceedings of Machine Learning Research  PMLR, 2017, pp. 962–970
  * [27] M B Zafar, I Valera, M Gomez-Rodriguez and K P Gummadi  “Fairness Constraints: A Flexible Approach for Fair Classification”  In _Journal of Machine Learning Research_ 20.75, 2019, pp. 1–42
  * [28] W Zhang et al.  “FARF: A Fair and Adaptive Random Forests Classifier”  In _CoRR_ abs/2108.07403, 2021  arXiv: https://arxiv.org/abs/2108.07403