# RobustFair: Adversarial Evaluation through Fairness Confusion Directed
Gradient Search

Xuran Li
State Key Laboratory of Computer Science,
Institute of Software, Chinese Academy of Sciences
University of Chinese Academy of Sciences
Beijing, China
<EMAIL_ADDRESS>
&Peng Wu
State Key Laboratory of Computer Science,
Institute of Software, Chinese Academy of Sciences
University of Chinese Academy of Sciences
Beijing, China
<EMAIL_ADDRESS>,
&Kaixiang Dong
Hangzhou Institute for Advanced Study,
University of Chinese Academy of Sciences
Hangzhou, China
State Key Laboratory of Computer Science,
Institute of Software, Chinese Academy of Sciences
Beijing, China
<EMAIL_ADDRESS>
&Zhen Zhang
State Key Laboratory of Computer Science,
Institute of Software, Chinese Academy of Sciences
University of Chinese Academy of Sciences
Beijing, China
<EMAIL_ADDRESS>
Corresponding author: Peng Wu

###### Abstract

The trustworthiness of deep neural networks is often challenged by their
vulnerability to minor adversarial perturbations, which may not only undermine
prediction accuracy (robustness) but also cause biased predictions for similar
inputs (individual fairness). Accurate fairness [1] has been recently proposed
to enforce a harmonic balance between accuracy and individual fairness. It
induces the notion of fairness confusion matrix to categorize predictions as
true fair, true biased, false fair, and false biased. In this paper, we
propose a harmonic evaluation approach, named RobustFair, for the accurate
fairness of a deep neural network, using adversarial perturbations crafted
through fairness confusion directed gradient search. By using Taylor
expansions to approximate the ground truths of adversarial instances,
RobustFair can particularly identify the robustness defects entangled for
spurious fairness, which are often elusive in robustness evaluation, and
missing in individual fairness evaluation. Moreover, RobustFair can boost
robustness and individual fairness evaluations by identifying robustness or
fairness defects simultaneously. Empirical case studies on fairness benchmark
datasets show that, compared with the state-of-the-art white-box robustness
and individual fairness testing approaches, RobustFair detects significantly
1.77-11.87 times adversarial perturbations, yielding 1.83-13.12 times biased
instances and 1.53-8.22 times false instances, including 1.62-3.00 times false
fair instances. The adversarial instances can then be effectively exploited to
improve the accurate fairness (and hence accuracy and individual fairness) of
the original deep neural network through retraining. The empirical case
studies further show that the adversarial instances identified by RobustFair
outperform those identified by the other testing approaches, in promoting 21%
accurate fairness and 19% individual fairness on multiple sensitive
attributes, without sacrificing accuracy at all or even promoting it by up to
4%.

##  1 Introduction

Deep neural networks (DNNs) have demonstrated impressive performance in
decision-making tasks. However, the widespread deployments of DNNs have
increasingly raised public concerns about their trustworthiness [2, 3, 4].
Studies have shown that DNNs are susceptible to minor adversarial
perturbations [5, 6], which can undermine not only the accuracy of predictions
[7] but also the fairness of decisions made for similar individuals [8].
Therefore, it is crucial to evaluate the impact of adversarial perturbations
comprehensively, particularly in cases where robustness and fairness issues
intertwine and may result in subtle defects. For example, in civil domains
such as income prediction [9], loan granting [10], and criminal justice risk
assessment [11], inaccurate or biased predictions can have significant
negative consequences, including reduced profits, compromised social security,
and violations of anti-discrimination laws and ethical principles. Thus, it is
challenging but essential to ensure DNNs are both functionally and ethically
correct.

Accurate fairness [1] has recently been proposed to reconcile accuracy with
individual fairness by enforcing that the predictions of an individual and its
similar counterparts shall all conform to the individual’s ground truth. An
individual and its similar counterparts differ only in their sensitive
features, such as gender, race, or age. Accurate fairness induces the notion
of fairness confusion matrix, as shown in Table 1, which categorizes
predictions as true fair, true biased, false fair, and false biased. _True
fair_ (TF) predictions satisfy exactly the accurate fairness criterion.

  * •

Instances with either _true biased_ (TB) or _false biased_ (FB) predictions
indicate that these instances and their similar counterparts receive different
predictions. These discriminations, no matter accurate or not, are targets of
individual fairness evaluation.

  * •

Instances with _false fair_ (FF) predictions indicate that the same
predictions made for these instances and their similar counterparts are
actually inaccurate for the instances themselves. This often emerges when
superficial fairness is achieved through bias mitigation but at the expense of
accuracy. These defects are certainly beyond the targets of individual
fairness evaluation, and may not be perceivable in robustness evaluation,
which usually does not take individual fairness into account.

Table 1: Fairness Confusion Matrix Accuracy Fairness | Fair | Biased
---|---|---
True | True Fair (TF) | True Biased (TB)
False | False Fair (FF) | False Biased (FB)

In this paper, we propose a novel approach, named RobustFair, to evaluate the
trustworthiness of DNNs from the perspective of accurate fairness. It utilizes
the fairness confusion matrix to guide a differentiated gradient search for
adversarial perturbations towards each category of false or biased defects.
Thus, RobustFair can integrate white-box robustness and individual fairness
evaluations organically. In the aspect of robustness, RobustFair employs
Taylor expansions to approximate the ground truths of perturbed instances in
order to identify adversarial instances with false fair or false biased
defects. In the aspect of individual fairness, RobustFair examines each
perturbed instance with all its similar counterparts in order to identify
adversarial instances with true biased or false biased defects.

We then implement RobustFair, and evaluate it on fairness benchmark datasets,
namely Adult (Census Income) [10], German Credit [12], Bank Marketing [9], and
ProPublica Recidivism (COMPAS) [11], to substantiate its effectiveness.
Compared to the state-of-the-art robustness and individual fairness testing
techniques, RobustFair can detect significantly 1.77-11.87 times false or
biased adversarial perturbations, yielding 1.15-12.17 times true biased
instances, 1.40-14.67 times false biased instances, and 1.62-3.00 times false
fair instances. This results in identifying 1.83-13.12 times biased instances
and 1.53-8.22 times false instances. Thus, RobustFair can boost a mutually
beneficial correlation between robustness and individual fairness evaluations
under the principle of accurate fairness, and especially discover those
robustness defects concealed by spurious fairness.

Furthermore, these identified adversarial instances can be effectively
exploited to augment original training data and retrain original DNNs for
higher accurate fairness. The empirical case studies show that the adversarial
instances identified by RobustFair can help promote 21% accurate fairness and
19% individual fairness on multiple sensitive attributes, without sacrificing
accuracy at all or even promoting it by up to 4%. This improvement is much
more superior than can be made with the instances identified by the other
testing approaches.

The main contributions of this paper are as follows:

  * •

We propose RobustFair, a novel adversarial evaluation approach for the
accurate fairness of DNNs. It categorizes adversarial instances directed by
the notion of fairness confusion matrix, and utilizes a gradient search
customized for each category of adversarial instances.

  * •

RobustFair constitutes an effective synergy of robustness and individual
fairness evaluations. It employs Taylor expansions to approximate the ground
truths of adversarial instances, which is a novel application in adversarial
evaluation.

  * •

Comparative case studies on fairness benchmark datasets demonstrate that
RobustFair outperforms the state-of-the-art robustness and individual fairness
evaluation approaches in both the quantity and quality of adversarial
instances identified. RobustFair can identify more adversarial instances with
more diversity, which can be exploited to promote more accurate fairness, and
individual fairness of DNNs, without sacrificing accuracy. This highlights the
superiority of our fairness confusion directed adversarial evaluation.

The rest of this paper is organized as follows. Section 2 reviews briefly
related work. Section 3 presents the RobustFair approach. Its evaluation
results are reported and analyzed in Section 4. The paper is concluded in
Section 5 with some future work.

##  2 Related Work

Efforts have been devoted to the trustworthiness of DNNs. In this section, we
mainly discuss gradient-based approaches for evaluating the robustness or
fairness of DNNs.

Robustness Evaluation The concept of DNN robustness concerns about a DNN
model’s ability to withstand perturbations in model inputs without degrading
its performance [13]. Typically, the robustness of a model is evaluated by
measuring or specifically attacking its accuracy in the presence of
adversarial perturbations [14]. The Fast Gradient Sign Method (FGSM) [13] and
Projected Gradient Descent (PGD) attack [15] are popular algorithms that may
mislead the model by imperceptibly perturbing an input in the direction of the
sign of the gradient of the loss function. The Auto Projected Gradient Descent
(APGD) [16], a more advanced variant of PGD, considers iteratively the inertia
direction of a search point with an adaptive step-size. The Auto Conjugate
Gradient (ACG) [17] utilizes the conjugate gradient method to extensively
search adversarial examples by updating a search point in more diverse
directions. However, these approaches pay less attention to robustness defects
resulted from the incompatible trade-off between accuracy and fairness [18].
For example, in an extreme case, enforcing the same predictions for every
input certainly meets any fairness criterion, but sacrifices accuracy
inevitably. Besides, a robust model may still perpetuate discrimination if
biased patterns exist in training data.

Fairness Evaluation Fairness asserts that DNNs treat similar groups or
individuals equally. To evaluate group fairness violations, [19] proposes a
statistical testing framework that uses optimal transport to project the
empirical measure onto the set of group-fair probability models. [20] presents
a statistical hypothesis testing to determine whether prediction errors are
distributed similarly across similar groups. For individual fairness testing,
the Adversarial Discrimination Finder (ADF) [8, 21] utilizes a two-phase
gradient search to enlarge prediction differences among similar individuals.
The Efficient Individual Discriminatory Instances Generator (EIDIG) [22] uses
a momentum optimization and intermediate gradient information to accelerate
individual fairness testing. However, current individual fairness evaluation
approaches do not take into account the impact of perturbations on prediction
accuracy, partly because the test oracles, e.g., the ground truths, for the
generated instances are generally not available. This may be an advantage of
individual fairness evaluation, but also limits its applicability as the
evaluation results may be intertwined with robust defects.

The above gradient-based approaches evaluate DNNS against adversarial
perturbations respectively from the perspectives of prediction accuracy and
individual fairness, but both criteria are not well coordinated on their own.
The recently proposed criterion of accurate fairness establishes an orthogonal
confusion aligning individual fairness with accuracy. RobustFair presented in
this paper is an accurate fairness evaluation approach that utilizes a
customized two-stage gradient search for finer-grained prediction defects
under the direction of fairness confusion.

##  3 RobustFair

In this section, we present the workflow of RobustFair, and especially focus
on its solutions of fairness confusion directed adversarial test generation
and Taylor expansions based test oracle generation. Assume a finite and
labeled dataset $V\subseteq X\times A$ with the domains of the non-sensitive
attributes, the sensitive attributes, and the ground truths denoted $A,X,Y$,
respectively. Each input $v=(x,a)\in V$ is associated with a ground truth
$y\in Y$. Let $I(x,a)\subseteq X\times A$ be the similar sub-population of
$(x,a)$, which is the set of the instances sharing the same non-sensitive
attributes values with $(x,a)$, i.e.,
$I(x,a)=\\{(x,a^{\prime})~{}|~{}a^{\prime}\in A\\}$. Obviously, $(x,a)\in
I(x,a)$. Let $f:X\times A\rightarrow{Y}$ denote a classifier learned from
training data, and $\hat{y}=f(x,a)$ the prediction result of classifier $f$
for input $(x,a)$. The high-level workflow of RobustFair is shown in Figure 1,
which consists of two phases: global generation (Algorithm 1) and local
generation (Algorithm 2).

Figure 1: The workflow of RobustFair

The global generation phase searches for diverse instances to cover a broad
range of an instance space. To scale the evaluation process, RobustFair starts
by calling getSeeds($V$), which cluster the given testing data $V$ and sample
randomly initial seeds uniformly from each cluster (Line 1). For each seed
$(v,y)$ with $v=(x,a)$, RobustFair selects the similar counterpart
$v^{\prime}\in I(x,a)$, for which the prediction $f(v^{\prime})$ is maximally
different from $f(v)$ in terms of the $L_{2}$-norm distance $\|\cdot\|_{2}$,
and then calculates the loss function gradients on seed $v$ and its similar
counterpart $v^{\prime}$ (Lines 6-8). These two gradients are examined to
determine fairness confusion perturbations towards false fair (FF), true
biased (TB), and false biased (FB) defects (Lines 9-11). For each directed
perturbation $dir$, RobustFair then perturbs the seed $v$ accordingly with a
fixed step size $p$, and approximates the ground truth $y_{p}$ for the
resulting instance $v_{p}$ with the Taylor expansion
groundTruth($v,y,g,f,v_{p}$) (Lines 13-14). The generated instance
$(v_{p},y_{p})$ with its ground truth approximation is kept in set $R_{T}$ for
the next iteration of global generation and in set $R_{G}$ for the subsequent
local generation and evaluation (Lines 15-16).

Algorithm 1 Global Generation

Input: model $f$, dataset $V$
Output: $R_{G}$

1: $R_{G}\leftarrow\emptyset$; $S\leftarrow$ getSeeds($V$);

2: for $i$ in range($iter$) do

3: $R_{T}\leftarrow\emptyset$;

4: for each $(v,y)\in S$ with $v=(x,a)$ do

5: $D\leftarrow\emptyset$;

6: $v^{\prime}\leftarrow\mathop{\arg\max}\limits_{v^{\prime}\in
I(x,a)}||f(v^{\prime})-f(v)||_{2}$;

7: $g\leftarrow\partial\mathrm{loss}(y,f(v))/\partial v$;

8: $g^{\prime}\leftarrow\partial\mathrm{loss}(y,f(v^{\prime}))/\partial
v^{\prime}$;

9: $D\leftarrow D\cup${dirFF($g,g^{\prime}$)};

10: $D\leftarrow D\cup${dirTB($g,g^{\prime}$)};

11: $D\leftarrow D\cup${dirFB($g,g^{\prime}$)};

12: for each $dir\in D$ do

13: $v_{p}\leftarrow v+p\cdot dir$;

14: $y_{p}\leftarrow$groundTruth($v,y,g,f,v_{p}$);

15: $R_{G}\leftarrow R_{G}\cup\\{(v_{p},y_{p})\\}$;

16: $R_{T}\leftarrow R_{T}\cup\\{(v_{p},y_{p})\\}$;

17: end for

18: end for

19: $S\leftarrow R_{T}$;

20: end for

21: return $R_{G}$

Algorithm 2 Local Generation

Input: model $f$
Output:$R_{L}$

1: $R_{L}$ $\leftarrow\emptyset$;

2: for each $(v,y)\in R_{G}$ with $v=(x,a)$ do

3: $D\leftarrow\emptyset$;

4: $v^{\prime}\leftarrow\mathop{\arg\max}\limits_{v^{\prime}\in
I(x,a)}||f(v^{\prime})-f(v)||_{2}$;

5: $g\leftarrow\partial\mathrm{loss}(y,f(v))/\partial v$;

6: $g^{\prime}\leftarrow\partial\mathrm{loss}(y,f(v^{\prime}))/\partial
v^{\prime}$;

7: $D\leftarrow D\cup${dirFF($g,g^{\prime}$)};

8: $D\leftarrow D\cup${dirTB($g,g^{\prime}$)};

9: $D\leftarrow D\cup${dirFB($g,g^{\prime}$)};

10: for each $dir\in D$ do

11: $min\leftarrow$ ascend($dir$);

12: for $i$ in range($iter$) do

13: $l_{dir}\leftarrow$ zerosLike($dir$);

14: $l_{dir}[min[i]]\leftarrow dir[min[i]]$;

15: $v_{p}\leftarrow v+p*l_{dir}$;

16: $y_{p}\leftarrow$groundTruth($v,y,g,f,v_{p}$);

17: $R_{L}\leftarrow R_{L}\cup(v_{p},y_{p})$;

18: end for

19: end for

20: end for

21: return $R_{L}$

In the local generation phase, RobustFair generates instances in the vicinity
of each instance $(v,y)\in R_{G}$ generated in the global generation phase.
RobustFair first determines the fairness confusion perturbations as in the
global generation phase (Lines 4-9). Then it sorts each perturbation direction
vector $dir$ in the ascending order of the absolute value of the derivative of
each non-sensitive attribute. At Line 11, $min[i]$ represents the attribute
index of the $i$-th least absolute derivative value. This allows RobustFair to
select the attributes with the less absolute derivative values to perturb,
which presumably results in the less deviations between the prediction
outputs. The minimal $iter$ absolute derivative values are selected one by one
for minimal perturbations, with the ground truths approximated through Taylor
expansions for the resulting instances (Lines 13-16). The generated instance
$(v_{p},y_{p})$ with its ground truth approximation is kept in set $R_{L}$ for
further evaluation (Line 17).

Finally, RobustFair utilizes the perturbed instances in $R_{G}\cup R_{L}$ to
evaluate the accurate fairness of classifier $f$, with respect to their
approximated ground truths.

###  3.1 Fairness Confusion Perturbations

RobustFair aims at the accurate fairness defects categorized by the fairness
confusion matrix under the alignment of accuracy and individual fairness.
Figure 2 illustrates the four categories of fairness confusion perturbations,
relating the predictions for a perturbed instance $(x,a)$ and its similar
counterpart $(x,a^{\prime})$ with $a^{\prime}\neq a$, and the instance’s
ground truth $y$. If a perturbation results in the predictions close to the
ground truth, i.e., $D(y,f(x,a^{\prime}))\leq Kd((x,a),(x,a^{\prime}))$, where
$D(\cdot,\cdot)$ and $d(\cdot,\cdot)$ are distance metrics, and constant
$K\geq 0$, the resulting predictions are _true fair_ [18]. If the prediction
$f(x,a)$ for instance $(x,a)$ is close to its ground truth $y$, but the
prediction $f(x,a^{\prime})$ for its similar counterpart $(x,a^{\prime})$ is
not, $(x,a)$ constitutes an adversarial instance with a _true biased_ (TB)
defect. On the contrary, if the prediction for $(x,a)$ is not close to its
ground truth, but the prediction for its similar counterpart is, $(x,a)$
constitutes an adversarial instance with a _false biased_ (FB) defect. If both
predictions are away from the ground truth, $(x,a)$ is an adversarial instance
with a _false fair_ (FF) defect.

Figure 2: Fairness Confusion Perturbations Algorithm 3 False Fair Direction
(dirFF)

Input: gradients $g$,$g^{\prime}$
Output: $dir$

1: $dir\leftarrow\text{zerosLike}(g)$

2: for each $g_{i}\in$ nonSensitive$(g)$ do

3: if sign($g_{i}$)==sign($g^{\prime}_{i}$) then

4: $dir[i]\leftarrow|g_{i}|$

5: end if

6: end for

7: return $dir$

To identify these adversarial instances, we leverage the loss function
gradients $g$ and $g^{\prime}$ on an instance and its similar counterpart, as
an intuitive and effective measure to direct perturbations on specific non-
sensitive attributes. Given an instance, its similar counterpart that is
maximally different from the instance in their predictions, is involved for
gradient calculation such that fairness confusion perturbations can be
constructed from the perspective of the most diverse counterpart of the
instance, promoting greater variety and inclusiveness in the RobustFair
evaluation process. We propose Algorithms 3, 4, and 5 that are devised for FF,
TB, and FB perturbations, respectively. In these algorithms, nonSensitive($g$)
returns the gradient component $g_{i}$ on each non-sensitive attribute, and
$g_{i}^{\prime}$ is the corresponding component of gradient $g^{\prime}$. Let
$|g_{i}|$ denote the absolute value of $g_{i}$.

Algorithm 4 True Biased Direction (dirTB)

Input: gradients $g$,$g^{\prime}$
Output: $dir$

1: $dir\leftarrow\text{zerosLike}(g)$;

2: for each $g_{i}\in$ nonSensitive$(g)$ do

3: if sign($g_{i}$) $\neq$ sign($g^{\prime}_{i}$) then

4: if sign($g_{i})\neq 0$ then

5: $dir[i]\leftarrow-|g_{i}|$;

6: else

7: $dir[i]\leftarrow|g^{\prime}_{i}|$;

8: end if

9: end if

10: end for

11: return $dir$

Algorithm 5 False Biased Direction (dirFB)

Input: gradients $g$,$g^{\prime}$
Output: $dir$

1: $dir\leftarrow\text{zerosLike}(g)$;

2: for each $g_{i}\in$ nonSensitive$(g)$ do

3: if sign($g_{i}$) $\neq$ sign($g^{\prime}_{i}$) then

4: if sign($g_{i})\neq 0$ then

5: $dir[i]\leftarrow|g_{i}|$

6: else

7: $dir[i]\leftarrow-|g^{\prime}_{i}|$

8: end if

9: end if

10: end for

11: return $dir$

Algorithm 3 selects non-sensitive attributes with the same gradient signs to
induce false fair perturbations (Line 3). Perturbing these attributes leads to
consistent changes in the losses for both the instance and its similar
counterpart. If sign($g_{i}$) is positive, a perturbation in the direction of
$g_{i}$ increases the losses, causing the predictions to deviate further from
the ground truth; otherwise, if sign($g_{i}$) is negative, a perturbation in
the opposite direction of $g_{i}$ moves the predictions away from the ground
truth (Line 4).

In contrast, Algorithms 4 and 5 select non-sensitive attributes with different
gradient signs to induce true biased and false biased perturbations,
respectively. Perturbing these attributes leads to opposite changes in the
losses for the instance and its similar counterpart. In Algorithm 4 for a true
biased perturbation, the prediction for the instance should get close to the
ground truth, while the one for its similar counterpart should deviate from
the ground truth. Thus, to enlarge the difference between the predictions, if
sign($g_{i}$) is not zero, a perturbation in the direction of $-|g_{i}|$ is
determined to maneuver the prediction for the instance towards the ground
truth (Line 5); otherwise, a perturbation in the direction of
$|g_{i}^{\prime}|$ is determined to maneuver the prediction for the similar
counterpart away from the ground truth (Line 7). The opposite strategy applies
for a false biased perturbation, as shown in Algorithm 5.

###  3.2 Ground Truth Approximation

Accurate fairness evaluation and fair confusion perturbations both rely on the
ground truths of the generated instances, which however are generally not
available. Taylor expansions [23] provide a powerful means of approximating
the behavior of a function near a given point by using its value and
derivatives. RobustFair approximates the loss of the generated instance
$v_{p}$ through the first-order Taylor expansion, as depicted below:

$\mathrm{loss}(y_{p},f(v_{p}))=\mathrm{loss}(y,f(v))+g(v_{p}-v)$ (1)

where $v_{p}$ is generated by perturbing instance $v$, $y$ is the ground truth
of $v$, $\mathrm{loss}(\cdot,\cdot)$ is the loss function, and $g$ is the loss
function gradient on $v$. Then, the approximated ground truth $y_{p}$ can be
derived from Equation (1). For the Mean Square Error ($\mathrm{MSE}$) loss
function, groundTruth($v,y,g,f,v_{p}$) can be implemented as below.

$\text{groundTruth}(v,y,g,f,v_{p})=\left\\{\begin{matrix}y^{-}&|y^{-}-y|<|y^{+}-y|\\\
y^{+}&\text{otherwise}\end{matrix}\right.$ (2)

where $y^{+}=f(v_{p})+\sqrt[2]{|\mathrm{MSE}(y,f(v))+g(v_{p}-v)|}$ and
$y^{-}=f(v_{p})-\sqrt[2]{|\mathrm{MSE}(y,f(v))+g(v_{p}-v)|}$. That is, the
approximated ground truth $y_{p}$ for $v_{p}$ is the one of $y^{-}$ and
$y^{+}$ that is closer to $y$.

##  4 Experimental Evaluation

We implement RobustFair in Python 3.8 with TensorFlow 2.4.1. The
implementation is evaluated on a Ubuntu 18.04.3 system, equipped with Intel
Xeon Gold 6154 @3.00GHz CPUs, GeForce RTX 2080 Ti GPUs, and 512GB memory. The
source code, experimental datasets, and models are publicly available at
https://github.com/Xuran-LI/RobustFairEvaluation.

###  4.1 Experiment Setup

Four popular fairness datasets Adult (Census Income) [10], German Credit [12],
Bank Marketing [9], and ProPublica Recidivism (COMPAS) [11] are considered for
experimental evaluation. The sizes, sensitive attributes, and the baseline
models (BL) trained with these datasets are reported in Table 2, where
$A$($k$) indicates that attribute $A$ has $k$ values, and FCNN($l$) denotes a
fully connected neural network classifier with $l$ layers. Specifically, we
consider all the sensitive attributes jointly, instead of separately, to
conduct complex case studies.

Table 2: Datasets and Models

Dataset | Size | Model | Sensitive Attributes
---|---|---|---
German Credit | 1000 |  FCNN(6) |  gender(2), age(51)
Bank Marking | 45211 |  FCNN(6) |  age(77)
Adult (Census Income) | 45222 |  FCNN(6) |  gender(2), age(71), race(5)
ProPublica Recidivism (COMPAS) | 6172 |  FCNN(6) |  gender(2), age(71), race(6)

The state-of-the-art robustness testing techniques, including Fast Gradient
Sign Method (FGSM) [13], Projected Gradient Descent (PGD) attack [15], Auto
Projected Gradient Descent (APGD) [16], and Auto Conjugate Gradient (ACG)
[17], and individual fairness testing techniques, including Adversarial
Discrimination Finder (ADF) [21] and Efficient Individual Discriminatory
Instances Generator (EIDIG) [22] are involved for comparison. We cluster each
testing dataset into four clusters using K-Means [24]. For ADF, EIDIG, and
RobustFair, 200 initial seeds are sampled from the clusters, and the numbers
of global and local iterations are all set to 5. For PGD, APGD, and ACG, 200
initial seeds are sampled but the number of iterations is set to 25. For FGSM,
$200\times 25$ are sampled and used directly. This is to ensure the equal
amount (200*25) of searches to be performed by each technique.

###  4.2 Results and Discussion

The objective of our experiments is to address the following three inquiries.

Q1: How efficient is RobustFair in accurate fairness evaluation?

Table 3: Statistics of Trustworthiness Evaluation

Dataset | Tech. | $N_{F}$ | $N_{B}$ | $N_{F|B}$ | $N_{TF}$ | $N_{TB}$ | $N_{FF}$ | $N_{FB}$ | SUM
---|---|---|---|---|---|---|---|---|---
German Credit | FGSM | 42.0 | 46.6 | 73.4 | 120.6 | 31.4 | 26.8 | 15.2 | 194.0
PGD | 152.4 | 70.3 | 193.9 | 138.9 | 41.5 | 123.6 | 28.8 | 332.8
APGD | 302.0 | 218.9 | 440.0 | 241.6 | 138.0 | 221.1 | 80.9 | 681.6
ACG | 270.9 | 204.4 | 403.5 | 297.5 | 132.6 | 199.1 | 71.8 | 701.0
ADF | 154.4 | 362.7 | 379.6 | 138.8 | 225.2 | 16.9 | 137.5 | 518.4
EIDIG | 110.9 | 194.5 | 234.2 | 416.7 | 123.3 | 39.7 | 71.2 | 650.9
RobustFair | 2481.9 | 4758.5 | 5222.7 | 80.3 | 2740.8 | 464.2 | 2017.7 | 5303.0
Adult | FGSM | 910.0 | 1213.6 | 1567.5 | 2582.3 | 657.5 | 353.9 | 556.1 | 4149.8
PGD | 168.1 | 213.3 | 277.3 | 233.0 | 109.2 | 64.0 | 104.1 | 510.3
APGD | 1023.4 | 1307.5 | 1667.5 | 932.6 | 644.1 | 360.0 | 663.4 | 2600.1
ACG | 861.2 | 1110.8 | 1468.9 | 846.0 | 607.7 | 358.1 | 503.1 | 2314.9
ADF | 329.6 | 1288.4 | 1300.2 | 287.2 | 970.6 | 11.8 | 317.8 | 1587.4
EIDIG | 71.9 | 367.2 | 377.6 | 684.2 | 305.7 | 10.4 | 61.5 | 1061.8
RobustFair | 2093.3 | 3623.5 | 4577.7 | 348.0 | 2484.4 | 954.2 | 1139.1 | 4925.7
Bank Marketing | FGSM | 523.5 | 826.0 | 1143.9 | 3033.9 | 620.4 | 317.9 | 205.6 | 4177.8
PGD | 101.2 | 158.2 | 211.2 | 495.2 | 110.0 | 53.0 | 48.2 | 706.4
APGD | 620.1 | 953.2 | 1325.6 | 3239.4 | 705.5 | 372.4 | 247.7 | 4565.0
ACG | 546.7 | 880.7 | 1215.0 | 3098.7 | 668.3 | 334.3 | 212.4 | 4313.7
ADF | 191.8 | 1382.2 | 1405.4 | 714.4 | 1213.6 | 23.2 | 168.6 | 2119.8
EIDIG | 48.5 | 261.9 | 282.4 | 1229.3 | 233.9 | 20.5 | 28.0 | 1511.7
RobustFair | 950.5 | 1742.9 | 2346.1 | 197.6 | 1395.6 | 603.2 | 347.3 | 2543.7
ProPublica Recidivism | FGSM | 26.6 | 200.9 | 221.6 | 990.5 | 195.0 | 20.7 | 5.9 | 1212.1
PGD | 10.0 | 70.2 | 77.9 | 241.1 | 67.9 | 7.7 | 2.3 | 319.0
APGD | 107.6 | 682.4 | 767.7 | 1306.1 | 660.1 | 85.3 | 22.3 | 2073.8
ACG | 73.7 | 642.4 | 697.2 | 1225.1 | 623.5 | 54.8 | 18.9 | 1922.3
ADF | 0.5 | 539.5 | 539.5 | 552.4 | 539.0 | 0.0 | 0.5 | 1091.9
EIDIG | 1.9 | 218.1 | 218.1 | 844.2 | 216.2 | 0.0 | 1.9 | 1062.3
RobustFair | 357.7 | 2378.1 | 2634.4 | 586.7 | 2276.7 | 256.3 | 101.4 | 3221.1

Table 3 presents the average statistics of ten runs of each technique testing
the baseline model across the German Credit, Adult, Bank Marketing, and
ProPublica Recidivism datasets. Columns $N_{F}$, $N_{B}$ show the numbers of
false, biased adversarial instances detected, respectively. The size of the
union of these false and biased adversarial instances is reported in Column
$N_{F|B}$ for each technique. Columns $N_{TF}$, $N_{TB}$, $N_{FF}$, and
$N_{FB}$ show the numbers of true fair instances, true biased, false fair, and
false biased adversarial instances, respectively. Note that by definitions,
$N_{FF}+N_{FB}=N_{F}$ for robustness evaluation, and $N_{TB}+N_{FB}=N_{B}$ for
individual fairness evaluation.

It can be seen that compared with the best performance of these state-of-the-
art evaluation techniques, RobustFair identifies significantly 1.17 to 11.87
times false or biased adversarial instances, specifically 1.15 to 12.17 times
true biased instances, 1.62 to 3.00 times false fair instances, and 1.40 to
14.67 times false biased instances. This results in detecting 1.83 to 13.12
times biased, and 1.53 to 8.22 times false adversarial instances. Thus,
RobustFair effectively evaluates accurate fairness through fairness confusion
directed adversarial tests, which also boost the performances of robustness
and individual fairness evaluations.

Answer to Q1: RobustFair can detect significantly more instances in each
adversarial category and exhibit the highest effectiveness in accurate
fairness evaluation, which is much greater than the sum of robustness and
individual fairness evaluations.

Q2: How effective is RobustFair evaluation for trustworthiness improvement?

Next, we augment training data by sampling 10% of the detected adversarial
instances to retrain models, and evaluate the performance of each retrained
model on the original testing data in terms of accuracy (ACC), individual
fairness (IF, the proportion of individually fair instances) and $R_{TF}$,
$R_{TB}$, $R_{FF}$, and $R_{FB}$ (the proportions of true fair, true biased,
false fair, and false biased instances, respectively). Recall that RobustFair
and these state-of-the-art evaluation techniques estimate the ground truths of
the generated instances in distinct ways: FGSM, PGD, APGD, and ACG reuse the
original ground truths under the classical robustness requirement; ADF and
EIDIG determine the ground truths by means of ensemble models; while
RobustFair applies Taylor expansions for ground truth approximation. Table 4
presents the average performance statistics of the baseline (BL) and retrained
models across the German Credit, Adult, Bank Marketing, and ProPublica
Recidivism datasets. Re-$M$ represents the model retrained with the
adversarial instances detected by techinque $M$.

Table 4: Statistics of Retraining

Dataset | Metric | BL | Re-FGSM | Re-PGD | Re-APGD | Re-ACG | Re-ADF | Re-EIDIG | Re-RobustFair
---|---|---|---|---|---|---|---|---|---
German Credit | ACC | 0.81 | 0.80 | 0.80 | 0.81 | 0.82 | 0.80 | 0.80 | 0.84
IF | 0.74 | 0.85 | 0.84 | 0.83 | 0.86 | 0.86 | 0.86 | 0.88
$R_{TF}$ | 0.63 | 0.70 | 0.70 | 0.71 | 0.73 | 0.71 | 0.71 | 0.76
$R_{TB}$ | 0.18 | 0.09 | 0.10 | 0.11 | 0.09 | 0.09 | 0.10 | 0.09
$R_{FF}$ | 0.11 | 0.15 | 0.14 | 0.12 | 0.13 | 0.15 | 0.15 | 0.13
$R_{FB}$ | 0.08 | 0.05 | 0.06 | 0.06 | 0.05 | 0.05 | 0.05 | 0.03
Adult | ACC | 0.85 | 0.84 | 0.84 | 0.84 | 0.84 | 0.84 | 0.85 | 0.85
IF | 0.73 | 0.72 | 0.73 | 0.72 | 0.70 | 0.71 | 0.73 | 0.75
$R_{TF}$ | 0.66 | 0.66 | 0.66 | 0.65 | 0.64 | 0.65 | 0.66 | 0.68
$R_{TB}$ | 0.19 | 0.19 | 0.18 | 0.20 | 0.21 | 0.20 | 0.19 | 0.17
$R_{FF}$ | 0.07 | 0.07 | 0.07 | 0.07 | 0.06 | 0.07 | 0.07 | 0.08
$R_{FB}$ | 0.09 | 0.09 | 0.08 | 0.09 | 0.10 | 0.09 | 0.09 | 0.08
Bank Marketing | ACC | 0.90 | 0.90 | 0.90 | 0.90 | 0.90 | 0.90 | 0.90 | 0.90
IF | 0.85 | 0.84 | 0.85 | 0.81 | 0.85 | 0.82 | 0.83 | 0.91
$R_{TF}$ | 0.79 | 0.78 | 0.79 | 0.76 | 0.80 | 0.76 | 0.77 | 0.84
$R_{TB}$ | 0.11 | 0.12 | 0.11 | 0.14 | 0.11 | 0.14 | 0.13 | 0.06
$R_{FF}$ | 0.06 | 0.06 | 0.06 | 0.05 | 0.06 | 0.05 | 0.05 | 0.07
$R_{FB}$ | 0.04 | 0.04 | 0.04 | 0.05 | 0.04 | 0.05 | 0.05 | 0.03
ProPublica Recidivism | ACC | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98 | 0.98
IF | 0.85 | 0.88 | 0.95 | 0.87 | 0.92 | 0.87 | 0.94 | 0.99
$R_{TF}$ | 0.84 | 0.87 | 0.93 | 0.86 | 0.91 | 0.85 | 0.93 | 0.97
$R_{TB}$ | 0.14 | 0.11 | 0.05 | 0.12 | 0.07 | 0.12 | 0.05 | 0.01
$R_{FF}$ | 0.02 | 0.02 | 0.02 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02
$R_{FB}$ | 0.00 | 0.01 | 0.00 | 0.01 | 0.00 | 0.01 | 0.00 | 0.00

It can be seen that the Re-RobustFair models exhibit the highest advancements
in accurate fairness (3%-21%). Notably, these models achieve significant
reductions of 11%-93% in $R_{TB}$ and 0-63% in $R_{FB}$, leading to a 3%-19%
improvement in individual fairness without compromising accuracy, across all
four datasets, even promoting 4% in accuracy on the German Credit dataset.
Therefore, the RobustFair evaluation plays a pivotal role in enhancing
trustworthiness by effectively promoting accurate fairness and individual
fairness while maintaining accuracy. In contrast, the other retrained models
do not consistently improve individual fairness or accurate fairness. Although
the models retrained on the German Credit and ProPublica Recidivism datasets
succeed in improving individual fairness and accurate fairness without
sacrificing accuracy, the majority of retrained models on the other two
datasets do not enhance individual fairness or accurate fairness. In fact,
some models even experience declines in accurate fairness, individual
fairness, or accuracy.

Answer to Q2: RobustFair evaluation demonstrates its significant contribution
to trustworthiness improvement in terms of accurate fairness, individual
fairness, and even accuracy, through retraining with the adversarial instances
detected.

Q3: What are the impacts of parameter settings on RobustFair evaluation.

Figure 3: Impacts of Parameter Settings

Herein, we discuss the impacts of different parameter settings on RobustFair
evaluation, including the number of initial seeds ranging in {100, 200, 300,
400, and 500}, the numbers of global and local iterations ranging in {5, 10,
15, 20, 25, 30}. The experimental results are presented in Figure 3 on the
four fairness datasets, by varying one parameter and keeping the others the
same as before.

It can be seen that as the number of local iterations increases, the numbers
of false, biased adversarial instances detected, the size of their union, and
the total number of all the generated instances first escalate rapidly and
then stabilize with minor fluctuations. However, with the increasing number of
initial seeds or global iterations, those numbers of instances continue to
increase steadily. This can be attributed to the two-phase generation scheme,
such that the global generation starting with the initial seeds intends to
enhance the diversity of the gradient search, while the local generation
intends to constrain the gradient search in limited neighbourhoods of
instances explored globally.

Answer to Q3: Increasing the number of global iterations or initial seeds can
enhance the coverage and diversity of the RobustFair evaluation process.
However, the advantages of the local gradient search may be offset by an
excessively large number of local iterations.

##  5 Conclusion and Future Work

In this paper, we present RobustFair, an innovative adversarial evaluation
approach for accurate fairness of deep neural networks. It applies the
fairness confusion matrix to guide the distinct gradient search directions for
false fair, true biased, and false biased adversarial perturbations. It
employs Taylor expansions to approximate the ground truths of adversarial
instances for gradient calculation and accurate fairness testing. Comparative
case studies on fairness benchmark datasets demonstrate the synergistic
effectiveness of RobustFair and accurate fairness evaluation, particularly in
detecting elusive robustness defects entangled with spurious fairness.
RobustFair outperforms state-of-the-art robustness and individual fairness
evaluation approaches in both the quantity and quality of diverse adversarial
instances detected, which can be leveraged to enhance the accurate fairness
and individual fairness of DNNs without compromising overall accuracy.

RobustFair is a white-box testing approach that requires to access the
gradients of DNNs. As part of future work, it is interesting to extend
accurate fairness evaluation to black-box scenarios. Furthermore, RobustFair
can be potentially expanded with other trustworthiness evaluation approaches
for various datasets, models, and tasks in practical applications.

## References

  * [1] Xuran Li, Peng Wu, and Jing Su.  Accurate fairness: Improving individual fairness without trading accuracy.  Thirty-Seven AAAI Conference on Artificial Intelligence, AAAI 2023, Washington, DC, USA, February 7-14, page abs/2205.08704, 2023.
  * [2] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [3] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel.  Fairness through awareness.  In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012, pages 214–226. ACM, 2012.
  * [4] Zhen Zhang, Peng Wu, Yuhang Chen, and Jing Su.  Out-of-distribution detection through relative activation-deactivation abstractions.  In 32nd IEEE International Symposium on Software Reliability Engineering, ISSRE 2021, Wuhan, China, October 25-28, 2021, pages 150–161. IEEE, 2021.
  * [5] Alex Gittens, Bülent Yener, and Moti Yung.  An adversarial perspective on accuracy, robustness, fairness, and privacy: Multilateral-tradeoffs in trustworthy ML.  IEEE Access, pages 120850–120865, 2022.
  * [6] Jae-Gil Lee, Yuji Roh, Hwanjun Song, and Steven Euijong Whang.  Machine learning robustness, fairness, and their convergence.  In KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pages 4046–4047. ACM, 2021.
  * [7] Jing Su, Zhen Zhang, Peng Wu, Xuran Li, and Jian Zhang.  Adversarial input detection based on critical transformation robustness.  In IEEE 33rd International Symposium on Software Reliability Engineering, ISSRE 2022, Charlotte, NC, USA, October 31 - Nov. 3, 2022, pages 390–401. IEEE, 2022.
  * [8] Peixin Zhang, Jingyi Wang, Jun Sun, Guoliang Dong, Xinyu Wang, Xingen Wang, Jin Song Dong, and Ting Dai.  White-box fairness testing through adversarial sampling.  In ICSE ’20: 42nd International Conference on Software Engineering, Seoul, South Korea, 27 June - 19 July, 2020, pages 949–960. ACM, 2020.
  * [9] S. Moro, P. Rita, and P. Cortez.  Bank Marketing.  UCI Machine Learning Repository, 2012.
  * [10] Adult.  UCI Machine Learning Repository, 1996.  DOI: 10.24432/C5XW20.
  * [11] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner.  Machine bias.  Ethics of Data and Analytics: Concepts and Cases, page 254, 2022\.
  * [12] Hans Hofmann.  Statlog (German Credit Data).  UCI Machine Learning Repository, 1994.  DOI: 10.24432/C5NC77.
  * [13] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [14] Muhammad Awais and Sung-Ho Bae.  A survey on efficient methods for adversarial robustness.  IEEE Access, 10:118815–118830, 2022.
  * [15] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and drian Vladu.  Towards deep learning models resistant to adversarial attacks.  In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
  * [16] Francesco Croce and Matthias Hein.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, 2020.
  * [17] Keiichiro Yamamura, Haruki Sato, Nariaki Tateiwa, Nozomi Hata, Toru Mitsutake, Issa Oe, Hiroki Ishikura, and Katsuki Fujisawa.  Diversified adversarial attacks based on conjugate gradient method.  In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, Proceedings of Machine Learning Research. PMLR, 2022.
  * [18] Carlos Pinzón, Catuscia Palamidessi, Pablo Piantanida, and Frank Valencia.  On the impossibility of non-trivial accuracy in presence of fairness constraints.  Proceedings of the AAAI Conference on Artificial Intelligence, pages 7993–8000, 2022.
  * [19] Nian Si, Karthyek Murthy, Jose Blanchet, and Viet Anh Nguyen.  Testing group fairness via optimal transport projections.  In Proceedings of the 38th International Conference on Machine Learning, pages 9649–9659, 2021.
  * [20] Furkan Gursoy and Ioannis A. Kakadiaris.  Error parity fairness: Testing for group fairness in regression tasks, 2022.
  * [21] Peixin Zhang, Jingyi Wang, Jun Sun, Xinyu Wang, Guoliang Dong, Xingen Wang, Ting Dai, and Jin Song Dong.  Automatic fairness testing of neural classifiers through adversarial sampling.  IEEE Transactions on Software Engineering, 48(9):3593–3612, 2022\.
  * [22] Lingfeng Zhang, Yueling Zhang, and Min Zhang.  Efficient white-box fairness testing through gradient search.  In ISSTA ’21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, Denmark, July 11-17, 2021, pages 103–114. ACM, 2021.
  * [23] Tom Apostol.  Mathematical Analysis.  Addison-Wesley, 1974.
  * [24] Stuart P. Lloyd.  Least squares quantization in pcm.  IEEE Transactions on Information Theory, 28(2):129–137, 1982.