# Distraction is All You Need for Fairness

Mehdi Yazdani-Jahromi, Amirarsalan Rajabi, Ali Khodabandeh Yalabadi, Aida
Tayebi, Ozlem Ozmen Garibay

###### Abstract

Bias in training datasets must be managed for various groups in classification
tasks to ensure parity or equal treatment. With the recent growth in
artificial intelligence models and their expanding role in automated decision-
making, it is vital to ensure that these models are not biased. There is an
abundance of evidence suggesting that these models could contain or even
amplify the bias present in the data on which they are trained, inherent to
their objective function and learning algorithms; Existing methods for
mitigating bias result in information loss and do not provide a suitable
balance between accuracy and fairness or do not ensure limiting the biases in
training. To this end, we propose a powerful strategy for training deep
learning models called the Distraction module, which can be effective in
controlling bias from affecting the classification results. This method can be
utilized with different data types (e.g., tabular, images, graphs). We
demonstrate the potency of the proposed method by testing it on _UCI Adult_
and _Heritage Health_ datasets (tabular), _POKEC-Z_ , _POKEC-N_ and _NBA_
datasets (graph), and _CelebA_ dataset (vision). Considering state-of-the-art
methods proposed in the fairness literature for each dataset, we exhibit that
our model is superior to these proposed methods in minimizing bias and
maintaining accuracy.

###### Index Terms:

Deep Learning, Fairness, Neural Networks, Adversarial Training, Game Theory

##  I Introduction

Artificial intelligence and machine learning models in real-world applications
have grown in past decades and led to automated decision-making in different
domains such as hiring pipelines, face recognition, financial services, the
healthcare system, and criminal justice. Algorithmic decision-making may cause
an algorithmic bias toward the central population subgroup and discrimination
and unfairness toward the minority. In recent years, fairness in artificial
intelligence has increased ethical concerns and received attention from
interdisciplinary research communities [1]. Several definitions of fairness
have been put forth as potential solutions to the problem of unwanted bias in
machine learning techniques. In most cases, the definitions may be separated
into two categories: individual fairness and collective fairness. A system
that is individual will treat users that are similar to each other in the same
manner, where the similarities between people may be determined by past
information [2, 3]. Group fairness metrics are measurements of the statistical
equality between different subgroups that are characterized by sensitive
characteristics such as race, or gender [4, 5, 6].
In this paper, our focus is on group fairness, and from this point on, we
refer to group fairness as fairness.
We introduced a novel in-process bias mitigation method that does not need
adversarial example generation to train a fair classifier and can be trained
on available datasets without alteration. we proposed a departure from the
typical adversarial paradigm. Rather than using a separate discriminator
network or injecting adversarial noise, we introduce a new training approach
that embeds a specialized module (Distraction Module) within the main neural
network. This embedded module independently manages a subset of weights within
the main network and is explicitly trained to optimize the network towards
specific criteria or alternative objective functions.
Our primary contributions are:

  * A novel training method that incorporates a specialized module within the main network to optimize towards specific criteria or alternative objective functions.

  * Empirical validation of our approach on multiple benchmarks with various data types, demonstrating improvements in network fairness.

  * New adversarial training procedure which significantly improves the state-of-the-art in both accuracy and fairness metrics.

##  II Related works

Fairness in machine learning has garnered significant attention, with methods
primarily spanning three categories:
Pre-process approaches adjust data prior to model training to achieve fair
outcomes. This includes changing or reweighing labels [7, 8] and modifying
feature distributions to make differentiation between privileged and
unprivileged groups [9, 10]. Recently, a GAN was introduced to generate
unbiased tabular datasets, focusing on both accuracy and fairness [11].
In-process approaches alter the algorithm during training. Some add
regularization terms to the objective function, balancing fairness and
accuracy. For example, mutual information between protected attributes and
classifier predictions was penalized [12], while others added constraints to
satisfy equalized odds proxies [13, 14].
Post-process approaches modify outcomes post-training. Strategies range from
flipping certain outcomes [6] to using different thresholds for privileged and
unprivileged groups, balancing fairness and accuracy [15, 16].
Different data types, namely tabular, graph, and images, require specialized
fairness approaches:
Tabular: Efforts in tabular data focus on mitigating bias. Approaches include
regularizing covariance between predictions and sensitive variables [17],
standardizing decision bounds [13], and restricting adversaries from inferring
sensitive characteristics [18]. Game-theoretic methods have been employed,
although scaling them remains challenging [19]. Other noteworthy techniques
leverage attention-based approaches, mutual information, and information-
theoretic methods [20, 21, 22, 23, 24].
Graph: Graphs can amplify biases, particularly in networks where nodes with
similar sensitive features are more likely to connect [25, 26]. This can lead
to severe decision-making biases in Graph Neural Networks (GNNs) [27]. Most
fair models were designed for i.i.d data and often don’t cater to graph data.
However, there have been pioneering efforts in learning fair node
representations from graphs [28, 26, 27].
Vision: Biases in vision models can manifest in various ways, such as gender
biases in action recognition or disparities in face recognition across racial
and gender categories [29, 29]. Solutions span from altering GAN utility
functions for fair image datasets [30, 31] to adversarial game formulations
[32] and methodologies for balanced data generation [33]. Apart from GANs,
techniques like U-Nets, deep information maximization adaptation networks,
reinforcement learning, and adversarial learning have been proposed to tackle
bias in image datasets [34, 35, 36, 19].

##  III Methodology

###  III-A Problem Definition

Our focus in this study is binary classification tasks. We posit that the
techniques proposed here can be extended to multi-class datasets without
constraint.
Let’s consider a dataset denoted as $E=\\{x^{(i)},a^{(i)},y^{(i)}\\}$, where
$x^{(i)}$, $y^{(i)}$, and $a^{(i)}$ are independently and identically
distributed samples drawn from the data distribution $P(x,y,a)$.

Here:

  * •

$x$ represents the features of the dataset.

  * •

$y\in\\{0,1\\}$ indicates the label.

  * •

$a$, which takes on discrete finite values, is the protected attribute of our
data.

###  III-B Model Architecture

Proposed method employs two distinct sets of weights for its neural network
classifier:

  * •

The primary set, aimed at the classification task, optimizes for accuracy
($\theta_{c}$).

  * •

The second set, associated with the ”Distraction module”, is designed to
enhance fairness within the model ($\theta_{d}$).

A graphical representation of this architecture can be found in Figure 1. This
model can be configured using a variety of neural network architectures such
as fully connected layers, graph convolution network, convolution neural
network, and etc.

Figure 1: Depiction of Distraction module in MLP model: The red weights are
controlled by the Distraction module and are optimized with Eq. 3 objective
function, based on Demographic parity (DP) loss. The blue weights have
different optimizer and Binary Cross Entropy (BCE) loss function. They are
controlled by the main classifier.

###  III-C Formulation

The goal of fairness in classification is to ensure that predictions remain
consistent regardless of the protected attribute. This concept can be formally
captured by:

$\begin{split}&p(C(x,D(x;\theta_{d});\theta_{c})=s|a,y)=\\\
&p(C(x,D(x;\theta_{d});\theta_{c})=s|a^{\prime},y)\end{split}$ (1)

as described in [37].

The methodology incorporates a game-theoretic approach. We treat the
Distraction module and the encompassing network as two distinct players in a
maximin game. Our objective is to train both sets of weights concurrently,
aiming to create a classifier that balances fairness with accuracy. The
weights of the Distraction module are entirely isolated from the leading
network. The Distraction module tries to make the classifier function results
as fair as possible, while the whole network tries to make the classifier
function results as accurate as possible. The first player is the Distraction
module, and the second is the whole network containing the Distraction module.
We train two sets of weights simultaneously to achieve a fair and accurate
classifier. This game is established with two utility functions, one for each
player. We denote the Distraction module as $D(x)$ and the whole network as
$C(x,D(x))$ throughout this paper. To put it differently, D and C play the
two-player maximin game with a utility function of $U_{1}(D,C)$ (Eq. 2) for
player C and $U_{2}(D,C)$ (Eq. 3) for player D.

$\min_{\theta_{c}}U_{1}(C,D)=\mathbb{E}_{x\sim X}\mathbb{E}_{y\sim
Y}\Big{[}-\log p_{\theta_{d},\theta_{c}}(y|x))\Big{]}$ (2)

$\begin{split}\max_{\theta_{d}}U_{2}(C,D)=&-\mathbb{E}_{s\sim
C(x,D(x;\theta_{d});\theta_{c})}\Big{[}\\\ &\mathbb{E}_{a\sim A}\big{[}-\log
p_{\theta_{d},\theta_{c}}(a|s)\big{]}\Big{]}\end{split}$ (3)

###  III-D Trade-off Between Fairness and Accuracy

A common challenge is that a classifier might not be both completely fair and
optimal. This is often because the protected attributes, along with their
proxies, can influence classifier decisions substantially.

To manage this, we introduce a parameter, $\eta$, which helps strike a balance
between model accuracy and fairness. The role of $\eta$ is central to the
Demographic parity loss (Eq. 4) utilized by the Distraction module. A smaller
$\eta$ leans towards accuracy, while a larger value promotes fairness. By
tuning $\eta$, we can generate a series of Pareto solutions for our multi-
objective optimization task.

$\eta\sum_{i=1}^{m}-\log
p_{\theta_{d},\theta_{c}}(a^{(i)}|C(x^{(i)},D(x^{(i)})))$ (4)

###  III-E Training Procedure

We employ a mini-batch stochastic gradient descent technique to train our
network. Algorithm 1 offers a detailed training procedure used for training
networks containing Distraction module.

During each iteration:

  * •

First, the network’s fairness loss is computed, followed by an update to the
Distraction module weights.

  * •

Then, the network undergoes another iteration with frozen weights of the
Distraction module where the classification loss is computed, and back-
propagation is employed for updating the weight of the remaining network.

Algorithm 1 Minibatch stochastic gradient descent for adversarial training of
a network with distraction module

Input: data $(X,A,Y)$, A is a set of protected attributes and Y is the label,
Batch Size $m$, $C$-Learning rate $lr_{1}$, $D$-Learning rate $lr_{2}$, $\eta$

for number of iterations in training do

sample minibatch of size $m$ samples $\\{x^{(1)},x^{(2)},...,x^{(m)}\\}$ from
data $P(x)$

updates the Distraction module by ascending the stochastic gradient with
learning rate $lr_{2}$:
$\displaystyle\nabla_{\theta_{d}}\eta\sum_{i=1}^{m}-\log
p_{\theta_{d},\theta_{c}}(a^{(i)}|C(x^{(i)},D(x^{(i)})))$

updates the classifier network by descending the stochastic gradient with
learning rate $lr_{1}$:  $\displaystyle\nabla_{\theta_{c}}\sum_{i=1}^{m}-\log
p_{\theta_{d},\theta_{c}}(y^{(i)}|x^{(i)})$

end for

In a practical perspective, this method needs two optimizers which can be any
standard gradient-based method. We used the Adam optimizer for both of the
functions in our experiments.

##  IV Experiments

TABLE I: Area over the curve of statistical demographic parity and accuracy (Higher is better). The proposed model (Distraction) significantly outperforms other benchmark models in this quantitative metric. Method | UCI Adult | Heritage Health
---|---|---
Distraction (Ours) | 0.411 | 0.503
FCRL (AAAI 2021) | 0.253 | 0.285
Attention | 0.213 | 0.139
CIVB (NeurIPS 2018) | 0.163 | 0.176
MIFR (AIStats 2019) | 0.221 | 0.166
MaxEnt-ARL (CVPR 2019) | 0.133 | 0
Adv Forgetting (AAAI 2020) | 0.077 | 0.172

This section compares our method to other benchmark methods in the literature.
The experiment section consists of three parts. First, we experiment with
tabular datasets. We compare the classification accuracy and statistical
parity of deep learning methods in the benchmark datasets. In the second
section, we use graph data for node classification tasks. We evaluate our
model on vision datasets in the third and final section.
The Distraction module used on all the datasets consists of only linear
layers, and the Distraction module is positioned one layer before the
classification layer. This choice was due to experiments conducted on the
vision and tabular datasets. The ablation study, and the additional results
can be found in the section IV-F. Additionally, we observed that the loss for
both the fairness metric and the accuracy is very volatile in training.
However, given enough steps, it always converges to a single point which is a
Pareto answer for this multi-objective optimization. We decided not to include
loss graphs in the paper due to the volatility of the losses.

###  IV-A Evaluation Metrics

We employ four evaluation metrics to compare the performance of our model with
baseline models. These metrics are as follows:

  1. 1.

Average Precision (AP):

     * •

Definition: The average precision is a measure that computes the average while
combining recall and precision at each threshold. It provides an aggregate
assessment of the classifier’s performance over all levels of precision-
recall.

     * •

Objective: Higher AP values are preferred, indicating better accuracy of the
classifiers.

  2. 2.

Accuracy:

     * •

Usage: Consistency with literature suggests using accuracy for tabular and
graph datasets.

     * •

Objective: A higher accuracy indicates a better performing model.

  3. 3.

Demographic Parity (DP):

     * •

Definition: DP is a widely-used fairness metric that captures the difference
in the probability of receiving a favorable decision between different
protected groups. It is calculated as the absolute difference
$(|P(Y=1|S=0)-P(Y=1|S=1)|)$. For more than two groups, DP can be computed
using
$\Delta_{DP}(a,\hat{y})=\max_{a_{i},a_{j}}|P(\hat{y}=1|a=a_{i})-P(\hat{y}=1|a=a_{j}))|$
[38, 24].

     * •

Objective: A smaller DP indicates a more fair classification, as it reduces
disparity between protected groups.

  4. 4.

Difference in Equality of Opportunity ($\Delta$EO):

     * •

Definition: Following [39] and [33], we use $\Delta$EO as a fairness metric.
It is defined as the absolute difference between the true positive rates for
different group expressions $(|TPR(S=0)-TPR(S=1)|)$.

     * •

Objective: Lower values of $\Delta$EO are preferred, indicating fairer
categorization between group expressions.

In the experiments, we used demographic parity as the fairness criterion
during training. This choice indicates that the model is optimized for
demographic parity and the demographic parity is the main metric that our
model is providing. The better fairness metrics provided are byproduct of
optimizing the model with the demographic parity objective.

###  IV-B Implementation details

The hyperparameters used in training the models on each tabular, graph, and
vision datasets can be found in the tables II, III, and IV respectively.
The training was performed on a single NVIDIA GeForce RTX 3090.

TABLE II: Summary of Parameter Setting for the Distraction on tabular datasets Hyperparameters | UCI Adult | Health Heritage
---|---|---
FC layers before the Distraction module | 2 | 2
FC layers of the Distraction module | 1 | 3
FC layers after the Distraction module | 1 | 1
Epoch | 50 | 50
Batch size | 100 | 100
Dropout | 0 | 0
Network optimizer | Adam | Adam
Distraction module optimizer | Adam | Adam
Network learning rate | 1e-3 | 1e-3
Distraction module learning rate | 1e-5 | 1e-5
$\eta$ | 100 | 100
TABLE III: Summary of Parameter Setting for the Distraction on graph datasets Hyperparameters | POKEC-Z | POKEC-N | NBA
---|---|---|---
GCN layer before the Distraction module | 2 | 2 | 2
Distraction module FC layers | 1 | 1 | 1
FC layers after the Distraction module | 1 | 1 | 1
Epoch | 5000 | 1000 | 1000
Batch size | 1 | 1 | 1
Dropout | 0 | 0.5 | 0.5
Network optimizer | Adam | Adam | Adam
Distraction module optimizer | Adam | Adam | Adam
Network learning rate | 1e-3 | 1e-3 | 1e-2
Distraction module learning rate | 1e-6 | 1e-8 | 1e-5
$\eta$ | 1000 | 100 | 1000
TABLE IV: Summary of Parameter Setting for the Distraction on vision dataset Hyperparameters | CelebA-Attractive | CelebA-Smiling | CelebA-WavyHair
---|---|---|---
Distraction module FC layers | 1 | 1 | 1
FC layers after the Distraction module | 1 | 1 | 1
Epoch | 30 | 15 | 15
Batch size | 128 | 128 | 128
Dropout | 0 | 0 | 0
Network optimizer | Adam | Adam | Adam
Distraction module optimizer | Adam | Adam | Adam
Network learning rate | 1e-3 | 1e-3 | 1e-3
Distraction module learning rate | 1e-6 | 1e-5 | 1e-5
$\eta$ | 1000 | 100 | 100

###  IV-C Tabular

We evaluated our method for bias mitigation to various current state-of-the-
art approaches. We concentrate on strategies specifically tuned to achieve the
best results in statistical parity metrics on tabular studies.

####  IV-C1 Datasets

We conducted experiments using two well-established benchmark datasets in this
field:
UCI Adult Dataset [40]: This dataset, based on demographic data collected in
1994, includes a training set of 30,000 samples and a test set of 15,000
samples. The task is to predict whether an individual’s salary exceeds $50,000
per annum, with gender serving as the binary protected attribute.

Heritage Health Dataset: This dataset involves predicting the Charleson Index,
a measure of a patient’s 10-year mortality risk. It comprises samples from
approximately 51,000 patients, split into a training set of 41,000 and a test
set of 11,000. The protected attribute, age, has nine possible values.

We selected the protected and target attributes in accordance with existing
literature.

(a)

(b)

Figure 2: Accuracy of different benchmark models and distraction model vs.
statistical demographic parity of each model for the (a) UCI Adult dataset and
(b) Heritage Health dataset. The ideal area for this graph is the bottom right
which indicates high accuracy and low statistical demographic parity. This
graph shows that our model performs significantly better than the other
benchmark models in both tabular datasets.

We used training sets to run the Distraction module technique and train the
network. Then we assess the performance by running classifiers for subsequent
prediction tasks.

####  IV-C2 Benchmark Methods

We compared our approach with the following state-of-the-art techniques:

  * •

CVIB [21]: This method employs a conditional variational autoencoder for bias
mitigation.

  * •

MIFR [22]: This technique optimizes fairness by leveraging an information
bottleneck factor combined with adversarial learning.

  * •

FCRL [24]: This approach uses specific approximations for contrastive
information to maximize theoretical goals, which can be employed to strike
appropriate trade-offs between statistical demographic parity and accuracy.

  * •

MaxEnt-ARL [32]: This method uses adversarial learning to reduce bias in
tabular data.

  * •

Adversarial Forgetting [23]: Another method that employs adversarial learning
for bias mitigation.

####  IV-C3 Results

We employed the training sets to execute our Distraction module method and
train the network. Subsequent prediction tasks were evaluated using
classifiers. The average accuracy, representing the most likely accuracy, and
maximum demographic parity, indicating worst-case bias, were computed over
five training iterations with random seeds. In contrast to [24], no
preprocessing was applied to the data before inputting it into our network.
Our reported results represent Pareto solutions for the neural network during
training with varying $\eta$s.

Figures 2a and 2b depict trade-offs between statistical demographic parity and
accuracy for various bias reduction techniques on the UCI Adult and Heritage
Health datasets, respectively. An ideal result would position the curve in the
lower right corner of the graph, indicating accurate and fair outcomes
concerning protected attributes. Our results demonstrate that the Distraction
method significantly outperforms competing methods. This finding is further
supported by the area-over-the-curve data for demographic parity and accuracy
(Table I), showing that our proposed strategy improves the area over the curve
for both datasets by a factor of $\sim$2\. This implies that our bias
reduction framework is the most effective mitigation strategy for tabular
data.

###  IV-D Graph

TABLE V: The comparisons of our proposed method with the baselines on Pokec-z
Method | ACC(%) | AUC(%) |  $\Delta_{DP}$(%) |  $\Delta_{EO}$(%)
---|---|---|---|---
ALFR | 65.4 ±0.3 | 71.3 ±0.3 | 2.8 ±0.5 | 1.1 ±0.4
ALFR-e | 68.0 ±0.6 | 74.0 ±0.7 | 5.8 ±0.4 | 2.8 ±0.8
Debias | 65.2 ±0.7 | 71.4 ±0.6 | 1.9 ±0.6 | 1.9 ±0.4
Debias-e | 67.5 ±0.7 | 74.2 ±0.7 | 4.7 ±1.0 | 3.0 ±1.4
FCGE | 65.9 ±0.2 | 71.0 ±0.2 | 3.1 ±0.5 | 1.7 ±0.6
FairGCN | 70.0 ±0.3 | 76.7 ±0.2 | 0.9 ±0.5 | 1.7 ±0.2
FairGAT | 70.1 ±0.1 | 76.5 ±0.2 | 0.5 ±0.3 | 0.8 ±0.3
NT-FairGNN | 70.0 ±0.1 | 76.7 ±0.3 | 1.0 ±0.4 | 1.6 ±0.2
GAT+Distraction (Ours) | 70.97 ±0.16 | 77.58 ±0.13 | 0.93 ±0.44 | 0.97 ±0.40
TABLE VI: The comparisons of our proposed method with the baselines on Pokec-n
Method | ACC(%) | AUC(%) |  $\Delta_{DP}$(%) |  $\Delta_{EO}$(%)
---|---|---|---|---
ALFR | 63.1 ±0.6 | 67.7 ±0.5 | 3.05 ±0.5 | 3.9 ±0.6
ALFR-e | 66.2 ±0.5 | 71.9 ±0.3 | 4.1 ±0.5 | 4.6 ±1.6
Debias | 62.6 ±0.9 | 67.9 ±0.7 | 2.4 ±0.7 | 2.6 ±1.0
Debias-e | 65.6 ±0.8 | 71.7 ±0.7 | 3.6 ±0.2 | 4.4 ±1.2
FCGE | 64.8 ±0.5 | 69.5 ±0.4 | 4.1 ±0.8 | 5.5 ±0.9
FairGCN | 70.1 ±0.2 | 74.9 ±0.4 | 0.8 ±0.2 | 1.1 ±0.5
FairGAT | 70.0 ±0.2 | 74.9 ±0.4 | 0.6 ±0.3 | 0.8 ±0.2
NT-FairGNN | 70.1 ±0.2 | 74.9 ±0.4 | 0.8 ±0.2 | 1.1 ±0.3
GAT+Distraction (Ours) | 70.07 ±0.5 | 75.8 ±0.38 | 0.62 ±0.14 | 3.0 ±1.0
TABLE VII: The comparisons of our proposed method with the baselines on NBA
Method | ACC(%) | AUC(%) |  $\Delta_{DP}$(%) |  $\Delta_{EO}$(%)
---|---|---|---|---
ALFR | 64.3 ±1.3 | 71.5 ±0.3 | 2.3 ±0.9 | 3.2 ±1.5
ALFR-e | 66.0 ±0.4 | 72.9 ±1.0 | 4.7 ±1.8 | 4.7 ±1.7
Debias | 63.1 ±1.1 | 71.3 ±0.7 | 2.5 ±1.5 | 3.1 ±1.9
Debias-e | 65.6 ±2.4 | 72.9 ±1.2 | 5.3 ±0.9 | 3.1 ±1.3
FCGE | 66.0 ±1.5 | 73.6 ±1.5 | 2.9 ±1.0 | 3.0 ±1.2
FairGCN | 71.1 ±1.0 | 77.0 ±0.3 | 1.0 ±0.5 | 1.2 ±0.4
FairGAT | 71.5 ±0.8 | 77.5 ±0.7 | 0.7 ±0.5 | 0.7 ±0.3
NT-FairGNN | 71.1 ±1.0 | 77.0 ±0.3 | 1.0 ±0.5 | 1.2 ±0.4
GAT+Distraction (Ours) | 77.09 ±0.45 | 77.99 ±0.58 | 0.34 ±0.21 | 12.78 ±2.9

We compare our proposed framework with state-of-the-art approaches for fair
classification and fair graph embedding learning, including ALFR [41], ALFR-e,
Debias [18], Debias-e, FCGE [28], FairGCN [27], and NT-FAIRGNN [42]. A brief
overview of these methods is as follows:

  * •

ALFR [41]: A pre-processing approach that removes sensitive information from
representations generated by an MLP-based autoencoder using a discriminator.
The debiased representations are then used to train a linear classifier.

  * •

ALFR-e: An extension of ALFR that incorporates graph structure information by
combining user features in ALFR with graph embeddings discovered by deepwalk
[43].

  * •

Debias [18]: An in-processing fair classification method that directly applies
a discriminator to the predicted probability of the classifier.

  * •

Debias-e: An extension of Debias that includes deepwalk embeddings into the
Debias features.

  * •

FCGE [28]: A method for learning fair node embeddings in graphs without node
characteristics. Discriminators filter out sensitive data in the embeddings.

  * •

FairGCN [27]: A graph convolutional network designed for fairness in graph-
based learning. It incorporates fairness constraints during training to reduce
disparities between protected groups.

  * •

NT-FAIRGNN [42]: A graph neural network that aims to achieve fairness by
balancing the trade-off between accuracy and fairness. It uses a two-player
minimax game between the predictor and the adversary, where the adversary aims
to maximize the unfairness.

####  IV-D1 Datasets

We conducted experiments using the following datasets obtained from the study
by [27]:

  * •

Pokec [44]: A widely-used social network dataset from Slovakia, akin to
Facebook and Twitter, containing anonymized data from the entire social
network of year 2012. The dataset includes user information such as gender,
age, interests, hobbies, and profession. Sampled subsets, Pokec-z and Pokec-n,
are created based on user provinces. The classification task involves
predicting users’ working environments.

  * •

NBA: A Kaggle dataset of approximately 400 NBA basketball players, featuring
2016-2017 season statistics, nationality, age, and salary. Graph connections
were established using relationships between NBA players on Twitter, collected
via the official API. Players are categorized as American or international, a
sensitive attribute. The classification task is to predict whether a player’s
wage is above the median.

####  IV-D2 Results

Each experiment was conducted five times, and Tables V, VI, and VII report the
mean and standard deviation of the runs for Pokec-z, Pokec-n, and NBA
datasets, respectively. These results represent the selected Pareto solutions
for comparison with the benchmarks. The tables reveal that, in comparison to
GAT, generic fair classification techniques and graph embedding learning
approaches exhibit inferior classification performance, even when utilizing
graph information. In contrast, our Distraction method performs comparably to
baseline GNNs. FairGCN is close to the baseline, but our Distraction technique
outperforms it. When sensitive information is scarce (e.g., NBA dataset),
baselines exhibit clear bias, with graph-based baselines performing worse.
However, our proposed model yields near-zero statistical demographic parity,
indicating effective discrimination mitigation.

###  IV-E Vision

(a)

(b)

Figure 3: ”Attractive” Attribute of CelebA Dataset as the Target Attribute.
(a) reflects the trade-off between Average Precision and Demographic Parity
Difference. (b) shows the trade-off between Average Precision and Equalized
Odds Difference. the Distraction module is showing competitive results to the
baseline.

(a)

(b)

Figure 4: _”Smiling” Attribute of CelebA Dataset as the Target Attribute_. (a)
reflects the trade-off between Average Precision and Demographic Parity
Difference. (b) shows the trade-off between Average Precision and Equalized
Odds Difference. the Distraction module is showing competitive results to the
baseline.

(a)

(b)

Figure 5: _”Wavy Hair” Attribute of CelebA Dataset as the Target Attribute_.
(a) reflects the trade-off between Average Precision and Demographic Parity
Difference. (b) shows the trade-off between Average Precision and Equalized
Odds Difference. the Distraction module is showing competitive results to the
baseline.

We compare our proposed method on vision tasks with several existing
approaches, including: (1) Empirical Risk Minimization (ERM) which achieves
the training task without any regularization, (2) Gap Regularization which
directly regularizes the model, (3) Adversarial Debiasing [18], and (4)
FairMixup [19].

####  IV-E1 Dataset and Setup

To demonstrate the efficacy of our method, we used the CelebA dataset of face
attributes [45], consisting of over 200,000 images of celebrities. Each image
in this dataset has been annotated with 40 binary attributes, including
gender, by human annotators. We chose three attributes – attractive, smiling,
and wavy hair – for binary classification tasks, using gender as the protected
attribute. These attributes were selected because each of them has a sensitive
group that receives a disproportionately high number of positive samples. For
each task, we employed a ResNet-18 architecture [46], augmented with two
additional layers for outcome prediction.

####  IV-E2 Results

The trade-off between Average Precision (AP), Demographic Parity (DP), and
Equality of Opportunity (EO) for attributes ”Attractive”, ”Smiling”, and ”Wavy
Hair” is illustrated in the figures 3, 4, and 5 respectively. Our proposed
Distraction module provides a more balanced trade-off between accuracy and
fairness. Instead of prioritizing one over the other, our method strikes a
better balance, ensuring that the trained model is both accurate and fair.
Moreover, our Distraction module consistently provides better equality of
opportunity across various accuracy levels compared to benchmark models.
Through empirical validation on multiple benchmarks, we’ve shown that the
Distraction module consistently outperforms other methods in achieving
equality of opportunity across various accuracy levels. This indicates that
our method can provide fair treatment to different protected groups while
still maintaining high predictive accuracy.

(a)

(b)

Figure 6: CelebA Dataset – t-SNE visualization $z$ without the Distraction
module (a) and with the Distraction module (b) labeled with gender classes.
The invariant encoding $z$ (a) shows clustering by gender. In contrast (b)
shows no recognizable clustering. These plots are generated using ”Attractive”
attribute.

To further illustrate the effectiveness of the Distraction module, we present
t-SNE visualizations of the output embeddings from the ResNet-18 model both
with and without the Distraction module. t-SNE (t-distributed Stochastic
Neighbor Embedding) is a dimensionality reduction technique that is
particularly well-suited for visualizing high-dimensional data in a low-
dimensional space.
In Figure 6a, we show the t-SNE plot of the embeddings, $z$, produced by the
ResNet-18 model without the Distraction module. In this plot, it is evident
that the embeddings are clustered by gender, suggesting that the model has
learned to rely on gender information for its classifications. This clustering
by gender is problematic as it indicates that the model is biased and may
exhibit unfair behavior when making predictions.
Conversely, in Figure 6b, we present the t-SNE plot of the embeddings,
$\tilde{z}$, generated by the ResNet-18 model with the Distraction module. In
this visualization, the gender-based clustering observed in the previous plot
is no longer apparent. Instead, the embeddings are distributed more evenly in
the low-dimensional space, indicating that the Distraction module has
successfully reduced the influence of gender information on the model’s
embeddings.
The comparison of these two t-SNE plots demonstrates the capability of the
Distraction module to mitigate bias in the model’s embeddings. By preventing
the model from relying on protected attributes such as gender for its
classifications, the Distraction module promotes fairness and reduces the risk
of discriminatory behavior in the model’s predictions.

###  IV-F Ablation Study

In this subsection, we perform an ablation study to investigate the effects of
different functions for the fairness layer in the Distraction module. The
fairness layer can be any differentiable function with controllable parameters
denoted as $\theta_{d}$. We experimented with three configurations for the
Distraction module: one linear layer, two linear layers, and three linear
layers on tabular datasets. The results of the ablation study are summarized
in Table VIII.
For the CelebA dataset, we explored three types of fairness layers: linear
layers, Residual Blocks (ResBlocks), and Convolutional Neural Network (CNN)
layers. The mean scores of each category of CelebA attributes for each type of
fairness layer are provided in Table IX. The ”Inconsistency between experts”
(Incons.), ”Gender-dependent” (G-dep), and ”Gender-independent” (G-indep)
columns in Table IX represent different fairness metrics related to the CelebA
dataset.

The justification for the performance differences between the ResBlock and the
fully connected models in our ablation study lies in the proportion of the
model occupied by the Distraction module and the specific contributions of
these modules to different parts of the network. In particular, there are two
primary factors that explain the observed performance differences: the roles
of the modules in the network and the flow of data through these modules.

Role in the Network: The ResBlock and the fully connected modules serve
different purposes within the network. The ResBlock contributes to the
embedding space of the image, which includes feature extraction and
representation learning. This enables the model to capture the essential
characteristics of the image while minimizing the effect of the protected
attributes (e.g., gender) on the classification task. In contrast, the fully
connected module is mainly involved in the classification part of the network,
where it contributes to the decision-making process based on the features
extracted from the previous layers. This distinction in roles explains why the
ResBlock provides more fair results, as it directly affects the representation
learning and reduces the influence of the protected attributes on the
embeddings.
Flow of Data: The flow of data through the ResBlock is different from the flow
through the fully connected and CNN modules. ResBlocks have skip connections
that allow the input to bypass some layers and directly flow to the subsequent
layers. These skip connections help in preserving the original information and
preventing the loss of critical features during the network’s forward pass. As
a result, the ResBlock is more effective in capturing the inherent
relationships in the data while mitigating the bias from the protected
attributes [46]. In contrast, CNNs involve multiple convolution and pooling
operations, which can cause the loss of some information relevant to fairness.
The fully connected module, with its dense layers, lacks the skip connections
present in the ResBlock, which can lead to less effective bias mitigation.

In conclusion, our ablation study demonstrates that the choice of fairness
layer in the Distraction module can significantly impact the fairness and
accuracy of the model. It is essential to strike a balance between fairness
and accuracy and to select the appropriate fairness layer for the specific
dataset and application at hand.

TABLE VIII: Area over the curve of statistical demographic parity and accuracy for model ablation Method | UCI Adult | Heritage Health
---|---|---
One Linear Layer | 0.411 | 0.492
Two Linear Layers | 0.404 | 0.513
Three Linear Layers | 0.349 | 0.531
TABLE IX: Accumulative comparison between different Distraction layers CNNBlock | AP |  $\Delta$DP |  $\Delta$EO
---|---|---|---
| Incons. | G-dep | G-indep | Incons. | G-dep | G-indep | Incons. | G-dep | G-indep
One Linear Layer | 0.646 | 0.755 | 0.841 | 0.072 | 0.115 | 0.085 | 0.084 | 0.069 | 0.089
CNN Res Block | 0.568 | 0.699 | 0.768 | 0.04 | 0.035 | 0.026 | 0.126 | 0.067 | 0.062
CNN Layer | 0.617 | 0.731 | 0.822 | 0.058 | 0.092 | 0.069 | 0.099 | 0.067 | 0.073

##  V conclusion

In this paper, we introduced a novel bias mitigation strategy, the Distraction
module, for training deep learning models on various data types, including
tabular, image, and graph data. Our proposed approach addresses the problem of
unwanted bias in machine learning models, which can lead to algorithmic
discrimination and unfairness towards certain population subgroups,
particularly minority groups. Unlike existing methods that may result in
information loss or fail to balance accuracy and fairness, the Distraction
module effectively controls bias while maintaining model performance.

The Distraction module operates within the main neural network, independently
managing a subset of weights and optimizing towards specific criteria or
alternative objective functions. This in-process bias mitigation method does
not require adversarial example generation and can be trained on available
datasets without any alterations.

Our empirical evaluation on multiple benchmarks, including the UCI Adult,
Heritage Health, POKEC-Z, POKEC-N, NBA, and CelebA datasets, demonstrates the
effectiveness of our approach. Compared to state-of-the-art methods in the
fairness literature for each dataset, our model exhibits superior performance
in minimizing bias while preserving accuracy. We also introduced a new
adversarial training procedure that further enhances both accuracy and
fairness metrics.

In conclusion, the Distraction module provides a powerful and flexible
strategy for mitigating bias in deep learning models. Its ability to operate
within the main neural network and optimize specific criteria enables more
precise control over bias without sacrificing model performance. As a result,
the Distraction module offers a promising approach for addressing the ethical
concerns associated with algorithmic bias and promoting fairness in artificial
intelligence applications.

##  VI acknowledgement

## References

  * [1] D. Pessach and E. Shmueli, “Algorithmic fairness,” _arXiv preprint arXiv:2001.09784_ , 2020.
  * [2] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in _Proceedings of the 3rd innovations in theoretical computer science conference_ , 2012, pp. 214–226.
  * [3] M. Yurochkin, A. Bower, and Y. Sun, “Training individually fair ml models with sensitive subspace robustness,” _arXiv preprint arXiv:1907.00020_ , 2019\.
  * [4] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in _International conference on machine learning_. PMLR, 2013, pp. 325–333.
  * [5] C. Louizos, K. Swersky, Y. Li, M. Welling, and R. Zemel, “The variational fair autoencoder,” _arXiv preprint arXiv:1511.00830_ , 2015.
  * [6] M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” _Advances in neural information processing systems_ , vol. 29, pp. 3315–3323, 2016.
  * [7] F. Kamiran and T. Calders, “Data preprocessing techniques for classification without discrimination,” _Knowledge and Information Systems_ , vol. 33, no. 1, pp. 1–33, 2012.
  * [8] B. T. Luong, S. Ruggieri, and F. Turini, “k-nn as an implementation of situation testing for discrimination discovery and prevention,” in _Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2011, pp. 502–510.
  * [9] M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian, “Certifying and removing disparate impact,” in _proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining_ , 2015, pp. 259–268.
  * [10] A. Tayebi, N. Yousefi, M. Yazdani-Jahromi, E. Kolanthai, C. J. Neal, S. Seal, and O. O. Garibay, “Unbiaseddti: Mitigating real-world bias of drug-target interaction prediction by using deep ensemble-balanced learning,” _Molecules_ , vol. 27, no. 9, p. 2980, 2022.
  * [11] A. Rajabi and O. O. Garibay, “Tabfairgan: Fair tabular data generation with generative adversarial networks,” _arXiv preprint arXiv:2109.00666_ , 2021\.
  * [12] T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma, “Fairness-aware classifier with prejudice remover regularizer,” in _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 2012, pp. 35–50.
  * [13] M. B. Zafar, I. Valera, M. Gomez Rodriguez, and K. P. Gummadi, “Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment,” in _Proceedings of the 26th international conference on world wide web_ , 2017, pp. 1171–1180.
  * [14] M. B. Zafar, I. Valera, M. G. Rogriguez, and K. P. Gummadi, “Fairness constraints: Mechanisms for fair classification,” in _Artificial Intelligence and Statistics_. PMLR, 2017, pp. 962–970.
  * [15] A. K. Menon and R. C. Williamson, “The cost of fairness in binary classification,” in _Conference on Fairness, Accountability and Transparency_. PMLR, 2018, pp. 107–118.
  * [16] S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, and A. Huq, “Algorithmic decision making and the cost of fairness,” in _Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining_ , 2017, pp. 797–806.
  * [17] A. Cotter, M. Gupta, H. Jiang, N. Srebro, K. Sridharan, S. Wang, B. Woodworth, and S. You, “Training well-generalizing classifiers for fairness metrics and other data-dependent constraints,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 1397–1405.
  * [18] B. H. Zhang, B. Lemoine, and M. Mitchell, “Mitigating unwanted biases with adversarial learning,” in _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_ , 2018, pp. 335–340.
  * [19] C.-Y. Chuang and Y. Mroueh, “Fair mixup: Fairness via interpolation,” _arXiv preprint arXiv:2103.06503_ , 2021.
  * [20] N. Mehrabi, U. Gupta, F. Morstatter, G. V. Steeg, and A. Galstyan, “Attributing fair decisions with attention interventions,” _arXiv preprint arXiv:2109.03952_ , 2021.
  * [21] D. Moyer, S. Gao, R. Brekelmans, G. V. Steeg, and A. Galstyan, “Invariant representations without adversarial training,” _Advances in Neural Information Processing Systems, volume 31, 9084–9093_ , 2018.
  * [22] J. Song, P. Kalluri, A. Grover, S. Zhao, and S. Ermon, “Learning controllable fair representations,” in _The 22nd International Conference on Artificial Intelligence and Statistics_. PMLR, 2019, pp. 2164–2173.
  * [23] A. Jaiswal, D. Moyer, G. Ver Steeg, W. AbdAlmageed, and P. Natarajan, “Invariant representations through adversarial forgetting,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 04, 2020, pp. 4272–4279.
  * [24] U. Gupta, A. Ferber, B. Dilkina, and G. Ver Steeg, “Controllable guarantees for fair outcomes via contrastive information estimation,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, no. 9, 2021, pp. 7610–7619.
  * [25] Y. Dong, O. Lizardo, and N. V. Chawla, “Do the young live in a “smaller world” than the old? age-specific degrees of separation in a large-scale mobile communication network,” _arXiv preprint arXiv:1606.07556_ , 2016.
  * [26] T. Rahman, B. Surma, M. Backes, and Y. Zhang, “Fairwalk: Towards fair graph embedding,” 2019.
  * [27] E. Dai and S. Wang, “Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information,” in _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_ , 2021, pp. 680–688.
  * [28] A. Bose and W. Hamilton, “Compositional fairness constraints for graph embeddings,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 715–724.
  * [29] J. Zhao, T. Wang, M. Yatskar, V. Ordonez, and K.-W. Chang, “Men also like shopping: Reducing gender bias amplification using corpus-level constraints,” _arXiv preprint arXiv:1707.09457_ , 2017.
  * [30] P. Sattigeri, S. C. Hoffman, V. Chenthamarakshan, and K. R. Varshney, “Fairness gan: Generating datasets with fairness properties using a generative adversarial network,” _IBM Journal of Research and Development_ , vol. 63, no. 4/5, pp. 3–1, 2019.
  * [31] S. Hwang, S. Park, D. Kim, M. Do, and H. Byun, “Fairfacegan: Fairness-aware facial image-to-image translation,” _arXiv preprint arXiv:2012.00282_ , 2020\.
  * [32] P. C. Roy and V. N. Boddeti, “Mitigating information leakage in image representations: A maximum entropy approach,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 2586–2594.
  * [33] V. V. Ramaswamy, S. S. Kim, and O. Russakovsky, “Fair attribute classification through latent space de-biasing,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 9301–9310.
  * [34] A. Rajabi, M. Yazdani-Jahromi, O. O. Garibay, and G. Sukthankar, “Through a fair looking-glass: mitigating bias in image datasets,” _arXiv preprint arXiv:2209.08648_ , 2022.
  * [35] M. Wang, W. Deng, J. Hu, X. Tao, and Y. Huang, “Racial faces in the wild: Reducing racial bias by information maximization adaptation network,” in _Proceedings of the ieee/cvf international conference on computer vision_ , 2019, pp. 692–702.
  * [36] M. Wang and W. Deng, “Mitigate bias in face recognition using skewness-aware reinforcement learning,” _arXiv preprint arXiv:1911.10692_ , 2019.
  * [37] G. Louppe, M. Kagan, and K. Cranmer, “Learning to pivot with adversarial networks,” _Proceedings of Advances in Neural Information Processing Systems 30 (NIPS 2017)_ , 2016.
  * [38] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” _ACM Computing Surveys (CSUR)_ , vol. 54, no. 6, pp. 1–35, 2021.
  * [39] V. S. Lokhande, A. K. Akash, S. N. Ravi, and V. Singh, “Fairalm: Augmented lagrangian method for training fair models with little regret,” in _European Conference on Computer Vision_. Springer, 2020, pp. 365–381.
  * [40] D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
  * [41] H. Edwards and A. Storkey, “Censoring representations with an adversary,” _arXiv preprint arXiv:1511.05897_ , 2015.
  * [42] E. Dai and S. Wang, “Learning fair graph neural networks with limited and private sensitive attribute information,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 35, no. 7, pp. 7103–7117, 2023.
  * [43] B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2014, pp. 701–710.
  * [44] L. Takac and M. Zabovsky, “Data analysis in public social networks,” in _International scientific conference and international workshop present day trends of innovations_ , vol. 1, no. 6. Present Day Trends of Innovations Lamza Poland, 2012.
  * [45] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in _Proceedings of International Conference on Computer Vision (ICCV)_ , December 2015.
  * [46] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.


# Constraints on neutron skin thickness and nuclear deformations using
relativistic heavy-ion collisions from STAR††thanks: Presented at Quark Matter
2022

Haojie Xu (For the STAR collaboration)  School of Science, Huzhou University,
Huzhou, Zhejiang 313000, China

###### Abstract

In these proceedings, we present the measurements of neutron skin thickness
and nuclear deformation using isobar ${}^{96}_{44}$Ru+${}^{96}_{44}$Ru and
${}^{96}_{40}$Zr+${}^{96}_{40}$Zr collisions at $\sqrt{s_{{}_{\rm NN}}}=200$
GeV by the STAR detector. The significant deviations from unity of the isobar
ratios of elliptic flow $v_{2}$, triangular flow $v_{3}$, mean $p_{\rm T}$
fluctuations $\langle\delta p_{\rm T}^{2}\rangle/\langle p_{\rm
T}\rangle^{2}$, and asymmetric cumulant ${\rm ac}_{2}\\{3\\}$ indicate large
differences in their quadrupole and octuple deformations. The significant
deviations of the isobar ratios of produced hadron multiplicity $N_{\rm ch}$,
mean transverse momentum $\langle p_{\rm T}\rangle$, and net charge number
$\Delta Q$ indicate a halo-type neutron skin for the Zr nucleus, much thicker
than for the Ru nucleus, consistent with nuclear structure calculations. We
discuss how we extract the neutron skin thickness, the symmetry energy slope
parameter, and deformation parameters from data.

##  1 Introduction

The isobar collisions, ${}^{96}_{44}$Ru+${}^{96}_{44}$Ru and
${}^{96}_{40}$Zr+${}^{96}_{40}$Zr, were originally proposed to search for the
chiral magnetic effect (CME) [1]. Based on the blind analysis with about 2
billion minimum-bias events for each species collected by the STAR experiment,
the initial premise of isobar collisions for CME search was not realized [2].
The CME-related backgrounds were found to differ between isobar collisions
[2], suggesting sizeable nuclear structure differences between the two isobar
nuclei. Those nuclear structure differences and their consequences in
experimental observables had been predicted by energy density functional
theory (DFT) calculations [3, 4]. Owing to the large statistics collected by
the STAR detector and robust cancellation of systematic uncertainties on the
observable ratio $R(X)\equiv\frac{X_{\rm RuRu}}{X_{\rm ZrZr}}$, isobar
collisions provide novel and accurate means to constrain the nuclear
structures and strong force parameters.

Nuclear deformation, a ubiquitous phenomenon for most atomic nuclei, reflects
collective motion induced by the interaction between valence nucleons and
shell structure. In most cases, the deformation has a quadrupole shape that is
characterized by overall strength $\beta_{2}$ and triaxiality $\gamma$, and/or
an octuple shape $\beta_{3}$. In relativistic collisions of two nuclei such
deformations enhance the fluctuations of bulk observables that are sensitive
to initial state geometry [5]. The deformation parameters can be constrained
from the precision measurements of the ratios between two isobar systems of
harmonic anisotropy coefficients $v_{2}$, $v_{3}$, mean transverse momentum
$[p_{\rm T}]$ fluctuations (mean, variance and skewness), and their Pearson
correlation coefficient $\rho(v_{2}^{2}\\{2\\},[p_{\rm T}])$ [6]. Similar
analysis has been performed in Au+Au and U+U collisions [7, 8].

Neutron skin thickness $\Delta r_{\rm np}$ ($\equiv\sqrt{\langle
r_{n}^{2}\rangle}-\sqrt{\langle r_{p}^{2}\rangle}$, the root mean square
difference between neutron and proton distributions) of nuclei can infer
nuclear symmetry energy. Such information is of critical importance to the
equation of state of dense nuclear matter in neutron stars. $\Delta r_{\rm
np}$ has traditionally been measured in low-energy hadronic and nuclear
scattering experiments. Recent measurement using parity-violating electroweak
interactions by the PREX-II experiment has yielded a large neutron skin
thickness of Pb nucleus [9], at tension with the world-wide data established
in hadronic collisions. In isobar collisions at relativistic energies, neutron
skin was predicted [3, 4] to affect event multiplicity and elliptic
anisotropy. Measurements of those quantities can, in turn, offer an
unconventional and perhaps more precise means to probe the neutron skin [10].
Specifically, the ratios between isobar collisions of the produced hadron
multiplicities ($N_{\rm ch}$) [10], the mean transverse momenta ($\langle
p_{\rm T}\rangle$) [11], and the net charge multiplicities ($\Delta Q$) [12]
can probe the neutron skin difference between the isobar nuclei.

##  2 Nuclear deformation measurements in isobar collisions

Anisotropic flow in most central collisions is exquisitely sensitive to
nuclear deformation. The deviations of $R(v_{2})$ and $R(v_{3})$ from unity
observed in most central isobar collisions [2] indicate a large quadrupole
deformation in Ru nucleus and a large octuple deformation in Zr nucleus [5].
Figure 1 shows the $R(v_{2})$ and $R(v_{3})$ as a function of charged track
multiplicity ($N_{\rm trk}^{\rm offline}$) with $|\eta|<0.5$. We simulate
events by a multi-phase transport (AMPT) model with varying $\beta_{2}$ for Ru
(and fixed $\beta_{2}=0.06$ for Zr [13]) and varying $\beta_{3}$ for Zr (and
fixed $\beta_{3}=0$ for Ru) to match the data to extract the best parameter
values. The extracted quadrupole deformation parameter for Ru is
$\beta_{2,Ru}=0.16\pm 0.02$ and the octuple deformation parameter for Zr is
$\beta_{3,Zr}=0.20\pm 0.02$, where the quoted uncertainty includes statistical
and systematic uncertainties. The AMPT results with those deformation
parameters are also shown in Fig. 1. Those deformation parameters extracted
from isobar collisions are consistent with the measurements from traditional
nuclear structure experiments [13, 14]. We also show in Fig. 1 right panel the
isobar ratios of mean $p_{\rm T}$ fluctuations $R(\langle\delta p_{\rm
T}^{2}\rangle/\langle p_{\rm T}\rangle^{2})$. The trend can be qualitatively
described by the Glauber model with the deformation parameters for Ru and Zr.

Figure 1: The $R(v_{2})$ (left), $R(v_{3})$ (middle), and $R(\langle\delta
p_{\rm T}^{2}\rangle/\langle p_{\rm T}\rangle^{2})$ (right) as a function of
$N_{\rm trk}^{\rm offline}$ in isobar collisions at $\sqrt{s_{{}_{\rm
NN}}}=200$ GeV.

The multi-particle correlations are also sensitive to nuclear deformation. In
Fig. 2, we present the difference between isobar collisions of the three-
particle asymmetric cumulant ${\rm ac}_{2}\\{3\\}$. Here ${\rm
ac}_{2}\\{3\\}\equiv\langle\langle
e^{i(2\varphi_{1}+2\varphi_{2}-4\varphi_{3}}\rangle\rangle$ is the average of
three-particle azimuthal correlations over an ensemble of events. The $R({\rm
ac}_{2}\\{3\\})$ trend is similar to that of $R(\langle v_{2}^{4}\rangle)$ as
shown in Fig. 2. The double ratio shown in the bottom panel of Fig. 2
indicates that the non-linear response coefficients $\chi_{4,22}={\rm
ac}_{2}\\{3\\}/\langle v_{2}^{4}\rangle$ are almost identical for the two
isobar systems. Both the trends of $R({\rm ac}_{2}\\{3\\})$ and
$R(\chi_{4,22})$ can be reproduced by the AMPT model simulations. The $R({\rm
ac}_{2}\\{3\\})$ data, being extra sensitive to flow fluctuations, can help
further constrain the deformation parameters of isobar nuclei [15].

Figure 2: The three-particle asymmetry cumulant ratio $R({\rm ac}_{2}\\{3\\})$
(top) and non-linear response coefficient ratio $R(\chi_{4,22})$ (bottom) as a
function of $N_{\rm trk}^{\rm offline}$ in isobar collisions at
$\sqrt{s_{{}_{\rm NN}}}=200$ GeV. AMPT simulations are also shown for
comparison.

Figure 3:  Left: Ratio of the measured multiplicity ($N_{\rm trk}^{\rm
offline}$) distributions in isobar collisions at $\sqrt{s_{{}_{\rm NN}}}=200$
GeV. Also shown are Monte Carlo Glauber model results with nuclear densities
calculated by DFT with three $L(\rho_{c})$ values. Right: Ratio of $\langle
N_{\rm trk}^{\rm offline}\rangle$ in top $2\%$ central collisions as function
of $L(\rho_{c})$ from Glauber model. The red line is the measured datum value
$1.0019$ with statistical uncertainties smaller than $0.0001$.

##  3 Neutron skin measurements

Nuclear density distributions, and thus the neutron skin thicknesses, depend
on the slope parameter $L$ of symmetry energy as a function of nuclear
density. With a given $L$ of the nuclear interaction potential, the nuclear
density can be calculated by the DFT framework. The event multiplicity
produced in heavy-ion collisions is sensitive to the density distributions of
the colliding nuclei, and thus the $L$. In Fig. 3, we present the ratio of the
multiplicity distributions measured in isobar collisions within pseudo-
rapidity $|\eta|<1$, and those computed by the Monte Carlo (MC) Glauber model
with the density distributions calculated by DFT with three values of
$L(\rho_{c})$ [10], where $\rho_{c}=0.11\rho_{0}/0.16\simeq 0.11$ fm-3 is
nuclear subsaturation cross density. The model result with $L(\rho_{c})=47.3$
MeV can reasonably describe the data including the high multiplicity range. In
the right panel of Fig. 3 we compare the ratios of mean multiplicity at top
$2\%$ centrality between data and model calculations and obtain
$L(\rho_{c})=53.8\pm 1.7({\rm stat.})\pm 7.8({\rm sys.})$ MeV. The
corresponding neutron skin thicknesses for Ru and Zr are $(\Delta r_{\rm
np})_{\rm Ru}=0.051\pm 0.009$ fm and $(\Delta r_{\rm np})_{\rm Zr}=0.195\pm
0.019$ fm, respectively. The systematic uncertainties are estimated with
different models (TRENTo vs. Glauber) and different $N_{\rm trk}^{\rm
offline}$ cutoffs, and considering nuclear deformations which are the dominant
contribution.

Figure 4: The mean transverse momentum ratio $R(\langle p_{\rm T}\rangle)$ as
a function of centrality in isobar collisions at $\sqrt{s_{{}_{\rm NN}}}=200$
GeV. The three thick curves are obtained from iEBE-VISHNU simulations with
three different $L(\rho_{c})$ values, the thin-dashed curve denotes the model
simulation with $\beta_{2,\rm Ru}=0.16$ and $\beta_{3,Zr}=0.20$ under
$L(\rho_{c})=47.3$ MeV.

The mean transverse momentum $\langle p_{\rm T}\rangle$ also depends on the
size of the colliding nuclei. The sensitivity is the strongest in most central
collisions. This provides another way to constrain the $L(\rho_{c})$
parameter. Although $\langle p_{\rm T}\rangle$ depends on bulk properties of
the QGP medium, the $R(\langle p_{\rm T}\rangle)$ in isobar collisions shows
weak sensitivity to shear and bulk viscosities of the QGP medium [11]. The
$R(\langle p_{\rm T}\rangle)$ as a function of centrality is shown in Fig. 4.
The trend can be described by iEBE-VISHNU (Event-By-Event Viscous Israel
Stewart Hydrodynamics and UrQMD) model simulations, with the nuclear densities
obtained from DFT [10, 16]. Based on the $R(\langle p_{\rm T}\rangle)$ values
at top $5\%$ centrality, we extract the slope parameter $L(\rho_{c})=56.8\pm
0.4({\rm stat.})\pm 10.4({\rm sys.})$ MeV, and the corresponding neutron skin
thicknesses of the isobar nuclei of $(\Delta r_{\rm np})_{\rm Ru}=0.052\pm
0.012$ fm and $(\Delta r_{\rm np})_{\rm Zr}=0.202\pm 0.024$ fm. The systematic
uncertainties are also dominated by the nuclear deformation effect which can
be improved in the future. The results extracted from $\langle p_{\rm
T}\rangle$ are consistent with those from multiplicity distributions above.

We compare in Fig. 5 our $L({\rho})$ (the slope parameter of symmetry energy
at saturation density $\rho_{0}$) results with a compilation of world data
from traditional nuclear structure experiments [17]. Our results are
consistent with world data with comparable precision.

Figure 5: The compilation of world data of $L(\rho)$ from Ref. [17]. The
$L(\rho_{c})$ values extracted from isobar data are converted into $L(\rho)$
for comparison. The PREX-II data are taken from Ref. [9]

##  4 Summary

The isobar ${}^{96}_{44}$Ru+${}^{96}_{44}$Ru and
${}^{96}_{40}$Zr+${}^{96}_{40}$Zr collision data collected by STAR at
$\sqrt{s_{{}_{\rm NN}}}=200$ GeV provide novel means to probe the nuclear
structure and deformation of the isobar nuclei. From the isobar ratios of the
measured anisotropic flow, multiplicity, and mean transverse momentum, with
the help of DFT calculations, we have extracted the nuclear deformation
parameters $\beta_{2},\beta_{3}$, and neutron skin thicknesses $\Delta r_{\rm
np}$ of the isobar nuclei, and the density slope parameter of symmetry energy
$L(\rho_{c})$. The results are consistent with world-wide data from
traditional nuclear scattering experiments with comparable precision.

This work was supported in part by the National Natural Science Foundation of
China under Grant Nos. 11905059, 12035006, 12075085.

## References

  * [1] S. A. Voloshin, Phys. Rev. Lett. 105, 172301 (2010).
  * [2] M. Abdallah et al. (STAR Collaboration), Phys. Rev. C 105, 014901 (2022).
  * [3] H. j. Xu, X. Wang, H. Li et al., Phys. Rev. Lett. 121, 022301 (2018).
  * [4] H. Li, H. j. Xu, J. Zhao et al., Phys. Rev. C 98, no.5, 054907 (2018).
  * [5] C. Zhang and J. Jia, Phys. Rev. Lett. 128, 022301 (2022).
  * [6] J. Jia and C. J. Zhang, [arXiv:2111.15559 [nucl-th]].
  * [7] J. Jia, S. Huang and C. Zhang, Phys. Rev. C 105, 014906 (2022).
  * [8] C. Zhang [STAR], [arXiv:2203.13106 [nucl-ex]].
  * [9] D. Adhikari et al. (PREX Collaboration), Phys.Rev.Lett. 126, 172502 (2021).
  * [10] H. Li, H. j. Xu, Y. Zhou et al., Phys. Rev. Lett. 125, 222301 (2020).
  * [11] H. j. Xu, W. Zhao, H. Li et al., [arXiv:2111.14812 [nucl-th]].
  * [12] H. j. Xu, H. Li, Y. Zhou et al., Phys. Rev. C 105, L011901 (2022).
  * [13] B. Pritychenko, M. Birch, B. Singh and M. Horoi, Atom. Data Nucl. Data Tabl. 107, 1-139 (2016) [erratum: ibid. 114, 371-374 (2017)].
  * [14] T. Kibédi and R. h. Spear, Atom. Data Nucl. Data Tabl. 80, 35-82 (2002).
  * [15] S. Zhao, H. j. Xu, Y. X. Liu and H. Song, [arXiv:2204.02387 [nucl-th]].
  * [16] Z. Zhang and L. W. Chen, Phys. Rev. C 94, no.6, 064326 (2016).
  * [17] B. A. Li, B. J. Cai, W. J. Xie and N. B. Zhang, Universe 7, no.6, 182 (2021).


SoftFlow: Automated HW-SW Confidentiality Verification for Embedded Processors

Lennart M. Reimann1, Jonathan Wiesner1, Dominik Sisejkovic2, Farhad Merchant4, and Rainer Leupers1

1RWTH Aachen University, Germany,
{lennart.reimann, wiesner<EMAIL_ADDRESS>
2Corporate Research, Robert Bosch GmbH, Germany<EMAIL_ADDRESS>
4Newcastle University, UK<EMAIL_ADDRESS>
SoftFlow: Automated HW-SW Confidentiality Verification for Embedded Processors

Anonymous Authors

Despite its ever-increasing impact, security is not considered as a design objective in commercial electronic design automation (EDA) tools. This results in vulnerabilities being overlooked during the software-hardware design process.
Specifically, vulnerabilities that allow leakage of sensitive data might stay unnoticed by standard testing, as the leakage itself might not result in evident functional changes. Therefore, EDA tools are needed to elaborate the confidentiality of sensitive data during the design process. However, state-of-the-art implementations either solely consider the hardware or restrict the expressiveness of the security properties that must be proven. Consequently, more proficient tools are required to assist in the software and hardware design. To address this issue, we propose SoftFlow, an EDA tool that allows determining whether a given software exploits existing leakage paths in hardware. Based on our analysis, the leakage paths can be retained if proven not to be exploited by software. This is desirable if the removal significantly impacts the design's performance or functionality, or if the path cannot be removed as the chip is already manufactured.
We demonstrate the feasibility of SoftFlow by identifying vulnerabilities in OpenSSL cryptographic C programs, and redesigning them to avoid leakage of cryptographic keys in a RISC-V architecture.

confidentiality, property checking, information flow analysis, risc-v

§ INTRODUCTION

Malicious modifications or unintended insecure software and hardware implementations must be detected at early design stages to avoid expensive post-silicon patches.
Therefore, Electronic Design Automation (EDA) tools must consider not only performance, power, and area objectives, but also the security implications of hardware and software. Commercial approaches to running secure kernels on secured hardware have already been developed, whereby leakage paths are not considered [1]. Moreover, the academic community continues to introduce security-aware EDA tools [2].

Information Flow Analysis (IFA) gains approval when analyzing software or hardware for the leakage of secret data [3]. However, existing IFA tools do not consider the bare-metal software running on a processor when analyzing the information flow, thus giving oversensitive results. Leakage paths that might not be exploited by software are presented as dangerous. Although it is desirable to remove all leakage paths to untrusted components in hardware, it might not always be possible to do so. Sometimes the vulnerabilities are only detected after the manufacturing process, such as Meltdown [4]. Moreover, removing all leakage paths might result in the loss of functionality or performance. A leakage path is a signal route that carries data from a sensitive source to untrusted components (Fig. <ref>).

A leakage path in a circuit through a multiplexer.

Fundamentally, there are two distinct flavors of information flow analysis: dynamic and static IFA. A dynamic analysis tracks the information flow within hardware for a given software running a set of test cases [5]. As only the program flow for the test cases is considered, vulnerabilities might be overlooked. Thus, static approaches that consider software independent of test cases yield more reliable verification results. Most state-of-the-art tools require manual translation of either software or hardware descriptions, or limit the expressiveness of security properties. Thus, a new EDA tool is required to overcome these constraints.
To address this challenge, we present SoftFlow, a framework that allows a hardware-software co-verification to analyze whether determined leakage paths are activated for a given software and arbitrary input data.
SoftFlow allows the designer to adapt the program and avoid leakage, or deploy software patches for manufactured chips.

The contributions of this paper are:
(1) An automated tool to convert leakage paths of a hardware description into provable hardware properties. (2) A complete framework that guarantees that determined leakage paths in hardware are not exploited by a given software. (3) An analysis of the state-of-the-art OpenSSL cryptographic algorithms to demonstrate the usability of the tool for a RISC-V architecture.

§ RELATED WORK

Block diagram illustrating SoftFlow's tool flow.

Sequential Extended Regular Expression (SERE) operators [6] used in this work.

Name       PSL Usage     Verilog Example      Interpretation
2*CONCAT     2*A;B     2*$(cntrl\_sig1 == 0x20);(cntrl\_sig2 == 0x11)$     Two cycles exist, so that first condition A is true, followed

by the cycle in which B is true.
FUSE     A:B     $(cntrl\_sig1 == 0x20):(cntrl\_sig2 == 0x11)$     A cycle exists in that both Boolean expressions A and B are true.
REPETITION_INF     A[*]     $(cntrl\_sig1 == 0x20)[*]$     The Boolean expression is true in every cycle.

As the complete system behavior can only be modeled when considering hardware and software together, a co-verification is used frequently to enforce security properties for a given design [7].
However, co-verification is a challenging task, as hardware and software are described in different domains.
Typically, one description is either converted into the other domain, or both descriptions are converted to allow the verification of both implementations [8].
These methods mainly depend on model checkers that formally verify the defined properties. In [9], a manual translation of the hardware description, program, and properties is conducted to allow a co-verification. However, a manual translation is error-prone and could introduce vulnerabilities into the description. A model checker, similar to the one used in this work, is used in [10][11]. However, both proposals limit their use in the variability of the observed software and the expressiveness of the security property.
With the introduction of SoftFlow, we overcome the mentioned limitations and establish a novel framework that allows an automated generation of security properties. Therefore, we enable a static security analysis of both hardware and software to facilitate the design of secure software for insecure hardware.

§ PRELIMINARIES

§.§ Threat Model

The threat model is based on the following assumptions:

* The vulnerability that exposes the secret is already present at the RTL-design stage of embedded processors.

* The adversary tries to gain information about secret signals, such as cryptographic keys.

* The complete hardware design is known to the attacker.

* The adversary can observe primary inputs and outputs of the design.

* The software cannot be exchanged by the attacker.

* Arbitrary input data can be used.

§.§ Information Flow Analysis

IFA allows the static or dynamic analysis of programs and hardware descriptions to elaborate on whether unintended information flow can occur. Security labels are used to flag components to be part of certain security classes [12]. Thus, an analysis determines whether sensitive information can be leaked to untrusted areas of the hardware. For the example in Fig. <ref>, it is analyzed whether the secret bit can be leaked to the observable output for all possible known input bit sequences.

State-of-the-art tools can conduct a static analysis of the Verilog hardware description to determine unwanted data leakages. QFlow [13, 14] and QIF-Verilog [15] use quantitative information flow to classify data leakages as negligible or a threat.
A scopechain is provided that allows identifying the path within the hardware description.
SoftFlow utilizes QFlow to gather the most suspicious leakage path in a system.

Separating the leakage paths into blocks to allow a suitable environment for property generation. The arrows illustrate signal assignments in which dotted arrows describe conditional assignments.

§.§ Property Specification Language (PSL)

PSL [16] is compatible with Verilog hardware descriptions and allows the modeling of the temporal behavior of digital architectures.

A simple example is a condition for a multiplexer:
$(control\_signal == 0x20)$.
The Boolean expression enables the model checker to determine when information is forwarded.
Moreover, Sequential Extended Regular Expressions (SERE) can model the temporal dependencies. CONCAT, FUSE, and REPETITION_INF of the SERE repertoire are used in this work, as explained in Table <ref>.
The operators can be combined into sequences, allowing the modeling of a leakage path.
The verification layer uses commands like  and .  indicates that the included property is expected to hold, thus
restricting the state-space of the evaluated architecture for the desired property verification.
However, the user must verify the assumptions to avoid introducing insecurities into the process.
-statements can be used to determine whether a property is true for at least one state in the state-space using a model checker.
If the property is proven to be uncoverable, the leakage cannot occur for the assumptions. SoftFlow automatically generates the assumptions and properties (Fig. <ref>).

\begin{equation}
\label{eq:path_sequence_example}
\underbrace{\underbrace{\text{\textbf{active(}$s_2$, $s_3$\textbf{)}}}_{AS}\text{ ; }\underbrace{\text{\textbf{alive(}$s_3$\textbf{)}[*]}}_{LS}}_{sb_1}\text{ ; } \underbrace{\underbrace{\text{\textbf{active(}$s_3$, $s_4$\textbf{)} : \textbf{active(}$s_4$, $s_5$\textbf{)} : \textbf{active(}$s_5$, $s_6$\textbf{)}}}_{AS}\text{ ; }\underbrace{\text{\textbf{alive(}$s_6$\textbf{)}[*]}}_{LS}}_{sb_2}\text{ ; }
\underbrace{\underbrace{\text{\textbf{active(}$s_7$, $s_8$\textbf{)}}}_{AS}}_{sb_3}
\end{equation}

§ SOFTFLOW

For the identification of the leakage paths, SoftFlow uses QFlow, due to its capability to analyze the information flow bitwise [13].
QFlow extracts the leakage paths from the sensitive source to the possibly harmful target.
The security properties are generated by SoftFlow (Fig. <ref>), as explained below.
Additionally, model-checking requires information about the software to constrain the model, reducing the number of false positives. Below, we clarify how the PSL rules are generated, verified and evaluated for multiple programs.

§.§ Property Generation

This subsection describes a tool that processes the leakage paths, given by a static quantitative IFA, and the hardware description to generate properties using PSL.

Combinational First, a combinational circuit is considered. In this case, a leakage path can consist of combinational operations and signal assignments (Fig. <ref>). These assignment types can be divided into two classes: conditional and unconditional. Unconditional assignments happen independently of any control signals, while conditional assignments depend on a multiplexer's control signal. When checking whether a leak can occur, one needs to analyze whether all conditional assignments are active at the same time.

Hereafter, active-function $A$ describes whether an information flow between two signals is active.

Sequential Blocks Due to the timing dependency, additional system behavior needs to be considered for sequential circuits. Fig. <ref> illustrates a single leakage path. The path leads from a trusted signal, storing a secret bit, to an untrusted signal.
Due to the timing behavior of sequential logic within this path, each leakage path needs to be separated into multiple blocks.

Assignments between non-sequential signals of the same block must occur in a single cycle. But the data can live for several cycles in sequential signals, such as registers, before moving on. Separating the leakage paths into blocks that always end with a sequential signal allows for a differentiation of this behavior.
The separation into the sequential blocks ($sb$) is depicted in Fig. <ref>.

Additionally, one must consider that the secret data is not overwritten within the sequential logic before forwarding it to the next $sb$.

Only conditional assignments influence the property generation.
For the first sequential block in Fig.  <ref>, only the second assignment holds a condition. Hence, the block active-function $BA$ for this sequential block $sb_1$ is true if the conditions for the assignment are true in a single cycle.

§.§.§ Leakage Path Activation

First, the sequential logic within the leakage path is identified. This is done to form the sequential blocks.
The secret datum can only pass a single sequential block in each cycle. For leakage to occur, the second sequential block needs to be activated after the previous one (Fig. <ref>).
Additionally, the two blocks do not have to be activated in consecutive cycles
as long as the information in the previous sequential logic stays alive and is not overwritten until it passes the next sequential block, which is described with the alive-function $L$.

Each sequential block results in a $BA$ and $L$ function, except for the last one.

In the example (Fig. <ref>), two alive-functions and three block active-functions are required to describe the activation of the path.

§.§.§ Active & Alive-Function Collection

First of all, the Verilog description is parsed to form a tree structure that can be iterated over.

Afterward, the algorithm iterates over the leakage paths to determine the individual activation conditions. The conditions for a single signal assignment are connected via an OR-operator as only a single true activation condition can lead to leakage. The resulting set of active-functions are temporally stored in a file to be processed to PSL.

The alive-function $L$ is determined similarly to the active-function.

The alive-function returns 'true' if the current value in a sequential logic is not overwritten for a fixed amount of cycles. An empty signal assignment describes a value that is not overwritten.

The activation condition from the empty node to the destination node is calculated and stored as the alive function.

§.§.§ Specifying SoftFlow's PSL Sequences

The collected active-functions and alive functions need to be translated to PSL to enable model-checking. For the example in Fig. <ref>, the PSL sequence is shown in Equation <ref>, with $AS$ and $LS$ describing the activation and liveness sequences.

The sequence is extended with the  command, forming the cover rules. If the model checker can cover this path, the leakage can occur for the given assumptions.

The individual Boolean activation conditions are connected using the FUSE operator (Table <ref>), as the conditions must be true in the same cycle.

After the activation conditions are converted, the liveness condition for the sequential logic is required. The returned liveness condition is converted to a PSL expression using the REPETITION_INF operator (Table <ref>).

The activation conditions and the alive-functions are combined into the full sequence for a single $sb$ using CONCAT (Table <ref>).

Uncoverable properties state that the leakage path cannot be taken.

§.§ Property Verification

By employing model-checking, one can establish a level of certainty regarding the reliability of the verification results.

The individual cover rules allow an independent evaluation for every leakage path, enabling the model checker to work in parallel for these properties.

The program memory address is used to identify the instruction and the related C code line.

SoftFlow's generated properties do not consider the compiler program for this processor. As we would like to elaborate on the security of a given C program and an insecure hardware implementation, the hardware model for the model-checking needs to be constrained before running the verification. This is done to reduce the false positives, which describe a leakage of secret data for programs that would not be implemented. Those constraints are implemented using .
Since the model permits the data memory to contain any information, it is necessary to contemplate all conceivable routes within a program. Conditional branches depend mostly on data in the data memory. Thus, all possible conditional jumps need to be considered. The user can choose from the following derived assumptions (A to F) to restrict the program flow.

A No illegal instructions:

A assumption is made which claims that the read port of the program memory can never hold the value of prohibited instructions.

B Only used instructions:

The formal model can be further constrained by analyzing the compiled machine-code statically. An assumption can be generated that further constrains the model by stating that only instructions present in this compiled program can be read from the program memory. The order is not yet considered.

C Replacing the program memory:

To avoid modeling an entire program memory, a lookup table is used instead, which also enables consideration of the order of instructions.
Furthermore, since return addresses of function calls are stored on the stack, it is possible for a return to transpire to any point within the program.

D Only legal return addresses:
The remaining three assumptions are generated using compiler information.
The assumption is used to remove undefined return addresses, which results in arbitrary program flows for the verification.

E Only correct hardware jumps:
Compiler information is processed to allow only valid start and return addresses for hardware loops during the evaluation.

F Call-return matching:
When a function is invoked from several locations, it is essential to account for both return addresses. Confirming a legitimate correlation between a call and its corresponding return is only achievable during runtime. Nonetheless, if an authentic hardware call-stack is utilized to store the return addresses, it can already be taken into consideration during the verification process.

§ EVALUATION

The evaluation is conducted for a RISC-V Verilog description and compiler.

The processor uses the RV32IC instruction-set, in which an instruction for a single round of AES, and an additional memory have been added that holds the cryptographic key.
A block diagram illustrating the architecture is shown in Fig. <ref>. The data input from the key memory is labeled sensitive in order for QFlow to yield the most suspicious leakage paths from the signal to all output ports of the design, including the memories' data and address ports.

Abstract block diagram of the RISC-V architecture.

The analysis is performed for several cryptographic algorithms from OpenSSL[17]: ChaCha20, AES-256, Camelia, Aria, and SHA-256.
SoftFlow is evaluated for different assumptions to elaborate on their efficiency and security.

§.§ Verification Cases

The used assumptions are listed behind each mode.

No assumptions are added to the model checker. The verification tool works solely with the properties generated from our property generator for the leakage paths.

Legal A:
The loading port of the program memory is restricted, disallowing all illegal instructions to be read.

Used A& B:
The data port of the program memory is restricted using an  statement that allows only instructions present in a compiled machine-code to be read.

Jumps A to E:
In this mode, only valid return addresses can be used for a return command and hardware loops, allowing a more realistic program flow.

Stack A to F:
A hardware stack allowing the return-call matching is added as supplemental Verilog code.

First, the assumptions for Used are utilized. If the property for this path cannot be proven to be uncoverable, all verification cases are tried until the model checker is successful.
Considering the valid states S of a processor model, the valid states for Stack are present in Used, resulting in:

\begin{equation*}
S_{\textsc{Stack}}\subseteq S_{\textsc{Jumps}}\subseteq S_{\textsc{Used}}\subseteq S_{\textsc{Legal}}\subseteq S_{\textsc{None}}
\end{equation*}

The inclusiveness of the more restrictive verification cases further reduces the state space that has to be analyzed.

§.§ Results

§.§.§ Coverages

The evaluation results of SoftFlow's functionality are presented below. The verification modes are applied to the five cryptographic algorithms for the RISC-V.

Without Software:

[thick, -stealth, red] (2.9,2.4) – (3.5,2.5);
[dashed, line width=0.8mm, red] (0, 2.15) – (0, 2.75);

ldk  x7,  1(x6!)
addi x10, x13, 40
sw   x13, 0(x2)
li   x6,  1
sw   x7,  4(x13!)
mv   x14, x7
ldk  x7,  1(x6!)
sw   x7,  4(x13!)
mv   x15, x7
ldk  x7,  1(x6!)
sw   x7,  4(x13!)
sw   x7,  4(x2)

Assembly program

[style=CStyle, basicstyle=]
int chess_storage(KM:0) km[4];
int Camellia_Ekeygen(int keyBitLength,
KEY_TABLE_TYPE k)

register u32 s0, s1, s2, s3;

k[0] = s0 = km[0];
k[1] = s1 = km[1];
k[2] = s2 = km[2];
k[3] = s3 = km[3];
s0 ^= k[0], s1 ^= k[1],
C program

A naive implementation of Camellia: The dashed line symbolizes the reported instructions activating the leakage path. The corresponding C-code is marked with an arrow.

The assumptions of the modes None and Legal are applied to the verification model. These two modes are independent of the actual software running.

QFlow yields 3776 leakage paths that need to be elaborated for the given architecture.

Leakage paths are uncoverable if the leakage cannot occur for the given assumptions. The paths that can be activated are labeled covered. Both, the Legal and None modes yield 857 uncoverable paths, which indicates that the processor only accepts legal instructions and that the 857 paths are false positives from QFlow. With SoftFlow, the false positives are removed entirely!

With OpenSSL Cryptographic Algorithms:
Next, the capabilities of SoftFlow are elaborated for the different OpenSSL algorithms for the remaining verification modes.
An example of how the applications are modified for the architecture is shown in Fig. <ref>. The key can be accessed with a pointer to the key memory KM. Table <ref> illustrates the results of the elaboration for the modes Stack, Jumps, and Used. As one can see, the number of uncoverable paths increases for all applications when more restrictive assumptions are used.

Summarized metrics for the five applications grouped by verification case.

Application    Aria     ChaCha20     Camellia     AES     SHA

Instr. count   2112     1347     1973     1409     649

Call depth     4     4     4     6     5

Covered     65     74     58     58     48

Uncoverable     3711     3702     3718     3718     3728

Covered     32     32     32     32     32

Uncoverable     3744     3744     3744     3744     3744

Covered     32     32     32     32     32

Uncoverable    3744     3744     3744     3744     3744

No difference in the results between the modes Stack and Jumps can be observed.

The call tree depths are limited for all five applications, which is required if the Stack mode is used. Otherwise, the hardware stack would have to be of unlimited size. Thirty-two leakage paths are still covered for the design and applications. The cause for this leakage detection can be seen in Fig. <ref> for the application Camellia.
The remaining exemplary elaborations are only presented for Camellia. Final results are presented for all applications. The assembly code in Fig. <ref> shows that the compiler loads the key values from the dedicated key memory (ldk, load key) and stores them in the untrusted data memory (sw, store word) for the array k. The programmer can avoid the usage of the leakage path by modifying the C program. Listing <ref> shows the replacement of the variable k with the pointer km, so that the values are not automatically stored in the untrusted data memory. Additionally, the pointer is marked as volatile (Listing <ref>), instructing the compiler not to optimize any data movements for KM's data. Table <ref> depicts the outcomes of SoftFlow subsequent to this modification. The leakage paths are not activated anymore except for the Aria application.

For Aria, the compiler places the key in a register that is directly connected to the address port of the key memory, which is marked as untrusted. Forcing the compiler to pick a different register can also avoid this leakage. However, the limited intrusion into the compiler does not allow this change. The final evaluation shows an example of finding software Trojans inside the program. In the four applications that achieved 100% avoidance of leakage, a software Trojan is implemented. An example Trojan for the Camellia application can be seen in Listing <ref>. For a particular input value stored in the data memory, the keys are written to the untrusted memory.

[style=CStyle, firstnumber=14, caption=Replacing k with km in the application camellia, label=lst:ktokm]
s0 ^= km[0], s1 ^= km[1],	s2 ^= km[2], s3 ^= km[3];

[style=CStyle, caption=Declaring km as volatile, label=lst:volatile]
volatile int chess_storage(KM:0) km[4];

[caption= The malicious lines implementing a simple Trojan in the application Camellia., label=fig:trojan_camellia, style=CStyle]
if (input[0] == 0x4d2)

u32* leakage = (u32*) malloc(sizeof(u32)*8);
for (int j=0; j<8; j++) leakage[j] = km[j];

Metrics of the modified applications.

Application    Aria     ChaCha20     Camellia     AES     SHA

covered     473     0     0     0     0

uncoverable     3303     3776     3776     3776     3776

time     16016     16780     3213     1337     4432

Verification of the modified applications with embedded Trojans.

Application    ChaCha20     Camellia     AES     SHA

covered     32     32     32     32

uncoverable     3744     3744     3744     3744

time     26045     2127     1386     1321

trigger (hex)    12345678     4d2     fedcba     12ef34dc

The results for the evaluations of the Full mode illustrate that all Trojans that use the leakage paths are detected despite their difference in the trigger, as shown in Table <ref>. The value of the trigger does not play a role in the detection, as the data coming from the untrusted data memory is assumed to be of any possible value.

scaled y ticks = false,

legend style=at=(0.495,1.3), anchor=north,legend columns=-1,
bar width=0.18cm,
ylabel=Time ($10^3$ s),
enlarge x limits=0.13,
symbolic x coords=None,Used,Jumps,Stack,Full,
ylabel near ticks,
ylabel shift=-0.55em,
xtick pos=left,
ytick pos=left,
legend image post style=scale=0.9,
y tick label style=font=,
y label style=font=
[fill=white] coordinates (None,1.085) (Used,1.041) (Jumps,6.733) (Stack,7.228)(Full,1.925);
[fill=rwth_blue!25] coordinates (None,1.085) (Used,0.856) (Jumps,14.356) (Stack,11.180)(Full,8.760);
[fill=rwth_blue!50] coordinates (None,1.085) (Used,0.852) (Jumps,4.874) (Stack,4.153)(Full,2.451);
[fill=rwth_blue!75] coordinates (None,1.085) (Used,0.826) (Jumps,4.416) (Stack,4.684)(Full,1.391);
[fill=rwth_blue] coordinates (None,1.085) (Used,0.789) (Jumps,4.537) (Stack,4.570)(Full,4.155);

Aria, Chacha20, Camellia, AES-256, SHA-256

The summarized runtime of the verification procedure for the different verification modes and applications.

§.§.§ Runtime

For the unmodified applications (e.g., Fig. <ref>) that still carry 32 covered paths, the verification is conducted by using the available verification modes. Legal is not presented here, as it yielded the same verification results as None. As shown in Fig. <ref>, the lowest runtimes are given by the verification modes that restrict the hardware model the least. However, some leakage paths could not be marked as uncoverable for the less restrictive assumptions (Table <ref>).

The runtime increases drastically for all applications at Jumps. Moreover, as expected, the lowest runtime can be achieved using the verification mode Full. The overall runtime is optimized, while yielding a precise result for all applications.
As the verification of the properties for every leakage path can be parallelized, depending on the available resources of the designer, the runtimes can be drastically reduced.

Fig. <ref> illustrates how successful the different verification modes are in their task. It can be observed that most leakage paths can be flagged as uncoverable by assuming that only the instructions given in a program will be used.

The higher number of cases verified using Used explains the reduced runtime of the Full-mode in Fig. <ref>.

§ LIMITATIONS

Overall, the tool can identify vulnerabilities and assist the designer in writing software that allows safe use of insecure hardware. However, some limitations must be considered.

Leakage paths can be triggered with instructions like load and store operations, as shown in Fig. <ref>. The data is leaked without any change. This results in a varied attack scenario from the one used by QFlow.
Additionally, the evaluation is only conducted on 3776 existing leakage paths, as QFlow only outputs the most suspicious ones. Although the elaboration of the paths was successful, all leakage paths must be considered.
It might not be possible for some applications and hardware combinations to avoid leakage. This could be due to unconditional leakages or leakages caused by instruction patterns required for the application.

§ CONCLUSION

We presented SoftFlow, an EDA tool that utilizes formal verification to conclusively indicate whether leakage paths in hardware are activated for a given software. Therefore, SoftFlow facilitates both a security-driven system design from the ground up and post-fabrication security patches. The usability of the proposed framework was demonstrated by analyzing OpenSSL cryptographic algorithms for a RISC-V architecture. The evaluation proves that secure software disabled all 3776 leakage paths. In addition, the consideration of both software and hardware mitigated all false positives of QFlow.
With SoftFlow, we enable a security-aware software-hardware co-verification process that takes into account the intricate interplay of dedicated hardware and software.
In the future, we plan to investigate security-aware compilers that facilitate the complete removal of vulnerabilities if allowed by both hardware and software.

[thick, red] (0, 2.65) – node[below, xshift=0.2cm]Total number of leakage paths(7.4, 2.65) ;

legend style=at=(0.535,1.3), anchor=north,legend columns=-1,
bar width=0.18cm,
ylabel=# uncoverable paths ($\cdot 10^3$),

enlarge x limits=0.13,
symbolic x coords=Used,Jumps,Stack,
ylabel near ticks,
ylabel shift=-0.55em,
xtick pos=left,
ytick pos=left,
legend image post style=scale=0.9,
y tick label style=font=,
y label style=font=
[fill=white] coordinates (Used,3.711) (Jumps,0.033) (Stack,0);
[fill=rwth_blue!25] coordinates (Used,3.702) (Jumps,0.042) (Stack,0);
[fill=rwth_blue!50] coordinates (Used,3.718) (Jumps,0.026) (Stack,0);
[fill=rwth_blue!75] coordinates (Used,3.718) (Jumps,0.026) (Stack,0);
[fill=rwth_blue] coordinates (Used,3.728) (Jumps,0.016) (Stack,0);

coordinates (None,3776) (Stack,3776)
Aria, Chacha20, Camellia, AES-256, SHA-256

A histogram showing what modes are mostly successful when verifying the model for different applications.

[1]
D. Sisejkovic et al., “A Secure Hardware-Software Solution Based on
RISC-V, Logic Locking and Microkernel,” ser. ACM SCOPES '20.

[2]
IEEE Council on Electronic Design Automation, “CADForAssurance,” 2020.
[Online]. Available: <https://cadforassurance.org/>

[3]
A. Ardeshiricham et al., “Register transfer level information flow
tracking for provably secure hardware design,” in DATE '2017.

[4]
M. Lipp et al., “Meltdown: Reading Kernel Memory from User Space,”
in 27th USENIX Security Symposium (USENIX Security 18), Baltimore.

[5]
O. Ruwase et al., “Parallelizing dynamic information flow tracking,”
in Proceedings of the Twentieth Annual Symposium on Parallelism in
Algorithms and Architectures, 01 2008, pp. 35–45.

[6]
A. Cimatti et al., “From Sequential Extended Regular Expressions to
NFA with Symbolic Labels,” in Implementation and Application of
Automata.1em plus 0.5em minus 0.4emSpringer Berlin Heidelberg,

[7]
F. Lugou et al., “Toward a Methodology for Unified Verification of
Hardware/Software Co-designs,” Journal of Cryptographic
Engineering, pp. 1–12, Nov. 2016.

[8]
B.-Y. Huang et al., “Formal Security Verification of Concurrent
Firmware in SoCs using Instruction-Level Abstraction for Hardware,” in
2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[9]
X. Guo et al., “Scalable SoC trust verification using integrated
theorem proving and model checking,” in 2016 IEEE International
Symposium on Hardware Oriented Security and Trust (HOST), 2016, pp.

[10]
M. D. Nguyen et al., “Formal hardware/software co-verification by
interval property checking with abstraction,” in 2011 48th
ACM/EDAC/IEEE Design Automation Conference (DAC), 2011, pp. 510–515.

[11]
D. Große et al., “HW/SW Co-Verification of Embedded Systems Using
Bounded Model Checking,” ser. GLSVLSI '06.

[12]
J. Oberg et al., “Theoretical analysis of gate level information flow
tracking,” in Design Automation Conference, 2010, pp. 244–247.

[13]
L. M. Reimann et al., “QFlow: Quantitative Information Flow for
Security-Aware Hardware Design in Verilog,” in 2021 IEEE 39th
International Conference on Computer Design (ICCD), 2021, pp. 603–607.

[14]
L. M. Reimann et al., “Quantitative information flow for hardware:
Advancing the attack landscape,” in 2023 IEEE 14th Latin America
Symposium on Circuits and System (LASCAS), 2023.

[15]
X. Guo et al., “QIF-Verilog: Quantitative Information-Flow based
Hardware Description Languages for Pre-Silicon Security Assessment,” in
IEEE HOST '19.

[16]
C. Eisner et al.,
???core.languages.en_GB???A Practical
Introduction to PSL Introduction, ser. Integrated Circuits and
Systems.1em plus 0.5em minus 0.4emSpringer, Boston, MA, 2006.

[17]
Open SSL Project, “OpenSSL,” 1998. [Online]. Available: