# Evolutionary Multi-Objective Optimisation for Fairness-Aware Self Adjusting
Memory Classifiers in Data Streams

Pivithuru Thejan Amarasinghe
La Trobe University
Melbourne
Australia
<EMAIL_ADDRESS>&Diem Pham
La Trobe University
Melbourne
Australia
Can Tho University
Can Tho
Vietnam
<EMAIL_ADDRESS>&Binh Tran
La Trobe University
Melbourne
Australia
<EMAIL_ADDRESS>&Su Nguyen
RMIT University
Melbourne
Australia
<EMAIL_ADDRESS>& Yuan Sun
La Trobe University
Melbourne
Australia
<EMAIL_ADDRESS>& Damminda Alahakoon
La Trobe University
Melbourne
Australia
<EMAIL_ADDRESS>
###### Abstract

This paper introduces a novel approach, evolutionary multi-objective
optimisation for fairness-aware self-adjusting memory classifiers, designed to
enhance fairness in machine learning algorithms applied to data stream
classification. With the growing concern over discrimination in algorithmic
decision-making, particularly in dynamic data stream environments, there is a
need for methods that ensure fair treatment of individuals across sensitive
attributes like race or gender. The proposed approach addresses this challenge
by integrating the strengths of the self-adjusting memory K-Nearest-Neighbour
algorithm with evolutionary multi-objective optimisation. This combination
allows the new approach to efficiently manage concept drift in streaming data
and leverage the flexibility of evolutionary multi-objective optimisation to
maximise accuracy and minimise discrimination simultaneously. We demonstrate
the effectiveness of the proposed approach through extensive experiments on
various datasets, comparing its performance against several baseline methods
in terms of accuracy and fairness metrics. Our results show that the proposed
approach maintains competitive accuracy and significantly reduces
discrimination, highlighting its potential as a robust solution for fairness-
aware data stream classification. Further analyses also confirm the
effectiveness of the strategies to trigger evolutionary multi-objective
optimisation and adapt classifiers in the proposed approach.

_K_ eywords fairness, multi-objective, data streams, swarm intelligence.

##  1 Introduction

In recent years, machine learning (ML) algorithms have been applied to support
decision-making across various domains such as finance and criminal Ghadami
and Epureanu [2022], Jena et al. [2020], Patil et al. [2022]. However, a
significant concern within these applications is the potential for
discrimination that affects individuals with respect to factors such as race,
gender, and socioeconomic status. For instance, healthcare providers and
health plans that rely on clinical algorithms for decision-making may
inadvertently discriminate against certain patients, resulting in some groups
losing access to healthcare services Cary Jr et al. [2023], Huang et al.
[2022]. Racial bias in ML poses a significant challenge, and addressing this
issue is essential to ensure fairness in implementing ML Mehrabi et al.
[2021].

Fairness-aware ML has gained significant attention in addressing the issue of
discrimination within algorithms. Since discrimination often arises from
sensitive features such as gender or race, techniques such as removing
sensitive attributes, applying feature selection, or assigning weights to
features to mitigate discrimination have played a crucial role in mitigating
discrimination. However, the trade-off arises because trying to reduce
discrimination can unintentionally lead to less accurate models. Therefore,
minimising discrimination and maximising accuracy can be considered as
conflicting objectives.

Swarm intelligence (SI), inspired by animal behaviours, is a subset of
evolutionary computation Mavrovouniotis et al. [2017]. SI algorithms leverage
swarm interactions for adaptation and cooperation, making them promising for
learning in dynamic environments and addressing challenges in changing
contexts. Peška et al. [2019]. SI algorithms such as Particle Swarm
Optimisation Ali and Malebary [2020] and Salp chain-based optimisation Al-
Zoubi et al. [2020] have been applied in feature weighting methods, allocating
higher weights to informative features and vice versa Niño-Adan et al. [2021].
SI has also demonstrated significant potential in multi-objective optimisation
tasks Zhou et al. [2011]. In recent work, SMPSO Nebro et al. [2009] has shown
promise in improving fairness for online classification, FOMOS Pham et al.
[2024].

Discrimination issues can occur in many ML applications, especially in
situations where data is constantly changing, like data stream scenarios
Pessach and Shmueli [2022]. While fairness in ML has received attention, there
are limited studies on fairness in streaming data. Several fairness-aware
classification algorithms, such as FAHT Zhang and Ntoutsi [2019], 2FAHT Zhang
et al. , and FAS Stream Pham et al. [2022], have been proposed for learning
streaming data. These methods aim to minimise the impact on sensitive features
during model learning. However, they require predefined constants for feature
weights (FAHT and 2FAHT) or proportions for optimising feature weights (FAS
Stream).

To overcome the limitations above, this paper proposes a new algorithm called
Evolutionary Multi-Objective Optimisation for fairness-aware Self Adjusting
Memory Classifiers (EMOSAM). The proposed algorithm combines the strength of
SAMKNN Losing et al.  in managing concept drift and the flexibility of EMO
Zhou et al. [2011], Nebro et al. [2009] in optimising feature weights and
handling multiple conflicting objectives. Specifically, SAMKNN is responsible
for streaming data classification, and SMPSO optimises feature weights that
SAMKNN use to mitigate discrimination and maximise accuracy. The main
contributions of this paper are as follows.

  * •

A new incremental classification method called EMOSAM to learn fairness-aware
SAMKNN classifiers for data stream classification.

  * •

An EMO component within EMOSAM to determine feature weights that maximise
accuracy and minimise discrimination simultaneously for SAMKNN.

  * •

Strategies to trigger EMO and adapt SAMKNN classifiers by selecting non-
dominated solutions obtained by EMO.

The remainder of the paper is organised as follows. The background and related
works are presented in Section 2. The details of our method are introduced in
Section 3.3. Section 4 shows the dataset and experimental setups for
validating our method. The conclusions and future work are discussed in
Section 5.

##  2 Background and Related Works

###  2.1 Data Stream Classification

There have been many algorithms for the classification of data streams
proposed in the literature Mehta et al. [2017]. Tree-based algorithms such as
the Hoeffding Tree Classifier (HT) Hulten et al. [2001], the Hoeffding
Adaptive Tree Classifier (HAT) [Bifet and Gavalda, 2007], and the Adaptive
Random Forest Classification Gomes et al. [2017] can quickly adapt decision
tree models to incorporate new concepts in streaming data Gomes et al. [2019].
Meanwhile, the Nearest Neighbour approach, such as k-Nearest-Neighbour (KNN),
is also widely used for learning from data streams and performs consistently
well across benchmark datasets. Ensemble-based algorithms synthesise the
results of individual learners to make final decisions, exploiting the
adaptability of multiple learners to handle diverse data distributions
Krawczyk et al. [2017]. Studies showcasing the accuracy weighted ensemble
classifier Wang et al. [2003], the hierarchical structure of the ensemble Yin
et al. [2015], You and Lin [2016], and dynamic ensemble sizes Xu and Wang
[2017] underscore the benefits of this approach. In particular, SAMKNN Losing
et al.  stands out as a state-of-the-art method that deals with changes in
data streaming using both short-term memory for current concepts and long-term
memory for past concepts.

###  2.2 Fairness-aware Machine Learning

#### Discrimination Measures

The initial phase to improve fairness within learnt models involves assessing
discrimination in ML models. Several discrimination measures, including
statistical parity, equalised odds, and absolute between-ROC area Žliobaitė
[2017], have been introduced for this purpose. Statistical parity, widely used
in numerous studies Zhang and Ntoutsi [2019], Le Quy et al. [2022], Iosifidis
et al. [2019], calculates the percentage difference between the unprotected
and protected groups when assigned a positive class ($Y=1$). In this paper, we
adopt statistical parity as the discrimination measure due to its simple
implementation and popular applications in the fairness-aware ML literature.

Consider a dataset $D$ used for binary classification tasks. Each instance in
$D$ consists of a set of features denoted by $x$, and a corresponding class
label denoted by $Y$, which can take on values of either $0$ or $1$.
Additionally, we have a sensitive feature denoted by $S$, which can take on
values from the set $\\{u,p\\}$. Here, $u$ represents the unprotected group
(e.g., females) and $p$ represents the protected group (e.g., males). The
predicted values $\hat{Y}$ are obtained from ML models. The discrimination of
a given ML model, based on the statistical parity criterion, can be calculated
as follows.

$Disc(D)=P(\hat{Y}=1\mid S=p)-P(\hat{Y}=1\mid S=u),$ (1)

where $P(\hat{Y}=1\mid S=p)$ represents the probability of the protected group
receiving a positive classification, while $P(\hat{Y}=1\mid S=u)$ denotes the
probability of the unprotected group receiving a positive classification. As a
result, the range of $Disc(D)$ is between -1 and 1. A value of 0 corresponds
to no discrimination, a value of 1 indicates discrimination against the
protected group, and a value of -1 signifies discrimination against the
unprotected group. This equation can also be used to assess discrimination in
the original dataset $D$ by replacing $\hat{Y}$ with $Y$.

#### Fairness-aware ML for Streaming Data

Fairness-aware ML techniques aim to mitigate discrimination while maintaining
the accuracy of their predictions. These techniques predominantly employ three
approaches: pre-processing, in-processing, and post-processing, which
correspond to intervening before, during, or after the model training process
to enhance fairness, respectively.

Pre-processing techniques aim to mitigate discrimination in a dataset. Two
popular methods within this approach are “massaging” and “reweighting” Kamiran
and Calders [2009], Calders et al. . The former involves relabelling data,
while the latter assigns weights to different sensitive groups. In streaming
classification, fairness enhancing interventions (FEI) adhere to a pre-
processing approach, modifying data chunks before inputting them into a
classification model Iosifidis et al. [2019]. These data modification methods
aim to mitigate discrimination.

FABBOO (Fairness-Aware Batch Based Online Oversampling) Iosifidis and Ntoutsi
[2020] is also a pre-processing technique tailored for fairness in online
learning with class imbalance. It aims to address biases by oversampling the
minority class. However, this method can lead to increased computation time.
Additionally, FABBOO relies on predefined thresholds for resampling,
potentially affecting its effectiveness.

In-processing methods adjust models by embedding fairness measurements into
their objective functions to address discrimination. These techniques focus on
mitigating the impact of sensitive features within ML models. In the decision
tree family, the adaptive fairness-aware decision tree classifier (FAHT) Zhang
and Ntoutsi [2019], 2FAHT Zhang et al. , and fairness-enhancing and concept-
adapting decision tree classifier (FEAT) Zhang and Bifet [2020] are derived
from Hoeffding Tree (HT). These models introduce novel splitting criteria that
consider discrimination during tree growth, showcasing success in mitigating
bias. However, they exhibit a trade-off between accuracy and discrimination
and rely on user input. The FAS stream Pham et al. [2022] handles
discrimination using swarm intelligence for feature weighting, but the balance
between accuracy and discrimination is user-defined.

Post-processing methods alter the decision boundary of a model or adjust its
prediction labels. Most of the methods are examined in static scenarios. Some
techniques introduce additional prediction thresholds, as demonstrated in
Hardt et al. [2016], to address discrimination, while others, such as in
Calders and Verwer [2010], modify the decision boundary of models for
fairness. These methods primarily target the output of a classifier. Adapting
these techniques to online learning poses challenges, as the evolving data
distribution in streaming data may lead to changes in the boundary or
predictions.

###  2.3 EMO in Fairness Machine Learning

Fairness ML aims to generate models that are fair and accurate. However,
previous studies Liu and Vicente [2022], Zhang et al. [2022] have shown that
fairness and accuracy are often conflicting objectives, meaning that improving
one usually comes at the expense of the other. Therefore, multi-objective
optimisation, especially evolutionary multi-objective optimisation (EMO), has
been applied to address these conflicts Padh et al. [2021], Liu and Vicente
[2022], Yu et al. [2022]. For example, Rehman et al. [2022] proposed an EMO
algorithm to select features that enhance both fairness and accuracy
objectives.Zhang et al. [2022, 2021] introduce a learning framework that uses
EMO to simultaneously optimise various metrics, including accuracy and
multiple fairness measures, within ML models. The results indicate the ability
of the EMO algorithms to discover a more diverse set of fair models using
solutions on the Pareto front to develop ensemble models with good
performance. However, these methods have been predominantly applied to a
limited set of benchmark datasets in static scenarios. In data stream
learning, FOMOS Pham et al. [2024] employs EMO with HT, automatically
optimising feature weights for fairer and more accurate trees. However, FOMOS
uses a singular strategy for leader selection from Pareto solutions and
triggers EMO solely when classifier discrimination increases. This motivates
our paper to introduce strategies for selecting solutions from the Pareto
front produced by EMO specifically tailored for streaming data.

##  3 Our Method

We introduce a novel approach called EMOSAM, which combines EMO with the SAM
algorithm to improve data stream classification. Our approach takes into
account both model accuracy and fairness (or discrimination) as objectives.
Figure 1 provides an overview of the EMOSAM approach. Similar to ensemble
methods, our algorithm maintains a set of weak learners that have different
trade-offs between model accuracy and fairness. These weak learners have
different feature weights that are evolved using an EMO algorithm. If a
significant model discrimination is detected, there is a good chance that the
previously identified feature weights are no longer suitable for the new data.
In such cases, the EMO algorithm is used to evolve a new set of feature
weights based on the discrimination trend. The remainder of this section will
provide details for each key component of EMOSAM.

Figure 1: An overview of our proposed EMOSAM algorithm for data stream
classification.

###  3.1 Weighted SAMKNN for Classification

SAMKNN, proposed by Losing et al. , is a cutting-edge approach for classifying
data streams. It uses the self-adjusting memory (SAM) architecture, which is
specifically designed to handle heterogeneous concept drift in streaming data.
SAMKNN uses long-term and short-term memory to make decisions when faced with
changes in streaming data. Short-term memory (STM) represents the current
concept, while long-term memory (LTM) preserves established knowledge as long
as it aligns with the STM. Incoming data points are stored in the STM, and a
cleaning process ensures that the LTM remains consistent with the STM. When
the size of the STM is reduced, its discarded knowledge is transferred to the
LTM. The accumulated knowledge is compressed when the available space is
exhausted. During prediction, both the STM and LTM are taken into account,
based on their past performance.

SAMKNN is known for its ability to handle concept drift, a common issue in
streaming data classification. However, one drawback of SAMKNN is that it uses
all features for predictions, which can make it susceptible to bias risks. In
order to address this limitation, our proposed method aims to enhance SAMKNN
by introducing feature weights. These feature weights are applied to each
feature to control its influence on the distance calculation for STM and LTM
in SAMKNN. The specific steps for making predictions using SAMKNN with feature
weights are outlined in Algorithm 1.

Algorithm 1 SAMKNN Prediction with Feature Weights

Input: data points $X$, feature weights $\alpha$, SAMKNN classifier
$\mathcal{M}$.

Output: the predicted class lable $\hat{y}$ for each data point $\mathbf{x}\in
X$.

Transform the data points $X$, the STM samples and LTM samples in
$\mathcal{M}$ by multiplying the feature weights $\alpha$.

for each data point $\mathbf{x}\in X$ do

$N_{1}\leftarrow$ the $k$ nearest neighbours of $\mathbf{x}$ in the STM
samples.

$N_{2}\leftarrow$ the $k$ nearest neighbours of $\mathbf{x}$ in the LTM
samples.

$N_{3}\leftarrow$ the $k$ nearest neighbours of $\mathbf{x}$ in the STM and
LTM.

$y_{1}\leftarrow$ the majority class among $N_{1}$.

$y_{2}\leftarrow$ the majority class among $N_{2}$.

$y_{3}\leftarrow$ the majority class among $N_{3}$.

$\hat{y}\leftarrow$ selected from $y_{1}$, $y_{2}$, and $y_{3}$ based on past
accuracy.

The original SAMKNN classifier can be seen as a specific case of the weighted
SAMKNN, where all feature weights are set to one. Determining the appropriate
feature weights can potentially enhance the accuracy and fairness of SAMKNN.
However, optimising these weights is challenging. In our approach, the feature
weights are initialised randomly and refined using an EMO algorithm when the
current classifier fails to mitigate discrimination.

###  3.2 Trigger for Feature Weight Optimisation

Applying EMO to optimise feature weights can be expensive, so it is not
necessary to apply it every time a new window arrives. Instead, we need to
determine the appropriate time to re-optimise the feature weights based on
discrimination. In the EMOSAM approach, we maintain a list of discrimination
values for previous windows. Within each new window, we analyse the
discrimination list ($PD$ in Algorithms 3) to determine if there is an
increasing trend in discrimination. We treat the model discriminations for
past windows as time series data and use the HP filter Hodrick and Prescott
[1997] to identify any trends in discrimination. The HP filter decomposes a
time series ($y_{t}$) into a trend component ($\Gamma_{t}$), a cyclical
component ($C_{t}$), and an error component ($\epsilon_{t}$):

$y_{t}=\Gamma_{t}+C_{t}+\epsilon_{t}.$ (2)

If a rising trend and a positive cycle are detected, we apply EMO to search
for new feature weights and reset the discrimination list. Furthermore, having
a positive cyclic component for the last value of the past discrimination
values will determine whether discrimination is starting to reduce or not. HP
filter is applied because of its robustness against randomness and noise,
which is the case in data stream classification. Validation of the HP filter’s
robustness will be further examined in the ablation study (Section 4.3).

###  3.3 SMPSO for Optimising Feature Weights

We address the issue of determining feature weights for SAMKNN as a multi-
objective optimisation problem. To optimise the feature weights, we employ the
Speed-constrained Multi-objective PSO (SMPSO) algorithm Nebro et al. [2009],
which aims to balance two conflicting objectives: (1) maximising accuracy and
(2) minimising discrimination. We choose SMPSO because it is widely recognised
as a highly effective multi-objective PSO algorithm, particularly in dynamic
optimisation problems Mavrovouniotis et al. [2017]. The success of SMPSO in
approximating Pareto fronts and its faster convergence towards these fronts,
as demonstrated in previous work Nebro et al. [2009], make it a suitable
choice for streaming learning tasks that require rapid convergence.

SMPSO starts by creating a swarm of particles that represent potential
solutions, as well as an archive of non-dominated solutions within the swarm
(i.e., solutions that are not dominated by any other solutions in terms of all
the objectives). In each iteration, the positions and velocities of the
particles are updated, and a polynomial mutation technique Deb [2011] is used
to generate new solutions. The resulting particles are then evaluated to
update both the swarm and the archive. After a specified number of iterations,
SMPSO returns the non-dominated solutions found in the archive.

Each solution or particle generated by SMPSO is a feature weight vector
($\alpha$), which will be used to calculate the distance between data points
in the SAMKNN classifier. The optimisation process aims to find the optimal
$\alpha$ that balances accuracy and discrimination. For a given number of
features $d$, a particle $\alpha$ can be represented as a $d$-dimensional
vector $\alpha=[\alpha_{1},\alpha_{2},\alpha_{3},\dots,\alpha_{d}]$, where
$\alpha_{i}$ is a real number between 0 and 1. Additionally, a particle
maintains an objective value vector $obj=[obj_{acc},obj_{disc}]$, where
$obj_{acc}$ and $obj_{disc}$ represent the accuracy and discrimination
achieved by applying $\alpha$ to the SAMKNN model as feature weights. The
discrimination $obj_{disc}$ is the absolute statistical parity in Equation
(1).

Algorithm 2 shows the process of evaluating a particle to determine its
objective values. The input to this algorithm includes the current data chunk
$D_{t}$, the position of a particle $\alpha$, and the SAMKNN classifier. In
order to assess the effectiveness of each particle, the feature weights are
incorporated into SAMKNN to generate predictions for $D_{t}$. The resulting
objective values, denoted as $obj=[obj_{acc},obj_{disc}]$, represent the
outcome of the evaluation.

Algorithm 2 Multi-objective evaluation

Input: Training data
$D_{t}=\\{(\mathbf{x}_{1},y_{1}),\dots,(\mathbf{x}_{n},y_{n})\\}$, SAMKNN
classifier $\mathcal{M}$, solution position or feature weights $\alpha$.

Output: objective values $obj=[obj_{acc},obj_{disc}]$.

Predict for $D_{t}$ using $\mathcal{M}$ with feature weights $\alpha$
(Algorithm 1).

$obj_{acc}$ $\leftarrow$ calculate accuracy of the prediction.

$obj_{disc}$ $\leftarrow$ calculate discrimination of the prediction.

###  3.4 Feature Weight Selection for SAMKNN

The selection of feature weight vectors from the Pareto front generated by
SMPSO is a challenging task, as each vector represents a unique trade-off
between model accuracy and fairness. In the EMOSAM approach, instead of
choosing a single weight vector, we consider all the weight vectors on the
Pareto front. Specifically, we utilise each weight vector to transform the
features and input them into the SAMKNN algorithm for making predictions. This
approach enables us to generate an ensemble of SAMKNNs. To determine the final
prediction of our ensemble, we employ a majority voting scheme, selecting the
prediction with the highest number of votes. The details of the selection of
the feature weight as well as other components of EMOSAM are described in
Algorithm 3.

Algorithm 3 EMOSAM Algorithm

Input: Data Stream $\mathcal{D}=\\{D_{1},\dots,D_{T}\\}$, and trend threshold
$\Phi$.

PF $\leftarrow$ randomly initialised feature weight vectors.

PD $\leftarrow$ an empty list to store past model discriminations.

for $t$ from $1$ to $T$ do

Get the features $X_{t}$ of the data chunk $t$ from $D_{t}$.

Initialise a list to store predictions: $L\leftarrow\\{\\}$.

for each feature weight vector $\alpha\in$ PF do

$\hat{Y}_{\alpha}\leftarrow$ predict for $X_{t}$ with feature weights $\alpha$
(Algorithm 1).

Add the predictions $\hat{Y}_{\alpha}$ to the list $L$.

$\hat{Y}_{t}\leftarrow$ majority voting of the predictions in $L$.

$Disc_{t}\leftarrow$ compute model discrimination based on $\hat{Y}_{t}$ and
$X_{t}$.

Add $Disc_{t}$ into the list PD and remove $Disc_{t-5}$ if exists.

Compute trend ($\Gamma_{t}$) and cycle ($C_{t}$) in PD using HP filter.

if last value of $\Gamma_{t}$ $\geq\Phi$ and last value of $C_{t}$ $>0$ then

Optimise SAMKNN feature weights using SMPSO on $D_{t}$.

PF $\leftarrow$ the Pareto front generated by SMPSO.

Remove all elements in PD. Fit SAMKNN with $D_{t}$ and update STM and LTM
samples.

##  4 Experiment and Result Analysis

###  4.1 Datasets and Experiment Design

Six classification datasets, commonly used in fairness learning Le Quy et al.
[2022], are used to examine the performance of EMOSAM. Table 1 shows the
information of each dataset, including the number of instances $(N)$,
dimension $(d)$, the sensitive feature $(SF)$, the unprotected group $(UP)$,
and the discrimination level $(Disc)$ calculated based on Equation (1).

Table 1: The datasets used in our experiments.

Dataset | N | d | SF | UP | Disc (%)
---|---|---|---|---|---
Law | 20,798 | 23 | Race | Non_White | 19.83
Credit | 30,000 | 91 | Gender | Female | 3.39
Adult | 32,561 | 36 | Gender | Female | 19.63
Bank | 41,188 | 53 | Age | Age$\leq$25 or Age$\geq$60 | 22.15
Dutch | 60,420 | 11 | Gender | Female | 29.51
Census | 284,556 | 370 | Gender | Female | 7.53

To examine the performance of EMOSAM, EMOSAM is compared with seven baseline
methods in terms of accuracy and discrimination. While HT, HAT [Bifet and
Gavalda, 2007] and SAM represent fairness-unaware methods, FAHT Zhang and
Ntoutsi [2019] and FABBOO Iosifidis and Ntoutsi [2020] represent traditional
fairness-aware methods, FOMOS Pham et al. [accepted, January 2024] and FAS
Stream Pham et al. [2022] are evolutionary fairness-aware methods. All methods
are run with default parameters in the original studies. For FOMOS, FOMOS_2obj
(maximising accuracy and minimising discrimination) is chosen to compare with
EMOSAM. As EMOSAM, FOMOS, and FAS Stream are stochastic algorithms, 30
independent runs were conducted for each method on each dataset with different
seeds.

Table 2 shows the experiment’s parameter settings. All methods utilise a
window size of 1000, a common choice in fairness learning on streaming data.
For SMPSO, the population size is set to 30, and the number of iterations is
10 to support online learning. The remaining SMPSO parameters are adopted from
the jMetalPy Benítez-Hidalgo et al. [2019].

Table 2: Parameter Settings. | Parameters | #Settings
---|---|---
All methods | $window\\_size$ | $1000$
EMOSAM | max STM size | $5000$
| Trend threshold $\phi$ | 10%
SMPSO | Population size | $30$
| Maximum iterations | $10$
| $c_{1},c_{2}$ | $1.49445$
| $w$ | $0.1$
| Leader size | $100$

###  4.2 Comparisons with Baseline Methods

Table 3: Accuracy and discrimination obtained by EMOSAM and other methods. Since all the standard deviations of accuracy are smaller than $0.01\%$, we do not display them to save space. Note that FAHT fails on the law and bank datasets. Dataset | Method | Accuracy (%) |  | Discrimination (%) |
---|---|---|---|---|---
Best | Avg | T1 | Best | Avg $\pm$ Std | T2
$\mathsf{Law}$ | HT | 89.29 |  | – | 23.72 |  | +
HAT | 89.18 |  | – | 21.06 |  | +
SAM | 88.94 |  | = | 7.22 |  | +
FABBOO | 86.56 |  | + | 0.99 |  | –
FOMOS | 89.51 | 89.28 | – | 18.96 | 19.11 $\pm$ 0.0255 | +
FAS Stream | 89.10 | 88.92 | + | 6.37 | 8.02 $\pm$ 0.0104 | +
EMOSAM | 89.32 | 89.11 |  | 2.23 | 7.22 $\pm$ 0.0152 |
$\mathsf{Credit}$ | HT | 79.41 |  | – | 1.58 |  | +
HAT | 81.19 |  | – | 1.58 |  | +
SAM | 77.27 |  | = | 0.61 |  | +
FAHT | 78.43 |  | – | 0.16 |  | –
FABBOO | 73.22 |  | + | 0.04 |  | –
FOMOS | 81.41 | 81.36 | – | 1.07 | 1.19 $\pm$ 0.0002 | +
FAS Stream | 77.53 | 77.15 | + | 0.76 | 1.13 $\pm$ 0.0020 | +
EMOSAM | 77.39 | 77.25 |  | 0.37 | 0.54 $\pm$ 0.0006 |
$\mathsf{Adult}$ | HT | 81.82 |  | – | 20.96 |  | +
HAT | 81.68 |  | – | 20.25 |  | +
SAM | 78.36 |  | + | 13.06 |  | +
FAHT | 79.42 |  | – | 16.78 |  | +
FABBOO | 74.76 |  | + | 0.33 |  | –
FOMOS | 82.22 | 82.01 | – | 17.77 | 18.23 $\pm$ 0.0019 | +
FAS Stream | 79.67 | 78.80 | = | 10.91 | 13.42 $\pm$ 0.0146 | +
EMOSAM | 78.66 | 78.65 |  | 9.44 | 11.97 $\pm$ 0.0089 |
$\mathsf{Bank}$ | HT | 88.20 |  | = | 2.87 |  | +
HAT | 88.46 |  | – | 13.55 |  | +
SAM | 89.15 |  | – | 11.93 |  | +
FABBOO | 87.10 |  | + | 2.26 |  | +
FOMOS | 88.14 | 88.08 | = | 2.36 | 2.73 $\pm$ 0.0028 | +
FAS Stream | 89.51 | 89.09 | – | 8.60 | 13.04 $\pm$ 0.0257 | +
EMOSAM | 88.66 | 88.16 |  | 0.49 | 1.79 $\pm$ 0.0067 |
$\mathsf{Dutch}$ | HT | 81.46 |  | – | 35.83 |  | +
HAT | 81.92 |  | – | 31.39 |  | +
SAM | 76.93 |  | + | 39.57 |  | +
FAHT | 73.33 |  | + | 19.37 |  | –
FABBOO | 70.86 |  | + | 0.06 |  | –
FOMOS | 81.87 | 81.55 | – | 27.35 | 28.28 $\pm$ 0.0172 | +
FAS Stream | 80.27 | 78.88 | + | 15.27 | 18.64 $\pm$ 0.0273 | –
EMOSAM | 80.25 | 79.42 |  | 22.47 | 26.87 $\pm$ 0.0256 |
$\mathsf{Census}$ | HT | 94.57 |  | – | 1.72 |  | +
HAT | 94.52 |  | – | 1.33 |  | +
SAM | 93.99 |  | + | 0.97 |  | –
FAHT | 94.28 |  | – | 3.20 |  | +
FABBOO | 88.18 |  | + | 0.07 |  | –
FOMOS | 94.55 | 94.54 | – | 1.43 | 1.48 $\pm$ 0.0001 | +
FAS Stream | 93.94 | 93.88 | + | 0.65 | 0.76 $\pm$ 0.0005 | –
EMOSAM | 94.37 | 94.22 |  | 0.90 | 1.09 $\pm$ 0.0006 |

Table 3 presents the accuracy and discrimination in percentage obtained by all
the compared methods. For stochastic methods (FOMOS, FAS, and EMOSAM), the
best and average results of 30 runs are reported. Wilcoxon statistical test is
conducted in each comparison to confirm their significance. Columns $T1$ and
$T2$ show the results of this test comparing EMOSAM accuracy and
discrimination with each baseline method, respectively. A ‘$+$’ indicates that
the accuracy/discrimination of EMOSAM is significantly better than the method
in the corresponding row, and conversely, a ‘$-$’ denotes the opposite. More
occurrences of the ‘$+$’ symbols indicate superior performance of EMOSAM. We
also highlight $T1$ and $T2$ in blue if they contain $(++)$, $(=+)$, or
$(+=)$, which means that EMOSAM dominates the corresponding method. On the
other hand, if EMOSAM is dominated by the corresponding method, i.e. $(--)$,
the cells will be highlighted in orange.

#### EMOSAM versus the fairness-unaware methods

Compared to HT, HAT, and SAM, EMOSAM’s accuracy obtained 3 ‘$+$’, 3 ‘$=$’, and
12 ‘$-$’, while its discrimination result is significantly lower in 17 cases
out of the 18 comparisons. In particular, the discrimination of EMOSAM is 16%
and 11% lower than that of HT on Law and Bank, respectively. It is worth
noting that HT, HAT, and SAM concentrate solely on improving the accuracy of
the models, while EMOSAM considers both accuracy and fairness. However, EMOSAM
still can dominate HT on one dataset and SAM on four datasets, as shown in the
blue highlighted rows.

#### EMOSAM versus the traditional fairness-aware methods

FAHT and FABBOO are popular fairness-aware methods for streaming data. The
Wilcoxon test results in Table 3 indicate that EMOSAM’s accuracy consistently
outperforms FABBOO across all datasets with 10% and 6% higher than FABBOO on
Dutch and Census, respectively. However, regarding discrimination, FABBOO wins
on all datasets except Bank. This shows a strong preference for discrimination
in FABBOO. Mixed results are shown when comparing EMOSAM with FAHT. While
EMOSAM is dominated by FAHT on Credit, EMOSAM ran successfully and achieved
competitive results on all datasets, including the two highly imbalanced
datasets (Law and Bank), where FAHT failed to run (i.e. no FAHT results for
the two datasets). Overall, the results show that EMOSAM produces prediction
models with a better balance between accuracy and discrimination than FABBOO
and FAHT.

#### EMOSAM versus evolutionary fairness-aware methods

Although FOMOS, FAS Stream, and EMOSAM employ evolutionary optimisation
methods for feature weighting to maximise accuracy and minimise
discrimination, EMOSAM dominates FAS Stream on three datasets and FOMOS on one
while not being dominated by any of these methods. The superior results of
EMOSAM confirm the contribution of the proposed multiobjective optimisation
approach to fairness-aware algorithms.

In general, the results show that EMOSAM generates fairer classifiers than the
fairness-unaware algorithms (HT, HAT, and SAM) while maintaining a better
balance between accuracy and discrimination than the fairness-aware methods.

$0$$10$$20$ $11$ $12$ $13$ (a) Law

$0$$0.5$$1$$1.5$ $18$ $20$ $22$ $24$ $26$ (b) Credit

$0$$10$$20$ $18$ $20$ $22$ $24$ (c) Adult

$5$$10$ $11$ $12$ $13$ (d) Bank

$0$$20$$40$ $20$ $25$ $30$ (e) Dutch

$0$$1$$2$$3$ $6$ $8$ $10$ $12$ HTHATSAMFAHTFABBOOFOMOSFAS StreamEMOSAM (f)
Census

Figure 2: The discrimination ($x$-axis) and error ($y$-axis) in percentage
produced by various methods on the test datasets. Our EMOSAM approach is
located on the Pareto front for five datasets and is only dominated by FAHT on
the Credit dataset.

#### Pareto dominance

To visualise the dominancy of EMOSAM over the baseline methods, we use scatter
plots (Figure 2) with model error on the $y$-axis and discrimination on the
$x$-axis. As FOMOS, FAS Stream, and EMOSAM are stochastic algorithms, the
average accuracy and discrimination results of the 30 independent runs on each
dataset are displayed in the figure. A desirable method exhibits low model
error and low discrimination, positioning methods in the bottom left corners
as favourable. Three different colours are used to highlight the results of
each group of baseline methods.

In general, Figure 2 shows that EMOSAM is located on the Pareto front for five
datasets and is only dominated by FAHT on the Credit dataset. Specifically,
EMOSAM positions towards the bottom left corner in two datasets, namely Law
and Census, while occupying a central position in the remaining four. Compared
to HT, HAT, and SAM (the green dots), EMOSAM is located on the left (lower
discrimination) in five datasets, including Law, Credit, Adult, Bank and
Dutch. However, EMOSAM is unbeneficial in model error in comparison with these
fairness-unaware methods that only focus on accuracy in their performance.
Notably, EMOSAM maintains a competitive position when contrasted with the
fairness-aware methods (blue and pink dots). For instance, in the Adult
dataset, EMOSAM is located in the middle, while FAHT, FOMOS and FAS Stream
tend to be in the lower right corner (higher discrimination and lower error).
Moreover, FABBOO is always on the upper left corner with very low
discrimination but very high error. This means EMOSAM maintains a better
balance between model error and discrimination. Overall, EMOSAM achieves
better performance than both fairness-unaware and fairness-aware methods.

(a) Model error

(b) Discrimination

(c) Processing Time

Figure 3: Comparison of weight selection methods. A positive bar means the
Majority Voting (used by EMOSAM) is better.

(a) Model error

(b) Discrimination

(c) Processing Time

Figure 4: Comparison of triggering approaches. A positive bar means the HP
filter approach (used by EMOSAM) is better.

###  4.3 Ablation Study

We further investigate the effectiveness of the two key components of our
EMOSAM method: Majority Voting for selecting feature weights and HP filter as
a trigger for optimising feature weights.

#### Feature Weight Selection

To examine the impact of the Majority Voting method on feature weight
selection used by EMOSAM, we replace it with two alternative approaches:

  * •

Random Selection, where a feature weight is selected randomly from the Pareto
front,

  * •

Knee Point, where the Knee point on the Pareto front is selected Zhang et al.
[2014].

To ensure a fair comparison, all other settings remain unchanged.

Figure 3 shows the difference between the Knee Point / Random Selection
approach and the Majority Voting method in terms of model error (3a),
discrimination (3b), and processing time (3c). A positive bar indicates that
the Majority Voting method outperforms the other approach, while a negative
bar suggests the opposite. In general, the Majority Voting approach achieves
lower error and discrimination values while requiring a similar computational
time compared to the other approaches. It is worth noting that Figure 3 uses
the same range as Figure 4.

In terms of model error, the Majority Voting method consistently achieves the
best results when compared to the other two methods. Specifically, according
to Wilcoxon signed-rank tests that utilise the mean errors from all six
datasets, our Majority Voting method significantly outperforms the Random
Selection approach with a p-value of 0.03. Although the errors produced by the
Majority Voting and Knee Point methods are not significantly different, our
method still manages to generate a slightly lower mean error of 15.53%
compared to 15.81% from the Knee Point method. Upon closer examination, it is
evident that our Majority Voting method performs exceptionally well on the
Dutch dataset, reducing the model error by 2% and 1% in comparison to the
Random Selection and Knee Point approaches, respectively.

Regarding model discrimination, our Majority Voting method consistently
generates equally good or better results than the Knee Point method,,
especially evident in the Dutch and Adult datasets. In combination with error
results, our Majority Voting method dominates the Knee Point method on the
Dutch and Adult datasets. The Random Selection method also performs well, with
a slightly lower mean discrimination rate of 8.16% compared to that of the
Majority Voting method (8.25%). However, according to the Wilcoxon test, there
is no statistically significant difference between these results.

The processing time for the three weight selection methods is similar, with a
difference of less than 3%. It is worth noting that the changes in processing
time are not as crucial as the changes in model error or discrimination, as a
3% increase in processing time is manageable. Furthermore, the Wilcoxon tests
show no statistically significant difference in the runtimes of the three
methods.

#### Weight Optimisation Trigger

To investigate the effects of the HP filter as a trigger for feature weight
optimsation, we substitute it with the following two alternative approaches
while keeping the other settings unchanged in EMOSAM:

  * •

Every Window, where feature weight optimsation is triggered in every window,
and

  * •

Previous Value, where feature weight optimsation is triggered only if the
discrimination in the current window is greater than that of the previous
window by a threshold $\theta$, which is set to 7% in our experiment.

In Figure 4, the HP filter method produces similar outcomes to the Previous
Value approach in terms of model error, discrimination, and runtime, as
evidenced by the Wilcoxon tests across the six datasets. However, since our HP
filter method considers the trend of multiple windows, it is more robust and
less sensitive to randomness than the Previous Value approach. Considering a
scenario where the model discrimination consistently increases in each window
but the difference is below the threshold $\theta$, the Previous Value method
will not trigger feature weight optimisation, even when the discrimination
reaches 100%.

In comparison to optimising feature weights in every window, our HP filter
method generates slightly larger model errors and discrimination rates but
with no statistically significant difference, as confirmed by the Wilcoxon
signed rank tests. However, our HP filter method requires significantly less
runtime. It is worth noting that the Every Window approach can be considered
the ideal scenario for our EMOSAM for producing model errors and
discrimination rates. This outcome implies that our HP filter effectively
identifies the appropriate windows for optimising feature weights when
necessary, while still maintaining a manageable runtime.

To further demonstrate the effectiveness of the HP filter, we present the
results of feature weight optimisation on the Dutch dataset in Figure 5. The
figure shows the accuracy and discrimination of EMOSAM and SAM over time, with
red arrows indicating when feature weight optimisation is triggered. It can be
seen that when the discrimination trend increases, such as in Window 2, the HP
filter successfully triggers the optimisation of feature weights to reduce
discrimination in the subsequent windows. Interestingly, this also leads to an
improvement in model accuracy, as it is one of the optimisation objectives. As
a result, EMOSAM achieves a lower discrimination rate and higher accuracy
compared to SAM over the 30 windows, and the difference is more significant
immediately after applying feature weight optimisation. In the Credit and
Census datasets, where the model discrimination is small, there is no need to
consider discrimination, and therefore, feature weight optimisation is never
triggered, resulting in EMOSAM degenerating to SAM, as shown in Figure 2. We
have also tested multiple values $\\{5\%,10\%,15\%\\}$ for the trend threshold
$\Phi$ of the HP filter and observed that it does not significantly change the
results. This further demonstrates the robustness of our HP filter method.

(a) Discrimination (b) Accuracy

Figure 5: The effect of feature weight optimisation on the Dutch dataset over
the first 30 windows.

##  5 Conclusion

This study presented an innovative evolutionary multi-objective optimisation
(EMO) approach aimed at improving both fairness and accuracy in data stream ML
models. We introduced EMOSAM, a novel incremental classification method
designed to develop fairness-aware SAMKNN classifiers for stream
classification. In EMOSAM, an EMO component optimises feature weights,
effectively balancing accuracy and discrimination.

The experiment results on six datasets show that EMOSAM effectively and
efficiently balances the conflicting objectives to dominate the baseline
methods in many cases. The ablation study further validated our proposed
strategies for EMO triggering and the selection of non-dominated solutions.
The activation of the EMO component can consistently enhance the
classification performance of SAMKNN over time, while the majority voting
strategy proposed in this paper can significantly improve both the accuracy
and fairness of the classification model.

Moving forward, our research will focus on several key areas. We plan to
investigate advanced EMO algorithms for a better balance between fairness and
accuracy, with an emphasis on adapting to dynamic data streams. Additionally,
we will explore the scalability of EMOSAM across larger and more varied
datasets to assess its robustness in different scenarios. Integrating
explainability into EMOSAM will also be a priority, enhancing model
transparency and understanding of fairness and accuracy determinants. Lastly,
we aim to include more comprehensive fairness criteria and metrics, broadening
our approach to fairness in machine learning. These steps are crucial for
advancing fair and accurate machine learning, especially in the complex world
of data streams.

## References

  * Al-Zoubi et al. [2020] Ala’ M Al-Zoubi, Ali Asghar Heidari, Maria Habib, Hossam Faris, Ibrahim Aljarah, and Mohammad A Hassonah.  Salp chain-based optimization of support vector machines and feature weighting for medical diagnostic information systems.  _Evolutionary machine learning techniques: algorithms and applications_ , pages 11–34, 2020.
  * Ali and Malebary [2020] Waleed Ali and Sharaf Malebary.  Particle swarm optimization-based feature weighting for improving intelligent phishing website detection.  _IEEE Access_ , 8:116766–116780, 2020.
  * Benítez-Hidalgo et al. [2019] Antonio Benítez-Hidalgo, Antonio J. Nebro, José García-Nieto, Izaskun Oregi, and Javier Del Ser.  jmetalpy: A python framework for multi-objective optimization with metaheuristics.  _Swarm and Evolutionary Computation_ , 51:100598, 2019.  ISSN 2210-6502.
  * Bifet and Gavalda [2007] Albert Bifet and Ricard Gavalda.  Learning from time-changing data with adaptive windowing.  In _Proceedings of the 2007 SIAM international conference on data mining_ , pages 443–448. SIAM, 2007.
  * Calders and Verwer [2010] Toon Calders and Sicco Verwer.  Three naive bayes approaches for discrimination-free classification.  _Data mining and knowledge discovery_ , 21:277–292, 2010.
  * [6] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy.  Building classifiers with independency constraints.  In _2009 IEEE International Conference on Data Mining Workshops_ , pages 13–18.
  * Cary Jr et al. [2023] Michael P Cary Jr, Anna Zink, Sijia Wei, Andrew Olson, Mengying Yan, Rashaud Senior, Sophia Bessias, Kais Gadhoumi, Genevieve Jean-Pierre, Demy Wang, et al.  Mitigating racial and ethnic bias and advancing health equity in clinical algorithms: A scoping review: Scoping review examines racial and ethnic bias in clinical algorithms.  _Health Affairs_ , 42(10):1359–1368, 2023.
  * Deb [2011] Kalyanmoy Deb.  Multi-objective optimisation using evolutionary algorithms: an introduction.  In _Multi-objective evolutionary optimisation for product design and manufacturing_ , pages 3–34. Springer, London, 2011.
  * Ghadami and Epureanu [2022] Amin Ghadami and Bogdan I Epureanu.  Data-driven prediction in dynamical systems: recent developments.  _Philosophical Transactions of the Royal Society A_ , 380(2229):20210213, 2022.
  * Gomes et al. [2017] Heitor M Gomes, Albert Bifet, Jesse Read, Jean Paul Barddal, Fabrício Enembreck, Bernhard Pfharinger, Geoff Holmes, and Talel Abdessalem.  Adaptive random forests for evolving data stream classification.  _Machine Learning_ , 106(9):1469–1495, 2017.
  * Gomes et al. [2019] Heitor Murilo Gomes, Jesse Read, Albert Bifet, Jean Paul Barddal, and João Gama.  Machine learning for streaming data: state of the art, challenges, and opportunities.  _ACM SIGKDD Explorations Newsletter_ , 21(2):6–22, 2019.
  * Hardt et al. [2016] Moritz Hardt, Eric Price, and Nati Srebro.  Equality of opportunity in supervised learning.  _Advances in neural information processing systems_ , 29, 2016.
  * Hodrick and Prescott [1997] Robert J Hodrick and Edward C Prescott.  Postwar us business cycles: an empirical investigation.  _Journal of Money, credit, and Banking_ , pages 1–16, 1997.
  * Huang et al. [2022] Jonathan Huang, Galal Galal, Mozziyar Etemadi, and Mahesh Vaidyanathan.  Evaluation and mitigation of racial bias in clinical machine learning models: scoping review.  _JMIR Medical Informatics_ , 10(5):e36388, 2022.
  * Hulten et al. [2001] Geoff Hulten, Laurie Spencer, and Pedro Domingos.  Mining time-changing data streams.  In _Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 97–106, 2001.
  * Iosifidis and Ntoutsi [2020] Vasileios Iosifidis and Eirini Ntoutsi.  -online fairness-aware learning under class imbalance.  In _International Conference on Discovery Science_ , pages 159–174. Springer, 2020.
  * Iosifidis et al. [2019] Vasileios Iosifidis, Thi Ngoc Han Tran, and Eirini Ntoutsi.  Fairness-enhancing interventions in stream classification.  In _International Conference on Database and Expert Systems Applications_ , pages 261–276, 2019.
  * Jena et al. [2020] Monalisa Jena, Ranjan Kumar Behera, and Santanu Kumar Rath.  Machine learning models for stock prediction using real-time streaming data.  In _International Conference on Biologically Inspired Techniques in Many-Criteria Decision Making_ , pages 101–108. Springer, 2020.
  * Kamiran and Calders [2009] Faisal Kamiran and Toon Calders.  Classifying without discriminating.  In _The 2nd international conference on computer, control and communication_ , pages 1–6, 2009.
  * Krawczyk et al. [2017] Bartosz Krawczyk, Leandro L Minku, João Gama, Jerzy Stefanowski, and Michał Woźniak.  Ensemble learning for data stream analysis: A survey.  _Information Fusion_ , 37:132–156, 2017.
  * Le Quy et al. [2022] Tai Le Quy, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi.  A survey on datasets for fairness-aware machine learning.  _Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery_ , page e1452, 2022.
  * Liu and Vicente [2022] Suyun Liu and Luis Nunes Vicente.  Accuracy and fairness trade-offs in machine learning: A stochastic multi-objective approach.  _Computational Management Science_ , 19(3):513–537, 2022.
  * [23] Viktor Losing, Barbara Hammer, and Heiko Wersing.  Knn classifier with self-adjusting memory for heterogeneous concept drift.  In _2016 IEEE 16th International Conference on data mining (ICDM)_ , pages 291–300.
  * Mavrovouniotis et al. [2017] Michalis Mavrovouniotis, Changhe Li, and Shengxiang Yang.  A survey of swarm intelligence for dynamic optimization: Algorithms and applications.  _Swarm and Evolutionary Computation_ , 33:1–17, 2017.
  * Mehrabi et al. [2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  _ACM computing surveys (CSUR)_ , 54(6):1–35, 2021.
  * Mehta et al. [2017] Shikha Mehta et al.  Concept drift in streaming data classification: algorithms, platforms and issues.  _Procedia computer science_ , 122:804–811, 2017.
  * Nebro et al. [2009] Antonio J Nebro, Juan José Durillo, Jose Garcia-Nieto, CA Coello Coello, Francisco Luna, and Enrique Alba.  Smpso: A new pso-based metaheuristic for multi-objective optimization.  In _2009 IEEE Symposium on computational intelligence in multi-criteria decision-making (MCDM)_ , pages 66–73. IEEE, 2009.
  * Niño-Adan et al. [2021] Iratxe Niño-Adan, Diana Manjarres, Itziar Landa-Torres, and Eva Portillo.  Feature weighting methods: A review.  _Expert Systems with Applications_ , 184:115424, 2021.
  * Padh et al. [2021] Kirtan Padh, Diego Antognini, Emma Lejal-Glaude, Boi Faltings, and Claudiu Musat.  Addressing fairness in classification with a model-agnostic multi-objective algorithm.  In _Uncertainty in artificial intelligence_ , pages 600–609. PMLR, 2021.
  * Patil et al. [2022] Rahul Patil, Pramod D Patil, Sayali Kanase, Nikita Bhegade, Vaishnavi Chavan, and Shreyas Kashetwar.  System for analyzing crime news by mining live data streams with preserving data privacy.  In _Sentimental Analysis and Deep Learning: Proceedings of ICSADL 2021_ , pages 799–811. Springer, 2022.
  * Peška et al. [2019] Ladislav Peška, Tsegaye Misikir Tashu, and Tomáš Horváth.  Swarm intelligence techniques in recommender systems-a review of recent research.  _Swarm and Evolutionary Computation_ , 48:201–219, 2019.
  * Pessach and Shmueli [2022] Dana Pessach and Erez Shmueli.  A review on fairness in machine learning.  _ACM Computing Surveys (CSUR)_ , 55(3):1–44, 2022.
  * Pham et al. [2022] Diem Pham, Binh Tran, Su Nguyen, and Damminda Alahakoon.  Fairness aware swarm-based machine learning for data streams.  In _Australasian Joint Conference on Artificial Intelligence_ , pages 205–219. Springer, 2022.
  * Pham et al. [2024] Diem Pham, Binh Tran, Su Nguyen, Damminda Alahakoon, and Mengjie Zhang.  Fairness optimisation with multi-objective swarms for explainable classifiers on data streams.  _Complex & Intelligent Systems_, pages 1–14, 2024.
  * Pham et al. [accepted, January 2024] Diem Pham, Binh Tran, Su Nguyen, Damminda Alahakoon, and Mengjie Zhang.  Fairness optimisation with multi-objective swarms for explainable classifiers on data streams.  accepted, January 2024.
  * Rehman et al. [2022] Ayaz Ur Rehman, Anas Nadeem, and Muhammad Zubair Malik.  Fair feature subset selection using multiobjective genetic algorithm.  In _Proceedings of the Genetic and Evolutionary Computation Conference Companion_ , pages 360–363, 2022.
  * Wang et al. [2003] Haixun Wang, Wei Fan, Philip S Yu, and Jiawei Han.  Mining concept-drifting data streams using ensemble classifiers.  In _Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 226–235, 2003.
  * Xu and Wang [2017] Shuliang Xu and Junhong Wang.  Dynamic extreme learning machine for data stream classification.  _Neurocomputing_ , 238:433–449, 2017.
  * Yin et al. [2015] Xu-Cheng Yin, Kaizhu Huang, and Hong-Wei Hao.  De2: dynamic ensemble of ensembles for learning nonstationary data.  _Neurocomputing_ , 165:14–22, 2015.
  * You and Lin [2016] Sheng-Chi You and Hsuan-Tien Lin.  A simple unlearning framework for online learning under concept drifts.  In _Advances in Knowledge Discovery and Data Mining: 20th Pacific-Asia Conference, PAKDD 2016, Auckland, New Zealand, April 19-22, 2016, Proceedings, Part I 20_ , pages 115–126. Springer, 2016.
  * Yu et al. [2022] Guo Yu, Lianbo Ma, Wei Du, Wenli Du, and Yaochu Jin.  Towards fairness-aware multi-objective optimization.  _arXiv preprint arXiv:2207.12138_ , 2022.
  * Zhang et al. [2021] Qingquan Zhang, Jialin Liu, Zeqi Zhang, Junyi Wen, Bifei Mao, and Xin Yao.  Fairer machine learning through multi-objective evolutionary learning.  In _International conference on artificial neural networks_ , pages 111–123. Springer, 2021.
  * Zhang et al. [2022] Qingquan Zhang, Jialin Liu, Zeqi Zhang, Junyi Wen, Bifei Mao, and Xin Yao.  Mitigating unfairness via evolutionary multi-objective ensemble learning.  _IEEE Transactions on Evolutionary Computation_ , 2022.
  * Zhang and Bifet [2020] Wenbin Zhang and Albert Bifet.  Feat: A fairness-enhancing and concept-adapting decision tree classifier.  In _International Conference on Discovery Science_ , pages 175–189, 2020.
  * Zhang and Ntoutsi [2019] Wenbin Zhang and Eirini Ntoutsi.  FAHT: An adaptive fairness-aware decision tree classifier.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)_ , pages 1480–1486, 2019.
  * [46] Wenbin Zhang, Mingli Zhang, Ji Zhang, Zhen Liu, Zhiyuan Chen, Jianwu Wang, Edward Raff, and Enza Messina.  Flexible and adaptive fairness-aware learning in non-stationary data streams.  In _2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)_ , pages 399–406.
  * Zhang et al. [2014] Xingyi Zhang, Ye Tian, and Yaochu Jin.  A knee point-driven evolutionary algorithm for many-objective optimization.  _IEEE Transactions on Evolutionary Computation_ , 19(6):761–776, 2014.
  * Zhou et al. [2011] Aimin Zhou, Bo-Yang Qu, Hui Li, Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang.  Multiobjective evolutionary algorithms: A survey of the state of the art.  _Swarm and evolutionary computation_ , 1(1):32–49, 2011.
  * Žliobaitė [2017] Indrė Žliobaitė.  Measuring discrimination in algorithmic decision making.  _Data Mining and Knowledge Discovery_ , 31(4):1060–1089, 2017.


# Variable Ion Compositions of Solar Energetic Particle Events in the Inner
Heliosphere:
A Field-line Braiding Model with Compound Injections

Fan Guo Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM
87545, USA  Lulu Zhao Department of Climate and Space Sciences and
Engineering, University of Michigan, Ann Arbor, MI, 48109, USA  Christina M.
S. Cohen California Institute of Technology, Pasadena, CA 91125, USA  Joe
Giacalone University of Arizona, Tucson, AZ 85721, USA  R. A. Leske California
Institute of Technology, Pasadena, CA 91125, USA  M. E. Wiedenbeck Jet
Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109,
USA  S. W. Kahler Air Force Research Laboratory, Space Vehicles Directorate,
3550 Aberdeen Ave., Kirtland AFB, NM 87117, USA  Xiaocan Li Dartmouth College,
Hanover, NH 03750, USA  Qile Zhang Theoretical Division, Los Alamos National
Laboratory, Los Alamos, NM 87545, USA  George C. Ho The Johns Hopkins
University, Applied Physics Laboratory, Laurel MD 20723, USA  Mihir I. Desai
Department of Physics and Astronomy, University of Texas at San Antonio, San
Antonio, TX 78249, USA  Southwest Research Institute, San Antonio, TX 78249,
USA

###### Abstract

We propose a model for interpreting highly variable ion composition ratios in
solar energetic particles (SEP) events recently observed by Parker Solar Probe
(PSP) at $0.3-0.45$ astronomical unit. We use numerical simulations to
calculate SEP propagation in a turbulent interplanetary magnetic field with a
Kolmogorov power spectrum from large scale down to the gyration scale of
energetic particles. We show that when the source regions of different species
are offset by a distance comparable to the size of the source regions, the
observed energetic particle composition He/H can be strongly variable over
more than two orders of magnitude, even if the source ratio is at the nominal
value. Assuming a 3He/4He source ratio of $10\%$ in impulsive 3He-rich events
and the same spatial offset of the source regions, the 3He/4He ratio at
observation sites also vary considerably. The variability of the ion
composition ratios depends on the radial distance, which can be tested by
observations made at different radial locations. We discuss the implication of
these results on the variability of ion composition of impulsive events and on
further PSP and Solar Orbiter observations close to the Sun.

Solar energetic particles, Interplanetary turbulence, Interplanetary magnetic
fields

##  1 Introduction

The acceleration and transport of energetic charged particles is a remarkable
problem in heliophysics and astrophysics. While much progress has been made,
many aspects of observations remain mysterious. Different acceleration and
transport effects are often entangled when energetic particles are observed,
making it challenging to distinguish these effects and evaluate their relative
importance. The Parker Solar Probe (PSP) and Solar Orbiter (SO) provide unique
opportunities for studying energetic particles closer to the Sun, where
interplanetary scattering, among other effects, is substantially reduced
compared to observations made at 1 AU (Desai & Giacalone, 2016; McComas et
al., 2016; Rodríguez-Pacheco et al., 2020).

During the first several orbits, PSP has observed a number of small Solar
Energetic Particle (SEP) events at $0.17$ \- $0.5$ AU. These have provided
much insight yet present new puzzles to particle acceleration and transport
processes. Especially, Cohen et al. (2021) reported that the helium to
hydrogen ratio (He/H)111We refer 4He as He throughout the paper unless we
specifically discuss their isotopes. showed substantial event-to-event
variation, $0.33\%\pm 0.13\%$, $1.6\%\pm 0.9$% and $17.6\%\pm 4.7\%$ for three
SEP events originating from the same active region, whereas the nominal ratio
measured at 1 AU is about $3.6\%$ (Reames, 1995). An earlier SEP event showed
a He/H ratio of $\sim 5.2\%$ (Leske et al., 2020). Variations of 3He/4He,
O/He, electron/proton and other heavy ions are also evident in the first SEP
events observed by PSP (Leske et al., 2020; Wiedenbeck et al., 2020; Cohen et
al., 2021). Using measurements at 1 AU, Reames (2019) reported a reduced He/C
ratio and possibly He/H ratio by more than one order of magnitude in some
events. It was also noted that these helium-poor events are among the smaller
impulsive events, indicating an event-size dependent variation. Kahler & Brown
(2021) reported gradual SEP events that show large variations of He/H ratios.
They find that the He/H ratios in the gradual SEP events are correlated with
that in the solar wind. While Reames (2019) suggested that these He-poor
events are due to the first ionization potential (FIP), it is interesting to
note that some of the reported events have He/C up to four times of the
“nominal” ratio in impulsive SEP events (Reames, 1995). The FIP effect may
generate a He reduction by several times (Laming, 2021), but has difficulties
in explaining the variation of He/H over nearly two orders of magnitude
observed by PSP. A relevant observation by Advanced Composition Explorer (ACE)
showed that in some events the 22Ne/20Ne ratio – with nearly the same charge
to mass ratio – has a three-fold event-to-event variation (0.7-2 times of the
solar wind value) (Leske et al., 1999b). Wiedenbeck et al. (2008) have
reported the variation over a number of heavy ion species and noted that heavy
ion isotope ratios (e.g., 26Mg/24Mg and 22Ne/20Ne) are more variable in 3He-
rich events than in large gradual SEP events. Together, these observations
suggested a systematic variability over a wide range of species, which has not
been explored much in either observation or theory (Leske et al., 1999a; Mason
et al., 1979). The ongoing PSP and SO missions provide a unique opportunity to
study the variable ion composition in SEP events. Here we propose a scenario,
where the observed variability of ion composition is due to the propagation of
SEPs in turbulent interplanetary magnetic fields from compound injections at
the source regions.

It has been well established that the transport of energetic particles in the
inner heliosphere is governed by magnetic turbulence (Jokipii, 1971; Zank,
2014). At small scales, particles resonantly interact with turbulence and get
scattered in their pitch angles (Jokipii, 1966, 1971), whereas at large
scales, magnetic field lines of force braid and meander, leading to a
transverse diffusion to the mean magnetic field (Jokipii & Parker, 1969;
Giacalone & Jokipii, 1999; Matthaeus et al., 2003). Observations of the
dropouts, namely sharp changes of energetic particle fluxes simultaneous at
all energies, indicate diffusion transverse to the local magnetic field is
weak (Mazur et al., 2000; Giacalone et al., 2000; Chollet et al., 2009;
Chollet & Giacalone, 2011; Guo & Giacalone, 2014). However, the particle can
still move transverse to the mean magnetic field through the field line
meandering (Giacalone et al., 2000; Guo & Giacalone, 2014). This perpendicular
diffusion is likely required to explain large longitudinal spread of SEPs
(Dresing et al., 2012; Giacalone & Jokipii, 2012; Wiedenbeck et al., 2013;
Zhao & Zhang, 2018). While these physical processes have been known, their
observational consequence has not been fully explored.

In this paper, we propose a model to understand the recent SEP events observed
by PSP. The model solves the propagation of SEPs instantaneously released at
compact sources in synthetic solar wind magnetic turbulence (Matthaeus et al.,
1990; Giacalone & Jokipii, 1999; Guo & Giacalone, 2014). We find that when the
spatial locations of source regions of different species (H, 4He, 3He) are
offset by a distance comparable to the radii of the source regions, the
observed energetic particle composition at different locations in
interplanetary space can vary significantly from the average compositions at
the combined source region. This spatial offset is more established for
energetic protons versus energetic electrons, as indicated by X-ray and
$\gamma$-ray observations (Hurford et al., 2003, 2006) and interplanetary
energetic particle observation (Chollet et al., 2009), but may exist between
other species as well. These compositional variations are stronger when
observed closer to the Sun, making PSP and SO unique for studying them more
systematically in the future. The rest of this Letter is organized as follows:
Section 2 briefly introduces the numerical model and parameters we use.
Section 3 discusses simulation results especially the effects of turbulent
magnetic fields on the slightly offset ion sources. More discussion and
further implications on future observations closer to the Sun by PSP and SO
are presented in Section 4.

##  2 Numerical Simulations for Charged Particle Propagation in the
interplanetary turbulence

We consider the propagation of SEPs (protons, 4He, and 3He) released from
spatially compact and instantaneous sources in turbulent solar wind magnetic
fields. We numerically integrate gyromotions of an ensemble of energetic
charged particles in an advecting, synthetic turbulent magnetic field similar
to that in the solar wind. The method we use is similar to our earlier work
(Giacalone & Jokipii, 1999; Guo & Giacalone, 2014, 2015). Here we lay out the
salient details of the model. We consider the propagation of SEPs from source
regions with small spatial offsets, as illustrated in Fig. 1 a). For
simplicity we only consider one turbulence model, namely the so-called
composite turbulence model (Matthaeus et al., 1990), which has been often used
to study energetic particle transport (e.g., Giacalone & Jokipii, 1999; Qin et
al., 2002). While some detailed differences between different turbulence
models have been noted before (Guo & Giacalone, 2014), they are not
particularly important for the purpose of the current study, and therefore we
defer a comparison among different turbulence models to a future work.

###  2.1 Turbulent Interplanetary Magnetic Field

The turbulent magnetic field in our model is in a three-dimensional Cartesian
geometry ($x,y,z$)

B $\displaystyle=$ $\displaystyle\textbf{B}_{0}+\delta\textbf{B}$ (1)

The fluctuating component is generated by summing over a range of plane wave
modes with a randomly distributed propagation direction, and with randomized
polarizations and phases. This is given by

$\displaystyle\delta\textbf{B}$ $\displaystyle=$
$\displaystyle\sum^{N_{m}}_{n=1}A_{n}\hat{\xi}_{n}\exp(ik_{n}z_{n}^{\prime}+i\beta_{n}),$
(2)

where

$\displaystyle\hat{\xi}_{n}=\cos\alpha_{n}\hat{x}_{n}^{\prime}+i\sin\alpha_{n}\hat{y}_{n}^{\prime},$
(3)

and

$\begin{pmatrix}\begin{matrix}x^{\prime}\\\ y^{\prime}\\\
z^{\prime}\end{matrix}\end{pmatrix}=\begin{pmatrix}\begin{matrix}\cos\theta_{n}\cos\phi_{n}&\cos\theta_{n}\sin\phi_{n}&-\sin\theta_{n}\\\
-\sin\phi_{n}&\cos\phi_{n}&0\\\
\sin\theta_{n}\cos\phi_{n}&\sin\theta_{n}\sin\phi_{n}&\cos\theta_{n}\end{matrix}\end{pmatrix}\begin{pmatrix}\begin{matrix}x\\\
y\\\ z\end{matrix}\end{pmatrix}.$

where $\beta_{n}$ is the phase of each wave mode $n$ with wavenumber $k_{n}$,
$A_{n}$ is its amplitude, $\alpha_{n}$ is the polarization angle, and
$\phi_{n}$ and $\theta_{n}$ determine the direction of the $k$-vector.
$\beta_{n}$, $\alpha_{n}$, and $\phi_{n}$ are random numbers between $0$ and
$2\pi$. This assumes a globally uniform background magnetic field
$\textbf{B}_{0}$ along a certain direction and a fluctuating magnetic field
component $\delta\textbf{B}$. We assume the background magnetic field is along
the $z$ direction. We use the composite model, which is a magnetostatic model
for the wave-vector spectrum of magnetic fluctuations based on observations of
the solar wind turbulence (Matthaeus et al., 1990). The fluctuating magnetic
field in the model is expressed as the sum of two parts: a slab component
$\delta B^{s}=(B_{x}^{s}(z),B_{y}^{s}(z),0)$ and a two-dimensional component
$\delta B^{2D}=(B_{x}^{2D}(x,y),B_{y}^{2D}(x,y),0)$. The two-dimensional
component only consists of magnetic fluctuation with wave vectors along the
transverse direction $x$ and $y$ ($\theta_{n}=\pi/2$ and $\alpha_{n}=\pi/2$),
whereas the slab component is a one-dimensional magnetic fluctuation where all
wave vectors are along the direction of the uniform magnetic field $z$
($\theta_{n}=0$). Solar wind observations have suggested that the magnetic
field fluctuation has components with wave vectors either nearly parallel or
perpendicular to the magnetic field, and more wave power is concentrated in
the perpendicular directions (about $80\%$) (Bieber et al., 1996). In the
model we consider, the wave propagation is not included and the time
dependence is only through the solar wind advection as the magnetic field is
frozen in the background plasma. This model captures the anisotropic
characteristic of the solar wind turbulence. We construct the magnetic
fluctuations, using the random phase approximation and assuming a Kolmogorov-
like magnetic power spectrum approximated by a large number of wave modes. The
slab component $\delta\textbf{B}^{s}$ and two-dimensional component
$\delta\textbf{B}^{2D}$ can be expressed as the real parts of the following
equations:

$\displaystyle\delta\textbf{B}^{s}$ $\displaystyle=$
$\displaystyle\sum^{N_{m}}_{n=1}A_{n}[\cos\alpha_{n}(\cos\phi_{n}\hat{x}+\sin\phi_{n}\hat{y})$
$\displaystyle+i\sin\alpha_{n}(-\sin\phi_{n}\hat{x}+\cos\phi_{n}\hat{y})]$
$\displaystyle\times\exp(ik_{n}z+i\beta_{n}),$

and,

$\displaystyle\delta\textbf{B}^{2D}$ $\displaystyle=$
$\displaystyle\sum^{N_{m}}_{n=1}A_{n}i(-\sin\phi_{n}\hat{x}+\cos\phi_{n}\hat{y})$
$\displaystyle\times\exp[ik_{n}(\cos\phi_{n}x+\sin\phi_{n}y)+i\beta_{n}],$

The amplitude of magnetic fluctuation at wave number $k_{n}$ follows a
Kolomogorov-like power law:

$\displaystyle A_{n}^{2}=\sigma^{2}\frac{\Delta
V_{n}}{1+(k_{n}L_{c})^{\gamma}}\left[\sum^{N_{m}}_{n=1}\frac{\Delta
V_{n}}{1+(k_{n}L_{c})^{\gamma}}\right]^{-1},$ (6)

where $\sigma^{2}=\delta B^{2}/B_{0}^{2}$ is the magnetic wave variance and
$\Delta V$ is a normalization factor $\Delta V=\Delta k_{n}$ for the 1D slab
fluctuations and $\Delta V=2\pi k_{n}\Delta k_{n}$ for the 2D fluctuations.
$\gamma=5/3$ for the slab component and $8/3$ for the 2D component,
respectively. A logarithmic spacing in wavenumbers $k_{n}$ is chosen so that
$\Delta k_{n}/k_{n}$ is a constant. In our simulations, we use parameters
similar to what is observed in the solar wind. The correlation length is taken
to be $L_{c}=0.01$ AU. The maximum and minimum wavelengths used to generate
the turbulence are $\lambda_{max}=0.3$ AU and $\lambda_{min}=5\times 10^{-5}$
AU, respectively. We choose $\Delta k_{n}/k_{n}=0.02$. These include the full
scale of magnetic fluctuation from the scales larger than $L_{c}$ down to the
scales that resonantly interact with particles. The variance of turbulence is
taken to be $\sigma^{2}=0.15$, and, after the above implementation we find the
constructed fluctuation has a variance slightly differs from the assigned
value by several percent. The mean magnetic field $B_{0}$ is taken to be $15$
nT, consistent with observations at PSP at 0.3 AU. The solar wind speed is set
to be $400$ km/s. These parameters are taken to not vary with distance from
the Sun because of our assumed geometry. Since the model assumes a uniform
solar wind speed in Cartesian coordinates, it does not include the effects of
an expanding solar wind such as adiabatic cooling, adiabatic focusing, and
drift motion in a Parker spiral magnetic field (e.g., Dalla et al., 2017).
Nevertheless, the model is sufficient to study the proposed physical processes
on the ion composition in SEP events. In the future, it would be interesting
to consider more realistic geometry and spatial dependence of background
solar-wind plasma parameters, magnetic field and their fluctuations, as they
may vary considerably in latitudinal and lonitudinal directions (e.g.,
Giacalone et al., 2000; Zhao et al., 2020).

###  2.2 Energetic Particle Motions

In the simulations, energetic particles are injected in source regions at
$z=0$. This is done by initializing upward propagating particles $v_{z}>0$
with a randomized pitch angle and gyrophase. The released particles follow a
power-law velocity distribution with $f\propto v^{-4}$ corresponding to
energies from $0.5$ MeV/nucleon to $100$ MeV/nucleon. After the injection the
equations of motions of particles are numerically integrated using the
Burlirsh-Stoer method (Press et al., 1986). The algorithm uses an adaptive
time step based on an evaluation of the local truncation error. It is very
efficient when the fields are fairly smooth compared with the particle
gyroradius. The energy is conserved to a high accuracy, with root of the mean
square difference of the total energy ranges from 0.045% (0.5 MeV/nucleon) to
0.01% (100 MeV/nucleon) during the simulation with no solar wind flow (zero
electric field). The charged particles are released impulsively and their
trajectories are numerically integrated until they reach boundaries at $z=0.5$
AU and $z=-0.03$ AU. The spacecraft observations at $0.3$ AU are mimicked by
collecting particles in windows of a dimension $L_{x}\times L_{y}=0.01\times
0.01$ AU2 when the particles pass the windows at $z=0.3$ AU. These collection
regions are uniformly distributed in the x and y directions ($-0.5$ AU < x <
$0.5$ AU, $-0.5$ AU < y < $0.5$ AU). The size of the collection regions is
about the same size as a correlation length $L_{c}$, which is smaller than the
typical scale of the field line random walk when the distance is much larger
than $L_{c}$ (e.g., Giacalone & Jokipii, 1999). We have also tested smaller
collection windows and the results are consistent with those presented in this
paper. We also include a case where particles are tracked until observers at 1
AU to study the radial evolution of ion composition. The collected particle
energies and arrival times in each window are analyzed as a spacecraft
observation. We release different ion species (H, 4He, and 3He) from the
source regions (4He$-$H simulations are made separately from 3He$-^{4}$He
simulations). The source regions are initially chosen to be circles at the
$z=0$ plane with a radius of $R_{0}=0.2L_{c}$ and the offset distance in the x
direction $D_{0}=R_{0}=0.2L_{c}$, is smaller than the correlation scale.
Considering that the supergranulation motion scale at the Sun is $\sim 30,000$
km and roughly corresponds to the correlation length (Rincon & Rieutord,
2018), this approximately corresponds to a source region of a diameter about
$12,000$ km on the Sun. We study how the offset distance and source size
influence the observations at $0.3$ AU, similar to what PSP can observe. We
examine different offset distances $D=0$, $0.5R_{0}$, and $2.0R_{0}$ and
different source sizes $R_{source}=0.5R_{0}$, $2.0R_{0}$ and $4.0R_{0}$. For
each run, we release about $8$ million particles for each species to get a
sufficient number of particles collected in the windows and then the ion
compositions of different species are calculated by assuming the composition
ratios at the combined source region (He/H)${}_{n}=0.036$ and (3He/4He)n =
0.1. To ensure good statistics in each observation sample, we only examine
windows that have more than $1000$ protons. Each parameter set is simulated
using $4$ different realizations by utilizing a different set of random
parameters. For the run with distance to 1 AU, we increased the total particle
number to $80$ million (assuming the SEP intensity decays as a function of
distance).

##  3 Simulation Results

Figure 1: Illustration of different field lines in a turbulent solar wind and
the energetic particle distribution in the turbulent magnetic fields. a) A
sketch of different field lines from two source regions with an offset. b) The
distribution of energetic hydrogen (turquoise) and helium (red) after 1 hour.
Observers at different locations naturally see quite different He/H ratios.

The turbulent, braiding magnetic field lines lead to complicated energetic
particle distributions. Figure 1 a) illustrates several sample field lines
from two source regions with an offset. In this case the source size is
$R_{source}=R_{0}$ and the offset distance is $D=R_{0}$. The blue source is
only filled with protons and the orange source region only has helium.
Particles have a tendency to move along a field line for a distance before
jumping to another. Therefore the observers connected to the blue field line
mostly see protons and the ones connected to the orange field line see more
helium. The brown field line has a mixture of both as it connects to the
overlapping region. The simulation results confirm this basic idea. Figure 1
b) shows the spatial distribution of protons (turquoise) and helium (red) 1
hour after the release time from the offset source. Note our magnetic field
model is fully 3D, which is important to model particle transport more
realistically (Jokipii et al., 1993; Jones et al., 1998). The distributions of
SEPs are highly nonuniform due to the effects of large-scale field line random
walk. With this offset between the source regions, the difference between
distributions of hydrogen and helium is quite significant. Spacecraft at 0.3
AU naturally see different He/H at different locations.

Figure 2: Two sample observations taken at different locations at 0.3 AU.
Upper panels show a case with low He/H = 0.181 and the lower panels show a
sample event with high He/H = 0.0024. Note that the ratio has been normalized
by the assumed nominal ratio (He/H)n = 0.036 (each helium count corresponds to
0.036 helium particle).

Figure 2 shows two sample observations at 0.3 AU mimicking PSP observations
taken at different collection locations. The upper panels show protons and
helium for a sample observation that is “helium-rich” (He/H = 0.181), whereas
the lower panels are for a helium-poor sample observation (He/H = 0.0024). In
both sample events, we observe clear velocity dispersion as faster (high
energy) particles arrive at observers earlier. Sample 1 is more than 5 times
of the nominal value of He/H and Sample 2 is only 1/15 of the nominal ratio
used in this study. These are similar to the variation of He/H of SEPs
observed by PSP (Cohen et al., 2021). We also briefly examined how the
observed He/H depends on particle energy. While the energy-dependent He/H
during individual observations can be quite variable, during these “extreme”
events with very high or very low He/H ratios, they often shows clear energy
dependence. At low energy, He/H shows extreme values (nearly 1.0 in Sample 1
and less than $10^{-3}$ in Sample 2), likely due to a good magnetic connection
with one of the source regions and the poor connectivity with the other one.
However, high energy particles can travel across field lines more easily and
mix, therefore He/H in high energy has a trend to be closer to the nominal
ratio (3.6 % in this case) than that of lower energies. We also observed other
sample events with different path lengths and dropouts similar to our earlier
work (Guo & Giacalone, 2014), which might be seen by PSP as more SEP events
are collected. Note that this variation can exist between any two species such
as 3He/4He (see below), as long as the source regions have an offset.

Figure 3: PDFs of He/H and 3He/4He observed at different locations at 0.3 AU
when $R=R_{0}$ and $D=R_{0}$. Vertical dashed lines represent the input source
ratios.

Since the observations would highly depend on how braiding magnetic field
lines connect with observers, it is interesting to study the likelihood of ion
composition ratios observed by spacecraft. We statistically study the He/H
ratios and also 3He/4He with $D=R_{0}$ and $R_{source}=R_{0}$. Figure 3 shows
the probability distribution functions (PDFs) for He/H and 3He/4He with more
than 1000 samples collected by different observers at 0.3 AU. One can observe
strong influence of interplanetary meandering magnetic fields on He/H. He/H is
seen to be variable over about nearly three orders of magnitude from about
$0.1\%$ to about $50\%$. Interestingly, 3He/4He (assuming a source ratio 0.1)
shows a similar variability ranging from $0.3\%$ to about $5$. These may have
strong implications on observations of ion compositions in the inner
heliosphere. It is worth noting that the distribution prefers helium-poor
events (He/H < (He/H)n) over helium-rich events (He/H > (He/H)n), which seems
to be a robust feature in the simulations we have done. The same situation
applies to the 3He/4He ratio as well.

Figure 4: Comparison between PDFs of He/H measured at 0.3 AU (orange) and 1 AU
(blue) with a vertical line showing the source region ratio.

It is instructive to explore how the distance between observers and the source
regions influence the composition ratio. Figure 4 shows the PDFs of He/H ratio
seen at 0.3 AU and 1 AU. Samples at 1 AU still show a broad distribution for
over more than one order of magnitude, but the distribution is significantly
narrower than that measured at 0.3 AU. As energetic particles propagate, they
are more and more mixed together even if their composition is very nonuniform
in the beginning. This suggests that the variation of the composition ratio
decays with distance. The PDF at 1 AU is also qualitatively similar to the
variable He/C distribution obtained by Reames (2019).

Next we examine how the distribution of He/H changes with offset distances (D
= 0, 0.5$R_{0}$, 2.0$R_{0}$) with fixed source radii $R=R_{0}$, as well as
different source radius (R = 0.5$R_{0}$, 2.0$R_{0}$, 4.0$R_{0}$) with a fixed
offset distance $D=R_{0}$. Since PSP observations have provided quantitative
measurements for He/H, and 3He/4He is likely more variable in the combined
source region (Petrosian & Liu, 2004), we focus on He/H, but note that 3He/4He
would have the same level of variation even for a fixed composition ratio in
the combined source regions. Figure 5 shows the PDF for different cases. In
the case with no displacement, the He/H is less than a factor of $2-3$ above
or below the source ratio, which may due to, for example, different particle
drift motions in the turbulent magnetic field. This variation becomes higher
when the displacement is larger. Over all, the variation of composition ratios
becomes significant when the offset distance is comparable to the radius of
the source region. We do notice that the PDFs of He/H have some difference
when both $R$ and $D$ change by the same ratio, indicating that the variation
of isotope ratios have some weak dependence on other factors, which we plan to
explore more in the future.

Figure 5: PDFs of He/H with different displacement and source radius. Vertical
lines show the assumed source region ratio $3.6\%$. a), b), and c) show
distribution of He/H for source radius $R=R_{0}$ and offset distance $D=0$,
$0.5R_{0}$, and $2.0R_{0}$, respectively. d), e), and f) show PDFs of He/H for
offset distance $D=R_{0}$, and source radius $R=0.5R_{0}$, $2.0R_{0}$, and
$4.0R_{0}$, respectively.

##  4 Discussion and Conclusions

Recent PSP observations within 0.5 AU suggest substantial variations in the
ion composition of SEP events (Cohen et al., 2021; Leske et al., 2020;
Wiedenbeck et al., 2020). We have shown in this paper that this variation can
possibly be due to the propagation of SEPs in braiding interplanetary magnetic
field lines of force, if the source regions are offset. This mechanism
predicts that observers at different locations will see different ion
composition ratios from the same SEP event, with possible energy dependences.
The variation of composition ratios decays with radial distance, which can
also be tested by spacecraft observations. The active observations by PSP, SO
and other spacecraft will likely advance our knowledge on this issue by
measuring events at distances that have never been seen before.

Earlier work has shown FIP effects may lead to a reduced He/H in the source
region ratios by a factor of a few (Reames, 2019; Laming, 2021), but may have
difficulties to reach extremely low He/H. Heating of helium particles can be
more efficient than that of protons during magnetic reconnection in the
presence of a strong guide field, potentially enhancing He/H in the energetic
component (Drake & Swisdak, 2014). Our model suggests that interplanetary
turbulent magnetic fields can lead to significant variability both above and
below the nominal ratios, without needing a mechanism to change the source
ratios. Since the simulated variation in our paper is biased toward smaller
He/H than the nominal values, it may be often seen as a suppression (Reames,
2019). In the future, it would be important to statistically study the
variable composition ratio over a large number of species in many SEP events
(e.g., Wiedenbeck et al., 2008; Bučík et al., 2021).

PSP has reported dropout-like features where proton flux suddenly drops (Leske
et al., 2020) similar to that observed at 1 AU (Mazur et al., 2000; Giacalone
et al., 2000; Ruffolo et al., 2003; Chollet & Giacalone, 2008; Guo &
Giacalone, 2014). If only one species suddenly changes its flux, this may lead
to a temporal change of He/H. If the helium has a similar dropout phenomenon
with a different timing, this can be used to test whether helium and hydrogen
source regions are offset. Chollet et al. (2009) have shown some electron
dropouts have different timings from the proton dropouts which may be due to
the separation of the source region. In addition, previous observations have
demonstrated that different observers will see different characteristics of
SEP features (Chollet & Giacalone, 2008). Future PSP and SO observations may
provide further insight on this.

We have shown that in our model the source region offset is essential for
producing the observed variations in ion compositions by PSP. Thus, on the one
hand, our results suggest that observations of composition variations observed
in situ provide important information about variations in the combined source
region close to the Sun. On the other hand, however, it is unclear what leads
to this offset. There have been observations showing the spatial offsets
between electrons (indicated by X-rays) and ions (indicated by $\gamma$-rays).
The observed offset in several flare events is about 10,000-15,000 km (Lin et
al., 2003; Hurford et al., 2003, 2006), which is comparable to the offset
distance needed to produce the observed composition variations reported. The
mechanism for this offset is not fully understood. Emslie et al. (2004)
suggest that this offset is due to the fact that different particle species
will interact with MHD turbulence differently during stochastic acceleration
process (Petrosian & Liu, 2004; Fu et al., 2020). Meanwhile, there have been
observations and theoretical work showing protons and electrons can be
accelerated simultaneously by the same mechanism (Shih et al., 2009; Guo &
Giacalone, 2012; Zhang et al., 2021). We comment that the drift motion in the
reconnection current layer may contribute to this. As an order of magnitude
estimate, we note that drift motions in the current sheet scales with
$v_{d}\sim\varepsilon/(qBR_{c})$ (Guo et al., 2014; Li et al., 2017, 2019;
Arnold et al., 2021). Assuming a magnetic field $B\sim 30$ G, curvature radius
$R_{c}\sim 1000$ km, and particle energy $\varepsilon\sim 30$ MeV, this can
lead to a drift speed of $\sim 10$ km/s. The time scale for this drift motion
may be limited by the scale of the reconnection layer and the outflow speed
$\tau\sim L/V_{out}$. We assume a current sheet of $L\sim 10^{5}$ km and an
outflow speed of $V_{out}\sim 100$ km/s (e.g., Tian et al., 2014). The
resulting drift distance can be $L_{d}\sim 10,000$ km, similar to what was
observed by RHESSI (Hurford et al., 2006), although more refined calculations
are definitely needed. Finally, we note that different time-dependent
injections of different species (Wang et al., 2016) would also be interesting
to explore in the future.

In summary, we have presented numerical simulations for interpreting recent
PSP observations on the highly variable He/H and possibly other ion
composition ratios. We showed that when the source regions of different
species are offset by a distance comparable to the size of the source region,
the observed energetic particle composition ratios can be strongly variable,
over more than two orders of magnitude. The composition variation decays with
distance, making PSP and SO unique for further understanding. The observed
3He/4He can also be quite variable if such offset exists between source
regions of different species, which is consistent with statistical
observations (Ho et al., 2005), although it is unknown if this variation is
due to the transport effect, or entirely due to the acceleration process.

We acknowledge the constructive comments from the reviewer of this paper,
which significantly helped improving the clarity of the paper. We thank
members of SolFER collaboration for helpful feedback. SolFER is a multi-
institutional collaboration for understanding the release of magnetic field
energy and associated particle acceleration in flares in the solar corona. F.
G., C. M. S. C., M. W., and Q. Z. acknowledge SolFER DRIVE center program with
grant number 80NSSC20K0627 and 80HQTR20T0040. F. G. and L. Zhao acknowledge
supports from NASA PSP GI Program 80HQTR21T0117. F. G. and J. G. acknowledge
support in part from NASA LWS Program grant 80HQTR21T0005. L. Z is supported
in part by NASA SOLSTICE DRIVE Science Center grant 80NSSC20K0600 and NASA LWS
grant 80NSSC21K0417. The work by X. L. and F. G. is funded in part by NASA
Grant 80NSSC21K1313 and 80HQTR21T0087. The work of J. G. was supported in part
by NASA’s Parker Solar Probe Mission under contract NNN06AA01C, NASA under
grant 80NSSC20K1283, and NSF/AGS under grant 1931252. Q. Z. also acknowledges
support from NASA Grant 80HQTR21T0103. F. G. is also supported by DOE through
the LDRD program at LANL and office of Fusion Energy Sciences.

## References

  * Arnold et al. (2021) Arnold, H., Drake, J. F., Swisdak, M., et al. 2021, Phys. Rev. Lett., 126, 135101. doi:10.1103/PhysRevLett.126.135101
  * Bieber et al. (1996) Bieber, J. W., Wanner, W., & Matthaeus, W. H. 1996, J. Geophys. Res., 101, 2511. doi:10.1029/95JA02588
  * Bučík et al. (2021) Bučík, R., Mulay, S. M., Mason, G. M., et al. 2021, ApJ, 908, 243. doi:10.3847/1538-4357/abd62d
  * Chhiber et al. (2020) Chhiber, R., Matthaeus, W. H., Cohen, C. M. S., et al. 2020, arXiv:2011.08329
  * Chollet & Giacalone (2008) Chollet, E. E. & Giacalone, J. 2008, ApJ, 688, 1368. doi:10.1086/592378
  * Chollet et al. (2009) Chollet, E. E., Giacalone, J., Skoug, R. M., et al. 2009, ApJ, 705, 1492. doi:10.1088/0004-637X/705/2/1492
  * Chollet & Giacalone (2011) Chollet, E. E. & Giacalone, J. 2011, ApJ, 728, 64. doi:10.1088/0004-637X/728/1/64
  * Cohen et al. (2021) Cohen, C. M. S., Christian, E. R., Cummings, A. C., et al. 2021, A&A, 650, A23. doi:10.1051/0004-6361/202039299
  * Dalla et al. (2017) Dalla, S., Marsh, M. S., Zelina, P., et al. 2017, A&A, 598, A73. doi:10.1051/0004-6361/201628618
  * Desai & Giacalone (2016) Desai, M. & Giacalone, J. 2016, Living Reviews in Solar Physics, 13, 3. doi:10.1007/s41116-016-0002-5
  * Drake & Swisdak (2014) Drake, J. F. & Swisdak, M. 2014, Physics of Plasmas, 21, 072903. doi:10.1063/1.4889871
  * Dresing et al. (2012) Dresing, N., Gómez-Herrero, R., Klassen, A., et al. 2012, Sol. Phys., 281, 281. doi:10.1007/s11207-012-0049-y
  * Emslie et al. (2004) Emslie, A. G., Miller, J. A., & Brown, J. C. 2004, ApJ, 602, L69. doi:10.1086/382350
  * Fu et al. (2020) Fu, X., Guo, F., Li, H., et al. 2020, ApJ, 890, 161. doi:10.3847/1538-4357/ab6d68
  * Giacalone & Jokipii (1999) Giacalone, J. & Jokipii, J. R. 1999, ApJ, 520, 204. doi:10.1086/307452
  * Giacalone et al. (2000) Giacalone, J., Jokipii, J. R., & Mazur, J. E. 2000, ApJ, 532, L75. doi:10.1086/312564
  * Giacalone et al. (2006) Giacalone, J., Jokipii, J. R., & Matthaeus, W. H. 2006, ApJ, 641, L61. doi:10.1086/503770
  * Giacalone & Jokipii (2012) Giacalone, J. & Jokipii, J. R. 2012, ApJ, 751, L33. doi:10.1088/2041-8205/751/2/L33
  * Guo & Giacalone (2012) Guo, F. & Giacalone, J. 2012, ApJ, 753, 28. doi:10.1088/0004-637X/753/1/28
  * Guo & Giacalone (2014) Guo, F. & Giacalone, J. 2014, ApJ, 780, 16. doi:10.1088/0004-637X/780/1/16
  * Guo et al. (2014) Guo, F., Li, H., Daughton, W., et al. 2014, Phys. Rev. Lett., 113, 155005. doi:10.1103/PhysRevLett.113.155005
  * Guo & Giacalone (2015) Guo, F. & Giacalone, J. 2015, ApJ, 802, 97. doi:10.1088/0004-637X/802/2/97
  * Ho et al. (2005) Ho, G. C., Roelof, E. C., & Mason, G. M. 2005, ApJ, 621, L141. doi:10.1086/429251
  * Hurford et al. (2003) Hurford, G. J., Schwartz, R. A., Krucker, S., et al. 2003, ApJ, 595, L77. doi:10.1086/378179
  * Hurford et al. (2006) Hurford, G. J., Krucker, S., Lin, R. P., et al. 2006, ApJ, 644, L93. doi:10.1086/505329
  * Jokipii (1966) Jokipii, J. R. 1966, ApJ, 146, 480. doi:10.1086/148912
  * Jokipii & Parker (1969) Jokipii, J. R. & Parker, E. N. 1969, ApJ, 155, 777. doi:10.1086/149909
  * Jokipii (1971) Jokipii, J. R. 1971, Reviews of Geophysics and Space Physics, 9, 27. doi:10.1029/RG009i001p00027
  * Jokipii et al. (1993) Jokipii, J. R., Kota, J., & Giacalone, J. 1993, Geophys. Res. Lett., 20, 1759. doi:10.1029/93GL01973
  * Jones et al. (1998) Jones, F. C., Jokipii, J. R., & Baring, M. G. 1998, ApJ, 509, 238. doi:10.1086/306480
  * Kahler & Brown (2021) Kahler, S. W. & Brown, D. 2021, ApJ, 908, 214. doi:10.3847/1538-4357/abd481
  * Laming (2021) Laming, J. M. 2021, ApJ, 909, 17. doi:10.3847/1538-4357/abd9c3
  * Leske et al. (1999a) Leske, R. A., Cohen, C. M. S., Cummings, A. C., et al. 1999, Geophys. Res. Lett., 26, 153. doi:10.1029/1998GL900173
  * Leske et al. (1999b) Leske, R. A., Mewaldt, R. A., Cohen, C. M. S., et al. 1999, Geophys. Res. Lett., 26, 2693. doi:10.1029/1999GL900561
  * Leske et al. (2020) Leske, R. A., Christian, E. R., Cohen, C. M. S., et al. 2020, ApJS, 246, 35. doi:10.3847/1538-4365/ab5712
  * Li et al. (2017) Li, X., Guo, F., Li, H., et al. 2017, ApJ, 843, 21. doi:10.3847/1538-4357/aa745e
  * Li et al. (2019) Li, X., Guo, F., & Li, H. 2019, ApJ, 879, 5. doi:10.3847/1538-4357/ab223b
  * Lin et al. (2003) Lin, R. P., Krucker, S., Hurford, G. J., et al. 2003, ApJ, 595, L69. doi:10.1086/378932
  * Mason et al. (1979) Mason, G. M., Gloeckler, G., & Hovestadt, D. 1979, ApJ, 231, L87. doi:10.1086/183010
  * Matthaeus et al. (1990) Matthaeus, W. H., Goldstein, M. L., & Roberts, D. A. 1990, J. Geophys. Res., 95, 20673. doi:10.1029/JA095iA12p20673
  * Matthaeus et al. (2003) Matthaeus, W. H., Qin, G., Bieber, J. W., et al. 2003, ApJ, 590, L53. doi:10.1086/376613
  * Mazur et al. (2000) Mazur, J. E., Mason, G. M., Dwyer, J. R., et al. 2000, ApJ, 532, L79. doi:10.1086/312561
  * McComas et al. (2016) McComas, D. J., Alexander, N., Angold, N., et al. 2016, Space Sci. Rev., 204, 187. doi:10.1007/s11214-014-0059-1
  * Petrosian & Liu (2004) Petrosian, V. & Liu, S. 2004, ApJ, 610, 550. doi:10.1086/421486
  * Press et al. (1986) Press, W. H., Flannery, B. P., & Teukolsky, S. A. 1986, Cambridge: University Press, 1986
  * Qin et al. (2002) Qin, G., Matthaeus, W. H., & Bieber, J. W. 2002, ApJ, 578, L117. doi:10.1086/344687
  * Reames (1995) Reames, D. V. 1995, Advances in Space Research, 15, 41
  * Reames (2019) Reames, D. V. 2019, Sol. Phys., 294, 32. doi:10.1007/s11207-019-1422-x
  * Rincon & Rieutord (2018) Rincon, F. & Rieutord, M. 2018, Living Reviews in Solar Physics, 15, 6. doi:10.1007/s41116-018-0013-5
  * Rodríguez-Pacheco et al. (2020) Rodríguez-Pacheco, J., Wimmer-Schweingruber, R. F., Mason, G. M., et al. 2020, A&A, 642, A7. doi:10.1051/0004-6361/201935287
  * Ruffolo et al. (2003) Ruffolo, D., Matthaeus, W. H., & Chuychai, P. 2003, ApJ, 597, L169. doi:10.1086/379847
  * Shih et al. (2009) Shih, A. Y., Lin, R. P., & Smith, D. M. 2009, ApJ, 698, L152. doi:10.1088/0004-637X/698/2/L152
  * Tian et al. (2014) Tian, H., Li, G., Reeves, K. K., et al. 2014, ApJ, 797, L14. doi:10.1088/2041-8205/797/2/L14
  * Wang et al. (2016) Wang, L., Krucker, S., Mason, G. M., et al. 2016, A&A, 585, A119. doi:10.1051/0004-6361/201527270
  * Wiedenbeck et al. (2008) Wiedenbeck, M. E., Leske, R. A., Cohen, C. M. S., et al. 2008, Particle Acceleration and Transport in the Heliosphere and Beyond: 7th Annual International AstroPhysics Conference, 1039, 149. doi:10.1063/1.2982438
  * Wiedenbeck et al. (2013) Wiedenbeck, M. E., Mason, G. M., Cohen, C. M. S., et al. 2013, ApJ, 762, 54. doi:10.1088/0004-637X/762/1/54
  * Wiedenbeck et al. (2020) Wiedenbeck, M. E., Bučík, R., Mason, G. M., et al. 2020, ApJS, 246, 42. doi:10.3847/1538-4365/ab5963
  * Zank (2014) Zank, G. P. 2014, Transport Processes in Space Physics and Astrophysics: Lecture Notes in Physics, Volume 877. ISBN 978-1-4614-8479-0. Springer Science+Business Media New York, 2014. doi:10.1007/978-1-4614-8480-6
  * Zhang et al. (2021) Zhang, Q., Guo, F., Daughton, W., et al. 2021, arXiv:2105.04521
  * Zhao & Zhang (2018) Zhao, L. & Zhang, M. 2018, ApJ, 859, L29. doi:10.3847/2041-8213/aac6cf
  * Zhao et al. (2020) Zhao, L., Zhang, M., & Lario, D. 2020, ApJ, 898, 16. doi:10.3847/1538-4357/ab97b3