#  Federated Behavioural Planes: Explaining the Evolution of Client Behaviour
in Federated Learning

Dario Fenoglio*
Università della Svizzera italiana
Lugano, Switzerland
<EMAIL_ADDRESS>&Gabriele Dominici*
Università della Svizzera italiana
Lugano, Switzerland
<EMAIL_ADDRESS>Pietro Barbiero
Università della Svizzera italiana
Lugano, Switzerland
<EMAIL_ADDRESS>
&Alberto Tonda
INRAE
Paris, France
<EMAIL_ADDRESS>
Martin Gjoreski
Università della Svizzera italiana
Lugano, Switzerland
<EMAIL_ADDRESS>
&Marc Langheinrich
Università della Svizzera italiana
Lugano, Switzerland
<EMAIL_ADDRESS>

###### Abstract

Federated Learning (FL), a privacy-aware approach in distributed deep learning
environments, enables many clients to collaboratively train a model without
sharing sensitive data, thereby reducing privacy risks. However, enabling
human trust and control over FL systems requires understanding the evolving
behaviour of clients, whether beneficial or detrimental for the training,
which still represents a key challenge in the current literature. To address
this challenge, we introduce _Federated Behavioural Planes_ (FBPs), a novel
method to analyse, visualise, and explain the dynamics of FL systems, showing
how clients behave under two different lenses: predictive performance (error
behavioural space) and decision-making processes (counterfactual behavioural
space). Our experiments demonstrate that FBPs provide informative trajectories
describing the evolving states of clients and their contributions to the
global model, thereby enabling the identification of clusters of clients with
similar behaviours. Leveraging the patterns identified by FBPs, we propose a
robust aggregation technique named _Federated Behavioural Shields_ to detect
malicious or noisy client models, thereby enhancing security and surpassing
the efficacy of existing state-of-the-art FL defense mechanisms.

##  1 Introduction

Federated Learning (FL), a privacy-aware deep learning (DL) approach in
distributed environments, is a dynamic system where many clients collaborate
to train a model without sharing sensitive data, thus mitigating privacy risks
[1, 2]. Analyzing the behaviour of FL systems is crucial to detect
anomalies—such as distribution shifts [3, 4], biased data [5], or adversarial
clients [6, 7, 8, 9]—which may compromise the global model’s predictive
performance and introduce biases into its decision-making process. Various
strategies have been developed to detect FL anomalies [10, 11, 12, 13, 14,
15]. However, existing techniques are not designed to track, visualise, and
explain how client behaviours affect the performance of the global model, thus
limiting human trust and control on FL dynamics.

Various studies have analysed models’ behaviour in terms of predictive
performance and decision-making processes independently. Predictive
performance behaviour is primarily investigated in the context of non-linear
optimisation using _behavioural spaces_ [16, 17, 18]. This technique allows
the visualisation of the predictive diversity of a set of regression models by
considering the vector of errors that each model produces on a set of samples.
The decision-making process is studied mainly in explainable AI (XAI) research
using, for example, _counterfactual explanations_ [19]. Counterfactuals can be
used to identify relevant input features used by a model to make predictions,
thus describing the position and orientation of the model’s decision
boundaries. However, these techniques have not yet been applied to FL systems,
resulting in a lack of insight into how the behaviour of individual clients
affects the overall model’s accuracy and decision-making capabilities, leading
to inefficiency in the training process.

To bridge this gap, we introduce _Federated Behavioural Planes_ (FBPs), a
method designed to visualise, explain, and give insights into the dynamics of
FL systems. Our key innovation involves the creation of two behavioural planes
for FL clients: one to highlight their predictive diversity and another to
emphasise their decision-making process diversity via counterfactuals.
Building on the client behaviour information provided by FBPs, to show their
practical utility, we propose _Federated Behavioural Shields_ , a robust
aggregation mechanism that enhances security against malicious or noisy
clients by accurately weighting the client models according to their
constructive contributions during training.

Figure 1:  Federated Behavioural Spaces framework makes it possible to visual
client behaviour in FL from two perspectives: predictive performance (_Error
Behavioural Plane_) and decision-making processes (_Counterfactuals
Behavioural Plane_). It highlights client trajectories and similarities,
offering insights into client interactions and supporting the introduction of
a new and effective robust aggregation mechanism with performance that
surpasses state-of-the-art baselines.

The results of our experiments demonstrate that: (i) counterfactual generators
jointly trained with FL systems produce valid and client-specific
counterfactual explanations which effectively describe clients’ decision-
making diversity; (ii) FBPs facilitate the identification of clusters of
clients with similar behaviours (e.g., normal vs. outlier clients), allowing
for tracking of their trajectories during the entire training; (iii) Federated
Behavioural Shields surpasses existing state-of-the-art defense mechanisms,
demonstrating that the information contained in FBPs provides valuable
descriptors of client behaviour.

##  2 Background

Federated learning FL systems [20, 3] involve a network of $K\in\mathcal{N}$
clients, coordinated by a central server, which collaboratively train a DL
model. Each client $k$ possesses a local and private dataset characterised by
a set of $z\in\mathcal{N}$ features $x^{(k)}\in
X^{(k)}\subseteq\mathcal{R}^{z}$ and a set of $u\in\mathcal{N}$ class labels
$y^{(k)}\in Y^{(k)}\subseteq\\{0,1\\}^{u}$. In each training round
$t\in\mathcal{N}$, each client trains a model $f:X^{(k)}\to Y^{(k)}$ on local
data to maximise the likelihood $\mathcal{L}(\theta^{(k)}\mid
x^{(k)},y^{(k)})$. Once trained, clients send their local model’s parameters
$\theta^{(k)}(t)\in\mathcal{R}^{q},q\in\mathcal{N}$ to a central server which
aggregates these parameters using a permutation-invariant aggregation
$\oplus:\mathcal{R}^{q\times K}\to\mathcal{R}^{q}$ (such as the mean or
median). The server then sends the aggregated model parameters $\theta(t+1)$
back to the clients to start a new training round:

$\displaystyle(\textit{local training})$
$\displaystyle\quad\theta^{(k)}(t)=\arg\max_{\theta^{(k)}(t)}\mathcal{L}(\theta^{(k)}(t)\mid
x^{(k)},y^{(k)})$ (1) $\displaystyle(\textit{aggregation})$
$\displaystyle\quad\theta(t+1)=\bigoplus_{k\in K}\theta^{(k)}(t)$ (2)

While FL is an efficient process to safeguard privacy, the inherent lack of
direct control over each individual client makes FL systems particularly
vulnerable to various poisoning attacks [6, 7, 8, 9, 21, 22, 14, 23, 24, 25,
26]. These can be categorised into model and data poisoning attacks. Model
poisoning involves altering gradients on compromised devices before
transmission to the server [9, 14, 23, 24], while data poisoning indirectly
manipulates gradients by tampering with training datasets on malicious devices
[25, 27, 21].

Counterfactual explanations Counterfactual explanations [19] describe a
model’s decision-making process by identifying minimal and plausible changes
to an observed input’s features that lead to a desired model prediction. In
explainable AI, finding counterfactual explanations is framed as an
optimisation problem where the objective is to identify, for each sample $x$,
the nearest data point $x^{\prime}$ such that the classifier
$f(\theta,x^{\prime})$ assigns a desired class label $y^{\prime}$:

$\arg\min_{x^{\prime}}||x-x^{\prime}||\quad\text{s.t.}\quad
f(\theta,x^{\prime})=y^{\prime}$ (3)

As a result, the variations in the input’s features between $x$ and
$x^{\prime}$ offer actionable insights into how the model’s decisions can be
altered, highlighting the most important features.

Semantic and behavioural spaces Semantic spaces [16, 17] represent the
semantics of a model by considering the error vector
$e=[\epsilon(f(\theta,x_{i}),y_{i})]_{i=1,\dots,n}$ that a model produces on a
set of $n\in\mathcal{N}$ samples, where $\epsilon$ could be the Mean Squared
Error, for instance. Given a set of $K$ models, the semantic space contains
$n$-dimensional data points $[e_{1},\dots,e_{K}]\in\mathcal{R}^{K\times n}$.
Behavioural spaces [18] summarise semantic spaces into lower-dimensional
spaces applying a transformation $\psi_{n\to
m}:\mathcal{R}^{n}\to\mathcal{R}^{m}$ with $m\ll n$ (typically $m=2$ for most
applications) i.e., $\psi_{n\to m}([e_{1},\dots,e_{K}])$.

##  3 Federated behavioural Planes

Problem definition: Given an FL system composed of a set of $K$ clients with
local data $(x^{(k)},y^{(k)})$ and models $f(\theta^{(k)})$, we aim to analyse
the evolution of the system to understand how clients impact the global
model’s predictive performance and decision-making over time. In Section 3.1,
we introduce what drives our method and formalise the problem. In Section 3.2,
we introduce Federated Behavioural Planes (FBPs), describing its components in
more detail: the Error Behavioural Plane (Section 3.3) and the Counterfactual
Behavioural Plane (Section 3.4). Finally, in Section 3.5, we introduce
Federated Behavioural Shields, a new robust aggregation mechanism to enhance
security in FL systems, showing a practical application of FBPs.

###  3.1 Dynamic behaviour of federated learning

FL is a dynamic process transitioning from a state where all participating
clients behave randomly to a state where the behaviour of participating
entities becomes coherent (the parameters of client models tend to converge to
values which lead to high model accuracy). Thus, FL systems can be effectively
analysed through the lens of dynamical systems using tools traditionally
employed for such studies, such as differential equations. Defining
$b(\theta^{(k)})$ as the evolving state of the behaviour of the client $k$ at
time $t$, we introduce two primary forces that influence $b(\theta^{(k)})$:
$g(\theta^{(k)},x^{(k)},y^{(k)})$, the local training dynamics, which drives
the client towards its local optimum by leveraging information from the local
dataset $(x^{(k)},y^{(k)})$; and $b\left(\bigoplus_{k\in
K}\theta^{(k)}\right)-b(\theta^{(k)})$, a correction term that periodically
aligns $b(\theta^{(k)})$ with the aggregated state of all clients within the
federated system. These dynamics can be encapsulated in the following
differential equation111For simplicity, we omit the dependency on the time
variable $t$ in our notation for variables $b$ and $\theta$.:

$\displaystyle\odv{b\left(\theta^{(k)}\right)}{t}=g\left(\theta^{(k)},x^{(k)},y^{(k)}\right)(1-\delta_{T})+\left[b\left(\bigoplus_{k\in
K}\theta^{(k)}\right)-b\left(\theta^{(k)}\right)\right]\cdot\delta$

Here, $\delta_{T}$ is characterised by a periodic Dirac delta function,
defined as $\delta_{T}=\sum_{r=0}^{\infty}\delta(t-r\cdot T)$, which triggers
instantaneous adjustments at intervals determined by period $T$.

###  3.2 Federated Behavioural Planes (FBPs)

Instead of finding a general analytical solution for Equation 3.1 (which is
not trivial and requires strong assumptions on its components), we aim to
empirically analyse the phase space of a FL system by considering different
descriptors of client behaviours. More specifically, we focus on investigating
(i) the predictive performance, evaluating how well the model is solving the
task, and (ii) the decision-making process, as it contains information on how
the model is solving the task. During each round, client behaviours are
assessed through their respective models on the server, utilising a server-
owned dataset reserved for this evaluation phase. This methodology aligns with
existing protocols [9, 14, 28, 29, 30, 31, 32]. Each client behaviour is
visualised through a two-dimensional plane: Error Behavioural Plane
representing predictive performance and the Counterfactual Behavioural Plane
to illustrate decision-making processes, collectively referred to as Federated
Behavioural Planes. However, this framework offers a general approach to
visualise and monitor different descriptors of the client behaviours
simultaneously, which can be customised through specific functions, enabling
the creation of additional planes.

###  3.3 Error Behavioural Plane (EBP)

To comprehensively evaluate each model from the predictive performance point
of view, we analyse the errors made by the model on all samples, rather than
relying solely on a simpler aggregate metric, such as loss or error [14]. This
approach enables a more detailed examination of the differences in the model’s
performance as observed by Mouret and Clune [18]. Following the methodology
proposed by Zhang et al. [33], we first construct a semantic error space for
each model and then map it to a reduced space, called Error Behavioural Plane
(EBP).

######  Definition 3.1 (Error Behavioural Plane).

Given a model $f$, parametrised with a set of weights $\theta^{(k)}(t)$,
related to the client $k$ at round $t$, a dataset
$(x^{(\text{server})},y^{(\text{server})})$, owned by the server, and a
dimensionality reduction technique $\psi_{n\to 2}$, the representation
$e^{(k)}(t)\in\mathcal{R}^{2}$ in the EBP of the client $k$ is the following:

$e^{(k)}(t)=\psi_{n\to
2}\left(\left[f(\theta^{(k)}(t),x_{i}^{(\text{server})})-y_{i}^{(\text{server})}\right]_{i=1,\dots,n}\right)$
(4)

It is worth noting that two clients, despite having similar accuracy or loss,
may receive significantly different representations in the EBP if they produce
errors on distinct subsets of samples. In contrast, clients whose trajectories
in the EBP converge over time form clusters representing clients whose
predictive performance is similar on the same set of samples. However,
clusters and trajectories in the EBP do not explain the decision-making
process that leads to a prediction. This information could be used to further
distinguish different types of clients and can be analysed using
counterfactuals.

###  3.4 Counterfactual Behavioural Plane (CBP)

The analysis of a model’s decision-making process is the main research
objective of explainable AI. Counterfactual explanations represent one of the
most effective techniques as they give insights concerning the position and
orientation of decision boundaries. Similar counterfactual explanations
indicate models having similar decision boundaries, i.e. models taking
decisions using a similar decision-making process. To this end, most
differentiable counterfactual generators are trained to model the training
data distribution [34, 35], thus potentially providing insights on non
identical distributions in clients’ data. To produce these explanations,
clients’ predictive models should be concurrently trained with a
counterfactual generator. Additional information on the optimisation objective
are provided in Appendix A. To obtain the Counterfactual Behavioural Plane
(CBP), we first compute the distances between the counterfactual distribution
generated by the server using a model of the client $k$ and the distribution
of other clients. Then, we apply dimensionality reduction to obtain the CBP.

######  Definition 3.2 (Counterfactual Behavioural Plane).

Given a model $f$ with parameters $\theta^{(k)}(t)$, related to the client $k$
at round $t$, a set of samples $x^{(\text{server})}\in\mathcal{R}^{n\times z}$
owned by the server with $n$ samples and $z$ features, a distance function
$d:\mathcal{R}^{n\times z}\to\mathcal{R}^{+}$, such as Wasserstein distance,
and a dimensionality reduction technique $\psi_{K\to 2}$, the representation
$c^{(k)}(t)\in\mathcal{R}^{2}$ in the Counterfactual Behavioural Plane of
client $k$ in an FL settings is the following:

$a^{(k)}(t)=\left[\arg\min_{x^{\prime}_{i}}||x^{(\text{server})}_{i}-x^{\prime}_{i}||\;\text{s.t.}\;f(\theta^{(k)}(t),x^{\prime}_{i})\neq
f(\theta^{(k)}(t),x^{(\text{server})}_{i})\right]_{i=1,\dots,n}$ (5)
$l^{(k)}(t)=\left[d(a^{(k)}(t),a^{(i)}(t))\right]_{i=1,\dots,K},\quad
c^{(k)}(t)=\psi_{K\to 2}\left(l^{(k)}(t)\right)$ (6)

CBP produces complementary information to EBP as clients which are similar in
the CBP might be far away in the EBP (as discussed in Appendix B).
Furthermore, as the purpose of CBP is to track clients’ behaviour rather than
explaining the model decision to a user, counterfactuals can be generated
based on predictive models’ embeddings, instead of input features, concealing
sensitive information.

###  3.5 Federated Behavioural Shields – FBPs as a defence mechanism

FBPs provide descriptors of client behaviours during training, enabling
various applications. Notably, trajectories in behavioural planes converging
over time form clusters representing clients with similar predictive
performance and decision-making process. This property can be used in practice
to identify anomalies, such as malicious clients attempting to compromise FL
training. In particular, leveraging this detailed information on client
behaviours, we propose Federated Behavioural Shields (FBSs), a new class of
robust aggregation strategies designed to enhance security in FL without
requiring prior knowledge on the attack. This defensive mechanism generates a
behavioural score in round $t$ for a client $k$, denoted as $s^{(k)}(t)$,
which is formulated through the composition of multiple scores $s_{j}^{(k)}$
computed on $S$ behavioural spaces, to guide the aggregation process in
creating the next round’s global model $\theta(t+1)$, as outlined below:

$s^{(k)}(t)=\frac{\prod_{j\in S}s_{j}^{(k)}(t)}{\sum_{i\in K}\prod_{j\in
S}s_{j}^{(i)}(t)},\quad\theta(t+1)=\bigoplus_{k\in
K}s^{(k)}(t)\theta^{(k)}(t)$ (7)

Specifically, based on the FBPs we previously defined, we can compute these
scores as follows:

$s^{(k)}(t)=\frac{s_{\text{error}}^{(k)}(t)s_{\text{cf}}^{(k)}(t)}{\sum_{i\in
K}s_{\text{error}}^{(i)}(t)s_{\text{cf}}^{(i)}(t)}\quad
s_{\text{error}}^{(k)}(t)=1-min(||e^{(k)}(t)||,1)\quad
s_{\text{cf}}^{(k)}(t)=\frac{1}{\frac{1}{K}\sum_{i\in K}l^{(k)}_{i}(t)}$

The error score $s^{(k)}_{\text{error}}(t)$ measures the distance between the
client $k$ and the optimal point at the center of the plane, while the
counterfactual score $s^{(k)}_{\text{cf}}$ measure the average distance
between the distribution of counterfactual generated by a client and all the
other clients. In addition, considering that honest clients may occasionally
deviate from the norm but generally contribute positively, we introduce a
moving average mechanism to track client behaviours (see Appendix A.2 for
details).

##  4 Experiments

The preliminary goal of our experiments is to assess whether counterfactual
generators can provide insights in an FL context without compromising the
performance of the predictor. We then visualise FBPs to verify that they can
reveal information about the behaviour of various clients through the error
and counterfatual behavioural planes (EBP and CBP). Lastly, we analyse the
effectiveness of Federated Behavioural Shields as a robust aggregation
mechanism, demonstrating the utility of the information provided by FBPs. Our
experiments aim to answer the following questions:

  * •

Counterfactuals in FL: Does the integration of counterfactual generators
impact clients’ predictive performance? Do counterfactuals have the same
quality in FL compared to centralised scenarios? Could counterfactual
generators be adapted for each client? Answering these questions is an
essential preliminary step to check whether counterfactuals can be used to
generate FBPs.

  * •

Explaining FL training: Can trajectories in FBPs describe the evolving client
behaviours during the training phase? Is it possible to visually identify
clusters of clients using FBPs?

  * •

Leveraging FBPs information: Do FBPs provide sufficient detail to Federated
Behavioural Shields to enhance the security of the FL training process against
security attacks?

This section describes essential information about the experiments. We provide
further details in Appendix A.2 such as counterfactual generators, model
optimisation, and training setup.

###  4.1 Data & task setup

In our experiments, we utilise four datasets: a Synthetic dataset (tabular) we
designed to have full control on clients’ data distributions, and thus test
our assumptions; the Breast Cancer Wisconsin [36] (tabular); the Diabetes
Health Indicator [37] (tabular); and small-MNIST [38] (image), reducing its
size by 76% to increase task difficulty and highlight client differences in
performance. For all the experiments with the small-MNIST dataset, our
approach involves generating counterfactuals at a non-interpretable internal
representation level of the model instead of the input space. To reflect the
most realistic cross-silo scenario [39], we distribute the training data among
various clients such as data are not independent and identically distributed
(IID) [40]. This represents the most challenging scenario to detect malicious
clients due to the significant variations even among benign clients. Further
details on the datasets and non-IID implementation are provided in Appendix
A.1. Additional experiments analysing setup characteristics such as window
length, local epochs, server validation set size, and differences between non-
IID and IID scenarios can be found in Appendix B.

###  4.2 Evaluation

Metrics To determine the efficacy of counterfactuals generated through end-to-
end training in FL, we measure several key metrics: task accuracy ($\uparrow$
– higher is better); counterfactual validity ($\uparrow$) [19], which checks
if the counterfactuals’ labels align with user-provided labels; proximity
($\downarrow$) [35], assessing whether counterfactuals are realistic by their
closeness to the training data; and sparsity ($\downarrow$) [19], which
quantifies the changes made to the input to generate the counterfactuals. The
latter is quantified using Euclidean distance, as counting the number of
changes provides less insight on the generated counterfactuals. To evaluate
the effectiveness of client-specific adaptation, we analyse the relative
change in client proximity between global and client-specific models,
expressed as $(P_{global}-P_{local})/P_{global}$. Finally, we measure the task
accuracy ($\uparrow$) of the FL system under different attacks and defenses.
All metrics are reported as the mean and standard error across five
experimental runs with distinct parameter initialization.

Baselines In our experiments, we compare Federated Behavioural Shields with
the following state-of-the-art robust aggregation methods: Median [11],
Trimmed-mean [11] and Krum [12]. Further information is provided in the
Appendix A.7.

Federated Attacks We focus on attacks with realistic assumptions and where
additional information outside the typical FL scenario are not available [10,
14, 23]. We test the following attacks: Label-flipping (Data Poisoning) [26,
41], changes each sample’s label to $1-y$ (performed in the context of binary
classification); Inverted-loss (Data Poisoning) [25], creates an update that
maximises the loss on the local dataset; Crafted-noise (Model Poisoning) [42],
adds noise $\mathcal{N}(0,\beta\cdot\sigma(w^{t}))$ to the previous global
model $w^{t}$, where $\sigma(w^{t})$ is the standard deviation of $w^{t}$ and
$\beta$ is a scale factor set to 1.2; and Inverted-gradient (Model Poisoning)
[24, 43], inverts the gradient derived from the server’s previous update,
misaligning it with the true gradient. Further details are provided in
Appendix A.7.

##  5 Key Findings & Results

Table 1: Performance comparison of our model, which includes a Predictor and
Counterfactual Generator (CF), across various settings: Local Centralised
(Local CL), Centralised Learning (CL), Federated Learning (FL), and FL with
only the Predictor in a non-IID setting.

Metric | Dataset | Local CL | CL | FL | FL
---|---|---|---|---|---
|  | Predictor + CF | Predictor + CF | Predictor + CF | Predictor
Accuracy (↑) | Diabetes |  55.9 $\pm$ 0.5% |  75.0 $\pm$ 0.2% |  74.7 $\pm$ 0.1% |  74.2 $\pm$ 0.1%
| Breast Cancer |  86.9 $\pm$ 0.7% |  97.7 $\pm$ 0.0% |  98.4 $\pm$ 0.1% |  97.7 $\pm$ 0.4%
| Synthetic |  75.0 $\pm$ 2.0% |  99.4 $\pm$ 0.2% |  99.8 $\pm$ 0.1% |  99.9 $\pm$ 0.1%
Validity (↑) | Diabetes |  87.6 $\pm$ 2.6% |  99.9 $\pm$ 0.1% |  99.9 $\pm$ 0.0% | N/A
| Breast Cancer |  100.0 $\pm$ 0.1% |  100.0 $\pm$ 0.0% |  100.0 $\pm$ 0.0% | N/A
| Synthetic |  97.1 $\pm$ 1.9% |  100.0 $\pm$ 0.0% |  100.0 $\pm$ 0.0% | N/A
Sparsity (↓) | Diabetes |  45.4 $\pm$ 2.1 |  34.5 $\pm$ 1.7 |  37.1 $\pm$ 1.2 | N/A
| Breast Cancer |  1459 $\pm$ 25 |  1325 $\pm$ 20 |  1448 $\pm$ 43 | N/A
| Synthetic |  8.63 $\pm$ 0.15 |  6.24 $\pm$ 0.22 |  6.14 $\pm$ 0.07 | N/A
Proximity (↓) | Diabetes |  8.91 $\pm$ 0.61 |  5.45 $\pm$ 0.40 |  6.23 $\pm$ 0.44 | N/A
| Breast Cancer |  61.2 $\pm$ 2.1 |  70.1 $\pm$ 1.8 |  72.1 $\pm$ 5.5 | N/A
| Synthetic |  0.142 $\pm$ 0.026 |  0.091 $\pm$ 0.003 |  0.089 $\pm$ 0.002 | N/A

###  5.1 Counterfactuals in FL

Integrating counterfactual generators in FL optimisation does not compromise
predictive performance. (Table 1) We compare the models’ accuracy across three
distinct settings and three different datasets under non-IID conditions. The
results indicate that our model, which involves the concurrent training of
both the predictor and the counterfactual generator, achieves performance
comparable to that of the predictor alone in FL. In the context of our model,
both Local Centralised Learning (CL), a CL performed on individual client
data, and FL, implemented across all clients, adhere to privacy standards [2].
However, only FL reaches the performance levels of the CL scenario, which
assumes local access to all client data. This result indicates that
incorporating counterfactuals during training does not compromise the
predictor’s performance, thus supporting their beneficial application without
adverse effects on performance.

Counterfactuals generated in FL have similar quality to the ones in CL. (Table
1) Unlike Local CL, FL leverages information from all clients, thereby
producing counterfactuals that more accurately reflect non-IID clients’ data.
As shown in Table 1, FL achieves higher validity compared to the Local CL
approach, indicating that the FL’s counterfactuals better match ground-truth
labels. Additionally, FL exhibits lower sparsity, which measures the number of
modifications needed to achieve the counterfactuals. Table 1 shows these
results using a counterfactual generator we adapted for this scenario (see
Appendix A.2), however, similar conclusions are also obtained using out-of-
the-box generators [34] (see Appendix B).

Figure 2: Relative variation of client-proximity across datasets.

Counterfactual generators can be adapted to specific clients. (Figure 2) In
our study, we explored the impact of client adaptation on counterfactual
generation within FL environments, as detailed in Section 4.2. Client-
personalisation is key whenever we need to extract client-specific
information. This adaptation can be achieved by training (or fine-tuning) a
counterfactual generator on local client’s data (which naturally happens at
each round). The effectiveness of client-specific adaptation can be measured
by the relative change in client proximity between global and client-specific
models, as shown in Figure 2. The figure shows a marked reduction in relative
proximity across the three datasets under non-IID conditions (up to 70% in the
Synthetic dataset). This suggests that client-specific counterfactuals after
adaptation are more representative of individual client datasets, providing
unique descriptors of client-specific behaviours in the CBP.

###  5.2 Explaining FL training

Figure 3: Client trajectories on Counterfactuals and Error Behavioural Planes
for Synthetic, Breast Cancer, and small-MNIST datasets, corresponding to
Inverted-loss, Crafted-noise, and Inverted-gradient attacks, respectively. The
figure highlights the deviation of the malicious client (red) from honest
clients, who tend to cluster together over time.

The trajectories in FBPs enable the identification of different client
behaviours during training. (Figure 3) Figure 3 illustrates FBPs’ client
trajectories over the last 15 rounds, where we introduced a different attack
on each dataset (from left to right: Inverted-loss, Crafted-noise, and
Inverted-gradient attacks). Each dataset was configured with five non-IID
clients and one attacker (red). In the Synthetic dataset, to highlight the
visualization of honest client clusters, the two largest clients were
subdivided into three (Client 2,3,4) and two smaller clients (Client 6,7),
respectively, forming distinct clusters. Specifically, on the EBP for the
Synthetic dataset, the attacker (Client 9) significantly deviates from the
trajectories of other clients, aiming to disrupt the server model (S). On the
CBP, clusters with similar data distributions converge, indicating similarity
in decision boundaries and client training data. In Breast Cancer and small-
MNIST, FBPs are also able to give insights on the type of attack. On the
Breast Cancer EBP, Client 6 exhibits a random trajectory around the server
trajectory, which indicates a Crafted-noise attack. In small-MNIST, both the
CBP and EBP reveal that at each round the trajectory of the attacker
consistently moves in the opposite direction to that of the server, which
indicates an Inverted-gradient attack. These insights might be useful for FL
users as they explain the nature of clients’ behaviours, allowing the
identification of as various types of attacks. Additional visualizations are
available in the Appendix B.

FBPs allow the identification of clusters of clients. (see Figure 3) In the
Synthetic dataset, CBPs’s clusters reflect client-specific data distributions
(Client 1,2-3-4,5,6-7,8 and Attacker 9). Clients sharing similar data
distributions tend to cluster closely in the CBP, which in the Synthetic
dataset is structured with adjacent slices in the feature space (for example,
Client 1 is positioned between Client 8 and Client 2). Similarly, in Breast
Cancer and small-MNIST, two primary clusters are identifiable in the CBP:
honest clients and attackers. This clustering capability is crucial for users
aiming to comprehend client characteristics without direct access to data or
models’ parameters, thereby informing strategic decisions in training a
federated model. For instance, one might consider reducing the number of
clients with identical data distributions to avoid redundancy and enhance
training efficiency. Conversely, identifying distinct clusters in FBPs can be
useful to maximise model performance (at the expense of the generalisation
ability) by forming a Clustered FL system where independent federated models
are trained using a subset of similar clients.

###  5.3 Leveraging FBPs information

FBPs offer detailed insights into client behaviors, enabling the Federated
Behavioural Shield to outperform existing state-of-the-art defense mechanisms.
(Figure 4) Our comparative analysis demonstrates that Federated Behavioural
Shields generally outperform traditional methods such as Krum, Median, and
Trimmed-mean across various datasets including Breast, Diabetes, and small-
MNIST. The only exception was in the scenario of the Inverted gradient on the
small-MNIST dataset, where Trimmed-mean performed better. Overall, our method
enhanced the performance up to 10 percentage points (pp) over Median—the most
robust aggregation baseline—when the FL system is under Label-flipping attack
and up to 16 pp when the system is not under attack on small-MNIST. The
proposed approach surpasses even a FedAvg aggregation in absence of attackers
on the Breast Cancer and on the small-MNIST datasets, under both normal and
crafted-noise conditions. These results suggest that an aggregation strategy
based on predictive performance or decision-making similarities is superior to
methods that solely consider sample count. Unlike other baselines, the
independence of our method from prior knowledge about attackers establishes it
as a robust aggregation tool in both adversarial and non-adversarial settings.
Contrary to previous studies [14, 28], we find that predictive performance
alone does not reliably identify malicious clients in non-IID settings, as
honest clients may also show consistently low performance, leading to
potential misclassification in the aggregation process. Occasionally, this
reliance on predictive performance can reduce our method’s accuracy compared
to scenarios where only counterfactual information is used. For instance,
under no-attack conditions, counterfactual information alone often offers more
effective behavioural descriptors than combining it with predictive metrics.
Further detailed analyses in Appendix A.7 indicate that using both descriptors
generally yields better results than relying solely on counterfactual
behaviour.

Figure 4: Comparative analysis of Federated Behavioural Shields and its
simpler version with only counterfactuals (cf) or predictive-performance
(error) versus Krum, Median, and Trimmed-mean defenses across five attack
types—No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-
loss—on three distinct datasets. Red dashed lines represent the accuracy
achieved using FedAvg without attackers.

##  6 Discussion

###  6.1 Related works

Federated Learning Client behaviours in FL have been analysed using integrated
visual tools [31, 44, 45] or indirectly through methods such as robust
aggregation [10, 11, 12, 13, 14, 28, 30, 32, 39, 46, 47, 48, 49, 50] and
clustering-based aggregation [51, 52, 53, 54, 55, 56, 57]. These studies
typically focus on similarities in model or gradient parameters [10, 11, 12,
13, 31, 39, 44, 45, 46, 47, 48, 49, 50, 53, 54, 55, 56, 57], under the premise
that distinct client data distributions manifest as unique model parameters.
Nevertheless, these methodologies might overlook valuable information about
how client models behave with the data, particularly in terms of predictive
performance. To mitigate this issue, previous studies adopted evaluation
methods that utilise a clean validation set on the server to delineate
clients’ behaviour. Common descriptors used in such cases are straightforward
metrics such as accuracy, error rates, and loss [14, 31, 44, 45, 51, 52].
Despite their utility, these metrics might not be able to reveal behavioural
patterns, such as those involved in the decision-making processes of models,
which may suggest subtle similarities or manipulations. Notably, Wang et al.
[44] introduced a post-hoc explainable approach, Grad-CAM [58], to explain
client model behaviours during training. However, Grad-CAM is limited to CNN
models and provides primarily qualitative visual insights, which cannot be
easily automated and thus still require human intervention. Similarly,
SentiNet [32] uses Grad-CAM to detect potentially malicious regions in input
images, yet client behaviour assessments continue to rely primarily on
evaluating predictive performance on manipulated and unmanipulated images. To
the best of our knowledge, this work represents the first systematic attempt
to formalise the evolving dynamics of clients in FL, showing how behavioural
client trajectories affect the predictive performance and decision-making
processes of the global model.

###  6.2 Limitations and future works

The primary constraint of our framework lies in its assumption that the server
possesses a minimal validation set for querying client models. While this
assumption is common across various methods, it can be mitigated as
demonstrated in Wang et al. [44], by generating synthetic data points. To this
end, our approach might already integrate a potential mechanism as
differentiable counterfactual methods can be used to generate synthetic data
[59]. A downside of counterfactual generators is that their concurrent
training in FL systems introduces an overhead both in the optimisation
objective and client-server communications. While in our experiments we did
not observe a decreased predictive performance, this overhead may lead to
higher computational demands and more advanced optimisation strategies in more
complex scenarios. Future works may consider the extensive information
available from FBPs to investigate other automatic strategies to optimise the
learning process, such as the development of Clustered FL among clusters of
clients and the fine-grained categorization of attack types. Incorporating
additional behavioural planes may also enhance the specificity of the FBPs
explanations.

###  6.3 Conclusions

In this work, we proposed Federated Behavioral Planes, a method to explain the
dynamics of FL systems and client behaviours. This innovative method allows to
visualise, track, and analyse client behaviours based on specific
characteristics. Our focus was twofold: examining predictive performance by
analysing prediction errors and investigating the decision-making process
through counterfactual generation. The results of our experiments showed that
Federated Behavioural Planes enable to track client behaviours over time,
cluster similar clients, and identify clients’ contributions to the global
model with respect to a specific descriptor. Based on Federated Behavioural
Planes’ information, we introduced a novel robust aggregation mechanism that
improves existing state-of-the-art methods by not requiring prior knowledge of
the attacker. This work lays the foundation to explain the evolution of client
behaviours, with the potential to enhance reliability and control over FL
systems.

## References

  * hip [1996] Health insurance portability and accountability act.  Public Law 104-191, 104th Congress, 1996.
  * gdp [2018] General data protection regulation.  Regulation (EU) 2016/679 of the European Parliament and of the Council, 2018.
  * Kairouz [2021] Peter et al. Kairouz.  Advances and open problems in federated learning.  _Foundations and Trends® in Machine Learning_ , 14(1-2):1–210, 2021.
  * Lu et al. [2018] Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang.  Learning under concept drift: A review.  _IEEE Transactions on Knowledge and Data Engineering_ , 31(12):2346–2363, 2018.
  * Chang and Shokri [2023] Hongyan Chang and Reza Shokri.  Bias propagation in federated learning.  _arXiv preprint arXiv:2309.02160_ , 2023.
  * Liu et al. [2022] Pengrui Liu, Xiangrui Xu, and Wei Wang.  Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives.  _Cybersecurity_ , 5(1):4, 2022.
  * Lyu et al. [2022] Lingjuan Lyu, Han Yu, Qiang Yang, et al.  Privacy and robustness in federated learning: Attacks and defenses.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2022\.
  * Lyu et al. [2020] Lingjuan Lyu, Han Yu, and Qiang Yang.  Threats to federated learning: A survey.  _arXiv preprint arXiv:2003.02133_ , 2020.
  * Bhagoji et al. [2019] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo.  Analyzing federated learning through an adversarial lens.  In _International Conference on Machine Learning_. PMLR, 2019.
  * Shejwalkar and Houmansadr [2021] Virat Shejwalkar and Amir Houmansadr.  Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning.  In _Proceedings of the Network and Distributed System Security Symposium (NDSS)_ , 2021.
  * Yin et al. [2018] Dong Yin, Yu-Xiang Chen, Ramchandran Kannan, and Peter Bartlett.  Byzantine-robust distributed learning: Towards optimal statistical rates.  In _Proceedings of the International Conference on Machine Learning_ , ICML. PMLR, 2018.
  * Blanchard et al. [2017] Peva Blanchard et al.  Machine learning with adversaries: Byzantine tolerant gradient descent.  In _Advances in Neural Information Processing Systems_ , volume 30, 2017.
  * Guerraoui and Rouault [2018] Rachid Guerraoui and Sébastien Rouault.  The hidden vulnerability of distributed learning in byzantium.  In _International Conference on Machine Learning_. PMLR, 2018.
  * Fang et al. [2020] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong.  Local model poisoning attacks to Byzantine-Robust federated learning.  In _29th USENIX Security Symposium (USENIX Security 20)_ , 2020.
  * Zeng and et al. [2023] Yi Zeng and et al.  Meta-Sift: How to sift out a clean subset in the presence of data poisoning?  In _32nd USENIX Security Symposium (USENIX Security 23)_ , 2023.
  * Vanneschi et al. [2014] Leonardo Vanneschi, Mauro Castelli, and Sara Silva.  A survey of semantic methods in genetic programming.  _Genetic Programming and Evolvable Machines_ , 15(2):195–214, January 2014.  ISSN 1573-7632.  doi: 10.1007/s10710-013-9210-0.  URL http://dx.doi.org/10.1007/s10710-013-9210-0.
  * Moraglio et al. [2012] Alberto Moraglio, Krzysztof Krawiec, and Colin G. Johnson.  _Geometric Semantic Genetic Programming_ , page 21–31.  Springer Berlin Heidelberg, 2012.  ISBN 9783642329371.  doi: 10.1007/978-3-642-32937-1_3.  URL http://dx.doi.org/10.1007/978-3-642-32937-1_3.
  * Mouret and Clune [2015] Jean-Baptiste Mouret and Jeff Clune.  Illuminating search spaces by mapping elites.  _CoRR_ , abs/1504.04909, 2015.  URL http://arxiv.org/abs/1504.04909.
  * Wachter et al. [2017] Sandra Wachter, Brent Mittelstadt, and Chris Russell.  Counterfactual explanations without opening the black box: Automated decisions and the gdpr.  _Harv. JL & Tech._, 31:841, 2017.
  * McMahan et al. [2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-Efficient Learning of Deep Networks from Decentralized Data.  In _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ , pages 1273–1282. PMLR, April 2017.
  * Shafahi et al. [2018] Ali Shafahi, W. Ronny Huang, Christoph Studer, et al.  Poison frogs! targeted clean-label poisoning attacks on neural networks.  In _Advances in Neural Information Processing Systems_ , volume 31, 2018.
  * Bagdasaryan et al. [2020] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov.  How to backdoor federated learning.  In _International Conference on Artificial Intelligence and Statistics_. PMLR, 2020.
  * Baruch et al. [2019] Gilad Baruch, Moran Baruch, and Yoav Goldberg.  A little is enough: Circumventing defenses for distributed learning.  In _Advances in Neural Information Processing Systems_ , volume 32, 2019.
  * Xie et al. [2020] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta.  Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation.  In _Uncertainty in Artificial Intelligence_. PMLR, 2020.
  * Gupta et al. [2023] Prajjwal Gupta, Fan Jin, Bingyan Li, Yuzheng Wu, et al.  A novel data poisoning attack in federated learning based on inverted loss function.  _Computers & Security_, 130:103270, 2023.
  * Ren et al. [2018] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun.  Learning to reweight examples for robust deep learning.  In _International Conference on Machine Learning_. PMLR, 2018.
  * Wang et al. [2020] Hongyi Wang, Jared Kaplan, Keith Saab, Ben Goodman, Junfeng Bai, et al.  Attack of the tails: Yes, you really can backdoor federated learning.  _Advances in Neural Information Processing Systems_ , 33:16070–16084, 2020.
  * Xie et al. [2019] Cong Xie, Sanmi Koyejo, and Indranil Gupta.  Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance.  In _International Conference on Machine Learning_ , page PMLR, 2019\.
  * Li et al. [2019] Suyi Li et al.  Abnormal client behavior detection in federated learning.  _arXiv preprint arXiv:1910.09933_ , 2019.
  * Cao et al. [2020] Xiaoyu Cao et al.  Fltrust: Byzantine-robust federated learning via trust bootstrapping.  _arXiv preprint arXiv:2012.13995_ , 2020.
  * Meng et al. [2021] Linhao Meng, Yating Wei, Rusheng Pan, Shuyue Zhou, Jianwei Zhang, and Wei Chen.  Vadaf: Visualization for abnormal client detection and analysis in federated learning.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ , 11(3-4):1–23, 2021.
  * Chou et al. [2020] Edward Chou, Florian Tramer, and Giancarlo Pellegrino.  Sentinet: Detecting localized universal attacks against deep learning systems.  In _2020 IEEE Security and Privacy Workshops (SPW)_. IEEE, 2020.
  * Zhang et al. [2023] Hengzhe Zhang, Qi Chen, Alberto Tonda, Bing Xue, Wolfgang Banzhaf, and Mengjie Zhang.  MAP-Elites with Cosine-Similarity for Evolutionary Ensemble Learning.  In _26th European Conference, EuroGP_ , volume 13986 of _Lecture Notes in Computer Science_ , pages 84–100, Brno, Czech Republic, April 2023. Springer Nature Switzerland.  doi: 10.1007/978-3-031-29573-7\\_6.  URL https://hal.science/hal-04230184.
  * Guyomard et al. [2023] Victor Guyomard, Françoise Fessant, Thomas Guyet, Tassadit Bouadi, and Alexandre Termier.  Vcnet: A self-explaining model for realistic counterfactual generation.  In Massih-Reza Amini, Stéphane Canu, Asja Fischer, Tias Guns, Petra Kralj Novak, and Grigorios Tsoumakas, editors, _Machine Learning and Knowledge Discovery in Databases_ , pages 437–453, Cham, 2023. Springer International Publishing.  ISBN 978-3-031-26387-3.
  * Pawelczyk et al. [2020] Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci.  Learning model-agnostic counterfactual explanations for tabular data.  In _Proceedings of The Web Conference 2020_ , WWW ’20. ACM, April 2020.  doi: 10.1145/3366423.3380087.  URL http://dx.doi.org/10.1145/3366423.3380087.
  * Wolberg et al. [1995] William Wolberg, Olvi Mangasarian, Nick Street, and W. Street.  Breast Cancer Wisconsin (Diagnostic).  UCI Machine Learning Repository, 1995.  DOI: https://doi.org/10.24432/C5DW2B.
  * Centers for Disease Control and Prevention (2017) [CDC] Centers for Disease Control and Prevention (CDC).  CDC Diabetes Health Indicators Dataset, 2017.
  * Deng [2012] Li Deng.  The mnist database of handwritten digit images for machine learning research.  _IEEE Signal Processing Magazine_ , 29(6):141–142, 2012.
  * Li et al. [2022] Yanli Li et al.  Enhancing federated learning robustness through clustering non-iid features.  In _Proceedings of the Asian Conference on Computer Vision_ , 2022\.
  * Shi et al. [2022] Junyu Shi et al.  Challenges and approaches for mitigating byzantine attacks in federated learning.  In _2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)_. IEEE, 2022.
  * Tolpegin et al. [2020] Vale Tolpegin et al.  Data poisoning attacks against federated learning systems.  In _Proceedings of the 25th European Symposium on Research in Computer Security (ESORICS)_ , volume 25 of _Computer Security - ESORICS 2020_ , pages 25–40, Guildford, UK, 2020. Springer International Publishing.
  * Lin et al. [2019] Jierui Lin, Min Du, and Jian Liu.  Free-riders in federated learning: Attacks and defenses.  _arXiv preprint arXiv:1911.12560_ , 2019.
  * Karimireddy et al. [2020] Sai Praneeth Karimireddy, Lie He, and Martin Jaggi.  Byzantine-robust learning on heterogeneous datasets via bucketing.  _arXiv preprint arXiv:2006.09365_ , 2020.
  * Wang et al. [2022] Xumeng Wang et al.  Hetvis: A visual analysis approach for identifying data heterogeneity in horizontal federated learning.  _IEEE Transactions on Visualization and Computer Graphics_ , 29(1):310–319, 2022.
  * Li et al. [2021] Quan Li et al.  Inspecting the running process of horizontal federated learning via visual analytics.  _IEEE Transactions on Visualization and Computer Graphics_ , 28(12):4085–4100, 2021.
  * Fung et al. [2020] Clement Fung, Chris JM Yoon, and Ivan Beschastnikh.  The limitations of federated learning in sybil settings.  In _23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020)_ , 2020.
  * Chen et al. [2018] Yudong Chen, Lili Su, and Jiaming Xu.  Distributed statistical machine learning in adversarial settings: Byzantine gradient descent.  In _Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems_ , 2018.
  * Mao et al. [2021] Yunlong Mao et al.  Romoa: Robust model aggregation for the resistance of federated learning to model poisoning attacks.  In _Computer Security–ESORICS 2021: 26th European Symposium on Research in Computer Security, Darmstadt, Germany, October 4-8, 2021, Proceedings, Part I_ , volume 26. Springer International Publishing, 2021.
  * Lewis et al. [2023] Cody Lewis, Vijay Varadharajan, and Nasimul Noman.  Attacks against federated learning defense systems and their mitigation.  _Journal of Machine Learning Research_ , 24(30):1–50, 2023.
  * Jebreel et al. [2024] Najeeb Moharram Jebreel et al.  Lfighter: Defending against the label-flipping attack in federated learning.  _Neural Networks_ , 170:111–126, 2024.
  * Ghosh et al. [2020] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran.  An efficient framework for clustered federated learning.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, _Advances in Neural Information Processing Systems_ , volume 33, pages 19586–19597. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper_files/paper/2020/file/e32cc80bf07915058ce90722ee17bb71-Paper.pdf.
  * Morafah et al. [2023] Mahdi Morafah, Saeed Vahidian, Weijia Wang, and Bill Lin.  Flis: Clustered federated learning via inference similarity for non-iid data distribution.  _IEEE Open Journal of the Computer Society_ , 4:109–120, 2023.  doi: 10.1109/OJCS.2023.3262203.
  * Sattler et al. [2021] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek.  Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints.  _IEEE Transactions on Neural Networks and Learning Systems_ , 32(8):3710–3722, 2021.  doi: 10.1109/TNNLS.2020.3015958.
  * Briggs et al. [2020] Christopher Briggs, Zhong Fan, and Peter Andras.  Federated learning with hierarchical clustering of local updates to improve training on non-iid data.  In _2020 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–9, 2020.  doi: 10.1109/IJCNN48605.2020.9207469.
  * Long et al. [2023] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang.  Multi-center federated learning: clients clustering for better personalization.  _World Wide Web_ , 26(1):481–500, 2023.  doi: 10.1007/s11280-022-01046-x.  URL https://doi.org/10.1007/s11280-022-01046-x.
  * Gong et al. [2022] Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen.  Adaptive client clustering for efficient federated learning over non-iid and imbalanced data.  _IEEE Transactions on Big Data_ , pages 1–1, 2022.  doi: 10.1109/TBDATA.2022.3167994.
  * Duan et al. [2022] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren.  Flexible clustered federated learning for client-level data distribution shift.  _IEEE Transactions on Parallel and Distributed Systems_ , 33(11):2661–2674, 2022.  doi: 10.1109/TPDS.2021.3134263.
  * Selvaraju et al. [2017] Ramprasaath R. Selvaraju et al.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE International Conference on Computer Vision_ , 2017.
  * Kingma and Welling [2022] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes, 2022.
  * Jin and Han [2010] Xin Jin and Jiawei Han.  K-means clustering.  In Claude Sammut and Geoffrey I. Webb, editors, _Encyclopedia of Machine Learning_ , pages 563–564. Springer US, Boston, MA, 2010.  ISBN 978-0-387-30164-8.  doi: 10.1007/978-0-387-30164-8_425.
  * Beutel et al. [2020] Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane.  Flower: A friendly federated learning research framework.  _arXiv preprint arXiv:2007.14390_ , 2020.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830, 2011\.
  * Hunter [2007] J. D. Hunter.  Matplotlib: A 2d graphics environment.  _Computing in Science & Engineering_, 9(3):90–95, 2007.  doi: 10.1109/MCSE.2007.55.
  * Waskom [2021] Michael L. Waskom.  seaborn: statistical data visualization.  _Journal of Open Source Software_ , 6(60):3021, 2021.  doi: 10.21105/joss.03021.  URL https://doi.org/10.21105/joss.03021.
  * Wes McKinney [2010] Wes McKinney.  Data Structures for Statistical Computing in Python.  In Stéfan van der Walt and Jarrod Millman, editors, _Proceedings of the 9th Python in Science Conference_ , pages 56 – 61, 2010.  doi: 10.25080/Majora-92bf1922-00a.

##  Appendix A Experimental setup

###  A.1 Datasets

In our experiments, we employ four distinct datasets: a Synthetic dataset
(tabular); the Breast Cancer Wisconsin dataset [36] (tabular); the Diabetes
Health Indicator dataset [37] (tabular); and the MNIST dataset [38] (image).
The Synthetic dataset consists of two features randomly extracted from a range
of -5 to +5. As shown in Figure 5.a, we partition this feature space into $K$
slices, where $K$ equals the number of clients. Each slice corresponds to a
specific client. We assign a label of 1 to all data points where $x_{1}>\alpha
x_{2}$, effectively creating a linear decision boundary that varies with
$\alpha$. For each client, we generally draw 1000 samples. This controlled
setup allows us to precisely manipulate and visually understand the data
distribution across different clients, as illustrated in Figure 5a. The Breast
Cancer dataset contains data from 569 patients with 30 continuous variables
derived from digitised images of a fine needle aspirate of a breast mass. The
variables describe characteristics of the cell nuclei present in the image
(e.g., radius, area, perimeter), aimed at predicting breast cancer. The
Diabetes dataset comprises data from 70,692 patients, encompassing
questionnaire-based variables (e.g., smoking, physical activity, fruit intake)
and medical measurements (e.g., BMI, cholesterol levels), with a total of 21
features, to predict the presence of diabetes. The MNIST dataset is a
comprehensive database of handwritten digits frequently used to benchmark
image classification algorithms. Each image is a 28x28 pixel grayscale
representation of a digit, ranging from 0 to 9. To increase task difficulty
and highlight client differences in performance, our experiments utilise only
10,000 of the available 70,000 images, referred to as small-MNIST.
Additionally, we transform these images into color, randomly assigning colors
with equal probabilities: red, green, or blue. To explore the decision-making
process through counterfactuals, we employed a ResNet-18 architecture to
extract 1,000 features from each image.

IID distributions across clients are achieved by randomly selecting samples
from each dataset. In contrast, non-IID distributions are created using KMeans
clustering [60] to form $K$ clusters of samples within each class—10 for the
Synthetic dataset and 5 for others. We then assign the samples of the two
nearest clusters from different classes to each client. To ensure each
client’s distribution is equally represented, we randomly extract 15% of the
samples from each client and combine them into the test set to assess the
performance of our experiments. We create a clean validation set on the server
by extracting 89 samples for Breast Cancer and 500 samples for other datasets.
Additionally, we partition each client’s dataset locally, allocating 80% for
training and 20% for validation.

###  A.2 Model configuration

In our work, the model is composed of a predictor, any Deep Neural Network,
which in our is a multi-layer perceptron that takes tabular data as input and
makes a prediction. It is composed of five layers with hidden dimension equal
to 512, 256, 256, 64, respectively. In addition, in our work, the model is
also composed of a counterfactual generator, which can be implemented in
different ways. The only requirement is that it can be trained concurrently
with the main model, in an end-to-end fashion. To generalise our results, we
tested the model with two different counterfactual generators. In Section 5,
we used a counterfactual generator designed by us that explicitly optimises
the generation of counterfactual with respect to specific labels using two
VAEs, named CFGen and described in Appendix A.3. On the other hand, in
Appendix B we also tested VCNet [34], which could generate counterfactual
training a VAE, implementing it according to the original paper. In the
context of the Federated Behavioral Space creation, PCA is employed as the
dimensionality reduction technique $\psi_{n\to 2}$ within the EBP, centering
the reduced space around a zero vector, which signifies the absence of errors.
For CBP, tSNE is utilised as the dimensionality reduction method $\psi_{K\to
2}$, ensuring the preservation of distances between clients and their
respective clusters. Finally, in terms of the implementation of Federated
Behavioural Shields, at each round $t$, the score $s^{(k)}(t)$ for each client
$k$ is computed using a moving average with a window length of $L$, as follow:

$s^{(k)}(t)=\frac{1}{L}\sum_{i=0}^{L-1}s^{(k)}(t-i)$ (22)

Additionally, client exclusion ($s^{(k)}(t)=0$) is automatically triggered if
a client exhibits a score approaching zero (i.e., $<10^{-7}$) within the
specified window.

###  A.3 CFGen Architecture

CFGen is a latent variable model that generates counterfactuals through
variational inference. To this end, we have two random variables $z$ and
$z^{\prime}$. These variables represent latent factors of variation whose
probability distributions are easier to model and sample compared to those for
$x$ and $x^{\prime}$. We also include dependencies from $y$ to the
counterfactual latent distribution $z^{\prime}$ in order to explicitly model
the dependency of $z^{\prime}$ on the class labels, resulting in the following
overall probabilistic graphical model:

$x$$\hat{x}$$y$$x^{\prime}$$y^{\prime}$$z$$z^{\prime}$

(23)

This way, the generative distribution factorises as:

$\displaystyle
p(\hat{x},y,z,x^{\prime},y^{\prime},z^{\prime},x)=p(\hat{x},y|z)p(x^{\prime},y^{\prime}|z^{\prime})p(x|z)p(\mathbf{z}|\hat{x},y)$
(24) $\displaystyle p(\hat{x},y|z)=p(y|\hat{x})p(\hat{x}|z),\quad
p(x^{\prime},y^{\prime}|z^{\prime})=p(y^{\prime}|x^{\prime})p(x^{\prime}|z^{\prime}),\quad
p(\mathbf{z}|\hat{x},y)=p(z)p(z^{\prime}|z,\hat{x},y)$ (25)

In our approach, $p(y|\hat{x})$ and $p(y^{\prime}|x^{\prime})$ are the task
predictor; $p(\hat{x}|z)$ and $p(x^{\prime}|z^{\prime})$ are the same decoder.
In practice, we assume that the input $x$ is always observed at test time,
making the term and $p(x|z)$ irrelevant. Finally, $p(z)$ is a standard normal
prior distribution and $p(z^{\prime}|z,\hat{x},y)$ is a learnable normal prior
whose mean and variance are parametrised by a pair of neural networks
$\phi_{p\mu}$ and $\phi_{p\sigma}$.

Amortised inference. CFGen amortise inference needed for training by
introducing two approximate Gaussian posteriors $q(z|\hat{x})$ and
$q(z^{\prime}|z,\hat{x},y,y^{\prime})$ whose mean and variance are
parametrised by a pair of neural networks $(\phi_{\mu},\phi_{\sigma})$
($(\phi_{\mu^{\prime}},\phi_{\sigma^{\prime}})$, respectively).

Optimization problem. In order to obtain these counterfactuals, it is
important to optimise their generation during training. CFGen is trained to
maximise the log-likelihood of tuples $(\hat{x},y,y^{\prime})$, while
observing $x$. Following a variational inference approach, we optimise the
evidence lower bound of the log-likelihood, which results in the following
objective function to maximise:

$\displaystyle\mathcal{L}=$ $\displaystyle\overbrace{\mathbb{E}_{z\sim
q(z|x)}[\log p(\hat{x}|z)]+\log p(y|\hat{x})}^{\text{ reconstruction of
$\hat{x}$ and $y$}}-\overbrace{D_{KL}[q(z|x)||p(z)]}^{\text{ prior
regularization on $z$}}$ $\displaystyle+$
$\displaystyle\overbrace{\mathbb{E}_{z,z^{\prime},x^{\prime}\sim
p(x^{\prime}|z^{\prime})q(z^{\prime}|\alpha)q(z|x))}[\log
p(y^{\prime}|x^{\prime})]}^{\text{ reconstruction of
$y^{\prime}$}}-\overbrace{D_{KL}[q(z^{\prime}|\alpha)||p(z^{\prime}|z,\hat{x},y)]}^{\text{prior
regularization on $z^{\prime}$ }}$ (26)

where $D_{KL}$ is the Kullback–Leibler divergence and
$\alpha=(z,\hat{x},y,y^{\prime})$. Moreover, in order to enforce the
counterfactuals to be as close as possible to the initial input, we add an
additional term to the objective:

$\displaystyle\mathcal{L}_{dz}=\overbrace{-D_{KL}\left[q(z|x)||q(z^{\prime}|\alpha)\right]}^{\text{posterior
distance}}-\overbrace{D_{KL}\left[p(z)||p(z^{\prime}|z,\hat{x},y)\right]}^{\text{prior
distance}}$ (27)

###  A.4 Training configuration

Gradient Descent was employed as the optimisation algorithm, with a batch size
equivalent to the dimension of the training dataset. Both the momentum and
learning rate were set at 0.9 and 0.01, respectively. For centralised training
scenarios, the model was trained over 1,000 epochs. The Flower library was
utilised to implement FL [61]. In all federated experiments, except for those
evaluating local epochs, we employed 2 local epochs. During the assessment of
various defense mechanisms, the number of communication rounds was capped at
200 and the window length for the moving average of 30 rounds. Additionally,
after evaluating the influence of test size on the server (see Appendix B),
test sizes were set at 89 for Breast Cancer and 500 for the other datasets.
For comparisons with centralised training, 1,000 rounds were used. For client
personalization, the generator was trained across 25 local epochs. In each
experiments, performance metrics were evaluated on the model that exhibited
the lowest aggregated loss during training. The aggregated loss represents the
weighted average of client losses, evaluated on each client’s local validation
set, proportional to the respective number of samples.

###  A.5 Code, licenses and hardware

For our experiments, we implement all baselines and methods in Python 3.9 and
relied upon open-source libraries such as PyTorch 2.2 [62] (BSD license),
Sklearn 1.4 [63] (BSD license), Flower 1.6 [61] (Apache License). In addition,
we used Matplotlib [64] 3.8.2 (BSD license) and Seaborn [65] 0.13 (BSD
license) to produce the plots shown in this paper. Data processing is
performed using Pandas [66] 2.2 (BSD license). The three datasets we used are
freely available on the web with licenses: Breast Cancer Wisconsin (CC BY-NC-
SA 4.0 license), Diabetes Health Indicators (CC0 licence), and MNIST (GNU
license).

We will publicly release the code with all the details used to reproduce all
the experiments under an MIT license. All experiments were conducted on a
workstation equipped with an NVIDIA RTX A6000 GPU, two AMD EPYC 7513 32-Core
processors, and 512 GB of RAM.

###  A.6 Robust aggregation baselines

This section details the robust aggregation methods used as baselines in our
experiments. These methods are designed to mitigate the effects of poisoning
attacks by malicious clients and include:

  * •

Median: This approach performs aggregation using the median of the client
updates [11]. Compared to the mean, the median is less affected by outliers.
Theoretical guarantees on the robustness of Median aggregation are provided in
[11], while empirical evidence demonstrates that it exhibits better robustness
than the more sophisticated Krum [14].

  * •

Trimmed-mean: This approach [11] aggregates each dimension of input gradients
separately. For each dimension $i$, it sorts the values of the $i^{th}$
dimension of all gradients, removes the $\beta_{t}$ largest and smallest
values, and averages the remaining values to obtain the aggregate for
dimension $i$. We use a default $\beta_{t}$ equal to 20% of the number of
clients.

  * •

Krum: Krum [12] is based on the intuition that malicious gradients must be far
from benign gradients to poison the global model. Assuming knowledge of the
upper bound on the number of malicious clients $m$, Krum selects the gradient
from the set of $K$ input gradients that is closest to its $K-m-2$ nearest
neighbors in terms of the squared Euclidean norm.

###  A.7 Federated attacks

This section outlines and provides additional details about the federated
attack scenarios implemented into our experiments, focusing on both model and
data poisoning without prior knowledge of honest client data or server
aggregation methods. We implemented these attacks assuming 20% of the clients
are malicious:

  * •

Label-flipping (Data Poisoning): In a multiclass scenario, this attack changes
all samples in the dataset with a source class $c_{src}$ into a target class
$c_{target}$ [26, 41]. In our implementation, which involves binary
classification, label flipping is performed as $1-c_{source}$ for all samples.

  * •

Inverted-loss (Data Poisoning): This attack aims to create an update that
maximises the loss and, consequently, causes a significant drop in accuracy
[25]. The implemented loss function follows the algorithm outlined in [25].

  * •

Crafted-noise (Model Poisoning): Inspired by the Free-riding attack described
in [42], this attack aims for stealthiness by adding noise to the previous
model received from the server, $w^{t}$. The noise is added as follows:
$w^{*}=w^{t}+\mathcal{N}(0,\beta\cdot\sigma(w^{t}))$, where $\sigma(w^{t})$
represents the standard deviation of $w^{t}$ and $\beta$ is a scale factor. We
tested different $\beta$ values: 0.3, 0.5, 0.8, 1.2, and 1.6.

  * •

Inverted-gradient (Model Poisoning): This attack modifies the inner product
between the true gradient and the malicious gradient updates sent by the
attacker, affecting the alignment with the true gradient [24, 43]. In our
implementation, the malicious gradient is inverted: $-\nabla w$. Given the
lack of information about honest clients, the true gradient is taken from the
previous update sent by the server, representing the previous improvement of
the global model.

For both data poisoning attacks, the attacker uses a local dataset drawn from
the same distribution as other clients and with an average number of samples.

##  Appendix B Additional experiments and analysis

#### Client-specific adaptation in IID and non-IID scenario

As depicted in Figure 2, there is a significant reduction in the relative
proximity measure between global and client-specific models across all three
datasets on average, indicating a high degree of customization in the client
models. Nonetheless, a few clients are still unable to tailor the model
effectively to their specific distribution. To explore this phenomenon, we
visually represent the generated counterfactuals pre-adaptation in Figure 5.a,
showcasing models with the highest and lowest degrees of client-specific
adaptation in Figures 5.c and 5.d, respectively. Notably, the most adapted
model corresponds to client 4, whose data is almost perpendicular to the class
boundary. In contrast, client 8, whose data distribution lies close to the
boundary, struggles to adapt, remaining akin to the pre-adaptation conditions.
This could highlight an increased capacity in personalisation in clients whose
data is perpendicular to the learnt decision boundary, also indicating that
personalization varies among clients, thereby offering more insights on the
behaviour of the clients.

((a))

((b))

((c))

((d))

Figure 5: (a) Synthetic dataset. (b) Counterfactuals generated by the server
on the synthetic test set. (c) Counterfactuals for Client 4 after adaptation.
(d) Counterfactuals for Client 8, similar to (c). Notably, Client 4, with a
data distribution perpendicular to the decision boundary, achieves effective
adaptation, whereas Client 8 encounters more challenges.

Furthermore, we analyse the effects of client-specific adaptation under both
IID and non-IID conditions. As shown in Figure 6, in contrast to non-IID,
under IID conditions where data distributions are uniform between clients, the
enhancements were minimal, indicating an alignment between global and local
optima. The lack of customization implies that all client models share a
similar decision-making processes, correctly reflecting with their IID
condition. However, this setting also facilitates the identification of
outliers or anomalous clients, which exhibit a higher degree of customization.

Figure 6: Relative variation of client-proximity across Diabetes, Breast
Cancer, and Synthetic datasets in both IID and non-IID settings. Client-
personalization is particularly effective in non-IID settings.

#### Architecture Independence in Counterfactual Generation

To explore the independence of Federated Behavioural Planes (FBPs) from the
counterfactual generator, we evaluate our methodology with an alternative
model named VCNet [34], maintaining identical experimental conditions. As
illustrated in Table 2, VCNet achieves comparable performance in both
Federated Learning (FL) and Centralised Learning (CL) configurations, where
data from all clients is placed on a single machine. Similarly to our
counterfactual generator, the results demonstrate that the privacy-preserving
training strategy (i.e., FL) does not compromise the VCNet’s effectiveness
compared to the ideal CL condition. Operating under similar privacy
constraints, FL enables VCNet to learn from the diverse distributions of
client data, thereby surpassing the performance of the Local CL approach,
where each client independently trains their own model.

To assess VCNet’s capacity for adapting the counterfactual distribution to
individual clients, we conduct client-specific adaptations starting from the
global model achieved through FL. Figure 7 illustrates the relative variation
in client-proximity across three datasets: Diabetes, Breast Cancer, and
Synthetic. The observed significant reduction in proximity indicates that
VCNet can be effectively personalised to each client’s distribution, thereby
confirming its utility in accurately reflecting client behaviours during the
training process.

Table 2: Comparison of model performance in Local Centralised, Federated Learning, and Centralised Learning (i.e., privacy-intrusive) for non-IID setting using VCNet [34] Metric | Dataset | Local CL | CL | FL
---|---|---|---|---
Accuracy (↑) | Diabetes |  56.8$\pm$0.0% |  73.8$\pm$0.0% |  73.9$\pm$0.0%
| Breast Cancer |  84.0$\pm$0.2% |  97.0$\pm$0.3% |  97.5$\pm$0.6%
| Synthetic |  74.6$\pm$0.1% |  99.5$\pm$0.1% |  99.8$\pm$0.1%
Validity (↑) | Diabetes |  100$\pm$0% |  100$\pm$0% |  100$\pm$0%
| Breast Cancer |  100$\pm$0% |  100$\pm$0% |  100$\pm$0%
| Synthetic |  100$\pm$0% |  100$\pm$0% |  100$\pm$0%
Sparsity (↓) | Diabetes |  51.1$\pm$0.1 |  42.1$\pm$0.1 |  35.4$\pm$0.0
| Breast Cancer |  2131$\pm$ 10 |  1555$\pm$4 |  1560$\pm$19
| Synthetic |  9.19$\pm$0.01 |  7.07$\pm$0.11 |  6.95$\pm$0.03
Proximity (↓) | Diabetes |  11.58$\pm$0.28 |  9.23$\pm$0.24 |  8.17$\pm$0.56
| Breast Cancer |  132.4$\pm$4.5 |  69.5$\pm$0.7 |  71.5$\pm$5.6
| Synthetic |  0.080$\pm$0.002 |  0.096$\pm$0.006 |  0.090$\pm$0.002
Figure 7: Relative variation of client-proximity across Diabetes, Breast
Cancer, and Synthetic datasets for VCNET.

#### Explaining FL training with Behavioural Planes

Figure 8 illustrates client trajectories within the FBPs for different
scenarios from those presented in Section 5. Specifically, the left side of
the figure displays the FBPs for the Synthetic dataset, now including an
attacker employing an Inverted Loss attack, while the right side shows the
FBPs for the Diabetes dataset under a Data Flip attack scenario. In both
cases, the largest clients are subdivided into smaller clients (three and
two), forming distinct clusters of clients (2,3,4 and 6,7 for Synthetic; 3,4,5
and 7,8 for Diabetes). These clusters are particularly evident on the CBP, and
even within the Synthetic dataset’s plane, one can observe clients with
similar data distributions (e.g., Client 8’s data distribution lies between
that of Clients 7 and 1). This observation holds across all clients in the
Synthetic dataset, underscoring the ability to highlight information about
data distribution similarities among clients without direct access to their
data. Moreover, Client 9 (the attacker) is noticeably diverging from the
others across all four planes. It is crucial to note the distinct behavioural
patterns of the attacker depending on the attack type. In the Inverted Loss
scenario, Client 9 moves in the exact opposite direction to the others,
converging at the same point in the EBP. Conversely, in the Diabetes dataset
with a Data Flip attack, Client 9 simply diverges from the others, each moving
towards different minima. This variation highlights the potential to identify
the type of attack deployed by the malicious client based on these behavioural
information.

Figure 8: Client trajectories on Counterfactuals and Error Behavioural Planes
for Synthetic, and Diabetes datasets, corresponding to Inverted-loss, and
Label-Flipping attacks, respectively. The figure highlights the deviation of
the malicious client (red) from honest clients, who tend to cluster together
over time.

#### The complementary roles of CBP and EBP

As previously observed in Figure 5, EBP and CBP provide orthogonal
descriptions of client behaviors. In the Synthetic dataset, CBP effectively
identifies clusters of clients with analogous data distributions. In contrast,
EBP shows all honest clients moving in the same direction, thereby obscuring
individual cluster distinctions but effectively contrasting the movements of
honest clients with that of the attacker. Similarly, in the Breast Cancer
dataset, the identification of an attack (i.e., Crafted-noise), is more
discernible through EBP, which elucidates the direction of the malicious
client relative to the server. These findings highlight the distinct yet
complementary roles that EBP and CBP play in the analysis of client behavior
across different datasets.

To illustrate the complexity of model behaviour and the utility of EBP and
CBP, consider the behavior of three different models with respect to the same
input changes. For instance, a data point with features [1, 2] is
misclassified as class 1 by Model A and correctly classified as class 0 by
Model B. However, both models would change their predictions to class 1 if the
features were altered to [2, 3]. In a similar fashion to Model A, a third
model, Model C, may also initially misclassify the point as class 1 but would
require a change in features to [2, -1] to alter its prediction to class 0,
highlighting different sensitivities and prediction dynamics in response to
input variations.

#### Federated Behavioural Shields on IID and non-IID scenarios

In our main study, we focus on the non-IID setting, which is the most
prevalent scenario for cross-silo FL in practice [39]. This setting presents
considerable challenges due to the substantial differences between updates
from honest clients, which can obscure the divergent behaviours of malicious
clients [40]. Consequently, we conducted experiments on the Breast Cancer
dataset in both IID and non-IID settings using our FBPs for comparative
analysis. As depicted in Figure 9, we initially visualized client trajectories
on both behavioural planes and within the counterfactual distance space $l$
(defined in Equation 6). The trajectories illustrate the uniform nature of
honest client behaviors on both planes, distinctly highlighting the divergent
behavior of the malicious client. Notably, the visualization of the
counterfactual distance space, employed by our Federated Behavioural Shields,
effectively identifies the malicious client. It indicates a high distance of
the attacker from all other clients, leading to a low score during the
aggregation process. To further validate our approach, we compared the
performance of our Federated Behavioural Shields under No-attack, Crafted-
noise, Inverted-gradient, Label-flipping, and Inverted-loss attacks in both
IID and non-IID settings. Table 3 demonstrates that higher accuracy is
achieved in the IID setting under almost all conditions compared to the non-
IID setting, underscoring the increased complexity of operating in non-IID
environments.

Figure 9: Client trajectories on Counterfactuals and Error Behavioural Planes for Breast Cancer datasets in IID setting under Inverted-loss attacks. The figure highlights the deviation of the malicious client (red - number 6) from honest clients, who tend to cluster together over time. Table 3: Comparison of our Federated Behavioural Shields across across five attacks types—No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-loss—on Breast Cancer dataset for both IID and non-IID configurations Condition | No attack | MP Noise | MP I.Grad | DP Flip | DP I.Loss | Mean
---|---|---|---|---|---|---
non-IID | 95.7$\pm$1.1 | 98.0$\pm$0.8 | 95.3$\pm$0.7 | 94.2$\pm$0.6 | 95.9$\pm$0.9 | 95.8$\pm$0.4
IID | 98.2$\pm$0.3 | 98.43$\pm$0.4 | 98.2$\pm$0.2 | 96.4$\pm$0.9 | 93.7$\pm$1.0 | 97.0$\pm$0.4

#### Impact of server validation set size

Considering the crucial role of the validation set size on the server in the
creation of behavioural planes, we initially examined its impact on
computational time and accuracy through 10 distinct trials. Specifically, we
evaluated our Federated Behavioural shields (without moving average) against a
malicious client executing an inverted gradient attack on the Diabetes
dataset. As depicted in Figure 10, computational time increases exponentially
with the size of the dataset. However, a dataset containing as few as 250
samples is sufficient to adequately represent the data distribution, achieving
performance on par with that observed in larger datasets. Additionally, with
fewer than 1000 samples, our method demonstrates greater efficiency compared
to the Krum algorithm.

Figure 10: Relationship between test set size, computational time, and
accuracy evaluated on Diabetes dataset against inverted gradient attack. As
test set size increases (logarithmic x-axis), computational time grows
exponentially (log10 scale, red line), while accuracy (blue line) only
slightly increases before plateauing.

#### Ablation study

We conducted an ablation study to examine the impact of various components on
the efficacy of our algorithm. These components include predictive performance
(error), decision-making process (counterfactuals), and the application of a
moving average. Table 4 presents the average accuracy and standard error under
five experimental conditions: No-attack, Crafted-noise, Inverted-gradient,
Label-flipping, and Inverted-loss, comparing different variations of our
method against baselines on the Breast Cancer and Diabetes datasets.

Notably, relying solely on predictive performance does not yield a
statistically significant benefit compared to the strongest baselines, such as
Trimmed-mean on Breast Cancer and Krum on Diabetes. In contrast,
counterfactual information provides deep insights into client behaviour,
enabling our method to outperform all baselines in identifying malicious
clients. Although the improvement is subtle, the combination of descriptors,
on average, provides better results than using counterfactuals alone.

The integration of a moving average notably enhances the performance of our
method, improving from $94.5\pm 0.5$ to $95.4\pm 0.3$ on the Breast Cancer
dataset. Smaller gains are observed on the Diabetes dataset, likely due to two
factors: the limited representativeness of the clean validation set on the
server for Breast Cancer, which consists of only 89 samples compared to 500
for Diabetes, and the small size of client datasets (77 training samples on
average), which may lead to unstable local training. By aggregating behaviour
across multiple rounds, the moving average technique helps to stabilise and
accurately assess client behaviour scores.

Table 4: Average accuracy (%) ± standard error for various defense strategies under five experimental conditions: No attack, Crafted-noise, Inverted-gradient, Label-flipping, and Inverted-loss. The table specifically compares the performance of our methods with and without the application of a moving average (MA). Defense Strategy | Breast | Diabetes
---|---|---
Krum | $91.9\pm 0.4$ | $70.6\pm 0.3$
Median | $92.6\pm 0.4$ | $67.0\pm 0.1$
Trimmed-mean | $93.4\pm 0.4$ | $69.0\pm 0.1$
Ours (error) w/o MA | $93.7\pm 0.4$ | $70.5\pm 0.8$
Ours (cf) w/o MA | $94.2\pm 0.4$ | $72.0\pm 0.1$
Ours w/o MA | $94.5\pm 0.5$ | $72.1\pm 0.2$
Ours (error) | $95.0\pm 0.4$ | $71.4\pm 0.2$
Ours (cf) | $95.4\pm 0.3$ | $72.3\pm 0.1$
Ours |  95.8 $\pm$ 0.4 |  72.4 $\pm$ 0.1

#### Impact of the window length in the moving average

We analyze the impact of window length on our Federated Behavioural Shields
and its two variations, one using only predictive performance (error) and the
other focusing solely on decision-making processes (counterfactual). The
results presented in Figure 11 depict the average accuracy across various
conditions—including No attack, Label-flipping, Inverted-loss, Crafted-noise,
and Inverted-gradient—for window lengths ranging from 3 to 30 rounds. While no
definitive patterns are clearly evident, on average, a longer window length
tends to be advantageous for accurately assessing the behavioural scores of
each client during the aggregation process. Noteworthy, the optimal accuracy
across both datasets was achieved with a window length of 30 rounds.
Particularly, a consistent and slight increase in accuracy is observed in the
Diabetes dataset when utilising only the counterfactual information.

Figure 11: Effect of window length on accuracy using Federated Behavioural
Shield. This table demonstrates the accuracy changes as a function of window
length in the moving average method across the Diabetes and Breast Cancer
datasets.

#### Impact of the local epochs in Federated Behavioural Shield

Local training epochs play a crucial role in FL, particularly in evaluating
client behaviours. An increased number of local epochs implies a greater
adaptation of the client-model to its local data distribution. For this
reason, we assess the performance of our Federated Behavioural Shields against
data poisoning attacks, including Label-flipping and Inverted-loss, across
Breast Cancer, Diabetes, and Synthetic datasets. Metrics are reported as the
mean and standard error across ten with distinct parameters’ initialization.
These attack types were selected because malicious updates are directly
influenced by the number of local epochs, similar to updates from honest
clients; differently, the behaviour of malicious clients in model poisoning
might remain unaffected. As depicted in Figure 12, the impact of local epochs
varies across datasets, with each achieving maximum accuracy at different
numbers of epochs. Generally, accuracy increases with the number of local
epochs until it reaches a peak, after which it begins to decline. This trend
may be attributed to the fact that initially, increasing the number of local
epochs enhances the detectability of anomalies in the behaviour of malicious
clients, as their models diverge more quickly from those of honest clients.
However, beyond a certain threshold, all client models become highly customd,
leading to the potential underweighting of even honest clients’ contributions
during the aggregation process on the server. This results in the system
relying predominantly on a few clients whose models are the most similar to
each other.

Figure 12: Impact of varying local epochs on accuracy for our Federated
Behavioural Shield. This table illustrates how changes in the number of local
training epochs affect accuracy across the Breast Cancer, Diabetes, and
Synthetic datasets.

#### Federated Behavioural Shields

In this section, we provide a detailed numerical breakdown of the results
depicted in Figure 4. This quantitative analysis aims to supplement the visual
data presented, offering precise values and statistical insights that underpin
the observations and conclusions discussed throughout the paper. Tables 5 for
Breast Cancer, 6 for Diabetes, and 7 for small-MNIST present a comprehensive
comparison of our Federated Behavioural Shields and its streamlined versions
using only the CBP or the EBP. These are evaluated against traditional
defenses such as Krum, Median, and Trimmed-mean across five scenarios: No
attack, Crafted-noise, Inverted-gradient, Label-flipping, and Inverted-loss.
This comparison elucidates the effectiveness of our approach under a range of
conditions, both adversarial and non-adversarial.

Table 5: Comparison of Federated Behavioural Shields and its simpler version with only counterfactuals (cf) or predictive-performance (error) versus Krum, Median, and Trimmed-mean defenses across five attacks types—No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-loss—on Breast Cancer dataset. |  | Model Poisoning | Data Poisoning |
---|---|---|---|---
| No-Attack | Crafted-Noise | Inv. Grad. | Label-Flip | Inv. Loss | Mean
Krum | 90.9$\pm$0.6 | 92.3$\pm$0.8 | 92.7$\pm$1.5 | 91.6$\pm$0.4 | 91.8$\pm$0.6 | 91.9$\pm$0.4
Median | 92.7$\pm$0.7 | 93.6$\pm$0.7 | 93.1$\pm$1.2 | 92.0$\pm$0.8 | 91.6$\pm$1.1 | 92.6$\pm$0.4
Trim | 93.2$\pm$0.5 | 96.1$\pm$0.7 | 95.5$\pm$0.4 | 91.8$\pm$1.1 | 90.4$\pm$0.8 | 93.4$\pm$0.3
Ours (error) | 96.6$\pm$0.8 | 97.7$\pm$1.2 | 95.0$\pm$0.8 | 91.4$\pm$1.0 | 94.1$\pm$0.8 | 95.0$\pm$0.4
Ours (cf) | 97.5$\pm$0.5 | 96.8$\pm$0.6 | 96.6$\pm$0.3 | 93.2$\pm$0.9 | 92.3$\pm$0.6 | 95.3$\pm$0.3
Ours | 95.7$\pm$1.1 | 98.0$\pm$0.8 | 95.3$\pm$0.7 | 94.2$\pm$0.6 | 95.9$\pm$0.9 | 95.8$\pm$0.4
Predictor | 96.6$\pm$ 0.4 | N/A | N/A | N/A | N/A | N/A
Table 6: Comparison of Federated Behavioural Shields and its simpler version with only counterfactuals (cf) or predictive-performance (error) versus Krum, Median, and Trimmed-mean defenses across five attacks types—No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-loss—on Diabetes dataset. |  | Model Poisoning | Data Poisoning |
---|---|---|---|---
| No-Attack | Crafted-Noise | Inv. Grad. | Label-Flip | Inv. Loss | Mean
Krum | 70.5$\pm$1.2 | 70.4$\pm$0.2 | 71.0$\pm$0.2 | 70.5$\pm$0.2 | 70.7$\pm$0.1 | 70.6$\pm$0.3
Median | 69.7$\pm$0.2 | 68.5$\pm$0.3 | 69.2$\pm$0.1 | 64.1$\pm$0.1 | 65.1$\pm$0.2 | 67.3$\pm$0.1
Trim | 67.9$\pm$0.1 | 69.6$\pm$0.4 | 71.7$\pm$0.2 | 65.1$\pm$0.1 | 70.5$\pm$0.2 | 69.0$\pm$0.1
Ours (error) | 71.41$\pm$0.6 | 72.0$\pm$0.5 | 72.2$\pm$0.4 | 72.3$\pm$0.2 | 69.3$\pm$0.8 | 71.4$\pm$0.2
Ours (cf) | 72.67$\pm$0.2 | 72.3$\pm$0.2 | 72.4$\pm$0.0 | 71.8$\pm$0.2 | 71.6$\pm$0.2 | 72.2$\pm$0.1
Ours | 72.47$\pm$0.3 | 73.1$\pm$0.4 | 72.3$\pm$0.0 | 72.0$\pm$0.4 | 71.9$\pm$0.3 | 72.4$\pm$0.1
Predictor | 73.5$\pm$0.2 | N/A | N/A | N/A | N/A | N/A
Table 7: Comparison of Federated Behavioural Shields and its simpler version with only counterfactuals (cf) or predictive-performance (error) versus Krum, Median, and Trimmed-mean defenses across five attacks types—No attack, Crafted-noise, Inverted-gradient, Label-flipping, Inverted-loss—on small-MNIST dataset. |  | Model Poisoning | Data Poisoning |
---|---|---|---|---
| No-Attack | Crafted-Noise | Inv. Grad. | Label-Flip | Inv. Loss | Mean
Krum | 56.3$\pm$2.4 | 60.3$\pm$1.3 | 62.0$\pm$1.5 | 59.3$\pm$2.5 | 55.9$\pm$3.0 | 58.8$\pm$1.0
Median | 68.8$\pm$1.5 | 69.5$\pm$2.0 | 63.7$\pm$1.6 | 64.2$\pm$1.7 | 64.5$\pm$1.0 | 66.1$\pm$0.7
Trim | 68.7$\pm$2.4 | 78.9$\pm$2.3 | 79.5$\pm$0.7 | 63.9$\pm$1.9 | 70.1$\pm$3.0 | 72.2$\pm$1.0
Ours (error) | 80.2$\pm$1.5 | 71.0$\pm$4.2 | 71.3$\pm$0.3 | 76.0$\pm$2.2 | 72.9$\pm$4.7 | 74.3$\pm$1.4
Ours (cf) | 84.2$\pm$0.2 | 84.1$\pm$0.4 | 74.5$\pm$1.7 | 77.8$\pm$1.7 | 80.1$\pm$0.6 | 80.1$\pm$0.5
Ours | 81.0$\pm$2.0 | 79.2$\pm$3.8 | 74.6$\pm$0.6 | 74.6$\pm$3.7 | 72.3$\pm$3.6 | 76.4$\pm$1.3
Predictor | 79.6$\pm$0.8 | N/A | N/A | N/A | N/A | N/A