# Fed-Safe: Securing Federated Learning in Healthcare Against Adversarial
Attacks

Erfan Darzi1,∗, Nanna M. Sijtsema2, P.M.A van Ooijen2
1 Harvard Medical School, Harvard University, Boston, MA, USA
2Department of Radiation Oncology, University Medical Center Groningen,
University of Groningen, The Netherlands

###### Abstract

This paper explores the security aspects of federated learning applications in
medical image analysis. Current robustness-oriented methods like adversarial
training, secure aggregation, and homomorphic encryption often risk privacy
compromises. The central aim is to defend the network against potential
privacy breaches while maintaining model robustness against adversarial
manipulations. We show that incorporating distributed noise, grounded in the
privacy guarantees in federated settings, enables the development of a
adversarially robust model that also meets federated privacy standards. We
conducted comprehensive evaluations across diverse attack scenarios,
parameters, and use cases in cancer imaging, concentrating on pathology,
meningioma, and glioma. The results reveal that the incorporation of
distributed noise allows for the attainment of security levels comparable to
those of conventional adversarial training while requiring fewer retraining
samples to establish a robust model.

††footnotetext: *Corresponding author<EMAIL_ADDRESS>
##  1 Introduction

Federated learning is a popular method for medical image analysis, due to its
promise of high accuracy while maintaining data privacy. One of its key
advantages over central storage of data is that it solves problems related to
data governance and privacy by training algorithms without exchanging
sensitive information. However, despite its limited communication overhead
compared to central storage methods, federated learning is not without its
privacy and security challenges. For example, malicious clients might be able
to acquire sensitive information from other clients or disrupt the training
process. These challenges hinder the large-scale deployment of federated
learning networks in medical image applications.

This growing threat from potential adversaries has incentivized research on
enhancing the robustness of federated networks. Various security-enhancing
methods have emerged, including adversarial training, secure aggregation, and
homomorphic encryption. Among these, adversarial training is the most popular
technique for mitigating the impact of adversarial attacks. However, these
approaches come with a fundamental drawback: they compromise the privacy of
both the model and the data [1, 2, 3]. Studies have shown that adversarial
training can amplify the efficacy of some privacy attacks which are otherwise
deemed ineffective against traditionally trained models. For example, one
study reported that adversarial training made it significantly easier to
perform multiple types of attacks, such as model inversion, data
reconstruction, and GAN-based attacks. The effectiveness of these attacks
increased by an order of magnitude when models were adversarially trained [2].

This compromise between security and privacy is commonly known as the
”robustness-privacy tradeoff of adversarial training.” It raises an important
question: Is the protection against adversarial attacks worth the high cost to
privacy?

### Our Contributions

In this paper, we provide certified and well-understood privacy guarantees to
adapt adversarial training in a federated environment. Our approach aims to
maintain adversarial robustness while minimizing privacy risks. Central to our
method is the leverage of inherent privacy guarantees in federated settings
which enables us attain privacy-preserving adversarial training.

Adversarial robustness We demonstrate that our federated adversarial
adaptation method achieves a level of robustness comparable to traditional
adversarial training. Specifically, we find that we can match or even surpass
the robustness offered by state-of-the-art adversarial training in various
scenarios.

Guaranteed Privacy Through Distributed Noise We demonstrate that our approach
not only enhances adversarial robustness but also maintains privacy. Our
modified adversarial training method utilizes distributed noise to ensure
certified privacy guarantees.

Efficient Data Utilization Our method is designed to minimize data
requirements for model retraining, which is important in medical data where
data sharing is a burden. Compared to traditional adversarial training
approaches, our adversarial adaptation strategy requires significantly less
data to train the global model, thereby achieving desired levels of robustness
with fewer data samples.

##  2 Related Works

While numerous studies have been conducted on the topic of federated learning
within the realm of medical image analysis, they have largely ignored the
aspect of security and privacy of these setups. Similarly, though there are
some research on adversarial robustness focused on medical images, these
primarily examine scenarios within a centralized computational framework. As
far as we are aware, our research stands as an initial effort in combining
these two important areas—federated learning and robustness—in the specific
context of medical imaging.

Federated Learning for Medical Image Analysis. Federated learning has been
explored in multiple studies for medical image analysis. For instance, Liu et
al. proposed a federated learning system for medical image classification that
achieved an accuracy of 89.17% for chest X-Ray,[4] using multiple hospitals as
clients[5]. Similarly, Malik et al. proposed a secure federated learning
framework for medical image classification, using a secure multi-party
computation protocol for secure aggregation and achieving an accuracy of
98.45% on the Chest X-Ray dataset[6].

Adversarial Attacks on Medical Images. Studies have shown that medical images
are vulnerable to adversarial attacks in a centralized setting. For instance,
Han et al. [7] proposed a deep learning-based system for classifying cardiac
diseases from ECG signals and assessed its vulnerability to various white-box
and black-box adversarial attacks. Similarly, Rodriguez et al.’s proposed
system for classifying chest x-ray abnormalities [8] was found to be highly
vulnerable to different types of adversarial attacks. Bortsova et al. further
studied the adversarial attacks and attack detection methods, in medical
images within a centralized setting [9].

Secure Federated Learning (SFL) has been widely studied, with research
focusing on both privacy and robustness. For example, So et al. proposed a
secure aggregation protocol for SFL to protect against various attack
scenarios [10]. Chen et al. presented an end-to-end secure aggregation
protocol to maintain client privacy while preserving model accuracy [11].
Studies concerning encryption and privacy-preserving approaches, such as
differential privacy, have also been conducted. Ma et al., for instance,
proposed a front-end framework using multi-key encryption [12], while Zhang et
al. created a platform using homomorphic encryption for secure aggregation
without considering DP or encrypted inference [13]. Sheller et al. implemented
a privacy-preserving FL segmentation model without using DP or assessing its
robustness[14], Vithana et al. showcased a segmentation task but relied on a
different technique (sparse vector) instead of DP [15], and Adnan et al.
provided a framework for FL with differential privacy concerning pathology
slides without covering secure aggregation or encrypted inference [16].
Adversarial training has been applied as an optimization technique for
individual clients in order to secure federated networks; however, this also
increases computational burden for all participants, potentially leading to
lack of convergence due to heterogeneity among models [17].

##  3 Adversarial adaptation with distributed noise

In federated learning, clients and a central server communicate via uplink and
downlink channels. The uplink channel allows clients to upload local model
parameters to the server, with the model designed to enforce
$(\epsilon,\delta)$-Differential Privacy (DP).

Parameters or functions of these parameters are uploaded, reducing exposure of
sensitive data. Differential privacy metrics evaluate the privacy cost of this
data transfer. Gaussian noise, scaled by query sensitivity, preserves the
privacy of uploaded models. Conversely, the downlink channel disseminates the
global model to the clients. Though this is an aggregate, it remains
susceptible to reverse-engineering attacks that could disclose sensitive
client data. The global privacy preserving requirements for federated learning
has been analyzed by Wei et al[18]. For the uplink, a clipping method bounds
each client’s training parameters $\mathbf{w}_{i}$ with a value $C$. Local
training at client $i$ is defined as:

$s_{\text{U}}^{\mathcal{D}_{i}}:=\mathbf{w}_{i}=\arg\min_{\mathbf{w}}{F_{i}(\mathbf{w},\mathcal{D}_{i})}=\frac{1}{|\mathcal{D}_{i}|}\sum_{j=1}^{|\mathcal{D}_{i}|}\arg\min_{\mathbf{w}}{F_{i}(\mathbf{w},\mathcal{D}_{i,j})},$
(1)

Sensitivity $\Delta s_{\text{U}}^{\mathcal{D}_{i}}$ is computed as:

$\Delta
s_{\text{U}}^{\mathcal{D}_{i}}=\max_{\mathcal{D}_{i},\mathcal{D}_{i}^{\prime}}{\|s_{\text{U}}^{\mathcal{D}_{i}}-s_{\text{U}}^{\mathcal{D}_{i}^{\prime}}\|}=\frac{2C}{|\mathcal{D}_{i}|},$
(2)

Global sensitivity in the uplink is denoted as $\Delta s_{\text{up}}$:

$\Delta s_{\text{up}}=\max_{i}\\{\Delta
s_{\text{up}}^{\mathcal{D}_{i}}\\},\quad\forall i.$ (3)

To control this sensitivity, each client should have a local dataset of at
least size $m$, leading to $\Delta s_{\text{up}}=\frac{2C}{m}$.

Noise scale $\sigma_{\text{up}}$ is introduced, and adjusted for multiple $L$
exposures as $\sigma_{\text{up}}=\frac{cL\Delta s_{\text{up}}}{\epsilon}$.

In the downlink channel, the aggregated dataset $\mathcal{D}_{i}$ is:

$s_{\text{down}}^{\mathcal{D}_{i}}\equiv\mathbf{w}=p_{1}\mathbf{w}_{1}+\cdots+p_{i}\mathbf{w}_{i}+\cdots+p_{N}\mathbf{w}_{N},$
(4)

where $1\leq i\leq N$ and $\mathbf{w}$ represents the unified parameters that
are subsequently disseminated by the server to the clients [18].
Adversarial Robustness In the context of federated learning, adversarial
robustness is crucial for enhancing the resilience of the federated model
against adversarial attacks. These attacks can compromise the privacy and
performance of the global model. To address this challenge, adversarial
training is often employed. This involves altering the training data with
small, malicious perturbations to deceive the model, aiming to make the model
more resilient in adversarial conditions.

A common measure for robustness is the adversarial loss
$\mathcal{L}_{\text{adv}}$, defined as:

$\mathcal{L}_{\text{adv}}(\mathbf{w},\mathcal{D})=\max_{\delta\in\Delta}\mathcal{L}(\mathbf{w},\mathcal{D}+\delta),$
(5)

Here, $\Delta$ is the set of allowable perturbations to the dataset
$\mathcal{D}$, and $\mathcal{L}$ is the original loss function. During
adversarial training, the objective is to minimize this adversarial loss by
iteratively updating the model parameters $\mathbf{w}$. Adversarial training
could be either done in local training, or in global server level with a
separate server dataset . This process enhances the robustness of the
federated learning model against adversarial attacks.

###  3.1 Distributed noise

The challenge of preserving privacy while training local models is a
significant concern, especially given that these models are often vulnerable
to reverse engineering and subsequent privacy breaches. Typically, these
models are trained without explicit privacy safeguards, thereby exposing them
to potential adversarial attacks aimed at decoding data from individual
clients. To mitigate these risks, we introduce a dual-faceted approach that
enhances both security and privacy.

Our distributed noise model serves as a robust solution to this challenge. It
comprises two key components:

First, we generate a series of synthetic samples to perform adversarial
adaptation on the global server. Additionally, we add distributed Gaussian
noise to the downlink channels to the client. By minimizing the adversarial
adaptation loss function, the network becomes resilient to potential
adversaries that may be present in the clients. Furthermore, this method of
noise-based protection does not interfere with local training. The adversarial
adaptation of the global model facilitates the learning of the data
distribution across all client models, thereby increasing its resilience
against potential threats.

Model Aggregation In federated learning networks, a central server aggregates
updates from multiple clients. For this study, we employ Federated Averaging
as our aggregation model, although the defense technique discussed is
applicable to other aggregation models as well. To further bolster the privacy
of our model updates, we introduce distributed noisy communication channels.

Algorithm 1 Federated adversarial adaptation

1:$X\in\mathcal{X}$, $Y\in\\{-1,1\\}$, $g_{\beta}$:
$\mathcal{X}\to\\{-1,1\\}$, $\epsilon$: privacy parameter, $\eta$: bound,
$\alpha$: step size

2:Initialize parameters: $\beta=\beta_{0}$ and $T=0$

3:while $T<T_{max}$ do

4: for each client do

5: Perform local federated training with differential privacy

6: $g_{\beta}\leftarrow g_{\beta_{T}}$

7: end for

8: Aggregate the models from all clients to create the global model

9: Update parameters: $\beta=\beta_{T+1}$ and $T=T+1$

10:end while

11:Generate adversarial samples using the global model:  $x_{adv}=x+\epsilon\
\text{sign}(\nabla_{x}\mathcal{L}(\theta,x,y))$

12:Perform federated adversarial adaptation $g_{\beta_{T+1}}$ with data set
$X_{T}$, $Y_{T}$ by performing the following optimization:
$g_{\beta_{i}}=\operatorname*{arg\,min}_{g_{\beta}}\mathcal{L}(g_{\beta},D_{i}\cup
D_{i}^{adv})$

13:for $k\in\\{1,\dots,N\\}$ do

14: Add distributed noise $\mathcal{N}(0,\sigma^{2})$ to $g_{\beta_{T+1}}$

15: Receive the noisy model $\hat{g}_{\beta_{T+1}}^{(k)}$

16:end for

Generating Adversarial Samples Within the client-side architecture, the global
model obtained from the central server undergoes an update process based on a
local dataset. To achieve this, an optimization algorithm is applied to the
model. Adversarial samples are generated using two distinct methods: First,
the global model is tested against a specific dataset to produce these
samples. The samples are generated using two methods, FGSM and PGD. The Fast
Gradient Sign Method (FGSM) is a technique aimed at generating adversarial
sample. It does so by taking a specific input $x\in\mathcal{X}$, where
$\mathcal{X}$ is the domain of all possible samples, and introducing
controlled perturbations to it. The key to this method is the use of the
gradient $\nabla$ of the model’s loss function $\mathcal{L}$[19]. The
perturbed or adversarial sample $x_{\text{adv}}$ is created in a single step
as follows:

$x_{\text{adv}}=x+\epsilon\ \text{sign}(\nabla_{x}\mathcal{L}(\theta,x,y)),$
(6)

Here, $\epsilon$ is a small positive constant that adjusts the extent of the
perturbation. The variable $\theta$ represents the trainable parameters of the
machine learning model. This approach is computationally efficient but may not
produce the most potent adversarial sample.

In contrast to FGSM, the Projected Gradient Descent Attack (PGD) provides a
more robust method for creating adversarial sample. It builds upon the
foundational principles of FGSM but employs a multi-step iterative process[20,
21]. The mathematical formulation for generating an adversarial sample using
PGD is:

$x^{i+1}=\text{clip}(x^{i}+\alpha\
\text{sign}(\nabla_{x^{i}}\mathcal{L}(\theta,x^{i},y)),$ (7)

In this equation, $i$ indicates the current iteration, and $\alpha$ is a
tunable constant that affects the magnitude of the disturbance at each
iteration. The function clip($\cdot$) ensures that the perturbations stay
within a certain limit, usually defined by $|\alpha\
\text{sign}(\nabla_{x^{i}}\mathcal{L}(\theta,x^{i},y))|\leq\eta$, where $\eta$
is another positive constant. PGD tends to generate stronger adversarial
samples at the cost of being computationally more demanding[21].

Suppose every instance $x\sim D$ corresponds to a fixed label
$y\in\\{-1,+1\\}$ and the label of $+1$ indicates that instance $x$ was
generated by an adversary. To evade the classifier, adversary would generate
an alternative instance $x^{\prime}\in X$, for which $g(x^{\prime})=-1$.
Traditionally in the field of machine learning, the focus is often on
minimizing the empirical risk, which can be formally expressed as:

$\operatorname*{arg\,min}_{\theta}\mathcal{L}(\theta)=\sum_{i}\ell(f_{\theta}(x_{i}),y_{i})$
(8)

In this equation, $\ell(\hat{y},y)$ represents the loss incurred when the
predicted outcome is $\hat{y}$ while the actual label is $y$. Here, $x_{i}$
refers to the input feature vector and $y_{i}$ corresponds to the associated
label within the training dataset.

However, the standard approach may not suffice in adversarial settings. When
dealing with malicious entities, the aim isn’t just to classify instances
correctly but also to anticipate evasive actions by these entities.
Specifically, a sample classified as malicious, denoted by $f_{\beta}(x)=+1$
in our notation, might be altered by an adversary to bypass detection.

To capture this adversarial behavior, consider a function defined as
$x^{adv}(\beta,x)$. Given a parameter vector $\beta$ and an original feature
vector $x$, this function returns a modified feature vector $x^{\prime}$ that
represents the instance after adversarial modification.

Federated Adversarial Adaptation After the adversarial samples are generated,
they are used to perform federated adversarial adaptation. The adaptation
process follows optimizing of an augmented dataset, to train a robust model
against adversarial attacks. The new model parameters are then sent to all the
clients for evaluation. Given the adversarial nature of the scenario, wherein
the adversary alters malicious instances based on the functions’s input, the
defender’s effective risk can no longer be represented solely by Equation 8.
Instead, we need to factor in the potential adversarial responses. As a
result, the global model now aims to reduce the so-called federated
adversarial adaptation loss. This loss, specific to the training data, is
expressed as:

$\min_{\theta}\mathcal{L}_{A}(\theta;x^{adv}_{i})=\sum_{i:y_{i}=-1}l(g_{\theta}(x_{i}),-1)+\sum_{i:y_{i}=+1}l(g_{\theta}(x^{adv}_{i}(\theta,x_{i}),+1)$
(9)

In this equation, $g_{\theta}(x)$ symbolizes the output of the classifier,
while $\mathcal{L}_{A}(\theta;x^{adv})$ represents the adversarial adaptation
loss. The function $x^{adv}(\theta,x)$ yields an adversarially modified
feature vector for the given input $x$ based on parameter $\theta$.

The risk function detailed in Equation 9 is constructed in a way that it
remains versatile. It can adapt to a variety of scenarios, given that we
aren’t making specific assumptions about the attacker’s function, denoted by
${x^{adv}}$.

Empirical risk evaluation To understand the outcome when the algorithm comes
to an end, we first identify the empirical risk during the final iteration of
the federated adversarial adaptation:

$\mathcal{L}^{R}_{N}(\beta,{x^{adv}})=\sum_{i\in D\cup
N}l(g_{\beta}(x_{i}),y_{i})$ (10)

Here, $N$ is a collective set of data points, integrated by the algorithm.
With this, we can now discern the relationship between
$\mathcal{L}^{R}_{N}(\beta,{x^{adv}})$ and the optimal adversarial adaptation
loss
$\mathcal{L}_{A}^{*}({x^{adv}})=\min_{\beta}\mathcal{L}_{A}(\beta,{x^{adv}})$.
Mathematically, for any parameter set $\beta$:

$\displaystyle\mathcal{L}^{R}_{N}(\beta,{x^{adv}})$
$\displaystyle\geq\mathcal{L}^{R}_{N}(\bar{\beta},{x^{adv}})$
$\displaystyle=\sum_{i:y_{i}=-1}l(g_{\bar{\beta}}(x_{i}),-1)+\sum_{i:y_{i}=+1}\sum_{j\in
N_{i}\cup x_{i}}l(g_{\bar{\beta}}(x_{i}),+1)$
$\displaystyle\geq\sum_{i:y_{i}=-1}l(g_{\bar{\beta}}(x_{i}),-1)+\sum_{i:y_{i}=+1}l(g_{\bar{\beta}}({x^{adv}}(\bar{\beta},x_{i})),+1)$
$\displaystyle\geq\min_{\beta}\mathcal{L}_{A}(\beta;{x^{adv}})=\mathcal{L}_{A}^{*}({x^{adv}})$

The logic behind the second inequality stems from the last iteration of our
algorithm, during which no new values are added. This implies that the
adversarial samples ${x^{adv}}(\beta,x_{i})$ fall within $N_{i}$ for every $i$
belonging to $I_{adv}$. This demonstrates how retraining the global model with
adversarial samples can improve the robustness of the model against
adversarial attacks. The proof shows that the empirical risk in the last
iteration of the algorithm is lower than the adversarial risk, and thus the
model is more robust against adversarial attacks.

##  4 Experiments and Results

###  4.1 Datasets

To simulate real-world conditions, we used non-independent and non-identically
distributed (non-IID) datasets across multiple clients. Data were divided into
non-overlapping chunks, each representing a segment of the general
distribution, to ensure that each client’s dataset is distinct, fulfilling the
requirements of federated learning scenarios.

In the domain of brain cancer classification, we utilized a public dataset of
MRI scans available on Kaggle [22]. This dataset comprises four classes: three
types of brain tumors and one healthy class. We used 2870 training images and
394 testing images, with 1437 and 1426 images specifically for Meningioma and
Glioma detection, respectively. The images were resized to 100$\times$100
pixels and underwent rotation, flipping, and normalization for data
augmentation.

For histopathologic cancer detection, we sourced high-resolution pathology
slices from previous work [23]. The dataset consists of 2150 images with a
62%/38% training/testing split. Metastatic tissue, when present, is centered
within a $32\times 32$ pixel region. These images were also transformed via
horizontal flipping and normalization to enhance the dataset.

Figure 1: (a) Original unmanipulated meningioma MRI. (b) Adversarial noise
generated with FGSM method with $\epsilon$ = 0.03 . (c) Manipulated meningioma
MRI (original image + noise), i.e., adversarial example.

###  4.2 Experimental Setup

Federated setup: A Convolutional Neural Network (CNN) with six layers of
convolution stacks, five fully connected layers and ReLU activation function
is used as the deep learning model (with 0.25 dropout parameter). The model is
trained with Cross-Entropy loss via SGD optimizer. For federated setting,
three clients equipped with random data with non-IID data distribution.
Federated averaging (FedAVG) method is used to aggregate the models with 20
epochs per communication round and 50 federated rounds in total, weighed based
on the size of the training dataset.
Evaluation Metrics : We use two metrics to evaluate the performance of
adversarial attacks, i.e. clean accuracy and attack success rate. Clean
accuracy is defined as the performance of models on uncorrupted test images.
Attack success rate (ASR) measures how much an adversary can change the
predicted labels produced by each model. Formally, ASR is defined as:

$ASR=\frac{1}{N}\sum\limits_{i=1}^{N}(Pre\text{-}attack\;label_{i}\neq
Post\text{-}attack\;label_{i})$ (11)

where $N$ is the number of images in the attack set. Attack accuracy (AA)
shows the accuracy of an adversary after performing an attack, and
transferability measures the effectiveness of attacks on target models. Figure
1 presents an illustration of a slice of a Meningioma MRI scan that has been
adversarially manipulated.

###  4.3 Robustness with federated adversarial adaptation

In this section, we conduct a comparative analysis between our proposed method
and traditional adversarial training. The evaluation aims to illustrate the
relative efficiencies of the methods in various scenarios. Figure 2 and Table
1 depict the comparison of distributed noise and adversarial training across
three classification tasks, under different attack settings, characterized by
varying degrees and steps of perturbation. Distributed noise training was able
to reduce the attacker’s self-ASR by up to 8%, and the attacker’s average ASR
by up to 27%, compared to the adversarial training method.

Figure 2: Effect of Error perturbation degree $\epsilon$ on attack transferability.PGD attack is performed.ASR is calculated on benign (left column) and the adversary client (right column). The higher ASR on benign clients shows higher transferability Table 1: Average values of ASR(self/avg) for different datasets as a result of distributed noise training and adversarial training technique. Lower value of ASR indicates a better defense against adversarial attacks. | Training Method
---|---
Dataset | Step | Adversarial training | Distributed Noise
|  | (ASR Self/Avg) | (ASR Self/Avg)
Glioma | 0.005 | 0.03/0.00 | 0.03/0.00
| 0.012 | 0.00/0.00 | 0.00/0.00
| 0.017 | 0.00/0.00 | 0.00/0.00
| 0.05 | 0.03/0.00 | 0.00/0.00
MRI | 0.005 | 0.12/0.05 | 0.10/0.02
| 0.012 | 0.22/0.09 | 0.22/0.08
| 0.017 | 0.25/0.11 | 0.23/0.04
| 0.05 | 0.46/0.27 | 0.41/0.15
Pathology | 0.005 | 0.32/0.08 | 0.31/0.08
| 0.012 | 0.43/0.12 | 0.47/0.10
| 0.017 | 0.46/0.17 | 0.49/0.12
| 0.05 | 0.54/0.32 | 0.65/0.34

The results also indicated that higher values of the perturbation parameter
led to a higher attack success rate in all scenarios. In both defense models,
the self ASR was around two to four times higher than the average ASR,
indicating that the differences between the models in a federated learning
network can lead to significant differences in the success of an attack. In
contrast, the error rate of the undefended model was similar for the
adversarial client and other clients, indicating high transferability of the
attack.

Table 2: Effect of DP on the performance of distributed noise and adversarial training. The average attack success rate (ASR) values are shown for each dataset. | No DP, DP, and $\epsilon$
---|---
| No DP | DP | $\epsilon$
Pathology | 10.21% | 5.09% | 0.01
| 9.97% | 8.72% | 0.03
| 9.53% | 7.87% | 0.05
| 19.04% | 16.67% | 0.07
Average | 12.19% | 9.58% |
MRI | 2.74% | 0.31% | 0.01
| 3.92% | 0.32% | 0.03
| 12.67% | 0.32% | 0.05
| 6.29% | 0.32% | 0.07
Average | 6.41% | 0.32% |

Figure 3: Effect of the number of adversarial samples on the average attack
success rate, for benign and adversarial clients. The results show that using
more retraining samples in the global model decreased the attack success rate
in all scenarios, with distributed noise outperforming adversarial training.

Distributed noise requires fewer training samples Figure 3 shows the effect of
the number of adversarial adaptation samples on the performance of distributed
noise and adversarial training, especially in the ASR for the benign clients.
We evaluated from 10,20,30,50,70 and 100 percent of the samples, and the
results indicate that using more retraining samples in the global model leads
to a lower attack success rate in all scenarios. Using all of the training
samples led to almost perfect defense in all scenarios, while distributed
noise outperformed adversarial training and achieved perfect defense with much
fewer samples. For example, using only 20% of the training samples would
result can reduce the attackers success on benign clients to 12.2% pathology,
9.7% in Meningioma, and 0.0% in Glioma usecase.

Effect of differential privacy on robustness While methods like adversarial
training have a compromise on privacy, our findings suggest that DP can
actually enhance robustness. Table 2 shows the results of the experiments
conducted to evaluate the effect of DP on the defense methods. In both defense
methods, adversarial training and distributed noise, DP reduced the average
attack success rate (ASR) and hence improved the defense. For all attack
values, the Glioma dataset resulted in perfect defense with ASR=0.0, and DP
enabled defense for MRI was also almost perfect with both defense methods,
resulting in average ASR= 0.32 percent. For the Pathology dataset, the average
ASR was reduced from 12.19 percent to 9.58 percent when differential privacy
was used. Our experiments showed that using distributed noise training can
reduce the attacker’s self-ASR by up to 8% and the attacker’s average ASR by
up to 27%, across three different datasets. Additionally, we observed that the
effectiveness of distributed noise is dependent on the step size used for the
attack, with smaller steps leading to greater effectiveness.

We found that using more retraining samples in the global model can lead to a
lower attack success rate, and that the addition of differential privacy can
improve defense performance. These findings are consistent with the existing
literature on adversarial defense, emphasizing the role of retraining samples
and differential privacy as viable defense mechanisms. However, the inherent
trade-off between differential privacy and clean accuracy necessitates further
exploration. A comparative analysis delineating the effectiveness,
computational cost, and implementation feasibility of our technique against
other defense mechanisms remains a subject for future research.

##  5 Conclusion

This paper focuses on robustness of federated learning networks against
adversarial attacks through the implementation of distributed noise. Our
methodology offers defense against a diverse array of adversarial scenarios
without compromising the robustness and privacy of the data involved. Our
empirical evaluations substantiate that the proposed system not only sustains
a high level of accuracy, comparable to models trained through conventional
methods, but also improves or matches the robustness of adversarial training.
The integration of distributed noise is pivotal, facilitating the safeguarding
of each client’s dataset while conforming to the standards of differential
privacy.

The promising results obtained underscore the efficacy of our system in
counteracting the threats from malicious entities in a federated setting, thus
promoting a secure and dependable federated learning ecosystem. Given its
enhanced performance and security attributes, our approach emerges as a
practical solution for various applications, especially in the medical sector
where the sanctity of data and privacy are of utmost importance.

We envision this work as a foundational step for future research endeavors
exploring the realms of robustness and privacy in federated learning systems.
It is our hope that the advancements made in this study will contribute
significantly to the development and deployment of privacy-preserving, and
robust federated learning models in real-world applications, paving the way
for innovations in medical and other critical domains.

## Acknowledgement

This research is supported by KWF Kankerbestrijding and the Netherlands
Organisation for Scientific Research (NWO) Domain AES, as part of their joint
strategic research programme: Technology for Oncology IL. The collaboration
project is co-funded by the PPP allowance made available by Health Holland,
Top Sector Life Sciences & Health, to stimulate public-private partnerships.

## References

  * [1] J. Zhang, Y. Chen, and H. Li, “Privacy leakage of adversarial training models in federated learning systems,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 108–114.
  * [2] F. A. Mejia, P. Gamble, Z. Hampel-Arias, M. Lomnitz, N. Lopatina, L. Tindall, and M. A. Barrios, “Robust or private? adversarial training makes models more vulnerable to privacy attacks,” _arXiv preprint arXiv:1906.06449_ , 2019\.
  * [3] L. Song, R. Shokri, and P. Mittal, “Privacy risks of securing machine learning models against adversarial examples,” in _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security_ , 2019, pp. 241–257.
  * [4] N. Bouacida and P. Mohapatra, “Vulnerabilities in federated learning,” _IEEE Access_ , vol. 9, pp. 63 229–63 249, 2021.
  * [5] B. Liu, B. Yan, Y. Zhou, Y. Yang, and Y. Zhang, “Experiments of federated learning for covid-19 chest x-ray images,” _arXiv preprint arXiv:2007.05592_ , 2020.
  * [6] H. Malik, A. Naeem, R. A. Naqvi, and W.-K. Loh, “Dmfl_net: A federated learning-based framework for the classification of covid-19 from multiple chest diseases using x-rays,” _Sensors_ , vol. 23, no. 2, p. 743, 2023.
  * [7] X. Han, Y. Hu, L. Foschini, L. Chinitz, L. Jankelson, and R. Ranganath, “Deep learning models for electrocardiograms are susceptible to adversarial attack,” _Nature medicine_ , vol. 26, no. 3, pp. 360–363, 2020.
  * [8] D. Rodriguez, T. Nayak, Y. Chen, R. Krishnan, and Y. Huang, “On the role of deep learning model complexity in adversarial robustness for medical images,” _BMC Medical Informatics and Decision Making_ , vol. 22, no. 2, pp. 1–15, 2022.
  * [9] G. Bortsova, C. González-Gonzalo, S. C. Wetstein, F. Dubost, I. Katramados, L. Hogeweg, B. Liefers, B. van Ginneken, J. P. Pluim, M. Veta, _et al._ , “Adversarial attack vulnerability of medical image analysis systems: Unexplored factors,” _Medical Image Analysis_ , vol. 73, p. 102141, 2021\.
  * [10] J. So, C. He, C.-S. Yang, S. Li, Q. Yu, R. E Ali, B. Guler, and S. Avestimehr, “Lightsecagg: a lightweight and versatile design for secure aggregation in federated learning,” _Proceedings of Machine Learning and Systems_ , vol. 4, pp. 694–720, 2022.
  * [11] W.-N. Chen, A. Ozgur, and P. Kairouz, “The poisson binomial mechanism for unbiased federated learning with secure aggregation,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 3490–3506.
  * [12] J. Ma, S.-A. Naas, S. Sigg, and X. Lyu, “Privacy-preserving federated learning based on multi-key homomorphic encryption,” _International Journal of Intelligent Systems_ , vol. 37, no. 9, pp. 5880–5901, 2022.
  * [13] L. Zhang, J. Xu, P. Vijayakumar, P. K. Sharma, and U. Ghosh, “Homomorphic encryption-based privacy-preserving federated learning in iot-enabled healthcare system,” _IEEE Transactions on Network Science and Engineering_ , 2022.
  * [14] M. J. Sheller, B. Edwards, G. A. Reina, J. Martin, S. Pati, A. Kotrotsou, M. Milchenko, W. Xu, D. Marcus, R. R. Colen, _et al._ , “Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data,” _Scientific reports_ , vol. 10, no. 1, pp. 1–12, 2020\.
  * [15] S. Vithana and S. Ulukus, “Model segmentation for storage efficient private federated learning with top $r$ sparsification,” _arXiv preprint arXiv:2212.11947_ , 2022.
  * [16] M. Adnan, S. Kalra, J. C. Cresswell, G. W. Taylor, and H. R. Tizhoosh, “Federated learning and differential privacy for medical image analysis,” _Scientific reports_ , vol. 12, no. 1, p. 1953, 2022.
  * [17] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” in _International Conference on Artificial Intelligence and Statistics_. PMLR, 2020, pp. 2938–2948.
  * [18] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 3454–3469, 2020.
  * [19] A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” in _Artificial intelligence safety and security_. Chapman and Hall/CRC, 2018, pp. 99–112.
  * [20] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [21] P. Liu, X. Xu, and W. Wang, “Threats, attacks and defenses to federated learning: issues, taxonomy and perspectives,” _Cybersecurity_ , vol. 5, no. 1, pp. 1–19, 2022.
  * [22] S. Bhuvaji, A. Kadam, P. Bhumkar, S. Dedge, and S. Kanchan, “Brain tumor classification (mri),” 2020. [Online]. Available: https://www.kaggle.com/dsv/1183165
  * [23] B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, and M. Welling, “Rotation equivariant cnns for digital pathology,” in _Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11_. Springer, 2018, pp. 210–218.