# Corrigendum and addendum to: Equidistribution of Eisenstein series in the
level aspect

Ikuya Kaneko  Tsukuba Kaisei High School
3315-10 Kashiwada, Ushiku, 300-1211 Japan<EMAIL_ADDRESS>and  Shin-ya
Koyama  Department of Biomedical Engineering, Toyo University, 2100 Kujirai,
Kawagoe, Saitama 350-8585 Japan<EMAIL_ADDRESS>
###### Abstract.

The second author formulated quantum unique ergodicity for Eisenstein series
in the prime level aspect in “Equidistribution of Eisenstein series in the
level aspect”, Comm. Math. Phys. 289, no. 3, 1131–1150 (2009). We point out
major errors and propose ideas to correct particular parts of the proofs with
partially weakened claims.

###### 2010 Mathematics Subject Classification:

Primary 58J51; Secondary 11F11, 11M12

We shortly indulge in retrospection on variations of the quantum unique
ergodicity (QUE) conjecture put fourth by Rudnick and Sarnak [RS94]. Luo and
Sarnak [LS95] established the arithmetic QUE for the continuous spectrum
spanned by Eisenstein series on
$\mathrm{SL}_{2}(\mathbb{Z})\backslash\mathbb{H}$. QUE in the level aspect was
spelled out by Kowalski, Michel and VanderKam [KMV02] for holomorphic cusp
forms: they conjectured that the masses of newforms of fixed weight with large
level $q$ are equidistributed amongst the fibers of the canonical projection
$Y_{0}(q)\to Y_{0}(1)$ with $Y_{0}(q)=\Gamma_{0}(q)\backslash\mathbb{H}$ the
modular curve. Nelson et al. [Nel11, NPS14] have affirmatively answered their
conjecture by following the strategy of Holowinsky and Soundararajan [HS10].
These types of variations intersect at second author’s work [Koy09], where QUE
in the prime level aspect for Eisenstein series was focused.

In the introduction, the term of the quantum ergodicity has incorrectly used,
and it refers to the equidistribution for a density one subsequence of
eigenfunctions of the Laplacian within the unit cotangent bundle in the large
eigenvalue limit. We wanted to generally mean QUE in configuration space.
Moreover the coefficient in the right hand side of the first display on p.1132
should be $6/\pi$ in the light of Stade’s formula on $\mathrm{GL}(2)$; for a
correct statement, see [HR92, Eq. (7.9)] or [Spi03, Eq. (1.1)]. That is, one
has

$\int_{\mathrm{SL}_{2}(\mathbb{Z})\backslash\mathbb{H}}\phi(z)|E(z,1/2+it)|^{2}\frac{dxdy}{y^{2}}=\frac{3}{\pi}\log(1/4+t^{2})\int_{\mathrm{SL}_{2}(\mathbb{Z})\backslash\mathbb{H}}\phi(z)\frac{dxdy}{y^{2}}+o(\log
t)$ (0.1)

as $t\to\infty$ for a fixed smooth and compactly supported function
$\phi:\mathrm{SL}_{2}(\mathbb{Z})\backslash\mathbb{H}\to\mathbb{R}$ (cf.
[You16] for a full main term with a power saving error term for (0.1)). As a
fatal matter, almost all the theorems and propositions should significantly be
modified, since the claimed results are stronger than what is justified. In
particular, we could no longer rely upon the Luo-Sarnak method when the level
is varying. By gathering together various modifications described below, we
see the correct version of Theorem 1.3 should morally be as follows:

######  Theorem 1.3.

Let $q$ be a prime or $q=1$, and let $t$ to be a fixed real number. We denote
by $\pi_{q}:Y_{0}(q)\to Y_{0}(1)$ the canonical projection with which fix a
map $\iota_{q}=\iota$ such that $\pi_{q}\circ\iota=\mathrm{id}$ for each $q$.
For any fixed compact Jordan measurable subsets $A,B\subset Y_{0}(1)$ having
positive measure and for an arbitrary sequence in
$\\{(\iota_{q},\kappa(q))\mid\text{$\pi_{q}\circ\iota_{q}=\mathrm{id}$,
$\kappa(q)=0,i\infty$ for each $q$}\\}$, we have

$\lim_{q\to\infty}\frac{\int_{A}|E_{q,\kappa(q)}(\iota
z,1/2+it)|^{2}d\mu}{\int_{B}|E_{q,\kappa(q)}(\iota
z,1/2+it)|^{2}d\mu}=\frac{\mathrm{vol}(A)}{\mathrm{vol}(B)},$

with respect to the Poincaré measure $d\mu=y^{-2}dxdy$ on the modular curve
$Y_{0}(1)$.

It should be mentioned that the assumption on $q$ could be weakened with more
work, but at the very least we have to treat finitely many more cusps. The
main issue in the original theorem was that the objects $A_{q}$ are too thin
and the error terms that appeared in Propositions 1.4 and 1.5 are not provably
smaller than the principal terms. It was also forgotten to ponder the error
term emerging from the manipulation of the spectral decomposition of the
characteristic function of a set $A_{q}^{0}$. Notice that $A_{q}^{0}$ is
changing with $q$, so that such an error term depends on $q$. We are much
indebted to Matthew Young and Jiakun Pan for their bringing those slips to our
attention. As a proper generalization of [Koy09], Jiakun Pan studied
Eisenstein series with general nebentypus characters and general levels and
succeeded in getting around the issues briefly described above. Apparently his
strategy relies upon the generalization of Zagier’s method of regularization
to Eisenstein series on general levels; Zagier developed a formula for
integrals of double and triple products of Eisenstein series on the level 1
case.

In order to circumvent many difficulties, it is conceivable to replace our
chief concern with a softer object. Fortunately, the method we used works fine
if instead of taking the thin sets of the form $A_{q}$, we select a set
$A_{1}$ that is not dependent on $q$. Fix a nice test function $\phi$ on
$\mathrm{SL}_{2}(\mathbb{Z})$ and $\iota=\iota_{q}$ to denote a map which
satisfies $\pi_{q}\circ\iota=\mathrm{id}$ for each $q$ (specifically, we have
$\iota z\in Y_{0}(q)$ for $z\in Y_{0}(1)$). We thus consider the following
“unbalanced” inner product against Eisenstein series on
$\Gamma_{0}(q)\backslash\mathbb{H}$,

$\langle|E|^{2},\phi\rangle_{1}=\int_{Y_{0}(1)}\phi(z)|E_{q,\kappa(q)}(\iota
z,1/2+it)|^{2}\frac{dxdy}{y^{2}},$ (0.2)

although the work of Jiakun Pan is concerned with
$\langle|E|^{2},\phi\rangle_{q}$ for more general Eisenstein series like
$E=E_{\infty}^{(q)}(z,s,\chi)$. We then spectrally decompose $\phi$ in (0.2)
into cusp forms and incomplete Eisenstein series. The Luo-Sarnak approach
proceeds by showing asymptotic formulæ for the contributions of Maaß forms and
incomplete Eisenstein series. Through working with (0.2), there are no forms
of level $q$ that enter after the application of the spectral decomposition,
so that one could adapt the Luo-Sarnak method to address (0.2). Actually,
there is a technical difficulty in analyzing the integral handled in [Koy09]
in the level aspect by our using only the spectral decomposition and
Parseval’s formula, since the Eisenstein series (absolute value squared) grows
too fast at the cusp. As we gave heed before, it is adequate to think that
renormalized integrals (such as in Michel and Venkatesh [MV10] using the
language of representation theory) would be the most useful implement to
sidestep the difficulty.

We also point out several problematic errors. However, the following
corrections are meant for the original paper [Koy09] to have a mostly self-
contained exposition, while they should be along with additional modifications
when we consider (0.2). First of all, throughout the paper the factor
$n^{s-1}$ should be $|n|^{s-1}$ for $n\in\mathbb{Z}$. Furthermore, the
condition on both sums in (2.3) and (2.4) should be recasted as
$c\in\mathcal{C}(\kappa,\kappa^{\prime})$ for a pair of cusps
$\kappa,\kappa^{\prime}$, where

$\mathcal{C}(\kappa,\kappa^{\prime})=\bigg{\\{}\gamma>0:\begin{pmatrix}\ast&\ast\\\
\gamma&\ast\end{pmatrix}\in\sigma_{\kappa}^{-1}\Gamma_{0}(q)\sigma_{\kappa^{\prime}}\bigg{\\}}.$

is the set of allowed moduli. By virtue of this, the change of variables
$c=\gamma\sqrt{q}$ on p.1138 can clearly be justified. On p.1139, the
definition of the Fourier coefficients $\rho_{j}(n)$ should be replaced with

$\rho_{j}(n)=\begin{cases}q^{1/2}\tau_{j}(n/q)&\text{if $q\mid n$},\\\
0&\text{otherwise}.\end{cases}$

and this propagates typos throughout the later parts of the paper when we
treat the oldforms of level 1 slashed by $q$. In particular, the assertion (2)
in Lemma 2.4 is wrong and should be reformed to the shape that includes the
extra $q^{1/2}$:

######  Lemma 2.4 (2).

When $u_{j}$ is an oldform for $\Gamma_{0}(q)$ and expressed by
$u_{j}(z)=v_{j}(qz)$ with $v_{j}$ a cusp form for
$\mathrm{SL}_{2}(\mathbb{Z})$, we have

$\sum_{n=1}^{\infty}\frac{\rho_{j}(n)\sigma_{\nu}(n)}{n^{s}}=\tau_{j}(1)q^{1/2-s}\frac{1+q^{\nu}-\tilde{\tau}_{j}(q)q^{\nu-s}}{1-q^{\nu-2s}}\frac{L(s,v_{j})L(s-\nu,v_{j})}{\zeta(2s-\nu)}$

and

$\sum_{n=1}^{\infty}\frac{\rho_{j}(n)\sigma_{\nu}(nq^{-\alpha})}{n^{s}}=\tau_{j}(1)q^{1/2-s}(1+q^{\nu})\frac{1-\tilde{\tau}_{j}(q)q^{\nu-s}}{1-q^{\nu-2s}}\frac{L(s,v_{j})L(s-\nu,v_{j})}{\zeta(2s-\nu)}.$

Here we simplified the complicated numerator in (2.13) as well. The proof of
this claim is always with the additional factor of $q^{1/2}$. In view of this
correction, it is better to incorporate Lemma 2.6 with Lemma 2.5 to conclude
the calculation of the contribution from cusp forms, because the proof of
Lemma 2.5 includes incorrect statements (and that of Lemma 2.6 is back-of-the-
envelope). For instance, on p.1143, the second-to-last line in the fourth
display should be modified into

$\displaystyle\frac{\rho_{j}(1)|L(s+it,u_{j})|^{2}}{\hat{\zeta}(1+2it)\zeta(2s)(1-q^{-2s})}\bigg{(}1-\frac{1-\tilde{\rho}_{j}(q)q^{-s-it}}{1-q^{-1-2it}}\bigg{)}$
$\displaystyle=\frac{\rho_{j}(1)|L(s+it,u_{j})|^{2}}{\hat{\zeta}(1+2it)\zeta(2s)(1-q^{-2s})}\Big{(}1-(1-\tilde{\rho}_{j}(q)q^{-s-it})(1+q^{-1-2it}+\cdots)\Big{)}$
$\displaystyle\ll\frac{\rho_{j}(q)q^{-s-it}}{1-q^{-2s}}\frac{|L(s+it,u_{j})|^{2}}{\hat{\zeta}(1+2it)\zeta(2s)}\ll
q^{\frac{6\vartheta}{5}-\frac{1}{10}+\epsilon}\ll q^{\frac{1}{32}+\epsilon},$

where we exploited the bound in the recent work of Blomer, Humphries, Khan and
Milinovich: $L(1/2,u_{j})\ll q^{1/5+\vartheta/15+\epsilon}$ for
$u_{j}\in\mathcal{B}^{\ast}(q)$ with $\mathcal{B}^{\ast}(q)$ being an
orthonormal basis of Hecke-Maaß newforms of level $q$ having spectral
parameter $t_{j}$. We denote by $\vartheta\in[0,7/64]$ an admissible exponent
towards the Ramanujan-Petersson conjecture and note that the appearance of
$\vartheta$ also stems from the Fourier coefficients $\rho_{j}(q)$. Even if an
optimal amplifier would yield the stronger bound
$L(1/2,u_{j})\ll_{\epsilon}q^{\frac{1}{4}-\frac{1-2\vartheta}{16}+\epsilon}$
of Burgess quality (that makes the limit of current technology), we cannot
obtain any error term to be smaller than the main term in our QUE problem,
unless we assume a slight improvement on the Ramanujan-Petersson conjecture.
This error is irretrievable, nonetheless our dealing with the level aspect QUE
problem in the shape (0.2) will be attended with no difficulties.

On the other hand, the full part of the last big display (2.15) on p.1143 is
also mistaken. In order to rectify it, we use the recasted form of Lemma 2.4
described above, obtaining the following plausible estimate:

$\displaystyle(2.15)$
$\displaystyle=\frac{1}{\hat{\zeta}(1+2it)}\bigg{(}\sum_{n=1}^{\infty}\frac{\rho_{j}(n)\sigma_{-2it}(n)}{n^{s-it}}-\frac{1}{1-q^{-1-2it}}\sum_{n=1}^{\infty}\frac{\rho_{j}(n)\sigma_{-2it}(nq^{-\alpha})}{n^{s-it}}\bigg{)}$
$\displaystyle=\tau_{j}(1)\frac{q^{1/2+it-s}}{1-q^{-2s}}\frac{L(s+it,v_{j})L(s-it,v_{j})}{\zeta(2s)\hat{\zeta}(1+2it)}$
$\displaystyle\hskip
56.9055pt\times\Big{(}1+q^{-2it}-\tilde{\tau}_{j}(q)q^{-s-it}-(1+q^{-2it})(1-\tilde{\tau}_{j}(q)q^{-s-it})(1+q^{-1-2it}+\cdots)\Big{)}$
$\displaystyle=\tau_{j}(q)\frac{q^{1/2-2(s+it)}}{1-q^{-2s}}\frac{L(s+it,v_{j})L(s-it,v_{j})}{\zeta(2s)\hat{\zeta}(1+2it)}+\text{(lower
order terms)}$ $\displaystyle\ll q^{\vartheta-1/2}.$

Hence, the contribution of the oldforms would not cause any problem in
analyzing the integral $I_{j}(q,s)$. In any case our method of asymptotic
evaluation for the inner product against incomplete Eisenstein series cannot
work as long as we do not substitute $A_{1}$ for the thin sets $A_{q}$. So, it
seems difficult to constructively approximate the projection of $\phi$ onto
$\mathcal{E}(Y_{0}(q))$ in terms of incomplete Eisenstein series, where
$\mathcal{E}(\Gamma\backslash\mathbb{H})$ is the orthogonal complement of the
span of the Maass forms $\mathcal{C}(\Gamma\backslash\mathbb{H})$. In passing,
the calculation of the contribution from incomplete Eisenstein series should
also be significantly corrected, namely the factor 24 in the first term in
Proposition 1.5 is incorrect. This error stems form a mis-calculation of a
Dirichlet series counted with divisor functions, and a certain integral
involving $K$-Bessel functions.

Although the work [Koy09] sticks to the $\mathrm{GL}(2)$ unitary Eisenstein
series, it is also interesting to speculate what happens if we change the
subject to higher ranks, such as to the $\mathrm{GL}(n)$ continuous spectrum
spanned by the degenerate Eisenstein series induced from the maximal parabolic
subgroup. Quite recently, Zhang [Zha19] studied an analogue of Luo and
Sarnak’s result in such direction, however QUE for higher rank non-degenerate
Eisenstein series is much more difficult problem as it is closely related to
the shifted convolution problem with generalized divisor functions and Fourier
coefficients of higher rank Maass cusp forms. This way, it would also be
possible to consider various versions of [Koy09] based solely on classical
ideas of Luo and Sarnak. For example, the authors [KK19] recently succeeded in
showing equidistribution results for Eisenstein series in the level aspect
over function fields. In the case of the function field analogue, fundamental
domains turn into Ramanujan graphs of the general form
$\Gamma\backslash\mathrm{PGL}_{2}(F)/K$ with $F$ a local non-archimedean
field, $K$ a maximal compact subgroup and $\Gamma$ a lattice in
$\mathrm{PGL}_{2}(F)$. The setting we are interested in is the homogeneous
space $G/K$, where $G=\mathrm{PGL}_{2}(k_{\infty})$ and
$K=\mathrm{PGL}_{2}(r_{\infty})$ with $k_{\infty}=\mathbb{F}_{q}((1/t))$ the
field of Laurent formal power series in the uniformizer $1/t$ over
$\mathbb{F}_{q}$ and $r_{\infty}$ the ring of local integers
$\mathbb{F}_{q}[[1/t]]$, respectively. Notice that $K$ is a maximal compact
subgroup of $G$. Then the number of the discrete spectrum of the adjacency
operator on $\mathcal{L}^{2}(\Gamma\backslash\mathcal{F})$ is known to be
finite, where $\mathcal{F}$ is a $(q+1)$-regular tree. But from the viewpoint
of the level aspect QUE problem, one can consider the Hecke congruence
subgroup of the form $\Gamma_{0}(A)$ as $\deg A$ tends to infinity. The
explicit forms of theorems in our work will be announced soon.

## References

  * [HR92] D. Hejhal and B. Rackner, _On the topography of Maass waveforms for $\mathrm{PSL}(2,\mathbf{Z})$_, Experiment. Math. 1 (1992), no. 4, 275–305.
  * [HS10] R. Holowinsky and K. Soundararajan, _Mass equidistribution for Hecke eigenforms_ , Ann. of Math. (2) 172 (2010), no. 2, 1517–1528.
  * [KK19] I. Kaneko and S. Koyama, _Quantum unique ergodicity for Eisenstein series in the level aspect over function fields_ , in preparation, 2019.
  * [KMV02] E. Kowalski, P. Michel, and J. VanderKam, _Rankin-Selberg $L$-functions in the level aspect_, Duke Math. J. 114 (2002), no. 1, 123–191.
  * [Koy09] S. Koyama, _Equidistribution of Eisenstein series in the level aspect_ , Commun. Math. Phys. 289 (2009), no. 3, 1131–1150.
  * [LS95] W. Luo and P. Sarnak, _Quantum ergodicity of eigenfunctions on $\mathrm{PSL}_{2}(\mathbf{Z})\backslash\mathbf{H}^{2}$_, Publ. Math. Inst. Hautes Études Sci. 581 (1995), no. 1, 207–237.
  * [MV10] P. Michel and A. Venkatesh, _The subconvexity problem for $\mathrm{GL}_{2}$_, Publ. Math. Inst. Hautes Études Sci. 111 (2010), no. 1, 171–271.
  * [Nel11] P. D. Nelson, _Equidistribution of cusp forms in the level aspect_ , Duke Math. J. 160 (2011), no. 3, 467–501.
  * [NPS14] P. D. Nelson, A. Pitale, and A. Saha, _Bounds for Rankin-Selberg integrals and quantum unique ergodicity_ , J. Amer. Math. Soc. 27 (2014), no. 1, 147–191.
  * [RS94] Z. Rudnick and P. Sarnak, _The behaviour of eigenstates of arithmetic hyperbolic manifolds_ , Commun. Math. Phys. 161 (1994), no. 1, 195–213.
  * [Spi03] F. Spinu, _The $L^{4}$-norm of the Eisenstein series_, Ph.D. thesis, Princeton University, Princeton, New York, 2003, ProQuest LLC, Ann Arbor, MI, 2003, http://www.math.jhu.edu/~fspinu/math/thesis.pdf.
  * [You16] M. P. Young, _The quantum unique ergodicity conjecture for thin sets_ , Adv. Math. 286 (2016), no. 2, 958–1016.
  * [Zha19] L. Zhang, _Quantum unique ergodicity of degenerate Eisenstein series on $\mathrm{GL}(n)$_, Commun. Math. Phys. 369 (2019), no. 1, 1–48.


# A Framework for testing Federated Learning algorithms using an edge-like
environment

Felipe Machado Schwanck  Marcos Tomazzoli Leipnitz  Joel Luís Carbonera
Juliano Araujo Wickboldt

###### Abstract

Federated Learning (FL) is a machine learning paradigm in which many clients
cooperatively train a single centralized model while keeping their data
private and decentralized. FL is commonly used in edge computing, which
involves placing computer workloads (both hardware and software) as close as
possible to the edge, where the data is being created and where actions are
occurring, enabling faster response times, greater data privacy, and reduced
data transfer costs. However, due to the heterogeneous data
distributions/contents of clients, it is non-trivial to accurately evaluate
the contributions of local models in global centralized model aggregation.
This is an example of a major challenge in FL, commonly known as data
imbalance or class imbalance. In general, testing and assessing FL algorithms
can be a very difficult and complex task due to the distributed nature of the
systems. In this work, a framework is proposed and implemented to assess FL
algorithms in a more easy and scalable way. This framework is evaluated over a
distributed edge-like environment managed by a container orchestration
platform (i.e. Kubernetes).

###### keywords:

Federated Learning , Edge Computing , Kubernetes , Microservices , Development
Framework

††journal: Future Generation Computer Systems

[inst1]organization=Federal University of Rio Grande do Sul (UFRGS), Institute
of Informatics,addressline=Av. Bento Gonçalvez, 9500, city=Porto Alegre,
postcode=91.509-900, state=RS, country=Brazil

##  1 Introduction

Federated Learning (FL) is a machine learning solution designed to train
machine learning models while keeping the data private and decentralized [1].
The main idea of FL is for each client to train its local model using its data
and, afterward, upload the generated local model to a single centralized
server, where the regional models of the participant clients will be
aggregated and weighted to create a global model. FL is commonly used in edge
computing, which involves placing computer workloads (both hardware and
software) as close as possible to the edge, where the data is being created
and where actions are occurring, enabling faster response times, greater data
privacy, and reduced data transfer costs [2]. Thus, FL can be used with a
variety of clients, such as smartphones, sensors, IoT devices, and data silos
(distributed databases that need to keep their data private from the outside
world). However, as seen in [3], in general, the data distribution of the
mobile systems and other similar settings is imbalanced, which can increase
the bias of the model and impact in a negative way its performance. Different
approaches have been proposed, such as Deep Reinforcement Learning (DRL) [4],
as a solution for this problem.

Although regular centralized machine learning may outperform FL in prediction
performance [5], the entire dataset must be shared. Its first application was
in Google GBoard [6], which learns from every smartphone using Gboard without
sharing user data. Since then, FL applicability has advanced to various fields
such as autonomous vehicles, traffic prediction and monitoring, healthcare,
telecom, IoT, pharmaceutics, industrial management, industrial IoT, and
healthcare and medical AI [7]. Moreover, the number of academic publications
with FL as main subject has increased significantly since its conception [1].

The increase in IoT devices has greatly enabled FL. The total number of IoT
connections will reach 83 billion by 2024, rising from 35 billion connections
in 2020, a growth of 130% over the next four years. The industrial sector has
been identified as a critical driver of this growth. Expansion will be driven
by the increasing use of private networks that leverage cellular network
standards [8]. The evolution of the IoT device’s computational power has also
enabled FL; since an edge computer can process data locally, its sensors (e.g.
cameras) could collect samples (e.g. images or frames) at a higher resolution
and a higher frequency (such as frame rate) than would be possible if the data
had to be sent to the cloud for processing [9].

One of the many challenges that have come up with the advance of FL is dealing
with data imbalance and heterogeneity, as seen in [7]. In FL, using their
local data, each edge node trains a shared model. As a result, data
distribution from those edge devices is based on their many uses. Imagine a
scenario, for example, of the distribution of cameras in a surveillance
system. Compared to cameras located in the wild, cameras in the park, for
example, capture more photographs of humans. Also, the size of the dataset
that each one of those cameras will have to train their local models might
differ by a large magnitude since a park might have much more data to input
than a camera in the wild. Furthermore, an approach that has been proposed in
many studies to dynamically address weight for the local models of clients
participating in the FL global model and, therefore, deal with the
heterogeneous data is DRL [10, 11, 12, 13].

DRL has gathered much attention recently. Recent studies have shown impressive
results in activities as diverse as autonomous driving [14], game playing
[15], molecular recombination [16], and robotics [17]. In all those
applications, it has been used in computer programs to teach them how to solve
complex problems, for example, how to fly model helicopters and perform
aerobatic maneuvers, and, in some applications, it has already outsmarted some
of the most skilled humans, such as in Atari, Go, poker and StarCraft.

Testing and evaluating FL algorithms can be a very difficult task to
accomplish. FL intrinsically creates complex distributed systems with non-
trivial interactions among its participants. Therefore, this work proposes and
implements a framework for testing FL algorithms that enables users to easily
create different training scenarios by simply changing configuration
parameters. These include computing and data distributions, datasets, global
model aggregations, local client models, and server/client training
parameters. The framework is also capable of collecting and visualizing
training results and resource usage metrics (e.g., CPU, memory, etc.).
Experiments have been conducted over a realistic edge-like environment managed
by a Kubernetes container orchestration platform, with varied parameters on
top of the proposed framework to demonstrate its capabilities.

The remainder of this work is organized as follows. Section 2 presents a
literature review for edge computing and FL, respectively. Sections 3 and 4
present the conceptual framework architecture proposed and the tools and
frameworks used to implement the PoC solution. Section 5 details the PoC
solution developed. Sections 6, 7, and 8 present the experimental setup, the
results obtained, and how resource usage and network traffic can be monitored,
respectively. Finally, Section 9 concludes the work with final remarks and a
perspective on future work.

##  2 Literature Review

This section presents the main topics of this work and provides an overview of
the current state of the art of edge computing, federated learning, and deep
reinforcement learning.

###  2.1 Edge Computing

Data is increasingly produced at the edge of the network; therefore,
processing the data at the network’s edge would be more efficient. With the
advancement of telecommunication services and the increase of the necessity
for low-latency computing, the edge computing paradigm has been motivated. In
edge computing, instead of having computer workloads (both hardware and
software) centralized in a data center (cloud), we have them as close as
possible to the edge, where the data is being created and where actions are
occurring, thus benefiting lower latency, greater data privacy, and reduced
data transfer costs. For [18], edge computing refers to the enabling
technologies allowing computation to be performed at the edge of the network,
on downstream data on behalf of cloud services, and upstream data on behalf of
IoT services. Therefore, edge devices can be any device with Internet access,
such as smartphones, smart cars, or other IoT devices.

Multi-access Edge Computing (MEC) [19] is proposed as a critical solution that
enables operators to open their networks to new services and IT ecosystems and
leverage edge-cloud benefits in their networks and systems since it places
storage and computation at the network edge. The proximity of the end users
and connected devices provides low latency and high bandwidth while minimizing
centralized cloud limitations such as delay, access bottlenecks, and single
points of failure.

MEC use cases can be seen in real-time traffic monitoring [20] and autonomous
vehicles [21]. In traffic monitoring, real-time and accurate video analysis is
critical and challenging work, especially in situations with complex street
scenes; therefore, edge computing-based video pre-processing is proposed to
eliminate the redundant frames edge devices need to process since a
considerable amount of vehicle video data is generated. Also, the
decentralized and highly available nature of multi-access edge computing is
taken advantage of to collect, store, and analyze city traffic data in
multiple sensors. For autonomous vehicles, a large amount of real-time data
processing from different sensors at high speed is needed to guarantee driver
safety.

###  2.2 Federated Learning

Federated learning (FL) is a distributed form of machine learning proposed by
Google [1] to train models at scale while allowing the user data to be
private. In Federated Averaging (FedAvg), the server aggregates the model
updates using simple averaging. It returns the new model parameters to the
client devices, which continue training using the updated model parameters.
Google’s proposal provided the first definition of federated learning, as well
as the Federated Optimization [22] approach to improve these federated
algorithms further. Advanced Federated Optimization [23] is a variant of the
Stochastic Gradient Descent (SGD) algorithm commonly used in centralized
training. In FedOpt, each local node applies SGD to its local data to compute
the gradients and then sends them to a central server. The server then
aggregates the gradients from all the nodes to update the global model.
Adaptive Federated Optimization with Yogi (FedYogi) incorporates a momentum-
based optimizer called Yogi after the central server aggregates the model
updates using the FedAvg algorithm to improve the convergence rate. Federated
Averaging with Momentum [24] incorporates a momentum-based optimizer, similar
to the Yogi optimizer in FedYogi. The critical difference is that it uses a
combination of the gradients from the current iteration and the gradients from
the previous iteration to update the model parameters. This allows the
optimizer to maintain a direction of movement even when the gradient changes
direction, which helps to smooth out noisy gradients and accelerate
convergence. In the “Federated Learning: Collaborative Machine Learning
without Centralized Training Data” blog post [25], Google explains how FL is
enabling mobile phones to collaboratively learn a shared prediction model
while keeping all the training data on the device, decoupling the ability to
do machine learning from the need to store the data in the cloud. In addition,
the current use of FL to predict keyboard words in Google’s Gboard [6] and how
it can be used for photo ranking and further improving language models.

FL usually deals with data distributed across multiple devices. In such
settings, data is usually non-independently and identically distributed (i.e.,
non-IID). One of the main challenges in FL is dealing with the heterogeneity
of the data distribution among the parties since data distribution from those
edge devices is based on their many uses [7].

Furthermore, many use cases in FL have data samples distributed among multiple
devices, which are not always synchronized and may have limited connectivity.
Thus, it cannot train these devices in parallel and directly aggregate them,
as it cannot guarantee device availability or data homogeneity. Understanding
how to properly select clients and weigh each client’s contributions in the
global model remains an open problem in FL. An example of a practical scenario
of data imbalance and heterogeneity in which DRL was used in FL as a solution
can be seen in recent work proposed to deal with blade icing detection in
distributed wind turbines [26]. Wind turbines closer to the sea experience
windy and snowy weather, while those closer to the continent deal with windy
and rainy conditions. This heterogeneity introduces a bias in the local models
since one might be more susceptible to icing than another. Therefore, since
our objective is to identify icing, clients who experience more icing should
have a different weight assigned to their contribution to the global model
than others without.

##  3 Conceptual Framework

The main objective of the proposed framework is to enable FL testing in a
platform where users can easily change parameters to create different
scenarios in a distributed computing environment. These include different
computing and data distributions, datasets, global model aggregations, client
models, and server and client training parameters. The framework should also
be capable of collecting training results and resource usage metrics. Figure 1
presents an overview of the proposed framework main components in a conceptual
architecture.

Figure 1: High-level overview of the conceptual framework proposed

The framework is designed in these layers to have the highest level of
independence within them. This enables easier development and segregation of
functions, so if improvements are done in one layer, it does not affect the
others. The following subsections detail each one of the components of the
layers.

###  3.1 Distributed Infrastructure Layer

This bottom layer contains all the infrastructure needed to create a
distributed computing environment. It is responsible for scaling computing,
storage, and networking from a pool of resources to meet user input
parameters. The user parameters will reflect how the resource layer will be
laid out to create the scenario desired for testing. For example, let’s say a
user wants to configure a scenario of FL using ten clients, whereas each
client has a specific computing requirement of four CPU cores and 4GB of RAM
per client. The distributed infrastructure manager will then configure the
resource layer from its pool of resources (computing, storage, and networking)
with ten clients, each with the specified specifications to meet user
requirements. Having an independent infrastructure from the resource and
application layers enables the usage of different computational devices since
the application only sees the resource layer. This will be fundamental to
allow different scenarios where other edge computing devices can be used as
clients.

###  3.2 Resource Layer

The resource layer can be found on top of the distributed infrastructure
layer. As mentioned before, it will match the configuration the user sets to
set the FL training scenario. We can categorize the distribution of resources
into four major categories: server resources, client resources, monitoring
resources, and experiment results resources. The server resource will contain
all the necessary resources to run the FL server matching the user
configuration, and the same can be said for the client resources. The
monitoring resource will include all the resources needed to run the
monitoring server, which will monitor the distributed infrastructure layer and
record the usage of the resources. The experiment results resource will have
all the resources necessary to run an application to display the results of
the experiments run in the framework.

###  3.3 Application Layer

The application layer will run on top of the resource layer in each
correspondent resource. It contains all the applications necessary to run an
FL algorithm in the framework, such as the FL server application, FL client
application, monitor agent application, monitor server application, and the
experiment results visualization application. Subsections 3.3.1 to 3.3.4
detail each of the mentioned applications.

####  3.3.1 FL Server Application

The FL Server application can be divided into five major components that
enable FL testing in a distributed environment.

Server

is responsible for communicating with the clients and controlling the entire
FL learning process, which includes selecting available clients to start
training; control of the number of FL server rounds and round timeout; global
model parameters aggregation and distribution and retrieval of model
parameters.

Model

is responsible for server-side initialization of the global model parameters
since some global model aggregation strategies need it to start FL learning
training and enable the usage of different models to learn weight balancing of
client models to distribute parameters.

Strategies

selects and configures supported global model aggregation strategies from the
user-specified configuration.

Dataset

is a collection of datasets supported by the framework used by the application
to handle the dataset configuration in memory and to properly obtain the raw
data to create data distributions via Storage Manager.

Storage Manager

is responsible for distributing the raw data of the configured dataset
throughout the distributed storage infrastructure. For example, suppose a user
wants to test an FL algorithm in a dataset using an unbalanced non-iid data
distribution with ten clients. The storage manager will separate the data into
ten unbalanced and biased non-iid data parts. The storage manager of the
server application is also responsible for managing where each experiment will
write its data and where the client will output their results data in the
system.

####  3.3.2 FL Client Application

The FL client application can be divided into three major components.

Client: is responsible for the training and testing algorithms run on the
client side and the connection with the server.

Model: is responsible for selecting the desired model configured by the user
to be used by the client to train it.

Storage Manager: is responsible for loading the distributed data for training
in the client and handling the experiments and test results of the client in
the storage.

####  3.3.3 Monitoring Application

The Monitoring Server Application is responsible for gathering resource usage
data of the distributed infrastructure and storing it for further
visualization and analysis by the Monitoring Visualization Application. Each
Monitoring Agent is responsible for gathering local resource usage data from
the distributed resources and sending the metrics to the Monitoring Server
Application.

####  3.3.4 Experiment Visualization Application

This application is responsible for querying data from already run experiments
and enabling the user to visualize the data for each step of the FL training
experiment in customizable dashboards.

##  4 Tools and Frameworks

This section provides an overview of the tools and frameworks considered to
implement the conceptual framework described in Section 3.

###  4.1 Orchestration, Deployment, and Building

This section presents tools related to orchestration, deployment, and building
of containers.

####  4.1.1 Docker

Docker111https://docker.com/ (accessed April 24th, 2024) provides the ability
to package and run an application in a loosely isolated environment called a
container. The isolation and security allow many containers to run
simultaneously on a host. Containers provide a robust solution for bundling
software and its dependencies into a transportable unit that can run
seamlessly across diverse computing environments. A containerized application
encapsulates all its libraries, settings, and tools within an isolated
environment. Developers can avoid the arduous task of addressing compatibility
issues with varied hardware and software and concentrate solely on the
application’s functionality. This inherent portability of containers
eradicates the need to reconfigure applications for distinct environments,
ensuring uniformity and dependability.

####  4.1.2 Kubernetes

Kubernetes222https://kubernetes.io/ (accessed April 24th, 2024) is a portable,
extensible, open-source platform for managing containerized workloads and
services, which facilitates both declarative configuration and automation. It
has a large, rapidly growing ecosystem. Kubernetes services, support, and
tools are widely available. A Kubernetes cluster consists of a set of worker
machines, called nodes, which run containerized applications. Every cluster
has at least one worker node. The worker node(s) host the Pods, which are the
components of the application workload. Heterogeneous edge computing devices
can be easily integrated and managed by Kubernetes as worker nodes. This also
facilitates the deployment of different FL algorithms and the creation of
varying testing scenarios in a distributed infrastructure environment.

###  4.2 Libraries and Frameworks

This section approaches the utilized libraries and frameworks to run federated
learning (FL).

####  4.2.1 PyTorch

PyTorch333https://pytorch.org/ (accessed April 24th, 2024) is a popular open-
source machine learning library that was created by Meta’s AI research team.
It is used to develop and train deep learning models and is written in Python,
which makes it easy to use and integrate with other Python libraries. PyTorch
was the main library for training and testing the FL models used in this work.

####  4.2.2 Poetry

Poetry444https://python-poetry.org/ (accessed April 24th, 2024) is a tool for
dependency management and packaging in Python. It allows you to declare the
libraries your project depends on, and it will manage (install/update) them
for you. Poetry offers a lock file to ensure repeatable installs and can build
your project for distribution. It was used in this work to create the packages
of the FL client and server applications.

####  4.2.3 Flower

As discussed in this work, the concept of federated learning emerged in
response to the need to leverage data from multiple devices while ensuring its
privacy. However, federated learning introduces two additional challenges not
present in traditional machine learning: scaling to various clients and
dealing with data heterogeneity.

To address these challenges, as proposed in [27], Flower (flwr) has been
developed as an open-source framework for building federated learning systems.
Flower provides two primary interfaces: the client and the server. These
interfaces enable the decentralization of standard centralized machine
learning solutions by implementing the necessary methods, making building and
deploying federated learning systems easier. In Flower’s
architecture555https://flower.dev/docs/architecture.html (accessed April 4th,
2024) each edge device in the training process runs a Flower client containing
a local machine-learning model. Flower provides a transparent connection via
the Edge Client Proxy using an RPC protocol such as gRPC to ensure
connectivity between the clients and the server.

###  4.3 Monitoring and Visualization

Finally, monitoring, storage, and data visualization tools are presented in
this section.

####  4.3.1 Prometheus

Prometheus666https://prometheus.io/ (accessed April 24th, 2024) is an open-
source systems monitoring and alerting toolkit originally built at SoundCloud.
Since its inception in 2012, many companies and organizations have adopted
this tool, and the project has a very active developer and user community. It
is now a standalone open-source project maintained independently of any
company. Prometheus was used as the primary application for monitoring the
usage of resources in the PoC solution of this work.

####  4.3.2 Rook Ceph

Rook Ceph777https://rook.io/ (accessed April 24th, 2024) is a storage solution
that combines the capabilities of the Rook storage orchestrator with the Ceph
distributed storage system. Rook is an open-source tool for managing storage
systems on Kubernetes, while Ceph is a distributed object and file storage
system that provides scalability, reliability, and performance. Rook Ceph was
the persistent storage for the client and server application.

####  4.3.3 Grafana

Grafana888https://grafana.com/ (accessed April 24th, 2024) is a data
visualization tool commonly used with Prometheus that allows you to query,
visualize, alert on, and understand your metrics. It was used as the primary
tool to visualize Prometheus’s resource usage.

##  5 Proof-of-Concept Implementation

Previous sections detailed the conceptual framework proposed and the tools
considered for its implementation. This section details the current PoC
implemented to run fully functional FL scenarios in a distributed
infrastructure with multiple clients and different data distributions. The
source code of the implementations of the software components described in
this section, as well as configuration files used to allow the deployment and
execution of the experiments discussed in the following sections, are
available at our GitHub repository999 https://github.com/Open-Digital-
Twin/framework-fl-testing.

Figure 2: Implementation architecture with all tools used in our PoC

The PoC consists of an end-to-end edge-like environment solution using edge
computing devices orchestrated by Kubernetes to run the FL applications, the
server and the clients, and the applications for monitoring resource usage.
Figure 2 shows the complete diagram of the PoC solution implemented. The
distributed infrastructure layer comprises the Kubernetes cluster deployed at
the Institute of Informatics, which serves as the foundation for each
resource. The application layer can be visualized on top of the resources as
containerized Docker images of each application running in the cluster pods.
Further sections will go through more details about the layers, including how
they were assembled with the tools and frameworks presented in Section 4, the
underlying infrastructure of the Kubernetes cluster, and how the components of
the PoC work.

###  5.1 Distributed Infrastructure Layer

The Kubernetes cluster at the Institute of Informatics includes a varied
number of computing resources, with 352 CPUs, 544 GiB of RAM, and 6.3 TiB of
disk space spread across 43 distinct nodes, as indicated in Table 1. These
nodes are conveniently categorized into three classifications: “computer”,
“edge” and “server” enabling experimentation with pods that are constrained to
specific machines with hardware that closely matches the real-life devices
being simulated. For instance, the “edge” label encompasses a total of twenty-
three Raspberry Pi devices (three Raspberry Pi 4s and twenty Raspberry Pi 3s)
with less powerful hardware specifications relative to the “computer” and
“server” machines.

Type | Nodes | Total RAM | Total CPUs cores | Total storage space
---|---|---|---|---
Computer | 14 | 110.9GiB | 56 | 587.2GiB
Edge | 23 | 40.6GiB | 92 | 559.2GiB
Server | 6 | 392.4GiB | 204 | 5.2TiB
Table 1: Institute of Informatics Kubernetes Cluster Capacity

A network switch interconnects the cluster nodes and has distributed storage
configured using Rook Ceph Filesystem (subsection 4.3.2) to build a layer on
top of the storage resources. This enables the mounting of Persistent Volumes
(PV) from the storage pool via a Persistent Storage Claim (PVC) into the
container images that can be shared between applications to enable data
distribution by the server and saving experiment results from each client.
When a pod requests storage resources, it creates a PVC that specifies the
amount of storage required and any other requirements, such as access mode and
storage class. The Kubernetes scheduler then looks for an available PV that
matches the requirements specified in the PVC. If a matching PV is found, it
is bound to the PVC, and the pod can use the storage resource provided by the
PV. The advantage of using PVs and PVCs is that they provide a level of
abstraction between the pod and the underlying storage infrastructure. This
allows pods to request storage resources without having to know the details of
the underlying storage infrastructure.

Knowing the context and making a parallel with the conceptual framework, in
the PoC, the infrastructure of our Kubernetes cluster serves as the
distributed infrastructure, and Kubernetes plays the infrastructure manager
role.

###  5.2 Resource Layer

Kubernetes utilizes labels to organize objects. They are key-value pairs that
can be attached to Kubernetes objects such as pods, services, nodes, and
deployments and can be used to identify and group related objects. This is a
powerful mechanism for selecting and manipulating subsets of objects based on
specific criteria and can also be used to manage nodes in a Kubernetes
cluster. Nodes are the worker machines that run containerized applications and
services in a Kubernetes cluster. By attaching labels to nodes, we can assign
specific roles or attributes to them and use those labels to manage and
schedule workloads on those nodes.

For example, one can label nodes based on their hardware characteristics, such
as CPU or memory capacity, and then use those labels to schedule workloads
that require specific hardware requirements. The Kubernetes Cluster of the
Institute of Informatics uses the “node-type” label to identify if a node is a
“computer”, “server” or “edge” type of computational device.

Finally, in the PoC solution, the resource layer used is a total reflex of the
configuration set in the Kubernetes deployment file. Labels were used to
assign where each containerized application would run and how many node
resources would be available for them to use.

###  5.3 Application Layer

The application layer of the PoC solution is composed of all the Docker images
built by the author, such as the FL server, client, and experiment results
images, which contain all the implemented logic for FL algorithm testing. For
monitoring purposes, Prometheus monitoring installed in the Kubernetes cluster
was used to capture resource usage by the applications, and Grafana was used
to visualize the data in charts.

Figure 3 shows how an experiment starts from the beginning to the end in the
PoC solution from a high-level overview. In the blue boxes, we can see the
actions of the Kubernetes cluster, the yellow ones from the FL Server
application, and the orange ones from the FL clients. From a high-level
perspective, the user configures a scenario in Lens101010Lens is a GUI to
simplify the interaction with kubectl: https://k8slens.dev/ (accessed April
24th, 2024) for deployment, and Kubernetes scales the necessary resources and
deploys the applications. Afterward, the FL server handles the data
distribution of the dataset and starts the FL training algorithm. The client
waits for the server to connect with it, runs local training rounds, and
returns the model parameters to the server. The server receives those
parameters, aggregates them using the configured aggregation method by the
user in the experiment layout, and distributes them back to the clients, who
will start local training rounds again with the new parameters. After all
server rounds are done, the FL server will save the results in the correct
output folder of the experiment, and the experiment will be over.

Figure 3: High-level flowchart of experiment execution in the PoC solution

The GitHub repository used in the development of the solution was organized
into isolated packages containing all necessary code, data, or declarations of
each component to maintain independence and enable better re-usability. The FL
server and client applications were also developed to run in bare-metal
environments. In addition, to allow easy application testing during
development, you can run a docker-
compose111111https://docs.docker.com/compose/ (accessed April 24th, 2024)
environment to deploy the application locally using the built-in docker
images. Further subsections will detail each one of the applications and their
role in the PoC solution.

####  5.3.1 Server Application

The server is a containerized Python application with the FlowerML server
framework as a dependency. The server storage manager is responsible for
initializing the experiment root path in the distributed storage, where the
results and logs of the currently deployed run of each client will be saved.
It will also be responsible for receiving the models trained by the clients,
averaging the received parameters using the selected strategy, and then
updating the clients’ models with the averaged parameters. The connection is
done to the clients through a gRPC connection with SSL encryption. The
algorithm is outlined in Algorithm 1.

1 load environment variables with user parameters;

2 if  _$experiment\\_path$ don’t exists_ then

3 initialize $experiment\\_path$ in storage;

4 create a temporary folder for the current experiment run;

5 initialize experiment configuration file ;

6

7load experiment configuration file;

8 if  _dataset configuration changed or is empty_ then

9 load raw data of the dataset;

10 create new data distribution using user parameters;

11 save the data batches in the experiment path ;

12 save the new dataset configuration in the configuration file;

13

14else

15 use current dataset configuration;

16

17configure server strategy with the selected $fl\\_strategy$;

18 initialize server global model parameters;

19 open connection in $server\\_address$;

20

21while  _$minimum\\_clients$ have not yet connected_ do

22 wait;

23

24for  _$i\leftarrow rounds$ to $0$_ do

25 receive parameters from clients;

26 aggregate client parameters;

27 send updated parameters back to clients;

28 evaluate current accuracy from clients;

29

30save temporary experiment folder with results to current experiment run
path;

31 update experiment configuration file;

Algorithm 1 Server application

The server address, number of server rounds, dataset, data distribution,
global aggregation strategy, client local rounds, and minimum number of
connected clients are parameterized for the server application and can be
changed in each deployment. From the folder structure and source code files
available at our GitHub repository, it is easy to correlate each part of the
application with the conceptual framework since each source code file
encapsulates its corresponding component, as we explain in the following.

dataset.py

contains all the implemented classes of the supported datasets by the
framework used by the application to handle the dataset configuration in
memory and how to obtain the raw data of the dataset properly. At the time of
this work, the CIFAR-10, CIFAR-100, and FMNIST datasets were implemented.
Further development of this class can enable any dataset to be compatible with
the framework. Section 6.1 will detail each one of the currently supported
datasets.

main.py

contains the main program loop and the server component of the application,
which is responsible for communicating with the clients and controlling the
entire FL learning process, which includes selecting available clients to
start training; control of the number of FL server rounds and round timeout;
global model parameters aggregation and distribution and retrieval of model
parameters. All the implemented packages are included here, and the central
server program runs as demonstrated in Algorithm 1.

model.py

contains the supported models and is responsible for server-side
initialization of global model parameters since some global model aggregation
strategies need it. At the time of this work, the current used models for
testing are a simple CNN, googlenet [28] and resnet [29]. These models were
chosen due to the simplicity and popularity. Further development of classes
can enable any models to be compatible with the framework.

storage.py

contains the storage manager class, which is responsible for distributing the
raw data of the configured dataset throughout the distributed infrastructure
storage to achieve user requirements for testing. The storage manager of the
server application is also responsible for managing where each experiment will
write its data and where the client will output their results data in the
system. At the time of this work, the storage manager can create data
distributions by changing the following parameters:

Balance

boolean parameter specifies if the data should be balanced between clients. If
$true$, the dataset is balanced, meaning the data batches have the same number
of samples. Otherwise, if $false$, the number of data samples differs between
them.

Non-IID

boolean parameter specifies whether the data should be Non-IID or IID between
clients. If $true$, the dataset is Non-IID, meaning there is a class imbalance
between the clients. Otherwise, if $false$, the dataset has a balanced class
distribution.

Distribution

this parameter specifies how data should be distributed regarding the dataset
classes if $Non-IID=true$. If set to pat, it will generate a pathological
scenario of class imbalance, whereas each client will have only a subset of
classes. If set to dir:, it will use a heterogeneous unbalanced Dirichlet
distribution of the classes, similar as seen in [30], an $\alpha$ parameter
can modify that to change the distribution aspects. Further development of
this class can enable more data distributions to be compatible with the
framework.

Further development of the storage manager class can enable more types of data
distributions to be compatible with the framework.

strategies.py

contains the strategy for supported global model aggregation strategies from
the user-specified configuration. At the time of this work, the supported
strategies implemented are FedAvg, FedAvgM, FedYogi, and FedOpt. Further
development of this class can enable more strategies to be compatible with the
framework.

####  5.3.2 Client Application

The client waits for the server storage manager to initialize the experiment
path in the distributed storage, connects to the server through a gRPC
connection encrypted with SSL, and performs the pre-defined number of epochs
received from the server, which is the number of times that the data set
passes through the neural network. When multiple clients are running, an
individual client is oblivious to the existence of the other clients – it can
only communicate with the server. The algorithm for the client application can
be seen in Algorithm 2.

1 load environment variables with user parameters while  _$experiment\\_path$
not initialized by the Server_ do

2 wait;

3

4start the connection with a server in $address$;

5 while  _server connection is open_ do

6 for  _$i\leftarrow epochs$ to $0$_ do

7 iterates through the dataset batch training the local model;

8 saves current results of a round in the experiment path temporary folder;

9

10 upload model to server;

11 receive updated parameters;

12 update local parameters;

13

14

Algorithm 2 Service service loop

The client will only upload the model when the server requests the models, and
the loop will end when the server finishes its rounds. From our folder
structure at GitHub, it is easy to correlate each part of the application with
the conceptual framework since each source code file encapsulates its
corresponding component. The client’s models and the address they will connect
are parameterized for the client application and can be changed in each
deployment.

client.py

contains the client classes with the utilized training and testing algorithms.
At the time of this work, it would only support one PyTorch training
algorithm.

main.py

contains the client application main loop described in Algorithm 2. It is
responsible for initializing the environment variables used to set parameters,
such as the local model utilized for training, and common variables, such as
the experiment path, to save results.

model.py

contains all the supported models described in the server application section.

storage.py

contains the storage manager class of the client, which is responsible for
loading the distributed data batch for training and testing the distributed
storage and saving the experiments test results of the client in the storage.

####  5.3.3 Monitoring Application

Prometheus uses the Kubernetes API to discover the various resources it needs
to monitor in the cluster, such as pods, services, deployments, nodes, and
more. It does this by querying the Kubernetes API server for information about
the desired resources and then collecting metrics data from these resources.

For example, to monitor a Kubernetes pod, Prometheus will query the Kubernetes
API server to get information about the pod’s name, namespace, labels, and
other metadata. Prometheus will then use this information to collect metrics
data from the pod, such as CPU and memory usage, network traffic, and other
metrics. Grafana queries the data from the Prometheus database to enable real-
time visualization of resource usage in dashboards. Some sample charts of
these dashboards are presented in Section 8.

####  5.3.4 Experiments Results Application

To retrieve the experiment results from the Kubernetes cluster, a container
running an Ubuntu base image from Docker was used to mount the used PVs and
access the results. Since the results are stored in files, we can copy them to
the local machine to read and interpret them. Figure 4 illustrates how the
experiments that run in the PoC are saved in the distributed storage.

.temp

contains the partial results while running the application. This directory is
deleted after the server application storage manager has moved the finalized
run into its correct directory.

data

contains each client’s training and testing data batches used in the
experiment.

runs

contains the results for each experiment run. The logs subfolder of a run
holds all the logs from the server and client applications, and the results
subfolder contains the evaluation matrix for each local epoch run in the
clients. A copy of the configuration used in the experiment run is saved to
enable visualization of how the data was distributed in each client, how the
classes were distributed inside each data batch, and which model and global
aggregation strategy was used.

config.json

contains the last configuration used in the experiment. This enables testing
of the same data distribution using several parameters, strategies, and
models.

Figure 4: Experiments output folder structure

##  6 Experimental Setup

This section presents the experimental setup used to evaluate the solution
proposed in Section 3. Three experiments were conducted to demonstrate the
framework’s capabilities. The adjusted parameters are the dataset, global
model aggregation, local client model, client epochs, and server rounds. Each
experiment was run with ten clients — named client-0 through client-9 and
taken from the set of computational devices labeled computer in the cluster —
running with different data distributions. Also, in the Flower framework, the
fraction fit was set to 1, and the minimum available clients was set to 10 to
ensure running every server round with all the clients participating. The
specific clients involved in a given experiment may vary from one run to
another, showing how the framework can help reproduce real-life scenarios
effectively.

###  6.1 Datasets

The datasets considered for the experiments were the following:

CIFAR-10

Consists of 60,000 32x32 color images divided into ten classes, with 6,000
images per class. There are 50,000 training images and 10,000 test images
[31].

CIFAR-100

Similar to CIFAR-10, except it has 100 classes containing 600 images each.
There are 500 training images and 100 testing images per class. The 100
classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes
with a fine label (the class to which it belongs) and a coarse label (the
superclass to which it belongs) [31].

Fashion-MNIST

A dataset of Zalando’s1212footnotemark: 12article images consisting of a
training set of 60,000 examples and a test set of 10,000 examples. Each
example is a 28x28 grayscale image, associated with a label from 10 classes.
We intend Fashion-MNIST to be a direct drop-in replacement for the original
MNIST dataset [32] for benchmarking machine learning algorithms. It shares the
exact image size and structure of training and testing splits.

###  6.2 Performance Metrics

The results from each experiment are saved in performance score matrices and
written to log files, as described in Section 5.3.4. The F1 score is a
standard metric for evaluating classification models and is particularly
useful in cases with imbalanced data. It is the harmonic mean of two competing
scores: precision and recall. Precision is the proportion of true positives
out of all predicted positives (true positives + false positives), measuring
how often the model correctly predicts a positive class, and recall is the
proportion of true positives out of all actual positives (true positives +
false negatives), measuring how well the model can detect positive classes.
Thus, for binary classification problems, the F1 score is calculated by

$\textit{F1 score}=2\times\frac{precision\times recall}{precision+recall}$ (1)

Given the F1 score of each class, as defined by (1), we can aggregate them
into different metrics typically used for evaluating the performance of multi-
class classification models: the micro, macro, and weighted average F1 scores.
These metrics are computed for each client and saved to their log files. In
short, the performance metrics analyzed were the following:

Accuracy

Measures the model’s accuracy as the ratio between the number of correctly
predicted samples and the total number of samples in the test dataset. For
binary class datasets, the micro average F1 score equals accuracy.

Macro Averaged F1 Score

Useful when dealing with balanced datasets or evaluating the model’s overall
performance across all classes equally. It is measured by computing the
average F1 score of all classes in the dataset.

Weighted Averaged F1 Score

Useful for imbalanced datasets, as it assigns more importance to classes with
more samples by considering their distribution. It is measured by computing
the F1 score of each class and weighting the average F1 score by the number of
instances in each class.

Losses Distributed

Represents the average loss across all clients. The server’s log file contains
the average loss of each client.

Accuracy Distributed

Useful to assess the overall FL accuracy. It is measured by computing the
weighted average accuracy of all clients, where the accuracy of each client is
weighted by the number of samples in its dataset. The server’s log file
contains the accuracy of each client.

It should be noted that while prediction performance is a critical metric for
any machine learning algorithm, it is not the focus of this work. The primary
purpose of the experiments is to demonstrate the framework’s capabilities and
how it can provide an easy way to completely change an FL testing scenario by
only changing input parameters in a deployment file; higher prediction
performances would require much more training time, making running numerous
experiments in a reasonable time impracticable. It would also require further
tweaks on each model run, which is out of the scope of this work.

##  7 Results

The results were collected directly from the output folder and aggregated into
visualization charts to ease analysis. In total, 143 containers (server and
clients) were run in the cluster nodes, resulting in 143 log files containing
performance measures through each server round. Each experiment run generates
11 log files, one from the server’s container and 10 from the clients’
containers. As the number of output files can grow exponentially, we
considered only the last run of each experiment, i.e., 33 files were analyzed
to generate the results.

We used external commercial tools to manipulate the data collected from all
log files and show the results through meaningful charts. As a future work,
the framework can include automatic chart generation from the performance
measures generated from the experiments.

###  7.1 Experiment 1

Table 2 details the parameters used for the first experiment and the data
distribution across clients. The FMNIST dataset was used with a non-IID and
unbalanced data distribution. Also, the classes were distributed using a
Dirichlet distribution with $\alpha$ = 0.1. The global model aggregation used
was FedOpt, and the experiment ran with ten local epochs and ten server
rounds.

Table 2: Configuration parameters and data distribution for Experiment 1 Configuration |  | Data Distribution |  |
---|---|---|---|---
Parameter | Value | Client | Train Batch Size | Test Batch Size
Dataset | FMNIST | client-0 | 6345 | 2116
Experiment Name | kubernetes-test-1 | client-1 | 135 | 46
FL Strategy | FedOpt | client-2 | 4137 | 1380
Model | Multilayer Perceptron (MLP) | client-3 | 600 | 200
Data Balance | False | client-4 | 6109 | 2037
Non-IID | True | client-5 | 7389 | 2464
Class Distribution | Dirchlet | client-6 | 8968 | 2990
$\alpha$ | 0.1 | client-7 | 5734 | 1912
Server Rounds | 10 | client-8 | 6665 | 2222
Local Rounds | 10 | client-9 | 6413 | 2138

(a) Evolution of each client’s Micro F1 score throughout the server rounds.

(b) Evolution of each client’s Macro F1 score throughout the server rounds.

(c) Evolution of each client’s Weighted F1 score throughout the server rounds.

(d) Accuracy and losses distributed throughout the server rounds.

Figure 5: Results from Experiment 1

Figure 5(a) shows each client’s _accuracy_ through the server rounds. The
x-axis indicates the server round number, and the y-axis indicates the
computed accuracy. It is important to notice that accuracy measures the
percentage of correctly classified cases regardless of their classes,
assigning the same importance to all instances. The figures indicate that
clients with large amounts of data had a relatively high initial accuracy in
the first rounds. In contrast, the accuracy of clients with little data was
initially low. Over the server rounds, the chart reveals that the aggregate
model had a subtly negative impact on the accuracy of clients with more data
available while benefiting the accuracy of clients with less data.

Figure 5(b) shows the macro F1 score of each client through the server rounds.
Notice that macro F1 assigns the same importance to each class of the dataset,
which means this metric tends to exhibit better results as the performance of
all classes in the dataset improves evenly. The results suggest that the
performance may seem unacceptable when focusing on averaging the performance
of different classes with the same weight, despite some clients having a
reasonable accuracy, as shown in Figure 5(a). Even so, as a general trend, the
chart reveals performance increases over the server rounds.

Another way to evaluate the model’s performance is to measure the clients’
weighted F1 score, i.e., the average F1 score weighted by the number of
instances in each class, as exemplified in Figure 5(c). The chart shows that
some clients (5, 6, and 8) had a relatively high initial performance with a
slight decrease over time. Contrasting with the results in Figure 5(b), we can
note that the performance of these clients in the majority classes (with more
instances) is considerably higher in the initial rounds. In these majority
classes, however, the aggregated model worsened the performance over the
server runs, even though the overall performance considering all classes
improved. When observing client-6 in Figure 5(b), for example, there is a
performance leap between rounds 4 and 5, suggesting that the aggregated model
improved the model’s overall performance despite reducing performance for the
majority class.

Figure 5(d) shows the distributed accuracy over all clients and the
distributed loss of the FedOpt aggregation strategy in each server round. The
x-axis indicates the server round number, the left y-axis indicates the
accuracy, and the second y-axis indicates the losses. We can see that the
overall performance of the FL algorithm converged to an accuracy between 60%
and 70% with less than 100 losses.

Further analysis could be done to understand better how the dataset’s class
distribution impacted the results, as the framework saves this information in
the configuration file of each experiment run. Recall that the purpose of this
work is not to present a comprehensive evaluation of FL algorithms but to
demonstrate how the proposed framework can aid in doing so. Therefore, further
analysis can be subject of future work.

###  7.2 Experiment 2

As shown in Table 3, the second experiment was run using the CIFAR-10 dataset
with non-IID and unbalanced data distribution, similar to the first
experiment. However, we used a pathological class distribution for two classes
per client and the FedAvg global model aggregation. The experiment ran with
fifteen local epochs and ten server rounds. The results analysis follows the
same approach used in Section 7.1.

Table 3: Configuration parameters and data distribution for Experiment 2 Configuration |  | Data Distribution |  |
---|---|---|---|---
Parameter | Value | Client | Train Batch Size | Test Batch Size
Dataset | CIFAR-10 | client-0 | 1422 | 474
Experiment Name | kubernetes-test-2 | client-1 | 7578 | 2526
FL Strategy | FedAvg | client-2 | 1950 | 651
Model | CNN | client-3 | 7049 | 2350
Data Balance | False | client-4 | 1278 | 426
Non-IID | True | client-5 | 7722 | 2574
Class Distribution | Pathological | client-6 | 2463 | 822
Class per Client | 2 | client-7 | 6536 | 2179
Server Rounds | 10 | client-8 | 2326 | 776
Local Rounds | 15 | client-9 | 6673 | 2225

Figure 6(a) shows the accuracy of all clients through the server rounds. We
can see that clients 0 and 1 can initially classify a good part of the test
instances, but most clients cannot classify any cases. As more server rounds
are run, however, the performance of clients with high accuracy decreases
while the performance of clients with low accuracy increases. As a result, the
overall accuracy stabilizes in a medium range between 30% and 60%, i.e., the
aggregation model was able to distribute the performance among clients.

(a) Evolution of each client’s Micro F1 score throughout the server rounds.

(b) Evolution of each client’s Macro F1 score throughout the server rounds
considering all classes.

(c) Evolution of each client’s Weighted F1 score throughout the server rounds.

(d) Accuracy and losses distributed throughout the server rounds.

Figure 6: Results from Experiment 2

Figure 6(b) presents the macro F1 score of all clients through the server
rounds. It shows that clients 0 and 8 initially managed to obtain an excellent
performance in the two classes from which they had data. However, the
aggregation drastically harmed their performance, reducing it to less than
20%. On the other hand, the aggregation positively affected clients who
performed quite poorly initially. Nevertheless, the general performance of all
clients, considering all their classes equally important, turned out to be
very low.

In contrast, Figure 6(c) shows the weighted F1 score, suggesting a more
erratic behavior over the server rounds. We can note that clients 0 and 8,
which had a noteworthy decline in performance in the first round when all
classes had the same weight, as shown in Figure 6(b), had a different behavior
when considering the importance of each class proportional to the number of
instances. The performance of these clients for their majority classes dropped
after round 0 to less than 40% but increased after round 1 to between 60% and
70% and then dropped again throughout the server rounds. Deeper analyses must
be carried out to investigate such a behavior. Conversely, we also can note
that the performance of clients who had poor performance at the beginning had
consistent performance improvements over the server rounds. In general, the
average performance of all clients stabilized in the range between 30% and
40%.

Figure 6(d) shows the distributed accuracy over all clients and the aggregated
loss of the FedAvg strategy in each server round. The overall performance of
the FL algorithm converged to an accuracy between 40% and 50% with almost 300
losses.

(a) Evolution of each client’s Micro F1 score throughout the server rounds.

(b) Evolution of each client’s Macro F1 score throughout the server rounds.

(c) Evolution of each client’s Weighted F1 score throughout the server rounds.

(d) Accuracy and losses distributed throughout the server rounds.

Figure 7: Results from Experiment 3

###  7.3 Experiment 3

The last experiment was run using the CIFAR-100 dataset, which also had non-
IID but a balanced data distribution. Similar to the second experiment, a
pathological distribution of twenty classes per client was used. The global
model aggregation used was FedYogi, and the experiment ran with ten local
epochs and fifteen server rounds. Table 4 shows the parameters and the
balanced data distribution used in the experiment.

Table 4: Configuration parameters and data distribution for Experiment 3 Configuration |  | Data Distribution |  |
---|---|---|---|---
Parameter | Value | Client | Train Batch Size | Test Batch Size
Dataset | CIFAR-100 | client-0 | 4500 | 1500
Experiment Name | kubernetes-test-3 | client-1 | 4500 | 1500
FL Strategy | FedYogi | client-2 | 4500 | 1500
Model | resnet | client-3 | 4500 | 1500
Data Balance | True | client-4 | 4500 | 1500
Non-IID | True | client-5 | 4500 | 1500
Class Distribution | Pathological | client-6 | 4500 | 1500
Class per Client | 20 | client-7 | 4500 | 1500
Server Rounds | 15 | client-8 | 4500 | 1500
Local Rounds | 10 | client-9 | 4500 | 1500

The experiment deals with a classification problem that is fairly difficult in
itself because there are more classes to predict, and originally, there were
relatively few instances per class available. Furthermore, in the distributed
context, each client has access to only 20 of the classes and less than half
of the original dataset samples per class. Due to this, we expect poor
performance in this scenario.

The chart in Figure 7(a) shows that the proportion of predicted instances
generally increases for all clients, with some exceptions (see between rounds
9 and 13), demonstrating that, in general, aggregation benefits clients over
time.

Figure 7(b) shows the f1-macro measure. Thus, considering the performance for
all classes and those with equal importance, the general performance of all
clients is poor, not even reaching 10% of the macro f-measure. However,
verifying the general beneficial effects of aggregation throughout the rounds
is possible, with some exceptions.

Figure 7(c), with the f1-weighted measure, shows a scenario similar to that
seen in charts 6.9 and 6.10. That is, in general, the performance of most
clients improves with aggregation over the rounds, with some exceptions.
However, in this scenario, the classes represented in each client gain more
importance, which is responsible for increasing the performance in this metric
compared to the macro average.

Figure 7(d) shows the aggregated accuracy over all the clients and the
aggregated loss of the strategy in each server round. The X-axis indicates the
server round number, the left Y-axis indicates the accuracy, and the second
Y-axis shows the losses. It can be noted that the overall performance of the
FL algorithm converged between 40-45% accuracy and closer to 800 of loss. It
is curious how the loss of the strategy decreased in the beginning and
afterward increased to a higher level than it began. It reflects the overall
poor performance of the clients.

##  8 Resource Usage Monitoring

The PoC solution uses the Grafana application to monitor the use of computing
and networking resources during the experiments, allowing the visualization of
the data collected by the Prometheus. To demonstrate some of the capabilities
of the PoC solution on monitoring, an analysis of CPU usage and network
traffic was made in Experiment 3 (subsection 7.3). It is important, once
again, to emphasize the focus on the demonstration of the capabilities and not
on the achieved performance. Similar analyzes can be done for all the
experiments run since all the metrics are saved in the Prometheus server
database.

###  8.1 CPU Usage

Figure 8 shows each client and server’s CPU resource usage in the third
experiment. The X-axis indicates the timestamp of the metric, and the Y-axis
indicates the CPU usage in percentage, and each data sample has a 10-second
interval between one another.

Figure 8: CPU Usage from server and clients in experiment 3

We can notice that the figure is quite chaotic since all the ten clients and
the server are plotted in it. However, we can see the execution of each of the
fifteen server rounds ran in the experiment and that some of the clients had a
CPU usage of 20-30% while others stayed between 1-5%. This can be explained by
the client’s CPU processing power heterogeneity since some clients run on
different hardware profiles. We also noticed that the workload was distributed
evenly across the clients since the data distribution was balanced.

The choice of using Grafana for experiment data visualization allows us to
dynamically select which data points to plot in the chart. So, to better
analyze the assumption about the first chart, we can choose some clients of
specific hardware and others of another specification to compare them. Figure
9 demonstrates this selection using the same chart with now only client-5,
client-4, and the server CPU usage being plotted. It becomes evident that
client-4 (Intel Core i5-3210M CPU @ 2.50GHz) presented higher CPU resource
usage and needed more time to process the same number of samples as client-5
(Intel Core i7-5500U CPU @ 2.40GHz).

Figure 9: CPU Usage from two clients with different hardware in experiment 3

As for the server (orange line in Figure 9), the peak of CPU usage is
noticeable at the beginning of the experiment as a brief bump (near 1%) before
the first training round. This can be explained by the fact that the data
distribution created by the server is done at the beginning of the experiment.
We can also see that the server’s CPU resource usage is almost negligible
compared to any client’s. The server only uses the CPU in between training
rounds to aggregate the parameters received from the clients, which is a
pretty simple task compared to the training done by the clients, thus
explaining this phenomenon. The hardware profile of the server node is the
same as client-5 (Intel Core i7-5500U CPU @ 2.40GHz), thus the amount of CPU
resources needed for client and server tasks can be analyzed and understood in
perspective.

Figure 10: CPU Usage from server and clients in experiment 1

To demonstrate that the analysis done in experiment 3 could be done in other
experiments, we present a sample chart from experiment 1. Figure 10 shows the
same chart of figure 8 plotted with data from the first experiment. It is
interesting to see how the data distribution affects the CPU usage of the
clients. We can visualize the gaps in between them due to the size of the
samples being different for each client, even for those running in similar
hardware profiles. Further analysis can be done in the Grafana dashboard to
understand the behavior of each one of the clients individually and in any
time frame specified. We will not present an exhaustive use of interactions
with Grafana charts in this paper for the sake of brevity.

###  8.2 Network Traffic

Similar to the CPU usage charts presented in the previous subsection, the same
type of analysis can be done to visualize the network traffic between the
clients and the server. Prometheus gathers information on traffic flowing in
and out of Pods, collecting metrics such as, packets in/out, bytes in/out, and
errors/drops, whereas Grafana enables interactive visualization and filtering
of these metrics in different types of charts. To demonstrate this we created
a simple dashboard with line charts containing bytes in/out of all Pods
participating in the experiments. Two of these charts are presented in the
following with data from the third experiment over the same time frame
previously used in Figures 8 and 9.

Figure 11 shows the received (inward) traffic for each one of the clients
involved in the third experiment as stacked areas. The X-axis indicates the
timestamp of the metric, and the Y-axis indicates the network traffic in
Megabytes per second (MBps). Each data sample is also collected with a
10-second interval between them. The first aspect to notice is that peaks of
traffic are presented at the beginning and end of each round, which is
expected since that is when communication between client and server should
occur. The aggregated traffic peak considering all 10 clients was around 80
MBps. We did not show traffic sent (outward) from each client Pod because the
communication follows a client-server paradigm, i.e., all the traffic flowing
out of clients goes into the server and vice-versa.

Figure 11: Receive bandwidth from server and clients in experiment 3

To better understand client and server behavior, we can filter only one client
and the server in the same chart. For example, Figure 12 shows the traffic
received (inward) in the server and client-9 for the third experiment. One can
see that the communication pattern is always the server receiving data and,
afterward, the client. This is expected since, initially, the clients connect
to the server, and the server accepts the connection. During training, the
pattern is maintained since the server receives the parameters from the
clients, and the client receives the aggregated parameters from the server. It
is possible to visualize that inward traffic at the server side (received
parameters from all clients) generates traffic peaks at most around 10-12
MBps, which means that in this experiment clients send less data than they
receive. In other words, the amount of data required to represent the client
parameters is smaller than the aggregated parameters generated at the server.
This type of deeper understanding of the resource consumption patterns in the
client-server interaction can help developers to better tune their FL-based
systems.

Figure 12: Receive bandwidth from server and client-9 in experiment 3

The Prometheus monitoring application collects many other metrics, such as
network packets, memory usage, or any other custom metric from applications
configured in the system. For the purposes of demonstration, in this work, we
considered only the CPU and the network due to their relevance in FL.
Nevertheless, the interested experimenter can easily create new dashboards to
monitor their experiment metrics in Grafana as they wish.

##  9 Conclusion

FL is an ML paradigm that is constantly being adopted due to its distributed
approach, the necessity of data privacy, and the vast increase in IoT devices.
Heterogeneous data distributions/contents of clients proved to be a real
challenge and gained great notoriety over the last few years. Proposals, such
as DRL algorithms to dynamically learn the weight of the contributions of each
client at each round, continue to be developed and will be fundamental to
increasing the accuracy and usability of FL in more applications. Testing and
assessing this FL algorithm can be a challenging and complex task due to the
systems’ distributed nature and the possible scenarios. To address this
complexity, this work proposed a conceptual framework to facilitate testing
federated learning scenarios in distributed computing environments with
different types of data distributions. This work achieves the intended
proposal by proposing a conceptual framework for testing federated learning
scenarios and demonstrating an implementation of those concepts in the PoC
solution. The solution developed shows that creating an edge-like FL testing
framework that can scale to several different types of real-life scenarios
using distributed heterogeneous computing and other data distributions is
possible, inspiring further development of the concepts and improvement of the
PoC solution.

To prove the capabilities of the PoC solution, three experiments with three
different FL scenarios were conducted. The results showed how it is possible
to analyze the impacts of class and data imbalance in a real-life distributed
system of FL through the framework via the f1-measures outputted in the
experiment results. It was also possible to see the resource usage of the
applications via the monitoring solution and to demonstrate the impact of the
underlying heterogeneous infrastructure used.

The critical point taken from this work is that designing a solution from the
beginning that has independence between infrastructure and applications has
shown to be very efficient and effective during the development phase.
Containers enabled re-usability, isolation, and easy testing of the
applications despite the environment in which they were deployed, either
locally or in the distributed cluster. Also, this allowed the system to scale
horizontally by adding more nodes to the cluster from an infrastructure
perspective or by creating more application replicas from an application point
of view.

Further improvement could be made to generate automated visualizations of each
client and the server’s aggregated results (section 7). For instance, it
should be possible to output the results of the FL algorithm’s performance to
the Prometheus server so that it can be further visualized by Grafana or
another visualization tool with access to its database.

This work also opens new opportunities to develop experiments with federated
learning scenarios. Fault tolerance experiments could also be done in the PoC
solution to enable further improvement in the framework’s reliability. From an
application point of view, one way would be to test limited connectivity
scenarios by disconnecting clients between training. Stress testing from an
infrastructure point of view would also be interesting to understand the
platform’s limitations.

All of the code is publicly available and has been developed with
extensibility in mind. Future works could also extend the datasets, models,
and FL strategies supported and better improve the applications as a whole.

## Acknowledgments

This study was partially funded by CAPES, Brazil - Finance Code 001. This work
was also supported by the São Paulo Research Foundation (FAPESP), Brazil grant
23/00673-7.

## References

  * [1] H. B. McMahan, E. Moore, D. Ramag, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data (2017).  arXiv:1602.05629.
  * [2] G. Darling, What is edge computing?, https://developer.ibm.com/articles/what-is-edge-computing/ (accessed August 28th, 2022) (2022).
  * [3] M. Duan, D. Liu, X. Chen, R. Liu, Y. Tan, L. Liang, Self-balancing federated learning with global imbalanced data in mobile systems, IEEE Transactions on Parallel and Distributed Systems 32 (1) (2021) 59–71.  doi:10.1109/TPDS.2020.3009406.
  * [4] A. Plaat, Deep Reinforcement Learning, Springer Nature Singapore, 2022.  doi:10.1007/978-981-19-0638-1.
URL https://doi.org/10.1007/978-981-19-0638-1

  * [5] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, M. Jirstrand, A performance evaluation of federated learning algorithms, DIDL ’18: Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning (2018) 1–8.
  * [6] A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, D. Ramage, Federated learning for mobile keyboard prediction, arXiv preprint arXiv:1811.03604 (2018).
  * [7] M. Shaheen, M. S. Farooq, T. Umer, B.-S. Kim, Applications of federated learning; taxonomy, challenges, and research trends, Electronics 11 (4) (2022).  doi:10.3390/electronics11040670.
URL https://www.mdpi.com/2079-9292/11/4/670

  * [8] J. Research, Iot the internet of transformation 2020, https://www.juniperresearch.com/whitepapers/iot-the-internet-of-transformation-2020 (accessed September 9th, 2022) (2020).
  * [9] G. Darling, Iot vs edge computing, https://developer.ibm.com/articles/iot-vs-edge-computing/ (accessed September 9th, 2022) (2022).
  * [10] X. W. Enwei Guo, W. Wu, Adaptive aggregation weight assignment for federated learning: A deep reinforcement learning, ACM (2022) 1–3.
  * [11] C. Huang, W. Chen, Y. Chen, S. Yang, Y. Zhang, Dearfsac: An approach to optimizing unreliable federated learning via deep reinforcement learning (2022).  arXiv:2201.12701.
  * [12] Y. Sun, S. Si, J. Wang, Y. Dong, Z. Zhu, J. Xiao, A fair federated learning framework with reinforcement learning (2022).  arXiv:2205.13415.
  * [13] J. Zheng, K. Li, N. Mhaisen, W. Ni, E. Tovar, M. Guizani, Exploring deep reinforcement learning-assisted federated learning for online resource allocation in privacy-persevering edgeiot (2022).  arXiv:2202.07391.
  * [14] H. Liu, Z. Huang, J. Wu, C. Lv, Improved deep reinforcement learning with expert demonstrations for urban autonomous driving, in: 2022 IEEE Intelligent Vehicles Symposium (IV), 2022, pp. 921–928.  doi:10.1109/IV51971.2022.9827073.
  * [15] X. Wang, J. Song, P. Qi, P. Peng, Z. Tang, W. Zhang, W. Li, X. Pi, J. He, C. Gao, et al., Scc: an efficient deep reinforcement learning agent mastering the game of starcraft ii, in: International Conference on Machine Learning, PMLR, 2021, pp. 10905–10915.
  * [16] B. Sridharan, S. Mehta, Y. Pathak, U. D. Priyakumar, Deep reinforcement learning for molecular inverse problem of nuclear magnetic resonance spectra to molecular structure, The Journal of Physical Chemistry Letters 13 (22) (2022) 4924–4933, pMID: 35635003.  arXiv:https://doi.org/10.1021/acs.jpclett.2c00624, doi:10.1021/acs.jpclett.2c00624.
URL https://doi.org/10.1021/acs.jpclett.2c00624

  * [17] N. Rudin, D. Hoeller, P. Reist, M. Hutter, Learning to walk in minutes using massively parallel deep reinforcement learning, in: A. Faust, D. Hsu, G. Neumann (Eds.), Proceedings of the 5th Conference on Robot Learning, Vol. 164 of Proceedings of Machine Learning Research, PMLR, 2022, pp. 91–100.
URL https://proceedings.mlr.press/v164/rudin22a.html

  * [18] W. Shi, J. Cao, Q. Zhang, Y. Li, L. Xu, Edge computing: Vision and challenges, IEEE INTERNET OF THINGS JOURNAL 3 (5) (2016) 637.
  * [19] F. Giust, X. Costa-Perez, A. Reznik, Multi-access edge computing: An overview of etsi mec isg, IEEE 5G Tech Focus 1 (4) (2017).
  * [20] S. Wan, S. Ding, C. Chen, Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles, Pattern Recognition 121 (2022) 108146.  doi:10.1016/j.patcog.2021.108146.
  * [21] S. Liu, L. Liu, J. Tang, B. Yu, Y. Wang, W. Shi, Edge computing for autonomous driving: Opportunities and challenges, Proceedings of the IEEE 107 (8) (2019) 1697–1716.  doi:10.1109/JPROC.2019.2915983.
  * [22] J. Konečný, H. B. McMahan, D. Ramage, P. Richtárik, Federated optimization: Distributed machine learning for on-device intelligence, Tech. rep., Google, Inc. (2016).
  * [23] S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečný, S. Kumar, H. B. McMahan, Adaptive federated optimization (2021).  arXiv:2003.00295.
  * [24] T.-M. H. Hsu, H. Qi, M. Brown, Measuring the effects of non-identical data distribution for federated visual classification (2019).  arXiv:1909.06335.
  * [25] Google, Federated learning: Collaborative machine learning without centralized training data, https://ai.googleblog.com/2017/04/federated-learning-collaborative.html (accessed November 2nd, 2021) (2017).
  * [26] X. Cheng, F. Shi, Y. Liu, J. Zhou, X. Liu, L. Huang, A class-imbalanced heterogeneous federated learning model for detecting icing on wind turbine blades, IEEE Transactions on Industrial Informatics (2022) 1–1doi:10.1109/TII.2022.3167467.
  * [27] D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão, N. D. Lane, Flower: A friendly federated learning research framework (2022).  arXiv:2007.14390.
  * [28] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions (2014).  arXiv:1409.4842.
  * [29] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition (2015).  arXiv:1512.03385.
  * [30] M. Yurochkin, M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang, Y. Khazaeni, Bayesian nonparametric federated learning of neural networks, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, Vol. 97 of Proceedings of Machine Learning Research, PMLR, 2019, pp. 7252–7261.
URL https://proceedings.mlr.press/v97/yurochkin19a.html

  * [31] A. Krizhevsky, G. Hinton, Learning multiple layers of features from tiny images, Tech. rep., University of Toronto (2009).
  * [32] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.  doi:10.1109/5.726791.