# Effects of direction reversals on patterns of active filaments

Leila Abbaspour<EMAIL_ADDRESS>Max Planck School
Matter to Life, University of Göttingen, Friedrich-Hund-Platz 1, 37077
Göttingen, Germany  Institute for Dynamics of Complex Systems, University of
Göttingen, Friedrich-Hund-Platz 1, 37077 Göttingen, Germany  Ali Malek
Institute for Dynamics of Complex Systems, University of Göttingen, Friedrich-
Hund-Platz 1, 37077 Göttingen, Germany  Stefan Karpitschka  Max Planck
Institute for Dynamics and Self-Organization, Am Faßberg 17, 37077 Göttingen,
Germany  Stefan Klumpp<EMAIL_ADDRESS>Institute for
Dynamics of Complex Systems, University of Göttingen, Friedrich-Hund-Platz 1,
37077 Göttingen, Germany  Max Planck School Matter to Life, University of
Göttingen, Friedrich-Hund-Platz 1, 37077 Göttingen, Germany

###### Abstract

Active matter systems provide fascinating examples of pattern formation and
collective motility without counterpart in equilibrium systems. Here, we
employ Brownian dynamics simulations to study the collective motion and self-
organization in systems of self-propelled semiflexible filaments, inspired by
the gliding motility of filamentous Cyanobacteria. Specifically, we
investigate the influence of stochastic direction reversals on the patterns.
We explore pattern formation and dynamics by modulating three relevant
physical parameters, the bending stiffness, the activity, and the reversal
rate. In the absence of reversals, our results show rich dynamical behavior
including spiral formation and collective motion of aligned clusters of
various sizes, depending on the bending stiffness and self-propulsion force.
The presence of reversals diminishes spiral formation and reduces the sizes of
clusters or suppresses clustering entirely. This homogenizing effect of
direction reversals can be understood as reversals providing an additional
mechanism to either unwind spirals or to resolve clusters.

##  I Introduction

Active matter is a class of systems that are inherently out of equilibrium
through coupling to an internal process of energy consumption that results,
for example in self-propulsion or growth Marchetti  _et al._ (2013); Bechinger
_et al._ (2016). Specifically, the case of self-propelled active particles has
received much attention in recent years, as it gives rise to intriguing
dynamics and mechanical behaviour at the collective level. Such systems are
ubiquitous in nature, examples include the cytoskeleton Hyman and Karsenti
(1996); Ndlec  _et al._ (1997); Surrey  _et al._ (2001), swarming bacteria
Mendelson  _et al._ (1999); Parrish  _et al._ (2002); Dombrowski  _et al._
(2004), tissues Salbreux and Jülicher (2017) ,and biofilms Hall-Stoodley  _et
al._ (2004). Finding suitable tools to study active matter systems is a
challenge for non-equilibrium statistical physics in order to elucidate the
dynamical behavior and the physical properties of these systems.

Self-propelled particles self-organize into macroscopic structures with
collective dynamics like clustering, swarming, and swirling. This type of
dynamical pattern formation has been subject to intensive research in recent
years, both from a theoretical or computational viewpoint Moore  _et al._
(2020); Joshi  _et al._ (2019); Bär  _et al._ (2020); Prathyusha  _et al._
(2018); Duman  _et al._ (2018); qing Shi and Chaté (2018); Wang  _et al._
(2018); Großmann  _et al._ (2016); Weitz  _et al._ (2015); Balagam and Igoshin
(2015); Isele-Holder  _et al._ (2015); Abkenar  _et al._ (2013); Wensink and
Löwen (2012, 2008); Abbaspour and Klumpp (2021) and experimentally Börner  _et
al._ (2002); Sumino  _et al._ (2012); Peruani  _et al._ (2012); Rabani  _et
al._ (2013); Dunkel  _et al._ (2013); Be’er  _et al._ (2013); Nishiguchi  _et
al._ (2017); Huber  _et al._ (2018), often using systems of bacteria either
swimming in a solution or gliding on surfaces as well as synthetic self-
propelled particles systems Mirfakhrai  _et al._ (2007).

On the theoretical side, most studies of collective effects in systems of
self-propelled particles have considered either spherical or point-like
particles Abbaspour and Klumpp (2021)or short and rigid rod-like particles, as
these are good approximations for the shape of the bacteria typically used in
experiments such as swimming Bacillus subtilis or Escherichia coli Nishiguchi
_et al._ (2017); Bär  _et al._ (2020); Wensink and Löwen (2012) and as
Myxococcus xanthus, gliding on surfaces Peruani  _et al._ (2012). The aspect
ratio of rod-like particles has a strong impact on the patterns, as high
aspect ratios promote (nematic) alignment of the particles and swarming
Wensink  _et al._ (2012).

At very large aspect ratios, the the finite rigidity of the particles is
expected to play a role. Therefore several computational recent studies have
addressed semi-flexible filamentous particles or self-propelled polymers
Prathyusha  _et al._ (2018); Duman  _et al._ (2018). Indeed, their collective
behavior is modulated by their bending rigidity. Moreover, flexibility even
results in new single-filament behavior, as individual flexible filaments are
seen to curl into spirals due to their self-propulsion Isele-Holder  _et al._
(2015).

Complicating the understanding of the behavior of the collective motion of
microbes, many of them in addition to active self-propulsion also perform
active directional changes such as the well-known run-and-tumble motion of
Escherichia coli Turner  _et al._ (2000). Typically, these active direction
changes are an integral part of the mechanisms for chemotaxis and other types
of tactic behaviors, through a coupling of the rates of directional change to
the direction of motion relative to a (chemical, light, etc.) gradient
direction. Such active directional changes have occasionally been included in
models for the motility and collective behavior of self-propelled particles
Großmann  _et al._ (2016, 2016); Santra  _et al._ (2021); Zhang  _et al._
(2018) or spreading of active polymers in porous media Kurzthaler  _et al._
(2021). For bacteria, the simplest direction change is a reversal, where the
whole filament reverses its direction of motion by $180^{\circ}$, as seen in
the gliding motility of M. xanthus, a short rigid rod Peruani  _et al._
(2012), as well as in gliding filamentous Cyanobacteria McBride (2001);
Tamulonis  _et al._ (2011a).

In this article, we use agent-based simulations to address how such reversals
affect the collective behaviors of self-propelled semi-flexible filaments at
overall relatively low density. To that end, we first consider the reference
case without reversals and analyze the formation of spirals and swarms or
clusters. Then we turn to the case with stochastic direction reversals,
generated by a Poisson process, and investigate their impact on these
patterns.

With that in mind, in this paper, we initially examine the self-assembly of
active filaments by modulating the bending stiffness and self-propulsion force
of the filaments, and then investigate how the introduction of reversals
modifies the collective behavior.

We focus on the interplay of the filaments’ self-propulsion force, bending
stiffness, and reversal rate as the control parameters that modulate the
pattern formation process. In the absence of reversals, the filaments form
spirals for high self-propulsion force and low bending stiffness, for larger
bending stiffness coherently moving clusters dominate the system. Introducing
reversals reduces the rates of forming either type of pattern, and thus the
system becomes gradually more isotropic with increasing reversal rate. The
suppression of either pattern can be explained by the dynamical pathway of
these structures: First, reversals add a mechanism that interrupts the
spooling of a filament onto itself and initiates unwinding. Second, similarly,
reversals admit filaments to leave a cluster as they break the coherence of
motion in clusters.

Figure 1:  Model of a self-propelled filament. (a) Sketch of a self-propelled
filament composed of monomers with diameter $\sigma$. The active force
inducing their self-propulsion acts tangentially along each bond. (Zoom) The
flexibility of the chain is modeled by a harmonic bending potential, where
$\theta_{i}$ is the angle formed by each successive triplet of monomers
$i-1,i,i+1$. (b) Sketch of the reversal mechanism of a self-propelled
filament: Upon a stochastic reversal with rate $\lambda$, a filament reverses
the direction of its self-propulsion force by $180^{\circ}$.

##  II Model

###  II.1 Self-propelled filaments

We consider a system of $N_{c}$ self-propelled semi-flexible filaments. The
individual filament is described as a chain of $N$ monomers located at
position s $(\vec{r}_{1},\vec{r}_{2},\cdots,\vec{r}_{N})$ in space (see Fig. 1
(a) for a schematic). Two consecutive monomers in the chain are connected by a
harmonic spring. In addition, the angle between three consecutive monomers
along the chain are subject to a harmonic bending potential, which controls
the flexibility of the chain. In the over-damped regime, the equation of
motion for each monomer can be written as:

$\zeta\dot{\vec{r}}_{i}=-\nabla_{i}U_{s}-\nabla_{i}U_{b}+\vec{F}^{a}_{i}+\vec{F}^{ex}_{i}.$
(1)

where $\dot{\vec{r}}_{i}$ is the velocity of the $i$-th monomer, $\zeta$ is
the friction coefficient. The four force contributions on the right hand side
of this equation represent stretching and bending of the chain, the active
force responsible for self-propulsion, and volume exclusion, respectively.

$U_{s}$ is the stretching potential due to the harmonic springs (each with an
equilibrium length of $r_{0}$ between consecutive monomers) with the form

$U_{s}=\frac{\kappa_{s}}{2}\sum_{j=2}^{N}(r_{j,j-1}-r_{0})^{2}.$ (2)

where $r_{j,j-1}=|\vec{r}_{j}-\vec{r}_{j-1}|$ is the distance between the
$j$-th and $j-1$-th monomer and $\kappa_{s}$ is the spring constant associated
with stretching.

$U_{b}$ is the harmonic bending potential which controls the flexibility of
the bonds. Considering $\theta_{i}$ as the angle formed by a consecutive
triplet of monomers $(i-1,i,i+1)$, given by
$\theta_{i}=cos^{-1}\left(\frac{{\vec{r}}_{i-1,i}\cdot{\vec{r}}_{i,i+1}}{|{\vec{r}}_{i-1,i}||{\vec{r}}_{i,i+1}|}\right)$,
this potential is defined as

$U_{b}=\frac{\kappa_{b}}{2}\sum_{j=2}^{N-1}(\theta_{j}-\pi)^{2}.$ (3)

where $\pi$ is the equilibrium angle of between adjacent pairs of monomers,
corresponding to a straight chain, and $\kappa_{b}$ is the spring constant
associated with bending.

The self-propulsion is induced by the active force $\vec{F}^{a}_{i}$,

$\vec{F}^{a}_{i}=f_{a}\hat{t}_{i},$ (4)

where $f_{a}$ fixes the absolute value of the self-propulsion force and
$\hat{t}_{i}$, the unit vector tangent to the chain at the position of $i$-th
monomer, defines its direction. For a chain consisting of discrete monomers,
this vector can be approximated as:

$\hat{t}_{i}=\frac{\alpha_{c}}{2}\left(\frac{\vec{r}_{i+1,i}}{|\vec{r}_{i+1,i}|}+\frac{\vec{r}_{i,i-1}}{|\vec{r}_{i,i-1}|}\right).$
(5)

Here $\alpha_{c}\in\\{-1,+1\\}$ specifies the state of the polarity (head-
tail) for each chain $c\in\\{1,\cdots,N_{c}\\}$ where $N_{c}$ is the number of
chains in the system. This variable will be important for the modelling of
reversal events, as we will discuss below.

Finally the presence of the repulsive force $\vec{F}^{ex}_{i}$ prevents
overlaps of the monomers. Such volume exclusion is implemented using the Week-
Chandler-Anderson potential:

$U_{ext}(r)=\begin{cases}4\epsilon\left[\left(\frac{\sigma}{r}\right)^{12}+\left(\frac{\sigma}{r}\right)^{6}\right]+\epsilon,&r<2^{1/6}\sigma\\\
0,&\text{otherwise}\end{cases}$ (6)

where $\sigma$ is the nominal interaction diameter and $\epsilon$ is the
energy scale of the interaction. In addition, adjacent monomers in the
filament overlapped, resulting in a smooth filament and $r_{0}=\sigma/2$

###  II.2 Direction reversals

So far, our model agrees with previous models for self-propelled filaments
Duman  _et al._ (2018); Prathyusha  _et al._ (2018); Isele-Holder  _et al._
(2015). In addition, we include spontaneous reversals of the direction of
self-propulsion, as they are often seen in the surface motion of bacteria
Taktikos  _et al._ (2013); Kurzthaler  _et al._ (2021) including filamentous
species such as gliding filamentous Cyanobacteria Häder and Burkart (1982);
Häder (1987); Tamulonis  _et al._ (2011b). Typically, the rates for such
reversals are modulated by various tactic behaviors such as chemotaxis,
phototaxis, etc. Here we consider the baseline case of spontaneous reversals
in a homogeneous environment.

We model random reversal events, where the gliding direction of a chain
changes instantaneously by $180^{\circ}$, as generated from a Poisson process
with a reversal rate $\lambda_{r}$ (see Fig. 1b). Mathematically, this means
that the polarity (head-tail) state $\alpha_{c}$ of a chain $c$ follows a
stochastic process

$\alpha_{c}=1\quad\xrightleftharpoons[\lambda_{r}]{\lambda_{r}}\quad\alpha_{c}=-1.$
(7)

In such a Poisson process, the distribution of waiting times $\tau$ elapsed
between two consecutive reversals is an exponential distribution of the form

$P_{\rm w}(\tau)=\lambda_{r}e^{-\lambda_{r}\tau}.$ (8)

Hence to implement the reversals for a single chain, we draw a random waiting
time $\tau_{0}$ from the exponential distribution described in Eq. 8 at time
$t_{0}$ . Between $t_{0}$ and $t_{1}=t_{0}+\tau_{0}$, the chain evolves
following Eq. 1. At $t_{1}$ the gliding direction of the chain is reversed and
another waiting time $\tau_{1}$ is drawn from the same waiting time
distribution. This process is done for each chain independently during the
simulation by drawing subsequent time intervals $\tau_{2},\tau_{3},...$, such
that reversals of any chain in the system is uncorrelated with the reversal of
any other chains.

Given a reversal rate $\lambda_{r}$ and a free propulsion speed of
$f_{a}/\zeta$, the free propagation distance between reversals on average is
$L_{r}=f_{a}/(\zeta\,\lambda_{r})$.

###  II.3 Simulations and parameters

All simulations presented here were run on GPUs using the simulation package
HOOMD-blue Anderson  _et al._ (2020) with custom extensions for self-
propulsion and for reversals, which were compiled together with the whole
software. The simulation integrates Eq. 1 over $t=10^{8}$ time steps.

In these simulations, lengths are measured in units of the monomer radius
$\sigma$ and the energy unit is $k_{B}T$. We choose the time unit to be the
self-diffusion time for a single monomer
$\tau_{D}=\frac{\sigma^{2}\zeta}{k_{B}T}$ where $\zeta$, which determines the
damping, is set to $\zeta=15$. Our system consists of $N_{f}=1666$ filaments
each having $N_{m}=59$ monomers.

We set the spring constant of the stretching spring to $\kappa_{s}=10^{4}$,
i.e. to a relatively high value to make sure that chains do not stretch too
much along their axis. Hence the distance between two successive monomers does
not fluctuate strongly and is approximately constrained to the fixed value
$r_{0}$. Furthermore, to obtain a smoother filament, we also set the value
$r_{0}=\frac{\sigma}{2}$. To investigate a large parameter space, we varied
$\kappa_{b}$ in the range $\\{1,1800\\}$ and $f_{a}$ between $\\{1,100\\}$.
The packing fraction, $\phi=N_{f}N_{m}r_{0}\sigma/L^{2}$, is set to $0.1$ in
all simulations, where $L$ denotes the box size and $L=167\sigma$

The aspect ratio, a dimensionless number representing the ratio between the
contour length and the diameter of the individual monomers
$a=\frac{L_{f}}{\sigma}$, is set to almost $30$ and $L_{f}=(N_{m}+1)r_{0}$ is
the contour length of the filaments. The reversal rates $\lambda_{r}$ are
varied between $\\{0-0.1\\}$ and are same for all the filaments.

##  III Results

To address the effect of direction reversals on the collective dynamics of
self-propelled filament, we performed systematic simulations varying three key
parameters, the bending stiffness $\kappa_{b}$, the strength of self-
propulsion $f_{a}$, and the reversal rate $\lambda_{r}$. We will show typical
snapshots of these simulations below. We start with the case without direction
reversals ($\lambda_{r}=0$), which has been studied before Prathyusha  _et
al._ (2018); Duman  _et al._ (2018) and which serves us as a reference
scenario here. In that case, it is known that a variety of non-equilibrium
patterns can be formed, for which we will present a detailed diagram of states
below.

Later on, we add the reversal mechanism to the filaments, inspired by the
reversal dynamics (due to phototaxis) observed in filamentous Cyanobacteria to
investigate the response of the system to an abrupt and random reversal in
gliding direction. We report a dramatic change in the individual as well as
collective dynamics in the state diagram as we crank up the reversal rate,
eventually destroying the spiral state completely and leading to destruction
of clusters and collective motion.

###  III.1 Non-reversing active filaments

Figure 2: Spiral ratio as a function of bending stiffness for different values
of self-propulsion force. (a) Sketch of different configuration of a single
filament characterised by the indicated spiral number indicated by $s$. (b)
Quantification of spiral formation by the spiral ratio $\tilde{s}$: An
increase in bending stiffness decreases the probability of spiral formation at
a constant self-propulsion force. Different values for self-propulsion are
shown in different colors. With increasing self-propulsion force at low
bending stiffness, spiral formation is promoted. For high bending stiffness,
spiral formation does not take place regardless of the self-propulsion force.
The aspect ratio of the filaments is $a=30$ and the packing fraction is
$\phi=0.1$.

####  III.1.1 Spiral formation

In the absence of direction reversals, the collective dynamics of the chains
is dominated by the interplay between self-propulsion, bending stiffness and
confinements due to the excluded volume effect. A key observation at low
density of filaments is that flexible filaments may form spirals while stiffer
filaments tend to form clusters Prathyusha  _et al._ (2018); Duman  _et al._
(2018).

Spirals form due to the self-interaction of an isolated active filament with
high self-propulsion and low bending stiffness. Spiral formation stands in
contrast to the relatively straight equilibrium structure of a semi-flexible
filament and depends on the self-propulsion. Spiral formation is initiated
when the head of the filament collides with a subsequent part of its own body,
such that the excluded volume interaction forces the filament to wind into
itself. If the self-propulsion is strong enough, filaments get trapped in
their own body, forming stable spirals. To characterise the dynamical state
with spiral formation we define a spiral number Isele-Holder  _et al._ (2015)
for each filament in the system which quantifies how many times a filament is
wound up around itself and is defined as:

$s=\left|\frac{\sum_{i=2}^{N-2}\Delta\theta_{i}}{2\pi}\right|$ (9)

where $\theta_{i}$ is the angle formed by the beads $i-1,i,i+1$. $|..|$
denotes the absolute value of the spiral number since they can be twisted
either in the clockwise or counterclockwise direction. $s=0$ indicates an
almost straight configuration (see Fig. 2 (a)) and $s=1$ a ring shape (see
Fig. 2 (a)). We consider a filament to be in a spiral state if $s\geq 1.2$
(see Fig. 2(a)).

To distinguish global patterns of filaments, we introduce the spiral ratio as
the ratio of the number of filaments in the spiral state (i.e., with $s\geq
1.2$), $N_{s}$, to that in the straight configuration,$N_{sf}$:

$\tilde{s}=\frac{N_{s}}{N_{sf}}.$ (10)

The spiral ratio is plotted in Fig.2(b) as a function of the bending stiffness
and the self-propulsion force. The figure shows a pronounced decrease of the
spiral ratio with increasing bending stiffness, which reflects the fact that
increasing bending stiffness ($\kappa_{b}$) hinders the self-interaction
needed for spiral formation. Fig.2(b) also shows that, for fixed bending
rigidity, spiral formation is promoted by an increase in the self-propulsion
force.

Figure 3:  Cluster ratio as a function of bending stiffness for different
values of self-propulsion force. (a) A binary collision of self-propelled
filaments that results in parallel alignment and coherent motion that
initiates cluster formation. (b) A binary collision of self-propelled
filaments that results in anti-parallel alignment and transient coherent
motion. The red monomer shows the head of the filament in both cases. (c)
Increasing the bending stiffness promotes cluster formation in the system.
Different values for the self-propulsion force are shown in different colors.
The aspect ratio of the filaments is $a=30$ and the packing fraction
$\phi=0.1$.

####  III.1.2 Cluster formation

In the limit of high bending stiffness, filaments behave like rod-shaped
agents. In this regime, filaments move in the direction of their long axis
and, upon collisions, they rotate to align due to their steric interaction. At
high bending stiffness, collisions between filaments are dominant over self-
interactions as the bending energy required for the self-interaction is rarely
reached. Dependent on the angle of incidence of the collision event, the two
filaments align either parallel or anti-parallel (Fig. 3(a) and (b),
respectively). In the first case, they continue their motion together, in the
second case, they stay in contact transiently Marchetti  _et al._ (2013);
Großmann  _et al._ (2016). Clusters of filaments form when additional
filaments collide with already aligned filaments, resulting in groups of
filaments that move coherently. The size and life-time of these clusters
depends on bending rigidity and activity, as well as the density of the
filaments in the system Wensink and Löwen (2012); Wensink  _et al._ (2012);
Marchetti  _et al._ (2013).

To characterise the clustering behaviour we identify clusters using the
following criteria: two monomers are considered to be part of the same cluster
if their centers are distanced less than $1.2\sigma$. We note that this
criterium automatically includes all monomers of one filament in the same
cluster. In addition, we only considered clusters with at least $N_{f}=10$
filaments to avoid counting micro clusters. Clusters are identified with the
data analysis framework Freud Ramasubramani  _et al._ (2020). Finally, in
analogy to the spiral ratio, we define a cluster ratio as the ratio of number
of filaments in clusters, $N_{c}$ to the number of free filaments, $N_{f}$.

$\tilde{c}=\frac{N_{c}}{N_{f}}$ (11)

Figure 4: Average cluster size as a function of bending stiffness. The average
cluster size is shown in different colors for different self-propulsion
forces.

Fig. 3 shows the cluster ratio as a function of the bending stiffness for
different values of the active force. The cluster ratio is seen to increase
monotonically with increasing bending stiffness ($\kappa_{b}$) and also with
increasing self-propulsion force. One can also see that high values of the
spiral number are associated with small clusters and vice versa: the regions
with higher values of spiral ratio in Fig. 2 correspond to the region of low
cluster ratio in Fig. 3. When the bending stiffness is low, spirals are formed
due to the self-interaction, when the bending stiffness increases, the
deformation of the filaments becomes unfavorable, which is why they usually
have a straight, rod-like conformation that promotes alignment and, thus,
clustering.

We also define the average cluster size, $CS_{m}$, which is the average total
number of filaments belonging to clusters and this seems to increase with
increasing bending stiffness indicating (see Fig. 4) the fact that bending
stiffness which suppress spiral formation, promotes clustering.

Since both spiral ratio and cluster ratio show opposing trends as function of
the bending rigidity and of the self-propulsion force, we wondered whether the
results could be written as functions of a dimensionless parameter
combination. A natural candidate is the flexure number, the ratio of activity
and bending rigidity Sekimoto  _et al._ (1995); Winkler  _et al._ (2017);
Duman  _et al._ (2018); Prathyusha  _et al._ (2018) can be written as:

$\Im=\frac{f_{a}L^{3}}{\kappa_{b}}$ (12)

Figure 5 shows the spiral and cluster ratio as functions of the flexure
number. Indeed the data from simulations with different bending stiffness and
self.propulsion forces are seen to collapse onto one curve. Only for the
largest flexure numbers, we see systematic deviations for the spiral ratio.
For smaller values of the flexure number, the spiral ratio increase as a power
law $\tilde{S}\sim\Im^{\delta}$ with $\delta\simeq 0.7$. Likewise, the
decrease of the cluster ratio also follows a power law
$\tilde{C}\sim\Im^{\delta^{\prime}}$ with $\delta^{\prime}\simeq-0.5$.

Figure 5: (a) Spiral ratio $\tilde{S}$ and (b) cluster ratio $\tilde{C}$ as
functions of the flexure number $\Im$: Data collapse for different self-
propulsion forces and bending rigidities when these are plotted as functions
of a dimensionless combination, the flexure number. The color of the points
indicates the value of the self-propulsion force. The red lines indicate power
laws fitted to the curves, with exponents $\delta$ and $\delta^{\prime}$.
Figure 6: Non equilibrium state diagram of non-reversing self-propelled
filaments. The state diagram is drawn as a function of self-propulsion force
$f_{a}$ and the bending stiffness $\kappa_{b}$ in the absence of reversals.
The different states are indicated by different symbols and the color of each
symbol is matched to the corresponding designation of the state. For details
on the characterisation and identification of the states, see text.

####  III.1.3 State diagram

We performed systematic simulations varying the self-propulsion force $f_{a}$
and the bending stiffness $\kappa_{b}$ to explore the variety of patterns in
the steady state of the system.

To distinguish different states of the system, we assign a state to all
filaments in a snapshot via the following criteria: A filament is considered
as a spiral if its spiral number is $<1.2$. The filament is taken to be in the
cluster category if it is not a spiral and part of a cluster of $\geq 10$
filaments. Since this criterion does not capture the diversity of cluster
patterns, we further subdivide the cluster state by the size of clusters and
distinguish a small-cluster category for filaments in clusters with size $\geq
10$ and $<100$ and a large-cluster category for filaments in clusters with
$\geq 100$ filaments. All other filaments are considered to be in the melt
state.

Using this classification and averaging over 200 snapshots per condition, we
classify the state of the whole system into the following categories: A large-
cluster state or a small-cluster state if $\geq 40\%$ of the filaments are in
large or small clusters, respectively. As the two criteria are not mutually
exclusive, coexistence of small and large clusters is identified if both are
satisfied simultaneously. Likewise, a spiral state is identified when more
than $20\%$ of the filaments form spirals (the threshold is chosen lower than
for the other states, as under the conditions simulated here, we hardly ever
see $40\%$ of the filaments being spirals). Finally, the system state is
classified as a melt state if $\geq 40\%$ of the filaments fall into this
category. Just like for small and large clusters, coexistence of other states
is possible and indeed seen frequently, e.g. between melt and small clusters
and melt and spirals. The diagram of states shown in Fig. 6 summarizes this
classification. Simulation snapshots corresponding to the different states are
shown in Fig. 7, where the color code of the filament shows the classification
at the filament level.

Figure 7: Snapshots of the simulation for different $f_{a}$ and $\kappa_{b}$.
(a) snapshot for $\kappa_{b}=10$, $f_{a}=10$ in ‘melt’ state (b)
$\kappa_{b}=5$, $f_{a}=100$ in spiral+melt state (c) Close-up view of spirals
in plot (a) (d)$\kappa_{b}=600$, $f_{a}=82$ (e) in small-cluster state
$\kappa_{b}=1000$, $f_{a}=24$ (f) in large-cluster state $\kappa_{b}=1600$,
$f_{a}=10$ in large + small cluster state.

For low bending stiffness and low self-propulsion force, the filaments are
mostly in the melt (or isotropic) phase and move in all directions, as can be
seen in Fig. 7 (a). We note that while the melt filaments are dominant, there
are also some small clusters. Increasing the self-propulsion force, while the
bending stiffness is kept at a small value, promotes spiral formation, as
shown in Fig. 7 (b) and (c). Formation of spirals is quite natural in this
regime because the filaments are flexible enough to bend onto themselves.
Their self-propulsion then results in spiral formation and, once formed, a
spiral is typically stable until another filament collides with it.

Collisions with other filament seems to be the dominant (if not only)
mechanism that breaks up spirals, since we have not included explicit
translational or rotational noise in our simulations. Under the conditions we
simulated, the spiral state typically coexists with the melt phase, as can be
seen in the snapshots in Fig. 7 (b) and (c). Consistent with this observation,
a pure spiral state, the ”gas of spirals” reported in earlier studies Duman
_et al._ (2018) is only observed at even lower densities than used here.

As shown above (Figure 2), the fraction of spirals decreases with increasing
bending rigidity. For sufficiently large bending stiffnesses, collective
behavior via the formation of clusters becomes dominant over individual
filament dynamics. In this regime, the self-interaction is less important
compared to inter-filament interaction, and when filaments collide, they can
bundle into small clusters with local alignment (nematic order). Over time,
the clusters grow as other filaments collide with them and join them. At
intermediate bending stiffness, there is a coexistence of dispersed individual
filaments and small clusters (Fig. 7 (d)). As we increase the bending
stiffness (Fig. 6d-f), the clusters become larger and the system enters a
large-cluster state, in particular for low self-propulsion force.

These observations show that the collective behavior of filaments is modulated
by their bending stiffness and their self-propulsion force. In agreement with
earlier work Prathyusha  _et al._ (2018); Duman  _et al._ (2018), we
distinguish several regimes, characterized by the formation of spirals and of
clusters of various sizes. Generally, we observed that, in the absence of
direction reversals, filaments with high self-propulsion force and low bending
stiffness form spirals, while filaments with high bending stiffness tend to
form clusters. This general picture will serve as a reference in the following
section, where we will include direction reversals and study how the variation
of the reversal rate affects the patterns formed by self-propelled filaments.

###  III.2 Active filaments with direction reversals

Figure 8: Snapshot of the simulation with reversal dynamics. We show snapshots
from the simulations keeping the bending stiffness, activity, density, and
aspect ratio same and changing the reversal rate. Applying reversal rate
changes the pattern formation drastically from its non-reversing counterpart.
a,b in the presence of a nonzero reversal rate, spiral formation has
disappeared. c,d Likewise, in the presence of reversals, the cluster sizes
become smaller than the non-reversing case. The coloring of the filaments
indicates their classification as spirals (cyan), filaments in small clusters
(red), large clusters (grey) and a melt (blue).

Next, we consider active filaments that exhibit reversals of their direction
of motion. Many microorganisms exhibit such a feature in their motility (both
swimming and surface-bound species), often as part of their chemo- or photo-
tactic strategy. Here we consider the case of spontaneous, non-biased
stochastic reversals. Our filaments are taken to change their direction of
motion abruptly, by $180^{\circ}$, according to a Poisson process with rate
$\lambda_{r}$. Varying $\lambda_{r}$, we explored the effect of reversals on
the structures and the collective motion of the active filaments.

Fig. 8 demonstrates the effect of a small reversal rate $\lambda_{r}=0.1$
(right column) compared to the case without reversals (left column) for two
opposite corners of the state diagram Fig. 6. These two parameters sets chosen
here correspond to the regime exhibiting spirals ($f_{a}=100$ and
$\kappa_{b}=1$, top) and to the regime dominated by small clusters ($f_{a}=24$
and $\kappa_{b}=1000$, bottom). In both cases, the visual appearance of
patterns clearly changes, indicating a substantial influence of the reversals.
In the spiral+melt regime (large $f_{a}$ and small $\kappa_{b}$), spirals
(light blue) are abundant (though in co-existence with straight filaments) in
absence of reversals, but efficiently suppressed by a small reversal rate
$\lambda_{r}=0.1$. In the large-cluster regime (small $f_{a}$ and large
$\kappa_{b}$), the change is less dramatic as collectively moving clusters are
seen both with and without reversals. However, the size of the clusters is
considerably decreased by the reversals.

Figure 9: Spiral and cluster ratio for different reversal rates $\lambda_{r}$
at an active force, $f_{a}=50$.

####  III.2.1 Unwinding and declustering

Figure 10: Diagram of state for reversing self-propelled filaments. The state
diagram is constructed as a function of the self-propulsion force $f_{a}$ and
the bending stiffness $\kappa_{b}$ and shown for four different values of the
reversal rate $\lambda_{r}$. States are marked by different symbols and
different colors as in Figure 6. When the reversal rate increases, on the one
hand the tendency of formation of large clusters decreases, on the other hand
the propensity of spiral formation in the system is progressively suppressed.
Figure 11: Unwinding of a spiral due to the reversal dynamics: Panel (a) shows
a filament in the spiral state in the center. In Panel (b) a reversal event
takes place. Panel (c): The spiral unwinds and the filament resumes the
straight configuration. The white monomer is the head of the filament, the
green is the body and the purple is the tail.

To quantify the weakening of spiral and cluster formation, we determined again
the spiral and cluster ratio, for a large range of reversal rates
$\lambda_{r}$, for an intermediate value of $f_{a}=50$. In Fig. 9, we plot the
two quantities as functions of the bending stiffness, comparing different
reversal rates. These results confirm the qualitative observation from the
snapshots: Reversals reduce both spiral and cluster abundance, for all values
of the bending stiffnesses. Higher reversal rates result in stronger
reduction.

Having noticed this stark effect of the reversal rate on pattern formation by
active filaments, we next investigate the effects of the reversal rate
$\lambda_{r}$ on the state diagram in the $\kappa_{b}$-$f_{a}$ space. The
results are shown in Fig. 10, for three non-zero values of $\lambda_{r}$, in
comparison to the case without reversals. Already for the smallest non-zero
reversal rate that we considered, $\lambda_{r}=0.001$, the spiral state as
well as the large-cluster state are strongly diminished. Increasing the
reversal rate further, also the small-cluster regime moves towards larger
bending stiffnesses and larger self-propulsion forces.

The suppression of spirals by reversals is rather straightforward. As
mentioned earlier, in the absence of reversals, spirals dissolve only through
collision with other filaments. Reversals add a second mechanism by the change
in the polarity (head-tail direction): When the reversal occurs, self-
propulsion is not directed inward, but rather outward, and thus the spiral
unwinds (see Fig. 11) unless another reversal occurs before unwinding was
complete. For small reversal rates $\lambda_{r}<f_{a}/\zeta L$, where a
filament propagates by more than its body length between reversals, this is
expected to happen. However, unwinding is faster than winding, because of the
additional release of bending energy, such that also repeated reversals lead
to a net unwinding.

Likewise, reversals add an additional mechanism for filaments to leave a
cluster. Clustered filaments move collectively in one direction, so that any
filament that reverses is likely to move out of the cluster. This shifts the
balance between filaments joining a cluster and filaments leaving a cluster,
resulting in smaller clusters. For very large reversal rates, clusters
disappear entirely and the system becomes isotropic again. In this limit, the
filaments do not show active directed motion any more, but rather move back
and forth in a fashion similar to passive diffusion. Thus a melt state like
for passive polymers is indeed expected, but the disorder results from an
entirely non-equilibrium mechanism.

##  IV Summary and Conclusion

In this study, we used particle-based simulations of self-propelled filaments
to study their pattern formation and, specifically, the impact of direction
reversal on these patterns. The simulations mimic bacterial gliding motility
on surfaces and their assembly into clusters and swarms. In contrast to the
well-studied spherical or rod-like self-propelled particles, we considered
long and flexible filaments.

First, we discussed the patterns that could form in the absence of direction
reversals, modulating the bending stiffness ($\kappa_{b}$) and self-propulsion
force ($f_{a}$) of the filaments at a relatively low filament density. The
results, in agreement with earlier work Prathyusha  _et al._ (2018); Duman
_et al._ (2018), reflect a competition between the self-interaction of
filaments, possible due to their flexibility, and the interaction between
filaments. At low bending stiffness and high self-propulsion force, the self-
interactions are dominant, leading to the formation of spirals. However, when
the bending stiffness increases, the self-interaction is reduced and steric
interactions between filaments become more important. In that case, filaments
form clusters through collisions and move collectively in these clusters.
Based on systematic simulations, we have determined a diagram of states in the
$\kappa_{b}$-$f_{a}$ space.

Then, to understand the impact of stochastic direction reversals, we added a
reversal rate to the filaments’ dynamics, mimicking the run-reverse movement
of bacteria, and examined the effects of this additional control parameter on
the formation of motility patterns. In general, reversals counteract both
clustering and spiral formation, thus resulting in more isotropic structures.
Both negative effects can be understood via dynamical pathways these
structures: First, reversals add a second mechanism to interrupt the spooling
of spirals and induce unspooling. Second, it also adds new mechanisms for
filaments to leave a cluster, as filaments typically leave a coherently moving
cluster upon reversal. The additional mechanism for leaving clusters results
in smaller clusters compared to the no-reversal reference case. Finally, high
reversal rates result in purely back-and-forth motion, similar to a passive
scenario.

Our results show that the direction reversals have a great impact on the
patterns of microorganisms in the casting media or other systems. The
direction reversals have unpleasant effects (like the destruction of spirals
and clustering) on the patterns formed by the bacteria, which could also
affect the dynamics of microorganisms in the system. Since the direction
reversal has a very unique effect on the self-assembly of active filaments, it
can also be extended to a higher density to study the effects of the reversal
rate on the pattern formed in that regime with/without obstacles in the
system.

## Acknowledgement

We are grateful to Rituparno Mandal for helpful discussions. This research was
conducted within the Max Planck School Matter to Life, supported by the German
Federal Ministry of Education and Research (BMBF) in collaboration with the
Max Planck Society.

## References

  * Marchetti  _et al._ (2013) M. C. Marchetti, J. F. Joanny, S. Ramaswamy, T. B. Liverpool, J. Prost, Madan Rao, and R. Aditi Simha, “Hydrodynamics of soft active matter,” Rev. Mod. Phys. 85, 1143–1189 (2013).
  * Bechinger  _et al._ (2016) Clemens Bechinger, Roberto Di Leonardo, Hartmut Löwen, Charles Reichhardt, Giorgio Volpe, and Giovanni Volpe, “Active particles in complex and crowded environments,” Rev. Mod. Phys. 88, 045006 (2016).
  * Hyman and Karsenti (1996) Anthony A Hyman and Eric Karsenti, “Morphogenetic properties of microtubules and mitotic spindle assembly,” Cell 84, 401–410 (1996).
  * Ndlec  _et al._ (1997) FJ Ndlec, Thomas Surrey, Anthony C Maggs, and Stanislas Leibler, “Self-organization of microtubules and motors,” Nature 389, 305–308 (1997).
  * Surrey  _et al._ (2001) Thomas Surrey, François Nédélec, Stanislas Leibler, and Eric Karsenti, “Physical properties determining self-organization of motors and microtubules,” Science 292, 1167–1171 (2001).
  * Mendelson  _et al._ (1999) Neil H. Mendelson, Adrienne Bourque, Kathryn Wilkening, Kevin R. Anderson, and Joseph C. Watkins, “Organized cell swimming motions in bacillus subtilis colonies: Patterns of short-lived whirls and jets,” American Society for Microbiology Journals 181, 600–609 (1999).
  * Parrish  _et al._ (2002) Julia K Parrish, Steven V Viscido, and Daniel Grunbaum, “Self-organized fish schools: an examination of emergent properties,” The biological bulletin 202, 296–305 (2002).
  * Dombrowski  _et al._ (2004) Christopher Dombrowski, Luis Cisneros, Sunita Chatkaew, Raymond E. Goldstein, and John O. Kessler, “Self-concentration and large-scale coherence in bacterial dynamics,” Phys. Rev. Lett. 93, 098103 (2004).
  * Salbreux and Jülicher (2017) Guillaume Salbreux and Frank Jülicher, “Mechanics of active surfaces,” Phys. Rev. E 96, 032404 (2017).
  * Hall-Stoodley  _et al._ (2004) Luanne Hall-Stoodley, J William Costerton, and Paul Stoodley, “Bacterial biofilms: from the natural environment to infectious diseases,” Nature reviews microbiology 2, 95–108 (2004).
  * Moore  _et al._ (2020) Jeffrey M. Moore, Tyler N. Thompson, Matthew A. Glaser, and Meredith D. Betterton, “Collective motion of driven semiflexible filaments tuned by soft repulsion and stiffness,” Soft Matter 16, 9436–9442 (2020).
  * Joshi  _et al._ (2019) Abhijeet Joshi, Elias Putzig, Aparna Baskaran, and Michael F Hagan, “The interplay between activity and filament flexibility determines the emergent properties of active nematics,” Soft Matter 15, 94–101 (2019).
  * Bär  _et al._ (2020) Markus Bär, Robert Großmann, Sebastian Heidenreich, and Fernando Peruani, “Self-propelled rods: Insights and perspectives for active matter,” Annual Review of Condensed Matter Physics 11, 441–466 (2020).
  * Prathyusha  _et al._ (2018) K. R. Prathyusha, Silke Henkes, and Rastko Sknepnek, “Dynamically generated patterns in dense suspensions of active filaments,” Phys. Rev. E 97, 022606 (2018).
  * Duman  _et al._ (2018) Özer Duman, Rolf E. Isele-Holder, Jens Elgeti, and Gerhard Gompper, “Collective dynamics of self-propelled semiflexible filaments,” Soft Matter 14, 4483–4494 (2018).
  * qing Shi and Chaté (2018) Xia qing Shi and Hugues Chaté, “Self-propelled rods: Linking alignment-dominated and repulsion-dominated active matter,” (2018), arXiv:1807.00294 [cond-mat.soft] .
  * Wang  _et al._ (2018) Yao-Kuan Wang, Chien-Jung Lo, and Wei-Chang Lo, “Formation of spiral coils among self-propelled chains,” Phys. Rev. E 98, 062613 (2018).
  * Großmann  _et al._ (2016) Robert Großmann, Fernando Peruani, and Markus Bär, “Mesoscale pattern formation of self-propelled rods with velocity reversal,” Phys. Rev. E 94, 050602 (2016).
  * Weitz  _et al._ (2015) Sebastian Weitz, Andreas Deutsch, and Fernando Peruani, “Self-propelled rods exhibit a phase-separated state characterized by the presence of active stresses and the ejection of polar clusters,” Phys. Rev. E 92, 012322 (2015).
  * Balagam and Igoshin (2015) Rajesh Balagam and Oleg A Igoshin, “Mechanism for collective cell alignment in myxococcus xanthus bacteria,” PLoS computational biology 11, e1004474 (2015).
  * Isele-Holder  _et al._ (2015) Rolf E. Isele-Holder, Jens Elgeti, and Gerhard Gompper, “Self-propelled worm-like filaments: spontaneous spiral formation structure and dynamics,” Soft Matter 11, 7181–7190 (2015).
  * Abkenar  _et al._ (2013) Masoud Abkenar, Kristian Marx, Thorsten Auth, and Gerhard Gompper, “Collective behavior of penetrable self-propelled rods in two dimensions,” Phys. Rev. E 88, 062314 (2013).
  * Wensink and Löwen (2012) H H Wensink and H Löwen, “Emergent states in dense systems of active rods: from swarming to turbulence,” Journal of Physics: Condensed Matter 24, 464130 (2012).
  * Wensink and Löwen (2008) H. H. Wensink and H. Löwen, “Aggregation of self-propelled colloidal rods near confining walls,” Phys. Rev. E 78, 031409 (2008).
  * Abbaspour and Klumpp (2021) Leila Abbaspour and Stefan Klumpp, “Enhanced diffusion of a tracer particle in a lattice model of a crowded active system,” Phys. Rev. E 103, 052601 (2021).
  * Börner  _et al._ (2002) Uwe Börner, Andreas Deutsch, Hans Reichenbach, and Markus Bär, “Rippling patterns in aggregates of myxobacteria arise from cell-cell collisions,” Phys. Rev. Lett. 89, 078101 (2002).
  * Sumino  _et al._ (2012) Yutaka Sumino, Ken H Nagai, Yuji Shitaka, Dan Tanaka, Kenichi Yoshikawa, Hugues Chaté, and Kazuhiro Oiwa, “Large-scale vortex lattice emerging from collectively moving microtubules,” Nature 483, 448–452 (2012).
  * Peruani  _et al._ (2012) Fernando Peruani, Jörn Starruß, Vladimir Jakovljevic, Lotte Søgaard-Andersen, Andreas Deutsch, and Markus Bär, “Collective motion and nonequilibrium cluster formation in colonies of gliding bacteria,” Phys. Rev. Lett. 108, 098102 (2012).
  * Rabani  _et al._ (2013) Amit Rabani, Gil Ariel, and Avraham Be’er, “Collective motion of spherical bacteria,” PloS one 8, e83760 (2013).
  * Dunkel  _et al._ (2013) Jörn Dunkel, Sebastian Heidenreich, Knut Drescher, Henricus H. Wensink, Markus Bär, and Raymond E. Goldstein, “Fluid dynamics of bacterial turbulence,” Phys. Rev. Lett. 110, 228102 (2013).
  * Be’er  _et al._ (2013) Avraham Be’er, Shinji K. Strain, Roberto A. Hernández, Eshel Ben-Jacob, and E.-L. Florin, “Periodic reversals in paenibacillus dendritiformis swarming,” Journal of Bacteriology 195, 2709–2717 (2013).
  * Nishiguchi  _et al._ (2017) Daiki Nishiguchi, Ken H. Nagai, Hugues Chaté, and Masaki Sano, “Long-range nematic order and anomalous fluctuations in suspensions of swimming filamentous bacteria,” Phys. Rev. E 95, 020601 (2017).
  * Huber  _et al._ (2018) L Huber, R Suzuki, T Krüger, E Frey, and AR Bausch, “Emergence of coexisting ordered states in active matter systems,” Science 361, 255–258 (2018).
  * Mirfakhrai  _et al._ (2007) Tissaphern Mirfakhrai, John D.W. Madden, and Ray H. Baughman, “Polymer artificial muscles,” Materials Today 10, 30–38 (2007).
  * Wensink  _et al._ (2012) Henricus H. Wensink, Jörn Dunkel, Sebastian Heidenreich, Knut Drescher, Raymond E. Goldstein, Hartmut Löwen, and Julia M. Yeomans, “Meso-scale turbulence in living fluids,” Proceedings of the National Academy of Sciences 109, 14308–14313 (2012).
  * Turner  _et al._ (2000) Linda Turner, William S. Ryu, and Howard C. Berg, “Real-time imaging of fluorescent flagellar filaments,” Journal of Bacteriology 182, 2793–2801 (2000).
  * Großmann  _et al._ (2016) R Großmann, F Peruani, and M Bär, “Diffusion properties of active particles with directional reversal,” New Journal of Physics 18, 043009 (2016).
  * Santra  _et al._ (2021) Ion Santra, Urna Basu, and Sanjib Sabhapandit, “Active brownian motion with directional reversals,” Phys. Rev. E 104, L012601 (2021).
  * Zhang  _et al._ (2018) Zhaoyang Zhang, Oleg A Igoshin, Christopher R Cotter, and Lawrence J Shimkets, “Agent-based modeling reveals possible mechanisms for observed aggregation cell behaviors,” Biophysical journal 115, 2499–2511 (2018).
  * Kurzthaler  _et al._ (2021) Christina Kurzthaler, Suvendu Mandal, Tapomoy Bhattacharjee, Hartmut Löwen, Sujit S Datta, and Howard A Stone, “A geometric criterion for the optimal spreading of active polymers in porous media,” Nature Communications 12, 1–10 (2021).
  * McBride (2001) Mark J McBride, “Bacterial gliding motility: multiple mechanisms for cell movement over surfaces,” Annual Reviews in Microbiology 55, 49–75 (2001).
  * Tamulonis  _et al._ (2011a) Carlos Tamulonis, Marten Postma, and Jaap Kaandorp, “Modeling filamentous cyanobacteria reveals the advantages of long and fast trichomes for optimizing light exposure,” PLoS One 6, e22084 (2011a).
  * Taktikos  _et al._ (2013) Johannes Taktikos, Holger Stark, and Vasily Zaburdaev, “How the motility pattern of bacteria affects their dispersal and chemotaxis,” PloS one 8, e81936 (2013).
  * Häder and Burkart (1982) Donat-P Häder and Uhland Burkart, “Enhanced model for photophobic responses of the blue-green alga, phormidium uncinatum,” Plant and cell physiology 23, 1391–1400 (1982).
  * Häder (1987) DP Häder, “Photosensory behavior in procaryotes,” Microbiological reviews 51, 1–21 (1987).
  * Tamulonis  _et al._ (2011b) Carlos Tamulonis, Marten Postma, and Jaap Kaandorp, “Modeling filamentous cyanobacteria reveals the advantages of long and fast trichomes for optimizing light exposure,” PLoS One 6, e22084 (2011b).
  * Anderson  _et al._ (2020) Joshua A. Anderson, Jens Glaser, and Sharon C. Glotzer, “Hoomd-blue: A python package for high-performance molecular dynamics and hard particle monte carlo simulations,” Computational Materials Science 173, 109363 (2020).
  * Ramasubramani  _et al._ (2020) Vyas Ramasubramani, Bradley D. Dice, Eric S. Harper, Matthew P. Spellings, Joshua A. Anderson, and Sharon C. Glotzer, “freud: A software suite for high throughput analysis of particle simulation data,” Computer Physics Communications 254, 107275 (2020).
  * Sekimoto  _et al._ (1995) Ken Sekimoto, Naoki Mori, Katsuhisa Tawada, and Yoko Y. Toyoshima, “Symmetry breaking instabilities of an in vitro biological system,” Phys. Rev. Lett. 75, 172–175 (1995).
  * Winkler  _et al._ (2017) Roland G. Winkler, Jens Elgeti, and Gerhard Gompper, “Active polymers — emergent conformational and dynamical properties: A brief review,” Journal of the Physical Society of Japan 86, 101014 (2017), https://doi.org/10.7566/JPSJ.86.101014 .


# Data Selection for Efficient Model Update in Federated Learning

Hongrui Shi<EMAIL_ADDRESS>University of Sheffield and  Valentin Radu
<EMAIL_ADDRESS>University of Sheffield

(2022)

###### Abstract.

The Federated Learning (FL) workflow of training a centralized model with
distributed data is growing in popularity. However, until recently, this was
the realm of contributing clients with similar computing capability. The fast
expanding IoT space and data being generated and processed at the edge are
encouraging more effort into expanding federated learning to include
heterogeneous systems. Previous approaches distribute light-weight models to
clients are rely on knowledge transfer to distil the characteristic of local
data in partitioned updates. However, their additional knowledge exchange
transmitted through the network degrades the communication efficiency of FL.
We propose to reduce the size of knowledge exchanged in these FL setups by
clustering and selecting only the most representative bits of information from
the clients. The partitioned global update adopted in our work splits the
global deep neural network into a lower part for generic feature extraction
and an upper part that is more sensitive to this selected client knowledge.
Our experiments show that only 1.6% of the initially exchanged data can
effectively transfer the characteristic of the client data to the global model
in our FL approach, using split networks. These preliminary results evolve our
understanding of federated learning by demonstrating efficient training using
strategically selected training samples.

federated learning, split learning, metadata, clustering

††journalyear: 2022††copyright: rightsretained††conference: 2nd European
Workshop on Machine Learning and Systems ; April 5–8, 2022; RENNES,
France††booktitle: 2nd European Workshop on Machine Learning and Systems
(EuroMLSys’22), April 5–8, 2022, RENNES, France††doi:
10.1145/3517207.3526980††isbn: 978-1-4503-9254-9/22/04††ccs: Computing
methodologies Model development and analysis††ccs: General and reference
Performance

##  1\. Introduction

Federated Learning (FL) is seen as the best solution for training machine
learning models on private data of many devices. The widespread adoption of
smartphones and their capability of generating essential training data have
contributed to the emergence of this privacy-preserving learning method
(McMahan et al., 2017). FL is used in many commercial applications, from
training the Google Keyboard for next work prediction (Kairouz et al., 2019)
to on-mobile image classification and more (Bonawitz et al., 2019).

In the classic FL setting, a global model is copied from a server to many
clients to be used for their local inference. A subset of clients are chosen
to contribute to the update of the global model by sending their local
training results on the local data to the server. The server aggregates all
the local training results to update the global model at the end of a FL
round. Subsequently, the updated global model is distributed to clients to
start a new round.

This training process works well when most clients have similar computing
capability. However, devices with a lot of local data or reduced computing
resources will fall behind the other clients and stall the training round. To
compensate for this system heterogeneity, recent research efforts have
proposed solutions based on model compression (Abdelmoniem and Canini, 2021)
and student models trained with attention transfer (Shi and Radu, 2021). Their
underlying idea is to reduce the computation load during the local updates by
training a light-weight model. In (Shi and Radu, 2021) we previously showed
the effectiveness of using attention transfer and metadata in the form of
activation maps from the Convolution Neural Network (CNN) to perform the model
update in a heterogeneous FL environment. However, it fails in a critical
point – if a client harbors a large number of local samples, the amount of
activation maps that needs to be sent to the server is critically large for
network transfer. Consequently, this adds a significant overhead to the
communication of our FL setting.

To this end, we aim to reduce the size of activation maps needed for updating
the global model, without deteriorating its generalization capability. Our
proposed solution is to identify the most informative samples for the
knowledge transfer. We do this by clustering and then select a smaller set of
activation maps to be used for the final stage of the global model update at
each round.

We alter the federated learning setting by introducing a split CNN paradigm.
The global CNN model is split into two parts, a lower part with layers
extracting generic features, and an upper part that is more sensitive to the
activation maps. For the lower part we keep a similar training process to the
federated average method, aggregating local updates into the lower part of the
global model. On the other hand, the upper part of the CNN model is trained
using a reduced set of activation maps created by the lower part of the
distributed global model on the local data. The adoption of activation maps
for updating the upper part of the global model is build on the solution
conceived in our previous work (Shi and Radu, 2021) introduced for the
challenge of heterogeneous devices.

Our split training paradigm preserves the privacy policy of FL by sending a
small fraction of activation maps to the server, and not the actual raw local
data. This approach has been previously explored (Osia et al., 2020) from a
privacy perspective, and proving the fallibility of raw data reconstruction
from feature maps is outside the scope of this paper.

Essentially, we make the following contributions:

  * •

We propose a new FL setting that splits the global neural network model into
two parts, a lower part for generic feature extraction trained with the
federated average method, and an upper part updated through activation maps.
We explore different levels for splitting the model, driven by empirical
observations.

  * •

To improve the communication efficiency, we introduce a clustering approach to
reduce the size of activation maps used for global update, without greatly
deteriorating the performance of our global model. This work advances FL in
heterogeneous systems, such as our previous system (Shi and Radu, 2021),
because the amount of computation performed by less powerful devices can be
significantly reduced using less local data.

  * •

By selecting representative samples from the clusters, our evaluation
indicates that as little as 1.6% of activation maps can update the global
model effectively with a negligible drop in test accuracy on the Cifar-10
image classification (Krizhevsky and Hinton, 2009). However, There is still
scope for improving these results and we indicate the route for this in future
work presented in Section 5.

##  2\. Related Work

#### Federated Learning (FL)

FL aims to train a global model in massively distributed networks (McMahan et
al., 2017) at scale (Bonawitz et al., 2019), over heterogeneous data of many
sources (Li et al., 2018), and with different training approaches (Shi and
Radu, 2021; Smith et al., 2017; He et al., 2020). For the system heterogeneity
challenge of training using devices of various computing capability, some
previous solutions abandon the single client model paradigm and replace this
with compressed client models to run efficiently on the client by matching the
model size with device computing capability (Shi and Radu, 2021; Abdelmoniem
and Canini, 2021).

#### System Heterogeneity.

The differences in hardware characteristics (processor, frequency, memory,
etc.) across clients and the amount of local data for training produce large
variations in the number of local updates performed by each client (Wang et
al., 2020). In FL, a deadline is imposed by the server on the time in order to
allow the clients to finish their local updates. Those clients who manage only
a few local updates or none at all in the given time are called stragglers and
often their work is discarded. FedNova (Wang et al., 2020) aggregates even the
updates from straggler by factoring in the number of local updates performed
by each client for weighting their contribution to the update of the global
model.

The other problem with system heterogeneity is that it causes objective
inconsistency. The federated optimisation convergence is built on the
assumption that clients perform a similar amount of updates from their local
data (Wang and Joshi, 2018; Stich, 2018). The best approach for assuring
convergence is to allow all clients to finish their round. But waiting for the
slow clients can significantly increase the training time (Wang et al., 2020).

Some most recent works allow less powerful clients to finish their local
updates in time by making the client models adaptive to local hardware.
(Abdelmoniem and Canini, 2021) uses model compression technique to reduce the
client models and (Shi and Radu, 2021) allows the client models to have
different sizes using attention transfer and metadata training to aggregate
local knowledge globally.

#### Non-IID data.

FL is expected to learn from non-IID data across devices that generate vastly
different distribution and volume of data. FedProx (Li et al., 2018), VRLSGD
(Liang et al., 2019) and SCAFFOLD (Karimireddy et al., 2020) have been
designed to handle non-IID local data. But these methods either result in
slower convergence or require additional communication and memory.

Our previous work (Shi and Radu, 2021) shows that the metadata created by the
client models can update the global model effectively in an extreme non-IID
settings for local data. However, the metadata sent to the server will
introduce additional overhead to the communication.

#### Training paradigms.

Different training paradigms have been explored for FL to address system and
data heterogeneity. Formulating the training as a multi-task learning (Smith
et al., 2017) results in larger global models that can accumulate the multiple
perspectives of client models trained on non-IID data as virtual tasks.
Knowledge transfer is used by Liang et al. (Liang et al., 2020a) to train
split networks. Knowledge transfer is performed at the network separation
point, running the first part of the model on the client and the final part on
the server. Unlike (Liang et al., 2020a), we see benefit in having the entire
global model on the server. This full model can be shared and used by clients
that have no local data to train a portion for their model on.

To facilitate the knowledge transfer between the server and clients in FL,
(Shi and Radu, 2021) proposes to use attention transfer to train smaller
models for each client from a large global model.

#### Clustering

In the field of knowledge transfer, knowledge selection is proposed for
seeking to select the most relevant knowledge for distillation. In (Li et al.,
2021), an entropy-based threshold is introduced to decide whether a piece of
knowledge is confident enough to be selected in a mutual knowledge
distillation paradigm.

Besides knowledge selection, another line of work is sample selection, where
clustering techniques are used. Clustering is commonly used for data analysis
as an unsupervised machine learning technique to group data points on the
basis of their properties (Seif, 2018). Data points grouped in the same
cluster are similar, whereas data points from different clusters have
discriminating patterns.

A popular clustering algorithm is K-means clustering. It clusters data points
by minimizing the within-cluster squared Euclidean distances with Expectation-
Maximum (EM) algorithm. In practice, K-means enjoys fast convergence by having
a linear complexity. On the other hand, K-means suffers from two major
problems. First, the number of clusters must be chosen manually, which is less
intuitive in many cases. Secondly, the initialization of K-means may lead to
the algorithm converging on different local minimums, making the results
inconsistent.

However, the computation efficiency of K-means makes it a good choice to
cluster the representations in resource-constrained systems.

#### Meta Learning and Few-shot Learning

Meta Learning or Learning to Learn (LTL) (Hochreiter et al., 2001) has emerged
as a popular technique for model adaption. It relies on meta-knowledge
extracted as generic information across tasks to generalise for a new task.
Recently, Model Agnostic Meta Learning (MAML) (Finn et al., 2017) has stemmed
from Meta Learning to learn a global model that can be used as an
initialization for further learning of a good model adaptive to a new task by
only a few local gradient steps (Kairouz et al., 2019). Both (Khodak et al.,
2019) and (Jiang et al., 2019) explore the relationship between MAML and FL in
the purpose to address the heterogeneity problem, particularly the non-IID
data. The global update in FL is compared to meta-training an initial model in
MAML. The local updates performed by each client can be considered as the
meta-testing to adapt the initial model to a individual local task.

To adapt the meta-learned initial model to a new task, Meta-Learning is
typically followed by Few-Shot learning (Fei-Fei et al., 2006), which aims to
use as less task-specify samples as possible. We see Few-Shot learning as
another potential solution to update the global model with metadata of reduced
size. This approach has not yet explored sufficiently in previous work as far
as we know, so we are encouraged to apply this technique to our future work.

Table 1. The list of notations used in this work. Notations | Description
---|---
$t$ | The $t$th global training round
$M$ | Total number of clients
$D_{M_{k}}$ | Metadata from client $k$
$D_{M}$ | The union of metadata from all clients
$A^{[j]}_{k}$ | Activation maps from client $k$ at level $j$
$M_{G}$ | The global model
$M_{C_{k}}$ | The $k$th client model
$M_{COM}$ | The composed model
$\bm{W}_{G}$ | Weights of the global model
$\bm{W}_{G}^{u}$ | Weights of upper part of the global model
$\bm{W}_{G}^{l}$ | Weights of lower part of the global model
$\bm{W}_{C_{k}}$ | Weights of the client model $k$
$\bm{W}_{S}^{u}$ | Weights of upper part of the updated global model
Figure 1. Block representation of the training performed between the server
and clients.The model is split into two parts for efficient distributed
training. The lower part of the network is trained using a federate learning
approach. The upper part of the network is trained using activation maps
(metadata) generated by clients using the local samples. A small fraction of
activation maps can be identified as most relevant for training. We do this
using clustering.

Initialization Initialize $M_{G}$ with weights $W_{G}(0)$;

for  _each round $t=1,2,...$_ do

On the client side:

for  _each client $k$_ do

Client model $M_{C_{k}}$ loads weights $W_{G}(t-1)$;

$D_{M_{k}}(t)\leftarrow$ Extract&Select $\left(D_{k},W_{G}^{l}(t-1)\right)$;

$W_{C_{k}}(t)\leftarrow$ LocalUpdate $\left(D_{k},W_{G}(t-1)\right)$;

end for

On the server side:

Aggregate all $D_{M_{k}}(t)$ into $D_{M}(t)$;

$W_{S}^{u}(t)\leftarrow$ MetaTraining $\left(D_{M}(t),W_{G}^{u}(0)\right)$;

$M_{COM}(t)\leftarrow$ ModelCompose
$\left(W_{G}^{l}(t-1),W_{S}^{u}(t)\right)$;

Test $M_{COM}(t)$ on test dataset;

$W_{G}(t)\leftarrow$ WeightAverage $\left(W_{C_{k}}(t),k=1,\cdots,m\right)$;

end for

return $W_{COM}$

Algorithm 1 Split Training with Metadata Selection

##  3\. Split Learning with Sample Selections

The global CNN model in our proposed method is trained on both the server side
and the client side. The lower part of the model is trained in the standard
federated average approach, with locally updated models being aggregated on
the server side for updating the lower part. The upper part of the model is
trained entirely on the server side, using the activation maps produced and
uploaded by the clients. The list of the notations used in this section are
presented in Table 1.

###  3.1. Activation Map Selection

At the start of each round $t$, the global model produced in the previous
round $t-1$ is distributed to the client side as $M_{C_{k}}$. There, the
activation maps (Equation (1)) are created by the local data $D_{k}$ on client
$k$, at a predefined level $j$ in the network that splits $M_{C_{k}}$ into a
lower part and a upper part.

(1) $A^{[j]}_{k}=f_{C_{k}}(\bm{W}^{[j]}_{C_{k}},\>x_{k})$

where $f_{C_{k}}$ is the function represented by the client model $k$,
$\bm{W}^{[j]}_{C_{k}}$ indicates the weights of the lower part of $M_{C_{k}}$
divided by level $j$, $(x_{k},y_{k})$ is the input-target pair of the local
dataset $D_{k}$. Figure 2 illustrates the activation maps created by a CNN
model at 3 different levels $j\in\\{G1,G2,G3\\}$ on the local data. Our level
selection is similar to the one proposed in (Zagoruyko and Komodakis, 2017).

After the activation maps are created, we select the activation maps that are
most representative alongside with their labels to form the metadata from
client $k$:
$D_{M_{k}}=Select\left(A^{[j]}_{k},y_{k}\right)$, sending them to the server
for updating the upper part of the global model. The global model will then
assimilate the local knowledge from the client models and local data encoded
in this metadata.

Figure 2. The collection of activation maps (metadata) from client model k
(global model refined in the previous round). The level in the model from
where to generate the activation maps is pre-determined and fix throughout the
training.

To identify the most representative activation maps we propose to use
Principal Component Analysis (PCA) for information compression and K-means for
clustering. PCA is first used to reduce the dimension of the activation maps.
Over the dimension-reduced activation maps, K-means is clusters these
representations belonging to the same class label.

Our insight of using PCA+K-means is that activation maps with similar
characteristics can be projected into the same cluster. Within each cluster,
we choose the sample that is closest to the cluster centre with respect to the
Euclidean distance as the most representative sample for its group. This
selected sample is assumed to include the most common characteristics of all
other samples in their own cluster. Finally, the metadata of client $k$,
$D_{M_{k}}$, is the union of the activation maps that are identified as the
most representative samples in the clustering stage.

The number of expected clusters is a predetermined hyperparameter. If we set a
small number of clusters, the size of metadata can be significantly reduced.
Most importantly, the selected activation maps are assumed to be capable of
updating the global model effectively without a significant loss of global
generalization because they contain the most common local information.

###  3.2. Local Update on the Client Side

The global model delivered to the $k$-th client is tuned on the local data
$D_{k}$ over a few local epochs. From these local epochs, the client optimiser
adjusts the weights $\bm{W}_{G}(t-1)$ locally such that it minimises the loss
function $\ell_{k}$ over the local data:

(2)
$\begin{split}&\underset{\bm{W}_{G}(t-1)}{\operatorname{arg}\,\operatorname{min}}\;F\left(\bm{W}_{G}(t-1),\;D_{k}\right)\\\
&=\frac{1}{N_{k}}\sum_{i=1}^{N_{k}}\ell_{k}\left(f_{G}(\bm{W}_{G}(t-1),\;x_{k}^{(i)}),\;y_{k}^{(i)}\right)\end{split}$

where $f_{G}$ is the function represented by the global model, $N_{k}$ is the
size of $D_{k}$, and $(x_{k}^{(i)},y_{k}^{(i)})$ is the $i$-th instance of
input-target pair of the local dataset. After the local updates,
$\bm{W}_{G}(t-1)$ is uniquely updated in the $k$-th client to
$\bm{W}_{C_{k}}(t)$.

###  3.3. Composing the Global Model on the Server Side

On the server side, the level for partitioning the lower part and upper part
of the global model is predetermined by $j$, using the same value for creating
the activation maps introduced in Section 3.1. Level $j$ is kept unchanged
across global model and clients so that $D_{m}$ is able to match the input
dimensions of the upper part of the global model. The server updates the upper
part and lower part separately by using metadata training and federated
average.

The upper part of the global model is updated to $\bm{W}_{S}^{u}(t)$ using the
metadata $D_{m}=\bigcup\limits_{k=1}^{m}D_{M_{k}}$ collected from the clients
at round $t$. To strengthen knowledge transfer for each round in the upper
layers, we keep the same initialization $\bm{W}_{G}^{u}(0)$ for the upper part
of the global model. Using a good number of metadata training epochs ensures
the global model effectively update its upper part. The optimization objective
of the metadata training is formulated as follows.

(3)
$\underset{\bm{W}_{S}^{u}(t)}{\operatorname{argmin}}F\left(\bm{W}_{S}^{u}(t),\;D_{m}\right)=\underset{\bm{W}_{S}^{u}(t)}{\operatorname{argmin}}\sum_{i=1}^{M}\ell_{G}\left(f_{G}(\bm{W}_{S}^{u}(t),\;A^{[j]}_{i}),\;y_{i}\right)$

where $f_{G}$ is the function represented by the global model, $M$ is the
total number of clients, $(A^{[j]}_{i},y_{i})$ is the metadata from the $i$-th
client in $D_{m}$.

To update the lower part of the global model, we adopt the conventional
federated average approach. In addition to the metadata training, the server
also gets an updated global model $M_{G}(t)$ by averaging the locally updated
model, $\bm{W}_{C_{k}}(t)$, from all clients as follows:

(4) $\bm{W}_{G}(t)=\frac{1}{m}\sum_{k=1}^{m}\bm{W}_{C_{k}}(t)$

Consequently, $M_{G}(t)$ contains the updated weights of the lower part.
Finally, the server composes a new global model $M_{COM}(t)$, with its upper
part from $\bm{W}_{S}^{u}(t)$ and lower part from $\bm{W}_{G}^{l}(t-1)$. It
should be noted that we use the lower part from the global model from the
previous round. This is because the metadata used to update the upper part to
$\bm{W}_{S}^{u}(t)$ is essentially created by the global model from the
previous round $M_{G}(t-1)$.

At the end of each global round, the composed global model, $M_{COM}(t)$, is
tested on the test dataset to evaluate its performance. The global model from
federated average, $M_{G}(t)$, is distributed to the clients for local updates
in the next round.

Our algorithm is summarized in Algorithm 1 and Figure 1 presents a block
representation of the algorithm.

##  4\. Evaluation

This section presents the setup for validating our proposed solution, results
and analysis of our experiments.

###  4.1. Experiment Setup

Our experiments are conducted on the CIFAR-10 dataset (Krizhevsky and Hinton,
2009) for the image classification task. CIFAR-10 has a total number of 60,000
images of size $32\times 32$ pixels, with 10 classes and 6,000 images for each
class. The training set and test set in CIFAR-10 contain 50,000 images and
10,000 images respectively. Our results indicate the accuracy on the test set
throughout the evaluate for the global model.

The Wide Residual Networks (WRN) (Zagoruyko and Komodakis, 2016) is used as
the global CNN model. The global model $M_{G}$ is built on a WRN with a depth
of 40 and width of 1 (WRN-40-1). The layers of WRN-40-1 are hierarchically
organized into 3 groups as discussed in Section 3.1. The dimension of
activation maps extracted at different level $j$ of the WRN-40-1 is shown in
Table 2.

Table 2. The dimension of the activation maps at different levels in the global CNN model (WRN-40-1). For an image input, the higher the level of the model, the lower resolutions of the activation map. Global Model | Level $j$ | Dimension of $A^{[j]}$
---|---|---
WRN-40-1 |  | $G1$
---
$G2$
$G3$
| $16\times 32\times 32$
---
$32\times 16\times 16$
$64\times 8\times 8$

We borrow the experiment settings from our previous work to approach the
heterogeneous system challenge (Shi and Radu, 2021). We set the number of
clients to 20 and assume that all the clients are able to complete their local
updates and metadata selection in time before finishing the round. To make a
fair comparison, the baselines also assumes a full client participation. To
simulate real-world applications, we exercise an extreme non-IID local data
setting used in (Liang et al., 2020b), where each client hosts only 2500
images randomly selected from two random classes of CIFAR-10.

###  4.2. Results and Analysis

We fix the level in the network from where we collect the activation maps.
This is first set to level $j=G1$, after the first group of convolutional
layers.

The first experiments assess the effectiveness of training the model with
selected metadata. We use the SGD optimizer with a learning rate of 0.1 and a
batch size of 50 for model update. L2 regularization with a weight of 0.0005
is applied to metadata training. The training epochs are set to 1 and 100
respectively for the local updates and metadata training. We train the global
model for 100 FL rounds. Regarding the hyperparameters of our proposed
clustering method introduced in Section 3.1, the number of PCA components is
set to 200, reducing the dimension of activation maps from $16\times 32\times
32$ to just 200 features. Dimension-reduced activation maps from the same
(image recognition) class are clustered into 20 groups by K-means. As such,
each CIFAR-10 class in $D_{k}$ will contribute 20 representative samples
towards the metadata $D_{M_{k}}$.

To evaluate the effect of using metadata selection, we use another baseline
produced by using the entire set of activation maps from the whole training
set on the composed model $D_{COM}$ (full metadata). This baseline collects
all the activation maps produced by the clients to form $D_{M}$. Table 3
compares our proposed metadata selection to the baseline of using the entire
set of metadata (activation maps). When the upper part of $M_{COM}$ is updated
on the entire activation maps without selection (50000 in total), $D_{COM}$
achieves a higher test accuracy of 70.03%. In contrast, the test accuracy of
$M_{COM}$ is just 48.47% due to $D_{M}$ being reduced to containing just 800
activation maps selected by our proposed clustering approach. A drop in
accuracy is understandable due to reduced training data. We aim to narrow this
performance gap by evaluating different levels in the network for extracting
activation maps.

Table 3. Comparison of the performance of $D_{COM}$ with metadata selection and without metadata selection (full activation maps to train on) on the CIFAR-10 image classification task. In metadata selection, each client selects 40 activation maps to form the metadata, uniformly distributed across classes. This is 1.6% of total activation maps available over the entire training set. The option without selection has no clusters. Composed Model |  | Metadata
---
Selection
| Cluster
---
Number
Test Acc.
WRN-40-1 |  | without
---
clustering
| ———
---
20
| 70.03%
---
48.47%

###  4.3. Metadata from Higher Levels

Table 4. Comparison among centralized training, Federated Average (FedAvg) and our FL approach with selected metadata. All use WRN-40-1 as the global model. The centralized version trains the model using the entire training data on a single machine, thus achieving the best performance. The FedAvg forms another strong baseline by assuming full client participation in each round. In our approach, we show the performance of the composed model trained on full activation maps and selected activation maps with on our clustering based selection (PCA+K-means). Our FL approach using split learning and metadata training outperforms the FedAvg baseline if all activation maps from either level $G1$ or $G2$ are used. With respect to using the selected metadata, we show that using selected metadata from level $G3$ produces a global model with a minimal performance drop. Missing values indicate not applicable settings.  Method |  | Metadata
---
Selection
| Test Acc.
---
| Global | $j=G1$ | $j=G2$ | $j=G3$
---|---|---|---
Centralized | ———– |  | 90.79% | ———– | ———– | ———–
---|---|---|---
FedAvg | ———– |  | 69.35% | ———– | ———– | ———–
---|---|---|---
| FL+metadata
---
FL+metadata
| without
---
clustering
|  | ———– | 70.03% | 73.47% | 68.55%
---|---|---|---
| ———– | 48.47% | 60.11% | 64.20%
---|---|---|---

The performance of the composed model $M_{COM}$ whose upper part is trained on
the selected activation maps from $G1$ of the client models degrades
significantly compared to the baseline. Our intuition for this behaviour is
that the selected metadata from a lower level is insufficient to train the
upper part of the WRN-40-1 model, which has more than half a million
parameters.

To justify this hypothesis, we further reduce the number of learnable
parameters in the upper part by predefining a higher $j$ level. Table 4 shows
our experiment for using metadata from all three levels. In addition to the
baseline without using metadata selection described in Section 4.2, we further
include a baseline where the global model is trained on the entire CIFAR-10
training set in a centralized manner and a FedAvg (McMahan et al., 2017)
baseline assuming full client participation.

The result depicts a narrow performance gap when a large $j$ is chosen. With a
fraction of activation maps (1.6% of all activation maps) from level $G3$ for
updating the global model, the performance of the model decreases by less than
5%. In contrast, using selected metadata from a lower level leads to a
significant performance deterioration. However, we notice that using a larger
$j$ makes our approach resemble the standard FedAvg because the lower part of
the global model, which is averaged in our approach, has a large size. This
observation comes also from the experimental result which show that our method
without metadata selection achieves 68.55%. This performance is very close to
the performance of the FedAvg baseline of 69.35%

On the other hand, if metadata from level $G1$ or $G2$ is fully used, our FL
approach outperforms the FedAvg baseline on the test set by over 4%. Compared
with averaging the entire global model, this result clearly demonstrates the
advantage of using metadata to update the upper part of the split global
model.

##  5\. Conclusions and Future Work

We propose a new split training approach for federated learning, with the aim
to reduce the size of metadata (activation maps) required from the client side
to train a global model. This approach splits the global CNN model into a
lower part that is trained using the federated learning standard approach, and
an upper part of the network that is trained on the server side using a
fraction of metadata selected from the client side. We select the metadata by
determining the most representative samples through clustering. Our
experiments show that with selecting just 1.6% of the activation maps, the
composed global model maintains a good performance, decreasing just 4.35% in
image classification accuracy on the CIFAR-10 test set. Our approach improves
the communication efficiency of our previous work to benefit federated
learning in heterogeneous systems due to reducing the local data needed for
updates.

There is still scope for improving our results. In future, we will explore two
main research directions:

  * •

replacing the distance-based clustering selection of representative activation
maps with stronger similarity metrics.

  * •

exploring more effective methods to train large models (holding many
parameters) with very few training samples, inspired from few-shot learning.

## References

  * (1)
  * Abdelmoniem and Canini (2021) Ahmed M Abdelmoniem and Marco Canini. 2021.  Towards Mitigating Device Heterogeneity in Federated Learning via Adaptive Model Quantization. In _Proceedings of the 1st Workshop on Machine Learning and Systems_. 96–103.
  * Bonawitz et al. (2019) Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečnỳ, Stefano Mazzocchi, H Brendan McMahan, et al. 2019\.  Towards federated learning at scale: System design.  _arXiv preprint arXiv:1902.01046_ (2019).
  * Fei-Fei et al. (2006) Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006.  One-shot learning of object categories.  _IEEE transactions on pattern analysis and machine intelligence_ 28, 4 (2006), 594–611.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017\.  Model-agnostic meta-learning for fast adaptation of deep networks. In _International Conference on Machine Learning_. PMLR, 1126–1135.
  * He et al. (2020) Chaoyang He, Murali Annavaram, and Salman Avestimehr. 2020\.  Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge.  _Advances in Neural Information Processing Systems_ 33 (2020).
  * Hochreiter et al. (2001) Sepp Hochreiter, A Steven Younger, and Peter R Conwell. 2001\.  Learning to learn using gradient descent. In _International Conference on Artificial Neural Networks_. Springer, 87–94.
  * Jiang et al. (2019) Yihan Jiang, Jakub Konečnỳ, Keith Rush, and Sreeram Kannan. 2019.  Improving federated learning personalization via model agnostic meta learning.  _arXiv preprint arXiv:1909.12488_ (2019).
  * Kairouz et al. (2019) Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2019\.  Advances and open problems in federated learning.  _arXiv preprint arXiv:1912.04977_ (2019).
  * Karimireddy et al. (2020) Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020.  SCAFFOLD: Stochastic controlled averaging for federated learning. In _International Conference on Machine Learning_. PMLR, 5132–5143.
  * Khodak et al. (2019) Mikhail Khodak, Maria-Florina Balcan, and Ameet Talwalkar. 2019.  Adaptive gradient-based meta-learning methods.  _arXiv preprint arXiv:1906.02717_ (2019).
  * Krizhevsky and Hinton (2009) Alex Krizhevsky and Geoffrey Hinton. 2009.  Learning multiple layers of features from tiny images.  (2009).
  * Li et al. (2018) Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2018\.  Federated optimization in heterogeneous networks.  _arXiv preprint arXiv:1812.06127_ (2018).
  * Li et al. (2021) Ziyun Li, Xinshao Wang, Haojin Yang, Di Hu, Neil M. Robertson, David A. Clifton, and Christoph Meinel. 2021.  Not All Knowledge Is Created Equal.  arXiv:2106.01489 [cs.LG]
  * Liang et al. (2020a) Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020a.  Think locally, act globally: Federated learning with local and global representations.  _arXiv preprint arXiv:2001.01523_ (2020).
  * Liang et al. (2020b) Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020b.  Think Locally, Act Globally: Federated Learning with Local and Global Representations.  (2020).
  * Liang et al. (2019) Xianfeng Liang, Shuheng Shen, Jingchang Liu, Zhen Pan, Enhong Chen, and Yifei Cheng. 2019\.  Variance reduced local SGD with lower communication complexity.  _arXiv preprint arXiv:1912.12844_ (2019).
  * McMahan et al. (2017) Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017\.  Communication-efficient learning of deep networks from decentralized data. In _AIStat_. PMLR.
  * Osia et al. (2020) Seyed Ali Osia, Ali Shahin Shamsabadi, Sina Sajadmanesh, Ali Taheri, Kleomenis Katevas, Hamid R Rabiee, Nicholas D Lane, and Hamed Haddadi. 2020.  A hybrid deep learning architecture for privacy-preserving mobile analytics.  _IEEE Internet of Things Journal_ 7, 5 (2020).
  * Seif (2018) George Seif. 2018\.  _The 5 Clustering Algorithms Data Scientists Need to Know — by George Seif — Towards Data Science_.  https://towardsdatascience.com/the-5-clustering-algorithms-data-scientists-need-to-know-a36d136ef68
  * Shi and Radu (2021) Hongrui Shi and Valentin Radu. 2021.  Towards Federated Learning with Attention Transfer to Mitigate System and Data Heterogeneity of Clients. In _Proceedings of the 4th International Workshop on Edge Systems, Analytics and Networking_ _(EdgeSys ’21)_. 61–66.
  * Smith et al. (2017) Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. 2017.  Federated multi-task learning.  _arXiv preprint arXiv:1705.10467_ (2017).
  * Stich (2018) Sebastian U Stich. 2018\.  Local SGD converges fast and communicates little.  _arXiv preprint arXiv:1805.09767_ (2018).
  * Wang and Joshi (2018) Jianyu Wang and Gauri Joshi. 2018.  Cooperative SGD: A unified framework for the design and analysis of communication-efficient SGD algorithms.  _arXiv preprint arXiv:1808.07576_ (2018).
  * Wang et al. (2020) Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. 2020.  Tackling the objective inconsistency problem in heterogeneous federated optimization.  _arXiv preprint arXiv:2007.07481_ (2020).
  * Zagoruyko and Komodakis (2016) Sergey Zagoruyko and Nikos Komodakis. 2016.  Wide residual networks.  _arXiv preprint arXiv:1605.07146_ (2016).
  * Zagoruyko and Komodakis (2017) Sergey Zagoruyko and Nikos Komodakis. 2017.  Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In _5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings_. International Conference on Learning Representations, ICLR.  arXiv:1612.03928 https://github.com/szagoruyko/attention-transfer.