# Combining Federated Learning and
Control: A Survey*

Jakob Weber1, Markus Gurtner1, Amadeus Lobe1, Adrian Trachte2, and Andreas
Kugi3 1 Jakob Weber, Markus Gurtner, and Amadeus Lobe are with the Center for
Vision, Automation & Control, AIT Austrian Institute of Technology GmbH,
Vienna, Austria<EMAIL_ADDRESS>Adrian Trachte is with the Robert Bosch
GmbH, Renningen, Germany3 Andreas Kugi is with the Automation and Control
Institute, TU Wien, Austria and the AIT Austrian Institute of Technology GmbH,
Vienna, Austria

###### Abstract

This survey provides an overview of combining Federated Learning (FL) and
control to enhance adaptability, scalability, generalization, and privacy in
(nonlinear) control applications. Traditional control methods rely on
controller design models, but real-world scenarios often require online model
retuning or learning. FL offers a distributed approach to model training,
enabling collaborative learning across distributed devices while preserving
data privacy. By keeping data localized, FL mitigates concerns regarding
privacy and security while reducing network bandwidth requirements for
communication. This survey summarizes the state-of-the-art concepts and ideas
of combining FL and control. The methodical benefits are further discussed,
culminating in a detailed overview of expected applications, from dynamical
system modeling over controller design, focusing on adaptive control, to
knowledge transfer in multi-agent decision-making systems.

###### Index Terms:

Federated Learning, Control Systems, Nonlinear Control

##  I Introduction

With the growing importance of data-driven models within control systems,
there is an increasing emphasis on integrating learning-based models directly
into the control loop. This integration enhances adaptability and allows for
broader generalizability across diverse and possibly unseen operational
scenarios. Nowadays, an increasing number of control system hardware options
include integrated connectivity solutions, exemplified by [1] or [2], thereby
creating opportunities, e.g., to integrate cloud-based solutions to enhance
system performance, resilience, and adaptability. While centralized approaches
leveraging connected Internet-of-Things (IoT) devices and cloud computing
infrastructure present viable options, bandwidth limitations and data privacy
challenges are still present when transmitting raw data, see, e.g., [3]
discussing privacy in the context of machine-learning and 6G communication,
and [4] for an in-depth discussion of privacy and security for distributed
machine-learning techniques. Furthermore, [5] offers an excellent overview of
communication-efficient and distributed learning. In this context, Federated
Learning (FL) offers a compelling solution.

Federated Learning enables collaborative model training across distributed
devices while preserving sensitive data, thereby addressing the challenges of
communication efficiency and privacy preservation. The core concept involves
computing model updates locally on individual devices, securely aggregating
these updates, and globally computing a combined model. This paradigm shift
from traditional centralized to decentralized training addresses data privacy
and security by keeping the raw data localized, thus ensuring minimal exposure
risk. Beyond regulatory considerations, see [6] for a discussion of FL under
the European Union Artificial Intelligence Act [7], FL offers a technical
advantage in reducing the required network bandwidth through a more efficient
information transfer.

Federated Learning has already gained substantial attention, particularly in
industries and domains characterized by abundant sensitive data with high data
privacy and communication efficiency demands, such as healthcare [8, 9, 10],
finance and economics [11, 12, 13, 14], manufacturing [15, 16, 17], and IoT
applications [18, 19, 20, 21, 22, 23]. The distributed nature of FL is
advantageous in scenarios featuring large datasets or devices spread across
diverse spatial locations, see e.g., [24] and [25].

This survey aims to offer a comprehensive overview of the potential of FL for
control problems, focusing on enhancing adaptability, scalability, resilience,
and privacy in (nonlinear) control applications. Our contribution is to
provide answers to the following research questions:

  1. Q1)

What is the current state-of-the-art at the intersection of FL and control?

  2. Q2)

What are the anticipated benefits of combining FL and control?

The paper is structured as follows: Section II introduces FL, presenting its
fundamental concept, prevalent algorithms, and various categorizations.
Section III is dedicated to research question Q1 and offers a detailed
overview of the existing literature with a focus on decentralized control and
learning in control, as well as on concepts at the intersection of FL and
control. In Section IV, we delve into the expected benefits of merging those
fields and discuss potential applications, which addresses research question
Q2. Finally, Section V gives some conclusions. A visualization of the paper’s
structure is given in Fig. 1.

Figure 1: The structure of this paper.

##  II Federated Learning

This section provides an overview of the fundamental principles of Federated
Learning (FL), laying the groundwork for addressing the research questions
posed earlier. We first introduce the primary goal of FL and a commonly used
optimization algorithm. We then categorize the various applications of FL to
highlight its versatility. The section concludes with a discussion of the key
challenges inherent to FL.

###  II-A Basic Concepts & Algorithms

Federated Learning emerged as a novel research area following the pioneering
work at Google, see [26] and [27]. This work introduced a distributed model
training approach particularly aimed at preserving privacy. The general idea
presented is that devices - mainly mobile phones - download the current model
from a central server, learn from the local, private data, and then transmit,
using encrypted communication, the model updates back to the central server.
The server immediately aggregates these updates to refine the shared global
model. Importantly, all training data remains on the device, and individual
updates are not stored in the cloud, see [28]. The present survey does not
claim to give a comprehensive description and discussion of all existing FL
algorithms, methods, and technologies in the context of FL. However, it
focuses on the connection between FL and control. For an in-depth exploration
of FL in general, the readers are referred to the literature, such as [24],
[29] and [30], and IEEE standards [31, 32].

Federated Learning encompasses a collection of directives and algorithms
tailored for distributed, privacy-preserving, and communication-efficient
learning. While this definition is informal, it can be precisely formulated as
a distributed optimization problem expressed by the following mathematical
representation

$\displaystyle\theta^{*}\leftarrow\arg\min_{\theta}\mathbb{E}_{i\sim\mathcal{P}}\left[\mathbb{E}_{x_{i}\sim\mathcal{D}_{i}}\left[f_{i}(x_{i},\theta)\right]\right].$
(1)

Here, the objective is to find the optimal value $\theta^{*}$, which minimizes
the expected value $\mathbb{E}_{i\sim\mathcal{P}}$, taken over clients $i$
sampled from the client distribution $\mathcal{P}$, of the expected value
$\mathbb{E}_{x_{i}\sim\mathcal{D}_{i}}$ of the local client cost functions
$f_{i}(x_{i},\theta)$, parameterized by $\theta$ and conditioned on the data
$x_{i}$ sampled from the local data-generating distribution $\mathcal{D}_{i}$.

Assuming standard regularity conditions, (1) is typically solved using
gradient descent techniques, wherein the global gradient is approximated as
the expected value of local gradients. This involves utilizing sample averages
over the local data-generating distributions $\mathcal{D}_{i}$ to estimate the
local gradients. Subsequently, the expected value of the global gradient is
computed through some form of weighted averaging of these local gradients.
Consequently, only gradient information and never any raw data $x_{i}$ are
exchanged, facilitating the privacy-preserving and communication-efficient
optimization of the global objective function.

A widely adopted FL algorithm, known as _Federated Averaging_ , introduced in
[27], performs the stochastic approximation of the global gradient through a
strategy of _partial participation_ of the clients. This involves
approximating the expected value over the client distribution $\mathcal{P}$
through a sample average over a number of clients $N$, coupled with executing
_local steps_ , encompassing multiple iterations of local gradient steps. The
standard Federated Averaging can be extended to provide flexibility in
choosing both the local and global optimization method, resulting in the
_Generalized Federated Averaging_ algorithm, also known as _FedOpt_. This
extension is detailed in [33] and presented in Algorithm 1. Examining
Algorithm 1, it becomes evident that three key degrees of freedom govern the
optimization of the FL objective in (1), namely

  * •

Local optimization (client-side),

  * •

Global optimization (server-side), and

  * •

Model aggregation.

Importantly, these components can be independently chosen, paving the way for
a diverse family of algorithms capable of solving a manifold of different
problems. [h]

1 Input: Initial model $\theta^{(0)}$, ClientOpt & ServerOpt with learning
rates $\eta$, $\eta_{s}$;

2 for  _$t=0,1,\ldots,T$_ do

3 Sample a subset $S_{t}$ of clients from $\mathcal{P}$;

4 for  _client $i\in S_{t}$ in parallel_ do

5 Initialize local model $\theta_{i}^{(t,0)}\leftarrow\theta^{(t)}$;

6 for  _$k=0,\ldots,\tau_{i}-1$_ do

7 Compute local gradient $g_{i}(\theta_{i}^{(t,k)})$;

8 Local update:
$\theta_{i}^{(t,k+1)}=\textrm{ClientOpt}(\theta_{i}^{(t,k)},g_{i}(\theta_{i}^{(t,k)}),\eta,t)$;

9

10 end for

11 Compute local changes
$\Delta^{(t)}_{i}=\theta_{i}^{(t,\tau_{i})}-\theta_{i}^{(t,0)}$;

12

13 end for

14 Aggregate local changes $\Delta^{(t)}=\sum_{i\in
S_{t}}p_{i}\Delta^{(t)}_{i}/\sum_{i\in S_{t}}p_{i}$;

15 Update global model
$\theta^{(t+1)}=\textrm{ServerOpt}(\theta^{(t)},\Delta^{(t)},\eta_{s},t)$;

16 end for

Algorithm 1 Generalized FedAvg (FedOpt) algorithm.

The optimization procedure of the _Generalized Federated Averaging_ algorithm
is depicted in detail in Fig. 2.

\begin{overpic}[width=433.62pt]{img/gen_fed_avg.pdf}
\put(3.6,0.6){$\theta^{(t)}$}
\put(9.0,3.25){\hbox{\pagecolor{white}\Romannum{1}}}
\put(2.75,5.0){\hbox{\pagecolor{white}\Romannum{2}}}
\put(36.0,10.0){\hbox{\pagecolor{white}$\Delta_{\mathrm{I}}^{(t)}$}}
\put(33.0,12.5){\hbox{\pagecolor{white}$\Delta_{\mathrm{II}}^{(t)}$}}
\put(61.0,11.0){\hbox{\pagecolor{white}$\Delta^{(t)}$}}
\put(91.0,13.0){$\theta^{(t+1)}$} \end{overpic} Figure 2: Sketch of the
optimization procedure of Generalized Federated Averaging for two clients.

Starting from the global model $\theta^{(t)}$, steps 6-9 entail local
optimization, where numerous local gradient steps are performed locally for
clients 1 and 2. Subsequently, in step 10, these steps are consolidated into
local model updates $\Delta_{\mathrm{I}}^{(t)}$ and
$\Delta_{\mathrm{II}}^{(t)}$ w.r.t. the initial model for each client. The
model aggregation in step 12 combines these local model updates to generate a
global model adjustment $\Delta^{(t)}$, where $p_{i}$ is introduced to weigh
the impact of each client. Finally, step 13 illustrates the global
optimization, wherein a single gradient step
$\theta^{(t+1)}=\theta^{(t)}+\eta_{s}\Delta^{(t)}$ with global learning rate
$\eta_{s}$ is executed. Typically, steps 4-13 are reiterated until convergence
of the global model is achieved. Apart from the proposed method for solving
(1), FL can also be formulated as a consensus optimization problem - tackled
using primal-dual methods for distributed convex-optimization, see [34] \- or
as the problem of privacy-sensitive fusion of multiple probability density
functions, see [35].

###  II-B Categorizations

The FL objective in (1) enables various applications, often categorized by
client heterogeneity, see [36], and learning tasks. An alternative FL
formulation, learning an implicit mixture of the global and pure local models,
is presented in [37]. A comprehensive survey and classification of FL is
provided in [38]. The most common distinction is between cross-device and
cross-silo FL. Cross-device FL embodies the original concept of
collaboratively learning a shared model across a large number of devices, as
detailed in [27] and [28]. Conversely, cross-silo FL, introduced in [24],
becomes particularly relevant for smaller sets of clients. Here, every client
engages in each round of learning and maintains a state that describes the
current model and its evolution. The primary characteristics of both
approaches are summarized and compared with distributed learning in Table I,
see [24]. The key characteristics are emphasized in a dark gray shade, where
the main differentiating features are the orchestration and client state.

TABLE I: Comparison of distributed learning, cross-silo and cross-device FL, adapted from [24]. |  Distributed learning |  Cross-silo FL |  Cross-device FL
---|---|---|---
Setting / Clients |  Training a model on a large dataset - clients are computing nodes in a single cluster or datacenter |  Training a model on siloed data - clients are different organizations or entities |  Clients are a very large number of mobile or IoT devices
Data distribution |  Data is centrally stored and can easily be shuffled and balanced across clients - any client can read any part of the dataset |  Data is generated locally and remains decentralized - each client stores its own data and cannot read the data of other clients
Orchestration |  All clients are almost always available |  Only a fraction of clients is available at any one time
Scale |  Typically, $10^{3}$ clients |  Typically, $10^{0}-10^{2}$ clients |  Massively parallel, up to $10^{10}$ clients
Primary bottleneck |  Computation, as very fast communication is available in datacenters |  Computation or communication |  Communication
Addressability |  Each client has an identity or name that allows the system to access it specifically |  Direct indexing of clients is not possible
Client state |  Stateful – each client may participate in each round of the computation, carrying a state |  Stateless – each client will likely only participate once or a few times
Connection |  Stable |  Relatively stable |  Highly unreliable
Data-partition axis |  Arbitrary across the clients |  Partition is fixed — horizontal or vertical |  Partition is fixed and horizontal

Another common distinction in FL is horizontal and vertical FL, primarily
based on the data space, as introduced in [29] and visualized in Fig. 3.
Adopting the notation from [29], horizontal FL, also known as sample-based FL,
is employed when clients share the same feature space $\mathcal{X}$ and target
space $\mathcal{Y}$ but differ in the sample space $\mathcal{I}$. This can be
thought of as a horizontal split through the large data matrix we would obtain
in a centralized setting (clients send their data to a central server). We
formalize this as

$\displaystyle\mathcal{X}_{i}=\mathcal{X}_{j},\
\mathcal{Y}_{i}=\mathcal{Y}_{j},\ \mathcal{I}_{i}\neq\mathcal{I}_{j}\ \forall\
\mathcal{D}_{i},\mathcal{D}_{j},i\neq j,$ (2)

wherein $\mathcal{D}$ denotes the local data. In horizontal FL, the objective
is to address the same learning task across all clients, involving identifying
the functional relationship between data samples drawn from $\mathcal{X}_{i}$
and $\mathcal{Y}_{i}$ for each client $i$. A prominent instance of this
approach can be found in mobile phones, for example, in next-word prediction,
emoji suggestion, and out-of-vocabulary word discovery, as detailed in [28].
It is crucial to emphasize that the FL task within horizontal FL is
effectively defined only when all clients share their true functional
relationship realized through their data-generating processes. While minor
disparities, e.g., through different noise levels, can be accommodated, more
significant deviations in the functional relationship necessitate the
application of advanced techniques such as clustered FL, as discussed in [39,
40, 41], or federated meta-learning (FedMeta), see [42].

Vertical FL, referred to as feature-based FL, is employed when clients share
the sample space $\mathcal{I}$ but differ in the feature space $\mathcal{X}$,
as outlined in [29]. This scenario can be formalized as

$\displaystyle\mathcal{X}_{i}\neq\mathcal{X}_{j},\
\mathcal{Y}_{i}\neq\mathcal{Y}_{j},\ \mathcal{I}_{i}=\mathcal{I}_{j}\ \forall\
\mathcal{D}_{i},\mathcal{D}_{j},i\neq j.$ (3)

It can be conceptualized as instances where the large data matrix we would
obtain in a centralized setting undergoes vertical splits across multiple
clients; see Fig. 3(b) for a graphical depiction [43]. Vertical FL is applied
in privacy-preserving regression tasks, as demonstrated in [44] or [45], as
well as in healthcare [8] and finance [11]. Note that the differences in
target space $\mathcal{Y}$ are implicit in the task definition, as only one
client (in our case client 1 as marked by the same color in Fig. 3(b)) can
provide targets to regress on. It is further important to highlight that
vertical FL necessitates integration with supplementary privacy-preservation
methods like differential privacy, homomorphic encryption, or secure multi-
party computation, further elaborated in [46, 47, 48, 49], respectively, as
sharing some encrypted data is necessary.

\begin{overpic}[width=433.62pt]{img/HFL_without_text.pdf}
\put(37.0,85.0){Features} \put(85.5,85.0){Targets} \put(35.0,68.5){Client 1}
\put(90.5,68.5){1} \put(35.0,43.75){Client 2} \put(90.5,43.75){2}
\put(35.0,13.0){Client $N$} \put(90.5,13.0){$N$} \put(45.0,28.0){$\vdots$}
\put(91.0,28.0){$\vdots$} \end{overpic} (a) Horizontal federated learning
(HFL)

\begin{overpic}[width=433.62pt]{img/VFL_without_text.pdf}
\put(37.0,85.0){Features} \put(85.5,85.0){Targets} \put(7.0,39.0){Client 1}
\put(91.0,39.0){1} \put(31.0,39.0){Client 2} \put(61.0,39.0){Client $N$}
\put(51.0,39.0){$\dots$} \end{overpic} (b) Vertical federated learning (VFL)

Figure 3: Sketch of the data partition for horizontal (a) and vertical
federated learning (b). Note that only one client (Client 1) provides target
values for VLF. The red lines indicate the splits through the centralized data
matrix.

Additionally introduced in[29] is the niche concept of Transfer FL or
federated transfer learning, applied in situations at the boundary between
horizontal and vertical FL. In this context, transfer learning techniques, as
delineated in [50], are employed in the FL task, see [51, 52]. This involves
learning a shared feature representation from a small common data set, which
can be applied subsequently to generate predictions for samples with features
from only one client. For further details, see [29, 53, 51].

Another significant categorization is based on the global model. In the
prevalent approach of FL, also termed centralized FL, a central entity - the
global server - is responsible for creating and managing the global model and
steering the federation process. This approach offers benefits such as ease of
implementation and increased control over client sampling. However, drawbacks
include heightened vulnerabilities to system failures and adversarial attacks
against a single point of failure, the global server. In contrast,
decentralized FL has no central entity, as discussed in detail in [54, 55].
Instead, some form of decentralized model aggregation occurs over a peer-to-
peer network, aiming to address the downsides of centralized FL mentioned
above. Another categorization is introduced in [56]. Herein, additional
servers (edge server) are introduced, allowing for near real-time responses to
a smaller number of clients, leading to the concept of edge-based FL.
Combining the advantages of edge servers with a central server then introduces
hierarchical FL, see [57, 58] for in-depth discussions of hierarchical FL.

###  II-C Challenges

Implementing FL methods and algorithms on dedicated applications is associated
with various technical challenges, discussed in detail in [59, 60, 61, 62].
Despite the communication-efficient distributed learning, there is still a
substantial communication overhead compared to local learning, with the
constant exchange of model updates between the clients and a central server.
This is particularly challenging in systems with limited bandwidth,
potentially leading to delays, increased latency, and increased resource
utilization, see [63, 64]. The heterogeneity of the clients’ computational
resources poses another hurdle, demanding adaptive strategies to manage
collaborative learning across devices with diverse computational power,
memory, and energy resources, see [65, 36, 66, 67, 68]. The non-independent
and identically distributed (non-IID) nature of data across clients introduces
further complexities in model training, necessitating algorithms that account
for variations in local data distributions, see[69, 70]. Finally, the
challenge of model aggregation complexity arises from the intricate process of
combining asynchronous updates, addressing learning rate discrepancies, and
ensuring robust mechanisms for aggregating disparate local models into a
coherent global model, see [71]. Addressing these challenges is pivotal to
successfully integrating FL in systems with diverse clients and data
distributions.

In addition to the technical challenges, FL faces critical hurdles related to
evaluation criteria, digital ethics, and incentive mechanisms. Robust
diagnostics capable of identifying and eliminating updates from clients with
faulty sensors or incorrect data are essential for ensuring the reliability
and accuracy of the aggregated FL models, see [72]. On the ethical front,
privacy and security issues are paramount [6]. FL’s core principle of
preserving user privacy while training models without exchanging raw data is a
delicate balance, requiring advanced encryption techniques to aggregate model
updates without compromising sensitive information. However, inherent security
issues, including data poisoning, adversarial attacks, and the potential for
reconstructing private raw data, are present challenges to FL systems, see
[73]. Designing effective incentive mechanisms for FL presents a formidable
challenge as participants must be motivated to contribute their computational
resources and data without direct access to the immediate benefits of the
global model, see[74, 75, 76]. This balancing act requires strategies to
encourage active and sustained participation in the collaborative learning
process. Despite the mentioned challenges, a key objective of FL is to obtain
strong, personalized models for individual clients, see [77, 78, 79, 80].

##  III State-of-the-Art at the Intersection of FL and Control

Building on the prior brief introduction to Federated Learning (FL), this
section focuses on the state-of-the-art at the intersection of FL and control.
First, the concept of _distributed control_ is introduced. Following that, we
investigate _learning and control_ , concentrating on system identification
and controller design. Here, we provide an overview of the literature we deem
interesting for FL. Finally, we provide a concrete overview of the current
literature at the intersection of FL and control.

###  III-A Distributed Control

Distributed Control (DC) is a subfield of control that focuses on designing
and implementing control systems where the control task is distributed across
multiple connected controllers. This allows for a robust framework for
managing complex and large-scale systems where centralized control would be
less effective. Each controller in the system manages its local operations and
integrates into the global control strategy through a network of communication
links to other controllers, see [81, 82, 83, 84, 85, 86, 87]. Again, the scope
of this section is not to provide a detailed introduction to DC, as there is
excellent literature available, e.g., [88, 89], but to provide the main
concepts, assumptions, and some examples. Similarities and synergies between
DC and FL, as well as major differences, are also discussed. Earlier work on
distributed multi-agent coordination can be found in, e.g., [90, 91, 92]. A
further overview focusing on decentralized control is given by [93].

In the context of DC, several foundational assumptions are essential. DC is
based on the decentralization of control, which means that each controller has
enough computational power and resources to manage local control tasks
autonomously. This decentralization demands robust connectivity for the
efficient exchange of information, guaranteeing that controllers can
coordinate and synchronize operations throughout the system. Nevertheless, the
DC architecture is also designed so that the failure of individual components
does not trigger a systemic failure, thereby ensuring continuous system
operation despite localized disturbances. Scalability is another critical
assumption underlying DC. The system design should allow for a seamless
addition or removal of nodes and modifications to the network’s configuration
without significant disruptions to its overall functionality. Synchronization
mechanisms within DC are assumed to ensure that, despite the autonomy of
individual controllers, operations are harmonized across the system,
particularly where processes are interconnected [89, 94]. Moreover, the
effectiveness of DC depends on the availability of accurate and timely data
from local sensors, which each node uses to make informed decisions. This
reliance on data assumes that all nodes have access to the necessary resources
- such as computational power and energy - to perform their functions
reliably. Finally, given the distributed nature of the system, security
measures are presumed to be robust, safeguarding data privacy against
potential adversaries. These assumptions serve as the foundation for the
design of DC systems.

Established application examples for DC include smart grids, multi-agent
systems, and connected autonomous vehicles (CAVs). In smart grids, various
components like renewable energy sources and consumer appliances must
coordinate to distribute electricity efficiently, e.g., [95, 96, 97, 94]. This
coordination is done using DC, where each node adjusts its operations based on
local data, e.g., energy demand or supply availability, and communicates with
neighboring nodes to maintain stability and efficiency. The work [98] also
provides an early example of applying ideas from DC to large-scale systems,
illustrated by a power system example, whereas [99] provides a summary of
recent research.

In distributed control of multi-agent systems, see [100, 101], formation
control involves multiple agents maintaining a predefined spatial arrangement
while moving, see, e.g., [102]. An early example of formation control of
unmanned aerial vehicles is provided by [103]. Each robot adjusts its position
based on the positions of its neighbors rather than central instructions, see
[104]. In [105], control performance and communication effort are compared in
multi-agent systems.

Furthermore, the platooning of connected automated vehicles (CAVs) is
projected to significantly alter road transportation by enhancing traffic
efficiency and decreasing fuel consumption, see [106, 107]. In [108], a
distributed model predictive control algorithm is proposed for the vehicle
platooning problem. The rapid development of vehicle-to-vehicle
communications, see [109] for a detailed discussion of vehicle-to-everything
(V2X) communication, also encourages using distributed control techniques. The
work [110] provides a detailed survey on cooperative longitudinal motion
control for multiple CAVs.

Further examples are given in [111], wherein decentralized, distributed and
centralized control systems are compared based on the objective of improved
system performance of a multi-zone furnace. The authors of [112] discuss the
problem of achieving multi-consensus in a linear multi-agent system using
distributed controllers. The distributed control of nonlinear interconnected
systems is studied in [113]. State observers are also investigated in the
setting of distributed control. For example, [114, 115, 116] discuss the
important issue of distributed observer design for LTI systems, whereas [117]
focuses on the design of distributed Luenberger observers. In [118], the
distributed Kalman filter is discussed.

DC and FL share several foundational principles based on their decentralized
nature. Both methodologies emphasize decentralization, where decision-making
for DC and learning for FL are distributed across agents rather than
centralized in a single entity. This structure reduces the risk associated
with single points of failure, increasing the system’s robustness. Both DC and
FL rely heavily on local processing. Agents in these systems handle data
locally, reducing the need to transport vast amounts of data across the
network. This not only saves bandwidth but also improves privacy by limiting
the accessibility of critical information. Scalability is another shared
feature; DC and FL are designed to handle an increasing number of agents
easily. This scalability means that growing the network or integrating more
agents does not affect the overall system performance, allowing the system to
remain manageable and operationally efficient as it grows. These shared
characteristics underline the adaptability and efficiency of both DC and FL in
managing complex, distributed tasks. Whether in controlling physical processes
or in data processing for machine-learning, both paradigms leverage
decentralization, local processing, and coordinated communication to meet the
respective goals effectively.

While Distributed Control (DC) and Federated Learning (FL) share some
fundamental principles, their differences are significant, reflecting
different aims, applications, and operating approaches.

Purpose and Application: DC is commonly used in engineering systems to control
physical processes and devices, such as grids, fleet robotics, and
infrastructure management. Its main goal is the direct control of physical
entities to ensure operational efficiency and reliability - fulfilling the
global control task. In contrast, FL is a machine-learning technique that
enables the training of models across multiple decentralized devices and is
particularly useful in data-driven applications where privacy and bandwidth
constraints are critical.

Data Handling: Data handling also underscores a fundamental difference between
the DC and FL. DC is concerned with real-time control, whereas FL uses data to
train machine-learning models without real-time considerations.

Algorithmic Focus: From an algorithmic perspective, DC concentrates on
algorithms that ensure stability, control, and optimization of system
dynamics, reflecting its direct interaction with physical systems. On the
other hand, FL’s algorithms are geared towards learning and inference, aiming
to optimize accuracy, reduce model bias, and, importantly, enhance data
privacy through its decentralized approach.

Privacy Concerns: In terms of privacy, while DC may handle sensitive
information, especially in critical infrastructure settings, its primary
concerns revolve around operational security and system reliability. FL
explicitly addresses privacy concerns, as it is designed to minimize data
exposure by ensuring that only model updates are communicated rather than raw
data.

In summary, while DC and FL operate on distributed principles, they apply
these principles to fundamentally different tasks - control and optimization
in real systems for DC and privacy-preserving collaborative learning in FL.
Their methodologies reflect their respective goals: immediate and direct
control of physical environments versus incremental and privacy-preserving
improvement of predictive models.

###  III-B Learning and Control with Focus on FL

This section briefly introduces _Learning and Control_ (LC), focusing on
concepts similar to Federated Learning (FL). While LC deserves extensive
coverage, we provide a concise overview of relevant topics and refer readers
to the key literature to explore this exciting intersection of artificial
intelligence and control theory. Within the field of LC, we see two main
categories that are promising for FL, namely

  * •

System identification

  * •

Controller design

The review of data-driven control [119] offers insights into the challenges of
model-based control (MBC) theory, the significance of data-driven control
(DDC) methods, the state-of-the-art, their classifications, and the
relationship between model-based and data-driven control.

####  III-B1 System Identification

Standard textbooks describe various ways to tackle the problem of system
identification, such as [120, 121, 122]. In the following, we discuss selected
works in this field that apply ideas from the learning literature to system
identification problem.

For the identification of stable LTI systems, [123] proposes a maximum
likelihood routine based on the Expectation Maximization (EM) algorithm with
latent states or disturbances and Lagrangian relaxation. The identification of
unstable linear systems is discussed in [124], where finite-time bounds on the
error of the Least Squares (LS) estimate of the dynamic matrix are derived for
a large class of heavy-tailed noise distributions. The work [125] presents a
novel statistical analysis of the LS estimator for LTI systems with stable,
marginally stable, and explosive dynamics. Learning autonomous linear
dynamical systems from a single trajectory or rollout is thematized in [126,
127], whereas [128] and [129] discuss identification based on multiple
trajectories. In [130, 131] spectral filtering is introduced to learn the
impulse response function for latent-state linear dynamical systems. When
working with nonlinear systems of the form $\dot{x}=f(x,u)$, a common practice
is linearizing around a reference point $\\{x_{r},u_{r}\\}$. The work [132]
follows this idea and uses regularized LS to obtain the linearized dynamics.

An overview of nonlinear dynamic system identification techniques is given in
[133]. A perspective based on kernel-methods and Bayesian statistics,
including support vector machines, Gaussian regression, and reproducing kernel
Hilbert spaces (RKHS), is proposed in [121] and further emphasized in [134]. A
kernel specifically tailored for Port-Hamiltonian systems is presented in
[135] and preserves the passive nature within this system class. Gaussian
mixture models are applied to encode the robot’s motion as a first-order
autonomous nonlinear ODE, see [136]. In [137], the combination of SINDY -
sparse identification of nonlinear dynamics, see [138] \- and MPC is proposed
to enhance the control performance. A modern perspective based on deep neural
networks is elaborated in [139] and [140]. Using neural networks in system
identification was already proposed in the 1990s [141, 142, 143]. The authors
of [144] formulate the system identification task based on high-dimensional
uncertain measurements, e.g., videos, as a neural network-based approximation
of the posterior of the control loss. Deep Learning was also applied to learn
representations of the Koopman operator and its eigenfunctions from data, as
seen in [145]. Another line of work focuses on _Neural State Space models_ ,
wherein neural networks are utilized to learn state space representations.
Early works trace back to [146], where recurrent neural networks are used for
nonlinear system identification. Recent works utilize autoencoders [147, 148],
genetic algorithms [149], or meta-learning [150, 151] for learning a nonlinear
state space representation. The problem of learning long sequence dependencies
is tackled via structured state space models in [152, 153, 154]. Learning
state representations can be interpreted as a generalization of system
identification, as it comprises learning forward and inverse models. An
excellent review summarizing state representation learning formalism, as well
as the learning objectives and building blocks, are given in [155].

Table II summarizes recent advances in learning-based identification for
linear and nonlinear systems, which we consider promising for FL applications.

TABLE II: Recent literature regarding linear and nonlinear system
identification with potential for FL.

Source |  Description |  Dynamics |  Techniques |  FL potential |  Input
---|---|---|---|---|---
[156] |  Introduces distributed stochastic gradient descent (SGD) with reversed experience replay for distributed online system identification of identical LTI systems |  LTI |  SGD |  High |  None
[157] |  Clustered system identification based on mean squared error (MSE) criterion |  LTI |  SGD; Cluster estimation |  High |  Gaussian
[158] |  System identification inspired by multi-task learning to estimate the dynamics of linear time-invariant systems jointly by leveraging structural similarities across the systems via regularized LS |  LTI |  regularized LS; Proximal gradient method |  High |  Constant
[159] |  Propose _Control-oriented regularization_ for LTI system identification using control specifications as Bayesian prior |  LTI |  Bayesian perspective; regularized LS |  Medium |  Linear
[160] |  Provide finite-time analysis for learning Markov parameters of LTI systems applying an ordinary LS estimator with multiple rollouts covering both stable and unstable systems |  part. observable LTI (open-loop stable or unstable) |  LS; Markov parameters |  High |  Gaussian
[161] |  Prefiltered Least Squares algorithm that provably estimates the dynamics of partially-observed linear systems |  part. observable LTI |  prefiltered LS |  Medium |  Gaussian
[162] |  Leverage data from auxiliary (similar) systems |  LTI |  weighted LS |  Medium |  Gaussian
[163] |  Jointly estimating transition matrices of multiple, related systems |  LTI |  SGD; Basis functions |  High |  None
[164] |  Automated tuning method for controller with safety constraints |  Linear |  Bayes opt.; Nonlin. regression |  High |  PI-control
[165] |  Simultaneously estimate states and explore structural dependencies between estimated dynamics |  Linear parameter varying (LPV) |  multiple neural networks |  Medium |  Necessary1
[166] |  Multi-robot transfer learning for SISO systems |  Nonlinear |  Transfer learning |  Medium |  Input-output linearization
[146], [167] |  Neural state space models |  Nonlinear |  Neural network |  High |  Possible2
[147, 148] |  Neural state space models via autoencoder |  Nonlinear |  neural network; Autoencoder |  Medium |  Necessary1
[151] |  Safe learning and control based on an online uncertainty-aware meta-learned dynamics model |  Nonlinear |  NN; Last layer adaptation |  High |  Necessary1
[168] |  Bayesian multi-task learning model using trigonometric basis functions to identify errors in the dynamics |  Nonlinear |  Basis functions; Max. Likelihood; Kalman filtering |  Medium |  MPC
[169] |  Stochastic MPC based on a learned input-feature model combined with (online) Bayesian linear regression and online model selection to leverage multiple input-feature models |  Nonlinear |  Bayesian; NN; |  High |  MPC
[170] |  Probabilistic Deep Learning to meta-learn building models using multi-source datasets |  Nonlinear |  NN; meta-learning; |  High |  None

  * 1

Inputs are necessary for system identification, but not specified in the
source.

  * 2

Method also applicable for autonomous systems.

####  III-B2 Controller Design

When given a system description, through first-principles modeling or data-
driven system identification, we typically want to control the system to
behave in a desired way. For this, a controller which determines the system’s
input is designed. The main distinction here is between controllers without
feedback (feedforward) and with feedback, wherein a feedback controller uses
measurements of the system’s output and closes the control loop. Learning
controllers using neural networks have a rich history, exemplified in works
like [171, 143, 142, 172]. Moreover, data-driven control (DDC) is thematized
in [173, 174], with a special focus on control design for nonlinear systems. A
distinction can be made between direct DDC and indirect DDC, where system
identification and control are performed sequentially. In [175], the
connection between indirect and direct DDC approaches is discussed. They
formulate these approaches using behavioral systems theory and parametric
mathematical programs and bridge them through a multi-criteria formulation,
trading off system identification and control objectives. The study reveals
that direct DDC can be derived as a convex relaxation of the indirect
approach, with regularization accounting for implicit identification steps.
Direct and indirect predictive control is the main topic of [176]. In this
comparative study based on stochastic LTI systems, two distinct non-asymptotic
regimes in control performance can be distinguished for direct and indirect
predictive control.

There is also much work on optimal control, especially for unknown, or
partially known systems. In [177], a Thompson sampling-based learning
algorithm is used to learn the dynamics, which are subsequently used for
Linear Quadratic Regulator (LQR) control design. They show robustness to time-
varying parameters of the controlled stochastic LTI system. Controlling an LTI
system with known noisy dynamics and adversarial quadratic loss is tackled
using semi-definite relaxation in [178] and [179], leading to strongly stable
policies. LQR control for unknown linear systems is further investigated in
[180, 181, 182, 183]. The problem of adversarial changing convex cost
functions with known linear dynamics is tackled in [184] and [185] using a
Disturbance-Action Controller (DAC) given by
$u_{t}=Kx_{t}+\sum_{i=1}^{H}M_{i-1}w_{t-i}$ combined with online convex
optimization. The papers [186, 187, 188, 189] focus on controlling unknown
linear dynamical systems subjected to non-stochastic, adversarial
perturbations. The class of Gradient Perturbation Controllers (GPC) is
introduced, combining a stabilizing linear controller $K$ with a DAC
parametrized by the matrices $M_{i},i=1,\dots,H$ for some horizon $H$ and
disturbance $w_{i}$. The general class of convex optimization control policies
(COCPs), including standard applications like LQR, approximate dynamic
programming, and model predictive control is discussed in [190]. They propose
updating the control parameters based on the projected stochastic gradients of
performance metrics (cost functions) instead of the standard way of tuning by
hand, or by grid search. The work of [191] considers Linear Quadratic Gaussian
(LQG) problems with unknown dynamics. They leverage Input-Output
Parameterization (IOP), see [192], for robust controller synthesis based on a
convex parameterization. Similar works [193] and [194] focus on the adaptive
aspect of the problem and introduce a control algorithm combining system
identification and adaptive control for LGQ systems.

In adaptive control, early work uses neural-network-based adaptive controllers
for trajectory tracking of robotic manipulators, see [195, 196, 197]. Radial
basis function networks were used even earlier for adaptive control, see
[198]. Recent work on model reference adaptive control (MRAC) based on
Gaussian Processes is given in [199], whereas [200] studies MRAC based on deep
neural networks. In [201], an adaptive nonlinear MPC is designed, so that
model uncertainties are learned online and immediately compensated for.
Adaptive control for high-dimensional systems is always challenging. This
problem is tackled in [202], wherein a non-parametric adaptive controller is
proposed that scales to high-dimensional systems by learning the unknown
dynamics in a reproducing kernel Hilbert space (RKHS) leveraging random
Fourier features. The work [203] introduces local Gaussian process regression
as a method that achieves high accuracy in online learning and real-time
prediction, applied to inverse dynamics model learning for model-based control
of two 7-DoF robot arms.

Table III summarizes the recent advances in learning-based controller design,
which we consider promising for FL applications.

TABLE III: Recent literature regarding controller design with potential for FL. Source |  Description |  System |  Techniques |  FL potential
---|---|---|---|---
[204, 205] |  Controlling an unknown system with varying latent parameters, aiming to learn approximate models for both the dynamics and the prior over latent parameters from observed trajectory data |  Nonlinear |  NN; meta-learning |  High
[206, 207] |  Control-oriented approach to learning parametric adaptive controllers through offline meta-learning from past trajectory data |  Nonlinear |  NN; meta-learning; |  Medium
[208] |  Meta-learning online control algorithm that learns across a sequence of similar control tasks |  Linear |  Projected gradient descent |  Medium
[209] |  Transferring knowledge from source system to design a stabilizing controller for target system |  Linear |  Transfer Learning |  High
[210] |  Combining prior knowledge and measured data for learning-based robust controller design, leading to stability and performance guarantees for closed-loop systems |  Linear |  Linear fractional transformations |  Medium
[211] |  Online multi-task learning approach for adaptive control with environment-dependent dynamics |  Nonlinear |  NN; Last-layer adaptation |  High
[212] |  Multi-task imitation learning via shared representations for linear dynamical systems |  Linear |  Multi-task imitation learning |  High
[213] |  Learning a time-varying, locally linear residual model of the dynamics to compensate for prediction errors of the controller’s design model |  Linear residual model |  Ridge regression; Replay buffer |  Medium
[214] |  Combine simulated source domains and adversarial training to learn robust policies |  Nonlinear |  Policy gradient; NN |  Medium
[215] |  Present a learning algorithm termed _Meta Strategy Optimization_ that learns a latent strategy space suitable for fast adaptation in training |  Nonlinear |  Meta learning; NN |  Medium
[216] |  Decomposing neural network policies into task-specific and robot-specific modules, enabling transfer learning across different robots and tasks with minimal additional training |  Nonlinear |  NN; multi-task & transfer learning; |  High
[217] |  LQR in a multi-task setting, with the MAML-LQR approach producing stabilizing controllers close to task-specific optimal controllers |  Nonlinear |  Policy gradient & NN; meta-learning |  High

###  III-C Existing Literature for FL and Control

Currently, existing works that combine Federated Learning (FL) methodologies
with control theory can be found in four major areas:

  * •

system identification,

  * •

controller design,

  * •

federated reinforcement learning, and

  * •

control-inspired aggregation.

Table IV summarizes the primary outcomes of these studies. This section refers
to the first research question, Q1, in Section I.

TABLE IV: Summary of the literature connecting FL and Control. Topic |  Summary and Literature References
---|---
System Identification |  Recent research explores collaborative learning of linear systems in system identification, using FedSysID to balance performance and system heterogeneity, see [218, 158]. A distributed Recursive Least Squares (RLS) algorithm, discussed in [219], addresses robust estimation in networked environments, aligning with FL principles.
Controller Design |  Practical applications of Federated LQR concepts have demonstrated their scalability and efficiency, marking significant strides in both linear and nonlinear systems [220, 221]. In [222], the integration of neural networks into an adaptive PID-control system, leveraging FL to overcome limitations posed by limited local training data, showcases the potential of this approach. In [223], FL is harnessed to learn static feedback controllers for a fleet of robots, while [224] explores the use of FL in swarm control of drones. Personalized FL is applied for learning in robotic control applications [225].
Reinforcement Learning |  Federated Learning mitigates reinforcement learning’s sample inefficiency and accommodates heterogeneous sensor data, see [226, 227, 228, 229].
Control-inspired Aggregation |  Novel research integrates control theory principles into FL by enhancing global model aggregation with integral and derivative terms, akin to PID control frameworks, see [230, 231, 232, 233].

In system identification, a strong focus has been put on collaboratively
learning linear, time-invariant (LTI) system dynamics from diverse
observations across multiple clients under privacy considerations, as
exemplified in [218] and [158]. In [218], the _FedSysID_ algorithm is
introduced, addressing Least Squares (LS) system identification for multiple
linear clients with similar dynamics, showing improved convergence compared to
individual learning. In [158], the focus lies on leveraging similarities among
multiple systems to accurately estimate LTI dynamics using LS, although
privacy considerations were not specifically addressed. Another notable
contribution, detailed in [219], introduces a distributed Recursive Least
Squares (RLS) algorithm tailored to robust estimation in networked
environments. In this scenario, each client measures samples linked by a
common, yet unknown, linear regression model. This algorithm can be
interpreted within the framework of FL, as knowledge is exchanged between
clients by transmitting and aggregating covariance matrices of an RLS
algorithm. Pioneering work in adaptive networks111Adaptive networks consist of
a collection of nodes with learning abilities. The nodes interact with each
other on a local level and diffuse information across the network to solve
inference and optimization tasks decentralized., a field not directly linked
but conceptually related to FL, comprehensively examines recent strides in
adaptation, learning, and optimization across networked systems, as surveyed
in [86] or [234]. These surveys offer valuable insights for comparing diverse
network topologies and facilitate an assessment of adaptive networks’
performance in contrast to centralized implementations.

Controller design has a line of work on federated LQR concepts, e.g., [220]
and [221]. In [220], the distributed LQR tracking problem is studied in a
setting of clients sharing linear but unknown dynamics and tracking different
targets. The proposed model-free federated zero-order policy gradient
algorithm capitalizes on a shared LQR matrix across clients, demonstrating a
linear speedup in the number of clients over communication-free, local
alternatives, even when clients have a heterogeneous component in their
objective, namely their tracking target. Simulation results demonstrate the
algorithm’s effectiveness in linear and nonlinear system settings, showcasing
its scalability. In [221], the focus is on the model-free federated LQR
problem involving multiple clients with distinct and unknown yet similar LTI
dynamics collaborating to minimize an average quadratic cost function while
maintaining data privacy. The proposed FL approach, named _FedLQR_ , allows
clients to periodically communicate with a central server to train policies,
addressing questions related to the stability of the learned common policy,
its proximity to individual clients’ optimal policies, and the speed of
learning with data from all clients. In [223], fleet-level learning of static
feedback controllers from distributed and potentially heterogeneous robotic
data is discussed. They proposed the _FLEET-MERGE_ algorithm, which
efficiently merges neural network-based controllers by accounting for
permutation invariance in recurrent neural networks without centralizing the
data.

An additional line of work delves into the integration of neural networks
within adaptive PID-control systems for connected autonomous vehicles (CAVs),
as exemplified in [75] and [222]. Control parameters are dynamically adjusted
to enable navigation under varying traffic and road conditions, which is
facilitated by a neural network-based auto-tuning unit learning the system
behavior, as detailed in [235]. The autonomous vehicle’s local training data,
restricted by onboard memory, limits the controller’s adaptability to specific
traffic scenarios, potentially compromising safety. To overcome this
constraint, the proposed approach advocates for implementing FL. In this
collaborative process, CAVs collectively learn the neural network-based auto-
tuning units facilitated by a wireless base station functioning as a global
server. This FL mechanism empowers CAVs to enhance neural network auto-tuning
units for local controller adjustments, effectively tackling the issue of
limited local training data and broadening the controller’s applicability
across diverse traffic patterns. In [225], personalized FL is applied for
learning trajectory forecasting models in robotic control applications,
wherein personalization is achieved by adjusting learning rates based on
parameter variances. An excellent example of the combination of FL and control
is given in [224]. Here, the swarm control of a large number of drones is
considered, combining mean-field game theory, Federated Learning, and Lyapunov
stability analysis. Numerical examples show the efficacy of the proposed
approach.

The intricate connections between control and reinforcement learning (RL)
arise from their common pursuit of effective decision-making and optimization.
Control theory contributes well-established principles for system regulation
and trajectory tracking, complemented by reinforcement learning’s adaptive
techniques designed for acquiring optimal behaviors in dynamic and uncertain
settings; see, e.g., [236, 237, 238] for detailed reviews of reinforcement
learning. The growth in computational power and the development of deep neural
networks have significantly enhanced the capabilities of reinforcement
learning algorithms. This technological growth has propelled the field into
new frontiers, enabling more sophisticated learning and decision-making
processes. Reinforcement learning algorithms, however, are confronted with the
challenge of sample inefficiency, often necessitating a substantial number of
interactions with the environment to acquire effective policies. This
inefficiency poses a significant drawback, particularly in real-world
scenarios where data collection can be resource-intensive and time-consuming.
Federated Learning techniques offer a promising avenue to mitigate these
challenges by facilitating collaborative learning among multiple agents
(clients) to derive optimal policies, see [226]. The authors propose a
federated RL scheme based on the actor-critic Proximal Policy Optimization
(PPO) algorithm to learn a classical nonlinear control problem: the upswing of
a pendulum. They clearly show the effectiveness of the proposed FL solution by
significantly reducing the convergence time, despite the slightly different
dynamics of the individual devices, namely three _Quanser QUBETM-Servo 2_ ,
see [239]. In [240], a federated reinforcement learning setup shows a linear
convergence speedup concerning the number of agents. Additionally, FL’s
capability to accommodate heterogeneous real-life or artificially generated
sensor data further broadens its applicability and effectiveness in diverse
and complex environments, as exemplified in [227, 228, 229, 241].

The discussion so far predominantly centers around applying FL techniques
within specific control areas, namely system identification, controller
design, and reinforcement learning. However, the landscape looks different
when examining control-inspired aggregation, representing a shift in
viewpoint. In this context, concepts derived from control are employed to
enhance FL algorithms. A noteworthy research path involves augmenting the
global model aggregation process by incorporating integral and derivative
terms akin to conventional PID-control frameworks. This approach, exemplified
in works such as [230, 231, 232, 233], represents a unique integration of
control theory principles to optimize the FL process. Another work,
conceptually similar to FL, is given by [99]. Here, they tackle the problem of
consumer scheduling under incomplete information.

##  IV FL and Control

In light of the literature review conducted in the previous section, this
section explores various scenarios in which Federated Learning (FL) concepts
can be integrated into control theory, encompassing system identification and
control, and knowledge transfer in multi-agent systems. Furthermore, FL’s
adaptive learning capabilities are discussed for dynamic systems, highlighting
its suitability for evolving environments and changing operating conditions.
This section refers to the second research question, Q2, in Section I and
presents FL as a promising framework for advancing various facets of control.

###  IV-A Methodical Benefits of Combining FL and Control

In control, the integration of FL principles offers a promising avenue for
advancing system adaptability via communication-efficient, collaborative
learning with preserved privacy as well as enhanced generalization and
robustness. Federated Learning’s adaptive learning capabilities find resonance
in control applications dealing with dynamic systems. Clients can continuously
update their knowledge using a system model or controller parameterization
based on changing environmental conditions. This makes FL suitable for control
scenarios where system dynamics or shared external influences evolve. The
decentralized approach inherent in FL aligns seamlessly with distributed
control systems. This synergy enables multiple clients engaged in control
tasks to collaboratively update their control policies or global model
representation based on local observations, fostering a coordinated and
efficient system behavior, as demonstrated in [226] and [242]. The emphasis on
communication efficiency is particularly beneficial for control systems with
many participants or low bandwidths. Clients engaged in collaborative learning
tasks can exchange minimal data while acquiring control policies, mitigating
communication challenges, and enhancing overall system efficiency. Transfer
learning proves valuable in control domains, mirroring FL’s practice of
training models on one task and adapting them to another, see [243]. This
concept facilitates transferring and fine-tuning of learned control policies
from one system to a similar yet distinct one, minimizing the necessity for
extensive and time-consuming retuning and retraining efforts. Privacy-
preserving control emerges as a critical application, leveraging FL’s emphasis
on local model training without raw data sharing. In scenarios where data
privacy is paramount, such as personal data in healthcare, critical
infrastructure, or strictly confidential production data, clients can
collaboratively learn control policies without disclosing sensitive
information, thus aligning with privacy requirements. By integrating robust
control techniques into FL algorithms, further guarantees of system
performance under uncertainties or adversarial conditions may be established,
bolstering the reliability and generalization of learned models in real-world
control applications.

In summary, including FL principles, encompassing adaptability,
generalization, communication efficiency, decentralization, and privacy
preservation, holds significant potential for advancing various facets of
control. This integration contributes to developing more efficient, flexible,
secure, and responsive control systems.

###  IV-B Expected Applications of Combining FL and Control

TABLE V: Overview of Expected Applications.

Application |  Core Idea |  Benefits |  Potential Downsides
---|---|---|---
Parametric federated SysID |  Inferring parameters of the structurally known dynamics of a system based on local input-output data from multiple clients |  Improved convergence rate over single client learning; persistent excitation of global model; _federated dynamics_ |  FL costs1 ; useful only for similar systems; high level of system knowledge required
Non-parametric federated SysID |  Collaboratively learn non-parametric dynamics based on local input-output data from multiple clients |  Improved convergence; increasing generalizability; _federated dynamics_ |  FL costs1 ; useful only for similar systems
Indirect Adaptive Control |  Learning _federated dynamics_ model and use local or global model as robust control design model |  Improved control performance and robustness under uncertainties in dynamic environments |  FL costs1 ; similar control requirements necessary;
Direct Adaptive Control |  Learn a control law on local data and aggregate at global server |  Improved generalization; improved initial model for local adaptive control; |  FL costs1 ; application to similar systems only; potential stability issues
Advanced server-side optimization |  Utilizes cloud computing power on global server |  Server-side update/training of neural-network-based control laws for low power hardware, e.g., Learning-based NMPC approximation |  FL costs1 ; powerful server necessary
Multi-agent decision-making |  Explicit distinction between decision-oriented and support-oriented agents |  Improves privacy-preserving decision-making and coordination among multiple agents |  FL costs1 ;
Sensor fusion & shared representation learning |  Combine different sensor modalities to extract a richer environment representation |  Enhances situational awareness and perception capabilities, e.g., merging visual, depth, and semantic maps |  FL costs1 ; scalability issues
Transfer learning |  Merges real-life and simulator data for knowledge transfer |  Facilitates safer exploration & knowledge transfer |  FL costs1 ;

  * 1

See Section II-C for a detailed discussion of FL costs.

There are many ways to include concepts from FL in control. Table V summarizes
the introduced applications.

####  IV-B1 System Identification

Foremost, Federated Learning presents a promising avenue for enhancing system
identification (SysID) within the control framework. Typically, system
identification is separated into parametric SysID and non-parametric SysID. In
parametric SysID, first principles and process knowledge are used to develop
parametric models, whose physics-based parameters are calibrated based on
input-output data. In non-parametric SysID, black-box models, e.g., highly-
capable function approximation algorithms like neural networks, are employed
to learn the system dynamics. Federated Learning extends this paradigm by
enabling multiple clients to collaboratively learn from decentralized data
sources while preserving data privacy, leading to federated SysID.

In this context, parametric federated SysID allows individual clients to
leverage their input-output data to calibrate local models, which are
subsequently aggregated to refine a global model representing the parametric
system dynamics, subsequently termed _federated dynamics_. This ranges from
sharing single parameter information to learning the complete dynamics of some
LTI or nonlinear system. Sharing information through the global model not only
copes with parameter scattering caused by minor, not modeled system
differences, e.g., component-aging effects, but also allows for some form of
persistent excitation of the global model; see [244] for further information
regarding persistent excitation. Furthermore, the scalability and
generalizability of the obtained _federated dynamics_ are enhanced, and
concerns regarding data privacy and security are addressed. An improved
convergence rate can be obtained compared to individual client learning. For
instance, smart pneumatic or hydraulic valves can come equipped with series
models that capture the nonlinear input-output relationships in a parametric
form. During operation, these valves encounter distinct application-specific
operating conditions and device-specific variations like wear and tear, often
not accounted for during end-of-line calibration. The valves can adapt to such
scenarios by embedding a local learning algorithm. These adaptations can also
be fed back to the manufacturer in a privacy-preserving manner to enhance the
overall series model. As a note on the downside, challenges inherent to FL, as
discussed in Section II-C, are also inherent for parametric SysID when
combined with FL. A high level of system knowledge is also required to
formulate the parametric models. Parametric federated SysID can only be
applied when the participating clients and systems follow similar dynamics. A
possible remedy here is the application of clustered FL techniques, wherein
the clients are clustered into similar groups sharing similar dynamics, see
[39, 40, 41]. This cluster assignment is either based on a priori known
criteria or obtained through applying clustering techniques on the client
model parameters.

In non-parametric SysID, the system dynamics are typically learned from input-
output data using sophisticated function approximation algorithms like neural
networks or Gaussian Processes. Non-parametric federated SysID enables the
learned client models to update a global model collaboratively, representing
the _federated dynamics_. This again fosters more robust and generalizable
models of dynamical systems and facilitates a more comprehensive exploration
of the systems dynamics. Furthermore, a faster convergence typical to FL is
obtained if the client dynamics are similar. Additionally, the computational
power of the global server can be exploited for further model refinement
during the global model update. This can enforce knowledge-based constraints
similar to the approach underlying the physics-informed neural networks
(PINNs), which seems intractable for embedded hardware; see [245] for more
information on physics-based Deep Learning or [246] for physics-informed
Machine Learning. FL can also support the development of confidence models,
which estimate the reliability of learned _federated dynamics_ based on
contextual features. These models provide valuable insights into the
confidence level associated with predictions, enabling better decision-making
in dynamic environments where uncertainty is prevalent. Integrating FL with
confidence modeling enhances the trustworthiness of purely data-driven
dynamical system models, contributing to more effective control and decision-
making processes. On the downside are again the challenges inherent to FL; see
Section II-C. Moreover, that non-parametric SysID can only be successfully
applied to similar systems. Again, clustered FL techniques, as discussed in
[39, 40, 41], may be helpful here.

####  IV-B2 Controller Design

Furthermore, FL techniques can also be applied to adaptive control problems,
where indirect and direct adaptive control methods are explored. Indirect
adaptive control involves learning a system model or its parameters from
distributed data and deriving a controller based on the acquired model, as
exemplified by the neural-network-based PID auto-tuning [235]. Federated
Learning can obtain a _federated dynamics_ model, serving as a feedback
controller design model for individual clients. The local controller can then
be designed using the _federated dynamics_ model or a locally adapted version
to ensure a high control performance and robustness under local uncertainties.
On the downside are again the challenges inherent to FL; see Section II-C.
Also, similar control requirements on the client level are necessary to
successfully apply indirect adaptive control combined with FL.

On the other hand, direct adaptive control entails learning the client
controller directly from its datasets. Federated Learning enables combining
these local client controllers in a robust, privacy-preserving way. The
resulting global controller model ensures robustness and generalizability in
various dynamic settings. For instance, hydraulic excavators can use data-
driven servo-compensation models to improve path or trajectory tracking for
tasks like leveling or grading, see [247]. This servo-compensation may be
equipped with a local learning algorithm, adapting individual machines to
various environmental conditions, e.g., the temperature influencing the
viscosity of the hydraulic oil. This information can be shared by applying FL
to obtain a global servo-compensation model, leading to a general improvement
in machine performance. Conversely, we again face challenges intrinsic to FL,
as elaborated in Section II-C. Additionally, there are potential stability
issues related to the global model.

Another way to use FL is to employ the global server’s computational power,
termed advanced server-side optimization, typically much larger than the
client devices, for further advanced control tasks, such as optimal control or
nonlinear model predictive control (NMPC). Especially for the latter, the
hardware requirements for a real-time implementation are still a significant
cost factor. In [248, 249, 250], this hurdle is overcome by optimizing the
corresponding operating range already in the development phase and
subsequently training a neural network (NN) to reproduce the solution in a
computationally efficient way. Federated Learning enables an extension of this
framework towards online adaptations, as the central server can perform the
computationally intensive optimization utilizing the _federated dynamics_ and
then distribute the solution to the clients, e.g., in the form of neural
networks. The global server must have suitable computational hardware to
perform this optimization. Also, challenges related to FL and federated SysID,
as already mentioned before, must be considered.

####  IV-B3 Multi-Agent Decision-Making

Progressing from system identification over controller design to knowledge
transfer, FL holds promise for advancing multi-agent decision-making scenarios
by facilitating collaborative decision-making among agents, including both
decision-oriented and support-oriented agents, see [242]. In such settings, FL
enables agents to learn and share knowledge while preserving the privacy of
their local data. Decision-oriented agents responsible for making critical
decisions can benefit from FL by leveraging insights from decentralized data
sources to improve decision-making accuracy and robustness. Support-oriented
agents providing auxiliary functions such as data preprocessing or information
aggregation can enhance their capabilities using FL by collectively learning
from distributed datasets. Additionally, FL fosters the development of
decentralized coordination mechanisms, allowing agents to coordinate their
actions efficiently without centralized control. By employing FL techniques,
multi-agent systems can adapt and evolve, leveraging the collective
intelligence of diverse agents to achieve more effective and resilient models,
exemplified in [227] and termed _Lifelong Federated Reinforcement Learning_.
For instance, decision-oriented mobile working machines in unstructured
environments may be assisted by support-oriented drones equipped with visual
sensors to solve tasks like handling cargo or material transport
collaboratively.

Furthermore, FL offers significant potential in multi-agent systems for
learning a shared representation of the environment, see [251]. By
collaboratively training models on decentralized data sources, FL enables
agents to collectively understand the environment, leading to more coherent
decision-making and coordination among agents. This becomes particularly
relevant in settings like construction sites where numerous stakeholders
interact, making privacy-preserving learning and adaptation of shared
environmental representations through FL methods of utmost interest.

Moreover, FL can enhance sensor fusion capabilities within multi-agent systems
by leveraging data from diverse sensor modalities distributed across agents,
see [228], wherein visual data (RGB and depth maps) and semantic segmentation
data are combined within an FL setting. This enables the extraction of richer
and more comprehensive information about the environment, improving
situational awareness and decision-making accuracy.

Additionally, FL facilitates transfer learning and multi-task learning (see
[252]) in control by allowing agents to transfer knowledge learned from one
environment to another. This is exemplified in [229], where sophisticated
simulators are combined with real-life LIDAR data to navigate autonomous model
cars in changing indoor environments with obstacles. Similarly, the work [253]
introduces an FL architecture for cooperative simultaneous localization and
mapping (SLAM) in cloud robotic systems. This adaptability is particularly
valuable in dynamic environments where conditions may change over time. By
leveraging transfer learning techniques with FL, multi-agent systems can
effectively adapt to new scenarios and improve overall performance and
robustness. Inherent to the presented applications regarding knowledge
transfer, namely multi-agent planning, shared representation learning, sensor
fusion, and transfer learning, are the already introduced challenges of FL,
see Section II-C.

##  V Conclusion

This paper presents an overview of state-of-the-art methods and ideas for
combining Federated Learning (FL) and control. A detailed literature review
reveals a scarcity of research at the intersection of FL and (nonlinear)
control. Building on the advancements in distributed control and learning,
combining FL and control can improve the system’s adaptability and privacy
preservation by allowing for decentralized controller updates, privacy-
preserving control, and adaptive learning, thereby improving control
efficiency, security, and responsiveness.

Practically, FL enriches control methodologies by enhancing system
identification, improving dynamical system modeling, and advancing control
techniques. Additionally, it facilitates knowledge transfer in multi-agent
decision-making via environment representation learning, sensor fusion, and
transfer learning, fostering adaptive and resilient control systems. Apart
from individual scientific activities in the synergetic combination of FL and
control, several further developments are required. The evident mismatch in
the main objectives between FL and control - FL attempts to achieve privacy-
preserving global model learning, whereas control usually concentrates on
localized optimal performance - may be the main reason for this open research
potential, see Section III. Tables II and III, combined with insights
presented in Section IV, provide important guidance and ideas for applying FL
to system identification and control. Up to the author’s knowledge, this work
marks the first comprehensive survey of FL and control, laying the groundwork
for further research to combine FL and control for collaborative control
applications. In essence, FL offers a valuable additional feature set that,
when integrated with control, can drive advancements across various domains.
However, substantial further research is required to realize the ideas
outlined in this work.

## References

  * [1] Robert Bosch GmbH. Bodas connect. [Online]. Available: https://www.boschrexroth.com/de/de/transforming-mobile-machines/elektronifizierung-und-iot/bodas-connect/
  * [2] Beckhoff Automation GmbH. Twincat. [Online]. Available: https://www.beckhoff.com/de-at/produkte/automation/twincat-3-fuer-industrie-4.0/
  * [3] Y. Sun, J. Liu, J. Wang, Y. Cao, and N. Kato, “When Machine Learning Meets Privacy in 6G: A Survey,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 4, pp. 2694–2724, 2020.
  * [4] C. Ma, J. Li, K. Wei, B. Liu, M. Ding, L. Yuan, Z. Han, and H. Vincent Poor, “Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning,” _Proceedings of the IEEE_ , vol. 111, no. 9, pp. 1097–1132, 2023.
  * [5] J. Park, S. Samarakoon, A. Elgabli, J. Kim, M. Bennis, S.-L. Kim, and M. Debbah, “Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications,” _Proceedings of the IEEE_ , vol. 109, no. 5, pp. 796–819, 2021.
  * [6] H. Woisetschläger, A. Erben, B. Marino, S. Wang, N. D. Lane, R. Mayer, and H.-A. Jacobsen, “Federated Learning Priorities Under the European Union Artificial Intelligence Act,” _arXiv preprint arXiv:2402.05968_ , 2024.
  * [7] European Commision. Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS - 2021. [Online]. Available: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206
  * [8] Nvidia Cooperation. Federated Learning powered by NVIDIA Clara. [Online]. Available: https://developer.nvidia.com/blog/federated-learning-clara/
  * [9] J. Chen, C. Yi, S. D. Okegbile, J. Cai, and X. S. Shen, “Networking Architecture and Key Supporting Technologies for Human Digital Twin in Personalized Healthcare: A Comprehensive Survey,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 1, pp. 706–746, 2023.
  * [10] S. K. Zhou, H. Greenspan, C. Davatzikos, J. S. Duncan, B. Van Ginneken, A. Madabhushi, J. L. Prince, D. Rueckert, and R. M. Summers, “A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises,” _Proceedings of the IEEE_ , vol. 109, no. 5, pp. 820–838, 2021.
  * [11] Nvidia Cooperation. Using Federated Learning to Bridge Data Silos in Financial Services. [Online]. Available: https://developer.nvidia.com/blog/using-federated-learning-to-bridge-data-silos-in-financial-services/
  * [12] G. Long, Y. Tan, J. Jiang, and C. Zhang, “Federated Learning for Open Banking,” in _Federated Learning: Privacy and Incentive_. Cham, Switzerland: Springer Nature, 2020, pp. 240–254.
  * [13] Y. Wang, F. Zobiri, and G. Deconinck, “Federated Learning for Collaborative Price Prediction and Optimal Trading in the Local Flexibility Market,” _SSRN_ , 2023. [Online]. Available: https://www.ssrn.com/abstract=4534499
  * [14] T. Liu, Z. Wang, H. He, W. Shi, L. Lin, R. An, and C. Li, “Efficient and Secure Federated Learning for Financial Applications,” _Applied Sciences_ , vol. 13, no. 10, p. 5877, 2023.
  * [15] T. Deng, Y. Li, X. Liu, and L. Wang, “Federated learning-based collaborative manufacturing for complex parts,” _Journal of Intelligent Manufacturing_ , vol. 34, no. 7, pp. 3025–3038, 2023.
  * [16] S. Savazzi, M. Nicoli, M. Bennis, S. Kianoush, and L. Barbieri, “Opportunities of Federated Learning in Connected, Cooperative, and Automated Industrial Systems,” _IEEE Communications Magazine_ , vol. 59, no. 2, pp. 16–21, 2021.
  * [17] V. Hegiste, T. Legler, and M. Ruskowski, “Application of Federated Machine Learning in Manufacturing,” in _International Conference on Industry 4.0 Technology_. Pune, Maharashtra, India: IEEE, 2022, pp. 1–8.
  * [18] D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. V. Poor, “Federated Learning for Internet of Things: A Comprehensive Survey,” _IEEE Communications Surveys & Tutorials_, vol. 23, no. 3, pp. 1622–1658, 2021.
  * [19] R. Tallat, A. Hawbani, X. Wang, A. Al-Dubai, L. Zhao, Z. Liu, G. Min, A. Y. Zomaya, and S. H. Alsamhi, “Navigating industry 5.0: A Survey of Key Enabling Technologies, Trends, Challenges, and Opportunities,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 2, pp. 1080–1126, 2024.
  * [20] S. Liu, B. Guo, C. Fang, Z. Wang, S. Luo, Z. Zhou, and Z. Yu, “Enabling Resource-Efficient AIoT System With Cross-Level Optimization: A Survey,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 1, pp. 389–427, 2024.
  * [21] A. Brecko, E. Kajati, J. Koziorek, and I. Zolotova, “Federated Learning for Edge Computing: A Survey,” _Applied Sciences_ , vol. 12, no. 18, p. 9124, 2022.
  * [22] H. Gu, L. Zhao, Z. Han, G. Zheng, and S. Song, “AI-Enhanced Cloud-Edge-Terminal Collaborative Network: Survey, Applications, and Future Directions,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 2, pp. 1322–1385, 2024.
  * [23] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-Efficient Edge AI: Algorithms and Systems,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 4, pp. 2167–2191, 2020.
  * [24] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings _et al._ , “Advances and Open Problems in Federated Learning,” _Foundations and Trends® in Machine Learning_ , vol. 14, no. 1–2, pp. 1–210, 2021.
  * [25] M. Nokleby, H. Raja, and W. U. Bajwa, “Scaling-Up Distributed Processing of Data Streams for Machine Learning,” _Proceedings of the IEEE_ , vol. 108, no. 11, pp. 1984–2012, 2020.
  * [26] J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated Optimization: Distributed Machine Learning for On-Device Intelligence,” _arXiv preprint arXiv:1610.02527_ , 2016.
  * [27] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ , ser. Proceedings of Machine Learning Research, vol. 54. PMLR, 2017, pp. 1273–1282.
  * [28] B. McMahan and D. Ramage. (2017) Google Research Blog. [Online]. Available: https://blog.research.google/2017/04/federated-learning-collaborative.html
  * [29] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated Machine Learning: Concept and Applications,” _ACM Transactions on Intelligent Systems and Technology (TIST)_ , vol. 10, no. 2, pp. 1–19, 2019.
  * [30] J. Wang, Z. Charles, Z. Xu, G. Joshi, H. B. McMahan, M. Al-Shedivat, G. Andrew, S. Avestimehr, K. Daly, D. Data _et al._ , “A Field Guide to Federated Optimization,” _arXiv preprint arXiv:2107.06917_ , 2021.
  * [31] “IEEE Guide for Architectural Framework and Application of Federated Machine Learning,” _IEEE Std 3652.1-2020_ , pp. 1–69, 2021.
  * [32] Q. Yang, L. Fan, R. Tong, and A. Lv, “IEEE Federated Machine Learning,” _IEEE Federated Machine Learning - White Paper_ , pp. 1–18, 2021.
  * [33] S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečný, S. Kumar, and H. B. McMahan, “Adaptive Federated Optimization,” in _9th International Conference on Learning Representations_ , Virtual Only Conference, 2021. [Online]. Available: https://openreview.net/forum?id=LkFG3lB13U5
  * [34] D. Jakovetić, D. Bajović, J. Xavier, and J. M. F. Moura, “Primal–Dual Methods for Large-Scale and Distributed Convex Optimization and Data Analytics,” _Proceedings of the IEEE_ , vol. 108, no. 11, pp. 1923–1938, 2020.
  * [35] G. Koliander, Y. El-Laham, P. M. Djurić, and F. Hlawatsch, “Fusion of Probability Density Functions,” _Proceedings of the IEEE_ , vol. 110, no. 4, pp. 404–453, 2022.
  * [36] A. Mitra, R. Jaafar, G. J. Pappas, and H. Hassani, “Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients,” in _Advances in Neural Information Processing Systems_ , vol. 34. Curran Associates, Inc., 2021, pp. 14 606–14 619.
  * [37] F. Hanzely and P. Richtárik, “Federated Learning of a Mixture of Global and Local Models,” _arXiv preprint arXiv:2002.05516_ , 2020.
  * [38] O. A. Wahab, A. Mourad, H. Otrok, and T. Taleb, “Federated Machine Learning: Survey, Multi-Level classification, Desirable Criteria and Future Directions in Communication and Networking Systems,” _IEEE Communications Surveys & Tutorials_, vol. 23, no. 2, pp. 1342–1397, 2021.
  * [39] F. Sattler, K.-R. Müller, and W. Samek, “Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 32, no. 8, pp. 3710–3722, 2020.
  * [40] C. Briggs, Z. Fan, and P. Andras, “Federated learning with hierarchical clustering of local updates to improve training on non-IID data,” in _International Joint Conference on Neural Networks_. Virtual Only Conference: IEEE, 2020, pp. 1–9.
  * [41] A. Taik, Z. Mlika, and S. Cherkaoui, “Clustered Vehicular Federated Learning: Process and Optimization,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 23, no. 12, pp. 25 371–25 383, 2022.
  * [42] X. Liu, Y. Deng, A. Nallanathan, and M. Bennis, “Federated Learning and Meta Learning: Approaches, Applications, and Directions,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 1, pp. 571–618, 2023.
  * [43] W. Xia, Y. Li, L. Zhang, Z. Wu, and X. Yuan, “A Vertical Federated Learning Framework for Horizontally Partitioned Labels,” _arXiv preprint arXiv:2106.10056_ , 2021.
  * [44] A. Gascón, P. Schoppmann, B. Balle, M. Raykova, J. Doerner, S. Zahur, and D. Evans, “Secure Linear Regression on Vertically Partitioned Datasets,” _IACR Cryptol. ePrint Arch._ , vol. 2016, p. 892, 2016.
  * [45] S. Hardy, W. Henecka, H. Ivey-Law, R. Nock, G. Patrini, G. Smith, and B. Thorne, “Private Federated Learning on Vertically Partitioned Data via Entity Resolution and Additively Homomorphic Encryption,” _arXiv preprint arXiv:1711.10677_ , 2017.
  * [46] C. Dwork, “Differential Privacy: A Survey of Results,” in _Theory and Applications of Models of Computation_. Springer Berlin Heidelberg, 2008, pp. 1–19.
  * [47] A. Acar, H. Aksu, A. S. Uluagac, and M. Conti, “A Survey on Homomorphic Encryption Schemes: Theory and Implementation,” _ACM Computing Surveys_ , vol. 51, no. 4, pp. 1–35, 2018.
  * [48] D. Bogdanov, S. Laur, and J. Willemson, “Sharemind: A Framework for Fast Privacy-Preserving Computations,” in _Computer Security - ESORICS 2008_. Springer Berlin Heidelberg, 2008, pp. 192–206.
  * [49] H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang, “Learning Differentially Private Recurrent Language Models,” in _6th International Conference on Learning Representations_ , Vancouver, BC, Canada, 2018. [Online]. Available: https://openreview.net/forum?id=BJ0hF1Z0b
  * [50] S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 22, no. 10, pp. 1345–1359, 2010.
  * [51] Y. Liu, Y. Kang, C. Xing, T. Chen, and Q. Yang, “A Secure Federated Transfer Learning Framework,” _IEEE Intelligent Systems_ , vol. 35, no. 4, pp. 70–82, 2020.
  * [52] S. Saha and T. Ahmad, “Federated Transfer Learning: Concept and Applications,” _Intelligenza Artificiale_ , vol. 15, no. 1, pp. 35–44, 2021.
  * [53] R. Razavi-Far, B. Wang, M. E. Taylor, and Q. Yang, _Federated and Transfer Learning_. Cham, Switzerland: Springer, 2022.
  * [54] E. T. M. Beltrán, M. Q. Pérez, P. M. S. Sánchez, S. L. Bernal, G. Bovet, M. G. Pérez, G. M. Pérez, and A. H. Celdrán, “Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges,” _IEEE Communications Surveys & Tutorials_, vol. 25, no. 4, pp. 2983–3013, 2023.
  * [55] A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach,” in _Advances in Neural Information Processing Systems_ , vol. 33. Curran Associates, Inc., 2020, pp. 3557–3568.
  * [56] D. Xu, T. Li, Y. Li, X. Su, S. Tarkoma, T. Jiang, J. Crowcroft, and P. Hui, “Edge Intelligence: Empowering Intelligence to the Edge of Network,” _Proceedings of the IEEE_ , vol. 109, no. 11, pp. 1778–1837, 2021.
  * [57] L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Client-Edge-Cloud Hierarchical Federated Learning,” in _2020 International Conference on Communications (ICC)_ , 2020, pp. 1–6.
  * [58] J. Zhang and K. B. Letaief, “Mobile Edge Intelligence and Computing for the Internet of Vehicles,” _Proceedings of the IEEE_ , vol. 108, no. 2, pp. 246–261, 2020.
  * [59] L. U. Khan, W. Saad, Z. Han, E. Hossain, and C. S. Hong, “Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges,” _IEEE Communications Surveys & Tutorials_, vol. 23, no. 3, pp. 1759–1799, 2021.
  * [60] V. P. Chellapandi, L. Yuan, C. G. Brinton, S. H. Żak, and Z. Wang, “Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges,” _IEEE Transactions on Intelligent Vehicles_ , vol. 9, no. 1, pp. 119–137, 2023.
  * [61] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated Learning in Mobile Edge Networks: A Comprehensive Survey,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 3, pp. 2031–2063, 2020.
  * [62] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated Learning: Challenges, Methods, and Future Directions,” _IEEE Signal Processing Magazine_ , vol. 37, no. 3, pp. 50–60, 2020.
  * [63] J. Konečný, H. B. McMahan, F. X. Yu, P. Richtarik, A. T. Suresh, and D. Bacon, “Federated Learning: Strategies for Improving Communication Efficiency,” in _NIPS Workshop on Private Multi-Party Machine Learning_ , 2016. [Online]. Available: https://arxiv.org/abs/1610.05492
  * [64] W. Luping, W. Wei, and L. Bo, “CMFL: Mitigating Communication Overhead for Federated Learning,” in _39th International Conference on Distributed Computing Systems_. Dallas, Texas, USA: IEEE, 2019, pp. 954–964.
  * [65] J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor, “Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization,” in _Advances in Neural Information Processing Systems_ , vol. 33. Curran Associates, Inc., 2020, pp. 7611–7623.
  * [66] A. Imteaj, K. Mamun Ahmed, U. Thakker, S. Wang, J. Li, and M. H. Amini, _Federated and Transfer Learning_. Cham, Switzerland: Springer International Publishing, 2023, ch. Federated Learning for Resource-Constrained IoT Devices: Panoramas and State of the Art, pp. 7–27.
  * [67] A. Imteaj, U. Thakker, S. Wang, J. Li, and M. H. Amini, “A Survey on Federated Learning for Resource-Constrained IoT Devices,” _IEEE Internet of Things Journal_ , vol. 9, no. 1, pp. 1–24, 2022.
  * [68] T. Nishio and R. Yonetani, “Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge,” in _2019 International Conference on Communications (ICC)_. Shanghai, China: IEEE, 2019, pp. 1–7.
  * [69] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated Learning with Non-IID Data,” _arXiv preprint arXiv:1806.00582_ , 2018.
  * [70] F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, “Robust and Communication-Efficient Federated Learning from Non-IID Data,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 31, no. 9, pp. 3400–3413, 2019.
  * [71] K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust Aggregation for Federated Learning,” _IEEE Transactions on Signal Processing_ , vol. 70, pp. 1142–1154, 2022.
  * [72] A. Imteaj and M. H. Amini, “FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots,” in _International Conference on Machine Learning and Applications_. Miami, Florida, USA: IEEE, 2020, pp. 1153–1160.
  * [73] R. Gosselin, L. Vieu, F. Loukil, and A. Benoit, “Privacy and Security in Federated Learning: A Survey,” _Applied Sciences_ , vol. 12, no. 19, 2022.
  * [74] Y. Zhan, P. Li, S. Guo, and Z. Qu, “Incentive Mechanism Design for Federated Learning: Challenges and Opportunities,” _IEEE Network_ , vol. 35, no. 4, pp. 310–317, 2021.
  * [75] T. Zeng, O. Semiari, M. Chen, W. Saad, and M. Bennis, “Federated Learning on the Road Autonomous Controller Design for Connected and Autonomous Vehicles,” _IEEE Transactions on Wireless Communications_ , vol. 21, no. 12, pp. 10 407–10 423, 2022.
  * [76] M. Xu, J. Peng, B. B. Gupta, J. Kang, Z. Xiong, Z. Li, and A. A. A. El-Latif, “Multiagent Federated Reinforcement Learning for Secure Incentive Mechanism in Intelligent Cyber–Physical Systems,” _IEEE Internet of Things Journal_ , vol. 9, no. 22, pp. 22 095–22 108, 2022.
  * [77] Y. Deng, M. M. Kamani, and M. Mahdavi, “Adaptive Personalized Federated Learning,” 2021. [Online]. Available: https://openreview.net/forum?id=g0a-XYjpQ7r
  * [78] Y. Jiang, J. Konečnỳ, K. Rush, and S. Kannan, “Improving Federated Learning Personalization via Model Agnostic Meta Learning,” 2020. [Online]. Available: https://openreview.net/forum?id=BkeaEyBYDB
  * [79] Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, “Three Approaches for Personalization with Applications to Federated Learning,” _arXiv preprint arXiv:2002.10619_ , 2020.
  * [80] K. Wang, R. Mathews, C. Kiddon, H. Eichner, F. Beaufays, and D. Ramage, “Federated Evaluation of On-device Personalization,” _arXiv preprint arXiv:1910.10252_ , 2019.
  * [81] D. D. Siljak, _Decentralized Control of Complex Systems_. North Chelmsford, Massachusetts, US: Courier Corporation, 2011.
  * [82] G. E. Dullerud and R. D’Andrea, “Distributed control of heterogeneous systems,” _IEEE Transactions on Automatic Control_ , vol. 49, no. 12, pp. 2113–2128, 2004.
  * [83] E. Tse, C. Chong, and S. Mori, “Distributed Control for Linear Systems,” in _1983 American Control Conference_. San Francisco, California, USA: IEEE, 1983, pp. 1116–1120.
  * [84] R. D’Andrea and G. E. Dullerud, “Distributed control design for spatially interconnected systems,” _IEEE Transactions on Automatic Control_ , vol. 48, no. 9, pp. 1478–1495, 2003.
  * [85] D. Trentesaux, “Distributed control of production systems,” _Engineering Applications of Artificial Intelligence_ , vol. 22, no. 7, pp. 971–978, 2009.
  * [86] G. Baggio, D. S. Bassett, and F. Pasqualetti, “Data-driven control of complex networks,” _Nature Communications_ , vol. 12, no. 1429, 2021.
  * [87] Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabási, “Controllability of complex networks,” _Nature_ , vol. 473, pp. 167–173, 2011.
  * [88] G. Antonelli, “Interconnected dynamic systems: An overview on distributed control,” _IEEE Control Systems Magazine_ , vol. 33, no. 1, pp. 76–88, 2013.
  * [89] Y. Cao, W. Yu, W. Ren, and G. Chen, “An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination,” _IEEE Transactions on Industrial Informatics_ , vol. 9, no. 1, pp. 427–438, 2013.
  * [90] J. N. Tsitsiklis, “Problems in decentralized decision making and computation,” Ph.D. dissertation, Massachusetts Institute of Technology, 1984.
  * [91] J. Tsitsiklis, D. Bertsekas, and M. Athans, “Distributed asynchronous deterministic and stochastic gradient optimization algorithms,” _IEEE Transactions on Automatic Control_ , vol. 31, no. 9, pp. 803–812, 1986.
  * [92] R. M. Murray, “Recent Research in Cooperative Control of Multivehicle Systems,” _Journal of Dynamic Systems, Measurement, and Control_ , vol. 129, no. 5, pp. 571–583, 2007.
  * [93] L. Bakule, “Decentralized control: An overview,” _Annual Reviews in Control_ , vol. 32, no. 1, pp. 87–98, 2008.
  * [94] Q. Zhou, M. Shahidehpour, A. Paaso, S. Bahramirad, A. Alabdulwahab, and A. Abusorrah, “Distributed Control and Communication Strategies in Networked Microgrids,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 4, pp. 2586–2633, 2020.
  * [95] J. Zhao and F. Dörfler, “Distributed control and optimization in DC microgrids,” _Automatica_ , vol. 61, pp. 18–26, 2015.
  * [96] D. K. Molzahn, F. Dörfler, H. Sandberg, S. H. Low, S. Chakrabarti, R. Baldick, and J. Lavaei, “A Survey of Distributed Optimization and Control Algorithms for Electric Power Systems,” _IEEE Transactions on Smart Grid_ , vol. 8, no. 6, pp. 2941–2962, 2017.
  * [97] F. Guo, C. Wen, and Y.-D. Song, _Distributed control and optimization technologies in smart grid systems_. Boca Raton, Florida: CRC Press, 2017.
  * [98] H. Khalil and P. v. Kokotovic, “Control strategies for decision makers using different models of the same system,” _IEEE Transactions on Automatic Control_ , vol. 23, no. 2, pp. 289–298, 1978.
  * [99] D. Kalathil and R. Rajagopal, “Online learning for demand response,” in _2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)_. Monticello, Illinois, United States: IEEE, 2015, pp. 218–222.
  * [100] F. L. Lewis, H. Zhang, K. Hengster-Movric, and A. Das, _Cooperative Control of Multi-Agent Systems: Optimal and Adaptive Design Approaches_. London, Great Britain: Springer London, 2013.
  * [101] Y. Tang, Z. Ren, and N. Li, “Zeroth-order feedback optimization for cooperative multi-agent systems,” _Automatica_ , vol. 148, p. 110741, 2023.
  * [102] G. Wang, W. Yao, X. Zhang, and Z. Li, “A Mean-Field Game Control for Large-Scale Swarm Formation Flight in Dense Environments,” _Sensors_ , vol. 22, no. 14, p. 5437, 2022.
  * [103] A. G. Mutambara and H. Durrant-Whyte, “Distributed decentralized robot control,” in _Proceedings of 1994 American Control Conference - ACC ’94_ , vol. 2. Baltimore, MD, USA: IEEE, 1994, pp. 2266–2267.
  * [104] D. M. Stipanović, G. Inalhan, R. Teo, and C. J. Tomlin, “Decentralized overlapping control of a formation of unmanned aerial vehicles,” _Automatica_ , vol. 40, no. 8, pp. 1285–1296, 2004.
  * [105] O. Demir and J. Lunze, “Cooperative control of multi-agent systems with event-based communication,” in _2012 American Control Conference (ACC)_. Montreal, Québec, Candada: IEEE, 2012, pp. 4504–4509.
  * [106] S. E. Li, Y. Zheng, K. Li, Y. Wu, J. K. Hedrick, F. Gao, and H. Zhang, “Dynamical Modeling and Distributed Control of Connected and Automated Vehicles: Challenges and Opportunities,” _IEEE Intelligent Transportation Systems Magazine_ , vol. 9, no. 3, pp. 46–58, 2017.
  * [107] S. E. Li, X. Qin, Y. Zheng, J. Wang, K. Li, and H. Zhang, “Distributed Platoon Control Under Topologies With Complex Eigenvalues: Stability Analysis and Controller Synthesis,” _IEEE Transactions on Control Systems Technology_ , vol. 27, no. 1, pp. 206–220, 2019.
  * [108] Y. Zheng, S. E. Li, K. Li, F. Borrelli, and J. K. Hedrick, “Distributed Model Predictive Control for Heterogeneous Vehicle Platoons Under Unidirectional Topologies,” _IEEE Transactions on Control Systems Technology_ , vol. 25, no. 3, pp. 899–910, 2017.
  * [109] M. Noor-A-Rahim, Z. Liu, H. Lee, M. O. Khyam, J. He, D. Pesch, K. Moessner, W. Saad, and H. V. Poor, “6G for Vehicle-to-Everything (V2X) Communications: Enabling Technologies, Challenges, and Opportunities,” _Proceedings of the IEEE_ , vol. 110, no. 6, pp. 712–734, 2022.
  * [110] Z. Wang, Y. Bian, S. E. Shladover, G. Wu, S. E. Li, and M. J. Barth, “A Survey on Cooperative Longitudinal Motion Control of Multiple Connected and Automated Vehicles,” _IEEE Intelligent Transportation Systems Magazine_ , vol. 12, no. 1, pp. 4–24, 2020.
  * [111] O. Demir and J. Lunze, “A decomposition approach to decentralized and distributed control of spatially interconnected systems,” _IFAC Proceedings Volumes_ , vol. 44, no. 1, pp. 9109–9114, 2011, 18th IFAC World Congress.
  * [112] L. V. Gambuzza and M. Frasca, “Distributed Control of Multiconsensus,” _IEEE Transactions on Automatic Control_ , vol. 66, no. 5, pp. 2032–2044, 2021.
  * [113] H. Xu, S. Liu, B. Wang, and J. Wang, “Distributed-Observer-Based Distributed Control Law for Affine Nonlinear Systems and Its Application on Interconnected Cruise Control of Intelligent Vehicles,” _IEEE Transactions on Intelligent Vehicles_ , vol. 8, no. 2, pp. 1874–1888, 2023.
  * [114] L. Wang and A. S. Morse, “A Distributed Observer for a Time-Invariant Linear System,” _IEEE Transactions on Automatic Control_ , vol. 63, no. 7, pp. 2123–2130, 2018.
  * [115] A. Mitra and S. Sundaram, “Distributed Observers for LTI Systems,” _IEEE Transactions on Automatic Control_ , vol. 63, no. 11, pp. 3689–3704, 2018.
  * [116] S. Park and N. C. Martins, “Design of Distributed LTI Observers for State Omniscience,” _IEEE Transactions on Automatic Control_ , vol. 62, no. 2, pp. 561–576, 2017.
  * [117] T. Kim, H. Shim, and D. D. Cho, “Distributed Luenberger observer design,” in _2016 IEEE 55th Conference on Decision and Control (CDC)_. Las Vegas, Nevada, USA: IEEE, 2016, pp. 6928–6933.
  * [118] U. A. Khan and A. Jadbabaie, “On the stability and optimality of distributed kalman filters with finite-time data fusion,” in _Proceedings of the 2011 American Control Conference_. San Francisco, California, USA: IEEE, 2011, pp. 3405–3410.
  * [119] Z.-S. Hou and Z. Wang, “From model-based control to data-driven control: Survey, classification and perspective,” _Information Sciences_ , vol. 235, pp. 3–35, 2013.
  * [120] L. Ljung, _System Identification: Theory for the User_ , 2nd ed. Upper Saddle River, New Jersey, USA: Prentice Hall, 1991.
  * [121] A. Chiuso and G. Pillonetto, “System Identification: A Machine Learning Perspective,” _Annual Review of Control, Robotics, and Autonomous Systems_ , vol. 2, pp. 281–304, 2019.
  * [122] O. Nelles, _Nonlinear Dynamic System Identification_. Heidelberg, Germany: Springer Berlin Heidelberg, 2020.
  * [123] J. Umenberger, J. Wågberg, I. R. Manchester, and T. B. Schön, “Maximum likelihood identification of stable linear dynamical systems,” _Automatica_ , vol. 96, pp. 280–292, 2018.
  * [124] M. K. S. Faradonbeh, A. Tewari, and G. Michailidis, “Finite time identification in unstable linear systems,” _Automatica_ , vol. 96, pp. 342–353, 2018.
  * [125] T. Sarkar and A. Rakhlin, “Near optimal finite time identification of arbitrary linear dynamical systems,” in _Proceedings of the 36th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 2019, pp. 5610–5618.
  * [126] M. Simchowitz, H. Mania, S. Tu, M. I. Jordan, and B. Recht, “Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification,” in _Proceedings of the 31st Conference On Learning Theory_ , ser. Proceedings of Machine Learning Research, vol. 75. PMLR, 2018, pp. 439–473.
  * [127] S. Oymak and N. Ozay, “Non-asymptotic Identification of LTI Systems from a Single Trajectory,” in _2019 American Control Conference (ACC)_. Philadelphia, Pennsylvania, USA: IEEE, 2019, pp. 5655–5661.
  * [128] L. Xin, L. Ye, G. Chiu, and S. Sundaram, “Identifying the Dynamics of a System by Leveraging Data from Similar Systems,” in _2022 American Control Conference (ACC)_. Atlanta, Georgia, USA: IEEE, 2022, pp. 818–824.
  * [129] S. Tu, R. Frostig, and M. Soltanolkotabi, “Learning from many trajectories,” _arXiv preprint arXiv:2203.17193_ , 2022.
  * [130] E. Hazan, K. Singh, and C. Zhang, “Learning Linear Dynamical Systems via Spectral Filtering,” in _Advances in Neural Information Processing Systems_ , vol. 30. Curran Associates, Inc., 2017.
  * [131] E. Hazan, H. Lee, K. Singh, C. Zhang, and Y. Zhang, “Spectral filtering for general linear dynamical systems,” in _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [132] L. Xin, G. Chiu, and S. Sundaram, “Learning Linearized Models from Nonlinear Systems with Finite Data,” in _2023 62nd IEEE Conference on Decision and Control (CDC)_. Marina Bay Sands, Singapore: IEEE, 2023, pp. 2477–2482.
  * [133] J. Schoukens and L. Ljung, “Nonlinear System Identification: A User-Oriented Road Map,” _IEEE Control Systems Magazine_ , vol. 39, no. 6, pp. 28–99, 2019.
  * [134] G. Pillonetto, T. Chen, A. Chiuso, G. De Nicolao, and L. Ljung, _Regularized System Identification: Learning Dynamic Models from Data_. Basel, Switzerland: Springer International Publishing, 2022.
  * [135] T. Beckers, J. Seidman, P. Perdikaris, and G. J. Pappas, “Gaussian process port-hamiltonian systems: Bayesian learning with physics prior,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_ , 2022, pp. 1447–1453.
  * [136] S. M. Khansari-Zadeh and A. Billard, “Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models,” _IEEE Transactions on Robotics_ , vol. 27, no. 5, pp. 943–957, 2011.
  * [137] E. Kaiser, J. N. Kutz, and S. L. Brunton, “Sparse identification of nonlinear dynamics for model predictive control in the low-data limit,” _Proceedings of the Royal Society A_ , vol. 474, p. 20180335, 2018.
  * [138] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Sparse Identification of Nonlinear Dynamics with Control (SINDYc),” _IFAC-PapersOnLine_ , vol. 49, no. 18, pp. 710–715, 2016, 10th IFAC Symposium on Nonlinear Control Systems.
  * [139] G. Pillonetto, A. Aravkin, D. Gedon, L. Ljung, A. H. Ribeiro, and T. B. Schön, “Deep networks for system identification: a survey,” _arXiv preprint arXiv:2301.12832_ , 2023.
  * [140] C. Legaard, T. Schranz, G. Schweiger, J. Drgoňa, B. Falay, C. Gomes, A. Iosifidis, M. Abkar, and P. Larsen, “Constructing Neural Network Based Models for Simulating Dynamical Systems,” _ACM Computing Surveys_ , vol. 55, no. 11, pp. 1–34, 2023.
  * [141] J. Sjöberg, H. Hjalmarsson, and L. Ljung, “Neural Networks in System Identification,” _IFAC Proceedings Volumes_ , vol. 27, no. 8, pp. 359–382, 1994, IFAC Symposium on System Identification.
  * [142] K. S. Narendra and K. Parthasarathy, “Identification and control of dynamical systems using neural networks,” _IEEE Transactions on Neural Networks_ , vol. 1, no. 1, pp. 4–27, 1990.
  * [143] K. J. Hunt, D. Sbarbaro, R. Żbikowski, and P. J. Gawthrop, “Neural Networks for Control Systems — A Survey,” _Automatica_ , vol. 28, no. 6, pp. 1083–1112, 1992.
  * [144] A. Achille and S. Soatto, “A Separation Principle for Control in the Age of Deep Learning,” _Annual Review of Control, Robotics, and Autonomous Systems_ , vol. 1, pp. 287–307, 2018.
  * [145] B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,” _Nature Communications_ , vol. 9, p. 4950, 2018.
  * [146] J. M. Zamarreño and P. Vega, “State space neural network. Properties and application,” _Neural networks_ , vol. 11, no. 6, pp. 1099–1112, 1998.
  * [147] D. Masti and A. Bemporad, “Learning nonlinear state–space models using autoencoders,” _Automatica_ , vol. 129, p. 109666, 2021.
  * [148] G. Beintema, R. Toth, and M. Schoukens, “Nonlinear state-space identification using deep encoder networks,” in _Proceedings of the 3rd Conference on Learning for Dynamics and Control_ , ser. Proceedings of Machine Learning Research, vol. 144. PMLR, 2021, pp. 241–250.
  * [149] E. Skomski, J. Drgoňa, and A. Tuor, “Automating Discovery of Physics-Informed Neural State Space Models via Learning and Evolution,” in _Proceedings of the 3rd Conference on Learning for Dynamics and Control_ , ser. Proceedings of Machine Learning Research, vol. 144. PMLR, 2021, pp. 980–991.
  * [150] A. Chakrabarty, “Optimizing Closed-Loop Performance with Data from Similar Systems: A Bayesian Meta-Learning Approach,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_. Cancun, Mexico: IEEE, 2022, pp. 130–136.
  * [151] T. Lew, A. Sharma, J. Harrison, A. Bylard, and M. Pavone, “Safe Active Dynamics Learning and Control: A Sequential Exploration–Exploitation Framework,” _IEEE Transactions on Robotics_ , vol. 38, no. 5, pp. 2888–2907, 2022.
  * [152] A. Gu, K. Goel, and C. Ré, “Efficiently Modeling Long Sequences with Structured State Spaces,” _arXiv preprint arXiv:2111.00396_ , 2021.
  * [153] ——, “Efficiently Modeling Long Sequences with Structured State Spaces,” in _Poster presented at 10th International Conference on Learning Representations_ , Virtual Only Conference, 2022.
  * [154] A. Gu and T. Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” 2024. [Online]. Available: https://openreview.net/forum?id=AL1fq05o7H
  * [155] T. Lesort, N. Díaz-Rodríguez, J.-F. Goudou, and D. Filliat, “State representation learning for control: An overview,” _Neural Networks_ , vol. 108, pp. 379–392, 2018.
  * [156] T.-J. Chang and S. Shahrampour, “Distributed Online System Identification for LTI Systems Using Reverse Experience Replay,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_. Cancun, Mexico: IEEE, 2022, pp. 6672–6677.
  * [157] L. F. Toso, H. Wang, and J. Anderson, “Learning Personalized Models with Clustered System Identification,” in _2023 62nd IEEE Conference on Decision and Control (CDC)_. Marina Bay Sands, Singapore: IEEE, 2023, pp. 7162–7169.
  * [158] Y. Chen, A. M. Ospina, F. Pasqualetti, and E. Dall’Anese, “Multi-Task System Identification of Similar Linear Time-Invariant Dynamical Systems,” in _2023 62nd IEEE Conference on Decision and Control (CDC)_. Marina Bay Sands, Singapore: IEEE, 2023, pp. 7342–7349.
  * [159] S. Formentin and A. Chiuso, “Control-oriented regularization for linear system identification,” _Automatica_ , vol. 127, p. 109539, 2021.
  * [160] Y. Zheng and N. Li, “Non-Asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectories,” _IEEE Control Systems Letters_ , vol. 5, no. 5, pp. 1693–1698, 2021.
  * [161] M. Simchowitz, R. Boczar, and B. Recht, “Learning Linear Dynamical Systems with Semi-Parametric Least Squares,” in _Proceedings of the Thirty-Second Conference on Learning Theory_ , ser. Proceedings of Machine Learning Research, vol. 99. PMLR, 2019, pp. 2714–2802.
  * [162] L. Xin, L. Ye, G. Chiu, and S. Sundaram, “Learning dynamical systems by leveraging data from similar systems,” _arXiv preprint arXiv:2302.04344_ , 2023, submitting to IEEE Transactions on Automatic Control.
  * [163] A. Modi, M. K. S. Faradonbeh, A. Tewari, and G. Michailidis, “Joint learning of linear time-invariant dynamical systems,” _Automatica_ , vol. 164, p. 111635, 2024.
  * [164] M. Khosravi, A. Eichler, N. Schmid, R. S. Smith, and P. Heer, “Controller Tuning by Bayesian Optimization An Application to a Heat Pump,” in _2019 18th European Control Conference (ECC)_. Naples, Italy: IEEE, 2019, pp. 1467–1472.
  * [165] Y. Bao, J. M. Velni, A. Basina, and M. Shahbakhti, “Identification of State-space Linear Parameter-varying Models Using Artificial Neural Networks,” _IFAC-PapersOnLine_ , vol. 53, no. 2, pp. 5286–5291, 2020, 21st IFAC World Congress.
  * [166] M. K. Helwa and A. P. Schoellig, “Multi-robot transfer learning: A dynamical system perspective,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. Vancouver, Canada: IEEE, 2017, pp. 4702–4708.
  * [167] M. Forgione and D. Piga, “dynoNet: A neural network architecture for learning dynamical systems,” _International Journal of Adaptive Control and Signal Processing_ , vol. 35, no. 4, pp. 612–626, 2021.
  * [168] E. Arcari, A. Scampicchio, A. Carron, and M. N. Zeilinger, “Bayesian multi-task learning using finite-dimensional models: A comparative study,” in _2021 60th IEEE Conference on Decision and Control (CDC)_. Austin, Texas, USA: IEEE, 2021, pp. 2218–2225.
  * [169] C. D. McKinnon and A. P. Schoellig, “Meta Learning With Paired Forward and Inverse Models for Efficient Receding Horizon Control,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 3240–3247, 2021.
  * [170] S. Zhan, G. Wichern, C. Laughman, A. Chong, and A. Chakrabarty, “Calibrating building simulation models using multi-source datasets and meta-learned Bayesian optimization,” _Energy and Buildings_ , vol. 270, p. 112278, 2022.
  * [171] K. Hunt and D. Sbarbaro, “Neural networks for nonlinear internal model control,” in _IEE Proceedings D (Control Theory and Applications)_ , vol. 138, no. 5. IET, 1991, pp. 431–438.
  * [172] E. Grant and B. Zhang, “A neural-net approach to supervised learning of pole balancing,” in _Proceedings. IEEE International Symposium on Intelligent Control 1989_. IEEE, 1989, pp. 123–129.
  * [173] C. De Persis and P. Tesi, “Formulas for Data-Driven Control: Stabilization, Optimality, and Robustness,” _IEEE Transactions on Automatic Control_ , vol. 65, no. 3, pp. 909–924, 2020.
  * [174] ——, “Learning controllers for nonlinear systems from data,” _Annual Reviews in Control_ , p. 100915, 2023.
  * [175] F. Dörfler, J. Coulson, and I. Markovsky, “Bridging Direct and Indirect Data-Driven Control Formulations via Regularizations and Relaxations,” _IEEE Transactions on Automatic Control_ , vol. 68, no. 2, pp. 883–897, 2023.
  * [176] V. Krishnan and F. Pasqualetti, “On Direct vs Indirect Data-Driven Predictive Control,” in _2021 60th IEEE Conference on Decision and Control (CDC)_. IEEE, 2021, pp. 736–741.
  * [177] Y. Ouyang, M. Gagrani, and R. Jain, “Control of unknown linear systems with Thompson sampling,” in _2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton)_. Monticello, Illinois, United States: IEEE, 2017, pp. 1198–1205.
  * [178] A. Cohen, A. Hasidim, T. Koren, N. Lazic, Y. Mansour, and K. Talwar, “Online Linear Quadratic Control,” in _Proceedings of the 35th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, vol. 80. PMLR, 2018, pp. 1029–1038.
  * [179] A. Cohen, T. Koren, and Y. Mansour, “Learning Linear-Quadratic Regulators Efficiently with only $\sqrt{T}$ Regret,” in _Proceedings of the 36th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 2019, pp. 1300–1309.
  * [180] S. Dean, H. Mania, N. Matni, B. Recht, and S. Tu, “Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulator,” in _Advances in Neural Information Processing Systems_ , vol. 31. Curran Associates, Inc., 2018.
  * [181] S. Dean, S. Tu, N. Matni, and B. Recht, “Safely Learning to Control the Constrained Linear Quadratic Regulator,” in _2019 American Control Conference (ACC)_. Philadelphia, Pennsylvania, USA: IEEE, 2019, pp. 5582–5588.
  * [182] S. Dean, H. Mania, N. Matni, B. Recht, and S. Tu, “On the Sample Complexity of the Linear Quadratic Regulator,” _Foundations of Computational Mathematics_ , vol. 20, no. 4, pp. 633–679, 2020.
  * [183] M. Ferizbegovic, J. Umenberger, H. Hjalmarsson, and T. B. Schön, “Learning robust LQ-controllers using application oriented exploration,” _IEEE Control Systems Letters_ , vol. 4, no. 1, pp. 19–24, 2019.
  * [184] N. Agarwal, E. Hazan, and K. Singh, “Logarithmic Regret for Online Control,” in _Advances in Neural Information Processing Systems_ , vol. 32. Curran Associates, Inc., 2019.
  * [185] N. Agarwal, B. Bullins, E. Hazan, S. Kakade, and K. Singh, “Online Control with Adversarial Disturbances,” in _Proceedings of the 36th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 2019, pp. 111–119.
  * [186] E. Hazan, S. Kakade, and K. Singh, “The Nonstochastic Control Problem,” in _Proceedings of the 31st International Conference on Algorithmic Learning Theory_ , ser. Proceedings of Machine Learning Research, vol. 117. PMLR, 2020, pp. 408–421.
  * [187] M. Simchowitz, “Making Non-Stochastic Control (Almost) as Easy as Stochastic,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 18 318–18 329, 2020.
  * [188] M. Simchowitz, K. Singh, and E. Hazan, “Improper Learning for Non-Stochastic Control,” in _Proceedings of Thirty Third Conference on Learning Theory_ , ser. Proceedings of Machine Learning Research, vol. 125. PMLR, 2020, pp. 3320–3436.
  * [189] X. Chen and E. Hazan, “Black-Box Control for Linear Dynamical Systems,” in _Proceedings of Thirty Fourth Conference on Learning Theory_ , ser. Proceedings of Machine Learning Research, vol. 134. PMLR, 2021, pp. 1114–1143.
  * [190] A. Agrawal, S. Barratt, S. Boyd, and B. Stellato, “Learning Convex Optimization Control Policies,” in _Proceedings of the 2nd Conference on Learning for Dynamics and Control_ , ser. Proceedings of Machine Learning Research, vol. 120. PMLR, 2020, pp. 361–373.
  * [191] Y. Zheng, L. Furieri, M. Kamgarpour, and N. Li, “Sample Complexity of Linear Quadratic Gaussian (LQG) Control for Output Feedback Systems,” in _Proceedings of the 3rd Conference on Learning for Dynamics and Control_ , ser. Proceedings of Machine Learning Research, vol. 144. PMLR, 2021, pp. 559–570.
  * [192] L. Furieri, Y. Zheng, A. Papachristodoulou, and M. Kamgarpour, “An Input–Output Parametrization of Stabilizing Controllers: Amidst Youla and System Level Synthesis,” _IEEE Control Systems Letters_ , vol. 3, no. 4, pp. 1014–1019, 2019.
  * [193] S. Lale, K. Azizzadenesheli, B. Hassibi, and A. Anandkumar, “Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting,” in _2021 American Control Conference (ACC)_. Virtual Only Conference: IEEE, 2021, pp. 2517–2522.
  * [194] ——, “Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems,” in _Advances in Neural Information Processing Systems_ , vol. 33. Curran Associates, Inc., 2020, pp. 20 876–20 888.
  * [195] F. Sun, Z. Sun, P. Woo, and Z. Zhang, “Neural adaptive controller design of robotic manipulators with an observer,” _IFAC Proceedings Volumes_ , vol. 32, no. 2, pp. 5485–5490, 1999, 14th IFAC World Congress 1999, Beijing, Chia, 5-9 July.
  * [196] M. K. Ciliz and C. Işik, “On-line learning control of manipulators based on artificial neural network models,” _Robotica_ , vol. 15, no. 3, pp. 293–304, 1997.
  * [197] Y. Jin, T. Pipe, and A. Winfield, “Stable neural network control for manipulators,” _Intelligent Systems Engineering_ , vol. 2, no. 4, pp. 213–222, 1993.
  * [198] R. M. Sanner and J.-J. E. Slotine, “Gaussian Networks for Direct Adaptive Control,” in _1991 American Control Conference_. Boston, Massachusetts, USA: IEEE, 1991, pp. 2153–2159.
  * [199] G. Joshi and G. Chowdhary, “Adaptive Control using Gaussian-Process with Model Reference Generative Network,” in _2018 IEEE Conference on Decision and Control (CDC)_. Miami Beach, FL, USA: IEEE, 2018, pp. 237–243.
  * [200] ——, “Deep Model Reference Adaptive Control,” in _2019 IEEE 58th Conference on Decision and Control (CDC)_. Nice, France: IEEE, 2019, pp. 4601–4608.
  * [201] D. Hanover, P. Foehn, S. Sun, E. Kaufmann, and D. Scaramuzza, “Performance, Precision, and Payloads: Adaptive Nonlinear MPC for Quadrotors,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 2, pp. 690–697, 2022.
  * [202] N. Boffi, S. Tu, N. Matni, J.-J. Slotine, and V. Sindhwani, “Learning Stability Certificates from Data,” in _Proceedings of the 2020 Conference on Robot Learning_ , ser. Proceedings of Machine Learning Research, vol. 155. PMLR, 2021, pp. 1341–1350.
  * [203] D. Nguyen-Tuong, J. Peters, and M. Seeger, “Local Gaussian Process Regression for Real Time Online Model Learning,” in _Advances in Neural Information Processing Systems_ , vol. 21. Curran Associates, Inc., 2008.
  * [204] J. Harrison, A. Sharma, R. Calandra, and M. Pavone, “Control adaptation via meta-learning dynamics,” in _2nd Workshop on Meta-Learning at NeurIPS_ , 2018.
  * [205] J. Harrison, A. Garg, B. Ivanovic, Y. Zhu, S. Savarese, L. Fei-Fei, and M. Pavone, “AdaPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems,” in _Robotics Research_. Cham, Switzerland: Springer International Publishing, 2020, pp. 437–453.
  * [206] S. M. Richards, N. Azizan, J.-J. Slotine, and M. Pavone, “Adaptive-control-oriented meta-learning for nonlinear systems,” in _Robotics: Science and Systems_ , Virtual Only Conference, 2021. [Online]. Available: https://roboticsconference.org/2021/program/papers/056/index.html
  * [207] ——, “Control-oriented meta-learning,” _The International Journal of Robotics Research_ , vol. 42, no. 10, pp. 777–797, 2023.
  * [208] D. Muthirayan, D. Kalathil, and P. P. Khargonekar, “Meta-Learning Online Control for Linear Dynamical Systems,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_. Cancun, Mexico: IEEE, 2022, pp. 1435–1440.
  * [209] L. Li, C. De Persis, P. Tesi, and N. Monshizadeh, “Data-Based Transfer Stabilization in Linear Systems,” _IEEE Transactions on Automatic Control_ , vol. 69, no. 3, pp. 1866–1873, 2024.
  * [210] J. Berberich, C. W. Scherer, and F. Allgöwer, “Combining Prior Knowledge and Data for Robust Controller Design,” _IEEE Transactions on Automatic Control_ , vol. 68, no. 8, pp. 4618–4633, 2023.
  * [211] G. Shi, K. Azizzadenesheli, M. O’Connell, S.-J. Chung, and Y. Yue, “Meta-Adaptive Nonlinear Control: Theory and Algorithms,” in _Advances in Neural Information Processing Systems_ , vol. 34. Curran Associates, Inc., 2021, pp. 10 013–10 025.
  * [212] T. T. Zhang, K. Kang, B. D. Lee, C. Tomlin, S. Levine, S. Tu, and N. Matni, “Multi-Task Imitation Learning for Linear Dynamical Systems,” in _Proceedings of The 5th Annual Learning for Dynamics and Control Conference_ , ser. Proceedings of Machine Learning Research, vol. 211. PMLR, 2023, pp. 586–599.
  * [213] Y. Sun, W. L. Ubellacker, W.-L. Ma, X. Zhang, C. Wang, N. V. Csomay-Shanklin, M. Tomizuka, K. Sreenath, and A. D. Ames, “Online Learning of Unknown Dynamics for Model-Based Controllers in Legged Locomotion,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 4, pp. 8442–8449, 2021.
  * [214] A. Rajeswaran, S. Ghotra, B. Ravindran, and S. Levine, “EPOpt: Learning Robust Neural Network Policies Using Model Ensembles,” in _5th International Conference on Learning Representations_. Toulon, France: OpenReview.net, 2017. [Online]. Available: https://openreview.net/forum?id=SyWvgP5el
  * [215] W. Yu, J. Tan, Y. Bai, E. Coumans, and S. Ha, “Learning Fast Adaptation With Meta Strategy Optimization,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 2950–2957, 2020.
  * [216] C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine, “Learning modular neural network policies for multi-task and multi-robot transfer,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. Marina Bay Sands, Singapore: IEEE, 2017, pp. 2169–2176.
  * [217] L. F. Toso, D. Zhan, J. Anderson, and H. Wang, “Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for the Model-free LQR,” _arXiv preprint arXiv:2401.14534_ , 2024.
  * [218] H. Wang, L. F. Toso, and J. Anderson, “FedSysID: A Federated Approach to Sample-Efficient System Identification,” in _Proceedings of The 5th Annual Learning for Dynamics and Control Conference_ , ser. Proceedings of Machine Learning Research, vol. 211. PMLR, 2023, pp. 1308–1320.
  * [219] I. A. Azzollini, M. Bin, L. Marconi, and T. Parisini, “Robust and scalable distributed recursive least squares,” _Automatica_ , vol. 158, p. 111265, 2023.
  * [220] Z. Ren, A. Zhong, and N. Li, “LQR with Tracking: A Zeroth-order Approach and Its Global Convergence,” in _2021 American Control Conference (ACC)_ , Virtual Only Conference, 2021, pp. 2562–2568.
  * [221] H. Wang, L. F. Toso, A. Mitra, and J. Anderson, “Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approach,” _arXiv preprint arXiv:2308.11743_ , 2023.
  * [222] T. Zeng, O. Semiari, M. Chen, W. Saad, and M. Bennis, “Federated Learning for Collaborative Controller Design of Connected and Autonomous Vehicles,” in _2021 60th IEEE Conference on Decision and Control (CDC)_. Austin, Texas, USA: IEEE, 2021, pp. 5033–5038.
  * [223] L. Wang, K. Zhang, A. Zhou, M. Simchowitz, and R. Tedrake, “Robot Fleet Learning via Policy Merging,” in _12th International Conference on Learning Representations_ , Vienna, Austria, 2024.
  * [224] H. Shiri, J. Park, and M. Bennis, “Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory,” _IEEE Transactions on Communications_ , vol. 68, no. 11, pp. 6840–6857, 2020.
  * [225] M. Nakanoya, J. Im, H. Qiu, S. Katti, M. Pavone, and S. Chinchali, “Personalized federated learning of driver prediction models for autonomous driving,” _arXiv preprint arXiv:2112.00956_ , 2021.
  * [226] H.-K. Lim, J.-B. Kim, J.-S. Heo, and Y.-H. Han, “Federated Reinforcement Learning for Training Control Policies on Multiple IoT Devices,” _Sensors_ , vol. 20, no. 5, p. 1359, 2020.
  * [227] B. Liu, L. Wang, and M. Liu, “Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 4, pp. 4555–4562, 2019.
  * [228] B. Liu, L. Wang, M. Liu, and C.-Z. Xu, “Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems With Heterogeneous Sensor Data,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 3509–3516, 2020.
  * [229] X. Liang, Y. Liu, T. Chen, M. Liu, and Q. Yang, “Federated transfer reinforcement learning for autonomous driving,” in _Federated and Transfer Learning_. Cham, Switzerland: Springer, 2022, pp. 357–371.
  * [230] L. Mächler, I. Ezhov, F. Kofler, S. Shit, J. C. Paetzold, T. Loehr, C. Zimmer, B. Wiestler, and B. H. Menze, “FedCostWAvg: A New Averaging for Better Federated Learning,” in _Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries_ , Springer. Virtual Event: Springer International Publishing, 2022, pp. 383–391.
  * [231] A. B. Mansour, G. Carenini, A. Duplessis, and D. Naccache, “Federated Learning Aggregation: New Robust Algorithms with Guarantees,” in _2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA)_. Atlantis Hotel, Bahamas: IEEE, 2022, pp. 721–726.
  * [232] L. Mächler, I. Ezhov, S. Shit, and J. C. Paetzold, “FedPIDAvg: A PID Controller Inspired Aggregation Method for Federated Learning,” in _International MICCAI Brainlesion Workshop_ , Springer. Cham, Switzerland: Springer Nature Switzerland, 2023, pp. 209–217.
  * [233] H. Gao, Q. Wu, X. Zhao, J. Zhu, and M. Zhang, “FedADT: An Adaptive Method Based on Derivative Term for Federated Learning,” _Sensors_ , vol. 23, no. 13, p. 6034, 2023.
  * [234] A. H. Sayed, “Adaptive networks,” _Proceedings of the IEEE_ , vol. 102, no. 4, pp. 460–497, 2014.
  * [235] L. Nie, J. Guan, C. Lu, H. Zheng, and Z. Yin, “Longitudinal speed control of autonomous vehicle based on a self-adaptive PID of radial basis function neural network,” _IET Intelligent Transport Systems_ , vol. 12, no. 6, pp. 485–494, 2018.
  * [236] D. Bertsekas, _Reinforcement learning and optimal control_. Nashua, New Hampshire, USA: Athena Scientific, 2019.
  * [237] S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” _arXiv preprint arXiv:2005.01643_ , 2020.
  * [238] M. Vidyasagar, “A tutorial introduction to reinforcement learning,” _SICE Journal of Control, Measurement, and System Integration_ , vol. 16, no. 1, pp. 172–191, 2023.
  * [239] Quanser. QUBE – Servo 2. [Online]. Available: https://www.quanser.com/products/qube-servo-2/
  * [240] S. Khodadadian, P. Sharma, G. Joshi, and S. T. Maguluri, “Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling,” in _Proceedings of the 39th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, vol. 162. PMLR, 2022, pp. 10 997–11 057.