# A Design Toolbox for the Development of Collaborative Distributed Machine
Learning Systems

David Jin1, Niclas Kannengießer12, Sascha Rank12, Ali Sunyaev12

1Karlsruhe Institute of Technology, Germany  2KASTEL Security Research Labs,
Germany  {david.jin, niclas.kannengiesser, sascha.rank<EMAIL_ADDRESS>
###### Abstract

To leverage training data for the sufficient training of ML models from
multiple parties in a confidentiality-preserving way, various collaborative
distributed machine learning (CDML) system designs have been developed, for
example, to perform assisted learning, federated learning, and split learning.
CDML system designs show different traits, for example, high agent autonomy,
machine learning (ML) model confidentiality, and fault tolerance. Facing a
wide variety of CDML system designs with different traits, it is difficult for
developers to design CDML systems with traits that match use case requirements
in a targeted way. However, inappropriate CDML system designs may result in
CDML systems failing their envisioned purposes. We developed a CDML design
toolbox that can guide the development of CDML systems. Based on the CDML
design toolbox, we present CDML system archetypes with distinct key traits
that can support the design of CDML systems to meet use case requirements.

###### Index Terms:

collaborative distributed machine learning (CDML), privacy-enhancing
technologies (PETs), assisted learning, federated learning (FL), split
learning, swarm learning, multi-agent systems (MAS).

##  I Introduction

The training of machine learning (ML) models requires sufficient training data
in terms of quantity and quality to make meaningful predictions with little
generalization error. Sufficient training data is, however, seldom available
from a single party (e.g., a bank or a hospital), which can prevent the
adequate training of ML models [1]. Inadequate training of ML models can
result in large generalization errors, rendering ML models ineffective [2].

To reduce generalization errors of ML models, developers request training data
from multiple third parties. Training data retrievals from third parties are
often subject to compliance, social, and technical challenges [3, 4, 5] that
hinder the acquisition of sufficient training data. For example, strict data
protection laws and regulations prohibit the disclosure of specific kinds of
data, such as personal data by the General Data Protection Regulation of the
European Union [6] and organizational data by the Healthcare Insurance
Portability and Accountability Act of the USA [7]. From a social perspective,
privacy behaviors of individuals restrict information flows to third parties
based on personal preferences [8], preventing access to their training data.
Insufficient computing resources inhibit the transfer of large data sets from
data centers to developers in an acceptable time [3, 4]. To reduce
generalization errors of ML models by using training data from multiple
parties, an ML paradigm is required that solves those challenges.

Collaborative distributed ML (CDML) is an ML paradigm that can be implemented
to overcome, in particular, compliance and technical challenges in using data
from multiple parties to train ML models [9, 10, 11, 12, 13, 14]. In CDML
systems, such as federated learning systems [10], split learning systems [11],
and swarm learning systems [14], each party operates at least one quasi-
autonomous agent (referred to as agent in the following). Agents in CDML
systems train (parts of) ML models on their local training data and self-
controlled compute in a distributed manner. Agents only share their locally
computed training results (interim results) with other agents, for example,
gradients [15], activations [11], and (pseudo-)residuals [12]. Reconstructing
training data from interim results is commonly difficult [9]. Using interim
results received from other agents, agents improve their local (parts of) ML
models. Following the CDML paradigm, parties can keep control over their
training data, which can help solve compliance challenges. Moreover, CDML can
help to solve technical challenges because large packages of training data are
not transferred to single parties to train ML models, saving bandwidth.
Moreover, computational resources for training ML models are distributed
across multiple agents, which decreases the amount of computational resources
a single party must possess to train ML models.

The potential of CDML to leverage large training data quantities in a
confidentiality-preserving and resource-efficient way has sparked enormous
interest in practice and research for various use cases with different
requirements for CDML systems. For instance, effective next-word prediction in
virtual smartphone keyboards requires language models to be trained on a large
quantity of heterogeneous training data representative of future user inputs.
To meet this requirement, CDML systems must be scalable to involve millions
[16] or even billions of agents [10]. Another CDML use case is the prediction
of financial risks in portfolio management [17, 18]. Financial institutions
rely on ML models to predict investment risks in portfolio management. As
customers pay for portfolio management, such ML models are core assets to
financial institutions. To protect such core assets, CDML systems must enable
collaborative training of ML models without disclosing ML models to
competitors.

To meet different use case requirements, practice and research developed
specialized CDML system designs. For instance, federated learning systems are
scalable to engage billions of agents to train ML models for next-word
prediction [19]. Assisted learning systems are unsuitable for this purpose due
to the sequential processing of interim results [17]. Conversely, assisted
learning seems to be suitable for training ML models for portfolio management
because ML model confidentiality is protected in the learning process.
Federated learning requires agents to disclose ML models and, thus, is
unsuitable for use cases requiring ML model confidentiality. Developers need
to understand how envisioned traits of CDML systems (e.g., high scalability,
ML model confidentiality) can be achieved by designing CDML systems in a
targeted manner.

The proliferation of a wide variety of specialized CDML system designs
introduced a large number of design options (e.g., regarding the structure of
interim results and the parts of ML models disclosed to other agents) that
constitute the CDML system design space. Developers must select and combine
design options from the CDML system design space to design CDML systems with
traits that meet use case requirements (e.g., high scalability, ML model
confidentiality, and high robustness of the training process). The targeted
selection and combination of design options requires developers to thoroughly
understand the CDML system design space and traits arising from the
implementation of design options in CDML systems. An insufficient
understanding of the CDML system design space can lead developers to select
design options that can cause CDML systems to fail their purposes, for
example, when ML models for portfolio management are inadvertently leaked in
unsuitable training processes. However, literature on CDML systems is
scattered, which is why the CDML system design space remains unclear and,
thus, how envisioned key traits can be achieved through targeted CDML system
designs. To support the targeted design of CDML systems suitable for use
cases, we ask the following research questions:

RQ1: What does the CDML system design space look like?

RQ2: What are the key traits of principal CDML system designs?

To answer our research questions, we applied a three-step research approach.
First, we developed the CDML design toolbox, which is a conceptualization of
the CDML system design space. The CDML design toolbox specifies the
fundamentals of CDML systems (e.g., agent roles and their interactions) and
design options for the customization of CDML systems (e.g., combinations of
agent roles in single agents, communication paths between agents, and types of
interim results). For the conceptualization, we analyzed literature on CDML
and developed agent-based models in the schemes presented in the Gaia
methodology [20]. These schemes are commonly used to develop agent-based
models that can serve as blueprints for implementing distributed software
systems, such as CDML systems. Then, we tested the validity of the CDML design
toolbox by modeling CDML system designs using the CDML design toolbox. Second,
we developed CDML archetypes based on commonalities and differences between
the modeled CDML systems. Third, we reviewed publications on CDML system
designs to extract key traits of the CDML archetypes.

This work has three principal contributions to practice and research. First,
by presenting the CDML design toolbox, we offer a consolidated design
knowledge base of CDML systems that introduces the main design commonalities
of CDML systems and offers design options for the customization of CDML system
designs to meet use case requirements. This consolidation of previously
scattered design knowledge in agent-based models (e.g., the roles model, the
interactions model) facilitates the application of the Gaia methodology for
systematically designing custom CDML systems. Moreover, by presenting design
options implemented in CDML system designs, the CDML design toolbox helps to
compare CDML system designs systematically. Second, by showcasing CDML
archetypes, we inform of combinations of design options commonly used in
practice and research. The CDML archetypes can be refined to develop
blueprints of CDML systems tailored to use cases using the CDML design
toolbox, which facilitates designing CDML systems. Third, by presenting key
traits of CDML archetypes, we support developers in understanding how design
options can be leveraged to achieve specific key traits. By using the CDML
archetypes and their key traits, developers are enabled to evaluate CDML
system designs in their suitability for use cases before implementing the
designs. Thereby, we support the targeted design of CDML systems for use
cases.

The remainder of this work is structured into six sections. First, we explain
the foundations of CDML, related research on CDML systems, and introduce basic
concepts of multi-agent systems (MAS). Second, we describe how we developed
the CDML design toolbox, including a brief introduction to the Gaia
methodology [20]. Moreover, we describe how we developed CDML archetypes using
the CDML design toolbox and how we identified their key traits. Third, we
present the CDML design toolbox. Fourth, we describe CDML archetypes and
explain how different combinations of design options can lead to key traits of
CDML systems. Fifth, we discuss our principal findings and describe the
contributions and limitations of this work. Moreover, we give an outlook for
future research directions. We conclude with a brief summary of this work and
our personal takeaways.

##  II Background and Related Research

###  II-A Collaborative Distributed Machine Learning

CDML combines the ML approaches of collaborative ML (CML) and distributed ML
(DML). Leveraging training data from various parties is the focus of CML [21,
22, 23]. In CML systems, training data from multiple parties is used in a
centralized or siloed way. In centralized CML, agents send their local data to
a central data server that various agents can access to train ML models using
the shared training data. To preserve training data confidentiality, data may
only be provided to the central data server in encrypted form. The used
cryptographic techniques (e.g., homomorphic encryption [23, 24]) allow agents
to train ML models on the encrypted data while the plain training data remains
confidential. However, the cryptographic techniques will likely lead the
centrally controlled computing system to consume more resources for the ML
model training [25]. Overall, agents in centralized CML depend on central data
servers. Crashes of central data servers can lead such CML systems to failure.

Distributed ML was developed to accelerate the training of large ML models,
such as deep learning models, by distributing training tasks to multiple
agents that train (parts of) ML models in parallel. Distributed ML systems can
train ML models in two ways [26, 27, 28]: data parallel and model parallel. In
data parallel training, partitions of the entire training data set are passed
to agents. Each agent trains the same ML model on individual subsets of the
whole training data set. In model parallel training, each agent uses identical
data but only trains a part of the ML model.

In preparation for DML, training data is usually gathered by a central party
that sets up the DML system (e.g., in computing clusters). The central party
then identically distributes the gathered training data across agents to
achieve a uniform workload for each. Through the uniform workload
distribution, idle times of agents are aimed to be low so that the ML model
training is performed with high computational efficiency [26].

The training process in DML is often coordinated by a central server, called
parameter server [29, 30, 31]. After the local training of the ML model,
agents transmit their ML model updates to the parameter server. The parameter
server stores ML model updates and offers the latest parameters to agents.
Agents fetch the parameters to proceed with the local training of the ML
model.

An alternative to using parameter servers in DML is all-reduce [28, 32, 33].
In all-reduce, all agents have similar roles, thus executing identical tasks.
The identical execution of tasks by all agents makes central parameter servers
obsolete. Each agent aggregates training results and distributes them to other
agents in the DML system. Any agent is notified about ML model updates to
proceed with the local training of the latest version of ML models.

In summary, CML centers on the sharing and collaborative use of training data,
while DML centers on performance improvements in training ML models. However,
DML hardly contributes to overcoming the legal and social challenges related
to leveraging training data from multiple parties in a confidentiality-
preserving way.

The combination of principles of CML (e.g., leveraging training data from
various parties) and DML (e.g., the distributed execution of ML tasks across
multiple agents) forms the foundation for CDML. In CDML systems, trainer
agents receive ML tasks from other agents and use local training data to
accomplish ML tasks. ML tasks specify the objectives pursued with ML models
(e.g., next-word prediction) and include information about the approach (e.g.,
what ML model architecture should be used). This approach can implement DML
techniques, which can eventually speed up the training process by
parallelization. However, because the training data is usually unknown to
participants in the ML system, identical distribution of training data, like
in purely DML, is hard to achieve. Thus, the performance benefits targeted in
DML systems may not be fully leveraged [34].

###  II-B Related Research on CDML

As one of the first CDML concepts, federated learning has established training
data confidentiality and distributed computing as a fundamental goal pursued
when applying the CDML paradigm [35, 10, 16]. Soon after its introduction,
various shortcomings of federated learning became apparent. For example,
federated learning systems have been shown to be inefficient due to high
communication costs [9] and prone to performance bottlenecks caused by the use
of a central parameter server [36]. From a security perspective, federated
learning systems are prone to failures due to an adversarial central parameter
server [9].

To tackle the shortcomings of federated learning, practice and research
brought forth other CDML concepts, including swarm learning, split learning,
and assisted learning. Like federated learning, swarm learning aims at the
collaborative and distributed training of global ML models known to all
parties involved in the training process. Other than federated learning
systems, swarm learning systems rely on redundant agents orchestrating the
training process in peer-to-peer networks [14]. The redundant execution of
tasks in swarm learning systems can make swarm learning systems more robust
than federated learning systems [14]. However, the strong redundancies render
swarm learning systems usually less resource-efficient and more complex
compared to federated learning systems.

In split learning systems [11], agents only train parts of ML models defined
by a so-called cut layer. Cut layers indicate the layers of neural networks
where the complete neural network is split. Agents only receive the
specifications of the cut layer as a kind of interface to input parameters for
the training of the rest of the ML model. By only disclosing parts of ML
models specified by cut layers, split learning helps to keep (at least parts
of) ML models confidential. However, the gain in ML model confidentiality in
split learning systems comes at the cost of the training performance of ML
models compared to federated learning [37].

In assisted learning [12], the focus on preserving the confidentiality of
training data is extended to ML models and even the purposes of ML models. In
assisted learning, a user agent requests feedback on statistics relevant to
training an ML model from service agents. Such feedback can include residuals
of its own ML model. The user agent incorporates feedback received from
service agents into its local ML model. This process can be executed
repeatedly until the ML model reaches sufficient prediction performance. By
enabling agents to decide which agents they want to assist, assisted learning
can improve the autonomy of agents. However, the increased autonomy comes with
coordination challenges, for example, how to assess the potential of agents to
assist in a learning task and in which order agents interact [38].

Various design options for customizing CDML systems to meet use case
requirements have been developed, such as federated learning systems with
multiple hierarchical levels. In each hierarchical level, a preprocessing of
previous training results is executed by aggregating a subset of training
results. The global ML model is then computed from multiple aggregated
training results [10]. Another design option for federated learning systems is
to form subnetworks to deal with heterogeneous computational resources between
trainer agents [39]. Agents with more computing resources (e.g., servers)
execute training tasks that consume more computational resources than agents
with only a few (e.g., smartphones).

Extant research has started to compare CDML systems to understand their
commonalities and differences. Such comparisons are often based on benchmarks,
for example, between systems of federated learning, split learning, and
SplitFed learning [13] and between systems of federated learning, swarm
learning, and decentralized federated learning [40]. CDML system benchmarks
commonly offer valuable help in understanding likely CDML system behaviors,
especially in terms of performance (e.g., convergence speed of ML models [13],
communication cost [13] and prediction performance [40]). Such benchmarks can
support practitioners in meeting performance requirements for CDML systems.
However, benchmark results are only helpful at a limited scale to understand
possible CDML system designs and their key traits, as they seldom explain how
CDML system designs lead to different system behaviors. Moreover, benchmark
studies only shed light on a few CDML system designs, leaving the entirety of
the CDML system design space unknown.

Other works compare CDML system designs. Several design options for federated
learning systems were revealed, describing different network topologies for
communication (e.g., via central servers and peer-to-peer) and computational
schedules [3], such as sequential training of ML models and parallel training
synchronized by a central server. Key traits that originate from the different
design options are discussed with a focus on confidentiality. Design
differences between other CDML systems (e.g., assisted learning systems and
split learning systems) remain unknown. In a comparison between federated
learning systems, split learning systems, and SplitFed learning systems [13],
key traits of those CDML systems are pointed out, with a focus on learning
performance, resource consumption, ML model confidentiality, and training data
confidentiality. Despite these valuable insights, several design options
(e.g., regarding the network topology and computational schedules) and their
influences on key traits of CDML systems remain unclear.

Since extant comparisons focus only on selected systems of a few CDML
concepts, it is still hard to understand the entirety of the CDML system
design space. To help developers design CDML systems that meet use case
requirements, the CDML system design space must be understood, including the
various CDML concepts, design options, and key traits of CDML system designs.
This knowledge of the CDML system design space needs to become available in
actionable form.

###  II-C Multi-Agent Systems

The multi-agent system (MAS) concept [41] offers a theoretical lens to model
systems based on agents (e.g., computing nodes) and their interactions in a
specified environment [20, 42]. The MAS concept is widely used in computer
science to model hardware systems and software systems, especially in the
field of artificial intelligence (AI) systems [43, 44]. Since the MAS concept
is established to develop blueprints of systems for their implementation [20,
45, 46], it seems to be adequate to represent the CDML system design space in
a CDML design toolbox that helps to design, analyze, and advance CDML systems.
In the following, we introduce the basic properties of the MAS concept
relevant to this work. Important MAS properties are summarized in Table I.

TABLE I: MAS properties and corresponding characteristics relevant in this work (adapted from [42]) Property |  Description |  Characteristic
---|---|---
Cardinality |  The number of objects and agents that are part of the MAS |  Finite
Infinite
Coalition Control |  The design of authority in the MAS |  Centralized
Decentralized
Goal Structure |  The number of goals in the MAS |  Multiple
Single
Interaction |  The way how agents work with other agents to achieve their goal(s) |  Competitive
Collaborative
Cooperative
Openness |  The possibilities to enter and leave the system |  Closed
Open
Population Diversity |  The presence of different types of agents |  Heterogeneous
Homogeneous

MASs are systems comprised of a population of agents. By design, MASs can
limit the population to a finite number of agents or allow an infinite number
of agents. Within MASs, agents can form groups, so-called coalitions.
Coalitions can comprise entire MAS populations or population partitions.
Agents can be part of multiple coalitions at the same time [20, 42]. We
consider each CDML system as a coalition within a superordinate MAS. As agents
can be part of multiple coalitions, agents can simultaneously participate in
multiple CDML systems.

Coalitions can be controlled in a centralized or decentralized way. In
centralized coalition control, a single or a few agents coordinate
interactions between agents in the coalition, for example, in federated
learning systems [16]. In decentralized coalition control, multiple or even
all agents have equitable influences on the coordination of the coalition.

In coalitions, there are two common goal structures. Agents can pursue
individual goals or common goals. Since agents can be part of multiple
coalitions, agents can pursue multiple goals at the same time. For example, an
agent may pursue an individual goal in one coalition (e.g., training its own
ML model in an assisted learning system) and a common goal in another
coalition (e.g., training a shared ML model in a swarm learning system).

Agents can have different kinds of interaction to reach their goals in
coalitions. They can act in a competitive, cooperative, and independent
manner. When agents compete with each other, they need to fight for scarce
resources to accomplish their tasks. Cooperative agents support each other in
the accomplishment of common goals, where individual agents (or subgroups of
agents) work on different tasks. In federated learning systems, for example,
some agents only train ML models, while other agents aggregate interim
training results [47, 16]. When agents collaborate, each agent is involved in
each task to accomplish shared goals. Swarm learning systems are mostly
collaborative, as most agents perform similar tasks in the ML model training
[14].

MASs and coalitions can differ in their openness to allowing agents to join
and leave arbitrarily. Closed MASs only allow specified agents to join. In
some federated learning systems, only selected agents are permitted to join
the coalitions [10]. Open MASs allow agents to join and leave arbitrarily, for
example, in many peer-to-peer learning systems [48, 49].

Population diversity refers to the heterogeneity of agent types in a
population. Agent types are sets of roles that are assigned to agents to
specify their tasks in a coalition [20]. If many agents in a population have
largely different agent types, the population is heterogeneous. For example,
hierarchical federated learning systems comprise up to four different agent
types that collaborate and execute different tasks in the training of ML
models. If most agents have identical agent types, the population is
homogeneous. Swarm learning systems, for example, can be considered
homogeneous because all agents execute identical tasks in the training of ML
models [14].

##  III Methods

We applied a three-step research approach to conceptualize the CDML design
space (RQ1) and extract key traits of CDML systems originating from different
designs (RQ2). First, we conceptualized CDML systems described in literature
(Section III-A). Based on the conceptualization, we developed the CDML design
toolbox. We modeled CDML systems using the CDML design toolbox to test its
applicability. Second, we used the models of the CDML systems to develop CDML
archetypes (Section III-B). Third, we extracted traits of CDML system designs
from literature. We assigned the CDML system designs, including their traits,
to the CDML archetypes and aggregated the traits to key traits (see Section
III-C). In the following, we describe our methods in detail.

###  III-A CDML Design Toolbox Development

To develop the CDML design toolbox, we adopted the Gaia methodology for agent-
oriented modeling [20]. Using the structures of the five agent-based models
presented in the Gaia methodology (see Section III-A1), we conceptualized CDML
systems presented in the literature by applying open coding, axial coding, and
selective coding [50] as described in Section III-A2. The literature analysis
revealed design options for CDML systems (e.g., agent role distributions,
optional communication paths, and structures of training processes). We tested
and refined our coding in three iterations by classifying CDML systems into
our coding (see Section III-A3).

####  III-A1 The Gaia Methodology

One main purpose of the Gaia methodology is to support the development of
agent-based models that can serve as blueprints for the implementation of
software systems [20]. The Gaia methodology is constituted of an analysis
stage and a design stage. In the analysis stage, a roles model and an
interactions model are developed, enabling an abstract view of a system. This
abstract view constitutes the concept level of the system description that
enables an analysis of system structures. The roles model describes the tasks
and basic processes, including the resources that agents can use. Roles
essentially describe the functions that an agent performs within the system.
Each role consists of four main aspects: responsibilities, permissions,
activities, and protocols. Responsibilities define the functions an agent of a
particular role needs to perform. An exemplary responsibility of an agent in
the role of an updater in CDML systems could be the aggregation of ML models
trained by other agents into a global ML model. Permissions describe which
resources are available to agents with specific roles to fulfill their
responsibilities. Exemplary resources for agents in the role of updater are
information about the ML model to be trained and local training data.
Activities are computations that agents perform locally without interaction
with other agents. In the case of agents in the trainer role, local training
of an ML model is an exemplary activity. Protocols as part of the roles model
reference protocol definitions in the interactions model that describe how
interactions between agents of specific roles are designed. For example,
updater agents must interact with agents with the trainer role to retrieve
interim training results and complete the training process.

The interactions model specifies how agents with specific roles interact with
each other in a purposeful way. Frequently recurring interactions between
agents with other agents, objects, or the environment of the MAS are recorded
as interaction patterns. Each interaction pattern is described in a protocol
definition. Protocol definitions include six attributes: purpose, initiator,
interactor, input, output, and processing. The purpose includes a textual
description of the meaning of interaction, for example, “passing an ML model
for its training”. Interactions originate from an agent (i.e., an initiator)
and are directed to an interaction partner (i.e., a responder). For
interaction, the initiator prepares an input and issues the input into the
interaction process. The output comprises the information received by the
responder at the end of the interaction.

Based on the roles model and the interactions model, envisioned CDML systems
can be detailed in the design stage of the Gaia methodology. The design stage
centers on the development of an agent model, a service model, and an
acquaintance model. These models form the design level of the system
representation. In combination, the concept level and the design level form
blueprints for the implementation of concrete software systems [20].

The agent model describes the agent types utilized by CDML systems. Agent
types are combinations of roles. Moreover, the agent model describes instances
of these agent types that will populate the CDML system. The service model
describes the main services that are necessary to implement an agent role. The
services that an agent can execute depend on its roles and corresponding
activities and protocols. The acquaintance model describes communication paths
between different agent types in the CDML system. The acquaintance model helps
to identify communication bottlenecks that may arise during run-time.

Similar to the structure of the Gaia methodology, the CDML design toolbox
comprises an abstract concept level and a more detailed design level. The
concept level describes the general design of CDML systems, focusing on their
commonalities (e.g., roles and interactions). On the design level, the CDML
design toolbox describes design options to customize and differentiate CDML
systems.

####  III-A2 Conceptualization of CDML Systems

To develop the CDML design toolbox, we conceptualized CDML systems in three
steps: start set compilation, development of an initial version of the CDML
design toolbox, and test and iterative refinement. We describe the three steps
in more detail in the following.

##### Start Set Compilation

For the development of the CDML design toolbox, we first compiled a start set
constituted of publications on CDML systems. To systematize the search for
potentially relevant publications, we specified the following inclusion
criteria (see Table II): English language, level of detail, topic fit, and
uniqueness. We excluded publications from the start set that did not meet all
inclusion criteria.

TABLE II: Criteria to be fulfilled for the inclusion of publications on CDML systems in our literature analysis Name |  Descriptions
---|---
English Language |  The publication must be in English.
Level of Detail |  The publication must present enough details to understand the design and functioning of CDML systems.
Topic Fit |  The publication must describe at least one CDML system design.
Uniqueness |  The publication must not be included in the set of relevant literature.

After specifying the inclusion criteria, each author independently generated
their own set of publications potentially relevant to developing the CDML
design toolbox. We searched for publications that cover a large variety of
CDML systems and offer detailed descriptions of CDML system designs. Then, we
consolidated the independently generated sets of publications into a
preliminary start set. The preliminary start set included peer-reviewed
scientific publications and grey literature. Next, we applied the inclusion
criteria to the publications in the preliminary start set (see Table II). We
removed one publication from the preliminary set of relevant literature
because it was a duplicate. Based on the full texts of the remaining 29
publications, we independently rated the relevance of each publication for the
conceptualization as “relevant”, “maybe relevant”, and “irrelevant” based on
the inclusion criteria (see Table II). Whenever we were at variance regarding
the relevance of publications (e.g., when one author felt the level of detail
of a publication was sufficient and another author disagreed), we discussed
the relevance of the publication in more detail until we concluded with
unanimous decisions to include or exclude the publication from the preliminary
start set. This relevance assessment led us to exclude 18 further publications
from the preliminary start set. The final start set included eleven
publications to be analyzed for the development of the initial version of the
conceptualization.

##### Development of an Initial Version of the CDML Design Toolbox

We analyzed the publications in the start set by applying open, axial, and
selective coding [50]. In open coding, we extracted aspects of CDML systems
relevant to explain their designs and functioning. After coding the literature
in the set of relevant publications, we iteratively refined our coding to
achieve mutual exclusiveness between our codes and the exhaustiveness of our
coding. For example, we merged the codes “client” and “device” into the code
“trainer” and the codes “sendParameters” and “sendGradients” into the code
“transmitInterimResult”.

In axial coding, we extracted relationships between the codes developed in
open coding. For example, we identified that the code “transmitInterimResult”
can be implemented differently. We coded each implementation (e.g.,
“activations” and “gradients”) and noted the relationship between
“transmitInterimResult” and “gradients”.

In selective coding, we classified the extracted codes into coding schemes.
The coding schemes correspond to five agent-oriented models (i.e., the roles
model, the interactions model, the agent model, the preliminary service model,
and the acquaintance model) introduced in the Gaia methodology [20]. For
example, we classified the code “trainer” as a role in the roles model and the
code “transmitInterimResult” as a protocol in the interactions model.

After the analysis, we refined the coding to improve the mutual exclusiveness
between codes and the exhaustiveness of our coding. For example, we abstracted
the code “aggregator” to “updater” to include DCML systems in which the ML
model is updated with and without aggregating interim results.

####  III-A3 Test and Iterative Refinement

We gathered evidence for the external validity of our CDML design toolbox by
testing whether CDML systems, which we did not use to develop our
conceptualization, can be successfully modeled with our CDML design toolbox.
To find CDML systems for testing the external validity of our
conceptualization, we applied a backward search and a forward search to the
set of relevant publications. We decided on the relevance of each publication
collated in the backward and forward searches based on the previously used
inclusion criteria (see Table II). If a publication met our inclusion
criteria, we added the publication to our set of relevant literature.

We again applied open, axial, and selective coding to analyze the new relevant
publications. Based on the coding, we classified the CDML systems into the
preliminary CDML design toolbox comprised of the agent-based models of the
Gaia methodology and the assigned codes.

When we recognized that a CDML system could not be classified into our
conceptualization, we refined our conceptualization accordingly and continued
with the test and iterative refinement until we had analyzed all relevant CDML
publications identified in the last round of backward and forward searches.
When our conceptualization needed to be refined, we repeated this third step
of our methods, “Test and Refinement”. We executed this step three times (see
Table III).

During the first iteration, we used four publications from the backward search
and five publications from the forward search, presenting eleven CDML systems.
When classifying the eleven CDML systems into our conceptualization, we
recognized the need for refinements of the CDML design toolbox. For example,
we added the role coordinator to map the sampling-service from the newly added
gossip learning system [49].

During the second iteration, we included one publication from the backward
search and eight publications from the forward search. When classifying the
nine CDML systems presented in those publications into the conceptualization,
we recognized the need to refine our CDML design toolbox. For example, we
needed to add activities and protocols while also requiring a revision of
existing definitions of activities and protocols. For instance, we added the
protocol “assignInterimResultRecepient” and redefined the protocol
‘SignalReadiness” so that agents with the roles trainer or updater can execute
the protocol.

In the third iteration, we tested the conceptualization based on nine CDML
systems presented in nine publications. We did not identify any further need
to refine our concept and decided our concept to be final. Overall, the
conceptualization was successfully tested on 43 CDML systems. 15 of these CDML
systems required refinements of our conceptualization.

TABLE III: Overview of our conceptualization of CDML systems for the development of the CDML design toolbox per iteration and in summary Category |  Initial
Conceptualization |  Iteration 1 |  Iteration 2 |  Iteration 3 |  Summary
---|---|---|---|---|---
Number of Publications
for the Validity Test |  10 |  5 (fs), 4 (bs) |  8 (fs), 1 (bs) |  5 (fs), 4 (bs) |  37
Number of CDML Systems
for the Validity Test |  13 |  5 (fs), 6 (bs) |  9 (fs), 1 (bs) |  5 (fs), 4 (bs) |  43
CDML Systems Successfully
Classified into Conceptualization |  n.a. |  3 |  3 |  9 |  15
CDML Systems Leading to
Conceptualization Refinements |  n.a. |  8 |  7 |  0 |  15
bs: backward search fs: forward search n.a.: not applicable

###  III-B CDML Archetype Development

Since the concept level of the CDML design toolbox points out commonalities
between CDML systems, we focused on the design level to identify CDML
archetypes. The design level allows for the differentiation between CDML
system designs. We developed an agent model, preliminary service model, and
acquaintance model for each CDML system. Using these models, we analyzed the
corresponding CDML system designs to identify similarities. Based on the
identified similarities, we developed CDML archetypes.

##### Agent Model

We started our analysis by examining role distributions in CDML systems to
extract common agent types. To identify agent types and their distribution in
CDML systems, we analyzed the agent models of the 43 CDML systems, which we
previously used for testing the validity of the CDML design toolbox (see
Section III-A2). We developed one agent model for each of the analyzed CDML
systems. Next, we compared the individual models with each other to identify
similarities and differences between the used agent types and their
distribution in the corresponding CDML systems. Based on similarities between
the agent models, we classified the 43 CDML systems into 18 groups of CDML
systems. Each CDML system was assigned to exactly one group.

##### Preliminary Service Model

We analyzed the grouped CDML systems to reveal similarities in the design
options implemented for activities and protocols. For example, CDML systems in
a group all use the design option “only interim result definition” for the
protocol provideMLTask. If CDML systems associated with different groups
showed similar uses of design options, we merged these groups into candidate
CDML archetypes. For example, we merged assisted learning systems with split
learning systems because both systems use the design option “activations” for
the protocol transmitInterimResult. Overall, we merged 18 groups of CDML
systems into six candidate CDML archetypes.

##### Acquaintance Model and Main Processes

We analyzed the communication paths of the individual CDML systems using their
acquaintance models. Whenever we observed similarities in acquaintance models
of CDML systems associated with different groups, we merged the groups. After
analyzing the acquaintance models, we merged our six candidate CDML archetypes
into four final CDML archetypes (i.e., the confidentiality archetype, the
control archetype, the flexibility archetype, and the robustness archetype).
Overall, we assigned each of the 43 CDML systems to one of the four CDML
archetypes.

###  III-C Identification of Key Traits of CDML Archetypes

Using the set of relevant publications on CDML systems that we used to develop
the CDML design toolbox (see Section III-A2), we performed open coding [50] to
extract preliminary traits of CDML system (e.g., robustness against the
participation of malicious agents) that authors point out to highlight
strengths and weaknesses of CDML system designs. We noted the referenced CDML
systems for all preliminary traits and noted explanations of how the trait
originates from the CDML design in axial coding [50]. For example, the key
trait “communication bottleneck” is referenced in several publications about
federated learning systems. This trait originates from the reliability of
federated learning systems on a central agent [51, 40, 52]. We added a
description of whether the referenced CDML system has a strength or weakness
in the respective trait. Our analysis revealed 132 codes representing
preliminary traits of 43 CDML systems. Subsequently, we harmonized the
preliminary traits in three iterations to ensure mutual exclusiveness and
exhaustiveness of our coding [50]. For example, we aggregated the preliminary
traits “does not rely on an orchestrator” and “no need to rely on a third
party” to the trait “fault-tolerant”. Our analysis revealed 38 traits of CDML
systems.

Next, we mapped the 38 traits to the CDML systems to their corresponding CDML
archetypes. We evaluated which traits of individual CDML systems apply to all
CDML systems assigned to corresponding CDML archetypes. We assigned the set of
traits shared by all CDML systems associated with a CDML archetype to the
corresponding CDML archetype as key traits. For example, we extracted the
trait “not reliant on single agents” from literature on blockchain-based
federated learning systems. To evaluate whether this trait also applies to all
CDML systems of the robustness archetype, we analyzed the CDML systems of the
robustness archetype (e.g., swarm learning) regarding their redundancy of
agent types. Since all CDML system designs of the robustness archetype show a
high redundancy of agent types, “not reliant on single agents” became a key
trait of the robustness archetype. We repeated this process for all traits
extracted from the literature analysis at the beginning of this step.

##  IV The CDML Design Toolbox

Our CDML design toolbox comprises a concept level and a design level. The
concept level (see Section IV-A) describes how CDML systems are designed in
principle, including agent roles and agent interactions. Roles are assigned to
agents in order to specify the activities and protocols to be executed by
corresponding agents. After the role assignment, agents keep their roles until
the coalition dissolves. Agents do not have to act in all their assigned roles
simultaneously but in at least one role. The design level (see Section IV-B)
includes design options that developers can use to design CDML systems.
Exemplary design options encompass the assignment of agent types (i.e.,
combinations of roles) to agents in the CDML system and the definition of
types of interim results to be transmitted between agents. The design options
are presented in an agent model, a preliminary service model, and an
acquaintance model. The agent model shows common combinations of agent types
used in CDML systems. In the preliminary service model, we describe design
options for implementing activities and protocols described in the roles
model. The acquaintance model illustrates communication paths between these
agent types in existing CDML systems.

To make the models incorporated in our CDML design toolbox tangible, we
describe them along the principal CDML life cycle. The CDML life cycle
incorporates three sequential phases each CDML system passes through: the
initialization phase, the operation phase, and the dissolution phase. In the
initialization phase, agents form and initialize a coalition that can become a
CDML system. The initialization phase described in this paper focuses on the
autonomous formation of CDML systems by agents in MASs. Alternatively,
developers can manually initialize CDML systems. However, the manual setup of
CDML systems is out of the scope of this work. In the operation phase, agents
interact in order to train or execute ML models. In the dissolution phase, the
agents end their collaboration and dissolve the CDML system. Because multiple
CDML systems may be formed in a single MAS (e.g., in open MAS), these phases
can be passed through in parallel. For simplicity, we describe these three
phases using the example of the formation of a single coalition that becomes a
CDML system and dissolves. We describe variants of the CDML system design
(e.g., in terms of numbers of agents with specific roles) in Section IV-B.

###  IV-A Concept Level of the CDML Design Toolbox

TABLE IV: Overview of roles and corresponding activities and protocols in the CDML design toolbox Role |  Description |  Activities |  Protocols
---|---|---|---
Configurator |  Approves agent applications for the coalition and defines the ML model and interim results to be transmitted in the coalition |  registerCoalition, defineInitialMLModel, defineInterimResult, awaitApplications, decideOnApplication |  applyForCoalition, informApplicant, provideMLTask
Coordinator |  Approves agent applications for the coalition and assigns communication paths among agents |  awaitApplications, decideOnApplication |  applyForCoalition, informApplicant, provideMLTask, assignInterimResultRecipient
Selector |  Chooses agents to train and update ML models |  selectAgent |  applyForCoalition, announceAgentSelection
Trainer |  Uses local data and computing resources under its control to update parameters of (parts of) ML models (e.g., to produce interim results) and transmits them to other agents in the coalition |  awaitSelectionSignal, trainMLModel, |  applyForCoalition, signalReadiness, transmitInterimResult
Updater |  Uses interim results received from other agents to compute updated parameters of (parts of) its ML model. |  awaitSelectionSignal, awaitInterimResults, updateMLModel |  applyForCoalition, signalReadiness, transmitInterimResult

The concept level of our CDML design toolbox incorporates a roles model and an
interactions model. The roles model comprises role descriptions, activities of
agents, and responsibilities. The interactions model includes protocols that
specify interactions between agents.

##### Initialization Phase

In the initialization phase, agents form a coalition of at least two agents
that aim to collaborate to accomplish an ML task. The formation of coalitions,
which can become CDML systems, is triggered by a configurator agent. The
configurator agent stores the CDML system specifications (activity:
registerCoalition) about the purpose of envisioned CDML systems (i.e., the
general prediction problem that ought to be addressed) and requirements for
agents that are searched to join the coalition (e.g., in terms of the needed
training data structure). The configurator agent defines (parts of) the
initial ML model (activity: defineInitialMLModel) to be trained. Definitions
of the (parts of) initial ML models are, for instance, the (first) layers of
neural networks, a (sub-) set of parameters of linear regressions, activation
functions, and the ML model architecture. Moreover, the configurator agent
defines the structure and type of interim results (activity:
defineInterimResult) to be transmitted between agents in the envisioned CDML
system. Interim results are updates that are computed by agents based on local
training data and the locally available (part of an) ML model. Then, the
configurator agent registers the coalition (activity: registerCoalition) with
a repository and starts an application process.

Agents fetch the CDML system specifications from the repository. Based on the
CDML system specifications, agents decide whether to participate in the CDML
system. Agents that decide to participate submit an application, including the
roles they apply for, to the configurator agent (protocol: applyForCoalition).
Commonly, agents can apply for the roles coordinator, selector, trainer, and
updater. The configurator agent iteratively checks for applications from
agents (activity: awaitApplications). Upon application receipt, the
configurator agent decides whether to accept or reject the agent for the CDML
system (activity: decideOnApplication). Then, the configurator agent responds
to the applying agent with an acceptance message or a rejection message
(protocol: informApplicant).

When trainer and updater agents join the coalition, the coordinator agent
assigns trainer agents to updater agents they will interact with in the
operation phase and inform the respective agents about the assignment
(protocol: assignInterimResultRecipient). The trainer agent sends its interim
result to its assigned updater agent. The updater agent can return interim
results to its assigned trainer agent(s) after updating (parts of) the ML
model.

The configurator agent sends the ML task (protocol: provideMLTask) to agents
in the coalition. ML tasks are a collection of information required to train
and update ML models and can include the initial ML model definition and the
interim result definition.

At the end of the initialization phase, at least two agents of the coalition
must have been assigned the following roles to form a CDML system:
configurator, coordinator, selector, trainer, and updater. Agents may have
multiple roles. We describe common combinations of roles on the design level
of the CDML design toolbox (see Section IV-B).

After the initialization phase, the coordinator agent handles applications of
agents on behalf of the configurator agent, which executes the activities
awaitApplications, decideOnApplication and the protocols applyForCoalition and
informApplicant. The coordinator agents send the ML task to the accepted
agents (protocol: provideMLTask). After the initialization of the CDML system,
ML models can be trained and executed in the operation phase.

##### Operation Phase

In the operation phase, agents participate in the training and execution of ML
models according to their assigned roles. At the beginning, the trainer agent
and the updater agent signal their readiness to the selector agent (protocol:
signalReadiness). Agents that have signaled their readiness iteratively check
for triggers from the selector agent to execute activities and protocols
required to collaboratively train and update ML models (activity:
awaitSelectionSignal).

The selector agent selects trainer agents and updater agents (activity:
selectAgent) to act in at least one of these roles. Then, the selector agent
requests the selected agents to act in the corresponding roles (protocol:
announceAgentSelection). Agents that are selected for the role trainer use
their locally available (parts of the) ML model and local training data to
compute interim results (activity: trainMLModel). The trainer agent sends its
interim result to the updater agent (protocol: transmitInterimResult). The
Updater agent waits until it receives interim results (activity:
awaitInterimResults) and then uses the interim results received from trainer
agents to compute a new version of the locally available (part of the) ML
model (activity: updateMLModel). The execution order of training, updating,
and transmitting interim results can vary between CDML systems (see Section
IV-B). The procedure outlined in the operation phase is typically executed
repeatedly. Protocols and activities may be executed in parallel or
sequentially.

##### Dissolution Phase

In the dissolution phase, agents stop executing the processes described in the
operation phase. This can be the case if agents decide that (parts of) the ML
model(s) have been sufficiently trained or, in case that other agents are
required to execute ML models, that they do not need to execute ML model
anymore. When agents end their collaboration, the CDML system dissolves.

###  IV-B Design Level of the CDML Design Toolbox

While the concept level of the CDML design toolbox offers an abstract
description CDML system designs, the design level can guide detailed
specifications of concrete CDML system designs as follows. The first step in
designing CDML systems entails the specification of an agent model (see
Section IV-B1) that presents the assignment of agent types to agents. Agent
types incorporate all roles that are simultaneously assigned to single agents.
The CDML design toolbox offers a set of agent types commonly used in CDML
systems in the agent model (see Section IV-B1). Second, developers need to
tailor the activities and protocols associated with agent types to the
requirements of the envisioned CDML system. In Section IV-B2, the CDML design
toolbox offers a range of design options on how activities and protocols can
be implemented to develop service models for CDML systems. Finally, the
acquaintance model needs to specify communication paths between agents (see
Section IV-B3). While some communication paths are integral to all CDML
systems (e.g., trainer agents sending interim results to updater agents, see
Section IV-B1), others are contingent on the characteristics of CDML systems
(e.g., updater agents returning interim results to trainer agents). The CDML
design toolbox introduces communication paths necessary to operate CDML
systems successfully. This list comprises necessary and optional communication
paths and helps developers consider communication efficiency and communication
bottlenecks when designing CDML systems.

In the following, we describe the three models (i.e., the agent model, the
preliminary service model, and the acquaintance model) that can be utilized to
develop CDML systems.

####  IV-B1 Agent Model

Agent types are a combination of roles identified in the roles model that can
serve as a blueprint to implement agents in CDML systems. Following the
concept level of the CDML design toolbox (see Section IV-A), CDML systems
require at least two agents with agent types that in combination comprise the
following roles: configurator, coordinator, selector, trainer, and updater.
These roles can be assigned to agents in seven combinations (see Table V),
each combination forming an individual agent type. Identical agent types can
be assigned to multiple agents, for example, to increase redundancies in the
processing of ML tasks [14] or to distribute workload in the processing of ML
tasks [10]. First, the Tra agent type only comprises the role trainer. Agents
of the Tra agent type only train the ML model without updating it with interim
results from other agents. The Tra agent type is utilized in CDML systems with
only one training round [53].

Second, the CooSel agent type comprises the roles coordinator and selector.
This agent type is utilized in CDML systems with a peer-to-peer structure. If
agent selection and the assignment of trainer agents to updater agents follow
a sophisticated rule (e.g., unbiased peer-to-peer sampling service [54]),
CooSel agents can be implemented that only focus on the selection and
assignment of agents [49, 55].

Third, the TraUpd agent type combines the roles trainer and updater. The
TraUpd agent type is implemented in many CDML systems since it combines the
two main roles accounting for training ML models. TraUpd agents can train ML
models but can include interim results into their local ML models [35, 47,
56].

Fourth, the ConTraUpd agent type combines the roles configurator, trainer, and
updater. The ConTraUpd agent type is mainly used in split learning systems and
assisted learning systems. The configurator role is required since agents in
these CDML systems define their own ML model [12, 11].

Fifth, the ConCooSelUpd agent type combines the roles configurator,
coordinator, selector, and updater. ConCooSelUpd agents primarily operate
central servers in federated learning systems [35, 47].

Sixth, the CooSelTraUpd agent type combines the roles coordinator, selector,
trainer, and updater. This agent type has a high degree of autonomy as it can
execute all activities and protocols except those with the configurator role.
The CooSelTraUpd agent type is used in CDML systems to create a high level of
redundancy [14, 57, 58].

Seventh, the ConCooSelTraUpd agent type combines the roles configurator,
coordinator, selector, trainer, and updater. This agent type is assigned to
central agents in federated learning (e.g., [59]) that train ML models or a
single agent that initiates the ML model to be trained in peer-to-peer-based
CDML systems (e.g., the BraintTorrent system [48] and the gossip learning
systems [49]).

TABLE V: Overview of agent types in CDML systems | Agent Roles
---|---
|  Configurator |  Coordinator |  Selector |  Trainer |  Updater
Agent Type
Tra |  |  |  |  ✓ |
CooSel |  |  ✓ |  ✓ |  |
TraUpd |  |  |  |  ✓ |  ✓
ConTraUpd |  ✓ |  |  |  ✓ |  ✓
ConCooSelUpd |  ✓ |  ✓ |  ✓ |  |  ✓
CooSelTraUpd |  |  ✓ |  ✓ |  ✓ |  ✓
ConCooSelTraUpd |  ✓ |  ✓ |  ✓ |  ✓ |  ✓

####  IV-B2 Preliminary Service Model

The key activities and protocols introduced at the concept level of the CDML
design toolbox (see Table IV) can be implemented based on various design
options. It is important to note that the following descriptions do not
represent a complete service model [20]. Complete service models are usually
highly context-dependent and, thus, out of scope for this work. The following
descriptions of design options for the key activities and protocols are
intended as a foundation for developing detailed service models.

##### Activities

We identified 12 design options for five key activities. The activity
awaitApplications has two design options. First, the agent population awaits
agent applications to join the coalition “only during the initialization
phase”. Applications are ignored when the CDML system is already initialized.
For example, in most variants of split learning systems [11], the ML model
layers to be trained need to be assigned to agents during the initialization
phase, which prevents agents from joining after the initialization phase.
Second, the agent population accepts applications “always” [14]. This allows
agents to join the CDML system arbitrarily.

The activity selectAgent has three design options. First, agents can be
selected for a role “based on votes from other agents” in the CDML system. The
selector collects the votes of other agents and decides which agents should
execute which activities and protocols; for example, all agents in the CDML
system can vote on which agent activates the updater role and executes the
updating of the ML model (activity: updateMLModel) [14]). Second, agents can
be selected “based on agent attributes”, for example, based on the size of
agents’ datasets [53]. Third, agents can be selected “randomly” to activate a
role and execute corresponding activities and protocols [60, 48].

The activity awaitInterimResults has two design options. To maintain liveness
in CDML systems, the waiting time of agents for interim results can be
“response-bound” or “time-bound”. If the waiting time of the agents is
“response-bound” [61], the updater agent waits for a specified number of
interim results before updating the ML model with the interim results
received. “Response-bound” waiting for interim results can decrease the
liveness of CDML systems if set too high; for example, when an agent with the
role updater awaits interim results from all trainer agent, but on trainer
agent may have crashed, the updater agent may theoretically wait infinitely.
“Time-bound” waiting tackles this issue [10]. If the waiting time exceeds a
specified time bound, the updater agent updates the ML model with all interim
results received during the waiting period. However, “time-bound” waiting may
lead the updater agent to ignore interim results received too late.

The activity updateMLModel has two design options. First, updater agents can
perform “batched updates” [62, 52, 53, 57]. In “batched updates”, updater
agents use a set of interim results received from trainer agents to update
their ML model at one time. Second, updater agents can perform “individual
updates” to separately update the ML model for each interim result received
from a trainer agent or an updater agent [61, 11].

The activity trainMLModel has three design options. First, trainer agents can
“train two complete ML models”. In this case, trainer agents compute two
separate ML models. A local ML model that learns representations of the
training data and a global ML model that is trained on the local ML model
instead of the raw training data. An advantage of this approach is that the
local ML model can protect confidential attributes from the global ML model,
thus improving training data confidentiality. Moreover, the communication
efficiency can be improved because the global ML model requires fewer
parameters due to the local ML model learning being the foundation for the
global ML model [63, 64]. Second, trainer agents can “train one complete ML
model”. A complete ML model refers to the entire set of parameters comprising
the ML model. In most CDML systems, trainer agents store and train one
complete ML model [47, 16]. Third, trainer agents can “train a part of an ML
model”. A part of an ML model refers to a subset of ML model parameters.
Exemplary parts of ML models are layers of a neural network or a subset of
coefficients of linear regression. Training only a part of an ML model has two
main advantages. First, trainer agents require less storage and computing
resources. Second, due to trainer agents only having access to a part of the
ML model, the complete ML model can remain confidential [11, 12].

##### Protocols

We identified nine design options for three key protocols. We identified two
design options for the protocol provideMLTask. First, the agent with the role
configurator can provide “only interim result definitions” to other agents in
the CDML system. The agent with the role configurator only provides the
interface between agents (e.g., exchange parameters or gradients). The exact
ML model to be used remains unknown to other agents (e.g., in terms of the ML
model architecture and its hyperparameters) [12]. Second, the configurator
agent provides both the interim result definition and initial ML model
definition (e.g., [35, 10]).

The protocol announceAgentSelection has two design options. First, the
selector agent can announce which agent should activate which role [10, 49].
Second, the selector agent can announce what agents should activate which role
and announce the training sample IDs that should be trained [12].

There are five design options for the protocol transmitInterimResult. First,
agents can transmit “parameter values” [19, 65]. Parameter values refer to a
set of variables or weights that the ML model learns from the training data
and that determine how the ML model makes predictions based on the input data.
Second, agents can transmit “gradients” [61, 35]. Gradients refer to the
directional slopes or change rates of a mathematical function. Third, agents
can transmit “activations with labels” [11, 66]. We refer to activations as
intermediate outputs of an ML model for a given input. When the ML model is
presented with input data, it propagates the data through its layers, applies
the learned parameters (weights and biases), and produces an output. We refer
to the output as “activations” if it is not the final output of the ML model.
If the output is from the final layer / includes all parameter values of the
ML model, we call the outputs predictions. Fourth, agents can transmit
“activations without labels” [11, 66]. Fifth, agents can transmit
“(pseudo-)residuals” [12]. Residuals refer to the differences between the
actual target values and the predicted values generated by an ML model.
Pseudo-residuals can be considered intermediate residuals and are often used
in boosting algorithms.

####  IV-B3 Acquaintance Model

Several communication paths between agents are required for the functioning of
CDML systems. Some of those communication paths are indispensable in every
CDML system; other communication paths only appear in some CDML systems. Based
on our concept level of CDML systems (see Section IV-A), we describe
indispensable communication paths and optional communication paths (design
options) in the following. Since communication paths differ between the
lifecycle phases of CDML systems, we describe the communication paths for each
phase separately.

##### Initialization Phase

The configurator agent must have a bidirectional communication path to all
other agents for two purposes: first, to participate in the coalition
application process (protocols: applyForCoalition, informApplicant); second,
to provide them with the ML task definition (protocol: provideMLTask).

The coordinator agent must have a unidirectional communication path to the
trainer agent to inform the agent to which updater agent they should send
their interim results (protocol: assignInterimResultRecipient). This
communication path allows for more flexibility by enabling sub-coalitions that
form around updater agents [67, 19, 10].

The coordinator agent may have a unidirectional communication path to the
updater agents. Via such a communication path, the coordinator agent can
inform the updater agents to which updater agents they should send
intermediate results (protocol: assignInterimResultRecipient). This
communication path can be used for a hierarchically organized CDML system, in
which updater agents communicate with each other to improve their local ML
model without using local training data [67, 19, 10].

##### Operation Phase

The selector agents must have a bidirectional communication path to the
trainer agent and the updater agent. This communication path enables the
selector agent to receive signals that these agents are ready to participate
in the training (protocol: signalReadiness) and to inform these agents that
they are selected for the training (protocol: announceAgentSelection).

The trainer agent must have a unidirectional communication path to the updater
agent to send it interim results (protocol: transmitInterimResult).

The coordinator agent can have a bidirectional communication path to all other
agent roles if applications can be received and processed after the
initialization phase. In this case, the coordinator agent take over handling
the applications from the configurator agent (protocols: applyForCoalition,
informApplicant). Because agents can apply and be admitted to a CDML system
after the initialization phase, this communication path enables the CDML
system to address issues in the agent population during the operation phase.
For example, if it becomes clear during the operation phase that the training
data is insufficient, more trainer agents can be admitted to the CDML system.

The updater agent can have unidirectional or bidirectional communication paths
with an other updater agent to exchange information about their ML model
update (e.g., [10, 19]). This communication path allows for hierarchical
structures with more than one updater agent.

The trainer agent can have bidirectional communication paths to the updater
agent, for example, to send and receive interim results (protocol:
transmitInterimResult). Such bidirectional communication paths are common in
CDML systems. In some CDML systems (e.g., one-shot federated learning [53]),
the trainer agent sends interim training results to the updater agent without
receiving interim results in return [53].

##### Dissolution Phase

During the dissolution phase, the communication paths between agents are
dissolved. Agents that have stored a local ML model can keep it and use it to
make predictions on their own.

##  V CDML Archetypes

We developed four CDML archetypes that reflect CDML system designs common in
practice and research: the confidentiality archetype, the control archetype,
the flexibility archetype, and the robustness archetype. The CDML archetypes
are distinguished by their agent models, acquaintance models, and principal
functioning, including preliminary service models. Table VI gives an overview
of the four CDML archetypes we describe in detail in the following. The
coalition-forming phase is outside the scope of the archetype descriptions
because developers can set up CDML systems that correspond to the CDML
archetypes. For each CDML archetype, we highlight common design variants.

TABLE VI: Overview of CDML Archetypes and Their Designs |  CDML Archetypes |  Confidentiality Archetype |  Control Archetype |  Flexibility Archetype |  Robustness Archetype
---|---|---|---|---|---
|  Exemplary CDML Systems |  Assisted learning systems [12, 68], Split learning systems [11, 66], SplitFed learning systems [37] |  Variants of federated learning systems [39, 10, 53, 61, 51, 59, 35, 63, 52] |  Gossip learning systems [49], Decentralized federated learning systems [40, 62, 69, 64], BrainTorrent systems [48] |  Swarm learning systems [14], Blockchain-based federated learning systems [60, 65, 57]
Distinguishing Design Options |  Hierarchy |  Strong |  Strong |  Weak |  Weak
Agent Types and their Number of Occurences |  ConTraUpd ($1\leq$),
ConCooSelUpd (1) |  TraUpd ($1\leq$),
ConCooSelUpd (1) |  ConCooSelTraUpd ($1\leq$),
CooSelTraUpd ($1\leq$) |  ConCooSelTraUpd (1),
CooSelTraUpd ($1\leq$)
Coalition-wide ML Model
Synchronization |  No |  Yes |  No |  Yes
Design Options for Activities |  awaitApplications |  Only during the initialization phase, Always |  Only during the initialization phase, Always |  Only during the initialization phase, Always |  Only during the initialization phase, Always
awaitInterimResults |  Waiting for a time-threshold, Waiting for a response-threshold |  Waiting for a time-threshold, Waiting for a response-threshold |  Waiting for a time-threshold, Waiting for a response-threshold |  Waiting for a time-threshold, Waiting for a response-threshold
selectAgent |  Based on agent attributes, Randomly |  Based on agent attributes, Randomly |  Based on agent attributes, Randomly |  Based on agent votes from other agents
trainMLModel |  Train a part of the ML model |  Train a part of the ML model, Train one complete ML model, Train two complete ML models |  Train one complete ML model, Train two complete ML models |  Train one complete ML model
updateMLModel |  Batched update, Individual update |  Batched update, Individual update |  Batched update, Individual update |  Batched update
Design Options for Protocols |  announceAgentSelection |  Role and training sample IDs |  Only role |  Only role |  Only role
provideMLTask |  Provide only interim result definition |  Provide ML model definition and interim result definition |  Provide ML model definition and interim result definition |  Provide ML model definition and interim result definition
|  transmitInterimResult |  Activations with labels, Activations without labels, (Pseudo- )Residuals, Gradients |  Gradients, Parameter values |  Gradients, Parameter values |  Gradients, Parameter values

###  V-A Confidentiality Archetype

The confidentiality archetype is suitable for use cases in which agents want
to preserve the confidentiality of ML models, ML tasks, and training data.
Agents only store parts of ML models. The full architectures of ML models
trained in the confidentiality archetype are not disclosed. Thus, no agent has
access to the global ML model. Instead, the global ML model is distributed
across several agents, which only store parts of it. ML models are not
synchronized coalition-wide during ML model training and for ML model
inference. Exemplary CDML systems of the confidentiality archetype are split
learning [11, 66, 70], assisted learning [12, 68], gradient assisted learning
[17], SplitFed learning [37], FDML [71], hierarchical SplitFed learning [19],
and FedLite [72].

####  V-A1 Agent Model

The confidentiality archetype comprises the agent types ConCooSelUpd
andConTraUpd. In its basic configuration, the confidentiality archetype
comprises one ConCooSelUpd agent and at least one ConTraUpd agent.

####  V-A2 Acquaintance Model

In the confidentiality archetype, the ConCooSelUpd agent can communicate with
all ConTraUpd agents on bidirectional communication paths (see Figure 1).
ConTraUpd agents do not communicate with each other directly.

Figure 1: Exemplary acquaintance model of the confidentiality archetype

####  V-A3 Principal Functioning

In the initialization phase, the ConCooSelUpd agent configures its local part
of the ML model and defines the interim results to be transmitted (activities:
defineInitialMLModel, defineInterimResult). Local parts of the ML model can be
specific layers of a neural network in split learning [11] or just parts of a
layer of a neural network in vertical split learning [11] and assisted
learning [12]. Examples of interim results include activations of a particular
layer of a neural network (e.g., referred to as the cut layer in split
learning) [11] or (pseudo-)residuals [17]. The ConCooSelUpd agent then
provides ConTraUpd agents with the interim result definition (protocol:
provideMLTask; design option: provide only interim result definition). After
receiving the interim result definition, ConTraUpd agents individually set up
their local parts of the ML model following the interim results definition.
For example, the ConTraUpd agents in split learning systems set up the layers
of a neural network from the input layer to the cut layer. The number of
outputs of the cut layer is set depending on the interim results definition.

The operation phase starts with the ConTraUpd agents signaling their readiness
to the ConCooSelUpd agent (protocol: signalReadiness) to participate in the
subsequent training round. Then, ConTraUpd agents wait for a response
(activity: awaitSelectionSignal). The ConCooSelUpd agent decides which
ConTraUpd agents to select for the next training round (activity:
selectAgent). For example, this selection can be made based on agent
attributes or randomly. After the selection, the ConCooSelUpd agent announces
its decision to the ConTraUpd agents (protocol: announceAgentSelection).
Selected ConTraUpd agents train their parts of the ML model (activity:
trainMLModel; design option: train a part of the ML model) and transmit their
interim results to the ConCooSelUpd agent (protocol: transmitInterimResult;
design option: activations with labels, (pseudo-)residuals). The ConCooSelUpd
agent waits for incoming interim results (protocol: awaitInterimResults). The
ConCooSelUpd agent uses the interim results to update (and train) its local
(part of the) ML model (activities: trainMLModel, updateMLModel). Depending on
the implementation, the ConCooSelUpd agent then transmits another interim
result back to the ConTraUpd agents (protocol: transmitInterResult; design
option: gradients). ConTraUpd agents use it to update their local part of the
ML model. The ConCooSelUpd agent decides how often this process is repeated.

####  V-A4 Key Traits

The confidentiality archetype relies on a strongly hierarchical agent
organization and does not have a coalition-wide synchronization of ML models.
The missing synchronization of ML models among agents leads to the fact that
ML models can be kept confidential. The main trait of the confidentiality
archetype is that confidentiality entails training data confidentiality and
the ML model confidentiality because agents only have access to parts of the
ML model. Next to enabling ML model confidentiality, The confidentiality
archetype can be very computation efficient since agents only have to store
and compute a part of the ML model, which can be potentially very large [11,
72]. The confidentiality archetype requires fewer training rounds than the
control archetype and converges quickly [66, 11]. The confidentiality
archetype has high communication costs due to the ML model partitioning and
the communication of both activations and gradients [72]. Some CDML systems
that correspond to the confidentiality archetype, such as split learning
systems (e.g., [11]), can have high idle times of trainer agents since the
trainer agents only interact with the updater agents sequentially [37]. Other
CDML systems, such as SplitFed learning systems, address this issue by
combining elements of split learning and federated learning and, thus, can
reduce the idle times [37]. As no agent has access to the entire ML model, the
coalition (or a subset of it) is required to make ML model inferences.
Therefore, the coalition can only be resolved when the ML model is not used
anymore.

####  V-A5 Variants of the Confidentiality Archetype

##### U-Shaped Split Learning [11]

U-shaped split learning systems can be used to train neural networks. A
selected ConTraUpd agent executes the forward propagation up to a specific
layer (i.e., the first cut layer) and only transmits activations to the
ConCooSelUpd agent (protocol: transmitInterimResults; design option:
activations without labels). The ConCooSelUpd continues the forward
propagation up to the second cut layer and transmits activations back to the
ConTraUpd agent. The ConTraUpd agent completes the forward propagation, starts
the backpropagation, and transmits the gradients of the second cut layer to
the ConCooSelUpd agent (protocol: transmitInterimResults; design option:
gradients). Using these gradients, the ConCooSelUpd agent continues the
backpropagation to the first cut layer and transmits the gradients of the
first cut layer to the ConTraUpd agent. The ConTraUpd agent executes the
backpropagation for the remaining layers and, thus, completes a training
round.

###  V-B Control Archetype

The control archetype is suitable for use cases in which one agent should have
control over the DCML system. The control archetype incorporates a
hierarchical communication structure with an agent on the top level that
controls the training process. The agent on top receives all interim results
and synchronizes the training process by deciding on the global ML model to be
trained in each training round. Exemplary CDML systems of the control
archetype implement variants of federated learning [10, 61, 35, 63], including
one-shot federated learning [53], semiFL [59], heteroFL [39], and hierarchical
federated learning[51, 52].

####  V-B1 Agent Model

CDML systems belonging to the control archetype comprise the agent types
ConCooSelUpd and TraUpd. The control archetype comprises one ConCooSelUpd
agent and at least one TraUpd agent.

####  V-B2 Acquaintance Model

The acquaintance model of the control archetype has the structure of a tree
(see Figure 2). Agents can bidirectionally communicate in a strictly
hierarchical manner along the vertexes of the tree. In its basic form, there
are two hierarchical levels (e.g., [10]): a root ConCooSelUpd agent forms the
top level of the hierarchy. At least one TraUpd agent resides on the bottom
level of the hierarchy. There can be additional levels between the top level
and the bottom level (e.g., [51, 52]). The inner nodes of the tree are
ConCooSelUpd agents, whereas TraUpd agents represent the leaves.

Figure 2: Exemplary acquaintance model of the control archetype

####  V-B3 Principal Functioning

In the initialization phase, the ConCooSelUpd agent on the top level of the
hierarchy defines the initial ML model and interim results (activities:
defineInitialMLModel, defineInterimResult). Suppose there are additional
ConCooSelUpd agents on lower levels of the acquaintance model. In that case,
the initial ML model and interim result definition are propagated to these
agents by executing the protocol provideMLTask (design option: ML model
definition and interim result definition). ConCooSelUpd agents on lower levels
of the acquaintance model can only forward parts of the ML model (i.e., sub-
models) to their child nodes. Thus, each ConCooSelUpd agent can individually
define the initial ML model and interim results for its descendants
(activities: defineInitialMLModel, defineInterimResult).

In the operation phase, TraUpd agents execute the signalReadiness protocol to
signal their availability to participate in a training round to their
respective parent ConCooSelUpd agent. Then, TraUpd agents wait for a selection
signal (activity: awaitSelectionSignal). ConCooSelUpd agents decide which of
their child ConCooSelUpd and TraUpd agents to include in a training round.
Once a sufficient number of child agents have signaled their readiness to a
ConCooSelUpd agent, it signals its readiness to its parent agent and waits for
a selection signal (activity: awaitSelectionSignal). This process is repeated
recursively throughout the hierarchy until it reaches the root ConCooSelUpd
agent. Then, the root ConCooSelUpd agent selects (a subset of) its subordinate
agents to participate in the upcoming training round (activity: selectAgent;
design option: based on agent attributes or randomly) and announces its
selection to its child agents (protocol: announceAgentSelection). Afterward,
it transmits the current version of the ML model, or a part thereof, to
selected child agents (protocol: transmitInterimResult; design option:
gradients or parameter values) and waits for interim results (activity:
awaitInterimResult; design option: waiting for a time-threshold or waiting for
a response-threshold). This selection process is repeated recursively by
descendant ConCooSelUpd agents until it reaches the leaf TraUpd agents. The
TraUpd agents update their local ML model based on the interim result received
(activity: updateMLModel; design option: batched update) and train it using
local training data and self-controlled compute (activity: trainMLModel;
design option: train one complete ML model or train a part of the ML model).
After training is completed, TraUpd agents initiate the transmitInterimResult
protocol (design option: gradients or parameter values) with their respective
parent ConCooSelUpd agent as the responder. The parent ConCooSelUpd agent
waits until a defined threshold is reached (activity: awaitInterimResult;
design option: waiting for a time-threshold or waiting for a response-
threshold) and update their (part of the) ML model based on the interim
results received (activity: updateMLModel; design option: batched update).
Each ConCooSelUpd agent can decide how often to repeat this training procedure
with its descendants. When the desired number of training rounds is completed,
ConCooSelUpd agents send the updated (part of the) ML model to their parent
nodes (protocol: transmitInterimResult; design option: gradients or parameter
values). Once the threshold of the root ConCooSelUpd agent is reached, a
coalition-wide training round is completed.

The procedure described for the operation phase is repeatedly executed until
the dissolution phase is initiated by the root ConCooSelUpd agent.

####  V-B4 Key Traits

The control archetype implements a strongly hierarchical organizational
structure of agents and requires the coalition-wide synchronization of ML
models. The combination of these traits leads to organizational structures in
which a small fraction of all agents wield the predominant control over the
CDML system. The control archetype is suitable for use cases with strict
hierarchies where one or a few agents should keep control over the CDML
system. The control archetype relies on only one root ConCooSelUpd agent. If
the one updater agent crashes, the whole CDML system crashes [49, 52, 48].
Thus, the control archetype is not crash-fault tolerant. The use of multiple
updater agents assigned to multiple layers of the hierarchy of the control
archetype can make the system tolerant to crashes of single updater agents
[19, 52]. If one updater agent crashes, updater agents can take the load in
aggregating interim results of the crashed one. However, this redistribution
of load to fewer updater agents can drastically reduce the overall performance
of the control archetype. The control archetype can be prone to performance
bottlenecks due to a few central agents having to execute numerous
computationally intensive activities and protocols [52, 53]. Such performance
bottlenecks include computation [40] (i.e., during updating) and communication
[40, 51] (i.e., sending and receiving interim results). Regarding the
predictive performance of the ML model trained collaboratively, the control
archetype usually performs better than the confidentiality archetype (e.g.,
[37]). The ML model usually converges faster than in CDML systems of the
flexibility archetype (e.g., [9]). The coalition can be dissolved after
training because the coalition is not required to make ML model inferences.

####  V-B5 Variants of the Control Archetype

##### TraUpd Agents as Tra Agents [53]

TraUpd agents lose their updater role and become Tra agents. In this variant,
the interim results are only transmitted from Tra agents to ConCooSelUpd
agents. No interim results are transmitted back to Tra agents. Tra agents do
not update their local ML models.

##### ConCooSelUpd Agents as ConCooSelTraUpd Agents [59]

ConCooSelUpd agents gain the trainer role and become ConCooSelTraUpd agents.
In these systems, the agents on higher levels of the hierarchy possess
training data on their own and use it to train (parts of) the ML model
themselves (e.g., [59]). ConCooSelTraUpd agents train the ML model (activity:
trainMLModel; design option: train one complete ML model or train a part of
the ML model) while waiting for interim results of subordinate agents in the
hierarchy.

##### TraUpd Agents Train Two Complete ML Models [63]

TraUpd agents train two complete ML models locally (activity: trainMLModel;
design option: train two complete ML models). TraUpd agents train one ML model
on local data. The second ML model is trained on the first ML model. Only the
gradients or parameter values resulting from the training of the second ML
model are transmitted to the superordinate agent.

###  V-C Flexibility Archetype

The flexibility archetype is suitable for use cases with communication
topologies that can change at run-time [40]. The flexibility archetype offers
a high degree of agent autonomy. Agents can arbitrarily join and leave the
flexibility archetype without impeding the functioning of the CDML system
[40]. In its basic variant, agents can select agents they want to collaborate
with. Moreover, agents can decide if and when they execute activities (e.g.,
trainMLModel or updateMLModel) and protocols (e.g., signalReadiness or
transmitInterimResult). The flexibility archetype is weakly hierarchically
organized. ML models are not synchronized coalition-wide during ML model
training. Exemplary CDML systems of the flexibility archetype implement gossip
learning [49], BrainTorrent [48], and decentralized federated learning [40,
62, 64, 69].

####  V-C1 Agent Model

The flexibility archetype comprises the agent types ConCooSelTraUpd and
CooSelTraUpd. In its basic configuration, the flexibility archetype comprises
one ConCooSelTraUpd agent and at least one CooSelTraUpd agent.

####  V-C2 Acquaintance Model

To participate in the training, agents must establish a bidirectional
communication path to at least one other agent. (see Figure 3). Other agents
include ConCooSelTraUpd agents and CooSelTraUpd agents. Agents decide with
which agents they interact on an equitable basis.

Figure 3: Acquaintance model of the flexibility archetype for an exemplary
training round

####  V-C3 Principal Functioning

In the initialization phase, the ConCooSelTraUpd agent first defines the ML
model (activity: defineInitialMLModel) and interim results (activity:
defineInterimResult). The ConCooSelTraUpd agent distributes the ML model and
the interim result definition to other agents in the CDML system (protocol:
provideMLTask; design option: provide initial ML model definition and interim
result definition). Agents can join at any time (protocol: applyForCoalition;
design option: always).

In the operation phase, each ConCooSelTraUpd and CooSelTraUpd agents train the
ML model locally using local training data and self-controlled computing
resources. Afterward, each agent signals its readiness to activate its updater
role for the upcoming training round (protocol: signalReadiness) and waits for
other agents to signal their readiness (activity: awaitAgentReadiness). Then,
at least one agent that signals its readiness is selected (activity:
selectAgent) to receive the interim results. Agents are usually selected
randomly (design option: randomly, but can also be selected in a targeted
manner (design option: based on agent attributes. The selection is announced
to the selected agent (protocol: announceAgentSelection). Agents that are
selected to activate the role updater wait (activity: awaitInterimResult)
until they receive the interim results from other agents using the protocol
transmitInterimResult (design option: gradients or parameter values). Lastly,
the selected agents use the interim results of other agents to update their
local ML model (activity: updateMLModel). The update can entail several
interim results (design option: batched update) or only one interim result
from another agent (design option: individual update).

This process is repeated until the dissolution phase is initiated. The
flexibility archetype dissolves when no agents engage in collaborative
training anymore.

####  V-C4 Key Traits

The flexibility archetype is weakly hierarchical and agents store different
states of ML models. ML models are not synchronized coalition-wide. Agents
have a high degree of autonomy and can individually decide when to train
collaboratively and with whom. Moreover, agents can individually decide to
activate roles and execute activities and protocols, which leads to agents
having little idle time [48].

The flexibility archetype can handle agent crashes better than the control
archetype[49]. An agent dropping out of the system may temporarily reduce the
performance of the flexibility archetype, but because a new agent can be
easily integrated into the training process due to the lack of rules, the
flexibility archetype can recover from the agent drop-out [9]. Because agents
can largely operate independently of each other, no single agent is vital for
the proper functioning of the CDML system. If agents are redundant, agents can
theoretically be replaced. However, this may not always be possible because
the flexibility archetype does not require redundant agents.

The flexibility archetype is not robust against malicious agents. Malicious
agents are agents that tamper with training processes and manipulate
collaboratively trained ML models [9]. Malicious agents can obfuscate their
identities by arbitrarily joining and dropping out of the CDML system and
arbitrarily switching their collaboration partners. Such obfuscation can
facilitate the engagement of agents in performing malicious activities without
detection (e.g., because reputation systems may not be applicable [42]).
Moreover, even when malicious agents are identified, it is hard to punish them
because rules (e.g., agents that act maliciously are forced to leave the
system) are hardly enforceable in the flexibility archetype. The coalition can
be dissolved after ML model training because the CDML system is not required
to make ML model inferences.

####  V-C5 Variants of the Flexibility Archetype

##### Additional CooSel Agent [49]

There can be a dedicated CooSel agent (e.g., [49]). The remaining agents lose
the selector role and become ConCooTraUpd and ConTraUpd agents. In each
training round, the CooSel agent selects a subset of the ConCooTraUpd and
CooTraUpd agents to function as the updater (activity: selectAgent; design
option: randomly) and assigns each of the remaining agents to one of the
agents selected as an updater. Each agent then sends its interim result to the
agent it was assigned to (protocol: transmitInterimResult; design option:
gradients or parameter values).

###  V-D Robustness Archetype

The robustness archetype is suitable for use cases in which agents may
inadvertently drop-out of the coalition during ML model training (e.g., due to
crashes or network failures) because a large fraction of agents is redundant
and, thus, can replace each other. The robustness archetype is weakly
hierarchically organized and performs coalition-wide synchronization of the ML
model. Exemplary CDML systems of the robustness archetype are swarm learning
system [14] and other blockchain-based CDML systems [57, 65].

####  V-D1 Agent Model

The robustness archetype comprises the agent types ConCooSelTraUpd and
CooSelTraUpd. In its basic configuration, the robustness archetype comprises
one ConCooSelTraUpd agent and at least one CooSelTraUpd agent.

####  V-D2 Acquaintance Model

As illustrated in Figure 4, there can be bidirectional communication paths
between all agents in the system. This includes both agents of the type
ConCooSelTraUpd and CooSelTraUpd.

Figure 4: Exemplary acquaintance model of the robustness archetype

####  V-D3 Principal Functioning

In the initialization phase of the robustness archetype, the ConCooSelTraUpd
agent defines the ML model and interim results and distributes corresponding
definitions to other agents in the coalition (protocol: provideMLTask; design
option: provide ML model definition and interim result definition). There must
always be at least one CooSelTraUpd agent and one ConCooSelTraUpd agent to
redundantly execute the roles coordinator, selector, trainer, and updater.
Additional CooSelTraUpd agents can join at any time (protocol:
applyForCoalition; design option: always).

In the operation phase, ConCooSelTraUpd and CooSelTraUpd agents broadcast
their readiness to activate their roles updater and trainer for the training
in the robustness archetype (protocol: signalReadiness). All agents that
received the broadcast individually decide whether the ConCooSelTraUpd or
CooSelTraUpd agent should activate the trainer and updater role (activity:
selectAgent). Agents broadcast their individual decisions to all agents in the
robustness archetype. The final selection of trainer and updater is made
through a consensus mechanism (design option: based on votes from other
agents). Next, ConCooSelTraUpd and CooSelTraUpd agents start training the ML
model using their locally available training data and compute (activity:
trainMLModel; design option: train a complete ML Model). All selected agents
receive identical interim results from agents that trained their ML model
(protocol: transmitInterimResult; design option: gradients or parameter
values). All agents use the identical interim results to update the ML model
(activity: updateMLModel). For the update, all selected updater agents use the
results of from all other agents (design option: batched update). All agents,
which computed ML model updates, broadcast their new interim results to all
agents in the system (protocol: transmitInterimResult).

This process is repeated until the start of the dissolution phase. The
dissolution phase starts when no agents engage in the collaborative training
anymore.

####  V-D4 Key Traits

The robustness archetype is weakly hierarchical and is designed to train
global ML models that are synchronized coalition-wide. Both of these traits
culminate in CDML systems where agent types are redundantly assigned to
agents. Agents process and store data of the global ML model redundantly,
increasing the robustness of CDML systems. The robustness archetype uses a
fully connected communication network [40]. Due to the high redundancy of
agents, except the agent with the role configurator, the robustness archetype
does not rely on single agents. This design prevents the robustness archetype
from failing if some agents drop-out of the CDML system [57], for example, due
to crashes and network failures. The robustness archetype allows for the
replacement of updater agents after each training round. Agents in the
robustness archetype usually require large computational resources, for
example, to compute ML model updates based on interim results from all other
agents in the CDML system [40]. The coalition can be dissolved after training
since the coalition is not required to make ML model inferences.

####  V-D5 Variants of the Robustness Archetype

##### A subset of agents activates the updater role per training round [14,
57]

Interim results are transmitted to and stored by all agents, but only a subset
of agents activate their udater role. From all ConCooSelTraUpd and
CooSelTraUpd agents that signal their readiness (protocol: signalReadiness),
not all agents are selected (activity: selectAgent; design options: based on
agent attributes, based on votes from other agents, or randomly) to activate
their updater role in every training round. In some cases, only one agent is
selected [14].

##  VI Discussion

###  VI-A Principal Findings

In this study, we present a CDML design toolbox, including a concept level and
a design level. The concept level of the CDML design toolbox includes five
roles (i.e., configurator, coordinator, selector, trainer, and updater), ten
activities (e.g., updateMLModel), and seven protocols (e.g.,
transmitInterimResult) inherent to CDML systems. On the design level, the CDML
design toolbox includes design options to customize CDML systems. For example,
the roles trainer and updater can be combined into the agent type TraUpd. We
present seven agent types and seven mandatory communication paths between
these agent types. For example, agents with the role updater can have
communication paths among each other. Moreover, the CDML design toolbox
presents design options for activities and protocols. Based on common
combinations of design options, we present four principal CDML archetypes
(i.e., the confidentiality archetype, control archetype, flexibility
archetype, and robustness archetype) and their key traits.

The design level of the CDML design toolbox shows different implementations of
roles, activities, and protocols in CDML systems that we describe as design
options. Different combinations of design options can lead to different CDML
systems. Our results show how CDML systems can be grouped and differentiated
on the basis of common combinations of design options and resulting key
traits. We observed significant similarities among CDML systems studied by
research communities with limited overlap. It turns out that split learning
systems and assisted learning systems implement similar design options; for
example, they comprise only ConCooSelUpd and ConTraUpd agents. Moreover, swarm
learning systems and blockchain-based decentralized federated learning systems
have similar design options. For example, both implement the agent types
ConCooSelTraUpd and CooSelTraUpd but differ regarding the number of agents
with an active updater role each training round.

The presented CDML archetypes and their key traits show that no one-size-fits-
all CDML system can be used for every use case. Developers must carefully
assess the suitability of CDML systems based on their designs and different
traits. For instance, the redundant distribution of roles in swarm learning
enhances robustness. However, in use cases where most agents have limited
resources, mandating that all agents perform all roles may result in the
failure of the CDML system because agents may be assigned roles that exceed
their resource capacities. Conversely, the redundancy in distributing agent
roles can be better suited for use cases characterized by frequent agent drop-
outs. Therefore, the careful assessment of CDML system suitability for use
case requirements is mandatory to operate CDML systems successfully.

In the agent model (see Section IV-B1), we present the agent types that we
identified in the analyzed publications. The presented agent types represent a
subset of the possible combinations of agent roles. For example, we did not
identify a Con agent or an Upd agent even though the implementation of such
agents could be possible as long as all roles are distributed to agents in
CDML systems. CDML systems that assign each agent only one role could also
have new traits, including agents requiring fewer resources, that might be
useful in many use cases. Because of the theoretical availability of more
agent types and combinations of design options, more CDML system designs with
different traits may become available in the future.

###  VI-B Contributions to Practice and Research

With this study, we contribute to practice and research in three principal
ways. First, by presenting the CDML design toolbox, we offer a consolidated
design knowledge base of previously scattered design knowledge of CDML
systems. Since the comparison of differences between CDML system designs has
focused on a few design aspects (e.g., the training process), the CDML design
toolbox enables systematic comparisons between CDML system designs covering a
broad set of design options. The agent-based models on the concept level
(i.e., the roles model and interactions model) of the CDML design toolbox
present the main design commonalities of CDML systems (e.g., the use of
specific agent roles and the principal training process). The three agent-
based models on the design level (i.e., agent model, service model, and
acquaintance model) can guide the systematic comparison between CDML system
designs and the customization of CDML system designs to meet use case
requirements. Moreover, the developed agent-based models can facilitate the
application of the Gaia methodology for developing custom CDML system designs.

Second, by showcasing CDML archetypes, we offer starting points for the
combination of design options to develop CDML system designs. The archetypes
inform of combinations of design options commonly used in practice and
research. The CDML archetypes can be customized by using the design options
presented in the CDML design toolbox to develop blueprints of CDML systems.
Thereby, in combination, the CDML archetypes and the CDML design toolbox offer
actionable help in guiding the design of CDML systems.

Third, by presenting key traits of CDML archetypes, we support developers in
deciding on combinations of design options to meet use case requirements. The
key traits of CDML archetypes enable developers to choose the most fitting
CDML archetype for use cases. Using the selected CDML archetype as a starting
point, developers can use the CDML design toolbox and customize the archetype
to show additional required traits. By executing this process, developers can
evaluate CDML system designs in their suitability for use cases prior to
implementing the designs.

###  VI-C Limitations

For the development of the CDML design toolbox, the CDML archetypes, and the
identification of key traits, we analyzed publications and CDML systems that
we deemed to be representative of the CDML field. With our selection of
publications and CDML systems for analysis, we aimed to cover the large
spectrum of different CDML system designs. However, the number of publications
and CDML systems significantly increased in the past years, making it
impossible to incorporate all publications in our study but only a
representative set of publications. The CDML design toolbox may not cover all
CDML system designs.

To conceptualize CDML systems, we strove to extract and understand their key
design aspects (e.g., activities, processes, and roles), requiring the
resolution of ambiguities, and to set extracted key aspects in relationships
(e.g., roles and responsibilities). Although well-suited to conduct such
research, qualitative research is inherently prone to subjective biases, for
example, because publications are individually interpreted depending on
personal conceptions. Despite our efforts to reduce such biases (e.g., through
feedback on our results from ML experts), we cannot guarantee that we have
completely eliminated them.

The analyzed publications focus on the core training process [48, 53, 11, 49,
40]. Other system components required to operate CDML systems are mostly
neglected. By triangulating descriptions of CDML systems based on our coding
and intense discussions with ML experts, we aimed to complete fragmented
descriptions of CDML systems. Still, the CDML design toolbox may lack aspects
not specifically mentioned in the analyzed publications. Similarly, a
significant number of the examined publications lacked sufficient detail in
their descriptions of permissions of roles, activities, and protocols. This
hindered us in describing permissions associated with agent roles at the
concept level and impeded the development of a complete service model.
Instead, we developed a preliminary service model that describes how
activities and protocols can be implemented.

###  VI-D Future Research

This work presents a wide range of CDML system designs that address the
different requirements of use cases. We noticed that research on CDML systems
remains predominantly theoretical, with only a few real-world implementations
of CDML systems (e.g., [16]). To gain a more comprehensive understanding of
the advantages and limitations of CDML systems in various use cases, future
research should prioritize empirical investigations of practical
implementations of CDML systems. This research should place particular
emphasis on real-world implications, encompassing socio-technical aspects such
as human perception and acceptance. The CDML design toolbox offers a
foundation for knowledge transfers within the CDML community (e.g., to develop
new CDML systems) and across multiple disciplines. In the following, we
describe three areas for knowledge transfer that may be particularly
interesting for improving CDML systems in future research.

##### Hyperparameter Optimization

Automated hyperparameter optimization (HPO) has become very important in the
development of ML models for manifold purposes [73], such as to improve ML
model performance and decrease necessary computations in the training of ML
models. For most automated HPO methods, such as Bayesian optimization [74, 75,
76], the availability of complete training data sets is assumed. This
assumption lies at odds with decentralized training data management in CDML
systems. Extant automated HPO methods are hardly applicable to CDML systems,
which may result in under-optimized ML models trained in CDML systems [73].
The CDML design toolbox can serve as a foundation for future research to
identify challenges in performing HPO in CDML systems with different designs
and develop corresponding solutions.

##### Data Confidentiality

The exchange of interim results instead of training data does not guarantee
training data confidentiality per se [77]. To protect training data
confidentiality, the combination of CDML and other privacy-enhancing
technologies (PETs), such as differential privacy and homomorphic encryption,
has become promising [56, 78]. Future research should develop guidelines for
how to combine the CDML paradigm with other PETs reasonably.

##### Robustness

Agents may pursue individual goals in CDML systems. However, ensuring the
accurate alignment between individual agent goals and the overarching goal of
the CDML system is critical. Misalignment can have detrimental consequences,
such as the introduction of the free-rider problem [79] and incentivizing
agents to poison training data or ML models [80, 81, 82]. The free-rider
problem is characterized by agents that provide subpar data while being able
to improve their ML model from interim results received from other agents.
Integrating robustness measures from diverse fields into CDML systems, such as
financial incentives in economics and normative principles in sociology for
agent behavior coordination [82, 42, 83, 84], could enhance the robustness of
CDML systems against challenges, such as anticipating malicious actions of
agents in CDML systems. Future research should extend the CDML design toolbox
to include design options that improve the robustness of CDML systems and
protect ML model training from malicious agent activity.

##  VII Conclusion

This work presents a CDML design toolbox that can be used to guide developers
in the development of CDML system designs. Leveraging the CDML design toolbox,
we developed four CDML archetypes with different key traits that can guide
developers in the design of CDML systems.

The CDML design toolbox is envisioned to offer a foundation for developers to
design CDML systems suitable for use cases. With our presentation of design
options, we aim to accelerate the design process and develop novel CDML
systems that can cover an even wider range of use cases.

During our investigation, we recognized the substantial expansion of the CDML
design space through contributions from practice and research. Following
federated learning systems, alternative CDML systems, such as split learning
systems, assisted learning systems, and gossip learning systems, have moved
into the focus of practice and research.

We hope that the CDML design toolbox will support the targeted design of CDML
systems suitable for use cases (e.g., by facilitating the use of the Gaia
method [20]) so that training of ML models on sufficient training data becomes
easier for developers. Owing to the considerable attention that CDML systems
have garnered in practice and research and the emergence of novel CDML
concepts beyond federated learning, we encourage the advancement of the CDML
design toolbox in the future.

## Acknowledgement

We thank Benjamin Sturm, Kathrin Brecker, Marc Zöller, Mikael Beyene, Richard
Guse, Simon Warsinsky, and Tobias Dehling for their valuable feedback on this
work. This work was supported by funding from the topic Engineering Secure
Systems of the Helmholtz Association (HGF) and by KASTEL Security Research
Labs.

## References

  * [1] A. Halevy, P. Norvig, and F. Pereira, “The Unreasonable Effectiveness of Data,” _IEEE Intelligent Systems_ , vol. 24, no. 2, pp. 8–12, 2009.
  * [2] A. Bansal, R. Sharma, and M. Kathuria, “A Systematic Review on Data Scarcity Problem in Deep Learning: Solution and Applications,” _ACM Computing Surveys_ , vol. 54, no. 10s, pp. 1–29, 2022.
  * [3] N. Rieke, J. Hancox, W. Li, F. Milletarì _et al._ , “The future of digital health with federated learning,” _npj Digital Medicine_ , vol. 3, no. 119, 2020.
  * [4] R. Shokri and V. Shmatikov, “Privacy-Preserving Deep Learning,” in _Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security_ , Denver, CO, USA, 2015, p. 1310–1321.
  * [5] G. J. Annas, “HIPAA Regulations–A New Era of Medical-Record Privacy?” _New England Journal of Medicine_ , vol. 348, no. 15, p. 1486–1490, 2003.
  * [6] European Parliament and Council of the European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council. [Online]. Available: https://data.europa.eu/eli/reg/2016/679/oj
  * [7] C. for Medicare & Medicaid Services. (1996) The Health Insurance Portability and Accountability Act of 1996 (HIPAA). Online at http://www.cms.hhs.gov/hipaa/.
  * [8] T. Dehling and A. Sunyaev, “A design theory for transparency of information privacy practices,” _Information Systems Research_ , vol. ePub ahead of print August 8, p. 1–22, 2023.
  * [9] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet _et al._ , “Advances and Open Problems in Federated Learning,” 2021. [Online]. Available: http://arxiv.org/abs/1912.04977
  * [10] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba _et al._ , “Towards Federated Learning at Scale: System Design,” 2019. [Online]. Available: https://arxiv.org/abs/1902.01046
  * [11] P. Vepakomma, O. Gupta, T. Swedish, and R. Raskar, “Split learning for health: Distributed deep learning without sharing raw patient data,” 2018. [Online]. Available: https://arxiv.org/abs/1812.00564
  * [12] X. Xian, X. Wang, J. Ding, and R. Ghanadan, “Assisted learning: A framework for multi-organization learning,” in _Advances in Neural Information Processing Systems_ , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 14 580–14 591.
  * [13] C. Thapa, P. C. Mahawaga Arachchige, S. Camtepe, and L. Sun, “SplitFed: When Federated Learning Meets Split Learning,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 8, pp. 8485–8493, 2022.
  * [14] S. Warnat-Herresthal, H. Schultze, K. L. Shastry, S. Manamohan _et al._ , “Swarm Learning for decentralized and confidential clinical machine learning,” _Nature_ , vol. 594, no. 7862, p. 265–270, 2021.
  * [15] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ , A. Singh and J. Zhu, Eds., vol. 54, 2017, pp. 1273–1282.
  * [16] A. Hard, C. M. Kiddon, D. Ramage, F. Beaufays _et al._ , “Federated Learning for Mobile Keyboard Prediction,” 2018. [Online]. Available: https://arxiv.org/abs/1811.03604
  * [17] E. Diao, J. Ding, and V. Tarokh, “GAL: Gradient Assisted Learning for Decentralized Multi-Organization Collaborations,” 2022. [Online]. Available: http://arxiv.org/abs/2106.01425
  * [18] A. M. Ozbayoglu, M. U. Gudelek, and O. B. Sezer, “Deep learning for financial applications: A survey,” _Applied Soft Computing_ , vol. 93, p. 106384, 2020.
  * [19] T. Xia, Y. Deng, S. Yue, J. He _et al._ , “HSFL: An Efficient Split Federated Learning Framework via Hierarchical Organization,” in _2022 18th International Conference on Network and Service Management_ , Thessaloniki, Greece, 2022, pp. 1–9.
  * [20] M. Wooldridge, N. R. Jennings, and D. Kinny, “The Gaia Methodology for Agent-Oriented Analysis and Design,” _Autonomous Agents and Multi-Agent Systems_ , vol. 3, no. 3, p. 285–312, 2000.
  * [21] C. Zhang, J. Xia, B. Yang, H. Puyang _et al._ , “Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning,” in _Proceedings of the ACM Symposium on Cloud Computing_ , Seattle, WA, USA, 2021, pp. 546–561.
  * [22] W. Zheng, R. Deng, W. Chen, R. A. Popa _et al._ , “Cerebro: A Platform for Multi-Party Cryptographic Collaborative Learning,” in _30th USENIX Security Symposium_ , Vancouver, Canada, 2021, pp. 2723–2740.
  * [23] Q. Wang, M. Du, X. Chen, Y. Chen _et al._ , “Privacy-Preserving Collaborative Model Learning: The Case of Word Vector Training,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 30, no. 12, pp. 2381–2393, 2018.
  * [24] B. Wang, M. Li, S. S. M. Chow, and H. Li, “A tale of two clouds: Computing on data encrypted under multiple keys,” in _2014 IEEE Conference on Communications and Network Security_ , San Francisco, CA, USA, 2014, pp. 337–345.
  * [25] B. Reagen, W.-S. Choi, Y. Ko, V. T. Lee _et al._ , “Cheetah: Optimizing and Accelerating Homomorphic Encryption for Private Inference,” in _2021 IEEE International Symposium on High-Performance Computer Architecture_ , Seoul, South Korea, 2021, pp. 26–39.
  * [26] J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and J. S. Rellermeyer, “A Survey on Distributed Machine Learning,” _ACM Computing Surveys_ , vol. 53, no. 2, p. 1–33, 2020.
  * [27] S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “Zero: Memory optimizations toward training trillion parameter models,” in _SC20: International Conference for High Performance Computing, Networking, Storage and Analysis_ , Atlanta, GA, USA, 2020, pp. 1–16.
  * [28] M. Diskin, A. Bukhtiyarov, M. Ryabinin, L. Saulnier _et al._ , “Distributed deep learning in open collaborations,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 7879–7897, 2021.
  * [29] M. Li, D. G. Andersen, J. W. Park, A. J. Smola _et al._ , “Scaling Distributed Machine Learning with the Parameter Server,” in _11th USENIX Symposium on Operating Systems Design and Implementation_ , Broomfield, CO, USA, 2014, p. 583–598.
  * [30] M. Diskin, A. Bukhtiyarov, M. Ryabinin, L. Saulnier _et al._ , “Distributed Deep Learning in Open Collaborations,” 2021. [Online]. Available: https://arxiv.org/abs/2106.10207
  * [31] M. Li, D. G. Andersen, J. W. Park, A. J. Smola _et al._ , “Scaling distributed machine learning with the parameter server,” in _11th USENIX Symposium on Operating Systems Design and Implementation_ , Broomfield, CO, USA, 2014, pp. 583–598.
  * [32] P. Patarasuk and X. Yuan, “Bandwidth optimal all-reduce algorithms for clusters of workstations,” _Journal of Parallel and Distributed Computing_ , vol. 69, no. 2, pp. 117–124, 2009.
  * [33] R. Thakur, R. Rabenseifner, and W. Gropp, “Optimization of collective communication operations in mpich,” _The International Journal of High Performance Computing Applications_ , vol. 19, no. 1, pp. 49–66, 2005.
  * [34] H. Zhu, J. Xu, S. Liu, and Y. Jin, “Federated learning on non-IID data: A survey,” _Neurocomputing_ , vol. 465, pp. 371–390, 2021.
  * [35] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ , Lauderdale, FL, USA, 2017.
  * [36] T. Zhang, L. Gao, C. He, M. Zhang _et al._ , “Federated Learning for the Internet of Things: Applications, Challenges, and Opportunities,” _IEEE Internet of Things Magazine_ , vol. 5, no. 1, pp. 24–29, 2022.
  * [37] C. Thapa, P. C. Mahawaga Arachchige, S. Camtepe, and L. Sun, “Splitfed: When Federated Learning Meets Split Learning,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 8, 2022, p. 8485–8493.
  * [38] X. Xian, X. Wang, J. Ding, and R. Ghanadan, “Assisted Learning: A Framework for Multi-Organization Learning,” in _34th Conference on Neural Information Processing Systems_ , vol. 33, Vancouver, Canada, 2020, pp. 14 580–14 591.
  * [39] E. Diao, J. Ding, and V. Tarokh, “HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients,” 2021. [Online]. Available: https://arxiv.org/abs/2010.01264
  * [40] Y. Yuan, J. Liu, D. Jin, Z. Yue _et al._ , “DeceFL: A Principled Fully Decentralized Federated Learning Framework,” _National Science Open_ , vol. 2, no. 1, pp. 1–17, 2023.
  * [41] H. S. Nwana, “Software Agents: An Overview,” _The Knowledge Engineering Review_ , vol. 11, no. 3, p. 205–244, 1996.
  * [42] D. Jin, N. Kannengießer, B. Sturm, and A. Sunyaev, “Tackling Challenges of Robustness Measures for Agent Collaboration in Open Multi-Agent Systems,” in _55th Hawaii International Conference on System Sciences_ , Honolulu, HI, USA, 2022, pp. 7585–7594.
  * [43] M. Wooldridge, “Temporal belief logics for modeling distributed artificial intelligence systems,” _Foundations of Distributed Artificial Intelligence. John Wiley & Sons_, pp. 269–286, 1996.
  * [44] X. Cao, Y. Chen, and K. J. R. Liu, “Data Trading With Multiple Owners, Collectors, and Users: An Iterative Auction Mechanism,” _IEEE Transactions on Signal and Information Processing over Networks_ , vol. 3, no. 2, pp. 268–281, 2017.
  * [45] L. Sterling and K. Taveter, _The art of agent-oriented modeling_. Cambridge, MA, USA: The MIT press, 2009.
  * [46] M. Wooldridgey and P. Ciancarini, “Agent-oriented software engineering: The state of the art,” in _First international workshop, AOSE 2000 on Agent-oriented software engineering_ , Limerick, Ireland, 2001, pp. 1–28.
  * [47] G. Long, Y. Tan, J. Jiang, and C. Zhang, “Federated learning for open banking,” in _Federated Learning: Privacy and Incentive_. Heidelberg, Germany: Springer, 2020, pp. 240–254.
  * [48] A. G. Roy, S. Siddiqui, S. Pölsterl, N. Navab, and C. Wachinger, “BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning,” 2019. [Online]. Available: https://arxiv.org/abs/1905.06731
  * [49] I. Hegedűs, G. Danner, and M. Jelasity, “Gossip Learning as a Decentralized Alternative to Federated Learning,” in _Distributed Applications and Interoperable Systems_ , J. Pereira and L. Ricci, Eds. Cham, Switzerland: Springer International Publishing, 2019, vol. 11534, pp. 74–90.
  * [50] J. M. Corbin and A. L. Strauss, _Basics of qualitative research: techniques and procedures for developing grounded theory_ , 4th ed. Los Angeles, CA, USA: SAGE Publications, Inc., 2015.
  * [51] Y. Zhao, Z. Liu, C. Qiu, X. Wang _et al._ , “An Incentive Mechanism for Big Data Trading in End-Edge-Cloud Hierarchical Federated Learning,” in _2021 IEEE Global Communications Conference_ , Madrid, Spain, 2021, pp. 1–6.
  * [52] A. Wainakh, A. S. Guinea, T. Grube, and M. Muhlhauser, “Enhancing Privacy via Hierarchical Federated Learning,” in _2020 IEEE European Symposium on Security and Privacy Workshops_ , Genoa, Italy, 2020, pp. 344–347.
  * [53] N. Guha, A. Talwalkar, and V. Smith, “One-Shot Federated Learning,” 2019. [Online]. Available: https://arxiv.org/abs/1902.11175
  * [54] D. Stutzbach, R. Rejaie, N. Duffield, S. Sen, and W. Willinger, “On Unbiased Sampling for Unstructured Peer-to-Peer Networks,” _IEEE/ACM Transactions on Networking_ , vol. 17, no. 2, pp. 377–390, 2009.
  * [55] I. Hegedüs, G. Danner, and M. Jelasity, “Decentralized learning works: An empirical comparison of gossip learning and federated learning,” _Journal of Parallel and Distributed Computing_ , vol. 148, pp. 109–124, 2021.
  * [56] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farhad, S. Jin, T. Q. S. Quek, and H. V. Poor, “Federated Learning with Differential Privacy: Algorithms and Performance Analysis,” 2019. [Online]. Available: https://arxiv.org/abs/1911.00222
  * [57] Y. Li, C. Chen, N. Liu, H. Huang _et al._ , “A Blockchain-Based Decentralized Federated Learning Framework with Committee Consensus,” _IEEE Network_ , vol. 35, no. 1, pp. 234–241, 2021.
  * [58] H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Blockchained On-Device Federated Learning,” _IEEE Communications Letters_ , vol. 24, no. 6, pp. 1279–1283, 2020.
  * [59] E. Diao, J. Ding, and V. Tarokh, “SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training,” in _Advances in Neural Information Processing Systems_ , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. New Orleans, LA, USA: Curran Associates, Inc., 2022, pp. 17 871–17 884.
  * [60] M. Xu, Z. Zou, Y. Cheng, Q. Hu _et al._ , “SPDL: A Blockchain-Enabled Secure and Privacy-Preserving Decentralized Learning System,” _IEEE Transactions on Computers_ , vol. 72, no. 2, pp. 548–558, 2023.
  * [61] R. Shokri and V. Shmatikov, “Privacy-Preserving Deep Learning,” in _Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security_ , Denver, CO, USA, 2015, pp. 1310–1321.
  * [62] W. Liu, L. Chen, and W. Zhang, “Decentralized Federated Learning: Balancing Communication and Computing Costs,” _IEEE Transactions on Signal and Information Processing over Networks_ , vol. 8, pp. 131–143, 2022.
  * [63] P. P. Liang, T. Liu, L. Ziyin, N. B. Allen _et al._ , “Think Locally, Act Globally: Federated Learning with Local and Global Representations,” 2020. [Online]. Available: https://arxiv.org/abs/2001.01523
  * [64] S. Kalra, J. Wen, J. C. Cresswell, M. Volkovs, and H. R. Tizhoosh, “Decentralized federated learning through proxy model sharing,” _Nature Communications_ , vol. 14, no. 1, p. 2899, 2023.
  * [65] J. Li, Y. Shao, K. Wei, M. Ding _et al._ , “Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation,” 2021. [Online]. Available: https://arxiv.org/abs/2101.06905
  * [66] O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” _Journal of Network and Computer Applications_ , vol. 116, p. 1–8, 2018.
  * [67] B. Wang, J. Fang, H. Li, X. Yuan, and Q. Ling, “Confederated Learning: Federated Learning With Decentralized Edge Servers,” _IEEE Transactions on Signal Processing_ , vol. 71, pp. 248–263, 2023.
  * [68] C. Chen, J. Zhou, J. Ding, and Y. Zhou, “Assisted Learning for Organizations with Limited Data,” 2022. [Online]. Available: https://arxiv.org/abs/2109.09307
  * [69] H. Xing, O. Simeone, and S. Bi, “Decentralized Federated Learning via SGD over Wireless D2D Networks,” in _2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications_ , Atlanta, GA, USA, 2020, pp. 1–5.
  * [70] X. Chen, J. Li, and C. Chakrabarti, “Communication and Computation Reduction for Split Learning using Asynchronous Training,” in _2021 IEEE Workshop on Signal Processing Systems_ , Coimbra, Portugal, 2021, pp. 76–81.
  * [71] Y. Hu, D. Niu, J. Yang, and S. Zhou, “FDML: A Collaborative Machine Learning Framework for Distributed Features,” in _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, Anchorage, AK, USA, 2019, pp. 2232–2240.
  * [72] J. Wang, H. Qi, A. S. Rawat, S. Reddi _et al._ , “FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients,” 2022. [Online]. Available: https://arxiv.org/abs/2201.11865