# Software Engineering for Collective Cyber-Physical Ecosystems

Roberto Casadei  0000-0001-9149-949X Università di Bologna50 Via
dell’Università, 47521CesenaItaly<EMAIL_ADDRESS>,  Gianluca Aguzzi
0000-0002-1553-4561 Università di Bologna50 Via dell’Università,
47521CesenaItaly<EMAIL_ADDRESS>,  Giorgio Audrito
0000-0002-2319-0375 Università di Torino185 Corso Svizzera, 10149TurinItaly
<EMAIL_ADDRESS>,  Ferruccio Damiani  0000-0001-8109-1706 Università
di Torino185 Corso Svizzera, 10149TurinItaly<EMAIL_ADDRESS>,
Danilo Pianini  0000-0002-8392-5409 Università di Bologna50 Via
dell’Università, 47521CesenaItaly<EMAIL_ADDRESS>,  Giordano Scarso
0009-0009-2114-7435 Università di Torino185 Corso Svizzera, 10149TurinItaly
<EMAIL_ADDRESS>,  Gianluca Torta  0000-0002-4276-7213 Università di
Torino185 Corso Svizzera, 10149TurinItaly<EMAIL_ADDRESS>and  Mirko
Viroli  0000-0003-2702-5702 Università di Bologna50 Via dell’Università,
47521CesenaItaly<EMAIL_ADDRESS>
(2024)

###### Abstract.

Today’s distributed and pervasive computing addresses large-scale cyber-
physical ecosystems, characterised by dense and large networks of devices
capable of computation, communication and interaction with the environment and
people. While most research focusses on treating these systems as “composites”
(i.e., heterogeneous functional complexes), recent developments in fields such
as self-organising systems and swarm robotics have opened up a complementary
perspective: treating systems as “collectives” (i.e., uniform, collaborative,
and self-organising groups of entities). This article explores the
motivations, state of the art, and implications of this “collective computing
paradigm” in software engineering, discusses its peculiar challenges, and
outlines a path for future research, touching on aspects such as
macroprogramming, collective intelligence, self-adaptive middleware, learning,
synthesis and experimentation of collective behaviour.

cyber-physical ecosystems, collective adaptive systems, swarm intelligence,
macroprogramming, edge-cloud continuum

††copyright: none††journalyear: 2024††conference: International Workshop on
Software Engineering in 2030; November 2024; Puerto Galinàs (Brazil)††ccs:
Software and its engineering Software development methods††ccs: Human-centered
computing Collaborative and social computing††ccs: Computing
methodologies††ccs: Software and its engineering System description
languages††ccs: Software and its engineering Software organization and
properties

##  1\. Introduction

Technological advances in computing systems and networking and trends like
pervasive computing (Conti et al., 2012) and the Internet of Things (IoT) (Li
et al., 2015), promote an increasing digitalisation of our world, which is
being filled with interconnected computing devices supporting a variety of
services and applications. In this work, we are concerned with the general
high-level problem of how to unleash the potential of _large-scale distributed
systems_ (van Steen et al., 2012) by proper engineering of the software
managing them or driving their overall behaviour.

Specifically, in this paper, we focus on _collective cyber-physical
ecosystems_. By _cyber-physical_ , we indicate that systems consist of
_situated computing_ devices able to sense and/or act in the physical
environment; without neglecting that systems may also comprise non-situated
and infrastructural devices (e.g., edge servers, the cloud, etc.). By
_ecosystem_ , we mean that devices _live_ and interact with each other within
the environment, functioning as an “ecological unit”, i.e., interacting to
provide functionalities in a long-term equilibrium; in particular, we consider
both short- and long-lived tasks which may require cooperation and must adapt
to perturbations. By _collective_ , we mean that multiple devices can be
regarded as a whole, solving distributed tasks through _ computational
collective intelligence (CCI)_ (Malone and Bernstein, 2022; Casadei, 2023a; He
et al., 2019). Examples of CCPEs include robotic swarms (Brambilla et al.,
2013), smart cities (Dharmawan et al., 2019), edge-cloud infrastructures (Xu
et al., 2018), and crowds of people with wearables (Ferscha et al., 2014).

In the research landscape of _ software engineering (SE)_ for CCPEs,
mainstream approaches – like (micro-)services, programming frameworks, and
artificial intelligence – are typically applied under a view of _“systems as
composites”_ ; in this view, a system is built as the _integration_ of
_heterogeneous_ components, which can be engineered largely independently of
the others. Recently, results from niches of several research threads – like
collective adaptive systems (Nicola et al., 2020), self-adaptive software
(Weyns, 2020), macro-programming (Casadei, 2023b; Júnior et al., 2022), and
swarm robotics (Brambilla et al., 2013) – promoted the complementary view of
_“systems as collectives”_. In this view, a system is engineered “as a whole”,
adopting (declarative) abstractions, tools, and methods suitable to
coordinating groups of interacting devices, letting their CCI emerge.

In this paper, we address the impact of adopting a collective stance in
software engineering for large-scale distributed systems, considering CCPEs as
main target and artificial CCI as main goal. To this end, we provide a review
of the state of the art on CCPE engineering, emphasising the different
approaches involved and then highlight key challenges that we expect may
contribute to the research and development of SE.

The rest of the manuscript is structured as follows. Section 2 details on the
motivation for the emerging of the “system as collectives” perspective in
software engineering. Section 3 reports on the state of the art on CCPE
engineering, and its main themes. Then, in Section 4, we highlight relevant
challenges fostering research in software engineering for CCPEs. Finally,
Section 5 offers a wrap-up.

##  2\. Systems as Collectives

In this section, we detail on the motivation for a collective viewpoint in
software engineering for large-scale distributed ecosystems.

###  2.1. Context and high-level challenges

Engineering pervasive and cyber-physical (eco-)systems is challenging (Broy
and Schmidt, 2014; Becker et al., 2019). Indeed, many challenges have to be
addressed, including the implications of _distribution_ (cf., lack of global
clock, multiple administrative domains, dynamic topologies, latency and cost
of communication, failure, inconsistency, security) (Tanenbaum and van Steen,
2007), the integration of _discrete_ and _continuous_ dynamics (Cassandras and
Lafortune, 2021), the implications of _large scale_ deployments (Orgerie et
al., 2013), and the _design of collaborative logic_ (cf. Section 3). For
instance, addressing large scale typically requires the design of
_decentralised_ solutions to avoid _single points of failure_ and _performance
bottlenecks_. To deal with different requirements in terms of latency, cost,
and performance, deployments often target multiple layers of the _ edge-cloud
continuum (ECC)_ (Xu et al., 2018). Also, since large-scale deployments
complicate _maintenance_ , the design should involve _autonomic capabilities_
(Kephart and Chess, 2003), endowing systems with some _adaptation_ or _self-*
properties_ (e.g., self-organisation (Gershenson et al., 2020; Ye et al.,
2017), self-reconfiguration (Coullon et al., 2023), self-improvement and self-
integration (Bellman et al., 2021)).

###  2.2. SE approaches for distributed systems

The majority of current solutions for distributed software design leverage
_(micro-)services_ (Dragoni et al., 2017), stream processing (de Assunção et
al., 2018), or _event-driven_ platforms (Cheng et al., 2016). These techniques
are suitable also for CCPEs, though challenges related to heterogeneity,
deployment, efficiency, integration, and security are hindering their adoption
in specific domains like manufacturing (Xu et al., 2018). General approaches
to _service composition_ (Peltz, 2003; Lemos et al., 2016) include
_orchestration_ by (possibly multiple) centralised entities (Vaquero et al.,
2019) or _choreographies_ for decentralised interaction protocols (Montesi,
2023). An interesting related SE practice is the use of _architectural
description languages_ (Pandey, 2010) describing the components and
constraints of an entire distributed system, and its specification in a single
codebase—cf. _multi-tier programming_ (Weisenburger et al., 2020).

Concerning _self-adaptive software systems_ (Weyns, 2020), there are multiple
classes of self-adaptation approaches. The _reference model_ for self-adaptive
systems (Kephart and Chess, 2003) distinguishes between the _managing_ and the
_managed_ system, and generally organises a feedback loop around the _MAPE-K
(Monitor, Analyse, Plan, Execute - Knowledge)_ components within the managing
system. Upon this basis, _architecture-based_ approaches emphasise runtime
reasoning about architectural models; _requirements-based_ approaches
emphasise the specification of requirements, often relaxed to deal with
uncertainty, and meta-requirements about the self-adaptation itself; _control-
based_ approaches leverage control theory to formally design and analyse the
self-adaptation logic; and _learning-based_ approaches exploit machine
learning to synthesise accurate models and plans.

In _decentralised_ self-adaptive systems, adaptation control is distributed
among different components equipped with feedback loops. This is similar but
often distinguished from _self-organising_ systems (Bonabeau et al., 1999; Ye
et al., 2017), which generally consist of a larger number of simpler
components that seek system goals by repeated local interaction. Engineering
self-organisation is a matter of identifying mechanisms and techniques to
_steer or guide the emergent behaviour_ of groups of interacting agents
(Rodríguez et al., 2007; Ye et al., 2017).

###  2.3. From composites to collectives

At this point, we deem useful to distinguish between two related but distinct
kinds of possible target _systems_ , where a system is generally meant,
coherently with systems theories (Mobus and Kalton, 2015), as a _plurality_ of
entities considered together within an arbitrary boundary. In particular, we
are inspired by the ontological distinction between composites and collectives
in Masolo et al. (Masolo et al., 2020). A _composite_ , a.k.a. an assembly or
complex, consists of different kinds of components, thus exhibiting
substantial _heterogeneity_ , organised in a recursively decomposable
functional structure with multiple roles. A distinct but related and non-
disjoint notion is that of a _collective_ , which is a more _homogeneous_ set
of entities related by some non-transitive _membership_ relationship. Roughly,
then, examples of composites include the distributed software modules that
collaboratively operate a manufacturing machine, or a team of agents with
significantly different capabilities interacting following a complex protocol;
whereas a swarm of similar robots, a cluster, or a sensor network are
exemplars of collectives. Research in fields such as swarm robotics (Brambilla
et al., 2013), collective adaptive systems (Nicola et al., 2020), and
artificial life (Gershenson et al., 2020; Brambilla et al., 2013) shows
peculiar traits of collectives, motivating new approaches to their
engineering. Abowd (Abowd, 2016) refers to _collective computing_ as the
fourth revolution after the mainframe, personal, and ubiquitous computing
revolutions. Indeed, the _collective intelligence_ (He et al., 2019; Suran et
al., 2021) emerging from the collaboration of humans and technologies has the
potential to overcome important societal challenges (Suran et al., 2022). Even
when humans are not an active part of a solution, the engineering of
_artificial collectives_ (Casadei, 2023a) can contribute by improving existing
applications (e.g., through increased autonomy and resilience), unlocking
novel ones (cf. _drone crowdsourcing_ (Alwateer et al., 2019) and _IoT -as-a-
Service_ (Hoque et al., 2022)), or sustaining the autonomous operation of
large ecosystems in smart infrastructures and smart cities (Pianini et al.,
2021; Dharmawan et al., 2019).

The interest in large-scale CCPEs is mounting. In these systems, besides the
local services and tasks (often resulting from _complex networks of
collaboration_), there is increasing interest in _global-level services_
(e.g., surveillance, waste collection, task allocation, group coordination)
and _global-level properties_ (e.g., overall energy consumption, system-level
resilience). Accordingly, several research efforts are noticing _fundamental
gaps in software engineering for these systems_ and consequently foster the
_collective viewpoint_ in system analysis, design, and implementation, with
_collectives_ and _collective phenomena_ (Wood and Galton, 2009) as first-
class citizens (Casadei, 2023b; Murgia et al., 2023; Júnior et al., 2022;
Inverso et al., 2020). In the next sections, we detail on these contributions,
further motivating interesting developments and challenges in software
engineering.

##  3\. SE for CCPE$\mathbf{s}$: Overview

In this section, we discuss the main issues encountered when engineering
CCPEs, and then provide a view of the state of the art.

###  3.1. Main Themes in SE for CCPEs

####  3.1.1. Local-to-global and global-to-local mapping problems

When dealing with CCPEs, there are two main issues (Tumer and Wolpert, 2004;
Beal et al., 2013; Casadei, 2023a).

The _local-to-global mapping problem_ , also known as the _forward_ or
_prediction problem_ , entails determining what will be the global outcome out
of an execution of the local behaviours making up the system, for a given set
of environment dynamics. Indeed, in the traditional “node-centric” engineering
viewpoint, the programmer thinks about how the behaviour of an _individual
agent_ will affect other agents and its surrounding environment. However, to
determine the global or system-level result, the behaviour of other agents and
the environment have to be considered as well, resulting in a complex network
of causes and effects, often leading to _emergent phenomena_. In emergence, it
is difficult to trace the micro-level causes that led to a given macro-level
observable. In other words, emergence makes prediction difficult or
impossible: the general solution consists of “running the system”, i.e.,
seeing where the chain of events and the historical contingencies lead the
system. This makes _simulation_ an invaluable tool for studying and analysing
collective behaviour.

The _global-to-local mapping problem_ , also known as the _inverse_ or
_engineering problem_ , entails determining, from a global outcome (_target
state_), what local behaviours will conduct to it and how. Clearly the two
problems are related, as two sides of the same coin.

Other names for these problems and corresponding solutions are also _micro-to-
macro_ and _macro-to-micro_ , respectively (Ye et al., 2022; Sawyer, 2003;
Casadei, 2023b). It is relevant to introduce this terminology because several
research threads do use these terms, and in our integrative effort we aim at
removing fictitious linguistic barriers. Indeed, in multi-agent systems, it is
frequent to talk about the micro-level of agent activity and the macro-level
of _agent societies/organisations_ (Wooldridge, 2009; Sawyer, 2003), and the
research thread of _macro-programming_ (Casadei, 2023b; Júnior et al., 2022)
is exactly where the forward and inverse problems are addressed by using
programming languages. Figure 1 depicts the general idea graphically, where
macro-observables are abstracted as functions of the inputs from the
individuals and environment, as in (He et al., 2019; Noël and Zambonelli,
2015).

Figure 1. The local/micro and global/macro levels of systems. The shapes at
the top are just abstract denotations of macroscopic information obtained by
an application-specific, abstracted function of the microscopic data taken
from the agents and environment (cf. (He et al., 2019; Gignoux et al., 2017)).
The stylised agents are not necessarily people but also artificial autonomous
agents.

In the traditional “node-centric” engineering viewpoint, there is an implicit
global-to-local solution that is limited to the designer’s mind or at the
requirements phase. In this paper, we analyze how recent research (Nicola et
al., 2020; Casadei, 2023b; Júnior et al., 2022; Beal et al., 2013; Pinciroli
and Beltrame, 2016; Mottola and Picco, 2011; Brambilla et al., 2013; Inverso
et al., 2020) shows that design can start from the “global” perspective,
ideally with declarative definitions that represent the intended global
behaviour, then implementing local behaviours by a formal global-to-local
mapping technique, and finally going forward for proper testing and validation
with a local-to-global mapping technique.

####  3.1.2. Automatic vs. manual design

Various works in the context of multi-agent systems and swarm robotics suggest
that techniques for the design of collective behaviour can be classified in
two main categories (Brambilla et al., 2013): automated techniques and manual
techniques.

_Automated_ design methods (Buchberger, 2023) work at the meta-level of
algorithms, defining procedures for learning or synthesis of the control
programs driving (the individual agents of) CCPEs. Prominent examples of this
family are _evolutionary methods_ (Trianni, 2008), _multi-agent reinforcement
learning_ (Busoniu et al., 2008), and _program synthesis_ (Gulwani et al.,
2017).

_Manual_ or _behaviour-based_ design methods, instead, include the classical
programming activity, where the programmer follows trial-and-error processes
to craft algorithms using general-purpose or domain-specific languages. When
addressing collective behaviour, this is also referred to as _macro-
programming_ (Casadei, 2023b) (discussed more deeply in Section 3.2.2).

Note that, being automated design the meta-level application of manual design,
these two classes are not necessarily disjoint: indeed, key research
opportunities lies in this overlap (Chaudhuri et al., 2021; Aguzzi et al.,
2022b, a; Yang et al., 2021).

####  3.1.3. Understanding and building “collective intelligence”

_ci_ can be generally intended as the property of groups of individuals
behaving in ways that seem intelligent (Malone and Bernstein, 2022). This
definition reuses possible definitions of “intelligence”, typically as a
collection of task-specific skills or, in a stronger sense, as a general
problem-solving and learning ability (Chollet, 2019). So, a CCPE is
intelligent if it is able to solve tasks in a variety of environments,
possibly adapting itself to cope with new problems.

Remark that the term “collective intelligence” leads to two main related
areas: socio-technical systems (Suran et al., 2021) and artificial collectives
(Casadei, 2023a). The former emphasises the complex collaboration of humans
and technologies, whereas the latter emphasises the engineering of multi-agent
systems consisting primarily of artificial devices. The two research areas are
related, and humans are indeed considered in the latter kind of systems as
well, but the problems addressed are quite different (roughly, more human-
computer interaction on one hand, and more behaviour control design on the
other hand). Since socio-technical systems tend to feature increased
complexity of agents and heterogeneity, hence leading to composites, in this
work we mainly focus on artificial collectives. We explicitly refer to the
interpretation of collective intelligence (CI) as _ computational collective
intelligence (CCI)_ (Szuba, 2001; Casadei, 2023a), meaning the intelligence
globally exhibited by groups of computing devices.

###  3.2. State of the Art in SE for CCPEs

It follows a non-exhaustive review of contributions to SE for CCPEs, aiming at
illustrating its main problems, methods, and techniques.

####  3.2.1. Learning/evolving collective behaviour

The automatic approach in CCPEs design has been a focal area for over 20
years, encompassing various fields such as swarm robotics (Brambilla et al.,
2013), traffic management, and crowd engineering. In this area, two main
approaches have been studied: _evolutionary approaches_ (Trianni, 2008) and _
reinforcement learning (RL)_ (Sutton and Barto, 2018). Both rely heavily on
simulation to provide necessary feedback for refining generated controllers.
Evolutionary approaches draw inspiration from natural evolution, starting with
a set of controllers that evolve based on simulation feedback. The design of a
controller is highly dependent on the scenario and desired complexity, using
finite state machines (Francesca et al., 2014), neural networks (Floreano and
Keller, 2010), and Boolean networks (Braccini et al., 2016). The state of the
art in this research area is attributed to AuToMoDe (Francesca et al., 2015),
a general framework for generating controllers for robot swarms.

In the realm of _collective RL_ (Li et al., 2022), initial efforts, such as
those by COIN (Wolpert and Tumer, 1999), focused on bottom-up CCI, aiming to
enable multiple agents to learn concurrently via RL. These early efforts
evolved towards prioritizing the collective as a primary entity, thus shifting
focus to designing individual agent controllers. For instance, a novel model
named swarMDP (Sosic et al., 2017) was introduced to simplify the modelling of
learning processes for swarm-like systems, characterised by homogeneous
entities (same controller) with a partial observability of the environment.
This model represents an initial top-down approach, where the controller,
possessing a global view, is then distributed into similar controllers for
individual robots.

In the state of the art, there are two main strategies. One strategy, aimed at
collaborative systems with few agents and exemplified by COMA (Foerster et
al., 2018) and MAPPO (Yu et al., 2022), addresses the individual behaviour of
agents that cooperate to achieve collective intelligence. Another strategy,
targetting more large-scale systems, let collective behaviour emerge through a
uniform controller, akin to swarMDP but leveraging more recent models like
mean-field games (Yang et al., 2018)—this is often known as _many-agent_
reinforcement learning (Yang, 2021). In the _centralised training and
decentralised execution_ approach (Gronauer and Diepold, 2022; Kraemer and
Banerjee, 2016), controllers are derived from a global perspective and then
distributed throughout the system. This methodology simplifies the learning of
collective behaviours and allows for the verification of outcomes during
simulation.

####  3.2.2. Macroprogramming

Macroprogramming is the umbrella term that gathers programming solutions for
defining and executing the macroscopic behaviour of software systems (Casadei,
2023b; Júnior et al., 2022). The definition is voluntarily broad, since it
mostly serves as a basis for integrating and finding commonalities among
different solutions for similar problems recurring in disparate domains, from
wireless sensor networks (Newton et al., 2005; Mottola and Picco, 2011) and
parallel computing (Tian et al., 2013) to swarm robotics (Pinciroli and
Beltrame, 2016) and software-defined networks (Kang et al., 2013). So, the
real question is not whether a programming solution is an instance of
macroprogramming but rather to _what extent it is so_ (Casadei, 2023b). In
particular, certain macro-programming approaches do expose _collective-level
abstractions_ , and programs can be read as instructions targetting not an
individual device, but rather a collective of devices, hence _the computing
machine is conceptually the whole distributed system of computers_. Achieving
that, often entails some forms of _declarativity_ at the language level, a
compilation of local programs from the global specification (cf. (Weisenburger
et al., 2020)), and a non-trivial interplay between the local/global programs
and the middleware (cf. Section 4.2).

For a detailed picture of the state of the art on macroprogramming, the reader
can refer to these two recent surveys (Casadei, 2023b; Júnior et al., 2022).
It should be noticed that the surveys do not cover research contributions that
are rather focussed on smaller-scale heterogeneous systems, i.e., composites,
and hence captured by the related fields of _choreographic programming_ (Cruz-
Filipe and Montesi, 2020), _multi-agent oriented programming_ (Boissier et
al., 2020), _architectural description languages_ (Malavolta et al., 2013),
and _multi-tier programming_ (Weisenburger et al., 2020).

####  3.2.3. Aggregate Computing

Aggregate computing is a prominent macroprogramming approach for CCPEs (Viroli
et al., 2019), developed for more than ten years. We briefly recall its
fundamental features: _bio-inspiration, formality, pragmatism,
compositionality, deployment-independence_.

Generally speaking, a major general theme in software engineering for CCPEs is
finding ways to map scientific theories about natural phenomena to software
engineering constructs (e.g., programming abstractions or platforms)—cf. _bio-
inspired_ mechanisms (Fernandez-Marquez et al., 2013). In the case of
aggregate computing, the mechanism of self-organisation (Gershenson et al.,
2020) is reflected in the execution model (implemented in the runtime or
middleware) of programs written in this paradigm, which assumes that each
device computes in repeated asynchronous sense–compute–interact rounds.

The programming paradigm is formally founded on _field calculi_ (Audrito et
al., 2019), functional core languages for expressing manipulations of
_computational fields_ : distributed data structures mapping sets of devices
to values, hence denoting collective inputs and outputs. This enables deriving
guarantees and proofs about programs, e.g. to assess that a collective
computation is _self-stabilising_ (roughly, eventually converging to the
“right” value) (Viroli et al., 2018).

These calculi are implemented by standalone or embedded _domain-specific
languages_ (Viroli et al., 2019), also providing pragmatic libraries of
reusable functions capturing recurrent patterns of collective behaviour. Using
these languages and libraries, the programmer defines a single program
expressing the whole behaviour of a collective as a _composition_ of building
blocks.

Then, the program can be deployed on a network of devices, using a simulator
or leveraging a proper middleware support on real systems (Casadei et al.,
2022). By distinguishing the logical model of the system from its physical
deployment, it is also possible to partition an application into deployable
components and hence offload computations across the edge-cloud continuum
(Casadei et al., 2022).

####  3.2.4. Models, Methods, and Tools for CCPEs

A few works provide early contributions addressing (parts of) the development
life-cycle for CCPEs, mainly from the research area of collective adaptive
systems (Nicola et al., 2020). Actually, the majority of these, briefly
reviewed in the following, lie at the intersection of composites and
collectives.

There is a number of contributions about the use of formal methods for the
specification and quantitative evaluation of collective adaptive systems
(Bernardo et al., 2016). One example is _CARMA (Collective Adaptive Resource-
sharing Markovian Agents)_ (Loreti and Hillston, 2016), a stochastic process
algebra supporting the modelling and verification, through a stochastic
simulator, of quantitative properties of collective behaviour like
performance, availability, dependability.

Another major source of contributions was the _ASCENS (Autonomic Service-
Component ENSembles)_ project (Wirsing et al., 2015). This project proposed
approaches and formal languages for the specification and analysis of
_ensembles_ , namely dynamic groups of devices exhibiting complex interactions
and working in complex environments (Wirsing et al., 2011). In the _SOTA
(State Of The Affairs)_ and _GEM (General Ensemble Model)_ approach
(Abeywickrama et al., 2020), collective behaviour is denoted as a trajectory
in a state space where each point denotes a single “state of the affairs”
comprising all the information about the ensemble and its environment.
Regarding the programming of ensembles, ASCENS features the _SCEL (Service
Component Ensemble Language)_ (Nicola et al., 2014), a process-algebraic
abstract language for specifying behaviour of individual entities, of
aggregations of entities, based on _attribute-based communication_ (Alrahman
et al., 2020), and in a way that is parametric to knowledge repositories and
adaptation policies.

A related approach to engineering dynamic ensembles is provided by Bucchiarone
et al. (Bucchiarone and Mongiello, 2019), based on _domain objects_
encapsulating behavioural processes and exporting _process fragments_ for
refinement and composition. These networks of domain objects (i.e., ensembles)
can be modelled using typed graph grammars (Hirsch et al., 1998).

A programming framework for hybrid collaborative systems is proposed by Scekic
et al. (Scekic et al., 2020). There, collectives are teams of _peers_
(abstracting _humans_ and _machines_) performing “collective-based tasks”,
e.g. in crowdsourcing settings. The approach, based on the SmartSociety
framework (Scekic et al., 2020), provides a Java-based platform and
Application Program Interface (API) for the _orchestration_ of such hybrid
ensembles, supporting _managed_ collectives, collective task management,
adaptation policies (expressed in terms of changing handlers attached to task
transitions), as well as other services for communication and incentive
management. The interesting part of the approach is the combination of
_collectiveness_ (where collectives are the first-class citizens managed by
the platform) and _human orchestration_ , and the positioning of the
contribution roughly half-way between composites and collectives engineering;
however the approach is quite ad-hoc and with limited methodological guidance.

There are other approaches similar to SCEL, domain objects, and SmartSociety
that share some of their motivation and abstractions, and provide related DSLs
and platforms supporting modelling, implementation, and limited forms of
verification. These include, e.g., _DEECo (Distributed Emergent Ensembles of
Components)_ (Bures et al., 2013).

Generally, any approach like aggregate computing, CARMA, or ASCENS comes with
its own toolchain, but there exist also “standalone” tools. One toolkit for
reasoning about collective adaptive systems is _Sibilla_ (Giudice et al.,
2022). It is a modular framework supporting multiple specification languages
(e.g., for interactive objects, population models, and agent-based systems),
organised with a three-layer architecture comprising: (i) a back-end
supporting modelling, simulation, and analysis; (ii) different front-ends
(APIs and shells), and (iii) a runtime system coordinating activities.

####  3.2.5. Methodologies for CCPEs

The aforementioned SOTA and GEM models promote _goal-oriented_ (Horkoff et
al., 2019) adaptation requirements engineering. However, there are also
studies on specific properties of CCPEs or peculiar aspects of their
engineering.

In (Ali et al., 2022), a systematic review is carried out studying how
existing approaches deal with _uncertainty_ and _adaptation_ in CCPEs. Also, a
_design guide_ is proposed for uncertainty-aware components in CCPEs,
highlighting the need of identifying uncertainty sources (e.g., failure,
delays, noise, missing information) and “border situations” to feed methods
for handling uncertainties (e.g., humans-in-the-loop, reconfiguration, and
declarative behaviour) before performing the adaptation decision-making.

Regarding engineering self-organisation and emergence, there exist some ideas
and high-level guidelines (Noël and Zambonelli, 2015; Prokopenko, 2013). For
instance, Nöel and Zambonelli (Noël and Zambonelli, 2015) contrast with
approaches based on micro-level design and rather suggest to focus on the
macro-level: _“the global behaviour must emerge, exploiting self-organisation
imposes some design constraints and decomposition plays an important role in
what can emerge”_. In particular, they delineate two different decomposition
strategies: _organisation-based_ (by distinguishing roles) and _functionality-
based_ (by breaking global tasks into global sub-tasks). In the former
strategy, literature on multi-agent organisational paradigms is especially
relevant (Horling and Lesser, 2004).

##  4\. SE Challenges for CCPE$\mathbf{s}$

In Section 3 we provided an overview of works contributing to the engineering
of CCPEs. At their core, there are notations and tools supporting automatic
and/or manual approaches helping to bridge the intended global behaviour with
the local control program of the agents of the CCPE at hand. The general goal
is promoting forms of collective intelligence out of whole networks of
devices. The key insight is that several research works (Nicola et al., 2020;
Casadei, 2023b; Júnior et al., 2022; Beal et al., 2013; Pinciroli and
Beltrame, 2016; Mottola and Picco, 2011; Brambilla et al., 2013; Inverso et
al., 2020) witness the need of embracing the collective viewpoint, possibly
throughout the software engineering process (Wirsing et al., 2015; Nicola et
al., 2020): cf. requirements expressing system-wide properties (Abeywickrama
et al., 2020), models and languages considering groups of entities and values
as first-class abstractions (Casadei, 2023b; Júnior et al., 2022; Inverso et
al., 2020), verification techniques leveraging model-checking or simulation to
assess the emergents (Loreti and Hillston, 2016; Giudice et al., 2022; Viroli
et al., 2018). In the following, we highlight gaps in the state of the art,
denoting key challenges to be addressed in the context of software engineering
of next-generation distributed systems.

###  4.1. Handling Homogeneity and Heterogeneity

Figure 2.  From composites to collectives. In composites (left) every entity
composing the system must be observed, understood, and designed/trained
individually; communication between entities needs to be modelled explicitly.
In collectives (right) all entities are observed, understood, and
designed/trained as a single entity; communication is implicit; dealing with
the heterogeneity of the single entities is still an active research topic.
Hybrid systems (centre) are possible: the system gets partitioned into
multiple subsystems, typically capturing homogeneous devices, building a de-
facto composite of collectives.

####  4.1.1. Integrating collectives and composites

A major challenge in engineering CCPEs is the integration of heterogeneous
entities into a coherent framework, where the system can be inspected,
understood, and designed (or trained) as a single entity (in the spirit of
macroprogramming, cf. Section 3.2.2). Macroprogramming approaches, in fact,
are particularly well-suited for systems composed of homogeneous entities,
where the collective behaviour can be abstracted and reasoned about at a high
level of abstraction. Thus far, however, there is a lack of formal and
practical methods to deal with heterogeneity from a collective perspective,
exploiting the peculiarities of every device without renouncing the benefit of
observing and designing the system as whole.

In Figure 2, we provide a graphical representation of the difference between
composites and collectives, including also somewhat _hybrid_ systems, where
the whole composed of heterogeneous devices is partitioned into multiple
subsystems based on their internal homogeneity. Although this idea helps with
managing complexity, it does not fully exploit the potential of a collective
viewpoint over a heterogeneous system. Consider, for instance, a system
composed of drones and ground robots: a truly collective approach would be
able to form mixed teams with capabilities that can only be achieved by
combining the two types of devices (for instance, exploiting the aerial view
in specific areas to assist ground robots in navigation and path-finding);
while a hybrid approach would partition the system into two homogeneous
communicating subsystems, losing the ability to create multiple small
independent teams.

Providing means to tame heterogeneity without losing the privileged
observation position of the collective has great potential to enable a leap
forward in several fields, including swarm robotics (Dorigo et al., 2020;
Kengyel et al., 2015), the IoT (Qiu et al., 2018; Casadei and Viroli, 2018),
and the edge-cloud continuum (Ferrer et al., 2021).

####  4.1.2. Multi-targeting and software product lines

The theme of heterogeneity, largely impacting on how abstractions can be
captured (cf. Section 4.2), has rippling effects on the software development
process. In fact, even though the software is designed from a collective
perspective, it will have to be executed on possibly very diverse devices,
ranging from edge servers and cloud instances to smartphones and wearables to
modest microcontrollers and sensors. Although decompositional approaches have
been proposed to tackle the problem of some devices being unable to
participate in some parts of the collective computation (Casadei et al.,
2020), they do not account for the production of multiple versions of the
software that can execute on very diverse devices. This problem can be
fractioned into two macro sub-problems:  (1) producing software _compatible_
with different target platforms from a single specification, and  (2)
customising the specific software configuration considering the final target.
The first problem imposes a significant maintenance burden on potential
middlewares (as they must be able to run on multiple platforms), potential
external DSLs (as they must be able to generate code for multiple targets),
and may severely limit the adoption of internal DSLs (as they are constrained
by the platforms targeted by the host language). The latter problem calls for
future CCPE-specific software product lines (ter Beek and Schaefer, 2023),
supporting the customisation of the final software in a target-specific
fashion.

###  4.2. Specifying and Running Collectives: DSLs, Middlewares, and Their
Interplay

Capturing collectives in a way that is actionable analysts and designers
requires appropriate abstractions to be reified. Thus, at some abstraction
level, multiple devices must operate as a single one, although in the respect
of their possible heterogeneity. Doing so, technically, can be performed in
two ways:

  * •

through a middleware, creating a software layer that abstracts the collective
as a single entity; or

  * •

through a DSL, capturing the collective behaviour at design time and
translating it into executable specifications.

Middlewares are the historically preferred approach to building a shared layer
multiple autonomous entities can interact through (Mamei and Zambonelli, 2009;
Poggi et al., 2002). Indeed, a middleware is a natural solution, as it
provides a shared unified layer, enforcing homogeneity over possibly
heterogeneous devices. This strategy has been used in the past for non-
distributed system with success: we may ascribe to the “middleware” category
all those execution runtimes that could be targeted by multiple programming
languages, such as the Java Virtual Machine (JVM) (targeted by Java, Ruby,
Scala, Kotlin, Groovy, and many other languages) or the Common Language
Runtime (CLR) (targeted by VisualBasic.NET, C#, F#, …). On the other hand,
however, middlewares can be impractical, as they normally require multiple
implementations for different platforms, and they impose certain requirements
on the underlying systems. In the exemplary case of the JVM (which is akin in
other runtimes) the middleware has to be implemented for each supported
platform, with each implementation possibly featuring slightly different
implementations. We observe that there is a trade-off between complexity and
portability: the middleware’s portability will be favoured by minimality; on
the other hand, a minimal middleware will hardly provide high-level
abstractions suitable for high-complexity systems, thus requiring further
layers, each one possibly imposing further restrictions on the underlying
platform. For instance, a middleware developed on top of the JVM will not be
capable to be executed on devices or runtimes for which a JVM is not
available, such as the browser, or low-power wearables.

DSLs come instead with a different set of trade-offs. The idea behind a DSLs
is to create new abstractions as part of the language itself, letting the
compiler (interpreter) figure out how to translate (execute) the program. DSLs
can be realised in two ways: as _standalone_ (or _external_) languages (Riti,
2017a), or _embedded_ (_internal_) into an existing General-Purpose Language
(GPL) (Riti, 2017b). The former approach is more flexible, but it requires
more development and maintenance effort, while the latter is constrained by
the syntax and semantics of the host GPL, but, in turn, can immediately
leverage its compiler/interpreter, its tooling, and feature a much gentler
learning curve for designers acquainted with the host GPL; an internal DSL, de
facto, is a library designed to exploit the syntactic features of the host
language in such a way that the feeling is akin to using a dedicated language.

The two approaches are not mutually exclusive, rather, they reinforce each
other. In fact, at some point, the code written in a DSL will perform
communications over a network; and these communications could well be mediated
by a middleware. On the other hand, the infrastructural support a middleware
provides will be exposed to the designers through an API, which may be
expressed, e.g., in form of an internal DSL. In practice, we have a spectrum
of possibilities, and understanding the better trade-off from the two extremes
of a pure DSL directly calling networking primitives under the hood and a pure
middleware driven through a regular API is yet to be investigated. This
investigation becomes even more challenging under the consideration that
different applications (with different contexts and objectives) developed by
different teams (with different expertises) may benefit from different
balances, and one specific design will hardly fit well in all conditions.
Clear guidance directed to CCPE DSL and middleware providers on the trade-off
each design choice brings on the table is yet to be devised.

###  4.3. Verification and Validation

#### Static analysis.

A first line of defence against bugs, smells, and vulnerabilities is static
analysis; however, the adoption of static methods based on model checking is
limited by the size, openness, and autonomy of CCPEs (Lim et al., 2015). In
response, formal methods are being developed to support CCPE-tailored analysis
(Stefano and Lang, 2023; Castiglioni et al., 2023). Further steps are required
in this direction, and, from a practitioner’s point of view, these research
efforts need to be reified into actionable tools (possibly language- or
middleware- specific).

#### Simulation.

In distributed systems, testing is a main challenge (Rafajlowicz, 2008):
executing the software on a single machine (even when software pieces are
isolated through Virtual Machines or containers) attenuates (or removes
altogether) many of the potential issues to be faced at runtime, such as
network failures, slowdowns, Content Delivery Network (CDN) updates, and
internal Domain Name System (DNS) lookup issues. In the case of CCPEs, the
problem is exacerbated by the large scale, which makes it unpractical (when
plainly impossible) to test the software on a single machine. The most
immediate consequence is that _simulation_ becomes a necessity in multiple
phases of the software production (Afzal et al., 2021): fast simulator with a
very simplified model of the world can support the developer in the early
stages of development; while more detailed simulations, with accurate physics
and network models (and much longer time to execute) are needed to support in-
depth testing (Arcaini et al., 2021). Capturing different degrees of
complexity with a single simulator is a significant challenge (Afzal et al.,
2020), we envision that using multiple simulators for different phases of the
development could be a viable solution. Orthogonally to scale, the dynamicity
of CCPEs, in which violations of the nominal states are generally
unpredictable, is a major challenge for simulation (Afzal et al., 2020):
generating critical scenarios automatically is another interesting challenge
ahead.

#### Test Cases: Inputs.

CCPEs are made of (possibly very large) groups of computational entities,
which require multiple levels of testing, as multiagent systems (Clark et al.,
2021). We can think of (at least) the following levels: single entity;
neighbouring entities; global system. The main challenges are posed by the
third level, since the number of possible system states is huge, and in the
test cases it is necessary to include states that trigger all the relevant
global system behaviour, e.g., an important self-adaptation capability (Lim et
al., 2015). The decision about which global system states should be actually
brought out by test cases is a difficult trade-off between keeping the number
of such states reasonably small, and covering all the relevant situations
(Siqueira et al., 2021). In order to find a good trade-off in a
(semi-)automatic way, the availability of a formal system model and/or of
formal requirements is a promising direction to be pursued.

#### Test Cases: Outcomes.

The nature of CCPEs poses challenges also for the determination of the test
outcome, in terms of the equality between an _expected_ result (test oracle),
and the _actual_ result of applying certain inputs to a certain system state.
The (automatic) computation of the expected result requires the availability
of a formal specification, similarly as the determination of relevant test
inputs discussed above (Siqueira et al., 2021). However, also the
determination of the actual result is not trivial, for at least two reasons.
First, the global outcome of the test must be determined as the value of a
formula that must be computed by potentially taking into account the local
states of a large number of system entities; in order to compute it in a
distributed way, techniques such as distributed Runtime Verification (RV) may
be adopted (Audrito et al., 2022). Second, the resulting global state may take
some time to materialize due to delays introduced by the communications
between the CCPE entities. Even when a CCPE behaviour is self-stabilising
(Viroli et al., 2018), then, it is key to allow enough time for the
stabilisation before finalising the test result.

#### Monitoring and runtime verification.

As the runtime execution of a CCPEs is subject to external and often
unpredictable influences, a-priori SE methods are usually not sufficient to
guarantee the desired outcomes. Therefore, they have to be complemented by
runtime monitoring approaches, both as separate tools flagging when manual
intervention is needed, and as integrated parts of the system triggering
automatic reactions. However, several challenges arise when performing runtime
monitoring on CCPE. A first challenge is designing appropriate specification
logics, that need to balance two opposite forces: expressiveness and
simplicity. On expressiveness, a logic should capture enough relevant
properties of systems situated in space and evolving over time. On simplicity,
it should allow an intuitive understanding to enhance its usability, also
among domain experts not proficient in modal logics. This tension has led to
many different logics being developed so far (Ma et al., 2021; Nenzi et al.,
2022; Audrito et al., 2022, 2021), and leaves room for improvement in future
works.

A further challenge is computing monitor results from CCPE data. Some
approaches such as have been restricted so far to _offline monitoring_ ,
performed remotely on a complete trace of events gathered from the whole
historical data of a CCPE, limiting their applicability in real-time (Nenzi et
al., 2022). Other approaches allow for _centralised online monitoring_ (Ma et
al., 2021), still requiring to gather all events on a single remote computer,
but allowing to compute monitor results in real-time from such event traces.
Limited research has been carried out so far on _decentralised monitoring_
(Audrito et al., 2022, 2021), in which monitors are integrated within the
monitored application and run on the same distributed devices. Each device
computes its own view of the monitored properties according to the knowledge
available to it, without need for a central coordinator. Although this
approach allows for greater resilience and faster reaction times of monitors,
it poses additional challenges that have so far limited the monitored logic
expressiveness. Future works could improve the current expressiveness limits
in this context.

###  4.4. Integration with Humans

An important characteristic of CCPEs is that they typically involve both
physical components (such as sensors, robots, drones, …) as well as humans.
The interaction between humans and the other cyber-physical entities can be
either direct (e.g., through voice and gestures directly captured with
microphones and cameras), or indirect, with the human interacting through
wearable devices. Two scenarios (of increasing complexity) of integration
between humans and CCPEs can be envisioned and pose peculiar challenges.

#### Humans controlling CCPEs.

Humans and physical devices interact with strict, rigid asymmetric roles. We
can consider humans as external, although strictly connected with the CCPE. In
this scenario, the humans typically request services from the CCPEs, and/or
perform some control over them (Villani et al., 2020). In a variation of the
scenario, instead of commands, the humans may provide relevant inputs by
carrying wearables or smartphones, e.g. in healthcare (Calvaresi et al.,
2019).

One crucial aspect in this kind of systems is the cognitive load affecting the
human(s) in charge of controlling the CCPE, especially when the latter
contains many elements, as in a swarm (Paas et al., 2022). Another important
aspect is the communication modality between humans and devices (e.g.,
robots), which can involve speech, gesture, haptics (if humans and devices are
physically close), or screen-based UIs and Virtual Reality (if the humans and
devices are remote) (Dahiya et al., 2023).

The main challenges for this kind of scenario are thus mainly related with
Human-Machine Interface (HMI), and in particular multi-modal interfaces that
improve the human user effectiveness and comfort. A fundamental issue that is
related with HMI is the level of autonomy of the CCPE w.r.t. the human: the
_right_ level of autonomy should be ideally adjusted dynamically to maximize
the performance of the human high-level control over the CCPE (Mostafa et al.,
2019).

#### Humans as CCPE Entities.

Humans and physical devices act as a single CCPE. In this kind of scenario,
there isn’t a rigid hierarchy where humans give (high-level) commands and CCPE
entities collectively execute them; instead, the roles of humans and robots
are equal, or at least overlapping, and (partially) interchangeable.
Interesting applications of these hybrid CCPEs include industrial processes
(Jost et al., 2017; Zhang et al., 2021; Guo, 2024), as well as rescue missions
(Habib et al., 2018). In order to fully take part into the CCPE, humans
typically have to carry some devices (smartphones, wearables, …) used to
provide inputs and outputs from/to the human. The main challenge posed by this
kind of systems is that they should run collective algorithms that take into
account and exploit the differences between humans and artificial entities
(e.g., intelligence, or speed), while at the same time considering them as
(partially) interchangeable elements of the system. This means that the system
must have a more or less sophisticated model of its entities, and adapt its
decisions to the involved entities.

###  4.5. Learning collective behaviour

Despite years of research into synthesizing collective behaviours, finding an
effective and scalable method to address the challenges associated with
learning in such systems remains a complex issue (Hernandez-Leal et al.,
2019). The intricacy of multi-agent systems escalates significantly as the
number of agents increases (Hernandez-Leal et al., 2018; Du and Ding, 2021);
this amplifies the complexity of the system’s dynamics, complicating the
process of discerning the influence of individual actions on the collective
outcome. Below, we delineate some paramount challenges associated with
learning in collective behaviour.

#### Multi-agent Credit Assignment

A paramount issue within multi-agent systems is the _multi-agent credit
assignment_ challenge, as highlighted by Nguyen et al. (Nguyen et al., 2018).
This problem revolves around the precise identification and rewarding of
individual agent actions that contribute positively towards the collective
goal. Techniques such as COMA and difference rewards (Tumer and Agogino, 2007)
have been developed to tackle this challenge. However, these approaches often
entail a significant computational burden, rendering them less viable for
deployment in large-scale systems. The core strategy of these methods involves
calculating rewards for agents by assessing the impact of their actions on the
collective outcome— a process that requires simulating the system’s state with
and without the agent’s action to find the differential effect (Agogino and
Tumer, 2004). This dual-simulation approach, while conceptually appealing for
its accuracy in attributing credit, is notably resource-intensive and may not
scale efficiently with the complexity or size of the system (Foerster et al.,
2018).

#### Online Learning

In such systems, agents should learn in real-time, adapting to changes in the
environment and the behaviour of other agents without leveraging a central
trainer. However, unlike single-agent systems, where the environment typically
exhibits stationary characteristics, online learning in multi-agent systems
introduces a non-stationary environment (Hernandez-Leal et al., 2018; Du and
Ding, 2021; Busoniu et al., 2008). This complexity makes it difficult to
determine whether changes are due to environmental responses or agent
behaviour change, potentially leading to undesirable emergent behaviours and
loss of control over collective learning (Yang and Wang, 2020; Oroojlooy and
Hajinezhad, 2023). Thus, moving from the current state of the art, which
mainly focuses on offline learning with a centralised training/decentralised
execution approach (Gronauer and Diepold, 2022; Kraemer and Banerjee, 2016),
to decentralised online learning is a crucial challenge to be further
investigated (Du and Ding, 2021).

#### Transfer learning

Another aspect of learning involves the _transfer_ of knowledge from one
domain to another. The capability to apply knowledge acquired in one context
to another, but similar, context presents a considerable challenge (da Silva
and Costa, 2019). Even if several solutions have been proposed so far to
address this issue (Boutsioukis et al., 2011), the complexity of transfer
learning in CCPEs underscores the need for novel approaches to facilitate
knowledge transfer across contexts effectively. Indeed, in multiagent systems
there are several axes is to be handled, such as the communication between
agents, the different roles of agents, the different capabilities of agents,
and the different environments in which agents operate (da Silva and Costa,
2019). In single agent learning, a novel trend consist in leveraging large
pre-trained models (i.e., _foundational_ models) to facilitate learning in new
tasks (Reed et al., 2022). The adoption of foundational models in CCPEs could
mark a shift toward more efficient and effective learning mechanisms,
suggesting a promising avenue for future exploration.

##  5\. Conclusion

In this paper, we presented challenges, current research efforts, and future
research directions for a collective computing paradigm in software
engineering. We showed the relevance of this paradigm given the growing
interest in large-scale cyber-physical ecosystems. By reviewing key themes and
relevant contributions in CCPEs engineering, we categorised current research
efforts on both techniques, abstractions, and tools. From this, we identified
the challenges ahead that will require innovative solutions. These include the
handling of homogeneity and heterogeneity; the development of DSLs, middleware
and tools; the bridging of global and local perspectives in system analysis,
design, and implementation; the integration with humans; and collective
learning approaches. Addressing these challenges will be essential for
unlocking the full potential of CCPEs and advancing the state of the art in
software engineering cyber-physical ecosystems research.

## Acknowledgements

This work has been partially supported by the Italian MUR PRIN 2020 Project
“COMMON-WEARS” (2020HCWWLP).

## References

  * (1)
  * Abeywickrama et al. (2020) Dhaminda B. Abeywickrama, Nicola Bicocchi, Marco Mamei, and Franco Zambonelli. 2020.  The SOTA approach to engineering collective adaptive systems.  _Int. J. Softw. Tools Technol. Transf._ 22, 4 (2020), 399–415.  https://doi.org/10.1007/S10009-020-00554-3
  * Abowd (2016) Gregory D. Abowd. 2016\.  Beyond Weiser: From Ubiquitous to Collective Computing.  _Computer_ 49, 1 (2016), 17–23.  https://doi.org/10.1109/MC.2016.22
  * Afzal et al. (2020) Afsoon Afzal, Claire Le Goues, Michael Hilton, and Christopher Steven Timperley. 2020\.  A Study on Challenges of Testing Robotic Systems. In _2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)_. 96–107.  https://doi.org/10.1109/ICST46399.2020.00020
  * Afzal et al. (2021) Afsoon Afzal, Deborah S. Katz, Claire Le Goues, and Christopher S. Timperley. 2021.  Simulation for Robotics Test Automation: Developer Perspectives. In _2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)_. 263–274.  https://doi.org/10.1109/ICST49551.2021.00036
  * Agogino and Tumer (2004) Adrian K. Agogino and Kagan Tumer. 2004.  Unifying Temporal and Structural Credit Assignment Problems. In _3rd International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2004), 19-23 August 2004, New York, NY, USA_. IEEE Computer Society, 980–987.  https://doi.org/10.1109/AAMAS.2004.10098
  * Aguzzi et al. (2022a) Gianluca Aguzzi, Roberto Casadei, and Mirko Viroli. 2022a.  Machine Learning for Aggregate Computing: a Research Roadmap. In _42nd IEEE International Conference on Distributed Computing Systems, ICDCS Workshops, Bologna, Italy, July 10, 2022_. IEEE, 119–124.  https://doi.org/10.1109/ICDCSW56584.2022.00032
  * Aguzzi et al. (2022b) Gianluca Aguzzi, Roberto Casadei, and Mirko Viroli. 2022b.  Towards Reinforcement Learning-based Aggregate Computing. In _Coordination Models and Languages - 24th IFIP WG 6.1 International Conference, COORDINATION 2022, Proceedings_ _(Lecture Notes in Computer Science, Vol. 13271)_. Springer, 72–91.  https://doi.org/10.1007/978-3-031-08143-9_5
  * Ali et al. (2022) Rima Al Ali, Lubomír Bulej, Jan Kofron, and Tomás Bures. 2022.  A guide to design uncertainty-aware self-adaptive components in Cyber-Physical Systems.  _Future Gener. Comput. Syst._ 128 (2022), 466–489.  https://doi.org/10.1016/J.FUTURE.2021.10.027
  * Alrahman et al. (2020) Yehia Abd Alrahman, Rocco De Nicola, and Michele Loreti. 2020\.  Programming interactions in collective adaptive systems by relying on attribute-based communication.  _Sci. Comput. Program._ 192 (2020), 102428.  https://doi.org/10.1016/J.SCICO.2020.102428
  * Alwateer et al. (2019) Majed Alwateer, Seng W. Loke, and Niroshinie Fernando. 2019\.  Enabling Drone Services: Drone Crowdsourcing and Drone Scripting.  _IEEE Access_ 7 (2019), 110035–110049.  https://doi.org/10.1109/ACCESS.2019.2933234
  * Arcaini et al. (2021) Paolo Arcaini, Xiao-Yi Zhang, and Fuyuki Ishikawa. 2021\.  Targeting Patterns of Driving Characteristics in Testing Autonomous Driving Systems. In _2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)_. 295–305.  https://doi.org/10.1109/ICST49551.2021.00042
  * Audrito et al. (2021) Giorgio Audrito, Roberto Casadei, Ferruccio Damiani, Volker Stolz, and Mirko Viroli. 2021\.  Adaptive distributed monitors of spatial properties for cyber-physical systems.  _J. Syst. Softw._ 175 (2021), 110908.  https://doi.org/10.1016/J.JSS.2021.110908
  * Audrito et al. (2022) Giorgio Audrito, Ferruccio Damiani, Volker Stolz, Gianluca Torta, and Mirko Viroli. 2022.  Distributed runtime verification by past-CTL and the field calculus.  _J. Syst. Softw._ 187 (2022), 111251.  https://doi.org/10.1016/j.jss.2022.111251
  * Audrito et al. (2019) Giorgio Audrito, Mirko Viroli, Ferruccio Damiani, Danilo Pianini, and Jacob Beal. 2019\.  A Higher-Order Calculus of Computational Fields.  _ACM Trans. Comput. Logic_ 20, 1, Article 5 (jan 2019), 55 pages.  https://doi.org/10.1145/3285956
  * Beal et al. (2013) Jacob Beal, Stefan Dulman, Kyle Usbeck, Mirko Viroli, and Nikolaus Correll. 2013.  Organizing the Aggregate: Languages for Spatial Computing.  In _Formal and Practical Aspects of Domain-Specific Languages: Recent Developments_. IGI Global, Chapter 16, 436–501.  https://doi.org/10.4018/978-1-4666-2092-6.ch016
  * Becker et al. (2019) Christian Becker, Christine Julien, Philippe Lalanda, and Franco Zambonelli. 2019.  Pervasive computing middleware: current trends and emerging challenges.  _CCF Trans. Pervasive Comput. Interact._ 1, 1 (2019), 10–23.  https://doi.org/10.1007/S42486-019-00005-2
  * Bellman et al. (2021) Kirstie L. Bellman, Jean Botev, Ada Diaconescu, Lukas Esterle, Christian Gruhl, Christopher Landauer, Peter R. Lewis, Phyllis R. Nelson, Evangelos Pournaras, Anthony Stein, and Sven Tomforde. 2021.  Self-improving system integration: Mastering continuous change.  _Future Gener. Comput. Syst._ 117 (2021), 29–46.  https://doi.org/10.1016/J.FUTURE.2020.11.019
  * Bernardo et al. (2016) Marco Bernardo, Rocco De Nicola, and Jane Hillston (Eds.). 2016\.  _Formal Methods for the Quantitative Evaluation of Collective Adaptive Systems - 16th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2016, Bertinoro, Italy, June 20-24, 2016, Advanced Lectures_. Lecture Notes in Computer Science, Vol. 9700. Springer.  https://doi.org/10.1007/978-3-319-34096-8
  * Boissier et al. (2020) Olivier Boissier, Rafael H Bordini, Jomi Hubner, and Alessandro Ricci. 2020.  _Multi-agent oriented programming: programming multi-agent systems using JaCaMo_.  MIT Press.
  * Bonabeau et al. (1999) Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. 1999\.  _Swarm intelligence: from natural to artificial systems_.  Oxford university press.
  * Boutsioukis et al. (2011) Georgios Boutsioukis, Ioannis Partalas, and Ioannis P. Vlahavas. 2011.  Transfer Learning in Multi-Agent Reinforcement Learning Domains. In _Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Athens, Greece, September 9-11, 2011, Revised Selected Papers_ _(Lecture Notes in Computer Science, Vol. 7188)_ , Scott Sanner and Marcus Hutter (Eds.). Springer, 249–260.  https://doi.org/10.1007/978-3-642-29946-9_25
  * Braccini et al. (2016) Michele Braccini, Andrea Roli, Marco Villani, and Roberto Serra. 2016.  Automatic Design of Boolean Networks for Cell Differentiation. In _Advances in Artificial Life, Evolutionary Computation, and Systems Chemistry - 11th Italian Workshop, WIVACE 2016, Fisciano, Italy, October 4-6, 2016, Revised Selected Papers_ _(Communications in Computer and Information Science, Vol. 708)_. 91–102.  https://doi.org/10.1007/978-3-319-57711-1_8
  * Brambilla et al. (2013) Manuele Brambilla, Eliseo Ferrante, Mauro Birattari, and Marco Dorigo. 2013.  Swarm robotics: a review from the swarm engineering perspective.  _Swarm Intell._ 7, 1 (2013), 1–41.  https://doi.org/10.1007/S11721-012-0075-2
  * Broy and Schmidt (2014) Manfred Broy and Albrecht Schmidt. 2014.  Challenges in Engineering Cyber-Physical Systems.  _Computer_ 47, 2 (2014), 70–72.  https://doi.org/10.1109/MC.2014.30
  * Bucchiarone and Mongiello (2019) Antonio Bucchiarone and Marina Mongiello. 2019.  Ten Years of Self-adaptive Systems: From Dynamic Ensembles to Collective Adaptive Systems. In _From Software Engineering to Formal Methods and Tools, and Back_ _(Lecture Notes in Computer Science, Vol. 11865)_. Springer, 19–39.  https://doi.org/10.1007/978-3-030-30985-5_3
  * Buchberger (2023) Bruno Buchberger. 2023\.  Automated programming, symbolic computation, machine learning: my personal view.  _Ann. Math. Artif. Intell._ 91, 5 (2023), 569–589.  https://doi.org/10.1007/S10472-023-09894-7
  * Bures et al. (2013) Tomás Bures, Ilias Gerostathopoulos, Petr Hnetynka, Jaroslav Keznikl, Michal Kit, and Frantisek Plásil. 2013.  DEECO: an ensemble-based component system. In _CBSE’13, Proceedings of the 16th ACM SIGSOFT Symposium on Component Based Software Engineering_. ACM, 81–90.  https://doi.org/10.1145/2465449.2465462
  * Busoniu et al. (2008) Lucian Busoniu, Robert Babuska, and Bart De Schutter. 2008\.  A Comprehensive Survey of Multiagent Reinforcement Learning.  _IEEE Trans. Syst. Man Cybern. Part C_ 38, 2 (2008), 156–172.
  * Calvaresi et al. (2019) Davide Calvaresi, Mauro Marinoni, Aldo Franco Dragoni, Roger Hilfiker, and Michael Schumacher. 2019\.  Real-time multi-agent systems for telerehabilitation scenarios.  _Artificial Intelligence in Medicine_ 96 (2019), 217–231.  https://doi.org/10.1016/j.artmed.2019.02.001
  * Casadei (2023a) Roberto Casadei. 2023a.  Artificial Collective Intelligence Engineering: A Survey of Concepts and Perspectives.  _Artif. Life_ 29, 4 (2023), 433–467.  https://doi.org/10.1162/ARTL_A_00408
  * Casadei (2023b) Roberto Casadei. 2023b.  Macroprogramming: Concepts, State of the Art, and Opportunities of Macroscopic Behaviour Modelling.  _ACM Comput. Surv._ 55, 13s (2023), 275:1–275:37.  https://doi.org/10.1145/3579353
  * Casadei et al. (2022) Roberto Casadei, Giancarlo Fortino, Danilo Pianini, Andrea Placuzzi, Claudio Savaglio, and Mirko Viroli. 2022.  A Methodology and Simulation-Based Toolchain for Estimating Deployment Performance of Smart Collective Services at the Edge.  _IEEE Internet Things J._ 9, 20 (2022), 20136–20148.  https://doi.org/10.1109/JIOT.2022.3172470
  * Casadei et al. (2020) Roberto Casadei, Danilo Pianini, Andrea Placuzzi, Mirko Viroli, and Danny Weyns. 2020.  Pulverization in Cyber-Physical Systems: Engineering the Self-Organizing Logic Separated from Deployment.  _Future Internet_ 12, 11 (2020), 203.  https://doi.org/10.3390/FI12110203
  * Casadei and Viroli (2018) Roberto Casadei and Mirko Viroli. 2018.  Collective Abstractions and Platforms for Large-Scale Self-Adaptive IoT. In _2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W), Trento, Italy, September 3-7, 2018_. IEEE, 106–111.  https://doi.org/10.1109/FAS-W.2018.00033
  * Cassandras and Lafortune (2021) Christos G. Cassandras and Stephane Lafortune. 2021.  _Introduction to Discrete Event Systems_ (3nd ed.).  Springer Publishing Company, Incorporated.
  * Castiglioni et al. (2023) Valentina Castiglioni, Michele Loreti, and Simone Tini. 2023\.  A framework to measure the robustness of programs in the unpredictable environment.  _Log. Methods Comput. Sci._ 19, 3 (2023).  https://doi.org/10.46298/LMCS-19(3:2)2023
  * Chaudhuri et al. (2021) Swarat Chaudhuri, Kevin Ellis, Oleksandr Polozov, Rishabh Singh, Armando Solar-Lezama, and Yisong Yue. 2021.  Neurosymbolic Programming.  _Found. Trends Program. Lang._ 7, 3 (2021), 158–243.  https://doi.org/10.1561/2500000049
  * Cheng et al. (2016) Bo Cheng, Da Zhu, Shuai Zhao, and Junliang Chen. 2016\.  Situation-Aware IoT Service Coordination Using the Event-Driven SOA Paradigm.  _IEEE Trans. Netw. Serv. Manag._ 13, 2 (2016), 349–361.  https://doi.org/10.1109/TNSM.2016.2541171
  * Chollet (2019) François Chollet. 2019\.  On the measure of intelligence.  _arXiv preprint arXiv:1911.01547_ (2019).
  * Clark et al. (2021) Andrew G. Clark, Neil Walkinshaw, and Robert M. Hierons. 2021\.  Test case generation for agent-based models: A systematic literature review.  _Information and Software Technology_ 135 (2021).  https://doi.org/10.1016/j.infsof.2021.106567
  * Conti et al. (2012) Marco Conti, Sajal K. Das, Chatschik Bisdikian, Mohan Kumar, Lionel M. Ni, Andrea Passarella, George Roussos, Gerhard Tröster, Gene Tsudik, and Franco Zambonelli. 2012\.  Looking ahead in pervasive computing: Challenges and opportunities in the era of cyber-physical convergence.  _Pervasive Mob. Comput._ 8, 1 (2012), 2–21.  https://doi.org/10.1016/J.PMCJ.2011.10.001
  * Coullon et al. (2023) Hélène Coullon, Ludovic Henrio, Frédéric Loulergue, and Simon Robillard. 2023\.  Component-Based Distributed Software Reconfiguration: A Verification-Oriented Survey.  _ACM Comput. Surv._ (may 2023).  https://doi.org/10.1145/3595376
  * Cruz-Filipe and Montesi (2020) Luís Cruz-Filipe and Fabrizio Montesi. 2020.  A core model for choreographic programming.  _Theor. Comput. Sci._ 802 (2020), 38–66.  https://doi.org/10.1016/j.tcs.2019.07.005
  * da Silva and Costa (2019) Felipe Leno da Silva and Anna Helena Reali Costa. 2019.  A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems.  _J. Artif. Intell. Res._ 64 (2019), 645–703.  https://doi.org/10.1613/JAIR.1.11396
  * Dahiya et al. (2023) Abhinav Dahiya, Alexander M. Aroyo, Kerstin Dautenhahn, and Stephen L. Smith. 2023.  A survey of multi-agent Human-Robot Interaction systems.  _Robotics and Autonomous Systems_ 161 (2023), 104335.  https://doi.org/10.1016/j.robot.2022.104335
  * de Assunção et al. (2018) Marcos Dias de Assunção, Alexandre Da Silva Veith, and Rajkumar Buyya. 2018.  Distributed data stream processing and edge computing: A survey on resource elasticity and future directions.  _J. Netw. Comput. Appl._ 103 (2018), 1–17.  https://doi.org/10.1016/J.JNCA.2017.12.001
  * Dharmawan et al. (2019) Audelia Gumarus Dharmawan, Gim Song Soh, Shaohui Foong, Roland Bouffanais, and Kristin L. Wood. 2019.  Design innovation of mesoscale robotic swarms: applications to cooperative urban sensing and mapping.  _Frontiers Inf. Technol. Electron. Eng._ 20, 12 (2019), 1618–1631.  https://doi.org/10.1631/FITEE.1900384
  * Dorigo et al. (2020) Marco Dorigo, Guy Theraulaz, and Vito Trianni. 2020\.  Reflections on the future of swarm robotics.  _Science Robotics_ 5, 49 (Dec. 2020), eabe4385.  https://doi.org/10.1126/scirobotics.abe4385
  * Dragoni et al. (2017) Nicola Dragoni, Saverio Giallorenzo, Alberto Lluch-Lafuente, Manuel Mazzara, Fabrizio Montesi, Ruslan Mustafin, and Larisa Safina. 2017\.  Microservices: Yesterday, Today, and Tomorrow.  In _Present and Ulterior Software Engineering_. Springer, 195–216.  https://doi.org/10.1007/978-3-319-67425-4_12
  * Du and Ding (2021) Wei Du and Shifei Ding. 2021.  A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications.  _Artif. Intell. Rev._ 54, 5 (2021), 3215–3238.  https://doi.org/10.1007/S10462-020-09938-Y
  * Fernandez-Marquez et al. (2013) Jose Luis Fernandez-Marquez, Giovanna Di Marzo Serugendo, Sara Montagna, Mirko Viroli, and Josep Lluís Arcos. 2013.  Description and composition of bio-inspired design patterns: a complete overview.  _Nat. Comput._ 12, 1 (2013), 43–67.  https://doi.org/10.1007/S11047-012-9324-Y
  * Ferrer et al. (2021) Ana Juan Ferrer, Sören Becker, Florian Schmidt, Lauritz Thamsen, and Odej Kao. 2021\.  Towards a Cognitive Compute Continuum: An Architecture for Ad-Hoc Self-Managed Swarms. In _21st IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2021_. IEEE, 634–641.  https://doi.org/10.1109/CCGRID51090.2021.00076
  * Ferscha et al. (2014) Alois Ferscha, Paul Lukowicz, and Franco Zambonelli. 2014\.  The superorganism of massive collective wearables. In _Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’14 Adjunct Publication_. ACM, 1077–1084.  https://doi.org/10.1145/2638728.2659396
  * Floreano and Keller (2010) Dario Floreano and Laurent Keller. 2010.  Evolution of Adaptive Behaviour in Robots by Means of Darwinian Selection.  _PLoS Biology_ 8, 1 (Jan. 2010), e1000292.  https://doi.org/10.1371/journal.pbio.1000292
  * Foerster et al. (2018) Jakob N. Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2018\.  Counterfactual Multi-Agent Policy Gradients. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18)_. AAAI Press, 2974–2982.  https://doi.org/10.1609/AAAI.V32I1.11794
  * Francesca et al. (2015) Gianpiero Francesca, Manuele Brambilla, Arne Brutschy, Lorenzo Garattoni, Roman Miletitch, Gaëtan Podevijn, Andreagiovanni Reina, Touraj Soleymani, Mattia Salvaro, Carlo Pinciroli, Franco Mascia, Vito Trianni, and Mauro Birattari. 2015\.  AutoMoDe-Chocolate: automatic design of control software for robot swarms.  _Swarm Intell._ 9, 2-3 (2015), 125–152.  https://doi.org/10.1007/S11721-015-0107-9
  * Francesca et al. (2014) Gianpiero Francesca, Manuele Brambilla, Arne Brutschy, Vito Trianni, and Mauro Birattari. 2014\.  AutoMoDe: A novel approach to the automatic design of control software for robot swarms.  _Swarm Intell._ 8, 2 (2014), 89–112.  https://doi.org/10.1007/S11721-014-0092-4
  * Gershenson et al. (2020) Carlos Gershenson, Vito Trianni, Justin Werfel, and Hiroki Sayama. 2020.  Self-Organization and Artificial Life.  _Artif. Life_ 26, 3 (2020), 391–408.  https://doi.org/10.1162/artl_a_00324
  * Gignoux et al. (2017) Jacques Gignoux, Guillaume Chérel, Ian D. Davies, Shayne R. Flint, and Eric Lateltin. 2017.  Emergence and complex systems: The contribution of dynamic graph theory.  _Ecological Complexity_ 31 (Sept. 2017), 34–49.  https://doi.org/10.1016/j.ecocom.2017.02.006
  * Giudice et al. (2022) Nicola Del Giudice, Lorenzo Matteucci, Michela Quadrini, Aniqa Rehman, and Michele Loreti. 2022\.  Sibilla: A Tool for Reasoning about Collective Systems. In _Coordination Models and Languages - 24th IFIP WG 6.1 International Conference, COORDINATION 2022, Proceedings_ _(LNCS, Vol. 13271)_. Springer, 92–98.  https://doi.org/10.1007/978-3-031-08143-9_6
  * Gronauer and Diepold (2022) Sven Gronauer and Klaus Diepold. 2022.  Multi-agent deep reinforcement learning: a survey.  _Artif. Intell. Rev._ 55, 2 (2022), 895–943.  https://doi.org/10.1007/S10462-021-09996-W
  * Gulwani et al. (2017) Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. 2017\.  Program Synthesis.  _Found. Trends Program. Lang._ 4, 1-2 (2017), 1–119.  https://doi.org/10.1561/2500000010
  * Guo (2024) Daqiang Guo. 2024\.  Fast scheduling of human-robot teams collaboration on synchronised production-logistics tasks in aircraft assembly.  _Robotics and Computer-Integrated Manufacturing_ 85 (2024).  https://doi.org/10.1016/j.rcim.2023.102620
  * Habib et al. (2018) Lydia Habib, Marie-Pierre Pacaux-Lemoine, and Patrick Millot. 2018\.  Human-Robots Team Cooperation in Crisis Management Mission. In _2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)_. 3219–3224.  https://doi.org/10.1109/SMC.2018.00545
  * He et al. (2019) Feijuan He, Yudai Pan, Qika Lin, Xianglin Miao, and Zhouguo Chen. 2019.  Collective Intelligence: A Taxonomy and Survey.  _IEEE Access_ 7 (2019), 170213–170225.  https://doi.org/10.1109/ACCESS.2019.2955677
  * Hernandez-Leal et al. (2018) Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. 2018\.  Is multiagent deep reinforcement learning the answer or the question? A brief survey.  _CoRR_ abs/1810.05587 (2018).  arXiv:1810.05587 http://arxiv.org/abs/1810.05587
  * Hernandez-Leal et al. (2019) Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. 2019\.  A survey and critique of multiagent deep reinforcement learning.  _Auton. Agents Multi Agent Syst._ 33, 6 (2019), 750–797.  https://doi.org/10.1007/S10458-019-09421-1
  * Hirsch et al. (1998) Dan Hirsch, Paola Inverardi, and Ugo Montanari. 1998\.  Graph grammars and constraint solving for software architecture styles. In _Proceedings of the Third International Workshop on Software Architecture, ISAW ’98_. ACM, 69–72.  https://doi.org/10.1145/288408.288426
  * Hoque et al. (2022) Mohammad Aminul Hoque, Md. Mahmud Hossain, Shahid Al Noor, S. M. Riazul Islam, and Ragib Hasan. 2022\.  IoTaaS: Drone-Based Internet of Things as a Service Framework for Smart Cities.  _IEEE Internet Things J._ 9, 14 (2022), 12425–12439.  https://doi.org/10.1109/JIOT.2021.3137362
  * Horkoff et al. (2019) Jennifer Horkoff, Fatma Basak Aydemir, Evellin Cardoso, Tong Li, Alejandro Maté, Elda Paja, Mattia Salnitri, Luca Piras, John Mylopoulos, and Paolo Giorgini. 2019.  Goal-oriented requirements engineering: an extended systematic mapping study.  _Requir. Eng._ 24, 2 (2019), 133–160.  https://doi.org/10.1007/S00766-017-0280-Z
  * Horling and Lesser (2004) Bryan Horling and Victor R. Lesser. 2004.  A survey of multi-agent organizational paradigms.  _Knowl. Eng. Rev._ 19, 4 (2004), 281–316.  https://doi.org/10.1017/S0269888905000317
  * Inverso et al. (2020) Omar Inverso, Catia Trubiani, and Emilio Tuosto. 2020\.  Abstractions for Collective Adaptive Systems. In _Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings, Part II_ _(LNCS, Vol. 12477)_. Springer, 243–260.  https://doi.org/10.1007/978-3-030-61470-6_15
  * Jost et al. (2017) Jana Jost, Thomas Kirks, and Benedikt Mättig. 2017.  Multi-agent systems for decentralized control and adaptive interaction between humans and machines for industrial environments. In _2017 7th IEEE International Conference on System Engineering and Technology (ICSET)_. 95–100.  https://doi.org/10.1109/ICSEngT.2017.8123427
  * Júnior et al. (2022) Iwens Gervásio Sene Júnior, Thalia S. Santana, Renato F. Bulcão-Neto, and Barry Porter. 2022\.  The state of the art of macroprogramming in IoT: An update.  _J. Internet Serv. Appl._ 13, 1 (2022), 54–65.  https://doi.org/10.5753/JISA.2022.2372
  * Kang et al. (2013) Nanxi Kang, Zhenming Liu, Jennifer Rexford, and David Walker. 2013\.  Optimizing the ”one big switch” abstraction in software-defined networks. In _Conference on emerging Networking Experiments and Technologies (CoNEXT’13), Proceedings of_. ACM, 13–24.  https://doi.org/10.1145/2535372.2535373
  * Kengyel et al. (2015) Daniela Kengyel, Heiko Hamann, Payam Zahadat, Gerald Radspieler, Franz Wotawa, and Thomas Schmickl. 2015.  Potential of Heterogeneity in Collective Behaviors: A Case Study on Heterogeneous Swarms. In _PRIMA 2015: Principles and Practice of Multi-Agent Systems - 18th International Conference, Proceedings_ _(LNCS, Vol. 9387)_. Springer, 201–217.  https://doi.org/10.1007/978-3-319-25524-8_13
  * Kephart and Chess (2003) Jeffrey O. Kephart and David M. Chess. 2003.  The Vision of Autonomic Computing.  _Computer_ 36, 1 (2003), 41–50.  https://doi.org/10.1109/MC.2003.1160055
  * Kraemer and Banerjee (2016) Landon Kraemer and Bikramjit Banerjee. 2016.  Multi-agent reinforcement learning as a rehearsal for decentralized planning.  _Neurocomputing_ 190 (2016), 82–94.  https://doi.org/10.1016/J.NEUCOM.2016.01.031
  * Lemos et al. (2016) Angel Lagares Lemos, Florian Daniel, and Boualem Benatallah. 2016.  Web Service Composition: A Survey of Techniques and Tools.  _ACM Comput. Surv._ 48, 3 (2016), 33:1–33:41.  https://doi.org/10.1145/2831270
  * Li et al. (2022) Meng Li, Pan Pei, F. Richard Yu, Pengbo Si, Yu Li, Enchang Sun, and Yanhua Zhang. 2022.  Cloud-Edge Collaborative Resource Allocation for Blockchain-Enabled Internet of Things: A Collective Reinforcement Learning Approach.  _IEEE Internet Things J._ 9, 22 (2022), 23115–23129.  https://doi.org/10.1109/JIOT.2022.3185289
  * Li et al. (2015) Shancang Li, Li Da Xu, and Shanshan Zhao. 2015.  The internet of things: a survey.  _Inf. Syst. Frontiers_ 17, 2 (2015), 243–259.  https://doi.org/10.1007/S10796-014-9492-7
  * Lim et al. (2015) Yoo Jin Lim, Eunkyoung Jee, Donghwan Shin, and Doo-Hwan Bae. 2015.  Efficient Testing of Self-Adaptive Behaviors in Collective Adaptive Systems. In _39th IEEE Annual Computer Software and Applications Conference, COMPSAC 2015, Volume 2_. IEEE Computer Society, 216–221.  https://doi.org/10.1109/COMPSAC.2015.131
  * Loreti and Hillston (2016) Michele Loreti and Jane Hillston. 2016.  Modelling and Analysis of Collective Adaptive Systems with CARMA and its Tools. In _Formal Methods for the Quantitative Evaluation of Collective Adaptive Systems - 16th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2016, Advanced Lectures_ _(LNCS, Vol. 9700)_. Springer, 83–119.  https://doi.org/10.1007/978-3-319-34096-8_4
  * Ma et al. (2021) Meiyi Ma, Ezio Bartocci, Eli Lifland, John A. Stankovic, and Lu Feng. 2021.  A Novel Spatial-Temporal Specification-Based Monitoring System for Smart Cities.  _IEEE Internet Things J._ 8, 15 (2021), 11793–11806.  https://doi.org/10.1109/JIOT.2021.3069943
  * Malavolta et al. (2013) Ivano Malavolta, Patricia Lago, Henry Muccini, Patrizio Pelliccione, and Antony Tang. 2013\.  What Industry Needs from Architectural Languages: A Survey.  _IEEE Trans. Software Eng._ 39, 6 (2013), 869–891.  https://doi.org/10.1109/TSE.2012.74
  * Malone and Bernstein (2022) Thomas W Malone and Michael S Bernstein. 2022.  _Handbook of collective intelligence_.  MIT press.
  * Mamei and Zambonelli (2009) Marco Mamei and Franco Zambonelli. 2009.  Programming pervasive and mobile computing applications: The TOTA approach.  _ACM Trans. Softw. Eng. Methodol._ 18, 4 (2009), 15:1–15:56.  https://doi.org/10.1145/1538942.1538945
  * Masolo et al. (2020) Claudio Masolo, Laure Vieu, Roberta Ferrario, Stefano Borgo, and Daniele Porello. 2020.  Pluralities, Collectives, and Composites. In _Formal Ontology in Information Systems - Proceedings of the 11th International Conference, FOIS 2020_ _(Frontiers in Artificial Intelligence and Applications, Vol. 330)_. IOS Press, 186–200.  https://doi.org/10.3233/FAIA200671
  * Mobus and Kalton (2015) George E. Mobus and Michael C. Kalton. 2015.  _Principles of Systems Science_.  Springer.
  * Montesi (2023) Fabrizio Montesi. 2023\.  _Introduction to Choreographies_.  Cambridge University Press.
  * Mostafa et al. (2019) Salama A Mostafa, Mohd Sharifuddin Ahmad, and Aida Mustapha. 2019.  Adjustable autonomy: a systematic literature review.  _Artificial Intelligence Review_ 51 (2019), 149–186.
  * Mottola and Picco (2011) Luca Mottola and Gian Pietro Picco. 2011.  Programming wireless sensor networks: Fundamental concepts and state of the art.  _ACM Comput. Surv._ 43, 3 (2011), 19:1–19:51.  https://doi.org/10.1145/1922649.1922656
  * Murgia et al. (2023) Maurizio Murgia, Riccardo Pinciroli, Catia Trubiani, and Emilio Tuosto. 2023.  Comparing perfomance abstractions for collective adaptive systems.  _Int. J. Softw. Tools Technol. Transf._ 25, 5 (2023), 785–798.  https://doi.org/10.1007/S10009-023-00728-9
  * Nenzi et al. (2022) Laura Nenzi, Ezio Bartocci, Luca Bortolussi, and Michele Loreti. 2022.  A Logic for Monitoring Dynamic Networks of Spatially-distributed Cyber-Physical Systems.  _Log. Methods Comput. Sci._ 18, 1 (2022).  https://doi.org/10.46298/LMCS-18(1:4)2022
  * Newton et al. (2005) Ryan Newton, Arvind, and Matt Welsh. 2005.  Building up to macroprogramming: an intermediate language for sensor networks. In _Proceedings of the 4th International Symposium on Information Processing in Sensor Networks, IPSN 2005_. IEEE, 37–44.  https://doi.org/10.1109/IPSN.2005.1440891
  * Nguyen et al. (2018) Duc Thien Nguyen, Akshat Kumar, and Hoong Chuin Lau. 2018\.  Credit Assignment For Collective Multiagent RL With Global Rewards. In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018_. 8113–8124.
  * Nicola et al. (2020) Rocco De Nicola, Stefan Jähnichen, and Martin Wirsing. 2020\.  Rigorous engineering of collective adaptive systems: special section.  _Int. J. Softw. Tools Technol. Transf._ 22, 4 (2020), 389–397.  https://doi.org/10.1007/s10009-020-00565-0
  * Nicola et al. (2014) Rocco De Nicola, Michele Loreti, Rosario Pugliese, and Francesco Tiezzi. 2014.  A Formal Approach to Autonomic Systems Programming: The SCEL Language.  _ACM Trans. Auton. Adapt. Syst._ 9, 2 (2014), 7:1–7:29.  https://doi.org/10.1145/2619998
  * Noël and Zambonelli (2015) Victor Noël and Franco Zambonelli. 2015.  Methodological Guidelines for Engineering Self-organization and Emergence.  In _Software Engineering for Collective Autonomic Systems - The ASCENS Approach_. Lecture Notes in Computer Science, Vol. 8998. Springer, 355–378.  https://doi.org/10.1007/978-3-319-16310-9_10
  * Orgerie et al. (2013) Anne-Cécile Orgerie, Marcos Dias de Assunção, and Laurent Lefèvre. 2013.  A survey on techniques for improving the energy efficiency of large-scale distributed systems.  _ACM Comput. Surv._ 46, 4 (2013), 47:1–47:31.  https://doi.org/10.1145/2532637
  * Oroojlooy and Hajinezhad (2023) Afshin Oroojlooy and Davood Hajinezhad. 2023.  A review of cooperative multi-agent deep reinforcement learning.  _Appl. Intell._ 53, 11 (2023), 13677–13722.  https://doi.org/10.1007/S10489-022-04105-Y
  * Paas et al. (2022) Anita Paas, Emily B. J. Coffey, Giovanni Beltrame, and David St-Onge. 2022.  Towards evaluating the impact of swarm robotic control strategy on operators’ cognitive load. In _2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)_. 217–223.  https://doi.org/10.1109/RO-MAN53752.2022.9900763
  * Pandey (2010) R. K. Pandey. 2010\.  Architectural description languages (ADLs) vs UML: a review.  _ACM SIGSOFT Softw. Eng. Notes_ 35, 3 (2010), 1–5.  https://doi.org/10.1145/1764810.1764828
  * Peltz (2003) Chris Peltz. 2003\.  Web Services Orchestration and Choreography.  _Computer_ 36, 10 (2003), 46–52.  https://doi.org/10.1109/MC.2003.1236471
  * Pianini et al. (2021) Danilo Pianini, Roberto Casadei, Mirko Viroli, and Antonio Natali. 2021.  Partitioned integration and coordination via the self-organising coordination regions pattern.  _Future Gener. Comput. Syst._ 114 (2021), 44–68.  https://doi.org/10.1016/J.FUTURE.2020.07.032
  * Pinciroli and Beltrame (2016) Carlo Pinciroli and Giovanni Beltrame. 2016.  Buzz: A Programming Language for Robot Swarms.  _IEEE Softw._ 33, 4 (2016), 97–100.  https://doi.org/10.1109/MS.2016.95
  * Poggi et al. (2002) Agostino Poggi, Giovanni Rimassa, and Paola Turci. 2002\.  What Agent Middleware Can (And Should) Do For You.  _Appl. Artif. Intell._ 16, 9-10 (2002), 677–698.  https://doi.org/10.1080/08839510290030444
  * Prokopenko (2013) M. Prokopenko. 2013\.  _Guided Self-Organization: Inception_.  Springer Berlin Heidelberg.
  * Qiu et al. (2018) Tie Qiu, Ning Chen, Keqiu Li, Mohammed Atiquzzaman, and Wenbing Zhao. 2018.  How Can Heterogeneous Internet of Things Build Our Future: A Survey.  _IEEE Commun. Surv. Tutorials_ 20, 3 (2018), 2011–2027.  https://doi.org/10.1109/COMST.2018.2803740
  * Rafajlowicz (2008) Ewaryst Rafajlowicz. 2008\.  Testing Homogeneity of Coefficients in Distributed Systems With Application to Quality Monitoring.  _IEEE Trans. Control. Syst. Technol._ 16, 2 (2008), 314–321.  https://doi.org/10.1109/TCST.2007.903398
  * Reed et al. (2022) Scott E. Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. 2022.  A Generalist Agent.  _Trans. Mach. Learn. Res._ 2022 (2022).  https://openreview.net/forum?id=1ikK0kHjvj
  * Riti (2017a) Pierluigi Riti. 2017a.  _External DSL_.  Apress, 59–69.  https://doi.org/10.1007/978-1-4842-3036-7_4
  * Riti (2017b) Pierluigi Riti. 2017b.  _Internal DSL_.  Apress, 45–57.  https://doi.org/10.1007/978-1-4842-3036-7_3
  * Rodríguez et al. (2007) Alejandro Rodríguez, Alexander Grushin, and James A. Reggia. 2007.  Swarm Intelligence Systems Using Guided Self-Organization for Collective Problem Solving.  _Adv. Complex Syst._ 10, supp01 (2007), 5–34.  https://doi.org/10.1142/S0219525907001069
  * Sawyer (2003) R. Keith Sawyer. 2003\.  Artificial Societies: Multiagent Systems and the Micro-Macro Link in Sociological Theory.  _Sociological Methods &; Research_ 31, 3 (Feb. 2003), 325–363.  https://doi.org/10.1177/0049124102239079
  * Scekic et al. (2020) Ognjen Scekic, Tommaso Schiavinotto, Svetoslav Videnov, Michael Rovatsos, Hong Linh Truong, Daniele Miorandi, and Schahram Dustdar. 2020\.  A Programming Model for Hybrid Collaborative Adaptive Systems.  _IEEE Trans. Emerg. Top. Comput._ 8, 1 (2020), 6–19.  https://doi.org/10.1109/TETC.2017.2702578
  * Siqueira et al. (2021) Bento R. Siqueira, Fabiano C. Ferrari, Kathiani E. Souza, Valter V. Camargo, and Rogério de Lemos. 2021.  Testing of adaptive and context-aware systems: approaches and challenges.  _Software Testing, Verification and Reliability_ 31, 7 (2021).  https://doi.org/10.1002/stvr.1772
  * Sosic et al. (2017) Adrian Sosic, Wasiur R. KhudaBukhsh, Abdelhak M. Zoubir, and Heinz Koeppl. 2017.  Inverse Reinforcement Learning in Swarm Systems.  (2017), 1413–1421.  http://dl.acm.org/citation.cfm?id=3091320
  * Stefano and Lang (2023) Luca Di Stefano and Frédéric Lang. 2023\.  Compositional Verification of Stigmergic Collective Systems. In _Verification, Model Checking, and Abstract Interpretation - 24th International Conference, VMCAI 2023, Proceedings_ _(LNCS, Vol. 13881)_. Springer, 155–176.  https://doi.org/10.1007/978-3-031-24950-1_8
  * Suran et al. (2021) Shweta Suran, Vishwajeet Pattanaik, and Dirk Draheim. 2021\.  Frameworks for Collective Intelligence: A Systematic Literature Review.  _ACM Comput. Surv._ 53, 1 (2021), 14:1–14:36.  https://doi.org/10.1145/3368986
  * Suran et al. (2022) Shweta Suran, Vishwajeet Pattanaik, Ralf H. J. M. Kurvers, Carina Antonia Hallin, Anna De Liddo, Robert Krimmer, and Dirk Draheim. 2022\.  Building Global Societies on Collective Intelligence: Challenges and Opportunities.  _Digit. Gov. Res. Pract._ 3, 4 (2022), 31:1–31:6.  https://doi.org/10.1145/3568169
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement Learning_ (2 ed.).  Bradford Books, Cambridge, MA.
  * Szuba (2001) Tadeusz M Szuba. 2001\.  _Computational collective intelligence_.  John Wiley & Sons, Inc.
  * Tanenbaum and van Steen (2007) Andrew S. Tanenbaum and Maarten van Steen. 2007.  _Distributed systems - principles and paradigms, 2nd Edition_.  Pearson Education.
  * ter Beek and Schaefer (2023) Maurice H. ter Beek and Ina Schaefer. 2023.  Systems and software product lines of the future.  _J. Syst. Softw._ 199 (2023), 111622.  https://doi.org/10.1016/J.JSS.2023.111622
  * Tian et al. (2013) Yuanyuan Tian, Andrey Balmin, Severin Andreas Corsten, Shirish Tatikonda, and John McPherson. 2013\.  From ”Think Like a Vertex” to ”Think Like a Graph”.  _Proc. VLDB Endow._ 7, 3 (2013), 193–204.  https://doi.org/10.14778/2732232.2732238
  * Trianni (2008) Vito Trianni. 2008\.  _Evolutionary Swarm Robotics - Evolving Self-Organising Behaviours in Groups of Autonomous Robots_. Studies in Computational Intelligence, Vol. 108.  Springer.
  * Tumer and Agogino (2007) Kagan Tumer and Adrian K. Agogino. 2007.  Distributed agent-based air traffic flow management. In _6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007)_. IFAAMAS, 255.  https://doi.org/10.1145/1329125.1329434
  * Tumer and Wolpert (2004) Kagan Tumer and David H Wolpert. 2004.  A Survey of Collectives.  In _Collectives and the design of complex systems_ , Kagan Tumer and David H Wolpert (Eds.). Springer Science & Business Media, Chapter 1.
  * van Steen et al. (2012) Maarten van Steen, Guillaume Pierre, and Spyros Voulgaris. 2012.  Challenges in very large distributed systems.  _J. Internet Serv. Appl._ 3, 1 (2012), 59–66.  https://doi.org/10.1007/S13174-011-0043-X
  * Vaquero et al. (2019) Luis M. Vaquero, Félix Cuadrado, Yehia Elkhatib, Jorge Bernal Bernabé, Satish Narayana Srirama, and Mohamed Faten Zhani. 2019.  Research challenges in nextgen service orchestration.  _Future Gener. Comput. Syst._ 90 (2019), 20–38.  https://doi.org/10.1016/J.FUTURE.2018.07.039
  * Villani et al. (2020) V. Villani, B. Capelli, and C. et al. Secchi. 2020.  Humans interacting with multi-robot systems: a natural affect-based approach.  _Auton Robot_ 44 (2020), 601–616.  https://doi.org/10.1007/s10514-019-09889-6
  * Viroli et al. (2018) Mirko Viroli, Giorgio Audrito, Jacob Beal, Ferruccio Damiani, and Danilo Pianini. 2018.  Engineering Resilient Collective Adaptive Systems by Self-Stabilisation.  _ACM Trans. Model. Comput. Simul._ 28, 2 (2018), 16:1–16:28.  https://doi.org/10.1145/3177774
  * Viroli et al. (2019) Mirko Viroli, Jacob Beal, Ferruccio Damiani, Giorgio Audrito, Roberto Casadei, and Danilo Pianini. 2019\.  From distributed coordination to field calculus and aggregate computing.  _J. Log. Algebraic Methods Program._ 109 (2019).  https://doi.org/10.1016/J.JLAMP.2019.100486
  * Weisenburger et al. (2020) Pascal Weisenburger, Johannes Wirth, and Guido Salvaneschi. 2020\.  A Survey of Multitier Programming.  _ACM Comput. Surv._ 53, 4 (2020), 81:1–81:35.  https://doi.org/10.1145/3397495
  * Weyns (2020) Danny Weyns. 2020\.  _An introduction to self-adaptive systems: A contemporary software engineering perspective_.  John Wiley & Sons.
  * Wirsing et al. (2015) Martin Wirsing, Matthias M. Hölzl, Nora Koch, and Philip Mayer (Eds.). 2015.  _Software Engineering for Collective Autonomic Systems - The ASCENS Approach_. Lecture Notes in Computer Science, Vol. 8998.  Springer.  https://doi.org/10.1007/978-3-319-16310-9
  * Wirsing et al. (2011) Martin Wirsing, Matthias M. Hölzl, Mirco Tribastone, and Franco Zambonelli. 2011\.  ASCENS: Engineering Autonomic Service-Component Ensembles. In _Formal Methods for Components and Objects, 10th International Symposium, FMCO 2011, Revised Selected Papers_ _(LNCS, Vol. 7542)_. Springer, 1–24.  https://doi.org/10.1007/978-3-642-35887-6_1
  * Wolpert and Tumer (1999) David H. Wolpert and Kagan Tumer. 1999.  An Introduction to Collective Intelligence.  https://doi.org/10.48550/ARXIV.CS/9908014
  * Wood and Galton (2009) Zena Wood and Antony Galton. 2009.  A taxonomy of collective phenomena.  _Appl. Ontology_ 4, 3-4 (2009), 267–292.  https://doi.org/10.3233/AO-2009-0071
  * Wooldridge (2009) Michael J. Wooldridge. 2009\.  _An Introduction to MultiAgent Systems, Second Edition_.  Wiley.
  * Xu et al. (2018) Hansong Xu, Wei Yu, David W. Griffith, and Nada Golmie. 2018\.  A Survey on Industrial Internet of Things: A Cyber-Physical Systems Perspective.  _IEEE Access_ 6 (2018), 78238–78259.  https://doi.org/10.1109/ACCESS.2018.2884906
  * Yang (2021) Yaodong Yang. 2021\.  _Many-agent reinforcement learning_.  Ph. D. Dissertation. UCL (University College London).
  * Yang et al. (2021) Yichen Yang, Jeevana Priya Inala, Osbert Bastani, Yewen Pu, Armando Solar-Lezama, and Martin C. Rinard. 2021.  Program Synthesis Guided Reinforcement Learning for Partially Observed Environments. In _Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021_. 29669–29683.
  * Yang et al. (2018) Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Weinan Zhang, and Jun Wang. 2018\.  Mean Field Multi-Agent Reinforcement Learning. In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018_ _(Proceedings of Machine Learning Research, Vol. 80)_. PMLR, 5567–5576.  http://proceedings.mlr.press/v80/yang18d.html
  * Yang and Wang (2020) Yaodong Yang and Jun Wang. 2020.  An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective.  _CoRR_ abs/2011.00583 (2020).  arXiv:2011.00583 https://arxiv.org/abs/2011.00583
  * Ye et al. (2017) Dayong Ye, Minjie Zhang, and Athanasios V. Vasilakos. 2017\.  A Survey of Self-Organization Mechanisms in Multiagent Systems.  _IEEE Trans. Syst. Man Cybern. Syst._ 47, 3 (2017), 441–461.  https://doi.org/10.1109/TSMC.2015.2504350
  * Ye et al. (2022) Peijun Ye, Yuanyuan Chen, Fenghua Zhu, Yisheng Lv, Wanze Lu, and Fei-Yue Wang. 2022\.  Bridging the Micro and Macro: Calibration of Agent-Based Model Using Mean-Field Dynamics.  _IEEE Trans. Cybern._ 52, 11 (2022), 11397–11406.  https://doi.org/10.1109/TCYB.2021.3089712
  * Yu et al. (2022) Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre M. Bayen, and Yi Wu. 2022.  The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games. In _Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022_.  http://papers.nips.cc/paper_files/paper/2022/hash/9c1535a02f0ce079433344e14d910597-Abstract-Datasets_and_Benchmarks.html
  * Zhang et al. (2021) Zequn Zhang, Dunbing Tang, and Qingwei Nie. 2021.  Research on Workers Integration in Smart Factories With Multi-Agent Control System.  _IEEE Access_ 9 (2021).  https://doi.org/10.1109/ACCESS.2021.3115339

  *[IoT]: Internet of Things
  *[CCI]: computational collective intelligence
  *[CCPEs]: collective cyber-physical ecosystem
  *[SE]: software engineering
  *[CCPE]: collective cyber-physical ecosystem
  *[ECC]: edge-cloud continuum
  *[CI]: collective intelligence
  *[RL]: reinforcement learning
  *[ RL]: reinforcement learning
  *[API]: Application Program Interface
  *[DSLs]: domain-specific language
  *[APIs]: Application Program Interface
  *[DSL]: domain-specific language
  *[JVM]: Java Virtual Machine
  *[CLR]: Common Language Runtime
  *[GPL]: General-Purpose Language
  *[CDN]: Content Delivery Network
  *[DNS]: Domain Name System
  *[RV]: Runtime Verification
  *[HMI]: Human-Machine Interface