# Deep Learning for Effective and Efficient Reduction of Large Adaptation
Spaces in Self-Adaptive Systems

Danny Weyns  Katholieke Universiteit Leuven, BelgiumLinnaeus University Sweden
<EMAIL_ADDRESS>,  Omid Gheibi  Katholieke Universiteit LeuvenBelgium
<EMAIL_ADDRESS>,  Federico Quin  Katholieke Universiteit
LeuvenBelgium<EMAIL_ADDRESS>and  Jeroen Van Der Donckt  Ghent
University (imec)Belgium<EMAIL_ADDRESS>
(2021)

###### Abstract.

Many software systems today face uncertain operating conditions, such as
sudden changes in the availability of resources or unexpected user behavior.
Without proper mitigation these uncertainties can jeopardize the system goals.
Self-adaptation is a common approach to tackle such uncertainties. When the
system goals may be compromised, the self-adaptive system has to select the
best adaptation option to reconfigure by analyzing the possible adaptation
options, i.e., the adaptation space. Yet, analyzing large adaptation spaces
using rigorous methods can be resource- and time-consuming, or even be
infeasible. One approach to tackle this problem is by using online machine
learning to reduce adaptation spaces. However, existing approaches require
domain expertise to perform feature engineering to define the learner, and
support online adaptation space reduction only for specific goals. To tackle
these limitations, we present ”Deep Learning for Adaptation Space Reduction
Plus” – DLASeR+ in short. DLASeR+ offers an extendable learning framework for
online adaptation space reduction that does not require feature engineering,
while supporting three common types of adaptation goals: threshold,
optimization, and set-point goals. We evaluate DLASeR+ on two instances of an
Internet-of-Things application with increasing sizes of adaptation spaces for
different combinations of adaptation goals. We compare DLASeR+ with a baseline
that applies exhaustive analysis and two state-of-the-art approaches for
adaptation space reduction that rely on learning. Results show that DLASeR+ is
effective with a negligible effect on the realization of the adaptation goals
compared to an exhaustive analysis approach, and supports three common types
of adaptation goals beyond the state-of-the-art approaches.

self-adaptation, adaptation space reduction, analysis, planning, deep
learning, threshold goals, optimization goal, set-point goal, Internet-of-
Things

††copyright: acmcopyright††journalyear: 2021††journal: TAAS††ccs: Software and
its engineering Software design engineering††ccs: Theory of computation Online
learning algorithms

##  1\. Introduction

Many software systems today face changing and uncertain operating conditions.
For such systems, employing a stationary approach that does not adapt to
changes may jeopardize the quality goals of the system. Consider for instance
a web-based system that runs a fixed number of servers based on an average
load. This configuration will result in a waste of resources when the load is
very low, but, the number of servers may be insufficient to handle the peak
demand (Klein et al., 2014).

Self-adaptation is one prominent approach to tackle such problems (Cheng et
al., 2009a; Weyns, 2017). A self-adaptive system reasons about itself and its
environment, based on observations, to determine whether adaptation is
required. In the case that adaptation is required, the system adapts itself in
order to meet its adaptation goals, or gracefully degrade if the goals may
temporarily not be achievable. Self-adaptation has been applied in a wide
range of application domains, ranging from service-based systems to cyber-
physical systems, Internet-of-Things, the Cloud, and robotics (Muccini et al.,
2016; Castañeda et al., 2014; Athreya et al., 2013; Edwards et al., 2009;
Jamshidi et al., 2016). In the example of the web-based system above,
enhancing the system with self-adaptation enables it to increase and decrease
the number of servers dynamically based on the monitored load. This results in
higher user satisfaction as well as improved economical and ecological use of
resources.

In this research, we apply architecture-based adaptation (Garlan et al., 2004;
Kramer and Magee, 2007; Weyns et al., 2012) that adds an external feedback
loop to the system. The feedback loop uses up-to-date architectural models as
first-class citizens to reason about changes. The feedback loop forms a
managing system on top of managed software system and is structured according
to the MAPE-K reference model, short for Monitor - Analyzer - Planner -
Executor - Knowledge (Kephart and Chess, 2003). MAPE-K divides adaptation in
four principle functions (Kephart and Chess, 2003; Weyns et al., 2013). The
monitor monitors the system and its environment. The analyzer determines
whether adaptation is required or not and if so it analyzes the adaptation
options for adapting the system. An adaptation option is a configuration of
the system that can be reached from the current configuration by changing
elements of the system through adaptation actions. Adaptation actions can
range from adjusting a parameter of the system up to an architectural re-
configuration of the system. We use the term adaptation space as the set of
all the possible adaptation options at some point in time, i.e., all the
possible configurations that can be reached from the current configuration of
the system by applying a set of adaptation actions to the system. The size of
the adaptation space (i.e., the number of adaptation options) may be constant
over time, or it may change dynamically. The planner then selects the best
adaptation option according to the adaptation goals and composes a plan for
adapting the managed system. Finally, the executor applies the adaptation
actions of the plan on the managed system. The four MAPE functions share
common knowledge (K), e.g., models of the system, the adaptation goals, the
set of adaptation options, an adaptation plan, among others.

This paper focuses on the analysis of the adaptation options of the adaptation
space, which is a task of the analyzer, and selecting the best option based on
the analysis results and the adaptation goals, which is a task of the planner.
Both tasks are essential for the decision-making in self-adaptive systems.
During the execution of these tasks the feedback loop estimates a set of
quality properties for each adaptation option of the adaptation space; each
quality property corresponding to an adaptation goal. We consider threshold
goals that require a system parameter to stay above/below a given threshold,
optimization goals that require to minimize or maximize a system parameter,
and set-point goals that require a system parameter to stay as close as
possible to a given value.

Selecting an adaptation option from a large adaptation space is often
computationally expensive (Cheng et al., 2009a; De Lemos et al., 2017; Weyns
et al., 2017). A common technique used to find the best adaptation option is
runtime verification of formal runtime models that represent the system and
its environment for one or more quality properties. These quality models have
parameters that can be instantiated for a particular adaptation option and the
actual values of the uncertainties. A classic approach to estimate the
qualities of the adaptation option is runtime quantitative verification, see
for example (Calinescu et al., 2011; Moreno et al., 2015; Weyns et al.,
2018a). It is important to note that the adaptation space exhibits dynamic
behavior that is difficult to predict upfront. On the one hand, the estimated
quality properties of the adaptation options vary over time as the
uncertainties the system is exposed to change over time. On the other hand,
the system configuration itself dynamically changes as the system is adapted
over time.

Different techniques have been studied to find the best adaptation option in a
large adaptation space. One particular approach to deal with the problem is
adaptation space reduction that aims at retrieving a subset of relevant
adaptation options from an adaptation space that are then considered for
analysis. An adaptation option is labeled relevant when it is predicted to
satisfy the adaptation goals. Techniques have been applied in this approach
including search-based techniques (Nair et al., 2018) and feature-based
techniques (Metzger et al., 2019). Recently, different machine learning
techniques have been investigated to reduce the adaptation space at runtime,
see for instance (Elkhodary et al., 2010; Quin et al., 2019; Jamshidi et al.,
2019). Among the applied learning techniques are decision trees,
classification, and regression (Gheibi et al., 2021a). However, most of these
techniques rely on domain expertise to perform feature engineering to define
the learner, which may hamper the applicability in practice. Further, most
existing approaches are limited to threshold goals and optimization goals. In
this paper, we tackle the following research question:
How to reduce large adaptation spaces and rank adaptation options effectively
and efficiently
at runtime for self-adaptive systems with threshold, optimization, and set-
point goals?
With effectively, we mean the solution should ensure that: (1) the reduced
adaptation space is significantly smaller, (2) the relevant adaptation options
should be covered well, that is, the adaptation options that satisfy (most of)
the adaptation goals should be included, (3) the effect of the state space
reduction on the realization of the adaptation goals is negligible. With
efficiently we mean the solution should ensure that: (4) the learning time is
small compared to the time needed for analysis, (5) there is no notable effect
on (1), (2), (3) and (4) for larger sizes of the adaptation space.

To answer the research question, we propose a novel approach for adaptation
space reduction called “Deep Learning for Adaptation Space Reduction Plus” –
DLASeR+ in short. DLASeR+ leverages on deep learning that relies on deep
artificial neural networks, i.e., neural networks with many layers. DLASeR+
offers an extendable framework that performs effective and efficient online
adaptation space reduction for threshold, optimization, and set-point goals.
While DLASeR+ can handle an arbitrary mix of goals, the concrete architecture
we present in this paper is tailored to a set of threshold and set-point goals
and one optimization goal. DLASeR+’s learning pipeline consists of an offline
and online part. During the offline part, the learning model is selected and
configured using training data. During the online part, the running system
uses the model to reduce adaptation spaces, and exploits newly obtained data
from analysis to continue the training and to update the learning model
enabling the system to deal with changing operating conditions.

We studied deep learning for four important reasons. First, classic learning
techniques usually require some form of human input for feature engineering,
whereas deep learning can handle raw data, without the need for feature
engineering. Second, besides some exceptions, classic machine learning models
are usually linear in nature, whereas deep learning can work with non-linear
models. Third, learned features and even entire models can be reused across
similar tasks. This type of transfer learning is a consequence of
representation learning, which is the basic core concept that drives deep
learning. We exploit representation learning in the DLASeR+ neural network
architecture. Fourth, given the success of deep learning in various other
domains, e.g., computer vision (Russakovsky et al., 2015) and natural language
processing (Radford et al., 2019), we were curious to explore how well deep
learning could perform for an important runtime problem in self-adaptive
systems.

In initial work, we explored the use of deep learning to reduce adaptation
spaces for threshold and optimization goals (Van Der Donckt et al.,
2020).111This initial version was denoted DLASeR; the + emphasizes that
DLASeR+ significantly extends DLASeR. The goal of that initial work was to
investigate the usefulness of deep learning for adaptation space reduction.
Compared to that exploratory work, DLASeR+ supports besides threshold and
optimization goals also set-point goals. Whereas the initial approach of (Van
Der Donckt et al., 2020) used a distinct model per goal, DLASeR+ works with an
integrated learning architecture that uses a single model, where layers are
shared and reused across different adaptation goals. Furthermore, DLASeR+
requires a single grid search process and a single prediction step in each
adaptation cycle, whereas the initial approach required grid search and
prediction for each goal in each cycle.

We evaluate DLASeR+ on two instances of DeltaIoT, an artifact for evaluating
self-adaptive systems (Iftikhar et al., 2017). The Internet-of-Things (IoT) is
a challenging domain to apply self-adaptation, given its complexity and high
degrees of uncertainties (Weyns et al., 2018b). The two instances of DeltaIoT
differ in the size of their adaptation space, enabling us to evaluate the
different aspects of effectiveness and efficiency. To that end, we define
appropriate metrics to evaluate DLASeR+ and compare it with a baseline that
applies exhaustive analysis, and two existing learning-based approaches for
adaptation space reduction: ML4EAS (Quin et al., 2019) that uses classic
learning techniques, and the initial DLASeR (Van Der Donckt et al., 2020).

The contributions of this paper are: (1) DLASeR+, a novel modular approach for
adaptation space reduction in self-adaptive systems that is able to handle
threshold, optimization, and set-point goals, and (2) a thorough evaluation of
the effectiveness and efficiency of the approach in the domain of IoT,
including a comparison with a baseline and two state of the art approaches.

Given the specific domain we use in the evaluation with relatively limited
sizes of adaptation spaces, we want to emphasize that additional validation is
required to generalize the findings.

The remainder of this paper is structured as follows. In Section 2 we provide
relevant background: we introduce DeltaIoT, we present a high-level
architecture of self-adaptation with adaptation space reduction, we zoom in on
the different types of adaptation goals, the adaptation space, and we
introduce the essential concepts of deep learning. Section 3 gives a high-
level overview of the research methodology. In Section 4, we introduce a set
of metrics that we use to measure the effectiveness and efficiency of DLASeR+
and compare the approach with alternative approaches. Sections 5 and 6 present
the core technical contribution of this paper: the architecture and learning
pipeline of DLASeR+ architecture respectively. In Section 7, we use the
metrics to evaluate DLASeR+ for two instances of DeltaIoT. Section 8 positions
DLASeR+ in the landscape of other related work. Finally, we draw conclusions
and look at future work in Section 9.

##  2\. Background

This section introduces the necessary background for this paper. We start with
introducing DeltaIoT. Then, we explain the basic architecture of a self-
adaptive system that integrates a verifier for runtime analysis and a learning
module for online adaptation space reduction. Next, we introduce the different
types of adaptation goals that are supported by DLASeR+. Then, we elaborate on
the concept of adaptation space. Finally, we introduce the relevant concepts
of deep learning.

###  2.1. DeltaIoT

DeltaIoT is a reference Internet-of-Things (IoT) application that has been
deployed at the Campus Computer Science of KU Leuven (Iftikhar et al., 2017).
DeltaIoT has been developed to support research on self-adaptation, i.e,
evaluate new self-adaptive approaches, e.g. to evaluate tradeoffs between non-
functional requirements in self-adaptive systems (Edwards and Bencomo, 2018)
or perform cost-benefit-analysis in runtime decision-making for self-
adaptation (Van Der Donckt et al., 2018). Next to the real physical setup
deployed by VersaSense,222VersaSense website: www.versasense.com DeltaIoT also
offers a simulator for offline experimentation. We use DeltaIoT as evaluation
case, but also as running example to illustrate the different parts that
follow in this section.

Figure 1. DeltaIoT deployment at the KU Leuven campus (borrowed from (Van Der
Donckt et al., 2018)). The gateway is marked by the blue icon in the center.
The data collected by the sensors is sent over multiple wireless links to this
gateway.

Figure 1 shows the physical setup of DeltaIoT. RFID sensors are used to
provide access control to labs, passive infrared sensors monitor the occupancy
of several buildings, and heat sensors are employed to sense the temperature.
The data of these sensors is relayed to the gateway by means of wireless
multi-hop communication. Each sensor is plugged into a battery-powered mote,
i.e., a networked tiny embedded computer. The motes take care of the routing
of sensor data to the gateway. The communication in DeltaIoT is time-
synchronized and organized in cycles with a fixed number of slots. Neighboring
motes are assigned such slots during which they can exchange packets. Motes
collect data (locally generated or received from other motes) in a buffer.
When a mote gets a turn to communicate with another mote, it forwards the
packets to the other mote. Packets that cannot be sent remain in the buffer
until the mote is assigned a next slot.

DeltaIoT has three main quality requirements: packet loss, latency, and energy
consumption. For these qualities we define corresponding adaptation goals,
e.g, average latency of packages delivery should not exceed a predefined
percentage and energy consumption of the network should be minimized. Ensuring
such goals is challenging since the IoT network is subject to various types of
uncertainties. Two main uncertainties are interference along network links
caused by external phenomena, and changing load in the network, that is, motes
only send packets when there is useful data, which may be difficult to
predict. The IoT network can be adapted using two parameters: power setting
and link distribution. The first parameter refers to the setting of the
transmission power of each mote. The options are discretized as an integer in
the range of 1 to 15. Increasing the transmission power will reduce packet
loss, but increase energy consumption. The second parameter refers to the way
data packets are relayed from the motes to the gateways. Observe in Figure 1
that several motes have multiple links over which data can be sent. We refer
to the distribution of packets sent by a mote to its parents as the link
distribution. If a mote has only one parent, it is obvious that it relays 100%
of its packets through that single parent. But when a mote has multiple
parents, the distribution of packets over the different links to the parents
can be selected and changed. Note that the sum of the distributions should
remain 100% (to optimize energy consumption). By changing the link
distribution, paths with more interference can be avoided. However, this may
cause delays at the buffers of the motes along these paths.

For practical reasons, we use the DeltaIoT simulator for the evaluation of
DLASeR+, since extensive experimentation on a physical IoT deployment is
particularly time consuming. The DeltaIoT simulator offers a realistic
alternative for the physical network where parameters of the simulator are
based on field experiments. We consider two instances of DeltaIoT. The first
one, referred to as DeltaIoTv1, consists of 15 motes (shown in Figure 1); the
second instance, referred to as DeltaIoTv2, consists of 37 motes. The larger
IoT network is more challenging in terms of the number of configurations that
are available to adapt the system representing the adaptation space. In
particular, the adaptation space of DeltaIoTv1 contains 216 possible
adaptation options, while DeltaIoTv2 has 4096 adaptation options. These
numbers are determined by the parameters of the IoT network that can be used
for adapting the system: power setting and link distribution.333Technically,
we apply the following approach to determine the adaptation options for the
IoT settings: first we determine the required power settings for each mote
along the links to its children such that the signal-to-noise ratio is at
least zero. These settings are determined based on the actual values of
signal-to-noise along the links. The settings are then fixed for all
adaptation options. The number of adaptation options is then determined by the
combinations of all possible settings of link distributions in the network.
This number is 216 for DeltaIoTv1 and 4096 for DeltaIoTv2. Hence, for both
versions, the number of adaptation options is constant. However, as will
explain in Section 2.4, the properties of the adaptation options change
dynamically with changing conditions.

###  2.2. Basic architecture self-adaptive system with adaptation space
reduction

Figure 2 shows the basic architecture of a self-adaptive system that uses
learning for adaptation space reduction. As explained in the introduction, in
this research we apply architecture-based adaptation with a MAPE-K feedback
loop. The figure highlights the main elements of architecture and high-level
flow of interactions between the elements to realize adaptation space
reduction. The managed system takes input from the environment and produces
output to realize the user goals. The managing system manages the managed
system to achieve a set of adaptation goals. Central to the managed system are
the MAPE elements that share knowledge and realize a feedback loop. The
feedback loop senses the managed system and adapts it to achieve the
adaptation goals.

Figure 2. A self-adaptive system that uses learning for adaptation space
reduction

When the MAPE feedback loop detects that the adaptation goals are violated or
may no longer be achievable, it reads the adaptation options from the
knowledge (1). The feedback loop then instructs the machine learning module to
reduce the adaptation space (2). The machine learning module will use its
learning model to make predictions about the expected quality properties for
the different adaptation options (3). Based on these predictions, the
adaptation space is reduced to the most relevant adaptation options. Next, the
MAPE feedback loop instructs the verifier module to analyze the adaptation
options of the reduced adaptation space (4). When the verifier completes the
verification (5), the MAPE feedback loop reads the results (6). It then
forwards the results to the machine learning module that exploits the results
to continue the training of its learning model (7). Finally, the MAPE feedback
loop generates a plan (8) that is the used to adapt the managed system.
Analysis adaptation option. We illustrate how an adaptation option is analyzed
for DeltaIoT. In particular, we explain how packet loss is estimated by the
analyzer of the MAPE loop using a statistical model checker (we apply this
approach in the evaluation in Section 7). Figure 3 shows the quality model for
packet loss that consists of two interacting automata: Topology and Network.

Figure 3. Runtime quality model for packet loss

The automata have two sets of parameters: (i) parameters to configure the
model for an adaptation option, and (ii) parameters of the uncertainties that
need to be set based on the current conditions.

To configure an adaptation option, the power settings of the motes per link
need to be set (with values between 0 to 15) and the distributions factors for
links of motes with two parents need to be set (each with a value of 0, 20,
40, 60, 80 or 100%). The power of the links is set based on the current levels
of interference along the links that are available in the knowledge
repository. The power settings are applied to all adaptation options. The
values for the distribution factors are different for each adaptation option;
these values are assigned to the variables $probLink1$ and $probLink2$ of the
topology model. Furthermore, the values of the uncertainties, network
interference (SNR) and traffic load, need to be set. These values are
available in the knowledge repository and are based on the recent
observations. The uncertainties apply to all adaptation options.

After initializing the model for a particular adaptation option, the Topology
automaton simulates the communication of data along a path selected for
verification, i.e., a sequence of links from one mote via other motes to the
gateway (see also Figure 1). The current link to send data is selected
probabilistically based on the distribution factors ($probLink1$ and
$probLink2$). The model then signals the Network automaton. Next, the
probability for packet loss is calculated (based on the SNR). Depending on the
result either the packet is lost or the message is received. In the latter
case, the network automaton returns to the start location, continuing with the
next hop of the communication along the path that is currently checked, until
the gateway is reached. If the packet is lost, the verification of the
communication along that path ends. The quality model allows determining the
packet loss of the adaptation options using the following query:

$Pr[$¡=$1](<>Network.PacketLoss)$

This query determines the probability that the state Network.PacketLoss is
reached for the different paths of an adaptation option (see the Network
automaton in Figure 3). To that end, the verifier performs a series of
simulations and returns the expected packet loss with a required accuracy and
confidence. These estimates together with the estimates of other quality
properties for the different adaptation options are then used to select an
adaptation option using the adaptation goals.

This example illustrates that rigorous runtime analysis can be a resource- and
time-consuming activity, implying the need for adaptation space reduction if
the number of adaptation options, i.e., the size of the adaptation space, is
too big to be completely verifiable within the available time period to make
an adaptation decision. We elaborate on the adaptation space below in Section
2.4.

###  2.3. Adaptation goals

An adaptation goal refers to a quality requirement of the managed system that
the managing system should realize. A violation (or an expected violation) of
one or more of the adaptation goals triggers an adaptation, aiming to satisfy
all the adaptation goals again.

In this research, we consider three types of adaptation goals: (1) threshold
goals, (2) set-point goals, and (3) optimization goals. Intuitively, a
threshold goal states that the value of some quality property of the system
should be below (or above) a certain threshold value. A set-point goal states
that some value of a quality property of the system should be kept at a
certain value, i.e., the set-point, with a margin of at most $\epsilon$.
Finally, an optimization goal states that some value of a quality property of
the system should be minimized (or maximized). Formally, we define the
satisfaction of goals by the adaptation options as follows.444For the
explanation, we consider only threshold goals below a value and optimization
goals that minimize a value; the other variants are defined similarly.
Consider $C$ the set of possible configurations, each configuration
representing an adaptation option of the adaptation space. We refer to a
particular quality property $q_{x}$ of an adaptation option $c_{i}\in C$ as
$c_{i}[q_{x}]$, with $x\in\\{t,s,o\\}$ referring to quality properties related
to threshold, set-point, and optimization goals respectively. Further,
consider a threshold goal $g_{t}$, a set-point goal $g_{s}$, and an
optimization goal $g_{o}$. The set of adaptation options $T$ that satisfy
$g_{t}$, the set $S$ that satisfy $g_{s}$, and the set $O$ that satisfy
$g_{o}$ are then defined as follows:555While a set-point goal may conceptually
be modeled as two threshold goals, there are good arguments to differentiate
them as a distinct type of goal. In particular, using a set-point to express
the corresponding goal is more straightforward and natural for stakeholders
compared to using two thresholds. Further, it makes maintenance easier, e.g.,
when updating the set-point value of the goal. Lastly, from a learning
perspective, if we use two threshold goals instead of one set-point goal, we
require two times the processing resources to train and infer.

(1) $\displaystyle T$ $\displaystyle=\\{c_{i}\in C\mid c_{i}[q_{t}]<g_{t}\\}$
(2) $\displaystyle S$ $\displaystyle=\\{c_{i}\in C\mid g_{s}-\epsilon\ \leq
c_{i}[q_{s}]\leq g_{s}+\epsilon\\}$ (3) $\displaystyle O$
$\displaystyle=\\{c_{i}\in C\mid c_{i}[q_{o}]<c_{j}[q_{o}],~{}~{}\forall
c_{j}\in C\setminus\\{c_{i}\\}\\}$

While DLASeR+ can handle an arbitrary mix of adaptation goals, in this paper,
we focus on systems that can have multiple threshold and set-point goals, but
only one optimization goal. In particular, the DLASeR+ architecture we present
in this paper maps each adaptation goal to a rule. Decisions are made by first
applying the rules of the threshold and set-point goals, and then the rule of
the optimization goal. Handling multiple optimization goals with different
decision-making techniques, e.g., based on Pareto optimality (Hwang and Masud,
2012), are outside the scope of this paper.

As an illustration, Figure 4 shows a latency goal for DeltaIoT specified in
three formats corresponding with the three types of goals (the diagrams are
simplified for didactic reasons).666In DeltaIoT, latency is defined as a
relative part of the cycle time of the time-synchronized communication in the
network. E.g., a latency of 5% means that the average time packets stay in the
network is 5% more as the cycle time. Each dot in a diagram represents an
adaptation option based on the values of latency and packet loss (we are here
mainly interested in the values of latency). 4(a) shows the latency as a
threshold goal `Latency` ¡ `6.5%`. All the dots below the red dotted line
represent adaptation options that satisfy the threshold goal. The adaptation
options above the red dotted line do not meet this goal. 4(b) shows the
latency as a set-point goal `Latency` = `6%`$~{}\pm~{}$`0.2%`. The two blue
dots are the only adaptation options that meet this set-point goal, since both
are in the range defined by the set-point value and the margin $\epsilon$. The
other adaptation options have values out of the range $[5.8\%,6.2\%]$ and do
not satisfy the goal. Finally, 4(c) shows the latency as an optimization goal.
The darker the dot, the lower the latency, hence, the dot at the bottom has
the lowest value for latency and this adaptation option optimizes the latency.

(a) Threshold goal.

(b) Set-point goal.

(c) Optimization goal.

Figure 4. Illustration of the three types of adaptation goals.

For the evaluation with DeltaIoT in Section 7, we consider three combinations
of adaptation goals: (1) two threshold goals: packet loss and latency, with
energy consumption as optimization goal, (2) the same threshold goals, with
energy consumption as set-point goal, and (3) latency as threshold goal,
energy consumption as set-point goal, and packet loss as optimization goal.

###  2.4. Adaptation space

An adaptation space comprises the set of adaptation options at some point in
time. The adaptation space is determined by the effectors (also called
actuators or “knobs”) that are available to adapt the managed system. The
actuators allow to set system variables. These variables are usually
characterized by a discrete domain (e.g., start/stop a number of servers,
launch a number of new virtual machines, select a power setting, send a number
of messages over a link, etc.). For variables with a continuous domain, we
assume in this work that the domain can be discretized.

If adaptation is required, it is the task of the analyzer and the planner of
the MAPE-K feedback loop to find the best adaptation option. This task
involves estimating the expected quality properties related to the adaptation
goals for all or at least a relevant subset of the adaptation options.
Different techniques can be used to make these predictions. One approach is to
represent the quality of the system as a network of stochastic timed automata
(Legay et al., 2010; Younes and Simmons, 2006; Weyns and Iftikhar, 2016) as we
illustrated in Sub-section 2.2. In this approach, the model is configured for
the effector settings of a particular adaptation option and parameters that
represent uncertainties are assigned up-to-date values. By utilizing
statistical model checking one can determine the expected quality of an
adaptation option. Another approach is to represent the quality of the system
as a parameterized Markov model (Calinescu et al., 2011; Moreno et al., 2015;
Cámara et al., 2020) and apply probabilistic model checking (Legay et al.,
2010; Younes and Simmons, 2006) to estimate the quality property of interest.
Statistical model checking is more efficient than probabilistic model
checking, but offers results bounded to a given accuracy and confidence,
depending on the number of simulation runs.

In the case of DeltaIoT, we are interested in the failure rate of the network,
its latency and the energy consumed to communicate the data to the gateway.
Figure 5 shows a representation of the adaptation space for DeltaIoTv2 at some
point in time.

Figure 5. Adaptation space of DeltaIoTv2 at some point in time.

The red lines denote two threshold goals for this particular scenario (latency
and packet loss). Each blue dot in this graph represents one adaptation
option. The dot in orange represents the current configuration. For this
particular instance, the number of adaptation options is over 4000. Analyzing
all these options within the available time slot may not be feasible. Hence,
the analysis should focus on the relevant options for adaptation, i.e., those
that are compliant with the goals, represented by the dots in the box bottom
left as determined by the two threshold goals. This paper is concerned with
analyzing such a large adaptation space in an effective and efficient manner.

It is important to notice that the adaptation space is dynamic, i.e., the
qualities of each adaptation option may change over time. In Figure 5, this
means that the position of the adaptation options on the diagram move over
time. The main cause for this dynamic behavior are the uncertainties the
system is subjected to. It is due to this dynamic behavior that the analysis
and verification needs to be performed at runtime, when the actual values of
the uncertainties are available. Figure 6 illustrates the dynamic behavior for
an instance of DeltaIoT. The figure illustrates how three qualities of one of
the adaptation options change over a series of cycles. These graphs are based
on field tests.

Figure 6. Dynamic behavior of an adaptation option over 50 cycles reflected in
changes of its quality properties.

###  2.5. Deep learning in a nutshell

Deep learning (DL) refers to a subset of machine learning mechanisms in which
the learning methods are based on deep artificial neural networks (ANNs) with
representation learning (Goodfellow et al., 2016).

####  2.5.1. Artificial neural networks

ANNs are the core of deep learning. These networks are commonly described in
terms of their input and output, which are the only externally observable
parts of an ANN. Both the input and the output are represented as a vector of
numbers. A neural network applies a chain of mathematical operations aiming to
transform the input to the desired output.

The basic building block of a neural network is a neuron, also called a
perceptron. In the case of a fully connected neural network, a neuron is
connected to all the inputs and produces a single output. To obtain its
output, the neuron applies two operations:

(4) $z=\vec{w}\cdot\vec{x}+b$ (5) $y=f(z)$

with $\vec{x}$ the current input vector, $\vec{w}$ the weights associated with
the inputs, $b$ a constant, $z$ an intermediate computed value, $y$ the
output, and $f(\cdot)$ an activation function. First, the weighted sum $z$ of
all the inputs is calculated (Equation 4). The weights associated with the
inputs are learnable. Intuitively, these weights represent the relative
importance of the inputs. When the weighted sum is computed, the neuron
applies an activation function $f$ (Equation 5). This function allows
introducing non-linearity in the neural network. Since, a weighted sum is a
linear function, and a linear combination of linear functions is still a
linear combination, a neural network without a non-linear activation function
will only be able to learn linear combinations of input values. Non-linearity
in learning is very important to learn more complex concepts. Examples of
common activation functions are hyperbolic tangent (tanh), sigmoid, and
rectified linear unit (ReLU).

A deep neural network is structured in two dimensions; width and depth. The
depth of a network corresponds to the number of layers, whereas each layer is
defined by its width, i.e., the number of neurons present in that layer.

####  2.5.2. Representation learning

A key concept of ANNs is representation learning (Goodfellow et al., 2016).
Representation learning learns representations of input data, typically
through transformations, which makes it easier to perform certain tasks
(Bengio et al., 2013). An important advantage of representation learning is
that it enables transfer learning, i.e., reuse of the learned representations,
complemented with fine-tuning for specific learning tasks (Pan and Yang,
2009). In deep learning, representations are learned through multiple non-
linear transformations on the input data. These learned representations
provide abstract and useful representations for the learning task. A simple
linear layer can then be stacked on top of the complex block with many non-
linear layers to tailor the learning network to the specific learning task at
hand, e.g., classification, regression, or prediction (Devlin et al., 2018).

####  2.5.3. Training a deep neural network

We distinguish three steps in the training of a ANN: (1) forward propagation,
(2) loss calculation, and (3) back propagation.

In the forward propagation step, the input data is passed through the layers
of the network, from input to output. The neurons apply their transformations
to the data they receive from all the connected neurons of the previous layer
and pass the result to the connected neurons of the next layer. In the final
layer, the neurons apply their transformations to obtain the (predicted)
output.

In the second step, a loss function estimates the loss (or error). The loss
captures how good or bad the predicted result is (i.e., the output predicted
by the network) compared to the correct result (i.e., the expected output). To
obtain a loss as close as possible to zero, the weights of the connections
between the neurons (that determine the weighted sums) are gradually adjusted
in the next step.

As final step, the loss value is propagated backwards through the network,
hence back propagation. You can think of this step as recursively applying the
chain rule to compute the gradients all the way from the output to the input
of the network. These gradients tell in which direction the loss is increasing
and indicate the influence of the computations of the neurons on the loss. An
optimizer exploits these gradients to update the weights of the neuron
connections aiming to minimize the loss. Examples of commonly used optimizers
are Adam, RMSprop, and Nadam (Ruder, 2016).

####  2.5.4. Classification and regression

Classification and regression are two important learning tasks that require
different predictions. Suppose that we have some machine learning model that
is described by the following equation:

(6) $\vec{y}=M(\vec{x})$

with $\vec{x}$ the input and $\vec{y}$ the output of the learner, and $M$ a
function that maps input to output.

In the case of classification, $M$ needs to map the input to a set of classes
that are represented by different labels with different encodings (e.g., rainy
= 0 and sunny = 1). A classification with only two labels is called binary
classification; while multi-class classification has more than two labels.
Regression on the other hand, maps the input data to continuous real values
instead of classes or discrete values. Whereas for classification the
predicted output is not ordered, for regression the output is ordered. In sum,
the main difference between both tasks is that the output variable $\vec{y}$
for classification is categorical (or discrete), while for regression it is
numerical (or continuous).

##  3\. Overview of the Research methodology

To tackle the research question, we followed a systematic methodology as shown
in Figure 7.

Figure 7. Overview of research methodology.

Based on a study of the state-of-the-art and our own experiences with applying
machine learning in self-adaptive systems, we defined the research question
(see Section 1). Once we determined the research question, we specified the
metrics that enabled us to measure the effectiveness and efficiency of DLASeR+
and compared it with other approaches. Then, we defined the DLASeR+
architecture that is able to deal with threshold goals, set-point goals, and
an optimization goal. Next, we devised DLASeR+’s learning pipeline that
applies deep learning for the three types of adaptation goals. We instantiated
this pipeline for various setups for the two evaluation cases of DeltaIoT
applying different combinations of adaptation goals. We then evaluated DLASeR+
and compared it with representative related approaches. Finally, we answered
the research question, and reflected and discussed the extent to which we met
our objectives.

In the next sections, we zoom in on the different steps of the methodology. We
start with explaining the metrics we used to design and evaluate DLASeR+
(Section 4). Then we zoom in on the DLASeR+ architecture (Section 5) and we
explain how to engineer a solution with DLASeR+ (Section 6). Next, we present
the evaluation of DLASeR+, answering the research question (Section 7).
Finally, we discuss related work (Section 8) and conclude the paper (Section
9).

##  4\. Metrics

To answer the research question, we determined metrics that allow us to
evaluate the different aspects of effectiveness and efficiency of the proposed
solution. Table 1 summarizes the metrics.

Effectiveness Coverage relevant adaptation options F1-score (for threshold and
set-point goals) Spearman’s rho (for optimization goal) Reduction adaptation
space Average adaptation space reduction (AASR) Average analysis effort
reduction (AAER) Effect on realization adaptation goals Differences in mean
values of quality properties of the goals with and without learning Efficiency
Learning time Time used for learning (training + prediction, expressed in s)
Time reduction for analysis with learning (expressed in %) Scalability Metrics
above for increasing size of adaptation space

Table 1. Left: different aspects of research question to be evaluated; right:
metrics for each aspect.

We use different metrics to capture the coverage of relevant adaptation
options for classification and regression. In particular, we use the F1-score
for classification and Spearman’s rank correlation coefficient (Spearman’s
rho) for regression.777A common metric to evaluate regression models is mean
squared error (MSE). However, since we are more interested in the ranking of
the regressed output, we use Spearman’rho to capture this ranking, which is
not covered by MSE. We define average adaptation space reduction (AASR) and
average analysis effort reduction (AAER) to capture and compare the reduction
of adaptation spaces. To capture the effect on the realization of the
adaptation goals, we use the differences in mean values over time for the
corresponding quality properties with and without learning. We measure the
time used for learning888The learning time refers to the time the system uses
for training and making predictions of the quality properties of interest for
the set of adaptation options that are considered for the respective quality
properties. and compare this with the time that would be necessary to analyze
the complete adaptation space. Finally, for scalability, we apply the
different metrics for scenarios with an increasing size of adaptation spaces.
We further elaborate on F1-score, Spearman’s rho, AASR, and AAER. The other
basic metrics are further explained in the evaluation section.

###  4.1. F1-score: precision and recall

The quality of a classification algorithm is often expressed in terms of
precision and recall. Applied to the classification of adaptation options
(relevant or not relevant according to the adaptation goals), precision is the
fraction of selected adaptation options that are relevant, while recall is the
fraction of the relevant options that are selected from the total number of
relevant options. Hence, precision and recall are a measure of relevance. The
F1-score combines both metrics with equal importance into a single number.
Concretely, the F1-score is defined as the harmonic mean of precision and
recall (Géron, 2019);

(7) $F1=2*\frac{precision*recall}{precision+recall}$

###  4.2. Spearman correlation

Spearman correlation, or Spearman’s rho, is a non-parametric metric that can
be used to measure rank correlation. We use this as a metric to capture the
ranking of the predicted values of quality properties of regression models.
The Spearman correlation essentially converts the predicted values to numeric
ranks and then assesses how well a monotonic function describes the
relationship between the predicted ranks and the true ranks. Spearman’s rho is
defined as (Kokoska and Zwillinger, 2000):

(8) $\rho_{xy}=\frac{n\sum x_{i}y_{i}-\sum x_{i}\sum y_{i}}{\sqrt{n\sum
x_{i}^{2}-(\sum x_{i})^{2}}\sqrt{n\sum y_{i}^{2}-(\sum y_{i})^{2}}}$

This formula computes the Spearman correlation between $x$ (the predictions of
the selected adaptation options) and $y$ (the true values for the selected
adaptation options), with $n$ the number of observations (the number of
selected adaptation options). The result is a value between 0 and 1 (for an
increasing monotonic trend, or -1 for a decreasing monotonic trend). Large
errors are penalized harder. For example, a swapping of the first and third
rank in the prediction results is worse (lower Spearman’s rho) compared to a
swapping of the first and second rank.

###  4.3. Average Adaptation Space Reduction

To capture how well the adaptation space is reduced, we define a new metric
called average adaptation space reduction ($AASR$). $AASR$ is defined as:

(9) $AASR=(1-\frac{selected}{total})\times 100$

with $selected$ the number of adaptation options selected by learning (over
multiple adaptation cycles) and $total$ the total number of adaptation options
(of multiple adaptation cycles). For instance, an average adaptation space
reduction of 70% means that after learning only 30% of the original adaptation
space is considered for analysis. $AASR$ is a particularly suitable metric for
stakeholders as it covers the high-end goal of adaptation space reduction and
allows comparing solutions.

Remark that the average adaptation space reduction is determined by the
system’s adaptation goals. In particular, for the three types of goals
considered in this work, the $AASR$ is determined by the threshold and set-
point goals, corresponding to the percentage of adaptation options that are
predicted to be conform with these goals. For systems with only optimization
goals, the $AASR$ is zero (since the selected and total number of adaptation
options are the same).

It is also interesting to note that the $AASR$ depends on the restrictiveness
of both threshold and set-point goals. Suppose that the threshold and set-
point goals are not very restrictive, thus many adaptation options will comply
with the adaptation goals. In this case, the reduction will be rather small.
In the other case, for very restrictive threshold and set-point goals the
opposite is true. A larger reduction for these goals can be expected.

###  4.4. Average Analysis Effort Reduction

To capture the effect of the adaptation space reduction on the effort required
for analysis, we define a new metric called average analysis effort reduction
($AAER$). $AAER$ is defined as:

(10)
${\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{}AAER}=(1-\frac{analyzed}{selected})\times
100$

with $analyzed$ the number of adaptation options that have been analyzed (over
multiple adaptation cycles) and $selected$ the number of adaptation options
selected by learning (over multiple adaptation cycles). For instance, an
average analysis effort reduction of 90% means that only 10% of the adaptation
options selected by learning were analyzed to find an option to adapt the
system. Similarly to $AASR$, $AAER$ also covers a high-end goal of adaptation
space reduction.

Note that the $AAER$ depends on the analysis approach that is used to find an
adaptation option from the reduced adaptation space (and not on the
constraints imposed by the adaptation goals as for $AASR$). In particular, the
analysis reduction corresponds to 100% minus the percentage of selected
adaptation options that have to be analyzed until an adaptation option is
found that meets all the threshold and set-point goals. For systems that
include threshold and set-point goals and an optimization goal, the selected
adaptation options are analyzed in the order predicted for the optimization
goal. For systems with only threshold and set-point goals, the most basic
method to analyze the selected adaptation options is random order. For the
evaluation of this specific case, we randomly shuffle the selected options per
cycle, resulting in a more representative $AAER$ score. Remark that for
systems with a large adaptation space and only an optimization goal,
${\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}{}AAER}\approx
100$% (since $analyzed$ is 1 and $selected$ is a large number equal to the
size of the adaptation space).

##  5\. DLASeR+ Architecture

We introduce now the novel adaptation space reduction approach “Deep Learning
for Adaptation Space Reduction Plus” – DLASeR+. We start with outlining how
this approach deals with different types of adaptation goals, and how learning
for these adaptation goals is combined into a unified architecture. Finally,
we zoom in on the neural network architecture of DLASeR+.

###  5.1. Adaptation space reduction for different types of adaptation goals

DLASeR+ can reduce the adaptation space for adaptation goals based on
thresholds, set-points, and optimization goals.

####  5.1.1. Threshold goals

To deal with threshold goals, DLASeR+ relies on classification deep learning.
Concretely we apply binary classification using class $1$ (true) if an
adaptation option meets the threshold goal and class $0$ (false) otherwise. We
say that an adaptation option meets a threshold goal when the associated
quality property of that adaptation option is below (or above) the given
threshold value $g_{t}$ of that goal. Hence, DLASeR+ reduces the adaptation
space to the adaptation options that are valid, i.e., that are classified as
$1$. In case of multiple threshold goals, the adaptation space is reduced to
the intersection of the subsets of adaptation options with predicted values
that are classified as $1$ for each of the different threshold goals.

####  5.1.2. Set-point goals

For set-points goals, DLASeR+ also relies on classification deep learning.
When a user defines a set-point goal, he or she has to specify a set-point
value $g_{s}$ and a bound $\epsilon$. This bound defines the range
$[g_{s}-\epsilon,g_{s}+\epsilon]$ in which adaptation options are considered
valid.999From a control theoretic perspective, this bound corresponds to the
steady state error. Hence, DLASeR+ again applies binary classification; class
1 for adaptation options that are predicted within the interval, and class 0
otherwise. In case the self-adaptive system has multiple set-point goals, the
reduced adaptation space is the intersection of the subsets of adaptation
options with predicted values that are classified as 1 for each of the
different set-point goals.

####  5.1.3. Optimization goals

DLASeR+ handles optimization goals using regression deep learning. Based on
the regressed (predicted) values of a quality property, the adaptation options
are ranked. From this ranking the adaptation option that maximizes or
minimizes the adaptation goal can be derived. The adaptation space reduction
is then determined by the number of adaptation options that need to be
analyzed to make an adaptation decision. In the case of a single optimization
goal, as we consider in this work, the adaptation space reduction is
determined by the number of ranked adaptation options that need to be analyzed
before one is found that complies with the other goals.

###  5.2. Unified DLASeR+ architecture

We explained how DLASeR+ handles single types of adaptation goals, however,
practical systems usually combine different types of adaptation goals. Over
the years, different techniques have been developed to combine adaptation
goals for the decision-making of self-adaptation. Classic examples are goals,
utility functions, and rules (Cheng et al., 2009b; Tesauro and Kephart, 2004;
Ghahremani et al., 2020). DLASeR+ offers a unified architecture for adaptation
space reduction for a class of systems that combine multiple threshold and
set-point goals with a single optimization goal. Our focus is on rule-based
goals that are representative for a large number of practical self-adaptive
systems. Figure 8 shows the unified DLASeR+ architecture.

Figure 8. Unified architecture of DLASeR+ for a mix of threshold and set-point
goals and one optimization goal.

The deep neural network takes as input the initial adaptation space that
consists of all the adaptation options, together with context data, such as
the actual values of uncertainties in the environment and the current values
of the relevant qualities. Internally, the DLASeR+ architecture is centered on
a single deep neural network that covers all the adaptation goals. The deep
neural network consists of a number of core layers, complemented with goal
specific heads. The core layers are shared among the different goals. Each
head deals with a specific goal. The output produced by the deep neural
network is then combined to produce the reduced adaptation space. The layered
structure of DLASeR+ with shared core layers and goal specific heads adds to
the modularity of the learning architecture supporting modifiability and
extensibility.

(a) Architecture deep neural network.

(b) Internal structure of the classification and regression heads in the
DLASeR+ architecture.

Figure 9. Overview of the internal deep neural network (NN) architecture of
DLASeR+.

####  5.2.1. Internal neural network architecture

The core of the DLASeR+ approach is a deep neural network that is kept updated
at runtime. Figure 9 shows the internal architecture of the neural network.
The figure at the top shows the structure of the network and the flow of the
data through the network. The network starts with concatenating the input into
input vectors, one vector per adaptation option. The input vectors include
high-level data relevant to adaptation in two parts: data of the adaptation
options, such as the settings of the system, and data of the context, such as
the current configuration, the recent load of the system and recent values of
uncertainty parameters. The data of the input vectors are then fed to the core
layers of the network that are modeled as a fully connected network of
neurons. The output of the last layer of the core layers encodes the input of
goal-specific heads. DLASeR+ supports two types of goal-specific heads that
can be added on top of the core layers: classification heads for threshold and
set-point goals, and a regression head for an optimization goal. Figure 9 at
the bottom illustrates the difference between the two types. Both types of
heads are fully connected neural networks that produce a single output, i.e.,
the output layer has a dimensionality of 1. However, the heads differ in the
output they produce. Classification heads that use a sigmoid activation
function produce values between 0 and 1. These values are classified based on
predefined thresholds; for instance, all values below 0.5 are classified as
class 0 and all values above (and including) 0.5 as class 1. The regression
head that uses a linear activation function produces values that predict the
quality property that needs to be optimized.

####  5.2.2. Adaptation space reduction stages

By using a single deep neural network, the input data passes only once through
the neural network to produce the output for the different adaptation goals.
This output is then combined to reduce the adaptation in two stages. The first
stage uses classification deep learning to reduce the adaptation space to the
adaptation options that are classified as being valid according to the
threshold and set-point goals. We refer to the first stage as the
classification stage. The second stage that deals with the optimization goal
further reduces the adaptation space obtained from stage 1 by ranking the
regressed values of the adaptations options. The adaptation options are ranked
from low to high in the case of a minimization goal, and vice versa for a
maximization goal. The ranked adaptation options can then be analyzed one by
one until an option is found that satisfies (e.g., determined by verification)
all the other goals. We refer to the second stage as the regression stage.

It is important to note that either of these stages can be omitted when the
corresponding type(s) of goals are not present for a problem at hand. For
instance, for a system with only threshold goals (for which DLASeR+ uses
classification), the second stage can be omitted.

Figure 10. Illustration of the two stages of DLASeR+. Each dot represents an
adaptation option. In the first stage the initial adaptation options are
classified for threshold and set-point goals (1.1): blue dots satisfy the
threshold goals, pink dots satisfy the set-point goals, and green dots satisfy
both types of goals. Then the intersection of adaptation options that satisfy
all threshold and set-point goals are kept, i.e., the green dots, (1.2). In
the second stage the relevant subset of adaptation options obtained in stage 1
are ranked (2.1).

Figure 10 schematically illustrates the two reduction stages, showing how the
predictions of the neural network for different adaptation goals are combined
to reduce an adaptation space.

In stage 1, all the adaptation options for all the threshold and set-point
goals are classified relevant or not relevant. Then, the results are
aggregated, reducing the initial adaptation space to the intersection of the
adaptation options that were classified relevant for all the aforementioned
goals. In stage 2, DLASeR+ ranks the subset of adaptation options obtained
from stage 1. This ranking is based on the regressed quality for which the
optimization goal was defined and depends on the optimization goal, i.e.,
ascending and descending order for respectively minimization and maximization.
The adaptation options of the reduced adaptation space can then be analyzed in
the order of the ranking until one is found that satisfies all the adaptation
goals.

##  6\. Engineering with DLASeR+

We explain now how to engineer a solution for adaptation space reduction with
DLASeR+. Central to this is the learning pipeline that consists of an offline
and online part. During the offline part, an engineer selects a model for the
deep neural network model and a scaler. During the online part, the running
system uses the model and scaler to reduce adaptation spaces, and exploits
newly obtained data from analysis to update the model and the scaler.

###  6.1. Offline part of the DLASeR+ learning pipeline

During the offline part of the pipeline, shown in Figure 11, the engineer
collects data of a series of adaptation cycles (via observation or simulation
of the system). This data comprises the adaptation options, context info, and
also the associated qualities of the adaptation options. The collected data is
then concatenated in input vectors, one per adaptation option. Each item of an
input vector refers to a measurable property or characteristic of the system
that affects the effectiveness of the learning algorithms. As an example, an
input vector for DeltaIoT contains:

  * •

Data of an adaptation option, i.e., the settings of the transmission power for
each link and the distribution factors per link that determine the percentage
of messages sent over the links;

  * •

Context data, i.e., the traffic load generated per mote and the signal-to-
noise ratio over the links (uncertainties), the current system configuration.

The qualities associated with the adaptation options are used to validate the
output of the generated goal-specific heads. As an example, the current
qualities (one per adaptation goal) for DeltaIoT are:

  * •

The packet loss along the links;

  * •

The energy consumed by the motes.

To successfully train a deep neural network, it is important that all relevant
data is collected.101010It is important to note that deep learning can work
with “raw data” without the need for transforming and aggregating features,
etc., as with classic machine learning that requires a substantial effort of
engineers. Using the raw data, the deep neural network will learn complex
relations automatically.

Figure 11. The offline part of the DLASeR+ learning pipeline. During model
selection the optimal hyper-parameters are determined for the shared layers
(Core layers selection) and for the goal-specific heads (Goal-specific head
selection). The output of the offline part of the pipeline is a completely
configured deep neural network model (of core layers complemented with the
various goal-specific heads) and a scaler.

The aggregated data is then used to perform so called back-testing on the
application to evaluate the performance of candidate learning solutions. To
that end, the aggregated data is split in two sets: training data and
validation data. It is crucial that both sets do not overlap. An overlap of
the two sets would introduce data-leakage, where knowledge of the test set
leaks into the dataset used to train the model. This would jeopardize the
validity of the results. (Brownley, 2020)

The main activity of the offline stage of the pipeline is selecting a deep
neural network model based on finding optimal hyper-parameter settings. Hyper-
parameters are non-learnable parameters that control the learning process. The
main hyper-parameters are the number of layers of the deep neural network and
the number of neurons per layer. Other hyper-parameters that apply for deep
learning architectures are the scaler algorithm (normalizes the data,
improving the learning process), the batch size (defines how much of the data
samples are considered before updating the model affecting the learning
speed), the learning rate LR (determines the impact of on update, affecting
the learning speed), and the optimizer (influences the updates of model
parameters utilizing the learning rate and the local gradient at the neurons).

In DLASeR+, we distinguish between hyper-parameters for the shared core layers
and for goal-specific heads. Core layers selection deals with the hyper-
parameters of the shared core layers, while goal-specific head selection deals
with the hyper-parameters of the goal-specific heads. To determine the optimal
values of the hyper-parameters for the DLASeR+ neural network model we applied
grid search (Géron, 2019). With grid search, the neural network model is
trained and then evaluated for every combination of hyper-
parameters.111111Technically, we consider the layout of the core layers and
the goal-specific heads (number of layers and neurons per layer) as distinct
hyper-parameters. However, these hyper-parameters are optimized together in a
single grid search process, since the loss function from the goal-specific
heads guides the learning process, including the learning of core layers. We
used different metrics during evaluation: (i) F1-score that combines precision
and recall of predicted classes of adaptation options, and (ii) Spearman
correlation that measures the ranking of predicted values of quality
properties of regression models, see the explanation in Section 4. Once the
models are trained (on the training data set), they are evaluated on the
validation data set, i.e., the predictions are evaluated using the validation
data. The best model is then selected based on the validation loss that is
determined by a loss function that sums the losses of the heads, capturing the
overall quality of the neural network model.

When the core layers and goal-specific heads are fine-tuned and a proper
scaler is found the integrated solution can be deployed, which brings us to
the second stage of the learning pipeline.

###  6.2. Online part of the DLASeR+ learning pipeline

During the online part of the learning pipeline, the deep neural network
supports the decision-making process of a self-adaptive system with reducing
large adaptation spaces. The online part, shown in Figure 12, consists of two
consecutive phases. In the training phase, which consists of a series of
training cycles, the deep learning model is initialized based on the current
state of the system exploiting all available runtime data. In the learning
phase, which consists of learning cycles, the deep learning model performs
adaptation space reduction and is updated using online learning.

Figure 12. The online part of the DLASeR+ learning pipeline. In the training
phase (training cycles), the deep neural network model is initialized for the
problem at hand. In the learning phase (learning cycles), adaptation space
reduction is applied and new data is used to update the deep neural network
model (online learning).

####  6.2.1. Training phase

The goal of the training phase is to initialize the learnable parameters of
the model for the problem at hand. During the training cycles, relevant
runtime data is collected to construct input vectors. This data is used to
update the scaler (e.g., updating min/max values of parameters, the means,
etc.) and the input vectors are adjusted accordingly. The deep neural network
model is then trained using the transformed data; i.e., the parameters of the
model are initialized, e.g., the weights of the contributions of neurons. The
training phase ends when the validation accuracy stagnates, i.e., when the
difference between the predicted values with learning and the actual
verification results are getting small.121212For the DeltaIoT, the training
phase ended after 45 training cycles (one per adaptation cycle), see Section
7. During the training cycles, the complete time slot available for adaptation
is used for analyzing adaptation options to generate data that is used to
initialize the deep neural network model; hence there is no reduction of the
adaptation space yet.

####  6.2.2. Learning phase

In the learning phase, the deep neural network model is actively used to make
predictions about the adaptation options for the various adaptation goals
aiming to reduce the adaptation space before the analysis of the reduced
adaptation space. Furthermore, the results of the analysis of the reduced
adaptation space are then used to incrementally update the learning model.
Concretely, in each learning cycle, an input vector is composed for each
adaptation option that is scaled using the scaler. Scaling normalizes the
range of independent variables of data, ensuring that each element contributes
approximately proportionately to the result. The deep neural network model
then makes predictions for the input vectors. Based on these predictions, the
adaptation space is reduced and the adaptation options of the reduced
adaptation space are then analyzed. In the evaluation of DLASeR+ (Section 7),
we use runtime statistical model checking (David et al., 2015; Agha and
Palmskog, 2018; Weyns and Iftikhar, 2016, 2019, 2022) to analyze the
adaptation options, however other analysis techniques can be applied.

The analysis of the reduced adaptation space depends on the types of
adaptation goals at hand. For a self-adaptive system with a mix of threshold
and set-point goals and an optimization goal, analysis consists of two stages:
the classification stage and the regression stage, see Section 5.2.2. For
systems without optimization goal, only the classification stage applies.
Similarly, for systems with only an optimization goal, only the regression
stage applies.

In the classification stage the adaptation space is reduced to the relevant
subset of adaptation options that comply with the threshold and set-point
goals. Algorithm 1 shows how the adaptation space reduction is applied and how
analysis is performed in the classification stage.

Algorithm 1 Stage 1: classification stage

1:$pred\\_subspace\leftarrow K.adaptation\\_options$

2:$DNN\\_output\leftarrow K.DNN\\_model$.predict($input\\_vectors$)

3:if $K.threshold\\_goals$ == None & $K.set\\_point\\_goals$ == None then

4: Proceed to the regression stage: Algorithm 2

5:end if

6:for each $class\\_goal\leftarrow K.classification\\_goals$ do

7: $pred\\_subspace\leftarrow pred\\_subspace\cap DNN\\_output.class\\_goal$

8:end for

9:if $K.optimization\\_goal\neq$ None then

10: Proceed to the regression stage: Algorithm 2

11:end if

12:$pred\\_subspace$.shuffle()

13:$valid\\_found\leftarrow$ False, $idx\leftarrow$ 0

14:$verified\\_subspace\leftarrow\emptyset$

15:while not $valid\\_found$ & idx $<pred\\_subspace.size$ do

16: $adapt\\_opt\leftarrow pred\\_subspace$[$idx$]

17: $Analyzer$.analyzeAdaptationOptions($adapt\\_opt$) $\triangleright$
Knowledge

18: $\\_,qualities\leftarrow K.verification\\_results$[$idx$]

19: if $qualities$ meet all threshold and set-point goals then

20: $valid\\_found\leftarrow$ True

21: end if

22: $verified\\_subspace$.add($adapt\\_opt$)

23: $idx\leftarrow idx$ \+ 1

24:end while

25:Select valid adaptation option $adapt\\_opt$, if none valid use fall back
option

26:$unselected\leftarrow K.adaptation\\_options\setminus pred\\_subspace$

27:$explore\leftarrow unselected$.randomSelect($exploration\\_rate$)

28:$Analyzer$.analyzeAdaptationOptions($explore$) $\triangleright$ Knowledge

29:$input\\_vectors$, $qualities\leftarrow K.analysis\\_results$

30:$K.DNN\\_model$.update($input\\_vectors$, $qualities$)

In lines 1 and 2, we initialize the variable $pred\\_subspace$ to the complete
adaptation space and store the predicted output of the deep neural network in
the $DNN\\_output$ variable. Both these variables are used in Algorithm 1 and
Algorithm 2. $K.DNN\\_model$ refers to the deep neural network model that is
stored in the Knowledge module ($K$). Since DLASeR+ uses a single deep
learning model for all the goals, prediction can be done in a single step.
Note that the predict() function in line 2 scales the elements of the input
vectors before making predictions. Lines 3 to 5 check whether there are any
threshold or set-point goals. If there are no threshold nor set-point goals,
we proceed to the regression stage, i.e., Algorithm 2. In the other case,
i.e., there are threshold and/or set-points goals, the adaptation space
($pred\\_subspace$) is reduced in lines 6 to 8. Concretely, we reduce the
adaptation space to the intersection of the predicted subspaces that are
relevant for each of these two types of classification goals. This step
represents the actual adaptation space reduction for all threshold and set-
point goals. The adaptation option is analyzed to check whether it complies
with the threshold and set-point goals. If this is the case this option is
selected for adaptation. If not, analysis is continued until an adaptation
option is found that satisfies the threshold and set-point goals. Line 9
checks whether there is an optimization goal. If there is such a goal, the
classification stage ends and the system continues the regression stage using
the reduced subset, i.e., $pred\\_subspace$ (see line 10). If there is no
optimization goal, the adaptation options for analysis are selected. We first
shuffle the relevant subspace to avoid bias in the way the adaptation options
are determined (see line 12), prepare the analysis (see lines 13 and 14), and
then iterate over the adaptation options top-down (see lines 15 to 24). In
this iteration, an adaptation option is analyzed to check whether it complies
with the threshold and set-point goals. If this is the case this option is
selected for adaptation and the iteration halts. If not, analysis is continued
until an adaptation option is found that satisfies the threshold and set-point
goals. After iterating over the subspace of adaptation options, line 25
selects a valid option to adapt the system, i.e., an adaptation option that
satisfies all the threshold and set-point goals. If no valid adaptation option
is found according to the goals, a fall-back option is used that implements a
graceful degradation strategy. Then, lines 26 and 27 use the exploration_rate
to select a random sample of adaptation options from the options that were not
analyzed. Adding this random sample aims at anticipating potential concept
drifts that might occur in dynamic environments after a large number of
adaptation cycles.131313Intuitively, one may argue to select this sample
nearer to the boundaries set by the thresholds rather than random, yet, this
may reduce the intended effect on potential concept drifts. Further study is
required to clarify this issue. These additional options are then also
analyzed (see line 28) and the analysis results are stored in the knowledge.
Finally, the analysis results are exploited to update the deep neural network
model (see lines 29 and 30). To that end, the input vectors (configurations
etc.) and the analysis results (i.e., the qualities per goal obtained by
verification) of the analyzed adaptation options are retrieved from the
knowledge. Based on this data, the neural network model is updated using the
same learning mechanism as used in the training cycle.

The regression stage starts either from the adaptation options selected in the
classification stage or from the complete adaptation space in case there is
only an optimization goal. The set of adaptation options are then ranked
according to the predicted quality of the optimization goal. Algorithm 2 shows
how the adaptation options are ranked and how one of the options is selected
based on the result of the analysis and its compliance with the threshold and
set-point goals.

Algorithm 2 Stage 2: regression stage

1:$pred\\_subspace$ retrieved from Algorithm 1

2:$opt\\_goal\leftarrow K.optimization\\_goal$

3:$ranking\leftarrow DNN\\_output.opt\\_goal$ $\triangleright$ Use prediction
of Algorithm 1, line 2

4:$ranked\\_subspace\leftarrow pred\\_subspace$.sort($ranking$)

5:$valid\\_found\leftarrow$ False, $idx\leftarrow$ 0

6:$verified\\_subspace\leftarrow\emptyset$

7:while not $valid\\_found$ & idx $<ranked\\_subspace.size$ do

8: $adapt\\_opt\leftarrow ranked\\_subspace$[$idx$]

9: $Analyzer$.analyzeAdaptationOptions($adapt\\_opt$) $\triangleright$
Knowledge

10: $\\_,qualities\leftarrow K.verification\\_results$[$idx$]

11: if $qualities$ meet all thresholds and set-point goals then

12: $valid\\_found\leftarrow$ True

13: end if

14: $verified\\_subspace$.add($adapt\\_opt$)

15: $idx\leftarrow idx$ \+ 1

16:end while

17:Select valid adaptation option $adapt\\_opt$, if none valid use fall back
option

18:$unselected\leftarrow K.adaptation\\_options\setminus verified\\_subspace$

19:$explore\leftarrow unselected$.randomSelect($exploration\\_rate$)

20:$Analyzer$.analyzeAdaptationOptions($explore$) $\triangleright$ Knowledge

21:$input\\_vectors$, $qualities\leftarrow K.verification\\_results$

22:$K.DNN\\_model$.update($input\\_vectors$, $qualities$)

In lines 2 to 4, the predictions of the previous stage are reused to obtain a
ranking of the relevant adaptation options. Since we have a single deep neural
network model, we require only a single prediction (see algorithm 1, line 2).
Lines 5 to 16 iterate over the ranked adaptation options in descending order
of the predicted value for the quality of the maximization goal (the opposite
order is used for a minimization goal). The adaptation option is analyzed to
check whether it complies with the threshold and set-point goals. If this is
the case this option is selected for adaptation. If not, analysis is continued
until an adaptation option is found that satisfies the threshold and set-point
goals. After iterating through the ranked adaptation (sub)space, a valid
option is selected to adapt the system, i.e., an adaptation option that
satisfies all the threshold and set-point goals (see line 17). If no valid
adaptation option is found according to the goals, a fall-back option is used
ensuring graceful degradation of the system. Then, lines 18 and 19 use the
exploration_rate to select a random sample of adaptation options from the
options that were not analyzed. These options are then also analyzed in line
20 and the analysis results are stored in the knowledge. Finally, lines 21 and
22 exploit the analysis results to update the deep neural network model. This
enables the model to cope with the dynamic behavior of the adaptation space.

####  6.2.3. Runtime integration of DLASER+ with MAPE-K

Figure 13 shows how DLASER+ is integrated with a MAPE-K feedback loop.

Figure 13. Runtime integration of DLASER+ with MAPE

We follow here the MAPE model and the tasks associated with the different MAPE
elements as specified in (Weyns, 2020). When the monitor finishes an update of
the knowledge with new runtime data, it triggers the analyzer. The analyzer
reads the data that is necessary for the deep neural network to determine the
relevant adaptation options. The deep neural network then produces predictions
that are written to the knowledge (steps 1 to 4). Next, the analyzer triggers
the planner that initiates adaptation space reduction; the ranked adaptation
options are then written to the knowledge (steps 5 to 7). Next, the planner
reads the ranked options and invokes the model verifier to analyze the options
one by one and writes the results to the knowledge (steps 8 to 10). When a
suitable adaptation is found, the planner invokes the deep neural network to
read the verification results and update the learning model accordingly (steps
11 to 13). During these steps, the deep neural network trains the learning
model using the most recent verification results that are generated by the
model verifier. This online learning in each adaptation cycle helps improving
the learner over time to deal with new conditions that the system encounters.
Finally, the planner generates a plan for the adaptation option that is
selected to adapt the system and triggers the executor to enact the actions of
the plan on the managed system (steps 14 and 15).

##  7\. Evaluation

We start with describing the evaluation setup and specify the various
combination of adaptation goals that are evaluated. Then we explain the
offline results we obtained. Finally, we zoom in on the online results for
effectiveness and efficiency of DLASeR+. All material used for the evaluation,
incl. all configurations and settings, and all evalution results are available
at the DLASeR website.141414
https://people.cs.kuleuven.be/danny.weyns/software/DLASeR/index.html

###  7.1. Evaluation setup

We evaluated DLASeR+ on two instances of DeltaIoT, as described in Section 2.
For both instances we used the same type of network settings. The stochastic
uncertainty profiles for the traffic load generated by the motes ranged from 0
to 10 messages per mote per cycle, while the network interference along the
links varied between -40 dB and +15 dB. The configurations of these profiles
are based on field tests. The MAPE-K feedback loop was designed using a
network of timed automata. These runtime models were executed by using the
ActivFORMS execution engine (Iftikhar and Weyns, 2014; Weyns and Iftikhar,
2022). The quality models were specified as stochastic timed automata. These
models are used to determine the quality estimates for the adaptation options.
Section 2 explains an example model for packet loss. We applied runtime
statistical model checking using Uppaal-SMC for the verification of adaptation
options (David et al., 2015). The exploration rate was set to 5%. For both
instances of DeltaIoT, we considered 275 online adaptation cycles,
corresponding with a wall clock time of 77 hours. We used 45 cycles to train
the network parameters. The remaining 230 cycles are evaluated as learning
cycles.

To evaluate the effect on the realization of the adaptation goals using
DLASeR+, we compare the results with a reference approach that analyzes the
whole adaptation space without learning. For the evaluation of coverage of
relevant adaptation options and the reduction of the adaptation space, we
could only compare the results for settings with threshold goals and an
optimization goal with the basic DLASeR approach (Van Der Donckt et al., 2020)
and ML4EAS (short for “Machine Learning for Efficient Adaptation Space
Reduction”) proposed by Quin et al. (Quin et al., 2019). The latter approach
applies classic machine learning techniques to reduce adaptation spaces for
threshold goals.

Table 2 summarizes the combinations of adaptation goals we evaluated for both
instances of DeltaIoT: TTO (2 Threshold goals and 1 Optimization goal), TTS (2
Threshold goals and 1 Set-point goal), and TSO (1 Threshold, 1 Set-point, and
1 Optimization goal).151515For TTO setting, ML4EAS applies classifiers to
reduce the adaptation space for the two threshold goals and then searches
within the reduced adaptation space to find the best adaptation option for the
optimization goal.

Reference | Goals DeltaIoTv1 | Goals DeltaIoTv2
---|---|---
TTO |  T1: $PL<10$% |  T1: $PL<10$%
T2: $LA<5$% |  T2: $LA<5$%
O1: minimize $EC$ |  O1: minimize $EC$
TTS |  T1: $PL<15$% |  T1: $PL<15$%
T2: $LA<10$% |  T2: $LA<10$%
S1: $EC\in[12.9\pm 0.1]$ |  S1: $EC\in[67\pm 0.3]$
TSO |  T1: $LA<10$% |  T1: $LA<10$%
S1: $EC\in[12.9\pm 0.1]$ |  S1: $EC\in[67\pm 0.3]$
O1: minimize $PL$ |  O1: minimize $PL$
Table 2. Overview of combinations of adaptation goals we evaluated for both
instances of DeltaIoT. The adaptation goals are defined for packet loss (PL),
latency (LA), and energy consumption (EC).

For the implementation of DLASeR+ we used the scalers from scikit-learn
(Pedregosa et al., 2011) and the neural networks from Keras and Tensorflow
(Abadi et al., 2016). The simulated IoT networks are executed on an i7-3770
CPU @ 3.40GHz with 12GB RAM; the deep learning models are trained and
maintained on an i7-3770 CPU @ 3.40GHz with 16GB RAM.

We start with presenting the results of the offline stage of the DLASeR+
learning pipeline. Then we present the results for effectiveness and
efficiency of the online stage of the learning pipeline for each combination
of goals.

###  7.2. Results offline settings

As explained in Section 6.1, DLASeR+ uses grid search for configuring and
tuning the deep neural network. We performed grid search on 30 sequential
adaptation cycles.161616Concretely, we used the data of all adaptation options
with their verification results over 30 adaptation cycles, i.e., 216
adaptation options with verification results per cycle for DeltaIoTv1 and 4096
adaptation options with verification results per cycle for DeltaIoTv2. Figure
5 and Figure 6 illustrate the performance of different adaptation options (for
DeltaIoTv2). Table 3 shows the best parameters of the network for each of the
three combinations of adaptation goals. Each row in the table corresponds to
one grid search process. In total, 6 grid search processes were completed; one
for each of the three combinations of adaptation goals and this for the two
instances of DeltaIoT.

Problem Goals Hyper parameters Scaler Batch size LR Optimizer Core layers
Class. layers Regr. layers DeltaIoTv1 TTO Standard 64 5e-3 Adam [50, 25, 15]
[20,10,5] [40, 20, 10, 5] DeltaIoTv2 TTO Standard 512 5e-3 Adam [150, 120,
100, 50, 25] [40, 20, 10, 5] [30, 40, 50, 40, 15, 5] DeltaIoTv1 TTS
MaxAbsScaler 64 5e-3 Adam [50, 25, 15] [40, 20,10,5] / DeltaIoTv2 TTS
MaxAbsScaler 512 5e-3 Adam [200, 100, 50, 25] [40, 20, 10, 5] / DeltaIoTv1 TSO
StandardScaler 16 5e-3 Adam [50, 80, 35, 15] [20,10,15] [40,20,10,5]
DeltaIoTv2 TSO StandardScaler 512 2e-3 RMSprop [150, 120, 100, 50, 25] [40,
20, 10, 5] [30, 40, 50, 40, 15, 5]

Table 3. Best grid search results for DLASeR+ on TTO, TTS, and TSO goal
combinations (see Table 2). LR refers to learning rate. The values between
brackets for the different layers represent the number of neurons.

Given that DLASeR+ comprises a single integrated neural network architecture
with multiple classification and regression heads, we use the validation loss
to select the best hyper-parameter configuration. The validation loss captures
how good the predictions are of the neural network model compared to the true
data (of the validation set). Here, the validation loss corresponds to the sum
of the losses for each head. In total, grid search evaluated 4120
configurations for DeltaIoTv1 (1728 for TTO and TSO goals and 768 for TTS
goals) and 3456 configurations for DeltaIoTv2 (1296 for TTO and TSO goals and
864 for TTS goals). The average time that was required for the offline
training of the deep neural network for a configuration was on 25s for
DeltaIoTv1 and 90s for DeltaIoTv2. We observe that the loss for the
configuration with TTO goals is significantly lower for DeltaIoTv1. Overall,
the loss for DeltaIotv2 is somewhat lower. Yet, as we will show in the
following subsection, the differences do not lead to significantly inferior
results.

We used the configurations with the best results for the different instances
of DLASeR+ shown in Table 3 to perform adaptation space reduction for the
three configurations with different adaptation goals (see Table 2) for both
versions of DeltaIoT.

###  7.3. Results online setting

We present the results for the different settings shown in Table 2, starting
with the metrics for effectiveness, followed by the metrics for efficiency.
The results allow us to to answer the different aspects of the research
question in a structured manner.

####  7.3.1. Effectiveness - Coverage of relevant adaptation options

To assess the first aspect of the effectiveness of DLASeR+, we look at the
F1-score for the threshold and set-point goals, and Spearman’s rho for the
optimization goal. Table 4 presents the results.

Note that the result of DLASeR+ for the classification task is a continuous
value that needs to be rounded to an integer class number. Hence, the
classification error can be computed in two ways. The first method measures
the error before rounding, e.g., if the real class of an input data is “1” and
the predicted class is “0.49”, the error will be “$1-0.49=0.51$”. The second
method measures the error after rounding, e.g., if the real class of an input
data is “1” and the predicted class is “0.49”, the error will be “$1-0=1$” as
“0.49” has been fixed to “0”. Depending on which method is selected for
computing the classification error, the F1-score can have different values. We
use the second method for calculating the classification error, enabling a
comparison of the results of DLASeR+ with ML4EAS, where the output is an
integer class number instead of a continuous value.

Problem | Setting | Method | F1 threshold | F1 set-point |  | Spearman’s rho
---
optimization
DeltaIoTv1 |  | DLASeR+ | 86.92% | / | 83.43%
TTO | DLASeR | 83.92% | / | 43.34%
| ML4EAS | 75.02% | / | /
TTS | DLASeR+ | 57.12% | 33.77% | /
TSO | DLASeR+ | 75.60% | 43.75% | 96.49%
DeltaIoTv2 |  | DLASeR+ | 63.16% | / | 21.53%
TTO | DLASeR | 62.35% | / | 5.81%
| ML4EAS | 89.87% | / | /
TTS | DLASeR+ | 60.48% | 35.14% | /
TSO | DLASeR+ | 64.19% | 35.20% | 95.99%
Table 4. Results for the coverage of relevant adaptation options for the
different evaluation settings.

For DeltaIoTv1, we notice an F1-score for the threshold goals of 86.92% for
the setting TTO, 57.12% for TTS, and 75.60% for TSO (with 100% being perfect
precision and recall). The F1-scores for the set-point goal for TTS and TSO
are respectively 33.77% and 43.75%. The differences can be explained by the
constraints imposed by the types of goals combined in the different settings,
with setpoint goals being most constraining, followed by threshold goals and
then optimization goals. Consequently, the F1-score is highest for TTO with
two threshold goals, followed by TSO with one threshold goal and one setpoint
goal, and finally TTS that combines two threshold goals with a setpoint goal.
For Spearman’s rho, we observe a difference between the settings with an
optimization goal, with 83.43% and 96.49% for TTO and TSO respectively. This
shows that regression is more difficult for energy consumption (TTO) compared
to packet loss (TSO).

For DeltaIoTv2, we observe an F1-score for the threshold goals of 63.16% for
the TTO setting, 60.48% for TTS, and 64.19% for TSO. The F1-score for the two
set-point goals, are around 35%. These results for F1-score are slightly lower
compared to the results for DeltaIoTv1, indicating that the setting is more
challenging for the learners. For Spearman’s rho, we observe values of 21.53%
and 95.99% for TTO and TSO respectively. The weak score for the setting with
TTO may point to a negative effect on the optimization goal caused by the
training of the core layers for the two threshold goals. In particular, this
may indicate that the knowledge shared in the core layers for the threshold
goals and the optimization goal is limited. While this may seem problematic,
the evaluation will show that lower Spearman’s rho values do not necessarily
imply inferior results.

Overall, we obtained acceptable to excellent results for the F1-score and
Spearman’s rho (with one exception). Compared to the results obtained for the
initial version of DLASeR (Van Der Donckt et al., 2020) and ML4EAS (Quin et
al., 2019), we observe similar results for the TTO setting (the results with
ML4EAS were somewhat better for DeltaIoTv2, but somewhat worse for
DeltaIoTv1). However, it is important to emphasize that these approaches do
not consider the other combinations of adaptation goals.

####  7.3.2. Effectiveness - Reduction adaptation space

Table 5 presents the results for adaptation space reduction obtained with the
different approaches for the different configurations. We observe that the
highest average adaptation space reduction (AASR) with DLASeR+ is achieved for
the TTO setting and this for both IoT instances. The results are slightly
better compared to the initial DLASeR but slightly worse compared to ML4EAS.
The AASR is lower for the other combinations of goals, which indicates that
reducing the adaptation space with DLASeR+ for settings with a setpoint goal
is more challenging. On the other hand, for the average analysis effort
reduction (AAER), we notice that only a limited number of adaptation options
need to be analyzed from the selected subspace before a valid option is found
that complies with the classification goals. This is particularly the case for
TTS and TSO settings that include a setpoint goal. This means that the
adaptation space reductions captured by AASR is of high quality.

Besides AASR and AAER, we also measured the fraction of the adaptation space
that was analyzed of the total adaptation space, which combines $AASR$ and
$AAER$,171717The definition of total reduction can be rewritten as: Total
Reduction $=100-(100-AASR)\times\left(1-\frac{AAER}{100}\right)$ defined as:

(11) $\textit{Total Reduction}\ =(1-\frac{analyzed}{total})\times 100$

For DeltaIoTv1, we measured a total reduction of 95.72%, 99.16%, and 96.41%
for TTO, TTS and TSO respectively, while the total reduction for DeltaIoTv2
was 99.52%, 99.94%, and 99.98%. These excellent results show that deep
learning with DLASeR+ is particularly effective in reducing adaptation spaces,
i.e., the total reduction is near to the optimum of what can be achieved.

Problem | Setting | Method | AASR | AAER | Total Reduction
---|---|---|---|---|---
DeltaIoTv1 |  | DLASeR+ | 56.77% | 90.11% | 95.72%
TTO | DLASeR | 54.84% | 88.88% | 94.98%
| ML4EAS | 62.61% | 0.00% | 62.61%
TTS | DLASeR+ | 42.88% | 98.54% | 99.16%
TSO | DLASeR+ | 38.27% | 94.18% | 96.41%
DeltaIoTv2 |  | DLASeR+ | 89.03% | 95.57% | 99.52%
TTO | DLASeR | 84.55% | 72.41% | 95.73%
| ML4EAS | 92.86% | 0.00% | 92.86%
TTS | DLASeR+ | 41.79% | 99.90% | 99.94%
TSO | DLASeR+ | 51.00% | 99.95% | 99.98%
Table 5. Adaptation space reductions on the three configurations of Table 2.

####  7.3.3. Effectiveness - Effect on realization of adaptation goals

To evaluate the effectiveness, we compare the median values of the quality
properties that correspond to the adaptation goals over 230 learning cycles
(i.e., representing about three days of operation of the IoT networks, see
evaluation setup). Note that a threshold goal is satisfied if the median of
the values over 230 cycles satisfies the goal. This does not necessarily mean
that the system satisfies the goal in all cycles.

The boxplots of Figures 14, 15, and 16 show the results for the quality
properties of the adaptation goals with DLASeR+ and other approaches. It is
important to note that the reference approach exhaustively analyzes the whole
adaptation space. This is the ideal case, but practically not always feasible
(as in DeltaIoTv2) due to time constraints on the time available to perform
adaptation.

For the setting with TTO goals, see Figure 14, the results for packet loss and
latency (threshold goals) are similar for all approaches. We observe that
DLASeR+ always satisfies the goals (i.e., all median values are below the
thresholds). For some adaptation cycles, no configurations are available that
satisfy the threshold goals (as shown by the reference approach that
exhaustively searches through the complete adaptation space). For energy
consumption (optimization goal), the results for DLASeR+ are slightly higher
compared to the reference approach, i.e., an increase of 0.03 C (0.24%) for
DeltaIoTv1 (median of 12.69 C for the reference approach; 12.72 C for
DLASeR+), and 0.03 C (0.05%) for DeltaIoTv2 (66.15 C for the reference
approach; 66.18 C for DLASeR+).

For the setting with TTS goals, see Figure 15, we observe similar results for
DLASeR+ and the reference approach. The threshold goals for packet loss and
latency are again always satisfied by DLASeR+. The set-point goal of energy
consumption is also satisfied for both instances of DeltaIoT (DLASeR+ shows
slightly more variability around the set-point).

Finally, for the setting with TSO goals, see Figure 16, the results show that
the threshold goal for latency is always satisfied with DLASeR+ and the same
applies for the set-point goal for energy consumption. As for the optimization
goal, the packet loss of DLASeR+ is slightly higher compared to the reference
approach, respectively 0.57% for DeltaIoTv1 (median of 6.33% for the reference
approach versus 6.90% for DLASeR+), and 0.62% for DeltaIoTv2 (6.33% for the
reference approach versus 6.95% for DLASeR+).

In summary, the results for the quality properties using DLASeR+ and the
reference approach are similar. For settings with two threshold goals and one
optimization goal, the results are similar as for two state-of-the-art
approaches, the initial DLASeR (Van Der Donckt et al., 2020) and ML4EAS (Quin
et al., 2019). However, contrary to these state-of-the-art approaches, DLASeR+
realizes adaptation space reduction for combinations of threshold, set-point
goals, and optimization goals with a small to negligible effect on the
qualities compared to the reference approach.

(a) DeltaIoTv1

(b) DeltaIoTv2

Figure 14. Effect on the realization of the adaptation goals for the TTO
setting.

(a) DeltaIoTv1

(b) DeltaIoTv2

Figure 15. Effect on the realization of the adaptation goals for the TTS
setting.

(a) DeltaIoTv1

(b) DeltaIoTv2

Figure 16. Effect on the realization of the adaptation goals for the TSO
setting.

####  7.3.4. Efficiency - Learning time

Table 6 presents the results for the analysis time, learning time, and overall
time reduction of DLASeR+ compared to the reference approach. As we can see,
the major part of the time is spent on analysis, i.e., the time used for the
verification of adaptation options. This is in contrast with the learning
time;181818Recall that the learning time is the sum of the time used for
online prediction and online training, cf. Table 1. on average 16.45% of the
time is spent on learning for DeltaIoTv1 (total time for verification of the
three settings is 4.45s versus 0.894s for learning) and 1.56% for DeltaIoTv2
(102.95s in total for verification of the three settings versus 1.63s for
learning). We observe high numbers for time reduction, on average 92.80% for
the three settings of DeltaIoTv1 and 94.84% for the settings of DeltaIoTv2.
For TTO settings, DLASeR+ realizes better time reductions than the other
approaches. Note that the optimal time reduction is 95%, since we use an
exploration rate of 5%, i.e., in each adaptation cycle, 5% of the adaptation
space is explored.

In sum, the learning time of DLASeR+ is only a fraction of the total time used
for analysis. DLASeR+ realizes an impressive reduction of the time required
for analysis and decision-making compared to the reference approach.

Problem | Setting | Method | Verification Time | Learning Time | Time Reduction
---|---|---|---|---|---
DeltaIoTv1 |  | DLASeR+ | 1.95 s | 0.004 s | 90.72%
TTO | DLASeR | 2.13 s | 0.003 s | 89.87%
| ML4EAS | 6.99 s | 0.002 s | 66.80%
TTS | DLASeR+ | 1.11 s | 0.45 s | 94.73%
TSO | DLASeR+ | 1.48 s | 0.44 s | 92.96%
DeltaIoTv2 |  | DLASeR+ | 36.44 s | 0.45 s | 94.52%
TTO | DLASeR | 61.21 s | 1.62 s | 90.79%
| ML4EAS | 68.44 s | 0.04 s | 89.71%
TTS | DLASeR+ | 33.27 s | 0.63 s | 95.00%
TSO | DLASeR+ | 33.24 s | 0.55 s | 95.00%
Table 6. Verification and learning time for DLASeR+. Time reduction compares
the total time used by DLASeR+ compared to the time used by the reference
approach that verifies the complete adaptation space.

####  7.3.5. Efficiency - Scalability

To evaluate the scalability of DLASeR+, we discuss the difference for the
metrics when scaling up the evaluation setting from DeltaIoTv1 with 216
adaptation options to DeltaIoTv2 with 4096 adaptation options; an increase of
adaptation space with a factor around 20.

For the coverage of relevant adaptation options (see Table 4), we observe a
decrease in F1-score for the threshold goals of 23.76% for the TTO setting
(86.92% for DeltaIoTv1 versus 63.16% for DeltaIoTv2), and a decrease of 11.41%
for the TSO setting (75.60% versus 64.18%). For the TTS settings we notice a
small increase of the F1-score for the threshold goals of 3.36% (57.12% versus
60.48%). For set-point goals, we observe a small increase in F1-score of 1.37%
for the setting TTS (33.77% versus 35.14%), and a small decrease of 8.55% for
the setting TSO (43.75% versus 35.20%). The results show that DeltaIoTv2 is
more challenging, but DLASeR+ scales well for threshold goals. For the
optimization goal, we notice a large decrease for Spearman’s rho with 61.9%
for the setting with TTO goals (from 83.43% for DeltaIoTv1 to 21.53% for
DeltaIoTv2), and a small decrease of 0.5% for the setting with TSO goals (from
96.49% to 95.99%). These results suggest that the optimization goal in TTO
(energy consumption) might be significantly harder to predict.

For the average adaptation space reduction (see Table 5), we measured a
substantial increase for two configurations and a small decrease for one
configuration when scaling up from DeltaIoTv1 to DeltaIoTv2. On average, we
measured an increase of 14.63% over the three configurations (from 56.77% to
89.03% for TTO and from 38.27% to 51.00% for TSO; from 42.88% to 41.79% for
TTS). The average analysis effort reduction also improved for DeltaIoTv2, with
an average of 4.20% (from 90.11% to 95.57% for TTO, from 98.54% to 99.90% for
TTS, and from 94.18% to 99.95% for TSO. This resulted in an average increase
of 2.72% for the total reduction (from 95.72% to 99.52% for TTO, from 99.16%
to 99.94% for TTS, and from 96.41% to 99.98% for TSO).

For the effect on realizing the adaptation goals (see Figure 14, 15, and 16),
we observe that DLASeR+ realizes the threshold and set-point goals for all
configurations of both DeltaIoTv1 and DeltaIoTv2. For the optimization goal
compared to the reference approach, we observe a slight increase of energy
consumption for the TTO setting with DLASeR+ from 0.24% extra with DeltaIoTv1
(medians 12.69 C versus 12.72 C) to 0.05% extra with DeltaIoTv2 (medians 66.15
C versus 66.18 C). Similarly, we observe a small increase of packet loss for
TSO with DLASeR+ from 0.57% extra with DeltaIoTv1 (medians 6.33% versus 6.90%)
to 0.62% extra with DeltaIoTv2 (medians 6.33% versus 6.95%).

Finally, the results for the learning time (see Table 6) show that relative
part of the time required for learning compared to the time required for
analysis decreases from 16.45% for DeltaIoTv1 to 1.56% for DeltaIoTv2, while
the total time reduction improves from 92.80% to 94.84% (the numbers are
averages over the three settings TTO, TTS, and TSO). The results show that
with increasing scale, the total time reduction improves substantially and
gets close to optimum for DeltaIoTv2.

In summary, while some of the indicators for the coverage of relevant
adaptation options are slightly inferior for configurations with larger
adaptation spaces, the other metrics show no negative effects on the
effectiveness and efficiency of DLASeR+ when the size of the adaptation space
is increased (in the evaluating setting with a factor of about 20).

###  7.4. Threats to validity

The evaluation results show that DLASeR+ is an effective and efficient
approach to reduce large adaptation spaces. However, the evaluation of DLASeR+
is subject to a number of validity threats.

External validity. Since we evaluated the approach in only one domain with a
particular adaptation spaces and adaptation goals, we cannot generalize the
conclusions, including the configuration of DLASeR+, and the effectiveness and
efficiency of adaptation space reduction. We mitigated this threat to some
extent by applying DLASeR+ to two IoT networks that differ in their topology
and size of adaptation space. Nevertheless, more extensive evaluation is
required in different domains to strengthen the validity of the results.
Furthermore, the reduction stages of the unified architecture of DLASeR+
presented in this paper target only a single optimization goal. Hence, the
approach is not directly applicable to systems with multi-objective
optimization goals. Additional research will be required to extend DLASeR+ for
systems with such types of goals. We also tested DLASeR+ for scenarios with up
to 4000 adaptation options in one domain. Further research is required to
study and evaluate other types of systems with much larger adaptation spaces.

Internal validity. We evaluated the effectiveness and efficiency of DLASeR+
using different metrics. It might be possible that the specifics of the
evaluation setting of the applications that we used, in particular the
topology of the network, the uncertainties, and the choices for the specific
goals that we considered may have an effect on the complexity of the problem
of adaptation space reduction. To mitigate this threat to some extent we
applied DLASeR+ to simulated settings of real IoT deployments that were
developed in close collaboration with an industry partner in IoT.

Reliability. For practical reasons, we performed the evaluation of DLASeR+ in
simulation. The uncertainties used in this simulation are based on stochastic
models. This may cause a threat that the results may be different if the study
would be repeated. We minimized this threat in three ways: (i) the profiles
are designed based on field tests, (ii) we evaluated DLASeR+ over a period of
several days, and (iii) the complete package we used for evaluation is
available to replicate the study.191919DLASeR website:
https://people.cs.kuleuven.be/danny.weyns/software/DLASeR/index.html

##  8\. Related work

Over the past years, we can observe an increasing interest in using machine
learning and related techniques to support self-adaptation (Gheibi et al.,
2021a). Three examples of initial work are Richert W. et al. (Richert and
Kleinjohann, 2008) that apply reinforcement learning to robots that learn to
select tasks that balance their own benefit with the needs of other robots,
Sykes et al. (Sykes et al., 2013) that use a probabilistic rule learning
approach to update the environment models of a self-adaptive system at runtime
relying on execution traces, and Bencomo et al. (Bencomo et al., 2013) that
use dynamic decision networks to model and support decision-making in self-
adaptive systems explicitly taking into account uncertainty.

In this section, we focus on the use of machine learning and other related
techniques that are used for adaptation space reduction and efficient
decision-making of systems with complex adaptation spaces. We have structured
the related work in four groups based on their main focus: (1) learning used
for the reduction of adaptation spaces, (2) learning used for efficient
decision-making, (3) search-based techniques to efficiently explore large
adaptation spaces, and finally (4) approaches for efficient verification. For
each group we present a representative selection of related work.

###  8.1. Learning used for adaptation space reduction

Elkhodary et al. (Elkhodary et al., 2010) propose the FUSION framework that
learns the impact of adaptation decisions on the system’s goals. In
particular, FUSION utilizes M5 decision trees to learn the utility functions
that are associated with the qualities of the system. The results show a
significant improvement in analysis. Whereas DLASeR+ targets the adaptation
space, FUSION targets the feature selection space, focusing on proactive
latency-aware adaptations relying on a separate model for each utility.

Chen et al. (Chen and Bahsoon, 2016) study feature selection and show that
different learning algorithms perform significantly different depending on the
types of quality of service attributes considered the way they fluctuate. The
work is centered on an adaptive multi-learners technique that dynamically
selects the best learning algorithms at runtime. Similar to our work, the
authors focus on efficiency and effectiveness, but the scope of that work is
on the features instead of adaptation options.

Quin et al. (Quin et al., 2019) apply classical machine learning techniques,
in particular classification and regression, to reduce large adaptation
spaces. These techniques require domain expertise to perform feature
engineering, which is not required in DLASeR+ (that only requires model
selection). That work also only considers threshold goals based on linear
learning models. In contract, our work considers threshold, optimization, and
set-point goals based on non-linear deep learning models.

Jamshidi et al. (Jamshidi et al., 2019) presents an approach that learns a set
of Pareto optimal configurations offline that are then used during operation
to generate adaptation plans. The approach reduces adaptation spaces, while
the system can still apply model checking with PRISM (Kwiatkowska et al.,
2002) at runtime to quantitatively reason about adaptation decisions. Compared
to that work, DLASeR+ is more versatile by reducing the adaptation space at
runtime in a dynamic and learnable fashion.

Metzger et al. (Metzger et al., 2019) apply online learning to explore the
adaptation space of self-adaptive systems using feature models. The authors
demonstrate a speedup in convergence of the online learning process. The
approach is centered on the adaptation of rules, whereas DLASeR+ is centered
on model-based self-adaptation. Furthermore, the work also looks at the
evolution of the adaptation space, while DLASeR+ only considers dynamics in
the adaptation space (not its evolution).

Camara et al. (Cámara et al., 2020) use reinforcement learning to select an
adaptation pattern relying on two long-short-term memory deep learning models.
Similar to our work, these authors demonstrate the benefits of integrating
machine learning with runtime verification. However, the focus differs in the
type of goals considered (they only consider threshold goals) and the type of
verification used (they use runtime quantitative verification); that work also
does not consider scalability.

Stevens et al. (Stevens and Bagheri, 2020) present Thallium that exploits a
combination of automated formal modeling techniques to significantly reduce
the number of states that need to be considered with each adaptation decision.
Thallium addresses the adaptation state explosion by applying utility bounds
analysis. The (current) solution operates on a Markov decision process that
represents the structure of the system itself, independent of the stochastic
environment. The authors suggest future work in combining learning-based
approaches employed on the reduced adaptation space from Thallium.

###  8.2. Learning used for efficient decision-making

Kim et al. (Kim and Park, 2009) present a reinforcement learning-based
approach that enables a software system to improve its behavior by learning
the results of its behavior and dynamically changing its plans under
environmental changes. Compared to DLASeR+, the focus is on effective
adaptation decision-making, without considering guarantees or the scalability
of the proposed approach.

Anaya et al. (Anaya et al., 2014) present a framework for proactive adaptation
that uses predictive models and historical information to model the
environment. These predictions are then fed to a reasoning engine to improve
the decision-making process. The authors show that their approach outperforms
a typical reactive system by evaluating different prediction models
(classifiers). Whereas they focus on leveraging proactive techniques to make
decision-making more effective, our work focuses on both the efficiency and
effectiveness of adaptation space reduction to improve the decision-making.

Qian et al. (Qian et al., 2015) study goal-driven self-adaptation centered on
case-based reasoning for storing and retrieving adaptation rules. Depending on
requirements violations and context changes, similar cases are used and if
they are not available, goal reasoning is applied. This way, the approach
realizes more precise adaptation decisions. The evaluation is done only for
threshold goals and the authors only provide some hints to scale up their
solutions to large-sized systems. In our work, we explicitly evaluate the
effect of scaling up the adaptation space.

Nguyen Ho et al. (Ho and Lee, 2015) rely on model-based reinforcement learning
to improve system performance. By utilizing engineering knowledge, the system
maintains a model of interaction with its environment and predicts the
consequence of its action. DLASeR+ relies on a different learning techniques.
Furthermore, we study the effect of scalability of the adaptation space, which
is not done in that paper.

###  8.3. Search-based techniques to explore large adaptation spaces

Cheng et al. (Cheng et al., 2013) argue for three overarching techniques that
are essential to address uncertainty in software systems: model-based
development, assurance, and dynamic adaptation. In relation to the work
presented in this paper, the authors argue for the application of search-based
software engineering techniques to model-based development, in particular, the
use of evolutionary algorithms to support an adaptive system to self-
reconfigure safely. In (Ramirez et al., 2010), the authors propose Hermes, a
genetic algorithmic approach, that adapts the system efficiently in time. In
(Ramirez et al., 2009), the authors propose Plato, an approach that maps data
monitored from the system or the environment into genetic schemes and evolves
the system by leveraging genetic algorithms. The main aim of these approaches
is ensuring safety under uncertainty in an efficient manner. In contrast,
DLASeR+ is conceptually different, relying on deep learning to explicitly
reduce large adaptation spaces, providing explicit support for different types
of adaptation goals.

Le Goues et al. (Le Goues et al., 2012) propose GenProg, an automated method
for repairing defects in legacy programs. GenProg relies on genetic
programming to evolve a program variant that retains required functionality
but is not susceptible to a given defect, using existing test suites to encode
both the defect and required functionality. The focus of this work is on
efficiently producing evolved programs that repair a defect, without
introducing substantial degradation in functionality. The focus of DLASeR+ on
the other hand is on reducing large adaptation spaces at architectural level
aiming to enhance the efficiency of the decision-making of self-adaptive
systems that need to deal with different types of quality properties.

Nair et al. (Nair et al., 2018) present FLASH that aims at efficiently finding
good configurations of a software system. FLASH sequentially explores the
configuration space by reflecting on the configurations evaluated so far to
determine the next best configuration to explore. FLASH can solve both single-
objective and multi-objective optimization problems. Whereas FLASH assumes
that the system is stationary, DLASeR+ uses incremental learning to stay up to
date during operation; i.e., DLASeR+ deals with dynamics in the environment at
runtime.

Kinneer et al. (Kinneer et al., 2018) propose a planner based on genetic
programming that reuses existing plans. Their approach uses stochastic search
to deal with unexpected adaptation strategies, specifically by reusing or
building upon prior knowledge. Their genetic programming planner is able to
handle very large search spaces. Similar to DLASeR+, the evaluation of this
work considers efficiency and effectiveness. However, the technique used is
different focusing on planning and that work put particular emphasis on reuse.

Chen et al. (Chen et al., 2018) present FEMOSAA, a framework that leverages a
feature model and a multi-objective evolutionary algorithm to optimize the
decision-making of adaptation at runtime. The authors show that FEMOSAA
produces statistically better and more balanced results for tradeoff with
reasonable overhead compared to other search-based techniques. Compared to
DLASeR+, the authors use a different analysis technique and rely on feature
models. The latter implies that the approach relies on domain engineers to
construct a feature model for the self-adaptive system.

In (Coker et al., 2015), Coker et al. use genetic programming planning and
combine this with probabilistic model checking to determine the fitness of
plans for a set of quality properties. The proposed search-based approach
provides an integrated solution for guiding the decision-making of a self-
adaptive system. This approach requires a well-defined objective function. In
contrast, DLASeR+ focuses on the reduction of the adaptation space for
different types of adaptation goals. With DLASeR+, different types of
decision-making mechanisms can be combined.

Pascual et al. (Pascual et al., 2013) apply a genetic algorithm to generate
automatically at runtime configurations for adapting a system together with
reconfiguration plans. The generated configurations are optimal in terms of
functionality taking into account the available resources (e.g. battery).
Concretely, the configurations are defined as variations of the application’s
software architecture based on a so called feature model. In contrast, DLASeR+
targets the reduction of large adaptation spaces targeting quality properties
of the system that are formulated as adaptation goals.

###  8.4. Approaches for efficient verification

Filieri et al. (Filieri et al., 2011) present a mathematical framework for
efficient run-time probabilistic model checking. Before deployment, a set of
symbolic expressions that represent satisfaction of the requirements is pre-
computed. At runtime, the verification step simply evaluates the formulae by
replacing the variables with the real values gathered by monitoring the
system. By shifting the cost of model analysis partially to design time, the
approach enables more efficient verification at runtime. In later work
(Filieri et al., 2016), the authors elaborate on this and explain how the
mathematical framework supports reasoning about the effects of changes and can
drive effective adaptation strategies.  Whereas DLASeR+ focuses on reducing
the set of adaptation options during operation, their work focuses on
efficient runtime verification by offloading work before system deployment.

Gerasimou et al. (Gerasimou et al., 2014) propose three techniques to speed up
runtime quantitative verification, namely caching, lookahead and nearly-
optimal reconfiguration. The authors evaluate several combinations of the
techniques on various scenarios of self-adaptive systems. The focus of this
work is different from DLASeR+, but the proposed techniques are complementary
and can perfectly be integrated in our work.

Moreno et al. (Moreno et al., 2018) present an approach for proactive latency-
aware adaptation that relies on stochastic dynamic programming to enable more
efficient decision-making. Experimental results show that this approach is
close to an order of magnitude faster than runtime probabilistic model
checking to make adaptation decisions, while preserving the same
effectiveness. Whereas our approach focus on reducing the set of adaptation
options to improve analysis, their work focuses on fast verification; here
too, a system may benefit from a combination of both approaches.

Goldsby et al. (Goldsby et al., 2008) and Zhang et al. (Zhang et al., 2009)
present AMOEBA-RT, a run-time approach that provides assurance that
dynamically adaptive systems satisfy their requirements. In AMOEBA-RT, an
adaptive program is instrumented with aspects that non-invasively collect
state of the system that can then be checked against a set of adaptation
properties specified in A-LTL, an extended linear temporal logic. At run-time,
the instrumented code sends the collected state information to a runtime model
checking server that determines whether the state of the adaptive program
satisfies the adaptation properties. The focus of this work is on assuring
properties using runtime model checking. In contrast, DLASeR+ focuses on
adaptation space reduction. AMOEBA-RT can be used in tandem with DLASeR+ to
enhance the efficiency of the decision-making process.

Junges et al. (Junges et al., 2021) present a runtime monitoring approach for
partially observable systems with non-deterministic and probabilistic
dynamics. The approach is based on traces of observations on models that
combine non-determinism and probabilities. The authors propose a technique
called forward filtering to estimate the possible system states in partially
observable settings along with a pruning strategy to enhance its efficiency.
Based on empirical results, the authors propose a tractable algorithm based on
model checking conditional reachability probabilities as a more tractable
alternative. In contrast, DLASeR+ focuses on the reduction of large adaptation
spaces of self-adaptive systems that are subject to uncertainties that can be
expressed as parameters of runtime models. Yet, DLASeR+ can be combined with
the proposed approach to enhance the performance of decision-making in self-
adaptive systems.

##  9\. Conclusions

In this paper, we studied the research question: “How to reduce large
adaptation spaces and rank adaptation options effectively and efficiently for
self-adaptive systems with threshold, optimization, and set-point goals?” To
answer this question, we presented DLASeR+. DLASeR+ relies on an integrated
deep neural network architecture that shares knowledge of different adaptation
goals. The approach is flexible as the core layers can be easily extended with
goal-specific heads. The evaluation shows that DLASeR+ is an effective and
efficient approach to reduce large adaptation spaces, including for settings
with large adaptation spaces. The approach realizes the threshold and set-
point goals for all the configurations we tested on the DeltaIoT artifact.
Compared to the theoretical optimal, we observe only a small tradeoff for the
quality property of the optimization goal. Yet this is a small cost for the
dramatic improvement of adaptation time.

We are currently applying DLASeR+ to service-based systems, which will provide
us insights in the effectiveness of the approach beyond the domain of IoT. For
these systems, we are studying the reduction of adaptation spaces with sizes
far beyond the adaptation spaces used in the evaluation of this paper, posing
more challenging learning problems to DLASeR+. We also plan to extend DLASeR+
for multi-objective optimization goals. Beyond DLASeR+ and learning-based
approaches, we also plan to compare the approach with conceptually different
approaches for improving the analysis of large adaptation spaces (as discussed
in related work) and perform a tradeoff analysis. In the mid term, we plan to
look into support for dynamically adding and removing adaptation goals. We
also plan to explore the use of machine learning in support of self-adaptation
in decentralized settings (Quin et al., 2021). In the long term, we aim at
investigating how we can define bounds on the guarantees that can be achieved
when combining formal analysis techniques, in particular runtime statistical
model checking, with machine learning, a starting point is (Gheibi et al.,
2021b).

## References

  * (1)
  * Abadi et al. (2016) Martin Abadi et al. 2016\.  Tensorflow: A system for large-scale machine learning. In _12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)_. 265–283.
  * Agha and Palmskog (2018) Gul Agha and Kalm Palmskog. 2018.  A Survey of Statistical Model Checking.  _ACM Trans. Model. Comput. Simul._ 28, 1, Article 6 (Jan. 2018), 39 pages.  https://doi.org/10.1145/3158668
  * Anaya et al. (2014) Ivan Dario Paez Anaya, Viliam Simko, Johann Bourcier, Noël Plouzeau, and Jean-Marc Jézéquel. 2014.  A prediction-driven adaptation approach for self-adaptive sensor networks. In _Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems_. 145–154.
  * Athreya et al. (2013) Arjun P Athreya, Bruce DeBruhl, and Patrick Tague. 2013\.  Designing for self-configuration and self-adaptation in the Internet of Things. In _9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing_. IEEE, 585–592.
  * Bencomo et al. (2013) Nelly Bencomo, Amel Belaggoun, and Valery Issarny. 2013\.  Bayesian artificial intelligence for tackling uncertainty in self-adaptive systems: The case of dynamic decision networks. In _2nd International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE)_. 7–13.
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ 35, 8 (2013), 1798–1828.
  * Brownley (2020) Jason Brownley. 2020\.  _Data Preparation for Machine Learning_.  https://machinelearningmastery.com/data-preparation-for-machine-learning/
  * Calinescu et al. (2011) Radu Calinescu, Lars Grunske, Martha Kwiatkowska, Raffaela Mirandola, and Giordana Tamburrelli. 2011\.  Dynamic QoS Management and Optimization in Service-Based Systems.  _IEEE Transactions on Software Engineering_ 37, 3 (May 2011), 387–409.  https://doi.org/10.1109/TSE.2010.92
  * Cámara et al. (2020) Javier Cámara, Henry Muccini, and Karthik Vaidhyanathan. 2020\.  Quantitative Verification-Aided Machine Learning: A Tandem Approach for Architecting Self-Adaptive IoT Systems. In _Proceedings International Conference on Software Architecture, ICSA_. 11–22.
  * Castañeda et al. (2014) Lorena Castañeda, Norha M Villegas, and Hausi A Müller. 2014.  Self-adaptive applications: On the development of personalized web-tasking systems. In _Proceedings of the 9th international symposium on software engineering for adaptive and self-managing systems_. 49–54.
  * Chen and Bahsoon (2016) Tao Chen and Rami Bahsoon. 2016.  Self-adaptive and online QoS modeling for cloud-based software services.  _IEEE Transactions on Software Engineering_ 43, 5 (2016), 453–475.
  * Chen et al. (2018) Tao Chen, Ke Li, Rami Bahsoon, and Xin Yao. 2018\.  FEMOSAA: Feature-guided and knee-driven multi-objective optimization for self-adaptive software.  _ACM Transactions on Software Engineering and Methodology (TOSEM)_ 27, 2 (2018), 1–50.
  * Cheng et al. (2013) Betty Cheng, Andres Ramirez, and Philip K. McKinley. 2013\.  Harnessing evolutionary computation to enable dynamically adaptive systems to manage uncertainty. In _2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE)_. 1–6.  https://doi.org/10.1109/CMSBSE.2013.6604427
  * Cheng et al. (2009a) Betty HC Cheng, Rogerio de Lemos, Holger Giese, Paola Inverardi, Jeff Magee, Jesper Andersson, Basil Becker, Nelly Bencomo, Yuriy Brun, Bojan Cukic, et al. 2009a.  Software engineering for self-adaptive systems: A research roadmap.  In _Software engineering for self-adaptive systems_. Springer, 1–26.
  * Cheng et al. (2009b) Betty H. C. Cheng, Pete Sawyer, Nelly Bencomo, and Jon Whittle. 2009b.  A Goal-Based Modeling Approach to Develop Requirements of an Adaptive System with Environmental Uncertainty. In _Model Driven Engineering Languages and Systems_ , Andy Schürr and Bran Selic (Eds.). Springer, 468–483.
  * Coker et al. (2015) Zack Coker, David Garlan, and Claire Le Goues. 2015.  SASS: Self-Adaptation Using Stochastic Search. In _2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems_. 168–174.  https://doi.org/10.1109/SEAMS.2015.16
  * David et al. (2015) Alexandre David, Kim G. Larsen, Axel Legay, Marius Mikučionis, and Danny Bøgsted Poulsen. 2015\.  Uppaal SMC tutorial.  _International Journal on Software Tools for Technology Transfer_ 17, 4 (2015), 397–415.