subfig

# Online Stochastic Variational Gaussian Process Mapping for Large-Scale SLAM
in Real Time

Ignacio Torroba1 Marco Cella1 Aldo Terán2 Niklas Rolleberg2 and John
Folkesson1 This work was supported by Stiftelsen for Strategisk Forskning
(SSF) through the Swedish Maritime Robotics Centre (SMaRC) (IRC15-0046)The
authors are with 1 the Robotics Perception and Learning Division and 2 the
Naval Architecture Division at KTH Royal Institute of Technology, SE-100 44
Stockholm, Sweden{torroba, mcella, aldot, nrol<EMAIL_ADDRESS>
###### Abstract

Rao-Blackwellized particle filter (RBPF) SLAM solutions with Gaussian Process
(GP) maps can both maintain multiple hypotheses of a vehicle pose estimate and
perform implicit data association for loop closure detection in continuous
terrain representations. Both qualities are of particular interest for SLAM
with autonomous underwater vehicles (AUVs) in the open sea, where
distinguishable features are scarce. However, the applicability of GP
regression to parallel, real-time mapping in an RBPF framework remains limited
by the size of the survey and the computational cost of the GP training. To
overcome these constraints, in this letter we propose the adaption of
Stochastic Variational GP (SVGP) regression to online mapping in combination
with a novel, efficient particle trajectory storing in the RBPF. We show how
the resulting RBPF-SVGP framework can achieve real-time performance in an
embedded platform on two AUV surveys containing millions of points. We further
test the framework on a live mission on an AUV and we make the implementation
publicly available.

##  I INTRODUCTION

Autonomous underwater vehicles (AUVs) are becoming standard tools for
underwater exploration and seabed mapping in both scientific and industrial
applications [1, 2]. Their capacity to dive untethered allows them to reach
areas inaccessible to surface vessels and to collect data more closely to the
seafloor, regardless of the water depth. However, their navigation autonomy
remains bounded by the accuracy of their dead reckoning (DR) estimate of their
global position, severely limited in the absence of a priori maps of the area
and GPS signal. Global localization systems equivalent to the later exists for
the underwater domain, such as LBL or USBL. However they involve expensive
external infrastructure and their reliability decreases with the distance to
the AUV, making them unsuitable for deep sea surveys.

Figure 1: Time-lapse of 1 out of 100 RBPF-SVGP particles learning a
bathymetric survey of 8M points in real time.

Simultaneous localization and mapping (SLAM) techniques can ameliorate these
limitations by allowing the vehicle to autonomously adjust its DR estimate
based on corrections against a map of the area being built at survey time. For
bathymetric mapping, the de facto sensor in industry is the multibeam
echosounder (MBES), which directly produces 3D representations of the seabed.
Thus, the current state of the art SLAM solutions for deep sea navigation
target AUVs equipped with an MBES. Two examples of these are [3], with a
maximum a posteriori (MAP) approach to solving SLAM and [4], based on a Rao-
Blackwellized Particle Filter (RBPF) solution. The later methodology presents
two advantages when compared to submap-based graph SLAM, as in the former.
These two strong points of the RBPF are: i) its capacity to keep several
hypotheses of the vehicle pose and ii) its ability to handle the data
association problem implicitly. This second point is particularly important in
bathymetric SLAM with MBES, where autonomous data association remains a
pervasive challenge due to the scarcity of distinguishable features [5]. These
reasons motivated the recent work in [6], which presented an RBPF solution for
bathymetric SLAM based on Gaussian Process (GP) models of the terrain. The use
of GP regression (GPR) to represent bathymetry yields continuous and light
maps with a readily available measure of map uncertainty. However, the
challenge in the use of parallel GPR in online and large scale surveys in
embedded hardware arises from their training complexity being cubic on the
number of training inputs [7]. To overcome this limitation, [6] presented a
GPU-targeted implementation of the GPR for a faster and more efficient
learning. However, despite improvements in speed, their framework still
required the bathymetry to be discretized a priori into tiles, each with their
own GP, and it is unsuitable for embedded platforms.

To enable handling of surveys of millions of points without discretizing them
while accomplishing real-time, in water, performance integrated on an AUV
system on its payload computer, we propose in this work an RBPF solution based
on Stochastic Variational Gaussian Process (SVGP) [8] maps. Combining an
adaption of SVGP to online mapping with a novel implementation of the handling
of particles lineages, we show how our RBPF framework achieves real-time
performance with datasets of millions of MBES data points in a NVIDIA Jetson
Orin, with up to $100$ particles. We assess the capabilities of the presented
SLAM solution in three surveys collected with two AUVs, one of them live, and
we make the framework publicly available to encourage further research.

##  II RELATED WORK

The Rao-Blackwellized factorization of SLAM, first proposed in [9], divides
its posterior into two separate problems, the vehicle localization and the map
construction. Thus, the localization estimate can be computed via a particle
filter, in which each particle represents a weighted hypothesis of the AUV
pose [10]. Each particle also maintains its own map, from which expected
measurements can be computed. By comparing them to the real ones, those
particles whose maps resemble the real one more closely are resampled.

Regarding the filter structure, an efficient implementation of the particles’
maps is proposed in [11] based on the Distributed Particle Mapping (DPM) from
[12]. Each particle’s trajectory is stored as a succession of the trajectory
segments of all its ancestors, for which a so-called ancestry tree is built.
Then, a common log of MBES measurements is shared across all particles in
synchrony with their trajectories, such that a particle’s map can be
reconstructed backtracing through the ancestry tree and transforming the
measurements into the particle’s trajectory. A similar distributed approach
has been adopted in this work.

Regarding the map representation, [13] employed 3D evidence grid maps, a type
of Octree structure that permitted to accomplish real-time RBPF SLAM in
underwater tunnels. Targeting open sea bathymetry instead, in [11] a 2D grid-
based map is used to represent seabed depths. GP was originally used to
regress seabed models within an RBPF framework in [4] with the approach later
extended in [6]. Both works modelled bathymetric maps online through GPR and
showed the advantages of using GP prediction to perform loop closures in
unmapped areas. However, they both suffered from their inability to model the
full survey with a single GP. In [4], the problem was sidestepped by
retraining the GP on the latest patch of collected bathymetry at determined
time intervals. This method, however, can suffer from ”memory loss” in the
kernel hyperparameters and therefore might not be able to handle big loop
closures. Instead, in [6] the survey was divided into patches or tiles of
predetermined size. Although effective, this divide-and-conquer approach
involved discretizing the space similarly to grid-based mapping, hindering the
continuous representation that GP is capable of. Furthermore, it implied
equally fixed training times per GP, regardless of the bathymetry in the tile,
data redundancies in the overlapping areas across tiles and the need to keep
track of which GP to query when computing expected measurements.

[4, 6] remained therefore limited by the cubic complexity of the GP training
on the number of training inputs $N$ [7]. Other application-based solutions to
adapt GP to large-scale mapping tasks [14] as well as sparse approximations to
the GP kernel [15] have been presented in the literature. However, in this
work we propose adopting the SVGP formulation from [8] to the online terrain
reconstruction problem in order to handle large datasets efficiently.

##  III SVGP BATHYMETRIC MAPPING

This section introduces large-scale bathymetric modelling with SVGP in an
offline context.

###  III-A Bathymetric surveying with a MBES

Autonomous bathymetric surveying tasks itself with creating a map of an
unknown area of seabed $E$ gathering MBES data with an AUV. The DR of the AUV
can be modelled as a Gaussian distribution
$r_{t}\sim\mathcal{N}(g(r_{t-1},c_{t}),W)$, where $r_{t}$ represents the AUV
pose estimate at time $t$. $c_{t}$ and $g()$ are the vehicle’s control input
and its motion model respectively and $W$ is the covariance of the zero-mean,
additive Gaussian noise that parameterizes the uncertainty in the DR.
Similarly, the MBES pings can be modelled as
$P_{t}\sim\mathcal{N}(h(r_{t},E_{t}),Q)$, where $P_{t}=\\{p_{i}\\}_{i=1}^{n}$
is a ping containing $n$ 3D beams $p_{i}=[x,y,z]$, $h()$ is the MBES
measurement model and $Q$ parameterizes the measurements uncertainty.
$E_{t}=\\{e_{i}\\}_{i=1}^{n}$ describes the patches of seabed, in the map
frame, where the beams hit the ground at time $t$.

After a bathymetric survey, a dataset of the form
$D=\\{p^{xy}_{i},p^{z}_{i}\\}_{i=1}^{N}$ can be constructed, containing the
$N$ beams collected in time $T$. In the reminder of this paper, we will
alternatively use the notation $D=\\{X,Y\\}$ and the former when necessary to
adapt to the standard notation in the GP literature.

###  III-B Stochastic Variational Gaussian Process maps

If the seabed is assumed to be an unknown continuous function of the form
$f:X\to Y$ from which the MBES beams are noisy samples, SVGP can be used to
regress the original $f$ from the dataset $D$. Applying Bayes rule, such
regression problem takes the following form:

$p(f(X)|Y)=\frac{p(Y|f(X))p(f(X))}{p(Y)}$ (1)

where we seek to determine the posterior distribution that best fits the
observations collected. The prior in Eq. 1 can be modelled through a GP as
$p(f(X))\sim\mathcal{N}(0,\mathcal{K}_{NN})$, parameterized be a zero mean and
a kernel matrix $\mathcal{K}_{NN}$. For a selected GP kernel, the optimal set
of its hyperparameters $\theta_{kernel}$ can be learned maximizing the
likelihood of collecting the N observations in $D$ for the given model.
However, such maximization involves the inversion of the $\mathcal{K}_{NN}$
matrix, which has historically prevented GP from being used in large mapping
applications. In order to alleviate this constraint, several sparse
formulations have been proposed [15] based on approximating the kernel via a
set of $S<<N$ latent variables, normally referred to as inducing variables
$u$. These inducing variables correspond to a set of input locations $u=f(Z)$.
In our problem $Z$ belongs in the same domain as $X$. Furthermore, in order to
handle intractable posteriors in Eq. 1, a Variational inference (VI) approach
was proposed in [16] to approximate them instead through a user-defined
variational term $q()$. Minimizing the KL divergence between the target
posterior and the selected $q()$, $KL[q(f(X),u)||p(f(X),u|Y)]$, is equivalent
to maximizing its VI evidence lower bound (ELBO), which yields the optimal set
of hyperparameters $\theta_{kernel}$. However, optimizing such ELBO remains
subject to manipulating the $N$ training points, limiting its applicability.
To overcome this, the SVGP formulation [8] decouples the process complexity
from the number of training points by parameterizing the variational term as
$q(f(X),u)=p(f(X)|u)q(u)$, with $q(u)\sim\mathcal{N}(\mu,\Sigma)$. This
results in the ELBO in Eq.2, which can be factorized into $N$ independent
terms (we represent $f(X)$ as $f$ in the following for the sake of space):

$\mathcal{L}=\sum^{N}_{i}\mathbb{E}_{q(u)p(f|u)}\left[\ln p(y_{i}\mid
f_{i})\right]-\mathrm{KL}\left[q(u)\mid\mid p(u)\right]$ (2)

Thanks to this factorization, stochastic gradient descent (SGD) techniques can
be applied to optimize the ELBO in Eq. 2 through minibatch estimates of size
$M<<N$. Such minibatch estimator has the form:

$\mathcal{\hat{L}}=\frac{N}{M}\sum^{M}_{i}\mathbb{E}_{q(u)p(f|u)}\left[\ln
p(y_{i}\mid f_{i})\right]-\mathrm{KL}\left[q(u)\mid\mid p(u)\right]$ (3)

The ELBO in Eq. 3 can be derived with respect to the locations of the $S$
inducing points $Z_{s}$ and therefore these can be added as hyperparameters in
the training process. Therefore the final set becomes
$\theta_{SVGP}=\\{\theta_{kernel},Z_{s}\\}$. Thus, the SVGP formulation has
permitted learning offline map models from datasets containing millions of
points, as in [10, 17].

##  IV THE RBPF-SVGP ALGORITHM

An RBPF algorithm approximates the SLAM posterior by maintaining an estimate
of the vehicle pose through a finite set $R_{t}$ of $J$ weighted particles,
each containing its own map estimate $SVGP^{j}_{t}$:

$R_{t}=\\{\langle r^{j}_{t},SVGP^{j}_{t}\rangle\\}_{j=1}^{J}$ (4)

In order to be able model the particles maps with SVGP, in this section we
present our approach to adapt SVGP to online bathymetry learning and its
efficient integration in an RBPF framework to achieve real-time management of
up to $100$ particles in an embedded platform.

###  IV-A Online mapping with SVGP

In the previous section, the focus has been on offline SVGP mapping. In order
to adapt it to online surveying, two minor modifications need to be adopted
involving the construction of the dataset $D$ and the initialization of the
inducing locations $Z_{s}$.

Regarding the dataset, so far we have assumed $D$ to have a fixed size $N$ and
contain all the data collected from a survey of duration $T$. However, in
order to implement online SVGP training, the dataset must be increased with
every new sample collected during the survey. Thus, we reformulate our dataset
as $D_{t}$, to denote its increasing size over time. We will show in our
experiments how the SGD minibatch estimators can handle a growing dataset
provided a sufficiently high rate of SVGP training iterations per new MBES
added to $D_{t}$.

Concerning $Z_{s}$, their initial positions must now be selected carefully to
account for the evolution of $D_{t}$. Although the learning of $Z_{s}$ will
eventually result in their optimal positions being found across the dataset,
this process is generally slower than the rate at which new data is collected
with a moving vehicle. Therefore, the results will be suboptimal if all the
$Z_{s}$ locations are naively initialized over the data in $D_{t}$ when the
survey starts. We overcome this by uniformly distributing $Z_{s}$ a priori
over the area to survey, whose boundaries are known at the time of planning
the mission.

###  IV-B SVGP minibatch construction

In an RBPF framework with SVGP maps, constructing minibatches for Eq. 3
entails randomly sampling from both the MBES log $D_{t}$ and a particle’s
history of trajectories $h_{t}^{j}=\\{r_{t}^{j}\\}_{t=0}^{t}$, with both of
them growing over time.

1 _svgp_iteration(_$D_{t},h^{j}_{t},\theta^{j}_{SVGP,s-1}$_)_:

2 $D_{M}=\emptyset$

3 for  _$m=0$ to $M$_ do

4 $\langle p_{b},t_{b}\rangle\sim\mathcal{U}(D_{t})$

5 $r^{j}_{tb}\leftarrow h^{j}_{t}(t_{b})$

6 $p^{j}_{b}=T(r^{j}_{tb})\>p_{b}$

7 $add$ $p^{j}_{b}$ $to$ $D_{M}$

8

9 $\theta^{j}_{SVGP,s}\leftarrow OptimizerStep(D_{M},\theta^{j}_{SVGP,s-1})$

10 return _$\theta^{j}_{SVGP,s}$_

Algorithm 1 $SVGP_{j}$ training step

Algorithm 1 describes the necessary steps involved in an SVGP training
iteration for particle $j$. Lines $3-7$ construct a minibatch $D_{M}$ and line
$8$ is an optimization step of Eq. 3. Line $4$ randomly samples a beam $p_{b}$
collected at time $t_{b}<=t$ from the MBES log. Line $5$ finds the
corresponding particle’s pose at $t_{b}$ and lines $6-7$ transforms $p_{b}$ to
that pose and adds it to the minibatch. Instructions $4-7$ cannot be
parallelized since they involve accessing the common MBES log, which is unique
due to memory constraints (note that line $8$ is indeed parallel, see Section
IV-D). Therefore they need to be very efficient in order to maximize SVGP
training iterations across particles. Backtracing through an ancestry tree in
line $5$ as in [4, 6] is therefore unsuitable here. Thus we present a new
implementation of the management of particles’ trajectories histories that
sidesteps the need to explicitly keep track on a tree of the particles’
lineages upon resampling. Our C++ approach represents histories $h_{t}^{j}$ as
vectors of managed pointers to the vectors containing the segments of
particles’ trajectories in between resamples. Our approach is illustrated in
Fig. 2.

Figure 2: Schematic of the proposed storing of particles trajectories as
vectors of pointers to segments of trajectories in between resamples.

The left image in Fig. 2 depicts an example with three particles’ trajectories
in 2D and three resampling steps, named $LC_{i}$. In red, the trajectory of
the winning particle after each resampling step. On the right, their three
corresponding trajectory histories $h_{t}^{j}$ as vectors of pointers evolving
over time. During the time before the first loop closure, $LC_{0}$, each
particle stores its own trajectory as a vector of poses managed by a pointer.
Once $LC_{0}$ is triggered and particle 1 resampled, the pointers in particle
0 and 2 delete their content and point at the trajectory segment in particle
1. After, a new pointer is added to each history vector and the process
continues so that after $LC_{2}$, each particles’ histories either contains or
points at the three corresponding segments of trajectories between LCs. Thus,
iterating through a particle’s history is trivial provided a common, auxiliary
vector containing the length of the trajectory segments (as integers). Given a
random beam at time $t_{b}<=t$, the vector will signal at the corresponding
pointer containing the particle’s pose at $t_{b}$, $r^{j}_{tb}$.

Additionally, as a result of this implementation computing the final RBPF
trajectory can be done directly just by averaging all the particles histories.
The need to explicitly keep track of their lineages has been sidestepped.

###  IV-C LC prompting and particle resampling

Given the set of $J$ particles $R_{t}$ in Eq. 4, the resampling procedure on
the RBPF follows the steps in algorithm 2.

1 _lc_prompting(_$R_{t},P_{t},Q$_)_:

2 $\hat{R_{t}}=\emptyset$

3 for  _$\langle r^{j}_{t},SVGP^{j}_{t}\rangle$ in $R_{t}$_ do

4 $\hat{P}_{t}^{j}=T(r_{t}^{j})\>P_{t}$

5
$w_{t}^{j}=\mathcal{N}(P^{z}_{t};SVGP^{j}_{\mu}(\hat{P}_{t}^{j,xy}),SVGP^{j}_{\Sigma}(\hat{P}_{t}^{j,xy})+Q)$

6 $add$ $\langle r^{j}_{t},w^{j}_{t},SVGP^{j}_{t}\rangle$ $to$ $\hat{R}_{t}$

7

8 $R_{t}=\emptyset$

9 for  _$j$ in $J$_ do

10 $draw$ $j$ $from$ $\hat{R}_{t}$ $with$ $probability$ $\propto w_{t}^{j}$

11 $add$ $\langle r^{j}_{t},SVGP^{j}_{t}\rangle$ $to$ $R_{t}$

12 return _$R_{t}$_

Algorithm 2 LC prompting

For each particle $j$, a new weight $w_{t}^{j}$ is computed in lines $4-6$
comparing the depths of the latest ping $P_{t}$ against the predictions of
that particle’s SVGP map. The weight will be proportional to the likelihood of
the difference between the real and the expected ping depths being zero. The
expected depths can be computed for a given set of 2D query locations
$X_{*}=\hat{P}_{t}^{j,xy}$ via the posterior of each SVGP from Eq. 1, which is
given by

$p(f(X_{*})|Y)\sim\mathcal{N}(K_{*s}K^{-1}_{ss}\mu,\\\
(K_{*s}K^{-1}_{ss})\Sigma(K_{*s}K^{-1}_{ss})^{T}+K_{**}-K_{*s}K^{-1}_{ss}K^{T}_{*s}))$
(5)

If the effective number of particles [18] is greater than half the number of
particles, a systematic resampling [19] takes place in lines $8-10$. The
resampling step in line $10$ might require copying particles. In our
framework, this process entails two operations: i) copying $h_{t}^{j}$, which
is extremely fast as it is a vector of pointers and ii) copying the SVGP map.
The later involves not only the SVGP hyperparameters $\theta_{SVGP}$, but also
the state of the particle’s optimizer, so that the training can continue in
the new particle.

Finally, to save computational power and avoid GPU congestion, the LC
prompting process in algorithm 2 occurs with a frequency $f_{LC}$, which is
user defined but constrained by the number of particles and the hardware
capacity. However, LC prompts are enabled only after every SVGP ELBO has
converged, according to the criterion defined in Section V-B.

###  IV-D The RBPF parallel structure

As opposed to a custom-made implementation of the SVGP regression for our
RBPF, our solution builds upon GPytorch [20]. The advantages are that GPytorch
is open-source, general and community-based and maintained. The trade off
rises from the so-called lazy management of the CUDA contexts carried out by
Torch. Currently, GPytorch lazily instantiates a new CUDA context (600-800 MB)
per SVGP created in a new process. This implies that a fully parallel
implementation of the SVGP maps would be quickly limited by memory
requirements. To overcome this, we have adopted a quasi-parallel
implementation of the SVGP training. In this solution, a set of $B$ parallel
nodes, each with its own CUDA context, hosts each $b$ SVGP maps, which run
optimization steps recursively within the node. Thus, the total number of
particles per nodes is $b=J/B$. An advantage of this implementation, however,
is that the maximum number of SVGP maps accessing the GPU at the same time is
now $B$, which can be limited by the user to prevent overloading it.

Figure 3: Schematic of the RBPF quasi-parallel structure, implemented in ROS

Fig. 3 depicts a simplified graph of the system, implemented in ROS [21]. Each
process from the set $O$ is a parallel ROS node, whose SVGP maps implement
recursively the key operations of SVGP training (line $8$ on algorithm 1) and
posterior sampling (line $5$ in algorithm 2) when attempting a LC. The RBPF
core node is written in C++ and both maintains the MBES log $D_{t}$ and
computes the particles propagation whenever new data from the AUV is received.
Furthermore, it stores the particles’ histories
$H_{t}=\\{h_{t}^{j}\\}_{j=0}^{J}$ and keeps them synchronized with $D_{t}$ for
an efficient minibatch construction when required by an SVGP map.
Additionally, it handles the LC prompting, the weights computation based on
the expected measurements from the SVGP maps and the particles resampling. All
these task are implemented in parallel.

##  V EXPERIMENTS

This section presents the evaluation methods designed and the datasets used to
test the performance of the RBPF-SVGP framework proposed.

###  V-A Datasets and vehicles

Three bathymetric surveys off the Swedish coast have been collected with two
AUVs for testing the RBPF. A Kongsberg Hugin 3000 equipped with a MBES
Kongsberg 2040 has collected mission $1$ without any external navigation aid.
Surveys $2$ and $3$ were executed by the AUV Lolo [22], with a R2Sonic MBES.
While Hugin is a commercially available vehicle with high-end specifications,
Lolo is a research platform under development at KTH university and as such it
has been a suitable vehicle to test our framework live in mission $2$ in its
payload computer, an NVIDIA Jetson Orin.

Figure 4: The AUVs Hugin (left) and Lolo (right) being deployed for the
surveys in the experiments.

The Jetson Orin features a 12-core ARM processor, 32GB of RAM and a 2048-core
NVIDIA Ampere architecture GPU with 64 tensor cores. All the experiments have
been conducted in the Orin, however only mission 2 was carried out live in
Lolo. The experiments on missions $1$ and $3$ were executed offline and
therefore the ROS environment from [10] has been used to replay the mission
from an external PC acting as the AUV. Table I summarizes the main
characteristics of three missions.

TABLE I: Parameters for the 3 experiments Experiment | 1 | 2 | 3
---|---|---|---
AUV | Hugin | Lolo | Lolo
Avg velocity (m/s) | 2.0 | 0.75 | 0.6
Duration (h) | 4.45 | 2.25 | 0.83
$N$ (beams) | 8.075.900 | 12.392.200 | 4.658.200
Minibatch size (M) | 1000 | 1000 | 1000
Inducing points (S) | 400 | 400 | 300
$f_{LC}$ | - | - | 0.1

###  V-B SVGP training

The Matern $\tau=1/2$ covariance function has been selected for the SVGP since
it has proved to work well on bathymetric data in [10]. The ELBO optimization
has been carried out with Adam [23] with a learning rate of $1e^{-1}$ across
missions. The size of the minibatch $M$ and the number of inducing points $S$
has been tuned for each mission, depending on the extension of the area to
survey. The final parameters can be seen in Table I.

A key requirement for the RBPF resampling to be successful is that the SVGP
maps resemble the true bathymetry close enough before attempting loop
closures. For this, an exponential moving average is computed over the ELBO of
each SVGP [24] at each optimization step in order to assess convergence at
time $t$. However, in our online setup, convergence at time $t$ does not
terminate the training, it only signals the RBPF core that the SVGP is ready
to be used for LC prompting. This is due to the fact that in online training
the SVGP ELBO can temporarily diverge when new data corresponding to
previously unseen seabed relief is being added to $D_{t}$. This motivates
continuously training the SVGP throughout the full mission. Fig. 6 showcases
such phenomena in a particle’s SVGP ELBO from experiment 1.

(a) (b) (c) (d)

Figure 5: Top row: experiments 1 with configuration $J=100$, $B=10$. Columns:
a) raw bathymetry and DR trajectory. b) SVGP posterior mean and c) variance
and inducing points locations of one particle randomly selected at the end of
the experiment. d) Consistency error map from comparing a) and b). Bottom row:
experiments 2 live in Lolo. Units are meters.

###  V-C Evaluation of the RBPF-SVGP framework

We now seek to assess the amenability of the proposed framework to real-time
SLAM in large seabed areas in an embedded platform. For this, we have designed
three sets of experiments carried out in the Jetson Orin. In experiment 1, we
have gauged the effects of the parallelization of the SVGP training in the
RBPF capacity to train as many SVGP maps as possible in real time during a
very large survey with Hugin. We have evaluated the capabilities of the
parallel structure proposed in terms of training iterations per particle
across the mission and the resulting SVGP maps resemblance to the original
bathymetry in terms of the consistency error metric from [25]. Experiment 2
shows the results from testing the same setup in Lolo live during a survey. In
experiment 3, we have evaluated the performance of the RBPF in terms of the
estimated vehicle trajectory compared against the ground truth computed from
GPS measurements [26] on Lolo data.

##  VI RESULTS

This section presents the results from testing the RBPF-SVGP framework on the
terrain modelling and localization tasks on the experiments introduced above.

###  VI-A Experiment 1: online SVGP mapping parallelization

As mentioned in the previous section, for the RBPF to converge towards the
true vehicle pose upon LC, the SVGP maps need to resemble the true underlying
bathymetry closely enough. This requires that each SVGP achieves enough
training iterations in real time, which is the reason behind the highly
efficient and parallel implementation of the RBPF presented. In order to
measure the resemblance of the SVGP maps with respect to the raw bathymetry,
we have tested the mapping-only capability of the SVGP-RBPF with the data from
mission 1 and applied the error metric from [25] to assess it. For this type
of experiment, the particles have been configured to follow the vehicles’ DR,
with zero motion noise added and the LC prompting turned off.

TABLE II: Results from the parallel mapping-only experiments with the Hugin survey | Experiment 1
---|---
Particles (J) | 1 | 10 | 10 | 100
Nodes (B) | 1 | 1 | 10 | 10
Avg it/particle | 202.327 | 21.425 | 39.373 | 3.714
Map RMSE | 1,636 | 1,709 | 1,712 | 1,995

The first row of images in Fig. 5 shows the results from this experiment with
the configuration $J=100$, $B=10$ for one particle randomly picked at the end
of the mission. Column a) depicts the original bathymetry map collected by
Hugin with the DR trajectories in red. Columns b) and c) represent the final
particle’s SVGP posterior mean and variance. The final locations of the
inducing points are plotted over the variance map. Column d) illustrates the
consistency error map from comparing a) and b) using the error metric from
[25]. Notice that the peak errors correspond to either bathymetry areas with
sharp changes, known to be difficult to model with GPR, or to noise in the
MBES data.

To gauge the effects of the parallelization of the SVGP training, the Hugin
survey has been replayed with four particles configurations, summarized in
Table II. Namely: single particle ($J=1$, $B=1$), no parallelization ($J=10$,
$B=1$), full parallelization ($J=10$, $B=10$) and quasi-parallelization
($J=100$, $B=10$). Notice how the fully parallel configuration has achieved
roughly double the total amount of iterations in the mission than the fully
sequential. Given that a $100\%$ utilization of the GPU was achieved, we
believe $\sim 400.000$ to be the maximum number of SVGP training iterations
achievable in this hardware with the proposed framework, regardless of the
degree of parallelization. Approximately the same amount has been achieved by
the quasi-parallel implementation for a $\times 10$ larger number of
particles, indicating maximum utilization of the GPU as well.

Regarding the resulting SVGP map, the fully-parallel configuration has
achieved the same consistency RMS error than the sequential one for double the
number of iterations per particle, which signals overfitting of the maps. We
therefore adopt the $RMSE\sim 1,7m$ as the baseline and we see how a tenfold
increment of the particles only causes an increase of $17,35\%$ on the error,
which hints at a good performance of our quasi-parallel implementation.

Figure 6: Resulting -ELBO of one particle for the four configurations in
experiment 1. Zoom-in: notice the effect of the full parallelization (red)
compared to the fully sequential training (green) on the shape of the bump
marked with arrows.

Finally, concerning the influence of the filter configuration in the temporal
aspect of the online training, Fig. 6 illustrates the evolution of the ELBO of
a particle randomly sampled from each configuration in Experiment 1. The
initial bump caused by the AUV entering the flat area of the survey in Fig. 5
a) row 1 has been marked with arrows. As expected, it is smoother the more
SVGP iterations are attainable per MBES ping collected. To enable
configurations with large number of particles, the LC prompting is only
enabled once a certain degree of convergence has been achieved by all the SVGP
maps, as explained in V-B. Thus, the initial bumps can be safely sidestepped.

###  VI-B Experiment 2: live survey in Lolo

The same setup as in experiment 1 has been tested live in Lolo while surveying
the area in the second row of Fig. 5, column a). Due to technical issues in
Lolo at the time of the data collection, the mission was carried out on the
surface. For this experiment, the particles configuration has been set to
$J=80,B=8$ to achieve a better training performance without overloading the
CPUs. The CPUs on the Jetson payload needed to be shared among the mission
behaviour tree [27] and other secondary processes. Therefore, although more
particles could have been set for the experiment, it would have been at the
expense of a lower performance of the GPU. The final average number of
iterations per particle was $2965$ and the consistency RMS error $1,003$. The
output of the live mapping for a particle randomly chosen can be seen in the
second row of images in Fig. 5, columns b)-d).

###  VI-C Experiment 3: RBPF localization estimate

To assess the overall performance of the RBPF framework in a SLAM problem, the
full RBPF-SVGP algorithm is tested in survey 3 with $J=100$ and $B=10$. This
is a challenging survey collected with Lolo that presents large coverage gaps
in the bathymetry and strong noise in the DR and the MBES, as can be seen in
Fig. 7 The area surveyed is approximately $250\times 150$ m. This figure
depicts the planned trajectory in red, with a yellow marker signalling the
starting point of the mission (the first swath was neglected) and a red one
the end. As in the previous experiment, due to technical problems with Lolo
the mission was carried out on the surface. This, however, has allowed
collecting GPS measurements that have been utilized as the ground truth
trajectory. Nevertheless, the AUV’s DR used in the experiment has been
estimated only from the velocities provided by the navigation system.

Figure 7: RViz visualization of the raw bathymetry and the planned trajectory
(red) on mission 3. The yellow marker signals the starting point (the first
swath was discarded) and the red one, the end of the mission. Figure 8: DR
(red) and RBPF (blue) trajectory errors against GT during experiment 3. Notice
the two loop closures at $t_{step}\sim 21000$ and $t_{step}\sim 28000$.

Fig. 8 depicts the evolution over time of the DR (red) position error, in
meters, against the GT position (from the GPS). In blue, the RBPF trajectory
estimate during the mission, computed online from the average of the
particles. The RBPF has registered two loop closures: a correct one at
$t_{step}\sim 21000$ and an erroneous one at $t_{step}\sim 28000$. The high
level of spurious measurements in the MBES data has led us to manually set the
uncertainty of the MBES noise model ($Q$) to a high level in order to make the
filter more robust against outliers, however resulting in only two loop
closures detected.

##  VII CONCLUSIONS AND FUTURE WORK

We have presented a Rao-Blackwellized particle filter with Stochastic
Variational Gaussian Process maps for online, large-scale SLAM in an embedded
platform. The key contributions proposed that have enabled our approach are:
i) the modifications of the SVGP training process, which have permitted to
adapt SVGP to online regression on surveys with a mobile platform, ii) a new
configuration of the RBPF particles’ lineage tracking, which has allowed
parallel, real-time SVGP training in a platform with limited resources.

The proposed framework has been implemented and tested in a Jetson Orin and
its performance has been assessed on the tasks of parallel, real-time mapping
and localization in three challenging surveys collected with two AUVs, one of
them live. When compared to the current state of the art solution for
underwater RBPF SLAM from [6], our method is capable of handling larger
surveys, both in terms of area covered and mission time, with a single SVGP
per particle map and with larger sets of particles. Regarding its
implementation, the SVGP regression module of our framework builds upon the
open-source GPytorch library and it does not require any modification of the
original SVGP formulation, facilitating its use. Additionally, the full
framework has been released to encourage further research on the topic
111https://github.com/ignaciotb/UWExploration/.

The current major limitation of the implementation concerns the lazy
initialization of the CUDA contexts per SVGP model carried out by Pytorch.
This limits the number of fully parallel SVGP maps by the amount of available
RAM of the host device. Although a quasi-parallel configuration has been
proposed, overcoming this constraint would entail a full SVGP implementation
in native CUDA. Nevertheless, when enough RAM is available, a fully-parallel
setup can be achieved setting $B=J$. Additionally, we propose as further
research the use of the particles’ ELBO value within the particles weighting
process. It has been shown that the ELBO relates to the online state of the
SVGP map and therefore factoring it in the particle weighting could
potentially result in more accurate resampling.

Finally, although this work has targeted bathymetric SLAM, the proposed
approached is not limited to it and it can be applied to different problems
and domains.

## ACKNOWLEDGMENT

The authors thank the Alice Wallenberg foundation for funding MUST, Mobile
Underwater System Tools, project that provided the Hugin AUV.

## References

  * [1] Alastair GC Graham, Anna Wåhlin, Kelly A Hogan, Frank O Nitsche, Karen J Heywood, Rebecca L Totten, James A Smith, Claus-Dieter Hillenbrand, Lauren M Simkins, John B Anderson, et al.  Rapid retreat of thwaites glacier in the pre-satellite era.  Nature Geoscience, pages 1–8, 2022.
  * [2] Ivan Stenius, John Folkesson, Sriharsha Bhat, Christopher Iliffe Sprague, Li Ling, Özer Özkahraman, Nils Bore, Zheng Cong, Josefine Severholt, Carl Ljung, et al.  A system for autonomous seaweed farm inspection with an underwater robot.  Sensors, 22(13):5064, 2022.
  * [3] Ignacio Torroba, Nils Bore, and John Folkesson.  Towards autonomous industrial-scale bathymetric surveying.  In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2019.
  * [4] Stephen Barkby, Stefan B Williams, Oscar Pizarro, and Michael V Jakuba.  Bathymetric slam with no map overlap using gaussian processes.  In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1242–1248. IEEE, 2011.
  * [5] Jiarui Tan, Ignacio Torroba, Yiping Xie, and John Folkesson.  Data-driven loop closure detection in bathymetric point clouds for underwater slam.  arXiv preprint arXiv:2209.08578, 2022.
  * [6] Kristopher E Krasnosky and Christopher Roman.  A massively parallel implementation of gaussian process regression for real time bathymetric modeling and simultaneous localization and mapping.  2022\.
  * [7] Carl Edward Rasmussen.  Gaussian processes in machine learning.  In Summer school on machine learning, pages 63–71. Springer, 2003\.
  * [8] James Hensman, Nicolo Fusi, and Neil D Lawrence.  Gaussian processes for big data.  arXiv preprint arXiv:1309.6835, 2013.
  * [9] Michael Montemerlo, Sebastian Thrun, Daphne Koller, Ben Wegbreit, et al.  Fastslam: A factored solution to the simultaneous localization and mapping problem.  Aaai/iaai, 593598, 2002.
  * [10] Ignacio Torroba, Christopher Iliffe Sprague, and John Folkesson.  Fully-probabilistic terrain modelling and localization with stochastic variational gaussian process maps.  IEEE Robotics and Automation Letters, 7(4):8729–8736, 2022.
  * [11] Stephen Barkby, Stefan B Williams, Oscar Pizarro, and Michael V Jakuba.  A featureless approach to efficient bathymetric slam using distributed particle mapping.  Journal of Field Robotics, 28(1):19–39, 2011.
  * [12] Austin I Eliazar and Ronald Parr.  Hierarchical linear/constant time slam using particle filters for dense maps.  Advances in neural information processing systems, 18, 2005.
  * [13] Nathaniel Fairfield, George Kantor, and David Wettergreen.  Real-time slam with octree evidence grids for exploration in underwater tunnels.  Journal of Field Robotics, 24(1-2):03–21, 2007.
  * [14] Shrihari Vasudevan, Fabio Ramos, Eric Nettleton, and Hugh Durrant-Whyte.  Gaussian process modeling of large-scale terrain.  Journal of Field Robotics, 26(10):812–840, 2009.
  * [15] Joaquin Quinonero-Candela and Carl Edward Rasmussen.  A unifying view of sparse approximate gaussian process regression.  The Journal of Machine Learning Research, 6:1939–1959, 2005.
  * [16] Michalis Titsias.  Variational learning of inducing variables in sparse gaussian processes.  In Artificial intelligence and statistics, pages 567–574. PMLR, 2009.
  * [17] Andrew C Miller, Lauren Anderson, Boris Leistedt, John P Cunningham, David W Hogg, and David M Blei.  Mapping interstellar dust with gaussian processes.  arXiv preprint arXiv:2202.06797, 2022.
  * [18] Jun S Liu.  Metropolized independent sampling with comparisons to rejection sampling and importance sampling.  Statistics and computing, 6(2):113–119, 1996.
  * [19] Genshiro Kitagawa.  Monte carlo filter and smoother for non-gaussian nonlinear state space models.  Journal of computational and graphical statistics, 5(1):1–25, 1996\.
  * [20] Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson.  Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration.  arXiv preprint arXiv:1809.11165, 2018.
  * [21] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, Andrew Y Ng, et al.  Ros: an open-source robot operating system.  In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
  * [22] Clemens Deutsch, Lázaro Moratelli, Sebastian Thuné, Jakob Kuttenkeuler, and Filip Söderling.  Design of an auv research platform for demonstration of novel technologies.  In 2018 IEEE/OES Autonomous Underwater Vehicle Workshop (AUV), pages 1–8. IEEE, 2018.
  * [23] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [24] Maximilian Balandat, Brian Karrer, Daniel R Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy.  Botorch: Programmable bayesian optimization in pytorch.  arxiv e-prints, pages arXiv–1910, 2019.
  * [25] Chris Roman and Hanumant Singh.  Consistency based error evaluation for deep sea bathymetric mapping with robotic vehicles.  In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., pages 3568–3574. Ieee, 2006.
  * [26] Edwin Olson and Michael Kaess.  Evaluating the performance of map optimization algorithms.  In RSS Workshop on Good Experimental Methodology in Robotics, volume 15, 2009.
  * [27] Christopher Iliffe Sprague, Özer Özkahraman, Andrea Munafo, Rachel Marlow, Alexander Phillips, and Petter Ögren.  Improving the modularity of auv control systems using behaviour trees.  In 2018 IEEE/OES Autonomous Underwater Vehicle Workshop (AUV), pages 1–6. IEEE, 2018.


# ACCORDANT: A Domain Specific Model and DevOps Approach for Big Data
Analytics Architectures111This research was carried out by the Center of
Excellence and Appropriation in Big Data and Data Analytics (CAOBA), supported
by the Ministry of Information Technologies and Telecommunications of the
Republic of Colombia (MinTIC) through the Colombian Administrative Department
of Science, Technology and Innovation (COLCIENCIAS) within contract No.
FP44842-anexo46-2015. We would like to make an special acknowledgement to the
Taxation Authority of Bogota at the Bogota’s Department of Finance (SHD), to
the office of telecommunications of the National Planning Department (DNP),
and to the CAOBA-Uniandes team, both faculty and students.

Camilo Castellanos<EMAIL_ADDRESS>Carlos A. Varela
<EMAIL_ADDRESS>Dario Correal<EMAIL_ADDRESS>System Engineering
and Computing Department, University of Los Andes, Bogota, Colombia
Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA

###### Abstract

Big data analytics (BDA) applications use machine learning algorithms to
extract valuable insights from large, fast, and heterogeneous data sources.
New software engineering challenges for BDA applications include ensuring
performance levels of data-driven algorithms even in the presence of large
data volume, velocity, and variety (3Vs). BDA software complexity frequently
leads to delayed deployments, longer development cycles and challenging
performance assessment. This paper proposes a Domain-Specific Model (DSM), and
DevOps practices to design, deploy, and monitor performance metrics in BDA
applications. Our proposal includes a design process, and a framework to
define architectural inputs, software components, and deployment strategies
through integrated high-level abstractions to enable QS monitoring. We
evaluate our approach with four use cases from different domains to
demonstrate a high level of generalization. Our results show a shorter
deployment and monitoring times, and a higher gain factor per iteration
compared to similar approaches.

###### keywords:

Software architecture , Big data analytics deployment , DevOps , domain
specific model , quality scenarios , performance monitoring

††journal: Journal of Systems and Software

##  1 Introduction

Big data analytics (BDA) applications use Machine Learning (ML) algorithms to
extract valuable insights from large, fast and heterogeneous data. These BDA
applications require complex software design, development, and deployment to
deal with big data characteristics: volume, variety, and velocity (3Vs), to
maintain expected performance levels. Specifically, BDA processing takes
advantage of cutting-edge technologies and infrastructures that enable
distributed stream computing. But the complexity involved in BDA application
development frequently leads to delayed deployments [1] and hinders
performance monitoring (e.g. throughput or latency) [2]. Regarding the 3Vs, a
BDA solution can be constrained to different performance metrics. For
instance, real-time stream analytics applications require low latency and
flexible scalability based on data volume fluctuation. On the other hand,
heavy workloads, which imply batch processing over big data, demand high
scalability and fault tolerance to achieve a particular deadline. One of the
key goals of software architecture is the design of the system’s structures
and their relationships to achieve expected quality properties.

The development of BDA solutions involves three knowledge domains: business,
analytics, and technology. In the business domain, business experts have to
define business goals and quality scenarios (QS) to drive analytics projects.
In the analytics domain, these business goals are translated into specific
analytics tasks by data scientists. Finally, in the technology domain,
software architects make decisions in terms of tactics, patterns, and
deployment considerations keeping in mind quality attributes. Stakeholders
from different domains face heterogeneous concerns and different abstraction
levels. Due to the lack of techniques, and tools to enable articulation and
integration of such domains, BDA solutions development presents a high cost
and error-prone transition between development and production environments [1,
3]. Though there is a growing interest of companies in big data adoption, real
deployments are still scarce (“Deployment Gap” phenomenon) [4].

In the same vein, previous surveys [5, 6, 7] have reported low deployment
frequency and delayed deployment procedures caused by analytics model
translation, lack of tools’ interoperability and stakeholders’ communication.
These pitfalls could be the result of the traditional approach of BDA
development where the data scientist produces the models as source code
implemented using machine learning-oriented tools which are focused on
analytics perspectives within a controlled environment (data lab). On the
other hand, software architects have to translate these models into software
products which usually implies rewriting code to obtain productive software
components deployed on specific IT infrastructures.

This paper proposes ACCORDANT (An exeCutable arChitecture mOdel foR big Data
ANalyTics), a DevOps and Domain-Specific Model (DSM) approach to develop,
deploy, and monitor BDA solutions bridging the gap between analytics and IT
domains. ACCORDANT allows to design BDA applications using QS, functional, and
deployment views. A QS specifies a quality attribute requirement for a
software artifact to support design and quality assessment. Functional view
defines the architectural elements that deliver the application’s
functionality. Deployment view describes how software is assigned to hardware-
processing and communication elements. Our deployment strategy incorporates
containerization since it offers consistent modularity to facilitate
portability, continuous integration, and delivery.

ACCORDANT is validated with four use cases from different domains by designing
functional and deployment models, and assessing performance QS. This
validation aims to reduce the time of design, deployment, and QS monitoring of
BDA solutions. These use cases range from public transportation and avionics
safety to weather forecasting and they include distributed batch, micro-batch,
and stream processing. Our results indicate improvements in design and
(re)deployment times to achieve the expected performance metrics. In summary,
the contributions of this paper are as follows:

  * 1.

A DSM framework to formalize and accelerate iteratively the development and
deployment of BDA solutions by specifying architectural functional, and
deployment views aligned to QS.

  * 2.

Three integrated domain-specific languages (DSLs) to specify architectural
inputs, component-connector models, and deployments, thus accelerating BDA
deployment cycle.

  * 3.

A containerization approach to promote automation delivery and performance
metrics monitoring for BDA applications aligned to QS.

  * 4.

The evaluation of this proposal applied to four use cases from diverse
domains, and using different deployment strategies and QS.

The rest of this paper is organized as follows. In Section 2, we describe the
background on DSM, big data analytics, and DevOps. Section 3 reviews related
work. Section 4 presents our methodology and proposal overview. Section 5
presents the use cases for experimentation. Section 6 illustrates the steps
followed to validate this proposal. Section 7 presents and discusses the
obtained results. Finally, Section 8 summarizes the conclusions and future
work.

##  2 Background

This section describes the core concepts in which this proposal is supported:
domain-specific modeling, software architecture, big data analytics, and
DevOps.

###  2.1 Domain-Specific Modeling (DSM) and Software Architecture

Domain-Specific Modeling enables the software to be modular and resilient to
changes through the separation of concerns (SoC) principle by specifying
technology-agnostic concepts, relationships, and constraints within the
domain. An important advantage of DSM is the close mapping problem and
solution domains to provide code generation. Moreover, DSM can speed up and
optimize the code generated for the specific platform improving productivity.
In order to enable code generation, the domain model requires to be narrow,
and it is constrained by a language specification, the metamodel. Furthermore,
due to the narrow metamodel’s scope, the models can be read, checked,
validated, and interpreted to generate specific implementations. Regarding
representations, DSM can be expressed in graphical, textual, or mixed notation
according to the domain context. It is possible to embed multiple views or
aspects (for example, analytics, software components, and deployment) using
different representations that share elements or mappings.

An architecture description language enables architects to express high-level
system structure by describing its coarse-grained components and connections
among them. These descriptions are contained in architectural views to address
different concerns, and these views are built based on collection of patterns,
templates, and conventions called Viewpoints [8]. The architectural design is
driven by quality scenarios and primary functional requirements through a
systematic design method, such as the Attribute-Driven Design method (ADD
[9]). ADD starts identifying inputs: QS, functional requirements, and
constraints. In each ADD iteration, a design goal is defined from these
inputs, and the selection of architectural structures, tactics, patterns, and
their application described across views, aims at achieving such goal. A
pattern is a standard, known and reusable solution to a common problem in
software architecture. Tactics are design primitives to achieve a response for
particular quality attributes. Previous studies have collected both patterns
[10, 11] and tactics [12, 13] to be applied in the BDA domain.

###  2.2 Big Data Analytics

In BDA context, data processing models aim at specific application
requirements: batch to process large stored datasets all at once with high
performance, and stream processing for an unbounded data flow in (near) real-
time. Due to the complexity of deploying and operating BDA solutions
integrating a myriad of technologies, complex analytics models and distributed
infrastructure, some research has been done to tackle such complexity by
raising the level of abstraction [14, 15, 16].

Due to the wide range of BDA technologies, portability plays a key role to
deploy, operate, and evolve BDA applications, and this is where portable
standards appear such as Predictive Model Markup Language
(PMML)222http://dmg.org/pmml/v4-3/GeneralStructure.html or Portable Format for
Analytics (PFA)333http://dmg.org/pfa/. PMML is the de facto standard proposed
by the Data Mining Group that enables portability of analytics models through
neutral-technology XML format. PMML allows specifying a set of machine
learning models and data transformations along with their metadata.

###  2.3 DevOps and IaC

According to Bass. et. al [17], DevOps is a set of practices aims to reduce
the time from software development to production environment, ensuring high
quality. DevOps includes activities as deploy, operate and monitor
applications, with the goals of improve deployment frequency and speed up the
time to market what is aligned to our proposal’s objectives. Infrastructure as
Code (IaC) arises from the necessity to handle the infrastructure setup,
evolution, and monitoring in an automated and replicable way through
executable specifications. IaC promotes the reduction of cost, time and risk
of IT infrastructure provision by offering languages and tools which allow to
specify concrete environments (bare-metal servers, virtual machines, operative
systems, middleware and configuration resources) and allocate them
automatically. In this context, technologies such as
Kubernetes444https://kubernetes.io/, an open source for automating deployment,
scaling, and management of container clusters which offers to decouple
application containers from the infrastructure details.

##  3 Related Work

Several works have proposed frameworks to build and deploy BDA applications.
We review and compare some of the most relevant works, that comprise building
blocks to construct and deploy BDA pipelines. Indeed, some works have tackled
DSM to describe functional and deployment viewpoints involving DevOps
practices. We summarize and compare the related work reviewed in Table 1,
addressing the identified problem and our vision of using separation of
concerns (SoC), domain-specific modeling and DevOps to deal with the
deployment gap.

Table 1 details in each column some features we identify in the related work
as follows. SoC is a key design principle for us, since the knowledge domains
involved in BDA (business, analytics, and IT) have to be tackled from
different perspectives (i.e. viewpoints). In terms of analytics domain, cross-
industry (CI), and technology-neutral models (TNM) promote applicability, and
BDA portability respectively. Regarding software architecture concepts, QS
specification (QSS), functional (FV) and deployment (DV) views allow us to
describe orthogonal concerns such as quality scenarios, components-and-
connector, and deployment models. Architectural tactics (AT) are design
decisions that influence the control of a QS response. A target-technology
assignment (TTA) complements DSM approaches by supporting a predefined
technologies set (P) or extensible code generators (C). Finally, considering
the DevOps practices, deployment specification column (DS) defines if only a
number of instances (I) per component or a whole deployment diagram (D) can be
described. Additional practices that facilitate the deployment and operation
processes are considered: continuous deployment (CD), QS monitoring (QSM), and
self-adaptation (SA).

Table 1: Related Work

Work SoC Business (Analytics) Software Architecture DevOps CI TNM QSS FV DV AT
TTA DS CD QSM SA Lechevalier et al. [18] Gribaudo et al. [14], Huang et al.
[16] D CloverDX [19] I OptiML [20] C Qualimaster [21] FastScore [22] C I
SpringXD [23] P I DICE [15][24][25] C D ACCORDANT C D

Some works have presented DSM to model analytics functions, however, they do
not tackle architecture concepts and deployment considerations because they
are only focused on functional definitions. Lechevalier et al. [18] introduce
a DSM framework for predictive analytics of manufacturing data using
artificial neural networks to generate analytics models. Sujeeth et al.
present in [20] OptiML, a DSL for machine learning which describes analytics
functions using a statistical model which cover a subset of ML algorithms,
this analytics functions are analyzed and optimized before the code
generation. CloverDX [19] is a commercial tool to design data transformations
and analytics workflows in a visual way integrating external APIs, and
including parallel processing in multiple nodes. CloverDX’s functional view
includes readers, processors, and writers for a predefined set of
technologies, but deployment view is not available and distributed processing
must be defined with specific parallel nodes in the functional view, which
prevents to use the same functional definition in different deployment
strategies. Finally, technology-neutral models, performance scenario
specifications, and architectural tactics are not supported.

In contrast, we found another group of studies interested in infrastructure
concerns of BDA applications leaving aside their functional components.
Gribaudo et al. [14] propose a modeling framework based on graph-based
language to evaluate the system’s performance of running applications which
follow the lambda architecture pattern. This modeling framework allows users
to define stream, batch, storage, and computation nodes along with performance
indices to be simulated and evaluated, but neither functional BDA application
nor real infrastructure provision are provided as a result. Huang et al. [16]
introduce a model to design, deploy, and configure Hadoop clusters through
architecture metamodel and rules, which describe BDA infrastructure and deploy
automation. Their work is focused on design, deployment, and evaluation of BDA
technology infrastructures. However, it leaves out functional analytics models
to get an integrated BDA solution.

QualiMaster [21] focuses on the processing of online data streams for real-
time applications such as the risk analysis of financial markets regarding
metrics of time behavior and resource utilization. The aim of QualiMaster is
to maximize the throughput of a given processing pipeline. Similarly, our
proposal generates software for BDA applications, but taking as input the
analytics specification of a predictive model, and the performance metrics to
be achieved. Unlike Qualimaster, our proposal is technology-neutral and cross-
industry which enables a more widespread application.

Fastscore [22] is a commercial framework to design and deploy analytics
models. Analytics components are conventionally developed using a determined
programming language or using a PMML file, and once imported to the platform,
they can be connected to data inputs and outputs. Quality scenarios cannot be
specified, but performance metrics can be visualized. Deployment is realized
through engines (containers) where models are executed, and the deployment
design is limited to engine replication factor to increase the concurrency of
analytics models.

SpringXD [23] is a unified, distributed, and extensible system for data
ingestion, analytics, processing, and export to simplify BDA development and
deployment. In SpringXD, modules are data processing units of one of three
types: source, processor, or sink, and they can be connected using messaging
abstractions called message bus to build BDA pipelines. Modules run over a
cluster of containers which can be replicated to a fixed number and monitored
to observe performance behavior, although these metrics are not application-
oriented, but infrastructure-oriented (e.g. CPU and memory use). Similar to
our approach, analytics processor can be defined through PMML models, but
target technologies are limited to a set of predefined options.

DICE project in [15][24] presents a DSM offering big data design which
comprises data, computation, technology-frameworks, and deployment concepts to
design and deploy data- intensive applications. DICE proposes a model-driven
engineering approach to develop application models which are automatically
transformed into IaC. In addition, DICE includes quality of service
requirements associated to elements within the application, which are
analogous to QS. Perez et al. in [25] presented a profile to enable
performance and reliability assessment. DICE supports configuration
management, service provisioning, and application deployment, but technology-
neutral models and architectural tactics are not considered which could hinder
portability and design decision tracing. Due to its focus, DICE requires
design at very detailed level, specifying different constructs regarding
target technologies, but in our proposal, the technology-specific generators
transform functional and deployment artifacts to code.

To summarize, the related work approaches reviewed tackle the BDA applications
design, but they are not concern about deployment architectural decisions.
Specifically, only four proposals follow the SoC principle
([21],[22],[23],[15]), and among them, only Qualimaster and DICE [15] offer a
deployment viewpoint. From the architecture perspective, tactics and QS
specifications are scarcely ever considered. Based on these findings, we argue
that our proposal aims to bridge such gaps.

##  4 ACCORDANT: A DevOps and Domain Specific Model Approach

Figure 1: ACCORDANT Process overview

This proposal aims at offering a high-level approach to design BDA solutions
starting from architectural artifacts, instead of source code. Specifically,
we propose ACCORDANT (An exeCutable arChitecture mOdel foR big Data ANalyTics)
to deal with functional, infrastructure and QS requirements. Our proposal
comprises: a design and deployment process, and a DSM framework to support
such process. This paper extends metamodel proposed in [26] by aligning
ACCORDANT process to ADD, and including architectural inputs, containerization
and serverless deployments in DV. Fig. 1 depicts the ACCORDANT’s process,
which adapts and integrates an architecture design method (ADD) and analytics
methodologies.

The steps performed using ACCORDANT modeling framework are framed in solid
lines, while the steps made with external tools are represented by dotted
lines. ACCORDANT process is iterative and, it is composed of seven steps: the
business user defines 1.1) business goals and 1.2) QS which will guide the
next steps. 2) The data scientist develops data transformations, build and
evaluates analytics models. The resulting analytics models are exported as
PMML files. 3) Architect design the software architecture using ACCORDANT
Metamodel in terms of Functional Viewpoint(FV) and Deployment Viewpoint(DV).
FV model makes use of PMML models to specify the software behavior. 4) FV and
DV models are interweaved to obtain an integrated model. 5) Code generation of
software and infrastructure is performed from integrated models. 6) The code
generated in the previous step is executed to provision infrastructure and
install the software. 7) QS are monitored in operation to be validated, and
design adjustments can be made to achieve QS, if necessary.

###  4.1 Architectural Inputs

According to architecture design methods such as Attribute-Driven Design
(ADD)[27], architecture design is driven by predefined quality scenarios (QS)
which must be achieved through design decisions compiled in well-known
catalogs of architectural patterns and tactics. Both QS and tactics are inputs
of the architecture design, therefore we include these initial building blocks
in the ACCORDANT metamodel along with other concepts defined in ADD. Fig. 2
details the main input building blocks grouped by the architectural input
package (InputPackage) which contains the elements required to start the
architectural design: Quality Scenario (QScenario), Analyzed QS (AnalyzedQS),
SentivityPoint and Tactic. A QScenario determines a quality attribute
requirement (i.e. latency, availability, scalability, etc) for a specific
Artifact. Thus, for instance, a QScenario could be defined as “latency $<=$ 3
seconds for an artifact X”, where artifact X corresponds to a software
component or connector. A QS is analyzed through a AnalyzedQS, and sensitivity
points. A SensitivityPoint is a property of a decision (a set of elements and
their relationships within an architectural view) that is critical for
achieving the QS, and that such decision is the application of a tactic to a
specific application context. Finally,Tactic elements synthesize BDA tactics
found in [12, 13] to be applied in an architecture instance, e.g.: dynamic
resource allocation, health monitoring, parallel processing, feature
selection, etc.

Once QScenarios, AnalyzedQS, and SensitivityPoints are defined in the step 1.2
of ACCORDANT process, the software architecture is designed in step 3 and
expressed on the views instantiating tactics in a concrete application. These
decisions are associated via SensitivityPoints, and they will be evaluated
against the initial QS to validated whether the architecture is achieving its
goal.

Figure 2: Excerpt of Architecture Inputs metamodel.

###  4.2 Functional Viewpoint (FV)

FV allows us to design analytics pipelines in terms of ingestion, preparation,
analysis and exporting building blocks. FV specifies functional requirements
of the analytics solution, and the constructs are described in a technology-
neutral way as detailed in the metamodel depicted in Fig. 3. FV is expressed
in a component-connector structure. Sensitivity points, from architectural
inputs, can be associated to components and connectors to represent where
architectural decisions have impact regarding the QS. Component metaclasses
are specialized in Ingestors, Transformers, Estimators and Sinks. Estimators
and Transformers are the software component realizations of PMML data model
and data transformer respectively, and the PMML file defines their behavior. A
Component exposes required and provided Port.

Connectors metaclasses transfer data or control flow among components through
an input or output Roles. A set of connector types are defined based on the
connector’s classification proposed by Taylor et al. in [28]: Procedure Call,
Event, Stream, Adaptor, Distributor and Arbitrator. A Procedure Call connector
models the flow control and communication through invocations. Similarly, an
Event connector affects the control flow and provides data transfer, but it is
subject to the occurrence of events to notify all interested parts. A Stream
connector is used to perform transfer of large amounts of data that is
continuously generated. Adaptors enable interaction between components that
have not designed to interoperate providing conversion features. Distributor
connectors identify interaction paths and communication routing. An Arbitrator
streamlines system operation and resolves conflicts thus offering intermediary
services.

Figure 3: Excerpt of Functional Viewpoint of ACCORDANT metamodel.

###  4.3 Deployment Viewpoint (DV)

The Deployment viewpoint integrates DevOps practices including
containerization, IaC, and serverless computing. The DV specifies how software
artifacts (components and connectors) are deployed on a set of computation
nodes. The main metaclasses are detailed in Fig. 4. DV metamodel comprises
Pod, ExposedPort, and Deployment metaclasses to operationalize BDA
applications in a specific technology. It is noteworthy that a FV model can be
deployed in different DV models either to use a different strategy, or to test
the fulfillment of predefined QScenarios.

DV contains Devices, Services, Deployments, serverless environments
(ServerlessEnv), and Artifacts. Sensitivity points can be assigned to
Deployments and Artifacts to map critical architectural decisions in the DV. A
Device is a worker machine (physical or virtual) on which the Pods are
deployed. A Pod is a group of one or more execution environments
(ExecEnvironment) which can share storage and network. An ExecEnvironment
represents a container with a Docker image, and specific resources
requirements (CPU, memory). A Deployment specifies the desired state for a
Pod’s group and its deployment strategy, including the number of replicas.
Services and ExposedPorts define the policies, addresses, ports, and protocols
by which to access to Pods from outside the cluster network. A ServerlessEnv
element describes a computing environment in which a cloud provider runs the
server, and dynamically manages the allocation of machine resources, as
opposition to ExecEnvironment where physical resources have to be defined and
managed. Artifacts correspond to executable or deployable representations of
functional elements (i.e. components and connectors from functional view)
which can be deployed on either execution or serverless environments.

Figure 4: Excerpt of Deployment Viewpoint metamodel

Once PMML, FV and DV models are designed and integrated, code generation takes
place by means of model-to-text transformations. Code generation is twofold:
software and infrastructure (IaC) code. On the software side, each component
and connector is assigned to a specific technology regarding its constraints
specified in the model (processing model, ML algorithm, delivery type, sync
type, etc). Such assignment enables us to generate code for target technology
restricted to these constraints. For instance, near real-time analytics
requires stream or micro-batch processing offered by Apache Storm or Spark
respectively, and Event connectors such as Apache Kafka or RabbitMQ. Regarding
the QS monitoring, code generators include specific machinery to log metrics
at an application level. It allows us to collect specific-QS from a high-level
abstraction, saving the cost of adding code for logging metrics for each
application and target technology. On the IaC side, DV model is transformed
into Kubernetes’ configuration files (in YAML format) used to create and
configure infrastructure over Kubernetes cluster. Kubernetes files contain
Nodes, Pods, Deployments, and Services which are executed through Kubectl.

In the last step, the performance metrics of the BDA application are gathered
to be compared to initial QS and evaluate the fulfillment of quality
requirements. In this step, the architect has to check the outputs, and to
make decisions in the architectural views, if QS is not achieved. This process
can take several iterations, and this is the whole cycle that we expect to
accelerate and using ACCORDANT.

##  5 Evaluation with four BDA Use Cases

Our experimentation aims to compare development and deployment time for each
iteration using accordant and other two frameworks reviewed in Section 3:
FastScore and SpringXD. We chose these frameworks because they are the closest
to our approach, and they support portable analytics models (PMML or PFA). We
validated our proposal in different domains through four use cases: UC1)
Transport delay prediction, UC2) Near mid-air collision detection, UC3) Near
mid-air collision risk analysis, and UC4) El Nino/Southern Oscillation cycles.
Table 2 summarizes the use cases, domains, processing models and quality
attributes. These use cases are applied to analytics models, they also
illustrate BDA facets as streaming and micro-batch to deal with the velocity
aspect, and batch processing is focused on volume, in terms of data size and
computation complexity. Fig. 5 details the component-connector model for each
use case to illustrate the functional building blocks, and their composition
as BDA pipelines. The ACCORDANT specification of these use cases is publicly
available555http://github.com/kmilo-castellanos/accordant-usecases, and the
use cases description will be presented below.

Table 2: Use Cases

Use Case Description Domain Analytics Model Processing Model QS Metric UC1
Transport Delay Prediction Transportation Regression Tree Stream Update time,
Latency UC2 NMAC Risk Analysis Avionics K-means Batch Deadline UC3 NMAC
Detection Avionics Decision Tree Micro-batch Latency UC4 El Nino/Southern
Oscillation Weather Polynomial Regression Batch Deadline

###  5.1 Use Case 1 (UC1)

The first use case was presented in [26], and it deals with delay prediction
of public transportation in Vancouver. Bus trips data is collected in real-
time from Vancouver Transport Operator, and it contains bus stops, routes and
time. A regression tree model to predict bus delays (in seconds) is built,
evaluated, and exported to PMML. The pipeline, described in Fig. 5a, starts
with an ingestor component which receives HTTP request and put it into an
event connector (message broker), then the request message is consumed by the
estimator to predict the delay time, and queue it, to be stored into a No-SQL
database (hierarchical). The PMML model is deployed into productive
environment as a delay predictor service, using OpenScoring, and Kafka message
broker, and MongoDB writer as target technologies. The QS were defined in
terms of performance and modifiability attributes. The QS specifies that users
make 1000 requests to delay prediction service under operations without load,
and the responses must have an average latency lower than 2 seconds. Second QS
states that when data scientist produces a new version of the predictive model
(new PMML file), it must be updated at runtime within 10 seconds.

###  5.2 Use Case 2 (UC2)

UC2 was applied in aviation safety to detect near mid-air collisions (NMAC) on
different air space ranges with different deployment models while performance
QS are monitored. This use case is described in Fig. 5b), and it was presented
in [7]. NMAC detection comprises a pairwise comparison of flights:
$C^{2}_{n}$), where n is the number of flights. Each comparison requires to
calculate distance and time based on location, speed and heading to determine
the risk level of NMAC, which implies an intensive computation of quadratic
time complexity. Eight-hours of data were stored in a distributed file system
to be loaded by JSON reader component. This ingestor calls NMAC detector which
computes the alert level. Once an alerting level is calculated for each flight
pair, the results are sent to the clustering estimator to be associated with a
specific cluster. NMACs are stored back in the file system. To compare
different data size magnitudes, we collected flight data for three air space
ranges in nautical miles (nmi): 2 nmi, 20 nmi, 200 nmi, and 1500 nmi around
John F. Kennedy Airport. These ranges represent different application scopes
to attend various demand levels: local, metropolitan, and regional areas. The
largest dataset (1500 nmi) is 1.4 GB of JSON files. This use case did not have
real-time requirements due to its heavy workload nature, and therefore a
performance QS for deadlines lower than one hour was defined.

###  5.3 Use Case 3 (UC3)

UC3 is a real-time application to detect NMAC within an air space range, and
its architecture is described in Fig. 5c). The ingestor component consumed
data through direct REST service. Flight data was pushed in a message queue to
be consumed by the NMAC detector component which performed the potential
collision detection to be finally stored in a relational DB through a message
broker connector. It is worth mentioning that the NMAC estimator of UC2 and
UC3 are the same, since its inputs, outputs, and behavior are identical, so we
can reuse such functional component definition, in spite of its deployment can
be different regarding the QS constraints. Given the near real-time nature of
this application, latency is the critical quality attribute, and we evaluated
this metric in two ranges of air space around John F. Kennedy Airport: 2 nmi
and 200 nmi, which demand different computation resources.

Figure 5: Component diagrams of Use Cases

###  5.4 Use Case 4 (UC4)

In this last use case, we used a public available data and PMML model
(polynomial regression) of El Nino/Southern Oscillation
(ENSO)666http://dmg.org/pmml/pmml_examples to implement a batch oriented
pipeline, see Fig. 5d. The El Nino/Southern Oscillation (ENSO) cycle, was the
strongest of the century which produced many problems throughout the world
affecting South and North America countries with destructive flooding in some
areas and strong drought in other areas. Data for this use case contains
oceanographic and surface meteorological readings (geolocation, humidity,
surface winds, sea surface temperatures, and subsurface temperatures) are
taken from a series of buoys positioned throughout the equatorial Pacific.
This data is expected to help with the understanding and prediction of ENSO
cycles. We read the historic data from 1980 to 1998 (178,080 records) using a
CSV reader (ingestor) component, which sends the data to the ENSO predictor
component. ENSO predictor is a estimator component that forecasts air
temperature, and stores the prediction in a distributed file system. The QS
defined for UC4 was a deadline for batch processing lower than 30 minutes.

###  5.5 Development, Deployment Time and Gain Factor

To compare ACCORDANT, SpringXD and FastScore, we measured the time invested in
development and deployment phases for each use case. Development phase
involves design and development of the functional components and connectors in
a specific technology. Deployment phase comprises the design and provisioning
of the technology infrastructure, the installation of software artifacts
developed in the previous phase, and the monitoring of the solution regarding
the predefined QS. These phases are performed iteratively, since in each
iteration some improvements and refinements are done until the QS are
achieved. Therefore, we measure the time invested in each iteration, and also
we calculate the gain factor $GF(uc,f)$, as a metric to estimate the
cumulative average of time reduction ratio for a use case uc, using framework
f over I iterations. $GF(uc,f)$ is defined as follows:

$GF(uc,f)=\dfrac{1}{I}\sum_{i=1}^{I}\dfrac{time\\_spent(uc,f)_{i}-time\\_spent(uc,f)_{i+1}}{time\\_spent(uc,f)_{i}}$
(1)

We define the gain factor as a form to measure the incremental improving of
using a high level abstractions to modify or refine an application until
achieve an expected QS. The time for each use case, phase, and iteration was
collected from two development teams which learnt and used the three
frameworks to develop and deploy two use cases each one, while they were
recording the time spent. The development and deployment process using
ACCORDANT will be illustrated with UC4 in the next Section.

##  6 Experimentation

To design, develop and deploy the four use cases, we followed ACCORDANT
process detailed previously in Figure 1. For the sake of brevity, this section
details the step-by-step implementation of UC4 as an example, more details
about the other use cases can be found in [26, 7]. The ACCORDANT projects are
available in a public repository 777http://github.com/kmilo-
castellanos/accordant as well as use cases and results
888http://github.com/kmilo-castellanos/accordant-usecases.

###  6.1 Definition of Quality Scenarios

QS are defined regarding the use case’s quality requirements. In UC4, a
scheduled job to estimate ENSO cycles for ten years of data is processed in
batch. In this vein, Fig 6 details architectural inputs of UC4 expressed using
the ACCORDANT’s input package DSL. The predictor component is required to have
a deadline lower than 1 hour in the QS UC4_QS1. Analyzing this QS, a
sensitivity point (UC4_SP1) is identified to achieve the deadline metric by
applying two tactics: introduce concurrency and increase. available resources.
These tactics will be materialized in the software architecture design.

Figure 6: Excerpt of Input Package Models of UC4 Using ACCORDANT DSLs

###  6.2 Development of Data Transformations and Analytics Model

Analytics Model is trained and evaluated by the data scientist outside the
ACCORDANT framework, and the resulting models were exported to PMML file to be
loaded in the ACCORDANT functional model. In this case, the polynomial
regression model of ENSO is downloaded and used. Fig 7 describes the structure
of the PMML, detailing some data fields, mining fields, and regression
coefficients. This PMML file will be embedded in the functional model in the
next step.

Figure 7: Excerpt of ENSO Polynomial Regression Model of UC4 in PMML format

###  6.3 Design of Software Architecture - Functional View

FV models were designed using ACCORDANT Functional DSL to specify a component-
connector structure for each use case. Two iterations of functional model were
designed for UC4, and the last iteration is depicted in Figure 8a. Since
architectural inputs are required in this design, this package is imported
using the keyword use inputPackage. The functional model specifies three
components: (CSVReader::Ingestor, ENSOPredictor::Estimator, and
HDFSWriter::Sink), and two connectors: procedure calls CallEnso::ProcCall and
CallExport::ProcCall which connect the components through ports. The
components also include some properties such as connections and formats.
Additionally, ENSOPredictor uses batch processing model, it has associated the
PMML “ElNinoPolReg.pmml”, obtained in the previous step, to provide the
predictive behavior. The sensitivity point UC4_SP1 aligns the architectural
input (QS and tactics explained in Section 6.1) to ENSOPredictor. It means
that ENSOPredictor becomes part of the introduce concurrency tactic
realization that will be translated into a distributed processing model which
has to be supported by the target technology.

Figure 8: Excerpt of Functional (a) and Deployment (b) Models of UC4 Using
ACCORDANT DSLs

###  6.4 Design of Software Architecture - Deployment View

The deployment view models were designed using ACCORDANT DSL for each use case
defined in the functional models. The UC4 deployment model had three
iterations, and Fig. 8b details the last version. Given that DV is based on
input package and functional view, they are imported by means of keyword use
inputPackage and functionalView respectively. This view includes the artifacts
that map connectors and components from functional view (e.g. ENSOPredictor)
to deployable elements (e.g. ENSOArtifact). Devices and deployments were
specified to support the computation requirements. For instance, deployments
of Spark master and worker nodes (e.g. SparkWorkerDep) details the number of
replicates, pods and execution environments (ExecEnv). ExecEnv defines the
docker image, CPU and memory requirements, ports, and commands along with the
artifacts to be deployed (ENSOArtifact). Finally, the sensitivity point
UC4_SP1 associates the deployment SparkWorkerDep to performance QS, and the
tactic increase available resources (see Section 6.1) to support distributed
computing over a Spark cluster.

###  6.5 Integration and Code Generation

Once the FV and DV models were designed and integrated, the code generation
produced both the functional code and IaC. On the one hand, the functional
code is a Spark driver program as detailed in Listing 1, where ENSOPredictor
component implements the PMML model in Spark technology. The Spark program
defines data input and output from the Data Dictionary and Mining Schema
embedded in PMML specifications. On the other hand, infrastructure code is the
configuration files which specify the provision and configuration policies of
Kubernetes cluster. Listing 2 shows an example of generated Kubernetes files.
The whole code of use cases is publicly available in the accordant-usecases
repository.

Listing 1: Generated Java Code of EnsoEstimator Component for Spark Streaming

⬇

SparkSession sparkSession = new SparkSession(sc.sc());

InputStream pmmlFile = new URL(”file:////path/ElNinoPolReg.pmml”)

EvaluatorBuilder b = new LoadingModelEvaluatorBuilder().load(pmmlFile);

Evaluator eval = builder.build();

TransformerBuilder pmmlTransformerBuilder =

new TransformerBuilder(evaluator)

.withTargetCols().exploded(true);

List<StructField> fields = new ArrayList<StructField>();

fields.add(DataTypes.createStructField(”latitude”, DataTypes.DoubleType,
true));

…

fields.add(DataTypes.createStructField(”s_s_temp”,DataTypes.DoubleType,
true));

StructType schema = DataTypes.createStructType(fields);

Transformer pmmlTransformer = pmmlTransformerBuilder.build();

Dataset<Row> inputDs =
sparkSession.read().schema(schema).csv(”data/Elnino.csv”);

TransformerBuilder tb = new TransformerBuilder(eval);

Transformer transformer = tb.build();

Dataset<Row> resultDs = transformer.transform(inputDs);

resultDs.write().option(”header”, ”true”).csv(”/enso-output/”);

…

Listing 2: Generated YAML Code from Deployment Specification for Kubernetes
(Extract)

⬇

apiVersion: apps/v1

kind: Deployment

metadata:

name: SparkWorkerDep

spec:

replicas: 3

spec:

containers:

- name: SparkWEnv

image: ramhiser/spark:2.0.1

command: [/spark-worker]

ports:

- containerPort: 8081

resources:

requests:

cpu: 0.3

…

###  6.6 Code Execution

Kubernetes code was executed on the AWS cloud using Amazon Elastic Container
Service for Kubernetes (Amazon EKS) and Elastic Compute Cloud (EC2). After
that, the software code was installed over the cluster to operationalize the
end-to-end solution.

###  6.7 Solution Monitoring

Performance metrics for each use case in operation were collected and
validated against QS defined in Section 6.1. As a result, different deployment
configurations were designed, deployed and monitored in each iteration to
observe the fulfillment of QS.

##  7 Results and Discussion

Revisiting the related work reviewed in Section 3, we have shown in practice
how ACCORDANT bridge the gap among analytics, software architecture, and
DevOps. As presented in Fig. 1, ACCORDANT follows the SoC principle by means
of three different languages to specify domain concerns. Analytics models in
ACCORDANT are cross-industry and technology-neutral. In terms of software
architecture, ACCORDANT supports QS specifications aligned to FV and DV, and
these models can be specified independently but in an integrated way.
Architectural tactics enable software architects to describe and communicate
their decisions. Code generators offer flexibility and impact positively the
development and deployment efficiency. Respecting DevOps practice, deployment
models allow us to design deployment diagrams, not limited to a number of
instances. Continuous deployment is supported via IaC and code generation, and
QS-monitoring is implemented by injecting logging code in the generated
applications. Finally, self-adaptation is not covered in the current version
of ACCORDANT.

To summarize, though a large variety of component-connector metamodels have
been previously proposed, as far as we know, our contribution resides in
specialize a component-connector metamodel in the BDA domain, and integrate it
with architectural inputs and deployment models to offer a holistic design.
Additionally, this section presents and discusses the experimental results
obtained during the iterative development and deployment phases of UC1, UC2,
UC3, and UC4.

###  7.1 Development and Deployment Time

Fig. 9 depicts the development and deployment time (in hours) accumulated for
all iterations per use case. It is worth noting that development time using
ACCORDANT is higher (between 23% and 47%) compared to SpringXD and Fastscore,
but the deployment time is significantly lower (between 50% and 81%) using
ACCORDANT. The higher development time can be explained by the time required
in ACCORDANT to specify architectural inputs, and many details in the FV. In
addition, the current version of the ACCORDANT prototype generates functional
code for estimators, but ingestor, sinks, and connectors still require manual.
Although ACCORDANT required more effort in the development phase, this effort
is rewarded during the deployment phase, where infrastructure and QS-
monitoring are provided automatically aligned to Inputs and FV, unlike other
approaches. This benefit can be observed on the deployment time across all use
cases using ACCORDANT, because they are more similar than the other
approaches.

The biggest time differences arise from UC2 that demands more time because it
includes a more complex pipeline involving two estimators: NMAC detector and
K-means clustering. Another interesting finding was that the high-level reuse
of previous architectural decisions (tactics) reduced the time of development
as shown the marked decreasing between use cases, and the growing gain factor
among iterations detailed in Fig. 9. These results suggest that ACCORDANT is
most suitable for application involving multiple iterations, or in subsequent
applications where reusing architectural decisions, models, and metrics can
reduce development times.

Figure 9: Development and Deployment Time for Use Case

###  7.2 Gain Factor Comparison

The gain factor metric presented in Section 5.5 was calculated for each use
case and iteration of development and deployment phases as depicted in Fig.
10. ACCORDANT’s gain factor was higher for all use cases, in the development
phase (Fig. 10a), what suggests that the high-level abstractions promote the
highest reduction of development time among consecutive iterations. The
highest gain factor was 0.46 in the UC3, it means reducing in 46% the
development time between consecutive iterations. The greatest gain factor
difference over the other approaches was 0.13 in the UC3. Regarding the
deployment gain factor (Fig. 10b), ACCORDANT also exhibited the highest gain
factor, on an even higher proportion, up to 0.75 in UC4. This means each
deployment iteration reduces the time in 75% compared to the previous one.
Similar to the deployment time in the previous section, we argue that the gain
factor in the deployment phase is greater because of the IaC generation is not
present in the other approaches.

Figure 10: Gain Factor for Use Case

##  8 Conclusions

We have presented a DevOps and DSM proposal to design, deploy, and monitor BDA
solutions. We have positioned the ACCORDANT contributions within the related
work. Four use cases from different domains were used to evaluate our approach
against two BDA frameworks. As a result, ACCORDANT has shown to facilitate and
accelerate iterative development and deployment phases by offering an
integrated and high-level design BDA applications. The greatest time reduction
was reported in the deployment phase, achieving up to 81% compared to other
approaches. In contrast, the development times offered by ACCORDANT were
greater. Despite the longer development time, deployment time is significantly
reduced thanks to the QS, FV, and DV alignment. ACCORDANT’s gain factor was
higher, which implies a higher reduction time in each iteration.

In contrast, some limitations have emerged from experimentation. The
development phase is slower than the other approaches for multiple reasons.
The current version of the ACCORDANT’s prototype requires supplementary manual
coding what increases the development time. ACCORDANT also requires more
design details and architectural inputs. These additional definitions are
rewarded in consecutive iterations, so ACCORDANT is most suitable for
application involving multiple iterations. Finally, our approach takes
advantage of reusing architectural decisions and models, hence, first-time or
one-time applications may not be benefited from our proposal.

As future work, the performance metrics collected along with FV and DV models
could allow us to propose a performance model to predict the expected
application-specific behavior based on the functional model, deployment model,
and target technology to recommend optimal architecture configuration for a
defined QS. Furthermore, we could include features to simulate and verify
correctness properties over the models such as technology selection in the FV
model and resource allocation in the DV model. Given that PMML provides a
model verification schema to validate results accuracy, a future extension
could incorporate automated model verification. This approach has been used
for deploying analytics components and connectors on virtual machines over
cloud infrastructure, but different paradigms such as serverless or fog
computing may open new research lines.

##  Appendix A Abbreviations

Table 3: Abbreviations ADD | Attribute Driven Design Method
---|---
ADS-B | Automatic Dependent Surveillance—Broadcast
BDA | Big Data Analytics
DSL | Domain Specific Language
DSM | Domain Specific Model
DV | Deployment View
ENSO | El Nino/Southern Oscillation
FV | Functional View
IaC | Infrastructure as Code
NMAC | Near Mid-air Collision
PFA | Portable Format for Analytics
PMML | Predictive Model Markup Language
QS | Quality Scenario

## References

  * [1] H.-M. Chen, R. Kazman, S. Haziyev, Agile Big Data Analytics for Web-Based Systems: An Architecture-Centric Approach, IEEE Transactions on Big Data 2 (3) (2016) 234–248.  doi:10.1109/TBDATA.2016.2564982.
  * [2] R. Ranjan, Streaming Big Data Processing in Datacenter Clouds, IEEE Cloud Computing (2014) 78—-83.
URL http://mahout.apache.org

  * [3] D. Wegener, S. Rüping, On reusing data mining in business processes-a pattern-based approach, in: International Conference on Business Process Management, Springer, 2010, pp. 264–276.
  * [4] H.-M. Chen, R. Schütz, R. Kazman, F. Matthes, How Lufthansa Capitalized on Big Data for Business Model Renovation, MIS Quarterly Executive 1615 (14) (2017) 299–320.
  * [5] K. Rexer, 2013 Data Miner Survey, Tech. rep., Rexer Analytics (2013).
  * [6] K. Rexer, P. Gearan, H. Allen, 2015 Data Science Survey, Tech. rep., Rexer Analytics (2016).
  * [7] C. Castellanos, B. Pérez, C. A. Varela, M. d. P. Villamil, D. Correal, A survey on big data analytics solutions deployment, in: T. Bures, L. Duchien, P. Inverardi (Eds.), Software Architecture, Springer International Publishing, Cham, 2019, pp. 195–210.
  * [8] N. Rozanski, E. Woods, Software Systems Architecture : Working with Stakeholders Using viewpoints and Perspectives, Addison-Wesley, 2005.
  * [9] H. Cervantes, R. Kazman, Designing software architectures: a practical approach, Addison-Wesley Professional, 2016.
  * [10] Thomas Erl, Wajid Khattak, Paul Buhler, Big Data Fundamentals: Concepts, Drivers & Techniques, Prentice Hall Press, 2016.
URL http://dl.acm.org/citation.cfm?id=2898954

  * [11] N. Marz, J. J. O. Warren, Big data : principles and best practices of scalable real-time data systems, 2015\.
URL https://www.manning.com/books/big-data

  * [12] I. Gorton, J. Klein, Distribution, data, deployment: Software architecture convergence in big data systems, IEEE Software 32 (3) (2014) 78–85.
  * [13] F. Ullah, M. A. Babar, Architectural tactics for big data cybersecurity analytics systems: A review, Journal of Systems and Software 151 (2019) 81–118.
  * [14] M. Gribaudo, M. Iacono, M. Kiran, A Performance Modeling Framework for Lambda Architecture Based Applications, Future Generation Computer Systemsdoi:10.1016/j.future.2017.07.033.
  * [15] M. Guerriero, S. Tajfar, D. Tamburri, E. Di Nitto, Towards A Model-Driven Design Tool for Big Data Architectures, in: 2nd IWBDSE, IEEE, 2016, pp. 37–43.
  * [16] Y. Huang, X. Lan, X. Chen, W. Guo, Towards Model Based Approach to Hadoop Deployment and Configuration, in: 12th WISA, IEEE, 2015, pp. 79–84.  doi:10.1109/WISA.2015.65.
  * [17] L. Bass, I. Weber, L. Zhu, DevOps: A Software Architect’s Perspective, Addison-Wesley Professional, 2015.
  * [18] D. Lechevalier, R. Ak, Y. T. Lee, S. Hudak, S. Foufou, A Neural Network Meta-Model and its Application for Manufacturing, in: 2015 IEEE International Conference on Big Data, 2015, pp. 1428–1435.
  * [19] CloverDX, CloverDX Data Integration.
URL https://www.cloverdx.com/

  * [20] A. K. Sujeeth, H. Lee, K. J. Brown, H. Chafi, M. Wu, A. R. Atreya, K. Olukotun, T. Rompf, M. Odersky, OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning, in: 28th ICML, 2011, pp. 609—-616.
  * [21] M. Alrifai, H. Eichelberger, C. Qui, R. Sizonenko, S. Burkhard, G. Chrysos, Quality-aware Processing Pipeline Modeling, Tech. rep., QualiMaster Project (2014).
  * [22] Open Data Group, FastScore.
URL https://www.opendatagroup.com/fastscore

  * [23] S. Anandan, M. Bogoevici, G. Renfro, I. Gopinathan, P. Peralta, Spring XD: a modular distributed stream and batch processing system, in: Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems - DEBS ’15, ACM Press, New York, New York, USA, 2015, pp. 217–225.  doi:10.1145/2675743.2771879.
  * [24] M. Artac, T. Borovsak, E. Di Nitto, M. Guerriero, D. Perez-Palacin, D. A. Tamburri, Infrastructure-as-Code for Data-Intensive Architectures: A Model-Driven Development Approach, in: 2018 IEEE International Conference on Software Architecture (ICSA), IEEE, 2018, pp. 156–165.  doi:10.1109/ICSA.2018.00025.
  * [25] D. Perez-Palacin, J. Merseguer, J. I. Requeno, M. Guerriero, E. Di Nitto, D. A. Tamburri, A uml profile for the design, quality assessment and deployment of data-intensive applications, Software and Systems Modeling 18 (6) (2019) 3577–3614.
  * [26] C. Castellanos, D. Correal, J.-D. Rodriguez, Executing Architectural Models for Big Data Analytics, in: C. E. Cuesta, D. Garlan, J. Pérez (Eds.), Software Architecture, Springer International Publishing, Cham, 2018, pp. 364–371.
  * [27] R. Wojcik, F. Bachmann, L. Bass, P. Clements, P. Merson, R. Nord, B. Wood, Attribute-driven design (add), version 2.0, Tech. rep., CARNEGIE-MELLON UNIV PITTSBURGH PA SOFTWARE ENGINEERING INST (2006).
  * [28] R. N. Taylor, N. Medvidovic, D. E. M., Software Architecture: Foundations, theory and practice, John Wiley and Sons, Inc, 2010.