# An Expert System for Redesigning Software for Cloud Applications

Rahul Yedida<EMAIL_ADDRESS>Rahul Krishna<EMAIL_ADDRESS>Anup Kalia
<EMAIL_ADDRESS>Tim Menzies<EMAIL_ADDRESS>Jin Xiao<EMAIL_ADDRESS>Maja Vukovic<EMAIL_ADDRESS>Department of Computer Science, North Carolina
State University, USA  IBM Research, USA

###### Abstract

Cloud-based software has many advantages. When services are divided into many
independent components, they are easier to update. Also, during peak demand,
it is easier to scale cloud services (just hire more CPUs). Hence, many
organizations are partitioning their monolithic enterprise applications into
cloud-based microservices.

Recently there has been much work using machine learning to simplify this
partitioning task. Despite much research, no single partitioning method can be
recommended as generally useful. More specifically, those prior solutions are
“brittle”; i.e. if they work well for one kind of goal in one dataset, then
they can be sub-optimal if applied to many datasets and multiple goals.

In order to find a generally useful partitioning method, we propose DEEPLY .
This new algorithm extends the CO-GCN deep learning partition generator with
(a) a novel loss function and (b) some hyper-parameter optimization. As shown
by our experiments, DEEPLY generally outperforms prior work (including CO-GCN,
and others) across multiple datasets and goals. To the best of our knowledge,
this is the first report in SE of such stable hyper-parameter optimization.

To enable the reuse of this research, DEEPLY is available on-line at
https://bit.ly/2WhfFlB.

###### keywords:

software engineering , microservices , deep learning , hyper-parameter
optimization , refactoring

††journal: Expert Systems with Applications

##  1 Introduction

As more and more enterprises move to the cloud, new tools are needed. For
example, IBM helps clients with millions of lines of code each year in this
refactoring process. In one such effort, IBM worked with a Fortune 100 company
to recommend partitions for a system with over one million lines of code.
These partitions were manually inspected by subject matter experts and
verified within weeks (as opposed to a year of manual
effort)111http://tiny.cc/mono2micro. Unfortunately, the tool support needed
for this process is still in its infancy. For example, consider the problem of
how to divide up old software for the cloud (i.e., into microservices).
Informally, we need to encourage cohesion and minimize coupling. However, AI-
based tools for doing this require more precise definitions of cohesion and
coupling. In this paper, we show six state-of-the-art tools focused on that
exact problem, which internally have several internal hyper-parameter choices.
The challenge with these tools is to tame that large hyper-parameter space.

Data mining is a powerful tool but, like any other software system ([40],
analysts are often puzzled by all the options for the control settings. For
example, consider the task of converting monolithic enterprise software into
cloud microservices. For the task, it is a common practice to apply some
clustering algorithm to decide how to break up the code into $k$ smaller
microservices. A common question asked by programmers is “what is a good value
for $k$"? More generally, across all the learners used for microservice
partitioning, currently there is little support for selecting appropriate
control settings ([13, 41].

Tools that can automatically learn settings for data miners are called hyper-
parameter optimizers (HPO). These tools can learn (e.g.) good $k$ values while
also optimizing for other goals including cluster coherence (which should be
maximized) and coupling (which should be minimized). But HPO suffers from
hyper-parameter brittleness. For example, Tu & Nair [38] reported that if an
optimizer works well for one kind of goal in one data set, they can be sub-
optimal if applied to multiple datasets and goals. In the case of redesigning
software monoliths as cloud microservices, Yedida et al. [41] recently
reported that different HPO tools perform best for different sets of goals
being explored on different datasets. To say that another way, based on past
results, no specific prior partitioning method can be recommended as generally
useful. This we consider this as a significant problem. As designs get more
complex, partitioning methods become very slow ([41]. For example, at the time
of this writing, we are running our algorithms for an industrial client. That
process has taken 282 CPU hours for 1 application. Hence it is less-than-ideal
to ask engineers to hunt through the output of multiple partitioning
algorithms, looking for results that work best for their particular domain.
This especially true when each of those algorithms runs very slowly. Instead,
we should be able to offer them one partitioning method that is generally most
useful across a wide range of problems.

To find a generally useful partitioning methods, this paper seeks HPO tools
that perform best across multiple datasets and goals (and prior work ([20, 19,
26, 27, 13] tended to explore just one or two partitioning methods). Thus, we
propose DEEPLY , which is a novel combination of optimization (using
Bergstra’s hyperopt tool ([7]) and a loss function. As shown by ur results,
DEEPLY generally works well across multiple goals and data sets.

To understand the benefits of DEEPLY , we investigate five research questions.

RQ1: How prevalent is hyper-parameter brittleness in automated microservice
partitioning? We verify Yedida et al. [41]’s results, who showed that hyper-
parameter optimizers are “brittle", i.e., they work well for a few metrics and
datasets, but are not useful across multiple goals and data sets. The
verification is crucial to set the motivation for our contribution.

RQ2: Is hyper-parameter optimization enough to curb the aforementioned
optimizer brittleness? Here we will show that standard hyper-parameter
optimization methods are insufficient for solving brittleness.

RQ3: Since hyper-parameter optimization methods are not enough, How else might
we fix the aforementioned brittleness problem? To that purpose, we propose
DEEPLY , a novel combination of hyper-parameter optimizers with a new loss
function.

RQ4: Does DEEPLY generate “dust” (where all functionality is loaded into one
partition) or “boulders” (where all partitions contain only one class each)?
Here, we show that DEEPLY avoids two anti-patterns. Specifically, DEEPLY does
not create “boulders” or “dust”.

The rest of this paper is structured as follows. We provide a detailed
background on the problem and the various attempts at solving it in §2. We
then formalize the problem and discuss our method in §3. In §4, we discuss our
experimental setup and evaluation system. We present our results in §5. Next,
we show how the crux of our approach extends beyond this problem in §6. We
discuss threats to the validity of our study in §7. Finally, we conclude in
§8.

Before we begin, just to say the obvious, when we say that hyper-parameter
optimization is becomes more generally useful method with DEEPLY , we mean
“generally useful across the data sets and metrics explored thus far”. It is
an open question, worthy of future research, to test if our methods apply to
other datasets and goals.

##  2 Designing for the Cloud

To fully exploit the cloud, systems have to be rewritten as “microservices”
comprising multiple independent, loosely coupled pieces that can scale
independently. Microservice architectures have many advantages ([3, 39] such
as technology heterogeneity, resilience (i.e., if one service fails, it does
not bring down the entire application), and ease of deployment. Therefore, it
is of significant business interest to port applications to the cloud under
the microservice architecture. For example, Netflix states that the elasticity
of the cloud and increased service availability are two of the primary reasons
it switched to a microservice
architecture222https://about.netflix.com/en/news/completing-the-netflix-cloud-
migration. They use the cloud for distributed databases, big data analytics,
and business logic. Further, they mention that moving to the cloud reduced
their averaged cost. .

Daya et al. [11] report on the long queue of applications waiting to be ported
to the cloud. Once ported to cloud-based microservices, code under each
microservice can be independently enhanced and scaled, providing agility and
improved speed of delivery. But converting traditional enterprise software to
microservices is problematic. Such traditional code is monolithic, i.e., all
the components are tightly coupled, and the system is designed as one large
entity. Thus, some application refactoring process is required to convert the
monolithic code into to a set of small code clusters.

Table 1:  Yedida et al. [41] report five widely used metrics to assess the quality of microservice partitions. In this table, [-] denotes metrics where less is better and [+] denotes metrics where more is better. For details on how these metrics are defined, see §4.4. Metric | Name |  Description |  Goal
---|---|---|---
BCS [-] ([21] | Business context sensitivity |  Mean entropy of business use cases per partition |  If minimized then more business cases handled locally
ICP [-] ([21] | Inter-partition call percentage |  Percentage of runtime calls across partitions |  If minimized then less traffic between clusters
SM [+] ([19] | Structural modularity |  A combination (see §4.4) of cohesion and coupling defined by Jin et al. [19] |  If maximized then more self-contained clusters with fewer connections between them
MQ [+] ([27] | Modular quality |  A ratio involving cohesion and coupling, defined by Mitchell & Mancoridis [27] |  If maximized then more processing is local to a cluster
IFN [-] ([27] | Interface number |  Number of interfaces needed in the microservice architecture |  If minimized then fewer calls between clusters
NED [-] ([21] | Non-extreme distribution |  Number of partitions with non-extreme values |  If minimized, fewer “boulders" (large monolithic partitions) or “dust” (partitions with only one class)
Table 2: Hyper-parameter choices studied in this paper. All the methods take runtime traces (or use cases) as their input. These methods return suggestions on how to build partitions from all the classes seen in the traces. For further details on these partitioning algorithms, see §3. Algorithm |  Hyper-parameter | (min, default, max)
---|---|---
Mono2Micro ([21, 20]) |  Number of clusters | (2, 5, 10)
FoSCI ([19]) |  Number of clusters | (5, 5, 13)
Number of iterations to run NSGA-II | (1, 3, 6)
Population size for NSGA-II | (30, 100, 200)
Parent size to use in NSGA-II | (10, 30, 50)
MEM ([26]) |  Maximum partition size | (17, 17, 17)
Number of partitions | (2, 5, 13)
Bunch ([27]) |  Number of partitions | (3, 5, 13)
Initial population size for hill climbing | (2, 10, 50)
Number of neighbors to consider in hill-climbing iterations | (2, 5, 10)
CoGCN ([13]) |  Number of clusters | (2, 5, 13)
Loss function coefficients $\alpha_{1},\alpha_{2},\alpha_{3}$ | (0, 0.1, 1)
Number of hidden units in each layer, $h_{1},h_{2}$ | (4, 32, 64)
Dropout rate | (0, 0.2, 1)

When done manually, such refactoring is expensive, time-consuming, and error-
prone. Hence, there is much current interest in automatically refactoring
traditional systems into cloud services.

Some of the authors of this paper are part of the IBM “Mono2Micro” that helps
client redesign their systems for the cloud. Some of that analysis is manual
since it replies on extensive domain knowledge. That said, increasingly, there
is automation applied to this task. For example, given a set of test cases or
use cases, it is reasonable to ask “how does this knowledge of frequently use
tests or use cases inform our microservices design?”. For that purpose, some
AI clustering can be used.

But when we try to use AI tools for this task, we often encounter the same
problem. Specifically, there are so many algorithms and partitioning goals:

  * •

Table 1 lists the various coupling and cohesion goals used by prior work on
partition generation. Note that in column one, any goal marked with “[-]”
should be minimized and all other goals should be maximized.

  * •

See also Table 2 which lists some of the partitioning tools, as well as their
control parameters. In the table, the core idea is to understand what parts of
the code call each other (e.g., by reflecting on test cases or use cases),
then cluster those parts of the code into separate microservices. These
algorithms assess the value of different partitions using scores generated
from the Table 1 metrics.

In such a rich space or options, different state-of-the-art AI approaches may
generate different recommendations. Yedida et al. [41] concluded that looking
at prior work, we seem to have a situation where analysts might run several
partitioning algorithms to find an algorithm that performs the best based on
their business requirements. For example, analysts might prefer Mono2Micro
([21, 20]) over others considering the clean separation of business use cases
where as they might prefer CoGCN ([13]) and FoSCI ([19]) for low coupling. As
stated in the introduction, this is hardly ideal since these partitioning
algorithms can be slow to execute. Accordingly, this paper seeks optimization
methods that support partitioning for a wide range of data sets and goals.

##  3 Algorithms for Microservice Partitioning

Informally, partitioning algorithms take domain knowledge and propose
partitions containing classes that often connect to each other. For example,
given a set of use cases or traces of test case execution:

  * •

Identity the entities (such as classes) used in parts of the use cases/test
cases;

  * •

Aggregate the frequently connecting entities;

  * •

Separate the entities that rarely connect.

This can be formalized as follows. Consider classes333This definition
trivially extends to languages without classes; translation units such as
functions could be used instead, for example. in an application $A$ as
$\mathcal{C}^{A}$ such that
$\mathcal{C}^{A}=\\{c_{1}^{A},c_{2}^{A},\ldots,c_{k}^{A}\\}$, where
$c_{i}^{A}$ represents an individual class. With this, we define a partition
as follows:

######  Definition 1.

A partition $\mathcal{P}^{A}$ on $\mathcal{C}^{A}$ is a set of subsets
$\\{P_{1}^{A},P_{2}^{A},\ldots,P_{n}^{A}\\}$ such that

  * •

$\bigcup\limits_{i=1}^{n}P_{i}^{A}=\mathcal{C}^{A}$, i.e., all classes are
assigned to a partition.

  * •

$P_{i}^{A}\neq\phi~{}\forall i=1,\ldots,n$, i.e., there are no empty
partitions.

  * •

$P_{i}^{A}\cap P_{j}^{A}=\phi~{}\forall i\neq j$, i.e., each partition is
unique.

######  Definition 2.

A partitioning algorithm $f$ is a function that induces a partition
$\mathcal{P}_{A}$ on an application with classes $\mathcal{C}^{A}$.

The goal of a microservice candidate identification algorithm is to identify a
function $f$ that jointly maximizes a set of metrics
$m_{1},m_{2},\ldots,m_{p}$ that quantify the quality of the partitions, i.e.,
given an application characterized by its class set $\mathcal{C}_{A}$, we aim
to find:

$\mathcal{P}^{*}_{A}=\operatorname*{arg\,max}\limits_{\mathcal{P}\in
B_{\mathcal{C}_{A}}}\sum\limits_{i=1}^{p}\alpha_{i}m_{i}(\mathcal{P};f)$ (1)

where $B_{C_{A}}$ denotes the set of all partitions of $C_{A}$ and $p$ is the
number of partitions.

The following is structured as follows. We first discuss prior approaches, and
the issues with them. These prior approaches come from a recent review of
automated microservice partitioning ([41]), which studied the effect of hyper-
parameter optimization on these approaches. Then, we will present the business
case for a new algorithm, and finally, we will discuss our approach.

Table 3: Deep learning: a tutorial. In this work, deep learning is used by
both CO-GCN and DEEPLY . A deep learner is a directed acyclic graph. Nodes are
arranged into “layers", which are proper subsets of the nodes. Each node
computes a weighted sum of its inputs, and then applies an “activation
function", yielding its output.
---
Deep learning (DL) ([17]) is an extension of prior work on multi-layer
perceptrons, where the adjective “deep” refers to the use of multiple layers
in the network.
The weights form the parameters of the model, which are learned via
backpropagation ([29]) using the rule
$\theta=\theta-\eta\nabla_{\theta}\mathcal{L}$ where $\theta$ represents the
parameters of the model, $\eta$ is the learning rate, and $\mathcal{L}$ is the
loss function (described below).
A deep learner with $L$ layers produces a prediction $\hat{y}$, and the
network learns parameters by using gradient descent to minimize the loss
function $\mathcal{L}(y,\hat{y})$. This loss function can be arbitrarily
designed, as done by the authors of CO-GCN ([13]) to suit the specific needs
of the application.
For static learning rates, there has been an increased interest in learning
rate schedules, where $\eta$ is replaced by a function $\eta(t)$, where $t$ is
the iteration. The proposed schedules have both theoretical ([33, 44]) and
empirical ([35, 34]) backing. Generally, however, all the papers agree on the
use of non-monotonic learning rate schedules. Specifically, Smith [34] argues
for cyclical learning rates, where the learning rate oscillates between
$(\eta_{l},\eta_{u})$, a preset range for a specified number of cycles. More
recently, Smith & Topin [36] proposed a “1cycle" learning rate policy, where a
single cycle is used, and showed this to be effective, but simple.
Dropout ([37]) is a technique which involves removing some nodes with a
probability $p$ during training, and adjusting the weights of the model during
testing. Srivastava et al. [37] argues that this enforces sparsity in the
model weights, which improves the model performance and makes it more robust
to noise.

###  3.1 FoSCI

FoSCI ([19]) uses runtime traces as a data source. They prune traces that are
subsets of others, and use hierarchical clustering with the Jaccard distance
to create “functional atoms". These are merged using NSGA-II ([12]), a multi-
objective optimization algorithm to optimize for the various goals. Since
FoSCI relies on a somewhat older optimization algorithm (from 2002 ([12])),
and all the goals are given equal priority, this can lead to suboptimal
results if the different goals have, for example, different scales.

###  3.2 Bunch

Bunch ([27]) is based on search techniques such as hill-climbing444Their tool
offers other heuristic-based approaches as well. to find a partition that
maximizes two specific metrics. It is well established that such greedy search
algorithms can get stuck in local optima easily ([30]).

###  3.3 Mono2Micro

Mono2Micro ([21, 20]) collects runtime traces for different business use
cases. Then, they use hierarchical clustering with the Jaccard distance to
partition the monolith into a set of microservices. However, as noted by
Yedida et al. [41], this approach takes as input the number of partitions,
which different architects may disagree on, or may not know the value of.

###  3.4 CO-GCN

CO-GCN uses the deep learning technology discussed in Table 3. More
specifically, CO-GCN ([13]) uses the call graph (built from the code) as input
to a graph convolutional neural network ([22]). They develop a custom loss
function (defined in Table 3) that relates to the metrics being optimized for,
and thus, the neural network can be seen as a black-box system that optimizes
for Equation (1). However, while their use of a custom loss function tailored
to the goals is novel, their approach has several hyper-parameters that a non-
expert may not know how to set, and their study did not explore hyper-
parameter optimization .

CO-GCN uses an exponential learning rate scheduling policy (see Table 3 for
details), where the learning decays exponentially. In our approach, we instead
use the 1cycle ([36]) policy, which has more experimental backing.

CO-GCN also uses dropout ([37]), which involves removing some nodes with a
probability $p$ during training, and adjusting the weights of the model during
testing. Srivastava et al. [37] argues that this enforces sparsity in the
model weights, which improves the model performance and makes it more robust
to noise.

CO-GCN is controlled by the settings of Table 2. Later in this paper, we show
that there are several benefits in using hyper-parameter optimization to
automatically select good subsets of these hyper-parameters. Note that (a) all
our partitioning methods (CO-GCN and all the others shown above) utilize
hyper-parameter optimization; (b) even better results can be obtained via
augmenting HPO with a novel loss function (and that combination of HPO+loss
function is what we call DEEPLY ).

###  3.5 The case for a new algorithm

The discussion so far shows that there are several approaches that perform
automatic microservice partitioning, with different goals, i.e., businesses
may be able to choose based on the goals they need, which application to use.
However, our industry partners have stressed that these are not widely adopted
for several reasons.

Key among these reasons is that these approaches all have hyper-parameters.
Industry practitioners, having worked with their monolithic systems, may not
know how many partitions would be “ideal" for a microservice architecture.
Furthermore, there may be disagreements between different practitioners on the
number of microservices (clusters), for example. Moreover, hiring an expert
such as a system architect can be expensive, and even then provides no
guarantees that just because the architect recommends, say, 4 microservices,
that the system will build the microservices envisioned by the architect.

On top of the above, refactoring a monolithic application into microservices
is expensive. It is a time-consuming process that can take months to complete,
with rigorous testing required to ensure that the outward functionality of the
system has not been changed. Therefore, it is crucial to large enterprises
that if and when such a change is made, that it is done well. What “well"
means can vary from business to business, or even application to application.
For example, while a commercial product that faces millions of users might
have the non-functional requirement that it be fast, an internal tool may not
have such a requirement.

Finally, we bring up the results of the study by Yedida et al. [41].
Specifically, they studied microservice partitioning algorithms, and showed
that “there is no best algorithm". That is, each algorithm that they studied
(when tuned with hyper-parameter optimization) was an expert at one of the
metrics, but did poorly on the rest. Of course, this does not inspire much
confidence in businesses who might be open to adopting these automated
systems, and whose requirements may be constantly changing.

###  3.6 DEEPLY

Summarizing the approaches in prior work, we note that they suffer from the
following limitations:

  1. (a)

They have hyper-parameters that practitioners may not understand or know how
to set.

  2. (b)

They rely on techniques that can easily get stuck in suboptimal, local minima.

  3. (c)

They treat all the metrics as having equal priority, when it may not be the
case (e.g., some feature scaling may be needed, or the business priorities are
different).

To address the limitations, we propose an extension to the CO-GCN ([13]) deep
learner. DEEPLY uses an hyper-parameter optimization technique that is known
to deal with local optima better, and (b) a novel reweighting of the metrics
based on data.

When algorithms produce a highly variable output, then hyper-parameter
optimization , described in §3.6.2, can be used to automatically learn the
control settings that minimize the variance. Specifically, we use the fact
that a hyper-parameter optimization algorithm, in searching for optimal
parameters, must test different hyper-parameter configurations, each of which
produces a different result. The results are aggregated together to form a
frontier, from which we can pick the “best" sample. To do so, we notice that
simply picking a model based on one metric sacrifices the performance in
others. Moreover, using a sum of all metrics has the disadvantages of (a)
different scales for different metrics (b) learners doing well in the “easy"
metrics but not in the others (c) ignoring correlations among the metrics.
Therefore, we design a custom loss function to choose the best sample.

For further details on all the concepts in the last two paragraphs, see the
rest of this section.

####  3.6.1 Feature Engineering

DEEPLY uses the deep learning technology discussed in Table 3.

We format our datasets into the form required by a graph convolutional network
([22]) as suggested in CO-GCN ([13]). A graph convolutional network models a
graph as a pair of matrices $(A,X)$, where $A$ is the (binary) adjacency
matrix and $X$ is the feature matrix. Our graph is characterized by nodes
corresponding to classes, and edges corresponding to function calls between
two classes. If a class is not used by any of the APIs published by the
software, then it is removed from the graph. The adjacency matrix is trivially
defined as $M_{ij}=1$ if an edge exists between the vertices $i,j$, and 0
otherwise.

For the feature matrix, we follow the approach of suggested in CO-GCN ([13]).
Specifically, we first define the entry point matrix
$E\in\\{0,1\\}^{|V|\times|P|}$ (where $V$ is the set of classes, and $P$ is
the set of entry points). Entry points refer to APIs published by a software,
each potentially for different functions. Then, for each such entry point
(i.e., API), we consider the set of classes invoked in its execution. We
consider the entry $E_{ij}=1$ if class $i$ is part of the execution trace of
entry point $j$, and 0 otherwise. Additionally, we consider the co-occurrence
matrix $C\in\\{0,1\\}^{|V|\times|V|}$ such that $C_{ij}=1$ if both classes
$i,j$ occur in the same execution trace, and 0 otherwise. Finally, we define
the dependence matrix $D\in\\{0,1\\}^{|V|\times|V|}$ as $D_{ij}=D_{ji}=1$ if
class $i$ inherits from class $j$ or vice versa, and 0 otherwise. The feature
matrix $X\in\mathbb{R}^{|V|\times(|P|+2|V|)}$ is the concatenation of $E,C,D$
(in that order), and is then row-normalized.

CO-GCN also uses dropout ([37]), which involves removing some nodes with a
probability $p$ during training, and adjusting the weights of the model during
testing. Srivastava et al. [37] argues that this enforces sparsity in the
model weights, which improves the model performance and makes it more robust
to noise.

####  3.6.2 Hyper-parameter Optimization

Hyper-parameter optimization is the systematic process of finding an optimal
set of hyper-parameters for a model. In the specific case of optimizing CO-
GCN, those parameters are shown in Table 2.

Several approaches for hyper-parameter optimization now exist. Of note is the
work by Bergstra et al. [6]555As of August 2021, this paper has 2,800
citations. which discusses three different approaches. In this paper, we use a
newer, widely used hyper-parameter optimization algorithm called Tree of
Parzen Estimators (TPE) ([6]). TPE divides data points seen so far into best
and rest, each of which are modeled by a Gaussian distribution. New candidates
are chosen so that they are highly likely to be in the best distribution.
Evaluating these candidates adds to the data points, and the algorithm
continues. This algorithm was open-sourced by its authors in a package called
hyperopt ([7]). Therefore, in this paper, whenever we say “hyperopt", we mean
TPE as implemented by this package.

####  3.6.3 Loss Function

Table 3 discussed loss functions $\mathcal{L}$ that offers weights to the
feedback seen by the learner during the inner-loop of the learner process.
Numerous researchers report that augmented loss functions can enhance
reasoning:

  * •

Ryu & Baik [31] used reweighted Naive Bayes for cross-project defect
prediction;

  * •

In a similar approach, a reweighted Naive Bayes was used by Arar & Ayan [5]
for defect prediction and Yedida & Menzies [42] explored weighted loss
functions in deep learners for defect prediction.

  * •

In the AI literature, Lin et al. [25] propose a “focal loss" function that
uses an exponential weighting scheme.

To the best of our knowledge, augmenting the loss function has not been
previously attempted for improving microservice partitions.

One challenge with designing a loss function is how to obtain the required
weights. In DEEPLY , we first run the algorithm 1,000 times to generate a set
of metrics shown in Table 1. Then, we check the correlation among the metrics
using the Spearman correlation coefficients. If any two metrics have strong
correlations we prefer to keep one to remove the redundant metrics for the
optimization. Figure 1 indicates the correlations among the metrics. Clearly,
some of the metrics are highly correlated. Therefore, we use a reduced set of
metrics for evaluating our approaches. We set a threshold of 0.6 to prune the
set of metrics. We observe that MQ has higher correlations with other metrics
than SM. Since IFN and ICP are highly correlated across all datasets, we
arbitrarily choose ICP. This leads to the final set of metrics: BCS, ICP, and
SM. Note that to ensure that we do not generate “dust" (individual classes in
a partition) or “boulders" (monolithic partitions), we also include NED in the
final metric set (for which we assigned a static weight of 0.2).

Figure 1: Spearman $\rho$ among metrics from 1,000 runs.

Based on these three metrics, we choose a loss function to pick one model from
the frontier of best solutions. This loss function is

$\mathcal{L}_{H}(P)=\sum_{i}w_{i}m_{i}(P)$ (2)

where $w_{i}$ are weights assigned to each of the metrics $m_{i}$, and $P$ is
the partition. Additionally, we divide the BCS weight by 10 (for an overall
weight of 0.1) to normalize its value to the same order of magnitude as the
other metrics. Finally, we add in a term for the NED metric, so that we do not
generate “dust" or “boulders" (e.g., a monolith with 20 classes is not
partitioned into 19 and 1).

Next, we change the clustering method used in CO-GCN ([13]). Rather than using
the k-means clustering, which assumes globular clusters, we use spectral
clustering, which can produce more flexible cluster shapes ([43]). We also
change the cluster loss function to

$\mathcal{L}_{clus}=\sum\limits_{i}\min\limits_{j}\left|x_{i}-x_{j}\right|$
(3)

where $x_{i}$ and $x_{j}$ are data points.

Next, we update the learning rate scheduling policy to the “1cycle" policy
([34, 36]), which has been shown to lead to faster convergence (see Table 3
for details). We update the learning rate every 200 steps.

Finally, we change all the activation functions in the network to ReLU
($f(x)=\max(0,x)$) ([28]), except the last layer (which we leave unchanged to
$f(x)=x$). Specifically, like Desai et al. [13], we use an encoder-decoder
architecture with 2 layers per each layer; each layer being a graph
convolution as defined by Kipf & Welling [22]. Therefore, for an input feature
matrix $X$, we consider

$\displaystyle E\leftarrow$
$\displaystyle\mathrm{ReLU}(\hat{A}(\mathrm{ReLU}(\hat{A}XW_{1}))W_{2})$ (4)
$\displaystyle D\leftarrow$
$\displaystyle\mathrm{ReLU}(\hat{A}(\mathrm{ReLU}(\hat{A}EW_{3}))W_{4})$ (5)

Algorithm 1 shows the overall approach. In the algorithm the lines in red
(19-22,25) are from the CO-GCN algorithm (and all else is our extension). We
first collect correlation statistics using a Monte Carlo-style data collection
(lines 1-4). Then we set weights (lines 4-7) and run hyperopt for 100
iterations (lines 10-15), while collecting the results of each run. The
weighted loss function then picks the “best" candidate in line 16, and returns
the optimal hyper-parameters in line 17. The actual model training itself is
delegated to CO-GCN-modified.

We form the graph Laplacian in lines 18-20, and pretrain (for 1 epoch) the
encoder (see Eq. (4)) and decoder (see Eq. (5)). We initialize the clusters in
line 23, but use spectral clustering instead. Then, we run the actual training
for 300 epochs.

In summary, our new method DEEPLY differs from CO-GCN in the following ways:

1

2Algorithm  _DEEPLY _

Input : graph dataset $D$

Output : optimal hyper-parameters $\theta^{*}$

3

4 for  _1,000 iterations_ do

5 run CO-GCN-modified($D$) with random settings and collect metrics

6

7 end for

$w_{SM}\leftarrow-1/corr(SM,MQ)$

// initialize weights

8 $w_{ICP}\leftarrow 1/corr(BCS,ICP)$

9 $w_{NED}\leftarrow 0.2,w_{BCS}\leftarrow 0.1$

$\theta=\phi$

// initialize our history to empty set

$M=\phi$

// initialize results to empty

10 for  _i=1..100_ do

11 $\theta_{i}\leftarrow$ hyperopt($\theta$)

$M_{i}\leftarrow$ CO-GCN-modified($D;\theta_{i}$)

// run the model

append $\theta_{i}$ to $\theta$

// log hyper-params

append $M_{i}$ to $M$

// log results

12

13 end for

14

15
$\theta^{*}=\operatorname*{arg\,min}\limits_{\theta}\sum\limits_{j=1}^{4}w_{j}M_{j}$

16 return $\theta^{*}$

17

18

19

20Algorithm  _CO-GCN-modified_

Input : graph dataset $D=(A,X)$

Output : partitioned microservices

// compute the graph Laplacian

21 $\tilde{A}\leftarrow A+I$

22 define diagonal matrix $\tilde{D}$ such that
$\tilde{D}_{ii}=\sum\limits_{j}\tilde{A}_{ij}$

23
$\hat{A}\leftarrow\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}$

24

25 pretrain the encoder and decoder using (4) and (5) respectively

26 initialize the clusters using spectral clustering on embeddings

27 train the network $(E,D)$ for 300 epochs using Alg. 1 lines 5-9 from ([13])
and 1cycle ([36])

28 get embeddings using (4)

29 cluster the embeddings using spectral clustering

30 return _clusters_

31

32

Algorithm 1 DEEPLY . Note CO-GCN-modified is a sub-routine containing our
changes to the original CO-GCN system. The the lines in red (18-21,24) are
from the original CO-GCN algorithm (and all else is our extension).

  * •

We use a better learning rate scheduling policy from deep learning literature
(discussed in Table 3).

  * •

We update the activation functions used in the neural network (also see Table
3).

  * •

We run hyper-parameter optimization and choose from the Pareto frontier using
a custom loss function over a reduced space of metrics (discussed in §3.6.2).

  * •

We update the clustering method employed, and the corresponding term in the
deep learner loss function (see §3.6.3).

##  4 Experimental Design

In this section, we detail our experimental setup for the various parts of our
research. Broadly, we follow the same experimental setup as Yedida et al.
[41]. However, that paper concluded that different optimizers performed
differently across datasets and metrics, and that there was no one winning
algorithm. In this paper, using the method described in §3.6, we show that our
approach wins most of the time across all our datasets and metrics.

###  4.1 Case studies

We use four open source projects to evaluate our approach against prior work.
These are acmeair666https://github.com/acmeair/acmeair (an airline booking
application), daytrader777https://github.com/WASdev/sample.daytrader7 (an
online stock trading application),
jpetstore888https://github.com/mybatis/jpetstore-6 (a pet store website), and
plants999https://github.com/WASdev/sample.plantsbywebsphere (an online web
store for plants and pets). These applications are built using Java Enterprise
Edition (J2EE), and common frameworks such as Spring and Spring Boot.

In Table 4, we show statistics about the datasets (number of classes and
methods) and the runtime traces that we used (number of traces, and their
class and method coverage). While the coverage here may seem low, we note that
applications tend to have a significant amount of “dead”, or unreachable code.
For example, Brown et al. [8] found between 30 to 50% of an industrial
software system was dead code. Eder et al. [14] found that for an industrial
software system written in .NET, 25% of method genealogies were dead.
Therefore, we were not worried about the coverage seeming low.

Table 4: Statistics about the datasets used in this study. Application | #classes | #methods |  #Runtime traces |  Class coverage |  Method coverage
---|---|---|---|---|---
acmeair | 33 | 163 |  11 |  28 (84%) |  108 (66%)
daytrader | 109 | 969 |  83 |  73 (66%) |  428 (44%)
jpetstore | 66 | 350 |  44 |  36 (54%) |  236 (67%)
plants | 37 | 463 |  43 |  25 (67%) |  264 (57%)

###  4.2 Hyper-parameter Optimization

We use the Tree of Parzen Estimators (TPE) ([6]) algorithm from the hyperopt
([23]) package for hyper-parameter optimization . As discussed in §3.6, we use
$\mathcal{L}_{H}(P)$ from (2) to guide hyperopt towards an optimal set of
hyper-parameters. Table 2 lists the hyper-parameters that we tune, along with
their ranges. We run the hyper-parameter optimizer for 100 iterations. We
train for 300 epochs using an initial learning rate of 0.01.

###  4.3 Baselines

Following our literature review, we use the baselines shown in Table 2. All
these approaches, except Mono2Micro and CO-GCN, were either open-source, or
re-implemented by us. For a fair evaluation, we tune each algorithm with
hyperopt for 100 iterations (which is the same as our approach). We use the
same loss function $\mathcal{L}_{H}(P)$ shown in (2), but set all the weights
to either 1 (for metrics we wish to minimize), or -1 (for metrics we wish to
maximize), since the correlation-based weights are a novel idea for our
approach.

###  4.4 Metrics

For a fair comparison with prior work, we must compare using the same set of
metrics. We choose a total of five metrics to evaluate our core hypothesis
that hyper-parameter tuning improves microservice extraction algorithms. These
are detailed below. These metrics have been used in prior studies, although
different papers used different set of metrics in their evaluations. For
fairness, we use metrics from all prior papers. These metrics evaluate
different aspects of the utility of an algorithm that might be more useful to
different sets of users (detailed in the RQ1), e.g., BCS evaluates how well
different business use cases are separated across the microservices.

Inter-partition call percentage (ICP) ([21]) is the percentage of runtime
calls across different partitions. For lower coupling, lower ICP is better.

Business context sensitivity (BCS) ([21]) measures the mean entropy of
business use cases per partition. Specifically,

$BCS=\frac{1}{K}\sum\limits_{i=1}^{K}\frac{BC_{i}}{\sum_{j}BC_{j}}\log_{2}\left(\frac{BC_{i}}{\sum_{j}BC_{j}}\right)$

where $K$ is the number of partitions and $BC_{i}$ is the number of business
use cases in partition $i$. Because BCS is fundamentally based on entropy,
lower values are better.

Structural modularity (SM), as defined by Jin et al. [19], combines cohesion
and coupling, is given by

$\displaystyle SM$
$\displaystyle=\frac{1}{K}\sum_{i=1}^{n}\frac{coh_{i}}{N_{i}^{2}}-\frac{1}{N_{i}(N_{i}-1)/2}\sum_{i<j<K}coup_{i,j}$
$\displaystyle coh_{i}$
$\displaystyle=\frac{2}{m(m-1)}\sum\limits_{\begin{subarray}{c}c_{i},c_{j}\in
C\\\ i<j\end{subarray}}\sigma(c_{i},c_{j})$ $\displaystyle coup_{i,j}$
$\displaystyle=\frac{\sum\limits_{\begin{subarray}{c}c_{1}\in C_{i}\\\
c_{2}\in C_{j}\end{subarray}}\sigma(c_{1},c_{2})}{\lvert C_{i}\rvert\lvert
C_{j}\rvert}$

where $N_{i}$ is the number of classes in partition101010Note that we use the
term “partition” to refer to both the set of all class subsets, as well as an
individual subset, but it is typically unambiguous. $i$, $K$ is the number of
partitions, $coh_{i}$ is the cohesion of partition $i$, and $coup_{i,j}$ is
the coupling between partitions $i$ and $j$, $c_{i}$ refers to a class in the
subset, $C_{i},C_{j}$ are two partitions, $\sigma$ is a general similarity
function bounded in $[0,1]$, and the $i<j$ condition imposes a general
ordering in the classes. Higher values of SM are better.

Modular quality (MQ) ([27]), coined by Mitchell & Mancoridis [27], is defined
on a graph $G=(V,E)$ as

$\displaystyle MQ$ $\displaystyle=\frac{2\mu_{i}}{2\mu_{i}+\epsilon}$
$\displaystyle\mu_{i}$ $\displaystyle=\sum\limits_{(e_{1},e_{2})\in
E}\mathbbm{1}(e_{1}\in C_{i}\land e_{2}\in C_{i})$ $\displaystyle\epsilon$
$\displaystyle=\sum\limits_{\begin{subarray}{c}(e_{1},e_{2})\in E\\\ i\neq
j\end{subarray}}\mathbbm{1}(e_{1}\in C_{i}\land e_{2}\in C_{j})$

where $\mathbbm{1}$ is the indicator function, and $C_{i},C_{j}$ are clusters
in the partition. Higher values of MQ are better.

The interface number (IFN) ([27]) of a partition is the number of interfaces
needed in the final microservice architecture. Here, an interface is said to
be required if, for an edge between two classes in the runtime call graph, the
two classes are in different clusters.

Finally, the non-extreme distribution (NED) is the percentage of partitions
whose distributions are not “extreme" (in our case, these bounds were set to
min=5, max=20).

However, as shown in Figure 1, some of these metrics are highly correlated
with others. Therefore, to avoid bias in the evaluation and the loss function
(i.e., if metrics $M_{1}$ and $M_{2}$ are correlated, and optimizer $O$
performs best on $M_{1}$, it likely performs best on $M_{2}$ as well), we use
a subset of these metrics. Specifically, we prune the metric set as discussed
in §3.6.3. Note that across all our figures and tables, a “[-]" following a
metric means lower values are better, and a “[+]" following a metric means
that higher values are better.

###  4.5 Statistics

For comparing different approaches, we use statistical tests due to the
stochastic nature of the algorithms. According to standard practice ([16]), we
run our algorithms 30 times to get a distribution of results, and run a Scott-
Knott test as used in recent work ([1, 42]) on them. The Scott-Knott test is a
recursive bi-clustering algorithm that terminates when the difference between
the two split groups is insignificant. The significance is tested by the
Cliff’s delta using an effect size of 0.147 as recommended by Hess & Kromrey
[18]. Scott-Knott searches for split points that maximize the expected value
of the difference between the means of the two resulting groups. Specifically,
if a group $l$ is split into groups $m$ and $n$, Scott-Knott searches for the
split point that maximizes

$\mathbb{E}[\Delta]=\frac{|m|}{|l|}\left(\mathbb{E}[m]-\mathbb{E}[l]\right)^{2}+\frac{|n|}{|l|}\left(\mathbb{E}[n]-\mathbb{E}[l]\right)^{2}$

where $|m|$ represents the size of the group $m$.

The result of the Scott-Knott test is ranks assigned to each result set;
higher the rank, better the result. Scott-Knott ranks two results the same if
the difference between the distributions is insignificant.

##  5 Results

RQ1: How prevalent is hyper-parameter brittleness in automated microservice
partitioning?

Here, we ran the approaches from Table 2, 30 times each, then compared results
from those treatments with the statistical methods of §4.5. The median results
are shown in Table 5(a) and the statistical analysis is shown in Table 5(b).

In those results, we see that across different datasets and metrics, those
ranks vary widely, with little consistency across the space of datasets and
metrics. For example:

  * •

For jpetstore, we see that the best algorithms for the three metrics are
Mono2Micro, CO-GCN, and MEM respectively;

  * •

For acmeair, the best algorithms are Mono2Micro, and Bunch;

  * •

For plants, the best algorithms are FoSCI, CO-GCN, and MEM;

  * •

For daytrader, the best are Mono2Micro, CO-GCN, and Bunch.

Hence we say:  For all the data sets and goals studied here, there is
widespread performance brittleness.  As to why this issue has not been
reported before in the literature, we note that much of the prior work was
focused on one algorithm exploring one case study. To the best of our
knowledge, this is first to perform a comparison across the same datasets and
metrics and also propose a novel approach.

Table 5: Results on all datasets. The top row shows our median performance
scores over 30 runs, while the bottom row shows the Scott-Knott ranks (lower
is better). Gray cells indicate the best result according to the Scott-Knott
test. All algorithms were tuned by hyperopt.

(a) Median performance scores over 30 runs Algorithm | jpetstore | acmeair | plants | daytrader
---|---|---|---|---
BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+]
Bunch | 2.43 | 0.48 | 0.21 | 1.67 | 0.55 | 0.17 | 2.29 | 0.56 | 0.22 | 1.86 | 0.57 | 0.27
Mono2Micro | 1.67 | 0.53 | 0.05 | 1.58 | 0.4 | 0.06 | 2.49 | 0.39 | 0.07 | 1.58 | 0.4 | 0.07
CO-GCN | 2.79 | 0.27 | 0.2 | 1.89 | 0.47 | 0.15 | 2.46 | 0.35 | 0.24 | 2.09 | 0.05 | 0.13
FoSCI | 1.99 | 0.66 | 0.05 | 2.08 | 0.54 | 0.07 | 1.94 | 0.45 | 0.07 | 2.04 | 0.48 | 0.07
MEM | 2.74 | 0.48 | 0.22 | 2.13 | 0.69 | 0.03 | 2.04 | 0.42 | 0.25 | 2 | 0.59 | 0.09
DEEPLY | 2.35 | 0.06 | 0.2 | 1.41 | 0.18 | 0.16 | 1.87 | 0.02 | 0.27 | 1.74 | 0.03 | 0.17

(b) Scott-Knott ranks. For all metrics, lower is better. Algorithm | jpetstore | acmeair | plants | daytrader
---|---|---|---|---
BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+] | BCS [-] | ICP [-] | SM [+]
Bunch | 4 | 3 | 2 | 2 | 4 | 1 | 4 | 6 | 4 | 3 | 5 | 1
Mono2Micro | 1 | 5 | 5 | 1 | 2 | 4 | 6 | 3 | 5 | 1 | 3 | 6
CO-GCN | 6 | 2 | 2 | 3 | 3 | 3 | 5 | 2 | 3 | 6 | 2 | 3
FoSCI | 2 | 6 | 4 | 4 | 4 | 5 | 2 | 5 | 6 | 5 | 4 | 5
MEM | 5 | 4 | 1 | 5 | 5 | 6 | 3 | 4 | 2 | 4 | 6 | 4
DEEPLY | 3 | 1 | 3 | 1 | 1 | 2 | 1 | 1 | 1 | 2 | 1 | 2

RQ2: Is hyper-parameter optimization enough to curb the aforementioned
optimizer brittleness?

Here we check of brittleness can be solved by tuning the partitioning methods.

Since all the algorithms in our study were tuned by hyperopt, the results of
Table 5 show that even post-tuning, there is a large brittleness problem.
Accordingly, we say:

Hyper-parameter optimization does not suffice to fix performance brittleness.

Since hyper-parameter optimization is not enough to tame brittleness, we ask:

RQ3: How else might we fix the aforementioned brittleness problem?

When we added the novel loss function (discussed above in §3.6.3), we obtained
the results shown in the last line of the two tables of Table 5(b). When
measured in terms of ranks seen amongst the different populations, the last
line of Table 5(b) is most insightful. Here we see that in general, DEEPLY
performs better compared to all other optimizers across all our case studies.
Hence we say:

Weighted losses together with hyper-parameter optimization fix the brittleness
problem.

Figure 2: Analysis of partitions generated by our approach. Distribution of
class size percentages across datasets (median results).

RQ4: Do we generate “dust" or “boulders"?

Two anti-patterns of partitioning are “dust” (generating too many partitions)
and “boulders” (generating too few partitions).

To analyze this, Figure 2 shows the distribution of partition sizes seen in 30
random samples. From that figure we say that our methods usually generate
small number of partitions. Also, since we are usually dealing with classes
ranging from 33 in acmeair to 109 in daytrader, Figure 2 tells us that we are
not generating partitions with only one item per partition (for example, on
plants, the median size is $5.4\%\times 111=6$ classes). Hence we say:

Our partitions are neither “boulders” nor “dust”.

##  6 Discussion

###  6.1 Lessons Learned

Yedida et al. [41] listed four lessons learned from their study. Our results
suggest that some of those lessons now need to be revised.

  1. 1.

We fully concur with Yedida et al. when they said “do not use partitioning
methods off-the-shelf since these these tend to have subpar results. That
said, currently we are planning to package (in Python) the DEEPLY system. Once
that is available then Algorithm 1 should accomplish much of the adjustments
required to commission DEEPLY for a new domain.

  2. 2.

Yedida et al. said “the choice of tuning algorithm is not as important as the
decision to tune. The random and hyperopt optimizers performed differently on
different metrics and datasets, but in a broader view, shared the same number
of highest statistical rankings. Therefore, the decision to tune is important,
but the choice of algorithm may not be as important.”. We disagree since, as
shown by Table 5(b), our methods demonstrate overall better results (as
indicated by the statistical ranks).

  3. 3.

Yedida et al. said “More research is required on tuning. While our results
shows that tuning is better than non-tuning, we also show that no single
approach is a clear winner across multiple datasets or multiple performance
goals.”. We concur but add that DEEPLY seems to resolve much of their
pessimism about the state of the art in hyper-parameter optimization.

  4. 4.

Yedida et al. said “There is no best partitioning algorithm… there is no one
best algorithm across all metrics.” We agree, but only partially. The last row
of Table 5(b) shows that even our preferred method DEEPLY does not always
obtain top rank. That said, DEEPLY often performs as well or better than
anything else.

###  6.2 On the value of dynamic traces

It is notable that one of the prior approaches we choose to compare against is
Bunch and its hill-climbing algorithm. Several years ago, an extensive
empirical study by Garcia et al. [15] demonstrated that Bunch performs poorly
in identifying meaningful abstractions in real-world systems it analyzes.
However, we still chose Bunch as a comparative system, since that study used
static attributes as the input to Bunch, while our inputs come from dynamic,
runtime traces, which we believe makes it meaningfully different. That is, we
use the results of Garcia et al. [15] as an indication that static traces may
not be sufficient for the microservice partitioning problem, and this study
explores dynamic traces instead.

###  6.3 Business Implications

At a higher level, hyper-parameter tuning is akin to searching for various
options and guessing which one is the best; weighted loss functions directly
encode business goals in this process, making it a more focused effort. For
example, changing business goals can be trivially implemented in our
framework, since the weighted loss function used in the hyper-parameter tuning
is based on the metrics (i.e., no additional mathematical derivations need to
be done).

Importantly, business users can (a) make a choice among many from the frontier
of best solutions of a size that they can choose, with the understanding that
higher sizes mean more options but takes slightly longer (b) see the trade-
offs between different architectural design choices directly in terms of
metrics (c) easily choose new metrics if they see fit. That is, our approach
provides businesses with flexibility and transparency into the working of the
model. While the deep learner itself is a black box, end users typically do
not care about the internal workings of the model; only in interpretable ways
to do better in their goals (this was studied by Chen et al. [10]).

For businesses, we offer the following advantages:

  * •

Stability: Business can be assured that our approach will perform well across
any dataset, no matter the metric used. This stability is important for time-
constrained businesses who need some guarantees of good results before using a
new approach.

  * •

Performance: Our approach achieved state-of-the-art performance across
different and uncorrelated metrics on four open-source datasets. This high
performance inspires confidence in our approach for businesses looking to
adopt an automated system for the architectural change.

  * •

Openness: Our code is fully open-source, but also modular. That is, businesses
are not limited to our specific approach (which builds upon CO-GCN and uses
hyperopt); rather, they are free to use our techniques to build upon any
existing infrastructure that they may have (e.g., IBM Mono2Micro).

###  6.4 Research Implications

The ideas of this paper extend to other tasks and domains. In this section, we
elaborate on the specific ideas and their broader utility.

Our approach is a general method that can be applied to any dataset, since the
components themselves can be changed. For example, a different hyper-parameter
optimizer such as Optuna ([2]) can be used instead of hyperopt, and a
different weighting mechanism can be chosen instead of our correlation-based
weights. Finally, a different set of preprocessing can be used. Therefore, a
generalized version of our approach is:

  1. (1)

Feature extraction. Different features sets can be extracted from code. For
example, one might use code2vec Alon et al. [4], which transforms code
snippets to fixed-length vectors.

  2. (2)

Hyper-parameter optimization. Any hyper-parameter optimization approach can be
plugged in here, such as random sampling, TPE, DODGE, etc.

  3. (3)

Loss-based selection. Any loss function can be applied to the frontier of best
solutions to select a configuration. Moreover, the loss function of the
learner itself can be modified, as done in CO-GCN.

From a research standpoint, we offer the following:

  * •

Advancing the state-of-the-art: Our approach consistently outperforms all the
prior state-of-the-art approaches we tested against across three different
metrics, on four datasets.

  * •

Modular approach: Our approach can be adapted by changing the base algorithm
(CO-GCN) and hyper-parameter optimization algorithm (hyperopt) and used on any
dataset as discussed above.

  * •

Documenting the success of weighted losses: Our paper adds to the body of
literature that documents the success of using weighted losses for different
problems, possibly motivating future work to also use them. More generally,
our idea can be applied to any multi-objective optimization problem, using the
general method from our paper: produce a frontier of best solutions (through
hyper-parameter optimization, swarm optimization, etc.) and use a weighted
loss to choose the best candidate. This idea has been used implicitly in the
deep learning field. In particular, Santurkar et al. [32], Li et al. [24] show
that a smoother loss function surface is beneficial for optimization. In the
case of Santurkar et al. [32], the adding of “skip-connections" in the deep
learner modify its loss function, making it smoother and therefore easier to
optimize in. This idea has also been used by Chaudhari et al. [9], who change
the loss function to shape it into a more amenable form.

##  7 Threats to Validity

Sampling bias: With any data mining paper, it is important to discuss sampling
bias. Our claim is that by evaluating on four different open-source projects
across different metrics, we mitigate this. Nevertheless, it would be
important future work to explore this line of research over more data.

Evaluation bias: While we initially considered comparing across all the
metrics we found in the literature, we noticed large correlations between
those metrics. Therefore, to reduce the effect of correlations, we chose a
subset of the metrics and evaluated all the approaches across those. Moreover,
in comparing the approaches, we tuned all of them using the same hyper-
parameter optimization algorithm, for the same number of iterations.

External validity: We tune the hyper-parameters of the algorithm, removing
external biases from the approach. Our baselines are also tuned using hyperopt
for the same number of iterations.

Internal validity: All algorithms compared were tuned using hyperopt. Because
our approach involves using weighted losses and other tweaks to CO-GCN, these
w ere not applied to the other algorithms.

##  8 Conclusion

In this paper, we presented a systematic approach for achieving state-of-the-
art results in microservice partitioning. Our approach consists of hyper-
parameter optimization and the use of weighted losses to choose a
configuration from the frontier of best solutions.

Broadly, the lesson from this work is:

At least for microservice partitioning, weighted loss functions can work
together with tuning to achieve superior results.

We first analyzed the existing state-of-the-art. Through this review, we
noticed (a) highly correlated, and therefore redundant, metrics used in the
literature (b) inconsistent comparisons being made in prior work (c) prior
work showing performance brittleness across different goals and datasets. We
argued that this creates an issue for businesses looking to use a microservice
partitioning tool for internal software. Through Monte Carlo-style sampling,
we chose a reduced, less correlated set of metrics, and used those as data for
choosing weights for each goal, accounting for their different scales. We
built upon an existing tool, CO-GCN ([13]), to build DEEPLY in this paper,
which fixes the issues listed above. To the best of our knowledge, ours is the
first structured attempt at (a) reviewing the literature for a list of state-
of-the-art microservice partitioning approaches (b) comparing all of them on
the same datasets and the same metrics (c) fixing the performance brittleness
issue using weighted losses and tuning. Finally, we discussed the broader
impacts of the approach specified in this paper, generalizing the concept
beyond the specific case of microservice partitioning.

Our approach is extensible and modular, consistently outperforms other
approaches across datasets and metrics, and can easily be adapted to any
metric that an enterprise is interested in. Moreover, the two loss functions
(for the deep learner and the hyper-parameter optimization algorithm) can be
tweaked to suit business goals.

##  9 Future Work

Our approach being modular leads to several avenues of future work, which we
discuss in this section.

Because we apply weighted losses at the hyper-parameter optimization level, we
can apply the same approach using a different base algorithm than CO-GCN.
Specifically, we could build a Pareto frontier using a different state-of-the-
art algorithm and then use our weighted loss function to choose a “best"
candidate.

Further, it would be useful to explore our methods on more datasets and
metrics. In particular, it would be beneficial to test our methods on large
enterprise systems. In addition, businesses might be interested in how much
faster our system is compared to human effort.

Finally, our method (using weighted losses to guide exploration of the Pareto
frontiers) is a general method that is not specific to microservice
partitioning. Specifically, since our losses choose from a Pareto frontier
generated by a hyper-parameter optimizer, (a) the choice of optimizer is left
to the user (b) the application that the optimizer is applied to can be freely
changed. Therefore our methods might offer much added value to other areas
where hyper-parameter optimization has been applied.

## Acknowledgments

This research was partially funded by an IBM Faculty Award. The funding source
had no influence on the study design, collection and analysis of the data, or
in the writing of this report.

## References

  * Agrawal et al. [2019] Agrawal, A., Fu, W., Chen, D., Shen, X., & Menzies, T. (2019).  How to" dodge" complex software analytics.  IEEE Transactions on Software Engineering, .
  * Akiba et al. [2019] Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019).  Optuna: A next-generation hyperparameter optimization framework.  In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 2623–2631).
  * Al-Debagy & Martinek [2018] Al-Debagy, O., & Martinek, P. (2018).  A comparative review of microservices and monolithic architectures.  In 2018 IEEE 18th International Symposium on Computational Intelligence and Informatics (CINTI) (pp. 000149–000154).  IEEE.
  * Alon et al. [2019] Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019).  code2vec: Learning distributed representations of code.  Proceedings of the ACM on Programming Languages, 3, 1–29.
  * Arar & Ayan [2017] Arar, Ö. F., & Ayan, K. (2017).  A feature dependent naive bayes approach and its application to the software defect prediction problem.  Applied Soft Computing, 59, 197–209.
  * Bergstra et al. [2011] Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011).  Algorithms for hyper-parameter optimization.  In 25th annual conference on neural information processing systems (NIPS 2011).  volume 24.
  * Bergstra et al. [2013] Bergstra, J., Yamins, D., Cox, D. D. et al. (2013).  Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms.  In Proceedings of the 12th Python in science conference.  Citeseer volume 13.
  * Brown et al. [1998] Brown, W. H., Malveau, R. C., McCormick, H. W. S., & Mowbray, T. J. (1998).  AntiPatterns: refactoring software, architectures, and projects in crisis.  John Wiley & Sons, Inc.
  * Chaudhari et al. [2019] Chaudhari, P., Choromanska, A., Soatto, S., LeCun, Y., Baldassi, C., Borgs, C., Chayes, J., Sagun, L., & Zecchina, R. (2019).  Entropy-sgd: Biasing gradient descent into wide valleys.  Journal of Statistical Mechanics: Theory and Experiment, 2019, 124018.
  * Chen et al. [2019] Chen, J., Chakraborty, J., Clark, P., Haverlock, K., Cherian, S., & Menzies, T. (2019).  Predicting breakdowns in cloud services (with spike).  In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 916–924).
  * Daya et al. [2016] Daya, S., Van Duy, N., Eati, K., Ferreira, C., Glozic, D., Gucer, V., Gupta, M., Joshi, S., Lampkin, V., Martins, M. et al. (2016).  Microservices from Theory to Practice: Creating Applications in IBM Bluemix Using the Microservices Approach.  IBM Redbooks.  URL: https://books.google.com/books?id=eOZyCgAAQBAJ.
  * Deb et al. [2002] Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002).  A fast and elitist multiobjective genetic algorithm: Nsga-ii.  IEEE transactions on evolutionary computation, 6, 182–197.
  * Desai et al. [2021] Desai, U., Bandyopadhyay, S., & Tamilselvam, S. (2021).  Graph neural network to dilute outliers for refactoring monolith application.  arXiv preprint arXiv:2102.03827, .
  * Eder et al. [2012] Eder, S., Junker, M., Jürgens, E., Hauptmann, B., Vaas, R., & Prommer, K.-H. (2012).  How much does unused code matter for maintenance?  In 2012 34th International Conference on Software Engineering (ICSE) (pp. 1102–1111).  IEEE.
  * Garcia et al. [2013] Garcia, J., Ivkovic, I., & Medvidovic, N. (2013).  A comparative analysis of software architecture recovery techniques.  In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) (pp. 486–496).  IEEE.
  * Ghotra et al. [2015] Ghotra, B., McIntosh, S., & Hassan, A. E. (2015).  Revisiting the impact of classification techniques on the performance of defect prediction models.  In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (pp. 789–800).  IEEE volume 1.
  * Goodfellow et al. [2016] Goodfellow, I., Bengio, Y., & Courville, A. (2016).  Deep learning.  MIT press.
  * Hess & Kromrey [2004] Hess, M. R., & Kromrey, J. D. (2004).  Robust confidence intervals for effect sizes: A comparative study of cohen’sd and cliff’s delta under non-normality and heterogeneous variances.  In annual meeting of the American Educational Research Association (pp. 1–30).  Citeseer.
  * Jin et al. [2019] Jin, W., Liu, T., Cai, Y., Kazman, R., Mo, R., & Zheng, Q. (2019).  Service candidate identification from monolithic systems based on execution traces.  IEEE Transactions on Software Engineering, .
  * Kalia et al. [2021] Kalia, A. K., Xiao, J., Krishna, R., Sinha, S., Vukovic, M., & Banerjee, D. (2021).  Mono2micro: a practical and effective tool for decomposing monolithic java applications to microservices.  In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 1214–1224).
  * Kalia et al. [2020] Kalia, A. K., Xiao, J., Lin, C., Sinha, S., Rofrano, J., Vukovic, M., & Banerjee, D. (2020).  Mono2micro: an ai-based toolchain for evolving monolithic enterprise applications to a microservice architecture.  In Proceedings of the 28th ACM Joint Meeting on ESEC/FSE (pp. 1606–1610).
  * Kipf & Welling [2016] Kipf, T. N., & Welling, M. (2016).  Semi-supervised classification with graph convolutional networks.  arXiv preprint arXiv:1609.02907, .
  * Komer et al. [2014] Komer, B., Bergstra, J., & Eliasmith, C. (2014).  Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn.  In ICML workshop on AutoML (p. 50).  Citeseer volume 9.
  * Li et al. [2017] Li, H., Xu, Z., Taylor, G., Studer, C., & Goldstein, T. (2017).  Visualizing the loss landscape of neural nets.  arXiv preprint arXiv:1712.09913, .
  * Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017).  Focal loss for dense object detection.  In Proceedings of the IEEE international conference on computer vision (pp. 2980–2988).
  * Mazlami et al. [2017] Mazlami, G., Cito, J., & Leitner, P. (2017).  Extraction of microservices from monolithic software architectures.  In 2017 IEEE International Conference on Web Services (ICWS) (pp. 524–531).  IEEE.
  * Mitchell & Mancoridis [2006] Mitchell, B. S., & Mancoridis, S. (2006).  On the automatic modularization of software systems using the bunch tool.  IEEE Transactions on Software Engineering, 32, 193–208.
  * Nair & Hinton [2010] Nair, V., & Hinton, G. E. (2010).  Rectified linear units improve restricted boltzmann machines.  In Icml.
  * Rumelhart et al. [1986] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986).  Learning representations by back-propagating errors.  nature, 323, 533–536.
  * Russell & Norvig [2002] Russell, S., & Norvig, P. (2002).  Artificial intelligence: a modern approach, .
  * Ryu & Baik [2016] Ryu, D., & Baik, J. (2016).  Effective multi-objective naïve bayes learning for cross-project defect prediction.  Applied Soft Computing, 49, 1062–1077.
  * Santurkar et al. [2018] Santurkar, S., Tsipras, D., Ilyas, A., & Mądry, A. (2018).  How does batch normalization help optimization?  In Proceedings of the 32nd international conference on neural information processing systems (pp. 2488–2498).
  * Seong et al. [2018] Seong, S., Lee, Y., Kee, Y., Han, D., & Kim, J. (2018).  Towards flatter loss surface via nonmonotonic learning rate scheduling.  In UAI2018 Conference on Uncertainty in Artificial Intelligence.  Association for Uncertainty in Artificial Intelligence (AUAI).
  * Smith [2017] Smith, L. N. (2017).  Cyclical learning rates for training neural networks.  In 2017 IEEE winter conference on applications of computer vision (WACV) (pp. 464–472).  IEEE.
  * Smith & Topin [2017] Smith, L. N., & Topin, N. (2017).  Super-convergence: Very fast training of neural networks using large learning rates.  arXiv preprint arXiv:1708.07120, .
  * Smith & Topin [2019] Smith, L. N., & Topin, N. (2019).  Super-convergence: Very fast training of neural networks using large learning rates.  In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications (p. 1100612).  International Society for Optics and Photonics volume 11006.
  * Srivastava et al. [2014] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014).  Dropout: a simple way to prevent neural networks from overfitting.  The journal of machine learning research, 15, 1929–1958.
  * Tu & Nair [2018] Tu, H., & Nair, V. (2018).  Is one hyperparameter optimizer enough?  In Proceedings of the 4th ACM SIGSOFT International Workshop on Software Analytics (pp. 19–25).
  * Wolff [2016] Wolff, E. (2016).  Microservices: flexible software architecture.  Addison-Wesley Professional.
  * Xu et al. [2015] Xu, T., Jin, L., Fan, X., Zhou, Y., Pasupathy, S., & Talwadker, R. (2015).  Hey, you have given me too many knobs!: Understanding and dealing with over-designed configuration in system software.  In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering ESEC/FSE 2015 (p. 307–319).  New York, NY, USA: Association for Computing Machinery.  URL: https://doi.org/10.1145/2786805.2786852. doi:10.1145/2786805.2786852.
  * Yedida et al. [2021a] Yedida, R., Krishna, R., Kalia, A., Menzies, T., Xiao, J., & Vukovic, M. (2021a).  Lessons learned from hyper-parameter tuning for microservice candidate identification.  In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) (pp. 1141–1145).  IEEE.
  * Yedida & Menzies [2021] Yedida, R., & Menzies, T. (2021).  On the value of oversampling for deep learning in software defect prediction.  IEEE Transactions on Software Engineering, .
  * Yedida & Saha [2021] Yedida, R., & Saha, S. (2021).  Beginning with machine learning: a comprehensive primer.  The European Physical Journal Special Topics, (pp. 1–82).
  * Yedida et al. [2021b] Yedida, R., Saha, S., & Prashanth, T. (2021b).  Lipschitzlr: Using theoretically computed adaptive learning rates for fast convergence.  Applied Intelligence, 51, 1460–1478.