# Sage: Leveraging ML to Diagnose Unpredictable Performance in Cloud
Microservices

1Yu Gan  1Mingyu Liang  2Sundar Dev  2David Lo  and 1Christina Delimitrou
1Cornell University  2Google
Contact author<EMAIL_ADDRESS>
###### Abstract

Cloud applications are increasingly shifting from large monolithic services,
to complex graphs of loosely-coupled microservices. Despite their advantages,
microservices also introduce cascading QoS violations in cloud applications,
which are difficult to diagnose and correct.

We present Sage, a ML-driven root cause analysis system for interactive cloud
microservices. Sage leverages unsupervised learning models to circumvent the
overhead of trace labeling, determines the root cause of unpredictable
performance online, and applies corrective actions to restore performance. On
experiments on both dedicated local clusters and large GCE clusters we show
that Sage achieves high root cause detection accuracy and predictable
performance.

##  1 Introduction

Cloud computing has reached proliferation by offering resource flexibility,
cost efficiency, and fast deployment [72, 85, 54]. As the cloud scale and
complexity increased, cloud services have undergone a major shift from large
monolithic designs to complex graphs of single-concerned, loosely-coupled
microservices. This shift is becoming increasingly pervasive, with large cloud
providers, such as Amazon, Twitter, Netflix, and eBay having already adopted
this application model [76, 75, 78]. Microservices are appealing for several
reasons, such as facilitating development, promoting elasticity, and enabling
software heterogeneity.

Despite their advantages, microservices also complicate resource management,
as dependencies between tiers introduce backpressure, causing unpredictable
performance to propagate through the system [82, 83]. Diagnosing such issues
empirically is cumbersome and prone to errors, especially as typical
microservices deployments include hundreds/thousands of tiers. Similarly,
current cluster managers [51, 53, 52, 54, 55, 56, 57, 101, 69, 85, 70, 71, 15,
93, 11]. are not expressive enough to account for the impact of dependencies,
putting more pressure on the need for automated root cause analysis systems.

Over the past few years, there has been increased attention on trace-based
methods to analyze [13], diagnose, and in some cases anticipate [60, 83]
performance issues in cloud services. While most of these systems target cloud
applications, the only one focusing on microservices is Seer [83], a DL-based
system that anticipates cases of unpredictable performance by inferring the
impact of outstanding requests on end-to-end performance. Despite its high
accuracy, Seer leverages supervised learning to anticipate QoS violations,
which require offline and online trace labeling. In a production system, this
is non-trivial, as it involves injecting resource contention in live
applications, hurting user experience.

We present Sage, a root cause analysis system for interactive microservices
that leverages unsupervised learning to identify the culprit of unpredictable
performance in complex graphs of microservices. Sage does not rely on data
labeling, hence it can be entirely transparent to both cloud users and
application developers, scales well with the number of microservices and
machines, and only relies on lightweight tracing that does not require
application changes or kernel instrumentaion. We have evaluated Sage both on
dedicated local clusters and large GCE settings and showed high root cause
detection accuracy and improved performance predictability.

##  2 ML for Performance Debugging

###  2.1 Overview

Sage is a performance debugging and root cause analysis system for large-scale
cloud applications. While the design centers around interactive microservices,
where dependencies between tiers are more impactful, Sage is also applicable
to traditional monolithic or SOA services. Sage relies on two broad
techniques, each of which is described in detail below; first, it uses Causal
Bayesian Networks (CBN) to model the RPC-level dependencies between
microservices, and the latency propagation from the backend to frontend.
Second, it uses a graphical variational auto-encoder (GVAE) to generate
examples of counterfactuals [22], and infer the hypothetical end-to-end
latency had some occurring events not happened. Using these two techniques,
Sage determines which set of microservices initiated an end-to-end QoS
violation, and moves to adjust deployment and/or resource allocation to
correct it.

###  2.2 Microservice Latency Propagation

Figure 1: The dependency graph and traces of nested RPCs.

Multiple RPCs between microservices form a tree of nested traces in a
distributed monitoring system. Figure 1 shows an example of the RPC dependency
graph containing five services, four RPCs, and its corresponding latency
traces.

The server-side latency of any non-leaf RPC is determined by the processing
time of the RPC itself and the waiting time (i.e., client-side latency) of its
child RPCs. The latency of any RPC will propagate through the RPC dependency
graph to the frontend and impact the end-to-end latency. Since the latency of
a child RPC cannot propagate to its parent without impacting its own latency,
the RPC latency propagation follows a local Markov property, where each
latency variable is conditionally independent on its non-ancestor RPC
latencies, given its child RPC latencies [29]. For instance, the latency of
RPC0 is conditionally independent on the latency of RPC2 and RPC3, given the
latency of RPC1.

###  2.3 Modeling Microservice Dependency Graphs

Causal Bayesian Networks (CBN) are a common tool to model causal relationships
[30, 31]. A CBN is a directed acyclic graph (DAG), where the nodes are
different random variables and the edges indicate their conditional
dependencies. More specifically, each edge represents a causal relationship,
directed from the cause to the effect. We define three types of nodes in the
Bayesian network:

  * •

Metric nodes ($X$): The metric nodes contain primarily resource-related
metrics of all services and network channels, such as CPU and memory
utilization and network bandwidth, collected via the in-place monitoring tools
[19, 20, 21].

  * •

Latency nodes ($Y$): The latency nodes include client-side latency ($Y^{c}$),
server-side latency ($Y^{s}$), and request/response network delay ($Y^{req}$
and $Y^{resp}$) of all RPCs.

  * •

Latent variables ($Z$): The latent variables contain the unobserved and
immeasurable factors that are responsible for latency stochasticity. They are
critical to generate the counterfactual latencies Sage relies on to diagnose
root causes of QoS violations.

We can construct the CBN among the three-node classes of all RPCs based on the
inherent causal relationships and latency propagation observations obtained
via a distributed tracing system, such as Dapper or Zipkin.

Figure 2 shows an example of the CBN constructed for a three-microser-vice
chain based on the RPC dependency graph. The nodes with solid lines ($X$ and
$Y$) are observed, while the nodes with dashed lines ($Z$) are latent
variables that need to be inferred. The arrows in the RPC dependency graph and
CBN have opposite directions because the latency of one RPC is determined by
the latency of its child RPCs.

###  2.4 Counterfactual Queries

In a typical cloud environment, site reliability engineers (SREs) can verify
if a suspected root cause is correct by reverting a microservice’s
configuration to a state known to be safe, while keeping the remaining
microservices unchanged. If the problem is resolved, the suspected culprit is
causally related to the QoS violation. Sage uses a similar process, where
“suspected root causes” are generated using counterfactuals, which determine
the causal effect by asking what the outcome would be if the state of a
microservice had been different [32, 33, 23].

Figure 2: Latency propagation CBN.

To avoid impacting the performance of live services during this process, Sage
leverages historical tracing data to generate realistic counterfactuals [31,
32], taking into account that the exact situation may not have occurred in the
past. If the probability that the end-to-end tail latency meets QoS after
intervention is greater than a threshold, then those services are the root
cause of the performance issue.

Conditional deep generative models, such as the conditional variational
autoencoders (CVAE) [34] and conditional generative adversarial nets, are
common tools to generate new data of a class from an original distribution.
Generally, they compress a high-dimensional target ($Y$) and tag ($X$) into
low-dimensional latent space variables ($Z$), and use the latent space
variables and tags to generate new target data. Recent studies have showed
that these techniques can also be used to generate counterfactuals for causal
inference [35, 36].

To generate counterfactuals, we build a network of CVAEs according to the
structure of the CBN. Although using one CVAE for the entire microservice
graph would be simple, it has several drawbacks. First, it lacks the CBN’s
structural information which is useful in terms of avoiding ineffectual
counterfactuals based on spurious correlations. Second, it prohibits partial
retraining and transfer learning, which is essential given the frequent update
cadence of microservices. Finally, the black-box model is less explainable
since it cannot reveal any information on how the latency of a problematic
service propagates to the frontend. Therefore, we construct one lightweight
CVAE per microservice, and connect the different CVAEs according to the
structure of the CBN to form the graphical variational autoencoder (GVAE).

The encoders and prior networks take the observed metrics as inputs, and are
trained in parallel. The decoders require the outputs of the parent decoders
in the CBN as inputs, and are trained serially. The maximum depth of the CBN
determines the maximum number of serially-cascaded decoders.

##  3 Sage Design

Figure 3: Overview of Sage’s system design.

Sage is a root cause analysis system for complex graphs of interactive
microservices. Sage relies on RPC-level tracing to compose the Causal Bayesian
network of the microservice topology, and per-node tracing to track the per-
tier latency distribution. Below we discuss the training and inference
pipeline (Sec. 3.1), Sage’s actuation system once a root cause has been
identified (Sec. 5.2), and the way Sage handles changes in application design
(Sec. 3.3).

Fig. 3 shows an overview of Sage. The system uses Jaeger [42], a distributed
RPC-level tracing system to collect end-to-end request execution traces.
Jaeger uses sampling to reduce tracing oveheads. Sage also uses the Prometheus
Node Exporter [46], Prometheus Blackbox Exporter [45], and cAdvisor [80] to
collect machine- and container-level hardware/OS metrics, and network
latencies. Each metric’s timeseries is stored in the Prometheus TSDB [44]. At
runtime, Sage queries Jaeger and Prometheus periodically to obtain real-time
latency and usage metrics. The GVAE then uses this data to infer the root
cause of any QoS violation(s). Once a root cause is diagnosed, Sage’s actuator
takes action to restore performance by adjusting the offending microservice’s
resources.

Sage uses a centralized master for trace processing and root cause analysis,
and per-node agents for trace collection and container deployment. It also
maintains two hot stand-by copies of the Sage master for fault tolerance.

###  3.1 Root Cause Analysis

Sage first uses the Data Streamer to fetch and pre-process the latency and
usage statistics. Sage initializes and trains the GVAE model offline with all
initially available data. It then periodically retrains the model, even when
there are no changes in application design, to account for changes in user
behavior [37, 49, 36]. Online learning models are prone to catastrophic
forgetting, where the model forgets previously-learned knowledge upon learning
new information [37, 28]. To avoid this, we interleave the current and
previous data in the training batches. In addition, to avoid class imbalance,
i.e., cases where the datapoints that meet QoS are significantly more than
those which violate it, the model oversamples the minority classes to create a
more balanced training dataset.

At runtime, Sage uses the latest version of the GVAE model to infer the root
cause of QoS violations. Sage first calculates the medians of usage and
performance metrics where QoS is met, and labels them as normal values. If at
any point QoS is not met, the GVAE will generate counterfactuals by replacing
a microservice’s metrics with their respective normal values. The service
whose counterfactual would resolve the QoS violation is identified as the
culprit behind it.

Sage implements a two-level approach to locate the root cause of a QoS
violation. It first uses service-level counterfactuals to locate the culprit
microservices, and then uses metric-level counterfactuals of the offending
service to identify the underlying reason that caused it to become the
culprit.

###  3.2 Actuation

Once Sage determines the root cause of a QoS violation it takes action.
Depending on which resource is identified by the GVAE as the one instigating
the QoS violation, Sage will dynamically adjust the CPU frequency, scale up or
scale out the problematic microservices, limit the rate of the collocated
interference jobs, partition the last level cache (LLC) with Intel Cache
Allocation Technology (CAT), and partition the network bandwidth with Linux
traffic control queuing disciplines. Sage first tries to resolve the
performance issue by only adjusting resources on the offending node, and only
when that is insufficient it scales out the problematic microservice on new
nodes and/or migrate it.

###  3.3 Handling Microservice Updates

Training the complete model from scratch for large clusters takes tens of
minutes to hours, so it is impractical to happen for every change in
application design/deployment. Sage instead implements selective partial
retraining and incremental retraining with a dynamically reshapable GVAE
similar to [36], thanks to VAE’s ability to be decomposed using the CBN. On
the one hand, with selective partial retraining, we only retrain the neurons
corresponding to the updated nodes and their descendents in the CBN, because
the causal relationships guarantee that all other nodes are not affected by
the change. On the other hand, with incremental retraining, we initialize the
parameters of the network to those of the previous model, while
adding/removing/reshaping the corresponding networks if microservices are
added/dropped/updated. The combination of these two transfer learning
approaches reduces retraining time by more than $10\times$, especially when
there is large fanout in the RPC dependency graph.

##  4 Methodology

###  4.1 Cloud Services

Generic Thrift microservices:  Apache Thrift [98] is a popular RPC framework.
We implement a code generator for composable Thrift microservices, and
evaluate two topologies; a Chain and a Fanout. In Chain, each microservice
receives a request from its parent, sends it to its downstream service, and
responds to its parent once it gets the results from its child. In Fanout, the
root service broadcast each request to all leaf services and returns to the
client only after all children have responded. Most real microservice
architectures are combinations of these two topologies [38, 82, 39].

Social Network:  One of the end-to-end microservice in the DeathStarBench
suite[82] that implements a broadcast-style social network with uni-
directional follow relationships.

###  4.2 Systems

Local Cluster:  We use a dedicated local cluster with five 2-socket 40-core
servers with 128GB RAM each, and two 2-socket 88-core servers with 188GB RAM
each. Each server is connected to a 40Gbps ToR switch over 10Gbe NICs.

Google Compute Engine:  We also deploy the Social Network service to Google
Compute Engine (GCE) with 84 nodes in us-central1-a to study Sage’s
scalability. All nodes are dedicated, so there is no interference from
external jobs.

###  4.3 Training Dataset for Validation

We use wrk2 [79] as the open-loop HTTP workload generator for all three
applications. To verify the ground truth during validation, we use stress-ng
[48] and tc-netem [40] to inject CPU-, memory-, disk-, and network-intensive
microbenchmarks to different, randomly-chosen subsets of microservices.

##  5 Evaluation

Figure 4: Detection accuracy and false negatives/positives.

###  5.1 Sage Validation

Fig. 4 shows the accuracy of Sage in the local cluster across services,
compared to two autoscaling techniques, and an oracular scheme that sets
thresholds for each tier and metric offline, beyond which point resources are
upscaled. Autoscale Strict increases resource allocation when the utilization
of a microservice exceeds 50% and Autoscale Relaxed when it exceeds 70% (on
par with AWS’s autoscaling policy). Sage significantly outperforms the other
methods, even the offline oracular one, by learning the impact of dependencies
between neighboring microservices. Similarly, Sage’s false negative and false
positive rates are marginal, which avoids QoS violations and resource
inefficiencies respectively.

Figure 5: End-to-end latency for Social Network in the presence of QoS
violations.

###  5.2 Actuation

Fig. 5 shows the impact of root cause analysis on the end-to-end tail latency
of the Social Network, with Sage and the offline oracular technique. To create
unpredictable performance whose source is known, we inject resource intensive
kernels in a randomly-selected subset of microservices. While there are cases
that the threshold-based scheme identifies correctly, more often performance
takes a long time to recover. In comparison, Sage immediately identifies the
correct root cause, and applies corrective action to restore performance.

We have also introduced changes to several microservices and have validated
that the transfer learning in Sage reduces training time by at least an order
of magnitude compared to retraining from scratch, without impacting accuracy.

###  5.3 Scalability

We now evaluate Sage’s accuracy when deploying the Social Network on 188
instances on GCE using Docker Swarm. We replicate the stateless microservices
and shard the caching systems and databases across instances during periods of
higher load. Each service has 1-10 replicas, depending on the maximum single-
process throughput. Accuracy on GCE is within 1% of the detection accuracy on
the local cluster, while the difference in false positives and false negatives
is also marginal. We also evaluate the difference in training and inference
time between the local cluster and GCE. We use two Intel Xeon 6152 processors
with 44 cores in total for training and inference. Training from scratch takes
124 minutes for the local cluster and 148 minutes for GCE. Inference takes
49ms on the local cluster and 62ms on GCE. Although we deploy 6.7x more
containers on GCE than on the local cluster, the training and inference time
only increase by 19.4% and 26.5% respectively.

##  6 Conclusions

We have presented Sage, an ML-driven root cause analysis system for
interactive, cloud microservices. Sage leverages entirely unsupervised models
to detect the sources of unpredictable performance, removing the need for
empirical diagnosis or expensive data labeling. Sage adapts to frequent design
changes, and takes action to restore QoS. We shows that Sage achieves high
root cause detection accuracy and improved performance. Given the increasing
complexity of cloud services, data-driven systems like Sage can improve
performance predictability without sacrificing efficiency.

## Acknowledgements

We sincerely thank Daniel Sanchez and the anonymous reviewers for their
feedback on earlier versions of this manuscript. This work was in part
supported by NSF grants NeTS CSR-1704742, CCF-1846046, a Google Faculty Award,
a Microsoft Research Faculty Fellowship, and a John and Norma Balen
Sesquisentennial Faculty Fellowship.

## References

  * [1] Decomposing twitter: Adventures in service-oriented architecture.  https://www.slideshare.net/InfoQ/decomposing-twitter-adventures-in-serviceoriented-architecture.
  * [2] Docker containers.  https://www.docker.com/.
  * [3] Locust.  https://locust.io/.
  * [4] Step and simple scaling policies for amazon ec2 auto scaling.  https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-simple-step.html.
  * [5] Why grpc?  https://grpc.io/.
  * [6] The evolution of microservices.  https://www.slideshare.net/adriancockcroft/evolution-of-microservices-craft-conference, 2016\.
  * [7] Microservices workshop: Why, what, and how to get there.  http://www.slideshare.net/adriancockcroft/microservices-workshop-craft-conference.
  * [8] Autoscale.  https://cwiki.apache.org/cloudstack/autoscaling.html.
  * [9] Aws autoscaling.  http://aws.amazon.com/autoscaling/.
  * [10] Jeffrey Chase, Darrell Anderson, Prachi Thakar, Amin Vahdat, and Ronald Doyle.  Managing energy and server resources in hosting centers.  In Proceedings of SOSP. Banff, CA, 2001.
  * [11] Shuang Chen, Christina Delimitrou, and José F Martínez.  Parties: Qos-aware resource partitioning for multiple interactive services.  In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 107–120. ACM, 2019.
  * [12] F. C. Eigler, V. Prasad, W. Cohen, H. Nguyen, M. Hunt, J. Keniston, and B. Chen, “Architecture of systemtap: a linux trace/probe tool,” 2005.
  * [13] R. Fonseca, G. Porter, R. H. Katz, S. Shenker, and I. Stoica, “X-trace: A pervasive network tracing framework,” in _Proceedings of the 4th USENIX Conference on Networked Systems Design & Implementation_, ser. NSDI’07. Berkeley, CA, USA: USENIX Association, 2007, pp. 20–20.
  * [14] Y. Gan, M. Pancholi, D. Cheng, S. Hu, Y. He, and C. Delimitrou, “Seer: Leveraging Big Data to Navigate the Complexity of Cloud Debugging,” in _Proceedings of the Tenth USENIX Workshop on Hot Topics in Cloud Computing (HotCloud)_ , July 2018.
  * [15] H. Yang, A. Breslow, J. Mars, and L. Tang, “Bubble-flux: precise online qos management for increased utilization in warehouse scale computers,” in _Proceedings of ISCA_. 2013.
  * [16] Tianqi Chen and Carlos Guestrin.  XGBoost: A scalable tree boosting system.  In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 785–794, New York, NY, USA, 2016. ACM.
  * [17] Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang.  Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems.  CoRR, abs/1512.01274, 2015.
  * [18] Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich, Marcus Fontoura, and Ricardo Bianchini.  Resource central: Understanding and predicting workloads for improved resource management in large cloud platforms.  In Proceedings of the 26th Symposium on Operating Systems Principles, pages 153–167. ACM, 2017.
  * [19] G. Cloud, _Cloud Monitoring documentation_ , 2020. [Online]. Available: https://cloud.google.com/monitoring/docs/apis
  * [20] A. W. Services, _Amazon CloudWatch User Guide Document History_ , 2020. [Online]. Available: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring
  * [21] M. Azure, _Azure Monitor documentation_ , 2020. [Online]. Available: https://docs.microsoft.com/en-us/azure/azure-monitor/
  * [22] C. Hitchcock, “Probabilistic causation,” in _The Stanford Encyclopedia of Philosophy_ , fall 2018 ed., E. N. Zalta, Ed. Metaphysics Research Lab, Stanford University, 2018.
  * [23] M. Höfler, “Causal inference based on counterfactuals,” _BMC medical research methodology_ , vol. 5, no. 1, p. 28, 2005.
  * [24] S. C. Hoi, D. Sahoo, J. Lu, and P. Zhao, “Online learning: A comprehensive survey,” _arXiv preprint arXiv:1802.02871_ , 2018.
  * [25] O. Ibidunmoye, F. Hernández-Rodriguez, and E. Elmroth, “Performance anomaly detection and bottleneck identification,” _ACM Comput. Surv._ , vol. 48, no. 1, Jul. 2015.
  * [26] O. Ivanov, M. Figurnov, and D. Vetrov, “Variational autoencoder with arbitrary conditioning,” 2018.
  * [27] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [28] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska _et al._ , “Overcoming catastrophic forgetting in neural networks,” _Proceedings of the national academy of sciences_ , vol. 114, no. 13, pp. 3521–3526, 2017.
  * [29] D. Koller and N. Friedman, _Probabilistic graphical models: principles and techniques_. MIT press, 2009.
  * [30] R. E. Neapolitan _et al._ , _Learning bayesian networks_. Pearson Prentice Hall Upper Saddle River, NJ, 2004, vol. 38.
  * [31] J. Pearl _et al._ , “Causal inference in statistics: An overview,” _Statistics surveys_ , vol. 3, pp. 96–146, 2009.
  * [32] S. L. Morgan and C. Winship, _Counterfactuals and causal inference_. Cambridge University Press, 2015.
  * [33] P. Menzies, “Counterfactual theories of causation,” _Stanford Encyclopedia of Philosophy_ , 2008.
  * [34] K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” in _Advances in Neural Information Processing Systems 28_ , C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds. Curran Associates, Inc., 2015, pp. 3483–3491.
  * [35] C. Louizos, U. Shalit, J. M. Mooij, D. Sontag, R. Zemel, and M. Welling, “Causal effect inference with deep latent-variable models,” in _Advances in Neural Information Processing Systems_ , 2017, pp. 6446–6456.
  * [36] J. Yoon, E. Yang, J. Lee, and S. J. Hwang, “Lifelong learning with dynamically expandable networks,” in _International Conference on Learning Representations_ , 2018. [Online]. Available: https://openreview.net/forum?id=Sk7KsfW0-
  * [37] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” _Neural Networks_ , 2019\.
  * [38] R. S. Kannan, L. Subramanian, A. Raju, J. Ahn, J. Mars, and L. Tang, “Grandslam: Guaranteeing slas for jobs in microservices execution frameworks,” in _Proceedings of the Fourteenth EuroSys Conference 2019_ , ser. EuroSys ’19. New York, NY, USA: Association for Computing Machinery, 2019.
  * [39] A. Sriraman and T. F. Wenisch, “$\mu$ suite: A benchmark suite for microservices,” in _2018 IEEE International Symposium on Workload Characterization (IISWC)_ , 2018, pp. 1–12.
  * [40] “tc-netem(8) - linux manual page,” http://man7.org/linux/man-pages/man8/tc-netem.8.html.
  * [41] “Inside azure search: Chaos engineering,” https://azure.microsoft.com/en-us/blog/inside-azure-search-chaos-engineering/.
  * [42] “Jaeger: open source, end-to-end distributed tracing,” https://www.jaegertracing.io/.
  * [43] “Opentracing,” https://opentracing.io/.
  * [44] “Prometheus,” https://prometheus.io/.
  * [45] “prometheus/blackbox_exporter,” https://github.com/prometheus/blackbox_exporter.
  * [46] “prometheus/node_exporter,” https://github.com/prometheus/node_exporter.
  * [47] “Spring framework,” https://spring.io/projects/spring-framework.
  * [48] “stress-ng,” https://wiki.ubuntu.com/Kernel/Reference/stress-ng.
  * [49] S. C. Hoi, D. Sahoo, J. Lu, and P. Zhao, “Online learning: A comprehensive survey,” _arXiv preprint arXiv:1802.02871_ , 2018.
  * [50] Joan Daemen and Vincent Rijmen.  Aes proposal: Rijndael.  1999\.
  * [51] Christina Delimitrou and Christos Kozyrakis.  Paragon: QoS-aware scheduling for heterogeneous datacenters.  In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 2013.
  * [52] Christina Delimitrou and Christos Kozyrakis.  QoS-Aware Scheduling in Heterogeneous Datacenters with Paragon.  In ACM Transactions on Computer Systems (TOCS), Vol. 31 Issue 4. 2014.
  * [53] Christina Delimitrou and Christos Kozyrakis.  QoS-Aware Admission Control in Heterogeneous Datacenters.  In Proceedings of the International Conference of Autonomic Computing (ICAC). 2013.
  * [54] Christina Delimitrou and Christos Kozyrakis.  Quasar: Resource-Efficient and QoS-Aware Cluster Management.  In Proceedings of the Nineteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). Salt Lake City, UT, USA, 2014.
  * [55] Christina Delimitrou and Daniel Sanchez and Christos Kozyrakis.  Tarcil: Reconciling Scheduling Speed and Quality in Large Shared Clusters.  In Proceedings of the Sixth ACM Symposium on Cloud Computing (SOCC). 2015.
  * [56] Christina Delimitrou and Christos Kozyrakis.  HCloud: Resource-Efficient Provisioning in Shared Cloud Systems.  In Proceedings of the Twenty First International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 2016.
  * [57] Christina Delimitrou and Christos Kozyrakis.  Bolt: I Know What You Did Last Summer… In The Cloud.  In Proceedings of the Twenty Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 2017.
  * [58] Yu Gan, Meghna Pancholi, Dailun Cheng, Siyuan Hu, Yuan He, and Christina Delimitrou.  Seer: leveraging big data to navigate the complexity of cloud debugging.  In HotCloud, 2018.
  * [59] Yu Gan, Meghna Pancholi, Dailun Cheng, Siyuan Hu, Yuan He, and Christina Delimitrou.  Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices.  In Proceedings of the Twenty Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2019.
  * [60] Yu Gan and Christina Delimitrou.  The architectural implications of cloud microservices.  In Computer Architecture Letters, 2018.
  * [61] Christina Delimitrou and Christos Kozyrakis.  QoS-Aware Scheduling in Heterogeneous Datacenters with Paragon.  In ACM Transactions on Computer Systems (TOCS). 2014.
  * [62] Peter J Denning.  The working set model for program behavior.  Communications of the ACM, 11(5):323–333, 1968.
  * [63] Yu Gan, Meghna Pancholi, Dailun Cheng, Siyuan Hu, Yuan He, and Christina Delimitrou.  Seer: leveraging big data to navigate the complexity of cloud debugging.  In Proceedings of the 10th USENIX Conference on Hot Topics in Cloud Computing, pages 13–13. USENIX Association, 2018.
  * [64] Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon Zaruvinsky, Mateo Espinosa, Yuan He, and Christina Delimitrou.  An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems.  In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 3–18. ACM, 2019.
  * [65] John Gittins, Kevin Glazebrook, and Richard Weber.  Multi-armed bandit allocation indices.  John Wiley & Sons, 2011.
  * [66] Kristina Gligorić, Ashton Anderson, and Robert West.  How constraints affect content: The case of twitter’s switch from 140 to 280 characters.  In Twelfth International AAAI Conference on Web and Social Media, 2018.
  * [67] Ben Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica.  Mesos: A platform for fine-grained resource sharing in the data center.  In Proceedings of NSDI. Boston, MA, 2011.
  * [68] Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon.  What is twitter, a social network or a news media?  In Proceedings of the 19th international conference on World wide web, pages 591–600. AcM, 2010.
  * [69] Ching-Chi Lin, Pangfeng Liu, and Jan-Jan Wu.  Energy-aware virtual machine dynamic provision and scheduling for cloud computing.  In Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing (CLOUD). Washington, DC, USA, 2011.
  * [70] David Lo, Liqun Cheng, Rama Govindaraju, Luiz André Barroso, and Christos Kozyrakis.  Towards energy proportionality for large-scale latency-critical workloads.  In Proceedings of the 41st Annual International Symposium on Computer Architecuture (ISCA). Minneapolis, MN, 2014.
  * [71] David Lo, Liqun Cheng, Rama Govindaraju, Parthasarathy Ranganathan, and Christos Kozyrakis.  Heracles: Improving resource efficiency at scale.  In Proc. of the 42Nd Annual International Symposium on Computer Architecture (ISCA). Portland, OR, 2015.
  * [72] Luiz Barroso and Urs Hoelzle and Parthasarathy Ranganathan  The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition.  In Synthesis Lectures on Computer Architecture October 2018.
  * [73] J. Dean and L. A. Barroso, “The tail at scale,” in _CACM, Vol. 56 No. 2_.
  * [74] “Zipkin,” http://zipkin.io.
  * [75] “The evolution of microservices,” https://www.slideshare.net/adriancockcroft/evolution-of-microservices-craft-conference, 2016\.
  * [76] “Microservices workshop: Why, what, and how to get there,” http://www.slideshare.net/adriancockcroft/microservices-workshop-craft-conference.
  * [77] “Apache thrift,” https://thrift.apache.org.
  * [78] “Decomposing twitter: Adventures in service-oriented architecture,” https://www.slideshare.net/InfoQ/decomposing-twitter-adventures-in-serviceoriented-architecture.
  * [79] “giltene/wrk2,” https://github.com/giltene/wrk2.
  * [80] “google/cadvisor,” https://github.com/google/cadvisor.
  * [81] “grpc: A high performance open-source universal rpc framework,” https://grpc.io/.
  * [82] Y. Gan, Y. Zhang, D. Cheng, A. Shetty, P. Rathi, N. Katarki, A. Bruno, J. Hu, B. Ritchken, B. Jackson, K. Hu, M. Pancholi, Y. He, B. Clancy, C. Colen, F. Wen, C. Leung, S. Wang, L. Zaruvinsky, M. Espinosa, R. Lin, Z. Liu, J. Padilla, and C. Delimitrou, “An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud and Edge Systems,” in _Proceedings of the Twenty Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)_ , April 2019.
  * [83] Y. Gan, Y. Zhang, K. Hu, Y. He, M. Pancholi, D. Cheng, and C. Delimitrou, “Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices,” in _Proceedings of the Twenty Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)_ , April 2019.
  * [84] Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean.  Boosting algorithms as gradient descent.  In Proceedings of the 12th International Conference on Neural Information Processing Systems, NIPS’99, pages 512–518, Cambridge, MA, USA, 1999\. MIT Press.
  * [85] David Meisner, Christopher M. Sadler, Luiz André Barroso, Wolf-Dietrich Weber, and Thomas F. Wenisch.  Power management of online data-intensive services.  In Proceedings of the 38th annual international symposium on Computer architecture, pages 319–330, 2011.
  * [86] Kay Ousterhout, Patrick Wendell, Matei Zaharia, and Ion Stoica.  Sparrow: Distributed, low latency scheduling.  In Proceedings of SOSP. Farminton, PA, 2013.
  * [87] Chenhao Qu, Rodrigo N Calheiros, and Rajkumar Buyya.  Auto-scaling web applications in clouds: A taxonomy and survey.  ACM Computing Surveys (CSUR), 51(4):73, 2018.
  * [88] Joy Rahman and Palden Lama.  Predicting the end-to-end tail latency of containerized microservices in the cloud.  In IEEE International Conference on Cloud Engineering, IC2E 2019, Prague, Czech Republic, June 24-27, 2019, pages 200–210. IEEE, 2019\.
  * [89] Charles Reiss, Alexey Tumanov, Gregory Ganger, Randy Katz, and Michael Kozych.  Heterogeneity and dynamicity of clouds at scale: Google trace analysis.  In Proceedings of SOCC. 2012.
  * [90] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  "why should I trust you?": Explaining the predictions of any classifier.  In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135–1144, 2016.
  * [91] Ryan A. Rossi and Nesreen K. Ahmed.  The network data repository with interactive graph analytics and visualization.  In AAAI, 2015.
  * [92] Krzysztof Rzadca, Pawel Findeisen, Jacek Swiderski, Przemyslaw Zych, Przemyslaw Broniek, Jarek Kusmierek, Pawel Nowak, Beata Strack, Piotr Witusowski, Steven Hand, and John Wilkes.  Autopilot: workload autoscaling at google.  In Proceedings of the Fifteenth European Conference on Computer Systems, pages 1–16, 2020.
  * [93] Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, and John Wilkes.  Omega: flexible, scalable schedulers for large compute clusters.  In Proceedings of EuroSys. Prague, Czech Republic, 2013.
  * [94] Zhiming Shen, Sethuraman Subbiah, Xiaohui Gu, and John Wilkes.  Cloudscale: elastic resource scaling for multi-tenant cloud systems.  In Proceedings of SOCC. Cascais, Portugal, 2011.
  * [95] Akshitha Sriraman and Thomas F. Wenisch.  µtune: Auto-tuned threading for OLDI microservices.  In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 177–194, Carlsbad, CA, October 2018. USENIX Association.
  * [96] Akshitha Sriraman and Thomas F Wenisch.  usuite: A benchmark suite for microservices.  In 2018 IEEE International Symposium on Workload Characterization (IISWC), pages 1–12. IEEE, 2018.
  * [97] Lalith Suresh, Peter Bodik, Ishai Menache, Marco Canini, and Florin Ciucu.  Distributed resource management across process boundaries.  In Proceedings of the 2017 Symposium on Cloud Computing, pages 611–623. ACM, 2017.
  * [98] Apache thrift.  https://thrift.apache.org.
  * [99] Torque resource manager.  http://www.adaptivecomputing.com/products/open-source/torque/.
  * [100] Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer, and Asser Tantawi.  An analytical model for multi-tier internet services and its applications.  SIGMETRICS Perform. Eval. Rev., 33(1):291–302, June 2005.
  * [101] Abhishek Verma, Luis Pedrosa, Madhukar R. Korupolu, David Oppenheimer, Eric Tune, and John Wilkes.  Large-scale cluster management at Google with Borg.  In Proceedings of the European Conference on Computer Systems (EuroSys), Bordeaux, France, 2015.
  * [102] Hailong Yang, Quan Chen, Moeiz Riaz, Zhongzhi Luan, Lingjia Tang, and Jason Mars.  Powerchief: Intelligent power allocation for multi-stage applications to improve responsiveness on power constrained cmp.  In Proceedings of the 44th Annual International Symposium on Computer Architecture, ISCA ’17, page 133–146, New York, NY, USA, 2017. Association for Computing Machinery.
  * [103] Hao Zhou, Ming Chen, Qian Lin, Yong Wang, Xiaobin She, Sifan Liu, Rui Gu, Beng Chin Ooi, and Junfeng Yang.  Overload control for scaling wechat microservices.  In Proceedings of the ACM Symposium on Cloud Computing, pages 149–161. ACM, 2018.