1

# SCOPE: Safe Exploration for Dynamic Computer Systems Optimization

Hyunji Kim  MIT EECSCambridge, MAUSA<EMAIL_ADDRESS>,  Ahsan Pervaiz
University of ChicagoChicago, ILUSA<EMAIL_ADDRESS>,  Henry Hoffmann
University of ChicagoChicago, ILUSA<EMAIL_ADDRESS>,  Michael
Carbin  MIT CSAILCambridge, MAUSA<EMAIL_ADDRESS>and  Yi Ding  MIT
CSAILCambridge, MAUSA<EMAIL_ADDRESS>
###### Abstract.

Modern computer systems need to execute under strict safety constraints (e.g.
a power limit), but doing so often conflicts with their ability to deliver
high performance (i.e. minimal latency). Prior work uses machine learning to
automatically tune hardware resources such that the system execution meets
safety constraints optimally. Such solutions monitor past system executions to
learn the system’s behavior under different hardware resource allocations
before dynamically tuning resources to optimize the application execution.
However, system behavior can change significantly between different
applications and even different inputs of the same applications. Hence, the
models learned using data collected a priori are often suboptimal and violate
safety constraints when used with new applications and/or inputs.

To address this limitation, we introduce the concept of an _execution space_ ,
which is the cross product of hardware resources, input features, and
applications. Thus, a _configuration_ is defined as a tuple of hardware
resources, input features, and application. To dynamically and safely allocate
hardware resources from the execution space, we present SCOPE 111Safe
Configuration Optimization for Performance and Efficiency, a resource manager
that leverages a novel safe exploration framework. SCOPE operates iteratively,
with each iteration (i.e., reallocation) having three phases: monitoring, safe
space construction, and objective optimization. To construct a safe set with
high coverage (i.e., a high number of safe configurations in the predicted
safe set), SCOPE introduces a locality preserving operator so that SCOPE’s
exploration will rarely violate the safety constraint and have small magnitude
violations if it does. We evaluate SCOPE’s ability to deliver improved latency
while minimizing power constraint violations by dynamically configuring
hardware while running a variety of Apache Spark applications. Compared to
prior approaches that minimize power constraint violations, SCOPE consumes
comparable power while improving latency by up to 9.5$\times$. Compared to
prior approaches that minimize latency, SCOPE achieves similar latency but
reduces power constraint violation rates by up to 45.88$\times$, achieving
almost zero safety constraint violations across all applications.

††copyright: none

##  1\. Introduction

A modern computer system needs to meet conflicting goals—for example,
minimizing latency while meeting some safety constraint (e.g., power limit)—in
the face of dynamic changes in its application and environment. To do so,
hardware architects expose a wide variety of resources for the system to
manage (Deng et al., 2017; Rahmani et al., 2018; Zhang et al., 2021), where
each type of hardware resource is controlled by a _hardware parameter_ and all
possible allocations of hardware resources constitute a _resource space_.

When managing these resources, the system needs to meet its safety
constraints. Rare and small magnitude violations can be tolerable if they do
not dramatically degrade system performance (Yuan and Nahrstedt, 2003; Fan et
al., 2016; Mishra et al., 2018). However, frequent and large magnitude
violations can cause serious damage, and even crash the system (Raghavendra et
al., 2008). For example, power capping systems deployed in hyperscale
datacenters (e.g., Amazon, Google, Microsoft) smooth out spikes from
occasional power overloading (Sakalkar et al., 2020; Li et al., 2020), but
cannot tolerate large-scale violations without causing significant degradation
in application performance (Ahmad and Vijaykumar, 2010).

To ensure that the system executes optimally while meeting the safety
constraint, existing resource managers use _samples from the resource space_
(i.e., hardware parameters and their measured system behavior) of past
executions to model system behavior (e.g., power and performance) as a
function of hardware resource usage. The model is then used to dynamically
adjust resources usage such that safety is maintained and application
performance is optimized (Ïpek et al., 2006; Mishra et al., 2015, 2018;
Hoffmann, 2015; Ding et al., 2019). However, samples collected from the
resource space may not be generalizable across different applications and
inputs. This means that safe and high-performing hardware resource allocation
for one execution can be unsafe and low-performing in another (Ding et al.,
2021; Yu et al., 2018). Specifically, when a new application or input leads to
significantly different system behavior, models learned using past executions
cannot provide safety guarantees and optimal performance.

Execution space. To address this limitation , we introduce _execution space_ ,
which is the cross product of hardware resources, input features, and
applications. We define a _configuration_ as a tuple of hardware resources,
input features (e.g., data size), and application. To ensure that the system
executes optimally while meeting the safety constraint, we need to _explore_
(i.e., evaluate a configuration that the system has not seen before) the
execution space, rather than the resource space. Exploring the execution space
is the process of evaluating a previously unseen configuration from the
execution space and it allows us to learn models of system behavior as a
function of both the hardware parameters, the current application, and input.

Our solution: safe exploration. To dynamically and safely allocate hardware
resources from the execution space, we present SCOPE, a resource manager that
leverages a novel safe exploration framework. _Safe exploration_ is a family
of sequential decision-making techniques that optimize an objective while
minimizing safety constraint violations (Sui et al., 2015, 2018; Turchetta et
al., 2019). Unlike static configuration that uses the same hardware resources
throughout application execution, SCOPE dynamically reallocates hardware
resources to optimize system performance and minimize safety constraint
violations while responding to dynamic runtime changes. SCOPE operates
iteratively, with each iteration (i.e., reallocation) having three phases:
_monitoring, safe space construction, and objective optimization_.

  * •

In the monitoring phase, SCOPE samples the execution space; i.e., it measures
the system behavior for the current application, input, and hardware resource
allocation. Specifically, it records both the objective and safety data, and
checks whether the safety constraint has been violated or not.

  * •

Safe space construction is the process of predicting a _safe set_ (i.e., a set
of configurations that will not violate the safety constraint) with high
coverage (i.e., a high number of safe configurations in the predicted safe
set). Furthermore, if an unsafe configuration is included in the safe set, its
violation magnitude should be small. To construct a safe set with these
properties, SCOPE introduces a locality preserving operator based on the
_locality preserving criterion_ (Belkin and Niyogi, 2003) (i.e., if two
configurations are close in distance, their system behavior is likely close as
well). This operator constrains SCOPE to explore only configurations within a
certain distance of the most recently executed safe configuration. Because any
new configurations are close to known safe configurations, exploring these new
configurations will rarely violate the safety constraint and have small
magnitude violations if it does.

  * •

In the objective optimization phase, SCOPE reallocates hardware resources with
the best predicted performance from the newly constructed safe set.

Results and contributions. We evaluate SCOPE’s ability to minimize latency
(the objective) while meeting a power (the safety) constraint for a variety of
Apache Spark applications (spa, ). For each input and application, SCOPE
dynamically configures hardware resources (e.g., CPU frequency, uncore
frequency, number of sockets, number of cores per socket, and whether
hyperthreads are enabled). Compared to prior approaches that minimize power
constraint violations, we find:

  * •

Compared to Intel’s RAPL (David et al., 2010), SCOPE decreases the violation
rate by 54.1$\times$ and violation magnitude by 1.04$\times$. These violation
reductions occur because SCOPE can reach even lower power caps than RAPL.
Since RAPL does not optimize latency, SCOPE is able to decrease latency by
9.5$\times$.

  * •

Compared to an existing state-of-the-art safe exploration approach from
domains outside of computer systems, SCOPE decreases latency by 1.11$\times$
across all evaluated applications while decreasing the violation rate and
magnitude by 11.93$\times$ and 1.40$\times$ respectively. SCOPE achieves these
results because it accounts for the unique features of computer systems by
continually monitoring the system; prior work assumes that samples taken early
in execution accurately capture behavior over the system lifetime.

  * •

With its locality preserving operator, SCOPE’s safe set has 1.96$\times$
higher coverage and 1.35$\times$ lower violation magnitude than SCOPE-NO, a
version of SCOPE that does not use the operator (section 6.3), which means
that even in the rare case where an unsafe configuration is selected from
SCOPE’s safe set, it will likely have lower violation magnitude than that of
SCOPE-NO’s.

We summarize the contributions as follows:

  * •

Expanding the exploration space from resource space to execution space, which
captures that system behavior is function of hardware resources, application,
and input.

  * •

Presenting SCOPE, a resource manager that leverages the safe exploration
framework to optimize the objective while minimizing the safety constraint
violations.

  * •

Introducing the locality preserving operator to construct the safe set with
high coverage.

Safe exploration is an important, emerging frontier of machine learning with a
wealth of applications in safety-critical systems. To the best of our
knowledge, SCOPE is the first demonstration of safe exploration in execution
space for computer systems optimization. SCOPE outperforms existing safe
exploration techniques from other domains by developing a locality preserving
operator and requiring fewer assumptions about application behavior than other
safe exploration techniques. Our work offers a foundation on which the
computer systems community can build new optimization tools that aid in
exploration (to improve system performance) while preserving safety.

##  2\. Related Work

This paper’s key insight is a methodology for using machine learning to
perform optimal resource management while meeting safety constraints in
execution space; i.e., without accounting for application and input. This
section focuses on related work in machine learning for computer systems
optimization (section 2.1, section 2.2). Rather than learning the
relationships between all three components (i.e., hardware resources, input,
and application) from the execution space, prior work learns from the resource
space by assuming that samples of the past executions from the resource space
can accurately predict the system behaviors of future executions from
potentially different applications or inputs. We also explain the difference
between existing safe exploration techniques (from domains other than computer
systems) and SCOPE (section 2.3).

###  2.1. Machine Learning for Unconstrained Optimization

Machine learning techniques have been increasingly applied to solve computer
systems optimization problems by modeling complex, nonlinear relationships
between system resource usage and quantifiable behavior (Penney and Chen,
2019; Zhang and Huang, 2019). Much prior work focuses on unconstrained
optimization problems with no safety constraint—i.e., optimizing a single
objective such as latency (Belay et al., 2014), throughput (Li et al., 2020),
power (Lee and Brooks, 2006), and energy consumption (Yuan and Nahrstedt,
2003). These works share a common methodology of building machine learning
models using training data collected by sampling the system resource space:
allocating different system resources and measuring behaviors. Specifically,
there are two types of system sampling, random and intelligent, and each leads
to different types of machine learning approaches. Random sampling typically
needs a large amount of samples to build a highly accurate model, but it is
free of biases that might arise from intelligent sampling (Yi et al., 2003;
Lee and Brooks, 2006; Bitirgen et al., 2008; Ansel et al., 2011; Chen et al.,
2011; Cochran et al., 2011; Oliner et al., 2013; Deng et al., 2017; Ansel et
al., 2012; Ponomarev et al., 2001; Sridharan et al., 2013; Garza et al., 2019;
Bhatia et al., 2019; Shi et al., 2019, 2021). For example, Lee et al. (2008)
build a regression model on simulated data to predict multiprocessor
performance. Paragon (Delimitrou and Kozyrakis, 2013) and Quasar (Delimitrou
and Kozyrakis, 2014) apply collaborative filtering to predict QoS performance
of workloads in datacenters. However, the large sampling effort required for
random sampling is often prohibitive due to is high computational cost and
inefficiency (Yu et al., 2018).

Unlike random sampling, intelligent sampling significantly reduces the
sampling effort required to achieve good systems outcomes (Ïpek et al., 2006;
Ipek et al., 2008; Petrica et al., 2013; Venkataraman et al., 2016; Canino et
al., 2018). A representative family of intelligent sampling techniques is
Bayesian optimization, which iteratively samples data points that are
predicted to contribute the most information to the learning model (Frazier,
2018). CherryPick (Alipourfard et al., 2017) and CLITE (Patel and Tiwari,
2020) use Bayesian optimization to schedule workloads in datacenters. GIL
(Ding et al., 2021) and Bliss (Roy et al., 2021) use Bayesian optimization to
optimize system performance. HyperMapper (Nardi et al., 2019) and BOCA (Chen
et al., 2021) use Bayesian optimization to tune compilers. These works reduce
the number of samples required to perform optimization, but do not consider
any safety constraints. This motivates the need for a learning-based resource
manager that both (1) works with reduced number of samples and (2) respects
safety constraints.

###  2.2. Machine Learning for Safe Optimization

Safe optimization problems in computer systems find the optimal point within a
tradeoff space (e.g., performance versus power)—i.e., optimizing a performance
objective under some safety constraint (Li and Martinez, 2006; Dubach et al.,
2010; Hoffmann, 2015; Deng et al., 2017; Ding et al., 2019). For example, Li
and Martinez (2006) collect samples to optimize power under a performance
constraint. Dubach et al. (2010) collect samples to build a dynamic control
system that optimizes energy and performance efficiency. LEO (Mishra et al.,
2015) and CALOREE (Mishra et al., 2018) develop hierarchical Bayesian models
to meet latency constraints and minimize energy.

These works provide safety under the assumption that the samples they collect
from the resource space of past executions can accurately capture the system
behaviors that will be seen during future executions. However, this assumption
can be violated when a new application or new input causes significantly
different system behaviors than those in the collected samples. The emergence
of such unsampled behavior would render the whole system unsafe. As such,
there is a need for approaches that can explore unseen configurations in the
execution space, rather than the resource space.

###  2.3. Safe Exploration in Other Problem Domains

Recent years have witnessed safe exploration applied to safety-critical
domains such as autonomous driving (Razin and Feigh, 2020), healthcare (Sui et
al., 2018), and robotics (Brunke et al., 2021). Safe exploration is a family
of sequential decision-making techniques that optimize an objective while
minimizing safety constraint violations (Garcia and Fernández, 2012; Sui et
al., 2015, 2018; Turchetta et al., 2019; Mao et al., 2019; Turchetta et al.,
2020; Wachi and Sui, 2020; Xu et al., 2021). To satisfy some safety property
with a high probability, these techniques either require extra supervision or
knowledge accumulated before exploration. They achieve probabilistic (i.e.,
not deterministic) safety guarantees based on the assumption that the changes
of safety measurements are continuous and bounded (i.e., Lipschitz
continuity). In computer systems, however, this assumption does not hold. A
typical example is power, where the power usage can change dramatically every
second (Zhang and Hoffman, 2015). Different from prior work, our solution,
SCOPE, is a new safe exploration framework that introduces a locality
preserving operator to eliminate the need for such assumptions. Evaluation
results show that SCOPE outperforms the existing state-of-the-art safe
exploration technique (e.g., StageOPT (Sui et al., 2018)) in both performance
and safety by tailoring our approach to the unique properties of computer
systems.

##  3\. Motivational Examples

We use two examples to demonstrate that to explore new configurations safely,
optimization must operate in execution space rather than resource space
because safe samples from the resource space of past executions do not
generalize to new applications and new inputs. To demonstrate a lack of
generalization across applications, we run two different applications and show
that the safe samples from the resource space for one application are no
longer safe for another. To demonstrate a lack of generalization across
inputs, we run one application with two different inputs and show that the
safe samples from the resource space for one are not safe for the other.

We consider two Apache Spark applications from HiBench (Huang et al., 2010)
and run them on the Chameleon configurable cloud computing platform (Keahey et
al., 2020) (details in section 5.1). We collect samples by profiling the
application at all possible assignments of hardware parameters (see Table 1)
and recording their latency and power data. Our goal is to minimize latency
while meeting a power constraint.

Safe samples from the resource space do not generalize across applications. We
show that safe samples from the resource space do not generalize across the
applications als and nweight. We construct a safe set for als by randomly
selecting a list of hardware allocations that do not violate the safety
constraint based on all samples collected for als. Then, we dynamically
configure als (at 20s intervals) using this safe set. We then use the same
safe set to dynamically configure nweight (again, at 20s intervals).

Figure 1 shows the resulting power as a function of execution time, where the
x-axis is the execution time, and the y-axis is the power. The blue line
represents als using its own safe set, the orange line represents running
nweight using als’s safe set, the red line represents running nweight using
safe hardware allocations constructed especially for nweight (again by
profiling all possible assignments of hardware parameters). The black dotted
horizontal line is the power limit which represents the safety constraint in
this example. We observe that als finishes safely in 125s with 0 power
violations, while nweight using als’s safe hardware allocations needs 190s
despite 32 power violations.

Figure 1. Results of power and latency for executing als and nweight.

The high number of violations and the significant difference in latency, show
that the safe samples from the resource space do not generalize across
different applications. In fact, nweight—with a properly constructed safe
set—finishes in 142s with 0 violations, indicating that it is possible to
improve nweight’s latency and safety, however, exploration operating only
within the resource space fails to do so.

Safe samples from the resource space do not generalize across different inputs
for the same application. We show that safe samples from the resource space do
not generalize across different inputs for the same application using als. We
create two different inputs with the same sizes (so the execution behavior
variations are due to properties of the data rather than data size): Input A
is the input we will sample to build a safe set and Input B is the target
input to be optimized. We construct a safe set for Input A by randomly
selecting a list of hardware allocations that do not violate constraints based
on all samples we collected for Input A. Then, we dynamically configure for
Input A (at 20s intervals) using this safe set, and then configure using the
same safe set for Input B (again, at 20s intervals).

Figure 2. Results of power and latency for executing als with two different
sets of inputs with same size.

Figure 2 shows the the resulting power as a function of execution time, where
the x-axis is the execution time, and y-axis is the power. The blue line
represents running Input A using its own safe set, the orange line represents
running Input B using Input A’s safe set, the red line represents running
Input B with its own safe set found by evaluating Input B in all possible
assignments of hardware parameters. The black dotted horizontal line is the
power threshold, or safety constraint in this example. We observe that Input A
finishes in 147s with 0 violation. Running Input B with Input A’s safe set,
however, finishes in 110s with 20 violations, mainly between 26s and 46s.
Input B is mostly under the power threshold, but its dynamic behavior causes
it to violate safety constraint, while Input A does not.

The high number of violations when running Input B shows that safe samples
from the resource space for Input A are no longer safe with different inputs,
even when the inputs are the same size. In fact, Input B can finish in 91s
with 0 violations with an appropriate safe set, which indicates that the safe
samples from the resource space for Input A do not generalize to Input B.

These examples demonstrate that safe samples from the resource space of past
executions fail to generalize to new applications and new inputs. As an
alternative to generalizing safe samples to new applications or inputs, the
next section describes SCOPE, our solution to optimize objectives and minimize
safety constraint violations in the execution space, rather than resource
space.

Figure 3. The workflow of SCOPE at each time interval while the application is
executing.

##  4\. SCOPE Design

SCOPE is a resource manager that dynamically and safely explores in the
execution space. In other words, SCOPE makes no assumptions about how past
executions, with different applications or inputs, affect the current system
behavior. SCOPE iteratively reallocates hardware resources, with each
iteration having three phases: monitoring, safe space construction, and
objective optimization. Figure 3 illustrates SCOPE’s workflow. In the first
phase of each time interval, SCOPE continually measures safety and objective
metrics for the current application, input, and hardware resource allocation (
A), and checks whether the safety constraint has been violated or not ( 1); if
a safety measurement violates the safety constraint, SCOPE moves to the next
phase immediately, otherwise it waits for a fixed time interval to expire.
Then, SCOPE goes to the phase of safe space construction to predict a safe set
with high coverage. Within this phase, SCOPE first builds a safety model using
the measured configurations and safety data ( B), and then constructs a safe
set based on the safety model ( C). Then, SCOPE’s objective optimization phase
reconfigures the system. Within this phase, SCOPE first builds an objective
model using the measured configurations and objective data ( D), and then
picks a predicted high-performing configuration from the safe set and puts the
system into that configuration.

The remainder of this section first sets up the core concepts, and then
describes SCOPE in detail.

###  4.1. Background and Definitions

SCOPE’s input includes an application, objective metric to optimize, safety
constraint, a starting safe configuration, number of measurements for each
time interval, and a list of configurations over which to optimize. A starting
safe configuration is needed to prevent SCOPE from violating the safety
constraint during the first iteration; users can conservatively choose this
configuration. For example, if the safety metric is power, users could start
in a configuration with minimal hardware resources. If latency is the safety
metric, then users could start with a configuration that makes all resources
available. SCOPE’s exploration will safely move the system out of this
conservative configuration to one that improves the objective metric.

Application.:

A program that runs on a computer system using hardware resources.

Configuration.:

A configuration ${\mathbf{x}}_{i}\in D$ is a $d$-dimensional vector that
includes $d$ parameters: ${\mathbf{x}}_{i}=[x_{i1},x_{i2},\ldots,x_{id}]$,
where $i$ is the $i$-th configuration, and $x_{ij}$ is the value for $j$-th
parameter, $j\in[d]$.

Safety constraint.:

The safety constraint is a threshold for a particular metric that the system
does not want to violate during application execution. Some common metrics
that can be used for safety constraints include energy (Temam, 2012; Hoffmann,
2015), power (Reagen et al., 2016), and latency (Mishra et al., 2015, 2018;
Ding et al., 2019). In this paper, we use $P$ to denote the safety constraint
and $y_{i}$ the safety measurement for the $i$-th configuration.

Safe configuration.:

The safe configuration is a configuration that does not violate the safety
constraint, i.e., $y_{i}<P$. Similarly, an unsafe configuration is a
configuration that violates the safety constraint, i.e., $y_{i}\geq P$.

Optimization objective.:

The optimization objective is the metric that SCOPE minimizes or maximizes.
Some common optimization objectives include energy (Mishra et al., 2015, 2018;
Ding et al., 2019), power (Reagen et al., 2016), and latency (Temam, 2012;
Hoffmann, 2015). In this paper, we use $z_{i}$ to denote the optimization
objective measurement for the $i$-th configuration.

Time interval.:

The time interval is a short period of time during application execution. In
this paper, we use $T_{i}$ to denote $i$-th time interval when the application
runs configuration ${\mathbf{x}}_{i}$. During each time interval, SCOPE will
execute all three of its phases: monitoring, safe set construction, and
objective optimization.

Measurement interval.:

The measurement interval is the amount of time over which SCOPE gets a pair of
safety and objective measurements.

###  4.2. Monitoring

At $i$-th time interval when the application runs at configuration
${\mathbf{x}}_{i}$, SCOPE continuously gets safety $y_{i}$ and objective
$z_{i}$ at each measurement interval for a maximum of $N$ times, where $N$ is
a input parameter of SCOPE ( A). To react to violations as quickly as
possible, if a safety measurement violates the safety constraint, SCOPE stops
monitoring and moves to the next phase immediately instead of finishing $N$
measurements ( 1). The measurement data collected over the time intervals are
the training data for the next phases of SCOPE; i.e., until the $i$-th time
interval, $X_{\rm train}=({\mathbf{x}}_{j})_{j=0}^{i}$, $Y_{\rm
train}=(y_{j})_{j=0}^{i}$, $Z_{\rm train}=(z_{j})_{j=0}^{i}$.

###  4.3. Safe Space Construction

After getting the training data from measurement, SCOPE goes to safe space
construction, which is the process of predicting a safe set with high
coverage—i.e., a high number of safe configurations in the predicted safe
set—and if an unsafe configuration is included, its violation magnitude should
be small. To construct a safe set with these properties, SCOPE introduces a
_locality preserving operator_ based on the _locality preserving criterion_
(Belkin and Niyogi, 2003) (i.e., if two configurations are close in distance,
their system behavior is also likely to be close). The translation of this
criterion to safe space construction is as follows:

  * •

If two configurations are close in distance, their corresponding safety and
objective measurements are likely close too.

  * •

If configuration A is safe and configuration B is unsafe but close to
configuration A in distance, the safety violation of configuration B is very
likely to be small.

This criterion matches our empirical observation that a smaller magnitude
configuration change (e.g., changing 2 cores to 4 cores) leads to smaller
system behavior changes than a larger change (e.g., changing 2 cores to 12
cores). The locality preserving operator constrains SCOPE to explore a subset
of configurations in the safe set that are within a neighborhood (i.e., close
area within some distance) of the most recently used safe configuration so
that SCOPE’s exploration will rarely violate the safety constraint and have
small magnitude violations if it does. Formally, $X_{c}$ is the candidate set
that is constructed with the locality preserving operator:

(1) $\displaystyle X_{c}=\\{{\mathbf{x}}\in D\setminus X_{\rm
train}|~{}\|{\mathbf{x}}-{\mathbf{x}}^{s}\|\leq\gamma\\},$

where ${\mathbf{x}}^{s}$ is the most recent safe configuration that SCOPE has
been used, and $\gamma\geq 0$ is the operator parameter. The operator
parameter $\gamma$ controls the distance that SCOPE explores. If $\gamma=0$,
the only configuration that SCOPE can explore is the starting safe
configuration. If $\gamma>0$, SCOPE can explore configurations that are within
distance $\gamma$ from ${\mathbf{x}}^{s}$. The higher $\gamma$ is, the larger
configuration space SCOPE can explore, and thus the more likely the unsafe
configurations can be included in the safe set. When $\gamma$ is large enough,
it is equivalent to exploring without constraints.

Upon obtaining the candidate set $X_{c}$, SCOPE constructs the safe set as
follows. At $i$-th time interval $T_{i}$, SCOPE trains the safety model
$f_{y}$ using the configurations and safety measurements collected so far
$(X_{\rm train},Y_{\rm train})=({\mathbf{x}}_{j},y_{j})_{j=0}^{i}$ ( B). SCOPE
uses this model to predict the safety values of the configurations in the
candidate set $X_{c}$, and include configurations that are predicted to meet
the safety constraint in the safe set ( C):

(2) $\displaystyle X_{s}=\\{{\mathbf{x}}\in X_{c}|f_{y}({\mathbf{x}})<P\\},$

where $X_{s}$ is the safe set, and $P$ is the safety constraint. The design of
safe space construction is compatible with any type of learning models such as
Gaussian process regression (Alipourfard et al., 2017), random forest (Nardi
et al., 2019), linear model (Ding et al., 2021), and neural networks (Ïpek et
al., 2006). SCOPE uses Gaussian process regression since it performs the best
in practice (section 6.6).

###  4.4. Objective Optimization

The objective optimization phase reallocates hardware resources with a high-
performing safe configuration. At $i$-th time interval, SCOPE trains the
objective model $f_{z}$ using the configurations and objective measurements
collected so far $(X_{\rm train},Z_{\rm
train})=({\mathbf{x}}_{j},z_{j})_{j=0}^{i}$ ( D). SCOPE uses this model to
predict the objective values of the configurations from the safe set.
Empirically, we find that the safe set could be empty when the operator
parameter $\gamma$ is too small, since there is not much neighborhood for
SCOPE to explore. To address the possible situation like this, SCOPE does the
following.

  * •

If the safe set is empty, SCOPE picks the configuration in the candidate set
$X_{c}$ that has the best predicted safety. In this way, SCOPE explores the
configuration space while avoiding safety violations.

  * •

If the safe set is not empty, SCOPE picks the configuration in the safe set
$X_{s}=\\{{\mathbf{x}}\in X_{r}|f_{y}({\mathbf{x}})<P\\}$ that has the best
predicted objective.

After SCOPE picks the new configuration and reallocates hardware sources based
on this configuration ( E), it goes to the next time interval until the
execution ends.

Algorithm 1 The SCOPE Resource Manager

1:${\mathbf{x}}_{0}$ $\triangleright$ Starting safe configuration.

2:$P$ $\triangleright$ Safety constraint threshold.

3:$\gamma$ $\triangleright$ Operator parameter.

4:$N$ $\triangleright$ Number of measurements.

5:$X_{\rm train}=\\{\\}$ $\triangleright$ Set of sampled configurations.

6:$Y_{\rm train}=\\{\\}$ $\triangleright$ Set for safety measurements.

7:$Z_{\rm train}=\\{\\}$ $\triangleright$ Set for objective measurements.

8:$i=0$

9:${\mathbf{x}}^{s}\leftarrow{\mathbf{x}}_{0}$ $\triangleright$ Assign current
safe configuration.

10:Run application at ${\mathbf{x}}_{0}$.

11:while application running do

12: for $t=1\text{,...}N$ do

13: Get safety $y_{i}$ and objective $z_{i}$.

14: if $y_{i}>P$ then

15: Stop monitoring.

16: Update training set $(X_{\rm train},Y_{\rm train},Z_{\rm train})$ with
$({\mathbf{x}}_{i},y_{i},z_{i})$.

17: Train safety model $f_{y}$ using $X_{\rm train}$ and $Y_{\rm train}$.

18: if $y_{i}<P$ then

19: ${\mathbf{x}}^{s}\leftarrow{\mathbf{x}}_{i}$ $\triangleright$ Update
current safe configuration.

20: $X_{c}=\\{{\mathbf{x}}\in D\setminus X_{\rm
train}|~{}\|{\mathbf{x}}-{\mathbf{x}}^{s}\|\leq\gamma\\}$ $\triangleright$
Update candidate configuration set.

21: $X_{s}=\\{{\mathbf{x}}\in X_{c}|f_{y}({\mathbf{x}})<P\\}$ $\triangleright$
Construct safe set.

22: Train objective model $f_{z}$ using $X_{\rm train}$ and $Z_{\rm train}$.

23: if $|X_{s}|==0$ then

24: ${\mathbf{x}}_{i+1}\leftarrow\operatorname*{arg\,min}_{{\mathbf{x}}\in
X_{c}}f_{y}({\mathbf{x}})$ $\triangleright$ If safe set is empty, pick
configuration with best predicted safety.

25: else

26: ${\mathbf{x}}_{i+1}\leftarrow\operatorname*{arg\,max}_{{\mathbf{x}}\in
X_{s}}f_{z}({\mathbf{x}})$ $\triangleright$ If safe set is not empty, pick
configuration with best predicted objective.

27: Reallocate hardware resources with ${\mathbf{x}}_{i+1}$.

28: $i\leftarrow i+1$

###  4.5. SCOPE Algorithm Summary

SCOPE is a general system design compatible with any type of learning models,
safety constraints, and optimization objectives. To demonstrate its
effectiveness in solving systems problems, we use SCOPE to minimize latency
while constraining the power usage to be below a fixed threshold, where the
safety metric is power, and the optimization objective is work done so far
(e.g., total instruction count). The input includes a starting safe
configuration ${\mathbf{x}}_{0}$, safety constraint $P$, number of
measurements $N$, and operator parameter $\gamma$. The starting safe
configuration can be set by the user since it is not desirable that the
application violates the safety constraint in the beginning. The safety
constraint depends on the optimization goal and is set by the user. The
operator parameter $\gamma$ is set by the user to control the exploration
space, where more insights can be found in section 6.4.

Algorithm 1 summarizes the procedure. In 10, SCOPE starts executing the
application, and records the current time $T_{0}$. While the application is
executing, SCOPE does the following steps iteratively. For each measurement
interval, SCOPE gets the safety and objective (13). If the safety measurement
violates the safety constraint (14), SCOPE stops measuring and moves to the
next phase (15). Otherwise, SCOPE continues measuring. In 16, SCOPE updates
the training set by adding the new measured data. In 17, SCOPE trains the
safety model using training configurations and safety data. In 19, SCOPE
updates the safe configuration that has been most recently executed. In 20,
SCOPE updates the candidate set by applying the locality preserving operator.
In 21, SCOPE constructs the safe set from the candidate set based on the
predictions using the safety model. In 22, SCOPE trains the objective model
using training configurations and objective data. Then, SCOPE will conduct
different steps based on the cardinality of the safe set. In 23 and 24, if the
safe set is empty, SCOPE will pick the configuration with the best predicted
safety in the candidate set. Otherwise, in 26, SCOPE will pick the
configuration with the best predicted objective in the safe set. In 27, SCOPE
reallocates hardware resources based on the newly pickled configuration. We
implement SCOPE in Python with libraries including numpy (Harris et al.,
2020), pandas (pandas development team, 2020), and scikit-learn (Pedregosa et
al., 2011). The code is released in https://anonymous.4open.science/r/scope-
code-9999.

##  5\. Experimental Setup

###  5.1. Hardware System

We experiment on the Chameleon configurable cloud computing platforms (Keahey
et al., 2020), where each experiment runs on a master node and four worker
nodes. Each node is a dual-socket system running Ubuntu 18.04 (GNU/Linux 5.4)
with 2 Intel(R) Xeon(R) Gold 6126 processors, 192 GB of RAM, hyperthreads and
TurboBoost. Each socket has 12 cores/24 hyperthreads and a 20 MB last-level
cache. We tune the hardware parameters in Table 1. These parameters have been
shown to influence both latency and power tradeoffs and are important to tune
to optimally meet a power cap (Zhang and Hoffmann, 2016). In total, there are
1920 possible allocations of hardware resources to be explored dynamically
while minimizing power cap violations.

Table 1. Hardware parameters. Parameter | Range
---|---
CPU frequency (GHz) | 1.0–3.7
Uncore frequency (GHz) | 1.0–2.4
Hyperthreading | on, off
Number of sockets | 1, 2
Number of cores per socket | 1–12

###  5.2. Software System

We use Apache Spark (spa, ) as our software system with the default Spark
configuration settings. We use 12 applications from HiBench’s benchmark suite
(Huang et al., 2010), which has been widely applied to configuration
optimization evaluations (Yu et al., 2018; Banerjee et al., 2021; Ding et al.,
2021; Wang et al., 2019). The applications cover various domains including
microbenchmarks, machine learning, websearch, and graph analytics (Table 2).

Table 2. HiBench applications. Application | Data size | Application | Data size
---|---|---|---
als | 0.7 GB | bayes | 1 GB
gbt | 0.1 GB | kmeans | 2.1 GB
linear | 36 GB | lr | 2 GB
nweight | 0.1 GB | pagerank | 0.2 GB
pca | 0.1 GB | rf | 1.6 GB
terasort | 3.2 GB | wordcount | 26 GB

###  5.3. Points of Comparison

We compare various approaches including static configuration, approaches with
extensive collected samples (Offline), approaches without collected samples
(RAPL, BO, StageOPT, SCOPE-NO), and an approach with perfect knowledge
(Oracle, which we create with brute force search and is, of course,
unrealizable in practice).

  * •

Static: run application at the starting safe configuration throughout the
execution.

  * •

Offline: before running the target application, randomly sample half of all
possible hardware resource allocations using the target application and input,
and build safety and objective models using Gaussian process regression. These
models are not updated during application execution. We reconfigure using the
predictions from these models at each time interval (Mishra et al., 2015,
2018).

  * •

RAPL: Intel’s Running Average Power Limit system that allows users to set a
power limit and tunes processor behavior to respect that limit using dynamic
voltage and frequency scaling technique (David et al., 2010); RAPL only
configures CPU and uncore frequency.

  * •

BO: use Bayesian optimization to reconfigure at each time interval, with
Bayesian Gaussian process regression being the learning model and expected
improvement being the acquisition function (Alipourfard et al., 2017; Nardi et
al., 2019; Patel and Tiwari, 2020; Chen et al., 2021).

  * •

StageOPT: use the StageOPT algorithm (Sui et al., 2018), a representative of
safe exploration approaches from other domains that assume continuous and
bounded changes in safety measurements to achieve probabilistic safety
guarantee. StageOPT separates safe set construction and objective optimization
into two stages: in the first few iterations, it focuses on constructing the
safe set only; and then it switches to optimizing the objective within the
safe space without updating the safe set. The lack of update to the safe set
can be very problematic in computer systems. As shown in our motivational
example, the discrete nature of computer systems means that applications that
look safe might quickly change to unsafe with no prior warning (see Figure 2).
In contrast, SCOPE continually updates its safe set.

  * •

SCOPE-NO: use the introduced safe exploration framework without the locality
preserving operator to reconfigure at each time interval.

  * •

SCOPE: use the introduced safe exploration framework with the locality
preserving operator to reconfigure at each time interval. Both SCOPE and
SCOPE-NO use Gaussian process regression as the learning model since it
performs the best (section 6.6).

  * •

Oracle: profile all latency and power data for the entire configuration space,
and pick the fastest configuration that meets the power constraint.

###  5.4. Evaluation Methodology

We start executing the application at the same safe configuration for each
approach in section 5.3, and then reconfigure it dynamically at each time
interval. For all approaches that reconfigure dynamically, we run the sweep
over different time intervals and pick the best time interval for each
(section 6.5). During each time interval, we use the maximum power for safety
and average of the work done (i.e. total instruction counts) for objective.

We evaluate on a wide range of power constraints that are set as
$[40,50,60,70,80]$-th percentiles of the power distributions that are
achievable across all configurations. This is a reasonable range in that a
small constraint value (e.g., $[10,20,30]$-th percentiles) leaves little room
for tuning and a large constraint value (e.g., $[90]$-th percentile) is often
too relaxed to constrain the power.

To make a comprehensive comparison, we identify 5 starting configurations as
fast and 5 as slow. For fast configurations, the latency is below p35 of the
latency distribution over all configurations that meet the power constraint.
For slow ones, the latency is above p65 of the latency distribution that meet
the power constraint. The reported results are averaged over different
constraints and 10 different starting safe configurations.

For SCOPE, we choose the operator parameter $\gamma=1$ for all applications
based on the best tradeoffs between the violation rates and speedups from
sensitivity analysis in section 6.4. We use 1 second as the measurement
interval.

We note that SCOPE runs on the same hardware whose power it is controlling.
Thus, SCOPE must account and compensate for its own power overhead. All
results include the power and latency overhead of running SCOPE.

###  5.5. Evaluation Metrics

For latency evaluation, we measure the latency $l_{\rm confg}$ obtained by
each approach, and compute its speedup compared to the latency $l_{\rm
static}$ of Static:

(3) $\displaystyle\text{speedup}=\frac{l_{\rm static}}{l_{\rm confg}}.$

For power evaluation, we record the power over 1 second interval. We note the
times that power exceeds the power threshold $n_{\rm violate}$, and then
divide it by the total number of measurements $n_{\rm total}$:

(4) $\displaystyle\text{violation~{}rate}=\frac{n_{\rm violate}}{n_{\rm
total}}\times 100\%.$

We also record the average of power usage that exceeds the power threshold $p$
throughout application execution, and then divide it by the power threshold
$P$:

(5) $\displaystyle\text{violation~{}magnitude}=\begin{cases}0,&\quad\text{if
}p\leq P\\\ \frac{p}{P},&\quad\text{if }p>P\end{cases}$

For safe set, we define coverage of the safe set:

(6) $\displaystyle\text{coverage}=\frac{m_{\rm safe}}{m_{\rm total}}\times
100\%,$

where $m_{\rm safe}$ is the number of true safe configurations in the
predicted safe set, and $m_{\rm total}$ is the number of all configurations in
the predicted safe set.

##  6\. Experimental Evaluation

We evaluate the following research questions (RQs):

  * •

RQ1: Does SCOPE reduce power violations? SCOPE reduces violation rates by
3.62–54.1$\times$ (Figure 4) and violation magnitudes by 1.04–1.49$\times$
(Figure 5) compared to other baselines.

  * •

RQ2: Does SCOPE improve application latency? SCOPE improves application
latency by 1.07–9.5$\times$ compared to other baselines (Figure 7).

  * •

RQ3: Does locality preservation produce better safe sets? With the locality
preserving operator, SCOPE’s safe set has 1.96$\times$ higher coverage and
1.35$\times$ lower violation magnitude than SCOPE-NO that does not use the
operator (Table 3), which means that even in the rare case where an unsafe
configuration is selected from SCOPE’s safe set, it will likely have lower
violation magnitude than that of SCOPE-NO.

  * •

RQ4: How sensitive is SCOPE to $\gamma$? The operator parameter $\gamma$
affects both speedup and violation rate, and all applications share a common
trend of the tradeoffs between speedup and violation rate (Figure 8).

  * •

RQ5: How sensitive is SCOPE to time intervals? SCOPE is the most robust to the
time interval between reallocations compared to other baselines due to its
ability to update models at all iterations and locality preserving operator
(Figure 9).

  * •

RQ6: How do different types of models perform? SCOPE can be used with any
types of learning models, and we have chosen Gaussian process regression for
all of our evaluations due to its lowest violation rates and comparably low
latency (Figure 10).

  * •

RQ7: What are the overheads? SCOPE achieves low overhead of 0.05s per sample
on average, which is negligible given the fact that we include the overheads
in all experiments and SCOPE has the best latency improvement (Figure 11).

Figure 4. Violation rates averaged over all power constraints for each
application. Lower is better. Figure 5. Violation magnitudes averaged over all
power constraints for each application. Lower is better. Figure 6. Power
results at different time points during execution for each application. Figure
7. Speedup averaged over all power constraints for each application. Higher is
better.

###  6.1. RQ1: Does SCOPE reduce power violations?

We use violation rate and magnitude to evaluate how well SCOPE reduces power
violations. Figure 4 and 5 summarize the average violation rates and
magnitudes for different approaches, where the x-axis is the application, the
y-axis is the violation rate or magnitude and the last column Mean is the
arithmetic mean over all applications for Figure 4 and the arithmetic mean
over all applications with non-zero violation magnitude for Figure 5, where we
put numbers of each bar to quantify the average results. To visualize better,
we cap the violation rate at 5% in Figure 4 and put numbers on the bars that
are capped. Static and Oracle were omitted since they do not violate at all;
Static runs the same starting safe configuration throughout the execution, and
Oracle runs the fastest configuration that is safe based on the exhaustive
search. Figure 6 shows power results at every time point for different
approaches, where x-axis is the running time, and y-axis is the power.
Overall, SCOPE has the lowest violation rate and magnitude, making rare and
small disruptions that disturb the system the least:

Violation rate.:

SCOPE is 3.62$\times$ better than Offline, 54.1$\times$ better than RAPL,
45.88$\times$ better than BO, and 11.93$\times$ better than StageOPT.

Violation magnitude.:

SCOPE is 1.05$\times$ better than Offline, 1.04$\times$ better than RAPL,
1.49$\times$ better than BO, 1.40$\times$ better than StageOPT.

Meanwhile, we observe the following:

  * •

Offline underperforms SCOPE in all but 2 applications. Despite utilizing a
large amount of samples collected a priori, the models Offline builds a priori
do not generalize to the environment changes that occur during execution,
while SCOPE dynamically updates its models to reconfigure. As a result,
Offline has higher violation rate, which also demonstrates the difficulty of
generalization of collected samples.

  * •

RAPL has the highest violation rate despite the fact that it only optimizes
for power. RAPL achieves low violation rate with higher power constraints
(e.g. nweight and terasort in Figure 6), but the lower power constraints (i.e.
[40,50]-th percentiles of the power distribution) are below the minimum power
threshold that RAPL can meet and cause RAPL to violate constantly. This
suggests that SCOPE performs much better than RAPL with a wider range of
constraints.

  * •

BO has the second highest violation rate and largest violation magnitudes
since it optimizes for latency only and does not consider power constraints.

  * •

StageOPT underperforms SCOPE because StageOPT achieves probabilistic safety
guarantees by assuming continuity and boundedness of power measurements, which
are not guaranteed to hold in computer systems. Thus StageOPT makes inaccurate
predictions, which leads it to have higher violation rates and magnitudes.

  * •

SCOPE has 1.38$\times$ lower violation rate and 1.04$\times$ lower violation
magnitude than SCOPE-NO. The results suggest that the introduced operator is
beneficial for reducing violations (more details in section 6.3).

Table 3. Summarized results of the percentage of configurations (POC) selected in the total configuration space, coverage, and the average violation magnitudes (AVM) of unsafe configurations in the safe set for SCOPE-NO and SCOPE when the application runs at the last time interval. Higher coverage is better. Lower AVM is better. | SCOPE-NO | SCOPE
---|---|---
| POC (%) | Coverage (%) |  | AVM
---
POC (%) | Coverage (%) |  | AVM
---
als | 95.78 | 38.48 | 1.47 | 19.13 | 71.08 | 1.16
bayes | 95.71 | 39.77 | 1.45 | 19.52 | 76.73 | 1.09
gbt | 95.66 | 27.97 | 1.61 | 20.81 | 55.88 | 1.16
kmeans | 94.28 | 24.85 | 1.66 | 18.31 | 56.04 | 1.19
linear | 95.30 | 43.24 | 1.42 | 20.08 | 76.32 | 1.12
lr | 95.74 | 31.73 | 1.52 | 18.60 | 65.47 | 1.10
nweight | 95.30 | 31.63 | 1.55 | 19.87 | 60.14 | 1.16
pagerank | 95.83 | 39.92 | 1.47 | 20.56 | 71.41 | 1.11
pca | 93.06 | 30.19 | 1.58 | 18.63 | 65.60 | 1.11
rf | 92.21 | 19.13 | 1.75 | 17.86 | 40.69 | 1.19
terasort | 95.55 | 34.87 | 1.51 | 19.04 | 74.32 | 1.12
wordcount | 95.66 | 39.83 | 1.47 | 18.74 | 73.61 | 1.19
Mean | 95.01 | 33.47 | 1.54 | 19.26 | 65.61 | 1.14
Figure 8. Violation rate and speedup as a function of $\gamma$ averaged over
all power constraints for each application. Lower is better for violation
rate. Higher is better for speedup.

###  6.2. RQ2: Does SCOPE improve application latency?

Figure 7 summarizes the speedups over Static for different approaches, where
the x-axis is the application, the y-axis is the speedup, and the last column
Mean is the arithmetic mean over all applications, where we put the number on
each bar to quantify the summarized results. We have omitted BO as it
optimizes for latency only and thus has a very high violation rate and
magnitude, which makes it an unfair comparison.

Oracle has the highest latency speedup since it has perfect prior knowledge to
choose the fastest safe configuration. SCOPE is second to Oracle: 1.07$\times$
higher speedup than Offline, 1.11$\times$ higher speedup than StageOPT, and
9.5$\times$ higher speedup than the slowest baseline, RAPL. In particular, we
observe the following:

  * •

All approaches outperform Static except RAPL, which demonstrates the
effectiveness of dynamic reconfiguration during execution for optimizing
latency. RAPL is the exception because it manages power only and disregards
latency. In addition, because RAPL only configures core frequency, it is
unable to take advantage of more complex tradeoffs that can reduce power
without harming latency as much (for example, reducing core usage for
applications with low parallel speedup) (Zhang and Hoffmann, 2016).

  * •

Offline, despite training on a large amount of samples collected prior to
running the target application, has lower speedup than SCOPE. This is because
Offline uses fixed models trained over early collected data that fail to adapt
to changes in execution.

  * •

StageOPT, despite achieving good results in other problem domains,
underperforms SCOPE for computer systems optimization. It is because StageOPT
focuses on constructing the safe set in the first stage and thus does not
optimize latency until the second stage, while SCOPE constructs the safe set
and optimizes latency throughout application execution.

  * •

SCOPE achieves 1.05$\times$ higher speedup than SCOPE-NO. This suggests that
the locality preserving operator not only reduces safety violations, but
improves latency. Detailed analyses can be found in section 6.3 and section
6.4.

###  6.3. RQ3: Does locality preservation produce better safe sets?

To better understand how the locality preserving operator improves the safe
set for SCOPE over SCOPE-NO (the one without the operator), we show the
percentage of configurations (POC) selected in the total configuration space,
coverage, and the average violation magnitudes (AVM) of unsafe configurations
in the safe set for SCOPE-NO and SCOPE in Table 3, where the numbers are
averaged over different power thresholds and starting configurations. SCOPE’s
results are obtained when the operator parameter $\gamma=1$ for all
applications based on the best tradeoffs between violation rate and speedup
(section 6.4). We observe the following:

  * •

SCOPE has 4.9$\times$ POC smaller than SCOPE-NO, which indicates that the
operator significantly reduces the number of configurations included in the
safe set.

  * •

Although SCOPE’s safe set has significantly fewer configurations, its coverage
is 1.96$\times$ higher than that of SCOPE-NO, which indicates that the
operator greatly improves the accuracy of the safe set prediction. The higher
accuracy that SCOPE has in predicting safe configurations leads SCOPE to
achieve 1.38$\times$ lower violation rate than SCOPE-NO (Figure 4).

  * •

SCOPE has 1.35$\times$ lower violation magnitude than SCOPE-NO for all unsafe
configurations in the safe set. This suggests that even when an unsafe
configuration from SCOPE is chosen for reconfiguration, it will be likely to
generate lower magnitude violation than that from SCOPE-NO. This is reflected
in Figure 5 where SCOPE has the lowest overall violation magnitudes.

###  6.4. RQ4: How sensitive is SCOPE to $\gamma$?

The operator parameter $\gamma$ controls the size of neighborhood space that
SCOPE explores (Eq. 1). To understand the effects of $\gamma$ on SCOPE, we
conduct sensitivity analysis of $\gamma$ on violation rate and speedup. For
better visualization, we normalize each hardware parameter by mean
normalization such that each parameter is within range $[-0.5,0.5]$ (Alam et
al., 2011). Figure 8 shows violation rate and speedup as a function of
$\gamma$, where the x-axis is the different values of $\gamma$ and the y-axis
is violation rate and speedup. Note that when $\gamma=0$, this is equivalent
to the Static approach since SCOPE must always choose the initial safe
configuration. In particular, we observe the following:

  * •

There is a tradeoff between violation rate and speedup; lower violation rate
is likely to have lower speedup and higher violation rate is likely to have
higher speedup, where ideally, we want low violation rate and high speedup.
For example, the average violation rate over all applications decreases from
3.04% (when $\gamma=0.5$) to 0.93% (when $\gamma=1$), while average speedup of
1.75 (when $\gamma=0.5$) also decreases to 1.60 (when $\gamma=1$). The
tradeoff occurs because usually when a violation occurs, higher power is
consumed, and more work is performed, allowing the application to finish
earlier.

  * •

The violation rate is generally higher when $0<\gamma<1$, compared to when
$\gamma\geq 1$. This is because we normalize all hardware parameters to range
from -0.5 to 0.5. Given binary parameters such as hyperthreading (on, off) and
number of sockets (1, 2), SCOPE needs $\gamma$ to be at least 1 to consider
updating those parameters. When $0<\gamma<1$, SCOPE is not enabled to change
hyperthreading or number of sockets at all, which limits the search space for
SCOPE to explore safely.

Figure 9. Violation rate as a function of time interval averaged over all
power constraints and applications for each approach. Figure 10. Violation
rate and speedup of SCOPE using different types of learning models, averaged
over all power constraints for each application. Lower is better for violation
rate. Higher is better for speedup. Figure 11. Overhead of processing each
sample for per reconfiguration averaged over all power constraints for each
application. Lower is better.

###  6.5. RQ5: How sensitive is SCOPE to time intervals?

For all approaches that reconfigure dynamically, we evaluate how they are
affected by different time intervals (i.e., the time period between
reallocations). We sweep over different time intervals $[5,10,20,30]$ seconds
and choose the best time interval (i.e. the interval that has the lowest
violation rates across all power constraints) for each method and each
application.

Figure 9 summarizes the violation rate over all power constraints and
applications as a function of time interval, where the x-axis is the different
time intervals, and the y-axis is the violation rate.

Compared to Offline, BO, and StageOPT, SCOPE has both the lowest violation
rate and the smallest variance at all intervals (SCOPE-NO is the second best
and very close to SCOPE). It is because SCOPE and SCOPE-NO construct safe set
and optimize latency together for each iteration, while BO optimizes for
latency only, and StageOPT separates expanding safe set and optimizing latency
into two separate stages. Critically, StageOPT only constructs the safe set at
the beginning of execution, so if a previously safe configuration becomes
unsafe, then StageOPT has no way to react. SCOPE is even better than SCOPE-NO
because it utilizes the locality preserving operator to reconfigure more
safely, which helps reduce the overall violation rate. These results show that
the novel safe exploration framework of SCOPE is robust to the reconfiguring
time intervals.

###  6.6. RQ6: How do different types of models perform?

SCOPE is a general framework compatible with any type of learning model.
Although Gaussian process regression is chosen as the learning models in all
our evaluations, we test the framework’s generality by using 4 common models:
multi-layer perceptron (Ïpek et al., 2006; Ipek et al., 2005) (SCOPE-MLP),
linear regression (Ding et al., 2021) (SCOPE-LINEAR), random forest (Nardi et
al., 2019; Chen et al., 2021; Roy et al., 2021) (SCOPE-RF), and Gaussian
process regression (Alipourfard et al., 2017; Patel and Tiwari, 2020) (SCOPE).
Note that for this experiment, we use the same type of models for training
both the safety and objective models. Figure 10 summarizes the average
violation rate and speedup, where the x-axis is the application and the y-axis
is either the average violation rate or speedup, with the last column Mean
being the arithmetic mean over all applications. We find that:

  * •

SCOPE has the lowest violation rate compared to other learning models. On
average, SCOPE has 4.54$\times$ lower violation rate than SCOPE-MLP,
10.82$\times$ lower violation rate than SCOPE-LINEAR and 12.0$\times$ lower
violation rate than SCOPE-RF.

  * •

The speedup from using different learning models is relatively similar. SCOPE
achieves 1.43$\times$ speedup over SCOPE-MLP. SCOPE-LINEAR and SCOPE-RF are
faster than SCOPE by 1% and 7% respectively. However, they are not good for
being safe since both SCOPE-LINEAR and SCOPE-RF have significantly higher
violations than SCOPE.

###  6.7. RQ7: What are the overheads?

We report the overhead of processing each sample, which includes updating the
learning models to predict future system behavior and choosing a new
configuration. Figure 11 shows the average overhead of processing each sample
for each application by different approaches, and the last column Mean is the
arithmetic mean over all applications. Static, Oracle and RAPL approaches were
omitted; Static and Oracle do not process samples or reconfigure and RAPL uses
Intel’s power control system to tune parameters in the background and thus the
overhead is not measurable. In particular, we find that on average:

  * •

BO has the lowest overhead of 0.03s because it only trains one objective model
without considering safety.

  * •

SCOPE has the second lowest average overhead and is better than SCOPE-NO
because of the locality preserving operator that reduces the number of
configurations for inference in predicting safe set.

  * •

The overhead of SCOPE, which is 0.05s on average, is negligible given the
total execution time. This is validated by SCOPE achieving lowest latency even
though we include overheads in all experiments. These overheads could be
further reduced in future work by porting the resource manager to a lower-
level language.

##  7\. Limitations

We note the following limitations for this work:

  * •

SCOPE, despite achieving the lowest violation rates and magnitudes, does not
predict the complete safe set or provide any formal guarantees. Future work
can explore formal guarantee for constructing the safe set.

  * •

Although Gaussian process regression provides confidence interval for
prediction, such information was not used in SCOPE. Future work can explore
techniques for incorporating uncertainty for safe exploration.

  * •

SCOPE currently uses fixed values for parameters such as time interval and the
operator parameter $\gamma$ throughout execution. Future work can explore
adaptively changing these parameters during execution to further reducing
safety violations.

##  8\. Conclusion

This paper presents SCOPE, a resource manager that leverages a novel safe
exploration framework that dynamically allocate hardware resources in the
execution space. SCOPE introduces a locality preserving operator that reduces
the violation rate and magnitudes compared to prior work. We hope this work
can inspire the computer systems community to build new optimization tools
that aid in exploration while preserving safety.

## References

  * Deng et al. [2017] Zhaoxia Deng, Lunkai Zhang, Nikita Mishra, Henry Hoffmann, and Frederic T Chong.  Memory cocktail therapy: A general learning-based framework to optimize dynamic tradeoffs in nvms.  In _Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture_ , pages 232–244, 2017.
  * Rahmani et al. [2018] Amir M Rahmani, Bryan Donyanavard, Tiago Mück, Kasra Moazzemi, Axel Jantsch, Onur Mutlu, and Nikil Dutt.  Spectr: Formal supervisory control and coordination for many-core systems resource management.  In _Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 169–183, 2018.
  * Zhang et al. [2021] Yanqi Zhang, Weizhe Hua, Zhuangzhuang Zhou, G Edward Suh, and Christina Delimitrou.  Sinan: Ml-based and qos-aware resource management for cloud microservices.  In _Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 167–181, 2021.
  * Yuan and Nahrstedt [2003] Wanghong Yuan and Klara Nahrstedt.  Energy-efficient soft real-time cpu scheduling for mobile multimedia systems.  _ACM SIGOPS Operating Systems Review_ , 37(5):149–163, 2003.
  * Fan et al. [2016] Songchun Fan, Seyed Majid Zahedi, and Benjamin C Lee.  The computational sprinting game.  _ACM SIGARCH Computer Architecture News_ , 44(2):561–575, 2016.
  * Mishra et al. [2018] Nikita Mishra, Connor Imes, John D. Lafferty, and Henry Hoffmann.  Caloree: Learning control for predictable latency and low energy.  In _Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems_ , ASPLOS’18, pages 184–198, New York, NY, USA, 2018.  ISBN 9781450349116.
  * Raghavendra et al. [2008] Ramya Raghavendra, Parthasarathy Ranganathan, Vanish Talwar, Zhikui Wang, and Xiaoyun Zhu.  No” power” struggles: coordinated multi-level power management for the data center.  In _Proceedings of the 13th international conference on Architectural support for programming languages and operating systems_ , pages 48–59, 2008.
  * Sakalkar et al. [2020] Varun Sakalkar, Vasileios Kontorinis, David Landhuis, Shaohong Li, Darren De Ronde, Thomas Blooming, Anand Ramesh, James Kennedy, Christopher Malone, Jimmy Clidaras, et al.  Data center power oversubscription with a medium voltage power plane and priority-aware capping.  In _Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 497–511, 2020.
  * Li et al. [2020] Shaohong Li, Xi Wang, Faria Kalim, Xiao Zhang, Sangeetha Abdu Jyothi, Karan Grover, Vasileios Kontorinis, Nina Narodytska, Owolabi Legunsen, Sreekumar Kodakara, et al.  Thunderbolt:$\\{$Throughput-Optimized$\\}$,$\\{$Quality-of-Service-Aware$\\}$ power capping at scale.  In _14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20)_ , pages 1241–1255, 2020.
  * Ahmad and Vijaykumar [2010] Faraz Ahmad and TN Vijaykumar.  Joint optimization of idle and cooling power in data centers while maintaining response time.  _ACM Sigplan Notices_ , 45(3):243–256, 2010.
  * Ïpek et al. [2006] Engin Ïpek, Sally A McKee, Rich Caruana, Bronis R de Supinski, and Martin Schulz.  Efficiently exploring architectural design spaces via predictive modeling.  _ACM SIGOPS Operating Systems Review_ , 40(5):195–206, 2006.
  * Mishra et al. [2015] Nikita Mishra, Huazhe Zhang, John D. Lafferty, and Henry Hoffmann.  A probabilistic graphical model-based approach for minimizing energy under performance constraints.  In _Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems_ , ASPLOS’15, pages 267–281, New York, NY, USA, 2015.  ISBN 9781450328357.
  * Hoffmann [2015] Henry Hoffmann.  Jouleguard: energy guarantees for approximate applications.  In _Proceedings of the 25th Symposium on Operating Systems Principles_ , pages 198–214, 2015.
  * Ding et al. [2019] Yi Ding, Nikita Mishra, and Henry Hoffmann.  Generative and multi-phase learning for computer systems optimization.  In _Proceedings of the 46th International Symposium on Computer Architecture_ , pages 39–52, 2019.
  * Ding et al. [2021] Yi Ding, Ahsan Pervaiz, Michael Carbin, and Henry Hoffmann.  Generalizable and interpretable learning for configuration extrapolation.  In _Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , pages 728–740, 2021.
  * Yu et al. [2018] Zhibin Yu, Zhendong Bei, and Xuehai Qian.  Datasize-aware high dimensional configurations auto-tuning of in-memory cluster computing.  In _Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 564–577, 2018.
  * Sui et al. [2015] Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause.  Safe exploration for optimization with gaussian processes.  In _International conference on machine learning_ , pages 997–1005. PMLR, 2015.
  * Sui et al. [2018] Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue.  Stagewise safe bayesian optimization with gaussian processes.  In _International conference on machine learning_ , pages 4781–4789. PMLR, 2018.
  * Turchetta et al. [2019] Matteo Turchetta, Felix Berkenkamp, and Andreas Krause.  Safe exploration for interactive machine learning.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Belkin and Niyogi [2003] Mikhail Belkin and Partha Niyogi.  Laplacian eigenmaps for dimensionality reduction and data representation.  _Neural computation_ , 15(6):1373–1396, 2003\.
  * [21] Apache spark.  https://spark.apache.org/docs/2.2.3/configuration.html.
  * David et al. [2010] Howard David, Eugene Gorbatov, Ulf R Hanebutte, Rahul Khanna, and Christian Le.  Rapl: Memory power estimation and capping.  In _2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED)_ , pages 189–194. IEEE, 2010.
  * Penney and Chen [2019] Drew D Penney and Lizhong Chen.  A survey of machine learning applied to computer architecture design.  _arXiv preprint arXiv:1909.12373_ , 2019.
  * Zhang and Huang [2019] Yiying Zhang and Yutong Huang.  ”learned”: Operating systems.  _Operating Systems Review_ , 53(1):40–45, 2019\.
  * Belay et al. [2014] Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion.  Ix: A protected dataplane operating system for high throughput and low latency.  In _11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14)_ , pages 49–65, 2014.  doi: 10.1145/2997641.
  * Lee and Brooks [2006] Benjamin C Lee and David M Brooks.  Accurate and efficient regression modeling for microarchitectural performance and power prediction.  _ACM SIGOPS operating systems review_ , 40(5):185–194, 2006.
  * Yi et al. [2003] Joshua J Yi, David J Lilja, and Douglas M Hawkins.  A statistically rigorous approach for improving simulation methodology.  In _The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings._ , pages 281–291. IEEE, 2003.
  * Bitirgen et al. [2008] Ramazan Bitirgen, Engin Ipek, and Jose F Martinez.  Coordinated management of multiple interacting resources in chip multiprocessors: A machine learning approach.  In _2008 41st IEEE/ACM International Symposium on Microarchitecture_ , pages 318–329. IEEE, 2008.
  * Ansel et al. [2011] Jason Ansel, Yee Lok Wong, Cy Chan, Marek Olszewski, Alan Edelman, and Saman Amarasinghe.  Language and compiler support for auto-tuning variable-accuracy algorithms.  In _International Symposium on Code Generation and Optimization (CGO 2011)_ , pages 85–96. IEEE, 2011.
  * Chen et al. [2011] Jian Chen, Lizy Kurian John, and Dimitris Kaseridis.  Modeling program resource demand using inherent program characteristics.  _ACM SIGMETRICS Performance Evaluation Review_ , 39(1):1–12, 2011.
  * Cochran et al. [2011] Ryan Cochran, Can Hankendi, Ayse K Coskun, and Sherief Reda.  Pack & cap: adaptive dvfs and thread packing under power caps.  In _2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)_ , pages 175–185. IEEE, 2011.
  * Oliner et al. [2013] Adam J Oliner, Anand P Iyer, Ion Stoica, Eemil Lagerspetz, and Sasu Tarkoma.  Carat: Collaborative energy diagnosis for mobile devices.  In _Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems_ , pages 1–14, 2013.
  * Ansel et al. [2012] Jason Ansel, Maciej Pacula, Yee Lok Wong, Cy Chan, Marek Olszewski, Una-May O’Reilly, and Saman Amarasinghe.  Siblingrivalry: online autotuning through local competitions.  In _Proceedings of the 2012 international conference on Compilers, architectures and synthesis for embedded systems_ , pages 91–100, 2012\.
  * Ponomarev et al. [2001] Dmitry Ponomarev, Gurhan Kucuk, and Kanad Ghose.  Reducing power requirements of instruction scheduling through dynamic allocation of multiple datapath resources.  In _Proceedings. 34th ACM/IEEE International Symposium on Microarchitecture. MICRO-34_ , pages 90–101. IEEE, 2001.
  * Sridharan et al. [2013] Srinath Sridharan, Gagan Gupta, and Gurindar S Sohi.  Holistic run-time parallelism management for time and energy efficiency.  In _Proceedings of the 27th international ACM conference on International conference on supercomputing_ , pages 337–348, 2013.
  * Garza et al. [2019] Elba Garza, Samira Mirbagher-Ajorpaz, Tahsin Ahmad Khan, and Daniel A Jiménez.  Bit-level perceptron prediction for indirect branches.  In _2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA)_ , pages 27–38. IEEE, 2019.
  * Bhatia et al. [2019] Eshan Bhatia, Gino Chacon, Seth Pugsley, Elvira Teran, Paul V Gratz, and Daniel A Jiménez.  Perceptron-based prefetch filtering.  In _2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA)_ , pages 1–13. IEEE, 2019.
  * Shi et al. [2019] Zhan Shi, Xiangru Huang, Akanksha Jain, and Calvin Lin.  Applying deep learning to the cache replacement problem.  In _Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture_ , pages 413–425, 2019.
  * Shi et al. [2021] Zhan Shi, Akanksha Jain, Kevin Swersky, Milad Hashemi, Parthasarathy Ranganathan, and Calvin Lin.  A hierarchical neural model of data prefetching.  In _Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 861–873, 2021.
  * Lee et al. [2008] Benjamin C Lee, Jamison Collins, Hong Wang, and David Brooks.  Cpr: Composable performance regression for scalable multiprocessor models.  In _2008 41st IEEE/ACM International Symposium on Microarchitecture_ , pages 270–281. IEEE, 2008.
  * Delimitrou and Kozyrakis [2013] Christina Delimitrou and Christos Kozyrakis.  Paragon: Qos-aware scheduling for heterogeneous datacenters.  In _Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems_ , ASPLOS’13, pages 77–88, New York, NY, USA, 2013.  ISBN 9781450318709.
  * Delimitrou and Kozyrakis [2014] Christina Delimitrou and Christos Kozyrakis.  Quasar: Resource-efficient and qos-aware cluster management.  In _Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems_ , ASPLOS’14, pages 127–144, New York, NY, USA, 2014.  ISBN 9781450323055.
  * Ipek et al. [2008] Engin Ipek, Onur Mutlu, José F. Martínez, and Rich Caruana.  Self-optimizing memory controllers: A reinforcement learning approach.  In _Proceedings of the 35th Annual International Symposium on Computer Architecture_ , ISCA’08, pages 39–50, USA, 2008.  ISBN 9780769531748.
  * Petrica et al. [2013] Paula Petrica, Adam M Izraelevitz, David H Albonesi, and Christine A Shoemaker.  Flicker: A dynamically adaptive architecture for power limited multicore systems.  In _Proceedings of the 40th Annual International Symposium on Computer Architecture_ , pages 13–23, 2013.
  * Venkataraman et al. [2016] Shivaram Venkataraman, Zongheng Yang, Michael Franklin, Benjamin Recht, and Ion Stoica.  Ernest: Efficient performance prediction for large-scale advanced analytics.  In _13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16)_ , pages 363–378, 2016.
  * Canino et al. [2018] Anthony Canino, Yu David Liu, and Hidehiko Masuhara.  Stochastic energy optimization for mobile gps applications.  In _Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , pages 703–713, 2018.
  * Frazier [2018] Peter I Frazier.  A tutorial on bayesian optimization.  _arXiv preprint arXiv:1807.02811_ , 2018.
  * Alipourfard et al. [2017] Omid Alipourfard, Hongqiang Harry Liu, Jianshu Chen, Shivaram Venkataraman, Minlan Yu, and Ming Zhang.  Cherrypick: Adaptively unearthing the best cloud configurations for big data analytics.  In _Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation_ , NSDI’17, page 469–482, USA, 2017. USENIX Association.  ISBN 9781931971379.
  * Patel and Tiwari [2020] Tirthak Patel and Devesh Tiwari.  Clite: Efficient and qos-aware co-location of multiple latency-critical jobs for warehouse scale computers.  In _2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)_ , pages 193–206, 2020.  doi: 10.1109/HPCA47549.2020.00025.
  * Roy et al. [2021] Rohan Basu Roy, Tirthak Patel, Vijay Gadepally, and Devesh Tiwari.  Bliss: auto-tuning complex applications using a pool of diverse lightweight learning models.  In _Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation_ , pages 1280–1295, 2021.
  * Nardi et al. [2019] Luigi Nardi, David Koeplinger, and Kunle Olukotun.  Practical design space exploration.  In _2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)_ , pages 347–358. IEEE, 2019.
  * Chen et al. [2021] Junjie Chen, Ningxin Xu, Peiqi Chen, and Hongyu Zhang.  Efficient compiler autotuning via bayesian optimization.  In _2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)_ , pages 1198–1209. IEEE, 2021.
  * Li and Martinez [2006] Jian Li and Jose F Martinez.  Dynamic power-performance adaptation of parallel computation on chip multiprocessors.  In _The Twelfth International Symposium on High-Performance Computer Architecture, 2006._ , pages 77–87. IEEE, 2006.
  * Dubach et al. [2010] Christophe Dubach, Timothy M Jones, Edwin V Bonilla, and Michael FP O’Boyle.  A predictive model for dynamic microarchitectural adaptivity control.  In _2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture_ , pages 485–496. IEEE, 2010.
  * Razin and Feigh [2020] Yosef S Razin and Karen M Feigh.  Hitting the road: Exploring human-robot trust for self-driving vehicles.  In _2020 IEEE International Conference on Human-Machine Systems (ICHMS)_ , pages 1–6. IEEE, 2020.
  * Brunke et al. [2021] Lukas Brunke, Melissa Greeff, Adam W Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, and Angela P Schoellig.  Safe learning in robotics: From learning-based control to safe reinforcement learning.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 5, 2021\.
  * Garcia and Fernández [2012] Javier Garcia and Fernando Fernández.  Safe exploration of state and action spaces in reinforcement learning.  _Journal of Artificial Intelligence Research_ , 45:515–564, 2012.
  * Mao et al. [2019] Hongzi Mao, Malte Schwarzkopf, Hao He, and Mohammad Alizadeh.  Towards safe online reinforcement learning in computer systems.  In _Workshop on ML for Systems In conjunction with 33rd conference on neural information processing systems_ , 2019.
  * Turchetta et al. [2020] Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, and Alekh Agarwal.  Safe reinforcement learning via curriculum induction.  _Advances in Neural Information Processing Systems_ , 33:12151–12162, 2020.
  * Wachi and Sui [2020] Akifumi Wachi and Yanan Sui.  Safe reinforcement learning in constrained markov decision processes.  In _International Conference on Machine Learning_ , pages 9797–9806. PMLR, 2020.
  * Xu et al. [2021] Wanqiao Xu, Kan Xu, Hamsa Bastani, and Osbert Bastani.  Safely bridging offline and online reinforcement learning.  _arXiv preprint arXiv:2110.13060_ , 2021.
  * Zhang and Hoffman [2015] Huazhe Zhang and H Hoffman.  A quantitative evaluation of the rapl power control system.  _Feedback Computing_ , 6, 2015.
  * Huang et al. [2010] Shengsheng Huang, Jie Huang, Yan Liu, Lan Yi, and Jinquan Dai.  Hibench: A representative and comprehensive hadoop benchmark suite.  In _Proc. ICDE Workshops_ , pages 41–51, 2010.
  * Keahey et al. [2020] Kate Keahey, Jason Anderson, Zhuo Zhen, Pierre Riteau, Paul Ruth, Dan Stanzione, Mert Cevik, Jacob Colleran, Haryadi S. Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha, and Joe Stubbs.  Lessons learned from the chameleon testbed.  In _Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC ’20)_. USENIX Association, July 2020.
  * Temam [2012] Olivier Temam.  A defect-tolerant accelerator for emerging high-performance applications.  In _2012 39th Annual International Symposium on Computer Architecture (ISCA)_ , pages 356–367. IEEE, 2012.
  * Reagen et al. [2016] Brandon Reagen, Paul Whatmough, Robert Adolf, Saketh Rama, Hyunkwang Lee, Sae Kyu Lee, José Miguel Hernández-Lobato, Gu-Yeon Wei, and David Brooks.  Minerva: Enabling low-power, highly-accurate deep neural network accelerators.  In _2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA)_ , pages 267–278. IEEE, 2016.
  * Harris et al. [2020] Charles R. Harris, K. Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant.  Array programming with numpy.  _Nature_ , 585:357–362, 2020.
  * pandas development team [2020] The pandas development team.  pandas-dev/pandas: Pandas, February 2020.
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830, 2011\.
  * Zhang and Hoffmann [2016] Huazhe Zhang and Henry Hoffmann.  Maximizing performance under a power cap: A comparison of hardware, software, and hybrid techniques.  _ACM SIGPLAN Notices_ , 51(4):545–559, 2016.
  * Banerjee et al. [2021] Subho S Banerjee, Saurabh Jha, Zbigniew Kalbarczyk, and Ravishankar K Iyer.  Bayesperf: minimizing performance monitoring errors using bayesian statistics.  In _Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems_ , pages 832–844, 2021.
  * Wang et al. [2019] Yuzhao Wang, Lele Li, You Wu, Junqing Yu, Zhibin Yu, and Xuehai Qian.  Tpshare: a time-space sharing scheduling abstraction for shared cloud via vertical labels.  In _2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA)_ , pages 499–512. IEEE, 2019.
  * Alam et al. [2011] Md Jahangir Alam, Pierre Ouellet, Patrick Kenny, and Douglas O’Shaughnessy.  Comparative evaluation of feature normalization techniques for speaker verification.  In _International Conference on Nonlinear Speech Processing_ , pages 246–253. Springer, 2011.
  * Ipek et al. [2005] Engin Ipek, Bronis R De Supinski, Martin Schulz, and Sally A McKee.  An approach to performance prediction for parallel applications.  In _European Conference on Parallel Processing_ , pages 196–205. Springer, 2005.

##  9\. Appendix

This appendix contains the following items:

  * •

section 9.1 breaks down the results of violation rate, violation magnitude,
and speedup in different safety constraint values.

  * •

section 9.2 shows how the number of samples used in training Offline on its
performance in optimizing for latency under power constraints.

###  9.1. Evaluation on Different Safety Constraints

In section 6.1 and section 6.2, we show the summarized results of each
approach by taking the average across different power constraints. Here, we
break down the results of each approach at each safety constraint to show how
SCOPE is sensitive to the safety constraint.

#### Violation rate and magnitude.

Figures 12 and 13 show the violation rates and magnitudes for different
approaches under different power constraints (p40-80), where the x-axis is the
application, the y-axis is the violation rate or magnitude, the last column
Mean is the arithmetic mean over all applications for Figures 12 and
arithmetic mean over all applications with non-zero violation magnitude for
Figure 13, and we put the number on each bar to quantify the mean results. To
visualize better, we cap the violation rate at 5% in Figure 12 and put numbers
on the bars that are capped.

We observe that SCOPE has the lowest violation rate and magnitude at almost
all power constraints. These results explicitly show that SCOPE has a
dominating advantage over other approaches no matter how we set the safety
constraint.

Figure 12. Violation rates averaged over all random starting configurations
for each power constraint and application. Lower is better. Figure 13.
Violation magnitudes averaged over all random starting configurations for each
power constraint and application. Lower is better.

#### Speedup.

Figure 14 shows the speedups over Static for different approaches under
different power constraints (p40-80), where the x-axis is the application, the
y-axis is the speedup, the last column Mean is the arithmetic mean over all
applications, and we put the number on each bar to quantify the mean results.
We observe that SCOPE is second to Oracle in all but the p50 power constraint,
where Offline has slightly higher speedup. This suggests that SCOPE’s
dominating advantage is robust to the value of safety constraint.

Figure 14. Speedup averaged over all random starting configurations for each
power constraint and application. Higher is better. Figure 15. Violation rates
and speedup of Offline using different amounts of samples, averaged over all
power constraints for each application. Lower is better for violation rate.
Higher is better for speedup.

###  9.2. Sensitivity Analysis for Offline Approach

The Offline approach that is compared against SCOPE in section 5.3 samples 50%
of all possible hardware resource allocations from the target application and
input for training. This section shows how Offline is affected by the number
of samples used. We sweep over different amount of samples, [25, 50, 75,
100]-% of all possible hardware resource allocations using the target
application and input. Figure 15 summarizes the average violation rate and
speedup, where the x-axis is the application, the y-axis is the violation rate
or speedup, the last column Mean is the arithmetic mean over all applications,
and we put the number on each bar to quantify the mean results. We observe
that using higher amount of samples has increased speedup for Offline.
However, using higher amount of samples does not necessarily lead to lower
violation rate, further demonstrating the difficulty of generalization of the
a priori collected samples.