# A case study of proactive auto-scaling for an ecommerce workload

Marcella Medeiros Siqueira Coutinho de Almeida
<EMAIL_ADDRESS>Federal University of Campina GrandeCampina
GrandeParaíbaBrazil ,  Thiago Emmanuel Pereira
<EMAIL_ADDRESS>Federal University of Campina GrandeCampina
GrandeParaíbaBrazil and  Fabio Morais<EMAIL_ADDRESS>Federal
University of Campina GrandeCampina GrandeParaíbaBrazil

(2022)

###### Abstract.

Preliminary data obtained from a partnership between the Federal University of
Campina Grande and an ecommerce company indicates that some applications have
issues when dealing with variable demand. This happens because a delay in
scaling resources leads to performance degradation and, in literature, is a
matter usually treated by improving the auto-scaling. To better understand the
current state-of-the-art on this subject, we re-evaluate an auto-scaling
algorithm proposed in the literature, in the context of ecommerce, using a
long-term real workload. Experimental results show that our proactive approach
is able to achieve an accuracy of up to 94 percent and led the auto-scaling to
a better performance than the reactive approach currently used by the
ecommerce company.

Cloud computing, auto-scaling, workload prediction, ARIMA

††copyright: acmcopyright††journalyear: 2022

##  1\. Introduction

Cloud computing refers to a model in which configurable computing resources
(e.g., networks, servers, storage, applications, and services) are outsourced
on an on-demand pay-per-use basis (Ilyushkin et al., 2018). Thus, the
computing resources could be rapidly provisioned and released with minimal
management effort or service provider interaction, providing an elastic
infrastructure (Mell et al., 2011). The auto-scaling techniques use this
elasticity to automatically adjust resources based on the application demand
running in the cloud infrastructure.

Specifically, auto-scaling can be defined as the process of dynamically
adapting the resources assigned to the elastic applications, depending on the
input workload, without the intervention of a human manager (Lorido-Botran et
al., 2014; Morais et al., 2013; Morais et al., 2017). In Amazon Web Services
(AWS), this feature allows users to automatically launch or terminate virtual
instances using its EC2 service based on predefined policies, schedules, and
regular health status checks (Radhika and Sadasivam, 2021).

There are two ways of scaling any application: vertical scaling and horizontal
scaling. The first one works on increasing (scale-up) and decreasing (scale-
down) the number of computing resources assigned to each application instance,
while the latter scales by increasing (scale-out) and decreasing (scale-in)
the number of application instances (Rossi et al., 2019).

As for the classification of auto-scalers, when grouped by their anticipation
capabilities (Lorido-Botran et al., 2014), it can be divided into two classes:
reactive and proactive. In reactive, the user has to specify thresholds for
workload metrics, meaning that scaling only occurs after such thresholds are
exceeded. In proactive, an application takes early scaling decisions based on
estimates of future demand (Shariffdeen et al., 2016).

Both strategies have trade-offs to take into account. While the reactive
strategy is simple and intuitive (Radhika and Sadasivam, 2021) (as consequence
is broadly utilized in commercial cloud providers (Lorido-Botran et al.,
2014)), the time taken by the auto-scaler to react and allocate resources
could be too long to avoid performance degradation. On the other hand, the
drawback of a proactive technique is that its reliability depends on the
accuracy of the predicted values (Shariffdeen et al., 2016).

The literature has shown a variety of approaches, evaluations, and comparisons
between these two models and their own subcategories (Lorido-Botran et al.,
2014). To the best of our knowledge, none of them has come to a conclusion
about which one is better in terms of performance for ecommerce companies. In
order to provide a better understanding of this matter, our goal is to
evaluate the reactive approach already used by an ecommerce company compared
to a proactive approach using the same configurations, simulating both auto-
scaling behaviors with a real collected workload. For that, we implemented a
time-series analysis technique, Autoregressive Integrated Moving Average.

Results show that our implemented model achieves an accuracy of 94% and
deviates very little from the actual data, less than 7%. Even though a good
prediction does not necessarily mean the model would make good allocation
decisions, our performance results were pretty optimistic when using our
forecast model in the auto-scaling simulation, since they were highly better
than the ones from the reactive approach.

The rest of this work is structured as follows: Section 2 presents previous
works on evaluating auto-scaling techniques, Section 3 exposes our prediction
model, its implementation, and metrics for analysis, and Section 4 discusses
our experimental results. Finally, Section 5 discloses our conclusions and
possible future work.

##  2\. Related Work

Auto-scaling and its techniques have been studied in many past works. Lorido-
Botran et al. proposed a classification of these techniques into five main
categories: static threshold-based rules, control theory, reinforcement
learning, queuing theory, and time series analysis, using this classification
for a literature review of previous proposals for auto-scaling (Lorido-Botran
et al., 2014). More recently, Radhika and Sadasivam also conducted a
comprehensive study on existing auto-scaling techniques, comparing newer
proactive and reactive auto-scaling strategies but discussing possible
research solutions for hybrid environments, too (Radhika and Sadasivam, 2021).
Both these studies do not evaluate the suggested approaches.

On the other hand, Ilyushkin et al. presented a detailed comparative study of
general state-of-the-art auto-scaling policies, this time conducting various
experiments and comparing their performance in pairwise and group comparisons
(Ilyushkin et al., 2018). Their work includes no predictive auto-scaling
policies in their experiments and discussions, only reactive and hybrid
techniques.

As for studies comparing proactive auto-scaling approaches, Wang et al.
proposed a predictive technique as a better alternative to the reactive one
already used by Amazon EKS, with experiments using real-world workloads (Wang
et al., 2021). Still, they deal only with vertical auto-scaling and a workload
with a duration of one day. Islam et al. (Islam et al., 2012) explored two
predictive algorithms related to time-series analysis, incorporating sliding
window to the training and prediction stages. Their benchmark is 135 minutes,
ending with the same issue as Wang et al. (Wang et al., 2021) of using a short
trace.

Although Gao et al. used an extensive trace from Google Cluster Trace and
evaluated different approaches of proactive auto-scaling (e.g., statistical,
machine learning, and deep learning approaches), they did not deepen how ARIMA
was implemented as the statistical approach (Gao et al., 2020). Calheiros et
al. implemented and compared ARIMA-based prediction to a trace of one week of
an actual workload, but the workload of an ecommerce company was not included
in their analysis (Calheiros et al., 2014).

Considering the current state-of-art, most auto-scaling techniques have been
evaluated using traces typically short spanning a few minutes, hours, or days
(Papadopoulos et al., 2016), and none of these evaluations were made testing a
workload of ecommerce directly. With the crescent use of cloud computing and
auto-scaling strategies for improvement, our work differs from the previous
articles mentioned as it aims at analyzing which approach has a better
performance for ecommerce traffic, conducting experiments with two weeks of
data from an ecommerce company workload.

##  3\. Methodology

The main goal of our work is to evaluate a proactive auto-scaling strategy
compared to the reactive approach already used by an ecommerce company. To do
so, we explored several existing predictive techniques through a
bibliographical review, searching for which algorithm would fit better with
the trace and metrics used by the ecommerce company from where the data were
collected. We considered proactive models utilizing horizontal scaling and CPU
load as the metric.

To better describe our methodology, the remainder of this section is divided
as follows: Section 3.1 describes our dataset and how the data were collected.
Sections 3.2 and 3.3 explains the reactive and the proactive strategies
evaluated in this work, respectively, while Sections 3.4 and 3.5 detail how we
implemented and simulated the auto-scaling behavior with our chosen
techniques. Finally, Section 3.6 defines each metric used for our evaluation,
of both prediction and performance.

###  3.1. Data collected

Our data were obtained from an application of the ecommerce company in which
its current team believes that the application uses more virtual machines than
needed at some moments of the day. We saw in this an opportunity to evaluate
the impact of changing from a reactive auto-scaling strategy to a proactive
one in case of significant infrastructure consumption. The data was collected
in February 2022 from AWS Cloud Watch, with a trace of two weeks, from
February 1st to 17th. The dataset we are working with has three important
variables:

  * •

`timestamp`: the moment of each observation, with a data grain of 60 seconds;

  * •

`instance_type`: the type of the instance, used to calculate the number of
vCPUs;

  * •

`utilization`: the CPU utilization in that given moment.

This means that each minute we have new data to consider in our forecast,
leaving us with historical data of more than 10 thousand observations to
process before predicting. As for “`instance_type`”, the application we are
analyzing uses Amazon EC2 instance types with four vCPUS available each (ec2,
[n. d.]a), and its auto-scaling configurations will be discussed in the
following section.

###  3.2. Reactive strategy

Widely studied in the past, reactive auto-scaling is usually based on
threshold-based rules techniques and tends to deal with some intrinsic
problems such as using cooldown times (also called inertia or calm) or dynamic
thresholds (Rampérez et al., 2021). The idea behind using the cooldown period
is to prevent the auto-scaling from launching or terminating additional
instances before the effects of previous activities are visible. The strategy
of cooldown is not used for scheduled, periodic scaling actions or, in our
case, predictive scaling.

In this study, we evaluate simple scaling, i.e., the current reactive strategy
used by our chosen application. On Amazon EC2, this policy allows the user to
create CloudWatch alarms for the scaling policies and specify the high and low
thresholds for these alarms. A CloudWatch alarm is a feature that sends
notifications when the chosen metrics fall outside of the levels (high or low
thresholds) configured by the user. The main issue with simple scaling is, as
mentioned before, the presence of cooldown periods: it must wait for the
scaling activity or health check replacement to complete and the cooldown
period to end before responding to additional alarms (ec2, [n. d.]b).

In our application, the simple scaling policy was configured as follows:

  * •

Due to confiability matters, the exact capacity numbers will be omitted. So,
it’s important to know the min and max capacities were of x and 7.5x
instances, respectively;

  * •

The evaluation period was 5 min;

  * •

The cooldown period was 6 min;

  * •

The scaling rules were based on CPU utilization. The lower threshold was 25%,
and the upper one was 50%. According to the rules:

    * –

when CPU utilization was lower than 25%, a number of y instances were removed;

    * –

when CPU utilization was higher than 50%, a number of 4y instances were added;

###  3.3. Proactive strategy

As it is hard to dynamically identify and adjust optimal threshold values to
maximize the utilization of the provisioned infrastructure, proactive auto-
scaling tries to offer the solution to some of these challenges (Iqbal et al.,
2018). For our evaluation of one of the proposed solutions in the literature,
we decided to take a better look at how a statistical approach would perform
in comparison to a reactive one, without the higher complexity that a machine
learning or deep learning technique would offer. Reviewing Calheiros et al.
work, we noticed that their studied algorithm, ARIMA, would fit our mentioned
parameters of horizontal scaling and CPU load, so we decided to reproduce this
approach (Calheiros et al., 2014).

Autoregressive Integrated Moving Average, i.e., ARIMA is a model utilized for
time series prediction in different fields such as finance and economics (Box
et al., 2015). It predicts a given time series based on its past values, which
“autoregressive” stands for, and the goal is to reach stationary data that is
not subject to seasonality (duk, [n. d.]). According to Hyndman and
Athanasopoulos (Hyndman and Athanasopoulos, 2018):

> If we combine differencing with autoregression and a moving average model,
> we obtain a non-seasonal ARIMA model. ARIMA is an acronym for AutoRegressive
> Integrated Moving Average (in this context, “integration” is the reverse of
> differencing). The full model can be written as
>
>
> $y^{\prime}_{t}=c+\phi_{1}y^{\prime}_{t-1}+\cdots+\phi_{p}y^{\prime}_{t-p}+\theta_{1}\varepsilon_{t-1}+\cdots+\theta_{q}\varepsilon_{t-q}+\varepsilon_{t}$
>
> where $y^{\prime}_{t}$ is the differenced series (it may have been
> differenced more than once). The “predictors” on the right hand side include
> both lagged values of $y_{t}$ and lagged errors. We call this an
> $ARIMA(p,d,q)$ model, where
>
>
> $p$ = order of the autoregressive part;
>
> $d$ = degree of first differencing involved;
>
> $q$ = order of the moving average part.

For our implementation of the ARIMA model, we used the auto.arima() function
in R. This function uses a variation of the Hyndman-Khandakar algorithm
(Hyndman and Khandakar, 2008), which combines unit root tests, minimization of
the AICc (Second-order Akaike Information Criterion), and MLE (Maximum
Likelihood Estimation) to obtain an ARIMA model. This function is used to
evaluate the model and estimate a configuration of the ARIMA parameters.

###  3.4. Algorithm implementation

We used the R package forecast for the prediction, passing our collected
historical data as a parameter in the auto.arima() function. In the context of
this work, the historical data to be passed in our function means the observed
number of CPU utilization in a determined timestamp, in our case, every 60
seconds, multiplied by the number of vCPUs available for each instance type,
divided by 100. This calculation gives us the CPU demand in that given moment,
and the function returns the predicted CPU demand in the horizon we settled as
a parameter.

In this scenario, the horizon is based on the application start-up time, i.e.,
the time the application takes to initialize a VM. We need to predict with
such an advance that by the time the application needs to add or remove
instances, our algorithm has already decided on the correct action considering
the time it would take for the start-up. The application chosen has a start-up
time of between 9 and 12 minutes, so we opted for the worst-case scenario and
settled our horizon to 12.

Another decision for our function was to incorporate the sliding window
technique, meaning that it will consider new input values throughout the loop
and leave older ones behind when predicting. The input $x$ is a vector of
resource usage samples over $k$ consecutive time intervals, and the predicted
output, $y$, is $r$ intervals ahead of this input window (Islam et al., 2012;
Dietterich, 2002). This left us with the following implementation:

Algorithm 1 Prediction model with ARIMA

1:$D$ is a list with size N ($N\in\mathbb{N}$) representing CPU demand, in d
should be the $d^{th}$ CPU demand in list ($d\in\mathbb{Q}^{+}$);

2:$D[i,j]$ is a sub sequence of CPU demand,
$D[i,j]=\\{d^{i},d^{i+1},...,d^{j-1},d^{j}\\}$;

3:$start$ and $end$ are variables that define the range of the sliding window
($start,end\in\mathbb{N}$);

4:$ArimaForecast(L,H)$ is a function that tries to predict the next $H$
elements of list $L$, returns the last element $p$ ($H\in\mathbb{N}$,
$p\in\mathbb{Q}^{+}$, $\exists_{i,j}D[i,j]=L$);

5:for each $i\in[initial\\_value,final\\_value]$ do

6: $prediction\leftarrow ArimaForecast(D[start,end],horizon)$;

7: if i + horizon ¡= N then

8: saves the last predicted value of $prediction$ in a new list M;

9: end if

10:end for

###  3.5. Auto-scaling simulation

To compare proactive and reactive auto-scaling techniques’ performances, we
need to simulate how each would behave with its respective policy
configurations. For that, we used an open-source auto-scaling simulator,
developed by the Laboratory of Distributed Systems (sim, [n. d.]), which
already provides the simple scaling policy, and implemented our proactive
scaling approach based on the function shown in the previous section.

This auto-scaling simulator receives a timestamp (the current time of the
observation), the instance type, and the CPU utilization as input, returning
the number of cores to be added or removed on the next observation based on
the chosen policy configuration. The configuration for the proactive technique
had a target value instead of upper and lower bounds like simple scaling, and
it was calculated as an approximated value between 25% and 50% (the current
configuration on our application), being settled as 35%. It should be noted
that this policy does not use cooldown periods when scaling.

Our model then had to predict the CPU demand for 12 minutes ahead and, to
reach the target of 35% in that future observation, it should decide to add or
remove cores. It receives a training file of historical data and a prediction
file so it gets new values as the sliding window goes by in the prediction.
For our experiment, we’ve run with a week of data in our training file (from
February 1st to 8th) and another week of data in our prediction file (from
February 9th to 17th), analyzing the results with the chosen metrics reported
in the next section.

###  3.6. Evaluation metrics

To evaluate our prediction’s accuracy, we chose different statistical metrics:
$R^{2}$ Prediction Accuracy, Mean Absolute Error (MAE), and Mean Absolute
Percentage Error (MAPE)(Kotz et al., 2005; Everitt and Skrondal, 2010). These
three metrics were selected because they are known for measuring accuracy in
forecast models, showing different perspectives of how the prediction
approximates the real data.

Since a high accuracy of the forecast does not necessarily imply a good
performance of our auto-scaling, we also need a metric to evaluate the results
of our simulation. The performance evaluation for both strategies was made
with the Auto-scaling Demand Index (ADI) metric (Netto et al., 2014), used for
analyzing and comparing system utilization. All these mentioned metrics will
be described in the following subsections.

####  3.6.1. $R^{2}$ Prediction Accuracy

The $R^{2}$ Prediction Accuracy measures how the prediction model approximates
the real data points, with its value being within the range from 0 to 1, which
means that the $R^{2}$ prediction accuracy of 1 indicates a perfect fit of the
fitted model (Islam et al., 2012). The formula for this metric is:

(1)
$R^{2}=1-\frac{\sum_{i=1}^{n}(\hat{y_{i}}-{y_{i}})^{2}}{\sum_{i=1}^{n}(\hat{y_{i}}-\bar{y})}$

Where $\hat{y}=\frac{1}{n}\sum_{i=1}^{n}{y_{i}}$, ${y_{i}}$ is the actual
output, $\hat{y_{i}}$ is the predicted output, and $n$ is the number of
observations.

####  3.6.2. Mean Absolute Error (MAE)

The Mean Absolute Error is a standard measure of forecast error in time series
analysis and is the mean of the absolute difference between a target value and
model prediction, i.e., the mean of all absolute errors. To reach this value,
the formula is:

(2) $MAE=\frac{1}{n}\sum_{i=1}^{n}\mid{\hat{y_{i}}}-{y_{i}}\mid$

Where $n$ is the number of errors and $\mid{\hat{y_{i}}}-{y_{i}}\mid$ the
absolute errors. This calculation tells us an average of how big of an error
it is expected from our prediction.

####  3.6.3. Mean Absolute Percentage Error (MAPE)

The metric Mean Absolute Percentage Error measures the accuracy of a forecast
model as a percentage and is given by the following formula:

(3)
$MAE=\frac{1}{n}\sum_{i=1}^{n}\frac{\mid{\hat{y_{i}}}-{y_{i}}\mid}{{y_{i}}}$

Where $n$ is the number of fitted points, ${y_{i}}$ is the actual value, and
$\hat{y_{i}}$ is the forecast value. The lower the MAPE value, the better the
prediction accuracy.

####  3.6.4. Auto-scaling Demand Index (ADI)

Auto-scaling Demand Index is a performance metric for evaluating auto-scaling
strategies. This metric is defined by the sum of all distances computed
between each utilization level reported by the system and the target
utilization interval, with upper and lower bounds (Netto et al., 2014). The
closer the result gets to zero, the better the auto-scaling performance, and
it’s calculated as follows:

(4) $\sigma=\sum_{t\in\tau}{\sigma_{t}}$

where

${\sigma_{t}}=\begin{cases}L-{u_{t}},&\text{if }{u_{t}}\leq L,\\\ 0,&\text{if
}L<{u_{t}}<U,\\\ {u_{t}}-U,&\text{otherwise.}\end{cases}$

U and L are the values for the upper and lower bounds settled on the
application configuration. Ut is the percentage of infrastructure utilization
at a time-step T, and the following calculation gives it:

${u_{t}}=\frac{{w_{t}}}{{m_{t}}}$

Where ${w_{t}}$ is the number of resources demanded by the application (e.g.,
number of cores) and ${m_{t}}$ is the number of allocated resources (e.g., the
sum of cores of the VMs running the application).

Even though our proactive auto-scaling policy works with a single target, we
calculated its ADI setting L and U values the same as the simple scaling
configuration, 25% and 50%, respectively, since our goal is to have a system
utilization between these thresholds and compare it with the performance of
our reactive strategy.

##  4\. Results

In our proactive approach, the auto-scaling simulator decides to add and
remove resources based on the cores’ demand predicted by ARIMA. To ensure it
will make suitable decisions, we first need to check the accuracy of our model
before analyzing our auto-scaling strategy results, so we evaluated the
prediction accuracy using three of the metrics mentioned in the previous
section, i.e., $R^{2}$ Prediction Accuracy, Mean Absolute Error (MAE), and
Mean Absolute Percentage Error (MAPE).

Table 1. Results of each metric for our prediction model. They show a high accuracy since it deviates very little from the original values. $R^{2}$ | MAE | MAPE (%)
---|---|---
0.94 | 2.50 | 6.82

Figure 1 compares the collected data on cores’ demand and the prediction made.
We can see very similar behavior, except for a few demand peaks in the actual
data that our model could not predict. This similarity is supported by the
numbers of MAE and MAPE, 2.50 and 6.82%, respectively, meaning our forecast
deviates less than 10% from the actual demand and is very close to reality.
The $R^{2}$ Prediction Accuracy had a result of 0.94 out of 1.0, which
indicates our model is a good fit for explaining 94% of the fitted data in the
regression model. All these results are presented in Table 1.

Figure 1. Actual cores’ demand and predicted demand throughout February 9th
and 17th. The similar behavior between them indicates a high accuracy of the
forecast model, deviating an average of only 2,50 cores from the actual
demand. Figure 2. Allocated cores when using proactive and reactive
strategies. The presence of cooldown periods in the reactive approach led it
to a more conservative behavior when allocating cores and a variation of
30.19% compared to a variation of 35.41% from the proactive one.

We then run our auto-scaling simulator for both proactive and reactive
strategies, noticing a higher variation in the decision to allocate cores when
running the proactive technique and a conservative behavior from the reactive
technique, as shown in Figure 2. The coefficient of variation for proactive
was 35.41%, while for reactive was 30.19%. This variation is also seen in
Figure 3, where it shows the variable Decision, which indicates how many cores
the algorithm decided to add or remove in each observation.

Figure 3. Adding and removing decisions from proactive and reactive
strategies. The proactive approach takes way more decisions of both adding and
removing cores, while the reactive one allocates way lower resources and
spends long periods of time without making any decisions. Figure 4. System
utilization when running proactive and reactive strategies. With the proactive
technique, the utilization tends to stay more between the upper and lower
bounds, illustrated in the plot as the dashed red lines.

For the evaluation of each approach’s performance, we used the Auto-scaling
Demand Index (ADI). This metric is calculated by taking values of system
utilization and the settled thresholds, shown in Figure 4. When calculating
ADI, the proactive strategy got a result of 1782.97, while the reactive
strategy got 7014.97. The goal is to get the result as close to zero as
possible, considering it is the sum of the differences between the thresholds
and the observed system utilization. This means our proactive approach reached
a way better value than the reactive approach.

A conservative behavior such as in simple scaling may have led to a
degradation of the performance, considering the time it takes to decide on
adding or removing instances, as seen in Figure 3, where the reactive auto-
scaling has extended periods of the variable Decision as zero. The
anticipation of scaling up and down resources, and taking into account the
start-up time for each VM in the application, could have been decisive in the
divergence of performance results for proactive and reactive auto-scaling
techniques.

##  5\. Conclusion

In this work, we aimed to evaluate the performance of a proactive strategy
compared to the performance of an already used reactive strategy by an
ecommerce company. Based on our metrics, ARIMA appeared to be a good method to
be considered for predictive technique implementation. An accuracy of 94% was
an optimistic first step in our experiments, especially considering that it
was a forecast of more than 10 thousand observations ahead, with a horizon of
12 minutes for each prediction.

Our auto-scaling simulation also showed a promising perspective when running
proactive policy configurations. It was not guaranteed that a great forecast
model would lead to great decisions of adding and removing instances, while
also meeting our defined thresholds, but the results indicated, in fact, much
better performance in reducing system utilization levels compared to the
reactive approach.

It should be noted, however, that this current work does not consider if a
performance improvement would lead to a cost reduction for the ecommerce
company since it did not analyze purchase plans and prices of VMs. For future
studies, we plan to implement and evaluate other proactive approaches,
incorporating a bigger trace of the same company workload and considering
capacity planning in our analysis.

###### Acknowledgements.

This work wouldn’t be possible without the help and support of many people. An
acknowledgment isn’t enough to my professors, Thiago Emmanuel and Fábio, who
were the best mentors a student could ask for — all their patience and
guidance were essential to finishing this work. To Ítallo and Fireman, who
also helped me through this whole process. To each of my friends, who rooted
for this achievement with me as if it were theirs. To my parents, Helda and
Marcelo. They taught me the importance and the power of education in life, but
the most important lesson I got from them is how love and comprehension give
you the strength to pursue your dreams. I am pursuing mine thanks to them. To
my grandfather, José Walter, who always wished to see his granddaughter
graduating. I hope he can see it somehow. This work has been funded by
MCTIC/CNPq-FAPESQ/PB (EDITAL Nº 010/2021) and by VTEX BRASIL (EMBRAPII
PCEE-1911.0140).

## References

  * (1)
  * ec2 ([n. d.]a) [n. d.]a.  Amazon EC2 Instance Types - Amazon Web Services.  https://aws.amazon.com/pt/ec2/instance-types/.  Accessed: 2022-08-04.
  * sim ([n. d.]) [n. d.].  Auto-Scaling Simulator.  https://github.com/ufcg-lsd/autoscaling-analyser/.  Accessed: 2022-08-02.
  * duk ([n. d.]) [n. d.].  Introduction to ARIMA: nonseasonal models.  https://people.duke.edu/~rnau/411arim.htm.  Accessed: 2022-08-02.
  * ec2 ([n. d.]b) [n. d.]b.  Step and simple scaling policies for Amazon EC2 Auto Scaling.  https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-simple-step.html.  Accessed: 2022-08-04.
  * Box et al. (2015) George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2015.  _Time series analysis: forecasting and control_.  John Wiley & Sons.
  * Calheiros et al. (2014) Rodrigo N Calheiros, Enayat Masoumi, Rajiv Ranjan, and Rajkumar Buyya. 2014.  Workload prediction using ARIMA model and its impact on cloud applications’ QoS.  _IEEE transactions on cloud computing_ 3, 4 (2014), 449–458.
  * Dietterich (2002) Thomas G Dietterich. 2002\.  Machine learning for sequential data: A review. In _Joint IAPR international workshops on statistical techniques in pattern recognition (SPR) and structural and syntactic pattern recognition (SSPR)_. Springer, 15–30.
  * Everitt and Skrondal (2010) Brian S Everitt and Anders Skrondal. 2010.  The Cambridge dictionary of statistics.  (2010).
  * Gao et al. (2020) Jiechao Gao, Haoyu Wang, and Haiying Shen. 2020.  Machine learning based workload prediction in cloud computing. In _2020 29th international conference on computer communications and networks (ICCCN)_. IEEE, 1–9.
  * Hyndman and Athanasopoulos (2018) Rob J Hyndman and George Athanasopoulos. 2018.  _Forecasting: principles and practice_.  OTexts.
  * Hyndman and Khandakar (2008) Rob J Hyndman and Yeasmin Khandakar. 2008.  Automatic time series forecasting: the forecast package for R.  _Journal of statistical software_ 27 (2008), 1–22.
  * Ilyushkin et al. (2018) Alexey Ilyushkin, Ahmed Ali-Eldin, Nikolas Herbst, André Bauer, Alessandro V Papadopoulos, Dick Epema, and Alexandru Iosup. 2018\.  An experimental performance evaluation of autoscalers for complex workflows.  _ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)_ 3, 2 (2018), 1–32.
  * Iqbal et al. (2018) Waheed Iqbal, Abdelkarim Erradi, and Arif Mahmood. 2018\.  Dynamic workload patterns prediction for proactive auto-scaling of web applications.  _Journal of Network and Computer Applications_ 124 (2018), 94–107.
  * Islam et al. (2012) Sadeka Islam, Jacky Keung, Kevin Lee, and Anna Liu. 2012\.  Empirical prediction models for adaptive resource provisioning in the cloud.  _Future Generation Computer Systems_ 28, 1 (2012), 155–162.
  * Kotz et al. (2005) Samuel Kotz, Narayanaswamy Balakrishnan, Campbell B Read, and Brani Vidakovic. 2005.  _Encyclopedia of Statistical Sciences, Volume 1_.  John Wiley & Sons.
  * Lorido-Botran et al. (2014) Tania Lorido-Botran, Jose Miguel-Alonso, and Jose A Lozano. 2014\.  A review of auto-scaling techniques for elastic applications in cloud environments.  _Journal of grid computing_ 12, 4 (2014), 559–592.
  * Mell et al. (2011) Peter Mell, Tim Grance, et al. 2011\.  The NIST definition of cloud computing.  (2011).
  * Morais et al. (2017) Fábio Jorge Almeida Morais et al. 2017\.  Provisionamento automático de recursos como um serviço de IaaS.  (2017).
  * Morais et al. (2013) Fabio Jorge Almeida Morais, Francisco Vilar Brasileiro, Raquel Vigolvino Lopes, Ricardo Araújo Santos, Wade Satterfield, and Leandro Rosa. 2013.  Autoflex: Service agnostic auto-scaling framework for iaas deployment models. In _2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing_. IEEE, 42–49.
  * Netto et al. (2014) Marco AS Netto, Carlos Cardonha, Renato LF Cunha, and Marcos D Assunçao. 2014.  Evaluating auto-scaling strategies for cloud computing environments. In _2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems_. IEEE, 187–196.
  * Papadopoulos et al. (2016) Alessandro Vittorio Papadopoulos, Ahmed Ali-Eldin, Karl-Erik Årzén, Johan Tordsson, and Erik Elmroth. 2016\.  PEAS: A performance evaluation framework for auto-scaling strategies in cloud applications.  _ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS)_ 1, 4 (2016), 1–31.
  * Radhika and Sadasivam (2021) EG Radhika and G Sudha Sadasivam. 2021.  A review on prediction based autoscaling techniques for heterogeneous applications in cloud environment.  _Materials Today: Proceedings_ 45 (2021), 2793–2800.
  * Rampérez et al. (2021) Víctor Rampérez, Javier Soriano, David Lizcano, and Juan A Lara. 2021.  Flas: A combination of proactive and reactive auto-scaling architecture for distributed services.  _Future Generation Computer Systems_ 118 (2021), 56–72.
  * Rossi et al. (2019) Fabiana Rossi, Matteo Nardelli, and Valeria Cardellini. 2019\.  Horizontal and vertical scaling of container-based applications using reinforcement learning. In _2019 IEEE 12th International Conference on Cloud Computing (CLOUD)_. IEEE, 329–338.
  * Shariffdeen et al. (2016) RS Shariffdeen, DTSP Munasinghe, HS Bhathiya, UKJU Bandara, and HMN Dilum Bandara. 2016.  Adaptive workload prediction for proactive auto scaling in PaaS systems. In _2016 2nd International Conference on Cloud Computing Technologies and Applications (CloudTech)_. IEEE, 22–29.
  * Wang et al. (2021) Thomas Wang, Simone Ferlin, and Marco Chiesa. 2021\.  Predicting CPU usage for proactive autoscaling. In _Proceedings of the 1st Workshop on Machine Learning and Systems_. 31–38.