HSF-CWP-2017-14

Software trigger and event reconstruction:
Executive Summary

Abstract: Realizing the physics programs of the planned and upgraded high-
energy physics (HEP) experiments over the next 10 years will require the HEP
community to address a number of challenges in the area of software and
computing. For this reason, the HEP software community has engaged in a
planning process over the past two years, with the objective of identifying
and prioritizing the research and development required to enable the next
generation of HEP detectors to fulfill their full physics potential. The aim
is to produce a Community White Paper (CWP) [1] which will describe the
community strategy and a roadmap for software and computing research and
development in HEP for the 2020s. The topics of event reconstruction and
software triggers were considered by a joint working group and are summarized
together in this document.

Editors: Vladimir Vava Gligorov12,a and David Lange20

Contributors: Albrecht, Johannes1; Bloom, Kenneth2; Boccali, Tommaso3; Boveia,
Antonio4,c; De Cian, Michel5; Doglioni, Caterina6,b; Dziurda, Agnieszka7;
Farbin, Amir8; Fitzpatrick, Conor9; Gaede, Frank10; George, Simon11; Gligorov,
Vladimir12,a; Grasland, Hadrien13; Grillo, Lucia14; Hegner, Benedikt7;
Kalderon, William6; Kama, Sami15; Koppenburg, Patrick16; Krutelyov, Slava17;
Kutschke, Rob18; Lampl, Walter19; Lange, David20; Moyse, Ed21; Norman,
Andrew18; Petric, Marko7; Polci, Francesco12; Potamianos, Karolos10; Ratnikov,
Fedor22; Raven, Gerhard16; Ritter, Martin23; Rizzi, Andrea3; Rodrigues,
Eduardo24; Rousseau, David13; Salzburger, Andy7; Sexton Kennedy, Liz18;
Sokoloff, Michael D24; Stewart, Graeme7; Ustyuzhanin, Andrey22; Viren,
Brett25; Williams, Mike26; Winklmeier, Frank27; Wuerthwein, Frank17

1 Technische Universität Dortmund, Dortmund, Germany

2 University of Nebraska-Lincoln, Lincoln, NE, USA

3 INFN Sezione di Pisa, Università di Pisa, Scuola Normale Superiore di Pisa,
Pisa, Italy

4 The Ohio State University, Columbus, OH, USA

5 Physikalisches Institut, Ruprecht-Karls-Universitat Heidelberg, Heidelberg,
Germany

6 Fysiska institutionen, Lunds Universitet, Lund, Sweden

7 CERN, Geneva, Switzerland

8 Department of Physics, The University of Texas at Arlington, Arlington, TX,
USA

9 Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL),
Lausanne, Switzerland

10 Deutsches Elektronen-Synchrotron, Hamburg, Germany

11 Department of Physics, Royal Holloway University of London, Surrey, United
Kingdom

12 LPNHE, Université Pierre et Marie Curie, Université Paris Diderot,
CNRS/IN2P3, Paris, France

13 LAL, Université Paris-Sud and CNRS/IN2P3, Orsay, France

14 School of Physics and Astronomy, University of Manchester, Manchester,
United Kingdom

15 Physics Department, Southern Methodist University, Dallas TX, United States
of America

16 Nikhef National Institute for Subatomic Physics and Vrije Universiteit
Amsterdam, Amsterdam, The Netherlands

17 University of California, San Diego, La Jolla, CA, USA

18 Fermi National Accelerator Laboratory, Batavia, IL, USA

19 Department of Physics, University of Arizona, Tucson, AZ, USA

20 Princeton University, Princeton, NJ, USA

21 Department of Physics, University of Massachusetts, Amherst, MA, USA

22 National Research University Higher School of Economics, Moscow, Russia;
Yandex School of Data Analysis, Moscow, Russia

23 Fakultät für Physik, Ludwig-Maximilians-Universität München, München,
Germany

24 University of Cincinnati, Cincinnati, OH, USA

25 Physics Department, Brookhaven National Laboratory, Upton, NY, USA

26 Laboratory for Nuclear Science, Massachusetts Institute of Technology,
Cambridge, MA, USA

27 Center for High Energy Physics, University of Oregon, Eugene, OR, USA

a Vladimir V. Gligorov acknowledges funding from the European Research Council
(ERC) under the European Union’s Horizon 2020 research and innovation
programme under grant agreement No 724777 “RECEPT”

b Caterina Doglioni acknowledges funding from the European Research Council
(ERC) under the European Union’s Horizon 2020 research and innovation
programme under grant agreement No 679305 “DARKJETS”

c Antonio Bovela acknowledges funding for US Department of Energy (Grant DE-
SC0011726).

##  1 Introduction

Realizing the physics programs of the planned and/or upgraded high-energy
physics (HEP) experiments over the next 10 years will require the HEP
community to address a number of challenges in the area of software and
computing. For this reason, the HEP software community has engaged in a
planning process over the past two years, with the objective of identifying
and prioritizing the research and development required to enable the next
generation of HEP detectors to fulfill their full physics potential. The aim
is to produce a Community White Paper (CWP) [1] which will describe the
community strategy and a roadmap for software and computing research and
development in HEP for the 2020s. This activity is organized under the
umbrella of the HEP Software Foundation (HSF).

The CWP process was carried out by working groups centered on specific topics.
The topics of event reconstruction and software triggers are summarized
together in this document and in more detail elsewhere [2]. The reconstruction
of raw detector data and simulated data and its processing in real time
represent a major component of today’s computing requirements in HEP. A recent
projection [Campana2016] of the ATLAS 2016 computing model results in $>$85%
of the HL-LHC CPU resources being spent on the reconstruction of data or
simulated events. We have evaluated the most important components of next
generation algorithms, data structures, and code development and management
paradigms needed to cope with highly complex environments expected in HEP
detector operations in the next decade. New approaches to data processing were
also considered, including the use of novel, or at least, novel to HEP,
algorithms, and the movement of data analysis into real-time environments.

This document will discuss software algorithms essential to the interpretation
of raw detector data into analysis-level objects. Specifically, these
algorithms can be broadly grouped:

  1. 1.

Online: Algorithms, or sequences of algorithms, executed on events read out
from the detector in near-real-time as part of the software trigger, typically
on a computing facility located close to the detector itself.

  2. 2.

Offline: As distinguished from online, any algorithm or sequence of algorithms
executed on the subset of events preselected by the trigger system, or
generated by a Monte Carlo simulation application, typically in a distributed
computing system.

  3. 3.

Reconstruction: The transformation of raw detector information into higher
level objects used in physics analysis. A defining characteristic of
“reconstruction” which separates it from “analysis” is that the quality
criteria used in the reconstruction to, for example, minimize the number of
fake tracks, should be general enough to be used in the full range of physics
studies required by the experimental physics program. This usually implies
that reconstruction algorithms use the entirety of the detector information to
attempt to create a full picture of each interaction in the detector.
Reconstruction algorithms are also typically run as part of the processing
carried out by centralized computing facilities.

  4. 4.

Trigger: the part of the online system responsible for classification of
events which reduces either the number of events which are kept for further
“offline” analysis, the size of such events, or both. In this working group we
were only concerned with software triggers, whose defining characteristic is
that they process data without a fixed latency. Software triggers are part of
the real-time processing path and must make decisions quickly enough to keep
up with the incoming data, possibly using substantial disk buffers.

  5. 5.

Real-time analysis: Data processing that goes beyond object reconstruction,
and is performed online within the trigger system. The typical goal of real-
time analysis is to combine the products of the reconstruction algorithms
(tracks, clusters, jets…) into complex objects (hadrons, gauge bosons, new
physics candidates…) which can then be used directly in analysis without an
intermediate reconstruction step.

##  2 Challenges

Software trigger and event reconstruction techniques in HEP face a number of
new challenges in the next decade. These are broadly categorized into 1) those
from new and upgraded accelerator facilities, 2) from detector upgrades and
new detector technologies, 3) increases in anticipated event rates to be
processed by algorithms (both online and offline), and 4) from evolutions in
software development practices.

Advancements in facilities and future experiments bring a dramatic increase in
physics reach, as well as increased event complexity and rates. At the HL-LHC,
the central challenge for object reconstruction is thus to maintain excellent
efficiency and resolution in the face of high pileup values, especially at low
transverse momenta. Detector upgrades such as increases in channel density,
high precision timing and improved detector geometric layouts are essential to
overcome these problems. For software, particularly for triggering and event
reconstruction algorithms, there is a critical need not to dramatically
increase the processing time per event.

A number of new detector concepts are proposed on the 5-10 year timescale in
order to help in overcoming the challenges identified above. In many cases,
these new technologies bring novel requirements to software trigger and event
reconstruction algorithms or require new algorithms to be developed. Ones of
particular importance at the HL-LHC include high-granularity calorimetry,
precision timing detectors, and hardware triggers based on tracking
information which may seed later software trigger and reconstruction
algorithms. Longer term projects with sufficiently mature software
infrastructure can include cost implications of the simulation and
reconstruction algorithms into the detector design considerations. This is
especially important when the computing cost is expected to be a substantial
part of the total construction and operation cost for an experiment.

Trigger systems for next-generation experiments are evolving to be more
capable, both in their ability to select a wider range of events of interest
for the physics program of their experiment, and their ability to stream a
larger rate of events for further processing. ATLAS and CMS both target
systems where the output of the hardware trigger system is increased by 10x
over the current capability, up to 1 MHz [4, 5]. In other cases, such as LHCb
[6] and ALICE [7], the full collision rate (between 30 to 40 MHz for typical
LHC operations) will be streamed to real-time or quasi-realtime software
trigger systems. The increase in event complexity also brings a “problem” of
overabundance of signal to the experiments, and specifically the software
trigger algorithms. The evolution towards a genuine real-time analysis of data
has been driven by the need to analyze more signal than can be written out for
traditional processing, and technological developments which make it possible
to do this without reducing the analysis sensitivity or introducing biases.

Evolutions in computing technologies are both opportunities to move beyond
commodity x86 technologies, which HEP has used very effectively over the past
20 years, and significant challenges to derive sufficient event processing
throughput per cost to reasonably enable our physics programs [8]. Specific
items identified included 1) the increase of SIMD capabilities (processors
capable of running a single instruction set simultaneously over multiple
data), 2) the evolution towards multi- or many-core architectures, 3) the slow
increase in memory bandwidth relative to CPU capabilities, 4) the rise of
heterogeneous hardware, and 5) the possible evolution in facilities available
to HEP production systems.

The move towards open source software development and continuous integration
systems brings opportunities to assist developers of software trigger and
event reconstruction algorithms. Continuous integration systems have already
allowed automated code quality and performance checks, both for algorithm
developers and code integration teams. Scaling these up to allow for
sufficiently high statistics checks is among the still outstanding challenges.
As the timescale for experimental data taking and analysis increases, the
issues of legacy code support increase. Code quality demands increase as
traditional offline analysis components migrate into trigger systems, or more
generically into algorithms that can only be run once.

##  3 Current approaches

Substantial computing facilities are in use for both online and offline event
processing across all experiments surveyed. Online facilities are dedicated to
the operation of the software trigger, while offline facilities are shared for
operational needs including event reconstruction, simulation (often the
dominant component) and analysis. CPU in use by experiments is typically at
the scale of tens or hundreds of thousands of x86 processing cores.
Projections to future needs, such as for the HL-LHC, show the need for a
substantial increase in scale of facilities without significant changes in
approach or algorithms.

The CPU needed for event reconstruction tends to be dominated by charged
particle reconstruction (tracking), especially when the need for efficiently
reconstructing low transverse momentum particles is considered. Calorimetric
reconstruction, particle flow reconstruction, particle identification
algorithms also make up significant parts of the CPU budget in some
experiments. The CPU required for event reconstruction and trigger area with
challenges and substantial potential risk to the computing cost of
experiments. In this respect, software for future experiments will continue to
evolve, to improve both the physics and technical performance characteristics
of algorithms, and the uncertainty and evolution due to detector performance
and operating conditions will continue throughout the experimental program.

Disk storage is typically 10s to 100s of PB per experiment. It is dominantly
used to make the output of the event reconstruction, both for real data and
simulation, available for analysis. Current generation experiments have moved
towards smaller, but still flexible, data tiers for analysis. These tiers are
typically based on the ROOT [9] file format and constructed to facilitate both
skimming of interesting events and the selection of interesting pieces of
events by individual analysis groups or through centralized analysis
processing systems. Initial implementations of real-time analysis systems are
in use within several experiments. These approaches remove the detector data
that typically makes up the raw data tier kept for offline reconstruction, and
keep only final analysis objects [10, 11, 12].

Detector calibration and alignment requirements were surveyed. Generally a
high level of automation is in place across experiments, both for very
frequently updated measurements and more rarely updated measurements. Often
automated procedures are integrated as part of the data taking and data
reconstruction processing chain. Some longer term measurements, requiring
significant data samples to be analyzed together remain as critical pieces of
calibration and alignment work. These techniques are often most critical for a
subset of precision measurements rather than for the entire physics program of
an experiment.

##  4 Research and development Roadmap and Goals

We identify seven broad areas to be critical for software trigger and event
reconstruction work over the next decade. These are:

  1. 1.

Enhanced vectorization programming techniques

  2. 2.

Algorithms and data structures to efficiently exploit many-core architectures

  3. 3.

Algorithms and data structures for non-x86 architectures (e.g., GPUs, FGAs)

  4. 4.

Enhanced quality assurance (QA) and quality control (QC) for reconstruction
techniques

  5. 5.

Real-time analysis

  6. 6.

Precision physics-object reconstruction, identification and measurement
techniques

  7. 7.

Fast software trigger and reconstruction algorithms for high-density
environments

Not all roadmap areas are directly applicable to the event reconstruction and
triggering approach taken by all experiments. However, we expect that each
area of proposed research and development will be broadly applicable to future
high-energy physics experimental programs.

### Roadmap area 1: Enhanced vectorization programming techniques

HEP developed toolkits and algorithms typically make poor use of vector units
on commodity computing systems. Improving this will bring speedups to
applications running on both current computing systems and most future
architectures. The goal for work in this area is to evolve current toolkit and
algorithm implementations, and best programming techniques to better use SIMD
capabilities of current and future computing architectures.

### Roadmap area 2: Algorithms and data structures to efficiently exploit
many-core architectures

Computing platforms are generally evolving towards having more cores in order
to increase processing capability. This evolution has resulted in multi-
threaded frameworks in use, or in development, across HEP. Algorithm
developers can improve throughput by being thread safe and enabling the use of
fine-grained parallelism. The goal is to evolve current event models, toolkits
and algorithm implementations, and best programming techniques to improve the
throughput of multithreaded software trigger and event reconstruction
applications.

### Roadmap area 3: Algorithms and data structures for non-x86 computing
architectures (e.g., GPUs, FPGAs)

Computing architectures using technologies beyond CPUs offer an interesting
alternative for increasing throughput of the most time consuming trigger or
reconstruction algorithms. Such architectures (e.g., GPUs, FPGAs) could be
easily integrated into dedicated trigger or specialized reconstruction
processing facilities (e.g., online computing farms). The goal is to
demonstrate how the throughput of toolkits or algorithms can be improved
through the use of new computing architectures in a production environment.

### Roadmap area 4: Enhanced QA/QC for reconstruction techniques

HEP experiments have extensive continuous integration systems, including
varying code regression checks that have enhanced the quality assurance and
quality control procedures for software development in recent years. These are
typically maintained by individual experiments and have not yet reached the
scale where statistical regression, technical, and physics performance checks
can be performed for each proposed software change. The goal is to enable the
development, automation, and deployment of extended QA and QC tools and
facilities for software trigger and event reconstruction algorithms.

### Roadmap area 5: Real-time analysis

Real-time analysis techniques are being adopted to enable a wider range of
physics signals to be saved by the trigger for final analysis. As rates
increase, these techniques can become more important and widespread by
enabling only the parts of an event associated with the signal candidates to
be saved, reducing the required disk space. The goal is to evaluate and
demonstrate the tools needed to facilitate real-time analysis techniques.
Research topics include compression and custom data formats; toolkits for
real-time detector calibration and validation which will enable full offline
analysis chains to be ported into real-time; and frameworks which will enable
non-expert offline analysts to design and deploy real-time analyses without
compromising data taking quality.

### Roadmap area 6: Precision physics-object reconstruction, identification
and measurement techniques

The central challenge for object reconstruction at HL-LHC is to maintain
excellent efficiency and resolution in the face of high pileup values,
especially at low transverse momenta. Both trigger and reconstruction
approaches need to exploit new techniques and higher granularity detectors to
maintain or even improve physics measurements in the future. Reconstruction in
very high pileup environments, such as the HL-LHC or FCC-hh, may also greatly
benefit from adding timing information to our detectors, in order to exploit
the finite beam crossing time during which interactions are produced. The goal
is to develop and demonstrate efficient techniques for physics object
reconstruction and identification in complex environments.

### Roadmap area 7: Fast software trigger and reconstruction algorithms for
high-density environments

Future experimental facilities will bring a large increase in event
complexity. The scaling of current-generation algorithms with this complexity
must be improved to avoid a large increase in resource needs. In addition, it
may be desirable or indeed necessary to deploy new algorithms, including
advanced machine learning techniques developed in other fields, in order to
solve these problems. The goal is to evolve or rewrite existing toolkits and
algorithms focused on their physics and technical performance at high event
complexity (e.g. high pileup at HL-LHC). Most important targets are those
which limit expected throughput performance at future facilities (e.g.,
charged-particle tracking). A number of such efforts are already in progress
across the community.

##  5 Conclusions

The next decade will see the volume and complexity of data being processed by
HEP experiments increase by at least one order of magnitude. While much of
this increase is driven by planned upgrades to the four major LHC detectors,
new experiments such as DUNE will also make significant demands on the HEP
data processing infrastructure. It is essential that software triggers and
event reconstruction algorithms continue to evolve so that they are able to
efficiently exploit future computing architectures and deal with this increase
in data rates without loss of physics capability.

We have identified seven key areas where R&D is necessary to enable the
community to exploit the full power of the enormous datasets which we will be
collecting. Three of these areas concern the increasingly parallel and
heterogeneous computing architectures which we will have to write our code
for. In addition to a general effort to vectorize our codebases, we must
understand what kinds of algorithms are best suited to what kinds of hardware
architectures, develop benchmarks that allow us to compare the physics-per-
dollar-per-watt performance of different algorithms across a range of
potential architectures, and find ways to optimally utilise heterogeneous
processing centres. The consequent increase in the complexity and diversity of
our codebase will necessitate both a determined push to educate tomorrow’s
physicists in modern coding practices, and a development of more sophisticated
and automated quality assurance and control for our codebases. The increasing
granularity of our detectors, and the addition of timing information to help
cope with the extreme pileup conditions at the HL-LHC, will require us to both
develop new kinds of reconstruction algorithms and to make them fast enough
for use in real-time. Finally, the increased signal rates will mandate a push
towards real-time analysis in many areas of HEP, in particular those with low
transverse momentum signatures.

The success of this R&D program will be intimately linked to challenges
confronted in other areas of HEP computing, most notably the development of
software frameworks which are able to support heterogeneous parallel
architectures, including the associated data structures and I/O, the
development of lightweight detector models which maintain physics precision
with minimal timing and memory consequences for the reconstruction, enabling
the use of offline analysis toolkits and methods within real-time analysis,
and the ability to integrarte machine learning reconstruction algorithms being
developed outside HEP into our workflows and apply them to our problems. For
this reason perhaps the most important task ahead of us is to maintain the
community which has coalesced together in this CWP process, so that the work
done in these sometimes disparate areas of HEP fuses coherently together into
a solution to the problems facing us over the next decade.

## References

  * [1] HEP Software Foundation, A Roadmap for HEP Software and Computing R&D for the 2020s, HSF-CWP-2017-01, arXiv:1712.06982v2 (Link).
  * [2] Software trigger and event reconstruction CWP group, To be published as HSF-CWP-2017-16 (Link).
  * [3] S. Campana, Presentation to the 2016 Aix-les-Bains ECFA HL-LHC workshop, Oct 3 2016 (Link).
  * [4] ATLAS Collaboration, ATLAS Phase-II Upgrade Scoping Document, CERN-LHCC-2015-020, LHCC-G-166 (2015) (Link).
  * [5] CMS Collaboration, Technical Proposal for the Phase-II Upgrade of the CMS Detector, CERN-LHCC-2015-010, LHCC-P-008, CMS-TDR-15-02 (2015) (Link).
  * [6] LHCb Collaboration, LHCb Trigger and Online Upgrade Technical Design Report, CERN-LHCC-2014-016 (2014) (Link).
  * [7] ALICE Collaboration, Technical Design Report for the Upgrade of the Online-Offline Computing System, (Link).
  * [8] I. Bird, et. al., Update of the Computing Models of the WLCG and the LHC Experiments, CERN-LHCC-2014-014, (2014), (Link).
  * [9] R. Brun and F. Rademakers, ROOT - An Object Oriented Data Analysis Framework, Proceedings AIHENP’96 Workshop, Lausanne, Sep. 1996, Nucl. Inst. & Meth. in Phys. Res. A 389 (1997) 81-86. See also ROOT homepage.
  * [10] R. Aaij, et. al., Tesla : an application for real-time data analysis in High Energy Physics, Comput. Phys. Commun. 208 (2016) 35-42 (Link).
  * [11] ATLAS Collaboration, Performance of the ATLAS Trigger System in 2015 , Eur. Phys. J.C77 5 317 (2017), . ATLAS Collaboration, Trigger-object Level Analysis with the ATLAS detector at the Large Hadron Collider: summary and perspectives, ATL-DAQ-PUB-2017-003 (2017) (Link).
  * [12] CMS Collaboration, Search for narrow resonances in dijet final states at s√= 8 TeV with the novel CMS technique of data scouting, Phys. Rev. Lett. 117 031802 (2016), (Link).


Vrije Universiteit Amsterdam

Honours Programme, Project Report

Failure Analysis of Big Cloud Service

Providers Prior to and During

Covid-19 Period

Author: Muhammad Ahsan (2663138)
---
1st supervisor: |  prof. dr. ir. Alexandru Iosup
---|---
daily supervisor: |  Sacheendra Talluri, MSc

A report submitted in fulfillment of the requirements for the Honours
Programme,
which is an excellence annotation to the VU Bachelor of Science degree in
Computer Science/Artificial Intelligence/Information Sciences
version 1.0

August 27, 2024

###### Abstract

Cloud services are important for societal function such as healthcare,
commerce, entertainment and education. Cloud can provide a variety of features
such as increased collaboration and inexpensive computing. Failures are
unavoidable in cloud services due to the large size and complexity, resulting
in decreased reliability and efficiency. For example, due to bugs, many high-
severity failures have been occurring in cloud infrastructure of popular
providers, causing outages of several hours and the unrecoverable loss of user
data (1, 2). There are prior studies about cloud failure analyses are limited
and use sources such as news articles. However, a detailed cloud failure
focused study is required that provides analyses for cloud failure data
gathered directly from the vendors. Furthermore, the Covid-19 cloud failures
should be studied as cloud services played a major role throughout the
Covid-19 period, as individuals relied on cloud services for activities such
as working from home. A program can be made for this task. As a result, we
will be able to better understand and mitigate cloud failures to reduce the
effect of cloud failures.

###### Contents

  1. 1 Introduction
    1. 1.1 Context
    2. 1.2 Problem Statement
    3. 1.3 Research Questions
    4. 1.4 Approach
  2. 2 Processing Cloud Failure Data
    1. 2.1 Data Description
    2. 2.2 Removing Duplicate Data
    3. 2.3 Handling Missing Data
    4. 2.4 Incorrect Dates in Data
    5. 2.5 Special cases in Data
    6. 2.6 Normalizing text
    7. 2.7 Organising Data
  3. 3 Results and Analysis
    1. 3.1 Monthly Cloud Failures
    2. 3.2 Comparing Cloud Service Providers
    3. 3.3 Failure duration and recovery
    4. 3.4 Failing Services
    5. 3.5 Weeks with Cloud Failures
    6. 3.6 Days of Cloud Failures
    7. 3.7 Location of Cloud failures
  4. 4 Limitations and Related Work
    1. 4.1 Analysis of Limitations
    2. 4.2 Related Work
  5. 5 Conclusion
  6. 6 Self-Reflection
    1. 6.1 Self-Reflection

##  Chapter 1 Introduction

Utilizing the cloud is a trend that continues to grow as it is used by an
increasing number of individuals and businesses (3, 4). Users may utilize
cloud services when and where they need them and only pay for the resources
they use. It provides massive computational capacity, scalability on demand,
and utility-like availability at a reasonable cost. Cloud Computing enables
efficiently on-demand network access to a shared pool of configurable
computing resources that can be rapidly provisioned and released with minimal
management effort or service provider interaction. The cloud is ubiquitous.
However, cloud failures occur frequently and affect users of the cloud
services. These failures can lead to great loss as people’s daily activities
such as many peoples income depends on the cloud services to work.

###  1.1 Context

A failure can be described as an unintended behavior of a service. The term
“cloud failure” refers to when a cloud service is unavailable for use for an
extended period of time. Unavailability can also relate to a service’s
inadequacy in terms of performance, as measured by the agreed-upon SLA
metrics. An SLA is an explicit or implicit contract between a cloud provider
and customer. It governs the obligations and responsibilities between both
parties regarding the provided service. In case the cloud provider is unable
to meet the terms of the agreed SLAs, the consequence is usually a financial
penalty or rebate, but it can take other forms as well. For example, if a data
centre was only partially impacted by a failure, the vendor may be forced to
do the appropriate repair and restoration actions. The end user may experience
downtime until the service is fully restored in accordance with the agreed-
upon SLA criteria. Despite all the potential and development cloud computing
has undergone over the years, failures continue to occur. Cloud failures are
of concern as they can result in reduction of quality of service,
availability, reliability and energy waste (5) that can ultimately lead to
economic loss for both cloud users and providers. By studying the failures, we
can protect users from being affected by these failures.

There are numerous cloud providers. Cloud failures are common among all
providers. Big cloud companies represent a large population of cloud users.
Therefore, it is more beneficial to study failures in big cloud providers.
This is the aim of the current study. Many big cloud vendors provide cloud
failure logs that can be used for the study. Cloud failures can be analyzed
over time to gain better understanding. Furthermore it is interesting to
compare failures among different vendors. Cloud failure analysis is done with
the aid of a program.

There are studies that describe different aspects of cloud. However there is
limited in-depth research on the topic of cloud failures. This paper will
provide analysis about cloud failures in three popular big cloud providers
Amazon Web Service (AWS), Microsoft Azure and Google Cloud Platform (GCP), and
comparison of failures during the period 2018 till 2020 (June).

###  1.2 Problem Statement

As the dependency on cloud computing increases, society demands high
availability, an ideal 24/7 service uptime, if possible. The cloud runs many
daily life activities including banking, healthcare, governance,
transportation, e-commerce, entertainment, etc. Cloud services played an
important part during Covid-19. A large population of the world trusts cloud
services to run their daily lives, for example, employees working remotely
from homes, online education for students and online businesses. However, many
cloud services continue to fail. These failures can lead to great loss as
people’s daily activities, especially as many people’s income depends on the
cloud services to work.

Is it correct to depend on the cloud to run daily lives? Why does the cloud
keep failing? Which services are failing? Where, when and how are the failures
occurring? How do the big cloud providers respond to failures? What is the
pattern of the failures in different periods? During difficult times like the
Covid-19 period, can the cloud fulfil the needs of people. Many similar
important questions can be explained by analysing the failures on cloud.

###  1.3 Research Questions

[RQ1] How to process raw cloud failure data to carry out analysis using the
data?

The amount of data is substantial, since most files include approximately 1000
instances. Any manual approach is likely to make mistakes and is inefficient
for solving the current problem because data is raw and would require
customize-able processing. Therefore an efficient method to solve the issue of
preparing data is to use an automated method which is reproducible. One of the
possibilities is to use a program that does the task. However, no known
program exists that can prepare the given sets of data.

[RQ2] How to statistically characterize and visualize cloud failure data to
understand the frequency of failures?

After approaching research question 1, we would have a software that outputs
the final data sets. At this point some approach is required that can extract
information and present the data. A possibility can be to provide the
processed data as an input to some available data analysis software. A
software like Microsoft Excel only provides basic ways to analyze and do not
meet the requirement of the current study. For example, if the study requires
a specific graph with specific features, the data analysis software may not
have those. Additionally, there are three different data sets, doing manual
analysis using some software for every data file is not efficient. In a
nutshell, general data analysis softwares provide limited options.

[RQ3] How did the pattern of failures change when comparing the Covid-19
period to the prior years?

Information such as peak cloud failures and their relation with other features
such as time and location need to be extracted to gain valuable insight in
cloud failures. Additionally, People relied on cloud services during Covid-19
example doing work from home. The change in usage can help us view failure
patterns that differ from normal days. Therefore comparison of Covid-19 period
cloud failures with the period before can be used to better understand cloud
failure.

###  1.4 Approach

RQ1. The question under consideration can be solved by creating a program that
takes the raw data file as the input, prepares it such as cleaning, filtering,
organising etc. The program outputs a file with ready to use data. There are a
variety of programming languages available for making the software. A
preferable language can be python because there are many data-centric Python
packages, for example Pandas and NumPy, which make the process of data
processing and data analysis a lot quick and convenient.

RQ2. The current research requires an analysis approach that is efficient,
reusable and provides a wide range of customizing options. The given
requirements can be fulfilled by coding a software that does the job. This
software can be an extension of the program made in research question 1. In
this way, the data can directly be available for analyses. Moreover, as
suggested in the approach to research question 1, python can be used as it
provides a variety of options example for plotting matplotlib and seaborn plot
can be used.

RQ3. The first part of the question can be approached by understanding the
general structure of the data files such that considering the data
fields(columns of data set), it is possible to predict some possible analyses.
The yearly data can be grouped by separate vendors. This can provide answers
to questions like which vendor has the most failures. The failure event start
time and date of failures can be used to analyze months that had the most
failures. Using event start time with event end time we can find out the
events that lasted the longest. This can further be used to analyse the
performance of service providers. The location data can tell which location
had the most failures. Similarly the service name column can be analyzed to
get services that failed the most. A possible plot can be a plot to show
failure count for each month with respect to separate cloud providers.

The second part of the question can be answered such that, the Covid-19
reduced the physical interaction among people and increased interaction
through cloud services, for example, online education and business. This rapid
increase in demand increased the burden on cloud services. Considering the
fact that Covid-19 was not expected, the cloud was not optimised for this, at
least not initially. Therefore it can be predicted that more failures should
have occurred during Covid-19 period. As a result, it is also interesting to
compare and provide analyses for the year 2020 (Covid-19 period) and the
previous years.

The study results in the following contributions:

  1. [C1]

A tool to process raw cloud failure data and extract useful information. The
first research question build the software to clean raw cloud data. Then
research question 2 extends the software from RQ1 adding visuals and statistic
results.

  2. [C2]

Analyses of cloud failures prior and during Covid-19 period. Additionally,
cloud failure analysis can be used to reduce cloud failures and for further
research.

##  Chapter 2 Processing Cloud Failure Data

This chapter describes the characteristics of the cloud failure data. The
operations performed to process the data such as cleaning and organising data
are also described in this chapter. Therefore, the first research question is
answered in the chapter. The operations mentioned here are in order. If a
process is mentioned before another process. It indicates that the process was
done before the next process unless specifically mentioned in the description.
Therefore the understanding of the next process should be gained assuming the
processes mentioned before is already completed. The cloud failure data used
by the study is obtained from the official website of the big cloud providers
(6, 7, 8). The study covers the period 2018 till 2020 (June).

###  2.1 Data Description

The failure information was initially put in rows and gathered in three
separate files for each year named ‘provider failures 2018’, ‘provider
failures 2019’ and ‘provider failures 2020’. All three files had 12 columns
that had information about service id, service name, location, status, event
start time, event end time, event duration hours, first notification, last
notification, description, vendor, monitor and orgtype. Most column headers
are self-explanatory. The event start time, event end time, first notification
and last notification column has information about the date and time. This is
originally represented in the form of a Unix timestamp. The orgtype column
means origin type. The origin type is ‘cloud’ for all rows as data is about
cloud services. The study covers three big cloud services so the vendor column
has three possibilities that are; AWS, Azure and GCP. The vendor and monitor
columns contain the same information. For consistency, the vendor column was
used when needed during the study. The program made for cloud analysis was
used to extract information about the data. This was done by choosing the
column and outputting unique values and their respective counts. Description
of columns are shown in Table 2.1.

Column name | Description
---|---
service id | The id of the service
service name | Name of service
location | Region of failure
status | Indicators such as 1,0. No meaningful interpretation
event start time | Start time of failure
event end time | End time of failure
event duration hours | Derived feature representing duration of cloud failure
first notification | First notification of the failure issued by the vendor.
last notification | Last notification of the failure issued by the vendor.
description | Details of cloud failure.
vendor | The cloud provider AWS or GCP or Azure.
monitor | Same as vendor.
org type | For all instances the origin is cloud.

Table 2.1: Data columns description.

The file ‘provider failures 2018’ had 965 rows, file ‘provider failures 2019’
had 1024 rows and ‘provider failures 2020’ had 639 rows. The rows of 2020 file
are less compared to other years files because 2020 covers cloud failures
until June while other files have full-year cloud failure data. The values
mentioned are of the raw data on which no operation had been performed. Each
file contains information about three big cloud vendors AWS, Azure and GCP. In
this initial data set rows were disorganised. The original failures count with
respect to each cloud provider is given in the first row of Table 2.2. The
data set contains rows with missing data. There are three vendors and they do
not provide information required to fill all 12 columns, see Section 2.3.
These cases are handled carefully at the right time in the right way, see
Section 2.

Many operations are the same for all three files because of the similar
structure. When the tool is run, input is given to the program which specifies
the file to process. Input 1 to process file ‘provider failures 2020’, 2 for
file ‘provider failures 2019’ and 3 for file ‘provider failures 2018’. When
files are processed the program reads/stores the processed versions files.
Reading all three files can enable comparing data such as combined plots. The
program prints the information to the screen. The processed results are
verified by automated tests built in the tool and manual checks. Manual checks
include observing different parts of the data before and after the processing.
Multiple people are involved so error is minimized. Depending on the tests,
the program is updated if required. The study includes 2020 data till June.
When 2020 is mentioned in this text, it means till June 2020.

###  2.2 Removing Duplicate Data

The initial data set had duplicate rows, described in Section 2.1. Removing
duplicates is the first step to prepare data. Duplicate rows are when a
failure event is reported more than once in the file. Duplicate rows have the
same service_id, service_name, location, event_start_time and vendor.

Description is not included in the criteria for duplicate rows. Duplicate rows
will have the same description because the event is the same. However, the
string length used for checking duplicates can be different. For example, two
duplicate rows, the first row the description starts with “The” but the second
row starts without the “The”, the remaining description is the same for both
rows, will have different string length. This example was observed during the
study. The tool outputs the duplicate rows which are used studied to verify
the process of duplicate removal. Similarly we ignore event end time when
removing duplicates. There is a special case in which the rows are duplicate
and have the same description length but have different end times. The
quantity of such rows is few. In this case we keep the row with higher end
time. Comparing descriptions and event end times of two rows would not be
beneficial as seen in the example. Instead of using description we use other
fields (mentioned above) to check for duplicates. The tool does the following
to remove duplicates: starting at the first row it selects that row and
compares with the rows below the selected row in the file. If a duplicate row
is found, it is removed from the file. In normal cases, when two rows are the
same, the selected row is kept and the other rows are deleted. The Table 2.2
shows the result of removing duplicates. The plus 1 in Table means that 1 row
had no vendor information. In both 2019 and 2018 files, this row gets removed
at a later step, see Section 2.4.

| Number of
---
Rows
2018 | 2019 | 2020
| AWS | Azure | GCP | Total | AWS | Azure | GCP | Total | AWS | Azure | GCP | Total
Initial | 325 | 284 | 355 |  | 964
---
+1
273 | 222 | 528 |  | 1024
---
+1
271 | 51 | 316 |  | 638
---
+1
Duplicate | 4 | 145 | 142 | 291 | 7 | 110 | 261 | 378 | 1 | 15 | 114 | 130
| After Removing
---
Duplicates
321 | 139 | 213 |  | 673
---
+1
266 | 112 | 267 |  | 645
---
+1
270 | 36 | 202 |  | 507
---
+1

Table 2.2: Results of duplicate removal.

###  2.3 Handling Missing Data

The unknown cells include the cells that are empty cells or cells with ‘-1’ or
‘0’. The reason is that not all vendors provide all information as addressed
by the current study, see Section 2.1. Furthermore, for consistency we aim to
replace the multiple representations and use ‘Unknown’ for string based
unknown cells and 0 for integral based unknown cells. The unknown cells are
GCP location cells in 2020 and 2019 data files, see Section 2.4. Another case
is Azure does not have service id in any row in any year. Azure also does not
provide information about the first notification and last notification. These
are also the limitations of the data and the study. All other rows are filled
with appropriate information. Table 2.3 gives the frequency of the unknown
cells as per vendor and total.

###  2.4 Incorrect Dates in Data

The initial data had rows with incorrect date and time. The rows had “-1” as
event start time and event end time. “-1” indicates that event time was not
known. Due to lack of information, the rows with “-1” were removed. There were
also rows that had event time of the previous year, for example, the file of
2020 contained failure events of 2019 and 2018. The old dated events were
already present in the correct year files and were repeated in the new year
file. The old dated rows were not required in the new year file so they were
removed from the file. The results are shown in table below.

| Number of Rows
---
2018 | 2019 | 2020
| After Removing
---
Duplicates
674 | 646 | 509
| Incorrect dated
---
Rows
275 | 271 | 386
| After removing
---
incorrect dated rows
399 | 375 | 123
Table 2.3: Missing dates and old dated rows.

###  2.5 Special cases in Data

The initial files also had other special cases related to event start, end
date and time. A few rows had the same event start and end time. Same event
start and end time is not possible so these rows are considered incorrect and
removed. Another case is that some rows had event end time before the event
start time. These rows were studied manually and checked with the time in the
description. A common thing that was noticed in rows having end time before
start time was that the start time was not converted to a 24-hour format. We
converted these to 24-hour time and the times matched the timing mentioned in
the description. In general these rare cases were present in only a few rows.
In total less than 15 rows per file had end time before start time. In this
case rows were kept in the files fixed by converting to 24-hour format.
Another case was that there were few rows that had corrupted description.
These rows count at most 5\. Corrupted description means the rows description
had symbols not understandable English. These rows were removed. After taking
care of the special cases, the failures count for each files is shown in Table
2.4.

|  Number of Rows
---
2018 | 2019 | 2020
| After removing
---
incorrect dated rows
399 | 375 | 123
Special cases rows | 13 | 18 | 9
Remaining rows | 386 | 357 | 114
Table 2.4: Result of fixing incorrect date and special cases.

###  2.6 Normalizing text

After cleaning the data and sorting rows, the data was more readable. The
quantity was reduced compared to initial files. At this stage data files were
manually observed. It was noticed that in the column of location and service
name, some rows needed to be fixed. Two rows have location ‘East US’ but the
second row uses lowercase ‘east us’. It is likely that when performing
comparisons between two cells, this can give incorrect results. Another
scenario is that ‘Networking’ and ‘Network’ represent the same service name.
Similarly ‘Southeast Asia’ and ‘South East Asia’ represent the same location.
When two cells intend to deliver the same information, we replace them with a
single representation, for example we replace ‘Southeast Asia’ and any other
representation of southeast asia with ‘South East Asia’. These measures are
helpful in reading and sorting data. This results in single representation and
provides the correct result when filtering, example when outputting unique
values. This operation is done before removing duplicate rows, this improves
the result of the duplicate removal process. The failure count presented in
the tables such as Table 2.2 is result obtained after doing this process.

| Number of
---
Rows
2018 | 2019 | 2020
| AWS | Azure | GCP | Total | AWS | Azure | GCP | Total | AWS | Azure | GCP | Total
Initial | 325 | 284 | 355 |  | 964
---
+1
273 | 222 | 528 |  | 1024
---
+1
271 | 51 | 316 |  | 638
---
+1
| Total
---
Removed
188 | 165 | 255 |  | 578
---
+1
155 | 144 | 367 |  | 667
---
+1
233 | 34 | 257 |  | 524
---
+1
| Remaining
---
after
cleaning
137 | 119 | 130 | 386 | 118 | 78 | 161 | 357 | 38 | 17 | 59 | 114

Table 2.5: Frequencies of data removed during cleaning process.

###  2.7 Organising Data

The data rows in the initial files were disorganized. Generally, there were
many sorting options. In particular, the order location, vendor, service id,
event start time, and then service name was found to be the optimal sorting
order. Rows are sorted by location first, then vendor, and so on. The
information was arranged in ascending order. The sorting operation was very
useful in certain analyses and also in outputting data.

The data has been cleaned and organized into groups. We have the data as
separate year data files initially. We store data after it has been cleaned
with respect to three different vendors for each year. This can be useful for
analyses and in other operations, such as graph creation.

##  Chapter 3 Results and Analysis

This chapter answers the second and third research question. To answer the
second research question the data is visualized using plots. Furthermore, the
plots are analyzed to extract patterns. A detailed comparison is laid down
between the period before and during Covid-19 for each plot. This answers the
third research question.

###  3.1 Monthly Cloud Failures

Figure 3.1: Plot of failure frequency in each month.

The Figure 3.1 shows the total number of monthly cloud failures in 2019 is
less than in 2018 except for March, April, and September (2020 not included).
That is, in 75 percent of cases, the cloud failures have decreased. In both
2018 and 2019, October and November are ’peak’ months where cloud failures are
relatively high compared to the rest of the year. During these months cloud
failures reach very close to the highest failure count of the year.

O-1: The months of October and November had the greatest failure frequency,
while December had the lowest number of cloud failures over the years.

In December the cloud failures are the least for both 2018 and 2019. Cloud
failures for March have been increasing every year, while in February and May
cloud failures have decreased continuously during the three years. The high
peak in March 2020 is likely related to Covid-19 the evidence being that this
is the first month since October 2018 that cloud failures reach such height
(approximately after 1.5 years). This month was the peak lockdown month and
most works had been shifted online. The sudden high usage of cloud services
during March due to Covid-19 can be a possible explanation for the high peak.

O-2: Covid-19 is likely one of the reasons for the peak of cloud failures in
March 2020.

An interesting point is that May 2020 had the least number of cloud failures
in three years while March 2020 was among the highest cloud failure months.
Overall, the pattern of bars shows that when failures are high in a particular
month, they do not continue to remain high in upcoming months (high referring
to more than 30 failures). The bars always come down in the next month.

###  3.2 Comparing Cloud Service Providers

Figure 3.2: Failure count in cloud service providers.

O-3: Microsoft Azure had fewer failures than AWS and GCP.

In 2018 AWS had the most cloud failures, in 2019 GCP had the most cloud
failures and in 2020 GCP is expected to have the most failures. Figure 3.2
shows that in these three years Azure had fewer failures than AWS and GCP. For
both AWS and Azure number of failures decreased from 2018 to 2019. However,
the number of failures for GCP increased from 2018 to 2019. Overall, during
the 2 and a half-year Azure has fewer failures compared to other vendors.

###  3.3 Failure duration and recovery

From the plots in Figures 3.3, 3.4 and 3.5 it can be observed that AWS and GCP
recover 70 to 90 percent of failure events in 5 hours while Azure recovers
only 50 to 60 percent. In general, during the 2 and a half-year time, AWS has
the quickest recovery of cloud failures. GCP recovery speed although behind
but is close to AWS. For Azure, the failure event recovery is slower compared
to other two vendors.

O-4: AWS had the fastest recovery, followed by GCP. Microsoft Azure recovery
was more delayed compared to the other vendors. For all vendors approximately
50% of cloud failures were recovered in less than 5 hours.

Overall, AWS outages have lasted no longer than 24 hours. GCP had the longest
failure occurrence in 2019, spanning up to ten days. In particular, the
longest failure occurrence in 2018 was of Azure, which lasted 120 hours. The
longest failure occurrence in the first half of 2020 was of GCP, which lasted
29 hours. Eighty percent of cloud outages were recovered within ten hours for
all three vendors. In addition, half of the failure events lasted fewer than 5
hours.

Figure 3.3: ECDF plot for 2020. Figure 3.4: ECDF plot for 2019. Figure 3.5:
ECDF plot for 2018.

###  3.4 Failing Services

Figures 3.6, 3.8 and 3.8 shows the name and frequency of the ten services that
had most failures in 2020, 2019 and 2018 respectively. A common service
failing in all three years is Network service, this is the most failing
service throughout the 2.5 years and this service also has the highest number
of failures in 2018 and 2019. Similarly Amazon Elastic Compute Cloud service
is the second most failing service in all 2.5 years.

O-5: Network service was among the frequently failing service and in total had
the most failures over the period. There are many services with continuing
failures over the years.

Services that were failing for two consecutive years (2018 and 2019) include
Google Cloud Storage service, Google Stackdriver, Google App Engine and Google
Compute Engine. In most cases the number of failures for these services have
decreased compared to the previous year. Services that have been failing for
exactly two consecutive years (2019 and 2020) include Google Cloud Functions
and Google Cloud Console.

Figure 3.6: 2020 services with most failures. Figure 3.7: 2019 services with
most failures. Figure 3.8: 2018 services with most failures.

###  3.5 Weeks with Cloud Failures

Figures 3.9, 3.11 and 3.11 shows the failures during the weeks of the year
2020, 2019 and 2018 respectively. The horizontal axis shows the week number
while the red dot shows the failure at the day of the week. In 2020, the
highest red dot in week 13 can be clearly noticed indicating failure of more
than 25 services on a day. In 2019, more than 10 failures on a day in the
weeks 30, 39 and week 46. Week 46 has the highest failures in a day that is
more than 15 failures in a day. Similarly in 2018, more than 10 failures in a
day in weeks 5, 7, 33 and 29. Week 29 had the highest failures. In all three
plots, we can notice that it is rare to have a week without failures which
highlights the importance of the study. From 2018 to 2019 failures are present
but in 2019 failures on a day are not high such as majority is fewer than 5
failures, as a result that is 2019 plot is less scattered vertically.

O-6: More than 25 services failed on a day in 2020. Furthermore, a week
without cloud failure is rare.

Figure 3.9: 2020 Failures till end of June. Figure 3.10: 2019 Failures
throughout year. Figure 3.11: 2018 Failures throughout year.

###  3.6 Days of Cloud Failures

Figure 3.12, Figure 3.13 and Figure 3.14 shows the failures at days of the
week in 2020, 2019 and 2018 respectively. In 2018 and 2019 we can observe and
it is also expected that more failure occur during working days (Monday to
Friday) than on weekend (Saturday and Sunday). In 2020 the pattern is present
but may not be clear as only half year data. For both 2018 and 2019 Thursday
has the highest number of failures. While in 2020 Friday has the highest
number of failures followed by Thursday. Throughout the 2.5 years Sunday has
the least number of failures.

O-7: Highest cloud failures around Thursday and Friday while lowest on Sunday.

Figure 3.12: 2020 Failures in week days. Figure 3.13: 2019 Failures in week
days. Figure 3.14: 2018 Failures in week days.

###  3.7 Location of Cloud failures

Figures 3.15, 3.16 and 3.17 shows the name and frequency of the ten locations
that had most failures in 2020, 2019 and 2018 respectively. The location with
highest number of failures during the 2.5 years is North Virginia reach 51
failures in 2018 which decrease to 38 in 2019. Another common location
throughout 2.5 years is Oregon having above 10 failures during 2018 and 2019
(2020 data is only of half year). Other common locations are East US, Ireland
and South US.

O-8: North Virginia and Oregon have the highest and most frequent cloud
failures.

The location analysis is limited to the data available that is we do not know
locations of all failures. In plots and analysis we also ignore the failure
that have multiple origins.

Figure 3.15: 2020 locations with most failures. Figure 3.16: 2019 locations
with most failures. Figure 3.17: 2018 locations with most failures.

##  Chapter 4 Limitations and Related Work

This chapter describes the threats to validity in the first section. The
second section analyzes related prior work.

###  4.1 Analysis of Limitations

The research looks at three different providers; Azure, AWS and GCP. The
vendors do not provide all data needed for the study. Missing data is handled
in Section 2.3. Another limitation of the data is that the results depend on
the accuracy of the data available. Furthermore, the study only includes data
from January 2018 to June 2020 which is two and a half years. When striving
for definite conclusions from the data, this quantity of data can be deemed
restricted; for example, data from more prior years can improve data
comparison.

The study’s validity is maintained by using a tool for analysis. The procedure
was repeated multiple times for the three files to ensure that the results
produced were consistent. Manual inspections were also carried out by
examining the program outputs and comparing them to the expected (manual)
output. Moreover, the majority of operations are automated, which reduces the
risk of human error. Automated validation checks were built in the tool to
ensure valid results.

###  4.2 Related Work

Currently, only a limited amount of work has attempted to analyze cloud
failures of big vendors. For example, the authors of (9) analyze outages and
incidents reported by companies and news outlets. The current study’s data was
gathered from official sources (6, 7, 8) of the vendors. Furthermore, the
current study is one of the first because it is based on recent periods. There
are studies that use data from newspaper articles to analyse cloud failures.
The study gets cloud failure data from official vendor sites. No study exists
that examines the failures of the three major cloud service providers, AWS,
Azure, and GCP.

##  Chapter 5 Conclusion

This chapter summarizes the work and contributions of the study. Cloud
services are beneficial to people all around the world. The cloud is viewed as
a solution to a variety of issues. For example, the cloud allows people to
collaborate and communicate with one another, particularly when they are in
different countries. There is, however, a long list of cloud failures that
could have a detrimental impact on billions of cloud users. Understanding
cloud failures is essential.

We created a tool to analyze and understand cloud failures. Initially the tool
was made to process raw cloud failure data. The tool was then extended to
output statistics and visuals from the data. These are then analyzed.
Comparison of failures prior to and during Covid-19 is studied. By studying
cloud failures, many of cloud failures can be prevented along with the loss
caused by cloud failures. This study covered the period 2018 till 2020-June
and provided analyses of cloud failures in big cloud providing companies; AWS,
Microsoft Azure and GCP. Furthermore, this study provides insight for reasons
of cloud failures. For further study, the tool can be extended to be used for
analysis of future cloud failures.

##  Chapter 6 Self-Reflection

###  6.1 Self-Reflection

Through this project I learned about growing demand and importance of cloud
services. It was interesting to learn about cloud failures including why,
where, how they occur and their possible solutions. During the research I
interacted with many professional members of the team that guided me through
out the study. I was greatly inspired by their words that introduced my to
methodologies that I had not known. The research showed me a variety the
techniques to analyse data. This study boosted my skills as a python
programmer as I gained experience by creating a software for the study. I
learned programming skills used for data analysis, applying statistical
methods, plotting graphs using code, various ways of representing data and
different types of graphs such as the ECDF plot. The most attractive part of
the study was the results of analysis.

During the research two-third of the time was spent on data cleaning and
visualization. Among which more time was spent by cleaning data (RQ1), than on
data transformation (RQ2). The data provided was raw data. Converting the raw
data in use-able data was done using a software created during the study.
Furthermore cleaning data involved many operations, see Section 2 The leftover
one-third time was spent on analysis and documentation.

## References

  * (1) Haryadi S. Gunawi, Mingzhe Hao, Riza O. Suminto, Agung Laksono, Anang D. Satria, Jeffry Adityatama, and Kurnia J. Eliazar.  Why Does the Cloud Stop Computing? Lessons from Hundreds of Service Outages.  In Marcos K. Aguilera, Brian Cooper, and Yanlei Diao, editors, Proceedings of the Seventh ACM Symposium on Cloud Computing, Santa Clara, CA, USA, October 5-7, 2016, pages 1–16. ACM, 2016.
  * (2) Haryadi Gunawi, Vincentius Martin, Desmond Anang, Mingzhe Hao, Tanakorn Leesatapornwongsa, Tiratat Patana-anake, Thanh Do, Jeffry Adityatama, Kurnia Eliazar, Agung Laksono, and Jeffrey Lukman.  What Bugs Live in the Cloud? pages 1–14, 11 2014.
  * (3) Katsantonis Konstantinos, Persefoni Mitropoulou, Evangelia Filiopoulou, Christos Michalakelis, and Mara Nikolaidou.  Cloud computing and economic growth.  10 2015.
  * (4) Gartner Inc.  Gartner forecasts worldwide public cloud revenue to grow 17% in 2020.  10 2019.
  * (5) Ana Gainaru, Franck Cappello, Marc Snir, and William Kramer.  Fault prediction under the microscope: A closer look into HPC systems.  In SC ’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pages 1–11, 2012.
  * (6) Microsoft Azure. Azure Status History. https://status.azure.com/en-us/status/history/. Accessed: 2020-04-15.
  * (7) Google Cloud Platform. Google Cloud Incidents JSON Feed. https://status.cloud.google.com/incidents.json. Accessed: 2020-04-15.
  * (8) AWS.  AWS Status JSON Feed. http://status.aws.amazon.com/data.json. Accessed: 2020-04-15.
  * (9) Lance Fiondella, Swapna S. Gokhale, and Veena B. Mendiratta.  Cloud Incident Data: An Empirical Analysis.  In 2013 IEEE International Conference on Cloud Engineering (IC2E), pages 241–249, 2013.