# A Milestone for FaaS Pipelines, Object Storage- vs VM-Driven Data Exchange

Germán T. Eizaguirre<EMAIL_ADDRESS>Universitat Rovira i
VirgiliTarragonaSpain ,  Marc Sánchez-Artigas<EMAIL_ADDRESS>Universitat Rovira i VirgiliTarragonaSpain and  Pedro García-López
<EMAIL_ADDRESS>Universitat Rovira i VirgiliTarragonaSpain

###### Abstract.

Serverless functions provide high levels of parallelism, short start-up times,
and “pay-as-you-go” billing. These attributes make them a natural substrate
for data analytics workflows. However, the impossibility of direct
communication between functions makes the execution of workflows challenging.
The current practice to share intermediate data among functions is through
remote object storage (e.g., IBM COS). Contrary to conventional wisdom, the
performance of object storage is not well understood. For instance, object
storage can even be superior to other simpler approaches like the execution of
shuffle stages (e.g., GroupBy) inside powerful VMs to avoid all-to-all
transfers between functions. Leveraging a genomics pipeline, we show that
object storage is a reasonable choice for data passing when the appropriate
number of functions is used in shuffling stages.

serverless computing, function-as-a-service

##  1\. Problem Statement and Goals.

Serverless computing platforms (such as AWS Lambda, IBM Cloud Functions, etc)
provide on-demand scalability and fine-grained resource allocation. This means
that a developer can leverage massive levels of parallelism in just a few
seconds to build applications with rapid horizontal scaling. Along with their
ease-of-use, these platforms have been recently used to execute serverless
workflows composed of a sequence of execution stages, which can be represented
as a directed acyclic graph (DAG). DAG nodes correspond to serverless
functions and edges correspond to the flow of data between dependent stages.

Unfortunately, serverless functions do not support point-to-point
communication. As a result, the standard practice for passing intermediate
data between serverless functions is through remote object storage (e.g., IBM
COS). If care is not taken, however, I/O-bound stages that require all-to-all
data transfers between functions (such as GroupBy and OrderBy) can end up
bottlenecking the system. This typically occurs due to the limited throughput
of object storage services (e.g., IBM COS only supports a few thousand
operations/s).

On the positive side, object storage is cheap and an “always-on” service,
requiring little intervention from the user. It is therefore the most
comfortable option for programmers and data analysts, despite its higher
latency and lower throughput compared to other alternatives such as AWS
ElastiCache. For this reason, some practitioners prefer to run I/O-bound
stages inside large-memory virtual machine (VM) instances to minimize data
transfers. Nevertheless, these solutions do not exploit the huge aggregated
bandwidth offered by object stores.

The objective of this demo is to show the practical utility of a pure
serverless implementation for workflows. That is, we demonstrate that object
storage performs well when the appropriate number of functions is used in
I/O-bound stages. To do so, we run a genomics pipeline in two manners. One way
that is “purely” serverless using object storage, and the other when the
shuffle operation runs inside a powerful VM. In this way, we qualitatively
evaluate the pros and cons of each strategy for serverless workflows.

##  2\. Research and Technical Approach.

###  2.1. Pipeline Description.

We employ METHCOMP (Peng et al., 2018), a compression method for DNA
methylation annotation files, as an evaluation workflow, since it is habitual
in the genomics community. For instance, the ENCODE project repository
contains bisulfite data for more than $500$ samples. Unfortunately, raw data
in structured BED format111https://www.encodeproject.org/data-standards/wgbs/
can amount to tens of GBs. METHCOMP presents a compression method tailored to
methylation data that yields about $10$x better compression ratio than gzip.
Particularly, this method operates in two consecutive stages. A first sort
stage that entails all-to-all data transfers between functions, and a second
stage that is embarrassingly parallel (encoding). For the purpose of this
demonstration, we port the METHCOMP pipeline to serverless.

###  2.2. Pipeline Implementations.

We use Lithops (Sampé et al., 2021), an open
source222https://github.com/lithops-cloud/lithops Python framework for
serverless analytics. Lithops allows the parallel execution of analytics
workflows on top of cloud functions. Moreover, it supports the provisioning of
heavyweight VM instances to run computations, which gives us the chance to
seamlessly run different incarnations of the same pipeline. We use IBM Cloud
as the cloud provider for both scenarios. IBM COS is used as data passing
mechanism in both pipelines.

Our first implementation is VM-based (Figure 1, A). Thanks to Lithops VM
provisioning, we execute the sort stage within a VM with sufficient physical
memory, while resorting to cloud functions only for the encoding stage.

Our second implementation is purely serverless (Figure 1, B). For the sort
operation, we use Primula (Sánchez-Artigas et al., 2020), an extension of
Lithops that optimizes all-to-all transfers between functions. Along with a
number of I/O optimizations for serverless all-to-all communication, Primula
finds the optimal number of functions for a given shuffle data size “on the
fly”. For I/O-bound tasks, using the optimal number of functions in terms of
remote storage resource utilization is crucial for good performance (Sánchez-
Artigas et al., 2020).

Figure 1. Purely serverless (A) and hybrid implementations (B) of the genomics
compression pipeline.

Purely serverless (upper branch) and hybrid implementations (lower branch) of
the genomics compression pipeline.

###  2.3. Experiment setup and results

In the demo, we will evaluate both configurations in the us-east region using
the $3.5$GB dataset:
ENCFF988BSW333https://www.encodeproject.org/experiments/ENCSR515MHO/. We will
allocate $2$GB of memory to cloud functions, and use a bx2-8x32 IBM Virtual
Server Instance as our VM for the sort stage.

Table 1 shows our results in terms of end-to-end latency and cost for a
parallelism degree of $8$ workers. End-to-end latency includes startup times.
Cost subsumes the following charges: the cost of cloud functions, storage
requests, and the VM expenses — i.e., execution time and storage volume. While
both configurations deliver similar costs, the “purely” serverless
architecture significantly outperforms the hybrid pipeline in terms of
latency. This shows that the execution of DAGs over a serverless
infrastructure can be superior to a VM-based, “serverful” infrastructure
thanks to a better exploitation of the aggregated bandwidth offered by object
storage when sharing intermediate data.

Table 1. Performance of METHCOMP pipeline in terms of latency and cost for a $3.5$GB input. Configuration | Latency (s) | Cost ($)
---|---|---
”Purely” serverless | 83.32 | 0.008
VM-supported | 142.77 | 0.010

###  2.4. User Interface and Configuration.

As our main objective is to show the feasibility of executing workflows using
a pure serverless architecture, we found it imperative to provide a
declarative programming interface to represent workflows. To this end, we
augmented Lithops with a module to create pipelines from JSON configuration
files. Further, we also developed a IPython interface for job tracking in real
time, which displays the workflow progress and breaks the cost down at each
stage.

##  3\. Related work

Research on serverless analytics systems is gaining attraction in the last few
years. Lithops (Sampé et al., 2018) is an open-source, general-purpose tool
for serverless analytics. Lithops is now multi-cloud (Sampé et al., 2021) and
for legacy support, it also enables the execution of Python code as is inside
VM instances. To improve the performance of all-to-all data transfers through
object storage, Lithops integrates Primula (Sánchez-Artigas et al., 2020).

###### Acknowledgements.

This work has been partially supported by EU Horizon $2020$ (No. $825184$) and
by the Spanish Government (PID2019-106774RB-C22). Marc Sánchez-Artigas is a
Serra Húnter Fellow.

## References

  * (1)
  * Peng et al. (2018) Jianhao Peng, Olgica Milenkovic, and Idoia Ochoa. 2018\.  METHCOMP: a special purpose compression platform for DNA methylation data.  _Bioinformatics_ 34, 15 (2018), 2654–2656.  https://doi.org/10.1093/bioinformatics/bty143
  * Sampé et al. (2018) Josep Sampé, Gil Vernik, Marc Sánchez-Artigas, and Pedro García-López. 2018\.  Serverless Data Analytics in the IBM Cloud. In _Proceedings of the 19th International Middleware Conference Industry_ (Rennes, France) _(Middleware ’18)_. Association for Computing Machinery, New York, NY, USA, 1–8.  https://doi.org/10.1145/3284028.3284029
  * Sampé et al. (2021) Josep Sampé, Pedro Garcia-Lopez, Marc Sanchez-Artigas, Gil Vernik, Pol Roca-Llaberia, and Aitor Arjona. 2021.  Toward Multicloud Access Transparency in Serverless Computing.  _IEEE Software_ 38, 01 (jan 2021), 68–74.  https://doi.org/10.1109/MS.2020.3029994
  * Sánchez-Artigas et al. (2020) Marc Sánchez-Artigas, Germán T. Eizaguirre, Gil Vernik, Lachlan Stuart, and Pedro García-López. 2020.  Primula: A Practical Shuffle/Sort Operator for Serverless Computing. In _Proceedings of the 21st International Middleware Conference Industrial Track_ (Delft, Netherlands) _(Middleware ’20)_. Association for Computing Machinery, New York, NY, USA, 31–37.  https://doi.org/10.1145/3429357.3430522