# Domain-Specific Computational Storage for Serverless Computing

Rohan Mahapatra Soroush Ghodrati Byung Hoon Ahn∗ Sean Kinzer Shu-ting Wang
Hanyang Xu
Lavanya Karthikeyan Hardik Sharma§ Amir Yazdanbakhsh♭ Mohammad Alian‡ Hadi
Esmaeilzadeh
Alternative Computing Technologies (ACT) Lab
University of California, San Diego
§Google Inc. ♭Google Research, Brain Team ‡University of Kansas
<EMAIL_ADDRESS>{soghodra, bhahn, skinzer, shw328<EMAIL_ADDRESS><EMAIL_ADDRESS>
{hardiksharma<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

While (1) serverless computing is emerging as a popular form of cloud
execution, datacenters are going through major changes: (2) storage
dissaggregation in the system infrastructure level and (3) integration of
domain-specific accelerators in the hardware level. Each of these three trends
individually provide significant benefits; however, when combined the benefits
diminish. Specifically, the paper makes the key observation that for
serverless functions, the overhead of accessing dissaggregated persistent
storage overshadows the gains from accelerators. Therefore, to benefit from
all these trends in conjunction, we propose Domain-Specific Computational
Storage for Serverless (_DSCS-Serverless_). This idea contributes a serverless
model that leverages a programmable accelerator within computational storage
to conjugate the benefits of acceleration and storage disaggregation
simultaneously. Our results with eight applications shows that integrating a
comparatively small accelerator within the storage (_DSCS-Serverless_) that
fits within its power constrains (15 Watts), significantly outperforms a
traditional disaggregated system that utilizes the NVIDIA RTX 2080 Ti GPU (250
Watts). Further, the work highlights that disaggregation, serverless model,
and the limited power budget for computation in storage require a different
design than the conventional practices of integrating microprocessors and
FPGAs. This insight is in contrast with current practices of designing
computational storage that are yet to address the challenges associated with
the shifts in datacenters. In comparison with two such conventional designs
that either use quad-core ARM A57 or a Xilinx FPGA, _DSCS-Serverless_ provides
3.7$\times$ and 1.7$\times$ end-to-end application speedup, 4.3$\times$ and
1.9$\times$ energy reduction, and 3.2$\times$ and 2.3$\times$ higher cost
efficiency, respectively.

000∗ Work done while pursuing PhD at University of California, San Diego.

##  I Introduction

(1) Serverless computing is emerging as a prevalent form of cloud execution
that has been adopted across different market sectors [1, 2, 3, 4, 5, 6, 7,
8]. This adoption is backed by the public cloud services such as AWS Lambda
[9], Google Cloud Functions [10], and Azure Serverless Computing [11]. The
popularity of serverless is driven by ease of programming, pay-as-you-go
pricing model, and alleviating the need of managing the cloud execution
environment [9, 10, 11].

Besides this shift in the cloud-native application development, the datacenter
is going through major changes: (2) storage disaggregation in the system
infrastructure level [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23], and the
(3) integration of domain-specific accelerators [24, 25, 26, 27] at the
hardware architecture level. Disaggregation is enabled by the increase in
network bandwidth to hundreds of Gbps and reduction in latency to single-digit
microseconds [16, 18, 20, 23]. Disaggregation has shown promising results in
resource utilization, elasticity, and failure mitigation in datacenters [28,
17, 29, 30]. While the improvements in networking is making storage
disaggregation a viable solution, the failure of Dennard scaling [31] and the
rise of dark silicon [32, 33, 34] has ignited a golden age of domain-specific
accelerators [35]. Accelerators [36, 25, 37, 38, 39, 40, 41, 42, 43, 44, 45,
46, 47, 48, 24, 49, 50, 51, 26, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62,
63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81,
82, 83, 84, 85, 86, 87, 88, 89, 90, 91] have made their way into the
datacenters of major cloud providers including Amazon [26], Google [24, 92],
Meta [93, 94], and Microsoft [95, 96].

Figure 1: This paper explores _DSCS-Serverless_ at the conjunction of three
different trends in cloud computing: (1) serverless functions in the
programming level; (2) storage disaggregation in the system infrastructure
level; and (3) domain-specific architectures in the hardware level.

The trend towards serverless has coincided with these two structural changes
in infrastructure and hardware. Each of these trends individually provide
significant benefits but collectively poses interesting challenges. On the one
hand, the gains from domain-specific accelerators can potentially expand
serverless usecases [97, 98, 99, 100] and/or potentially improve their speed
and efficiency. On the other hand, serverless functions operate on ephemeral
data and they need to read and store their inputs and outputs from persistent
storage for every invocation [18, 101, 102, 103]. This paper makes the
observation that for serverless applications with disaggregated storage, the
overhead of moving input and output data from remote storage limits the
benefits from acceleration. The gain will be limited since current
accelerators are inherently designed to myopically focus on the compute. They
are not meant to deal with the significant data movement cost in serverless
functions that also involves further networking when the storage is
disaggregated. Observing these insights, as shown in Figure 1, the paper
explores the confluence of the three trends and provides a pathway towards
utilizing accelerators for serverless computing on storage-disaggregated
datacenters. We propose Domain-Specific Computational Storage for Serverless
computing or for short _DSCS-Serverless_.

This idea contributes a serverless model that leverages a relatively small
programmable accelerator within computational storage to conjugate the
benefits of disaggregation and acceleration simultaneously. The proposed model
does not advocate moving back heavy compute to the storage but take _a more
balanced approach_ by integrating a rather small accelerator to mitigate the
communication overheads when applicable. In other words, some of the remote
storage nodes will have a tiny accelerator closely coupled next to the storage
media. These programmable accelerators specifically target a domain of
function and accelerate when a workload falls into the domain.

However, placing a compute near the storage comes with challenges, specially
when it is a domain-specific accelerator.

Tight power constraints. While every accelerator has its own Power,
Performance and Area (PPA), the storage device imposes a strict upper bound on
the power budget (<25 watts [104]). Further, this power budget is divided
amongst the flash and the accelerator. As such, one of the primary challenges
is to architect a near-storage accelerator that not only covers a broad range
of applications, but also adheres to the tight design constraints. We explore
using various compute platforms (ARM CPU, Low-power GPU, FPGA, and Domain-
Specific Accelerator) near-storage for a domain of serverless application
while abiding by the constraints imposed by the storage. Considering the
constraints, we perform a Pareto design space exploration that examines more
than 650 configurations.

System software stack changes. Serverless functions use frameworks such as
OpenFaaS [105] and Kubernetes [106] for deployment and orchestration. The
challenge here is how to minimally change the system stack and frameworks to
integrate the _DSCS-Serverless_ such that serverless functions can be
offloaded seamlessly. Furthermore, the changes should not interfere with the
normal execution with disaggregated storage. To identify if a serverless
function can utilize _DSCS-Serverless_ , we use software hints provided at
function deployment time. We further develop a OpenCL based runtime that can
be used to from within the serverless functions to utilize the near-storage
accelerator.

The paper makes the following contributions:

  * •

_The observation that with disaggregated storage, the overhead of moving data
from remote storage limits the benefits from acceleration in serverless
functions._ The results show that integrating a comparatively small
accelerator within the storage that fits within its power constrains
significantly outperforms traditional disaggregated systems that even use
high-end GPUs.

  * •

_The _DSCS-Serverless_ serverless model that leverages a relatively small
programmable accelerator within computational storage to opportunistically
accelerate a domain of serverless functions._ _DSCS-Serverless_ does not aim
to move back heavy compute to the storage but shows that taking a more
balanced approach by integrating a rather small accelerator can unlock the
significant gains for a domain of serverless functions amenable to
acceleration.

  * •

_The work highlights that disaggregation, serverless model, and the limited
power budget for computation in storage require an alternative design than the
conventional practices of integrating microprocessors and FPGAs._ This insight
is in contrast with current practices of designing computational storage that
are yet to address the challenges associated with the shifts in datacenters
with respect to disaggregation.

We choose machine learning domain to design a programmable accelerator that
can accelerate algorithms such as linear regression, image classification,
object detection, semantic segmentation, neural machine translation, vision
transformers, etc. and showcase an implementation of _DSCS-Serverless_. This
is because machine learning services are commonly deployed as serverless
pipelines [107, 108, 109, 110, 111, 112, 113, 114, 115, 7] and need to operate
under tight Service Level Objective (SLO) and cost budget constraints. We
evaluate the system through a rigorous study with eight real-world, latency
critical, _end-to-end_ applications inspired from AWS Lambda case studies
[109, 108, 116, 115, 117, 112, 118, 114] and model the applications as a
sequence of serverless functions on AWS. _DSCS-Serverless_ performs better
than existing computational storage solutions that either use microprocessors
[119, 120] (3.7$\times$ end-to-end application speedup, 4.3$\times$ energy
reduction, and 3.2$\times$ higher cost efficiency) or FPGAs [121, 122]
(1.7$\times$ end-to-end application speedup, 1.9$\times$ energy reduction, and
2.3$\times$ higher cost efficiency). Evaluations show that integrating a
comparatively small accelerator for _DSCS-Serverless_ significantly
outperforms a traditional disaggregated system that utilizes the NVIDIA RTX
2080 Ti GPU. In comparison, _DSCS-Serverless_ achieves 2.7$\times$ end-to-end
speedup, 4.2$\times$ energy reduction, and 3.0$\times$ higher cost efficiency.

##  II Background and Motivation

This section describes a model execution flow in serverless computing and
characterizes benchmarks on AWS [9] to identify the bottlenecks of the current
serverless computing platform.

###  II-A Life of Serverless Functions

In serverless, applications are broken down into functions having following
phases:

Figure 2: Serverless computing workflow for an end-to-end application. The
application consists of three functions exchanging data via disaggregared
persistent storage. Figure 3: Traditional serverless computing system and
_DSCS-Serverless_ that augments some storage drives with a tiny domain
specific accelerator (DSA).

Deployment. Figure 2 depicts a widely used serverless pipeline for machine
learning application that consists of three functions: Data Pre-Processing,
Machine Learning Inference, and Post-Processing & Notification Service [123,
113, 115, 108, 112]. During deployment, the application provider configures
metadata constraints (timeout, trigger mechanisms, hardware requirements,
etc.) into a YAML file for each of the functions. Since serverless functions
are stateless, the provider also allocates a persistent storage (such as AWS
S3) that is used by the functions to read and store data. The provider deploys
application modelled as chain of functions to serverless frameworks [9, 10,
124, 105].

Invocation. An application is launched when an user’s request (with data)
arrives at the storage and triggers an event. The serverless framework (AWS
Lambda [9] or OpenFaaS [105]) based on the deployment constraints from the
function’s YAML file, schedules the function to a compute node. The compute
node initiates data read using an RPC to the persistent storage node. At the
storage node, this RPC invokes a system call that reads data from the physical
storage over PCIe. The data is then serialized [125], converted to network
packets and transmitted to the compute node. After function execution, the
output data (ephemeral or not) is written back to the persistent storage
following similar steps discussed above for reading the data. Moreover, if the
function utilizes a specialized domain specific accelerator (DSA) such as
GPUs, ASICs or FPGA at the compute node, the compute node has to initiate a
data transfer (e.g. cudaMemcpy for GPUs [126]) to the DSA devices’ memory
generally over PCIe [127, 97, 128]. Overall, these steps are expensive for
serverless functions with strict Service Level Objective requirements [129,
103] since they involve RPCs [130, 101], system calls [131], and I/Os [103].

Figure 4: Runtime latency breakdown for application modeled as serverless
functions deployed on AWS EC2 with remote S3 storage.

###  II-B Characterization of Serverless Functions

As demonstrated above, there are variegated components that contribute to the
end-to-end latency of the serverless functions. We profile the benchmarks (Ref
Table I) on AWS EC2 instances using the methodology mentioned in Section VI-A.

Computation vs. Communication. Figure 4 shows the compute, communication
(network + I/O), and the system stack overhead of launching the function using
OpenFaaS [105] and Kubernetes [106]. We observe that latency to access the
storage makes up a significant portion of the end-to-end latency (on average
$>55\%$). The average for reading and writing the data to the remote storage
is greater than the time it takes to perform the computation. In fact, Credit
Risk Assessment, Asset Damage Detection, and Content Moderation consists of
more than 70% communication. Our analysis commensurate with prior studies
[132, 133] that demonstrate $\approx$75% communication overhead. This
communication overhead is naturally expected because of the serverless
function execution flows discussed earlier (refer II-A).

Figure 5: Cumulative Density Function for (a) reading inputs and (b) writing
outputs from/to AWS S3 for different data sizes.

Communication in storage disaggregated datacenters. Figure 5 shows the
cumulative density function of (a) read and (b) write latency of remote S3
storage across a range of benchmarks. The results show that both read/write
latency suffer from tail latency. However, the average latency difference
between the median and the 99th percentile is a factor of 110% and 75% for
read and write accesses, respectively. The higher average latency difference
for reads is due to larger input data size compared to the outputs (refer
Table I). This long tail latency is primarily because of disaggregation
mechanism that increases the network communication overhead. Our analyses
about tail latency of serverless functions commensurate with prior studies
[134, 135, 101, 136]. Indeed, recent work has devised solutions to mitigate
network latency for microservices or serverless functions through RPC
accelerations [130], QUIC protocol [137], and communication bypassing/fused
functions [102, 103].

Domain specific acceleration. Accelerators have been integrated into major
cloud providers including Google [24, 92], Amazon [26], Meta [93], and
Microsoft [96]. The improved efficiency of these accelerators unlocked
additional usecases in serverless computing [97, 99, 100]. However, the
primary target of these accelerators is commonly focused on computation
efficiency. From Figure 4, we observe that _with disaggregated storage, the
overhead of moving input and output data limits the benefits from acceleration
in serverless applications._ As such, the overall benefits of the current
paradigm of acceleration in a disaggregated system is strictly capped by the
Amdahl’s Law [138].

###  II-C Opportunities in Near-Data Computational Storage in Serverless
Computing

To relieve the communication overhead, we can revisit the conventional wisdom
of _computing close to the data storage_ in the scope of serverless systems.
Employing near-data computation can notably reduce the network and I/O latency
overhead. Such approach is naturally beneficial because it effectively
relocates the system bottleneck towards computation element for which we can
employ domain accelerators. This paper, _DSCS-Serverless_ , sets out to
explore this new model for serverless computing. In this model, we leverage
computational storage as a scaffold to enable near-storage domain acceleration
while being faithful to disaggregated architecture. In particular, we devise a
relatively small accelerator, under tight storage power constraints, for a
range of computationally intensive serverless functions. Finally, to offset
the network and I/O latency, _DSCS-Serverless_ leverages peer-to-peer [139,
140] channel connecting accelerator and storage units. Next, we expound the
_DSCS-Serverless_ system architecture.

##  III DSCS-Serverless Overview

Figure 3(b) outlines the high-level system architecture of _DSCS-Serverless_.
_DSCS-Serverless_ leverages computational storage drive (CSD) to architect and
integrate a relatively small near-storage domain specific accelerator (DSA) on
which a domain of functions are opportunistically executed. As shown in Figure
6, the _DSA_ directly communicates with the storage unit (e.g. flash NAND,
SSD, etc.) using a dedicated peer-2-peer (P2P) PCIe link [140]. With such
system architecture, the functions that can be accelerated with the near-
storage _DSA_ obviate the extravagant network and I/O data transfer overheads
(Remote Read/Write parts in Figure 4). This also improves the resource
utilization of the storage node by enhancing it with computational capability,
but does not replace the conventional disaggregated storage nodes. In essence,
_DSCS-Serverless_ system architecture closely follows the design philosophies
commonly-used in disaggregated storage data centers [17, 141, 142]. Note that
in case a particular serverless function is not supported by DSA, _DSCS-
Serverless_ falls back to the traditional serverless computing execution flow
with its inherent data transfer overheads.

###  III-A Life of Serverless Functions in _DSCS-Serverless_

We contrast the life of a serverless function in _DSCS-Serverless_ and the
traditional system architecture (refer Figure 3(a)).

A

$\mapsto$ In contrast to the traditional system, _DSCS-Serverless_ directly
deploys the serverless functions, for which DSA offers acceleration, to the
_DSA_ integrated storage node that has the data, thereby eliminating the
costly invocation of a compute node. Note that each invocation of compute node
in the traditional confers a notable data read latency. Section V describes
the details of how _DSCS-Serverless_ identifies the serverless functions
amenable to acceleration, followed by corresponding DSA invocations.

B

$\mapsto$ Instead of fetching data from a remote storage via costly RPC
requests, _DSCS-Serverless_ employs its driver to instead initiate a peer-to-
peer (P2P) data transfer from storage to _DSA_ memory. Specifically, in the
traditional system for an AWS S3 Read API, the RPC upon reaching the storage
node requires at least a ProtoBuf deserialized [92] and a _read_ system call
to access the file over PCIe [143]. In contrast, _DSCS-Serverless_ circumvents
the costly ProtoBuf deserialization by performing a single system calls to
initiate P2P data communication [139].

The P2P data transfers between the SSD and the _DSA_ memory subsystem bypasses
the host’s entire stack ***In evaluations, we use PCIe Gen3 with $\times$4
lanes that is used in conventional CSDs such as Samsung SmartSSD [144]. We
conduct sensitivity analysis against number of PCIe lanes in Section VI-C..
Once the data are entirely transferred to _DSA_ memory, the execution of the
serverless function commences.

C

$\mapsto$ Once the execution completes, the _DSA_ sends an interrupt over PCIe
to the host to initiate a P2P transfer of the results to the storage. Once
transfer completes, the host may invoke subsequent serverless function calls.
This is in contrast to the traditional system in which data transfer over
network and I/O events occur to write the results to persistent storage.

Once the serverless function concludes, _DSCS-Serverless_ interrupts the host
to notify the completion of the job. Using our proposed system architecture,
_DSCS-Serverless_ offers this opportunity to execute serverless function on a
single (storage) node, thereby improving overall end-to-end latency and energy
consumption. In addition, the specialized _DSA_ units economize the precious
CPU cycles on compute nodes leading to additional cost efficiency in
datacenters.

##  IV DSCS-Serverless Architecture Design

This section first discusses the architecture of the programmable domain
specific accelerator (_DSA_). Then, we showcase a methodology to derive the
optimal _DSA_ configuration under tight computational storage power
constraints. We also demonstrate a technology scaling analysis to project the
performance numbers for more recent technology nodes. Note that, while we
present the design space exploration framework for Machine learning and Data
analytics applications, it can be readily employed to find optimal _DSA_ s for
alternative application domains.

###  IV-A Architecture Design of Domain Specific Accelerator

Machine learning and Data analytics applications are one of the fastest
growing domains in serverless, accounting for more than 40% of services [145,
7, 11]. As such, we set the primary target of our design to accelerate such
applications. The important design decisions that we considered for the
architecture were: programmability such that it can cater to a wide range of
commonly deployed machine learning workloads and low-power such that it can
abide by the power constraint imposed by the storage. Specifically, the
architecture can support a wide range of machine learning inference tasks such
as image classification, object detection, semantic segmentation,
linear/logistic regression, neural machine translation, conversational AI,
data pre-processing, etc. Figure 6 demonstrates our systolic array based _DSA_
architecture, inspired by recent industry designs such as Google TPU [24,
146], Amazon [26], Meta [93], and academic architectures [147, 148, 36, 38,
39, 40, 52]. Additionally, we design and integrate a vector engine in _DSA_.

Figure 6: Architecture diagram of _DSA_ placed near storage for _DSCS-
Serverless_ as highlighted in Figure 3

Systolic array accelerator. The systolic unit consists of a 2D array of
Processing Elements (PEs) and dedicated multi-bank scratchpads for input
activations, weights, and outputs as shown in Figure 6. Each bank of
scratchpad units is exclusively shared across PEs within a row. The execution
flow of such architecture is similar to conventional systolic-array
accelerators [149, 43, 150]. At each cycle input activation tensors are
fetched from input scratchpads and shared across the PE units within a row.
The partial sum from the PEs are forwarded in a waterfall fashion per column.
Once the computations across the array of PEs conclude, the results are either
fed to the Vector Engine for ensuing operations or written back to DRAM.

Vector engine unit. The Vector Engine is a Single Instruction Multiple Data
(SIMD) architecture designed primarily to execute activation functions (e.g.
Relu, LeakyRelu, Tanh, Sigmoid), pooling, quantization, vector arithmetic
computations, and datatype casting which are prevalent in emerging machine
learning models [151, 152, 153].

Furthermore, machine learning models generally require data pre-
processing/post-processing such as image scaling, normalization, and datatype
casting [123, 117, 109, 110]. These transformations are commonly packaged as
separate serverless functions. For instance, _Asset Damage Detection_
benchmark [117] shows the pipeline for image classification that contains
function solely for data pre-processing. We utilize the Vector Engine to
execute these functions as well. In summary, the tightly-couple design of a
systolic array accelerator and a vector engine unit, enables the execution of
a broad range of serverless function services, from heavy GEMM operations to
simple data processing functions.

###  IV-B Design Space Exploration for Optimal Near-Storage DSA

To effectively support the design space exploration of _DSA_ , we designate
the number of PEs, systolic array X-Y dimension, on-chip scratchpad sizes, and
memory bandwidth as configurable parameters. Such design space exploration is
crucial to ensure that _DSA_ architecture satisfies tight power and thermal
constraints of computational storage drives †††The commercial Samsung SmartSSD
[144] reserves 18 watts and 93°C power and thermal constraints, respectively.
This tight design constraints are primarily because of the limited PCIe power
budget ($\leq$ 25 watts [104]) that is the exclusive source of power for CSD
units Note that the power source of each CSD unit is shared between storage
(flash) and compute devices [119, 122, 121]. To provide an estimate of the
CSD’s limited power budget, Samsung’s SmartSSD [122, 121] has merely a TDP
(ideal) of 18 watts. In addition, prior work has shown that computational
storage, when appropriately designed, can significantly outperform
conventional servers with high-end CPUs and GPUs [154, 155]. As such,
performing proper design space exploration for _DSA_ is crucial.

DSE objective. There are various conflicting factors to take into account to
identify optimal design points for CSDs. Commensurate with prior work [156,
157], we use throughput (frames per second) as the performance metric. Capital
expense incurred by ASIC fabrication is another determining factor for
designing data centers [158, 159]. However, measuring the precise capital
expense is not pragmatic and kept confidential by major cloud providers [24,
160]. Therefore, we use chip area as a proxy for the ASIC fabrication cost. We
use _DSA_ power consumption to assess the feasibility of a design point [156],
which is capped by the CSD’s limited power budget. The objective of DSE is to
find the design points that are on the Pareto frontier of the
power$\leftrightarrow$performance and area$\leftrightarrow$performance.
Particularly, we only select _DSA_ architecture configurations that are on
Pareto frontiers while abiding by the tight power and area constraints of
CSDs. We choose the open source 45 nm FreePDK [161] technology node for our
baseline analysis in the design space exploration.

FPGA implementation and simulation methodology. To build the design space
exploration infrastructure, we first implement and synthesize _DSA_ (refer
Section VI-A). Since hardware simulations for the entire design points ($>$
650) is not practical, we develop a cycle-accurate simulator to closely model
the latency and power of our designed _DSA_ (refer Section VI-A). For power
and area numbers, we use the synthesized values using 45 nm technology node.
We followed the methodology in [162] to scale the results to 14 nm, which is
relatively similar to the technology node of Samsung SmardSSD [144]. We
provide the details of methodology in Section VI-A.

Design space exploration search space. We use Google TPUv1 [24] with
256$\times$256 PEs, 28 MB of on-chip scratchpads, and 34 GB/s memory bandwidth
as the standard design point. We then scale this design by varying the number
of PEs from 4$\times$4 to 1024$\times$1024 with a power of 2 stride. We
proportionally scale the scratchpads to provide sufficient on-chip resources
for PEs. However, we set the maximum scratchpads size to 32 MB because large
scratchpad sizes significantly increase the power consumption, exceeding the
tight power constraints for _DSA_. We use three realistic memory bandwidth in
the search space, namely DDR4 (19.2 GB/s), DDR5 (38 GB/s) and HBM2 (460 GB/s)
[163, 164].

Figure 7: Performance frontiers, 45 nm Tech Node.

Pareto-optimal design points. Figure 7 demonstrate the
area$\leftrightarrow$performance and power$\leftrightarrow$performance results
across a range of design points. We exclude the design points that are either
infeasible because of design constraints or significantly inefficient in terms
of throughput. The power and throughput of each design point is an average
across set of target benchmarks (See Table I). The design point at the bottom
left portion of Figure 7(a), represents a 4x4 systolic array with 128 KB on-
chip scratchpad and DDR4 memory. The top right design configuration shows a
128$\times$128 systolic array with 4 MB on-chip scratchpad and DDR5 memory.
The results of our design space exploration indicates that a 1024$\times$1024
systolic array delivers significantly lower throughput compared with
128$\times$128 array. This is because the _DSA_ employs a tiling-based
execution mechanism. For a batch size of one‡‡‡We use batch size 1 for our
baseline design because serverless functions generally target real-time user
facing applications with tight service level objectives  [103] . In these
scenarios, additional latency for batching is not desirable. Nonetheless,
Section VI-C provides sensitivity results to batch sizes., a 1024$\times$1024
systolic array does _not_ show the most optimal performance because the
compiler (Section V) aims to obtain the optimal tiling such that the _DSA_
overlaps the memory transfers for a tile with the computation of preceding
tile. If the tile sizes are large, the cycles spent on memory transfer
outweigh the compute cycles. This is similar to CPU pipelines where stalls
reduce the IPC. Through our design space exploration, we find the most optimal
_DSA_ configuration to be a 128$\times$128 systolic array with 4 MB on-chip
scratchpad, and DDR5 memory.

##  V DSCS-Serverless: System Consideration

_DSCS-Serverless_ requires minimal modifications to (1) the programming model
to express near-data acceleration using _DSA_ , and (2) the system software
stack to identify and leverage nodes capable of near-storage acceleration
using _DSA_.

System software stack. _DSCS-Serverless_ is deployed atop containerized
serverless frameworks. We implement and deploy the _DSCS-Serverless_ software
stack using OpenFaaS [105], an open source serverless framework. However, the
_DSCS-Serverless_ can be readily deployed into other serverless platforms such
as Apache OpenWhisk [124]. OpenFaaS is deployed on Kubernetes [106] to
orchestrate containers, and uses Promethus [165] for cluster monitoring and
telemetry.

Programming model. To utilize serverless functions using OpenFaaS, users
define their target applications as a Directed Acyclic Graph (DAG) of
decoupled serverless functions. During deployment of these functions, the user
provides a YAML file to describe the properties and constraints of the
function (eg. dependencies, timeout, access mechanism, storage, etc.). _DSCS-
Serverless_ extends this YAML file format to enable users to mark near-data
_DSA_ acceleratable functions. In addition, users provide a container that
packages the accelerated serverless function with the appropriate device
drivers and libraries.

Data placement. To enable effective utilization of near-storage accelerators,
it is essential that the data to be processed is situated on the same storage
drive that houses the accelerator. Cloud service providers today offer various
storage classes for different types of data (hot and frequently accessed data,
cold data, and archived data, etc.) [141, 142]. We extend the storage class
[166] to add a new class named _Acceleratable_Storage_. When a function is
designated as acceleratable during deployment, the storage provisioned for the
data comprises _DSCS-Serverless_ storage drives. During invocation, all data
requests for this application are directed to _DSCS-Serverless_ capable
storage drives. Further, we exploit the fact that serverless functions have a
strict maximum payload size per request (e.g., 256 KB on AWS Lambda [167]) to
ensure that the entire payload is routed to the same drive. As the number of
requests (data) increases, it is possible to store different requests on
separate drives that support _DSCS-Serverless_. This is because requests
(data) are independent of each other, and therefore, the scheduling of
requests (data) can be distributed across different drives. The scheduler
relies on Prometheus [165] telemetry to decide whether to employ near-storage
acceleration or execute the function in a conventional manner.

Function scheduling and fallback options. We extend the centralized Kubernetes
scheduler to expose storage nodes that can utilize near-data _DSA_ and
opportunistically map acceleratable serverless functions execution to such
nodes if the data resides on the node. The scheduler uses a simple FCFS
scheduling policy for the incoming function execution requests, leveraging
Prometheus [105] to monitor availability and prevent overloading a single
Kubernetes pod. Note that similar to traditional serverless [9, 10, 11], a
function instance on the _DSA_ also does _not_ support multi-tenancy and
follows a run-to-completion execution policy. Once a function is offloaded for
computation on _DSCS-Serverless_ , the storage node marks its compute status
as _busy_. The scheduler does not offload more functions until the node
becomes _available_. When no _DSA_ -capable storage node is available or a
_DSA_ is already processing some other function, the scheduler falls back to
traditional execution and utilizes remote compute nodes (CPU) for function
execution instead. This is possible because the _DSCS-Serverless_ can be used
as a traditional storage drive as well. Finally, we leverage the existing
Kubernetes mechanisms for fail-over support and container migration.

Storage scalability. Remote storage can be scaled independently from compute,
thereby enabling virtually unlimited storage capacity. This independent
scalability is not limited by _DSCS-Serverless_ because it can still operate
as a traditional storage node for applications that do not require the compute
acceleration capabilities of _DSCS-Serverless_. Moreover, serverless functions
do not require complex scaling of compute and storage since they have strict
storage and compute quotas [167] and serverless deploys horizontal scaling to
launch more function instances to handle new requests. Therefore, _DSCS-
Serverless_ does not restrict independent storage scalability.

Cold starts. Functions in _DSCS-Serverless_ incurs the same cold start as
functions in traditional platforms. Cold starts occur when the function is
deployed for the first time to a node where the function’s container image is
pulled from a remote registry, unpacked, and has to pass a health check.
During horizontal scaling, each time a function scale from N to N+1 replicas
of a function, the same process described above takes place. However, we
configure the serverless framework to preemptively lunch some container (until
the horizontal scaling threshold is reached) if all other containers are busy.
This hides the cold start latency because we have at least some replicas to
cater to the current load of the system. Further, similar to traditional
mechanism where the function is kept warm on the compute node’s memory for
certain amount of time, _DSCS-Serverless_ also stores the function on the
_DSA_ ’s memory for some duration preemptively waiting for new requests.

Device driver and libraries. To support near-storage acceleration, _DSCS-
Serverless_ includes a OpenCL device driver [168]. The driver implements
standard interfaces for mapping storage space to physical address space of
both the storage node and the _DSA_ ’s configuration registers and memory.
Additionally, the driver orchestrates direct peer-to-peer (P2P) data transfer
between the storage and the _DSA_ that bypass the storage node’s system stack
and utilizes dedicated PCIe links. The OpenCL-based driver also abides by the
host OS security checks for access control to both storage and _DSA_. The
acceleratable function’s container include all dependent libraries such as
OpenCL framework, runtime and associated tools that is used to program _DSA_.

Compiler support. We develop a compilation stack capable of code generation
for different _DSA_ configurations and for a range of evaluated machine
learning models. The serverless functions that utilized the _DSA_ are
implemented using Pytorch [169] and stored as ONNX (Open Neural Network
Exchange) files [170]. The front-end part of the compiler performs a range of
optimizations, including operator fusion to minimize off-chip data movement.
Then the compiler performs _DSA_ design configuration (e.g. number of PEs,
memory bandwidth) specific optimizations such as padding and tiling to
maximize the _DSA_ ’s utilization. Once these optimization passes complete,
the compiler generates the hardware configuration specific optimized
executable code. This code is packaged along with the serverless function in
the container. Note that we rely on the user to partition their application
into server functions that can or cannot be accelerated using a near-storage
_DSA_.

Host and storage communication. Prior work [171, 172, 173] suggested the use
of multi-channel DMA over PCIe for high throughput. This work follows the same
methodology and configures our device driver to use multi-channel DMA. The
computational storage contains a PCIe switch as shown in Figure 6 similar to
prior work [174, 175]. CSD employs this switch to route incoming requests to
either the storage or _DSA_. _DSA_ s communicate with the storage device over
peer-2-peer PCIe links.

##  VI Evaluation

TABLE I: The benchmarks evaluated, serverless functions utilized, machine
learning model, and input/output size for the model.

Application | Description | Serverless Functions | DNN Model (# Params) | Input/Output Shape
---|---|---|---|---
Credit Risk Assessment [176, 114] | Identify positive and negative credit risks for loan approval. | f1: Normalization; f2: Logistic Regression; f3: Notification Service |  Logistic Regression [45] (200) | (1,200)/ (1)
Asset Damage Detection [117] | Detect damages to objects using images captured by CCTV. | f1: Image Preprocessing; f2: Image Classification; f3: Notification Service |  ResNet-50 [177] (25 million) | (1,3,224,224)/ (1,1000)
Auto-PPE Detection [113] | Detect protective gear on factory workers to prevent hazards. | f1: Image Preprocessing; f2: Object Detection; f3: Notification Service |  YOLOv3 [178] (65 million) | (1,3,416,416)/ (1,255,52,52)
Clinical Analysis [111, 112] | Identify salient elements in medical scans. | f1: Image Preprocessing; f2: Semantic Segmentation; f3: Notification Service |  FCN [179] (54 million) | (1,3,224,224)/ (1,3,224,224)
Content Moderation [109, 110] | Detect inappropriate or offensive content. | f1: Image Preprocessing; f2: Image Classification; f3: Notification Service |  EfficientNet [180] (11.5 million) | (1,3,224,224)/ (1,1000)
Conversational Chatbot [115] | Answers input text questions. | f1: Text Preprocessing; f2: Question & Answering; f3: Response |  BERT-Base [151] (110 million) | (1,128,768)/ (1,128,768)
Document Translation [107, 108, 116] | Localize content on websites for cross-lingual communication. | f1: Text Preprocessing; f2: Neural Machine Translation; f3: Response |  GPT-2 [152] (1.5 billion) | (1,1,128)/ (1,1,128)
Remote Sensing [118] | Traffic monitoring using unmanned aerial vehicles. | f1: Image Preprocessing; f2: Scene Classification; f3: Notification Service |  Vision Transformer [153, 181] (632 million) | (1,3,224,224)/ (1,1000)

###  VI-A Methodology

TABLE II: Specification of Traditional, Conventional Near-Storage, and
Proposed platforms used for evaluation.

| TRADITIONAL PLATFORMS | CONVENTIONAL NEAR-STORAGE PLATFORMS (NS) | PROPOSED
---|---|---|---
| Baseline (CPU) | GPU | FPGA | NS-ARM | NS-Mobile-GPU | NS-FPGA | DSCS-Serverless
Chip |  Intel Xeon Platinum 8275CL [182] |  Nvidia GeForce RTX 2080 Ti [183] |  Xilinx Alveo U280 Data Center Accelerator Card [163] |  Arm Cortex A57 [184] |  NVIDIA Pascal GPU [185] |  Samsung SmartSSD [121] | TPU-like [149]
Cores/ PEs | 16 cores | 4352 CUDA Cores | 1024 PEs | 4 cores | 256 CUDA cores | 256 PEs | 16,384 PEs
Memory | 32 GB | 11 GB GDDR6 | 32 GB(2 MB on-chip buffer) | 4 GB | 4 GB | 4 GB (2 MB on-chip buffer) | 4 GB (4 MB on-chip buffer)
TDP | 240 W | 250 W | 225 W | 15 W | 15 W | 18 W | 4.2 W
Frequency | 3 GHz | 1.35 Ghz | 250 Mhz | 2 GHz | 1.3 GHz | 250 MHz | 1 GHz
Tech. Node | 14 nm | 12 nm | 16nm | N/A | N/A | 16 nm | 14 nm
PCIe | N/A | Gen3x16 | Gen4x8 | N/A | N/A | Gen3x4 | Gen3x4

Benchmarks. To evaluate the efficacy of _DSCS-Serverless_ , we use eight real
world latency critical applications representing serverless pipelines deployed
on AWS Lambda [9]. Table I shows the suite of benchmarks, their description,
serverless functions, and the corresponding inputs/outputs sizes. Each
application is executed in a pipeline similar to the one shown in Figure 2,
where we offload both data pre-processing (f1) and machine learning inference
(f2) to the compute platform (GPU, FPGA, ARM, _DSA_). Since the exact machine
learning models used in AWS Lambda functions are not publicly available for
some benchmarks, we use representative and state-of-the-art inference models
from MLCommons [186] or Hugging Face [187] that provide similar functionality
as AWS Lambda functions (e.g. AWS Rekognition [113] offers image
classification service and we use ResNet-50 [188]). We containerize all the
serverless functions by using OpenFaaS [105]. The model weights of ML
functions are stored on the container while the input/outputs use remote
storage through AWS S3 API calls.

Baseline system setup. For the _baseline_ , we use _Amazon EC2 c5.4xlarge_
with _Intel(R) Xeon(R) Platinum 8275CL_ CPU cloud instance and use an _IAM_
account to connect the EC2 virtual machine to a S3 object storage in the same
region. The EC2 instances runs _Ubuntu 20.04.4 LTS_ with kernel version
_5.13.0-1029-aws_. We spawn a cluster using Kubernetes [106] on the EC2
instance and deploy OpenFaaS [105] as a pod on the Kubernetes cluster. All the
benchmarks are deployed and registered with the OpenFaaS function registry at
the deployment time. All evaluations are based on warm containers unless
specificed otherwise. We use _hey_ [189], a http load generator tool to invoke
the serverless application. Each request follow a run-to-completion model.

Evaluation of compute platforms. We consider two scenarios: First the
_Traditional Platforms_ , when the compute device (refer Table II) is a
separate node and accesses the remote storage via network as shown in Fig. 3.
We consider three commodity high-end compute platforms commonly used in data
centers for this scenario: an Intel Xeon CPU (baseline), a NVIDIA GPU, and a
Xilinx datacenter FPGA [190] (programmed with a relatively smaller _DSA_ due
to resource constraints of the FPGA). Second is the _Conventional Near-Storage
platforms (NS)_ , where the compute is placed near the storage and connected
via a peer-2-peer PCIe link to the storage. Since these are not available on
datacenters, we setup the infrastructure locally similar to the baseline
setup. We consider three low power near storage platforms: a quad-core ARM
CPU, a Mobile-GPU (Nvidia Jetson TX2), and the near-storage FPGA (Samsung
SmartSSD [121]). Since we did not have access to ARM Cortex A53 which is used
in commercially available CSDs [119, 120], we use an even powerful ARM core
(Cortex A-57) for our evaluation. Table II reports the specifications of all
the evaluated compute platforms.

System performance measurements. For the baseline measurement, we use the
aforementioned baseline setup and invoke the application by generating 10000
sequential requests using hey, a http load generator to measure the latency.
We use the 95th percentile latency for all our analysis similar to prior work
[103, 102]. To measure the latency for all other _traditional compute
platforms_ , we create containers with the required environments (e.g.
onnxruntime for GPU or Xilinx XRT for FPGA in addition to their corresponding
drivers). For the case of _conventional near-storage platforms_ , we develop
an analytical model where we modify the baseline system by omitting the
latency to access the remote storage and add the peer-2-peer latency for
transferring the data between the storage and compute platform, and other
required latencies (e.g. instructions, model weights, etc.). To obtain the P2P
latency of data transfer, we emulate this communication on a Samsung Xilinx
SmartSSD and measure the time it takes to transfer data between the FPGA and
storage for all the benchmarks 10000 times and sample the 95th percentile
latency.

Hardware implementation and synthesis. We implement the _DSA_ in 15k lines of
Verilog and synthesize it using Synopsys Design Compiler R-2020.09-SP4 with
FreePDK 45nm standard cell library. The design was able to achieve 1GHz
frequency. Further, to synthesize the _DSA_ and program it on Samsung Xilinx
SmartSSD FPGA, we use Xilinx Vitis/Vivado [191] and Xilinx XRT Runtime [192] .
We also use the Xilinx Vivado to obtain the resource utilization, timing,
power and thermal statistics for the FPGA analysis.

Simulation infrastructure. We compile each machine learning model to our
domain-specific accelerator ISA and generate executable binaries. We develop a
cycle-accurate simulator for _DSA_ ASIC implementation, which uses compiler-
generated instructions, and provides cycle counts and energy statistics. We
compare the simulator provided results with the FPGA implementation of _DSA_
on the SmartSSD card for the same design configuration and frequency to verify
the closeness of the cycles by error margin of $\leq$ 10%. We use this
simulator to obtain the performance/energy numbers for _DSA_ ASIC
implementation and design projections mentioned in Section III. Further, we
will be open-sourcing both the hardware implementation and the software
simulator.

Power measurements. We measure the compute, PCIe and system stack power
dissipation and combine them to report the energy efficiency of the system.
Although, serverless systems also use the network (Ethernet/Internet),
measuring the power for it is not realistic and therefore we omit the network
power for all the traditional systems. We use the Intel RAPL[193, 194] and MSR
registers to get the Server-CPU power. To obtain the power for the ARM CPU and
GPU on Jetson TX2, we use NVPModel tool from NVIDIA Jetson TX2 Development Kit
[195]. We use Xilinx Vivado [191] to measure the power for FPGA implementation
of _DSA_ on Samsung Xilinx CSD and Alveo u280. To obtain power for the ASIC
_DSA_ , we use synthesis results to measure the logic cell power and CACTI-P
[196] to model on-chip memory energy. For PCIe, we use the per-bit PCIe power
reported in prior work [197].

Cost efficiency model. To assess if a new design offers cost savings over
other systems, we evaluate cost efficiency [159], which is the average peak
throughput over total cost and time of ownership as shown in the equation
below.

$\vspace{-1ex}\text{Cost Efficiency}=\frac{\text{Throughput}\times
T}{\text{CAPEX}+\text{OPEX}}$ $\text{where OPEX }=\Sigma(Power\times T\times
Electricity)$

_CAPEX_ is the one time capital expenditure ($) to setup the hardware
(including the networking, host server, etc.). We identify the cost to procure
a compute platform from the respective company website [122, 183, 182, 163,
119]. To compute the _CAPEX_ for the proposed _DSA_ , we use the analytical
model from ASIC Clouds [158]. Further, _OPEX_ is the day to day operating cost
($) of hardware. It is the product of the power (watts) for various components
in the cluster, the time for which the cluster is active (T) and the average
price of electricity in U.S $0.0733/kWh [159]. We compute the _Cost
Efficiency_ for a period of 3 years [160].

Throughput measurement. Throughput is the maximum requests per second achieved
by the system while meeting the SLA of 99%. To measure throughput, we create
an application trace by randomly sampling functions from the benchmarks (Table
I) using Poisson distribution, and induce load on the system for 30 minutes.
We emulate 100 _DSCS-Serverless_ on the baseline system by launching 100
serverless functions that busy-wait for the duration it would take to execute
the function on the _DSA_. The choice of 100 is for feasibility of evaluation.

###  VI-B Experimental Results

Figure 8: Normalized speedup for application designed as serverless function
shown in Figure 2. Figure 9: Normalized runtime breakdown.

Performance comparison. Figure 8 compares the performance of various platforms
listed in Table II for both traditional (with remote storage) and Near-Storage
(denoted as NS) scenarios across all studied benchmarks. The speedups are
normalized to the baseline (see Table II). On average, _DSCS-Serverless_
provides 3.6$\times$ speedup over the baseline across all benchmarks.
_Moreover, the results suggest that leveraging a lightweight accelerator (4.2
watts) for _DSCS-Serverless_ outperforms a high-end GPU (250 watts) with
remote storage by 2.7$\times$._ This is because: First, the inherent data
movement latency in remote storage setting limits the performance benefits
from the high-end GPU. Second, the usage of batch size one for serverless
scenarios causes underutilization in GPUs. Using a FPGA slightly underperforms
the baseline due to the driver overhead and limited FPGA resources.

To tackle the communication overheads, we analyze near-storage computing. As
shown in Figure 8, using near-storage with a quad-core ARM CPU slightly
underperforms the baseline. Compared to the baseline, mobile GPU provides
1.35$\times$ speedup, while leveraging a NS-FPGA unlocks 2.2$\times$ speedup.
The speedup for low-power NS-FPGA seems counter-intuitive compared to high-
power FPGA (with remote storage) because the latter was bottlenecked by the
communication overhead. _This analysis shows that the overhead of moving input
and output data from remote storage limits the benefits from acceleration._
Nevertheless, FPGA’s performance is still bounded by it’s limited resources
and low frequency. As shown in Figure 8, _leveraging a domain specific
architecture near the storage unlocks additional benefits and provides 3.7
$\times$ and 1.7$\times$ speedups over the conventional approaches of using
microprocessor and SmartSSD near the storage, respectively._ _Credit Risk
Assessment_ exhibits the least speedup because logistic regression is not
computationally intensive while _PPE Detection_ achieves the maximum speedup
because moving compute to near-storage reduces the significant data movement
which the benchmark otherwise incurs.

Runtime breakdown analysis. Figure 9 shows the runtime breakdown across the
individual system components for all the benchmarks and platforms. We see that
for _traditional platforms_ with GPU/FPGA (with remote storage), the compute
portion is significantly reduced due to hardware acceleration. However, the
data transfer over network limits the effective speedup achieved by the
hardware acceleration. This significant data transfer is addressed by the
near-storage platforms where moving compute closer to storage reduces the data
movement, shifting the bottleneck back to the compute. The _DSA_ further
accelerates this compute portion unlocking additional performance gains.
Overall, we observe that leveraging _DSCS-Serverless_ shifts the bottleneck
from compute and communication to other components such as the system stack.

For benchmark _Credit Risk Assessment_ , Figure 4 shows that data movement
accounts for approximately 75% of the runtime. Intuitively, moving compute to
near-storage should provide 4x speedup. However, we observe 1.8x speedup
because of two reasons. First, using near-storage has it’s driver overhead
that contributes significantly towards the end-to-end runtime (order of
O(ms)). Second, as mentioned in the methodology VI-A, f3 function is launched
on the CPU and experiences the network and IO latency similar to traditional
systems. As depicted in Figure 9, for _DSCS-Serverless_ the bottleneck now is
the latency incurred by the function f3 to read the data from persistent
storage and the system stack overheads.

Figure 10: Normalized system energy reduction for application.

Energy reduction comparison. Figure 10 analyzes the end-to-end system energy
reduction achieved by _DSCS-Serverless_. On average, _DSCS-Serverless_
provides 3.5$\times$ energy reduction over the baseline system and 1.9$\times$
reduction over the NS-FPGA (SmartSSD), the most competitive baseline. FPGAs
have significantly higher static energy dissipation and thus cannot match the
energy efficiency on an ASIC. Although leveraging _DSA_ provides significant
energy reduction (29$\times$ over CPU baseline), the total system energy
reduction is bounded by the system stack and _f3_ function being executed on
the CPU. The trends in energy reduction are similar to the speedup, with _PPE
Detection_ showing the maximum gains (8x) and _Credit Risk Assessment_ showing
the minimum (1x).

Figure 11: Normalized cost efficiencies.

Cost efficiency. Figure 11 shows the cost efficiency for various platforms
normalized to the baseline. Results show _DSCS-Serverless_ offers the highest
cost efficiency (3.4$\times$) compared to the CPU baseline, while NS-FPGA
(SmartSSD) ranks second (1.6$\times$). This result is intuitive, since over
the initial period of usage, the _CAPEX_ cost of building a hardware is
dominant. As the time goes, the _OPEX_ cost, that is cost of operating
(electricity cost) becomes more dominant. Since _DSCS-Serverless_ consumes
lesser energy compared to other platforms, it’s cost efficiency increases over
time.

Figure 12: Throughput improvement over CPU baseline.

Throughput. Figure 12 compares the throughput of _DSCS-Serverless_ relative to
baseline CPU for requests sampled from the application trace (Refer Sec VI-A).
_DSCS-Serverless_ improves the throughput by 3.1x on average. For _Credit Risk
Assessment_ the throughput improvement is not significant since it is not
compute intensive and therefore acceleration does not provide significant
benefits. Overall, _DSCS-Serverless_ improves the throughput of the system
since each _DSCS-Serverless_ instance can process more requests per second as
compared to the baseline. On the contrary, the baseline queues more request
and results in a greater number of requests that violate the 99% SLA.

###  VI-C Sensitivity Analysis

Figure 13: Sensitivity to batchsize

Batch size. Figure 13 shows the sensitivity of the _DSCS-Serverless_ end-to-
end performance with respect to batch size (refer I). We sweep the batch size
from one to 64 across all benchmarks and report the performance of _DSCS-
Serverless_ normalized to the baseline (CPU) with remote storage, running the
same batch size. The rationale behind limiting the batch size to 64 is because
AWS Lambda has a strict cap on the network payload size for serverless
functions [167]. Relative to the baseline, the performance improvements of
_DSCS-Serverless_ increases from 3.6x for batch size 1 to 15.9x for batch size
64. This performance improvement stems from (1) reducing the communication
overheads of transferring batched data to compute node compared to the
baseline and (2) the capability of the _DSA_ in reusing the weights across the
batch, thereby improving the computation significantly. From all the
benchmarks, the improvements are more pronounced for _Conversational Chatbot_
and _Document Translation_ , since these benchmark applications deploy
language transformer models with large number of weights, where the _DSCS-
Serverless_ leverages batching to amortize the cost of loading weights by
reusing them across the input batch.

Figure 14: Sensitivity to the number of accelerated functions.

Number of accelerated functions. To analyze the sensitivity of the _DSCS-
Serverless_ performance to the number of accelerated functions, we create
synthetic benchmarks by adding either one, two, or three additional
accelerated functions to the application. These added functions are all
replicated from the _f2_ function of the original benchmarks. The label in
Figure 14 refers to the number of replicated functions. The performance is
normalized to the baseline (CPU) running the same function configuration.
Results show that increasing the number of functions that are offloaded to
_DSCS-Serverless_ , the improvements escalates (from 3.6$\times$ to
8.1$\times$). Increasing the number of functions in fact emulates the
scenarios in which the serverless applications are composed of more complex
pipelines with multiple functions [198, 199]. Using these complex pipelines
would incur more pronounced computation and communication overheads to the
end-to-end execution, both are addressed significantly by domain-
specialization and near-storage computation of _DSCS-Serverless_.

Figure 15: Sensitivity to PCIe bandwidth.

Sensitivity to PCIe bandwidth. Figure 15 illustrates the sensitivity of the
_DSCS-Serverless_ performance with respect to PCIe bandwidth used for P2P
communication (normalized to _DSCS-Serverless_ with PCIe x1). We measure the
communication time for all benchmarks with x4, x8, and x16 PCIe lanes using
the NS-FPGA, FPGA, and GPU (refer Table II), and use these numbers to
extrapolate the communication time for x1, x2, and x32 lanes. As the results
show, scaling the number of lanes has a negligible impact on performance. This
is due to the fact that PCIe provides enough bandwidth for P2P communication
as the size of the data is relatively small for serverless functions. As such,
in these cases the performance of _DSCS-Serverless_ is bounded by the latency
of the communication as opposed to the bandwidth.

Figure 16: Network and I/O Tail latency effect.

Tail latency effect. Accessing remote storage can incur long tail latency
[135, 18, 101]. To understand this variability and it’s implications on _DSCS-
Serverless_ performance, we perform a sweep across various latency
distribution for the PCIe, P2P and network. Figure 16 shows the implications
of tail latency normalized to baseline (CPU) with the same latency
distribution. Results suggest _DSCS-Serverless_ is robust to network and I/O
tail latency, since it in fact removes the data movement over them. On
average, _DSCS-Serverless_ provides 5.0$\times$ speedup for the 99th
percentile and 3.1$\times$ speedup for 50th percentile.

Figure 17: Cold start effect on functions.

Cold start. Figure 17 shows the speedup of _DSCS-Serverless_ over baseline.
Both _DSCS-Serverless_ and the baseline use cold containers where they pull
the container image (including the weights for the model) and load it to the
memory of the _DSA_. Since the models are large, the time to load a model
accounts for a significant portion of the end-to-end latency, thereby reducing
the speedup from 3.6x to 2.6x. However, as mentioned in Section V, cold
latency is incurred by both _DSCS-Serverless_ and the baseline systems.
Further, only the first invocation incurs a cold latency while all subsequent
invocations can potentially hide the cold latency using preemptive horizontal
scaling (Refer Section V).

##  VII Related Work

Individually, the emergence of serverless computing, the shift towards storage
disaggregation, and adoption of domain-specific accelerators has provided
significant benefits. However, collectively they poses interesting challenges.
The paper explores the confluence of the three trends and provides a pathway
towards utilizing accelerators for serverless computing on storage
disaggregated datacenters. Below, we discuss related work to these different
trends.

Serverless and storage. Serverless functions are stateless and ephemeral [11,
9, 200]. They use persistent storage to transfer intermediate data between
functions [103]. Pocket [18] proposed an storage system to allocate different
storage resources depending on workloads to reduce cost. Locus [133] focused
on deriving an optimal combination of slow remote and fast in-memory storage
while SONIC [132] used local and remote storage to pass data between
functions. NumPyWren [201] identified appropriate block size to remote storage
for serverless linear algebra. Jiffy [202] used in-memory caching on remote
servers to accommodate large variable sized intermediate data. However, it
still incurs the network latency to remote storage. These papers consider
multi-tier storage and the possibility of efficient data passing between
functions. They are orthogonal to this work since _DSCS-Serverless_ proposes a
model of serverless computing by leveraging near-storage _DSA_ to reduce the
data movement and unlock additional benefits from acceleration.

Acceleration of serverless functions. SmartNICs have been used to accelerate
serverless functions with low compute intensity [203]. Speedo [204] placed the
serverless function dispatcher on SmartNIC to avoid latency overhead. Dagger
[130] accelerated RPCs using FPGA-based NIC thereby reducing the communication
latency. BlastFunction [128] exposes FPGAs to serverless framework to
accelerate functions. Molecule [97] and Hardless [100] propose runtimes to
enable hardware accelerators for serverless computing. HiveMind [205] proposes
a hardware-software solution for serverless edge swarms. These solutions
either enable data-movement-aware acceleration on programmable NICs or
compute-focused acceleration on GPUs/FPGAs. _DSCS-Serverless_ enables both by
placing properly-sized _DSA_ near-storage.

Near-storage acceleration. Near-storage acceleration reduces data movement and
lowers communication latency. A plethora of work proposed near-data ASICs for
workloads (graph analytics, databases, sorting, etc.) demanding large amount
of data transfer. [174, 175, 206, 207, 208, 62, 209, 210, 211, 212, 213, 214,
215, 216, 86, 217]. INSIDER [218] proposes a runtime system that abstracts the
storage-side FPGA computation and provides transparent data movement between
CPU and FPGA. There are commercially available products such as Eideticom’s
NoLoad [219] for transparent compression, Samsung SmartSSD for utilities
(encryption, compression, etc.) [122, 121], and NGD system’s Newport for
encryption on ARM cores [119, 120]. Some existing computational storage [220,
221, 222] use ASIC hardware to perform inline compression and encryption, In
contrast, _DSCS-Serverless_ exclusively proposes domain-specific programmable
accelerators for serverless functions.

##  VIII Conclusion

Emergence of serverless computing coupled with disaggregation and hardware
specialization introduces unique challenges and opportunities. The paper
proposes a serverless computing model that integrates a domain-specific
accelerator next to the storage unit. The paper evaluates the model by
analyzing it for the domain of machine learning. Evaluation with a diverse set
of serverless applications against variety of compute platforms shows
significant gains in terms of performance, energy and cost efficiency.

## Acknowledgement

This work was in part supported by generous gifts from Google, Samsung,
Qualcomm, Microsoft, Xilinx as well as the National Science Foundation (NSF)
awards CCF#2107598, CNS#1822273, National Institute of Health (NIH) award
#R01EB028350, Defense Advanced Research Project Agency (DARPA) under agreement
number #HR0011-18-C-0020, and Semiconductor Research Corporation (SRC) award
#2021-AH-3039. The U.S. Government is authorized to reproduce and distribute
reprints for Governmental purposes not withstanding any copyright notation
thereon. The views and conclusions contained herein are those of the authors
and should not be interpreted as necessarily representing the official
policies or endorsements, either expressed or implied of Google, Qualcomm,
Microsoft, Xilinx, Samsung, NSF, SRC, NIH, DARPA or the U.S. Government. We
would like to extend our gratitude towards Berkin Akin, Andrey Ayupov, Daniel
Kaufman, and Stella Aslibekyan for reviewing the paper and providing feedback.
We also thank the extended team at Google Research, Brain Team who enabled
this collaboration and helped us with the paper.

## References

  * bmw [2020] BMW uses AWS.  https://aws.amazon.com/solutions/case-studies/bmw-group-case-study/, 2020\.
  * luf [2021] Lufthansa Technik: Keeping airlines flying optimally with AI-powered TechOps platform AVIATAR.  https://cloud.google.com/customers/lufthansa/, 2021.
  * coc [2020] Coca Cola uses AWS Serverless.  https://aws.amazon.com/solutions/case-studies/coca-cola-freestyle/, 2020\.
  * the [2021] Guardian News & Media Automates Subscription Fulfillment Using AWS Step Functions.  https://aws.amazon.com/solutions/case-studies/the-guardian/, 2021\.
  * vog [2021] PhotoVogue Case Study.  https://aws.amazon.com/solutions/case-studies/photovogue/, 2021\.
  * net [2021] Netflix & AWS Lambda Case Study.  https://aws.amazon.com/solutions/case-studies/netflix-and-aws-lambda, 2021\.
  * ibm [2021] Serverless in the enterprise: IBM Market Development and Insights.  https://www.ibm.com/downloads/cas/ZJLWQOAQ, 2021.
  * pwc [2020] PwC helps make compliance easier, automates regulatory obligation identification with Microsoft Azure Cognitive Search.  https://customers.microsoft.com/en-us/story/811347-pwc-partner-professional-services-azure, 2020\.
  * aws [2021a] AWS Lambda.  https://aws.amazon.com/lambda/, 2021a.
  * goo [2021] Google Cloud Functions.  https://cloud.google.com/functions/docs/concepts/overview, 2021\.
  * [11] Azure serverless.  Azure serverless.  https://azure.microsoft.com/en-us/solutions/serverless/#overview, 2021.
  * Klimovic et al. [2016] Ana Klimovic, Christos Kozyrakis, Eno Thereska, Binu John, and Sanjeev Kumar.  Flash Storage Disaggregation.  In _EuroSys_ , 2016.
  * Nanavati et al. [2017] Mihir Nanavati, Jake Wires, and Andrew Warfield.  Decibel: Isolation and Sharing in Disaggregated Rack-Scale Storage.  In _NSDI_ , 2017.
  * Legtchenko et al. [2017] Sergey Legtchenko, Hugh Williams, Kaveh Razavi, Austin Donnelly, Richard Black, Andrew Douglas, Nathanael Cheriere, Daniel Fryer, Kai Mast, Angela Demke Brown, Ana Klimovic, Andy Slowey, and Antony Rowstron.  Understanding Rack-Scale Disaggregated Storage.  In _HotStorage_ , 2017.
  * Guz et al. [2017] Zvika Guz, Harry (Huan) Li, Anahita Shayesteh, and Vijay Balakrishnan.  NVMe-over-Fabrics Performance Characterization and the Path to Low-Overhead Flash Disaggregation.  In _SYSTOR_ , 2017.
  * Klimovic et al. [2017] Ana Klimovic, Heiner Litz, and Christos Kozyrakis.  ReFlex: Remote Flash $\approx$ Local Flash.  In _ASPLOS_ , 2017.
  * Shan et al. [2018] Yizhou Shan, Yutong Huang, Yilun Chen, and Yiying Zhang.  LegoOS: A Disseminated, Distributed OS for Hardware Resource Disaggregation.  In _OSDI_ , 2018.
  * Klimovic et al. [2018] Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis.  Pocket: Elastic Ephemeral Storage for Serverless Analytics.  In _OSDI_ , 2018.
  * Jia et al. [2019] Yichen Jia, Eric Anger, and Feng Chen.  When NVMe over Fabrics Meets Arm: Performance and Implications.  In _MSST_ , 2019.
  * Hwang et al. [2020] Jaehyun Hwang, Qizhe Cai, Ao Tang, and Rachit Agarwal.  TCP$\approx$RDMA: CPU-efficient Remote Storage Access with i10.  In _OSDI_ , 2020.
  * Li et al. [2020] Huaicheng Li, Mingzhe Hao, Stanko Novakovic, Vaibhav Gogte, Sriram Govindan, Dan R. K. Ports, Irene Zhang, Ricardo Bianchini, Haryadi S. Gunawi, and Anirudh Badam.  LeapIO: Efficient and Portable Virtual NVMe Storage on ARM SoCs.  In _ASPLOS_ , 2020.
  * Xue et al. [2020] Shuai Xue, Shang Zhao, Quan Chen, Gang Deng, Zheng Liu, Jie Zhang, Zhuo Song, Tao Ma, Yong Yang, Yanbo Zhou, Keqiang Niu, Sijie Sun, and Minyi Guo.  Spool: Reliable Virtualized NVMe Storage Pool in Public Cloud Infrastructure.  In _ATC_ , 2020.
  * Min et al. [2021] Jaehong Min, Ming Liu, Tapan Chugh, Chenxingyu Zhao, Andrew Wei, In Hwan Doh, and Arvind Krishnamurthy.  Gimbal: Enabling Multi-Tenant Storage Disaggregation on SmartNIC JBOFs.  In _SIGCOMM_ , 2021.
  * Jouppi et al. [2017a] Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al.  In-Datacenter Performance Analysis of a Tensor Processing Unit.  In _ISCA_ , 2017a.
  * Fowers et al. [2018] Jeremy Fowers, Kalin Ovtcharov, Michael Papamichael, Todd Massengill, Ming Liu, Daniel Lo, Shlomi Alkalay, Michael Haselman, Logan Adams, Mahdi Ghandi, Stephen Heil, Prerak Patel, Adam Sapek, Gabriel Weisz, Lisa Woods, Sitaram Lanka, Steven K. Reinhardt, Adrian M. Caulfield, Eric S. Chung, and Doug Burger.  A Configurable Cloud-Scale DNN Processor for Real-Time AI.  In _ISCA_ , 2018.
  * aws [2022a] AWS Inferentia.  https://aws.amazon.com/machine-learning/inferentia/, 2022a.
  * nvi [2020a] Nvidia V100 Overview.  https://www.nvidia.com/en-us/data-center/v100/, 2020a.
  * Lim et al. [2009] Kevin Lim, Jichuan Chang, Trevor Mudge, Parthasarathy Ranganathan, Steven K. Reinhardt, and Thomas F. Wenisch.  Disaggregated Memory for Expansion and Sharing in Blade Servers.  In _ISCA_ , 2009.
  * Singhvi et al. [2020] Arjun Singhvi, Aditya Akella, Dan Gibson, Thomas F. Wenisch, Monica Wong-Chan, Sean Clark, Milo M. K. Martin, Moray McLaren, Prashant Chandra, Rob Cauble, Hassan M. G. Wassel, Behnam Montazeri, Simon L. Sabato, Joel Scherpelz, and Amin Vahdat.  1RMA: Re-Envisioning Remote Memory Access for Multi-Tenant Datacenters.  In _SIGCOMM_ , 2020.
  * Lee et al. [2022] Youngmoon Lee, Hasan Al Maruf, Mosharaf Chowdhury, Asaf Cidon, and Kang G. Shin.  Hydra: Resilient and Highly Available Remote Memory.  In _FAST_ , 2022.
  * Dennard et al. [1974] R. H. Dennard, F. H. Gaensslen, V. L. Rideout, E. Bassous, and A. R. LeBlanc.  Design of Ion-Implanted MOSFET’s with Very Small Physical Dimensions.  _JSSC_ , 1974.
  * Hardavellas et al. [2011] N. Hardavellas, M. Ferdman, B. Falsafi, and A. Ailamaki.  Toward dark silicon in servers.  _IEEE Micro_ , 2011.
  * Esmaeilzadeh et al. [2011a] Hadi Esmaeilzadeh, Emily Blem, Renee St. Amant, Karthikeyan Sankaralingam, and Doug Burger.  Dark silicon and the end of multicore scaling.  In _ISCA_ , 2011a.
  * Venkatesh et al. [2010] Ganesh Venkatesh, Jack Sampson, Nathan Goulding, Saturnino Garcia, Vladyslav Bryksin, Jose Lugo-Martinez, Steven Swanson, and Michael Bedford Taylor.  Conservation Cores: Reducing the Energy of Mature Computations.  In _ASPLOS_ , 2010.
  * Hennessy and Patterson [2019] John L Hennessy and David A Patterson.  A New Golden Age for Computer Architecture.  _CACM and Turing Lecture_ , 2019.
  * Chen et al. [2014] Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam.  DianNao: A Small-footprint High-throughput Accelerator for Ubiquitous Machine-learning.  In _ASPLOS_ , 2014.
  * Zhang et al. [2015] Chen Zhang, Peng Li, Guangyu Sun, Yijin Guan, Bingjun Xiao, and Jason Cong.  Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks.  In _FPGA_ , 2015.
  * Liu et al. [2015] Daofu Liu, Tianshi Chen, Shaoli Liu, Jinhong Zhou, Shengyuan Zhou, Olivier Teman, Xiaobing Feng, Xuehai Zhou, and Yunji Chen.  Pudiannao: A Polyvalent Machine Learning Accelerator.  In _ASPLOS_ , 2015.
  * Du et al. [2015] Zidong Du, Robert Fasthuber, Tianshi Chen, Paolo Ienne, Ling Li, Tao Luo, Xiaobing Feng, Yunji Chen, and Olivier Temam.  ShiDianNao: Shifting Vision Processing Closer to the Sensor.  In _ISCA_ , 2015.
  * Liu et al. [2016] Shaoli Liu, Zidong Du, Jinhua Tao, Dong Han, Tao Luo, Yuan Xie, Yunji Chen, and Tianshi Chen.  Cambricon: An Instruction Set Architecture for Neural Networks.  In _ISCA_ , 2016.
  * Zhang et al. [2016] Shijin Zhang, Zidong Du, Lei Zhang, Huiying Lan, Shaoli Liu, Ling Li, Qi Guo, Tianshi Chen, and Yunji Chen.  Cambricon-X: An Accelerator for Sparse Neural Networks.  In _MICRO_ , 2016.
  * Song et al. [2016] Lili Song, Ying Wang, Yinhe Han, Xin Zhao, Bosheng Liu, and Xiaowei Li.  C-brain: A Deep Learning Accelerator That Tames the Diversity of CNNs Through Adaptive Data-level Parallelization.  In _DAC_ , 2016.
  * Sharma et al. [2016] Hardik Sharma, Jongse Park, Divya Mahajan, Emmanuel Amaro, Joon Kim, Chenkai Shao, Asit Misra, and Hadi Esmaeilzadeh.  From High-Level Deep Neural Models to FPGAs.  In _MICRO_ , 2016.
  * Alwani et al. [2016] Manoj Alwani, Han Chen, Michael Ferdman, and Peter Milder.  Fused-layer cnn accelerators.  In _MICRO_ , 2016.
  * Mahajan et al. [2016] Divya Mahajan, Jongse Park, Emmanuel Amaro, Hardik Sharma, Amir Yazdanbakhsh, Joon Kyung Kim, and Hadi Esmaeilzadeh.  TABLA: A Unified Template-based Framework for Accelerating Statistical Machine Learning.  In _HPCA_ , 2016.
  * Shen et al. [2017a] Yongming Shen, Michael Ferdman, and Peter Milder.  Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer.  In _FCCM_ , 2017a.
  * Song et al. [2017] Linghao Song, Xuehai Qian, Hai Li, and Yiran Chen.  PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning.  In _HPCA_ , 2017.
  * Parashar et al. [2017] Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, Antonio Puglielli, Rangharajan Venkatesan, Brucek Khailany, Joel Emer, Stephen W Keckler, and William J Dally.  SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks.  In _ISCA_ , 2017.
  * Shen et al. [2017b] Yongming Shen, Michael Ferdman, and Peter Milder.  Maximizing CNN Accelerator Efficiency Through Resource Partitioning.  In _ISCA_ , 2017b.
  * Kwon et al. [2018] Hyoukjun Kwon, Ananda Samajdar, and Tushar Krishna.  Maeri: Enabling flexible dataflow mapping over dnn accelerators via reconfigurable interconnects.  _ASPLOS_ , 2018.
  * Lee et al. [2018] Jinmook Lee, Changhyeon Kim, Sanghoon Kang, Dongjoo Shin, Sangyeob Kim, and Hoi-Jun Yoo.  Unpu: A 50.6 tops/w unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision.  In _ISSCC_ , 2018.
  * Chen et al. [2019] Yu-Hsin Chen, Tien-Ju Yang, Joel Emer, and Vivienne Sze.  Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices.  _JETCAS_ , 2019.
  * Shao et al. [2019] Yakun Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matthew Fojtik, Nan Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, et al.  Simba: Scaling deep-learning inference with multi-chip-module-based architecture.  In _MICRO_ , 2019.
  * Gao et al. [2019] Mingyu Gao, Xuan Yang, Jing Pu, Mark Horowitz, and Christos Kozyrakis.  Tangram: Optimized coarse-grained dataflow for scalable nn accelerators.  In _ASPLOS_ , 2019.
  * Geng et al. [2020] Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, and Martin C. Herbordt.  AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing.  In _MICRO_ , 2020.
  * Yan et al. [2020] Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie.  HyGCN: A GCN Accelerator with Hybrid Architecture.  In _HPCA_ , 2020.
  * Ghodrati et al. [2020] Soroush Ghodrati, Byung Hoon Ahn, Joon Kyung Kim, Sean Kinzer, Brahmendra Reddy Yatham, Navateja Alla, Hardik Sharma, Mohammad Alian, Eiman Ebrahimi, Nam Sung Kim, et al.  Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks.  In _MICRO_ , 2020.
  * Qin et al. [2020] Eric Qin, Ananda Samajdar, Hyoukjun Kwon, Vineet Nadella, Sudarshan Srinivasan, Dipankar Das, Bharat Kaul, and Tushar Krishna.  SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training.  _HPCA_ , 2020.
  * Liang et al. [2021] Shengwen Liang, Ying Wang, Cheng Liu, Lei He, Huawei LI, Dawen Xu, and Xiaowei Li.  EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks.  _IEEE Transactions on Computers_ , 70(9), 2021.
  * Li et al. [2021] Jiajun Li, Ahmed Louri, Avinash Karanth, and Razvan Bunescu.  GCNAX: A Flexible and Energy-efficient Accelerator for Graph Convolutional Neural Networks.  In _HPCA_ , 2021.
  * Ham et al. [2016] Tae Jun Ham, Lisa Wu, Narayanan Sundaram, Nadathur Satish, and Margaret Martonosi.  Graphicionado: A high-performance and energy-efficient accelerator for graph analytics.  In _MICRO_ , 2016.
  * Lee et al. [2017] Jinho Lee, Heesu Kim, Sungjoo Yoo, Kiyoung Choi, H. Peter Hofstee, Gi-Joon Nam, Mark R. Nutter, and Damir Jamsek.  ExtraV: Boosting Graph Processing near Storage with a Coherent Accelerator.  _Proc. VLDB Endow._ , 10(12), 2017.
  * Yao et al. [2018] Pengcheng Yao, Long Zheng, Xiaofei Liao, Hai Jin, and Bingsheng He.  An Efficient Graph Accelerator with Parallel Data Conflict Management.  In _PACT_ , 2018.
  * Mukkara et al. [2018] Anurag Mukkara, Nathan Beckmann, Maleen Abeydeera, Xiaosong Ma, and Daniel Sanchez.  Exploiting Locality in Graph Analytics through Hardware-Accelerated Traversal Scheduling.  In _MICRO_ , 2018.
  * Zhang et al. [2018a] Mingxing Zhang, Youwei Zhuo, Chao Wang, Mingyu Gao, Yongwei Wu, Kang Chen, Christos Kozyrakis, and Xuehai Qian.  GraphP: Reducing Communication for PIM-Based Graph Processing with Efficient Data Partition.  In _HPCA_ , 2018a.
  * Song et al. [2018] Linghao Song, Youwei Zhuo, Xuehai Qian, Hai Li, and Yiran Chen.  GraphR: Accelerating Graph Processing Using ReRAM.  In _HPCA_ , 2018.
  * Zhang et al. [2018b] Dan Zhang, Xiaoyu Ma, Michael Thomson, and Derek Chiou.  Minnow: Lightweight Offload Engines for Worklist Management and Worklist-Directed Prefetching.  In _ASPLOS_ , 2018b.
  * Boroumand et al. [2019] Amirali Boroumand, Saugata Ghose, Minesh Patel, Hasan Hassan, Brandon Lucia, Rachata Ausavarungnirun, Kevin Hsieh, Nastaran Hajinazar, Krishna T. Malladi, Hongzhong Zheng, and Onur Mutlu.  CoNDA: Efficient Cache Coherence Support for near-Data Accelerators.  In _ISCA_ , 2019.
  * Mukkara et al. [2019] Anurag Mukkara, Nathan Beckmann, and Daniel Sanchez.  PHI: Architectural Support for Synchronization- and Bandwidth-Efficient Commutative Scatter Updates.  In _MICRO_ , 2019.
  * Rahman et al. [2020] Shafiur Rahman, Nael Abu-Ghazaleh, and Rajiv Gupta.  GraphPulse: An Event-Driven Hardware Accelerator for Asynchronous Graph Processing.  In _MICRO_ , 2020.
  * Zhang et al. [2021] Yu Zhang, Xiaofei Liao, Hai Jin, Ligang He, Bingsheng He, Haikun Liu, and Lin Gu.  DepGraph: A Dependency-Driven Accelerator for Efficient Iterative Graph Processing.  In _HPCA_ , 2021.
  * Rahman et al. [2021] Shafiur Rahman, Mahbod Afarin, Nael Abu-Ghazaleh, and Rajiv Gupta.  JetStream: Graph Analytics on Streaming Data with Event-Driven Hardware Accelerator.  In _MICRO_ , 2021.
  * Zhao et al. [2021] Jin Zhao, Yu Zhang, Xiaofei Liao, Ligang He, Bingsheng He, Hai Jin, and Haikun Liu.  LCCG: A Locality-Centric Hardware Accelerator for High Throughput of Concurrent Graph Processing.  In _SC_ , 2021.
  * Turakhia et al. [2018] Yatish Turakhia, Gill Bejerano, and William J. Dally.  Darwin: A Genomics Co-Processor Provides up to 15,000X Acceleration on Long Read Assembly.  In _ASPLOS_ , 2018.
  * Fujiki et al. [2018] Daichi Fujiki, Arun Subramaniyan, Tianjun Zhang, Yu Zeng, Reetuparna Das, David Blaauw, and Satish Narayanasamy.  GenAx: A Genome Sequencing Accelerator.  In _ISCA_ , 2018.
  * Cong et al. [2018] Jason Cong, Licheng Guo, Po-Tsang Huang, Peng Wei, and Tianhe Yu.  SMEM++: A Pipelined and Time-Multiplexed SMEM Seeding Accelerator for Genome Sequencing.  In _FPL_ , 2018.
  * Banerjee et al. [2019] Subho Sankar Banerjee, Mohamed El-Hadedy, Jong Bin Lim, Zbigniew T. Kalbarczyk, Deming Chen, Steven S. Lumetta, and Ravishankar K. Iyer.  ASAP: Accelerated Short-Read Alignment on Programmable Hardware.  _IEEE Transactions on Computers_ , 2019.
  * Nag et al. [2019] Anirban Nag, C. N. Ramachandra, Rajeev Balasubramonian, Ryan Stutsman, Edouard Giacomin, Hari Kambalasubramanyam, and Pierre-Emmanuel Gaillardon.  GenCache: Leveraging In-Cache Operators for Efficient Sequence Alignment.  In _MICRO_ , 2019.
  * Huangfu et al. [2019] Wenqin Huangfu, Xueqi Li, Shuangchen Li, Xing Hu, Peng Gu, and Yuan Xie.  MEDAL: Scalable DIMM Based Near Data Processing Accelerator for DNA Seeding Algorithm.  In _MICRO_ , 2019.
  * Cali et al. [2020] Damla Senol Cali, Gurpreet S. Kalsi, Zülal Bingöl, Can Firtina, Lavanya Subramanian, Jeremie S. Kim, Rachata Ausavarungnirun, Mohammed Alser, Juan Gomez-Luna, Amirali Boroumand, Anant Norion, Allison Scibisz, Sreenivas Subramoneyon, Can Alkan, Saugata Ghose, and Onur Mutlu.  GenASM: A High-Performance, Low-Power Approximate String Matching Acceleration Framework for Genome Sequence Analysis.  In _MICRO_ , 2020.
  * Kim et al. [2020] Yeseong Kim, Mohsen Imani, Niema Moshiri, and Tajana Rosing.  GenieHD: Efficient DNA Pattern Matching Accelerator Using Hyperdimensional Computing.  In _DATE_ , 2020.
  * Huangfu et al. [2020] Wenqin Huangfu, Krishna T. Malladi, Shuangchen Li, Peng Gu, and Yuan Xie.  NEST: DIMM based Near-Data-Processing Accelerator for K-mer Counting.  In _ICCAD_ , 2020.
  * Laguna et al. [2020] Ann Franchesca Laguna, Hasindu Gamaarachchi, Xunzhao Yin, Michael Niemier, Sri Parameswaran, and X. Sharon Hu.  Seed-and-Vote Based in-Memory Accelerator for DNA Read Mapping.  In _ICCAD_ , 2020.
  * Fujiki et al. [2020] Daichi Fujiki, Shunhao Wu, Nathan Ozog, Kush Goliya, David Blaauw, Satish Narayanasamy, and Reetuparna Das.  SeedEx: A Genome Sequencing Accelerator for Optimal Alignments in Subminimal Space.  In _MICRO_ , 2020.
  * Haghi et al. [2021] Abbas Haghi, Santiago Marco-Sola, Lluc Alvarez, Dionysios Diamantopoulos, Christoph Hagleitner, and Miquel Moreto.  An FPGA Accelerator of the Wavefront Algorithm for Genomics Pairwise Alignment.  In _FPL_ , 2021.
  * Mansouri Ghiasi et al. [2022] Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu.  GenStore: A High-Performance in-Storage Processing System for Genome Sequence Analysis.  In _ASPLOS_ , 2022.
  * Cali et al. [2022] Damla Senol Cali, Konstantinos Kanellopoulos, Joël Lindegger, Zülal Bingöl, Gurpreet S. Kalsi, Ziyi Zuo, Can Firtina, Meryem Banu Cavlak, Jeremie Kim, Nika Mansouri Ghiasi, Gagandeep Singh, Juan Gómez-Luna, Nour Almadhoun Alserr, Mohammed Alser, Sreenivas Subramoney, Can Alkan, Saugata Ghose, and Onur Mutlu.  SeGraM: A Universal Hardware Accelerator for Genomic Sequence-to-Graph and Sequence-to-Sequence Mapping.  In _ISCA_ , 2022.
  * Murray et al. [2016] Sean Murray, William Floyd-Jones, Ying Qi, George Konidaris, and Daniel J. Sorin.  The Microarchitecture of a Real-time Robot Motion Planning Accelerator.  In _MICRO_ , 2016.
  * Sacks et al. [2018] Jacob Sacks, Divya Mahajan, Richard C Lawson, and Hadi Esmaeilzadeh.  Robox: An End-to-End Solution to Accelerate Autonomous Control in Robotics.  In _ISCA_ , 2018.
  * Lin et al. [2021] Yujun Lin, Zhekai Zhang, Haotian Tang, Hanrui Wang, and Song Han.  PointAcc: Efficient Point Cloud Accelerator.  In _MICRO_ , 2021.
  * Neuman et al. [2021] Sabrina M. Neuman, Brian Plancher, Thomas Bourgeat, Thierry Tambe, Srinivas Devadas, and Vijay Janapa Reddi.  Robomorphic Computing: A Design Methodology for Domain-Specific Accelerators Parameterized by Robot Morphology.  In _ASPLOS 2021_ , 2021.
  * Karandikar et al. [2021] Sagar Karandikar, Chris Leary, Chris Kennelly, Jerry Zhao, Dinesh Parimi, Borivoje Nikolic, Krste Asanovic, and Parthasarathy Ranganathan.  A Hardware Accelerator for Protocol Buffers.  In _MICRO_ , 2021.
  * Anderson et al. [2021] Michael Anderson, Benny Chen, Stephen Chen, Summer Deng, Jordan Fix, Michael Gschwind, Aravind Kalaiah, Changkyu Kim, Jaewon Lee, Jason Liang, et al.  First-generation Inference Accelerator Deployment at Facebook.  _arXiv preprint_ , 2021.
  * met [2019] Mount Shasta for Video Transcoding.  https://engineering.fb.com/2019/03/14/data-center-engineering/accelerating-infrastructure/, 2019\.
  * Putnam et al. [2014] Andrew Putnam, Adrian Caulfield, Eric Chung, Derek Chiou, Kypros Constantinides, John Demme, Hadi Esmaeilzadeh, Jeremy Fowers, Gopi Prashanth, Jan Gray, Michael Haselman, Scott Hauck, Stephen Heil, Amir Hormati, Joo-Young Kim, Sitaram Lanka, James R. Larus, Eric Peterson, Aaron Smith, Jason Thong, Phillip Yi Xiao, and Doug Burger.  A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services.  In _ISCA_ , 2014.
  * Chung et al. [2017] Eric Chung, Jeremy Fowers, Kalin Ovtcharov, Michael Papamichael, Adrian Caulfield, Todd Massengil, Ming Liu, Daniel Lo, Shlomi Alkalay, Michael Haselman, Christian Boehn, Oren Firestein, Alessandro Forin, Kang Su Gatlin, Mahdi Ghandi, Stephen Heil, Kyle Holohan, Tamas Juhasz, Ratna Kumar Kovvuri, Sitaram Lanka, Friedel van Megen, Dima Mukhortov, Prerak Patel, Steve Reinhardt, Adam Sapek, Raja Seera, Balaji Sridharan, Lisa Woods, Phillip Yi-Xiao, Ritchie Zhao, and Doug Burger.  Accelerating persistent neural networks at datacenter scale.  In _HotChips_ , 2017.
  * Du et al. [2022] Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen.  Serverless Computing on Heterogeneous Computers.  In _ASPLOS_ , 2022.
  * Roy et al. [2022] Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari.  IceBreaker: Warming Serverless Functions Better with Heterogeneity.  In _ASPLOS_ , 2022.
  * Fingler et al. [2022] Henrique Fingler, Zhiting Zhu, Esther Yoon, Zhipeng Jia, Emmett Witchel, and Christopher J. Rossbach.  DGSF: Disaggregated GPUs for Serverless Functions.  In _IPDPS_ , 2022.
  * Werner and Schirmer [2022] Sebastian Werner and Trever Schirmer.  Hardless: A Generalized Serverless Compute Architecture for Hardware Processing Accelerators, 2022.  URL https://arxiv.org/abs/2208.03192.
  * Roy et al. [2021] Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari.  Characterizing and Mitigating the I/O Scalability Challenges for Serverless Applications.  In _IISWC_ , 2021.
  * Mahgoub et al. [2022a] Ashraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Eshaan Minocha, Sameh Elnikety, Saurabh Bagchi, and Somali Chaterji.  WISEFUSE: Workload Characterization and DAG Transformation for Serverless Workflows.  In _SIGMETRICS_ , 2022a.
  * Mahgoub et al. [2022b] Ashraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Sameh Elnikety, Somali Chaterji, and Saurabh Bagchi.  ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGs.  In _OSDI_ , July 2022b.
  * Barbalace and Do [2021] Antonio Barbalace and Jaeyoung Do.  Computational Storage: Where Are We Today?  In _CIDR_ , 2021.
  * ope [2016] OpenFaaS: Serverless Functions, Made Simple.  https://www.openfaas.com, 2016.
  * kub [2014] Kubernetes.  https://kubernetes.io, 2014.
  * aws [2021b] Create a Serverless Pipeline to Translate Large Documents with Amazon Translate.  https://aws.amazon.com/blogs/machine-learning/create-a-serverless-pipeline-to-translate-large-documents-with-amazon-translate/, 2021b.
  * nlp [2021a] AWS Translate.  https://aws.amazon.com/translate/, 2021a.
  * con [2021a] AWS Content Moderation using Serverless.  https://docs.aws.amazon.com/rekognition/latest/dg/moderation.html?pg=ln&sec=ft, 2021a.
  * con [2021b] Content Moderation in Social Media.  https://www.mygreatlearning.com/blog/content-moderation-in-social-media-with-aws-services/, 2021b.
  * cli [2019] Using DNN to classify acute myeloidlymphoblastic.  https://www.intel.com/content/www/us/en/developer/articles/technical/inception-v3-deep-convolutional-architecture-for-classifying-acute-myeloidlymphoblastic.html, 2019\.
  * cli [2021] Genomics using Serverless.  https://aws.amazon.com/blogs/publicsector/genomic-and-medical-big-data-go-serverless/, 2021\.
  * aws [2021c] PPE Detection using Amazon Rekognition.  https://aws.amazon.com/blogs/machine-learning/automatically-detecting-personal-protective-equipment-on-persons-in-images-using-amazon-rekognition/, 2021c.
  * cre [2021a] AWS.  https://aws.amazon.com/blogs/industries/transforming-credit-risk-analysis-with-aws-fargate-and-aws-step-functions-at-societe-generale/, 2021a.
  * nlp [2021b] AWS Serverless-bot-framework.  https://aws.amazon.com/solutions/implementations/serverless-bot-framework/, 2021b.
  * nlp [2021c] Translate Large Documents using Serverless Pipeline with Amazon Translate.  https://aws.amazon.com/blogs/machine-learning/create-a-serverless-pipeline-to-translate-large-documents-with-amazon-translate/, 2021c.
  * ass [2020] Spot product Defects using Computer Vision to Automate Quality Inspection.  https://aws.amazon.com/lookout-for-vision/, 2020.
  * Bazi et al. [2021] Yakoub Bazi, Laila Bashmal, Mohamad M Al Rahhal, Reham Al Dayil, and Naif Al Ajlan.  Vision Transformers for Remote Sensing Image Classification.  _Remote Sensing_ , 13(3):516, 2021.
  * ngd [2021] NGD system NVMe Computational Storage.  https://ngdsystems.com/nvme-computational-storage-a-compelling-solution-for-bringing-intelligence-to-the-edge/, 2021\.
  * Gorji et al. [2021] Ali Heydari Gorji, Mahdi Torabzadehkashi, Siavash Rezaei, Hossein Bobarshad, Vladimir Castro Alves, and Pai H. Chou.  In-storage Processing of I/O Intensive Applications on Computational Storage Drives.  _CoRR_ , abs/2112.12415, 2021.  URL https://arxiv.org/abs/2112.12415.
  * Lee et al. [2020] Joo Hwan Lee, Hui Zhang, Veronica Lagrange, Praveen Krishnamoorthy, Xiaodong Zhao, and Yang Seok Ki.  SmartSSD: FPGA Accelerated Near-Storage Data Analytics on SSD.  _IEEE Computer Architecture Letters_ , 19(2), 2020.
  * sma [2022] Innovation With SmartSSD for Green Computing.  https://www.snia.org/sites/default/files/PM-Summit/2022/PMCS22-Ki-Innovation-With-Smart-SSD-for-Green-Computing.pdf, 2022\.
  * Ao et al. [2018] Lixiang Ao, Liz Izhikevich, Geoffrey M. Voelker, and George Porter.  Sprocket: A Serverless Video Processing Framework.  In _SoCC_ , 2018.
  * ope [2021] Apache OpenWhisk.  https://openwhisk.apache.org/, 2021.
  * Inc. [2015] Google Inc.  Protocol Buffers.  https://developers.google.com/protocol-buffers/, 2015.
  * Nvidia [2007] Nvidia.  Compute unified device architecture (CUDA) programming guide, 2007.
  * Sriraman and Dhanotia [2020] Akshitha Sriraman and Abhishek Dhanotia.  Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale.  In _ASPLOS_ , 2020.
  * Damiani et al. [2022] Andrea Damiani, Giorgia Fiscaletti, Marco Bacis, Rolando Brondolin, and Marco D. Santambrogio.  BlastFunction: A Full-Stack Framework Bringing FPGA Hardware Acceleration to Cloud-Native Applications.  _ACM Trans. Reconfigurable Technol. Syst._ , 2022.
  * Schall et al. [2022] David Schall, Artemiy Margaritov, Dmitrii Ustiugov, Andreas Sandberg, and Boris Grot.  Lukewarm Serverless Functions: Characterization and Optimization.  In _ISCA_ , 2022.
  * Lazarev et al. [2021] Nikita Lazarev, Shaojie Xiang, Neil Adit, Zhiru Zhang, and Christina Delimitrou.  Dagger: Efficient and Fast RPCs in Cloud Microservices with near-Memory Reconfigurable NICs.  In _ASPLOS_ , 2021.
  * McClure et al. [2022] Sarah McClure, Amy Ousterhout, Scott Shenker, and Sylvia Ratnasamy.  Efficient Scheduling Policies for Microsecond-Scale Tasks.  In _NSDI_ , 2022.
  * Mahgoub et al. [2021] Ashraf Mahgoub, Karthick Shankar, Subrata Mitra, Ana Klimovic, Somali Chaterji, and Saurabh Bagchi.  SONIC: Application-aware Data Passing for Chained Serverless Applications.  In _ATC_ , 2021.
  * Pu et al. [2019] Qifan Pu, Shivaram Venkataraman, and Ion Stoica.  Shuffling, Fast and Slow: Scalable Analytics on Serverless Infrastructure.  In _NSDI_ , 2019.
  * Ustiugov et al. [2021] Dmitrii Ustiugov, Theodor Amariucai, and Boris Grot.  Analyzing Tail Latency in Serverless Clouds with STeLLAR.  In _IISWC_ , 2021.
  * Scheuner et al. [2022] Joel Scheuner, Simon Eismann, Sacheendra Talluri, Erwin Van Eyk, Cristina L. Abad, Philipp Leitner, and Alexandru Iosup.  Let’s Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications.  _ArXiv_ , abs/2205.07696, 2022.
  * aws [2019] Amazon CloudWatch Percentiles on Amazon S3.  https://aws.amazon.com/blogs/storage/amazon-s3-cloudwatch-percentiles/, 2019\.
  * Hou et al. [2022] Kaiyu Hou, Sen Lin, Yan Chen, and Vinod Yegneswaran.  QFaaS: Accelerating and Securing Serverless Cloud Networks with QUIC.  In _SoCC_ , 2022.
  * Amdahl [1967] Gene M. Amdahl.  Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities.  In _Proceedings of the Spring Joint Computer Conference_ , 1967.
  * Gummadi et al. [2003] Krishna P. Gummadi, Richard J. Dunn, Stefan Saroiu, Steven D. Gribble, Henry M. Levy, and John Zahorjan.  Measurement, Modeling, and Analysis of a Peer-to-Peer File-Sharing Workload.  In _ACM Symposium on Operating Systems Principles (SOSP)_ , 2003\.
  * pci [2019] Device-to-device memory-transfer offload with P2PDMA.  https://lwn.net/Articles/767281/, 2019.
  * ama [2021a] Cloud Object Storage |Store & Retrieve Data Anywhere |Amazon Simple Storage Service (S3).  https://aws.amazon.com/s3/, 2021a.
  * azu [2021] Azure Cloud Storage Solutions and Services | Microsoft azure.  https://azure.microsoft.com/en-us/product-categories/storage/, 2021.
  * McKusick et al. [1984] Marshall K. McKusick, William N. Joy, Samuel J. Leffler, and Robert S. Fabry.  A Fast File System for UNIX.  _ACM Trans. Comput. Syst._ , 2(3):181–197, aug 1984.  ISSN 0734-2071.  doi: 10.1145/989.990.  URL https://doi.org/10.1145/989.990.
  * xil [2021] Xilinx SmartSSD Computational Storage Drive Product Brief.  https://www.xilinx.com/content/dam/xilinx/publications/product-briefs/xilinx-smartssd-computational-storage-drive-product-brief.pdf, 2021\.
  * Shafiei et al. [2022] Hossein Shafiei, Ahmad Khonsari, and Payam Mousavi.  Serverless Computing: A Survey of Opportunities, Challenges, and Applications.  _ACM Comput. Surv._ , 54(11s), nov 2022.  ISSN 0360-0300.  doi: 10.1145/3510611.  URL https://doi.org/10.1145/3510611.
  * Seshadri et al. [2022] Kiran Seshadri, Berkin Akin, James Laudon, Ravi Narayanaswami, and Amir Yazdanbakhsh.  An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks.  In _IISWC_ , 2022.
  * Genc et al. [2019] Hasan Genc, Ameer Haj-Ali, Vighnesh Iyer, Alon Amid, Howard Mao, John Wright, Colin Schmidt, Jerry Zhao, Albert Ou, Max Banister, et al.  Gemmini: An Agile Systolic Array Generator Enabling Systematic Evaluations of Deep-Learning Architectures.  _arXiv_ , 2019.
  * Chen et al. [2016a] Yu-Hsin Chen, Joel Emer, and Vivienne Sze.  Eyeriss: A Spatial Architecture for Energy-efficient Dataflow for Convolutional Neural Networks.  In _ISCA_ , 2016a.
  * Jouppi et al. [2017b] Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al.  In-Datacenter Performance Analysis of a Tensor Processing Unit.  In _ISCA_ , 2017b.
  * Chen et al. [2016b] Yu-Hsin Chen, Tushar Krishna, Joel S Emer, and Vivienne Sze.  Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks.  _JSSC_ , 2016b.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint_ , 2018.
  * Cohen and Gokaslan [2020] Vanya Cohen and Aaron Gokaslan.  OpenGPT-2: Open Language Models and Implications of Generated Text.  _XRDS_ , 27(1), 2020.
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.  In _ICLR_ , 2021.
  * Kim et al. [2017] Namhyung Kim, Jeongseob Ahn, Sungpack Hong, Hassan Chafi, and Kiyoung Choi.  How Much Computation Power do you need for Near-Data Processing in Cloud?  In _ASBD_ , 2017.
  * HeydariGorji et al. [2020] Ali HeydariGorji, Mahdi Torabzadehkashi, Siavash Rezaei, Hossein Bobarshad, Vladimir Alves, and Pai H. Chou.  Stannis: Low-Power Acceleration of DNN Training Using Computational Storage Devices.  In _DAC_ , 2020.
  * Esmaeilzadeh et al. [2011b] Hadi Esmaeilzadeh, Emily Blem, Renée St. Amant, Karthikeyan Sankaralingam, and Doug Burger.  Dark Silicon and the End of Multicore Scaling.  In _ISCA_ , 2011b.
  * Hameed et al. [2010] Rehan Hameed, Wajahat Qadeer, Megan Wachs, Omid Azizi, Alex Solomatnikov, Benjamin C Lee, Stephen Richardson, Christos Kozyrakis, and Mark Horowitz.  Understanding sources of inefficiency in general-purpose chips.  In _ISCA_ , 2010.
  * Magaki et al. [2016] Ikuo Magaki, Moein Khazraee, Luis Vega Gutierrez, and Michael Bedford Taylor.  ASIC Clouds: Specializing the Datacenter.  In _ISCA_ , 2016.
  * Liu et al. [2019] Ming Liu, Simon Peter, Arvind Krishnamurthy, and Phitchaya Mangpo Phothilimthana.  E3: Energy-Efficient Microservices on SmartNIC-Accelerated Servers.  In _ATC_ , 2019.
  * Barroso et al. [2018] Luiz André Barroso, Urs Hölzle, and Parthasarathy Ranganathan.  The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition.  _Synthesis Lectures on Computer Architecture_ , 13(3):i–189, 2018.  doi: 10.2200/S00874ED3V01Y201809CAC046.  URL https://doi.org/10.2200/S00874ED3V01Y201809CAC046.
  * Stine et al. [2007] James E. Stine, Ivan Castellanos, Michael Wood, Jeff Henson, Fred Love, W. Rhett Davis, Paul D. Franzon, Michael Bucher, Sunil Basavarajaiah, Julie Oh, and Ravi Jenkal.  FreePDK: An Open-Source Variation-Aware Design Kit.  In _MSE_ , pages 173–174, 2007.  doi: 10.1109/MSE.2007.44.
  * Sarangi and Baas [2021] Satyabrata Sarangi and Bevan Baas.  DeepScaleTool: A Tool for the Accurate Estimation of Technology Scaling in the Deep-Submicron Era.  In _ISCAS_ , 2021.
  * alv [2020a] Alveo U280 Data Center Accelerator Card.  https://www.xilinx.com/products/boards-and-kits/alveo/u280.html, 2020a.
  * alv [2020b] Alveo U250 Data Center Accelerator Card.  https://www.xilinx.com/products/boards-and-kits/alveo/u250.html, 2020b.
  * pro [2020] Prometheus: From Metrics to Insight.  https://prometheus.io/, 2020.
  * Garfinkel [2007] Simson L. Garfinkel.  An Evaluation of Amazon’s Grid Computing Services: EC2, S3, and SQS.  _Center For. Citeseer_ , 2007.
  * aws [2021d] AWS Lambda Quotas.  https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html, 2021d.
  * Suda et al. [2016] Naveen Suda, Vikas Chandra, Ganesh Dasika, Abinash Mohanty, Yufei Ma, Sarma Vrudhula, Jae-sun Seo, and Yu Cao.  Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks.  In _FPGA_ , 2016.
  * Paszke et al. [2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic Differentiation in PyTorch.  In _NIPS-W_ , 2017.
  * Microsoft [2017] Facebook Research Microsoft.  ONNX: An Open Format to Represent Deep Learning Models.  http://onnx.ai/, 2017.
  * Lin and Liu [2016] Felix Xiaozhu Lin and Xu Liu.  Memif: Towards Programming Heterogeneous Memory Asynchronously.  In _ASPLOS_ , 2016.
  * Li et al. [2016] Bojie Li, Kun Tan, Layong (Larry) Luo, Yanqing Peng, Renqian Luo, Ningyi Xu, Yongqiang Xiong, Peng Cheng, and Enhong Chen.  ClickNP: Highly Flexible and High Performance Network Processing with Reconfigurable Hardware.  In _SIGCOMM_ , 2016.
  * Neugebauer et al. [2018] Rolf Neugebauer, Gianni Antichi, José Fernando Zazo, Yury Audzevich, Sergio López-Buedo, and Andrew W. Moore.  Understanding PCIe Performance for End Host Networking.  In _SIGCOMM_ , 2018.
  * Salamat et al. [2021] Sahand Salamat, Armin Haj Aboutalebi, Behnam Khaleghi, Joo Hwan Lee, Yang Seok Ki, and Tajana Rosing.  NASCENT: Near-Storage Acceleration of Database Sort on SmartSSD.  In _FPGA_ , 2021.
  * Salamat et al. [2022] Sahand Salamat, Hui Zhang, Yang Seok Ki, and Tajana Rosing.  NASCENT2: Generic Near-Storage Sort Accelerator for Data Analytics on SmartSSD.  _ACM Trans. Reconfigurable Technol. Syst._ , 15(2), 2022\.
  * cre [2021b] IBM.  https://www.ibm.com/docs/en/spss-statistics/saas?topic=regression-using-binary-logistic-assess-credit-risk, 2021b.
  * He et al. [2016a] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In _CVPR_ , 2016a.
  * Redmon and Farhadi [2018] Joseph Redmon and Ali Farhadi.  Yolov3: An Incremental Improvement.  _arXiv_ , 2018.
  * Long et al. [2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully Convolutional Networks for Semantic Segmentation.  In _CVPR_ , June 2015.
  * Tan and Le [2019] Mingxing Tan and Quoc V. Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In _arXiv_ , 2019.  doi: 10.48550/ARXIV.1905.11946.  URL https://arxiv.org/abs/1905.11946.
  * Wu et al. [2020] Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, and Peter Vajda.  Visual Transformers: Token-based Image Representation and Processing for Computer Vision, 2020.
  * int [2019] Intel Platanium Xeon Scalable Processors.  https://www.intel.com/content/www/us/en/products/details/processors/xeon/scalable/platinum/products.html, 2019\.
  * nvi [2019] NVIDIA-Turing-Architecture-Whitepaper.  https://images.nvidia.com/aem-dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf, 2019\.
  * tx2 [2019] Nvidia Jetson TX2 Module.  https://developer.nvidia.com/embedded/jetson-tx2, 2019.
  * nvi [2018] Nvidia Pascal Architecture.  https://www.nvidia.com/en-us/data-center/pascal-gpu-architecture/, 2018\.
  * Reddi et al. [2019] Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, et al.  MLPerf Inference Benchmark.  _arxiv_ , 2019.
  * hug [2022] Hugging Face.  https://huggingface.co/docs/transformers/model_doc/vit/, 2022.
  * He et al. [2016b] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In _CVPR_ , 2016b.
  * hey [2021] hey http Load Generator.  https://github.com/rakyll/hey, 2021.
  * ama [2021b] Amazon EC2 F1 Instances.  https://aws.amazon.com/ec2/instance-types/f1/, 2021b.
  * xil [2020a] Xilinx Vitis.  https://docs.xilinx.com/v/u/2020.2-English/ug1416-vitis-documentation, 2020a.
  * xil [2020b] Xilinx Runtime Library.  https://www.xilinx.com/products/design-tools/vitis/xrt.html, 2020b.
  * Rotem et al. [2012] Efraim Rotem, Alon Naveh, Avinash Ananthakrishnan, Eliezer Weissmann, and Doron Rajwan.  Power-Management Architecture of the Intel Microarchitecture Code-Named Sandy Bridge.  _IEEE Micro_ , 32(2), 2012.
  * Khan et al. [2018] Kashif Nizam Khan, Mikael Hirki, Tapio Niemi, Jukka K. Nurminen, and Zhonghong Ou.  RAPL in Action: Experiences in Using RAPL for Power Measurements.  _ACM Trans. Model. Perform. Eval. Comput. Syst._ , 3(2), 2018.
  * nvi [2020b] Nvidia Jetson TX2.  https://developer.nvidia.com/embedded/jetson-tx2, 2020b.
  * Li et al. [2011] S. Li, K. Chen, J. H. Ahn, J. B. Brockman, and N. P. Jouppi.  CACTI-P: Architecture-level Modeling for SRAM-based Structures with Advanced Leakage Reduction Techniques.  In _ICCAD_ , 2011.
  * Beck et al. [2018] Noah Beck, Sean White, Milam Paraschou, and Samuel Naffziger.  Zeppelin: An SoC for Multichip Architectures.  In _IEEE ISSCC_ , 2018.
  * Kannan et al. [2019] Ram Srivatsa Kannan, Lavanya Subramanian, Ashwin Raju, Jeongseob Ahn, Jason Mars, and Lingjia Tang.  GrandSLAm: Guaranteeing SLAs for Jobs in Microservices Execution Frameworks.  In _EuroSys_ , 2019.
  * Romero et al. [2021] Francisco Romero, Mark Zhao, Neeraja J. Yadwadkar, and Christos Kozyrakis.  Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines.  In _SoCC_ , 2021.
  * aws [2022b] AWS Serverless Data Analytics Pipeline Reference Architecture.  https://aws.amazon.com/blogs/big-data/aws-serverless-data-analytics-pipeline-reference-architecture/, 2022b.
  * Shankar et al. [2020] Vaishaal Shankar, Karl Krauth, Kailas Vodrahalli, Qifan Pu, Benjamin Recht, Ion Stoica, Jonathan Ragan-Kelley, Eric Jonas, and Shivaram Venkataraman.  Serverless Linear Algebra.  In _SoCC_ , 2020.
  * Khandelwal et al. [2022] Anurag Khandelwal, Yupeng Tang, Rachit Agarwal, Aditya Akella, and Ion Stoica.  Jiffy: Elastic Far-Memory for Stateful Serverless Analytics.  In _EuroSys_ , 2022.
  * Choi et al. [2020] Sean Choi, Muhammad Shahbaz, Balaji Prabhakar, and Mendel Rosenblum.  $\lambda$-NIC: Interactive Serverless Compute on Programmable SmartNICs.  In _IEEE ICDCS_ , 2020.
  * Daw et al. [2021] Nilanjan Daw, Umesh Bellur, and Purushottam Kulkarni.  Speedo: Fast Dispatch and Orchestration of Serverless Workflows.  In _SoCC_ , 2021.
  * Patterson et al. [2022] Liam Patterson, David Pigorovsky, Brian Dempsey, Nikita Lazarev, Aditya Shah, Clara Steinhoff, Ariana Bruno, Justin Hu, and Christina Delimitrou.  HiveMind: A Hardware-Software System Stack for Serverless Edge Swarms.  In _ISCA_ , 2022.
  * Kang et al. [2021] Seongyoung Kang, Jiyoung An, Jinpyo Kim, and Sang-Woo Jun.  MithriLog: Near-Storage Accelerator for High-Performance Log Analytics.  In _MICRO_ , 2021.
  * Mailthody et al. [2019] Vikram Sharma Mailthody, Zaid Qureshi, Weixin Liang, Ziyan Feng, Simon Garcia de Gonzalo, Youjie Li, Hubertus Franke, Jinjun Xiong, Jian Huang, and Wen-mei Hwu.  DeepStore: In-Storage Acceleration for Intelligent Queries.  In _MICRO_ , 2019.
  * Soltaniyeh et al. [2022] Mohammadreza Soltaniyeh, Veronica Lagrange Moutinho Dos Reis, Matt Bryson, Xuebin Yao, Richard P. Martin, and Santosh Nagarakatte.  Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®.  In _ICPE_ , 2022.
  * Kwon et al. [2022] Miryeong Kwon, Donghyun Gouk, Sangwon Lee, and Myoungsoo Jung.  Hardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep Learning Service on Large-Scale Graphs.  In _FAST_ , 2022.
  * Do et al. [2013] Jaeyoung Do, Yang-Suk Kee, Jignesh M. Patel, Chanik Park, Kwanghyun Park, and David J. DeWitt.  Query Processing on Smart SSDs: Opportunities and Challenges.  In _SIGMOD_ , 2013.
  * Kang et al. [2013] Yangwook Kang, Yang-suk Kee, Ethan L. Miller, and Chanik Park.  Enabling cost-effective data processing with smart SSD.  In _MSST_ , 2013.
  * Koo et al. [2017] Gunjae Koo, Kiran Kumar Matam, Te I, H. V. Krishna Giri Narra, Jing Li, Hung-Wei Tseng, Steven Swanson, and Murali Annavaram.  Summarizer: Trading Communication with Computing near Storage.  In _MICRO_ , 2017.
  * Seshadri et al. [2014] Sudharsan Seshadri, Mark Gahagan, Sundaram Bhaskaran, Trevor Bunker, Arup De, Yanqin Jin, Yang Liu, and Steven Swanson.  Willow: A User-Programmable SSD.  In _OSDI_ , 2014.
  * Tiwari et al. [2013] Devesh Tiwari, Simona Boboila, Sudharshan Vazhkudai, Youngjae Kim, Xiaosong Ma, Peter Desnoyers, and Yan Solihin.  Active Flash: Towards Energy-Efficient, In-Situ Data Analytics on Extreme-Scale Machines.  In _FAST_ , 2013.
  * Tiwari et al. [2012] Devesh Tiwari, Sudharshan S. Vazhkudai, Youngjae Kim, Xiaosong Ma, Simona Boboila, and Peter J. Desnoyers.  Reducing Data Movement Costs Using Energy-Efficient, Active Computation on SSD.  In _HotPower_ , 2012.
  * Gu et al. [2016] Boncheol Gu, Andre S. Yoon, Duck-Ho Bae, Insoon Jo, Jinyoung Lee, Jonghyun Yoon, Jeong-Uk Kang, Moonsang Kwon, Chanho Yoon, Sangyeun Cho, Jaeheon Jeong, and Duckhyun Chang.  Biscuit: A Framework for Near-Data Processing of Big Data Workloads.  In _ISCA_ , 2016.
  * Dhar et al. [2019] Ashutosh Dhar, Sitao Huang, Jinjun Xiong, Damir Jamsek, Bruno Mesnet, Jian Huang, Nam Sung Kim, Wen-mei Hwu, and Deming Chen.  Near-Memory and In-Storage FPGA Acceleration for Emerging Cognitive Computing Workloads.  In _ISVLSI_ , 2019.
  * Ruan et al. [2019] Zhenyuan Ruan, Tong He, and Jason Cong.  INSIDER: Designing In-Storage Computing System for Emerging High-Performance Drive.  In _ATC_ , 2019.
  * eid [2020] Eideticom NoLoad SmartSSD.  https://www.eideticom.com/media/attachments/2020/11/09/noload_smartssd_product_brief1.pdf, 2020\.
  * sca [2021] ScaleFlux CSD 3000.  https://oss.scaleflux.com/202111/file_20211104_133214475.pdf, 2021\.
  * pli [2021] Pliops Extreme Data Processor.  https://pliops.com/product/, 2021.
  * net [2020] Netint Codensity Solid State Drives.  https://netint.ca/wp-content/uploads/2020/07/NETINT_Codensity_D408_SSD_Product_Brief_20PB001-1.pdf, 2020\.