# Towards Variable-Length In-Network Caching

Gyuyeong Kim
Sungshin Women’s University

###### Abstract

We present StarCache, a new in-network caching architecture that can cache
variable-length items to balance a wide range of key-value workloads. Unlike
existing works, StarCache does not cache hot items in the switch memory.
Instead, we make hot items revisit the switch data plane continuously by
exploiting packet recirculation. Our approach keeps cached key-value pairs in
the switch data plane while freeing them from item size limitations caused by
hardware constraints. We implement a StarCache prototype on an Intel Tofino
switch. Our experimental results show that StarCache can balance highly skewed
workloads with various key and value sizes.

##  1 Introduction

Key-value stores are the fundamental building blocks for online services owing
to fast access to variable-length data [4, 17, 5, 31]. Workloads are generally
read-intensive, and most items are hundreds of bytes [13, 8, 9]. A challenge
in scaling key-value stores is mitigating load imbalance among storage servers
caused by skewed key popularity (e.g., trending events). The load imbalance
leads to the overloading of hot item servers, causing significant performance
degradation [22, 30, 28].

In-network caching is a promising load-balancing approach that builds a cache
in the switch by leveraging programmable switch ASICs like Intel Tofino [6].
The idea is to store hot items in the switch memory. Requests read the item
value by referring to the cache lookup table where the item key is the table
index. Compared to server-based caching, in-network caching provides much
higher performance without extra cache nodes or accelerators thanks to a Tbps-
scale processing throughput.

Unfortunately, existing solutions [22, 36, 33] can cache only tiny items due
to hardware constraints. Specifically, the key size is limited to 16 bytes due
to the maximum match-key width of the match-action table. A limited number of
match-action stages and a small accessible byte size per stage make it
difficult for the value size to exceed 128 bytes. These numbers are far from
typical workloads where key and value sizes are still small enough to be
stored in a single packet, but larger than the limit [8, 34, 13, 9, 38]. For
example, in 54 Twitter workloads [38], most keys are tens of bytes and many
values are less than 1024 bytes. However, the existing solutions cannot cache
even a single item in 42 out of 54 workloads because the keys and values are
generally larger than the limits. Between the other 12 workloads, only 2
workloads have a portion of cacheable items larger than 50%. This trend is
similar in Facebook workloads as well [13]. In this context, we ask the
following question: how can we cache variable-length items in the programmable
switch for balancing a wide range of key-value workloads?

We present StarCache, a new in-network caching architecture that can cache
variable-length key-value items in the switch data plane. Our idea is to make
hot items keep visiting the switch data plane continuously in the form of
cache packets, instead of caching the hot items in the switch memory. We
efficiently use packet recirculation, a built-in feature of the programmable
switch ASIC. This enables a packet to revisit the switch data plane through an
internal loopback port without going outside. The switch can cache variable-
length key-value pairs without being limited by the hardware constraints owing
to circulating cache packets. For cache serving, the switch maintains small
request metadata in the switch memory and forwards the matched cache packet to
the client if there is a pending request. To support variable-length keys, we
use key hashes for cache lookups and resolve hash collisions at the client by
comparing the requested key and the returned key, which is contained in the
cache packet.

The key insight behind the idea is that it is hard to overcome the memory
access constraint of switch hardware if we stick to the approach of caching
items in the switch memory. For example, the idea of using key hashes for
variable-length keys is difficult to realize in the existing architecture.
This is because the switch does not have enough hardware resources to store
keys and values together. In StarCache, cache packets contain both keys and
values, allowing to handle hash collisions at the client. Meanwhile, we may
recirculate requests multiple times to read large values. However, this limits
scalability because the number of in-flight packets in the recirculation port
increases proportionately to the number of requests. In addition, the switch
has tens of ports but only one recirculation port. In ours, the recirculation
port is not a performance bottleneck. This is because 1) requests are never
recirculated; 2) the number of in-flight cache packets is small; 3) the switch
can process multiple cache packets simultaneously with massive hardware-level
parallelism, thus minimizing queueing between cache packets.

Realizing the idea imposes various technical challenges. First, the switch
should buffer multiple request metadata for cached items. We design a request
table as a circular queue structure using multiple register arrays. Our
request table provides isolated data access among different keys, which means
that requests for a key do not have to wait until requests for other keys are
handled. Second, a cache packet should serve multiple requests once fetched,
rather than a single request. To achieve this, the switch clones the cache
packet before forwarding it to the client using the packet replication engine
(PRE), a special module of the switch ASIC. Next, we should ensure cache
coherence. We design an invalidation-based coherence protocol that can fetch
new values and reply to the client simultaneously. Lastly, the switch should
deal with dynamic workloads where key popularity and value size change over
time. We design a switch control plane mechanism that considers dynamic
workloads.

We have implemented a StarCache prototype on an Intel Tofino switch. To
evaluate StarCache, we build a testbed consisting of commodity servers, which
are interconnected via an Intel Tofino switch. Our experimental results show
that StarCache provides load balancing for skewed workloads having diverse
item sizes. In addition, StarCache is robust to various workload conditions.
We also show that StarCache can handle dynamic workloads where key popularity
changes over time.

In summary, this work makes the following contributions.

  * •

We design StarCache, an in-network caching architecture where the switch can
cache variable-length key-value items to balance a wide range of workloads.

  * •

We propose several techniques to address various challenges in designing a
custom switch data plane.

  * •

We implement a StarCache prototype on Intel Tofino switches and show the
efficiency and robustness of StarCache through extensive testbed experiments.

##  2 Background and Motivation

###  2.1 In-Network Load-Balancing Caches

Balancing key-value stores with a small cache. Today’s online services rely on
key-value stores since a user request may generate hundreds of key-value store
requests. In distributed key-value stores, balancing imbalanced loads between
storage servers caused by different key popularity is a primary challenge.
Caching is a powerful technique to address the challenge, which is based on a
theoretical foundation that we can balance loads for $N$ servers (or
partitions) by caching the $O(N\log N$) hottest items, regardless of the
number of items [16]. Caching on commodity servers is a natural option to
build the load-balancing cache [17, 16, 30]. However, the performance of a
cache node is not sufficient to handle many requests due to the limited packet
processing throughput of CPUs.

(a) The NetCache architecutre [22] (b) The proposed StarCache architecture

Figure 1: Comparison of the high-level idea with the NetCache architecture.
In StarCache, clients submit requests, and then circulating cache packets read
request metadata. Since both keys and values are in cache packets, hardware
constraints do not limit the item size. For variable-length keys, StarCache
uses key hashes while handling hash collisions at the client.

Why in-network caching? Building a load-balancing cache in the switch by
leveraging programmable switch ASICs like Intel Tofino [6] is an attractive
approach. Unlike server-based caching, this approach can provide high-
performance load balancing without additional nodes. This is because the
switch is highly optimized for packet I/O and can process a few billion
packets per second, which is an order of magnitude higher than that of the CPU
in servers. The switch also provides a low packet processing delay within
hundreds of nanoseconds.

Limited cacheable item size. Existing works [22, 33, 36] demonstrate the
efficiency of in-network caching, but they limit the cacheable item size.
Specifically, they support items of up to 16-byte keys and 128-byte values. In
the reconfigurable match table (RMT) switch architecture [11] like Intel
Tofino, the switch data plane consists of $n$ match-action stages [6]. Each
stage has a static memory and a few ALUs that can perform simple arithmetic
operations on $k$ bytes. The existing works store the value of cached items
across multiple stages after fragmentation, limiting the maximum value size to
$n\times k$ bytes. Unfortunately, $n\times k$ of the current switch is quite
small, and the available number of stages for accessing the value is less than
$n$ since other non-caching functions also consume match-action stages. The
key size is also limited by the maximum match-key width of the match-action
table. This is because the existing works use a match-action table to
implement the cache lookup table where the item key is the match key.

Why is it not enough? Many key-value items are indeed small, but typically
exceed the existing size limits. We have analyzed 54 Twitter workloads [38],
and observe that existing solutions are not enough to deal with typical
workloads. For example, only 3.7% of the workloads have over 80% of keys $\leq
16$ B. 38.9% of the workloads have over 80% of values $\leq 128$ B. The
existing works can cache less than 10% of items for 85% of the workloads
because, to be cacheable, both key and value sizes must be within the limits.
Furthermore, they cannot cache even a single item for 77.8% of the workloads.
Facebook workloads [13] show similar size distributions. The average key size
is 27.1 bytes, and the median value size is 235 bytes. These numbers exceed
the size limits of the existing architecture.

###  2.2 Variable-Length In-Network Caching

Design rationale. We argue that it is hard to overcome the memory access
constraint if we stick to the approach of caching data in the switch memory
because the constraint is determined at the time of manufacture. Therefore, we
propose StarCache, a different approach to in-network caching.

Our high-level idea is to keep cached key-value pairs circulating within the
switch data plane through packet recirculation, a built-in feature of the
programmable switch that makes the packet revisit the switch data plane. In
our approach, cached items read requests rather than the requests read the
cached items. Specifically, clients submit requests, and the switch maintains
small request metadata like the client IP address. Cache packets, which are
reply packets containing the key and the value of cached items, continue to
revisit the data plane via an internal recirculation port while actively
checking pending requests. If a request for the key is found, the matched
cache packet is forwarded to the client. Otherwise, the cache packet is
recirculated. In Figure 1, we illustrate the difference between StarCache and
the NetCache architecture [22], which is the reference architecture of
existing in-network caching solutions [22, 33, 36].

Variable-length keys. Our approach enables us to support variable-length key-
value pairs. For variable-length keys, one possible solution within the
limited match-key width of the match-action table is to use a fixed-size key
hash as the match key. To handle potential hash collisions, we should compare
the requested key and the key of the value that has been read. To do this, the
switch data plane should also store the item keys alongside values.
Unfortunately, it is hard to realize in the existing architecture because
there are not enough match-action stages to store both keys and values.
StarCache can realize the idea of using key hashes since the switch data plane
maintains cached key-value pairs in the form of circulating reply packets.
Therefore, the client can get the correct value from the storage server if the
requested and returned keys differ.

Variable-length values. For variable-length values, a possible approach in the
existing architecture is recirculating requests. Specifically, requests can
access the switch memory multiple times using packet recirculation. However,
this approach is not scalable because the number of in-flight packets per
second in the recirculation port increases in proportion to the number of
requests. For example, if every request is recirculated 8 times to read a
1024-byte value, the effective throughput of the recirculation port is reduced
to 1/8 of the bandwidth. Unfortunately, a pipeline in the programmable switch
has only one internal recirculation port, while there are tens of regular
front ports. This means that the recirculation port can limit the performance.

In StarCache, the recirculation port is not a bottleneck. The switch never
recirculates requests as they are in the memory until cache packets read them.
We recirculate only a small number of cache packets. The time to recirculate
and process a cache packet is a few hundred nanoseconds like normal packets.
Furthermore, the switch can process multiple packets simultaneously thanks to
hardware-level parallelization, which reduces queueing between cache packets.

Trade-off. There is a trade-off of using recirculation to achieve variable-
length in-network caching. Although we are free from size limitations, we
should reasonably limit the number of cache entries to avoid long request wait
times. Otherwise, the switch may not have enough space to store requests due
to many pending requests, or the request latency would increase excessively
even if it is stored. This is because, although the switch can process many
packets simultaneously, the queueing between cache packets may still happen
due to many cache packets beyond the capability of switch hardware.
Fortunately, caching a small number of hot items is enough to balance skewed
workloads [16, 28].

Technical challenges. To balance the trade-off and translate the idea into a
working system, we should address several technical challenges as follows.

  * •

The switch should maintain multiple requests, especially for the same cached
item. If not, many requests for cached items would be forwarded to the server
as they cannot be stored in the data plane.

  * •

A cache packet should serve multiple requests once fetched. Otherwise, the
switch must fetch the cache packet from the server again, degrading
performance.

  * •

We should design a cache coherence mechanism between the switch and storage
servers. If not, a request may read a stale value for the requested key.

  * •

The switch should adapt to changes in the workload, such as key popularity.

Figure 2: StarCache architecture.

##  3 StarCache Design

###  3.1 StarCache Architecture

Figure 2 shows the overview of the StarCache architecture.

Switch data plane. The switch data plane in StarCache consists of several
custom tables and modules as follows.

  * •

Lookup table is for cache lookups, which is a match-action table that uses a
hash of item keys as the match key. The table returns a table index needed to
access other tables and modules. Cache entries are managed by the controller
in the switch control plane.

  * •

State table, implemented as a register array, maintains the validity of the
value of cached items. This is needed to prevent read requests from obtaining
the stale value when there are pending writes for the requested key.

  * •

Request table, implemented with multiple register arrays and registers, stores
request metadata like client IP address, L4 port number, and request sequence
number.

  * •

Key counters consist of two registers and one register array. The key
popularity counter is a register array that tracks the key popularity for each
key. The cache hit counter and the overflow request counter are registers that
track the total number of cache hits and the total number of overflow requests
for all cached keys. The controller uses these for cache sizing.

  * •

Cloning module comprises a few match-action tables for cloning reply packets.
Packet cloning is done by the PRE, a special hardware module in the switch
ASIC [6]. The switch clones packets with negligible overhead since the switch
duplicates only a packet descriptor pointing to the packet instead of the
whole packet.

Meanwhile, the switch invokes the custom processing logic only for StarCache
packets by referring to reserved L4 ports. We use UDP to handle messages for
better latency like existing works [28, 30, 22] while using TCP for top-$k$
item reports in cache updates. For normal packets, the switch only applies the
traditional packet forwarding logic.

Switch control plane. The controller in the switch control plane performs
cache updates and cache sizing. It evicts the least popular keys and inserts
new hot keys based on server-side periodic top-$k$ reports for uncached keys
and the switch-side report for cached keys. Note that while the controller
handles key insertion, fetching values is done via the switch data plane. The
controller also adjusts the cache size dynamically to deal with dynamic
workloads. It also handles switch configurations.

Clients and servers. Clients should specify the operation type, the item key
to request, and the key hash. Storage servers run a server application that
acts as a shim layer that translates StarCache messages to API calls for key-
value stores and vice versa. The server returns replies for read and write
requests like regular storage servers. However, in cases of write requests for
cached items, the server returns a write reply with the item value to fetch
the latest value.

Figure 3: StarCache packet format.

###  3.2 Packet Format

Figure 3 depicts the StarCache packet format. The switch only parses the
header. The payload of StarCache message consists of the key and the value.
Our header is 22 bytes. Therefore, StarCache supports a key-value pair of up
to 1438 bytes for a single packet. For example, the switch can cache an item
with a 16-byte key and a 1422-byte value. Our header fields are as follows.

(a) Read request (b) Read reply (c) Write request (d) Write reply

Figure 4:  Request processing.  (a) the switch drops the request after
inserting request metadata into the request table; (b) If a circulating cache
packet reads request metadata, the switch clones the packet so that the
original packet is forwarded to the client and the cloned one is recirculated
again for further serving; (c) the switch invalidates the item to avoid
inconsistent reads if a write request is for a cached item; (d) upon receiving
a write reply for a cached item, the switch validates the item. After that,
the switch clones the packet similar to (b). The cloned packet is recirculated
as a read reply after the first recirculation.

  * •

OP (1 byte): the operation type, which can be R-REQ (Read request), W-REQ
(Write request), R-REP (Read reply), W-REP (Write reply), F-REQ (Fetch
request), F-REP (Fetch reply), and CRN-REQ (Correction request).

  * •

SEQ (4 bytes): a sequence number as request IDs for handling hash collisions.

  * •

HKEY (16 bytes): the key hash as cache lookup index.

  * •

FLAG (1 byte): a versatile flag field 1) to distinguish write requests for
cached items from those for uncached items; 2) to store the number of packets
comprising a multi-packet item.

###  3.3 Basic Request and Reply Processing

In Figure 4, we illustrate the packet processing logic.

Request generation at clients. To send a request, clients should specify the
OP and HKEY fields. The SEQ field is increased by one for every request, which
is used to resolve hash collisions. As we consider hash-partitioned storage,
the destination storage server is determined by hashing the key.

Read requests. The switch first refers to the lookup table using a key hash to
get a table index, which is used to access the slots in other tables. If
missed, the switch forwards the packet to the server as the request is for an
uncached item. In case of a cache hit, the key popularity counter and the
cache hit counter are incremented by one. This key popularity is collected by
the controller in the switch control plane periodically for cache updates.
Next, the switch checks the validity of the requested key by looking into the
state table. The state is binary: valid or invalid. Being invalid means that
there are pending write requests for the key. In this case, the switch
forwards the read request to the server to avoid reading stale item values. If
the key is valid, the switch checks the request table to see whether there is
a free slot. The switch puts the request metadata into the table when a free
slot is found. Otherwise, the request is destined to the server after the
overflow request counter is increased. Request metadata includes the client IP
address, L4 port number, and SEQ as request IDs. After insertion, the switch
drops the packet. This is acceptable since a cache packet will soon service
the stored request.

Read replies. When receiving a read reply, the switch first checks to see if
the ingress port is the recirculation port. If it is, the reply is a cache
packet; otherwise, it is a reply for an uncached item sent by the server. For
cache packets, if the cache misses or the state of the value is invalid, the
switch drops the packet. A cache miss for a cache packet means that the
controller evicted the key from the lookup table due to a change in key
popularity or the cache size. The invalid state indicates a write request with
a new value is in progress.

The switch goes through the lookup table and the state table, and then looks
for pending requests in the request table. If a request is found, the switch
forwards the cache packet to the client after updating the header with
metadata and removing the metadata from the table slot. Here, to make the
cache packet serve more requests, the switch clones the packet before packet
forwarding. The switch forwards the original packet to the client and the
cloned one to the recirculation port. The switch also recirculates the packet
if there are no pending requests in the table.

Write requests. The first switch checks whether the requested key is in the
cache lookup table. If it is, the value for the key is invalidated to prevent
reading the outdated value. In addition, the switch sets the FLAG field to 1
so that the storage server can append the value in the write reply. Regardless
of a cache hit, the switch forwards the request to the storage server to
update the value of the key in the server.

Write replies. If the key is cached, the switch validates the value to allow
read requests to get the latest value. Similar to reads, the switch clones the
packet so that the client receives the write reply while the switch has a new
cache packet simultaneously. The switch updates the OP field to R-REP of the
cloned packet after the first recirculation since cache packets are read
replies. For a cache miss, the switch forwards the packet to the client since
the reply is for an uncached item.

Other types of messages. Fetch requests, fetch replies, and correction
requests are delivered to the server as standard packets. The fetch reply is
forwarded to the controller, who has sent a fetch request for cache updates.

Figure 5: An example of queueing operations in the request table.  Read
requests enqueue new metadata and cache packets dequeue the stored metadata.

###  3.4 Buffering Request Metadata

Circular queue-based request table. We should store multiple request metadata
for cached items because the requests should wait until cache packets serve
them. The switch should be able to buffer concurrent read requests for the
same cached item. If not, many requests for the cached item would be forwarded
to the storage server due to the lack of vacancies. Furthermore, a request for
a key should be isolated from other requests for other keys since the request
may undergo excessive queueing delay due to the different requests.

We design a request table based on a circular queue structure. Our request
table provides a logical queue for each cached key and the queue can be
accessed in $O(1)$ by efficient indexing. This provides fast and isolated
queue access for a key regardless of other keys. The request table consists of
6 register arrays: the first three arrays are for storing request metadata for
each key (i.e., client IP address, request sequence number array, and L4 port
number). The other three arrays are for managing queue operations. The queue
length array maintains the number of stored request metadata for each key. The
front pointer array handles dequeue operations by tracking the first request
metadata for each key. The rear pointer arrays perform enqueue operations by
keeping track of the last request metadata for each key.

The queue management arrays are indexed using a table index ($CacheIdx$)
returned by the cache lookup table. The request metadata arrays are accessed
via a request index, which is defined as $ReqIdx=CacheIdx\times S+i$ where $S$
is the maximum queue size per key and $i$ is the offset for $i$th slot in the
logical queue for a key and is given by the pointer arrays. The switch uses
three match-action stages for a request table. Specifically, the switch first
checks the queue status (Stage 1), and performs en/dequeue operations if the
queue is not empty or not full (Stage 2). Next, the switch gets/puts metadata
(Stage 3).

An example for table operations. Figure 5 shows a simple example of table
operations. We assume that the request table can maintain up to 4 requests for
each key. We consider enqueueing and dequeueing operations using read requests
and cache packets, respectively. In stage 2, the rear pointer changes to 0
from 3 since we implement a circular queue. It is also easy to see that the
request metadata for different keys does not collide since we partition the
metadata arrays using the indexing formula of $ReqIdx=CacheIdx\times S+i$.

###  3.5 Cache Serving

Handling multiple requests via packet cloning. In StarCache, cache packets
pass through the switch data plane repeatably to serve pending requests for
cached items. One challenge is serving multiple requests with a cache packet
fetched only once because the cache packet is forwarded to the client after
updating the header using the retrieved metadata. A strawman is to fetch the
cache packet from the server again, but this approach is inefficient as the
switch cannot serve pending requests for the key until the fetching is
completed. Fetching multiple copies of the cache packet may be a solution, but
this incurs excessive queueing delay between cache packets. In addition, it is
hard to know how many cache packets should be fetched because we cannot
predict the number of requests for each cached item in advance.

Figure 6: An example of cache packet cloning using multicast.  Packet cloning
is done in the PRE, a hardware module of the switch ASIC for packet cloning.

To address this, we utilize packet cloning, another built-in feature of the
programmable switch ASIC besides packet recirculation. Packet cloning is done
by the PRE, a hardware module specialized for packet cloning in the switch
ASIC. Cloning has a negligible overhead for the following reasons. First, the
PRE is located after the ingress pipeline. This means the switch does not
repeat the ingress pipeline processing for the cloned packet, not causing
extra ingress processing delay. Second, the switch does not copy the entire
packet. It only copies the small descriptor pointing to the memory location of
the packet and reuses the packet.

We use multicast to forward the original and clone packets. Specifically,
through a match-action table using the destination IP address acquired from
the request table, the switch gets a multicast group ID that specifies the
regular port number directed to the client and the internal recirculation port
number. The switch forwards the original packet to the client and the cloned
one to the switch data plane again, making it serve more requests.

An example of cache cloning. In Figure 6, a cache packet gets the client IP
address of 10.0.1.102 and other metadata from the request table. The packet
obtains the multicast group ID as 2 using the destination IP address (i.e.,
the client IP address). The cache packet then gets a pair of destination port
numbers. 52 is the port number directed to the client, and 68 is the
recirculation port number. After that, the switch clones the packet and
forwards the original and cloned ones to each port.

Figure 7: An example of handling hash collisions.  The client gets the correct
value by sending a correction request.

###  3.6 Handling Hash Collisions

The existing works use the item key as the match key of the cache lookup
table, which is implemented as a match-action table. Since the match-key width
is limited by hardware, we cannot use the key exceeding the size limit. We may
use a register array, but the size limitation in the register index is
stricter than in the match-action table. Therefore, we use the fixed-sized key
hash as the match key. A challenge is how to resolve potential hash
collisions. We should handle this because a request may read the wrong value
of other keys.

Client-side collision resolution. We handle hash collisions at the client
level by maintaining a list of the keys for each request that has not yet
received a reply. The list is indexed by $pkt.seq$, a request ID. Request
packets contain both the original key and the key hash since we should use the
original key to get the value in servers. Upon receiving read replies, the
client checks whether the requested key in the list and the returned key in
the reply header are identical. If different, the client sends a correction
request to the storage server. The switch then forwards the request without
applying the cache logic so that the client gets the correct item value from
the storage server. One issue is that requests for uncached but hash-collided
keys always undergo this process. However, this is uncommon since our 128-bit
key hash provides a low collision probability. In our experience, hash
collisions only occur occasionally during cache updates.

An example of resolving hash collisions. We plot an example of handling hash
collisions in Figure 7 where the requested key is DDDD but the returned key is
AAAA. Upon receiving the reply, the client detects that the retrieved value is
a wrong value by referring to the key list using $pkt.seq=1$. The client then
sends a correction request to the storage server. The switch bypasses the
cache logic, and forwards the packet to the server. The storage server returns
the value as a read reply, and the client finally gets the correct value for
the key DDDD. The client removes the key from the list. $pkt.seq$ is finite
and wraps around if it reaches the maximum value.

###  3.7 Cache Coherence

To ensure cache coherence between the switch and servers, we design an
invalidation-based coherence protocol illustrated in Figure 4 (c) and (d). The
switch invalidates the value when handling a write request for a cached item
and revalidates the value upon receiving a write reply. Ideally, the outdated
cache packets should be replaced with the latest cache packet. However,
requests may be returned with the stale value since the outdated packets are
still circulating in the switch data plane even if the item value becomes
invalid. To address this, the switch drops the cache packet if the item is
cached but its value is invalid. Owing to the fact that the cache packet is
dropped before accessing the request table, we can prevent read requests from
retrieving the stale value until a new cache packet is fetched.

In StarCache, the server sends a single reply packet, and the switch updates
the value and replies to the client simultaneously by cloning the packet. Note
that the existing architecture [22] sends two different packets for updating
the value and replying to the client, which degrades write performance.

Figure 8: Cache update.  The controller in the switch control plane updates
the entries of the cache lookup table and the cache size by tracking workload
changes.

###  3.8 Cache Updates

StarCache can handle dynamic workloads by tracking workload changes as
follows.

Dealing with key popularity changes. To handle key popularity changes over
time, we update cache entries based on periodic popularity reports from the
switch and storage servers. In particular, we count the popularity of cached
keys in the switch data plane using the key popularity counter. The controller
keeps track of key popularity by reading the counter periodically. Meanwhile,
storage servers periodically report the top-$k$ keys to the controller, which
are popular uncached keys. The servers use a count-min sketch with five hash
functions to track key popularity in a memory-efficient manner while ensuring
accuracy. To reflect the recent status only, we reset all the counters to zero
after reporting.

If cache entries need to be updated due to a change in key popularity, the
controller removes the victim key from the cache lookup table and inserts a
new popular key instead. The controller then sends a fetch request to the
storage server containing the latest value for the key. Upon receiving the
request, the server fetches a new cache packet to the switch data plane and
replies to the controller.

Dynamic cache sizing. Since StarCache uses recirculation, it is important to
avoid the queueing between cache packets. If not, many requests for cached
items may be forwarded to the storage server since they did not find a free
slot in the request table where cache serving is delayed due to the queueing
between cache packets. Although we try to set the best cache size at first,
the cache size may cause a negative effect as the workload changes over time.

To address this, the controller dynamically changes the cache size to maximize
the performance while avoiding the adversarial effect. The controller
calculates the request overflow ratio, which is defined as the number of
requests for cached items that were sent to the server divided by the total
number of requests for cached items. The controller compares the ratio with a
pre-defined threshold (e.g., 1%). If the ratio exceeds the threshold, the
controller decreases the cache size by one. Otherwise, the cache size is
increased by one.

###  3.9 Handling Practical Requirements

Multi-packet items. Many key-value items are less than the MTU size. However,
some items may exceed the MTU size, such as articles and photo objects (e.g.,
Wikipedia and Flickr workloads [9]). To cache multi-packet items, we fetch
multiple cache packets with fragmented values for the same key. We maintain
the number of forwarded cache packets for each item by placing another
register array alongside the request table, the ACKed packet counter. The
initial value of each slot is 1 since most items are single-packet. When
fetching item values, the storage server puts the number of packets comprising
the item in the FLAG field. When handling cache packets, the switch retrieves
metadata as usual, but it does not manipulate the slots of all register arrays
in the request table if the slot value in the ACKed packet counter is not
equal to the value in FLAG. Instead, the switch increases the slot value by
one. When the slot value in the counter equals FLAG, the switch removes
request metadata as it is ready to be finished.

However, this may reduce the performance gains since StarCache has to limit
the cache size to avoid queueing between cache packets, and multiple cache
packets occupy the cache for one large item. Therefore, the balancing
efficiency may decrease if many items consist of multiple packets. This can be
mitigated if the bandwidth of the recirculation port increases in next-
generation switch hardware.

Failure handling. For packet loss, we can use an application-level mechanism.
For example, our controller uses UDP with a timeout-based mechanism to send
fetch requests and receive fetch replies. Storage servers use TCP to report
top-$k$ items to the controller. Meanwhile, server failures do not cause a
StarCache-specific problem. Switch failures result in the loss of cached
items, but the cache can be reconstructed quickly after the switch is
recovered.

Multi-rack deployment. StarCache can scale to multiple racks as items of
servers are only cached in the ToR switch. For example, assume that we have
rack 1 and rack 2 where ToR switches, $ToR1$ and $ToR2$, are interconnected
via a spine switch $SPN$. When $CLI$, a client in rack 1, sends a request to
$SRV$, a server in rack 2, the path is $CLI-ToR1-SPN-ToR2-SRV-ToR2-SPN-
ToR1-CLI$. If the requested item is cached, $ToR2$ is the only switch that
applies the cache logic. There is no issue even if rack 1 is the replicated
rack of rack 2 where the two ToR switches have the same cache entries because
the item is returned by $ToR1$.

###  3.10 Discussion

Write-back caching. Although most key-value workloads are read-intensive, some
workloads are write-intensive. Since we use write-through caching that updates
the switch and the server simultaneously with invalidation, our performance
gain decreases as the write ratio grows. FarReach [36] is a recent work that
enables write-back caching. Although it still cannot serve many workloads due
to the size limitation, it provides high performance regardless of the write
ratio. The difference between write-through caching and write-back caching is
whether a write request for cached items updates the storage server or not.
Therefore, StarCache can use write-back caching as well by letting the switch
return write replies upon receiving write requests after updating the cache
only. However, the switch with Intel Tofino 1 does not have sufficient stages
to implement other key modules like snapshot generation alongside the
StarCache modules. We expect that we may fully implement write-back caching in
StarCache using Tofino 2 [2] whose the number of stages is much higher than
Tofino 1 by roughly double, but this is beyond the scope of this work.

Multi-pipeline deployment. The multi-terabits performance of the programmable
switch ASIC comes from the multi-pipelined architecture. Each pipeline
consists of tens of regular ports and one internal recirculation port. In a
switch with 4 pipelines and 64 front ports, each pipeline has 16 ports and one
recirculation port statically. The switch resources are partitioned, and
metadata and memory data are not shared between the pipelines. Therefore, the
pipelines of client-directed ports and server-directed ports should be the
same. Otherwise, cache packets in a pipeline may not read requests since they
are in another pipeline. This can be achieved by mapping each pipeline of the
ToR switch to a client-directed port on the spine switches, which is feasible
only when the clients and servers are located in different racks. We expect
this to be mitigated with a new programmable switch architecture like MP5 [37]
that achieves high performance with a logical single pipeline.

We describe how StarCache can ensure cache coherence between switch pipelines.
First, the switch forwards write requests to the recirculation ports of other
pipelines and the server. In the other pipelines, the switch drops the request
after invalidation as the request has already been delivered to the server in
the source pipeline. Meanwhile, write reply packets are cloned in the PRE to
fetch the new cache packet for all pipelines. The switch forwards the original
reply to the client while recirculating the cloned reply via the recirculation
ports of all pipelines. We have implemented this and validated the
functionality and performance, but do not include this in the current design.
This is because write requests for cached items in a pipeline should be
propagated to the other pipelines to invalidate the item value via the
recirculation ports, and this may make the port a bottleneck.

##  4 Implementation

Client-server application. We develop an open-loop application in C using
NVIDIA Messaging Accelerator library (VMA) [3] that reduces host packet
processing time. This enables the processing of packets in user space by
implementing native RDMA verbs and bypassing kernel network stacks. The client
application measures throughput and latency by generating requests. The time
gap between consecutive requests follows an exponential distribution. The
server application has multiple threads where each thread is pinned to a
disjoint CPU core. To emulate multiple storage servers, we assign a partition
per thread so that each thread acts as an independent server. This technique
is used in NetCache [22] as well. In a similar vein, we implement a key-value
store with TommyDS [1], a high-performance hash table library.

Switch. We implement the switch data plane in P416 [10] for Intel Tofino [6].
We use Intel P4 Studio SDE 9.7.0 to compile the switch data plane. Our
prototype uses 10 match-action stages and 12.38% SRAM. The request table has
64K slots that can maintain up to 4K cached keys with a maximum queue size of
16 for each key. The controller is written in Python 3. In our prototype, we
use an additional register array for the request table to store the timestamp
of the request for latency measurement. The StarCache header also has 3 extra
fields of 1-B Cached, 4-B Lat, and 1-B ParID. The first two fields are
required to separately measure the latency of requests for cached and uncached
items. The last field is to store the partition ID as we emulate multiple
storage servers as dedicated threads in a physical node.

##  5 Evaluation

###  5.1 Methodology

Testbed setup. We build a cluster consisting of 8 nodes, which are connected
by an APS BF6064X-T switch with the Intel Tofino 1 [6]. The servers are
equipped with a 10-core CPU (Intel i5-12600K @ 3.7 Ghz), 32 GB of DDR5 memory,
and a 100GbE NVIDIA CX-5 NIC. The servers run Ubuntu 22.04 LTS with Linux
kernel 6.5.0. The 5 nodes act as clients to generate requests, and the
remaining 3 nodes are used to emulate multiple storage servers.

Compared schemes. We compare our work with NoCache and NetCache [22]. NoCache
is a mechanism without any cache logic. NetCache is the representative in-
network caching architecture. FarReach [36] and DistCache [33] also adopt the
NetCache architecture. We implement the core caching logic of NetCache, but
our implementation provides items up to 64-byte values across 8 stages with an
8-byte accessible size per stage. We find that the P4 compiler allocates only
two cache read tables per stage, even with pragma statements. We suspect that
this is due to compiler restrictions on our code. We clarify that this does
not mislead the conclusions of our experiments, as there is a negligible
difference in the latency required to read 64 bytes and 128 bytes at line rate
using the same number of match-action stages.

Workloads. We emulate 36 storage servers using 12 partitioned threads per
node. We basically consider a workload with 10M key-value pairs whose key
popularity skewness follows a Zipfian distribution with $\alpha=0.99$, since
it is regarded as typical skewness [14, 8]. Although both key and value sizes
impact whether an item is cacheable by NetCache, we use fixed 16-byte keys by
default for simplicity. Instead, we represent the portion of cacheable items
using different ratios between 64 bytes and 1024 bytes values. The 64-byte
value represents a cacheable item value of NetCache. The 1024-byte value is a
typical-sized item value in many workloads and is considered in many research
papers [30, 31, 32, 18, 7, 27]. We use a bimodal distribution with 12% 64-byte
and 88% 1024-byte values by considering the Cluster017 workload of Twitter
[13]. In the workload, 12% of values are $\leq 128$ B and 87% of values are
$>128$ and $\leq 1024$ B. 1% of values are between 1024 B and 1280 B, but we
consider them 1024 B for simplicity. Most experiments are for read-only
workloads like Cluster017 as we target read-intensive workloads.

Except for dynamic workloads, we try to preload 10K and 32 of static hottest
items for NetCache and StarCache, respectively. 32 is a nearly optimal cache
size for StarCache with the given workload. Since NetCache can cache only 12%
of items in the workload, the actual number of cached items is 1.2K. We choose
12% of keys among the 10K hottest keys with a uniform distribution. We store
the chosen keys as a text file to make experimental results consistent. The
controller loads the file and puts the keys to the switch cache.

Figure 9: Throughput with different skewness.  Figure 10:  Load on individual
storage servers.

###  5.2 Main Results

Throughput with different key access distributions. We measure the throughput
with different skewness and plot the results in Figure 9. StarCache (switch)
is the throughput of the cache in the switch. In other words, StarCache
(servers+switch) is the total throughput, the sum of server throughput and
switch throughput. We can see that StarCache provides high throughput
regardless of skewness, unlike the others whose throughput decreases as the
workload becomes more skewed. For example, StarCache provides high throughput
even for highly skewed workloads with Zipf-0.99. NetCache does not provide
high throughput as expected since many hot items are not cacheable. In the
Zipf-0.99 workload, StarCache has higher throughput than NoCache and NetCache
by 1.46$\times$ and 1.39$\times$, respectively.

Throughput on individual servers. We plot the load on individual servers in
Figure 10. We can see that NoCache and NetCache do not balance loads well.
This is due to the lack of caching logic for NoCache and many uncacheable hot
items for NetCache. However, StarCache can balance the loads between the
servers sufficiently since StarCache can cache variable-length items.

Latency. We measure the latency by varying target throughput. Figure 11 shows
the median and the 99th percentile latencies as a function of throughput.
Since there is no cache logic, NoCache does not provide low latency with
limited throughput. NetCache is better than NoCache, but still does not
provide the best performance. On the other hand, StarCache provides low
latency in both the median and 99th percentile latencies while providing the
highest throughput.

Impact of write ratios. To investigate the performance of StarCache in various
workloads, we measure the throughput by varying the write ratios. Figure 12
reports the throughput as a function of write ratios. As expected, the
throughput of StarCache decreases as the write ratio grows. This is because
the switch invalidates the cached key when handling a write request for a
cached item to ensure cache coherence. The switch cache provides benefits for
only read requests, the throughput is decreased with higher write ratios since
read requests for invalid cached items are forwarded to storage servers. With
the 100% write ratio, the throughput of StarCache converges to NoCache since
the cache does not provide any benefit. Meanwhile, similar to StarCache,
NetCache offers reduced throughput as the write ratio grows since it also
invalidates the cached key if there is a write for the key.

(a) Median (b) 99th percentile

Figure 11: Throughput vs. latency.  Figure 12:  Impact of write ratios.

Impact of item size. Figure 13 shows the throughput with different key sizes
and value sizes. In Figure 13 (a), we can see that the throughput of StarCache
decreases as the key size grows. This is because larger keys consume more
resources in storage servers and the switch. The latency to recirculate and
process a cache packet especially increases, though it still remains at a few
hundred nanoseconds. NoCache and NetCache also show similar results. The
results with different value sizes in Figure 13 (b) are similar to those with
different key sizes. StarCache shows the best performance except in a case
with 64-B values. NetCache provides the highest throughput for the case since
all items can be cacheable.

(a) Impact of key size (b) Impact of value size

Figure 13:  Impact of item size.  Figure 14:  Impact of number of servers.

###  5.3 Deep Dive

Impact of number of servers. Figure 14 shows the individual throughput of 9,
18, and 36 storage servers. We also present the result with 36 servers for the
uniform workload as the baseline. We can see that StarCache can balance loads
sufficiently regardless of the number of servers. StarCache does not achieve
perfect load balancing. This is because there are trade-offs to achieve
variable-length caching, like limited cache size and extra latency caused by
packet recirculation.

Impact of cache size. Figure 15 plots the throughput breakdown of StarCache
with different cache sizes and the corresponding overflow request ratios. We
can see that the total throughput increases as the number of cached entries
grows. However, the throughput is saturated around 16-32 cached items. The
throughput of the switch cache is also saturated as well. Figure 15 (b)
clarifies the reason. From 64 cached items, the overflow request ratio rapidly
increases, where the overflow requests indicate the requests for cached items
forwarded to the storage servers due to the lack of free slots in the request
table. Although the request table can maintain 256 request metadata for each
key simultaneously, the slots get full quickly if the queueing between cache
packets occurs due to too many cache packets.

(a) Throughput (b) Overflow request ratio

Figure 15:  Impact of cache size.

(a) Median (b) 99th percentile

Figure 16: Latency breakdown.

Latency breakdown. Figure 16 (a) and (b) plot the median and the 99th
percentile latency breakdowns, respectively. For the median latency, we can
see that there is no distinct difference between NetCache and StarCache for
servers. However, for StarCache, the latency of requests handled by the switch
is slightly higher than NetCache by $1\mu s$. This is because StarCache
requires requests to wait until circulating cache packets handle them. The
difference in the caching logic between NetCache and StarCache results in more
different tail latency as throughput grows. In Figure 16 (b), we can see that
the tail latency of requests handled by the switch cache in StarCache
gradually increases as throughput grows, unlike NetCache showing a
deterministic latency even if throughput is saturated. This is because the
queueing delay between requests for the same cached item occurs when the
throughput gets closer to the saturated throughput.

Figure 17:  Impact of multi-packet items.

Impact of multi-packet items. Figure 17 shows the performance of StarCache by
considering that all cached items are multi-packet items. For one packet, we
use a 1416-B value. For example, a 4-packet item has 16-B key and 5664-B
values. We can see that StarCache generally balances loads even with multi-
packet items. We observe that the balancing efficiency slightly decreases as
the value sizes grow. This is because we decrease the cache size by
considering that multiple cache packets take up the cache for a single item.

Impact of cacheable item ratios for NetCache. Figure 18 shows the throughput
and the balancing efficiency of NetCache and StarCache with different ratios
of 64-B values. This is to inspect whether NetCache is better than StarCache
when it can cache more items. The balancing efficiency is defined as
$LowSrv/HighSrv$ where $LowSrv$ is the storage server where the throughput is
the lowest between the storage servers, and $HighSrv$ is the storage server
with the highest throughput (i.e., the server with the hottest item). We use a
near-optimal cache size for StarCache. The number of cached items is 32 for
the ratios between 0-60%. We use 64 cache entries for 70-90%. For the ratio of
100%, we cache 128 hot items. Figure 18 (a) shows mixed results. StarCache
outperforms until 50%, but is beaten by NetCache after 60%. We note that the
throughput of NetCache in 80-100% is limited by the Tx throughput. However,
NetCache does not balance loads sufficiently. In Figure 18 (b), we see that
NetCache has poor balancing efficiency even if only 10% items are uncacheable.
Unlike NetCache, StarCache can balance loads across all the ratios. We recall
that the number of workloads where the portion of cacheable items for NetCache
is higher than 50% is only 2 between 54 workloads [13].

(a) Throughput (b) Balancing efficiency

Figure 18:  Impact of cacheable item ratios for NetCache.

Handling dynamic workloads. We investigate the performance of StarCache with
dynamic workloads. We use a hot-in pattern, which is the most radical workload
change. Specifically, every 10 seconds, the popularity of the 256 coldest
items and the 256 hottest items is swapped. This means that all cache entries
should be replaced. These sudden popularity changes are not common in
practice, but this enables us to evaluate the robustness of StarCache to
dynamic workloads.

Figure 19 (a) shows the throughput during 60 seconds. We can see that the
throughput decreases when item popularity changes but recovers within a few
seconds. This is because the controller in the switch control plane quickly
updates the cache entries and fetches cache packets based on the top-$k$
report of servers and the cache popularity counter of the switch data plane.
Figure 19 (b) plots the change in the cache size over time with a given
threshold of 0.05% and a minimum cache size of 32. The controller adjusts the
cache size dynamically by monitoring the overflow request ratio. We can see
that the cache size fluctuates between 32 and 36, hence the switch can
maintain the overflow request ratio within 0.05%.

(a) Throughput (b) Cache Size

Figure 19:  Performance with dynamic workloads.

##  6 Related work

In-network caching. We have witnessed a surge of domain-specific acceleration
through in-network computing (e.g., consensus [15], coordination [21], machine
learning [35], replication [26]). Storage systems are major target domains of
recent works (e.g., [22, 26, 39, 24, 25, 40, 28, 20, 29, 19, 12, 23]).
NetCache [22] is an early work that has a great impact not only on in-network
caching but also on the entire field of in-network computing. It shows the
potential and limitations of the programmable switch as an in-network
accelerator for distributed systems. DistCache [33] resolves inter-rack load
imbalance and FarReach [36] realizes write-back caching, but they can still
cache a limited size of items. StarCache enables variable-length in-network
caching by leveraging built-in features of the programmable switch while
sacrificing the cache size reasonably.

Other alternatives. SwitchKV [30] employs the OpenFlow switch for cache
lookups while items are cached in cache nodes, which provide limited
performance. Pegasus [28] is a selective replication solution where the switch
replicates hot items across storage servers for load balancing, functioning
similarly to caching. Unfortunately, its performance is limited by the
aggregate throughput of storage servers. StarCache provides high-performance
caching beyond the aggregate throughput with no extra hardware costs.

##  7 Conclusion

We proposed StarCache, an in-network caching architecture that is capable of
variable-length caching in programmable switches. The key idea of StarCache is
to make cached key-value pairs revisit the switch data plane continuously by
leveraging packet replication efficiently. We addressed technical challenges
to realize the idea of StarCache. We have implemented a StarCache prototype on
an Intel Tofino switch, and experimental results demonstrated that StarCache
can balance skewed workloads with diverse sizes of items. We believe that
StarCache is a fine example of overcoming fundamental switch hardware
constraints that put limits on application semantics, which are major hurdles
of in-network computing research.

## Acknowledgement

The author thanks Yeon-sup Lim for providing constructive feedback on the
earlier draft of the paper.

## References

  * [1] Tommyds c library.  https://www.tommyds.it/, Last accessed date: , 2018.
  * [2] Advanced congestion & flow control with programmable switches.  https://opennetworking.org/wp-content/uploads/2020/04/JK-Lee-Slide-Deck.pdf, Last accessed date: , April 2020.
  * [3] Nvidia messaging accelerator (vma).  https://docs.nvidia.com/networking/display/vmav9840, Last accessed date: , 2021.
  * [4] Redis key-value store.  https://redis.io/, 2023.
  * [5] Rocksdb: A persistent key-value store for flash and ram storage.  https://rocksdb.org/, Last accessed date: , 2023.
  * [6] Tofino switch.  https://github.com/barefootnetworks/Open-Tofino, Last accessed date: , 2023.
  * [7] David G. Andersen, Jason Franklin, Michael Kaminsky, Amar Phanishayee, Lawrence Tan, and Vijay Vasudevan.  Fawn: a fast array of wimpy nodes.  In Proc. of ACM SOSP, page 1–14, New York, NY, USA, 2009. Association for Computing Machinery.
  * [8] Berk Atikoglu, Yuehai Xu, Eitan Frachtenberg, Song Jiang, and Mike Paleczny.  Workload analysis of a large-scale key-value store.  In Proc. of ACM SIGMETRICS, page 53–64, New York, NY, USA, 2012.
  * [9] Michaela Blott, Ling Liu, Kimon Karras, and Kees Vissers.  Scaling out to a single-node 80gbps memcached server with 40terabytes of memory.  In Proc. of USENIX HotStorage, pages 8–12, USA, 2015.
  * [10] Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker.  P4: Programming protocol-independent packet processors.  SIGCOMM Comput. Commun. Rev., 44(3):87–95, July 2014.
  * [11] Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz.  Forwarding metamorphosis: Fast programmable match-action processing in hardware for sdn.  In Proc. of ACM SIGCOMM, pages 99–110, 2013.
  * [12] Tommaso Caiazzi, Mariano Scazzariello, and Marco Chiesa.  Millions of low-latency state insertions on asic switches.  Proc. of ACM CoNEXT, 1(CoNEXT3), nov 2023.
  * [13] Zhichao Cao, Siying Dong, Sagar Vemuri, and David H.C. Du.  Characterizing, modeling, and benchmarking rocksdb key-value workloads at facebook.  In Proc. of USENIX FAST, Santa Clara, CA, February 2020.
  * [14] Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears.  Benchmarking cloud serving systems with ycsb.  In Proc. of ACM SoCC, page 143–154, New York, NY, USA, 2010. Association for Computing Machinery.
  * [15] Huynh Tu Dang, Pietro Bressana, Han Wang, Ki Suh Lee, Noa Zilberman, Hakim Weatherspoon, Marco Canini, Fernando Pedone, and Robert Soulé.  P4xos: Consensus as a network service.  IEEE/ACM Trans. Netw., 28(4):1726–1738, aug 2020.
  * [16] Bin Fan, Hyeontaek Lim, David G. Andersen, and Michael Kaminsky.  Small cache, big effect: Provable load balancing for randomly partitioned cluster services.  In Proc. of ACM SoCC, New York, NY, USA, 2011.
  * [17] Brad Fitzpatrick.  Distributed caching with memcached.  Linux J., 2004(124):5–, August 2004.
  * [18] Zerui Guo, Hua Zhang, Chenxingyu Zhao, Yuebin Bai, Michael Swift, and Ming Liu.  Leed: A low-power, fast persistent key-value store on smartnic jbofs.  In Proc. of ACM SIGCOMM, page 1012–1027, New York, NY, USA, 2023. Association for Computing Machinery.
  * [19] Matthias Jasny, Lasse Thostrup, Tobias Ziegler, and Carsten Binnig.  P4db - the case for in-network oltp.  In Proc. of ACM SIGMOD, page 1375–1389, 2022.
  * [20] Theo Jepsen, Alberto Lerner, Fernando Pedone, Robert Soulé, and Philippe Cudré-Mauroux.  In-network support for transaction triaging.  Proc. VLDB Endow., 14(9):1626–1639, may 2021.
  * [21] Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica.  Netchain: Scale-free sub-rtt coordination.  In Proc. of USENIX NSDI, pages 35–49, Renton, WA, April 2018.
  * [22] Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica.  Netcache: Balancing key-value stores with fast in-network caching.  In Proc. of ACM SOSP, pages 121–136, 2017.
  * [23] Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan.  Tea: Enabling state-intensive network functions on programmable switches.  In Proc. of ACM SIGCOMM, page 90–106, 2020.
  * [24] Gyuyeong Kim.  Holistic in-network acceleration for heavy-tailed storage workloads.  IEEE Access, 11:77416–77428, 2023.
  * [25] Gyuyeong Kim.  Netclone: Fast, scalable, and dynamic request cloning for microsecond-scale rpcs.  In Proc. of ACM SIGCOMM, page 195–207, September 2023.
  * [26] Gyuyeong Kim and Wonjun Lee.  In-network leaderless replication for distributed data stores.  Proc. VLDB Endow., 15(7):1337–1349, Mar. 2022.
  * [27] Baptiste Lepers, Oana Balmau, Karan Gupta, and Willy Zwaenepoel.  Kvell: the design and implementation of a fast persistent key-value store.  In Proc. of ACM SOSP, page 447–461, New York, NY, USA, 2019. Association for Computing Machinery.
  * [28] Jialin Li, Jacob Nelson, Ellis Michael, Xin Jin, and Dan R. K. Ports.  Pegasus: Tolerating skewed workloads in distributed storage with in-network coherence directories.  In Proc. of USENIX OSDI, pages 387–406, November 2020.
  * [29] Junru Li, Youyou Lu, Yiming Zhang, Qing Wang, Zhuo Cheng, Keji Huang, and Jiwu Shu.  Switchtx: Scalable in-network coordination for distributed transaction processing.  Proc. VLDB Endow., 15(11):2881–2894, jul 2022.
  * [30] Xiaozhou Li, Raghav Sethi, Michael Kaminsky, David G. Andersen, and Michael J. Freedman.  Be fast, cheap and in control with switchkv.  In Proc. of USENIX NSDI, pages 31–44, USA, 2016.
  * [31] Hyeontaek Lim, Dongsu Han, David G. Andersen, and Michael Kaminsky.  Mica: A holistic approach to fast in-memory key-value storage.  In Proc, of USENIX NSDI, page 429–444, USA, 2014.
  * [32] Ming Liu, Liang Luo, Jacob Nelson, Luis Ceze, Arvind Krishnamurthy, and Kishore Atreya.  Incbricks: Toward in-network computation with an in-network cache.  In Proc. of ACM ASPLOS, page 795–809, New York, NY, USA, 2017.
  * [33] Zaoxing Liu, Zhihao Bai, Zhenming Liu, Xiaozhou Li, Changhoon Kim, Vladimir Braverman, Xin Jin, and Ion Stoica.  Distcache: Provable load balancing for large-scale storage systems with distributed caching.  In Proc. of USENIX FAST, pages 143–157, Boston, MA, February 2019.
  * [34] Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, David Stafford, Tony Tung, and Venkateshwaran Venkataramani.  Scaling memcache at facebook.  In Proc. of USENIX NSDI, pages 385–398, Berkeley, CA, USA, 2013.
  * [35] Amedeo Sapio, Marco Canini, Chen-Yu Ho, Jacob Nelson, Panos Kalnis, Changhoon Kim, Arvind Krishnamurthy, Masoud Moshref, Dan Ports, and Peter Richtarik.  Scaling distributed machine learning with In-Network aggregation.  In Proc. of USENIX NSDI, pages 785–808, April 2021.
  * [36] Siyuan Sheng, Huancheng Puyang, Qun Huang, Lu Tang, and Patrick P. C. Lee.  FarReach: Write-back caching in programmable switches.  In Proc. of USENIX ATC, pages 571–584, Boston, MA, July 2023. USENIX Association.
  * [37] Vishal Shrivastav.  Stateful multi-pipelined programmable switches.  In Proc. of ACM SIGCOMM, page 663–676, New York, NY, USA, 2022. Association for Computing Machinery.
  * [38] Juncheng Yang, Yao Yue, and K. V. Rashmi.  A large scale analysis of hundreds of in-memory cache clusters at twitter.  In Proc. of USENIX OSDI, pages 191–208. USENIX Association, November 2020.
  * [39] Zhuolong Yu, Yiwen Zhang, Vladimir Braverman, Mosharaf Chowdhury, and Xin Jin.  Netlock: Fast, centralized lock management using programmable switches.  In Proc. of ACM SIGCOMM, page 126–138, 2020.
  * [40] Hang Zhu, Zhihao Bai, Jialin Li, Ellis Michael, Dan R. K. Ports, Ion Stoica, and Xin Jin.  Harmonia: Near-linear scalability for replicated storage with in-network conflict detection.  Proc. VLDB Endow., 13(3):376–389, November 2019.