# fpgaDDS: An Intra-FPGA Data Distribution Service for ROS 2 Robotics
Applications

Christian Lienen1, Sorel Horst Middeke1, and Marco Platzner1 1Christian
Lienen, Sorel Horst Middeke, and Marco Platzner are with the Department of
Computer Science, Paderborn University, 33098 DE Paderborn, Germany
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Modern computing platforms for robotics applications comprise a set of
heterogeneous elements, e.g., multi-core CPUs, embedded GPUs, and FPGAs. FPGAs
are reprogrammable hardware devices that allow for fast and energy-efficient
computation of many relevant tasks in robotics. ROS is the de-facto
programming standard for robotics and decomposes an application into a set of
communicating nodes. ReconROS is a previous approach that can map complete ROS
nodes into hardware for acceleration. Since ReconROS relies on standard ROS
communication layers, exchanging data between hardware-mapped nodes can lead
to a performance bottleneck.

This paper presents fpgaDDS, a lean data distribution service for hardware-
mapped ROS 2 nodes. fpgaDDS relies on a customized and statically generated
streaming-based communication architecture. We detail this communication
architecture with its components and outline its benefits. We evaluate fpgaDDS
on a test example and a larger autonomous vehicle case study. Compared to a
ROS 2 application in software, we achieve speedups of up to 13.34 and reduce
jitter by two orders of magnitude.

##  I Introduction

The computation demand of robotics applications has increased in the last few
years because modern robots gather more and more sensory information about
their environment and employ sophisticated algorithms for decision-making. The
rapid adoption of advanced machine learning for robotics additionally
accelerates this trend.

Modern robotics hardware architectures comprise a heterogeneous set of
computing elements to provide the needed performance in an energy-efficient
way. Computing elements include multi-core CPUs, embedded general-purpose
GPUs, and FPGAs. FPGAs are reprogrammable hardware devices that are flexible
since they can provide efficient acceleration for a wide range of workloads.

The Robot Operating System (ROS) is the de-facto standard for robotics
applications. Following the design philosophy of ROS, a robotics application
is decomposed into nodes, which implement sub-functions of the overall
application. Such functional decomposition facilitates easier development,
maintenance, and increased reusability of code. ROS nodes can leverage
different types of communication for data exchange, e.g., many-to-many
publish-subscribe communication where one or more nodes publish data on
specific topics, and other nodes that subscribe to these topics receive the
data.

Recently, the integration of FPGA-based hardware accelerators into ROS-based
applications has been proposed. The presented approaches either integrate
hardware accelerator kernels into ROS nodes and trigger kernel execution
similar to remote procedure calls [1, 2] or move complete nodes into the
hardware domain [3].

The latter approach is rather flexible and uses a standard ROS communication
layer for data transfer. This, however, can induce significant overhead
because all data passes shared main memory. Even for directly communicating
ROS nodes in hardware, the data is transferred via main memory under the
control of the software in the ROS communication layer.

The novel contribution of this paper is the presentation of fpgaDDS, a lean
data distribution service for ROS nodes that are mapped to hardware. fpgaDDS
is based on a customized streaming communication architecture and integrates
into the ReconROS architecture and programming model [3]. The presented
approach leads to significant improvements in performance and reduction of
jitter when executing ROS applications while at the same time maintaining the
programming philosophy and functional decomposition model of ROS.

The remainder of the paper is structured as follows: Section II provides
background, especially about ROS and hardware acceleration approaches for ROS.
Section III elaborates on the novel intra-FPGA data distribution service
fpgaDDS. Section IV reports on the experimental evaluation of fpgaDDS based on
measurements. Section V concludes the paper and provides an outlook on feature
work.

##  II Background

In this section, we first introduce the Robot Operating System before we
review hardware acceleration in ROS 2-based applications and ReconROS in more
detail.

###  II-A Robot Operating System

The Robot Operating System (ROS) comprises a set of software libraries, tools,
and a methodology for the design of robotics applications. Following the
design methodology of ROS, the overall application is decomposed into so-
called ROS nodes. ROS nodes use different communication types to interact and
exchange data. A topic-based publish-subscribe mechanism supports many-to-many
communication in ROS. One-to-one communication follows a remote-procedure-call
scheme leveraged by ROS services and actions. ROS applications are typically
described by their computation graph, which comprises the ROS nodes and their
communication as edges.

ROS 2 uses a message infrastructure for consistent information exchange
between nodes. Messages are hierarchically specified using a simple definition
language, starting from simple messages of built-in datatypes such as integers
or floating point numbers up to complex multi-level message structures. Per
default, the standard installation of ROS 2 provides a set of predefined
messages and the option to define custom user-specified message types.

ROS 2 leverages standard communication middleware implementations following
the Data Distribution Service (DDS) standard. Since DDS implementations differ
slightly, the ROS middleware layer (rmw) provides individual adapters for each
DDS implementation. In addition, the middleware layer provides communication
interfaces for the ROS client library, which summarizes common client
functionality [4]. Figure 1 shows the overall communication stack of ROS 2.

Figure 1: ROS 2 communication stack

There are middlewares for intra-network communication and intra-platform
communication. A prominent example of an intra-network communication
middleware is FastDDS [5] by eprosima. Iceoryx [6] is an example of an intra-
platform communication middleware that uses shared memory for data exchange.
Further, there are also mixed forms that optimize communication depending on
the destination. For instance, CycloneDDS [7] combines intra-network and
intra-platform communication and employs Iceoryx internally.

###  II-B Hardware Acceleration for ROS applications

In recent years, hardware accelerators for ROS-based applications have been
presented. Most works focus on ROS nodes, identify runtime-extensive parts and
move them to a hardware accelerator. At runtime, the remaining ROS node calls
the hardware accelerator like a remote-procedure call and possibly blocks
during accelerator execution. Examples of this approach are described in [1,
2], or are leveraged by industry products, e.g., Xilinx KRIA [8].

There is also research to accelerate communication. In [9], the authors reduce
communication overhead by separating control and data communication traffic
handled in hardware. Follow-up work [10] proposed to interpret ROS messages
with high-level hardware synthesis tool-flows, which reduces development time
and creates more compact code.

In [11], the authors present two approaches for reducing communication
overheads between nodes: kernel fusion and dedicated streaming queues. Kernel
fusion merges acceleration kernels of different ROS nodes into one combined
acceleration kernel. Dedicated streaming queues connect acceleration kernels
using data-transmission interfaces. Both approaches break with ROS’ node
decomposition methodology, i.e., the original ROS computational graph gets
transformed.

FPGA-ROS [12] allows for implementing ROS nodes and their communication
interfaces altogether in hardware. A streaming network connects application-
specific hardware cores to a shared central Ethernet gateway, which realizes
TCP/IP communication in hardware. A central manager instance supervises the
overall application.

###  II-C ReconROS

ReconROS [3] combines the reconfigurable hardware operating system ReconOS
[13] with ROS 2 and provides a framework for hardware acceleration of ROS 2
applications.

Figure 2: ReconROS Architecture (from [14])

Figure 2 shows the ReconROS architecture implemented on a platform FPGA
comprising a multi-core CPU and a reconfigurable fabric. ROS 2 nodes
implemented in hardware (HW nodes) are mapped to reconfigurable slots (RS),
which are rectangular areas on the reconfigurable fabric. HW nodes use a
memory interface (MEMIF) to access the shared virtual memory space, including
memory-mapped peripherals. Further, each HW node relies on an operating system
interface (OSIF) to connect to its delegate thread, a lightweight software
thread that performs operating system calls on behalf of the HW node.

ReconROS extends a guest operating system, e.g., Linux, by ReconOS and ROS 2
and allows for the execution of ROS 2 nodes in hardware and software through
the ReconROS stack and the ReconROS API. The ReconROS stack extends ReconOS by
ROS-related objects, e.g., ROS publishers or subscribers. The ReconROS API is
available for both SW and HW node implementations. It ensures a consistent
programming model that even allows for dynamically changing the mapping of a
ROS node to software or hardware or assigning a HW node to a specific
reconfigurable slot at runtime. In this paper, however, we only discuss static
mappings.

For instance, a HW node that wants to receive a new message from a ROS topic
sends the corresponding command via its OSIF to its delegate thread. The
delegate receives the command, relays it to the ReconROS stack, and waits in a
blocked state until lower ROS layers provide a new message. This wakes up the
delegate that then sends the address of the new message via the OSIF to the HW
node. Finally, the HW node can read the message using its MEMIF. ReconROS
builds on a shared memory architecture where all HW nodes share one access
port to the main memory. Hence, the HW nodes share the bandwidth for the
memory port, and together with the multi-core CPU, they share the total
bandwidth to the main memory.

The ReconROS build flow starts from a configuration file that captures the
information about the target platform, synthesis toolchains, ROS version,
number of reconfigurable slots, the HW nodes with their ROS-specific
resources, e.g., subscribers and publishers, and the mapping of HW nodes to
reconfigurable slots and generates all required software binaries and the FPGA
configuration bitstream. ReconROS supports the design of HW nodes using either
the hardware description languages VHDL and Verilog or high-level synthesis
from C/C++.

##  III Intra-FPGA Data Distribution Service

The ReconROS framework uses ROS 2 standard communication middlewares for data
transport between nodes, including HW nodes. Although this excels in
flexibility, as HW nodes can communicate with arbitrary SW and HW nodes, the
overall application performance may be suboptimal due to the HW nodes’
mechanism of calling ROS functions via their delegate threads and due to
competing memory accesses of all HW nodes.

This paper presents fpgaDDS, a novel and lean intra-FPGA data distribution
service for ReconROS applications. fpgaDDS maps ROS communication between HW
nodes to the reconfigurable fabric and thus avoids many memory accesses while
maintaining the ROS programming model for standard ROS 2 publish-subscribe
communication, i.e., no changes to the ROS nodes are required. Figure 1
sketches how fpgaDDS and the corresponding ReconROS DDS adapter integrates
into the ROS 2 communication stack.

The new intra-FPGA DDS has two benefits: First, mapping the data transport
between communicating HW nodes to the reconfigurable fabric saves many memory
accesses and thus improves application performance. Second, moving nodes and
communication from software to parallel executing hardware further reduces
application execution time and jitter, which helps achieve predictable real-
time behavior under ROS 2.

Figure 3: Extensions of the ReconROS communication stack include fpgaDDS and
the corresponding ReconROS DDS adapter

###  III-A Intra-FPGA Communication Architecture

The publish-subscribe communication principle of ROS 2 inspires the structural
design of the fpgaDDS communication architecture. The fpgaDDS-extended
ReconROS build flow generates an application-specific static AXI-streaming
(AXIS) network for each ROS topic separately. Such AXIS-based hardware-mapped
topics (HMT) are lean, resulting in relatively simple communication protocols
executed during runtime and high-performance implementations due to the total
available bandwidth per HMT.

Figure 4: Example for an instance of the communication architecture with two
hardware-mapped topics A and B

Figure 4 shows a fpgaDDS example comprising six ReconROS HW nodes connected to
two HMTs, A and B, by AXIS. In the figure, a directed connection from a HW
node to a HMT represents a publisher, and a connection from a HMT to a HW node
is a subscription.

The internal design of a HMT depends on the required number of publishers and
subscribers. In the simplest case, where the HMT receives data from one
publisher and provides that data to one subscriber, the HMT includes a simple
AXIS connection from input to output. In case there are more publishers for an
HMT, the inputs are gathered by an AXI Interconnect IP block which arbitrates
the input messages. The AXI-interconnect is configured to implement an
arbitration based on complete messages, ensuring consistency in the sense that
always complete messages are forwarded. If there are more HMT subscribers, the
message is broadcast to all subscribers by an AXI Broadcast IP block.
Naturally, AXI Interconnect and Broadcast IP blocks can be concatenated to
realize HMTs with arbitrary numbers of publishers and subscribers.

Figure 4 also indicates a FIFO buffer for the subscription of topic A by HW
node 4. Such FIFO buffers are optional components for subscribers of HMTs. By
adding FIFO buffers, the data processing of the subscriber nodes can be
decoupled from the topic. This behavior can be helpful for asynchronous
communication, for example, where the subscriber reads the data from the topic
at a different time.

In ROS 2, DDS layers allow developers to specify various Quality-of-Service
(QoS) parameters. fpgaDDS, with its AXIS streaming architecture, realizes, by
default, publish-subscribe communication with the QoS parameters Keep All and
Reliable, where no messages are discarded. The number of stored messages
depends on the size of the FIFO. If the FIFO runs full, transmission blocks.
In addition, due to its static DDS structure, fpgaDDS provides infinite
Lifespan Duration and infinite Lease Duration.

The communication architecture for the fpgaDDS is synthesized based on the
ReconROS project configuration file during design time. First, all subscribers
and publishers with a hardware property attribute are identified and grouped
for their topics. Then, for each HMT, the ReconROS build flow extends the HW
nodes by input (subscription) and output (publishing) ports, inserts the HMT
AXIS infrastructure, and connects the ports and the infrastructure
accordingly. The overall procedure corresponds to the discovery procedure in
standard DDS implementations.

###  III-B ReconROS DDS Adapter

The ReconROS DDS adapter aims to close the gap between the programming model
for streaming networks at the fpgaDDS layer and the publish/subscribe
communication mechanism in ReconROS. As a result, the programmer can use
similar blocking and non-blocking functions for interaction with software-
mapped and hardware-mapped topics.

In order to maintain compatibility between standard message structures
provided by ROS 2 and the message structures used in our streaming fpgaDDS
communication architecture, which is generated by a high-level synthesis tool
flow, we need to serialize message objects on the transmitter side and de-
serialize them on the receiver side.

Due to the multi-layer architecture of ROS 2 and the support of several
different communication middlewares, the ROS 2 framework already provides
functionality for ROS message serialization in software, even for complex
nested message types. However, these functions can not be directly used for
hardware generation since Vivado HLS provides limited support for pointer
arithmetic, casting, and recursion.

Therefore, we extract the structure of messages, i.e., their components and
data types, from the message definition file and replace all non-primitives
with their sub-components. This also allows for resolving multi-level nested
messages. The resulting list of primitives and arrays is then transformed into
a C macro for message publishing and subscribing. During hardware synthesis,
these macros are in-lined and generate hardware to write and read data
sequentially to and from AXI streaming interfaces in blocking and non-blocking
versions.

###  III-C Execution Modes

Communicating HW nodes can be operated in two execution modes with fpgaDDS.
The first mode is the same as when using a software-based DDS and operates the
receive, compute, and send phases sequentially. Figure 5-a shows an example of
a chain of three nodes with publish-subscribe communication.

With fpgaDDS, we can additionally leverage the dataflow option of the high-
level synthesis tool to create an implementation where the phases are
overlapped as much as possible, constrained only by the data flow. This
operation mode is exemplified in Figure 5-b and can substantially improve
overall execution time. This mode is useful for sets of nodes that operate on
data in a streaming manner, which is typical, for example, for many low-level
image processing tasks.

Figure 5: Execution modes for HW nodes with communication ($t_{com})$ and
executio ($t_{exec})$ phases: (a) sequential execution and (b) dataflow
execution

##  IV Evaluation

This section reports on experiments with fpgaDDS. First, we describe the
experimental setup, followed by experiments with HW nodes to demonstrate the
benefit of using fpgaDDS. Finally, we present measurements on a real-world
application of autonomous driving.

###  IV-A Experimental Setup

All HW nodes have been implemented in C++ and synthesized to hardware with the
high-level synthesis tool Xilinx Vitis 2021.2. The computing platform is a
ZCU104 evaluation board comprising an UltraScale+ MPSoC FPGA running Ubuntu
Linux 18.04, ReconROS, and ROS 2 dashing. The ReconROS infrastructure and the
HW nodes run at 100 MHz. For the autonomous driving example, we additionally
use a desktop PC with an Intel Core i5-8000 CPU running Ubuntu Linux 18.04 and
ROS 2 dashing, connected to the FPGA evaluation board via Gigabit Ethernet.

###  IV-B Hardware-Mapped Topic Evaluation

In the first experiment, we implemented two HW nodes connected by a topic and
measured the time it took from the start of sending to the end of receiving.
Data transfers have been repeated 1000 times, and the average values are
reported. Table I presents the resulting transfer times $t_{avg}$ and their
standard deviation $\sigma$ for HW nodes using the software-mapped CycloneDDS
and the hardware-mapped fpgaDDS for different message sizes. The table also
lists the speedups achieved by fpgaDDS. The speedups show that the overhead
for using CycloneDDS dominates for small data sizes. However, the speedup
converges close to $2$ for larger data sizes because data has to be
transferred twice over the MEMIF compared to one needed transmission via the
HMT. However, the measurements for CycloneDDS represent the most optimistic
case. With higher utilization of the MEMIF, the transmission time increases as
the nodes share MEMIFs bandwidth.

Increasing the number of publishers and subscribers increases the transfer
time for CycloneDDS. For fpgaDDS, the transfer time only increases with more
publishers since the HMT arbitrates the incoming messages. However, adding
more subscribers in fpgaDDS does not influence the transfer time.

Message Size |  CycloneDDS $t_{avg}$ $(\sigma)$ [ms] |  fpgaDDS $t_{avg}$ $(\sigma)$ [ms] |  Speedup
---|---|---|---
3 kB | 0.06 (0.11) |  $<$0.01 ($<$0.01) | 17.21
12 kB | 0.07 (0.06) | 0.02 ($<$0.01) | 4.71
50 kB | 0.19 (0.08) | 0.06 ($<$0.01) | 3.11
196 kB | 0.63 (0.04) | 0.24 ($<$0.01) | 2.54
786 kB | 2.34 (0.03) | 0.98 ($<$0.01) | 2.38
3146 kB | 9.24 (0.04) | 3.93 ($<$0.01) | 2.35
TABLE I: Communication times and speedup for CycloneDDS and fpgaDDS for
different message sizes

###  IV-C Autonomous Vehicle Example

Figure 6: Computation graph (a) and the resulting architecture graph (b) for
the autonomous vehicle example

In this experiment, we elaborate and extend the autonomous driving application
from [14]. This application is inspired by the Turtlebot auto race challenge,
follows a street lane, and handles traffic lights during the drive. The target
robot is the Turtlebot 3 platform comprising a 0.3-megapixel camera for lane
and traffic light detection. In our setup, the resulting ReconROS application
runs on the FPGA evaluation board, connected to the desktop PC executing a
Gazebo physics simulation environment.

Figure 6-a displays the computational graph for the application. The only
input of the computation graph is the camera image of the robot. The Image
Compensation node calculates a histogram of the input image and uses it to
remove outlier pixels. The resulting image is then published to Topic A. The
Gaussian Blur node subscribes to the image data, applies a low-pass filter,
and forwards the result to the image projection node through Topic B. The
Image Projection node applies a warp transformation for an orthogonal view of
the lane, then resizes the image to 1000x600 pixels. Finally, the Lane
Following node extracts the lane’s left and right stripes from image data and
calculates the center point between them. This center point is sent to the
node Lane Control that calculates driving commands from the center point.

In addition to the Gaussian Blur node, the Red Traffic Light Detection and the
Green Traffic Light nodes subscribe to Topic A. Both nodes transform the image
into the HSV colorspace and then detect a red or green traffic light. If a red
traffic light is detected, the node publishes a stop command to Topic E, which
leads to a stop of the robot by the lane control node. Additionally, that
command activates green traffic light detection. When a green light is
detected, the node publishes a start command to Topic F, which activates the
lane control node again.

We have implemented the ROS 2 application of Figure 6-a in three different
versions: (i) Software, where nodes run on the CPU based on CycloneDDS for
communication, (ii) hardware, where all nodes are implemented using high-level
synthesis but still use CycloneDDS for communication, and (iii) hardware,
where the nodes use our novel fpgaDDS. The architecture graph for the third
version is sketched in Figure 6-b. In addition, the subscribers of HMT Topic E
and F are equipped with buffers to enable asynchronous communication via these
HMTs.

The software implementation does not use any reconfigurable logic resources.
Table II summarizes the resource utilization of the reconfigurable fabric for
the hardware implementations. The table lists the resource types lookup-tables
(CLB LUTs), dedicated memory blocks (BRAM, URAM), and arithmetic function
blocks (DSP). The main insight from Table II is that fpgaDDS does not require
more logic resources than the version with CycloneDDS. This is because the
AXIS-based communication architecture requires relatively few logic resources.
CycloneDDS, on the other hand, demands substantially more CLB LUTs for
implementing in-node buffers.

Implementation | CLB LUTs |  BRAM |  URAM |  DSPs
---|---|---|---|---
Hardware (CycloneDDS) |  134252 (58%) |  250.5 (80%) |  95 (99%) |  230 (13%)
Hardware (fpgaDDS) |  44939 (20%) |  255.0 (82%) |  91 (95%) |  230 (13%)
TABLE II: Resource utilization of the hardware implementations (% of the used
XCZU7EV-2FFVC1156)

To evaluate the performance of the different communication middlewares, we
have measured the execution times of the node chain $\alpha$ indicated as a
red dashed line in Figure 6. The node chain starts with the image compensation
node and ends with the lane control node. We have selected this chain of nodes
since it subsumes most nodes regularly executed in the lane-following mode of
the vehicle and feeds the topics periodically with message data. In contrast,
topics E and F receive data only in the event of detected red or green traffic
lights.

We have determined the execution of time node chain $\alpha$ by calculating
the difference between the publishing time of the lane control node and the
subscription time of the image compensation node. Again, we repeat the
experiments 1000 times and report on the average and the standard deviation in
Table III. Additionally, the table presents the speedups achieved over a pure
software implementation. For example, implementing the ROS nodes in hardware
but keeping the software-based CycloneDDS results in a speedup of 2.48.
Mapping also communication to hardware with fpgaDDS gives a speedup of 13.34.
Furthermore, with fpgaDDS, the standard deviation, which relates to jitter,
reduces by two orders of magnitude.

Implementation |  $t_{avg}$ ($\sigma$) [ms] | Speedup
---|---|---
Software (CycloneDDS) | 274.44 (14.96) | 1.00
Hardware (CycloneDDS) | 110.72 (07.73) | 2.48
Hardware (fpgaDDS) | 20.58 (00.13) | 13.34
TABLE III: Execution times and speedups for node chain $\alpha$ of Figure 6
and different implementation versions

##  V Conclusion and Future Work

This paper has presented fpgaDDS, a lean data distribution service for ROS
nodes mapped to hardware. fpgaDDS is a statically generated, customized
streaming communication architecture integrated into ReconROS that ensures low
latency and high bandwidth communication between hardware nodes. Experiments
have demonstrated significantly improved performance and reduced jitter.

Future work is planned along two lines: First, we aim to extend fpgaDDS for
ROS 2 services and actions and support a richer set of Quality-of-Service
parameters. Second, we want to make available fpgaDDS for dynamically
scheduled ROS 2 nodes. ROS 2 foresees a so-called executor that dispatches
callback functions on specific events for dynamic scheduling. On FPGA
platforms, dynamic scheduling is supported by partial dynamic reconfiguration.
Combining such dynamic techniques with fpgaDDS poses challenges since fpgaDDS
is currently statically generated.

## References

  * [1] K. Yamashina, H. Kimura, T. Ohkawa, K. Ootsu, and T. Yokota, “CReComp: Automated Design Tool for ROS-Compliant FPGA Component,” in _Proc. IEEE 10th International Symposium on Embedded Multicore/Many-Core Systems-on-Chip, MCSoC 2016_. IEEE, 2016, pp. 138–145.
  * [2] M. Eisoldt, S. Hinderink, M. Tassemeier, M. Flottmann, J. Vana, T. Wiemann, J. Gaal, M. Rothmann, and M. Porrmann, “ReconfROS: Running ROS on Reconfigurable SoCs,” in _Proc. 2021 Drone Systems Engineering and Rapid Simulation and Performance Evaluation: Methods and Tools Proceedings_. ACM, 2021, p. 16–21.
  * [3] C. Lienen and M. Platzner, “Design of Distributed Reconfigurable Robotics Systems with ReconROS,” _ACM Trans. Reconfigurable Technol. Syst._ , vol. 15, no. 3, dec 2022.
  * [4] S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” _Science Robotics_ , vol. 7, no. 66, 2022.
  * [5] “eprosima fast dds,” https://github.com/eProsima/Fast-DDS, accessed: 2023-02-28.
  * [6] “iceoryx - true zero-copy inter-process-communication,” https://github.com/eclipse-iceoryx/iceoryx, accessed: 2023-02-28.
  * [7] “Eclipse cyclone dds,” https://github.com/eclipse-cyclonedds/cyclonedds, accessed: 2023-02-28.
  * [8] V. Mayoral-Vilches, “Kria Robotics Stack,” https://www.xilinx.com/applications/industrial/robotics/wp540-kria-robotics-stack.html, 2021, accessed: 2022-01-13.
  * [9] Y. Sugata, T. Ohkawa, K. Ootsu, and T. Yokota, “Acceleration of Publish/Subscribe Messaging in ROS-Compliant FPGA Component,” in _Proc. of the 8th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies (HEART2017)_. ACM, 2017.
  * [10] T. Ohkawa, Y. Sugata, H. Watanabe, N. Ogura, K. Ootsu, and T. Yokota, “High Level Synthesis of ROS Protocol Interpretation and Communication Circuit for FPGA,” in _Proc. 2019 IEEE/ACM 2nd International Workshop on Robotics Software Engineering (RoSE)_ , 2019, pp. 33–36.
  * [11] V. Mayoral-Vilches, S. M. Neuman, B. Plancher, and V. J. Reddi, “Robotcore: An open architecture for hardware acceleration in ros 2,” in _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2022, pp. 9692–9699.
  * [12] A. Podlubne and D. Göhringer, “FPGA-ROS: Methodology to Augment the Robot Operating System with FPGA Designs,” in _Proc. International Conference on ReConFigurable Computing and FPGAs (ReConFig)_ , 2019.
  * [13] E. Lübbers and M. Platzner, “ReconOS: Multithreaded Programming for Reconfigurable Computers,” _ACM Transactions on Embedded Computing Systems_ , vol. 9, no. 1, pp. 8:1–8:33, 2009.
  * [14] C. Lienen and M. Platzner, “Task mapping for hardware-accelerated robotics applications using reconros,” in _2022 Sixth IEEE International Conference on Robotic Computing (IRC)_ , 2022, pp. 148–155.