# RobotCore: An Open Architecture for Hardware Acceleration in ROS 2

Víctor Mayoral-Vilches1,2,3, Sabrina M. Neuman4, Brian Plancher4, Vijay Janapa
Reddi4 This material is based upon work funded by Xilinx and supported by the
National Science Foundation under Grant 2030859 to the Computing Research
Association for the CIFellows Project. Any opinions, findings, conclusions, or
recommendations expressed in this material are those of the authors and may
not reflect those of the funding organizations.1Víctor Mayoral-Vilches is with
Acceleration Robotics, Ecuador 3, 1 I, Vitoria, Álava, Spain
<EMAIL_ADDRESS>Mayoral-Vilches is with the System
Security Group, Universität Klagenfurt, Universitätsstr. 65-67 9020
Klagenfurt, Austria<EMAIL_ADDRESS>Mayoral-Vilches is with Alias
Robotics, Venta de la Estrella 6, pab 130, Vitoria 01006, Spain
<EMAIL_ADDRESS>M. Neuman, Brian Plancher, and Vijay Janapa
Reddi are with the John A. Paulson School of Engineering and Applied Sciences,
Harvard University, Cambridge, MA, USA<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Hardware acceleration can revolutionize robotics, enabling new applications by
speeding up robot response times while remaining power-efficient. However, the
diversity of acceleration options makes it difficult for roboticists to easily
deploy accelerated systems without expertise in each specific hardware
platform. In this work, we address this challenge with RobotCore, an
architecture to integrate hardware acceleration in the widely-used ROS 2
robotics software framework. This architecture is target-agnostic (supports
edge, workstation, data center, or cloud targets) and accelerator-agnostic
(supports both FPGAs and GPUs). It builds on top of the common ROS 2 build
system and tools and is easily portable across different research and
commercial solutions through a new firmware layer. We also leverage the Linux
Tracing Toolkit next generation (LTTng) for low-overhead real-time tracing and
benchmarking. To demonstrate the acceleration enabled by this architecture, we
use it to deploy a ROS 2 perception computational graph on a CPU and FPGA.

We employ our integrated tracing and benchmarking to analyze bottlenecks,
uncovering insights that guide us to improve FPGA communication efficiency. In
particular, we design an intra-FPGA ROS 2 node communication queue to enable
faster data flows, and use it in conjunction with FPGA-accelerated nodes to
achieve a $24.42\%$ speedup over a CPU.

##  I Introduction

Recent work has seen an explosion of specialized robotics acceleration on
nontraditional computing platforms such as GPUs, FPGAs, and ASICs [1, 2, 3, 4,
5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21]. This has been
sparked by the decline of Moore’s Law and Dennard Scaling, which limits the
performance of traditional CPU computing, positioning hardware acceleration as
an emerging solution to achieve high performance and power efficiency in
robotics applications.

However, this increased diversity of computing platforms leads to a dramatic
growth in design space complexity that makes it difficult for users to easily
deploy robotics applications on hardware accelerators without substantial
expertise in each specific accelerator platform. The Open Computing Language
(OpenCL) [22] is an effort to standardize hardware acceleration under a common
language, but its adoption across silicon vendors has been uneven and support
for it varies. As a result, current hardware acceleration usage is often tied
to a particular vendor’s solutions and platforms. This not only impedes
interoperability and reuse of acceleration kernels, but presents yet another
layer of complexity that users must overcome while implementing robot systems.

Figure 1: The open architecture for hardware acceleration in ROS 2 extends the
ROS 2 build system to seamlessly support vendor and platform-agnostic
deployment of robotics applications on accelerator hardware. The integrated
tracing and benchmarking infrastructure enables users to analyze the system
and make strategic design improvements to optimize performance.

To address this challenge, in this work we present _RobotCore_ , an _open
architecture for hardware acceleration_ that extends the Robot Operating
System (ROS) [23], the _de facto_ standard for robot application development.
ROS is widely used by academia and industry, and early work has demonstrated
its potential for hardware-accelerated robotics applications [2, 3, 24]. We
facilitate this emerging direction by implementing a vendor and platform-
agnostic abstraction layer for hardware acceleration in robotics (Fig. 1).
Starting with a popular robotics API as the foundation, our ROS 2-based
acceleration architecture provides a common ground for both academic
researchers and silicon vendors alike to develop specialized robotics
acceleration kernels, and deploy them for easy usage by a large, established
user base.

Once roboticists can easily harness hardware acceleration across multiple
platforms, the next major challenge is profiling and benchmarking the
application. Benchmarking is needed to determine the best mapping of the
robotics computational graph to the different hardware resources available to
optimize overall robot system performance. This is a difficult task, however,
since every application is different and deployment scenarios are widespread.
Full end-to-end system analysis is required to understand how different
implementation tradeoffs impact overall performance. To enable this analysis,
we demonstrate how to leverage prior work [25] and benchmark our ROS hardware
acceleration framework with a low-overhead framework for real-time tracing
based on the Linux Tracing Toolkit next generation (LTTng) [26]. We
demonstrate analysis of a case study deployment using CPU and FPGA nodes for a
simple perception pipeline.

Using our framework and benchmarking, we diagnose that substantial latency
bottlenecks in this computational graph come from inter-node interactions
across ROS 2 layers in the CPU. We recognize this as an opportunity for design
optimization in hardware accelerators, because interaction with the CPU should
not be necessary for dataflow between nodes co-located on the same non-CPU
platform (e.g., FPGA).

Based on the benchmarking analysis, we demonstrate two novel separate paths
toward _hardware acceleration_ : (1) kernel fusion, and (2) improved message
passing. Kernel fusion results in the highest speedup, an average of
$26.96\%$, but it requires manual redesign of the underlying kernels. To avoid
manual redesign entirely and improve design re-use and portability, we
alternatively develop an intra-FPGA ROS 2 node communication queue that
leverages AXI4-Stream interfaces [27] and transfers data in a sequential
streaming manner directly between acceleration kernels. This improves the
overall inter-node performance in our computational graph by $24.42~{}\%$ on
average, while requiring no change in the accelerated kernels. This design
extends to applications beyond our case study, since this same pattern is
valid for any ROS 2 inter- or intra-process communication, and aligns with ROS
2 composition capabilities.

In summary, key contributions of this work are that we:

  * •

Create a new open infrastructure to increase the performance of robotics
applications by enabling _seamless integration of hardware acceleration_ into
ROS 2 that is _portable across accelerator platforms_ (e.g., FPGAs, GPUs) and
system deployments (e.g., edge devices, workstations, data centers, and
cloud);

  * •

Expose insights into how to optimize overall system-wide performance by
introducing a _low-overhead tracing and benchmarking framework_ to analyze
application performance across ROS 2 computational graphs, laying foundation
to analyze mixed-platform systems (e.g., combinations of CPU and FPGA-based
nodes); and

  * •

Increase ROS 2 node-to-node dataflow performance to achieve an average overall
accelerator speedup of $24.42\%$ over CPU in our experiments by designing
_intra-FPGA ROS 2 node communication queues_ , based on insights uncovered
using our open acceleration infrastructure and low-overhead benchmarking on a
case study analysis of a simple perception graph.

The core components of our architecture are disclosed under a commercially
friendly open-source license and are available and maintained at the ROS 2
Hardware Acceleration Working Group GitHub organization:
https://github.com/ros-acceleration.

##  II Background and Related Work

###  II-A ROS and ROS 2

The Robot Operating System (ROS) is an open-source collection of software
frameworks and tools designed to provide a _structured communications layer_
for robotics applications running on heterogenous computer hardware [28].

ROS applications are designed around event driven graphs of _Nodes_ which
communicate through _Messages_ on various _Topics_ , _Services_ , and
_Actions_. Each Node can be thought of as a software process which applies an
algorithm to the input message and then broadcasts the resulting output
message. By managing all inter-Node communications across abstraction layers
(e.g., rclcpp, rcl, rmw), ROS simplifies the robotic system deployment process
and enables roboticists to quickly develop and test new algorithms. ROS also
provides substantial infrastructure to facilitate the automatic building,
evaluation, and deployment of robotic systems, including dependency managers,
package managers, build systems and tools, simulators, and visualizers.

ROS 2 is a re-design of ROS that modernizes and updates all of its components
while adhering to its core design principles. ROS 2 provides a stronger
partitioning of the communication middleware from the robotics logic, enabling
more flexibility, scalability, and reliability [2]. ROS 2 also provides an
updated build system, ament, and a new universal build tool, colcon. This
provides a single simple interface for managing the building and deployment of
complete robotics applications. Leveraging these tools, roboticists can write
new algorithms and rely on ROS 2 to handle all lower level operations and
middleware management.

###  II-B Hardware Acceleration for ROS and ROS 2

There has been previous work that has focused on ways to accelerate robotics
applications by developing tools and methodologies to help roboticists
leverage hardware acceleration for selected ROS Nodes and to optimize the ROS
computational graph through adaptive computing [29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 43, 24]. There has also been some work to
accelerate the scheduling and communication layers used by ROS and ROS 2 [44,
45, 46, 47, 48, 49, 50, 51]. Unfortunately, the majority of these efforts
assume an end-user has substantial experience with embedded systems and
embedded hardware flows, or is customized to a specific hardware acceleration
board or deployment scenario.

Our proposed open architecture takes a ROS-centric approach to integrate the
hardware and embedded flows directly into the core ROS 2 ecosystem. This
provides end-users with a build and deployment experience for hardware
accelerators similar to the standard, non-accelerated ROS 2 experience.

##  III An Open Architecture for HW Acceleration

ROS 2 C++ stack Our open architecture for hardware acceleration
github.com/ros-acceleration  1 build system (ament) 1 ament_acceleration
1 ament_vitis
1 ament_jetpack
... 2 build tools (colcon) 2 colcon-acceleration
Xilinx Vitis Nvidia Jetpack 3 firmware 3 acceleration_firmware
3 acceleration_
firmware_kv260 3 acceleration_
firmware_jetson ... 4 tracing drivers libraries cloud image_pipeline
userland tooling rclcpp rcl rmw adapter middleware abcde$e_{2}$$e_{1}$ $
colcon build ---merge-install a b c d $ source install/setup.bash $ colcon
build ---merge-install ---mixin kv260
\---install-dir install-kv260
\---build-dir build-kv260 e $e_{1}$ $e_{2}$

Figure 2: An open architecture for hardware acceleration in ROS 2. Our
contributions are highlighted in green. The architecture include the following
extensions to ROS 2: 1 extensions to the build system (ament), 2 extensions to
the build tools (colcon), 3 adds a new firmware pillar to workspaces
simplifying the production and deployment of acceleration kernels and 4
introduces a low-overhead real-time tracing and benchmarking approach based on
the Linux Tracing Toolking next generation (LTTng [26]). A walkthrough on the
interaction amongst the ROS 2 packages in the open architecture is also
depicted. A first invocation to colcon build a will prepare the ROS 2
workspace for hardware acceleration and deploy files in the local overlay as
illustrated by b, c and d. After that, from the overlay, a second invocation
of colcon build with the \--mixin <target-board> flag e will cross-compile
$e_{1}$ and produce the accelerators $e_{2}$ as needed for the target
accelerator. From this point on, the colcon_acceleration package helps further
automate additional processes.

Our open architecture (Fig. 2) extends the core ROS 2 build system and tools
to provide platform-agnostic (i.e., supports edge, workstation, data center,
or cloud targets), technology-agnostic (i.e., supports FPGAs and GPUs), and
portable hardware-accelerated ROS 2 capabilities for roboticists. We: 1 extend
the ROS 2 build system, ament; 2 extend the ROS 2 meta build tool, colcon; and
3 develop integrated ROS 2 firmware extensions. We also 4 integrate a low-
overhead tracing and benchmarking framework to enable the analysis of holistic
application performance across ROS graphs. This section describes these
extensions in detail.

###  III-A Extending the ROS 2 Build System - 1

The first pillar of our open architecture allows roboticists to generate
acceleration kernels directly from the ROS 2 build system (ament) in the same
way they generate CPU binaries. To do so, the ament_acceleration ROS 2 package
and its extensions abstract the ROS build system from vendor-specific
accelerators (e.g. FPGAs or GPUs), including their frameworks and software
platforms. This allows the build system to easily support hardware
acceleration across commercial solutions while using the same syntax,
simplifying the work of ROS 2 package maintainers.

Under the hood, each hardware-specific extension of ament_acceleration
abstracts away the corresponding vendor-specific firmware. For example,
ament_vitis111github.com/ros-acceleration/ament_vitis relies on the
proprietary Xilinx Vitis [52] and on the Xilinx Runtime (XRT) library [53].
This simplifies the creation of acceleration kernels and separates firmware
concerns from algorithm development. This way, robotics engineers can focus on
improving their computational graphs with a ROS-centric development flow.
Separately, hardware experts, potentially sponsored by silicon vendors, can
improve acceleration kernels for a particular commercial solution. Overall,
these extensions help achieve the objective of simplifying the creation and
integration of acceleration kernels from different vendors into ROS 2
computational graphs.

Fig. 2 depicts the build system extensions showing how ament_acceleration
abstracts the build system from vendor-specific solutions. As an example of an
alternative acceleration technology supported, ament_jetpack is included and
illustrates the integration of Nvidia JetPack [54].

###  III-B Extending the ROS 2 Build Tools - 2

The second pillar of our open architecture extends the colcon ROS 2 meta build
tool to integrate hardware acceleration flows into the ROS 2 Command Line
Interface (CLI) commands. Examples of these extensions include the selection
of the target accelerator and build-time through mixins, emulation
capabilities to speed-up the development process and facilitate design without
access to the real hardware, raw disk image production tools, and simplified
configuration of hypervisors. These extensions are implemented by the colcon-
acceleration222github.com/ros-acceleration/colcon-acceleration ROS 2 package.
As in Section III-A, colcon_acceleration further enables roboticists to
leverage hardware accelerators while using the same commands and flows to
build their applications.

###  III-C Adding Firmware Extensions - 3

Represented by the abstract acceleration_firmware ROS package and its
corresponding specializations (e.g.
acceleration_firmware_kv260333github.com/ros-
acceleration/acceleration_firmware_kv260 for the Xilinx Kria KV260 board), the
third pillar of our open architecture, firmware extensions, are meant to
provide firmware artifacts for each supported technology solution. This again
simplifies the process for ROS package consumers and maintainers, and further
aligns hardware acceleration workflows with typical ROS development flows.
Each ROS 2 workspace can leverage multiple firmware packages, but can only use
one at a time. As colcon_acceleration supports the selection of the active
firmware in the ROS workspace, by separating the firmware out into their own
packages, our open architecture enables silicon vendors to maintain an
acceleration_firmware_<solution> ROS package that automatically integrates
into standard ROS 2 workflows.

###  III-D Low-Overhead Real-Time Tracing & Benchmarking - 4

In the context of hardware acceleration in robotics, it is fundamental to be
able to inspect performance improvements. To that end, it is important to
benchmark and trace the system. Benchmarking is the process of running a
computer program to assess its relative performance, whereas tracing is a
technique used to understand what is happening in a system while it is
running. Tracing helps determine which pieces of a Node are consuming more
compute cycles or generating indeterminism, and are thereby good candidates
for hardware acceleration. Benchmarking instead helps investigate the relative
performance of an acceleration kernel versus its CPU scalar computing
baseline. Similarly, benchmarking also helps with comparing acceleration
kernels across different hardware acceleration technology solutions (e.g.,
Kria KV260 vs. Jetson Nano) and across kernel implementations within the same
hardware acceleration technology solution.

In order to trace and evaluate the relative performance of both ROS 2
individual Nodes and complete computational graphs, we leverage Linux Tracing
Toolkit next generation (LTTng [26]) for tracing and benchmarking. Building
upon prior work [25], LTTng provides a collection of flexible tracing tools
and multipurpose instrumentation for ROS 2 that allow collecting runtime
execution information in real-time in distributed systems using low-overhead
tracers. Instrumentation probes introduced in ROS 2 Nodes can be enabled or
disabled at build-time. When enabling all ROS 2 instrumentation, end-to-end
message latency overhead is below $5.5$us [25], making it suitable for a wide
variety of hardware acceleration use cases. This infrastructure also lays a
foundation for future integration with platform-specific performance counters
and tracing tools that can extend analysis to more fine-grained introspection
and profiling of the kernels running onboard an accelerator device.

##  IV Accelerating ROS 2 Perception

Figure 3: Computational graph of our case study perception application,
image_pipeline, containing two ROS 2 nodes: (1) RectifyNode subscribes to the
/camera/image_raw and /camera/camera_info topics from Gazebo [55] and
publishes a _rectified image_ to (2) ResizeNode, which publishes the final
_resized image_.

For our case study, we trace, benchmark, and accelerate a subset of
image_pipeline [56], one of the most popular packages in the ROS 2 ecosystem,
and a core piece of the ROS perception stack. We compose a simple
computational graph consisting of two nodes, resize and rectify, as shown in
Fig. 3. We then leverage our open architecture for hardware acceleration
(Section III) to benchmark, trace and accelerate our computational graph,
comparing a CPU to an FPGA implementation. In this section we describe the
methodology of our approach, and analyze our timing results.

Figure 4: Tracepoints instrumented across ROS 2 abstraction layers on CPU for
case study computational graph (Fig. 3). Breakdown summary in Fig. 5.

###  IV-A Methodology

We propose the following steps to analyze a ROS 2 application and design
appropriate acceleration: (i) instrument both the core components of ROS 2 and
the target kernels; (ii) trace and benchmark the kernels on the CPU to
establish a baseline; (iii) develop a hardware accelerated implementation on
alternate hardware (e.g., GPU, FPGA); and (iv) trace, benchmark against the
CPU baseline, and improve the accelerated implementation.

Following this methodology, in our case study we begin by instrumenting both
ROS 2 and our target kernels with LTTng probes. Reusing past work and probes
[25] allows us to easily get a grasp of the dataflow interactions within rmw,
rcl, and rclcpp ROS 2 layers. We then also instrument the ResizeNode and
RectifyNode components of the image_pipeline package used in our case study.
The relevant tracepoints placed in our computational graph across ROS 2 stack
layers are listed in Fig. 4 and 5 (full list in Pull Request $717$ in the
image_pipeline repository [56]). On the CPU, these tracepoints enable us to
isolate the latency of computation within a node from the time it takes ROS 2
to package and pass information between nodes.

In the following sections we report timing results from using a Xilinx Kria®
KV260 Vision AI Starter Kit [57], which has an onboard integrated Quad-core
Arm® Cortex®-A53 CPU and an FPGA containing 256K System Logic Cells and 1.2K
DSP Slices. All benchmark results report the mean value obtained from a $60$
second continuous run of the computational graph. The FPGA kernels are
synthesized, placed and routed with a $250$MHz clock.

###  IV-B CPU-Only Tracing Results

Fig. 4 demonstrates the results of instrumenting and tracing our target
computational graph (Fig. 3) across multiple ROS 2 stack layers on the CPU,
and Fig. 5 summarizes the breakdown of timing results across operations,
establishing the CPU baseline for our application. The breakdown in Fig. 5
shows the time taken to do the computations within each node, as well as the
time taken by the ROS 2 lower-level message-passing system across the various
abstraction layers. We find that the message-passing overhead in our
application consumes more than $73.3\%$ of the total time and is therefore a
large bottleneck in the total computation time of the full graph. We next
explore FPGA hardware acceleration options, comparing performance to the CPU
baseline.

Figure 5: Breakdown of CPU runtime derived from tracing and benchmarking.
Total computation time of our case study graph is dominated by message passing
overheads, a bottleneck consuming over $73.3\%$ of total runtime.

###  IV-C Accelerating and Benchmarking CPU & FPGA

In this section, we explore hardware acceleration options for an FPGA for our
case study application (Fig. 3). In Section IV-C1, we first explore hardware
acceleration kernels for the core logic of each of the Nodes (_rectify_ and
_resize_), harnessing our open architecture for implementation. In Section
IV-C2, we then explore two different FPGA designs to accelerate the
computational graph by optimizing dataflow interactions between FPGA-based
nodes, addressing the ROS 2 _communication infrastructure_ performance
bottleneck revealed by the CPU baseline in Section IV-B.

####  IV-C1 Accelerating Nodes & Components on an FPGA

We first accelerate the computations at each one of the graph nodes. The
RectifyNode and ResizeNode _Components_ of Fig. 3 are accelerated using
Xilinx’s HLS, XRT, and OpenCL targeting the Kria KV260444github.com/ros-
acceleration/image_pipeline/blob/ros2/image_proc/src/
{rectify,resize}_fpga.cpp. Each ROS 2 _Component_ has an associated
acceleration kernel555github.com/ros-
acceleration/image_pipeline/tree/ros2/image_proc/src/ image_proc that
leverages the Vitis Vision Library, a computer vision library optimized for
Xilinx silicon solutions and based on OpenCV APIs. These accelerated
_Components_ and their kernels seamlessly integrate with the rest of the ROS
meta-package through our open architecture (Fig. 2), and are openly available
to the public. Building the accelerators is abstracted away from roboticist
end-users, and takes no significant additional effort than the standard build
of the image_pipeline.

After benchmarking the accelerated _Components_ using the trace points of
Section IV-B, we observe an average $6.22\%$ speedup in the total computation
time of the perception pipeline when offloading tasks to the FPGA (see Fig.
6). For this case study example, it is not surprising that accelerating the
computational nodes and components alone only gives a modest performance
increase because, as we saw in Section IV-B, the performance bottleneck in the
baseline CPU system was communication overhead, not computation.

####  IV-C2 Accelerating the Computational Graph on an FPGA

In our case study application, message-passing overheads across the ROS 2
abstraction layers far outweigh other operations, so in this section we focus
on optimizing these dataflows. Addressing performance bottlenecks in our
system leads to overall lower computational graph latency, and to faster
robots. To seize this acceleration opportunity in our case study example, we
optimize the dataflow within the computational graph and across ROS 2 Nodes
and Components through two different design approaches: (a) kernel fusion, and
(b) dedicated streaming queues.

Figure 6: Total runtime of CPU baseline and FPGA, FPGA-Integrated, and FPGA-
Streaming hardware-accelerated implementations of case study application.
Acceleration enables up to $26.96\%$ speedup over CPU.

The speedup obtained by integrating both ROS _Components_ on the FPGA into a
_single unified kernel_ is shown in Fig. 6. The benefits of doing this are
two-fold. First, we avoid any message-passing between the _Rectify_ and
_Resize_ Nodes’ _Components_. Second, we avoid the compute cycles wasted while
memory is mapped back and forth between the host CPU and the FPGA. This
results in an overall latency speedup of $26.96\%$ over the CPU. In addition
to speeding up the perception stage, another added benefit of this improvement
is that such speedups make room for other robot tasks in a complete end-to-end
system. Note, however, that this improvement required the construction of an
entirely new ROS Node and unified acceleration kernel on the FPGA.

The results of a different approach, developing an _accelerated ROS 2 message
passing_ interface on the FPGA, are also shown in Fig. 6. This interface is
_Node_ and _Component_ -agnostic and can be leveraged by roboticists to
accelerate the communication channels of any computational graph on an FPGA.
This is done by leveraging an AXI4-Stream interface to create an intra-FPGA
ROS 2 communication queue which is then used to pass data across Nodes in the
FPGA without sending messages to the CPU. This allows us to completely bypass
the original CPU-centric ROS 2 message-passing system and optimizes dataflow,
achieving an overall latency improvement of $24.42\%$ over the CPU in our
application.

Based on these results, for this case study, we show that implementing FPGA-
accelerated versions of key ROS 2 _Components_ is easily feasible, and that
addressing the right bottleneck is key to improving performance. Tracing and
benchmarking the CPU baseline suggested that communication is the bottleneck
in our case study. In fact, independent examination of, e.g., a single run of
the fused-kernel accelerator using a device-specific tool (Fig. 7), confirms
that this is also the case on the FPGA—we note that integrating device-
specific profiling tools into our foundational tracing infrastructure in
future work can further automate this type of fine-grained introspection of
kernels onboard accelerator devices. We can achieve overall performance
improvements by either combining Nodes or streamlining intra-FPGA
communication. While combining nodes may result in slightly higher
performance, it is a much more labor-intensive design effort. By contrast, our
accelerated intra-FPGA-Node communication queues can be easily applied by any
roboticist, to any computational graph.

Figure 7: Screenshot thumbnail of single run of case study FPGA-Integrated
kernel accelerator using Xilinx Vitis Analyzer. Data transfer overheads
consume a substantial portion of runtime compared to kernel computation,
confirming the communication bottleneck on the FPGA, similar to CPU.

##  V Conclusion and Future Work

In this work we present a new open infrastructure to introduce hardware
acceleration in ROS 2 in a scalable and technology-agnostic manner. Our
architecture allows us to increase the performance of robotics applications
through _seamless integration of hardware acceleration_ with ROS 2 APIs and
its conventional flows. We do so by extending ROS 2 in a way that is _portable
across accelerator platforms_ (e.g., FPGAs, GPUs) and system deployments
(e.g., edge devices, workstations, data centers, and cloud). We also present a
_low-overhead tracing and benchmarking framework_ to analyze performance
across ROS 2 computational graphs.

We use our open architecture and our tracing and benchmarking infrastructure
to demonstrate a principled design methodology for ROS 2 hardware
acceleration, exposing insights into how to optimize overall system-wide
performance by analyzing a CPU baseline, and comparing accelerator design
iterations to that original baseline. We examine a case study using the Xilinx
Kria KV260 platform to demonstrate FPGA acceleration of one of the most
popular packages in the ROS perception pipeline: image_pipeline. We first
demonstrate a modest performance speedup of $6.22\%$ from offloading
perception tasks to the FPGA, and then increased speedup by additionally
addressing the communication overheads that we identified as bottlenecks by
analyzing our CPU baseline. We achieved a speedup of $26.96\%$ from re-
architecting the graph to combine nodes and avoid inter-FPGA-node
communication delays inflicted by interactions with the CPU, but this approach
requires substantial effort from users to re-architect their graphs. Instead,
to avoid this overhead and stay in alignment with the ROS 2 programming model,
we then design a novel _intra-FPGA ROS 2 Node communication queue_ that allows
ROS _Nodes_ and _Components_ to deliver faster dataflows, achieving a
$24.42\%$ speedup over a CPU without excessive manual per-kernel design
effort.

We contribute our open architecture to the ROS community, so that future work
can use our infrastructure and extend to new applications beyond our case
study example. Promising directions for future work include: benchmarking
computational graphs with other hardware solutions (e.g., GPUs) to establish
consistent cross-accelerator comparisons; extending our LTTng tracing and
benchmarking approach to include additional tracing information (e.g.,
profiling within FPGA or GPU devices) for more fine-grained introspection of
kernels running onboard accelerators; and applying our open architecture and
analysis to other ROS 2 packages.

Our code is disclosed under a commercially friendly open-source license and is
available and maintained at the ROS 2 Hardware Acceleration Working Group
GitHub organization: https://github.com/ros-acceleration. This work is being
further integrated into the ROS ecosystem through a community standardization
effort, REP-2008 [58].

## References

  * [1] Z. Wan, B. Yu, T. Y. Li, J. Tang, Y. Zhu, Y. Wang, A. Raychowdhury, and S. Liu, “A survey of fpga-based robotic computing,” _IEEE Circuits and Systems Magazine_ , vol. 21, no. 2, pp. 48–74, 2021.
  * [2] V. Mayoral-Vilches and G. Corradi, “Adaptive computing in robotics, towards ros 2 software-defined hardware,” _Xilinx, WP537_ , 2021.
  * [3] V. Mayoral-Vilches, “Kria robotics stack, a ros 2-centric approach for hardware acceleration in robotics,” _Xilinx, WP540_ , 2021.
  * [4] S. Murray, W. Floyd-Jones, Y. Qi, D. J. Sorin, and G. D. Konidaris, “Robot motion planning on a chip.” in _Robotics: Science and Systems_ , 2016.
  * [5] S. Murray, W. Floyd-Jones, Y. Qi, G. Konidaris, and D. J. Sorin, “The microarchitecture of a real-time robot motion planning accelerator,” in _2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)_. IEEE, 2016, pp. 1–12.
  * [6] S. Murray, W. Floyd-Jones, G. Konidaris, and D. J. Sorin, “A programmable architecture for robot motion planning acceleration,” in _2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP)_ , vol. 2160. IEEE, 2019, pp. 185–188.
  * [7] B. Plancher, S. M. Neuman, T. Bourgeat, S. Kuindersma, S. Devadas, and V. J. Reddi, “Accelerating robot dynamics gradients on a cpu, gpu, and fpga,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 2335–2342, 2021\.
  * [8] S. M. Neuman, B. Plancher, T. Bourgeat, T. Tambe, S. Devadas, and V. J. Reddi, “Robomorphic computing: a design methodology for domain-specific accelerators parameterized by robot morphology,” in _Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems_ , 2021, pp. 674–686.
  * [9] B. Plancher, S. M. Neuman, R. Ghosal, S. Kuindersma, and V. J. Reddi, “Grid: Gpu-accelerated rigid body dynamics with analytical gradients,” in _2022 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2022.
  * [10] J. Austin, R. Corrales-Fatou, S. Wyetzner, and H. Lipson, “Titan: A parallel asynchronous library for multi-agent and soft-body robotics using nvidia cuda,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 7754–7760.
  * [11] C. D. Freeman, E. Frey, A. Raichuk, S. Girgin, I. Mordatch, and O. Bachem, “Brax–a differentiable physics engine for large scale rigid body simulation,” _arXiv preprint arXiv:2106.13281_ , 2021.
  * [12] A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, and V. Sze, “Navion: A 2-mw fully integrated real-time visual-inertial odometry accelerator for autonomous navigation of nano drones,” _IEEE Journal of Solid-State Circuits_ , vol. 54, no. 4, pp. 1106–1119, 2019.
  * [13] Y. Liu, C. E. Derman, G. Calderoni, and R. I. Bahar, “Hardware acceleration of robot scene perception algorithms,” in _2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD)_. IEEE, 2020, pp. 1–8.
  * [14] B. Asgari, R. Hadidi, N. S. Ghaleshahi, and H. Kim, “Pisces: power-aware implementation of slam by customizing efficient sparse algebra,” in _2020 57th ACM/IEEE Design Automation Conference (DAC)_. IEEE, 2020, pp. 1–6.
  * [15] W. Liu, B. Yu, Y. Gan, Q. Liu, J. Tang, S. Liu, and Y. Zhu, “Archytas: A framework for synthesizing and dynamically optimizing accelerators for robotic localization,” in _MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture_ , 2021, pp. 479–493.
  * [16] V. Mayoral-Vilches, A. Hernández, R. Kojcev, I. Muguruza, I. Zamalloa, A. Bilbao, and L. Usategi, “The shift in the robotics paradigm: The hardware robot operating system (h-ros); an infrastructure to create interoperable robot components,” in _2017 NASA/ESA Conference on Adaptive Hardware and Systems (AHS)_ , July 2017, pp. 229–236.
  * [17] G. Williams, A. Aldrich, and E. A. Theodorou, “Model predictive path integral control: From theory to parallel computation,” _Journal of Guidance, Control, and Dynamics_ , vol. 40, no. 2, pp. 344–357, 2017.
  * [18] J. Sacks, D. Mahajan, R. C. Lawson, and H. Esmaeilzadeh, “Robox: an end-to-end solution to accelerate autonomous control in robotics,” in _2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 2018, pp. 479–490.
  * [19] B. Plancher and S. Kuindersma, “A performance analysis of parallel differential dynamic programming on a gpu,” in _International Workshop on the Algorithmic Foundations of Robotics (WAFR)_. Merida, Mexico: Springer, Dec. 2018, pp. 656–672.
  * [20] ——, “Realtime model predictive control using parallel ddp on a gpu,” in _Toward Online Optimal Control of Dynamic Robots Workshop at the 2019 International Conference on Robotics and Automation (ICRA)_ , Montreal, Canada, May. 2019.
  * [21] K. Gupta, P. Z. X. Li, S. Karaman, and V. Sze, “Efficient computation of map-scale continuous mutual information on chip in real time,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 6464–6470.
  * [22] A. Munshi, “The opencl specification,” in _2009 IEEE Hot Chips 21 Symposium (HCS)_. IEEE, 2009, pp. 1–314.
  * [23] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler, and A. Ng, “Ros: an open-source robot operating system,” in _Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA) Workshop on Open Source Robotics_ , Kobe, Japan, May 2009.
  * [24] NVIDIA, “Nvidia isaac ros,” Accessed 2022, github.com/NVIDIA-ISAAC-ROS.
  * [25] C. Bédard, I. Lütkebohle, and M. Dagenais, “ros2_tracing: Multipurpose low-overhead framework for real-time tracing of ros 2,” Accessed 2022, gitlab.com/ros-tracing/ros2˙tracing.
  * [26] M. Desnoyers and M. R. Dagenais, “The lttng tracer: A low impact performance and behavior monitor for gnu/linux,” in _OLS (Ottawa Linux Symposium)_ , vol. 2006. Citeseer, 2006, pp. 209–224.
  * [27] A. AMBA, “Axi4-stream protocol specification,” _Volume IHI A_ , vol. 51, p. 4, 4.
  * [28] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in _ICRA workshop on open source software_ , vol. 3, no. 3.2. Kobe, Japan, 2009, p. 5.
  * [29] K. Yamashina, T. Ohkawa, K. Ootsu, and T. Yokota, “Proposal of ros-compliant fpga component for low-power robotic systems: case study on image processing application,” _2nd International Workshop on FPGAs for Software Programmers (FSP 2015)_ , 2015.
  * [30] K. Yamashina, H. Kimura, T. Ohkawa, K. Ootsu, and T. Yokota, “crecomp: Automated design tool for ros-compliant fpga component,” in _2016 IEEE 10th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSOC)_. IEEE, 2016, pp. 138–145.
  * [31] A. Podlubne and D. Göhringer, “Fpga-ros: Methodology to augment the robot operating system with fpga designs,” in _2019 International Conference on ReConFigurable Computing and FPGAs (ReConFig)_. IEEE, 2019, pp. 1–5.
  * [32] M. Eisoldt, S. Hinderink, M. Tassemeier, M. Flottmann, J. Vana, T. Wiemann, J. Gaal, M. Rothmann, and M. Porrmann, “Reconfros: Running ros on reconfigurable socs,” in _Proceedings of the 2021 Drone Systems Engineering and Rapid Simulation and Performance Evaluation: Methods and Tools Proceedings_ , 2021, pp. 16–21.
  * [33] C. Lienen, M. Platzner, and B. Rinner, “Reconros: Flexible hardware acceleration for ros2 applications,” in _2020 International Conference on Field-Programmable Technology (ICFPT)_. IEEE, 2020, pp. 268–276.
  * [34] D. P. Leal, M. Sugaya, H. Amano, and T. Ohkawa, “Automated integration of high-level synthesis fpga modules with ros2 systems,” in _2020 International Conference on Field-Programmable Technology (ICFPT)_ , 2020, pp. 292–293.
  * [35] T. Ohkawa, K. Yamashina, T. Matsumoto, K. Ootsu, and T. Yokota, “Architecture exploration of intelligent robot system using ros-compliant fpga component,” in _2016 International Symposium on Rapid System Prototyping (RSP)_. IEEE, 2016, pp. 1–7.
  * [36] S. Panadda, J. Nattha, P. L. Daniel, and O. Takeshi, “Low-power high-performance intelligent camera framework ros-fpga node,” in _Proceedings of Asia Pacific Conference on Robot IoT System Development and Platform_ , no. 2020, 2021, pp. 73–74.
  * [37] J. P. Queralta, F. Yuhong, L. Salomaa, L. Qingqing, T. N. Gia, Z. Zou, H. Tenhunen, and T. Westerlund, “Fpga-based architecture for a low-cost 3d lidar design and implementation from multiple rotating 2d lidars with ros,” in _2019 IEEE SENSORS_ , 2019, pp. 1–4.
  * [38] T. K. Maiti, “Ros on arm processor embedded with fpga for improvement of robotic computing,” in _2021 International Symposium on Devices, Circuits and Systems (ISDCS)_ , 2021, pp. 1–4.
  * [39] T. Ohkawa, K. Yamashina, H. Kimura, K. Ootsu, and T. Yokota, “Fpga components for integrating fpgas into robot systems,” _IEICE TRANSACTIONS on Information and Systems_ , vol. 101, no. 2, pp. 363–375, 2018.
  * [40] D. P. Leal, M. Sugaya, H. Amano, and T. Ohkawa, “Fpga acceleration of ros2-based reinforcement learning agents,” in _2020 Eighth International Symposium on Computing and Networking Workshops (CANDARW)_ , 2020, pp. 106–112.
  * [41] H. Amano, H. Mori, A. Mizutani, T. Ono, Y. Yoshimoto, T. Ohkawa, and H. Tamukoh, “A dataset generation for object recognition and a tool for generating ros2 fpga node,” in _2021 International Conference on Field-Programmable Technology (ICFPT)_. IEEE, 2021, pp. 1–4.
  * [42] Y. Nitta, S. Tamura, and H. Takase, “A study on introducing fpga to ros based autonomous driving system,” in _2018 International Conference on Field-Programmable Technology (FPT)_. IEEE, 2018, pp. 421–424.
  * [43] K. E. Chen, Y. Liang, N. Jha, J. Ichnowski, M. Danielczuk, J. Gonzalez, J. Kubiatowicz, and K. Goldberg, “Fogros: An adaptive framework for automating fog robotics deployment,” in _2021 IEEE 17th International Conference on Automation Science and Engineering (CASE)_. IEEE, 2021, pp. 2035–2042.
  * [44] Y. Sugata, T. Ohkawa, K. Ootsu, and T. Yokota, “Acceleration of publish/subscribe messaging in ros-compliant fpga component,” in _Proceedings of the 8th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies_ , 2017, pp. 1–6.
  * [45] T. Ohkawa, Y. Sugata, H. Watanabe, N. Ogura, K. Ootsu, and T. Yokota, “High level synthesis of ros protocol interpretation and communication circuit for fpga,” in _2019 IEEE/ACM 2nd International Workshop on Robotics Software Engineering (RoSE)_. IEEE, 2019, pp. 33–36.
  * [46] H. Choi, Y. Xiang, and H. Kim, “Picas: New design of priority-driven chain-aware scheduling for ros2,” in _2021 IEEE 27th Real-Time and Embedded Technology and Applications Symposium (RTAS)_. IEEE, 2021, pp. 251–263.
  * [47] Y. Suzuki, T. Azumi, S. Kato, and N. Nishio, “Real-time ros extension on transparent cpu/gpu coordination mechanism,” in _2018 IEEE 21st International Symposium on Real-Time Distributed Computing (ISORC)_. IEEE, 2018, pp. 184–192.
  * [48] C. S. V. Gutiérrez, L. U. S. Juan, I. Z. Ugarte, and V. Mayoral-Vilches, “Time-sensitive networking for robotics,” _arXiv preprint arXiv:1804.07643_ , 2018.
  * [49] ——, “Real-time linux communications: an evaluation of the linux communication stack for real-time robotic applications,” _arXiv preprint arXiv:1808.10821_ , 2018.
  * [50] ——, “Towards a distributed and real-time framework for robots: Evaluation of ros 2.0 communications for real-time robotic applications,” _arXiv preprint arXiv:1809.02595_ , 2018.
  * [51] C. S. V. Gutiérrez, L. U. S. Juan, I. Z. Ugarte, I. M. Goenaga, L. A. Kirschgens, and V. Mayoral-Vilches, “Time synchronization in modular collaborative robots,” _arXiv preprint arXiv:1809.07295_ , 2018.
  * [52] Xilinx, “Vitis unified software platform,” Accessed 2022, xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vitis.html.
  * [53] ——, “Xilinx runtime (xrt),” Accessed 2022, github.com/Xilinx/XRT.
  * [54] NVIDIA, “Nvidia jetpack sdk,” Accessed 2022, developer.nvidia.com/embedded/jetpack.
  * [55] N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in _Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on_ , vol. 3. IEEE, 2004, pp. 2149–2154.
  * [56] P. Mihelich and J. Bowman, “image_pipeline ros metapackage,” Accessed 2022, github.com/ros-perception/image˙pipeline.
  * [57] Xilinx, “Kria® kv260 vision ai starter,” Accessed 2022, xilinx.com/products/som/kria/kv260-vision-starter-kit.html.
  * [58] ROS 2 Hardware Acceleration Working Group (HAWG), “Ros enhancement proposal (rep): Rep-2008 - ros 2 hardware acceleration architecture and conventions,” Accessed 2022, github.com/ros-infrastructure/rep/pull/324.


To bound also the right-hand side, we first use the safety strain condition
(1.13), which implies that there exists a $\boldsymbol{\mathsf{T}}_{0}\in
L^{1}(Q;\mathbb{R}^{d\times d})$ such that

$\alpha\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0})+\beta\partial_{t}\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0})=\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}_{0})\qquad\textrm{
a.e. in }Q.$

Thus, using the monotonicity of $\boldsymbol{\mathsf{G}}$, we see that

$\displaystyle\boldsymbol{\mathsf{T}}^{n}\cdot((\alpha\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0})+\beta\partial_{t}\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0}))-\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n}))$
$\displaystyle\leq\boldsymbol{\mathsf{T}}^{n}\cdot(\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}_{0})-\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}^{n}))\leq\boldsymbol{\mathsf{T}}_{0}\cdot(\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}_{0})-\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}^{n})).$

Using the convergence results (4.20)–(4.27) applied to all terms in (4.31)
with the above inequality yields the following:

$\begin{split}&\frac{1}{4}\int_{\Omega}\beta|\partial_{t}(\boldsymbol{u}-\boldsymbol{u}_{0})(t)|^{2}+\beta\left|\partial_{t}(\boldsymbol{u}-\boldsymbol{u}_{0})(t)+\frac{2\alpha}{\beta}(\boldsymbol{u}-\boldsymbol{u}_{0})(t)\right|^{2}\,\mathrm{d}x\\\
&\leq\int_{0}^{t}\int_{\Omega}\boldsymbol{\mathsf{T}}_{0}\cdot((\alpha\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0})+\beta\partial_{t}\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{u}_{0}))-\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}))+\alpha|\partial_{t}(\boldsymbol{u}-\boldsymbol{u}_{0})|^{2}\,\mathrm{d}x\,\mathrm{d}\tau\\\
&\quad+\int_{0}^{t}\int_{\Omega}(\boldsymbol{f}-\partial_{tt}\boldsymbol{u}_{0})\cdot(\alpha(\boldsymbol{u}-\boldsymbol{u}_{0})+\beta\partial_{t}(\boldsymbol{u}-\boldsymbol{u}_{0}))+\frac{2\alpha^{2}}{\beta}\partial_{t}(\boldsymbol{u}-\boldsymbol{u}_{0})\cdot(\boldsymbol{u}-\boldsymbol{u}_{0})\,\mathrm{d}x\,\mathrm{d}\tau\\\
&\leq
C\int_{0}^{t}\|\boldsymbol{\mathsf{T}}_{0}\|_{1}+\|\boldsymbol{f}\|_{2}+\|\partial_{tt}\boldsymbol{u}_{0}\|_{2}+1\,\mathrm{d}\tau.\end{split}$
(4.32)

Letting $t\to 0_{+}$, we see that

$\lim_{t\to
0_{+}}(\|\boldsymbol{u}(t)-\boldsymbol{u}_{0}(0)\|_{2}^{2}+\|\partial_{t}\boldsymbol{u}(t)-\partial_{t}\boldsymbol{u}_{0}(0)\|^{2}_{2})=0.$

In addition, it also follows from (4.23) that
$\boldsymbol{u}\in\mathcal{C}^{1}_{loc}(0,T;L^{2}(\Omega;\mathbb{R}^{d}))$,
which combined with the above result gives that
$\boldsymbol{u}\in\mathcal{C}^{1}([0,T];L^{2}(\Omega;\mathbb{R}^{d}))$.

###  4.4 Validity of the equation in the limit

To summarize the results so far, we have found a couple
$(\boldsymbol{u},\boldsymbol{\mathsf{T}})$ that satisfies (1.3)–(1.18) and
(1.20), (1.21). It remains to show (1.19). To do so, we use the method
developed in [1]. Let $g$ be a smooth nonnegative nonincreasing function
satisfying

$g(s)=\left\\{\begin{aligned} &1,&&\textrm{for }s\in[0,1],\\\ &0,&&\textrm{for
}s>2.\end{aligned}\right.$

For each $k\in\mathbb{N}$, let us define

$g_{k}(s):=g(s/k).$

It is clear that $g_{k}\nearrow 1$. Next let
$\boldsymbol{v}\in\mathcal{C}^{\infty}_{0}(Q;\mathbb{R}^{d})$ be arbitrary but
fixed. Thanks to (4.23) and (4.29), all terms in (1.19) are well-defined for
almost all $t\in(0,T)$ and we just need to check that the equality holds.

Using the properties of $g_{k}$, we have

$\begin{split}I&:=\int_{Q}\partial_{tt}\boldsymbol{u}\cdot\boldsymbol{v}+\boldsymbol{\mathsf{T}}\cdot\nabla\boldsymbol{v}-\boldsymbol{f}\cdot\boldsymbol{v}\,\mathrm{d}x\,\mathrm{d}t\\\
&=\lim_{k\to\infty}\int_{Q}\partial_{tt}\boldsymbol{u}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}|)+\boldsymbol{\mathsf{T}}\cdot\nabla\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}|)-\boldsymbol{f}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}|)\,\mathrm{d}x\,\mathrm{d}t.\end{split}$
(4.33)

Using (4.23), (4.28), the fact that $\boldsymbol{\mathsf{T}}^{n}\in
L^{2}_{loc}(0,T;W^{1,2}_{loc}(\Omega;\mathbb{R}^{d\times d}))$, which follows
from (4.17), and the fact that $g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)$ is
supported only in the set where $|\boldsymbol{\mathsf{T}}^{n}|\leq k$, we can
rewrite the right-hand side of (4.33) in the following way:

$\begin{split}I&=\lim_{k\to\infty}\lim_{n\to\infty}\int_{Q}\partial_{tt}\boldsymbol{u}^{n}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)+\boldsymbol{\mathsf{T}}^{n}\cdot\nabla\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)-\boldsymbol{f}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)\,\mathrm{d}x\,\mathrm{d}t\\\
&=\lim_{k\to\infty}\lim_{n\to\infty}\int_{Q}\partial_{tt}\boldsymbol{u}^{n}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)+\boldsymbol{\mathsf{T}}^{n}\cdot\nabla(\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|))-\boldsymbol{f}\cdot\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)\,\mathrm{d}x\,\mathrm{d}t\\\
&\quad-\lim_{k\to\infty}\lim_{n\to\infty}\int_{Q}\boldsymbol{\mathsf{T}}^{n}\cdot(\nabla
g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)\otimes\boldsymbol{v})\,\mathrm{d}x\,\mathrm{d}t\\\
&=-\lim_{k\to\infty}\lim_{n\to\infty}\int_{Q}\boldsymbol{\mathsf{T}}^{n}\cdot(\nabla
g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)\otimes\boldsymbol{v})\,\mathrm{d}x\,\mathrm{d}t,\end{split}$
(4.34)

where for the last equality we have used (4.4) with
$\boldsymbol{w}:=\boldsymbol{v}g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)$. It
remains to show that the right-hand side of (4.34) vanishes. We define

$M_{k,n}(s):=\int_{0}^{s}\frac{g^{\prime}_{k}(t)}{\frac{\phi^{\prime}(t)}{t}+n^{-1}}\,\mathrm{d}t\leq\int_{0}^{s}\frac{tg^{\prime}_{k}(t)}{\phi^{\prime}(t)}\,\mathrm{d}t=:M_{k}(s).$

Then, using that $|g_{k}^{\prime}(s)|\leq Cs^{-1}\chi_{\\{s\in(k,2k)\\}}$, we
see that

$M_{k}(s)\left\\{\begin{aligned} &\leq C\min\\{s,k\\}&&\textrm{for all }s\geq
0,\\\ &=0&&\textrm{for }s\leq k.\end{aligned}\right.$ (4.35)

Next we can use the structural assumption (A4) to rewrite the term under the
limit in (4.34) as

$\displaystyle-\int_{Q}$ $\displaystyle\boldsymbol{\mathsf{T}}^{n}\cdot(\nabla
g_{k}(|\boldsymbol{\mathsf{T}}^{n}|)\otimes\boldsymbol{v})\,\mathrm{d}x\,\mathrm{d}t$
(4.36)
$\displaystyle=-\int_{Q}\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n})\cdot(\nabla|\boldsymbol{\mathsf{T}}^{n}|\otimes\boldsymbol{v})\frac{g^{\prime}_{k}(|\boldsymbol{\mathsf{T}}^{n}|)}{\frac{\phi^{\prime}(|\boldsymbol{\mathsf{T}}^{n}|)}{|\boldsymbol{\mathsf{T}}^{n}|}+n^{-1}}\,\mathrm{d}x\,\mathrm{d}t$
$\displaystyle=-\int_{Q}\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n})\cdot(\nabla
M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\otimes\boldsymbol{v})\,\mathrm{d}x\,\mathrm{d}t$
$\displaystyle=\int_{Q}\mathop{\mathrm{div}}\nolimits\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n})\cdot\boldsymbol{v}M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\,\mathrm{d}x\,\mathrm{d}t+\int_{Q}\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n})\cdot\nabla\boldsymbol{v}M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\,\mathrm{d}x\,\mathrm{d}t.$

For the first term on the right-hand side of (4.36), we use the definition of
$\mathcal{A}_{n}$ alongside the Cauchy–Schwarz inequality to obtain

$\displaystyle|\mathop{\mathrm{div}}\nolimits\boldsymbol{\mathsf{G}}_{n}(\boldsymbol{\mathsf{T}}^{n})\cdot\boldsymbol{v}M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)|=\left|\sum_{i,j,a,b=1}^{d}(\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n}))^{ij}_{ab}\partial_{j}\boldsymbol{\mathsf{T}}_{ab}^{n}\boldsymbol{v}_{i}M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\right|$
$\displaystyle=\left|\sum_{m=1}^{d}\sum_{i,j,a,b=1}^{d}(\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n}))^{ij}_{ab}\partial_{m}\boldsymbol{\mathsf{T}}_{ab}^{n}\delta_{mj}\boldsymbol{v}_{i}M_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\right|$
$\displaystyle\leq\left|\sum_{m=1}^{d}\left(\partial_{m}\boldsymbol{\mathsf{T}}^{n},\partial_{m}\boldsymbol{\mathsf{T}}^{n}\right)^{\frac{1}{2}}_{\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n})}\left(\sum_{i,j,a,b=1}^{d}(\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n}))^{ij}_{ab}\delta_{mj}\boldsymbol{v}_{i}\delta_{ma}\boldsymbol{v}_{b}M^{2}_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\right)^{\frac{1}{2}}\right|$
$\displaystyle\leq\left|\sum_{m=1}^{d}\left(\partial_{m}\boldsymbol{\mathsf{T}}^{n},\partial_{m}\boldsymbol{\mathsf{T}}^{n}\right)^{\frac{1}{2}}_{\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n})}\left((n^{-1}+\frac{C}{1+|\boldsymbol{\mathsf{T}}^{n}|})|\boldsymbol{v}|^{2}M^{2}_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\right)^{\frac{1}{2}}\right|.$

Using this bound in (4.36) and then in (4.34), recalling the fact that
$\boldsymbol{v}$ is compactly supported, we deduce with the help of Hölder’s
inequality and the uniform bound (4.16) that

$\begin{split}|I|&\leq\lim_{k\to\infty}\lim_{n\to\infty}\int_{Q}\left|\sum_{m=1}^{d}\left(\partial_{m}\boldsymbol{\mathsf{T}}^{n},\partial_{m}\boldsymbol{\mathsf{T}}^{n}\right)^{\frac{1}{2}}_{\mathcal{A}_{n}(\boldsymbol{\mathsf{T}}^{n})}\left(\left(n^{-1}+\frac{C}{1+|\boldsymbol{\mathsf{T}}^{n}|}\right)|\boldsymbol{v}|^{2}M^{2}_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\right)^{\frac{1}{2}}\right|\,\mathrm{d}x\,\mathrm{d}t\\\
&\leq
C(\boldsymbol{v})\lim_{k\to\infty}\lim_{n\to\infty}\left(\int_{Q}\left(n^{-1}+\frac{C}{1+|\boldsymbol{\mathsf{T}}^{n}|}\right)M^{2}_{k,n}(|\boldsymbol{\mathsf{T}}^{n}|)\,\mathrm{d}x\,\mathrm{d}t\right)^{\frac{1}{2}}\\\
&=C(\boldsymbol{v})\lim_{k\to\infty}\left(\int_{Q}\frac{M^{2}_{k}(|\boldsymbol{\mathsf{T}}|)}{|\boldsymbol{\mathsf{T}}|}\,\mathrm{d}x\,\mathrm{d}t\right)^{\frac{1}{2}},\end{split}$
(4.37)

where for the last equality we used (4.29) and the boundedness of $M_{k}$.
Consequently, using that $\boldsymbol{\mathsf{T}}\in
L^{1}(Q;\mathbb{R}^{d\times d})$ and the structure of $M_{k}$ (4.35), we
deduce that

$\begin{split}|I|&\leq
C(\boldsymbol{v})\lim_{k\to\infty}\left(\int_{Q}\frac{M^{2}_{k}(|\boldsymbol{\mathsf{T}}|)}{|\boldsymbol{\mathsf{T}}|}\,\mathrm{d}x\,\mathrm{d}t\right)^{\frac{1}{2}}\leq
C(\boldsymbol{v})\lim_{k\to\infty}\left(\int_{Q\cap\\{|\boldsymbol{\mathsf{T}}|>k\\}}|\boldsymbol{\mathsf{T}}|\,\mathrm{d}x\,\mathrm{d}t\right)^{\frac{1}{2}}=0.\end{split}$

Since $\boldsymbol{v}$ is arbitrary, we see that (1.19) holds for almost all
$t\in(0,T)$ and all smooth compactly supported $\boldsymbol{w}$. Finally,
using a weak∗ density argument based on [1, Lemma A.3] we deduce that (1.19)
holds for an arbitrary $\boldsymbol{w}\in W^{1,2}_{0}(\Omega,\mathbb{R}^{d})$
fulfilling $\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{w})\in
L^{\infty}(Q;\mathbb{R}^{d\times d})$. This concludes the proof of the
existence of a solution as asserted in Theorem 1.2.

###  4.5 Uniqueness of the solution

It remains to prove the uniqueness of such weak solutions. Let
$(\boldsymbol{u}_{1},\boldsymbol{\mathsf{T}}_{1})$ and
$(\boldsymbol{u}_{2},\boldsymbol{\mathsf{T}}_{2})$ be two solutions emanating
from the same data and denote
$\boldsymbol{v}:=\boldsymbol{u}_{1}-\boldsymbol{u}_{2}$. Then it follows from
(1.19) that, for almost all $t\in(0,T)$ and for every $\boldsymbol{w}\in
W^{1,\infty}_{0}(\Omega;\mathbb{R}^{d})$,

$\int_{\Omega}\partial_{tt}\boldsymbol{v}\cdot\boldsymbol{w}+(\boldsymbol{\mathsf{T}}_{1}-\boldsymbol{\mathsf{T}}_{2})\cdot\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{w})\,\mathrm{d}x=0.$
(4.38)

Since $\partial_{t}\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{v})$ and
$\boldsymbol{\mathsf{\varepsilon}}(\boldsymbol{v})$ belong to
$L^{\infty}(\Omega;\mathbb{R}^{d\times d})$ for almost all $t\in(0,T)$, we can
again use the weak∗ density argument as in the previous section to deduce that
(4.38) holds with
$\boldsymbol{w}:=\alpha\boldsymbol{v}+\beta\partial_{t}\boldsymbol{v}$.
Consequently, since we have

$\alpha\boldsymbol{v}+\beta\partial_{t}\boldsymbol{v}=\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}_{1})-\boldsymbol{\mathsf{G}}(\boldsymbol{\mathsf{T}}_{2}),$

we can use the monotonicity of $\boldsymbol{\mathsf{G}}$ and integration over
$(t_{0},t)$, with $0<t_{0}<t<T$, to deduce from (4.38) that

$\displaystyle 0$ $\displaystyle\geq
2\int_{t_{0}}^{t}\int_{\Omega}\partial_{tt}\boldsymbol{v}\cdot(\alpha\boldsymbol{v}+\beta\partial_{t}\boldsymbol{v})\,\mathrm{d}x\,\mathrm{d}\tau$
$\displaystyle\quad=\beta\int_{\Omega}|\partial_{t}\boldsymbol{v}(t)|^{2}-|\partial_{t}\boldsymbol{v}(t_{0})|^{2}+2\alpha\partial_{t}\boldsymbol{v}(t)\cdot\boldsymbol{v}(t)-2\alpha\boldsymbol{v}(t_{0})\cdot\boldsymbol{v}(t_{0})\,\mathrm{d}x-2\alpha\int_{t_{0}}^{t}\int_{\Omega}|\partial_{t}\boldsymbol{v}|^{2}\,\mathrm{d}x\,\mathrm{d}\tau.$

We note that this procedure is rigorous for every such $t_{0}>0$ thanks to the
regularity of $\boldsymbol{u}_{1}$ and $\boldsymbol{u}_{2}$ asserted in
(1.15). Since
$\boldsymbol{v}\in\mathcal{C}^{1}([0,T];L^{2}(\Omega;\mathbb{R}^{d}))$ as a
result of (1.15), we can use (1.21) and let $t_{0}\to 0_{+}$ in the above
inequality in order to deduce that

$\displaystyle 0$
$\displaystyle\geq\beta\int_{\Omega}|\partial_{t}\boldsymbol{v}(t)|^{2}+2\alpha\partial_{t}\boldsymbol{v}(t)\cdot\boldsymbol{v}(t)\,\mathrm{d}x-2\alpha\int_{0}^{t}\int_{\Omega}|\partial_{t}\boldsymbol{v}|^{2}\,\mathrm{d}x\,\mathrm{d}\tau$
$\displaystyle=\beta\int_{\Omega}|\partial_{t}\boldsymbol{v}(t)|^{2}+2\alpha\partial_{t}\boldsymbol{v}(t)\cdot\left(\int_{0}^{t}\partial_{t}\boldsymbol{v}(\tau)\,\mathrm{d}\tau\right)\,\mathrm{d}x-2\alpha\int_{0}^{t}\int_{\Omega}|\partial_{t}\boldsymbol{v}|^{2}\,\mathrm{d}x\,\mathrm{d}\tau$
$\displaystyle\geq\frac{\beta}{2}\left(\|\partial_{t}\boldsymbol{v}(t)\|_{2}^{2}-C(\alpha,\beta,T)\int_{0}^{t}\|\partial_{t}\boldsymbol{v}(\tau)\|_{2}^{2}\,\mathrm{d}\tau\right)$
$\displaystyle=\mathrm{e}^{-tC(\alpha,\beta,T)}\frac{\mathrm{d}}{\mathrm{d}t}\left(\mathrm{e}^{-tC(\alpha,\beta,T)}\int_{0}^{t}\|\partial_{t}\boldsymbol{v}(\tau)\|_{2}^{2}\,\mathrm{d}\tau\right).$

Simple integration with respect to $t$ then gives that
$\partial_{t}\boldsymbol{v}\equiv 0$ almost everywhere in $Q$ and consequently
$\boldsymbol{u}_{1}=\boldsymbol{u}_{2}$. By strict monotonicity, we
necessarily also have that
$\boldsymbol{\mathsf{T}}_{1}=\boldsymbol{\mathsf{T}}_{2}$ almost everywhere in
$Q$ and, hence, uniqueness follows.

## References

  * Beck et al. [2017] L. Beck, M. Bulíček, J. Málek, and E. Süli.  On the existence of integrable solutions to nonlinear elliptic systems and variational problems with linear growth.  _Arch. Ration. Mech. Anal._ , 225(2):717–769, 2017.  ISSN 0003-9527.  doi: 10.1007/s00205-017-1113-4.  URL https://doi.org/10.1007/s00205-017-1113-4.
  * Beck et al. [2020] L. Beck, M. Bulíček, and Gmeineder F.  On a Neumann problem for variational functionals of linear growth.  Accepted for publication in _Annali della Scuola Normale Superiore di Pisa, Classe di Scienze_ , DOI: 10.2422/2036-2145.201802_005, 2020\.
  * Bulíček et al. [2012] M. Bulíček, J. Málek, and K. R. Rajagopal.  On Kelvin–Voigt model and its generalizations.  _Evol. Equ. Control Theory_ , 1(1):17–42, 2012\.  ISSN 2163-2472.  doi: 10.3934/eect.2012.1.17.  URL https://doi.org/10.3934/eect.2012.1.17.
  * Bulíček et al. [2013] M. Bulíček, P. Kaplický, and M. Steinhauer.  On existence of a classical solution to a generalized Kelvin–Voigt model.  _Pacific J. Math._ , 262(1):11–33, 2013.  ISSN 0030-8730.  doi: 10.2140/pjm.2013.262.11.  URL https://doi.org/10.2140/pjm.2013.262.11.
  * Bulíček et al. [2014] M. Bulíček, J. Málek, K. R. Rajagopal, and E. Süli.  On elastic solids with limiting small strain: Modelling and analysis.  _EMS Surv. Math. Sci._ , 1(2):283–332, 2014.
  * Bulíček et al. [2020] M. Bulíček, V. Patel, Y. Şengül, and E. Süli.  Existence of large-data global weak solutions to a model of a strain-limiting viscoelastic body.  Submitted. arXiv:2011.07490, 2020.
  * Şengül [2020a] Y. Şengül.  Viscoelasticity with limiting strain.  _Disc. Cont. Dyn. Syst. S_ , 2020a.  doi: 10.3934/dcdss.2020330.
  * Şengül [2020b] Y. Şengül.  Nonlinear viscoelasticity of strain-rate type: an overview.  Submitted, 2020b.
  * Erbay and Şengül [2015] H. A. Erbay and Y. Şengül.  Traveling waves in one-dimensional nonlinear models of strain-limiting viscoelasticity.  _Int. J. Nonlinear Mech._ , 77:61–68, 2015.
  * Erbay and Şengül [2020] H. A. Erbay and Y. Şengül.  A thermodynamically consistent stress-rate type model of one-dimensional strain-limiting viscoelasticity.  _Z. Angew. Math. Phys._ , 71(3):94, 2020.  ISSN 0044-2275.  doi: 10.1007/s00033-020-01315-7.  URL https://doi.org/10.1007/s00033-020-01315-7.
  * Erbay et al. [2020] H. A. Erbay, A. Erkip, and Y. Şengül.  Local existence of solutions to the initial-value problem for one-dimensional strain-limiting viscoelasticity.  _J. Differential Equations_ , 269:9720–9739, 2020.
  * Rajagopal [2003] K. R. Rajagopal.  On implicit constitutive theories.  _Appl. Math._ , 48:279–319, 2003.
  * Rajagopal [2006] K. R. Rajagopal.  On implicit constitutive theories for fluids.  _J. Fluid Mech._ , 550:243–249, 2006.
  * Rajagopal [2010] K. R. Rajagopal.  On new class of models in elasticity.  _J. Math. Comp. Appl._ , 15(4):506–528, 2010\.
  * Rajagopal and Saccomandi [2014] K. R. Rajagopal and G. Saccomandi.  Circularly polarized wave propagation in a class of bodies defined by a new class of implicit constitutive relations.  _Z. Angew. Math. Phys._ , 65(5):1003–1010, 2014\.  ISSN 0044-2275.  doi: 10.1007/s00033-013-0362-9.  URL https://ezproxy-prd.bodleian.ox.ac.uk:2095/10.1007/s00033-013-0362-9.


# New light rings from multiple critical curves as observational signatures of
black hole mimickers

Gonzalo J. Olmo<EMAIL_ADDRESS>Departamento de Física Teórica and IFIC,
Centro Mixto Universidad de Valencia - CSIC. Universidad de Valencia,
Burjassot-46100, Valencia, Spain  Departamento de Física, Universidade Federal
da Paraíba, 58051-900 João Pessoa, Paraíba, Brazil  Diego Rubiera-Garcia
<EMAIL_ADDRESS>Departamento de Física Teórica and IPARCOS, Universidad
Complutense de Madrid, E-28040 Madrid, Spain  Diego Sáez-Chillón Gómez
<EMAIL_ADDRESS>Departamento de Física Teórica, Atómica y Óptica, Campus
Miguel Delibes,
Universidad de Valladolid UVa, Paseo Belén, 7, 47011 - Valladolid, Spain

(August 28, 2024)

###### Abstract

We argue that the appearance of additional light rings in a shadow observation
- beyond the infinite sequence of exponentially demagnified self-similar rings
foreseen in the Kerr solution - would make a compelling case for the existence
of black hole mimickers having multiple critical curves. We support this claim
by discussing three different scenarios of spherically symmetric wormhole
geometries having two such critical curves, and explicitly work out the
optical appearance of one such object when surrounded by an optically and
geometrically thin accretion disk.

##  I Introduction

For decades after Schwarzschild found its celebrated solution, the
relativistic community had to deal with the conceptual novelties that the
presence of (what is nowadays known as) an event horizon raised. It was not
until the Kerr(-Newman) family of axisymmetric electrovacuum solutions of
Einstein’s field equations of General Relativity (GR) was found Waldbook ,
together with the understanding of gravitational collapse Joshibook , that
black holes (BHs) were accepted as a reliable representation of the final fate
of massive enough fuel-exhausted stars. The Kerr solution has passed numerous
tests for its astrophysical viability, most notoriously those based on
electromagnetic radiation Bambi:2015kza , which recently include the shadows
surrounding the object EventHorizonTelescope:2019dse . The latter corresponds
to the dark silhouette of a BH when illuminated by an accretion disk
Luminet:1979nyg ; Narayan:2019imo , where the gravitational deflection of
light rays approaching an unstable circular orbit (a critical curve) defines
the boundary of the central brightness depression. Moreover, by assuming the
disk to be optically thin, this boundary is made up of a set of strongly
lensed and gravitationally redshifted light rings superimposed on the direct
emission Falcke:1999pj .

The current constraints on shadows shapes set by the EHT Collaboration
measurements EventHorizonTelescope:2019dse  point to their compatibility with
the rough expectations of the Kerr solution. However, such shadows might be
produced by other compact objects having a critical curve Guerrero:2021ues .
Indeed, since Nature could breed creatures beyond our imagination, it is
pertinent to characterize the zoo of BH mimickers Johnson-Mcdaniel:2018cdu
able to cast a shadow. Moreover, since two different BH (mimickers) of
comparatively similar masses are able to cast the same shadow Lima:2021las ,
there is a major effort in conceiving new physical signatures capable to
distinguish unmistakably a BH from a mimicker Cardoso:2019rvt ;
Guerrero:2021ues . The investigation of such signatures is of particular
interest given the present great opportunity to carry out precise tests of the
Kerr solution in the strong field regime EventHorizonTelescope:2020qrl . The
main aim of the present Letter is to bring forward one of such signatures,
which is driven by the existence of multiple critical curves in some BH
mimickers.
For the sake of this work, we shall consider one of the leading proposals for
BH mimickers, namely, the wormhole (WH) paradigm. Likewise BHs in the past,
WHs have awaken a special interest by decades in the community. Usually
interpreted as structures allowing to connect far away regions, rather than a
GR-framed solution they are a generic possibility in gravitational theories
based on a curved space-time. Since they naturally restore geodesic
completeness Carballo-Rubio:2019fnb , which is a deep-rooted problem in GR
BHs, and may even shed new light on quantum information problems
Maldacena:2013xja , they are worth bearing in mind. Hence, we shall discuss
three different WH scenarios where two critical curves are present. We shall
argue that the presence of the new critical curve in optically thin disks
generically brings in additional light rings not present in the Kerr solution,
some of which may yield a non-negligible contribution to the total luminosity
of the object. This claim will be further strengthened by explicitly working
out the shape and luminosities of such new light rings in a toy-model
consisting of a WH extension of the Schwarzschild solution, illuminated by
three different emission profiles of a geometrically thin accretion disk. The
observation of such new light rings would act as compelling evidence for the
existence of this kind of BH mimickers.

##  II Critical curves, ray-tracing and thin accretion disk

We consider a spherically symmetric (WH) geometry written in suitable
coordinates as:

$ds^{2}=-A(x)dt^{2}+A^{-1}(x)dx^{2}+r^{2}(x)d\Omega^{2}\ ,$ (1)

where the (non-vanishing) area of the two-spheres $S=4\pi r^{2}(x)$ is
parametrized in terms of the coordinate $x\in(-\infty,+\infty)$. Though
broadly astrophysical compact objects are expected to rotate, the size and
shape of the shadow and its boundary are weakly dependent on the spin and the
orientation of the observer Johannsen:2010ru ; Perlick:2021aok ;
Gralla:2019xty , so the spherical symmetry restriction will allow us a cleaner
scenario to capture the main observational novelties brought by the existence
of multiple critical curves. For a null geodesic and taking $\theta=\pi/2$, a
freedom granted by the spherical symmetry restriction, the corresponding
geodesic equation reads

$\dot{x}^{2}=\frac{1}{b^{2}}-V_{eff}(x)\ ,$ (2)

where $\dot{x}\equiv dx/d\lambda$ with $\lambda$ being an affine parameter and
$b=L/E$ the impact parameter ($E=A\dot{t}$ and $L=r^{2}(x)\dot{\phi}$ being
the photon’s conserved energy and angular momentum, respectively) whereas the
effective potential is given by:

$V_{eff}=\frac{A(x)}{r^{2}}\ .$ (3)

Unstable circular orbits (the so-called photon sphere) will exist as far as
$\dot{x}=0$ for every $\lambda$, such that the right-hand side of equation (2)
vanishes at the circular radius, given by $x_{m}$, and representing a maximum
of the effective potential (3), i.e.,
$V_{eff}(x_{ps})=1/b^{2},V_{eff}^{\prime}(x_{ps})=0,V_{eff}^{\prime\prime}(x_{ps})<0$.
This defines a critical impact parameter

$b_{c}=\sqrt{\tfrac{r^{2}(x_{ps})}{A(x_{ps})}}\ ,$ (4)

where a light ray would turn the object an infinite number of times. For the
Schwarzschild BH, one has $b_{c}=3\sqrt{3}M$ with $r_{ps}=x_{ps}=3M$.

We consider an accretion disk whose emission is confined to the equatorial
plane. In the optically thin limit, which seems to be a reliable enough
description for the accretion disk of supermassive BHs Johnson:2015iwg , each
pixel on the observer’s screen collects the integrated emission along a given
null geodesic. Thus, a light ray may intersect the disk several times in their
winding around the object, picking up additional brightness according to the
emission profile of the disk. Astrophysical observations of the intensity on
the observer’s screen are thus computed by backward ray-tracing the null
geodesics equation (2), suitably rewritten as the variation of the azimuthal
angle with respect to the radial coordinate:

$\frac{d\phi}{dx}=\frac{b}{r^{2}(x)\sqrt{1-\frac{b^{2}A(x)}{r^{2}(x)}}}\ .$
(5)

This procedure splits the light rays issued from the observer into two
classes: those with $b>b_{c}$ are deflected at a minimum distance to
asymptotic infinity, and those with $b<b_{c}$ spiral down until meeting the
event horizon (if any). The image on the observer’s screen is thus formed by
an infinite set of self-similar light rings characterized by the number of
photon (half-)orbits in their winding around the compact object. Since such
rings will be exponentially thinner and dimmer Luminet:1979nyg , the image of
the object will be largely dominated by the direct emission (a single
intersection with the disk) appearing as a thick bright lump of radiation. As
discussed in Gralla:2019xty , only the lensing ring (two intersections with
the front and back of the disk, respectively) produces a neatly visible
additional bright ring, while for the sum of higher-orbit contributions
(trajectories intersecting the disk at least thrice) only the first sub-ring
is barely visible (what the authors of Gralla:2019xty  dub as “photon ring”,
dismissing all the subsequent orbits). It is worth pointing out that, in those
models of accretion disks whose inner edge extends below the critical curve
(up to the horizon), there will be emissions from inside that region towards
the observer’s screen, effectively enlarging the spaces of
direct/lensed/photon ring trajectories inside the region $b<b_{c}$. Therefore,
the optical appearance of the object via its pattern of luminous and dark
regions will be highly influenced by the emission profile of the disk. At the
same time, and despite their relative dimness, the fact that the location of
higher-order rings are very sensitive to the background geometry rather than
to the details of the accretion disk, makes them suitable signatures to probe
the metric itself Wielgus:2021peu .

The geometrically thin accretion disk is described by the radiative transfer
(Boltzmann) equation for unpolarized photons Gold:2020iql . Neglecting
scattering and absorption effects, and assuming a source with an isotropic
emission in the rest frame of the disk with specific intensity
$I^{em}_{\nu}=I(x)$ and a frequency-independent emissivity, one finds that
$I_{\nu}^{em}/\nu^{3}$ is conserved along a null geodesic Eichhorn:2021iwq .
In the observer’s frame with photon’s frequency $\nu^{\prime}$, its intensity
is given by $I^{ob}_{\nu^{\prime}}=(\nu^{\prime}/\nu)^{3}I^{em}_{\nu}$. For
the geometries considered here this implies
$I^{ob}_{\nu^{\prime}}=A(x)^{3/2}I^{em}_{\nu}$, so that the integrated
intensity becomes $I^{ob}=\int
d\nu^{\prime}I_{\nu^{\prime}}^{ob}=A^{2}(x)I(x)$. The total intensity comes
from taking into account all the possible intersections of every photon with
the disk as:

$I=\sum_{m=1}^{3}A^{2}(x)I(x)_{|_{x=x_{m}(b)}}\ ,$ (6)

where the so-called transfer function $x_{m}(b)$ (with $m=1,2,3$ for
direct/lensed/photon ring trajectories) collects the relation between the
emission radius of the disk and the impact parameter $b$, with its slope
encoding the corresponding demagnification of the image.

Going beyond Schwarzschild spacetimes, more than one critical curve might
naturally arise. In such a case light rays will be able to spiral around the
additional critical curve(s) following a similar description of
direct/lensed/photon ring trajectories. Moreover, in the intermediate region
between every pair of critical curves the (local) minimum of the potential
will act as an anti-photon sphere, around which light rays may oscillate in
elliptical orbits Cardoso:2014sna . Therefore, a complex pattern of new
trajectories will arise in such objects with multiple critical curves, in such
a way that, depending on the emission profile of the disk and, in particular,
on how deeply it penetrates inside the new critical curve(s), one would expect
the appearance of new luminous and dark regions associated to the dominant
sources of emission (direct/lensed/photon ring). In the next section we shall
present three scenarios of WHs having two critical curves and work out the new
pattern of light rings for one of such objects using three toy-models of
accretion disk emissions.

##  III Light rings and shadows from WHs with multiple critical curves

###  III.1  Absolute maximum of the potential at the center

Let us consider the uniparametric toy-model defined by

$A(x)=1-\frac{2Mx^{2}}{(x^{2}+a^{2})^{3/2}}\hskip 5.69046pt;\hskip
5.69046ptr^{2}(x)=x^{2}+a^{2}$ (7)

which recovers the Schwarzschild solution in the limit $x\to\infty$, so $M$ is
the mass of the object as seen by a far away observer. This toy-model is
actually an extension of the well known Bardeen’s solution Bardeen  where the
radial function is infused with a bouncing behaviour at $x\to 0$ thanks to the
presence of the parameter $a$, which in turn restores geodesic completeness
and removes the presence of curvature divergences everywhere. We shall take a
theory-agnostic approach, in which (7) may arise in GR (supported e.g. by some
non-linear theory of electrodynamics) or in some modified gravity theory
Lobo:2020ffi . This family of solutions describes either two-horizon BHs with
a hidden WH inside it ($0<a/M\leq 4\sqrt{3}/9$) or traversable WHs otherwise.
The corresponding effective potential has always an absolute maximum at $x=0$
(the throat), while a second (local) maximum is present for $a/M\leq
2\sqrt{5}/5$. Hence, within $4\sqrt{3}/9<a/M\leq 2\sqrt{5}/5$ there are two
critical curves Tsukamoto:2021caq . For the sake of this work we shall choose
$a/M=6/7$, whose corresponding potential is depicted in Fig. 1. The full set
of images for every possible configuration within this family will be treated
in a separate publication.

Figure 1: The effective potential $V_{eff}(x)=A/r^{2}$ for the model (7) with
$a/M=6/7$ (blue). The inset panel depicts the outermost (local) maximum, while
the absolute maximum is located at the throat $x=0$, a feature which is
missing in the Schwarzschild solution (dashed black). Figure 2: The number of
half-orbits (in blue) as a function of $b$ (here $M=1$), as compared to the
Schwarzschild solution (dashed black). The two critical curves
$b_{c}^{1},b_{c}^{2}$ drive the direct/lensed/photon ring emissions, whose
corresponding regions are enclosed by the horizontal dashed lines. We also
depict the inner shadow limit, $b_{is}<b_{c}^{1}$, to be discussed later.

We implement the ray-tracing procedure by considering a disk located on the
equatorial plane (so that the observer lies on the north pole). The
(normalized) number of (half-)orbits is defined by $n\equiv\phi/2\pi$, so that
light rays in straight motion (outside the outer critical curve) is defined by
$n=1/2$, while direct/lensed/photon ring trajectories correspond to
$1/2<n<3/4$, $3/4<n<5/4$, and $n>5/4$, respectively. Since we have now two
critical curves with corresponding impact parameters $b_{c}^{1}/M=6/7$ and
$b_{c}^{2}/M\approx 4.589$, these three types of trajectories arise around
each of them, whose number of half-orbits is depicted in Fig. 2. The two
critical curves appear in this plot as two spikes such that in the region
$b_{c}^{1}<b<b_{c}^{2}$ the description of the direct/lensed/photon ring
emissions strongly deviates from the Schwarzschild spacetime expectations, as
shown in Fig. 3. The expected contribution of such emissions to the image
within the different impact parameter regions is hinted by the transfer
function $x_{m}(b)$ depicted in Fig. 4.

Figure 3: Ray-tracing of the trajectories ($M=1$) within the two critical
curves, $b_{c}^{1}<b<b_{c}^{2}$, where the different colours are identified
with their photon ring/lensed/direct character according to Fig. 2. The dashed
yellow and black circumferences correspond to the outer and inner critical
curves, while the black trajectories denote the inner shadow limit,
$b_{is}<b_{c}^{1}$. Figure 4: The transfer function $x_{m}(b)$ for the direct
(blue), lensed (orange) and photon ring (green) emissions.

We consider next three accretion disk canonical toy-models where the maximum
of the emission is attained at some relevant surface: i) innermost stable
circular orbit for time-like observers, ii) (outer) critical curve, and iii)
WH’s throat, respectively, being zero in the interior region to them, while
smoothly falling off outwards (see Guerrero:2021ues  for details of these
models). This is intended to simulate different types of (geometrically thin)
disks or, alternatively, different stages in the temporal evolution of the
same disk, and to collect and integrate the luminosity associated to the
direct/lensed/photon ring trajectories of the two critical curves in different
ways.

The optical appearances for these three accretion disk models are depicted in
Fig. 5. In the first model (left panel) we see five concentric neat circular
rings. The three outermost ones are akin to the standard light rings of the
Schwarzschild solution (associated to the direct/lensed/photon ring
trajectories of the outer critical curve), slightly modified in their
corresponding impact parameters and luminosities due to the geometry (7). The
additional two visible light rings are associated to photon ring/lensed
trajectories of the inner critical curve, collecting $\sim 2.8\%$ and $\sim
2.5\%$ of the total luminosity, respectively. Moreover, there is yet another
(innermost) light ring associated to further photon ring emission of the inner
critical curve but it is so dimmed ($\sim 10^{-6}$) that it is not visible at
naked eye in this figure. In the second model (middle panel) the disk emission
induces an overlapping between several contributions of these trajectories,
abruptly changing the distribution of bright and dark regions. Now the
innermost visible contribution is a new lensing ring associated to the inner
critical curve accounting for a $\sim 9.2\%$ of the luminosity budget, the
latter being largely dominated by the extended bright central ring (a mixing
of direct and inner photon ring emissions), with another visible thin ring in
the outermost edge. Finally, in the third model (right panel) there are
several blurred rings with intermediate dark-brown regions and the image is
dominated by the concentrated bright ring of radiation, where different
contributions associated to both critical curves are combined, enclosing a
central black region greatly diminished in size.

The bottom line of the discussion above is that, while a given background
geometry is the solely actor determining the location of the critical
curve(s), very different images are produced depending on the assumed emission
profile of the accretion disk, inducing different degrees of superposition of
the individual light rings on its interaction with the geometry of the
critical curve(s). Let us also point out here that the model (7) and its
parameter are chosen on purpose in order to grossly exaggerate the features
brought by the presence of the new critical curve as compared with the
expected images of the GR (Schwarzschild) BH, though much less radical
proposals do exist in the market Gan:2021pwu .

Figure 5: The optical appearance (viewed in face-on orientation) of the
traversable WH (7) with $a=6/7$ and $M=1$ for an accretion disk whose inner
edge extends to the innermost stable circular orbit for time-like observers
(left), to the outer critical curve (middle), and all the way down to the WH
throat (right). Three extra rings (one not visible) appear inner to those
present in the Schwarzschild solution, associated to the new lensed/photon
ring emissions. Depending on the accretion disk model, these rings can be
neatly separated from each other (left) or superimposed with the direct
emission (middle and right).

###  III.2 Accretion disk on the opposite side of the throat

For a BH, when the inner edge of the disk extends up to the horizon, the size
of the central brightness depression is reduced towards the inner shadow,
defined as the impact parameter $b_{is}<b_{c}^{1}$ below which a light ray
terminates on the event horizon without crossing the equatorial plane once
($n<1/4$ in Fig. 2). The fact that no light ray can be originated neither from
inside the horizon nor from the accretion disk itself makes this region become
an utter brightness depression. Moreover, being a feature uniquely ascribed to
the background geometry, simultaneous observations of it and of the critical
curve(s) might help to break the degeneracy of the shadow’s caster parameters
Chael:2021rjo .

Let us now consider a traversable WH described by a symmetric potential with
respect to the throat, with a single critical curve on each side. This can be
achieved, for instance, in thin-shell scenarios by gluing together two patches
of space-time $\mathcal{M}^{\pm}$ at the shell (the throat),
$\Sigma=\mathcal{M}^{+}\cap\mathcal{M}^{-}$ via a suitable junction conditions
formalism in different theories of gravity Senovilla:2013vra ; Olmo:2020fri .
Let us place our observer on $\mathcal{M}^{+}$, and assume an accretion disk
on $\mathcal{M}^{-}$ only. Light rays coming from the $\mathcal{M}^{-}$ side
with impact parameter $b<b_{is}$ will reach the mouth without having any
intersection with the disk, and cross it to reach the observer’s screen on
$\mathcal{M}^{+}$. Therefore, unless the back-reaction of the matter fields
threading the shell $\Sigma$ is capable to significantly alter the light ray’s
impact parameter, the image in the region $b<b_{is}$, despite not being pitch
black, will only be given by those rays coming from the $\mathcal{M}^{-}$ sky,
not capable to yield additional light rings. However, if there is an accretion
disk on $\mathcal{M}^{-}$ whose emission extends inside the critical curve
there, some of those light rays with $b_{is}<b<b_{c}$ may intersect that disk
several times before crossing the throat to yield additional light rings on
the $\mathcal{M}^{+}$ observer’s screen within that impact parameter region.
Moreover, if there is another accretion disk on $\mathcal{M}^{+}$ whose
emission also extends inside the critical curve on that side, one would expect
a superposition of the images originated on both sides on top of the direct
emission(s). Likewise in the example above of the absolute maximum of the
potential at the throat, whether these images will resolve in neatly separated
light rings will be model-dependent both on the background geometry as on the
features of the accretion disks on $\mathcal{M}^{\pm}$, as shown, for
instance, in the two rotating WH geometries considered in Paul:2019trt .

###  III.3 Reflection-asymmetric critical curves

Let us assume now the effective potential to have different shapes on
$\mathcal{M}^{\pm}$. These reflection-asymmetric WHs have been considered
recently in the literature Wielgus:2020uqz ; Guerrero:2021pxt ; Peng:2021osd .
Suppose again the observer to be located on $\mathcal{M}^{+}$ and the
asymmetric potential to have its absolute maximum on $\mathcal{M}^{-}$. Now
there is a new set of light trajectories within the range of impact parameters
$b_{c}^{-}<b<b_{c}^{+}$, with $b_{c}^{\pm}$ the critical impact parameters
associated to the maxima of the potentials on $\mathcal{M}^{\pm}$,
respectively. Such trajectories correspond to light rays originated on
$\mathcal{M}^{+}$, which cross the throat $\Sigma$, hit the potential slope on
$\mathcal{M}^{-}$, and get reflected back to $\mathcal{M}^{+}$. Therefore,
whenever the emission of the accretion disk penetrates this region (on
$\mathcal{M}^{-}$), additional light rings will appear associated to
lensed/photon ring trajectories, again depending on the emission profile of
the disk.

##  IV Conclusion and remarks

Though the application of Ockham’s razor bids us to admit that every time we
observe a shadow, a (Kerr) BH is the most likely candidate, the difficulty to
disentangle the effects of the background geometry and the accretion disk
modelling on such an image Vincent:2020dij  drives us to consider alternative
scenarios for the shadow caster, where clear cut observational signatures may
arise. In this regard, the next generation of experiments together with new
analytical tools might be capable to distinguish among those effects
Lara:2021zth . In this Letter we have argued that within the large zoo of BH
mimickers, those having multiple critical curves (illustrated here with
certain WH geometries) in optically thin scenarios generically display
additional light rings driven by the new lensed/photon ring emissions
associated to the new critical curve(s). Using geometrically thin toy-models
of the accretion flow we have shown that such new rings can be located in far-
removed regions of the impact parameter space, and contribute non-negligibly
to the total luminosity, even significantly larger than those of the infinite
sequence of strongly-lensed rings associated to the standard critical curve of
the Schwarzschild (Kerr) solution. While these features could facilitate their
detection, depending on the emission properties of the disk these new light
rings can be superimposed with the direct emission as well as with the
canonical rings. This can spoil a neat separation of the (old and new) light
rings, forming instead a reduced number of rings with wider amplitudes and
larger combined luminosities. As a consequence, the optical appearance of any
such mimickers may strongly deviate from the Schwarzschild BH one (as
illustrated with the images presented in Fig. 5). Since very long baseline
interferometry could be able to resolve out the diffuse but sharp
contributions from higher-order light rings in the near future Johnson:2019ljv
, the observation of any such new rings from multiple critical curves could
make a compelling case for the existence of non-Kerr beasts in the cosmic zoo.

## Acknowledgements

DRG is funded by the _Atracción de Talento Investigador_ programme of the
Comunidad de Madrid (Spain) No. 2018-T1/TIC-10431, and acknowledges further
support from the FCT projects No. PTDC/FIS-PAR/31938/2017 and PTDC/FIS-
OUT/29048/2017. DS-CG is funded by the University of Valladolid (Spain), Ref.
POSTDOC UVA20. This work is supported by the Ministerio de Ciencia e
Innovación (Spain) projects Nos. FIS2017-84440-C2-1-P, PID2019-108485GB-I00,
PID2020-116567GB-C21, and PID2020-117301GA-I00, the project PROMETEO/2020/079
(Generalitat Valenciana), and the Edital 006/2018 PRONEX (FAPESQPB/CNPQ,
Brazil, Grant 0015/2019). This article is based upon work from COST Action
CA18108, supported by COST (European Cooperation in Science and Technology).

## References

  * (1) R. M. Wald, General Relativity (The University of Chicago Press, 1984).
  * (2) P. S. Joshi, Gravitational collapse and spacetime singularities (CUP, New York, 2007).
  * (3) C. Bambi, Rev. Mod. Phys. 89 (2017) 025001.
  * (4) K. Akiyama, et al. [Event Horizon Telescope], Astrophys. J. Lett. 875 (2019) L1.
  * (5) J. P. Luminet, Astron. Astrophys. 75 (1979) 228.
  * (6) R. Narayan, M. D. Johnson and C. F. Gammie, Astrophys. J. Lett. 885 (2019) L33.
  * (7) H. Falcke, F. Melia and E. Agol, Astrophys. J. Lett. 528 (2000) L13.
  * (8) M. Guerrero, G. J. Olmo, D. Rubiera-Garcia and D. Sáez-Chillón Gómez, JCAP 08 (2021) 036.
  * (9) N. K. Johnson-Mcdaniel, A. Mukherjee, R. Kashyap, P. Ajith, W. Del Pozzo and S. Vitale, Phys. Rev. D 102 (2020) 123010.
  * (10) H. C. D. Lima, Junior., L. C. B. Crispino, P. V. P. Cunha and C. A. R. Herdeiro, Phys. Rev. D 103 (2021) 084040.
  * (11) V. Cardoso and P. Pani, Living Rev. Rel. 22 (2019) 4.
  * (12) D. Psaltis, et al. [Event Horizon Telescope], Phys. Rev. Lett. 125 (2020) 141104.
  * (13) R. Carballo-Rubio, F. Di Filippo, S. Liberati and M. Visser, Phys. Rev. D 101 (2020) 084047.
  * (14) J. Maldacena and L. Susskind, Fortsch. Phys. 61 (2013) 781.
  * (15) T. Johannsen and D. Psaltis, Astrophys. J. 718 (2010) 446.
  * (16) V. Perlick and O. Y. Tsupko, [arXiv:2105.07101 [gr-qc]].
  * (17) S. E. Gralla, D. E. Holz and R. M. Wald, Phys. Rev. D 100 (2019) 024018.
  * (18) M. D. Johnson, et al. Science 350 (2015) 1242.
  * (19) M. Wielgus, [arXiv:2109.10840 [gr-qc]].
  * (20) R. Gold, et al. Astrophys. J. 897 (2020) 148.
  * (21) A. Eichhorn and A. Held, JCAP 05 (2021) 073.
  * (22) V. Cardoso, L. C. B. Crispino, C. F. B. Macedo, H. Okawa and P. Pani, Phys. Rev. D 90 (2014) 044069.
  * (23) J, Bardeen, presented at GR5, Tiflis, U.S.S.R., and published in the conference proceedings in the U.S.S.R. (1968).
  * (24) F. S. N. Lobo, M. E. Rodrigues, M. V. d. S. Silva, A. Simpson and M. Visser, Phys. Rev. D 103 (2021) 084052.
  * (25) N. Tsukamoto, Phys. Rev. D 104 (2021) 064022.
  * (26) Q. Gan, P. Wang, H. Wu and H. Yang, Phys. Rev. D 104 (2021) 024003.
  * (27) A. Chael, M. D. Johnson and A. Lupsasca, Astrophys. J. 918 (2021) 6.
  * (28) J. M. M. Senovilla, Phys. Rev. D 88 (2013) 064015.
  * (29) G. J. Olmo and D. Rubiera-Garcia, Class. Quant. Grav. 37 (2020) 215002.
  * (30) S. Paul, R. Shaikh, P. Banerjee and T. Sarkar, JCAP 03 (2020) 055.
  * (31) M. Wielgus, J. Horak, F. Vincent and M. Abramowicz, Phys. Rev. D 102 (2020) 084044.
  * (32) M. Guerrero, G. J. Olmo and D. Rubiera-Garcia, JCAP 04 (2021) 066.
  * (33) J. Peng, M. Guo and X. H. Feng, [arXiv:2102.05488 [gr-qc]].
  * (34) F. H. Vincent, et al. Astron. Astrophys. 646 (2021) A37.
  * (35) G. Lara, S. H. Völkel and E. Barausse, [arXiv:2110.00026 [gr-qc]].
  * (36) M. D. Johnson, et al. Sci. Adv. 6 (2020) eaaz1310.