# R2SNet: Scalable Domain Adaptation for Object Detection in Cloud–Based
Robots Ecosystems via Proposal Refinement

Michele Antonazzi, Matteo Luperto, N. Alberto Borghese, Nicola Basilico All
authors are with the Department of Computer Science, University of Milan,
Milano, Italy<EMAIL_ADDRESS>
###### Abstract

We introduce a novel approach for scalable domain adaptation in cloud robotics
scenarios where robots rely on third–party AI inference services powered by
large pre–trained deep neural networks. Our method is based on a downstream
proposal–refinement stage running locally on the robots, exploiting a new
lightweight DNN architecture, R2SNet. This architecture aims to mitigate
performance degradation from domain shifts by adapting the object detection
process to the target environment, focusing on relabeling, rescoring, and
suppression of bounding–box proposals. Our method allows for local execution
on robots, addressing the scalability challenges of domain adaptation without
incurring significant computational costs. Real–world results on mobile
service robots performing door detection show the effectiveness of the
proposed method in achieving scalable domain adaptation.

##  I Introduction

Robot–assisted services are today present in a wide range of real–world
applications, including healthcare, logistics, domestic assistance, and
agriculture [1]. While becoming more and more ubiquitous, autonomous mobile
robots are facing a growing need to tackle increasingly complex perception and
decision–making tasks for which the recent wave of AI and deep learning offers
solutions of unprecedented potential, often available as very large Deep
Neural Networks (DNNs) pre–trained on public or third–party datasets.

The computational capabilities that such a need brings are at odds with the
typical profiles of mobile robots: not only they are devices with limited
resources, but they need to be. Keeping affordable hardware costs and
preserving energy consumption at operational time are mandatory requisites in
many real–world scenarios. This is the reason why offloading the
computationally demanding inference with DNNs is an emerging trend in the
field, for which third–party AI services deployed in the cloud are a
convenient solution. Such services have great capabilities, but, as many
robotic practitioners are well aware of, also have access constraints and
performance barriers. Constraints typically entail that they can only be
accessed with queries. Among performance limiting factors, domain shifts are
perhaps the most relevant to the field–AI paradigm that robots embody: the
data distribution encountered in their target environments can significantly
diverge from the distribution on which the cloud–based DNN has been trained.
This discrepancy can inevitably result in substantial performance degradation.

Figure 1: A general overview of the cloud–based scenario we consider.

Consider these challenges in the scope of a robotic ecosystem where multiple
independent units are deployed across different environments and rely on a
cloud–based DNN inference service. Assume that the robots’ working
environments are initially unknown and the number of robots in the system is
expected to increase by deploying new units in novel environments. Standard
_domain adaptation_ techniques [2, 3], where fine–tuning and feature alignment
are exploited to train models that can withstand the shift to a target domain,
face scalability issues in such a scenario. The fields of Cloud Robotics [4],
and more recently Fog Robotics [5], come in handy by studying
inference–serving solutions that distribute, in an adaptive way, the
computational and storage loads across robots and cloud services. However, the
application of domain adaptation techniques over these architectures is
subject to scalability issues. First, it requires full access to the cloud
DNN; this is not always feasible if the cloud DNN is provided by an external
vendor. Then, performing domain adaptation on the full DNN model would pose
significant costs; each time a new robot is deployed, a new DNN is to be
trained, deployed, and maintained, as it cannot be shared by multiple robots
after performing domain adaptation.

In this work, we focus on scalable domain adaptation, a problem at the
intersection of cloud robotics and deep learning that, despite being relevant
to many real–world settings, is still largely underexplored. We focus on the
task of object detection in the general scenario represented by Fig. 1: a set
of robots need to carry out such a task from RGB images acquired in their
respective environments. To such end, they rely on a general–purpose
pre–trained DNN, called here TaskNet, which is provided as a third–party cloud
service, making it accessible exclusively through queries.

The core contribution of our approach is to perform domain adaptation as an
efficient downstream proposal–refinement stage, running locally on the robots.
As we shall detail in Section III-A, this strategy is inspired by the
observation that state–of–the–art object detectors typically work by
generating dense sets (up to thousands) of bounding–box proposals which then
undergo heuristic post–processing via confidence thresholding and non–maximum
suppression [6]. Our findings indicate that a substantial portion of the
performance degradation due to domain shifts can be mitigated by introducing
before such post–processing heuristics a proposal–refinement step adapted to
the target environment. To such end, we introduce R2SNet, a novel lightweight
DNN architecture for proposal refinement that focuses on three different types
of corrective actions: relabeling, rescoring, and suppression of bounding
boxes. To carry out such a task, R2SNet leverages the acquired images and the
geometrical features of the corresponding bounding–box proposals, and it can
be run downstream and locally on the robot.

We evaluate this method in a real–world testbed where mobile service robots
must perform real–time _door detection_ , that is identifying the location and
status (open/closed) of doors/passages through visual recognition [7]. For
service robots, this object detection task is key for navigation, but also one
recognized as very much affected by domain shifts [8]. The obtained results
show how our method enables scalable adaptation, effectively mitigating the
performance losses due to domain shifts encountered with the general
pre–trained model, all while avoiding the need for substantial computational
costs in training and inference.

##  II Related Works

Cloud Robotics [4] is an active area of research focusing on engineering the
distribution of storage and computational tasks away from robotic platforms to
web–enabled architectures [9]. In the last years, this area faced the wave of
cyber–physical systems’ increasing reliance on large, pre–trained DNNs. Such
models pose substantial computational demands that fostered the development of
strategies for distributing their workload to the edge, towards ecosystems
where edge computing and deep learning become interlinked. Object detection
represents one of the most significant testbeds against this background [10],
[11].

One of the mainstream approaches for cloud–based DNN workload distribution is
Model Splitting [12] where, essentially, the model is divided into two or more
portions that are run collaboratively across the network. Examples of this
method for object detection are [13] where YOLOv3 undergoes a process of
cloud–edge distribution and [14] where inference follows a hierarchical
structure from the cloud to the end device.

A series of works, falling under the umbrella of “Fog Robotics”, investigated
solutions based on a continuum of computing resources from robots to cloud
data centers. This paradigm is becoming increasingly widespread [15], with
distributed object detection (typically in synergy with grasping) being among
its real–world challenges. Examples of works on this line include [5], where
models are initially trained in the cloud and subsequently adapted at the
edge, [16] where authors focus on reducing latency by means of a
Q–learning–based policy for load balancing, and [17] where object detection
based on SSD [18] is served to the robot from a fog node cluster whose
resources can be adapted to guarantee service quality. Other examples of
similar offloading strategies for object detection in mobile robots have been
proposed in [19] and [20].

Most of these works primarily focus on enhancing service quality but overlook
the challenge of scalable domain adaptation in the constrained cloud setting
we adopt. In this paper, we directly address this issue with an architecture
related to the one proposed in [21] where cooperation between a large
cloud–based model and a smaller one operating locally on the robot is
exploited. Our method differentiates in the role and design of the smaller
model. In such work, the smaller model is essentially a scaled–down, less
accurate variant of the larger one, aimed at reducing costs. In contrast, our
approach enhances the smaller model’s role to not just serve as a
cost–effective alternative but to specifically refine and adapt the cloud
model’s predictions for the robot’s unique operational environment in a
scalable way.

##  III Method

###  III-A Proposals Filtering in Object Detection

Object Detection (OD) amounts to identifying the location and dimension of
objects in an image. Deep learning is today the leading approach to building
detectors, which are typically based on architectures that analyze the input
image through different stages to ensure its comprehensive coverage [22].
Two–stage detectors (such as Faster R–CNN [23]) use a Region Proposal Network
(RPN) to predict, in a first stage, proposals of bounding boxes from
multi–scale image embeddings. In the second stage, such proposals are
classified into object categories. Differently, one–stage models (such as YOLO
[24]) directly predict object classes for a set of predefined bounding boxes
called _anchors_ , which uniformly cover the image with multiple scales and
sizes.

Both architectures share a characteristic: they produce many overlapping
bounding–box proposals, typically numbering in the thousands, which are
independently scored using the image’s features. To distill meaningful
detections from this dense set, a heuristic two–step post–processing is
commonly executed. The first step, called Non–Maximum Suppression (NMS) [6],
iteratively selects pairs of proposals whose Intersection over Union area
(IoU) exceeds a threshold $\rho_{IoU}$ and suppresses the one with the lowest
confidence. In the second step, any proposal with a confidence lower than a
threshold $\rho_{c}$ is also discarded.

For achieving scalable adaptation, we suggest relocating the post–processing
step to operate locally on each robot. This entails integrating it as a
downstream module of a global cloud–based object detector we call TaskNet (see
Fig. 1) which has been configured to return raw proposals by modifying
hyperparameters. Additionally, we augment this post–processing by
incorporating R2SNet, a lightweight deep architecture tailored to the robot’s
target environment.

From an image $x$, we obtain from the TaskNet a set of raw proposal
$\hat{Y}=\\{\hat{y}\\}$, with

$\hat{y}=\big{[}\hat{c}_{x},\hat{c}_{y},\hat{w},\hat{h},\hat{c},\text{hot}(\hat{o})\big{]}_{1\times
f}$ (1)

where $\hat{c}_{x},\hat{c}_{y}\in[0,1]$ are the center coordinates,
$\hat{w},\hat{h}\in[0,1]$ represent width and height, $\hat{c}$ is the
confidence, $\hat{o}\in\mathcal{O}$ is an integer indicating the object
category, and $\text{hot}(\cdot)$ is its one–hot encoding (so
$f=5+|\mathcal{O}|$). Once received by the robot, the $k$ most confident
proposals, where $k\gg O$, with $O$ indicating the maximum number of
identifiable objects in an image, are given as input to R2SNet. Before
presenting its architecture, we examine the three primary types of
interventions along which the network is trained and used: Relabeling,
Rescoring, and Suppression (hence the acronym R2SNet). In the remainder of the
paper, we focus on a specific object detection task, _door detection_ , as it
is particularly significant for this task, as discussed in the examples below.
However, our considerations are general to other detection tasks for
autonomous robots.

a – Relabeling |  |  |
---|---|---|---
b – Rescoring |  |  |
c – Suppression |  |  |
| TaskNet | TaskNet + NMS | TaskNet + R2SNet + NMS
Figure 2: R2SNet refinements in filtering dense proposals, compared to
standard post–processing. Green/red bounding boxes are open/closed doors.

#### Relabeling

Frequently, a TaskNet generates several overlapping proposals over the same
target object; some of these proposals often have contrasting labels. Fig. 2a
shows an example of different overlapping proposals that label the same door
both as closed and open. These errors are frequent when involve objects that
might resemble each other (e.g., open and closed doors or chairs and
armchairs). The standard post–processing based on NMS would select the
proposal with the highest confidence disregarding the correctness of its
object category. In our method, we improve on this by relabeling all
overlapping proposals to a single category, forcing a consensus. Also, we
identify isolated proposals not overlapping with any others as spurious. Based
on our empirical observations, these isolated proposals often correspond to
errors in object localization. Consequently, we relabel them as background, an
additional category introduced in this stage.

#### Rescoring

It is well–known that confidence scores may not consistently reflect the
actual uncertainty, and thus the likelihood of correctness, of the proposals
computed with TaskNet [25]. When a poorly localized proposal receives high
confidence, NMS might erroneously reject nearby proposals that better match
with the object. Conversely, if a properly localized proposal receives low
confidence, the thresholding step could erroneously discard it. We observed
this phenomenon in challenging instances, such as the one depicted in Fig. 2b,
where an open door is partially hidden behind a corner, often leading to
errors. To address this, we correlate the confidence of each proposal to the
IoU area they have with the best overlapping ground truth box. In this way,
the IoU threshold $\rho_{IoU}$ becomes the only hyperparameter for the
post–processing techniques.

#### Suppression

Other frequent errors occur when dense sets of proposals are situated in parts
of the image where no objects are present. This might happen because the
features in these regions mimic an object category that the detector is
trained to identify. For instance, Fig. 2c highlights instances where cabinets
(or windows) are misclassified as doors. While relabeling partially mitigates
this issue, we introduce a suppression phase to directly address it by
learning a feature embedding from the image portion of each proposal to
differentiate between background areas and those containing an object.

###  III-B R2SNet

Figure 3: The R2SNet architecture. Batch normalization and ReLU activation
functions are applied to all layers of the shared MLPs.

First, given the $k$ most confident proposals computed by the TaskNet, R2SNet
extracts a matrix of descriptors
$BD=\big{[}\hat{y}_{1},\hat{y}_{2},\ldots,\hat{y}_{k}\big{]}_{k\times f}$ by
stacking the vectors defined in Eq. 1. Additionally, it extracts a feature
vector from the portion of the image $x$ corresponding by each proposal, using
a convolutional architecture we call BFNet (Bounding–box Feature Network,
detailed in Sec. III-C), to compute a matrix of image descriptors
$ID_{[k\times 8]}$. $BD$ and $ID$ can be seen as projections of the $k$ most
confident bounding boxes in two distinct spaces $\mathbb{R}^{f}$ and
$\mathbb{R}^{8}$, which are meant to capture their geometrical and visual
features.

Given these preliminaries, R2SNet (depicted in Fig. 3) is inspired by PointNet
[26], which is designed to classify and segment dense point clouds and to be
invariant to input permutation (proposals, in our setting). R2SNet processes
$BD$ and $ID$ with two symmetric sub–networks. At first, each of them maps the
input to a high–dimensional space using a Multi–Layer Perceptron (MLP) shared
across the $k$ proposal descriptors, to obtain local features $L_{[k\times
l]}$. In the MLPs, the same weights are applied to each descriptor, making the
size of the network fixed regardless of the number $k$ of proposals. After
this step, the local features are expanded again with another MLP and then
aggregated using $\max$, to obtain a global feature vector $G_{[1\times g]}$.
This last one is concatenated with each row of $L_{[k\times l]}$ and then
mixed with a shared MLP, obtaining an embedding $LG_{[k\times 128]}$ that
represents both local and global features of the $k$ proposals. The outputs
$LG_{BD}$ and $LG_{ID}$ of the two sub–networks are fed into three heads to
handle the relabeling, rescoring, and suppression of the proposals.

Figure 4: The BFNet architecture.

We denote as $Y$ the set of ground truth bounding boxes for image $x$ where
each $y\in Y$ is encoded as per Eq. 1 by setting $c=1$. We define a matching
rule to assign a proposal $\hat{y}$ to a ground–truth bounding box
$\hat{y}^{GT}=\arg\max_{y\in Y}a_{IoU}(\hat{y},y)$, where $a_{IoU}(\hat{y},y)$
is the IoU area between $\hat{y}$ and $y$.

The relabeling head, starting from the concatenation of $LG_{BD}$ and
$LG_{ID}$, assigns to each proposal $\hat{y}$ the probabilities for each
object class in the set $\mathcal{O}\cup\\{\texttt{background}\\}$, producing
an output $O^{cls}_{[k\times|\mathcal{O}|+1]}$. This head is trained with the
following log–loss:

$\mathcal{L}_{cls}(O^{cls})=-\frac{1}{k}\sum_{p=1}^{k}\log\big{(}O^{cls}_{p}\big{)}\cdot\text{hot}\big{(}\hat{o}_{p}\big{)},$
(2)

where $(\cdot)$ is the dot product and $\hat{o}_{p}$ is the true class for the
$p$-th proposal determined by our matching rule:

$\hat{o}_{p}=\begin{cases}Class(\hat{y}^{GT}_{p})&\text{if
}a_{IoU}(\hat{y}^{GT}_{p},\hat{y}_{p})\geq\rho_{IoU}\\\
\texttt{background}&\text{otherwise.}\end{cases}$

The rescoring head aligns the confidence of a proposal $\hat{y}$ to its IoU
area with its associated ground truth $\hat{y}^{GT}$. To achieve this, the
confidence score $c\in[0,1]$ is discretized into $10$ intervals. The rescoring
head is then tasked with predicting the likelihood that the confidence score
falls within each of these intervals, yielding an output matrix
$O^{conf}_{[k\times 10]}$. For training, we construct a target vector
$v(\hat{y})$ for a proposal $\hat{y}$, whose values peak at the interval
corresponding to the IoU score between $\hat{y}$ and its corresponding ground
truth $\hat{y}^{GT}$, and decrease in a Gaussian–like manner on either side of
the peak. In such a way, we obtain a measure of the error that increases with
the distance between the predicted and true peaks. This error is adopted for
the rescoring loss:

$\mathcal{L}_{res}(O^{conf})=\frac{1}{k}\sum_{p=1}^{k}\|O^{conf}_{p}-v(\hat{y}_{p})\|_{1}.$
(3)

The confidence assigned to each $\hat{y}_{p}$ is $\arg\max_{j}O^{conf}_{p,j}$.

Finally, the suppression head is trained with a loss obtained by adapting Eq 2
for binary classification between proposals that correspond to an object
(those for which $\hat{o}_{p}\neq\texttt{background}$) and those falling on
the background.

###  III-C BFNet

BFNet, shown in Fig. 4, guides R2SNet to identify those proposals that are
wrongly placed on the background and that can be suppressed. This task is
challenging when only the descriptors $BD$ (location, confidence, and class)
are used, thus image descriptors $ID$ are needed. BFNet partitions the input
image with a low–resolution grid mask $M_{[W\times H]}$. Then it extracts a
feature encoding $IF_{[8\times W\times H]}$ ($8$ channels for each cell),
which is mapped to each proposal’s region of interest.

More precisely, BFNet extracts a multi–scale feature hierarchy of the input
image using a CNN–based backbone with residual connections [27]. The last
three embeddings are re–scaled with dimensions $[W\times H]$,
$\big{[}\frac{W}{2}\times\frac{H}{2}\big{]}$, and
$\big{[}\frac{W}{4}\times\frac{H}{4}\big{]}$ using adaptive average pooling
layers. To aggregate features at different scales, Feature Pyramid Networks
[28] (FPN) are commonly used. Differently from what is done in FPNs, to have
more descriptive features, each embedding is processed by three parallel
convolutional backbones and step–by–step top–down aggregated through
upsampling and summation. The resulting embeddings are concatenated and mixed
through convolution to generate a feature map $IF$.

Then, we need to obtain the portion of $IF$ covered by each proposal. Rather
than iteratively slicing $IF$ according to each bounding box coordinates, we
perform a faster parallel end–to–end mask generation process. More precisely,
we use a series of MLPs that produces a binary mask $M_{[W\times
H]}^{\hat{y}}$ for each proposal $\hat{y}$ where an element is set to 1 (0) if
inside (outside) the area of $\hat{y}$. This mask is used to suppress the
features of $IF$ exceeding the bounding box’s boundaries. First, BFNet
receives in input a matrix
$BC=[\phi(\hat{y}_{1})\ldots\phi(\hat{y}_{k})]_{k\times 4}$, where
$\phi:\mathbb{R}^{f}\rightarrow\mathbb{N}^{4}$ encodes an input proposal
$\hat{y}$ to a vector $[x_{0},y_{0},x_{1},y_{1}]$ containing the coordinates
of the bottom–left and top–right corners in the grid mask $M_{[W\times H]}$.
It then computes, for each proposal $\hat{y}$, four binary grids defined as

$\displaystyle\begin{split}&M^{x_{j}}=\scalebox{1.2}{$\mathds{1}$}_{\scalebox{0.7}{$\leq$}}\Big{(}(-1)^{j}(x_{j}-A)\Big{)}\\\
&M^{y_{j}}=\scalebox{1.2}{$\mathds{1}$}_{\scalebox{0.7}{$\leq$}}\Big{(}(-1)^{j}(y_{j}-B^{T})\Big{)}\end{split}$
(4)

for $j\in\\{0,1\\}$, where $A=\operatorname{diag}(I_{H})\times[0\ldots W-1]$
and $B=\operatorname{diag}(I_{W})\times[H-1\ldots 0]$. We obtain the matrices
using four MLPs, each comprising one input and $W\times H$ output neurons. The
weights are initialized according to Eq. 4 and remain fixed during the
training process. The mask of each proposal $\hat{y}$, obtained as

$M^{\hat{y}}_{[W\times H]}=\bigwedge\limits_{j=0}^{1}M^{x_{j}}\wedge
M^{y_{j}},$

is combined with the embedding $IF$ to suppress the features outside the
bounding box boundaries. The results are then compressed along the last two
dimensions with a $\max$ operation, obtaining $ID_{[k\times 8]}$ that encodes
the image descriptors of each proposal for R2SNet.

Before training the whole R2SNet, BFNet is pre–trained for addressing a
low–resolution binary segmentation task. The image features $IF_{[8\times
W\times H]}$ are convoluted into a binary grid mask $M^{seg}_{[2\times W\times
H]}$ obtained by training on this loss:

$\mathcal{L}_{seg}(M^{seg})=-\frac{1}{WH}\sum_{w=1}^{W}\sum_{h=1}^{H}\log\big{(}M^{seg}_{w,h}\big{)}\cdot\text{hot}\big{(}l_{w,h}\big{)},$
(5)

where the ground truth label for each cell $l_{w,h}$ is

$l_{w,h}=\begin{cases}1&\text{if }\exists~{}y\in Y\mid\phi(y)\text{ contains
the cell }w,h\\\ 0&\text{otherwise. }\end{cases}$

$e_{1}$ $e_{2}$ $e_{3}$ $e_{4}$ $\overline{e}$ Exp. mAP$\uparrow$
$\mathbf{TP}$$\uparrow$ $\mathbf{FP}$$\downarrow$ $\mathbf{BFD}$$\downarrow$
mAP$\uparrow$ $\mathbf{TP}$$\uparrow$ $\mathbf{FP}$$\downarrow$
$\mathbf{BFD}$$\downarrow$ mAP$\uparrow$ $\mathbf{TP}$$\uparrow$
$\mathbf{FP}$$\downarrow$ $\mathbf{BFD}$$\downarrow$ mAP$\uparrow$
$\mathbf{TP}$$\uparrow$ $\mathbf{FP}$$\downarrow$ $\mathbf{BFD}$$\downarrow$
mAP$\uparrow$ $\mathbf{TP}$$\uparrow$ $\mathbf{FP}$$\downarrow$
$\mathbf{BFD}$$\downarrow$ TaskNet 33 41% 10% 13% 27 33% 5% 18% 13 24% 7% 34%
47 47% 6% 14% 30 36% 7% 20% $\text{R2S}^{30}_{25}$ 39 50% 7% 11% 30 36% 4% 10%
20 26% 7% 12% 58 64% 4% 11% 37 44% 6% 11% $\text{R2S}^{30}_{50}$ 40 49% 8% 9%
35 40% 6% 6% 22 29% 7% 10% 59 63% 5% 9% 39 45% 6% 9% $\text{R2S}^{30}_{75}$ 49
54% 5% 7% 35 39% 6% 6% 26 31% 8% 10% 62 62% 1% 5% 43 46% 5% 7%

TABLE I: R2SNet performance evaluation when trained with an increasing amount
of data. $\overline{e}$ is the average.

##  IV Experimental Evaluation

###  IV-A Experimental Setting

Training is performed using 2 publicly available datasets (see [7, 8] for more
details) for door detection in RGB images. The first one, _DeepDoors2_
($\mathcal{D}_{\textbf{{DD2}}}$) [29], has 3K real–world images containing
doors, taken from a human perspective. The second one,
$\mathcal{D}_{\textbf{{G}}}$, is taken in 10 environments of Gibson [30] and
contains around 5K photorealistic images acquired from the viewpoint of a
mobile robot. The experimental evaluation is performed on a third, real–world,
dataset, $\mathcal{D}_{\textbf{{real}}}$, collected with our Giraff–X robot
(Fig. 1) [31]; it contains four runs collected when different indoor
environments $e_{*}$ (three university facilities and an apartment, see Fig.
2) have been fully mapped. All datasets have
$\mathcal{O}=\\{\texttt{closed},\texttt{open}\\}$. We split each run in each
environment of $\mathcal{D}_{\textbf{{real}}}$ in $75/25\%$ for
training/testing R2SNet.

We use a Faster R–CNN [23] as TaskNet; including its ResNet–50 [27] backbone,
it has 41M parameters. We train the TaskNet on the full
$\mathcal{D}_{\textbf{{DD2}}}$ and $\mathcal{D}_{\textbf{{G}}}$ for 60 epochs
with a batch size of 4. We then deploy it for inference disabling the NMS and
thresholding.

R2SNet needs to be trained from scratch on a large sample of dense proposals
obtained by a TaskNet on unseen data; to do so, we augment
$\mathcal{D}_{\textbf{{DD2}}}$ and $\mathcal{D}_{\textbf{{G}}}$ as in the
following. For each image, alongside the ground–truth bounding boxes of doors,
we must include the corresponding TaskNet proposals. To do this, generating
these proposals using images unseen by TaskNet during its training is key,
ruling out the use of the reference TaskNet trained on the full
$\mathcal{D}_{\textbf{{DD2}}}$ and $\mathcal{D}_{\textbf{{G}}}$. To overcome
this, we train 11 versions of Faster R–CNN, dividing the datasets into 11
segments (the first includes $\mathcal{D}_{\textbf{{DD2}}}$, the others
contain images from one of the $10$ environments of
$\mathcal{D}_{\textbf{{G}}}$), using a leave–one–out approach. Each TaskNet,
trained on $10$ segments, is used to extract proposals from the remaining,
unseen, one. R2SNet is thus pre–trained with the dataset obtained by combining
the 11 segments. We run a first pass training only BFNet using
$\mathcal{L}_{seg}$, then in a second pass we train the whole architecture
using
$\mathcal{L}_{R2S}=\mathcal{L}_{cls}+\mathcal{L}_{conf}+\mathcal{L}_{sup}$
(both passes with 60 epochs and batch size of 16, $k=30$, and $W=H=32$).

The pre–trained R2SNet is adapted with data specific to its deployment
environment using examples from $\mathcal{D}_{\textbf{{real}}}$. We label this
customized version as $\text{R2S}^{\\#proposals}_{\\#data}$, where the
superscript denotes the number of proposals $k$ per training example, and the
subscript indicates the percentage of data utilized relative to the total
available data in $\mathcal{D}_{\textbf{{real}}}$ from the deployment
environment. In particular, we assessed the impact of using $10$, $50$, and
$100$ proposals, alongside $25\%$, $50\%$, and $75\%$ of the data (which
correspond to $\approx 80$, $160$, and $240$ training images, respectively).
The remaining $25\%$ of the data are used for testing the R2SNet, using the
TaskNet as a baseline. We apply a NMS step to the TaskNet and TaskNet+R2SNet
proposals, setting $\rho_{IoU}=50\%$ and $\rho_{c}=75\%$ for the former and
$\rho_{IoU}=\rho_{c}=50\%$ for the latter. Note that the domain adaptation of
R2SNet is performed once, using the data acquired during the initial
deployment of the robot, and is used later on for the whole operative life of
the robot.

The performance metrics are the mean Average Precision [32] overall object
categories (mAP) with 3 additional indicators (formally defined in [7])
measuring the percentage of doors detected with the correct (wrong) label
denoted as $TP$ ($FP$) and the rate between the background detections (i.e.,
false positives detections placed on the background) and the total number of
ground truth objects, named _BFD_. We release the implementation of R2SNet and
the code to run the experiments in a publicly available
repository111https://aislabunimi.github.io/r2snet.

###  IV-B Results

Fig. 5 shows the performance of $\text{R2S}_{75}^{k}$ varying the number of
proposals $k$. A value $k\in\\{30,50\\}$ improves the mAP of $\approx 45\%$,
while also reducing both $FP$ and $BFD$, showing the effectiveness of our
R2SNet. Despite the filtering becomes challenging due to the high number of
noisy low–confidence proposals, a higher value of $k=100$ results in a further
increase of the mAP, at the expense of higher $BFD$ (that are still close to
those obtained by the TaskNet). Interestingly, also a low value of $k=10$ has
some positive aspects; while it does not improve the mAP, it still reduces the
$FP$ and $BFD$. Fig. 2 shows the benefits brought by the
$\text{R2S}_{75}^{100}$ refinement to the TaskNet raw proposals.

Another interesting remark, confirming the solidity of our approach, comes
from evaluating the results when a different and increasing number of samples
obtained in the robot target environment are used to train R2SNet, as shown in
Table I. For this test, we used $k=30$. Even with a few examples,
$\text{R2S}^{30}_{25}$ increases both the mAP and $TP$ of $\approx 20\%$ while
halving the percentage of $BFD$. Of course, exposing the R2SNet to a higher
number of examples, as in $\text{R2S}^{30}_{50-75}$, increases performances,
but our findings suggest that a few examples are enough for R2SNet to improve
the TaskNet performance on the target environment.

$\displaystyle e_{1}$$\displaystyle e_{2}$$\displaystyle e_{3}$$\displaystyle
e_{4}$$\displaystyle\overline{e}$$0$$20$$40$$60$3132434652

mAP

TaskNet$\text{R2S}^{10}_{75}$$\text{R2S}^{30}_{75}$$\text{R2S}^{50}_{75}$$\text{R2S}^{100}_{75}$

$\displaystyle e_{1}$$\displaystyle e_{2}$$\displaystyle e_{3}$$\displaystyle
e_{4}$$\displaystyle\overline{e}$40200204060803734465360Environment

Additional
---
indicators

$TP$$FP$$BFD$

Figure 5: $\text{R2S}_{75}^{k}$ performance when varying $k$ expressed with
the mAP (top row) and the additional indicators (bottom row).

To further evaluate the contribution of each network’s component, we conduct
an ablation study where the relabeling, rescoring, and refinement heads are
incrementally activated. To do this, we use as a reference
$\text{R2S}_{75}^{100}$, which has the best performances. Table II reports the
results averaged over the 4 environments. From this analysis, it can be seen
how the relabeling head alone ensures a significant mAP and $TP$ improvement
while reducing the $FP$ and $BFD$. The use of the rescoring head similarly
improves the mAP and $TP$, without reducing $FP$ and $BFD$s. The best overall
performance is shown when all three heads are used. However, the suppression
head has less impact on the performance than the other two heads; still, it is
needed for training, as disabling it during training increases the $BFD$ of
the $3\%$.

|  |  | $\overline{e}$
---|---|---|---
Rel. | Res. | Sup. | mAP$\uparrow$ |  $\mathbf{TP}$$\uparrow$ |  $\mathbf{FP}$$\downarrow$ |  $\mathbf{BFD}$$\downarrow$
|  |  | 34 | 44% | 10% | 35%
✓ |  |  | 44 | 48% | 4% | 6%
| ✓ |  | 41 | 54% | 15% | 34%
|  | ✓ | 37 | 43% | 9% | 14%
✓ | ✓ |  | 52 | 61% | 6% | 20%
✓ |  | ✓ | 44 | 47% | 4% | 5%
| ✓ | ✓ | 41 | 53% | 15% | 31%
✓ | ✓ | ✓ | 52 | 60% | 6% | 19%
TABLE II: Ablation study results.

Our R2SNet has 8M parameters. To assess its computational demands for
inference, we installed it on an edge device, an NVIDIA Jetson TX2, mounted on
the Giraff–X robot. On this hardware, commonly available in service robots,
R2SNet demonstrates remarkable efficiency, processing images at a rate of 16.7
Hz on the GPU and 2.6 Hz on the CPU. As a reference, TaskNet processes at
significantly lower frequencies of 1.1 Hz and 0.06 Hz, respectively, on the
same platforms. The R2SNet training with a RTX 3090 GPU took a few minutes.
These results show how our method can be used in real–time on a mobile robot,
illustrating R2SNet’s capability for efficient deployment in robots that
utilize edge devices, even those without a GPU.

##  V Conclusions

In this work, we presented a scalable domain adaptation solution for robotics
ecosystems using a shared third–party object detection service. We propose
R2SNet, a novel lightweight architecture to refine the proposals locally in
the robot to mitigate the performance degradation caused by domain shifts. We
validate our framework with real–world experiments addressing the door
detection task.

In future work, we will extend our approach enabling the robots to upload
domain–independent images to enhance the TaskNet performance. Furthermore, we
plan to investigate techniques for adaptive learning for evolving
environments.

## Acknowledgements

This work was partially funded by Grant Number G53D23002860006 - PRIN2022, by
the Italian Ministery of Research and University.

## References

  * [1] M. B. Alatise and G. P. Hancke, “A review on challenges of autonomous mobile robot and sensor fusion methods,” _IEEE Access_ , vol. 8, pp. 39 830–39 846, 2020.
  * [2] P. Oza, V. A. Sindagi, V. V. Sharmini, and V. M. Patel, “Unsupervised domain adaptation of object detectors: A survey,” _IEEE Trans. Pattern Anal. Mach. Int._ , 2023.
  * [3] Y. Lee, A. S. Chen, F. Tajwar, A. Kumar, H. Yao, P. Liang, and C. Finn, “Surgical fine-tuning improves adaptation to distribution shifts,” in _Proc. ICLR_ , 2023.
  * [4] G. Hu, W. P. Tay, and Y. Wen, “Cloud robotics: architecture, challenges and applications,” _IEEE Netw._ , vol. 26, no. 3, pp. 21–28, 2012.
  * [5] A. K. Tanwani, N. Mor, J. Kubiatowicz, J. E. Gonzalez, and K. Goldberg, “A fog robotics approach to deep robot learning: Application to object recognition and grasp planning in surface decluttering,” in _Proc. ICRA_. IEEE, 2019, pp. 4559–4566.
  * [6] J. Hosang, R. Benenson, and B. Schiele, “Learning non-maximum suppression,” in _Proc. CVPR_ , July 2017.
  * [7] M. Antonazzi, M. Luperto, N. A. Borghese, and N. Basilico, “Development and adaptation of robotic vision in the real-world: the challenge of door detection,” 2024.
  * [8] M. Antonazzi, M. Luperto, N. Basilico, and N. A. Borghese, “Enhancing door-status detection for autonomous mobile robots during environment-specific operational use,” in _Proc. ECMR_ , 2023.
  * [9] M. Afrin, J. Jin, A. Rahman, A. Rahman, J. Wan, and E. Hossain, “Resource allocation and service provisioning in multi-agent cloud robotics: A comprehensive survey,” _IEEE Commun. Surv. Tutor._ , vol. 23, no. 2, pp. 842–870, 2021.
  * [10] X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” _IEEE Commun. Surv. Tutor._ , vol. 22, no. 2, pp. 869–904, 2020.
  * [11] Y. Guo, B. Zou, J. Ren, Q. Liu, D. Zhang, and Y. Zhang, “Distributed and efficient object detection via interactions among devices, edge, and cloud,” _IEEE Trans. Multimed._ , vol. 21, no. 11, pp. 2903–2915, 2019.
  * [12] S. Abuadbba, K. Kim, M. Kim, C. Thapa, S. A. Camtepe, Y. Gao, H. Kim, and S. Nepal, “Can we use split learning on 1d cnn models for privacy preserving training?” in _Proc. ASIACCS_ , 2020, pp. 305–318.
  * [13] I. Chakroun, T. Vander Aa, R. Wuyts, and W. Verachtert, “Distributing intelligence for object detection using edge computing,” in _Proc. CLOUD_. IEEE, 2021, pp. 681–687.
  * [14] S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Distributed deep neural networks over the cloud, the edge and end devices,” in _Proc. ICDCS_. IEEE, 2017, pp. 328–339.
  * [15] J. Ichnowski, K. Chen, K. Dharmarajan, S. Adebola, M. Danielczuk _et al._ , “Fogros2: An adaptive platform for cloud and fog robotics using ros 2,” in _Proc. ICRA_ , 2023, pp. 5493–5500.
  * [16] A. K. Tanwani, R. Anand, J. E. Gonzalez, and K. Goldberg, “Rilaas: Robot inference and learning as a service,” _IEEE RA-L_ , vol. 5, no. 3, pp. 4423–4430, 2020.
  * [17] D. Vinod and P. SaiKrishna, “Development of an autonomous fog computing platform using control-theoretic approach for robot-vision applications,” _ROBOT AUTON SYST_ , vol. 155, p. 104158, 2022.
  * [18] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in _Proc. ECCV_. Springer, 2016, pp. 21–37.
  * [19] W. J. Beksi, J. Spruth, and N. Papanikolopoulos, “Core: A cloud-based object recognition engine for robotics,” in _Proc. IROS_. IEEE, 2015, pp. 4512–4517.
  * [20] M. Penmetcha, S. S. Kannan, and B.-C. Min, “Smart cloud: Scalable cloud robotic architecture for web-powered multi-robot applications,” in _Proc. SMC_. IEEE, 2020, pp. 2397–2402.
  * [21] S. Chinchali, A. Sharma, J. Harrison, A. Elhafsi, D. Kang, E. Pergament, E. Cidon, S. Katti, and M. Pavone, “Network offloading policies for cloud robotics: a learning-based approach,” _Auton. Robot._ , vol. 45, no. 7, pp. 997–1012, 2021.
  * [22] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” _Proc. IEEE_ , vol. 111, no. 3, pp. 257–276, 2023.
  * [23] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” _Adv. Neur. In._ , vol. 28, no. 6, 2015.
  * [24] A. Farhadi and J. Redmon, “YoloV3: An incremental improvement,” 2018.
  * [25] T. Popordanoska, A. Tiulpin, and M. B. Blaschko, “Beyond classification: Definition and density-based estimation of calibration in object detection,” in _Proc. WACV_ , January 2024, pp. 585–594.
  * [26] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proc. CVPR_ , July 2017.
  * [27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proc. CVPR_ , 2016, pp. 770–778.
  * [28] T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in _Proc. CVPR_ , July 2017.
  * [29] J. Ramôa, V. Lopes, L. Alexandre, and S. Mogo, “Real-time 2d–3d door detection and state classification on a low-power device,” _SN Appl. Sci._ , 2021.
  * [30] F. Xia, A. R. Zamir _et al._ , “Gibson env: Real-world perception for embodied agents,” in _Proc. CVPR)_ , 2018.
  * [31] M. Luperto, M. Romeo, J. Monroy, J. Renoux, A. Vuono _et al._ , “User feedback and remote supervision for assisted living with mobile robots: A field study in long-term autonomy,” _ROBOT AUTON SYST_ , vol. 155, p. 104170, 2022.
  * [32] M. Everingham, L. V. Gool, C. K. I. Williams, J. M. Winn, and A. Zisserman, “The pascal visual object classes (VOC) challenge,” _Int. J. Comput. Vision_ , vol. 88, pp. 303–338, 2009.


# Nontrivial low-frequency topological waves at the boundary of a magnetized
plasma

Roopendra Singh Rajawat<EMAIL_ADDRESS>School of Applied and Engineering
Physics, Clark Hall, Cornell University, Ithaca, NY, USA-14853  Tianhong Wang
School of Applied and Engineering Physics, Clark Hall, Cornell University,
Ithaca, NY, USA-14853  Gennady Shvets  School of Applied and Engineering
Physics, Clark Hall, Cornell University, Ithaca, NY, USA-14853

###### Abstract

The topological properties of a magnetized cold gaseous plasma have recently
been explored and the existence of topologically protected edge states has
been established. These studies are limited to a magnetized plasma, where ions
are infinitely massive and provide a neutralizing background. When ion motion
is included, a new class of low-frequency unidirectional topological waves
emerges in the dispersion relation. The group velocity of these waves is in
the opposite direction of high-frequency topological electron waves for a
given magnetic field direction. The Berry curvature and Chern numbers are
calculated to establish nontrivial topological phase. These finding broadens
the possible applications and observations of these exotic excitations in
space and laboratory plasmas.

††preprint: APS/123-QED

Introduction.- Topological edge states have attracted wide attention due to
their unidirectional nature and robustness against perturbation. These modes
are found within common bulk-band gaps and characterized by an integer
invariant, known as the Chern numberOzawa  _et al._ (2019). The Chern number
follows bulk-boundary correspondence Hasan and Kane (2010); Silveirinha (2015)
principle which states that a gapless surface mode exists within a common band
gap at the interface of two topologically distinct materials and allows to
determine the number of unidirectional edge states from the topological
properties of bulk modes. This principle has originated in condensed matter
physicsKlitzing  _et al._ (1980) and later extended to photonicsHaldane and
Raghu (2008); Raghu and Haldane (2008), cold atomic gasesGoldman  _et al._
(2016), electron gas Jin  _et al._ (2016) and classical fluids Silveirinha
(2015); Delplace  _et al._ (2017); Tauber  _et al._ (2019, 2020) .

Recently, nontrivial topology of high-frequency edge states (surface plasmon
polariton; SPP’s), occurring at the boundary of a cold magnetized plasma, have
been established Gao  _et al._ (2016); Yang  _et al._ (2016); Hassani Gangaraj
_et al._ (2019); Pakniyat  _et al._ (2020); Parker  _et al._ (2020a, b);
Parker (2021); Fu and Qin (2021, 2022); Qin and Fu (2022) for gyrotropic
plasmonic materials Gao  _et al._ (2016); Yang  _et al._ (2016); Hassani
Gangaraj  _et al._ (2019); Pakniyat  _et al._ (2020), and continuum plasma
fluids Parker  _et al._ (2020a, b); Parker (2021); Fu and Qin (2021, 2022);
Rajawat  _et al._ (2022); Qin and Fu (2022); Shen  _et al._ (2022). It has
been shown that for certain choices of plasma density ($n$), stationary
external magnetic field (${\bf B_{0}}=B_{0}\hat{z}$) and parallel wavenumber
$k_{z}$, SPP’s can exist at the plasma-vacuum (Parker  _et al._ , 2020b;
Parker, 2021) or plasma-plasma interfaceFu and Qin (2021, 2022), provided both
mediums have different topological phase .

Study of nontrivial topology of low-frequency edge states is still in infancy.
In plasmonic materials, the low-frequency SPPs ceases to exist due to typical
Drude behavior and resulting dissipation. One can however make an artificial
material out of very thin metallic wires, which supports low-frequency SPPs
Pendry  _et al._ (1996); Shelby  _et al._ (2001); Hassani Gangaraj  _et al._
(2020) and removes dissipation. In continuum fluids, like a plasma, low-
frequency electrostatic Emeleus (1966); Hong and Lee (1999) and
electromagnetic Cramer (1995); Breizman and Arefiev (2000) surface waves
arises naturally due to ion motion and exist well below plasma frequencies.
For a cold magnetized plasma, low-frequency electromagnetic waves Cramer
(1995); Breizman and Arefiev (2000); Parker  _et al._ (2020a) have been
studied, however, nontrivial topology of low-frequency waves has not been
discussed. In a recent work, the nontrivial topology of a magnetized fusion
plasma in the Alfven continuum has been discussed in the presence of magnetic
shearParker  _et al._ (2020a) and found unidirectional reversed-shear Alfven
eigenmodes. These low-frequency modes are non-dissipative and exist at the
neutral layer of magnetic shear.

In this letter, without going into much complexity, we use a simplified cold
magnetized plasma slab model to observe the effect of ion motion on nontrivial
topological edge states, occurring at the plasma-vacuum interface, and
calculate Chern numbers of energy bands for mobile ions. We discuss the
limitation on parallel wavenumber $k_{z}$ for a chosen value of ion mass,
numerically obtain the dispersion relation of low-frequency topological edge
sates, demonstrate their existence and topological protection by exciting an
edge state enroute to a rectangular discontinuity using a full 3D particle-in-
cell simulation code.

We consider the cold-plasma slab model of a magnetized, stationary plasma,
where electron and ion collisions are neglected. The external magnetic field
is uniform (${\bf B_{0}}=B_{0}\hat{z}$) and is kept in $\hat{z}$ direction.
The linearized set of governing equations for an infinite homogeneous plasma
is

$\displaystyle\frac{\partial{\tilde{\bf v}_{e}}}{\partial t}$ $\displaystyle=$
$\displaystyle-{\omega_{pe}{\bf E}}-\Omega_{e}{\tilde{\bf
v}_{e}}\times\hat{z},$ (1) $\displaystyle\frac{\partial{\tilde{\bf
v}_{i}}}{\partial t}$ $\displaystyle=$ $\displaystyle{\omega_{pi}{\bf
E}}+\Omega_{i}{\tilde{\bf v}_{i}}\times\hat{z},$ (2)
$\displaystyle\frac{\partial{\bf E}}{\partial t}$ $\displaystyle=$
$\displaystyle\nabla\times{\bf B}+\omega_{pe}\tilde{\bf
v}_{e}-\omega_{pi}\tilde{\bf v}_{i},$ (3) $\displaystyle\frac{\partial{\bf
B}}{\partial t}$ $\displaystyle=$ $\displaystyle-\nabla\times{\bf E}.$ (4)

where ${\bf v}_{e}$ and ${\bf v}_{i}$ are the electron and ion fluid
velocities, $\bf E$ the electric field, ${\bf B}_{0}$ the external stationary
background magnetic field, $\bf B$ the perturbation magnetic field, $|e|$ the
charge on an electron, $m$ and $M$ are the mass of electron and ion,
$\alpha=m/M$, $c$ the speed of light, $\omega_{pe}=\sqrt{4\pi n_{e}e^{2}/m}$
the electron plasma frequency, $\omega_{pi}=\sqrt{4\pi n_{i}e^{2}/M}$ the ion
plasma frequency, $\Omega_{i}=eB_{0}/M$ the ion cyclotron frequency and
$\Omega_{e}=eB_{0}/m$ the electron cyclotron frequency. We have used
renormalized velocities $\tilde{\bf v}_{e}={\bf v}_{e}/\omega_{pe}$ and
$\tilde{\bf v}_{i}={\bf v}_{i}/\omega_{pi}$, reference electric field $E_{0}$,
and reference frequency $\omega_{p0}$. The time is normalized to
${\omega_{p0}}^{-1}$, frequency to $\omega_{p0}$, length to
$k_{p}^{-1}=c/\omega_{p0}$, electric field to $E_{0}$ and magnetic field to
$E_{0}/c$, electron and ion velocities to $eE_{0}/m$ and $eE_{0}/M$,
respectively.

Using Fourier transformation $\partial_{t}\rightarrow-i\omega$ and
$\nabla\rightarrow i{(k_{x},k_{y},k_{z})}$, we obtain the eigenvalue equation
$H\mid\psi\rangle=\omega\mid\psi\rangle$, where $\mid\psi\rangle=\left({\bf
v_{e}\,v_{i}\,E\,B}\right)^{T}$ is a 12-element eigenvector , and $H$ is a
$12\times 12$ Hermitian matrix and given by

$H=\begin{pmatrix}-i\Omega_{e}\hat{z}\times&0&-i\omega_{pe}&0\\\
0&-i\Omega_{i}\hat{z}\times&i\omega_{pi}&0\\\
i\omega_{pe}&-i\omega_{pi}&0&-{\bf k\times}\\\ 0&0&{\bf
k\times}&0\end{pmatrix}.$

The dispersion relation is obtained by solving equation $det\left(H-\omega{\bf
I}_{12}\right)=0$. For each nonzero value of $k_{z}$, the system has 12
eigenvalues $\omega_{\pm n}$, where $n=1,2,3,4,5$ is the index of bands. Note
that, we have two zero frequency bands $\omega_{0}=0$. If thermal corrections
are introduced, zero frequency bands become positive/negative frequency slow
magnetosonic modes in the limit $k_{\perp}=0$ Stix (1992). We change the
problem into 2D by fixing $k_{z}$ and consider a parameter space
$k_{\perp}=(k_{x}^{2}+k_{y}^{2})^{1/2}$. Since the problem is isotropic in the
plane perpendicular to the magnetic field, we keep $k_{x}=0$. All bandgaps
closes at $k_{z}=0$, however, a gap always exist between band 1 and 2 for a
non-zero value of $k_{z}\neq 0$. At a fixed $k_{z}$, in the limit
$k_{\perp}\rightarrow\infty$,
$(\omega_{1},\omega_{2},\omega_{3},\omega_{4},\omega_{5})\rightarrow(0,\omega_{LH},\omega_{UH},ck_{\perp},ck_{\perp})$,
where $\omega_{UH}=\left(\omega^{2}_{pe}+\Omega^{2}_{e}\right)^{1/2}$ is the
upper-hybrid resonance frequency and
$\omega_{LH}=\left(\Omega_{i}^{2}+\frac{\Omega_{e}^{2}\omega_{pi}^{2}}{\omega_{UH}^{2}}\right)^{1/2}$
is the lower-hybrid resonance frequency.

Figure 1: Spectrum of a magnetized, homogeneous, cold plasma as a function of
$k_{y}$, when both electron and ion species are mobile. The spectrum is
plotted for parameters $\Omega_{e}/\omega_{p0}=0.5,\,M/m=5$, (a)
$k_{z}/k_{p}=0.4$, (b) $k_{z}/k_{p}=1.0$. The Chern numbers for positive
frequency bands are shown.

The topology of a band gap is characterized by its gap Chern number Hatsugai
(1993) , which is defined by $C_{i,i+1}=\Sigma_{n=-5}^{i}C_{n}$. The Berry
curvature and Chern number are calculated as described in (Parker  _et al._ ,
2020b; Parker, 2021; Fu and Qin, 2021). The Chern numbers associated with
bands 1, 4, and 5 are integers, however, Chern numbers for bands 2 and 3 are
not integers, which is a result of an insufficient smooth linear operator at
small scales in continuum fluids that does not compactify the space in finite
Brillouin zone Silveirinha (2015). To compactify Brillouin zone in a fluid
continuum and to ensure that Chern numbers are integers, we use same
regularization strategy as discussed in Silveirinha (2015); Parker  _et al._
(2020b); Parker (2021). We choose
$\omega_{p,s}=\omega_{p,s}/\left(1+k_{\perp}^{2}/k_{c}^{2}\right)$, where
$k_{c}$ is a very big cutoff wavenumber, and subscript $s=e,i$ stands for
species electron and ion. The positive frequency bands spectrum
$\left(\omega,k_{y}\right)$ for parameters
$\Omega_{e}/\omega_{p0}=0.5,\,\Omega_{i}/\omega_{p0}=0.1,\,M/m=5$ and two
different value of parallel wavenumbers $k_{z}/k_{p}=0.4,\,1$ are shown in
Fig. 1(a,b), respectively. We have also shown band Chern numbers obtained
using $k_{c}=100$. The Chern number for each positive frequency band is
$\left(C_{1},C_{2},C_{3},C_{4},C_{5}\right)$ = $(1,-2,2,0,1)$ for parameters
$k_{z}/k_{p}=0.4,\Omega_{e}/\omega_{p0}=0.5,M/m=5$ and
$\left(C_{1},C_{2},C_{3},C_{4},C_{5}\right)$ = $(1,-2,1,1,-1)$ for parameters
$k_{z}/k_{p}=1.0,\Omega_{e}/\omega_{p0}=0.5,M/m=5$. The gap Chern number for a
low frequency band gap is $C_{1,2}=1$ and for a high frequency band gap is
$C_{2,3}=-1$. Thus multiple bands are topologically nontrivial and can support
multiple topological gapless edge states. Note that, as discussed in Parker
_et al._ (2020b), bands 3 and 4 touch at
$k_{z}^{*}=\Omega_{e}/\sqrt{1+\Omega_{e}}$. Since band gap closes at
$k_{z}^{*}$, we observe different Chern numbers for bands 3 and 4 for
wavenumber ranges $k_{z}<k_{z}^{*}$ and $k_{z}>k_{z}^{*}$ as shown in Fig.
1(a) and 1(b), respectively. Interestingly, the Chern number of low-frequency
bands (band 1 and 2) are universal and do not change with changing the
parallel wavenumber or finite ion mass. Notwithstanding, changing the magnetic
field direction flips the sign of the Chern numbers. We have calculated Chern
numbers for various ion mass ranges $M/m=5-18360$, and found that ion mass
does not affect the Chern number of any band. Surprisingly, the Chern number
for band 1 ($C_{1}=1$) agrees with the Chern number obtained by earlier study
Parker  _et al._ (2020a). For infinitely massive ions, the Chern numbers for
bands $\omega_{\pm 1}$ become zero (trivial) and for bands $\omega_{\pm 2}$
the Chern number becomes $\mp 1$, respectively, which is exactly same result
earlier studies obtained for single mobile specie plasma Parker  _et al._
(2020b); Parker (2021); Fu and Qin (2021, 2022).

Figure 2:  (a) The band spectrum $\omega(k_{y})$ with two band gaps is shown
for parameters
$\Omega_{e}/\omega_{p0}=0.5,\,k_{z}/k_{p}=1.0,\,M/m=5,\,k_{p}\delta l=1$. The
gray color lines show bulk modes. The high (electron dynamics dominated) and
low frequency (ion dynamics dominated) topological edges states are shown by
blue and red color lines, respectively.(b) Zoom in view of low-frequency
modes. (c) The non-zero eigenvectors (left $y-$scale) for a high frequency
mode at $k_{y}/k_{p}=-1$. (d) The non-zero eigenvectors (left $y-$scale) for a
low frequency mode at $k_{y}/k_{p}=1$. The density profile of plasma in
$x-$direction is shown in (c,d) by a black color line (right $y-$scale).

Bulk-edge correspondence states that an edge mode exists in the common band
gaps at the interface between two topologically distinct materials with
different gap Chern numbers. We have identified all existing topological
nontrivial band gaps in a band spectrum. To prove existence of gapless edge
states between band gaps, we use a numerical eigenvalue solver Fu and Qin
(2021) to solve a set 1D differential equations (1) - (4) in an inhomogeneous
plasma by replacing $x-$coordinate dependent electron and ion plasma
frequencies $\omega_{pe}\rightarrow\omega_{pe}(x)$ and
$\omega_{pi}\rightarrow\omega_{pi}(x)$, respectively. The background magnetic
field is uniform (${\bf B}=B_{0}\hat{z}$) and constant in z-direction. The
plasma density is nonuniform only in the $x-$direction, shown in Fig. 2(c,d)
by a black color line. We choose the density profile
$n(x)=\frac{n_{0}}{2}\left(1+tanh\left[\left(x_{0}-x\right)/\delta
l\right]\right)$, where $n_{0}$ is a maximum density of plasma, $x_{0}$ the
location of the interface and $\delta l$ is width of the interface. We use
artificial conducting boundary conditions at $x=0$ and $x=l$. The artificial
conducting boundary condition yields a nonphysical mode which is uninteresting
and not shown here.

It has been already established (Parker  _et al._ , 2020b; Parker, 2021; Fu
and Qin, 2021) that for a magnetized plasma an edge state arises between bands
2 and 3, and can also be confirmed in our results as shown by the blue color
line in Fig. 2(a). This high-frequency edge state is dominated by electron
dynamics, hence we name it electron gaseous plasmon polariton (EGPP). EGPPs
are localized near the center of the density profile which is confirmed by
Fig. 2(c), where the centroid of non-zero components of electric field
(eigenmodes), obtained using a numerical eigenvalue solver, is localized near
the center of the density profile. Nonetheless, we have another band gap exist
between band 1 and 2 and the gap Chern number for this gap is $C_{1,2}=-1$.
According to the bulk-boundary correspondence, a gapless edge state must exist
in this band gap. We present that a low-frequency edge state exists between
bands 1 and 2 for a finite ion mass as shown by the red color line in Fig.
2(b). This edge state is mostly governed by ion dynamics, hence we name it ion
gaseous polarized polaritons (IGPP). The IGPPs are localized near the plasma-
vacuum interface, which is confirmed by Fig. 2(d), where the centroid of non-
zero components of electric fields is localized near the interface.
Interestingly, the sign of group velocity of low-frequency edge states is in
the opposite direction of high-frequency waves, i.e., for a given magnetic
field direction, EGPPs and IGPPs will propagate in opposite directions. The
IGPPs unidirectional frequency window is confined between band 1 and ion
cyclotron resonance frequency. The LH resonance frequency bulk bands exist
from ion cyclotron resonance frequency to band 2. The Band 1 at $k_{\perp}=0$
limit becomes left-hand-circularly polarized (LCP) mode and its dispersion
relation is given by Stix (1992).

$k_{z}^{LCP}=\omega\sqrt{1+\alpha+\frac{1}{(\Omega_{i}-\omega)(\Omega_{e}+\omega)}}.$
(5)

The second band gap remains open for every finite value of $k_{z}$ and closes
only at $k_{z}=\infty$, however, in the limit $k_{z}\rightarrow\infty$, band
gap becomes infinitesimally small. A finite band gap can be tuned using
equation (5) at an appropriate frequency value $\omega<\Omega_{i}$. It should
be noted that higher the mass of an ion, lower would be the value of $k_{z}$
to open a finite band gap to obtain unidirectional frequency window. One needs
to use a very small value of $k_{z}$ to open a finite band gap for higher
massive ions. For example, to open a unidirectional frequency window of
$\Delta\omega=0.2\Omega_{i}$ for mass ratios $M/m=5$ and $M/m=1836$, one needs
$k_{z}/k_{p}=0.75$ and $k_{z}/k_{p}=0.042$, respectively. The dispersion for a
realistic mass ratio $M/m=1836$ is shown in Fig. 3(a) which is plotted for
parameters $k_{z}/k_{p}=0.01,\Omega_{e}/\omega_{p0}=0.5,M/m=1836$ where red
color line shows a gapless edge mode and gray color lines represent bulk
modes. It is worth mentioning that for higher massive ions, edge states become
almost like plane waves. The higher ion mass will also yield slower frequency
waves.

Figure 3:  The band spectrum $\omega(k_{y})$ for low frequency modes for the
parameters $\Omega_{e}/\omega_{p0}=0.5,\,k_{z}/k_{p}=0.01;\,M/m=1836$.

3D particle-in-cell simulation:- We demonstrate existence and verify their
topological protection using a 3D particle-in-cell simulation codePUKHOV
(1999). The plasma density profile is shown in Fig. 4(c), which varies
linearly (ramp) from $n_{0}$ to $0$ within $k_{p}\delta l=1.0$ length. A
$z-$direction periodic dipole source chain is kept at
$(x,\,y)=(17,\,0)k^{-1}_{p}$, which excites propagating edge modes in
$-yz\,(+yz)$ plane for positive (negative) external magnetic field ($\bf
B_{0}$). The simulation box size is $L_{x}\times L_{y}\times
L_{z}=30k^{-1}_{p}\times 50k^{-1}_{p}\times 6.28k^{-1}_{p}$. The spatial and
temporal resolutions are $\Delta x=0.02k^{-1}_{p}$, $\Delta y=0.2k^{-1}_{p}$,
$\Delta z=0.25k^{-1}_{p}$ and $\Delta t=0.8\Delta x$, respectively. We have
used 4 particles per cell. The simulation box has periodic boundaries in the
$z-$direction for both particles and electromagnetic fields, absorbing in $x-$
and $y-$directions for electromagnetic fields and reflecting for particles. A
rectangular discontinuity is kept at $k_{p}y=12$.

Figure 4: The simulation is carried out for parameters
$\,k_{z}/k_{p}=1.0,\Omega_{ce}/\omega_{p0}=0.5,\,M/m=5,\,k_{p}\delta l=1.0$.
(a) A time snapshot of normalized intensity of electric field $E_{z}$ at
$k_{p}z=\pi$. (b) The $x-$direction variation of normalized electric fields
(left scale) $E_{x}$ (green line) and $E_{z}$ (violet color) at $k_{p}y=4\pi$.
The density profile as a function $x-$coordinate (black line; left scale). To
save computational time, we have considered incompressible plasma fluid, which
does not give any nonlinear effects.

The IGPP mode is excited for the wavenumber $k_{y}/k_{p}=1$ as shown in Fig.
2(b) by a black star. The mode frequency and wavenumber $k_{y}$ obtained from
the simulation results agrees with the theoretical dispersion relation. It is
clear from Fig. 4(a,b) that edge states are unidirectional, robust and
propagate without any defect and scattering around the discontinity. The
centroid of the edge state is localized near the vacuum as shown in Fig. 4(c),
which agrees with the theory.

In conclusion, we have presented that, finite ion mass opens a second
nontrivial low-frequency band gap. The Chern numbers are calculated for all
bands. The Chern numbers for low-frequency bands are found to be independent
of both parallel wavenumber $k_{z}$ and the mass of an ion. An edge state
arises within the low-frequency band gap for a few nonzero values of $k_{z}$.
The dispersion relation has been obtained using a numerical solution of
governing equations for a magnetized two-fluid cold plasma.

The excited edge state frequency is smaller than ion cyclotron frequency and
no naturally occurring resonance continuums exist below ion cyclotron
frequency, thus we do not expect any collisionless damping. However, a quasi-
edge state Rajawat  _et al._ (2022); Rajawat and Shvets (2022) between ion
cyclotron frequency and band 2 can be obtained in the limit $\delta l\ll
1/k_{p}$, where a continuum of lower-hybrid frequency exist. The edge state
would undergo collisionless damping via resonant interaction with lower-hybrid
resonance and would become a quasi-edge state Rajawat and Shvets (2022). It is
also worthwhile to mention that, since quasi-edge states are confined between
$\Omega_{i}$ and $\omega_{2}$, they can exist at arbitrary values of $k_{z}$.

On the separate note, we do not observe any Alfven resonance bands from our
solver. The Alfven resonance may not exist in two-fluid description of a cold
plasma and a detailed explanation can be found in Bellan (1994) and its
subsequent comments.

## References

  * Ozawa  _et al._ (2019) T. Ozawa, H. M. Price, A. Amo, N. Goldman, M. Hafezi, L. Lu, M. C. Rechtsman, D. Schuster, J. Simon, O. Zilberberg, and I. Carusotto, Rev. Mod. Phys. 91, 015006 (2019).
  * Hasan and Kane (2010) M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010).
  * Silveirinha (2015) M. G. Silveirinha, Phys. Rev. B 92, 125153 (2015).
  * Klitzing  _et al._ (1980) K. v. Klitzing, G. Dorda, and M. Pepper, Phys. Rev. Lett. 45, 494 (1980).
  * Haldane and Raghu (2008) F. D. M. Haldane and S. Raghu, Phys. Rev. Lett. 100, 013904 (2008).
  * Raghu and Haldane (2008) S. Raghu and F. D. M. Haldane, Phys. Rev. A 78, 033834 (2008).
  * Goldman  _et al._ (2016) N. Goldman, J. C. Budich, and P. Zoller, Nature Physics 12, 639 (2016).
  * Jin  _et al._ (2016) D. Jin, L. Lu, Z. Wang, C. Fang, J. D. Joannopoulos, M. Soljačić, L. Fu, and N. X. Fang, Nature Communications 7, 13486 (2016).
  * Delplace  _et al._ (2017) P. Delplace, J. B. Marston, and A. Venaille, Science 358, 1075 (2017), https://www.science.org/doi/pdf/10.1126/science.aan8819 .
  * Tauber  _et al._ (2019) C. Tauber, P. Delplace, and A. Venaille, Journal of Fluid Mechanics 868, R2 (2019).
  * Tauber  _et al._ (2020) C. Tauber, P. Delplace, and A. Venaille, Phys. Rev. Research 2, 013147 (2020).
  * Gao  _et al._ (2016) W. Gao, B. Yang, M. Lawrence, F. Fang, B. Béri, and S. Zhang, Nature Communications 7, 12435 (2016).
  * Yang  _et al._ (2016) B. Yang, M. Lawrence, W. Gao, Q. Guo, and S. Zhang, Scientific Reports 6, 21461 (2016).
  * Hassani Gangaraj  _et al._ (2019) S. A. Hassani Gangaraj, G. W. Hanson, M. G. Silveirinha, K. Shastri, M. Antezza, and F. Monticone, Phys. Rev. B 99, 245414 (2019).
  * Pakniyat  _et al._ (2020) S. Pakniyat, A. M. Holmes, G. W. Hanson, S. A. H. Gangaraj, M. Antezza, M. G. Silveirinha, S. Jam, and F. Monticone, IEEE Transactions on Antennas and Propagation 68, 3718 (2020).
  * Parker  _et al._ (2020a) J. B. Parker, J. W. Burby, J. B. Marston, and S. M. Tobias, Phys. Rev. Research 2, 033425 (2020a).
  * Parker  _et al._ (2020b) J. B. Parker, J. B. Marston, S. M. Tobias, and Z. Zhu, Phys. Rev. Lett. 124, 195001 (2020b).
  * Parker (2021) J. B. Parker, Journal of Plasma Physics 87, 835870202 (2021).
  * Fu and Qin (2021) Y. Fu and H. Qin, Nature Communications 12, 3924 (2021).
  * Fu and Qin (2022) Y. Fu and H. Qin, Journal of Plasma Physics 88, 835880401 (2022).
  * Qin and Fu (2022) H. Qin and Y. Fu,  (2022), 10.48550/ARXIV.2205.02381.
  * Rajawat  _et al._ (2022) R. S. Rajawat, V. Khudik, and G. Shvets,  (2022), 10.48550/ARXIV.2203.06693.
  * Shen  _et al._ (2022) X. Shen, L. Reichwein, and A. Pukhov,  (2022), 10.48550/ARXIV.2209.06685.
  * Pendry  _et al._ (1996) J. B. Pendry, A. J. Holden, W. J. Stewart, and I. Youngs, Phys. Rev. Lett. 76, 4773 (1996).
  * Shelby  _et al._ (2001) R. A. Shelby, D. R. Smith, and S. Schultz, Science 292, 77 (2001), https://www.science.org/doi/pdf/10.1126/science.1058847 .
  * Hassani Gangaraj  _et al._ (2020) S. A. Hassani Gangaraj, C. Valagiannopoulos, and F. Monticone, Phys. Rev. Research 2, 023180 (2020).
  * Emeleus (1966) K. Emeleus, Physics Letters 21, 633 (1966).
  * Hong and Lee (1999) H. S. Hong and H. J. Lee, Physics of Plasmas 6, 3422 (1999), https://doi.org/10.1063/1.873599 .
  * Cramer (1995) N. F. Cramer, Physica Scripta T60, 185 (1995).
  * Breizman and Arefiev (2000) B. N. Breizman and A. V. Arefiev, Phys. Rev. Lett. 84, 3863 (2000).
  * Stix (1992) T. Stix,  _Waves in Plasmas_ (American Inst. of Physics, 1992).
  * Hatsugai (1993) Y. Hatsugai, Phys. Rev. Lett. 71, 3697 (1993).
  * PUKHOV (1999) A. PUKHOV, Journal of Plasma Physics 61, 425–433 (1999).
  * Rajawat and Shvets (2022) R. S. Rajawat and G. Shvets, “manuscript in prep.” (2022).
  * Bellan (1994) P. M. Bellan, Physics of Plasmas 1, 3523 (1994), https://doi.org/10.1063/1.870888 .

###### Acknowledgements.

This work was supported by Air Force of Scientific Research (AFOSR) through
Stanford University under MURI Award no. FA9550-21-1-0244. The authors thank
the Texas Advanced Computing Center (TACC) for providing HPC resources.