$\displaystyle=\prod_{j=1}^{N}\mathbb{E}\exp(\lambda(\tilde{N}^{k,j}(s,a)-B\mu(s,a)))$
$\displaystyle\leq\exp\left(\frac{8N(B+2\tau_{\mathsf{mix}})\mu(s,a)\tau_{\mathsf{mix}}\lambda^{2}}{1-20\lambda\tau_{\mathsf{mix}}}\right)$
(48)

Following the Chernoff bound in the proof of (Paulin, 2015, Theorem 3.4), we
conclude:

$\displaystyle\mathbb{P}(|\tilde{N}^{k}(s,a)-NB\mu(s,a)|>t)\leq
2\exp\left(-\frac{t^{2}}{16\tau_{\mathsf{mix}}(B+2\tau_{\mathsf{mix}})K\mu(s,a)+40t\tau_{\mathsf{mix}}}\right)$
(49)

Taking $t=\frac{1}{2}NB\mu(s,a)$, whenever $B\geq\tau_{\mathsf{mix}}$ (by
assumption), we must have:

$\mathbb{P}\left(\tilde{N}^{k}(s,a)\leq\frac{1}{2}NB\mu(s,a)\right)\leq
2\exp\left(-C\tfrac{KB\mu(s,a)}{\tau_{\mathsf{mix}}}\right)$

for some constant $C$. Further, via a union bound over all state action pairs
$(s,a)$, we conclude the statement of the lemma.

###  L.7 Proof of Lemma 13

We first prove a simple consequence of multivariate calculus.

######  Lemma 17.

Let $f:\mathbb{R}^{n}\to\mathbb{R}$ be defined by $f(x)=\sup_{i}x_{i}$. Then,
for every $x,y\in\mathbb{R}^{n}$, there exists $\zeta\in\mathbb{R}^{d}$ such
that $\|\zeta\|_{1}=1$, $\zeta_{i}\geq 0$ and:

$f(x)-f(y)=\langle\zeta,x-y\rangle$

###### Proof.

We consider the log-sum-exp function. Given $L\in\mathbb{R}^{+}$, define
$f_{L}(x):=\frac{1}{L}\log\left(\sum_{i=1}^{n}\exp(Lx_{i})\right)$. An
elementary calculation shows that:

$\frac{\log(n)}{L}+\sup_{i}x_{i}\geq f_{L}(x)\geq\sup_{i}x_{i}$

Therefore, for any fixed $x$,

$\lim_{L\to\infty}f_{L}(x)=f(x)\,.$

Now, $\nabla f_{L}(x)=(p_{1},\dots,p_{n})$ where
$p_{i}=\frac{\exp(Lx_{i})}{\sum_{j=1}^{n}\exp(Lx_{j})}$. Clearly,
$\langle\nabla f_{L}(x),e_{i}\rangle\geq 0$ and $\|\nabla f_{L}(x)\|_{1}=1$.
By the mean value theorem there exists $\beta_{L}$ such that:

$f_{L}(x)-f_{L}(y)=\langle\nabla f_{L}(\beta_{L}),x-y\rangle\,.$ (50)

Now, the simplex in $\mathbb{R}^{n}$ (denoted by $\Delta_{n}$) is compact.
Therefore, there exists a sub-sequence $L_{k}\to\infty$ such that
$\lim_{k\to\infty}\nabla f_{L_{k}}(\beta_{L_{k}})=\zeta\in\Delta_{n}$. Taking
limit along the sub-sequence $L_{k}$ in Equation (50), we conclude the result.
∎

###### Proof of Lemma 13.

In the tabular setting, we have the following expression using Equation (40):

$\text{L.H.S}=\biggr{|}\eta\gamma\sum_{j=1}^{K}\sum_{i=1}^{B}(1-\eta)^{\tilde{N}^{k,j}_{i}(s,a)}\mathbbm{1}\left[(\tilde{s}^{k,j}_{i},\tilde{a}^{k,j}_{i})=(s,a)\right]\left[\tilde{\epsilon}^{k,j}_{i}(w)-\tilde{\epsilon}^{k,j}_{i}(v)\right]\biggr{|}$
(51)

Where

$\tilde{\epsilon}^{k,j}_{i}(w):=\left[\sup_{a^{\prime}\in\mathcal{A}}\langle
w,\phi(\tilde{s}^{k,j}_{i+1},a^{\prime})\rangle-\mathbb{E}_{s^{\prime}\sim
P(\cdot|\tilde{s}^{k,j}_{i},\tilde{a}^{k,j}_{i})}\sup_{a^{\prime}\in
A}\langle\phi(s^{\prime},a^{\prime}),w\rangle\right]$

Now, by an application of Lemma 17, we show that for some
$\eta(\cdot|s^{\prime},w,v)\in\Delta(\mathcal{A})$, depending only on
$(s^{\prime},w,v)$, we have:

$\sup_{a^{\prime}\in A}\langle
w,\phi(s^{\prime},a^{\prime}),w\rangle-\sup_{a^{\prime}\in A}\langle
w,\phi(s^{\prime},a^{\prime}),v\rangle=\sum_{a^{\prime}\in\mathcal{A}}\zeta(a^{\prime}|s^{\prime},w,v)\langle\phi(s^{\prime},a^{\prime}),w-v\rangle$

Now, using the definition of $\hat{P}_{k}(\cdot|s,a)$ and $\bar{P}(\cdot|s,a)$
in the discussion preceding Lemma 12, we can simplify Equation (51) to:

$\text{L.H.S.}=\gamma\biggr{|}\sum_{\begin{subarray}{c}s^{\prime}\in\mathsf{supp}(P(\cdot|s,a))\\\
a^{\prime}\in\mathcal{A}\end{subarray}}\left(\hat{P}_{k}(s^{\prime}|s,a)-\bar{P}_{k}(s^{\prime}|s,a)\right)\zeta(a^{\prime}|s^{\prime},w,v)\langle\phi(s^{\prime},a^{\prime}),w-v\rangle\biggr{|}$
(52)

We conclude the statement of the lemma by an application of the Holder
inequality. ∎

###  L.8 Proof of Lemma 11

###### Proof.

We solve for $f(u^{*})=u^{*}$ to obtain the relation:
$u^{*}=\alpha+\beta\sqrt{u^{*}}$, which after discarding the negative solution
for $\sqrt{u^{*}}$ yields the unique solution:
$u^{*}=\left(\frac{\beta+\sqrt{\beta^{2}+4\alpha}}{2}\right)^{2}$. To prove
the second part, consider for arbitrary $x,y\in\mathbb{R}^{+}$, the following
hyper-contractivity:

$\displaystyle|f(x)-f(y)|$ $\displaystyle=\beta|\sqrt{x}-\sqrt{y}|$
$\displaystyle\leq\beta\sqrt{|x-y|}$ (53)

The second step follows from the fact that
$|\sqrt{a}-\sqrt{b}|\leq\sqrt{|a-b|}$. Since $u^{*}=f(u^{*})$, we apply the
inequality above:

$\displaystyle|f^{(t)}(u)-u^{*}|$ $\displaystyle=|f^{(t)}(u)-f^{(t)}(u^{*})|$
$\displaystyle\leq\beta\sqrt{|f^{(t-1)}(u)-f^{(t-1)}(u^{*})|}$ (54)

We then conclude the result by induction. ∎


# Sparse Image based Navigation Architecture to Mitigate the need of precise
Localization in Mobile Robots

Pranay Mathur  Mobile Robotics
Addverb Technologies
Noida, India
<EMAIL_ADDRESS>Rajesh Kumar  Mobile Robotics
Addverb Technologies
Noida, India
<EMAIL_ADDRESS>Sarthak Upadhayay  Mobile Robotics
Addverb Technologies
Noida, India
<EMAIL_ADDRESS>
###### Abstract

Traditional simultaneous localization and mapping (SLAM) methods focus on
improvement in the robot’s localization under environment and sensor
uncertainty. This paper, however, focuses on mitigating the need for exact
localization of a mobile robot to pursue autonomous navigation using a sparse
set of images. The proposed method consists of a model architecture - RoomNet,
for unsupervised learning resulting in a coarse identification of the
environment and a separate local navigation policy for local identification
and navigation. The former learns and predicts the scene based on the short
term image sequences seen by the robot along with the transition image
scenarios using long term image sequences. The latter uses sparse image
matching to characterise the similarity of frames achieved vis-a-vis the
frames viewed by the robot during the mapping and training stage. A sparse
graph of the image sequence is created which is then used to carry out robust
navigation purely on the basis of visual goals. The proposed approach is
evaluated on two robots in a test environment and demonstrates the ability to
navigate in dynamic environments where landmarks are obscured and classical
localization methods fail.

###### Index Terms:

vision-based navigation, scene-recognition, localization

##  I Introduction

Figure 1: RoomNet Network Architecture: The input consists of a queue
$Q_{I_{t}}$ of current images and a queue $Q_{I_{h}}$ of images at a fixed
interval. The network output is a tensor which gives us the room ID $r$

Simultaneous Localization and Mapping (SLAM) based methods have been popular
since the 2000s. The methods focus on localizing the robot within an
environment while simultaneously creating the map of the environment.
Traditional SLAM methods are known to perform remarkably well on high quality
data [10, 7, 1]. However, the methods are prone to errors in situations where
the sensor data is noisy or when the environment itself is non-static leading
to errors. The SLAM methods focus on the odometry data attached to the robot
which is prone to drift and then rely on matching the environment to the map
in order to prescribe the exact location within the environment. The
localisation is usually described in terms of the Euclidean coordinates
described in $\mathbb{R}^{2}$, requiring an exact prescription of a globally
fixed zero frame. Intermediate localisation errors lead to incorrect actions
by the robot. Numerous methods have been proposed in the literature in order
to improve the robot’s localization in the presence of inaccurate sensor
input, changing environment, change in the location of the robot within the
environment and others. Existence of repetitive features within an environment
is another issue which leads to inaccurate loop closure and hence, inaccurate
localizations. Outdoor localisation has been aided by the existence of Global
Positioning Systems (GPS) but the indoor localisation has been dependent on
the on board sensors on the robot [4, 5].

Recently, different methods have been proposed to pursue robot navigation in
different conditions. This includes robust odometry estimation techniques
including the utility of visual odometry [22, 8, 34], visual-inertial odometry
[18, 41] and LiDAR (Light Detection and Ranging) based odometry techniques
[36]. Apart from traditional LiDAR based localization, image based
localization techniques are popular [37].

Apart from the classical models based on fundamentals from vision, deep
learning methods are very popular recently as they tend to capture non linear
behaviour which is difficult to be modelled using classical strategies. Visual
Odometry techniques like DeepVO [30] have focussed on utilising the recurrent
convolutional neural networks leveraging the memory saving feature of the
same. Structure from motion (Sfm) synthesis techniques have also been
developed [39] for visual odometry.

In order to develop a robust SLAM pipeline, different methods of scene
representation are popular. Dense scenes are reconstructed by fusing the RGB
and the depth images [16]. Monocular image based understanding of the scene
provides a compact descriptor of the scene itself [31, 17]. Other
representations of the scene like the point cloud representation [21], voxel
representation [14], mesh representation [35], and semantic representation [2,
11] have been used. Again, the neural network based SLAM methods including the
CNN-SLAM [27], CodeSLAM [3], SceneCode [38], FASTSLAM [20], SLAM-net [15] are
able to train a robust end-to-end SLAM architecture. CodeSLAM and SceneCode
focuses on training a reduced version of the scene whereas CNN-SLAM is an end-
to-end predictor. SLAM-net architecture proposes a differentiable architecture
where the end-to-end SLAM architecture is fused into a differentiable
computation graph, ensuring backpropagation of the loss for each of the sub-
module of the traditional SLAM pipeline.

Many of the methods focus on improvement on the existing SLAM and mapping
based navigation pipelines using neural network based methods. It is well
known that most of the industrial use case for mobile robots using SLAM
pipelines have utility in warehouse scenarios [40]. The challenging and
changing industrial environments have been well documented in the literature
and challenging datasets like the Scene Change Detection dataset are made
available to tackle [23]. Due to the existence of featureless walls and
rapidly changing scenarios, exact localization based methods fail. The fail
cases are usually related to the occasions of moving features, existence of
featureless structures and rapidly changing methods. Some of the real life
scenarios where the autonomous mobile robot, Dynamo, failed to localize based
on the existing map has been shown in Figure 4 . It can be seen that in a true
industrial scenario many of the features are dynamic and repeated exact
localization of the robot in the environment is difficult. However, there are
certain elements like the pillar or the exact room exits are stationary and do
not change throughout. The researchers have been trying to mitigate the issue
by incorporating the methods [24, 19]. There has been an attempt to relate the
semantically understood feature components within the scene [29]. Infact,
visual navigation language (VLN) models have been increasingly popular [33,
32].

This paper focuses on mitigating the case of exact localization of the robot
within the environment and characterizing the understanding of the robot’s
location in terms of entry-exit image scenarios for the region. A sparse image
based graph consists of the entry exit image the robot needs to see while
navigating the environment. Additionally, the goal to the robot is an image
scene rather than an exact coordinate to the robot. The method reduces the
dependency of exact localization of the robot in terms of the Euclidean
coordinates. The main contributions of our work titled RoomNet can be
quantified as:

  * •

Describing a neural network leveraging the short term and the long term scenic
memory of the robot to locate itself with respect to a room ID and trained on
videos stored while passively navigating

  * •

Describing an image based local navigation policy where the robot repeatedly
aims for a sequence of goal images in order to achieve the desired location.

##  II Network

Figure 2: Illustration of our method

The paper focuses on describing a network so that the robot is able to
understand a sparse location in terms of the image scenes needed to cross to
exit the scene. Throughout the paper, we call the scene a room. The room can
be either characterised by a physical room entity or a hypothetical scenario
associated with a discrete change in the scene. A R-CNN based neural network
is created based on long term and short term memory. The short term memory is
characterised by a LSTM (Long Short Term Memory) architecture whereas the long
term memory is defined from a memory bank. A set of pretrained VGG-16 backbone
is used to characterise the features that the robot sees. The pretrained
backbone architecture results in quick feature generation and less number of
trainable parameters which can be trained during the initial mapping stage.

Let the scene be discretized into ‘m’ rooms, each of them with two transition
locations. Each of the transition locations is characterised using a set of
transition frames that the robot sees (coined as ‘transit’ in the paper). The
short term memory bank constitutes a sequence of ‘$n_{1}$’ images, sampled at
the time interval of $t_{1}$. The long term memory bank constitutes a set of
‘$n_{2}$’ images, sampled at the time intervals of $t_{2}$ with
$t_{2}>>t_{1}$. All the $n_{1}+n_{2}$ frames sequentially pass through the
VGG-16 feature extractor backbone. The concatenated set of short term memory
features pass through the LSTM block. However, the long term features are
sparsely selected over a long range of time and might not capture the exact
understanding of the room for longer instances. In order to mitigate the
issue, an attention model is described to capture the long term memory of the
robot.

VGG-16: VGG16 is a CNN based model architecture which was proposed during 2014
[26]. A representation of the backbone is presented in the Figure 1. The
network uses small $3\times 3$ sized filters with a fixed stride of 1, thereby
capturing small discrete features within the image. The padding is used such
that the spatial resolution is affine to the convolution operation. The model,
although has a lot of trainable parameters has been widely used for the
operations of object detection and classification. In this work, the feature
backbone is used to create an ensemble of features to be trained.

LSTM: LSTM or Long-Short Term Memory is a sequence based recurrent neural
network [13]. A representative view of the LSTM structure is shown in the
Figure 1. The LSTM network consists of a forget gate, input gate and the
output gate. The short term memory features of the local scene seen by the
robot is mapped to the room ID and thence, to the transition sequence of the
frames.

Attention Block: Attention models have shown promise in context of complex
sequence modelling where the elements of sequence are not sampled at a
consistent frequency [28]. It was assumed that the robot’s local velocity is
constant, thereby leading to approximate constant feature sampling for the
short term memory component of the frames. However, the long term feature
vectors when sampled based on a constant frame frame, cannot be at a constant
frequency because the robot itself does not move at a globally constant
velocity. The attention block is presented in the Figure 1. The context
features are described from the saved frames in the long term memory whereas
the current input feature is based on the current image frame.

Complete Model: The network model utilises a combination of the instantaneous
current frame, a set of short term sequence of frames recently seen by the
robot and a set of long term frames seen by the robot. The context features
for each of the frames is described by the feature detector backbone (VGG-16)
and then bifurcated to an LSTM and an attention block. The two outputs are
concatenated to a final softmax layer which presents the probability of the
robot being indexed with a set of entry - exit sequence of images.

##  III Method

The proposed method divides the problem into two sub-modules. A network
architecture - RoomNet for coarse identification of the scene and a separate
module for navigation and finer recognition using a local navigation policy.

RoomNet: RoomNet operates in two stages in an unsupervised manner. A training
and mapping stage during which a predefined set of maneuvers is executed and
an inference stage for navigation. During the training and mapping stage, a
sparse graph $G_{r}(V,E)$ is created with the room ID number $r\in\mathbb{W}$,
as vertices being mapped to the scene viewed by the robot. From the queue of
image frames, $Q_{I}$ associated with $r$, an image sub-queue, $Q_{I_{t}}$, is
chosen by selecting frames post a fixed time interval. These frames are used
to train the model along with the older images, $Q_{I_{h}}$ serving the
purpose of a long-term memory. They take into account the transition images
associated with an edge $E$ of $G_{r}(V,E)$ and the frames the robot could
have seen prior to the current scene. The sparse graph formulates the
connections between different nodes thereby providing additional constraints
on the output inference of RoomNet.

Once the sparse graph is created, the goal to the robot is fed in terms of a
goal image, $I_{g}$. The goal image associates with it a set of frames based
on the environment that it is located in. These are the frames the robot
“needs to view” to associate it with a room ID $r$ where the scene is known to
be available. During initialization, the network RoomNet is fed the queue,
$Q_{I}$, consisting of the latest image frames. This provides a location that
is used to initialize the additional module. It associates the room ID with an
initial frame $I_{o}$ acting as the source node in the graph. Simultaneously,
the user-defined goal image $I_{g}$, is passed. Using $Dijkstra^{\prime}s$
algorithm[6] a hierarchy is then generated from the sparse graph for the room
ID’s, $r\in(0,1,...m)$ to visit and the transition images, $I_{r}$ the robot
needs to look for. Thus, the output of RoomNet is a $1\times(m+1)$ tensor with
an associated probability $p_{m}$ corresponding to the prediction. The tensor
is multiplied with $G_{r}(V,E)$ to get the final output matrix.

$G_{r}=\begin{bmatrix}r_{00}&r_{01}&r_{02}&...&r_{0m}\\\
r_{10}&r_{11}&r_{12}&...&r_{1m}\\\ \vdots&\vdots&\vdots&...&\vdots\\\
r_{m0}&r_{m1}&r_{m2}&...&r_{mm}\\\ \end{bmatrix}$ (1)

Local Navigation Policy: Once a frame has been associated with a scene the
policy is used for local navigation. Images are matched using graph neural-
network based feature-matching [25]. Intermediate images are said to have been
identified when the matching score $m_{s}$ calculated as shown in (2) crosses
a pre-defined threshold $m_{s0}$. The intermediate goal image $I_{r}$, is said
to have been reached when the gradient $v$, of scores generated based on
matching the current frame to the intermediate goal image $I_{k}$ turns
negative and the product of the instantaneous matching scores and uncertainty
measure of inference from RoomNet $C_{t}$, over a moving window of $l$
previous predictions, drops below a threshold $C_{o}$.

$m_{s}=\frac{Features\;Matched}{Keypoints\;Detected\;in\;Query\;Image}$ (2)

$v=\eta\nabla m_{s}$ (3)

$C_{t}=\sum_{i=0}^{l}\frac{1}{e^{{|r-r_{t-i}|}}}$ (4)

Figure 3: Test Environment: The left column presents the original scene while
the right column visualizes the same scene with new obstacles and altered
configurations. The test robots Dynamo 100 and Dynamo 1500 are visible in the
last row on the left and right column respectively Figure 4: Evaluation: The
red arrows indicate the trajectory of the robot. In case 1, The goal points
highlighted in green are placed in the shape of a trapezium. In case 2, the
goal points highlighted in green are placed to form the vertices of a
triangle. The red points are visualized laser scans

###  III-A Implementation

Our approach has been implemented on two robots - a Dynamo 100 and
Dynamo-1500111https://addverb.com/product/autonomous-mobile-robot robot shown
in Figure 3, each equipped with a 2.3 GHz Intel celeron processor, SICK
LiDARs222https://www.sick.com/in/en/detection-and-ranging-solutions and an
Intel Realsense D435i depth camera333https://www.intelrealsense.com/depth-
camera-d435i. The industrial safety grade LiDAR is used purely for obstacle
avoidance and plays no role in our algorithm. Training of RoomNet is performed
using videos collected while passively navigating. The scenes are annotated
with a room ID according to the the scene they belong to. Upon initialization
we perform in-place rotation for RoomNet to provide an initial inference. Post
this an image associated with the inference is identified as the first target
image. Upon identification of the target image the robot moves along the
shortest path possible while avoiding obstacles as described in the section
above. When the target is reached a new image with the next room ID, $r$ is
associated. This process repeats till the final goal is reached. In case the
robot enters a room that is out of the hierarchical sequence of room ID’s,
$Dijkstra^{\prime}s$ algorithm[6] is run again taking this node as the source
and the goal node as the destination.

##  IV Results and Evaluation

We evaluate our results in a scene that consists of 4 rooms. An exemplary set
of scene change images is presented in the Figure 3. The desired image goal
target associated with our local navigation policy is highlighted in dotted
green rectangles. We perform evaluation by comparing our method with the
GMapping SLAM[12] and adaptive Monte Carlo localization[9] based trajectory
planner in a dynamic environment.

Consider the first case where the robot is localized as is made evident by the
laser scans in red coinciding with the obstacle boundaries. The traditional
SLAM architecture based trajectory planner plans a quadrilateral shaped path
with the goal points highlighted in green. The trajectory followed by the
robot is presented in the Figure 4. The result of our approach is demonstrated
in the next row of images. The local planner followed the path more accurately
and the time taken was lower in comparison.

Consider the second case, where the room had changes and dynamic obstacles.
Here we observe the traditional SLAM architecture led to an error in
localization as is evident by the altered laser scans. The trajectory followed
by the robot is presented in the Figure 4 and the trajectory post
delocalization have been highlighted in red. Our approach was able to follow
the planned triangle shaped trajectory accurately and reach the final goal.
For the purpose of visualization of the trajectory we manually initialized the
pose upon delocalization. The image goals were reached successfully. We
reiterate that the map is used purely for visualization purposes and our
approach does not require a metric map. A video of our results can be found at
https://youtu.be/KwP2NuIrdx4

##  V Conclusion

This work focuses on mitigating the need of exact localization of the mobile
robot in terms of exact euclidean coordinates and focusing on the local short
term and the long term scenes that the robot sees. A neural network is
described to associate a set of transition images with the current scene
viewed by the robot. The robot then directs itself towards repetitive
transition views and ultimately to the goal. The use of long term and short
term memory scenes have helped sustain consistency even when the environment
suffered dynamic changes.

## References

  * [1] T. Bailey and H. Durrant-Whyte, “Simultaneous localization and mapping (slam): Part ii,” _IEEE robotics & automation magazine_, vol. 13, no. 3, pp. 108–117, 2006.
  * [2] Y. Bao, Y. Pan, Z. Yang, and R. Huan, “Utilization of semantic planes: Improved localization and dense semantic map for monocular slam in urban environment,” _IEEE Robotics and Automation Letters_ , 2021.
  * [3] M. Bloesch, J. Czarnowski, R. Clark, S. Leutenegger, and A. J. Davison, “Codeslam—learning a compact, optimisable representation for dense visual slam,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 2560–2568.
  * [4] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” _IEEE Transactions on robotics_ , vol. 32, no. 6, pp. 1309–1332, 2016.
  * [5] C. Chen, B. Wang, C. X. Lu, N. Trigoni, and A. Markham, “A survey on deep learning for localization and mapping: Towards the age of spatial machine intelligence,” _arXiv preprint arXiv:2006.12567_ , 2020.
  * [6] E. W. Dijkstra, “A note on two problems in connexion with graphs,” _Numerische mathematik_ , vol. 1, no. 1, pp. 269–271, 1959.
  * [7] H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping: part i,” _IEEE robotics & automation magazine_, vol. 13, no. 2, pp. 99–110, 2006.
  * [8] C. Forster, M. Pizzoli, and D. Scaramuzza, “Svo: Fast semi-direct monocular visual odometry,” in _2014 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2014, pp. 15–22.
  * [9] D. Fox, W. Burgard, F. Dellaert, and S. Thrun, “Monte carlo localization: Efficient position estimation for mobile robots,” 01 1999, pp. 343–349.
  * [10] J. Fuentes-Pacheco, J. Ruiz-Ascencio, and J. M. Rendón-Mancha, “Visual simultaneous localization and mapping: a survey,” _Artificial intelligence review_ , vol. 43, no. 1, pp. 55–81, 2015.
  * [11] M. Gonzalez, E. Marchand, A. Kacete, and J. Royan, “S3lam: Structured scene slam,” _arXiv preprint arXiv:2109.07339_ , 2021.
  * [12] G. Grisetti, C. Stachniss, and W. Burgard, “Improved techniques for grid mapping with rao-blackwellized particle filters,” _IEEE Transactions on Robotics_ , vol. 23, no. 1, pp. 34–46, 2007.
  * [13] S. Hochreiter and J. Schmidhuber, “Lstm can solve hard long time lag problems,” _Advances in neural information processing systems_ , pp. 473–479, 1997.
  * [14] M. Ji, J. Gall, H. Zheng, Y. Liu, and L. Fang, “Surfacenet: An end-to-end 3d neural network for multiview stereopsis,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 2307–2315.
  * [15] P. Karkus, S. Cai, and D. Hsu, “Differentiable slam-net: Learning particle slam for visual navigation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 2815–2825.
  * [16] C. Kerl, J. Sturm, and D. Cremers, “Dense visual slam for rgb-d cameras,” in _2013 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2013, pp. 2100–2106.
  * [17] J. Lee, M. Back, S. S. Hwang, and I. Y. Chun, “Improved real-time monocular slam using semantic segmentation on selective frames,” _arXiv preprint arXiv:2105.00114_ , 2021.
  * [18] M. Li and A. I. Mourikis, “High-precision, consistent ekf-based visual-inertial odometry,” _The International Journal of Robotics Research_ , vol. 32, no. 6, pp. 690–711, 2013.
  * [19] B. Liu, X. Xiao, and P. Stone, “A lifelong learning approach to mobile robot navigation,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 1090–1096, 2021.
  * [20] M. Montemerlo, S. Thrun, D. Koller, B. Wegbreit _et al._ , “Fastslam: A factored solution to the simultaneous localization and mapping problem,” _Aaai/iaai_ , vol. 593598, 2002.
  * [21] R. A. Newcombe, S. J. Lovegrove, and A. J. Davison, “Dtam: Dense tracking and mapping in real-time,” in _2011 international conference on computer vision_. IEEE, 2011, pp. 2320–2327.
  * [22] D. Nistér, O. Naroditsky, and J. Bergen, “Visual odometry,” in _Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004._ , vol. 1. Ieee, 2004, pp. I–I.
  * [23] J.-M. Park, J.-H. Jang, S.-M. Yoo, S.-K. Lee, U.-H. Kim, and J.-H. Kim, “Changesim: Towards end-to-end online scene change detection in industrial indoor environments,” _arXiv preprint arXiv:2103.05368_ , 2021.
  * [24] D. M. Rosen, J. Mason, and J. J. Leonard, “Towards lifelong feature-based mapping in semi-static environments,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2016, pp. 1063–1070.
  * [25] P. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” _CoRR_ , vol. abs/1911.11763, 2019. [Online]. Available: http://arxiv.org/abs/1911.11763
  * [26] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 1–9.
  * [27] K. Tateno, F. Tombari, I. Laina, and N. Navab, “Cnn-slam: Real-time dense monocular slam with learned depth prediction,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 6243–6252.
  * [28] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [29] H. Wang, W. Wang, W. Liang, C. Xiong, and J. Shen, “Structured scene memory for vision-language navigation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 8455–8464.
  * [30] S. Wang, R. Clark, H. Wen, and N. Trigoni, “Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 2043–2050.
  * [31] X. Wang, M. Christie, and E. Marchand, “Tt-slam: Dense monocular slam for planar environments,” in _IEEE International Conference on Robotics and Automation, ICRA’21_ , 2021.
  * [32] X. Wang, Q. Huang, A. Celikyilmaz, J. Gao, D. Shen, Y.-F. Wang, W. Y. Wang, and L. Zhang, “Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6629–6638.
  * [33] X. Wang, W. Xiong, H. Wang, and W. Y. Wang, “Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 37–53.
  * [34] N. Yang, L. v. Stumberg, R. Wang, and D. Cremers, “D3vo: Deep depth, deep pose and deep uncertainty for monocular visual odometry,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 1281–1292.
  * [35] M. Young, C. Pretty, J. McCulloch, and R. Green, “Sparse point cloud registration and aggregation with mesh-based generalized iterative closest point,” _Journal of Field Robotics_ , 2021.
  * [36] J. Zhang and S. Singh, “Loam: Lidar odometry and mapping in real-time.” in _Robotics: Science and Systems_ , vol. 2, no. 9, 2014.
  * [37] W. Zhang and J. Kosecka, “Image based localization in urban environments,” in _Third international symposium on 3D data processing, visualization, and transmission (3DPVT’06)_. IEEE, 2006, pp. 33–40.
  * [38] S. Zhi, M. Bloesch, S. Leutenegger, and A. J. Davison, “Scenecode: Monocular dense semantic reconstruction using learned encoded scene representations,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 11 776–11 785.
  * [39] T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, “Unsupervised learning of depth and ego-motion from video,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1851–1858.
  * [40] Q. Zou, Q. Sun, L. Chen, B. Nie, and Q. Li, “A comparative analysis of lidar slam-based indoor navigation for autonomous vehicles,” _IEEE Transactions on Intelligent Transportation Systems_ , 2021.
  * [41] D. Zuñiga-Noël, A. Jaenal, R. Gomez-Ojeda, and J. Gonzalez-Jimenez, “The uma-vi dataset: Visual–inertial odometry in low-textured and dynamic illumination environments,” _The International Journal of Robotics Research_ , vol. 39, no. 9, pp. 1052–1060, 2020.