Alejandro<EMAIL_ADDRESS>Javier<EMAIL_ADDRESS>Michael<EMAIL_ADDRESS>QUT Centre for Robotics
Queensland University of Technology
Brisbane, Australia School of Engineering
University of Zaragoza
Zaragoza, Spain Motion-Bias-Free Feature-Based SLAM

# Motion-Bias-Free Feature-Based SLAM

###### Abstract

For SLAM to be safely deployed in unstructured real world environments, it
must possess several key properties that are not encompassed by conventional
benchmarks. In this paper we show that SLAM _commutativity_ , that is,
consistency in trajectory estimates on forward and reverse traverses of the
same route, is a significant issue for the state of the art. Current pipelines
show a significant bias between forward and reverse directions of travel, that
is in addition inconsistent regarding which direction of travel exhibits
better performance. In this paper we propose several contributions to feature-
based SLAM pipelines that remedies the motion bias problem. In a comprehensive
evaluation across four datasets, we show that our contributions implemented in
ORB-SLAM2 substantially reduce the bias between forward and backward motion
and additionally improve the aggregated trajectory error. Removing the SLAM
motion bias has significant relevance for the wide range of robotics and
computer vision applications where performance consistency is important.

††This research was partially supported by funding from ARC Laureate
Fellowship FL210100156 to MM and the QUT Centre for Robotics, the Spanish
Government under Grant PID2021-127685NB-I00 and TED2021-131150B-I00 and the
Aragon Government under Grant DGA_FSET45_20R.

##  1 Introduction

Simultaneous Localization and Mapping (SLAM) refers to the concurrent
estimation of a mobile robot state and a model of the environment, known as
the map, from its onboard sensor readings. Visual SLAM utilizes video as the
main input to build a 3D map of an unknown environment, which is then used in
another processing thread to determine the camera pose within the environment
in real-time [Cadena et al.(2016)Cadena, Carlone, Carrillo, Latif, Scaramuzza,
Neira, Reid, and Leonard].

The progress in this field has been remarkable over the past three decades,
leading to the extensive adoption of SLAM in real-world applications across
various industries (e.g., robotics, autonomous driving or augmented and
virtual reality). Starting from the introduction of probabilistic formulations
for Simultaneous Localization and Mapping (SLAM) [Durrant-Whyte and
Bailey(2006), Bailey and Durrant-Whyte(2006), Davison et al.(2007)Davison,
Reid, Molton, and Stasse, Klein and Murray(2007)], researchers have
investigated the fundamental properties of SLAM, such as observability,
convergence, and consistency [Civera et al.(2008)Civera, Davison, and Montiel,
Chli and Davison(2009), Strasdat et al.(2010)Strasdat, Montiel, and Davison,
Newcombe et al.(2011)Newcombe, Lovegrove, and Davison, Dissanayake et
al.(2011)Dissanayake, Huang, Wang, and Ranasinghe, Kerl et al.(2013)Kerl,
Sturm, and Cremers]. Current pipelines [Engel et al.(2014)Engel, Schöps, and
Cremers, Mur-Artal et al.(2015)Mur-Artal, Montiel, and Tardos, Forster et
al.(2016)Forster, Zhang, Gassner, Werlberger, and Scaramuzza, Mur-Artal and
Tardós(2017), Engel et al.(2017)Engel, Koltun, and Cremers, Schops et
al.(2019)Schops, Sattler, and Pollefeys, Teed and Deng(2021)] show an
impressive performance in terms of accuracy and robustness. However, although
successful implementations in certain domains might rise the question “is
fundamental research in SLAM still necessary?”, challenges still remain,
particularly in complex and dynamic environments [Bescos et al.(2018)Bescos,
Fácil, Civera, and Neira, Xu et al.(2019)Xu, Li, Tzoumanikas, Bloesch,
Davison, and Leutenegger, Ballester et al.(2021)Ballester, Fontán, Civera,
Strobl, and Triebel]. Achieving truly robust performance is critical for a
wide range of applications. To ensure that SLAM can provide reliable
perception and navigation for long-lived autonomous robots, fundamental
aspects such as self-tuning, resource awareness, and task-driven perception
are yet to be addressed [Cadena et al.(2016)Cadena, Carlone, Carrillo, Latif,
Scaramuzza, Neira, Reid, and Leonard].

In this paper we address motion bias, a particular problem that mainly affects
feature-based visual SLAM. Several works [Engel et al.(2016)Engel, Usenko, and
Cremers, Engel et al.(2017)Engel, Koltun, and Cremers, Yang et al.(2018)Yang,
Wang, Gao, and Cremers] and our own results show that variations in the
direction of the camera motion (i.e., playing the same image sequence forwards
and backwards) lead to significant differences in the performance of SLAM
methods. Figure 1 contains experimental results showing this bias in KITTI
[Geiger et al.(2013)Geiger, Lenz, Stiller, and Urtasun] and TUM Mono VO [Engel
et al.(2016)Engel, Usenko, and Cremers].

(a) KITTI Dataset (10 sequences)

(b) TUM Mono VO Dataset (50 sequences)

Figure 1: Motion bias and errors in KITTI [Geiger et al.(2013)Geiger, Lenz,
Stiller, and Urtasun] 1(a) and TUM Mono VO [Engel et al.(2016)Engel, Usenko,
and Cremers] 1(b) datasets. Top row displays the motion bias as the arithmetic
error difference between forward (f) and backward (b) passes. Bigger $\pm$
values represent higher motion errors. Note in the box-and-whiskers plots the
substantial biases in the Vanilla ORB-SLAM2 version, and how our
implementation is able to remove them. Bottom row shows forward (green),
backward (blue) and total (black) errors for Vanilla ORB-SLAM2 and our
version. Note that our errors are similar for forward and backward passes, in
contrast to substantial differences in Vanilla ORB-SLAM2. Moreover, observe
the opposite bias sign in KITTI and TUM Mono VO, indicating data dependency.

The existence of this motion bias in SLAM pipelines is relevant and should be
addressed. On the one hand, motion bias should not appear from an optimization
perspective, as the geometric configuration and scene appearance do not change
when playing an image sequence forwards or backwards. This reveals flaws in
current feature-based pipelines, and lack of explainability of their inner
working modes. It may also indicate that they are tuned to overfit benchmarks
and reported results do not generalize in more general setups.

On the other hand, bias-free forward and backward estimates are desirable for
SLAM to generalize across a wide range of use cases. We can cite several
application domains in which agents perform varied motions, such as robots
equipped with a front-facing and a rear-facing camera [Agarwal et
al.(2020)Agarwal, Vora, Pandey, Williams, Kourous, and McBride, Maddern et
al.(2017)Maddern, Pascoe, Linegar, and Newman, Meyer et al.(2021)Meyer,
Smíšek, Fontan Villacampa, Oliva Maza, Medina, Schuster, Steidle, Vayugundla,
Müller, Rebele, et al.], highly maneuverable robots such as vacuum cleaners,
drones or robotic arms, and systems operating in environments with strong
motion constraints, such as mine exploration or endoscopic surgery [Recasens
et al.(2021)Recasens, Lamarca, Fácil, Montiel, and Civera].

It is true that there might be applications with dominant motion patterns,
such as autonomous driving. However, it should be remarked that the motion
bias in current SLAM pipelines is neither consistent nor predictable,
impacting in their reliability and trustworthiness in critical applications.
Figure 1 shows how ORB-SLAM2 presents opposite biases in KITTI and TUM Mono
VO: forward motion estimates are more accurate in KITTI, whereas backward ones
are more accurate in TUM Mono VO.

In this paper we make the following contributions: (1) we extend previous
evaluations [Engel et al.(2016)Engel, Usenko, and Cremers, Engel et
al.(2017)Engel, Koltun, and Cremers, Yang et al.(2018)Yang, Wang, Gao, and
Cremers] that characterized the bias present in feature-based SLAM for forward
and backward camera motion, (2) we identify potential sources that may cause
such bias and provide theoretical insights and empirical analysis for a better
design of new methods, (3) we implement several novel contributions for motion
bias removal on the state-of-the-art ORB-SLAM2 [Mur-Artal and Tardós(2017)],
specifically in a deterministic version111Our deterministic implementation of
ORB-SLAM2 [Mur-Artal and Tardós(2017)] described in Section 4 can be found in
https://github.com/alejandrofontan/ORB_SLAM2_Deterministic. to facilitate
ablation studies and analyses, and (4) we evaluate our proposals in 4
different and complementary datasets. Our results show that we are able to
reduce drastically the motion bias in feature-based SLAM, while keeping the
general accuracy.

##  2 Related Work

Motion Bias was unveiled by Engel et al. [Engel et al.(2017)Engel, Koltun, and
Cremers], that showed the difference in accuracy of feature-based monocular
ORB-SLAM2 [Mur-Artal and Tardós(2017)] between forward and backward runs in
the TUM Mono VO Dataset [Engel et al.(2016)Engel, Usenko, and Cremers]. They
arrived at three significant outcomes: 1) They showed that ORB-SLAM2 performed
significantly better for backwards motion, 2) that, contrary to ORB-SLAM2,
their direct sparse odometry (DSO) was largely unaffected by motion bias, and
3) that image resolution was a relevant factor for ORB-SLAM2, while DSO was
only marginally affected due to subpixel accuracy. From these outcomes they
also concluded that benchmarking on large datasets, covering a diverse range
of environments and motion patterns, was of high importance. While Engel et
al. raised these issues, no analysis, conclusion, or possible remedies were
given in their work.

Yang et al. [Yang et al.(2018)Yang, Wang, Gao, and Cremers] performed
systematic and quantitative evaluations of motion bias on the three most
popular SLAM formulations, namely direct, feature-based and semi-direct
methods. Their initial findings agreed with those of Engel et al. [Engel et
al.(2017)Engel, Koltun, and Cremers]. They claim that the performance
degradation of feature-based SLAM come from implementation details, and
proposed that future work should take into consideration the following issues:
depth representation, point sampling strategy, point management and
discretization artifacts. They implemented a version of ORB-SLAM2 with sub-
pixel matching accuracy that reduced the errors of the forward passes but
maintained a substantial gap between forward and backward motions. Motion bias
remained hence unsolved, and Yang et al.’s final conclusion highlighted the
importance of the problem, stating that more efforts should be taken to
address such problem for the adoption of feature-based SLAM in safety-critical
applications like autonomous driving. In this paper we continue their work by
formulating and evaluating six novel contributions that realistically minimize
motion bias with minimal impact in SLAM general accuracy.

##  3 Motion-Bias-Free Feature-Based SLAM

In this section, we formulate the contribution of visual covariances to motion
bias and introduce an alternative robust cost function for outliers. We also
address the sources of the motion bias by providing theoretical insights and
proposing motion-bias-free models for five components within feature-based
SLAM, specifically related to point representation, optimization and data
association.

###  3.1 Point Representation

Feature-based SLAM estimates a sparse map of 3D points
$\\{\textbf{p}\in\mathcal{P}\mid\textbf{p}\in\mathbb{R}^{3}\\}$ and keyframes
$k\in\mathcal{K}$. Keyframes are composed by a grayscale image
$\textbf{I}_{k}$ and its 6DOF camera pose, represented by a rotation matrix
$\textbf{R}_{k}\in\mathrm{SO}(3)$ and translation vector
$\textbf{t}_{k}\in\mathbb{R}^{3}$. Each point p is associated with a set of 2D
keypoints
$\mathcal{U}\equiv\\{\mathbf{u}_{k1},\mathbf{u}_{k2},\dots,\mathbf{u}_{kn}\\},\mathbf{u}_{ki}\in\boldsymbol{\Omega}_{ki}$
with corresponding feature descriptors
$\mathcal{B}\equiv\\{\mathbf{b}_{k1},\mathbf{b}_{k2},\dots,\mathbf{b}_{kn}\\},\mathbf{b}_{ki}\in\mathbb{R}^{d}$.
The keypoints correspond to the projections of p in the keyframes
$\mathcal{K}$ in which it is visible, and $\boldsymbol{\Omega}_{ki}$ are their
image domains.

Reference Descriptor Selection. To enhance the matching process and acquire
more observations with new keypoints, a common technique is to choose a
distinctive reference descriptor $\mathbf{b}_{r}$ and match subsequent
keypoints against it [Mur-Artal et al.(2015)Mur-Artal, Montiel, and Tardos,
Mur-Artal and Tardós(2017), Campos et al.(2021)Campos, Elvira, Rodríguez,
Montiel, and Tardós]. Traditionally, the reference descriptor is selected
based on image appearance. For instance, one approach is to choose the
descriptor with the least median Hamming distance $d_{H}$ to the remaining
descriptors:
$\mathbf{b}_{r}=\operatorname{argmin}_{\mathbf{b}_{ki}}\operatorname{median}(d_{H}(\mathbf{b}_{ki},\mathcal{B})))$
[Mur-Artal and Tardós(2017)]. In contrast, we propose to leverage 3D geometry
by selecting the descriptor with the smallest Euclidean distance $d_{E}$
between the translation vectors of its holding keyframe $\textbf{t}_{k}$ and
the keyframe $\textbf{t}_{i}$ containing the keypoint being matched:
$\mathbf{b}_{r}=\operatorname{argmin}_{\mathbf{b}_{k}}d_{E}(\textbf{t}_{k},\textbf{t}_{i}))$.
With this policy, we aim to enhance matching quality by utilizing the
descriptor that exhibits closest geometric proximity.

Point Appearance Invariance. In order to reduce the number of incorrect
observations resulting from the matching process, we apply a geometric
constraint that is robust to motion bias to filter the query keypoints
$u_{q}$. The image keypoints $\mathcal{U}$ are detected at different octaves
$l\in(0,l_{max}\in\mathbb{Z})$ corresponding to scaled images, where
$w(l)=s^{l}\cdot w_{0}$ and $h(l)=s^{l}\cdot h_{0}$. Here, $s$ represents the
scaling factor, and $w_{0}$ and $h_{0}$ are the widths and heights at the
highest resolution. To establish the appearance invariance interval for the
depth $z_{q}$ of the query keypoints $u_{q}$, we set a maximum change in the
octave of the keypoints denoted by $\Delta l$:

$\small{z_{q}\in[\operatorname{max}_{k}(z_{k}\cdot s^{-\Delta
l-0.5}),\quad\operatorname{min}_{k}(z_{k}\cdot s^{\Delta l+0.5})],}$ (1)

where $z_{k}$ is the depth of the point p with respect to the keyframe $k$
(see Figure 2(a)).

###  3.2 Optimization

Visual Covariances contribution to motion bias. The camera pose optimization
in feature-based SLAM is based on minimizing a weighted residual
$\textbf{r}\boldsymbol{\sigma}^{-2}_{r}\textbf{r}^{\top}$, in which r and
$\boldsymbol{\sigma}^{2}_{r}$ are respectively the reprojection error and its
covariance. In this section we formulate the contribution to motion bias of
the standard approximation of visual covariances and propose a new cost
function that is robust to motion bias.

We identify an observation as the projection of a 3D point p in two different
keyframes $\\{i,j\\}\in\mathcal{K}$. We label with subscript $j$ the reference
frame, and with $i$ any other frame from which the point is visible. The image
coordinates of the projection of p in reference frame $j$ and its depth are
denoted as $\textbf{u}\in\boldsymbol{\Omega}$ and $z\in\mathbb{R}$
respectively (we drop subindex $j$ for simplicity).

The function $\varphi(\textbf{u})$ projects a point p from its camera
coordinates u in the reference frame $j$ into the frame $i$,

$\small{\varphi(\textbf{u})=\Pi(\textbf{R}\Pi^{-1}(\textbf{u},z)+\textbf{t}),}$
(2)

where $\Pi(\textbf{p})$ (determined by the intrinsic camera parameters)
projects p in the image; and $\Pi^{-1}(\textbf{u},z)$ back-projects the image
point with coordinates $\bf{u}$ at depth $z$. $\textbf{R}\in\mathrm{SO}(3)$
and $\textbf{t}\in\mathbb{R}^{3}$ stand here for the relative rotation and
translation between keyframes $j$ and $i$.

Feature-based errors are usually defined as variations of the following
expression:

$\small{\textbf{r}=\textbf{u}_{i}-\varphi(\textbf{u}),\quad\boldsymbol{\sigma}_{r}^{2}=\boldsymbol{\sigma}^{2}_{u_{i}}+\varphi(\boldsymbol{\sigma}^{2}_{u}),}$
(3)

where $\textbf{u}_{i}$
$\\{\textbf{u}_{i}\in\mathcal{U}\mid\textbf{u}_{i}\in\mathbb{R}^{2}\\}$ stands
for the feature point in image $i$ and $\varphi(\textbf{u})$ for the
corresponding point in the frame $j$ reprojected in the frame $i$. Hence the
residual covariance is expressed as the sum of the projection covariance and
the feature subpixel noise $\boldsymbol{\sigma}_{u_{i}}^{2}$.

We approximate the residual function (3) by its first-order Taylor
approximation

$\small{\varphi(\textbf{u}+d\textbf{u})\approx\varphi(\textbf{u})+\nabla_{\textbf{u}}\varphi
d\textbf{u},\quad\boldsymbol{\sigma}^{2}_{r}\approx\boldsymbol{\sigma}^{2}_{u_{i}}+\nabla_{\textbf{u}}\varphi\boldsymbol{\sigma}^{2}_{u}\nabla_{\textbf{u}}\varphi^{T},}$
(4)

with the perspective deformation gradient tensor $\nabla_{\textbf{u}}\varphi$
[Fontan et al.(2022)Fontan, Oliva, Civera, and Triebel] being the Jacobian of
the projection function (2).

Assuming pure forward/backward movement, $\textbf{R}=\textbf{I}_{3}$,
$\textbf{t}=(0,0,t)^{T}$, the feature subpixel noise, the gradient tensor and
the residual covariances can be modeled as isotropic Gaussians,

$\small{\boldsymbol{\sigma}^{2}_{r}=\sigma^{2}_{r}\textbf{I}_{2}=(\sigma^{2}_{u_{i}}+\varepsilon_{ij}^{2}\sigma^{2}_{u_{j}})\textbf{I}_{2}\quad,\text{with}\quad\boldsymbol{\sigma}^{2}_{u_{i}}=\sigma_{u_{i}}^{2}\textbf{I}_{2}\quad,\boldsymbol{\sigma}^{2}_{u_{j}}=\sigma_{u_{j}}^{2}\textbf{I}_{2}\quad,\text{and}\quad\nabla_{\textbf{u}}\varphi=\varepsilon_{ij}\textbf{I}_{2}.}$
(5)

The standard approach for visual covariance is to approximate the residual
covariance of the keypoint as $\sigma^{2}_{r}|_{appr}=2\sigma^{2}_{u_{i}}$
[Mur-Artal et al.(2015)Mur-Artal, Montiel, and Tardos]. Compared against the
value from the linearization

$\small{\alpha=\frac{\sigma^{2}_{r}|_{appr}}{\sigma^{2}_{r}}=\frac{2\sigma^{2}_{u_{i}}}{(\sigma^{2}_{u_{i}}+\varepsilon_{ij}^{2}\sigma^{2}_{u_{j}})}=\frac{2}{(1+\varepsilon_{ij}^{2})}\left\\{\begin{array}[]{c
l}>1&\text{if \hskip 5.69054pt}\varepsilon_{ij}^{2}<1\text{\hskip 5.69054pt {
, over-estimation} }\\\ &\\\ <1&\text{if \hskip
5.69054pt}\varepsilon_{ij}^{2}>1\text{\hskip 5.69054pt { , under-estimation.}
}\end{array}\right.}$ (6)

this leads to over- and under-estimation effects that result in different
outlier rejection criteria at forward and backward motions. To address this
issue, we compute the reprojection error in both the projection and reference
keyframes (assuming constant depth) and normalize them with the covariance of
the keypoints in their respective views

$\small{\textbf{r}_{i}=(\textbf{u}_{i}-\varphi(\textbf{u}))^{2},\textbf{\hskip
11.38109pt}\textbf{r}_{j}=(\textbf{u}-\varphi(\textbf{u}_{i}))^{2},\textbf{\hskip
11.38109pt}\boldsymbol{\sigma}_{r_{i}}^{2}=2\boldsymbol{\sigma}^{2}_{u_{i}},\textbf{\hskip
11.38109pt}\boldsymbol{\sigma}_{r_{j}}^{2}=2\boldsymbol{\sigma}^{2}_{u_{j}}}.$
(7)

Figure 2(b) compares the robustness of the standard approach and our
reprojection function against motion bias

$\small{\alpha=\frac{\sigma^{2}_{r}|_{appr}}{\sigma^{2}_{r}}=\frac{2\sigma^{2}_{u_{i}}+2\sigma^{2}_{u_{j}}}{(1+\varepsilon_{ij}^{2})\sigma^{2}_{u_{j}}+(1+\varepsilon_{ji}^{2})\sigma^{2}_{u_{i}}}\stackrel{{\scriptstyle\sigma^{2}_{u_{i}}=\sigma^{2}_{u_{j}}}}{{=}}\frac{4}{2+\varepsilon_{ij}^{2}+\varepsilon_{ji}^{2}}.}$
(8)

(a) Depth invariance interval.

(b) Covariance over-/under-estimation.

Figure 2: Left: We show the depth invariance interval (1) for every octave of
a point found at $z=2(m)$. Right: Relative error in the estimation of visual
covariances (6), (8).

Outlier Rejection. ORB-SLAM2 [Mur-Artal and Tardós(2017)] and DSO [Engel et
al.(2017)Engel, Koltun, and Cremers], representative pipelines for feature-
based and photometric SLAM, implement two strategies to reject outliers.
Firstly, they minimize a Huber cost function [Concha and Civera(2015)] of the
geometric and photometric errors respectively in order to downweight their
influence. And secondly, both approaches remove point observations that have a
Mahalanobis residual above a certain threshold after optimization. Although
this second strategy may have benefits, it may have a negative effect if
removal is done too early. For instance, for a frame with a large number of
noisy observations (due to dynamic objects, motion blur or lighting changes),
accurate matches can suffer from early removal due to inaccurate initial
estimates.

Early removal of potential outliers may not have a significant influence for
photometric methods, as observations are abundant and very efficiently
computed by simply retrieving the intensity values of the image points [Fontan
et al.(2023)Fontan, Giubilato, Oliva, Civera, and Triebel]. However, it can
degrade the performance of feature-based methods. As features have a high
extraction footprint, their re-detection after removal comes with certain
delay. In order to avoid this effect, we do not remove potential outliers and
keep all observations in the optimization.

###  3.3 Data Association

Symmetric Data Association. The implementation of data association varies
depending on its specific function within the SLAM pipeline. The left part of
Figure 3 provides pseudo-code for the tracking and mapping threads of ORB-
SLAM2 [Mur-Artal and Tardós(2017)]. Design choices differ based on whether the
association aims to create new map points (i.e., Search For Triangulation),
search for new observations (i.e., Search By Projection), or fuse map points
(i.e., Fuse).

The ray from a point observation $\textbf{u}_{i}\in\mathbb{R}^{3}$ on an image
$I\in\\{i,j\\}$ is $\textbf{r}_{i}=R^{wc}_{i}(K^{-1}\textbf{u}_{i})$, where
$K^{-1}\in\mathbb{R}^{3\times 3}$ contains the camera intrinsics and
$R_{i}^{wc}$ the rotation matrix of the corresponding camera pose. The
parallax angle between two rays is
$\cos(\alpha)=(\textbf{r}_{i}\cdot\textbf{r}_{j})/(|\textbf{r}_{i}||\textbf{r}_{j}|)$.
Figure 3 shows the angle between point observations. Despite the angle at the
end of both forward and backward passes being the same, it varies throughout
the trajectory. Consequently, if a threshold for the angle is set during point
initialization, the number of past and future observations may differ between
the forward and backward passes. This discrepancy becomes more prominent when
matching functions utilize distinct parameters or heuristics. These variations
can result in a different number of observations during forward and backward
motion, ultimately leading to differences in performance.

In our modification of ORB-SLAM2, we have changed the implementation of
association functions to ensure the utilization of consistent geometry and
appearance constraints.

1:function Tracking ($\mathcal{I}$)

2: $\triangleright$ $\mathcal{I}$ = RGB image

3: if Not Initialized then

4: Try to initialize ($\mathcal{I}$)

5: end if

6: Track With Motion Model () Search By Projection v1 ($c_{1}$)

7: Track Local Map () Search By Projection v2 ($c_{2}$)

8: if Need New KeyFrame () then

9: $\mathcal{K}_{n}$ $\leftarrow$ Create New KeyFrame ()

10: end if

11:end function

1:function Mapping ($\mathcal{K}_{n}$)

2: $\triangleright$ $\mathcal{K}_{n}$ = new keyframe

3: $\mathcal{P}$ $\leftarrow$ Map Point Culling ($\mathcal{P}$)

4: $\mathcal{P}_{n}$ $\leftarrow$ Create New Pts
($\mathcal{K}_{n}$,$\mathcal{K}$) Search For Triangulation ($c_{3}$)

5: Search In Neighbors () Fuse ($\mathcal{P}_{n}$,$\mathcal{K}$,$c_{4}$) Fuse
($\mathcal{P}$,$\mathcal{K}_{n}$,$c_{4}$)

6: $\mathcal{P}\leftarrow\mathcal{P}\cup\mathcal{P}_{n}$ ,
$\mathcal{K}\leftarrow\mathcal{K}\cup\mathcal{K}_{n}$

7: Local Bundle Adjustment ($\mathcal{P}$,$\mathcal{K}$) Erase Outliers ()

8: $\mathcal{K}$ $\leftarrow$ KeyFrame Culling ($\mathcal{K}$)

9:end function

Figure 3: Left: Tracking and mapping threads of [Mur-Artal and Tardós(2017)]
illustrating the distribution of data association functions throughout a
standard pipeline (underlined in the pseudo-code). $c_{1,2,3,4}$ represents
variations in the chosen descriptor distance thresholds used for feature
matching. Right: Asymmetry in the angle between observations of a point under
fwd/bwd motion.

Robust Data Association. A common approach to expedite data association is to
employ greedy strategies, where the matching of keypoints is performed
strictly in sequential order. This sequential ordering can lead to biased
matching results, as keypoints may be ordered based on their position in the
image or their resolution octave. We have modified the matching strategy in
the ORB-SLAM2 code to avoid its potential bias. We prioritize matches based on
the lowest Hamming distance instead of relying on sequential ordering.

##  4 Ablation Study

In this section, we evaluate quantitatively how the methods proposed in
Section 3 contribute to the reduction of motion bias. Specifically, we examine
the effects of descriptor selection, point appearance invariance, visual
covariances, outlier rejection, symmetric data association, and robust data
association. Following [Yang et al.(2018)Yang, Wang, Gao, and Cremers], we
evaluate the accuracy of the trajectories by calculating an alignment error,
denoted as $e_{r}$. This error is estimated as the translational root-mean-
squared-error (RMSE) between the estimated trajectory, after it is aligned to
the start and end segments of the ground truth trajectory. For further
details, please refer to [Engel et al.(2016)Engel, Usenko, and Cremers]. To
quantify the motion bias, we define it as the difference between the alignment
error of the forward pass and the alignment error of the backward pass:
$e_{r}(f)-e_{r}(b)$.

###  4.1 Implementation Details

Deterministic ORB-SLAM2. SLAM systems are susceptible to non-deterministic
effects that can significantly impede their performance and hinder our ability
to perform ablation studies on specific processes or variables. In this study,
we introduce a modified version of ORB-SLAM2 [Mur-Artal and Tardós(2017)] that
has been specifically designed to guarantee full determinism, rendering it
highly suitable for conducting ablation studies222Our deterministic
implementation of ORB-SLAM2 [Mur-Artal and Tardós(2017)] maintained all
capabilities of the original baseline. However, similar to [Engel et
al.(2016)Engel, Usenko, and Cremers, Yang et al.(2018)Yang, Wang, Gao, and
Cremers], we deactivated loop closure to maximize the impact of motion bias in
our experiments.. Specifically, 1) we have altered both the tracking and
mapping threads to ensure that they operate sequentially, thereby eliminating
online non-deterministic behavior resulting from the real-time implementation.
2) In addition, we have addressed implicit non-deterministic behavior that
arose from dynamic memory allocation and random sample consensus (RANSAC).

Keyframe management. Selecting a subset of representative keyframes in order
to enable real-time global bundle adjustment and relocalization is an open
problem in SLAM [Konolige and Agrawal(2008), Strasdat et al.(2010)Strasdat,
Montiel, and Davison, Leutenegger et al.(2013)Leutenegger, Furgale, Rabaud,
Chli, Konolige, and Siegwart, Younes et al.(2017)Younes, Asmar, Shammas, and
Zelek, Schmuck and Chli(2019), Fontan et al.(2023)Fontan, Giubilato, Oliva,
Civera, and Triebel]. State-of-the-art approaches often rely on policies that
are designed to maximize performance on specific benchmarks, and may degrade
when they are tested on out-of-distribution sequences. To ensure homogeneous
executions in our forward/backward experiments across four different datasets,
we process every new frame as a keyframe but keep only one out of every five
together with the latest five keyframes. These numbers are a reasonable
compromise in our experiments between computational efficiency and
performance.

###  4.2 Ablation Results

Table 1 illustrates the impact of each proposed method on accuracy and motion
bias. We assess the relative contribution of each method by implementing all
of them and then conducting separate evaluations by deactivating each method
one at a time.

The main factor contributing to motion bias is the outlier rejection, which
creates a significant gap between forward and backward runs by artificially
improving one pass while degrading the other. Our contributions effectively
balance the forward and reverse passes in both datasets, thereby eliminating
motion bias.

|  Kitti Dataset [Geiger et al.(2013)Geiger, Lenz, Stiller, and Urtasun]
---|---
| $e_{r}(f)$ | $e_{r}(b)$ | $e_{r}(f)-e_{r}(b)$
| rmse | mean/std | rmse | mean/std | rmse | mean/std
Ours (full) | 72.7 | 58.3/45.7 | 71.4 | 56.8/45.4 | 6.9 | 1.4/ 7.1
wo. Descriptor Sel. | 74.4 | 59.4/47.2 | 79.0 | 63.4/49.6 | 15.1 | -3.9/15.3
wo. Point App. Inv. | 64.7 | 52.7/39.5 | 71.9 |  56.7/46.5 | 9.4 | -4.0/8.9
wo. Robust Assoc. | 72.4 | 57.0/47.1 | 69.3 | 57.6/40.6 | 17.8 |  -0.6/18.7
wo. Sym. Assoc. | 90.6 |  72.4/57.3 | 71.8 | 59.1/43.0 | 24.9 | 13.3/22.2
wo. Visual Cov. | 74.4 | 59.4/47.1 | 72.2 | 59.4/43.5 | 13.2 | 3.0/14.1
wo. Outlier Reject. | 59.8 |  50.7/33.4 | 88.9 |  69.7/58.2 | 31.2 |  -19.0/26.1

TUM Mono VO Dataset [Engel et al.(2016)Engel, Usenko, and Cremers]
---
$e_{r}(f)$ | $e_{r}(b)$ | $e_{r}(f)-e_{r}(b)$
rmse | mean/std | rmse | mean/std | rmse | mean/std
6.7 | 4.8/4.7 | 6.8 | 5.3/4.3 | 3.9 | -1.3/3.7
13.0 | 5.9/11.7 | 5.3 | 4.3/3.2 | 5.4 | -0.5/5.5
6.7 | 5.0/4.5 | 6.2 | 4.9/3.7 | 2.8 |  -0.1/2.9
5.8 | 4.5/3.8 | 8.3 |  5.8/6.0 | 4.3 | -0.5/4.3
7.6 |  4.4/6.2 | 4.7 | 4.1/2.4 | 4.1 | -1.3/3.9
8.0 | 5.0/6.2 | 7.0 | 5.3/4.6 | 6.2 | 0.3/6.3
15.2 |  7.3/13.5 | 3.3 |  2.4/ 2.4 | 6.3 |  3.3/5.4

Table 1: Ablation Study in KITTI [Geiger et al.(2013)Geiger, Lenz, Stiller,
and Urtasun] and TUM Mono VO [Engel et al.(2016)Engel, Usenko, and Cremers].
We report the root-mean-squared-error (RMSE), mean and standard deviation
(std) of the trajectory alignment error ($e_{r}$) in meters (all better if
closer to zero). We show results for the forward (f) and backward (b) passes,
and underline in red/green the worst/best result for each metric. Values in
the last columns evaluate the motion bias effect (smaller with errors closer
to zero). Every row evaluates a configuration where all contributions except
one are implemented.

(a) # Map Points

(b) # Local Keyframes

(c) # Observation Inliers

Figure 4: Other motion bias metrics. We show the differences on the total
number [#] of map points (Pt), local keyframes (Key), and observation inliers
(Inl.) between forward (f) and backward (b) passes. As before, our
contributions effectively minimize the differences across the three
indicators, providing evidence of reduced motion bias.

An alternative approach to evaluate motion bias without relying on camera
trajectory ground truth is by comparing key statistics of the SLAM graph.
Figure 4 illustrates how our proposed approaches effectively reduce the
differences across three indicators, providing evidence of reduced motion
bias.

##  5 Results

We selected four publicly available datasets, encompassing a wide range of
potential sources for motion bias, including indoor/outdoor environments,
walking/driving scenarios, high/low video frequencies, and strong/soft
forward/backward motion.

Table 2 provides an overview of these dataset characteristics and shows the
errors and motion bias metrics obtained by our deterministic implementation of
ORB-SLAM2. Note first how motion bias appear in the four datasets, evidencing
that it is a general problem in SLAM. With our contributions, we successfully
eliminate motion bias across all datasets, effectively balancing the accuracy
between forward and backward passes. Notably, in the KITTI dataset, motion
bias is completely eradicated, leading to a significant improvement in the
accuracy of the backward pass. For detailed results per sequence, refer to
Table 3.

|  |  |  |  | $e_{r}$ (f) |  $e_{r}$ (b) |  $e_{r}$ (f) - $e_{r}$ (b)
---|---|---|---|---|---|---|---
Dataset |  |  | Hz | Fwd/Bwd | Van. | Ours | Van. | Ours | Van. | Ours
TUM Mono VO [Engel et al.(2016)Engel, Usenko, and Cremers] | Indoor | Walk | 40 | Medium | 6.10 | 6.74 | 1.89 | 6.86 | 4.51 | 3.93
RGB-D SLAM [Sturm et al.(2012)Sturm, Engelhard, Endres, Burgard, and Cremers] | Indoor | Walk | 30 | Soft | 0.07 | 0.06 | 0.03 | 0.05 | 0.02 | 0.01
EuRoC [Burri et al.(2016)Burri, Nikolic, Gohl, Schneider, Rehder, Omari, Achtelik, and Siegwart] | Indoor | Drone | 20 | Soft | 0.07 | 0.16 | 0.09 | 0.07 | 0.04 | 0.01
KITTI [Geiger et al.(2013)Geiger, Lenz, Stiller, and Urtasun] | Outdoor | Car | 10 | Strong | 76.42 | 72.72 | 132.24 | 71.40 | 65.67 | 6.94

Table 2: Final Evaluation. Our approach eliminates motion bias in the four
datasets (third column in orange) while keeping global accuracy constant. As
expected the reduction of motion bias is stronger in Kitti and TUM Mono VO
datasets where the motion occurs mainly in the forward/backward direction of
the camera. Values are the medians after outlier removal.

Sequence | 00 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10
---|---|---|---|---|---|---|---|---|---|---
$e_{r}(f)$ Van. | 128.3 | 38.0 | 9.8 | 4.1 | 65.1 | 93.2 | 40.1 | 123.1 | 100.4 | 23.4
$e_{r}(b)$ Van. | 254.9 | 179.3 | 6.9 | 0.9 | 115.3 | 111.7 | 53.3 | 183.2 | 120.3 | 29.9
$e_{r}(f)-e_{r}(b)$ Van. | -126.6 | -141.3 | 2.9 | 3.2 | -50.1 | -18.4 | -13.1 | -60.0 | -19.9 | -6.5
$e_{r}(f)$ Ours | 140.1 | 27.1 | 4.1 | 4.8 | 72.0 | 96.0 | 43.6 | 102.6 | 71.6 | 21.0
$e_{r}(b)$ Ours | 136.6 | 41.8 | 2.0 | 0.7 | 66.2 | 100.7 | 38.9 | 102.9 | 59.4 | 19.3
$e_{r}(f)-e_{r}(b)$ Ours | 3.5 | -14.7 | 2.0 | 4.1 | 5.7 | -4.6 | 4.7 | -0.2 | 12.2 | 1.68

Table 3: Per-sequence results in KITTI. Our motion-bias-free implementation
removes the bias ($e_{r}(f)-e_{r}(b)$) between forward and backward passes in
most sequences.

##  6 Conclusion & Future Work

In this paper, we are motivated by the significant impact of motion bias on
state-of-the-art feature-based SLAM systems. Our findings reveal substantial
inconsistencies in performance, not only between forward and reverse
directions of travel but also in terms of which direction exhibits better
performance.

Our study reveals that the primary factor contributing to the observed
inconsistencies is the outlier rejection process. To address this issue, we
propose an alternative policy and introduce a new model for visual covariances
that effectively removes motion bias. By implementing this and other
contributions in ORB-SLAM2, we demonstrate a substantial enhancement in the
consistency between forward and backward motion, along with improved average
trajectory error in both directions, across four diverse datasets. This novel
approach yields more trustable and generalizable performance, which is of
great significance in critical applications where consistency of performance
is crucial for usability and practicality.

Future work should focus on investigating how outlier management techniques
can effectively eliminate motion bias while maintaining the highest level of
trajectory accuracy (Section 4.2). Additionally, we will further explore the
impact of image resolution (Section 2) and keyframe management strategies
(Section 4.1) on inducing motion bias.

## References

  * [Agarwal et al.(2020)Agarwal, Vora, Pandey, Williams, Kourous, and McBride] Siddharth Agarwal, Ankit Vora, Gaurav Pandey, Wayne Williams, Helen Kourous, and James McBride.  Ford multi-av seasonal dataset.  _The International Journal of Robotics Research_ , 39(12):1367–1376, 2020.
  * [Bailey and Durrant-Whyte(2006)] Tim Bailey and Hugh Durrant-Whyte.  Simultaneous localization and mapping (slam): Part ii.  _IEEE robotics & automation magazine_, 13(3):108–117, 2006.
  * [Ballester et al.(2021)Ballester, Fontán, Civera, Strobl, and Triebel] Irene Ballester, Alejandro Fontán, Javier Civera, Klaus H Strobl, and Rudolph Triebel.  Dot: Dynamic object tracking for visual slam.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 11705–11711. IEEE, 2021.
  * [Bescos et al.(2018)Bescos, Fácil, Civera, and Neira] Berta Bescos, José M Fácil, Javier Civera, and José Neira.  Dynaslam: Tracking, mapping, and inpainting in dynamic scenes.  _IEEE Robotics and Automation Letters_ , 3(4):4076–4083, 2018.
  * [Burri et al.(2016)Burri, Nikolic, Gohl, Schneider, Rehder, Omari, Achtelik, and Siegwart] Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart.  The euroc micro aerial vehicle datasets.  _The International Journal of Robotics Research_ , 35(10):1157–1163, 2016.
  * [Cadena et al.(2016)Cadena, Carlone, Carrillo, Latif, Scaramuzza, Neira, Reid, and Leonard] Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J Leonard.  Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age.  _IEEE Transactions on robotics_ , 32(6):1309–1332, 2016.
  * [Campos et al.(2021)Campos, Elvira, Rodríguez, Montiel, and Tardós] Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, and Juan D Tardós.  Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam.  _IEEE Transactions on Robotics_ , 37(6):1874–1890, 2021.
  * [Chli and Davison(2009)] Margarita Chli and Andrew J Davison.  Active matching for visual tracking.  _Robotics and Autonomous Systems_ , 57(12):1173–1187, 2009.
  * [Civera et al.(2008)Civera, Davison, and Montiel] Javier Civera, Andrew J Davison, and JM Martinez Montiel.  Inverse depth parametrization for monocular slam.  _IEEE transactions on robotics_ , 24(5):932–945, 2008.
  * [Concha and Civera(2015)] Alejo Concha and Javier Civera.  An evaluation of robust cost functions for rgb direct mapping.  In _2015 European Conference on Mobile Robots (ECMR)_ , pages 1–8. IEEE, 2015.
  * [Davison et al.(2007)Davison, Reid, Molton, and Stasse] Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse.  Monoslam: Real-time single camera slam.  _IEEE transactions on pattern analysis and machine intelligence_ , 29(6):1052–1067, 2007.
  * [Dissanayake et al.(2011)Dissanayake, Huang, Wang, and Ranasinghe] Gamini Dissanayake, Shoudong Huang, Zhan Wang, and Ravindra Ranasinghe.  A review of recent developments in simultaneous localization and mapping.  In _2011 6th International Conference on Industrial and Information Systems_ , pages 477–482. IEEE, 2011.
  * [Durrant-Whyte and Bailey(2006)] Hugh Durrant-Whyte and Tim Bailey.  Simultaneous localization and mapping: part i.  _IEEE robotics & automation magazine_, 13(2):99–110, 2006.
  * [Engel et al.(2014)Engel, Schöps, and Cremers] Jakob Engel, Thomas Schöps, and Daniel Cremers.  Lsd-slam: Large-scale direct monocular slam.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II 13_ , pages 834–849. Springer, 2014.
  * [Engel et al.(2016)Engel, Usenko, and Cremers] Jakob Engel, Vladyslav Usenko, and Daniel Cremers.  A photometrically calibrated benchmark for monocular visual odometry.  _arXiv preprint arXiv:1607.02555_ , 2016.
  * [Engel et al.(2017)Engel, Koltun, and Cremers] Jakob Engel, Vladlen Koltun, and Daniel Cremers.  Direct sparse odometry.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(3):611–625, 2017.
  * [Fontan et al.(2022)Fontan, Oliva, Civera, and Triebel] Alejandro Fontan, Laura Oliva, Javier Civera, and Rudolph Triebel.  Model for multi-view residual covariances based on perspective deformation.  _IEEE Robotics and Automation Letters_ , 7(2):1960–1967, 2022.
  * [Fontan et al.(2023)Fontan, Giubilato, Oliva, Civera, and Triebel] Alejandro Fontan, Riccardo Giubilato, Laura Oliva, Javier Civera, and Rudolph Triebel.  Sid-slam: Semi-direct information-driven rgb-d slam.  _IEEE Robotics and Automation Letters_ , 2023.
  * [Forster et al.(2016)Forster, Zhang, Gassner, Werlberger, and Scaramuzza] Christian Forster, Zichao Zhang, Michael Gassner, Manuel Werlberger, and Davide Scaramuzza.  Svo: Semidirect visual odometry for monocular and multicamera systems.  _IEEE Transactions on Robotics_ , 33(2):249–265, 2016.
  * [Geiger et al.(2013)Geiger, Lenz, Stiller, and Urtasun] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun.  Vision meets robotics: The kitti dataset.  _The International Journal of Robotics Research_ , 32(11):1231–1237, 2013.
  * [Kerl et al.(2013)Kerl, Sturm, and Cremers] Christian Kerl, Jürgen Sturm, and Daniel Cremers.  Robust odometry estimation for rgb-d cameras.  In _2013 IEEE international conference on robotics and automation_ , pages 3748–3754. IEEE, 2013.
  * [Klein and Murray(2007)] Georg Klein and David Murray.  Parallel tracking and mapping for small ar workspaces.  In _2007 6th IEEE and ACM international symposium on mixed and augmented reality_ , pages 225–234. IEEE, 2007.
  * [Konolige and Agrawal(2008)] Kurt Konolige and Motilal Agrawal.  Frameslam: From bundle adjustment to real-time visual mapping.  _IEEE Transactions on Robotics_ , 24(5):1066–1077, 2008.
  * [Leutenegger et al.(2013)Leutenegger, Furgale, Rabaud, Chli, Konolige, and Siegwart] Stefan Leutenegger, Paul Furgale, Vincent Rabaud, Margarita Chli, Kurt Konolige, and Roland Siegwart.  Keyframe-based visual-inertial slam using nonlinear optimization.  _Proceedings of Robotis Science and Systems (RSS) 2013_ , 2013.
  * [Maddern et al.(2017)Maddern, Pascoe, Linegar, and Newman] Will Maddern, Geoffrey Pascoe, Chris Linegar, and Paul Newman.  1 year, 1000 km: The oxford robotcar dataset.  _The International Journal of Robotics Research_ , 36(1):3–15, 2017.
  * [Meyer et al.(2021)Meyer, Smíšek, Fontan Villacampa, Oliva Maza, Medina, Schuster, Steidle, Vayugundla, Müller, Rebele, et al.] Lukas Meyer, Michal Smíšek, Alejandro Fontan Villacampa, Laura Oliva Maza, Daniel Medina, Martin J Schuster, Florian Steidle, Mallikarjuna Vayugundla, Marcus G Müller, Bernhard Rebele, et al.  The madmax data set for visual-inertial rover navigation on mars.  _Journal of Field Robotics_ , 38(6):833–853, 2021.
  * [Mur-Artal and Tardós(2017)] Raul Mur-Artal and Juan D Tardós.  Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras.  _IEEE transactions on robotics_ , 33(5):1255–1262, 2017.
  * [Mur-Artal et al.(2015)Mur-Artal, Montiel, and Tardos] Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos.  Orb-slam: a versatile and accurate monocular slam system.  _IEEE transactions on robotics_ , 31(5):1147–1163, 2015.
  * [Newcombe et al.(2011)Newcombe, Lovegrove, and Davison] Richard A Newcombe, Steven J Lovegrove, and Andrew J Davison.  Dtam: Dense tracking and mapping in real-time.  In _2011 international conference on computer vision_ , pages 2320–2327. IEEE, 2011.
  * [Recasens et al.(2021)Recasens, Lamarca, Fácil, Montiel, and Civera] David Recasens, José Lamarca, José M Fácil, JMM Montiel, and Javier Civera.  Endo-depth-and-motion: reconstruction and tracking in endoscopic videos using depth networks and photometric constraints.  _IEEE Robotics and Automation Letters_ , 6(4):7225–7232, 2021.
  * [Schmuck and Chli(2019)] Patrik Schmuck and Margarita Chli.  On the redundancy detection in keyframe-based slam.  In _2019 International Conference on 3D Vision (3DV)_ , pages 594–603. IEEE, 2019.
  * [Schops et al.(2019)Schops, Sattler, and Pollefeys] Thomas Schops, Torsten Sattler, and Marc Pollefeys.  BAD SLAM: Bundle Adjusted Direct RGB-D SLAM.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019.
  * [Strasdat et al.(2010)Strasdat, Montiel, and Davison] Hauke Strasdat, JMM Montiel, and Andrew J Davison.  Real-time monocular slam: Why filter?  In _2010 IEEE International Conference on Robotics and Automation_ , pages 2657–2664. IEEE, 2010.
  * [Sturm et al.(2012)Sturm, Engelhard, Endres, Burgard, and Cremers] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers.  A Benchmark for the Evaluation of RGB-D SLAM Systems.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2012.
  * [Teed and Deng(2021)] Zachary Teed and Jia Deng.  Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.  _Advances in neural information processing systems_ , 34:16558–16569, 2021.
  * [Xu et al.(2019)Xu, Li, Tzoumanikas, Bloesch, Davison, and Leutenegger] Binbin Xu, Wenbin Li, Dimos Tzoumanikas, Michael Bloesch, Andrew Davison, and Stefan Leutenegger.  Mid-fusion: Octree-based object-level multi-instance dynamic slam.  In _2019 International Conference on Robotics and Automation (ICRA)_ , pages 5231–5237. IEEE, 2019.
  * [Yang et al.(2018)Yang, Wang, Gao, and Cremers] Nan Yang, Rui Wang, Xiang Gao, and Daniel Cremers.  Challenges in monocular visual odometry: Photometric calibration, motion bias, and rolling shutter effect.  _IEEE Robotics and Automation Letters_ , 3(4):2878–2885, 2018.
  * [Younes et al.(2017)Younes, Asmar, Shammas, and Zelek] Georges Younes, Daniel Asmar, Elie Shammas, and John Zelek.  Keyframe-based monocular slam: design, survey, and future directions.  _Robotics and Autonomous Systems_ , 98:67–88, 2017.