NORMAL  $\BODY$

Hao Zhang, Human-Centered Robotics Lab, Colorado School of Mines, Golden, CO
80401, USA

# Self-Reflective Terrain-Aware Robot Adaptation for Consistent Off-Road
Ground Navigation

Sriram Siva11affiliationmark:  Maggie Wigness22affiliationmark:  John G.
Rogers22affiliationmark:  Long Quang22affiliationmark:  and Hao
Zhang11affiliationmark:  11affiliationmark:  Human-Centered Robotics Lab,
Colorado School of Mines, Golden, CO 80401, USA.
22affiliationmark:  Army Research Laboratory (ARL), Adelphi, MD 20783, USA.
<EMAIL_ADDRESS>
###### Abstract

Ground robots require the crucial capability of traversing unstructured and
unprepared terrains and avoiding obstacles to complete tasks in real-world
robotics applications such as disaster response. When a robot operates in off-
road field environments such as forests, the robot’s actual behaviors often do
not match its expected or planned behaviors, due to changes in the
characteristics of terrains and the robot itself. Therefore, the capability of
robot adaptation for consistent behavior generation is essential for
maneuverability on unstructured off-road terrains. In order to address the
challenge, we propose a novel method of self-reflective terrain-aware
adaptation for ground robots to generate consistent controls to navigate over
unstructured off-road terrains, which enables robots to more accurately
execute the expected behaviors through robot self-reflection while adapting to
varying unstructured terrains. To evaluate our method’s performance, we
conduct extensive experiments using real ground robots with various
functionality changes over diverse unstructured off-road terrains. The
comprehensive experimental results have shown that our self-reflective
terrain-aware adaptation method enables ground robots to generate consistent
navigational behaviors and outperforms the compared previous and baseline
techniques.

###### keywords:

Ground navigation, robot learning, terrain adaptation, self-reflection

##  1 Introduction

Autonomous ground robots require the crucial capability of navigating over
unstructured and unprepared terrains in off-road environments and avoiding
obstacles to complete tasks in real-world applications such as search and
rescue, disaster response, and reconnaissance (Kawatsuma et al. (2012); Park
et al. (2017); Schwarz et al. (2017); Kuntze et al. (2012)). When operating in
field environments, ground robots need to navigate over a wide variety of
unstructured off-road terrains with changing types, slope, friction, and other
characteristics that cannot be fully modeled beforehand. In addition, ground
robots operating over a long-period of time often experience changes in their
own functionalities, including damages (e.g., failed robot joints and flat
tires), natural wear and tear (e.e., reduced tyre traction), and varying robot
configurations (e.g., varying payload). As illustrated in Figure 1, such
challenges make robot navigation in unstructured off-road environments a
challenging problem. Thus, robot adaptation to changes in terrains and robot
functionalities is essential to the success of offroad ground navigation.

Figure 1:  A motivating scenario of self-reflective terrain-aware robot
adaptation for generating consistent robot navigational behaviors. When robots
operate in unstructured off-road environments, the actual behaviors often do
not match their expected behaviors, due to changes in the characteristics of
terrains and the robots themselves. Therefore, the capability of robot
adaptation for consistent behavior generation is essential for maneuverability
over unstructured off-road terrains.

Given its importance, robot adaptation to unstructured off-road terrains has
been an active area of research over the past decades. Previous methods can be
broadly categorized into model-based and learning-based approaches. Model-
based methods often use control theories to model ground robots’ dynamics for
navigation (Bhattacharyya et al. (2018); Haruno et al. (2001)). However, they
typically ignore terrain characteristics. Learning-based approaches can learn
terrain complexity and estimate the possibility of traversing terrains through
terrain recognition (Bermudez et al. (2012); Peynot et al. (2014); Hudjakov
and Tamre (2009)). Learning-based methods were also developed to adapt
navigational behaviors according to the environment (Posada et al. (2010); Han
et al. (2017)) and transfer expert demonstrations on navigational controls to
autonomous ground robots based upon learning from demonstration (LfD) (Silver
et al. (2010); Wigness et al. (2018); Siva et al. (2019)).

Although previous learning-based approaches have shown promising performance,
the expected navigational behaviors generated by previous approaches cannot
always be executed accurately by a robot when it navigates over unstructured
off-road terrains. That is – a robot’s actual navigational behaviors may not
be consistent with their expected behaviors. This inconsistency is caused
mainly by two reasons. First, while ground robots navigate in real-world off-
road environments, they experience terrains with a wide variety of
characteristics that cannot be modeled beforehand, for example, tall grass
terrain with hidden rocks as illustrated in Figure 1. Second, robots may also
experience negative effects or the so-called setbacks (Borges et al. (2019);
Knight et al. (2001)), that are defined as the changes in the robot’s
functionalities that increase the difficulty for a robot to accomplish its
expected behaviors. Examples of the setbacks include malfunctioning robot
joints, reduced wheel traction, and heavy payload. The previous methods were
not able to learn to jointly adapt to changes in both terrains and setbacks.

In order to these above shortcomings, we develop a novel method of self-
reflective terrain-aware robot adaptation for consistent navigational behavior
generation, which enables robots to accurately execute the expected behaviors
through robot self-reflection while adapting to varying unstructured off-road
terrains. In psychology, self-reflection is considered as the humans’ ability
to modulate our behaviors by being aware of ourselves (Marcovitch et al.
(2008); Smith (2002); Ardelt and Grunwald (2018)). We use the term self-
reflection in robotics to refer to the robot’s capability of adopting self-
awareness (e.g., a robot knows it has slower speeds than expected) to modulate
navigational behavior generation. In order to enable robot self-reflection,
our method monitors the difference of expected and actual behaviors that is
caused by the robot’s setbacks, and accordingly adapts the robot’s
navigational behaviors to minimize this difference in order to generate
consistent navigational behaviors. Our approach also learns representations of
unstructured terrains, which are jointly used to classify unstructured
terrains and generate navigational behaviors. In addition, our approach is
able to fuse historical observations acquired from different onboard sensors
to characterize terrains, and automatically identify the historical
observations that provide the most important information for generating
consistent behaviors. The above components are all integrated into a unified
mathematical framework of regularized constrained optimization with a
theoretical convergence guarantee.

The contribution of this paper focuses on the introduction of the first method
of self-reflective terrain-aware adaptation for ground robots to generate
consistent navigational controls to traverse unstructured off-road
environments. The specific novelties of this paper include:

  * •

We introduce a novel method for terrain-aware robot adaptation to unstructured
off-road terrains, which is able to simultaneously recognize terrains and
generate corresponding navigational behaviors, as well as to identify the most
important terrain features for terrain adaptation.

  * •

We introduce the novel idea and implement one of the first approaches of self-
reflection for ground robots to monitor their navigational behaviors and
setbacks in order to enhance consistent ground maneuverability.

  * •

We implement a new optimization algorithm to effectively address the
regularized constrained optimization problem that is formulated for self-
reflective terrain-aware robot adaptation, which holds a theoretical guarantee
to converge to the global optimal solution.

Furthermore, as an experimental contribution, we perform extensive experiments
and provide a comprehensive performance evaluation of terrain adaptation
methods by designing a set of scenarios for ground robots with various
functionality changes over diverse individual and complex unstructured off-
road terrain.

The remainder of this paper is structured as follows. A review of related work
is offered in Section 2. Our proposed methods for self-reflective terrain-
aware robot adaptation is discussed in Section 3. We derive the optimization
algorithm in 4. After discussing the experimental results in Section 5, we
finally conclude the paper in Section 6.

##  2 Related Work

In this section, we provide a review of the related research in the field of
robot adaptation to unstructured terrains. Existing methods for terrain
adaptation can be broadly classified into two categories, including model-
based and learning-based methods. Previous learning-based approaches can be
further divided into methods for terrain classification and learning-based
adaptation.

###  2.1 Control Methods for Ground Robot Navigation

Control theory has been commonly applied to implement ground robot navigation
over terrains in robotics. Many early methods used fuzzy logic to implement
robot control for ground navigation over unstructured terrains (Babunski et
al. (2020); Adib and Masoumi (2017); Fries (2018)). As these methods didn’t
consider robot dynamics, methods were proposed to generate navigational
controls that also model a robot’s dynamics (Meng and Kak (1993); Kelly
(1994); Dash et al. (2015); Verginis and Dimarogonas (2021); Tsunoda et al.
(2020)). Under control theory, system identification has been commonly
considered to model unknown dynamic systems by observing the behavior of the
system to various robot inputs(Johansson (1993); Rabiner et al. (1978)). For
the specific case of non-linear robotic systems koopman operator theory
(Koopman (1931)) provided a way of using infinite dimensional linear
observable space to model robot behaviors (Proctor et al. (2018); Williams et
al. (2015)).

Robot navigation problems were solved by generating optimal trajectories from
robot dynamics model. The trajectory optimization modules are based on dynamic
programming, specifically differential dynamic programming and iterative
linear quadratic regulator (Yong and Zhou (1999); Tassa et al. (2014); Li and
Todorov (2004)). These methods are essentially an open loop control and thus
susceptible to model errors or disturbances in deployment. To address these
problems, model predictive control (MPC) (Camacho and Alba (2013)) generates
controls given the systems current state by repeatedly solving the optimal
control problem over a prediction horizon and only the first optimal control
is executed leading to a closed-loop control form.

Although classical control provide abundant models that allow for efficient
algorithms for high-dimensional tasks. However, these models can only
approximately the dynamics of the mobile robot. To cope with this challenge,
learning-based methods obtain robot’s dynamics directly from the data.

###  2.2 Terrain Classification Methods

Terrain classification methods make use of robot’s sensory data to classify
the terrains the robot traverses over. Many of the earlier methods were
developed to address the navigational challenge of larger vehicles (Bekker
(1969)) and classification was done in a manual fashion. There was an enormous
amount of research on terramechanics (Meyerhof (1970)), which is the guidance
of autonomous vehicles through rough terrains. A similar genre of speed
selection was used in DARPA grand challenge, to address the problem of
unstructured terrain navigation though fixing an optimal speed for each
terrain type by collecting data for months (Urmson et al. (2004)). Then
methods were used to generate digital terrain maps and then fitted a Gaussian
model to classify the terrains (Shimoda et al. (2005); Sidek and Sarkar
(2013)). A few techniques were also designed to use terrain ruggedness data to
evaluate navigational behaviors in an online fashion for ground vehicles
(Brooks and Iagnemma (2005); Sadhukhan et al. (2004)).

In learning based methods, terrain classification was usually performed by
learning from terrain data to classify the terrain for navigational behaviors.
For example, an Support Vector Machine (SVM) based classifier was used on
features learned from Hidden Markov Models to identify terrains (Trautmann and
Ray (2011)). Color based terrain classification was performed to label
obstacles and generate navigational behaviors (Manduchi et al. (2005)). More
recently, semantic segmentation methods based on deep learning (Long et al.
(2015)) were successfully used to classify off-road terrains (Jiang et al.
(2020); Wigness et al. (2019)) and perform the task of terrain navigation.

Although effective, these methods solely rely in discrete categorization of
terrain types. In real world environments, unstructured off-road terrains have
a wide variety of characteristics and cannot be categorized into distinctive
types. Thus these methods cannot enable in a mobile robot to adapt in real-
world off-road field environments.

###  2.3 Learning-based Adaptation Methods

This category of methods focus on enabling robots to acutely adapt to the
various terrains. High-level models were developed and implemented to solve
the general problem of robot terrain adaptation in (He et al. (2019);
Nikolaidis et al. (2017); Papadakis (2013); Parker (1996, 2000)). A different
genre of methods used case-based reasoning to adapt a robot in dynamic
environments (Watson and Marir (1994)). Many model-based approaches were
formulated to learn a single global model using function approximators such as
the Gaussian processes (Ko and Fox (2009); Deisenroth and Rasmussen (2011))
and neural networks (Lenz et al. (2015)). Some methods then used probabilistic
Gaussian models (Plagemann et al. (2008)) to learn terrain models and update
the models efficiently using sparse approximation. The use of ground speed as
an optimization variable and formulating methods for trading progress and
velocity with changing terrains is observed in earlier works (Fox et al.
(1997)). Recently, methods were developed that used an array of inertial and
ultrasonic sensors to recognize soil properties and perform terrain adaptation
(Nabulsi et al. (2006)).

With the popularity of learning based methods in the field of robot learning,
learning based terrain adaptation methods also started to gain significant
attention due to their effectiveness and robustness (Thrun (1998)). Terrain
adaptation was initially addressed from the perspective of online learning
where the model parameters were updated continuously in the execution stage
(Kleiner et al. (2002)). Although successful, these methods lacked the ability
to adapt on the fly with the changes in unstructured terrains. This led to
development of open-loop control methods that generated navigational behaviors
according to the terrain characteristics (Plagemann et al. (2008)).
Reinforcement learning based navigation was employed to generate stable
locomotion patterns for terrain adaptation (Erden and Leblebicioğlu (2008)).
Techniques based on inverse reinforcement learning were employed to mimic
expert navigational behaviors to achieve near human-level maneuverability
(Wigness et al. (2018)). More recently, an approach for terrain aware
apprenticeship learning has showed promising results for ground robot
navigation on unstructured terrains (Siva et al. (2019)).

Given the promise of previous learning based terrain adaptation methods,
almost all previous approaches focus on learning the expected navigational
behaviors, without addressing the need of self-reflection to generate
consistent navigational behaviors in these terrains.

##  3 Approach

In this section, we discuss our novel principled approach to enable ground
robots to adapt their navigational behaviors to unstructured off-road terrains
and perform self-reflection to generate consistent navigational behaviors. An
overview of our approach is shown in Figure 2.

Notation: We denote scalars using lowercase italic letters (e.g.,
$m\in\mathbb{R}$), vectors using boldface lowercase letters (e.g.,
$\mathbf{m}\in\mathbb{R}^{p}$), matrices using boldface capital letters, e.g.,
$\mathbf{M}=\\{{m}_{j}^{i}\\}\in\mathbb{R}^{p\times q}$ with its $i$-th row
and $j$-th column denoted as $\mathbf{m}^{i}$ and $\mathbf{m}_{j}$,
respectively. We use boldface capital Euler script letters to denote tensors
(i.e., 3D matrices), e.g.,
${\mathbfcal{M}}=\\{{{m}}_{j}^{i(k)}\\}\in\mathbb{R}^{p\times q\times r}$.
Unstacking tensor ${\mathbfcal{M}}$ along its height ($p$), width ($q$) and
depth ($r$) provides slices of matrices $\mathbf{M}^{i}$, $\mathbf{M}_{j}$ and
$\mathbf{M}^{(k)}$, respectively (Rabanser et al. (2017)). The key variables
used in our discussion are defined and briefly explained in Table 1.

Table 1:  Definition of the key variables used in our approach.  Variables | Definition
---|---
$\mathbf{x}_{t}\in\mathbb{R}^{d}$ |  Terrain feature vector extracted at time $t$
$\mathbf{X}\in\mathbb{R}^{d\times c}$ | Terrain feature instance matrix
$\mathbfcal{X}\in\mathbb{R}^{d\times n\times c}$ |  Feature tensor from $n$-different instances
$\mathbf{Z}\in\mathbb{Z}^{l\times n}$ |  Terrain indicator matrix corresponding to $\mathbfcal{X}$
$\mathbf{Y}\in\mathbb{R}^{b\times n}$ |  Expected robot navigational behaviors for $\mathbfcal{X}$
$\mathbf{A}\in\mathbb{R}^{b\times n}$ |  Actual robot navigational behaviors for $\mathbfcal{X}$
$\mathbfcal{E}\in\mathbb{R}^{b\times n\times c}$ | Behavior difference tensor
$\mathbfcal{W}\in\mathbb{R}^{l\times d\times c}$ | Weight tensor to encode terrain features
$\mathbfcal{V}\in\mathbb{R}^{b\times d\times c}$ | Weight tensor to learn estimated behaviors
$\mathbfcal{U}\in\mathbb{R}^{b\times b\times c}$ | Weight tensor to learn offset behaviors
$d$ | Dimensionality of the feature vector
$c$ | Past time steps used to acquire observations
$n$ | Number of instances
$l$ | Number of terrain types
$b$ | Number of robot behavior controls

###  3.1 Terrain-Aware Ground Navigation

We use multiple sensors installed on a ground robot to collect terrain
observations, including RGBD images, LiDAR scans and IMU readings, at each
time step when the robot traverses unstructured terrains. At each time step
$t$, we extract $m$ types of features (such as visual features and local
elevations) from the multi-sensory data. These features are concatenated into
a feature vector denoted as $\mathbf{x}^{(t)}\in\mathbb{R}^{d}$, where
$d=\sum_{i=1}^{m}d_{i}$ with $d_{i}$ denoting the dimensionality of the $i$-th
feature type. Features extracted from a sequence of $c$ past time steps are
stacked into a matrix as a terrain feature instance denoted as
$\mathbf{X}=[\mathbf{x}^{(t)};\dots;\mathbf{x}^{(t-c)}]\in\mathbb{R}^{d\times
c}$. To train our approach, we collect a set of $n$ feature instances that are
obtained when the robot traverses over various terrains, and denote this
training set as a terrain feature tensor
$\mathbfcal{X}=[\mathbf{X}_{1},\dots,\mathbf{X}_{n}]\in\mathbb{R}^{d\times
n\times c}$.

We denote the terrain types that are associated with $\mathbfcal{X}$ as
$\mathbf{Z}=[\mathbf{z}_{1},\dots,{\mathbf{z}_{n}}]\in\mathbb{Z}^{l\times n}$,
where $\mathbf{z}_{i}\in\mathbb{Z}^{l}$ denotes the vector of terrain types
and $l$ represents the number of terrain types. Each element
${z}_{j}^{i}\in\\{0,1\\}$ indicates whether $\mathbf{X}_{i}$ has the $j$-th
terrain type. During training, ground truth of the terrain types is provided
by human experts. Given $\mathbfcal{X}$ and $\mathbf{Z}$, terrain recognition
can be formulated as an optimization problem:

$\displaystyle\min_{\mathbfcal{W}}$
$\displaystyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})$
s.t. $\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}^{\top}=\mathbfcal{I}$ (1)
$\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}=\mathbfcal{W}^{\top}\otimes(\mathbfcal{W}^{\top})^{\top}$

where $\mathbfcal{W}\in\mathbb{R}^{l\times d\times c}$ is a weight tensor used
to encode the importance of each element in $\mathbfcal{X}$ towards estimating
terrain types. Each tensor element $w_{j}^{i(k)}\in\mathbfcal{W}$ denotes the
weight of the $j$-th terrain feature from the $k$-th past time step to
recognize the $i$-th terrain type. $\mathbf{W}^{i}$, $\mathbf{W}_{j}$ and
$\mathbf{W}^{(k)}$ are the slices of matrices that are obtained by unstacking
$\mathbfcal{W}$ along its height, width and depth, respectively.
$\mathbfcal{I}\in\mathbb{R}^{l\times d\times c}$ is the identity tensor (Qi
(2017)), which is used to mathematically define the orthogonality in tensors.
The operator $\otimes$ denotes the vector Kronecker tensor product that
performs the matrix-wise multiplication of the two tensors (Neudecker (1969)).
In Eq. (3.1), the tensor product $\otimes$ takes each terrain feature instance
$\mathbf{X}_{i}\in\mathbfcal{X}$, and multiplies it with the weight tensor
$\mathbfcal{W}$.

We adopt the Log-Cosh loss $\mathcal{L}_{lc}(\cdot)=log(cosh(\cdot))$ (Chen et
al. (2018); Xu et al. (2020)) in the objective function in Eq. (3.1) to encode
the error of using terrain features $\mathbfcal{X}$ to recognize terrain types
$\mathbf{Z}$, through the learning model that is parameterized by
$\mathbfcal{W}$. The advantage of using the Log-Cosh loss (e.g., over the
$\ell_{2}$-loss) is twofold. First, the Log-Cosh loss is robust to outliers,
because a linear error becomes a much smaller value using the logarithmic
scale, thus it can reduce the learning bias over outliers. Because
observations acquired by a robot when navigating over unstructured off-road
terrain are usually noisy, the use of the Log-Cosh loss is desirable. Second,
$\mathcal{L}_{lc}(\cdot)$ provides a better optimization of $\mathbfcal{W}$
near the optimality. This is because, near the optimal value of
$\mathbfcal{W}$, the objective value computed by $\mathcal{L}_{lc}(\cdot)$ in
Eq. (3.1) can change significantly when $\mathbfcal{W}$ shows small changes
(Chen et al. (2018)). The two constraints together in Eq. (3.1) are the
necessary conditions for orthogonality. Orthogonality in tensors makes each
slice of matrices that stack up a tensor to be full rank, which means that
each slice of matrices is independent and orthogonal (or perpendicular) to
each other and thus sharing the least similarity (Li et al. (2018)). In our
formulation, the orthogonality makes $\mathbf{W}^{i},i=1,\dots,l$ that
correspond to all the terrain types to be independent of each other.

Figure 2:  Overview of our proposed method for self-reflective terrain-aware
robot adaptation to enable ground robots to adapt their navigational behaviors
to unstructured off-road terrains and perform self-reflection to generate
consistent navigational behaviors under the unified constrained optimization
framework.

The first main novelty of this paper is that we propose a new method for
terrain-aware navigation adaptation by joint terrain classification and
behavior learning under a unified constrained optimization framework. This is
a achieved by projecting an observation’s feature space into a terrain type
space and then further projects the terrain type space into a robot
navigational behavior space.

Formally, we denote the robot’s navigational behaviors as
$\mathbf{Y}=[\mathbf{y}_{1},\dots,\mathbf{y}_{n}]\in\mathbb{R}^{b\times n}$,
where $\mathbf{y}_{i}\in\mathbb{R}^{b}$ represents the navigational behaviors
that the robot is expected to execute while traversing over terrain instance
$\mathbf{X}_{i}$, and $b$ denotes the number of distinctive robot controls
(e.g., linear and angular velocities). The expected behaviors can be obtained
through demonstrations recorded from human experts when the experts control
ground robots to traverse various terrains. Similar to terrain classification,
navigational behaviors are learned using a history of observations from the
pas $c$ time steps, which allows our formulation to implicitly consider the
dynamics of a ground robot. Then, the problem of terrain-aware navigation
adaptation through joint terrain recognition and behavior generation can be
formulated as:

$\displaystyle\min_{\mathbfcal{W},\mathbfcal{V}}$
$\displaystyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})+\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})$
s.t. $\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}^{\top}=\mathbfcal{I}$ (2)
$\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}=\mathbfcal{W}^{\top}\otimes(\mathbfcal{W}^{\top})^{\top}$

where $\mathbfcal{V}\in\mathbb{R}^{b\times d\times c}$ is a weight tensor that
indicates the importance of the input features $\mathbfcal{X}$ to generate
navigational behaviors $\mathbf{Y}$, and each element $v_{j}^{i(k)}$
represents the weight of using $j$-th terrain feature from $k$-th past time
step towards generating the $i$-th navigational behavior. We further adopt
$\mathbf{V}^{i}$, $\mathbf{V}_{j}$ and $\mathbf{V}^{(k)}$ to denote the slices
of matrices obtained by unstacking $\mathbfcal{V}$ along its height, width and
depth, respectively.

The second term in the objective function in Eq. (3.1) is a loss that encodes
the difference between the learning model and the demonstrated robot’s
navigational behaviors. This loss aims to learn a projection from a history of
observations to terrain types, and then to use the terrain types to generate
the robot’s behaviors, thus enabling terrain awareness for our navigational
behavior generation method.

Different terrain features capture different characteristics of unstructured
terrains, e.g., color, texture, and slope. These features typically have
different contributions towards terrain recognition and behavior generation.
Similarly, features extracted in specific time steps can be more important
than others. Thus, it is essential to identify the most discriminative
features when multiple types of features from a history of time steps are
used. To achieve this capability, we design a new regularization term called
the behavior norm, which is mathematically defined as:

$\|\mathbfcal{V}\otimes\mathbfcal{W}\|_{B}=\sum_{j=1}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}\mathbf{W}_{j}^{(k)}\|_{F}$
(3)

where $\mathbf{V}_{i}^{(k)}$ and $\mathbf{W}_{i}^{(k)}$ denote the slices of
matrices of $\mathbfcal{V}$ and $\mathbfcal{W}$ obtained at the $k$-th past
time step for the $i$-th type of features, respectively. This norm enforces
sparsity between feature modalities obtained over the past $c$ time steps in
order to identify the most discriminative features and time steps during
training.

Finally, the problem of terrain-aware behavior generation for terrain
adaptation is formulated as:

$\displaystyle\min_{\mathbfcal{W},\mathbfcal{V}}$
$\displaystyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})+\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})$
$\displaystyle+\lambda_{1}\|\mathbfcal{V}\otimes\mathbfcal{W}\|_{B}$ s.t.
$\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}^{\top}=\mathbfcal{I}$ (4)
$\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}=\mathbfcal{W}^{\top}\otimes(\mathbfcal{W}^{\top})^{\top}$

where $\lambda_{1}$ is a trade-off hyperparamter to control the amount of
regularization. The problem formulation in Eq. (3.1) allows for terrain-aware
navigational behavior generation. However, ground robots operating in an
unstructured environment over a long-period of time often experience changes
in their own functionalities, such as damages (e.g., failed motors and flat
tires), natural wear and tear (e.g., reduced tyre traction), and changed robot
configurations (e.g., varying payload). These negative effects, also called
_setbacks_ reduce the effectiveness of robot navigation and cause the robot’s
actual navigational behaviors not to match the planned or expected behaviors.

###  3.2 Self-Reflection for Consistent Navigational Behavior Generation

The second main novelty of this paper is to propose the idea of self-
reflection for a ground robot to generate consistent navigational behaviors by
adapting to changes in the robot’s functionalities. In psychology, self-
reflection is considered as the humans’ capability of modulating our behaviors
by being aware of ourselves (Marcovitch et al. (2008); Smith (2002); Ardelt
and Grunwald (2018)). We use the term self-reflection in robotics to refer to
the robot’s capability of adopting self-awareness (e.g., knowing it has slower
speeds than expected) to modulate the terrain-aware behavior generation.
Through self-reflection, we enable a ground robot to generate terrain-aware
navigational behaviors that are also conditioned on its self-awareness. The
robot can be self-aware in the way that it continuously monitors the
difference between the actual and expected navigational behaviors, where this
difference is caused by setbacks or negative effects. Then, our method adapts
the parameters of terrain-aware behavior generation according to robot’s self-
awareness, and computes an offset control to generate consistent navigational
behaviors without explicitly modeling the robot setbacks.

Mathematically, we denote the robot’s actual navigational behaviors as
$\mathbf{A}=[\mathbf{a}_{1},\dots,\mathbf{a}_{n}]\in\mathbb{R}^{b\times n}$,
where $\mathbf{a}_{i}\in\mathbb{R}^{b}$ is the actual navigational behaviors
corresponding to the $i$-th instance $\mathbf{X}_{i}$. The actual behaviors
can be estimated when the ground robot navigates over unstructured terrains
using pose estimation methods such as SLAM or visual odometry (Shan and Englot
(2018); Shan et al. (2016)). Because of setbacks, the robot’s actual behaviors
usually do not match its expected or planned navigational behaviors. We
compute the difference in navigational behaviors over the past $c$ time steps
as
$\mathbf{E}=[({\mathbf{a}}^{(t)}-\mathbf{y}^{(t)}),\dots,(\mathbf{a}^{(t-c)}-\mathbf{y}^{(t-c)})]\in\mathbb{R}^{b\times
c}$, and further denote the behavior differences for all instances
$\mathbfcal{X}$ as a tensor
$\mathbfcal{E}=[\mathbf{E}_{1},\dots,\mathbf{E}_{n}]\in\mathbb{R}^{b\times
n\times c}$.

Then, we introduce a novel loss function to encode self-reflection for
achieving consistent terrain-aware navigational behavior generation as
follows:

$\mathcal{L}_{lc}(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))$
(5)

where
$\mathbfcal{U}=[\mathbf{U}^{(1)},\dots,\mathbf{U}^{(c)}]\in\mathbb{R}^{b\times
b\times c}$ is the weight tensor with $\mathbf{U}^{(k)}\in\mathbb{R}^{b\times
c}$ indicating the importance of the $k$-th past behavior difference
${\mathbf{a}}^{(k)}-\mathbf{y}^{(k)}$ to generate offset behaviors
$\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}$. Because the loss
function in Eq. (5) encodes the error between the behavior differences and the
generated offset behaviors, minimizing this loss function is equivalent to
minimizing the behavior differences using the generated offset behaviors, or
maximizing the consistency of the actual and expected behaviors. By monitoring
the differences over the past $c$ time steps to improve the navigational
behavior at the current time step, this loss function allows our approach to
be self-reflective and act as a closed-loop controller for consistent behavior
generation while being terrain aware.

During robot navigation, because of inertia, historical data from the past
time steps often contributes differently towards generating offset behaviors.
For example, a heavier payload increases the inertia of the robot; thus a
longer time history needs to be considered for generating offset behaviors.
Thus, we introduce an additional regularization term to learn the most
informative behavior differences in the past time steps for efficient self-
reflection, which is defined as:

$\|\mathbfcal{U}\|_{R}=\sum_{k=1}^{c}\|\mathbf{U}^{(k)}\|_{F}$ (6)

where $\mathbf{U}^{(k)}$ denotes the slice of matrices obtained from
$\mathbfcal{U}$ at the $k$-th past time step.

Finally, integrating all components together, we formulate the problem of
self-reflective terrain-aware robot adaptation for generating consistent robot
navigational behaviors as a regularized optimization problem in a unified
mathematical framework:

$\displaystyle\min_{\mathbfcal{W},\mathbfcal{V},\mathbfcal{U}}$
$\displaystyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})+\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})$
$\displaystyle+\mathcal{L}_{lc}(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))$
$\displaystyle+\lambda_{1}\|\mathbfcal{V}\otimes\mathbfcal{W}\|_{B}+\lambda_{2}\|\mathbfcal{U}\|_{R}$
s.t. $\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}^{\top}=\mathbfcal{I}$ (7)
$\displaystyle\mathbfcal{W}\otimes\mathbfcal{W}=\mathbfcal{W}^{\top}\otimes(\mathbfcal{W}^{\top})^{\top}$

where $\lambda_{2}$, similar to $\lambda_{1}$, is a hyperparameter to control
the trade-off between the loss functions and regularization terms. An
illustration of our complete approach for self-reflective terrain-aware robot
adaptation is presented in Figure 2.

After computing the optimal values of the weight tensors $\mathbfcal{W}$,
$\mathbfcal{U}$ and $\mathbfcal{V}$ according to Algorithm 1 in the training
phase, a robot can apply our self-reflective terrain-aware adaptation method
to generate consistent navigational behaviors during execution. At each time
step $t$ in the execution phase, the robot computes multi-model features
$\mathbf{X}_{t}$ from observations obtained by its onboard sensors over the
past $c$ time steps. The robot also estimates the corresponding actual
behaviors using pose estimation techniques (such as SLAM or visual odometry
(Shan and Englot (2018); Shan et al. (2016))) and computes the matrix of the
behavior differences $\mathbf{E}_{t}$. Then our approach can be used by the
robot to generate self-reflective terrain-aware navigational behaviors by:

$\displaystyle\mathbf{y}=\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbf{X}_{t}+\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbf{E}_{t}$
(8)

The first term in Eq. (8) generates the navigational behaviors that are aware
of terrain types, which allows a ground robot to adapt its navigational
behaviors to unstructured terrains. The second term in Eq. (8) provides offset
controls based on the monitoring of the differences between the expected and
actual behaviors in order to compensate for the setbacks and improve
consistency in navigational behaviors.

1

Input : $\mathbfcal{X}\in\mathbb{R}^{d\times n\times c}$,
$\mathbf{Y}\in\mathbb{R}^{b\times n}$, $\mathbf{A}\in\mathbb{R}^{b\times n}$,
and $\mathbfcal{E}\in\mathbb{R}^{b\times n\times c}$

2

Output : The weight tensors $\mathbfcal{W}\in\mathbb{R}^{l\times d\times c}$,
$\mathbfcal{V}\in\mathbb{R}^{b\times d\times c}$ and
$\mathbfcal{U}\in\mathbb{R}^{b\times b\times c}$

3

4Initialize $\mathbfcal{W}\in\mathbb{R}^{l\times d\times c}$,
$\mathbfcal{V}\in\mathbb{R}^{b\times d\times c}$ and
$\mathbfcal{U}\in\mathbb{R}^{b\times b\times c}$;

5while  _not converge_ do

6 Calculate the block diagonal matrix $\mathbf{Q}^{B}$ with $j$-th column and
$k$-th row element given as
$\frac{\mathbf{I}_{1}}{\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}\mathbf{W}_{j}^{(k)}\|_{F}}$;

7 Calculate the block diagonal matrix $\mathbf{Q}^{l}$ with $i$-th diagonal
block given as
$\frac{\mathbf{I}^{l}}{\|\mathbf{W}^{i}(\mathbf{W}^{i})^{\top}-\mathbf{I}^{l}\|_{F}}$;

8 Calculate the block diagonal matrix $\mathbf{Q}_{d}$ with $j$-th diagonal
block given as
$\frac{\mathbf{I}_{d}}{\|\mathbf{W}_{j}(\mathbf{W}_{j})^{\top}-\mathbf{I}_{d}\|_{F}}$;

9 Calculate the block diagonal matrix $\mathbf{Q}^{(c)}$ with $k$-th diagonal
block given as
$\frac{\mathbf{I}^{(c)}}{\|\mathbf{W}^{(k)}(\mathbf{W}^{(k)})^{\top}-\mathbf{I}^{c}\|_{F}}$;

10 Compute each slice of matrices $\mathbf{W}^{i}$ from Eq. (4);

11 Calculate the block diagonal matrix $\mathbf{O}^{B}$ with $j$-th column and
$k$-th row element given as
$\frac{\mathbf{I}_{1}}{\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}\mathbf{W}_{j}^{(k)}\|_{F}}$;

12 Compute the matrix $\mathbf{V}^{i}$ from Eq. (12);

13 Calculate the block diagonal matrix $\mathbf{P}^{(R)}$ with $k$-th diagonal
block given as
$\frac{\mathbf{I}_{2}}{\sum_{k=t}^{t-c}\|\mathbf{U}^{(k)}\|_{F}}$;

14 Compute the matrix $\mathbf{U}^{i}$ from Eq. (13);

15return:  $\mathbfcal{W}\in\mathbb{R}^{l\times d\times c}$,
$\mathbfcal{V}\in\mathbb{R}^{b\times d\times c}$ and
$\mathbfcal{U}\in\mathbb{R}^{b\times b\times c}$

Algorithm 1 The implemented algorithm to solve the formulated constrained
regularized optimization problem in Eq. (3.2).

##  4 Optimization Algorithm

In this section, we implement a new iterative algorithm, as shown in in
Algorithm 1, to compute the optimal solution to the formulated constrained
regularized optimization problem in Eq. (3.2). The optimization problem is
challenging to solve because of the constrains added to the objective function
and the two non-smooth structured regularization terms.

To solve the constrained optimization problem in Eq. (3.2), we first convert
the constrains into a matrix form as:

$\min_{\mathbfcal{W},\mathbfcal{V},\mathbfcal{U}}$
$\displaystyle\scriptstyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})+\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})$
$\displaystyle\scriptstyle+\mathcal{L}_{lc}(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))+\lambda_{1}\|\mathbfcal{V}\otimes\mathbfcal{W}\|_{B}+\lambda_{2}\|\mathbfcal{U}\|_{R}$
s.t
$\displaystyle\scriptstyle\mathbf{W}^{i}(\mathbf{W}^{i})^{\top}=\mathbf{I}^{l};\forall
i=1,\dots,l$ (9)
$\displaystyle\scriptstyle\mathbf{W}_{j}(\mathbf{W}_{j})^{\top}=\mathbf{I}_{d};\forall
j=1,\dots,d$
$\displaystyle\scriptstyle\mathbf{W}^{(k)}(\mathbf{W}^{(k)})^{\top}=\mathbf{I}^{(c)};\forall
k=1,\dots,c$

where $\mathbf{I}^{l}$, $\mathbf{I}_{d}$ and $\mathbf{I}^{(c)}$ are identity
matrices of size $l,d,c$. In Eq. (4), the constrains are written using a
matrix form, This form still enforces orthogonality, the same as the tensor
form in Eq. (3.2), as the matrix-form constraints make each column in each
slice of matrices $\mathbf{W}^{i}$, $\mathbf{W}_{j}$ and $\mathbf{W}^{(k)}$ to
be full rank. Thus, each slice of matrix in tensor $\mathbfcal{W}$ is also
full rank and $\mathbfcal{W}$ is orthogonal.

Then we can rewrite the objective function in Eq. (4) using Lagrangian
multipliers $\lambda^{l},\lambda_{d}$ and $\lambda^{(c)}$ as:

$\min_{\mathbfcal{W},\mathbfcal{V},\mathbfcal{U}}$
$\displaystyle\scriptstyle\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})\scriptstyle+\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})\scriptstyle+\scriptstyle\mathcal{L}_{lc}(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))$
$\displaystyle\scriptstyle+$
$\displaystyle\scriptstyle\lambda_{1}\|\mathbfcal{V}\otimes\mathbfcal{W}\|_{B}+\lambda_{2}\|\mathbfcal{U}\|_{R}+\Sigma_{i=1}^{l}\lambda^{l}\|\mathbf{W}^{i}(\mathbf{W}^{i})^{\top}-\mathbf{I}^{l}\|_{F}$
(10) $\displaystyle\scriptstyle+$
$\displaystyle\scriptstyle\Sigma_{j=1}^{d}\lambda_{d}\|\mathbf{W}_{j}(\mathbf{W}_{j})^{\top}-\mathbf{I}_{d}\|_{F}+\Sigma_{k=1}^{c}\lambda^{(c)}\|\mathbf{W}^{(k)}(\mathbf{W}^{(k)})^{\top}-\mathbf{I}^{(c)}\|_{F}$

Each of the Lagrangian multipliers is designed to have a high value (i.e.,
$\lambda^{l},\lambda_{d},\lambda^{(c)}>\\!\\!>0$), such that even a small
variation in orthogonality in each slices of matrix from weight tensor
$\mathbfcal{W}$ can result in a high cost. To compute the optimal weight
tensor $\mathbfcal{W}$, we minimize Eq. (4) with respect to $\mathbf{W}^{i}$,
$i=1,\dots,l$, resulting in:

$\displaystyle\scriptstyle\tanh{(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})}((\mathbf{X}^{i})^{\top}\mathbf{W}^{i})\tanh{(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})}((\mathbf{X}^{i}\mathbf{V}^{i})^{\top}\mathbf{W}^{i})$
$\displaystyle\scriptstyle+\tanh{(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))}((\mathbf{E}^{i}\mathbf{V}^{i})^{\top}\mathbf{W}^{i})$
(11)
$\displaystyle\scriptstyle+(\lambda_{1}\mathbf{Q}^{B}+\Sigma_{i=1}^{l}\lambda^{l}\mathbf{Q}^{l}+\Sigma_{j=1}^{d}\lambda_{d}\mathbf{Q}_{d}+\Sigma_{k=1}^{c}\lambda^{(c)}\mathbf{Q}^{(c)})\mathbf{W}^{i}=0$

where $\mathbf{Q}^{B},\mathbf{Q}^{l},\mathbf{Q}_{d}$ and $\mathbf{Q}^{(c)}$
denote block diagonal matrices that are dependent on the weight tensor
$\mathbfcal{W}$. Mathematically, we express each element in the $j$-th column
and $k$-th row in $\mathbf{Q}^{B}$ as
$\frac{1}{\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}\mathbf{W}_{j}^{(k)}\|_{F}}$.
Each of the $i$-th block in $\mathbf{Q}^{l}$ is given by
$\frac{\mathbf{I}^{i}}{\|\mathbf{W}^{i}(\mathbf{W}^{i})^{\top}-\mathbf{I}^{l}\|_{F}}$.
The $j$-th diagonal block in $\mathbf{Q}_{d}$ is given as
$\frac{\mathbf{I}_{d}}{\|\mathbf{W}_{j}(\mathbf{W}_{j})^{\top}-\mathbf{I}_{d}\|_{F}}$
and each block diagonal element in $\mathbf{Q}^{(c)}$ is given by
$\frac{\mathbf{I}^{(c)}}{\|\mathbf{W}^{(k)}(\mathbf{W}^{(k)})^{\top}-\mathbf{I}^{(c)}\|_{F}}$.
Because each slice of matrix $\mathbf{W}^{i}$ and the block diagonal matrices
$\mathbf{Q}^{B},\mathbf{Q}^{l},\mathbf{Q}_{d}$, $\mathbf{Q}^{(c)}$ are
interdependent, we need an iterative algorithm to compute them.

We use the optimal slices of weight matrices $\mathbf{W}^{i}$ to solve weight
tensors $\mathbfcal{V}$ and $\mathbfcal{U}$. Accordingly, we differentiate Eq.
(4) with respect to $\mathbf{V}^{i}$ and $\mathbf{U}^{i}$, $i=1,\dots,b$,
results in:

$\displaystyle\scriptstyle\tanh{(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})}((\mathbf{W}^{i}\mathbf{X}^{i})^{\top}\mathbf{V}^{i}-\mathbf{WX}^{i}\mathbf{y}^{i})+\lambda_{1}\mathbf{O}^{B}=0$
(12)

and

$\displaystyle\scriptstyle\tanh{(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))}((\mathbf{W}^{i}\mathbf{E}^{i})^{\top}\mathbf{U}^{i}-\mathbf{E}^{i})+\lambda_{2}\mathbf{P}^{(R)}=0$
(13)

where $\mathbf{O}^{(B)}$ and $\mathbf{P}^{(R)}$ represent block diagonal
matrices that are dependent on $\mathbfcal{V}$ and $\mathbfcal{U}$
respectively. Mathematically, we calculate each element in the $j$-th column
and $k$-th row in $\mathbf{O}^{B}$ as
$\frac{1}{\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}\mathbf{W}_{j}^{(k)}\|_{F}}$.
The $k$-th block diagnoal element in $\mathbf{P}^{(R)}$ is calculated by
$\frac{\mathbf{I}_{(k)}}{\sum_{k=1}^{c}\|\mathbf{U}^{(k)}\|_{F}}$. The optimal
values of $\mathbf{V}^{i}$ and $\mathbf{U}^{i}$ are employed to update
$\mathbf{W}^{i}$ in the next iteration. Similarly, $\mathbf{U}^{i}$ and
$\mathbf{P}$ are interdependent, and thus an iterative algorithm is needed to
update their values.

In the following, we prove that Algorithm 1 decreases the value of the
objective function in Eq. (3.2) with each iteration and converges to the
global optimal solution. But first, we present a lemma from Nie et al. (2010):

######  Lemma 1.

Given any two matrices $\mathbf{A}$ and $\mathbf{B}$, the following inequality
relation holds:

$\|\mathbf{B}\|_{F}-\frac{\|\mathbf{B}\|_{F}^{2}}{2\|\mathbf{A}\|_{F}}\leq\|\mathbf{A}\|_{F}-\frac{\|\mathbf{A}\|_{F}^{2}}{2\|\mathbf{A}\|_{F}}$
(14)

######  Theorem 1.

Algorithm 1 converges to the global optimal solution of the constrained
regularized optimization problem in Eq. (3.2).

###### Proof.

According to Step 7 in Algorithm 1, the value of $\mathbf{W}^{i}(s+1)$ is
computed from $\mathbfcal{W}(s)$ in the $s$-th iteration by:

$\displaystyle\scriptstyle\mathbf{W}^{i}(s+1)$
$\displaystyle\scriptstyle=\mathcal{L}_{lc}(\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Z})+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i\top})\mathbf{Q}^{B}(s+1)(\mathbf{V}^{i}\mathbf{W}^{i})$
$\displaystyle\scriptstyle+\lambda^{l}Tr$
$\displaystyle\scriptstyle(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l^{\top}})\mathbf{Q}^{l}(s+1)(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l})$
(15) $\displaystyle\scriptstyle+\lambda_{d}Tr$
$\displaystyle\scriptstyle(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}_{d})^{\top}\mathbf{Q}_{d}(s+1)(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}^{d})$
$\displaystyle\scriptstyle+\lambda^{(c)}Tr$
$\displaystyle\scriptstyle(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)\top})\mathbf{Q}^{(c)}(s+1)(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)})$

where the operator $Tr$ denotes the trace of a matrix. From Step 9 in
Algorithm 1, we obtain:

$\displaystyle\scriptstyle\mathbf{V}^{i}(s+1)\scriptstyle=\mathcal{L}_{lc}(\mathbfcal{V}\otimes\mathbfcal{W}\otimes\mathbfcal{X}-\mathbf{Y})\scriptstyle+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i})^{\top}\mathbf{O}^{B}(s+1)(\mathbf{V}^{i}\mathbf{W}^{i})$
(16)

From Step 11 in Algorithm 1 we also obtain:

$\displaystyle\scriptstyle\mathbf{U}^{i}(s+1)=\mathcal{L}_{lc}(\mathbfcal{U}\otimes\mathbfcal{W}\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))+\lambda_{2}Tr\mathbf{U}^{i}\mathbf{P}(s+1)\mathbf{U}^{i}$
(17)

Then, we derive that:

$\displaystyle\scriptstyle\mathbfcal{F}_{1}(s+1)+\mathbfcal{F}_{2}(s+1)+\mathbfcal{F}_{3}(s+1)+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i})^{\top}\mathbf{Q}^{B}(s+1)(\mathbf{V}^{i}\mathbf{W}^{i})$
$\displaystyle\scriptstyle+\lambda^{l}Tr(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l})^{\top}\mathbf{Q}^{l}(s+1)(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l})$
$\displaystyle\scriptstyle+\lambda_{d}Tr(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}_{d})^{\top}\mathbf{Q}_{d}(s+1)(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}^{d})$
$\displaystyle\scriptstyle+\lambda^{(c)}Tr(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)\top}\mathbf{Q}^{(c)}(s+1)(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)})$
$\displaystyle\scriptstyle+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i})^{\top}\mathbf{O}^{B}(s+1)(\mathbf{V}^{i}\mathbf{W}^{i})+\lambda_{2}Tr\mathbf{U}^{i}\mathbf{P}(s+1)\mathbf{U}^{i}$
$\displaystyle\scriptstyle\leq\mathbfcal{F}_{1}(s)+\mathbfcal{F}_{2}(s)+\mathbfcal{F}_{3}(s)+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i})^{\top}\mathbf{Q}^{B}(s)(\mathbf{V}^{i}\mathbf{W}^{i})$
(18)
$\displaystyle\scriptstyle+\lambda^{l}Tr(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l})^{\top}\mathbf{Q}^{l}(s)(\mathbf{W}^{i}\mathbf{W}^{i\top}-\mathbf{I}^{l})$
$\displaystyle\scriptstyle+\lambda_{d}Tr(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}_{d})^{\top}\mathbf{Q}_{d}(s)(\mathbf{W}_{j}\mathbf{W}_{j}^{\top}-\mathbf{I}^{d})$
$\displaystyle\scriptstyle+\lambda^{(c)}Tr(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)})^{\top}\mathbf{Q}^{(c)}(s)(\mathbf{W}^{(k)}\mathbf{W}^{(k)\top}-\mathbf{I}^{(c)})$
$\displaystyle\scriptstyle+\lambda_{1}Tr(\mathbf{V}^{i}\mathbf{W}^{i})^{\top}\mathbf{O}^{B}(s)(\mathbf{V}^{i}\mathbf{W}^{i})+\lambda_{2}Tr\mathbf{U}^{i}\mathbf{P}(s)\mathbf{U}^{i}$

where:

$\displaystyle\mathbfcal{F}_{1}(s)=\mathcal{L}_{lc}(\mathbfcal{W}(s)\otimes\mathbfcal{X}-\mathbf{Z})$
$\displaystyle\mathbfcal{F}_{2}(s)=\mathcal{L}_{lc}(\mathbfcal{V}(s)\otimes\mathbfcal{W}(s)\otimes\mathbfcal{X}-\mathbf{Y})$
$\displaystyle\mathbfcal{F}_{3}(s)=\mathcal{L}_{lc}(\mathbfcal{U}(s)\otimes\mathbfcal{W}(s)\otimes\mathbfcal{E}-(\mathbf{A}-\mathbf{Y}))$

After substituting $\mathbf{Q}^{B},\mathbf{Q}^{l},\mathbf{Q}_{d}$,
$\mathbf{Q}^{(c)}$, $\mathbf{O}^{B}$ and $\mathbf{P}^{R}$ in Eq. (4), we
obtain:

$\displaystyle\scriptstyle\mathbfcal{F}_{1}(s+1)+\mathbfcal{F}_{2}(s+1)+\mathbfcal{F}_{3}(s+1)+\lambda_{2}\sum_{k=1}^{c}\frac{\|\mathbf{U}^{(k)}(s+1)\|_{F}^{2}}{2\|\mathbf{U}^{(k)}(s)\|_{F}}$
$\displaystyle\scriptstyle+\lambda_{1}\sum_{i=j}^{m}\sum_{k=1}^{c}\frac{\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s+1)\|_{F}^{2}}{2\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}}$
$\displaystyle\scriptstyle+\lambda^{l}\sum_{i=1}^{l}\frac{\|\mathbf{W}^{i}(s+1)(\mathbf{W}(s+1)^{i})^{\top}-\mathbf{I}^{l}\|_{F}^{2}}{\|2\mathbf{W}^{i}(s)(\mathbf{W}^{i}(s))^{\top}-\mathbf{I}^{l}\|_{F}}$
$\displaystyle\scriptstyle+\lambda_{d}\sum_{j=1}^{d}\frac{\|\mathbf{W}_{j}(s+1)(\mathbf{W}_{j}(s+1))^{\top}-\mathbf{I}_{d}\|_{F}^{2}}{\|2\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}}$
$\displaystyle\scriptstyle+\lambda^{(c)}\sum_{k=1}^{c}\frac{\|\mathbf{W}^{(k)}(s+1)(\mathbf{W}^{(k)}(s+1))^{\top}-\mathbf{I}^{(c)}\|_{F}^{2}}{\|2\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}}$
$\displaystyle\scriptstyle\leq\mathbfcal{F}_{1}(s)+\mathbfcal{F}_{2}(s)+\mathbfcal{F}_{3}(s)+\lambda_{2}\sum_{k=1}^{c}\frac{\|\mathbf{U}^{(k)}(s)\|_{F}^{2}}{2\|\mathbf{U}^{(k)}(s)\|_{F}}$
(19)
$\displaystyle\scriptstyle+\lambda_{1}\sum_{i=j}^{m}\sum_{k=1}^{c}\frac{\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}^{2}}{2\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}}$
$\displaystyle\scriptstyle+\lambda^{l}\sum_{i=1}^{l}\frac{\|\mathbf{W}^{i}(s)(\mathbf{W}(s)^{i})^{\top}-\mathbf{I}^{l}\|_{F}^{2}}{\|2\mathbf{W}^{i}(s)(\mathbf{W}^{i}(s))^{\top}-\mathbf{I}^{l}\|_{F}}$
$\displaystyle\scriptstyle+\lambda_{d}\sum_{j=1}^{d}\frac{\|\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}^{2}}{\|2\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}}$
$\displaystyle\scriptstyle+\lambda^{(c)}\sum_{k=1}^{c}\frac{\|\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}^{2}}{\|2\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}}$

Using Lemma 1, we obtain the following inequalities in Eqs. (4-4). From
$\mathbf{Q}^{B}$ and $\mathbf{O}^{B}$, we obtain:

$\displaystyle\scriptstyle\sum_{i=1}^{m}\sum_{k=1}^{c}\Big{(}\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s+1)\|_{F}-\frac{\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s+1)\|_{F}^{2}}{2\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}}\Big{)}$
$\displaystyle\scriptstyle\leq\sum_{i=1}^{m}\sum_{k=1}^{c}\Big{(}\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}-\frac{\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}^{2}}{2\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}}\Big{)}$
(20)

From $\mathbf{Q}^{l}$, we obtain:

$\displaystyle\scriptstyle\sum_{i=1}^{m}\Big{(}\|\mathbf{W}^{i}(s+1)\mathbf{W}^{i\top}(s+1)-\mathbf{I}^{l}\|_{F}-\frac{\|\mathbf{W}^{i}(s+1)\mathbf{W}^{i\top}(s+1)-\mathbf{I}^{l}\|_{F}^{2}}{2\|\mathbf{W}^{i}(s)\mathbf{W}^{i\top}(s)-\mathbf{I}^{l}\|_{F}}\Big{)}$
$\displaystyle\scriptstyle\leq\sum_{i=1}^{m}\Big{(}\|\mathbf{W}^{i}(s)\mathbf{W}^{i\top}(s)-\mathbf{I}^{l}\|_{F}-\frac{\|\mathbf{W}^{i}(s)\mathbf{W}^{i\top}(s)-\mathbf{I}^{l}\|_{F}^{2}}{2\|\mathbf{W}^{i}(s)(\mathbf{W}^{i\top}(s))-\mathbf{I}^{l}\|_{F}}\Big{)}$
(21)

From $\mathbf{Q}_{d}$, we obtain:

$\displaystyle\scriptstyle\sum_{j=1}^{d}\Big{(}\|\mathbf{W}_{j}(s+1)(\mathbf{W}_{j}(s+1))^{\top}-\mathbf{I}_{d}\|_{F}-\frac{\|\mathbf{W}_{j}(s+1)(\mathbf{W}_{j}(s+1))^{\top}-\mathbf{I}_{d}\|_{F}^{2}}{2\|\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}}\Big{)}$
$\displaystyle\scriptstyle\leq\sum_{j=1}^{d}\Big{(}\|\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}-\frac{\|\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}^{2}}{2\|\mathbf{W}_{j}(s)(\mathbf{W}_{j}(s))^{\top}-\mathbf{I}_{d}\|_{F}}\Big{)}$
(22)

From $\mathbf{Q}^{(c)}$, we obtain:

$\displaystyle\scriptstyle\sum_{k=1}^{c}\Big{(}\|\mathbf{W}^{(k)}(s+1)(\mathbf{W}^{(k)}(s+1))^{\top}-\mathbf{I}^{(c)}\|_{F}-$
(23)
$\displaystyle\scriptstyle\quad\quad\quad\quad\quad\frac{\|\mathbf{W}^{(k)}(s+1)(\mathbf{W}^{(k)}(s+1))^{\top}-\mathbf{I}^{(c)}\|_{F}^{2}}{2\|\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}}\Big{)}\leq$
$\displaystyle\scriptstyle\sum_{k=1}^{c}\Big{(}\|\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}-\frac{\|\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}^{2}}{2\|\mathbf{W}^{(k)}(s)(\mathbf{W}^{(k)}(s))^{\top}-\mathbf{I}^{(c)}\|_{F}}\Big{)}$

From $\mathbf{P}^{(R)}$, we obtain:

$\displaystyle\scriptstyle\sum_{k=1}^{c}\Big{(}\|\mathbf{U}^{(k)}(s+1)\|_{F}-\frac{\|\mathbf{U}^{(k)}(s+1)\|_{F}^{2}}{2\mathbf{U}^{(k)}(s)\|_{F}}\Big{)}$
$\displaystyle\scriptstyle\leq\sum_{k=1}^{c}\Big{(}\|\mathbf{U}^{(k)}(s)\|_{F}-\frac{\|\mathbf{U}^{(k)}(s)\|_{F}^{2}}{2\mathbf{U}^{(k)}(s)\|_{F}}\Big{)}$
(24)

Then, adding Eqs. (4-4) to Eq. (4) on both sides results in the following
inequality:

$\displaystyle\scriptstyle\mathbfcal{F}_{1}(s+1)+\mathbfcal{F}_{2}(s+1)+\mathbfcal{F}_{3}(s+1)+\lambda_{2}\sum_{k=1}^{c}\|\mathbf{U}^{(k)}(s+1)\|_{F}$
$\displaystyle\scriptstyle+\lambda_{1}\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s+1)\|_{F}$
$\displaystyle\scriptstyle+\lambda^{l}\sum_{i=1}^{l}\|\mathbf{W}^{i}(s+1)\mathbf{W}^{i\top}(s+1)-\mathbf{I}^{l}\|_{F}$
$\displaystyle\scriptstyle+\lambda_{d}\sum_{j=1}^{d}\|\mathbf{W}_{j}(s+1)\mathbf{W}_{j}^{\top}(s+1)-\mathbf{I}_{d}\|_{F}$
$\displaystyle\scriptstyle+\lambda^{(c)}\sum_{k=1}^{c}\|\mathbf{W}^{(k)}(s+1)\mathbf{W}^{(k)\top}(s+1)-\mathbf{I}^{(c)}\|_{F}$
$\displaystyle\scriptstyle\leq\mathbfcal{F}_{1}(s)+\mathbfcal{F}_{2}(s)+\mathbfcal{F}_{3}(s)+\lambda_{2}\sum_{k=1}^{c}\|\mathbf{U}^{(k)}(s)\|_{F}$
(25)
$\displaystyle\scriptstyle+\lambda_{1}\sum_{i=j}^{m}\sum_{k=1}^{c}\|\mathbf{V}_{j}^{(k)}(s)\mathbf{W}_{j}^{(k)}(s)\|_{F}$
$\displaystyle\scriptstyle+\lambda^{l}\sum_{i=1}^{l}\|\mathbf{W}^{i}(s)\mathbf{W}^{i\top}(s)-\mathbf{I}^{l}\|_{F}+\lambda_{d}\sum_{j=1}^{d}\|\mathbf{W}_{j}(s)\mathbf{W}_{j}^{\top}(s)-\mathbf{I}_{d}\|_{F}$
$\displaystyle\scriptstyle+\lambda^{(c)}\sum_{k=1}^{c}\|\mathbf{W}^{(k)}(s)\mathbf{W}^{(k)\top}(s)-\mathbf{I}^{(c)}\|_{F}$

This inequality in Eq. (4) proves that the objective value is decreased in
each iteration. The Log-Cosh losses in Eq. (4) are convex (Gangal et al.
(2007)) and the Frobenius norms in Eq. (4) are also convex (Negahban and
Wainwright (2012)). Thus, as a sum of convex functions, the objective function
in Eq. (4) is also convex. Therefore, Algorithm 1 is guaranteed to converge to
the global optimal solution to the formulated constrained regularized
optimization problem in Eq. (3.2). ∎

Time Complexity. As the formulated optimization problem in Eq. (3.2) is
convex, Algorithm 1 converges fast (e.g., within tens of iterations only). In
each iteration of our algorithm, computing Steps 3, 4, 5, 6, 8 and 10 is
trivial. Steps 7, 9 and 11 can be computed by solving a system of linear
equations with quadratic complexity.

##  5 Experiments

In this section, we first discuss on the experimental setup and implementation
details of our approach. Then we analyze the experimental results obtained by
our method and compare it with the previous state-of-the-art methods in
multiple ground navigation scenarios in real-world off-road environments.

Figure 3:  Multi-modal features used in our experimentation to characterize
robot’s traversing terrain.

###  5.1 Experimental Setups

We extensively evaluate our approach using a Clearpath Husky unmanned ground
vehicle robot to navigate over a variety of unstructured off-road terrains.
The robot is equipped with multiple exteroceptive and proprioceptive sensors.
Exteroceptive sensors include an Intel Realsense D435 color camera and an
Ouster OS1-64 LIDAR to observe the terrain. Proprioceptive sensors including
wheel encoders and Microstrain 3DM-GX5-25 Internal Measurement Unit (IMU) are
used to measure the robot’s internal states while navigating over these
terrains. All these terrain observations are obtained from the robot and are
linearly interpolated to get observations constantly at 15 Hz.

Figure 4: Various terrain tracks used for for individual types of terrains.
From left: concrete, grass, gravels, large-rocks and snow terrain.

For representing unstructured terrains, we implement multiple visual features
extracted from color images to depict the characteristics ot the unstructured
terrain. These features include Histogram of Oriented Gradients (HOG) (Dalal
and Triggs (2005)) to depict the shape of the terrain and Local Binary
Patterns (LBP) (Ahonen et al. (2006)) to detail the texture of the terrain. To
show the various terrain characteristics such as the slope, step height, and
terrain normals, we also compute an elevation map (Fankhauser et al. (2014))
of the terrain using LIDAR data to form a grid-wise robot centric elevation
map. Apart from these features, we also concatenate sensory readings from the
Inertial Measurement Unit (IMU) into a vector of features. The various
features used in our experiments are visualized in Figure 3.

In addition, the expected robot behaviors are recorded as demonstrations from
an expert human operator The actual robot navigational behaviors have been
recorded using SLAM techniques (Shan and Englot (2018)). We use a sequence of
five frames (i.e., $c=15$), and the hyperparameters $\lambda_{1}=1$ and
$\lambda_{2}=0.1$ for all of our experiments.

We compare our approach against three state-of-the-art approaches. These three
approaches can be broadly classified into two categories. The first category
is model-based approaches which includes the methods of Model Predictive Path
Integral (MPPI) control (Williams et al. (2016)) and Non-Linear Control
Optimization tool (NLOPT) (Johnson (2011)). The second category of methods
includes learning-based approach of terrain representation and apprenticeship
learning (TRAL) (Siva et al. (2019)).

All the aforementioned methods are quantitatively evaluated against our over
various terrains. We define the following evaluation metrics to illustrate the
observations:

  * •

_Failure Rate_ : This metric is defined as the number of times the robot fails
to complete the navigation task across a set of experimental trails. A failure
is when the robot gets stuck in the terrain or in an obstacle or flips over in
the terrain. Lower values of failure rate indicate better performance and is
preferred.

  * •

_Traversal Time_ : This metric is defined as the average time taken to
complete the navigational task in all the successive runs over the given
terrain. Generally, smaller values of traversal time indicate better
performance and may be preferred.

  * •

_Inconsistency_ : This metrics is defined as the error between the robot’s
expected navigational behavior and its actual behavior in terms of robot poses
(the linear and angular position). A lower value of inconsistency metric
indicate better performance and is preferred.

  * •

_Jerkiness_ : This metric is defined as the average sum of the acceleration
derivatives along all axes, with lower values indicating better performance.
Jerkiness indicates how smooth a robot can traverse over a terrain. This is a
useful metric for state estimation and SLAM methods that may assume smooth
robot motions.

Figure 5:  Off-road terrain circuit (left) consisting of multiple terrains of
grass, gravels, large rocks, mud and sand. These tracks also show a
combination of transitions between these individual terrains (right) and also
have a varying slope of $0^{\circ}$-$30^{\circ}$ throughout the track.

###  5.2 Terrain Adaptation for Ground Robot Navigation over Unstructured
Terrains

In this set of experiments, we evaluate our proposed approach against its
ability to generate terrain aware navigational behaviors. The ability to
generate terrain aware behaviors are required mainly when robots are i)
navigating over individual terrain tracks and ii) navigating over complex
terrains. Therefore, we test our approach along with comparison methods in
these two kinds of scenarios. More discussion on each of these experimental
scenarios are as follows:

Table 2: Quantitative results based on ten runs for scenarios when the robot traverses over individual types of unstructured terrain shown in Fig. 4. Successful runs (with no failures) are used to calculate the metrics of traversal time, inconsistency and jerkiness. Our approach is compared with the methods of MPPI (Williams et al. (2016)), NLOPT (Johnson (2011)) and TRAL (Siva et al. (2019)).  | Failure Rate (/10) | Traversal Time (s) | Inconsistency | Jerkiness (m/s3)
---|---|---|---|---
Terrain | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours
Concrete | 0 | 0 | 0 | 0 | 12.82 | 17.82 | 16.04 | 12.49 | 0.65 | 0.79 | 0.69 | 0.56 | 5.13 | 4.98 | 5.19 | 4.35
Grass | 0 | 0 | 0 | 0 | 13.45 | 18.02 | 16.39 | 13.32 | 0.61 | 0.90 | 0.88 | 0.62 | 7.82 | 8.52 | 8.32 | 6.98
Gravels | 0 | 0 | 0 | 0 | 13.96 | 16.39 | 15.32 | 14.84 | 1.21 | 1.67 | 1.72 | 1.03 | 16.11 | 16.66 | 16.43 | 15.09
L.Rock | 0 | 1 | 0 | 0 | 13.45 | 17.87 | 16.71 | 16.31 | 3.97 | 5.64 | 3.21 | 2.91 | 13.75 | 17.61 | 14.85 | 12.32
Snow | 0 | 0 | 0 | 0 | 14.22 | 18.09 | 17.49 | 14.08 | 7.18 | 9.93 | 8.12 | 6.64 | 5.12 | 5.76 | 5.49 | 5.42

####  5.2.1 Experimental Analysis on Individual Terrain Tracks

In this experimental scenario, we evaluate our proposed approach on various
individual types of terrains. That is each of the testing tracks used in this
scenario is made of one type of terrain. The length of the tracks are kept
approximately ten meters long. We use five different terrains: concrete,
grass, gravels, large-rocks and snow terrains, and are illustrated in Figure
4. Each of these five terrains has distinctive characteristics and are
frequently observed in real-world environments. Our approach is trained on
driving data collected while the robot is manually being controlled by an
expert human operator to traverse over each of the terrains. The learned model
is then deployed on the robot to generate navigational behaviors as the robot
autonomously navigates over the terrain. Specifically, our approach is used as
a local controller which operates under a local planner. Then the evaluation
metrics for each of the methods are computed across ten trials on each type of
individual terrain tracks.

The quantitative results achieved by our approach and its comparison with
other methods are provided in Table 2. We observe that nearly all the
approaches successfully navigate over all the individual types of terrain with
an exception of the NLOPT approach that fails in a single test run on large
rocks terrain. The presented traversal time is computed by averaging the
traversal time across all successful runs, i.e., it excludes the failed trials
captured by the failure rate metric. In terms of traversal time, our approach
achieves better traversal time in the terrains of concrete, grass and snow,
followed by the MPPI approach. In the terrains of gravels and large-rocks we
observe that our approach has a slower traversal time as compared to the MPPI
approach and performs the second best. Both the NLOPT and TRAL approach both
have a slower traversal time over all the terrains with the NLOPT approach
performing the slowest throughout all the terrains. Also in model-based
approaches of MPPI and NLOPT we do not observe the terrain aware nature of the
approach, that is there is no adaptation in navigational behaviors as the
traversal time remains nearly the same. On the other hand, learning-based
approaches do show variation in traversal over different terrains.
Specifically, our approach shows a faster traversal time in plain terrains of
concrete, grass and snow, and a slower traversal time on both the gravels and
large rocks terrain.

Table 2 also presents the inconsistency and the jerkiness metrics. We observe
that our approach achieves the best inconsistency throughout all the terrains
except the grass terrain, followed by both the MPPI and TRAL approach. The
MPPI approach has a best inconsistency value in grass terrain. The TRAL
approach obtains a better performance to MPPI approach in the large-rocks
terrains but otherwise has a poorer performance. The NLOPT approach obtains a
highest inconsistency metric throughout all the terrains and thus has the
least performance in terms of the inconsistency metric. In general, we observe
that learning based methods obtain a better performance over model-based
approaches. While comparing against the jerkiness metric, we observe that our
approach obtains the least value of jerkiness in all terrains but snow. The
MPPI approach obtains the best performance in snow terrain and trails our
approach in all the remaining terrains. Both the NLOPT and TRAL approach
obtain highest value of jerkiness in specific terrains and perform poorly as
compared to both MPPI and our approach. We observe that both the MPPI approach
and our approach perform best in the individual type of terrains across all
the metrics with our approach performing the best in most of the scenarios.

Table 3: Quantitative results based on ten runs for scenarios when the robot traverses over complex terrain tracks shown in Fig. 5. Successful runs (with no failures) are used to calculate the metrics of traversal time, inconsistency and jerkiness. Our approach is compared with classical control based approaches of MPPI (Williams et al. (2016)), NLOPT (Johnson (2011)) and TRAL (Siva et al. (2019)).  | Failure Rate (/10) | Traversal Time (s) | Inconsistency | Jerkiness (m/s3)
---|---|---|---|---
Terrain | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours
Mud-Sand | 0 | 0 | 0 | 0 | 26.61 | 31.11 | 41.08 | 36.17 | - | - | - | - | - | - | - | -
Sand-Grass | 0 | 0 | 0 | 0 | 23.69 | 31.78 | 36.57 | 28.79 | - | - | - | - | - | - | - | -
Grass-Mud | 0 | 0 | 0 | 0 |  18.83 | 20.69 | 37.35 | 26.74 |  - | - | - | - |  - | - | - | -
Mud-Rock | 0 | 0 | 0 | 0 |  19.31 | 21.86 | 39.88 | 33.32 |  - | - | - | - |  - | - | - | -
Rock-Sand | 0 | 0 | 0 | 0 | 20.44 | 22.67 | 36.08 | 36.59 | - | - | - | - | - | - | - | -
Sand-Gravel | 0 | 0 | 0 | 0 | 20.93 | 23.98 | 41.9 | 43.02 | - | - | - | - | - | - | - | -
Gravel-Grass | 0 | 0 | 0 | 0 | 22.38 | 26.04 | 39.17 | 29.07 | - | - | - | - | - | - | - | -
Grass-Rock | 0 | 0 | 0 | 0 | 19.81 | 26.55 | 36.15 | 35.82 | - | - | - | - | - | - | - | -
Rock-Gravel | 0 | 0 | 0 | 0 | 17.62 | 27.03 | 34.92 | 42.41 | - | - | - | - | - | - | - | -
Figure 6: Various terrain tracks used to evaluate the self-reflection capability of the robot. From left: mixed terrain I (MT-I), mixed terrain II (MT-II), forest, reduced-traction snow-grass (RT-Snow-Grass), reduced-traction mud-snow (RT-Snow-Mud), reduced-traction hill (RT-Hill), reduced-traction forest (RT-Snow-Forest), reduced-traction payload snow-grass (RT-P-Snow-Grass) and reduced-traction payload hill terrain (RT-P-Hill).  Table 4: Quantitative results based on ten runs for scenarios shown in Fig. 6 used to evaluate the self-reflection capability of the robot. Successful runs (with no failures) are used to calculate the metrics of traversal time, inconsistency and jerkiness. Our approach is compared with classical control based approaches of MPPI (Williams et al. (2016)), NLOPT (Johnson (2011)) and TRAL (Siva et al. (2019)). | Failure Rate (/10) | Traversal Time (s) | Inconsistency | Jerkiness (m/s3)
---|---|---|---|---
Terrain | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours | MPPI | NLOPT | TRAL | Ours
MT-I | 0 | 0 | 0 | 0 | 14.51 | 17.68 | 16.84 | 13.74 | 0.65 | 0.79 | 0.69 | 0.56 | 5.68 | 5.57 | 5.13 | 4.95
MT-II | 1 | 2 | 1 | 0 | 13.44 | 17.48 | 16.74 | 13.64 | 0.78 | 0.98 | 0.88 | 0.71 | 7.66 | 8.75 | 8.35 | 6.52
Forest | 2 | 5 | 3 | 1 | 14.67 | 13.67 | 15.29 | 15.96 | 1.21 | 1.67 | 1.72 | 1.11 | 16.25 | 16.62 | 16.55 | 16.78
RT-Snow-Grass | 3 | 4 | 4 | 2 | 13.24 | 17.66 | 16.18 | 13.72 | 4.61 | 5.90 | 5.88 | 4.11 | 9.12 | 9.52 | 9.32 | 8.98
RT-Snow-Mud | 5 | 7 | 4 | 2 | 13.28 | 17.84 | 16.10 | 12.03 |  3.97 | 5.64 | 3.21 | 3.01 | 13.14 | 17.15 | 14.12 | 12.35
RT-Hill | 3 | 6 | 3 | 2 | 13.07 | 15.85 | 16.42 | 13.91 | 14.98 | 15.23 | 14.91 | 13.07 | 13.75 | 17.61 | 14.85 | 12.32
RT-Snow-Forest | 0 | 0 | 0 | 0 | 13.55 | 16.63 | 14.39 | 14.34 | 7.18 | 9.93 | 8.12 | 6.62 | 6.72 | 6.88 | 6.92 | 6.47
RT-P-Snow-Grass | 4 | 7 | 5 | 2 | 13.96 | 16.39 | 15.32 | 13.84 | 9.12 | 9.96 | 9.57 | 8.54 | 16.11 | 16.66 | 16.43 | 16.09
RT-P-Hill | 5 | 8 | 6 | 4 | 14.59 | 15.87 | 14.38 | 15.77 | 11.18 | 15.93 | 14.12 | 10.62 | 13.12 | 17.56 | 14.86 | 12.51

####  5.2.2 Experimental Analysis on Complex Terrain Tracks

In the second experimental scenario, we evaluate our approach when robots
navigate over complex off-road unstructured environments. To replicate the
challenges of these real-world complex terrains we built an off-road terrain
circuit track. The testing track has different combination of terrains of
varying slopes of $0^{\circ}$-$30^{\circ}$ throughout the track. We then test
our robot in this testing track by taking two terrains at once to evaluate how
different approaches perform when there is a transition between terrains. We
then have multiple complex terrains made using a combination of grass,
gravels, large rocks, mud and sand terrains. An aerial view of this testing
track along with the multiple complex terrains is shown in 5. The dimension of
this testing track is 21 m x 10 m and each segment of terrain making up the
track is of length 5 meters long. Therefore, each segment of complex terrain
track (i.e., two terrain tracks combined) is either of 10 meters or 8 meters
depending on the section of track the robot is navigating over. The same
learned model used in the previous experimental scenario is deployed in this
experiment with no additional training. In this experiments we compute the
evaluation metrics based on five trial runs on each terrain track.

The quantitative results obtained by our approach and its comparison with
other state of the art methods are provided in Table 3.

###  5.3 Self-Reflection for Ground Navigation

In this set of experiments, we aim to evaluate all the approaches in
challenging scenarios where the robot would need the essential ability of
self-reflection. Specifically, we evaluate the performance in scenarios when
robots experience a change in their own functionality. Therefore, we introduce
in robots three categories of setbacks.W Each of these setbacks are used to
increase the difficulty of robot navigation in a terrain and would
increasingly demand the ability of self-reflection. These three setbacks are
each tested in specific terrains each listed below:

  * •

We _over-inflate_ all the robot tyres and with this setback robot generally
has a higher center of mass and reduced traction with the terrain. This
setback also causes increased jerkiness in the robot due to less damping from
the over-inflated tyres. With this setback we test the robots in three
different terrains of i) mixed terrain I (MT-I), ii) mixed terrain II (MT-II)
and iii) the forest terrain. None of the three terrains have any significant
slope.

  * •

We further reduce robot tyre traction by completely _deflating and duct taping
the tyres_. With this setback the wheel traction reduces significantly and the
robot has a lower center of mass. This setback in robot is evaluated in four
different terrains of i) reduced traction snow-grass (RT-Snow-Grass), ii)
reduced traction snow-mud (RT-Snow-Mud), iii) reduced traction hill (RT-Hill)
and iv) reduced traction snow-forest (RT-Snow-Forest). Each of these terrains
but the snow-forest terrain have a varying slope of $0^{\circ}$\- $45^{\circ}$
which further enhances the affect of this setback.

  * •

Finally, we introduce in the robot the setback of _reduced tyre pressure,
reduced wheel traction and increased payload_. That is on top of the
completely deflated and duct taped tyres, we further add a payload of fifteen
pounds. This setback is evaluated in two different terrain of i) reduced
traction payload snow-grass (RT-P-Snow-Grass) and ii) reduced traction payload
hill (RT-P-Hill). Both of these terrains have a varying slope of $0^{\circ}$\-
$45^{\circ}$.

These nine different testing terrains are illustrated in Figure 6. No
additional training is performed for this set of experiments and we use the
same model from Section 5.2.1. Besides, all the other parameters for
deployment are kept the same. In this set of experiments the evaluation
metrics for each of the methods are calculated based on ten trials of robot
runs on each of the terrains.

The quantitative results obtained by our approach and its comparison with
other approaches are presented in Table 3. For the first category of setback,
we observe that the failure rates is less in the tracks of mixed terrains and
the highest failure is observed in the forest terrain. Our approach achieves
the least failure rate in all of these three different terrains followed by
the MPPI and TRAL approach. The NLOPT approach achieves a higher failure rate
in both MT-II and forest terrain and is the poor performer amongst the
different approaches. In terms of the traversal time, we do observe that our
approach obtains a lesser traversal time in the terrains of MT-I but also has
a slowest traversal time in the forest terrain. This again shows the ability
of our approach to adapt its navigational behaviors as the terrain becomes
more and more cluttered. The MPPI approach performs better in the MT-II
terrain and the surprisingly we see that the NLOPT approach has the fastest
traversal time in the Forest terrain which otherwise performs the slowest in
the remaining two terrains. In terms of the inconsistency metric our approach
with the ability to generate consistent navigational behaviors is able to
obtain the least inconsistency metric overall followed by the MPPI approach.
Both TRAL and NLOPT approach have high inconsistency values in respective
terrains with TRAL approach achieving a poor performance in the Forest
terrain. In terms of the jerkiness metric, we observe that all the approaches
have an increased value of jerkiness indicating the cluttered nature of
terrain in a forest environment. Our approach has a lower jerkiness measure in
MT-I and MT-II terrain but also performs poorly in the Forest terrain. The
MPPI approach obtains the second best performance in both the MT-I and MT-II
terrain but also obtains the best performance in the Forest terrain. Both the
NLOPT and TRAL approach perform poorly and have a high jerkiness metric
overall.

In the second category of setbacks, we observe an increased failure rate as
compared to the previous terrains. Especially in the terrains with slope,
i.e., RT-Snow-Mud, RT-Hill and RT-Snow-Grass, failure rate is high as compared
to RT-Snow-Forest which has no slope. Our approach has lesser failure rate
throughout all the terrains in this category and the NLOPT approach performs
the worst. In terms of traversal time, we do see that both MPPI performs
better than our approach in all but RT-Snow-Mud terrain. Our approach trails
MPPI approach by a small window but does show terrain aware behaviors as the
velocities differ with the terrain as opposed to the MPPI approach. The
approach of TRAL has a poorer performance as compared to MPPI and our
approach, and the NLOPT obtains the highest traversal time values indicating
the slowest traversal in all the terrains. For the inconsistency metric, we
observe that our approach obtains the least value of inconsistency metric
followed by both MPPI and TRAL approach with MPPI performing better in RT-
Snow-Grass and RT-Snow-Forest and TRAL performing well in the remaining
terrains. The NLOPT approach obtains the highest inconsistency value
indicating a poor performance compared to other approaches. When observing the
jerkiness metrics obtained by the different approaches, we see a similar trend
of our approach performing the best in all the different terrains follows by
both MPPI and TRAL approach. The NLOPT approach has the lowest jerkiness value
in all the terrains but RT-Snow-Forest signaling a poor performance overall.
Additionally, we see a lesser value of jerkiness over RT-Snow-Forest terrain
as compared to the Forest terrain. This indicates that when covered with snow,
much of the cluttered nature of the forest terrain is subdued.

Finally, we evaluate and compare our approach in the third category of
setbacks. Having reduced tyre traction and payload especially on terrains with
slope is very challenging for mobile robots Grand et al. (2004). With these
challenges we observe that the failure rates are very high. Specifically in
the RT-P-Hill terrain we see that all the approaches fail in nearly half the
runs. This shows that with increased setbacks, it is really hard for robots to
achieve their expected behaviours. We do observe our approach to have a
slightly better performance over the other approaches followed by the MPPI,
TRAL and NLOPT approaches, in that order. In terms of traversal time, we see
mixed results. We observe that our approach obtains least traversal time in
RT-P-Snow-Grass terrain where as is slow in RT-P-Hill terrain. In the RT-P-
Hill terrain the TRAL approach obtains the lowest traversal time. The MPPI
approach obtains the second best performance throughout and the NLOPT approach
having the slowest traversal time in both the terrains. In terms of
inconsistency we observe that our approach has a lower inconsistency on both
the terrains followed by the MPPI approach and then the TRAL approach. Again,
we see that the inconsistency values obtained by the NLOPT approach is higher
than the other approaches. A similar trend is seen in terms of the jerkiness
metric as well. We see our approach has a better performance followed by the
MPPI approach in all the terrains. Both the TRAL and NLOPT approaches have a
higher value of jerkiness suggesting a poor performance.

Across all the three different category of setbacks, it is observed that by
generating inconsistent navigational behaviors and lowering inconsistency
values while navigating over various terrains, our approach is able to reduce
the number of failures by trading-off the traversal time in some scenarios.

((a)) Performance on Various Frame Sequences

((b)) Hyper-parameter Analysis

((c)) Historical Time Steps

((d)) Discriminative Feature Modalities

Figure 7: Analysis of various parameters used in our experimental runs.

###  5.4 Discussion

We further discuss on the additional experimental results to investigate the
characteristics of our approach. These investigations derive the basis for the
various parameters used in our experimental runs. This includes performance on
various frame sequences, hyperparameter analysis, relative weights of each
time step and the discriminative feature modalities.

####  5.4.1 Performance on Various Frame Sequences:

Our approach uses a sequence of historical frames ’$c$’, to generate terrain
aware behavior and self-reflective consistent behaviors. Figure 7(a) uses a
boxplot to show the distribution of inconsistency value for each value of
historical frame sequence $c=1,\dots,10$; under various values of
hyperparameters $\lambda_{1}\in[0.001,10]$ and $\lambda_{2}\in[0.001,10]$. The
boxplot displays a distribution of results based on a five number summary
(i.e., minimum, first quartlie, medium, thirds quartile, and maximum). It is
observed that our approach performs well in specific values of
$c=\\{1,5,10\\}$, and we observe the inconsistency metric being worst at
$c=\\{6,8\\}$. The best inconsistency value of 1.3 is observed when $c=5$,
that is the barplot with lowest minimum. This value of historical frame
sequences in used in all of our experiments.

####  5.4.2 Hyper-parameter Analysis:

Our approach utilizes two hyper-parameters $\lambda_{1}$ and $\lambda_{2}$
that balances the amount of loss from our objective function and the
regularization terms. Figure 7(b) depicts how the inconsistency metric changes
with each of these hyperparameter values for five historical frame sequences,
i.e., $c=5$. In general it is seen that the inconsistency metric changes
significantly with variations in $\lambda_{2}$ as compared to $\lambda_{1}$,
signifying the importance of modeling historical time steps. Altogether, it is
observed that $\lambda_{1}\in(0.01,1)$ an $\lambda_{2}\in(0.01,10)$ result in
a good performance. The best results are observed with $\lambda_{1}=0.1$ and
$\lambda_{2}=1$. These are the fixed hyperparameters were used during
execution for all the experiments.

####  5.4.3 Historical Time Steps:

Our approach models features and behavior difference from a sequences of
historical time steps to perform self-reflection. We therefor analyze the
importance of each time steps from the sequence of frames in generating
robot’s navigational behaviors. Figure 7(c) presents the importance of each
time steps in the historical sequence of time steps. It is observed that the
terrain features and behavior differences from most recent time steps have an
higher importance for generating navigational behaviors. We observed that the
present and past three time steps are equally important with the third past
time step having a weight of $20\%$. The fourth and fifth past time step have
a smaller importance of nearly $10\%$.

####  5.4.4 Discriminative Feature Modalities:

Our approach has the ability to automatically estimate the importance of the
various feature modalities. The results from evaluating the modality
importance is demonstrated in Figure 7(d). It is observed that the two feature
modalities, i.e., HOG and LBP features are most discriminative in performing
self-reflective terrain adaptation. These two features combined together
account for relative importance of $96\%$. The IMU features and elevation
features are comparatively not much important.

##  6 Conclusion

In this paper, we introduce a novel method of self-reflective terrain-aware
adaptation for autonomous ground robots to generate consistent behaviors to
navigate over unstructured off-road environments. Formulated in a unified
regularized constrained optimization framework, our method is also able to
identify discriminative terrain features and fuse them to perform effective
terrain adaptation. Additionally, we design a new algorithm that addresses the
formulated optimization problem, which holds a theoretical guarantee to
converge to the global optimal solution. To evaluate our approach, we conduct
extensive experiments using physical ground robots with varying functionality
changes over diverse unstructured off-road terrains. Experimental results have
shown that our self-reflective terrain-aware adaptation method outperforms
previous and baseline techniques and enables ground robots to generate
consistent behaviors when navigating in off-road environments.

## References

  * Adib and Masoumi (2017) Adib A and Masoumi B (2017) Mobile robots navigation in unknown environments by using fuzzy logic and learning automata.  In: _Artificial Intelligence and Robotics_.
  * Ahonen et al. (2006) Ahonen T, Hadid A and Pietikainen M (2006) Face description with local binary patterns: Application to face recognition.  _Transactions on Pattern Analysis and Machine Intelligence_ (12): 2037–2041.
  * Ardelt and Grunwald (2018) Ardelt M and Grunwald S (2018) The importance of self-reflection and awareness for human development in hard times.  _Research in Human Development_ .
  * Babunski et al. (2020) Babunski D, Berisha J, Zaev E and Bajrami X (2020) Application of fuzzy logic and pid controller for mobile robot navigation.  In: _Mediterranean Conference on Embedded Computing_.
  * Bekker (1969) Bekker MG (1969) Introduction to terrain-vehicle systems .
  * Bermudez et al. (2012) Bermudez FLG, Julian RC, Haldane DW, Abbeel P and Fearing RS (2012) Performance analysis and terrain classification for a legged robot over rough terrain.  In: _International Conference on Intelligent Robots and Systems_.
  * Bhattacharyya et al. (2018) Bhattacharyya SP, Datta A and Keel LH (2018) _Linear control theory: structure, robustness, and optimization_.  CRC press.
  * Borges et al. (2019) Borges CDB, Almeida AMA, Júnior ICP and Junior JJdMS (2019) A strategy and evaluation method for ground global path planning based on aerial images.  _Expert Systems With Applications_ 137: 232–252.
  * Brooks and Iagnemma (2005) Brooks CA and Iagnemma K (2005) Vibration-based terrain classification for planetary exploration rovers.  _Transactions on Robotics_ 21(6): 1185–1191.
  * Camacho and Alba (2013) Camacho EF and Alba CB (2013) _Model predictive control_.  Springer science & business media.
  * Chen et al. (2018) Chen P, Chen G and Zhang S (2018) Log hyperbolic cosine loss improves variational auto-encoder .
  * Dalal and Triggs (2005) Dalal N and Triggs B (2005) Histograms of oriented gradients for human detection.  In: _Conference on Computer Vision and Pattern Recognition_.
  * Dash et al. (2015) Dash T, Nayak T and Swain RR (2015) Controlling wall following robot navigation based on gravitational search and feed forward neural network.  In: _Proceedings of the international conference on perception and machine intelligence_.
  * Deisenroth and Rasmussen (2011) Deisenroth M and Rasmussen CE (2011) PILCO: A model-based and data-efficient approach to policy search.  In: _International Conference on Machine Learning_.
  * Erden and Leblebicioğlu (2008) Erden MS and Leblebicioğlu K (2008) Free gait generation with reinforcement learning for a six-legged robot.  _Robotics and Autonomous Systems_ 56(3): 199–212.
  * Fankhauser et al. (2014) Fankhauser P, Bloesch M, Gehring C, Hutter M and Siegwart R (2014) Robot-centric elevation mapping with uncertainty estimates.  In: _Mobile Service Robotics_. World Scientific, pp. 433–440.
  * Fox et al. (1997) Fox D, Burgard W and Thrun S (1997) The dynamic window approach to collision avoidance.  _Robotics and Automation Magazine_ 4(1): 23–33.
  * Fries (2018) Fries TP (2018) Autonomous robot navigation in diverse terrain using a fuzzy evolutionary technique.  In: _Annual Conference of the Industrial Electronics Society_.
  * Gangal et al. (2007) Gangal AS, Kalra P and Chauhan D (2007) Performance evaluation of complex valued neural networks using various error functions.  _Enformatika_ 23: 27–32.
  * Grand et al. (2004) Grand C, Benamar F, Plumet F and Bidaud P (2004) Stability and traction optimization of a reconfigurable wheel-legged robot.  _The International Journal of Robotics Research_ 23(10-11): 1041–1058.
  * Han et al. (2017) Han F, Yang X, Zhang Y and Zhang H (2017) Sequence-based multimodal apprenticeship learning for robot perception and decision making.  In: _International Conference on Robotics and Automation_.
  * Haruno et al. (2001) Haruno M, Wolpert DM and Kawato M (2001) Mosaic model for sensorimotor learning and control.  _Neural computation_ .
  * He et al. (2019) He B, Wang S and Liu Y (2019) Underactuated robotics: a review.  _International Journal of Robotics Research_ 16(4).
  * Hudjakov and Tamre (2009) Hudjakov R and Tamre M (2009) Aerial imagery terrain classification for long-range autonomous navigation.  In: _International Symposium on Optomechatronic Technologies_.
  * Jiang et al. (2020) Jiang P, Osteen P, Wigness M and Saripalli S (2020) Rellis-3d dataset: Data, benchmarks and analysis.  _arXiv preprint arXiv:2011.12954_ .
  * Johansson (1993) Johansson R (1993) _System modeling and identification_.  Prentice-hall.
  * Johnson (2011) Johnson SG (2011) The nlopt nonlinear-optimization package.  _?_ ?(?): ?
  * Kawatsuma et al. (2012) Kawatsuma S, Fukushima M and Okada T (2012) Emergency response by robots to fukushima-daiichi accident: summary and lessons learned.  _Industrial Robot: An International Journal_ .
  * Kelly (1994) Kelly A (1994) A feedforward control approach to the local navigation problem for autonomous vehicles.  Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA ROBOTICS INST.
  * Kleiner et al. (2002) Kleiner A, Dietl M and Nebel B (2002) Towards a life-long learning soccer agent.  In: _Robot Soccer World Cup_.
  * Knight et al. (2001) Knight R, Fisher F, Estlin T, Engelhardt B and Chien S (2001) Balancing deliberation and reaction, planning and execution for space robotic applications.  In: _International Conference on Intelligent Robots and Systems_.
  * Ko and Fox (2009) Ko J and Fox D (2009) GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models.  _Autonomous Robots_ 27(1): 75–90.
  * Koopman (1931) Koopman BO (1931) Hamiltonian systems and transformation in hilbert space.  _Proceedings of the national academy of sciences of the united states of america_ 17(5): 315.
  * Kuntze et al. (2012) Kuntze HB, Frey CW, Tchouchenkov I, Staehle B, Rome E, Pfeiffer K, Wenzel A and Wöllenstein J (2012) Seneka-sensor network with mobile robots for disaster management.  In: _IEEE Conference on Technologies for Homeland Security_.
  * Lenz et al. (2015) Lenz I, Knepper RA and Saxena A (2015) DeepMPC: Learning Deep Latent Features for Model Predictive Control. In: _Robotics: Science and Systems_.
  * Li and Todorov (2004) Li W and Todorov E (2004) Iterative linear quadratic regulator design for nonlinear biological movement systems.  In: _ICINCO (1)_. Citeseer, pp. 222–229.
  * Li et al. (2018) Li Z, Nakatsukasa Y, Soma T and Uschmajew A (2018) On orthogonal tensors and best rank-one approximation ratio.  _SIAM Journal on Matrix Analysis and Applications_ 39(1): 400–425.
  * Long et al. (2015) Long J, Shelhamer E and Darrell T (2015) Fully convolutional networks for semantic segmentation.  In: _Conference on Computer Vision and Pattern Recognition_. pp. 3431–3440.
  * Manduchi et al. (2005) Manduchi R, Castano A, Talukder A and Matthies L (2005) Obstacle detection and terrain classification for autonomous off-road navigation.  _Autonomous Robots_ 18(1): 81–102.
  * Marcovitch et al. (2008) Marcovitch S, Jacques S, Boseovski JJ and Zelazo PD (2008) Self-reflection and the cognitive control of behavior: Implications for learning.  _Mind, Brain, and Education_ 2(3): 136–141.
  * Meng and Kak (1993) Meng M and Kak AC (1993) Mobile robot navigation using neural networks and nonmetrical environmental models.  _Control Systems Magazine_ .
  * Meyerhof (1970) Meyerhof G (1970) Introduction to terrain-vehicle systems.
  * Nabulsi et al. (2006) Nabulsi S, Armada M and Montes H (2006) Multiple terrain adaptation approach using ultrasonic sensors for legged robots.  In: _International Conference on Climbing and Walking Robots_.
  * Negahban and Wainwright (2012) Negahban S and Wainwright MJ (2012) Restricted strong convexity and weighted matrix completion: Optimal bounds with noise.  _The Journal of Machine Learning Research_ 13: 1665–1697.
  * Neudecker (1969) Neudecker H (1969) A note on kronecker matrix products and matrix equation systems.  _SIAM Journal on Applied Mathematics_ 17(3): 603–606.
  * Nie et al. (2010) Nie F, Huang H, Cai X and Ding CH (2010) Efficient and robust feature selection via joint l2, 1-norms minimization.  In: _Advances in Neural Information Processing Systems_.
  * Nikolaidis et al. (2017) Nikolaidis S, Zhu YX, Hsu D and Srinivasa S (2017) Human-robot mutual adaptation in shared autonomy.  In: _International Conference on Human-Robot Interaction_.
  * Papadakis (2013) Papadakis P (2013) Terrain traversability analysis methods for unmanned ground vehicles: A survey.  _Engineering Applications of Artificial Intelligence_ 26(4): 1373–1385.
  * Park et al. (2017) Park S, Oh Y and Hong D (2017) Disaster response and recovery from the perspective of robotics.  _International Journal of Precision Engineering and Manufacturing_ 18(10): 1475–1482.
  * Parker (1996) Parker LE (1996) L-ALLIANCE: Task-oriented multi-robot learning in behavior-based systems.  _Advanced Robotics_ 11(4): 305–322.
  * Parker (2000) Parker LE (2000) Lifelong adaptation in heterogeneous multi-robot teams: Response to continual variation in individual robot performance.  _Autonomous Robots_ 8(3): 239–267.
  * Peynot et al. (2014) Peynot T, Lui ST, McAllister R, Fitch R and Sukkarieh S (2014) Learned stochastic mobility prediction for planning with control uncertainty on unstructured terrain.  _Journal of Field Robotics_ 31(6): 969–995.
  * Plagemann et al. (2008) Plagemann C, Mischke S, Prentice S, Kersting K, Roy N and Burgard W (2008) Learning predictive terrain models for legged robot locomotion.  In: _International Conference on Intelligent Robots and Systems_.
  * Posada et al. (2010) Posada LF, Narayanan KK, Hoffmann F and Bertram T (2010) Floor segmentation of omnidirectional images for mobile robot visual navigation.  In: _International Conference on Intelligent Robots and Systems_.
  * Proctor et al. (2018) Proctor JL, Brunton SL and Kutz JN (2018) Generalizing koopman theory to allow for inputs and control.  _Journal on Applied Dynamical Systems_ 17(1): 909–930.
  * Qi (2017) Qi L (2017) Transposes, l-eigenvalues and invariants of third order tensors.  _arXiv preprint arXiv:1704.01327_ .
  * Rabanser et al. (2017) Rabanser S, Shchur O and Günnemann S (2017) Introduction to tensor decompositions and their applications in machine learning.  _arXiv preprint arXiv:1711.10781_ .
  * Rabiner et al. (1978) Rabiner L, Crochiere R and Allen J (1978) Fir system modeling and identification in the presence of noise and with band-limited inputs.  _Transactions on Acoustics, Speech, and Signal Processing_ .
  * Sadhukhan et al. (2004) Sadhukhan D, Moore C and Collins E (2004) Terrain estimation using internal sensors.  In: _International Conference on Robotics and Automation_.
  * Schwarz et al. (2017) Schwarz M, Rodehutskors T, Droeschel D, Beul M, Schreiber M, Araslanov N, Ivanov I, Lenz C, Razlaw J, Schüller S et al. (2017) Nimbro rescue: Solving disaster-response tasks with the mobile manipulation robot momaro.  _Journal of Field Robotics_ 34(2): 400–425.
  * Shan et al. (2016) Shan M, Bi Y, Qin H, Li J, Gao Z, Lin F and Chen BM (2016) A brief survey of visual odometry for micro aerial vehicles.  In: _Annual Conference of the Industrial Electronics Society_.
  * Shan and Englot (2018) Shan T and Englot B (2018) LeGO-LOAM: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain.  In: _International Conference on Intelligent Robots and Systems_.
  * Shimoda et al. (2005) Shimoda S, Kuroda Y and Iagnemma K (2005) Potential field navigation of high speed unmanned ground vehicles on uneven terrain.  In: _International Conference on Robotics and Automation_.
  * Sidek and Sarkar (2013) Sidek N and Sarkar N (2013) Exploiting wheel slips of mobile robots to improve navigation performance.  _Advanced Robotics_ 27(8): 627–639.
  * Silver et al. (2010) Silver D, Bagnell JA and Stentz A (2010) Learning from demonstration for autonomous navigation in complex unstructured terrain.  _International Journal of Robotics Research_ 29(12): 1565–1592.
  * Siva et al. (2019) Siva S, Wigness M, Rogers J and Zhang H (2019) Robot adaptation to unstructured terrains by joint representation and apprenticeship learning.  In: _Robotics: Science and Systems_.
  * Smith (2002) Smith R (2002) Self-reflection and the self.  In: _Rewriting the Self_.
  * Tassa et al. (2014) Tassa Y, Mansard N and Todorov E (2014) Control-limited differential dynamic programming.  In: _International Conference on Robotics and Automation_.
  * Thrun (1998) Thrun S (1998) Lifelong learning algorithms.  In: _Learning to Learn_. Springer.
  * Trautmann and Ray (2011) Trautmann E and Ray L (2011) Mobility characterization for autonomous mobile robots using machine learning.  _Autonomous Robots_ 30(4): 369–383.
  * Tsunoda et al. (2020) Tsunoda Y, Sueoka Y, Wada T and Osuka K (2020) Sheepdog-type robot navigation: Experimental verification based on a linear model.  In: _International Symposium on System Integration_.
  * Urmson et al. (2004) Urmson C, Anhalt J, Clark M, Galatali T, Gonzalez JP, Gowdy J, Gutierrez A, Harbaugh S, Johnson-Roberson M, Kato H et al. (2004) High speed navigation of unrehearsed terrain: Red team technology for grand challenge 2004.  _Robotics Institute - Carnegie Mellon University_ .
  * Verginis and Dimarogonas (2021) Verginis CK and Dimarogonas DV (2021) Adaptive robot navigation with collision avoidance subject to 2nd-order uncertain dynamics.  _Automatica_ 123: 109303.
  * Watson and Marir (1994) Watson I and Marir F (1994) Case-based reasoning: A review.  _The Knowledge Engineering Review_ .
  * Wigness et al. (2019) Wigness M, Eum S, Rogers JG, Han D and Kwon H (2019) A rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments.  In: _International Conference on Intelligent Robots and Systems_.
  * Wigness et al. (2018) Wigness M, Rogers JG and Navarro-Serment LE (2018) Robot navigation from human demonstration: Learning control behaviors.  In: _International Conference on Robotics and Automation_.
  * Williams et al. (2016) Williams G, Drews P, Goldfain B, Rehg JM and Theodorou EA (2016) Aggressive driving with model predictive path integral control.  In: _International Conference on Robotics and Automation_.
  * Williams et al. (2015) Williams MO, Kevrekidis IG and Rowley CW (2015) A data–driven approximation of the koopman operator: Extending dynamic mode decomposition.  _Journal of Nonlinear Science_ 25(6): 1307–1346.
  * Xu et al. (2020) Xu X, Li J, Yang Y and Shen F (2020) Towards effective intrusion detection using log-cosh conditional variational autoencoder.  _IEEE Internet of Things Journal_ .
  * Yong and Zhou (1999) Yong J and Zhou XY (1999) _Stochastic controls: Hamiltonian systems and HJB equations_ , volume 43.  Springer Science & Business Media.