# Learning to Model and Plan for Wheeled Mobility on Vertically Challenging
Terrain

Aniket Datar, Chenhui Pan, and Xuesu Xiao
Department of Computer Sciences
George Mason University
{adatar, cpan7<EMAIL_ADDRESS>

###### Abstract

Most autonomous navigation systems assume wheeled robots are rigid bodies and
their 2D planar workspaces can be divided into free spaces and obstacles.
However, recent wheeled mobility research, showing that wheeled platforms have
the potential of moving over vertically challenging terrain (e.g., rocky
outcroppings, rugged boulders, and fallen tree trunks), invalidate both
assumptions. Navigating off-road vehicle chassis with long suspension travel
and low tire pressure in places where the boundary between obstacles and free
spaces is blurry requires precise 3D modeling of the interaction between the
chassis and the terrain, which is complicated by suspension and tire
deformation, varying tire-terrain friction, vehicle weight distribution and
momentum, etc. In this paper, we present a learning approach to model wheeled
mobility, i.e., in terms of vehicle-terrain forward dynamics, and plan
feasible, stable, and efficient motion to drive over vertically challenging
terrain without rolling over or getting stuck. We present physical experiments
on two wheeled robots and show that planning using our learned model can
achieve up to 60% improvement in navigation success rate and 46% reduction in
unstable chassis roll and pitch angles.

> Keywords: Wheeled Mobility, Vertically Challenging Terrain, Motion Planning

Figure 1: Front and side view (1st and 2nd row) of a wheeled robot navigating
vertically challenging terrain: (from left to right) large roll angle, stable
chassis, suspended wheel, roll-over, and get-stuck.

##  1 Introduction

Wheeled robots, arguably the most commonly used mobile robot type, have
autonomously moved from one point to another in a collision-free and efficient
manner in the real world, e.g., transporting materials in factories or
warehouses [1], vacuuming our homes or offices [2], and delivering food or
packages on sidewalks [3]. Thanks to their simple motion mechanism, most
wheeled robots are treated as rigid bodies moving through planar workspaces.
After tessellating their 2D workspaces into obstacles and free spaces,
classical planning algorithms plan feasible paths in the free spaces that are
free of collisions with the obstacles [4, 5, 6].

However, recent advances in wheeled mobility have shown that even conventional
wheeled robots (i.e., without extensive hardware modification such as active
suspensions [7, 8, 9] or adhesive materials [10]) have previously unrealized
potential to move over vertically challenging terrain (e.g., in mountain
passes with large boulders or dense forests with fallen trees) [11, 12, 13],
where vehicle motion is no longer constrained to a 2D plane [14] (Figure 1).
In those environments, neither assumptions of rigid vehicle chassis and clear
delineation between obstacles and free spaces in a simple 2D plane are valid
[15, 16, 17, 18]. Thanks to the long suspension travel and reduced tire
pressure, off-road vehicle chassis are able to drive _over_ obstacles (rather
than to _avoid_ them) and experience significant deformation to conform with
the irregular terrain underneath the robot, which will be otherwise deemed as
non-traversable according to conventional navigation systems. Therefore,
autonomously navigating wheeled robots in vertically challenging terrain
without rolling over or getting stuck requires a precise understanding of the
3D vehicle-terrain interaction.

In this paper, we investigate learning approaches to model vehicle-terrain
interactions and plan vehicle trajectories to drive wheeled robots on
vertically challenging terrain. Considering the difficulty in analytically
modeling and computing vehicle poses using complex vehicle dynamics [19, 20,
21] in real time, we adopt a data-driven approach to model the forward
vehicle-terrain dynamics based on terrain elevation maps along potential
future trajectories. We develop a Wheeled Mobility on Vertically Challenging
Terrain (WM-VCT) planner, which uses our learned model’s output in a novel
cost function and produces feasible, stable, and efficient motion plans to
autonomously navigate wheeled robots on vertically challenging terrain. We
present extensive physical experiment results on two wheeled robot platforms
and compare our learning approach against four existing baselines and show
that our learned model can achieve up to 60% improvement in navigation success
rate and 46% reduction in unstable chassis roll and pitch angles.

##  2 Related Work

In this section, we review related work in 2D robot navigation in planar
workspaces, facilitating mobility on vertically challenging terrain, and
machine learning approaches for mobile robots.

###  2.1 2D Rigid-Body Navigation in Planar Workspaces

Roboticists have been developing classical ground navigation planners for
decades [4, 5, 6]. Recently, researchers have also investigated high-speed
[22, 23], off-road [24, 25, 26], and social [27, 28, 29, 30] navigation. In
the aforementioned research thrusts, most robots are modeled as 2D rigid
bodies (e.g., bounding boxes) and most feasible navigation plans are in 2D and
collision-free, regardless of the robot type (e.g., wheeled or tracked). In
order to allow wheeled mobile robots to venture into other difficult-to-reach
spaces, recent work has extended wheeled mobility to vertically challenging
terrain [14], considering that vertical protrusions from the ground are not
uncommon in real-world unstructured environments [11, 12]. However, both rigid
body and planar workspace assumptions are no longer valid in such spaces,
requiring new methods to model the interactions between the non-rigid robots
and 3D environments.

###  2.2 Robot Mobility on Vertically Challenging Terrain

Most research aiming at allowing robots to move in vertically challenging
environments are from the hardware side. Still treating vehicles as rigid
bodies, tracked vehicles are expected to crawl over more rugged terrain than
wheeled platforms due to the increased surface contact and therefore
propulsion [31], while adhesive materials [10] and tethers [13] allow robots
to overcome gravity while climbing vertical slopes. Relaxing the assumption of
rigid body, vehicles with active suspensions have been developed [7, 8, 9] to
proactively maintain a stable pose of the chassis on vertically challenging
terrain. Highly articulated systems, e.g., legged [32, 33], wheel-legged [34,
35], or snake [36, 15] robots, are another choice to negotiate through such
terrain with a stable torso, or virtual chassis for limbless snake robots
[37], by solving many Degrees-of-Freedom (DoFs) of the robot joints. However,
despite the versatility to overcome verticality, such specialized hardware are
expensive, inefficient (especially on flat terrain), and not as common as
conventional wheeled robots.

###  2.3 Machine Learning for Robot Mobility

In addition to the aforementioned classical methods, roboticists have also
started utilizing data-driven approaches for robot mobility [38]. Learning
from data, robots no longer need hand-crafted models [23, 39, 40], cost
functions [41, 42, 30], or planner parameters [43, 44, 45, 46, 47], see beyond
sensor range [48, 49], and acquire navigation behaviors end-to-end from sensor
data [50, 51, 52]. Researchers have also used end-to-end Behavior Cloning (BC)
[53] to address wheeled mobility on vertically challenging terrain [14].
Considering the power of machine learning and drawbacks of end-to-end BC
(e.g., data-hungry, prone to overfitting, and not generalizable), this work
takes a structured learning approach to only model the vehicle-terrain forward
dynamics, then constructs a novel cost function tailored for vertically
challenging terrain, and finally plans trajectories to move robots toward
their goal.

##  3 Approach

The difficulties in navigating a wheeled mobile robot on vertically
challenging terrain are two fold: (1) the high variability of vehicle poses
due to the irregular terrain underneath the robot may overturn the vehicle
(rolling-over, 4th column in Figure 1); (2) not being able to identify that a
certain terrain patch is beyond the robot’s mechanical limit and therefore
needs to be circumvented may get the robot stuck (immobilization, 5th column
in Figure 1). Therefore, this work takes a structured learning approach to
address both challenges by learning a vehicle-terrain forward dynamics model
based on the vertically challenging terrain underneath the vehicle, using it
to rollout sampled receding-horizon trajectories, and minimizing a cost
function to reduce the chance of rolling-over and immobilization and to move
the vehicle toward the goal.

###  3.1 Motion Planning Problem Formulation

Consider a discrete vehicle dynamics model of the form
$\mathbf{x}_{t+1}=f(\mathbf{x}_{t},\mathbf{u}_{t})$, where $\mathbf{x}_{t}\in
X$ and $\mathbf{u}_{t}\in U$ denote the state and input space respectively. In
the normal case of 2D navigation planning (Figure 2 left),
$X\subset\mathbb{SE}(2)$ and $X=X_{\textrm{free}}\cup X_{\textrm{obs}}$, where
$X_{\textrm{free}}$ and $X_{\textrm{obs}}$ denote free spaces and obstacle
regions. $\mathbf{x}_{t}$ includes translation along the $\mathbf{x}$ and
$\mathbf{y}$ axis ($x$ and $y$) and the rotation along the
$\mathbf{z}=\mathbf{x}\times\mathbf{y}$ axis (yaw) of a fixed global
coordinate system. For input, $U\subset\mathbb{R}^{2}$ and
$\mathbf{u}_{t}=(v_{t},\omega_{t})$, where $v_{t}$ and $\omega_{t}$ are the
linear and angular velocity. Finally, let $X_{\textrm{goal}}\subset X$ denote
the goal region. The motion planning problem for the conventional 2D
navigation case is to find a control function
$u:\\{t\\}_{t=0}^{T-1}\rightarrow U$ that produces an optimal path
$\mathbf{x}_{t}\in X_{\textrm{free}},\forall t\in\\{t\\}_{t=0}^{T}$ from an
initial state $\mathbf{x}_{0}=\mathbf{x}_{\textrm{init}}$ to the goal region
$\mathbf{x}_{T}\in X_{\textrm{goal}}$ that follows the system dynamics
$f(\cdot,\cdot)$ and minimizes a given cost function $c(x)$, which maps from a
state trajectory $x:\\{t\\}_{t=0}^{T}\rightarrow X$ to a positive real number.
In many cases, $c(x)$ is simply the total time step $T$ to reach the goal.
Considering the difficulty in finding the absolute minimal-cost state
trajectory, many mobile robots use sampling-based motion planners to find
near-optimal solutions [54, 55].

Conversely, in our case of wheeled mobility on vertically challenging terrain,
vehicle state $X\subset\mathbb{SE}(3)$ (i.e., translations and rotations along
the $\mathbf{x}$, $\mathbf{y}$, and $\mathbf{z}$ axis) with the same input
$\mathbf{u}_{t}=(v_{t},\omega_{t})\in U\subset\mathbb{R}^{2}$. The system
dynamics enforces that $\mathbf{x}_{t}$ is always “on top of” a subset of
$X_{\textrm{obs}}$ (i.e., vertically challenging terrain underneath and
supporting the robot) or some boundary of $X$ (i.e., on a flat ground) due to
gravity, requiring a vehicle-terrain dynamics model (Figure 2 right).

Figure 2: 2D navigation in $\mathbb{SE}(2)$ vs. navigation on vertically
challenging terrain in $\mathbb{SE}(3)$.

###  3.2 Vehicle-Terrain Dynamics Model Learning

Compared to the simple 2D vehicle dynamics in $\mathbb{SE}(2)$, our non-rigid
vehicle-terrain dynamics on vertically challenging terrain in $\mathbb{SE}(3)$
becomes more difficult to model, considering the complex interaction between
the terrain and chassis via the long suspension travel and deflated tire
pressure of off-road vehicles to assure adaptivity and traction (Figure 1).
Therefore, this work adopts a data-driven approach to learn the vehicle-
terrain dynamics model, which can be used to rollout trajectories for
subsequent planning.

To be specific, $\mathbf{x}_{t}=(x_{t},y_{t},z_{t},r_{t},p_{t},\phi_{t})$,
where the first and last three denote the translational ($x$, $y$, $z$) and
rotational (roll, pitch, yaw) component respectively along the $\mathbf{x}$,
$\mathbf{y}$, and $\mathbf{z}$ axis. Note that unlike most 2D navigation
problems in which the next vehicle state $\mathbf{x}_{t+1}$ only relies on the
current vehicle state $\mathbf{x}_{t}$ and input $\mathbf{u}_{t}$ alone, our
next vehicle state is additionally affected by the vertically challenging
terrain underneath and in front of the vehicle in the current time step,
denoted as $\mathbf{m}_{t}$. Therefore, the forward dynamics on vertically
challenging terrain can be formulated as

$\mathbf{x}_{t+1}=f_{\theta}(\mathbf{x}_{t},\mathbf{u}_{t},\mathbf{m}_{t}),$
(1)

which is parameterized by $\theta$ and will be learned in a data-driven
manner. Training data of size $N$ can be collected by driving a wheeled robot
on different vertically challenging terrain and recording the current and next
state, current terrain, and current input:
$\mathcal{D}=\\{\langle\mathbf{x}_{t},\mathbf{x}_{t+1},\mathbf{m}_{t},\mathbf{u}_{t}\rangle_{t=1}^{N}$}.
Then we learn $\theta$ by minimizing a supervised loss function:

$\theta^{*}=\operatorname*{arg\,min}_{\theta}\sum_{(\mathbf{x}_{t},\mathbf{x}_{t+1},\mathbf{m}_{t},\mathbf{u}_{t})\in\mathcal{D}}\lVert
f_{\theta}(\mathbf{x}_{t},\mathbf{u}_{t},\mathbf{m}_{t})-\mathbf{x}_{t+1}\rVert_{H},$
(2)

where $||\mathbf{v}||_{H}=\mathbf{v}^{T}H\mathbf{v}$ is the norm induced by a
positive definite matrix $H$, used to weigh the learning loss of the different
dimensions of the vehicle state $\mathbf{x}_{t}$. The learned vehicle-terrain
forward dynamics model, $f_{\theta}(\cdot,\cdot,\cdot)$, can then be used to
rollout future trajectories for minimal-cost planning.

###  3.3 Sampling-Based Receding-Horizon Planning

We adopt a sampling-based receding-horizon planning paradigm, in which the
planner first uniformly samples input sequences up until a short horizon $H$,
uses the learned model $f_{\theta}$ to rollout state trajectories, evaluates
their cost based on a pre-defined cost function, finds the minimal-cost
trajectory, executes the first input, replans, and thus gradually moves the
horizon closer to the final goal. In this way, the modeling error can be
corrected by frequent replanning. However, an under-actuated wheeled robot,
i.e., using $\mathbf{u}_{t}=(v_{t},\omega_{t})\in U\subset\mathbb{R}^{2}$ to
actuate $\mathbf{x}_{t}=\\{x_{t},y_{t},z_{t},r_{t},p_{t},\phi_{t})\in
X\subset\mathbb{SE}(3)$ subject to $f_{\theta}$, may easily end up in many
terminal states outside of $X_{\textrm{goal}}$, which the vehicle cannot
escape and recover from, i.e., rolling over due to excessive roll and pitch
angles or immobilization after getting stuck, e.g., on a large rock.
Therefore, while our goal is still to minimize the traversal time $T$ leading
to $X_{\textrm{goal}}$, for our receding-horizon planner, we seek to optimize
five cost terms on a state trajectory
$\mathbf{x}_{0:H}=\\{\textbf{x}_{t}\\}_{t=0}^{H},\textrm{s.t.},\mathbf{x}_{t+1}=f_{\theta}(\mathbf{x}_{t},\mathbf{u}_{t},\mathbf{m}_{t}),\forall
t<H$, which starts at the current time $0$ and ends at the horizon $H$, to
avoid these two types of terminal states on vertical challenging terrain and
also move the robot towards the goal:

$c(\mathbf{x}_{0:H})=w_{1}c_{\textrm{ro}}(\mathbf{x}_{0:H})+w_{2}c_{\textrm{im}}(\mathbf{x}_{0:H})+w_{3}c_{\textrm{hc}}(\mathbf{x}_{0:H})+w_{4}c_{\textrm{mb}}(\mathbf{x}_{0:H})+w_{5}c_{\textrm{est}}(x_{H}),$
(3)

where $c_{\textrm{ro}}(\cdot)$ and $c_{\textrm{im}}(\cdot)$ denote the cost
corresponding to the robot’s rolling over and being immobilized respectively;
$c_{\textrm{hc}}$ is the cost for experiencing large height change;
$c_{\textrm{mb}}$ is the cost of moving out of the existing map boundary;
$c_{\textrm{est}}$ is the estimated cost to reach the final goal region
$X_{\textrm{goal}}$ from the state on the horizon $x_{H}$, which can be
computed by the Euclidean distance
$c_{\textrm{est}}(x_{H})=||x_{H}-x_{G}||_{2}$, where $x_{G}$ is any state
inside $X_{\textrm{goal}}$. $w_{1}$ to $w_{5}$ are corresponding weights for
the cost terms.

###  3.4 Modeling Rolling-Over and Immobilization

Vehicle roll-over is often associated with large roll and pitch angles, which
we therefore seek to minimize along the state trajectory. Note that roll
$r_{t}$ and pitch $p_{t}$ are part of the vehicle state $\mathbf{x}_{t}$.
Therefore, we design a cost term that considers the absolute values of roll
and pitch:

$c_{\textrm{ro}}(\mathbf{x}_{0:H})=w_{1,1}\sum_{t=0}^{H}|r_{t}|+w_{1,2}\sum_{t=0}^{H}|p_{t}|,$
(4)

where $w_{1,1}$ and $w_{1,2}$ weigh the effect of the absolute value of roll
and pitch.

Similarly, vehicle immobilization often happens when the vehicle state does
not change from time to time. Therefore, trajectories on which vehicle state
significantly changes, especially along the translational dimension $x$ and
$y$, are encouraged:

$c_{\textrm{im}}(\mathbf{x}_{0:H})=-w_{2,1}\sum_{t=1}^{H}|x_{t}-x_{t-1}|-w_{2,2}\sum_{t=1}^{H}|y_{t}-y_{t-1}|,$
(5)

where $w_{2,1}$ and $w_{2,2}$ weigh the effect of displacement along $x$ and
$y$ direction.

Furthermore, the vehicle should prefer gentle slope rather than large height
change, so we encourage small displacement in the $z$ direction along the
trajectory:

$c_{\textrm{hc}}(\mathbf{x}_{0:H})=\sum_{t=1}^{H}|z_{t}-z_{t-1}|.$ (6)

With the cost function (Equation (3)) and cost terms (Equation (4), (5), and
(6)) defined, we can use any motion planner to find the minimal-cost 6-DoF
state trajectory $\mathbf{x}_{0:H}$ (see details in Section 4).

##  4 Experiments

In this section, we present implementation details and physical experiment
results of our WM-VCT navigation planner. We also compare WM-VCT’s performance
against other navigation baselines designed for vertically challenging
terrain.

###  4.1 Implementation and Baselines

We implement our WM-VCT planner on two open-source wheeled robot platforms,
the Verti-Wheelers (VWs) [14], one with six wheels (V6W,
$0.863\textrm{m}\times 0.249\textrm{m}\times 0.2\textrm{m}$) and the other
with four wheels (V4W, $0.523\textrm{m}\times 0.249\textrm{m}\times
0.2\textrm{m}$). Both robots are equipped with a Microsoft Azure Kinect RGB-D
camera with a 1-DoF gimbal actuated by a servo to fixate the field of view on
the terrain in front of the vehicle regardless of chassis pose. NVIDIA Jetson
computers (ORIN and Xavier for V6W and V4W respectively) provide onboard
computation. We use low-gear and lock both front and rear differentials to
improve mobility on vertically challenging terrain. Both robots are tested on
a $3.1\textrm{m}\times 1.3\textrm{m}$ rock testbed (with the highest vertical
point of $X_{\textrm{obs}}$ reaching 0.6m) composed of hundreds of rocks and
boulders of an average size of 30cm, similar to the size of the robots (Figure
1). The rocks and boulders on the testbed are shuffled many times during
experiments.

The vehicle state estimation for $\mathbf{x}_{t}$ is provided by an online
Visual Inertia Odometry system provided by the rtabmap_ros package [56]. In
order to represent $\mathbf{m}_{t}$, we process the RGB-D input into an
elevation map [57], a 2D grid where each pixel ($8$mm resolution) indicates
the height of the terrain at that point. For simplicity, we decompose the full
6-DoF vehicle-terrain dynamics model
$\mathbf{x}_{t+1}=(x_{t+1},y_{t+1},z_{t+1},r_{t+1},p_{t+1},\phi_{t+1})=f_{\theta}(\mathbf{x}_{t},\mathbf{u}_{t},\mathbf{m}_{t})$
into two parts: we utilize a planar Ackermann-steering model to obtain
approximate $(x_{t+1},y_{t+1},\phi_{t+1})$ based on $(x_{t},y_{t},\phi_{t})$
and $\mathbf{u}_{t}$, while $z_{t+1}$ is determined by the value of the
elevation map $\mathbf{m}_{t}$ at $(x_{t+1},y_{t+1})$; We use a neural network
to predict the roll $r_{t+1}$ and pitch $p_{t+1}$ angles based on
$\mathbf{m}_{t}$, $r_{t}$, and $p_{t}$. In practice, we find such a
decomposition very efficient and sufficiently accurate in capturing the 6-DoF
vehicle dynamics (see experiment results) and leave learning the full dynamics
model as future work. We use two $40$$\times$$100$ ($0.32$m$\times 0.8$m)
elevation maps centered at both the robot’s current and next position and
aligned with the current and next yaw angle, i.e., underneath
$(x_{t},y_{t})$/$(x_{t+1},y_{t+1})$ and aligned with $\phi_{t}$/$\phi_{t+1}$.
In our neural network model, a fully connected sequential head (8000-64-32-8
neurons) processes the $2\times 40\times 100$ elevation maps into a
8-dimensional embedding, which is then concatenated with the current roll
$r_{t}$ and pitch $p_{t}$ angles and fed into two more fully connected layers
(10-8-2), before finally producing the next roll $r_{t+1}$ and pitch $p_{t+1}$
values. To train the model for each vehicle, we use the open-source Verti-
Wheeler datasets [14], which include 43161 data frames for V6W and 41367 data
frames for V4W crawling over different vertically challenging testbeds.

Table 1: Cost Function Weights $w_{1}$ | $w_{2}$ | $w_{1,1}$ | $w_{1,2}$ | $w_{2,1}$ | $w_{2,2}$ | $w_{3}$ | $w_{4}$ | $w_{5}$
---|---|---|---|---|---|---|---|---
1 | 8 | 0.4 | 0.4 | 1 | 1 | 0.07 | 10 | 4

For the sampling-based receding-horizon planner, we keep the linear velocity
$v$ constant ($0.2\textrm{m/s}$) and sample 11 angular velocities $\omega$
(evenly from $[-0.78\textrm{rad/s},0.78\textrm{rad/s}]$). With a step size of
1 second, we roll out our vehicle dynamics model with the 11 $(v,\omega)$
pairs five times, evaluate the trajectory costs (Equation (3) to (6)) with
corresponding cost weights listed in Table 1, and expand the search tree again
from the 3rd state on the lowest-cost trajectory using the same 11 velocity
pairs. We repeat this process five times and find the overall minimal-cost
trajectory of horizon 15 ($\mathbf{x}_{0:15}$).

We implement a low-level controller operating at a frequency of 30Hz, with the
objective of tracking the trajectory that incurs the lowest cost.
Specifically, aiming to reach the next state $\mathbf{x}_{t+1}$ from the
current state $\mathbf{x}_{t}$, the linear velocity controller determines the
throttle command (ranging from $[-1.0,1.0]$) by considering the current pitch
angle $p_{t}$: the controller switches throttle command among three intervals,
i.e., $0.15$ for pitch less than -5°, 0.20 for pitch from -5° to 5°, and 0.30
for pitch greater than 5°. Such a mechanism approximately maintains constant
velocity with respect to changing terrain. The steering angle is calculated by
measuring the angle between the current yaw angle $\phi_{t}$ and the line
connecting the current 2D position $(x_{t},y_{t})$ to the next position
$(x_{t+1},y_{t+1})$. Meanwhile, while the receding-horizon planner constantly
replans at 2Hz, the controller will also trigger instant replanning when the
distance between the robot and the planned trajectory exceeds a predefined
threshold (0.4m).

Our proposed WM-VCT navigation planner is compared against four baselines. The
three baseline algorithms developed with the open-source Verti-Wheelers
project [14], i.e., Open-Loop (OL), Rule-Based (RB), and Behavior Cloning
(BC), are implemented on our robots and compared against WM-VCT. We also
compare our method against the Art planner [58], a state-of-the-art motion
planner based on learned motion cost for quadruped robots to navigate on rough
terrain, which is shown to be not applicable for our wheeled vehicle-terrain
dynamics. In Figure 3, we show the V6W navigating the testbed (top middle),
front (top left) and top (top right) view of the elevation map with the
planned 6-DoF vehicle state trajectory, and pitch and roll values in two
example environments (bottom left and right). In the first environment, while
both BC (red) and WM-VCT (green) succeed, the former experiences larger roll
and pitch values; in the second environment, BC (red) fails due to the
excessive roll angle around 7.5s, while WM-VCT is able to successfully
navigate through.

Figure 3: Physical Experiments: The V6W (top middle) navigates through a vertical challenging environment (front and top view of the elevation map shown in top left and top right with planned 6-DoF vehicle poses and trajectory in black); roll and pitch values of two successful WM-VCT trials (green lines at the bottom) are shown, while BC suffers from larger values in the first (red lines bottom left) and fails the second (red lines bottom right). Table 2: Experiment Results of BC and WM-VCT: Number of successful trials (Out of 5), mean traversal time (of successful trials in seconds), and average roll/pitch angles (in degrees). OL, RB, and Art fail all trials. Best results are shown in bold.  | V6W | V4W
---|---|---
| BC | WM-VCT | BC | WM-VCT
Easy |  5, 15.8s, 7.3°/7.9° |  5, 24s, 5.1°/7.5° |  2, 18.0s, 9.2°/17.5° |  2, 27.5s, 5.8°/9.5°
Medium | 3, 17.0s, 9.4°/8.3° |  4, 24.5s, 6.1°/8.6° | 1, 16.0s, 12°/8.5° |  2, 32.5s, 7.9°/11.4°
Difficult | 1, 20.0s, 8.3°/10.7° |  4, 22.7s, 6.2°/7.4° | N/A | N/A

###  4.2 Testbed Experiment Results

We randomly shuffle the testbed three times and test all four baselines
against our WM-VCT planner. In all our three test environments, both OL and RB
do not finish one single trial, either rolling over or getting stuck on
challenging rocks. The Art planner aims at planning trajectories that go
through flat surfaces for a quadruped to have stable foothold, while the
quadruped torso’s roll and pitch angles can be simply stabilized by the many
DoFs on the four limbs, which does not apply to wheeled vehicles. It is also
designed for a large quadruped robot carrying heavy-duty onboard computation
and takes more than 10 seconds for one single planning cycle on the small VWs.
Without timely replanning, our controller is not able to follow the outdated
trajectory tailored for legged robots and therefore fails every time at
difficult scenarios after deviating from the planned path.

We present our experiment results in Table 2. The left half shows the V6W
results while the right half V4W in the three obstacle courses labeled with
three difficulty levels, five trials each. In general, our WM-VCT planner
achieves better results on both six-wheeled and four-wheeled platforms,
compared to BC, the only baseline that can occasionally navigate through, in
terms of navigation success rate and average roll and pitch angles. For V6W,
both BC and WM-VCT finish all five trials in the easy environment, with BC
being more aggressive and therefore achieve faster traversal time but larger
roll and pitch angles; for the medium difficulty, BC fails two trials due to
rolling-over and immobilization, while WM-VCT fails only one; in the difficult
environment, BC succeeds only one while WM-VCT fails only one. Note the
shorter traversal time of BC is calculated based on only the successful
trials, showing its aggressiveness, while WM-VCT has lower roll and pitch
angles in most cases (other than slightly larger pitch for Medium). V4W is a
much less mechanically capable platform, and therefore fails more than V6W (it
fails all trials in the difficult environment with both BC and WM-VCT). But
the overall comparison between BC and WM-VCT remains the same: WM-VCT finishes
more trials, is slower but more stable, and achieves lower roll and pitch
angles overall (except pitch for Medium).

###  4.3 Outdoor Mobility Demonstration

Figure 4: Outdoor Mobility Demonstration: BC rolls over (left) and WM-VCT
succeeds (right).

In addition to the controlled testbed experiments, we also deploy our WM-VCT
outdoors in natural vertically challenging terrain with different rock sizes
to test our planner’s generalizability. The outdoor environment is unseen in
the training set and will cause challenge for BC’s generalizability, while we
expect the limited learning scope of WM-VCT is able to overcome such out-of-
distribution scenarios. Both V6W and V4W are able to successfully avoid
excessive roll and pitch angles and getting stuck on large, unseen rocks in
the outdoor environment. As shown in Figure 4, BC fails because the learned
end-to-end policy causes excessive roll angle and leads to roll-over (left),
while WM-VCT’s vehicle-terrain dynamic model successfully generalizes to the
unseen outdoor environment.

##  5 Limitation

Currently our method only uses learning for the vehicle-terrain forward
dynamics, while the cost function being used to select minimal-cost state
trajectory is still manually designed and requires fine-tuning of the
corresponding cost weights. In future work, those design choices can be
optimized by learning from expert demonstration using Inverse Optimal Control
[59] or trial-and-error data using Reinforcement Learning [60]. Another
limitation is the intensive computation to rollout the full forward dynamics
based on the learned dynamics model, therefore requiring our decomposition for
simplification. More efficient rollout methods need to be investigated in the
future. Furthermore, our uniform sampling strategy and constant control input
along the entire planning horizon remain to be improved possibly by learning
the sampling distribution based on the information embedded in the elevation
map [61]. Lastly, a versatile wheeled robot should not only be able to
locomote on vertically challenging terrain, but also efficiently drive through
relatively easy environments by switching to high-gear and releasing
unnecessary differential locks (front and/or rear), which is still beyond the
capability of our method.

##  6 Conclusions

We present a learning approach to enable wheeled mobility on vertically
challenging terrain. Going beyond the current 2D motion planning assumptions
of rigid vehicle bodies and 2D planar workspaces which can be divided into
free spaces and obstacles, our WM-VCT planner first learns to model the non-
rigid vehicle-terrain forward dynamics in $\mathbb{SE}(3)$ based on the
current vehicle state, input, and underlying terrain. Leveraging the
trajectory roll-outs under a sampling-based receding-horizon planning paradigm
using the learned vehicle-terrain forward dynamics, WM-VCT constructs a novel
cost function to prevent the vehicle from rolling-over and immobilization when
facing previously non-traversable obstacles. We show that WM-VCT can produce
feasible, stable, and efficient motion plans to drive robots over vertically
challenging terrain toward their goal and outperform several state-of-the-art
baselines on two physical wheeled robot platforms.

## References

  * IEEE Spectrum [2008] IEEE Spectrum.  Kiva systems – three engineers, hundreds of robots, one warehouse.  https://spectrum.ieee.org/three-engineers-hundreds-of-robots-one-warehouse, 2008\.  Accessed: 2023-05-16.
  * iRobot [2023] iRobot.  iRobot – robot vacuum and mop.  https://www.irobot.com/, 2023.  Accessed: 2023-05-16.
  * Amazon [2023] Amazon.  Meet scout.  https://www.aboutamazon.com/news/transportation/meet-scout, 2023\.  Accessed: 2023-05-16.
  * Fox et al. [1997] D. Fox, W. Burgard, and S. Thrun.  The dynamic window approach to collision avoidance.  _IEEE Robotics & Automation Magazine_, 4(1):23–33, 1997.
  * Quinlan and Khatib [1993] S. Quinlan and O. Khatib.  Elastic bands: Connecting path planning and control.  In _[1993] Proceedings IEEE International Conference on Robotics and Automation_ , pages 802–807. IEEE, 1993.
  * Rösmann et al. [2017] C. Rösmann, F. Hoffmann, and T. Bertram.  Kinodynamic trajectory optimization and control for car-like robots.  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 5681–5686. IEEE, 2017.
  * Cordes et al. [2014] F. Cordes, C. Oekermann, A. Babu, D. Kuehn, T. Stark, F. Kirchner, and D. Bremen.  An active suspension system for a planetary rover.  In _Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS)_ , pages 17–19, 2014\.
  * Islam et al. [2017] M. R. Islam, F. H. Chowdhury, S. Rezwan, M. J. Ishaque, J. U. Akanda, A. S. Tuhel, and B. B. Riddhe.  Novel design and performance analysis of a mars exploration robot: Mars rover mongol pothik.  In _2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN)_ , pages 132–136. IEEE, 2017.
  * Jiang et al. [2019] H. Jiang, G. Xu, W. Zeng, F. Gao, and K. Chong.  Lateral stability of a mobile robot utilizing an active adjustable suspension.  _Applied Sciences_ , 9(20):4410, 2019.
  * Liu and Seo [2018] Y. Liu and T. Seo.  Anyclimb-ii: Dry-adhesive linkage-type climbing robot for uneven vertical surfaces.  _Mechanism and Machine Theory_ , 124:197–210, 2018.
  * Murphy [2014] R. R. Murphy.  _Disaster robotics_.  MIT press, 2014.
  * Xiao and Murphy [2018] X. Xiao and R. Murphy.  A review on snake robot testbeds in granular and restricted maneuverability spaces.  _Robotics and Autonomous Systems_ , 110:160–172, 2018.
  * McGarey et al. [2016] P. McGarey, F. Pomerleau, and T. D. Barfoot.  System design of a tethered robotic explorer (trex) for 3d mapping of steep terrain and harsh environments.  In _Field and Service Robotics: Results of the 10th International Conference_ , pages 267–281. Springer, 2016.
  * Datar et al. [2023] A. Datar, C. Pan, M. Nazeri, and X. Xiao.  Toward wheeled mobility on vertically challenging terrain: Platforms, datasets, and algorithms.  _arXiv preprint arXiv:2303.00998_ , 2023.
  * Xiao et al. [2015] X. Xiao, E. Cappo, W. Zhen, J. Dai, K. Sun, C. Gong, M. J. Travers, and H. Choset.  Locomotive reduction for snake robots.  In _2015 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 3735–3740. IEEE, 2015.
  * Murphy et al. [2016] R. Murphy, J. Dufek, T. Sarmiento, G. Wilde, X. Xiao, J. Braun, L. Mullen, R. Smith, S. Allred, J. Adams, et al.  Two case studies and gaps analysis of flood assessment for emergency management with small unmanned aerial systems.  In _2016 IEEE international symposium on safety, security, and rescue robotics (SSRR)_ , pages 54–61. IEEE, 2016.
  * Xiao et al. [2017] X. Xiao, J. Dufek, T. Woodbury, and R. Murphy.  Uav assisted usv visual navigation for marine mass casualty incident response.  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 6105–6110. IEEE, 2017.
  * Xiao et al. [2021] X. Xiao, J. Dufek, and R. R. Murphy.  Autonomous visual assistance for robot operations using a tethered uav.  In _Field and Service Robotics: Results of the 12th International Conference_ , pages 15–29. Springer, 2021.
  * Jazar [2008] R. N. Jazar.  _Vehicle dynamics_ , volume 1.  Springer, 2008.
  * Yan et al. [2019] G. Yan, M. Fang, and J. Xu.  Analysis and experiment of time-delayed optimal control for vehicle suspension system.  _Journal of Sound and Vibration_ , 446:144–158, 2019.
  * Aly and Salem [2013] A. A. Aly and F. A. Salem.  Vehicle suspension systems control: a review.  _International journal of control, automation and systems_ , 2(2):46–54, 2013.
  * Atreya et al. [2022] P. Atreya, H. Karnan, K. S. Sikand, X. Xiao, S. Rabiee, and J. Biswas.  High-speed accurate robot control using learned forward kinodynamics and non-linear least squares optimization.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 11789–11795. IEEE, 2022.
  * Xiao et al. [2021] X. Xiao, J. Biswas, and P. Stone.  Learning inverse kinodynamics for accurate high-speed off-road navigation on unstructured terrain.  _IEEE Robotics and Automation Letters_ , 6(3):6054–6060, 2021.
  * Manduchi et al. [2005] R. Manduchi, A. Castano, A. Talukder, and L. Matthies.  Obstacle detection and terrain classification for autonomous off-road navigation.  _Autonomous robots_ , 18:81–102, 2005.
  * Jackel et al. [2006] L. D. Jackel, E. Krotkov, M. Perschbacher, J. Pippine, and C. Sullivan.  The darpa lagr program: Goals, challenges, methodology, and phase i results.  _Journal of Field robotics_ , 23(11-12):945–973, 2006.
  * Mousazadeh [2013] H. Mousazadeh.  A technical review on navigation systems of agricultural autonomous off-road vehicles.  _Journal of Terramechanics_ , 50(3):211–232, 2013\.
  * Mirsky et al. [2021] R. Mirsky, X. Xiao, J. Hart, and P. Stone.  Prevention and resolution of conflicts in social navigation–a survey.  _arXiv preprint arXiv:2106.12113_ , 2021.
  * Karnan et al. [2022] H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone.  Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation.  _IEEE Robotics and Automation Letters_ , 7(4):11807–11814, 2022.
  * Chen et al. [2017] Y. F. Chen, M. Everett, M. Liu, and J. P. How.  Socially aware motion planning with deep reinforcement learning.  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 1343–1350. IEEE, 2017.
  * Xiao et al. [2022] X. Xiao, T. Zhang, K. M. Choromanski, T.-W. E. Lee, A. Francis, J. Varley, S. Tu, S. Singh, P. Xu, F. Xia, S. M. Persson, L. Takayama, R. Frostig, J. Tan, C. Parada, and V. Sindhwani.  Learning model predictive controllers with real-time attention for real-world navigation.  In _Conference on robot learning_. PMLR, 2022.
  * Vu et al. [2008] Q.-H. Vu, B.-S. Kim, and J.-B. Song.  Autonomous stair climbing algorithm for a small four-tracked robot.  In _2008 International Conference on Control, Automation and Systems_ , pages 2356–2360. IEEE, 2008.
  * Fankhauser et al. [2018] P. Fankhauser, M. Bloesch, and M. Hutter.  Probabilistic terrain mapping for mobile robots with uncertain localization.  _IEEE Robotics and Automation Letters (RA-L)_ , 3(4):3019–3026, 2018.  doi:10.1109/LRA.2018.2849506.
  * Kumar et al. [2021] A. Kumar, Z. Fu, D. Pathak, and J. Malik.  Rma: Rapid motor adaptation for legged robots.  In _Robotics: Science and Systems_ , 2021.
  * Zheng et al. [2022] C. Zheng, S. Sane, K. Lee, V. Kalyanram, and K. Lee.  $\alpha$-waltr: Adaptive wheel-and-leg transformable robot for versatile multiterrain locomotion.  _IEEE Transactions on Robotics_ , 2022.
  * Xu et al. [2023] K. Xu, Y. Lu, L. Shi, J. Li, S. Wang, and T. Lei.  Whole-body stability control with high contact redundancy for wheel-legged hexapod robot driving over rough terrain.  _Mechanism and Machine Theory_ , 181:105199, 2023.
  * Wright et al. [2007] C. Wright, A. Johnson, A. Peck, Z. McCord, A. Naaktgeboren, P. Gianfortoni, M. Gonzalez-Rivero, R. Hatton, and H. Choset.  Design of a modular snake robot.  In _2007 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 2609–2614. IEEE, 2007.
  * Rollinson and Choset [2011] D. Rollinson and H. Choset.  Virtual chassis for snake robots.  In _2011 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 221–226. IEEE, 2011.
  * Xiao et al. [2022] X. Xiao, B. Liu, G. Warnell, and P. Stone.  Motion planning and control for mobile robot navigation using machine learning: a survey.  _Autonomous Robots_ , 46(5):569–597, 2022.
  * Sivaprakasam et al. [2021] M. Sivaprakasam, S. Triest, W. Wang, P. Yin, and S. Scherer.  Improving off-road planning techniques with learned costs from physical interactions.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 4844–4850. IEEE, 2021.
  * Karnan et al. [2022] H. Karnan, K. S. Sikand, P. Atreya, S. Rabiee, X. Xiao, G. Warnell, P. Stone, and J. Biswas.  Vi-ikd: High-speed accurate off-road navigation using learned visual-inertial inverse kinodynamics.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 3294–3301. IEEE, 2022.
  * Vasquez et al. [2014] D. Vasquez, B. Okal, and K. O. Arras.  Inverse reinforcement learning algorithms and features for robot navigation in crowds: an experimental comparison.  In _2014 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 1341–1346. IEEE, 2014.
  * Wigness et al. [2018] M. Wigness, J. G. Rogers, and L. E. Navarro-Serment.  Robot navigation from human demonstration: Learning control behaviors.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1150–1157. IEEE, 2018.
  * Xiao et al. [2020] X. Xiao, B. Liu, G. Warnell, J. Fink, and P. Stone.  Appld: Adaptive planner parameter learning from demonstration.  _IEEE Robotics and Automation Letters_ , 5(3):4541–4547, 2020.
  * Wang et al. [2021a] Z. Wang, X. Xiao, B. Liu, G. Warnell, and P. Stone.  Appli: Adaptive planner parameter learning from interventions.  In _2021 IEEE international conference on robotics and automation (ICRA)_ , pages 6079–6085. IEEE, 2021a.
  * Wang et al. [2021b] Z. Wang, X. Xiao, G. Warnell, and P. Stone.  Apple: Adaptive planner parameter learning from evaluative feedback.  _IEEE Robotics and Automation Letters_ , 6(4):7744–7749, 2021b.
  * Xu et al. [2021] Z. Xu, G. Dhamankar, A. Nair, X. Xiao, G. Warnell, B. Liu, Z. Wang, and P. Stone.  Applr: Adaptive planner parameter learning from reinforcement.  In _2021 IEEE international conference on robotics and automation (ICRA)_ , pages 6086–6092. IEEE, 2021.
  * Xiao et al. [2022] X. Xiao, Z. Wang, Z. Xu, B. Liu, G. Warnell, G. Dhamankar, A. Nair, and P. Stone.  Appl: Adaptive planner parameter learning.  _Robotics and Autonomous Systems_ , 154:104132, 2022.
  * Everett et al. [2019] M. Everett, J. Miller, and J. P. How.  Planning beyond the sensing horizon using a learned context.  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 1064–1071. IEEE, 2019.
  * Meng et al. [2023] X. Meng, N. Hatch, A. Lambert, A. Li, N. Wagener, M. Schmittle, J. Lee, W. Yuan, Z. Chen, S. Deng, et al.  Terrainnet: Visual modeling of complex terrain for high-speed, off-road navigation.  _arXiv preprint arXiv:2303.15771_ , 2023.
  * Pan et al. [2020] Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. A. Theodorou, and B. Boots.  Imitation learning for agile autonomous driving.  _The International Journal of Robotics Research_ , 39(2-3):286–302, 2020.
  * Faust et al. [2018] A. Faust, K. Oslund, O. Ramirez, A. Francis, L. Tapia, M. Fiser, and J. Davidson.  Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 5113–5120. IEEE, 2018.
  * Pfeiffer et al. [2017] M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart, and C. Cadena.  From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots.  In _IEEE International Conference on Robotics and Automation_. IEEE, 2017.
  * Bojarski et al. [2016] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, et al.  End to end learning for self-driving cars.  _arXiv preprint arXiv:1604.07316_ , 2016.
  * Karaman and Frazzoli [2010] S. Karaman and E. Frazzoli.  Incremental sampling-based algorithms for optimal motion planning.  _Robotics Science and Systems VI_ , 104(2), 2010.
  * Karaman et al. [2011] S. Karaman, M. R. Walter, A. Perez, E. Frazzoli, and S. Teller.  Anytime motion planning using the rrt.  In _2011 IEEE international conference on robotics and automation_ , pages 1478–1483. IEEE, 2011.
  * ROS [2023] ROS.  rtabmap_ros - ros wiki.  http://wiki.ros.org/rtabmap_ros, 2023.  Accessed: 2023-06-07.
  * Miki et al. [2022] T. Miki, L. Wellhausen, R. Grandia, F. Jenelten, T. Homberger, and M. Hutter.  Elevation mapping for locomotion and navigation using gpu, 2022.
  * Wellhausen and Hutter [2023] L. Wellhausen and M. Hutter.  Artplanner: Robust legged robot navigation in the field.  In _Field Robotics_ , 2023.
  * Mombaur et al. [2010] K. Mombaur, A. Truong, and J.-P. Laumond.  From human to humanoid locomotion—an inverse optimal control approach.  _Autonomous robots_ , 28:369–383, 2010.
  * Barto and Sutton [1995] A. G. Barto and R. S. Sutton.  Reinforcement learning.  _Handbook of brain theory and neural networks_ , pages 804–809, 1995\.
  * Williams et al. [2017] G. Williams, A. Aldrich, and E. A. Theodorou.  Model predictive path integral control: From theory to parallel computation.  _Journal of Guidance, Control, and Dynamics_ , 40(2):344–357, 2017.