#  CoverNav: Cover Following Navigation Planning in Unstructured Outdoor
Environment with Deep Reinforcement Learning

Jumman Hossain  Department of Information Systems
University Of Maryland, Baltimore County
<EMAIL_ADDRESS>Nirmalya Roy  Department of Information Systems
University Of Maryland, Baltimore County
<EMAIL_ADDRESS>Jumman Hossain1, Abu-Zaher Faridee1, Nirmalya Roy1, Anjan
Basak2, Derrik E. Asher2<EMAIL_ADDRESS><EMAIL_ADDRESS>1Department of Information Systems, University of Maryland, Baltimore County,
USA  2DEVCOM Army Research Lab, USA  1{jumman.hossain, faridee1,
<EMAIL_ADDRESS>
###### Abstract

Autonomous navigation in off-road environments has been extensively studied in
the robotics field. However, navigation in covert situations where an
autonomous vehicle needs to remain hidden from outside observers remains an
under-explored area. In this paper, we propose a novel Deep Reinforcement
Learning (DRL) based algorithm, called CoverNav, for identifying covert and
navigable trajectories with minimal cost in off-road terrains and jungle
environments in the presence of observers. CoverNav focuses on unmanned ground
vehicles seeking shelters and taking covers while safely navigating to a
predefined destination. Our proposed DRL method computes a local cost map that
helps distinguish which path will grant the maximal covertness while
maintaining a low-cost trajectory using an elevation map generated from 3D
point cloud data, the robot’s pose, and directed goal information. CoverNav
helps robot agents to learn the low-elevation terrain using a reward function
while penalizing it proportionately when it experiences high elevation. If an
observer is spotted, CoverNav enables the robot to select natural obstacles
(e.g., rocks, houses, disabled vehicles, trees, etc.) and use them as shelters
to hide behind. We evaluate CoverNav using the Unity simulation environment
and show that it guarantees dynamically feasible velocities in the terrain
when fed with an elevation map generated by another DRL-based navigation
algorithm. Additionally, we evaluate CoverNav’s effectiveness in achieving a
maximum goal distance of 12 meters and its success rate in different elevation
scenarios with and without cover objects. We observe competitive performance
comparable to state-of-the-art (SOTA) methods without compromising accuracy.

###### Index Terms:

deep reinforcement learning, simulated terrain environment, off-road path
planning, cover detection, semantic segmentation

##  I Introduction

Figure 1:  During navigation in unstructured outdoor environments, our
proposed CoverNav algorithm directs the Clearpath Husky robot towards nearby
obstacles that provide the highest cover. In simulation, the robot detects the
position of obstacles in its vicinity and moves towards them to find cover. If
there are multiple obstacles in the robot’s field of view, CoverNav
prioritizes the obstacle with the highest cover. In the accompanying image, we
show the robot navigating using our DRL-based method in blue, and state-of-
the-art navigation planning algorithms, TERP [1] in red and GA-Nav [2] in
yellow. In the image, the robot initially detects a bush as its cover, moves
towards it, and then changes direction when it no longer perceives any cover
in that direction. It quickly finds shelter between a truck and a cottage
until it reaches its destination.

Humans and animals can traverse challenging terrain and off-road environments
with ease to reach their destination. As the fields of human-autonomous agent
teaming and artificial intelligence progress, it becomes essential for robots
to showcase a similar level of adaptability, enabling them to navigate diverse
terrains effectively and become valuable companions in various scenarios.
Autonomous ground vehicles, particularly, have great potential as companions
to humans, seamlessly collaborating with or actively participating in tasks.
They can strategically position themselves in advantageous locations to
optimize operations and achieve objectives more efficiently. In order for
autonomous ground vehicles to be effective in various scenarios, they must be
able to navigate through complex terrains. These terrains include urban
environments with paved surfaces as well as off-road areas with natural
obstacles like forests, jungles, deserts, and uneven terrain with wet gaps.
Additionally, rural settings with boundary fences, walls, and sparse
structures pose unique challenges. In each scenario, various physical features
can offer sufficient terrain cover for autonomous ground vehicles to minimize
their visibility. This allows them to increase their chances of mission
success, especially in situations where it’s necessary to move the robot while
remaining hidden from observers. By utilizing these physical features
effectively, autonomous ground vehicles can enhance their ability to remain
concealed, reduce the risk of being detected, and improve the overall
effectiveness of their missions.

Al Marzouqi and Jarvis [3] conducted one of the earliest comprehensive surveys
on robotic covert path planning, outlining the various strategies and
techniques used in the field. Subsequently, Ninomiya et al. [4] presented a
planning framework that satisfies multiple spatial constraints imposed on the
path, such as staying behind a building, walking along walls, or avoiding the
line of sight of patrolling agents. As evident from these works, to achieve
effective covert path planning, UGVs must have an integrated navigation
planning system that allows them to adapt their movements based on the
environment and select routes that optimize cover and concealment.

In order to effectively utilize the terrain covers to keep unmanned ground
vehicles (UGVs) hidden, proper logic must be integrated into their navigation
planning system. UGVs must navigate the ground without direct supervision and
adapt their movements based on the environment instead of following a fixed
path. It is important for UGVs to select routes that optimize cover and
concealment as if they expect to be seen by an observer at any moment.
Developing such an effective off-road navigation planning method presents a
significant challenge: distinguishing safe and traversable regions for a
mobile robot while considering its properties, constraints, and terrain cover.
Many existing methods predominantly focus on environmental elements such as
surface properties and often overlook the dynamic constraints of the robot,
undermining the potential impacts on their interactions with the environment.
Semantic segmentation-based algorithms categorize various terrain surfaces by
learning their visual characteristics, such as color and texture [2, 5, 6, 7,
8, 9, 10, 11, 12, 13]. However, these algorithms often fail to consider the
dynamic constraints of the UGV, thereby neglecting the impact of these
constraints on the environmental interaction.

Deep Reinforcement Learning (DRL) is a promising approach that can potentially
address some of these limitations for navigation planning [14, 15] and UGV can
learn to make strategic decisions based on the environment’s current state to
maximize a reward function. DRL techniques have excelled in detecting unsafe
elevation changes, efficient trajectory planning, and guiding UGVs around
moving objects [16]. However, they often do not account for the crucial aspect
of minimizing the likelihood of being seen by an outside observer.
Incorporating these strategic considerations into the DRL framework is an
essential step toward creating more effective and practical UGVs for off-road
navigation.

In this work, we present a novel Deep Reinforcement Learning (DRL) based
method for identifying covert and navigable trajectories in off-road terrains
and jungle environments. Our approach focuses on seeking shelters and covers
while safely navigating to a predefined destination with the objective of
minimizing the visibility of the robot to potential observers. In the context
of this problem, cover refers to any physical object that can shield or
conceal the robot from detection by observers. Examples of such objects
include buildings, trees, hills, vehicles, and any other obstacles that can
obscure the robot’s view or hide it from the line of sight of an observer.

Our main contributions are summarized as the following:

Figure 2:  Overview of the CoverNav Navigation Planning System: The process
begins with Perception and Semantic Segmentation, where the environment is
sensed and objects are classified (block 1). This feeds into the Cover
Analysis and Selection module (block 2), which identifies potential cover
objects and selects the optimal one. The selected cover object is then used by
the Path Planning and Motion Control modules (block 3) to both generate a safe
and optimal trajectory towards the object and executes this trajectory through
the robot’s physical movements.

  * •

Designing a Novel Reward Function : Our proposed DRL network utilizes a novel
reward function to effectively learn optimal off-road terrain features with
covers in simulated environments. By rewarding the agent for moving towards
the objective, penalizing for deviating from it, promoting low altitude,
encouraging proximity to cover, and maintaining stability, our approach is
able to learn quickly and efficiently. This results in the ability to plan
reliable and steady trajectories while maintaining a safe distance from
obstacles in order to seek out covers.

  * •

Designing a Novel Cover Detection Algorithm : We present a novel cover
detection algorithm that leverages state-of-the-art semantic segmentation
techniques to precisely identify and locate cover objects in unstructured
outdoor environments. By exploiting the rich semantic information provided by
segmentation, our algorithm can accurately identify and localize cover
objects, providing the necessary information to the robot to take appropriate
actions for seeking shelter when needed. The proposed approach thus enhances
the safety and robustness of the robot navigation system in challenging
outdoor environments.

  * •

Integrated Cover-Following Navigation Planning Approach: We propose an
integrated approach that combines Terrain Elevation-based Reliable Path
Planning (TERP) [1] and Dynamic Window Approach-based Reinforcement Learning
(DWA-RL) [17] with our cover detection algorithm to enhance the navigation of
a robot in an unstructured outdoor environment. The TERP method provides
reliable and stable path planning for the robot, while the DWA-RL approach
facilitates navigation in an environment with mobile obstacles. The
integration of these methods enables the robot to locate cover and shelter,
navigate safely (i.e., maintain a stable trajectory and move without being
detected by potential threats) to a predetermined destination, and avoid
dynamic obstacles. This combined approach provides a more robust and efficient
navigation strategy for the robot in complex outdoor environments.

  * •

Experimenting in Various Simulated Unstructured Environments: In order to
evaluate the effectiveness of our proposed approach, we conducted extensive
simulations using a Clearpath Husky robot in various unstructured
environments, such as off-road terrains and jungles with varying levels of
vegetation density. CoverNav exhibits competitive performance in success rate
and trajectory length, comparable to state-of-the-art (SOTA) methods while
maintaining high accuracy. We also tested our strategy in a realistic forest
environment with diverse terrain features, extensive areas of cover, and
different elevations. Our experiments demonstrate that _our approach can
successfully achieve the highest possible goal distance of 12 meters and high
success rates_ (i.e., the percentage of trials in which the robot successfully
completes the specified goal distance) in navigating through terrains with
varying degrees of elevation and cover. These results provide valuable
insights into the potential real-world performance of our approach.

The rest of the article is organized in the following structure. Section II
covers related prior works in off-road path planning. Section III provides a
brief overview of the architecture, the background information as well as the
proposed methodology. A comprehensive analysis of the experiments and their
outcomes can be found in section IV. In the following section V, there is a
brief discussion about the limitations of our approach. In the final Section
(VI), we summarize our findings and discuss potential future directions.

##  II Related Work

In this section, we discuss existing work on robot navigation in unstructured
outdoor environments.

###  II-A Semantic Segmentation for Off-Road Surface Traversability

In many scenarios, a robot encounters unstructured outdoor terrains with
different levels of navigability. Many of these terrain classes have extremely
similar visual features and overlaps that can lead to misclassification and
result in dangerous navigation. Several recent works have addressed these
challenges of off-road navigation by proposing novel methods with semantic
segmentation [18, 19]. GA-Nav [2] is a learning-based semantic segmentation
method that identifies different terrain groups in off-road and unstructured
outdoor terrains. It balances segmentation accuracy and computational overhead
and outperforms state-of-the-art segmentation methods by using coarse-grained
segmentation. For instance, it recognizes trees and poles simply as obstacles
instead of categorizing them as individual classes. In the context of our
work, we utilize the potential of the semantic segmentation method to
effectively detect and classify cover objects in the environment.

###  II-B Deep Reinforcement Learning Based Off-road Navigation

In recent studies, Deep Reinforcement Learning (DRL) has been utilized to
improve navigation strategies in complex, off-road environments. Taking the
challenge of uncertainty, Zhang et al. [14] demonstrated the potential of DRL
for navigating environments with unknown rough terrain. Wiberg et al. [20]
proposed a DRL approach for vehicle control in rough terrains, concentrating
on developing an effective mechanism to handle complex and uneven surfaces.
Similarly, Josef and Degani [21] presented a DRL method for safe local
planning of a ground vehicle in unknown rough terrain, underscoring the
necessity for robust navigation in the face of unpredictability and hazards
associated with rough terrains. Further expanding the DRL framework, Zhou et
al. [15] introduced a multimodal approach to navigating rough terrain,
reinforcing the efficacy of integrating multiple strategies within a DRL
framework for enhanced navigation robustness. Another recent DRL-based method,
TERP [1] is specifically designed to facilitate robust navigation in uneven
outdoor terrains. TERP excels in identifying unsafe regions characterized by
high elevation gradients and in calculating local least-cost waypoints to
circumvent the untraversable areas.

While existing techniques have achieved considerable advancements in
autonomous navigation, one important aspect that has been overlooked is the
ability to find places for hiding or taking cover in outdoor terrains. This
crucial feature is particularly relevant in military scenarios, where
maintaining concealment from potential observers is paramount. In our method,
we propose a novel approach to incorporate the feature of cover-seeking into
the DRL-based navigation of unmanned ground vehicles in off-road terrains. Our
approach represents a promising step towards enabling more robust, and
intelligent navigation in complex environments.

##  III Methodology

In this section, we detail the building blocks of CoverNav; we first start
with a very high-level overview of the primary modules and progressively
provide details on where our contributions (the novel reward function, the
cover detection algorithm, the integrated cover following navigation planning)
fit in.

###  III-A Overview of CoverNav Navigation Planning

The CoverNav navigation planning system (Figure 2) comprises three
interdependent modules that enable a mobile robot to navigate safely through
unstructured outdoor environments in search of cover while avoiding obstacles
and remaining undetected. The details of the modules are articulated below:

####  III-A1 Perception and Semantic Segmentation

This module receives and processes image and point cloud sensor data from the
environment. We first infer the robot’s pose, twist, and elevation map from
this data. Semantic segmentation techniques are applied to raw sensor data to
detect and classify objects in the environment.

####  III-A2 Cover Analysis and Selection

As the semantic segmentation data is passed to this module, we utilize our
novel cover detection Algorithm (described later Section III-B2) to identify
potential cover providing natural obstacles such as trees, rocks, bushes, etc.
This process takes into account the distance of objects from the robot, as
well as the potential for collisions and the amount of cover provided. Once
potential cover objects are identified, this module calculates the 3D
Euclidean distance between the robot and each potential cover object. It then
applies predefined selection criteria, including factors such as distance, and
visibility, to determine the most suitable cover object.

####  III-A3 Path Planning and Motion Control

This module integrates the path planning and motion control functionalities to
facilitate effective navigation of the robot. In this module, we combine TERP
[1] (a Deep Deterministic Policy Gradient (DDPG) [22] algorithm) with our
novel reward function to generate a trajectory that guides the robot towards
the selected cover object, while avoiding obstacles. The computed trajectory
takes into consideration the robot’s current position, the cover object’s
location, and the surrounding environment. The goal is to determine a safe and
optimal trajectory that maximizes cover, minimizes visibility, and assures
obstacle avoidance. This trajectory is dynamically updated in response to
changes in the environment or the detection people within the environment. The
motion control aspect utilizes the DWA-RL [17] algorithm, which combines the
strengths of the Dynamic Window Approach (DWA) [23, 24] and Deep Reinforcement
Learning (DRL) methods. It determines dynamically feasible velocities that
enable the robot to adhere to the trajectory while ensuring collision
avoidance and maintaining stability.

These three modules work together to enable the robot to autonomously and
intelligently navigate towards its goal, maximizing safety, and minimizing the
risk of detection. The system architecture is shown in Fig. 3, where the
inputs from the sensors are processed through different modules to produce the
output of dynamically feasible velocities for the robot to follow. The cover
detection and reward function modules use semantic segmentation information to
identify potential covers for the robot to use, and the reward function
provides feedback to the DRL network for optimizing the cover-following
velocities. The elevation cost map provides information about the terrain and
helps the robot avoid elevation gains that may make it more visible.

Figure 3:  Overview of CoverNav System Architecture.

###  III-B Problem Formulation

Navigating a UGV in unstructured outdoor environments with covers poses a
challenging problem due to the need for obstacle avoidance, cover detection,
and reliable path planning. The objective is for the robot to approach a
designated goal location, while ensuring obstacle avoidance, pose stability,
and maintaining proximity to available covers. In order to effectively tackle
the aforementioned challenge, we must account for various scenarios and
uncertainties. Therefore, we formulate our problem as a Markov decision
process (MDP). The MDP formulation allows us to model the environment, the
robot’s actions, and the resulting outcomes in a probabilistic manner,
enabling us to develop an optimal policy that maximizes the expected reward
over time. Our MDP consisting of a tuple $(S,A,P,R,\gamma)$, where:

  * •

$S$ is the state space representing the robot’s state. It includes the robot’s
position and orientation, as well as the presence and location of obstacles
and covers in the environment.

  * •

$A$ is the action space representing the robot’s actions. It includes the
robot’s movement in different directions and its rotation.

  * •

$P$ is the transition function representing the probability of moving from one
state to another after taking an action.

  * •

$R$ is the reward function representing the immediate reward the robot
receives after taking an action.

  * •

$\gamma$ is the discount factor representing the importance of future rewards.

Our objective is to find a policy $\pi(s)$ that maps a state $s$ to action
$a$, such that the expected cumulative reward of the robot is maximized.

####  III-B1 Reward Function Design

Our reward function is designed to encourage desirable robot behaviors and
discourage undesirable ones. It includes five components:

$r_{goal}$:

This term measures progress towards the goal by subtracting the current
distance from the previous distance. The closer the robot gets to the goal,
the higher the reward. It is calculated as the difference between the distance
to the goal in the previous state ($d_{t-1}$) and the distance to the goal in
the current state ($d_{t}$). A positive reward is assigned if the robot moves
closer to the goal, indicating progress, while a negative reward is assigned
if the robot moves farther away.

$r_{\text{goal}}=d_{t-1}-d_{t}$ (1)

$r_{dir}$:

This reward component encourages the robot to maintain a consistent heading
toward the goal. It is calculated as the absolute difference between the
heading in the current state ($\theta_{t}$) and the heading in the previous
state ($\theta_{t-1}$). A smaller difference indicates a more consistent
heading, resulting in a higher reward.

$r_{\text{dir}}=-|\theta_{t}-\theta_{t-1}|$ (2)

$r_{stab}$:

This reward component evaluates the stability of the robot based on its roll
and pitch angles ($\psi_{t}$). The reward is calculated using an exponential
function, where the stability reward decreases as the roll and pitch angles
deviate from zero. Smaller values of $\psi_{t}$ result in higher stability
rewards.

$r_{\text{stab}}=\exp(-\psi_{t}^{2})$ (3)

$r_{elev}$:

This term encourages the robot to move towards areas of higher elevation by
penalizing deviations from the target elevation. It is computed as the sum of
absolute differences between the current and target elevations, weighted by a
vector of elevation weights. Mathematically, it can be represented as:

$r_{elev}=\sum_{i=1}^{n}w_{elev}\Delta h_{i}$ (4)

where $n$ is the number of previous positions, $\Delta h_{i}$ is the
difference in elevation between the current position and the $i^{th}$ previous
position, and $w_{elev}$ is the weight given to the elevation component. The
calculation of $\Delta h_{i}$ depends on the method used to obtain elevation
information. In this work, the elevation information is obtained from the
Unity terrain data. $\Delta h_{i}$ is calculated as the difference between the
elevation at the current position and the $i^{th}$ previous position, given
by:

$\Delta h_{i}=h_{cur}-h_{i}$ (5)

where $h_{cur}$ is the elevation at the current position and $h_{i}$ is the
elevation at the $i^{th}$ previous position.

$r_{cover}$:

This term encourages the robot to seek cover objects to hide behind or
beneath. It is computed as follows:

$\displaystyle r_{\text{cover}}=\begin{cases}0,&\text{if
}d_{\text{coverObject}}>1.5w_{\text{min}}\\\
d_{\text{coverObject}}-0.5w_{\text{min}},&\text{if }0.5w_{\text{min}}\leq
d_{\text{coverObject}}\leq 1.5w_{\text{min}}\\\ -1000,&\text{if
}d_{\text{coverObject}}<0.5w_{\text{min}}\end{cases}$ (6)

It is calculated based on the distance to the nearest cover object,
represented by $d_{\text{coverObject}}$, and the shortest external dimension
of the robot, denoted as $w_{\text{min}}$. If the distance to the nearest
cover object ($d_{\text{coverObject}}$) is greater than 1.5 times the shortest
external dimension of the robot, then the reward is 0. If the distance to the
nearest cover object is between 0.5 and 1.5 times the shortest external
dimension of the robot, then the reward is the distance minus half the
shortest external dimension of the robot. If the distance to the nearest cover
object is less than 0.5 times the shortest external dimension of the robot,
then the reward is -1000. The reward function is designed to encourage the
robot to move towards the goal while avoiding obstacles and seeking cover. The
different terms provide incentives for different behaviors, such as moving in
a straight line, maintaining stability, and seeking higher ground. The
weighting of the terms can be adjusted to optimize the performance of the
robot.

The reward at current state $R_{t}$ is the sum of these five components,
summarized at the end of Table I.

$d_{t}$ |  | distance from robot to goal at current state
---|---|---
$d_{t-1}$ |  | distance from robot to goal at previous state
$\theta_{t}$ |  | heading of the robot at current state
$\theta_{t-1}$ |  | heading of the robot at previous state
$\psi_{t}$ |  | roll and pitch angles of the robot at current state
$w$ |  | weight vector for elevation changes
$d_{\text{coverObject}}$ |  | distance to the nearest cover object
$w_{min}$ |  | shortest external dimension of the robot
$r_{goal}$ | = | $d_{t-1}-d_{t}$
$r_{dir}$ | = | $-|\theta_{t}-\theta_{t-1}|$
$r_{stab}$ | = | $\exp(-\psi_{t}^{2})$
$r_{elev}$ | = | $\sum_{i}w_{e}lev|\Delta h_{i}|$
$r_{cover}$ | = | $\tiny\begin{cases}0,&\text{if }d_{\text{coverObject}}>1.5w_{\text{min}}\\\ d_{\text{coverObject}}-0.5w_{\text{min}},&\text{if }0.5w_{\text{min}}\leq d_{\text{coverObject}}\leq 1.5w_{\text{min}}\\\ -1000,&\text{if }d_{\text{coverObject}}<0.5w_{\text{min}}\end{cases}$
$\bm{R_{t}}$ | = | $\bm{r_{goal}+r_{dir}+r_{stab}+r_{elev}+r_{cover}}$
TABLE I: Summary of Reward Function Components

####  III-B2 Cover Detection

To accurately locate and label objects, we utilize the ARL Unity Framework’s
semantic segmentation framework. This algorithm analyzes the scene and assigns
relevant class labels to each pixel in the image. The resulting segmentation
map provides valuable information about the different objects present,
including trees, buildings, rocks, and grass (refer to Listing 1 for an
example). However, to specifically identify objects that can serve as cover
for the robot, we apply additional criteria to the semantic labels. We focus
on objects that have the potential to offer protection and concealment, such
as buildings, trees, junk cars, etc. By narrowing down the selection to these
cover objects, we ensure that our system targets the most suitable options for
the robot’s safety and strategic positioning. Once we have identified the
desired cover object, we proceed to calculate the distance between the robot’s
position and the cover object in a 3-dimensional space. This distance metric
provides crucial information about the proximity of the cover and helps the
robot in making informed decisions during navigation and planning.

To calculate the distance, we use the 3D Euclidean distance formula. In the
context of the reward function, this formula is used to calculate the distance
between the robot and a cover object in 3D space, as shown in Equation (7):

$\displaystyle d_{\text{coverObject}}$
$\displaystyle=\sqrt{\big{(}}(x_{\text{robot}}-x_{\text{cover}})^{2}$ (7)
$\displaystyle+(y_{\text{robot}}-y_{\text{cover}})^{2}+(z_{\text{robot}}-z_{\text{cover}})^{2}\big{)}$

where $d_{\text{coverObject}}$ represents the distance between the robot and
the cover object, and $(x_{\text{robot}},y_{\text{robot}},z_{\text{robot}})$
and $(x_{\text{cover}},y_{\text{cover}},z_{\text{cover}})$ represent the 3D
coordinates of the robot and the cover object, respectively. If the system is
able to recognize any of the relevant cover categories, such as trees, bushes,
or rocks, they are passed on to Algorithm 1.

⬇

frame_id: 991904

detections:

class_id: 0

class_name: ”Rock”

confidence: 1.0

x_min: 151.64044189453125

y_min: 118.20899963378906

x_max: 168.4332275390625

y_max: 112.97318267822266

object_id: 68

x_pos: -0.18053817749023438

y_pos: -0.46726706624031067

z_pos: 18.7108097076416

kpts: []

Listing 1: Detected Object Information

Given the details of detected objects, such as object class, object location,
robot location, and object detection confidence, Algorithm 1 determines
whether the robot is currently in cover or not. Algorithm 1 operates by
iterating through each detected object and computing the distance between the
object and the robot. The algorithm considers objects that belong to classes
such as trees, bushes, rocks, etc., and has a detection confidence score of at
least 0.85. The confidence threshold of 0.85 ensures that only objects with a
sufficiently high detection confidence are considered for cover detection. The
algorithm stores the minimum distance among these objects in the variable
coverDistance. If the minimum distance is less than or equal to 10 meters, the
algorithm returns True, indicating that the robot is currently in cover.
Otherwise, the algorithm returns False, indicating that the robot is not in
cover. The cover detection algorithm offers a time complexity of O(N), where N
is the number of detected objects. The algorithm efficiently iterates through
the detected objects, computes distances, and determines if the robot is in
cover based on specified thresholds. The chosen threshold values, such as
detection confidence and distance, influence the object selection and cover
determination process without affecting the algorithm’s time complexity.

Input: $objC$ $\triangleright$ Object Classes

Input: $objL$ $\triangleright$ Object Locations

Input: $rL$ $\triangleright$ Robot Location

Input: $cf$ $\triangleright$ Confidence Values

Output: $isCover$ $\triangleright$ Cover or not cover

1 Function  _DetectCover(_$objC,objL,rL,cf$_)_:

2 $coverDistance\longleftarrow\infty$;

3 $isCover\longleftarrow False$

4 $N\longleftarrow length(objC)$

5 foreach  _$i\in N$_ do

6 if  _ $objC_{i}\in\\{trees,bushes,rocks,cottage,building,houses,\newline
disabled~{}vehicles\\}$ and $cf_{i}\geq 0.85$_ then

7 $dist_{i}\longleftarrow||objL_{i}-rL||$

8 if  _$dist_{i} <coverDistance$_ then

9 $coverDistance\longleftarrow dist_{i}$

10

11 end if

12

13 end if

14

15 end foreach

16 if  _$coverDistance\leq 10$_ then

17 $isCover\longleftarrow True$

18

19 end if

20 return $isCover;$

21

Algorithm 1 Algorithm for Detecting Cover

####  III-B3 Dynamic Window Approach with Deep Reinforcement Learning

The Dynamic Window Approach (DWA) [23, 24] is a well-known navigation
algorithm that takes into account the dynamic constraints of a robot and
ensures collision-free and feasible velocities within a given time interval.
It operates within a 2-dimensional velocity space known as the dynamic window,
which contains a discrete set of velocity vectors that the robot can achieve
in the next time interval. The DWA algorithm consists of two stages to
determine a suitable velocity vector. Firstly, it evaluates the collision-free
velocities within the dynamic window by considering the robot’s dynamic
constraints. This stage ensures that the selected velocities are within the
robot’s capability to execute safely. However, the DWA algorithm has
limitations when it comes to navigating in the presence of mobile obstacles.
It relies solely on the robot’s immediate sensor data, which may not provide
enough time for the robot to react to changes in the environment. As a result,
it may struggle to effectively avoid collisions with moving obstacles.

To overcome this problem, we employ a recent work [17] that combines the
advantages of DWA and DRL-based methods for navigation in dynamic environments
where mobile obstacles exist. This method functions as our navigation module,
which is advantageous to our method in a number of different respects. First,
it contains a reward function that is shaped such that the robot’s navigation
is more spatially aware of the obstacles’ motion. That is, the robot is
rewarded for navigating in the direction opposite to the heading direction of
obstacles, which leads to the robot taking maneuvers around moving obstacles.
Second, using our DRL agent in conjunction with a high-level path planner like
this can reduce the complexity of the DRL problem and speed up the training
process. That means, calculating the robot’s feasible velocity set at a
particular time instant and the costs that correspond to making use of that
robot’s velocity over the course of the previous n time instants is how the
observation space is built up in DWA-RL [17]. This formulation maintains the
dynamic feasibility guarantees provided by DWA while also incorporating the
time-dependent change in the state of the environment. In contrast to the
outcomes of other DRL methods, this one results in a significantly reduced
dimension of the observation space.

###  III-C Training Strategy

The training strategy we employ is a combination of exploration with a DRL
agent and exploitation of DWA’s dynamic feasibility guarantees. In the
beginning, the DRL agent explores the environment, using its policy to
determine the robot’s action. Exploration is done using random actions to
enable the agent to learn the optimal policy. Once a sufficient number of
episodes have been run, the DRL agent is tested with DWA. During testing,
DWA’s dynamic feasibility guarantees ensure that the robot does not collide
with obstacles while moving towards the goal.

The learning rate for both actor and critic networks was set to $10^{-4}$. The
batch size was set to 64, and the replay buffer size was set to 100,000. We
use the Adam optimizer with a decay rate of 0.9 and 0.999 for the first and
second moments, respectively. The discount factor $\gamma$ was set to 0.99,
and the exploration noise scale $\sigma$ was set to 0.1.

The training process consists of a total of 100 episodes. Each episode lasts
for 100 time-steps, during which the robot navigates through the environment,
avoiding obstacles and maximizing cover while minimizing visibility. At the
beginning of each episode, the robot’s initial position is randomized within
the start zone. During each time step, the robot receives an observation from
the environment, and the DRL agent selects an action based on the observation.
The action is then executed in the environment, and the robot’s new state is
observed, along with the reward. The process repeats until the episode ends,
either because the robot has reached the goal or has exceeded the time limit.
We made the decision to stop the training at episode 100 after careful
analysis and consideration of various factors. By episode 100, the agent had
already shown significant progress and achieved competitive performance.
Further training beyond this point did not yield substantial improvements in
terms of success rate and trajectory length. Considering the computational
resources required for continued training, we determined episode 100 as an
appropriate stopping point.

The reward function is designed to encourage the robot to move towards the
goal while maximizing cover and minimizing visibility. The maximum cover and
minimum visibility that can be obtained during the episode are tracked, and
the final reward is normalized by these values. This normalization encourages
the robot to achieve maximum cover and minimum visibility during each episode.

##  IV Experiments and Results

We explain the implementation of our technique as well as the experimental
details on a Husky robot in the ARL unity simulation environment.

###  IV-A Implementation Details

The model is tested in the ARL Unity Simulation [25], a realistic outdoor
environment that has a broad range of ground textures and elevation levels, as
well as buildings, vegetation, lake, etc. Our DRL network is implemented in
PyTorch. We use simulated uneven terrains with a Clearpath Husky robot created
using ROS Melodic [26] and ARL Unity Simulation framework to train the DRL
network. The simulated Husky robot is mounted with a Velodyne VLP16 3D LiDAR.
The network is trained in a workstation with a 10th Generation Intel Core
i9-10850K processor and an NVIDIA GeForce RTX 3090 GPU.

We utilize the existing semantic segmentation framework from ARL Unity
simulation and apply some custom labeling to it. The custom labeling includes
ignoring small objects that cannot provide cover for the robot, which has been
done to serve the specific purposes of our experiment. We use the Octomap [27]
and Grid-map [28] ROS packages to obtain the elevation map using the point
clouds from the Velodyne ROS package. The computational load of our network is
low enough that it can function effectively in real-time on the laptop with
the ROS tools that were discussed earlier.

###  IV-B Experiment Details

The following details provide insights into our experimental setup.

Number of Training Episodes: We conduct extensive training sessions consisting
of 100 episodes to ensure thorough exploration and learning. Each episode
represents a complete navigation mission, allowing the robot to learn and
adapt to various environmental conditions. We empirically found that 100 is an
adequate number of episodes to ensure robust learning and the acquisition of
diverse navigation strategies.

Destination Placement: The destinations for the navigation missions are
strategically set within a radius of 12 meters from the robot’s initial
location. This range provides a challenging yet realistic scenario for the
robot to navigate to different targets. It enables us to assess the robot’s
capability to explore and reach destinations in various environmental
conditions, including terrain variations and obstacle configurations.

Reward Scores: The reward scores obtained during the training sessions serve
as a metric to evaluate the performance improvement of our model over time. We
monitor the average reward score throughout the training process. The gradual
increase in reward scores indicates the learning progress and the
effectiveness of our method in achieving navigation goals. Higher reward
scores demonstrate improved navigation capabilities and successful adherence
to cover-following strategies.

###  IV-C Baselines

We compare the performance of our algorithm against the following navigation
strategies:

  * •

DWA [23]: A popular algorithm that focuses on real-time obstacle avoidance and
trajectory planning. It calculates feasible velocities based on the current
robot state and dynamically selects the best velocity combination to avoid
collisions and reach the goal.

  * •

GA-Nav [2]: A learning-based semantic segmentation method that aims to
identify different terrain groups in unstructured and outdoor environments. It
has shown promising results in accurately classifying terrain surfaces and
improving navigation performance in complex terrains.

  * •

TERP [1]: A DRL-based planner specifically designed for navigating uneven
outdoor terrains. It incorporates terrain elevation information to identify
unsafe regions and computes local least-cost waypoints to avoid traversing
those areas.

###  IV-D Evaluation Metrics

Due to fundamental differences between some of the baselines and CoverNav
(e.g., neither DWA and GA-Nav are DRL-based methods), direct comparison of
reward scores between CoverNav and the baseline methods is not meaningful.
Instead, we focus on evaluating navigation performance using the following
metrics for a fair comparison.

Success Rate: The ratio (in percentage) between the number of times the robot
successfully accomplished its objective and the number of total tries, while
avoiding obstacles, high elevations, going through cover, and low elevations.

(a)

(b)

(c)

(d)

Figure 4: A sequential overview of the robot’s process in finding shelter. (a)
The robot initially encounters an obstacle along its intended path. (b) It
navigates around the obstacle while still heading towards the goal location.
(c) The robot identifies and selects a strategic position near the goal that
offers sufficient cover or shelter. (d) Finally, the robot successfully
reaches the goal location, utilizing the nearby cottage as a shelter or cover
during its navigation. The step-by-step visualization demonstrates the robot’s
adaptive behavior in identifying and utilizing available shelter to accomplish
its objectives.

Execution time: The time (in seconds) taken by the robot to complete a
navigation task. Shorter execution times indicate a more efficient and
responsive navigation system, making it suitable for time-sensitive
applications. Optimizing execution time is crucial for practical and effective
navigation in real-world scenarios.

Trajectory length: The distance (in meters) traveled by the robot to reach its
destination. This metric provides insights into the efficiency of navigation.
Shorter trajectories indicate more efficient navigation, while longer
trajectories may suggest suboptimal paths or difficulty navigating in a
complex environment.

###  IV-E Testing Scenarios

We evaluate our CoverNav navigation in different environment scenarios,
including normal elevation, low-high elevation, and forest and jungle
environments.

  * •

Normal Elevation: This refers to a terrain with a relatively flat or gently
sloping surface.

  * •

Low Elevation: The terrain has a normal height and features such as buildings,
fences, and other minor impediments. The maximum elevation ($\leq$ 1m).

  * •

Low-High Elevation: The terrain includes both high and low elevations. It has
elevations ranging from 1m to 3m.

  * •

Forest and Jungle: A realistic forest environment that includes diverse
terrain and extensive areas of cover. The diverse terrain includes grass, an
open hiking path, water, and some hills[29]. The maximum elevation $\geq$ 4m.

###  IV-F Results

In this section, we analyze the navigation performance of CoverNav in various
testing scenarios mentioned in the previous section. We assess CoverNav’s
ability to navigate through different terrains, overcome obstacles, and locate
suitable shelter spots. Finally, we compare the navigation performance
(according to three metrics from Section IV-D) with the three baselines
discussed in Section IV-C.

####  IV-F1 Navigation Performance

Fig. 4 details the methodical process of the robot attempting to locate a
place to stay while using a house as shelter. It can be observed from Fig. 5
that the agent consistently improves its performance across all scenarios as
the number of episodes progresses, demonstrating effective learning. In the
normal elevation scenario, the robot demonstrated successful navigation
through the environment, efficiently locating a suitable shelter spot without
encountering significant challenges. The relatively even terrain allowed for
smooth traversal and effective decision-making. In the low elevation scenario,
which includes features such as buildings, fences, and other minor
impediments, the robot successfully navigated while taking into account these
obstacles. It showcased its ability to identify and utilize cover in the
presence of such environmental elements. In the low-high elevation scenario,
the terrain presented a mix of high and low elevations, posing increased
complexity for the robot’s navigation. Despite the challenges, the robot was
able to adapt and successfully locate shelter locations. However, the learning
curve in this scenario was steeper compared to the normal elevation scenario,
requiring the robot to refine its strategies for navigating uneven terrains.
In the forest and jungle environment, characterized by dense foliage and
varied terrain, the robot faced the most challenging conditions. Navigating
through the intricate landscape, avoiding obstacles, and identifying suitable
shelter spots proved to be challenging tasks. The robot encountered
difficulties such as getting stuck or colliding with obstacles in this
scenario. However, as the training progressed, the robot gradually improved
its navigation skills and adapted to the complex terrain. The learning curve
showed a positive trend, indicating the robot’s ability to learn and make
progress in navigating through the forest environment over time.

Figure 5: Performance comparison of CoverNav under four scenarios: (a) Normal
Elevation, (b) Low Elevation, (c) Low-High Elevation, (d) Forest and Jungle

####  IV-F2 Comparison with Baselines

Table II provides a comparison of the performance metrics of the DWA, GA-Nav,
TERP, and CoverNav methods for robot navigation in various terrains. CoverNav
demonstrates competitive performance in terms of goal-reaching and obstacle
avoidance, with success rates similar to GA-Nav and TERP. However, it is worth
noting that the execution time for CoverNav is longer compared to the other
methods, attributed to the more complex algorithms used for cover detection
and following. Despite the slightly longer execution time, the superior
performance of CoverNav in success rate and trajectory length makes it a
promising method for robot navigation in various terrains, particularly in
scenarios where cover following is of utmost importance

Metrics | Method |  | Normal
---
Elevation
| Low
---
Elevation
| Low-High
---
Elevation
| Forest
---
and
Jungle
Success Rate (%) | DWA | 72 | 82 | 67 | X
GA-Nav | 79 | 83 | 71 | X
TERP | 82 | 83 | 69 | X
CoverNav (Ours) | 81 | 82 | 70 | 86
Trajectory Length (m) | DWA | 5 | 4 | 6 | X
GA-Nav | 6 | 5 | 7 | X
TERP | 9 | 5 | 7 | X
CoverNav (Ours) | 9 | 8 | 7 | 12
Execution Time (s) | DWA | 12 | 14 | 17 | X
GA-Nav | 14 | 13 | 16 | X
TERP | 10 | 12 | 20 | X
CoverNav (Ours) | 15 | 17 | 22 | 25

TABLE II: Performance Metrics Comparison of DWA, GA-Nav, TERP, and CoverNav
(Ours) Methods for Robot Navigation in Various Terrains.

##  V Discussion

In this work, we presented a navigation strategy that effectively navigates
outdoor environments while following covers. Our approach demonstrates a
significant increase in success rate with the number of episodes played,
making it capable of successfully navigating through a variety of
environments, including normal elevations, low-high elevations, and
forest/jungle environments.

Furthermore, our analysis highlights the significance of the training episode
length, complexity, and the distance of randomly generated goals in
determining the performance of our proposed strategy. We observed that when
longer-distanced goals, such as those set at 50 meters, were assigned, the
robot exhibited instability, making it challenging to accomplish the
exploration objective. This instability was evident as the robot continued to
collide even after a new goal was generated, leading to a persistent negative
reward signal and a failure to converge. To address this issue, several
strategies can be employed. Firstly, adjusting the reward function or fine-
tuning the training process can provide clearer guidance to the learning
agent, encouraging it to prioritize collision avoidance and convergence. By
emphasizing the importance of avoiding collisions and rewarding successful
navigation, the learning agent can better understand the desired behavior.

In addition, modifying the training process to gradually increase the
difficulty of the assigned goals can be beneficial. Starting with shorter
distances and simpler environments allows the agent to gradually develop the
necessary skills and strategies for navigating challenging environments
without succumbing to instability. This progressive approach enables the agent
to acquire robust and adaptive navigation capabilities, enhancing its
performance in longer-distanced goal scenarios.

It is important to strike a balance between training difficulty and stability
to ensure effective learning. By carefully designing the training curriculum,
gradually introducing more complex scenarios, and providing adequate training
resources, we can foster the development of a resilient and proficient
navigation policy that can handle a wide range of goal distances and
environmental complexities.

Moreover, the current implementation of the navigation strategy is performed
offline, and there is a need for further quantitative validation to assess its
real-time performance in unpredictable jungle environments. While evaluations
were conducted in emulated environments using Unity, which provides useful
insights, it is essential to validate the strategy in real-world scenarios to
ensure its effectiveness. A small-scale validation in a real-world setup would
provide valuable evidence of its applicability and help bridge the gap between
simulated and real-world use cases.

##  VI Conclusion and Future Work

We introduced a novel DRL-based method for guiding a robot in unstructured
terrain environments, prioritizing maximum cover to enhance its safety and
covert capabilities. Our approach enables the robot to identify and utilize
natural shelters such as trees and bushes as hiding spots from potential
observer agents. Although our method may result in longer trajectories
compared to some existing approaches, it provides the distinct advantage of
actively seeking and utilizing cover, which is crucial in contested
environments.

In our future work, we plan to enhance the obstacle and cover detection
process by incorporating additional factors such as geometry and dynamic
reward systems. Additionally, we intend to leverage RGB images from large-
scale datasets to train our model specifically for real off-road unstructured
terrain environments. This will help address challenges such as detecting
small objects within grassy areas. Furthermore, we intend to investigate the
integration of imitation learning and deep reinforcement learning techniques
to accelerate the initial training phase. We can expedite the learning process
and improve the convergence of our navigation model by utilizing pre-existing
expert demonstrations. To facilitate this research, we will utilize RGB images
and robot odometry data from the TartanDrive dataset [30], a comprehensive and
large-scale off-road driving dataset. By training our model on this rich
dataset, we aim to improve its ability to recognize different terrain surfaces
and generate more stable and adaptive trajectories.

By advancing these research directions, we aim to enhance the practicality and
effectiveness of our navigation method in real-world scenarios. This will open
up opportunities for its deployment in various applications, including
autonomous vehicles, and military operations.

## Acknowledgment

This work has been partially supported by U.S. Army Grant #W911NF2120076, ONR
Grant #N00014-23-1-2119, NSF CAREER Award #1750936, NSF REU Site Grant
#2050999 and NSF CNS EAGER Grant #2233879.

The authors would also like to thank Avijoy Chakma, Zahid Hasan, and Dr.
Anuradha Ravi for their constructive feedback on this work, Arthur C. Schang
for setting up the Semantic Segmentation in ARL Unity Environment, and special
thanks to Wanying Zhu for conducting the initial experiments.

## References

  * [1] Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Utsav Patel, and Dinesh Manocha.  Terp: Reliable planning in uneven outdoor environments using deep reinforcement learning.  In 2022 International Conference on Robotics and Automation (ICRA), pages 9447–9453. IEEE, 2022.
  * [2] Tianrui Guan, Divya Kothandaraman, Rohan Chandra, Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, and Dinesh Manocha.  Ga-nav: Efficient terrain segmentation for robot navigation in unstructured outdoor environments.  IEEE Robotics and Automation Letters, 7(3):8138–8145, 2022.
  * [3] Mohamed Al Marzouqi and Ray A Jarvis.  Robotic covert path planning: A survey.  In 2011 IEEE 5th international conference on robotics, automation and mechatronics (RAM), pages 77–82. IEEE, 2011.
  * [4] Kai Ninomiya, Mubbasir Kapadia, Alexander Shoulson, Francisco Garcia, and Norman Badler.  Planning approaches to constraint-aware navigation in dynamic environments.  Computer Animation and Virtual Worlds, 26(2):119–139, 2015.
  * [5] Noriaki Hirose, Amir Sadeghian, Marynel Vázquez, Patrick Goebel, and Silvio Savarese.  Gonet: A semi-supervised deep learning approach for traversability estimation.  In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3044–3051. IEEE, 2018.
  * [6] Amirreza Shaban, Xiangyun Meng, JoonHo Lee, Byron Boots, and Dieter Fox.  Semantic terrain classification for off-road autonomous driving.  In 5th Annual Conference on Robot Learning, 2021.
  * [7] Kasi Viswanath, Kartikeya Singh, Peng Jiang, PB Sujit, and Srikanth Saripalli.  Offseg: A semantic segmentation framework for off-road driving.  In 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), pages 354–359. IEEE, 2021.
  * [8] Anukriti Singh, Kartikeya Singh, and PB Sujit.  Offroadtranseg: Semi-supervised segmentation using transformers on offroad environments.  arXiv preprint arXiv:2106.13963, 2021.
  * [9] Daniel Maturana, Po-Wei Chou, Masashi Uenoyama, and Sebastian Scherer.  Real-time semantic mapping for autonomous off-road navigation.  In Field and Service Robotics: Results of the 11th International Conference, pages 335–350. Springer, 2018.
  * [10] Abhinav Valada, Gabriel L Oliveira, Thomas Brox, and Wolfram Burgard.  Deep multispectral semantic scene understanding of forested environments using multimodal fusion.  In 2016 International Symposium on Experimental Robotics, pages 465–477. Springer, 2017.
  * [11] Lalitha Dabbiru, Chris Goodin, Nicklaus Scherrer, and Daniel Carruth.  Lidar data segmentation in off-road environment using convolutional neural networks (cnn).  SAE International Journal of Advances and Current Practices in Mobility, 2(2020-01-0696):3288–3292, 2020.
  * [12] Peng Jiang, Philip Osteen, Maggie Wigness, and Srikanth Saripalli.  Rellis-3d dataset: Data, benchmarks and analysis.  In 2021 IEEE international conference on robotics and automation (ICRA), pages 1110–1116. IEEE, 2021.
  * [13] Maggie Wigness, Sungmin Eum, John G Rogers, David Han, and Heesung Kwon.  A rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments.  In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5000–5007. IEEE, 2019.
  * [14] Kaicheng Zhang, Farzad Niroui, Maurizio Ficocelli, and Goldie Nejat.  Robot navigation of environments with unknown rough terrain using deep reinforcement learning.  2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages 1–7, 2018.
  * [15] Bo Zhou, Jianjun Yi, and Xinke Zhang.  Learning to navigate on the rough terrain: A multi-modal deep reinforcement learning approach.  In 2022 IEEE 4th International Conference on Power, Intelligent Computing and Systems (ICPICS), pages 189–194. IEEE, 2022.
  * [16] Jinyoung Choi, Kyungsik Park, Minsu Kim, and Sangok Seok.  Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view.  In 2019 International Conference on Robotics and Automation (ICRA), pages 5993–6000. IEEE, 2019.
  * [17] Utsav Patel, Nithish Kumar, Adarsh Jagan Sathyamoorthy, and Dinesh Manocha.  Dynamically feasible deep reinforcement learning policy for robot navigation in dense mobile crowds.  arXiv preprint arXiv:2010.14838, 2020.
  * [18] Biao Gao, Shaochi Hu, Xijun Zhao, and Huijing Zhao.  Fine-grained off-road semantic segmentation and mapping via contrastive learning.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5950–5957. IEEE, 2021.
  * [19] Amirreza Shaban, Xiangyun Meng, JoonHo Lee, Byron Boots, and Dieter Fox.  Semantic terrain classification for off-road autonomous driving.  In Conference on Robot Learning, pages 619–629. PMLR, 2022.
  * [20] Viktor Wiberg, Erik Wallin, Tomas Nordfjell, and Martin Servin.  Control of rough terrain vehicles using deep reinforcement learning.  IEEE Robotics and Automation Letters, 7(1):390–397, 2021.
  * [21] Shirel Josef and Amir Degani.  Deep reinforcement learning for safe local planning of a ground vehicle in unknown rough terrain.  IEEE Robotics and Automation Letters, 5(4):6748–6755, 2020.
  * [22] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra.  Continuous control with deep reinforcement learning.  arXiv preprint arXiv:1509.02971, 2015.
  * [23] Dieter Fox, Wolfram Burgard, and Sebastian Thrun.  The dynamic window approach to collision avoidance.  IEEE Robotics & Automation Magazine, 4(1):23–33, 1997.
  * [24] Marija Seder and Ivan Petrovic.  Dynamic window based approach to mobile robot motion control in the presence of moving obstacles.  In Proceedings 2007 IEEE International Conference on Robotics and Automation, pages 1986–1991. IEEE, 2007.
  * [25] J. Fink, G. Gremillion, S. Nogar, and A. Schang.  ARL Robotics Simulation.  https://gitlab.sitcore.net/arl/robotics-simulation/, 2019.
  * [26] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, Andrew Y Ng, et al.  Ros: an open-source robot operating system.  In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
  * [27] Armin Hornung, Kai M Wurm, Maren Bennewitz, Cyrill Stachniss, and Wolfram Burgard.  Octomap: An efficient probabilistic 3d mapping framework based on octrees.  Autonomous robots, 34:189–206, 2013.
  * [28] Péter Fankhauser and Marco Hutter.  A universal grid map library: Implementation and use case for rough terrain navigation.  Robot Operating System (ROS) The Complete Reference (Volume 1), pages 99–120, 2016.
  * [29] Avi Spector, Wanying Zhu, Jumman Hossain, and Nirmalya Roy.  Simulated forest environment and robot control framework for integration with cover detection algorithms.  In Proceedings of 9th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT2022), 2022.
  * [30] Samuel Triest, Matthew Sivaprakasam, Sean J. Wang, Wenshan Wang, Aaron M. Johnson, and Sebastian Scherer.  Tartandrive: A large-scale dataset for learning off-road dynamics models.  In 2022 International Conference on Robotics and Automation (ICRA), pages 2546–2552, 2022.