# ShadowNav: Crater-Based Localization for Nighttime and Permanently Shadowed
Region Lunar Navigation

Abhishek Cauligi*
<EMAIL_ADDRESS>R. Michael Swan*
<EMAIL_ADDRESS>Hiro Ono
<EMAIL_ADDRESS>Shreyansh Daftry
<EMAIL_ADDRESS>John Elliott
<EMAIL_ADDRESS>Larry Matthies
<EMAIL_ADDRESS>Deegan Atha
<EMAIL_ADDRESS>Jet Propulsion Laboratory  California Institute of
Technology
Pasadena  CA 91109  USA *Abhishek Cauligi and R. Michael Swan contributed
equally to this work.978-1-6654-9032-0/23/$\$31.00$ ©2023. California
Institute of Technology. Government sponsorship acknowledged.
The research was carried out at the Jet Propulsion Laboratory, California
Institute of Technology, under a contract with the National Aeronautics and
Space Administration (80NM0018D0004).

###### Abstract

There has been an increase in interest in missions that drive significantly
longer distances per day than what has currently been performed. For example,
Endurance-A proposes driving several kilometers a day in order to reach its
target traverse of 2000 km in 4 years. Additionally, some of these proposed
missions, including Endurance-A and rovers for Permanently Shadowed Regions
(PSRs) of the moon, require autonomous driving and absolute localization in
darkness. Endurance-A proposes to drive 1200 km of its total traverse at
night. The lack of natural light available during these missions limits what
can be used as visual landmarks and the range at which landmarks can be
observed. In order for planetary rovers to traverse long-ranges, onboard
absolute localization is critical to the rover’s ability to maintain its
planned trajectory and avoid known hazardous regions. Currently, the
localization performed onboard rovers is relative to the rover’s frame of
reference and is performed through the integration of wheel and visual
odometry and inertial measurements. To accomplish absolute localization, a
“ground-in-the-loop” (GITL) operation is performed wherein a human operator
matches local maps or images from onboard with orbital images and maps. This
GITL operation places a limit on the distance that can be driven in a day to a
few hundred meters, which is the distance that the rover can maintain
acceptable localization error via relative methods. Previous work has shown
that using craters as landmarks is a promising approach for performing
absolute localization on the moon during the day. In this work we present a
method of absolute localization that utilizes craters as landmarks and matches
detected crater edges on the surface with known craters in orbital maps. We
focus on a localization method based on a perception system which has an
external illuminator and a stereo camera. While other methods based on lidar
exist, lidar is not currently planned for deployment on the current proposed
nighttime and PSR missions. In this paper, we evaluate (1) both monocular and
stereo based surface crater edge detection techniques, (2) methods of scoring
the crater edge matches for optimal localization, and (3) localization
performance on simulated Lunar surface imagery at night. We demonstrate that
this technique shows promise for maintaining absolute localization error of
less than 10 m required for most planetary rover missions.

###### Contents

  1. 1 Introduction
  2. 2 Related Works
  3. 3 Approach
    1. 3.1 A. Surface Crater Detection
      1. 3.1.1 Positive Obstacle False Positive Rejection
    2. 3.2 B. Particle Filter
      1. 3.2.1 Q-Score
      2. 3.2.2 Overview
    3. 3.3 C. Surface to Orbital Crater Transformation
      1. 3.3.1 Stereo hole filling
  4. 4 Datasets Overview
    1. 4.1 A. Simulated Lunar Environment
    2. 4.2 B. Simulated Craters for Detection Analysis
    3. 4.3 C. Simulated Trajectories for Localization Analysis
    4. 4.4 D. Real Data of Negative Obstacles at Night
  5. 5 Crater Detection Performance
    1. 5.1 A. Metrics
    2. 5.2 B. Detection Results on Simulated Data
    3. 5.3 C. Detection Results on Real Data
  6. 6 Localization Performance
    1. 6.1 A. Resampling Scheme Comparison
    2. 6.2 B. Baseline Performance Evaluation
    3. 6.3 C. Debugging
  7. 7 Conclusions
    1. 7.1 D. Future Work

Figure 1: The ShadowNav localization algorithm performs absolute localization
for a Lunar rover mission located at the red position in the left image by
matching known craters from (left) an orbital map against (right) detected
craters from the rover stereo cameras.

##  1 Introduction

Figure 2: Schematic of the ShadowNav algorithm proposed to perform absolute
localization on the Moon. A particle filter is used to match craters detected
by the rover stero cameras with known craters from an orbital map.

Long-range Lunar navigation, and specifically navigating within darkness, has
gained a significant amount of traction recently. For example, missions to
Permanently Shadowed Regions (PSRs) of the moon have been proposed such as the
VIPER mission [1, 2] and the Lunar Polar Volatiles Explorer mission concepts.
Furthermore, there are missions that have proposed driving during the Lunar
night in order to traverse longer distances. For example, the new Decadal
Survey [3] recommends the Endurance-A Lunar rover mission should be
implemented as a strategic medium-class mission as the highest priority of the
Lunar Discovery and Exploration Program. The Endurance-A rover proposal plans
to drive $2000\text{\,}\mathrm{km}$ in the South Pole-Aitken (SPA) Basin to
collect $100\text{\,}\mathrm{kg}$ of samples, which would be delivered to
Artemis astronauts. This mission concept study [4] identified several key
capabilities required to complete this mission which are: (1) Endurance will
need to drive 70% of its total distance during the night to enable daytime
hours dedicated to science and sampling. (2) The mission will require onboard
autonomy for the majority of its operations, while the ground only handles
contingencies. (3) Global localization is necessary to maintain an error of
$<$$10\text{\,}\mathrm{m}$ relative to orbital maps.

At present, existing rovers perform onboard localization relative to their own
reference frame. This is accomplished by using wheel and visual odometry and
inertial measurements. Absolute localization is performed periodically with a
“ground-in-the-loop” (GITL) operation. This is acceptable for current driving
distances which are a few hundred meters a day. Existing relative localization
has around 2% drift and therefore can only drive at most
$500\text{\,}\mathrm{m}$ before the error will be larger than
$10\text{\,}\mathrm{m}$. In order to traverse longer distances, on the order
of several kilometers a day proposed by missions such as Endurance-A,
autonomous absolute localization becomes critical. At present the Lunar
surface does not have continuous communication with Earth. Therefore, having
to perform several GITL operations for absolute localization in a day will
significantly reduce the distance that can be driven. The lack of frequent
absolute localization for the rover would lead to errors greater than the
maximum $10\text{\,}\mathrm{m}$ localization error which would present
significant risks to the mission through deviations from the desired
trajectory and risk for unidentified obstacles.

Craters as landmarks have been shown to be promising for absolute localization
on the Moon [5, 6]. However, the lack of natural light available while driving
within a PSR or during the Lunar night limits what can be used as a landmark
and the range at which the landmarks can be observed. Using craters is still
promising as the average distance between craters of
$\geq$$10\text{\,}\mathrm{m}$ in diameter is $100\text{\,}\mathrm{m}$ on
terrain with relatively fresh craters and $10\text{\,}\mathrm{m}$ on terrain
with old craters [7]. Additionally the Lunar Reconnaissance Orbiter Camera
(LROC) provides digital elevation models (DEMs) with a resolution between
$0.5\text{\,}\mathrm{m}$-$5\text{\,}\mathrm{m}$ per pixel [8] and there are
some DEMs within PSRs [9].

In this work, we propose using a stereo camera with an illuminator positioned
below the stereo camera in order to detect crater rims within the darkness.
The use of such an illuminator is motivated by the Endurance-A mission concept
study [4], which proposes the use of a stereo camera with an illumination
source as the perception system for a rover operating in darkness. Global
localization is then accomplished by matching the detected crater rims against
known craters from an orbital image as shown in Figure 2. To handle the
uncertainty and nonlinearity of the crater rim detection model, we utilize a
particle filter with a novel _Q-Score_ metric for ranking potential crater
matches in order to estimate the absolute position of the rover within an
orbital map. This paper demonstrates the initial results of both crater
detection within darkness and absolute localization within simulation which
are the results of the first two years of a planned three year effort to
validate this approach. Work is ongoing to collect and validate this approach
in a real-world Lunar analogue test location.

Statement of Contributions: This paper presents an approach to absolute
localization on the Moon that can be performed while a rover is in darkness,
such as within a PSR or during the Lunar night. The main contributions of the
work as summarized below:

1\. We developed a simulator based on Blender [10] which renders simulated
surface stereo imagery of the Lunar surface in darkness located within a known
orbital position. The rendering process utilizes the Hapke lighting model for
more accurate surface reflectance as well as DEMs captured by LROC for
realistic crater distributions.

2\. We evaluated different crater-edge detection techniques and demonstrate a
method which captures 80% of the leading crater arc at $10\text{\,}\mathrm{m}$
and can detect crater arcs out to $20\text{\,}\mathrm{m}$.

3\. We present a method to localize a rover within an orbital map using
surface crater-edge detections and known orbital craters based on a particle
filter and a metric we call the Q-Score which is detailed in Section 3.2.1.

4\. We demonstrate our absolute localization technique can achieve less than
$2\text{\,}\mathrm{m}$ absolute error with an assumed odometry drift of 2% and
an initial 3-sigma uncertainty of $3\text{\,}\mathrm{m}$.

Figure 3: Figure demonstrating the impact of placement of light source on
crater rim shadows. Left: Sample render of a crater with light source even
with camera. Right Sample render of a crater with light source below the
camera.

##  2 Related Works

Absolute localization on planetary surfaces is critical for expanding the
range rovers can travel in a day and over the course of a mission and there
have been many previous works that investigate this problem. There have been
techniques proposed for the Martian surface. Works such as [11, 12] consider
far range and horizon features which are at ranges that are beyond what is
expected can be seen in the dark. [13] proposes a technique on the Martian
surface for absolute localization that uses rocks and DEMs surface features.

In our work, we focus on the problem of global localization in darkness which
is relevant for permanently shadowed regions of the moon, for which there has
been a surge of interest in conducting scientific measurements and activities
[14]. Our solution approach is inspired by a host of recent works that seek to
leverage orbital maps for global rover localization in these shadowed regions.
In [13], the authors propose a localization procedure that matches an observed
rover image with an orbital map, but this approach neglects the rover motion
model and yields a deterministic estimate of the robot belief. A purely data-
driven approach is presented in [15], wherein a convolutional neural network
is trained on synthetic data to match the rover observations with orbital
imagery. Closest to our approach, [16] presents a particle filtering technique
to compare rover monocular camera imagery with orbital imagery and uses a
Siamese neural network approach to assign each particle a likelihood weight.
The authors in [6] propose a similar approach for Lunar absolute localization
known as LunarNav. However, LunarNav focuses on the daytime localization
problem and therefore considers different methods of crater matching that rely
on greater knowledge of the surface geometry than available in the nighttime
case.

##  3 Approach

In this work, we propose an absolute localization approach which utilizes a
crater’s leading edge as landmarks for localization. The end result of this
approach will be an estimated position and uncertainty within the orbital
frame. At present, this approach only considers position localization. Rover
orientation is assumed to be given by a star tracker which can compute
orientation in three dimensions from celestial measurements. Our approach
consists of two primary components:

  1. 1.

A leading-edge crater detection methodology for use with a Lunar rover
equipped with a stereo camera system and illumination source.

  2. 2.

A particle filter for computing a position belief based on a score computed
based on the association of crater edges and known orbital ground truth
craters, which we call the Q-Score, and the robot motion model.

###  3.1 A. Surface Crater Detection

In order to identify craters on the surface, the system was designed to be
used in conjunction with a perception system that contained a stereo camera
and an illumination source. This perception system was configured where the
illumination source was beneath the stereo camera. Examples of simulated
images with the light at the same height as the cameras and the light
positioned beneath the cameras are in Figure 3. It was observed that placing
the illumination source below the camera results in a shadow at the leading
edge of a negative obstacle. Furthermore, offsetting the light with the
cameras reduced the impact of the Hapke model washing out some of the surface
texture. Further details on the Hapke model and its impact on surface terrain
are provided in Section 4.2.

Here, we first review the three different techniques studied in this work for
detecting a crater’s leading edge: (1) a method of detecting jumps within
stero disparities, (2) a Canny edge detector used to find the shadow on the
leading edge, and (3) a convolutional neural network (CNN)-based edge detector
that uses both the monocular and disparity image as input.

1\. Stereo Disparity Discontinuity Method The first approach for leading edge
crater detection relies on detecting discontinuities within the stereo
disparity image. To accomplish this, the stereo disparity image must first be
generated using methods such as the JPLV algorithm [17] or the Semi-Global
Block Matching (SGBM) approach [18], among others. To account for the low
contrast that may be present in the Lunar rover case, Contrast Limited
Adaptive Histogram Equalization (CLAHE) is first run on the input images prior
to running stereo. CLAHE is an adaptive histogram equalization and operates on
sub-regions of an image which allows more consistent equalization across
different lighting conditions within an image. This is useful for this
application as there is a light-to-dark gradient from near-to-far within the
images. The resulting disparity image is then scanned column-by-column and,
when the difference between any two disparities is greater than some pre-
defined threshold, the larger column index is marked as a crater edge.
Further, any numerical issues stemming from stereo holes are accounted for by
omitting any pixels with spurious values during comparison.

2\. Canny Edge Detector Method For sensor configurations that contain an
illuminator located beneath the stereo cameras, shadows appear on the leading
edge of negative obstacles. In such cases, a Canny edge detector can be used
to distinguish the stark contrasting dark line along the rim. In this work,
the Canny edge detector from OpenCV [19] is used to find these shadows.

3\. CNN-Based Edge Detector Method The Holistically-Nested Edge Detection
(HED) approach presents a CNN-based deep learning based method for leading
edge crater detection [20]. This method uses the HED approach and can be
performed by directly using the publicly released neural network weights. HED
is capable of performing both monocular and stereo depth based edge detection.
For HED to perform edge detection within a depth image, it generates a three
channel image that contains horizontal disparity, height above ground, and
angle of the local surface normal with the inferred direction of gravity. The
RGB and depth predictions of the CNN are then merged to generate the desired
output.

####  3.1.1 Positive Obstacle False Positive Rejection

One shortcoming of the aforementioned leading edge crater detection approach
is the susceptibility of false positive cases in the presence of positive
obstacles. In order to account for this positive obstacle issue, the detected
edge points are passed through a filter that removes points which have hits on
the far side of the crater edge with a detected negative or flat slope.
Detected edge points are kept only if, within the region directly beyond the
detected edge, there exists a positive slope or if there is not enough stereo
to accurately compute the slope. Thus, the case of a detected positive slope
is assumed to correspond to the rising edge of the crater under the assumption
that the detected edge is the leading edge of a negative obstacle.
Alternatively, a detected edge is also retained if the far edge is not
captured due to low light conditions, as this is assumed to be an indication
of the presence of a large crater.

Algorithm 1 Q-Score Computation

0: Belief $b_{i}^{t}$, set of crater observations
$\\{z_{0,\textrm{rover}}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},z_{m,\textrm{rover}}^{t}\\}$,
set of ground truth craters
$\\{c_{0,\textrm{world}}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},c_{\ell,\textrm{world}}^{t}\\}$,
positive value $\varepsilon$

1: $\mathcal{Q}_{\textrm{inc}}\leftarrow\varepsilon$

2: for $i=1,\ldots,m$ do

3:
$z_{0,\textrm{world}}^{t}\leftarrow\textrm{rover\\_to\\_world}(z_{0,\textrm{rover}}^{t})$

4:
$d_{\textrm{cr}}\leftarrow\min\|c_{j,\textrm{world}}-z_{0,\textrm{world}}^{t}\|$

5:
${\mathcal{Q}}_{\textrm{inc}}\leftarrow{\mathcal{Q}}_{\textrm{inc}}+d_{\textrm{cr}}$

6: end for

7:
$Q_{\textrm{score}}\leftarrow\min\Big{(}1,(\frac{1}{m}{\mathcal{Q}}_{\textrm{inc}})^{-1}\Big{)}$

8: return $Q_{\textrm{score}}$

Algorithm 2 ShadowNav Particle Filtering Algorithm

0: Initial belief distribution $(\mu_{0},\Sigma_{0})$, number of particles
$N_{s}$, number of effective particles threshold $N_{\textrm{eff,thresh}}$

1:
$\\{b_{1}^{0},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},b_{N_{s}}^{0}\\}\leftarrow\textrm{sample\\_beliefs}(\mu_{0},\Sigma_{0})$

2:
$\\{w_{1}^{0},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},w_{N_{s}}^{0}\\}\leftarrow\\{1,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},1\\}$

3: $t\leftarrow 1$

4: while particle filter running do

5:
$\\{z_{0}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},z_{m}^{t}\\}\leftarrow\textrm{get\\_observations()}$

6:
$\\{q_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},q_{N_{s}}^{t}\\}\leftarrow\\{0,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},0\\}$

7: for $i=1,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},N_{s}$ do

8: $b_{i}^{t}\leftarrow\textrm{propagate\\_sample}(b_{i}^{t-1})$

9:
$q_{i}^{t}\leftarrow\log\textrm{Q\\_score}(b_{i}^{t},\\{z_{0}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},z_{m}^{t}\\})$

10: end for

11:
$q_{\textrm{min}}^{t}\leftarrow\min(q_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},q_{N_{s}}^{t})$

12: for $i=1,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},N_{s}$ do

13: $w_{i}^{t}\leftarrow w_{i}^{t-1}+q_{i}^{t}-q_{\textrm{min}}^{t}$

14: end for

15:
$N_{\textrm{eff}}\leftarrow\textrm{compute\\_N}_{\textrm{eff}}(w_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},w_{N_{s}}^{t})$

16: if $N_{\textrm{eff}}\leq N_{\textrm{eff,thresh}}$ then

17:
$\\{b_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},b_{N_{s}}^{t}\\}\leftarrow\textrm{resample\\_beliefs}(\\{b_{i}^{t}\\}_{i=1}^{N_{s}},\\{w_{i}^{t}\\}_{i=1}^{N_{s}})$

18:
$\\{w_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},w_{N_{s}}^{t}\\}\leftarrow\\{1,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},1\\}$

19: end if

20: $t\leftarrow t+1$

21: end while

Algorithm 3 Systematic Resampling

0: Particles
$\\{b_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},b_{N_{s}}^{t}\\}$ and
associated weights
$\\{w_{1}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},w_{N_{s}}^{t}\\}$

1: $n^{t}=\log\Big{(}\sum_{i=1}^{N_{s}}\exp(b_{i}^{t})\Big{)}$

2:
$\\{\tilde{w}_{0}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},\tilde{w}_{N_{s}}^{t}\\}\leftarrow\\{0,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},0\\}$

3: for $i=1,\makebox[9.00002pt][c]{.\hfil.\hfil.}{},N_{s}$ do

4: $\tilde{w}_{i}^{t}\leftarrow\exp(w_{i}^{t}-n^{t})$

5: end for

6:
$\\{q_{0},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},q_{N_{s}}\\}\leftarrow\textrm{cum\\_sum}(\\{\tilde{w}_{0}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},\tilde{w}_{N_{s}}^{t}\\})$

7: $n\leftarrow 0$

8: $m\leftarrow 0$

9: $u_{0}\sim{\mathbb{U}}(0,\frac{1}{N_{s}})$

10: while $n\leq N_{s}$ do

11: $u=u_{0}+\frac{n}{N_{s}}$

12: while $q_{m}\leq u$ do

13: $m\leftarrow m+1$

14: end while

15: $n\leftarrow n+1$

16: $b_{n}^{t}\leftarrow b_{m}^{t}$

17: end while

18: return
$\\{b_{0}^{t},\makebox[9.00002pt][c]{.\hfil.\hfil.}{},b_{N_{s}}^{t}\\}$

###  3.2 B. Particle Filter

Here, we provide an overview of the proposed ShadowNav particle filtering
approach. First, we provide further details on the Q-Score metric that is used
in the belief update step.

####  3.2.1 Q-Score

The Q-Score provided the measurement probability of some position belief based
on rover frame observations and an orbital map. The procedure for computing
the Q-Score is given in Algorithm 1. The algorithm takes as input a given
belief $b_{i}^{t}$, a set of $m$ observed edges in rover frame, and a set of
$\ell$ ground truth crater observations to associate these measurements with
(Line  ‣ 1). A value $\mathcal{Q}_{\textrm{inc}}$ is initialized to some
negligibly small, positive value $\varepsilon$ to later avoid divide-by-zero
issues (Line 1). Next, for each measurement $z_{i}^{t}$ in the rover frame,
the detected edge is converted to world frame (Line 3) and the minimum
distance to an edge from the ground truth map computed (Line 4). The
$\mathcal{Q}_{\textrm{inc}}$ is incremented by the distance between the
observed edge and its associated ground truth observation (Line 5). The
Q-Score is computed as the reciprocal of $\mathcal{Q}_{\textrm{inc}}$ and a
$\min$ operation is applied to ensure that the score provided by any
particular run is between 0 and 1 (Line 7). This implies that observations and
belief pairs which are less than $1\text{\,}\mathrm{m}$ away from ground truth
will receive the same score as those exactly 1m away from ground truth, which
is seen as acceptable given the orbital DEM resolution and mission concept
localization requirements.

In addition to the shortest distance formulation from Line 4, additional
approaches were also explored for determining the Q-Score. One alternate
approach investigated included fitting a Gaussian normal distribution on the
orbital map crater edges and the Q-Score value was them computed based on the
intensity (i.e., distance to the computed mean) of the point hit by
observations or 0 in cases when no point was hit. In practice, it was
determined that the shortest distance formulation provided the most robust
results for use with the particle filter and also did not require additional
projection calculations to project each belief from the orbital frame to rover
frame.

####  3.2.2 Overview

A description of the ShadowNav particle filtering algorithm is given in Alg.
2. The algorithm takes as input a Gaussian belief distribution
$(\mu_{0},\Sigma_{0})$ assumed for the initial robot position, the number of
particles $N_{s}$ to use in the particle filter, and a threshold for the
effective number of beliefs $N_{\textrm{eff,thresh}}$ used to trigger
resampling (Line  ‣ 2). The filter is initialized by sampling $N_{s}$
particles from the initial belief distribution and assigning a weight of equal
importance for each particle (Lines 1-2). As common in particle filtering
implementations [21], we note that we used the $\log$ of the weights for
improved numerical stability of the weight update step [22]. Given a new set
of crater observations (Line 5), a set of Q-Score measurements is initialized
for computing for each individual particle (Line 6). After applying the motion
model update to each particle (Line 8), the Q-Score for each updated particle
is computed using the procedure from Alg. 1 by comparing against the current
measurements (Line 9). The particle weights are then updated in $\log$-domain
(Line 13) with a normalization step to ensure non-negative weights (Line 11).
Next, the number of effective samples $N_{\textrm{eff}}$ at the current
iteration is calculated (Line 15). A common pitfall of particle filters is
“degeneracy”, wherein the weights $\\{w_{i}^{t}\\}$ collapse around a handful
of particles and computational resources are wasted on propagating low
likelihood particles [21]. If $N_{\textrm{eff}}$ is below the threshold
$N_{\textrm{eff,thresh}}$, then this indicates that the filter is degenerating
and a resample operation is triggered (Line 17).

Further details on the systematic resampling approach used in this work are
provided in Algorithm 3. Given a set of particles and their associated
weights, the weights are first normalized to $(0,1]$ from $\log$-domain (Lines
1-4) and the cumulative sum of these normalized weights $\tilde{w}_{i}^{t}$
computed (Line 6). The key step in systematic resampling is to sample a random
value $u_{0}$ from a uniform distribution inversely proportional to $N_{s}$
(Line 9) and then incrementally sample a new particle from this “bin” of width
$\frac{1}{N_{s}}$ . This ensures that, after resampling, at least one particle
is retained from each $\frac{1}{N_{s}}$ interval from the previous belief
distribution.

(a) Sample of terrain with 90∘ from camera.

(b) Sample opposition effect during the day.

(c) Sample effect of surface reflectance at night with an illuminator.

Figure 4: The opposition effect simulated during the day and its effect at
night with an external illuminator.

###  3.3 C. Surface to Orbital Crater Transformation

For every observation step, rover frame crater edges were detected with a
stereo camera pair that provided the depth, and thus a relative position for
the crater edge was saved. This relative crater distance was added to each
particle’s belief position to form an estimate of the observed crater position
in the world frame for each particle. The orbital map was projected to the
world frame and then the shortest distance metric noted in the Q-Score
algorithm was used to determine which particle belief positions were most
likely and thus which observed crater was the most likely one to match the
known orbital craters.

####  3.3.1 Stereo hole filling

As some crater edge detections do not rely on depth information, not all
pixels in the stero camera depth or disparity image will have a detected depth
value and, in such cases, no relative position would be available for matching
rover observations to the orbital map. For such observations, a simple plane
fit can be carried out to fill in the depth information. A future area of
investigation includes carrying out an improved stereo hole filling approach,
in particular using existing knowledge on what the regional terrain looks
like.

Figure 5: The trajectories used for the numerical experiments are overlaid on
the orbital map here with the crater numbers in black. A red square indicator
is at the start and a green circle indicator is at the end of each trajectory.
Trajectory 1 is in blue, trajectory 2 is in orange, and trajectory 3 is in
purple.

##  4 Datasets Overview

###  4.1 A. Simulated Lunar Environment

At the time of writing, a Lunar dataset with images captured in the dark with
an illuminator did not exist. Therefore, to evaluate the approach, a
simulation environment was developed using the Blender software [10]. In order
to simulate images as realistically as possible, the Hapke lighting model [23,
24, 25] was implemented. This model approximates the Lunar surface reflectance
and will simulate the “opposition effect”. This effect leads to a focused
point of extreme saturation at a location within an image where the camera ray
and light source are at zero phase angle. The Hapke lighting model was
implemented using the “old highland” parameters of the moon provided in [26],
as these most closely match the poles of the moon where PSRs can be found. The
coherent backscattering opposition effect (CBOE) was left out of our
implementation and only the shadow hiding opposition effect (SHOE) was
implemented as it dominates most or all lighting calculations in our use case,
while CBOE has a negligible or very small effect. Initial implementation was
done using the Open Shading Language (OSL), however not all rays are available
for calculation due to optimizations made in OSL, so workarounds were needed
to implement the Hapke lighting model in Blender using OSL. While this was
partially successful, it was not very robust and we had numerous issues.
Instead of using OSL, we opted to modify the source of Blender to add the
Hapke bidirectional reflectance distribution function (BRDF) directly into the
Blender Cycles renderer code which also reduced the render time by greater
than a factor of 2 through the use of Nvidia CUDA.

In order to represent a realistic 3D model of the surface geometry, DEMs
produced from LROC were utilized. While LROC has enough resolution to resolve
craters of around $10\text{\,}\mathrm{m}$, its resolution is not quite good
enough for generating smooth surface imagery. In order to have smooth surface
image renders, the DEMs from LROC were scaled down to be
$0.25\text{\,}\mathrm{m}$ resolution. Crater measurements in future
discussions were based on this scaled resolution. This scaled DEM was imported
into Blender and a surface texture was added. The surface texture comprised of
two scales of fractal Brownian motion, which is a natural noise that was added
to the DEM in order to simulate Lunar surface texture for stereo to utilize.
Figure 4 demonstrates three sample renders, two in the daylight and one at
night with an illumination source from our simulation. It demonstrates what
the surface looks like in daytime conditions as well as the effect of the
Hapke model during the day with the sun behind the camera and the effect of
the illumination source. From this it was observed that the full amount of
daytime texture is not observed during the night with an illumination source.

Table 1: Table of crater sizes in crater detection dataset. Crater | Diameter (m) | Depth (m)
---|---|---
1 | 9.2 | 1.0
2 | 9.1 | 0.75
3 | 11.3 | 0.84
4 | 4.4 | 0.55
5 | 3.7 | 0.40
6 | 8.3 | 0.27
7 | 11.9 | 0.44
8 | 3.9 | 0.48
9 | 4.1 | 0.49
10 | 2.3 | 0.25

###  4.2 B. Simulated Craters for Detection Analysis

In order to evaluate the performance of different crater detection techniques,
a dataset with different sized craters was built. This dataset was built using
the simulation process within Blender and captured stereo pair renders between
$5\text{\,}\mathrm{m}$ and $20\text{\,}\mathrm{m}$ from the front crater rim
in increments of $0.1\text{\,}\mathrm{m}$. This dataset contained 10 different
craters with varying sizes and depths. The sizes of the craters within this
dataset are in Table 1 and their locations corresponding to the crater ID in
our simulated environment are marked in Figure 5.

###  4.3 C. Simulated Trajectories for Localization Analysis

In order to evaluate the localization performance, several trajectories were
run in the simulated environment. These trajectories were run to generate an
image every $1\text{\,}\mathrm{m}$ and were designed to approach craters in
different ways that might present challenges to our filtering approach. The
$1\text{\,}\mathrm{m}$ observation delta was used to reduce render times of
our dataset, as rendering every $0.1\text{\,}\mathrm{m}$ did not result in a
significant localization performance change. An overview of the trajectories
within the orbital environment are displayed in Figure 5.

###  4.4 D. Real Data of Negative Obstacles at Night

In addition to the simulated data generated, a dataset was collected in the
Arroyo, which is a dry river bed near the NASA Jet Propulsion Laboratory. This
dataset contained a few different negative obstacles that were imaged at 5,
10, and $15\text{\,}\mathrm{m}$ away from the leading edge. This dataset was
used to validated that the stereo and crater edge detection algorithms work on
real data collected at night with an external illuminator.

Figure 6: Plots of different metrics evaluating crater detection performance.
Left: Plot that shows image-based crater edge detection score versus range for
all craters evaluated. Right: Plot that shows percent of the crater front arc
detected for all craters evaluated.

##  5 Crater Detection Performance

Figure 7: Sample stereo results using JPLV stereo on a sample negative
obstacle.

(a) Ground Truth at $7\text{\,}\mathrm{m}$

(b) Ground Truth at $12\text{\,}\mathrm{m}$

(c) Ground Truth at $17\text{\,}\mathrm{m}$

(d) JPLV Disparity + Canny at $7\text{\,}\mathrm{m}$

(e) JPLV Disparity + Canny at $12\text{\,}\mathrm{m}$

(f) JPLV Disparity + Canny at $17\text{\,}\mathrm{m}$

(g) JPLV HED at $7\text{\,}\mathrm{m}$

(h) JPLV HED at $12\text{\,}\mathrm{m}$

(i) JPLV HED at $17\text{\,}\mathrm{m}$

Figure 8: The efficacy of the JPLV HED approach over JPLV Disparity + Canny is
demonstrated in simulations of crater rim detection overlay samples for crater
1.

(a) Negative obstacle $5\text{\,}\mathrm{m}$ away

(b) Negative obstacle $10\text{\,}\mathrm{m}$ away

(c) Negative obstacle $10\text{\,}\mathrm{m}$ away

(d) Negative obstacle $5\text{\,}\mathrm{m}$ away

(e) Negative obstacle $15\text{\,}\mathrm{m}$ away

(f) Negative obstacle $10\text{\,}\mathrm{m}$ away

Figure 9: Qualitative edge detections using JPLV disparity discontinuity
detection and Canny hybrid on negative obstacles on a real dataset collected
in a dry river bed at night. These results demonstrate the transferability of
the crater detection algorithms from simulated data to a real environment.

###  5.1 A. Metrics

In order to evaluate the performance of surface crater detection, the dataset
referenced in Section 4.2 was utilized. Five different combinations of
algorithms were evaluated. These were disparity discontinuity detection within
SGBM stereo, disparity discontinuity detection within JPLV stereo, HED using
SGBM stereo, HED using JPLV stereo, and a hybrid JPLV disparity discontinuity
detection and canny edge detection approach. The hybrid discontinuity
detection and canny approach was implemented so that Canny only ran on the
portion of the image that was $10\text{\,}\mathrm{m}$ away or further. This
was done since it was observed the discontinuity detection worked well in the
near range but stereo began to degrade beyond $10\text{\,}\mathrm{m}$.

These algorithms were evaluated with two different metrics. The first was an
image based edge scoring method which captures an average Gaussian probability
that a detected edge is on a ground truth crater edge. It utilizes a distance
error computed in image space as represented in Equation 1 where
$\mathrm{Error}_{\mathrm{dist}_{px}}$ is the pixel error from ground truth to
detection, $\mathrm{range}_{\mathrm{gt}}$ is the known ground truth range,
$\mathrm{fl}$ is the focal length of camera, $\mathrm{ss}$ is the sensor size
of the camera, and $\mathrm{Error}_{\mathrm{dist}}$ is the error in meters of
the detection.

$\mathrm{Error}_{\mathrm{dist}}=\mathrm{Error}_{\mathrm{dist}_{px}}*\frac{\mathrm{range}_{\mathrm{gt}}}{(\mathrm{fl}*\mathrm{ss})}~{}$
(1)

The distance error was then passed into a Gaussian. The Gaussian probabilities
for all of the detected pixels were summed together and normalized by number
of detected points to obtain a score. This scoring method infused ground truth
range values to remove the impact of stereo holes and stereo range uncertainty
on the projection in order to better isolate the specific performance of the
crater detection algorithms. The sigma value for the Gaussian that was used in
these experiments was $0.25\textrm{m}$. This was chosen because the resolution
of the DEM utilized was $0.25\text{\,}\mathrm{m}$. Therefore most detections
should fall within this boundary if they are highly accurate. The second
metric used was ”percent of front arc detected”. In this metric, there is a
ground truth circle of the orbital crater. Depending on the pose of the
simulated cameras, the half arc of the ground truth circle that was nearest
the simulated camera was projected into image space. The crater detection was
then matched to the half arc and the percentage of the half arc that was
successfully identified was determined. This metric removes the Gaussian
component from the first metric; however, it does not capture false positives
like the first metric.

###  5.2 B. Detection Results on Simulated Data

The results of running the different algorithms on the simulated dataset are
observed in Figure 6. There were several notable observations from the
results. The first was that the algorithms tended to perform the best around
$10\text{\,}\mathrm{m}$ and did not improve as craters came closer. This was
believed to be because as the camera gets closer to the crater, more of the
crater becomes visible and the discontinuities become smaller. However, as the
crater becomes further than $10\text{\,}\mathrm{m}$, the stereo began to
degrade. Additionally, for the hybrid stereo and Canny technique, the Canny
detection started detection at $10\text{\,}\mathrm{m}$ and led to a
significant jump in performance.

In terms of algorithm comparison, JPLV disparity discontinuity performed
better than SGBM disparity discontinuity which is likely due JPLV having more
holes than SGBM. These holes at the boundary helped the disparity
discontinuity detector find a better edge. However, for HED, it performed well
with either stereo technique, likely due to its representation of depth
containing height values. HED was used with its out of the box weights from
its authors. It likely could be improved with finetuning on a Lunar dataset.

In addition to quantitative results, samples of crater rim detection overlays
are in Figure 8. These results were on crater 1 which is a nearly
$10\text{\,}\mathrm{m}$ in diameter crater. Both methods were able to detect
the craters well, but JPLV HED did have more falloff at
$17\text{\,}\mathrm{m}$ than the Canny detector. However, the Canny edge
detector was optimized for this environment where as HED was a generalized
detector. Overall the generalization of HED was extremely promising as a
crater rim detection approach.

###  5.3 C. Detection Results on Real Data

As described previously, data was collected from a location with negative
obstacles at night. This dataset was used to validate the performance of
stereo and crater detection algorithms. Figure 7 presents a sample of
$5\text{\,}\mathrm{m}$ and $10\text{\,}\mathrm{m}$ negative obstacles and the
corresponding stereo results from JPLV. From this figure is was observed that
stereo is dense up unto the leading edge of the negative obstacle.
Additionally, at $5\text{\,}\mathrm{m}$, the far edge of the negative obstacle
was captured in the disparity values. At $10\text{\,}\mathrm{m}$, the far
edge, did contain some disparity values but it was sparse. While not fully
representative of the Lunar surface, this demonstrated that current stereo
techniques do have the capability to work in low light conditions at the
ranges necessary. The data was also used to evaluate the edge detection
techniques. The JPLV disparity discontinuity and Canny edge detection hybrid
was found to be the best on simulation data and therefore it was used on the
real data. Figure 9 demonstrates sample detections at different ranges. These
detection results did contain false positives on some of the vegetation as the
false positive rejection was not run. Vegetation is not present on the moon,
however, objects such as rocks could present similar issues. Overall, the
negative obstacle edge detection qualitatively performs well.

(a) Ground truth error for traj. 1.

(b) Filter covariance for traj. 1.

(c) Ground truth error for traj. 2.

(d) Filter covariance for traj. 2.

Figure 10: A comparison of the four proposed resampling schemes demonstrated
that systematic resampling empirically outperforms the other scheme in terms
of relatively lower ground truth error and reduced uncertainty in the filter.

(a) Ground truth error.

(b) Filter covariance.

Figure 11: Monte Carlo simulations for trajectories 1–3 demonstrated the
efficacy of the Q-Score based particle filtering approach at accomplishing
global rover localization.

(a) Traj. 1 traverse – case A.

(b) Traj. 1 traverse – case B.

Figure 12: Two Monte Carlo trials for trajectory 1 are illustrated with the
ground truth in red and the weighted average belief $\mu^{t}$ in blue. The
comparatively better performance of the filter in case A (left) was due to
false positive crater rim measurements in case B (right) that led to worse
localization.

##  6 Localization Performance

In this section, we provide Monte Carlo results on the performance of the
proposed ShadowNav filtering algorithm. For each simulation, we analyzed the
performance of the ShadowNav filter on the basis of the following metrics:

Ground truth error: We computed the weighted average mean
$\mu^{t}=\sum_{i=1}^{N_{s}}w_{i}^{t}b_{i}^{t}$ at time $t$ for the filter
using the particle weights and beliefs and compute the $\ell_{2}$-distance to
the ground truth ${\textrm{gt}}^{t}$, i.e.,
$\|\mu^{t}-{\textrm{gt}}^{t}\|_{2}$.

Particle filter uncertainty: To capture the uncertainty associated with the
current belief, we additionally computed the weighted covariance matrix
$\Sigma^{t}=\sum_{i=1}^{N_{s}}\tilde{w}_{i}^{t}(b_{i}^{t}-\mu^{t})(b_{i}^{t}-\mu^{t})$,
where $\tilde{w}_{i}^{t}$ are the normalized weights detailed in Alg. 3. The
metric we report at each time step was the square root of the largest
eigenvalue $\sqrt{\lambda_{\textrm{max}}(\Sigma^{t})}$, which corresponded to
the worst case variance of the estimation error [27, 28].

Mahalanobis distance: The final metric we computed was the Mahalanobis
distance, which measures the distance between and the particle filter
distribution and ground truth position. We approximately computed this by
fitting a Gaussian distribution ${\mathcal{N}}(\mu^{t},\Sigma^{t})$ to the
particle filter distribution, for which the Mahalnobis distance is simply a
weighted $\ell_{2}$-norm
$\sqrt{(\mu^{t}-{\textrm{gt}}^{t})^{T}(\Sigma^{t})^{-1}(\mu^{t}-{\textrm{gt}}^{t})}$.

###  6.1 A. Resampling Scheme Comparison

In this section, we compared the baseline systematic resampling approach
detailed in Alg. 3 against three other resampling methods utilized:
multinomial, residual, and stratified (we refer the reader to [21, 29, 30] for
a thorough review of these approaches.) Figure 10 presents the ground truth
error and filter uncertainty for the four different resampling approaches. We
saw that, for the two trajectories compared in Figure 10, systematic
resampling led to comparable ground truth error as the other resampling
approaches, but that systematic resampling outperformed the other approaches
in terms of the overall uncertainty of the filter. Indeed, we note that
multinomial resampling, the most commonly employed resampling technique, fared
quite poorly in terms of the variance of the filter uncertainty (Fig. 10(b)
and 10(d)).

###  6.2 B. Baseline Performance Evaluation

Finally, we evaluated the performance of the proposed ShadowNav particle
filter approach on three test trajectories. Our analysis consisted of Monte
Carlos simulations with 25 seeds and utilizing 2% odometry noise and initial
belief distribution with $\sigma_{0}=$$3\text{\,}\mathrm{m}$. Each simulation
was run with $N_{s}=100$ particles and systematic resampling as the resampling
scheme with $N_{\textrm{eff,thresh}}=50$ as the resampling threshold.

Figure 11 shows Monte Carlo simulation results for the three test
trajectories. We saw that the initial uncertainty in the filter began at
approximately $3\text{\,}\mathrm{m}$ as expected by sampling from a
distribution with $\sigma_{0}=$$3\text{\,}\mathrm{m}$. Thereafter, the filter
was able to improve the rover position estimate, which led to an absolute
error reduction of $4\text{\,}\mathrm{m}$. Further, we see in Table 2 that the
metrics computed at the final time step indicate convergence of the filter,
with an average final error of $\leq$$4\text{\,}\mathrm{m}$ and an absolute
error reduction of $4\text{\,}\mathrm{m}$.

(a) Trajectory 1

(b) Trajectory 2

(c) Trajectory 3

Figure 13: The final ground truth error distribution for 25 Monte Carlo simulations showed filter convergence to $\leq 4\textrm{m}$ error in all cases for trajectories 2 and 3 and for the majority of cases for trajectory 1. Table 2: The metrics computed at the end of a long-range lunar traverse indicate convergence of the particular filter on trajectories 2 and 3, but spurious measurements from unlabeled crater lead to relatively poor performance on trajectory 1. | Error | Uncertainty | Mahalanobis Dist.
---|---|---|---
Traj. 1 | $3.84\pm 2.78$ | $1.84\pm 1.12$ | $8.74\pm 10.03$
Traj. 2 | $1.75\pm 0.78$ | $1.32\pm 0.76$ | $2.75\pm 1.88$
Traj. 3 | $1.68\pm 0.7$ | $1.39\pm 0.61$ | $2.92\pm 1.91$

As seen in Figure 13, while the filter performed well on trajectories 2 and 3,
the filter was less performant for the trajectory 1 test case. Figure 12
illustrates the performance of the filter on trajectory 1 for two different
random seeds as the rover starts from the northern edge of the orbital map and
moves southward. During the middle portion of this traverse, the craters were
out-of-sight for the rover and, as we see in Figure 11, false positive
observations led to increases in the error and uncertainty in the filter. As
the crater in the southern portion of the orbital map became observable for
the rover, we saw that the estimate quickly improved in case A (Fig. 12(a)),
but continues to have a residual error in case B (Fig. 12(b)). This poor
convergence behavior was also explained by false positive observations,
wherein the filter had difficulty reconciling the front edge of the rim with
the back edge, an issue that requires further investigation.

###  6.3 C. Debugging

When testing the particle filter, we found it helpful to generate “perfect”
datasets where ground truth depth was generated directly from the simulator as
shown in Figure 14(b)) and crater edges were plotted into the rover frame
using their exact known world coordinates (see Fig. 8(a)-8(c)). This approach
uncovered bugs with our perception and projection pipeline as well as the
particle filter pipeline and it is highly recommended to build such a dataset
for all similar work.

(a) Simulated image from Blender.

(b) Perfect depth: blue is close, red is far

Figure 14: Crater 1 viewed from $5\text{\,}\mathrm{m}$ away from front rim.

##  7 Conclusions

In this work we present a system to perform autonomous absolute localization
on a Lunar rover while it is in darkness. This system entails using a stereo
camera and illuminator. We enhanced a Blender based simulation with a custom
Lunar texture and an implementation of the Hapke model to model surface
reflectance as accurately as possible. We further demonstrate both geometric
and learning based techniques for detecting the leading edge of a crater with
ability to detect some craters out to $20\text{\,}\mathrm{m}$ range. We
propose a method of matching the detected leading crater rims with known
craters within an orbital map and using these matches to score observations
with our Q-Score. Finally we demonstrate absolute localization within our
simulation environment with less than $4\text{\,}\mathrm{m}$ error, and an
absolute error reduction of $4\text{\,}\mathrm{m}$ upon detecting craters.
These results show promise for further investigation in the future on more
simulation environments as well as on to be collected real analogue datasets.

###  7.1 D. Future Work

In the future, we seek to perform several updates and additional evaluations.
The primary focus is to experimentally collect a nighttime dataset using
representative hardware in an analogue Lunar environment with negative
obstacles to evaluate the system. Additional evaluation is planned to evaluate
the performance of the proposed approach along longer trajectories, on more
varied Lunar type locales, and for different rover specific parameters such as
camera height off of the ground. Finally, we plan to validate our proposed
approach on a flight-like embedded computer (e.g., a Snapdragon) to
demonstrate that it is computationally feasible for use onboard a Lunar rover.

###### Acknowledgements.

The research was carried out at the Jet Propulsion Laboratory, California
Institute of Technology, under a contract with the National Aeronautics and
Space Administration (80NM0018D0004). The authors would like to thank Yang
Cheng, Olivier Lamarre, and Scott Tepsuporn for their discussions during the
development of this work.

## References

  * [1] K. Ennico-Smith, A. Colaprete, R. Elphic, J. Captain, J. Quinn, and K. Zachny, “The Volatiles Investigating Polar Exploration Rover payload,” in _Lunar and Planetary Science Conference_ , 2020.
  * [2] A. Colaprete, R. C. Elphic, M. Shirley, K. Ennico-Smith, D. S. S. Lim, Z. Zacny, and J. Captain, “The Volatiles Investigating Polar Exploration Rover (VIPER) mission – measurements and constraints,” in _Lunar and Planetary Science Conference_ , 2021.
  * [3] National Academies of Sciences, Engineering, and Medicine, “Origins, worlds, and life: A decadal strategy for planetary science and astrobiology 2023–2032.” National Academy Press, Tech. Rep., 2022.
  * [4] J. T. Keane, S. M. Tikoo, and J. Elliott, “Endurance: Lunar South Pole-Atken Basin traverse and sample return rover,” National Academy Press, Tech. Rep., 2022.
  * [5] L. Matthies, S. Daftry, S. Tepsuporn, Y. Cheng, D. Atha, R. M. Swan, S. Ravichandar, and M. Ono, “Lunar rover localization using craters as landmarks,” in _IEEE Aerospace Conference_ , 2022.
  * [6] S. Daftry, Z. Chen, Y. Cheng, S. Tepsuporn, B. Coltin, U. Naam, M. M. Lanssie, S. Khattak, M. Deans, and L. Matthies, “LunarNav: Crater-based localization for long-range autonomous rover navigation,” in _IEEE Aerospace Conference_ , 2023.
  * [7] H. Hiesinger, C. H. van der Bogert, J. H. Pasckert, L. Funcke, L. Giacomini, L. R. Ostrach, and M. S. Robinson, “How old are young lunar craters?” _Journal of Geophysical Research_ , vol. 117, no. 12, pp. 1–15, 2012.
  * [8] M. S. Robinson, S. M. Brylow, M. Tschimmel, D. Humm, S. J. Lawrence, P. C. Thomas, B. W. Denevi, E. Bowman-Cisneros, J. Zerr, M. A. Ravine, M. A. Caplinger, F. T. Ghaemi, J. A. Schaffner, M. C. Malin, P. Mahanti, A. Bartels, J. Anderson, T. N. Tran, E. M. Eliason, A. S. McEwen, E. Turtle, B. L. Jolliff, and H. Hiesinger, “Lunar Reconnaissance Orbiter (LROC) camera instrument overview,” _Space Science Reviews_ , vol. 150, 2010.
  * [9] E. Cisneros, A. Awumah, H. M. Brown, A. C. Martin, K. N. Paris, R. Z. Povilaitis, A. K. Boyd, M. S. Robinson, and LROC Team, “Lunar Reconnaissance Orbiter camera permanently shadowed region imaging – atlas and controlled mosaics,” in _Lunar and Planetary Science Conference_ , 2017.
  * [10] Blender - a 3D modelling and rendering package. Software available from https://www.blender.org/.
  * [11] J. V. Hook, R. Schwartz, K. Ebadi, K. Coble, and C. Padgett, “Topographical landmarks for ground-level terrain relative navigation on Mars,” in _IEEE Aerospace Conference_ , 2022.
  * [12] K. Ebadi, K. Coble, D. Atha, R. Schwartz, C. Padgett, and J. V. Hook, “Semantic mapping in unstructured environments: Toward autonomous localization of planetary robotic explorers,” in _IEEE Aerospace Conference_ , 2022.
  * [13] J. W. Hwangbo, K. Di, and R. Li, “Integration of orbital and ground image networks for the automation of rover localization,” in _American Society for Photogrammetry and Remote Sensing Annual Conference_ , 2009.
  * [14] D. Hurley, P. Prem, A. Stickle, C. Hibbitts, A. Deutsch, A. Colaprete, R. Elphic, S. Li, P. Lucey, Y. Liu, S. Hosseini, K. D. Retherford, K. Zacny, J. Atkinson, M. Benna, W. Farrell, D. Needham, L. Gertsch, M. Delitsky, and P. Hayne, “Science from the lunar permanently shadowed regions,” in _Lunar and Planetary Science Conference_ , 2020.
  * [15] B. Wu, R. W. K. Potter, P. Ludivig, A. S. Chung, and T. Seabrook, “Absolute localization through orbital maps and surface perspective imagery: A synthetic lunar dataset and neural network approach,” in _IEEE/RSJ Int. Conf. on Intelligent Robots & Systems_, 2019.
  * [16] V. Franchi and E. Ntagiou, “Planetary rover localisation via surface and orbital image matching,” in _IEEE Aerospace Conference_ , 2022.
  * [17] A. Howard, A. I. Ansar, T. E. Litwin, and S. B. Goldberg. (2009) Jet Propulsion Laboratory Stereo Vision software suite (JPLV). Software available from https://software.nasa.gov/software/NPO-18593-1T.
  * [18] H. Hirschmuller, “Stereo processing by semiglobal matching and mutual information,” _IEEE Transactions on Pattern Analysis & Machine Intelligence_, vol. 30, no. 2, pp. 328–341, 2008.
  * [19] J. Canny, “A computational approach to edge detection,” _IEEE Transactions on Pattern Analysis & Machine Intelligence_, vol. 8, no. 6, pp. 679–698, 1986.
  * [20] S. Xie and Z. Tu, “Holistically-nested edge detection,” in _IEEE Int. Conf. on Computer Vision_ , 2015.
  * [21] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking,” _IEEE Transactions on Signal Processing_ , vol. 50, no. 2, pp. 174–188, 2002.
  * [22] C. Gentner, S. Zhang, and T. Jost, “Log-PF: Particle filtering in logarithm domain,” _Journal of Electrical and Computer Engineering_ , vol. 2018, 2018\.
  * [23] B. Hapke, _Theory of reflectance and emittance spectroscopy_. Cambridge Univ. Press, 2012.
  * [24] ——, “Bidirectional reflectance spectroscopy: 5. the coherent backscatter opposition effect and anisotropic scattering,” _Icarus_ , vol. 157, no. 2, pp. 523–534, 2002.
  * [25] F. Schmidt and S. Bourguignon, “Efficiency of BRDF sampling and bias on the average photometric behavior,” _Icarus_ , vol. 317, pp. 10–26, 2019.
  * [26] X. Xu, J. Liu, D. Liu, B. Liu, and R. Shu, “Photometric correction of Chang’E-1 interference imaging spectrometer’s (IIM) limited observing geometries data with Hapke model,” _Remote Sensing_ , vol. 12, no. 22, p. 3676, 2020.
  * [27] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” _IEEE Transactions on Signal Processing_ , vol. 57, no. 2, 2009.
  * [28] L. Carlone and S. Karaman, “Attention and anticipation in fast visual-inertial navigation,” _IEEE Transactions on Robotics_ , vol. 35, no. 1, 2019.
  * [29] T. Li, M. Bolić, and P. M. Djurić, “Resampling methods for particle filtering,” _IEEE Transactions on Signal Processing_ , vol. 32, no. 3, pp. 70–86, 2015.
  * [30] C. Kuptametee and N. Aunsri, “A review of resampling techniques in particle filtering framework,” _Measurement_ , vol. 193, 2022.

Abhishek Cauligifig/headshots/cauligi.png is a Robotics Technologist in the
Robotic Surface Mobility Group at NASA Jet Propulsion Laboratory, California
Institute of Technology. Abhishek received his B.S. in Aerospace Engineering
from the University of Michigan - Ann Arbor and his PhD. in Aeronautics and
Astronautics from Stanford University. His research interests lie in
leveraging recent advances in nonlinear optimization, machine learning, and
control theory towards planning and control for complex robotic systems.

R. Michael Swanfig/headshots/swan.jpeg is a Robotics Systems Engineer at NASA
Jet Propulsion Laboratory, California Institute of Technology. He received his
B.S. in Computer Engineering from Walla Walla University and his M.S. in
Computer Science from the University of Southern California. He is interested
in robotic surface and aerial autonomy, perception, simulation, and robotic
system architecture.

Hiro Onofig/headshots/ono.jpg is the Group Leader of the Robotic Surface
Mobility Group at NASA Jet Propulsion Laboratory, California Institute of
Technology. Since he joined JPL in 2013, he has led a number of research
projects on Mars rover autonomy, as well as three NIAC studies on Enceladus
Vent Explorer and Comet Hitchhiker. Hiro was a flight software developer of
M2020’s Enhanced AutoNav and the lead of M2020 Landing Site Traversability
Analysis. He also led the development of a machine learning-based Martian
terrain classifier, SPOC (Soil Property and Object Classification), which won
JPL’s Software of the Year Award in 2020.

Shreyansh Daftryfig/headshots/daftry.jpg is a Robotics Technologist at NASA
Jet Propulsion Laboratory, California Institute of Technology. He received his
M.S. degree in Robotics from the Robotics Institute, Carnegie Mellon
University, and his B.S. degree in Electronics and Communication Engineering.
His research interest lies at the intersection of space technology and
autonomous robotic systems, with an emphasis on machine learning applications
to perception, planning, and decision making. At JPL, he is the Group Leader
of the Perception Systems group, is working on the Mars Sample Recovery
Helicopter mission, and has led/contributed to technology development for
autonomous navigation of ground, airborne, and subterranean robots.

John Elliottfig/headshots/elliott.jpg is a principal engineer in JPL’s Mission
Concept Systems Development group. He currently serves as Program Engineer for
the Planetary Science Formulation office. His recent tasks have included
serving as study lead for the Planetary Decadal Survey’s three lunar rover
mission concept studies, Intrepid, INSPIRE, and Endurance, and performing
systems engineering and leadership roles on a number of recent Discovery and
New Frontiers mission proposals. Mr. Elliott’s past experience includes six
years in the terrestrial nuclear power industry with Bechtel Corporation in
addition to 30 years in aerospace systems at TRW and JPL.

Larry Matthiesfig/headshots/larry.png is the technology co- ordinator in the
Mars Exploration Pro- gram Office at JPL. He received B.S., M. Math,and PhD
degrees in Computer Science from the University of Regina (1979), University
of Waterloo (1981), and Carnegie Mellon University (1989). He has been with
JPL for more than 32 years. He has conducted technology development in
perception systems for autonomous navigation of robotic vehicles for land,
sea, air, and space. He supervised the JPL Computer Vision group for 21 years.
He led development of computer vision algorithms for Mars rovers, landers, and
helicopters. He is a Fellow of the IEEE and a member of the editorial boards
of Autonomous Robots and the Journal of Field Robotics.

Deegan Athafig/headshots/deegan.jpg is a Robotics Technologist within the
Perception Systems Group of the Mobility and Robotic Systems Section at the
Jet Propulsion Laboratory. He received his B.S. degree from Purdue University
in Electrical Engineering and his M.S. in Computer Science from the Georgia
Institute of Technology. He is currently the Principal Investigator for the
ShadowNav task and leading the semantic perception effort for the DARPA RACER
project. His interests are in the infusion of machine learning and robotic
perception into autonomous systems operating in unstructured environments.