11institutetext: School of Electrical and Electronic Engineering, Yonsei
University, Seoul, Korea
11email<EMAIL_ADDRESS>
# Decomposition of Neural Discrete Representations for Large-Scale 3D Mapping

Minseong Park  0000-1111-2222-3333 Suhan Woo  0009-0008-1947-750X Euntai Kim
Corresponding author 0000-0002-0975-8390

###### Abstract

Learning efficient representations of local features is a key challenge in
feature volume-based 3D neural mapping, especially in large-scale
environments. In this paper, we introduce Decomposition-based Neural Mapping
(DNMap), a storage-efficient large-scale 3D mapping method that employs a
discrete representation based on a decomposition strategy. This decomposition
strategy aims to efficiently capture repetitive and representative patterns of
shapes by decomposing each discrete embedding into component vectors that are
shared across the embedding space. Our DNMap optimizes a set of component
vectors, rather than entire discrete embeddings, and learns composition rather
than indexing the discrete embeddings. Furthermore, to complement the mapping
quality, we additionally learn low-resolution continuous embeddings that
require tiny storage space. By combining these representations with a shallow
neural network and an efficient octree-based feature volume, our DNMap
successfully approximates signed distance functions and compresses the feature
volume while preserving mapping quality. Our source code is available at
https://github.com/minseong-p/dnmap.

###### Keywords:

Large-scale 3D Mapping Neural discrete representation

##  1 Introduction

Figure 1: The comparison between indexing [27] and our proposed decomposition
strategy. Indexing requires exploring the entire embedding space ($2^{B}$) to
optimize discrete representations. In contrast, our approach only needs to
learn $B$ indicators.

Accurate and dense mapping in large-scale 3D environments plays a crucial role
in the fields of computer vision and robotics, with a wide range of
applications ranging from autonomous navigation to virtual environments. One
of the most commonly used sensors for large-scale 3D mapping is LiDAR, known
for its ability to provide accurate geometry and robustness to illumination
changes. Due to these advantages, LiDAR is widely used for mapping in large-
scale outdoor environments [10, 16, 19]. A key practical challenge is
balancing geometric accuracy with the memory consumption of the map. Since
vehicles and mobile robots often have limited resources, including memory and
processing power, efficient representation of maps is crucial to ensure that
the map remains usable in these resource-constrained scenarios.

In recent years, implicit neural representations in 3D reconstruction have
gained significant attention. Implicit neural representations learn a neural
network to approximate implicit functions, such as signed distance functions
(SDFs), enabling accurate, high-fidelity reconstruction using compact
representations [20, 1, 22, 28, 15]. Due to these advantages, there have been
several attempts to leverage implicit neural representations for 3D mapping
using [31, 1, 35, 8]. However, most of these methods only work in relatively
small-scale indoor environments due to the limited capacity of neural networks
or computational costs.

When it comes to large-scale outdoor environments, a few works have been
reported. They have achieved some success in large-scale 3D mapping by
learning and storing local features through efficiently designed feature
volumes [34, 3, 6, 25]. However, they focused on designing the structure of
the feature volume and relatively overlooked how to efficiently represent
local features themselves. Understandably, as the scale of the environment
increases, reducing the storage of each local feature becomes as critical as
an efficient structure of the feature volume.

In this paper, we propose Decomposition-based Neural Mapping (DNMap) to
successfully adopt discrete representation in storage-efficient large-scale 3D
mapping. Our DNMap approximates SDFs based on a decomposition-based discrete
representation and octree-based structure. As shown in Fig. 1, we learn the
decomposition and composition of the discrete embedding space, different from
the existing discrete representation proposed in [27]. We decompose each
discrete embedding into component vectors that are shared across the embedding
space. Then, we learn how to compose the component vectors using composition
indicators, which can be expressed as binary vectors. The process of
composition is designed to have the same effect as indexing into
multidimensional binary vectors, which are more computationally efficient than
one-hot vectors, and the binary vectors are stored via an octree-based feature
volume. Along with these techniques, our DNMap reconstructs accurate and
compact large-scale outdoor environment maps by simultaneously optimizing the
component vector set, the octree-based feature volume, and the decoder
parameters.

Our main contributions are summarized as follows: 1) The proposed discrete
representation based on a decomposition strategy significantly reduces the
storage of the feature volume compared to continuous representation, while
preserving mapping quality; 2) Learning composition, rather than indexing
directly, reduces memory usage during training, enabling practical utilization
of discrete representation in large-scale 3D mapping.

##  2 Related Works

###  2.1 Implicit Neural Representations

Implicit neural representations approximate implicit surfaces as neural
networks, enabling compact representation of complex environments. Early works
on implicit neural representations represent an environment using a large,
fixed single neural network [20, 1, 22, 28, 30, 14, 33]. Although there is an
advantage that the amount of computation is constant regardless of the scale
of the environment, the large amount of computations for inference was pointed
out as a limitation.

Recent works are based on the combination of a feature volume with a shallow
neural network for scalable representations or real-time rendering [31, 1, 35,
8, 12]. Most of these existing works have focused on representing objects or
relatively small-scale indoor environments. There are several studies [9, 21,
24, 11] on implicit representation in large-scale outdoor environments using
LiDAR, but they focus on representation in single frames.

Implicit neural mapping, which aims to reconstruct 3D or visual maps using
implicit neural representations is a relatively new task. Incremental mapping
using a single neural network [31, 32] aims at representing sequential data in
a replay-based manner. For large-scale 3D mapping using implicit neural
representations, a few works [34, 3, 6, 25] focus on designing efficient data
structures and have shown some success. However, we still think there are ways
to represent features more efficiently, and this inspires our work.

###  2.2 Neural Discrete Representation

VQ-VAE [17] was reported as a pioneering work to learn neural discrete
representations. In this framework, the encoder outputs discrete latent codes
via vector quantization (VQ), encouraging the network to generate meaningful
latent codes, thereby enhancing the fidelity of reconstruction. Motivated by
the VQ framework, VQ-AD [27] was reported to compress feature volumes for an
encoder-less framework, which is widely adopted in implicit representations.
Despite its success in feature volume compression, VQ-AD raises memory usage
issues during training because indexing discrete representations requires
exploring the entire embedding space. This challenge greatly motivates our
work.

##  3 Preliminaries

In this paper, we follow the large-scale 3D mapping framework proposed in
SHINE-Mapping [34]. Given a 3D environment
$\mathcal{W}\subset{\mathbb{R}^{3}}$ sequentially observed by LiDAR in known
poses, SHINE-Mapping approximates signed distance functions
$d=\mathit{SDF}\left(\mathbf{x}\right)$ that map a 3D coordinate
$\mathbf{x}\in\mathcal{W}$ to the shortest signed distance $d\in\mathbb{R}$
from the input coordinate to the surface.

Specifically, it consists of the following parts: an octree $\mathcal{O}$ with
height $L$ is built from a given 3D environment $\mathcal{W}$; learnable
embeddings are stored in voxel corners of an octree-based structure; given a
sampled query point $\mathbf{x}$, the voxels to which the query point belongs
are determined at each level, and the query feature
$\mathbf{z}\in\mathbb{R}^{D}$ is determined as the sum of the embeddings
interpolated from the voxel corners of each level, where $D$ indicates the
dimension of the embedding space; the query feature $\mathbf{z}$ is decoded
into a signed distance value through a decoder $f_{\Theta}$ which is a shallow
MLP parameterized by $\Theta$. Then, the entire process
$\Phi:\mathcal{W}\to\mathbb{R}$ can be represented as:

$\mathbf{z}=\sum\limits_{l=0}^{L-1}{\mathit{interp}\left(\mathbf{x};\mathcal{O}_{l}\right)},$
(1) $d=\Phi\left(\mathbf{x}\right)=f_{\Theta}\left(\mathbf{z}\right),$ (2)

where $\mathcal{O}_{l}$ is the set of embeddings of octree level $l$;
$\mathit{interp}$ is a trilinear interpolation operator. Next, we will
describe the details of the octree-based feature volume and the loss
functions.

####  3.0.1 Octree-Based Feature Volume.

For a determined octree height $L$, SHINE-Mapping build an octree
$\mathcal{O}$ from observed surfaces, allocating only the voxels that contain
surfaces. The octree at each level consists of sparse voxels with sizes
corresponding to each level. Based on the voxel size of the highest level, the
lower the level, the larger the voxel size. The locations of voxels are stored
as Morton codes. To speed up the mapping process, a hash table from voxels to
the eight surrounding corners is maintained during mapping, and after mapping
is complete, the hash table is removed to free storage space. After the octree
is built, learnable embedding vectors are stored in the voxel corners.

####  3.0.2 Loss Functions.

The loss function $\mathcal{L}$ consists of the SDF loss function
$\mathcal{L}_{\mathit{sdf}}$ and the Eikonal loss function $\mathcal{L}_{E}$.
First, a binary cross entropy loss with a sigmoid function is applied as the
SDF loss function $\mathcal{L}_{\mathit{sdf}}$ to supervise SDF prediction:

$\mathcal{L}_{\mathit{sdf}}\left(\Phi\left(\mathbf{x}\right),\mathit{SDF}\left(\mathbf{x}\right)\right)={\mathit{BCE}\left(\sigma\left(\Phi\left(\mathbf{x}\right)\right),\sigma\left(\frac{\mathit{SDF}\left(\mathbf{x}\right)}{s}\right)\right)},$
(3)

where
$\mathit{BCE}\left(x,y\right)=y\log{x}+\left(1-y\right)\log{\left(1-x\right)}$
is the binary cross entropy loss, and $s$ is a hyper-parameter to control the
flatness of the sigmoid function
$\sigma\left(x\right)=1/\left(1+e^{-x}\right)$. This loss function emphasizes
the impact of the query points sampled near the surfaces. Second, the Eikonal
loss function $\mathcal{L}_{e}$ is adopted to encourage the L2-norm of the
gradient at the query point $\mathbf{x}$ to equal one [5], since the gradient
of SDF is a unit normal vector. The Eikonal loss function $\mathcal{L}_{e}$ is
defined as:

$\mathcal{L}_{e}\left(\Phi\left(\mathbf{x}\right)\right)=\left({\lVert{\nabla\Phi\left(\mathbf{x}\right)}\rVert}_{2}-1\right)^{2}.$
(4)

Consequently, the final loss $\mathcal{L}$ is defined as:

$\mathcal{L}\left(\Phi\left(\mathbf{x}\right),\mathit{SDF}\left(\mathbf{x}\right)\right)=\mathcal{L}_{\mathit{sdf}}\left(\Phi\left(\mathbf{x}\right),\mathit{SDF}\left(\mathbf{x}\right)\right)+\lambda\mathcal{L}_{e}\left(\Phi\left(\mathbf{x}\right)\right),$
(5)

where $\lambda=0.1$ is weight of the Eikonal loss function $\mathcal{L}_{e}$.

##  4 Method

Figure 2: The pipeline of our DNMap. When a query point $\mathbf{x}$ is given,
the voxels to which the query point belongs are determined at each level of
the octree $\mathcal{O}$. For the determined discrete embedding voxels at each
level, the voxel corner embeddings are composed from the component vector set
$\mathcal{E}$ according to the composition indicators stored in the corners.
The query embeddings at each level
$\mathbf{z}_{0},\cdot\cdot\cdot,\mathbf{z}_{L-1}\mathbf{z}_{c}$ are determined
via trilinear interpolation of the voxel corner embeddings. Finally, the query
embedding $\mathbf{z}$ is determined as the sum of all interpolated features,
and the decoder $f_{\Theta}$ takes the query embedding as input and outputs a
signed distance value.

Advancing from SHINE-Mapping, we introduce Decomposition-based Neural Mapping
(DNMap) that learns discrete embeddings based on a decomposition strategy. Our
DNMap differs from SHINE-Mapping in two key ways: composition indicators,
rather than continuous embeddings, are stored in the voxel corners of the
octree $\mathcal{O}$; a corner embedding becomes $\mathbf{e}_{k}$, composed
from the component vector set $\mathcal{E}$. Additionally, we store low-
resolution continuous embeddings in the voxel corners at the $0$-level of the
octree to complement global location information. With these embeddings, the
query feature $\mathbf{z}$ is determined as the sum of discrete and continuous
terms, different from Eq. (1):

$\mathbf{z}=\underbrace{\sum\limits_{l=0}^{L-1}{\textit{interp}\left(\mathbf{x};\mathcal{D}_{l}\right)}}_{\text{discrete}}+\underbrace{\textit{interp}\left(\mathbf{x};\mathcal{C}\right)}_{\text{continuous}},$
(6)

where $\mathcal{D}_{l}$ is a set of decomposition-based discrete embeddings
which are composed from the component vector set $\mathcal{E}$ at level $l$;
$\mathcal{C}$ is a set of low-resolution continuous embeddings. The framework
of our DNMap is illustrated in Fig. 2.

Finally, the objective of our DNMap is to simultaneously optimize the discrete
feature volume $\mathcal{D}$, low-resolution continuous feature volume
$\mathcal{C}$, the component vector set $\mathcal{E}$, and the decoder
parameter $\Theta$ by minimizing the final loss $\mathcal{L}$, and can be
represented as:

$\underset{\mathcal{D},\mathcal{C},\mathcal{E},\Theta}{\mathop{\arg\min}}{\frac{1}{|\mathcal{W}|}\sum\limits_{\mathbf{x}\in\mathcal{W}}{\mathcal{L}\left(\Phi\left(\mathbf{x}\right),\mathit{SDF}\left(\mathbf{x}\right)\right)}}.$
(7)

###  4.1 Decomposition-Based Neural Discrete Representation

We define the discrete embedding space
$\left\\{\mathbf{e}_{i}\in\mathbb{R}^{D}\right\\}_{i=0}^{2^{B}-1}$ at bitwidth
$B$, where $\mathbf{e}_{i}$ is the $i$-th discrete embedding. A simple way to
optimize the embedding space is to learn differentiable indices whose
dimension is the size of the embedding space [27]. However, this approach has
a problem when applied to large-scale 3D mapping because it requires extensive
computation to explore the entire embedding space. To solve this problem, we
adopt a simple decomposition strategy.

####  4.1.1 Decomposition.

We decompose the discrete embedding space into a set of $B+1$ component
vectors, which consists of one embedding bias
$\mathbf{e}_{b}\in\mathbb{R}^{D}$ and $B$ embedding offsets
$\Delta{\mathbf{e}}\in\mathbb{R}^{D}$ that are shared across the embedding
space. Then the component vector set $\mathcal{E}\subset{\mathbb{R}^{D}}$ is
defined as:

$\mathcal{E}=\left\\{\mathbf{e}_{b},\Delta{\mathbf{e}}_{0},\cdots,\Delta{\mathbf{e}}_{B-1}\right\\}.$
(8)

####  4.1.2 Composition.

We represent a discrete embedding $\mathbf{e}$ as a composition of the
component vectors, and the composition process is formulated as:

$\mathbf{e}=\mathbf{e}_{b}+\sum\limits_{j=0}^{B-1}{{b}_{j}\Delta{\mathbf{e}}_{j}},$
(9)

where $b_{j}\in\left\\{0,1\right\\}$ is the $j$-th composition indicator. To
update the $j$-th embedding offset $\Delta{\mathbf{e}}_{j}$ by gradient
descent, the gradient can be calculated using the chain rule:

$\frac{\partial\mathcal{L}}{\partial\Delta{\mathbf{e}}_{j}}=\frac{\partial\mathcal{L}}{\partial
d}\frac{\partial
d}{\partial\mathbf{e}}\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}_{j}}.$
(10)

Since the term $\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}_{j}}$ in
Eq. (10) is not defined when $b_{j}=0$, we reformulate Eq. (9) as follows so
that each embedding offset is updated frequently:

$\begin{split}\mathbf{e}&=\underbrace{\mathbf{e}^{\prime}_{b}+\sum\limits_{j=0}^{B-1}{\Delta{\mathbf{e}}^{0}_{j}}}_{\mathbf{e}_{b}}+\sum\limits_{j=0}^{B-1}{{b}_{j}\underbrace{\left(\Delta{\mathbf{e}}^{1}_{j}-\Delta{\mathbf{e}}^{0}_{j}\right)}_{\Delta{\mathbf{e}}_{j}}}\\\
&=\mathbf{e}^{\prime}_{b}+\sum\limits_{j=0}^{B-1}{\left\\{\left(1-{b}_{j}\right)\Delta{\mathbf{e}}^{0}_{j}+{b}_{j}\Delta{\mathbf{e}}^{1}_{j}\right\\}}.\end{split}$
(11)

Then, the term $\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}_{j}}$ can
be defined as follows, regardless of $b_{j}$:

$\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}_{j}}=\begin{cases}\displaystyle\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}^{1}_{j}}\frac{\partial\Delta{\mathbf{e}}^{1}_{j}}{\partial\Delta{\mathbf{e}}_{j}}&\text{if
}b_{j}=1\\\
\displaystyle\frac{\partial\mathbf{e}}{\partial\Delta{\mathbf{e}}^{0}_{j}}\frac{\partial\Delta{\mathbf{e}}^{0}_{j}}{\partial\Delta{\mathbf{e}}_{j}}&\text{if
}b_{j}=0\end{cases}.$ (12)

This approach helps the embedding space converge to an optimal representation
of a large-scale 3D environment.

####  4.1.3 Composition indicator.

We define a composition indicator vector that indicates which component vector
to compose, which can be expressed as a binary vector
$\mathbf{b}={\begin{bmatrix}b_{0}&\cdots&b_{B-1}\end{bmatrix}}^{T}\in\mathbb{R}^{B}$.
Actually, the real gradient of a binary vector $\mathbf{b}$ is not defined
because the binary vector is deterministic and non-differentiable. To
approximate the gradient of the binary vector $\mathbf{b}$, we adopt a
straight-through estimator approach [2] using a sigmoid function
$\sigma\left(\cdot\right)$ and an indicator function
$\mathds{1}_{\mathbb{R}^{+}}\left(\cdot\right)$. The indicator function
$\mathds{1}_{\mathbb{R}^{+}}:\mathbb{R}\to\left\\{0,1\right\\}$ is a function
that outputs one if $x>0$, and zero otherwise. Specifically, we first store
the randomly initialized differentiable continuous vectors $\mathbf{v}$ in the
octree $\mathcal{O}$. When an index is called, we generate two types of
vectors from the vector $\mathbf{v}$: non-differentiable binary vector
$\mathds{1}_{\mathbb{R}^{+}}\left(\mathbf{v}\right)$ and differentiable vector
$\sigma\left(\mathbf{v}\right)$. Then, the binary vector $\mathbf{b}$ is
calculated as:

$\mathbf{b}=\text{sg}\left[\mathds{1}_{\mathbb{R}^{+}}\left(\mathbf{v}\right)-\sigma\left(\mathbf{v}\right)\right]+\sigma\left(\mathbf{v}\right),$
(13)

where $\text{sg}\left[\cdot\right]$ the indicates stop-gradient operator,
which is defined to treat a specific variable as a constant. This simple
technique allows us to optimize the binary vector $\mathbf{b}$ via gradient
descent.

####  4.1.4 Implementation.

The composition process in Eq. (11) is implemented using a linear layer with
$\mathbf{W}={\begin{bmatrix}\Delta{\mathbf{e}}^{0}_{0}&\cdots&\Delta{\mathbf{e}}^{0}_{B-1}&\Delta{\mathbf{e}}^{1}_{0}&\cdots&\Delta{\mathbf{e}}^{1}_{B-1}\end{bmatrix}}\in\mathbb{R}^{D\times{2B}}$
as the weight matrix and $\mathbf{e}^{\prime}_{b}$ as the bias as follows:

$\mathbf{e}=\mathbf{W}\mathbf{b}^{*}+\mathbf{e}^{\prime}_{b}$ (14)

where $\mathbf{b}^{*}\in\mathbb{R}^{2B}$ is the concatenation of two vectors,
$1-\mathbf{b}$ and $\mathbf{b}$.

###  4.2 Low-Resolution Continuous Embedding

In order to complement mapping quality, we additionally learn low-resolution
continuous embedding $\mathbf{c}\in\mathbb{R}^{D}$ which are stored in voxel
corners at the 0-level octree to help the discrete embeddings that lack global
location information. Because the number of voxels in the 0-level octree is a
very small percentage of the number of voxels in all levels, it does not
significantly affect the overall storage. For example, when building an octree
with a height of 3 and a smallest voxel size of 0.2 m, the ratio of the number
of 0-level voxels to the number of total level voxels in the MaiCity [29] and
NewerCollege [23] datasets are 4.7% and 3.7%, respectively.

###  4.3 Efficient Implementation

Figure 3: Comparison between (a) basic implementation and (b) efficient
implementation. In most cases during training, (b) is more efficient than (a)
because $N_{V}>>N_{Q}$, where $N_{V}$ is the number of total voxels, and
$N_{Q}$ is the number of query points processed at once.

Because the region where the query points in the batch are located is part of
a large-scale 3D environment, it is inefficient to compose all voxel corners
every time during training. To address this problem, we first interpolate the
binary vectors of the corners of the voxel to which the query point belongs
and then perform indexing. Since the processes described in Eq. (6) and Eq.
(14) are linear, the output query features are the same. This approach is more
efficient because the number of query points in one batch is generally less
than the total number of voxels. The $l$-level query feature with efficient
implementation can be calculated as:

$\mathbf{z}_{l}=\mathbf{W}\textit{interp}\left(\mathbf{x};\mathcal{B}^{*}_{l}\right)+\mathbf{e}^{\prime}_{b},$
(15)

where $\mathcal{B}^{*}_{l}$ is the set of $\mathbf{b}^{*}$ at level $l$. This
technique allows accessing the component vector set as many times as only the
number of query points, which reduces redundant composition when querying
features. Therefore, the execution time is reduced in most cases during
training because the total number of voxels is much greater than the number of
query points processed at once. A comparison between basic and efficient
implementation is illustrated in Fig. 3.

##  5 Experiments

###  5.1 Experimental Setting

####  5.1.1 Datasets.

To validate our DNMap, we use two publicly available outdoor LiDAR datasets
that provide ground truth mesh: MaiCity [29], and Newer College [23]. MaiCity
consists of 64-beam noise-free synthetic LiDAR scans in urban roads. Newer
College consists of handheld LiDAR scans collected at Oxford University with
cm-level noise and motion distortion.

####  5.1.2 Data Preparation.

To train SDFs, we prepare pairs of query points and SDF labels following
SHINE-Mapping [34]. We sample query points along the ray of LiDAR point clouds
and use the signed distance values from sampled query points to the endpoints
of the rays as labels. Since our goal is to reconstruct surfaces, the query
points sampled near the surface are more important than those sampled in free
space. Therefore, we sample the query points near the surface and the query
points in free space separately. Specifically, for each LiDAR ray, $N_{F}$
query points are uniformly sampled in free space, and $N_{S}$ query points
inside the truncation band $\pm 3s$ are sampled near the surfaces, where $s$
is a hyperparameter as described in Eq. (3).

####  5.1.3 Evaluation Metric.

To compare methods with each other, we measure metrics in terms of mapping
quality. We use the commonly used 3D reconstruction metrics adopted in most
works [34]. We evaluate accuracy, completion, Chamfer-L1 distance, and
F-score, obtained by comparing the reconstructed mesh with the ground truth
mesh. The reconstructed mesh can be generated from learned implicit neural
representations using marching cubes [13]. We generate explicit mesh with a
voxel size of 10 cm. Furthermore, we report the storage of neural
representations and total storage, including the spatial model.

####  5.1.4 Implementation Details.

For the spatial model, we set the height of the octree to $L=3$. For the
representation, we set the embedding space dimension to $D=8$. For data
preparation, we set $N_{F}=6$, $N_{S}=3$, and $s=0.05$ in MaiCity [29] and
$N_{F}=6$, $N_{S}=3$, and $s=0.1$ in Newer College [23]. The decoder is
implemented as a MLP containing two 32-dimensional hidden layers with ReLU
activation. All methods are trained for 20000 iterations with 8192 batch size.
An Adam optimizer is used with an initial learning rate 0.01, which decays to
0.001 after 10000 iterations.

###  5.2 Experimental Results

To verify the mapping quality and storage efficiency of our method, we compare
our DNMap with SHINE-Mapping [34] and VQ-SHINE-Mapping [34, 27]. Based on the
SHINE-Mapping framework, VQ-SHINE-Mapping [34, 27] is a method of replacing
the original continuous embedding with the discrete embedding proposed in VQ-
AD [27]. We also report the results of DNMap-discrete, which does not use low-
resolution continuous embedding, to verify the effectiveness of the proposed
discrete representation. All methods were evaluated in the same experimental
setting and GPU, changing only the bitwidth from 4 to 8 when discrete
embedding is used.

####  5.2.1 Quantitative Comparisons.

Table 1: Quantitative comparisons on MaiCity [29] dataset. The error threshold for calculating F-score is 10 cm. |  | Mapping Quality |  | Storage(kB) |  |  |  |
---|---|---|---|---|---|---|---|---
Method |  |  Acc.(cm)$\downarrow$ |  Com.(cm)$\downarrow$ |  C-l1(cm)$\downarrow$ |  F-score$\uparrow$ |  | Rep. | Total |  | Time |
Discrete (4-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping [34, 27] |  | 4.57 | 15.05 | 9.81 | 86.07 |  | 249 | 1937 |  | 31m | 59s |
DNMap-discrete |  | 4.11 | 12.44 | 8.28 | 87.14 |  | 248 | 1936 |  | 30m | 04s |
Discrete (6-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping [34, 27] |  | 4.29 | 15.47 | 9.88 | 87.33 |  | 376 | 2064 |  | 34m | 40s |
DNMap-discrete |  | 4.12 | 12.21 | 8.16 | 87.64 |  | 371 | 2059 |  | 30m | 45s |
Discrete (8-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping [34, 27] |  | 4.19 | 15.38 | 9.79 | 87.90 |  | 519 | 2207 |  | 43m | 40s |
DNMap-discrete |  | 4.08 | 12.42 | 8.25 | 87.63 |  | 496 | 2184 |  | 30m | 57s |
Non-discrete |  |  |  |  |  |  |  |  |  |  |  |
SHINE-Mapping [34] |  | 3.19 | 17.75 | 10.47 | 89.23 |  | 15775 | 17463 |  | 30m | 03s |
DNMap (4-bit) |  | 3.13 | 17.52 | 10.33 | 89.45 |  | 1089 | 2777 |  | 35m | 27s |
DNMap (6-bit) |  | 3.11 | 16.96 | 10.04 | 89.55 |  | 1212 | 2900 |  | 35m | 46s |
DNMap (8-bit) |  | 3.14 | 16.46 | 9.80 | 89.56 |  | 1337 | 3025 |  | 36m | 00s |
Table 2: Quantitative comparisons on Newer College [23] dataset. The error threshold for calculating F-score is 20 cm. ⋆ denotes out-of-memory. |  | Mapping Quality |  | Storage(kB) |  |  |  |
---|---|---|---|---|---|---|---|---
Method |  |  Acc.(cm)$\downarrow$ |  Com.(cm)$\downarrow$ |  C-l1(cm)$\downarrow$ |  F-score$\uparrow$ |  | Rep. | Total |  | Time |
Discrete (4-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping [34, 27] |  | 8.57 | 10.10 | 9.78 | 91.31 |  | 662 | 6458 |  | 42m | 33s |
DNMap-discrete |  | 7.69 | 11.01 | 9.35 | 92.86 |  | 661 | 6457 |  | 36m | 24s |
Discrete (6-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping [34, 27] |  | 8.00 | 10.70 | 9.35 | 92.01 |  | 995 | 6791 |  | 49m | 06s |
DNMap-discrete |  | 7.57 | 10.76 | 9.16 | 92.83 |  | 990 | 6786 |  | 36m | 46s |
Discrete (8-bit) |  |  |  |  |  |  |  |  |  |  |  |
VQ-SHINE-Mapping⋆ [34, 27] |  | - | - | - | - |  | - | - |  | - |
DNMap-discrete |  | 7.24 | 10.93 | 9.08 | 93.35 |  | 1321 | 7117 |  | 37m | 27s |
Non-discrete |  |  |  |  |  |  |  |  |  |  |  |
SHINE-Mapping [34] |  | 7.42 | 10.47 | 8.94 | 92.96 |  | 42187 | 47983 |  | 35m | 58s |
DNMap (4-bit) |  | 6.76 | 10.50 | 8.63 | 93.63 |  | 2207 | 8003 |  | 44m | 34s |
DNMap (6-bit) |  | 6.58 | 10.50 | 8.54 | 93.75 |  | 2536 | 8332 |  | 45m | 12s |
DNMap (8-bit) |  | 6.68 | 10.64 | 8.67 | 93.60 |  | 2867 | 8663 |  | 45m | 27s |

Tab. 1 and Tab. 2 show the 3D reconstruction results on the MaiCity [29]
dataset and Newer College [23] dataset, respectively. We conducted the
experiments by setting all methods to share the same spatial model with an
octree with a height of 3 and a voxel size of 0.2 m. In the tables, Discrete
indicates methods that use only discrete representations, and Non-discrete
indicates methods that use continuous representations. In terms of storage,
Rep. refers to the capacity used to store the feature, and the Total storage
capacity includes Morton codes to represent the spatial model.

In the experiments on the MaiCity [29] dataset, our DNMap outperforms all
other methods even with less storage. Compared to VQ-SHINE-Mapping when the
bitwidth is 8, our DNMap-discrete shows slightly lower mapping quality in
F-score, but trains about 1.4$\times$ faster. When comparing our DNMap (8-bit)
with SHINE-Mapping, DNMap reconstructs a dense and accurate mesh even with
8.5% of the feature storage.

In the experiments on the Newer College [23] dataset, our DNMap outperforms
SHINE-Mapping in all metrics, even in an environment with significant noise
and motion distortion. These results imply that our DNMap represents large-
scale environments with a limited number of component vectors, making it
robust to noise. Also, when the bitwidth is 8, VQ-SHINE-Mapping cannot be
trained due to large memory usage, whereas our DNMap can be trained. The
weakness in the Newer College dataset is that our DNMap shows slightly lower
performance in terms of completeness compared to other methods. Nevertheless,
DNMap achieves better performance than other methods in all other metrics,
especially when comparing our DNMap-discrete (8-bit) with SHINE-Mapping. Even
without using continuous embeddings, our DNMap reconstructs accurate mesh with
only 3% of the feature storage.

####  5.2.2 Qualitative Comparisons.

(a) MaiCity

(b) Newer College

Figure 4: Qualitative comparisons on (a) MaiCity [29] and (b) Newer College
[23] datasets. The highlighted areas of the figures are shown below.

Qualitative comparisons are illustrated in Fig. 4. As shown in the figure, our
DNMap reconstructs the shape of the structure as well as SHINE-Mapping. This
implies that the proposed decomposition-based representation can capture
repetitive and representative structural cues in a large-scale 3D environment,
and each discrete embedding is located in an optimal region. Furthermore, our
DNMap reconstructs a smoother mesh than SHINE-Mapping, as shown in the
highlighted areas. We can confirm that our proposed discrete representation is
more robust to noise than the continuous representation.

###  5.3 Incremental Mapping

Figure 5: The reconstruction result of incremental mapping on KITTI [4]
odometry dataset sequence 00 using the component vector set and the decoder
pre-trained at Newer College dataset [23].

The reconstruction of incremental mapping is illustrated in Fig. 5. For
incremental mapping we adopt replay-based approach [26, 18, 31] that stores
the previous data pairs and trains them with current observed data pairs. We
perform mapping on the KITTI [4] odometry dataset 00 sequence using the
component vector set and the decoder pre-trained at Newer College dataset. As
shown in the figure, DNSMap is verified to capture repetitive and
representative patterns of the structure well without additional supervision.

Table 3: Ablation Study. The F-score is reported for every 10000 iterations. EI denotes efficient implementation. |  |  |  F-Score $\uparrow$ |  |  it/sec $\uparrow$
---|---|---|---|---|---
|  |  | 10k | 20k | 30k | 40k | 50k |  | basic | EI
SHINE-Mapping |  |  | 91.8 | 91.9 | 91.9 | 91.8 | 91.8 |  | 30.7 | -
DNMap (8-bit) |  |  | 91.7 | 92.1 | 92.1 | 92.2 | 92.2 |  | 23.9 |  27.1(1.13$\times$)
$B=4$ |  |  |  |  |  |  |  |  |  |
Indexing |  |  | 88.1 | 89.1 | 89.2 | 89.3 | 88.9 |  | 25.3 |  28.2(1.11$\times$)
Indexing-Gumbel |  |  | 86.4 | 86.5 | 86.6 | 87.2 | 86.9 |  | 24.3 |  27.3(1.12$\times$)
Decomposition-naive |  |  | 86.7 | 87.5 | 88.2 | 88.2 | 88.2 |  | 27.1 |  32.5(1.20$\times$)
Decomposition |  |  | 87.6 | 88.6 | 88.9 | 88.9 | 89.0 |  | 25.8 |  30.5(1.18$\times$)
$B=6$ |  |  |  |  |  |  |  |  |  |
Indexing |  |  | 89.1 | 89.7 | 89.7 | 89.7 | 89.7 |  | 20.6 |  22.1(1.07$\times$)
Indexing-Gumbel |  |  | 74.0 | 80.4 | 83.1 | 84.3 | 81.6 |  | 18.5 |  19.9(1.08$\times$)
Decomposition-naive |  |  | 87.3 | 88.5 | 88.7 | 88.4 | 88.7 |  | 26.6 |  31.5(1.18$\times$)
Decomposition |  |  | 88.7 | 89.8 | 90.0 | 89.9 | 89.8 |  | 25.9 |  29.9(1.15$\times$)
$B=8$ |  |  |  |  |  |  |  |  |  |
Indexing |  |  | 89.0 | 89.4 | 89.5 | 89.5 | 89.6 |  | 12.3 |  12.4(1.01$\times$)
Indexing-Gumbel |  |  | 77.8 | 83.0 | 80.5 | 83.5 | 85.0 |  | 9.8 |  9.9(1.01$\times$)
Decomposition-naive |  |  | 88.7 | 89.7 | 90.0 | 89.8 | 90.0 |  | 26.3 |  30.4(1.16$\times$)
Decomposition |  |  | 88.7 | 90.2 | 90.3 | 90.3 | 90.3 |  | 25.6 |  29.5(1.15$\times$)

###  5.4 Ablation Studies (Tab. 3)

We conduct ablation studies on the proposed method to analyze our proposed
method by comparing Indexing, Indexing-Gumbel, Decomposition-naive, and
Decomposition. Indexing indicates VQ-SHINE-Mapping that learns indexing using
Softmax function [34, 27]. Indexing-Gumbel is a method of providing randomness
to indexing in VQ-SHINE-Mapping using the Gumbel-Softmax function [7, 34, 27].
Decomposition-naive is a naive version of our decomposition-based discrete
representation where the composition process is Eq. (9). Decomposition
indicates our decomposition-based discrete representation where the
composition process is Eq. (11).

All ablation studies are trained with some variation on the experimental
setting in Section 5.1.4: $N_{F}=4$, $N_{S}=8$, and $s=0.05$ for data
preparation; 50000 iterations with 4096 batch size for training. We report the
F-score for every 10000 iterations and the training speed (it/sec).

####  5.4.1 Analysis on Neural Discrete Representation Learning.

From the table, Indexing shows degraded performance as the bitwidth increases,
and the learning speed decreases significantly. In our experience, indexing-
based representation fails to utilize all embeddings when the size of the
embedding space becomes too large and falls into a local minimum. Moreover,
the performance of Indexing-Gumbel is very poor, likely because the introduced
randomness confuses the feature volume and the decoder that optimizes neural
fields. Decomposition-naive has the fastest learning speed but relatively low
performance, whereas Sigmoid-proposed, which can express a more diverse
embedding space, outperforms all other methods.

####  5.4.2 Analysis on Training Speed.

From the table, we can see that decomposition-based representation can
optimize the discrete embedding space with faster training speeds compared to
other methods. Additionally, the efficient implementation (EI) described in
Section 4.3 significantly boosts training speed. However, if accessing the
parameter set representing the embedding space is not the main cause of
slowing down the learning speed, there is no significant effect.

##  6 Discussion

Although our proposed decomposition-based discrete representation shows
improved performance and storage efficiency compared to existing
representations, it still has weaknesses. While our DNMap is robust to noise
and specialized for learning structural cues in large-scale 3D environments,
it still has difficulties representing thin surfaces, which are generally a
weak point in implicit neural representations. To solve this problem, we can
reduce the voxel size to represent more details and increase the height of the
octree, but this is not a fundamental solution. Therefore, our future work
will focus on maintaining local structural details while still being memory
efficient in large-scale environments.

##  7 Conclusion

In this paper, we present a novel 3D mapping framework named Decomposition-
based Neural Mapping (DNMap) to capture repetitive and representative patterns
in a large-scale environment in a storage-efficient manner. To this end, we
adopt an implicit neural discrete representation based on a decomposition
strategy that learns the decomposition and composition of the discrete
embedding space. First, we decompose each discrete embedding into component
vectors that are shared across the embedding space. Then, we learn the
composition indicators and store them in an octree-based structure. All
parameters are simultaneously optimized without additional supervision. With
these techniques, we significantly reduce the storage of local features in
large-scale outdoor datasets while preserving mapping accuracy.

#### Acknowledgments.

This work was supported by Korea Evaluation Institute Of Industrial Technology
(KEIT) grant funded by the Korea government(MOTIE) (No.20023455, Development
of Cooperate Mapping, Environment Recognition and Autonomous Driving
Technology for Multi Mobile Robots Operating in Large-scale Indoor Workspace)

## References

  * [1] Azinović, D., Martin-Brualla, R., Goldman, D.B., Nießner, M., Thies, J.: Neural rgb-d surface reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6290–6301 (June 2022)
  * [2] Bengio, Y., Léonard, N., Courville, A.: Estimating or propagating gradients through stochastic neurons for conditional computation (2013)
  * [3] Deng, J., Wu, Q., Chen, X., Xia, S., Sun, Z., Liu, G., Yu, W., Pei, L.: Nerf-loam: Neural implicit representation for large-scale incremental lidar odometry and mapping. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 8218–8227 (October 2023)
  * [4] Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. pp. 3354–3361 (2012). https://doi.org/10.1109/CVPR.2012.6248074
  * [5] Gropp, A., Yariv, L., Haim, N., Atzmon, M., Lipman, Y.: Implicit geometric regularization for learning shapes. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 3789–3799. PMLR (13–18 Jul 2020), https://proceedings.mlr.press/v119/gropp20a.html
  * [6] Isaacson, S., Kung, P.C., Ramanagopal, M., Vasudevan, R., Skinner, K.A.: Loner: Lidar only neural representations for real-time slam. IEEE Robotics and Automation Letters 8(12), 8042–8049 (2023). https://doi.org/10.1109/LRA.2023.3324521
  * [7] Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016)
  * [8] Johari, M.M., Carta, C., Fleuret, F.: Eslam: Efficient dense slam system based on hybrid representation of signed distance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 17408–17419 (June 2023)
  * [9] Kwon, Y., Sung, M., Yoon, S.: Implicit lidar network: Lidar super-resolution via interpolation weight prediction. In: 2022 International Conference on Robotics and Automation (ICRA). pp. 8424–8430 (2022). https://doi.org/10.1109/ICRA46639.2022.9811992
  * [10] Kühner, T., Kümmerle, J.: Large-scale volumetric scene reconstruction using lidar. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). pp. 6261–6267 (2020). https://doi.org/10.1109/ICRA40945.2020.9197388
  * [11] Li, P., Shi, Y., Liu, T., Zhao, H., Zhou, G., Zhang, Y.Q.: Semi-supervised implicit scene completion from sparse lidar (2021)
  * [12] Liu, L., Gu, J., Zaw Lin, K., Chua, T.S., Theobalt, C.: Neural sparse voxel fields. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 15651–15663. Curran Associates, Inc. (2020), https://proceedings.neurips.cc/paper_files/paper/2020/file/b4b758962f17808746e9bb832a6fa4b8-Paper.pdf
  * [13] Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. SIGGRAPH Comput. Graph. 21(4), 163–169 (aug 1987). https://doi.org/10.1145/37402.37422, https://doi.org/10.1145/37402.37422
  * [14] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: Computer Vision – ECCV 2020. pp. 405–421. Springer International Publishing, Cham (2020)
  * [15] Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. 41(4), 102:1–102:15 (Jul 2022). https://doi.org/10.1145/3528223.3530127, https://doi.org/10.1145/3528223.3530127
  * [16] Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality. pp. 127–136 (2011). https://doi.org/10.1109/ISMAR.2011.6092378
  * [17] van den Oord, A., Vinyals, O., kavukcuoglu, k.: Neural discrete representation learning. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017), https://proceedings.neurips.cc/paper_files/paper/2017/file/7a98af17e63a0ac09ce2e96d03992fbc-Paper.pdf
  * [18] Ortiz, J., Clegg, A., Dong, J., Sucar, E., Novotny, D., Zollhoefer, M., Mukadam, M.: isdf: Real-time neural signed distance fields for robot perception (2022)
  * [19] Pan, Y., Kompis, Y., Bartolomei, L., Mascaro, R., Stachniss, C., Chli, M.: Voxfield: Non-projective signed distance fields for online planning and 3d reconstruction. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 5331–5338 (2022). https://doi.org/10.1109/IROS47612.2022.9981318
  * [20] Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
  * [21] Park, M., Son, H., Kim, E.: Implicit point function for lidar super-resolution in autonomous driving. IEEE Robotics and Automation Letters 8(11), 7003–7009 (2023). https://doi.org/10.1109/LRA.2023.3313925
  * [22] Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., Geiger, A.: Convolutional occupancy networks. In: European Conference on Computer Vision (ECCV) (2020)
  * [23] Ramezani, M., Wang, Y., Camurri, M., Wisth, D., Mattamala, M., Fallon, M.: The newer college dataset: Handheld lidar, inertial and vision with ground truth. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 4353–4360 (2020). https://doi.org/10.1109/IROS45743.2020.9340849
  * [24] Rist, C.B., Emmerichs, D., Enzweiler, M., Gavrila, D.M.: Semantic scene completion using local deep implicit functions on lidar data. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(10), 7205–7218 (2022). https://doi.org/10.1109/TPAMI.2021.3095302
  * [25] Shi, C., Tang, F., Wu, Y., Jin, X., Ma, G.: Accurate implicit neural mapping with more compact representation in large-scale scenes using ranging data. IEEE Robotics and Automation Letters 8(10), 6683–6690 (2023). https://doi.org/10.1109/LRA.2023.3311355
  * [26] Sucar, E., Liu, S., Ortiz, J., Davison, A.J.: imap: Implicit mapping and positioning in real-time. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6229–6238 (October 2021)
  * [27] Takikawa, T., Evans, A., Tremblay, J., Müller, T., McGuire, M., Jacobson, A., Fidler, S.: Variable bitrate neural fields. In: ACM SIGGRAPH 2022 Conference Proceedings. SIGGRAPH ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3528233.3530727, https://doi.org/10.1145/3528233.3530727
  * [28] Takikawa, T., Litalien, J., Yin, K., Kreis, K., Loop, C., Nowrouzezahrai, D., Jacobson, A., McGuire, M., Fidler, S.: Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 11358–11367 (June 2021)
  * [29] Vizzo, I., Chen, X., Chebrolu, N., Behley, J., Stachniss, C.: Poisson Surface Reconstruction for LiDAR Odometry and Mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2021), http://www.ipb.uni-bonn.de/pdfs/vizzo2021icra.pdf
  * [30] Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems. vol. 34, pp. 27171–27183. Curran Associates, Inc. (2021), https://proceedings.neurips.cc/paper_files/paper/2021/file/e41e164f7485ec4a28741a2d0ea41c74-Paper.pdf
  * [31] Yan, Z., Tian, Y., Shi, X., Guo, P., Wang, P., Zha, H.: Continual neural mapping: Learning an implicit scene representation from sequential observations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 15782–15792 (October 2021)
  * [32] Yan, Z., Yang, H., Zha, H.: Active neural mapping. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 10981–10992 (October 2023)
  * [33] Zhang, K., Riegler, G., Snavely, N., Koltun, V.: Nerf++: Analyzing and improving neural radiance fields (2020)
  * [34] Zhong, X., Pan, Y., Behley, J., Stachniss, C.: Shine-mapping: Large-scale 3d mapping using sparse hierarchical implicit neural representations. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2023)
  * [35] Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M.R., Pollefeys, M.: Nice-slam: Neural implicit scalable encoding for slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12786–12796 (June 2022)