# An Empirical Study of Training State-of-the-Art LiDAR Segmentation Models

Jiahao Sun♡,1,2 Chunmei Qing${}^{\heartsuit,1,\textrm{\Letter}}$ Xiang Xu♡,3
Lingdong Kong♡,2,4 Youquan Liu♡,5
Li Li♡,6 Chenming Zhu♡,1,7 Jingwei Zhang♡,2 Zeqi Xiao♡,8 Runnan Chen♡,7
Tai Wang♡,2 Wenwei Zhang♡,2 Kai Chen♡,2

♡MMDetection3D Contributors
1South China University of Technology 2Shanghai AI Laboratory
3Nanjing University of Aeronautics and Astronautics 4National University of
Singapore
5Fudan University 6Durham University 7University of Hong Kong
8Nanyang Technological University, Singapore
https://github.com/open-mmlab/mmdetection3d

###### Abstract

In the rapidly evolving field of autonomous driving, precise segmentation of
LiDAR data is crucial for understanding complex 3D environments. Traditional
approaches often rely on disparate, standalone codebases, hindering unified
advancements and fair benchmarking across models. To address these challenges,
we introduce MMDetection3D-lidarseg, a comprehensive toolbox designed for the
efficient training and evaluation of state-of-the-art LiDAR segmentation
models. We support a wide range of segmentation models and integrate advanced
data augmentation techniques to enhance robustness and generalization.
Additionally, the toolbox provides support for multiple leading sparse
convolution backends, optimizing computational efficiency and performance. By
fostering a unified framework, MMDetection3D-lidarseg streamlines development
and benchmarking, setting new standards for research and application. Our
extensive benchmark experiments on widely-used datasets demonstrate the
effectiveness of the toolbox. The codebase and trained models have been
publicly available, promoting further research and innovation in the field of
LiDAR segmentation for autonomous driving.

††🖂 Corresponding Author<EMAIL_ADDRESS>
##  1 Introduction

Figure 1: Performance comparisons of state-of-the-art LiDAR segmentation
models [20, 102] from different codebases on the validation sets of the
SemanticKITTI [4] and nuScenes [28] datasets. Table 1: An overview of the
supported models, sparse convolution backends, and 3D data augmentation
techniques from existing LiDAR segmentation codebases. MMDet3D-lidarseg is an
abbreviation of our codebase, “✓” denotes features that are officially
supported.

|  MMDet3D-lidarseg |  LaserMix [44] |  Pointcept [22] |  PCSeg [58] |  SPVNAS [78] |  Cylin3D [102] |  PVKD [36]
---|---|---|---|---|---|---|---
Model
MinkUNet [20] |  ✓ |  ✓ |  ✓ |  ✓ |  ✗ |  ✗ |  ✗
SPVCNN [78] |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✗ |  ✗
Cylinder3D [102] |  ✓ |  ✓ |  ✗ |  ✓ |  ✗ |  ✓ |  ✓
PolarNet [100] |  ✓ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
CENet [17] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗ |  ✓
FRNet [95] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
Backend
Minkowski Engine [20] |  ✓ |  ✓ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
SpConv v1 [97] |  ✓ |  ✓ |  ✗ |  ✗ |  ✗ |  ✓ |  ✓
SpConv v2 [23] |  ✓ |  ✗ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
TorchSparse [77] |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✗ |  ✗
TorchSparse++ [79] |  ✓ |  ✗ |  ✗ |  ✗ |  ✓ |  ✗ |  ✗
Data Augmentation
LaserMix [44] |  ✓ |  ✓ |  ✗ |  ✓ |  ✗ |  ✗ |  ✗
PolarMix [89] |  ✓ |  ✗ |  ✗ |  ✓ |  ✗ |  ✗ |  ✗
FrustumMix [95] |  ✓ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗ |  ✗
Test Time Augmentation |  ✓ |  ✗ |  ✓ |  ✓ |  ✗ |  ✗ |  ✓

LiDAR (Light Detection and Ranging) is an advanced remote sensing technology
that uses laser pulses to measure distances to objects, enabling the capture
of high-resolution, three-dimensional information about the environment when
mounted on vehicles, drones, or other platforms [25, 86, 32]. This capability
is crucial for applications such as autonomous driving, where understanding
complex 3D environments is essential for safe navigation [3, 61, 35]. LiDAR
provides rich spatial details that complement other sensors like cameras and
radars, offering a robust foundation for perceiving and interpreting the
surrounding world.

LiDAR segmentation is the process of classifying individual points in a LiDAR-
generated point cloud into distinct semantic categories, such as vehicles,
pedestrians, and roadways [4, 28, 8, 90]. This segmentation is fundamental for
interpreting the structure and type of objects within the environment, which
is critical for situational awareness and decision-making in autonomous
systems [72, 47, 54, 2, 18, 7]. Effective LiDAR segmentation enhances an
autonomous vehicle’s ability to navigate complex scenarios, avoid obstacles,
and make informed decisions [64, 99, 74, 9, 10, 57].

Despite its importance, the recent development of LiDAR segmentation models
has faced significant challenges. The landscape is fragmented, with numerous
standalone and often incompatible codebases [102, 78, 36]. As shown in Tab. 1,
this fragmentation creates inefficiencies in research and development, making
it difficult to perform fair and comprehensive comparisons between different
models [87, 44, 58]. Additionally, many existing solutions often struggle to
meet the demands of dynamic and varied autonomous driving environments, which
require segmentation frameworks that are flexible, scalable, and robust [21,
61, 47, 62, 15, 65, 69, 68].

One major challenge in LiDAR segmentation is the integration of different
sparse convolution backends [77, 79, 23, 97, 20], which are essential for
efficiently processing the sparse and irregular nature of LiDAR point clouds
[5, 50, 49, 53]. Sparse convolutional networks have demonstrated significant
improvements in performance and computational efficiency for 3D point cloud
processing [78]. However, exploring and comparing these backends within a
unified framework has been challenging due to the lack of standardized tools
and benchmarks [21, 58].

In response to the aforementioned challenges, in this work, we present
MMDetection3D-lidarseg, a state-of-the-art toolbox designed to unify,
optimize, and streamline the training and benchmarking of LiDAR segmentation
models. MMDetection3D-lidarseg integrates multiple advanced features to
facilitate cohesive and efficient development, offering a unified framework
that enhances the comparability and reproducibility of research findings. Our
toolbox supports the exploration and comparison of various sparse
convolutional backends, providing a standardized benchmark that improves the
performance of widely used segmentation models.

Our design emphasizes several key principles as follows:

  * •

Unified Framework: By consolidating various LiDAR segmentation models and
techniques into a single, comprehensive toolbox, we eliminate the
fragmentation of codebases, enabling more efficient development and easier
benchmarking of models.

  * •

Optimization & Efficiency: Our toolbox includes optimized implementations of
state-of-the-art algorithms, ensuring that models can be trained and evaluated
quickly and effectively. This is crucial for both academic research and real-
world applications, where time and resource efficiency are paramount.

  * •

Flexibility & Scalability: The MMDetection3D-lidarseg toolkit is designed to
be flexible and scalable, capable of adapting to different driving scenarios
and handling large-scale point cloud data. This flexibility ensures that the
toolbox can meet the evolving needs of researchers and practitioners in the
field.

  * •

Comprehensive Benchmarking: To facilitate fair and comprehensive comparisons,
we provide a suite of standardized benchmarks that cover a wide range of
tasks, including fully-, semi-, and weakly-supervised LiDAR segmentation.
These benchmarks help ensure that different models can be evaluated on a
common ground, promoting transparency and rigor in research.

  * •

Public Availability & Collaborations: By making the codebase and trained
models publicly available, we encourage collaboration and open innovation in
the field of LiDAR segmentation. This openness helps accelerate the pace of
advancements and fosters a community-driven approach to problem-solving.

MMDetection3D-lidarseg represents a significant step forward in the
development of LiDAR segmentation models, addressing key challenges and
setting new standards for research and application. As shown in Fig. 1, our
implemented models achieve consistent and significant performance improvements
over the existing codebases. By providing a comprehensive and flexible
platform, our toolbox aims to accelerate innovation in autonomous driving
technologies, ultimately contributing to the development of safer and more
reliable autonomous systems.

##  2 Related Work

###  2.1 LiDAR Semantic Segmentation

Segmenting LiDAR-acquired point clouds into meaningful semantic categories is
indispensable for achieving comprehensive 3D scene understanding, especially
in the context of autonomous driving [82, 30, 91]. The evolution of LiDAR
semantic segmentation methodologies has been characterized by a transition
from early hand-crafted features [88, 48] and traditional machine learning
techniques [26, 27] to the dominance of deep learning approaches [37, 38, 81].
Convolutional neural networks (CNNs), initially developed for image analysis,
have been adapted for processing unstructured point cloud data through
innovative strategies [84]. One such strategy is voxelization [20, 78, 102,
35], which transforms point clouds into a structured 3D grid format suitable
for standard 3D CNN operations. However, voxelization often incurs high
computational costs and memory usage due to the sparsity of point cloud data.
Alternatively, projection-based methods [67, 93, 24, 100, 56] rasterize point
clouds into 2D range images or bird’s eye view maps, leveraging the efficiency
of 2D CNNs. Despite their efficiency, these methods may sacrifice some 3D
spatial information [40, 1, 41, 95]. The evolution of LiDAR semantic
segmentation has been marked by continuous exploration of these methodologies,
each offering unique trade-offs in terms of accuracy, computational
efficiency, and scalability [70, 3, 76, 85]. Recent advancements have focused
on integrating multi-modal data sources [39, 60, 13, 59, 96, 19, 14, 71, 55],
enhancing real-time processing capabilities [95, 101, 17, 43, 12, 83], and
improving model robustness under diverse environmental conditions [42, 45, 34,
33, 92, 98]. These advancements highlight the dynamic nature of research in
LiDAR segmentation, striving to balance the competing demands of precision,
efficiency, and robustness [31, 11, 98].

###  2.2 3D Operators

Sparse convolution is a fundamental operator in LiDAR segmentation. To
understand its significance, we first consider the $N$-dimensional dense
convolution. Let $x^{\text{in}}_{i}$ and $x^{\text{out}}_{j}$ represent the
input and output features, respectively, where $i$ and $j$ are $N$-dimensional
coordinates (_e.g_., $i=(2,1,1)$ in 3D space). The size of the convolution
kernel is $K$, with kernel offsets $\Delta^{N}(K)$. The weights of the
convolution kernel, $W$, are of shape $K^{N}\times C^{\text{in}}\times
C^{\text{out}}$, where $C^{\text{in}}$ and $C^{\text{out}}$ represent the
dimension of input and out features, respectively. For each offset $\delta$
(_e.g_., $\delta=(0,1,-1)\in\Delta^{3}(3)$), the weights $W$ can be broken
down into $K^{N}$ matrices of shape $C^{\text{in}}\times C^{\text{out}}$,
denoted as $W_{\delta}$. Dense convolution with stride $s$ can then be
formulated as follows:

$x^{\text{out}}_{i}=\sum_{\delta\in\Delta^{N}(K)}x^{\text{in}}_{s\cdot
i+\delta}\cdot W_{\delta}~{}.$ (1)

Here, the bias in convolution is ignored for simplicity. However, in datasets
commonly used for LiDAR segmentation, voxelized point cloud samples often have
less than $1\%$ of nonzero voxels. This sparsity makes dense convolution
inefficient due to the large number of meaningless computations, leading to
limited model performance. Sparse convolution addresses this inefficiency by
computing only when the input voxel is nonzero. It can be formulated as
follows:

$x^{\text{out}}_{i}=\sum_{\delta\in\Delta^{N}(K)}\sum_{j}\mathbbm{1}[s\cdot
j+\delta]\cdot x^{\text{in}}_{s\cdot j+\delta}\cdot W_{\delta}~{},$ (2)

where $\mathbbm{1}[\cdot]$ is a binary indicator. The computation of sparse
convolution is determined by maps $\mathcal{M}={(i,j,\delta)}$. For given
input coordinates and convolution kernel size, sparse convolution first
calculates the output coordinates and generates maps $\mathcal{M}$, then
performs the convolution operation. In essence, sparse convolution modifies
the definition of dense convolution by performing computations only at a
sparse set of output locations rather than across the entire feature map. It
is arguably the most critical building block for state-of-the-art voxel-based
LiDAR segmentation models. Various backends have been developed to efficiently
handle sparse convolution tasks, and MMDetection3D-lidarseg supports five
prevailing sparse convolution backends: SpConv [97], MinkowskiEngine [20],
TorchSparse [77], SpConv2 [23], and TorchSparse++ [79]. We also compare the
advantages and disadvantages of these sparse convolutions in the experimental
section, aiming to assist researchers in selecting the most suitable library
for their specific needs.

###  2.3 3D Data Augmentations

Data augmentation plays a crucial role in enhancing the performance and
generalizability of deep learning models, especially in domains characterized
by limited training data or substantial variance in operational conditions
[89, 44, 95]. In the context of LiDAR segmentation, 3D data augmentations are
essential for simulating a wide range of scenarios and environmental
conditions that autonomous vehicles may encounter [42]. Techniques such as
random rotations, scaling, and jittering of point clouds help models become
robust to variations in object orientations and sizes. More sophisticated
methods, including synthetic occlusion, simulating varying sensor ranges, and
generating adversarial examples, further enrich the model’s exposure to
challenging conditions, thereby improving its real-world performance [46, 54].
Additionally, domain adaptation techniques aim to bridge the gap between
synthetic and real-world data, enabling models trained primarily on simulated
environments to transfer their capabilities to real-world settings [43, 39,
52, 94, 66, 73, 6]. Continuous innovation in 3D data augmentation techniques
is pivotal for advancing the state-of-the-art in LiDAR segmentation, ensuring
models are not only accurate but also resilient to the diverse challenges they
face in real-world applications [42, 92]. These augmentations, coupled with
robust training frameworks like MMDetection3D-lidarseg, provide a
comprehensive toolkit for developing next-generation LiDAR segmentation models
capable of meeting the demands of modern autonomous systems [47].

##  3 MMDetection3D-lidarseg

MMDetection3D-lidarseg includes high-quality implementations of popular LiDAR
semantic segmentation models and sparse convolution backends. A summary of
supported models and backends, compared to other codebases, is provided in
Tab. 1. MMDetection3D-lidarseg supports a wider range of models and backends
than other codebases, offering unparalleled flexibility and comprehensiveness.
The following is a brief introduction to the datasets, models, data
augmentation techniques, and sparse convolution backends that we support.

###  3.1 Datasets

  * •

SemanticKITTI [4] is a pioneering dataset for LiDAR segmentation, offering an
extensive collection of annotated 3D point clouds derived from real-world
driving scenarios. It extends the KITTI Vision Benchmark Suite [32] by
providing dense, per-point annotations across a wide range of categories,
including various types of vehicles, pedestrians, and both natural and urban
structures. Notably, this dataset features temporal consistency, capturing the
dynamic nature of urban environments through sequences of scans over time. The
extensive coverage and challenging scenarios make SemanticKITTI a fundamental
resource for advancing and benchmarking LiDAR segmentation models in the
context of autonomous driving.

  * •

nuScenes-lidarseg [28], a variant of the well-known nuScenes [8] dataset,
serves as a diverse and large-scale benchmark for semantic segmentation in
autonomous vehicle sensor data. This dataset includes detailed annotations for
a variety of object classes across numerous scenes captured in different urban
environments and weather conditions, utilizing LiDAR, radar, and camera data.
The complexity and diversity of nuScenes, featuring night scenes and adverse
weather conditions, present significant challenges for segmentation models,
making it an invaluable resource for testing robustness and versatility.
Additionally, its multi-modal nature supports research into sensor fusion
techniques, enhancing its contribution to the development of comprehensive
perception systems for autonomous driving.

  * •

ScribbleKITTI [85] introduces an innovative approach to dataset annotation for
LiDAR segmentation. It shares the same scenes with SemanticKITTI [4] but is
annotated with line scribbles rather than dense, per-point labeling. This
method significantly reduces the annotation effort while still providing high-
quality training data for weakly-supervised learning algorithms. This
efficiency of scribble-based annotations enables the creation of extensive
datasets with less manual effort, facilitating the development and testing of
advanced segmentation models in autonomous driving applications.

###  3.2 Models

  * •

MinkUNet [20] is a highly efficient model for LiDAR segmentation, built on the
Minkowski Engine. It employs sparse convolutional operations to efficiently
process point cloud data, extending the U-Net architecture into 3D spaces.
This design enables the segmentation of large-scale point clouds by leveraging
the inherent sparsity of LiDAR data, significantly enhancing computational
efficiency.

  * •

SPVCNN [78] combines the strengths of point-based and voxel-based processing
methods to achieve high efficiency and accuracy. This hybrid approach provides
a scalable solution that adapts to the density variations of point cloud data,
ensuring detailed feature extraction while maintaining computational
efficiency across diverse scenes.

  * •

Cylinder3D [102] introduces a cylindrical partitioning strategy that aligns
more naturally with the distribution of point clouds captured by circular
LiDAR scans. This approach enhances segmentation performance, particularly for
cylindrical objects like poles and trees, by preserving their geometric
integrity.

  * •

CENet [17] focuses on capturing and utilizing contextual information from the
range view of LiDAR point clouds for efficient segmentation, with multi-level
feature fusion and .

  * •

PolarNet [100] transforms LiDAR point clouds into a polar bird’s eye view
representation, facilitating efficient processing through partitioning the
space into polar bins. This representation is particularly advantageous for
LiDAR data, aligning with the sensor’s native data structure and simplifying
the detection of objects in 360-degree surroundings.

  * •

FRNet [95] introduces a novel Frustum-Range representation for scalable LiDAR
segmentation and proposes two novel 3D data augmentation techniques,
FrustumMix and RangeInterpolation, to assist in training robust and
generalizable models. It achieves competitive performance while still
maintaining promising scalability for real-time LiDAR segmentation.

Although the architectures of various LiDAR segmentation models differ, they
can generally be categorized into voxel-based and projection-based segmentors,
each sharing common components [41, 1, 96]. Voxel-based segmentors [20, 102,
78] typically consist of a voxelization module, a voxel feature encoder, a 3D
backbone, and a decode head. These components work together to convert raw
point cloud data into a structured voxel grid, extract meaningful features,
process these features through a deep 3D neural network, and finally decode
the processed features into semantic labels. Conversely, projection-based
segmentors [17, 95] include a projection module, a pixel feature encoder, a 2D
backbone, and a decode head. The projection module converts the 3D point cloud
into a 2D image representation, such as a range image or bird’s eye view map
[67, 100]. The pixel feature encoder and 2D backbone then process this image
to extract features, which are subsequently decoded into semantic labels [87,
93].

Figure 2: Overview of voxel-based and projection-based LiDAR segmentors
illustrated with abstractions in the MMDetection3D-lidarseg codebase. Modules
marked with $*$ are optional.

Both types of segmentors may also incorporate a point feature encoder to
directly process raw point cloud data and an auxiliary head to provide
additional outputs or intermediate supervision, enhancing the model’s overall
performance and robustness [95, 102, 42]. With these abstractions, the
framework of voxel-based and projection-based segmentors is illustrated in
Fig. 2. The modular design of these segmentors allows researchers to develop
their own models by creating new components and assembling them with existing
ones. This flexibility not only accelerates the development process but also
facilitates experimentation and innovation in the field of LiDAR segmentation.

###  3.3 Data Augmentation Techniques

  * •

LaserMix [44] introduces a novel augmentation technique specifically designed
for LiDAR point clouds, inspired by the concept of mixup for image data. This
technique involves mixing multiple point clouds along the azimuth and
inclination directions, significantly enhancing the model’s ability to learn
generalized features and improve robustness against overfitting.

  * •

PolarMix [89] generates synthetic samples by mixing polar representations of
LiDAR point clouds. This method is particularly effective for augmenting data
in scenarios where the spatial relationships and orientations of objects are
critical for understanding the scene.

  * •

FrustumMix [95] enhances context awareness by swapping frustum regions from
two scenes. This method splits two point clouds into several non-overlapping
frustum regions along the inclination or azimuth direction. FrustumMix
maintains the invariant structure within the frustum while enhancing the
contextual relationships between frustums.

  * •

Test Time Augmentation (TTA), typically applied during model inference,
enhances segmentation accuracy by applying various transformations (_e.g_.,
rotations, scaling) to the input data at test time and aggregating the
predictions. TTA mitigates the impact of noise and variations in the point
cloud data, resulting in more consistent and reliable segmentation outcomes,
particularly in challenging or ambiguous scenes. However, it requires
significantly more time during inference, making it less suitable for real-
time applications such as autonomous driving perception.

###  3.4 Sparse Convolution Backends

  * •

SpConv [97] can be regarded as a vanilla implementation of the sparse
convolution. SpConv employs the gather-GEMM-scatter dataflow on GPUs, which is
weight-stationary and features an outer host loop over kernel offsets. For
each offset, it computes a kernel map, gathers all input features into a
$K^{2}\times C^{\text{in}}$ matrix in DRAM, and multiplies the weights using
dense matrix multiplication, handled by existing vendor libraries like cuBLAS
and cuDNN. SpConv then scatters the results back to output positions. This
implementation only needs to optimize scatter and gather in CUDA. However,
this data flow is fundamentally inefficient due to the lack of overlap between
computation and memory access.

  * •

MinkowskiEngine [20] takes a fetch-on-demand approach and merges the gather,
matrix multiplication, and scatter kernel calls into a single CUDA kernel.
Instead of materializing the $K^{2}\times C^{\text{in}}$ matrix gather buffer
in DRAM, it fetches input features on demand into the L1 shared memory,
performs matrix multiplication in the on-chip storage, and directly scatters
the partial sums (residing in the register file) to corresponding outputs
position without first instantiating them in a DRAM scatter buffer. This
method overlaps computation with memory access and saves DRAM writes to gather
and scatter buffers. However, it cannot avoid DRAM writes to the final output
tensor, resulting in $4\times$ to $10\times$ larger write-back traffic than
the theoretical optimal value. Furthermore, the block-fused fetch-on-demand
dataflow suffers from write-back contentions between different threads.

  * •

TorchSparse [77] directly optimizes the two bottlenecks of the vanilla gather-
GEMM-scatter dataflow: irregular computation and data movement. Due to sparse
convolutions with odd kernel size and a stride of 1, the maps corresponding to
kernel offsets $(a,b,c)$ always have the same size as the maps corresponding
to the symmetric kernel offset $(-a,-b,-c)$, TorchSparse applies adaptive
matrix multiplication grouping to trade computation for better regularity,
achieving a $1.4\times$ to $1.5\times$ speedup for matrix multiplication.
TorchSparse also optimizes data movement by adopting vectorized, quantized,
and fused locality-aware memory access, reducing the memory movement cost by
$2.7\times$.

  * •

SpConv v2 [23] extends the well-known implicit GEMM formulation for 2D
convolution to 3D sparse convolution. The sparse convolution workload is
equivalent to a dense GEMM. Similar to fetch-on-demand, implicit GEMM overlaps
computation with memory access, hiding memory latency through pipelining. Like
“im2col” in 2D convolution, an implicit GEMM implementation is output-
stationary, achieving the theoretical minimum DRAM write-back traffic.
However, despite lower DRAM traffic compared to fetch-on-demand, implicit GEMM
has non-negligible redundant computation. To address this issue, SpConv v2
excludes unsorted implicit GEMM in their design space and uses bitmask sorting
to minimize computation overhead. Each output point is assigned a $K^{2}$
dimensional bitmask indicating the presence of its neighbors. These bitmasks
are treated as numbers and sorted, adjusting the computation order for
different outputs accordingly. In practical applications, sorting can reduce
redundant computation by up to $3\times$.

  * •

TorchSparse++ [79] simultaneously supports gather-GEMM-scatter, fetch-on-
demand, and sorted implicit GEMM dataflows. Overlapped computation and memory
access can be achieved with relatively low engineering complexity compared to
SpConv2. TorchSparse++ creates a highly efficient Sparse Kernel Generator that
generates performant sparse convolution kernels at less than one-tenth of the
engineering cost of current state-of-the-art systems. Additionally,
TorchSparse++ designed the Sparse Autotuner, which extends the design space of
existing sparse convolution libraries and searches for the best dataflow
configurations for training and inference workloads.

##  4 Experiments

In this section, we embark on comprehensive benchmarks of diverse LiDAR
segmentation models facilitated by MMDetection3D-lidarseg. Firstly, we provide
an overview of the datasets, metrics, and detailed implementations employed
within the codebase. Subsequently, we delve into experimental results from
distinct setups, encompassing fully-, semi-, and weakly-supervised learning
paradigms. Finally, we delve into a series of ablation studies, aiming to
scrutinize and compare the various components facilitated within the codebase.
Through these rigorous evaluations, we aim to provide valuable insights into
the performance and efficacy of LiDAR segmentation modes, contributing to
advancements in this critical field.

Table 2: Results of state-of-the-art LiDAR segmentation models [20, 102, 100,
17, 95, 78] on the validation sets of SemanticKITTI [4] and nuScenes [28]. Mix
denote a combination of LaserMix [44] and PolarMix [89] or FrustumMix [95].
TTA denotes test time augmentation. All scores are given in percentage (%).
The best scores under each metric are highlighted in bold.

Model |  Mix |  TTA | SemKITTI | nuScenes
---|---|---|---|---
|  |  mIoU |  mAcc |  mIoU |  mAcc
Representation: Voxel
MinkUNet |  ✗ |  ✗ |  $66.9$ |  $92.4$ |  $76.4$ |  $94.1$
✓ |  ✗ |  $70.4$ |  $92.8$ |  $77.6$ |  $94.2$
✓ |  ✓ |  $\mathbf{71.8}$ |  $\mathbf{93.2}$ |  $80.1$ |  $94.7$
Cylinder3D |  ✗ |  ✗ |  $63.7$ |  $91.0$ |  $75.8$ |  $93.7$
✓ |  ✗ |  $67.0$ |  $91.7$ |  $79.3$ |  $94.3$
✓ |  ✓ |  $69.4$ |  $92.5$ |  $\mathbf{80.9}$ |  $\mathbf{94.5}$
Representation: Bird’s Eye View
PolarNet |  ✗ |  ✗ |  $57.2$ |  $91.0$ |  $71.7$ |  $93.1$
✓ |  ✗ |  $60.1$ |  $91.3$ |  $72.3$ |  $93.2$
✓ |  ✓ |  $60.7$ |  $91.7$ |  $72.3$ |  $93.4$
Representation: Range View
CENet |  ✗ |  ✗ |  $61.9$ |  $90.3$ |  - |  -
✓ |  ✗ |  $62.2$ |  $90.5$ |  - |  -
FRNet |  ✗ |  ✗ |  $64.1$ |  $92.2$ |  $76.8$ |  $93.4$
✓ |  ✗ |  $67.6$ |  $92.3$ |  $77.7$ |  $94.0$
✓ |  ✓ |  $68.7$ |  $92.5$ |  $79.0$ |  $94.3$
Representation: Fusion
SPVCNN |  ✗ |  ✗ |  $66.4$ |  $92.5$ |  $76.0$ |  $93.9$
✓ |  ✗ |  $68.4$ |  $92.3$ |  $77.0$ |  $94.1$

Table 3: Results of data-efficient learning algorithms [80, 103, 29, 16, 44,
95, 85, 51] with state-of-the-art LiDAR segmentation backbones [101, 95, 100,
102, 20, 78] on the semi-supervised learning benchmarks of the SemanticKITTI
[4], nuScenes [28], and ScribbleKITTI [85] datasets, respectively. All methods
are tested on the official validation sets. The labeled data are uniformly
sampled with the 1%, 10%, 20%, and 50% quota from the training set of the
original dataset. Sup.-Only denotes training with the labeled data only, while
the data-efficient learning methods use both labeled and unlabeled data.
†Lim3D [51] adopts a different data sampling strategy instead of uniform
sampling. All scores are given in percentage (%). The best scores under each
metric are highlighted in bold.

Model | Venue | Backbone | SemanticKITTI | nuScenes | ScribbleKITTI
---|---|---|---|---|---
1% |  10% |  20% |  50% |  1% |  10% |  20% |  50% |  1% |  10% |  20% |  50%
Representation: Range View
Sup.-Only | - | FRNet |  $44.9$ |  $60.4$ |  $61.8$ |  $63.1$ |  $51.9$ |  $68.1$ |  $70.9$ |  $74.6$ |  $42.4$ |  $53.5$ |  $55.1$ |  $57.0$
LaserMix [44] | CVPR’23 |  $52.9$ |  $62.9$ |  $63.2$ |  $65.0$ |  $58.7$ |  $71.5$ |  $72.3$ |  $75.0$ |  $45.8$ |  $56.8$ |  $57.7$ |  $59.0$
FrustumMix [95] | arXiv’23 |  $55.8$ |  $64.8$ |  $65.2$ |  $65.4$ |  $61.2$ |  $72.2$ |  $74.6$ |  $75.4$ |  $46.6$ |  $57.0$ |  $59.5$ |  $61.2$
Representation: Bird’s Eye View
Sup.-Only | - | PolarNet |  $45.1$ |  $54.6$ |  $55.6$ |  $56.5$ |  $50.9$ |  $67.5$ |  $69.5$ |  $71.0$ |  $42.6$ |  $52.8$ |  $53.4$ |  $54.4$
MeanTeacher [80] | NeurIPS’17 |  $47.4$ |  $55.6$ |  $56.6$ |  $57.1$ |  $51.9$ |  $68.1$ |  $69.7$ |  $71.1$ |  $43.7$ |  $53.4$ |  $54.$4 |  $54.9$
LaserMix [44] | CVPR’23 |  $51.0$ |  $57.7$ |  $58.6$ |  $60.0$ |  $54.0$ |  $69.5$ |  $70.8$ |  $71.9$ |  $45.7$ |  $55.5$ |  $56.0$ |  $56.6$
Representation: Voxel
Sup.-Only | - | Cylinder3D |  $45.4$ |  $56.1$ |  $57.8$ |  $58.7$ |  $50.9$ |  $65.9$ |  $66.6$ |  $71.2$ |  $39.2$ |  $48.0$ |  $52.1$ |  $53.8$
MeanTeacher [80] | NeurIPS’17 |  $45.4$ |  $57.1$ |  $59.2$ |  $60.0$ |  $51.6$ |  $66.0$ |  $67.1$ |  $71.7$ |  $41.0$ |  $50.1$ |  $52.8$ |  $53.9$
LaserMix [44] | CVPR’23 |  $50.6$ |  $60.0$ |  $61.9$ |  $62.3$ |  $55.3$ |  $69.9$ |  $71.8$ |  $73.2$ |  $44.2$ |  $53.7$ |  $55.1$ |  $56.8$
LiM3D† [51] | CVPR’23 |  - |  $61.6$ |  $62.6$ |  $62.8$ |  - |  - |  - |  - |  - |  $60.3$ |  $60.5$ |  $60.9$
FrustumMix [95] | arXiv’23 |  $55.7$ |  $62.5$ |  $63.0$ |  $64.9$ |  $60.0$ |  $70.0$ |  $72.6$ |  $74.1$ |  $45.6$ |  $55.7$ |  $58.2$ |  $60.8$
Sup.-Only | - | MinkUNet |  $53.9$ |  $64.0$ |  $64.6$ |  $65.4$ |  $58.3$ |  $71.0$ |  $73.0$ |  $75.1$ |  $48.6$ |  $57.7$ |  $58.5$ |  $60.0$
MeanTeacher [80] | NeurIPS’17 |  $56.1$ |  $64.7$ |  $65.4$ |  $66.0$ |  $60.1$ |  $71.7$ |  $73.4$ |  $75.2$ |  $49.7$ |  $59.4$ |  $60.0$ |  $61.7$
LaserMix [44] | CVPR’23 |  $\mathbf{60.9}$ |  $\mathbf{66.6}$ |  $\mathbf{67.2}$ |  $\mathbf{68.0}$ |  $62.8$ |  $73.6$ |  $\mathbf{74.8}$ |  $\mathbf{76.1}$ |  $\mathbf{57.2}$ |  $\mathbf{61.1}$ |  $\mathbf{61.4}$ |  $\mathbf{62.4}$
Representation: Fusion
Sup.-Only | - | SPVCNN |  $52.7$ |  $64.1$ |  $64.5$ |  $65.1$ |  $57.9$ |  $71.7$ |  $73.0$ |  $74.6$ |  $47.2$ |  $57.3$ |  $58.2$ |  $58.8$
MeanTeacher [80] | NeurIPS’17 |  $54.4$ |  $64.8$ |  $65.2$ |  $65.7$ |  $59.4$ |  $72.5$ |  $73.1$ |  $74.7$ |  $49.9$ |  $58.3$ |  $58.6$ |  $59.1$
LaserMix [44] | CVPR’23 |  $60.3$ |  $\mathbf{66.6}$ |  $67.0$ |  $67.6$ |  $\mathbf{63.2}$ |  $\mathbf{74.1}$ |  $74.6$ |  $75.8$ |  $57.1$ |  $60.8$ |  $60.7$ |  $61.0$

###  4.1 Experimental Settings

Datasets. We conduct comprehensive experiments on three popular LiDAR-based
semantic segmentation benchmarks. SemanticKITTI [4] is a large-scale outdoor
dataset consisting of $22$ sequences collected from various scenarios in
Karlsruhe, Germany. Typically, sequences $00$ to $10$ are used for training,
while sequence $08$ is used for validation. All points are annotated with $28$
classes, and a merged set of $19$ classes is used for evaluation. nuScenes
[28] is a multi-modal dataset widely used in autonomous driving. It is
collected around streets in Boston and Singapore and features sparser LiDAR
points. The dataset includes $1,000$ driving scenes, with $750$ scenes for
training, $150$ scenes for validation, and $100$ scenes for testing. It is
annotated with $32$ semantic categories, and a merged set of $16$ labels is
used for evaluation. ScribbleKITTI [85] shares the same scenes as
SemanticKITTI [4] but is annotated with line scribbles. It contains $189$
million labeled points, with approximately $8.06\%$ labeled points for
training.

Figure 3: Performance comparisons of state-of-the-art LiDAR segmentation
models [20, 102, 78, 95, 17] from different LiDAR representation groups
(voxel, bird’s eye view, range view, fusion) on the validation set of
SemanticKITTI [4]. We report the segmentation accuracy (mIoU), inference
latency (FPS), and model parameters. The larger the area coverage, the larger
the model capacity.

Implementation Details. In our experiments, we employ AdamW [63] as the
default optimizer. We set the initial learning rate to 0.01 and utilize the
OneCycle scheduler [75] to dynamically adjust the learning rate during
training. Training is performed across eight A100 GPUs for 50 epochs on
SemanticKITTI [4] and ScribbleKITTI [85] datasets. For the nuScenes [28]
dataset, we extend the training to 80 epochs to ensure optimal performance.
Each GPU operates with a batch size of 2, making efficient use of parallel
processing capabilities. Our default data augmentation pipeline includes point
cloud rotation, translation, scaling, and flipping on the global view.
Additionally, we introduce a randomized choice of LaserMix [44] and PolarMix
[89] for the voxel-based, fusion-based, and bird’s eye view models, while
adopting FrustumMix [95] for range-view models to achieve better performance.
For detailed configurations and reproducibility, please refer to our GitHub
repository111https://github.com/open-mmlab/mmdetection3d.

Evaluation Metrics. For performance evaluation, we adhere to standard
protocols by employing mean Intersection-over-Union (mIoU) as the primary
metric and mean Accuracy (mAcc) as a supplementary metric. In addition to
these metrics, We evaluate the real-time processing applicability of models by
measuring their inference speed. We quantify this using two common metrics:
Frames Per Second (FPS) and Iteration Per Second (Iter/s). By incorporating
these metrics into our evaluation framework, we ensure a comprehensive
assessment of model performance, encompassing both accuracy and efficiency
considerations.

Evaluation Protocol. To uphold fair evaluation practices in our comparisons,
we employ consistent evaluation protocols across different LiDAR segmentation
models and data augmentation techniques. One crucial aspect of our methodology
is reporting evaluation metrics at the point level rather than voxels or range
images. This approach ensures fair comparisons even when models use different
rasterizations, as it directly assesses segmentation accuracy at the finest
granularity of the input data. Furthermore, we conduct ablation studies to
isolate the effects of individual components and augmentations. By
systematically varying and analyzing each factor independently, we gain a
deeper understanding of its impact on model performance.

###  4.2 Benchmark Experiments

Fully-Supervised LiDAR Semantic Segmentation. We benchmark benchmarking on
voxel-based, fusion-based, bird’s eye view, and range view segmentors using
the validation sets of SemanticKITTI [4] and nuScenes [28]. The evaluated
LiDAR segmentation models include MinkUNet [20], SPVCNN [78], Cylinder3D
[102], PolarNet [100], CENet [17], and FRNet [95], as detailed in Tab. 2. We
report performance under default data augmentation, mixing data augmentation,
and, where applicable, test-time augmentation (TTA) for each model. MinkUNet
[20] achieves the highest mIoU of $71.8\%$ on SemanticKITTI [4], while
Cylinder3D [102] attains $80.9\%$ mIoU on nuScenes [28]. Our results in Tab. 2
illustrate that our default training settings and advanced data augmentation
techniques significantly enhance model performance. For instance, MinkUNet’s
mIoU on SemanticKITTI improved by $5.0\%$ with our default settings, and an
additional $3.5\%$ with mixing data augmentation. TTA further boosted
performance by $1.4\%$, resulting in nearly a $10\%$ overall improvement.
These findings underscore the importance of comprehensive training strategies
and robust data augmentation in maximizing model effectiveness.

Inference Speed. Fig. 3 illustrates the inference speed of each model on the
SemanticKITTI [4] dataset. All FPS measurements are conducted on a single
NVIDIA GeForce RTX 3060 GPU with the data type float32. For fusion- and voxel-
based segmentors, SPVCNN [78] employs Torchsparse as its sparse convolution
backend, Cylinder3D [102] uses SpConv, and all variants of MinkUNet [20]
utilize SpConv v2 as their sparse convolution backend, respectively. The
results indicate that range view methods exhibit faster inference speeds, with
CENet [17] achieving $33$ FPS. While voxel-based methods generally offer
higher accuracy, their larger model sizes result in slower inference speeds.
Among the MinkUNet variants, MinkUNet-50-w32 achieves the highest accuracy but
the slowest inference, whereas MinkUNet-18-w16 offers faster inference with
lower accuracy. MinkUNet-34-w32 provides a balanced trade-off between
performance and speed, making it a versatile choice for various applications.
We believe these insights could become critical for researchers and
practitioners who need to optimize for either speed or accuracy depending on
the specific requirements of their deployment scenarios, such as real-time,
in-vehicle driving scene understanding and segmentation.

Table 4: Results of state-of-the-art LiDAR segmentation models [20, 102, 100,
17, 95, 78] under weakly-supervised learning setups on ScribbleKITTI [85]. Mix
denote a combination of LaserMix [44] and PolarMix [89] or FrustumMix [95].
TTA denotes test time augmentation. All scores are given in percentage (%).
The best scores under each metric are highlighted in bold.

Model |  Mix |  TTA | ScribbleKITTI
---|---|---|---
|  |  |  mIoU |  mAcc
Representation: Voxel
MinkUNet |  ✗ |  ✗ |  $61.2$ |  $88.5$
|  ✓ |  ✗ |  $62.6$ |  $89.9$
|  ✓ |  ✓ |  $\mathbf{65.7}$ |  $\mathbf{90.5}$
Cylinder3D |  ✗ |  ✗ |  $58.8$ |  $87.4$
|  ✓ |  ✗ |  $59.8$ |  $88.8$
Representation: Bird’s Eye View
PolarNet |  ✗ |  ✗ |  $55.7$ |  $87.6$
|  ✓ |  ✗ |  $57.2$ |  $88.4$
Representation: Range View
FRNet |  ✗ |  ✗ |  $57.6$ |  $88.3$
|  ✓ |  ✗ |  $60.2$ |  $88.5$
|  ✓ |  ✓ |  $63.1$ |  $89.9$
Representation: Fusion
SPVCNN |  ✗ |  ✗ |  $60.1$ |  $88.5$
|  ✓ |  ✗ |  $62.9$ |  $89.9$
|  ✓ |  ✓ |  $\mathbf{65.7}$ |  $90.4$

Semi-Supervised LiDAR Semantic Segmentation. Under the data-efficient learning
setting, we leverage partially labeled datasets to evaluate how models can
effectively learn with limited supervision. We conduct this study on
SemanticKITTI [4], nuScenes [28], and ScribbleKITTI [85]. By using a
combination of labeled and unlabeled data, we benchmark methods that employ
techniques such as consistency regularization and pseudo-labeling to improve
segmentation accuracy, including MeanTeacher [80], LaserMix [44], Lim3D [51],
and FrustumMix [95]. Consistency regularization ensures that predictions
remain stable under small perturbations of the input, while pseudo-labeling
generates labels for the unlabeled data to iteratively refine the model.
Preliminary results, shown in Tab. 3, indicate that semi-supervised learning
methods can achieve performance comparable to fully supervised counterparts,
demonstrating their potential in reducing the need for extensive manual
annotations. These methods not only lower the cost and effort associated with
data labeling but also enable the utilization of larger datasets that include
both labeled and unlabeled samples, enhancing the robustness and
generalization of different LiDAR segmentation approaches.

Weakly-Supervised LiDAR Segmentation. For the label-efficient learning task,
we utilize the ScribbleKITTI [85] dataset to evaluate LiDAR segmentors trained
with weak annotations. This approach significantly reduces the annotation
effort by using line scribbles instead of dense labels (which corresponds to
around $8.06\%$ of the original labels), providing a practical solution for
large-scale dataset creation [85]. Weakly-supervised learning leverages these
sparse annotations to guide the model in learning robust features. The
performance metrics, presented in Tab. 4, reveal that state-of-the-art
segmentors [20, 102, 100, 95, 78] can attain competitive accuracy with a
fraction of the annotation cost, highlighting their efficiency for large-scale
applications. These results are particularly promising for applications where
detailed annotations are impractical or too costly, suggesting that weak
annotations can be a viable alternative for training high-performance models.

Figure 4: Performance comparisons of different sparse convolution backends
[20, 23, 97, 77, 79] on the validation sets of SemanticKITTI [4] and nuScenes
[28] datasets. All experiments are conducted using the MinkUNet-34-w32
backbone [20]. Both the training speed (Iter/s, iterations per second) and
inference speed (FPS, frames per second) are measured using a single NVIDIA
A100 GPU.

###  4.3 Ablation Study

Sparse Convolution Backends. One of the key features of our
MMDetection3D-lidarseg codebase is its support for five common sparse
convolution backends: SpConv [97], Minkowski Engine [20], Torchsparse [77],
SpConv v2 [23], and Torchsparse++[79]. We compare the efficacy of these sparse
convolution backends using the MinkUNet-34-w32 backbone [20] on the validation
sets of SemanticKITTI [4] and nuScenes [28]. Fig. 4 reports the training speed
(Iter/s, iterations per second) and inference speed (FPS, frames per second)
of different backends, with measurements taken on a single NVIDIA A100 GPU.
Experiments show that, for all sparse convolution backends, enabling Automatic
Mixed Precision (AMP) reduces memory footprint and increases both training
iteration speed and inference speed, except for the Minkowski Engine [20],
which is not compatible with the float16 data type. In terms of training
speed, SpConv v2 [23] with AMP turned on is the fastest, reaching $8.3$
iterations per second on SemanticKITTI and $12.5$ iterations per second on
nuScenes. For inference speed, Torchsparse++ [79] using float16 data type
leads, achieving $35.8$ FPS on SemanticKITTI and $46.7$ FPS on nuScenes. The
results also show that all sparse convolutional backends perform faster on
nuScenes than on SemanticKITTI due to the denser point clouds in
SemanticKITTI, which require higher resolution during voxelization. These
findings provide valuable insights into the trade-offs and benefits of each
backend, guiding researchers in selecting the appropriate backend based on
their specific requirements for training efficiency, memory usage, and
inference speed.

Table 5: Ablation study on model capacities. We benchmark six variants of the
MinkUNet [20] on SemanticKITTI [4]. Param: the number of trainable parameters.
AMP: whether to use automatic mixed precision during training. Mem: memory
consumption. Train: the number of training interactions per second. Infer: the
frame-per-second rate during inference. The best scores in terms of training
speed (Iter/s), inference speed (FPS), and segmentation accuracy (mIoU) are
highlighted in bold.

Param (M) |  Amp |  Mem (GB) |  Train (Iter/s) |  Infer (FPS) |  mIoU (%)
---|---|---|---|---|---
Variant: MinkUNet-18-W16
$5.4$ |  ✗ |  $1.9$ |  $7.1$ |  $30.7$ |  $64.1$
$5.4$ |  ✓ |  $1.1$ |  $\mathbf{10.0}$ |  $\mathbf{32.4}$ |  $64.8$
Variant: MinkUNet-18-W20
$8.8$ |  ✗ |  $2.4$ |  $5.3$ |  $25.0$ |  $65.2$
$8.8$ |  ✓ |  $1.4$ |  $8.3$ |  $30.3$ |  $65.4$
Variant: MinkUNet-18-W32
$21.7$ |  ✗ |  $3.9$ |  $8.3$ |  $18.5$ |  $66.0$
$21.7$ |  ✓ |  $2.3$ |  $\mathbf{10.0}$ |  $31.8$ |  $65.8$
Variant: MinkUNet-34-W32
$37.9$ |  ✗ |  $4.7$ |  $2.4$ |  $14.7$ |  $66.9$
37.9 |  ✓ |  $2.8$ |  $8.3$ |  $28.7$ |  $66.6$
Variant: MinkUNet-50-W32
$31.8$ |  ✗ |  $11.5$ |  $2.8$ |  $11.0$ |  $65.5$
$31.8$ |  ✓ |  $6.0$ |  $7.7$ |  $13.7$ |  $\mathbf{67.6}$
Variant: MinkUNet-101-W32
$70.8$ |  ✗ |  $14.2$ |  $1.9$ |  $8.0$ |  $66.9$
$70.8$ |  ✓ |  $8.8$ |  $5.6$ |  $9.9$ |  $67.2$

Table 6: Ablation study on the use of 3D data augmentation techniques. We
benchmark PolarMix [89], LaserMix [44], and FrustumMix [95] with the MinkUNet
[20], Cylinder3D [102], and FRNet [95] backbones, respectively. The best score
under each backbone configuration is highlighted in bold.

PolarMix |  LaserMix |  FrustumMix |  mIoU
---|---|---|---
[89] |  [44] |  [95] |  (%)
Model: MinkUNet
✗ |  ✗ |  ✗ |  $66.9$
✓ |  ✗ |  ✗ |  $68.9$
✗ |  ✓ |  ✗ |  $67.7$
✓ |  ✓ |  ✗ |  $\mathbf{70.4}$
Model: Cylinder3D
✗ |  ✗ |  ✗ |  $63.7$
✓ |  ✗ |  ✗ |  $64.5$
✗ |  ✓ |  ✗ |  $65.6$
✓ |  ✓ |  ✗ |  $\mathbf{67.0}$
Model: FRNet
✗ |  ✗ |  ✗ |  $64.1$
✓ |  ✗ |  ✗ |  $66.5$
✗ |  ✓ |  ✗ |  $65.1$
✗ |  ✗ |  ✓ |  $\mathbf{67.6}$

Table 7: Ablation study on the use of the test time augmentation (TTA) during
the model inference stage. We benchmark different combinations with the
MinkUNet [20], Cylinder3D [102], and FRNet [95] backbones, respectively, on
SemanticKITTI [4]. The best score under each backbone configuration is
highlighted in bold.

Flip |  Rotate |  Scale |  Translate | mIoU | Time
---|---|---|---|---|---
|  |  |  | (%)
Model: MinkUNet
✗ |  ✗ |  ✗ |  ✗ | $70.4$ | $1\times$
✓ |  ✗ |  ✗ |  ✗ | $71.2$ | $4\times$
✓ |  ✓ |  ✗ |  ✗ | $71.4$ | $12\times$
✓ |  ✓ |  ✓ |  ✗ | $71.7$ | $36\times$
✓ |  ✓ |  ✓ |  ✓ | $\mathbf{71.8}$ | $108\times$
Model: Cylinder3D
✗ |  ✗ |  ✗ |  ✗ | $67.0$ | $1\times$
✓ |  ✗ |  ✗ |  ✗ | $65.3$ | $4\times$
✓ |  ✓ |  ✗ |  ✗ | $65.4$ | $12\times$
✓ |  ✓ |  ✓ |  ✗ | $69.0$ | $36\times$
✓ |  ✓ |  ✓ |  ✓ | $\mathbf{69.4}$ | $108\times$
Model: FRNet
✗ |  ✗ |  ✗ |  ✗ | $67.6$ | $1\times$
✓ |  ✗ |  ✗ |  ✗ | $68.1$ | $4\times$
✓ |  ✓ |  ✗ |  ✗ | $68.1$ | $12\times$
✓ |  ✓ |  ✓ |  ✗ | $68.4$ | $36\times$
✓ |  ✓ |  ✓ |  ✓ | $\mathbf{69.0}$ | $108\times$

LiDAR Segmentation Model Capacities. We compare six variants of the MinkUNet
[20] segmentor, evaluating their training memory footprint, training speed,
accuracy, and inference speed on SemanticKITTI, as shown in Tab. 5. Using an
NVIDIA A100 GPU for measurements, we find that MinkUNet-50-w32 with mixed
precision training achieves the highest mIoU of $67.6\%$, while
MinkUNet-18-w16 offers the fastest inference speed at $32.4$ FPS. The commonly
used MinkUNet-34-w32 provides an optimal balance between the LiDAR
segmentation accuracy and speed, making it a preferred choice for various
applications. This detailed comparison allows researchers to make informed
decisions about which model variant to use based on their specific needs.
MinkUNet-18-w16 and MinkUNet-34-w32 are composed of BasicBlocks, while
MinkUNet-50-w32 and MinkUNet-101-w32 are built with Bottleneck Blocks, similar
to ResNet architectures. This variety in model complexity and performance
metrics provides flexibility for deployment in diverse scenarios, where either
high accuracy or faster inference speeds may be prioritized.

Mixing-Based Data Augmentations. We evaluate the effectiveness of mixing-based
data augmentation techniques on SemanticKITTI [4], with results detailed in
Tab. 6. Using PolarMix [89] and LaserMix [44] individually enhances the
performance of MinkUNet [20] by $2\%$ mIoU and $1.1\%$ mIoU, respectively.
Combining PolarMix and LaserMix with a random choice strategy yields a $3.5\%$
improvement over using them separately. Similar improvements are observed with
Cylinder3D [102]. For range view segmentors like FRNet [95], FrustumMix offers
superior performance gains compared to PolarMix or LaserMix, establishing it
as the default setting for range view segmentation benchmarks. These results
underscore the importance of advanced data augmentation techniques in
enhancing model robustness and performance, providing a clear path for further
improvements in LiDAR segmentation models. The combination of multiple data
augmentation strategies maximizes the exposure of the model to diverse
scenarios, improving its ability to generalize and perform well in real-world
dense scene understanding applications.

Test Time Augmentation (TTA). Tab. 7 presents the impact of various TTA
strategies on model performance. Starting with flipping along different axes,
TTA improves the performance of MinkUNet [20] by $1.6\%$ mIoU. Adding
rotation, scaling, and translation further boosts MinkUNet’s performance to
$71.8\%$ mIoU on SemanticKITTI [4]. While TTA also benefits Cylinder3D [102]
and FRNet [95], extensive use of TTA significantly increases inference time.
For example, comprehensive TTA techniques result in a $108$-fold increase in
inference time compared to no TTA, underscoring the trade-off between
performance gains and computational cost. These findings highlight the
potential of TTA in improving model accuracy in controlled environments while
also illustrating the practical limitations in real-time applications, such as
in-vehicle LiDAR segmentation. The choice of TTA techniques should be
carefully considered based on the specific requirements of the application,
balancing the need for improved accuracy with the constraints of computational
resources and inference speed.

##  5 Discussions & Future Directions

The comprehensive evaluation of LiDAR segmentation models using the
MMDetection3D-lidarseg toolbox has provided several key insights into the
performance and utility of various models and techniques in real-world
applications. Our experiments have highlighted the strengths and weaknesses of
different segmentation strategies, data augmentation techniques, and sparse
convolution backends, contributing valuable knowledge to the field of LiDAR
segmentation. The key takeaways are summarized as follows:

  * •

Our codebase unified framework streamlines development and benchmarking by
supporting diverse models and backends. This reduces fragmentation and sets
new standards for model evaluation.

  * •

Advanced 3D data augmentation techniques, such as LaserMix, PolarMix, and
FrustumMix significantly enhance model robustness and generalization.
Combining these strategies yields notable performance improvements and helps
models adapt to diverse environmental conditions.

  * •

The comparison of sparse convolution backends shows that SpConv v2 and
Torchsparse++ offer superior training and inference speeds with AMP. This is
crucial for deploying LiDAR segmentation models in resource-constrained and
real-time environments.

  * •

Evaluations of MinkUNet variants reveal that larger models like
MinkUNet-50-w32 achieve higher accuracy but at increased computational costs,
while smaller models offer faster inference at the expense of accuracy.
MinkUNet-34-w32 strikes a good balance between accuracy and efficiency.

  * •

Test Time Augmentation (TTA) strategies improve model performance but
significantly increase inference time, highlighting a trade-off crucial for
real-time applications like autonomous driving.

Future work will expand MMDetection3D-lidarseg to include more state-of-the-
art LiDAR segmentation models, providing a broader range of tools for
researchers. Further research into novel data augmentation techniques will aim
to enhance model robustness and generalization by simulating a wider variety
of real-world scenarios. Optimization of current backends will continue,
aiming for greater efficiency and performance. Enhancing the toolbox’s
capabilities to support semi-supervised and weakly-supervised learning will
reduce the need for extensive manual annotations, facilitating large-scale
dataset creation.

Beyond autonomous driving, the principles and techniques from
MMDetection3D-lidarseg can be applied to robotics, augmented reality, and
urban planning. Future research will explore these applications, adapting and
optimizing the toolbox for various use cases. By making the codebase and
trained models publicly available, we aim to foster a collaborative research
environment. Contributions from the broader 3D computer vision research
community will drive innovation and accelerate the development of more
reliable LiDAR segmentation models.

##  6 Broader Impact

Positive Societal Impacts. MMDetection3D-lidarseg can significantly advance
autonomous driving and related fields by improving LiDAR segmentation. This
enhancement can lead to safer and more efficient autonomous systems, reducing
traffic accidents and fatalities. Beyond transportation, applications in
robotics, augmented reality, urban planning, and environmental monitoring can
benefit from improved 3D environment understanding, leading to advancements in
automation, immersive experiences, infrastructure development, and accurate
environmental mapping.

Negative Societal Impacts. The potential negative impacts include the misuse
of enhanced LiDAR technology for surveillance, which might raise privacy
concerns. Additionally, increased automation could reduce the demand for human
labor in industries like transportation and manufacturing, potentially leading
to job displacement. To mitigate these risks, it is crucial to adhere to
ethical guidelines, ensuring privacy-preserving techniques are integrated into
technology development.

##  7 Conclusion

In this work, we introduced MMDetection3D-lidarseg, a comprehensive toolbox
designed to facilitate the training and evaluation of state-of-the-art LiDAR
segmentation models. By supporting a wide range of segmentation models and
integrating advanced data augmentation techniques, our codebase enhances the
robustness and generalization capabilities of LiDAR segmentation models.
Furthermore, the toolbox provides support for multiple leading sparse
convolution backends, optimizing computational efficiency and performance
across various hardware configurations. Our extensive benchmark experiments on
widely-used datasets, including SemanticKITTI, nuScenes, and ScribbleKITTI,
demonstrate the effectiveness of MMDetection3D-lidarseg in diverse autonomous
driving scenarios. The results highlight the toolbox’s ability to streamline
development, improve benchmarking consistency, and set new standards for
research and application in the field of LiDAR segmentation. By making the
codebase and trained models publicly available, MMDetection3D-lidarseg
promotes further research, collaboration, and innovation within the community.
We believe that this unified framework will accelerate advancements in
autonomous driving technologies, ultimately contributing to the development of
safer and more reliable autonomous systems. Future work will focus on
expanding the capabilities of MMDetection3D-lidarseg, including the
incorporation of additional segmentation models, further optimization of
sparse convolution backends, and the integration of more advanced data
augmentation techniques. We also plan to explore the application of this
toolbox in other domains that can benefit from precise 3D environment
understanding, such as robotics and augmented reality.

## Acknowledgements

We acknowledge the use of the following public resources, during the course of
this work:

  * •

MMCV222https://github.com/open-mmlab/mmcv. Apache License 2.0

  * •

MMDetection333https://github.com/open-mmlab/mmdetection. Apache License 2.0

  * •

MMDetection3D444https://github.com/open-mmlab/mmdetection3d. Apache License
2.0

  * •

MMEngine555https://github.com/open-mmlab/mmengine. Apache License 2.0

  * •

OpenPCSeg666https://github.com/PJLab-ADG/OpenPCSeg. Apache License 2.0

  * •

nuScenes777https://www.nuscenes.org/nuscenes. CC BY-NC-SA 4.0

  * •

nuScenes-devkit888https://github.com/nutonomy/nuscenes-devkit. Apache License
2.0

  * •

SemanticKITTI999http://semantic-kitti.org. CC BY-NC-SA 4.0

  * •

SemanticKITTI-API101010https://github.com/PRBonn/semantic-kitti-api. MIT
License

  * •

ScribbleKITTI111111https://github.com/ouenal/scribblekitti. Unknown

  * •

lidar-bonnetal121212https://github.com/PRBonn/lidar-bonnetal. MIT License

  * •

CENet131313https://github.com/huixiancheng/CENet. MIT License

  * •

FRNet141414https://github.com/Xiangxu-0103/FRNet. Apache License 2.0

  * •

PolarSeg151515https://github.com/edwardzhou130/PolarSeg. BSD 3-Clause License

  * •

MinkowskiEngine161616https://github.com/NVIDIA/MinkowskiEngine. MIT License

  * •

TorchSparse171717https://github.com/mit-han-lab/torchsparse. MIT License

  * •

SPVNAS181818https://github.com/mit-han-lab/spvnas. MIT License

  * •

Cylinder3D191919https://github.com/xinge008/Cylinder3D. Apache License 2.0

  * •

SpConv202020https://github.com/traveller59/spconv. Apache License 2.0

  * •

LaserMix212121https://github.com/ldkong1205/LaserMix. CC BY-NC-SA 4.0

  * •

PolarMix222222https://github.com/xiaoaoran/polarmix. MIT License

## References

  * [1] Angelika Ando, Spyros Gidaris, Andrei Bursuc, Gilles Puy, Alexandre Boulch, and Renaud Marlet.  Rangevit: Towards vision transformers for 3d semantic segmentation in autonomous driving.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5240–5250, 2023.
  * [2] Mehmet Aygun, Aljosa Osep, Mark Weber, Maxim Maximov, Cyrill Stachniss, Jens Behley, and Laura Leal-Taixé.  4d panoptic lidar segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5527–5537, 2021.
  * [3] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Jürgen Gall, and Cyrill Stachniss.  Towards 3d lidar-based semantic scene understanding of 3d point cloud sequences: The semantickitti dataset.  International Journal of Robotics Research, 40:959–96, 2021.
  * [4] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Juergen Gall.  Semantickitti: A dataset for semantic scene understanding of lidar sequences.  In IEEE/CVF International Conference on Computer Vision, pages 9297–9307, 2019.
  * [5] Jens Behley, Andres Milioto, and Cyrill Stachniss.  A benchmark for lidar-based panoptic segmentation based on kitti.  In IEEE International Conference on Robotics and Automation, pages 13596–13603, 2021.
  * [6] Alexandre Boulch, Corentin Sautier, Björn Michele, Gilles Puy, and Renaud Marlet.  Also: Automotive lidar self-supervision by occupancy estimation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13455–13465, 2023.
  * [7] Christopher Lang Alexander Braun, Lars Schillingmann, and Abhinav Valada.  A point-based approach to efficient lidar multi-task perception.  arXiv preprint arXiv:2404.12798, 2024.
  * [8] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.  nuscenes: A multimodal dataset for autonomous driving.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11621–11631, 2020.
  * [9] Anh-Quan Cao and Raoul De Charette.  Monoscene: Monocular 3d semantic scene completion.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991–4001, 2022.
  * [10] Anh-Quan Cao, Angela Dai, and Raoul de Charette.  Pasco: Urban 3d panoptic scene completion with uncertainty awareness.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
  * [11] Jun Cen, Peng Yun, Shiwei Zhang, Junhao Cai, Di Luan, Mingqian Tang, Ming Liu, and Michael Yu Wang.  Open-world semantic segmentation for lidar point clouds.  In European Conference on Computer Vision, pages 318–334, 2022.
  * [12] Qi Chen, Sourabh Vora, and Oscar Beijbom.  Polarstream: Streaming lidar object detection and segmentation with polar pillars.  In Advances in Neural Information Processing Systems, volume 34, 2021.
  * [13] Runnan Chen, Youquan Liu, Lingdong Kong, Nenglun Chen, Xinge Zhu, Yuexin Ma, Tongliang Liu, and Wenping Wang.  Towards label-free scene understanding by vision foundation models.  In Advances in Neural Information Processing Systems, volume 36, 2023.
  * [14] Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang.  Clip2scene: Towards label-efficient 3d scene understanding by clip.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7020–7030, 2023.
  * [15] Xieyuanli Chen, Shijie Li, Benedikt Mersch, Louis Wiesmann, Jürgen Gall, Jens Behley, and Cyrill Stachniss.  Moving object segmentation in 3d lidar data: A learning-based approach exploiting sequential data.  IEEE Robotics and Automation Letters, 6(4):6529–6536, 2021.
  * [16] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang.  Semi-supervised semantic segmentation with cross pseudo supervision.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2613–2622, 2021.
  * [17] Huixian Cheng, Xianfeng Han, and Guoqiang Xiao.  Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving.  In IEEE International Conference on Multimedia and Expo, pages 1–6, 2022.
  * [18] Huixian Cheng, Xianfeng Han, and Guoqiang Xiao.  Transrvnet: Lidar semantic segmentation with transformer.  IEEE Transactions on Intelligent Transportation Systems, 24(6):5895–5907, 2023.
  * [19] Ran Cheng, Ryan Razani, Ehsan Taghavi, Enxu Li, and Bingbing Liu.  Af2-s3net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12547–12556, 2021.
  * [20] Christopher Choy, JunYoung Gwak, and Silvio Savarese.  4d spatio-temporal convnets: Minkowski convolutional neural networks.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075–3084, 2019.
  * [21] MMDetection3D Contributors.  MMDetection3D: OpenMMLab next-generation platform for general 3D object detection.  https://github.com/open-mmlab/mmdetection3d, 2020.
  * [22] Pointcept Contributors.  Pointcept: A codebase for point cloud perception research.  https://github.com/Pointcept/Pointcept, 2023.
  * [23] Spconv Contributors.  Spconv: Spatially sparse convolution library.  https://github.com/traveller59/spconv, 2022.
  * [24] Tiago Cortinhal, George Tzelepis, and Eren Erdal Aksoy.  Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds.  In International Symposium on Visual Computing, pages 207–222, 2020.
  * [25] Bertrand Douillard, James Underwood, Noah Kuntz, Vsevolod Vlaskine, Alastair Quadros, Peter Morton, and Alon Frenkel.  On the segmentation of 3d lidar point clouds.  In IEEE International Conference on Robotics and Automation, pages 2798–2805, 2011.
  * [26] Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu.  A density-based algorithm for discovering clusters in large spatial databases with noise.  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 226–231, 1996.
  * [27] Martin A. Fischler and Robert C. Bolles.  Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography.  Communications of the ACM, 24(6):381–395, 1981.
  * [28] Whye Kit Fong, Rohit Mohan, Juana Valeria Hurtado, Lubing Zhou, Holger Caesar, Oscar Beijbom, and Abhinav Valada.  Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking.  IEEE Robotics and Automation Letters, 7(2):3795–3802, 2022.
  * [29] Geoff French, Timo Aila, Samuli Laine, Michal Mackiewicz, and Graham Finlayson.  Semi-supervised semantic segmentation needs strong, high-dimensional perturbations.  In British Machine Vision Conference, 2020.
  * [30] Biao Gao, Yancheng Pan, Chengkun Li, Sibo Geng, and Huijing Zhao.  Are we hungry for 3d lidar data for semantic segmentation? a survey of datasets and methods.  IEEE Transactions on Intelligent Transportation Systems, 23(7):6063–6081, 2021.
  * [31] Stefano Gasperini, Mohammad-Ali Nikouei Mahani, Alvaro Marcos-Ramiro, Nassir Navab, and Federico Tombari.  Panoster: End-to-end panoptic segmentation of lidar point clouds.  IEEE Robotics and Automation Letters, 6(2):3216–3223, 2021.
  * [32] Andreas Geiger, Philip Lenz, and Raquel Urtasun.  Are we ready for autonomous driving? the kitti vision benchmark suite.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3354–3361, 2012.
  * [33] Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc Van Gool.  Lidar snowfall simulation for robust 3d object detection.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16364–16374, 2022.
  * [34] Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Fog simulation on real lidar point clouds for 3d object detection in adverse weather.  In IEEE/CVF International Conference on Computer Vision, pages 15283–15292, 2021.
  * [35] Fangzhou Hong, Lingdong Kong, Hui Zhou, Xinge Zhu, Hongsheng Li, and Ziwei Liu.  Unified 3d and 4d panoptic segmentation via dynamic shifting networks.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3480–3495, 2024.
  * [36] Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen Change Loy, and Yikang Li.  Point-to-voxel knowledge distillation for lidar semantic segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition, pages 8479–8488, 2022.
  * [37] Qingyong Hu, Bo Yang, Sheikh Khalid, Wen Xiao, Niki Trigoni, and Andrew Markham.  Towards semantic segmentation of urban-scale 3d point clouds: A dataset, benchmarks and challenges.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4977–4987, 2021.
  * [38] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham.  Randla-net: Efficient semantic segmentation of large-scale point clouds.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11108–11117, 2020.
  * [39] Maximilian Jaritz, Tuan-Hung Vu, Raoul de Charette, Emilie Wirbel, and Patrick Pérez.  xmuda: Cross-modal unsupervised domain adaptation for 3d semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12605–12614, 2020.
  * [40] Alok Jhaldiyal and Navendu Chaudhary.  Semantic segmentation of 3d lidar data using deep learning: a review of projection-based methods.  Applied Intelligence, 53(6):6844–6855, 2023.
  * [41] Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, and Ziwei Liu.  Rethinking range view representation for lidar segmentation.  In IEEE/CVF International Conference on Computer Vision, pages 228–240, 2023.
  * [42] Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu.  Robo3d: Towards robust and reliable 3d perception against corruptions.  In IEEE/CVF International Conference on Computer Vision, pages 19994–20006, 2023.
  * [43] Lingdong Kong, Niamul Quader, and Venice Erin Liong.  Conda: Unsupervised domain adaptation for lidar segmentation via regularized domain concatenation.  In IEEE International Conference on Robotics and Automation, pages 9338–9345, 2023.
  * [44] Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu.  Lasermix for semi-supervised lidar semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21705–21715, 2023.
  * [45] Lingdong Kong, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng, Benoit R. Cottereau, and Wei Tsang Ooi.  Robodepth: Robust out-of-distribution depth estimation under corruptions.  In Advances in Neural Information Processing Systems, volume 36, 2023.
  * [46] Lingdong Kong, Xiang Xu, Jun Cen, Wenwei Zhang, Liang Pan, Kai Chen, and Ziwei Liu.  Calib3d: Calibrating model preferences for reliable 3d scene understanding.  arXiv preprint arXiv:2403.17010, 2024.
  * [47] Lingdong Kong, Xiang Xu, Jiawei Ren, Wenwei Zhang, Liang Pan, Kai Chen, Wei Tsang Ooi, and Ziwei Liu.  Multi-modal data-efficient 3d scene understanding for autonomous driving.  arXiv preprint arXiv:2405.05258, 2024.
  * [48] Loic Landrieu, Hugo Raguet, Bruno Vallet, Clément Mallet, and Martin Weinmann.  A structured regularization framework for spatially smoothing semantic labelings of 3d point clouds.  ISPRS Journal of Photogrammetry and Remote Sensing, 132:102–118, 2017.
  * [49] Jiale Li, Hang Dai, and Yong Ding.  Self-distillation for robust lidar semantic segmentation in autonomous driving.  In European Conference on Computer Vision, pages 659–676, 2022.
  * [50] Jiale Li, Hang Dai, Hao Han, and Yong Ding.  Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21694–21704, 2023.
  * [51] Li Li, Hubert P. H. Shum, and Toby P. Breckon.  Less is more: Reducing task and model complexity for 3d point cloud semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9361–9371, 2023.
  * [52] Rong Li, Raoul de Charette, and Anh-Quan Cao.  Coarse3d: Class-prototypes for contrastive learning in weakly-supervised 3d point cloud segmentation.  In British Machine Vision Conference, 2022.
  * [53] Rong Li, Shijie Li, Xieyuanli Chen, Teli Ma, Wang Hao, Juergen Gall, and Junwei Liang.  Tfnet: Exploiting temporal cues for fast and accurate lidar semantic segmentation.  arXiv preprint arXiv:2309.07849, 2023.
  * [54] Ye Li, Lingdong Kong, Hanjiang Hu, Xiaohao Xu, and Xiaonan Huang.  Optimizing lidar placements for robust driving perception in adverse conditions.  arXiv preprint arXiv:2403.17009, 2024.
  * [55] Guibiao Liao, Jiankun Li, and Xiaoqing Ye.  Vlm2scene: Self-supervised image-text-lidar learning with foundation models for autonomous driving scene understanding.  In AAAI Conference on Artificial Intelligence, pages 3351–3359, 2024.
  * [56] Venice Erin Liong, Thi Ngoc Tho Nguyen, Sergi Widjaja, Dhananjai Sharma, and Zhuang Jie Chong.  Amvnet: Assertion-based multi-view fusion network for lidar semantic segmentation.  arXiv preprint arXiv:2012.04934, 2020.
  * [57] Minghua Liu, Yin Zhou, Charles R. Qi, Boqing Gong, Hao Su, and Dragomir Anguelov.  Less: Label-efficient semantic segmentation for lidar point clouds.  In European Conference on Computer Vision, pages 70–89, 2022.
  * [58] Youquan Liu, Yeqi Bai, Lingdong Kong, Runnan Chen, Yuenan Hou, Botian Shi, and Li Yikang.  Openpcseg: An open source point cloud segmentation codebase.  https://github.com/PJLab-ADG/PCSeg, 2023.
  * [59] Youquan Liu, Runnan Chen, Xin Li, Lingdong Kong, Yuchen Yang, Zhaoyang Xia, Yeqi Bai, Xinge Zhu, Yuexin Ma, Yikang Li, Yu Qiao, and Yuenan Hou.  Uniseg: A unified multi-modal lidar segmentation network and the openpcseg codebase.  In IEEE/CVF International Conference on Computer Vision, pages 21662–21673, 2023.
  * [60] Youquan Liu, Lingdong Kong, Jun Cen, Runnan Chen, Wenwei Zhang, Liang Pan, Kai Chen, and Ziwei Liu.  Segment any point cloud sequences by distilling vision foundation models.  In Advances in Neural Information Processing Systems, volume 36, 2023.
  * [61] Youquan Liu, Lingdong Kong, Xiaoyang Wu, Runnan Chen, Xin Li, Liang Pan, Ziwei Liu, and Yuexin Ma.  Multi-space alignments towards universal lidar segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
  * [62] Romain Loiseau, Mathieu Aubry, and Loïc Landrieu.  Online segmentation of lidar sequences: Dataset and algorithm.  In European Conference on Computer Vision, pages 301–317, 2022.
  * [63] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  In International Conference on Learning Representations, 2018.
  * [64] Rodrigo Marcuzzi, Lucas Nunes, Louis Wiesmann, Jens Behley, and Cyrill Stachniss.  Mask-based panoptic lidar segmentation for autonomous driving.  IEEE Robotics and Automation Letters, 8(2):1141–1148, 2023.
  * [65] Qiang Meng, Xiao Wang, JiaBao Wang, Liujiang Yan, and Ke Wang.  Small, versatile and mighty: A range-view perception framework.  arXiv preprint arXiv:2403.00325, 2024.
  * [66] Bjoern Michele, Alexandre Boulch, Gilles Puy, Tuan-Hung Vu, Renaud Marlet, and Nicolas Courty.  Saluda: Surface-based automotive lidar unsupervised domain adaptation.  In International Conference on 3D Vision, 2024.
  * [67] Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss.  Rangenet++: Fast and accurate lidar semantic segmentation.  In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4213–4220, 2019.
  * [68] Aljoša Ošep, Tim Meinhardt, Francesco Ferroni, Neehar Peri, Deva Ramanan, and Laura Leal-Taixé.  Better call sal: Towards learning to segment anything in lidar.  arXiv preprint arXiv:2403.13129, 2024.
  * [69] Aldi Piroli, Vinzenz Dallabetta, Johannes Kopp, Marc Walessa, Daniel Meissner, and Klaus Dietmayer.  Label-efficient semantic segmentation of lidar point clouds in adverse weather conditions.  IEEE Robotics and Automation Letters, 9(6):5575–5582, 2024.
  * [70] Gilles Puy, Alexandre Boulch, and Renaud Marlet.  Using a waffle iron for automotive point cloud semantic segmentation.  In IEEE/CVF International Conference on Computer Vision, pages 3379–3389, 2023.
  * [71] Gilles Puy, Spyros Gidaris, Alexandre Boulch, Oriane Siméoni, Corentin Sautier, Patrick Pérez, Andrei Bursuc, and Renaud Marlet.  Three pillars improving vision foundation model distillation for lidar.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
  * [72] Giulia Rizzoli, Francesco Barbato, and Pietro Zanuttigh.  Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives.  Technologies, 10(4):90, 2022.
  * [73] Corentin Sautier, Gilles Puy, Alexandre Boulch, Renaud Marlet, and Vincent Lepetit.  Bevcontrast: Self-supervision in bev space for automotive lidar point clouds.  In International Conference on 3D Vision, 2024.
  * [74] Kshitij Sirohi, Rohit Mohan, Daniel Büscher, Wolfram Burgard, and Abhinav Valada.  Efficientlps: Efficient lidar panoptic segmentation.  IEEE Transactions on Robotics, 38(3):1894–1914, 2021.
  * [75] Leslie N Smith and Nicholay Topin.  Super-convergence: Very fast training of neural networks using large learning rates.  In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, volume 11006, pages 369–386, 2019.
  * [76] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov.  Scalability in perception for autonomous driving: Waymo open dataset.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2446–2454, 2020.
  * [77] Haotian Tang, Zhijian Liu, Xiuyu Li, Yujun Lin, and Song Han.  Torchsparse: Efficient point cloud inference engine.  Proceedings of Machine Learning and Systems, 4:302–315, 2022.
  * [78] Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han.  Searching efficient 3d architectures with sparse point-voxel convolution.  In European Conference on Computer Vision, pages 685–702, 2020.
  * [79] Haotian Tang, Shang Yang, Zhijian Liu, Ke Hong, Zhongming Yu, Xiuyu Li, Guohao Dai, Yu Wang, and Song Han.  Torchsparse++: Efficient point cloud engine.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 202–209, 2023.
  * [80] Antti Tarvainen and Harri Valpola.  Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.  In Advances in Neural Information Processing Systems, volume 30, 2017.
  * [81] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas.  Kpconv: Flexible and deformable convolution for point clouds.  In IEEE/CVF International Conference on Computer Vision, pages 6411–6420, 2019.
  * [82] Larissa T. Triess, Mariella Dreissig, Christoph B. Rist, and J. Marius Zöllner.  A survey on deep domain adaptation for lidar perception.  In IEEE Intelligent Vehicles Symposium Workshops, pages 350–357, 2021.
  * [83] Larissa T. Triess, David Peter, Christoph B. Rist, and J. Marius Zöllner.  Scan-based semantic segmentation of lidar point clouds: An experimental study.  In IEEE Intelligent Vehicles Symposium, pages 1116–1121, 2020.
  * [84] Marc Uecker, Tobias Fleck, Marcel Pflugfelder, and J. Marius Zöllner.  Analyzing deep learning representations of point clouds for real-time in-vehicle lidar perception.  arXiv preprint arXiv:2210.14612, 2022.
  * [85] Ozan Unal, Dengxin Dai, and Luc Van Gool.  Scribble-supervised lidar semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2697–2707, 2022.
  * [86] Nina Varney, Vijayan K. Asari, and Quinn Graehling.  Dales: A large-scale aerial lidar data set for semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 186–187, 2020.
  * [87] Yuan Wang, Tianyue Shi, Peng Yun, Lei Tai, and Ming Liu.  Pointseg: Real-time semantic segmentation based on 3d lidar point cloud.  arXiv preprint arXiv:1807.06288, 2018.
  * [88] Martin Weinmann, Boris Jutzi, Stefan Hinz, and Clément Mallet.  Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers.  ISPRS Journal of Photogrammetry and Remote Sensing, 105:286–304, 2015.
  * [89] Aoran Xiao, Jiaxing Huang, Dayan Guan, Kaiwen Cui, Shijian Lu, and Ling Shao.  Polarmix: A general data augmentation technique for lidar point clouds.  In Advances in Neural Information Processing Systems, volume 35, pages 11035–11048, 2022.
  * [90] Aoran Xiao, Jiaxing Huang, Dayan Guan, Fangneng Zhan, and Shijian Lu.  Transfer learning from synthetic to real lidar point cloud for semantic segmentation.  In AAAI Conference on Artificial Intelligence, pages 12795–2803, 2022.
  * [91] Aoran Xiao, Jiaxing Huang, Dayan Guan, Xiaoqin Zhang, Shijian Lu, and Ling Shao.  Unsupervised point cloud representation learning with deep neural networks: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):11321–11339, 2023.
  * [92] Shaoyuan Xie, Lingdong Kong, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu.  Robobev: Towards robust bird’s eye view perception under corruptions.  arXiv preprint arXiv:2304.06719, 2023.
  * [93] Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka.  Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation.  In European Conference on Computer Vision, pages 1–19, 2020.
  * [94] Jingyi Xu, Weidong Yang, Lingdong Kong, Youquan Liu, Rui Zhang, Qingyuan Zhou, and Ben Fei.  Visual foundation models boost cross-modal unsupervised domain adaptation for 3d semantic segmentation.  arXiv preprint arXiv:2403.10001, 2024.
  * [95] Xiang Xu, Lingdong Kong, Hui Shuai, and Qingshan Liu.  Frnet: Frustum-range networks for scalable lidar segmentation.  arXiv preprint arXiv:2312.04484, 2023.
  * [96] Xu Yan, Jiantao Gao, Chao Zheng Chaoda Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li.  2dpass: 2d priors assisted semantic segmentation on lidar point clouds.  In European Conference on Computer Vision, pages 677–695, 2022.
  * [97] Yan Yan, Yuxing Mao, and Bo Li.  Second: Sparsely embedded convolutional detection.  Sensors, 18(10):3337, 2018.
  * [98] Kaicheng Yu, Tang Tao, Hongwei Xie, Zhiwei Lin, Zhongwei Wu, Zhongyu Xia, Tingting Liang, Haiyang Sun, Jiong Deng, Dayang Hao, Yongtao Wang, Xiaodan Liang, and Bing Wang.  Benchmarking the robustness of lidar-camera fusion for 3d object detection.  arXiv preprint arXiv:2205.14951, 2022.
  * [99] Dimitris Zermas, Izzat Izzat, and Nikolaos Papanikolopoulos.  Fast segmentation of 3d point clouds: A paradigm on lidar data for autonomous vehicle applications.  In IEEE International Conference on Robotics and Automation, pages 5067–5073, 2017.
  * [100] Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, and Hassan Foroosh.  Polarnet: An improved grid representation for online lidar point clouds semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9601–9610, 2020.
  * [101] Yiming Zhao, Lin Bai, and Xinming Huang.  Fidnet: Lidar point cloud semantic segmentation with fully interpolation decoding.  In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4453–4458, 2021.
  * [102] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin.  Cylindrical and asymmetrical 3d convolution networks for lidar segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9939–9948, 2021.
  * [103] Yang Zou, Zhiding Yu, B V K Vijaya Kumar, and Jinsong Wang.  Unsupervised domain adaptation for semantic segmentation via class-balanced self-training.  In European Conference on Computer Vision, pages 289–305, 2018.