# SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention
Clustering

Enxu Li*, Ryan Razani*, Yixuan Xu, and Liu Bingbing
Huawei Noah’s Ark Lab, Toronto, Canada
{thomas.enxu.li, ryan.razani, richard.xu2<EMAIL_ADDRESS>Indicates
equal contribution.

###### Abstract

Panoptic segmentation aims to address semantic and instance segmentation
simultaneously in a unified framework. However, an efficient solution of
panoptic segmentation in applications like autonomous driving is still an open
research problem. In this work, we propose a novel LiDAR-based panoptic
system, called SMAC-Seg. We present a learnable sparse multi-directional
attention clustering to segment multi-scale foreground instances. SMAC-Seg is
a real-time clustering-based approach, which removes the complex proposal
network to segment instances. Most existing clustering-based methods use the
difference of the predicted and ground truth center offset as the only loss to
supervise the instance centroid regression. However, this loss function only
considers the centroid of the current object, but its relative position with
respect to the neighbouring objects is not considered when learning to
cluster. Thus, we propose to use a novel centroid-aware repel loss as an
additional term to effectively supervise the network to differentiate each
object cluster with its neighbours. Our experimental results show that SMAC-
Seg achieves state-of-the-art performance among all real-time deployable
networks on both large-scale public SemanticKITTI and nuScenes panoptic
segmentation datasets.

##  I INTRODUCTION

Scene understanding is a crucial task in many applications such as autonomous
driving and robotics, attracting research attention in domains like computer
vision and deep learning. Recently, the topic of panoptic segmentation is
introduced to unify the instance segmentation and semantic segmentation in a
single trainable network. The purpose of panoptic segmentation is to identify
the class labels for points/pixels in the “stuff” classes and both class
labels and instance ID’s for points/pixels in the “thing” classes. “Thing” is
referred to all countable objects such as pedestrians, cars, and bikes, while
“stuff” is referred to uncountable semantics (background) such as building,
sidewalk, and road.

Panoptic segmentation in the image domain [1, 2] has reached a mature state
thanks to the structured representation of input images being processed by
standard convolutional networks. However, there are few panoptic segmentation
methods presented for LiDAR point cloud [3, 4]. LiDARs have become a pivotal
sensor modality in perception applications used in autonomous driving and
robotics due to their accurate geometry and light independence. Nonetheless,
the sparsity and non-uniform density of point clouds pose new challenges.

Figure 1: Overview of LiDAR-based SMAC-Seg architecture.

To this end, several deep learning approaches have been introduced to address
these problems. They either process the input point cloud directly [5, 6],
divide the 3D scene into cubical or cylindrical voxel grids [7, 8], or project
the point cloud into a 2D image plane either in spherical range-view (RV) [9,
10, 11] or top-down birds-eye-view (BEV) [12, 13]. While the first two
categories achieve high accuracy compared to projection-based, they are
inefficient and require large memory consumption. In contrast, the projection-
based can achieve real-time operation but suffers from information loss due to
projection operation. Therefore, there is a need to address the panoptic
segmentation problem with a method that offers a real-time and lightweight
computation while alleviating the problems due to the projection of point
clouds.

In this work, we propose a novel LiDAR-based panoptic system, called SMAC-Seg,
as depicted in Fig. 1. We adopt a 2D projection-based approach for the
semantic backbone due to its capability of running in real-time. To recover
the information lost during projection, we propose a novel convolution with
Cross Local Spatial Attention (CLSA). Further, we introduce a learnable
clustering module called SMAC to dynamically aggregate points with kernels in
multiple directions, relocating points towards their centroids and speeding up
the clustering process. Most clustering-based approaches in the literature
[14, 15] are supervised with L2 loss to regress instance centers by penalizing
the difference between predicted and ground truth centers. However, the
relative position of the centers with respect to the neighbouring objects is
not considered when learning to cluster. Therefore, we introduce a novel
centroid-aware repel loss in the training to ensure each object cluster is
differentiated from its closest neighbour.

To summarize, our main contributions are as follows: 1) we propose an end-to-
end LiDAR-based proposal-free panoptic segmentation network that efficiently
clusters the instances, 2) a novel Sparse Multi-directional Attention
Clustering (SMAC) to dynamically aggregate each object cluster in birds-eye-
view (BEV) and segment in real-time, 3) a unique CLSA block to recover the 3D
geometry information lost due to the process of spherical projection, 4) a
novel centroid-aware repel loss to effectively reduce the confusion of each
object cluster with its neighbouring objects, and 5) a comprehensive analysis
on panoptic segmentation performance of our method against existing methods on
two public outdoor datasets, SemanticKITTI [16], and nuScenes [17].

##  II RELATED WORK

Panoptic segmentation task is firstly introduced in images to provide semantic
labels and differentiate each object in the pixel-level [18]. Most of the
panoptic segmentation works in the LiDAR domain are largely built upon
advances made in the image domain. Thus, we first review some major and recent
works in image panoptic segmentation, followed by approaches introduced on
LiDAR point clouds.

###  II-A Image Panoptic Segmentation

The first introduced panoptic segmentation baseline from [18] combines outputs
from PSPNet [19] and Mask R-CNN [20] with a fusion module. Subsequently,
proposal-based methods [21, 1] are the most popular approaches among the
literature where researchers use a combination of RPN and Mask R-CNN to
segment instances. In addition, there are also a handful of proposal-free
approaches. For instance, DeeperLab [22] focuses on locating key points and
regressing offsets followed by a clustering algorithm to segment objects.
Panoptic-DeepLab [2] builds on DeeperLab and introduces a dual-decoder
structure to regress instance object centers and semantic labels,
respectively. In contrast, some research works [23, 24] have also used graph-
based approaches to provide panoptic segmentation results on images.

###  II-B LiDAR Panoptic Segmentation

As a joint task of semantic and instance segmentation, panoptic task usually
builds upon a semantic segmentation network. Existing LiDAR semantic
segmentation networks could be categorized as 2D-based [10, 11], 3D-based [25,
7, 8, 26], and point-based [5, 27, 28, 6]. 2D-based methods usually process
the point cloud in a range view (RV) [10, 11], birds-eye-view (BEV) [12], or a
combination to obtain multi-view [29]. The biggest advantage of this approach
is its high efficiency to achieve real-time performance, benefited from the
fast inference speed of 2D CNNs. However, the 3D geometry information is lost
in the projected 2D representation, which leads to a lack of performance
comparing to 3D-based and point-based approaches. In comparison, 3D-based
methods usually voxelize the point cloud and use sparse 3D convolutions to
process and obtain the segmentation results, which struggle to operate in
real-time despite their state-of-the-art performances. Lastly, point-based
approaches process the raw point cloud directly, yet they require high
computation resources when the point cloud is large-scale, resulting in the
slowest inference speed.

Some early LiDAR panoptic works, as an extended task from semantic
segmentation task, focus on two-stage systems where the semantic segmentation
network is followed by a detection network to generate proposals for objects
in the scene. For instance, [30] uses RangeNet++ [10] or KPConv [6] to predict
the semantics followed by box proposals from PointPillars [31] to segment
objects. PanopticTrackNet [32] is another proposal-based approach where the
instance segmentation proposal head is adapted from Mask R-CNN. EfficientLPS
[33] brings [1] from the image domain and introduce blocks that learns range-
aware features targeting LiDAR point clouds.

Many researchers [3, 34, 14, 15] have also explored using clustering to
segment the foreground point cloud into objects. The pioneering panoptic work
in the LiDAR domain is LPSAD [3]. The authors use a shared encoder with two
decoders where the first decoder predicts the semantic embedding and the
second decoder regresses object centroids for the foreground. Then, a
clustering algorithm is applied to segment instances based on predicted
semantic embedding and the predicted object centroids in the 3D space.
Panoster [34] introduces a learnable clustering to assign instance class
labels to every point and uses post-processing techniques such as DBSCAN to
merge points located close in the 3D space into the same cluster. DS-Net [15]
builds upon meanshift clustering and introduces a learnable dynamic shifting
module to shift points in 3D towards the object centroids in an iterative
manner.

Recently, GP-S3Net [4] introduces a graph-based approach to do instance
segmentation where the network embeds each foreground cluster as a graph node
and predicts a connection between each pair of nodes to form instances.
However, this approach requires a clustering algorithm (e.g., HDBSCAN) to
bridge the semantic and instance network and is not real-time.

##  III PROPOSED METHOD

###  III-A Network Architecture

Figure 2: Illustration of SMAC-Seg. The network consists of a shared encoder
and dual decoder Unet, and a SMAC module. Please note that the instance mask
(top left) is obtained from the ground truth semantic label during training
and from the semantic prediction during testing.

A block diagram of our proposed SMAC-Seg is illustrated in Fig. 2. The input
3D point cloud is denoted as $\textbf{P}=\\{(x,y,z,r,l_{sem},l_{ins})_{i}\mid
i\in\\{1,...,N\\}\\}$ where $N$ is the number of points in the LiDAR point
cloud; $(x,y,z)$ are the Cartesian coordinates in the reference frame centered
at the LiDAR sensor; $r$ is the measure of reflectance returned by the LiDAR
beam. The input LiDAR point cloud P is first projected into a RV image,
denoted as $\mathbf{P}_{RV}\in\mathbb{R}^{H\times W\times C_{i}}$, where $H,W$
are the height and width of the range image and $C_{i}$ is the input features
(Cartesian coordinates, remission and depth). The RV image is passed to a
shared encoder which includes a CLSA block followed by three Cross blocks and
one residual bottleneck block to extract contextual and global features. Then,
the down-sampled feature maps are further processed by a dual decoder to be
mapped to the same resolution as input, $H\times W$. The semantic decoder
predicts semantic classes, denoted as
$\hat{\mathbf{P}}_{sem}\in\mathbb{R}^{H\times W\times C_{cls}}$ with $C_{cls}$
number of classes, while the instance decoder regresses the 2D $x,y$ offset
$\mathbf{P}_{O}$ $\in$ $\mathbb{R}^{H\times W\times 2}$.

To further obtain instance IDs for the foreground, we first apply an instance
mask to filter the point cloud in RV such that only the thing points are
remained, denoted as $\mathbf{P}_{th}$ $\in$ $\mathbb{R}^{M\times 2}$ where
$M$ is the number of remaining foreground points and $2$ is the original $x$
and $y$ coordinates. Note that the mask is obtained from the ground truth
semantic labels during training and is computed from the predicted semantic
labels during testing. Next, we obtain $\mathbf{P}_{s}$ by using
$\mathbf{P}_{O}$, the learned 2D center offset of the corresponding foreground
points from the instance decoder and shifting them towards the object centers.
Further, we project $\mathbf{P}_{s}$ onto a BEV map,
$\mathbf{C}_{bev}\in\mathbb{R}^{h\times w\times 2}$ ($h,w$ are the dimension
of the BEV map different from the dimension of the range image), using the
shifted and discretized $x$ and $y$ coordinates as indices. Consequently, we
generate a binary occupancy mask, $\mathbf{O}\in\mathbb{B}^{h\times w}$, to
mark the the occupied cells as valid entries. At the same time, we build a
hash table, $\mathbf{H}_{f}$, to keep track of the features of the
corresponding location in the BEV map with valid entries as well as another
hash table, $\mathbf{H}_{i}$, for their original indices on range image. In
case of multiple points getting projected to the same BEV location, we take
the mean of their features.

We then apply Sparse Multi-directional Attention (SMA) to aggregate each
cluster in $\mathbf{C}_{bev}$ using attention weights obtained from its
corresponding features from $\mathbf{H}_{f}$. Subsequently, BFS clustering
with a radius of $r$ is used on $\mathbf{C}_{f}$, BEV map generated as the
output of SMA, to differentiate each object thus obtain instance label
$\hat{\mathbf{P}}_{ins}\in\mathbb{R}^{h\times w}$. Lastly, we map the instance
label back to RV using the hash table, $\mathbf{H}_{i}$. The semantic and
instance segmentation RV predictions are then mapped to original 3D domain and
are concatenated as panoptic predictions. To address any conflicts between
semantic and instance predictions, we use majority-voting.

Cross Local Spatial Attention (CLSA)

We extend the Diamond Feature Extractor module in [29] and introduce a
spatially adaptive feature extractor for RV images to incorporate the local 3D
geometry as shown in Fig. 3. Specifically, we replace the regular convolutions
in the second half of the Diamond Block with CLSA convolutions. A 2D
convolution operation can be written as,

$\mathbf{x}^{out}_{\mathbf{u}}=[\mathbf{x}^{in}_{\mathbf{u}}*\mathbf{W}]_{\mathcal{V}^{2}(K)}=\sum_{\mathbf{i}\in\mathcal{V}^{2}(K)}\mathbf{W}_{\mathbf{i}}\mathbf{x}^{in}_{\mathbf{u}+\mathbf{i}}$
(1)

where $\mathbf{u}$ denotes the 2D index to locate each point in the feature
map; $\textbf{W}\in\mathbb{R}^{K\times K\times N^{out}\times N^{in}}$ is the
kernel weight, shared among each sliding window, with $N^{in},N^{out}$ being
the number of input and output feature channels respectively;
$\mathcal{V}^{2}(K)$ is the list of offsets in 2D square with length $K$
centered at the origin. Here, we would like W to be adaptive to the geometry
of each neighbourhood, in particular, with attention built from the relative
positions of the points. Formally, we introduce a 2D convolution with cross
local spatial attention as the following:

$\mathbf{x}^{out}_{\mathbf{u}}=[\mathbf{x}^{in}_{\mathbf{u}}*\tilde{\mathbf{W}}_{\mathbf{u}}]_{\mathcal{N}^{2}(K)}$
(2)
$\tilde{\mathbf{W}}_{\mathbf{u}}=\sigma\Big{[}w\Big{(}\underbrace{\bigcup_{\mathbf{i}\in\mathcal{N}^{2}(K)}\mathbf{c}_{\mathbf{u}+\mathbf{i}}-\mathbf{c}_{\mathbf{u}}}_{\Delta\mathbf{c}_{\mathbf{u}}}\Big{)}\Big{]}$
(3)

Figure 3: Convolution with Cross Local Spatial Attention (CLSA).

where $\tilde{\textbf{W}}\in\mathbb{R}^{(2K-1)\times N^{out}\times
N^{in}\times H\times W}$ is the spatially adaptive kernel weight computed from
the relative geometric positions of the points within the cross-shaped
neighbourhood; $w(.)$ is a 3-layer MLP; $\mathbf{c}_{\mathbf{u}}$ is the
corresponding spatial coordinate feature of $\mathbf{x}_{\mathbf{u}}^{in}$
(i.e. Cartesian $xyz$, depth, and occupancy); $\bigcup$ is the concatenation
operator; $\sigma$ denotes the softmax operation on the spatial dimension to
ensure the attention weights in the neighbourhood for each feature channel are
summed up to 1; and $\mathcal{N}^{2}(K)$ is a set of offsets that define the
shape of a cross kernel with size of $K$. e.g. $\mathcal{N}^{2}(3)$ $=$
$\\{(-1,0),$ $(0,0),$ $(1,0),(0,1),(0,-1)\\}$. We compare the cross kernel
sampling with other regularly used kernel shapes (e.g. square kernel) in
section 7.

CrossUnet Our proposed CrossNet backbone is depicted at the bottom right of
Fig. 2. It includes 3 layers of Cross blocks which are designed to capture
multi-scale features, followed by a bilateral fusion to obtain rich
information at each block. In particular, the input feature is first passed to
a multi branch convolution layers and each branch further processes the
features with Convolution layers of different receptive field (followed by
Relu and BatchNorm layer) to obtain fine-grained information. Next, a
Bilateral fusion module is applied on each branch to fuse the features of
different resolutions. Finally, all the feature maps are concatenated and
their channel numbers are reduced through a final convolution layer for
efficient processing.

Sparse Multi-directional Attention (SMA) The foreground point cloud in BEV,
$\mathbf{C}_{bev}$, is processed by SMA to ensure the points are more
aggregated towards the object centers; thus, a simple and fast clustering
algorithm like BFS can easily differentiate each cluster. Further, for every
valid entry in $\mathbf{C}_{bev}$, we obtain the center of mass of its
neighbourhood in five directions using Eq.4, denoted as
$\mathbf{C}_{W},\mathbf{C}_{E},\mathbf{C}_{N},\mathbf{C}_{S}$, and
$\mathbf{C}_{C}$.

$\mathbf{C}_{\mathbf{u}}=\frac{1}{n_{\mathbf{u}}}\sum_{\mathbf{i}\in\Omega(K)}\mathbf{C}_{bev,\mathbf{u+i}}\cdot\mathbf{O}_{\mathbf{u+i}}$
(4)

where $\Omega(K)$ is a set of 2D indices for each neighbourhood sampling
region with size $K$ and $n$ is the number of valid entries in the
neighbourhood acting as a normalization factor, formally,
$n_{\mathbf{u}}=\sum_{\mathbf{i}\in\Omega}\mathbf{O}_{\mathbf{u+i}}$. We
denote multi-directional neighbour sampling as $\Omega_{W}(K):\\{(0,i)$ $\mid$
$\forall i\in[-K,0]\\}$, $\Omega_{E}(K):\\{(0,i)$ $\mid$ $\forall
i\in[0,K]\\}$, $\Omega_{N}(K):\\{(i,0)$ $\mid$ $\forall i\in[-K,0]\\}$,
$\Omega_{S}(K):\\{(i,0)$ $\mid$ $\forall i\in[0,K]\\}$. The foreground point
cloud at location $\mathbf{u}$ in BEV representation after being processed by
SMA module, $\mathbf{C}_{f,\mathbf{u}}$, can be expressed as,

$\mathbf{C}_{f,\mathbf{u}}=\mathbf{C}_{all,\mathbf{u}}\times\underbrace{\sigma(MLP(\mathbf{H}_{f,\mathbf{u}}))}_{\mathbf{\pi}_{\mathbf{u}}}$
(5)

where $\mathbf{C}_{all,\mathbf{u}}\in\mathbb{R}^{2\times
5}=cat(\mathbf{C}_{W,\mathbf{u}}$,$\mathbf{C}_{E,\mathbf{u}}$,$\mathbf{C}_{N,\mathbf{u}}$,$\mathbf{C}_{S,\mathbf{u}}$,$\mathbf{C}_{C,\mathbf{u}})$
is the concatenated $xy$ centers of mass from applying kernels in five
directions at location $\mathbf{u}$;
$\mathbf{\pi}_{\mathbf{u}}\in\mathbb{R}^{5}$ is the attention weights computed
using the MLP from the foreground features at location $\mathbf{u}$ of the BEV
map; $\sigma$ denotes the softmax operator to ensure the attention weights in
all directions summing up to 1, and $\times$ is matrix multiplication.
Essentially, $\mathbf{C}_{f}$ is the final location of the foreground point
cloud in BEV, shifted towards its neighbouring points after receiving the
directional guidance from the network.

###  III-B Centroid-aware Repel Loss

The main purpose of the SMAC module is not to have an accurate prediction of
the object center, but to have each object forming a cluster that could be
easily differentiated from others in the 2D BEV space. In order to tackle this
problem, we propose a novel Centroid-aware Repel Loss to supervise this
module.

$L_{repel}=\frac{1}{I}\sum_{i=1}^{I}\frac{1}{P_{i}}\sum_{p=1}^{P_{i}}max\\{0,d_{i}-\hat{d}_{i,p}\\}$
(6) $d_{i}=\min_{j\in[1,I]\cap j\neq
i}\lVert\mathbf{C}_{gt,i}-\mathbf{C}_{gt,j}\rVert_{2}$ (7)
$\hat{d}_{i,p}=\min_{\begin{subarray}{c}q\in[1,P_{j}]\\\ j\in[1,I]\cap j\neq
i\end{subarray}}\lVert\mathbf{C}_{f,(i,p)}-\mathbf{C}_{f,(j,q)}\rVert_{2}$ (8)

where $I$ is the total number of instances, $P_{i}$ is the number of occupied
points in $\mathbf{C}_{bev}$ for instance $i$,
$\mathbf{C}_{f,(i,p)}\in\mathbb{R}^{2}$ is the final 2D position after $SMAC$
at point $p$ that belongs to instance $i$, and
$\mathbf{C}_{gt,i}\in\mathbb{R}^{2}$ is the ground truth 2D centroid of
instance $i$. Essentially, $\hat{d}$ represents the closest distance from
itself (final shifted position) to any other point from other objects, and $d$
represents the distance between the ground truth centroid of current object to
the other closest instance. This loss term penalizes if the ground truth
distance, $d$ is larger than $\hat{d}$, meaning the network still needs to
learn such that each foreground cluster is repelled from others.

Figure 4: Illustrations of how the centroid-aware repel loss penalizes the
network. Three instances are present in the scene where black points are from
the original point cloud in BEV, cyan points are the GT centroid for each
instance, and red points are the final location of the points after SMAC. In
this case, the final predicted centroid of instance 2 and 3 are close
($\hat{d}_{2,0}$ is small compare to ground truth $d_{2}$). This loss function
will repel $\mathbf{C}_{f,(2,0)}$ away from $\mathbf{C}_{f,(3,0)}$.

Further, we use an additional loss term to enforce the variance of each
cluster is minimized, as used in [3].

$L_{attract}=\frac{1}{I}\sum_{i=1}^{I}\frac{1}{P_{i}}\sum_{p=1}^{P_{i}}\lVert\mathbf{C}_{f,(i,p)}-\bar{\mathbf{C}}_{f,i}\rVert_{2}$
(9)

where $\bar{\mathbf{C}}_{f,i}$ is the average of the all the point locations
in BEV after SMAC for instance $i$ and the rest of the terms are defined as
the same as in Eq.6. We use three loss terms to supervise the semantic
segmentation, similar to [29] and L2 regression loss, similar to [3, 15] to
supervise the center offset from the instance decoder. Thus, the total loss is
the weighted combination illustrated as follows:

$\begin{subarray}{c}L_{total}=\underbrace{\beta_{wce}L_{wce}+\beta_{ls}L_{ls}+\beta_{tv}L_{tv}}_{\text{semantic
decoder}}+\underbrace{\beta_{l2}L_{l2}}_{\text{instance decoder}}\\\
+\underbrace{\beta_{repel}L_{repel}+\beta_{attract}L_{attract}}_{\text{SMAC}}\end{subarray}$
(10)

##  IV EXPERIMENTS

In this section, we discuss the experimental setup and demonstrate the
performance of SMAC-Seg on popular driving-scene datasets for panoptic
segmentation using 3D LiDAR point clouds. We compared our results with state-
of-the-art approaches and conducted ablation studies to evaluate the
importance of the main components.

###  IV-A Datsets and Evaluation Metrics

In terms of datasets, we used SemanticKITTI dataset [16] and nuScenes dataset
[17]. SemanticKITTI is the first publically available dataset on LiDAR-based
panoptic segmentation for driving scenes. For the panoptic segmentation task,
each point in the dataset is assigned a label of 19 classes. The instance IDs
are available for humans and vehicles, which account for 8 things classes.
Other objects without instance IDs belong to stuff classes.

NuScenes is a popular large-scale driving-scene dataset, where official point-
level panoptic segmentation labels for LiDAR scans are unavailable 111at the
time of writing. Given the 3D bounding box annotations and the semantic
labels, we generated our own labels for training and evaluation. Specifically,
we assigned the same instance ID for all points that have the same semantic
labels and are located within the same bounding box. The 16 labeled classes in
the lidarseg dataset were divided into 8 things classes and 8 stuff classes.
During evaluation, we followed [14] to discard instances that contain fewer
than 20 points. Given the official training-validation split, we trained our
model on 700 training scenes and provided validation results on 150 validation
scenes.

Figure 5: Comparison of SMAC-Seg with LPSAD on both SemanticKITTI and nuScene
dataset.

For metrics, we follow [18] to use the mean Panoptic Quality (PQ) for
evaluation and comparison with other models. We also reported the mean
Recognition Quality (RQ) and the mean Segmentation Quality (SQ) and calculated
these metrics separately on stuff and things classes, providing PQSt, SQSt,
RQSt and PQTh, SQTh, RQTh.

###  IV-B Experimental Setup

We trained our network end-to-end for 100 epochs on SemanticKITTI training
split and evaluated on the test split. All ablation experiments were validated
on seq8 (validation set). The range image resolution was set as $64\times
2048$. For nuScenes, we trained our network for 150 epochs on the training
scenes and evaluated on the validation scenes. We used $32\times 1024$ as the
resolution for the range image as the LiDAR sensor has 32 beams. We provided
result of an additional hi-res model specifically for nuScenes, incorporating
a range image with size $64\times 2048$ to reduce the information loss. For
both datasets, we trained our model using SGD optimizer with a learning rate
of 0.01 and weight decay of $10^{-5}$. 4 NVIDIA V100 GPUs were used and the
batch size per GPU was 4. The BEV grid size was set to be $0.5$m. The SMAC
kernel size was set as $7$. The weights on the loss functions,
$\beta_{wce},\beta_{ls},\beta_{tv},\beta_{l2},\beta_{repel},\beta_{attract}$
were set to be 1.0, 1.0, 5.0, 0.1, 0.1, 0.1 respectively.

We use a dual-decoder U-net based on residual blocks as the baseline where the
first decoder predicts the 2D offsets to object centers and second decoder
predicts semantic class labels. Then we add a clustering algorithm after the
first decoder to segment the foreground into objects. We include both
traditional (e.g. BFS, HDBSCAN [35], and meanshift [36]) and learning-based
(e.g. Dynamic Shifting [15]) clustering in comparison. In particular, the
baseline with Dynamic Shifting clustering is adapted from [15] where we attach
the DS clustering head to our first decoder.

###  IV-C Quantitative Evaluation

Figure 6: Three examples of foreground points before and after SMAC module.
The black points ($\mathbf{P}_{th}$) are from the original LiDAR sensor; blue
points ($\mathbf{P}_{s}$) are the positions of the points after applying the
2D offsets from the instance decoder; red points ($\mathbf{C}_{f}$) are the
final shifted position after the proposed SMAC module.

The performance of SMAC-Seg in comparison to other models is shown in Table I
and Table II. For evaluations on SemanticKITTI test dataset (Table I), the
models are divided into two groups based on latency. The models in row 1-7
either do not have an available inference speed or are not capable to operate
in real-time, while those in row 8-10 are known to be able to operate in real-
time, with FPS’s greater than 10Hz. With a PQ of $56.1\%$ and an FPS of
10.1Hz, our proposed model (row 10) obtains a good balance between run-time
and accuracy. Specifically, it achieves the highest PQ for real-time models,
outperforming the best real-time baseline, Panoptic-PolarNet, by $2.0\%$.

A deeper look into the other metrics reveals that SMAC-Seg is better than
other real-time models in recognizing and differentiating between instances,
achieving an improvement of $1.2\%$ in RQTh over Panoptic-PolarNet. This
demonstrates that the addition of the SMA module to refine predictions of the
instance decoder before clustering is helpful for identifying instances. In
terms of metrics correlated to semantic segmentation performance, despite a
reduction of $1.6\%$ in SQTh over Panoptic-PolarNet, other metrics such as SQ,
PQSt and mIOU see improvements of $0.6\%$, $3.6\%$ and $3.8\%$, respectively.
This means that although the semantic decoder within SMAC-Seg has not obtained
the best results in segmenting certain thing classes, it performs particularly
well in labeling background classes. This higher mIoU and SQ can be mainly
attributed to the CLSA module, which incorporates 3D geometry features that
are otherwise lost after spherical projection.

For evaluations on nuScenes validation dataset (Table II), because the methods
for creating the instance labels for panoptic segmentation may be different
across publications, we separated the models into three groups. The first (row
1-4) and second group (rows 5-6) are existing published methods grouped by
inference speed, similar to Table I. The third group (row 7-10) contains the
baseline models and proposed models used in our experiments. SMAC-Seg HiRes
(rows 10) achieves the highest PQ out of all models, with a $15.8\%$ and
$5.1\%$ improvement over the baseline models Dual Decoder (BFS) and Dual
Decoder (HDBSCAN), respectively.

However, since nuScenes dataset is obtained in an urban setting and contains
more objects per scene than semanticKITTI dataset, the inference time of SMAC-
Seg with a high-resolution range image (row 10) is no longer real-time. By
reducing the resolution from $64\times 2048$ to $32\times$1024, SMAC-Seg (row
9) again achieves a real-time performance with only a small trade-off in PQ.

Method | PQ | PQ$\dagger$ | RQ | SQ | PQTh | RQTh | SQTh | PQSt | RQSt | SQSt | mIoU | FPS
---|---|---|---|---|---|---|---|---|---|---|---|---
RangeNet++ [10] \+ PointPillars [31] | $37.1$ | $45.9$ | $47.0$ | $75.9$ | $20.2$ | $25.2$ | $75.2$ | $49.3$ | $62.8$ | $76.5$ | $52.4$ | $2.4$
PanopticTrackNet [32] | $43.1$ | $50.7$ | $53.9$ | $78.8$ | $28.6$ | $35.5$ | $80.4$ | $53.6$ | $67.3$ | $77.7$ | $52.6$ | 6.8
KPConv [6] \+ PointPillars [31] | $44.5$ | $52.5$ | $54.4$ | $80.0$ | $32.7$ | $38.7$ | $81.5$ | $53.1$ | $65.9$ | $79.0$ | $58.8$ | $1.9$
Panoster [34] | $52.7$ | $59.9$ | $64.1$ | $80.7$ | $49.4$ | $58.5$ | $83.3$ | $55.1$ | $68.2$ | $78.8$ | $59.9$ | $-$
DS-Net [15] | $55.9$ | $62.5$ | $66.7$ | $82.3$ | $55.1$ | $62.8$ | $87.2$ | $56.5$ | $69.5$ | $78.7$ | $61.6$ | $3.4^{\dagger}$
EfficientLPS [33] | $57.4$ | $63.2$ | $68.7$ | 83.0 | $53.1$ | $60.5$ | 87.8 | 60.5 | 74.6 | 79.5 | $61.4$ | $-$
GP-S3Net [4] | 60.0 | 69.0 | 72.1 | $82.0$ | 65.0 | 74.5 | $86.6$ | $56.4$ | $70.4$ | $78.7$ | 70.8 | $3.7^{*}$
LPSAD [3] | $38.0$ | $47.0$ | $48.2$ | $76.5$ | $25.6$ | $31.8$ | $76.8$ | $47.1$ | $60.1$ | $76.2$ | $50.9$ | 11.8
Panoptic-PolarNet [14] | $54.1$ | $60.7$ | $65.0$ | $81.4$ | 53.3 | $60.6$ | 87.2 | $54.8$ | $68.1$ | $77.2$ | $59.5$ | $11.6$
SMAC-Seg [Ours] | 56.1 | 62.5 | 67.9 | 82.0 | $53.0$ | 61.8 | $85.6$ | 58.4 | 72.3 | 79.3 | 63.3 | $10.1$

TABLE I: Comparison of LiDAR panoptic segmentation performance on
SemanticKITTI[16] test dataset. Metrics are provided in [%] and FPS is in
[Hz].(*: measured with our implementation based on [4]; $\dagger$: measured
using official codebase released by the authors.)

Method | PQ | PQ$\dagger$ | RQ | SQ | PQTh | RQTh | SQTh | PQSt | RQSt | SQSt | mIoU | FPS
---|---|---|---|---|---|---|---|---|---|---|---|---
DS-Net [15] | $42.5$ | $51.0$ | $50.3$ | $83.6$ | $32.5$ | $38.3$ | $83.1$ | $59.2$ | $70.3$ | 84.4 | $70.7$ | $-$
PanopticTrackNet [32] | $50.0$ | $57.3$ | $60.6$ | $80.9$ | $45.1$ | $52.4$ | $80.3$ | $58.3$ | $74.3$ | $81.9$ | $63.1$ | $-$
EfficientLPS [33] | $59.2$ | $62.8$ | $70.7$ | $82.9$ | $51.8$ | $62.7$ | $80.6$ | 71.5 | 84.1 | $84.3$ | $69.4$ | $-$
GP-S3Net [4] | 61.0 | 67.5 | 72.0 | 84.1 | 56.0 | 65.2 | 85.3 | $66.0$ | $78.7$ | $82.9$ | 75.8 | $-$
LPSAD [3] | $50.4$ | $57.7$ | $62.4$ | $79.4$ | $43.2$ | $53.2$ | $80.2$ | $57.5$ | $71.7$ | $78.5$ | $62.5$ | $\textbf{22.3}^{*}$
Panoptic-PolarNet [14] | 67.7 | 71.0 | 78.1 | 86.0 | 65.2 | 74.0 | 87.2 | 71.9 | 84.9 | 83.9 | 69.3 | $10.1$
Dual-Dec UNet w/ BFS [Our Baseline] | $52.6$ | $58.0$ | $64.3$ | $79.7$ | $39.2$ | $49.0$ | $77.5$ | $66.0$ | $79.5$ | $81.9$ | $71.1$ | 20.1
Dual-Dec UNet w/ HDBSCAN [Our Baseline] | $63.3$ | $68.8$ | $75.2$ | $83.5$ | $60.8$ | $70.9$ | $85.0$ | $66.0$ | $79.5$ | $81.9$ | $71.1$ | $18.8$
SMAC-Seg [Ours] | $67.0$ | $71.8$ | $78.2$ | $85.0$ | $65.2$ | $74.2$ | $87.1$ | 68.8 | 82.2 | $82.9$ | 72.2 | $14.5$
SMAC-Seg HiRes [Ours] | 68.4 | 73.4 | 79.7 | 85.2 | 68.0 | 77.2 | 87.3 | 68.8 | $82.1$ | 83.0 | $71.2$ | $6.2$

TABLE II: Comparison of LiDAR panoptic segmentation performance on nuScenes
[17] validation dataset. Metrics are provided in [%] and FPS is in [Hz].(*:
measured with our implementation based on [3])

###  IV-D Qualitative Evaluation

In Fig. 5, we demonstrate the panoptic segmentation performance of the SMAC-
Seg model by comparing its inference results to LPSAD, our implementation
based on [3]. For a relatively simple scene from SemanticKITTI dataset with a
row of cars, SMAC-Seg segments the instance points without errors, while LPSAD
identifies the car on the right as two separate instances. In the example from
nuScenes, the scene is more complex, with an intersection and a mix of cars,
trucks, and cyclists present. With few sparse points describing each instance
and variations in object size, correctly predicting object centroids and
segmenting instances is more difficult. In cases like this, LPSAD is prone to
errors. The large truck on the left of the frame is segmented into two
instances, while the cyclist immediately to its right is also segmented into
two. At the bottom of the frames, as two cars are close to each other, LPSAD
mistakenly assigns half of the car on the right to be part of the car on the
left. In comparison, SMAC-Seg is able to segment each instance in a complex
scene with high accuracy. The ability for SMAC-Seg to accurately group points
within the same instance can be primarily attributed to the directional
guidance during offset correction within the SMA module and the supervision of
attract and loss. Meanwhile, the strength of SMAC-Seg in accurately
differentiating between two closely positioned objects is made possible by the
supervision of the centroid-aware repel loss function.

From Fig. 6, we further present the effectiveness of the SMA module. The blue
points are the predicted centroids after shifting each foreground point by the
regressed offsets from instance decoder. Although most points are shifted
towards the center of the instance, some indicated in the figure are shifted
to a wrong direction. A clustering algorithm applied here is not ideal, as the
object will be over-segmented. This indicates that the supervision by L2-loss
within instance decoder is insufficient to aid clustering. On the other hand,
the addition of the SMA module before BFS clustering effectively corrects the
offset predictions and aggregates each object cluster towards its centroid.
This additional relocation of points allows the clustering algorithm to return
a more accurate result, as shown by the final shifted points in red.

###  IV-E Ablation Studies

To further demonstrate the influence of each proposed component, we conducted
ablation studies on the validation set of SemanticKITTI. We first compare our
proposed SMAC module against various clustering algorithms (row 1-5 of Table
III), including BFS, HDBSCAN [35], Meanshift [36], and DS [15]. The results
show that HDBSCAN and Meanshift clustering achieve a slightly better PQ when
segmenting the shifted foreground points, while DS and SMAC with the baseline
backbone are the only candidates here that are capable of operating in real-
time. When benchmarking against DS, our proposed SMAC runs $30ms$ faster with
a PQ improvement of $0.5\%$. Note that SMAC module is a combination of SMA and
BFS clustering, when comparing with the BFS baseline (row 1), the proposed
module not only increases PQ from $44.9\%$ to $51.9\%$ but decreases the
runtime spent on clustering significantly as the foreground points are located
together with other neighbouring points that belong to the same instance and
away from others (as seen in Fig.6). In addition, from row 6-10, we show the
contribution of each proposed components, namely Cross Block, CLSA, Centroid-
aware Repel Loss, and the combinations of them. We also report the FPS of each
entry and bold the ones that operate in real-time.

Architecture |  crossBlock |  CLSA |  Repel |  BFS |  HDB |  MS |  DS |  SMAC | mPQ | FPS
---|---|---|---|---|---|---|---|---|---|---
Baseline |  |  |  |  |  |  |  |  | $44.9$ |  $8.6$
|  |  |  |  |  |  |  | $52.7$ |  $4.8$
|  |  |  |  |  |  |  | $52.8$ |  $3.2$
|  |  |  |  |  |  |  | $51.4$ |  11.1
Proposed |  |  |  |  |  |  |  |  | $51.9$ |  16.1
|  |  |  |  |  |  |  | $46.2$ |  $7.7$
|  |  |  |  |  |  |  | $48.6$ |  $6.6$
|  |  |  |  |  |  |  | $54.1$ |  10.1
|  |  |  |  |  |  |  | $52.9$ |  15.5
|  |  |  |  |  |  |  | 55.8 |  10.1

TABLE III: Ablation study of the proposed components vs baseline.

In addition, We evaluated the effectiveness of convolutions with CLSA on the
semantic segmentation tasks with various kernel shapes as illustrated in Fig.
7. From the results in Table V, cross kernel is the most effective in
utilizing the local neighbourhood geometry. Using CLSA with square dense
kernels performs relatively worse than others; with large square kernels, the
network tends to memorize some of the local geometry and introduces
overfitting to the training set. Nevertheless, CLSA is highly effective as it
improves the mean IoU by $3.3\%$ comparing to the baseline. It learns to
recover the local 3D geometry features, which range-based (RV) approaches lose
after the spherical projection.

Figure 7: Examples of kernel size of 5. From left to right: cross kernel,
diamond kernel, square dense kernel, and square kernel with dilation.

Architecture | Sampling Region | mIoU | Improvement
---|---|---|---
baseline | $-$ | $58.9$ |  $-$
CLSA | Cross Kernel | 62.2 |  +3.3
Diamond Kernel | $61.7$ |  $+2.8$
Square Dense Kernel | $61.1$ |  $+2.2$
Square Kernel with Dilation | $61.7$ |  $+2.8$

TABLE IV: Ablation study of using different neighbourhood sampling with local
spatial attention.

Lastly, we conducted experiments on the SMAC module to see how various BEV
grid sizes and directional kernel sizes could affect the performance. In
particular, we chose to run a total of six experiments with grid size of
$\\{0.3,0.5,1.0\\}$m, and with kernel size of $3$ and $7$ for each. From
Tab.V, the combination of grid size $0.5$ and kernel size $7$ achieves the
best performance.

Grid Size | Kernel Size | PQ | PQTh | RQTh | SQTh
---|---|---|---|---|---
0.3 | $3$ | $55.5$ | $56.2$ | $63.7$ |  76.6
$7$ | $55.5$ | $57.1$ | $64.9$ |  $76.4$
0.5 | $3$ | $55.3$ | $56.2$ | $64.3$ |  $76.0$
$7$ | 55.8 | 57.4 | 65.5 |  $76.4$
1.0 | $3$ | $55.0$ | $55.6$ | $63.8$ |  $75.8$
$7$ | $55.1$ | $55.4$ | $63.4$ |  $76.1$

TABLE V: Ablation study of using different grid size and kernel size in SMAC.

##  V CONCLUSION

We have proposed SMAC-Seg, a novel end-to-end real-time panoptic segmentation
network for LiDAR point cloud. In particular, the model comprises of a robust
encoder including a novel CLSA block to recover 3D geometric features lost due
to projection and multiple Cross blocks to learn and fuse multi-scale
features. To segment each object, each foreground cluster is learned to
effectively aggregate towards its centroid in the proposed SMAC module.
Further, a novel centroid-aware repel loss is added to supervise the model in
differentiating each cluster with its neighbour. From the evaluation on both
large-scale datasets of SemanticKITTI and nuScenes, we have shown that SMAC-
Seg achieves state-of-the-art performance among all published real-time
methods at the time of submission.

## References

  * [1] R. Mohan and A. Valada, “Efficientps: Efficient panoptic segmentation,” _International Journal of Computer Vision_ , vol. 129, pp. 1551 – 1579, 2020\.
  * [2] B. Cheng, M. Collins, Y. Zhu, T. Liu, H. Adam, and L.-C. Chen, “Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation,” 06 2020, pp. 12 472–12 482.
  * [3] A. Milioto, J. Behley, C. McCool, and C. Stachniss, “Lidar panoptic segmentation for autonomous driving,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2020, pp. 8505–8512.
  * [4] R. Razani, R. Cheng, E. Li, E. Taghavi, Y. Ren, and L. Bingbing, “Gp-s3net: Graph-based panoptic sparse semantic segmentation network,” _arXiv preprint arXiv:2108.08401_ , 2021.
  * [5] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 652–660.
  * [6] H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 6411–6420.
  * [7] R. Cheng, R. Razani, E. Taghavi, E. Li, and B. Liu, “(af)2-s3net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021.
  * [8] X. Zhu, H. Zhou, T. Wang, F. Hong, Y. Ma, W. Li, H. Li, and D. Lin, “Cylindrical and asymmetrical 3d convolution networks for lidar segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021, pp. 9939–9948.
  * [9] T. Cortinhal, G. Tzelepis, and E. E. Aksoy, “Salsanext: Fast semantic segmentation of lidar point clouds for autonomous driving,” in _2020 IEEE Intelligent Vehicles Symposium (IV)_ , 2020, pp. 655–661.
  * [10] A. Milioto, I. Vizzo, J. Behley, and C. Stachniss, “Rangenet++: Fast and accurate lidar semantic segmentation,” in _Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS)_ , 2019.
  * [11] R. Razani, R. Cheng, E. Taghavi, and L. Bingbing, “Lite-hdseg: Lidar semantic segmentation using lite harmonic dense convolutions,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021.
  * [12] Y. Zhang, Z. Zhou, P. David, X. Yue, Z. Xi, B. Gong, and H. Foroosh, “Polarnet: An improved grid representation for online lidar point clouds semantic segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 9601–9610.
  * [13] M. Simon, S. Milz, K. Amende, and H.-M. Gross, “Complex-yolo: An euler-region-proposal for real-time 3d object detection on point clouds,” in _Computer Vision – ECCV 2018 Workshops_ , L. Leal-Taixé and S. Roth, Eds. Cham: Springer International Publishing, 2019, pp. 197–209.
  * [14] Z. Zhou, Y. Zhang, and H. Foroosh, “Panoptic-polarnet: Proposal-free lidar point cloud panoptic segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021.
  * [15] F. Hong, H. Zhou, X. Zhu, H. Li, and Z. Liu, “Lidar-based panoptic segmentation via dynamic shifting network,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021\.
  * [16] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “Semantickitti: A dataset for semantic scene understanding of lidar sequences,” in _2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019_. IEEE, 2019, pp. 9296–9306. [Online]. Available: https://doi.org/10.1109/ICCV.2019.00939
  * [17] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 621–11 631.
  * [18] A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dollar, “Panoptic segmentation,” _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019.
  * [19] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” 07 2017, pp. 6230–6239.
  * [20] K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask r-cnn,” 10 2017, pp. 2980–2988.
  * [21] Y. Li, X. Chen, Z. Zheng, L. Xie, G. Huang, D. Du, and X. Wang, “Attention-guided unified network for panoptic segmentation,” 06 2019, pp. 7019–7028.
  * [22] T. Yang, M. D. Collins, Y. Zhu, J. Hwang, T. Liu, X. Zhang, V. Sze, G. Papandreou, and L. Chen, “Deeperlab: Single-shot image parser,” _CoRR_ , vol. abs/1902.05093, 2019. [Online]. Available: http://arxiv.org/abs/1902.05093
  * [23] N. Gao, Y. Shan, Y. Wang, X. Zhao, and K. Huang, “Ssap: Single-shot instance segmentation with affinity pyramid,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. PP, pp. 1–1, 04 2020.
  * [24] Y. Wu, Z. Gengwei, Y. Gao, X. Deng, K. Gong, X. Liang, and L. Lin, “Bidirectional graph reasoning network for panoptic segmentation,” 06 2020, pp. 9077–9086.
  * [25] R. Cheng, R. Razani, Y. Ren, and B. Liu, “S3net: 3d lidar sparse semantic segmentation network,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021\.
  * [26] H. Tang, Z. Liu, S. Zhao, Y. Lin, J. Lin, H. Wang, and S. Han, “Searching efficient 3d architectures with sparse point-voxel convolution,” in _European Conference on Computer Vision_ , 2020.
  * [27] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” in _Advances in neural information processing systems_ , 2017, pp. 5099–5108.
  * [28] Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, N. Trigoni, and A. Markham, “Randla-net: Efficient semantic segmentation of large-scale point clouds,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 108–11 117.
  * [29] M. Gerdzhev, R. Razani, E. Taghavi, and B. Liu, “Tornado-net: multiview total variation semantic segmentation with diamond inception module,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021.
  * [30] J. Behley, A. Milioto, and C. Stachniss, “A benchmark for lidar-based panoptic segmentation based on KITTI,” _CoRR_ , vol. abs/2003.02371, 2020. [Online]. Available: https://arxiv.org/abs/2003.02371
  * [31] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [32] J. V. Hurtado, R. Mohan, W. Burgard, and A. Valada, “Mopt: Multi-object panoptic tracking,” _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Scalability in Autonomous Driving_ , 2020.
  * [33] K. Sirohi, R. Mohan, D. Büscher, W. Burgard, and A. Valada, “Efficientlps: Efficient lidar panoptic segmentation,” 2021.
  * [34] S. Gasperini, M.-A. Mahani, A. Marcos-Ramiro, N. Navab, and F. Tombari, “Panoster: End-to-end panoptic segmentation of lidar point clouds,” _IEEE Robotics and Automation Letters_ , vol. PP, pp. 1–1, 02 2021.
  * [35] R. J. G. B. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” _Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science_ , p. 160–172, 2013\.
  * [36] D. Comaniciu and P. Meer, “Mean shift: a robust approach toward feature space analysis,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 24, no. 5, p. 603–619, 2002.