# DMNR: Unsupervised De-noising of Point Clouds Corrupted by Airborne
Particles

Chu Chen
Department of Mathematics
City University of Hong Kong
Hong Kong
<EMAIL_ADDRESS>
&Yanqi Ma, Bingcheng Dong
School of Mathematical Science
Dalian University of Technology
Dalian
mayq697<EMAIL_ADDRESS>
Junjie Cao
School of Mathematical Science
Dalian University of Technology
Dalian
<EMAIL_ADDRESS>
Corresponding Author.

###### Abstract

LiDAR sensors are critical for autonomous driving and robotics applications
due to their ability to provide accurate range measurements and their
robustness to lighting conditions. However, airborne particles, such as fog,
rain, snow, and dust, will degrade its performance and it is inevitable to
encounter these inclement environmental conditions outdoors. It would be a
straightforward approach to remove them by supervised semantic segmentation.
But annotating these particles point wisely is too laborious. To address this
problem and enhance the perception under inclement conditions, we develop two
dynamic filtering methods called Dynamic Multi-threshold Noise Removal (DMNR)
and DMNR-H by accurate analysis of the position distribution and intensity
characteristics of noisy points and clean points on publicly available WADS
and DENSE datasets. Both DMNR and DMNR-H outperform state-of-the-art
unsupervised methods by a significant margin on the two datasets and are
slightly better than supervised deep learning-based methods. Furthermore, our
methods are more robust to different LiDAR sensors and airborne particles,
such as snow and fog.

_K_ eywords LiDAR $\cdot$ environmental perception $\cdot$ de-noising
algorithm

##  1 Introduction

As a commonly used 3D measurement sensor for machine environmental perception,
LiDAR has been widely used in automatic driving, scene reconstruction, and so
on. By directly obtaining the exact distance and azimuth of the surrounding
objects, LiDAR has the natural advantage of high precision compared to other
sensors. However, when it comes to outdoor application scenarios like
automatic driving, the scanned point clouds are vulnerable to airborne
particles such as snowflakes (Fig. 1(a)), raindrops, fog (Fig. 1(e)), dust,
and so on. A large number of noise points will be generated which have serious
adverse effects on downstream tasks [1].

To remove the corruption of airborne particles, some efforts have been made.
They are mainly divided into conventional filter-based methods and deep
learning-based approaches. Deep learning perception methods generally tend to
achieve better quantitative results [2, 3]. However, it is not a proper option
to deploy these pre-processing networks together with their downstream
applications on one edge device with limited computational resources.

Besides, the dependency on point-wise annotations also limits their
application. Although there are some efficient 3D point annotation methods,
manual annotation of 3D airborne particles is still laborious and time-
consuming. Because it is hard to take advantage of the assistance of camera
view when labeling these particles, since they may be not visible in camera
images or the images are also inconsistently degraded [4, 5, 6]. In contrast,
the conventional filter-based methods not only have the advantage in this
aspect, but they also have no training cost and significantly fewer
computational resources.

The conventional filter-based methods [7, 8, 9, 10] can filter out noise
points effectively. Meanwhile, a large number of environmental details are
misjudged, as shown in Fig. 1(b)-1(c) and Fig. 1(f)-1(g). More environment
points can also be preserved by lowering the aggressiveness of the methods but
with the cost of unsatisfying filtering results. Since neither of the two
situations are acceptable to downstream algorithms, the ability to make a
balance is rather crucial for classical filters. What’s more, previous methods
fail to maintain the de-noising effect when dealing with different kinds of
airborne particles.

The paper addresses these issues by proposing an unsupervised airborne
particle removal strategy for LiDAR in adverse weather. By staging multiple
filtering processes with adaptive spatial and intensity thresholds, our
strategy shows its effective de-noising performance and robustness when
dealing with various airborne particles and LiDAR sensors (32 or 64 channels).
Following the strategy, we introduce two algorithms called Dynamic Multi-
threshold Noise Removal (DMNR), which is a standard multi-stage filtering
method with multiple dynamic thresholds, and DMNR-H, which includes an extra
post-processing step by HDBSCAN. The contributions of this paper can be
described as follows:

(a) Snowy Condition

(b) DSOR

(c) DDIOR

(d) DMNR

(e) Foggy Condition

(f) DSOR

(g) DDIOR

(h) DMNR

Figure 1: The proposed DMNR method outperforms previous unsupervised methods
for snow and dog removal. The left column shows a snowy point cloud and a
foggy point cloud. The rest columns are results of DSOR [8], DDIOR [10], and
our DMNR. Noise points are rendered in red and non-noise points are in blue.

  * •

Two unsupervised LiDAR point cloud de-noising methods are proposed. They
follow a multi-stage filtering strategy with multiple dynamic thresholds
determined by a thorough analysis of the position distribution and intensity
characteristics of the scanned points on the WADS and DENSE datasets.

  * •

Compared with previous unsupervised methods, our methods are more robust to
different kinds of airborne particles and LiDAR devices.

  * •

Quantitative and qualitative evaluations show that our methods outperform
previous methods by a large margin for airborne particle removal.

##  2 Related Works

LiDAR sensors are vulnerable to airborne particles caused by adverse weather
conditions such as rain, snow, and fog. In these conditions, airborne
particles may result in intensity attenuation and spurious coordinates of
scanned points. Therefore, noise filters are crucial to the quality of the
point cloud data. Some denoising algorithms dealing with these noises have
been developed in recent years. We would like to focus on unsupervised
methods.

The Point Cloud Library (PCL) [11] contains several filters such as
Statistical Outlier Removal (SOR) and Radius Outlier Removal (ROR). But none
of these filters are specifically designed for noisy point clouds corrupted by
airborne particles, and their performance is far from ideal. To deal with
point clouds affected by snowflakes, Charron et al. [7] proposed Dynamic
Radius Outlier Removal (DROR) by improving ROR to a dynamic search radius in
order to preserve more environmental details far from LiDAR sensor. Kurup et
al. [8] also proposed a de-snowing algorithm called Dynamic Statistical
Outlier Removal (DSOR). DSOR can remove more snow than DROR, but it also
removes more environmental features.

Park et al. [9] noticed that the intensity of snow points is generally lower
than that of non-snow points. Based on this observation, they proposed Low-
intensity Outlier Removal (LIOR). However, under heavy snow conditions, when
the intensity of snow points is close to that of other points, the performance
of LIOR will be reduced significantly. The latest de-snowing algorithm is the
Dynamic Distance-intensity Outlier Removal (DDIOR) proposed by Wang et al.
[10]. DDIOR has a pre-processing step to retain points with high intensity and
long distances and adds intensity information into the dynamic threshold.
Compared with DSOR, DDIOR slightly improves the precision of snow removal, but
it still cannot fully retain environmental details. It still needs to modify
parameters to maintain performance when handling different scenes.

There is no unsupervised algorithm that can effectively remove the noise
caused by multiple kinds of airborne particles at present. In this work, we
compare our DMNR and DMNR-H filters with DSOR and DDIOR on de-snowing based on
Winter Adverse Driving dataSet (WADS [8]). In addition, we also show the
superiority of our methods for different airborne particles by applying snow
simulation [12] and fog simulation [13] on DENSE dataset [14]. Note that the
LiDAR devices used in the two datasets are also different.

##  3 Methods

###  3.1 DMNR

Wang et al.[10] analyzed the fundamental characteristics of snow noise points
including low density, low intensity, close range, and fast decay. Hahner et
al. [12] also reached a similar conclusion about fog noise from an optical
perspective. These characteristics of these noise points are very important to
the study of de-noising algorithms. But they all ignored the height
characteristics of the noisy points. Fig. 2 shows the height of the points in
different ranges. Clean points collected at higher altitudes tend to have
lower densities. It can be seen from Fig. 3 that previous de-noising
algorithms failed to distinguish clean points at high places from noise
points, resulting in misjudgments and loss of environmental details.

Figure 2: Relationship between distance from sensor and height ($z$ value) of
scanned points. Noise points are rendered in red, and non-noise points in
blue.

The first stage of DMNR is to distinguish clean environmental points from
noise points by a height threshold. For each point
$p_{n}=(x_{n},y_{n},z_{n})$, we have:

$H_{n}=\frac{h_{1}}{d_{n}}+h_{2},$ (1)

where $d_{n}=\sqrt{x_{n}^{2}+y_{n}^{2}+z_{n}^{2}}$ is the distance from the
point to the sensor. When dealing with one scan of point cloud, $h_{1}$ and
$h_{2}$ are constants, which can be determined by
$h_{1}=\mathop{max}\limits_{i=1,...,N}\left\\{d_{i}/2\right\\}$ and
$h_{2}=\mathop{min}\limits_{i=1,...,N}\left\\{z_{i}-1\right\\}$ respectively.
Points with higher height ($z_{n}>H_{n}$) will be retained directly, while
those with lower height ($z_{n}\leq H_{n}$) will go to the next stage of the
algorithm.

(a) Raw Point Cloud

(b) DSOR

(c) DDIOR

(d) DMNR

Figure 3: Horizontal view of a raw point cloud (a) and the results of DSOR
(b), DDIOR (c), and DMNR (d). The point cloud are rendered according to their
labels and noise points are rendered in yellow.

The second stage of DMNR is to filter out noise points by a point-wise dynamic
threshold about density, intensity, and range:

$T_{n}=\mu\times(k_{1}\times{\rm e}^{k_{2}\times d_{n}}+k_{3}\times i)\times
d_{n},$ (2)

where $k_{1}$ is an indirect aggressive parameter, $k_{2}$ is an aggressive
range parameter, $i$ is the intensity of the point, $k_{3}$ is the weight of
intensity, and $\mu$ is the global average density. $\mu$ measures the overall
density of the point cloud. It can be obtained by averaging the local average
density $ad_{n}$:

$\mu=\frac{\sum_{n=1}^{N}ad_{n}}{N},$ (3)

where $N$ is the number of points. $ad_{n}$ is computed as the mean distance
of point $p_{n}$ and its K nearest neighbors.

We can control the aggressive level of DMNR through $k_{1}$ and $k_{2}$, which
can be determined according to weather conditions and the range of noise
points respectively. Algorithm 1 shows the pseudo-code of DMNR. DMNR divides
the raw point cloud $P$ into two sets: the filtered point cloud $F$ and the
outliers $O$.

Algorithm 1 Dynamic Multi-threshold Outlier Removal

1:Point Cloud $P=p_{n},p_{n}=(x_{n},y_{n},z_{n},i_{n}),n=1,...,N$
$K=$number of nearest neighbors
$k_{1}=$aggressive parameter
$k_{2}=$aggressive range parameter
$k_{3}=$weight of intensity

2:Filtered Point Cloud $F=p_{n_{j}},p_{n_{j}}\in P,j=1,...,N^{\prime}$
Outliers $O=P\backslash F$

3:$P\leftarrow KdTree$;

4:for $p_{n}\in P$ do

5: $average\\_distances$ $ad_{n}\leftarrow nearestKsearch(k)$;

6:end for
calculate:global average distance $\mu\leftarrow ad_{n}$

7:for $p_{n}\in P$ do

8: $distanced_{n}\leftarrow\sqrt{x_{n}^{2}+y_{n}^{2}+z_{n}^{2}}$

9: $H_{n}\leftarrow 100/d_{n}-5$

10: if $z_{n}>H_{n}$ then

11: $F\leftarrow p_{n}$

12: else

13: $T_{n}\leftarrow\mu\times(k_{1}\times{\rm e}^{k_{2}\times
d_{n}}+k_{3}\times i_{n})\times d_{n}$

14: if $ad_{n}<T_{n}$ then

15: $F\leftarrow p_{n}$

16: else

17: $O\leftarrow p_{n}$

18: end if

19: end if

20:end for

21:return $F,O$

###  3.2 DMNR-H

Hierarchical Density-Based Spatial Clustering of Applications with Noise
(HDBSCAN) [15] is a clustering algorithm, which can process data with uneven
density and large differences in cluster spacing, which are the significant
characteristics of point cloud data. We use it as a post-processing method to
restore some misclassified clean points from the noise points $O$ detected by
DMNR. The point cloud is clustered into multiple classes by HDBSCAN
adaptively. DMNR-H will select $h$ classes containing most of the filtered
point cloud $F$, and the rest points of the same $h$ classes will be
transferred from $O$ to $F$ as clean points. Complete process of DMNR-H can be
referred to Fig. 4. Consequently the results of DMNR are effectively improved
and the environmental structure can be better preserved.

Figure 4: Process of DMNR-H algorithm.

##  4 Experiments

(a) Raw Point Cloud

(b) DSOR

(c) DDIOR

(d) DMNR

Figure 5: Qualitative comparison of snow removal between DSOR, DDIOR, and our
DMNR. (a) Raw Point Cloud; (b) DSOR; (c) DDIOR; (d) DMNR. The left-hand image
of each sub-image is the overall visualization of de-noising effect while the
right-hand image is a zoomed-in version of that on the left. Colors of the
point cloud are rendered according to labels, and noise points are those in
yellow.

We compare our methods with DSOR [8] and DDIOR [10] on WADS [8] and DENSE [14]
for both de-snowing and de-fogging. WADS are scanned by 64-channel LiDAR
sensors under snowy weather and the snow particles are annotated manually.
DENSE are scanned by 32-channel LiDAR sensors under clear weather. Snow
simulation [12] with snowfall rate $r_{s}=2mm/h$ and fog simulation [13] with
spatially varying attenuation coefficient $\alpha=0.015$ are employed to
synthesis snow and fog data over 1,800 frames point clouds in DENSE. Since not
all sequences are available in WADS (some sequences are labeled incorrectly or
have too few frames), we use sequences 13, 23-24, 26, 28, 30, and 34-36 for
experiments and visualization. All the experiments are conducted on a laptop
with an Intel ® CoreTM i5-8265U CPU and 8GB RAM. We use the default parameters
of DSOR [8] and DDIOR [10]. Table 1 shows the parameters of DMNR and DMNR-H in
all the experiments.

Table 1: Parameters of DMNR and DMNR-H. $K$ |  $k_{1}$ |  $k_{2}$ |  $k_{3}$ |  $h$
---|---|---|---|---
10 |  0.015 |  0.055 |  100 |  5

(a) Raw Point Cloud

(b) DSOR

(c) DDIOR

(d) DMNR

(e) DMNR-H

Figure 6: Visualization of de-noising effect (left) and noise points judged by
the corresponding filter (right). (a) Raw Point Cloud and Ground Truth Noise;
(b) DSOR; (c) DDIOR; (d) DMNR; (d) DMNR-H. Colors of the point cloud are
rendered according to labels, and noise points are those in yellow.

###  4.1 Qualitative Evaluation

Fig.3 shows the effect of combining height threshold at pre-processing stage,
where DMOR performs the best on preserving environmental details with high
altitude. The overall de-noising effect is shown in Fig.5 and Fig.6 which
visualized the point cloud that retained and removed respectively. By
comparing with the raw point cloud image (Fig.5(a)) and referring to the color
of the point cloud rendered by labels, DSOR removed large part of the
environmental details (Fig.6(b)) while retained lots of noise (Fig.5(b)).
Although DDIOR removed noise thoroughly (Fig.5(c)), even more environmental
features were removed (Fig.6(c)) than DSOR were. Our DMOR filter can remove
noise completely (Fig.5(d)) while nearly no environmental information was
misjudged (Fig.6(d)).

Besides Fig. 3, the overall de-noising effects are also shown in Fig. 5 and
Fig. 6. By comparing with the raw point cloud (Fig. 5(a)), DSOR removes a
large part of the environmental details (Fig. 6(b)) while retaining lots of
noise points (Fig. 5(b)). Although DDIOR removes noise more thoroughly (Fig.
5(c)), more environmental points are also removed (Fig. 6(c)) than DSOR. Our
DMNR and DMNR-H filter can remove noise more completely (Fig. 5(d) and Fig.
6(e)) while nearly no environmental information is misjudged (Fig. 6(d) and
Fig. 6(e)).

Table 2: Quantitative Comparison on de-snowing

Filters |  | WADS[8]* |  |  | DENSE [14]** |
---|---|---|---|---|---|---
$precision$ | $recall$ | $F1$ | $precision$ | $recall$ | $F1$
DSOR[8] | 65.07 | 95.60 | 77.43 | 5.60 | 98.37 | 10.60
DDIOR[10] | 69.87 | 95.23 | 80.60 | 18.38 | 91.47 | 30.60
DMNR | 91.82 | 90.69 | 91.25 | 80.25 | 61.61 | 69.71
DMNR-H | 92.86 | 89.88 | 91.35 | 83.52 | 60.03 | 69.86
WeatherNet$\dagger$ [2] | 94.88 | 85.13 | 89.74 | - | - | -

  * •

* 909 frames of outdoor collected snow. ** 1,816 frames of simulated snow [12].

  * •

$\dagger$ Deep learning based method.

###  4.2 Quantitative Evaluation

Generally, $precision$ and $recall$ are used for quantitative assessment of
de-noising performance:

$precision=\frac{TP}{TP+FP},recall=\frac{TP}{TP+FN}.$ (4)

$TP$ (True Positive) refers to noise points filtered out from the point cloud.
$FP$ (False Positive) refers to non-noise points filtered out from the point
cloud. $FN$ (False Negative) refers to noise points retained after filtering.
As its definition, $precision$ evaluates the ability of a filter to maintain
environmental details while $recall$ evaluates the ability to remove noise
points. An aggressive filter increases $recall$ but decreases $precision$
while it would be an opposite effect for a mild filter, in which case we will
not be able to make a comparison among filters. Thus, a comprehensive value
for assessment is necessary. We use $F1-score$, denoted by $F1$, to
comprehensively evaluate the de-noising ability of filters:

$F1=\frac{2}{\frac{1}{recall}+\frac{1}{precision}}.$ (5)

Table 2 shows the quantitative comparison among DSOR, DDIOR, DMNR, DMNR-H and
WeatherNet [2] on WADS [8] and DENSE [14] datasets. The results on the WADS
dataset illustrate that DSOR and DDIOR filters have higher $recall$ but lower
$precision$. While our methods have the highest $F1$, $91.25$ and $91.35$,
which are higher than the deep learning based supervised method WeatherNet
(89.74), and outperform DSOR and DDIOR by a large margin (77.43 and 80.60).

Table 3: Quantitative Comparison on de-fogging

Filters |  |  fog* |
---|---|---|---
$precision$ |  $recall$ |  $F1$
DSOR[8] |  23.74 |  99.99 |  38.37
DDIOR[10] |  26.47 |  99.88 |  41.85
DMNR |  82.36 |  83.26 |  81.73
DMNR-H |  83.49 |  82.36 |  81.77

  * •

* 1,816 frames of simulated fog [13] in DENSE [14].

The results of DSOR and DDIOR filters on the DENSE dataset show that they
perform poorly for different kinds of snow and dog. The point clouds of DENSE
dataset are scanned with 32-channel LiDARs, and have less number of points,
smaller scanning range, and higher overall intensity than that of WADS
dataset. Thus they own sparser density and a huge variety of intensity when
corrupted by airborne particles. However, our methods still maintain higher
$F1$ for the two cases. What’s more, with the help of HDBSCAN, DMNR-H achieves
even better $F1$ than DMNR for all the cases.

##  5 Conclusions

We propose an unsupervised de-noising algorithm DMNR for airborne particle
removal in LiDAR data. It includes an environmental structure preservation
stage and a dynamic filtering stage. The first stage effectively protects
sparse environment points at high places. The second stage identifies noise
points by an adaptive threshold considering density, intensity, and range
together. Besides, DMNR-H is proposed by appending a HDBSCAN clustering
algorithm to further improve DMNR’s ability to preserve environmental
information.

Thanks to the excellent ability of both environmental structure preserving and
noise filtering, our methods far outperform the state-of-the-art unsupervised
methods in terms of comprehensive de-noising ability when processing point
clouds corrupted by various of airborne particles obtained by different LiDAR
devices.

## References

  * [1] Jeremy P Bos, Derek Chopp, Akhil Kurup, and Nathan Spike.  Autonomy at the end of the earth: an inclement weather autonomous driving data set.  In Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2020, volume 11415. SPIE, 2020.
  * [2] Robin Heinzler, Florian Piewak, Philipp Schindler, and Wilhelm Stork.  CNN-based lidar point cloud de-noising in adverse weather.  IEEE Robotics and Automation Letters, 5(2):2514–2521, 2020.
  * [3] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4510–4520, 2018.
  * [4] Shree K. Nayar and Srinivasa G. Narasimhan.  Vision in bad weather.  Proceedings of the Seventh IEEE International Conference on Computer Vision, 2:820–827 vol.2, 1999.
  * [5] Md Nasim Khan and Mohamed M. Ahmed.  Snow detection using in-vehicle video camera with texture-based image features utilizing k-nearest neighbor, support vector machine, and random forest.  Transportation Research Record, 2673:221 – 232, 2019.
  * [6] Jun Mao, Uthai Phommasak, Shinya Watanabe, and Hiroyuki Shioya.  Detecting foggy images and estimating the haze degree factor.  Journal of Computer Science & Systems Biology, 7:226–228, 2014\.
  * [7] Nicholas Charron, Stephen Phillips, and Steven L. Waslander.  De-noising of lidar point clouds corrupted by snowfall.  2018 15th Conference on Computer and Robot Vision (CRV), pages 254–261, 2018.
  * [8] Akhil M. Kurup and Jeremy P. Bos.  Dsor: A scalable statistical filter for removing falling snow from lidar point clouds in severe winter weather.  ArXiv, abs/2109.07078, 2021.
  * [9] Ji il Park, Jihyuk Park, and Kyung-Soo Kim.  Fast and accurate desnowing algorithm for lidar point clouds.  IEEE Access, 8:160202–160212, 2020.
  * [10] Weiqi Wang, Xiong You, Lingyu Chen, Jiangpeng Tian, Fengying Tang, and Lantian Zhang.  A scalable and accurate de-snowing algorithm for lidar point clouds in winter.  Remote. Sens., 14:1468, 2022.
  * [11] Radu Bogdan Rusu and Steve B. Cousins.  3d is here: Point cloud library (pcl).  2011 IEEE International Conference on Robotics and Automation, pages 1–4, 2011.
  * [12] Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc Van Gool.  Lidar snowfall simulation for robust 3d object detection.  ArXiv, abs/2203.15118, 2022.
  * [13] Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Fog simulation on real lidar point clouds for 3d object detection in adverse weather.  2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 15263–15272, 2021.
  * [14] Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide.  Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [15] Ricardo J. G. B. Campello, Davoud Moulavi, and Jörg Sander.  Density-based clustering based on hierarchical density estimates.  In PAKDD, 2013.