# Variable Rate Compression for Raw 3D Point Clouds

Md Ahmed Al Muzaddid and William J. Beksi The authors are with the Department
of Computer Science and Engineering, University of Texas at Arlington,
Arlington, TX, USA. Emails<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we propose a novel variable rate deep compression architecture
that operates on raw 3D point cloud data. The majority of learning-based point
cloud compression methods work on a downsampled representation of the data.
Moreover, many existing techniques require training multiple networks for
different compression rates to generate consolidated point clouds of varying
quality. In contrast, our network is capable of explicitly processing point
clouds and generating a compressed description at a comprehensive range of
bitrates. Furthermore, our approach ensures that there is no loss of
information as a result of the voxelization process and the density of the
point cloud does not affect the encoder/decoder performance. An extensive
experimental evaluation shows that our model obtains state-of-the-art results,
it is computationally efficient, and it can work directly with point cloud
data thus avoiding an expensive voxelized representation.

###### Index Terms:

RGB-D Perception; Deep Learning for Visual Perception; Big Data in Robotics
and Automation

##  I Introduction

Among various data modalities such as audio, image, text, etc., 3D data in the
form of point clouds constitutes a growing portion of the spectrum. Sensors
for capturing 3D point cloud data such as light detection and ranging (LiDAR),
stereo, structured light, and time-of-flight (ToF) have become increasingly
popular and economical. In comparison to images, high-dimensional information
can be described in 3D with immunity to variations in color, illumination, and
scale. Besides point clouds, there are numerous ways to depict 3D data
including meshes, CAD models, and volumetric representations. However, when
compared to the other data representations, point clouds offer the advantage
of providing a simpler, denser, and closer-to-reality description [1].

A point cloud consists of an unordered set of points where each point has a
Cartesian coordinate value in Euclidean space. Additional attributes may
include the color (i.e., RGB value) and surface normal associated with each
point. Robotics is at the forefront of utilizing point clouds created by 3D
sensor technology. Portrayal of the environment provided by these sensors can
facilitate 3D learning-based tasks like object/scene reconstruction, motion
planning and navigation, object detection and classification, pose estimation,
grasp detection, etc. [2]. Other pertinent applications where point clouds are
widely used include augmented and virtual reality, cultural heritage,
geographical information systems, self-driving vehicles, and many more [3].

3D sensors generate enormous amounts of point cloud data at high frame rates.
For example, a 3D point cloud with 0.7 million points per frame at 30 frames
per second needs a bandwidth of approximately 500 MB/s for video [4]. Working
with uncompressed point cloud data can lead to congestion and delays in
communication networks [5]. Thus, efficient compression coding technologies
are indispensable for ensuring the compact storage and transmission of such
data. There exists an array point cloud compression techniques in the
literature which are primarily based on the use of octrees and 2D-projection.
Nevertheless, a number of deep learning-based point cloud compression methods
have emerged which offer performance improvements over traditional methods.

Figure 1: 3D point clouds produced by LiDAR, stereo, structured light, and ToF
sensors require a large amount of memory capacity. The transfer of point
clouds to and from machines necessitates compression of the data. Our approach
can reconstruct dense point clouds from a highly-compressed representation, at
variable bitrates, using a single trained model.

Even though most machine learning point cloud compression research has been
inspired by the success of learning-based image compression [6], there are
fundamental differences between these two domains. Unlike a traditional 2D
image, the structure of a 3D point cloud is irregular. Dealing with unordered
3D points is a significant dilemma. For instance, proven techniques such as
convolutional layers that extract features in the image domain can not be
plainly applied to raw point clouds. These 2D and 3D convolution operators are
suitable for grid-like data structures. Therefore, point clouds are typically
voxelized whereby points are represented as a binary occupancy cell. Although
this provides a representation suitable for use with a convolutional layer, it
also introduces a number of difficulties (e.g., computing and memory overhead,
loss of information, etc.).

To address these limitations, we propose a model that can compress raw point
clouds without converting them into a voxelized representation, Fig. 1. There
are two prominent benefits of our approach: (i) there is no loss of
information due to voxelization; (ii) the compression and reconstruction
performance is not affected by the underlying point cloud density. At the same
time, our technique reduces the unnecessary computational burden induced by
the processing of unoccupied voxels. In summary, our contributions are the
following.

  * •

We introduce a weighted entropy loss function and inference strategy to
compress point clouds at different bitrates using a single trained model.

  * •

Our architecture eliminates the need to train multiple models for a range of
bitrates thus benefiting a diversity of applications.

  * •

We establish raw 3D point cloud compression benchmarks for a variety of tasks
on publicly available datasets.

Our source code is available at [7].

The remainder of this paper is organized as follows. Relevant related
literature is discussed in Section II. Our approach for variable rate deep 3D
point cloud compression is presented in Section III. The design and results of
our experiments are demonstrated and explained in Section IV. In Section V,
the paper is concluded and future work is discussed.

##  II Related Work

###  II-A Tree Structure-Based Compression

Octrees provide an efficient way of representing point clouds through the
partitioning of space [8]. An octree-based representation builds a tree data
structure that starts from the root node, where a node represents a 3D
bounding box, and recursively subdivides the space into eight child nodes
until a desired depth is reached. Conventional octree-based coding methods [9,
10, 11, 12, 13, 14] directly exploit the spatial correlations among points via
this data structure.

Other types of tree-based representations, such as k-d trees trees, [15] and
spanning trees [16], can achieve analogous results for lossless coding [17,
18, 19]. The Moving Picture Experts Group (MPEG) has published a geometry-
based point cloud compression standard (MPEG V-PCC) based on an octree data
structure [20, 21]. In addition, the Point Cloud Library [22] provides octree-
based implementations for point cloud compression.

###  II-B Projection-Based Compression

Projection-based approaches follow the principle of converting a 3D point
cloud into 2D images by projection or mapping [23, 24, 25, 26, 27, 28].
Usually, the output of the projection step is represented as a set of depth
images based on multiple dissimilar viewpoints or canonical planes. This
provides a regular grid structure of the points, which are further partitioned
into a set of patches or point clusters. Subsequent steps compress the patches
using well-known image/video codecs such as JPEG [29] and HEVC [30]. The main
challenge with this technique is to find an optimum set of transformations
from the 3D point set to the corresponding 2D representation. A combination of
octree and projection-based methods [31] is also possible as well.

###  II-C Learning-Based Compression

Learning-based methods typically exploit the same transform-based coding
principles used in conventional lossy image coding such as JPEG. Most of the
learning-based models search for an optimum nonlinear transformation in terms
of rate distortion trade-off during training time by using an appropriate
differentiable objective function. For instance, the estimated entropy of a
quantized latent vector can be used along with the reconstruction loss in an
objective function [32, 33]. Within the image domain, convolutional neural
networks (CNNs) are used to extract the translation-invariant features that
encapsulate spatial correlations among neighboring pixels. Inspired by the
success of CNN layers [34], complementary approaches have been adopted in
learning-based point cloud compression [35, 36, 37, 38, 39].

After converting a raw point cloud into a regular voxel grid, autoencoder-
based machine learning models may be used to extract a latent representation
which is then quantized. The quantized latent representation is further
processed by a decoder to reconstruct the point cloud. The whole model can
then be trained end-to-end to optimize both reconstruction loss and entropy
loss. Due to the non-differentiable nature of the quantization step, special
treatment is needed inside a conventional autoencoder structure. For example,
[40] carried out nearest neighbor assignments to compute the quantized value
of the latent vector which relied on (differentiable) soft quantization.
Motivated by theoretical links to dithering, [41] proposed to replace
quantization by additive uniform noise. [42] used a stochastic rounding
operation in the forward pass, while in the backward pass the derivative is
replaced with the derivative of the expectation.

##  III Variable Rate Deep Compression

Figure 2: An overview of our variable rate deep compression model for
compressing raw 3D point clouds. The input to the model is an $n\times f$
matrix, where $n$ is the number of sample points and $f$ is the number of
features per point. The top and bottom branches of the encoder correspond to
the local and global feature extractors, respectively. At the first level in
the local feature extractor, the set abstraction (SA) layer transforms the per
point input features into combined features of size $n_{1}\times f_{1}$. These
combined features are then processed by the next level SA layer. All of the
features are combined into a single feature vector of size $f_{3}$ at the last
level SA layer. The bottom branch converts per point input features of size
$n\times f$ into transformed features of size $n\times f^{\prime}$. Next, a
pooling operation combines the features into a single feature vector of size
$f^{\prime}$. Lastly, the two feature vectors of size $f_{3}$ and $f^{\prime}$
are concatenated and processed by the compressor. The decoder first
decompresses the compressed vector into $\hat{y}$ which is then processed by
both of the decoding branches (top and bottom) to produce approximately $n$
points in total.

Our unified architecture directly takes raw point clouds as input, Fig. 2. A
PointNet-based [43] feature extractor is utilized for processing point cloud
data as follows. The encoder transforms a point cloud $x$ using a parametric
analysis transform, $g_{a}(x;\theta_{a})$, into a latent representation $y$.
The latent representation is then quantized to produce $\hat{y}$. On the
receiving side, the decoder recovers $\hat{y}$ from the compressed signal. A
synthesis transformer function, $g_{s}(\hat{y};\theta_{s})$, uses $\hat{y}$ to
recover the reconstructed point cloud $\tilde{x}$. $\theta_{a}$ and
$\theta_{s}$ are the weight parameters of the encoder and decoder,
respectively.

Our goal is to derive uncorrelated global and local features from the point
cloud input. To do this, we propose a deep learning-based approach to extract
structural features from 3D data. Transforms (e.g., discrete cosine transform
(DCT) [44]) have been used for decades to represent 2D image and video data in
a more condensed format. Although we employ a feature extractor that encodes
low-level features into compact vector representations, floating-point vectors
are not suited for sending and receiving over a transmission channel in an
efficient way. This is due to the fact that precise floating-point data
requires a high bitrate to transmit.

To reduce the bitrate, we further quantize the embedded vector using an
adaptive quantizer coupled with a probability density model. The probability
density model is applied to estimate the bitrate required to compress the
quantized vector representation. A fully-connected decoder is used as a
synthesis transformer to decode the quantized latent features into the
reconstructed point cloud. The reconstructed point cloud is then compared
against the input point cloud and the reconstruction loss is calculated. The
whole model is trained end-to-end to simultaneously optimize reconstruction
loss and bitrate. In the following subsections, we describe each component of
the model.

###  III-A 3D Transform

Within the image domain, transformations are used to express data in a
consolidated form. Learned 2D transforms have demonstrated improved coding
performance in image compression [45, 34, 46]. A learned transform usually
performs better than a traditional transform such as the DCT. This is due to
the ability of the learned transform to allocate more or less bits for
contrasting parts of an image based on the rate distortion constraints it is
trained on. For point cloud data, a transform is typically learned via a
convolutional autoencoder which takes a voxelized occupancy grid as input.

Instead of using a convolution-based feature extractor, we opt for a fully-
connected layer-based feature extractor. Concretely, we use two parallel
branches in our encoder module. One branch handles the global structure of the
point cloud and the other branch focuses on local patterns along the disparate
levels of granularity. In the global feature extractor branch, we apply a
multilayer perceptron network to every point. The network weights are shared
among all the points. A global feature vector is then derived by applying a
symmetric function on the transformed features.

In the local feature extractor branch, features are aggregated in a
hierarchical fashion to capture the geometric structure of the neighboring
point cloud distribution. The points are sampled based on an iterative
farthest point strategy [47] and used as keypoints to extract nearby local
structure. Similar to a CNN, sampled features only focus on a small
neighborhood within the initial layers. At the higher layers, features are
captured at larger scales. This is comparable to the increasing receptive
field size within the deeper layers of a CNN. Both feature vectors from the
local and global branch are then concatenated and passed through fully-
connected layers which combine them into a single latent vector $y$.

###  III-B Quantization

After obtaining the latent representation, we perform a quantization step at
compression time based on the desired target quality or rate. During training,
to ensure differentiability we model the discretization using additive uniform
noise [45]. More formally, we define

$q(\tilde{y}\,|\,x,\theta_{s})=\prod_{i}U\left(\tilde{y}_{i}\,|\,y_{i}-\frac{1}{2},y_{i}+\frac{1}{2}\right),$
(1)

where $\tilde{y}$ is the noisy approximation, $U$ denotes a uniform
distribution centered on $y_{i}$, and $i\in[1,\ldots,l]$ where $l$ is the
length of $y$. During compression, the elements of $y$ are quantized by
rounding to the nearest integer.

###  III-C Rate-Distortion Modeling

Following quantization, the latent vector becomes discrete-valued.
Consequently, it can be compressed losslessly using entropy coding techniques
such as arithmetic coding [48]. The compressed code can then be transmitted as
a sequence of bits. On the receiving end, the decoder recovers the 3D
coordinates from the compressed latent vector to reconstruct the original
point cloud. Given an approximate distribution $P_{\hat{y}}$ of the quantized
vector $\hat{y}$, we calculate the bitrate as

$R=E_{x}[-\log P_{\hat{y}}(Q(g_{a}(x;\theta_{a})))],$ (2)

where $Q$ represents the quantization function. The distortion is the expected
difference between the reconstructed point cloud $\tilde{x}$ and the inceptive
point cloud $x$ as measured by a suitable distance metric. To control the rate
distortion trade-off we use a traditional formulation in which an equilibrium
is found between the code length and the distortion. Formally, we minimize the
combined loss, $L=D+\lambda R$, where $D$ is the distortion and $\lambda$ acts
as a hyperparameter that balances the two competing objectives.

###  III-D Decoder

Given a vector decompressed by the decompressor, the decoder module regresses
the 3D coordinate values of the point cloud. To support variable bitrates, we
use a weighted entropy loss function. Through the use of variable weights for
individual vector elements, we gradually reduce the importance of the elements
occurring at positions later on in the compressed vector. More precisely,

$R_{weighted}=E_{x}\left[\sum_{i}-\omega_{i}\log P(\hat{y}_{i})\right],$ (3)

where

$\omega_{i}=ae^{-b\times i}$ (4)

is an exponential decaying function for assigning the $i$-th weight to each
element in the compressed vector.

With this weighted entropy loss the overall loss function becomes $L=D+\lambda
R_{weighted}$. During the reconstruction phase, elements appearing towards the
end of the vector contribute less to the overall reconstruction. As a result,
we can ignore the latter part of the vector if we have a tight budget on the
bitrate. This allows us to train a single model that produces variable
bitrates. We show the relation between the effective bitrate versus the
reconstruction loss through a broad experimental assessment (Section IV).

##  IV Experimental Evaluation

###  IV-A Datasets, Training, and Runtime Details

We make use of the ShapeNet [49] and ModelNet40 [50] datasets for our
experimental evaluation. ShapeNet is composed of 16,881 objects from 16
classes and 50 parts in total. In addition, the category label for each object
is provided. Each point cloud contains 2,048 points uniformly sampled from the
surface of an object. The sampled point clouds are normalized with zero mean
and scaled to a unit sphere. ModelNet40 consists of 12,311 CAD-generated
objects (e.g., airplane, car, plant, lamp, etc.) from 40 categories. Within
ModelNet40, 9,843 objects are used for training while the rest (2,468) are
reserved for testing.

Our model was trained in batches where each sample consists of $n=2048$
points. Only the $x,y,z$ coordinates are considered as input features (i.e.,
$f=3$) akin to [43]. For the local feature extractor branch, three SA layers
were used. The number of sampling points in each SA layer are $n_{1}=512$,
$n_{2}=128$, $n_{3}=1$, and the number of features are $f_{1}=256$,
$f_{2}=512$, $f_{3}=512$. In the global feature extractor branch the number of
features is $f^{\prime}=512$. On the decoder side, we used a small fully-
connected neural network of size $512\times 512\times 1024$. Max pooling was
utilized for all pooling operations.

Using the Adam [51] optimizer, our network was trained for 300 iterations with
an initial learning rate 0.0001, a batch size of 16, momentum value of 0.9,
and momentum2 value of 0.999. Training took place on a CentOS 7.6.1810 machine
using an Intel Xeon E5-2620 2.10 GHz CPU, 132 GB of memory, and an NVIDIA
GeForce GTX 1080 Ti GPU. It took approximately 23 hours to train our model.
All experimental tests were performed using a Ubuntu 18.04 machine with an
Intel Core i7-8700 3.20 GHz CPU, 32 GB of memory, and an NVIDIA Quadro P4000
GPU. Table I displays the elapsed wall-clock times, per batch size, for 3D
point cloud compression and decompression. Note that due to the parallel
processing capability of the GPU, compression and decompression in larger
batch sizes takes less time per point cloud.

Task |  Time (msec)
---|---
|  Batch size of 32 |  Batch size of 16 |  Batch size of
1
Compression |  $8.50\pm 0.10$ |  $8.60\pm 0.10$ |  $15.20\pm 0.40$
Decompression |  $0.40\pm 0.01$ |  $0.50\pm 0.02$ |  $2.30\pm 0.10$
TABLE I: Elapsed wall-clock times for 3D point cloud compression and decompression tasks averaged over ten trial runs. Category | CD ($\times 10^{-2}$)  $\downarrow$ | EMD ($\times 10^{-2}$)  $\downarrow$ |  F-score $\uparrow$
---|---|---|---
1.69 bpp |  0.35 bpp |  0.03 bpp |  1.69 bpp |  0.35 bpp |  0.03 bpp |  1.69 bpp |  0.35 bpp |  0.03 bpp
Airplane |  0.063 |  0.068 |  0.074 |  2.041 |  2.361 |  2.170 |  0.995 |  0.995 |  0.992
Bag |  0.343 |  0.362 |  0.392 |  2.733 |  3.346 |  2.439 |  0.788 |  0.764 |  0.744
Cap |  0.550 |  0.600 |  0.595 |  2.386 |  2.298 |  3.151 |  0.652 |  0.591 |  0.623
Car |  0.230 |  0.255 |  0.257 |  3.032 |  2.394 |  2.837 |  0.891 |  0.884 |  0.870
Chair |  0.202 |  0.211 |  0.273 |  2.575 |  2.673 |  3.016 |  0.919 |  0.913 |  0.879
Earphone |  0.463 |  0.457 |  0.656 |  3.237 |  2.192 |  2.822 |  0.762 |  0.784 |  0.735
Guitar |  0.055 |  0.056 |  0.067 |  1.846 |  1.815 |  2.406 |  0.990 |  0.990 |  0.986
Knife |  0.072 |  0.070 |  0.086 |  1.408 |  1.901 |  1.963 |  0.987 |  0.989 |  0.983
Lamp |  0.321 |  0.349 |  0.389 |  2.274 |  3.054 |  3.091 |  0.845 |  0.826 |  0.807
Laptop |  0.127 |  0.132 |  0.165 |  2.313 |  2.535 |  2.566 |  0.987 |  0.985 |  0.977
Motorbike |  0.256 |  0.280 |  0.292 |  2.501 |  3.085 |  2.444 |  0.868 |  0.857 |  0.848
Mug |  0.369 |  0.407 |  0.453 |  2.446 |  2.305 |  2.239 |  0.726 |  0.694 |  0.623
Pistol |  0.110 |  0.128 |  0.138 |  2.611 |  2.276 |  2.206 |  0.974 |  0.970 |  0.963
Rocket |  0.162 |  0.156 |  0.173 |  2.072 |  2.481 |  2.314 |  0.917 |  0.921 |  0.911
Skateboard |  0.174 |  0.197 |  0.203 |  2.050 |  2.477 |  2.648 |  0.941 |  0.933 |  0.930
Table |  0.179 |  0.195 |  0.230 |  2.039 |  2.475 |  2.690 |  0.934 |  0.923 |  0.903
Mean |  0.230 |  0.245 |  0.278 |  2.347 |  2.479 |  2.562 |  0.886 |  0.876 |  0.861
TABLE II: Reconstruction loss via the chamfer distance (CD) and earth mover’s
distance (EMD) across a range of bits per point (bpp) for each object category
in the ShapeNet [49] dataset. The F-score was calculated at a distance
threshold of $d=0.05$.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

Figure 3: The reconstruction results for a subset of objects from the ShapeNet
[49] dataset. The first row shows the ground-truth point clouds. The second
and third rows display the reconstructed point clouds using the chamfer
distance and earth mover’s distance, respectively.
$0$$0.5$$1$$1.5$$0.82$$0.84$$0.86$$0.88$BitrateClassification
Accuracy$Uncompressed$$CD$$EMD$ Figure 4: A comparison of the classification
accuracy for uncompressed and decompressed point cloud data using the chamfer
distance (CD) and earth mover’s distance (EMD).
$0$$0.5$$1$$1.5$$10^{-2}$$10^{-1}$BitrateDistance$Draco_{p2p}$$Ours_{p2p}$$Draco_{\perp}$$Ours_{\perp}$
Figure 5: The reconstruction quality of objects from the ShapeNet [49] dataset
in terms of the point-to-point (p2p) and point-to-surface ($p_{\perp}$)
distances at different bitrates.
$700$$800$$900$$1{,}000$$0.32$$0.34$$0.36$$0.38$$0.4$Latent Vector SizeChamfer
Distance Figure 6: The increase in reconstruction loss using a truncated
latent vector to enable variable bitrates.

###  IV-B Reconstruction Evaluation

Measuring the reconstruction capability of our model is crucial to evaluating
its performance. However, direct quantitative comparisons between two point
clouds is complicated by the property that the set of points comprising a
point cloud is invariant to permutations. To address this issue, we utilized
the chamfer distance (CD) [52] and earth mover’s distance (EMD) [53]. The CD
and EMD are two permutation-invariant metrics for comparing sets of points
introduced in [54].

Initially, we used the same metrics (CD and EMD) to calculate the
reconstruction loss and to train our model. Table II establishes a benchmark
on the ShapeNet dataset using the CD, EMD, and F-score metrics. Despite the
fact that both of the losses are differentiable almost everywhere, the CD is
more efficient to compute in comparison to the EMD. In our experiments, we
found that the training process converges faster when using the CD compared to
the EMD. Notwithstanding, the EMD metric produces slightly higher-fidelity
qualitative reconstruction results as shown in Fig. 3.

In a second experiment, we measured the performance of our network against
Draco 1.4.1 [55], a state-of-the-art open-source library for compressing and
decompressing 3D geometric meshes and point clouds. For this experiment, we
again used ShapeNet and trained our model using the CD loss. The comparison
was done using the point-to-point and point-to-surface distance metrics. These
metrics are widely used to quantify 3D surface reconstruction quality and are
described in [56]. Fig. 5 shows that our architecture outperforms Draco,
especially at lower bitrates.

###  IV-C Task-Specific Evaluation

An important and practical use case of compressed point cloud data is to store
large amounts of 3D models and then preprocess them later on for downstream
functions. To quantify the reconstruction quality of such tasks, we used a
PointNet [43] model to classify point cloud objects from ModelNet40. In this
experiment, we trained a model using uncompressed data and determined its
initial accuracy. Then, we used the trained model to classify the decompressed
data and examined its accuracy against the original data. Note that both the
uncompressed and decompressed data were compared using the same metric (e.g.,
CD or EMD).

Fig. 4 shows that our compression method preserves the semantic structure of
the objects, even at low bitrates, while training with different loss
functions. We can achieve similar accuracy with the CD metric on the
compressed data (86%) when compared against the actual data (88%). An
interesting observation from this experiment is that the classification
accuracy increases gradually with the bitrate, up to a certain optimal point,
and then decreases. Based on these results, we hypothesize that the bitrate
constraint acts as a regularizer that helps the model to better generalize.

###  IV-D Variable Bitrate Evaluation

This experiment showcases the possibility of using a single trained model to
compress point cloud data at varying bitrates. At training time, the weighted
entropy loss function (3) penalizes the lower-indexed vector elements more
during the compression process. Consequently, this ensures that the high-
frequency information is carried through the higher-indexed vector elements
while the lower-indexed vector elements carry the overall shape information.
The exponential decaying function (4) is used for assigning a weight to each
vector element where $\omega_{i}$ is the $i$-th element in the weight vector.

For this experiment our model was trained using the CD metric, and the values
15 and 0.003 for variables $a$ and $b$, respectively, were used for (4). At
test time, we truncate the latent vector according to the bitrate setting and
only compress the truncated vector instead of the whole vector. In Fig. 6, we
illustrate the relationship between the loss and the number of vector elements
used in the reconstruction. As we truncate the higher-indexed vector elements
the distance between the actual and compressed point cloud gradually
increases. This naturally results in a less accurate reconstruction.

##  V Conclusion and Future Work

We presented a deep learning-based approach that is capable of variable rate
raw point cloud compression. Variable-rate compression is enabled by a tunable
loss function that distributes information non-uniformly among the latent
vector elements. This method alleviates the need for the training and storage
of multiple models over a range of bitrates. The efficacy of our model was
shown through the analysis of different components and the presentation of an
extensive evaluation on the individual impact of these factors. What’s more,
we established new benchmarks on publicly available datasets and we showed
that our model surpasses state-of-the-art open-source software for compressing
and decompressing 3D point clouds. In future work, we will investigate
extending the range of bitrates a single model can support.

## Acknowledgments

The authors acknowledge the Texas Advanced Computing Center (TACC) at the
University of Texas at Austin for providing software, computational, and
storage resources that have contributed to the research results reported
within this paper.

## References

  * [1] Z. Cai, J. Han, L. Liu, and L. Shao, “Rgb-d datasets using microsoft kinect or similar sensors: a survey,” _Multimedia Tools and Applications_ , vol. 76, no. 3, pp. 4313–4355, 2017.
  * [2] Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3d point clouds: A survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2020.
  * [3] K. Sugimoto, R. A. Cohen, D. Tian, and A. Vetro, “Trends in efficient representation of 3d point clouds,” in _Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference_. IEEE, 2017, pp. 364–369.
  * [4] C. Cao, M. Preda, and T. Zaharia, “3d point cloud compression: A survey,” in _Proceedings of the International Conference on 3D Web Technology_ , 2019, pp. 1–9.
  * [5] W. J. Beksi and N. Papanikolopoulos, “Point cloud culling for robot vision tasks under communication constraints,” in _Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2014, pp. 3747–3752.
  * [6] M. I. Patel, S. Suthar, and J. Thakar, “Survey on image compression using machine learning and deep learning,” in _Proceedings of the International Conference on Intelligent Computing and Control Systems_. IEEE, 2019, pp. 1103–1105.
  * [7] https://github.com/robotic-vision-lab/Variable-Rate-Compression-For-Raw-3D-Point-Clouds.
  * [8] D. Meagher, “Geometric modeling using octree encoding,” _Computer Graphics and Image Processing_ , vol. 19, no. 2, pp. 129–147, 1982.
  * [9] Y. Huang, J. Peng, C.-C. J. Kuo, and M. Gopi, “Octree-based progressive geometry coding of point clouds,” in _Proceedings of the Symposium on Point-Based Graphics_ , vol. 6, 2006, pp. 103–110.
  * [10] R. Schnabel and R. Klein, “Octree-based point-cloud compression,” in _Proceedings of the Symposium on Point-Based Graphics_ , vol. 6, 2006, pp. 111–120.
  * [11] J. Kammerl, N. Blodow, R. B. Rusu, S. Gedikli, M. Beetz, and E. Steinbach, “Real-time compression of point cloud streams,” in _Proceedings of the IEEE International Conference on Robotics and Automation_ , 2012, pp. 778–785.
  * [12] J. Elseberg, D. Borrmann, and A. Nüchter, “One billion points in the cloud–an octree for efficient processing of 3d laser scans,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 76, pp. 76–88, 2013.
  * [13] A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard, “Octomap: An efficient probabilistic 3d mapping framework based on octrees,” _Autonomous Robots_ , vol. 34, no. 3, pp. 189–206, 2013.
  * [14] R. Mekuria, K. Blom, and P. Cesar, “Design, implementation, and evaluation of a point cloud codec for tele-immersive video,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 27, no. 4, pp. 828–842, 2016\.
  * [15] J. L. Bentley, “Multidimensional binary search trees used for associative searching,” _Communications of the ACM_ , vol. 18, no. 9, pp. 509–517, 1975\.
  * [16] D. Cheriton and R. E. Tarjan, “Finding minimum spanning trees,” _SIAM Journal on Computing_ , vol. 5, no. 4, pp. 724–742, 1976.
  * [17] P.-M. Gandoin and O. Devillers, “Progressive lossless compression of arbitrary simplicial complexes,” _ACM Transactions on Graphics_ , vol. 21, no. 3, pp. 372–379, 2002.
  * [18] S. Gumhold, Z. Kami, M. Isenburg, and H.-P. Seidel, “Predictive point-cloud compression,” in _ACM SIGGRAPH 2005 Sketches_ , 2005.
  * [19] B. Merry, P. Marais, and J. Gain, “Compression of dense and regular point clouds,” in _Proceedings of the International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa_ , 2006, pp. 15–20.
  * [20] V. Zakharchenko, “Algorithm description of mpeg-pcc-tmc2,” in _ISO/IEC JTC1/SC29/WG11/N17767_ , 2018.
  * [21] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivokuća, S. Lasserre, Z. Li _et al._ , “Emerging mpeg standards for point cloud compression,” _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , vol. 9, no. 1, pp. 133–148, 2018.
  * [22] R. B. Rusu and S. Cousins, “3d is here: Point cloud library (pcl),” in _Proceedings of the IEEE International Conference on Robotics and Automation_ , 2011, pp. 1–4.
  * [23] T. Ochotta and D. Saupe, “Compression of point-based 3d models by shape-adaptive wavelet coding of multi-height fields,” in _Proceedings of the Symposium on Point-Based Graphics_. The Eurographics Association, 2004.
  * [24] P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in _Proceedings of the IEEE International Conference on Image Processing_ , vol. 1, 2007, pp. 1–201.
  * [25] T. Golla and R. Klein, “Real-time point cloud compression,” in _Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2015, pp. 5087–5092.
  * [26] L. He, W. Zhu, and Y. Xu, “Best-effort projection based attribute compression for 3d point cloud,” in _Proceedings of the Asia-Pacific Conference on Communications_. IEEE, 2017, pp. 1–6.
  * [27] X. Sun, H. Ma, Y. Sun, and M. Liu, “A novel point cloud compression algorithm based on clustering,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 2132–2139, 2019.
  * [28] C. Tu, E. Takeuchi, A. Carballo, and K. Takeda, “Point cloud compression for 3d lidar sensor using recurrent neural network with residual blocks,” in _Proceedings of the IEEE International Conference on Robotics and Automation_ , 2019, pp. 3274–3280.
  * [29] G. K. Wallace, “The jpeg still picture compression standard,” _IEEE Transactions on Consumer Electronics_ , vol. 38, no. 1, pp. xviii–xxxiv, 1992\.
  * [30] G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (hevc) standard,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 22, no. 12, pp. 1649–1668, 2012\.
  * [31] K. Ainala, R. N. Mekuria, B. Khathariya, Z. Li, Y.-K. Wang, and R. Joshi, “An improved enhancement layer for octree based point cloud compression with plane projection approximation,” in _Proceedings of the Applications of Digital Image Processing_ , vol. 9971. International Society for Optics and Photonics, 2016, pp. 223–231.
  * [32] J. Wang, H. Zhu, Z. Ma, T. Chen, H. Liu, and Q. Shen, “Learned point cloud geometry compression,” _arXiv preprint arXiv:1909.12037_ , 2019.
  * [33] A. F. Guarda, N. M. Rodrigues, and F. Pereira, “Deep learning-based point cloud geometry coding: Rd control through implicit and explicit quantization,” in _Proceedings of the IEEE International Conference on Multimedia & Expo Workshops_, 2020, pp. 1–6.
  * [34] F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. Van Gool, “Conditional probability models for deep image compression,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 4394–4402.
  * [35] A. F. Guarda, N. M. Rodrigues, and F. Pereira, “Deep learning-based point cloud coding: A behavior and performance study,” in _Proceedings of the European Workshop on Visual Information Processing_. IEEE, 2019, pp. 34–39.
  * [36] ——, “Point cloud coding: Adopting a deep learning-based approach,” in _Proceedings of the Picture Coding Symposium_. IEEE, 2019, pp. 1–5.
  * [37] M. Quach, G. Valenzise, and F. Dufaux, “Learning convolutional transforms for lossy point cloud geometry compression,” in _Proceedings of the IEEE International Conference on Image Processing_ , 2019, pp. 4320–4324.
  * [38] ——, “Improved deep point cloud geometry compression,” in _Proceedings of the IEEE International Workshop on Multimedia Signal Processing_ , 2020, pp. 1–6.
  * [39] L. Wiesmann, A. Milioto, X. Chen, C. Stachniss, and J. Behley, “Deep compression for dense point cloud maps,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 2060–2067, 2021.
  * [40] E. Agustsson, F. Mentzer, M. Tschannen, L. Cavigelli, R. Timofte, L. Benini, and L. Van Gool, “Soft-to-hard vector quantization for end-to-end learning compressible representations,” _arXiv preprint arXiv:1704.00648_ , 2017.
  * [41] J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimization of nonlinear transform codes for perceptual quality,” in _Proceedings of the IEEE Picture Coding Symposium_ , 2016, pp. 1–5.
  * [42] L. Theis, W. Shi, A. Cunningham, and F. Huszár, “Lossy image compression with compressive autoencoders,” _arXiv preprint arXiv:1703.00395_ , 2017\.
  * [43] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 652–660.
  * [44] N. Ahmed, T. Natarajan, and K. R. Rao, “Discrete cosine transform,” _IEEE Transactions on Computers_ , vol. 100, no. 1, pp. 90–93, 1974.
  * [45] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” _arXiv preprint arXiv:1802.01436_ , 2018.
  * [46] Y. Choi, M. El-Khamy, and J. Lee, “Variable rate deep image compression with a conditional autoencoder,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 3146–3154.
  * [47] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _arXiv preprint arXiv:1706.02413_ , 2017.
  * [48] J. Rissanen and G. Langdon, “Universal modeling and coding,” _IEEE Transactions on Information Theory_ , vol. 27, no. 1, pp. 12–23, 1981.
  * [49] A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su _et al._ , “Shapenet: An information-rich 3d model repository,” _arXiv preprint arXiv:1512.03012_ , 2015.
  * [50] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3d shapenets: A deep representation for volumetric shapes,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2015, pp. 1912–1920.
  * [51] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [52] H. Barrow, J. Tenenbaum, R. Bolles, and H. Wolf, “Parametric correspondence and chamfer matching: Two new techniques for image matching,” in _Proceedings of the Image Understanding Workshop_. Science Applications, Inc Arlington, VA, 1977, pp. 21–27.
  * [53] Y. Rubner, C. Tomasi, and L. J. Guibas, “The earth mover’s distance as a metric for image retrieval,” _International Journal of Computer Vision_ , vol. 40, no. 2, pp. 99–121, 2000.
  * [54] H. Fan, H. Su, and L. J. Guibas, “A point set generation network for 3d object reconstruction from a single image,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 605–613.
  * [55] Google, _Draco 3D Data Compression_ , 2021, https://google.github.io/draco/.
  * [56] D. Tian, H. Ochimizu, C. Feng, R. Cohen, and A. Vetro, “Geometric distortion metrics for point cloud compression,” in _Proceedings of the IEEE International Conference on Image Processing_ , 2017, pp. 3460–3464.