# Learning-Based Biharmonic Augmentation for Point Cloud Classification

Jiacheng Wei, Guosheng Lin, Henghui Ding, Jie Hu and Kim-Hui Yap Jiacheng Wei
and Kim-Hui Yap are with the School of Electrical and Electronic Engineering,
Nanyang Technological University, Singapore. (E-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Guosheng Lin and Henghui Ding are
with the School of Computer Science and Engineering, Nanyang Technological
University, Singapore. (e-mail: gslin@ntu.edu.sg).Jie Hu is with State Key Lab
of Computer Science, ISCAS & University of Chinese Academy of
Sciences.Manuscript received October 23, 2023.

###### Abstract

Point cloud datasets often suffer from inadequate sample sizes in comparison
to image datasets, making data augmentation challenging. While traditional
methods, like rigid transformations and scaling, have limited potential in
increasing dataset diversity due to their constraints on altering individual
sample shapes, we introduce the Biharmonic Augmentation (BA) method. BA is a
novel and efficient data augmentation technique that diversifies point cloud
data by imposing smooth non-rigid deformations on existing 3D structures. This
approach calculates biharmonic coordinates for the deformation function and
learns diverse deformation prototypes. Utilizing a CoefNet, our method
predicts coefficients to amalgamate these prototypes, ensuring comprehensive
deformation. Moreover, we present AdvTune, an advanced online augmentation
system that integrates adversarial training. This system synergistically
refines the CoefNet and the classification network, facilitating the automated
creation of adaptive shape deformations contingent on the learner status.
Comprehensive experimental analysis validates the superiority of Biharmonic
Augmentation, showcasing notable performance improvements over prevailing
point cloud augmentation techniques across varied network designs.

###### Index Terms:

Data Augmentation, Point Cloud Classification, Shape Deformation.

(a)

(b)

(c)

Figure 1: (a) Original airplane point cloud depicted from three distinct
perspectives. (b) Set of deformation prototypes derived from the initial
shape, each representing a feasible deformation basis. (c) Augmented samples
formed by blending the deformation prototypes. While these samples exhibit
notable shape variations compared to the original, their recognition as
airplanes remains straightforward.

##  I Introduction

Over recent years, the potential for 3D point cloud recognition has expanded
considerably, driven by enhancements in 3D sensor technology. Deep learning
methodologies, as evidenced by works such as [1, 2], have furthered
advancements in 3D computer vision. Nonetheless, 3D vision tasks are
particularly vulnerable to data limitations compared to their 2D counterparts.
Notably, while 2D datasets like ImageNet [3] and COCO [4] boast extensive size
and diversity, 3D point cloud datasets such as ModelNet40 [5] are
comparatively limited in both aspects. Furthermore, given the rising demand,
there exists a noticeable research gap in the development of augmentation
techniques for 3D point cloud data.

Traditional augmentation approaches for point cloud data, termed Conventional
Data Augmentation (CDA)[6, 7], chiefly encompass translation, scaling,
rotation, and point-wise noise addition. PointAugment[8] brings forth an auto-
augmentation framework [9], employing an auxiliary network to produce a global
transformation matrix paired with point-wise noise. Inspired by techniques
such as Mixup [10] and ManifoldMixup [11], PointMixup [12] formulates virtual
samples by interpolating two samples from discrete classes, both spatially and
feature-wise. RSMix [13], adopting the CutMix approach [14], constructs
virtual samples by substituting portions from alternate samples. Nevertheless,
a notable limitation of these strategies is their neglect of modifications to
the geometric shape or local structures of individual samples, which might be
pivotal for bolstering data diversity.

Two very recent works explore region-based augmentation for point cloud data.
PatchAugment [15] employs various Conventional Data Augmentation (CDA)
techniques within each ball query [7] localized area. However, this approach
can introduce discontinuities, yielding samples that may appear artificial. An
inherent limitation is its integration within the ball query, making it
unsuitable for networks lacking ball query functionalities.

Conversely, PointWOLF [16] applies CDA methodologies on localized areas but
incorporates the Nadaraya-Watson kernel regression [17, 18] to refine local
transformations. This leads to more authentic-looking augmentations. Yet, the
augmentation process entails executing the kernel regression technique $M$
times for each of the $M$ transformed local regions. As these local
transformations are generated in isolation, there is a risk of producing
incongruent deformations, yielding potentially inauthentic augmentations that
might impede effective training. This risk can be mitigated by confining
transformations within a meticulously crafted narrow search domain. Hence, the
crux of the challenge is the efficient creation of augmentations that embody
both plausible and authentic non-rigid deformations.

Thus, the central challenge is to adeptly produce augmentations characterized
by both plausible and authentic non-rigid deformations

(a) Original.

(b) $\sigma=0.01$.

(c) $\sigma=0.05$

(d) $\sigma=0.1$.

Figure 2: Deformations of a sample lamp with random offsets
$O\sim\mathcal{N}(0,\,\sigma^{2})$ draw from a normal distribution for each
individual control points.

In this work, we introduce Biharmonic Augmentation (BA), an innovative point
cloud data augmentation technique designed to surmount prevailing challenges
and amplify dataset diversity. As depicted in Figure 3, our approach refrains
from directly crafting deformation offsets for each data point, either on a
local or global scale. Instead, we leverage classical deformation handles from
the realm of computational geometry to parameterize smooth deformations. These
handles consist of a concise set of control points extracted from the dataset.
We adopt biharmonic coordinates [19], anchored on these control points, as our
deformation function. This facilitates the sculpting of the shape by adjusting
the coordinates of control points. The inherent advantage of biharmonic
coordinates is their capability to yield smooth deformations through
straightforward matrix multiplication. Each control point inherently affects
the deformation in a localized subspace. Nonetheless, individual modifications
to these control points can lead to unnatural deformations. For instance,
Figure 2 presents smooth but incongruous deformations created by superimposing
random coordinate offsets on control points of a sample lamp, making its
identification as a lamp challenging for human viewers. To counteract this, we
derive inspiration from [20] to cultivate a suite of deformation prototypes.
As illustrated in Figure 1(b), each prototype encompasses a set of coordinate
offsets for every control point, epitomizing a viable deformation foundation.
By amalgamating all prototypes with diverse coefficients, we can craft a
myriad of credible deformations. As a result, our deformation methodology
ensures a harmonious balance between localized smoothness and overarching
authenticity in the enhanced point clouds.

For optimal augmentations tailored to classification tasks, our goal is to
pinpoint target shapes conducive to classification training. This involves
determining the most apt coefficients for the deformation prototypes. To
achieve this, we introduce CoefNet, designed to predict these coefficients for
automated augmentation creation. Instead of directly generating the
coefficients, CoefNet predicts offsets for coefficients chosen at random,
rectifying any unfavorable deformations. This strategy ensures that CoefNet
does not gravitate towards particular deformations, thereby preventing both
networks from becoming ensnared in local minima.

Drawing inspiration from notable works [21, 22, 23, 24, 25], we introduce
AdvTune, an online augmentation framework. AdvTune seamlessly integrates
adversarial learning methodologies, enabling concurrent training of both the
CoefNet and classification networks. Consequently, our Biharmonic Augmentation
is adept at autonomously producing augmentations that are both smooth and
credible, dynamically adapting to the state of the learner network

In summary, our contributions can be summarized as:

  * •

We present Biharmonic Augmentation, a pioneering data augmentation technique
adept at enhancing data diversity by proficiently crafting smooth and
authentic non-rigid deformations within point cloud data.

  * •

We architect AdvTune, an online adversarial auto-augmentation framework,
adeptly producing augmentations that are attuned to the current state of the
learner network.

  * •

Through empirical studies, we validate that our Biharmonic Augmentation yields
substantial advancements in point cloud shape classification efficacy,
surpassing prevailing augmentation approaches across diverse network
backbones.

##  II Related Works

Data Augmentation for 2D Images: In 2D vision tasks, common random data
augmentations [26] include flipping, random cropping, and rotation applied to
image datasets. Subsequent methods, such as Gaussian noise [27] and regional
dropout [28], introduce noise or eliminate certain pixels within images.
Techniques like Mixup [10] employ linear interpolation between two images to
craft virtual samples. Advanced approaches, such as ManifoldMixup [11], blend
two images in the feature domain, while CutMix [14] merges images by
substituting certain regions from one image into another. AutoAugment [9] and
its derivatives [29, 30] integrate AutoML strategies [31] to autonomously
discern optimal augmentation policies. It is worth noting that these
aforementioned strategies are typically classified as offline data
augmentation techniques, given that the augmentation process remains distinct
from the primary model training phase.

Adversarial Training for Image Data Augmentation:  DAGAN [32] employs the
Generative Adversarial Network (GAN)[33] to produce augmentations. Works such
as[22, 24, 25] harness adversarial training as a data augmentation tactic,
where the augmented data is infused with adversarial perturbations. While
these methods can bolster model robustness, they occasionally compromise
generalizability. Efforts like Online Augment [21] and AdvProp [23] seek to
enhance generalizability during adversarial training. These techniques are
typically categorized as online data augmentation methods, given their ability
to dynamically generate augmented data in accordance with the evolving state
of the training model.

Deep Learning on 3D Point Clouds: While 2D image data is structured with grid
pixels, 3D point cloud data consists of unordered sets. This distinction
complicates the direct application of established convolutional neural network
architectures to point cloud datasets. PointNet [6] offers a solution by
leveraging shared multi-layer perceptions for individual points and
subsequently deriving a global feature through a max-pooling operation. Its
enhanced successor, PointNet++[7], utilizes a hierarchical framework,
integrating multiple PointNet-inspired architectures to capture and group
local data attributes. Several studies [34, 1] have applied convolution
operations to point clouds, focusing on nearest neighbors. On the other hand,
voxel-based techniques [2] segment point cloud data into discrete voxels,
conducting sparse convolutions on these segmented units. In the realm of
neural networks and learning systems, deep learning methods on point clouds
are drawing more attentions, [35, 36] focus on the point cloud classification
task using attention modules. [37, 38] proposed graph neural network based
methods for point cloud segmentation tasks. [39] also utilizes point cloud
deformation for reconstruction tasks.

Data Augmentation for 3D point clouds: Conventional data augmentation (CDA)
[6, 6] applies translation, scaling, rotation and point-wise noise to point
cloud data. PointAugment [8] introduces an auto-augmentation [9] framework to
learn to generate a global transformation matrix and point-wise noise with
another network. Mixup based methods [12, 13] create virtual samples by mixing
two point cloud samples. PatchAugment [15] and PointWOLF [16] performs
different conventional data augmentation techniques to different local regions
in the point cloud samples.

Figure 3: The framework of Biharmonic Augmentation. (a) Shows the entire
workflow of the proposed method. (b) Shows the details of the Biharmonic
Augmentation during the training of the target network. During the adversarial
training, the pretrained PrototypeNet is frozen. The PrototypeNet generates
deformation prototypes from input point clouds and the sampled control points.
The CoefNet takes the deformation prototypes and the control points to
generate the combination weights of the prototypes. Then, the augmented point
cloud is derived from the linear combination of the prototypes. The CoefNet
and the classification network are jointly updated through adversarial
training.

##  III Method

###  III-A Biharmonic Deformation

The primary objective of our method is to introduce smooth and plausible
deformations of existing point cloud data, thereby enriching data diversity.
Traditional shape deformation tasks typically involve constraints, such as
target shapes, which may limit flexibility. To overcome this limitation, we
adopt handle-based deformation, a well-established technique in computational
geometry. Unlike directly manipulating individual point positions, handle-
based deformation offers a lower degree of freedom, making it more manageable
and allowing for smoother and more realistic augmentations. The deformation
functions are derived by solving biharmonic equations, leveraging selected
handles and their corresponding boundary conditions, as demonstrated in prior
studies by Jacobson et al. [40, 41]. This approach empowers our method to
create augmented point cloud data with enhanced realism and versatility.

We adopt the efficient solution proposed by Wang et al. [42] to compute the
matrix $\mathbf{W}$, which involves solving biharmonic functions, leading to
the derivation of biharmonic coordinates. These coordinates enable us to
represent the deformation function as $f(\mathbf{C})=\mathbf{W}\mathbf{C}$,
where $\mathbf{C}$ corresponds to the coordinates of the control points.
Consequently, with the coordinates of the control points $f(\mathbf{C})$, we
can effortlessly reconstruct the original point cloud by employing simple
matrix multiplication. This methodology allows us to efficiently and
accurately generate smooth and plausible deformations, enhancing the data
diversity and realism of the augmented point cloud samples.

In this context, each control point corresponds to the shape of a local
region. Deformation can be achieved by adding position offsets
$\mathbf{O}\in\mathbb{R}^{c\times 3}$ to the control points, indicating the
displacements of the control points. Then, the new control point coordinates
will be $\mathbf{C}=\mathbf{C_{0}}+\mathbf{O}$, where
$\mathbf{C_{0}}\in\mathbb{R}^{c\times 3}$ represents the original rest
position of the control points, and the deformation function can be rewritten
as

$f(\mathbf{O})=\mathbf{W}(\mathbf{C_{0}}+\mathbf{O}),$ (1)

The deformation function has $3c$ degrees of freedom. As demonstrated in
Figure 2, we can generate smooth deformations by moving the control points
individually, but this may result in implausible deformations that contradict
the inherent characteristics of the original shapes.

Figure 4: Our CoefNet takes the PrototypeNet features, the learned deformation
prototypes, the control point rest positions and a set of random coefficient
sampled from a normal distribution and generates a coefficient offsets to
correct the random coefficients.

###  III-B Deformation Prototypes

We adopt the approach presented in [20] to learn a set of deformation
prototypes that impose constraints and correlations on the control points in a
lower-dimensional subspace. For each point cloud, we generate $m$ deformation
prototypes $\mathbf{M}_{i}\in\mathbb{R}^{c\times 3}$. Each deformation
prototype contains a set of offsets for all the control points:

$\mathbf{M}_{i}=[\vec{o}_{i1},\cdots,\vec{o}_{ic}]^{T},$ (2)

where $\vec{o}_{ij}\in\mathbb{R}^{3}$ is the offset of the $j$-th control
point in deformation prototypes $\mathbf{M}_{i}$. Each deformation prototype
indicates a plausible global deformation direction for the point cloud sample.
By introducing a set of combination coefficients
$\mathbf{a}=[a_{1},\cdots,a_{m}]$, the overall control point offsets can be
derived by a linear combination of the deformation prototypes:
$\mathbf{O}=\sum_{i=1}^{m}a_{i}\mathbf{M}_{i}$. Then, the new deformation
function is defined as:

$g\left(\mathbf{a};\\{\mathbf{M}_{i}\\}_{i=1}^{m}\right)=\mathbf{W}(\mathbf{C_{0}}+\sum_{i=1}^{m}a_{i}\mathbf{M}_{i}),$
(3)

The new deformation function $g:\mathbb{R}^{m}\rightarrow\mathbb{R}^{n\times
3}$ thus has only $m$ degrees of freedom. However, with different combination
coefficients, we can generate arbitrary many plausible deformations.

We generate the deformation prototypes by adopting Meta-HandleNet proposed by
[20] and denote it PrototypeNet. The PrototypeNet is trained by deforming a
source point cloud to a target point cloud sample in the same category. We
adopt most of the training strategy from [20] with some modifications to adapt
to the shape classification task. Instead of training separate models for each
category, we train a single model for all the classes for efficiency. Due to
the limited space, we will detail the training process of PrototypeNet in
supplementary materials. Since the target samples are all other samples from
the same category, the deformation prototypes are learned to factorize the
subspace that covers all the plausible deformations. To improve the efficiency
in the later data augmentation process, we save the generated deformation
prototypes and an intermediate feature for each point cloud training sample
for later usage.

We use the biharmonic deformation as a tool to generate augmentations.
However, shape deformation methods require a target shape as input. Therefore,
the problem lies in finding the optimal target shapes, which are the augmented
shapes that effectively help the classification training.

###  III-C CoefNet

The naive way to perform the data augmentation is to draw a set of random
coefficients from a normal distribution. However, as discussed in Section II,
many previous works show that random augmentation is less efficient and
effective than searching or learning-based augmentation methods. Hence, we
design a CoefNet to automatically generate the combination coefficients
$\mathbf{a}$. Meanwhile, to prevent the CoefNet from converging to some local
minima and producing constant deformations. It is still important to bring
randomness into the network. A conventional way is to feed random noise along
with the inputs to the network. However, the generated coefficients may not
provide enough diversity in the augmentation. We combine the above two methods
and design our CoefNet to predict offsets to the randomly sampled
coefficients. The randomly sampled coefficients provide sufficient diversity
to the augmentation while the predicted offsets can correct unwanted
deformation and guide the augmentation to adapt to the learner state.

Specifically, as shown in Figure 4, given a random coefficient
$\mathbf{a^{rand}}=[a_{1}^{rand},\cdots,a_{m}^{rand}]\sim\mathcal{N}(\mu_{a},\,\sigma_{a}^{2})$,
we expand it by $c$ times, where $c$ is the number of control points.
Similarly, we expand the control points coordinates and the stored
PrototypeNet features $\mathbf{F_{mh}}$ by $m$ times. Then, we concatenate
these inputs together with the generated deformation prototypes and form an
overall feature size $m\times c\times 71$. Then, we feed this feature to three
MLP layers and generate a $m\times c\times 1$ tensor. We apply max-pooling on
the tensor along the $c$ dimension to get the coefficient offsets
$\mathbf{a^{off}}$. The overall coefficients is derived by
$\mathbf{a}=\mathbf{a^{rand}}+\beta\mathbf{a^{off}}$.

Algorithm 1 summarizes the whole process to generate deformed augmentation
samples.

Figure 5: The network architecture of PrototypeNet during training, and we
only save the generated deformation prototypes and the intermediate feature
for our further process. Here $g$ is the deformation function in Equation 3.

###  III-D AdvTune

In the following section, we present AdvTune, an online data augmentation
framework with adversarial training, designed to jointly optimize the
classification networks and CoefNet. This process guides the CoefNet to
generate deformations that best fit the classification network.

The original purpose of adversarial training [43] is to enhance model
robustness against adversarial attacks. However, such trained models exhibit
poor generality on clean or original samples [44, 43]. As discussed by [23],
the problem is caused by a distribution mismatch, where the original samples
and adversarial samples lie on two different distributions. Adversarial
training has been proven useful[23, 21] in assisting network training to
achieve both robustness and generality by utilizing its regularization
ability, provided that the distribution mismatch is well managed.

We propose two strategies to overcome the distribution mismatch. The first
strategy involves applying regularizations on the augmentor CoefNet during
training, preventing excessive distribution displacement from the original
data. The second strategy is an iterative training scheme.

Data: control points $\mathbf{C_{0}}\in\mathbb{R}^{c\times 3}$,

biharmonic coordinates $\mathbf{W}\in\mathbb{R}^{n\times c}$,

deformation prototypes $\mathbf{M}_{i=0}^{m}\in\mathbb{R}^{m\times c\times
3}$,

CoefNet $h(.;\mathbf{F_{mh}},\mathbf{C},\mathbf{W})$

Result: augmented point cloud $\mathbf{P^{d}}\in\mathbb{R}^{n\times 3}$

Sample random coefficients
$\mathbf{a^{rand}}=[a_{1}^{rand},\cdots,a_{m}^{rand}]\sim\mathcal{N}(\mu_{a},\,\sigma_{a}^{2})$

Update guided coefficients $\mathbf{a}\leftarrow
h(\mathbf{a^{rand}};\mathbf{F_{mh}},\mathbf{C},\mathbf{W})$

Generate augmentation
$\mathbf{P^{d}}=\mathbf{W}(\mathbf{C_{0}}+\sum_{i=1}^{m}a_{i}\mathbf{M}_{i})$.

return _$\mathbf{P^{d}}$_

Algorithm 1 Biharmonic Augmentation

We denote the parameters in the classification network and CoefNet as $\theta$
and $\phi$. Given a point cloud dataset $\\{\mathbf{P}_{k}\\}_{k=1}^{K}$ with
$K$ samples, for each sample $\mathbf{P}_{k}$, a biharmonic coordinates
$\mathbf{W}_{k}$, a set of control points $\mathbf{C_{0}}_{k}$ and deformation
prototypes $\mathbf{M}_{k}$. A recovered sample $\mathbf{P^{r}}_{k}$ can be
derive as $\mathbf{P^{r}}_{k}=\mathbf{W}_{k}\mathbf{C_{0}}_{k}$ where
$\mathbf{P^{r}}_{k}$ is very similar to $\mathbf{P}_{k}$. Then, we generate a
set of coefficients $\mathbf{a}_{k}$ using CoefNet and produce an augmented
sample
$\mathbf{P^{d}}_{k}=\mathbf{W}_{k}(\mathbf{C_{0}}_{k}+\sum_{i=1}^{m}a_{ki}\mathbf{M}_{ki})$.

[24, 25] suggest training the classification network with a mixture of the
original and augmented data. We use stochastic gradient descent for
illustration. For each mini-batch, $\theta$ is updated by gradients:

$\nabla_{\theta}\mathcal{L}(\mathbf{P},y;\theta)+\nabla_{\theta}\mathcal{L}(\mathbf{P^{r}},y;\theta)+\nabla_{\theta}\mathcal{L}(\mathbf{P^{d}},y;\theta,\phi),$
(4)

where $\mathcal{L}$ is the loss function, in our case, a cross-entropy loss
and $y$ is the ground truth label.

We apply two regularization terms in optimizing the $\phi$ in CoefNet to
prevent overwhelming distribution displacement. The first term is a Chamfer
Distance [45] between the recovered and augmented samples.

$\displaystyle\mathcal{R}_{\texttt{CD}}(\mathbf{P^{d}},\mathbf{P^{r}})=$
$\displaystyle\sum_{p_{1}\in\mathbf{P^{d}}}\min_{p_{2}\in\mathbf{P^{r}}}\|p_{1}-p_{2}\|_{2}^{2}$
$\displaystyle+$
$\displaystyle\sum_{p_{2}\in\mathbf{P^{r}}}\min_{p_{1}\in\mathbf{P^{d}}}\|p_{1}-p_{2}\|_{2}^{2},$
(5)

The second term is a symmetry loss [46] of the augmented sample since most of
the objects are symmetric in certain directions. Since the shape
classification datasets are usually unaligned, we developed an orientation
invariant symmetry loss by taking the minimum value of $v$ symmetry loss
calculated along $v$ directions separated by $\pi/v$:

$\mathcal{R}_{\texttt{sym}}(\mathbf{P^{d}})=\min_{\delta\in\mathbf{\Delta}}\mathcal{R}_{CD}(Mir(Rot(\mathbf{P^{d}},\delta)),Rot(\mathbf{P^{d}},\delta)),$
(6)

where $Mir$ takes the mirror of the point cloud along x-axis, $Rot$ is a
rotation function along the up-axis, $\Delta$ is the set of $v$ rotation
angles. With these two regularization terms, $\phi$ is updated by:

$\begin{split}\lambda_{cd}\nabla_{\phi}\mathcal{R}_{cd}(\mathbf{P^{r}},&\mathbf{P^{d}};\phi)\\\
+\lambda_{sym}\nabla_{\phi}\mathcal{R}_{sym}(\mathbf{P^{d}};\phi)-&\lambda_{adv}\nabla_{\phi}\mathcal{L}(\mathbf{P^{d}},y;\theta,\phi),\end{split}$
(7)

where $-\mathcal{L}(\mathbf{P^{d}},y;\theta,\phi)$ is the adversarial loss
from the classification network and $\lambda$s are the balancing weights.

Fine-tuning is proved to be another solution to relief distribution mismatch
by [23]. However, a final fine-tuning following the whole training process may
not be strong enough to recover all the performance degradation caused by
distribution mismatch, especially for small-scale networks. We enhance the
ability to finetune to neutralize distribution mismatch with a simple yet
effective iterative training scheme.

Algorithm 2 summarizes the procedure of AdvTune. We separate the
classification network training by training with the mixture of
$\mathbf{P^{r}}$ and $\mathbf{P^{d}}$ as the adversarial learning step and
training with $\mathbf{P}$ as the fine-tuning step. Then, Equation 4 is
divided into two parts:

$\nabla_{\theta}\mathcal{L}(\mathbf{P^{r}},y;\theta)+\nabla_{\theta}\mathcal{L}(\mathbf{P^{d}},y;\theta,\phi),$
(8) $\nabla_{\theta}\mathcal{L}(\mathbf{P},y;\theta).$ (9)

We establish an inner loop consisting of $E_{i}$ iterations and an outer loop
with $E_{o}$ iterations. For each iteration in the outer loop, we initially
update $\theta$ using Equation 8 for $E_{i}$ iterations. Subsequently,
$\theta$ is updated with Equation 9 for an additional $E_{i}$ iteration to
counteract distribution displacement, a process we term distribution tuning.
Following this, we update $\phi$ employing Equation 7, taking into account the
classification network’s state with the adversarial loss. Ultimately, after
completing $E_{o}$ iterations of the outer loop, we perform an additional
$E_{f}$ iterations, updating $\theta$ using Equation 9 as a final fine-tuning
step to further mitigate the effects of distribution mismatch.

Data: control points $\mathbf{C_{0}}$, biharmonic coordinates $\mathbf{W}$,
deformation prototypes $\mathbf{M}_{i=0}^{m}$, original point cloud
$\mathbf{P}$, Initialized $\theta$ and $\phi$ from classification network and
CoefNet

Result: Updated $\theta$ and $\phi$

for  _each outer loop_ do

for  _each inner loop_ do

Generate $\mathbf{P^{r}}=\mathbf{W}\mathbf{C_{0}}$

Generate $\mathbf{P^{d}}$ using Algorithm 1

Update $\theta$ using Equation 8

end for

for  _each inner loop_ do

Update $\theta$ using Equation 9

end for

for  _each inner loop_ do

Generate $\mathbf{P^{r}}=\mathbf{W}\mathbf{C_{0}}$

Generate $\mathbf{P^{d}}$ using Algorithm 1

Update $\phi$ using Equation 7

end for

end for

for  _Final fine-tuning iterations_ do

Update $\theta$ using Equation 9

end for

return _$\theta$ , $\phi$_

Algorithm 2 AdvTune

Model | CDA [6] | CDA(w/o R) | PointAugment [8] | PointMixup [12] | RSMix [13] | PointWOLF [16] | BA(Ours)
---|---|---|---|---|---|---|---
PointNet | 89.2 | 89.7 | 90.9 | 89.9 | 88.7 | 91.1 | 91.6
PointNet++ | 90.8 | 92.5 | 92.5 | 91.7 | 91.6 | 93.2 | 93.3
DGCNN | 91.9 | 92.7 | 92.9 | 93.1 | 93.2 | 93.2 | 93.4

TABLE I: The overall accuracy on ModelNet40. model | CDA | PointAugment | BA(Ours)
---|---|---|---
PointNet | 91.9 | 94.1 | 94.3
PointNet++ | 93.3 | 94.9 | 95.5
DGCNN | 94.8 | 95.1 | 95.6
TABLE II: The overall accuracy on ModelNet10.

##  IV Experiments

###  IV-A Datasets

We conduct object classification experiments on the widely recognized
ModelNet40 [5] dataset and its subset, ModelNet10. ModelNet40 comprises 9,843
training samples and 2,468 testing samples across 40 classes, generated from
CAD models. ModelNet10, a subset of ModelNet40, consists of 10 classes with
aligned orientations, containing 3,991 training samples and 908 testing
samples. For ModelNet40, we compute the biharmonic coordinates utilizing the
raw dataset, but assess our model using the test set from the extensively
adopted preprocessed data provided by PointNet [6] to ensure a fair
comparison. For each sample, we employ 1,024 points as input data,
disregarding the surface normal information.

###  IV-B Implementation

Backbone Networks. Our method is model agnostic and we conduct experiments on
three popular networks for point clouds, namely PointNet [6], PointNet++ [7]
and DGCNN [47].

Data Preparation To compute the biharmonic coordinates on ModelNet40. The
process begins by establishing watertight manifolds through the utilization of
the method proposed by Huang et al. [48] on the raw ModelNet40 data. Following
this step, both tetrahedral meshes and surface meshes are generated from the
aforementioned manifold, employing the fTetWild algorithm [49]. Subsequently,
the biharmonic coordinates are computed on the resulting surface mesh using
the libigl library [50].

Training BA on ModelNet. All neural networks and experiments are implemented
using PyTorch. We employ the trained PrototypeNet to generate deformation
prototypes. For all experiments, the CoefNet is trained with the Adam
optimizer, with an initial learning rate of 0.001. The parameter $\beta$ is
empirically set at 0.1, and $v$ is selected as 2. The balancing weights
$\lambda_{cd}$, $\lambda_{sym}$, and $\lambda_{adv}$ are empirically set at
0.1, 0.1, and 0.01, respectively. The mean $\mu_{a}$ is set to 0, and the
standard deviation $\sigma_{a}$ is set at 0.5. For PointNet and PointNet++, we
use the implementation from [51]. For DGCNN, we employ its official Pytorch
implementation. We adhere to most of the training specifications from the
implementations, and we train each model with a batch size of 32. The outer
iteration is set to 15, and the inner iteration is set to 10, with each inner
iteration corresponding to 1 epoch. The final fine-tuning epoch number is 100.

Training of PrototypeNet. Our PrototypeNet is adapted from the MetaHandleNet
of [20] and follows most of the training strategies. Figure 5 illustrates the
architecture of PrototypeNet during training. Unlike [20], which trains a
model for each class, we train a single network for all classes. Each step
takes a source point cloud and a target point cloud from the same category.
The two-point clouds are fed into a PointNet[6] encoder to generate global
features. Subsequently, the source feature with control point handles and
biharmonic coordinates is concatenated and fed into a group of MLPs to
generate PrototypeNet features and predict deformation prototypes with
additional MLP layers. Concurrently, the source and target features with
control point coordinates and the PrototypeNet features are concatenated and
fed into another group of MLPs to generate coefficients that deform the source
to the target. Finally, the deformed point cloud is generated using Equation
3. We employ the same loss functions $\mathcal{L}_{fit}$,
$\mathcal{L}_{symm}$, $\mathcal{L}_{nor}$, $\mathcal{L}_{Lap}$,
$\mathcal{L}_{sp}$, $\mathcal{L}_{cov}$, $\mathcal{L}_{ortho}$, and
$\mathcal{L}_{SVD}$ to train the network. $\mathcal{L}_{fit}$ and
$\mathcal{L}_{symm}$ represent the Chamfer Distance and symmetry loss
calculated between the deformed point cloud and the target point cloud.
$\mathcal{L}_{nor}$ is the surface normal loss, and $\mathcal{L}_{Lap}$ is the
mesh Laplacian loss; these two losses are calculated on the reconstructed
surface meshes of the deformed and target point clouds. $\mathcal{L}_{sp}$,
$\mathcal{L}_{cov}$, $\mathcal{L}_{ortho}$, and $\mathcal{L}_{SVD}$ are
calculated on the deformation prototypes to encourage disentanglement of the
deformation prototypes, enabling the deformation prototypes to correspond to
different deformation directions. Please refer to [20] for the details of the
loss functions.

The PrototypeNet is designed for shape deformation datasets, which are aligned
datasets that all the samples have the same orientation. However, the data are
not always aligned in shape classification datasets. Therefore, during the
training of PrototypeNet, we rotate the target point cloud along up-axis $v$
times for $2\pi/v$ degrees of each rotation and we calculate the Chamfer
Distance between the $v$ rotations of target point cloud with the source point
cloud. Then, we take the rotation with minimum Chamfer Distance to the input
point cloud as the input target point cloud to eliminate the orientation
problem for shape classification datasets. For ModelNet40, we use $v=4$. For
each training epoch, we select 20 target point clouds for each source point
cloud. Since we train all the classes in a single network, we train the
network for 60 epochs instead of the default 30 epochs. We use an Adam
optimizer with an initial learning rate of 0.001.

###  IV-C Shape Classification

We compare our proposed BA with several data augmentation methods on point
cloud data. CDA denotes conventional data augmentation including global
translation, scaling, rotation along up-axis and point-wise jittering. Similar
to [16], we found that rotation along up-axis could be harmful to the network
training. Therefore, we also provide the result of CDA without rotation. For
fair comparison, we reproduce the results of other methods using their
official implementations with the same backbone implementation. PointAugment
[8] and PointMix [12] are two poineer works in the area. RSMix [13] and
PointWolF [16] are two recent works with good performance.

Our method produces the largest performance gain on all three backbone
networks.

$\sigma$ | 0 | 0.01 | 0.02 | 0.05 | 0.1 | 0.2
---|---|---|---|---|---|---
accuracy | 88.0 | 86.9 | 86.9 | 86.8 | 87.2 | 86.9
TABLE III: Moving individual control points with random offsets draw from a
normal distribution with different standard deviation ($\mu=0$), no
convectional data augmentation during training.

###  IV-D Experiments on Deformation Prototypes

In Table III, several PointNet models are trained utilizing augmentations
generated by adding random offsets to individual control points, while
excluding conventional data augmentation techniques. The offsets are derived
from normal distributions with $\sigma=(0.01,0.02,0.05,0.1,0.2)$. When
compared to the baseline result of 88.0, all models with random offsets
exhibit a decline in classification performance. Additionally, experiments
were conducted to train PointNet with randomly generated combination
coefficients for deformation prototypes, as shown in Table IV. This can be
perceived as a comparison with [20] when random target shapes are employed.
Evidently, deformations generated by random coefficients can enhance network
performance in classification, provided they are sampled within a reasonable
range ($\sigma_{a}\leq 0.5$). Moreover, as depicted in Figure 2, implausible
and unrealistic deformation is introduced by individual offsets applied to
control points. Consequently, it can be concluded that unrealistic
augmentation negatively impacts network training, while deformation prototypes
address this issue as they are learned to represent plausible deformation
directions.

$\sigma_{a}$ | 0.1 | 0.2 | 0.3 | 0.5 | 0.7 | 1
---|---|---|---|---|---|---
random coefficients | 89.3 | 89.6 | 89.5 | 90.1 | 87.8 | 78.4
BA | 90.3 | 90.9 | 91.1 | 91.6 | 91.1 | 90.8
TABLE IV: BA comparing to randomly generated combination coefficients. Figure
6: From left to right. We show the original point cloud, the deformed point
cloud with random offset draw from $\mathcal{N}(0,0.1)$ on each control
points, the deformed point cloud with random coefficients draw from
$\mathcal{N}(0,0.5)$ on the deformation prototypes and the deformed point
cloud with the same random coefficients but corrected by CoefNet on the
deformation prototypes. It is obvious that random offsets on control points
will lead to unrealistic and twisted deformations, random coefficients on
deformation prototypes would not generate twisted samples but could produce
some overwhelming deformation that make the augmented samples unreal and our
CoefNet is able to correct those overwhelming deformation and generate
realistic deformations.

###  IV-E Experiments on CoefNet

Table IV presents a comparison of PointNet classification performances between
random coefficients and coefficients generated by CoefNet. As discussed in
Section III-C, the coefficients produced by CoefNet are also randomly drawn
from normal distributions. However, CoefNet predicts a set of offsets to guide
and correct the random coefficients. The results clearly indicate that CoefNet
facilitates the generation of more diverse deformations, as it achieves
superior performance. Furthermore, the learned coefficients exhibit robustness
when the initial coefficients are drawn from distributions with larger
variances. Specifically, for $\sigma_{a}=1$ and $\sigma_{a}=0.7$, the
classification performance significantly declines with random coefficients,
while the performance remains robust for learned coefficients. This
demonstrates CoefNet’s capacity to correct undesirable augmentations. Table V
compares the performance of three backbone networks when CoefNet directly
generates the coefficients with input noises and predicts coefficient offsets
to guide random coefficients. Predicting offsets to random coefficients
outperforms directly predicting coefficients across all three networks,
validating our assumption that input noise may not offer sufficient randomness
to enhance the diversity of datasets and demonstrating the effectiveness of
our designed CoefNet.

| PointNet | PointNet++ | DGCNN
---|---|---|---
Direct coefficient | 90.8 | 92.0 | 92.9
Coefficient offsets | 91.6 | 93.3 | 93.4
TABLE V: Directly predicting coefficients compares to predicting coefficient
offsets in CoefNet.

###  IV-F Ablation Study on AdvTune

We conduct ablation studies to evaluate the effectiveness of AdvTune. In Table
VI, we present the classification performance of PointNet trained with and
without the two regularization terms. It is evident that both the Chamfer
distance and the orientation invariant symmetry loss contribute to the
performance of the final result. This substantiates that the two
regularization terms can prevent the CoefNet from learning malicious
adversarial deformations, thereby mitigating the issue of distribution
mismatch.

$\mathcal{R}_{sym}$ | $\mathcal{R}_{cd}$ | accuracy
---|---|---
|  | 91
✓ |  | 91.3
| ✓ | 91.3
✓ | ✓ | 91.6
TABLE VI: Ablation study on the two regularization terms Symmetry Loss and
Chamfer Distance on PointNet. Figure 7: Visualizations of the augmented
samples generated by Biharmonic Augmentation in different outer iterations.

Table VII presents the ablation study of the two fine-tuning modules in
AdvTune. DF denotes distribution fine-tuning, corresponding to Equation 9, in
which the network is trained with clean samples only after training with
augmented samples. FF represents final fine-tuning, corresponding to the last
loop in Algorithm 2, where the classification model is fine-tuned after the
adversarial training process. The experiments reveal that for PointNet and
PointNet++, the classification performance declines considerably without fine-
tuning, indicating a severe distribution mismatch issue. The performance of
PointNet++ is even lower than the baseline result of conventional data
augmentation. The second row supports our assumption that solely employing
final fine-tuning is inadequate for neutralizing the distribution displacement
accumulated during adversarial training, but it can alleviate the effect. The
third row demonstrates that distribution fine-tuning, with our designed
iterative training scheme, effectively reduces distribution displacement
during adversarial training. It yields improvements of +2.6%, +4.7%, and 1.1%
for PointNet, PointNet++, and DGCNN compared to the first row results without
fine-tuning. In each outer loop, distribution fine-tuning is performed
immediately after training with augmented samples, where distribution
displacement occurs. Subsequently, CoefNet training follows distribution fine-
tuning, which implies that displacement does not accumulate during the
training. The fourth row indicates that, with the iterative training scheme,
final fine-tuning serves as a complementary module that brings minor
improvements to the final results. Thus, the experiments demonstrate that our
AdvTune plays a significant role in training both the classification network
and the CoefNet.

Training | PointNet | PointNet++ | DGCNN
---|---|---|---
w/o DF+FF | 88.9 | 87.0 | 92.3
FF w/o DF | 90.5 | 91.7 | 92.5
DF w/o FF | 91.5 | 93.0 | 93.4
DF + FF | 91.6 | 93.3 | 93.4
TABLE VII: Ablation study on AdvTune on three classification networks. DF denotes distribution fine-tuning and FF denotes final fine-tuning. Corruption | w/o DA | CDA | BA
---|---|---|---
jitter $\sigma=0.01$ | 87.1 | 88.1 | 91.0
jitter $\sigma=0.03$ | 84.3 | 83.5 | 88.6
jitter $\sigma=0.05$ | 66.2 | 62.8 | 72.6
Dropout 0.25 | 87.8 | 88.3 | 91.2
Dropout 0.5 | 86.5 | 87.5 | 90.9
Dropout 0.75 | 80.0 | 78.2 | 84.6
Scale 0.9 | 85.8 | 88.4 | 91.6
Scale 1.1 | 86.3 | 88.9 | 91.3
TABLE VIII: Robustness test on PointNet with different corruptions in the
testing data.

###  IV-G Robustness Test

As shown in Table VIII, we perform a robustness test to our model with
corrupted testing data. The experiments are performed on PointNet. We corrupt
the testing data with different levels of random jittering, random dropout and
scaling. And we compare our method with models trained with no data
augmentation and conventional data augmentations. For random jittering, the
performance difference is larger when the jittering gets stronger.
Conventional data augmentation even performs worse than the model trained with
no data augmentation. Our method also shows robustness to random dropout and
scaling compared to conventional data augmentation.

###  IV-H Qualitative Results

Figure 6 compares four types of point clouds: the original point cloud, the
deformed point cloud with random offsets drawn from $\mathcal{N}(0,0.1)$ on
each control point, the deformed point cloud with random coefficients drawn
from $\mathcal{N}(0,0.5)$ on the deformation prototypes, and the deformed
point cloud with the same random coefficients but corrected by CoefNet on the
deformation prototypes. As shown in the second column of Figure 6, randomly
moving individual control points can easily create distorted deformations that
are difficult for humans to recognize their correct categories. This issue is
resolved by using deformation prototypes. However, although deformation
prototypes with random coefficients do not create distorted deformations, they
may generate excessive deformations on specific prototypes. As illustrated in
the third column of Figure 6, overwhelming deformation can also produce
unrealistic deformations. In contrast, when we feed the same random
coefficients to our proposed CoefNet, as shown in the fourth column of Figure
6, we can correct the excessive deformation and generate realistic
deformations. The experiment results in Table IV show that performance
declines with random coefficients when they are drawn from a distribution with
a larger standard deviation, which experience the same issue illustrated in
the third column of Figure 6.

Figure 7 presents additional visualizations of augmented samples at different
training stages. We can observe that the deformed shapes vary during the
training, indicating that the deformations do not converge to specific shapes,
and thus providing sufficient diversity throughout the training process.

This demonstrates that our proposed CoefNet is capable of correcting the
random coefficients to produce realistic deformations while preserving the
diversity offered by the random coefficients.

##  V Future works

The proposed method exhibits potential for future extensions, particularly
towards scene-level applications. A plausible avenue for advancement involves
augmenting 3D scenes by manipulating individual object shapes within the
scene. Adversarial training can be leveraged to identify optimal augmentations
that enhance performance in scene-understanding tasks. Additionally, as a part
of prospective research, exploration into alternative non-rigid deformation
techniques presents an opportunity to expedite the training process for
deformation prototype networks. These prospective endeavors hold promise for
further enhancing the capabilities and efficiency of the method.

##  VI Conclusion

This paper introduces Biharmonic Augmentation, a novel and impactful data
augmentation approach designed for point cloud classification tasks. Our
method achieves smooth and realistic deformations on the original data,
effectively expanding the dataset. These deformations are efficiently
generated through straightforward matrix multiplication techniques.
Additionally, we present AdvTune, an online auto-augmentation framework
empowered by adversarial training, which empowers the augmentor network
CoefNet to generate more diverse and meaningful augmentations tailored to the
target network. The utilization of deformation-based data augmentation offers
significant potential, addressing the data scarcity challenge in 3D point
cloud data by producing authentic and expanded dataset augmentations.

## References

  * [1] H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6411–6420.
  * [2] C. Choy, J. Gwak, and S. Savarese, “4d spatio-temporal convnets: Minkowski convolutional neural networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3075–3084.
  * [3] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Advances in neural information processing systems_ , vol. 25, pp. 1097–1105, 2012.
  * [4] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _European conference on computer vision_. Springer, 2014, pp. 740–755.
  * [5] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3d shapenets: A deep representation for volumetric shapes,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 1912–1920.
  * [6] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 652–660.
  * [7] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _arXiv preprint arXiv:1706.02413_ , 2017.
  * [8] R. Li, X. Li, P.-A. Heng, and C.-W. Fu, “Pointaugment: an auto-augmentation framework for point cloud classification,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 6378–6387.
  * [9] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “Autoaugment: Learning augmentation strategies from data,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 113–123.
  * [10] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” _arXiv preprint arXiv:1710.09412_ , 2017.
  * [11] V. Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, D. Lopez-Paz, and Y. Bengio, “Manifold mixup: Better representations by interpolating hidden states,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 6438–6447.
  * [12] Y. Chen, V. T. Hu, E. Gavves, T. Mensink, P. Mettes, P. Yang, and C. G. Snoek, “Pointmixup: Augmentation for point clouds,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16_. Springer, 2020, pp. 330–345.
  * [13] D. Lee, J. Lee, J. Lee, H. Lee, M. Lee, S. Woo, and S. Lee, “Regularization strategy for point cloud via rigidly mixed sample,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 15 900–15 909.
  * [14] S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable features,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6023–6032.
  * [15] S. V. Sheshappanavar, V. V. Singh, and C. Kambhamettu, “Patchaugment: Local neighborhood augmentation in point cloud classification,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 2118–2127.
  * [16] S. Kim, S. Lee, D. Hwang, J. Lee, S. J. Hwang, and H. J. Kim, “Point cloud augmentation with weighted local transformations,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 548–557.
  * [17] E. A. Nadaraya, “On estimating regression,” _Theory of Probability & Its Applications_, vol. 9, no. 1, pp. 141–142, 1964.
  * [18] G. S. Watson, “Smooth regression analysis,” _Sankhyā: The Indian Journal of Statistics, Series A_ , pp. 359–372, 1964.
  * [19] Y. Wang, A. Jacobson, J. Barbič, and L. Kavan, “Linear subspace design for real-time shape deformation,” _ACM Transactions on Graphics (TOG)_ , vol. 34, no. 4, pp. 1–11, 2015.
  * [20] M. Liu, M. Sung, R. Mech, and H. Su, “Deepmetahandles: Learning deformation meta-handles of 3d meshes with biharmonic coordinates,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 12–21.
  * [21] Z. Tang, Y. Gao, L. Karlinsky, P. Sattigeri, R. Feris, and D. Metaxas, “Onlineaugment: Online data augmentation with less domain knowledge,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16_. Springer, 2020, pp. 313–329.
  * [22] X. Zhang, Q. Wang, J. Zhang, and Z. Zhong, “Adversarial autoaugment,” _arXiv preprint arXiv:1912.11188_ , 2019.
  * [23] C. Xie, M. Tan, B. Gong, J. Wang, A. L. Yuille, and Q. V. Le, “Adversarial examples improve image recognition,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 819–828.
  * [24] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” _CoRR_ , vol. abs/1412.6572, 2015.
  * [25] A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” _ArXiv_ , vol. abs/1611.01236, 2017.
  * [26] P. Y. Simard, D. Steinkraus, J. C. Platt _et al._ , “Best practices for convolutional neural networks applied to visual document analysis.” in _Icdar_ , vol. 3, no. 2003, 2003.
  * [27] R. G. Lopes, D. Yin, B. Poole, J. Gilmer, and E. D. Cubuk, “Improving robustness without sacrificing accuracy with patch gaussian augmentation,” _ArXiv_ , vol. abs/1906.02611, 2019.
  * [28] T. Devries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” _ArXiv_ , vol. abs/1708.04552, 2017.
  * [29] D. Ho, E. Liang, I. Stoica, P. Abbeel, and X. Chen, “Population based augmentation: Efficient learning of augmentation policy schedules,” _ArXiv_ , vol. abs/1905.05393, 2019.
  * [30] D. Lee, H. Park, T. X. Pham, and C. D. Yoo, “Learning augmentation network via influence functions,” _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 10 958–10 967, 2020.
  * [31] B. Baker, O. Gupta, N. Naik, and R. Raskar, “Designing neural network architectures using reinforcement learning,” _ArXiv_ , vol. abs/1611.02167, 2017.
  * [32] A. Antoniou, A. J. Storkey, and H. Edwards, “Data augmentation generative adversarial networks,” _ArXiv_ , vol. abs/1711.04340, 2017.
  * [33] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in _NIPS_ , 2014.
  * [34] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” _Acm Transactions On Graphics (tog)_ , vol. 38, no. 5, pp. 1–12, 2019.
  * [35] C.-Q. Huang, F. Jiang, Q.-H. Huang, X.-Z. Wang, Z.-M. Han, and W.-Y. Huang, “Dual-graph attention convolution network for 3-d point cloud classification,” _IEEE Transactions on Neural Networks and Learning Systems_ , pp. 1–13, 2022.
  * [36] J. Dong, Y. Cong, G. Sun, L. Wang, L. Lyu, J. Li, and E. Konukoglu, “Inor-net: Incremental 3-d object recognition network for point cloud representation,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 34, no. 10, pp. 6955–6967, 2023.
  * [37] Z. Du, H. Ye, and F. Cao, “A novel local-global graph convolutional method for point cloud semantic segmentation,” _IEEE Transactions on Neural Networks and Learning Systems_ , pp. 1–15, 2022.
  * [38] S. Li, Y. Liu, and J. Gall, “Rethinking 3-d lidar point cloud segmentation,” _IEEE Transactions on Neural Networks and Learning Systems_ , pp. 1–12, 2021.
  * [39] A.-D. Nguyen, S. Choi, W. Kim, J. Kim, H. Oh, J. Kang, and S. Lee, “Single-image 3-d reconstruction: Rethinking point cloud deformation,” _IEEE Transactions on Neural Networks and Learning Systems_ , pp. 1–15, 2022.
  * [40] A. Jacobson, I. Baran, J. Popović, and O. Sorkine-Hornung, “Bounded biharmonic weights for real-time deformation,” _Communications of the ACM_ , vol. 57, pp. 99 – 106, 2014.
  * [41] A. Jacobson, T. Weinkauf, and O. Sorkine-Hornung, “Smooth shape‐aware functions with controlled extrema,” _Computer Graphics Forum_ , vol. 31, 2012.
  * [42] Y. Wang, A. Jacobson, J. Barbi, and L. Kavan, “Linear subspace design for real-time shape deformation,” _ACM Transactions on Graphics (TOG)_ , vol. 34, pp. 1 – 11, 2015.
  * [43] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _ArXiv_ , vol. abs/1706.06083, 2018.
  * [44] C. Xie, Y. Wu, L. van der Maaten, A. L. Yuille, and K. He, “Feature denoising for improving adversarial robustness,” _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 501–509, 2019.
  * [45] H. Fan, H. Su, and L. J. Guibas, “A point set generation network for 3d object reconstruction from a single image,” _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2463–2471, 2017.
  * [46] W. Wang, D. Ceylan, R. Mech, and U. Neumann, “3dn: 3d deformation network,” _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 1038–1046, 2019.
  * [47] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” _ACM Transactions on Graphics (TOG)_ , 2019.
  * [48] J. Huang, H. Su, and L. Guibas, “Robust watertight manifold surface generation method for shapenet models,” _arXiv preprint arXiv:1802.01698_ , 2018.
  * [49] Y. Hu, T. Schneider, B. Wang, D. Zorin, and D. Panozzo, “Fast tetrahedral meshing in the wild,” _ACM Trans. Graph._ , vol. 39, no. 4, Jul. 2020.
  * [50] A. Jacobson and D. Panozzo, “libigl: prototyping geometry processing research in c++,” _SIGGRAPH Asia 2017 Courses_ , 2017.
  * [51] X. Yan, “Pointnet/pointnet++ pytorch,” _https://github.com/yanx27/ Pointnet_Pointnet2_pytorch_ , 2019.