# Cloud-RAIN: Point Cloud Analysis with Reflectional Invariance

Yiming Cui1 Lecheng Ruan2 Hang-Cheng Dong3 Qiang Li4 Zhongming Wu5 Tieyong
Zeng5 Feng-Lei Fan5∗ ∗Feng-Lei Fan<EMAIL_ADDRESS>is the
corresponding author.1Yiming Cui<EMAIL_ADDRESS>is with Department of
Electrical and Computer Engineering, University of Florida, Gainesville, FL,
USA.2Lecheng Ruan is with BIGAI, Beijing, China.3Hang-Cheng Dong is with
School of Instrumentation Science and Engineering, Harbin Institute of
Technology, Harbin, China.4Qiang Li is with Department of Electrical and
Computer Engineering, McMaster University, Hamilton, ON, Canada. 5Zhongming
Wu, Tieyong Zeng, and Feng-Lei Fan are with Department of Mathematics, The
Chinese University of Hong Kong, N. T. Hong Kong.

###### Abstract

The networks for point cloud tasks are expected to be invariant when the point
clouds are affinely transformed such as rotation and reflection. So far,
relative to the rotational invariance that has been attracting major research
attention in the past years, the reflection invariance is little addressed.
Notwithstanding, reflection symmetry can find itself in very common and
important scenarios, e.g., static reflection symmetry of structured streets,
dynamic reflection symmetry from bidirectional motion of moving objects (such
as pedestrians), and left- and right-hand traffic practices in different
countries. To the best of our knowledge, unfortunately, no reflection-
invariant network has been reported in point cloud analysis till now. To fill
this gap, we propose a framework by using quadratic neurons and PCA canonical
representation, referred to as Cloud-RAIN, to endow point Cloud models with
ReflectionAl INvariance. We prove a theorem to explain why Cloud-RAIN can
enjoy reflection symmetry. Furthermore, extensive experiments also corroborate
the reflection property of the proposed Cloud-RAIN and show that Cloud-RAIN is
superior to data augmentation. Our code is available at
https://github.com/YimingCuiCuiCui/Cloud-RAIN.

##  I Introduction

Recently, point clouds have been intensively studied due to their broad
applications as a basic representation of 3D models in autonomous driving and
robotics. Deep learning techniques are introduced into this field for either
direct analysis [1, 2, 3, 4] or indirect analysis represented by PointNet [5]
and its successors [6, 7]. As a point cloud is an unordered set, previous
direct analysis models focus on the invariance to sample permutation. However,
in addition to the invariance to point permutation, the networks are also
supposed to infer consistently when the point clouds are affinely transformed
such as translation, rotation, and reflection. While the bias caused by
translation can be counteracted easily by centralization, past years have
witnessed a plethora of works [8, 9, 10, 11] proposed to address the
invariance to rotation and achieved the encouraging success, e.g., rotation
data augmentation, spherical-related convolutions, explicit local rotation-
invariant features, and canonical poses.

Figure 1: In self-driving scenarios, front view of cars often exhibits
reflection symmetry, which could include (a) static reflection symmetry of
structured streets, (b) dynamic reflection symmetry from bidirectional motion
of moving objects (such as pedestrians), and (c) left- and right-hand traffic
practices in different countries (such as UK and US).

However, relative to the rotational symmetry, the reflection symmetry is
little addressed. Notwithstanding, reflection symmetry can find itself in very
common and important scenarios. Often, reflection symmetry appears due to the
symmetric city planning. Figure 1(a) shows one such example in autonomous
driving, where the driver’s front view exhibits reflection symmetry in well-
structured traffic scenes; Also, reflection symmetry is obtained by the
direction of movement. Figure 1(b) manifests that pedestrians crossing the
street to the left and to the right are symmetric in reflection. A devastating
car accident may happen if a self-driving car only identifies the pedestrians
walking along one direction and fails the opposite; In addition, reflectional
symmetry can be given rise by the left- and right-hand traffic practices in
different countries, as Figure 1(c) shows. Furthermore, we argue that
reflection symmetry is more common than rotation symmetry in some important
scenarios such as driving, where the environment around an autonomous vehicle
is unlikely to rotate severely but likely to exhibit reflection symmetry.
Therefore, reflectional invariance should be examined dedicatedly in a point
cloud model.

However, to the best of our knowledge, no reflection-invariant network has
been claimed in point cloud analysis till now. To deal with this issue, here
we shift our attention to the quadratic neurons and networks. Recently,
motivated by introducing neuronal diversity in deep learning, the so-called
quadratic neuron [12] was designed by using a simplified quadratic function to
replace the inner product in the conventional neuron. Hereafter, we call a
network made of quadratic neurons a quadratic network and a network made of
conventional neurons a conventional network. Why can quadratic networks
address reflectional symmetry? Here, we prove that a quadratic network can
approximate a reflectionally symmetric function well compared to conventional
networks. The idea is that due to the invariance to the sign flipping of the
input, the power term in the quadratic neuron can directly deal with the
reflection of the input with respect to axis-aligned planes. Next, we combine
quadratic networks with PCA to build a point Cloud method with ReflectionAl
INvariance, referred to as Cloud-RAIN. While quadratic neurons deal with the
reflectional symmetry due to the sign flipping, the PCA canonical transform
derives canonical poses of a point cloud, which can transform the problem of
reflection across arbitrary planes into a problem of sign flipping. Thus,
Cloud-RAIN can tackle a general arbitrary-plane reflectional invariance
problem. Our contributions are threefold:

• We identify reflection invariance as an essential yet overlooked concern in
point cloud analysis.

• We prove a theorem showing that a quadratic network is powerful in
approximating a reflectionally-invariant function. Then, we propose Cloud-RAIN
that combines quadratic neurons and the PCA transform for point cloud
applications.

• Empirically, Cloud-RAIN is invariant to reflection and much better than
reflection data augmentation.

##  II Related Works

Deep Learning-based Point Cloud Analysis. Currently, deep learning methods for
point cloud analysis can be divided into two classes: indirect and direct.
Indirect methods transform point clouds either to multi-view images or
volumetric data followed by a 2D CNN or a 3D CNN, including VoxelNet [1],
SPLATNet [13], and PCNN [4]. In contrast, direct methods represented by
PointNet [5], PointNet++ [6], RS-CNN [14], PointWeb [15], KPConv [16],
PointConv [17], and so on [18, 19, 20, 21]. Direct methods are able to capture
extrinsic features like 3D corners, edges, and local shape parts.
Concurrently, graph-based methods build a graph by considering each point in a
point cloud as a vertex and generating directed edges based on the vertices’
neighborships. Then, graph networks perform feature learning in spatial or
spectral domains. Representative graph models include DGCNN [22], LDGCNN [23],
DeepGCNs[24], PointGCN [25], GAC-Net [26], and so on. Recently, Transformers
[27] were more and more introduced in the point cloud tasks [28, 29, 30, 31].
One common concern regarding deep models is that these models are vulnerable
to symmetric transforms when data augmentation is not applied.

Rotation Symmetry. Intensive investigations have been conducted on the
rotation-invariant property. These studies are roughly divided into three
categories: i) rotation-invariant operations [32, 33, 9, 10, 34, 35], which
designs rotation-invariant convolutions or attentions in a network to achieve
invariance; ii) explicit local rotation-invariant features [11, 8, 36, 37,
38], which handcrafts rotationally invariant features based on the intrinsic
geometric relations of point clouds such as relative distances and angles; and
iii) canonical poses [39, 40, 41], which transforms point cloud data to their
PCA-based canonical poses that preserve shape information to achieve
rotational invariance. We think that reflection symmetry is more important
than rotation symmetry in some important applications such as driving, where
buildings, views, and pedestrians around a normally-moving car are unlikely to
encounter severe rotation but very likely to exhibit reflection symmetry.
However, unfortunately, endowing a point cloud model with the reflection-
invariant property was little studied.

Polynomial Neural Networks. The story of polynomial neurons begins with the
Group Method of Data Handling (GMDH [42]), which gradually learns higher-order
terms as a feature extractor. Furthermore, the so-called higher-order unit was
proposed in [43, 44] by adding a nonlinear activation $\sigma(\cdot)$ after
GMDH: $y=\sigma(Y(\mathbf{x}_{1},\cdots,\mathbf{x}_{n}))$. To balance the
power of high-order units and parametric efficiency, Milenkoiv et al. [45]
only utilized linear and quadratic terms and proposed to use the annealing
technique to find optimal parameters. Recently, higher-order, particularly
quadratic units were intensively studied [46, 47, 48, 12, 49] from the
perspective of neuronal diversity. The neurons in [47, 50, 51, 49] are a
special case of [52]. [47] excludes the power term, and [50] only has the
power term, which renders an incomplete quadratic representation. This work
intends to make the first attempt to introduce quadratic neurons into point
cloud analysis to realize reflection invariance.

##  III Power of Quadratic Networks in Reflectional Invariance

Here, we illustrate why quadratic neurons are suitable for point cloud tasks
requiring reflection invariance. We provide a theorem that a quadratic network
is powerful in approximating a symmetric function, e.g., using much fewer
parameters than a conventional one. This suggests that for tasks requiring a
model to be reflectionally symmetric, quadratic networks are favored over
conventional ones, which makes the employment of quadratic networks in
reflectional invariance well justified. The core idea of this theorem is that
the power term in a quadratic neuron are straightforwardly reflection-
invariant, while it is costly for the conventional to learn power terms.

Quadratic neurons. Given an input vector
$\mathbf{f}=(f_{1},f_{2},\cdots,f_{n})$, a conventional neuron computes an
inner product plus a bias $b$ followed by a nonlinear activation $\sigma$:

$\sigma(\mathbf{f}^{\top}\mathbf{w}+b),$ (1)

while a quadratic neuron computes two inner products and one power term and
then feeds them into a nonlinear activation function. Mathematically, its
inner-working is

$\sigma\big{(}(\mathbf{f}^{\top}\mathbf{w}_{1}+b_{1})(\mathbf{f}^{\top}\mathbf{w}_{2}+b_{2})+(\mathbf{f}\circ\mathbf{f})^{\top}\mathbf{w}_{3}+b_{3}\big{)},$
(2)

where $\circ$ is a Hadamard product, $\mathbf{w}_{1}$, $\mathbf{w}_{2}$, and
$\mathbf{w}_{3}$ are parameters. Compared to the designs of quadratic neurons
in [46, 53] with a complexity $\mathcal{O}(n^{2})$, Eq. 2 is much more compact
with the complexity of $\mathcal{O}(3n)$, scaling linearly with the number of
features $n$.

######  Definition 1 (reflectionally invariant function).

We call a function
$h(\mathbf{x})=h(\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{d}):\left(\mathbb{R}^{N}\right)^{d}\to\mathbb{R}$
reflectionally invariant, if for any $\mathbf{\pi}\in\\{-1,1\\}^{d}$,
$h(\mathbf{x})$ satisfies

$h(\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{d})=h(\mathbf{\pi}_{1}\mathbf{x}_{1},\mathbf{\pi}_{2}\mathbf{x}_{2},\cdots,\mathbf{\pi}_{d}\mathbf{x}_{d}),$
(3)

where $N$ is the number of samples, $d$ is the number of features,
$\mathbf{x}_{i}\in\mathbb{R}^{N}$, and $\prod_{i=1}^{d}\pi_{i}=-1$.

When $\prod_{i=1}^{d}\pi_{i}=-1$,
$\mathbf{\pi}_{1}\mathbf{x}_{1},\mathbf{\pi}_{2}\mathbf{x}_{2},\cdots,\mathbf{\pi}_{d}\mathbf{x}_{d}$
is a reflection of $\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{d}$. When
$\prod_{i=1}^{d}\pi_{i}=1$,
$\mathbf{\pi}_{1}\mathbf{x}_{1},\mathbf{\pi}_{2}\mathbf{x}_{2},\cdots,\mathbf{\pi}_{d}\mathbf{x}_{d}$
is a rotation of $\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{d}$. It is
also important for a model to maintain rotation invariance over samples.
However, previous literature has studied rotation invariance. Therefore,
rotation invariance is not the focus of our draft.

######  Proposition 1 (Proposition 2 in [54]).

Given $M>0$ and $\epsilon\in(0,1)$, there is a ReLU network with two inputs
that implements a function
$\widetilde{\times}:\mathbb{R}^{2}\rightarrow\mathbb{R}$ so that a) for any
inputs $x,y$, if $|x|\leq M$ and $|y|\leq M$, then
$|\widetilde{\times}(x,y)-xy|\leq\epsilon$; b) if $x=0$ or $y=0$, then
$\widetilde{\times}(x,y)=0$; c) the number of computation units in this ReLU
network is $O(\ln(1/\epsilon))$.

######  Proposition 2 (Lemma 1 in [55]).

There exists a one-hidden-layer quadratic ReLU network that can implement a
mapping $\tilde{\times}:\mathbb{R}^{2}\rightarrow\mathbb{R}$, satisfying that
i) $\tilde{\times}(x,y)=xy$ ; ii) $\tilde{\times}(x,y)$ has 2 quadratic
neurons and accordingly 4 parameters.

######  Theorem 1.

Let $f:\left(\mathbb{R}^{N}\right)^{d}\rightarrow\mathbb{R}$ be a continuously
differentiable and totally symmetric function, where $\Omega$ is a compact
subset of $\mathbb{R}^{d}$. Let $\delta>0$, then there exists
$\tilde{\mathbf{F}}:\left(\mathbb{R}^{N}\right)^{d}\rightarrow\mathbb{R}$,
such that for any $X$, we have

$\sup_{X}\left|f(X)-\tilde{\mathbf{F}}(X)\right|\leq\delta,$ (4)

where
$\tilde{\mathbf{F}}(X)=\sum_{l=1}^{L}c_{l}\left(\widetilde{\times}_{i=1}^{N}\widetilde{\times}_{\alpha=1}^{d}X_{i,\alpha}^{\phi_{i,\alpha}(l)}\right)$,
and $L$ is a fundtion of $\delta$, $c_{l}$ is a coefficient for different $l$,
and $\phi_{i,\alpha}(l)$ is the power function dependent on $i$ and $\alpha$.
To represent $\tilde{\mathbf{F}}$ in the sense of the max norm, a conventional
network needs $NdLO(\ln(1/\delta))$ parameters, while a quadratic network only
needs $4NdL$ parameters.

The sketch of proof: We first show a polynomial approximation scheme to the
symmetric function $f$, which is $F(\mathbf{x})$. Then, we construct the
quadratic and the conventional networks to represent $F(\mathbf{x})$,
respectively, with a comparison of the approximation error and the number of
parameters.

###### Proof.

Step 1. We first consider the case of $N=1$. According to the Stone-
Weierstrass theorem, a polynomial can approximate any continuous function
defined on a closed interval. Similarly, based on the fundamental theorem of
symmetric polynomials [56], $f$ can be naturally approximated by a polynomial
of elementary symmetric polynomials made of power terms. Mathematically, for
any symmetric polynomial $P$, we have
$P(X)=Q\left(e_{1}(X),\ldots,e_{d}(X)\right)$, where $Q$ is some polynomial,
and $e_{k}$ is as the following:

$e_{k}(X)=\sum_{1\leq j_{1}<j_{2}<\cdots<j_{k}\leq
d}x_{j_{1}}^{2}x_{j_{2}}^{2}\cdots x_{j_{k}}^{2}.$ (5)

To extend the result to the case of $N>1$, the monomial takes the form

$F(X)=\prod_{i=1}^{d}\prod_{\alpha=1}^{N}X_{i,\alpha}^{\phi_{i,\alpha}},$ (6)

where $\phi_{i,\alpha}$ is an even integer, which ensures that this monomial
is a reflectionally invariant function. $\phi_{i,\alpha}$ is determined by the
target function and the index $\alpha,i$. The approximation using a symmetric
polynomial can be a linear combination of $L$ symmetrized monomials.

$\mathbf{F}(X)=\sum_{l=1}^{L}c_{l}F_{i}^{(l)}(X)=\sum_{l=1}^{L}c_{l}\left(\prod_{i=1}^{d}\prod_{\alpha=1}^{N}X_{i,\alpha}^{\phi_{i,\alpha}(l)}\right),$
(7)

where $L$ is a function of $\delta$, such that

$\sup_{X}\left|f(X)-\mathbf{F}(X)\right|\leq\delta/2.$ (8)

Step 2. Now, let us use a conventional and a quadratic neural network to
approximate $\mathbf{F}(X)$.

Approximation by a conventional network: We can approximate this product by a
conventional neural network with the help of Proposition 1. Let
$\widetilde{\times}$ be the approximate multiplication. Applying
$\widetilde{\times}$ in a compositional manner,

$\underset{X}{\sup}\left|\widetilde{\times}_{i=1}^{N}\widetilde{\times}_{\alpha=1}^{d}X_{i,\alpha}^{\phi_{i,\alpha}}-\prod_{i=1}^{N}\prod_{\phi=1}^{d}X_{i,\alpha}^{\phi_{i,\alpha}}\right|\leq\delta/2.$
(9)

Let
$\tilde{\mathbf{F}}=\sum_{l=1}^{L}c_{l}\left(\widetilde{\times}_{i=1}^{N}\widetilde{\times}_{\alpha=1}^{d}X_{i,\alpha}^{\phi_{i,\alpha}(l)}\right)$,
it can also approximate $f(X)$ based on the triangle inequality:

$\sup_{X}\left|f(X)-\tilde{\mathbf{F}}(X)\right|\leq\delta.$ (10)

The number of parameters used to construct $\tilde{\mathbf{F}}$ is
$NdLO(\ln(1/\delta))$.

Approximation by a quadratic network: A quadratic neuron can precisely
approximate a product function without incorporating extra error like a
conventional network. To express
$\tilde{\mathbf{F}}=\sum_{l=1}^{L}\widetilde{\times}_{i=1}^{N}\widetilde{\times}_{\alpha=1}^{d}X_{i,\alpha}^{\phi_{i,\alpha}(l)}$,
the number of parameters used in a quadratic network is $4NdL$. ∎

Remark 1. The above constructive theorem informs us that a conventional
network is hard to learn reflectionally invariant functions, i.e., much more
parameters, but it is much easier for quadratic networks because quadratic
neurons can easily represent a power term and a multiplication operation. Our
main theorem heavily relies on [57]. But our highlight is to permute the
dimension, while their result permutes samples. Moreover, we intend to use
quadratic neurons to reimplement this scheme to show their superiority in
representing reflectional symmetry. One may ask if there are other kinds of
constructs. Actually, one can also partition the domain into a lattice with a
small grid size to approximate $f(X)$ instead of using the linear combination.
But still, the quadratic network will show the advantages because the power
term in the quadratic neuron is essentially suitable for reflectional
invariance.

##  IV Cloud-RAIN

Inspired by the above encouraging theoretical result, here we propose a point
Cloud method which enjoys the ReflectionAl INvariance (Cloud-RAIN) and whose
spotlight is the incorporation of quadratic neurons. In addition, we also
equip Cloud-RAIN with the PCA canonical transform, because the quadratic
neuron alone can only tackle reflection across axis-aligned planes. When
combining the PCA canonical transform, quadratic neurons can deal with
reflection with respect to arbitrary planes.

###  IV-A Quadratic Neurons

Given a set of points $\mathbf{X}\in\mathbb{R}^{N\times C}$, where $N$
represents the number of points, and $F$ is the number of features. Usually,
$C=3+c$ denotes $(x,y,z)$ position in the geometric space, and the feature
number (e.g., $c=3$ for points only with color; $c=6$ for those with color and
normals). Existing point cloud analysis methods usually focus on how to
engineer $\mathbf{w}$ or $\mathbf{f}$ to fulfill the needs and characters of
the problems. For example, RS-CNN [14] restricts $\mathbf{w}$ to learn the
relations between $p_{i}$ and $p_{j}$ in geometric space and extends a regular
rigid CNN to the irregular configuration. DGCNN [22] proposes the EdgeConv
that operates on the edges between the central point and its neighbors and
treats $\mathbf{f}$ as the edge features. Point Transformer [28] uses the
vector self-attention operator to aggregate local features and the subtraction
relation to generate $\mathbf{w}$. Despite these arts, they all use
conventional neurons which integrate $\mathbf{w}$ and $\mathbf{f}$ in a linear
fashion, as Eq. (1) shows.

Here, the proposed Cloud-RAIN replaces the simple linear relation between
$\mathbf{f}$ and $\mathbf{w}$ with a quadratic interaction by quadratic
neurons. Theorem 1 shows that quadratic neurons are powerful in learning a
reflectionally invariant function.

###  IV-B PCA Canonical Transform

Note
$(\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{D})\to(\mathbf{\pi}_{1}\mathbf{x}_{1},\mathbf{\pi}_{2}\mathbf{x}_{2},\cdots,\mathbf{\pi}_{D}\mathbf{x}_{D})$
is a special case of reflection symmetry, where the planes of symmetry align
with the axes. Here, we introduce PCA canonical transform to empower the
quadratic aggregation to deal with reflection across an arbitrary plane. We
first show how the PCA transform is performed on the point cloud so as to
obtain its canonical representation. Without loss of generality, let the point
cloud be $\mathbf{X}\in\mathbb{R}^{N\times 3}$,
$\mathbf{X}_{i}\in\mathbb{R}^{3\times 1}$ be the $i$-th point of $\mathbf{X}$,
and
$\overline{\mathbf{X}}=\large(\sum_{i=1}^{N}\mathbf{X}_{i}\large)/N\in\mathbb{R}^{3\times
1}$ be the geometric center of $\mathbf{X}$, we perform the PCA decomposition
for $\mathbf{X}$ as follows:

$\frac{\sum_{i=1}^{N}\left(\mathbf{X}_{i}-\overline{\mathbf{X}}\right)\left(\mathbf{X}_{i}-\overline{\mathbf{X}}\right)^{T}}{N}=\mathbf{V}\boldsymbol{\Lambda}\mathbf{V}^{\top},$
(11)

where $\mathbf{V}\in\mathbb{R}^{3\times 3}$ is the matrix composed of
eigenvectors ($\mathbf{v}_{1},\mathbf{v}_{2},\mathbf{v}_{3}$) and
$\boldsymbol{\Lambda}=\operatorname{diag}\left(\lambda_{1},\lambda_{2},\lambda_{3}\right)$
are the corresponding eigenvalues. By multiplying the point cloud with the
eigenvector matrix, we obtain the canonical representation of the point cloud
$\mathbf{X}$ as $\mathbf{X}_{\text{can }}=\mathbf{XV}$.

The reflection-invariant property of $\mathbf{X}_{\text{can }}$ is illustrated
in the following: Applying a random reflection matrix
$\mathbf{F}\in\mathbb{R}^{3\times 3}$ on the point cloud $\mathbf{X}$ to get a
reflected point cloud $\mathbf{XF}^{\top}$ and performing PCA decomposition
for $\mathbf{FX}$ as shown in Eq. (11), we have the following:

$\displaystyle\frac{\sum\left(\mathbf{FX}_{i}-\mathbf{F}\overline{\mathbf{X}}\right)\left(\mathbf{FX}_{i}-\mathbf{F}\overline{\mathbf{X}}\right)^{\top}}{N}$
(12) $\displaystyle=$
$\displaystyle\mathbf{F}\left(\frac{\sum\left(\mathbf{X}_{i}-\overline{\mathbf{X}}\right)\left(\mathbf{X}_{i}-\overline{\mathbf{X}}\right)^{\top}}{N}\right)\mathbf{F}^{\top}$
$\displaystyle=$
$\displaystyle(\mathbf{FV})\boldsymbol{\Lambda}(\mathbf{FV})^{\top},$

where $\mathbf{FV}$ is the eigenvector matrix of the reflected point cloud
$\mathbf{XF}^{\top}$. Therefore, the canonical representation
$\mathbf{XF}^{\top}$ is
$\left(\mathbf{XF}^{\top}\right)_{\text{can}}=\mathbf{X}\mathbf{F}^{T}\cdot\mathbf{FV}=\mathbf{XV}$.
Thus, the effect of the reflection is counteracted in $\mathbf{X}_{\text{can
}}$.

Although employing the canonical representation can realize the reflection
invariance with respect to arbitrary planes, it brings up the issue of sign
ambiguity. Given an eigenvector $\mathbf{v}$, both choosing $+\mathbf{v}$ and
$-\mathbf{v}$ fulfills the rule of PCA transform. What’s undesirable is we
don’t know if it is $+\mathbf{v}$ or $-\mathbf{v}$ being selected. Because
$\mathbf{V}=[\mathbf{v}_{1},\mathbf{v}_{2},\mathbf{v}_{3}]$, there are eight
possible combinations
($[\pm\mathbf{v}_{1},\pm\mathbf{v}_{2},\pm\mathbf{v}_{3}]$) that cause sign
ambiguity in the canonical representation of $\mathbf{X}_{\text{can }}$. This
is the intrinsic problem of using the canonical representation. However,
quadratic aggregation can overcome the sign ambiguities due to its power term
based on our earlier analyses.

Remark 2 (Theoretical guarantee). The canonical representation alone can deal
with reflection across the arbitrary plane but suffers sign ambiguity, while
the quadratic aggregation alone is invariant to the sign flipping but only
works for reflection across axis-aligned planes. It is the combination between
the canonical representation and quadratic aggregation that realizes the
invariance to reflection across arbitrary planes. The advantage of such a
combination is that it has a theoretical guarantee to be invariant to
reflection, which will be confirmed by our experiments later.

Dataset | S3DIS | ScanNet
---|---|---
Methods | mAcc | mIOU | mAcc | mIOU
PointNet[5] | 53.1 | 45.0 | 44.3 | 41.9
PointNet[5]+Ours | 56.5 | 48.1 | 45.6 | 43.7
PointNet++[6] | 62.0 | 53.9 | 53.4 | 54.4
PointNet++[6]+Ours | 64.4 | 56.0 | 57.3 | 57.6
DGCNN[22] | 57.0 | 49.3 | 48.0 | 46.7
DGCNN[22]+Ours | 61.9 | 50.6 | 51.0 | 49.8
LDGCNN[23] | 59.1 | 49.9 | 50.6 | 49.9
LDGCNN[23]+Ours | 61.4 | 50.6 | 51.6 | 51.8
GSNet[58] | 52.7 | 44.5 | 43.7 | 45.2
GSNet[58]+Ours | 54.5 | 46.2 | 44.9 | 46.4
ShellNet[59] | 58.3 | 49.5 | 49.7 | 49.0
ShellNet[59]+Ours | 61.2 | 50.4 | 50.8 | 51.7
PointMLP[20] | 80.3 | 70.1 | 68.9 | 69.3
PointMLP[20] \+ Ours | 81.5 | 70.9 | 70.1 | 70.5
PointMixer[60] | 80.5 | 70.7 | 69.7 | 70.6
PointMixer[60] \+ Ours | 81.7 | 71.8 | 70.9 | 71.8

TABLE I: Experiments on S3DIS and ScanNet.

Model |  $x$ axis |  $y$ axis |  $z$ axis |  $xyz$ axis
---|---|---|---|---
|  $\Delta$mAcc |  $\Delta$mIOU |  $\Delta$mAcc |  $\Delta$mIOU |  $\Delta$mAcc |  $\Delta$mIOU |  $\Delta$mAcc |  $\Delta$mIOU
PointNet [5] | $\downarrow 3.5\%$ | $\downarrow 4.1\%$ | $\downarrow 2.6\%$ | $\downarrow 3.1\%$ | $\downarrow 45.0\%$ | $\downarrow 43.8\%$ | $\downarrow 45.8\%$ | $\downarrow 44.2\%$
PointNet [5] \+ Ours | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$
DGCNN [22] | $\downarrow 3.8\%$ | $\downarrow 4.4\%$ | $\downarrow 3.5\%$ | $\downarrow 3.6\%$ | $\downarrow 44.3\%$ | $\downarrow 49.2\%$ | $\downarrow 45.3\%$ | $\downarrow 49.3\%$
DGCNN [22] \+ Ours | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$
PointMLP [20] | $\downarrow 4.3\%$ | $\downarrow 5.1\%$ | $\downarrow 3.9\%$ | $\downarrow 4.1\%$ | $\downarrow 64.7\%$ | $\downarrow 68.9\%$ | $\downarrow 66.1\%$ | $\downarrow 69.2\%$
PointMLP [20] \+ Ours | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$
PointMixer [60] | $\downarrow 4.9\%$ | $\downarrow 5.7\%$ | $\downarrow 4.4\%$ | $\downarrow 4.9\%$ | $\downarrow 65.1\%$ | $\downarrow 69.4\%$ | $\downarrow 66.5\%$ | $\downarrow 69.9\%$
PointMixer [60] \+ Ours | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$ | $\downarrow 0\%$

TABLE II: Performance drops on different reflection operations.

##  V Experiments

To validate the reflection invariance of the proposed Cloud-RAIN, we conduct
extensive experiments on point cloud semantic segmentation with multiple
recent networks as a backbone. Other supplementary results such as model
complexity analysis are put into the supplementary materials.

###  V-A Experimental Setups

We mainly use S3DIS [61] and ScanNet [62], the mainstream point cloud semantic
segmentation benchmarks, to evaluate our proposed methods. We use the widely-
adopted mean accuracy (mAcc) and IOU (mIOU) as evaluation metrics. For S3DIS,
following the protocol of previous work [22, 63, 64, 5, 59, 6], we train for
100 epochs with 4 Tesla V100 GPUs. The batch size is set to 32. Following the
standard practice, the raw input points are firstly grid-sampled to generate
$4,096$ points. Unless otherwise specified, we use scale and jitter as data
augmentation. For ScanNet, we train for 100 epochs with weight decay and batch
size set to 0.1 and 32, respectively. The number of input points is set to be
$8,192$ by sampling. Except for random jitter, the data augmentation of the
ScanNet is the same as that of the S3DIS. We re-implement the original models
for a fair comparison and only replace the linear aggregation with the
quadratic one without changing other elements of the model.

###  V-B Segmentation Results

We report the basic segmentation results in Table I. All compared models are
either classic models or recently published in flagship venues. The spotlight
is that our proposed Cloud-RAIN can universally boost the existing models’
performance by a reasonably large margin. In detail, integrated with our
Cloud-RAIN, the current models’ mAcc and mIOU scores are improved by at least
$1.8\%$ and $0.7\%$ on the S3DIS benchmark, respectively. For a simple model
like PointNet, our proposed Cloud-RAIN can boost the performance on mAcc and
mIOU by $3.4\%$ and $3.1\%$. On the ScanNet benchmark, the improvement over
ScanNet is not on par with that on the S3DIS dataset. However, our proposed
methods can still escalate the mAcc and mIOU scores by at least $1.0\%$
relative to the existing methods. We think that if normal features over the
ScanNet benchmark are considered, the improvement could be more significant.
Although reflection symmetry is the primary goal of this manuscript, we would
like to note to readers that the Cloud-RAIN is an effective drop-in
replacement to escalate the performance of the original models.

###  V-C Reflection Invariance Analysis

Now, we provide detailed experiments to analyze the reflection-invariance
properties of our proposed Cloud-RAIN. In the first part, we concentrate on
reflections across axis-aligned planes that are very common in realistic
scenarios such as symmetric street views. No PCA transform needs to be used
because quadratic aggregation alone can deal with such reflection operations.
In the second part, reflection across arbitrary planes is tested.

Reflection invariance (axis-aligned planes). To analyze the effect of data
reflection, we select PointNet, DGCNN, PointMLP, and PointMixer as examples to
conduct experiments on the S3DIS benchmark. In the experiment, we flip the
point clouds along the $x$, $y$, $z$, and $xyz$ axis, respectively, and test
the segmentation performance of the well-trained models with the reflected
inputs, accordingly. During the training process, reflection augmentation is
not applied for the original models and those integrated with our proposed
methods. The results are listed in Table II.

As seen in Table II, our proposed Cloud-RAIN can keep the performance for the
reflected inputs thanks to the effect of power terms. This is mathematically
guaranteed because power terms return the same value for the flipped inputs.
In contrast, the existing methods cannot withstand the input reflection,
evidenced by the dramatic performance drop. Especially when the reflection is
applied to the $z$ axis, the performance drop is devastating. This might be
because the dynamic range of positions along the $z$ axis is far more
extensive than that along the $x$ and $y$ axes. Thus, the destruction of
flipping the $z$ axis to the model is severer.

Figure 2: Comparison of the segmentation results with different reflections.
Figure 3: Comparison of the segmentation results from different models with
reflection across randomly-selected planes.

Figure 2 offers semantic segmentation results in different reflection
settings. For better visualization, we view the semantic segmentation results
from the same angle regardless of their reflection. An interesting finding
from Figure 2 is that when the point clouds are reflected along the $z$ axis,
the original model is handicapped in distinguishing the categories of ceiling
and floors. Many points with the ceiling category are misclassified into the
floor class, indicating that the model just learns a reflectionally asymmetric
mapping.

Model |  $\Delta$mAcc |  $\Delta$ mIOU
---|---|---
PointNet | $\downarrow 46.3\%(\pm 0.52\%)$ | $\downarrow 42.7(\pm 0.32\%)$
PointNet + Aug | $\downarrow 2.6\%(\pm 0.46\%)$ | $\downarrow 2.4\%(\pm 0.28\%)$
PointNet + PCA | $\downarrow 1.7\%(\pm 0.38\%)$ | $\downarrow 1.9\%(\pm 0.32\%)$
PointNet + Ours | $\downarrow 0\%$ | $\downarrow 0\%$
DGCNN | $\downarrow 45.9\%(\pm 0.22\%)$ | $\downarrow 40.4\%(\pm 0.18\%)$
DGCNN + Aug | $\downarrow 8.0\%(\pm 0.28\%)$ | $\downarrow 8.5\%(\pm 0.32\%)$
DGCNN + PCA | $\downarrow 6.8\%(\pm 0.22\%)$ | $\downarrow 7.7\%(\pm 0.26\%)$
DGCNN + Ours | $\downarrow 0\%$ | $\downarrow 0\%$
PointMLP | $\downarrow 66.9\%(\pm 0.22\%)$ | $\downarrow 69.8\%(\pm 0.16\%)$
PointMLP + Aug | $\downarrow 10.2\%(\pm 0.32\%)$ | $\downarrow 11.3\%(\pm 0.38\%)$
PointMLP + PCA | $\downarrow 7.4\%(\pm 0.20\%)$ | $\downarrow 8.6\%(\pm 0.22\%)$
PointMLP + Ours | $\downarrow 0\%$ | $\downarrow 0\%$
PointMixer | $\downarrow 67.1\%(\pm 0.38\%)$ | $\downarrow 70.0\%(\pm 0.42\%)$
PointMixer + Aug | $\downarrow 11.0\%(\pm 0.30\%)$ | $\downarrow 11.9\%(\pm 0.34\%)$
PointMixer + PCA | $\downarrow 8.6\%(\pm 0.26\%)$ | $\downarrow 9.4\%(\pm 0.28\%)$
PointMixer + Ours | $\downarrow 0\%$ | $\downarrow 0\%$
TABLE III: Performance of PointNet and DGCNN drops dramatically for reflection
across arbitrary planes.

Reflection invariance (arbitrary planes). We conduct experiments with
PointNet, DGCNN, PointMLP, and PointMixer on the S3DIS benchmark to study the
reflection across arbitrary planes. We randomly generate reflection planes,
across which we flip the point clouds. To contrast the effectiveness of the
proposed Cloud-RAIN, we apply data augmentation during the training process,
where the training data are flipped along randomly generated planes. In the
inference stage, we generate a plane randomly and flip the point clouds across
the plane. For reliability of results, we run $5$ experiments and calculate
the mean as well as the standard deviation. We also train the original models
aided by PCA transform. All those comparative results are listed in Table III.

From Table III, we notice that without reflection data augmentation, all
models suffer a severe performance loss for both mAcc and mIOU scores when
there is no training augmentation. Even when the models are trained with
randomly generated data augmentation, PointNet is still subjected to at least
a $2.0\%$ drop on both evaluation metrics, while DGCNN’s mAcc and mIOU scores
drop at least $8.0\%$. What is favorable is that the performance of our models
does not have any compromise with randomly generated reflection planes, which
agrees with our theoretical analyses that Cloud-RAIN has the theoretical
guarantee for reflection invariance. Furthermore, aided by PCA transform, the
existing models are still subjected to the performance drop, which means these
models cannot overcome the sign ambiguity given rise by PCA. This means that
quadratic aggregation is necessary for reflectional symmetry over arbitrary
planes.

We visualize and compare a representative example from the S3DIS with respect
to an arbitrary plane reflection as Figure 3. We sample $4,096$ points from
the point clouds of each room and reflect those sampled points across the same
randomly generated plane. Then, we merge the segmentation results to be the
original room. From Figure 3, we notice that when the test data are reflected
across an arbitrary plane, the models trained without data augmentation cannot
even accurately distinguish the categories like walls, ceilings, and so on.
When it comes to the model trained with reflection data augmentation, though
it is not severely affected by the arbitrary reflection during the inference
time, there is still a structural distortion in distinguishing the boundaries
compared with our proposed method, as highlighted in the red rectangle.

##  VI Conclusion

In this manuscript, we have identified the reflection invariance, alongside
the rotation invariance, as a practical concern in point cloud applications.
Then, we have derived a theorem suggesting that quadratic neurons can learn a
reflectionally symmetric function much easier than conventional neurons do.
Next, we have introduced quadratic neurons and PCA canonical transform to
prototype a reflectionally-invariant model. Lastly, extensive comparison
experiments show that embedding quadratic aggregation in the existing models
can not only strengthen the reflection invariance as our theoretical analyses
illustrate. In the future, more efforts can be invested in exploring
polynomial aggregation to unify rotational and reflectional invariance
ultimately.

## References

  * [1] Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud based 3d object detection,” in CVPR, 2018.
  * [2] J. Xie, G. Dai, F. Zhu, E. K. Wong, and Y. Fang, “Deepshape: Deep-learned shape descriptor for 3d shape retrieval,” TPAMI, vol. 39, no. 7, pp. 1335–1345, 2017.
  * [3] H. Su, S. Maji, E. Kalogerakis, and E. G. Learned-Miller, “Multi-view convolutional neural networks for 3d shape recognition,” in ICCV, 2015\.
  * [4] M. Atzmon, H. Maron, and Y. Lipman, “Point convolutional neural networks by extension operators,” TOG, vol. 37, no. 4, pp. 1–12, 2018.
  * [5] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in CVPR, 2017.
  * [6] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” NIPS, 2017.
  * [7] X. Liu, C. R. Qi, and L. J. Guibas, “Flownet3d: Learning scene flow in 3d point clouds,” in CVPR, 2019.
  * [8] X. Sun, Z. Lian, and J. Xiao, “Srinet: Learning strictly rotation-invariant representations for point cloud classification and segmentation,” in ACM Multimedia, 2019.
  * [9] C. Esteves, C. Allen-Blanchette, A. Makadia, and K. Daniilidis, “Learning so (3) equivariant representations with spherical cnns,” in ECCV, 2018.
  * [10] Z. Zhang, B.-S. Hua, D. W. Rosen, and S.-K. Yeung, “Rotation invariant convolutions for 3d point clouds deep learning,” in 3DV, 2019.
  * [11] X. Li, R. Li, G. Chen, C.-W. Fu, D. Cohen-Or, and P.-A. Heng, “A rotation-invariant framework for deep point cloud analysis,” TVCG, 2021\.
  * [12] F. Fan, W. Cong, and G. Wang, “A new type of neurons for machine learning,” International journal for numerical methods in biomedical engineering, vol. 34, no. 2, p. e2920, 2018.
  * [13] H. Su, V. Jampani, D. Sun, S. Maji, E. Kalogerakis, M.-H. Yang, and J. Kautz, “SPLATNet: Sparse lattice networks for point cloud processing,” in CVPR, 2018.
  * [14] Y. Liu, B. Fan, S. Xiang, and C. Pan, “Relation-shape convolutional neural network for point cloud analysis,” in CVPR, 2019.
  * [15] H. Zhao, L. Jiang, C.-W. Fu, and J. Jia, “PointWeb: Enhancing local neighborhood features for point cloud processing,” in CVPR, 2019.
  * [16] H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in ICCV, 2019.
  * [17] W. Wu, Z. Qi, and L. Fuxin, “Pointconv: Deep convolutional networks on 3d point clouds,” in CVPR, 2019.
  * [18] Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, N. Trigoni, and A. Markham, “Randla-net: Efficient semantic segmentation of large-scale point clouds,” in CVPR, 2020.
  * [19] L. Tang, Y. Zhan, Z. Chen, B. Yu, and D. Tao, “Contrastive boundary learning for point cloud segmentation,” in CVPR, 2022.
  * [20] X. Ma, C. Qin, H. You, H. Ran, and Y. Fu, “Rethinking network design and local geometry in point cloud: A simple residual MLP framework,” in ICLR, 2022\.
  * [21] D. Nie, R. Lan, L. Wang, and X. Ren, “Pyramid architecture for multi-scale processing in point cloud segmentation,” in CVPR, 2022.
  * [22] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” TOG, vol. 38, no. 5, pp. 1–12, 2019.
  * [23] K. Zhang, M. Hao, J. Wang, C. W. de Silva, and C. Fu, “Linked dynamic graph cnn: learning on point cloud via linking hierarchical features,” arXiv:1904.10014 [cs], Apr. 2019.
  * [24] G. Li, M. Müller, A. Thabet, and B. Ghanem, “Deepgcns: Can gcns go as deep as cnns?,” in ICCV, 2019.
  * [25] Y. Zhang and M. Rabbat, “A graph-cnn for 3d point cloud classification,” in ICASSP, 2018.
  * [26] L. Wang, Y. Huang, Y. Hou, S. Zhang, and J. Shan, “Graph attention convolution for point cloud semantic segmentation,” in CVPR, 2019.
  * [27] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” NIPS, 2017\.
  * [28] H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V. Koltun, “Point transformer,” in ICCV, 2021.
  * [29] X. Lai, J. Liu, L. Jiang, L. Wang, H. Zhao, S. Liu, X. Qi, and J. Jia, “Stratified transformer for 3d point cloud segmentation,” in CVPR, 2022\.
  * [30] C. Park, Y. Jeong, M. Cho, and J. Park, “Fast point transformer,” in CVPR, 2022.
  * [31] C. Zhang, H. Wan, X. Shen, and Z. Wu, “Patchformer: An efficient point transformer with patch attention,” in CVPR, 2022.
  * [32] A. Poulenard, M.-J. Rakotosaona, Y. Ponty, and M. Ovsjanikov, “Effective rotation-invariant point cnn with spherical harmonics kernels,” in 3DV, 2019.
  * [33] Y. Rao, J. Lu, and J. Zhou, “Spherical fractal convolutional neural networks for point cloud recognition,” in CVPR, 2019.
  * [34] F. Fuchs, D. Worrall, V. Fischer, and M. Welling, “Se (3)-transformers: 3d roto-translation equivariant attention networks,” NIPS, 2020.
  * [35] C. Deng, O. Litany, Y. Duan, A. Poulenard, A. Tagliasacchi, and L. J. Guibas, “Vector neurons: A general framework for so (3)-equivariant networks,” in ICCV, 2021.
  * [36] C. Chen, G. Li, R. Xu, T. Chen, M. Wang, and L. Lin, “Clusternet: Deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis,” in CVPR, 2019.
  * [37] J. Xu, X. Tang, Y. Zhu, J. Sun, and S. Pu, “Sgmnet: Learning rotation-invariant point cloud representations via sorted gram matrix,” in ICCV, 2021.
  * [38] R. Yu, X. Wei, F. Tombari, and J. Sun, “Deep positional and relational feature learning for rotation-invariant point cloud analysis,” in ECCV, 2020.
  * [39] K. Fujiwara and T. Hashimoto, “Neural implicit embedding for point cloud analysis,” in CVPR, 2020.
  * [40] Z. Xiao, H. Lin, R. Li, L. Geng, H. Chao, and S. Ding, “Endowing deep 3d models with rotation invariance based on principal component analysis,” in ICME, 2020.
  * [41] F. Li, K. Fujiwara, F. Okura, and Y. Matsushita, “A closer look at rotation-invariant deep point cloud analysis,” in ICCV, 2021.
  * [42] A. G. Ivakhnenko, “Polynomial theory of complex systems,” TSMC, no. 4, pp. 364–378, 1971.
  * [43] T. Poggio, “On optimal nonlinear associative recall,” Biological Cybernetics, vol. 19, no. 4, pp. 201–209, 1975.
  * [44] C. L. Giles and T. Maxwell, “Learning, invariance, and generalization in high-order neural networks,” Applied optics, vol. 26, no. 23, pp. 4972–4978, 1987.
  * [45] S. Milenkovic, Z. Obradovic, and V. Litovski, “Annealing based dynamic learning in second-order neural networks,” in ICNN, 1996.
  * [46] G. Zoumpourlis, A. Doumanoglou, N. Vretos, and P. Daras, “Non-linear convolution filters for cnn-based learning,” in ICCV, 2017.
  * [47] N. Tsapanos, A. Tefas, N. Nikolaidis, and I. Pitas, “Neurons with paraboloid decision boundaries for improved neural network classification performance,” TNNLS, vol. 30, no. 1, pp. 284–294, 2018.
  * [48] G. G. Chrysos, S. Moschoglou, G. Bouritsas, J. Deng, Y. Panagakis, and S. Zafeiriou, “Deep polynomial neural networks,” TPAMI, vol. 44, no. 8, pp. 4021–4034, 2021.
  * [49] Z. Xu, F. Yu, J. Xiong, and X. Chen, “Quadralib: A performant quadratic neural network library for architecture optimization and design exploration,” MLSys, 2022.
  * [50] M. Goyal, R. Goyal, and B. Lall, “Improved polynomial neural networks with normalised activations,” in IJCNN, 2020.
  * [51] J. Bu and A. Karpatne, “Quadratic residual networks: A new class of neural networks for solving forward and inverse problems in physics involving pdes,” in SDM, 2021.
  * [52] F. Fan, H. Shan, M. K. Kalra, R. Singh, G. Qian, M. Getzin, Y. Teng, J. Hahn, and G. Wang, “Quadratic autoencoder (q-ae) for low-dose ct denoising,” TMI, vol. 39, no. 6, pp. 2035–2050, 2019.
  * [53] Y. Jiang, F. Yang, H. Zhu, D. Zhou, and X. Zeng, “Nonlinear cnn: improving cnns with quadratic convolutions,” Neural Computing and Applications, vol. 32, no. 12, pp. 8507–8516, 2020.
  * [54] D. Yarotsky, “Error bounds for approximations with deep relu networks,” Neural Networks, vol. 94, pp. 103–114, 2017.
  * [55] F.-L. Fan, H.-C. Dong, Z. Wu, L. Ruan, T. Zeng, Y. Cui, and J.-X. Liao, “One neuron saved is one neuron earned: On parametric efficiency of quadratic networks,” arXiv preprint arXiv:2303.06316, 2023.
  * [56] I. G. Macdonald, Symmetric functions and Hall polynomials.  Oxford university press, 1998.
  * [57] J. Han, Y. Li, L. Lin, J. Lu, J. Zhang, and L. Zhang, “Universal approximation of symmetric and anti-symmetric functions,” Communications in Mathematical Sciences, vol. 20, no. 5, pp. 1397–1408, 2022.
  * [58] M. Xu, Z. Zhou, and Y. Qiao, “Geometry sharing network for 3d point cloud classification and segmentation,” in AAAI, 2020.
  * [59] Z. Zhang, B.-S. Hua, and S.-K. Yeung, “Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics,” in ICCV, 2019.
  * [60] J. Choe, C. Park, F. Rameau, J. Park, and I. S. Kweon, “Pointmixer: Mlp-mixer for point cloud understanding,” in ECCV, 2022.
  * [61] I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese, “3d semantic parsing of large-scale indoor spaces,” in CVPR, 2016.
  * [62] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in CVPR, 2017.
  * [63] Y. Cui, X. Liu, H. Liu, J. Zhang, A. Zare, and B. Fan, “Geometric attentional dynamic graph convolutional neural networks for point cloud analysis,” Neurocomputing, vol. 432, pp. 300–310, 2021.
  * [64] Y. Wang and T. Derr, “Tree decomposed graph neural network,” in CIKM, 2021\.