# Self-organized stationary patterns in networks of bistable chemical
reactions

Nikos E. Kouvaris∗,a aDepartment of Physics, University of Barcelona, Martí i
Franquès 1, 08028 Barcelona, Spain
bDepartment of Chemistry, Saint Louis University, 3501 Laclede Ave., St.
Louis, Missouri 63103, USA
cDepartment of Physical Chemistry, Fritz Haber Institute of the Max Planck
Society, Faradayweg 4-6, D-14195 Berlin, Germany
<EMAIL_ADDRESS>Michael Sebekb aDepartment of Physics, University of
Barcelona, Martí i Franquès 1, 08028 Barcelona, Spain
bDepartment of Chemistry, Saint Louis University, 3501 Laclede Ave., St.
Louis, Missouri 63103, USA
cDepartment of Physical Chemistry, Fritz Haber Institute of the Max Planck
Society, Faradayweg 4-6, D-14195 Berlin, Germany
<EMAIL_ADDRESS>Alexander S. Mikhailovc aDepartment of Physics,
University of Barcelona, Martí i Franquès 1, 08028 Barcelona, Spain
bDepartment of Chemistry, Saint Louis University, 3501 Laclede Ave., St.
Louis, Missouri 63103, USA
cDepartment of Physical Chemistry, Fritz Haber Institute of the Max Planck
Society, Faradayweg 4-6, D-14195 Berlin, Germany
<EMAIL_ADDRESS>István Z. Kissb aDepartment of Physics, University of
Barcelona, Martí i Franquès 1, 08028 Barcelona, Spain
bDepartment of Chemistry, Saint Louis University, 3501 Laclede Ave., St.
Louis, Missouri 63103, USA
cDepartment of Physical Chemistry, Fritz Haber Institute of the Max Planck
Society, Faradayweg 4-6, D-14195 Berlin, Germany
<EMAIL_ADDRESS>
###### Abstract

Experiments with a network of bistable electrochemical reactions, organized in
regular and non-regular tree networks, are presented to confirm an alternative
to Turing mechanism for formation self-organized stationary patterns. The
results show that the pattern formation can be described by identification of
domains that can be activated individually or in combinations. The method was
also demonstrated to localization of chemical reactions to network
substructures and identification of critical sites whose activation results in
complete activation of the system. While the experiments were performed with a
specific nickel electrodissolution system, they reproduce all the salient
dynamical behavior of a general network model with a single nonlinearity
parameter. This indicates that the considered pattern formation mechanism is
very robust and similar behavior can thus be expected in other natural or
engineered networked systems, which exhibit, at least locally, a tree-like
structure.

In biological context, many chemical reactions take place in discrete units,
e.g., in cells, that form a complex network.[1, 2] Similarly, recent advances
in microfabrication allow the generation of engineered networks of reaction
units, e.g., with lab-on-chip microdroplets,[3] electrode arrays,[4, 5] BZ
beads,[6] or microelectrofusion.[7] The interplays between local reaction
kinetics (nodes), the physical processes that create coupling (link), and the
architecture of the network in such systems can lead to a wealth of self-
organized phenomena, including synchronization,[4, 6, 8] stationary Turing and
oscillatory patterns,[9, 10, 11, 12, 13] or excitation waves.[14, 15, 16]

Stationary patterns generated via the Turing[17] mechanism have been observed
in experiments for both continuous[18] and networked[19] systems. Here, an
alternative mechanism for emergence of stationary patterns in networks is
experimentally explored. We focus on network-organized systems of bistable
elements with diffusive connections between them. Bistable elements can be
found in a broad class of chemical reactions (e.g., with autocatalysis[20,
21]) but also in cellular[22] and engineered systems.[23] Such elements can
have local or diffusive connections between them. For regular lattices and
linear chains (i.e., for relatively simple networks), it is known that, under
sufficiently weak coupling, the fronts fail to propagate, and thus stationary
domains can be formed,[24, 25, 26, 27] whereas at strong coupling the fronts
spread[27, 28] and a uniform state is eventually established. Recently,
analogues phenomena were theoretically investigated for complex networks and
the formation of stationary domains, sensitive to the network topology, was
predicted based on a simple model of regular trees and one-component bistable
elements.[29, 30] The aim of the present study is to experimentally identify
the stationary pattern formation, induced by network structure, with chemical
reactions in which autocatalysis can produce local bistable behavior.

Suppose that some substance can undergo chemical reactions in reactors
occupying nodes $i$ ($i=1,\ldots,N$) of a network and this substance can
spread diffusively from one node to another. Such a network-organized
reaction-diffusion system is generally described by equations
$\dot{u}_{i}=f(u_{i})+K\sum_{j=1}^{N}\\!\left(A_{ij}u_{j}-A_{ji}u_{i}\right)\\!\,$,
where $u_{i}$ is the chemical concentration in the node $i$, the function
$f(u)$ specifies the local dynamics at the nodes, and the coefficient $K$
characterizes the strength of diffusive coupling. The network structure is
determined by a symmetric adjacency matrix whose elements are $A_{ij}=1$, if
there is a connection between nodes $i$ and $j$ ($i\neq j$), and $A_{ij}=0$
otherwise. Function $f(u)$ can be chosen in such a way that individual
elements are bistable, e.g., with autocatalysis (Supporting Information). For
such models an approximate analytical theory is available for special networks
representing regular trees with fixed branching ratio.[29]

Figure 1: a) The bifurcation diagram. Region $I$: Center and periphery
activations are pinned. Region $II$: Center activation is pinned whereas
periphery activation propagates towards the center of the tree. Region $III$:
Center and periphery activations propagate in both directions, to the
periphery and the center. Region $IV$: Center activation retreats whereas
periphery activation propagates towards the center. b) Different activity
domains in a non-regular tree network. Active domains $A$, $B$, $C$, and $D$,
are distinguished by different colors. Activation of any node with the cream
color ($\ocircle,\ \vartriangle,\ \varhexagon$) results in spreading of the
activation over all nodes in the respective network domains $A$, $A\cup B$,
$A\cup C$, or $A\cup D$. Activation of any node with the brown color
($\Square,\ \diamondsuit$) remains pinned on that node. Activation of a single
node with the magenta color ($\pentagon$) retreats and vanishes resulting in
the uniform passive state.

The analytical and numerical results are summarized in Figure 1. Under given
nonlinearity factors (Supporting Information), the two major parameters are
the coupling strength $K$ and the branching ratio $r$. For weak coupling, the
activation is pinned (i.e., it does not propagate) in region $I$. At the
intermediate coupling strength there is a range of branching ratios (region
$II$), where the center activation is pinned while the periphery activation
propagates towards the center. At sufficiently strong coupling and relatively
low branching ratios (region $III$), the center activation spreads towards to
periphery, whereas the periphery activation propagates towards the center. At
strong coupling and large branching ratios (region $IV$), the center
activation retreats and the periphery activation propagates towards the
center. The behavior of regular or non-regular tree networks can fully be
interpreted in terms of these four regions.

Figure 1b shows a non-regular tree network with branching ratios varying from
one or two, three, four and five. Numerical simulations for this network were
performed with the coupling constant $K=0.04$, so that these domains
corresponded to regions $II$, $III$ or $IV$ in the bifurcation diagram.
Simulations were done by applying initial activation to one of the nodes and
following the subsequent evolution until a stationary state was reached.
Stationary patterns were then classified by identifying all the possible
stationary states that could be obtained by such an initial activation.

The developed patterns can be interpreted by the formation of domains
activated individually or in combinations. The domains consist of groups of
nodes that correspond to the same dynamical region and thus behave similarly.
For example, domains of elements in region $III$ support spreading of
activation towards both the center and the periphery. When such domains are
adjacent to, or surrounded by nodes that are amenable to pinning or to
retreating (e.g., in region $I$, $II$, or $IV$), the propagating fronts get
pinned and stationary structures develop. Therefore, the observed structures
consist of uniform domains separated by nodes amenable to pinning or to
retreating. The exact configuration of the patterns depends on the
architecture of the network, the applied coupling strength, and the
nonlinearity of the reaction that altogether determine the assignment of the
nodes to the different regions and the configuration of the domains.

For the specific network, shown in Figure 1b, activation of a node with the
cream color resulted in spreading of activation to all nodes of the domains
$A$, $A\cup B$, $A\cup C$, or $A\cup D$. Activation of a node with the brown
color remained pinned on that node. Activation of a node with the magenta
color retreated and vanished resulting in the uniform final passive state.
Simulations were also performed by initially activating several network nodes,
but essentially the same final patterns were then reached.

Experiments with networks of bistable electrochemical reactions were performed
(Supporting Information). Each unit represented a corroding metal (nickel)
wire that accommodated a complex reaction system (including, e.g., formation
of multiple forms of metal oxides, bisulfate adsorption, oxygen evolution and
metal dissolution) that exhibited bistable behavior. Moreover, coupling was
established in the form of the charge flow between the wires (due to
difference in electrode potential) which affected the rate of metal
dissolution of the coupled electrodes.[4] Electrodes and external connections
between them, correspond to the nodes and the links in all network diagrams.

Figure 2: Evolution of center and periphery activations on a regular tree with
the branching ratio $r=3$ at different coupling strengths $K$. a) Initial
condition of center activation. b) Spreading front, $K=0.67\ mS$; final state
at $t=1008\ s$ is shown. c) Pinned front, $K=2.0\ mS$, $t=2310\ s$. d)
Retreating front, $K=4.0\ mS$, $t=68\ s$. e) Initial condition of periphery
activation. If $K=0.04\ mS$ the front is pinned, $t=480\ s$. f) Spreading
front, $K=0.10\ mS$, $t=464\ s$. $V=1300mV$; color coding indicated by the bar
is used to display currents in network nodes.

First, experiments with regular trees were undertaken. Figure 2 shows that
center activation in a four-layer tree with the branching ratio $3$ could
result in spreading fronts (Figure 2b, region $III$) for weak coupling, pinned
fronts (Figure 2c, region $II$) for moderate coupling, and retreating fronts
(Figure 2d, region $IV$) for strong coupling. Similarly, periphery activation
(Figure 2e) yielded either pinned fronts (Figure 2e, region $I$) at very weak
coupling or spreading fronts (Figure 2f) at stronger coupling; such behavior
was found in all regions II, III, or IV.

Figure 3: a) Pinned fronts in both directions, $K=0.10\ mS$; final state at
$t=402\ s$ is shown. b) Spreading front to the center and pinned front toward
the periphery, $K=0.20\ mS$, at $t=602\ s$. c) Spreading fronts in both
directions, $K=0.67\ mS$, at $t=538\ s$. d) A spreading front towards center
and a retreating front from periphery, $K=2.5\ mS$. Middle snapshot at $t=25\
s$ and final state at $t=50\ s$. $V=1300mV$.

Figure 3 shows the evolution observed in the same network after activation of
an intermediate node at different coupling strengths. In region $I$ (Figure
3a) the fronts are pinned on both sides. In region $II$ (Figure 3b) the center
facing front spreads towards the center and the periphery facing front is
pinned. In region $III$ fronts spread in both directions and finally activate
the entire network (Figure 3c). In region $IV$ the center facing front
propagates towards the center but the periphery facing front is retreated from
the periphery finally establishing the passive state in the entire network
(Figure 3d). All these experiments confirm the theoretical predictions for the
regular tree networks.

Furthermore, we could also build the same non-regular tree as in the
simulations in Figure 1b. By performing experiments with regular trees under
the same experimental setup, we could find (Supporting Figure, Figure S1) that
at $K\approx 1\ mS$ regions $II$, $III$, and $IV$ correspond to the branching
ratios $r=3,\ 4$, $r=1,\ 2$, and $r\geq 5$, respectively. Similar to
theoretical predictions, we could see that activations with a single (Figure
4a; Supporting Information, Video S1) or multiple nodes (Figure 4b; Supporting
Information, Figure Video S2) within domain $A$ resulted in activation of all
elements in domain $A$. When activation was applied to an intermediate element
of domain $B$, pattern $A\cup B$ was observed (Figure 4c; Supporting
Information, Video S3). To achieve complete activation of the network,
peripheral nodes of the branches with the highest branching ratios $r=3,\ 4$
and $5$ had to be initially activated (Figure 4d; Supporting Information,
Video S4). When a root node of a branch with high branching ratios $r=4$ or
$r=5$ was initially activated, the activation could not spread and died out
(Figure 4e). Furthermore, activation of more central node in the branch with
$r=3$ resulted in a pinned front (Figure 4f).

Figure 4: a) Activation of a single node from the linear chain yields
spreading fronts that activate the domain $A$ ($V=1300\ mV$, final state at
$t=654\ s$). b) Intermediate activation of four nodes in the domain $A$ with
$r=2$ yields spreading fronts to activate the domain $A$ ($V=1295\ mV$, final
state at $t=241\ s$). c) Near periphery activation of a single node in the
branch with $r=3$ yields spreading fronts which activate the domains $A\cup B$
($V=1275\ mV$, final state at $t=336\ s$). d) Periphery activation of the
branches with $r=3,\ 4$, and $5$ yields spreading fronts to complete network
activation ($V=1280\ mV$, final state at $t=201\ s$). e) Activation of a
single node in the branch with $r=5$ yields retreating fronts to complete
network passivation ($V=1300\ mV$, final state at $t=54\ s$). f) Activation of
a single more central node in the branch with $r=3$ yields pinned fronts
($V=1270\ mV$, final state at $t=801\ s$). All experiments performed at
$K=1.3\ mS$.

Thus, we have experimentally demonstrated that bistable tree networks can
support a rich variety of stationary patterns, determined both by the network
architecture and initial activation conditions. Many features of the
stationary patterns in a large, non-regular network, can be predicted from the
network topology based on experiments with four-layer trees that identify the
dependence of the dynamical behavior as a function of the branching ratio. The
experiments were performed with an electrochemical system; nonetheless, a
surprisingly good agreement with theoretical predictions has been found
despite the fact that the experimental system did not perfectly meet the
idealizations made in the theory,[29] i.e., the state of a network element was
described by more than a single variable and coupling between the elements was
local, but not exactly of the diffusive form. This indicates that the found
behavior is generic and robust; therefore it can be expected in various
natural and engineered bistable networks.

While only (regular or non-regular) tree networks were considered, our results
are also relevant for large random networks that possess locally the tree
structure.[31] Hence, formation of stationary domains should also be
characteristic for random networks provided that such domains remain
sufficiently small. Finally, it should be noted that, as has theoretically
been shown,[30] self-organized stationary domains can be controlled by
introducing global feedbacks and experimental realization of such control
would further facilitate the design of complex stationary structures on
networks.

## Acknowledgements.

N. E. K. acknowledge financial support by the LASAGNE (Contract No.318132) EU
project. M. S. and I. Z. K. acknowledge support from National Science
Foundation CHE-1465013.

## References

  * [1] L. R. Varshney, et al., PLoS Comp. Biol. 2011, 7, e1001066.
  * [2] M. E. J. Newman, SIAM Rev. 2003, 45, 167.
  * [3] M. Toiya, V. G. Vanag, I. R. Epstein, Angew. Chem. Int. Ed. Engl. 2008, 47, 7753.
  * [4] M. Wickramasinghe, I. Z. Kiss, PloS ONE 2013, 8, e80586.
  * [5] Y. Jia, I. Z. Kiss, J. Phys. Chem. C 2012, 116, 19290.
  * [6] M. R. Tinsley, S. Nkomo, K. Showalter, Nature Phys. 2012, 8, 662.
  * [7] M. Karlsson, K. Sott, M. Davidson, A-S. Cans, P. Linderholm, D. Chiu, Proc. Natl. Acad. Sci. USA 2002, 99, 11573.
  * [8] A. Arenas, et al., Phys. Rep. 2008, 469, 93.
  * [9] W. Horsthemke, K. Lam, P. K. Moore, Phys. Lett. A 2004, 328, 444-.
  * [10] H. Nakao, A. S. Mikhailov, Nature Phys. 2010, 6, 544.
  * [11] N. E. Kouvaris, S. Hata, A. Díaz-Guilera, Sci. Rep. 2015, 5, 10840.
  * [12] S. Hata, H. Nakao, A. S. Mikhailov, Sci. Rep. 2014, 4, 3585.
  * [13] M. Asllani, J. D. Challenger, F. S. Pavone, L. Sacconi, D. Fanelli, Nat. Commun. 2014, 5, 4517.
  * [14] N. E. Kouvaris, T. Isele, A. S. Mikhailov, E. Schöll Europhys. Lett. 2014, 106, 68001.
  * [15] T. Isele, E. Schöll, New J. Phys. 2015, 17, 023058.
  * [16] A. J. Steele, M. Tinsley, K. Showalter Chaos 2006, 16, 015110.
  * [17] A. M. Turing, Philos. Trans. R. Soc. 1952, 237, 37.
  * [18] J. Horvath, I. Szalai, P. De Kepper Science 2009, 324, 772.
  * [19] N. Tompkins, et al., Proc. Natl. Acad. Sci. USA 2014, 111, 4397.
  * [20] I. R. Epstein, J. A. Pojman, An Introduction to Nonlinear Chemical Dynamics: Oscillations, Waves, Patterns, and Chaos, Oxford University Press, 1998.
  * [21] A. S. Mikhailov, Foundations of Synergetics I: Distributed Active Systems, Springer, Berlin 1990.
  * [22] T. G. W. Graham, S. M. Ali Tabei, A. R. Dinner, I. Rebay, Development 2010, 137, 2265.
  * [23] K. Ikeda, H. Daido, O. Akimoto, Phys. Rev. Lett. 1980, 45, 709.
  * [24] V. Booth, T. Erneux, Physica D 1992, 188, 206.
  * [25] T. Erneux, G. Nicolis, Physica D 1993, 67, 237.
  * [26] I. Mitkov, K. Kladko, J. Pearson, Phys. Rev. Lett. 1998, 81, 5453.
  * [27] L. P. Laplante, T. Erneux Physica A 1992, 188, 89.
  * [28] G. Flatgen, K. Krischer, Phys. Rev. E 1995, 51, 3997.
  * [29] N. E. Kouvaris, H. Kori, A. S. Mikhailov, PLoS ONE 2012, 7, e45029.
  * [30] N. E. Kouvaris, A. S. Mikhailov, Europhys. Lett. 2013, 102, 16003.
  * [31] A. Barrat, M. Barthélemy, A. Vespignani, Dynamical Processes on Complex Networks, Cambridge Univ. Press, 2008.


# Cross-Level Cross-Scale Cross-Attention Network for Point Cloud
Representation

Xian-Feng Han<EMAIL_ADDRESS>0000-0002-4869-4537 College of Computer
and Information Science, Southwest UniversityNo.2 Tiansheng RoadBeibei
DistrictChongqingChina400712 ,  Zhang-Yue He<EMAIL_ADDRESS>College of Computer and Information Science, Southwest UniversityNo.2
Tiansheng RoadBeibei DistrictChongqingChina400712 ,  Jia Chen
<EMAIL_ADDRESS>College of Computer and Information Science, Southwest
UniversityNo.2 Tiansheng RoadBeibei DistrictChongqingChina400712 and  Guo-
Qiang Xiao<EMAIL_ADDRESS>College of Computer and Information Science,
Southwest UniversityNo.2 Tiansheng RoadBeibei DistrictChongqingChina400712

###### Abstract.

Self-attention mechanism recently achieves impressive advancement in Natural
Language Processing (NLP) and Image Processing domains. And its permutation
invariance property makes it ideally suitable for point cloud processing.
Inspired by this remarkable success, we propose an end-to-end architecture,
dubbed Cross-Level Cross-Scale Cross-Attention Network (CLCSCANet), for point
cloud representation learning. First, a point-wise feature pyramid module is
introduced to hierarchically extract features from different scales or
resolutions. Then a cross-level cross-attention is designed to model long-
range inter-level and intra-level dependencies. Finally, we develop a cross-
scale cross-attention module to capture interactions between-and-within scales
for representation enhancement. Compared with state-of-the-art approaches, our
network can obtain competitive performance on challenging 3D object
classification, point cloud segmentation tasks via comprehensive experimental
evaluation.

Point Cloud, Self Attention, Classification, Segmentation

††copyright: none††conference: ; ††booktitle: ;††submissionid: 816††ccs: 3D
point cloud analysis††ccs: Computing methodologies Neural Networks Figure 1.
Applications of our Cross-Level Cross-Scale Cross-Attention Network.

##  1\. Introduction

The rapid development of 3D acquisition devices makes point cloud become a
preferred 3D geometric representation for arbitrarily-shaped objects in real
word (Te et al., 2018). And thanks to its flexibility, analyzing and
understanding 3D point clouds, therefore, begin to serve as a fundamental
processing demanded by a number of applications, such as robotics (Nezhadarya
et al., 2020), autonomous driving (Guo et al., 2020) and VR/AR (Gojcic et al.,
2020).

Recently, deep learning techniques has revolutionized 2D computer vision with
powerful discrete convolution as essential component for impressive
representation learning on regular grid. Their outstanding performance
inspires more and more attention paid to deep learning on 3D point clouds.
However, since 3D point cloud can be considered as an unordered collection of
points with irregular structure, it is not always reasonable to directly
transfer deep learning techniques for image processing to point cloud based
tasks.

In order to sovle this challenge, several attempts have been made to transform
the unstructured 3D point clouds into regular girds, such as voxel grid
(Maturana and Scherer, 2015), multi-view images (Su et al., 2015), so that
standard 3D CNNs or 2D CNNs can be applied for feature learning. Nevertheless,
these approaches usually requires high memory footprint, computational
consumption, and may suffer from information loss during transformation. More
recently, deep neural networks directly manipulating raw point clouds have
been specifically developed. As a pioneering work, PointNet (Qi et al., 2017a)
encodes each individual point identically, and then accumulates point features
into a global vector via a symmetric function. PointNet++ (Qi et al., 2017b)
improves PointNet by taking into account local information of 3D point cloud.
However, these point-wise methods essentially treat points independently (Han
et al., 2020) and fail to capture their long-range correlation relationship
(Wang et al., 2018).

To better leverage point dependencies between and in different levels and
scales, we propose a novel architecture, named Cross-Level Cross-Scale Cross-
Attention network (CLCSCANet), for effective point feature learning, which
mainly contains three components. Specifically, we first design a Point
Feature Pyramid (PFP) module to hierarchically extract point-wise features
with different receptive fields. Then, a Cross-Level Cross-Attention model is
developed to investigate inter-level and intra-level correlations for
geometric and semantic information aggregation. Finally, a Cross-Level Cross-
Attention module is designed to learn inter-scale and intra-scale interactions
to further improve the discriminative ability of feature representations.

We conduct extensive quantitative and qualitative evaluation on ModelNet40 (Wu
et al., 2015) for 3D object classification, ShapNet (Yi et al., 2016) for part
segmentation. Experimental results demonstrate the effectiveness and
capability of better feature representation learning of our CLCSCANet with
achievement of comparable performance.

Summarily, the major contributions of this work are concluded as follows:

  * •

Three well-designed modules, namely Point-wise Feature Pyramid, Cross-Level
Cross-Attention, Cross-Scale Cross-Attention, are developed to aggregate
features across different levels and scales, while modeling their long-range
correlation dependencies.

  * •

Based on these models, we build an end-to-end Cross-Level Cross-Scale Cross-
Attention Network (CLCSCANet) architecture taking point clouds as input for
highly geometric and semantic representation learning.

  * •

We perform extensive experiments on publicly available benchmark datasets
across three tasks (i.e. classification and segmentation) and verify the
effectiveness of our CLCSCANet framework which can obtain competitive
performance compared with the state-of-the-arts.

##  2\. Related Work

###  2.1. Deep Learning on Point Clouds

The recent breakthroughs from deep learning in 2D computer vision domain has
been shifting the research trend gradually to its application in 3D point
clouds. However, this task is quite challenging, since standard CNNs are only
suitable for data with regular structure.

Grid-based Methods. In order to solve such problem, early attempts mainly
focus on transforming irregular point clouds into an intermediate grid
structure. View-based methods (Su et al., 2015)(Feng et al., 2018) intuitively
aims to turn the 3D problems into 2D problems by projecting the 3D point
clouds to a set of images from different views. Then, conventional
convolutional neural networks in 2D images can be used to perform feature
extraction. Finally, these features are aggregated back into 3D space.
Although remarkable performance has been achieved in classification task, it
is nontrivial to perform segmentation task due to the loss of intrinsic
geometric relationship of 3D points during projection (Han et al., 2020). On
the other hand, the choice of images including the number of views also
seriously impact on performance (Zhao et al., 2019).

Alternatively, voxel-based approaches attempt to assign the scattered 3D
points to a regular 3D grid structure, such as occupancy voxel grid (Maturana
and Scherer, 2015), that can be processed by standard 3D convolutional neural
networks. However, the cubic growth of memory requirements and computational
demanding limit their applicable to high-resolution voxelization (Lei et al.,
2019), and also constrains the expressiveness and efficiency of voxelized
representation of point clouds (Yang et al., 2020). To alleviate this problem,
the octree-based (Lei et al., 2019) and kd-tree-based (Klokov and Lempitsky,
2017) models are proposed to refine the performance (Lin et al., 2020a). For
example, OctNet (Riegler et al., 2017) improves the resolution up to
$256\times 256\times 256$. Nevertheless, the grid-based representation still
suffers from quantization artifacts and fine-grained information loss.

Point-based Methods.The appearance of PointNet (Qi et al., 2017a) leads to a
new trend of deep learning on point clouds. This network directly takes
coordinates of points as input and uses shared MLPs to independently learn
point-wise features, which are then aggregated into a global representation
via the channel-wise max-pooling. A major limitation is that PointNet lacks
ability to acquiring local information of point clouds. To remedy this,
PointNet++ (Qi et al., 2017b) extends PointNet with a hierarchical structure
to understand local features using farthest ponit sampling and a geometric
grouping layer (Zhao et al., 2019)(Xie et al., 2018). Some latest works focus
on generalizing convolution operator for grid-structured data to handle
unstructured point clouds. And these methods usually define spatial kernels by
exploring the spatial localization information in a point clouds (Thomas et
al., 2019). Kernel Point Convolution uses a small collection of kernel points
for local feature learning. PointConv (Wu et al., 2019) can be considered as
Monte Carlo estimation of 3D continuous convolution, which uses MLP and a
density function to estimate and re-weight the learned weight function. (Lin
et al., 2020b) designs a surface-style convolution, named FPConv, for point
cloud analysis. (Engelmann et al., 2020) proposes Dilated Point Convolutions
to increase the receptive field size.

Unlike these methods, we pay attention not only to acquiring information in
local regions, but also to capturing long-range context dependencies among
points in point clouds.

###  2.2. Self-Attention

Attention mechanism originally from natural language processing domain
(Bahdanau et al., 2014)(Lin et al., 2017), nowadays, has been attracting more
and more attention in the field of computer vision, since it has the ability
of capturing global contextual information by modeling much broader range
dependencies between the components of the input sequence. For example, Fu et
al. (Fu et al., 2019) developed position attention and channel attention
modules based on self-attention mechanism for scene segmentation. On the other
hand, the usage of matrix multiplication and summation in self-attention makes
it invariant to permutation, which is an ideal choice for point cloud
processing.

Inspired by this remarkable property, we design the cross-level cross-
attention and cross-scale cross-attention modules with the fundamental idea of
self-attention for discriminative representation learning.

##  3\. Cross-Level Cross-Scale Cross-Attention Network

###  3.1. Overview

Figure 2 illustrates the overall architecture of our proposed end-to-end
Cross-Level Cross-Scale Cross-Attention Network (CLCSCANet). Our goal is to
learn a projection function $f:P\rightarrow R^{F}$ for various point cloud
based tasks including 3D classification and segmentation. The network directly
undertakes 3D point cloud wiht $N$ points $P=\\{p_{i}\in
R^{3+a},i=1,2,...,N\\}$ as input, where $3+a$ represents 3D coordinates
$(x_{i},y_{i},z_{i})$ and additional attributes, such as color, surface
normal. First, Farthest Point Sampling algorithm is initially used to obtain
three sub point clouds with different resolutions, each of which will be fed
into the corresponding path of our Point Feature Pyramid module (Section 3.2)
to construct hierarchical features. Then, we explore intra-level and inter-
level point feature relationships by building a Cross-Level Cross-Attention
(Section 3.3) model for aggregating geometric and semantic information
simultaneously. Finally, a Cross-Scale Cross-Attention module (Section 3.4) is
defined to fully investigate the correlation among points within and from
different scales for feature representation enhancement. In the following
sections, we will elaborate these three modules.

Figure 2. Overall Architecture of proposed Cross-Level Cross-Scale Cross-
Attention Network for Point Cloud Analysis. The network mainly consists of
three core modules, namely Point-wise Feature Pyramid, Cross-Level Cross-
Attention and Cross-Scale Cross-Attention. C and D denote the number of
classes for Classification and Segmentation tasks.

###  3.2. Point-wise Feature Pyramid

Empirically, the receptive field size is directly related to contextual
information learning (larger receptive field usually means description of much
broader context ), which makes significant contribution to the performance of
3D point cloud processing (Engelmann et al., 2020). Hence, in order to extract
hierarchical features to deal with objects with diverse scales, we develop a
Point-Wise Feature Pyramid (PFP) module, which is composed of three individual
branches (from top to bottom, the resolution is decreasing while the receptive
field size is increasing).

Specifically, as illustrated in Figure 2, we first perform Furthest Point
Sampling (FPS) algorithms (Qi et al., 2017b) on initial input point cloud $P$
to obtain three subsets of points with different resolutions $\\{P_{1}\in
R^{N1\times(3+a)})=FPS(P,N_{1}),P_{2}\in
R^{N2\times(3+a)}=FPS(P,N_{2}),P_{3}\in R^{N3\times(3+a)}=FPS(P,N_{3})\\}$,
since FPS can generate limited centroids ensuring better coverage of point
cloud and maintain the original shape (Lin et al., 2020b). Then, $K$ points
around each centroid that are determined via ball query with a given radius
are grouped to form a local region for capturing local feature. Finally, the
three independent paths in PFP module apply their corresponding mapping
functions $f_{scale}^{i}:R^{N_{i}\times(3+a)}\rightarrow R^{N_{i}\times
C},i=1,2,3$ to these three downsampled point clouds to extract three
individual latent representations. In our work, we define each path by
stacking multiple shared Multi-Layer Perception (MLP) modules.

###  3.3. Cross-Level Cross-Attention Module

The spatial dependencies among points in the same level and among different
levels can provide both semantically-rich and geometrically-rich information.
In order to better model intra-level and inter-level correlation and make full
use features from different levels, we formulate the Cross-Level Cross-
Attention (CLCA) model for learning more comprehensive feature representation.
The overall architecture is shown in Figure 3.

Intra-Level Correlation. For each path $i=1,2,3$, we separately extract low-
level, mid-level and high-level feature maps, denoting as
$\mathcal{F}_{low}^{i}$, $\mathcal{F}_{mid}^{i}$, $\mathcal{F}_{high}^{i}$,
from corresponding shared MLP layers. Here, we will take the high-level
feature map $\mathcal{F}_{high}^{i}$ as example for simplified description.
First, we apply poinwise feature transformations to $\mathcal{F}_{high}^{i}$
to calculate the query, key and value matrices, which are defined as:

(1)
$Q_{high}^{i}=\phi(\mathcal{F}_{high}^{i})=\mathcal{F}_{high}^{i}W_{high-i}^{q}$
(2)
$K_{high}^{i}=\psi(\mathcal{F}_{high}^{i})=\mathcal{F}_{high}^{i}W_{high-i}^{k}$
(3)
$V_{high}^{i}=\beta(\mathcal{F}_{high}^{i})=\mathcal{F}_{high}^{i}W_{high-i}^{v}$

Then, the formulation for point self correlations $F_{i}$ is modeled as
follows:

(4)
$SC^{i}_{high}=\sigma(Q_{high}^{i}\cdot(K_{high}^{i})^{T}/\sqrt{C‘})V_{high}^{i}+\mathcal{F}_{high}^{i}$

where $\phi(\cdot)$, $\psi(\cdot)$, $\beta(\cdot)$ are linear transformations
in our work, and $W_{mid-i}^{q}\in R^{C\times C^{\prime}}$, $W_{mid-i}^{k}\in
R^{C\times C^{\prime}}$, $W_{mid-i}^{v}\in R^{C\times C}$ are corresponding
learnable weight parameters matrices. We set $C^{\prime}=C/4$ for
computational efficiency. $\sigma(\cdot)$ is a normalization function, and
softmax is chosen in our work. Simultaneously, the same procedures are applied
to low-level and mid-level features to obtain the corresponding
outputs$SC^{i}_{low}$ and $SC^{i}_{mid}$.

Inter-Level Correlation.To further explore correlations across different
levels for aggregating more discriminate point-wise features, we introduce the
cross-attention for level interaction, which can be formulated as follows:

(5)
$F_{CLCA}^{i}=\mathcal{AT}(SC^{i}_{low},SC^{i}_{mid},SC^{i}_{high})+SC^{i}_{low}+SC^{i}_{mid}+SC^{i}_{high}$

where,

(6) $\displaystyle\mathcal{AT}(\cdot)$
$\displaystyle=\sigma(f_{1}(SC^{i}_{low})\cdot(f_{2}(SC^{i}_{mid})^{T})\cdot
f_{3}(SC^{i}_{high})$ (7)
$\displaystyle=\delta((SC^{i}_{low}W_{clca-i}^{1})\cdot(SC^{i}_{mid}W_{clca-i}^{2})^{T})\cdot(SC^{i}_{high}W_{clca-i}^{3})$

Here, $W_{clca-i}^{1}\in R^{C\times C/4}$, $W_{clca-i}^{2}\in R^{C\times C/4}$
and $W_{clca-i}^{3}\in R^{C\times C}$ are linear projection parameters
matrices.

Lastly, the final output of our Cross-Level Cross-Attention module contains
three individual representations $F_{CLCA}^{i}\in R^{N_{i}\times C},i=1,2,3$,
each having the same scales as the corresponding $P_{i}$. And the usage of
multi-level information is actually of important in point cloud understanding.

Figure 3. Cross-Level Cross-Attention Architecture for inter-level and intra-
level dependencies learning.

###  3.4. Cross-Scale Cross-Attention Module

In PFP module, we see that the top branch has the highest resolution but
lowest receptive field, while the bottom with lowest resolution and largest
receptive field. Empirically, point-wise feature at different scales generally
contains different semantic or contextual information. Therefore, to fully
exploit long-range self correlation within the same scale and aggregate
dependencies across different scale correlation learning, we formulate Cross-
Scale Cross-Attention module for enhancing the discriminative power of
representation.

Up-Sampling Layer. Specifically, to perform point-wise prediction task
conveniently, such as segmentation, we should first propagate point-wise
feature maps $F_{CLCA}^{i}$ from CLCA up to the original resolution of input
point cloud through up-sampling layer that mainly uses $K$ nearest neighbors
(KNN) interpolation and shared MLPs.

(8) $F_{cs-in}^{i}=UP(F_{CLCA}^{i})=MLP(KNN-Interpo(F_{CLCA}^{i},P))$

Where $F_{CSCA-input}^{i}\in R^{N\times D}$ represents the interpolated point-
wise feature maps from the $i$th scale ($i=1,2,3$).

Intra-Scale Interaction.Then, based on the fundamental idea of self-attention
mechanism, relationship among points of $i$th scale can be constructed as:

(9) $SC_{scale}^{i}=\sigma((F_{cs-in}^{i}W_{SC}^{q})\cdot(F_{cs-
in}^{i}W_{SC}^{k})^{T}/sqrt(D^{\prime}))\cdot(F_{cs-in}^{i}W_{SC}^{v})+F_{cs-
in}^{i}$

where $W_{SC}^{q}\in R^{D\times D^{\prime}}$, $W_{SC}^{k}\in R^{D\times
D^{\prime}}$ and $W_{SC}^{v}\in R^{D\times D}$ are the weight parameters of
three fully-connected layers. And $D^{\prime}$ is set to $D/4$.

Inter-Scale Interaction. Similar to Cross-Level Cross-Attention model, we also
consider the cross scale attention to fuse multi-scale information. And this
interaction model is formalized as:

(10)
$F_{CSCA}=\mathcal{AT^{\prime}}(SC_{scale}^{1},SC_{scale}^{2},SC_{scale}^{3})+SC_{scale}^{1}+SC_{scale}^{2}+SC_{scale}^{3}$

Here,

(11)
$\begin{split}AT^{\prime}=\sigma((SC_{scale}^{1}W^{1})\cdot(SC_{scale}^{2}W^{2})^{T})\cdot((SC_{scale}^{3}W^{3}))\\\
W^{1}\in R^{D\times D^{\prime}},W^{2}\in R^{D\times D^{\prime}},W^{3}\in
R^{D\times D}\end{split}$

The Cross-Scale Cross-Attention module finally outputs semantically meaningful
point-wise feature representation.

###  3.5. Architecture and Supervision

From Figure 2, it can be clearly seen that these three modules depicted in the
previous subsections, together with fully connected layers, constitute the
overall architecture of our proposed CLCSCANet, which is simple and easy to
implement.. The detailed configuration of 3D object classification and point
cloud segmentation networks are provided in Table LABEL:. During training,
cross-entropy is employed to supervise the learning process of our model in an
end-to-end manner.

(12) $CE(G,P)=\left\\{\begin{aligned} -\sum_{i=1}^{C}G_{i}log(P_{i})&\qquad
for\;classification\\\ -\sum_{n=1}^{N}\sum_{i=1}^{C}G_{n,i}log(P_{n,i})&\qquad
for\;segmentation\\\ \end{aligned}\right.$

where G and P represent the ground truth and prediction, respectively. N
refers to the number of input points. C denotes the number of classes.

###  3.6. Network Configurations

The detailed configurations of network for classification and segmentation
tasks are summarized in Table 1

Table 1. Network configurations for classification and segmentation. NN($r,K$)
denotes the radius of ball query and the number of points to group for local
information extraction. MLP($C_{in},C_{out}$) represents the Multiple Layer
Perceptron, taking $C_{in}$ input features and generating $C_{out}$ features.
CLCA($C_{in}$) and CSCA($C_{in}$) denotes cross-level cross-attention and
cross-scale cross-attention operations for modeling interactions.
FC($C_{in},C_{out}$) means fully connected layer. $C$ is the number of
categories of corresponding dataset.

Tasks | Scales | Resolution | Layer1 | Layer2 | Layer3 | Layer4 | Layer5 | Layer6 | Layer7
---|---|---|---|---|---|---|---|---|---
Classification | Path1 | 512 |  | NN(r=0.2, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.4, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.8, K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) | - | CSCA(512) |  FC(1024, 512) FC(512, 256) FC(256, C)
Path2 | 256 |  | NN(r=0.2, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.4, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.8, K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) | -
Path3 | 128 |  | NN(r=0.2, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.4, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.8,K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) | -
Segmentation | Path1 | 512 |  | NN(r=0.1, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.2, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.4,, K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) |  | UP(2048)
---
MLP(278, 512)
MLP(512, 256)
MLP(256, 128)
CSCA(128) |  FC(128, 128) FC(128, C)
Path2 | 256 |  | NN(r=0.1, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.2, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.4, K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) |  | UP(2048)
---
MLP(278, 512)
MLP(512, 256)
MLP(256, 128)
Path3 | 128 |  | NN(r=0.1, K=16)
---
MLP(3, 64)
MLP(64, 128)
MLP(128, 128)
| NN(r=0.2, K=32)
---
MLP(256,128)
MLP(128, 128)
| NN(r=0.4,K=64)
---
MLP(256,128)
MLP(128, 128)
CLCA(256) |  | UP(2048)
---
MLP(278, 512)
MLP(512, 256)
MLP(256, 128)

##  4\. Experiments

We evaluate our CLCSCANet architecture with extensive experiments on multiple
challenging benchmark datasets, including ModelNet (Wu et al., 2015) for
classification, ShapNetPart (Yi et al., 2016) for part segmentation. And for
all experiments, we use Pytorch framework to implement our proposed model on a
NVIDIA RTX TITAN 24G GPU. Adam optimizer and step learning rate decay are
adopted to train the network in end-to-end manner.

###  4.1. 3D Shape Classification

ModelNet40 classification benchmark includes 12,311 meshed CAD models in 40
different object categories, where we randomly choose 9,843 models for
training and 2,468 samples for evaluation. Following PointNet++ (Qi et al.,
2017b), 1,024 points without normals are uniformly sample from each object
instance. Meanwhile, for fair comparison, data preprocessing operations,
including random point dropout, random shifting, and random scale, are applied
to augment the input. During training, we set the initial learning rate to
0.001, which is decayed 0.7 every 20 epochs. The classification network is
trained for 150 epochs with a batch size of 20.

In Table 2, we report experimental settings and the quantitative comparison
results with several state-of-the-art methods. From the table, it can be
explicitly stated that 1) our CLCSCANet achieves comparable classification
accuracy (92.2%) among these state-of-the-art methods that only takes point
cloud as input, such as PointNet (Qi et al., 2017a), OctreeGCNN (Lei et al.,
2019) and SPH3D-GCN (Lei et al., 2020), etc. 2) Our model is slightly better
than the PointNet++ with 5,000 points and normals as input, obtaining 0.3%
improvement, while it is a little worse than SFCNN (Rao et al., 2019) that
also uses normal as additional input. 3)Meanwhile, our CLCSCANet performs
favorably against volumetric-based, like OctNet (Riegler et al., 2017) and
multi-view based models (e.g., MVCNN (Su et al., 2015)). 4) Experimental
results show the effectiveness of our model in 3D object classification.

Table 2. 3D Object Classification results on ModelNet40 dataset. Method | Representation | Input Size | ModelNet40
---|---|---|---
3DShapeNets (Wu et al., 2015) | Volumetric | $30^{3}$ | 77.3%
VoxNet (Maturana and Scherer, 2015) | Volumetric | $32^{3}$ | 83.0%
OctNet (Riegler et al., 2017) | Volumetric | $128^{3}$ | 86.5%
MVCNN (Su et al., 2015) | Multi-view | $12\times 224^{2}$ | 90.1%
DeepNet (Ravanbakhsh et al., 2016) | Points | $5000\times 3$ | 90.0%
Kd-Net (Klokov and Lempitsky, 2017) | Points | $2^{15}\times 3$ | 88.5%
PointNet (Qi et al., 2017a) | Points | $1024\times 3$ | 89.2%
ECC (Simonovsky and Komodakis, 2017) | Points | 1000 $\times$3 | 83.2%
KC-Net (Shen et al., 2018) | Points | $1024\times 3$ | 91.0%
FoldingNet (Yang et al., 2018) | Points | $2048\times 3$ | 88.4%
OctreeGCNN (Lei et al., 2019) | Points | $1024\times 3$ | 92.0%
3D-GCN (Lin et al., 2020a) | Points | $1024\times 3$ | 92.1%
SPH3D-GCN (Lei et al., 2020) | Points | $1000\times 3$ | 92.1%
CLCSCANet | Points | $1024\times 3$ |  92.2%
SFCNN (Rao et al., 2019) | Points+normals | $1024\times 6$ | 92.3%
PointNet++ (Qi et al., 2017b) | Points+normals | $5000\times 6$ | 91.9%

###  4.2. Part Segmentation

For 3D point cloud segmentation task, we use ShapeNetPart (Yi et al., 2016)
dataset to validate the effectiveness of our CLCSCANet. This dataset consists
of 16,881 3D CAD shapes from 16 different categories with 50 parts in total.
We use the official split 14,007 shapes for training and 2,874 instances for
testing, where each shape have 2 to 6 part labels, and each point is
associated with a part label for segmentation task. Following the previous
works, we adopt the average Intersection over Union (mIoU) over all instances
and category-wise IoU over shapes under that category to evaluate the
performance of our method. For fair comparison, we sample 2,048 points from
each shape model. The initial learning rate is 0.0005 and is decayed half
every 20 epochs. We use batch size of 8 to train the our segmentation network
for 120 epochs. The momentum is 0.9.

The 3D point cloud part segmentation results of our CLCSCANet are presented in
Table 3, in which we also make an comparison with several state-of-the-art
methods, such as SO-Net (Li et al., 2018), DGCNN (Wang et al., 2018), 3D-GCN
(Lin et al., 2020a), etc. From our experimental results, we can report that
our model achieves the highest mIoU of 85.3%, and performs much better in 4 of
16 categories. Figure 4 visualizes the qualitative comparison between the part
segmentation results of our CLCSCANet and the ground truth. Both quantitative
and visualization results demonstrate the success of our CLCSCANet in point
cloud segmentation task.

Table 3. Experimental comparison of part segmentation with the state-of-the-
art approaches on ShapeNet part dataset. The mean IoU across all the shape
instances and IoU for each category are reported.

Method | mIoU | aero | bag | cap | car | chair | ep | guitar | knife | lamp | laptop | motor | mug | pistol | rocket | skate | table
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
ShapeNet (Yi et al., 2016) | 81.4 | 81.0 | 78.4 | 77.7 | 75.7 | 87.6 | 61.9 | 92.0 | 85.4 | 82.5 | 95.7 | 70.6 | 91.9 | 85.9 | 53.1 | 69.8 | 75.3
PointNet (Qi et al., 2017a) | 83.7 | 83.4 | 78.7 | 82.5 | 74.9 | 89.6 | 73.0 | 91.5 | 85.9 | 80.8 | 95.3 | 65.2 | 93.0 | 81.2 | 57.9 | 72.8 | 80.6
PointNet++ (Qi et al., 2017b) | 85.1 | 82.4 | 79.0 | 87.7 | 77.3 | 90.8 | 71.8 | 91.0 | 85.9 | 83.7 | 95.3 | 71.6 | 94.1 | 81.3 | 58.7 | 76.4 | 82.6
KD-Net (Klokov and Lempitsky, 2017) | 82.3 | 80.1 | 74.6 | 74.3 | 70.3 | 88.6 | 73.5 | 90.2 | 87.2 | 71.0 | 94.9 | 57.4 | 86.7 | 78.1 | 51.8 | 69.9 | 80.3
SO-Net (Li et al., 2018) | 84.9 | 82.8 | 77.8 | 88.0 | 77.3 | 90.6 | 73.5 | 90.7 | 83.9 | 82.8 | 94.8 | 69.1 | 94.2 | 80.9 | 53.1 | 72.9 | 83.0
RGCNN (Te et al., 2018) | 84.3 | 80.2 | 82.8 | 92.6 | 75.3 | 89.2 | 73.7 | 91.3 | 88.4 | 83.3 | 96.0 | 63.9 | 95.7 | 60.9 | 44.6 | 72.9 | 80.4
DGCNN (Wang et al., 2018) | 85.2 | 84.0 | 83.4 | 86.7 | 77.8 | 90.6 | 74.7 | 91.2 | 87.5 | 82.8 | 95.7 | 66.3 | 94.9 | 81.1 | 63.5 | 74.5 | 82.6
3D-GCN (Lin et al., 2020a) | 85.1 | 83.1 | 84.0 | 86.6 | 77.5 | 90.3 | 74.1 | 90.9 | 86.4 | 83.8 | 95.6 | 66.8 | 94.8 | 81.3 | 59.6 | 75.7 | 82.6
ELM (Fujiwara and Hashimoto, 2020) | 85.2 | 84.0 | 80.4 | 88.0 | 80.2 | 90.7 | 77.5 | 91.2 | 86.4 | 82.6 | 95.5 | 70.0 | 93.9 | 84.1 | 55.6 | 75.6 | 82.1
Weak Sup. (Xu and Lee, 2020) | 85.0 | 83.1 | 82.6 | 80.8 | 77.7 | 90.4 | 77.3 | 90.9 | 87.6 | 82.9 | 95.8 | 64.7 | 93.9 | 79.8 | 61.9 | 74.9 | 82.9
CLCSCANet | 85.3 | 82.7 | 71.3 | 77.8 | 77.8 | 90.8 | 75.2 | 91.0 | 87.0 | 84.7 | 95.5 | 71.6 | 93.3 | 82.8 | 56.9 | 74.1 | 83.5

Figure 4. Qualitative comparison between the prediction of our CLCSCANet and
the ground truth on ShapeNet Part dataset.

###  4.3. Ablation Study and Analysis

To validate the importance of our proposed modules in CLCSCANet, ablation
studies are conducted on the classification task of ModelNet40. Here, we adopt
a single-scale point-wise network that stacks multiple shared MLPs as our
baseline. Table 4 reports the accuracy average class (ACC) and overall
accuracy (OA) of different design choices of CLCSCANet.

Table 4. Ablation studies on the ModelNet40 dataset. We analyze the effects of point-wise self-attention (PWSA) and channel-wise self-attention (CWSA). Method | ACC | OA
---|---|---
Baseline | 85.1 | 87.1
Baseline + CLCA | 89.5 | 91.6
Baseline + CSCA | 88.7 | 90.8
CLCSCANet |  90.3% |  92.2%

From the table, we can see that both CLCA and CLCS modules significantly
improve the classification performance in terms of ACC and OA. This remarkable
improvements further convincingly demonstrate that the introduction of CLCA
and CLCS modules enhance the geometric and semantic power of representation by
modeling point-wise feature interactions across different levels and scales.

##  5\. Conclusion

Following the remarkable advances in deep learning and self attention
techniques, we propose a Cross-Level Cross-Scale Cross-Attention network
architecture for better analyzing and understanding 3D point clouds in this
paper. The introduction of Pointwise Feature Pyramid, Cross-Level Cross-
Attention and Cross-Scale Cross-Attention models contributes to progressively
aggregating geometric and semantic information and enhancing the
representation capability. Results from extensive experiments show the
effectiveness and competitive performance of our network.

## References

  * (1)
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014\.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_ (2014).
  * Engelmann et al. (2020) Francis Engelmann, Theodora Kontogianni, and Bastian Leibe. 2020.  Dilated point convolutions: On the receptive field size of point convolutions on 3D point clouds. In _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 9463–9469.
  * Feng et al. (2018) Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. 2018.  Gvcnn: Group-view convolutional neural networks for 3d shape recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 264–272.
  * Fu et al. (2019) Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu. 2019.  Dual attention network for scene segmentation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 3146–3154.
  * Fujiwara and Hashimoto (2020) Kent Fujiwara and Taiichi Hashimoto. 2020.  Neural Implicit Embedding for Point Cloud Analysis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 11734–11743.
  * Gojcic et al. (2020) Zan Gojcic, Caifa Zhou, Jan D Wegner, Leonidas J Guibas, and Tolga Birdal. 2020.  Learning multiview 3D point cloud registration. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 1759–1769.
  * Guo et al. (2020) Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. 2020\.  Deep learning for 3d point clouds: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ (2020).
  * Han et al. (2020) Wenkai Han, Chenglu Wen, Cheng Wang, Xin Li, and Qing Li. 2020.  Point2Node: Correlation learning of dynamic-node for point cloud feature modeling. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 10925–10932.
  * Klokov and Lempitsky (2017) Roman Klokov and Victor Lempitsky. 2017.  Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In _Proceedings of the IEEE International Conference on Computer Vision_. 863–872.
  * Lei et al. (2019) Huan Lei, Naveed Akhtar, and Ajmal Mian. 2019.  Octree guided cnn with spherical kernels for 3d point clouds. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9631–9640.
  * Lei et al. (2020) Huan Lei, Naveed Akhtar, and Ajmal Mian. 2020.  Spherical kernel for efficient graph convolution on 3d point clouds.  _IEEE transactions on pattern analysis and machine intelligence_ (2020).
  * Li et al. (2018) Jiaxin Li, Ben M Chen, and Gim Hee Lee. 2018.  So-net: Self-organizing network for point cloud analysis. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 9397–9406.
  * Lin et al. (2020b) Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, and Xiaoguang Han. 2020b.  Fpconv: Learning local flattening for point convolution. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4293–4302.
  * Lin et al. (2017) Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017.  A structured self-attentive sentence embedding.  _arXiv preprint arXiv:1703.03130_ (2017).
  * Lin et al. (2020a) Zhi-Hao Lin, Sheng-Yu Huang, and Yu-Chiang Frank Wang. 2020a.  Convolution in the cloud: Learning deformable kernels in 3D graph convolution networks for point cloud analysis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 1800–1809.
  * Maturana and Scherer (2015) Daniel Maturana and Sebastian Scherer. 2015.  Voxnet: A 3d convolutional neural network for real-time object recognition. In _2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 922–928.
  * Nezhadarya et al. (2020) Ehsan Nezhadarya, Ehsan Taghavi, Ryan Razani, Bingbing Liu, and Jun Luo. 2020.  Adaptive hierarchical down-sampling for point cloud classification. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 12956–12964.
  * Qi et al. (2017a) Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a.  Pointnet: Deep learning on point sets for 3d classification and segmentation. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 652–660.
  * Qi et al. (2017b) Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b.  Pointnet++: Deep hierarchical feature learning on point sets in a metric space.  _arXiv preprint arXiv:1706.02413_ (2017).
  * Rao et al. (2019) Yongming Rao, Jiwen Lu, and Jie Zhou. 2019.  Spherical fractal convolutional neural networks for point cloud recognition. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 452–460.
  * Ravanbakhsh et al. (2016) Siamak Ravanbakhsh, Jeff Schneider, and Barnabas Poczos. 2016\.  Deep learning with sets and point clouds.  _arXiv preprint arXiv:1611.04500_ (2016).
  * Riegler et al. (2017) Gernot Riegler, Ali Osman Ulusoy, and Andreas Geiger. 2017\.  Octnet: Learning deep 3d representations at high resolutions. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3577–3586.
  * Shen et al. (2018) Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. 2018\.  Mining point cloud local structures by kernel correlation and graph pooling. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 4548–4557.
  * Simonovsky and Komodakis (2017) Martin Simonovsky and Nikos Komodakis. 2017.  Dynamic edge-conditioned filters in convolutional neural networks on graphs. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3693–3702.
  * Su et al. (2015) Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. 2015.  Multi-view convolutional neural networks for 3d shape recognition. In _Proceedings of the IEEE international conference on computer vision_. 945–953.
  * Te et al. (2018) Gusi Te, Wei Hu, Amin Zheng, and Zongming Guo. 2018\.  Rgcnn: Regularized graph cnn for point cloud segmentation. In _Proceedings of the 26th ACM international conference on Multimedia_. 746–754.
  * Thomas et al. (2019) Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. 2019.  Kpconv: Flexible and deformable convolution for point clouds. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 6411–6420.
  * Wang et al. (2018) Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2018.  Dynamic graph cnn for learning on point clouds.  _arXiv preprint arXiv:1801.07829_ (2018).
  * Wu et al. (2019) Wenxuan Wu, Zhongang Qi, and Li Fuxin. 2019.  Pointconv: Deep convolutional networks on 3d point clouds. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9621–9630.
  * Wu et al. (2015) Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015.  3d shapenets: A deep representation for volumetric shapes. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1912–1920.
  * Xie et al. (2018) Saining Xie, Sainan Liu, Zeyu Chen, and Zhuowen Tu. 2018\.  Attentional shapecontextnet for point cloud recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 4606–4615.
  * Xu and Lee (2020) Xun Xu and Gim Hee Lee. 2020.  Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13706–13715.
  * Yang et al. (2020) JuYoung Yang, Chanho Lee, Pyunghwan Ahn, Haeil Lee, Eojindl Yi, and Junmo Kim. 2020\.  PBP-Net: Point Projection and Back-Projection Network for 3D Point Cloud Segmentation.  _arXiv preprint arXiv:2011.00988_ (2020).
  * Yang et al. (2018) Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. 2018\.  Foldingnet: Point cloud auto-encoder via deep grid deformation. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 206–215.
  * Yi et al. (2016) Li Yi, Vladimir G Kim, Duygu Ceylan, I Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, Leonidas Guibas, et al. 2016\.  A scalable active framework for region annotation in 3d shape collections.  _ACM Transactions on Graphics (TOG)_ 35, 6 (2016), 210\.
  * Zhao et al. (2019) Hengshuang Zhao, Li Jiang, Chi-Wing Fu, and Jiaya Jia. 2019\.  Pointweb: Enhancing local neighborhood features for point cloud processing. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 5565–5573.