# Anomalous Diffusion

Mendeli H. Vainstein<EMAIL_ADDRESS>Institute of Physics and
International Center of Condensed Matter Physics, University of Brasília, P.O.
Box: 04513, 70919-970 Brasília-DF, Brazil  Luciano C. Lapas
<EMAIL_ADDRESS>Institute of Physics and International Center of Condensed
Matter Physics, University of Brasília, P.O. Box: 04513, 70919-970 Brasília-
DF, Brazil  Fernando A. Oliveira<EMAIL_ADDRESS>Institute of Physics and
International Center of Condensed Matter Physics, University of Brasília, P.O.
Box: 04513, 70919-970 Brasília-DF, Brazil

###### Abstract

Recent investigations call attention to the dynamics of anomalous diffusion
and its connection with basic principles of statistical mechanics. We present
here a short review of those ideas and their implications.

1\. Introduction. - Diffusion is a fundamental problem in statistical physics
Brown28 ; Einstein05 ; Smoluchowski06 ; Langevin08  and much more simple to
describe than reaction rates Kramers40 ; Hanggi90 ; Oliveira98 , for example.
From the studies of diffusion, it is possible to obtain a direct description
of important concepts in physics, such as the ergodic hypothesis (EH) Lee07a ;
Khinchin49 ; Boltzmann74 ; Lee01  and the fluctuation-dissipation theorem
(FDT) Langevin08 ; Kubo66 ; Costa03 . Some of the first concepts of
statistical mechanics, for instance the Gaussian distribution of particles,
were obtained in the study of diffusion, which also led promptly to other
concepts as the Boltzmann equilibrium distribution. In our study of diffusion,
we use the Mori formalism Mori65  which has a well defined memory function for
non-Markovian systems. In this context, the Kubo response function Zwanzgi01
is of special importance, since it was formulated in terms of correlation
functions which can be obtained by the use of linear response theory and are
directly connected with experiments, such as light or neutron scattering
Oliveira81 . In this article we display some recent results in this area, and
we point the reader to selected literature directly related to this topic.

2\. Anomalous Diffusion. - Anomalies in the traditional Einstein diffusion
have been the focus of extensive research in many disciplines Astumian02 ;
Morgado02 ; Bulashenko02 ; Costa03 ; Bao03 ; Budini04 ; Dorea06 ; Bao06 . In
order to address this problem let us consider the generalized Langevin
equation (GLE) in the form Mori65 ; Kubo66 ; Zwanzgi01

$\frac{dP(t)}{dt}=-\int_{0}^{t}\Pi(t-t^{\prime})P(t^{\prime})dt^{\prime}+F(t),$
(1)

where $P$ is the particle momentum, $\Pi(t)$ is a kernel, or memory function,
and $F(t)$ is a random force, which fulfills $\langle F(t)\rangle=0$ and the
fluctuation-dissipation theorem (FDT):

$\langle F(t)F(t^{\prime})\rangle=\langle P^{2}\rangle_{eq}\Pi(t-t^{\prime}).$
(2)

The usual manner to study the diffusive dynamics is to investigate the mean
square displacement of the particles, given by

$\lim_{t\rightarrow\infty}\langle x^{2}(t)\rangle\propto t^{\alpha}.$ (3)

The exponent $\alpha$ classifies the type of diffusion: for $\alpha=1$, we
have normal diffusion; for $0<\alpha<1$, subdiffusion; and $\alpha>1$,
superdiffusion. To obtain average values, it is important to define the
correlation function for the dynamical operator $P$ as

$C_{P}(t)=\langle P(t)P(0)\rangle,$ (4)

where the brackets $\left\langle\right\rangle$ denote an ensemble average. One
can also define the normalized correlation function

$R(t)=C_{P}(t)/C_{P}(0),$ (5)

which obeys the equation

$\frac{dR(t)}{dt}=-\int_{0}^{t}\Pi(t-t^{\prime})R(t^{\prime})dt^{\prime}.$ (6)

Its Laplace transform yields

$\tilde{R}(z)=\frac{1}{z+\tilde{\Pi}(z)}.$ (7)

Recently, an important achievement in the dynamics of diffusion has been
obtained by Morgado _et al._ Morgado02 . They have shown that if

$\lim_{z\rightarrow 0}\tilde{\Pi}(z)\approx cz^{\nu},$ (8)

where $c$ is a positive nondimensional constant, then

$\alpha=1+\nu.$ (9)

Consequently, the behavior of the memory $\tilde{\Pi}(z)$ for small $z$
determines the long range behavior of the diffusion, Eq. (3). This result has
found many applications: the study of molecular motors Bao03 ; Bao06 ,
anomalous diffusion Costa03 ; Dorea06 ; Vainstein06 ; Lapas07 , and the
dynamics of dipolar chains Toussaint04  are a few examples.

In a subsequent work, Costa _et al._ Costa03  have shown that the average
value of the momentum is

$\langle P^{2}(t)\rangle=\langle P^{2}\rangle_{eq}+R^{2}(t)[\langle
P^{2}(0)\rangle-\langle P^{2}\rangle_{eq}].$ (10)

In this equation the average value will be the equilibrium value if

$\lim_{t\rightarrow\infty}R(t)=0,$ (11)

i. e., irreversibility is a necessary and sufficient condition for ergodicity
to hold in diffusion. For systems which violate the condition given in Eq.
(11), there will be no ergodicity. Using the final value theorem and Eq. (7)
we obtain

$\lim_{t\rightarrow\infty}R(t)=\lim_{z\rightarrow
0}z\tilde{R}(z)=\lim_{z\rightarrow 0}\frac{1}{1+cz^{\alpha-2}}.$ (12)

This relation shows that for most of the diffusive regimes, $0<\alpha<2$, the
equilibrium condition, Eq. (11), holds. However, for ballistic diffusion,
$\alpha=2$, it is not fulfilled

$\lim_{t\rightarrow\infty}R(t)=\lim_{z\rightarrow 0}\frac{1}{1+c}\neq
0\text{.}$ (13)

This means that ballistic diffusion violates ergodicity and the fluctuation
dissipation theorem Costa03 . In other words, if the ballistic system is not
initially equilibrated, then it will never reach equilibrium and the final
result of any measurement will depend on the initial conditions. In this
situation the EH will not be valid. For $\alpha>2$, no memory of the initial
condition is lost ($R(t\rightarrow\infty)=1$) and the process is not
diffusive, being an activated process for which the GLE does not work Costa03
. The results of this letter apply to all kinds of diffusion, $0<\alpha<2$,
described by a GLE independent of the memory range. This gives origin to new
studies in ballistic diffusion Bao03 ; Bao06 ; Lapas07 .

For any initial distribution of values $P(0)$, with $\langle
F(t)P(0)\rangle=0$, it is possible to obtain the temporal evolution of the
moments of $P$,

$\langle P(t)\rangle=\langle P(0)\rangle R(t),$ (14)

Again, after an infinite time, we expect that $\langle
P(t\rightarrow\infty)\rangle=0$. However, for ballistic diffusion, this is not
the case, as can be seen from Eq. (13). The astonishing result here is the
existence of a residual current such as in superfluids.

3\. Nonexponential behavior. - From the above results, it is quite clear that
the correlation for ballistic diffusion will not decay exponentially to
equilibrium. Besides the ballistic case, any anomalous regime will present
nonexponential decay. Even for normal diffusion, a large number of relaxations
may be nonexponential Morgado02 . There are a large number of phenomena where
the systems do not relax immediately to equilibrium. Those phenomena, usually
associated with non-aging, have nonexponential relaxation and are most
commonly described by power laws or stretched exponentials. The study of
anomalous relaxation has produced quite interesting results Kauzmann48 ;
Parisi97 ; Ricci00 ; Metzler00 ; Metzler04 ; Holek01 ; Rubi03 ; Hentschel07 .

For a system described by a GLE of the form Eq. (1), the evolution relies on
the noise that drives the particles. For a harmonic noise Morgado02

$F(t)=\frac{1}{\sqrt{2k_{B}T}}\int\sqrt{\rho(\omega)}\cos(\omega
t+\phi(\omega))d\omega,$ (15)

where $\rho$ is the noise spectral density, $k_{B}$ is the Boltzmann constant,
and $\phi$ is a set of random phases in the range $0\leq\phi\leq 2\pi$. A
systematic study carried on by Vainstein _et al._ Vainstein06  has shown that
the spectral density plays a fundamental role in the description of stochastic
processes as we shall see. First, the memory function $\Pi(t)$ can be easily
obtained by the use of the FDT, Eq. (2), as

$\Pi(t)=\int\rho(\omega)\cos(\omega t)d\omega,$ (16)

in such a way that the average cancels the random terms and obviously the
memory is a deterministic even function. The Laplace transform of the memory
$\tilde{\Pi}(z)$ is an odd function in $z$; therefore, $\tilde{R}(z)$ given in
Eq. (7) is also an odd function in $z$. This implies that by inverting the
Laplace transform, $R(t)$ is an even function of $t$

$R(-t)=R(t).$ (17)

Second, the condition given by Eq. (8) to determine the exponent $\alpha$, Eq
(9), for a spectral density of the form

$\lim_{\omega\rightarrow 0}{\rho}(\omega)\approx\omega^{\beta},$ (18)

becomes

$\nu=\left\\{\begin{array}[]{ll}\beta,&\mbox{ $\beta<1$};\\\ \\\ 1,&\mbox{
$\beta\geq 1$}.\end{array}\right.$ (19)

This shows that a noise spectral density in the form of a power law can
produce only diffusive motion in the region $0\leq\alpha\leq 2$. Diffusive
motion beyond ballistic is not allowed, what can be observed by using the
Laplace transform.

To study relaxation we need to know $R(t)$, which can be calculated
analytically in restricted cases, being obtained numerically most of the
times. A recent study shows that exponential decay, power laws or even Mittag-
Leffler functions are particular cases of a more general function Vainstein06
which approximates the decay. Indeed, considering time reversal symmetry
Oliveira81  and the discussion above (see Eq. (17)), the correlation function
must be even and cannot be any of those forms. We shall expose here the
conditions under which it is possible to obtain an approximately exponential
decay, what happens in certain circumstances for normal diffusion. In this
case,

$\gamma=\lim_{z\rightarrow 0}\tilde{\Pi}(z)=\frac{\pi}{2}\rho(0).$ (20)

Consequently, the friction in the usual Langevin equation is nothing more than
the noise spectral density for the lower modes. For an arbitrary memory, the
system has a rich behavior; even for normal diffusion it is possible to show
the existence of at least three time ranges Vainstein06 . A normal diffusion
can be obtained using a spectral density of the form Morgado02

$\rho(\omega)=\left\\{\begin{array}[]{ll}\frac{2\gamma}{\pi},&\mbox{
$\omega<\omega_{s}$};\\\ \\\ 0,&\mbox{
$\omega>\omega_{s}$}.\end{array}\right.$ (21)

For a broad band noise spectral density, $\gamma/\omega_{s}<1$, and long times
$t>\gamma^{-1}$ it is possible to decouple Eq. (6) to obtain an exponential
decay of the form $R(t)=\exp{(-\gamma t)}$, which will bring the system to
equilibrium, that is, $R(t\rightarrow\infty)\rightarrow 0$.

Most of the experimental situations where anomalous relaxation is present
arise in complex, nonlinear or far from equilibrium structures in which
detailed balance does not hold. Good examples can be found in supercooled
liquids Kauzmann48  and in glasses Parisi97 ; Ricci00 . Those systems,
however, apparently do not have any easy analytical solution. On the another
hand, diffusion can present closed solution for the main expectation values,
and arises as a simple laboratory for the discussion of those properties.

4\. Nonlinear dynamics I: Chain Dynamics - As mentioned before Toussaint04 ,
chain dynamics is a quite interesting subject where the application of those
ideas may lead to important results. There is a particular problem that has
attracted our attention: the breaking process of a chain of $N$ monomers of
mass $M$ subject to a strain $S$. The equation of motion for monomer $j$,
$j=1,2,..N$, is Oliveira94 ; Oliveira98b

$M\frac{d^{2}X_{j}}{dt^{2}}=G_{j}-G_{j+1}-M\gamma\frac{dX_{j}}{dt}+F(t),$ (22)

where the forces

$G_{j}=-\frac{\partial U(a+S+y)}{\partial y}|_{X_{j}-X_{j-1}}$ (23)

are derived from the interparticle potential $U$. In this situation, it is
simpler to treat the random force $F(t)$ as delta correlated, i.e.,
$\Pi(t)=2\gamma\delta(t)$. We then define an effective potential and consider
the system as a one body Kramers problem. However, simulations show that the
breaking rate is around a hundred times smaller than the usual Kramers rate
Kramers40 . Several approaches were used in order to overcome the problem; one
of them was to consider that the collective motion of the chain generates a
harmonic noise, which is correlated and has an associated memory. Those were
to be added to Eq. (22) in order to consider a non-Markovian analysis
Oliveira95 . This improves the results, but does not solve the problem. This
is an important issue, because the simulation really describes a very
important experiment. Polymers used as an additive in a turbulent flow have
breaking rates which are up to $10^{-6}$ smaller than those computed using the
simple Kramers theory; the simulation Maroja01 , on the other hand, is in
perfect agreement with the experiments.

5\. Nonlinear dynamics II: Synchronization - Nonlinear dynamics plus noise is
an explosive mixture with a large number of unexpected results. The study of
the evolution of maps subject to a common noise shows different examples of
synchronization Fahy92 ; Longa96 . Besides that, the study of Langevin
trajectories shows very nice patterns both for systems without memory Ciesla01
and with memory Morgado07 . Synchronization of many different phenomena arises
continuously in the literature Acebron05 .

6\. Final remarks. - Diffusion, one of the simplest phenomena in physics, is a
starting point for the study of simple and complex fluids. Many theorems in
statistical physics and even proper applications of its formalism, such as a
“simple linear response theory”, rely on the ergodic hypothesis and its
validation. Again, diffusion shows up as a simple way to address the problem.
In this work we have discussed various regimes of anomalous diffusion, which
are ergodic in the range of exponents $0<\alpha<2$, where $\alpha$ defines the
asymptotic behavior of the diffusion, Eq. (3). For $\alpha=2$, we have the
special ballistic case, for which ergodicity is not valid, as we have seen. In
recent years, molecular motors have been receiving a lot of attention Bao03 ;
Bao06  because of their large potential for pure and applied science. In this
subject, a discussion such as the one presented in this paper may be very
useful. Moreover, there are many situations that present violation of the EH,
particularly in glassy systems Parisi97 ; Ricci00 , and others were the EH
holds. For example, dynamical simulations and equilibrium statistical
mechanics were recently used to treat glass transition calculations
Hentschel07 . The agreement found between the two approaches is a strong
indication of the validity of the EH. This is a quite surprising result for
such a complex system. Disordered systems are still a large universe in which
to explore the basic assumptions of statistical mechanics.

###### Acknowledgements.

This work was supported by Coordenação de Pessoal de Nível Superior (CAPES),
Conselho Nacional de Desenvolmento Científico (CNPq), Financiadora de Estudos
e Projetos (FINEP) and Fundação de Amparo à Pesquisa do Distrito Federal
(FAPDF).

## References

  * [1] R. Brown,  Phil. Mag., 4 161 (1828);  Ann. Phys. Chem. B, 14 294 (1828).
  * [2] A. Einstein,  Investigations on the theory of the Brownian Movement,  (Dover, New York) 1956.
  * [3] M. von Smoluchowski,  Ann. Phys., 21 756 (1906).
  * [4] P. Langevin,  Comptes Rendus, 146 530 (1908).
  * [5] H. A. Kramers,  Physica, 7 284 (1940).
  * [6] P. Hänggi, P. Talkner and M. Borkovec,  Rev. Mod. Phys., 62 251 (1990).
  * [7] F. A. Oliveira,  Physica A, 257 128 (1998).
  * [8] M. H. Lee,  Phys. Rev. Lett., 98 190601 (2007).
  * [9] A. I. Khinchin,  Mathematical Foundations of Statistical Mechanics (Dover, New York) 1949.
  * [10] L. Boltzmann,  On the Development of the Methods of Theoretical Physics in Recent Times, in Theoretical Physics and Philosophical Problems: Selected Writings  (Kluwer Academic Publishers) 1974.
  * [11] M. H. Lee,  Phys. Rev. Lett., 87 250601 (2001).
  * [12] R. Kubo,  Rep. Prog. Phys., 29 255 (1966). R. Kubo, M. Toda and N. Hashitsume,  Statistical Physics II (Springer, Berlin) 1991.
  * [13] I. V. L. Costa, R. Morgado, M. V. B. T. Lima, and F. A. Oliveira,  Europhys. Lett., 63 173 (2003).
  * [14] H. Mori,  Prog. Theor. Phys., 423 33 (1965).
  * [15] R. Zwanzig,  Nonequilibrium Statistical Mechanics,  (Oxford University Press, New York) 2001.
  * [16] F. A. Oliveira,  Solid State Commun., 40 859 (1981).
  * [17] R. D Astumian, and P. Hänggi,  Physics Today, 55 33 (2002).
  * [18] R. Morgado, F. A. Oliveira, G. G. Batrouni and A. Hansen,  Phys. Rev. Lett., 89 100601 (2002).
  * [19] O. M. Bulashenko and J. M. Rubí,  Phys. Rev. B, 66 045310 (2002).
  * [20] J. D. Bao and Y. Z. Zhuo,  Phys. Rev. Lett., 91 138104 (2003).
  * [21] A. A. Budini and M. O. Caceres,  J. Phys. A: Math. Gen., 37 5959 (2004);  Phys. Rev. E, 70 046104 (2004).
  * [22] C. C. Y. Dorea and A. V. Medino,  J. Stat. Phys., 123 685 (2006).
  * [23] J. D. Bao, Y. Z. Zhuo, F. A. Oliveira and P. Hänggi,  Phys. Rev. E, 74 061111 (2006).
  * [24] L. C. Lapas, I. V. L. Costa, M. H. Vainstein and F. A. Oliveira,  Europhys. Lett., 77 37004 (2007).
  * [25] M. H. Vainstein, I. V. L. Costa, R. Morgado and F. A. Oliveira,  Europhys. Lett., 73 726 (2003).
  * [26] R. Toussaint, G. Helgesen and E. G. Flekkøy,  Phys. Rev. Lett. 93 108304 (2004).
  * [27] W. Kauzmann,  Chem. Rev. 43 219 (1948).
  * [28] G. Parisi,  Phys. Rev. Lett. 79 3660 (1997).
  * [29] F. Ricci-Tersenghi, D. A. Stariolo, and J. J. Arenzon,  Phys. Rev. Lett. 84 4473 (2000).
  * [30] R. Metzler and R. Klafter,  Phys. Rep., 339 1 (2000).
  * [31] R. Metzler and R. Klafter,  J. Phys. A: Math. Gen., 37 161 (2004).
  * [32] I. Santamaria-Holek, D. Reguera and J. M. Rubí,  Phys. Rev. E , 63 051106 (2001).
  * [33] A. Pérez-Madrid, D. Reguera, J. M. Rubí,  Physica A, 329 357 (2003).
  * [34] H. G. E. Hentschel, V. Ilyin, N. Makedonska, I. Procaccia, N. Schupper,  Phys. Rev. E , 75 050404 (2007).
  * [35] F. A. Oliveira and P. L. Taylor,  J. Chem. Phys. 101 10118 (1994).
  * [36] F. A. Oliveira,  Phys. Rev. B 57 10576 (1998).
  * [37] F. A. Oliveira,  Phys. Rev. B 52 1009 (1995).
  * [38] A. Maroja, F. A. Oliveira, M. Ciesla and L. Longa,  Phys. Rev. E 63 061801 (2001).
  * [39] S. Fahy, D. R. Hamann,  Phys. Rev. Lett. , 69 761 (1992).
  * [40] L. Longa, E. M. F. Curado and F. A. Oliveira,  Phys. Rev. E , 54 R2201 (1996).
  * [41] M. Ciesla, S. P. Dias, L. Longa and F. A. Oliveira,  Phys. Rev. E , 63 065202 (2001).
  * [42] R. Morgado, M. Ciesla, L. Longa and F. A. Oliveira,  Europhys. Lett. , 79 10002 (2007).
  * [43] J. A. Acebron,L. L. Bonilla, C. J. P. Vicente, F. Ritort and R. Spliger,  Rev. Mod. Phys. , 77 137 (2005).


# Enforcing exact permutation and rotational symmetries
in the application of quantum neural network on point cloud datasets

Zhelun Li<EMAIL_ADDRESS>Lento Nagano
<EMAIL_ADDRESS>Koji Terashi<EMAIL_ADDRESS>International Center for Elementary Particle Physics (ICEPP),
The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan

###### Abstract

Recent developments in the field of quantum machine learning have promoted the
idea of incorporating physical symmetries in the structure of quantum
circuits. A crucial milestone in this area is the realization of
$S_{n}$-permutation equivariant quantum neural networks (QNN) that are
equivariant under permutations of input objects. In this work, we focus on
encoding the rotational symmetry of point cloud datasets into the QNN. The key
insight of the approach is that all rotationally invariant functions with
vector inputs are equivalent to a function with inputs of vector inner
products. We provide a novel structure of QNN that is exactly invariant to
both rotations and permutations, with its efficacy demonstrated numerically in
the problems of two-dimensional image classifications and identifying high-
energy particle decays, produced by proton-proton collisions, with the
$SO(1,3)$ Lorentz symmetry.

††preprint: APS/123-QED

##  I Introduction

Recently, there has been an increasing interest in Quantum Machine Learning
(QML), which is expected to have an advantage over its classical counterpart
for a specific task (see e.g. [1, 2, 3, 4] for review articles) in a wide
range of fields including quantum chemistry, condensed-matter physics, high-
energy physics (HEP). Among variants of QMLs, the (variational) Quantum Neural
Network (QNN) e.g. [5, 6, 7, 8] is widely used architecture. Specifically, in
the field of HEP, a number of recent papers have pioneered the application of
QNN in the analysis of high energy collider datasets [9], including
classifying new physics events using quantum annealing [10], variational
quantum circuits [11, 12, 13], and quantum kernel algorithms [14, 15, 16].

In recent years, researchers in both classical and quantum machine learning
communities have started to experiment with ideas of encoding problem-specific
symmetries into QNNs. In classical Machine Learning (ML), equivariant Neural
Networks (NN) have been proven to be an effective method to exploit symmetries
in the dataset, which often leads to substantial improvements in model
performances. An example of the equivariance in NN is the algorithm known as
Convolutional Neural Networks (CNNs) [17, 18], which was designed to address
the translational symmetry in images. A standard CNN would operate on gridded
pixel inputs of images and perform sliding window convolutions on nearby
pixels such that the network could pick up patterns independent of their
translational positions. This idea of convolution-based NN was also realized
in QNNs by introducing quantum convolution layers made of quasi-local unitary
transformations applied in a translationally invariant fashion [19, 20]. The
equivariance of this architecture, called Quantum Convolutional Neural
Networks (QCNNs), has been proven extremely useful in the study of quantum
many-body problems [19, 21, 22, 23, 24, 25], especially when the Hamiltonian
under investigation possesses an exact, or approximate, translational
symmetry. In the last few years, general theories of achieving equivariance in
quantum circuits have been built using ansatz choices that accommodate the
algebraic properties of the underlying symmetries [26, 27, 28]. One example of
these developments is the introduction of the twirling method [29, 26, 27],
which provides a general method to encode full permutation symmetry into the
QNN. Several recent papers have designed rotationally equivariant QNN for
image classification by studying the properties of discretized approximation
of the rotational group [30, 31, 32, 33]. Other examples include permutation
equivariance [34, 35, 36, 37, 38], $SU(N)$ equivariance [39, 40, 41, 42, 43],
and equivaraince in graph problems [44, 45, 46, 47]. Despite the success in
equivariant QNN, the realization of arbitrary symmetries, especially
continuous symmetries, in QNN is still considered a challenging task [26, 32,
31] in general. Since the convolution-based method requires the model to
perform convolutions across all group elements [48], its application in
datasets with continuous symmetry inevitably runs into difficulties induced by
the infinite number of elements in continuous groups. Furthermore, the
traditional convolution-based treatment in classical ML requires the use of
complicated convolutional layers targeting a discretized approximation, often
realized by Fourier transforms, of the continuous symmetry [49, 50, 51, 52].
Such treatments are proven to be very hard to compute even in classical NNs.
It is still unclear how a QNN could replicate these architectures using
hardware-efficient ansatz consisting of single-qubit rotations and 2-qubit
entanglements. We also observe that most realizations of equivariant QNNs
utilize the amplitude encoding technique, which requires complicated circuit
structures to fully encode the classical data and is therefore challenging to
implement on noisy intermediate-scale quantum machines [30, 31, 32].

In this work, we aim to design a new framework of equivariant QNNs that is
compatible with near-term quantum machines. We consider an alternative
approach to enforcing exact rotational symmetries in point cloud datasets,
which consist of collections of data points in arbitrary dimension spaces. To
achieve the exact equivariance, we invoke Weyl’s work [53], which asserted
that all invariant functions of the orthogonal groups $O(N)$ with vector
inputs could be expressed as functions solely dependent on the group-invariant
inner products of these vectors. This key insight enables us to perform simple
data preprocessing to obtain $O(N)$-invariant inputs to a network that is
therefore guaranteed to be exactly $O(N)$-invariant at the input level. This
approach was first developed by Villar et al. [54] who proved that this simple
method involving only pair-wise scalars could significantly improve the
performance of NN. Recently, another work in the field of HEP elaborated on
this idea to produce the Permutation Equivariant and Lorentz Invariant or
Covariant Aggregator Network (PELICAN) with the full permutation and
${SO}(1,3)$ Lorentz symmetry [55]. The PELICAN was tested on the top-quark
identification task, which is an example of real-world problems involving
${SO}(1,3)$ Lorentz and permutation symmetries. When compared with state-of-
the-art models in top-quark identifications, the PELICAN was reported to have
achieved a similar performance while having significantly lower model
complexity. We note that this idea of enforcing ${SO}(N)$ rotational symmetry
by taking inner products can be easily extended to QNNs since it does not
require any specific choice of the quantum circuit ansatz.

This paper is organized as follows. In Section II, we discuss the theoretical
aspect of the implementations of both the rotational and permutation
symmetries. For the rotational symmetry, we will briefly go through the
theorem that offers the theoretical guarantee for our approach. And then we
proceed to work out the architecture of quantum circuits that could further
enforce permutation equivariance on the rotationally invariant inputs. In
Section III, we first test this architecture in a dataset made of rotated 2D
images to demonstrate its power. Then a slightly more complex version is
applied to the task of particle decay classifications. Finally, we conclude
our work and discuss its effectiveness, limitations, and possible future
extensions in Section IV.

##  II Methods

In this Section, we will first briefly introduce the general recipe to build a
QNN with trainable parameters in Section II.1. Section II.2 gives the details
of the three QNN models that are investigated in this paper: The baseline QNN
with no symmetry, the QNN with rotational symmetry, and the QNN with both
rotational and permutation symmetries. Finally, we introduce the loss function
used in the training process in Section II.3.

###  II.1 General construction

Suppose that our data is a set of $\\{\vec{x}_{i}\\}$ and associated labels
$\\{y_{i}\\}$. First, the classical vectors $\vec{x}_{i}$ are encoded by the
encoding unitary $U_{\text{E}}$ which acts on the initial state as
$U_{\text{E}}(\\{\vec{x}_{i}\\})|\psi_{0}\rangle$. Then these states are
processed by a QNN unitary $U_{\text{QNN}}(\theta)$ to have a final state

$|\psi(\\{\vec{x}_{i}\\};\theta)\rangle=U_{\text{QNN}}(\theta)U_{\text{E}}(\\{\vec{x}_{i}\\})|\psi_{0}\rangle\,.$
(1)

After the feature encoding and the hardware efficient ansatz, the output of
the QNN, $\hat{y}$, is evaluated by measuring the expectation value of an
observable, $O$, which is taken to be the product of Pauli-$Z$ operators
across all qubits in this work:

$\displaystyle\hat{y}(\\{\vec{x}_{i}\\};\theta)$
$\displaystyle=\langle\psi(\\{\vec{x}_{i}\\};\theta)|O|\psi(\\{\vec{x}_{i}\\},\theta)\rangle\,$
(2) $\displaystyle=\langle\psi(\\{\vec{x}_{i}\\};\theta)|Z^{\otimes
N_{q}}|\psi(\\{\vec{x}_{i}\\},\theta)\rangle\,.$ (3)

The output layer of the QNN uses the global observable, $Z^{\otimes N_{q}}$,
which is more prone to the barren plateau phenomenon than local observables
[56]. However, the global observable is fully compatible with the twirling-
based permutation equivariant ansatz that will be introduced in Section
II.2.3. 111One can avoid the use of global observable e.g. by using
$\sum_{n}Z_{n}$ instead. We do not investigate the (potential) difference that
comes from such a replacement in this paper. The output will then be compared
to the truth label of the events to perform classification. The rotation
angles in the ansatz act as trainable weights, the values of which are to be
optimized in the learning process.222Typically, the weights are optimized via
gradient descent, which relies on the computation of the first-order
derivatives of the output through the parameter-shift rule [5, 57]. It is
known that the gradient of QNN often exhibits barren plateau [58], in which
the gradient will diminish exponentially as the number of qubits increases. In
this paper, we use a gradient-free optimizer known as COBYLA [59] to update
the weights in the learning process.

###  II.2 QNN architectures

####  II.2.1 Baseline model

Figure 1: An example of the baseline QNN model that uses the $Z$ feature map
and a hardware efficient ansatz. The data-encoding part of the circuit in the
blue box encodes two 2D vectors using the $Z$ feature map. The ansatz in the
red brackets is repeated $N$ times.

In this subsection, we introduce the basic QNN [60] as a baseline model to
deal with point cloud datasets. For a typical point cloud dataset, each input
data consists of a series of vector points, the value of which indicates the
geometrical positions of objects in space. In this paper, we denote a vector
in the point cloud by $\vec{p}$. For example, a two-dimensional point cloud
dataset contains a collection of $x$ and $y$ axis coordinates of data points
in the form of: $\\{\vec{p}_{i}\\}=\\{(p_{i}^{0},p_{i}^{1})\\}$, where the
superscript denotes the coordinate index and the subscript denotes the index
of the data point. To use these inputs for a QNN, we first flatten the input
into a one-dimensional array,
$\\{p_{0}^{0},p_{0}^{1},p_{1}^{0},p_{1}^{1},...,p_{n}^{0},p_{n}^{1}\\}$, and
then encode these values in qubits using $Z$ feature maps. A $Z$ feature map
consists of single qubit $Z$-rotations with angles corresponding to the values
of the input array. If the input array is normalized to have values within
$[0,2\pi]$, the $Z$ feature map will then offer a unique encoding that maps
the input array onto the quantum state of qubits:
$\lvert\psi(\vec{p})\rangle$. For a dataset consisting of $n$ data points in
the $d$ dimensional space, it will require $N_{q}=n\cdot d$ number of qubits
to fully encode the input data. After the input data has been encoded onto the
qubits, a QNN will then use multiple layers of hardware efficient ansatz,
denoted by $U({\vec{\theta}})$, consisting of single qubit rotations and CNOT
gates, as shown in Figure 1.

####  II.2.2 QNN with rotational symmetry

It is obvious that the baseline QNN circuit given in Figure 1 is not
rotationally invariant. To construct a circuit with the exact rotational
symmetry, we first introduce a theorem that provides some insights into this
problem. From Weyl’s work [53] and its application in modern ML architecture
[54, 55], we note that for any rotationally invariant function with vector
inputs, $I(p_{1},...,p_{N})$, there always exists an equivalent rotationally
invariant function $F(\\{p_{i}\cdot p_{j}\\})$ with pair-wise inner products
as inputs:

$I(p_{1},...,p_{N})=F(\\{p_{i}\cdot p_{j}\\}_{i,j}).$ (4)

This suggests that the pair-wise inner products provide enough information to
build any rotationally invariant function that could be built with the vector
inputs. Although such a change in the inputs’ format might seem trivial at
first, a closer examination will reveal that this approach offers a
significantly easier way of encoding the rotational symmetry. On the left-hand
side of Equation (4), the function, $I$, has to be specifically chosen to have
rotational invariance, which, in the context of NN, often involves layers of
convolutions. However, the right-hand side of Equation (4) is a function that
has the rotational symmetry guaranteed at the input level since all inner
products are rotational invariants. This provides great freedom in the
construction of $F(\\{p_{i}\cdot p_{j}\\})$, the architecture of which has no
constraint as far as the rotational symmetry is concerned. This is
particularly beneficial in QNNs since it allows for the use of hardware-
efficient ansatz in the construction of a rotationally invariant circuit. In
classical machine learning, this approach was used to build the PELICAN
network, which can identify top quarks at the state-of-the-art $94.25\%$
accuracy [55].

As an example, we consider the input of four spatial points in arbitrary
dimension space. The circuit in Figure 2 computes the pair-wise inner
products, $p_{i}\cdot p_{j}$, and uses them as the inputs in the feature map
encoding shown in the blue box.

####  II.2.3 QNN with rotational and permutation symmetries

Figure 2: An example of a QNN setup with both rotational and permutation
symmetry of four input vectors. The rotational symmetry is enforced by having
pair-wise inner products as inputs, which requires $4\cdot(4+1)/2=10$ qubits
in the encoding process: $\\{p_{1}\cdot p_{1}\\},...,\\{p_{3}\cdot p_{4}\\}$.
The ansatz is decomposed into two blocks as discussed in Section II.2.3. Each
twirled operator is required to have the same parameter by the twirling method
to achieve equivariance. Each $ZZ$ gate takes exactly two inputs. If a
horizontal line goes through a $ZZ$ gate, it implies that the corresponding
qubit is not one of the $ZZ$ gate’s inputs.

For point cloud datasets, there often exists a permutation symmetry, in which
the label is invariant under permutations of the input data points. Suppose
that the data points are invariant under the symmetry induced by a group
$\mathcal{S}$, which could be taken to be the permutation group in this
section. Let $V[s]$ be a representation of the symmetry group $\mathcal{S}$ on
the data points space $\\{p_{i}\\}$. The goal here is to construct a QNN
circuit which satisfies

$\hat{y}(V[s]\cdot p;\theta)=\hat{y}(p;\theta)\,,$ (5)

where we denote $p=\\{p_{i}\\}$, and $V[s]\cdot p$ is the symmetry group
elements acting on the data points space. The QNN with this condition is
called an equivariant QNN.

First, the action of symmetry group on the encoding unitary can be expressed
using the so-called induced representation:

$U_{\text{E}}(V[s]\cdot
p)=V^{\text{(ind)}}[s]U_{\text{E}}(p)V^{\text{(ind)}{\dagger}}[s]\,.$ (6)

Then, our task is to symmetrize QNN unitary $U_{\text{QNN}}$ using
$V^{\text{(ind)}}[s]$. Suppose that our QNN is composed of the products of $G$
from group $\mathcal{G}$: $R_{G}(\theta)=\exp(-iG\theta)$. Then the
symmetrization is equivalent to

$\displaystyle[R_{G}(\theta),V^{\text{(ind)}}[s]]$
$\displaystyle=0,\quad\text{for
all}\quad\theta\in[0,2\pi],G\in\mathcal{G},s\in\mathcal{S}\,,$ (7)
$\displaystyle\Leftrightarrow[G,V^{\text{(ind)}}[s]]$
$\displaystyle=0,\quad\text{for all}\quad G\in\mathcal{G},s\in\mathcal{S}\,.$
(8)

A common approach to symmetrize generators $G\in\mathcal{G}$ is the use of the
twirling formula [29, 26, 27], in which $\mathcal{T}[G]$ is equivariant to all
elements in group $\mathcal{S}$.

$\mathcal{T}[G]=\frac{1}{|{\mathcal{S}}|}\sum_{s\in\mathcal{S}}V^{\text{(ind)}}[s]GV^{\text{(ind)}{\dagger}}[s]$
(9)

Then the resulting QNN output $\hat{y}$ becomes equivariant, if the observable
commutes with the induced unitary operators. For a one-index input vector, the
twirl operator simply requires the operator at hand, $G$, to be looped through
all permutations of the input vector elements to achieve equivariance. In our
network with pair-wise inner products between data points described in Section
II.2.2, we have two-indexed objects, $p_{i}\cdot p_{j}$, as inputs. For
convenience, we will from now on denote the two-indexed object with the
notation: $q_{ij}=p_{i}\cdot p_{j}$. The transformation of the pair-wise inner
products with respect to the permutation of the datapoints is not the same as
those considered in References [29, 26, 27]. To obtain some intuitions about
the permutation of the two-indexed objects, we first decompose the vector
$\vec{Q}=\\{q_{ij}\\}$ into two separate blocks of elements: the self-
interaction terms, $\vec{Q}_{\text{self}}=(q_{11},...,q_{ii})$, and the pair-
wise interaction terms, $\vec{Q}_{\text{pair}}=(q_{12},...,q_{ij})$ for $i\neq
j$. Under the $S_{n}$ permutation of point cloud data points, we observe that
both blocks transform into themselves. As a result, these two blocks of terms
are respectively transforming like $S_{n}$-permutated one-index vectors. It is
then easy to construct permutation-equivariant ansatz in both blocks by
applying the twirling formula separately. In Appendix A, we give the full
detail of constructing such an ansatz for the case of three input vectors.

The circuit in Figure 2 is an example of permutation and rotationally
invariant QNN with four vectors as inputs. We see that the inputs are divided
into two blocks: the first four elements are the self-interactions and the
remaining terms are the pair-wise interactions. The ansatz is built by
twirling the Pauli-$Y$ rotations and the entangling $ZZ$ rotations in both
blocks.

###  II.3 Loss function

The output of the QNN, $\hat{y}(p_{i};\theta)$, is compared to the truth label
of the events to perform classification. The loss function is taken to be the
standard Mean Squared Error (MSE) defined as:
$\mathcal{L}(\theta)=\frac{1}{4n}\sum_{i=0}^{n}(\hat{y}(p_{i};\theta)-y_{i})^{2}$,
where $n$ is the size of the training set and the extra factor of $4$ is used
to normalize the MSE into the range of $[0,1]$ given that the output is
between $-1$ and $1$, and the labels are given by $y\in\\{-1,1\\}$. The QNN
introduced in the last few subsections could be viewed as a linear function,
represented by the linear unitary function of the circuit, acting on the input
qubits. However, many real-world problems require nonlinear treatment of the
input data. In such cases, we introduce an ad-hoc nonlinear function at the
end of the QNN as follows.

$\begin{split}\mathcal{L^{\prime}}(\theta,b)=\sum_{i=0}^{n}(-|\hat{y}(p_{i};\theta)-b|-y_{i})^{2},\end{split}$
(10)

where $b$ is an offset parameter to be optimized by the learning process. The
function has a nonlinearity at $\hat{y}(p_{i};\theta)=b$ by introducing the
absolute difference between $\hat{y}(p_{i};\theta)$ and $b$. In this paper, we
consider the task of applying QNN to identify signal data, labeled by $y=1$,
out of datasets containing a mixture of signal and background data, in which
the background data is labeled by $y=-1$. Intuitively, the nonlinear function
in Equation (10) will be handy if the QNN output of the signal data is
concentrated at a particular value while background data has QNN outputs on
both sides. In such case, the function $-|\hat{y}(p_{i};\theta)-b|$ will learn
to spot such signal concentrations while it is generally difficult for a
linear model to do so. Since the offset parameter is explicitly defined in the
cost function, its derivative is well-defined for all values except at the
exact discontinuity. In each iteration, we update $b$, similar to other
parameters, through either gradient descent or COBYLA.

##  III Results

In this section, we provide the experimental results of the implementation of
the symmetric QNNs. In Section III.1, we create datasets with 2D rotations of
certain geometrical figures and test the performance of QNN with symmetry
against that of the baseline QNN. In Section III.2, we consider the particle
decay classification, which involves the $SO(1,3)$ Lorentz symmetry. In
Section III.3, we investigate the trainability of our models by numerically
evaluating the variance of the first-order gradients upon random
initializations. All numerical results presented here are obtained by the
classical quantum circuit simulator implemented via Qiskit [61].

###  III.1 Classification of 2D rotations

(a) Square template

(b) Triangle template

(c) Transformed square.

(d) Transformed triangle.

Figure 3: Templates and transformed images of the 2D dataset containing
squares and triangles. The templates of the two geometrical objects are shown
in the first two subfigures at the top. The two images at the bottom show two
examples of transformed images, in which the template went through spatial
translation, re-sizing, rotations, and smearings.

To test the performance of our new architecture, we first create a dataset
with rotated images of two geometrical objects: the square and the triangle.
The templates of these two objects are shown in Figure 3(a) and Figure 3(b),
respectively. The two templates consist of four data points, three of which
are placed in the same locations. To further impose hurdles on the learning
process, we consider various transformations to be applied to these templates.
The transformations are summarized in the following list:

(a) Loss function vs iterations.

(b) ROC curve.

Figure 4: The Loss function during the training process and the ROC curve for various QNN models in the 2D image classification task. The loss values shown in the figure are the median values of all ten initializations, whereas the error bands are computed at $25\%$ and $75\%$ quantiles. The ROC curves are obtained by averaging the training results and the error bands are obtained by taking the standard deviations. Methodology | Baseline | Rotation | Rotation + Permutation
---|---|---|---
$\\#$ parameters | 24 | 30 | 8
$\\#$ qubits | 8 | 10 | 10
Depth | 2 | 2 | 12
AUC | $0.720\pm 0.060$ | $0.880\pm 0.070$ | $0.966\pm 0.030$
Table 1: Specifications and performances of the QNNs used in the 2D image
classification task.

  * •

Translation: The data points in the templates are translated collectively as a
group in both the $x$ and $y$-axis. The maximum translational distance is set
at 5.

  * •

Re-sizing: The size of the template is varied by a factor randomly drawn
between 0.5 and 5.

  * •

Rotation: The template is rotated around its center by an arbitrary angle.

  * •

Shuffling: The order of data points in the dataset is subject to random
shuffling such that there is no preferred order in the four data points.

  * •

Smearing: Each data point in the template is smeared such that it is moved
around its nominal position by a random amount drawn between -0.5 and 0.5. The
smearing is performed independently for each data point.

Two examples of the transformed images are shown in Figure 3(c) and Figure
3(d). Overall, we created $1,600$ different images, each of which is a
template undergone all five random transformations listed above. In this
dataset, $1,200$ images are used as training data while the other $400$ are
used as testing data to verify performances.

(a) Loss function vs iterations.

(b) ROC curve.

Figure 5: The Loss function during the training process and the ROC curve for various QNN models in the particle decay identification task. The loss values shown in the figure are the median values of all ten initializations, whereas the error bands are computed at $25\%$ and $75\%$ quantiles. The ROC curves are obtained by averaging the training results and the error bands are obtained by taking the standard deviations. In the ROC curve, the QNN results are compared to the performance of the mass cut, which is regarded as the theoretical upper limit of performance. Methodology | Baseline | Rotation | Rotation + Permutation | Mass cut
---|---|---|---|---
$\\#$ parameters | 48 | 18 | 4 | N/A
$\\#$ qubits | 16 | 6 | 6 | N/A
Depth | 2 | 2 | 12 | N/A
AUC | $0.544\pm 0.060$ | $0.813\pm 0.076$ | $0.982\pm 0.011$ | $0.988\pm 0.001$
Table 2: Specifications and performances of the QNNs used in the particle
decay classification task.

We consider three models to tackle this 2D image classification problem: the
baseline model with no symmetry, the QNN model with rotational symmetry, and
the QNN model with full permutation and rotational symmetries. The baseline
model is the same as the one described in Section II.2. The input of the
baseline model uses the flattened array containing the $x$ and $y$ coordinates
of the data points in the image. The baseline model employs
$N_{\text{base}}=2$ layers of hardware efficient ansatz with 24 parameters in
total. We also consider a model that respects the rotational symmetry but not
the permutation symmetry. The rotationally symmetric model uses pair-wise
inner products between the four data points as inputs, giving
$4\cdot(4+1)/2=10$ qubits in total. This model also uses $N_{\text{rot}}=2$
layers of hardware efficient ansatz, which gives 30 parameters in total.
Finally, we construct the model with the full rotational and permutation
symmetries according to the recipe shown in Figure 2. It is clear that, due to
the symmetry constraints, the ansatz in the fully symmetric model contains a
much smaller number of parameters. In practice, the fully symmetric model has
$N_{\text{full}}=2$ layers of the ansatz with 8 parameters in total. However,
this fully symmetric model appears to be much more effective in the
classification task when compared to the baseline one. To obtain numerical
results, we train each model ten times, each with a different initialization.
The training results of all three models are shown in Figure 4(a), where the
regression loss is plotted against the number of iterations in training. The
median values of all initializations are shown in the figure with error bands
computed at the $25\%$ and $75\%$ quantiles. The regression loss is taken to
be the MSE loss, as defined in Section II.3, across the whole dataset. The
fully symmetric model is observed to have converged much faster than the two
alternative models. In terms of performance, we quantitatively measure the
classification performance using the Receiver Operating Characteristic (ROC)
curve shown in Figure 4(b). The performances of the trained models are
quantitatively measured by the Area Under the Curve (AUC) of their ROC curves,
which are listed in Table 1 333We also perform simulation for the baseline and
rotation equivariant models with the number of layers and the number of
iterations increased but the fully symmetric model presented here still gives
the best performance. The detail is given in Appendix B.. The circuit depth
shown in Table 1, obtained directly via Qiskit [61], is a measure of the
circuit’s effective number of layers during parallel execution. The QNN model
with full rotation and permutation symmetries has the best performance with an
AUC of $0.966\pm 0.030$.

###  III.2 Classification of high energy particle decay with ${SO}(1,3)$
Lorentz symmetry

Another task we have considered is the classification of particle decays. We
use a simulated Monte Carlo dataset that corresponds to the running conditions
at the large hadron collider, where protons collide at high energies to
produce various high-energy particles that subsequently decay [62]. In this
task, we use the Higgs boson decay as the signal process, where a Higgs boson
decays into four final state leptons through the $ZZ^{*}$ channel:
$H\xrightarrow[]{}ZZ^{*}\xrightarrow[]{}\ell^{\pm}\ell^{\mp}\ell^{\pm}\ell^{\mp}$.
The background consists of four lepton events from various Standard Model
processes excluding Higgs boson production. A total number of $6,000$ particle
decay events are used in the training, while $2,000$ events are used for
testing performances. For the baseline model, we use the four momenta of the
final state leptons as inputs. To enforce rotational symmetry, we acknowledge
that the ${SO}(1,3)$ Lorentz symmetry also satisfies the theorem given in
Equation (4) [53, 54, 55] and thus use the pair-wise inner products between
final state particles as inputs. In this task, the invariant masses of leptons
are negligible compared to the energy scale of the decay, which is at the
scale of the Higgs mass. As a result, it is convenient to drop the self-
interacting terms in the first block of Figure 2. In practice, this
implementation does not show any loss of performance when compared to the
model with the self-interactions. To further increase the performance of the
QNN models, we implement the nonlinear activation function in Equation (10) on
all QNN models for the particle decay classification problem.

The training loss is shown in Figure 5(a), in which we observe that the fully
symmetric model again converges much faster. In the ROC curve shown in Figure
5(b), we also include the traditional classification method of window cuts on
the four-body invariant mass. Since this traditional method directly yields
the Higgs mass and cuts around it, it could be viewed as the most optimal
method given the available information. The AUC of all models is shown in
Table 2, in which the fully symmetric model has achieved similar performance
as that of the mass cut method.

(a) Derivative variance versus input dimensions.

(b) Derivative variance versus number of input data points.

(c) Derivative variance versus number of qubits.

Figure 6: The variance of first-order derivatives of the QNN output. The
dimension and number of data points in the input data are varied to observe
their impact on the gradients. The variance is computed by sampling $100$
randomly initiated parameters. The third plot shows the number of qubits used
in the ansatz on the x-axis.

###  III.3 Barren plateau

In previous sections, the QNN with the full symmetries has been observed to
converge much faster during the training procedures. This can be further
investigated by checking the variance of the first-order gradients, which is
exponentially suppressed for particular models, such as circuits with large
depths, high expressibility, and high entanglement (see e.g. [58, 63, 56, 64,
65, 66]). We take the circuits used in Section III.1 and vary the
dimensionality and number of input data points to observe their impact on the
trainability of our models.

First, we fix the number of input data points to three and vary the dimension
of the point cloud input between two and four. The first-order derivatives are
sampled based on $100$ randomly initiated sets of parameters. Since the
baseline model requires more qubits as the dimension increases, we can easily
observe the barren plateau effect in Figure 6(a), where the derivative
variance of the baseline model decreases exponentially. However, since our
symmetric QNN model uses inner products as inputs, the increase in dimensions
has practically no impact on the gradients. We then proceed to fix the
dimension of the input at two and vary the number of data points between three
and five. Again, as shown in Figure 6(b), the baseline model exhibits clear
signs of a barren plateau due to its expressive nature. In the case of the
symmetric model, it does not appear to be expressive enough for the gradient
to vanish since we only used two layers. The exploitation of symmetries has
allowed us to use a much shallower circuit ansatz with fewer parameters that
had proven to be much more trainable. Finally, we combine the first two plots
into Figure 6(c) to show that the baseline model observes a clear barren
plateau phenomenon where the variance of derivatives decreases exponentially
as the number of qubits increases.

##  IV Discussion and conclusions

In this work, we have demonstrated the performance of the QNN model with
rotational and permutation symmetries. The fully symmetric model has shown
great potential in the task of classifying point cloud datasets, where the
traditional QNN model without symmetries struggles. The new model exploits the
fact that point cloud datasets often exhibit rotational and permutation
symmetries, which could be encoded into quantum circuits via taking inner
products of data vectors and using permutation equivariant ansatz.

In the classification tasks of 2D images, the fully symmetric QNN model
achieved an AUC of ROC curve above $0.966\pm 0.030$ while having converged
typically within a few dozen iterations. The fast converging is likely a
result of the lower number of parameters used in the symmetric circuits.
Similarly, in the task of the particle decay classification, the fully
symmetric model achieved an AUC of $0.982\pm 0.011$, which is on par with that
of the mass cut method. In both tasks, the number of parameters needed in the
symmetric circuit to achieve optimal results is proven to be much lower than
that in other QNN models. By construction, the symmetric circuit only searches
for solutions to the task that is invariant under the symmetry groups.
Intuitively, this greatly restricts the search space, in which the
optimization is conducted, and thus yields a much faster convergence speed
with a lower number of parameters. This observation is in agreement with
classical equivariant NN studies using the same principle in Reference [55].

An obvious limitation of our approach is that the input datasets are required
to be point clouds, in which the transformation of global rotational
symmetries is captured by taking inner products. For position-dependent local
symmetries, this approach is no longer valid. Another important note on this
model is that the number of qubits needed grows polynomially with respect to
the number of vectors in the input dataset. For a dataset with $n$ vectors in
$d$ dimensional space, the traditional method would require $n\cdot d$ number
of qubits, whereas our approach would require $n(n+1)/2$ qubits. Therefore,
our approach will require a lot more qubits when $n\gg d$. However, in the
appendix of Reference [54], the authors showed that, for any invariant
function to be expressed by the set of inner products, it is sufficient to use
only a subset of an approximate size of $n(d+1)$.

It would be an interesting study to further extend this approach to larger
datasets with more complex structures. In particular, we would like to point
out that an ensemble of quantum state vectors could be viewed as a form of
point clouds. In such cases, global unitary transformations acting on the
ensemble are equivalent to rotations of state vectors in the Hilbert space.
This hints that our approach might be useful in recognizing features of
quantum state ensembles that are invariant under unitary transformations.

Another avenue for future research is the choice of nonlinearity
implementation. Since our approach guarantees the permutation equivariance by
the twirling method, we are then obliged to measure all qubits to maintain the
equivariance, which essentially ruled out a popular choice of nonlinearity
based on measuring only a subset of qubits [19]. As a result, we implement an
ad hoc nonlinearity as shown in Equation (10). This particular choice of
nonlinearity is in no way the only option, nor is it proven to be an optimal
one. It would be a potentially fruitful study to investigate the introduction
of nonlinearities that respect the existing symmetries in the circuit.

###### Acknowledgements.

We would like to thank Dr. Chase Shimmin for the discussions that inspired
this project. This work is supported by the Center of Innovations for
Sustainable Quantum AI (JST Grant Number JPMJPF2221).

##  Appendix A Construction of permutation equivariant model

For $n=3$, we have:

$Q_{\text{pair}}=\begin{pmatrix}q_{12}\\\ q_{13}\\\
q_{23}\end{pmatrix}\,,\quad Q_{\text{self}}=\begin{pmatrix}q_{11}\\\ q_{22}\\\
q_{33}\end{pmatrix}\,.$ (11)

The representation matrices act on these vectors independently, in other
words, the representation is reducible: $V=V_{\text{pair}}\oplus
V_{\text{self}}$.

###  A.1 Pair vector

The corresponding representation of ${S}_{3}$ acting on the “pair” vector is
obtained as

$\displaystyle V_{\text{pair}}[\mathbb{I}_{{S}_{3}}]$
$\displaystyle=\begin{pmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\end{pmatrix}\,,$
$\displaystyle V_{\text{pair}}[\sigma_{(12)}]$
$\displaystyle=\begin{pmatrix}1&0&0\\\ 0&0&1\\\ 0&1&0\end{pmatrix}\,,$ (12)
$\displaystyle V_{\text{pair}}[\sigma_{(23)}]$
$\displaystyle=\begin{pmatrix}0&1&0\\\ 1&0&0\\\ 0&0&1\end{pmatrix}\,,$
$\displaystyle V_{\text{pair}}[\sigma_{(13)}]$
$\displaystyle=\begin{pmatrix}0&0&1\\\ 0&1&0\\\ 1&0&0\end{pmatrix}\,,$ (13)
$\displaystyle V_{\text{pair}}[\sigma_{(123)}]$
$\displaystyle=\begin{pmatrix}0&0&1\\\ 1&0&0\\\ 0&1&0\end{pmatrix}\,,$
$\displaystyle V_{\text{pair}}[\sigma_{(132)}]$
$\displaystyle=\begin{pmatrix}0&1&0\\\ 0&0&1\\\ 1&0&0\end{pmatrix}\,,$ (14)

where $\mathbb{I}_{{S}_{n}}\in{S}_{n}$ is the identity element in ${S}_{n}$
and $\sigma_{(i_{1}\cdots i_{k})}$ is a cycle 444If $\sigma\in{S}_{n}$
satisfies $\sigma(i_{1})=i_{2},\sigma(i_{2})=i_{3},\cdots,\sigma(i_{k})=i_{1}$
and $\sigma(i)=i$ for other $i$’s, then this is called a cycle and denoted by
$\sigma_{(i_{1},\cdots,i_{k})}$. For example, $\sigma_{(ij)}$ is a permutation
between $i$-th and $j$-th elements..

Suppose that we use the encoding $U_{\text{E}}(q)=R_{Z}(q_{12})\otimes
R_{Z}(q_{13})\otimes R_{Z}(q_{23})$, then the induced representation is given
by

$\displaystyle V_{\text{pair}}^{\text{(ind)}}[\mathbb{I}_{\mathcal{S}_{3}}]$
$\displaystyle=\mathbb{I}_{\mathcal{H}_{3}}\,,$ $\displaystyle
V_{\text{pair}}^{\text{(ind)}}[\sigma_{(12)}]$
$\displaystyle=\text{SWAP}_{\llbracket 13\rrbracket,\llbracket
23\rrbracket}\,,$ (15) $\displaystyle
V_{\text{pair}}^{\text{(ind)}}[\sigma_{(23)}]$
$\displaystyle=\text{SWAP}_{\llbracket{12}\rrbracket,\llbracket{13}\rrbracket}\,,$
$\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(13)}]$
$\displaystyle=\text{SWAP}_{\llbracket{12}\rrbracket,\llbracket{23}\rrbracket}\,,$
(16) $\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(123)}]$
$\displaystyle=\text{SWAP}_{\llbracket{12}\rrbracket,\llbracket{23}\rrbracket}\cdot\text{SWAP}_{\llbracket{12}\rrbracket,\llbracket{13}\rrbracket}\,,$
(17) $\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(132)}]$
$\displaystyle=\text{SWAP}_{\llbracket{12}\rrbracket,\llbracket{23}\rrbracket}\cdot\text{SWAP}_{\llbracket{13}\rrbracket,\llbracket{23}\rrbracket}\,.$
(18)

where $\text{SWAP}_{\llbracket ij\rrbracket,\llbracket kl\rrbracket}$ is the
SWAP operator between the qubits encoding $q_{ij}$ and $q_{kl}$ and
$\mathbb{I}_{\mathcal{H}_{n}}$ is the identity operator on the $n$-qubits
Hilbert space.

First, let us consider the $Y$-generators
$\mathcal{G}_{Y}=\\{Y_{\llbracket{12}\rrbracket},Y_{\llbracket{13}\rrbracket},Y_{\llbracket{23}\rrbracket}\\}$,
where we denote a Pauli matrix $P\in\\{I,X,Y,Z\\}$ acting on the qubit
encoding $q_{ij}$ as $P_{\llbracket{ij}\rrbracket}$. The twirling formula for
this representation leads to

$\displaystyle\mathcal{T}[Y_{\llbracket{12}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{12}\rrbracket}V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$
(19)
$\displaystyle=\frac{Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}+Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}}{6}$
(20)
$\displaystyle=\frac{1}{3}Y_{\llbracket{12}\rrbracket}+\frac{1}{3}Y_{\llbracket{13}\rrbracket}+\frac{1}{3}Y_{\llbracket{23}\rrbracket}\,,$
(21) $\displaystyle\mathcal{T}[Y_{\llbracket{13}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{13}\rrbracket}V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$
(22)
$\displaystyle=\frac{Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}+Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}+Y_{\llbracket{12}\rrbracket}}{6}$
(23)
$\displaystyle=\frac{1}{3}Y_{\llbracket{12}\rrbracket}+\frac{1}{3}Y_{\llbracket{13}\rrbracket}+\frac{1}{3}Y_{\llbracket{23}\rrbracket}\,,$
(24) $\displaystyle\mathcal{T}[Y_{\llbracket{23}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{23}\rrbracket}V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$
(25)
$\displaystyle=\frac{Y_{\llbracket{23}\rrbracket}+Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}+Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{13}\rrbracket}}{6}$
(26)
$\displaystyle=\frac{1}{3}Y_{\llbracket{12}\rrbracket}+\frac{1}{3}Y_{\llbracket{13}\rrbracket}+\frac{1}{3}Y_{\llbracket{23}\rrbracket}\,.$
(27)

Thus, the symmetrized ansatz can be constructed by

$\displaystyle\exp\left[i\left(\theta^{\prime}_{1}\mathcal{T}[Y_{\llbracket{12}\rrbracket}]+\theta^{\prime}_{2}\mathcal{T}[Y_{\llbracket{13}\rrbracket}]+\theta^{\prime}_{3}\mathcal{T}[Y_{\llbracket{23}\rrbracket}]\right)\right]=\exp\left[i\theta\left(Y_{\llbracket{12}\rrbracket}+Y_{\llbracket{13}\rrbracket}+Y_{\llbracket{23}\rrbracket}\right)\right]\,,$
(28)

where

$\displaystyle\theta$
$\displaystyle=\frac{1}{3}\theta^{\prime}_{1}+\frac{1}{3}\theta^{\prime}_{2}+\frac{1}{3}\theta^{\prime}_{3}\,.$
(29)

Next, let us consider $ZZ$-generators
$\mathcal{G}_{ZZ}=\\{Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket},Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket},Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\\}$.
The twirling formula gives

$\displaystyle\mathcal{T}[Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot(Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket})\cdot
V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$ (30)
$\displaystyle=\frac{1}{6}\big{[}Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+Z_{\llbracket{12}\rrbracket}Z_{\llbracket{23}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{12}\rrbracket}$
$\displaystyle\quad\quad\quad+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{13}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\big{]}$
(31)
$\displaystyle=\frac{1}{3}Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+\frac{1}{3}Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+\frac{1}{3}Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\,,$
(32)
$\displaystyle\mathcal{T}[Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot(Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket})\cdot
V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$ (33)
$\displaystyle=\frac{1}{6}\big{[}Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+Z_{\llbracket{12}\rrbracket}Z_{\llbracket{23}\rrbracket}$
$\displaystyle\quad\quad\quad+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{12}\rrbracket}+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}+Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}\big{]}$
(34)
$\displaystyle=\frac{1}{3}Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+\frac{1}{3}Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+\frac{1}{3}Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\,,$
(35)
$\displaystyle\mathcal{T}[Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{pair}}^{\text{(ind)}}[s]\cdot(Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket})\cdot
V_{\text{pair}}^{\text{(ind)}}[s]^{{\dagger}}$ (36)
$\displaystyle=\frac{1}{6}\big{[}Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{12}\rrbracket}+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{13}\rrbracket}+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}$
$\displaystyle\quad\quad\quad+Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}\big{]}$
(37)
$\displaystyle=\frac{1}{3}Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+\frac{1}{3}Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+\frac{1}{3}Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\,,$
(38)

Thus the equivariant ansatz is obtained as

$\displaystyle\quad\exp\left[i\left(\theta^{\prime}_{1}\mathcal{T}[Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}]+\theta^{\prime}_{2}\mathcal{T}[Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}]+\theta^{\prime}_{3}\mathcal{T}[Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}]\right)\right]$
$\displaystyle=\exp\left[i\theta\left(Z_{\llbracket{12}\rrbracket}Z_{\llbracket{13}\rrbracket}+Z_{\llbracket{13}\rrbracket}Z_{\llbracket{23}\rrbracket}+Z_{\llbracket{23}\rrbracket}Z_{\llbracket{12}\rrbracket}\right)\right]\,,$
(39)

where
$\theta=(\theta^{\prime}_{1}+\theta^{\prime}_{2}+\theta^{\prime}_{3})/3$.

###  A.2 Self vector

The representation acting on this space is given by

$\displaystyle V_{\text{self}}[\mathbb{I}_{\mathcal{S}_{3}}]$
$\displaystyle=\begin{pmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\end{pmatrix}\,,$
$\displaystyle V_{\text{self}}[\sigma_{(12)}]$
$\displaystyle=\begin{pmatrix}0&1&0\\\ 1&0&0\\\ 0&0&1\end{pmatrix}\,,$ (40)
$\displaystyle V_{\text{self}}[\sigma_{(23)}]$
$\displaystyle=\begin{pmatrix}1&0&0\\\ 0&0&1\\\ 0&1&0\end{pmatrix}\,,$
$\displaystyle V_{\text{self}}[\sigma_{(13)}]$
$\displaystyle=\begin{pmatrix}0&0&1\\\ 0&1&0\\\ 1&0&0\end{pmatrix}\,,$ (41)
$\displaystyle V_{\text{self}}[\sigma_{(123)}]$
$\displaystyle=\begin{pmatrix}0&1&0\\\ 0&0&1\\\ 1&0&0\end{pmatrix}\,,$
$\displaystyle V_{\text{self}}[\sigma_{(132)}]$
$\displaystyle=\begin{pmatrix}0&0&1\\\ 1&0&0\\\ 0&1&0\end{pmatrix}\,,$ (42)

The corresponding induced representation is obtained as

$\displaystyle V_{\text{pair}}^{\text{(ind)}}[\mathbb{I}_{{S}_{3}}]$
$\displaystyle=\mathbb{I}_{\mathcal{H}_{3}}\,,$ $\displaystyle
V_{\text{pair}}^{\text{(ind)}}[\sigma_{(12)}]$
$\displaystyle=\text{SWAP}_{\llbracket 11\rrbracket,\llbracket
22\rrbracket}\,,$ (43) $\displaystyle
V_{\text{pair}}^{\text{(ind)}}[\sigma_{(23)}]$
$\displaystyle=\text{SWAP}_{\llbracket{22}\rrbracket,\llbracket{33}\rrbracket}\,,$
$\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(13)}]$
$\displaystyle=\text{SWAP}_{\llbracket{11}\rrbracket,\llbracket{33}\rrbracket}\,,$
(44) $\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(123)}]$
$\displaystyle=\text{SWAP}_{\llbracket{22}\rrbracket,\llbracket{33}\rrbracket}\cdot\text{SWAP}_{\llbracket{11}\rrbracket,\llbracket{22}\rrbracket}\,,$
(45) $\displaystyle V_{\text{pair}}^{\text{(ind)}}[\sigma_{(132)}]$
$\displaystyle=\text{SWAP}_{\llbracket{11}\rrbracket,\llbracket{33}\rrbracket}\cdot\text{SWAP}_{\llbracket{11}\rrbracket,\llbracket{22}\rrbracket}\,.$
(46)

Then the twirling formula gives

$\displaystyle\mathcal{T}[Y_{\llbracket{11}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{self}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{12}\rrbracket}V_{\text{self}}^{\text{(ind)}}[s]^{{\dagger}}$
(47)
$\displaystyle=\frac{1}{3}Y_{\llbracket{11}\rrbracket}+\frac{1}{3}Y_{\llbracket{12}\rrbracket}+\frac{1}{3}Y_{\llbracket{33}\rrbracket}\,,$
(48) $\displaystyle\mathcal{T}[Y_{\llbracket{22}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{self}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{22}\rrbracket}V_{\text{self}}^{\text{(ind)}}[s]^{{\dagger}}$
(49)
$\displaystyle=\frac{1}{3}Y_{\llbracket{11}\rrbracket}+\frac{1}{3}Y_{\llbracket{12}\rrbracket}+\frac{1}{3}Y_{\llbracket{33}\rrbracket}\,,$
(50) $\displaystyle\mathcal{T}[Y_{\llbracket{33}\rrbracket}]$
$\displaystyle=\frac{1}{6}\sum_{s\in{S}_{3}}V_{\text{self}}^{\text{(ind)}}[s]\cdot
Y_{\llbracket{33}\rrbracket}V_{\text{self}}^{\text{(ind)}}[s]^{{\dagger}}$
(51)
$\displaystyle=\frac{1}{3}Y_{\llbracket{11}\rrbracket}+\frac{1}{3}Y_{\llbracket{22}\rrbracket}+\frac{1}{3}Y_{\llbracket{33}\rrbracket}\,.$
(52)

Thus the equivariant ansatz is

$\displaystyle\exp\left[i\left(\theta^{\prime}_{1}\mathcal{T}[Y_{\llbracket{11}\rrbracket}]+\theta^{\prime}_{2}\mathcal{T}[Y_{\llbracket{22}\rrbracket}]+\theta^{\prime}_{3}\mathcal{T}[Y_{\llbracket{33}\rrbracket}]\right)\right]=\exp\left[i\theta\left(Y_{\llbracket{11}\rrbracket}+Y_{\llbracket{22}\rrbracket}+Y_{\llbracket{33}\rrbracket}\right)\right]\,.$
(53)

##  Appendix B Performances with alternative settings

In Section III, we used two layers of hardware efficient ansatz for both the
baseline and the rotational invariant QNN. In this appendix, we increase the
number of layers to four and also train for $400$ iterations to investigate
the performances of more expressive models. Since the fully symmetric model
performs well in Section III, we kept it in the same configuration.

We consider the same task of 2D image classification presented in Section
III.1. The training loss function is shown in Figure 7(a) and the ROC curve is
shown in Figure 7(b). We observe that the performances of the baseline and the
rotation-only model are about the same as those listed in Table 1. The extra
layers in the ansatz increase the model complexities and thus require mode
iterations for the model to converge to the optimized solution. It could be
argued that the performance of the two models might be on par with that of the
fully symmetric model if it has a sufficient number of layers and was trained
for large numbers of iterations far beyond the scope of this experiment.
However, given the limited computational resources, we claim it is sufficient
to observe a clear advantage in the fully symmetric model since it yields the
best performance with a much faster convergence speed due to the lower number
of parameters.

Methodology | Baseline | Rotation | Rotation + Permutation
---|---|---|---
$\\#$ parameters | 40 | 50 | 8
$\\#$ qubits | 8 | 10 | 10
Depth | 4 | 4 | 12
AUC | $0.772\pm 0.033$ | $0.901\pm 0.039$ | $0.966\pm 0.030$
Table 3: Specifications and performances of the QNNs used in the 2D image
classification task. The fully symmetric model is the same as the one
presented in Section III.1. The baseline and the rotationally invariant models
have four layers of hardware-efficient ansatz and are trained for $400$
iterations.

(a) Loss function vs iterations.

(b) ROC curve.

Figure 7: The Loss function during the training process and the ROC curve for
various QNN models in the 2D image classification task. The fully symmetric
model is the same as the one presented in Section III.1. The baseline and the
rotationally invariant models have four layers of hardware-efficient ansatz
and are trained for $400$ iterations. The loss values shown in the figure are
the median values of all ten initializations, whereas the error bands are
computed at $25\%$ and $75\%$ quantiles. The ROC curves are obtained by
averaging the training results and the error bands are obtained by taking the
standard deviations.

## References

  * Schuld  _et al._ [2014] M. Schuld, I. Sinayskiy, and F. Petruccione, An introduction to quantum machine learning, Contemporary Physics 56, 172–185 (2014).
  * Biamonte  _et al._ [2017] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, Quantum machine learning, Nature 549, 195–202 (2017).
  * Mangini  _et al._ [2021] S. Mangini, F. Tacchino, D. Gerace, D. Bajoni, and C. Macchiavello, Quantum computing models for artificial neural networks, Europhysics Letters 134, 10002 (2021).
  * Cerezo  _et al._ [2022] M. Cerezo, G. Verdon, H.-Y. Huang, L. Cincio, and P. Coles, Challenges and opportunities in quantum machine learning, Nature Computational Science 2 (2022).
  * Mitarai  _et al._ [2018] K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, Quantum circuit learning, Physical Review A 98, 10.1103/physreva.98.032309 (2018).
  * Farhi and Neven [2018] E. Farhi and H. Neven, Classification with quantum neural networks on near term processors, arXiv preprint arXiv:1802.06002 (2018), arXiv:1802.06002 .
  * Benedetti  _et al._ [2019a] M. Benedetti, E. Lloyd, S. Sack, and M. Fiorentini, Parameterized quantum circuits as machine learning models, Quantum Sci. Technol. 4, 043001 (2019a), arXiv:1906.07682 [quant-ph] .
  * Schuld  _et al._ [2020] M. Schuld, A. Bocharov, K. M. Svore, and N. Wiebe, Circuit-centric quantum classifiers, Phys. Rev. A 101, 032308 (2020).
  * Guan  _et al._ [2021] W. Guan, G. Perdue, A. Pesah, M. Schuld, K. Terashi, S. Vallecorsa, and J.-R. Vlimant, Quantum machine learning in high energy physics, Machine Learning: Science and Technology 2, 011003 (2021).
  * Mott  _et al._ [2017] A. Mott, J. Job, J.-R. Vlimant, D. Lidar, and M. Spiropulu, Solving a higgs optimization problem with quantum annealing for machine learning, Nature 550, 375 (2017).
  * Terashi  _et al._ [2021] K. Terashi, M. Kaneda, T. Kishimoto, M. Saito, R. Sawada, and J. Tanaka, Event classification with quantum machine learning in high-energy physics, Computing and Software for Big Science 5, 10.1007/s41781-020-00047-7 (2021).
  * Wu  _et al._ [2021a] S. L. Wu, J. Chan, W. Guan, S. Sun, A. Wang, C. Zhou, M. Livny, F. Carminati, A. Di Meglio, A. C. Y. Li, J. Lykken, P. Spentzouris, S. Y.-C. Chen, S. Yoo, and T.-C. Wei, Application of quantum machine learning using the quantum variational classifier method to high energy physics analysis at the LHC on IBM quantum computer simulator and hardware with 10 qubits, Journal of Physics G: Nuclear and Particle Physics 48, 125003 (2021a).
  * Blance and Spannowsky [2021] A. Blance and M. Spannowsky, Quantum machine learning for particle physics using a variational quantum classifier, Journal of High Energy Physics 2021, 10.1007/jhep02(2021)212 (2021).
  * Wu  _et al._ [2021b] S. L. Wu, S. Sun, W. Guan, C. Zhou, J. Chan, C. L. Cheng, T. Pham, Y. Qian, A. Z. Wang, R. Zhang, M. Livny, J. Glick, P. K. Barkoutsos, S. Woerner, I. Tavernelli, F. Carminati, A. Di Meglio, A. C. Y. Li, J. Lykken, P. Spentzouris, S. Y.-C. Chen, S. Yoo, and T.-C. Wei, Application of quantum machine learning using the quantum kernel algorithm on high energy physics analysis at the LHC, Physical Review Research 3, 10.1103/physrevresearch.3.033221 (2021b).
  * Wozniak  _et al._ [2023] K. A. Wozniak, V. Belis, E. Puljak, P. Barkoutsos, G. Dissertori, M. Grossi, M. Pierini, F. Reiter, I. Tavernelli, and S. Vallecorsa, Quantum anomaly detection in the latent space of proton collision events at the LHC, (2023), arXiv:arXiv:2301.10780 [quant-ph] .
  * Schuhmacher  _et al._ [2023] J. Schuhmacher, L. Boggia, V. Belis, E. Puljak, M. Grossi, M. Pierini, S. Vallecorsa, F. Tacchino, P. Barkoutsos, and I. Tavernelli, Unravelling physics beyond the standard model with classical and quantum anomaly detection, Machine Learning: Science and Technology 4, 045031 (2023).
  * LeCun and Bengio [1998] Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, in  _The Handbook of Brain Theory and Neural Networks_ (MIT Press, Cambridge, MA, USA, 1998) p. 255–258.
  * O’Shea and Nash [2015] K. O’Shea and R. Nash, An introduction to convolutional neural networks, (2015), arXiv:arXiv:1511.08458 [cs.NE] .
  * Cong  _et al._ [2019] I. Cong, S. Choi, and M. D. Lukin, Quantum convolutional neural networks, Nature Physics 15, 1273–1278 (2019).
  * Hur  _et al._ [2022] T. Hur, L. Kim, and D. K. Park, Quantum convolutional neural network for classical data classification, Quantum Machine Intelligence 4, 10.1007/s42484-021-00061-x (2022).
  * Monaco  _et al._ [2023] S. Monaco, O. Kiss, A. Mandarino, S. Vallecorsa, and M. Grossi, Quantum phase detection generalization from marginal quantum neural network models, Phys. Rev. B 107, L081105 (2023).
  * Herrmann  _et al._ [2022] J. Herrmann _et al._ , Realizing quantum convolutional neural networks on a superconducting quantum processor to recognize quantum phases, Nature Commun. 13, 4144 (2022), arXiv:2109.05909 [quant-ph] .
  * Nagano  _et al._ [2023] L. Nagano, A. Miessen, T. Onodera, I. Tavernelli, F. Tacchino, and K. Terashi, Quantum data learning for quantum simulations in high-energy physics, Phys. Rev. Res. 5, 043250 (2023), arXiv:2306.17214 [quant-ph] .
  * Wrobel  _et al._ [2021] N. Wrobel, A. Baul, J. Moreno, and K.-M. Tam, An application of quantum machine learning on quantum correlated systems: Quantum convolutional neural network as a classifier for many-body wavefunctions from the quantum variational eigensolver, (2021), arXiv:arXiv:2111.05076 [cond-mat.str-el] .
  * Chinzei  _et al._ [2024] K. Chinzei, Q. H. Tran, K. Maruyama, H. Oshima, and S. Sato, Splitting and parallelizing of quantum convolutional neural networks for learning translationally symmetric data, Phys. Rev. Res. 6, 023042 (2024).
  * Nguyen  _et al._ [2022] Q. T. Nguyen, L. Schatzki, P. Braccia, M. Ragone, P. J. Coles, F. Sauvage, M. Larocca, and M. Cerezo, Theory for equivariant quantum neural networks, (2022), arXiv:arXiv:2210.08566 [quant-ph] .
  * Meyer  _et al._ [2023] J. J. Meyer, M. Mularski, E. Gil-Fuster, A. A. Mele, F. Arzani, A. Wilms, and J. Eisert, Exploiting symmetry in variational quantum machine learning, PRX Quantum 4, 10.1103/prxquantum.4.010328 (2023).
  * Ragone  _et al._ [2022] M. Ragone, P. Braccia, Q. T. Nguyen, L. Schatzki, P. J. Coles, F. Sauvage, M. Larocca, and M. Cerezo, Representation Theory for Geometric Quantum Machine Learning, (2022), arXiv:arXiv:2210.07980 [quant-ph] .
  * Schatzki  _et al._ [2024] L. Schatzki, M. Larocca, Q. Nguyen, F. Sauvage, and M. Cerezo, Theoretical guarantees for permutation-equivariant quantum neural networks, npj Quantum Information 10 (2024).
  * T West  _et al._ [2023] M. T West, M. Sevior, and M. Usman, Reflection equivariant quantum neural networks for enhanced image classification, Machine Learning: Science and Technology 4, 035027 (2023).
  * Chang  _et al._ [2023] S. Y. Chang, M. Grossi, B. Le Saux, and S. Vallecorsa, Approximately equivariant quantum neural network for p4m group symmetries in images, in  _2023 IEEE International Conference on Quantum Computing and Engineering (QCE)_ (IEEE, 2023) arXiv:2310.02323 .
  * West  _et al._ [2024] M. T. West, J. Heredge, M. Sevior, and M. Usman, Provably trainable rotationally equivariant quantum machine learning, (2024), arXiv:arXiv:2311.05873 [quant-ph] .
  * Sebastian  _et al._ [2024] P. S. Sebastian, M. Cañizo, and R. Orús, Image classification with rotation-invariant variational quantum circuits, (2024), arXiv:arXiv:2403.15031 [quant-ph] .
  * Zheng  _et al._ [2023a] H. Zheng, Z. Li, J. Liu, S. Strelchuk, and R. Kondor, Speeding up learning quantum states through group equivariant convolutional quantum ansätze, PRX Quantum 4, 020327 (2023a).
  * Zheng  _et al._ [2022a] H. Zheng, C. Kang, G. S. Ravi, H. Wang, K. Setia, F. T. Chong, and J. Liu, SnCQA: A hardware-efficient equivariant quantum convolutional circuit architecture 10.1109/QCE57702.2023.00034 (2022a), arXiv:2211.12711 [quant-ph] .
  * Das and Caruso [2024] S. Das and F. Caruso, Permutation-equivariant quantum convolutional neural networks, (2024), arXiv:arXiv:2404.18198 [quant-ph] .
  * Mansky  _et al._ [2023] M. B. Mansky, S. L. n. Castillo, V. R. Puigvert, and C. Linnhoff-Popien, Permutation-invariant quantum circuits, (2023), arXiv:arXiv:2312.14909 [quant-ph] .
  * Heredge  _et al._ [2023] J. Heredge, C. Hill, L. Hollenberg, and M. Sevior, Permutation invariant encodings for quantum machine learning with point cloud data, (2023), arXiv:arXiv:2304.03601 [quant-ph] .
  * Zheng  _et al._ [2023b] H. Zheng, Z. Li, J. Liu, S. Strelchuk, and R. Kondor, Speeding Up Learning Quantum States Through Group Equivariant Convolutional Quantum Ansätze, PRX Quantum 4, 020327 (2023b), arXiv:2112.07611 [quant-ph] .
  * Le  _et al._ [2023] I. N. M. Le, O. Kiss, J. Schuhmacher, I. Tavernelli, and F. Tacchino, Symmetry-invariant quantum machine learning force fields, (2023), arXiv:arXiv:2311.11362 [quant-ph] .
  * Li  _et al._ [2023] Z. Li, H. Zheng, Y. Wang, L. Jiang, Z.-W. Liu, and J. Liu, SU(d)-Symmetric Random Unitaries: Quantum Scrambling, Error Correction, and Machine Learning, (2023), arXiv:arXiv:2309.16556 [quant-ph] .
  * East  _et al._ [2023] R. D. P. East, G. Alonso-Linaje, and C.-Y. Park, All you need is spin: SU(2) equivariant variational quantum circuits based on spin networks, (2023), arXiv:arXiv:2309.07250 [quant-ph] .
  * Zheng  _et al._ [2022b] H. Zheng, Z. Li, J. Liu, S. Strelchuk, and R. Kondor, On the Super-exponential Quantum Speedup of Equivariant Quantum Machine Learning Algorithms with SU($d$) Symmetry (2022) arXiv:2207.07250 [quant-ph] .
  * Skolik  _et al._ [2023] A. Skolik, M. Cattelan, S. Yarkoni, T. Bäck, and V. Dunjko, Equivariant quantum circuits for learning on weighted graphs, npj Quantum Inf. 9, 47 (2023), arXiv:2205.06109 [quant-ph] .
  * Mernyei  _et al._ [2021] P. Mernyei, K. Meichanetzidis, and I. I. Ceylan, Equivariant Quantum Graph Circuits, (2021), arXiv:2112.05261 [cs.LG] .
  * Verdon  _et al._ [2019] G. Verdon, T. McCourt, E. Luzhnica, V. Singh, S. Leichenauer, and J. Hidary, Quantum Graph Neural Networks, (2019), arXiv:1909.12264 [quant-ph] .
  * Sauvage  _et al._ [2024] F. Sauvage, M. Larocca, P. J. Coles, and M. Cerezo, Building spatial symmetries into parameterized quantum circuits for faster training, Quantum Sci. Technol. 9, 015029 (2024), arXiv:2207.14413 [quant-ph] .
  * Cohen and Welling [2016] T. Cohen and M. Welling, Group equivariant convolutional networks, in  _Proceedings of The 33rd International Conference on Machine Learning_, Proceedings of Machine Learning Research, Vol. 48, edited by M. F. Balcan and K. Q. Weinberger (PMLR, New York, New York, USA, 2016) pp. 2990–2999.
  * Cohen  _et al._ [2018] T. S. Cohen, M. Geiger, J. Koehler, and M. Welling, Spherical cnns, in  _International Conference on Learning Representations (ICLR)_ (2018) arXiv:1801.10130 .
  * Weiler  _et al._ [2018] M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. Cohen, 3d steerable cnns: Learning rotationally equivariant features in volumetric data, in  _5th International Conference on Learning Representations, ICLR_ (2018) arXiv:1807.02547 .
  * Veeling  _et al._ [2018] B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, and M. Welling, Rotation equivariant cnns for digital pathology, in  _Medical Image Computing and Computer Assisted Intervention–MICCAI_ (2018) arXiv:1806.03962 .
  * Shimmin [2021] C. Shimmin, Particle convolution for high energy physics, (2021), arXiv:arXiv:2107.02908 [hep-ph] .
  * Weyl [1946] H. Weyl,  _The Classical Groups: Their Invariants and Representations_, Princeton landmarks in mathematics and physics (Princeton University Press, 1946).
  * Villar  _et al._ [2021] S. Villar, D. W. Hogg, K. Storey-Fisher, W. Yao, and B. Blum-Smith, Scalars are universal: Equivariant machine learning, structured like classical physics, in  _Neural Information Processing Systems_ (2021).
  * Bogatskiy  _et al._ [2022] A. Bogatskiy, T. Hoffman, D. W. Miller, and J. T. Offermann, PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics, (2022), arXiv:2211.00454 [hep-ph] .
  * Cerezo  _et al._ [2021] M. Cerezo, A. Sone, T. Volkoff, L. Cincio, and P. J. Coles, Cost function dependent barren plateaus in shallow parametrized quantum circuits, Nature Communications 12, 10.1038/s41467-021-21728-w (2021).
  * Schuld  _et al._ [2019] M. Schuld, V. Bergholm, C. Gogolin, J. Izaac, and N. Killoran, Evaluating analytic gradients on quantum hardware, Physical Review A 99, 10.1103/physreva.99.032331 (2019).
  * McClean  _et al._ [2018] J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, Barren plateaus in quantum neural network training landscapes, Nature Communications 9, 10.1038/s41467-018-07090-4 (2018).
  * Powell [1994] M. J. D. Powell, A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation (1994).
  * Benedetti  _et al._ [2019b] M. Benedetti, E. Lloyd, S. Sack, and M. Fiorentini, Parameterized quantum circuits as machine learning models, Quantum Science and Technology 4, 043001 (2019b).
  * [61] Qiskit contributors, Qiskit: An open-source framework for quantum computing, zenodo.org/records/8190968 10.5281/zenodo.2573505.
  * Javier Duarte [2020] H. P. Javier Duarte, Fermilab LHC physics center machine learning hands-on advanced tutorial session datasets, https://doi.org/10.5281/zenodo.3901869 10.5281/zenodo.3901869 (2020).
  * Holmes  _et al._ [2022] Z. Holmes, K. Sharma, M. Cerezo, and P. J. Coles, Connecting ansatz expressibility to gradient magnitudes and barren plateaus, PRX Quantum 3, 10.1103/prxquantum.3.010313 (2022).
  * Patti  _et al._ [2021] T. L. Patti, K. Najafi, X. Gao, and S. F. Yelin, Entanglement devised barren plateau mitigation, Phys. Rev. Res. 3, 033090 (2021).
  * Sharma  _et al._ [2022] K. Sharma, M. Cerezo, L. Cincio, and P. J. Coles, Trainability of dissipative perceptron-based quantum neural networks, Phys. Rev. Lett. 128, 180505 (2022).
  * Ortiz Marrero  _et al._ [2021] C. Ortiz Marrero, M. Kieferová, and N. Wiebe, Entanglement-induced barren plateaus, PRX Quantum 2, 040316 (2021).