# Towards Antisymmetric Neural Ansatz Separation

Aaron Zweig  Courant Institute of Mathematical Sciences, New York University,
New York  Joan Bruna  Courant Institute of Mathematical Sciences, New York
University, New York  Center for Data Science, New York University

###### Abstract

We study separations between two fundamental models (or _Ansätze_) of
antisymmetric functions, that is, functions $f$ of the form
$f(x_{\sigma(1)},\ldots,x_{\sigma(N)})=\text{sign}(\sigma)f(x_{1},\ldots,x_{N})$,
where $\sigma$ is any permutation. These arise in the context of quantum
chemistry, and are the basic modeling tool for wavefunctions of Fermionic
systems. Specifically, we consider two popular antisymmetric Ansätze: the
Slater representation, which leverages the alternating structure of
determinants, and the Jastrow ansatz, which augments Slater determinants with
a product by an arbitrary symmetric function. We construct an antisymmetric
function in $N$ dimensions that can be efficiently expressed in Jastrow form,
yet provably cannot be approximated by Slater determinants unless there are
exponentially (in $N^{2}$) many terms. This represents the first explicit
quantitative separation between these two Ansätze.

##  1 Introduction

Neural networks have proven very successful in parametrizing non-linear
approximation spaces in high-dimensions, thanks to the ability of neural
architectures to leverage the physical structure and symmetries of the problem
at hand, while preserving universal approximation. The successes cover many
areas of engineering and computational science, from computer vision [14] to
protein folding [13].

In each case, modifying the architecture (e.g. by adding layers, adjusting the
activation function, etc.) has intricate effects in the approximation,
statistical and optimization errors. An important aspect in this puzzle is to
first understand the approximation abilities of a certain neural architecture
against a class of target functions having certain assumed symmetry [16, 2].
For instance, symmetric functions that are permutation-invariant, ie
$f(x_{\sigma(1)},\ldots,x_{\sigma(N)})=f(x_{1},\ldots x_{N})$ for all
$x_{1},\ldots,x_{N}$ and all permutations $\sigma:\\{1,N\\}\to\\{1,N\\}$ can
be universally approximated by several neural architectures, e.g DeepSets [29]
or Set Transformers [17]; their approximation properties [30] thus offer a
first glimpse on their efficiency across different learning tasks.

In this work, we focus on quantum chemistry applications, namely
characterizing ground states of many-body quantum systems. These are driven by
the fundamental Schröndinger equation, an eigenvalue problem of the form

$H\Psi=\lambda\Psi~{},$

where $H$ is the Hamiltonian associated to a particle system defined over a
product space $\Omega^{\otimes N}$, and $\Psi$ is the _wavefunction_ , a
complex-valued function $\Psi:\Omega^{\otimes N}\to\mathbb{C}$ whose squared
modulus $|\Psi(x_{1},\ldots,x_{N})|^{2}$ describes the probability of
encountering the system in the state $(x_{1},\ldots,x_{N})\in\Omega^{\otimes
N}$. A particularly important object is to compute the _ground state_ ,
associated with the smallest eigenvalue of $H$. On Fermionic systems, the
wavefunction satisfies an additional property, derived from Pauli’s exclusion
principle: the wavefunction is _antisymmetric_ , meaning that

$\Psi(x_{\sigma(1)},\ldots,x_{\sigma(N)})=\text{sign}(\sigma)\Psi(x_{1},\ldots,x_{N})~{}.$
(1)

The antisymmetric constraint is an uncommon one, and therefore demands unique
architectures to enforce it. The quintessential antisymmetric function is a
_Slater determinant_ [26], that we now briefly describe. Given functions
$f_{1},\dots,f_{N}:\Omega\to\mathbb{C}$, they define a rank-one tensor mapping
$f_{1}\otimes\dots\otimes f_{N}:\Omega^{\otimes N}\to\mathbb{C}$ by
$(f_{1}\otimes\dots\otimes f_{N})(x_{1},\ldots,x_{N}):=\prod_{j\leq
N}f_{j}(x_{j})$. The Slater determinant is then the orthogonal projection of a
tensor rank one function into antisymmetric space. In other words, the rank
one tensor $f_{1}\otimes\dots\otimes f_{N}$ is projected to
$\mathcal{A}(f_{1}\otimes\dots\otimes f_{N}):=\frac{1}{N!}\sum_{\sigma\in
S_{N}}(-1)^{\sigma}f_{\sigma(1)}\otimes\dots\otimes f_{\sigma(N)}$. In
coordinates, this expression becomes

$\displaystyle\mathcal{A}(f_{1}\otimes\dots\otimes
f_{N})(x)=\frac{1}{N!}\text{det}\begin{bmatrix}f_{1}(x_{1})&\ldots&f_{1}(x_{N})\\\
f_{2}(x_{1})&\ldots&f_{2}(x_{N})\\\ &\dots&\\\
f_{N}(x_{1})&\ldots&f_{N}(x_{N})\end{bmatrix}~{},$

which shows that is antisymmetric following the alternating property of the
determinant.

The Slater Ansatz is then simply a linear combination of several Slater
determinants, of the form $F(x)=\sum_{l\leq
L}\mathcal{A}(f_{1}^{l}\otimes\dots\otimes f_{N}^{l})$, similarly as a shallow
(Euclidean) neural network formed as a linear combination of simple non-linear
ridge functions. While this ansatz defines a universal approximation class for
antisymmetric functions (as a direct consequence of Weierstrass universal
approximation theorems for polynomials), the approximation rates will
generally be cursed by the dimensionality of the input space, as is also the
case when studying lower bounds for standard shallow neural networks [19].

In the case of particles in $\Omega=\mathbb{R}$ or $\mathbb{C}$, it is
classical that all antisymmetric functions can be written as a product of a
symmetric function with the Vandermonde (see Section 3). This setting is
generally considered much easier than settings with higher-dimensional
particles, as this Vandermonde factorization no longer applies above one
dimension, though there are still ansätze that mimic this formulation [7].

A more powerful variant is the Jastrow Ansatz, where each Slater determinant
is ‘augmented’ with a symmetric prefactor [12], ie
$G=p\cdot\mathcal{A}(f_{1}\otimes\dots\otimes f_{N})$ where $p$ is
permutation-invariant. Clearly, $G$ is still antisymmetric, since the product
of an antisymmetric function with a symmetric one is again antisymmetric, but
grants more representational power. Other parametrisations building from
Jastrow are popularly used in the literature, e.g. _backflow_ [5], which
models particle interactions by composing the Slater determinant with a
permutation equivariant change of variables. Among practitioners, it is common
knowledge that the Slater Ansatz is inefficient, compared to Jastrow or other
more advanced parameterizations. Yet, there is no proven separation evinced by
a particular hard antisymmetric function. We note that the Jastrow ansatz is
strictly generalized by backflow (see Section 3), so separations between
Slater and Jastrow would have immediate consequences for separations from the
stronger architectures as well.

In this work, we are interested in understanding quantitative differences in
approximation power between these two classes. Specifically, we wish to find
antisymmetric target functions $G$ such that $G$ can be efficiently
approximated with the Jastrow ansatz, i.e. approximated to $\epsilon$ error in
the infinity norm with some modest dependence on the parameters $N$ and
$\epsilon$ by a single Slater determinant with a single symmetric prefactor,
yet no Slater representation can approximate $G$ for reasonably small widths.
This question mirrors the issue of depth separation in deep learning theory,
where one seeks functions that exhibit a separation between, for example, two
layer and three layer networks [4], as well as recent separations between
classes of symmetric representations [30].

#### Main Contribution:

We prove the first explicit separation between the two ansätze, and construct
an antisymmetric function $G$ such that:

  * •

In some norm, $G$ cannot be approximated better than constant error by the
Slater ansatz, unless there are $\Omega(e^{N^{2}})$ many Slater Determinants.

  * •

$G$ can be written in the Jastrow ansatz with neural network widths bounded by
$poly(N)$ using analytic activation functions

##  2 Related Work

###  2.1 Machine Learning for Quantum Chemistry

Numerous works explore how to use neural network parameterizations effectively
to solve Schrödinger’s equation. These include works in first quantization,
which try to parameterize the wavefunction $\Psi$ directly [22, 8], and second
quantization, where the wavefunction is restricted to an exponentially large
but finite-dimensional Hilbert space, and then the problem is mapped to a spin
system [1].

###  2.2 Antisymmetric Ansätze

Numerous architectures enforce antisymmetry. In this work we focus primarily
on the Slater ansatz and Jastrow ansatz, but others exist and are used in
practice, with associated guarantees of universality [6]. The backflow ansatz
enables interaction between particles while preserving antisymmetry [18]. More
recently, an ansatz that introduces hidden additional fermions was introduced
in [21].

###  2.3 Architecture Separations

A large body of work studies the difference in expressive power between
different neural network architectures. These works frequently center on the
representational gap between two-layer and three-layer networks [4, 3].
Relatedly, several works have considered the representational power of
different networks architectures constrained to be symmetric [27, 28, 30].

The most closely related work to ours is [9], which proves a non-constructive
limit on the representability of the backflow ansatz, but requires exact
representation rather than approximation in some norm. Conversely, [10]
demonstrates the universality of a single backflow ansatz, but requires a
highly discontinuous backflow transform that may not be efficiently
representable with a neural network.

##  3 Preliminaries and Main Theorem

###  3.1 Antisymmetric Ansätze

We consider $N$ particles restricted to the complex unit circle. That is,
$x\in\Omega^{N}$ with $\Omega=\\{z\in\mathbb{C};|z|=1\\}$. We denote the
tensor product $\otimes$ where, for $f,g:\Omega\rightarrow\mathbb{C}$, we have
$f\otimes g:\Omega^{2}\rightarrow\mathbb{C}$ such that $(f\otimes
g)(x,y)=f(x)g(y)$. Given a permutation $\sigma\in\mathcal{S}_{N}$, and
$x\in\Omega^{N}$, we denote by
$\sigma.x=(x_{\sigma(1)},\ldots,x_{\sigma(N)})\in\Omega^{N}$ the natural group
action.

Let $\mathcal{A}$ denote the antisymmetric projection, defined via:

$\displaystyle\mathcal{A}(\phi_{1}\otimes\dots\otimes\phi_{N})=\frac{1}{N!}\sum_{\sigma\in
S_{N}}(-1)^{\sigma}\phi_{\sigma(1)}\otimes\dots\otimes\phi_{\sigma(N)}$ (2)

So up to rescaling we can consider Slater determinants as terms of the form
$\mathcal{A}(\phi_{1}\otimes\dots\otimes\phi_{N})$. Each $\phi_{n}$ is called
an _orbital_. Intuitively, a Slater determinant is the simplest way to write
an antisymmetric function, inheriting the antisymmetry property from the
determinant itself.

Thus the Slater determinant ansatz with $L$ terms can be written as:

$\displaystyle F=\sum_{l=1}^{L}\mathcal{A}(f_{1}^{l}\otimes\dots\otimes
f_{N}^{l})~{}.$ (3)

Similarly, the Jastrow ansatz (with only one term) [12] can be written as:

$\displaystyle G=p\cdot\mathcal{A}(g_{1}\otimes\dots\otimes g_{N})$ (4)

where $p$ is a symmetric function, namely $p(\sigma.x)=p(x)$ for any $\sigma$
and $x$. It is immediate to verify that $G$ is antisymmetric. Finally, the
Backflow ansatz [5] (considering again a single term) is defined as

$\displaystyle\widetilde{G}(x)=\mathcal{A}(\tilde{g}_{1}\otimes\dots\tilde{g}_{N})(\Phi(x))~{},$
(5)

where $\Phi:\Omega^{N}\to\widetilde{\Omega}^{N}$ is an equivariant flow,
satisfying $\Phi(\sigma.x)=\sigma.\Phi(x)$, and where in general
$\widetilde{\Omega}$ might be higher-dimensional than $\Omega$.

In particular, we verify that

$\displaystyle\Phi:\Omega^{N}$ $\displaystyle\to(\mathbb{C}\times\Omega)^{N},$
(6) $\displaystyle x$
$\displaystyle\mapsto\Phi(x):=((p^{1/N}(x);x_{1}),\dots,(p^{1/N}(x);x_{N}))$
(7)

is equivariant. Given a collection of $N$ orbitals
$\phi_{1},\ldots\phi_{N}:\Omega\to\mathbb{C}$, we verify that the Jastrow
Ansatz $G=p\cdot\mathcal{A}(g_{1}\otimes\dots\otimes g_{N})$ can be written as
$G=\mathcal{A}(\tilde{g}_{1}\otimes\dots\tilde{g}_{N})\circ\Phi$, with
$\tilde{g}_{j}:\mathbb{C}\times\Omega\to\mathbb{C}$ defined as
$\tilde{g}_{j}(z,x)=z\cdot g_{j}(x)$. Thus, the Jastrow ansatz can be
recovered as a particular case of the more general backflow ansatz. Therefore,
quantitative separations between Slater and Jastrow automatically imply the
same rates for Backflow.

###  3.2 Inner Products

To measure the distance between the Slater Determinant ansatz and the Jastrow
ansatz, we need an appropriate norm.

For univariate functions $f,g:{S^{1}}\rightarrow\mathbb{C}$, define the inner
product:

$\displaystyle\langle
f,g\rangle:=\frac{1}{(2\pi)}\int_{0}^{2\pi}f(e^{i\theta})\overline{g(e^{i\theta})}d\theta~{}.$
(8)

For functions acting on $N$ particles,
$F,G:({S^{1}})^{N}\rightarrow\mathbb{C}$, the associated inner product is

$\displaystyle\langle
F,G\rangle:=\frac{1}{(2\pi)^{N}}\int_{[0,2\pi]^{N}}F(e^{i\bm{\theta}})\overline{G(e^{i\bm{\theta}})}d\bm{\theta}~{}.$
(9)

Let us introduce the notation that for $x\in\mathbb{C}^{N}$ and
$\alpha\in\mathbb{N}^{N}$, $x^{\alpha}=\prod_{i=1}^{N}x_{i}^{\alpha_{i}}$.
Then the orthogonality of the Fourier basis may be written as $\langle
x^{\alpha},x^{\beta}\rangle=\delta_{\alpha\beta}$.

###  3.3 Theorem Statement

With this, we may state our main result explicitly:

######  Theorem 3.1.

Consider a Slater ansatz with $L$ terms:

$\displaystyle F=\sum_{l=1}^{L}\mathcal{A}(f_{1}^{l}\otimes\dots\otimes
f_{N}^{l})$ (10)

parameterized by orbitals $f_{n}^{l}:{S^{1}}\rightarrow\mathbb{C}$, and a
Jastrow ansatz

$\displaystyle\hat{G}=p\cdot\mathcal{A}(g_{1}\otimes\dots\otimes g_{N})$ (11)

parameterized by orbitals $g_{n}:{S^{1}}\rightarrow\mathbb{C}$ and a symmetric
Jastrow factor $p:({S^{1}})^{N}\rightarrow\mathbb{C}$.

Let $N\geq 6$ and $1>\epsilon>0$. Then there is a hard function $G$ with
$\|G\|=1$, such that $\hat{G}$ parameterized by a neural network with width,
depth, and weights scaling in $O(poly(N,1/\epsilon))$ that can approximate
$G$:

$\displaystyle\|\hat{G}-G\|_{\infty}<\epsilon$ (12)

but, for a number of Slater determinants $L\leq e^{N^{2}}$:

$\displaystyle\min_{F}\|F-G\|^{2}\geq\frac{3}{10}~{}.$ (13)

Note the constraint $\|G\|=1$ is just to prevent vacuous solutions by scaling
$G$ to be arbitrarily big. We will describe the exact network structure of the
hard function $G$ in Section 5.4.

We also remark that $N^{O(N)}\ll e^{N^{2}}$, so this is a true separation.
Furthermore, the construction only requires $N^{O(N)}$ parameters due to the
analysis of a generic complex ReLU activation. For particular choices of the
activation, the network only requires $poly(N)$ parameters. Finally, the
restriction of $\hat{G}$ to only have one determinant is artificial, here to
demonstrate the nature of the separation. In practice, the Jastrow ansatz
allows for multiple determinant terms in learning.

###  3.4 Schur Polynomials

To build up the difficult function $G$, we use several identities related to
the symmetric Schur polynomials. First, we introduce partitions as they will
be used to index Schur polynomials:

centertableaux 5,4,1

(a) $\lambda=(5,4,1)$

centertableaux 3,2,2,2,1

(b) $\lambda^{\prime}=(3,2,2,2,1)$

Figure 1: Example of Young diagram and conjugate partition

######  Definition 3.2.

An _integer partition_ $\lambda$ is non-increasing, finite sequence of
positive integers $\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{k}$. The
weight of the partition is given by $|\lambda|=\sum_{i=1}^{k}\lambda_{i}$. The
length of a partition $l(\lambda)$ is the number of terms in the sequence. We
call a partition _even_ if every $\lambda_{i}$ is even.

Partitions can be represented by their Young diagram, see Figure 1.
Furthermore, we will need the notion of a conjugate partition:

######  Definition 3.3.

Given a partition $\lambda$, the _conjugate partition_ $\lambda^{\prime}$ is
gotten by reflecting the Young diagram of $\lambda$ along the line $y=-x$. We
call a partition _doubly even_ if $\lambda$ and $\lambda^{\prime}$ are both
even.

First, we introduce the Vandermonde written as:

$\displaystyle V(x)=\prod_{i<j}(x_{j}-x_{i})~{}.$ (14)

Then we denote the Schur polynomial indexed by partition $\lambda$ as:

$\displaystyle
s_{\lambda}(x):=\begin{cases}V(x)^{-1}\text{det}\left[x_{i}^{\lambda_{j}+N-j}\right]&l(\lambda)\leq
N~{},\\\ 0&l(\lambda)>N~{}.\end{cases}$ (15)

Given two partitions $\lambda$ and $\mu$, the following fact follows easily
from linearity of the determinant:

$\displaystyle\langle s_{\lambda}\cdot V,s_{\mu}\cdot
V\rangle=N!\cdot\delta_{\lambda\mu}~{}.$ (16)

We will in the sequel assume $N$ is even, then we can cite the following
formal identity of a particular Pfaffian:

######  Theorem 3.4 ([25] Theorem 5.2, [11] Corollary 4.2).

$\displaystyle\sum_{\lambda\text{ doubly even}}s_{\lambda}\cdot V$
$\displaystyle=\mathrm{Pf}\left[\frac{x_{i}-x_{j}}{1-x_{i}^{2}x_{j}^{2}}\right]$
(17) $\displaystyle=\prod_{i<j}\frac{1}{1-x_{i}^{2}x_{j}^{2}}\cdot
N!\cdot\mathcal{A}(\phi_{1}\otimes\dots\otimes\phi_{N})~{},$ (18)

where we set the $\phi$ maps to be:

$\displaystyle\phi_{j}(x_{i})=\begin{cases}x_{i}(x_{i}^{2})^{N/2-j}(1+x_{i}^{4})^{j-1}&1\leq
j\leq N/2\\\ (x_{i}^{2})^{N-j}(1+x_{i}^{4})^{j-1-N/2}&N/2+1\leq j\leq
N\end{cases}$ (19)

We note the requirement for $N$ to be even is to ensure the existence of
doubly even partitions. We will leverage this identity to show the existence
of a function that requires many Slater determinants to approximate, but can
nevertheless be written in the form of the Jastrow ansatz.

##  4 Proof Sketch

We give here a sketch of the tactic of the proof. Suppose we consider the
domain of our antisymmetric functions to be only a finite set of points.
Equivalently, we simplify the problem to a question of antisymmetric tensor
products of vectors instead of functions. Then inapproximability of Slater
determinants amounts to finding a high-rank antisymmetric tensor, that cannot
be approximated in the $l_{2}$ norm by a small number of rank-one tensors.

Let’s further simplify, and forget about antisymmetry for a moment. The usual
trick for questions of high-rank tensors, is to flatten all the tensors and
rewrite them as matrices, so the problem reduces to approximating a high rank
matrix by a low rank one. This is handily solved by SVD. To make the SVD
tractable, the high-rank matrix is typically taken to be diagonal.

In our setting, we cannot simply choose a tensor that will be diagonal after
flattening, because the constraints of antisymmetry will enforce certain
matrix elements to be equal. It turns out we can focus on a particular
subtensor, where it’s possible to flatten to a diagonal matrix, while
nevertheless keeping our hard function representable by the Jastrow ansatz.

Indeed, the hard function $G$ can be written exactly in the Jastrow ansatz by
the above identity:

$\displaystyle G$ $\displaystyle:=\frac{C}{\sqrt{N!}}\sum_{\lambda\text{
doubly even}}r^{\left(|\lambda|+\frac{N(N-1)}{2}\right)}s_{\lambda}\cdot V$
(20)
$\displaystyle=C\sqrt{N!}\cdot\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\cdot\mathcal{A}(\phi_{1}^{(r)}\otimes\dots\otimes\phi_{N}^{(r)})~{},$
(21)

where $\phi_{j}^{(r)}(z)=\phi_{j}(rz)$ and $C,r$ are constants with $|r|<1$.
The second equality gives a Jastrow ansatz. So it remains to demonstrate why
the first expression in terms of Schur polynomials is difficult to
approximate, i.e. why it encodes a high-rank antisymmetric tensor.

Let us explain the significance of restricting the support to doubly even
Schur functions. Let $\delta=(N-1,N-2,\dots,1,0)$. Then by simply canceling
the Vandermonde factor, for an appropriate partition $\lambda$, we have:

$s_{\lambda}(x)\cdot
V(x)=\text{det}\left[x_{i}^{\lambda_{j}+\delta_{j}}\right]$

Furthermore, if $\lambda$ is doubly even, then $\lambda+\delta$ will take the
form $(2a+1,2a,2b+1,2b,\dots)$ with alternating odd and even terms with the
odd term one above the subsequent even term. See Figure 2 for an example.

The significance of this structure is that, by knowing only the odd values of
$\lambda+\delta$, the even values are determined. We will make essential use
of this property to flatten an antisymmetric tensor into a matrix with sets of
odd indices as rows, and sets of even indices as columns. For this matrix, the
above structure implies diagonality, and from there we can proceed with a
usual proof of low-rank approximation from SVD.

The complete proof is given in Appendix 5.

centertableaux 4,4,2,2

(a) $\lambda=(4,4,2,2)$

centertableaux

[*(white) ∙] 4+3,4+2,2+1,2+0 *[*(white)]4,4,2,2

(b) $\lambda+\delta=(7,6,3,2)$

Figure 2: $\lambda$ and $\lambda+\delta$ for $\lambda$ doubly even.
Note that $\lambda+\delta\sim(6+1,2+1)\cup(6,2).$

##  5 Proof

###  5.1 Building the hard function $G$

For any $r\in\mathbb{R}$ with $|r|<1$, by mapping $x\mapsto rx$ and using
homogeneity of $s_{\lambda}$ and $V$, we define $G$ via the generating
function identity:

$\displaystyle G:=\frac{C}{\sqrt{N!}}\sum_{\lambda\text{ doubly
even}}r^{\left(|\lambda|+\frac{N(N-1)}{2}\right)}s_{\lambda}\cdot V$
$\displaystyle=C\sqrt{N!}\cdot\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\cdot\mathcal{A}(\phi_{1}^{(r)}\otimes\dots\otimes\phi_{N}^{(r)})~{},$
(22)

where

$\displaystyle\phi_{j}^{(r)}(x_{i})=\begin{cases}rx_{i}((rx_{i})^{2})^{N/2-j}(1+(rx_{i})^{4})^{j-1}&\text{
if }1\leq j\leq N/2\\\ ((rx_{i})^{2})^{N-j}(1+(rx_{i})^{4})^{j-1-N/2}&\text{
if }N/2+1\leq j\leq N\end{cases}$ (23)

where $C$ is chosen to normalize $G$.

Note that from the RHS, it is clear that $G$ is written in the form of a
Jastrow ansatz. We will discuss efficiency of computing $G$ further below.

It remains to choose $r$ and $C$ such that $\|G\|=1$. Note that, if $p(k)$
denotes the number of partitions of $k$, and and $p^{\prime}(k)$ denotes the
number of doubly even partitions of $k$ , it’s easy to see that

$\displaystyle p^{\prime}(k)=\begin{cases}p(k/4)&k\equiv 0\mod 4\\\
0&else\end{cases}$ (24)

So we calculate by orthogonality:

$\displaystyle\|G\|^{2}$
$\displaystyle=\frac{C^{2}}{N!}\left\langle\sum_{\lambda\text{ doubly
even}}r^{\left(|\lambda|+\frac{N(N-1)}{2}\right)}s_{\lambda}\cdot
V,\sum_{\mu\text{ doubly
even}}r^{\left(|\mu|+\frac{N(N-1)}{2}\right)}s_{\mu}\cdot V\right\rangle$ (25)
$\displaystyle=C^{2}r^{N(N-1)}\sum_{\lambda\text{ doubly even}}r^{2|\lambda|}$
(26) $\displaystyle=C^{2}r^{N(N-1)}\sum_{k=0}^{\infty}r^{2k}p^{\prime}(k)$
(27) $\displaystyle=C^{2}r^{N(N-1)}\sum_{k=0}^{\infty}r^{8k}p(k)$ (28)
$\displaystyle=C^{2}r^{N(N-1)}\prod_{k=1}^{\infty}\frac{1}{1-r^{8k}}$ (29)

where in the last line we employ the generating function for partition
numbers. Then setting
$C=\left(r^{-N(N-1)}\prod_{k=1}^{\infty}1-r^{8k}\right)^{1/2}$ gives
$\|G\|=1$.

###  5.2 From Tensors to Matrices

The point of choosing $G$ in this way, is it enables a simple flattening
argument, where we can reduce comparing tensors to comparing matrices.

Note again that terms of the form $x^{\alpha}$ for $\alpha\in\mathbb{N}^{N}$
are orthonormal. Hence, we derive an initial lower bound by Bessel’s
inequality:

$\displaystyle\|F-G\|^{2}$
$\displaystyle\geq\sum_{\alpha\in\mathbb{N}^{N}}\left(\left\langle
F,x^{\alpha}\right\rangle-\left\langle
G,x^{\alpha}\right\rangle\right)^{2}~{}.$ (30)

Note that by antisymmetry of $F$ and $G$, if $\alpha$ doesn’t have distinct
elements then

$\displaystyle\left\langle F,x^{\alpha}\right\rangle=\left\langle
G,x^{\alpha}\right\rangle=0~{}.$ (31)

To see this, suppose $\alpha_{1}=\alpha_{2}$, and let $P_{12}$ be the
permutation operator defined by

$\displaystyle P_{12}F(x_{1},x_{2},x_{3},\dots)=F(x_{2},x_{1},x_{3},\dots)$
(32)

It’s easy to see $P_{12}$ is a symmetric operator with respect to
$\langle\cdot,\cdot\rangle$. Then for any antisymmetric function $H$,

$\displaystyle\langle H,x^{\alpha}\rangle$ $\displaystyle=\langle
H,P_{12}x^{\alpha}\rangle$ (33) $\displaystyle=\langle
P_{12}H,x^{\alpha}\rangle$ (34) $\displaystyle=-\langle H,x^{\alpha}\rangle$
(35)

which implies $\langle H,x^{\alpha}\rangle=0$.

Furthermore, let us define the equivalence class $\sim$ as via
$\alpha\sim\alpha^{\prime}$ if there exists a permutation $\pi$ such that
$\alpha=\pi\circ\alpha^{\prime}$. Then by similar reasoning,
$\alpha\sim\alpha^{\prime}$ implies:

$\displaystyle\left\langle F,x^{\alpha}\right\rangle$
$\displaystyle=(-1)^{\pi}\left\langle F,x^{\alpha^{\prime}}\right\rangle$ (36)
$\displaystyle\left\langle G,x^{\alpha}\right\rangle$
$\displaystyle=(-1)^{\pi}\left\langle G,x^{\alpha^{\prime}}\right\rangle$ (37)

So define $\mathbb{N}_{\geq}^{N}$ to be the set of strictly decreasing non-
negative integer vectors of length $N$, then we have:

$\displaystyle\|F-G\|^{2}$ $\displaystyle\geq
N!\cdot\sum_{\alpha\in\mathbb{N}_{\geq}^{N}}\left(\left\langle
F,x^{\alpha}\right\rangle-\left\langle G,x^{\alpha}\right\rangle\right)^{2}$
(38)

Now, we can consider a flattening argument, by passing from tensors to
matrices. Define

$\displaystyle\mathfrak{A}_{1}$
$\displaystyle=\\{\beta\in\mathbb{N}_{\geq}^{N/2}:\beta_{i}\equiv 1\mod 2\\}$
(39) $\displaystyle\mathfrak{A}_{2}$
$\displaystyle=\\{\gamma\in\mathbb{N}_{\geq}^{N/2}:\gamma_{i}\equiv 0\mod
2\\}$ (40)

For $\beta\in\mathfrak{A}_{1}$ and $\gamma\in\mathfrak{A}_{2}$, let
$\beta\cup\gamma\in\mathbb{N}^{N}$ be the concatenation of $\beta$ and
$\gamma$.

Then given a function acting on $N$ particles such as $G$, we can map $G$ to a
(infinite-dimensional) matrix $M$ indexed by the sets $\mathfrak{A}_{1}$ and
$\mathfrak{A}_{2}$:

$\displaystyle M(G)=\left[\langle
G,x^{\beta\cup\gamma}\rangle\right]_{\beta,\gamma}$ (41)

Let us calculate the entries of this matrix. Let $\delta=(N-1,N-2,\dots,1,0)$,
and observe that:

$\displaystyle\langle s_{\lambda}\cdot
V,x^{\beta\cup\gamma}\rangle=\begin{cases}\pm
1&\lambda+\delta\sim\beta\cup\gamma~{},\\\ 0&\text{otherwise.}\end{cases}$
(42)

Note that ambiguity in sign depends on the sign of the permutation that maps
$\lambda+\delta$ to $\beta\cup\gamma$.

By definition, $G$ is a sum of terms of the form $s_{\lambda}\cdot V$ where
$\lambda$ is doubly even. This implies that
$\lambda+\delta=(2a_{1}+1,2a_{1},2a_{2}+1,2a_{2},\dots)$ with $a_{i}>a_{i+1}$.
In other words, $\lambda+\delta\sim(\gamma+\mathbf{1})\cup\gamma$ with
$\gamma+\mathbf{1}\in\mathfrak{A}_{1}$ and $\gamma\in\mathfrak{A}_{2}$, where
$\mathbf{1}$ is the all-ones vector. See Figure 2 for an example.

It follows that we may write:

$\displaystyle\langle
G,x^{\beta\cup\gamma}\rangle=\begin{cases}\pm\frac{C}{\sqrt{N!}}r^{\left(|\lambda|+\frac{N(N-1)}{2}\right)}&\beta=(\gamma+\mathbf{1}),\quad\lambda+\delta\sim(\gamma+\mathbf{1})\cup\gamma~{},\\\
0&\text{otherwise.}\end{cases}$ (43)

Suppose we index $M(G)$ such that the $i$th column is indexed by
$\gamma^{(i)}$, and the $i$th row is indexed by $\gamma^{(i)}+\mathbf{1}$.
Then $M(G)$ is in fact a diagonal matrix. And given the functional form of
$G$, we have that the diagonal terms will include:

  * •

$\pm\frac{C}{\sqrt{N!}}r^{\left(0+\frac{N(N-1)}{2}\right)}$ repeated $p(0)$
times,

  * •

$\pm\frac{C}{\sqrt{N!}}r^{\left(4+\frac{N(N-1)}{2}\right)}$ repeated $p(1)$
times,

  * •

$\pm\frac{C}{\sqrt{N!}}r^{\left(8+\frac{N(N-1)}{2}\right)}$ repeated $p(2)$
times,

  * •

…

  * •

$\pm\frac{C}{\sqrt{N!}}r^{\left(4k+\frac{N(N-1)}{2}\right)}$ repeated $p(k)$
times.

Second, let us consider $M(f_{1}\otimes\dots\otimes f_{N})$. We can calculate
the inner product of a rank-one function as the product of orbital inner
products:

$\displaystyle\langle f_{1}\otimes\dots\otimes
f_{N},x^{\beta\cup\gamma}\rangle$ $\displaystyle=\prod_{n=1}^{N/2}\langle
f_{n},y^{\beta_{n}}\rangle\prod_{n=1}^{N/2}\langle
f_{N/2+n},y^{\gamma_{n}}\rangle$ (44)

where we introduce $y\in\mathbb{C}$ as a one-dimensional dummy variable for
integration.

Define vectors $u\in\mathbb{C}^{|\mathfrak{A}_{1}|}$ and
$v\in\mathbb{C}^{|\mathfrak{A}_{2}|}$ such that

$\displaystyle u_{\beta}$ $\displaystyle=\prod_{n=1}^{N/2}\langle
f_{n},y^{\beta_{n}}\rangle~{},$ (45) $\displaystyle v_{\gamma}$
$\displaystyle=\prod_{n=1}^{N/2}\langle f_{N/2+n},y^{\gamma_{n}}\rangle~{}.$
(46)

Then it’s clear that $M(f_{1}\otimes\dots\otimes f_{N})=uv^{T}$, i.e. it is
rank-one. Consequently, because $F$ is the sum of $L\cdot N!$ rank-one
tensors, $M(F)$ will be rank at most $L\cdot N!$.

So we finally pass from tensors to matrices, and lower bound via the Frobenius
norm $\|\cdot\|_{F}$:

$\displaystyle\|F-G\|^{2}$ $\displaystyle\geq
N!\cdot\sum_{\alpha\in\mathbb{N}_{\geq}^{N}}\left(\left\langle
F,x^{\alpha}\right\rangle-\left\langle G,x^{\alpha}\right\rangle\right)^{2}$
(47) $\displaystyle\geq N!\cdot\|M(F)-M(G)\|_{F}^{2}$ (48)

Thus, because $M(F)$ is low-rank and $M(G)$ is chosen to be diagonal, we have
an approachable infinite-dimensional matrix low-rank approximation problem.

###  5.3 Deriving the bound

By SVD, the optimal choice for $F$ is to produce a diagonal matrix $M(F)$ of
rank $L\cdot N!$ with the maximal singular values of $G$ along the diagonal.
So it only remains to calculate these terms, and lower bound the
approximation.

So suppose we choose $L\leq e^{N^{2}}$. Noting that $N^{N}\leq e^{N^{2}}/14$
for $N\geq 6$:

$\displaystyle L\cdot N!$ $\displaystyle\leq e^{N^{2}}N^{N}$ (49)
$\displaystyle\leq e^{2N^{2}}/14$ (50) $\displaystyle\leq p(N^{4})$ (51)

where the last line follows from Corollary 3.1 in [20].

So clearly $L\leq e^{N^{2}}$ guarantees that $L\cdot
N!\leq\sum_{k=0}^{N^{4}}p(k)$.

Thus, since $M(F)$ is constrained to have rank $\leq L\cdot N!$, it will be
diagonal with $\leq\sum_{k=0}^{N^{4}}p(k)$ terms, so that:

$\displaystyle\|F-G\|^{2}$ $\displaystyle\geq N!\cdot\|M(F)-M(G)\|_{F}^{2}$
(52) $\displaystyle\geq
N!\cdot\sum_{k=N^{4}+1}^{\infty}\left(\pm\frac{C}{\sqrt{N!}}r^{\left(4k+\frac{N(N-1)}{2}\right)}\right)^{2}p(k)$
(53) $\displaystyle=C^{2}r^{N(N-1)}\sum_{k=N^{4}+1}^{\infty}r^{8k}p(k)$ (54)
$\displaystyle=1-C^{2}r^{N(N-1)}\sum_{k=0}^{N^{4}}r^{8k}p(k)$ (55)

where the last line follows as $C$ was chosen so that
$C^{2}r^{N(N-1)}\sum_{k=0}^{\infty}r^{8k}p(k)=1$.

Note that

$\displaystyle\sum_{k=0}^{N^{4}}r^{8k}p(k)\leq\prod_{k=1}^{N^{4}}\frac{1}{1-r^{8k}}$
(56)

as the LHS is the generating function for partitions $\lambda$ with
$|\lambda|\leq N^{4}$, and the RHS is the generating function for partitions
with all parts $\leq N^{4}$, which clearly dominates the LHS termwise.

So plugging back in the definition of
$C=\left(r^{-N(N-1)}\prod_{k=1}^{\infty}1-r^{8k}\right)^{1/2}$:

$\displaystyle\|F-G\|^{2}$ $\displaystyle\geq
1-C^{2}r^{N(N-1)}\prod_{k=1}^{N^{4}}\frac{1}{1-r^{8k}}$ (57)
$\displaystyle=1-\prod_{k=N^{4}+1}^{\infty}1-r^{8k}~{}.$ (58)

Finally, by choosing $r=1-\frac{1}{8N^{4}+8}$, we have:

$\displaystyle 1-\prod_{k=N^{4}+1}^{\infty}1-r^{8k}$ $\displaystyle\geq
1-\left(1-r^{8N^{4}+8}\right)$ (59)
$\displaystyle=\left(1-\frac{1}{8N^{4}+8}\right)^{8N^{4}+8}$ (60)
$\displaystyle\geq\left(1-\frac{1}{16}\right)^{16}$ (61)
$\displaystyle\geq\frac{3}{10}~{},$ (62)

where we use that the limit $\left(1-\frac{1}{n}\right)^{n}$ increases
monotonically in $n$. Hence, we conclude:

$\displaystyle\|F-G\|^{2}$ $\displaystyle\geq\frac{3}{10}~{}.$ (63)

###  5.4 Efficiency of representing $G$

We remind the representation, where with $r=1-\frac{1}{8N^{4}+8}$ we have:

$\displaystyle
G=C\sqrt{N!}\cdot\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\cdot\mathcal{A}(\phi_{1}^{(r)}\otimes\dots\otimes\phi_{N}^{(r)})$
(64)

with

$\displaystyle\phi_{j}^{(r)}(x_{i})=\begin{cases}rx_{i}((rx_{i})^{2})^{N/2-j}(1+(rx_{i})^{4})^{j-1}&1\leq
j\leq N/2~{},\\\ ((rx_{i})^{2})^{N-j}(1+(rx_{i})^{4})^{j-1-N/2}&N/2+1\leq
j\leq N~{}.\end{cases}$ (65)

So it remains to characterize some $\hat{G}$ in the Jastrow ansatz,
parameterized with neural networks, that approximates $G$.

Given inputs of the form $z\in\mathbb{C}^{n}$, we consider network layers of
the form $z\mapsto\sigma(Az+B\overline{z}+c)$ where $A,B,c$ are learnable
weights, and $\sigma$ is an analytic activation function with coefficients
that decay at least factorially (such as $\exp$, $\sinh$ or $\sin$). It is
easy to see by the complex Stone-Weierstrauss theorem that linear combinations
of these terms, i.e. two layer networks, are universal approximators for
continuous complex-valued functions.

Observe further, the following fact:

######  Lemma 5.1.

Fix $J$ and let $\gamma$ be a primitive $J$th root of unity. Then

$\displaystyle\frac{1}{J}\sum_{j=0}^{J-1}\gamma^{ij}=\begin{cases}1&i\equiv
0\mod J\\\ 0&i\not\equiv 0\mod J\end{cases}$ (66)

We will use this fact to exponentially well-approximate all the components of
the hard function $G$.

######  Lemma 5.2.

For any $k,J\in\mathbb{N}$ with $2ek<J$, there exists a shallow neural network
$f^{(k)}$ using either the $\exp,\sinh,\sin$ activations, with $O(J)$ neurons
and O(k) weights, such that

$\displaystyle\sup_{|\xi|\leq 2}\left|f^{(k)}(\xi)-\xi^{k}\right|$
$\displaystyle\leq 2\left(\frac{2ek}{J}\right)^{J}$ (67)

###### Proof.

The result is obvious if $k=0$, so assume $k\geq 1$.

Let $\gamma$ be a primitive $J$th root of unity, $t\in\mathbb{C}$ some
constant, and let $k\in\mathbb{N}$ such that $0\leq k\leq J-1$. Let $\sigma$
denote a chosen activation, with Taylor expansion
$\sigma(z)=\sum_{i=0}^{\infty}c_{i}z^{i}$.

By applying Lemma 5.1 we can define a network $f^{(k)}$ and expand as:

$\displaystyle f^{(k)}(\xi)$
$\displaystyle:=\sum_{j=0}^{J-1}\frac{\gamma^{-kj}}{J}\sigma(\gamma^{j}t\xi)$
(69)
$\displaystyle=\sum_{j=0}^{J-1}\frac{\gamma^{-kj}}{J}\sum_{i=0}^{\infty}c_{i}(\gamma^{j}t\xi)^{i}$
(70)
$\displaystyle=\sum_{i=0}^{\infty}c_{i}(t\xi)^{i}\left[\frac{1}{J}\sum_{j=0}^{J-1}\gamma^{(i-k)j}\right]$
(71) $\displaystyle=\sum_{i=0}^{\infty}c_{i}(t\xi)^{i}\mathbbm{1}_{i\equiv
k\mod J}$ (72) $\displaystyle=\sum_{i=0}^{\infty}c_{iJ+k}(t\xi)^{iJ+k}$ (73)
$\displaystyle=c_{k}(t\xi)^{k}+\sum_{i=1}^{\infty}c_{iJ+k}(t\xi)^{iJ+k}$ (74)

Let’s choose $t=c_{k}^{-1/k}$. Therefore, it follows:

$\displaystyle\sup_{|\xi|\leq
1}|f^{(k)}(\xi)-\xi^{k}|\leq\sum_{i=1}^{\infty}|c_{iJ+k}||(2t)^{iJ+k}|$ (75)

If $\sigma(\xi)=\exp(\xi)$, it follows $c_{k}=\frac{1}{k!}$. So by Stirling’s
approximation we have $|t|\leq k$ and therefore if $J>2ek$, then:

$\displaystyle\sum_{i=1}^{\infty}|c_{iJ+k}||(2t)^{iJ+k}|$
$\displaystyle\leq\sum_{i=1}^{\infty}\frac{(2k)^{iJ+k}}{(iJ+k)!}$ (76)
$\displaystyle\leq\sum_{i=1}^{\infty}\left(\frac{2ek}{J}\right)^{iJ}$ (77)
$\displaystyle\leq 2\left(\frac{2ek}{J}\right)^{J}$ (78)

We cannot apply this argument directly for the $\sin$ activation, because it
is an odd function, and $c_{k}=0$ if $k$ is even. However, a subset of
networks with the $\sin$ activations is networks with the activation
$\sigma(\xi)=\sin\xi+\sin(\pi/2-\xi)=\sin\xi+\cos\xi$, for which the above
argument applies as before.

Similarly, using the identity $\sinh(\xi+1)=\sinh\xi\cosh 1+\cosh\xi\sinh 1$,
the same bound applies with the activation $\sinh$.

∎

We consider first the Jastrow factor of $G$. We will approximate it in
$\hat{G}$ using a Relational Network [24] with multiplication pooling. In what
follows, we consider the infinity norm restricted to the unit complex circle.

Define $y_{ij}=f^{(2)}(x_{i}+x_{j})-f^{(2)}(x_{i})-f^{(2)}(x_{j})$. Note that
because $x_{i}x_{j}=(x_{i}+x_{j})^{2}-x_{i}^{2}-x_{j}^{2}$, by Lemma 5.2, we
have for an appropriate choice of $J$:

$\displaystyle\|y_{ij}-x_{i}x_{j}\|_{\infty}$
$\displaystyle\leq\|f^{(2)}(x_{i}+x_{j})-(x_{i}+x_{j})^{2}\|_{\infty}+\|f^{(2)}(x_{i})-x_{i}^{2}\|_{\infty}+\|f^{(2)}(x_{j})-x_{j}^{2}\|_{\infty}$
(79) $\displaystyle\leq 6\left(\frac{4e}{J}\right)^{J}$ (80)

Clearly it follows that for sufficiently large $J$, $\|y_{ij}\|_{\infty}\leq
2$. Therefore, applying Lemma 5.2 again we have

$\displaystyle\left\|f^{(2k)}(y_{ij})-x_{i}^{2k}x_{j}^{2k}\right\|_{\infty}$
$\displaystyle\leq\left\|f^{(2k)}(y_{ij})-y_{ij}^{2k}\right\|_{\infty}+\left\|y_{ij}^{2k}-x_{i}^{2k}x_{j}^{2k}\right\|_{\infty}$
(81) $\displaystyle\leq
2\left(\frac{2ek}{J}\right)^{J}+\sum_{l=0}^{2k-1}\|y_{ij}^{2k-l}(x_{i}x_{j})^{l}-y_{ij}^{2k-l-1}(x_{i}x_{j})^{l+1}\|_{\infty}$
(82) $\displaystyle\leq
2\left(\frac{2ek}{J}\right)^{J}+\sum_{l=0}^{2k-1}\|y_{ij}^{2k-l-1}\|_{\infty}\|(x_{i}x_{j})^{l}\|_{\infty}\|y_{ij}-x_{i}x_{j}\|_{\infty}$
(83) $\displaystyle\leq 2\left(\frac{2ek}{J}\right)^{J}+2k\cdot 2^{2k}\cdot
6\left(\frac{4k}{J}\right)^{J}$ (84)

Now, consider a network $g$ that takes in two inputs, defined via

$\displaystyle
g(x_{i},x_{j})=1+\sum_{k=1}^{K}r^{4k}f^{(2k)}\left(f^{(2)}(x_{i}+x_{j})-f^{(2)}(x_{i})-f^{(2)}(x_{j})\right)$
(85)

Then it follows that:

$\displaystyle\left\|\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}-g(x_{i},x_{j})\right\|_{\infty}$
$\displaystyle\leq\sum_{k=1}^{K}r^{4k}O\left(k2^{2k}\left(\frac{2ek}{J}\right)^{J}\right)+\left\|\sum_{k=K+1}^{\infty}(r^{4}x_{i}^{2}x_{j}^{2})^{k}\right\|_{\infty}$
(86) $\displaystyle\leq
O\left(K^{2}2^{2K}\left(\frac{2eK}{J}\right)^{J}\right)+O\left(\frac{r^{4K}}{1-r}\right)=:\delta_{1}$
(87)

Let us assume we choose $J,K$ such that $\delta_{1}\leq 1$. One can confirm
that
$\max\left(\left\|\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\right\|_{\infty},\|g(x_{i},x_{j})\|_{\infty}\right)\leq\frac{1}{1-r}+\delta_{1}\leq\frac{2}{1-r}$.
So it follows from routine calculation that

$\displaystyle\left\|\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}-\prod_{i<j}g(x_{i},x_{j})\right\|_{\infty}\leq
N\left(\frac{2}{1-r}\right)^{N^{2}-1}\delta_{1}$ (88)

Consider second the antisymmetric factor. Following the row transforms given
in the proof of Lemma 3.4 in [11], the antisymmetric term may be equivalently
written as:

$\displaystyle\mathcal{A}(\phi_{1}^{(r)}\otimes\dots\otimes\phi_{N}^{(r)})=\mathcal{A}(\psi_{1}^{(r)}\otimes\dots\otimes\psi_{N}^{(r)})$
(89)

with

$\displaystyle\psi_{j}^{(r)}(x_{i})=\begin{cases}rx_{i}((rx_{i})^{2})^{N/2-j}(1+(rx_{i})^{4(j-1)})&1\leq
j\leq N/2~{},\\\ ((rx_{i})^{2})^{N-j}(1+(rx_{i})^{4(j-1-N/2)})&N/2+1\leq j\leq
N~{}.\end{cases}$ (90)

Expanding, we see that $\psi_{j}^{(r)}$ is a polynomial in degree $\leq 3N$
with only two non-zero coefficients, each bounded by $1$. So by Lemma 5.2 it
is easy to construct networks $\hat{\psi}_{j}$ such that

$\displaystyle\|\psi_{j}^{(r)}-\hat{\psi}_{j}\|_{\infty}\leq
O\left(\left(\frac{6eN}{J}\right)^{J}\right)=:\delta_{2}$ (91)

.

Choose $J$ to ensure $\delta_{2}\leq 1$, then we also clearly have that
$\max\left(\|\hat{\psi}_{j}^{(r)}\|_{\infty},\|\hat{\psi}_{j}\|_{\infty}\right)\leq
2+\delta_{2}\leq 3$. Now, we calculate:

$\displaystyle\left\|\mathcal{A}(\psi_{1}^{(r)}\otimes\dots\otimes\psi_{N}^{(r)})-\mathcal{A}(\hat{\psi}_{1}\otimes\dots\otimes\hat{\psi}_{N})\right\|_{\infty}$
$\displaystyle=\left\|\frac{1}{N!}\sum_{\sigma}(-1)^{\sigma}\left(\psi_{\sigma(1)}^{(r)}\otimes\dots\otimes\psi_{\sigma(N)}^{(r)}-\hat{\psi}_{\sigma(1)}\otimes\dots\otimes\hat{\psi}_{\sigma(N)}\right)\right\|_{\infty}$
(92)
$\displaystyle\leq\frac{1}{N!}\sum_{\sigma}\left\|\left(\psi_{\sigma(1)}^{(r)}\otimes\dots\otimes\psi_{\sigma(N)}^{(r)}-\hat{\psi}_{\sigma(1)}\otimes\dots\otimes\hat{\psi}_{\sigma(N)}\right)\right\|_{\infty}$
(93) $\displaystyle<N3^{N-1}\delta_{2}$ (94)

Finally, we combine the Jastrow factor and antisymmetric component. Let

$\displaystyle\hat{G}(x)=C\sqrt{N!}\prod_{i<j}g(x_{i},x_{j})\mathcal{A}(\hat{\psi}_{1}\otimes\dots\otimes\hat{\psi}_{N})(x)~{}.$
(95)

Again, we have the simple bounds
$\left\|\mathcal{A}(\psi_{1}^{(r)}\otimes\dots\otimes\psi_{N}^{(r)})\right\|_{\infty}\leq
2^{N}$ and
$\left\|\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\right\|_{\infty}\leq\left(\frac{2}{1-r}\right)^{N^{2}}$.

Then we calculate:

$\displaystyle\|G-\hat{G}\|_{\infty}$
$\displaystyle=C\sqrt{N!}\left\|\prod_{i<j}\frac{1}{1-r^{4}x_{i}^{2}x_{j}^{2}}\cdot\mathcal{A}(\phi_{1}^{(r)}\otimes\dots\otimes\phi_{N}^{(r)})-\prod_{i<j}g(x_{i},x_{j})\cdot\mathcal{A}(\hat{\psi}_{1}\otimes\dots\otimes\hat{\psi}_{N})\right\|_{\infty}$
(96) $\displaystyle\leq
C\sqrt{N!}\left(N3^{N-1}\delta_{2}\left(\frac{2}{1-r}\right)^{N^{2}}+N\left(\frac{2}{1-r}\right)^{N^{2}-1}\delta_{1}2^{N}\right)$
(97)
$\displaystyle\leq\sqrt{N!}N3^{N}\left(\frac{2}{1-r}\right)^{N^{2}}(\delta_{1}+\delta_{2})$
(98)

From the choice $r=1-\frac{1}{8N^{4}+8}$ and the assumption that $N\geq 6$, we
can further bound

$\displaystyle\|G-\hat{G}\|_{\infty}$ $\displaystyle\leq
N^{2N}(9N^{4})^{N^{2}}(\delta_{1}+\delta_{2})$ (99) $\displaystyle\leq
N^{5N^{2}}(\delta_{1}+\delta_{2})$ (100)

Choosing $J\geq 12eK$, and $K\geq 2$ so that $K^{2}\leq 2^{K}$, we recall that

$\displaystyle\delta_{1}+\delta_{2}$ $\displaystyle\leq
O\left(K^{2}2^{2K}\left(\frac{6eK}{J}\right)^{J}\right)+O\left(\frac{r^{4K}}{1-r}\right)$
(101) $\displaystyle\leq
O\left(2^{3K-J}\right)+O\left(N^{4}\left(1-\frac{1}{N^{4}}\right)^{4K}\right)$
(102) $\displaystyle\leq
O\left(2^{-9K}\right)+O\left(N^{4}e^{-4K/N^{4}}\right)$ (103)
$\displaystyle\leq O\left(N^{4}e^{-4K/N^{4}}\right)$ (104)

because the right-most term dominates in the second-to-last line.

Hence, if we’d like to control the error $\|G-\hat{G}\|_{\infty}$ by some
$\epsilon$, we require that for some universal constant $C$,

$\displaystyle\epsilon\geq CN^{5N^{2}+4}e^{-4K/N^{4}}$ (105)

Note that for $N\geq 6$, we have $N^{5N^{2}+4}\leq e^{N^{3}}$, and therefore
it suffices to choose $K$ such that

$\displaystyle\epsilon\geq Ce^{N^{3}-4K/N^{4}}$ (106)

And this condition is equivalent to the bound

$\displaystyle K\geq\frac{1}{4}\left(N^{4}\log\frac{C}{\epsilon}+N^{7}\right)$
(107)

Note that $J$ is subject to the same bound up to constant factors.

##  6 Experiments

We illustrate the nature of this exponential separation in finite case,
specifically by seeking to learn our hard function in the Slater ansatz and
Jastrow ansatz with $N=4,6$ particles. In particular, we try to learn the hard
function $G$ with the two given ansätze (rescaled with the $C$ term as this
constant is extremely small to give normalization in the $L_{2}$ norm).

(a) All training iterations with $N=4$

(b) Last quarter of iterations with $N=4$

(c) All training iterations with $N=6$

(d) Last quarter of iterations with $N=6$

Figure 3: Training MSE for Slater ansatz of varying number of determinants and
Jastrow ansatz of one determinant.

We consider the empirical approximation of mean squared training error in
$\|\cdot\|$, where we compare learning with the Slater ansatz and a large
number of determinants, vs. the Jastrow ansatz with a single determinant. We
parameterize each orbital with a three layer neural network with hidden width
30, and activation the complex ReLU that maps $a+bi\mapsto ReLU(a)+ReLU(b)i$.
We further parameterize the Jastrow term with the symmetric architecture of a
Relational Network [24] and multiplication pooling where all networks are
three layers and all hidden widths are also 30. This is justified by the
structure of the Jastrow term in the definition of $G$, which is a symmetric
product of terms depending on pairs of particles.

Learning rate is set to 0.0005 in all runs, for 200000 iterations of full
batch gradient descent on 10000 samples drawn i.i.d. from the complex unit
circle. We plot the normalized MSE, i.e. MSE divided by the error attained by
the naive zero function.

The results are given in Figure 3. Each Slater ansatz is labeled by the number
of determinants it is parameterized with. We observe that one Jastrow
determinant suffices where a large number of Slater determinants fails to
achieve low MSE, alluding to the exponential nature of the separation as the
number of particles $N$ increases.

##  7 Discussion

###  7.1 Proof limitations

The proof technique relies on finding a symmetric function that is supported
exclusively on doubly even Schur polynomials. This is established in the
Pfaffian identity given in Theorem 3.4. This yields a function that requires
exponentially many Slater determinants but may be written exactly in Jastrow
form.

However, the large magnitude of the Jastrow factor precludes efficient
approximation in the infinity norm. This cannot be overcome by changing the
value of $r$: as $r$ approaches 1, the magnitude of support on high
dimensional doubly even Schur polynomials increases while simultaneously the
magnitude of the Jastrow factor increases. So we must choose a sufficiently
large $r$ in order to guarantee the induced matrices are effectively high-
rank.

An alternative tactic would be to control approximation in the $L_{2}$ norm
given by $\|\cdot\|$. However, calculating the $L_{2}$ norm is most easily
done after decomposing into the orthogonal basis of multinomials, which is
challenging when multiplying terms in the Jastrow product.

This proof also only uses 1-dimensional particles to evince a separation. A
nearly identical proof could be employed for higher-dimensional particles by
only utilizing the first component, but there would be no dependence on the
dimension $d$. Understanding the simultaneous dependence on $N$ and $d$ would
therefore require a new proof technique.

###  7.2 Open Questions

The main result of this work represents a first step in understanding
separations between relevant antisymmetric ansätze. We conclude with a
discussion of the natural open questions in this domain.

#### Practical Wavefunctions

Extending the analysis to consider wavefunctions that appear in more practical
applications would be informative. For example, in the one-dimensional case,
famously the eigenfunctions of the Sutherland model are known [15], and the
representability of these functions in particular is an open question.

#### More Powerful Ansätze

The separation we demonstrate in this work is between the two simplest
ansätze. Demonstrating separations among the more expressive network
architectures, for example the backflow or hidden fermion models discussed
previously, would prove the merit of these more complicated and time-intensive
methods.

#### Learnability separations

Our current separation concerns exclusively the approximation properties of
the two parametric families of antisymmetric functions, and as such neglects
any optimization question. It would be interesting to integrate the
optimization aspect in the separation, similarly as in [23] for fully-
connected networks.

#### Acknowledgements:

We are thankful of discussions with Giuseppe Carleo and Michael Lindsey. This
work is partially funded by NSF CAREER CIF-1845360, NSF CCF-1814524 and NSF
DMS-2134216.

## References

  * Carleo & Troyer [2017] Giuseppe Carleo and Matthias Troyer.  Solving the quantum many-body problem with artificial neural networks.  _Science_ , 355(6325):602–606, 2017.
  * Cohen et al. [2018] Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling.  Spherical cnns.  _arXiv preprint arXiv:1801.10130_ , 2018.
  * Daniely [2017] Amit Daniely.  Depth separation for neural networks.  In _Conference on Learning Theory_ , pp. 690–696. PMLR, 2017.
  * Eldan & Shamir [2016] Ronen Eldan and Ohad Shamir.  The power of depth for feedforward neural networks.  In _Conference on learning theory_ , pp. 907–940. PMLR, 2016.
  * Feynman & Cohen [1956] RP Feynman and Michael Cohen.  Energy spectrum of the excitations in liquid helium.  _Physical Review_ , 102(5):1189, 1956.
  * Han et al. [2019a] Jiequn Han, Yingzhou Li, Lin Lin, Jianfeng Lu, Jiefu Zhang, and Linfeng Zhang.  Universal approximation of symmetric and anti-symmetric functions.  _arXiv preprint arXiv:1912.01765_ , 2019a.
  * Han et al. [2019b] Jiequn Han, Linfeng Zhang, and E Weinan.  Solving many-electron schrödinger equation using deep neural networks.  _Journal of Computational Physics_ , 399:108929, 2019b.
  * Hermann et al. [2020] Jan Hermann, Zeno Schätzle, and Frank Noé.  Deep-neural-network solution of the electronic schrödinger equation.  _Nature Chemistry_ , 12(10):891–897, 2020.
  * Huang et al. [2021] Hang Huang, JM Landsberg, and Jianfeng Lu.  Geometry of backflow transformation ansatz for quantum many-body fermionic wavefunctions.  _arXiv preprint arXiv:2111.10314_ , 2021.
  * Hutter [2020] Marcus Hutter.  On representing (anti) symmetric functions.  _arXiv preprint arXiv:2007.15298_ , 2020.
  * Ishikawa et al. [2006] Masao Ishikawa, Soichi Okada, Hiroyuki Tagawa, and Jiang Zeng.  Generalizations of cauchy’s determinant and schur’s pfaffian.  _Advances in Applied Mathematics_ , 36(3):251–287, 2006.
  * Jastrow [1955] Robert Jastrow.  Many-body problem with strong forces.  _Physical Review_ , 98(5):1479, 1955.
  * Jumper et al. [2021] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al.  Highly accurate protein structure prediction with alphafold.  _Nature_ , 596(7873):583–589, 2021.
  * Krizhevsky et al. [2017] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  _Communications of the ACM_ , 60(6):84–90, 2017\.
  * Langmann [2005] Edwin Langmann.  A method to derive explicit formulas for an elliptic generalization of the jack polynomials.  _arXiv preprint math-ph/0511015_ , 2005.
  * LeCun et al. [1995] Yann LeCun, Yoshua Bengio, et al.  Convolutional networks for images, speech, and time series.  _The handbook of brain theory and neural networks_ , 3361(10):1995, 1995.
  * Lee et al. [2019] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh.  Set transformer: A framework for attention-based permutation-invariant neural networks.  In _International Conference on Machine Learning_ , pp. 3744–3753. PMLR, 2019.
  * Luo & Clark [2019] Di Luo and Bryan K Clark.  Backflow transformations via neural networks for quantum many-body wave functions.  _Physical review letters_ , 122(22):226401, 2019\.
  * Maiorov & Meir [1998] Vitaly Maiorov and Ron S Meir.  Approximation bounds for smooth functions in c (r/sup d/) by neural and mixture networks.  _IEEE Transactions on Neural Networks_ , 9(5):969–978, 1998.
  * Maróti [2003] Attila Maróti.  On elementary lower bounds for the partition function.  _Integers: Electronic J. Comb. Number Theory_ , 3:A10, 2003\.
  * Moreno et al. [2021] Javier Robledo Moreno, Giuseppe Carleo, Antoine Georges, and James Stokes.  Fermionic wave functions from neural-network constrained hidden states.  _arXiv preprint arXiv:2111.10420_ , 2021.
  * Pfau et al. [2020] David Pfau, James S Spencer, Alexander GDG Matthews, and W Matthew C Foulkes.  Ab initio solution of the many-electron schrödinger equation with deep neural networks.  _Physical Review Research_ , 2(3):033429, 2020\.
  * Safran & Lee [2022] Itay Safran and Jason Lee.  Optimization-based separations for neural networks.  In _Conference on Learning Theory_ , pp. 3–64. PMLR, 2022.
  * Santoro et al. [2017] Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap.  A simple neural network module for relational reasoning.  _Advances in neural information processing systems_ , 30, 2017.
  * Sundquist [1996] Thomas Sundquist.  Two variable pfaffian identities and symmetric functions.  _Journal of Algebraic Combinatorics_ , 5(2):135–148, 1996.
  * Szabo & Ostlund [2012] Attila Szabo and Neil S Ostlund.  _Modern quantum chemistry: introduction to advanced electronic structure theory_.  Courier Corporation, 2012.
  * Wagstaff et al. [2019] Edward Wagstaff, Fabian Fuchs, Martin Engelcke, Ingmar Posner, and Michael A Osborne.  On the limitations of representing functions on sets.  In _International Conference on Machine Learning_ , pp. 6487–6494. PMLR, 2019.
  * Wagstaff et al. [2022] Edward Wagstaff, Fabian B Fuchs, Martin Engelcke, Michael A Osborne, and Ingmar Posner.  Universal approximation of functions on sets.  _Journal of Machine Learning Research_ , 23(151):1–56, 2022.
  * Zaheer et al. [2017] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola.  Deep sets.  _Advances in neural information processing systems_ , 30, 2017.
  * Zweig & Bruna [2022] Aaron Zweig and Joan Bruna.  Exponential separations in symmetric neural networks.  _arXiv preprint arXiv:2206.01266_ , 2022.