# Neural Network Field Theories:

Non-Gaussianity, Actions, and Locality

Mehmet Demirtas1,2<EMAIL_ADDRESS>, James Halverson1,2
<EMAIL_ADDRESS>, Anindita Maiti1,2,4
<EMAIL_ADDRESS>,
Matthew D. Schwartz1,3<EMAIL_ADDRESS>, and Keegan Stoner1,2
<EMAIL_ADDRESS>
###### Abstract

Both the path integral measure in field theory and ensembles of neural
networks describe distributions over functions. When the central limit theorem
can be applied in the infinite-width (infinite-$N$) limit, the ensemble of
networks corresponds to a free field theory. Although an expansion in $1/N$
corresponds to interactions in the field theory, others, such as in a small
breaking of the statistical independence of network parameters, can also lead
to interacting theories. These other expansions can be advantageous over the
$1/N$-expansion, for example by improved behavior with respect to the
universal approximation theorem. Given the connected correlators of a field
theory, one can systematically reconstruct the action order-by-order in the
expansion parameter, using a new Feynman diagram prescription whose vertices
are the connected correlators. This method is motivated by the Edgeworth
expansion and allows one to derive actions for neural network field theories.
Conversely, the correspondence allows one to engineer architectures realizing
a given field theory by representing action deformations as deformations of
neural network parameter densities. As an example, $\phi^{4}$ theory is
realized as an infinite-$N$ neural network field theory.

Keywords: Neural Network field theory correspondence; Feynman rules for Neural
Network field theories; non-perturbative field theories via Neural Networks.

Contents

toc

##  1 Introduction

The last decade has seen remarkable progress in machine learning (ML) in a
wide variety of fields, including traditional ML fields such as natural
language processing, image recognition, and gameplay (see [1, 2] for reviews,
and [3, 4] for some breakthroughs in the literature), but also in the physical
sciences [5], and more recently to obtain rigorous results in pure mathematics
[6, 7, 8, 9]. This progress has been facilitated in part by the increasing
complexity of deep neural networks, both in terms of the number of parameters
appearing in them and their architecture. However, despite their empirical
success, the theoretical foundations of deep neural networks are still not
fully understood. Natural questions emerge:

  * •

Are ideas from the sciences, such as physics, useful in neural network theory?

  * •

As it develops, does ML theory lead to progress in the sciences?

A growing literature (see below), gives an affirmative answer to the first,
but the second is less clear; it is applied ML, not theoretical ML, that is
primarily used in the sciences.

In this paper we explore both of these questions by further developing a
correspondence between neural networks and field theory. This connection was
already implicit in Neal’s Ph.D. thesis [10] in the $1990$’s, where he
demonstrated that an infinite width single-layer neural network is (under
appropriate assumptions) a draw from a Gaussian process (GP). This is the so-
called neural network / Gaussian process (NNGP) correspondence, and in recent
years it has been shown that most modern NN architectures [11, 12, 13, 14, 15,
16] have a parameter $N$ such that the NN is drawn from a GP in the
$N\to\infty$ limit. The NNGP correspondence is of interest from a physics
perspective because Gaussian processes are generalized non-interacting (free)
field theories, and neural networks provide a novel way to realize them. Non-
Gaussianities emerge at finite-$N$, which correspond to turning on
interactions that are generally non-local, and may be captured by statistical
cumulant functions, known as connected correlators in physics. As we will see,
since Gaussianity in the $N\to\infty$ limit emerges by the Central Limit
Theorem (CLT), non-Gaussianities may be studied more generally by
parametrically violating necessary conditions of the CLT.

These results provide a first glimpse that there is a more general NN-FT
correspondence that should be developed in its own right, taking inspiration
from both physics and ML. In this introduction we will review the central
ideas of the correspondence and introduce principles for understanding the
literature, which we review in part. Readers familiar with the background are
directed to Section 1.3 for a summary of our results.

###  1.1 NN-FT Correspondence

At first glance, neural networks and field theories seem very different from
one another. However, in both cases, the central objects of study are random
functions. The random function $\phi$ associated to a neural network is
defined by its _architecture_ , which is a composition of simpler functions
that involves parameters $\theta$. At program initialization, parameters are
drawn as $\theta\sim P(\theta)$, yielding a randomly initialized neural
network, i.e. a random function. In field theory, the random functions are
simply the fields themselves, typically described by specifying their
probability density function directly, $P(\phi)=\exp(-S[\phi])$, via the
Euclidean action functional $S[\phi]$; we work in Euclidean signature
throughout.

NNFT Fig. 1: In a NN-FT correspondence, ideas from one may give insights into
the other. In this paper we are primarily interested in understanding when
neural network field theories exhibit physical principles such as non-
Gaussianity and locality, with an eye towards applications in both ML and
especially physics in the future.

We therefore have two different origins for the statistics of a field theory.
To exemplify the point, consider a field theory defined by an ensemble of
networks or fields $\phi:\mathbb{R}\to\mathbb{R},$

$\phi(x)=a\,\sigma(b\,\sigma(c\,x))\qquad\qquad a\sim P(a),\,\,b\sim
P(b),\,\,c\sim P(c),$ (1.1)

where $\sigma:\mathbb{R}\to\mathbb{R}$ acts element-wise and is generally
taken to be non-linear. Here the statistics of the ensemble arise from how it
is _constructed_ , rather than from the density $\exp(-S[\phi])$ over
functions from which it is drawn. We will refer to such a description as the
_parameter space_ description of a _neural network field theory_. The
construction of $\phi$ defined in (1.1) has two parts, the architecture that
defines its functional form, and the choice of distributions from which the
parameters $a$, $b$, and $c$ are drawn. This particular architecture is a
feedforward network with depth two, width one, and activation function
$\sigma$. In this description of the field theory, one does not necessarily
know the action $S[\phi]$, but the theory may nevertheless be studied because
the architecture and parameter densities define its statistics.

For instance, the correlation functions of a neural network field theory can
be expressed as

$G^{(n)}(x_{1},\dots,x_{n}):=\mathbb{E}[\phi(x_{1})\dots\phi(x_{n})]=\int
d\theta\,P(\theta)\,\phi(x_{1})\dots\phi(x_{n}),$ (1.2)

where we denote the set of parameters of the neural network by $\theta$, and
the network / field $\phi$ depends on parameters through its architecture.
Alternatively, we could provide a _function space_ description of the theory
by specifying the action $S[\phi]$ and express the correlation functions as

$G^{(n)}(x_{1},\dots,x_{n})=\int
D\phi\,e^{-S[\phi]}\,\phi(x_{1})\dots\phi(x_{n}),$ (1.3)

as in a first course on quantum field theory. These expressions may be derived
from the partition function

$Z[J]=\mathbb{E}[e^{\int d^{d}xJ(x)\phi(x)}],$ (1.4)

where the parameter space and function space results arise by specifying how
the expectation value is computed,

$\displaystyle Z[J]$ $\displaystyle=\int d\theta\,P(\theta)\,e^{\int
d^{d}xJ(x)\phi(x)}$ (1.5) $\displaystyle Z[J]$ $\displaystyle=\int
D\phi\,e^{-S[\phi]+\int d^{d}xJ(x)\phi(x)}.$ (1.6)

In this work, many calculations will be carried out in terms of a general
expectation value $\mathbb{E}[\cdot]$ that denotes agnosticism towards the
origin of the statistics; explicit calculations may be carried out by
replacing $\mathbb{E}$ with one description or the other, as in passing from a
general expression (1.4) to those of parameter space (1.5) and function space
(1.6).

Parameter space and function space provide two different descriptions of a
field theory, which could be thought of as different duality frames [17]. When
one defines a field theory by a neural network architecture, the parameter
space description is readily available, but the action is not known, a priori.
However, if the parameter distributions are easy to sample then the fields are
also easy to sample: one just initializes neural networks on the computer. On
the other hand, in field theory we normally proceed by first specifying an
action; in this case, the probability of a given field configuration is known
because $P[\phi]=\exp(-S[\phi])$ is known, but fields are notoriously hard to
sample, as evidenced by the proliferation of Monte Carlo techniques in lattice
field theory.

_Example: NNGP Correspondence in Parameter Space and Function Space_

Let us study an example to make the abstract notions more concrete. Consider a
fully-connected feedforward network $\phi:\mathbb{R}^{d}\to\mathbb{R}$ with
depth one and width $N$,

$\phi(x)=\sum_{i=1}^{N}\sum_{j=1}^{d}a_{i}\,\sigma\,(b_{ij}x_{j}),\qquad\qquad
a\sim\mathcal{N}(0,\sigma^{2}/N),\,\,b\sim\mathcal{N}(0,\sigma^{2}/d),$ (1.7)

where $\sigma$ is an elementwise non-linearity such as tanh or
$\text{ReLU}(z):=\text{max}(0,z)$. Here, the set of parameters $\theta$ is
given by the union of the $a$-parameters and the $b$-parameters. As we will
see in detail in Section 2, if the parameters are drawn independently then the
connected correlation functions

$G^{(2k)}_{c}(x_{1},\dots,x_{2k})\propto\frac{1}{N^{k-1}},$ (1.8)

and the odd-point correlation functions vanish due to $a$ having zero mean. In
the $N\to\infty$ limit, also known as the Gaussian Process (GP) limit, then,
the only non-vanishing connected correlator has two points,

$G^{(2)}_{c}(x_{1},x_{2})$ (1.9)

which demonstrates that the theory is Gaussian; this is the NNGP
correspondence. Concretely, following [18], we may compute the two-point
function as

$G^{(2)}(x,y)=\mathbb{E}[\phi(x)\phi(y)]=\int\,da\,db\,P(a)P(b)\,\,a_{i_{1}}\sigma(b_{i_{1}j_{1}}x_{j_{1}})\,\,a_{i_{2}}\sigma(b_{i_{2}j_{2}}y_{j_{2}})$
(1.10)

where we have used Einstein summation and left the details of the Gaussian
parameter densities $P(a)$ and $P(b)$ implicit. For a fixed choice of $\sigma$
one may evaluate this integral analytically or via Monte Carlo sampling,
resulting in the two-point function; analytic integrated results for
$\sigma=\text{tanh}$ and $\sigma=\text{Erf}$ are presented in [18]. Since the
parameter space calculation establishes Gaussianity of the theory, we infer
the action

$S[\phi]=\int d^{d}x\,d^{d}y\,\phi(x)\,G^{(2)}(x,y)^{-1}\,\phi(y),$ (1.11)

where the inverse of the two-point function satisfies $\int
d^{d}y\,G^{(2)}(x,y)^{-1}G^{(2)}(y,z)=\delta^{(d)}(x-z)$.

As a concrete example, we refer the reader to Section 4.2, which recalls a
neural network realization of free scalar field theory from [19] that uses a
$\cos$ activation. In that case we have

$G^{(2)}(x,y)^{-1}=\delta(x-y)(\nabla^{2}+m^{2})$ (1.12)

which reproduces the usual free scalar action

$S[\phi]=\int d^{d}x\,\phi(x)\left(\nabla^{2}+m^{2}\right)\phi(x),$ (1.13)

in this case realized via a concrete neural network architecture.

Thus, in the GP limit, both the parameter space and function space
descriptions of the field theory are readily available. Building on [20] using
the Edgeworth expansion, we will see methods for computing approximate actions
at finite-$N$, and we will also develop techniques to engineer desired
actions.

###  1.2 Organizing Principles and Related Work

We have discussed a foundational principle underlying the NN-FT
correspondence, that parameter space and function space provide two different
descriptions of the statistics of an ensemble of neural networks or fields.
Though we have given an example, and there are many more, we are still in very
general territory and it is not clear where to go. Accordingly, we would like
to provide other organizing principles:

  * •

NN-for-FT vs. FT-for-NN: are we aiming to better understand physics or ML?

  * •

Fixed Initialization vs. Learning: are we aiming to understand a fixed NN-FT
at initialization, or a one-parameter family of NN-FTs defined by some
dynamics, such as ML training dynamics or field theory flows?

Much of the existing literature can be classified within each of these
principles, and they also set context for discussing our results. We will
first review some results for network ensembles at initialization, and then
during and after training. With these ideas in place, we will turn to the idea
of using NN-FT in service of field theory.

For literature that is most similar in perspective to this introduction (prior
to this reference section), see [19] and the works that preceded it [21, 17],
by subsets of the authors.

Initialization. A neural network with parameters $\theta$ and parameter
distribution $P(\theta)$ is initialized on a computer by drawing $\theta\sim
P(\theta)$ and inserting them into the architecture, generating a random
function $\phi(x)$ that is sampled from a distribution $P(\phi)$ that may or
may not be known. In the $N\to\infty$ NNGP limit, $P(\phi)$ is Gaussian. This
was shown for feed forward networks in Neal’s thesis [10], as well as more
recently in [11, 12, 13]; was generalized to a plethora of architectures, e.g.
convolutional layers [14, 15, 16, 22, 23, 24, 25, 26], recurrent layers, graph
convolutions [27, 28, 29, 30, 31, 32], skip connections [33, 34], attention
[35, 36], and batch /layer normalization in [37, 38], pooling [25, 26], and
transformers [39, 40]. The generality of this result arises from the
generality in which central limit theorem behavior manifests itself in neural
networks; see [14, 15, 16] for a systematic treatment in the tensor programs
formalism.

Since Gaussianity follows from the central limit theorem, one generally
expects non-Gaussianities in the form of $1/N$-corrections. Study of these
non-Gaussianities was initiated a few years ago; e.g., [41] computed leading
non-Gaussianities via the connected four-point function, [42] showed for deep
feedforward networks how $P(\phi)$ is perturbed by $1/N$-corrections, [21]
proposed using effective field theory to model non-Gaussian $P(\phi)$ for
neural networks, and [43] developed an effective theory approach and an $L/N$
expansion that controls feature learning in deep feedforward networks; for
concreteness in our examples, we are interested in the distribution of
networks at initialization and take $L=1$. This $L/N$ expansion allowed [43]
to also study signal propagation through the network, identify universality
classes, and tune hyperparameters to criticality.

Methods borrowed from field theory have been useful in studying NNs at
initialization. For example, perturbative methods like Feynman diagrams were
employed in [44, 21, 45, 46]. Various schemes for renormalization group flow,
including non-perturbative ones, were applied to NNs in [47, 48]. Global
symmetries of NN-FTs were shown to arise from symmetry invariances of NN
parameter distributions in [17]. While the results of this paper were being
finalized, a recent paper [49] brought forward a different diagrammatic
approach to effective field theories in deep feedforward networks.

Learning. Although we do not study the dynamics of learning in this paper, it
is a goal for future work. Therefore, we would like to review some of the
literature.

Neural networks may be trained to perform useful tasks via a variety of
learning schemes, such as supervised learning or reinforcement learning, that
utilizes a learning algorithm to update the system, such as stochastic
gradient descent. In practice this involves training one or a handful of
randomly initialized neural networks to convergence. However, in general there
is nothing special about the initial networks that were trained; in the
absence of compute limitations, one would prefer to train _all_ the networks
and compute an ensemble average at convergence. Theoretically, this amounts to
tracking the distributional flow of the neural network ensemble, and in
principle it may be done in either parameter space or function space.

In the $N\to\infty$ limit, most known architectures define neural networks
that are draws from Gaussian processes. Since the architecture defines a GP,
it could be used as a prior in Bayesian inference, the learning algorithm of
interest in Neal’s original work [10]. On the other hand, gradient descent
with continuous time is governed by the neural tangent kernel (NTK) [50],
which becomes deterministic and training time $t$-independent in the so-called
frozen-NTK limit. In this limit, $N\to\infty$ and the neural network dynamics
is well-approximated by that of a model that is linear in the neural network
parameters. This frozen behavior is a vast simplification of the dynamics and
is known to exist for many architectures, such as convolutional neural
networks[51], graph neural networks[52], recurrent networks[53, 54], and
attention layers[39]. For supervised learning with MSE loss, the neural
network ensemble trained under gradient descent remains a GP for all times
$t$, including $t\to\infty$, with known mean and covariance; the dynamics
becomes that of kernel regression, with kernel given by the frozen-NTK. How is
this related to Neal’s desire to relate Bayesian inference and trained neural
networks? If all but the last layer’s weights are frozen, then the NTK is the
NNGP kernel and the distribution of the neural network ensemble converges to
the GP Bayesian posterior as $t\to\infty$.

In summary, in the $N\to\infty$ limit, the distribution of the neural network
ensemble is Gaussian. If it undergoes supervised training with MSE loss, it
remains Gaussian at all times and converges to the Bayesian GP posterior in a
particular case [55]. In general, however, gradient descent induces non-
Gaussianities.

At finite-$N$, the neural network ensemble is non-Gaussian. In the Bayesian
context, this defines a non-Gaussian prior, and inference may be performed for
weakly non-Gaussian priors via a $1/N$-expansion [41]. In the gradient descent
context, the NTK is no longer frozen and evolves during training,
significantly complicating the dynamics. Work by Roberts, Yaida, and Hanin
develops a theory of an evolving NTK in [43]. They apply it in detail to
fully-connected networks of depth $L$, demonstrate the relevance of $L/N$ as
an expansion parameter, and develop an effective model for the dynamics. Such
$1/N$ corrections to dynamical NTK were previously studied by other authors in
[56, 57, 44]. Pehlevan et. al. have developed a systematic understanding of
the evolution of NTK and parametric interpolations between rich and lazy
training regimes using the framework of dynamical mean field theory, see [58].
Some of these authors have studied the $O(1/N)$ suppressed corrections to
training dynamics of finite width Bayesian NNs in [59]. A separate work, [60],
presents close-to-Gaussian NN processes including stationary Bayesian
posteriors in the joint limit of large width and large data set, using $1/N$
as an expansion parameter. Moreover, the authors of [61] explore a
correspondence between learning dynamics in the continuous time limit and
early Universe cosmology, and [62, 63, 64] analyzes connected correlation
functions propagating through neural networks.

NN-for-FT.

Neural networks, including the ones we have discussed thus far, generally have
$\mathbb{R}^{n}$ as their domain and therefore naturally live in Euclidean
signature. They define statistical field theories that may or may not have
analytic continuations to quantum field theories in Lorentzian signature.
Nevertheless, statistical field theories are interesting in their own right
and NN-FT provides a novel way to study them.

Using an architecture to define a field theory enables a parameter space
description that makes sampling, and therefore numerical simulation on a
lattice, easy. If one can determine an easily sampled NN architecture that
engineers standard Euclidean $\phi^{4}$ theory, for instance, this could lead
to improved results on the lattice by avoiding Monte Carlo entirely 111This
lattice approach should be contrasted with works [65, 66, 67] that train a
normalizing flow to give proposals for the accept / reject step of MCMC. This
is an engineering problem that is work-in-progress; it is not clear that the
$\phi^{4}$ NN-FT realization in this work is easily sampled. Alternatively, by
simply fixing an easily sampled architecture with interesting physical
properties such as symmetries and strong coupling, lattice simulation could be
performed immediately.

For uses in fundamental and formal quantum physics, one might wish to know
when a neural network architecture defines a _quantum_ field theory (QFT).
Since NN architectures are usually defined in Euclidean signature, we may
instead ask when a Euclidean field theory admits an analytic continuation to
Lorentzian signature that defines a QFT. The situation is complicated by the
fact that in general we do not know the action, but instead have access to the
Euclidean correlation functions, expressed in parameter space.

Fortunately, the Osterwalder-Schrader (OS) theorem [68] of axiomatic field
theory gives necessary and sufficient conditions, expressed in terms of the
correlators, for the existence of a QFT after continuation. The axioms include

  * •

Euclidean Invariance. Correlation functions must be Euclidean invariant, which
becomes Lorentz invariance after analytic continuation. See [19] for an
infinite ensemble of NN architectures realizing Euclidean invariance.

  * •

Permutation Symmetry. Correlation functions must be invariant under
permutations of their arguments, a collection of points in Euclidean space.
This is automatic in NN-FTs with scalar outputs.

  * •

Reflection Positivity. Correlation functions must satisfy a positivity
condition known as reflection positivity, which is necessary for unitarity and
the absence of negative-norm states in the analytically continued theory.

  * •

Cluster Decomposition. Correlation functions must satisfy cluster
decomposition, which says that interactions must shut off at infinite
distance. As a condition on connected correlators, cluster decomposition is

$\lim_{b\to\infty}\,\,G^{(n)}_{c}(x_{1},\dots,x_{p},x_{p+1}+b,\cdots,x_{n}+b)\to
0,$ (1.14)

for any value of $1<p<n$. We have assumed permutation symmetry to simplify
notation, putting the shifts into $x_{p+1}$ into $x_{n}$.

These ideas were utilized in [19] to define neural network _quantum_ field
theories: a NN-QFT is a neural network architecture whose correlation
functions satisfy the OS axioms, and therefore defines a QFT upon analytic
continuation. To date, the only known example is a NN architecture that
engineers a standard free scalar field theory in $d$-dimensions, though we
improve the situation in this work by developing techniques to engineer local
Lagrangians, which automatically satisfy the OS axioms. To make further
progress on NN-QFT in a general setting, one needs especially a deeper
understanding of reflection positivity and cluster decomposition in
interacting NN-FTs; we study the latter.

###  1.3 Summary of Results and Paper Organization

Since there are a number of different themes and concepts in this paper, we
would like to highlight some of the major conceptual results:

  * •

Parametric Non-Gaussianity: $1/N$ and Independence Breaking.
Section 2 approaches interactions in NN-FT (non-Gaussianity) by parametrically
breaking necessary conditions for the central limit theorem to hold. Violating
the infinite-$N$ limit is well studied, but we also systematically study
interactions arising from the breaking of statistical independence, and apply
these ideas in examples.

  * •

Computing Actions with Feynman Diagrams.
In Section 3 we develop a general field theory technique for computing the
action diagrammatically. The coupling functions are computed with a new type
of connected Feynman diagram, whose vertices are the connected correlators.
This is a swapping of the normal role of couplings and connected correlators,
which arises from a “duality” that becomes apparent via the Edgeworth
expansion. The technique is also applied to NN-FT, including an analysis of
how actions may be computed in the two regimes of parameteric non-Gaussianity
developed in Section 2, $1/N$ and independence breaking.

  * •

Engineering Actions in NN-FT.
In Section 4 we develop techniques for engineering actions in NN-FT. This is
to be distinguished from the approach of Section 3: instead of fixing an
architecture, computing its correlators, and then computing its action via
Feynman diagrams, in Section 4 we fix a desired action and develop techniques
for designing architectures that realize the action. Adding a desired term to
the action manifests itself in NN-FT by deforming the parameter distribution,
which breaks statistical independence if it is a non-Gaussianity. Using this
technique, local actions may be engineered at infinite-$N$.

  * •

$\phi^{4}$ as a NN-FT.
In Section 4.2 we design an infinite width neural network architecture that
realizes $\phi^{4}$ theory, using the techniques that we developed.

  * •

The Importance of $N\to\infty$ for Interacting Theories. In physics,
interesting theories defined by a fixed action $S$ generally have a wide
variety of finite action field configurations, which have non-zero probability
density. This is potentially at odds with the universal approximation theorem:
if a single finite-action configuration cannot be realized by an architecture
$A$, but only approximated, then any NN-FT associated to $A$ cannot realize
the field theory associated to $S$. If the $1/N$ is an expansion parameter for
both non-Gaussianities and the degree of approximation, as e.g. with single-
layer width-$N$ networks, this simple no-go theorem suggests that exact NN-FT
engineering of well-studied theories in physics occurs most naturally at
infinite-$N$, as we saw in the case of $\phi^{4}$ theory.

These are highlights of the paper. For more detailed summaries of results, we
direct you to the beginning of each section.

##  2 Connected Correlators and the Central Limit Theorem

Interacting field theories with a Lagrangian description are defined by non-
Gaussian field densities $\exp(-S[\phi])$. If the non-Gaussianities are small,
the theory is close to Gaussian and weakly interacting, in which case
correlation functions may be computed in perturbation theory using Feynman
diagrams. The non-Gaussianities are captured by the higher connected
correlation functions, which vanish in the Gaussian limit. They are known as
cumulants in the statistics literature and may be obtained from a generating
functions $W[J]$ as

$G^{(n)}_{c}(x_{1},\dots,x_{n}):=\left(\frac{\delta}{\delta
J(x_{1})}\dots\frac{\delta}{\delta
J(x_{n})}W[J]\right)\Bigg{|}_{J=0},\qquad\qquad W[J]:=\ln Z[J].$ (2.1)

In the absence of a known Lagrangian description, connected correlators still
encode the presence of non-Gaussianities, since the theory is Gaussian if
$G^{(n)}_{c}=0$ for $n>2$.

In this section we systematically study non-Gaussianities in NN-FT. Since the
parameter space description exists for any NN-FT, we choose to study non-
Gaussianities via connected correlators (rather than actions), which may be
studied in parameter space even when the action is unknown. We are interested
in non-Gaussianities in NN-FT for a number of reasons. In the NN-for-FT
direction, it is important for understanding interactions in the associated
field theories. Conversely, in the FT-for-NN direction, understanding non-
Gaussianities is important for capturing the statistics of finite networks and
networks with correlations in the parameter distributions, which generally
develop during training.

The essential idea in our approach is to recall the origin of Gaussianity, and
then parametrically move away from it. Specifically, many field theories
defined by neural network architectures admit an $N\to\infty$ limit in which
they are Gaussian, and the Gaussianity has a statistical origin: the Central
Limit Theorem (CLT). The CLT states that the distribution of the standardized
sum of $N$ independent and identically distributed random variables approaches
a Gaussian distribution in the limit $N\to\infty$. Therefore we may
systematically study non-Gaussianities in neural network field theories by
violating assumptions of the CLT, e.g. via $1/N$ corrections and breaking the
independence condition, both of which affect connected correlators.

There are a number of results and themes in this section, which is organized
as follows:

  * •

Central Limit Theorem. In Section 2.1 we review the CLT from the perspective
of cumulant generating functionals, which will be useful in NN-FT since in
general we do not have a simple expression for the action but do have access
to cumulants.

  * •

Independence Breaking. In Section 2.2 we introduce how non-Gaussianities may
also arise by violating the statistical independence assumption of the CLT. We
characterize this by a family of joint densities with parameter $\alpha$ that
factorize (become independent) when $\alpha=0$. We study the
$\alpha$-dependence of cumulants via Taylor series, showing that $\alpha$
controls non-Gaussianities independently of those arising from
$1/N$-corrections. A simple example of independence-breaking induced non-
Gaussianities at $N=\infty$ is given in Section 2.2.

  * •

Connected Correlators and Interactions in NN-FT. In Section 2.3 we study non-
Gaussianities in NN-FT, decomposing the field $\phi(x)$ into $N$ constituent
neurons as in [19]. We study the case of independent neurons in Section 2.3.1,
where we present the $N$-scaling of connected correlators and also two
examples: single-layer Cos-net, which exhibits full Euclidean symmetry in all
of its correlators, and $d=1$ ReLU-net, which we show exhibits an interesting
bilocal structure in its two-point and four-point functions.

In Section 2.3.2 we turn to breaking neuron independence in NN-FT, building on
the independence breaking results of [19], which gives a new source of
interactions and a generalized formula for connected correlators.
Specifically, we introduce a general formalism for the expansion of the
cumulant generating functional in terms of independence-breaking parameters,
and therefore the computation of connected correlators. As an example, we
deform the Cos-net theory to have non-independent neurons via non-independent
input weights, doing the deformation in a way that preserves Euclidean
invariance, and compute the independence-breaking correction to the connected
four-point function.

  * •

Identical-ness Breaking. Interactions may also arise from breaking the
identical-ness assumption of the CLT. See Appendix Appendix B. Details of
Examples for an example of a NN-FT with non-Gaussianities arising from
identical-ness breaking.

Equipped with two different types of parameters that induce non-Gaussianity in
connected correlators, $1/N$ and independence-breaking parameters, we will see
how this may be used to approximate actions in Section 3.

###  2.1 Review: Central Limit Theorem from Generating Functions

In order to understand non-Gaussianities in NN-FTs, it is useful to recall
essential aspects of the Central Limit Theorem in the case of a single random
variable, since they carry over to the NN-FT case. We will do so using the
language of generating functions and cumulants (connected correlators), since
we may use them to study Gaussianity and non-Gaussianity even if the NN-FT
action is unknown.

Of course, the CLT is among the most fundamental theorems of statistics. There
are many variants of it in the literature, with different sets of assumptions.
Here, we will describe a particularly simple version of it and provide a
proof, showing how key assumptions come into play. For a more in depth
discussion of the CLT, see e.g. [69].

Consider $N$ random variables $X_{i}$. Assume that they are _identical_ ,
_independent_ , _mean-free_ , and have _finite variance_. The CLT states that
the standardized sum

$\phi=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}X_{i}$ (2.2)

is drawn from a Gaussian distribution in the limit $N\to\infty$. In other
words, even if the $X_{i}$ are sampled from complicated, non-Gaussian
distributions, these details wash out and their sum is drawn from a Gaussian
distribution.

To see the Gaussianity in a way that may be extrapolated to NN-FT, it is
useful to introduce generating functions. The moment generating function of
$\phi$ is defined as

$Z_{\phi}[J]:=\mathbb{E}[e^{J\phi}]=\mathbb{E}[e^{J\sum_{i}X_{i}/\sqrt{N}}],$
(2.3)

from which we can extract the moments by taking derivatives,

$\mu^{\phi}_{r}:=\mathbb{E}[\phi^{r}]=\Bigl{(}\frac{\partial}{\partial
J}\Bigr{)}^{r}Z_{\phi}[J].$ (2.4)

In physics language, $J$ is the source, $Z_{\phi}[J]$ is the partition
function, and $\mu^{\phi}_{r}$ is the $r^{\text{th}}$ correlator of $\phi$.
The cumulant generating functional (CGF) of $\phi$ is the logarithm of the
moment generating functional

$W_{\phi}[J]:=\log\,\mathbb{E}[e^{J\phi}]=\log\,\mathbb{E}[e^{J\sum_{i}X_{i}/\sqrt{N}}],$
(2.5)

and the cumulants $\kappa^{\phi}_{r}$ are computed by taking derivatives of
$W_{\phi}[J]$,

$\kappa^{\phi}_{r}:=\Bigl{(}\frac{\partial}{\partial
J}\Bigr{)}^{r}W_{\phi}[J].$ (2.6)

A random variable is Gaussian only if its cumulants $\kappa^{\phi}_{r>2}$
vanish. Fundamental properties of CGFs include

$\displaystyle W_{X+c}[J]=cJ+W_{X}[J],$ (2.7) $\displaystyle
W_{cX}[J]=\log\,\mathbb{E}[e^{J\,cx}]=W_{X}[cJ]$ (2.8)

where $c\in\mathbb{R}$ is a constant, which imply

$\displaystyle\kappa^{X+c}_{1}=\kappa^{X}_{1}+c\qquad\kappa^{X+c}_{r>1}=\kappa^{X}_{r>1}$
(2.9) $\displaystyle\kappa^{cX}_{r}=c^{r}\,\kappa^{X}_{r},$ (2.10)

respectively.

We would like to see the Gaussianity of $\phi$ under CLT assumptions by
computing cumulants. This is possible since $\kappa_{r>2}=0$ is necessary for
Gaussianity; conversely, we may study non-Gaussianities in terms of non-
vanishing higher cumulants. Specifically, for a sum of _independent_ random
variables the moment generating function factorizes,

$Z_{X_{1}+\dots+X_{N}}[J]=\prod_{i}^{N}Z_{X_{i}}[J].$ (2.11)

Consequently, the CGF and the cumulants become

$\displaystyle W_{X_{1}+\dots+X_{N}}[J]=W_{X_{1}}[J]+\dots+W_{X_{N}}[J],$
(2.12)
$\displaystyle\kappa_{r}^{X_{1}+\dots+X_{N}}=\kappa_{r}^{X_{1}}+\dots+\kappa_{r}^{X_{N}}.$
(2.13)

Using the identities in $\eqref{eqn:cumiden}$ we can write the cumulants of
$\phi$ as

$\kappa_{r}^{\phi}=\frac{\kappa_{r}^{X_{1}}+\dots+\kappa_{r}^{X_{N}}}{N^{r/2}}.$
(2.14)

When the $X_{i}$ are _identical_ this simplifies to

$\kappa_{r}^{\phi}=\frac{\kappa^{X_{i}}_{r}}{N^{r/2-1}}.$ (2.15)

The cumulants $\kappa^{\phi}_{r>2}$ vanish in the $N\to\infty$ limit. To
establish that $\phi$ is Gaussian, we also need to show that
$\kappa^{\phi}_{1}$ and $\kappa^{\phi}_{2}$ are finite. As the $X_{i}$ are
mean-free, $\kappa_{1}^{\phi}=\kappa_{1}^{X_{i}}/\sqrt{N}=0$, while
$\kappa_{2}^{\phi}=\kappa_{2}^{X_{i}}$ is finite by assumption. Thus, $\phi$
is Gaussian distributed. This is the Central Limit Theorem, cast into the
language of cumulants.

We emphasize that this result relies not only on the $N\to\infty$ limit, but
also on the independence assumption (2.13).

###  2.2 Non-Gaussianity from Independence Breaking

We wish to study the emergence of non-Gaussianity by breaking the independence
condition.

To do so, we must parameterize the breaking of statistical independence. Let
$p(X;\alpha)$ be a family of joint distributions on $X_{i}$ parameterized by a
hyperparameter $\alpha$ that must be chosen in order to define the problem. We
choose the family of joint distributions to be of the form

$p(X;\alpha=0)=\prod_{i}p(X_{i}),$ (2.16)

i.e. $p(X)$ is independent in the $\alpha\to 0$ limit, but $\alpha\neq 0$ in
general controls the breaking of independence. Then we obtain

$W_{\phi}[J]=\log\mathbb{E}[e^{J\sum_{i}X_{i}/\sqrt{N}}]=\log\int\prod_{j}dX_{j}\,p(X;\alpha)\,e^{J\sum_{i}X_{i}/\sqrt{N}}$
(2.17)

which when expanded around $\alpha=0$ yields

$W_{\phi}[J]=\log\left[\prod_{j}\mathbb{E}_{p(X,\alpha=0)}[e^{JX_{j}/\sqrt{N}}]+\sum_{k=1}^{\infty}\frac{\alpha^{k}}{k!}\,\int\prod_{j}dX_{j}\,\,e^{J\sum_{i}X_{i}/\sqrt{N}}\,\,\partial_{\alpha}^{k}p(X;\alpha)|_{\alpha=0}\right],$
(2.18)

where the first term of the log uses independence of $p(X;\alpha=0)$.

To deal with the $\alpha$-dependent terms, we generalize a trick appearing
regularly in machine learning, e.g. in the policy gradient theorem in
reinforcement learning. There, the fact that $p\,\partial_{\alpha}\log
p=\partial_{\alpha}p$ allows us to write

$\partial_{\alpha}\mathbb{E}[\mathcal{O}]=\mathbb{E}[\mathcal{O}\,\partial_{\alpha}\log
p]$ (2.19)

for any $\alpha$-independent operator $\mathcal{O}$. Generalizing, we define

$\mathcal{P}_{k}:=\frac{1}{p}\partial^{k}_{\alpha}p,$ (2.20)

and note that it satisfies the recursion relation

$\mathcal{P}_{k+1}=\mathcal{P}_{1}\mathcal{P}_{k}+\partial_{\alpha}\mathcal{P}_{k},$
(2.21)

which allows for efficient computation. We can then write (2.18) as

$W_{\phi}[J]=\log\left[\prod_{j}\mathbb{E}_{p(X,\alpha=0)}\Big{[}e^{JX_{j}/\sqrt{N}}\Big{]}+\sum_{k=1}^{\infty}\frac{\alpha^{k}}{k!}\,\mathbb{E}_{p(X,\alpha=0)}\Big{[}e^{J\sum_{i}X_{i}/\sqrt{N}}\,\mathcal{P}_{k}|_{\alpha=0}\Big{]}\right].$
(2.22)

In the limit $\alpha\to 0$, the $X_{j}$ become independent, and we have

$\lim_{\alpha\to
0}W_{\phi}[J]=\sum_{j}\log\mathbb{E}_{p(X_{j})}\left[e^{JX_{j}/\sqrt{N}}\right]=\sum_{j}\lim_{\alpha\to
0}W_{X_{j}/\sqrt{N}}[J],$ (2.23)

where $\phi$ is now a sum of $N$ independent variables $X_{j}$, and its CGF is
the sum of CGFs of $X_{j}/\sqrt{N}$, as expected; details of the calculations
are in Appendix (Appendix C. CGF and Edgeworth Expansion for NNFT).

We have now discussed two mechanisms that result in non-Gaussianities: $1/N$
corrections and independence breaking. While one can use either or both of
these mechanisms to generate and control non-Gaussianities, more caution is
required to use independence breaking alone, at infinite $N$. This is because
the non-Gaussianities that are generated by independence breaking might depend
on $N$ as well as $\alpha$. For example, if the leading corrections to higher
cumulants $\kappa^{\phi}_{r}$ scale as $\alpha N^{a_{r}}$ with $a_{r}<0$ for
all $r>2$, $\phi$ will be Gaussian regardless of independence breaking. While
if $a_{r}>0$, $\kappa^{\phi}_{r}$ will diverge, which is undesirable. In the
following, we will present an example where $a_{r}=0$ for all $r$ and the non-
Gaussianities are generated by independence breaking alone.

#### Example: Independence Breaking at Infinite N

Let us provide an example of independence breaking non-Gaussianities that
persist in the $N\to\infty$ limit, showing how one can control higher
cumulants by adjusting the correlations between random variables. Consider the
normalized sum of $N$ random variables,

$\phi=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}X_{i},$ (2.24)

where $X_{i}$ is the product of two random variables $a_{i}$ and $h_{i}$,

$X_{i}=a_{i}h_{i}.$ (2.25)

This architecture can be interpreted as the last layer of a fully connected
neural network, where $h_{i}$ are the outputs of the neurons in the previous
layer, $a_{i}/\sqrt{N}$ are the weights, and $\phi$ is the output. First, let
us consider the simple case where $a_{i}$ and $h_{i}$ are independent,
Gaussian random variables222The word “Gaussian” happens to appear many times
in this example. To clarify: though $a$ and $h$ are both Gaussian by
construction, $ah$ is not, and $\phi$ is Gaussian in the CLT limit.,

$P(\vec{a},\vec{h})=P_{\text{ind}}(\vec{a},\vec{h})=(2\pi\sigma_{a}\sigma_{h})^{-N}\exp\Bigl{(}-\frac{1}{2\sigma_{a}^{2}}\sum_{i=1}^{N}a_{i}^{2}-\frac{1}{2\sigma_{h}^{2}}\sum_{i=1}^{N}h_{i}^{2}\Bigr{)},\\\
$

where $\sigma_{a}$ and $\sigma_{h}$ are positive and finite. Since $a_{i}$ and
$h_{i}$ are independent, so are $X_{i}$. The CLT applies and $\phi$ is
Gaussian.

Next, we will perturb $P(\vec{a},\vec{h})$ to break independence. To that end,
we introduce an auxiliary random variable $H$ and define,

$\displaystyle P(\vec{a},\vec{h},H)$
$\displaystyle=P_{\text{ind}}(\vec{a},\vec{h},H)$
$\displaystyle=(2\pi\sigma_{a}\sigma_{h})^{-N}(\sqrt{2\pi}\sigma_{h})^{-1}\exp\Bigl{(}-\frac{1}{2\sigma_{a}^{2}}\sum_{i=1}^{N}a_{i}^{2}-\frac{1}{2\sigma_{h}^{2}}\sum_{i=1}^{N}h_{i}^{2}-\frac{1}{2\sigma_{h}^{2}}H^{2}\Bigr{)},$
(2.26)

where we set the standard deviation of $H$ to $\sigma_{h}$, for simplicity. We
then define a correction term,

$P_{\text{corr}}(\vec{a},\vec{h},H)=P_{\text{ind}}(\vec{a},\vec{h},H)\cdot\exp\Bigl{(}-\frac{1}{2\sigma_{h}^{2}}\sum_{i=1}^{N}(h_{i}-H)^{2}\Bigr{)}.$
(2.27)

Finally, putting these together we define,

$P(\vec{a},\vec{h},H;\alpha)=(1-\alpha)P_{\text{ind}}(\vec{a},\vec{h})+\alpha
P_{\text{corr}}(\vec{a},\vec{h}).$ (2.28)

When $\alpha=0$, the second term vanishes and both $a_{i}$ and $h_{i}$ are
independent. As we turn on $\alpha>0$, the $a_{i}$ remain independent, but
correlations are induced between the $h_{i}$ through a direct coupling to $H$
in $P_{\text{corr}}(\vec{a},\vec{h})$.

To quantify the non-Gaussianity of $\phi$ as a function of $\alpha$, we
compute the CGF,

$\displaystyle W_{\phi}[J]$
$\displaystyle=\log\mathbb{E}[e^{J\sum_{i}X_{i}/\sqrt{N}}]$
$\displaystyle=\log\int\prod_{i=1}^{N}da_{i}dx_{i}P(\vec{a},\vec{h},H;\alpha)e^{J\sum_{i}a_{i}h_{i}/\sqrt{N}}.$
(2.29)

As $P(\vec{a},\vec{h},H;\alpha)$ is Gaussian, (2.29) can be evaluated
analytically to give

$W_{\phi}[J]=\log\Biggl{[}(1-\alpha)\Biggl{(}\frac{N}{N-J^{2}\sigma_{a}^{2}\sigma_{h}^{2}}\Biggr{)}^{N/2}+\alpha\Biggl{(}\frac{N^{N/2}(N-J^{2}\sigma_{a}^{2}\sigma_{h}^{2})^{\frac{1-N}{2}}}{\sqrt{N-(N+1)J^{2}\sigma_{a}^{2}\sigma_{h}^{2}}}\Biggr{)}\Biggr{]}.$
(2.30)

The odd cumulants vanish, as the $\phi$ ensemble has a $\mathbb{Z}_{2}$
symmetry $\phi\to-\phi$ (due to evenness of P(a)), while the even cumulants
$\kappa_{r}^{\phi}$ can be computed by taking derivatives of $W_{\phi}[J]$.
For example, the second and the fourth cumulants are

$\displaystyle\kappa_{2}^{\phi}$
$\displaystyle=\sigma_{a}^{2}\sigma_{h}^{2}(1+\alpha),$ (2.31)
$\displaystyle\kappa_{4}^{\phi}$
$\displaystyle=\sigma_{a}^{4}\sigma_{h}^{4}\Bigl{(}9\alpha-3\alpha^{2}+\frac{6+12\alpha}{N}\Bigr{)}.$
(2.32)

In the limit $N\to\infty,\alpha\to 0$, the second cumulant is finite while all
higher cumulants vanish, and $\phi$ is Gaussian as expected. At finite
$\alpha>0$, all even cumulants are finite and in general nonzero. The ability
to tune $\alpha$ thus allows one to control the degree of non-Gaussianity of
$\phi$. Note that breaking independence in the large $N$ limit is not a
particularly efficient way to sample from a non-Gaussian distribution of a
single variable.

###  2.3 Connected Correlators in NN-FT

We wish to establish that the ideas exemplified above — that non-Gaussianities
may arise via finite-$N$ corrections or independence breaking — generalize to
continuum NN-FT.

In outline, one may think of this conceptually as passing from a single random
variable $\phi$ ($0$d field theory) to a discrete number of random variables
$\phi_{i}$ (lattice field theory), and finally to a continuous number of
random variables $\phi(x)$ (continuum field theory), where
$x\in\mathbb{R}^{d}$. This is a textbook procedure in the context of the
function-space path integral. Here we wish to instead emphasize the general
procedure and parameter space perspective.

Consider the case that the continuum field $\phi(x)$ is built out of neurons
$h_{i}(x)$ [19] as

$\phi(x)=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}h_{i}(x).$ (2.33)

If the $h_{i}(x)$ are independent, the CLT states that $\phi(x)$ is Gaussian
in the limit $N\to\infty$. This is the essence of the NNGP correspondence.

Motivated by the single variable case, we will study non-Gaussianities arising
from both finite-$N$ corrections and breaking of the independence condition.
The cumulant generating functional of $\phi(x)$ is

$W_{\phi}[J]=\log
Z_{\phi}[J]=\sum_{r=1}^{\infty}\int\prod_{i=1}^{r}d^{d}x_{i}\frac{J(x_{1})\dots
J(x_{r})}{r!}\,G_{c}^{(r)}(x_{1},\dots,x_{r}),$ (2.34)

where we have performed a series expansion in terms of the cumulants, a.k.a.
the connected correlation functions $G_{c}^{(r)}$ of $\phi$. This is a
straightforward generalization of (2.5) to the continuum. When the odd-point
functions vanish the connected four-point function is

$G^{(4)}_{c}(x_{1},\dots,x_{4})=G^{(4)}(x_{1},\dots,x_{4})-(G^{(2)}(x_{1},x_{2})G^{(2)}(x_{3},x_{4})+\text{2
perms}),$ (2.35)

which will capture leading-order non-Gaussianities in many of our examples.

In the following, we will quantify non-Gaussianities in terms of non-vanishing
cumulants, as well as directly in the action via an Edgeworth expansion.

####  2.3.1 Finite-$N$ Corrections with Independent Neurons

We first study non-Gaussianities in the case where the neurons $h_{i}(x)$ are
i.i.d. but $N$ is finite, e.g. single hidden layer networks, shown in [70]. We
can express the CGF (2.34) in terms of the connected correlation functions of
the neurons,

$\displaystyle W_{\phi(x)}[J]$
$\displaystyle=\log\mathbb{E}\Bigl{[}\exp{\Bigl{(}\frac{1}{\sqrt{N}}\sum_{i=1}^{N}\int
d^{d}xJ(x)h_{i}(x)\Bigr{)}}\Bigr{]}$
$\displaystyle=\sum_{r=1}^{\infty}\int\prod_{i=1}^{r}d^{d}x_{i}\,\,\frac{J(x_{1})\cdots
J(x_{r})}{r!}\,\frac{G_{c,h_{i}}^{(r)}(x_{1},\cdots,x_{r})}{N^{r/2-1}}.$
(2.36)

This result relies on the fact that for independent $h_{i}$, the expectation
of the product is the product of the expectations, which turns the first
expression into a sum on neuron CGFs. For identically distributed neurons the
sum gives a factor of $N$, and the normalization $1/\sqrt{N}$ gives the
$r$-dependent $N$-scaling. This result lets us express the connected
correlators of $\phi(x)$ in terms of the connected correlators of $h_{i}(x)$,

$G^{(r)}_{c}(x_{1},\cdots,x_{r})=\frac{G^{(r)}_{c,h_{i}}(x_{1},\cdots,x_{r})}{N^{r/2-1}}.$
(2.37)

This $N$-scaling implies

$\lim_{N\to\infty}G^{(r>2)}_{c}(x_{1},\cdots,x_{r})=0,$ (2.38)

establishing Gaussianity in the limit.

##### Examples: Single Layer Cos-net and ReLU-net

We will now consider two single hidden layer architectures with finite $N$ and
i.i.d. parameters. While the methods we describe in this section can be
employed to study neural networks with arbitrary depth $L>1$, inducing
statistical correlations among neurons [70], single hidden layer architectures
suffice to demonstrate their utility.

##### ReLU-net

First, we will consider an architecture with a single hidden layer and ReLU
activation functions. As ReLU activations are ubiquitous in machine learning
applications, this is a natural example to study. Consider

$\phi(x)=W^{1}_{i}R(W^{0}_{ij}x_{j})~{}~{}\text{where}~{}R(z)=\begin{cases}z,~{}\text{for}~{}z\geq
0\\\ 0,~{}\text{otherwise}\end{cases},$ (2.39)

with $d=d_{\text{out}}=1$,
$W^{0}\sim\mathcal{N}(0,\frac{\sigma^{2}_{W_{0}}}{d})$,
$W^{1}\sim\mathcal{N}(0,\frac{\sigma^{2}_{W_{1}}}{N})$.

We compute the two-point function in the parameter space description 1.2 to
obtain

$\displaystyle
G^{(2)}_{c,\text{ReLU}}(x,y)=\frac{\sigma_{W_{0}}^{2}\sigma_{W_{1}}^{2}}{2}\Big{(}R(x)R(y)+R(-x)R(-y)\Big{)},$
(2.40)

which has a factorized structure in the terms that one might call bi-local:
the function depends independently on $x$ and $y$, regardless of any relation
between them. This result is exact and does not receive $1/N$ corrections.
Non-Gaussianities induced by $1/N$ corrections manifest as a nonzero $4$-pt
connected correlation function,

$\displaystyle G^{(4)}_{c,\text{ReLU}}(x_{1},x_{2},x_{3},x_{4})=$
$\displaystyle~{}\frac{1}{N}\Bigg{(}\frac{15\sigma^{4}_{W_{0}}\sigma^{4}_{W_{1}}}{4d^{2}}\Big{(}\sum\limits_{j=\pm
1}R(jx_{1})R(jx_{2})R(jx_{3})R(jx_{4})\Big{)}$
$\displaystyle-\frac{\sigma^{4}_{W_{0}}\sigma^{4}_{W_{1}}}{4d^{2}}\Big{(}\sum\limits_{\mathcal{P}(abcd)}\sum\limits_{j=\pm
1}R(jx_{a})R(jx_{b})R(-jx_{c})R(-jx_{d})\Big{)}\Bigg{)}.$ (2.41)

As expected, $G^{(4)}_{c,\text{ReLU}}(x_{1},x_{2},x_{3},x_{4})$ scales as
$1/N$.

##### Cos-net

Next, let us study a single hidden layer network with cosine activation
functions. The NN-FT associated to Cos-net (and its generalizations) is
Euclidean invariant [19], which is interesting on physical grounds, e.g. to
satisfy one of the Osterwalder-Schrader axioms to establish an NN-QFT.
Euclidean invariance may be established using the mechanism of [17] for
determining symmetries from parameter space correlators, which absorbs
symmetry transformations into parameter redefinitions, yielding invariant
correlators when the relevant parameter distributions are invariant under the
symmetry.

Cos-net was defined in [19], where its $2$-point function and connected
$4$-point function were also computed. The architecture is

$\phi(x)=W^{1}_{i}\cos(W^{0}_{ij}x_{j}+b^{0}_{i})$ (2.42)

where $W^{1}\sim\mathcal{N}(0,\sigma^{2}_{W_{1}}/N)$,
$W^{0}\sim\mathcal{N}(0,\sigma^{2}_{W_{0}}/d)$, and
$b^{0}\sim\text{Unif}[-\pi,\pi]$. As before, the correlation functions are
computed in parameter space (1.2). The $2$-pt function

$\begin{split}G^{(2)}_{c,\text{Cos}}(x_{1},x_{2})&=\frac{\sigma_{W_{1}}^{2}}{2}e^{-\frac{1}{2d}\sigma_{W_{0}}^{2}(\Delta
x_{12})^{2}}\end{split}$ (2.43)

is manifestly translation invariant, with $\Delta x_{12}=x_{1}-x_{2}$. The
$4$-pt correlation function is

$\begin{split}&G^{(4)}_{c,\text{Cos}}(x_{1},x_{2},x_{3},x_{4})=\frac{\sigma_{W_{1}}^{4}}{8N}\sum\limits_{\mathcal{P}(abcd)}\bigg{(}3e^{-\frac{\sigma_{W_{0}}^{2}(\Delta
x_{ab}+\Delta x_{cd})^{2}}{2d}}-2e^{-\frac{\sigma_{W_{0}}^{2}\left((\Delta
x_{ab})^{2}+(\Delta x_{cd})^{2}\right)}{2d}}\bigg{)},\end{split}$ (2.44)

where $\Delta x_{ij}:=x_{i}-x_{j}$ and $\mathcal{P}(abcd)$ denotes the three
independent ways of drawing pairs $(x_{a},x_{b}),(x_{c},x_{d})$ from the list
of external vertices $(x_{1},x_{2},x_{3},x_{4})$.

We see the manifest Euclidean invariance of these correlators, and that non-
Gaussianities are encoded in $G^{(4)}_{c,\text{Cos}}$ as a $1/N$ corrections.

####  2.3.2 Generalized Connected Correlators from Independence Breaking

We now wish to generalize our theories and connected correlators to including
the possibility that non-Gaussianities arise not only from $1/N$-corrections,
but also from independence breaking, e.g. by developing correlations between
the neurons $h_{i}(x)$. Previously, [19, 70] studied mixed non-Gaussianities
at finite $N$ and statistical correlations among neurons.

Generalizing our approach from section 2.2, we parameterize breaking of
statistical independence by promoting the distribution of neurons $P(h)$ to
depend on a vector of hyperparameters $\vec{\alpha}\in\mathbb{R}^{q}$,
$P(h;\vec{\alpha})$.

Since independence is necessary for Gaussianity via the CLT, and we will
sometimes wish to perturb around the Gaussian fixed point, we require

$P(h;\vec{\alpha}=\vec{0})=\prod_{i}P(h_{i}),$ (2.45)

where the hyperparameter vector $\vec{\alpha}$ must be chosen as part of the
architecture definition. From this expression, the neurons become independent
when $\vec{\alpha}=0$.

For a general $P(h;\vec{\alpha})$, the CGF is

$W_{\phi}[J]=\log\Bigg{[}\int\Big{(}\prod_{i=1}^{N}Dh_{i}\Big{)}P(h;\vec{\alpha})e^{\frac{1}{\sqrt{N}}\sum\limits_{i=1}^{N}\int
dx\,h_{i}(x)J(x)}\Bigg{]}.\\\ $ (2.46)

For small values of $\alpha$, we can expand $P(h;\vec{\alpha})$,

$P(h;\vec{\alpha})=P(h;\vec{\alpha}=0)+\sum_{r=1}^{\infty}\sum_{s_{1},\cdots,s_{r}=1}^{q}\frac{\alpha_{s_{1}}\cdots\alpha_{s_{r}}}{r!}\partial_{\alpha_{s_{1}}}\cdots\partial_{\alpha_{s_{r}}}P(h;\vec{\alpha})\Big{|}_{\vec{\alpha}=0}.$
(2.47)

Analogous to the single variable case, we define

$\mathcal{P}_{r,\\{s_{1},\cdots,s_{r}\\}}:=\frac{1}{P(h;\vec{\alpha})}\partial_{\alpha_{s_{1}}}\cdots\partial_{\alpha_{s_{r}}}P(h;\vec{\alpha})$
(2.48)

satisfying the recursion relation

$\displaystyle\mathcal{P}_{r+1,\\{s_{1},\cdots,s_{r+1}\\}}=\frac{1}{r+1}\sum_{\gamma=1}^{r+1}(\mathcal{P}_{1,s_{\gamma}}+\partial_{\alpha_{s_{\gamma}}})\mathcal{P}_{r,\\{s_{1},\cdots,s_{r+1}\\}\backslash
s_{\gamma}}.$ (2.49)

Finally, we can expand (2.34) in $\vec{\alpha}$,

$W_{\phi}[J]=\log\Bigg{[}e^{W_{\phi,\vec{\alpha}=0}[J]}+\sum_{r=1}^{\infty}\sum_{s_{1},\cdots,s_{r}=1}^{q}\frac{\alpha_{s_{1}}\cdots\alpha_{s_{r}}}{r!}\prod_{i=1}^{N}\mathbb{E}_{P_{i}(h_{i})}\Big{[}e^{\frac{1}{\sqrt{N}}\int
d^{d}x\,h_{i}(x)J(x)}\cdot\mathcal{P}_{r,\\{s_{1},\cdots,s_{r}\\}}\big{|}_{\vec{\alpha}=0}\Big{]}\Bigg{]},$
(2.50)

where $W_{\phi,\vec{\alpha}=0}[J]$ is given in (2.3.1). This form of
$W_{\phi}[J]$ makes it clear how one can tune $N$ and $\vec{\alpha}$ to
generate and manipulate non-Gaussianities; for details see Appendix (Appendix
C. CGF and Edgeworth Expansion for NNFT).

For appropriately small independence breaking hyperparameter $\vec{\alpha}$,
and other attributes of the architecture, the ratio of second term to first
term in the logarithm of (2.50) is small. In such cases, one can approximate
(2.50) using Taylor series expansion $\log(1+x)\approx x$ around $x=0$. The
CGF becomes

$\displaystyle
W_{\phi}[J]=W_{\phi,\vec{\alpha}=0}[J]+\sum_{s=1}^{q}\frac{\alpha_{s}}{e^{W_{\phi,\vec{\alpha}=0}[J]}}\prod_{i=1}^{N}\mathbb{E}_{P_{i}(h_{i})}\Big{[}e^{\frac{1}{\sqrt{N}}\int
d^{d}x\,h_{i}(x)J(x)}\cdot\mathcal{P}_{1,s}\big{|}_{\vec{\alpha}=0}\Big{]},$
(2.51)

and the cumulants

$\displaystyle G^{(r)}_{c}(x_{1},\cdots,x_{r})$
$\displaystyle=\frac{\partial^{r}W_{\phi}[J]}{\partial J(x_{1})\cdots\partial
J(x_{r})}\Big{|}_{J=0},$
$\displaystyle=G^{(r),\text{i.i.d.}}_{c}+\vec{\alpha}\cdot\Delta
G^{(r)}_{c}+O(\vec{\alpha}^{2}).$ (2.52)

are proportional to $\vec{\alpha}$ at the leading order. The leading order
expression in $\vec{\alpha}$ is evaluated in (C.21).

##### Example: Single Layer Cos-net

Let us exemplify the non-Gaussianities generated by statistical independence
breaking of a single layer Cos-net architecture given in (2.42). We can break
this independence by modifying the distribution from which the weights
$W^{0}_{ij}$ (an $N\times d$ matrix) are sampled

$\displaystyle
P(W^{0})=c\exp{\Bigg{[}-\sum_{i,j}\Big{(}\frac{d}{2\sigma_{W_{0}}^{2}}(W^{0}_{ij})^{2}+\frac{\alpha_{\text{IB}}}{N^{2}}\sum_{i_{1},j_{1},i_{2},j_{2}}(W^{0}_{i_{1}j_{1}})^{2}(W^{0}_{i_{2}j_{2}})^{2}\Big{)}\Bigg{]}},$
(2.53)

where $c$ is a normalization constant. The rotational invariance preserving
term $\frac{\alpha_{\text{IB}}(\text{Tr}({W^{0}}^{T}W^{0}))^{2}}{N^{2}}$
introduces mixing between the weights $W^{0}_{ij}$ and parametric independence
is explicitly broken. The degree of independence breaking can be controlled by
tuning $\alpha_{\text{IB}}$.

We wish to compute the connected correlation functions to quantify the non-
Gaussianities generated by independence breaking. In general, this is a
difficult problem. However, when $\alpha_{\text{IB}}\ll 1$, we can perform a
perturbative expansion in $\alpha_{\text{IB}}$. Setting $d=1$ for simplicity,
we obtain

$\displaystyle G^{(2)}_{c,\text{Cos}}(x_{1},x_{2})=$
$\displaystyle\frac{\alpha_{\text{IB}}\sigma_{W_{0}}^{4}\sigma_{W_{1}}^{2}e^{-\frac{\sigma_{W_{0}}^{2}(\Delta
x_{12})^{2}}{2}}}{2N}\bigg{[}-\frac{\left(1-5\sigma_{W_{0}}^{2}(\Delta
x_{12})^{2}+\sigma_{W_{0}}^{4}(\Delta x_{12})^{4}\right)}{N}$
$\displaystyle+\sigma_{W_{0}}^{2}(\Delta x_{12})^{2}\bigg{]},$ (2.54)
$\displaystyle
G^{(4)}_{c,\text{Cos}}(x_{1},\cdots,x_{4})=G^{(4)}_{\begin{subarray}{c}c,\text{Cos}\\\
\text{i.i.d.}\end{subarray}}(x_{1},\cdots,x_{4})+\frac{\alpha_{\text{IB}}\sigma_{W^{0}}^{4}\sigma_{W^{1}}^{4}}{8N^{2}}\sum\limits_{\mathcal{P}(abcd)}\Bigg{[}6-\Big{(}2\sigma_{W^{0}}^{2}(\Delta
x_{ab}^{2}+\Delta x_{cd}^{2})$ $\displaystyle+2\sigma_{W^{0}}^{4}\Delta
x_{ab}^{2}\Delta x_{cd}^{2}\Big{)}e^{-\frac{\sigma_{W_{0}}^{2}}{2}\left(\Delta
x_{ab}^{2}+\Delta x_{cd}^{2}\right)}+\Big{(}3+3\sigma_{W^{0}}^{2}(\Delta
x_{ab}+\Delta x_{cd})^{2}\Big{)}e^{-\frac{\sigma_{W_{0}}^{2}}{2}(\Delta
x_{ab}+\Delta x_{cd})^{2}}\Bigg{]},$ (2.55)

to leading order in $\alpha_{\text{IB}}$, where
$G^{(4)}_{\begin{subarray}{c}c,\text{Cos}\\\
\text{i.i.d.}\end{subarray}}(x_{1},\cdots,x_{4})$ is obtained at $d=1$ from
(2.44). Non-Gaussianities at finite $N$, and $\alpha_{\text{IB}}\neq 0$ still
preserve the translation invariance of the $2$nd and $4$th cumulants of Cos-
net architecture.

We refer the reader to Appendix (Cos-net Cumulants at Finite $N$, Non-i.i.d.
Parameters) for details, where we also compute leading order non-Gaussian
corrections to first two cumulants in a single hidden layer Gauss-net at
$\alpha_{\text{IB}}\neq 0$, finite $N$, for $d=1$.

##  3 Computing Actions from Connected Correlators

In Section 2 we systematically studied non-Gaussianities in neural network
field theories by parametrically violating two assumptions of the Central
Limit Theorem: infinite-$N$ and independence. The study was performed at the
level of connected correlators, rather than actions, because every NN-FT
admits a parameter space description of connected correlators, even if an
action is not known.

In this section we will develop these techniques for calculating actions from
connected correlators, including in terms of Feynman diagrams in which the
connected correlators are vertices. More specifically:

  * •

Field Density from Connected Correlators: Edgeworth Expansion. In Section 3.1
we review how knowledge of the cumulants of a single random variable may be
used to approximate its probability density, and then we generalize to the
field theory case, which has a continuum of random variables. This gives an
expression for $P[\phi]=\exp(-S[\phi])$ in terms of connected correlation
functions. We present an explicit example in the case of a single variable.

  * •

Computing the Action with Feynman Diagrams. Given the Edgeworth expansion, we
develop a method to compute the action perturbatively via Feynman diagrams,
which becomes clear due to a formal similarity between the Edgeworth expansion
and the partition function of a field theory. This is a result that is
applicable to general field theories.

  * •

Neural Network Field Theory Actions. In Section 3.3 we specify the analysis of
Section 3.2 to the case of neural network field theories. We derive the
leading order form of the action for the case of non-Gaussianities induced
either by $1/N$-corrections or independence breaking.

  * •

Neural Network Field Theory Examples. In Section 3.4 we derive the leading-
order action in $1/N$ for concrete neural network architectures.

###  3.1 Field Density from Connected Correlators: Edgeworth Expansion

The Edgeworth expansion from statistics (see e.g. [71] for a textbook
statistics description and [20] for an ML study) can be used to construct the
probability density from the cumulants. The key observation which allows the
Edgeworth expansion to be applied in a field theory is that the normal
relation for the generating function in terms of the action

$e^{W[J]}=\int d\phi\,e^{-S[\phi]+J\phi}$ (3.1)

can be inverted to express the action in terms of the generating functional.
Adding a source term in the exponent, mapping $J\to iJ$ and integrating over
$J$, we have

$\int dJe^{W[iJ]-iJ\phi}=\int dJe^{-iJ\phi}\int
d\phi^{\prime}e^{-S[\phi^{\prime}]+iJ\phi^{\prime}}=e^{-S[\phi]}$ (3.2)

where

$\int dJe^{iJ(\phi^{\prime}-\phi)}=\delta[\phi^{\prime}-\phi]$ (3.3)

has been used. Deforming the $J$ integration contour back to real $J$ then
results in

$P[\phi]=e^{-S[\phi]}=\int dJe^{W[J]-J\phi},$ (3.4)

This gives the probability density and action in terms of $W[J]$. This result
can also be thought of as arising from an inverse Fourier transform of the
characteristic function.

Then to apply the Edgeworth expansion for a single random variable $\phi$, we
write $W[J]$ in terms of cumulants

$W[J]=\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r!}J^{r},$ (3.5)

which lets us write

$\displaystyle P[\phi]$
$\displaystyle=\exp\left[\sum_{r=3}^{\infty}\frac{\kappa_{r}}{r!}(-\partial_{\phi})^{r}\right]\int
dJe^{\kappa_{2}\frac{J^{2}}{2}+\kappa_{1}J-J\phi},$
$\displaystyle=\exp\left[\sum_{r=3}^{\infty}\frac{\kappa_{r}}{r!}(-\partial_{\phi})^{r}\right]e^{-\frac{(\phi-\kappa_{1})^{2}}{2\kappa_{2}}},$
(3.6)

where the Gaussian integral has been performed by mapping $J\to iJ$
(alternatively, working with the characteristic function the whole time) and
we have neglected the normalization factor. We have an expression for the
density $P_{\phi}$ as an expansion around the Gaussian with mean $\kappa_{1}$
and variance $\kappa_{2}$.

The result may be extended to the field theory case, where $\phi$ is replaced
by $\phi(x)$, a continuum of mean free random variables. Then the relation is

$e^{-S[\phi]}=\frac{1}{Z}\exp{\Big{(}\sum_{r=3}^{\infty}\frac{(-1)^{r}}{r!}\int\prod_{i=1}^{r}d^{d}x_{i}G_{{c}}^{(r)}(x_{1},\cdots,x_{r})\frac{\delta}{\delta\phi(x_{1})}\cdots\frac{\delta}{\delta\phi(x_{r})}\Big{)}}e^{-S_{G}[\phi]},$
(3.7)

where the Gaussian Process action $S_{G}$ is defined as

$S_{G}[\phi]=\frac{1}{2}\int
d^{d}x_{1}d^{d}x_{2}\,\phi(x_{1})G_{{c}}^{(2)}(x_{1},x_{2})^{-1}\phi(x_{2}),$
(3.8)

To the extent that there is a perturbative ordering to the correlators through
some expansion parameter (such as $\frac{1}{N}$ or independence breaking),
this expression can be evaluated perturbatively to systematically construct an
action from the cumulants. 333 In the finite-dimensional version of the
Edgeworth expansion, it is sometimes convenient to further express the powers
of derivatives in terms of probabilist’s Hermite polynomials using
$(-\partial_{x})^{r}\,e^{-\frac{x^{2}}{2}}=:H_{r}(x)\,e^{-\frac{x^{2}}{2}},$
(3.9) In the field theory case, using Hermite polynomials provides no obvious
advantage.

#### 1D Example: Sum of $N$ Uniform Random Variables

Let us demonstrate the Edgeworth expansion in a simple example. Consider the
standardized sum of $N$ i.i.d. random variables sampled from a uniform
distribution

$\phi=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}X_{i},\quad
X_{i}\sim\text{Unif}(-1/2,1/2)\,\,\forall i.$ (3.10)

The cumulants of $X_{i}$ are

$\displaystyle\kappa_{1}^{X_{i}}$ $\displaystyle=0,$ (3.11)
$\displaystyle\kappa_{r}^{X_{i}}$
$\displaystyle=\frac{B_{r}}{r}\,\,\text{for}\,\,r\geq 2,$ (3.12)

where $B_{r}$ is the $r^{\text{th}}$ Bernoulli number.444$B_{r}$ vanishes for
odd $r\geq 2$. First few nonzero Bernoulli numbers are:
$B_{2}=\frac{1}{6},B_{4}=-\frac{1}{30},B_{6}=\frac{1}{42}$. Plugging this into
(2.15), the cumulants of $\phi$ are

$\kappa_{r}^{\phi}=\frac{B_{r}}{rN^{r/2-1}}.$ (3.13)

At finite $N$, the cumulants $\kappa_{r>2}^{\phi}$ are nonzero and $\phi$ is
non-Gaussian. Using these cumulants, we can write down the probability
distribution function of $\phi$ via an Edgeworth expansion,

$\displaystyle P_{\phi}$
$\displaystyle=\frac{1}{Z}\exp{\Biggl{[}\sum_{r=3}^{\infty}\frac{\kappa_{r}^{\phi}}{r!}(-\partial_{\phi})^{r}}\Biggr{]}e^{-\phi^{2}/2\kappa_{2}^{\phi}}$
$\displaystyle=\frac{1}{Z}\exp{\Biggl{[}\sum_{r=3}^{\infty}\frac{B_{r}}{r!rN^{r/2-1}}(-\partial_{\phi})^{r}}\Biggr{]}e^{-\phi^{2}/B_{2}}$
(3.14)

Truncating the sum at $r=4$, expanding the exponential, and keeping terms up
to $O(1/N)$ we get

$\displaystyle P_{\phi}$
$\displaystyle=\frac{1}{Z}\Bigg{[}1+\kappa_{4}^{\phi}\Big{(}\frac{1}{8(\kappa_{2}^{\phi})^{2}}-\frac{1}{4(\kappa_{2}^{\phi})^{3}}\phi^{2}+\frac{1}{24(\kappa_{2}^{\phi})^{4}}\phi^{4}\big{)}+O(1/N^{3/2})\Bigg{]}e^{-\phi^{2}/2\kappa_{2}^{\phi}},$
$\displaystyle=\frac{1}{Z^{\prime}}\exp\Bigg{[}-\Big{(}\frac{1}{2\kappa_{2}^{\phi}}+\frac{\kappa_{4}^{\phi}}{4(\kappa_{2}^{\phi})^{3}}\Big{)}\phi^{2}+\frac{\kappa_{4}^{\phi}}{24(\kappa_{2}^{\phi})^{4}}\phi^{4}+O(1/N^{3/2})\Bigg{]},$
$\displaystyle=\frac{1}{Z^{\prime}}\exp\Bigg{[}\Big{(}-6+\frac{18}{5N}\Big{)}\phi^{2}-\frac{36}{5N}\phi^{4}+O(1/N^{3/2})\Bigg{]},$
(3.15)

where on the second line we absorbed the constant term into the normalization
constant $Z^{\prime}$. At order $O(N^{0})$, the exponent in (3.1) is quadratic
and $\phi$ is Gaussian distributed. Gaussianity is then broken by a quartic
interaction at order $O(1/N)$.

It is worth noting that the cumulants of $\phi$ are given by simple closed
form expressions, see Equation (3.13), while $P_{\phi}$ involves a
perturbative expansion in $1/N$. This is in contrast to weakly coupled field
theories, where we often start from a simple action expressed in closed form
and calculate the connected correlation functions via a perturbative expansion
in the coefficients of interaction terms.

###  3.2 Computing the Action with Feynman Diagrams

In a field theory a powerful tool for organizing a perturbation expansion is
with Feynman diagrams. Just as Feynman diagrams can be used to compute the
cumulants perturbatively in an expansion parameter from an action, they can
also be used to compute the action perturbatively from the cumulants. To
understand the derivation, recall the expression for the partition function

$\displaystyle e^{W[J]}$
$\displaystyle=Z[J]=c^{\prime}\exp{\Big{(}\sum_{r=3}^{\infty}\int\prod_{i=1}^{r}d^{d}x_{i}\,g_{r}(x_{1},\cdots,x_{r})\frac{\delta}{\delta
J(x_{1})}\cdots\frac{\delta}{\delta J(x_{r})}\Big{)}}e^{-S_{0}[J]},$ (3.16)

where we have introduced couplings $g_{r}$ instead of $g_{r}/r!$,

$S_{0}[J]=\int dx_{1}dx_{2}J(x_{1})\Delta(x_{1},x_{2})J(x_{2}),$ (3.17)

and $\Delta(x_{1},x_{2})$ is the free propagator. The expression (3.16) arises
by taking the usual expression for the partition function

$Z[J]=\int D\phi\,e^{-S_{\text{free}}[\phi]-S_{\text{int}}+\int
d^{d}xJ(x)\phi(x)}$ (3.18)

and replacing the $\phi$’s in the interaction terms

$S_{\text{int}}=\sum_{r=3}^{\infty}\int\prod_{i=1}^{r}d^{d}x_{i}\,g_{r}(x_{1},\dots,x_{r})\,\phi(x_{1})\dots\phi(x_{r})$
(3.19)

by $\delta/\delta J$’s. Pulling the $J$-derivatives outside of the $\int
D\phi$ in (3.18) and performing the Gaussian integral yields (3.16). These
manipulations closely mirror the Edgeworth expansion.

| Field Picture | Source Picture
---|---|---
Field | $\phi(x)$ | $J(x)$
CGF | $W[J]=\log(Z[J])$ | $S[\phi]=-\log(P[\phi])$
Cumulant | $G^{(r)}_{c}(x_{1},\dots,x_{r})$ | $g_{r}(x_{1},\dots,x_{r})$
Table 1: The Edgeworth expansion for $P[\phi]$ and the interaction expansion
of $Z[J]$ are formally related by a change of variables, given here up to
constant factors. Due to this relationship, non-local couplings and connected
correlators may both be computed by appropriate connected Feynman diagrams.

The Edgeworth expansion (3.7) is related to the partition function (3.16) by a
simple change of variables, given in Table 1, which one might think of as a
duality map between a field picture and a source picture. This relationship
between the Edgeworth expansion and the partition function immediately tells
us that the analog of $g_{r}(x_{1},\dots,x_{n})$ are the connected correlation
functions $G_{{c}}^{(r)}(x_{1},\cdots,x_{r})$ in (3.7).

We may therefore compute the couplings $g_{r}(x_{1},\dots,x_{n})$ in the same
way that we compute the connected correlators
$G_{{c}}^{(r)}(x_{1},\cdots,x_{r}).$ In a weakly coupled field theory, one can
compute the connected correlation functions
$G_{{c}}^{(r)}(x_{1},\cdots,x_{r})$ in terms of the couplings
$g_{r}(x_{1},\cdots,x_{r})$ perturbatively via Feynman diagrams. An Edgeworth
expansion allows us to do the converse and compute the couplings
$g_{r}(x_{1},\cdots,x_{r})$ in terms of the connected correlation functions
$G_{{c}}^{(r)}(x_{1},\cdots,x_{r})$. The similarity between (3.7) and (3.16)
suggests that the terms in the expansion for $g_{r}(x_{1},\cdots,x_{r})$ can
be represented by Feynman diagrams, whose vertices are connected correlators,
e.g.

$G^{(6)}_{c}$ (3.20)

in the case of a six-point vertex. Notably, the vertex is itself a function
and lines enter the $n$-point vertex at $n$ locations.

To compute the coupling $g_{r}(x_{1},\dots,x_{r})$ in terms of Feynman
diagrams, one sums over all connected $r$-point Feynman diagrams made out of
$G^{(n)}_{c}$ vertices. By convention, we do not label internal points on the
vertices, in order to simplify the combinatorics. For instance, the four-point
coupling $g_{4}(x_{1},\dots,x_{4})$ has a diagram

$G^{(4)}_{c}$$x_{1}$$x_{2}$$x_{3}$$x_{4}$ (3.21)

where it is to be understood that connections to internal points in a vertex
appear in all possible combinations. Analytic expressions may be obtained from
the diagrams via the Feynman rules given in Table 2. If
$G_{{c}}^{(2)}(x_{i},y_{j})^{-1}=\frac{\delta^{2}S_{G}}{\delta\phi(x_{i})\delta\phi(y_{j})}$
involves differential operators, it can be evaluated by Fourier
transformation, see Appendix (Appendix D. Fourier Transformation Trick for
$G^{(2)}_{c}(x,y)^{-1}$)

Feynman Rules for $g_{r}(x_{1},\dots,x_{r})$.

  1. 1.

Internal points associated to vertices are unlabeled, for diagrammatic
simplicity. Propagators therefore connect to internal points in all possible
ways.

  2. 2.

For each propagator between $z_{i}$ and $z_{j}$,

$\displaystyle\vbox{\hbox{ \leavevmode\hbox to0pt{\vbox
to0pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[label=left:$z_{i}$] (a1) at
(-0.4,0) ; \vertex[label=right:$z_{j}$] (b1) at ( 0.4,0); \diagram* { (a1) --
[dashed] (b1)}; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}=G_{{c}}^{(2)}(z_{i},z_{j})^{-1}.$
(3.22)

  3. 3.

For each vertex,

$\displaystyle\vbox{\hbox{ \leavevmode\hbox to0pt{\vbox
to0pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(n)}_{c}$}}; \vertex(a) at (-1,0.5) ; \vertex(b) at (
-1,-0.5); \vertex(c) at (1, 0.5); \vertex(b1) at (0,1); \vertex(d) at
(1,-0.5); \vertex(a1) at (0,-1) ; \vertex(an) at (-0.31,0.15) ; \vertex(bn) at
( -0.10,-0.47); \vertex(cn) at (1.0, -0.15); \vertex(b1n) at (0.85,0.45);
\vertex(dn) at (0.11,0.63); \vertex(a1n) at (0.55,-0.65) ; \diagram* { (a) --
[dashed] (m) -- [dashed](b), (c) -- [dashed] (m) -- [dashed] (d), (a1) --
[dashed] (m) -- [dashed] (b1)}; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}=(-1)^{n}\int
d^{d}y_{1}\cdots d^{d}y_{n}\,G_{{c}}^{(n)}(y_{1},\cdots,y_{n}).$ (3.23)

  4. 4.

Divide by symmetry factor and insert overall $(-)$.

Table 2: Feynman rules for computing $g_{r}$ from each connected diagram with
$G^{(n)}_{c}$ vertices.

As an example, let us compute a contribution to the quartic coupling
$g_{4}(x_{1},x_{2},x_{3},x_{4})$ from a $G^{(4)}_{c}$ vertex

$\displaystyle g_{4}(x_{1},\dots,x_{4})=$
$\displaystyle\,\,-\frac{1}{4!}\Big{[}\int
dy_{1}dy_{2}dy_{3}dy_{4}\,G^{(4)}_{c}(y_{1},y_{2},y_{3},y_{4})\,G_{{c}}^{(2)}(y_{1},x_{1})^{-1}G_{{c}}^{(2)}(y_{2},x_{2})^{-1}$
$\displaystyle\quad
G_{{c}}^{(2)}(y_{3},x_{3})^{-1}G_{{c}}^{(2)}(y_{4},x_{4})^{-1}+\text{perms}\Big{]}+\dots$
(3.24) $\displaystyle=$ $\displaystyle\qquad\vbox{\hbox{ \leavevmode\hbox
to0pt{\vbox to0pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(4)}_{c}$}}; \vertex[label=left:$x_{1}$] (a) at (-1,1)
; \vertex[label=left:$x_{2}$] (b) at ( -1,-1); \vertex[label=right:$x_{3}$]
(c) at (1, 1); \vertex[label=right:$x_{4}$] (d) at (1,-1); \vertex(a1) at
(-0.31,0.31) ; \vertex(b1) at ( -0.31,-0.31); \vertex(c1) at (0.31, 0.31);
\vertex(d1) at (0.31,-0.31); \diagram* { (a) -- [dashed] (m) -- [dashed](b),
(c) -- [dashed] (m) -- [dashed] (d) }; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}+\dots,$ (3.25)

where the dots represent contributions from other diagrams, and “perms”
represents other diagrams from permutations over internal points. A
combinatoric factor of $4!$ from summing over internal points cancels out the
prefactor $1/4!$ from Edgeworth expansion.

The Edgeworth expansion (3.7) involves an infinite sum. Correspondingly,
computing $g_{r}(x_{1},\cdots,x_{r})$ requires summing over infinitely many
Feynman diagrams. When all but finitely many terms in the expansion are
parametrically suppressed, the expansion can be truncated at finite order to
provide an approximation of $g_{r}(x_{1},\cdots,x_{r})$. We will apply these
rules to concrete examples later in this section and demonstrate how
approximations to $g_{r}(x_{1},\cdots,x_{r})$ can be obtained systematically.

While our focus is on neural network field theories, we emphasize that
Edgeworth expansions can be utilized in any field theory where the connected
correlation functions are known, and the expansion in (3.7) is not divergent.

####  3.2.1 Example: Non-local $\phi^{4}$ Theory.

Aside from any application in NN-FT, it is interesting to study the self-
consistency of the Edgeworth expansion. We do so in a famous case, $\phi^{4}$
theory, generalized to the case of non-local quartic interactions, in order to
demonstrate the ability of the Edgeworth method to handle non-locality.
Consider the action

$\displaystyle S[{\phi}]$ $\displaystyle=\int
d^{d}x_{1}d^{d}x_{2}\,\frac{1}{2}\phi(x_{1})G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}\phi(x_{2})$
$\displaystyle+\frac{1}{4!}\int
d^{d}x_{1}d^{d}x_{2}d^{d}x_{3}d^{d}x_{4}\lambda(x_{1},x_{2},x_{3},x_{4})\phi(x_{1})\phi(x_{2})\phi(x_{3})\phi(x_{4}),$
(3.26)

where $G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}$ and
$\lambda(x_{1},x_{2},x_{3},x_{4})$ are both totally symmetric, and
$G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}$ is the operator in the free action
$S_{G}[\phi]$. We denote the free propagator $D(x_{1},x_{2})$ so that $\int
d^{d}x^{\prime}\,G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}\,D(x^{\prime},x_{2})=\delta^{d}(x_{1}-x_{2})$.
We can then expand $G_{c}^{(2)}(x_{1},x_{2})$ in
$\lambda(x_{1},x_{2},x_{3},x_{4})$, and at leading order,

$\displaystyle G_{c}^{(2)}(x_{1},x_{2})=D(x_{1},x_{2})+\frac{1}{2}\int
d^{d}y_{1}\cdots
d^{d}y_{4}\lambda(y_{1},y_{2},y_{3},y_{4})D(x_{1},y_{1})D(y_{2},y_{3})D(y_{4},x_{2}),$
(3.27)

where the $\frac{1}{2}$ is a symmetry factor. Similarly,

$\displaystyle G^{(4)}_{c}(x_{1},\cdots,x_{4})=$ $\displaystyle\,\int
d^{d}x^{\prime}_{1}\cdots
d^{d}x^{\prime}_{4}\,\lambda(x^{\prime}_{1},x^{\prime}_{2},x^{\prime}_{3},x^{\prime}_{4})D(x_{1},x^{\prime}_{1})D(x_{2},x^{\prime}_{2})D(x_{3},x^{\prime}_{3})D(x_{4},x^{\prime}_{4})$
$\displaystyle+O(\lambda^{2}).$ (3.28)

There are no other connected correlators that have contributions at
$O(\lambda)$. To perform an Edgeworth expansion, we first need to write down
the inverse propagator,

$G_{{c}}^{(2)}(x_{1},x_{2})^{-1}=G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}-\frac{1}{2}\int
d^{d}x_{3}d^{d}x_{4}\lambda(x_{1},x_{2},x_{3},x_{4})D(x_{3},x_{4})+O(\lambda^{2}).$
(3.29)

Given (3.29), it is easy to verify that

$\int
dx^{\prime}G_{{c}}^{(2)}(x_{1},x^{\prime})^{-1}G^{(2)}_{{c}}(x^{\prime},x_{2})=\delta(x_{1}-x_{2})+O(\lambda^{2}).$
(3.30)

At this point, let us introduce a shorthand notation to improve readability,
rewriting $\int d^{d}x_{1}d^{d}x_{2}\,G_{c}^{(2)}(x_{1},x_{2})$, $\int
d^{d}x_{1}d^{d}x_{2}\,G_{{c}}^{(2)}(x_{1},x_{2})^{-1}$ and $\int
d^{d}x_{1}\cdots d^{d}x_{4}\,G^{(4)}_{c}(x_{1},\cdots,x_{4})$ as,

$\displaystyle G_{xy}$
$\displaystyle=D_{xy}+\frac{1}{2}\lambda_{1234}D_{1x}D_{23}D_{4y}+O(\lambda^{2}),$
(3.31) $\displaystyle G_{xy}^{-1}$
$\displaystyle=G_{G,\phi}^{(2)}(x,y)^{-1}-\frac{1}{2}\lambda_{xy12}D_{12}+O(\lambda^{2}),$
(3.32) $\displaystyle G_{1234}$
$\displaystyle=\lambda_{1^{\prime}2^{\prime}3^{\prime}4^{\prime}}D_{1^{\prime}1}D_{2^{\prime}2}D_{3^{\prime}3}D_{4^{\prime}4}+O(\lambda^{2}),$
(3.33)

respectively. Finally, we obtain the Edgeworth expansion at $O(\lambda)$ by
plugging in (3.29) and (3.2.1) into (3.7),

$P[\phi]=\frac{1}{Z}\exp{\Bigl{(}\frac{1}{4!}G_{1234}\delta_{1}\delta_{2}\delta_{3}\delta_{4}\Bigr{)}}\exp{\Bigl{(}-\frac{1}{2}\phi_{x}G^{-1}_{xy}\phi_{y}\Bigr{)}}+O(\lambda^{2}),$
(3.34)

where $\delta_{1}:=\delta/\delta\phi(x_{1})$. Expanding the first exponential
and performing the derivatives we obtain

$P[\phi]=\frac{1}{Z}\Big{[}1-\frac{\lambda_{1234}}{8}D_{12}D_{34}-\frac{\lambda_{1234}}{4!}\phi_{1}\phi_{2}\phi_{3}\phi_{4}\Big{]}\exp{\Big{(}-\frac{1}{2}\phi_{x}G_{G,\phi}^{(2)}(x,y)^{-1}\phi_{y}\Big{)}}+O(\lambda^{2}),$
(3.35)

with $\lambda_{1234}:=\int d^{d}x_{1}\cdots
d^{d}x_{4}\,\lambda(x_{1},\cdots,x_{4})$, and $\phi_{x}:=\phi(x)$. The second
term does not depend on $\phi$ and can be absorbed into the normalization
factor, resulting in

$P[\phi]=\frac{1}{Z^{\prime}}\exp{\Big{(}-\frac{1}{2}\phi_{x}G_{G,\phi}^{(2)}(x_{1},x_{2})^{-1}\phi_{y}-\frac{\lambda_{1234}}{4!}\phi_{1}\phi_{2}\phi_{3}\phi_{4}\Big{)}}+O(\lambda^{2}).$
(3.36)

We have recovered the $\phi^{4}$ action at $O(\lambda)$, as expected.

###  3.3 General Interacting Actions in NN-FT

We now study the Edgeworth expansion in neural network field theories. We will
modify the general analysis of the previous section to the case where non-
Gaussianities are generated by the two mechanisms we described in Section 2,
namely, by violating assumptions of the CLT by finite $N$ corrections and
independence breaking.

Interactions from $1/N$-corrections. As we discussed in section 2.3.1, non-
Gaussianities arising due to $1/N$ corrections result in connected correlation
functions that scale as

$G_{{c}}^{(r)}(x_{1},\cdots,x_{r})\propto\frac{1}{N^{r/2-1}},$ (3.37)

for a single hidden layer network. At large $N$, the action can be
approximated systematically by organizing the Edgeworth expansion in powers of
$1/N$, calculating the couplings via Feynman diagrams, and truncating at a
fixed order in $1/N$.

To do so, we need to know how the couplings scale with $N$. We have studied a
case in (3.25) where only the even-point correlators are non-zero, and clearly
there is a $1/N$ contribution to $g_{4}$ from a single $G^{(4)}_{c}$ vertex;
any higher order correlator $G^{(r>4)}_{c}$ contributes at $1/N^{r/2-1}$ and
higher. Consider now contributions to the couplings $g_{r>4}$. There is a
tree-level $1/N^{r/2-1}$ contribution from a single $G^{(r)}_{c}$ vertex and
there are $1/N^{n/2-1}$ contributions from a $G^{(n>r)}_{c}$ vertex with an
appropriate number of loops; both are more suppressed than the $1/N$
contribution to $g_{4}$. Finally, consider contributions from $V$ number of
$G^{(n<r)}_{c}$ vertices. Forming a connected diagram requires $nV>r$, which
implies $V\geq 2$ and therefore the contribution is of order $1/N^{\geq n-1}$,
which is more suppressed than $1/N$ since $n$ begins at $3$ in the Edgeworth
expansion. Therefore, the single-vertex tree-level contribution to $g_{4}$ is
the leading contribution in $1/N$.

The quartic coupling $g_{4}(x_{1},x_{2},x_{3},x_{4})$, at leading order in
$G_{{c}}^{(4)}\propto 1/N$, is

$\displaystyle g_{4}(x_{1},\dots,x_{4})=$
$\displaystyle\,\,-\frac{1}{4!}\Bigg{[}\int
dy_{1}dy_{2}dy_{3}dy_{4}\,G^{(4)}_{c}(y_{1},y_{2},y_{3},y_{4})\,G_{{c}}^{(2)}(y_{1},x_{1})^{-1}G_{{c}}^{(2)}(y_{2},x_{2})^{-1}$
$\displaystyle\quad
G_{{c}}^{(2)}(y_{3},x_{3})^{-1}G_{{c}}^{(2)}(y_{4},x_{4})^{-1}\,+\text{perms}\Bigg{]}+O\left(\frac{1}{N^{2}}\right),$
(3.38) $\displaystyle=$ $\displaystyle\quad\vbox{\hbox{ \leavevmode\hbox
to0pt{\vbox to0pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(4)}_{c}$}}; \vertex[label=left:$x_{1}$] (a) at (-1,1)
; \vertex[label=left:$x_{2}$] (b) at ( -1,-1); \vertex[label=right:$x_{3}$]
(c) at (1, 1); \vertex[label=right:$x_{4}$] (d) at (1,-1); \vertex(a1) at
(-0.31,0.31) ; \vertex(b1) at ( -0.31,-0.31); \vertex(c1) at (0.31, 0.31);
\vertex(d1) at (0.31,-0.31); \diagram* { (a) -- [dashed] (m) -- [dashed](b),
(c) -- [dashed] (m) -- [dashed] (d) }; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}+O\left(\frac{1}{N^{2}}\right).$
(3.39)

We may compute this coupling in a NN-FT by first computing $G^{(4)}_{c}$ in
parameter space.

In summary, the leading-order in $1/N$ action for a single layer NN-FT is

$S=S_{G}+\int d^{d}x_{1}\dots
d^{d}x_{4}\,\,\,g_{4}(x_{1},\dots,x_{4})\,\phi(x_{1})\dots\phi(x_{4})+O\left(\frac{1}{N^{2}}\right),$
(3.40)

where $g_{4}$ at $O(1/N)$ is given in (3.39), under the assumption that the
odd-point functions are zero, as in the architectures of Section (3.4).

Interactions from Independence Breaking. Non-Gaussianities generated via
independence breaking alone are qualitatively different than those from $1/N$
corrections.

We wish to determine the leading-order action due to independence breaking.
Focusing on the case where independence breaking is controlled by a single
parameter $\alpha$ for simplicity, it follows from (2.52), that the connected
correlation functions scale as

$G_{{c}}^{(r)}(x_{1},\cdots,x_{r})\propto\alpha\quad\forall r>2$ (3.41)

at $N\rightarrow\infty$ limit, since the connected correlators
$G_{{c}}^{(r),\text{free}}|_{r>2}$ of the free theory vanish.

As a result, each coupling $g_{r}(x_{1},\cdots,x_{r})$ receives contributions
from tree-level diagrams of all connected correlators, at leading order in
$\alpha$. More generally, at any given order in $\alpha$, there are infinitely
many diagrams from all connected correlators to $g_{r}(x_{1},\cdots,x_{r})$.
For example, the expansion for $g_{4}(x_{1},x_{2},x_{3},x_{4})$ at $O(\alpha)$
includes terms proportional to $G_{{c}}^{(2n)}(x_{1},\cdots,x_{2n})$ for all
$n>1$,

$\displaystyle
g_{4}(x_{1},x_{2},x_{3},x_{4})=-\sum_{n=2}^{\infty}\frac{(-1)^{2n}}{(2n)!}\Big{[}\int
dy_{1}\cdots
dy_{2n}\,G_{{c}}^{(2n)}(y_{1},\cdots,y_{2n})\,G_{{c}}^{(2)}(y_{1},x_{1})^{-1}\,$
$\displaystyle\quad
G_{{c}}^{(2)}(y_{2},x_{2})^{-1}\,G_{{c}}^{(2)}(y_{3},x_{3})^{-1}\,G_{{c}}^{(2)}(y_{4},x_{4})^{-1}\,\prod_{m=5}^{2n-1}G_{{c}}^{(2)}(y_{m},y_{m+1})^{-1}+\text{perms}\Big{]}+O(\alpha^{2}),$
(3.42) $\displaystyle=$ $\displaystyle(-1)^{2n}\Bigg{(}\vbox{\hbox{
\leavevmode\hbox to0pt{\vbox to0pt{\pgfpicture\makeatletter\hbox{\hskip
0.0pt\lower 0.0pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(4)}_{c}$}}; \vertex[label=left:$x_{1}$] (a) at
(-0.7,0.7) ; \vertex[label=left:$x_{2}$] (b) at ( -0.7,-0.7);
\vertex[label=right:$x_{3}$] (c) at (0.7, 0.7); \vertex[label=right:$x_{4}$]
(d) at (0.7,-0.7); \vertex(a1) at (-0.31,0.31) ; \vertex(b1) at (
-0.31,-0.31); \vertex(c1) at (0.31, 0.31); \vertex(d1) at (0.31,-0.31);
\diagram* { (a) -- [dashed] (m) -- [dashed](b), (c) -- [dashed] (m) --
[dashed] (d) }; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}+\vbox{\hbox{
\leavevmode\hbox to0pt{\vbox to0pt{\pgfpicture\makeatletter\hbox{\hskip
0.0pt\lower 0.0pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(6)}_{c}$}}; \vertex[label=left:$x_{1}$] (a) at
(-0.9,0.5) ; \vertex[label=left:$x_{2}$] (b) at ( -0.7,-0.7);
\vertex[label=right:$x_{3}$] (c) at (0.9, 0.5); \vertex[label=right:$x_{4}$]
(d) at (0.7,-0.7); \vertex(a1) at (-0.35,0.11) ; \vertex(b1) at (
-0.25,-0.51); \vertex(c1) at (0.35, 0.11); \vertex(d1) at (0.25,-0.51);
\vertex(d2) at (0.15,0.47); \vertex(d3) at (-0.15,0.47); \vertex(d4) at
(0,1.1); \diagram* { (a) -- [dashed] (m) -- [dashed](b), (c) -- [dashed] (m)
-- [dashed] (d), (d3) -- [dashed,half left] (d4) -- [dashed, half left] (d2)}
; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}+\vbox{\hbox{
\leavevmode\hbox to0pt{\vbox to0pt{\pgfpicture\makeatletter\hbox{\hskip
0.0pt\lower 0.0pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\feynman every vertex/.style={red,
dot}; every particle/.style={blue}; \vertex[empty dot] (m) at (0,0)
{\contour{black}{}{$G^{(8)}_{c}$}}; \vertex[label=left:$x_{1}$] (a) at
(-1.0,0.5) ; \vertex[label=left:$x_{2}$] (b) at ( -1.0,-0.5);
\vertex[label=right:$x_{3}$] (c) at (0.9, 0.5); \vertex[label=right:$x_{4}$]
(d) at (0.9,-0.5); \vertex(a1) at (-0.35,0.11) ; \vertex(b1) at (
-0.35,-0.11); \vertex(c1) at (0.35, 0.11); \vertex(d1) at (0.35,-0.11);
\vertex(d2) at (0.15,0.47); \vertex(d3) at (-0.15,0.47); \vertex(d4) at
(0,1.1); \vertex(d6) at (0.15,-0.47); \vertex(d5) at (-0.15,-0.47);
\vertex(d7) at (0,-1.1); \diagram* { (a) -- [dashed] (m) -- [dashed](b), (c)
-- [dashed] (m) -- [dashed] (d), (d6) -- [dashed,half left] (d7) -- [dashed,
half left] (d5), (d3) -- [dashed,half left] (d4) -- [dashed, half left] (d2)}
; \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}}+\cdots\Bigg{)}+O(\alpha^{2}),$
(3.43)

where summing over internal points $y_{i}$ cancels out $\frac{1}{2n!}$
prefactor from each $G^{(2n)}_{c}$. The terms in the parenthesis constitute an
infinite sum.

This structure makes it impossible to systematically approximate
$g_{r}(x_{1},\cdots,x_{r})$ with a finite number of terms via a perturbative
expansion in $\alpha$, unless some other structure correlates with it. Note
that this is a feature of neural network field theories where non-
Gaussianities are generated only by independence breaking. Approximation via a
finite number of terms would be possible in cases where connected correlation
functions scale with both $\alpha$ and $1/N$. In the limit of
$N\rightarrow\infty$, the leading-order in $\alpha$ action for a NN-FT is

$S=S_{G}+\sum_{r=4}^{\infty}\int d^{d}x_{1}\dots
d^{d}x_{r}\,g_{r}(x_{1},\dots,x_{r})\,\phi(x_{1})\dots\phi(x_{r})+O(\alpha^{2}),$
(3.44)

where $g_{r>4}$’s are computed similar to (3.43). Such an action can not be
approximated by a finite truncation, unless the theory exhibits additional
structure.

###  3.4 Example Actions in NN-FT

Next, we exemplify the Feynman rules from Section (3.2) in a few single layer
NN architecture examples at finite width and i.i.d. parameters, and evaluate
the leading order in $1/N$ quartic coupling and NN-FT action. The quartic
coupling is

$\displaystyle g_{4}(x_{1},\cdots,x_{4})=-\frac{1}{4!}\Big{[}\int
d^{d}y_{1}\cdots
d^{d}y_{4}\,G^{(4)}_{c}(y_{1},\cdots,y_{4})G_{{c}}^{(2)}(y_{1},x_{1})^{-1}\cdots
G_{{c}}^{(2)}(y_{4},x_{4})^{-1}+\text{perms}\Big{]},$ (3.45)

at $O(1/N)$. When $G_{{c}}^{(2)}(x_{1},y_{1})^{-1}$ involves differential
operators, we use the methods from Appendix (Appendix D. Fourier
Transformation Trick for $G^{(2)}_{c}(x,y)^{-1}$) to evaluate $g_{4}$.

#### Single Layer Cos-net

Recall the Cos-net architecture introduced earlier,
$\phi(x)=W^{1}_{i}\cos(W^{0}_{ij}x_{j}+b^{0}_{i})$, for
$W^{1}\sim\mathcal{N}(0,\sigma^{2}_{W_{1}}/N)$,
$W^{0}\sim\mathcal{N}(0,\sigma^{2}_{W_{0}}/d)$, and
$b^{0}\sim\text{Unif}[-\pi,\pi]$. We will consider the case where all
parameters are independent and non-Gaussianities arise due to finite $N$
corrections. To evaluate the leading order quartic coupling for this NNFT, let
us first compute the inverse propagator
$G^{(2)}_{c,\text{Cos}}(x_{1},x_{2})^{-1}$, starting from the $2$-pt function

$\displaystyle
G^{(2)}_{c,\text{Cos}}(x_{1},x_{2})=\frac{\sigma_{W_{1}}^{2}}{2}e^{-\frac{\sigma_{W_{0}}^{2}(x_{1}-x_{2})^{2}}{2d}},$
(3.46)

and inversion relation $\int
d^{d}y\,G^{(2)}_{c,\text{Cos}}(x,y)^{-1}\,G_{{c,\text{Cos}}}^{(2)}(y,z)=\delta^{d}(x-z)$.
Translation invariance of the $2$-pt function and delta function constraints
$G^{(2)}_{c,\text{Cos}}(x,y)^{-1}$ as a translation invariant operator. Then,
performing a Fourier transformation of the $2$-pt function and its inverse
operator, followed by an inverse Fourier transformation, we obtain

$G^{(2)}_{c,\text{Cos}}(x,y)^{-1}=\frac{2\sigma^{2}_{W_{0}}}{\sigma_{W_{1}}^{2}d}\,e^{-\frac{\sigma_{W_{0}}^{2}\,\nabla^{2}_{x}}{2d}}\delta^{d}(x-y),$
(3.47)

where $\nabla^{2}_{x}:=\partial^{2}/\partial x^{2}$. Here, we use (D.3) to
evaluate the quartic coupling as,

$\displaystyle g_{4}^{\text{Cos}}(x_{1},\cdots,x_{4})=$ $\displaystyle-\int
d^{d}p_{1}\cdots
d^{d}p_{4}\,\tilde{G}_{{c,\text{Cos}}}^{(4)}(p_{1},\cdots,p_{4})\,\tilde{G}_{{c,\text{Cos}}}^{(2)}(-p_{1})^{-1}\cdots\tilde{G}_{{c,\text{Cos}}}^{(2)}(-p_{4})^{-1}$
$\displaystyle e^{-ip_{1}x_{1}\cdots-ip_{4}x_{4}},$ (3.48)

where $\tilde{G}_{{c,\text{Cos}}}^{(4)}(p_{1},\cdots,p_{4})$ is from (Cos-net
Cumulants at Finite $N$, Non-i.i.d. Parameters), and
$\tilde{G}_{{c,\text{Cos}}}^{(2)}(-p)^{-1}=\frac{2\sigma_{W_{0}}}{\sqrt{d}\sigma^{2}_{W_{1}}}e^{\frac{dp^{2}}{2\sigma^{2}_{W_{0}}}}$.
Using this,

$\displaystyle
g_{4}^{\text{Cos}}(x_{1},x_{2},x_{3},x_{4})=-\frac{4\sqrt{6}\pi^{3/2}\sigma^{4}_{W_{0}}}{Nd^{2}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(abcd)}e^{-\frac{\sigma^{2}_{W_{0}}\nabla^{2}_{r_{abcd}}}{6d}}+\frac{8\pi\sigma^{4}_{W_{0}}}{Nd^{2}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(ab,cd)}e^{-\frac{\sigma^{2}_{W_{0}}(\nabla^{2}_{r_{ab}}+\nabla^{2}_{r_{cd}})}{2d}}.$
(3.49)

We introduce the abbreviation $r_{abcd}:=x_{a}+x_{b}-x_{c}-x_{d}$, and
$\mathcal{P}(abcd)=12$ refers to the number of ways ordered list of indices
$a,c,b,d\in\\{1,2,3,4\\}$ can be chosen. Similarly, $r_{ab}:=x_{a}-x_{b}$, and
$\mathcal{P}(ab,cd)=12$ is the number of ways ordered pairs
$(a,c),(b,d)\in\\{1,2,3,4\\}$ can be drawn.

With this, Cos-net field theory action at $O(1/N)$ is

$\displaystyle S_{\text{Cos}}[\phi]=$
$\displaystyle\,\frac{2\sigma^{2}_{W_{0}}}{\sigma_{W_{1}}^{2}{d}}\int
d^{d}x\,\phi(x)\,e^{-\frac{\sigma_{W_{0}}^{2}\,\nabla^{2}_{x}}{2d}}\phi(x)\,-\int
d^{d}x_{1}\cdots
d^{d}x_{4}\Bigg{[}\frac{4\sqrt{6}\pi^{3/2}\sigma^{4}_{W_{0}}}{Nd^{2}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(abcd)}e^{-\frac{\sigma^{2}_{W_{0}}\nabla^{2}_{r_{abcd}}}{6d}}$
$\displaystyle-\frac{8\pi\sigma^{4}_{W_{0}}}{Nd^{2}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(ab,cd)}e^{-\frac{\sigma^{2}_{W_{0}}(\nabla^{2}_{r_{ab}}+\nabla^{2}_{r_{cd}})}{2d}}\Bigg{]}\phi(x_{1})\cdots\phi(x_{4})+O(1/N^{2}).$
(3.50)

The NNGP action is local, but the leading order quartic interaction is non-
local.

#### Single Layer Gauss-net

As our next example, consider the output of a single-layer Gauss-net

$\displaystyle\phi(x)=\frac{W^{1}_{i}\exp(W^{0}_{ij}x_{j}+b^{0}_{i})}{\sqrt{\exp[2(\sigma_{b_{0}}^{2}+\frac{\sigma^{2}_{W_{0}}}{d}x^{2})]}},$
(3.51)

for parameters drawn i.i.d. from
$W^{0}\sim\mathcal{N}(0,\frac{\sigma^{2}_{W_{0}}}{d})$,
$W^{1}\sim\mathcal{N}(0,\frac{\sigma^{2}_{W_{1}}}{N})$, and
$b^{0}\sim\mathcal{N}(0,\sigma^{2}_{b_{0}})$. The propagator is identical to
Cos-net field theory, and so is $G^{(2)}_{c,\text{Gauss}}(x_{1},x_{2})^{-1}$.
We evaluate Gauss-net quartic coupling $g_{4}$, using (D.3), and (Gauss-net at
Finite $N$, Non-i.i.d. Parameters) for $\tilde{G}_{{c,\text{Gauss}}}^{(4)}$,
as

$\displaystyle
g_{4}^{\text{Gauss}}(x_{1},\cdots,x_{4})=-\frac{4\sqrt{2}\,\pi^{3/2}\sigma^{4}_{W_{0}}}{\sqrt{3}N^{2}d^{4}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(abcd)}\bigg{[}d^{2}N+2\sigma^{4}_{W_{0}}-\frac{\sigma^{5}_{W_{0}}(d-\sigma^{2}_{W_{0}}\nabla^{2}_{r_{abcd}})}{d^{3/2}}\bigg{]}e^{-\frac{\sigma^{2}_{W_{0}}\nabla^{2}_{r_{abcd}}}{6d}}$
$\displaystyle\qquad+\frac{8\pi\sigma^{4}_{W_{0}}}{N^{2}d^{4}\sigma^{4}_{W_{1}}}\sum_{\mathcal{P}(ab,cd)}\bigg{[}d^{2}N+6\sigma^{4}_{W_{0}}-4d^{3}\sigma^{5}_{W_{0}}+\frac{\sigma^{6}_{W_{0}}}{d}+\Big{(}\frac{2\sigma^{7}_{W_{0}}}{d^{3/2}}-\frac{\sigma^{8}_{W_{0}}}{d^{2}}\Big{)}(\nabla^{2}_{r_{ab}}+\nabla^{2}_{r_{cd}})$
$\displaystyle\qquad+\frac{\sigma^{10}_{W_{0}}}{d^{3}}\nabla^{2}_{r_{ab}}\nabla^{2}_{r_{cd}}\bigg{]}e^{-\frac{\sigma^{2}_{W_{0}}(\nabla^{2}_{r_{ab}}+\nabla^{2}_{r_{cd}})}{2d}},$
(3.52)

where $\mathcal{P}(ab,cd)$ and $\mathcal{P}(abcd)$ are defined as before.

Thus, Gauss-net field theory action at $O(1/N)$,

$\displaystyle
S_{\text{Gauss}}[\phi]=\,\frac{2\sigma^{2}_{W_{0}}}{\sigma_{W_{1}}^{2}{d}}\int
d^{d}x\,\phi(x)\,e^{-\frac{\sigma_{W_{0}}^{2}\,\nabla^{2}_{x}}{2d}}\phi(x)\,+\int
d^{d}x_{1}\cdots
d^{d}x_{4}\,g_{4}^{\text{Gauss}}\,\phi(x_{1})\cdots\phi(x_{4}),$ (3.53)

differs from Cos-net field theory at the level of quartic interaction.

##  4 Engineering Actions: Generalities, Locality, and $\phi^{4}$ Theory

In Section 3 we used the Edgeworth expansion and a “duality” between fields
and sources to compute couplings (including non-local ones) in the action as
connected Feynman diagrams whose vertices are given by the usual connected
correlators $G^{(n)}_{c}(x_{1},\dots,x_{n})$. This general field theory result
is applicable in NN-FT of fixed architectures, but it doesn’t answer the
question of how to engineer an architecture that realizes a given action.

In this section we study how to design actions of a given type by deforming a
Gaussian theory by an arbitrary operator. The result is simple and exploits
the duality between the parameter-space and function-space descriptions of a
field theory. The main results are:

  * •

Action Deformations. We develop a mechanism for expressing an arbitrary
deformation of a Gaussian action as a deformation of the parameter density of
a NN-FT.

  * •

Local Lagrangians. We utilize the mechanism to engineer local interactions.

  * •

$\phi^{4}$ Theory as a NN-FT. Using a previous result that achieves free
scalar field theory as a NN-FT, we engineer local $\phi^{4}$ theory as an NN-
FT.

  * •

Cluster Decomposition. We develop an approach to cluster decomposition,
another notion of locality that is weaker than local interactions.

We also discuss why it might have been expected that $\phi^{4}$ theory (and
other well-studied field theories) arises naturally at infinite-$N$.

To begin our analysis, consider the partition function of a Gaussian theory

$Z_{G}[J]=\mathbb{E}_{G}[e^{\int d^{d}x\,J(x)\phi(x)}],$ (4.1)

where we have labelled both the partition function and the expectation with a
$G$ subscript to emphasize Gaussianity.

Now we wish to define a deformed theory that differs from the original only by
an operator insertion, treating it in both function space and parameter space.
The deformed partition function is given by

$Z[J]=\mathbb{E}_{G}[e^{-\lambda\int d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi}(x_{1},\dots,x_{r})}e^{\int
d^{d}x\,J(x)\phi(x)}],$ (4.2)

where $\mathcal{O_{\phi}}$ is a non-local operator (though it may be chosen to
be local) that has a subscript $\phi$, denoting that it may depend on $\phi$
and its derivatives. In the function space, the partition function of the
Gaussian theory is

$Z_{G}[J]=\int D\phi\,e^{-S_{G}[\phi]+\int d^{d}xJ(x)\phi(x)},$ (4.3)

and the operator insertion corresponds to a deformation of the partition
function to

$Z[J]=\int D\phi\,e^{-S[\phi]+\int d^{d}xJ(x)\phi(x)}$ (4.4)

where the action has been deformed

$S_{G}[\phi]\to S[\phi]=S_{G}[\phi]+\lambda\int d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi}(x_{1},\dots,x_{r}).$ (4.5)

We may treat this theory in perturbation theory in the usual way: correlators
in the non-Gaussian theory are expanded perturbatively in $\lambda$ and
evaluated using the Gaussian expectation $\mathbb{E}_{G}$, which utilizes the
Gaussian action when expressed in function-space.

How is this deformation expressed in parameter space, i.e., how do we think of
this deformation from a neural network perspective? In parameter space, the
Gaussian partition function is

$Z_{G}[J]=\int d\theta P_{G}(\theta)\,e^{\int d^{d}xJ(x)\phi_{\theta}(x)},$
(4.6)

We remind the reader that in such a case Gaussianity is not obvious, but
requires a judicious choice of parameter density $P(\theta)$ and architecture
$\phi_{\theta}(x)$ such that we have a neural network Gaussian process via the
central limit theorem. In parameter space, the deformation yields

$Z[J]=\int d\theta P_{G}(\theta)\,\,e^{-\lambda\int d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi_{\theta}}(x_{1},\dots,x_{r})}e^{\int
d^{d}xJ(x)\phi_{\theta}(x)},$ (4.7)

where we assume that where the operator $\mathcal{O}_{\phi_{\theta}}$ doesn’t
involve an explicit $\phi(x)$, but instead its parameter space representation;
we will exemplify this momentarily. Again, correlators may be computed in
perturbation theory in $\lambda$ by expanding and evaluating in the Gaussian
expectation, this time in the parameter space formulation.

We emphasize that if the function space and parameter space descriptions (4.3)
and (4.6) represent the same partition function, then the deformed theories
(4.4) and (4.7) are the same theory. That is, we see how an arbitrary
deformation of the action induces an associated deformation of the parameter
space description. We will use this in Section 4.2 to engineer $\phi^{4}$
theory as a neural network field theory, and in 4.1 we will more explicitly
deform a neural network Gaussian process.

We end our general discussion with some theoretical considerations in neural
network field theory, interpreting a non-Gaussian deformation
$\mathcal{O}_{\phi_{\theta}}$ in terms of the framework of Section 2, and also
taking into account the universal approximation theorem.

A non-Gaussian deformation $\mathcal{O}_{\phi_{\theta}}$ must violate an
assumption of the CLT. The architecture itself is still the same
$\phi_{\theta}(x)$ as in the Gaussian theory. Instead, in (4.7) we may
interpret the operator insertion as

$P(\theta):=P_{G}(\theta)\,\,e^{-\lambda\int d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi_{\theta}}(x_{1},\dots,x_{r})},$ (4.8)

i.e., same architecture, but with a deformed parameter distribution. This
makes it clear that our non-Gaussian theory is still at infinite-$N$ and
therefore cannot receive non-Gaussianities in $1/N$-corrections. Instead, it
receives non-Gaussianities because the deformed parameter distribution has
independence breaking via the non-trivial relationship amongst the parameters
in the deformation. There may also exist schemes for controlling non-Gaussian
deformations in $1/N$, instead of via independence breaking, but it is beyond
our scope.

Was it inevitable that systematic control over non-Gaussianities arises most
naturally via independence breaking rather than $1/N$-corrections? The general
answer is not clear, but we may use the control over non-Gaussianities to
yield common theories, such as $\phi^{4}$ theory in the next section. In that
context we may ask a related question: was it inevitable that we obtain common
interacting theories via independence breaking rather than $1/N$ corrections?
This question has a better answer. Finite action configurations of a common
theory, say $\phi^{4}$ theory

$S[\phi]=\int
d^{d}x\,\left[\phi(x)(\nabla^{2}+m^{2})\phi(x)+\frac{\lambda}{4!}\,\phi(x)^{4}\right],$
(4.9)

are not _arbitrary_ functions, since there may be some functions $\phi(x)$
that have infinite action. However, finite action configurations are still
fairly general functions, and since they have finite action they occur with
non-zero probability in the ensemble.

On the other hand, there are universal approximation theorems for neural
networks, where the error in the approximation to a target function may
decrease with increasing $N$. In such a case this theorem that is usually
cited as a feature in ML may actually be a bug: at finite-$N$ there exist
functions that can’t be explicitly realized by a fixed architecture, but only
approximated. We therefore find it reasonable to expect that there is at least
one finite-action configuration $\phi(x)$ in $\phi^{4}$ theory that cannot be
realized by a finite-$N$ neural network of fixed architecture; in such a case,
a NN-FT realization of $\phi^{4}$ theory must be at infinite-$N$. This comment
only scratches the surface, but we find the interplay between universal
approximation theorems and realizable field theories at finite-$N$ to be
worthy of further study.

###  4.1 Non-Gaussian Deformation of a Neural Network Gaussian Process

To make the general picture more concrete, we would like to consider non-
Gaussian deformations of any neural network Gaussian process. The main result
is that we may deform any NNGP by any operator we like, which breaks
independence by deforming the parameter density, explaining the origin of non-
Gaussianities by violating the independence.

As before, we consider a field built out of neurons,

$\phi_{\theta}(x)=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}a_{i}h_{i}(x)$ (4.10)

where the full set of parameters $\theta$ is realized by the set of parameters
$a_{i}$ and the set of parameters $\theta_{h}$ of the post-activations or
neurons $h$. This equation forms the field out of a linear output layer with
weights $a_{i}$ acting on the post-activations, which could themselves be
considered as the $N$-dimensional output of literally any neural network. If
the reader wishes, one may take $\phi$ to be a single-layer network by further
choosing

$h_{i}(x)=\sigma(b_{ij}x_{j}+c_{i})$ (4.11)

with $\sigma:\mathbb{R}\to\mathbb{R}$ a non-linear activation function such as
ReLU or tanh; with this additional choice we now have $\theta_{h}$ comprised
of $b$-parameters and $c$-parameters. Taking the parameter densities
$P_{G}(a)$ and $P_{G}(\theta_{h})$ to be independent and $N\to\infty$,
$\phi(x)=\phi_{\theta}(x)$ is drawn from a Gaussian process; we have again
used a subscript $G$ to emphasize that these are the parameter densities of
the Gaussian theory.

Deforming the Gaussian theory by an operator insertion, which in general is
non-Gaussian, we have

$Z[J]=\int da\,d\theta_{h}\,\,P_{G}(a)P_{G}(\theta_{h})\,\,e^{-\lambda\int
d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi_{a,\theta_{h}}}(x_{1},\dots,x_{r})}e^{\int
d^{d}xJ(x)\phi_{\theta}(x)}.$ (4.12)

We may interpret the operator insertion as deforming the independent Gaussian
parameter density $P_{G}(a)P_{G}(\theta_{h})$ to a non-trivial joint density

$P(a,\theta_{h})=P_{G}(a)P_{G}(\theta_{h})\,e^{-\lambda\int d^{d}x_{1}\dots
d^{d}x_{r}\,\mathcal{O}_{\phi_{a,\theta_{h}}}(x_{1},\dots,x_{r})}.$ (4.13)

The partition function is then

$Z[J]=\int da\,d\theta_{h}\,\,P(a,\theta_{h})\,e^{\int
d^{d}xJ(x)\phi_{\theta}(x)},$ (4.14)

an infinite-$N$ non-Gaussian NN-FT where the operator insertion deforms the
parameter density. At initialization, if one draws the parameters $\theta_{h}$
first, one may think of this as affecting the density from which the
$a$-parameters are drawn; the draws of $a$-parameters are no longer
independent.

For the sake of concreteness, consider the case of the single-layer network
and take a general non-local quartic deformation. Then the operator insertion
is

$e^{-\int d^{d}x_{1}\dots d^{d}x_{4}\,g_{4}(x_{1},\dots
x_{4})\,\phi_{a,b,c}(x_{1})\dots\phi_{a,b,c}(x_{4})},$ (4.15)

where Einstein summation is implied and we have absorbed the overall $\lambda$
into the definition of the non-local coupling $g_{4}(x_{1},\dots,x_{4})$.
Inserting the equation for the neural network

$\phi_{a,b,c}(x)=\frac{1}{\sqrt{N}}a_{i}\sigma(b_{ij}x_{j}+c_{i}),$ (4.16)

into the deformation, we obtain

$e^{-\int d^{d}x_{1}\dots d^{d}x_{4}\,g_{4}(x_{1},\dots x_{4})\,a_{i_{1}}\dots
a_{i_{4}}\sigma(b_{i_{1}j_{1}}x_{j_{1}}+c_{i_{1}})\dots\sigma(b_{i_{4}j_{4}}x_{j_{4}}+c_{i_{4}})/N^{2}},$
(4.17)

which defines a deformed parameter density

$P(a,b,c)=P_{G}(a)P_{G}(b)P_{G}(c)\,e^{-\int d^{d}x_{1}\dots
d^{d}x_{4}\,g_{4}(x_{1},\dots x_{4})\,a_{i_{1}}\dots
a_{i_{4}}\sigma(b_{i_{1}j_{1}}x_{j_{1}}+c_{i_{1}})\dots\sigma(b_{i_{4}j_{4}}x_{j_{4}}+c_{i_{4}})/N^{2}}.$
(4.18)

Then

$Z[J]=\int da\,db\,dc\,\,\,P(a,b,c)\,e^{\int
d^{d}xJ(x)\,a_{i}\sigma(b_{ij}x_{j}+c_{i})/\sqrt{N}}$ (4.19)

is the partition function of a infinite-$N$ NN-FT, as we impose $\lim
N\to\infty$, with quartic non-Gaussianity induced by the breaking of
independence in the joint parameter density $P(a,b,c)$.

###  4.2 $\phi^{4}$ Theory as a Neural Network Field Theory

To end this section and demonstrate the power of this technique, we would like
to engineer the first interacting theory that any student learns: local
$\phi^{4}$ theory. The action is

$S[\phi]=\int
d^{d}x\,\left[\phi(x)(\nabla^{2}+m^{2})\phi(x)+\frac{\lambda}{4!}\,\phi(x)^{4}\right].$
(4.20)

Following our prescription, we

  * •

Engineer the NNGP. Using the result of [19], we take

$\phi_{a,b,c}(x)=\sum_{i}\frac{a_{i}\,\cos(b_{ij}x_{j}+c_{i})}{\sqrt{\textbf{b}_{i}^{2}+m^{2}}},$
(4.21)

where the sum runs from $1$ to $N=\infty$, $\textbf{b}_{i}$ is the vector that
is the $i^{\text{th}}$ row of the matrix $b_{ij}$, and the parameter densities
of the Gaussian theory are

$\displaystyle P_{G}(a)$
$\displaystyle=\prod_{i}e^{-\frac{N}{2\sigma_{a}^{2}}a_{i}a_{i}}$ (4.22)
$\displaystyle P_{G}(b)$
$\displaystyle=\prod_{i}P_{G}(\textbf{b}_{i})\,\,\,\text{with}\,\,\,P_{G}(\textbf{b}_{i})=\text{Unif}(B^{d}_{\Lambda})$
(4.23) $\displaystyle P_{G}(c)$
$\displaystyle=\prod_{i}P_{G}(c_{i})\,\,\,\text{with}\,\,\,P_{G}(c_{i})=\text{Unif}([-\pi,\pi]),$
(4.24)

where $B^{d}_{\Lambda}$ is a $d$-sphere of radius $\Lambda$. The density
$P_{G}(\textbf{b}_{i})$ is not independent in the vector index $j$, but all
that is needed for Gaussianity is independence in the $i$ index, which is
clear due to the product nature of $P_{G}(b)$. The power spectrum (Fourier-
transform of the two-point function) is

$G^{(2)}(p)=\frac{\sigma_{a}^{2}(2\pi)^{d}}{2\,\text{vol}(B^{d}_{\Lambda})}\frac{1}{p^{2}+m^{2}},$
(4.25)

which becomes the standard free scalar result $1/(p^{2}+m^{2})$ by a trivial
rescaling

$\phi_{a,b,c}(x)=\sqrt{\frac{2\,\text{vol}(B^{d}_{\Lambda})}{\sigma_{a}^{2}(2\pi)^{d}}}\,\,\,\sum_{i,j}\frac{a_{i}\,\cos(b_{ij}x_{j}+c_{i})}{\sqrt{\textbf{b}_{i}^{2}+m^{2}}}.$
(4.26)

This neural network Gaussian process is equivalent to the free scalar theory
of mass $m$ in $d$ Euclidean dimensions, with

$G^{(2)}(p)=\frac{1}{p^{2}+m^{2}},$ (4.27)

where $\Lambda$ plays the role of a hard UV cutoff on the momentum.

  * •

Introduce the Operator Insertion. Given the NNGP above, or any other NNGP
realizing the free scalar field theory, we wish to insert the operator

$e^{-\frac{\lambda}{4!}\int d^{d}x\,\phi_{a,b,c}(x)^{4}},$ (4.28)

associated to a local $\phi^{4}$ interaction.

  * •

Absorb the Operator into a Parameter Density Deformation. The non-Gaussian
operator insertion deforms the parameter density to

$P(a,b,c)=P_{G}(a)P_{G}(b)P_{G}(c)\,\,e^{-\frac{\lambda}{4!}\int
d^{d}x\,\phi_{a,b,c}(x)^{4}},$ (4.29)

where for $\phi_{a,b,c}(x)$ it is to be understood that the RHS of (4.26) is
inserted, yielding an expression that is only a function of $a$’s, $b$’s, and
$c$’s.

  * •

Write the Partition Function. We then have a partition function for the
deformed theory, given by

$Z[J]=\int da\,db\,dc\,\,\,P(a,b,c)\,\,e^{\int d^{d}xJ(x)\,\phi_{a,b,c}(x)},$
(4.30)

where again it is to be understood that we insert the RHS of (4.26) for
$\phi_{a,b,c}$ and (4.29) for $P(a,b,c)$; there are no explicit fields in the
expression, it depends only on the architecture (which includes parameters
a,b,c) and the joint parameter density.

Thus, the architecture (4.26) and parameter density (4.29) realize local
$\phi^{4}$ theory via the partition function (4.30). We discuss the
connections between Gaussian Processes, locality, and translation invariance
in Appendix (Appendix E. Gaussian Processes: Locality and Translation
Invariance).

Let us briefly address RG flows. The definition of a fixed non-Gaussian theory
here involves the choice of a fixed value of $\lambda$, in addition to the
choice of a fixed value of $\Lambda$ that was implicit in the fixing of the
GP. From that starting point, decreasing $\Lambda$ while keeping the
correlators fixed induces an RG flow for $\lambda$ governed by the usual
Callan-Symanzik equation. In the language of the neural network architecture,
this is interpreted as a flow in the parameter density that is necessary to
fix the correlators as $\Lambda$ is decreased.

###  4.3 Cluster Decomposition and Space Independence Breaking

We now turn to a weaker notion of locality: cluster decomposition. Given a
field $\phi(x)$ (or neural network in our context) we say that it satisfies
cluster decomposition if all connected correlation functions
$G^{(r)}_{c}(x_{1},\dots,x_{r})$ asymptote to zero in the limit where the
separation between any two space points $x_{i},x_{j},i\neq j$ is taken to
$\infty$,

$\lim_{|x_{i}-x_{j}|\to\infty}G^{(r)}_{c}(x_{1},\dots,x_{r})=0.$ (4.31)

If the probability density function of $\phi$ has the form

$P(\phi)=\frac{1}{Z}\exp{\Bigl{(}-\int
dx\,\mathcal{L}\Bigl{(}x,\phi(x),\frac{\partial\phi}{\partial
x},\dots,\frac{\partial^{n}\phi}{\partial x^{n}}\Bigr{)}\Bigr{)}}$ (4.32)

where $Z$ is a normalization constant and $n$ is finite, we say that $\phi(x)$
has a local Lagrangian density. This is a stronger notion of locality compared
to cluster decomposition, as any theory with a local Lagrangian density
satisfies cluster decomposition, but the converse is not true [72].

Checking whether a theory satisfies cluster decomposition requires knowledge
of the asymptotic behavior of correlation functions, but not the probability
density function. As calculating the probability density function of an NN-FT
is more challenging than computing the correlation functions, checking cluster
decomposition is easier than determining whether there exists a local
Lagrangian density that describes the system.

The main result we describe in this section is a framework that enables
engineering neural network architectures that satisfy cluster decomposition.

### Space Independent Field Theory

We will perform our analysis by studying, and then moving away from, a case
with a very strong assumption: field theories that are defined by fields that
have independent statistics at different space (or space) points $x_{I}$. We
call these field theories _space independent_ (SI) field theories. While one
can still view such fields as random functions defined on a continuously
differentiable space, in general the field configurations are discontinuous;
avoiding this would require statistical correlations between nearest
neighbors, violating the assumption. This “$d$-dimensional” field theory is
really a collection of uncountably many independent $0$-d theories. This means
that the partition function factorizes

$Z_{\phi_{\text{SI}}}[J]=\mathbb{E}[e^{\int
d^{d}x_{I}J(x_{I})\phi(x_{I})}]=\prod_{I}\mathbb{E}_{\phi_{\text{SI}}}[e^{J(x_{I})\phi(x_{I})}],$
(4.33)

where the product runs over all space points $x_{I}$. This form is agnostic
about the origin of the statistics and may be specified in either the function
space or parameter space description. In parameter space, independent
statistics at different space points $x_{I}$ means that the SI theory has
partition function

$Z_{\phi_{\text{SI}}}[J]=\prod_{i}\int
d\theta_{I}\,P_{I}(\theta_{I})\,e^{J(x_{I})\,\phi_{\theta_{I}}(x_{I})},$
(4.34)

i.e. each space point $x_{I}$ has its own ensemble of neural networks
$\phi_{\theta_{I}}(x_{I})$ with its own set of parameters $\theta_{I}$ that is
independent of $\theta_{J}$ for $I\neq J$. In function space, independence
means that

$Z_{\phi_{\text{SI}}}[J]=\prod_{I}\int
D\phi_{I}\,e^{-S[\phi(X_{I})]+{J(x_{I})\phi(x_{I})}},$ (4.35)

i.e., the action is such that the path integral factorizes. An immediate
consequence of this factorization is that the action cannot contain
derivatives of $\phi(x_{I})$, as these would depend on the value of $\phi$ not
only at point $x_{I}$, but a local neighborhood around it. Then, the action is
of the form,

$S(\phi(x_{I}))=V[\phi(x_{I})],$ (4.36)

which, turning the product into a sum in the exponent, gives the more
canonical form

$Z{\phi_{\text{SI}}}[J]=\int\left(\prod_{I}D\phi_{I}\right)\,e^{-\int
d^{d}x_{I}\,(V[\phi(X_{I})]-{J(x_{I})\phi(x_{I}))}}.$ (4.37)

This is a field theory with a potential, but no derivatives. The field values
at different points of space are independent random variables. If they are
identically distributed $V[\phi(x_{I})]$ is fixed $\forall I$ and the
different factors in $Z_{\text{SI}}[J]$ enjoy an $S_{L}$ permutation symmetry,
where the number of space points $L$ is infinite in the continuum limit.

Before introducing correlations between the field values at different space
points, let us first study the statistics of the SI theory. Denote the
cumulants of $\phi$ at a given point $x_{I}$ as $\kappa_{r}^{\phi}(x_{I})$.
For simplicity, we will assume that the field values at different space points
are identically distributed, i.e. $\kappa_{r}^{\phi}(x_{I})=\kappa_{r}^{\phi}$
is fixed for all $I$, which will also be important for translation invariance.
We also assume that they are mean free, $\kappa_{1}^{\phi}=0$. Next, we
consider the cumulant generating functional, which takes the form

$\displaystyle W_{\phi_{SI}}[J]$
$\displaystyle=\log\Bigl{(}Z_{\phi_{SI}}[J]\Bigr{)}=\log\Bigl{(}\prod_{I}\mathbb{E}_{\phi_{\text{SI}}}[e^{J(x_{I})\phi(x_{I})}]\Bigr{)},$
$\displaystyle=\int
dx\log\Bigl{(}\mathbb{E}_{\phi_{\text{SI}}}[e^{J(x)\phi(x)}]\Bigr{)},$
$\displaystyle=\int dx\,W[J;x],$ (4.38)

where $W[J;x]$ is the CGF of $\phi$ at space point $x$. Just as the partition
function $Z[J]$ factorizes into a product of partition functions associated to
individual space points, the CGF $W[J]=\log Z[J]$ becomes a sum (or integral,
in this case). The connected correlators are easily computed by taking
derivatives 555We remind the reader that space derivatives are ill-defined, as
$\phi(x)$ is discontinuous everywhere. However, derivatives with respect to
$J(x_{I})$ are still well defined.  , where $\partial J(x_{I})/\partial
J(x_{J})=\delta(x_{I}-x_{J})$,

$\displaystyle G^{(n)}_{c}(x_{1},\ldots,x_{n})$
$\displaystyle=\Bigl{(}\prod_{I=1}^{n}\frac{\partial}{\partial
J(x_{I})}\Bigr{)}W_{\phi_{SI}}[J],$ $\displaystyle=\int
dx\Bigl{(}\prod_{I=1}^{n}\frac{\partial}{\partial J(x_{I})}\Bigr{)}W[J;x].$
(4.39)

and the connected correlation functions of SI networks $\phi_{\text{SI}}$
simplifies to

$\displaystyle G^{(n)}_{c}(x_{1},\ldots,x_{n})$ $\displaystyle=\int
dx\Bigl{(}\frac{\partial}{\partial
J(x)}\Bigr{)}^{n}W[J;x]\prod_{I=1}^{n}\delta(x-x_{I}),$ $\displaystyle=\int
dx\,\kappa_{n}^{\phi}\prod_{I=1}^{n}\delta(x-x_{I}),$ (4.40)

with $n$ delta functions. The $n$-point connected correlator is nonzero only
when $x_{1}=x_{2}=\dots=x_{n}$, and its magnitude is determined by
$\kappa_{n}^{\phi}$.

The correlation functions can be written in terms of the connected
correlators. For example, the two point function of $\phi$ is

$\displaystyle G^{(2)}(x_{1},x_{2})$
$\displaystyle=\mathbb{E}_{\phi}[\phi(x_{1})\phi(x_{2})],$
$\displaystyle=\kappa_{2}^{\phi}\,\delta(x_{1}-x_{2})+(\kappa_{1}^{\phi})^{2}$
$\displaystyle=\kappa_{2}^{\phi}\,\delta(x_{1}-x_{2}).$ (4.41)

As $\phi(x_{1})$ and $\phi(x_{2})$ are independent and mean free,
$G^{(2)}_{\phi_{\text{SI}}}(x_{1},x_{2})$ is nonzero only when $x_{1}=x_{2}$.
Similarly, the four point function is

$\displaystyle G^{(4)}(x_{1},x_{2},x_{3},x_{4})$
$\displaystyle=\kappa_{4}^{\phi}\,\delta(x_{1}-x_{2})\delta(x_{1}-x_{3})\delta(x_{1}-x_{4})+(\kappa_{2}^{\phi})^{2}\Big{(}\delta(x_{1}-x_{2})\delta(x_{3}-x_{4})$
$\displaystyle+\delta(x_{1}-x_{3})\delta(x_{2}-x_{4})+\delta(x_{1}-x_{4})\delta(x_{2}-x_{3})\Big{)}.$
(4.42)

The statistics of the theory is completely determined by the space
independence assumption and the cumulants $\kappa_{r}^{\phi}$. The general
$n$-point function can be expressed as

$\displaystyle G^{(n)}(x_{1},\ldots,x_{n})=\sum_{\alpha\in
S_{n}}\prod_{r\in\alpha}G^{(r)}_{\phi_{\text{SI}},c}(x_{1},\ldots,x_{n}),$
(4.43)

where $S_{n}$ denotes partitions of the set $\\{1,\ldots,n\\}$.

### Space-Time Independence Breaking

Clearly we don’t want to stop with space independent theories. We will now
introduce correlations between different space points to ‘stitch together’ the
$L$ 0-dimensional theories (associated to the space independent fields) into a
$d$-dimensional field theory. This requires modifying the theory in some way
so that there are non-trivial correlations between field values at different
points.

One way to do so is to define new field variables $\Phi(x_{I})$ as a function
of the SI fields $\phi(x_{I})$,

$\Phi(x_{I})=\Phi\Big{(}\phi(x_{1}),\ldots,\phi(x_{L})\Big{)}.$ (4.44)

As the value of $\Phi$ at site $x_{I}$ in principle depends on the values of
$\phi$ at all space points, $\Phi(x_{I})$ and $\Phi(x_{J})$ are correlated in
general, even when $I\neq J$. The statistics of $\Phi(x_{I})$ are then
determined by the functional form of (4.44), as well as the statistics of
$\phi(x_{I})$. However, such a general formulation (4.44) is unwieldy, and we
therefore simplify the picture.

We will describe a family of architectures where $\Phi(x_{I})$ is constructed
by a simpler ansätz, a smearing of $\phi(a)_{a\in\\{x_{1},\cdots,x_{L}\\}}$
across all space points, and write down a necessary and sufficient condition
to satisfy cluster decomposition. Consider the architecture,

$\Phi(x_{I})=\int_{-\infty}^{\infty}da\,f(x_{I}-a)\,\phi(a)$ (4.45)

for some continuous and differentiable function $f(x_{I}-a)$. First, note that
although a generic draw of $\phi(a)$ is discontinuous due to independence
across different points in space, $\Phi(x_{I})$ is rendered continuous by the
smearing. Furthermore, if the function $f$ is nonzero everywhere,
$\Phi(x_{I})$ will have correlations between all pairs of lattice sites.

We wish to check whether cluster decomposition is satisfied, and therefore
need to compute correlation functions of $\Phi(x)$. The $\Phi$-correlators are
given by

$\displaystyle G^{(n)}_{\Phi}(x_{1},\ldots,x_{n})$
$\displaystyle=\mathbb{E}_{\phi}\big{[}\Phi(x_{1})\cdots\Phi(x_{n})\big{]},$
$\displaystyle=\mathbb{E}_{\phi}\Big{[}\prod_{i=1}^{n}\int
da_{i}\,f(x_{i}-a_{i})\,\phi(a_{i})\Big{]}.$ (4.46)

As $f$ does not depend on $\phi$, we can carry out the expectation value over
$\phi$ to obtain

$\displaystyle G^{(n)}_{\Phi}(x_{1},\ldots,x_{n})$
$\displaystyle=\int\prod_{i=1}^{n}da_{i}\,f(x_{i}-a_{i})\,G^{(n)}_{\phi}(a_{1},\ldots,a_{n}),$
(4.47)

where $G^{(n)}_{\phi}(a_{1},\ldots,a_{n})$ is the $n$-point correlation
function of $\phi$. The only contribution to the connected correlator of
$\Phi(x)$ comes from the connected piece of
$G^{(n)}_{\phi}(a_{1},\ldots,a_{n})$ with $n$ delta functions666The remaining
terms factorize and do not contribute to the connected correlator.,

$G^{(n)}_{c}(x_{1},\ldots,x_{n})=\kappa_{n}^{\phi}\int
dx\prod_{i=1}^{n}da_{i}\,f(x_{i}-a_{i})\,\delta(x-a_{i}).$ (4.48)

Evaluating the integral, we obtain

$G^{(n)}_{c}(x_{1},\ldots,x_{n})=\kappa_{n}^{\phi}\int
dx\,\prod_{i}^{n}f(x_{i}-x).$ (4.49)

Cluster decomposition is satisfied if and only if (4.49) asymptotes to zero in
the limit where the separation between any two of the space points
$x_{I},x_{J}$ is taken to $\infty$. Any smearing function $f(x)$ that decays
faster than $1/x$ asymptotically satisfies this condition.777Note that the SI
theory automatically satisfies cluster decomposition as the connected
correlator, c.f. (4.40), vanishes unless all space points coincide.

#### Example: Gaussian Smearing

We now present an example with a particular choice of the smearing function
$f$ and show that the resulting theory satisfies cluster decomposition. Let

$\displaystyle f(x)$ $\displaystyle=e^{-\frac{x^{2}}{\beta}},$ (4.50)
$\displaystyle\Phi(x)$ $\displaystyle=\int
da\,e^{-\frac{(x-a)^{2}}{\beta}}\phi(a),$ (4.51)

for some $\beta>0$. As before, we will consider a case where $\phi(x)$ at
different space points are identically distributed, with cumulants
$\kappa_{\phi}^{n}$.888As $\phi(x)$ are identically distributed, the cumulants
do not depend on the space coordinates $x$. Following equation (4.49), the
cumulants of $\Phi(x)$ are then given by

$\displaystyle G^{(n)}_{c}(x_{1},\ldots,x_{n})$
$\displaystyle=\kappa_{n}^{\phi}\int
dx\,\prod_{i=1}^{n}e^{-\frac{(x_{i}-x)^{2}}{\beta}},$
$\displaystyle=\kappa_{n}^{\phi}\sqrt{\frac{\pi\beta}{n}}\exp\Bigl{[}M_{ij}x_{i}x_{j}/\beta\Bigr{]},$
(4.52)

where

$M_{ij}=\begin{cases}\frac{2}{n}-2,&\text{if }i=j\\\
\frac{2}{n},&\text{otherwise}\end{cases}$ (4.53)

This matrix is negative semidefinite, with eigenvalues
$\lambda_{1}=\cdots=\lambda_{n-1}=-\beta/2$, $\lambda_{n}=0$, and the
eigenvector corresponding to $\lambda_{n}$ is $(1,\cdots,1)$. Consequently,
the cumulant $G^{(n)}_{c}(x_{1},\ldots,x_{n})$ vanishes when any of the
$x_{i}$ are taken to be large, unless they coincide $x_{1}=\cdots=x_{n}$. This
theory thus satisfies cluster decomposition.

The dependence of the connected correlators (4) on the space coordinates
$x_{i}$ is completely determined by the choice of smearing function $f$, while
their magnitudes depend both on $f$ as well as the cumulants
$\kappa_{\phi}^{n}$. Although our main motivation here has been to engineer
neural network architectures that satisfy cluster decomposition, smearing
layers offer great flexibility in manipulating the connected correlators and
might be useful in designing neural networks with other desired properties.

##  5 Conclusions

In this paper we continued the development of neural network field theory (NN-
FT), a new approach to field theory in which a theory is specified by a neural
network architecture and a parameter density. This description enables a
parameter space description of the statistics, yielding a different method for
computing correlation functions. For a more detailed introduction to NN-FT,
see the introduction and references therein.

We focused on three foundational aspects of NN-FT: non-Gaussianity, actions,
and locality. Via the central limit theorem (CLT), many architectures admit an
$N\to\infty$ limit in which the associated NN-FT is Gaussian, i.e., a
generalized free field theory. In the machine learning literature, these are
called neural network Gaussian processes (NNGPs). In Section 2 we demonstrated
that interactions arise from parametrically violating assumptions of the CLT,
yielding non-Gaussianities arising from $1/N$-corrections, as well as the
breaking of statistical independence and the identicalness assumption. These
interactions are apparent via parameter-space calculations of connected
correlation functions, but manifest themselves as non-Gaussianities in the
field density $P[\phi]=\exp(-S[\phi])$. In Section 3 we developed a technique
that allows for the action to be computed from the connected correlation
functions, via connected Feynman diagrams. This is an inversion of the usual
approach in field theory: we compute coupling functions in terms of connected
correlators, rather than the other way around. The technique was applied to
NN-FT, including an analysis involving the parametric non-Gaussianities we
studied. In Section 4 we studied how to design architectures that realize a
given action. We do so by deforming an NNGP by an operator insertion that,
from a function-space perspective, corresponds to a deformation of the GP
action. However, since we know the architecture we may also express the
deformation in parameter space, in which case the non-Gaussianity associated
to a given deformation of the action has a natural interpretation as a
deformation of the neural network parameter density. That is, the interactions
arise from independence breaking. We apply this technique to induce local
interactions, and derive an architecture that realizes $\phi^{4}$ theory as an
infinite NN-FT.

Acknowledgements. We thank Sergei Gukov, Mathis Gerdes, Jessica Howard, Ro
Jefferson, Gowri Kurup, Joydeep Naskar, Fabian Ruehle, Jiahua Tian, Jacob
Zavatone-Veth, and Kevin Zhang for discussions. This work is supported by the
National Science Foundation under Cooperative Agreement PHY-2019786 (The NSF
AI Institute for Artificial Intelligence and Fundamental Interactions). This
work was performed in part at Aspen Center for Physics, which is supported by
National Science Foundation grant PHY-2210452. A.M. thanks ECT* and the
ExtreMe Matter Institute EMMI at GSI, Darmstadt, for support in the framework
of an ECT* Workshop during which part of this work has been completed. J.H. is
supported by NSF CAREER grant PHY-1848089.

## Appendix A. Continuum Hermite Polynomials

Let us first recall the definition of continuum Hermite polynomials for
convenience,

$H(\phi,x_{1},\cdots,x_{n})=(-1)^{n}e^{S_{G}}\frac{\delta}{\delta\phi(x_{1})}\cdots\frac{\delta}{\delta\phi(x_{n})}e^{-S_{G}}.$
(A.1)

Defining

$S_{i}=\frac{\delta S_{G}}{\delta\phi(x_{i})},\qquad
S_{i,j}=\frac{\delta^{2}S_{G}}{\delta\phi(x_{i})\delta\phi(x_{j})},$ (A.2)

the first six Hermite polynomials are,

$\displaystyle H_{1}(\phi,x_{1})=S_{1},$ $\displaystyle
H_{2}(\phi,x_{1},x_{2})=S_{1}S_{2}-S_{1,2},$ $\displaystyle
H_{3}(\phi,x_{1},x_{2},x_{3})=S_{1}S_{2}S_{3}-S_{1,2}S_{3}[3],$ $\displaystyle
H_{4}(\phi,x_{1},x_{2},x_{3},x_{4})=S_{1}S_{2}S_{3}S_{4}-S_{1,2}S_{3}S_{4}[6]+S_{1,2}S_{3,4}[3],$
$\displaystyle
H_{5}(\phi,x_{1},x_{2},x_{3},x_{4},x_{5})=S_{1}S_{2}S_{3}S_{4}S_{5}-S_{1,5}S_{2}S_{3}S_{4}[10]+S_{1,2}S_{3,4}S_{5}[15],$
$\displaystyle
H_{6}(\phi,x_{1},x_{2},x_{3},x_{4},x_{5},x_{6})=S_{1}S_{2}S_{3}S_{4}S_{5}S_{6}-S_{1,6}S_{2}S_{3}S_{4}S_{5}[15]$
$\displaystyle\,\qquad\qquad\qquad\qquad+S_{1,2}S_{3,4}S_{5}S_{6}[45]-S_{1,2}S_{3,4}S_{5,6}[15],$
(A.3)

where the square brackets denote sums over all terms with a given index
structure, for example