# Population Level Activity in Large Random Neural Networks

James MacLaurin, Moshe Silverstein, Pedro Vilanova

###### Abstract

We determine limiting equations for large asymmetric ‘spin glass’ networks.
The initial conditions are not assumed to be independent of the disordered
connectivity: one of the main motivations for this is that allows one to
understand how the structure of the limiting equations depends on the energy
landscape of the random connectivity. The method is to determine the
convergence of the double empirical measure (this yields population density
equations for the joint distribution of the spins and fields). An additional
advantage to utilizing the double empirical measure is that it yields a means
of obtaining accurate finite-dimensional approximations to the dynamics.

##  1 Introduction

This paper concerns the high-dimensional dynamics of asymmetric random neural
networks of the form, for $j\in I_{N}=\\{1,2,\ldots,N\\}$.

$dx^{j}_{t}=\big{(}-x^{j}_{t}/\tau+\beta
N^{-1/2}\sum_{k=1}^{N}J^{jk}\lambda(x^{k}_{t})\big{)}dt+\sigma_{t}dW^{j}_{t},$
(1)

where $\lambda$ is a Lipschitz function, $\tau$ is a constant, and
$\\{J^{jk}\\}_{j,k\in I_{N}}$ are sampled independently from a centered normal
distribution of variance $1$, $\\{W^{j}_{t}\\}_{j\in I_{N}}$ are Brownian
Motions. We study the convergence of the double empirical measure

$\displaystyle N^{-1}\sum_{j\in
I_{N}}\delta_{(\mathbf{z}^{j}_{[0,T]},\mathbf{G}^{j}_{[0,T]})},$ (2)

where $G^{j}_{t}=N^{-1/2}\sum_{k=1}^{N}J^{jk}\lambda(x^{k}_{t})\big{)}$. The
dynamics of high-dimensional recurrent neural networks have many applications.
They have been heavily applied to neuroscientific problems: many scholars
think that they can be used to explain how the brain balances excitation and
inhibition [7, 32, 19, 28, 12]. They have been used to study spatially-
extended patterns in the brain [33, 34]. Most recently, it has been recognized
that they are of fundamental importance to data science [6, 2, 22, 35]. For
more applications, see the mongraph of Helias and Dahmen [26] and the recent
survey in [13].

There exist limiting ‘correlation equations’ [14, 26] that describe the
effective dynamics of high dimensional random neural networks. These
constitute delayed integro-differential equations that have proven very
difficult to analyze, particularly over short timescales. A related problem is
that the correlation equations have only been determined from initial
conditions that are independent of the connectivity. This means that they may
not be accurate over longer timescales that diverge with $N$. For example,
many scholars are interested in understanding the nature of the limiting
dynamics after the system attains a particular state (such as, if it enters an
‘energy well’ of specified characteristics, does it escape?). To address this
question, one needs to start the dynamics at a particular point in the energy
landscape of the connectivity (and therefore the initial condition is
disorder-dependent).

The literature concerning large $N$ limiting equations for random neural
networks has a complex history. Sompolinsky, Crisanti and Sommers anticipated
that Path Integral methods would yield limiting dynamical equations [36] \-
the derivation was published in a later work [14]. We refer the reader to the
excellent discussion in the monograph of Helias and Dahmen [26].

Path Integral methods (as practiced by physicists) yield population density
equations by determining where the probability measure for the $N$-dimensional
system concentrates. In the probability literature, one of the most powerful
means of addressing this question is the theory of Large Deviations [18].
Large Deviations theory was used to determine spin glass dynamics in the
pioneering papers of Ben Arous and Guionnet [3, 5, 24]; they obtained the
first rigorous results concerning the large $N$ limit of random neural
networks. After this work, Grunwald employed Large Deviations theory to obtain
correlation / response equations for random neural networks whose spins flip
randomly between discrete states [23]. Moynot and Samuelides studied the non-
Gaussian case [31]. Faugeras and MacLaurin extended the work of Ben Arous and
Guionnet to include correlations in the connectivity [20]. Touboul and Cabana
determined limiting equations for spatially-extended systems [10, 11].
Faugeras, Soret and Tanre [21] determined novel integral equations to describe
the state of these systems.

On a related note, correlation-response equations for symmetric random neural
networks were first derived by Crisanti, Horner Sommers [15] and Cugliandolo
and Kurchan [16]. Ben Arous, Dembo and Guionnet [4] proved the accuracy of
these correlation / response equations for symmetric random neural networks,
employing concentration inequalities.

Broadly-speaking, this paper follows the approach of Ben Arous and Guionnet
[3]. We employ the theory of Large Deviations to determine the large $N$ limit
of the empirical measure. However the main novelties of our approach are:

  * •

We employ a general class of connectivity-dependent initial conditions. This
unsurprisingly yields a different limiting dynamics as $N\to\infty$.
Connectivity-dependent initial conditions were employed in the papers of Ben
Arous and Guionnet [5] (who studied dynamics started at the equilibrium
distribution, in the high temperature regime) and Dembo and Subag [17].

  * •

We study the double empirical measure, that includes information about both
the spins and the fields. This has several advantages: it facilitates finite-
dimensional approximations to the dynamics that are very accurate, and it
facilitates a broader class of disorder-dependent initial condition. For spin-
glass dynamics, the Large Deviations of the double empirical measure was
determined by Grunwald [23] for jump-Markov systems.

  * •

We include Replicas (i.e. $M$ copies of the system with the same connectivity,
but independent Brownian Motions). This broadens the class of admissible
disorder-dependent initial conditions.

  * •

The function $\lambda$ can be unbounded and the diffusion coefficient
$\sigma_{t}$ can vary in time. The time-varying nature of $\sigma_{t}$ is
essential for studying how periodic environmental noise in the brain shapes
the dynamics of random neural networks.

Notation: Let $I_{N}=\\{1,2,\ldots,N\\}$ be the set of neuron indices. For any
Polish space $\mathcal{X}$, let $\mathcal{P}(\mathcal{X})$ denote all
probability measures over $\mathcal{X}$. The space
$\mathcal{C}([0,T],\mathbb{R})$ is always endowed with the supremum topology
(unless indicated otherwise), i.e.

$\left\|x_{[0,T]}\right\|=\sup_{t\in[0,T]}|x_{t}|$

For $\mathbf{y}\in\mathbb{R}^{N}$, $\left\|\mathbf{y}\right\|$ is the
Euclidean norm. For any probability measures $\mu$ and $\nu$ over a Polish
Space, let $\mathcal{R}(\mu||\nu)$ denotes the relative entropy of measure
$\mu$ with respect to $\nu$. For any two measures on the same metric space
with metric $d$, $d_{W}(\cdot,\cdot)$ indicates the Wasserstein distance, i.e.

$d_{W}(\mu,\nu)=\inf_{\zeta}\mathbb{E}^{\zeta}\big{[}d(x,y)\big{]},$ (3)

where the infimum is taken over all $\zeta$ on the product space such that the
marginal of the first variable is equal to $\mu$ and the marginal of the
second variable is equal to $\nu$. In the particular case that
$\mu,\nu\in\mathcal{C}([0,T],\mathbb{R}^{M})$, the distance is (unless
otherwise indicated) $d(x,y)=\sup_{t\in[0,T]}\sup_{p\in
I_{M}}\big{|}x^{p}_{t}-y^{p}_{t}\big{|}$.

For any
$\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$, we
write
$\mu^{(1)},\mu^{(2)}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$
to be the marginals over (respectively) the first $M$ variables and last $M$
variables.

##  2 Outline of Model and Main Results

We are going to rigorously determine the limiting dynamics of multiple
replicas (with identical connections $\mathbf{J}$, but with independent
initial conditions and independent Brownian Motions). We let the superscript
$a$ denote replica $a\in I_{M}=\\{1,2,\ldots,M\\}$, and consider the system

$\displaystyle dz^{a,j}_{t}=$
$\displaystyle\big{(}-z^{a,j}_{t}/\tau+G^{a,j}_{t}\big{)}dt+\sigma_{t}dW^{a,j}_{t}\text{
where }$ (4) $\displaystyle G^{a,j}_{t}=$ $\displaystyle N^{-1/2}\sum_{k\in
I_{N}}J^{jk}\lambda(z^{a,k}_{t}).$ (5)

We assume that $\lambda\in\mathcal{C}^{2}(\mathbb{R})$: this means in
particular that there is a constant $C_{\lambda}$ such that
$|\lambda(x)-\lambda(y)|\leq C_{\lambda}|x-y|$. The noise intensity
$t\to\sigma_{t}$ is taken to be continuous and non-random, and such that for
constants $\underline{\sigma}$ and $\bar{\sigma}$,

$0<\underline{\sigma}\leq\sigma_{t}\leq\bar{\sigma}.$ (6)

Our major motivation for time-varying diffusivity lies in neuroscience: often
synaptic noise exhibits particular rhythms. It has been of major interest how
these rhythms shape pattern formation [8].

The connectivities $\\{J^{jk}\\}$ are taken to be independent centered
Gaussian variables, with variance

$\mathbb{E}\big{[}J^{jk}J^{lm}\big{]}=\delta(j,l)\delta(k,m).$

Let $\gamma^{N}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\big{)}$ be their joint
probability law. There are two cases for the initial conditions
$\\{z^{a,j}_{0}\\}_{j\in I_{N},a\in I_{M}}$ that are considered in this paper.

###  2.1 Assumptions on the Initial Conditions

####  2.1.1 Case 1: Connectivity-Dependent Initial Conditions

The probability law of the initial conditions is assumed to be such that for
any measurable set $\mathcal{A}\subset\mathbb{R}^{MN}$,

$\displaystyle\mathbb{P}\big{(}\mathbf{Z}_{0}\in\mathcal{A}\big{)}=\int_{\mathcal{A}}\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x},$
(7)

where the probability density
$\rho^{N}_{\mathbf{J}}:\mathbb{R}^{MN}\to\mathbb{R}^{+}$ is defined as
follows.

Let $\mu^{N}$ be the uniform Lebesgue Measure over the set $\mathbb{R}^{MN}$.
Conditionally on a realization $\mathbf{J}$ of the random connections, let the
probability density $\rho^{N}_{\mathbf{J}}:\mathbb{R}^{MN}\to\mathbb{R}^{+}$
be such that for $\delta_{N}>0$ (with $\delta_{1}=1$ and $\delta_{N}$
decreasing to $0$ as $N\to\infty$), there exists some
$\kappa\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}$ such that

$\displaystyle\rho^{N}_{\mathbf{J}}(\mathbf{g})=$
$\displaystyle\chi\big{\\{}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0}),\kappa\big{)}\leq\delta_{N}\big{\\}}/Z^{N}_{\mathbf{J}}\text{
where }$ (8) $\displaystyle Z^{N}_{\mathbf{J}}=$
$\displaystyle\mu^{N}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0}),\kappa\big{)}\leq\delta_{N}\big{)}\text{
and }$ (9) $\displaystyle\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})=$
$\displaystyle N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{z}^{j}_{0},\mathbf{G}^{j}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}$
(10)

Roughly speaking, we need to assume that as $N\to\infty$, the law of
$\mathbf{z}^{N}_{0}$ behaves like its annealed average. Its assumed that (i)
$\kappa$ has a finite second moment in each of its variables, and (ii) we have
the bound

$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log\mathbb{E}[Z^{N}_{\mathbf{J}}]>-\infty,$
(11)

It is also assumed that for any $\epsilon>0$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\big{|}N^{-1}\log
Z^{N}_{\mathbf{J}}-\mathbb{E}[Z^{N}_{\mathbf{J}}]\big{|}\geq\epsilon\big{)}<0.$
(12)

Define $\mathfrak{V}_{0},\tilde{\mathfrak{V}}_{0}\in\mathbb{R}^{M\times M}$ to
be the covariance matrix with entries, $p,q\in I_{M}$,

$\displaystyle\mathfrak{V}_{0}^{pq}$
$\displaystyle=\mathbb{E}^{\kappa}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}$
(13) $\displaystyle\tilde{\mathfrak{V}}_{0}^{pq}$
$\displaystyle=\mathbb{E}^{\kappa}\big{[}z^{p}_{0}z^{q}_{0}\big{]}.$ (14)

It is also assumed that both $\mathfrak{V}_{0}$ and $\tilde{\mathfrak{V}}_{0}$
are invertible.

####  2.1.2 Case 2: Connectivity-Independent Initial Conditions

One can also assume that the initial conditions $(z^{j}_{0})_{j\in I_{N}}$ are
(i) independent of the connectivity, and (ii) sampled independently from a
$\mathbb{R}^{M}$-valued probabilistic distribution of bounded variance. This
distribution is written as $\hat{\kappa}\in\mathcal{P}(\mathbb{R}^{M})$.

###  2.2 Main Result

Our main result is that the empirical measure converges to a fixed point of a
mapping $\Phi:\mathcal{U}\to\mathcal{U}$. Here
$\mathcal{U}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$
is defined in (54), consisting of (i) a broad class of measures with nice
regularity properties, and (ii) such that the empirical measure inhabits
$\mathcal{U}$ with unit probability.

For any $\mu\in\mathcal{U}$, in the case of connectivity-dependent initial
conditions $\Phi(\mu)$ is specified as follows. It is defined to be the law of
Gaussian random variables $\big{(}z^{p}_{t},G^{p}_{t}\big{)}_{p\in
I_{M}\fatsemi t\in[0,T]}$ such that (i) $(z^{p}_{0},G^{p}_{0})_{p\in I_{M}}$
are distributed according to $\kappa$, and (ii) conditionally on the initial
conditions, $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ is a
Gaussian system such that
$\mathbb{E}[G^{p}_{s}]=\mathfrak{m}^{p}_{s}(\mu,\mathbf{G}_{0})$ and the
conditional variance is

$\displaystyle\mathbb{E}\big{[}\big{(}G^{p}_{s}-\mathfrak{m}^{p}_{s}(\mu,\mathbf{G}_{0})\big{)}\big{(}G^{q}_{t}-\mathfrak{m}^{q}_{t}(\mu,\mathbf{G}_{0})\big{)}\;|\;\mathbf{G}_{0},\mathbf{z}_{0}\big{]}=\mathfrak{W}^{\mu,pq}_{st}.$
(15)

Here

$\displaystyle\mathfrak{W}^{\mu,pq}_{st}$ $\displaystyle=\sum_{a,b\in
I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0})\big{]}\mathbb{E}^{\mu}\big{[}\lambda(z^{q}_{t})\lambda(z^{b}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}$
(16) $\displaystyle\mathfrak{V}_{\mu,0}^{pq}$
$\displaystyle=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}$
(17) $\displaystyle\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})$
$\displaystyle=\sum_{a,b\in
I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}g^{b}$
(18)

Letting $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ be Brownian Motions that
are independent of $\mathbf{G}^{\mu}$ , we define $(z^{p}_{t})_{p\in
I_{M}\fatsemi t\in[0,T]}$ to be the strong solution to the stochastic
differential equation

$\displaystyle
dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.$
(19)

In the case of connectivity-independent initial conditions, $\Phi$ is defined
as follows. One first defines $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi
t\in[0,T]}$ to be a centered Gaussian system such that

$\mathbb{E}\big{[}G^{p}_{t}G^{q}_{s}\big{]}=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{t})\lambda(z^{q}_{s})\big{]}.$

$(z^{p}_{0})_{p\in I_{M}}$ is independent of $\big{(}G^{p}_{t}\big{)}_{p\in
I_{M}\fatsemi t\in[0,T]}$ and distributed according to $\hat{\kappa}$. For
Brownian Motions $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$, that are
independent of $\mathbf{G}^{\mu}$, $z^{p}_{t}$ is the strong solution of (19).

######  Theorem 1.

The mapping $\Phi$ is well-defined for all $\mu\in\mathcal{U}$. Furthermore
there exists a unique probability measure
$\xi\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ such
that with unit probability,

$\lim_{N\to\infty}N^{-1}\sum_{j\in
I_{N}}\delta_{(\mathbf{z}^{j}_{[0,T]},\mathbf{G}^{j}_{[0,T]})}=\xi.$ (20)

$\xi$ is the unique measure such that $\Phi(\xi)=\xi$. Furthermore,

$\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},$ (21)

where $\xi^{(n+1)}=\Phi(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in
$\mathcal{U}$.

Remark. This theorem is useful because it also implies a means to efficiently
determine the large $N$ limiting equations, through repeated application of
the mapping $\Phi$. Because the limiting system is Gaussian, one only needs to
solve for its covariance matrix. See the discussion in Helias and Dahmen [26]
for an alternative formulation of the limiting covariance function in terms of
a PDE.

###  2.3 An Example System that Satisfies the Conditions of Section 2.1.1

We now outline a general system that satisfies the conditions of Section
2.1.1. Suppose that $\lambda:\mathbb{R}\to\mathbb{R}$ is an odd function.
Define
$\iota:\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\to\mathbb{R}^{+}$
to be

$\displaystyle\iota(\mu)=\mathcal{R}(\mu||p).$ (22)

Here $p\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}$ is
centered and Gaussian, the law of random variables
$(z^{1}_{0},g^{1}_{0},z^{2}_{0},g^{2}_{0})$ that are such that
$\mathbb{E}[(z^{p}_{0})^{2}]=1$, $\mathbb{E}[z^{p}_{0}g^{q}_{0}]=0$ and
$\mathbb{E}[g^{p}_{0}g^{q}_{0}]=\mathbb{E}[\lambda(z^{p}_{0})\lambda(z^{q}_{0})]$.
Let $\Xi:\mathcal{C}\big{(}\mathbb{R}^{2}\big{)}\to\mathbb{R}$ be bounded.

######  Lemma 2.

Suppose that there is a unique
$\eta\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}$ such that

$\displaystyle\sup_{\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}}\big{\\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})]-\iota(\mu)\big{\\}}=\mathbb{E}^{\eta}[\Xi(z^{1},g^{1})]-\iota(\eta).$
(23)

Suppose also that

$\sup\big{\\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]-\iota(\mu)\;:\;\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\text{
and }\mu^{(1)}=\eta^{(1)}\text{ and }\mu^{(2)}=\eta^{(1)}\big{\\}}\\\
=2\big{\\{}\mathbb{E}^{\eta}[\Xi(z^{1},g^{1})]-\iota(\eta)\big{\\}}.$ (24)

Then the conditions of Section 2.1.1 are satisfied, substituting $M=1$,
$\kappa=\eta^{(1)}$ and $\delta_{N}=(\log N)^{-1}$.

###### Proof.

We show that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\big{\\{}\log\mathbb{E}^{\gamma}\big{[}(Z^{N}_{\mathbf{J}})^{2}\big{]}-2\log\mathbb{E}^{\gamma}\big{[}Z^{N}_{\mathbf{J}}\big{]}\big{\\}}=0.$
(25)

It is immediate from (25) that (12) must be satisfied. Let
$\\{y^{p,j}_{0}\\}_{j\in I_{N}\fatsemi 1\leq p\leq 2}$ be iid
$\mathcal{N}(0,1)$ random variables. Form the empirical measure
$\hat{\mu}^{N}_{0}:=N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{y}^{j},\mathbf{G}^{j}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$.

Define

$\displaystyle\mathcal{A}_{N}=\chi\bigg{\\{}d_{W}\big{(}\eta^{(1)},N^{-1}\sum_{j\in
I_{N}}\delta_{y^{1,j},g^{1,j}}\big{)}\leq(\log
N)^{-1}\;,\;d_{W}\big{(}\eta^{(2)},N^{-1}\sum_{j\in
I_{N}}\delta_{y^{2,j},g^{2,j}}\big{)}\leq(\log N)^{-1}\bigg{\\}}.$

We will first demonstrate that

$\lim_{N\to\infty}N^{-1}\log\mathbb{E}\big{[}\mathcal{A}_{N}\exp\big{(}N\mathbb{E}^{\hat{\mu}^{N}}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]\big{)}\big{]}=\\\
\sup\big{\\{}\mathbb{E}^{\mu}[\Xi(z^{1},g^{1})+\Xi(z^{2},g^{2})]-\iota(\mu)\;:\;\mu\in\mathcal{P}\big{(}\mathbb{R}^{2}\times\mathbb{R}^{2}\big{)}\text{
and }\mu^{(1)}=\eta^{(1)}\text{ and }\mu^{(2)}=\eta^{(1)}\big{\\}}.$ (26)

It is straightforward to prove that for any $a>0$ there exists a compact
subset $\mathfrak{U}_{a}\subset\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$ such
that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}_{0}\notin\mathfrak{U}_{a}\big{)}\leq-a.$
(27)

A simple Large Deviations estimate yields that, for any
$\mu\in\mathcal{P}\big{(}\mathbb{R}^{4}\big{)}$, and $\epsilon\ll 1$,

$\displaystyle N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}_{0}\in
B_{\epsilon}(\mu)\big{)}=-\iota(\mu)+O(\sqrt{\epsilon}).$ (28)

(In fact this could also be proved using Corollary 11). Our choice that
$\delta_{N}$ goes to zero sufficiently slowly (i.e. $\delta_{N}=(\log
N)^{-1}$) ensures that the rate function in (28) dominates the asymptotic
estimate of the probability as $N\to\infty$. Thus discretizing
$\mathfrak{U}_{a}$ into $\epsilon$ balls, and then taking $\epsilon\to 0^{+}$,
one obtains (26).

Now (26) also implies that

$\displaystyle\lim_{N\to\infty}N^{-1}\log\mathbb{E}\big{[}\mathcal{A}_{N}\big{]}=-2\iota(\eta).$
(29)

This holds because since
$\mu\to\mathbb{E}^{\mu}[\Xi(y^{1},g^{1}+\Xi(y^{2},g^{2})]$ is continuous, it
becomes almost constant as $N\to\infty$, as the radius of $\mathcal{A}_{N}$
shrinks. (29) in turn implies (25). ∎

##  3 Proof Outline

The main goal of this paper is to prove Theorem 1 employing the theory of
Large Deviations [18]. The method - similarly to the original work by Ben
Arous and Guionnet [3] \- is to (i) prove a Large Deviations Principle for the
uncoupled system, and then (ii) perform an exponential change-of-measure using
Girsanov’s Theorem to obtain the Large Deviations Principle for the coupled
system, before (iii) proving that the rate function has a unique zero.

The three main differences between this paper and the early papers of Ben
Arous and Guionnet is that we (i) study the convergence of the double
empirical measure (2) (whereas Ben Arous and Guionnet study the convergence of
the annealed empirical measure in their earlier papers [3]) (in the later
works [25, 4] quenched asymptotics are determined) (ii) we employ disorder-
dependent initial conditions and (iii) we employ replicas.

Our main focus is on proving Case 1 (i.e. the connectivity-dependent initial
conditions). The proofs are broadly similar, however Case 1 is more difficult
because it requires a uniform Large Deviations Principle for the conditioned
probability laws.

###  3.1 Large Deviations of the Uncoupled System

We start by noting a Large Deviation Principle for the uncoupled system.
Because we are employing general disorder-dependent initial conditions, we
must determine a Large Deviations Principle for the conditioned probability
law. To this end, we must first define the set $\mathcal{Y}^{N}$ of all ‘valid
initial conditions’ (basically the set of all initial points such that the
empirical measure at time $0$ is close to its limit). More precisely, we
define $\mathcal{Y}^{N}\subset\mathbb{R}^{NM}\times\mathbb{R}^{NM}$ to be such
that

$\displaystyle\mathcal{Y}^{N}=\big{\\{}(\mathbf{z}_{0},\mathbf{g}_{0})\;:\;d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{g}_{0}),\kappa\big{)}\leq\tilde{\delta}_{N}\big{\\}},$
(30)

where $\tilde{\delta}_{N}=\max\big{\\{}\delta_{N},(\log N)^{-1}\big{\\}}$ and
$\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{g}_{0})=N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{z}^{j}_{0},\mathbf{g}^{j}_{0}}\in\mathcal{P}(\mathbb{R}^{2M})$.
Define the uncoupled dynamics,

$\displaystyle y^{p,j}_{t}=z^{p,j}_{0}+\int_{0}^{t}\sigma_{s}dW_{s}^{p,j},$
(31)

and let
$P^{N}_{\mathbf{z}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
be the law of $\\{y^{j}_{[0,T]}\\}_{j\in I_{N}}$, conditioned on the values at
time $0$. Write

$\displaystyle\tilde{G}^{p,j}_{t}=N^{-1/2}\sum_{k\in
I_{N}}J^{jk}\lambda(y^{p,k}_{t}).$ (32)

Define
$\gamma^{N}_{\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\big{)}$ to
be the law of the connections $\\{J^{jk}\\}_{j,k\in I_{N}}$, conditioned on
the event

$\displaystyle\big{\\{}\tilde{G}^{p,j}_{0}=g^{p,j}_{0}\text{ for each }j\in
I_{N},\;p\in I_{M}\big{\\}}.$ (33)

We note that $\gamma^{N}_{\mathbf{g}_{0}}$ is Gaussian, but no longer of zero
mean (in general). The mean of $\tilde{G}^{p,j}_{t}$ is a function of the
empirical measure and $\mathbf{g}^{j}_{0}$ (explicit formulae are outlined
further below). Let
$Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathbb{R}^{N^{2}}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
be the joint law of the uncoupled system, i.e.

$Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}=\gamma^{N}_{\mathbf{g}_{0}}\otimes
P^{N}_{\mathbf{z}_{0}}.$ (34)

The first main result is a uniform Large Deviations Principle for the
conditioned system.

######  Theorem 3.

Let
$\mathcal{A},\mathcal{O}\in\mathcal{B}\big{(}\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$,
such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{A}\big{)}$
$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{I}(\mu)$ (35)
$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{O}\big{)}$
$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{I}(\mu).$ (36)

Here the rate function
$\mathcal{I}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}^{+}$
is such that $\mathcal{I}(\mu)=\infty$ if $\mu\notin\mathcal{U}$, else
otherwise

$\displaystyle\mathcal{I}(\mu)=\tilde{I}_{\mu}(\mu),$ (37)

where, $\tilde{I}_{\mu}(\mu)$ is defined in (117). Furthermore $\mathcal{I}$
is lower-semi-continuous and has compact level sets.

We will prove Theorem 3 by locally freezing the dependence of the fields
$\\{\tilde{G}^{p,j}_{t}\\}$ on the empirical measure. In order that we may do
this, we must first define a regular subset $\mathcal{Q}_{\mathfrak{a}}$ (for
a positive integer $\mathfrak{a}\gg 1$) which is such that (i) the empirical
measure $\hat{\mu}^{N}(\mathbf{y})=N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$
inhabits with high probability, and (ii) there exist uniform bounds on the
fluctuations in time. To this end, writing $\mathcal{K}_{\mathfrak{a}}$ to be
the compact set specified in Lemma 24, define the set

$\mathcal{Q}_{\mathfrak{a}}=\bigg{\\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\mu\in\mathcal{K}_{\mathfrak{a}}\text{
and }\sup_{p\in
I_{M}}\mathbb{E}^{\mu}[\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq\mathfrak{a}\text{
and }\\\ \text{ For all integers }m\geq\mathfrak{a}\text{ it holds that
}\sup_{0\leq i\leq m}\mathbb{E}^{\mu}\big{[}\sup_{p\in
I_{M}}(w^{p}_{t^{(m)}_{i+1}}-w^{p}_{t^{(m)}_{i}})^{2}\big{]}\leq\Delta_{m}^{1/4}\bigg{\\}}$
(38)

where $\Delta_{m}=T/m$ and $t^{(m)}_{i}=iT/m$. Write

$\displaystyle\mathfrak{Q}=\bigcup_{\mathfrak{a}\geq
1}\mathfrak{Q}_{\mathfrak{a}}.$ (39)

######  Lemma 4.

For any $L>0$, there exists $a>0$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{a}\big{)}\leq-L$
(40)

The above lemma is proved in the Appendix. Next, for any $\nu\in\mathcal{Q}$,
we define a centered Gaussian law
$\beta_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$ as
follows. We stipulate that $\beta_{\nu}$ is the law of Gaussian random
variables $\\{G^{\nu,p}_{t}\\}_{t\in[0,T],p\in I_{M}}$ with covariance
structure

$\displaystyle\mathbb{E}^{\beta_{\nu}}\big{[}G^{\nu,p}_{s}G^{\nu,q}_{t}\big{]}=\mathbb{E}^{\nu}\big{[}\lambda(x^{p}_{s})\lambda(x^{q}_{t})\big{]}$
(41)

This definition will be useful because for any $j\in I_{N}$, the law of
$\tilde{G}^{j}_{[0,T]}$ under $\gamma^{N}$ is
$\beta_{\hat{\mu}^{N}(\mathbf{y})}$. In the following Lemma we collect some
regularity estimates for the Gaussian Law $\beta_{\nu}$.

######  Lemma 5.

(i) $\beta_{\nu}$ is a well-defined Gaussian probability law. (ii)
Furthermore, the map $t\to G^{\nu,p}_{t}$ is ‘uniformly continuous’ for all
measures in $\mathcal{U}_{a}$, in the following sense. For any $a>0$, and any
$\epsilon>0$, there exists $\delta(a,\epsilon)$ such that for all
$\nu\in\mathcal{U}_{a}$,

$\displaystyle\sup_{\nu\in\mathcal{U}_{a}}\sup_{p\in
I_{M}}\mathbb{E}^{\beta_{\nu}}\bigg{[}\sup_{s,t\in[0,T]\fatsemi|s-t|\leq\delta(a,\epsilon)}\big{|}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{|}\bigg{]}\leq\epsilon$
(42)

In order that we may make sense of the disorder-dependent initial condition,
we also require an understanding of the distribution of $\beta_{\nu}$,
conditioned on the value of $\tilde{G}_{0}$. To this end, for any
$\nu\in\mathcal{U}$ and any $\mathbf{g}\in\mathbb{R}^{M}$, let
$\beta_{\nu,\mathbf{g}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$
be the probability law $\beta_{\nu}$ conditioned on the event
$\mathbf{G}^{\nu}_{0}=\mathbf{g}$. Standard Gaussian identities [29] imply
that $\beta_{\nu,\mathbf{g}}$ is also Gaussian, with the following mean and
variance,

$\displaystyle\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\big{[}G^{\nu,p}_{s}\big{]}=$
$\displaystyle\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})$ (43)
$\displaystyle\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\big{[}\big{(}G^{\nu,p}_{s}-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})\big{)}\big{(}G^{\nu,q}_{t}-\mathfrak{m}^{q}_{t}(\nu,\mathbf{g}\big{)}\big{]}=$
$\displaystyle\mathfrak{W}^{\nu,pq}_{st},$ (44)

where

$\displaystyle\mathfrak{W}^{\mu,pq}_{st}$ $\displaystyle=\sum_{a,b\in
I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0})\big{]}\mathbb{E}^{\mu}\big{[}\lambda(z^{q}_{t})\lambda(z^{b}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}$
(45) $\displaystyle\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})$
$\displaystyle=\sum_{a,b\in
I_{M}}\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{s})\lambda(z^{a}_{0}\big{]}\big{(}\mathfrak{V}_{\mu,0}^{-1}\big{)}^{ab}g^{b}$
(46) $\displaystyle\mathfrak{V}_{\mu,0}^{pq}$
$\displaystyle=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{0})\lambda(z^{q}_{0})\big{]}.$
(47)

######  Corollary 6.

(i) For any $a>0$, and any $\epsilon>0$, there exists
$\tilde{\delta}(a,\epsilon)$ such that for all $\nu\in\mathcal{U}_{a}$, and
all $\mathbf{g}\in\mathbb{R}^{M}$,

$\displaystyle\sup_{\nu\in\mathcal{U}_{a}}\sup_{p\in
I_{M}}\mathbb{E}^{\beta_{\nu,\mathbf{g}}}\bigg{[}\sup_{s,t\in[0,T]\fatsemi|s-t|\leq\tilde{\delta}(a,\epsilon)}\big{|}G^{\nu,p}_{s}-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})-G^{\nu,p}_{t}+\mathfrak{m}^{p}_{t}(\nu,\mathbf{g})\big{|}\bigg{]}\leq\epsilon$
(48)

(ii) For any $\epsilon,a>0$, there exists a compact set
$\mathcal{C}_{\epsilon,a}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that
for all $\nu\in\mathcal{Q}_{a}$,

$\displaystyle\beta_{\nu}(\mathcal{C}_{\epsilon,a})\geq 1-\epsilon,$ (49)

and for all $\mathbf{g}_{0}\in\mathbb{R}^{M}$ such that
$\left\|\mathbf{g}_{0}\right\|\leq a$,

$\displaystyle\beta_{\nu,\mathbf{g}_{0}}(\mathcal{C}_{\epsilon,a})\geq
1-\epsilon.$ (50)

(iii) For $1\leq j\leq N$ and any $\mathbf{g}_{0}\in\mathbb{R}^{M}$, the law
of $\tilde{G}^{j}_{[0,T]}$ under $\gamma^{N}_{\mathbf{g}_{0}}$ is identical to
$\beta_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}$.

###  3.2 Exponential Tightness

To prove a Large Deviation Principle, one requires that the empirical measure
inhabits a compact set with arbitrarily high probability. For any
$\mathbf{y}\in\mathcal{C}([0,T],\mathbb{R}^{M})^{N}$, write
$\tilde{\gamma}^{N}_{\mathbf{y}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
to be the law of the random variables $(\tilde{G}^{p,j}_{t})_{j\in
I_{N}\fatsemi p\in I_{M}\fatsemi t\in[0,T]}$, and write
$\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}$ to be
$\tilde{\gamma}^{N}_{\mathbf{y}}$ conditioned on the event in (33).

The following lemmas are needed for this proof.

######  Lemma 7.

For any $L>0$, there exists a compact set
$\tilde{\mathcal{C}}_{L}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$
such that the following holds. For any $N\geq 1$, and any
$\\{\mathbf{y}^{j}_{[0,T]}\\}_{j\in I_{N}}$ such that
$\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$,

$\displaystyle
N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y}}\big{(}\hat{\mu}^{N}(\tilde{\mathbf{G}})\notin\tilde{\mathcal{C}}_{L}\big{)}\leq-L.$
(51)

Also, as long as $(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\tilde{\mathbf{G}})\notin\tilde{\mathcal{C}}_{L}\big{)}\leq-L.$
(52)

For
$\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$,
write $\mu^{(1)}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$
to be the marginal of $\mu$ over its first $M$ variables, and $\mu^{(2)}$ to
be the marginal of $\mu$ over its last $M$ variables. Next, define the set

$\mathcal{U}_{a}=\bigg{\\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}\;:\;\mu^{(1)}\in\mathcal{Q}_{a},\;\mu^{(2)}\in\tilde{\mathcal{C}}_{a}\text{
and }\\\ \sup_{t\in[0,T]}\sup_{p\in
I_{M}}\mathbb{E}^{\mu}[(G^{p}_{t})^{2}]\leq C_{\lambda}^{2}a,\;\text{for all
}0\leq s,t\leq T,\;\sup_{p\in
I_{M}}\mathbb{E}^{\mu}[(G^{p}_{t}-G^{p}_{s})^{2}]\leq
aC_{\lambda}^{2}|t-s|^{1/2}\bigg{\\}},$ (53)

and let

$\displaystyle\mathcal{U}=\bigcup_{a\geq 0}\mathcal{U}_{a}.$ (54)

It follows immediately from the above definition that $d_{W}(\mu,\nu)<\infty$
for any $\mu,\nu\in\mathcal{U}$. We can now prove an ‘exponential tightness’
result.

######  Lemma 8.

For any $a\geq 0$, $\mathcal{U}_{a}$ is compact. For any $L>0$, there exists
$a>0$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\notin\mathcal{U}_{a}\big{)}\leq-L.$
(55)

###### Proof.

Since the sets $\mathcal{Q}_{a}$ and $\tilde{\mathcal{C}}_{a}$ are compact,
this follows almost immediately from Lemmas 4 and 7. ∎

###  3.3 The Coupled System (with connectivity-dependent initial conditions)

Define
$\mathcal{J}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}$
to be such that $\mathcal{J}(\mu)=\infty$ if $\mu\notin\mathcal{U}$, or if the
marginal of $\mu$ at time $0$ is not $\kappa$. Else otherwise, for any
$\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ and
$\mathbf{z}_{0},\mathbf{g}_{0}\in\mathbb{R}^{M}$, writing
$\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}$ for $\mu$ conditioned on the values of
its variables at time $0$, define

$\displaystyle\mathcal{J}(\mu)=\mathbb{E}^{\kappa}\bigg{[}\mathcal{R}\big{(}\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}||S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{)}\bigg{]}$
(56)

Here
$S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$
is defined to be the probability law of $(\mathbf{z},\mathbf{G}^{\mu})$, where
$\mathbf{G}^{\mu}$ is distributed according to $\beta_{\mu,\mathbf{g}_{0}}$,
and for Brownian Motions $\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ that are
independent of $\mathbf{G}^{\nu}$

$\displaystyle
dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}$
(57)

Define $\Phi:\mathcal{U}\to\mathcal{U}$ to be the following map. For some
$\mu\in\mathcal{U}$, write $\Phi(\mu)$ to be the law of the following random
variables $(\mathbf{x},\mathbf{h})$. First, it is stipulated that
$(\mathbf{x}_{0},\mathbf{h}_{0})$ have probability law $\kappa$. Second,
conditionally on $(\mathbf{x}_{0},\mathbf{h}_{0})$, the distribution of
$(\mathbf{x}_{[0,T]},\mathbf{h}_{[0,T]})$ is given by
$S_{\mu,\mathbf{x}_{0},\mathbf{g}_{0}}$.

######  Lemma 9.

The probability law $S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}$ is well-defined.
Furthermore, there exists a unique zero $\xi$ of the rate function
$\mathcal{J}$. $\xi$ is the unique measure in $\mathcal{U}$ such that
$\Phi(\xi)=\xi$.

###### Proof.

We have already proved in Lemma 5 that $\big{(}G^{\mu,p}_{[0,T]}\big{)}_{p\in
I_{M}}$ is well-defined. It is straightforward to check that for any path
$\big{(}G^{\mu,p}_{[0,T]}\big{)}_{p\in I_{M}}$, there exists a unique strong
solution to the stochastic differential equation (57). Thus the probability
law is well-defined.

It is well-known that the Relative Entropy is zero if and only if its two
arguments are identical [9]. Thus, from the form of $\mathcal{J}$ in (56), any
zero must be a fixed point of $\Phi$. Furthermore, there must exist at least
one zero of the rate function (if not, the total probability mass could not be
one as $N\to\infty$). The uniqueness of the zero is proved in Lemma 22. ∎

######  Theorem 10.

For any $\epsilon>0$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\xi)\geq\epsilon\big{)}<0.$
(58)

Thus with unit probability,

$\lim_{N\to\infty}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})=\xi.$ (59)

Furthermore,

$\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},$ (60)

where $\xi^{(n+1)}=\Phi(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in
$\mathcal{U}$.

###  3.4 Connectivity-Independent Initial Conditions

The above reasoning can be adapted to prove a Large Deviation Principle for
the unconditioned system. This is needed for proving the main theorem for Case
2 (connectivity-independent initial conditions). Write
$Q^{N}=\gamma^{N}\otimes P^{N}$ to be the law of the random variables
$(\mathbf{y},\mathbf{G})$ (with no conditioning), and for any
$\nu\in\mathfrak{Q}$, define
$S_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$ to
be $S_{\nu}=P\otimes\beta_{\nu}$. In the following corollary to Theorem 3, we
prove the Large Deviation Principle for the unconditioned and uncoupled
system.

######  Corollary 11.

Let
$\mathcal{A},\mathcal{O}\in\mathcal{B}\big{(}\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$,
such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log
Q^{N}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{A}\big{)}$
$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{R}\big{(}\mu||S_{\mu^{(1)}}\big{)}$
(61) $\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log
Q^{N}\big{(}\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})\in\mathcal{O}\big{)}$
$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{R}\big{(}\mu||S_{\mu^{(1)}}\big{)}.$
(62)

Here the rate function $\mu\to\mathcal{R}\big{(}\mu||S_{\mu^{(1)}}\big{)}$ is
lower semi-continuous and has compact level sets.

We now specify the operator $\tilde{\Phi}:\mathcal{U}\to\mathcal{U}$. Fix
$\mu\in\mathcal{U}$ and defined $\tilde{\Phi}(\mu)$ to be the law of processes
$\big{(}z^{p}_{[0,T]},G^{p}_{[0,T]}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$.
One first defines $\big{(}G^{p}_{t}\big{)}_{p\in I_{M}\fatsemi t\in[0,T]}$ to
be centered Gaussian system such that

$\mathbb{E}\big{[}G^{p}_{t}G^{q}_{s}\big{]}=\mathbb{E}^{\mu}\big{[}\lambda(z^{p}_{t})\lambda(z^{q}_{s})\big{]}.$

$(z^{p}_{0})_{p\in I_{M}}$ is independent of $\big{(}G^{p}_{t}\big{)}_{p\in
I_{M}\fatsemi t\in[0,T]}$ and distributed according to $\hat{\kappa}$. Letting
$\big{(}W^{p}_{[0,T]}\big{)}_{p\in I_{M}}$ be Brownian Motions that are
independent of $\mathbf{G}^{\mu}$ , we define $(z^{p}_{t})_{p\in I_{M}\fatsemi
t\in[0,T]}$ to be the strong solution to the stochastic differential equation

$\displaystyle
dz^{p}_{t}=\big{(}-\tau^{-1}z^{p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.$
(63)

######  Theorem 12.

Assume the connectivity-independent initial conditions (Case 2). For any
$\epsilon>0$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\xi)\geq\epsilon\big{)}<0.$
(64)

Thus with unit probability,

$\lim_{N\to\infty}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})=\xi.$ (65)

Furthermore,

$\displaystyle\xi=\lim_{n\to\infty}\xi^{(n)},$ (66)

where $\xi^{(n+1)}=\tilde{\Phi}(\xi^{(n)})$ and $\xi^{(1)}$ is any measure in
$\mathcal{U}$.

##  4 Proofs

We have divided the proofs into three main sections. In Section 4.1, we prove
general regularity properties of the stochastic processes. In Section 4.2, we
prove the LDP for the uncoupled system. In Section 4.3, we determine the
limiting dynamics of the coupled system.

###  4.1 Regularity Estimates and Compactness

We first prove Lemma 5.

###### Proof.

We first check that the covariance function is positive definite (when
restricted to a finite set of times). Let $\\{t_{i}\\}_{1\leq i\leq
m}\subset[0,T]$ be a finite set of times. Then evidently for any constants
$\\{\alpha^{p}_{i}\\}_{p\in I_{M},1\leq i\leq m}$, it must be that

$\displaystyle\sum_{p,q\in I_{M}}\sum_{1\leq i,j\leq
m}\alpha^{p}_{i}\alpha^{q}_{j}\mathbb{E}^{\nu}\big{[}\lambda\big{(}x^{p}_{t_{i}}\big{)}\lambda\big{(}x^{q}_{t_{j}}\big{)}\big{]}=\mathbb{E}^{\nu}\big{[}\big{(}\sum_{p\in
I_{M}}\sum_{1\leq i\leq
m}\alpha^{p}_{i}\lambda\big{(}x^{p}_{t_{i}}\big{)}\big{)}^{2}\big{]}\geq 0.$
(67)

This means that there exists a finite set of centered Gaussian variables
$\\{G^{\nu,p}_{t^{(m)}_{i}}\\}_{p\in I_{M}\fatsemi 1\leq i\leq m}$ such that
(41) holds. It then follows from the Komolgorov Extension Theorem that
$\beta_{\nu}$ is well-defined on any countably dense subset of times of
$[0,T]$. It remains for us to demonstrate continuity, i.e. that there exists a
Gaussian probability law such that (41) holds for all time. We do this using
standard theory for the continuity of Gaussian Processes (following Chapter 2
of [1]).

First, we notice that

$\displaystyle\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\mathbb{E}\big{[}(G^{\nu,p}_{t})^{2}\big{]}<\infty.$
(68)

Now define the canonical metric,

$\displaystyle\bar{d}_{p}(s,t)=$
$\displaystyle\mathbb{E}\big{[}\big{(}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{)}^{2}\big{]}^{\frac{1}{2}}=\mathbb{E}^{\nu}\big{[}\big{(}\lambda(x^{p}_{s})-\lambda(x^{p}_{t})\big{)}^{2}\big{]}^{\frac{1}{2}}$
(69) $\displaystyle\leq$ $\displaystyle\rm{Const}\sup_{p\in
I_{M}}\mathbb{E}^{\nu}\big{[}\big{|}x^{p}_{s}-x^{p}_{t}\big{|}^{2}\big{]}^{\frac{1}{2}}\leq
a\;(t-s)^{\frac{1}{4}}$ (70)

thanks to properties of the set $\mathcal{Q}_{a}$, for all $s,t$ such that
$|s-t|$ is smaller than some constant depending on $a$. It follows from
Theorem 1.4.1 of [1] that the Gaussian Process is almost-surely continuous.

Write $B_{t}(\epsilon)=\big{\\{}s\in[0,T]:\bar{d}(s,t)\leq\epsilon\big{\\}}$
to be the $\epsilon$-ball about $t$, and let $\mathcal{N}(\epsilon)$ denote
the smallest number of such balls that cover $T$. We see that there exists a
constant $\mathfrak{c}_{a}>0$ such that

$\displaystyle\mathcal{N}(\epsilon)\leq\mathfrak{c}_{a}\epsilon^{-4}.$ (71)

Writing $H(\epsilon)=\log\mathcal{N}(\epsilon)$, it follows from Theorem 1.3.5
in [1] that there exist $M$ Gaussian Processes $(G^{\nu,p}_{t})_{t\in[0,T]}$
such that $t\to G^{\nu,p}_{t}$ is almost-surely continuous, and there exists a
universal constant $\mathfrak{K}>0$ and a random $\eta>0$ such that for all
$\delta<\eta$,

$\displaystyle\sup_{p\in I_{M}\fatsemi s,t\leq
T\fatsemi\bar{d}(s,t)\leq\delta}\big{|}G^{\nu,p}_{s}-G^{\nu,p}_{t}\big{|}$
$\displaystyle\leq\mathfrak{K}\int_{0}^{\delta}H^{1/2}(\epsilon)d\epsilon$
(72)
$\displaystyle\leq\mathfrak{K}\int_{0}^{\delta}\big{(}4\log\big{(}\epsilon^{-1}\big{)}+\log\mathfrak{c}_{a}\big{)}^{\frac{1}{2}}d\epsilon,$
(73)

and we note that the above goes to 0 as $\delta\to 0^{+}$. This also implies
(42). ∎

We next prove Corollary 6.

###### Proof.

The proof of (48) is analogous to the proof of (42). Notice that
$\mathfrak{m}^{p}_{t}(\mu,\mathbf{g})$ depends continuously on $\mathbf{g}$.

(42) and (48) imply (respectively) (49) and (50) . ∎

We can now prove Lemma 7.

###### Proof.

We prove (52) only. The other proof is very similar.

It follows from Lemma 5 that for any $\epsilon>0$, there exists a compact set
$\mathcal{C}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that for
any $\mu\in\mathcal{Q}_{L}$, and all $\mathbf{g}_{0}\in\mathbb{R}^{M}$ such
that $\left\|\mathbf{g}_{0}\right\|\leq\epsilon^{-1}$,

$\displaystyle\beta_{\mu,\mathbf{g}_{0}}\big{(}G^{\mu}_{[0,T]}\notin\mathcal{C}_{\epsilon}\big{)}\leq\epsilon.$
(74)

It has already been noted above that for any
$\\{\mathbf{y}_{[0,T]}^{j}\\}_{j\in
I_{N}}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$,
$\\{\tilde{G}^{j}_{[0,T]}\\}_{j\in I_{N}}$ are independent, and the
probability law of $\tilde{G}^{j}_{[0,T]}$ is
$\beta_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}$. Thus as long as
$\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$, the estimate in (74) holds for
any $\tilde{G}^{j}_{[0,T]}$ such that
$\|\mathbf{g}^{j}_{0}\|\leq\epsilon^{-1}$.

Define

$\displaystyle\theta_{N,\epsilon}(\mathbf{g}_{0})=N^{-1}\sum_{j\in
I_{N}}\chi\\{\|\mathbf{g}^{j}_{0}\|>\epsilon^{-1}\\}.$ (75)

Our construction of $\mathcal{Y}^{N}$ implies that

$\displaystyle\lim_{\epsilon\to
0^{+}}\lim_{N\to\infty}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}\theta_{N,\epsilon}(\mathbf{g}_{0})=0.$
(76)

Write
$I_{N,\epsilon}=\big{\\{}j\;:\|\mathbf{g}^{j}_{0}\|\leq\epsilon^{-1}\big{\\}}$.
Next, for some $\delta>\epsilon/2$, and $b>0$, by Chernoff’s Inequality, for
any $N\geq 1$ and any $(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$,

$\displaystyle\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in
I_{N}}\\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\\}\geq\delta\bigg{)}$
$\displaystyle\leq\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in\tilde{I}_{N,\epsilon}}\\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\\}\geq\delta/2\bigg{)}$
$\displaystyle\leq\mathbb{E}^{\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}}\bigg{[}\exp\bigg{(}b\sum_{j\in\tilde{I}_{N,\epsilon}}\\{G^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\\}-bN\delta/2\bigg{)}\bigg{]}$
$\displaystyle\leq\big{\\{}\epsilon\exp(b)+1-\epsilon\big{\\}}^{N}\exp\big{(}-Nb\delta/2\big{)}$

We thus find that (by taking small enough $\epsilon$, and $1\ll
b\ll-\log\epsilon$), for any integer $n$, that there must exist a compact set
$\mathcal{C}^{(n)}$ such that for all $N\geq 1$, all
$(\mathbf{y}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$ and all $\mathbf{y}$ such
that $\hat{\mu}^{N}(\mathbf{y})\in\mathcal{Q}_{L}$

$\displaystyle
N^{-1}\log\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}N^{-1}\sum_{j\in
I_{N}}\\{G^{j}_{[0,T]}\notin\mathcal{C}^{(n)}\\}\geq n^{-1}\big{)}\leq-n^{2}.$
(77)

This motivates us to define the compact set
$\tilde{\mathcal{C}}_{L}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$
to consist of all measures $\mu$ such that for all $n\geq m$,

$\displaystyle\mu\big{(}\mathcal{C}^{(n)}\big{)}\geq 1-n^{-1}.$ (78)

Thus using a union-of-events bound,

$\displaystyle\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{G})\notin\tilde{\mathcal{C}}_{L}\big{)}$
$\displaystyle\leq\sum_{n\geq
m}\tilde{\gamma}^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}N^{-1}\sum_{j\in
I_{N}}\\{G^{j}_{[0,T]}\notin\mathcal{K}^{(n)}\\}\geq n^{-1}\bigg{)}$ (79)
$\displaystyle\leq\sum_{n\geq m}\exp\big{(}-Nn^{2}\big{)}$ (80)
$\displaystyle\leq\exp(-NL),$ (81)

for all $N\geq 1$, as long as $m$ is large enough.

∎

The following bound on the operator norm of the connectivity matrix is well-
known (and the proof is omitted).

######  Lemma 13.

For any $L>0$, there exists $\ell$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\geq\ell\big{)}\leq-L,$
(82)

where $\mathcal{J}_{N}\in\mathbb{R}^{N\times N}$ has $(j,k)$ entry

$\mathcal{J}_{N,jk}=N^{-1/2}J^{jk}$

######  Lemma 14.

For any $\ell>0$, there exists $L>0$ such that for all $p\in I_{M}$ and all
$N\geq 1$,

$\displaystyle
N^{-1}\log\mathbb{P}\big{(}\mathcal{A}_{c}\;,\;\sup_{t\in[0,T]}\sum_{j\in
I_{N}}(z^{p,j}_{t})^{2}\geq N\ell\big{)}\leq-L$ (83)

where

$\mathcal{A}_{c}=\bigg{\\{}\left\|\mathcal{J}_{N}\right\|\leq c\;,\;\sup_{p\in
I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}\leq
N\mathbb{E}^{\kappa}[(z^{p}_{0})^{2}]+N\bigg{\\}}.$

###### Proof.

Write

$u_{t}=N^{-1}\sum_{j\in I_{N}}(z^{p,j}_{t})^{2}.$

If the event $\mathcal{A}_{c}$ holds, then thanks to Ito’s Lemma it must be
that

$\displaystyle du_{t}=$
$\displaystyle\big{\\{}-2\tau^{-1}u_{t}+1+N^{-1}\sum_{j\in
I_{N}}z^{p,j}_{t}G^{p,j}_{t}\big{\\}}dt+N^{-1}\sum_{j\in
I_{N}}z^{p,j}_{t}dW^{p,j}_{t}$ (84) $\displaystyle\leq$
$\displaystyle\big{\\{}-2\tau^{-1}u_{t}+1+cC_{\lambda}u_{t}\big{\\}}dt+N^{-1}\sum_{j\in
I_{N}}z^{p,j}_{t}dW^{p,j}_{t},$ (85)

since $N^{-1}\sum_{j\in I_{N}}\lambda(z^{p,j}_{t})^{2}\leq
C_{\lambda}^{2}u_{t}$. Write

$\displaystyle v_{t}=\sup_{s\in[0,t]}N^{-1}\bigg{|}\sum_{j\in
I_{N}}\int_{0}^{t}z^{p,j}_{s}dW^{p,j}_{s}\bigg{|},$ (86)

and define the stopping time, for a constant $A>0$,

$\displaystyle\tau_{A}=\inf\big{\\{}t\geq 0\;:\;v_{t}\geq\exp(At)+A\big{\\}}.$
(87)

Gronwall’s Inequality implies that for all $t\leq\tau_{A}$,

$\displaystyle u_{t}\leq\big{(}A+u_{0}+t\big{)}\exp\big{(}\tilde{c}t\big{)}$

where $\tilde{c}=A+cC_{\lambda}-2\tau^{-1}$. The quadratic variation of
$x(t):=N^{-1}\sum_{j\in I_{N}}\int_{0}^{t}z^{p,j}_{s}dW^{p,j}_{s}$ is

$\displaystyle(QV)_{t}^{N}=N^{-2}\sum_{j\in
I_{N}}\int_{0}^{t}(z^{p,j}_{s})^{2}ds.$ (88)

For all $t\leq\tau_{A}$,

$\displaystyle(QV)_{t}^{N}$ $\displaystyle\leq
N^{-1}\tilde{c}^{-1}\big{(}A+u_{0}+t\big{)}\exp\big{(}\tilde{c}t\big{)}:=N^{-1}h_{t},$
(89)

and notice that $h_{t}$ is independent of the Brownian Motions. Now define the
stochastic process $w(t)$ to be such that

$\displaystyle w(t)=$ $\displaystyle x\big{(}\alpha^{N}_{t}\big{)}\text{ where
}$ (90) $\displaystyle\alpha^{N}_{t}=$ $\displaystyle\inf\big{\\{}s\geq
0\;:\;(QV)_{s}^{N}=t\big{\\}}$ (91)

Thanks to the time-rescaled representation of a stochastic integral, $w(t)$ is
a Brownian Motion [27]. Writing $f(t)=\exp(At)+A$, it follows that

$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }$ $\displaystyle s\leq
T\text{ such that }\big{|}x(s)\big{|}\geq f(s)\bigg{)}$ $\displaystyle\leq$
$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that
}\big{|}w(N^{-1}h_{s})\big{|}\geq f(s)\bigg{)}$ $\displaystyle\leq$
$\displaystyle\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that
}\big{|}w(N^{-1}h_{s^{(m)}})\big{|}\geq f(s_{(m)})\bigg{)}$

and we have written

$\displaystyle s^{(m)}$
$\displaystyle=\inf\big{\\{}t^{(m)}_{a}\;:t^{(m)}_{a}\geq s\big{\\}}$ (92)
$\displaystyle s_{(m)}$
$\displaystyle=\sup\big{\\{}t^{(m)}_{a}\;:t^{(m)}_{a}\leq s\big{\\}}.$ (93)

and we recall that $t^{(m)}_{a}=Ta/m$. Employing a union-of-events bound,

$\mathbb{P}\bigg{(}\text{ There exists }s\leq T\text{ such that
}\big{|}w(h_{s^{(m)}})\big{|}\geq f(s_{(m)})\bigg{)}\leq\\\
\sum_{a=0}^{m-1}\bigg{\\{}\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\geq
f\big{(}t_{a}^{(m)}\big{)}\bigg{)}+\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\leq-f\big{(}t_{a}^{(m)}\big{)}\bigg{)}\bigg{\\}}$
(94)

Now since $w(t)$ is centered and Gaussian, with variance of $t$,

$\displaystyle
N^{-1}\log\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\geq
f\big{(}t_{a}^{(m)}\big{)}\bigg{)}=$
$\displaystyle-\frac{N}{2}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}+O\big{(}\log
N\big{)}$ (95) $\displaystyle
N^{-1}\log\mathbb{P}\bigg{(}w\big{(}N^{-1}h_{t_{a+1}^{(m)}}\big{)}\leq-f\big{(}t_{a}^{(m)}\big{)}\bigg{)}=$
$\displaystyle-\frac{N}{2}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}+O\big{(}\log
N\big{)}.$ (96)

We fix $m=A$, and take $A$ to be arbitrarily large. Then

$\lim_{A\to\infty}\inf_{0\leq a\leq
m-1}f\big{(}t_{a}^{(m)}\big{)}^{2}\big{(}h_{t_{a+1}^{(m)}}\big{)}^{-1}=\infty.$

We thus find that, for large enough $A$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\bigg{(}\mathcal{A}_{c},\text{
There exists }s\leq T\text{ such that }\big{|}x(s)\big{|}\geq
f(s)\bigg{)}\leq-L.$ (97)

We have already demonstrated in the course of the proof that if the event
$\mathcal{A}_{c}$ holds, and $\sup_{s\in[0,T]}|x(s)|\leq f(s)$, then there
exists a constant such that $\sup_{t\in[0,T]}u_{t}\leq\rm{Const}$. We have
thus established the Lemma. ∎

The following $L^{2}$-Wasserstein distance provides a very useful way of
controlling the dependence of the fields $(G^{\nu}_{t})$ on the measure $\nu$.
Define $d^{(2)}_{t}(\cdot,\cdot)$ to be such that for any
$\mu,\nu\in\mathcal{U}$,

$\displaystyle
d^{(2)}_{t}(\mu,\nu)=\inf_{\zeta}\mathbb{E}^{\zeta}\bigg{[}\sum_{p\in
I_{M}}\int_{0}^{t}\big{\\{}(y^{p}_{s}-\tilde{y}^{p}_{s})^{2}+(G^{p}_{s}-\tilde{G}^{p}_{s})^{2}\big{\\}}ds\bigg{]}^{1/2},$
(98)

where the infimum is over all
$\zeta\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{2M})\times\mathcal{C}([0,T],\mathbb{R}^{2M}\big{)}$,
such that the law of the first $2M$ processes is given by $\mu$, and the law
of the last $2M$ processes is given by $\nu$. Let
$d^{(2)}(\mu,\nu):=d^{(2)}_{T}(\mu,\nu)$.

######  Lemma 15.

For any $a>0$, $d^{(2)}(\cdot,\cdot)$ metrizes weak convergence in
$\mathcal{U}_{a}$. Furthermore,

$\displaystyle\lim_{\epsilon\to
0^{+}}\sup\big{\\{}d_{W}(\mu,\nu)\;:\mu,\nu\in\mathcal{U}_{a}\text{ and
}d^{(2)}(\mu,\nu)\leq\epsilon\big{\\}}=0.$ (99)

###### Proof.

Since $\mathcal{U}_{a}$ is compact, Prokhorov’s Theorem implies that for any
$\tilde{\epsilon}>0$, there exists a compact set
$\mathcal{D}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})^{2}$ such that
for all $\mu\in\mathcal{U}_{a}$,

$\displaystyle\mu\big{(}\mathcal{D}_{\epsilon}\big{)}\geq 1-\tilde{\epsilon}.$
(100)

Since $\mathcal{D}_{\epsilon}$ is compact, it follows from the Arzela-Ascoli
Theorem that for any $\delta>0$, there exists $\upsilon(\epsilon,\delta)$ such
that for all $f,g\in\mathcal{D}_{\epsilon}$ such that for all $p\in I_{2M}$,

$\displaystyle\int_{0}^{T}(f^{p}(t)-g^{p}(t))^{2}dt\leq\upsilon(\epsilon,\delta),$
(101)

it necessarily holds that

$\displaystyle\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}f^{p}(t)-g^{p}(t)\big{|}\leq\delta.$ (102)

Let $\zeta$ be any measure that is within $\eta\ll 1$ of realizing the infimum
in (98). Then, writing

$\mathcal{A}_{\epsilon}=\chi\big{\\{}\text{ For each }p\in
I_{M},\;\;y^{p},\tilde{y}^{p},g^{p},\tilde{g}^{p}\in\mathcal{D}_{\epsilon}\big{\\}},$

we have the bound

$\mathbb{E}^{\zeta}\bigg{[}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{]}\\\
\leq\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\mathcal{A}_{\epsilon}\bigg{]}+\\\
\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\bigg{]}$
(103)

Now we take $d^{(2)}(\mu,\nu)\to 0^{+}$, and $\eta\to 0^{+}$ too. Since
$\mathcal{A}_{\epsilon}$ is closed, thanks to the Portmanteau Theorem, we thus
find that for any $\epsilon>0$,

$\displaystyle\mathbb{E}^{\zeta}\bigg{[}\mathcal{A}_{\epsilon}\sum_{p\in
I_{M}}\int_{0}^{T}\big{\\{}(y^{p}_{s}-\tilde{y}^{p}_{s})^{2}+(G^{p}_{s}-\tilde{G}^{p}_{s})^{2}\big{\\}}ds\bigg{]}\to
0.$ (104)

which in turn implies that (making use of the uniform convergence over
$\mathcal{D}_{\epsilon}$ in (102))

$\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\mathcal{A}_{\epsilon}\bigg{]}\to
0.$ (105)

For the other term on the RHS of (103), for $b>0$, write

$\displaystyle\mathcal{B}_{b}=\chi\bigg{\\{}\text{ For each }p\in
I_{M},\;\sup_{t\in[0,T]}\big{|}y^{p}_{t}\big{|}\leq
b\;,\;\sup_{t\in[0,T]}\big{|}\tilde{y}^{p}_{t}\big{|}\leq
b\;,\;\sup_{t\in[0,T]}\big{|}g^{p}_{t}\big{|}\leq
b\;,\;\sup_{t\in[0,T]}\big{|}\tilde{g}^{p}_{t}\big{|}\leq b\bigg{\\}}$

Then,

$\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\bigg{]}\\\
\leq\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\mathcal{B}_{b}\bigg{]}\\\
+\mathbb{E}^{\zeta}\bigg{[}\bigg{(}\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}y^{p}(t)-\tilde{y}^{p}(t)\big{|}+\sup_{p\in
I_{M}}\sup_{t\in[0,T]}\big{|}g^{p}(t)-\tilde{g}^{p}(t)\big{|}\bigg{)}\big{(}1-\mathcal{A}_{\epsilon}\big{)}\big{(}1-\mathcal{B}_{b}\big{)}\bigg{]}.$
(106)

Thanks to the fact that, for all $\mu\in\mathcal{U}_{a}$,

$\sup_{p\in
I_{M}}\mathbb{E}^{\mu}\big{[}\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq a,$

one finds that the second term on the RHS of (106) goes to $0$ as
$b\to\infty$, uniformly for all $\epsilon>0$ and all
$\mu,\nu\in\mathcal{U}_{a}$ . For any fixed $b\gg 1$, the first term on the
RHS of (106) must go to zero as $\epsilon\to 0^{+}$, thanks to (100). We have
thus proved the Lemma. ∎

For $\mu,\in\mathfrak{Q}$, we define $d^{(2)}_{t}(\mu,\nu)$ analogously to
(98).

######  Lemma 16.

There exists a constant $\mathfrak{C}>0$ such that for all
$\mu,\nu\in\mathfrak{Q}$ and all $t\in[0,T]$,

$\displaystyle d^{(2)}_{t}(\beta_{\nu},\beta_{\mu})$
$\displaystyle\leq\mathfrak{C}d^{(2)}_{t}(\nu,\mu).$ (107)

Also for all $\mu,\nu\in\mathfrak{Q}$ such that for some $b>0$,
$\det(\mathfrak{V}_{\mu,0}),\det(\mathfrak{V}_{\nu,0})\geq b>0$, there exists
a constant $\mathfrak{C}_{b}$ such that

$\displaystyle
d^{(2)}_{t}\big{(}\beta_{\nu,\mathbf{g}},\beta_{\mu,\mathbf{g}}\big{)}$
$\displaystyle\leq\tilde{\mathfrak{C}}_{b}(1+\left\|\mathbf{g}\right\|)d^{(2)}_{t}(\nu,\mu),$
(108)

and $\left\|\cdot\right\|$ is the Euclidean norm on $\mathbb{R}^{M}$.

###### Proof.

We start by proving (107). Recalling the definition of the distance
$d^{(2)}_{t}$ in (98), let $\zeta_{\epsilon}$ be such that

$\displaystyle
d^{(2)}_{t}(\mu,\nu)^{2}\geq\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in
I_{M}}\int_{0}^{t}(z^{p}_{s}-y^{p}_{s})^{2}ds\bigg{]}+\epsilon.$ (109)

Furthermore define centered Gaussian processes
$G^{(\epsilon),\mu,p}_{s},G^{(\epsilon),\nu,p}_{s}$ to be such that for any
$p,q\in I_{M}$ and any $s,t\in[0,T]$,

$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\mu,p}_{s}G^{(\epsilon),\nu,q}_{t}\big{]}$
$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(z^{p}_{s})\lambda(y^{q}_{t})\big{]}$
(110)
$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\mu,p}_{s}G^{(\epsilon),\mu,q}_{t}\big{]}$
$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(z^{p}_{s})\lambda(z^{q}_{t})\big{]}$
(111)
$\displaystyle\mathbb{E}\big{[}G^{(\epsilon),\nu,p}_{s}G^{(\epsilon),\nu,q}_{t}\big{]}$
$\displaystyle=\mathbb{E}^{\zeta_{\epsilon}}\big{[}\lambda(y^{p}_{s})\lambda(y^{q}_{t})\big{]}.$
(112)

This definition is possible thanks to a trivial modification of Lemma 5
(switching $M\to 2M$). We thus find that

$\displaystyle\lim_{\epsilon\to 0^{+}}\mathbb{E}\bigg{[}\sum_{p\in
I_{M}}\int_{0}^{r}(G^{(\epsilon),\mu,p}_{t}-G^{(\epsilon),\nu,p}_{t})^{2}dt\bigg{]}\leq$
$\displaystyle\lim_{\epsilon\to
0^{+}}\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in
I_{M}}\int_{0}^{r}\big{\\{}\lambda(z^{p}_{t})-\lambda(y^{p}_{t})\big{\\}}^{2}dt\bigg{]}$
(113) $\displaystyle\leq$ $\displaystyle\lim_{\epsilon\to
0^{+}}C_{\lambda}^{2}\mathbb{E}^{\zeta_{\epsilon}}\bigg{[}\sum_{p\in
I_{M}}\int_{0}^{r}\big{\\{}z^{p}_{t}-y^{p}_{t}\big{\\}}^{2}dt\bigg{]}$ (114)
$\displaystyle=$ $\displaystyle C_{\lambda}^{2}d_{r}^{(2)}(\mu,\nu).$ (115)

Now as $\epsilon\to 0^{+}$, the LHS of (113) must decrease to
$d^{(2)}(\beta_{\mu},\beta_{\nu})$. (108) follows analogously.

The proof of (108) is analogous, since the mean and variance functions in (45)
and (46) are such that for all $\mu,\nu\in\mathcal{U}$, there is a constant
$C_{b}>0$ such that

$\displaystyle\sup_{p,q\in I_{M}\fatsemi
s.t\in[0,T]}\big{|}\mathfrak{W}^{\mu,pq}_{st}-\mathfrak{W}^{\nu,pq}_{st}\big{|}$
$\displaystyle\leq C_{b}\lim_{\epsilon\to 0^{+}}\sup_{r\in I_{M}\fatsemi
s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$
$\displaystyle\sup_{p,q\in I_{M}\fatsemi
s.t\in[0,T]}\big{|}\mathfrak{m}^{p}_{s}(\mu,\mathbf{g})-\mathfrak{m}^{p}_{s}(\nu,\mathbf{g})\big{|}$
$\displaystyle\leq C_{b}\big{(}1+\|\mathbf{g}\|\big{)}\lim_{\epsilon\to
0^{+}}\sup_{r\in I_{M}\fatsemi
s\in[0,T]}\mathbb{E}^{\zeta_{\epsilon}}\big{[}\big{(}\lambda(z^{r}_{s})-\lambda(y^{r}_{s})\big{)}^{2}\big{]}$

We have also employed the fact that $\det(\mathfrak{V}_{\mu,0})$ is uniformly
lower-bounded by a positive constant $b$ (as noted in the statement of the
Lemma). ∎

###  4.2 Large Deviations of the Uncoupled System

Our first aim is to prove a Large Deviation Principle in the case of fields
with a frozen interaction structure (in Lemma 17 below). This would ordinarily
be a trivial application of Sanov’s Theorem. However the proof is slightly
complicated by the need for the LDP to be uniform with respect to the
variables
$(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathbb{R}^{MN}\times\mathbb{R}^{MN}$ that
the probability laws are conditioned on.

For any $\mathbf{g}_{0}\in\mathbb{R}^{MN}$ and $\nu\in\mathcal{U}$, define
$\tilde{\gamma}^{N}_{\nu,\mathbf{g}_{0}}:=\otimes_{j=1}^{N}\beta_{\nu,\mathbf{g}_{0}^{j}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$.
In other words, $\tilde{\gamma}^{N}_{\nu,\mathbf{g}_{0}}$ is the law of $N$
independent $\mathcal{C}([0,T],\mathbb{R}^{M})$-valued Gaussian variables
$\\{\tilde{G}^{\nu,j}\\}_{j\in I_{N}}$. The mean and variance of these
variables is specified in (43) and (44).

Let
$Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
be the joint law of the uncoupled system, i.e.

$Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}=\gamma^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\otimes
P^{N}_{\mathbf{z}_{0}}.$ (116)

For $\nu\in\mathcal{U}$, define
$\tilde{I}_{\nu}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}$
as follows. We specify that $\tilde{I}_{\nu}(\mu)=\infty$ if either the
marginal of $\mu$ at time $0$ is not equal to $\kappa$, and / or
$\mu\notin\mathcal{U}$. Otherwise, for any $\zeta\in\mathcal{U}$, writing
$\zeta_{\mathbf{z}_{0},\mathbf{g}_{0}}$ to be the law of $\zeta$, conditioned
on the values of its variables at time $0$, define

$\displaystyle\tilde{I}_{\nu}(\zeta)=\mathbb{E}^{\kappa}\big{[}\mathcal{R}\big{(}\zeta_{\mathbf{z}_{0},\mathbf{g}_{0}}||P_{\mathbf{z}_{0}}\otimes\tilde{\gamma}_{\nu,\mathbf{g}_{0}}\big{)}\big{]}.$
(117)

Define the empirical measure
$\tilde{\mu}^{N}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R})^{M}\times\mathcal{C}([0,T],\mathbb{R})^{M}\big{)}$
to be

$\displaystyle\tilde{\mu}^{N}=N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]},\tilde{G}^{\nu,j}_{[0,T]}},$ (118)

where we recall that

$\displaystyle y^{p}_{t}=z^{p}_{0}+\int_{0}^{t}\sigma_{s}dW_{s}^{p}.$ (119)

######  Lemma 17.

Fix some $\nu\in\mathcal{U}$. Let
$\mathcal{A},\mathcal{O}\subseteq\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$,
such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$
$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu)$ (120)
$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$
$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\nu}(\mu).$ (121)

Furthermore $\tilde{I}_{\nu}(\cdot)$ is lower semi-continuous, and has compact
level sets.

###### Proof.

First, fix any sequence
$\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}_{N\geq 1}$, such that
$\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\in\mathcal{Y}^{N}$.
Necessarily, thanks to the definition of $\mathcal{Y}^{N}$, it must be that

$\displaystyle\hat{\mu}^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\to\kappa\in\mathcal{P}\big{(}\mathbb{R}^{2M}\big{)}.$
(122)

It follows from (122) that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$
$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu)$ (123)
$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$
$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\nu}(\mu).$ (124)

See for instance [30] for a proof of this fact. Furthermore
$\tilde{I}_{\nu}:\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\to\mathbb{R}^{+}$
is lower-semi-continuous and has compact level sets.

We next have to show that the convergence is uniform over $\mathcal{Y}^{N}$
(as in the statement of the Theorem). To do this, we first wish to show that
for any measurable set
$\mathcal{E}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$
and any $\epsilon>0$, for all large enough $N$,

$\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{E}\big{)}\leq\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{E}^{(\epsilon)}\big{)}$
(125)

and $\mathcal{E}^{(\epsilon)}$ is the closed $\epsilon$-blowup of
$\mathcal{E}$ with respect to $d_{W}$. To do this, we are going to compare the
conditioned probability to the conditioned probability induced by any other
sequence in $\mathcal{Y}^{N}$. This comparison is facilitated by using the
following permutation-averaged probability law.

Define the set

$\mathfrak{S}^{N}=\bigg{\\{}(\mathbf{y},\mathbf{g})\in\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}:\\\
N^{-1}\sum_{j\in I_{N}}\big{\\{}\sup_{p\in
I_{M}}\big{|}y^{p,j}_{0}-z^{(N),p,j}_{0}\big{|}+\sup_{p\in
I_{M}}\big{|}g^{p,j}_{0}-g^{(N),p,j}_{0}\big{|}\big{\\}}\leq
2\tilde{\delta}_{N}\bigg{\\}},$ (126)

and we recall that $(\tilde{\delta}_{N})_{N\geq 1}$ is a sequence that
decreases to $0$, as defined in (30). We endow $\mathfrak{S}^{N}$ with the
topology that it inherits from
$\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}$.
Write $\mathfrak{P}^{N}$ to be the set of all permutations on $I_{N}$, and
define the measure
$\bar{Q}^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\in\mathcal{P}\big{(}\mathfrak{S}^{N}\big{)}$
to be the average over all permutations, i.e. for any measurable
$\mathcal{A}\subseteq\mathfrak{S}^{N}$,

$\displaystyle\bar{Q}^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}(\mathcal{A})=\big{|}\mathfrak{P}^{N}\big{|}^{-1}\sum_{\pi\in\mathfrak{P}^{N}}Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\pi\cdot\mathcal{A}\big{)}$
(127)

and here we denote
$\pi:\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}\times\mathcal{C}\big{(}[0,T],\mathbb{R}^{M}\big{)}^{N}$
to be the permutation,

$\displaystyle\big{(}\pi\cdot(\mathbf{y},\mathbf{g})\big{)}^{j}=(\mathbf{y}^{\pi(j)},\mathbf{g}^{\pi(j)}).$
(128)

Since the empirical measure is invariant under any permutation of its
arguments, for any measurable
$\mathcal{A}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}$

$\displaystyle
Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}=\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}.$
(129)

We can without loss of generality take
$\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}\big{)}\in\mathfrak{S}^{N}$.
Now consider any other sequence
$\big{(}\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}\in\mathfrak{S}^{N}\cap\mathcal{Y}^{N}$.
Let $d_{W}$ be the $1$-Wasserstein Distance on
$\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
induced by the norm

$\displaystyle\left\|(\mathbf{y},\mathbf{G})-(\bar{\mathbf{y}},\bar{\mathbf{G}})\right\|=N^{-1}\sum_{j\in
I_{N}}\big{\\{}\sup_{p\in I_{M}\fatsemi
t\in[0,T]}\big{|}y^{p,j}_{t}-\bar{y}^{p,j}_{t}\big{|}+\sup_{p\in I_{M}\fatsemi
t\in[0,T]}\big{|}G^{p,j}_{t}-\bar{G}^{p,j}_{t}\big{|}\big{\\}}.$ (130)

We claim that

$\displaystyle
d_{W}\big{(}\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}},\bar{Q}^{N}_{\nu,\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}}\big{)}\leq$
$\displaystyle N^{-1}\sum_{j\in I_{N}}\big{\\{}\sup_{p\in
I_{M}}\big{|}z^{(N),p,j}_{0}-\grave{z}^{(N),p,j}_{0}\big{|}$
$\displaystyle+\sup_{p\in I_{M}\fatsemi
t\in[0,T]}\big{|}\mathfrak{m}^{p}_{t}(\nu,\mathbf{g}^{(N),j}_{0})-\mathfrak{m}^{p}_{t}(\nu,\grave{\mathbf{g}}^{(N),j}_{0})\big{|}\big{\\}}$
(131) $\displaystyle:=$ $\displaystyle
f^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0},\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}$
(132)

This identity follows from the fact that
$\tilde{z}^{p,j}_{t}:=z^{p,j}_{t}-z^{p,j}_{0}$ and
$\tilde{g}^{p,j}_{t}:=G^{p,j}_{t}-\mathfrak{m}^{p}_{t}(\nu,\mathbf{G}^{j}_{0})$
are identically distributed, both (i) for all $j\in I_{N}$, and (ii) with
respect to both probability laws
$\bar{Q}^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}$ and
$\bar{Q}^{N}_{\nu,\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}}$.

It follows from the definition of $\mathcal{Y}^{N}$ that

$\displaystyle\lim_{N\to\infty}\sup_{(\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}),(\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0})\in\mathfrak{S}^{N}\cap\mathcal{Y}^{N}}f^{N}\big{(}\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0},\grave{\mathbf{z}}^{(N)}_{0},\grave{\mathbf{g}}^{(N)}_{0}\big{)}=0.$
(133)

We have thus proved (125).

To now must prove the Large Deviations bounds in the statement of the theorem.
We start with the upper-bound (120). It follows from (123) and (125) that for
any $\epsilon>0$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}\leq-\inf_{\mu\in\mathcal{A}^{(\epsilon)}}\tilde{I}_{\nu}(\mu).$
(134)

The lower-semi-continuity of $\tilde{I}_{\nu}(\mu)$ dictates that

$\lim_{\epsilon\to
0^{+}}\inf_{\mu\in\mathcal{A}^{(\epsilon)}}\tilde{I}_{\nu}(\mu)=\inf_{\mu\in\mathcal{A}}\tilde{I}_{\nu}(\mu),$

and we have proved the upperbound. For the lower-bound, let $\mathcal{O}$ be
open, and for any $\mu\in\mathcal{O}$, take $\epsilon>0$ to be such that
$B_{2\epsilon}(\mu)\subset\mathcal{O}$. Then

$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}$
$\displaystyle N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$
(135)
$\displaystyle\geq\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in
B_{2\epsilon}(\mu)\big{)}$ (136)
$\displaystyle\geq\underset{N\to\infty}{\underline{\lim}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}^{(N)}_{0},\mathbf{g}^{(N)}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in
B_{\epsilon}(\mu)\big{)}$ (137) $\displaystyle\geq-\inf_{\zeta\in
B_{\epsilon}(\mu)}\tilde{I}_{\nu}(\zeta),$ (138)

using the Large Deviations estimate (124). Taking $\epsilon\to 0^{+}$, it must
be that for any $\mu\in\mathcal{O}$,

$\displaystyle\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}\geq-\tilde{I}_{\nu}(\mu).$
(139)

Since $\mu\in\mathcal{O}$ is arbitrary, (121) follows immediately. ∎

We can now state the proof of Theorem 3.

###### Proof.

We start with the upper bound (35). We write
$\hat{\mu}^{N}:=\hat{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}_{[0,T]})$.
Using a union-of-events bound, for any $a>0$,

$\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\big{)}\leq\\\
\max\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)},\\\
\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\notin\mathcal{U}_{a}\big{)}\bigg{\\}}\\\
\leq\max\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)},-L\bigg{\\}},$
(140)

for any $L>0$, as long as $a$ is sufficiently large, thanks to the exponential
tightness proved in Lemma 8. By taking $a\to\infty$, it thus suffices that we
prove that for arbitrary $\mathcal{U}_{a}$ such that
$\mathcal{A}\cap\mathcal{U}_{a}\neq\emptyset$,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}=-\inf_{\mu\in\mathcal{A}\cap\mathcal{U}_{a}}\tilde{I}_{\mu}(\mu).$
(141)

Since $\mathcal{A}\cap\mathcal{U}_{a}$ is compact, for any $\epsilon>0$ we can
always find an open covering of the form, for some positive integer
$\mathcal{N}_{\epsilon}$, $\\{\zeta_{i}\\}_{1\leq
i\leq\mathcal{N}_{\epsilon}}\subseteq\mathcal{A}\cap\mathcal{U}_{a}$,

$\displaystyle\mathcal{A}\cap\mathcal{U}_{a}\subseteq\bigcup_{i=1}^{\mathcal{N}_{\epsilon}}B_{\epsilon}(\zeta_{i}).$
(142)

We thus find that

$\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}\\\
\leq\sup_{1\leq
i\leq\mathcal{N}_{\epsilon}}\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in
B_{\epsilon}(\zeta_{i})\big{)}\bigg{\\}}.$ (143)

Thus, employing Lemma 5 in the third line below,

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}$
$\displaystyle N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in
B_{\epsilon}(\zeta_{i})\big{)}$ (144)
$\displaystyle=\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\gamma^{N}_{\mathbf{y},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in
B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$ (145)
$\displaystyle=\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\tilde{\gamma}^{N}_{\hat{\mu}^{N}(\mathbf{y}),\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in
B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$ (146)
$\displaystyle\leq\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\sup_{\nu\in
B_{\epsilon}(\zeta_{i})}\gamma^{N}_{\nu^{(1)},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in
B_{\epsilon}(\zeta_{i})\bigg{)}\bigg{]}$ (147)
$\displaystyle=-\inf_{\mu,\nu\in
B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu),$ (148)

thanks to Theorem 17. We thus find that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{A}\cap\mathcal{U}_{a}\big{)}$
$\displaystyle\leq-\inf_{1\leq i\leq\mathcal{N}_{\epsilon}}\inf_{\nu,\mu\in
B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu).$ (149)

Now, it is proved in Lemma 16 that $\nu\to
Q_{\nu,\mathbf{z}_{0},\mathbf{g}_{0}}$ is continuous. Since the Relative
Entropy is lower-semi-continuous in both of its arguments, we thus find that
the following map is lower-semi-continuous,

$(\nu,\mu)\to\tilde{I}_{\nu}(\mu).$

Thus taking $\epsilon\to 0^{+}$, we obtain that

$\displaystyle\lim_{\epsilon\to 0^{+}}\inf_{1\leq
i\leq\mathcal{N}_{\epsilon}}\inf_{\nu,\mu\in
B_{\epsilon}(\zeta_{i})}\tilde{I}_{\nu}(\mu)=\inf_{\mu\in\mathcal{A}\cap\mathcal{U}_{a}}\tilde{I}_{\mu}(\mu),$
(150)

and we have proved (141).

Turning to the lower bound (36), consider an arbitrary open set $\mathcal{O}$.
If $\mathcal{O}\cap\mathcal{U}=\emptyset$, then

$\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}=-\infty=-\inf_{\mu\in\mathcal{O}}\mathcal{I}(\mu),$

since $\mathcal{I}$ is identically $\infty$ outside of $\mathcal{U}$. In this
case, its clear that (121) holds.

We can thus assume that $\mathcal{O}\cap\mathcal{U}\neq\emptyset$. Let
$\mu\in\mathcal{O}$ be such that $\mu$ is in the interior of
$\mathcal{U}_{a}$, for some $a>0$. We can thus find a sequence of
neighborhoods $\\{\mathcal{N}_{i}\\}_{i\geq 1}$ of $\mu$ such that
$\mathcal{N}_{j}\subseteq\mathcal{O}\cap\mathcal{U}_{a}\cap B_{j^{-1}}(\mu)$.
We thus find that for any $j\geq 1$,

$\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}\geq\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}.$
(151)

Similarly to the bound for the closed sets, we obtain that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}\geq-\sup_{\nu\in\mathcal{N}_{j}}\inf_{\mu\in\mathcal{N}_{j}}\tilde{I}_{\nu}(\mu).$
(152)

Taking $j\to\infty$, since $(\nu,\mu)\to\tilde{I}_{\nu}(\mu)$ is lower
semicontinuous, it must be that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{N}_{j}\big{)}\geq-\tilde{I}_{\mu}(\mu).$
(153)

Since $\mu\in\mathcal{O}$ is arbitrary, it must be that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}\in\mathcal{O}\big{)}\geq-\inf_{\mu\in\mathcal{O}}\tilde{I}_{\mu}(\mu).$
(154)

∎

####  4.2.1 Uncoupled System (with no conditioning)

In this subsection we prove Corollary 11.

For some $\nu\in\mathfrak{Q}$, let
$Q^{N}_{\nu}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\times\mathcal{C}([0,T],\mathbb{R}^{M})^{N}\big{)}$
be the joint law of the uncoupled system (with no conditioning), i.e.

$Q^{N}_{\nu}=\big{(}\beta_{\nu}\otimes P_{\mathbf{z}}\big{)}^{\otimes N}.$
(155)

We reach a corollary to Lemma 17.

######  Corollary 18.

Fix some $\nu\in\mathcal{U}$. Let
$\mathcal{A},\mathcal{O}\subseteq\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})^{2}\big{)}\big{)}$,
such that $\mathcal{O}$ is open and $\mathcal{A}$ closed. Then

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log
Q^{N}_{\nu}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{A}\big{)}$
$\displaystyle\leq-\inf_{\mu\in\mathcal{A}}\mathcal{R}(\mu||S_{\nu})$ (156)
$\displaystyle\underset{N\to\infty}{\underline{\lim}}N^{-1}\log
Q^{N}_{\nu}\big{(}\tilde{\mu}^{N}(\mathbf{y}_{[0,T]},\tilde{\mathbf{G}}^{\nu}_{[0,T]})\in\mathcal{O}\big{)}$
$\displaystyle\geq-\inf_{\mu\in\mathcal{O}}\mathcal{R}(\mu||S_{\nu}).$ (157)

Furthermore $\mu\to\mathcal{R}(\mu||S_{\nu})$ is lower semi-continuous, and
has compact level sets.

###### Proof.

This is a consequence of Sanov’s Theorem. ∎

The proof of Corollary 11 now follows analogously to the proof of 3.

###  4.3 Coupled System

Girsanov’s Theorem implies that

$\displaystyle\frac{dP^{N}_{\mathbf{J},\mathbf{z}_{0}}}{dP^{N}_{\mathbf{z}_{0}}}\bigg{|}_{\mathcal{F}_{T}}(\mathbf{y})=\exp\big{(}N\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})\big{)}$
(158)

where $\Gamma^{N}_{\mathbf{J},T}:\mathbb{R}^{MN}\to\mathbb{R}$ is

$\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})=N^{-1}\sum_{j\in I_{N}\fatsemi p\in
I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}dy^{p,j}_{s}-\frac{1}{2}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}^{2}ds.$
(159)

We wish to specify a map $\Gamma:\mathcal{U}\to\mathbb{R}$ with (i) as nice
regularity properties as possible, and (ii) such that with unit probability

$\displaystyle\Gamma^{N}_{\mathbf{J},T}(\mathbf{y})=\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}.$
(160)

It is well-known that the stochastic integral is not a continuous function of
the driving Brownian motion. Thus we define the map $\Gamma$ to be a limit of
time-discretized approximations, and we will show that this limit must always
converge for any measure in $\mathcal{U}$.

Our precise definition of $\Gamma:\mathcal{U}\to\mathbb{R}$ is as follows. We
first define a time-discretized approximation to $\Gamma$.
$\Gamma^{(m)}:\mathcal{U}\to\mathbb{R}^{+}$,

$\Gamma^{(m)}(\mu)=\sum_{p\in
I_{M}}\sum_{a=0}^{m-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(m)}_{a}}^{-2}\big{(}G^{p}_{t_{a}^{(m)}}-\tau^{-1}z^{p}_{t_{a}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{a+1}}-z^{p}_{t^{(m)}_{a}}+\Delta_{m}\tau^{-1}z^{p}_{t^{(m)}_{a}}\big{)}\\\
-\frac{1}{2}\sigma_{t^{(m)}_{a}}^{-2}\Delta_{m}\big{(}G^{p}_{t_{a}^{(m)}}-\tau^{-1}z^{p}_{t_{a}^{(m)}}\big{)}^{2}\bigg{]}.$
(161)

We now define $\Gamma:\mathcal{U}\to\mathbb{R}$ to be such that (in the case
that the following limit exists)

$\displaystyle\Gamma(\mu)=\lim_{j\to\infty}\Gamma^{(m_{j,j})}(\mu),$ (162)

where $m_{j,j}$ is a positive integer defined further below in Lemma 20. If
the above limit does not exist, then we define $\Gamma(\mu)=0$ (in fact we
will see that the limit always exists if $\mu\in\mathcal{U}$). It may be
observed that $\Gamma$ is a well-defined measurable function.

######  Lemma 19.

For every $N\geq 1$, every
$(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$, and for
$Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}$ almost every
$(\mathbf{y},\tilde{\mathbf{G}})$, the following limit exists

$\lim_{j\to\infty}\Gamma^{(m_{j,j})}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}$
(163)

With unit probability, the Radon-Nikodym Derivative in (158) is such that

$\displaystyle\frac{dP^{N}_{\mathbf{J},\mathbf{z}_{0}}}{dP^{N}_{\mathbf{z}_{0}}}\bigg{|}_{\mathcal{F}_{T}}=\exp\big{(}\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\big{)}$
(164)

Also for any $\epsilon,L>0$, there exists $k\in\mathbb{Z}^{+}$ such that for
all $N\geq 1$,

$\displaystyle\sup_{j\geq
k}\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\bigg{(}\big{|}\Gamma^{(m_{j,j})}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\big{|}\geq\epsilon\bigg{)}\leq-L$
(165)

###### Proof.

Define the set

$\displaystyle\mathcal{A}_{j}=\big{\\{}\mu\in\mathcal{U}:\big{|}\Gamma^{(m_{j,j})}(\mu)-\Gamma^{(m_{j+1,j+1})}(\mu)\big{|}\geq
2^{1-j}\big{\\}}$ (166)

Thanks to a union-of-events bound, for any $N\geq 1$, and using the bound in
Lemma 20,

$\displaystyle\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\bigg{(}\hat{\mu}^{N}\in\bigcup_{j\geq
k}\mathcal{A}_{j}\bigg{)}\leq\sum_{j=k}^{\infty}\exp\big{(}-N2^{j}\big{)}.$
(167)

It thus follows from the Borel-Cantelli Lemma that there exists a random $k$
such that $\hat{\mu}^{N}\notin\mathcal{A}_{j}$ for all $j\geq k$, and so the
limit in (163) exists (almost surely). (165) follows analogously.

∎

######  Lemma 20.

(i) $\Gamma^{(m)}:\mathcal{U}\to\mathbb{R}$ is continuous. (ii) Moreover, for
any $a,j\in\mathbb{Z}^{+}$, there exists $m_{a,j}$ such that for all $m\geq
m_{a,j}$ and all $n\geq m$,

$\displaystyle\sup_{\mathbf{z}_{0},\mathbf{g}_{0}\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\big{|}\Gamma^{(m)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma^{(n)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}\geq
2^{-j}\big{)}\leq-2^{a}.$ (168)

###### Proof.

(i) The continuity of $\Gamma^{(m)}$ is almost immediate from the definition.

(ii) For any $t\in[0,T]$, write $t^{(m)}=\sup\\{t^{(m)}_{b}\;:t^{(m)}_{b}\leq
t\\}$. Starting with the discrete approximation to the stochastic integral, we
can thus write

$\displaystyle\sum_{b=0}^{m-1}\sigma_{t^{(m)}_{a}}^{-2}\big{(}G^{p}_{t_{b}^{(m)}}-\tau^{-1}z^{p}_{t_{b}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{b+1}}-z^{p}_{t^{(m)}_{b}}\big{)}=\int_{0}^{T}\sigma^{-2}_{t^{(m)}}\big{(}G^{p}_{t^{(m)}}-\tau^{-1}z^{p}_{t^{(m)}}\big{)}dz^{p}_{t}.$
(169)

Hence,

$\sum_{b=0}^{m-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(m)}_{b}}^{-2}\big{(}G^{p}_{t_{b}^{(m)}}-\tau^{-1}z^{p}_{t_{b}^{(m)}}\big{)}\big{(}z^{p}_{t^{(m)}_{b+1}}-z^{p}_{t^{(m)}_{a}}\big{)}\bigg{]}-\\\
\sum_{ab=0}^{n-1}\mathbb{E}^{\mu}\bigg{[}\sigma_{t^{(n)}_{b}}^{-2}\big{(}G^{p}_{t_{b}^{(n)}}-\tau^{-1}z^{p}_{t_{b}^{(n)}}\big{)}\big{(}z^{p}_{t^{(n)}_{b+1}}-z^{p}_{t^{(n)}_{b}}\big{)}\bigg{]}\\\
=\mathbb{E}^{\mu}\bigg{[}\int_{0}^{T}\bigg{\\{}\sigma^{-2}_{t^{(m)}}\big{(}G^{p}_{t^{(m)}}-\tau^{-1}z^{p}_{t^{(m)}}\big{)}-\sigma^{-2}_{t^{(n)}}\big{(}G^{p}_{t^{(n)}}-\tau^{-1}z^{p}_{t^{(n)}}\big{)}\bigg{\\}}dz^{p}_{t}\bigg{]}\\\
=\mathbb{E}^{\mu}\bigg{[}\int_{0}^{T}\sum_{p\in
I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{]}$ (170)

where $f^{p}_{t}=\sigma_{t}^{-2}\big{(}G^{p}_{t}-\tau^{-1}z^{p}_{t}\big{)}$.
Writing

$\displaystyle
f^{p,j}_{t}=\sigma_{t}^{-2}\big{(}G^{p,j}_{t}-\tau^{-1}z^{p,j}_{t}\big{)},$
(171)

we obtain that

$\displaystyle\mathbb{E}^{\hat{\mu}^{N}}\bigg{[}\int_{0}^{T}\sum_{p\in
I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{]}=N^{-1}\sum_{j\in
I_{N}\fatsemi p\in
I_{M}}\int_{0}^{T}\big{(}f^{p,j}_{t^{(m)}}-f^{p,j}_{t^{(n)}}\big{)}dy^{p,j}_{t}.$
(172)

The quadratic variation of this stochastic integral is

$\displaystyle(QV)^{(m,n),N}_{t}=N^{-2}\sum_{j\in I_{N}\fatsemi p\in
I_{M}}\int_{0}^{t}\big{(}f^{p,j}_{s^{(m)}}-f^{p,j}_{s^{(n)}}\big{)}^{2}\sigma_{s}^{2}ds$
(173)

By definition of the set $\mathcal{U}_{a}$, if
$\hat{\mu}^{N}\in\mathcal{U}_{a}$, then for any $\delta>0$, one can find
$m_{\delta}$ such that as long as $m,n\geq m_{\delta}$, necessarily

$(QV)^{(m,n),N}_{T}\leq N^{-1}\delta.$

Then writing $w(\cdot)$ to be a standard Brownian Motion, using the Dambin-
Dubins-Schwarz [27] time-rescaled representation of the stochastic integral,
as long as $(m,n)\geq m_{\delta}$,

$\displaystyle\mathbb{P}\bigg{(}\hat{\mu}^{N}\in\mathcal{U}_{a}\;,\;\bigg{|}\int_{0}^{T}\sum_{p\in
I_{M}}(f^{p}_{t^{(m)}}-f^{p}_{t^{(n)}})dz_{t}^{p}\bigg{|}\geq\frac{\epsilon}{2}\bigg{)}\leq$
$\displaystyle\mathbb{P}\big{(}\big{|}w\big{(}N^{-1}\delta\big{)}\big{|}\geq\epsilon\big{)}$
(174) $\displaystyle=$
$\displaystyle\exp\big{(}-N\epsilon^{2}/(8\delta)\big{)}\leq\exp(-NL),$ (175)

as long as we choose $\delta$ sufficiently small.

The other terms in

$\Gamma^{(m)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}-\Gamma^{(n)}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\big{)}$

are treated similarly (observe that they are just Riemann Sums, so its
straightforward to control their difference from the limiting integral).

∎

We now prove Theorems 10 and 12.

###### Proof.

In the case of connectivity-independent initial conditions (Case 2 of the
Assumptions), the theorem follows from Corollary 11. Since the relative
entropy is only zero when its two arguments are identical, any zero must be a
fixed point of the operator $\Phi$. It is proved in the following Lemma that
there is a unique zero.

For the rest of this proof, we prove the theorem in the case of connectivity-
dependent initial conditions. We start by proving that for any $\epsilon>0$,
there must exist a measure $\mu\in\mathcal{U}$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu)\leq\epsilon\big{)}=0.$
(176)

Write $\mathfrak{U}=\mathcal{U}_{a}$, where $a$ is large enough that

$\underset{N\to\infty}{\overline{\lim}}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
Q^{N}_{\mathbf{z}_{0},\mathbf{g}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y},\tilde{\mathbf{G}})\in\mathcal{U}_{a}\big{)}<-C.$

where $C$ is the upperbound for $\Gamma$ in Lemma 21. This is possible thanks
to the Exponential Tightness. Thanks to the Radon-Nikodym derivative identity
in (160), we thus find that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\notin\mathfrak{U}\big{)}<0.$
(177)

Thus for (176) to hold, it suffices that we prove that there exists
$\mu\in\mathfrak{U}$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G})\in\mathfrak{U}\;,\;d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu)\leq\epsilon\big{)}=0.$
(178)

Since $\mathfrak{U}$ is compact, for any $\epsilon>0$, we can obtain a finite
covering of $\mathfrak{U}$ of the form

$\displaystyle\mathfrak{U}\subseteq\bigcup_{i=1}^{\mathcal{N}_{\epsilon}}B_{\epsilon}(\mu_{i}),$
(179)

where $\mu_{i}\in\mathfrak{U}$. By a union of events bound,

$\displaystyle 0=$
$\displaystyle\lim_{N\to\infty}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\in\mathfrak{U}\big{)}$
(180) $\displaystyle\leq$ $\displaystyle\max_{1\leq
i\leq\mathcal{N}_{\epsilon}}\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{z},\tilde{\mathbf{G}})\in
B_{\epsilon}(\mu_{i})\big{)}\bigg{\\}}$ (181)

If our proposition in (178) were to be false, then (181) would be strictly
negative, which would be a contradiction.

Write $\mu_{(k)}\in\mathfrak{U}$ to be such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}(\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu_{(k)})\geq
k^{-1}\big{)}=0.$ (182)

Let $\mu\in\mathfrak{U}$ be any measure such that for some subsequence
$(p_{k})_{k\geq 1}$, $\lim_{k\to\infty}\mu_{(p_{k})}=\mu$ (this must be
possible because $\mathfrak{U}$ is compact).

We next claim that

$\displaystyle\lim_{\epsilon\to
0^{+}}\underset{N\to\infty}{\underline{\lim}}N^{-1}\log\mathbb{P}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}=-\mathcal{I}(\mu)+\Gamma(\mu)$
(183)

Indeed writing
$\mathcal{A}_{\epsilon}=\big{\\{}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{\\}}$,

$\displaystyle\mathbb{P}\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}=$
$\displaystyle\mathbb{E}^{\gamma}\bigg{[}\int_{\mathbb{R}^{MN}}P^{N}_{\mathbf{J},\mathbf{x}}(\mathcal{A}_{\epsilon})\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x}\bigg{]}$
(184) $\displaystyle=$
$\displaystyle\mathbb{E}^{\gamma}\bigg{[}\int_{\mathbb{R}^{MN}}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\\{\mathcal{A}_{\epsilon}\\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})d\mathbf{x}\bigg{]}$
(185) $\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\\{\mathcal{A}_{\epsilon}\\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})\bigg{]}d\mathbf{x}$
(186) $\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{\gamma}\bigg{[}\mathbb{E}^{P^{N}_{\mathbf{x}}}\big{[}\exp\big{(}N\Gamma(\hat{\mu}^{N})\big{)}\chi\\{\mathcal{A}_{\epsilon}\\}\big{]}\rho^{N}_{\mathbf{J}}(\mathbf{x})\;\bigg{|}\;\mathbf{G}_{0}\bigg{]}\bigg{]}d\mathbf{x}$
(187)

and in this last step we first perform the conditional expectation, for
$\gamma$ conditioned on the values of $\\{G^{p,j}_{0}\\}_{j\in I_{N}\fatsemi
p\in I_{M}}$.

Now, recall that

$\rho^{N}_{\mathbf{J}}(\mathbf{z}_{0})=(Z^{N}_{\mathbf{J}})^{-1}\chi\big{\\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in
B_{\delta_{N}}(\kappa)\big{\\}}.$

Furthermore, writing

$u_{N}=N^{-1}\log\mathbb{E}[Z^{N}_{\mathbf{J}}],$

our assumption on the initial condition dictates that for any $\delta>0$,

$\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\big{|}N^{-1}\log
Z^{N}_{\mathbf{J}}-u_{N}\big{|}\geq\delta\big{)}<0.$ (188)

Next, we claim that

$\displaystyle\lim_{\epsilon\to
0^{+}}\inf_{\nu\in\mathfrak{U}\cap\mathcal{A}_{\epsilon}}\Gamma(\nu)=\Gamma(\mu).$
(189)

Indeed (189) is a consequence of Lemma 19: this Lemma implies that $\Gamma$
can be approximated arbitrarily well by continuous functions over
$\mathfrak{U}$.

We thus obtain that

$\displaystyle\lim_{\epsilon\to
0^{+}}\underset{N\to\infty}{\underline{\lim}}\inf_{(\mathbf{z}_{0},\mathbf{G}_{0})}N^{-1}\log\big{(}d_{W}\big{(}\hat{\mu}^{N}(\mathbf{z},\mathbf{G}),\mu\big{)}\leq\epsilon\big{)}$
(190) $\displaystyle=$ $\displaystyle\Gamma(\mu)+\lim_{\epsilon\to
0^{+}}\underset{N\to\infty}{\underline{\lim}}\big{\\{}-u_{N}+N^{-1}\log\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\big{[}Q^{N}_{\mathbf{x},\mathbf{G}_{0}}\big{(}\mathcal{A}_{\epsilon}\big{)}\big{]}\chi\big{\\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in
B_{\delta_{N}}(\kappa)\big{\\}}d\mathbf{x}\big{\\}}$ (191) $\displaystyle=$
$\displaystyle\Gamma(\mu)-\lim_{\epsilon\to
0^{+}}\inf_{\nu\in\mathcal{A}_{(\epsilon)}}\mathcal{I}(\nu),$ (192)

since (by definition)

$N^{-1}\log\int_{\mathbb{R}^{MN}}\mathbb{E}^{\gamma}\big{[}\chi\big{\\{}\hat{\mu}^{N}(\mathbf{z}_{0},\mathbf{G}_{0})\in
B_{\delta_{N}}(\kappa)\big{]}d\mathbf{z}_{0}=u_{N},$

and we have employed the uniform lower bound in (36). The lower semi-
continuity of $\mathcal{I}$ implies that

$\lim_{\epsilon\to
0^{+}}\inf_{\nu\in\mathcal{A}_{(\epsilon)}}\mathcal{I}(\nu)=\mathcal{I}(\mu).$

We thus obtain (183), as required.

Next, we must show that $\mathcal{J}(\mu)=\mathcal{I}(\mu)-\Gamma(\mu)$
(recall the definition of $\mathcal{J}(\mu)$ in (56). Now

$\displaystyle\frac{dS_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}}{dQ_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}}\bigg{|}_{\mathcal{F}_{T}}=\exp\bigg{(}\sum_{p\in
I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}g^{p}_{s}-\tau^{-1}z^{p}_{s}\big{)}dz^{p}_{s}-\frac{1}{2}\sigma_{s}^{-2}\big{(}g^{p}_{s}-\tau^{-1}z^{p}_{s}\big{)}^{2}ds\bigg{)}$
(193)

Substituting this identity into the proposed rate function definition in (56),

$\displaystyle\mathbb{E}^{\kappa}\bigg{[}\mathcal{R}\big{(}\mu_{\mathbf{z}_{0},\mathbf{g}_{0}}||S_{\mu,\mathbf{z}_{0},\mathbf{g}_{0}}\big{)}\bigg{]}=\mathcal{I}(\mu)-\Gamma(\mu),$
(194)

as required.

The above reasoning dictates that there must be at least one $\xi$ such that
$\mathcal{J}(\xi)=0$. In fact, it is proved in Lemma 22 that there can only be
one measure $\xi$ such that $\mathcal{J}(\xi)=0$. Furthermore it follows from
Lemma 22, over small enough time increments, the mapping $\Phi_{t}$ must be a
contraction. This implies (66). ∎

######  Lemma 21.

There exists a constant $C>0$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C\big{)}<0.$ (195)

###### Proof.

For any $\ell>0$,

$\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\bigg{(}\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C\bigg{)}\leq\max\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|>\ell\big{)},\\\
\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell,\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C\big{)}\bigg{\\}}$ (196)

Thanks to Lemma 13,
$\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|>\ell\big{)}$
converges to $-\infty$ as $\ell\to\infty$. It thus suffices that we prove
that, for abitrary $\ell>0$, there exists $C_{\ell}>0$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell,\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C_{\ell}\big{)}<0.$ (197)

Now, leaving out the negative-semi-definite terms, we find that

$\displaystyle\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\leq N^{-1}\sum_{j\in
I_{N}\fatsemi p\in
I_{M}}\int_{0}^{T}\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}dy^{p,j}_{s}$
(198)

Furthermore, writing
$h^{p}_{s}=\sigma_{s}^{-2}\big{(}\tilde{G}^{p,j}_{s}-\tau^{-1}y^{p,j}_{s}\big{)}$,
and assuming that $\left\|\mathcal{J}_{N}\right\|\leq\ell$, one finds that

$\displaystyle\sum_{j\in I_{N}}(h^{p,j}_{s})^{2}\leq$ $\displaystyle
2\sigma_{s}^{-4}\sum_{j\in
I_{N}}\big{\\{}(\tilde{G}^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\\}}$
(199) $\displaystyle\leq$ $\displaystyle 2\sigma_{s}^{-4}\sum_{j\in
I_{N}}\big{\\{}\ell\lambda(y^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\\}}$
(200) $\displaystyle\leq$ $\displaystyle 2\sigma_{s}^{-4}\sum_{j\in
I_{N}}\big{\\{}\ell
C_{\lambda}^{2}(y^{p,j}_{s})^{2}+\tau^{-2}(y^{p,j}_{s})^{2}\big{\\}}.$ (201)

We thus find that, thanks to Lemma , for any $L>0$ there exists a constant
$\bar{C}_{L}>0$ such that

$\displaystyle\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\sup_{p\in
I_{M}}\sum_{j\in I_{N}}(h^{p,j}_{s})^{2}\geq N\bar{C}_{L}\big{)}\leq-L.$ (202)

Write

$\mathcal{H}_{N}=\bigg{\\{}\sup_{p\in I_{M}}\sum_{j\in
I_{N}}(h^{p,j}_{s})^{2}\leq N\bar{C}_{L}\bigg{\\}}.$

We thus find that,

$\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C_{\ell}\big{)}\\\
\leq\max\bigg{\\{}\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\mathcal{H}_{N}^{c}\big{)},\\\
\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C_{\ell}\big{)}\bigg{\\}}\\\
\leq\max\bigg{\\{}-L,\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C_{\ell}\big{)}\bigg{\\}}$ (203)

Furthermore, using the Dambins-Dubins Schwarz Theorem [27], and writing $w(t)$
to be 1D Brownian Motion,

$\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\left\|\mathcal{J}_{N}\right\|\leq\ell\;,\;\mathcal{H}_{N}\;,\;\Gamma^{N}_{\mathbf{J},T}(\mathbf{z})\geq
C_{\ell}\big{)}\\\
\leq\underset{N\to\infty}{\overline{\lim}}N^{-1}\log\mathbb{P}\big{(}\sup_{s\in[0,T]}\big{|}w\big{(}\bar{C}N^{-1}s\big{)}\big{|}\geq
C_{\ell}\big{)}\leq-L,$ (204)

as long as $C_{\ell}$ is sufficiently large, using standard properties of
Brownian Motion. ∎

We now prove that the rate function $\mathcal{J}$ has a unique minimizer (i.e.
we prove Lemma 9).

######  Lemma 22.

There exists a unique fixed point $\xi$ of $\Phi$ in $\mathcal{U}$.
Furthermore $\xi$ is such that for any $\mu\in\mathcal{U}$, writing
$\xi_{(1)}=\mu$ and $\xi_{(n+1)}=\Phi(\xi_{(n)})$, it holds that

$\displaystyle\xi=\lim_{n\to\infty}\xi_{(n)}$ (205)

###### Proof.

We start by considering the following restricted map
$\tilde{\Phi}:\mathfrak{Q}\to\mathfrak{Q}$. This is such that
$\tilde{\Phi}(\mu)=\nu^{(1)}$, where $\nu=\Phi(\alpha)$ for any
$\alpha\in\mathcal{U}$ such that $\alpha^{(1)}=\mu$. Define
$d^{(2)}_{t}:\mathfrak{Q}\times\mathfrak{Q}\to\mathbb{R}^{+}$ analogously.

We are going to demonstrate that there is a constant $c>0$ such that for all
$\mu,\nu\in\mathfrak{Q}$,

$\displaystyle
d^{(2)}_{t}\big{(}\tilde{\Phi}_{t}(\mu),\tilde{\Phi}_{t}(\nu)\big{)}\leq
c\sqrt{t}d^{(2)}_{t}(\mu,\nu).$ (206)

For any $\mu,\nu\in\mathfrak{Q}$, we construct a particular $\zeta$ that is
within $\eta\ll 1$ of realizing the infimum in the definition of the
Wasserstein distance in (98). To do this, we employ the construction of Lemma
16. Let $\mathbf{G}^{\mu},\mathbf{G}^{\nu}$ be
$\mathcal{C}([0,T],\mathbb{R}^{M})$-valued random variables (in the same
probability space), with joint probability law $\beta_{\mu,\nu}$. Then for
Brownian motions $\big{(}W^{p}_{t}\big{)}_{p\in I_{M}}$, define

$\displaystyle dz^{\nu,p}_{t}$
$\displaystyle=\big{(}-\tau^{-1}z^{\nu,p}_{t}+G^{\nu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}$
(207) $\displaystyle dz^{\mu,p}_{t}$
$\displaystyle=\big{(}-\tau^{-1}z^{\mu,p}_{t}+G^{\mu,p}_{t}\big{)}dt+\sigma_{t}dW^{p}_{t}.$
(208)

The initial conditions are identical: $z^{\nu,p}_{0}=z^{\mu,p}_{0}$. We
immediately see that

$\displaystyle\frac{d}{dt}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}=-\tau^{-1}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}+G^{\nu,p}_{t}-G^{\mu,p}_{t},$
(209)

and hence

$\displaystyle\frac{d}{dt}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}$
$\displaystyle=-2\tau^{-1}\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}+2\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}\big{(}G^{\nu,p}_{t}-G^{\mu,p}_{t}\big{)}\text{
and thus }$ (210) $\displaystyle\big{(}z^{\nu,p}_{t}-z^{\mu,p}_{t}\big{)}^{2}$
$\displaystyle=\int_{0}^{t}\exp\big{(}2(s-t)/\tau\big{)}2\big{(}z^{\nu,p}_{s}-z^{\mu,p}_{s}\big{)}\big{(}G^{\nu,p}_{s}-G^{\mu,p}_{s}\big{)}ds.$
(211)

It follows from this that there exists a constant $c>0$ such that for all
$t\in[0,T]$,

$\displaystyle
d^{(2)}_{t}\big{(}\tilde{\Phi}_{t}(\mu),\tilde{\Phi}_{t}(\nu)\big{)}$
$\displaystyle\leq ctd^{(2)}_{t}\big{(}\beta_{\mu},\beta_{\nu}\big{)}$ (212)
$\displaystyle\leq cC_{\lambda}td^{(2)}_{t}(\mu,\nu),$ (213)

using Lemma 16. Thus for small enough $t$, there is a unique fixed point of
$\tilde{\Phi}_{t}$ (the mapping upto time $t$). Iterating this argument, we
find a unique fixed point for $\tilde{\Phi}$. The uniqueness for
$\tilde{\Phi}$ in turn implies uniqueness for $\Phi$, thanks to the identity
in Lemma 16.

To see why (205) holds. First consider arbitrary $\nu_{(1)}\in\mathfrak{Q}$,
and define $\nu_{(n+1)}=\tilde{\Phi}(\nu_{(n)})$. The above bound in (213)
implies that necessarily $\big{(}\nu_{(n)}\big{)}_{n\geq 1}$ is Cauchy. It
then immediate follows that for any $\xi_{(1)}\in\mathcal{U}$ with first
marginal equal to $\nu_{(1)}$, and writing $\xi_{(n+1)}=\Phi(\xi_{(n)})$, it
must be that $\big{(}\xi_{(n)}\big{)}_{n\geq 1}$ is Cauchy.

Finally we note that $d^{(2)}$ metrizes weak convergence, thanks to Lemma 15.

∎

##  Appendix A Bounding Fluctuations of the Noise

For the processes $(\mathbf{y}^{j}_{[0,T]})_{j\in I_{N}}$ that are defined in
(31), define the empirical measure

$\displaystyle\hat{\mu}^{N}(\mathbf{y})=N^{-1}\sum_{j\in
I_{N}}\delta_{\mathbf{y}^{j}_{[0,T]}}\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}.$
(214)

Next, we bound the probability of the empirical being in the set
$\mathfrak{Q}_{a}$, defined in (38), which we recall

$\mathcal{Q}_{\mathfrak{a}}=\bigg{\\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\sup_{m\geq\mathfrak{a}}\sup_{0\leq
i\leq m}\mathbb{E}^{\mu}\big{[}\sup_{M\in
I_{M}}(w^{p}_{t^{(m)}_{i+1}}-w^{p}_{t^{(m)}_{i}})^{2}\big{]}>\Delta_{m}^{1/4}\text{
and }\\\ \mu\in\mathcal{K}_{\mathfrak{a}}\text{ and }\sup_{p\in
I_{M}}\mathbb{E}^{\mu}[\sup_{t\in[0,T]}(y^{p}_{t})^{2}\big{]}\leq\mathfrak{a}\bigg{\\}}$
(215)

where $\Delta_{m}=T/m$ and $t^{(m)}_{i}=iT/m$. The main result of this section
is the following.

######  Lemma 23.

For any $L>0$, there exists $\mathfrak{a}\in\mathbb{Z}^{+}$ such that for all
$N\geq 1$,

$\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})}N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{\mathfrak{a}}\big{)}\leq-L.$
(216)

###### Proof.

Employing a union-of-events bound, or any
$(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$,

$N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{Q}_{\mathfrak{a}}\big{)}\leq
N^{-1}\log\bigg{\\{}P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in
I_{M}}N^{-1}\sum_{j\in
I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}\\\
+P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{0\leq t\leq\Delta_{m}}\sum_{j\in
I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in
I_{M}}\big{|}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{|}^{2}\geq
Na\Delta_{m}\bigg{)}+P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{\mathfrak{a}}\big{)}\bigg{\\}}.$
(217)

With a view to bounding the first term on the RHS, since
$y^{p,j}_{0}=z^{p,j}_{0}$,

$(y^{p,j}_{t})^{2}\leq
2\big{(}y^{p,j}_{t}-y^{p,j}_{0}\big{)}^{2}+2(z^{p,j}_{0})^{2}.$

Thus for a positive constant $b>0$,

$\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}b\sup_{p\in
I_{M}}\sum_{j\in I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}\bigg{)}\bigg{]}\\\
\leq\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}2b\sup_{p\in
I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}+2b\sum_{j\in
I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t}-z^{p,j}_{0})^{2}\bigg{)}\bigg{]}.$ (218)

Thus, thanks to Chernoff’s Inequality,

$\displaystyle N^{-1}\log P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in
I_{M}}N^{-1}\sum_{j\in
I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}\leq\frac{2b}{N}\sup_{p\in
I_{M}}\sum_{j\in I_{N}}(z^{p,j}_{0})^{2}$ (219)
$\displaystyle+N^{-1}\log\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}2b\sum_{j\in
I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t}-z^{p,j}_{0})^{2}\bigg{)}\bigg{]}-b\mathfrak{a}.$
(220)

The first term on the RHS is bounded for all $N$ and all
$(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$. For the second term on
the RHS, standard theory on stochastic processes implies that the exponential
moment exists, as long as $b$ is small enough. Thus, taking
$\mathfrak{a}\to\infty$, the RHS can be made arbitrarily small. We thus find
that

$\displaystyle\lim_{\mathfrak{a}\to\infty}\sup_{N\geq
1}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{p\in I_{M}}N^{-1}\sum_{j\in
I_{N}}\sup_{t\in[0,T]}(y^{p,j}_{t})^{2}>\mathfrak{a}\bigg{)}=-\infty.$ (221)

The Lemma now follows from applying (221), Lemma 25 and Lemma 24 to (217). ∎

The following result is well-known. Nevertheless we sketch a quick proof for
clarity.

######  Lemma 24.

For any $L>0$, there exists a compact set $\mathcal{K}_{L}$ such that for all
$N\geq 1$,

$\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}\leq-L$
(222)

###### Proof.

The following property follows straightforwardly from properties of the
stochastic integral (noting that the diffusion coefficient is uniformly
bounded): for any $\epsilon>0$, there exists a compact set
$\mathcal{C}_{\epsilon}\subset\mathcal{C}([0,T],\mathbb{R}^{M})$ such that for
all $j\in I_{N}$ such that $\|z_{0}^{j}\|\leq\epsilon^{-1}$,

$\displaystyle\sup_{j\in
I_{N}}P^{N}_{\mathbf{z}_{0}}\big{(}y^{j}_{[0,T]}\notin\mathcal{C}_{\epsilon}\big{)}\leq\epsilon.$
(223)

Write

$\displaystyle
u^{N}_{\epsilon}=\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\sum_{j\in
I_{N}}\chi\\{\|\mathbf{y}^{j}_{0}\|\geq\epsilon^{-1}\\},$ (224)

and note that our assumptions on $\mathcal{Y}^{N}$ dictates that

$\displaystyle\lim_{\epsilon\to 0^{+}}\lim_{N\to\infty}u_{\epsilon}^{N}=0.$
(225)

For any $m\in\mathbb{Z}^{+}$, define the set
$\mathcal{L}_{m,\delta}\subset\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}$
to be such that

$\displaystyle\mathcal{L}_{m,\delta}=\big{\\{}\mu\in\mathcal{P}\big{(}\mathcal{C}([0,T],\mathbb{R}^{M})\big{)}\;:\;\mu(\mathcal{C}_{m^{-1}})\geq\delta\big{)}\big{\\}}$
(226)

We claim that for any $m\geq 1$, there exists $\delta_{m}>0$ such that

$\displaystyle\sup_{N\geq
1}\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}\big{(}\mathbf{y}\big{)}\notin\mathcal{L}_{m,\delta_{m}}\big{)}\leq-m$
(227)

To see this, employing a Chernoff Inequality, for a constant $b>0$, for any
$(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}$,

$\displaystyle N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}\big{(}\mathbf{y}\big{)}\notin\mathcal{L}_{m,\delta}\big{)}\leq$
$\displaystyle\mathbb{E}^{P^{N}_{\mathbf{z}_{0}}}\bigg{[}\exp\bigg{(}b\sum_{j\in
I_{N}}\chi\\{\mathbf{y}^{j}_{[0,T]}\notin\mathcal{C}_{m^{-1}}\\}-Nb\delta\bigg{)}\bigg{]}$
(228) $\displaystyle\leq$
$\displaystyle-b\delta+N^{-1}\log\big{\\{}(\epsilon+u^{N}_{\epsilon})\big{(}\exp(b)-1\big{)}+1\big{\\}}^{N}$
(229) $\displaystyle=$
$\displaystyle-b\delta+\log\big{\\{}(\epsilon+u^{N}_{\epsilon})\big{(}\exp(b)-1\big{)}+1\big{\\}}$
(230)

Taking $\epsilon$ to be sufficiently small, and $b$ sufficiently large, we
obtain (227).

Now, for an integer $m_{L}$ to be specified further below, define
$\mathcal{K}_{L}=\bigcap_{m\geq m_{L}}\mathcal{L}_{m,\delta_{m}}$. Prokhorov’s
Theorem implies that $\mathcal{K}_{L}$ is compact. Employing a union-of-events
bound, we obtain that

$\displaystyle
P^{N}_{\mathbf{z}_{0}}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}$
$\displaystyle\leq\sum_{m\geq m_{L}}\exp(-mN)$ (231)
$\displaystyle\leq\exp(-m_{L}N)\sup_{n\geq 1}\sum_{j=0}^{\infty}\exp(-jN).$
(232)

We thus find that, for large enough $m_{L}$,

$\sup_{N\geq
1}N^{-1}\log\mathbb{P}\big{(}\hat{\mu}^{N}(\mathbf{y})\notin\mathcal{K}_{L}\big{)}\leq-L,$

as required. ∎

######  Lemma 25.

There exists a constant $\mathfrak{C}$ such that for any positive integer $m$
and any $a>0$, writing $\Delta_{m}=Tm^{-1}$ and $t^{(m)}_{i}=Ti/m$, for any
$N\geq 1$,

$\displaystyle\sup_{(\mathbf{z}_{0},\mathbf{g}_{0})\in\mathcal{Y}^{N}}N^{-1}\log
P^{N}_{\mathbf{z}_{0}}\bigg{(}\sup_{0\leq t\leq\Delta_{m}}\sum_{j\in
I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in
I_{M}}\big{|}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{|}^{2}\geq
Na\Delta_{m}\bigg{)}\leq\mathfrak{C}+\log m-\frac{a}{4}$ (233)

###### Proof.

Define, for $t\in[0,\Delta_{m})$,

$f^{N}_{t}=\sum_{j\in I_{N}}\sup_{0\leq i\leq m-1}\sup_{p\in
I_{M}}\big{(}y^{p,j}_{t+t^{(m)}_{i}}-y^{p,j}_{t^{(m)}_{i}}\big{)}^{2}$

Notice that $t\to f^{N}_{t}$ is a submartingale. Thus, writing
$a=(4\Delta_{m})^{-1}$, $\exp\big{(}af^{N}_{t}\big{)}$ is a submartingale.
Therefore, thanks to Doob’s Submartingale Inequality,

$\displaystyle\mathbb{P}\big{(}f^{N}_{t}\geq Nx\big{)}$
$\displaystyle\leq\mathbb{E}\bigg{[}\exp\big{(}af^{N}_{T}-aNx\big{)}\bigg{]}$
(234)
$\displaystyle\leq\big{\\{}mM\big{(}1-2\Delta_{m}\bar{\sigma}a\big{)}^{-1/2}\big{\\}}^{N}\exp(-aNx)$
(235)
$\displaystyle=\big{\\{}mM2^{1/2}\big{\\}}^{N}\exp\big{(}-Nx/(4\Delta_{m})\big{)}$
(236)

∎

## References

  * [1] Robert Adler and Jonathan Taylor.  Random Fields and Geometry, volume 53.  Springer, 2019.
  * [2] Ahmed El Alaoui, Andrea Montanari, and Mark Sellke.  Optimization of mean-field spin glasses.  Arxiv Preprint, 2020.
  * [3] G Ben Arous and A Guionnet.  Large deviations for langevin spin glass dynamics.  Probability Theory and Related Fields, 102, 1995.
  * [4] Gerard Ben Arous, Amir Dembo, and Alice Guionnet.  Cugliandolo-kurchan equations for dynamics of spin-glasses.  Probability Theory and Related Fields, 136:619–660, 2006.
  * [5] Gerard Ben Arous and Alice Guionnet.  Symmetric langevin spin glass dynamics.  The Annals of Probability, 25:1367–1422, 1997.
  * [6] Gerard Ben Arous, Song Mei, Andrea Montanari, and Mihai Nica.  The landscape of the spiked tensor model.  Communications on Pure and Applied Mathematics, 72:2282–2330, 11 2019.
  * [7] Nicolas Brunel.  Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons.  Journal of Computational Neuroscience, 8:183–208, 2000.
  * [8] Nicolas Brunel and Xiao-Jing Wang.  What determines the frequency of fast network oscillations with irregular neural discharges? i. synaptic dynamics and excitation-inhibition balance.  Journal of Neurophysiology, 90:415–430, 2003.
  * [9] Amarjit Budhiraja and Paul Dupuis.  Analysis and Approximation of Rare Events, volume 94.  Springer, 2019.
  * [10] Tanguy Cabana and Jonathan D. Touboul.  Large deviations for randomly connected neural networks: I. spatially extended systems.  Advances in Applied Probability, 50:983–1004, 2018.
  * [11] Tanguy Cabana and Jonathan D. Touboul.  Large deviations for randomly connected neural networks: Ii. state-dependent interactions.  Advances in Applied Probability, 50:983–1004, 2018.
  * [12] B. Cessac.  Linear response in neuronal networks: from neurons dynamics to collective response.  5 2019.
  * [13] Patrick Charbonneau, Enzo Marinari, Mark Mezard, Giorgio Parisi, Federico Ricci-Tersenghi, Gabriella Sicuro, and Francesco Zamponi, editors.  Spin Glass Theory and Far Beyond.  World Scientific, 2023.
  * [14] A. Crisanti and H. Sompolinsky.  Path integral approach to random neural networks.  Physical Review E, 98:1–20, 2018.
  * [15] Andrea Crisanti, Heinz Horner, and H.J. Sommers.  The spherical p-spin interaction spin-glass model.  Zeitschrift fur Physik B Condensed Matter, 92:257–271, 1993.
  * [16] L. F. Cugliandolo and J. Kurchan.  Analytical solution of the off-equilibrium dynamics of a long-range spin-glass model.  Physical Review Letters, 71:173–176, 1993.
  * [17] Amir Dembo and Eliran Subag.  Dynamics for spherical spin glasses: Disorder dependent initial conditions.  Journal of Statistical Physics, 181:465–514, 2020.
  * [18] Amir Dembo and Ofer Zeitouni.  Large Deviations Techniques and Applications 2nd Edition.  Springer, 1998.
  * [19] Diego Fasoli and Stefano Panzeri.  Stationary-state statistics of a binary neural network model with quenched disorder.  Entropy, pages 1–30, 2019.
  * [20] Olivier Faugeras and James MacLaurin.  Asymptotic description of neural networks with correlated synaptic weights.  Entropy, 17:4701–4743, 2015.
  * [21] Olivier Faugeras, Emilie Soret, and Etienne TanrÃ©.  Asymptotic behaviour of a network of neurons with random linear interactions.  Preprint HAL Id : hal-01986927, 2019.
  * [22] David Gamarnik.  The overlap gap property: A topological barrier to optimizing over random structures.  Proceedings of the National Academy of Scientists, 2021.
  * [23] M Grunwald.  Sanov results for glauber spin-glass dynamics.  Probability Theory and Related Fields, 106:187–232, 1996.
  * [24] A Guionnet.  Averaged and quenched propagation of chaos for spin glass dynamics.  Probability Theory and Related Fields, 109:183–215, 1997.
  * [25] Alice Guionnet and Boguslaw Zegarlinski.  Decay to equilibrium in random spin systems on a lattice.  Journal of Statistical Physics, 86:899–904, 1997.
  * [26] Moritz Helias and David Dahmen.  Statistical Field Theory for Neural Networks.  Springer, 2020.
  * [27] Ioannis Karatzas and Steven Shreve.  Brownian motion and stochastic calculus 2nd edition, 1991.
  * [28] Itamar Daniel Landau and Haim Sompolinsky.  Coherent chaos in a recurrent neural network with structured connectivity.  PLoS Computational Biology, 14:1–27, 2018.
  * [29] George Lindgren, Holger Rootzen, and Maria Sandsten.  Stationary Stochastic Processes for Scientists and Engineers.  Chapman Hall, 2013.
  * [30] Eric Lucon.  Quenched large deviations for interacting diffusions in random media.  Journal of Statistical Physics, 166:1405–1440, 2017.
  * [31] Olivier Moynot and Manuel Samuelides.  Large deviations and mean-field theory for asymmetric random recurrent neural networks.  Probability Theory and Related Fields, 123:41–75, 5 2002.
  * [32] Gabriel Koch Ocker, Yu Hu, Michael A. Buice, Brent Doiron, Kresimir Josic, Robert Rosenbaum, and Eric Shea-Brown.  From the statistics of connectivity to the statistics of spike times in neuronal networks.  Current Opinion in Neurobiology, 46:109–119, 2017.
  * [33] Robert Rosenbaum and Brent Doiron.  Balanced networks of spiking neurons with spatially dependent recurrent connections.  Physical Review X, pages 1–9, 2014.
  * [34] Robert Rosenbaum, Matthew A. Smith, Adam Kohn, Jonathan E. Rubin, and Brent Doiron.  The spatial structure of correlated neuronal variability.  Nature Neuroscience, 20:107–114, 2017.
  * [35] Kai Segadlo, Bastian Epping, Alexander Van Meegen, David Dahmen, Michael KrÃ¤mer, and Moritz Helias.  Unified field theoretical approach to deep and recurrent neuronal networks.  Journal of Statistical Mechanics: Theory and Experiment, 2022, 10 2022.
  * [36] H. Sompolinsky, A. Crisanti, and H. J. Sommers.  Chaos in random neural networks.  Physical Review Letters, 61:259–262, 1988.