Hong-Bin Chen
[Hong-Bin Chen]Institut des Hautes Études Scientifiques, France

Jean-Christophe Mourrat
[Jean-Christophe Mourrat]Department of Mathematics, ENS Lyon and CNRS, Lyon, France

[2010]

On the free energy of vector spin glasses with non-convex interactions

The limit free energy of spin-glass models with convex interactions can be represented as a variational problem involving an explicit functional. Models with non-convex interactions are much less well-understood, and simple variational formulas involving the same functional are known to be invalid in general. We show here that a slightly weaker property of the limit free energy does extend to non-convex models. Indeed, under the assumption that the limit free energy exists, we show that this limit can always be represented as a critical value of the said functional. Up to a small perturbation of the parameters defining the model, we also show that any subsequential limit of the law of the overlap matrix is a critical point of this functional. We believe that these results capture the fundamental conclusions of the non-rigorous replica method.

§ INTRODUCTION

§.§ Informal summary of the main results

We fix an integer $D \ge 1$, and let $(H_N(\sigma))_{\sigma \in (\R^N)^\D}$ be a centered Gaussian field such that, for every $\sigma = (\sigma_1,\ldots, \sigma_D)$ and $\tau = (\tau_1, \ldots, \tau_D) \in (\R^N)^\D$, we have

\begin{equation}
\label{e.def.xi}
\E \Ll[ H_N(\sigma) H_N(\tau) \Rr] = N \xi \Ll( \frac{\si \tau^\intercal}{N} \Rr) ,
\end{equation}

where $\xi \in C^\infty(\R^{D\times D}; \R)$ is a smooth function admitting an absolutely convergent power-series expansion with $\xi(0) = 0$, and where $\sigma \tau^\intercal$ denotes the matrix of scalar products

\begin{equation}
\label{e.def.sigma.tau}
\sigma \tau^\intercal = (\sigma_d \cdot \tau_{d'})_{1 \le d,d' \le D}.
\end{equation}

The notation in (<ref>) is natural if we think of $\si$ and $\tau$ as $D$-by-$N$ matrices, and we often identify $(\R^N)^D$ with $\R^{D \times N}$. The class of functions $\xi$ such that there exists a Gaussian random field satisfying (<ref>), together with an explicit construction of the Gaussian process $H_N$, are given in Subsection <ref> below. We are mostly interested in models for which the function $\xi$ is non-convex. A representative example in this class is the bipartite model, which is obtained by choosing  $D = 2$ and $\xi((A_{d,d'})_{1 \le d,d' \le 2}) = A_{1,1} A_{2,2}$. This model can be realized explicitly by setting $H_N(\sigma)$ to be

\begin{equation*}  %\label{e.}
\frac 1 {\sqrt{N}} \sum_{i,j = 1}^N J_{i,j} \sigma_{1,i} \sigma_{2,j},
\end{equation*}

where $(J_{i,j})_{1 \le i,j \le N}$ are independent standard Gaussian random variables.

We also fix a probability measure $P_1$ with compact support, and for each integer $N \ge 1$, we denote by $P_N = P_1^{\otimes N}$ the $N$-fold tensor product of $P_1$. We think of $P_N$ as a probability measure on $\R^{D \times N} \simeq (\R^N)^D$.
We are interested in understanding, for each fixed $\beta \ge 0$, the large-$N$ limit of the free energy

\begin{equation*}
\frac 1 N \E \log \int \exp \Ll(\beta H_N(\si) \Rr) \, \d P_N(\si).
\end{equation*}

For $D = 1$, $\xi(r) = r^2$, and $P_1 = \tfrac 1 2 \de_{-1} + \tfrac 1 2  \de_{1}$, this corresponds to the Sherrington–Kirkpatrick model [94]. In this context, a candidate for the limit free energy was derived in [63, 84, 85, 86, 87] using sophisticated non-rigorous arguments within the framework of the replica method. In this approach, a number of formal manipulations are performed in order to be in a position to appeal to the Laplace principle for integrals of exponential functions, say of the form

\begin{equation}
\label{e.laplace}
\frac 1 N \log \int \exp(N \mathscr J(x)) \, \d x.
\end{equation}

The actual expression that shows up within the replica method also involves the number of replicas as a parameter. As this number, a positive integer, is sent to zero, unexpected things happen. In particular, the idea that the limit of (<ref>) ought to be the supremum of $\mathscr J$ is no longer valid; for the SK model and all other scalar models with Ising spins, the limit becomes the infimum of $\mathscr J$ instead. For more general models, even this “inverted variational principle” is no longer necessarily valid. To the best of our understanding, in general, the main takeaway of the replica method is that the limit free energy can be represented as a critical value of this function $\mathscr J$. By this, we mean that the limit can be represented in the form of $\mathscr J(x)$ for some $x$ such that $\nabla \mathscr J(x) = 0$. Crucially, one also expects that the critical point itself encodes the asymptotic law of the overlap between two independent replicas sampled according to the corresponding Gibbs measure.

The main results of this paper are rigorous versions of statements of this form, for general models and for a suitable choice of $\mathscr J$. Precisely, under the assumption that the free energy converges to some limit, we show that this limit must be a critical value of some explicit functional. Whether or not the free energy converges, we also show that, up to a small perturbation of the energy function $H_N$, any subsequential limit of the law of the overlap matrix is described by a critical point of the same functional.

A related route towards obtaining a fully unambiguous characterization of the limit free energy was explored in [68, 71]; see also [37]. The idea there is to consider an enriched version of the free energy that depends on more parameters, and aim to show that this enriched free energy is the solution to an explicit Hamilton–Jacobi equation.

We will also work with this enriched version of the free energy throughout this paper.

One possible approach to solving Hamilton–Jacobi equations is through the method of characteristics. In our context, the characteristics are straight lines, and formal calculations suggest that the gradient of the free energy ought to be constant along those lines. Using the equation then yields a prescription for the value of the solution itself along each of these lines. As long as the characteristic lines do not cross each other, one can show that defining a function according to this prescription yields a valid solution to the equation (see for instance <cit.> for more precision). In this perspective, our main result can be rephrased as saying that in fact, the limit free energy can always be represented as the value prescribed by one of these characteristic lines. The only remaining source of ambiguity is that for large times, or in other words at low temperature, there may be multiple characteristic lines reaching a point, and we only assert that one of these characteristic lines prescribes the correct value.

§.§ Precise statement of the main results

Let $\S^\D$ be the space of $D\times D$ real symmetric matrices, and $\S^\D_+$ be the subset of positive semi-definite matrices.

\begin{equation}
\label{e.def.mclQ}
\mcl Q := \Ll\{ q : [0,1) \to S^\D_+ \ : \ q \text{ is right-continuous with left limits, and is increasing} \Rr\} ,
\end{equation}

where “$q$ is increasing” means that, for every $u,v \in \Ll[0,1\Rr)$,

\begin{equation*}  %\label{e.}
u \le v \quad \implies \quad q(u) \le q(v),
\end{equation*}

and the latter inequality is interpreted as $q(v) - q(u) \in S^D_+$. For every $r \in [1,\infty]$, we set $\mcl Q_r := \mcl Q \cap L^r([0,1]; S^D)$. The enriched free energy will be a function defined over $\R_+ \times \mcl Q_1$.
Here and throughout, we set $\R_+=[0,\infty)$.
When $q \in \mcl Q$ is constant, i.e. $q = h$ with $h \in S^\D_+$, this enriched free energy is defined as

\begin{multline}
\label{e.def.simple.F}
\bar F_N(t,h)
\\
:= -\frac 1 N \E \log \int \exp \Ll( \sqrt{2t} H_N(\sigma) - t N \xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr) + \sqrt{2h} z \cdot \sigma - h \cdot \sigma \sigma^\intercal \Rr) \, \d P_N(\sigma),
\end{multline}

where $z = (z_1,\ldots, z_D)$ is a random element of $(\R^{N})^D \simeq \R^{D\times N}$ with independent standard Gaussian entries, independent of $(H_N(\sigma))_{\sigma \in (\R^N)^\D}$, and $\E$ denotes the expectation with respect to all randomness. In (<ref>) and throughout the paper, whenever $a$ and $b$ are two matrices of the same size, we denote by $a \cdot b = \tr(a b^\intercal)$ the entrywise scalar product, and by $|a| = (a \cdot a)^{1/2}$ the associated norm. The compensating term $t N \xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)$ is equal to half the variance of the Gaussian random variable that immediately precedes it in the expression (<ref>); and as a consequence, an application of Jensen's inequality yields that $\bar F_N$ is non-negative. This compensating term is helpful for the analysis, and we expect that it can be removed a posteriori using similar but simpler arguments as those presented here (in the sense that the final results are in terms of the critical points of a functional that has been enriched with a finite number of additional parameters). Here we will content ourselves with the observation that this term is constant for the bipartite model with $\pm 1$ spins, or more generally for any model with $\pm 1$ spins for which the function $\xi$ only depends on the diagonal of its argument, so its presence causes no harm in these cases. (We also explain how this compensating term can be removed in the case when $\xi$ is convex in Section <ref>.) The definition of $\bar F_N(t,q)$ for non-constant $q \in \mcl Q_1$ is more involved, as one needs to replace the “replica-symmetric” random magnetic field $\sqrt{2h} z$ appearing in (<ref>) by a more complicated one that is coupled with a Poisson–Dirichlet cascade; we refer to (<ref>) and Proposition <ref> for a precise definition. Even with general $q \in \mcl Q_1$, the structure of this term remains that of a “one-body” potential, in the sense that it does not feature any interaction between spins indexed by different indices in $\{1,\ldots, N\}$. In particular, for every $N \ge 1$, we have that

\begin{equation}
\label{e.def.psi}
\psi(q) := \bar F_1(0,q) = \bar F_N(0,q).
\end{equation}

For every $q,q' \in \mcl Q_2$, $p \in L^2([0,1];S^\D)$, and $t \ge 0$, we define

\begin{equation}
\label{e.def.Parisi.functional}
\mcl J_{t,q}(q',p) := \psi(q') +  \la p , q-q'\ra_\cH + t \int_0^1 \xi(p).
\end{equation}

In this expression, we use the shorthand notation $\int_0^1 \xi(p) = \int_0^1 \xi(p(u)) \, \d u$, and the inner product between $p$ and $q-q'$ is that of the space $L^2([0,1], S^\D)$, that is,

\begin{equation*}  %\label{e.}
\la p, q-q'\ra_\cH = \int_0^1 p(u) \cdot (q(u)-q'(u)) \, \d u.
\end{equation*}

The mapping in (<ref>) has a close relationship with the Hamilton–Jacobi equation

\begin{equation}
\label{e.hj}
\Ll\{
\begin{array}{ll}
\displaystyle{\dr_t f - \int_0^1 \xi(\partial_q f)  = 0} & \quad \text{ on } \R_+ \times \mcl Q_2, \\
\displaystyle{f(0,\cdot) = \psi} & \quad \text{ on } \mcl Q_2.
\end{array}
\Rr.
\end{equation}

A first indication of this fact is that, for each fixed $q'$ and $p$, the mapping $(t,q) \mapsto \mcl J_{t,q}(q',p)$ is an affine solution to the equation (<ref>) (with a different initial condition). More importantly, under convexity conditions on $\xi$ and $\psi$ respectively, the Hopf–Lax and Hopf formulas allow us to write the solution to (<ref>) evaluated at $(t,q)$ as a saddle-point problem for the functional $\mcl J_{t,q}$, as was shown in [29] (see also <cit.> for the case of equations in finite dimension). For these reasons, we call the mapping $\mcl J_{t,q}$ the Hamilton–Jacobi functional. While neither $\xi$ nor $\psi$ need to be convex nor concave in general, the conjecture of [68, 71] (see also <cit.>) is that $\bar F_N$ converges to the solution to (<ref>). The function $\mcl J_{t,q}$ will play the role of the function $\mathscr J$ discussed informally in the previous subsection. We say that the pair $(q',p) \in \mcl Q_2 \times L^2([0,1]; S^D)$ is a critical point of the mapping $\mcl J_{t,q}$ if

\begin{equation}
\label{e.crit.p}
q = q' - t \nabla \xi(p)
\end{equation}

\begin{equation}
\label{e.crit.q'}
p = \partial_q \psi(q').
\end{equation}

In more explicit notation, these identities state that, for almost every $u \in [0,1)$, we have

\begin{equation*}  %\label{e.}
q(u) = q'(u) - t \nabla \xi(p(u)) \qquad \text { and } \qquad p(u) = \partial_q \psi(q',u),
\end{equation*}

and $\partial_q \psi(q,\cdot)$ denotes the Gateaux derivative of $\psi$ at $q$, see (<ref>) for a precise definition.
The condition (<ref>) is the first-variation relation for the variable $p$ in the mapping $\mcl J_{t,q}$, while the condition (<ref>) is the same for the variable $q'$ (ignoring the constraint $q' \in \mcl Q_2$). Notice that inserting (<ref>) into (<ref>) yields that

\begin{equation}
\label{e.charact.line}
q = q' - t \nabla \xi(\dr_q \psi(q')).
\end{equation}

At least formally, the characteristic line associated with the equation (<ref>) and emanating from the point $q' \in \mcl Q_2$ is the trajectory

\begin{equation*}  %\label{e.}
t' \mapsto \big(t', q'-t'\nabla \xi(\dr_q \psi(q'))\big)
\end{equation*}

(cf. <cit.>).
The relation (<ref>) can therefore be rephrased as saying that the characteristic line emanating from $q'$ passes through the point $(t,q)$.

In order to clarify the relationship between the Hamilton–Jacobi functional and the Parisi formula, we first present a characterization of the limit free energy that is valid under the assumption that $\xi$ is convex over $S^D_+$. The limit free energy was first identified for a class of scalar models including the Sherrington–Kirkpatrick model in [49, 102]. The argument was then extended to general scalar models in [76, 79, 77], using an alternative approach based on the ultrametricity property of the Gibbs measure. This approach was then extended to models as in (<ref>) in [13, 80, 81, 82], under the assumption that the function $\xi$ is convex over $\R^{D\times D}$. Here we show that the more general assumption that $\xi$ is convex over $S^D_+$ is sufficient for this result to be valid. To generate examples of functions that satisfy the weaker condition but not the stronger one, we note that if $\xi_0 : \R^{D\times D} \to \R$ is convex over $\R^{D\times D}$, then any function of the form

\begin{equation*}  %\label{e.}
\xi : A = (A_{d,d'})_{d,d' \le 1} \mapsto  \xi_0(A) + \sum_{d = 1}^D \sum_{k = 1}^{+\infty} \beta_{d,k}^2 A_{d,d}^k,
\end{equation*}

with $(\beta_{d,k})_{d \le D, k \ge 1}$ decaying sufficiently rapidly to ensure the absolute summability of the series, is convex over $S^D_+$, but it is not necessarily convex over $\R^{D\times D}$.

If $\xi$ is a convex function over $S^\D_+$, then for every $t\ge 0$ and $q \in \mcl Q_2$, we have

\begin{equation}
\label{e.parisi.convex}
\lim_{N \to \infty} \bar F_N(t,q)= \sup_{q' \in q + \mcl Q_\infty} \inf_{p \in \mcl Q_\infty} \mcl J_{t,q}(q',p) = f(t,q) ,
\end{equation}

where $f$ is the solution to (<ref>).

The fact that the Parisi formula ($q = 0$) can be recast in terms of the Hamilton–Jacobi functional in the case $D = 1$ is from [69], and was extended to arbitrary $q$ in [72]. The fact that the solution to  the Hamilton–Jacobi equation (<ref>) with convex $\xi_{|S^D_+}$ admits a variational representation is from [29], and
the inequality

\begin{equation*}  %\label{e.}
\liminf_{N \to \infty} \bar F_N(t,q) \ge f(t,q)
\end{equation*}

is shown in [68, 71] for arbitrary $\xi$. The converse bound will follow from the results obtained below, which are more precise than necessary in this case.

To clarify further the connection between Theorem <ref> and the classical expression of the Parisi formula, when $\xi$ is convex over $S^\D_+$, we define its convex dual $\xi^*$ by setting, for every $a \in \R^{D \times D}$,

\begin{equation}
\label{e.def.xistar}
\xi^*(a) := \sup_{b \in S^\D_+} \Ll\{ a \cdot b - \xi(b)  \Rr\} .
\end{equation}

As will be shown here, one can rewrite the saddle-point problem in (<ref>) as

\begin{equation}
\label{e.parisi.convex.var}
\sup_{q' \in \mcl Q_\infty} \Ll\{ \psi(q + q') - t \int_0^1 \xi^* \Ll( \frac{q'}{t} \Rr)  \Rr\} ,
\end{equation}

which perhaps most closely resembles the classical finite-dimensional Hopf–Lax formula adapted to the equation in (<ref>).
For scalar models and $q = 0$, it was shown in [69] that one can recover the classical expression of the Parisi formula from the formula in (<ref>) via a change of variables.
This change of variables essentially consists in replacing the variable $q'$ with the variable $p$ defined so that $q' = t \nabla \xi(p)$, so that (<ref>) holds; see also <cit.> for more precision.

For possibly non-convex $\xi$, the first identity in (<ref>) is false in general [68]. Another plausible candidate saddle-point formula is

\begin{equation}
\label{e.hopf.formula}
\sup_{p \in \mcl Q_\infty} \inf_{q' \in \mcl Q_\infty} \mcl J_{t,q}(q',p) ,
\end{equation}

which is the Hopf formula for the solution to (<ref>) when $\psi$ is convex [29]. A formula of this form arises in the context of certain problems of statistical inference in high dimensions, see [10, 11, 12, 30, 26, 28, 54, 58, 59, 60, 61, 64, 66, 67, 90, 91], as well as <cit.> for a presentation following the Hamilton–Jacobi approach. However, it is shown in <cit.> that the variational formula in (<ref>) cannot be the limit free energy in general, because the function $\psi$ is not convex in general. A similar phenomenon seems to occur for certain problems of statistical inference on sparse graphs, see [38, 36, 55]. More broadly, there is no arrangement of inf's and sup's in front of the Hamilton–Jacobi functional that could be equal to the limit free energy in general [68]. In analogy with the change of variables described below (<ref>), one might alternatively want to use (<ref>) and ask whether the limit free energy could be written in the form of the supremum or the infimum over $p \in \mcl Q_\infty$ of the functional

\begin{equation}
\label{e.parisi-type.functional}
p \mapsto \psi(q + t \nabla \xi(p)) - t \int_0^1 (p\cdot \nabla \xi(p) - \xi(p)),
\end{equation}

but these are also invalid candidates in general [68].

Our first main result states that, under the assumption that the limit free energy exists, it must be a critical value of the Hamilton–Jacobi functional.

Assume that $\bar F_N$ converges to some $f$. Then for every $t \ge 0$ and $q \in \mcl Q_2$, there exists $(q',p) \in \mcl Q_\infty^2$ that is a critical point of $\mcl J_{t,q}$ and is such that

\begin{equation}
\label{e.main1.sg}
\lim_{N \to \infty} \bar F_N(t,q) = \mcl J_{t,q}(q',p).
\end{equation}

We understand that the assumption of convergence of $\bar F_N$ to $f$ is pointwise, that is, for every $(t,q) \in \R_+ \times \mcl Q_1$, we assume that $\lim_{N \to \infty} \bar F_N(t,q) = f(t,q)$. As will be explained in Propositions <ref> and <ref>, the function $\bar F_N$ is Lipschitz in its arguments, with a Lipschitz constant that is uniformly bounded in $N$, so the existence of subsequential limits is a consequence of the Arzelà-Ascoli theorem (and pointwise convergence on a dense subset of $\R_+\times \mcl Q_1$ implies convergence everywhere). The assumption of convergence could be weakened to a local statement around the point $(t,q)$ for which we want (<ref>) to hold, but we prefer to refrain from doing so for clarity. As shown in Proposition <ref>, one can also represent the limit free energy at $(t,q)$ as a critical value of the function in (<ref>).

In fact, for most choices of $(t,q)$, we can identify an explicit choice for the critical point $(q',p)$ appearing in Theorem <ref>, which is expressed in terms of the limit free energy $f$. Readers who are familiar with the method of characteristics (or see <cit.>) may guess our choice of $(q',p)$: since the derivative of the solution ought to be constant along the characteristic line, we aim to choose $q'$ so that $\dr_q f(t,q) = \dr_q \psi(q')$, and thus $p = \dr_q \psi(q') = \dr_q f(t,q)$.

To make this statement precise, we define $\mcl Q_{\uparrow}$ to be the set of all $q \in \mcl Q_2$ such that $q(0) = 0$ and there exists a constant $c > 0$ satisfying, for every $u \le v \in [0,1]$,

\begin{equation}
\label{e.def.Qup}
q(v) - q(u) \ge c (v-u)\, \id \quad \text{ and } \quad \ellipt(q(v) - q(u)) \le c^{-1},
\end{equation}

where $\id$ denotes the identity matrix, and $\ellipt(a)$ denotes the ratio between the largest and the smallest eigenvalues of the matrix $a \in S^D_+$.

Roughly speaking, the space $\mcl Q_\uparrow$ is the set of all $q \in \mcl Q_2$ that are “uniformly increasing” with a bounded ellipticity ratio. Although the space $\mcl Q_2$ has empty interior as a subspace of $L^2([0,1];S^\D)$, the space $\mcl Q_\uparrow$ will functionally play the role of the interior of $\mcl Q_2$. This space is for instance convenient when exploring the notion of Gateaux differentiability of a function defined on $\mcl Q_2$; we refer to Definition <ref> for the precise notion of Gateaux differentiability we use. We will show below that if $\bar F_N$ converges to some function $f$, then the limit $f$ must be Gateaux differentiable almost everywhere in $\mcl Q_2$. Our usage of the phrase “almost everywhere” requires some explanation here, since the space $\mcl Q_2 \subset L^2([0,1];S^\D)$ is infinite-dimensional. What we mean is that the set of points at which $f$ is not Gateaux differentiable is a Gaussian null set (see Definition <ref>). In particular, if $\bar F_N$ converges to some function $f$, then $f$ must be differentiable on a subset of $\R_+ \times (\mcl Q_\uparrow \cap L^\infty)$ that is dense in $\R_+ \times \mcl Q_2$.

Assume that $\bar F_N$ converges to some limit $f$. For every $t \ge 0$ and $q \in \mcl Q_\uparrow \cap L^\infty([0,1];S^D)$, if $f(t,\cdot)$ is Gateaux differentiable at $q$, then letting $p = \dr_q f(t,q)$ and $q' = q + t \nabla \xi(p)$, we have that $(q',p) \in \mcl Q_\infty^2$ is a critical point of $\mcl J_{t,q}$ and is such that (<ref>) holds.

Up to a small perturbation of the free energy, we can also identify the limit of the law of the overlap matrix in terms of this critical point. We denote by $\langle \cdot \rangle$ the Gibbs measure associated with the enriched free energy with parameters $(t,q)$, see (<ref>) for a precise definition. The canonical random variable under this measure is denoted by $\sigma$, and we denote by $\sigma'$ an independent copy of $\sigma$ under $\la \cdot \ra$, also called a replica. The overlap matrix is the random variable $N^{-1} \sigma \sigma'^\intercal$, taking values in $\R^{D\times D}$.

We now introduce the perturbation to the Hamiltonian that we need to add. We let $(\hat H_N(\sigma))_{\sigma \in (\R^\D)^N}$ be the centered Gaussian field such that, for every $\sigma, \tau \in (\R^\D)^N$,

\begin{equation}\label{e.hatH_N}  %\label{e.}
\E \Ll[ \hat H_N(\sigma) \hat H_N(\tau) \Rr] = N \Ll| \frac{\sigma \tau^\intercal}{N} \Rr| ^2.
\end{equation}

This Gaussian field can be constructed explicitly by setting

\begin{equation*}  %\label{e.}
\hat H_N(\sigma) := \frac{1}{\sqrt{N}} \sum_{d = 1}^\D \sum_{i,j = 1}^N W_{i,j} \sigma_{d,i} \sigma_{d,j},
\end{equation*}

where $(W_{i,j})$ are independent standard Gaussians. We impose this new Gaussian field to be independent of all other sources of randomness. For every $\hat t \ge 0$, we define $\hat F_N(t,\hat t,q)$ to be the enriched free energy as discussed earlier and to which we add the following quantity in the exponential:

\begin{equation}
\label{e.def.tdH}
\sqrt{2\hat t} \, \hat H_N(\si) - \hat t N \Ll| \frac{\sigma \sigma^\intercal}{N} \Rr| ^2;
\end{equation}

see also (<ref>) for more precision. The corresponding functional $\mcl J$ then becomes

\begin{equation}  \label{e.hatJ=}
\hat {\mcl J}_{t,\hat t,q}(q',p) := \psi(q') +  \la p , q-q'\ra_\cH + t \int_0^1 \xi(p) + \hat t \int_0^1 |p|^2.
\end{equation}

Assume that $\hat F_N$ converges to some limit $\hat f$ along a subsequence. Suppose also that, for some $t \ge 0$, $\hat t > 0$, and $q \in \mcl Q_\uparrow \cap L^\infty([0,1];S^D)$, we have that $\hat f(t,\hat t, \cdot)$ is Gateaux differentiable at $q$, and that $\hat f(t,\cdot,q)$ is differentiable at $\hat t$. Then letting $p = \dr_q \hat f(t,\hat t, q)$ and $q' = q + t \nabla \xi(p) + 2 \hat t p$, we have that $(q',p)\in \mcl Q_\infty^2$ is a critical point of $\hat {\mcl J}_{t,\hat t, q}$. Moreover, as $N$ tends to infinity along the said subsequence, the overlap matrix converges in law under $\E \la \cdot \ra$ to the random variable $p(U)$, where $U$ is a uniform random variable over $[0,1]$.

Notice that in Theorem <ref>, we do not require that the free energy converges as $N$ tends to infinity, as convergence along a subsequence suffices; and we recall that the Lipschitz continuity properties of $\hat F_N$ guarantee that we can always extract convergent subsequences. If we do make the assumption that $\hat F_N$ converges along the full sequence, then an application of Theorem <ref> yields a representation of the free energy itself as

\begin{equation*}  %\label{e.}
\lim_{N \to \infty} \bar F_N(t,q) = \hat{\mcl J}_{t,\hat t,q}(q',p),
\end{equation*}

for the same critical point $(q',p)$ of $\hat{\mcl J}_{t,\hat t,q}$. To be clear, we mention that the statement that $(q',p)$ is a critical point of $\hat{\mcl J}_{t,\hat t,q}$ means that

\begin{equation*}  %\label{e.}
q = q'-t \nabla \xi(p) - 2\hat t  p \quad \text{ and } \quad p = \dr_q \psi(q').
\end{equation*}

As for Theorem <ref>, for each $t \ge 0$, almost every choice of $(\hat t, q) \in \R_+ \times (\mcl Q_\uparrow \cap L^\infty)$ is a point that satisfies the assumptions of Theorem <ref>. In particular, the set of such points is dense in $\R_+ \times \mcl Q_2$.

In fact, we will prove a stronger version of Theorem <ref> in which we identify the limit law of the array of overlaps involving arbitrarily many independent copies of $\sigma$ under $\la \cdot \ra$.

A natural question is whether results such as Theorem <ref> characterize the limit free energy $f$ uniquely. The next proposition answers this question positively at high temperature.

There exists $t_c > 0$ such that for every $t \in [0,t_c)$ and $q \in \mcl Q_2$, the function $\mcl J_{t,q}$ has a unique critical point in $\mcl Q_2^2$.

For $t> 0$ sufficiently small and $q = 0$, Theorem <ref> is shown in [34]. Showing Theorem <ref> at high temperature for non-zero paths $q$ is not a straightforward adaptation of the argument in [34] because the system is no longer replica-symmetric in this case, no matter how small $t$ is.

The Hamilton–Jacobi functional may have multiple critical points in general. Interestingly, this fact does not completely rule out the possibility that Theorem <ref> might fully characterize the limit free energy. Indeed, one can exclude some spurious critical points by exploiting the continuity properties of the free energy, for instance by using the following simple observation.

For each integer $n \ge 1$, let $(t_n,q_n) \in \R_+ \times \mcl Q_2$, and let $(q'_n,p_n) \in \mcl Q_2^2$ be a critical point of $\mcl J_{t_n,q_n}$ such that

\begin{equation*}
\lim_{N \to \infty} \bar F_N(t_n,q_n) = \mcl J_{t_n,q_n}(q'_n,p_n).
\end{equation*}

Suppose that $(t_n,q_n)$ converges towards $(t,q) \in \R_+ \times \mcl Q_2$. Then the sequence $(q'_n,p_n)$ is precompact in $\mcl Q_2^2$. Moreover, any subsequential limit $(q',p) \in \mcl Q_2^2$ is a critical point of $\mcl J_{t,q}$, and is such that

\begin{equation}
\label{e.stability.conclusion}
\lim_{N \to \infty} \bar F_N(t,q) = \mcl J_{t,q}(q',p).
\end{equation}

Proposition <ref> implies that among all the candidate critical points of $\mcl J_{t,q}$, we can exclude those that cannot be represented as limits of critical points at nearby locations. While we will not prove this here, we expect that some spurious critical points can indeed be excluded using this property.

In order to avoid possible confusion, we mention that the notion of stability of critical points implied by Proposition <ref> is different from another possible notion of stability sometimes evoked by physicists and that has its clearest meaning for convex models. Indeed, since Theorem <ref> gives a variational formula for the limit free energy in this case, we can deduce constraints on the Hessian of the Hamilton–Jacobi functional at the relevant critical point, at least at a formal level. From the arguments of <cit.>, we do not expect that one can strengthen Theorem <ref> and prescribe the “orientation” of the Hessian of the Hamilton–Jacobi functional at a critical point $(q',p)$ in general. Our understanding is that physicists sometimes proceed by inspecting the orientation of the Hessian of the unique critical point of the high-temperature phase, and by then postulating that the same orientation must be valid at low temperature. While this might work in some cases, we would be surprised if this procedure was valid in general.

We now comment on our assumption of convergence of the free energy in Theorems <ref> and <ref>. Even for certain scalar models (with $\xi$ that is not convex on $\R$), the only technique we are aware of for showing the convergence of the free energy to some limit consists in obtaining a full characterization of the limit; in other words, it consists in showing that Theorem <ref> holds. In view of this, we find it more promising to direct further effort towards a full identification of the limit for general models, rather than towards the discovery of an argument that would only guarantee convergence.

§.§ Related works

We have already mentioned a number of works that are related to the present paper. In particular, we recall that the limit free energy of the Sherrington–Kirkpatrick model was determined in [49, 102, 103, 104]. The proof for general scalar models ($D = 1$) was obtained in [76, 77] by showing that the Ghirlanda–Guerra identities [2, 45, 47] suffice to ensure the ultrametricity of the Gibbs measure. The version of Theorem <ref> with $\xi$ assumed to be convex over $\R^{D\times D}$ was obtained in [13, 80, 81, 82]. The main ingredient of the proof there is that by enforcing ultrametricity of the overlaps associated with a large family of linear combinations of the spins of different types, one can deduce that the overlaps corresponding to these different types synchronize with one another. These properties of ultrametricity and synchronization will also be used crucially in the present work. Related results for spherical models were obtained in [31, 101] for scalar models, and in [16, 17, 57, 83] for models with multiple types of spins.

Connections between mean-field models of statistical mechanics and Hamilton–Jacobi equations have already been noted in [23, 75]; we also refer to [18] for a recent survey on related topics. In the context of spin glasses, these connections were first explored in [1, 14, 15, 48], under a replica-symmetric or one-step replica symmetry breaking assumption. The fact that the Parisi formula can be seen as the Hopf–Lax formula of an infinite-dimensional Hamilton–Jacobi equation is from [70, 73]; see also [37] for a pedagogical presentation. The rigorous justification of the fact that this Hopf–Lax formula is indeed the solution to the Hamilton–Jacobi equation is from [29]. An inequality between the limit free energy and the solution to the Hamilton–Jacobi equation is obtained in [68, 71] for general models.

For certain spherical models with multiple types and non-convex $\xi$ called “pure models”, the limit free energy was identified explicitly in [97, 99, 98, 100], assuming that this limit free energy is well-defined; see also [9] for the case of the bipartite spherical Sherrington–Kirkpatrick model. A more geometric analysis of the energy landscape is in [20, 53, 56, 62]; see also  [6, 7, 41, 95] for scalar models.

In the low-temperature regime, the free energy is a good approximation for the maximum of the energy function $\sigma \mapsto H_N(\sigma)$ over the support of $P_N$ (see for instance <cit.>). We think that it would be interesting to work out what consequences our results entail for the asymptotic behavior of this maximum, in the limit of large system size. For some convex models, a Parisi-like variational formula capturing this was obtained in [8, 32, 35].  Relatedly, there has been much recent progress on the problem of determining whether there exists a polynomial-time algorithm that identifies a configuration $\sigma$ in the support of $P_N$ such that $H_N(\sigma)$ is close to its maximal value. The answer turns out to depend on the specifics of the model, and relates to the overlap gap property [43]; see also [42, 44] for surveys. In general, one can identify a specific value $\mathsf{ALG}$ such that there exists a polynomial-time algorithm that can identify a configuration $\sigma$ with $H_N(\sigma) \simeq N \mathsf{ALG}$ with high probability, but such that any algorithm within a broad class will demonstrably fail to improve upon this limit value $\mathsf{ALG}$, as shown first for scalar models in [40, 50, 65, 93, 96]. Remarkably, they have been successfully extended to general, possibly non-convex spherical models with multiple types in [51, 52], even though the asymptotic behavior of the maximum of $H_N$ is currently not understood at this level of generality.

§.§ Outline of the paper

In Section <ref>, we explore the differentiability properties of Lipschitz functions on $\mcl Q_2$. In particular, we show that any such function is Gateaux-differentiable outside of a Gaussian null set. In Section <ref>, we define the free energy $\bar F_N(t,q)$ for piecewise-constant paths $q$ using discrete Poisson–Dirichlet cascades, and then extend it to any path $q \in \mcl Q_1$ by continuity. We also show that the sequence $(\bar F_N)_{N \in \N}$ is precompact, and that $\bar F_N$ is locally semi-concave uniformly over $N$. It is convenient to also give a direct definition of the free energy and its associated Gibbs measure for arbitrary paths $q \in \mcl Q_\infty$. For this purpose, we derive a number of preliminary results involving continuous Poisson-Dirichlet cascades in Section <ref>, and then give this direct definition of the free energy in Section <ref>. In that section, we also study the differentiability properties of $\bar F_N$ and of any subsequential limit with respect to the path $q$. We turn to cavity calculations in Section <ref>. Using the ultrametricity and synchronization results from [76] and  [80, 81, 82], we extend the Aizenman–Simms–Starr scheme [3] and obtain precise representations of $\bar F_{N+1} - \bar F_N$ and of $\dr_q \bar F_N$. Information about $\dr_q \bar F_N$ gives us access to the asymptotic law of the overlap matrix. In Section <ref>, we combine these results to show Theorem <ref> and then Theorem <ref>, as well as some refinements involving overlap arrays. We also show that any subsequential limit $f$ of the free energy $\bar F_N$ must be such that, at every point $(t,q)$ of Gateaux differentiability of $f$,  we have

\begin{equation}
\label{e.satisfy.eqn.intro}
\dr_t f(t,q) = \int_0^1 \xi(\dr_q f(t,q,u)) \, \d u.
\end{equation}

In particular, this shows that the function $f$ must satisfy the Hamilton–Jacobi equation (<ref>) everywhere outside of a Gaussian null set. Finally, Section <ref> derives further consequences of our results in the special case when $\xi$ is convex over $S^D_+$. In particular, we prove Theorem <ref> there.

§.§ Explicit representation of the Gaussian field H_N

Before closing this introduction, we clarify the class of functions $\xi$ such that (<ref>) can hold for some Gaussian process $H_N$, and give an explicit construction of $H_N$ in this case. By <cit.>, the function $\xi$ must take the form

\begin{equation}
\label{e.explicit.xi}
\xi(a) = \sum_{p=1}^{+\infty}\mathsf{C}^{(p)}\cdot a^{\otimes p},
\end{equation}

where $\mathsf{C}^{(p)}\in \S^{\D^p}_+$ for each $p \ge 1$. In (<ref>), the notation $a^{\otimes p}$ stands for the $p$-fold tensor product of the matrix $a \in \R^{D\times D}$, which we interpret as a $D^p$-by-$D^p$ matrix. We recall that we assume that the series in (<ref>) is absolutely convergent for every $a \in \R^{D\times D}$. The summation in (<ref>) starts at $p = 1$ since we assume that $\xi(0) = 0$. There is no loss of generality in this assumption, since the general case can be obtained by simply adding a fixed Gaussian random variable to $H_N$, and this changes neither the averaged free energy nor the Gibbs measure.

To give an explicit construction of the Hamiltonian $H_N$, it suffices to consider the case when only one of the terms in the series in (<ref>) is non-zero, since the general case can then be obtained by adding independent Gaussian fields corresponding to each term in the series. We therefore fix $p \ge 1$, and let
$(J^{(\mathbf d)})_{\mathbf d \in \{1,\ldots D\}^p}$ be a centered Gaussian vector with covariance matrix $\msf C^{(p)}$. We let $((J^{(\mathbf d)}_{\mathbf i})_{\mathbf d \in \{1,\ldots, D\}^p})_{\mathbf{i} \in \{1,\ldots, N\}^p}$ denote $N^p$ independent copies of the $D^p$-dimensional random vector $(J^{(\mathbf d)})_{\mathbf d \in \{1,\ldots D\}^p}$. In particular, we have for every $\mathbf{d}, \mathbf{d}' \in \{1,\ldots, D\}^p$ and $\mathbf{i}, \mathbf{i}'\in \{1,\ldots, N\}^p$ that

\begin{equation*}  %\label{e.}
\E \Ll[ J_{\mathbf{i}}^{(\mathbf d)}  J_{\mathbf{i}'}^{(\mathbf d')} \Rr] = \1_{\{\mathbf{i} = \mathbf{i}'\}} \, \msf C^{(p)}_{\mathbf{d}, \mathbf{d}'} .
\end{equation*}

We then set, for every $\sigma = (\sigma_{d,i})_{1 \le d \le D, 1 \le i \le N} \in \R^{D\times N}$,

\begin{equation*}  %\label{e.}
H_N(\si) := N^{-\frac{p-1}{2}} \sum_{d_1, \ldots, d_p = 1}^D \ \sum_{i_1,\ldots, i_p = 1}^N J_{i_1, \ldots, i_p}^{(d_1, \ldots, d_p)} \si_{d_1,i_1} \cdots \, \si_{d_p,i_p}.
\end{equation*}

This Gaussian process satisfies (<ref>)  for the function $\xi$ given by

\begin{equation*}
\xi(a) = \msf C^{(p)} \cdot a^{\otimes p}.
\end{equation*}

§.§ Assumption on the support of P_1

All the results stated above are valid under the assumption that the reference measure $P_N = P_1^{\otimes N}$ is such that $P_1$ has compact support in $\R^D$. For convenience, we assume from now on that

\begin{equation}
\label{e.bound.support}
\mbox{the support of $P_1$ is contained in the closed unit ball of $\R^D$.}
\end{equation}

Up to a rescaling of the variables, this entails no loss in generality.

§ DIFFERENTIABILITY OF LIPSCHITZ FUNCTIONS

The main goal of this section is to show that a Lipschitz function on the space of paths $\C$ is differentiable “almost everywhere”. We do not know whether a statement in this spirit is valid if the notion of differentiability is meant in the sense of Fréchet. We will show that it is indeed valid if the notion of differentiability is somewhat weakened, and with “almost everywhere” being understood in the sense that the set of points of non-differentiability is a Gaussian null set.

Throughout this section, we let $E$ be a separable Banach space, and let $G$ be a subset of $E$. In our application, we will choose $E$ to be $L^2([0,1]; S^D)$ and $G = \C_2$. We denote by $|\cdot|_E$ the norm on the space $E$, by $E^*$ the (continuous) dual of $E$, and by $\la \cdot, \cdot \ra_E$ the canonical pairing of $(E^*,E)$. We start by giving precise definitions of the notions of differentiability we use.

A function $g:G\to\R$ is Fréchet differentiable at ${q} \in G$ if there is a unique $y^* \in E^*$ such that

\begin{align}
\label{e.def.frechet}
\lim_{r\to 0}\sup_{\substack{{q'}\in G\setminus\{{q}\}\\ |{q'}-{q}|_E\leq r}}\frac{\Ll|g({q'}) - g({q}) - \la y^* , {q'}-{q}\ra_E\Rr|}{|{q'}-{q}|_E} =0.
\end{align}

In this case, we call $y^*$ the Fréchet derivative of $g$ at $q$.

The condition in (<ref>) can be rewritten as

\begin{align*}
g({q'}) - g({q}) = \la y^*, {q'}-{q} \ra_E + o\Ll(\Ll|{q'}-{q}\Rr|_E\Rr)
\end{align*}

for ${q'} \in G$ tending to ${q}$. The notion of Fréchet differentiability is often defined only when $G$ is an open set, in which case if there exists $y^* \in E^*$ satisfying (<ref>), then it is necessarily unique. But we do not require that $G$ be open here.

For every $q\in G$, we define

\begin{align*}
\mathrm{Adm}(G,q) :=\Ll\{e\in E\ \big|\  \exists r>0 \, : \, \forall t \in [0,r],\ q+te\in G\Rr\}
\end{align*}

to be the set of admissible directions at $q$ along which a small line segment is contained in $G$.

A function $g:G\to \R$ is Gateaux differentiable at $q\in G$ if the following two conditions hold:

* for every $e\in \mathrm{Adm}(G,q)$, the following limit exists:

\begin{equation}
\label{e.def.gateaux.lim}
g'(q,e) := \lim_{r\searrow 0} \frac{g(q+re)-g(q)}{r};
\end{equation}

* there is a unique $y^*\in E^*$ such that  for every $e\in \mathrm{Adm}(G,q)$, we have $g'(q,e) = \la y^*,e\ra_E$.

In this case, we call $y^*$ the Gateaux derivative of $g$ at $q$.

We stress that our definition of Gateaux differentiability may differ from the  most classical one. For one thing, the notion is most commonly used only when $G$ is an open set. Most importantly, the notion of Gateaux differentiability is often understood in the sense that only the first of the two properties we listed is valid, while here we require in particular that the mapping $e \mapsto g'(q,e)$ be linear and continuous. We refer to [74] for a comparison of the different notions.  When $G$ is an open set, our definition coincides with those of <cit.>, <cit.>, and <cit.> (we impose $r > 0$ in (<ref>) while the limit is two-sided in [88, 21], but these are equivalent since we also have that $g'(q,e) = -g'(q,-e)$). An example of a Lipschitz function that is nowhere Fréchet differentiable is the $\ell^1$ norm on the space of absolutely summable sequences; this function is Gateaux differentiable at every nowhere-vanishing sequence.

In the problem we are ultimately interested in, the function defined on $G$ admits a Lipschitz extension on the full space $E$. Under the assumption that the norm $|\cdot |_E$ is Fréchet differentiable away from the origin, it was shown in [89] that a Lipschitz function on $E$ must be Fréchet differentiable on a dense set. However, what we really want is to identify a dense set of points of differentiability in $G$. Since in our application, the set $G$ has empty interior, the existence of a dense set of points of differentiability in $E$ does not guarantee the existence of any such point in $G$.

We will therefore rely instead on the fact, proved in [5, 88], that a Lipschitz function on $E$ must be Gateaux differentiable outside of a Gaussian null set. As will be explained next, a Gaussian null set is sufficiently “thin” that in our application, we can deduce in particular the existence in $G$ of a dense set of points of Gateaux differentiability.

We now give a precise definition of the notion of a Gaussian null set, and present the relevant results from [88]. We equip $E$ with the Borel $\sigma$-algebra generated by its norm topology.

A probability measure $\mu$ on $E$ is said to be a non-degenerate Gaussian measure with mean $a \in E$ if for every non-zero $e^*\in E^*$, the measure $\mu\circ (e^*)^{-1}$ is a Gaussian measure with mean $e^*(a)$ and non-zero variance.

Although we will not make use of this fact, we mention in passing that when $E$ is a separable Banach space as we assume here, the mean of a Gaussian measure can always be represented as an element of $E$, by <cit.>. This is easier to see when $E$ a separable Hilbert space (e.g. <cit.>), or more generally when $E$ is reflexive. In our application the space $E$ will be a separable Hilbert space.

A Borel subset $B$ of $E$ is said to be a Gaussian null set if for every non-degenerate Gaussian measure $\mu$ on $E$, we have $\mu(B)=0$.

The following lemma is a simple modification of <cit.>; we provide a complete proof for the reader's convenience.

Let $(w_n)_{n\in\N}$ be a sequence in $E$ that has dense linear span and satisfies $\lim_{n\to\infty}|w_n|_E=0$, and let $K$ be the closed convex hull of $\{0\}\cup\{w_n:n\in\N\}$. For every $x\in E$, the set $x+K$ is not a Gaussian null set.

We denote by $\ell^2$ the Hilbert space of square-summable sequences indexed by $\N$, and decompose the proof into two steps.

Step 1. Let $\mu$ be a non-degenerate Gaussian measure on $\ell^2$ with mean $a \in \ell^2$. In this step, we show that $\mu$ attributes a positive mass to every open set; or equivalently, that the support of $\mu$ is the full space $\ell^2$. By <cit.>, there exists a continuous self-adjoint operator $A : \ell^2 \to \ell^2$ such that for every $u \in \ell^2$, the measure $\mu \circ u^{-1}$ is Gaussian with mean $u \cdot a$ and variance $u \cdot A u$. In the notation $\mu \circ u^{-1}$, we interpret $u$ as the element of the dual of $\ell^2$ canonically associated with $u$, namely the mapping $x \mapsto u \cdot x$. Since $\mu$ is non-degenerate, the kernel of $A$ must be trivial. Since $A$ is self-adjoint, this implies that the image of $A$ is dense in $\ell^2$. We also observe that for every $h \in \ell^2$, the translation of $\mu$ by $Ah$ is equivalent to $\mu$, since its Radon–Nikodym derivative with respect to $\mu$ is the mapping

\begin{equation*}  %\label{e.}
x \mapsto \exp \Ll( (x-a) \cdot h - \frac 1 2 h \cdot A h  \Rr) .
\end{equation*}

Now suppose that there exists an open set $U \subset \ell^2$ such that $\mu(U) = 0$. We deduce that the translation of $U$ by any $Ah$, $h \in \ell^2$ also has zero measure. But since the image of $A$ is dense in $\ell^2$, we can find a sequence $(h_n)_{n \in \N}$ such that

\begin{equation*}  %\label{e.}
\bigcup_{n \in \N} (U + A h_n) = \ell^2.
\end{equation*}

We have thus reached the absurd conclusion that $\mu(\ell^2) = 0$; so there exists no open set $U \subset \ell^2$ such that $\mu(U) = 0$.

Step 2.
We define the linear map $L:\ell^2\to E$ such that, for every $y = (y_n)_{n\in\N}\in \ell^2$,

\begin{align*}
Ly := \sum_{n\in\N}2^{-n}y_n w_n.
\end{align*}

Let $U$ be the unit ball of $\ell^2$. Since the set $\{0\}\cup\{w_n:n\in\N\}\cup\{-w_n:n\in\N\}$ is compact in $E$, its closed convex hull $G$ is also compact (see for instance <cit.>).
Since $|y_n|\leq 1$ for every $y\in U$ and $n \in \N$, we have $LU \subset G$. Compactness of $G$ implies that $G$ is bounded and thus $L$ is continuous. We define $T:\ell^2\to E$ such that, for every $y \in \ell^2$,

\begin{equation*}  %\label{e.}
T(y) := x+\frac{1}{2}Ly + \sum_{n\in\N} 2^{-n-1}w_n =x+ \sum_{n\in\N} 2^{-n}\Ll(\frac{y_n+1}{2}\Rr)w_n.
\end{equation*}

Since $\frac{y_n+1}{2} \in [0,1]$ for every $y\in U$ and $n \in \N$, we have $T(U)\subset x+K$. We let $\mu$ be a non-degenerate Gaussian measure on $\ell^2$. By the result of the previous step, we have that $\mu$ attributes a positive mass to $U$. We denote $\nu := \mu \circ T^{-1}$. Since $T$ is affine and continuous, the measure $\nu$ is a Gaussian measure on $E$ (with a mean in $E$); and since $T(U) \subset x + K$, it attributes a positive mass to $x + K$. To see that it is non-degenerate, let $e^* \in E^*$ be non-zero. For every $y \in \ell^2$, we have

\begin{equation*}  %\label{e.}
\la e^*,T(y)\ra_E = \la z, y\ra_{\ell^2} + c ,
\end{equation*}

\begin{equation*}  %\label{e.}
z= \Ll(2^{-n-1}\la e^*,w_n\ra_E\Rr)_{n\in\N} \quad \text{ and } \quad c = \la e^*,x+\sum_{n\in\N}2^{-n-1}w_n\ra_E.
\end{equation*}

Recall that the linear span of $(w_n)_{n \in \N}$ is dense in $E$. If $z$ was zero, then by linearity and continuity of $e^*$, we would deduce that $e^*$ is zero as well, which is excluded. Since $\nu \circ (e^{*})^{-1}$ is a translation of the measure $\mu \circ z^{-1}$, and $\mu$ is non-degenerate, we conclude that $\nu$ is non-degenerate as well. To sum up, we have found a non-degenerate Gaussian measure $\nu$ such that $\nu(x+K) > 0$, so the set $x + K$ is not a Gaussian null set.

The next theorem is a special case of <cit.>. This result is itself based on <cit.>, which shows a similar result using another notion of thin sets now called “Aronszajn null sets”, and the main contribution of [88] is to show that these sets are Gaussian null sets.  We also refer to <cit.> for a recent monograph presentation of this result.

If a function $g: E\to\R$ is  Lipschitz continuous, then it is Gateaux differentiable outside of a Gaussian null subset of $E$.

We now specialize this result to the sets $E$ and $G$ of interest to us, namely $E= L^2([0,1]; S^D)$ and $G = \mcl Q_2$. We recall that the $\mcl Q_2$ is the intersection of the set $\C$ defined in (<ref>) with $L^2([0,1];S^D)$. For every $p \in [1,\infty]$, we use the shorthand notation $L^p := L^p([0,1];S^D)$. We also recall that the set $\C_\uparrow$ is defined in the sentence containing (<ref>). We observe that the set $\C_\uparrow\cap L^\infty$ is dense in $\C_2$. Indeed, denoting by $S^D_{++}$ the set of positive definite matrices, we have that

\begin{equation*}  %\label{e.}
\Ll\{ u \mapsto cu\, \id + \sum_{k = 0}^K q_k \1_{ \Ll[ \frac{k}{K+1}, \frac{k+1}{K+1} \Rr) } \mid c > 0, \ q_0, \ldots, q_K \in S^D_{++} \Rr\} ,
\end{equation*}

is dense in $\mcl Q_2$ and is a subset of $\mcl Q_\uparrow \cap L^\infty$.

If $g:\C_2\to\R$ is Lipschitz continuous, then there exists a Gaussian null set $\mathcal{N}$ of $\cH$ such that $g$ is Gateaux differentiable on $\C_2\setminus\mathcal{N}$. Moreover, $(\C_\uparrow\cap L^\infty)\setminus\mathcal{N}$ is dense in $\C_2$.

Similarly, if $g:\R_+\times\C_2\to\R$ is Lipschitz continuous (in its two variables), then there exists a Gaussian null set $\mathcal{N}$ of $\R\times\cH$ such that $g$ is Gateaux differentiable jointly in its two variables on $(\R_+\times\C_2)\setminus\mathcal{N}$. Moreover, $(\R_+\times(\C_\uparrow\cap L^\infty))\setminus\mathcal{N}$ is dense in $\R_+\times\C_2$.

We only prove the first part of the proposition, since the second part is obtained similarly. We decompose the proof into two steps.

Step 1. We construct a Lipschitz extension $\bar g : L^2 \to \R$ to the Lipschitz function $g : \mcl Q_2 \to \R$.

We denote by $P : L^2 \to \mcl Q_2$ the orthogonal projection to $\mcl Q_2$. By definition, for every $q \in L^2$, the point $P(q)$ is the unique minimizer in $\mcl Q_2$ of the mapping

\begin{equation*}  %\label{e.}
\Ll\{
\begin{array}{rcl}  %\label{}
\mcl Q_2 & \to & \R \\
q' & \mapsto & |q-q'|_{L^2}^2.
\end{array}
\Rr.
\end{equation*}

We refer to <cit.> for a justification that the mapping $P$ is well-defined, based on the fact that $\mcl Q_2$ is a closed and convex subset of $L^2$. It is also shown there using the convexity of $\mcl Q_2$ that, for every $q \in L^2$ and $p \in \mcl Q_2$, we have

\begin{equation}
\label{e.first.variation.P}
\la q - P(q) , p - P(q) \ra_\cH \le 0.
\end{equation}

We show that, for every $q,q' \in L^2$,

\begin{equation}
\label{e.lipschitz.projection}
|P(q') - P(q)|_{L^2} \le|q'-q|_{L^2}.
\end{equation}

For every $q,q' \in L^2$, we use the identity (<ref>) with $p$ substituted with $P(q')$, and similarly with $q$ and $q'$ interchanged, to get that

\begin{equation*}  %\label{e.}
\la q- P(q), P(q') - P(q) \ra_\cH \le 0 \le \la q'-P(q'), P(q') - P(q) \ra_\cH.
\end{equation*}

Rearranging and using the Cauchy–Schwarz inequality, this yields that

\begin{equation*}  %\label{e.}
|P(q') - P(q)|_{L^2}^2 \le \la q'-q, P(q') - P(q) \ra_\cH \le |q'-q|_{L^2} \, |P(q') - P(q)|_{L^2}.
\end{equation*}

This shows (<ref>). To construct the extension of $g$ to $L^2$, we set, for every $q \in \mcl Q_2$,

\begin{equation*}  %\label{e.}
\bar g(q) := g(P(q)).
\end{equation*}

Since $P$ is the identity on $\mcl Q_2$, the function $\bar g$ is indeed an extension of the function $g$. In view of (<ref>) and of the fact that $g$ is Lipschitz, the function $\bar g$ is also Lipschitz, as desired.

Step 2.
We let $\bar g : L^2 \to \R$ be a Lipschitz extension of $g$ as given by the previous step. By Theorem <ref>, there exists a Gaussian null set $\mathcal{N}\subset \cH$ such that $\bar g$ is Gateaux differentiable on $\cH\setminus \mathcal{N}$.
Letting $x \in \C_2 \setminus \mcl N$, we verify that $g$ is Gateaux differentiable at $x$. Let $y^* \in (L^2)^* \simeq L^2$ denote the Gateaux derivative of $\bar g$ at $x$. The limit $g'(x,e)$ in (<ref>) exists for every $e \in \mathrm{Adm}(\C_2,x)$, and it is equal to $\la y^*, e \ra_{L^2}$. One can check that the orthogonal of $\C_2$ in $L^2$ is trivial, so the closed linear span of $\C_2$ is $L^2$. Moreover, we clearly have $\C_2\subset \mathrm{Adm}(\C_2,x)$, so there can be only one $y^*$ such that $g'(x,e) = \la y^*, e \ra_{L^2}$ for every $e \in \mathrm{Adm}(\C_2,x)$. This shows that $g$ is Gateaux differentiable at $x \in \C_2\setminus\mathcal{N}$.

We next show that $(\C_\uparrow\cap L^\infty)\setminus\mathcal{N}$ must be dense in $\C_2$.
Notice that the set

\begin{equation*}  %\label{e.}
W:=\{q\in \C:\: q(0)=0\ \text{and}\ \forall u\leq v\in[0,1),\ q(v)-q(u)\leq (v-u)\id \}
\end{equation*}

has dense linear span in $\cH$.
Let $(w_n)_{n\in\N}$ be a sequence in $W$ whose linear span is dense in $\cH$. We can rescale $w_n$ to ensure that $|w_n|_\cH\leq 1$ and $\lim_{n\to\infty} |w_n|_\cH=0$. Let $K$ be as given in Lemma <ref>. We fix any $x\in \C_\uparrow\cap L^\infty$ and any $\eps>0$. By Lemma <ref>, we have that $x+\eps K$ is not a Gaussian null set, which implies that $(x+\eps K)\not\subset\mathcal{N}$. Hence, there is $y \in (x+\eps K)\setminus\mathcal{N}$ such that $g$ is Gateaux differentiable at $y$.

To complete the argument, we show that $x+\eps K \subset \C_\uparrow\cap L^\infty$. Let $\kappa\in K$ and let $c$ be the constant in (<ref>) for $x$. We write $x':=x+\eps \kappa$ and we clearly have $x'(0)=0$. Fix any $0\leq u\leq v\leq 1$. We have $x'(v)-x'(u)\geq x(v)-x(u) \geq c(v-u)\id$. Let $\lambda_{\max}$ and $\lambda_{\min}$ be the largest and the smallest eigenvalue of $x(v)-x(u)$, respectively. Note that $\lambda_{\min}\geq c(v-u)$ and that the eigenvalues of $\kappa(v)-\kappa(u)$ lie in $[0,v-u]$. So, we get

\begin{align*}
\ellipt(x'(v)-x'(u)) \leq \frac{\lambda_{\max}+\eps(v-u)}{\lambda_{\min}+0} = \frac{\lambda_{\max}}{\lambda_{\min}} + \frac{\eps(v-u)}{\lambda_{\min}} \leq c^{-1}+\frac{\eps}{c}.
\end{align*}

So, we conclude that $x'\in \C_\uparrow$, and we clearly also have that $x'\in L^\infty$. Therefore, we have verified $x+\eps K \subset \C_\uparrow\cap L^\infty$.

Hence, $y\in (\C_\uparrow\cap L^\infty)\setminus\mathcal{N}$.
Since $|y-x|_\cH\leq \eps$ and $\ep > 0$ was arbitrary, we conclude that $(\C_\uparrow\cap L^\infty)\setminus\mathcal{N}$ is dense in $\C_\uparrow\cap L^\infty$ and thus also in $\C_2$.

§ DEFINITION AND REGULARITY PROPERTIES OF THE FREE ENERGY

In this section, we define the free energy $\bar F_N(t,q)$ for arbitrary choices of the path $q \in \mcl Q_1$. When the path $q$ is constant equal to $h \in S^D_+$, we have already displayed the definition of the free energy in (<ref>). Here we will first extend this definition to piecewise-constant $q \in \mcl Q$. One can then verify that the free energy is $L^1$-Lipschitz continuous in $q$, and this therefore defines the free energy $\bar F_N(t,q)$ for arbitrary choices of $q \in \mcl Q_1$ by density.

Another consequence of this Lipschitz continuity property is that the sequence of functions $(\bar F_N)_{N \in \N}$ is precompact. We also show that the free energy is locally semi-concave. This property will be used later to assert the convergence of the derivatives of $\bar F_N$ at any point of differentiability of the limit.

We start by defining the free energy $\bar F_N(t,q)$ for piecewise-constant $q \in \mcl Q$. When $q$ is constant equal to $h \in S^D_+$, we had introduced in (<ref>) the external random magnetic field $\sqrt{2h} z$ acting on the spin configuration $\sigma$, where $z$ takes values in $\R^{D\times N}$ and has independent standard Gaussian entries. We will here replace this random magnetic field by a richer object that, in the language of physics, has a finite number of replica-symmetry breaks. Precisely, for each choice of integer $K \in \N$ (the number of replica-symmetry breaks) and parameters $(q_k)_{0 \le k \le K} \subset S^D_+$ and $(\zeta_k)_{1 \le k \le K} \subset [0,1]$ satisfying the constraints

\begin{equation}
\label{e.def.q}
0= q_{-1} \le q_0 < q_1 < \cdots < q_K
\end{equation}

\begin{equation}
\label{e.def.zeta}
0 = \zeta_0 < \zeta_1 < \cdots < \zeta_K < \zeta_{K+1} = 1,
\end{equation}

we aim to make sense of $\bar F_N(t,q)$ where $q$ is given by

\begin{equation}
\label{e.def.piece.const.q}
q = \sum_{k = 0}^K q_k \1_{ \Ll[ \zeta_{k}, \zeta_{k+1} \Rr) }.
\end{equation}

In (<ref>) and throughout the paper, whenever $a,b \in S^D$, we write $a < b$ to mean that $b-a \in S^D_+$ and $a \neq b$. The construction involves the notion of Poisson–Dirichlet cascades, which we now introduce briefly. We let $\mcl A$ denote the infinite-ary tree of depth $K$, which we encode as

\begin{align}\label{e.tree}
\mcl {A} := \N^0 \cup \N^1\cup\cdots \cup \N^K,
\end{align}

with the understanding that $\N^0 =\{\emptyset\}$. We understand that $\emptyset$ is the root of the tree $\mcl A$, and that for each $k \in \{0,\ldots,K-1\}$ and $\alpha = (n_1,\ldots, n_k) \in \N^k$, the children of $\alpha$ are all the nodes

\begin{equation*}  %\label{e.}
\alpha n := (n_1, \ldots, n_k,n) \in \N^{k+1},
\end{equation*}

where $n$ ranges in $\N$. For every $\alpha = (n_1,\ldots,n_k) \in \mcl A$, we denote its depth by $|\alpha| := k$, and for every $\ell \le k$, we write

\begin{equation*}
\alpha_{|\ell} := (n_1,\ldots,n_\ell)
\end{equation*}

to denote the ancestor of $\alpha$ at depth $\ell$. For any two leaves $\alpha, \beta \in \N^K$, we write

\begin{equation}
\label{e.def.wedge}
\alpha \wedge \beta := \max \Ll\{ k \le K \mid \alpha_{|k} = \beta_{|k} \Rr\}
\end{equation}

to denote the depth of the most recent common ancestor to $\alpha$ and $\beta$. The Poisson–Dirichlet cascade (also called Ruelle probability cascade after [92]) with parameters $(\zeta_k)_{1 \le k \le K}$  is a random probability measure on the leaves $\N^K$, whose weights we denote by $(v_\alpha)_{\alpha \in \N^K}$; in particular, $v_\alpha \ge 0$ and $\sum_{\al \in \N^K} v_\al = 1$. Briefly, this probability measure is constructed as follows (more details can be found in <cit.> or <cit.>). With each non-leaf vertex $\alpha$ at depth $k \le K-1$, we attach an independent Poisson point process on $\R_+$ with intensity measure $x^{-1-\zeta_{k+1}} \, \d x$. This Poisson point process can be ordered decreasingly, say $u_{\alpha 1} \ge u_{\alpha 2} \ge \cdots$, and we attribute the weight $u_{\alpha n}$ to the child vertex $\alpha n$. This attributes a weight to each vertex except the root. Finally, for each leaf vertex $\alpha \in \N^K$, we compute the product of all the weights attached to the nodes from the root to the leaf vertex $\alpha$, excluding the root, and we denote the result by $w_\alpha$, that is, $w_\alpha := \prod_{k = 1}^{K} u_{\alpha_{|k}}$. The family $(v_\alpha)_{\alpha \in \N^K}$ is then obtained by normalizing the family $(w_\alpha)_{\alpha \in \N^K}$ into a probability measure, that is, $v_\alpha := w_\alpha / \sum_\beta w_\beta$. We construct this object independently of $(H_N(\sigma))_{\sigma \in \R^{D\times N}}$.

In order to define the random magnetic field, we also give ourselves $(z_\beta)_{\beta\in \mathcal{A}}$ a collection of independent standard $\R^{D \times N}$-valued Gaussian vectors, independent of the Gaussian field $(H_N(\sigma))$ and the cascade $(v_{\alpha})$.
For every $\alpha \in \N^K$, we set

\begin{align}\label{e.Z(h_alpha)=}
\msf w^q(\al) := \sum_{k = 0}^K(q_k - q_{k-1})^{1/2} z_{\alpha_{|k}}.
\end{align}

The random variable $\msf w^q(\alpha)$ takes values in $\R^{D\times N}$.
For every $\al, \al' \in \N^K$, we have

\begin{align*}%\label{e.Eww=}
\E \msf w^q(\al)\msf w^q(\al')^\intercal = N q_{\al\wedge\al'}.
\end{align*}

Recalling the definition of the piecewise-constant path $q$ in (<ref>), we define

\begin{equation}
\label{e.def.FN.piece.const}
\bar F_N(t,q) := -\frac 1 N \E \log \int \sum_{\alpha \in \N^K} \exp \Ll( H_N^{\mathrm{disc}}(t,q,\sigma,\alpha) \Rr) \, v_\alpha \, \d P_N(\sigma),
\end{equation}

\begin{equation*}
H_N^{\mathrm{disc}}(t,q,\sigma,\alpha) := \sqrt{2t} H_N(\sigma) -  t N \xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)
+ \sqrt{2} \msf w^q \cdot \sigma - \sigma \cdot q_K  \sigma.
\end{equation*}

We denote by $\langle \cdot \rangle$ the associated Gibbs measure, and by $(\sigma,\alpha)$ the canonical random variable. That is, for every bounded measurable $f : \R^N \times \N^K \to \R$,

\begin{equation}
\label{e.def.disc.gibbs}
\la f(\sigma,\alpha) \ra := \frac{ \int \sum_{\alpha \in \N^K} f(\sigma, \alpha) \exp \big(H_N^{\mathrm{disc}}(t,q,\sigma,\alpha)\big) v_\alpha \, \d P_N(\sigma) } { \int \sum_{\alpha \in \N^K} \exp \big(H_N^{\mathrm{disc}}(t,q,\sigma,\alpha)\big) v_\alpha \, \d P_N(\sigma) }.
\end{equation}

We denote by $((\sigma^\ell, \alpha^\ell))_{\ell \in \N}$ independent copies of $(\sigma, \alpha)$ under $\la \cdot \ra$, also called replicas. We may also write $(\sigma',\alpha')$, $(\sigma'',\alpha'')$ for independent copies of $(\sigma,\alpha)$ in calculations in which only two or three such copies suffice. As is classical and will be explained in more details below (or see <cit.>), one can show that for every $k \in \{0,\ldots, K\}$,

\begin{equation}
\label{e.first.cascade.identity}
\E \la \1_{\{\alpha \wedge \alpha'= k\}} \ra = \zeta_{k+1} - \zeta_k.
\end{equation}

Using this, one can show the following Lipschitz continuity property of $\bar F_N$ as a function of the path $q$.

For every $t,t'\ge 0 $ and every piecewise-constant ${q},{q'} \in \C$, we have

\begin{align}
\label{e.barF_N_Lip}
\Ll|\bar F_N(t,{q}) - \bar F_N(t',{q'})\Rr|\leq \Ll|{q}-{q'}\Rr|_{L^1} + |t-t'| \, \sup_{|a| \le 1} |\xi(a)|.
\end{align}

As a consequence, the free energy $\bar F_N$ admits a unique extension to $\R_+ \times \C_1$ satisfying (<ref>) for every $t,t' \ge 0$ and $q,q' \in \C_1$.

In (<ref>) and throughout, for every $p \in [1,\infty]$, we use the notation

\begin{equation*}
|q-q'|_{L^p} := \Ll( \int_0^1 |q-q'|^p \Rr) ^{1/p},
\end{equation*}

with the usual understanding as an essential supremum when $p = \infty$. We also often use the shorthand $L^p := L^p([0,1]; S^D)$, and we recall that $\C_p := \C \cap L^p$.

The Lipschitz continuity in $q$ is in <cit.>; see also <cit.> for a proof when $D = 1$ using similar notation as here. Besides the verification of (<ref>), the main step of the proof is to show that if we allow for repetitions in the sequence $(q_k)_{0 \le k \le K}$, replacing the strict inequalities “$<$” in (<ref>) by “$\le$”, and follow along with the exact same construction of $\bar F_N$, then we end up with the same quantity as if we had removed the repetitions in the sequence $(q_k)_{0 \le k \le K}$ (and thereby reduced the value of $K$ and removed some “unused” $\zeta_k$'s in the representation (<ref>)) to begin with.

For the Lipschitz continuity in $t$, we use the Gaussian integration by parts in <cit.>, to write, for every $t \ge 0$ and piecewise-constant $q \in \mcl Q$,

\begin{equation}
\label{e.der.t}
\partial_t \bar F_N(t,q) = -\frac 1 N \E \la  \frac 1 {\sqrt{2t}} H_N(\sigma) - N \xi \Ll( \frac{\sigma \sigma^\intercal}{N} \Rr) \ra = \E \la \xi \Ll( \frac{\sigma \sigma'^\intercal}{N} \Rr) \ra .
\end{equation}

By the assumption (<ref>) on the support of $P_1$ and the Cauchy–Schwarz inequality, we have for every $\sigma$ in the support of $P_N$ that

\begin{equation}
\label{e.bound.on.sigma}
|\sigma \sigma^\intercal|^2 = \sigma \sigma^\intercal \cdot \sigma \sigma^\intercal =  |\sigma^\intercal \sigma|^2 = \sum_{i,j = 1}^N \Ll( \sum_{d= 1}^D \sigma_{d,i} \sigma_{d,j} \Rr) ^2 \le N^2.
\end{equation}

In particular, we have that

\begin{equation*}  %\label{e.}
\Ll| \partial_t \bar F_N(t,q) \Rr| \le \sup_{|a| \le 1} |\xi(a)|,
\end{equation*}

so the proof is complete.

We also recall the following observation justifying the identity in (<ref>).

For every $N \in \N$ and $q \in \mcl Q_1$, we have

\begin{equation}
\label{e.init.cond}
\bar F_N(0,q) = \bar F_1(0,q).
\end{equation}

The proof of this result is based on the fact that we chose $P_N$ to be a product measure, $P_N = P_1^{\otimes N}$.
By the Lipschitz property in Proposition <ref>, it suffices to show the identity (<ref>) for piecewise-constant $q$. The proof is then classical, for $D = 1$ it can be found in the first part of <cit.> using similar notation as here, and the argument applies verbatim to arbitrary $D$.

Given two metric spaces $\mathcal{X}$ and $\mathcal{Y}$, we say that a sequence $(g_n)_{n\in\N}$ of functions $g_n:\mathcal{X}\to\mathcal{Y}$ converges locally uniformly to some $g:\mathcal{X}\to\mathcal{Y}$, if $(g_n)_{n\in\N}$ converges to $g$ uniformly on every metric ball of $\mathcal{X}$.
By combining Propositions <ref> and <ref>, we can verify that the sequence $\bar F_N$ is precompact.

For every $p \in (1,+\infty]$, the sequence $(\bar F_N)_{N \in\N}$ is precompact for the topology of local uniform convergence in $\R_+ \times \C_p$.

The proof of this proposition relies on the following lemma, which states that for every $r  < p \in [1,+\infty]$, the embedding $\mcl Q_r \subset \mcl Q_p$ is compact.

Let $r \in (1,+\infty]$, and let $\Ll(q^{(n)}\Rr)_{n \in \N}$ be a sequence of paths in $\mcl Q$ such that

\begin{equation}
\label{e.compact.embed}
\sup_{n \in \N} \Ll|q^{(n)}\Rr|_{L^r} < +\infty.
\end{equation}

There exist a subsequence $\Ll(q^{(n_k)}\Rr)_{k \in \N}$ from $\Ll(q^{(n)}\Rr)_{n \in \N}$ and a path $q \in \mcl Q_r$ such that for every $p \in [1,+\infty)$ with $p < r$, the subsequence $\Ll(q^{(n_k)}\Rr)_{k \in \N}$ converges almost everywhere and in $L^p$ to $q$.

Let $M < +\infty$ denote the supremum in (<ref>). Since the path $q^{(n)}$ is increasing, we use Jensen's inequality to write, for every $u < 1$,

\begin{equation*}  %\label{e.}
\Ll|q^{(n)}(u)\Rr| \le \frac{1}{1-u} \int_u^1 \Ll|q^{(n)}(t)\Rr| \, \d t \le \frac 1 {1-u} \Ll|q^{(n)}\Rr|_{L^1} \le  \frac 1 {1-u} \Ll|q^{(n)}\Rr|_{L^r} \le \frac{M}{1-u}.
\end{equation*}

By a diagonal extraction argument, we can therefore extract a subsequence from $(q^{(n)})_{n \in \N}$ that converges pointwise at every rational number in $[0,1)$;  for every $u \in [0,1) \cap \Q$, we denote by $q^{(\infty)}(u)$ this limit. For every $u \in [0,1)$, we set

\begin{equation*}  %\label{e.}
q(u) := \lim_{\substack{t \in \Q \\ t \searrow u}} q^{(\infty)}(u).
\end{equation*}

Since $q^{(\infty)}$ is increasing, this limit is well-defined. By construction, the path $q$ belongs to $\mcl Q$, and whenever $u < 1$ is a point of continuity of $q$, we must have that $q^{(n)}(u)$ converges to $q(u)$ along the subsequence. In particular, Fatou's lemma ensures that ${q \in \mcl Q_r}$. Moreover, by the dominated convergence theorem, the convergence of $q^{(n)}$ to $q$ along the subsequence is valid in $L^p([0,u]; S^D)$ for every $u < 1$ and $p < +\infty$. So it suffices to verify that, for each $p < r$,

\begin{equation}
\label{e.neglect.last.piece}
\lim_{u \nearrow 1} \sup_{n \in \N} \int_u^1 \Ll|q^{(n)}(t)\Rr|^p \, \d t = 0.
\end{equation}

By Jensen's inequality, we have

\begin{equation*}  %\label{e.}
\Ll(\frac{1}{1-u} \int_u^1 \Ll|q^{(n)}(t)\Rr|^p \, \d t\Rr)^{1/p} \le \Ll(\frac{1}{1-u} \int_u^1 \Ll|q^{(n)}(t)\Rr|^r \, \d t\Rr)^{1/r} \le \frac{M}{(1-u)^{1/r}}.
\end{equation*}

The right-hand side is understood as $M$ when $r = +\infty$. The inequality above implies (<ref>) and therefore completes the proof.

We fix $p \in (1,+\infty]$. In view of Propositions <ref> and <ref>, it suffices to show that the intersection of any $L^p$ ball with the set $\mcl Q$ is compact in $L^1$, by the Arzelà-Ascoli theorem.  This follows from Lemma <ref>.

Proposition <ref> implies that if there exists a subsequence along which $\bar F_N$ converges pointwise to some limit, then the convergence in fact holds locally uniformly in $\R_+ \times \mcl Q_p$ for every $p > 1$.

Our next result records a monotonicity property of the mapping $q \mapsto \bar F_N(t,q)$. One way to think of this result is to say that the derivative $\partial_q \bar F_N(t,q,\cdot)$ always belongs to $\C_2$. Since we would like to use common language also concerning possible limits of $f$, which are not necessarily differentiable everywhere, we prefer to phrase it as a monotonicity property per se, using the notion of dual cones. We denote by $\C_2^*$ the cone dual to $\C_2$, which is defined by

\begin{align*}%\label{e.C_2^*}
\C_2^* := \Ll\{\kappa \in\cH \ \mid \ \forall q \in \C_2, \ \la \kappa, q \ra_\cH\geq 0\Rr\}.
\end{align*}

The following result can be found in <cit.>. We also give a proof here for the reader's convenience.

We have

\begin{equation}
\label{e.charact.Cstar}
\C_2^* = \left\{\kappa\in \cH \ \mid \ \forall t \in [0,1), \ \int_t^1\kappa(s)\d s\in\S^\D_+\right\}.
\end{equation}

We start by recalling, e.g. from <cit.>, that for every $a \in S^D$, we have the equivalence

\begin{equation}
\label{e.charact.SD+}
a \in S^D_+ \quad \iff \quad \forall b \in S^D_+, \ a \cdot b \ge 0.
\end{equation}

(In other words, the cone $S^D_+$ is self-dual.)
We denote by $K$ the set on the right side of (<ref>). Let $\kappa \in K$ and $q \in \C_2$ be a continuously differentiable path, whose derivative we denote by $\dot q$. Integrating by parts, we see that

\begin{align}
\label{e.calc.charact.C2star}
\la \kappa,q \ra_\cH
& = \int_0^1 \kappa(u) \cdot q(u) \,\d u
\\
\notag
& = q(0) \cdot \int_0^1 \kappa(u) \, \d u + \int_0^1 \kappa(u) \cdot \int_0^u \dot q(t) \, \d t \, \d u
\\
\notag
& = q(0) \cdot \int_0^1 \kappa(u) \, \d u + \int_0^1 \dot q(t) \cdot \int_t^1 \kappa(u) \, \d u \, \d t.
\end{align}

Using that $q$ is increasing and (<ref>), we see that $\dot q$ must take values in $S^D_+$. Since we also have $q(0) \in S^D_+$, we conclude that $\la \kappa,q \ra_\cH \ge 0$. Arguing by approximation, this result can be extended to every $q \in \C_2$. We have thus shown that $K \subset \C_2^*$.

Conversely, let $\kappa \in \C_2^*$. For every continuous path $\dot q \in C([0,1];S^D_+)$, the path defined for every $u \in [0,1]$ by

\begin{equation*}  %\label{e.}
q(u) := \int_0^u \dot q(t) \, \d t
\end{equation*}

belongs to $\C_2$, and the notation $\dot q$ is consistent. We conclude from the calculation in (<ref>) that

\begin{equation*}  %\label{e.}
\int_0^1 \dot q(t) \cdot \int_t^1 \kappa(u) \, \d u \, \d t \ge 0.
\end{equation*}

Using that this property holds for every choice of $\dot q \in C([0,1];S^D_+)$, that the mapping $t \mapsto \int_t^1 \kappa(u) \, \d u$ is continuous, and the characterization in (<ref>), we conclude that $\kappa \in K$, thereby finishing the proof.

For every subset $G$ of $\cH$ and function $g:G\to\R$, we say that $g$ is  $\C_2^*$-increasing if for every $q, q' \in  G$, we have

\begin{align}\label{e.C_2^*-nondec}
{q}-{q'}\in \C^*_2\quad \implies \quad g({q}) \geq g({q'}).
\end{align}

The following result is a rephrasing of <cit.>.

For every $t\geq 0$ and $N\in\N$, the mapping $\bar F_N(t,\cdot)$ is $\C_2^*$-increasing.

It will be very useful for our purposes to notice that the free energy $\bar F_N$ is locally semi-concave as a function of $q$. We first obtain a local semi-concavity property for regularly-spaced piecewise-constant paths, and then also derive a version that concerns sufficiently regular continuous paths. We recall that, for every $a \in S^D_{+}$, we denote by $\ellipt(a)$ the ratio between the largest and the smallest eigenvalues of $a$.

There exists a constant $C < +\infty$ depending only on $D$ and $\xi$ such that the following holds. Let $(\zeta_k)_{1 \le k \le K}$ be as in (<ref>), let $t,t' \ge 0$, $0 = q_{-1} \le q_0 < \cdots < q_K\in S^D_+$, and $0 = q'_{-1} \le q'_0 < \cdots < q'_K\in S^D_+$, and set

\begin{equation}
\label{e.def.disc.q.q'}
q := \sum_{k = 0}^K q_k \1_{\Ll[\zeta_k, \zeta_{k+1} \Rr)} \quad \text{ and } \quad q' := \sum_{k = 0}^K q_k' \1_{\Ll[\zeta_k, \zeta_{k+1} \Rr)}.
\end{equation}

Letting $c \in (0,1]$ be such that $t,t' \ge c$ and

\begin{equation}
\label{e.semiconc.cond.q}
\forall k \in \{0, \ldots, K\}, \qquad \frac{c}{K+1} \id \le q_{k} - q_{k-1}  \quad \text{ and } \quad \ellipt(q_k - q_{k-1}) \le c^{-1},
\end{equation}

and similarly with $q$ replaced by $q'$ in (<ref>), we have, for every  $\lambda \in [0,1]$,

\begin{multline}
\label{e.semiconc.disc}
(1-\lambda) \bar F_N(t,q) + \lambda \bar F_N(t',q') - \bar F_N \big( (1-\lambda)(t,q) + \lambda(t',q') \big)
\\
\le C\lambda (1-\lambda) c^{-2} \Ll(  (t-t')^2
+  (K+1)\sum_{k=0}^K\Ll|\Ll(q_k-q_{k-1}\Rr)-\Ll(q'_k-q'_{k-1}\Rr)\Rr|^2\Rr).
\end{multline}

We start with a preliminary step.

Step 0.
Recall that we denote by $S^D_{++}$ the set of $D$-by-$D$ positive definite matrices. For every $h \in S^D_{++}$ and  $a\in\S^\D$, we set

\begin{align*}
\mathsf{D}_{\sqrt{h}}(a) := \lim_{\eps\to 0}\eps^{-1}\Ll(\sqrt{h+\eps a}-\sqrt{h}\Rr),
\end{align*}

as well as

\begin{align*}
\mathsf{D}^2_{\sqrt{h}}(a) = \lim_{\eps\to  0} \ep^{-1} \Ll(\mathsf{D}_{\sqrt{h+\eps a}}(a) - \mathsf{D}_{\sqrt{h}}(a)\Rr).
\end{align*}

We denote by $\lambdamin(h)$ the smallest eigenvalue of $h$.
In this step, we show that

\begin{align}
\label{e.|D|<...}
\Ll|\mathsf{D}_{\sqrt{h}}(a)\Rr|\leq \frac 1 2 |a|\lambdamin(h)^{-\frac 1 2}
\end{align}

\begin{align}\label{e.D^2<...}
\Ll|\mathsf{D}^2_{\sqrt{h}}(a)\Rr|\leq \frac 1 4|a|^2 \lambdamin(h)^{-\frac 3 2}.
\end{align}

For two matrices $b,b'\in \R^{D\times D}$, we write $b\circledast b' = \frac{1}{2}( bb' + b'b)$ for the symmetrized matrix multiplication.
Writing $h+\eps a = \Ll(\sqrt{h} +\eps \mathsf{D}_{\sqrt{h}}(a) +o(\eps)\Rr)^2$ and expanding the square, we see that

\begin{align}\label{e.sqrt(h)D+...}
2\sqrt{h}\circledast \mathsf{D}_{\sqrt{h}}(a)=a
\end{align}

Let $\lambda = \mathrm{diag}(\lambda_1,\dots,\lambda_D)$ and $o$ be an orthogonal matrix such that $h = o \lambda o^\intercal$. We have $\sqrt{h} = o\sqrt{\lambda} o^\intercal$, where $\sqrt{\lambda}=\mathrm{diag}(\sqrt{\lambda_1},\dots,\sqrt{\lambda_D})$.
Writing $D' := o^\intercal \mathsf{D}_{\sqrt{h}}(a)  o $ and $a' := o^\intercal a o$, we get from (<ref>) that for every $d,d' \in \{1,\ldots D\}$,

\begin{equation*}  %\label{e.}
\end{equation*}

This implies that

\begin{equation*}  %\label{e.}
|\mathsf{D}_{\sqrt{h}}(a)| = |D'|\leq \frac 1 2 |a'| \ {\max_{d \le D} \lambda_d^{-1/2}}= \frac 1 2 |a| \lambdamin(h)^{-\frac{1}{2}},
\end{equation*}

as announced in (<ref>).

Turning to (<ref>), we start with (<ref>) applied to $h+\eps a$:

\begin{align*}
2\sqrt{h+\eps a }\circledast \mathsf{D}_{\sqrt{h+\eps a}}(a) =a.
\end{align*}

By the definitions of the derivatives, this means that, as $\eps$ tends to zero,

\begin{align*}
2\Ll(\sqrt{h}+\eps \mathsf{D}_{\sqrt{h}}(a)+o(\eps)\Rr)\circledast \Ll(\mathsf{D}_{\sqrt{h}}(a)+\eps \mathsf{D}^2_{\sqrt{h}}(a)+o(\eps)\Rr) = a
\end{align*}

Using (<ref>) to cancel out terms and sending $\eps$ to zero, we get

\begin{align*}
\sqrt{h}\circledast \mathsf{D}^2_{\sqrt{h}}(a) = -  \Ll(\mathsf{D}_{\sqrt{h}}(a)\Rr)^2.
\end{align*}

Using the same diagonalization argument as for (<ref>) yields that

\begin{align}
\label{e.D2squareroot}
\Ll|\mathsf{D}^2_{\sqrt{h}}(a)\Rr|\leq  \Ll|\Ll(\mathsf{D}_{\sqrt{h}}(a)\Rr)^2\Rr| \lambdamin(h)^{-\frac{1}{2}}.
\end{align}

Observing also that, for every matrix $b \in S^D$ with eigenvalues $\lambda_1,\ldots, \lambda_D$,

\begin{equation*}  %\label{e.}
\Ll|b^2\Rr|^2 =  \sum_{d = 1}^D\lambda_d^4 \le \Ll(\max_{d \le D} \lambda_d^2 \Rr)\, \sum_{d = 1}^D\lambda_d^2 \le \Ll(\sum_{d = 1}^D \lambda_d^2\Rr)^2 = |b|^4,
\end{equation*}

we obtain (<ref>) from (<ref>) and (<ref>).

Now that this preliminary step is complete, we turn to the proof of Proposition <ref> per se, which will be decomposed into two steps. For notational clarity we will only show the semi-concavity property in (<ref>) in the special case of $t = t'$, but it will be clear from the proof that the argument also covers the general case. To lighten notation, we therefore fix $t \ge 0$ from now on.

Step 1.
For every $s\geq0$ and $y = (y_0,\ldots, y_K) \in (\S^\D_+)^{K+1}$, we define

\begin{align*}
\tilde H_N^{s,y}(\sigma,\alpha) & =\sqrt{2}sH_N(\sigma) - s^2N\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr) +  \sqrt{2}\sum_{k=0}^Ky_k {z}_{\alpha_{|k}}\cdot \sigma - \sum_{k=0}^K y_k^2\cdot\sigma\sigma^\intercal,
\\
G_N(s,y)  & = -\frac{1}{N}\E\log \int \sum_{\alpha \in \N^K}  \exp\Ll(\tilde H_N^{s,y}(\sigma,\alpha)\Rr)\, v_\alpha \, \d P_N(\sigma).
\end{align*}

Notice that, with notation as in (<ref>)-(<ref>), we have

\begin{equation}
\label{e.turn.F.to.G}
\bar F_N(t,{q}) = G_N(\sqrt{t},(\sqrt{{q}_k-{q}_{k-1}})_{0\leq k\leq K}).
\end{equation}

As already mentioned, we will keep $t$ fixed in this proof for notational clarity, but the required modifications for the general case should be clear. We therefore fix $s = \sqrt{t}$ from now on and suppress it from the notation, simply writing $G_N(y)$ in place of $G_N(\sqrt{t}, y)$.

In this step, we show that the Hessian of $G_N$ is uniformly bounded from above. With the notation $y  = (y_0,\ldots, y_K) \in (S^D_+)^{K+1}$, we denote by $\partial_{y_k} G_N(y)$ the derivative with respect to the $k$-th coordinate of $y$. This derivative is an element of $S^D$. In this proof, it is at times convenient to think of each coordinate $y_k$ of $y$ simply as a $\frac{D(D+1)}{2}$-dimensional vector, so that we can view $\partial_{y_k} \partial_{y_{l}} G_N(y)$ as a $\frac{D(D+1)}{2}$-by-$\frac{D(D+1)}{2}$ matrix, and we can for instance write the Taylor expansion, as $y' =(y'_0,\ldots, y_K') \in (S^D_+)^{K+1}$ tends to $y \in (S^D_+)^{K+1}$,

\begin{multline*}  %\label{e.}
G_N(y') = G_N(y)  + \sum_{k = 0}^K (y_k'-y_k)\cdot \partial_{y_k} G_N(y) \\
+ \frac 1 2 \sum_{k,l = 0}^K (y'_k - y_k) \cdot \partial_{y_k} \partial_{y_l} G_N(y) (y'_l-y_l) + o(|y-y'|^2).
\end{multline*}

Notice that our definitions of scalar products are consistent regardless of whether we regard $y_k$ as a matrix or as a $\frac{D(D+1)}{2}$-dimensional vector. We also abbreviate the expansion above as

\begin{equation*}  %\label{e.}
G_N(y') = G_N(y)  + (y'-y)\cdot \nabla G_N(y)
+ \frac 1 2 (y'-y) \cdot \nabla^2 G_N(y) (y'-y) + o(|y-y'|^2).
\end{equation*}

With this notation in place, the goal of this step is to show that, for every $k \in \{0,\ldots, K\}$, we have

\begin{align}\label{e.|nablaG_N|<}
\Ll|\partial_{y_k} G_N(y)\Rr|\leq 2|y_k|,
\end{align}

and that for every $a \in (S^D)^{K+1}$,

\begin{align}\label{e.zGz<2}
a\cdot  \nabla^2 G_N(y)a\leq 2|a|^2.
\end{align}

We start with the first-order derivative of $G_N$. For every $k\in\{0,\dots,K\}$ and $a \in \S^\D_+$, using the Gaussian integration by parts in <cit.>, we can compute

\begin{align*}
a\cdot \partial_{y_k} G_N(y)
& = \frac{\d}{\d\eps}G_N(y_1,\dots, y_{k-1},y_k+\eps a,y_{k+1},\dots,y_K)\Big|_{\eps =0}
\\
& = -\frac{1}{N} \E \la \sqrt{2} az_{\alpha_{|k}}\cdot\sigma - 2 ay_k \cdot \sigma\sigma^\intercal \ra = \frac{2}{N}\E \la ay_k \cdot \sigma\sigma'^\intercal \1_{\{\alpha_{|k} = \alpha'_{|k}\}}\ra.
\end{align*}

By the Cauchy–Schwarz inequality and (<ref>), we have

\begin{align}
\label{e.bound.on.sisi'}
|\sigma \sigma'^\intercal|^2
= \sigma \sigma'^\intercal \cdot \sigma \sigma'^\intercal
= \sigma^\intercal \sigma \cdot \sigma'^\intercal \sigma'
\le |\sigma^\intercal \sigma| \, |\sigma'^\intercal \sigma'| \le N^2.
\end{align}

Combining the last two displays yields that

\begin{equation*}  %\label{e.}
\Ll| a\cdot \partial_{y_k} G_N(y) \Rr| \le 2 |a| \, |y_k|,
\end{equation*}

which is (<ref>).

Turning to the proof of (<ref>), we first observe that, for every $a,b \in \R^{D\times D}$ and $\lambda \in \R$,

\begin{align*}
(1-\lambda)a^2 + \lambda b^2 = ((1-\lambda)a+\lambda b)^2 + \lambda(1-\lambda)(a-b)^2.
\end{align*}

Using this and Hölder's inequality, we obtain that, for every $y,y' \in (S^D)^{K+1}$ and $\lambda \in [0,1]$,

\begin{align*}
&(1-\lambda) G_N(y)+\lambda G_N(y') = -\frac{1}{N}\E\log (\cdots)^{1-\lambda}(\cdots)^\lambda
\\
&\leq -\frac{1}{N}\E \log \int \sum_{\alpha \in \N^K}  \exp\Ll((1-\lambda)\tilde H_N^{s,y}(\sigma,\alpha) +\lambda \tilde H_N^{s,y'}(\sigma,\alpha)\Rr)\, v_\alpha \, \d P_N(\sigma)
\\
&= -\frac{1}{N}\E \log \int\sum_{\alpha \in \N^K}  \exp\Ll(\tilde H_N^{s,(1-\lambda)y+\lambda y'}(\sigma,\alpha)+ r \Rr)\, v_\alpha \, \d P_N(\sigma)
\end{align*}

\begin{align*}
r := - \lambda(1-\lambda) \sum_{k=0}^K\Ll(y_{k}-y_{k}'\Rr)^2\cdot \sigma\sigma^\intercal  \ge - \lambda(1-\lambda) N\Ll|y-y'\Rr|^2,
\end{align*}

using also (<ref>) in the last inequality.
We thus deduce that

\begin{align*}
(1-\lambda) G_N\Ll(y\Rr)+\lambda G_N\Ll(y'\Rr) \leq G_N\Ll((1-\lambda)y+\lambda y'\Rr) + \lambda(1-\lambda)\Ll|y-y'\Rr|^2.
\end{align*}

Substituting $y+\eps a$ for $y'$ in the above display, setting $\lambda$ to be $\frac{1}{2}$, and then sending $\eps$ to zero, we obtain (<ref>).

Step 2.
In this step, we change variables and use the results of the previous step to bound the second derivatives in the $q_k$'s of $\bar F_N$.
For every $x\in (\S^\D_{+})^{K+1}$, we write $\sqrt{x} := \Ll(\sqrt{x_k}\Rr)_{1\leq k\leq K}$ and set $\tilde G_N(x) := G_N\Ll(\sqrt{x}\Rr)$. For every $x \in (S^D_{++})^{K+1}$ and $a \in (S^D)^{K+1}$, we have

\begin{align*}  %\label{e.}
\frac{\d}{\d \ep} \tilde G_N(x+\ep a) = \sum_{k = 0}^K D_{\sqrt{x_k + \ep a_k}}(a_k) \cdot \partial_{y_k} G_N(\sqrt{x + \ep a}),
\end{align*}

and thus

\begin{align*}
a \cdot  \nabla^2 \tilde G_N(x) a
& = \frac{\d^2}{\d \eps^2} \tilde G_N(x+\eps a)\Big|_{\eps =0}
\\
&= \sum_{k,l=0}^K \mathsf{D}_{\sqrt{x_k}}(a_k)\cdot \partial_{y_k} \partial_{y_l} G_N\Ll(\sqrt{x}\Rr) \mathsf{D}_{\sqrt{x_l}}(a_l)  + \sum_{k=0}^K \mathsf{D}_{\sqrt{x_k}}^2(a_k)\cdot \partial_{y_k} G_N\Ll(\sqrt{x}\Rr).
\end{align*}

Applying the estimates (<ref>) and  (<ref>), and then (<ref>) and (<ref>), we get

\begin{align*}
a\cdot   \nabla^2 \tilde G_N(x) a
& \leq 2\sum_{k=0}^K \Ll| \mathsf{D}_{\sqrt{x_k}}(a_k) \Rr|^2+ 2\sum_{k=0}^K \Ll|\mathsf{D}^2_{\sqrt{x_k}}(a_k)\Rr|\Ll|\sqrt{x_k}\Rr|
\\
& \le \frac 1 2 \sum_{k = 0}^K |a_k|^2 \lambdamin(x_k)^{-1} + \frac 1 2 \sum_{k = 0}^K |a_k|^2 \lambdamin(x_k)^{-\frac 3 2} |\sqrt{x_k}|.
\end{align*}

Notice that

\begin{equation*}  %\label{e.}
|\sqrt{x_k}|^2 \le D \, \ellipt(x_k) \lambdamin(x_k).
\end{equation*}

In particular, for every $x \in (S^D_+)^{K+1}$ satisfying

\begin{equation}
\label{e.constraints.x}
\mbox{for every $k \in \{0,\ldots, K\}$, } \ \frac{c}{K+1} \id \le x_k \quad \text{and} \quad \ellipt(x_k) \le c^{-1},
%,  \quad \frac{c}{K+1} \id \le x'_k, \quad \ellipt(x'_k) \le c^{-1},
\end{equation}

we have that, for some constant $C < +\infty$ depending only on $D$,

\begin{equation*}  %\label{e.}
a \cdot  \nabla^2 \tilde G_N(x) a  \le C(K+1)c^{-2} |a|^2.
\end{equation*}

For every $x,x' \in (S^D_+)^{K+1}$ with $x$ satisfying (<ref>) and $x'$ satisfying these same bounds as $x$ displayed in (<ref>), we therefore have that, for every $\lambda \in [0,1]$,

\begin{align*}
(1-\lambda) \tilde G_N(x) + \lambda \tilde G_N\Ll(x'\Rr) \leq \tilde G_N\Ll((1-\lambda)x +\lambda x'\Rr) + C \lambda (1-\lambda)  c^{-2} (K+1)|x-x'|^2.
\end{align*}

We now recall from (<ref>) that for $0 = q_{-1} < q_0 < \cdots < q_K$, and for $q$ as in (<ref>), we have that $\bar F_N(t,q) = \tilde G_N\big((q_k - q_{k-1})_{0 \le k \le K}\big)$. For $q,q'$ as in the assumptions of the theorem, a change of variables in the previous display therefore yields the desired result.

We now extend this semi-concavity result to the case of continuous paths. For every $c > 0$, we write

\begin{multline}
\label{e.def.C_c}
\C_{\uparrow,c} :=\big \{{q}\in\C_2 \ \mid \ q(0) = 0 \ \text{ and } \ \forall u \le v \in [0,1), \quad
q(v) - q(u) \ge c (v-u) \id \quad
\\ \text{and } \quad \ellipt(q(v) - q(u)) \le c^{-1}
\big\}.
\end{multline}

We observe that $\bigcup_{c>0}\C_{\uparrow,c}=\C_\uparrow$, where we recall that $\C_\uparrow$ was introduced in the sentence containing (<ref>). For every path $q \in \C$, we denote by $\dot q$ its distributional derivative. Whenever this distributional derivative can be represented as an element of $L^2$, we write that $\dot q \in L^2$.

There exists a constant $C < +\infty$ such that, for every $N\in\N$, $c>0$, $t,t' \ge c$, $q,q' \in \C_{\uparrow,c}$ with $\dot q- \dot q' \in L^2$, and $\lambda \in [0,1]$,

\begin{multline}
\label{e.semiconc}
(1-\lambda)\bar F_N(t,{q}) + \lambda \bar F_N(t',q')
- \bar F_N \big((1-\lambda)(t,q) + \lambda(t',q')\big)
\\
\leq  C\lambda(1-\lambda)c^{-2}\Ll((t-t')^2+\Ll|\dot{q}-\dot{q}'\Rr|^2_\cH\Rr).
\end{multline}

By a density argument and Proposition <ref>, it suffices to show the result for smooth paths $q, q'$. We therefore choose two paths $q,q'$ in $\C_{\uparrow,c} \cap C^\infty([0,1];S^D)$, and for each $K\in\N$ and $k\in\{0,\dots,K\}$, we set

\begin{equation*}  %\label{e.}
q_k := q \Ll( \frac{k}{K+1} \Rr) ,
\end{equation*}

as well as the piecewise-constant path $q^K \in \C$ defined by

\begin{equation*}  %\label{e.}
q^K := \sum_{k = 0}^{K} q_k \1_{ \Ll[ \frac{k}{K+1}, \frac{k+1}{K+1} \Rr)} .
\end{equation*}

We do the same construction for the path $q'$, obtaining a family of parameters $(q'_k)_{0 \le k \le K}$ and a path $q'^K$. As always, we set $q_{-1} = q'_{-1} = 0$. An application of Proposition <ref> yields

\begin{multline}
\label{e.semiconc.K}
(1-\lambda)\bar F_N\Ll(t,q^K\Rr) + \lambda \bar F_N\Ll(t',q'^K\Rr) - \bar F_N \Ll( (1-\lambda)\Ll(t,q^K\Rr) + \lambda\Ll(t',q'^K\Rr)\Rr)
\\
\le
C\lambda (1-\lambda) c^{-2} \Ll(  (t-t')^2
+  (K+1)\sum_{k=0}^K\Ll|\Ll(q_k-q_{k-1}\Rr)-\Ll(q'_k-q'_{k-1}\Rr)\Rr|^2\Rr).
\end{multline}

By Proposition <ref>, the left side of this inequality converges to the left side of (<ref>). Under our smoothness assumption on $q$, the difference

\begin{equation*}  %\label{e.}
\dot q \Ll( \frac{k}{K+1} \Rr) - \frac{q_k - q_{k-1} }{K+1} = \dot q \Ll( \frac{k}{K+1} \Rr)  - \frac{1}{K+1} \Ll( q \Ll( \frac{k}{K+1} \Rr) -q \Ll( \frac{k-1}{K+1} \Rr) \Rr)
\end{equation*}

converges to zero as $K$ tends to infinity, uniformly over $k \in \{0,\ldots, K\}$. Since the same goes for the path $q'$, we can replace the second term in the large parentheses on the right side of (<ref>) by

\begin{equation*}  %\label{e.}
\frac{1}{K+1} \sum_{k = 0}^K \Ll| \dot q \Ll( \frac{k}{K+1} \Rr) - \dot q' \Ll( \frac{k}{K+1} \Rr) \Rr|^2 ,
\end{equation*}

up to an error that tends to zero as $K$ tends to infinity. This is a Riemann-sum approximation of the integral $|\dot q - \dot q'|_{L^2}^2$, so under our smoothness assumption on $q$ and $q'$, it converges to this integral as $K$ tends to infinity. Recalling (<ref>), we thus obtain the desired result.

In short, we think of Proposition <ref> as stating that the function $q \mapsto \bar F_N(t,q)$ is locally semi-concave with respect to the quadratic form $q \mapsto |\dot q|_{L^2}^2$. One of the useful consequences of this fact for our purposes is that if $\bar F_N$ converges to some limit $f$, and if $q \in \C_\uparrow$ is a point of differentiability of $f(t,\cdot)$, then the $q$-derivative of $\bar F_N$ at $(t,q)$ must converge to the $q$-derivative of $f$ at $(t,q)$. We postpone a precise statement to this effect to Section <ref>.

§ CONTINUOUS CASCADES AND OVERLAP APPROXIMATIONS

Recall that in Section <ref>, we defined the free energy $\bar F_N(t,q)$ for piecewise-constant paths $q$, and then extended this function to all $q \in \C_1$ by continuity. In this section, we lay the groundwork that will allow us to give a direct definition of $\bar F_N(t,q)$ for arbitrary paths $q$. We also show some useful properties of the objects involved, and how to identify them with the discrete objects introduced in Section <ref> when the path $q$ is piecewise constant. One of the advantages of the direct construction is that it allows us to also directly manipulate the associated Gibbs measure for continuous paths $q$.

The basic object in this construction is a Poisson–Dirichlet cascade whose overlap is uniformly distributed over the interval $[0,1]$.
For the reader's convenience, we briefly review its construction here. Recall from the previous section that for each choice of the parameters $(\zeta_{k})_{1 \le k \le K}$ as in (<ref>), we can build the associated Poisson–Dirichlet cascade $(v_\alpha)_{\alpha \in \N^K}$, which encodes the weights of a random probability measure on $\N^K$. For each realization of $(v_\alpha)_{\alpha \in \N^K}$, we construct independent random variables $(\alpha^\ell)_{\ell \ge 1}$ taking values in $\N^K$ that are sampled according to these weights. Choosing also real numbers $(q_k)_{0 \le k \le K}$ satisfying (<ref>), we then build the “overlap” array $(q_{\alpha^\ell \wedge \alpha^{\ell'}})_{\ell, \ell' \ge 1}$, where we recall the notation $\wedge$ from (<ref>). We have thus defined a mapping taking any piecewise-constant path $q \in L^\infty([0,1];\R)$ of the form in (<ref>) as input, and returning a random array indexed by $\N\times \N$ (and in truth it is rather the law of this random array that is unambiguously well-defined). If we take any sequence of piecewise-constant paths $q^{(n)}$ that converges in $L^\infty([0,1];\R)$ to the identity map, then the sequence of resulting overlap arrays converges in law in the sense of finite-dimensional distributions, by <cit.>. The law of the limit does not depend on the approximating sequence, and we denote it by $R = (R_{\ell,\ell'})_{\ell,\ell' \ge 1}$. By the Dovbysh–Sudakov theorem in <cit.>, this limit array can be represented as a “true” overlap array. That is, there exists a separable Hilbert space $\mathfrak{H}$, whose scalar product we denote by $\wedge$, and a random probability measure $\fR$ on the unit sphere of $\mathfrak{H}$, such that if $(\alpha^\ell)_{\ell \ge 1}$ denote independent random variables sampled according to $\fR$, then the overlap array $(\alpha^\ell \wedge \alpha^{\ell'})_{\ell, \ell' \ge 1}$ has the same law as the limit array $R$ constructed above. In this statement, the law of $(\alpha^\ell \wedge \alpha^{\ell'})_{\ell, \ell' \ge 1}$ is understood after we average both over the sampling of the random variables $(\alpha^\ell)_{\ell \ge 1}$ and over the randomness inherent to the random probability measure $\fR$ itself.

We denote by $\langle \cdot \rangle_{\fR}$ the expectation with respect to the sampling of the independent variables $(\alpha^\ell)_{\ell \ge 1}$, defined for each fixed realization of $\fR$, and we denote by $(\bs \Omega, \P)$ the probability space, with associated expectation $\E$, corresponding to the sampling of $\fR$ itself. It will at times be convenient to denote explicitly that $\fR$ is a random variable, and we use the notation $\bomega \mapsto \fR_{\bomega}$ when we want to clarify that $\fR$ is a mapping from $\bs{\Omega}$ to the space of probability measures on $\mfk H$.

By construction, the overlap $\alpha^1 \wedge \alpha^2$ is distributed uniformly over $[0,1]$ under $\P \fR$. (Strictly speaking, we should rather write $\P \, \fR^{\otimes \N}$ or $\P \, \fR^{\otimes 2}$ in place of $\P \fR$, but we allow ourselves this abuse of notation.) In particular, we have $\P \fR$-almost surely that $\alpha^1 \wedge \alpha^2 \in [0,1]$. Equivalently, denoting by $\mfk U$ the (random) support of the measure $\fR$, we have with $\P$-probability one that for every $\alpha^1, \alpha^2 \in \mfk U$,

\begin{equation*}
\al^1 \wedge \al^2 \ge 0.
\end{equation*}

As a direct consequence of the construction of $\fR$ from discrete tree structures, we have with $\P\fR$-probability one that

\begin{equation}
\label{e.fR.ultrametric}
\alpha^1 \wedge \alpha^3 \ge \min\Ll(\alpha^1 \wedge \alpha^2, \alpha^2 \wedge \alpha^3\Rr).
\end{equation}

Equivalently, this states that $\P$-almost surely, the relation (<ref>) holds for every $\alpha^1, \alpha^2, \alpha^3$ in $\mfk U$. Since $\mfk U$ is a subset of the unit sphere of $\mfk H$, this property states that the set $\mfk U$ is ultrametric. Explicitly, this means that $\P$-almost surely, we have for every $\alpha^1, \alpha^2, \alpha^3 \in \mfk U$ that

\begin{equation*}
\Ll|\alpha^1 - \alpha^3\Rr|_{\mfk H} \leq \max \Ll( \Ll|\alpha^1 - \alpha^2\Rr|_{\mfk H}, \Ll|\alpha^2 - \alpha^3\Rr|_{\mfk H} \Rr) ,
\end{equation*}

where $|\cdot|_{\mfk H}$ denotes the norm in $\mfk H$.
The set $\mfk U$ is the continuous analogue of the leaves $\N^K$ of the tree $\mcl A$ that appeared in the construction of the discrete cascades.

To summarize, there exists a measurable subset $\bs \Omega^*$ of $\bs \Omega$ with $\P(\bs \Omega^*) = 1$ such that for every $\bomega \in \bs \Omega^*$, we have that

\begin{equation}
\label{e.good.mfkU}
\begin{cases}
\mbox{$\mfk U_\bomega$ is a subset of the unit sphere in $\mfk H$},\\
\forall \al^1, \al^2 \in \mfk U_\bomega, \quad \al^1 \wedge \al^2 \ge 0,\\
\forall \al^1, \al^2, \al^3 \in \mfk U_\bomega, \quad \alpha^1 \wedge \alpha^3 \ge \min(\alpha^1 \wedge \alpha^2, \alpha^2 \wedge \alpha^3).
\end{cases}
\end{equation}

Recall that in the construction of the free energy for piecewise-constant paths, we used the random Gaussian field in (<ref>). Once such a Gaussian field is constructed for $N = 1$, it is easy to extend the construction to general $N$ by using independent copies. Our next goal is to construct an analogous object in the setting of continuous cascades. We recall that we write $\mcl Q_\infty = \mcl Q \cap L^\infty$. By monotonicity, for every $q \in \mcl Q_\infty$, the limit

\begin{equation}
\label{e.def.q.1}
q(1) := \lim_{u \nearrow 1} q(u) \in S^D_+
\end{equation}

is well-defined. We always extend a path $q \in \mcl Q_\infty$ by continuity at $1$ according to (<ref>). With this convention, every path $q \in \mcl Q_\infty$ is continuous at $1$.

We fix $\mfk U = \mfk U_\bomega$ such that (<ref>) holds, and let $q \in \mcl Q_\infty$. There exists an $\R^\D$-valued  centered Gaussian process $(w^q(\alpha))_{\alpha \in \mathfrak{U}}$ such that for every $\alpha, \alpha' \in \mfk U$,

\begin{align}
\label{e.gaussian_cascade}
\E  \Ll[w^q(\alpha) w^q(\alpha')^\intercal \Rr] = q(\alpha\wedge\alpha').
\end{align}

Moreover, if we display the randomness explicitly by writing $\omega \mapsto w^q(\alpha,\omega)$ for $\omega$ varying in the underlying probability space $\Omega$, then we can construct $w^q$ in such a way that the mapping $(\alpha,\omega)\mapsto w^q(\alpha,\omega)$ is jointly measurable.

We understand that the set $\mathfrak{U}$ is equipped with the Borel $\sigma$-algebra induced by the norm topology of $\mfk H$.

Since we aim to prove Proposition <ref> by appealing to the Kolmogorov extension theorem, we need to construct such a process indexed by any finite subset of $\mathfrak{U}$ and verify the consistency condition. When $D = 1$, we thus need to show that for every $n \in \N$ and $\alpha^1, \ldots, \alpha^n \in \mfk U$, the matrix $(q(\alpha^i\wedge\alpha^j))_{1\leq i,j\leq n}$ is positive semi-definite. This can be obtained by observing that the matrix $(q(\alpha^i\wedge\alpha^j))_{1\leq i,j\leq n}$ is ultrametric in the sense of <cit.>, and by quoting <cit.> to conclude.

We can also give a direct proof, which is more in the spirit of the construction of the Gaussian process itself and clearly applies for general $D \in \N$.
As a first step, we show that any finite ultrametric set can be embedded into the leaves of a tree in a natural way. This result is classical but we give a brief proof for the reader's convenience.

Let $\{\alpha^1,\dots,\alpha^n\}$ be an ultrametric subset of the unit sphere in $\mathfrak{H}$. Let $K\ge 0$ be an integer and let real numbers

\begin{align*}
s_0< s_1< \dots< s_K=1
\end{align*}

be such that $\{s_k\}_{k=0}^K = \{\alpha^i\wedge\alpha^j\}_{1\leq i,j\leq n}$. There exists a rooted tree of depth $K$ with leaves $\{\bal^i\}_{1\leq i\leq n}$ such that for every $i,j \in \{1,\ldots, n\}$, we have

\begin{align}\label{e.overlap=tree-overlap}
\alpha^i\wedge\alpha^j = s_{\bal^i\wedge \bal^j},
\end{align}

where $\bal^i\wedge \bal^j$ denotes the depth of the most recent common ancestor to $\bal^i$ and $\bal^j$.

We recall that for every $i,j,l \in \{1\ldots, n\}$, we have

\begin{equation}
\label{e.ultrametric}
\alpha^{i} \wedge \alpha^{l} \ge \min(\alpha^i \wedge \alpha^{j}, \alpha^{j} \wedge \alpha^{l}).
\end{equation}

For each $k \in \{0,\ldots, K\}$ and $i,j \in \{1,\ldots, n\}$, we write
$i \sim_k j$ if and only if $\alpha^i \wedge \alpha^j \ge s_k$. By (<ref>), the relation $\sim_k$ is an equivalence relation. We define the set of nodes at depth $k$ (of the tree we are going to construct) to be the set of equivalence classes of the relation $\sim_k$. For $k = K$, the equivalence classes are simply singletons since $s_K = 1$, and we write $\Lambda^i := \{i\}$. By definition, for each $k \ge 1$, an equivalence class for the relation $\sim_k$ must be a subset of exactly one equivalence class for the relation $\sim_{k-1}$. We draw an edge between the equivalence classes at levels $k$ and $k-1$ whenever this occurs. This construction results in a graph of depth $K$ without loops. At level $0$, there is only one equivalence class since $\al^i \wedge \al^j \ge s_0$ for every $i,j \in \{1,\ldots,n\}$. So our graph is connected, i.e. it is a tree, and we take this unique equivalence class at level $0$ as the root of the tree. For every $i,j \in \{1,\ldots,n\}$ and $k \in \{0,\ldots, K\}$, if $\alpha^i \wedge \alpha^j = s_k$, then by definition $i$ and $j$ belong to the same equivalence class for the relation $\sim_\ell$ if and only if $\ell \le k$. This implies that $\Lambda^i \wedge \Lambda^j = k$, and thus that the identity (<ref>) holds.

We first show the existence of the Gaussian process $(w^q(\alpha))_{\alpha\in \mathfrak{U}}$. With a view towards applying the Kolmogorov extension theorem, we  verify that for every $n \in \N$ and $\al^1, \ldots, \al^n \in \mfk U$, there exists a centered Gaussian vector $(w(\alpha^i))_{1 \le i \le n}$ with the desired covariance structure. Letting $K \ge 0$ and $s_0 < \cdots < s_K = 1$ be such that $\{s_k\}_{k = 0}^K = \{\al^i \wedge \al^j\}_{1 \le i,j \le n}$, we appeal to Lemma <ref> to find a tree of depth $K$ with leaves $\Lambda^1,\ldots, \Lambda^n$ such that (<ref>) holds. We denote by $\mathscr{A}$ the set of vertices of the tree, and let $(g_\beta)_{\beta\in\mathscr{A}}$ consist of independent standard $\R^\D$-valued Gaussian vectors. For every $i \in \{1,\ldots, n\}$ and $k \in \{0,\ldots, K\}$, we write $\Lambda^i_{|k}$ to denote the ancestor of the leaf $\Lambda^i$ at depth $k$. For each $i \in \{1,\ldots, n\}$, we set

\begin{align*}
w\Ll(\alpha^i\Rr) :=  \sum_{k=0}^K(q(s_k)-q(s_{k-1}))^{1/2} g_{\bal^i_{|k}},
\end{align*}

with the understanding that $q(s_{-1}) = 0$.
Using (<ref>), we see that

\begin{align*}
\E w\Ll(\alpha^i\Rr)w\Ll(\alpha^j\Rr)^\intercal =  q\Ll(s_{\bal^i\wedge\bal^j}\Rr) =  q\Ll(\alpha^i\wedge\alpha^j\Rr)\quad\forall i,j\in\{1,\dots,n\}.
\end{align*}

This is the desired form for the covariance. It is straightforward to verify the consistency condition in the Kolmogorov extension theorem. Appealing to this theorem (see e.g. <cit.>), we thus conclude for the existence of the Gaussian process $(w^q(\alpha))_{\alpha\in \mathfrak{U}}$.

Next, we verify the measurability. For every $\al, \al' \in \mfk U$, we have

\begin{align}\label{e.cts_at_1}
\E \Ll|w^q(\alpha) - w^q(\alpha')\Rr|^2 = \tr \Ll(q(1) - q(\alpha\wedge\alpha')\Rr).
\end{align}

Since moreover, $q$ is continuous at $1$ due to (<ref>),
we deduce that $w^q$ is stochastically continuous; in other words, for every $\alpha\in\mathfrak{U}$ and $\eps>0$, we have that $\P\Ll\{\Ll|w^q(\alpha) - w^q(\alpha')\Rr|>\eps\Rr\}$
converges to $0$ as $\alpha'$ approaches $\alpha$.
We can therefore apply <cit.> to find a version of the process that is jointly measurable.

To summarize and introduce more explicit notation, there exists a measurable subset $\bs \Omega^*$ of $\bs \Omega$ with $\P(\bs \Omega^*)$ such that (<ref>) holds for every $\bomega \in \bs \Omega^*$. For each such $\bomega$ and $q \in \mcl Q_\infty$, we can construct a probability space $(\Omega_{\bomega, q}, \P_{\bomega,q})$ with expectation $\E_{\bomega,q}$, and a centered Gaussian process $(w^q(\alpha))_{\alpha \in \mfk U_\bomega}$ on $(\Omega_{\bomega, q}, \P_{\bomega,q})$, such that for every $\al, \al' \in \mfk U_\bomega$, we have

\begin{equation}
\label{e.summary.gaussian}
\E_{\bomega,q} \Ll[ w^q(\alpha) w^q(\alpha')^\intercal \Rr] =  q(\alpha\wedge\alpha'),
\end{equation}

and the mapping

\begin{equation*}  %\label{e.}
\Ll\{
\begin{array}{rcl}  %\label{}
\mfk U_\bom\times \Omega_{\bom, q} & \to & \R^D
\\
(\al, \omega) & \mapsto & w^q(\alpha,\omega)
\end{array}
\Rr.
\end{equation*}

is measurable. The object $w^q$ will serve as the continuous analogue of the process in (<ref>).

Notice that we do not assert any measurability property of this Gaussian process with respect to $\bomega$. We do not know of a simple way to improve upon this. Fortunately, as will be explained shortly, this will not be a problem as long as we take $\E_{\bomega,q}$-expectations first, provided that we show that these expectations themselves are measurable with respect to $\bomega$. This is the content of Lemma <ref> below.

Lest the reader thinks that this is a technical detail that they would rather skip, it is worth noting that the justification of this measurability property will involve ingredients that we need to appeal to later as well. In a nutshell, we will show that the relevant $\E_{\bomega,q}$-expectations can be approximated arbitrarily closely by continuous functions of the overlap arrays. This result will not be of much surprise to specialists, as we will essentially state a general version of <cit.>.

For a measurable subset $S$ of a Euclidean space, we say that a pair $(\Gamma,\mathcal{R})$ is an $S$-valued overlap structure if

* $\Gamma$ is a probability measure on some Polish space;

* $\mathcal{R}:(\supp\Gamma)\times(\supp\Gamma)\to S$ is measurable and satisfies $|\mathcal{R}|\leq 1$ a.s.

For $d \in\N$ and a locally bounded function $C:S\to\R^{d \times d}$, $(\Gamma,\mathcal{R})$ is said to be $C$-admissible if there exists an $\R^d$-valued centered Gaussian process $(z(\rho))_{\rho\in\supp\Gamma}$ on some probability space $\Omega_{\Gamma,\mathcal{R}}$ such that

* the covariance is given by

\begin{align*}
\E_{\Omega_{\Gamma,\mathcal{R}}} \Ll[z(\rho)z(\rho')^\intercal \Rr] = C(\mathcal{R}(\rho,\rho')),\quad\forall \rho,\rho'\in\supp\Gamma;
\end{align*}

* the process $z$ is measurable as a function on $(\supp\Gamma)\times \Omega_{\Gamma,\mathcal{R}}$.

For every $\eps>0$ and $a,b\in\R$, we write

\begin{align}\label{e.approx_eps}
a \approx_\eps b \quad \Longleftrightarrow \quad |a-b|\leq \eps.
\end{align}

We take the setting and notation as in Definition <ref>. For every Lipschitz $\bg:\R^d\times \R^{d\times d}\to\R$, every bounded measurable $\bh:S^n\to\R$ with $n\in\N$, and every $\eps>0$, there are $n_1,n_2\in\N$ and bounded continuous functions $F^i_\eps: (\R^{d\times d})^{n_i}\to\R$, with $i\in\{1,2\}$, such that

\begin{align}
\E_{\Omega_{\Gamma,\mathcal{R}}} \log \int \exp\Ll(\mathbf{g}\Ll(z(\rho),C( \mathcal{R}(\rho,\rho))\Rr)\Rr) \d \Gamma(\rho)
&\approx_\eps \la F^1_\eps \Ll(C^{\leq n_1}\Rr) \ra_\Gamma,\label{e.1st_approx_os}
\\
\E_{\Omega_{\Gamma,\mathcal{R}}} \la \mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr) \ra_{\Gamma^\mathbf{g}} &\approx_\eps \la \mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr)F^2_\eps \Ll(C^{\leq n_2}\Rr) \ra_\Gamma, \label{e.2nd_approx_os}
\end{align}

uniformly over all $C$-admissible $S$-valued overlap structures $(\Gamma,\mathcal{R})$.
$\la\cdot\ra_\Gamma$ and $\la\cdot\ra_{\Gamma^\mathbf{g}}$ denote the expectations with respect to $\Gamma^{\otimes \N}$ and $({\Gamma^\mathbf{g}})^{\otimes\N}$ respectively, with canonical random variable $(\rho^\ell)_{\ell \ge 1}$; $\mathcal{R}^{\leq n} := \Ll(\mathcal{R}\Ll(\rho^{\ell},\rho^{\ell'}\Rr)\Rr)_{1\leq \ell,\ell'\leq n}$, $C^{\leq n_i} = \Ll(C\Ll( \mathcal{R}\Ll(\rho^{\ell},\rho^{\ell'}\Rr)\Rr)\Rr)_{1\leq \ell,\ell'\leq n_i}$; and

\begin{align*}
\d \Gamma^\mathbf{g}(\rho) := \frac{\exp\Ll(\mathbf{g}\Ll(z(\rho),\mathcal{R}(\rho,\rho)\Rr)\Rr) \d \Gamma(\rho)}{\int \exp\Ll(\mathbf{g}\Ll(z(\rho'),\mathcal{R}(\rho',\rho')\Rr)\Rr) \d \Gamma(\rho')}.
\end{align*}

For brevity, we write $\E = \E_{\Omega_{\Gamma,\mathcal{R}}}$. The estimates below are uniform in $(\Gamma,\mathcal{R})$ due to $|\mathcal{R}|\leq 1$.
For the first approximation (<ref>), we can use the same argument as in <cit.>. We sketch the main steps, omitting some details.
For $a> 0$, define $\log_a=\max\{-a, \min\{a,\log\}\}$ and $\mathbf{g}_a$ similarly. Since $\mathbf{g}$ is Lipschitz, we can use a Gaussian concentration inequality such as <cit.> to approximate the left-hand side in (<ref>) by

\begin{align*}
\E \log_a \la\exp\Ll(\mathbf{g}_a\Ll(z(\rho),C(\mathcal{R}(\rho,\rho))\Rr)\Rr) \ra_\Gamma
\end{align*}

for large $a$.
Noticing that $\log_a$ is continuous and that its argument in the expression above remains in a compact set, we can approximate it by polynomials, so that the expression above becomes a linear combination of terms of the form

\begin{align*}
\E \la\exp\Ll(\mathbf{g}_a\Ll(z(\rho),C(\mathcal{R}(\rho,\rho))\Rr)\Rr) \ra_\Gamma^j
&= \E \la \prod_{\ell=1}^j\exp\Ll(\mathbf{g}_a\Ll(z(\rho^\ell),C(\mathcal{R}(\rho^\ell,\rho^\ell))\Rr)\Rr) \ra_\Gamma,
\end{align*}

for $j\in\N$. We can move $\E$ inside $\la\cdot\ra_\Gamma$ to see that the right-hand side is equal to $ \la F^{1}_{\eps,j}\Ll(C^{\leq j}\Rr)\ra_\Gamma$ for a continuous function $F^1_{\eps,j}$ satisfying

\begin{align*}
F^{1}_{\eps,j}\Ll(C^{\leq j}\Rr) = \E\prod_{\ell=1}^j\exp\Ll(\mathbf{g}_a\Ll(z(\rho^\ell),C(\mathcal{R}(\rho^\ell,\rho^\ell))\Rr)\Rr).
\end{align*}

This is possible since we can write the right-hand side above explicitly as a Gaussian integral.
Collecting $F^{1}_{\eps,j}$ back into the linear combination, we obtain $F^1_\eps$.

The second approximation (<ref>) is done in a similar way.
Instead of $\log$, we approximate the function $x\mapsto \frac{1}{x}$. The detail can be seen in <cit.>, more precisely, the approximation after <cit.>.
One can also see the argument in the second half of <cit.>. Again, we only give a sketch. Writing $V(\rho) := \exp\Ll(\mathbf{g}(z(\rho),C(\mathcal{R}(\rho,\rho)))\Rr)$, we have

\begin{align*}
\E \la \mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr) \ra_{\Gamma^\mathbf{g}} = \E \frac{\la\mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr)\prod_{\ell=1}^n V\Ll(\rho^\ell\Rr)\ra_\Gamma}{\la V\Ll(\rho\Rr) \ra_\Gamma^n}.
\end{align*}

Using the Gaussian concentration, we can approximate $\la V\Ll(\rho\Rr) \ra_\Gamma^{-n}$ by a polynomial of $\la V\Ll(\rho\Rr) \ra_\Gamma$. Hence, the right-hand side in the above is approximated by a linear combination of

\begin{align*}
\E \la\mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr)\prod_{\ell=1}^n V\Ll(\rho^\ell\Rr)\ra_\Gamma \la V\Ll(\rho\Rr) \ra_\Gamma^j = \E \la\mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr)\prod_{\ell=1}^{n+j} V\Ll(\rho^l\Rr)\ra_\Gamma =  \la\mathbf{h}\Ll(\mathcal{R}^{\leq n}\Rr)F^2_{\eps,j}\Ll(C^{\leq n+j}\Rr)\ra_\Gamma
\end{align*}

for some $j\in\N$, where the continuous function $F^2_{\eps,j}$ satisfies

\begin{align*}
F^2_{\eps,j}\Ll(C^{\leq n+j}\Rr) = \E \prod_{\ell=1}^{n+j} V\Ll(\rho^\ell\Rr).
\end{align*}

Again, collecting $F^2_{\eps,j}$ back into the linear combination, we obtain $F^2_\eps$.

We now come back to the setting explained in the paragraph containing (<ref>), and settle the question of measurability of $\E_{\bom, q}$-expectations.

For every $q\in \mcl Q_\infty$, every Lipschitz $\mathbf{g}:\R^\D\to\R$, and every bounded measurable $\mathbf{h}:\R^{n\times n}\to \R$ with $n\in\N$, the functions

\begin{align}
\bomega &\mapsto \E_{{\bomega,q}} \log\int \exp\Ll(\mathbf{g}\Ll(w^q(\alpha)\Rr)\Rr) \d \fR_\bomega(\alpha),\label{e.1_bomega_mapsto}
\\
\bomega &\mapsto \E_{{\bomega,q}} \la  \mathbf{h}\Ll(\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)_{1\leq \ell,\ell'\leq n}\Rr) \ra_{\fR^\mathbf{g}_\bomega}\label{e.2_bomega_mapsto}
\end{align}

are measurable on $\boldsymbol{\Omega}^\star$, where $\la\cdot\ra_{\fR^\mathbf{g}_\bomega}$ denotes the expectation with respect to the measure $(\fR^\mathbf{g}_\bomega)^{\otimes \N}$ with canonical random variables $(\alpha^\ell)_{\ell \in \N}$, and with

\begin{align}\label{e.dfR^g}
\d \fR^\mathbf{g}_\bomega (\alpha):= \frac{\exp\Ll(\mathbf{g}\Ll(w^q(\alpha)\Rr)\Rr) \d \fR_\bomega(\al)}{\int \exp\Ll(\mathbf{g}\Ll(w^q(\al')\Rr)\Rr) \d \fR_\bomega(\al')}.
\end{align}

First of all, due to the joint measurability of the Gaussian process, both functions are well-defined. We show that they can be approximated uniformly in $\bomega$ by measurable functions.
For the first function (<ref>), we can apply Proposition <ref> (with $\Gamma=\fR_\bomega$, $R(\alpha,\alpha')=\alpha\wedge\alpha'$, and $C=q$) to get that, for every $\eps>0$, there is an integer $n_1$ and a continuous function $F_\eps^1:(\R^{\D\times\D})^{n_1}\to\R$ such that
the the function (<ref>) is approximated by the function

\begin{align}\label{e.3_bomega_mapsto}
\bomega\mapsto \la F^1_\eps\Ll(\Ll(q\Ll(\alpha^l\wedge\alpha^{l'}\Rr)\Rr)_{1\leq l,l'\leq n_1}\Rr) \ra_{\fR_\bomega}
\end{align}

uniformly in $\bomega$ up to an error bounded by $\eps$.
Since (<ref>) is measurable, we can conclude the measurability of the function (<ref>).
Similarly, for the second function (<ref>), we can deduce the measurability using Proposition <ref>.

Henceforth, we omit $\omega$ and $\bomega$ for brevity, and simply write $\P$ and $\E$ to denote the probability and the expectation over both $\bs \Omega$ and $\Omega_{\bomega,q}$, with the understanding that we always integrate over the space $\Omega_{\bomega,q}$ first.

We now show that for piecewise-constant paths, the free energies and Gibbs expectations defined using the discrete construction introduced in the previous section coincide with those defined using the continuous cascade.

Let $K \in \N$, let $(q_k)_{0 \le k \le K} \subset S^D_+$ and $(\zeta_k)_{1 \le k \le K} \subset \R_+$ be such that (<ref>) and (<ref>) hold, let $(\nu_\bal)_{\bal\in\N^K}$ be the associated Poisson–Dirichlet cascade as constructed below (<ref>), let $q \in \mcl Q$ be the piecewise-constant path in (<ref>), and let $(\msf w^q(\bal))_{\bal\in\N^K}$ and $(w^q(\alpha))_{\alpha \in \mfk U}$ be as in (<ref>) (with $N = 1$) and (<ref>) respectively.
For every Lipschitz $\mathbf{g}:\R^\D\to\R$ and every bounded measurable $\mathbf{h}:\R^{n\times n}\to\R$ with $n\in\N$, it holds that

\begin{gather}\label{e.l.equiv_cascade_1}
\E \log \sum_{\bal \in \N^K} \nu_\bal \exp\Ll(\mathbf{g}\Ll(\msf w^q(\bal)\Rr)\Rr) = \E \log \int \exp\Ll(\mathbf{g}\Ll(w^{q}(\alpha)\Rr)\Rr) \d \fR(\alpha),
\\
\label{e.l.equiv_cascade_2}
\E  \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu^{\mathbf{g}}_{\bal^1}\cdots \nu^{\mathbf{g}}_{\bal^n} \mathbf{h}\Ll(\Ll(\zeta_{\bal^\ell\wedge\bal^{\ell'}}\Rr)_{1\leq \ell,\ell'\leq n}\Rr) = \E  \la \mathbf{h}\Ll(\Ll(\zeta\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\Rr)_{1\leq \ell,\ell'\leq n}\Rr)\ra_{\fR^\mathbf{g}}
\end{gather}

where $\fR^\mathbf{g}$ is as defined in (<ref>), $\langle \cdot \rangle_{\fR^\mathbf{g}}$ as in Lemma <ref>,

\begin{align}\label{e.nu^g_bal=}
\nu^\mathbf{g}_\bal := \frac{ \nu_\bal\exp\Ll(\mathbf{g}\Ll(\msf w^q(\bal)\Rr)\Rr)}{\sum_{\bal' \in \N^K} \nu_{\bal'} \exp\Ll(\mathbf{g}\Ll(\msf w^q(\bal')\Rr)\Rr)}, \qquad\forall \bal \in \N^K.
\end{align}

In (<ref>) and (<ref>), the expectations integrate all the randomness. For the expectations on the right side, we first integrate the Gaussian randomness in $(w^{q}(\alpha))_{\alpha \in \mfk U}$ and then the randomness of $\mathfrak{R}$, which is justified by Lemma <ref>.

We start by observing that, for every $\bal, \bal' \in \N^K$,

\begin{equation*}  %\label{e.}
\E \Ll[ \msf w^q(\bal) \msf w^q(\bal')^\intercal \Rr]  = q \Ll( \zeta_{\bal \wedge \bal'} \Rr) ,
\end{equation*}

where we recall that $(q(u))_{u \in [0,1)}$ denotes the piecewise-constant path in (<ref>). In the language of Definition <ref>, the probability measure with weights $(\nu_\bal)_{\bal \in \N^K}$ and the field $(\msf w^q(\bal))_{\bal \in \N^K}$ form a $q$-admissible overlap structure, with the mapping $\mcl R$ chosen to be $(\bal, \bal') \mapsto \zeta_{\bal \wedge \bal'}$.

By (<ref>), the measure $\mfk R$ and the Gaussian field $(w^q(\al))_{\al \in \mfk U}$ also form a $q$-admissible overlap structure, with $\mcl R$ chosen to be $(\al,\al') \mapsto \al \wedge \al'$. Since $q$ is constant on each interval $[\zeta_k, \zeta_{k+1})$, we can also choose $\mcl R$ to be the mapping $(\al,\al') \mapsto \zeta(\al \wedge \al')$, where we write

\begin{equation}
\label{e.def.zetapath}
\zeta := \sum_{k = 0}^K \zeta_k \1_{ \Ll[ \zeta_k, \zeta_{k+1} \Rr) }.
\end{equation}

Denoting by $(\bal^\ell)_{\ell \in \N}$ and $(\alpha^\ell)_{\ell \in \N}$ two families of independent random variables sampled according to $(\nu_\bal)_{\bal \in \N^K}$ and $\mfk R$ respectively, we see from Proposition <ref> that in order to prove the result, it suffices to argue that for each $n \in \N$, the overlap arrays $(\zeta_{\bal^\ell \wedge \bal^{\ell'}})_{\ell, \ell' \le n}$ and $(\zeta(\al^\ell \wedge \al^{\ell'}))_{\ell, \ell' \le n}$ have the same law after averaging over all sources of randomness. By <cit.> (or   <cit.>), every discrete Poisson–Dirichlet cascade satisfies the Ghirlanda–Guerra identities, and these identities are preserved whether we look at the overlap as being the mapping $(\bal, \bal') \mapsto \bal \wedge \bal'$ or the mapping $(\bal, \bal') \mapsto \zeta_{\bal \wedge \bal'}$. We recall that we built the continuous Poisson–Dirichlet cascade as a limit of discrete Poisson–Dirichlet cascades, using e.g. <cit.>. It therefore follows that the continuous Poisson–Dirichlet cascade $\mfk R$ still satisfies the Ghirlanda–Guerra identities, and in particular the overlap array built from the overlap mapping $(\al,\al') \mapsto \zeta(\al \wedge \al')$ satisfies the Ghirlanda–Guerra identities. By e.g. <cit.>, the laws of the overlap arrays $(\zeta_{\bal^\ell \wedge \bal^{\ell'}})_{\ell, \ell' \le n}$ and $(\zeta(\al^\ell \wedge \al^{\ell'}))_{\ell, \ell' \le n}$ are uniquely determined by the laws of $\zeta_{\bal \wedge \bal'}$ and $\zeta(\al \wedge \al')$ respectively. By <cit.> (or <cit.>), we have that the probability for $\zeta_{\bal \wedge \bal'}$ to be equal to $\zeta_k$ is $\zeta_{k+1} - \zeta_k$, for each $k \in \{0, \ldots, K\}$. By the construction of $\mfk R$ from <cit.>, we have that $\al \wedge \al'$ is uniformly distributed over $[0,1]$, and in particular, by (<ref>), the probability for $\zeta(\al \wedge \al')$ to be equal to $\zeta_k$ is equal to $\zeta_{k+1} - \zeta_k$ as well. The proof is therefore complete.

To close this section, we show that the law of a Poisson–Dirichlet cascade under $\E \la \cdot \ra_\fR$ remains the same under any probability measure of the form of $\E \la \cdot \ra_{\fR^\mathbf{g}}$, with $\fR^{\mathbf{g}}$ as in (<ref>) with $\mathbf{g}$ an arbitrary Lipschitz function. In particular, the overlap $\alpha \wedge \alpha'$ under $\E \la \cdot \ra_{\fR^{\mathbf{g}}}$ is uniformly distributed over $[0,1]$, a fact that we will use repeatedly later.

We start by recalling the following invariance property of discrete Poisson–Dirichlet cascades from <cit.>.

Let $K \in \N$, let $\mathcal{A}$ be the rooted tree in (<ref>),  let $(q_k)_{0 \le k \le K} \subset S^D_+$ and $(\zeta_k)_{1 \le k \le K} \subset \R_+$ be such that (<ref>) and (<ref>) hold, let $(\nu_\bal)_{\bal\in\N^K}$ be the associated Poisson–Dirichlet cascade as constructed below (<ref>), and let $(\msf w^q(\bal))_{\bal\in\N^K}$ be as in (<ref>).
For every Lipschitz $\mathbf{g}:\R^\D\to\R$, there is a random bijection $\pi:\mathcal{A}\to\mathcal{A}$, preserving the parent-child relation, such that

\begin{align*}
\Ll(\nu^\mathbf{g}_{\pi(\bal)}\Rr)_{\bal \in\N^K} \stackrel{\d}{=} \Ll(\nu_\bal\Rr)_{\bal\in\N^K}
\end{align*}

where $ \Ll(\nu^\mathbf{g}_{\bal}\Rr)_{\bal \in\N^K}$ is given in (<ref>).

In particular, for every bounded measurable function $\mathbf{h}:\R^{n\times n}\to\R$ with $n\in\N$, we have

\begin{align*}
\E  \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu^{\mathbf{g}}_{\bal^1}\cdots \nu^{\mathbf{g}}_{\bal^n} \mathbf{h}\Ll(\Ll(\zeta_{\bal^\ell\wedge\bal^{\ell'}}\Rr)_{1\leq \ell,\ell'\leq n}\Rr) = \E  \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu_{\bal^1}\cdots \nu_{\bal^n} \mathbf{h}\Ll(\Ll(\zeta_{\bal^\ell\wedge\bal^{\ell'}}\Rr)_{1\leq \ell,\ell'\leq n}\Rr).
\end{align*}

We can now prove that this invariance property transfers to continuous cascades.

For every $q \in \mcl Q_\infty$, every Lipschitz $\mathbf{g}:\R^\D\to\R$, and every bounded measurable $\mathbf{h}:\R^{n\times n}\to\R$ with $n\in\N$, we have

\begin{align*}
\E  \la \mathbf{h}\Ll(\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)_{1\leq \ell,\ell'\leq n}\Rr)\ra_{\fR^\mathbf{g}}=\E  \la \mathbf{h}\Ll(\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)_{1\leq \ell,\ell'\leq n}\Rr)\ra_{\fR}
\end{align*}

where $\fR^\mathbf{g}$ is defined in (<ref>), and we recall that $\langle \cdot \rangle_\fR = \langle \cdot \rangle_{\fR^0}$.

For every $\ell, \ell',n \in \N$, we write $Q^{\ell, \ell'} := \al^\ell \wedge \al^{\ell'}$ and $Q^{\le n} := (Q^{\ell,\ell'})_{1 \le \ell, \ell' \le n}$.
Without loss of generality, we can assume that $\mathbf{h}$ is bounded and continuous. We fix $\ep > 0$. By Proposition <ref>, there exists $n \in \N$ and a bounded continuous function $F_\ep$ such that

\begin{align}\label{e.cts_inv_eq_1}
\E \la \mathbf{h}\Ll(Q^{\leq n}\Rr)\ra_{\fR^\mathbf{g}} \approx_\eps \E \la  \mathbf{h}\Ll(Q^{\leq n}\Rr) F_\eps\Ll( q\Ll(Q^{\leq n}\Rr)\Rr)\ra_\fR,
\end{align}

where $q$ acts entrywise. For each $K \in \N$, we write

\begin{equation*}
\zeta^{(K)} := \sum_{k = 0}^K \frac{k}{K+1} \1_{ \Ll[ \frac{k}{K+1}, \frac{k+1}{K+1} \Rr)  },
\end{equation*}

with the usual understanding that $\zeta^{(K)}(1) = \lim_{u \nearrow 1} \zeta^{(K)}(u) = K/(K+1)$.
In particular, the sequence $(\zeta^{(K)})_{K \in \N} \subset \mcl Q_\infty$ converges in $L^1$ to the identity map $x \mapsto x$ as $K$ tends to infinity. For every $\ell, \ell',n \in \N$, we write $Q_K^{\ell, \ell'} := \zeta^{(K)}(\al^\ell \wedge \al^{\ell'})$ and $Q_K^{\le n} := (Q_K^{\ell, \ell'})_{1 \le \ell, \ell' \le n}$. Since $Q$ satisfies the Ghirlanda–Guerra identities by construction of the continuous Poisson–Dirichlet cascade, it is immediate that $Q_K$ also satisfies the Ghirlanda–Guerra identities. By <cit.>, we have that $Q_K^{\le n}$ converges in law to $Q^{\le n}$. We want to argue that $(Q_K^{\le n}, q(Q_K^{\le n}))$ also converges in law to $(Q^{\le n}, q(Q^{\le n}))$ as $K$ tends to infinity. This is not immediate since $q$ is not continuous. Notice first that the on-diagonal overlaps are deterministic, and for every $\ell \in \N$, we have

\begin{equation*}  %\label{e.}
Q_K^{\ell, \ell} = \frac{K}{K+1} \xrightarrow{K \to \infty} 1 = Q^{\ell,\ell}.
\end{equation*}

Since $q$ is continuous at $1$, the question thus only concerns the off-diagonal overlaps. We recall that each of these off-diagonal overlaps is uniformly distributed over $[0,1]$. Since the set of points of discontinuity of $q$ is countable, a union bound guarantees that the probability for $Q^{\le n}_K$ to belong to the set of points of discontinuity of the mapping $q$ (as applied entrywise to the array) is zero.
We can therefore apply the continuous mapping theorem (e.g. <cit.>) to deduce that indeed $(Q_K^{\le n}, q(Q_K^{\le n}))$ converges in law to $(Q^{\le n}, q(Q^{\le n}))$ as $K$ tends to infinity. In particular, for every $K$ sufficiently large, we have that

\begin{equation*}  %\label{e.}
\E \la  \mathbf{h}\Ll(Q^{\leq n}\Rr) F_\eps\Ll( q\Ll(Q^{\leq n}\Rr)\Rr)\ra_\fR \approx_\ep \E \la  \mathbf{h}\Ll(Q_K^{\leq n}\Rr) F_\eps\Ll( q\Ll(Q_K^{\leq n}\Rr)\Rr)\ra_\fR.
\end{equation*}

By Proposition <ref>, the latter quantity can be represented using the discrete Poisson–Dirichlet cascade based on the path $q \circ \zeta^{(K)}$, which is defined on a tree of depth $K$. Denoting the weights of this discrete Poisson–Dirichlet cascade by $(\nu_{\bal})_{\bal \in \N^K}$, Proposition <ref> ensures that

\begin{equation*}  %\label{e.}
\E \la  \mathbf{h}\Ll(Q_K^{\leq n}\Rr) F_\eps\Ll( q\Ll(Q_K^{\leq n}\Rr)\Rr)\ra_\fR = \E  \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu_{\bal^1}\cdots \nu_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr) F_\ep \Ll( q \Ll( R^{\le n} \Rr)  \Rr) ,
\end{equation*}

where we used the shorthand $R^{\ell,\ell'} := (\bal^\ell \wedge \bal^{\ell'})/(K+1)$ and $R^{\le n} := (R^{\ell, \ell'})_{1 \le \ell, \ell' \le n}$. Another application of Proposition <ref>, in the same setting of $q$-admissible overlap structures as for (<ref>), yields that

\begin{equation*}  %\label{e.}
\E  \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu_{\bal^1}\cdots \nu_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr) F_\ep \Ll( q \Ll( R^{\le n} \Rr)  \Rr)
\approx_\ep
%\E \la \mathbf{h}\Ll({R}^{\leq n}\Rr) \ra_{\bal^\mathbf{g}} ,
\sum_{\bal^1,\dots,\bal^n \in \N^K} \nu^{\mathbf{g}}_{\bal^1}\cdots \nu^{\mathbf{g}}_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr).
\end{equation*}

By the invariance property from Lemma <ref>, we have that

\begin{equation*}  %\label{e.}
\sum_{\bal^1,\dots,\bal^n \in \N^K} \nu^{\mathbf{g}}_{\bal^1}\cdots \nu^{\mathbf{g}}_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr)
= \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu_{\bal^1}\cdots \nu_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr).
\end{equation*}

To summarize, we have shown that

\begin{equation*}  %\label{e.}
\E \la \mathbf{h}\Ll(Q^{\leq n}\Rr)\ra_{\fR^\mathbf{g}} \approx_{3\ep} \sum_{\bal^1,\dots,\bal^n \in \N^K} \nu_{\bal^1}\cdots \nu_{\bal^n} \mathbf{h}\Ll(R^{\le n}\Rr).
\end{equation*}

Using this relation also with $\mathbf{g} = 0$, we conclude that

\begin{equation*}  %\label{e.}
\E \la \mathbf{h}\Ll(Q^{\leq n}\Rr)\ra_{\fR^\mathbf{g}} \approx_{6\ep}    \E \la \mathbf{h}\Ll(Q^{\leq n}\Rr)\ra_{\fR} .
\end{equation*}

Since $\ep > 0$ was arbitrary, the proof is complete.

All results in this section also hold if, instead of $\C_\infty$, we take $q$ from the set

\begin{align}\label{e.Pi=}
\Pi:= \Ll\{ q : [0,1] \to S^\D_+ \ : \ q \text{ is left-continuous with right limits, and is increasing} \Rr\}.
\end{align}

The only place where any kind of continuity of a path is needed is in the proof of Proposition <ref>. There, to show that $w^q$ is jointly measurable, one requires $q$ to be continuous at $1$, as used in (<ref>).  Every path in $\Pi$ is also continuous at $1$ by definition.

§ THE FREE ENERGY WITH CONTINUOUS CASCADES

We recall from the beginning of the previous section that we denote by $\fR$ a Poisson–Dirichlet cascade such that the law of one overlap is uniformly distributed over $[0,1]$. The object $\fR$ is a (random) probability measure on the Hilbert space $\mfk H$, and we denote its (random) support by $\mfk U$. For each realization of $\mfk R$, we denote by $\la \cdot \ra_{\fR}$ the expectation with respect to the measure $\fR^{\otimes \N}$, with canonical random variables $(\alpha^\ell)_{\ell \in \N}$. The expectation $\E$ takes the average over the randomness of $\fR$ itself. The event in (<ref>) has $\P$-probability one, and whenever this event is realized and for each $q \in \mcl Q_\infty$, there exists an $\R^D$-valued Gaussian process $(w^q(\al))_{\al \in \mfk U}$ with covariance given by (<ref>). On this same event, we take $(w^{q}_i)_{i\in\N}$ to be i.i.d. copies of $w^{q}$ and define, for every $N\in\N$ and $\alpha \in \mfk U$,

\begin{align}\label{e.W^mu(alpha)}
W^{q}_N(\alpha) := (w^{q}_1(\alpha),\dots, w^{q}_N(\alpha)).
\end{align}

The random variable $W^q_N(\alpha)$ takes values in $\R^{D\times N}$. We can now define, for each $t \ge 0$, $q \in \mcl Q_\infty$, and $N \in \N$, the free energy

\begin{equation}
\label{e.def.barFN.continuous}
\bar F_N(t,q) := -\frac{1}{N}\E \log \iint \exp \Ll(H_N^{t,{q}}(\sigma,\alpha) \Rr)\d P_N(\sigma)\d \fR(\alpha),
\end{equation}

\begin{equation}
\label{e.H^t,mu_N}
H_N^{t,{q}}(\sigma,\alpha)  = \sqrt{2t}H_N(\sigma) - tN\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)+\sqrt{2}W^{q}_N(\alpha)\cdot \sigma -  {q}(1)\cdot \sigma\sigma^\intercal.
\end{equation}

We also define the associated Gibbs measure

\begin{align}
\label{e.<>_N}
\la \cdot\ra_N \propto  \exp\Ll( H^{t,q}_N(\sigma,\alpha)\Rr) \d P_N(\sigma) \d \fR(\alpha),
\end{align}

defined for each fixed realization of $\fR$ and of the random Gaussian field $(w^q(\alpha))_{\alpha \in \mfk U}$. We use the symbol $\propto$ as shorthand for more precise expressions as in (<ref>).  The canonical random variable under $\la \cdot \ra_N$ is denoted by $(\si, \al)$. We also write $(\si^\ell, \al^\ell)_{\ell \in \N}$ to denote independent copies of $(\si, \al)$ under $\la\cdot \ra_N$. The expectation $\E$ in (<ref>) integrates all sources of randomness, and we recall that for measurability reasons, we always take the average over the Gaussian process $(W_N^q(\alpha))_{\alpha \in \mfk U}$ first.

In (<ref>), we allowed ourselves to use the same notation as in (<ref>). When $q$ is piecewise constant, these two quantities coincide, by Proposition <ref>. This proposition also ensures that Gibbs averages computed under the measures in (<ref>) and in (<ref>) coincide.

Recall that we extended the free energy defined in (<ref>) to all paths $q \in \mcl Q_1$ using the Lipschitz continuity property from Proposition <ref>. The next proposition states that the right-hand side of (<ref>) satisfies the same Lipschitz continuity property, and therefore the two constructions do coincide for all paths and there is no abuse of notation here. This proposition in fact identifies the derivatives of $\bar F_N$ with respect to $t$ and $q$ precisely.
We set

\begin{align}\label{e.L^infty<1}
L^\infty_{\leq 1} := \Ll\{q \in L^\infty :\: |\kappa|_{L^\infty}\leq 1\Rr\}.
\end{align}

Let $N \in \N$, and let $\bar F_N : \R_+ \times \mcl Q_\infty \to \R$ be the free energy defined in (<ref>). We have for every $t,t' \in \R_+$ and $q,q' \in \mcl Q_\infty$ that

\begin{equation*}
\Ll|\bar F_N(t,{q}) - \bar F_N(t',{q'})\Rr|\leq \Ll|{q}-{q'}\Rr|_{L^1} + |t-t'| \, \sup_{|a| \le 1} |\xi(a)|.
\end{equation*}

In particular, the free energy in (<ref>) can be extended by continuity to $\R_+\times \mcl Q_1$, and it coincides everywhere with the free energy defined in (<ref>) and extended by continuity to $\R_+\times \mcl Q_1$ in Proposition <ref>. Moreover, the restriction of the function $\bar F_N$ to $\R_+ \times \mcl Q_2$ is Fréchet (and Gateaux) differentiable everywhere, jointly in its two variables. We denote its Fréchet (and Gateaux) derivative in $q$ by $\dr_q \bar F_N(t,q) = \dr_q \bar F_N(t,q, \cdot) \in L^2([0,1]; S^D)$. For every $t \ge 0$, we have, for every $q \in \mcl Q_2$,

\begin{equation}
\label{e.bounds.der.FN}
\partial_q \bar F_N(t,q) \in \mcl Q \cap L^\infty_{\le 1}, \qquad \Ll|\partial_t  \bar F_N(t,q) \Rr| \le \sup_{|a| \le 1} |\xi(a)|,
\end{equation}

and, for every $q \in \mcl Q_\infty$  and $\kappa \in L^2([0,1]; S^D)$,

\begin{equation}
\label{e.def.der.FN}
\la \kappa,\dr_q \bar F_N(t,{q})\ra_\cH = \E \la \kappa \Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N}\ra_N,\qquad \partial_t \bar F_N(t,{q}) = \E \la \xi \Ll(\frac{\sigma\sigma'^\intercal}{N}\Rr)\ra_N.
\end{equation}

Finally, for every $r \in [1,+\infty]$,  $t \ge 0$ and $q, q' \in \mcl Q_2$ with $q'-q \in L^r$, we have

\begin{equation}
\label{e.continuity.der.FN}
\Ll|\dr_q \bar F_N(t,{q'}) -\dr_q \bar F_N(t,{q})\Rr|_{L^r}\leq 16N\Ll|q'-{q}\Rr|_{L^r}.
\end{equation}