In particular, the mapping $q \mapsto \dr_q \bar F_N(t,q)$ can be extended to $\mcl Q_1$ by continuity, and the properties in (<ref>) and (<ref>) remain valid with $q, q' \in \mcl Q_1$.

For the statement of (<ref>), we impose that $q \in \mcl Q_\infty$ so that the Gibbs measure appearing there	is well-defined.

For clarity of presentation we only prove the results concerning $\dr_q \bar F_N$; showing the joint differentiability in $(t,q)$ does not pose additional difficulties, the property of $\partial_t \bar F_N(t,q)$ in (<ref>) is a Gaussian integration by parts as in (<ref>), and the property of $\partial_t \bar F_N(t,q)$ in (<ref>) follows from that in (<ref>) and (<ref>).

We fix $t \ge 0$, $q, q' \in \mcl Q_\infty$, and write $\kappa := q'-q \in L^\infty([0,1]; S^D)$.
Conditionally on $\fR$, we construct the Gaussian processes $(w^{q}(\alpha))_{\alpha \in \mfk U}$ and $(w^{\kappa}(\alpha))_{\alpha \in \mfk U}$ from Proposition <ref> independently of one another, and we observe that, for every $s \in [0,1]$,

\begin{align}
\label{e.wq.distr.identity}
\Ll(w^{{q}+ s  \kappa}(\alpha) \Rr)_{\alpha\in \mfk U}\stackrel{\d}{=} %\Ll(\sqrt{ s } w^{{q}+ \kappa}(\alpha) + \sqrt{1- s } w^{{q}}(\alpha)\Rr)_{\alpha \in \mfk U}.
\Ll(w^{{q}}(\alpha) + \sqrt{s} w^{{\kappa}}(\alpha)\Rr)_{\alpha \in \mfk U}.
\end{align}

We may as well take the right-hand side of (<ref>) as our definition of $w^{q+s\kappa}$, so that these processes are realized on the same probability space and coupled together in a natural way as $s$ varies in $[0,1]$. It follows from (<ref>) that, for every $\al, \al' \in \mfk U$,

\begin{align*}
\E\Ll[ \Ll(\frac{\d}{\d  s }w^{{q}+ s  \kappa}(\alpha) \Rr)\Ll(w^{{q}+ s  \kappa}(\alpha')\Rr)^\intercal \Rr] = \frac{1}{2}\kappa\Ll(\alpha\wedge\alpha'\Rr).
\end{align*}

By this, the Gaussian integration by parts in <cit.>, and the fact that $\alpha\wedge\alpha=1$ almost surely under $\P\fR$, we deduce that

\begin{align*}
\frac{\d}{\d  s }\bar F_N(t,{q}+ s \kappa) \big|_{ s  =0}
& = -\frac{1}{N}\E \la \sqrt{2}\Ll(\frac{\d}{\d  s }w^{{q}+ s  \kappa}(\alpha)\Rr)\cdot\sigma - \kappa(1)\cdot\sigma\sigma^\intercal\ra_N
= \E \la R^{1,2}\ra_N,
\end{align*}

where we write

\begin{equation}
\label{e.def.R.kappa}
R^{\ell,\ell'} := \kappa(\alpha^\ell\wedge\alpha^{\ell'})\cdot \frac{\sigma^\ell(\sigma^{\ell'})^\intercal}{N},
\end{equation}

and we observe that $\kappa(1)$ is well-defined since $q(1)$ and $q'(1)$ are well-defined. Differentiating once more yields that

\begin{align}
\label{e.diff.q.second}
\frac{\d^2}{\d  s ^2}\bar F_N(t,{q}+ s \kappa)\big|_{ s  =0} = 2N\E\la R^{1,2}R^{1,2}-4R^{1,2}R^{1,3}+3R^{1,2}R^{3,4}\ra_N.
\end{align}

We recall from (<ref>) that for every $\sigma, \sigma'$ in the support of $P_N$, we have $\Ll|\frac{\si \si'^\intercal}{N}\Rr| \le 1$. Using also  the invariance of Poisson–Dirichlet cascades in Proposition <ref> and the Cauchy–Schwarz inequality, we obtain that

\begin{align}
\label{e.|d/dF_N|<}
\Ll|\frac{\d}{\d  s }\bar F_N(t,{q}+ s \kappa)\big|_{ s  =0}\Rr| &  \leq \E \la \Ll|R^{1,2}\Rr|\ra_N\leq \E \la |\kappa\Ll(\alpha\wedge\alpha'\Rr)| \ra_N = \E \la |\kappa\Ll(\alpha\wedge\alpha'\Rr)| \ra_\fR = |\kappa|_{L^1},
\end{align}

as well as

\begin{multline}
\label{e.|d^2F_N|<}
\Ll|\frac{\d^2}{\d  s ^2}\bar F_N(t,{q}+ s \kappa)\big|_{ s =0}\Rr|  \leq 16 N\E \la \Ll|R^{1,2}\Rr|^2\ra_N \leq  16 N\E \la \Ll|\kappa\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_N
\\
=16 N \E \la \Ll|\kappa\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_\fR =16 N|\kappa|_\cH^2.
\end{multline}

We define the function $\partial_q \bar F_N(t,q) = \dr_q \bar F_N(t,q,\cdot) \in L^2([0,1]; S^D)$ by duality so that for every $\kappa \in L^2([0,1]; S^D)$,

\begin{equation}
\label{e.def.drq.FN}
\la \kappa, \dr_q\bar F_N(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot\frac{\sigma\sigma'^\intercal}{N}\ra_N.
\end{equation}

The estimates (<ref>) and (<ref>) imply that

\begin{equation}
\label{e.FN.frechet}
\Ll| \bar F_N(t,q') - \bar F_N(t,q) - \la q'-q, \dr_q\bar F_N(t,{q})\ra_\cH\Rr| \le 8N|q'-q|_{L^2}^2.
\end{equation}

It follows from (<ref>) and (<ref>) that  for every $\kappa \in L^2([0,1]; S^D)$,

\begin{equation}
\label{e.bound.drq}
\la \kappa, \dr_q\bar F_N(t,{q})\ra_\cH \leq |\kappa|_{L^1}.
\end{equation}

By duality, we deduce that $\partial_q \bar F_N(t,q) \in L^\infty_{\le 1}$.
Therefore, we also have that

\begin{equation*}  %\label{e.}
|\bar F_N(t,q) - \bar F_N(t,q')| \le |q-q'|_{L^1}.
\end{equation*}

In particular, we can extend the free energy $\bar F_N$ in (<ref>) to all $\R_+ \times \mcl Q_1$ by continuity. We have already observed that the definitions of $\bar F_N(t,q)$ in (<ref>) and (<ref>) coincide when $q$ is piecewise constant, and each has been extended continuously to $\R_+\times \mcl Q_1$, so the two definitions coincide everywhere on $\R_+ \times \mcl Q_1$.

By continuity, the estimate in (<ref>), which we showed for $q,q' \in \mcl Q_\infty$, extends to every $q' \in \mcl Q_2$. This already shows that every point $q \in \mcl Q_\infty$ is a point of Fréchet differentiability of $\bar F_N(t,\cdot) : \mcl Q_2 \to \R$, and the Fréchet derivative at this point is indeed $\partial_q \bar F_N(t,q)$. We now wish to extend this to every $q \in \mcl Q_2$. We thus fix $q \in \mcl Q_2$ and take a sequence $(q_n)_{n \in \N}$ of elements of $\mcl Q_\infty$ that converges to $q$ in $L^2$. Since $\partial_q \bar F_N(t,q_n) \in L^\infty_{\le 1}$ for every $n \in \N$, we can extract a subsequence that converges weakly in $L^2$; we denote the limit by $p$. Since $(q_n)_{n \in \N}$ converges to $q$ strongly in $L^2$, substituting $q$ by $q_n$ in (<ref>) and letting $n$ tend to infinity along the subsequence yields that, for every $q' \in \mcl Q_2$,

\begin{equation*}  %\label{e.}
\Ll| \bar F_N(t,q') - \bar F_N(t,q) - \la q'-q, p\ra_\cH\Rr| \le 8N|q'-q|_{L^2}^2.
\end{equation*}

This characterizes $p$ uniquely, so in fact the subsequence extraction was not necessary and $\partial_q \bar F_N(t,q_n)$ converges weakly to an object that depends only on $q$ itself, not on the approximating sequence. We denote this object by $\dr_q \bar F_N(t,q)$, a notation that is consistent with the previous definition that was restricted to the case $q \in \mcl Q_\infty$. We have thus shown that $\bar F_N(t,\cdot) : \mcl Q_2 \to \R$ is Fréchet differentiable everywhere.

Since the inequality (<ref>) is preserved by taking weak $L^2$ limits on $\dr_q \bar F_N(t,q)$, we also have that $\dr_q \bar F_N(t,q)$ belongs to $L^\infty_{\le 1}$ for every $q \in \mcl Q_2$.

We recall from Proposition <ref> that $\bar F_N(t,\cdot)$ is $\mcl Q_2^\star$-increasing, as defined in (<ref>). For every smooth $q \in \mcl Q$ with derivative bounded away from zero and every smooth $\kappa \in \mcl Q_2^\star$, we must therefore have that

\begin{equation*}  %\label{e.}
\la \kappa, \dr_q\bar F_N(t,{q})\ra_\cH   \ge 0.
\end{equation*}

By density, this is in fact valid for every $\kappa \in \mcl Q_2^\star$. This means that $\partial_q \bar F_N(t,q)$ belongs to the dual cone

\begin{equation*}  %\label{e.}
(\C^*_2)^*:=\Ll\{{q}\in L^2([0,1]; S^D) \, \mid \, \forall {q'} \in \C_2^*, \  \la {q},{q'}\ra_\cH\geq 0 \Rr\}.
\end{equation*}

Since $\C_2$ is a non-empty closed convex cone in a Hilbert space, we have by <cit.>  that $(\C^*_2)^* = \C_2$. (One can also prove this more directly by proceeding similarly to the proof of Lemma <ref>.) We have therefore verified that $\partial_q \bar F_N(t,q) \in \mcl Q_2$ for every smooth $q \in \mcl Q$ whose derivative is bounded away from zero. Since $\mcl Q_2$ is closed and convex, it is also weakly closed, by <cit.> (or again by direct arguments). For general $q \in \mcl Q_2$, since we can obtain $\dr_q \bar F_N(t,q)$ as a weak $L^2$ limit of similar objects computed at smooth paths with derivative bounded away from zero, we deduce that $\dr_q \bar F_N(t,q)$ belongs to $\mcl Q_2$ for every $q \in \mcl Q_2$.

So far we have only observed the continuity of the mapping $q \mapsto \dr_q \bar F_N(t,q)$ in the sense of weak $L^2$ convergence. Now that we also know that $\dr_q \bar F_N(t,q) \in \mcl Q \cap L^\infty_{\le 1}$, we can appeal to Lemma <ref> and upgrade the continuity property of the mapping $q \mapsto \dr_q \bar F_N(t,q)$ to the strong $L^r$ topology, for every $r < +\infty$. Precisely, for every $r < +\infty$ and every sequence $(q_n)_{n \in \N} \subset \mcl Q_2$ converging to $q \in \mcl Q_2$ in $L^2$, we have

\begin{equation}
\label{e.conv.der.Lr}
\lim_{n \to + \infty} \Ll|\dr_q \bar F_N(t,q_n) - \dr_q \bar F_N(t,q) \Rr|_{L^r} = 0.
\end{equation}

It remains to show (<ref>).
We temporarily display the dependence of $\la\cdot\ra_N$ on ${q}$ explicitly by writing $\la\cdot\ra_N = \la\cdot\ra_{N,{q}}$. We fix ${q}, {q}'\in\C_\infty$ as in the beginning of this proof, with $\kappa = q'-q$, and for each $\eta \in L^\infty$, we aim to compute the derivative in $s$ of

\begin{equation*}  %\label{e.}
\E \la \eta\Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N}\ra_{N, {q} + s \kappa }.
\end{equation*}

We realize the Gaussian processes $(w^{q + s \kappa}(\alpha))_{\alpha \in \mfk U}$ as discussed below (<ref>). As for (<ref>), recalling the notation $R^{\ell, \ell'}$ in (<ref>) and denoting
$Q^{\ell,\ell'} := \eta(\alpha^\ell\wedge\alpha^{\ell'})\cdot \frac{\sigma^\ell(\sigma^{\ell'})^\intercal}{N}$, we have

\begin{align*}
\frac{\d}{\d s } \E \la \eta\Ll(\alpha\wedge\alpha'\Rr)\cdot\frac{ \sigma\sigma'^\intercal}{N}\ra_{N,{q} + s\kappa } = 2N\E \la Q^{1,2}\Ll(R^{1,2}-4R^{1,3}+3R^{3,4}\Rr)\ra_{N,{q} + s \kappa }.
\end{align*}

By Hölder's inequality and the invariance of Poisson–Dirichlet cascades from Proposition <ref>, we get, for every $r, r' \in [1,+\infty]$ with $\frac 1 r + \frac 1 {r'} = 1$,

\begin{align*}
\Ll|\frac{\d}{\d s } \E \la \eta\Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N}\ra_{N,{q} + s \kappa }\Rr|
&\leq 16N \E \la \Ll|Q^{1,2}\Rr|^{r'}\ra_{N,{q}  + s \kappa}^{1/r'}\E \la \Ll|R^{1,2}\Rr|^r \ra_{N,{q}  + s \kappa}^{1/r}
\\
&\leq 16N\E \la \Ll|\eta\Ll(\alpha\wedge\alpha'\Rr)\Rr|^{r'} \ra_{N,{q}  + s \kappa }^{1/r'}\E \la \Ll|\kappa\Ll(\alpha\wedge\alpha'\Rr)\Rr|^r \ra_{N,{q}  + s \kappa }^{1/r}\notag
\\
&= 16N \E \la \Ll|\eta\Ll(\alpha\wedge\alpha'\Rr)\Rr|^{r'} \ra_{\fR}^{1/r'}\la \Ll|\kappa\Ll(\alpha\wedge\alpha'\Rr)\Rr|^r \ra_{\fR}^{1/r}
\notag
\\
& = 16N |\eta|_{L^{r'}}|q'-q|_{L^r}. \notag
\end{align*}

Integrating the above in $ s $ from $0$ to $1$ and using (<ref>), we get

\begin{align}
\label{e.estim.der.der}
\Ll|\la \eta, \dr_q\bar F_N(t,{q}')\ra_\cH - \la \eta, \dr_q\bar F_N(t,{q})\ra_\cH\Rr|\leq 16 N|\eta|_{L^{r'}}|{q}'-{q}|_{L^r}.
\end{align}

We established (<ref>) for every $q, q' \in \mcl Q_\infty$ and $\eta \in L^\infty$. By duality, this implies that, for every $q, q' \in \mcl Q_\infty$,

\begin{equation*}  %\label{e.}
\Ll| \dr_q \bar F_N(t,q') - \dr_q \bar F_N(t,q) \Rr|_{L^r} \le 16N|q'-q|_{L^r}.
\end{equation*}

For $r < +\infty$, we can use (<ref>) to extend this estimate to every $q,q' \in \mcl Q_2$ with $q'-q \in L^r$. The case $r = +\infty$ can then be obtained by using the result for finite $r$ and letting $r$ tend to infinity. The proof is therefore complete.

Recall from Proposition <ref> that the initial condition is given by $\psi = \bar F_1(0,\cdot)$, which can be written explicitly as

\begin{align}\label{e.psi=}
\psi({q}) = - \E\log  \iint \exp\Ll(\sqrt{2}w^{q}(\alpha)\cdot \sigma - {q}(1)\cdot \sigma\sigma^\intercal\Rr) \d P_1(\sigma) \d \fR(\alpha).
\end{align}

Proposition <ref> immediately yields similar statements concerning the function $\psi$. We record them here for future reference. For every ${q}\in\C_\infty$, we write

\begin{align}\label{e.<>_mu}
\la \cdot \ra_{q}\propto \exp \Ll(w^{{q}}(\alpha)\cdot\sigma-{q}(1)\cdot\sigma\sigma^\intercal\Rr) \d P_1(\sigma)\d\fR(\alpha).
\end{align}

The function $\psi$ is $L^1$-Lipschitz continuous, with Lipschitz constant equal to $1$.
The restriction of the function $\psi$ to $\mcl Q_2$ is Fréchet (and Gateaux) differentiable everywhere; we denote its Fréchet (and Gateaux) derivative by $\dr_q \psi(q) = \dr_q \psi(q, \cdot) \in L^2([0,1]; S^D)$. We have, for every $q \in \mcl Q_2$,

\begin{equation}
\label{e.bound.der.psi}
\partial_q \psi(q) \in \mcl Q \cap L^\infty_{\le 1},
\end{equation}

and, for every $q \in \mcl Q_\infty$ and $\kappa \in L^2([0,1]; S^D)$,

\begin{equation}
\label{e.def.der.psi}
\la \kappa,\dr_q \psi({q})\ra_\cH = \E \la \kappa \Ll(\alpha\wedge\alpha'\Rr)\cdot \sigma\sigma'^\intercal\ra_q.
\end{equation}

Moreover, for every $r \in [1,+\infty]$ and $q, q' \in \mcl Q_2$ with $q'-q \in L^r$, we have

\begin{equation}
\label{e.continuity.der.psi}
\Ll|\dr_q \psi({q}) -\dr_q \psi({q'})\Rr|_{L^r}\leq 16\Ll|{q}-{q'}\Rr|_{L^r}.
\end{equation}

In particular, the mapping $q \mapsto \dr_q \psi(q)$ can be extended to $\mcl Q_1$ by continuity, and the properties in (<ref>) and (<ref>) remain valid with $q, q' \in \mcl Q_1$.

We recall that Proposition <ref> asserts that the sequence $(\bar F_N)_{N \in \N}$ is precompact. The regularity estimates on $\bar F_N$ that hold uniformly over $N$ naturally transfer to any subsequential limit. Using also the results from Section <ref>, we infer that any such limit must be Gateaux differentiable “almost everywhere”. We summarize these points in the next proposition.

Suppose that $\bar F_N$ converges to some limit $f$ pointwise along some subsequence.
Then for every $r \in (1,+\infty]$, the function $\bar F_N$ converges locally uniformly to $f$ in $\R_+ \times \mcl Q_r$ along the subsequence. We have that $f$ satisfies the same Lipschitz and local semi-concavity properties as $\bar F_N$ stated in Propositions <ref> and <ref>, and that $f$ is $\mcl Q_2^\star$-increasing as $\bar F_N$ is from Proposition <ref>. Moreover,

* for each $t\geq 0$,
there is a Gaussian null set $\mathcal{N}_t$ of $\cH$ such that $f(t,\cdot) : \mcl Q_2 \to \R$ is Gateaux differentiable at every point in $\C_2\setminus\mathcal{N}_t$ and $ (\C_\uparrow\cap L^\infty)\setminus\mathcal{N}_t$ is dense in $\C_2$;

* there is a Gaussian null set $\mathcal{N}$ of $\R\times\cH$ such that $f : \R_+ \times \mcl Q_2 \to \R$ is Gateaux differentiable on $(\R_+\times \C_2)\setminus\mathcal{N}$ and $(\R_+\times (\C_\uparrow\cap L^\infty))\setminus\mathcal{N}$ is dense in $\R_+\times \C_2$,

In the statement of Proposition <ref> and throughout the paper, we allow ourselves some imprecision when saying that $\bar F_N$ converges pointwise to $f$ along a subsequence, since we do not clearly specify over which set this pointwise convergence is valid. As long as there exists a dense set of points $(t,q) \in \R_+ \times \mcl Q_1$ such that $\bar F_N(t,q)$ converges to $f(t,q)$ along the subsequence, the convergence immediately extends to every $(t,q) \in \R_+ \times \mcl Q_1$ by the Lipschitz continuity property in Proposition <ref>.

The fact that the convergence of $\bar F_N$ to $f$ along the subsequence holds locally uniformly in $\R_+\times \mcl Q_r$, for every $r \in (1,+\infty]$, was already mentioned and follows from Proposition <ref>. The fact that $f$ is Lipschitz, locally semi-concave, and $\mcl Q_2^*$-increasing is immediate from Propositions <ref> and <ref>. The differentiability statements follow from Proposition <ref>.

The next proposition asserts that at points of differentiability of the limit $f$, the derivatives of $\bar F_N$ converge to those of $f$.

Suppose that $\bar F_N$ converges pointwise to some limit $f$ along a subsequence $(N_k)_{k \in \N}$.

(1) For each $t \ge 0$, if $f(t,\cdot)$ is Gateaux differentiable at $q \in \C_\uparrow$, then $\partial_q \bar F_{N_k}(t, q, \cdot)$ converges in $L^r$ to $\partial_q f(t,q,\cdot)$ for every $r \in [1,+\infty)$.

(2) For each $q \in \mcl Q_1$, if $f(\cdot,q)$ is differentiable at $t > 0$, then $\partial_t \bar F_{N_k}(t, q)$ converges to $\partial_t f(t,q)$.

We only show part (1) of the statement, since the second part is only easier. We decompose the proof into three steps.

Step 1. We start by recalling that for every $a,b \in S^D$, the largest eigenvalue of $a+b$ is smaller than the sum of the largest eigenvalues of $a$ and $b$, and similarly with the smallest eigenvalue. This can be seen from the variational representation of the largest and smallest eigenvalues of a symmetric matrix.

Recall also the definition of $\mcl Q_{\uparrow, c}$ in (<ref>), and that $\mcl Q_{\uparrow} = \bigcup_{c > 0} \mcl Q_{\uparrow, c}$. Let $q \in \mcl Q_{\uparrow, c}$, and let $\kappa \in C^\infty([0,1]; S^D)$ be a smooth function with $\kappa(0) = 0$. In this step, we show that there is $\delta > 0$ such that for every $s \in [-\de, \de]$, we have $q + s \kappa \in \mcl Q_{\uparrow,\frac{c}{2}}$.

Let $\eta \in (0, c)$ be a parameter to be determined in the course of the argument, and let $\delta > 0$ be sufficiently small that for every $u\le v \in [0,1]$, every eigenvalue of $\delta (\kappa(u)- \kappa(v))$ lies in the interval $[-\eta (v-u),\eta (v-u)]$. (This amounts to taking $\delta > 0$ sufficiently small that every eigenvalue of $\de \dot \kappa(u)$ belongs to $[-\eta , \eta ]$ for every $u \in [0,1]$.) For  every $s \in [-\de, \de]$ and $0 \le u \le v < 1$, we have

\begin{equation*}  %\label{e.}
(q+s\kappa)(v)-(q+s\kappa)(u)\geq (c-\eta )(v-u)\id.
\end{equation*}

Let $\lambda_{\max}$ and $\lambda_{\min}$ be the largest and the smallest eigenvalues of $q(v)-q(u)$ respectively. We have

\begin{align*}
\ellipt((q+s\kappa)(v)-(q+s\kappa)(u))\leq
\frac{\lambda_{\max} + \eta(v-u)}{\lambda_{\min} - \eta(v-u)} \leq \frac{1 + \tfrac \eta c}{1 - \tfrac \eta c} \, \frac{\lambda_{\max}}{\lambda_{\min}},
%    \frac{\lambda_{\max}}{\lambda_{\min}-\tfrac c 4(u-v)} + \frac{\tfrac c 4 (u-v)}{\lambda_{\min}-\tfrac c 4(u-v)}
\end{align*}

where we used that $\lambda_{\max} \ge \lambda_{\min}\geq c(v-u)$ in the second inequality. In view of the last two displays, we obtain the announced result by choosing $\eta = \frac c 3$.

Step 2. In this step, we show part (1) of the statement except that we only prove that $\partial_q \bar F_{N_k}(t, q) = \partial_q \bar F_{N_k}(t_{N_k}, q_{N_k}, \cdot)$ converges weakly in $L^2$ to $\partial_q f(t,q) = \partial_q f(t,q,\cdot)$. The notion of convergence will be upgraded in the next step.

By assumption, there exists $c > 0$ such that $q \in \mcl Q_{\uparrow, c}$. Let $\kappa \in C^\infty([0,1]; S^D)$ be a smooth function with $\kappa(0) = 0$. By the result of the previous step, we can find $\delta > 0$ such that for every $s \in [-\de, \de]$, we have $q + s \kappa \in \mcl Q_{\uparrow, \frac c 2}$.
By Proposition <ref>, there is a constant $C < +\infty$ (depending on $c$) such that for every $N \in \N$, $s \in [-\delta, \delta]$, and  $\lambda \in [0,1]$, we have

\begin{align*}
(1-\lambda)\bar F_{N_k}(t, {q}) + \lambda \bar F_{N_k}(t,{q}+ s \kappa) - \bar F_{N_k}(t, {q}+\lambda s \kappa) \leq C\lambda(1-\lambda)s^2 |\dot\kappa|^2_\cH.
\end{align*}

Dividing both sides by $\lambda$ and sending $\lambda$ to zero, we get

\begin{align*}
\bar F_{N_k}(t,{q}+s\kappa) - \bar F_{N_k}(t, {q}) -s\la \kappa,\dr_q \bar F_{N_k}(t, {q})\ra_\cH \leq Cs^2 |\dot\kappa|^2_\cH.
\end{align*}

Since $|\dr_q \bar F_{N_k}(t, {q})|_\cH$ is bounded uniformly in $k \in \N$ due to Proposition <ref>, for any subsequence $(N'_k)_{k\in\N}$ of $(N_k)_{k\in\N}$, we can extract a further subsequence $(N''_k)_{k\in\N}$ along which $\dr_q \bar F_{N}(t, {q})$ converges weakly to some ${p}\in\cH$. Passing to the limit along $(N''_k)_{k\in\N}$, we get

\begin{align*}
f(t,{q}+s\kappa) -f(t,{q}) - s \la \kappa, {p}\ra_\cH \leq Cs^2|\dot\kappa|^2_\cH
\end{align*}

for every $s\in[-\delta,\delta]$.
Sending $s$ to zero and using the Gateaux differentiability of $f(t,\cdot)$ at ${q}$, we obtain that

\begin{equation*}  %\label{e.}
\la \kappa ,\dr_q f(t,{q}) \ra_\cH = \la \kappa, {p}\ra_\cH.
\end{equation*}

Since the set of $\kappa$'s that we have considered is dense in $\cH$, we  conclude that $\dr_q f(t, {q}) = {p}$.

We have thus shown that any subsequence $(N'_k)_{k\in\N}$ of $(N_k)_{k\in\N}$ has a further subsequence $(N''_k)_{k\in\N}$ along which $\dr_q \bar F_{N}(t, {q})$ converges weakly to $\dr_q f(t, {q})$. This therefore completes the argument for the weak convergence in $L^2$ of $\dr_q \bar F_{N_k}(t,q)$ to $\dr_q f(t,q)$.

Step 3. In this step, we improve the convergence obtained in the previous step from weakly in $L^2$ to strongly in $L^r$ for every $r \in [1,+\infty)$.  We recall from (<ref>) that $\dr_q \bar F_N(t,q) \in \mcl Q \cap L^\infty_{\le 1}$. The strong convergence in $L^r$ therefore follows from Lemma <ref>. The proof is therefore complete.

§ CAVITY CALCULATIONS AND ULTRAMETRICITY

The goal of this section is to present a number of cavity calculations and exploit the asymptotic ultrametricity of the Gibbs measure. Our arguments contain and extend the method introduced in [3] and now often called the Aizenman–Sims–Starr scheme. Roughly speaking, the cavity calculation of [3] consists in comparing $\bar F_{N+1}$ with $\bar F_N$ by separating out the integration of one of the spin variables. Next, if one knows that the Gibbs measure is approximately ultrametric, then one can observe that the integration of the spun-off spin variable can be expressed in the form of

\begin{equation*}  %\label{e.}
\psi({q}+t\nabla\xi({p})) -t \int_0^1\big(p \cdot \nabla \xi(p) - \xi(p)\big),
\end{equation*}

for some $p \in \C_\infty$ which may depend on $N$. Ultrametricity can indeed be enforced by means of small perturbations of the Hamiltonian, as was shown in [76] and reviewed in detail in [77]. Intuitively, for each fixed $N$, the natural choice of $p$ is such that the law of the overlap under the “cavity Gibbs measure” with $N$ spins is the law of $p(U)$, where $U$ is uniformly distributed over $[0,1]$.

We will perform a family of other cavity calculations that will allow us to keep track of arbitrary continuous observables of the overlap. This will have two main benefits that will be detailed in the next section. The first one is that we will indeed be able to track the identity of $p$ in the calculation sketched above. Under a differentiability assumption on the limit free energy $f$, we will justify that we can choose $p = \dr_q f(t,q)$. The second crucial benefit is that we will discover another property that this $p$ must satisfy, namely, that

\begin{equation*}  %\label{e.}
p = \dr_q \psi(q + t \nabla \xi(p)).
\end{equation*}

Collecting all this information then essentially yields a proof of our main results.

This section is organized as follows. We first define the various terms that will enter into the cavity calculations. We next perform the Aizenman–Simms–Starr calculation for the free energy. We then do a cavity calculation for a general observable of the overlap of the spun-off spin variables. We continue by establishing that the quantities we obtain in these calculations can be approximated arbitrarily closely by continuous functions of the law of the overlap array. We next discuss Ghirlanda–Guerra identities, which are enforced by small perturbations of the Hamiltonian and ensure ultrametricity. The ultrametricity property implies that the law of the overlap array is a continuous function of the law of the overlap between only two replicas; and Poisson–Dirichlet cascades provide us with a canonical representation of these objects. So all the “cavity averages” we obtained in the cavity calculations, which involve the “cavity Gibbs measure”, can in fact be re-expressed as cavity averages with respect to some fixed Poisson–Dirichlet cascade.

Throughout this section, we fix $(t,{q}) \in \R_+\times \C_\infty$, and the integer $M\in \N$ denotes the number of cavity spins. For the purpose of this work, we only need to consider the case when $M=1$, but we prefer to carry the cavity calculations with a general $M$, as these might be useful for future applications.

§.§ Definitions and notation

§.§.§ Hamiltonians and perturbation

We start by introducing the various Hamiltonians that will enter into the cavity calculations. The basic idea of a cavity calculation is to decompose an element $\rho \in \R^{D \times (N+M)}$ into $\rho = (\sigma, \tau)$, with $\sigma \in \R^{D \times N}$ and $\tau \in \R^{D \times M}$, and to express the free energy of Gibbs averages involving $N+M$ variables as averages of the $\tau$ variables under some Gibbs measure over the $\sigma$ variables. We are in particular interested in separating out the dependence in $\sigma$ and $\tau$ of the Hamiltonian $H_{N+M}(\rho) = H_{N+M}((\sigma,\tau))$. Temporarily focusing on the case $M = 1$, we would like for instance to argue that

\begin{equation}
\label{e.expansion.HN}
H_{N+1}(\sigma,\tau) \simeq H_{N+1}(\sigma,0) + \tau \cdot \nabla_\tau H_{N+1}(\sigma,0),
\end{equation}

where we use the somewhat informal notation $\nabla_\tau H_{N+1}$ to denote the gradient in $\tau$ of the mapping $(\sigma,\tau) \mapsto H_{N+1}(\sigma,\tau)$. Since $\rho \rho^\intercal = \si \si^\intercal + \tau \tau^\intercal$, we have from (<ref>) that for every $(\sigma,\tau), (\sigma',\tau') \in \R^{D\times (N+1)}$,

\begin{equation*}  %\label{e.}
\E \Ll[ H_{N+1}(\sigma,\tau) H_{N+1}(\sigma',\tau') \Rr] = (N+1) \xi \Ll( \frac{\sigma \sigma'^\intercal + \tau \tau'^\intercal}{N+1} \Rr) .
\end{equation*}

Differentiating this expression, we get that for every $a, b \in \R^D$,

\begin{equation}
\label{e.diff1.defxi}
\E \Ll[ H_{N+1}(\sigma,\tau) \big(b\cdot \nabla_\tau H_{N+1}(\sigma',\tau')\big) \Rr] = \tau b^\intercal \cdot \nabla \xi \Ll( \frac{\sigma \sigma'^\intercal + \tau \tau'^\intercal}{N+1} \Rr)
\end{equation}

\begin{equation}
\label{e.diff2.defxi}
\E \Ll[ \big(a \cdot \nabla_\tau H_{N+1}(\sigma,0)\big) \big(b \cdot \nabla_\tau H_{N+1}\big)(\sigma',0) \Rr] = b a^\intercal \cdot \nabla \xi \Ll( \frac{\sigma \sigma'^\intercal}{N+1} \Rr) .
\end{equation}

We see from (<ref>) that $(H_{N+1}(\sigma,0))_{\sigma \in \R^{D\times N}}$ and $(\nabla_\tau H_{N+1}(\sigma,0))_{\sigma \in \R^{D \times N}}$ are independent Gaussian fields. The Hamiltonian $H_{N+1}(\sigma,0)$ will serve as our reference Hamiltonian, the common part between $H_{N+1}$ and $H_N$, and we will similarly expand $H_N(\sigma)$ into $H_{N+1}(\sigma,0)$ plus an independent Gaussian field. From now on, we revert to the setting with general $M \in \N$, and introduce convenient notation to denote the various Hamiltonians entering our calculations. While we may not always repeat it, each time we introduce a new Hamiltonian or other Gaussian field, we impose that it be independent of all the previously-defined random fields; and the expectation $\E$ is understood to average all these processes.

In order to enforce the asymptotic validity of the Ghirlanda–Guerra identities, and thereby the ultrametricity of the Gibbs measure, we need to introduce a rich family of perturbations of the Hamiltonian.
Let $(\lambda_n)_{n\in\N}$ be an enumeration of $[0,1]\cap \Q$, and $(a_n)_{n\in\N}$ be an enumeration of $\{a\in \S^\D_+ :|a|\leq 1\}\cap \Q^{D\times D}$. For every $h\in \N^4$, let $(H^h_N(\sigma,\alpha))_{\sigma\in\R^{\D\times N},\,\alpha\in \mfk U}$
be an independent centered Gaussian process with covariance

\begin{align*}%\label{e.H^h}
\E \Ll[H^h_N(\sigma,\alpha)H^h_N(\sigma',\alpha')\Rr] = N\Ll(a_{h_1}\cdot \Ll(\frac{\sigma\sigma'^\intercal}{N}\Rr)^{\odot h_2}+\lambda_{h_3} \alpha\wedge\alpha'\Rr)^{h_4}
\end{align*}

where $\odot$ is the Schur product of matrices, i.e. $a\odot b = (a_{i,j}b_{i,j})_{i,j}$.
Let us explain why this process exists. By <cit.>, there exists a Gaussian process with covariance $a\cdot \Ll(\frac{\sigma\sigma'^\intercal}{N}\Rr)^{\odot n}$ for every $a\in\S^\D_+$ and $n\in\N$. Proposition <ref> ensures the existence of a Gaussian process with covariance $\alpha\wedge\alpha'$. Linear combinations with non-negative coefficients of covariance functions are still covariance functions, since we can simply take superpositions of independent processes to realize them, and <cit.> also allows for the multiplication of two covariance functions. Finally, the measurability can be handled as in the proof of Proposition <ref>.

For each $h \in \N^4$, let $c_h>0$ be a constant such that

\begin{align*}%\label{e.c_h}
c_h \sqrt{\frac{1}{N}\E  H^h_N(\sigma,\alpha)H^h_N(\sigma,\alpha) }\leq 2^{-|h|_1},
\end{align*}

uniformly over $\sigma \in \supp P_N$, $\alpha \in \mfk U$, and $N \in \N$, and where we write $|h|_1 := \sum_{i=1}^4 h_i$.
Let $\{e_i\}_{i=1}^{D(D+1)/2}$ be an orthonormal basis of $\S^{D}$.
For every

\begin{align}\label{e.x_pert}
x = \Ll((x_h)_{h\in\N^4}, (x_i)_{i=1}^{D(D+1)/2} \Rr)\in [0,3]^{\N^4 \times \frac{D(D+1)}{2}},
\end{align}

we set

\begin{align}\label{e.H^pert}
\begin{split}
H_{N}^{x}(\sigma,\alpha) &:= \check H_N^{x}(\sigma,\alpha)+ \sum_{i=1}^{D(D+1)/2} x_i e_i\cdot \sigma\sigma^\intercal, \qquad \text{where}
\\
\check H_N^{x}(\sigma,\alpha)&:= \sum_{h\in \N^4} x_h  c_h H^h_N(\sigma,\alpha).
\end{split}
\end{align}

The second term in the definition of $H_N^x$ will ensure the concentration of the “self-overlap” $\frac{\sigma \sigma^\intercal}{N}$. Notice that we impose each coordinate of $x$ to take values in the interval $[0,3]$.
We define the free energy with perturbation

\begin{align}
\bar F^x_N(t,{q}) &:= -\frac{1}{N}\E\log \iint \exp\Ll(H^{t,{q}}_N(\sigma,\alpha) + N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr)\d P_N(\sigma) \d \fR(\alpha),
\label{e.F^x_N=}
\end{align}

and its associated Gibbs measure

\begin{align}
\la\cdot\ra^\circ_{N,x} &\propto  \exp\Ll(H^{t,{q}}_N(\sigma,\alpha) + N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr)\d P_N(\sigma) \d \fR(\alpha).\label{e.<>^circ_Nx=}
\end{align}

We keep denoting by $(\sigma,\alpha)$ the canonical random variable under $\la\cdot\ra^\circ_{N,x}$, and we write $(\sigma^\ell, \alpha^\ell)_{\ell \ge 1}$ to denote independent copies of $(\sigma,\alpha)$.
The expectation $\E$ in (<ref>) integrates all the sources of Gaussian randomness and the randomness of $\fR$.

Our “reference” Hamiltonian for the cavity calculation is denoted by $(\tilde H_N(\sigma))_{\sigma\in \R^{D\times N}}$, and it is the centered Gaussian process such that, for every $\si, \si' \in \R^{D\times N}$,

\begin{align*}
\E \Ll[\tilde H_N(\sigma)\tilde H_N(\sigma')\Rr] = (N+M)\xi\Ll(\frac{\sigma\sigma'^\intercal}{N+M}\Rr).
\end{align*}

This is simply an independent copy of what we were denoting $H_{N+M}(\sigma,0)$ at the opening of this section. Notice that the parameter $M$ appears in the definition of the covariance above. Abusing notation, we prefer to keep the dependence of $\tilde H_N$ on $M$ implicit, as it only takes $\sigma \in \R^{D\times N}$ as input.
We let $\tilde W^{q}_N$ be an independent copy of $W^{q}_N$ defined in (<ref>), and, for every $\sigma \in \R^{D\times N}$ and $\alpha \in \mfk U$, we set

\begin{align*}%\label{e.tilde_H^t,mu}
\tilde H^{t,{q}}_N(\sigma,\alpha) := \sqrt{2t}\tilde H_N(\sigma) - t(N+M)\xi\Ll(\frac{\sigma\sigma^\intercal}{N+M}\Rr) +\tilde W^{q}_N(\alpha)\cdot \sigma - {q}(1)\cdot\sigma\sigma^\intercal.
\end{align*}

We denote the free energy and the Gibbs measure arising in the cavity calculation by

\begin{align}\label{e.tildeF_N=}
\tilde F_N^x(t,{q}) := -\frac{1}{N}\E\log\iint \exp\Ll(\tilde H^{t,{q}}_N(\sigma,\alpha)+N^{-\frac{1}{16}} H^{x}_{N}(\sigma,\alpha)\Rr)\d P_N(\sigma)\d\fR(\alpha)
\end{align}

\begin{align}\label{e.<>_N,x}
\la\cdot\ra_{N,x} \propto \exp\Ll(\tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr)\d P_N(\sigma)\d\fR(\alpha).
\end{align}

§.§.§ Notation for the overlaps

For every $N\in\N$, $\ell,\ell'\in\N$, $h\in \N^4$, and $n\in \N$, we write

\begin{align}\label{e.overlap_notation}
\begin{cases}
R^{\ell,\ell'}_{N,\sigma} := \frac{\sigma^\ell\Ll(\sigma^{\ell'}\Rr)^\intercal}{N},\qquad R^{\ell,\ell'}_\alpha :=\alpha^\ell\wedge\alpha^{\ell'},\qquad R^{\ell,\ell'}_N :=\Ll(R^{\ell,\ell'}_{N,\sigma}, R^{\ell,\ell'}_\alpha\Rr);
\\
R^{\ell,\ell'}_{N,h} := \Ll(a_{h_1}\cdot \Ll(R^{\ell,\ell'}_{N,\sigma}\Rr)^{\odot h_2}+\lambda_{h_3} R^{\ell,\ell'}_\alpha\Rr)^{h_4};
\\
R_N := \Ll(R^{\ell,\ell'}_N\Rr)_{\ell,\ell'\in\N},\qquad R^{\leq n}_N := \Ll(R^{\ell,\ell'}_N\Rr)_{\ell,\ell'\leq n} .
\end{cases}
\end{align}

For every ${p} \in \C_\infty$, we set

\begin{align}\label{e.overlap_alpha}
Q^{\ell,\ell'}_{p} := \Ll({p}\Ll(R^{\ell,\ell'}_\alpha\Rr), R^{\ell,\ell'}_\alpha\Rr),\qquad
Q_{p} := \Ll(Q^{\ell,\ell'}_{p}\Rr)_{\ell,\ell'\in \N},\qquad Q^{\leq n}_{p} := \Ll(Q^{\ell,\ell'}_{p}\Rr)_{1\leq \ell,\ell'\leq n}.
\end{align}

Under some circumstances, we will be able to argue that the $R_{N,\sigma}$-overlaps synchronize with the $R_\alpha$-overlaps; this means that the $R_N$-overlaps are close to the $Q_p$-overlaps for a suitable choice of $p$.

§.§.§ Definitions for the free-energy cavity calculation

We now define the Gibbs average that will show up in the free-energy cavity calculation, or in other words in the Aizenman–Sims–Starr scheme [3]. We define $\theta:\R^{\D\times\D}\to\R$ such that, for every $a \in \R^{D \times D}$,

\begin{align}\label{e.theta=}
\theta(a) := a\cdot \nabla \xi(a) - \xi(a).
\end{align}

We define the following independent centered Gaussian processes indexed by $\sigma\in \R^{D\times N}$:

* let $\msf Z(\sigma) = (\msf Z_1(\sigma),\dots, \msf Z_M(\sigma))$ be an $M$-tuple of independent $\R^{D}$-valued centered Gaussian vectors $\msf Z_i(\sigma)$ with covariance $\E \msf Z_i(\sigma) \msf Z_i(\sigma')^\intercal =\nabla \xi (\frac{\sigma\sigma'^\intercal}{N})$;

* let $\msf Y(\sigma)$ be real-valued with covariance $\E \msf Y(\sigma)\msf Y(\sigma') =  M\theta (\frac{\sigma\sigma'^\intercal}{N})$.

The fact that there indeed exists such a Gaussian process $\msf Z$ follows from (<ref>) and a simple rescaling. Concerning the existence of the process $\msf Y$, we recall from Subsection <ref> that $\xi$ takes the form given in (<ref>), where the matrices $\mathsf{C}^{(p)}$ belong to $S^{D^p}_+$ and the series is absolutely convergent. A direct calculation yields that

\begin{equation*}  %\label{e.}
\theta(a) = \sum_{p = 1}^\infty (p-1)\mathsf{C}^{(p)}\cdot a^{\otimes p}.
\end{equation*}

This is of the same form as (<ref>), so the existence of the process $\msf Y$ follows from the second part of Subsection <ref>.

For every $\sigma \in \R^{D\times N}$, $\tau \in \R^{D \times M}$, and $\alpha \in \mfk U$, we set

\begin{align}\label{e.U=}
U(\sigma,\alpha,\tau) := \sqrt{2t}\msf Z(\sigma)\cdot \tau - t\nabla\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)\cdot\tau\tau^\intercal + \sqrt{2}W^{{q}}_M(\alpha)\cdot\tau-{q}(1)\cdot\tau\tau^\intercal,
\end{align}

and for every $x \in[0,3]^{\N^4 \times \frac{D(D+1)}{2}}$,

\begin{multline}
\label{e.A_N(x)}
A_N(x)  := \E \log \la \int \exp\Ll(U(\sigma,\alpha,\tau)\Rr)\d P_M(\tau) \ra_{N,x}
\\
- \E \log \la \exp\Ll(\sqrt{2t}\msf Y(\sigma)- tM\theta \Ll( \frac{\sigma\sigma^\intercal}{N}\Rr)\Rr) \ra_{N,x}.
\end{multline}

With $\psi$ and $\theta$ defined in (<ref>) and (<ref>), we set, for every ${p}\in \C_\infty$,

\begin{align}\label{e.mathscrP}
\mathscr{P}_{t,{q}}({p}) := \psi({q}+t\nabla\xi({p})) -t \int_0^1\theta({p}(s))\d s.
\end{align}

We will show that the limit of $A_N(x)$ is related to $\mathscr{P}_{t,{q}}({p})$ for some ${p}$.

§.§.§ Definitions for the Gibbs-average cavity calculation

We write $\rho \in \R^{\D\times (N+ M)}$ as $\rho = (\sigma,\tau) \in \R^{\D\times N}\times \R^{\D\times M}$.
For a bounded continuous function $g:\R^{\D\times\D}\times \R\to\R$, we want to find the limit of $\E \la g(\tau\tau'^\intercal,\alpha\wedge\alpha')\ra_{N+M,x}^\circ$ as $N$ tends to infinity. To study this, we need a few more definitions.
We define the Gibbs measure $\la\cdot\ra_{N+M,x}^\star$ by

\begin{align}\label{e.<>_N^star}
\la\cdot\ra_{N+M,x}^\star\propto\exp\Ll(\tilde H_N^{t,{q}}(\sigma,\alpha) +  U(\sigma,\alpha,\tau)
+ N^{-\frac{1}{16}}  H^{x}_N(\sigma,\alpha)\Rr)\d P_{N+M}(\sigma,\tau) \d\fR(\alpha).
\end{align}

The canonical random variable under $\la\cdot\ra_{N+M,x}^\star$ is denoted by $(\rho, \alpha) = (\sigma,\tau,\alpha)$.

For every $\pi \in \C_\infty$, we define the Gibbs measure $\la\cdot\ra_{\fR,\pi}$ by

\begin{align}\label{e.<>_R,pi}
\la \cdot\ra_{\fR,\pi} \propto \exp\Ll(\sqrt{2}W^\pi_M(\alpha)\cdot\tau -\pi(1)\cdot \tau\tau^\intercal\Rr) \d P_M(\tau) \d \fR(\alpha),
\end{align}

and we denote by $(\tau,\alpha)$ the canonical random variable under $ \la \cdot\ra_{\fR,\pi}$.
We will show that the limit of $\E \la g(\tau\tau'^\intercal,\alpha\wedge\alpha')\ra_{N+M,x}^\circ$ is related to $\E \la g(\tau\tau'^\intercal,\alpha\wedge\alpha') \ra_{\fR, \pi}$ for some $\pi$.

§.§ Cavity calculations per se

§.§.§ Free-energy cavity calculation

We have, uniformly over $M,N \in \N$ and $x \in [0,3]^{\N^4 \times \frac{D(D+1)}{2}}$,

\begin{align*}%\label{e.cavityF^x}
&-(N+M)\bar F_{N+M}^x(t,{q}) + N\bar F_N^x(t,{q}) = A_N(x)  + O\Ll(M^2N^{-\frac{1}{16}}\Rr).
\end{align*}

The argument consists in a rigorous justification of the expansion (<ref>) and of a similar expansion for $H_N$.
We keep writing $\rho = (\sigma,\tau) \in \R^{D\times (N+M)}$, so that $\rho\rho^\intercal = \sigma\sigma^\intercal + \tau\tau^\intercal$ and

\begin{align*}
W^{q}_{N+M}(\alpha)\cdot \rho \stackrel{\d}{=} \tilde W^{q}_N(\alpha)\cdot \sigma + W^{q}_M(\alpha)\cdot \tau.
\end{align*}

Using the Taylor expansion of $\xi$, the local Lipschitzness of $\nabla \xi$, and $|\frac{\sigma\sigma^\intercal}{N}-\frac{\sigma\sigma^\intercal}{N+M}| = O(\frac{M}{N})$, we get, for every $\rho$ in the support of $P_{N+M}$,

\begin{align}\label{e.(N+M)xi(...)=}
\begin{split}
(N+M)\xi\Ll(\frac{\rho\rho^\intercal}{N+M}\Rr) & = (N+M)\xi \Ll(\frac{\sigma\sigma^\intercal}{N+M}\Rr)+ \nabla\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)\cdot \tau\tau^\intercal + O\Ll(\frac{M^2}{N}\Rr),
\\
N\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr) & = (N+M)\xi\Ll(\frac{\sigma\sigma^\intercal}{N+M}\Rr) + M \theta \Ll(\frac{\sigma\sigma^\intercal}{N}\Rr) + O\Ll(\frac{M^2}{N}\Rr).
\end{split}
\end{align}

These two relations yield, respectively,

\begin{align}\label{e.EHH=EHH}
\begin{split}
\E H_{N+M}(\rho) H_{N+M}(\rho') &= \E \tilde H_N(\sigma)\tilde H_N(\sigma') + \E (\msf Z(\sigma)\cdot \tau)(\msf Z(\sigma')\cdot \tau') + O\Ll(\frac{M^2}{N}\Rr),
\\
\E H_N(\sigma)H_N(\sigma') &= \E\tilde H_N(\sigma)\tilde H_N(\sigma') + \E \msf Y(\sigma)\msf Y(\sigma') + O\Ll(\frac{M^2}{N}\Rr)
\end{split}
\end{align}

uniformly over $\rho$ and $\rho'$ in the support of $P_{N+M}$.
Recalling $H^{x}_N(\sigma,\alpha)$ and $\check H^{x}_N(\sigma,\alpha)$ in (<ref>), we verify that

\begin{multline}
\label{e.EHpertHpert=}
(N+M)^{-\frac{1}{8}}\E \check H^{x}_{N+M}(\rho,\alpha) \check H^{x}_{N+M}(\rho',\alpha')
\\ =N^{-\frac{1}{8}} \E \check H^{x}_N(\sigma,\alpha) \check H^{x}_N(\sigma',\alpha') + O\Ll(MN^{-\frac{1}{8}}\Rr).
\end{multline}

Using again that $\rho\rho'^\intercal = \sigma\sigma'^\intercal + \tau\tau'^\intercal$, we see that the second term on the right-hand side in (<ref>) satisfies

\begin{align}\label{e.sumxg(sigma)=}
(N+M)^{-\frac{1}{16}}\sum_{i=1}^{D(D+1)/2} x_ie_i\cdot \rho\rho^\intercal = N^{-\frac{1}{16}}\sum_{i=1}^{D(D+1)/2} x_ie_i\cdot \sigma\sigma^\intercal+ O\Ll(MN^{-\frac{1}{16}}\Rr).
\end{align}

We want to insert the above relations into

\begin{align*}
&-(N+M)\bar F_{N+M}^x(t,{q}) + N\bar F_N^x(t,{q})
\\ &= \E \log \iint \exp \Ll(H^{t,{q}}_{N+M}(\rho,\alpha)+(N+M)^{-\frac{1}{16}}H^{x}_{N+M}(\rho,\alpha) \Rr)\d P_{N+M}(\rho) \d \fR(\alpha)
\\
&- \E \log \iint \exp\Ll( H^{t,{q}}_{N}(\sigma,\alpha) +N^{-\frac{1}{16}}H^{x}_{N}(\sigma,\alpha)\Rr)\d P_{N}(\sigma) \d \fR(\alpha)=: \mathrm{I}_0 - \mathrm{II}_0 .
\end{align*}

Recalling $\la\cdot\ra_{N,x}$ in (<ref>),
we rewrite $A_N(x)$ in (<ref>) as

\begin{align*}
A_N(x)  = \E \log  \iint \exp\Ll(U(\sigma,\alpha,\tau)+ \tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr)\d P_{N+M}(\sigma,\tau) \d \fR(\alpha)
\\
- \E \log \iint\exp\Ll(\sqrt{2t}\msf Y(\sigma)- tM\theta \Ll( \frac{\sigma\sigma^\intercal}{N}\Rr) +\tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr)\d P_N(\sigma) \d \fR(\alpha)
\\
=: \mathrm{I}_1 - \mathrm{II}_1.
\end{align*}

Note that all the Hamiltonians are of the form $\mathbf{G}(\mathbf{s}) + \mathbf{D}(\mathbf{s})$ where the first term $\mathbf{G}(\mathbf{s})$ is centered Gaussian, the second term $\mathbf{D}(\mathbf{s})$ is deterministic, and $\mathbf{s}$ is $(\rho,\alpha)$ or $(\sigma,\alpha)$.
To compare $\mathrm{I}_0$ with $\mathrm{I}_1$ and $\mathrm{II}_0$ with $\mathrm{II}_1$, we consider two interpolations of the form

\begin{gather*}
\varphi(r) := \E \log\int \exp \Ll(\sqrt{1-r}\mathbf{G}^{0}(\mathbf{s})+(1-r)\mathbf{D}^{0}(\mathbf{s})+\sqrt{r}\mathbf{G}^{1}(\mathbf{s})+r\mathbf{D}^{1}(\mathbf{s})\Rr)\d\mathfrak{P}(\mathbf{s}),
\end{gather*}

where $\mathbf{G}^{0}$ and $\mathbf{G}^{1}$ are independent and $\mathfrak{P}$ is $P_{N+M}\otimes \fR$ or $P_N\otimes \fR$.
For the first interpolation between $\mathrm{I}_0$ and $\mathrm{I}_1$, we take

\begin{align}
\label{e.def.GD0}
\mathbf{G}^{0}(\mathbf{s})+\mathbf{D}^{0}(\mathbf{s}) & = H^{t,{q}}_{N+M}(\rho,\alpha)+N^{-\frac{1}{16}}H^{x}_{N+M}(\rho,\alpha),
\\
\label{e.def.GD1}
\mathbf{G}^{1}(\mathbf{s})+\mathbf{D}^{1}(\mathbf{s})  & = U(\sigma,\alpha,\tau)+ \tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha).
\end{align}

We understand that the Gaussian fields $H_{N+M}^x$ and $H_{N}^x$ appearing in (<ref>) and (<ref>) respectively are independent.

For the second interpolation, between $\mathrm{II}_0$ and $\mathrm{II}_1$, we take

\begin{align}
\label{e.def.GD0.bis}
\mathbf{G}^{0}(\mathbf{s})+\mathbf{D}^{0}(\mathbf{s}) & = H^{t,{q}}_{N}(\sigma,\alpha) +N^{-\frac{1}{16}}H^{x}_{N}(\sigma,\alpha),
\\
\label{e.def.GD1.bis}
\mathbf{G}^{1}(\mathbf{s})+\mathbf{D}^{1}(\mathbf{s})  & = \sqrt{2t}\msf Y(\sigma)- tM\theta \Ll( \frac{\sigma\sigma^\intercal}{N}\Rr) +\tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha).
\end{align}

To remain faithful with what we announced, we should take the Gaussian fields $H^x_N(\si, \al)$ appearing in (<ref>) and (<ref>) to be independent. This is of course not necessary here, and taking them to be equal is just as convenient; this can be phrased in our setting by absorbing this term into the definition of $\mathfrak{P}$.

We control $|\varphi(1)-\varphi(0)|$ by computing the derivative

\begin{align*}
\frac{\d}{\d r}\varphi(r) = \E \la \frac{1}{2\sqrt{r}}\mathbf{G}^{1}(\mathbf{s}) - \frac{1}{2\sqrt{1-r}}\mathbf{G}^{0}(\mathbf{s}) + \mathbf{D}^{1}(\mathbf{s}) -\mathbf{D}^{0}(\mathbf{s})\ra_r,
\end{align*}

where $\la\cdot\ra_r$ is the Gibbs measure naturally associated with the free energy expression defining $\varphi(r)$.
We use Gaussian integration by parts (e.g. <cit.>) to get

\begin{align*}
\frac{\d}{\d r}\varphi(r) = \E \la \mathbf{V}(\mathbf{s},\mathbf{s})-\mathbf{V}(\mathbf{s},\mathbf{s}')+ \mathbf{D}^{1}(\mathbf{s}) -\mathbf{D}^{0}(\mathbf{s})\ra_r,
\end{align*}

where $\mathbf{s}'$ is an independent copy of $\mathbf{s}$ under $\la\cdot\ra_r$, and

\begin{align}\label{e.V(s,s')=}
\mathbf{V}(\mathbf{s},\mathbf{s}') := \frac{1}{2}\Ll(\E \mathbf{G}^{1}(\mathbf{s})\mathbf{G}^{1}(\mathbf{s}') - \E \mathbf{G}^{0}(\mathbf{s})\mathbf{G}^{0}(\mathbf{s}') \Rr).
\end{align}

We can then bound $\mathbf{V}$ and $\mathbf{D}^{1} -\mathbf{D}^{0}$ using the estimates in (<ref>), (<ref>), (<ref>), and (<ref>).
For both interpolations, we have $|\frac{\d}{\d r}\varphi(r)|=O(M^2N^{-\frac{1}{16}})$ where $M^2$ comes from (<ref>) and (<ref>), and $N^{-\frac{1}{16}}$ comes from (<ref>) and (<ref>).

For each ${p}\in\mathcal{Q}_\infty$, conditionally on $\fR$ and on the event where $\mfk U = \supp \fR$ satisfies (<ref>), we define the following independent centered Gaussian processes  indexed by $\alpha \in \mfk U$:

* let $Z(\alpha) = (Z_1(\alpha),\dots, Z_M(\alpha))$ be an $M$-tuple of independent $\R^\D$-valued centered Gaussian vectors $Z_i(\alpha)$ with covariance $\E Z_i(\alpha) Z_i(\alpha')^\intercal = \nabla\xi({p}\Ll(\alpha\wedge\alpha'\Rr))$;

* let $Y(\alpha)$ be real-valued with covariance $\E Y(\alpha)Y(\alpha') = M\theta({p}\Ll(\alpha\wedge\alpha'\Rr))$.

Letting also

\begin{equation*}  %\label{e.}
V(\alpha,\tau):= \sqrt{2t} Z(\alpha)\cdot \tau - t \nabla\xi\Ll({p}(1)\Rr)\cdot \tau\tau^\intercal + \sqrt{2}W^{q}_M(\alpha)\cdot \tau - {q}(1)\cdot \tau\tau^\intercal,
\end{equation*}

we have

\begin{multline}
\label{e.-MP^t}
=\E \log  \iint \exp\Ll(V(\alpha,\tau)\Rr)\d P_M(\tau)\d\fR(\alpha)
\\
- \E \log \int \exp\Ll(\sqrt{2t}Y(\alpha)- tM\theta \Ll( {p}(1)\Rr)\Rr) \d\fR(\alpha).
\end{multline}

We denote the right-hand side of (<ref>) by $\mathrm{I}-\mathrm{II}$.
We have

\begin{equation*}  %\label{e.}
\sqrt{2t}Z(\alpha)+\sqrt{2}W^{q}_M(\alpha) \stackrel{\d}{=} \sqrt{2}W^{{q} + t\nabla\xi({p})}_M(\alpha),
\end{equation*}

and thus

\begin{align*}
\mathrm{I} = -M\bar F_M(0,{q} + t\nabla\xi({p})) =- M\psi(0,{q} + t\nabla\xi({p})),
\end{align*}

where we used Proposition <ref> in the last equality.

To compute $\mathrm{II}$, we assume that ${p}$ is a step function; the general case can be recovered by approximation (one can verify that $\mathrm{II}$ is continuous in ${p}$ by a similar argument to that for the continuity in $q$ of $\bar F_N$ from Proposition <ref>). Hence, we assume that ${p}=\sum_{l=1}^kp_l\1_{\Ll[s_{l-1},s_l\Rr)}$ for $0=s_0<s_1<\dots<s_k=1$ and $0\leq p_0\leq p_1\leq\cdots \leq p_k$. For brevity, we also write $\tilde \theta := 2tM\theta$. Following the computation of the second term in <cit.> (comparing the second term in <cit.> with that in <cit.>; substituting $s_l, p_l, \tilde\theta$ for $\zeta_p, q_p, \theta$ therein), we obtain that

\begin{align*}
&\E \log \int \exp\sqrt{2t}Y(\alpha) \d\fR(\alpha) = \frac{1}{2}\sum_{ l=0}^{ k-1}s_l\left(\tilde\theta(p_{l+1})-\tilde\theta(p_l)\right)
\\
&= \frac{1}{2}\left(-\sum_{l=1}^k (s_{l}-s_{l-1})\tilde\theta(p_l) + s_k\tilde\theta(p_k)-s_0\tilde\theta(p_0)\right)
=\frac{1}{2}\left( -\int_0^1\tilde\theta({p}(s))\d s + \tilde\theta({p}(1))\right),
\end{align*}

which yields

\begin{align*}
\mathrm{II}= \E \log \int \exp\sqrt{2t}Y(\alpha) \d\fR(\alpha) -tM\theta({p}(1)) = -tM\int_0^1\theta({p}(s))\d s.
\end{align*}

Comparing $\mathrm{I}$ and $\mathrm{II}$ with the definition of $\mathscr{P}_{t,q}(p)$ in (<ref>), we get the desired result.

§.§.§ Gibbs-average cavity calculation

Recall $\la\cdot\ra^\circ_{N,x}$ in (<ref>) and $\la\cdot\ra_{N+M,x}^\star$ in (<ref>).

For every continuous function $g:\R^{\D\times\D}\times \R\to\R$, we have uniformly over $M, N \in \N$ and $x \in [0,3]^{\N^4 \times \frac{D(D+1)}{2}}$ that

\begin{align*}
\E \la g\Ll(\tau\tau'^\intercal,\alpha\wedge\alpha'\Rr)\ra_{N+M,x}^\circ = \E \la g\Ll(\tau\tau'^\intercal,\alpha\wedge\alpha'\Rr)\ra_{N+M,x}^\star + O\Ll(M^2 N^{-\frac{1}{16}}\Rr) .
\end{align*}

We use the estimates in the proof of Proposition <ref> and a similar interpolation computation.
We write $\Theta := g\Ll(\tau\tau'^\intercal,\alpha\wedge\alpha'\Rr)$, $\mathbf{s}:=(\rho,\alpha)$, and $\mathfrak{P} := P_{N+M}\otimes \fR$.
We rewrite $\la\cdot\ra_{N+M,x}^\circ$ and $\la\cdot\ra_{N+M,x}^\star$ as follows,

\begin{gather*}
\la\cdot\ra_{N+M,x}^\circ \propto \exp\Ll(\mathbf{G}^0(\mathbf{s}) + \mathbf{D}^0(\mathbf{s})\Rr) \d \mathfrak{P}(\mathbf{s}),
\\
\la\cdot\ra_{N+M,x}^\star \propto \exp\Ll(\mathbf{G}^1(\mathbf{s}) + \mathbf{D}^1(\mathbf{s})\Rr) \d \mathfrak{P}(\mathbf{s}),
\end{gather*}

where $\mathbf{G}^i(\mathbf{s})$ collects centered Gaussian terms, $\mathbf{D}^i(\mathbf{s})$ collects deterministic terms in the respective Hamiltonians, and they satisfy

\begin{align*}
\mathbf{G}^0(\mathbf{s}) + \mathbf{D}^0(\mathbf{s}) &= H^{t,{q}}_{N+M}(\rho,\alpha) + (N+M)^{-\frac{1}{16}}H^{x}_{N+M}(\rho,\alpha),
\\
\mathbf{G}^1(\mathbf{s}) + \mathbf{D}^1(\mathbf{s}) &= \tilde H_N^{t,{q}}(\sigma,\alpha) +  U(\sigma,\alpha,\tau)
+ N^{-\frac{1}{16}}  H^{x}_N(\sigma,\alpha).
\end{align*}

Again we understand that $H^x_{N+M}$ and $H^x_N$ are independent here.
For every $r\in[0,1]$, we define the interpolating Gibbs measure

\begin{align*}
\la\cdot\ra_r \propto \exp\Ll(\sqrt{1-r}\mathbf{G}^0(\mathbf{s})+\sqrt{r}\mathbf{G}^1(\mathbf{s})+(1-r)\mathbf{D}^0(\mathbf{s})+r \mathbf{D}^1(\mathbf{s})\Rr)\d \mathfrak{P}(\mathbf{s}),
\end{align*}

and we set $\varphi(r) := \E \la \Theta\ra_r$. We aim to show that $|\frac{\d}{\d r}\varphi(r)|=o(1)$ uniformly in $r$. Notice that $\Theta$ depends on two independent copies of $\mathbf{s}$, which we  denote by $\mathbf s^1$ and $\mathbf s^2$. We also give ourselves $\mathbf{s}^3$ and $\mathbf{s}^4$ two additional independent copies of $\mathbf{s}$. We write the Gibbs measure in the definition of $\varphi(r)$ explicitly as a ratio and differentiate to obtain that $\frac{\d}{\d r} \varphi(r)$ is the $\E$-expectation of the $\langle \cdot \rangle_r$-covariance between $\Theta$ and

\begin{equation*}  %\label{e.}
\frac{1}{2\sqrt{r}}\big(\mathbf{G}^{1}(\mathbf{s}^1) + \mathbf{G}^{1}(\mathbf{s}^2)\big) - \frac{1}{2\sqrt{1-r}}\big(\mathbf{G}^{0}(\mathbf{s}^1) + \mathbf{G}^0(\mathbf{s}^2)\big) + \mathbf{D}^{1}(\mathbf{s}^1)+ \mathbf{D}^{1}(\mathbf{s}^2) -\mathbf{D}^{0}(\mathbf{s}^1) - \mathbf{D}^{0}(\mathbf{s}^2).
\end{equation*}

We can rewrite this covariance as a Gibbs average by utilizing the additional variable $\mathbf{s}^3$. We next perform a Gaussian integration by parts as in e.g. <cit.> to obtain a Gibbs average of the form

\begin{align*}
\frac{\d}{\d r}\varphi(r) = \E \la \Theta\Ll(\sum_{i,j=1}^4 c_{i,j}\mathbf{V}^{i,j}+ \sum_{i=1}^3 c_i \mathbf{U}^i\Rr)\ra_r,
\end{align*}

\begin{align*}
\mathbf{V}^{i,j} &= \E \Ll[\mathbf{G}^1\Ll(\mathbf{s}^i\Rr)\mathbf{G}^1\Ll(\mathbf{s}^j\Rr) - \mathbf{G}^0\Ll(\mathbf{s}^i\Rr)\mathbf{G}^0\Ll(\mathbf{s}^j\Rr)\Rr],\qquad \mathbf{U}^i = \mathbf{D}^1\Ll(\mathbf{s}^i\Rr) - \mathbf{D}^0\Ll(\mathbf{s}^i\Rr),
\end{align*}

and $(c_{i,j})_{i,j=1}^4$ and $(c_i)_{i=1}^3$ are absolute constants.
Then, we can use the estimates (<ref>), (<ref>), (<ref>), and (<ref>) to see that $|\mathbf{V}^{i,j}|$ and $|\mathbf{U}^i|$ are $O\Ll(M^2N^{-\frac{1}{16}}\Rr)$. Since $\Theta$ is also bounded a.s., we deduce $|\frac{\d}{\d r}\varphi(r)|=O\Ll(M^2N^{-\frac{1}{16}}\Rr)$ uniformly in $r$. The desired result thus follows.

§.§.§ Other useful interpolation estimates
We now collect two more useful estimates that are obtained using similar arguments. Recall $\bar F^x_N(t,{q})$ in (<ref>) and $\tilde F^x_N(t,{q})$ in (<ref>).

For each fixed $M \in \N$, we have

\begin{align*}
\lim_{N\to\infty} \sup_{x} \Ll|\bar F^x_N(t,{q}) - \bar{F}_N(t,{q})\Rr| =0,\qquad \lim_{N\to\infty} \sup_{x} \Ll|\tilde F^x_N(t,{q}) - \bar F_N(t,{q})\Rr| =0 ,
\end{align*}

where the suprema are over $x \in [0,3]^{\N^4 \times \frac{D(D+1)}{2}}$.

We use a similar interpolation as in the proof of Proposition <ref>. We recall the normalizing factor $\frac{1}{N}$ in the definition of the free energy.
The first convergence follows from the interpolation and the presence of $N^{-\frac{1}{16}}$ in front of $H^{x}_N$. The second convergence follows from these facts and, additionally, the second line in (<ref>) together with the boundedness of $\E \msf Y(\sigma)\msf Y(\sigma')$.

There exists $C < +\infty$ such that for every $N, M \in \N$, $t \ge 0$ and $q \in \mcl Q_1$,

\begin{align*}
\Ll|N\bar F_N(t,{q})- (N+M)\bar F_{N+M}(t,{q})\Rr|\leq CM\Ll(t + |{q}|_{L^1}\Rr).
\end{align*}

Let us first assume that ${q} \in \C_\infty$.
As before, we write $\rho =(\sigma,\tau)$, and carry out an interpolation computation.
Let $\mathbf{s}:=(\rho,\alpha)$, and $\mathfrak{P} := P_{N+M}\otimes \fR$.
We set

\begin{align*}
\mathbf{G}^0(\mathbf{s}) + \mathbf{D}^0(\mathbf{s}) &= H^{t,{q}}_{N+M}(\rho,\alpha) ,\quad
\mathbf{G}^1(\mathbf{s}) + \mathbf{D}^1(\mathbf{s}) =  H^{t,{q}}_{N}(\sigma,\alpha)
\end{align*}

where $\mathbf{G}^i(\mathbf{s})$ collects centered Gaussian terms and $\mathbf{D}^i(\mathbf{s})$ collects deterministic terms.
For every $r\in[0,1]$, we set

\begin{gather*}
\varphi(r) := \E \log\int \exp \Ll(\sqrt{1-r}\mathbf{G}^{0}(\mathbf{s})+(1-r)\mathbf{D}^{0}(\mathbf{s})+\sqrt{r}\mathbf{G}^{1}(\mathbf{s})+r\mathbf{D}^{1}(\mathbf{s})\Rr)\d\mathfrak{P}(\mathbf{s}).
\end{gather*}

We have $\varphi(0) = (N+M)\bar F_{N+M}(t,{q})$ and $\varphi(1) = N\bar F_N(t,{q})$.
Again, using Gaussian integration by parts, we get

\begin{align*}
\frac{\d}{\d r}\varphi(r) = \E \la \mathbf{V}(\mathbf{s},\mathbf{s})-\mathbf{V}(\mathbf{s},\mathbf{s}')+ \mathbf{D}^{1}(\mathbf{s}) -\mathbf{D}^{0}(\mathbf{s})\ra_r
\end{align*}

for $\mathbf{V}$ given in (<ref>), and where $\la\cdot\ra_r$ is the corresponding Gibbs measure.
This time, we bound $\mathbf{V}$ and $\mathbf{D}^{1} -\mathbf{D}^{0}$ using

\begin{align*}
Nt\xi\Ll(\frac{\sigma\sigma'^\intercal}{N}\Rr) & = (N+M)t\xi\Ll(\frac{\rho\rho'^\intercal}{N+M}\Rr) + O(M)t,
\\
{q}\Ll(\alpha\wedge\alpha'\Rr)\cdot \sigma\sigma'^\intercal &= {q}\Ll(\alpha\wedge\alpha'\Rr)\cdot \rho\rho'^\intercal + O\Ll(M\Rr)\Ll|{q}\Ll(\alpha\wedge\alpha'\Rr)\Rr|
\end{align*}

uniformly in $N$. We get

\begin{align*}
\Ll|\frac{\d}{\d r}\varphi(r) \Rr|\leq O(M)\Ll(1 + \E \la \Ll|{q}\Ll(\alpha\wedge\alpha'\Rr)\Rr|\ra_r\Rr) = O(M)\Ll(t+|q|_{L^1}\Rr)
\end{align*}

and the announced result follows for ${q}\in \C_\infty$. The full result follows by approximation using Proposition <ref>.

§.§ Overlap approximations

Recall the notation $\approx_\eps$ in (<ref>).

§.§.§ Overlap approximations for the free-energy cavity calculation

We show that $A_N(x)$ in (<ref>) and $-M\mathscr{P}_{t,q}({p})$ in (<ref>) can be approximated by continuous functions of finitely many entries in the overlap arrays. Recall $R^{\leq n}_N$ in (<ref>), $Q^{\leq n}_{p}$ in (<ref>),  $L^\infty_{\leq 1}$ in (<ref>), and that $\la\cdot\ra_\fR$ denotes the expectation with respect to the measure $\fR^{\otimes \N}$.

For every $\eps>0$, there exists $n\in\N$ and a bounded continuous function $F_\eps : (\R^{D\times D}\times \R^{D\times D})^{n\times n}\to \R $ such that the following two properties hold:

* $A_N(x) \approx_\eps \E \la F_\eps\Ll(R^{\leq n}_{N},q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr) \ra_{N,x}$ uniformly over $N \in \N$ and $x \in [0,3]^{\N^4 \times \frac{D(D+1)}{2}}$;

* $-M\mathscr{P}_{t,{q}}({p})\approx_\eps \E \la F_\eps \Ll(Q^{\leq n}_p,q\Ll(R^{\leq n}_\alpha\Rr)\Rr) \ra_\fR$ uniformly over ${p} \in \C\cap L^\infty_{\leq 1}$.

In the above $q$ acts entrywise (as $p$ does in the definition of $Q_p$).

We show that $A_N(x)$ and $-M\mathscr{P}_{t,{q}}({p})$ have similar overlap structures as defined in Definition <ref>, which will allow us to apply Proposition <ref>. Let $\Gamma$ be a probability measure on some separable Hilbert space $\mathcal{X}$, and let $\mathcal{R} =(\mathcal{R}_1,\mathcal{R}_2) :\mathcal{X}\times \mathcal{X}\to \R^{D\times D}\times \R$ be a measurable function satisfying $|\mathcal{R}_1|,|\mathcal{R}_2|\leq 1$.

Assume that there are independent centered Gaussian processes indexed by $\rho\in \supp\Gamma$:

* $Z(\rho) = (Z_1(\rho),\dots, Z_M(\rho))$ is an $M$-tuple of independent $\R^\D$-valued centered Gaussian vectors $z_i(\rho)$ with covariance $\E Z_i(\rho)Z_i(\rho')^\intercal = \nabla \xi\Ll(\mathcal{R}_1(\rho,\rho')\Rr)$;

* $W(\rho)=(w_1(\rho),\dots,w_M(\rho))$ is an $M$-tuple of independent $\R^\D$-valued centered Gaussian vectors $w_i(\rho)$ with covariance $\E w_i(\rho)w_i(\rho')^\intercal = {q} \Ll(\mathcal{R}_2(\rho,\rho')\Rr)$;

* $Y(\rho)$ is real-valued with covariance $\E Y(\rho)Y(\rho') = M\theta (\mathcal{R}_1(\rho,\rho'))$.

We also assume a stochastic continuity property of these processes so that we can construct jointly measurable versions of these as in Proposition <ref>.
Writing $\la\cdot\ra_\Gamma$ to denote the expectation with respect to $\Gamma ^{\otimes \N}$, we consider

\begin{align*}
\mathcal{F}_1(\Gamma,\mathcal{R}) & :=\E \log \bigg\langle\int \exp\Big(\sqrt{2t} Z(\rho)\cdot \tau - t \nabla\xi\Ll(\mathcal{R}_1(\rho,\rho)\Rr)\cdot \tau\tau^\intercal
\\
&\qquad \qquad+ \sqrt{2}W(\rho)\cdot \tau - {q}(\mathcal{R}_2(\rho,\rho))\cdot \tau\tau^\intercal\Big)\d P_M(\tau)\bigg\rangle_\Gamma,
\\
\mathcal{F}_2(\Gamma,\mathcal{R}) &:= \E \log \la \exp\Ll(\sqrt{2t}Y(\rho)- tM\theta \Ll( \mathcal{R}_1(\rho,\rho)\Rr)\Rr)\ra_\Gamma,
\end{align*}

where $\E$ integrates the Gaussian randomness in $Z(\rho)$, $W(\rho)$ and $Y(\rho)$.
Comparing with the expression for $A_N(x)$ in (<ref>), we see that $A_N(x)$ is the average of $\mathcal{F}_1(\Gamma,\mathcal{R})-\mathcal{F}_2(\Gamma,\mathcal{R})$ with $(\frac{\sigma}{\sqrt{N}},\alpha)$, $R_N$, and $\la\cdot\ra_{N,x}$ substituted for $\rho$, $\mathcal{R}$, and $\la\cdot\ra_\Gamma$ (this further averaging is with respect to the randomness intrinsic to $\la \cdot \ra_{N,x}$ itself).
Using Lemma <ref>, we see that $-M\mathscr{P}_{t,{q}}({p})$ is the average of $\mathcal{F}_1(\Gamma,\mathcal{R})-\mathcal{F}_2(\Gamma,\mathcal{R})$ with $\alpha$, $Q_{p}$, and $\fR$ substituted for $\rho$, $\mathcal{R}$, and $\Gamma $.

We will apply Proposition <ref> to $\mathcal{F}_1(\Gamma,\mathcal{R})$ and $\mathcal{F}_2(\Gamma,\mathcal{R})$. For $\mathcal{F}_1(\Gamma,\mathcal{R})$, the corresponding function $C$ is $C_1:\mathcal{R}\mapsto 2t\nabla\xi(\mathcal{R}_1)+2q(\mathcal{R}_2)$. One can also identify the corresponding Lipschitz function $\mathbf{g}$ as an integral in $\tau$. Note that there are $M$ independent copies of the $\R^\D$-valued processes here. Proposition <ref> is still applicable and we can approximate

\begin{align*}
\mathcal{F}_1(\Gamma,\mathcal{R}) \approx_{\frac{\eps}{2}} \E \la F_{1,\eps}(C_1(\mathcal{R}^{\leq n_1}))\ra_\Gamma
\end{align*}

for some $n_1$ and some continuous $F_{1,\eps}$, where $C_1$ acts entrywise.

For $\mathcal{F}_2(\Gamma,\mathcal{R})$, the corresponding function $C$ is $C_2:\mathcal{R}\mapsto M\theta(\mathcal{R}_1)$ and $\mathbf{g}$ is linear. Proposition <ref> is directly applicable and we obtain the approximation

\begin{align*}
\mathcal{F}_2(\Gamma,\mathcal{R}) \approx_{\frac{\eps}{2}} \E \la F_{2,\eps}(C_2(\mathcal{R}^{\leq n_2}))\ra_\Gamma
\end{align*}

for some $n_2$ and some continuous $F_{2,\eps}$, where $C_2$ acts entrywise.

Since both $\nabla\xi$ and $\theta$ are continuous, we can absorb them into continuous functions to get

\begin{align*}
\mathcal{F}_1(\Gamma,\mathcal{R}) -\mathcal{F}_1(\Gamma,\mathcal{R})\approx_\eps \E \la F_\eps(\mathcal{R}^{\leq n}_1,q(\mathcal{R}^{\leq n}_2))\ra_\Gamma.
\end{align*}

We emphasize that $q$ cannot be absorbed because it is in general not continuous.
For simplicity, we also add the variable $\mathcal{R}_2$ (which forms $\mathcal{R}$ together with $\mathcal{R}_1$) and express the approximating function as $F_\eps(\mathcal{R}^{\leq n},q(\mathcal{R}^{\leq n}_2))$.
In view of (<ref>) and (<ref>), for $\mathcal{R}$ being $R_N$ or $Q_p$, we have $\mathcal{R}_2 = R_\alpha$ in both cases. Hence, we arrive at the approximations of the announced form.

§.§.§ Overlap approximations for the Gibbs-average cavity calculation

In the same fashion, we want to approximate $\E \la g(\tau\tau'^\intercal, \alpha\wedge\alpha')\ra_{N+M,x}^\star$ and $\E \la g(\tau\tau'^\intercal, \alpha\wedge\alpha')\ra_{\fR,{q}+t\nabla\xi({p})}$, where the two Gibbs measures are defined in (<ref>) and (<ref>), respectively. Also, recall the definition of $\la\cdot\ra_{N,x}$ in (<ref>), and that $\la \cdot\ra_\fR$ denotes the expectation with respect to $\fR^{\otimes \N}$.

For every bounded continuous function $g:\R^{D\times D}\times \R\to\R$ and every $\eps>0$, there is $n\in\N$ and a bounded continuous function $F_\eps : (\R^{D\times D}\times \R^{D\times D}\times \R)^{n\times n}\to \R $ such that the following two properties hold:

* $\E \la g(\tau\tau'^\intercal, \alpha\wedge\alpha')\ra_{N+M,x}^\star \approx_\eps \E \la F_\eps\Ll(R^{\leq n}_{N},q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr) \ra_{N,x}$ uniformly over $N,x$;

* $\E \la g(\tau\tau'^\intercal, \alpha\wedge\alpha')\ra_{\fR,{q}+t\nabla\xi({p})}\approx_\eps \E \la F_\eps \Ll(Q^{\leq n}_p,q\Ll(R^{\leq n}_\alpha\Rr)\Rr) \ra_\fR$ uniformly over ${p} \in \C\cap L^\infty_{\leq 1}$.

Again, $q$ acts entrywise.

Comparing the definition of $\la\cdot\ra^\star_{N+M,x}$ and that of $\la\cdot\ra_{N,x}$, we can rewrite

\begin{align}
&\E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{N+M,x}^\star \notag
\\
&= \E \frac{\la \iint g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr) e^{U(\sigma,\alpha,\tau)+U(\sigma',\alpha',\tau')} \d P_1(\tau) \d P_1(\tau')\ra_{N,x}}{\la \iint e^{U(\sigma,\alpha,\tau)+U(\sigma',\alpha',\tau')} \d P_1(\tau)\d P_1(\tau')\ra_{N,x} } \label{e.<>/<>_1}
\end{align}

for $U$ given in (<ref>).
Similarly, we can rewrite

\begin{align}%\label{e.<nu,nablaF>=_(4)}
&\E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{\fR,{q}+t\nabla\xi({p})}\notag
\\
&= \E \frac{\la \iint g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)e^{V(\alpha,\tau)+V(\alpha',\tau')}\d P_1(\tau) \d P_1(\tau')\ra_\fR}{\la \iint e^{V(\alpha,\tau)+V(\alpha',\tau')}\d P_1(\tau) \d P_2(\tau')\ra_\fR}   \label{e.<>/<>_2}
\end{align}

$V(\alpha,\tau)$ is in Lemma <ref> and satisfies $V(\alpha,\tau) {=} \sqrt{2t} Z(\alpha)\cdot \tau - t \nabla\xi\Ll({p}(1)\Rr)\cdot \tau\tau^\intercal + \sqrt{2}W^{q}_M(\alpha)\cdot \tau - {q}(1)\cdot \tau\tau^\intercal$.

In the formalism of Definition <ref>, we can see that (<ref>) and (<ref>) are expressed in the same overlap and covariance structure. Here, $\Gamma$ corresponds to $\la\cdot\ra_{N,x}$ and $\fR$; $\mathcal{R}=(\mathcal{R}_1,\mathcal{R}_2)$ (which is $\R^{\D\times\D}\times\R$-valued) corresponds to $R_N$ and $Q_p$; and $C$ is $\mathcal{R}\mapsto 2t\nabla\xi(\mathcal{R}_1)+2q(\mathcal{R}_2)$.

To apply Proposition <ref>, let us first assume that $g$ does not depend on $\tau\tau'^\intercal$.  Then, we can identify the function $\mathbf{g}$ as an integral in $\tau$ and set $\mathbf{h}=g$ so that both (<ref>) and (<ref>) can be rewritten as the left-hand side of (<ref>). Applying Proposition <ref>, we can approximate (<ref>) and (<ref>) by

\begin{align*}
\E \la g\Ll(\mathcal{R}^{1,2}_2\Rr) F_\eps \Ll(C(\mathcal{R}^{\leq n})\Rr) \ra_\Gamma
\end{align*}

for some $n\in\N$ and continuous $F_\eps$, where we used the observation that $\mathcal{R}_2=R_\alpha$ for $\mathcal{R}$ being either $R_N$ or $Q_p$ (see (<ref>) and (<ref>)). Since the functions $g$ and $\nabla\xi$ in the definition of $C$ are continuous, we can absorb them into $F_\eps$ so that the above approximating term becomes

\begin{align*}
\E \la F_\eps \Ll(\mathcal{R}^{\leq n},q\Ll(\mathcal{R}_2^{\leq n}\Rr)\Rr) \ra_\Gamma,
\end{align*}

as announced. Since $q$ is not continuous in general, it cannot be absorbed.

If $g$ depends on $\tau\tau'^\intercal$, then Proposition <ref> is not directly applicable. Nevertheless, we can easily modify the approximation argument in its proof to get the same result.

§.§ Ghirlanda–Guerra identities

Recall the notation of overlaps in (<ref>).
We denote by $\E_x$ the expectation under which $x$ is an i.i.d. sequence of uniform random variables over $[1,2]$.
For $N\in\N$, integer $n\geq 2$, $h\in\N^4$, and a bounded measurable function $\bff:\Ll(\R^{\D\times \D} \times \R\Rr)^{n\times n}\to \R$, we define, for $\la\cdot\ra = \la \cdot \ra_{N,x}$ in (<ref>),

\begin{align}\label{e.Delta^x}
\Delta^x_N(\bff,n,h) = \Ll|\E \la \bff\Ll(R^{\leq n}_N\Rr)R^{1,n+1}_{N,h} \ra - \frac{1}{n}\E\la \bff\Ll(R^{\leq n}_N\Rr)\ra \E \la  R^{1,2}_{N,h}\ra - \frac{1}{n}\sum_{l=2}^n \E \la \bff\Ll(R^{\leq n}_N\Rr) R^{1,l}_{N,h}\ra\Rr|.
\end{align}

In (<ref>) and throughout, $\E$ integrates the Gaussian randomness in the Hamiltonian and the randomness in $\fR$. In particular, it does not integrate $x$.

The following holds:

*  $\lim_{N\to\infty} \E_x \E \la \Ll|R^{1,1}_{N,\sigma} - \E \la R^{1,1}_{N,\sigma} \ra_{N,x}\Rr|\ra_{N,x} =0$;

*  $\lim_{N\to\infty} \E_x \Delta^x_N(\bff,n,h)=0$ for every $n$, $h$, and $\bff$.

We start with a lemma which leads to part (<ref>) of the proposition. We recall the notation for the coordinates of $x$ in (<ref>).

There exists a constant $C < +\infty$ not depending on $x$ such that, for every $i\in \{1,\dots,D(D+1)/2\}$,

\begin{align*}
\int_1^2\E \la\Ll|e_i\cdot \sigma\sigma^\intercal - \E \la e_i\cdot \sigma\sigma^\intercal\ra\Rr| \ra_{N,x} \d x_i \leq CN^{\frac{25}{32}}.
\end{align*}

We write $\la\cdot\ra=\la \cdot\ra_{N,x}$.
Fixing any $x$ except for the entry $x_i$,
we set

\begin{align*}
\varphi(x_i) := \log \iint \exp\Ll(\tilde H^{t,{q}}_N(\sigma,\alpha) +  N^{-\frac{1}{16}}H^{x}_N(\sigma,\alpha)\Rr) \d P_N(\sigma) \d \fR(\alpha)
\end{align*}

and $\phi(x_i) :=\E \varphi(x_i)$.
We compute the derivatives:

\begin{gather*}
\varphi'(x_i)  = N^{-\frac{1}{16}} \la e_i\cdot \sigma\sigma^\intercal\ra, \qquad \varphi''(x_i) = N^{-\frac{1}{8}}  \la (e_i\cdot \sigma\sigma^\intercal)^2 - \la e_i\cdot \sigma\sigma^\intercal\ra^2 \ra,
\\
\phi'(x_i)  = N^{-\frac{1}{16}} \E \la e_i\cdot \sigma\sigma^\intercal\ra,\qquad
\phi''(x_i)  = N^{-\frac{1}{8}} \E \la (e_i\cdot \sigma\sigma^\intercal)^2 - \la e_i\cdot \sigma\sigma^\intercal\ra^2 \ra.
\end{gather*}

Since $|e_i\cdot \sigma\sigma^\intercal|\leq N$, we have, for all $x_i\in[0,3]$,

\begin{align}\label{e.|phi'|<}
|\phi'(x_i)|\leq N^\frac{15}{16}.
\end{align}

Integrating $\phi''(x_i)$ over $x_i\in[1,2]$ yields

\begin{align*}
\int_1^2 \E \la (e_i\cdot \sigma\sigma^\intercal)^2 - \la e_i\cdot \sigma\sigma^\intercal\ra^2 \ra \, \d x_i=N^\frac{1}{8}(\phi'(2)-\phi'(1)) \leq 2N^\frac{17}{16}.
\end{align*}

By a  concentration argument such as <cit.>, we have

\begin{align*}
\sup_{x}\E |\varphi-\E \varphi| \leq CN^\frac{1}{2}
\end{align*}

for some constant $C< +\infty$. Defining

\begin{align*}
\delta := |\varphi(x_i+y_i) -\phi(x_i+y_i)| + |\varphi(x_i-y_i)-\phi(x_i-y_i)| + |\varphi(x_i)-\phi(x_i)|,
\end{align*}

we thus have $\int_1^2\delta \d x_i\leq CN^\frac{1}{2}$ for all $y_i\in[0,1]$. By Hölder's inequality (as in e.g. <cit.>), the functions $\varphi$ and $\phi$ are convex. By <cit.>, we can obtain from basic properties of convex functions that

\begin{align*}
|\varphi'(x_i) - \phi'(x_i)|\leq \phi'(x_i+y_i) - \phi'(x_i-y_i) + \frac{\delta}{y_i}.
\end{align*}

Using (<ref>) and the mean value theorem, we get

\begin{align*}
\int_1^2 \Ll(\phi'(x_i+y_i)-\phi'(x_i-y_i)\Rr)\d x_i = \phi(2+y_i) -\phi(2-y_i) - \phi(1+y_i) +\phi(1-y_i) \leq 4 N^\frac{15}{16} y_i.
\end{align*}

\begin{align*}
\int_1^2 \E|\varphi'(x_i) - \phi'(x_i)| \d x_i \leq 4 N^\frac{15}{16} y_i + \frac{CN^\frac{1}{2}}{y_i}.
\end{align*}

Inserting the expressions of $\varphi'$ and $\phi'$ and setting $y_i= N^{-\frac{7}{32}}$, we get the desired result.

We write $\la\cdot\ra = \la\cdot\ra_{N,x}$.
The previous lemma implies that

\begin{align*}
\int_1^2 \E \la   \Ll|e_i\cdot\frac{\sigma\sigma^\intercal }{N} - \E \la e_i\cdot \frac{\sigma\sigma^\intercal }{N} \ra \Rr|\ra\d x_i \leq C N^{-\frac{7}{32}}
\end{align*}

for every $i \in \{1,\ldots D(D+1)/2\}$.
Since $\{e_i\}_{i=1}^{D(D+1)/2}$ is an orthonormal basis, we deduce that

\begin{align*}
\lim_{N\to\infty} \int_{\Ll[1,2\Rr]^{D(D+1)/2}} \E \la \Ll|\frac{\sigma\sigma^\intercal }{N} - \E \la \frac{\sigma\sigma^\intercal }{N} \ra \Rr|\ra\d x=0,
\end{align*}

which proves Proposition <ref> (<ref>).

To prove the second part, we proceed as in <cit.> (with $h$, $N^\frac{7}{16}$, $N^\frac{1}{2}$, $N^{-\frac{1}{2}}x_hc_h H^h_N(\sigma,\alpha)$ substituted for $p$, $s$, $\nu_N(s)$, $g_p(\sigma)$ therein) to get that for every $h$, there is a constant $C_h$ such that

\begin{align*}
\int_1^2 \E \la \Ll| H^h_N -\E \la H^h_N\ra\Rr|\ra \d x_h \leq C_h N^\frac{3}{4}.
\end{align*}

Let $\bff$ be bounded and measurable and write $\bff = \bff(R^{\leq n})$. Without loss of generality, we assume $\|\bff\|_\infty\leq 1$. Then

\begin{align*}
\Ll|\E\la \bff H^h_N\ra - \E \la \bff\ra \E \la H^h_N\ra\Rr| \leq  \E \la \Ll|H^h_N - \E \la H^h_N\ra \Rr|\ra.
\end{align*}

For brevity, we omit the subscript $N$ from overlaps.
By the Gaussian integration by parts in e.g. <cit.>, we have

\begin{align*}
&\E\la \bff H^h_N\ra - \E \la \bff\ra \E \la H^h_N\ra
\\
& = x_hc_hN^\frac{15}{16}\Ll(\E \la \bff\Ll(\sum_{\ell=1}^n R^{1,\ell}_h - n R^{1,n+1}_h\Rr)\ra + \E\la \bff\ra \E \la  R^{1,2}_h -R^{1,1}_h\ra\Rr).
\end{align*}

The above three displays yield

\begin{align*}
\E_x\Ll| n\E \la \bff R^{1,n+1}_h \ra - \E \la \bff\ra \E \la R^{1,2}\ra  -  \sum_{\ell=2}^n \E \la \bff R^{1,\ell}_h\ra\Rr|
\leq  \E_x \E \la \Ll|R^{1,1}_h - \E \la R^{1,1}_h\ra  \Rr|\ra +C_h N^{-\frac{3}{16}} .
\end{align*}

Since $R^{1,1}_\alpha =1$ and $R^{1,1}_\sigma$ is bounded, there exists a constant $C'_h < +\infty$ such that $\Ll|R^{1,1}_h - R^{2,2}_h\Rr|\leq C'_h \Ll|R^{1,1}_\sigma - R^{2,2}_\sigma\Rr|$ and thus

\begin{align*}
\E \la \Ll|R^{1,1}_h - \E \la R^{1,1}_h\ra  \Rr|\ra \leq \E \la \Ll| R^{1,1}_h - R^{2,2}_h \Rr|\ra  \leq C'_h \E \la \Ll| R^{1,1}_\sigma - R^{2,2}_\sigma \Rr|\ra
\\
\leq 2C'_h \E \la \Ll| R^{1,1}_\sigma  -  \E \la R^{1,1}_\sigma\ra \Rr|\ra.
\end{align*}

Taking $\E_x$ in this display, by Proposition <ref> (<ref>), we see that the right-hand side vanishes as $N\to\infty$.
Inserting this into the previous display, we get Proposition <ref> (<ref>).

§.§ Identifying the limits

Let $\Delta^x_N(\bff,n,h)$ be as given in (<ref>). We enumerate as $((\bff_j,n_j,h_j))_{j\in\N}$ all the triples $(\bff,n,h)$ where $\bff:\Ll(\R^{\D\times\D}\times\R\Rr)^{n\times n}\to\R$ is a monomial (with coefficient $1$), $n\in\N$, $h\in\N^4$.
Then, we modify each $\bff_j$ in two steps:
first, since $R^{\leq n}$ is bounded, we can adjust $\bff_j$ outside a bounded set to make it bounded;
secondly, we rescale $\bff_j$ to make sure that $\Delta^x_N(\bff_j,n_j,h_j)\leq 1$. For each $N \in \N$ and $x \in [0,3]^{\N^4 \times \frac{D(D+1)}{2}}$, we set

\begin{align}\label{e.Delta_N(x)}
\Delta_N(x) := \E \la \Ll|R^{1,1}_{N,\sigma} -\E \la R^{1,1}_{N,\sigma}\ra_{N,x}\Rr|\ra_{N,x}
+\sum_{j=1}^\infty 2^{-j} \Delta^x_N(\bff_j,n_j,h_j).
\end{align}

\begin{align}\label{e.mathcal_K}
\mbox{We denote by $\mathcal{K}$ the convex hull of the set} \Ll\{\tau\tau^\intercal:\:\tau\in\supp P_1\Rr\}.
\end{align}

The set $\mathcal{K}$ is closed, is such that $\mathcal{K}\subset \S^\D_+$, and we recall from (<ref>) that we have $|a|\leq 1$ for every $a\in\mathcal{K}$.

Recall the definitions of $R_N$ in (<ref>), $A_N(x)$ in (<ref>), $\sP_{t,{q}}$ in (<ref>), $\la \cdot\ra_{N,k}$ in (<ref>), $\la \cdot\ra_{N,x}^\circ$ in (<ref>), and $\la\cdot\ra_{\fR,\pi}$ in (<ref>). Whenever we speak of the convergence in law of an overlap array, we always mean this in the sense of finite-dimensional distributions.

We fix $M\in\N$ and $(t,q)\in\R_+\times \C_\infty$, and suppose that there is a sequence $(N_k,x_k)_{k\in\N}$ such that $\lim_{k\to\infty} N_k=+\infty$ and $\lim_{k\to\infty}\Delta_{N_k}(x_k)=0$.
Then, there are a subsequence $(N'_k,x'_k)_{k\in\N}$, ${p}\in \C\cap L^\infty_{\leq 1}$, and $a\in\mathcal{K}$ satisfying $a\geq{p}$ such that

*  $R_{N'_k}$ under $\E\la\cdot\ra_{N'_k,x'_k}$ converges in law to

\begin{align*}
\Ll(Q^{\ell,\ell'}_{p}\1_{\ell\neq \ell'}+ (a,1) \1_{\ell =\ell'}\Rr)_{\ell,\ell'\in\N}
%\Ll({p}\Ll(R^{l,l'}_\alpha\Rr)\1_{l\neq l'}+ a \1_{l =l'},\ R^{l,l'}_\alpha\Rr)_{l,l'\in\N}
\end{align*}

under $\E \la\cdot\ra_\fR$ as $k$ tends to infinity;

*  we have $\lim_{k\to\infty}A_{N'_k}\Ll(x'_k\Rr) =- M \mathscr{P}_{t,{q}}({p})$;

for every bounded continuous $g:\R^{\D\times\D}\times \R\to\R$,

\begin{equation*}  %\label{e.}
\lim_{k\to\infty} \E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{N'_k+M,x'_k}^\circ =\E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{\fR,{q}+t\nabla\xi({p})}.
\end{equation*}

In the statement of Proposition <ref>, when we say that $(N_k', x_k')_{k \in \N}$ is a subsequence, we mean that $\lim_{k \to \infty} N_k' = +\infty$ and that $\{(N_k', x_k') \mid k \in \N\} \subset \{(N_k, x_k) \mid k \in \N\}$.

We define the quantile function $\kappa:[0,1)\to \R$ of a real-valued random variable $X$ as the right-continuous increasing function satisfying

\begin{align*}
\E g(X) = \int_0^1 g(\kappa(s))\d s
\end{align*}

for every bounded measurable function $g:\R\to\R$. We can obtain $\kappa$ by taking the right-continuous inverse of the probability distribution function of $X$.
Consistently with (<ref>), we also set $\kappa(1) := \lim_{s\nearrow 1}\kappa(s)$ whenever $\kappa$ is bounded.

We denote by $R^\mathrm{diag}_N := \Ll(R^{\ell,\ell}_N\Rr)_{\ell\in\N}$ the diagonal part consisting of self-overlaps, and by $R^\mathrm{off}_N := \Ll(R^{\ell,\ell'}_N\Rr)_{\ell\neq \ell'}$ the off-diagonal part. We will use similar notation for the diagonal and off-diagonal parts of other overlap arrays.

Part (<ref>).
Since $R^{1,1}_{N,\sigma} \in \S^\D_+$ and $\Ll|R^{1,1}_{N,\sigma}\Rr|\leq 1$, we can extract a subsequence $\Ll(N'_k,\,x'_k\Rr)_{k\in\N}$ from $\Ll(N_k,\,x_k\Rr)_{k\in\N}$ such that

\begin{align}\label{e.E<R11>=q}
\lim_{k\to\infty} \E \la R^{1,1}_{N'_k,\sigma} \ra_{x'_k} = z
\end{align}

for some $z\in\S^\D_+$.
Since each $ R^{\ell,\ell'}_N$ is bounded, by passing to a further subsequence, we can assume that

\begin{align}\label{e.RtoR}
\mbox{$R_{N'_k}$ under $\E \la\cdot\ra_{N'_k,x'_k}$ converges in law to $R_\infty$ as $k$ tends to infinity,}
\end{align}

for some random array $R_\infty$ (and in the sense of finite-dimensional distributions).
We will show that the law of $R_\infty$ has a natural representation under $\E\la\cdot\ra_{\fR}$.
We write $R^{\ell,\ell'}_\infty := \Ll(R^{\ell,\ell'}_{\infty,\sigma}, R^{\ell,\ell'}_{\infty,\alpha}\Rr)$ where $R^{\ell,\ell'}_{\infty,\bullet}$ is the limit of $R^{\ell,\ell'}_{N,\bullet}$ with $\bullet$ being $\sigma$ or $\alpha$. For $R^{\ell,\ell'}_\infty$, the subscripts $\sigma$ and $\alpha$ are purely symbolic.

First, we determine $R^\mathrm{diag}_\infty$.
Due to the first term on the right of (<ref>), the assumption $\lim_{k\to\infty}\Delta_{N_k}\Ll(x^{N_k}\Rr)=0$ ensures the concentration of $R^{\ell,\ell}_{N,\sigma}$.
Also, it is clear from $R^{\ell,\ell}_{N,\alpha}=1$ that $R^{\ell,\ell}_{\infty,\alpha}=1$, which together with (<ref>) yields that $R_\infty^{\mathrm{diag}}$ is deterministic and

\begin{align}\label{e.diag_R}
R_{\infty}^{\ell,\ell} =(z,1) \quad \forall \ell\in\N.
\end{align}

Next, we determine $R^\mathrm{off}_\infty$.
The same assumption on $\Delta_{N_k}(x^{N_k})$ also ensures that $R_\infty$ satisfies the Ghirlanda–Guerra identities. Panchenko's synchronization result <cit.> guarantees the existence of a Lipschitz function $\Psi:\R_+\to\S^\D_+\times \R_+$ satisfying $\Psi(s)\geq \Psi(s')$ for all $s\geq s'$ such that

\begin{align}\label{e.R=Phi(tr(R))}
R^{\ell,\ell'}_\infty = \Psi\Ll(\tr\Ll(R^{\ell,\ell'}_\infty\Rr)\Rr) \quad \text{a.s.}\ \forall \ell,\ell'\in\N,
\end{align}

where we used the shorthand $\tr\Ll(R^{\ell,\ell'}_\infty\Rr):=\tr\Ll(R^{\ell,\ell'}_{\infty,\sigma}\Rr)+ R^{\ell,\ell'}_{\infty,\alpha}$.

To proceed, we determine the distribution of $\tr\Ll(R^{\ell,\ell'}_\infty\Rr)$.
We denote the quantile function of $\tr\Ll(R^{1,2}_\infty\Rr)$ by $\kappa$. The assumption $\lim_{k\to\infty}\Delta_{N_k}\Ll(x^{N_k}\Rr)=0$ implies that the Ghirlanda–Guerra identities hold for $\Ll(\tr\Ll(R^{\ell,\ell'}_\infty\Rr)\Rr)_{\ell,\ell'\in\N}$. Using the Dovbysh–Sudakov representation in <cit.> together with the characterization of Poisson–Dirichlet cascades by the overlap distribution in <cit.>, we deduce that $\Ll(\tr\Ll(R^{\ell,\ell'}_\infty\Rr)\Rr)_{\ell\neq \ell'}$ has the same law as the overlap array associated with a Poisson–Dirichlet cascade, and this cascade must be such that the quantile function of one overlap is $\kappa$.
Using this and (<ref>), we set $r:=\tr(z)+1$ so that

\begin{align}\label{e.tr(R)=kappa}
\tr\Ll(R^{\ell,\ell'}_\infty\Rr) = \kappa\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\1_{\ell\neq \ell'} + r\1_{\ell=\ell'},
\end{align}

where $\Ll(\alpha^{\ell}\Rr)_{\ell\in\N}$ is sampled from $\la\cdot\ra_\fR$. To be precise, we have argued that the two sides of (<ref>) have the same law, jointly over $\ell, \ell' \in \N$, where the law of the right-hand side is under $\E \la \cdot \ra_\fR$. Since so far we had only specified the law of $R_\infty$ but not its realization itself, we may as well choose the probability space to be that for $\E \langle \cdot \rangle_\fR$ and realize $\tr\Ll(R^{\ell,\ell'}_\infty\Rr)$ according to (<ref>); the full array $R_\infty$ is then specified according to (<ref>), and we get the representation

\begin{align}\label{e.R^l,l'_first}
R^{\ell,\ell'}_\infty = \Psi\Ll(\kappa\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\1_{\ell\neq \ell'}+r \1_{\ell=\ell'}\Rr),\quad\forall \ell,\ell'\in\N.
\end{align}

In order to simplify (<ref>), we write $\Psi=:(\Psi_\sigma,\Psi_\alpha)$ so that $\Psi_\sigma$ is $\S^\D_+$-valued and $\Psi_\alpha$ is $\R_+$-valued.

By the invariance of Poisson–Dirichlet cascades from Proposition <ref>, we have that $R^{\ell,\ell'}_{N,\alpha}$ under $\E\la\cdot\ra_{N,x}$ has the same law as $\alpha^\ell\wedge \alpha^{\ell'}$ under $\E\la\cdot\ra_\fR$. Therefore, $R^{\ell,\ell'}_{\infty,\alpha}$ must also have the same law, which means that $\Psi_\alpha(\kappa(\alpha^\ell\wedge\alpha^{\ell'}))$ is uniformly distributed over $[0,1]$ under $\E\la\cdot\ra_\fR$ for $\ell\neq \ell'$. Since $\alpha^\ell\wedge\alpha^{\ell'}$ is itself uniform over $[0,1]$ and $\Psi\circ\kappa$ is increasing and right-continuous, we must have

\begin{align*}%\label{e.Psi_alpha(kappa)=s}
\Psi_\alpha(\kappa(s))= s,\quad\forall s\in[0,1].
\end{align*}

Thus, we can simplify (<ref>) into

\begin{align}\label{e.R^l,l'}
R^{\ell,\ell'}_\infty = \bigg(\Psi_\sigma\Ll(\kappa\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\1_{\ell\neq \ell'}+r \1_{\ell=\ell'}\Rr),\quad \alpha^\ell\wedge\alpha^{\ell'}\bigg),\quad\forall \ell,\ell'\in\N.
\end{align}

Before concluding, we show that

\begin{align}\label{e.zeta(s)<r}
\kappa(s)\leq r,\quad\forall s\in[0,1].
\end{align}

Note that (<ref>) implies that $R^{\ell,\ell'}_{\infty,\sigma} \in\S^\D_+$ for $\ell\neq \ell'$. The Cauchy–Schwarz inequality implies that $v\cdot R^{\ell,\ell'}_{N,\sigma} v\leq \frac{1}{2} v\cdot \Ll(R^{\ell,\ell}_{N,\sigma}+R^{\ell',\ell'}_{N,\sigma}\Rr)v$ for every $v\in\R^\D$, which yields

\begin{align}\label{e.R<(R+R)/2}
R^{\ell,\ell'}_{\infty,\sigma} \leq \frac{1}{2}\Ll(R^{\ell,\ell}_{\infty,\sigma}+R^{\ell',\ell'}_{\infty,\sigma}\Rr).
\end{align}

Taking the trace on both sides of (<ref>) and using (<ref>), we get

\begin{align*}
\kappa\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr) \leq r.
\end{align*}

Since $\alpha^\ell\wedge\alpha^{\ell'}$ is uniformly distributed on $[0,1]$ for $\ell\neq \ell'$ and $\kappa$ is right-continuous, we arrive at (<ref>).

We set ${p}:= \Psi_\sigma\circ \kappa$ and $a := \Psi_\sigma(r)$.
Since $a$ is the subsequential limit of $\frac{\sigma\sigma^\intercal}{N}$, we have that $a$ belongs to the set $\mcl K$ defined in (<ref>), and in particular $a\in\S^\D_+$ and $|a|\leq 1$. Since the function $\Psi_\sigma$ is increasing, the inequality (<ref>) implies that ${p}\leq a$. Since $\kappa$, as a quantile function, is also increasing, we have ${p} \in \C$.
Notice that, for $y,y'\in \S^\D_+$ satisfying $y\geq y'$, we have $(y+y')\cdot (y-y')\geq 0$ and thus $|y|^2\geq |y'|^2$. This along with ${p}\leq a$ gives $|{p}|_{L^\infty}\leq |a|\leq 1$.
We have thus verified the conditions on ${p}$ and $a$ stated in the proposition.
Part (<ref>) follows from (<ref>) and the representation in (<ref>).

Part (<ref>).
Due to the possibility that $r>\kappa(1)$, in general, a representation of $R_\infty$ as a Poisson–Dirichlet cascade might be off on the diagonal.
To circumvent this, we build a sequence of Poisson–Dirichlet cascades whose overlap arrays approximate $R_\infty$ arbitrarily closely.
Allowed by (<ref>), we can
choose a sequence $(\kappa_m)_{m\in\N}$ of right-continuous increasing
step functions from $[0,1)$ to $[0,r]$ that converges to $\kappa$ in $L^1([0,1);\R)$ as $m$ tends to infinity. For each $m \in \N$, we can also impose that $\lim_{s \nearrow 1}\kappa_m(s) = r$ while preserving the properties mentioned in the previous sentence. As usual, we set $\kappa_m(1) := \lim_{s\nearrow1}\kappa_m(s)$, and thus have $\kappa_m(1)=r$.
Setting ${p}_m:=\Psi_\sigma\circ\kappa_m$, we have that ${p}_m\in \C_\infty$ and

\begin{align}\label{e.zeta_m}
{p}_m(1) = \Psi_\sigma(r),\qquad \lim_{m\to\infty}\Ll|{p}_m - {p}\Rr|_{L^1} =0.
\end{align}

For each $m$, we consider $Q_{{p}_m}$ defined in (<ref>). Notice that, from our choice of $p$ and (<ref>), we have $\Ll(Q^{\ell,\ell'}_{p}\Rr)_{\ell\neq \ell'}=\Ll(R^{\ell,\ell'}_\infty\Rr)_{\ell\neq \ell'}$.

By the second property in (<ref>) and the continuity of Poisson–Dirichlet cascades in the overlap distribution (<cit.> or <cit.>), we deduce that $\Ll(Q^{\ell,\ell'}_{{p}_m}\Rr)_{\ell\neq \ell'}$ converges in law to $\Ll(Q^{\ell,\ell'}_{p}\Rr)_{\ell\neq \ell'}=\Ll(R^{\ell,\ell'}_\infty\Rr)_{\ell\neq \ell'}$ under $\E\la\cdot\ra_\fR$. Due to the first property in (<ref>), we have $Q^{\ell,\ell}_{{p}_m} =R^{\ell,\ell}_\infty$, which is deterministic. We thus obtain that, under $\E\la\cdot\ra_\fR$,

\begin{align}\label{e.QtoR}
\mbox{$Q_{{p}_m}$ converges in law to $R_\infty$ as $m$ tends to infinity. }
\end{align}

In order to proceed, we would like to improve upon the convergences in (<ref>) and (<ref>) into

\begin{multline}
\label{e.joint_cvg_in_law_1}
\mbox{$\big(R_{N'_k}, q(R_\al)\big)$ under $\E \la\cdot\ra_{N'_k,x'_k}$ converges in law }
\\ \mbox{to $\big(R_\infty, q(R_\alpha)\big)$ under $\E \la \cdot \ra_\fR$ as $k$ tends to infinity,}
\end{multline}

and, under $\E \la \cdot \ra_\fR$,

\begin{equation}
\label{e.joint_cvg_in_law_2}
\mbox{$\big(Q_{p_m},q(R_\alpha)\big)$ converges in law to $\big(R_\infty,q(R_\alpha)\big)$ as $m$ tends to infinity.}
\end{equation}

These convergences can be obtained by appealing to the continuous mapping theorem as in the proof of Proposition <ref>: in short, the off-diagonal elements of $R_\alpha$ are uniformly distributed over $[0,1]$, so they have probability zero to belong to the set of points of discontinuity of $q$.

We now fix any $\eps>0$ and recall the notation $\approx_\eps$ from (<ref>).

For the function $F_\eps$ given by Lemma <ref> and using the convergences in law in (<ref>) and  (<ref>), we have for sufficiently large $k$ and $m$ that

\begin{align*}
\stackrel{\ref{l.approx_finite_overlap}}{\approx_\eps}  \E \la F_\eps\Ll(R^{\leq n}_{N'_k},q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr) \ra_{N'_k,x'_k}
\stackrel{\eqref{e.joint_cvg_in_law_1}}{\approx_\eps} \E \la F_\eps\Ll(R^{\leq n}_{\infty}, q\Ll(R^{\leq n}_{\sigma}\Rr)\Rr)\ra_\fR
\\
\stackrel{\eqref{e.joint_cvg_in_law_2}}{\approx_\eps} \E \la F_\eps\Ll(Q^{\leq n}_{{p}_m},q\Ll(R^{\leq n}_\alpha\Rr)\Rr)\ra_\fR
\stackrel{\ref{l.approx_finite_overlap}}{\approx_\eps} -M\mathscr{P}_{t,{q}}({p}_m).
\end{align*}

The symbol $\approx_\ep$ does not define a transitive relation, and cumulatively we obtain an error of $4\ep$ between the first and the last term in the string of approximate equalities.
Recall the expression of $\mathscr{P}_{t,{q}}$ in (<ref>).
These along with (<ref>) and the Lipschitzness of $\psi$ from Corollary <ref> imply $\lim_{m\to\infty}\mathscr{P}_{t,{q}}({p}_m) = \mathscr{P}_{t,{q}}({p})$.
Hence, sending $k$ and $m$ to infinity, we get

\begin{align*}
\liminf_{k\to\infty}  A_{N'_k}\Ll(x'_k\Rr) \approx_{4\eps} - M\mathscr{P}_{t,{q}}({p}) \approx_{4\eps} \limsup_{k\to\infty}  A_{N'_k}\Ll(x'_k\Rr).
\end{align*}

Sending $\eps$ to zero, we obtain part (<ref>).

Part (<ref>).
Recall $\la\cdot\ra^\star_{N+M,x}$ in (<ref>).
For every $k\in\N$ and ${p}' \in \C_\infty$, we write

\begin{align*}
r(k) := \E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{N'_k+M,x'_k}^\star,\qquad \quad \mathfrak{r}\Ll({p}'\Rr) :=\E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{\fR,{q}+t\nabla\xi({p}')}.
\end{align*}

For every $\eps > 0$, we denote by $F_\ep$ the function given by Lemma <ref> and we use (<ref>) and (<ref>) to get that, for every sufficiently large $k$ and $m$,

\begin{align*}
\stackrel{\ref{l.approx_finite_overlap_2}}{\approx_\eps} \E \la F_\eps \Ll(R^{\leq n}_{N'_k},q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr)\ra_{N'_k,x'_k}
\stackrel{\eqref{e.joint_cvg_in_law_1}}{\approx_\eps} \E \la F_\eps\Ll(R^{\leq n}_\infty,q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr)\ra_\fR
\\
\stackrel{\eqref{e.joint_cvg_in_law_2}}{\approx_\eps} \E \la F_\eps\Ll(Q^{\leq n}_{{p}_m},q\Ll(R^{\leq n}_{\alpha}\Rr)\Rr)\ra_\fR
\stackrel{\ref{l.approx_finite_overlap_2}}{\approx_\eps} \mathfrak{r}\Ll({p}_m\Rr).
\end{align*}

Using the second property in (<ref>) and a computation similar to the one in (<ref>), we have $\lim_{m\to\infty} \mathfrak{r}\Ll({p}_m\Rr)=\mathfrak{r}({p})$. Therefore, sending $m$ and $k$ to infinity and then $\ep$ to zero, we obtain that $\lim_{k\to\infty} r(k) = \mathfrak{r}({p})$. Lemma <ref> allows us to replace $\la\cdot\ra^\star_{N'_k+M,x'_k}$ in $r(k)$ by $\la\cdot\ra^\circ_{N'_k+M,x'_k}$ in the limit. Hence, part (<ref>) follows.

§.§ Corollaries

We now combine Propositions <ref> and <ref>.

Recall the definition of $\la\cdot\ra_{N,x}$ in (<ref>), and that $\la\cdot\ra_{\fR}$ denotes the expectation with respect to $\fR^{\otimes \N}$. Recall also $\sP_{t,{q}}$ from (<ref>) and the definitions of the overlaps in (<ref>) and (<ref>).

For every $(t,{q})\in \R_+\times \C_\infty$, there are sequences $(N^\pm_k)_{k\in\N}$ of increasing integers, $(x_k^\pm)_{k\in\N}$, ${p}_\pm \in \C\cap L^\infty_{\leq 1}$, and $a_\pm \in \mathcal{K}$ satisfying $a_\pm\geq p_\pm$ such that

*  $R_{N^\pm_k}$ under $\E\la\cdot\ra_{N^\pm_k,x^\pm_k}$ converges in law to

\begin{align*}
\Ll(Q^{\ell,\ell'}_{{p}_\pm}\1_{\ell\neq \ell'}+ (a_\pm,1) \1_{\ell =\ell'}\Rr)_{\ell,\ell'\in\N}
\end{align*}

under $\E \la\cdot\ra_\fR$ as $k$ tends to infinity;

we have

\begin{equation*}  %\label{e.}
\mathscr{P}_{t,{q}}({p}_-)\leq \liminf_{N\to\infty}\bar F_N(t,{q}) \leq \limsup_{N\to\infty}\bar F_N(t,{q}) \leq \mathscr{P}_{t,{q}}({p}_+).
\end{equation*}

Using Lemma <ref> and Proposition <ref> with $M=1$, we have

\begin{align*}
\liminf_{N\to\infty}\bar F_N(t,{q}) = \liminf_{N\to\infty} \E_x \bar F_N^x(t,{q}) \geq -\limsup_{N\to\infty} \E_x A_N(x).
\end{align*}

Lemmas <ref> and <ref> imply that $\sup_{N,x}|A_N(x)|\leq c$ for some constant $c$.
Proposition <ref> and (<ref>) yield that $\lim_{N\to\infty} \E_x \Delta_N(x) =0$. Using these two facts and following the argument in <cit.>, we can find a sequence $(x_N)_{N\in\N}$ satisfying $\lim_{N\to\infty}\Delta_N(x_N) = 0$ and $A_N(x_N) \geq \E_x A_N(x)+o(1)$ as $N$ tends to infinity.

We remark that <cit.> is originally stated to get an upper bound for $A_N(x_N)$. Examining the proof, one can see that as long as $A_N(x)$ is bounded uniformly, the argument still works for the other side needed here. Hence,

\begin{align*}
\liminf_{N\to\infty}\bar F_N(t,{q}) \geq -\limsup_{N\to\infty} A_N(x_N).
\end{align*}

We extract $(N_k)_{k\in\N}$ along which the $\limsup_{N\to\infty} A_N(x_N)$ is achieved.
By Proposition <ref> at $M=1$, we can find a subsequence $(N^-_k,\, x^-_k)_{k\in\N}$ of $(N_k,\,x_{N_k})_{k\in\N}$ and ${p}_-\in\C\cap L^\infty$ such that part (<ref>) holds and the first inequality in part (<ref>) holds.
The other half follows from a similar argument.

Recall $\la\cdot\ra_{N,x}$ in (<ref>) and $\la\cdot\ra_{\fR,\pi}$ in (<ref>).

For any $M\in\N$, any $(t,{q})\in \R_+\times (\C\cap L^\infty)$, and any sequence of increasing integers, there are a subsequence $(N_k)_{k\in\N}$, $(x_k)_{k\in\N}$, ${p} \in \C\cap L^\infty_{\leq 1}$, and $a\in \mathcal{K}$ satisfying $a\geq p$ such that

$R_{N_k}$ under $\E\la\cdot\ra_{N_k,x_k}$ converges in law to

\begin{align*}
\Ll(Q^{\ell,\ell'}_{p}\1_{\ell\neq \ell'}+ (a,1) \1_{\ell =\ell'}\Rr)_{\ell,\ell'\in\N}
\end{align*}

under $\E \la\cdot\ra_\fR$  as $k$ tends to infinity;

*  for every bounded continuous $g:\R^{\D\times\D}\times \R\to\R$, we have

\begin{equation*}  %\label{e.}
\lim_{k\to\infty} \E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{N_k+M,x_k}^\circ =\E \la g\Ll(\tau\tau'^\intercal, \alpha\wedge\alpha'\Rr)\ra_{\fR,{q}+t\nabla\xi({p})}.
\end{equation*}

Recall $\Delta_N(x)$ in (<ref>).
By Proposition <ref>, we have $\lim_{N\to\infty}\E_x \Delta_N(x)=0$. As discussed in the previous proof, we can find $(x_N)_{N\in\N}$ such that $\lim_{N\to\infty} \Delta_N(x_N)=0$. The desired result then follows from Proposition <ref>.

§ PROOFS OF THE MAIN RESULTS

We now leverage the results of the previous section to complete the proofs of our main results. We first show Theorem <ref> as well as the fact that any subsequential limit of the free energy $\bar F_N$ must solve the equation “almost everywhere”, as was announced in (<ref>). We then deduce Theorem <ref> by a continuity argument. We next identify, for “almost every” choice of $q$, the limit law of the array of the conditional expectations of the $\sigma$-overlaps given the $\alpha$-overlaps. Up to the additional perturbation provided by the parameter $\hat t$ introduced around (<ref>), we can then slightly upgrade this result and thereby obtain a proof of Theorem <ref>.

§.§ Critical point identification

Recall the definition of $\sP_{t,q}$ in (<ref>). In the next proposition, we assume the convergence of the entire sequence $\Ll(\bar F_N\Rr)_{N\in\N}$.

Suppose that $\Ll(\bar F_N\Rr)_{N\in\N}$ converges pointwise to some limit $f$, and let $t \ge 0$ and $q \in \mcl Q_\uparrow \cap L^\infty$. If $ f(t,\cdot)$ is Gateaux differentiable at ${q} \in \C_\uparrow\cap L^\infty$, then $f(t,{q}) = \mathscr{P}_{t,{q}}(\dr_q  f(t,{q}))$.

Let $(N^\pm_k)_{k\in\N}$, $(x^\pm_k)_{k\in\N}$, and ${p}_\pm$ be given by Corollary <ref>, whose part (<ref>) implies

\begin{align}\label{e.P<limF<P}
\mathscr{P}_{t,{q}}({p}_-) \leq f(t,{q})\leq  \mathscr{P}_{t,{q}}({p}_+).
\end{align}

Recall the definitions of $\bar F^x_N$ in (<ref>) and of $\la\cdot\ra_{N,x}$ in (<ref>). We set $\bar F_k^\pm := \bar F^{x^\pm_k}_{N^\pm_k}$.
For every continuous $\kappa\in L^\infty$, by a computation similar to that for (<ref>), we have, for every $k \in \N$,

\begin{align*}
\la \kappa, \dr_q \bar F_k^\pm(t,{q}) \ra_{\cH} = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N^\pm_k}  \ra_{N^\pm_k,x^\pm_k}.
\end{align*}

By Lemma <ref>, the function $\Ll(\bar F_k^\pm\Rr)_{k\in\N}$ also converges pointwise to $f$ as $k$ tends to infinity. Following the proof of Proposition <ref> verbatim, we obtain that $\dr_q \bar F_k^\pm(t,{q})$ converges to $\dr_q  f(t,{q})$ in $L^2$.
Using this and Corollary <ref> (<ref>) yields that

\begin{align*}
\la \kappa, \dr_q  f(t,{q})\ra_{L^2} = \E \la \kappa\Ll(R^{1,2}_\alpha\Rr)\cdot{p}_\pm(R^{1,2}_\alpha)\ra_{\fR}  = \int_0^1 \kappa(s)\cdot{p}_\pm(s)\d s.
\end{align*}

Since the set of continuous $\kappa\in L^\infty$ is dense in $\cH$, we have ${p}_\pm = \dr_q  f(t,q)$, which along with (<ref>) gives the desired result.

In the following result, the limit is subsequential.

Suppose that $\Ll(\bar F_{N}\Rr)_{N\in\N}$ converges pointwise to some limit $f$ along a subsequence $(N_k)_{k\in\N}$, and let $t \ge 0$.
If $f(t,\cdot)$ is Gateaux differentiable at some ${q} \in \C_\uparrow\cap L^\infty$, then

\begin{equation}
\label{e.fixed_pt_nbla_f}
\dr_q f(t,{q}) = \dr_q \psi ({q}+t\nabla\xi(\dr_q f(t,{q}))).
\end{equation}

If, in addition, $t>0$ and $f(\cdot,q)$ is differentiable at $t$, then

\begin{align}\label{e.satify_eqn}
\partial_t f(t,{q}) = \int_0^1\xi(\dr_q f(t,{q})).
\end{align}

By passing to a further subsequence, we may assume that $(N_k)_{k\in\N}$ is the subsequence given in Corollary <ref> (for $M=1$) along with $(x_k)_{k\in\N}$ and ${p} \in \C\cap L^\infty_{\leq 1}$.

Recall $\tilde F_N^x$ in (<ref>) and $\la\cdot\ra_{N,x}$ in (<ref>).
As in the proof of (<ref>), we have for any bounded continuous $\kappa:[0,1]\to\S^\D$ that

\begin{align}\label{e.dtildeF}
\la \kappa, \dr_q \tilde F_{N_k}^{x_k}(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N_k} \ra_{N_k,x_k},\quad \partial_t \tilde F_{N_k}^{x_k}(t,{q}) = \E \la \xi\Ll(\frac{\sigma\sigma'^\intercal}{N_k}\Rr) \ra_{N_k,x_k}.
\end{align}

By Lemma <ref>, we have that $\tilde F_{N_k}^{x_k}$ converges pointwise to $f$.
Following the proof of Proposition <ref>, we deduce that $\dr_q \tilde F_{N_k}^{x_k}(t,{q})$ converges to $\dr_q f(t,q)$ in $L^2$. Using also the convergence of the overlaps from Corollary <ref> (<ref>), we let $k$ tend to infinity in the first relation in (<ref>) to get that

\begin{align*}
\la \kappa, \dr_q f(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr) \cdot {p}\Ll(\alpha\wedge\alpha'\Rr)\ra_\fR = \la \kappa,{p}\ra_\cH.
\end{align*}

Since the set of bounded continuous functions spans $L^2$, we deduce that

\begin{align}\label{e.zeta=nabla_f}
{p}= \dr_q f(t,{q}).
\end{align}

Under the additional assumption on the differentiability in $t$ (and an adaptation  to $\tilde F_{N_k}^{x_k}$ of the part of Proposition <ref> concerning the derivative in $t$), we can pass to the limit in the second relation in (<ref>) and get that

\begin{align*}%\label{e.dt_f=int_xi(d_f)}
\partial_t f(t,{q}) = \E \la \xi\Ll({p}\Ll(\alpha\wedge\alpha'\Rr)\Rr)\ra_\fR = \int_0^1\xi({p}(u))\d u .
\end{align*}

This and (<ref>) yield (<ref>).

To obtain (<ref>), we relate ${p}$ to the derivative of $\psi$. Recall $\bar F_N^x$ in (<ref>) and $\la\cdot\ra^\circ_{N,x}$ in (<ref>).
By Lemmas <ref> and <ref>, we have that $\bar F_{N_k+1}^{x_k}$ converges pointwise to $f$. Following the proof of (<ref>), we have for any bounded continuous $\kappa$ that

\begin{align*}
\la \kappa, \dr_q \bar F_{N_k+1}^{x_k}(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\rho\rho'^\intercal}{N_k+1} \ra_{N_k+1,x_k}^\circ.
\end{align*}

We write $\rho$ as an $(N_k +1)$-tuple of $D$-dimensional coordinates $\rho =\Ll(\rho_1,\dots,\rho_{N_k},\rho_{N_k+1}\Rr)$, which are i.i.d. under $P_{N_k+1} = P_1^{\otimes (N_k+1)}$. Since the Hamiltonian in $\la\cdot\ra_{N_k+1,x_k}^\circ$ only depends on $\rho\rho^\intercal$ and $\rho\rho'^\intercal$, we deduce that $\la\cdot\ra_{N_k+1,x_k}^\circ$ is invariant under permutations of the coordinates of $\rho$. Therefore, writing $\sigma := (\rho_1,\dots\rho_{N_k})$ and $\tau := \rho_{N_k+1}$, we have

\begin{align*}
\la \kappa, \dr_q \bar F_{N_k+1}^{x_k}(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot \tau\tau'^\intercal \ra_{N_k+1,x_k}^\circ.
\end{align*}

By an adaptation of Proposition <ref>, we have that $\dr_q \bar F_{N_k+1}^{x_k}(t,{q})$ converges in $L^2$ to $\dr_q  f(t,q)$. Using also Corollary <ref> (<ref>) (with $M=1$), we can pass to the limit in the previous display and get

\begin{align*}
\la \kappa, \dr_q f(t,{q})\ra_\cH = \E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot \tau\tau'^\intercal \ra_{\fR,q+t\nabla\xi({p})},
\end{align*}

where $\la\cdot\ra_{\fR,q+t\nabla\xi({p})}$ is given in (<ref>) with $M=1$. Notice that since $M = 1$, the Gibbs measure $\la\cdot\ra_{\fR,{q}+t\nabla\xi({p})}$ is the same as the Gibbs measure $\la \cdot\ra_{{q}+t\nabla\xi({p})}$ defined in (<ref>).
By (<ref>), the right-hand side in the previous display is equal to $\la \kappa, \dr_q \psi({q}+t\nabla\xi({p}))\ra_\cH$. Combining this with (<ref>) yields that

\begin{align}\label{e.nabla_psi=zeta}
\dr_q \psi({q}+t\nabla\xi({p})) = {p}.
\end{align}

In conclusion, (<ref>) follows from (<ref>) and (<ref>).

Recall $\mathscr{P}_{t,{q}}$ in (<ref>), $\theta$ in (<ref>), and $\mcl J_{t,{q}}$ in (<ref>). We have

\begin{align}\label{e.P=J}
\mathscr{P}_{t,{q}}({p}) = \mcl J_{t,{q}}\Ll({q}+ t \nabla\xi({p}),{p}\Rr).
\end{align}

Let ${p}$ and ${q'}$ be as given in the statement. The relation (<ref>) in Proposition <ref> implies that $({q'},p)$ is a critical point.
The convergence in (<ref>) follows from Proposition <ref> and (<ref>).

Proposition <ref> also proves that any subsequential limit of the free energy must satisfy the Hamilton–Jacobi equation (<ref>) at every point of differentiability of the limit. We recall that this limit function must be Gateaux differentiable jointly in its two variables “almost everywhere”, by Proposition <ref>.

§.§ Critical point representation

Suppose that $\Ll(\bar F_N\Rr)_{N\in\N}$ converges pointwise to some limit $f$.
For every $(t,{q})\in\R_+\times \C_2$, there is ${p} \in \C\cap L^\infty_{\leq 1}$ such that

\begin{gather}\label{e.p.1st_stronger_id}
f(t,{q}) = \mathscr{P}_{t,{q}}({p}), \qquad
{p} =\dr_q\psi({q} + t\nabla\xi({p})).
\end{gather}

Proposition <ref> yields a sequence $({q}_n)_{n\in\N}$ in $  \C_\uparrow \cap L^\infty$ convergent to ${q}$ in $\cH$ such that $f(t,\cdot)$ is Gateaux differentiable at every ${q}_n$. By the continuity of $f$ given there, we have

\begin{align}\label{e.f(t,mu)=limf(t,mu_n)}
f(t,{q}) = \lim_{n\to\infty} f(t,{q}_n).
\end{align}

We write ${p}_n := \dr_q f(t,{q}_n)$. Proposition <ref> also implies that $\sup_{n\in\N}|{p}_n|_{L^\infty} \leq 1$.
By passing to a subsequence, we can assume that ${p}_n$ converges weakly in $\cH$ to some limit ${p}$. Arguing as in Step 3 of the proof of Proposition <ref>, we can upgrade this to the strong convergence in $L^2$, and we also see that ${p} \in \C\cap L^\infty_{\leq 1}$.

Recall $\mathscr{P}_{t,{q}}$ in (<ref>). By Corollary <ref> and the local Lipschitzness of $\nabla\xi$ and $\theta$, we have

\begin{align*}
\Ll|\mathscr{P}_{t,{q}_n}({p}_n) - \mathscr{P}_{t,{q}}({p})\Rr| &\leq  \Ll|{q}_n-{q}\Rr|_{L^1}+t\Ll|\nabla\xi({p}_n)-\nabla\xi({p})\Rr|_{L^1}  + t\Ll|\theta({p}_n)-\theta({p})\Rr|_{L^1}
\\
&\leq Ct \Ll|{p}_n-{p}\Rr|_{L^1} + \Ll|{q}_n-{q}\Rr|_{L^1},
\end{align*}

for some constant $C$ depending only on $\xi$. By Jensen's inequality, we have that $q_n$ and $p_n$ converge in $L^1$ to $q$ and $p$ respectively as $n$ tends to infinity, and therefore

\begin{equation*}  %\label{e.}
\lim_{n\to\infty} \mathscr{P}_{t,{q}_n}({p}_n) = \mathscr{P}_{t,{q}}({p}).
\end{equation*}

By construction, the function $f(t,\cdot)$ is Gateaux differentiable at ${q}_n$. Proposition <ref> therefore ensures that

\begin{equation*}  %\label{e.}
f(t,{q}_n) = \mathscr{P}_{t,{q}_n}({p}_n).
\end{equation*}

These along with (<ref>) yields $f(t,{q}) = \mathscr{P}_{t,{q}}({p})$, which gives the first relation in (<ref>).

Proposition <ref> ensures that ${p}_n = \dr_q\psi({q}_n+t\nabla\xi({p}_n))$.
Using (<ref>) and the local Lipschitzness of $\nabla\xi$, we get

\begin{align*}
\Ll|\dr_q\psi({q}_n+t\nabla\xi({p}_n))- \dr_q\psi({q}+t\nabla\xi({p}))\Rr|_{L^2} \leq C|{q}_n-{q}|_{L^2} + C|{p}_n-{p}|_{L^2},\quad\forall n\in\N,
\end{align*}

for some $C$ only depending on $\xi$. Sending $n$ to infinity, we get ${p} = \dr_q\psi({q}+t\nabla\xi({p}))$, which gives the second relation in (<ref>).

Let ${p}$ be given in Proposition <ref> and set ${q'}:= {q} + t\nabla\xi({p})$. The second relation in (<ref>) ensures that $({q'},p)$ is a critical point. The first relation in (<ref>) along with (<ref>) yields (<ref>).

§.§ Convergence of the overlap

§.§.§ Convergence of the conditional overlap

Recall the Gibbs measure $\la \cdot\ra_N$ associated with the original free energy given in (<ref>).
For $\ell,\ell'\in\N$, we consider the conditional expectation of the spin overlap

\begin{align*}
:=\E \la \frac{\sigma^\ell\Ll(\sigma^{\ell'}\Rr)^\intercal}{N} \,\Big|\,\alpha^\ell\wedge \alpha^{\ell'}\ra_N,
\end{align*}

where the conditioning is with respect to $\E \la \cdot\ra_N$ (not $\la\cdot\ra_N$). We also recall that $\la \cdot \ra_{\fR}$ is the expectation with respect to the measure $\fR^{\otimes \N}$.

Suppose that $\Ll(\bar F_N\Rr)_{N\in\N}$ converges pointwise to some limit $f$  along a subsequence $(N_k)_{k\in\N}$, and let $t \ge 0$.
If $f(t,\cdot)$ is Gateaux differentiable at some ${q} \in \C_\uparrow\cap L^\infty$, then $\Ll(R_{N_k,\sigma|\alpha}^{\ell,\ell'}\Rr)_{\ell,\ell'\in\N:\:\ell\neq \ell'}$ under $\E \la \cdot\ra_{N_k}$ converges in law to $\Ll({p}\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\Rr)_{\ell,\ell'\in\N:\:\ell\neq \ell'}$ under $\E \la\cdot\ra_\fR$ as $k$ tends to infinity, where ${p} = \dr_q f(t,q)$.

We write ${p}_N := \dr_q \bar F_N \Ll(t,{q}\Rr)$.
Using the invariance of the Poisson–Dirichlet cascade in Proposition <ref>, the computation of $\dr_q \bar F_N \Ll(t,{q}\Rr)$ in (<ref>), and the definition of conditional expectation, we have the following identities, for any bounded continuous $\kappa : [0,1] \to S^D$:

\begin{multline*}
\E \la \kappa\Ll(\alpha\wedge\alpha'\Rr)\cdot {p}_N\Ll(\alpha\wedge\alpha'\Rr) \ra_N = \int_0^1\kappa(s)\cdot {p}_N(s)\d s = \la \kappa, \dr_q \bar F_N \Ll(t,{q}\Rr)\ra_\cH
\\
=\E \la \kappa \Ll(\alpha\wedge\alpha'\Rr)\cdot \frac{\sigma\sigma'^\intercal}{N}\ra_N = \E \la \kappa \Ll(\alpha\wedge\alpha'\Rr)\cdot \E \la\frac{\sigma\sigma'^\intercal}{N}\,\Big|\, \alpha\wedge\alpha' \ra_N\ra_N.
\end{multline*}

Hence, we deduce that

\begin{align}\label{e.cond_overlap=p_N()}
\E \la\frac{\sigma\sigma'^\intercal}{N}\,\Big|\, \alpha\wedge\alpha'\ra_N = {p}_N\Ll(\alpha\wedge\alpha'\Rr)
\end{align}

a.s. under $\E\la\cdot\ra_N$. For every ${p}'\in \C_\infty$, we write $O_{{p}'}^{\ell,\ell'} := {p}'\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)$,
so that under $\E \la\cdot\ra_N$,

\begin{align}\label{e.R=O}
R_{N,\sigma|\alpha}^{\ell,\ell'} = O_{{p}_N}^{\ell,\ell'},\quad\forall \ell,\ell'\in \N:\: \ell\neq \ell'.
\end{align}

Proposition <ref> ensures that ${p}_N$ converges to ${p}$ in $L^1$ along the subsequence, and thus

\begin{align}\label{e.E<|p_N-p|>}
\E \la \Ll|{p}_N\Ll(\alpha\wedge\alpha'\Rr)-{p}\Ll(\alpha\wedge\alpha'\Rr)\Rr|\ra_\fR = \Ll|{p}_N- {p}\Rr|_{L^1}
\end{align}

tends to zero as $N$ tends to infinity along the subsequence.
We fix any $n\in\N$ and any bounded Lipschitz $g:\Ll(\R^{\D\times\D}\times \R\Rr)^{n^2-n} \to \R$. We write $R^\square_{N,\sigma|\alpha} := \Ll(R_{N,\sigma|\alpha}^{\ell,\ell'}\Rr)_{\ell,\ell'\leq n; \ell\neq \ell'}$ and similarly for $O^\square_{{p}_N}$ and $O^\square_{p}$.
The invariance of the Poisson–Dirichlet cascade from Proposition <ref> implies that $O_{{p}_N}$ under $\E\la\cdot\ra_N$ has the same distribution as $O_{{p}_N}$ under $\E \la\cdot\ra_\fR$.
Combining this with (<ref>) yields that

\begin{align*}
&\Ll|\E \la g\Ll(R^\square_{N,\sigma|\alpha}\Rr) \ra_N - \E \la g\Ll(O^\square_{p}\Rr) \ra_\fR \Rr| = \Ll| \E \la g\Ll(O^\square_{{p}_N}\Rr) \ra_N - \E \la g\Ll(O^\square_{p}\Rr) \ra_\fR \Rr|
\\
&=  \Ll| \E \la g\Ll(O^\square_{{p}_N}\Rr) \ra_\fR - \E \la g\Ll(O^\square_{p}\Rr) \ra_\fR \Rr|
\leq \|g\|_\mathrm{Lip}\E \la \Ll|O^\square_{{p}_N} - O^\square_{p}\Rr| \ra_\fR.
\end{align*}

Using also that (<ref>) tends to zero along the subsequence therefore yields the announced result.

§.§.§ Convergence of the overlap under perturbation

Let $\Ll(\hat H_N(\sigma)\Rr)_{\sigma\in\R^{D\times N}}$ be the Gaussian process in (<ref>).
For every $\Ll(t,\hat t,{q}\Rr)\in \R_+ \times \R_+\times \C_\infty$, we consider

\begin{multline}
\label{e.def.FN.t.hat}
\hat F_N\Ll(t,\hat t,{q}\Rr)
\\
:= -\frac{1}{N} \E\log \int \exp\Ll(H_N^{t,{q}}(\sigma,\alpha)+\sqrt{2\hat t}\hat H_N(\sigma)- \hat t N\Ll|\frac{\sigma\sigma^\intercal}{N}\Rr|^2\Rr) \d P_N(\sigma)\d \fR(\alpha),
\end{multline}

where $H^{t,{q}}_N(\sigma,\alpha)$ is given in (<ref>).
We denote the associated Gibbs measure by $\la \cdot\ra_{N,\hat t}\, $.

Recall the functional $\hat {\mcl J}_{t,\hat t,q}$ defined in (<ref>), and the overlaps $R_N$ and $Q_{p}$ defined in (<ref>) and (<ref>).

Suppose that $\Ll(\hat F_N\Rr)_{N\in\N}$ converges pointwise to some limit $f$  along a subsequence $(N_k)_{k\in\N}$.
Then, for each $t \ge 0$, the function $f\Ll(t,\cdot,\cdot\Rr) : \R_+ \times \mcl Q_2 \to \R$ is Gateaux differentiable (jointly in its two variables) on a subset of $\R_+ \times (\C_\uparrow\cap L^\infty)$ that is dense in $\R_+ \times \C_2$. Moreover, for every $\hat t \ge 0$ and every point ${q} \in \C_\uparrow\cap L^\infty$ of Gateaux differentiability of $f(t,\hat t, \cdot)$, the following holds for ${p} = \dr_q f\Ll(t,\hat t, {q}\Rr)$ and ${q'} = {q} + t\nabla \xi\Ll({p}\Rr) + 2 \hat t {p}$:

*  ${p} = \dr_q \psi \Ll({q'}\Rr)$;

*  if $(N_k)_{k\in\N}$ is the full sequence $(N)_{N\in\N}$, then $\lim_{N\to\infty}\hat F_N\Ll(t,\hat t, {q}\Rr) = \hat{\mathcal{J}}_{t,\hat t, {q}}({q'},{p})$;

*  if $\hat t>0$ and $f(t,\cdot,{q})$ is differentiable at $\hat t$, then $\Ll(R_{N_k}^{\ell,\ell'}\Rr)_{\ell,\ell'\in\N:\:\ell\neq \ell'}$ under $\E \la \cdot\ra_{N_k,\hat t}$ converges in law to $\Ll({p}\Ll(\alpha^\ell\wedge\alpha^{\ell'}\Rr)\Rr)_{\ell,\ell'\in\N:\:\ell\neq \ell'}$ under $\E \la\cdot\ra_\fR$ as $k$ tends to infinity.

By straightforward modifications, the differentiability follows from Proposition <ref>, part (<ref>) follows from Proposition <ref>, and part (<ref>) follows from Proposition <ref>.

Hence, we focus on part (<ref>).
For convenience, we drop the subsequence $(N_k)_{k \in \N}$ in the notation, but we understand that we only take limits along the subsequence throughout the proof.

By an adaptation of Proposition <ref>, we have (along the subsequence)

\begin{align}\label{e.cvg_mathscrF}
\lim_{N\to\infty} \Ll|\dr_q \hat F_N \Ll(t,\hat t,{q}\Rr) - \dr_q f\Ll(t,\hat t,{q}\Rr)\Rr|_\cH = 0\quad \text{ and } \quad \lim_{N\to\infty} \partial_{\hat t} \hat F_N\Ll(t,\hat t,{q}\Rr) = \partial_{\hat t} f\Ll(t,\hat t,{q}\Rr).
\end{align}

For brevity, we write ${p}_N := \dr_q \hat F_N \Ll(t,\hat t,{q}\Rr)$, ${p} := \dr_q f\Ll(t,\hat t,{q}\Rr)$, and $\la \cdot\ra_N := \la\cdot\ra_{N,\hat t}\, $.
We have

\begin{align*}
\partial_{\hat t} \hat F_N\Ll(t,\hat t,{q}\Rr) = \E \la \Ll|\frac{\sigma\sigma'^\intercal}{N}\Rr|^2\ra_N.
\end{align*}

Adapting the proof of Proposition <ref>, we have that  (<ref>) holds, which implies that

\begin{align}
\E \la \Ll|\frac{\sigma\sigma'^\intercal}{N}-{p}_N\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_N
&= \E \la \Ll|\frac{\sigma\sigma'^\intercal}{N}\Rr|^2\ra_N - \E \la \Ll|{p}_N\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_N  \notag
\\
&= \partial_{\hat t} \hat F_N\Ll(t,\hat t,{q}\Rr) - \Ll|\dr_q\hat F_N\Ll(t,\hat t,{q}\Rr)\Rr|^2_\cH \label{e.E<>=E<>-E<>}
\end{align}

where in the second equality we used the invariance of the Poisson–Dirichlet cascade.

We can now argue as for (<ref>) in Proposition <ref> to conclude. Indeed, since $t$ is fixed here, we view $\hat F_N(t,\cdot,\cdot)$ and $f(t,\cdot,\cdot)$ as functions of $(\hat t,{q})$. We can substitute $\hat t$ and $|\cdot|^2$ for $t$ and $\xi$ in Proposition <ref> to get

\begin{gather}\label{e.f(s,mu)_PDE}
\partial_{\hat t} f\Ll(t,\hat t,{q}\Rr) = \Ll|\dr_q f\Ll(t,\hat t,{q}\Rr)\Rr|^2_\cH.
%{p} = \dr_q\psi\Ll({q}+\frac{1}{2}\nabla\xi({p})+2s{p}\Rr).\label{e.zeta_fixedpt}
\end{gather}

Combining (<ref>), (<ref>), and (<ref>), we obtain that (along the subsequence)

\begin{align}
\label{e.RNQN.close}
\lim_{N\to\infty} \E \la \Ll|\frac{\sigma\sigma'^\intercal}{N}-{p}_N\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_N =0.
\end{align}

The first convergence in (<ref>) ensures that (along the subsequence)

\begin{align}
\label{e.pNp.close}
\lim_{N\to\infty} \E \la \Ll|{p}_N\Ll(\alpha\wedge\alpha'\Rr)-{p}\Ll(\alpha\wedge\alpha'\Rr)\Rr|^2\ra_\fR =0.
\end{align}

We fix any $n\in\N$ and any bounded Lipschitz $g:\Ll(\R^{\D\times\D}\times \R\Rr)^{n^2-n} \to \R$.
We write $R^\square_N := \Ll(R_N^{\ell,\ell'}\Rr)_{\ell,\ell'\leq n; \ell\neq \ell'}$ and similarly for $Q^\square_{{p}_N}$ and $Q^\square_{p}$.
The invariance of the Poisson–Dirichlet cascade implies that $Q_{{p}_N}$ under $\E\la\cdot\ra_N$ has the same distribution as $Q_{{p}_N}$ under $\E \la\cdot\ra_\fR$.
We therefore have that

\begin{align*}
&\Ll|\E \la g\Ll(R^\square_N\Rr) \ra_N - \E \la g\Ll(Q^\square_{p}\Rr) \ra_\fR \Rr|
\\
&=  \Ll|\E \la g\Ll(R^\square_N\Rr) \ra_N - \E \la g\Ll(Q^\square_{{p}_N}\Rr) \ra_N +\E \la g\Ll(Q^\square_{{p}_N}\Rr) \ra_\fR - \E \la g\Ll(Q^\square_{p}\Rr) \ra_\fR \Rr|
\\
& \leq \|g\|_\mathrm{Lip}\Ll(\E \la \Ll|R^\square_N - Q^\square_{{p}_N}\Rr| \ra_N + \E \la \Ll|Q^\square_{{p}_N} - Q^\square_{p}\Rr| \ra_\fR\Rr).
\end{align*}

Combining this with (<ref>) and (<ref>) therefore completes the proof.

The theorem follows from Proposition <ref>.

§.§ Stability and high-temperature uniqueness of the critical point

We now turn to a proof of Proposition <ref> on the uniqueness of the critical point at high temperature.

The fact that $\mcl J_{t,q}$ has at least one critical point is a consequence of Theorem <ref> (and could also be proved directly).
For each $i \in \{1,2\}$, let $(q'_i,{p}_i)\in \mcl Q_2^2$ be a critical point of $\mcl J_{t,q}$.
Inserting (<ref>) into (<ref>), we get ${p}_i = \dr_q \psi(q+t\nabla \xi({p}_i))$.
By (<ref>), we have

\begin{align*}
\Ll|{p}_1-{p}_2\Rr|_{L^2}\leq 16 t \Ll|\nabla\xi({p}_1)-\nabla\xi({p}_2)\Rr|_{L^2}.
\end{align*}

By the local Lipschitzness of $\nabla\xi$, we have

\begin{align*}
C := \sup_{a,a'\in \S^\D_+:\:|a|,|a'|\leq 1;\; a\neq a'}\frac{\Ll|\nabla\xi(a)-\nabla\xi(a')\Rr|}{|a-a'|}<+\infty.
\end{align*}

The property in (<ref>) implies that $|{p}_i(s)|\leq 1$ for each $i\in\{1,2\}$ and $s\in[0,1)$. Therefore,

\begin{align*}
\Ll|{p}_1-{p}_2\Rr|_{L^2}\leq 16  Ct \Ll|{p}_1-{p}_2\Rr|_{L^2}.
\end{align*}

Hence, if $ t < t_c := (16 C)^{-1}$, we must have ${p}_1 ={p}_2$. Since $q'_i$ is determined by (<ref>), the uniqueness follows.

We close this section with a proof of Proposition <ref> on the stability of relevant critical points.

We take $(t_n, q_n) \in \R_+ \times \mcl Q_2$ and $(q'_n,p_n) \in \mcl Q_2^2$ as in the statement. The fact that $(q'_n,p_n)$ is a critical point of $\mcl J_{t_n, q_n}$ (see (<ref>) and (<ref>)) means that

\begin{equation}
\label{e.stability.crit}
p_n = \partial_q \psi(q'_n) \quad \text{ and } \quad  q'_n = q_n + t_n \nabla \xi(p_n).
\end{equation}

By (<ref>) from Corollary <ref>, we have that $p_n \in \mcl Q \cap L^\infty_{\le 1}$. By Lemma <ref>, the sequence $(p_n)_{n \in \N}$ is therefore precompact in $L^2$. By the second relation in (<ref>) and the convergence of $(t_n,q_n)_{n \in \N}$ in $\R_+ \times L^2$, we deduce that the sequence $(q'_n, p_n)_{n \in \N}$ is precompact. Moreover, if $(p_n)_{n \in \N}$ converges in $L^2$ along a subsequence, then $(q_n', p_n)_{n \in \N}$ converges in $(L^2)^2$ along the same subsequence. Recall also from (<ref>) that

\begin{equation}
\label{e.stability.J.tn}
\mcl J_{t_n,q_n}(q'_n,p_n) = \psi(q'_n) + \la p_n , q_n-q'_n\ra_\cH + t_n \int_0^1 \xi(p_n).
\end{equation}

Letting $(q',p) \in \mcl Q_2^2$ denote a subsequential limit of $(q'_n,p_n)_{n \in \N}$, we have that $\psi(q'_n)$ converges to $\psi(q')$ along the subsequence, by the continuity of $\psi$ from Corollary <ref>. The scalar product in (<ref>) converges to $\la p, q-q'\ra_{\cH}$ along the subsequence, by the Cauchy–Schwarz inequality. Finally, the integral $\int_0^1 \xi(p_n)$ converges to $\int_0^1 \xi(p)$ along the subsequence, because $\xi$ is continuous and $p_n \in L^\infty_{\le 1}$. So we conclude that as $n$ tends to infinity along the subsequence, we have that $\mcl J_{t_n, q_n} (q'_n, p_n)$ converges to $\mcl J_{t,q}(q',p)$. Using also the uniform Lipschitz continuity of $\bar F_N$ from Proposition <ref>, we obtain (<ref>). Using the continuity of $\dr_q \psi$ from Corollary <ref> and (<ref>), we also verify that $(q',p)$ is a critical point of $\mcl J_{t,q}$, and thus the proof is complete.

§ FURTHER RESULTS IN THE CONVEX CASE

In this section, we give a proof of Theorem <ref>. We also derive a number of alternative representations of the variational problem describing the limit free energy, and explain how to remove the compensating term $-N t \xi(\sigma \sigma^\intercal)/N$ appearing in (<ref>). We also show that the limit free energy is Gateaux differentiable everywhere in $(0,\infty) \times (\mcl Q_\uparrow \cap L^\infty)$, and use this to show that it satisfies the Hamilton–Jacobi equation (<ref>) at every point in this set. We collect all results into a final statement that also describes the convergence of the conditional $\sigma$-overlaps for every $(t,q) \in (0,\infty) \times (\mcl Q_\uparrow \cap L^\infty)$.

§.§ Parisi formula when xi is convex on SD̂_+

When $\xi$ is convex over the whole space $\R^{D\times D}$, an interpolation argument allows one to establish a lower bound on any subsequential limit of $\bar F_N(t,q)$ [49, 13]. When $\D=1$, one can generalize this bound to the case when $\xi$ is only assumed to be convex over $\S^1_+=\R_+$ using the Talagrand positivity principle (see <cit.>). If $\D>1$ and $\xi$ only depends on the $D$ diagonal entries of its argument, then one may as well think of $\xi$ as being defined on $\R^D$, and one can simplify the setup and only consider paths that take values in $\R^D_+$ instead of $S^D_+$. In this context, one can still apply the scalar positivity principle to get the Parisi formula under the assumption that $\xi$ is convex over $\R^\D_+$, see <cit.>.

For general vector spin glass models, to the best of our knowledge, existing results all require $\xi$ to be convex on $\R^{\D\times\D}$. Here we obtain the lower bound on the limit free energy by combining the results of [71] and [29]. We derive the converse bound from the results of this paper, but as already mentioned, these results are much more precise than necessary in this case.

Recall the definitions of $\sP_{t,{q}}$ in (<ref>) and of $\mathcal{K}$ in (<ref>).

If $\xi$ is convex on $\S^\D_+$, then for every $t \ge 0$ and $q \in \mcl Q_2$, we have

\begin{align}\label{e.parisi_hj}
\lim_{N\to\infty} \bar F_N(t,{q}) = \sup_{{p}\in\C:\: \exists a \in \mathcal{K},\, p\leq a}\sP_{t,{q}}({p})= \sup_{{p}\in\C_\infty}\sP_{t,{q}}({p}).
\end{align}

We fix any $(t,q)$.
Under the stated convexity assumption, the lower bound in <cit.> expressed in terms of the solution to (<ref>) is proved in <cit.> to admit a representation by the Hopf–Lax formula:

\begin{align}\label{e.hopf-lax}
\liminf_{N\to\infty} \bar F_N(t,q)  \geq \sup_{{q'}\in \C_\infty} \inf_{p'\in\C_\infty}\Ll\{\psi({q}+{q'})- \la q',p'\ra_\cH +t\int_0^1 \xi({p'}(s))\d s\Rr\}.
\end{align}

On the other hand, by Corollary <ref>, we can find ${p} \in \C\cap L^\infty_{\leq 1}$ and $a\in\mathcal{K}$ satisfying $a\geq p$ such that

\begin{align}\label{e.limsup<}
\limsup_{N\to\infty} \bar F_N(t,{q}) \leq \sP_{t,{q}}({p}) = \psi({q}+t\nabla\xi({p}))-t \int_0^1\theta({p}(s))\d s,
\end{align}

where we recall that $\theta$ is defined in (<ref>).
To compare these two formulas, we need to show that

\begin{align}\label{e.theta=xi*()}
\theta(b) = \xi^*(\nabla\xi(b)),\quad\forall b\in \S^\D_+,
\end{align}

with $\xi^*$ as in (<ref>).
Fix any $b\in\S^\D_+$. It is clear from their definitions that $\theta(b) \leq  \xi^*(\nabla\xi(b))$. For any $b'\in\S^\D_+$, the convexity of $\xi$ on $S^D_+$ implies that for every $\lambda \in [0,1]$,

\begin{equation*}  %\label{e.}
\lambda \xi(b') + (1-\lambda)\xi(b) \ge \xi(\lambda b'+ (1-\lambda)b).
\end{equation*}

Rearranging terms and sending $\lambda$ to zero, we obtain that

\begin{equation*}  %\label{e.}
\xi(b')-\xi(b)\geq (b'-b)\cdot \nabla\xi(b),
\end{equation*}

and thus

\begin{equation*}  %\label{e.}
b\cdot\nabla\xi(b)-\xi(b)\geq b'\cdot\nabla\xi(b)-\xi(b').
\end{equation*}

Taking the supremum over $b'\in\S^\D_+$, we get $\theta(b)\geq \xi^*(\nabla\xi(b))$, and thus complete the argument for (<ref>).

Using (<ref>), we have

\begin{multline*}
\int_0^1\theta({p}(s))\d s  = \int_0^1 \xi^*(\nabla\xi({p}(s))) \d s \geq \sup_{p'\in\C_\infty}\Ll\{\la \nabla\xi(p), p'\ra_\cH-\int_0^1\xi(p'(s))\d s \Rr\}
\\
\geq \la \nabla\xi(p), p\ra_\cH-\int_0^1\xi(p(s))\d s = \int_0^1\theta({p}(s))\d s ,
\end{multline*}

which implies

\begin{align}\label{e.int_theta=sup}
\int_0^1\theta({p}(s))\d s = \sup_{p'\in\C_\infty}\Ll\{\la \nabla\xi(p), p'\ra_\cH-\int_0^1\xi(p'(s))\d s \Rr\}.
\end{align}

Applying this to (<ref>), we get

\begin{align*}
\liminf_{N\to\infty} \bar F_N(t,q)  & \geq  \inf_{p'\in\C_\infty}\Ll\{\psi({q}+t\nabla\xi(p))- t\la \nabla\xi(p),p'\ra_\cH +t\int_0^1 \xi({p'}(s))\d s\Rr\}
\\
&\geq \psi({q}+t\nabla\xi(p))- t\int_0^1 \theta({p}(s))\d s,
\end{align*}

which along with (<ref>) yields the first equality in (<ref>). The second equality can be deduced similarly.

Fix any $(t,q)\in\R_+\times \C_2$.
Applying (<ref>) to (<ref>) and comparing it with (<ref>), we get

\begin{align*}
\lim_{N\to\infty} \bar F_N(t,q)  &= \sup_{{q'}\in \C_\infty} \inf_{p'\in\C_\infty}\Ll\{\psi({q}+{q'})- \la q',p'\ra_\cH +t\int_0^1 \xi({p'}(s))\d s\Rr\}
\\
& = \sup_{q'\in q+\C_\infty} \inf_{p\in\C_\infty} \mcl J_{t,q}(q',p),
\end{align*}

where the last equality follows from the definition of $\mcl J_{t,{q}}(q',{p})$ in (<ref>).
Hence, we have verified the first identity in (<ref>).
It is proved in <cit.> that the unique viscosity solution $f$ to (<ref>), defined in <cit.>, admits the Hopf–Lax representation under the assumption that $\xi$ is convex on $\S^\D_+$. The Hopf–Lax formula for $f$ evaluated at $(t,q)$ has the same expression as the one on the first line of the above display. Since this identity holds at every $(t,q)$, we conclude that $\lim_{N\to\infty}\bar F_N =f$ pointwise everywhere on $\R_+\times \C_2$.

In the above proof, to show that $\Ll(\bar F_N\Rr)_{N\in\N}$ converges to the solution $f$, we directly match the variational formula for $\lim_{N\to\infty}\bar F_N$ with the Hopf–Lax representation of $f$. We mention another approach via the comparison principle for viscosity solutions. Let $p$ be as given in (<ref>) and consider the function $g: (t',q')\mapsto \sP_{t',q'}(p)$. Using the convexity of $\xi$, one can verify that $g$ is a subsolution of (<ref>). The comparison principle (e.g. <cit.>) implies that $g\leq f$ everywhere. This along with (<ref>) yields $\limsup \bar F_N(t,q)\leq g(t,q)\leq f(t,q)$. Since $(t,q)$ was arbitrary, we have $\limsup \bar F_N\leq f$ everywhere on $\R_+\times \C_2$. The matching lower bound is provided by <cit.>. This argument can be seen in [25] for $D=1$.

In the corollary below, we verify the version of the Hopf–Lax formula appearing in (<ref>).

If $\xi$ is convex on $\S^\D_+$, then for every $t \ge 0$ and $q \in \mcl Q_2$, we have

\begin{align}
\label{e.parisi.convex2}
\lim_{N\to\infty} \bar F_N(t,q) = \sup_{q' \in \mcl Q_\infty} \Ll\{ \psi(q + q') - t \int_0^1 \xi^* \Ll( \frac{q'}{t} \Rr)  \Rr\} .
\end{align}

We fix any $(t,q)$ and denote the variational formula on the right-hand side of (<ref>) by $\rhs$.
Using (<ref>) in (<ref>), we get

\begin{equation*}  %\label{e.}
\limsup_{N\to\infty}\bar F_N(t,q)\leq\rhs.
\end{equation*}

To show the converse bound, we recall the definition of $\xi^*$ in (<ref>) to see that, for every $q' \in \mcl Q_\infty$,

\begin{align*}
\sup_{p'\in\C_\infty}\Ll\{\frac{1}{t}\la q',p'\ra_\cH -\int_0^1\xi(p'(s))\d s\Rr\} \leq \int_0^1 \xi^* \Ll(\frac{q'(s)}{t}\Rr)\d s.
\end{align*}

Using this in (<ref>), we get $\liminf_{N\to\infty}\bar F_N(t,q)\geq \rhs$. This gives the announced identity at $(t,q)$.

We can extract from Proposition <ref> a result that takes a form that is more common in the literature. For every $\pi\in\C_\infty$ and $x\in \S^\D$, we set

\begin{align} \label{e.sP(pi,x)}
\begin{split}
\sP(\pi,x) := \E\log\iint\exp\Ll(w^{\nabla\xi\circ\pi}(\alpha)\cdot\tau-\frac{1}{2}\nabla\xi\circ\pi(1)\cdot\tau\tau^\intercal+x\cdot \tau\tau^\intercal \Rr)\d P_1(\sigma) \d \fR(\alpha)
\\+\frac{1}{2}\int_0^1\theta\Ll(\nabla\xi\circ\pi(s)\Rr)\d s.
\end{split}
\end{align}

This resembles the classic Parisi functional for the Sherrington–Kirkpatrick model.

If $\xi$ is convex on $\S^\D_+$, then

\begin{align*}
\lim_{N\to\infty} \frac{1}{N}\E \log \int \exp\Ll(H_N(\sigma) - \frac{N}{2}\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)\Rr)\d P_N(\sigma)
= \inf_{\pi\in\C_\infty} \sP(\pi,0).
\end{align*}

Recall the expression of $\bar F_N(t,q)$ from (<ref>) and the random Hamiltonian $H_N(\sigma)$ with covariance (<ref>).
Setting $t=\frac{1}{2}$ and $q=0$, we get

\begin{align*}
-\bar F_N\Ll(\frac{1}{2},0\Rr)=\frac{1}{N}\E \log \int \exp\Ll(H_N(\sigma) - \frac{N}{2}\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)\Rr)\d P_N(\sigma).
\end{align*}

Recalling the definition of $\sP_{t,q}$ in (<ref>), we have $-\sP_{\frac{1}{2},0}(p) = \sP(p,0)$ for every $p\in\C_\infty$, where we also used $\sqrt{2}w^{\frac{1}{2}q'}\stackrel{\d}{=}w^{q'}$ for $q'\in\C_\infty$, which is clear from (<ref>).
Now, the announced identity follows from (<ref>) in Proposition <ref> evaluated at $t=\frac{1}{2}$ and $q=0$.

We can further remove the correction term $- \frac{N}{2}\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)$ and get the following.

If $\xi$ is convex on $\S^\D_+$, then

\begin{align*}
\lim_{N\to\infty} \frac{1}{N}\E \log \int \exp\Ll(H_N(\sigma)\Rr)\d P_N(\sigma)
& = \sup_{y\in\S^\D_+}\inf_{\pi\in\C_\infty}\Ll\{\sP(\pi,y)-\frac{1}{2}\xi^*(2y)\Rr\}
\\
& = \sup_{z\in\S^\D_+}\inf_{y\in\S^\D_+,\ \pi\in\C_\infty}\Ll\{\sP(\pi,y)-y\cdot z + \frac{1}{2}\xi(z)\Rr\}.
\end{align*}

The method is the same as in <cit.> and <cit.>. For $N\in\N$ and $(t,x) \in \R_+\times \S^\D$, we consider

\begin{align*}
\cF_N(t,x) := \frac{1}{N}\E\log\int\exp\Ll(H_N(\sigma)- \frac{N}{2}\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)+ tN\xi\Ll(\frac{\sigma\sigma^\intercal}{N}\Rr)+x\cdot \sigma\sigma^\intercal\Rr)\d P_N(\sigma).
\end{align*}

It is shown in <cit.> that the convexity of $\xi$ over $\S^\D_+$ implies that the limit of $(\cF_N)_{N\in\N}$ in the local uniform topology is a viscosity solution $f$ of $\partial_t f - \xi(\nabla_x f) =0$ on $(0,\infty)\times \S^\D$. The notion of viscosity solutions is defined in <cit.>.
Recall that $\S^\D_{++}$ is the set of $\D$-by-$\D$ positive definite matrices, which is the interior of $\S^\D_+$.
Since $\S^\D_{++}$ is an open subset of $\S^\D$, we immediately have that $f$ is a viscosity solution of

\begin{align}\label{e.cF_hj}
\partial_t f - \xi(\nabla_x f) =0\qquad \text{on}\quad (0,\infty)\times \S^\D_{++}.
\end{align}

We want to use variational formulas for the solution of (<ref>) to conclude. For this, we need to first identify the initial condition of $f$ and verify more properties.

Applying Corollary <ref> with $e^{x\cdot\tau\tau^\intercal} \d P_1(\tau)$ substituted for $P_1$ therein, we have, for every $x\in\S^\D$,

\begin{align}\label{e.f(0,x)=}
f(0,x) = \lim_{N\to\infty} \cF_N(0,x)  = \inf_{\pi\in\Pi} \sP(\pi,x).
\end{align}

It is verified in <cit.> that $\cF_N$ is Lipschitz uniformly in $N$ and convex, and satisfies that $\cF_N(t,x) \leq \cF_N(t,x')$ whenever $x\geq x'$. Passing to the limit, we have that $f$ is Lipschitz and convex and that $f(t,\cdot)$ is increasing for every $t\geq 0$. These properties along with the well-posedness <cit.> of (<ref>) implies that $f$ is the viscosity solution of (<ref>) with the above initial condition, which is unique in the class of Lipschitz functions that are increasing for each fixed $t$.

Since $\xi$ is convex on $\S^\D_+$ and $f(0,\cdot)$ is also convex, we can represent $f$ by the Hopf–Lax formula and by the Hopf formula described in <cit.>: for every $t>0$ and $x\in\S^\D_+$,

\begin{align}\label{e.f(t,x)=}
\begin{split}
f(t,x) & =\sup_{y\in\S^\D_+}\Ll\{f(0,x+y)-t\xi^*\Ll(\frac{y}{t}\Rr)\Rr\}
\\
& = \sup_{z\in\S^\D_+}\inf_{y\in\S^\D_+}\Ll\{f(0,y)+(x-y)\cdot z+ t\xi(z)\Rr\}.
\end{split}
\end{align}

Notice that, for every $N\in\N$,

\begin{align*}
\cF_N\Ll(\frac{1}{2},0\Rr) = \frac{1}{N}\E\log\int \exp(H_N(\sigma))\d P_N(\sigma).
\end{align*}

Hence, setting $t=\frac{1}{2}$ and $x=0$ in (<ref>), inserting (<ref>) to (<ref>), and using the convergence of $\cF_N$ to $f$, we can obtain the announced identities.

In the spin glass literature, sometimes it is preferred to consider left-continuous paths in $\Pi$ defined in (<ref>) instead of the right-continuous ones in $\C_\infty$.
We explained in Remark <ref> that, for $\pi\in\Pi$, the Gaussian process $w^\pi$ exists and satisfies all the properties in Section <ref>. Hence, $\sP(\pi,x)$ is defined in the same way as in (<ref>) for $\pi\in\Pi$ and $x\in\S^\D$. By the $L^1$-Lipschitzness of $\psi$ in Corollary <ref>, the value of $\sP(\pi,x)$ does not change if we replace $\pi$ by its left-continuous version.
Therefore, $\inf_{\pi\in \C_\infty}$ in Corollary <ref> and Proposition <ref> can be replaced by $\inf_{\pi \in\Pi}$.
Due to the first equality in Proposition <ref>, we can further refine $\inf_{\pi\in\Pi}$ therein to $\inf_{\pi\in\Pi:\:\exists a \in \mathcal{K},\, \pi\leq a}$ for $\mathcal{K}$ defined in (<ref>).

§.§ Differentiability of the Parisi formula

We show that the limit free energy is differentiable. We use a similar argument as that for the so-called generic models, which is presented, for instance, in <cit.>.

Suppose that $\xi$ is convex on $\S^\D_+$, and let $f$ be the limit of $\Ll(\bar F_N\Rr)_{N\in\N}$ as identified in Proposition <ref>.

* For each $t\in \R_+$, the function $f(t,\cdot)$ is Gateaux differentiable everywhere on $\C_\uparrow\cap L^\infty$.

* The function $f$ is Gateaux differentiable everywhere on $(0,\infty)\times (\C_\uparrow\cap L^\infty)$.

We show the first property; the second one can be verified in an analogous way.
Fix any $(t,{q}) \in \R_+\times (\C_\uparrow\cap L^\infty)$.

We write $f({q'}) :=f(t,{q'})$ and $\sP_{{q'}}(\pi) := \sP_{t,{q'}}(\pi)$.
By Proposition <ref>, we can write, for every $q' \in \mcl Q_2$,

\begin{align}\label{e.limF_N=sP}
\lim_{N\to\infty} \bar F_N(t,q' )= f(q') = \sup_{\pi\in\C\cap L^\infty_{\leq 1}} \sP_{q'}(\pi).
\end{align}

This identity at $q$ together with (<ref>) implies that there is $\pi\in \C\cap L^\infty_{\leq 1}$ such that

\begin{align}\label{e.sP(pi)=f(q)}
\sP_q(\pi) =  f(q).
\end{align}

Step 1.
We show that $f$ is differentiable at $q$ along nice directions.
Let $\kappa\in C^{\infty}([0,1]; S^D)$ be such that $\kappa(0) = 0$. Due to $q\in \C_\uparrow$, by the same argument in Step 1 of the proof of Proposition <ref>, there are $c,\delta>0$ such that ${q} + r \kappa\in \C_{\uparrow,c}\cap L^\infty$ for all $r\in [-\delta,\delta]$ (we recall that $\C_{\uparrow,c}$ is defined in (<ref>)).
By Proposition <ref>, we have that the function $r\mapsto  f(q+r\kappa)$ is semi-concave on $[-\delta,\delta]$. We want to show that this function is differentiable at $r=0$. For this, we need to introduce the notion of superdifferential (e.g. <cit.>). For a function $g:I\to\R$ defined on an open interval $I\subset\R$, the superdifferential of $g$ at $r\in I$ is the set of real numbers $a$ satisfying

\begin{align*}
\limsup_{s\to r}\frac{g(s)-g(r)-a(s-r)}{|s-r|}\leq 0.
\end{align*}

It is well-known (e.g. <cit.>, or <cit.> for a variant with convex functions) that for a semi-concave function $g : I \to \R$ and $r\in I$, the superdifferential at $r$ is not empty, and if it contains exactly one point $a \in \R$, then $g$ is differentiable at $r$ with $\frac{\d}{\d r}g(r)=a$.
So we let $a \in \R$ be any element of the superdifferential of the mapping $r\mapsto  f(q+r\kappa)$ at $r=0$, and proceed to identify it uniquely.
In the following, we denote by $C$ a constant that depends only on $(t,{q})$, $c$, and $\kappa$, and may vary from instance to instance.
By Proposition <ref>, we have

\begin{align}\label{e.sP_semi_concave}
(1-\lambda)  f(q) + \lambda  f(q+r\kappa) -  f({q}+ \lambda r \kappa) \leq C\lambda(1-\lambda)r^2
\end{align}

uniformly in $\lambda\in[0,1]$ and $r$. Dividing both sides by $\lambda$, sending $\lambda$ to zero, and using the definition of the superdifferential, we get

\begin{align*}
f(q+r\kappa) -  f(q) -a r\leq Cr^2.
\end{align*}

Dividing both sides by $r>0$ and using (<ref>) and (<ref>), we get

\begin{align*}