# Central Limit Theorem of Overlap for the Mean Field Ghatak-Sherrington model

Yueqi Sheng School of Engineering & Applied Sciences, Harvard University,
Cambridge, Massachusetts, USA.
Email<EMAIL_ADDRESS>Qiang Wu School of Mathematics, University of
Minnesota, MN, USA.
Email<EMAIL_ADDRESS>
###### Abstract

The Ghatak-Sherrington (GS) spin glass model is a random probability measure
defined on the configuration space $\\{0,\pm 1,\pm
2,\ldots,\pm\mathcal{S}\\}^{N}$ with system size $N$ and $\mathcal{S}\geqslant
1$ finite. This generalizes the classical Sherrington-Kirkpatrick (SK) model
on the boolean cube $\\{-1,+1\\}^{N}$ in order to capture more complex
behaviors, including the spontaneous inverse freezing phenomenon. Although
many results on the physics side have been established to understand the GS
model, mathematical exploration of the model remains scarce. Overlap, the
normalized inner product of two configurations, acts as the system order
parameter to understand the phase transition of mean-field spin glass models.
In this paper, we use moment method combined with the cavity approach to
rigorously establish a quantitative joint central limit theorem for the
overlap and self-overlap array. The results hold at high temperatures under
arbitrary crystal and external fields. Compared to the SK model, the main
challenge comes from the non-trivial self-overlap terms which also correlated
with the standard overlap terms.

###### Contents

  1. 1 Introduction
    1. 1.1 Setting of the model
    2. 1.2 Relation to prior works
    3. 1.3 Notations
    4. 1.4 Main result
    5. 1.5 Proof outline
    6. 1.6 Organization of the paper
    7. 1.7 Acknowledgement
  2. 2 Cavity method and second moment estimates
    1. 2.1 Preliminaries
    2. 2.2 Variance of overlaps and self-overlaps
      1. 2.2.1 Variance of $T_{1,2}$
      2. 2.2.2 Variance of $T_{1}$ and $S_{1}$
      3. 2.2.3 Covariance: $S_{1}T_{1}$ term
  3. 3 General moments computation
    1. 3.1 Induction on $T_{k,l}$
    2. 3.2 Recursive relation for correlated "basis"
      1. 3.2.1 Induction on $T_{k}$ and $S_{k}$
      2. 3.2.2 Proof of Lemma 3.6
      3. 3.2.3 Induction on $T$ and $S$
      4. 3.2.4 Proof of Lemma 3.14
    3. 3.3 Proof of Lemma 3.1
  4. 4 Appendix
    1. 4.1 Proof of Lemma 3.3
    2. 4.2 Proof of Lemma 3.9, 3.10
    3. 4.3 Proof of Lemma 3.18, 3.19

##  1 Introduction

Mean field spin glass theory has undergone a flourishing development in the
last 20 years, in particular, it culminated with the seminal work of proof for
the celebrated Parisi’s formula by Talagrand [Tal06] and Panchenko [Pan14].
After that, a lot of rigorous results for the mean field spin glass system
have been established. Among those, the most notable models are the
Sherrington-Kirkpatrick model and its $p$-spin variants. Those models take the
standard $N$ dimensional boolean cube $\\{-1,+1\\}^{N}$ as the underlying
configuration space. An extensive list of those exciting developments can be
found in [CMP+23]. However, the binary spin value configuration is rather
restrictive. There are more realistic but also more complicated spin glass
models that are much less understood, where the spin values are beyond binary
and even general vectors in Euclidean space. Some examples include the Ghatak-
Sherrington model, Potts spin glass, XY-spin glass, etc. In this work, we take
one step forward to study the mean-field Ghatak-Sherrington model.

Compared to the classical SK model and its $p$-spin variants, where the spin
value is restricted to be binary. The Ghatak-Sherrington model, where
configuration space is the general hypercube, was first introduced in [GS77]
to study the so-called inverse freezing phenomenon. More specifically, this
inverse freezing phenomenon predicts that at a low enough temperature, there
is another replica symmetric regime. This is in sharp contrast to the binary
spin-valued models, such as SK and its $p$-spin variants, where the model in
the low-temperature regime is widely believed to exhibit replica symmetry
breaking only. One major reason for the inverse freezing phenomenon is due to
the crystal field effect (see the Hamiltonian definition in (1.2)). The
crystal field term is just trivially reduced to the system size $N$ in the SK
case. There are many results [DCdA00, dCYS94, KH99, LDA82, MS85, Leu07] due to
physicists in order to understand this phenomenon. However, to the best of our
knowledge, the number of mathematically rigorous results concerning the GS
model is quite limited. In [Pan05], Panchenko first proved a variational
formula for limiting free energy by generalizing Talagrand’s method to the GS
model and also later in [Pan18] via a different approach. Recently, Auffinger
and Chen [AC21] used the cavity method to establish the Thouless-Anderson-
Palmer equation for local magnetization. In this work, we focus on
investigating the behavior of overlap, which acts as the system order
parameter. The asymptotics of overlap contain fruitful information about the
Gibbs measure. In particular, it can be used to understand the inverse-
freezing phenomenon. The main contribution of this work is proving a joint
central limit theorem for the overlap in the high-temperature regime. Our
approach is based on moment computations and the cavity idea.

Let us first introduce the concrete setting for the mean-field Ghatak-
Sherrington model as follows.

###  1.1 Setting of the model

The Ghatak-Sherrington (GS) model is defined as follows: for each
configuration

$\displaystyle\bm{\sigma}=(\sigma_{1},\sigma_{2},\cdots,\sigma_{N})\in{}_{N,\operatorname{\mathcal{S}}}:=\\{0,\pm
1,\cdots,\pm\operatorname{\mathcal{S}}\\}^{N},$ (1.1)

where $\operatorname{\mathcal{S}}\geqslant 1$, the Hamiltonian of GS model is
given by

$\displaystyle
H_{N}(\bm{\sigma})=\frac{\beta}{\sqrt{N}}\sumop\slimits@_{i<j}g_{i,j}\sigma_{i}\sigma_{j}+D\sumop\slimits@_{i=1}^{N}\sigma_{i}^{2}+h\sumop\slimits@_{i=1}^{N}\sigma_{i},$
(1.2)

where $\beta>0$ is the inverse temperature, and $h\geqslant 0$ and
$D\in\px@BbbR$ represent the external and crystal fields respectively. One of
the fundamental questions in statistical physics is to compute the limiting
free energy

$F_{N}(\beta,h,D):=\frac{1}{N}\log Z_{N}(\beta,h,D),$

as the volume size $N\to\infty$, where the partition function of the GS model
is given by

$Z_{N}(\beta,D,h):=\sumop\slimits@_{\sigma\in{}_{N,S}}\exp(H_{N}(\sigma)).$

The associated GS Gibbs measure is

$\displaystyle
dG_{\beta,h,D}(\bm{\sigma})=\frac{\exp(H_{N}(\bm{\sigma}))}{Z_{N}(\beta,h,D)}\cdot
d\bm{\sigma},$ (1.3)

where $d\sigma$ is the uniform reference measure on
${}_{N,\operatorname{\mathcal{S}}}$. In the following context, we will
suppress the dependence on $\beta,h,D$ for the above objects unless it causes
confusion. Furthermore, we will use angular bracket $\langle\cdot\rangle$ to
denote the expectation w.r.t the Gibbs measure $dG(\bm{\sigma})$.

Another important object in the mean field spin glass theory, acting as the
order parameters of the system, is the overlap between two configurations
$\bm{\sigma}^{1},\bm{\sigma}^{2}\in{}_{N,\operatorname{\mathcal{S}}}$,

$\displaystyle
R_{1,2}=\frac{1}{N}\sumop\slimits@_{i=1}^{N}\sigma^{1}_{i}\sigma^{2}_{i},\quad
<EMAIL_ADDRESS>(1.4)

If $\sigma^{1}=\sigma^{2}$, the overlap becomes the self overlap, which is
defined below

<EMAIL_ADDRESS>
Compared to the Sherrington-Kirkpatrick (SK) model, where the configuration
space is $\\{-1,+1\\}^{N}$, the self-overlap is no longer trivially reduced to
1. The self-overlap in the GS model not just acts as an extra term, but also
correlates with the usual overlap. This makes the analysis more challenging.
Before presenting our main results, let us briefly discuss some existing works
addressing the fluctuation problem.

###  1.2 Relation to prior works

In this section, we review existing fluctuation results on the overlap in the
mean field spin glass theory and its related applications in other problems of
interest.

For the classical SK model, a central limit theorem of overlap in the zero
external fields was first proved in [CN95] via a stochastic calculus approach.
In the presence of a positive external field, the central limit theorem for
overlap for the array of overlaps was proved in [Tal11, GT02] using the moment
method combined with cavity method computations. Proving a CLT for overlap is
not just interesting itself, since overlap acts as the order parameter of the
system, it also has many further implications. For instance, in the proof of
Hanen’s theorem [Han07] for the limiting law of spin covariances, the moment
estimates for the overlap arrays play an important role. Another application
is on the spin covariance matrix, this question was recently studied in [AG23]
while deriving a sharp upper bound for the operator norm of the spin
covariance matrix, the CLT of overlap for the SK model was crucially used.
Investigating overlap in the low-temperature regime is a highly challenging
open problem for Ising spin glass models. In the spherical SK model, due to a
nice contour integral representation of the partition function, the
fluctuation results for the overlap have been well understood in the near-
critical temperature [NS19] and low-temperature regime [LS22]. Moreover, a
recent result by [CCM23] proved a central limit theorem for the overlap in the
Ising SK model on the so-called Nishimori line. On the other hand, oftentimes
establishing fluctuation results for the overlap to other generalized spin
glass models can be a quite challenging task, even at the high-temperature
regime. In [DW21], for the multi-species SK model, some second-moment
computation was done to compute the variance-covariance matrix for the overlap
array. However, the general moments’ computation involves many matrix
operations and can be highly technical if not impossible.

Besides the classical SK type model, the central limit theorems of overlap in
various regimes for the Hopfield model [Hop82, Tal98] were also established in
[GL99, Gen96a, Gen96b] by Gentz et.al. In both Hopefield and SK models, the
spin values are restricted as binary. The goal of this work is to extend the
fluctuation results to the non-binary spin settings.

###  1.3 Notations

  * •

We denote $\langle\cdot\rangle$ as the Gibbs average and
$\nu(\cdot):=\operatorname*{\px@BbbE}[\langle\cdot\rangle]$, where
$\operatorname*{\px@BbbE}$ denotes the average w.r.t the disorder.

  * •

Let $n$ be the number of replicas, $N$ be the number of spins (or the system
size) and $\operatorname{\mathcal{S}}:=\|\bm{\sigma}\|_{\infty}$ be the
largest value of a spin can take.

  * •

For $k,l\in[n]$, denote $R_{k,l}$ as the overlap for the configuration
$\bm{\sigma}^{k},\bm{\sigma}^{l}\in{}_{N,\operatorname{\mathcal{S}}}$.
(Setting $k=l$ gives the self-overlaps, $R_{k,k}$). We use $Q_{k,l}$ to denote
where the overlap/self-overlap concentrates. The specific values of $Q_{k,l}$
are given in Proposition 1.1.

  * •

We use $b:=\langle\sigma_{1}\rangle$ and
$\tilde{b}:=\langle\sigma_{1}\sigma_{1}\rangle$ to denote the first and second
moment for a single spin under quenched Gibbs measure, i.e., for fixed
disorder.

  * •

We use $\varepsilon_{l}$ to denote the last spin of $\bm{\sigma}^{l}$ and
$\varepsilon_{k,l}:=\varepsilon_{k}\varepsilon_{l}$. Moreover,
$R^{-}_{k,l}=R_{k,l}-\frac{1}{N}\varepsilon_{k,l}$ is the overlap without
counting contribution from the last spin.

  * •

For a positive integer $n$, denote $[n]=\\{1,2,\cdots,n\\}$ as the set of all
positive integer up to $n$. Let
$\operatorname{\mathcal{C}}_{n}:=\\{(k,l):k,l\in[n],k\leqslant l\\}$ be the
set of all replica pairs contained in $[n]$.

  * •

We say a term is of order $H$, denote it as $O(H)$, if it is asymptotically of
order $N^{-H/2}$.

###  1.4 Main result

To prepare the main results, a joint central limit theorem for the array of
overlaps and self-overlaps of the GS model, we begin by introducing some
necessary notations and existing results.

In the high-temperature regime, it is expected that the GS model is replica
symmetric in the sense that the overlap and self-overlap concentrate on some
fixed points respectively. Those concentration results first appeared in
[AC21], we recall it here.

######  Proposition 1.1 ([AC21, Proposition 2]).

There exist a $\tilde{\beta}>0$ s.t for $\beta\in[0,\tilde{\beta})$,
$h\geqslant 0$ and $\operatorname{\mathcal{D}}\in\px@BbbR$, we have

$\operatorname*{\px@BbbE}\langle(R_{1,2}-q)^{2}\rangle\leqslant\frac{16\mathcal{S}^{2}}{N},$
$\operatorname*{\px@BbbE}\langle(R_{1,1}-p)^{2}\rangle\leqslant\frac{16\mathcal{S}^{4}}{N}.$

where

$\displaystyle
p=\operatorname*{\px@BbbE}\left[\frac{\sumop\slimits@_{\gamma=1}^{\operatorname{\mathcal{S}}}\gamma^{2}\cdot
2\cosh[\gamma(\sqrt{q}\beta\eta+h)]\exp(\gamma^{2}[\operatorname{\mathcal{D}}+\frac{\beta^{2}}{2}(p-q)])}{1+\sumop\slimits@_{\gamma=1}^{\operatorname{\mathcal{S}}}\gamma^{2}\cdot
2\cosh[\gamma(\sqrt{q}\beta\eta+h)]\exp(\gamma^{2}[\operatorname{\mathcal{D}}+\frac{\beta^{2}}{2}(p-q)])}\right],$
(1.5) $\displaystyle
q=\operatorname*{\px@BbbE}\left[\frac{\sumop\slimits@_{\gamma=1}^{\operatorname{\mathcal{S}}}\gamma^{2}\cdot
2\sinh[\gamma(\sqrt{q}\beta\eta+h)]\exp(\gamma^{2}[\operatorname{\mathcal{D}}+\frac{\beta^{2}}{2}(p-q)])}{1+\sumop\slimits@_{\gamma=1}^{\operatorname{\mathcal{S}}}\gamma^{2}\cdot
2\cosh[\gamma(\sqrt{q}\beta\eta+h)]\exp(\gamma^{2}[\operatorname{\mathcal{D}}+\frac{\beta^{2}}{2}(p-q)])}\right]^{2},$
(1.6)

where $\eta\sim N(0,1)$ in the above.

Note that in high-temperature regime, the solution to the fixed point
equations (1.5) and (1.6) was also shown to be unique in [AC21]. In this
paper, for convenience, we will use the following notation to denote where the
overlap and self-overlap concentrate.

######  Definition 1.2.

Let

$Q_{k,l}:=\begin{cases}p,&\text{if }k\neq l,\\\ q,&\text{if }k=l.\end{cases}$

where $p,q$ are the solutions to (1.5), (1.6) respectively.

Our main result is a quantitative joint central limit theorem for the overlap
and self-overlap array among a set of replicas $[n]$, i.e.
$\\{R_{k,l}\\}_{(k,l)\in\operatorname{\mathcal{C}}_{n}}$, see
$\operatorname{\mathcal{C}}_{n}$ in Section 1.3. We show that the overlap and
self-overlap array behave like a family of correlated Gaussians asymptotically
$N\to\infty$. We achieve this by computing all the moments of the
(self-)overlaps
$\\{R_{k,l}-Q_{k,l}:(k,l)\in\operatorname{\mathcal{C}}_{n}\\}$.

######  Theorem 1.3.

Consider a set of nonnegative integers $\\{p(k,l)\in\px@BbbN:1\leqslant
k\leqslant l\leqslant n\\}$. Set

$p=\sumop\slimits@_{1\leqslant k\leqslant l\leqslant n}p(k,l),$

and let $\\{\eta_{k,l}:(k,l)\in\operatorname{\mathcal{C}}_{n}\\}$ be a family
of centered Gaussian with covariances

$\mathrm{Cov}(\eta_{k,l},\eta_{k^{\prime},l^{\prime}}):=\begin{cases}A_{2}^{2}\delta(|(k,l)\cap(k^{\prime},l^{\prime})|=2)+|(k,l)\cap(k^{\prime},l^{\prime})|A_{1}^{2}+A_{0}^{2},&\text{if}\
|(k,l)|=|(k^{\prime},l^{\prime})|=2,\\\
B_{1}^{2}\delta(|(k,l)\cap(k^{\prime},l^{\prime})|=1)+B_{0}^{2},&\text{if}\
|(k,l)|=|(k^{\prime},l^{\prime})|=1,\\\
C_{1}^{2}\delta(|(k,l)\cap(k^{\prime},l^{\prime})|=1)+C_{0}^{2},&\text{if}\
|(k,l)|\neq|(k^{\prime},l^{\prime})|.\end{cases}$

where the constants
$A_{2}^{2},A_{1}^{2},B_{1}^{2},C_{1}^{2},A_{0}^{2},B_{0}^{2},C_{0}^{2}$ are
defined in Lemma 3.1. There exists $\beta^{\prime}\in(0,\tilde{\beta}]$ s.t.
for $\beta<\beta^{\prime}$, we have

$N^{\frac{p}{2}}\cdot\nu\left({}_{k\leqslant
l}(R_{k,l}-Q_{k,l})^{p(k,l)}\right)=\operatorname*{\px@BbbE}\left[{}_{(k,l)\in\operatorname{\mathcal{C}}_{n}}\eta_{k,l}^{p(k,l)}\right]+O(N^{-1/2}).$

The structure of the covariance matrix is inherently related to the
decomposition of overlaps presented below in (1.7), by first rewriting each
centered overlap and self-overlap as a sum of "basis"
$\\{T_{k,l},T_{k},S_{k},T,S\\}$ as in (1.7) and expand the product. By Lemma
1.4, the only pairs of basis that are not independent are $(T_{k},S_{k})$ for
any $k\in[n]$ and $(T,S)$. The covariance structures essentially correspond to
the variance-covariance of "basis" components. In particular, in the case
$\lvert(k,l)\rvert=\lvert(k^{\prime},l^{\prime})\rvert=2$ i.e.,, $k\neq
l,k^{\prime}\neq l^{\prime}$, which corresponds to the overlap case.

Theorem 1.3 says that moments of (self-)overlap array asymptotically equals to
the corresponding moment of a correlated Gaussian. The key idea of the proof
is based on moment computations via the cavity approach, similar to [Tal11,
Chapter 1.10] for the SK model. However, in the GS model, the challenging part
is to handle self-overlap and more importantly, the correlation between
overlap and self-overlap. This makes the analysis much more involved than in
the SK case.

###  1.5 Proof outline

We briefly sketch the proof idea in this section. The first step is to
decompose the (self-)overlaps as the sum of some "basis" that are mostly
pairwise independent. This allows us to rewrite the moments of (self-)overlaps
as a homogeneous polynomial over the "basis" terms, where each term is a
moment of the basis. Our main technical Lemma (Lemma 3.1) says the moments of
the basis behave like moments of Gaussian asymptotically.

The "correct basis" we use to decompose (self-) overlap is a generalization of
those of the SK model. For overlap, let $b:=\langle\sigma_{1}\rangle$, we
define the following basis components,

$T_{k,l}:=\frac{1}{N}\sumop\slimits@_{i=1}^{N}(\sigma^{k}_{i}-b)(\sigma^{l}_{i}-b),\quad
T_{k}:=\frac{1}{N}\sumop\slimits@_{i=1}^{N}(\sigma^{k}_{i}-b)b,\quad
T:=b^{2}-q.$

For self-overlap, similarly denote
$\tilde{b}:=\langle\sigma_{1}\sigma_{1}\rangle$, and the corresponding basis
components are

$S_{l}=\frac{1}{N}\sumop\slimits@_{i=1}\sigma^{l}_{i}\sigma^{l}_{i}-\tilde{b},\
\text{and}\ S=\tilde{b}-p.$

It’s clear to see that by definition, we have the following decomposition

$\displaystyle R_{k,l}-q=T_{k,l}+T_{k}+T_{l}+T,\quad\text{and}\quad
R_{l,l}-p=S_{l}+S.$ (1.7)

The following lemma states that the terms in the above decomposition are
nearly pair-wise independent of each other under $\nu(\cdot)$.

######  Lemma 1.4.

The random variables $\\{\\{T_{k,l}\\}_{1\leqslant k<l\leqslant
n},\\{T_{k}\\}_{k\leqslant n},T,\\{S_{k}\\}_{k\leqslant n},S\\}$ are pairwise
independent under $\nu(\cdot)$ except the pairs $\\{S_{k},T_{k}\\}$ for
$k\leqslant n$ and $\\{S,T\\}$.

Now in order to show Theorem 1.3, it suffices to show that the set of basis
$\\{T_{k,l},T_{k},S_{k},T,S:k,l\in[n]\\}$ are asymptotically Gaussian. This is
the statement of our main technical lemma below.

######  Lemma 1.5 (Informal version of Lemma 3.1).

For any set of integers $\\{h(k,l):1\leqslant k<l\leqslant n\\}$ and
$\\{h(k):1\leqslant k\leqslant n\\}$ and $\\{h^{\prime}(k):1\leqslant
k\leqslant n\\}$ and $h^{\prime},h$. Let

$H:=\sumop\slimits@_{1\leqslant k<l\leqslant
n}h(k,l)+\sumop\slimits@_{1\leqslant k\leqslant
n}h(k)+\sumop\slimits@_{1\leqslant l\leqslant n}h^{\prime}(l)+h+h^{\prime},$

Then there exists a family of centered Gaussians indexed by all possible
"basis" s.t.

$\displaystyle\nu\left({}_{k,l}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right)$
$\displaystyle=\operatorname*{\px@BbbE}[{}_{k,l}g_{T_{k,l}}^{h(k,l)}{}_{k}g_{T_{k}}^{h(k)}{}_{l}g_{S_{l}}^{h^{\prime}(l)}g_{T}^{h}g_{S}^{h}]+O(N^{-\frac{H+1}{2}}).$

Moreover, for any pair of "basis" $(X,Y)$, the corresponding Gaussians
$g_{X},g_{Y}$ are independent unless
$\\{X,Y\\}\in\\{\\{S_{k},T_{k}\\}:1\leqslant k\leqslant
n\\}\cup\\{\\{S,T\\}\\}.$

Note that the family of Gaussains in Lemma 1.5 are independent except the
cases $\\{S_{k},T_{k}\\}$ and $\\{S,T\\}$. It’s easy to check that Theorem 1.3
follows from Lemma 1.5 by setting

$\eta_{k,l}:=\begin{cases}g_{T_{k,l}}+g_{T_{k}}+g_{T_{l}}+g_{T},&\text{if
}k\neq l,\\\ g_{S_{k}}+g_{S},&\text{if }k=l.\end{cases}$

The proof of Lemma 1.5 is based on the cavity method, which was used to prove
CLT of overlap in the classical SK model, The main difference here is that we
need to handle terms corresponding to self overlaps, i.e. $S_{k},S$, and
characterize the correlation between $T_{k}$ and $S_{k}$. In the rest of this
paper, we will focus on the proof of Lemma 1.5.

###  1.6 Organization of the paper

The paper is strcutured as follows. In Section 2, we first introduce the setup
for the cavity method and give some technical preliminaries. The second moment
computations for the variance-covariance estimation are carried out in Section
2.2. In Section 3, we generalize the second moment computation in Section 2.2
to general moments of the "basis" $T_{k,l},T_{k},S_{k},T,S$. The results are
formally stated in Lemma 3.1. More specifically, the inductive relations on
different "basis" are given in Section 3.1 and 3.2. Some lemmas involving
technical but repetitive computations are deferred to the Apppendix Section 4.
Finally, as we pointed out in Section 1.5, to prove Theorem 1.3, it suffices
to prove Lemma 3.1, whose proof is included in Section 3.3.

###  1.7 Acknowledgement

We thank Juspreet Singh Sandhu for the discussions in the initial stage.

##  2 Cavity method and second moment estimates

We begin with the idea of the cavity method and show how one can use it to
obtain the second-moment estimation of the "basis". The idea of the cavity
method is based on an induction from the size $N-1$ model to size $N$. We are
interested in studying the effects of this small change on the spin glass
system. The cavity idea is formally formulated into the following
interpolation scheme.

For $t\in[0,1]$, the interpolated Hamiltonian at time $t$ is given by

$\displaystyle H_{N}^{t}(\bm{\sigma})=H_{N-1}(\bm{\rho})$
$\displaystyle+\sigma_{N}\left(\sqrt{t}\cdot\frac{\beta}{\sqrt{N}}\sumop\slimits@_{i=1}^{N-1}g_{i,N}\sigma_{i}+\sqrt{1-t}\cdot\beta\eta\sqrt{q}\right)$
(2.1)
$\displaystyle\qquad\qquad+(1-t)\cdot\frac{\beta^{2}}{2}(p-q)\sigma_{N}^{2}+D\sigma_{N}^{2}+h\sigma_{N},$
(2.2)

where $\bm{\rho}:=(\sigma_{1},\ldots,\sigma_{N-1})$, $\eta\sim N(0,1)$
independent of $g_{i,j}$ and

$H_{N-1}(\bm{\rho}):=\frac{\beta}{\sqrt{N}}\sumop\slimits@_{1\leqslant
i<j\leqslant
N-1}g_{i,j}\sigma_{i}\sigma_{j}+D\sumop\slimits<EMAIL_ADDRESS>
At $t=0$, the last spin is decoupled from the original system, which brings
out a small change, heuristically known as "cavity"; at $t=1$,
$H_{N}^{1}(\bm{\sigma})$ is just the original GS Hamiltonian.

In the following, we use $\varepsilon_{l}$ to denote the last spin of $l$-th
replica, that is, $\varepsilon_{l}:=\sigma_{N}^{l}$. For a pair of replica
$k,l\in[n]$, we denote the (self-)overlap without the last spin as

$R^{-}_{k,l}:=R_{k,l}-\frac{1}{N}\varepsilon_{k,l}.$

In this paper, we use $\langle\cdot\rangle_{t}$ as the corresponding Gibbs
average at time $t$ and
$\nu_{t}(\cdot):=\operatorname*{\px@BbbE}[\langle\cdot\rangle_{t}]$. In
particular, at $t=1$,
$\nu_{1}(\cdot)=\nu(\cdot)=\operatorname*{\px@BbbE}\langle\cdot\rangle$.

###  2.1 Preliminaries

Recall that the goal is to compute the joint moments of (self-)overlaps, the
first step is the decomposition to "basis" terms given in (1.7). We begin by
proving some basic properties of the "basis".

##### Properties of "basis"

First, we show that the set of random variables
$\\{T_{k,l},T_{k},S_{k},T,S\\}$ are mostly pari-wise independent as stated in
Lemma 1.4.

See 1.4

###### Proof of Lemma 1.4.

For pairs of random variable that doesn’t involve $S_{k}$, the proof is the
same as in SK mode. We will present the proof for the pairwise independence of
$S_{l}$ and $\\{T_{k,k^{\prime}},T_{k},S_{k}:k\neq l\\}$. For
$X,Y\in\\{T_{k,l},T_{k}\\}$, $\nu(XY)=0$ follows directly from symmetry of
types of (self-)overlaps.

For pairs of term involving $T_{k,l},S_{h}$: Consider a set of constants
$\\{k,l,h\\}$ s.t. $k\neq l$ and some constant $h^{\prime}\notin\\{k,l,h\\}$.

$\displaystyle\nu(T_{k,l}S_{h})=\nu((R_{h,h}-R_{h^{\prime},h^{\prime}})T_{k,l})$

Note that there exists a replica in $\\{k,l\\}$ that does not appear in
$S_{h}$. WLOG, assume $h\neq l$, the integrate w.r.t. $\sigma^{l}$ gives

$\nu(T_{k,l}S_{h})=0$

For pair of term involving $T_{k},S_{h}$: if $k\neq h$, then by symmetry

$\nu(T_{k}S_{h})=\nu((R_{h,h}-R_{h^{\prime},h^{\prime}})T_{k})=0$

∎

To continue, we introduce another trick to express the "basis" random
variables with (self)-overlaps by introducing a new replica for each
occurrence of $b,\tilde{b}$. This trick has been used many times in [Tal11,
Chapter 1.8], and we record it here for completeness.

######  Claim 2.1.

Fix some integer $n$. For $k,l\notin[n]$ s.t. $k\neq l$,

$\nu(T_{1,2})=\nu(R_{1,2}-R_{1,l}-R_{k,2}+R_{k,l}),$
$\nu(T_{1})=\nu(R_{1,l}-R_{k,l}),\quad\nu(S_{1})=\nu(R_{1,1}-R_{k,k}),$
$\nu(T)=\nu(R_{k,l}-q),\quad\nu(S)=\nu(R_{k,k}-p).$

###### Proof.

The proof follows from the linearity of expectation. We will show a proof for
$S_{1}$, the other terms can be proved using the same technique.

$\displaystyle\nu(S_{1})$
$\displaystyle=\operatorname*{\px@BbbE}[\langle\frac{1}{N}\sumop\slimits@_{i}(\sigma_{1}^{i})^{2}-\tilde{b}\rangle]=\operatorname*{\px@BbbE}[\langle\frac{1}{N}\sumop\slimits@_{i}(\sigma_{1}^{i})^{2}-\langle\varepsilon_{k}^{2}\rangle\rangle]$
$\displaystyle=\operatorname*{\px@BbbE}[\langle\frac{1}{N}\sumop\slimits@_{i}\left((\sigma_{1}^{i})^{2}-(\sigma_{i}^{k})^{2}\right)\rangle]$
$\displaystyle=\nu(R_{1,1}-R_{k,k}).$

where the second equality is the definition of $\tilde{b}$ and the third
equality uses symmetry between sites. ∎

This implies that we can expand moments of basis as a homogeneous polynomial
of (self-)overlaps over a set of replicas.

##### Approximation of moments

We use the following definition to capture the degree of a term.

######  Definition 2.2.

For $f:{}^{\otimes n}_{N,S}\to\px@BbbR$, we say $f$ is of order $H$ if $f$ is
a product of $H$ centered overlaps or self-overlaps, $R_{k,l}-Q_{k,l}$ for
$k,l\in[n]$.

Estimating the magnitude of order $H$ functions follows a standard application
of concentration of overlaps and Hölder’s inequality. The following Lemma
generalizes the second-moment estimates of centered (self-)overlaps in
Proposition 1.1.

######  Lemma 2.3 ([Che22, Proposition 5]).

For $\beta<\tilde{\beta}$, there exist some constant $C>0$ such that for any
$k\geqslant 1$ and $l,l^{\prime}\in[n]$, we have

$\nu\left((R_{l,l^{\prime}}-Q_{l,l^{\prime}})^{2k}\right)\leqslant\left(\frac{Ck}{N}\right)^{k},$

This implies that if $f$ is an order $H$ function, then there exists a
constant $C$ that doesn’t depend on $N$ s.t.

$\nu(f)\leqslant C\cdot N^{-\frac{H}{2}}.$

We will see this type of dependency on $N$ many times below. To lighten the
notation, we overwrite the big $O$ notation and say a quantity $A=O(H)$ if

$|A|\leqslant K\cdot N^{-\frac{H}{2}},$

for some constant $K$ that does not depend on $N$. Note that the constant $K$
can depend on other parameters such as $\beta,n,\operatorname{\mathcal{S}}$.

One of the main tools we use in the cavity method is $\nu_{1}(f)\approx
v_{0}(f)+v^{\prime}_{0}(f)$. Let’s first recall the structure of
$\nu^{\prime}_{t}(f)$.

######  Lemma 2.4 ([AC21, Lemma 3]).

Let $f:{}_{N,\mathcal{S}}^{\otimes n}\to\px@BbbR$ be any function of $n$
replicas, for $t\in(0,1)$, we have

$\displaystyle 2\frac{d}{dt}\nu_{t}(f)=$
$\displaystyle\beta^{2}\sumop\slimits@_{1\leqslant k,l\leqslant
n}\nu_{t}(\varepsilon_{k}\varepsilon_{l}(R^{-}_{k,l}-Q_{k,l}))f)$
$\displaystyle-2\beta^{2}\sumop\slimits@_{1\leqslant k\leqslant n;n+1\leqslant
l\leqslant 2n}\nu_{t}(\varepsilon_{k}\varepsilon_{l}(R^{-}_{k,l}-q)f)$
$\displaystyle+\beta^{2}n(n+1)\nu_{t}(\varepsilon_{n+1,n+2}(R^{-}_{n+1,n+2}-q)f)-\beta^{2}n\nu_{t}(\varepsilon_{n+1}^{2}(R^{-}_{n+1,n+1}-p)f).$

######  Remark 2.5.

We present a convenient way of rewriting the above lemma. For $a,b\in[2n]$,
let

$\displaystyle\text{sgn}(a,b):=(-1)^{|\\{a,b\\}\cap[n]|},$ (2.3)

then we have

$\displaystyle\frac{d}{dt}\nu_{t}(f)=$
$\displaystyle\operatorname{\mathcal{R}}_{n,f}+\frac{\beta^{2}}{2}\sumop\slimits@_{a,b\in[2n]}\text{sgn}(a,b)\nu_{t}(\varepsilon_{a,b}(R^{-}_{a,b}-Q_{a,b})f),$
(2.4)

where $\operatorname{\mathcal{R}}_{n,f}$ corresponds to additional terms from
replicas independent from $f$. For $a\in[2n]$, denote $a^{\prime}=2n+a$

$\displaystyle\operatorname{\mathcal{R}}_{n,f}:=\frac{\beta^{2}}{2}\sumop\slimits@_{a,b\in[2n]}\text{sgn}(a,b)\nu_{t}(\varepsilon_{a^{\prime},b^{\prime}}(R^{-}_{a^{\prime},b^{\prime}}-Q_{a^{\prime},b^{\prime}})f).$
(2.5)

To quantify the difference between $\nu_{1}(f)$ and $\nu_{0}(f)$ by the
"degree" of $f$, we have

######  Proposition 2.6.

For $f:{}^{\otimes n}_{N,S}\to\px@BbbR$ s.t. $f$ is a product of $H$ centered
overlaps or self-overlaps, $R_{a,b}-Q_{a,b}$ for $a,b\in[n]$,

$\displaystyle|\nu_{0}(f)-\nu(f)|=O(H+1),$ (2.6)
$\displaystyle|\nu_{0}(f)+\nu_{0}^{\prime}(f)-\nu(f)|=O(H+2).$ (2.7)

The proof of Proposition 2.6 is based on the concentration of overlaps and
Hölder’s inequality. For the mean field GS spin glass model, those types of
results were already established in [AC21, Che22]. First, we have an upper
bound for $\nu_{t}(f)$.

######  Lemma 2.7 ([AC21, Lemma 4]).

For $f:{}_{N,S}^{\otimes n}\to[0,\infty)$, we have

$\nu_{t}(f)\leqslant\exp(6n^{2}\beta^{2}\operatorname{\mathcal{S}}^{4})\nu(f).$

The overlap and self-overlap concentration results already stated in
Proposition 1.1, the following presents higher order moments estimate.
Consequently, we get similar results for $R^{-}_{l,l^{\prime}}$.

###### Proof of Proposition 2.6.

To prove (2.6), note that

$|\varepsilon_{k,l}(R^{-}_{k,l}-Q_{k,l})|\leqslant|\varepsilon_{k,l}(R_{k,l}-Q_{k,l})|+\frac{\operatorname{\mathcal{S}}^{4}}{N},$

we will bound $\nu^{\prime}_{t}(f)$ using Lemma 2.4, 2.7 and Hölder’s
inequality.

$\displaystyle|\nu_{1}(f)-\nu_{0}(f)|$
$\displaystyle\leqslant\sup_{t}\Bigg{\\{}3n^{2}\operatorname{\mathcal{S}}^{2}\beta^{2}\left(\nu_{t}(|f|^{p})^{\frac{1}{p}}\nu_{t}(|R_{1,2}-Q_{1,2}|^{q})^{\frac{1}{q}}+\nu_{t}(|f|^{p})^{\frac{1}{p}}\nu_{t}(|R_{1,1}-Q_{1,1}|^{q})^{\frac{1}{q}}+\nu_{t}(|f|)\frac{\operatorname{\mathcal{S}}^{2}}{N}\right)\Bigg{\\}}$
$\displaystyle\leqslant\exp(6n^{2}\beta^{2}S^{4})3n^{2}\operatorname{\mathcal{S}}^{2}\beta^{2}\left(\nu_{1}(|f|^{p})^{\frac{1}{p}}\nu_{1}(|R^{-}_{1,2}-Q_{1,2}|^{q})^{\frac{1}{q}}+\nu_{t}(|f|^{p})^{\frac{1}{p}}\nu_{t}(|R_{1,1}-Q_{1,1}|^{q})^{\frac{1}{q}}+\nu_{1}(|f|)\frac{\operatorname{\mathcal{S}}^{2}}{N}\right),$

Since $f$ is of order $H$, by Lemma 2.3, apply Hölder’s inequality with
$p=q=2$ gives the desired result.

For (2.7), observe that

$|\nu_{1}(f)-\nu_{0}(f)-\nu^{\prime}_{0}(f)|\leqslant\sup\limits_{\begin{subarray}{c}0\leqslant
t\leqslant 1\end{subarray}}|\nu_{t}^{\prime\prime}(f)|.$

By Lemma 2.4, $\nu^{\prime\prime}(f)$ brings an additional factor of
$R^{-}_{i,j}-Q_{i,j}$. Apply the above proof on $f(R_{1,2}-p)$ and
$f(R_{1,1}-q)$ gives the desired result. ∎

Later in the proof, we will need to study terms involving (self-)overlaps
without the last spin, i.e. $R^{-}_{k,l}-Q_{k,l}$. Here we establish the
analog of the above results on $R^{-}_{k,l}-Q_{k,l}$.

######  Corollary 2.8 (of Lemma 2.3).

For $\beta<\tilde{\beta}$, there exist some constant $C^{\prime}>0$ such that
for any $k\geqslant 1$, we have

$\nu\left((R^{-}_{l,l^{\prime}}-Q_{l,l^{\prime}})^{2k}\right)\leqslant\left(\frac{C^{\prime}}{N}\right)^{k},$

###### Proof.

By Minkowski’s inequality,

$\nu\left((R^{-}_{l,l^{\prime}}-Q_{l,l^{\prime}})^{2k}\right)^{\frac{1}{2k}}\leqslant\nu\left((R_{l,l^{\prime}}-Q_{l,l^{\prime}})^{2k}\right)^{\frac{1}{2k}}+\frac{\operatorname{\mathcal{S}}^{2}}{N}\leqslant\left(\frac{C^{\prime}}{N}\right)^{\frac{1}{2}},$

where the last inequality follows from Lemma 2.3. Raise both sides to $2k$-th
power gives the desired result. ∎

######  Lemma 2.9.

Fix an integer $n$ and $H$, for each $1\leqslant v\leqslant H$, consider
$v_{1},v_{2}\in[n]$ and
$U_{v}\in\\{T_{v_{1},v_{2}},T_{v_{1}},T,S_{v_{1}},S\\}$. Let
$f:{}_{N,\operatorname{\mathcal{S}}}^{\otimes n}\to\px@BbbR$ s.t.
$f={}_{1\leqslant v\leqslant H}U_{v}$. Denote $f^{-}:={}_{v}U^{-}_{v}$. We
have

$|\nu(f)-\nu(f^{-})|=O(H+1).$

###### Proof.

Observe that each term of $f$ is of the form
$\nu(T_{1,2}),\nu(T_{1}),\nu(S_{1}),\nu(T),\nu(S)$ and can be written as
linear combination of (self-)overlaps where each occurance of
$b:=\langle\sigma_{1}\rangle$ and $\tilde{b}:=\langle\sigma_{1}^{2}\rangle$
corresponds to a new replica. For example, it’s easy to check that

$\nu(T_{1,2})=\nu(\langle\sigma_{1}-\sigma_{3},\sigma_{2}-\sigma_{4}\rangle)=\nu((R_{1,2}-q)-(R_{1,4}-q)-(R_{2,3}-q)+(R_{1,4}-q)).$

The rest of the terms can be rewritten in a similar way.

Since $f$ is the product of $H$ such terms, expanding the product shows that
it’s a sum of functions of order $H$. By Lemma 2.3 and Holder’s inequality,
$f=O(H)$. Moreover, for $I\subset[H]$, $\nu({}_{u\notin
I}U_{p(u),q(u)})=O(|I|)$. Again by Lemma 2.3

$\displaystyle|\nu(f)-\nu(f^{-})|$
$\displaystyle\leqslant\Bigg{|}\sumop\slimits@_{I\subset[H]}\nu\left({}_{v\in
I}\frac{\varepsilon(v)}{N}{}_{u\notin I}U_{p(u),q(u)}\right)\Bigg{|}=O(H+1).$

∎

The following Corollary tells us that the error to approximate $\nu(f)$ by
$\nu_{0}(f)$ is small.

######  Corollary 2.10.

Let $f:{}^{n}_{N,\operatorname{\mathcal{S}}}\to\px@BbbR$ be an order $H$
function, we have

$\nu_{0}(f^{-})=\nu(f)+O(H+1).$

###### Proof.

Applying Corollary 2.8 and equation (2.6) for $f^{-}$, combining with Lemma
2.9 gives

$\nu_{0}(f^{-})=\nu(f^{-})+O(H+1)=\nu(f)+O(H+1).$

∎

###  2.2 Variance of overlaps and self-overlaps

In this section, we compute the variance-covariance structure of a subset of
the "basis": $T_{1,2},T_{1},S_{1}$. The variance-covariance computation of
$S,T$ follows the same idea as $T_{1},S_{1}$ and we will show it as a special
case of general moments in Theorem 3.17. The main goal is to get a sense of
how to handle the additional self-overlap terms. We further note that the
following variance results hold at sufficiently high temperature, that is,
$\beta<\beta^{\prime}$ from some $\beta^{\prime}$ as in the Theorem 1.3. While
stating the results in the following context, we might not repeatedly specify
the high temperature condition ($\beta<\beta^{\prime}$).

We begin by demonstrating how the cavity method is used to compute the second
moment of the basis random variables. With some abuse of notation, let $X$ be
the expansion given in Claim 2.1 using (self-)overlaps for
$\\{T_{1,2},T_{1},S_{1}\\}$ and $\varepsilon_{X}$ be the expression by
replacing each overlap in
$R_{k,l}=\frac{1}{N}\sumop\slimits@_{i}\sigma^{k}_{i}\sigma^{l}_{i}$ $X$ by
the last spin $\varepsilon_{k}\varepsilon_{l}$. Let
$X^{-}:=X-\frac{1}{N}\varepsilon_{X}$ be the part of the basis that depends
only on the first $N-1$ spins.

Note that for $X\in T_{1,2},T_{1},S_{1}$, by symmetry of spins

$\nu(X^{2})=\nu(\varepsilon_{X}X).$

We can further decouple the last spin from the expression to get

$\displaystyle\nu(X^{2})$
$\displaystyle=\frac{1}{N}\nu(\varepsilon_{X}^{2})+\nu(\varepsilon_{X}X^{-})$
(2.8)
$\displaystyle=\frac{1}{N}\nu(\varepsilon_{X}^{2})+\nu^{\prime}_{0}(\varepsilon_{X}X^{-})+O(3),$
(2.9)

where the last equality follows from (2.7) and
$\nu_{0}(\varepsilon_{X}X^{-})=\nu_{0}(\varepsilon_{X})\nu_{0}(X^{-})=0$.
(Note that in the above expression, each copy of $X$ will introduce at least
one new replica.)

This is the starting point of the variance-covariance calculations. To
simplify notations, we record some constants corresponding to the expectation
of the last spins.

######  Definition 2.11.

We define the following constants,

  * •

From $\nu(\varepsilon_{S_{1}}^{2})$,
$D:=\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1}^{2})$,

  * •

From $\nu(\varepsilon_{S_{1}}\varepsilon_{T_{1}})$,
$E:=\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1,3})$ and
$H:=\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon^{2}_{1}))=E$,

  * •

From $\nu(\varepsilon_{T_{1}}^{2})$,
$F:=\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,3})$ and
$G:=\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,4})$,

  * •

From $\nu(\varepsilon_{T_{1,2}}^{2})$,
$A:=\nu_{0}\left((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})\varepsilon_{1,2}\right)=F-G$.

The list of constants below will occur many times in computation involving
$S_{1},T_{1},S,T$ as normalizing constants and we record them here for future
references.

  * •

$M_{1}:=1-\beta^{2}(F-3G)$,

  * •

$M_{2}:=1-\frac{\beta^{2}}{2}D$,

  * •

$M_{3}:=(1-\beta^{2}(F-G))$,

  * •

$M=M_{1}M_{2}+\beta^{4}E^{2}$.

Note that $M_{1},M_{2},M_{3},M$ are independent from $N$.

######  Remark 2.12.

When $\beta^{\prime}<\frac{1}{2\operatorname{\mathcal{S}}^{2}}$: check that
$F-G=\operatorname*{\px@BbbE}(\langle\varepsilon_{1,1}-\varepsilon_{1,2}\rangle_{0}^{2})$,
$G=\operatorname*{\px@BbbE}[b^{2}\left(\langle\varepsilon_{1,1}\rangle_{0}-\langle\varepsilon_{1}\rangle_{0}^{2}\right)]$
and
$D=\operatorname*{\px@BbbE}[\langle\varepsilon_{1}^{4}\rangle_{0}-\langle\varepsilon_{1}^{2}\rangle_{0}^{2}]$,
we have $F-G,F-3G,D\in(0,4\operatorname{\mathcal{S}}^{4}]$. For
$0\leqslant\beta\leqslant\beta^{\prime}<\frac{1}{2\operatorname{\mathcal{S}}^{2}}$,
we have $M_{1},M_{2},M_{3}>0$.

####  2.2.1 Variance of $T_{1,2}$

We begin by checking $\nu(T_{12}^{2})$. By Lemma 1.4, we should expect this
term to behave the same as in the SK model ([Tal11, Proposition 1.8.7.]). This
is indeed the case as we will show below.

######  Lemma 2.13.

For $\beta<\beta^{\prime}$, we have

$\nu(T_{12}^{2})=A_{2}^{2}+O(3),$

where

$A_{2}^{2}:=\frac{A}{N(1-\beta^{2}A)}.$

###### Proof.

Using (2.9) with $X=T_{1,2}$, we have

$\displaystyle\nu(T_{1,2}^{2})$
$\displaystyle=\nu\left(\frac{(\bm{\sigma}^{1}-\bm{b})\cdot(\bm{\sigma}^{2}-\bm{b})}{N}\frac{(\bm{\sigma}^{1}-\bm{b})\cdot(\bm{\sigma}^{2}-\bm{b})}{N}\right)$
$\displaystyle=\nu\left(\frac{(\bm{\sigma}^{1}-\bm{\sigma}^{3})\cdot(\bm{\sigma}^{2}-\bm{\sigma}^{4})}{N}\frac{(\bm{\sigma}^{1}-\bm{\sigma}^{5})\cdot(\bm{\sigma}^{2}-\bm{\sigma}^{6})}{N}\right),$

where the last equality follows from replacing each occurrence of
$\bm{b}=(\langle\sigma_{1}\rangle,\cdots,\langle\sigma_{N}\rangle)$ by a new
replica. Rewrite the above formula by expanding the inner products and
replacing each term with appropriate overlaps, we get

$\displaystyle\nu(T_{1,2}^{2})$
$\displaystyle=\nu((R_{1,2}-R_{1,4}-R_{2,3}+R_{3,4})(R_{1,2}-R_{1,6}-R_{2,5}+R_{5,6}))$
$\displaystyle=\frac{1}{N}\nu((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(\varepsilon_{1}-\varepsilon_{5})(\varepsilon_{2}-\varepsilon_{6}))+\nu((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(R^{-}_{1,2}-R^{-}_{1,6}-R^{-}_{2,5}+R^{-}_{5,6}))$
$\displaystyle=\frac{1}{N}\nu_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(\varepsilon_{1}-\varepsilon_{5})(\varepsilon_{2}-\varepsilon_{6}))+\nu_{0}^{\prime}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(R^{-}_{1,2}-R^{-}_{1,6}-R^{-}_{2,5}+R^{-}_{5,6}))+O(3),$

where the first equality follows from the symmetry of spins and isolates the
last spin from the overlaps and the second step is due to Proposition 2.6. For
the first term

$\displaystyle\nu_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(\varepsilon_{1}-\varepsilon_{5})(\varepsilon_{2}-\varepsilon_{6}))$
$\displaystyle=\nu_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})\varepsilon_{1,2})$
$\displaystyle=A.$

For the second term,

$\displaystyle\nu^{\prime}_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(R^{-}_{1,2}-R^{-}_{1,6}-R^{-}_{2,5}+R^{-}_{5,6}))$
$\displaystyle=$
$\displaystyle\frac{\beta^{2}}{2}\sumop\slimits@_{a,b}\text{sgn}(a,b)\nu_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})\varepsilon_{ab})\cdot\nu_{0}((R^{-}_{a,b}-Q_{a,b})(R^{-}_{1,2}-R^{-}_{1,6}-R^{-}_{2,5}+R^{-}_{5,6}))+\operatorname{\mathcal{R}}_{4,T_{1,2}^{2}}.$

Observe that the term involving last spins,
$\nu_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})\varepsilon_{ab})$,
is non-zero only when
$\\{a,b\\}\in\\{\\{1,2\\},\\{1,4\\},\\{2,3\\},\\{3,4\\}\\}$. Summing over all
such $a,b$, by Corollary 2.10, we have

$\nu^{\prime}_{0}((\varepsilon_{1}-\varepsilon_{3})(\varepsilon_{2}-\varepsilon_{4})(R^{-}_{1,2}-R^{-}_{1,6}-R^{-}_{2,5}+R^{-}_{5,6}))=A\nu(T_{1,2}^{2})+O(3).$

Together, the two terms give

$\nu(T_{1,2}^{2})=\frac{A}{N}+\beta^{2}A\nu(T_{1,2}^{2}).$

∎

The following relation involving $\nu(T_{1,2}^{2})$ will be useful later and
we record it here for convenience.

######  Claim 2.14.

By definition, $A=F-G$

$\displaystyle\beta^{2}A_{2}^{2}+\frac{1}{N}$
$\displaystyle:=\frac{1}{N(1-\beta^{2}(F-G))}=\frac{1}{NM_{3}}$ (2.10)

for $A_{2}^{2}$ given in Lemma 2.13.

####  2.2.2 Variance of $T_{1}$ and $S_{1}$

We will now check the variance of $S_{1},T_{1}$. Unlike in the SK model, the
basis are not independent each other any more. This hints that we should
handle $S_{1},T_{1}$ together.

######  Theorem 2.15.

For $\beta<\tilde{\beta}$, the variance of $T_{1},S_{1}$ are given by

$\nu(T_{1}^{2})=A_{1}^{2}+O(3),$

where

$A_{1}^{2}:=\frac{GM_{2}+\frac{\beta^{2}}{2}E^{2}}{M}\cdot\frac{1}{NM_{3}}=\frac{1}{2\beta^{2}N}\left(\frac{1}{M_{3}}-\frac{M_{2}}{M}\right),$

and

$\nu(S_{1}^{2})=B_{1}^{2}+O(3),$

where

$B_{1}^{2}=\frac{DM_{1}-2\beta^{2}E^{2}}{NM}=\frac{2}{N\beta^{2}}(\frac{M_{1}}{M}-1),$

The covariance is

$\nu(S_{1}T_{1})=C_{1}^{2}+O(3),$

where

$C_{1}^{2}:=\frac{E}{NM}.$

The above theorem could be viewed as a generalization of showing
$\nu(T_{1}^{2})$ in the SK model, with the addition of handling self-overlap
terms from $\nu^{\prime}_{0}(f)$ in (2.9). We will compute each part of the
theorem in Lemma 2.16, Lemma 2.19, and Lemma 2.21.

######  Lemma 2.16.

For $\beta\leqslant\beta^{\prime}$, we have

$\nu(T_{1}^{2})=A_{1}^{2}+O(3),$

where

$A_{1}^{2}:=\frac{GM_{2}+\frac{\beta^{2}}{2}EH}{M}\cdot\frac{1}{NM_{3}}=\frac{1}{2\beta^{2}N}\left(\frac{1}{M_{3}}-\frac{M_{2}}{M}\right).$

To prove this, we will use the following lemma to characterize the relation
between $\nu(T_{1}^{2})$ and $\nu(T_{1}S_{1})$.

######  Lemma 2.17.

We have

$\displaystyle(1-\beta^{2}(F-3G))\nu(T_{1}^{2})=G\left(\frac{1}{N}+\beta^{2}A_{2}^{2}\right)+\frac{\beta^{2}}{2}H\nu(S_{1}T_{1})+O(3),$
(2.11)

and

$\displaystyle\frac{(1-\frac{\beta^{2}}{2}D)}{E}\nu(S_{1}T_{1})=(\frac{1}{N}+\beta^{2}A_{2}^{2})-2\beta^{2}\nu_{0}(T_{1}^{2})+O(3).$
(2.12)

Note that Lemma 2.16 follows immediately from Lemma 2.17.

###### Proof of Lemma 2.16.

Plug (2.12) into (2.11) and rearrange gives

$\displaystyle\left(1-\beta^{2}(F-3G)+\frac{\beta^{4}EH}{1-\frac{\beta^{2}}{2}D}\right)\nu(T_{1}^{2})$
$\displaystyle=\left(G+\frac{\beta^{2}EH}{2(1-\frac{\beta^{2}}{2}D)}\right)\left(\frac{1}{N}+\beta^{2}A_{2}^{2})\right)+O(3)$
(2.13)
$\displaystyle\stackrel{{\scriptstyle(\ref{claim:T12})}}{{=}}\left(G+\frac{\beta^{2}EH}{2(1-\frac{\beta^{2}}{2}D)}\right)\frac{1}{NM_{3}}+O(3).$
(2.14)

∎

We now turn to the proof of Lemma 2.17.

###### Proof of Lemma 2.17.

Using (2.9) with $X=T_{1}$, we can rewrite $\nu(T_{1}^{2})$ by introducing a
new replica for each occurrence of $b$ and get

$\displaystyle\nu(T_{1}^{2})$
$\displaystyle=\nu((R_{1,3}-R_{2,3})(R_{1,5}-R_{4,5}))$ (2.15)
$\displaystyle=\frac{1}{N}\nu((\varepsilon_{1,3}-\varepsilon_{2,3})(\varepsilon_{15}-\varepsilon_{4,5}))+\nu^{\prime}_{0}((\varepsilon_{13}-\varepsilon_{23})(R^{-}_{1,5}-R^{-}_{4,5}))+O(3).$
(2.16)

For the first term, not that by symmetry,
$\nu((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{4,5})=0$. Thus we have

To expand the second term, we use (2.4) with $N=5$ gives

$\displaystyle\nu^{\prime}_{0}((\varepsilon_{13}-\varepsilon_{23})(R^{-}_{1,5}-R^{-}_{4,5}))=$
$\displaystyle\frac{\beta^{2}}{2}\sumop\slimits@_{a,b\in[10]}\text{sgn}(a,b)\nu_{0}((\varepsilon_{13}-\varepsilon_{23})\varepsilon_{ab})\nu_{0}((R^{-}_{a,b}-\mu_{a,b})(R^{-}_{1,5}-R^{-}_{4,5}))+R_{5,T_{1}^{2}}.$

Many terms will vanish due to
$\nu_{0}((\varepsilon_{13}-\varepsilon_{23})\varepsilon_{ab})=0$. We will see
that the non-vanishing pairs of replica $(a,b)$ introduce some structures that
correspond to either $T_{1}$ or $S_{1}$.

To capture which pair of $(a,b)$ having
$\nu_{0}((\varepsilon_{13}-\varepsilon_{23})\varepsilon_{ab})\neq 0$, let’s
expand the product into two terms. Observe the value of
$\nu_{0}(\varepsilon_{13}\varepsilon_{ab})$ is characterized by the type of
multiset $\\{1,3,a,b\\}$ and that the replica $2$ in
$\nu_{0}(\varepsilon_{23}\varepsilon_{ab})$ is equivalent to replica $1$ in
$\nu_{0}((\varepsilon_{13}\varepsilon_{ab})$. Thus we have that

$\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{a,b})\neq
0\iff|\\{a,b\\}\cap\\{1,2\\}|=1.$

What’s left to do is to check
$\nu_{0}((R^{-}_{a,b}-\mu_{a,b})(R^{-}_{1,5}-R^{-}_{4,5}))$ for such pair
$(a,b)\in\operatorname{\mathcal{C}}_{10}$

  * •

If $a=b$: in this case $a\in\\{1,2\\}$. Combine the two cases gives

$\frac{1}{2}\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,1})\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,5}-R^{-}_{4,5}))\stackrel{{\scriptstyle\ref{cor:
last spin}}}{{=}}\frac{1}{2}H\nu(S_{1}T_{1})+O(3).$

  * •

For $\\{a,b\\}\in\\{\\{1,3\\},\\{2,3\\}\\}$, we have

$\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,3})\nu_{0}((R^{-}_{1,3}-R^{-}_{2,3})(R^{-}_{1,5}-R^{-}_{4,5}))\stackrel{{\scriptstyle\ref{cor:
last spin}}}{{=}}F\nu(T_{1}^{2})+O(3).$

  * •

Now we count the case when $a\in\\{1,2\\}$, $b\notin\\{1,2,3\\}$. Here is when
the rectangles appear. Recall that for each of the $5$ replicas, we introduce
a new replica. Let’s index them with $\\{k+5:k\leqslant 5\\}$. Gather terms
for $b\in\\{4,9\\}$ (equivalently $\\{5,10\\}$)

$\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,4})\nu_{0}((R^{-}_{1,4}-R^{-}_{2,4}-R^{-}_{1,9}+R^{-}_{2,9})(R^{-}_{1,5}-R^{-}_{4,5})).$

Using (2.6) and Lemma 2.9, we can rewrite the second term with
$T_{k,l},T_{k},T_{l}$ involving those new replicas,

$\nu_{0}((R^{-}_{1,4}-R^{-}_{2,4}-R^{-}_{1,9}+R^{-}_{2,9})(R^{-}_{1,5}-R^{-}_{4,5}))=\nu((T_{1,4}-T_{2,4}-T_{1,9}+T_{2,9})(T_{1,5}-T_{4,5}+T_{1}-T_{4})).$

We see that there are no even moments of $T_{k,l}$ here, thus this term is
$O(3)$ by Lemma 2.13. For $b\in\\{5,10\\}$,

$\displaystyle\nu_{0}((R^{-}_{1,5}-R^{-}_{2,5}-R^{-}_{1,10}+R^{-}_{2,10})(R^{-}_{1,5}-R^{-}_{4,5}))$
$\displaystyle\stackrel{{\scriptstyle\ref{cor: last spin}}}{{=}}$
$\displaystyle\nu((T_{1,5}-T_{2,5}-T_{1,10}+T_{2,10})(T_{1,5}-T_{4,5}+T_{1}-T_{4}))=\nu(T_{1,5}^{2}).$

Thus the total contribution from this case is

$G\nu(T_{1,5}^{2})+O(3).$

  * •

Now we left with the cases $a\in\\{1,2\\}$ and $b\in\\{6,7,8\\}$ which are the
new replica corresponds to $\\{1,2,3\\}$. For those terms, WLOG,

$3\nu_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})\varepsilon_{1,4})\nu_{0}((-R^{-}_{1,6}+R^{-}_{2,6})(R^{-}_{1,5}-R^{-}_{4,5})).$

Note that since the new replica is not used by our second copy of $T_{1}$,
namely $R^{-}_{1,5}-R^{-}_{4,5}$, this term can be written as

$-3G\nu(T_{1}^{2})+O(3).$

Combining all the terms for the second term,

$\displaystyle\nu^{\prime}_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})(R^{-}_{1,5}-R^{-}_{4,5}))=$
$\displaystyle\frac{\beta^{2}}{2}H\nu(S_{1}T_{1})+\beta^{2}F\nu(T_{1}^{2})$
$\displaystyle+\beta^{2}G\nu(T_{12}^{2})-3\beta^{2}G\nu(T_{1}^{2})+O(3).$

Plugging this back into (2.15), we have

$\displaystyle\nu(T_{1}^{2})=$
$\displaystyle\frac{1}{N}\nu((\varepsilon_{1,3}-\varepsilon_{2,3})((\varepsilon_{1,5}-\varepsilon_{4,5})))+\nu((\varepsilon_{1,3}-\varepsilon_{2,3})(R^{-}_{1,5}-R^{-}_{4,5})),$
$\displaystyle=$ $\displaystyle
G(\frac{1}{N}+\beta^{2}\nu(T_{1,2}^{2}))+\frac{\beta^{2}}{2}H\nu(S_{1}T_{1})+\beta^{2}\left[F-3G\right]\nu(T_{1}^{2})+O(3).\
$

Rearranging gives (2.11),

$(1-\beta^{2}(F-3G))\nu(T_{1}^{2})=G(\frac{1}{N}+\beta^{2}\nu(T_{12}^{2}))+\frac{\beta^{2}}{2}H\nu(S_{1}T_{1})+O(3).$

Plug in $\nu(T_{12}^{2})=A_{2}^{2}+O(3)$ gives (2.11).

######  Remark 2.18.

In SK, the mixed term $S_{1}T_{1}$ vanishes. If we look at the constant for
$T_{1}^{2}$,

$F=\nu_{0}((\varepsilon_{13}-\varepsilon_{23})\varepsilon_{13})=b(2)-b(1),$
$G=\nu_{0}((\varepsilon_{13}-\varepsilon_{23})\varepsilon_{14})=b(1)-b(0).$

Combining them, we get back the original constants $1-4q+3\hat{q}$, which is
one of the "eigenvalues" and thus we get back the second moment of $T_{1}$
(see the equation (1.259) in [Tal11]).

##### A way of writing covariance of $S_{1},T_{1}$

To handle the occurrence of $\nu(S_{1}T_{1})$ in the final expression, we will
use the symmetry of spin to write

$\displaystyle\nu(S_{1}T_{1})=\nu((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})(R_{1,4}-R_{3,4}))=\frac{1}{N}\nu((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1,3})+\nu((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})(R^{-}_{1,4}-R^{-}_{3,4})).$
(2.17)

This type of expansion helps reduce the moment of $S_{1}$. As shown above in
$\nu(T_{1}^{2})$, to control the second term, it is enough to look at
$\nu^{\prime}_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})(R^{-}_{14}-R^{-}_{34}))$.

$\displaystyle\nu^{\prime}_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})(R^{-}_{1,4}-R^{-}_{3,4}))=$
$\displaystyle\frac{\beta^{2}}{2}\sumop\slimits@_{a,b}\text{sgn}(a,b)\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-\mu_{a,b})(R^{-}_{1,4}-R^{-}_{3,4}))$
$\displaystyle+R_{4,S_{1}T_{1}}.$

Observe that

$\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{a,b})\neq
0\iff|\\{a,b\\}\cap\\{1,2\\}|=1,$

Let’s iterate over those pairs $(a,b)$:

  * •

For $a=b$: either $a=b=1$ or $a=b=2$,

$\frac{1}{2}\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1}^{2})\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,4}-R^{-}_{3,4}))\stackrel{{\scriptstyle(\ref{lemma:last
spin})}}{{=}}\frac{1}{2}D\nu(S_{1}T_{1})+O(3).$

  * •

For $|\\{a,b\\}\cap\\{1,2\\}|=1$, assume $a\in\\{1,2\\}$ and
$b\in\\{3,4,\cdots 8\\}$. As shown above, for $b\in\\{3,7\\}$ or $\\{4,8\\}$,
we have

$\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1,3})\nu_{0}((R^{-}_{1,3}-R^{-}_{2,3}+R^{-}_{1,7}-R^{-}_{2,7})(R^{-}_{1,4}-R^{-}_{3,4}))=O(3),$

and

$\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{13})\nu_{0}((R^{-}_{1,4}-R^{-}_{2,4}+R^{-}_{1,8}-R^{-}_{2,8})(R^{-}_{1,4}-R^{-}_{3,4}))=E\nu(T_{1,4}^{2})+O(3).$

For $b\in\\{5,6\\}$, we have

$-\nu_{0}((\varepsilon_{1}^{2}-\varepsilon_{2}^{2})\varepsilon_{1,3})\nu_{0}((R^{-}_{1,5}-R^{-}_{2,5})(R^{-}_{1,4}-R^{-}_{2,4}))=-E\nu(T_{1}^{2})+O(3).$

Thus

$\displaystyle\nu^{\prime}_{0}(S_{1}T_{1})=$
$\displaystyle\frac{\beta^{2}}{2}D\nu(S_{1}T_{1})+\beta^{2}E\nu(T_{1,2}^{2})-2\beta^{2}E\nu(T_{1}^{2})+O(3).$

Plugging this back to the equation (2.17) gives (2.12)

$\displaystyle(1-\frac{\beta^{2}}{2}D)\nu(S_{1}T_{1})=$
$\displaystyle(\frac{1}{N}+\beta^{2}\nu(T_{12}^{2}))E-2\beta^{2}E\nu(T_{1}^{2})+O(3),$
(2.18)

Plugging in $\nu(T_{12}^{2})=A_{2}^{2}+O(3)$ gives (2.12). ∎

######  Lemma 2.19.

For $\beta\leqslant\beta^{\prime}$, we have

$\nu(S_{1}^{2})=B_{1}^{2}+O(3),$

where

$B_{1}^{2}=\frac{DM_{1}-2\beta^{2}E^{2}}{NM}=\frac{2}{N\beta^{2}}\left(\frac{M_{1}}{M}-1\right)$

To prove the Lemma 2.19, we need to show the following two relations.

######  Lemma 2.20.

We have

$\left(1-\frac{\beta^{2}}{2}D\right)\nu(S_{1}^{2})=\frac{1}{N}D-2\beta^{2}E\cdot\nu(S_{1}T_{1})+O(3),$
$(1-\beta^{2}(F-3G))\nu(S_{1}T_{1})=\frac{1}{N}E+\frac{\beta^{2}}{2}H\cdot\nu(S_{1}^{2})+O(3).$

###### Proof of Lemma 2.19.

As in the $\nu(T_{1}^{2})$ case, Lemma 2.19 follows from combining the above
two relations and the definition of $M_{1},M$.

$\left(\frac{(1-\frac{\beta^{2}}{2}D)(1-\beta^{2}(F-3G))+\beta^{4}EH}{(1-\beta^{2}(F-3G))}\right)\nu(S_{1}^{2})=\frac{D((1-\beta^{2}(F-3G)))-2\beta^{2}E^{2}}{N(1-\beta^{2}(F-3G))}+O(3).$

Rearrange gives for $B_{1}^{2}=\frac{2}{N\beta^{2}}(\frac{M_{1}}{M}-1)$,
$\nu(S_{1}^{2})=B_{1}^{2}+O(3)$.

∎

###### Proof of Lemma 2.20.

The proof is similar to the previous case. Denote
$\varepsilon_{k,k}=(\sigma^{k}_{N})^{2}$,

$\displaystyle\nu(S_{1}^{2})$
$\displaystyle=\nu((R_{1,1}-R_{2,2})(R_{1,1}-R_{3,3})),$ (2.19)
$\displaystyle=\frac{1}{N}\nu((\varepsilon_{1,1}-\varepsilon_{2,2})(\varepsilon_{1,1}-\varepsilon_{3,3}))+\nu((\varepsilon_{1,1}-\varepsilon_{2,2})(R^{-}_{1,1}-R^{-}_{3,3})),$
(2.20)
$\displaystyle=\frac{1}{N}\nu((\varepsilon_{1,1}-\varepsilon_{2,2})\varepsilon_{1,1})+\nu((\varepsilon_{1,1}-\varepsilon_{2,2})(R^{-}_{1,1}-R^{-}_{3,3})).$
(2.21)

To control the second term, observe that by (2.7), and
$\nu_{0}((\varepsilon_{1,1}-\varepsilon_{2,2})(R^{-}_{1,1}-R^{-}_{3,3}))=0$,

$\nu((\varepsilon_{1,1}-\varepsilon_{2,2})(R^{-}_{1,1}-R^{-}_{3,3}))=\nu^{\prime}_{0}((\varepsilon_{1,1}-\varepsilon_{2,2})(R^{-}_{1,1}-R^{-}_{3,3}))+O(3).$

By (2.4)

$\displaystyle\nu^{\prime}_{0}((\varepsilon_{11}-\varepsilon_{22})(R^{-}_{1,1}-R^{-}_{3,3}))=$
$\displaystyle\frac{\beta^{2}}{2}\sumop\slimits@_{a,b}\text{sgn}(a,b)\nu_{0}((\varepsilon_{1,1}-\varepsilon_{2,2})\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-Q_{a,b})(R^{-}_{1,1}-R^{-}_{3,3}))$
$\displaystyle+R_{3,S_{1}^{2}}.$

Note that

$\nu_{0}((\varepsilon_{1,1}-\varepsilon_{2,2})\varepsilon_{a,b})\neq
0\iff|\\{a,b\\}\cap\\{1,2\\}|=1.$

To count the contribution for all such $a,b$,

  * •

For $a=b$, combine the contribution of two terms gives

$\frac{\beta^{2}}{2}D\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,1}-R^{-}_{3,3}))\stackrel{{\scriptstyle(\ref{lemma:last
spin}),\eqref{eq:1st approx}}}{{=}}\frac{\beta^{2}}{2}D\nu(S_{1}^{2})+O(3).$

  * •

If $a\neq b$, WLOG, suppose $a\in\\{1,2\\}$ and $b\in\\{3,6\\}$: by Lemma 2.9

$\beta^{2}E\nu_{0}((R^{-}_{13}-R^{-}_{2,3}-R^{-}_{1,6}+R^{-}_{2,6})(R^{-}_{1,1}-R^{-}_{3,3}))=\beta^{2}E\nu_{0}((R_{13}-R_{2,3}-R_{1,6}+R_{2,6})(R_{1,1}-R_{3,3}))+O(3).$

Rewrite the last part using 1.2,

$\displaystyle\beta^{2}E\nu_{0}((R_{13}-R_{2,3}-R_{1,6}+R_{2,6})(R_{1,1}-R_{3,3}))+O(3),$
$\displaystyle=$
$\displaystyle\beta^{2}E\nu_{0}((T_{13}-T_{2,3}-T_{1,6}+T_{2,6})(S_{1}-S_{3})+O(3),$
$\displaystyle=$ $\displaystyle O(3).$

  * •

For $a\in\\{1,2\\}$ and $b\in\\{4,5\\}$, combine the two terms gives

$-2\beta^{2}E\nu_{0}((R^{-}_{1,4}-R^{-}_{2,4})(R^{-}_{1,1}-R^{-}_{3,3}))=-2\beta^{2}E\nu(S_{1}T_{1})+O(3).$

Plug this back in (2.19)

$\displaystyle\nu(S_{1}^{2})$
$\displaystyle=\frac{1}{N}\nu((\varepsilon_{1,1}-\varepsilon_{2,2})\varepsilon_{1,1})+\frac{\beta^{2}}{2}D\nu(S_{1}^{2})-2\beta^{2}E\nu(S_{1}T_{1})+O(3),$
(2.22)
$\displaystyle=\frac{1}{N}D+\frac{\beta^{2}}{2}D\nu(S_{1}^{2})-2\beta^{2}E\nu(S_{1}T_{1})+O(3).$
(2.23)

##### Alternative way of writing $\nu(S_{1}T_{1})$

We’ve seen one way of decomposing $S_{1}T_{1}$ in lemma 2.17, which reduces
the moment of $S_{1}$. While we may directly apply (2.12) here, we show
another way of decomposing $S_{1}T_{1}$ by reducing the moment of $T_{1}$, as
it will be helpful in the general case. The idea is same

$\displaystyle\nu(S_{1}T_{1})=\nu((R_{1,1}-R_{2,2})(\varepsilon_{1,4}-\varepsilon_{3,4}))=\frac{1}{N}\nu((\varepsilon_{1,1}-\varepsilon_{2,2})(\varepsilon_{1,4}-\varepsilon_{3,4}))+\nu((R^{-}_{1,1}-R^{-}_{2,2})(\varepsilon_{1,4}-\varepsilon_{3,4}))$

We then rewrite the second term as before

$\displaystyle\nu((R^{-}_{1,1}-R^{-}_{2,2})(\varepsilon_{1,4}-\varepsilon_{3,4}))=\nu_{0}^{\prime}((R^{-}_{1,1}-R^{-}_{2,2})(\varepsilon_{1,4}-\varepsilon_{3,4}))+O(3)$
$\displaystyle=\frac{\beta^{2}}{2}\sumop\slimits@_{a,b}\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{a,b}-Q_{a,b}))\nu_{0}(\varepsilon_{a,b}(\varepsilon_{1,4}-\varepsilon_{3,4}))$

As shown in Lemma 2.17,

$\nu_{0}((\varepsilon_{1,4}-\varepsilon_{3,4})\varepsilon_{a,b})\neq
0\iff|\\{a,b\\}\cap\\{1,3\\}|=1$

Let’s go over all cases of such size two subsets:

  * •

If $a=b$: this term gives

$\frac{\beta^{2}}{2}H\nu_{0}((R^{-}_{1,1}-R^{-}_{3,3})(R^{-}_{1,1}-R^{-}_{2,2}))\stackrel{{\scriptstyle(\ref{lemma:last
spin}),\eqref{eq:1st approx}}}{{=}}\frac{\beta^{2}}{2}H\nu(S_{1}^{2})+O(3)$

  * •

For $\\{a,b\\}\in\\{\\{1,4\\},\\{3,4\\}\\}$, we have

$\beta^{2}F\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,4}-R^{-}_{3,4}))\stackrel{{\scriptstyle(\ref{lemma:last
spin}),\eqref{eq:1st approx}}}{{=}}\beta^{2}F\nu(S_{1}T_{1})+O(3)$

  * •

Now we count the case when $a\in\\{1,3\\}$, $b\notin\\{1,3,4\\}$. Gather terms
for $b\in\\{2,6\\}$ and rewrite

$\displaystyle\beta^{2}G\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,2}-R^{-}_{3,2}-R^{-}_{1,6}+R^{-}_{3,6}))$
$\displaystyle\stackrel{{\scriptstyle(\ref{lemma:last spin}),\eqref{eq:1st
approx}}}{{=}}\beta^{2}G\nu_{0}((S_{1}-S_{2})(T_{1,2}-T_{3,2}-T_{1,6}+T_{3,6}))$
$\displaystyle=O(3)$

  * •

Now we are left with $a\in\\{1,3\\}$ and $b\in\\{5,7,8\\}$

$-3\beta^{2}G\nu_{0}((R^{-}_{1,1}-R^{-}_{2,2})(R^{-}_{1,5}-R^{-}_{3,5}))\stackrel{{\scriptstyle(\ref{lemma:last
spin}),\eqref{eq:1st approx}}}{{=}}-3\beta^{2}G\nu(S_{1}T_{1})+O(3)$

Combine we get

$\nu(S_{1}T_{1})=\frac{1}{N}E+\frac{\beta^{2}}{2}H\nu(S_{1}^{2})+\beta^{2}(F-3G)\nu(S_{1}T_{1})+O(3)$

∎

####  2.2.3 Covariance: $S_{1}T_{1}$ term

######  Lemma 2.21.

For $\beta\leqslant\beta^{\prime}$, we have

$\nu(S_{1}T_{1})=C_{1}^{2}+O(3),$

where

$C_{1}^{2}:=\frac{E}{NM}.$

###### Proof.

Note that one can deduce $\nu(S_{1}T_{1})$ from both 2.17 and 2.20. From lemma
2.17, we get

$\displaystyle\nu(S_{1}T_{1})$
$\displaystyle=\frac{E}{M_{2}}\left[\frac{1}{NM_{3}}-2\beta^{2}A_{1}^{2}\right]+O(3)$
$\displaystyle=\frac{E}{M_{2}}\left(\frac{1}{NM_{3}}-\frac{1}{N}\left(\frac{1}{M_{3}}-\frac{M_{2}}{M}\right)\right)+O(3)=\frac{E}{MN}+O(3).$

From lemma 2.20

$\displaystyle\nu(S_{1}T_{1})$
$\displaystyle=\frac{1}{M_{1}}\left(\frac{E}{N}+\frac{\beta^{2}}{2}HB_{1}^{2}\right)+O(3)=\frac{E}{NM}+O(3),$

where the last equality follows from $E=H$. ∎

##  3 General moments computation

In Section 2.2, we obtained the variance and covariance:
$\nu(T_{1,2}^{2}),\nu(T_{1}^{2}),\nu(S_{1}^{2})$, $\nu(S_{1}T_{1})$ by
rewriting moments with lower order terms. In this section, we extend this idea
to general moments of $T_{1,2},T_{1},S_{1},T,S$.

######  Lemma 3.1 (Formal version of Lemma 1.5).

Fix an integer $n$, consider the following sets of integers
$\\{h(k,l):1\leqslant k<l\leqslant n\\}$, $\\{h(k):1\leqslant k\leqslant n\\}$
and $\\{h^{\prime}(k):1\leqslant k\leqslant n\\}$ and $h^{\prime},h$. Let

$H:=\sumop\slimits@_{1\leqslant k<l\leqslant
n}h(k,l)+\sumop\slimits@_{1\leqslant k\leqslant
n}h(k)+\sumop\slimits@_{1\leqslant l\leqslant n}h^{\prime}(l)+h+h^{\prime},$

let $g_{X}$ be a centered Gaussians vector where the index $X$ belongs to

$\\{T_{k,l},T_{k},S_{k},T,S:1\leqslant k<l\leqslant n\\},$

and its covariance matrix is

$\mathrm{Cov}(g_{X},g_{Y})=\begin{cases}A_{2}^{2},&\text{ if }X=Y=T_{k,l},\\\
A_{1}^{2},&\text{ if }X=Y=T_{k},\\\ A_{0}^{2},&\text{ if }X=Y=T,\\\
B_{1}^{2},&\text{ if }X=Y=S_{k},\\\ B_{0}^{2},&\text{ if }X=Y=S,\\\
C_{1}^{2},&\text{ if }\\{X,Y\\}=\\{T_{k},S_{k}\\},\\\ C_{0}^{2},&\text{ if
}\\{X,Y\\}=\\{T,S\\}.\\\ \end{cases}$

then for $\beta<\beta^{\prime}\leqslant\tilde{\beta}$, we have

$\displaystyle\nu\left({}_{k,l}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right)$
$\displaystyle=\operatorname*{\px@BbbE}[{}_{k,l}g_{T_{k,l}}^{h(k,l)}{}_{k}g_{T_{k}}^{h(k)}{}_{l}g_{S_{l}}^{h^{\prime}(l)}g_{T}^{h}g_{S}^{h}]+O(H+1).$

Similar to the proof of CLT in SK model, the proof for Lemma 3.1 consists of
three parts, first we separate any $T_{k,l}$ terms s.t. $h(k,l)>0$ from the
mixed moments, then $(T_{k},S_{k})$ terms and then the $(T,S)$ term. This is
based on the Lemma 1.4, which states that the set of random variables is
pairwise independent besides $(T_{k},S_{k})$ for some $k\in[n]$ and $(T,S)$.
Thus, we expect the mixed moments to be decomposed into

${}_{k,l}\nu\left(T_{k,l}^{h(k,l)}\right)\cdot{}_{k}\nu\left(T_{k}^{h(k)}S_{k}^{h^{\prime}(k)}\right)\cdot\nu(T^{h}S^{h^{\prime}}).$

Therefore, it is then enough to characterize the moments of the form:
$T_{k,l}^{h(k,l)},T_{k}^{h(k)}S_{k}^{h^{\prime}(k)},T^{h}S^{h^{\prime}}$. The
formal statements can be found in Theorem 3.2, 3.5 and 3.11.

Before we start the proofs, we will introduce the necessary notations to index
each term within the mixed moments. Let’s first rewrite each term using
(self-)overlaps by the expansion given in Claim 2.1. For $v\in[H]$, denote
$V_{v}=\\{v_{1},v_{2},\cdots\\}$ as the set of replicas appeared in the
corresponding term $U_{v}$. Define $U_{v}$ as

$U_{v}:=\begin{cases}R_{v_{1},v_{2}}-R_{v_{1},v_{4}}-R_{v_{3},v_{2}}+R_{v_{3},v_{4}},&\text{
if }v\text{-th term corresponds to }T_{k,l},\\\
R_{v_{1},v_{3}}-R_{v_{2},v_{3}},&\text{ if }v\text{-th term corresponds to
}T_{k},\\\ R_{v_{1},v_{1}}-R_{v_{2},v_{2}},&\text{ if }v\text{-th term
corresponds to }S_{l},\\\ R_{v_{1},v_{2}}-p,&\text{ if }v\text{-th term
corresponds to }T,\\\ R_{v_{1},v_{1}}-q,&\text{ if }v\text{-th term
corresponds to }S.\end{cases}$

Then the general moments can be rewritten as

$\displaystyle\nu\left({}_{k,l}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right)=\nu({}_{v\geqslant
1}U_{v}).$ (3.1)

By symmetry of spins, we can replace one of $U_{v}$ by the same expression on
the last spin. To do this, let’s define the following notation: For $v\in[H]$,
let

$\varepsilon(v):=\begin{cases}\varepsilon_{v_{1},v_{2}}-\varepsilon_{v_{1},v_{4}}-\varepsilon_{v_{3},v_{2}}+\varepsilon_{v_{3},v_{4}},&\text{
if }v\text{-th term corresponds to }T_{k,l},\\\
\varepsilon_{v_{1},v_{3}}-\varepsilon_{v_{2},v_{3}},&\text{ if }v\text{-th
term corresponds to }T_{k},\\\
\varepsilon_{v_{1},v_{1}}-\varepsilon_{v_{2},v_{2}},&\text{ if }v\text{-th
term corresponds to }S_{l},\\\ \varepsilon_{v_{1},v_{2}}-p,&\text{ if
}v\text{-th term corresponds to }T,\\\ \varepsilon_{v_{1},v_{1}}-q,&\text{ if
}v\text{-th term corresponds to }S.\end{cases}$

Finally, define $U^{-}_{v}$ as the part of $U_{v}$ that doesn’t depend on the
last spin

$U^{-}_{v}:=U_{v}-\frac{1}{N}\varepsilon(v).$

Finally, following the cavity method, one should try to separate as many parts
of the expression that depend on the last spin as possible. To this end, let’s
further decompose (3.1) as

$\displaystyle\nu\left({}_{k,l}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right)$
$\displaystyle=\nu({}_{v\geqslant 1}U_{v})$ (3.2)
$\displaystyle=\nu(\varepsilon(1){}_{v>1}U^{-}_{v})+\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu(\varepsilon(1)\varepsilon(u){}_{v\neq 1,u}U^{-}_{v})+O(H+1).$ (3.3)

###  3.1 Induction on $T_{k,l}$

We first generalize the result in Lemma 2.13 to show that $T_{1,2}$ behaves
like independent Gaussian w.r.t. other basis terms.

######  Theorem 3.2.

For $\beta<\beta^{\prime}$, we have

$\nu({}_{(k,l)}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{k}S_{k}^{h^{\prime}(k)}S^{h^{\prime}})={}_{(k,l)}a(h(k,l))A_{2}^{h(k,l)}\nu({}_{k}T_{k}^{h(k)}T^{h}{}_{k}S_{k}^{h^{\prime}(k)}S^{h^{\prime}})+O(H+1),$

where $a(x)=\operatorname*{\px@BbbE}[g^{x}]$ with $g\sim N(0,1)$.

The proof of this theorem is the same as its analog in the SK model. We
include the proof for completeness.

###### Proof.

The proof goes by inducting on $\sumop\slimits@_{k,l}h(k,l)$. WLOG, we assume
that $h(1,2)\geqslant 1$ and reduce the moment of $T_{1,2}$. For the sake of
simplicity, let’s define a function $g_{1,2}(x)$ that tracks the moment of
$T_{1,2}$ s.t.

$g_{1,2}(x)=\begin{cases}T_{1,2}^{x}{}_{(k,l)\nequiv(1,2)}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}},&\text{if
}h(1,2)\geqslant 0,\\\ 0,&\text{ otherwise}.\end{cases}$

Assume that $U_{1}$ corresponds to a copy of $T_{1,2}$. Using (3.1), we have

$\displaystyle\nu(g(h(1,2)))=\nu(\varepsilon(1){}_{v>1}U^{-}_{v})+\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu(\varepsilon(1)\varepsilon(u){}_{v\neq 1,u}U^{-}_{v})+O(H+1),$ (3.4)

where

$\varepsilon(1)=\varepsilon_{1_{1},1_{2}}-\varepsilon_{1_{1},1_{4}}-\varepsilon_{1_{3},1_{2}}+\varepsilon_{1_{3},1_{4}}.$

The second term is again approximated by $\nu^{\prime}_{0}(\cdot)$ using
(2.7).

######  Lemma 3.3.

For $\beta<\beta^{\prime}$, suppose $h(1,2)\geqslant 1$ and $U_{1}$
corresponds to a copy of $T_{1,2}$, we have

$\nu^{\prime}_{0}(\varepsilon(1){}_{v>1}U^{-}_{v})=\beta^{2}A\cdot\nu(g_{1,2}(h(1,2)))+O(H+1).$

The proof of the above lemma is essentially the same as in the SK model; we
include it in the appendix for completeness. For the first term, by (2.6),

$\displaystyle\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu(\varepsilon(1)\varepsilon(u){}_{v\neq 1,u}U^{-}_{v})$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu_{0}(\varepsilon(1)\varepsilon(u){}_{v\neq 1,u}U^{-}_{v})+O(H+1),$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu_{0}(\varepsilon(1)\varepsilon(u))\nu_{0}({}_{v\neq
1,u}U^{-}_{v})+O(H+1).$

Note that following a similar arguement as in Lemma 1.4,
$\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\neq 0$ only when $a,b$ appears in
the expression of $\varepsilon(1)$. However, by construction, $1_{2},1_{3}$ do
not appear in any other terms besides $U_{1}$, thus the only possible pair of
replicas that appears in $U_{1}$ that also appears in other terms are when
$(u_{1},u_{2})\equiv(1,2)$.

$\displaystyle\nu_{0}(\varepsilon(1)\varepsilon(u))=\begin{cases}A,&\text{ if
}U_{u}\text{ corresponds to a copy of }T_{1,2},\\\
0,&\text{otherwise}.\end{cases}$ (3.5)

Summing up all non-zero terms and applying Corollary 2.10, we have

$\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu(\varepsilon(1)\varepsilon(u){}_{v\neq
1,u}U^{-}_{v})=(h(1,2)-1)\cdot\frac{A}{N}\cdot\nu(g_{1,2}(h(1,2)-2))+O(H+1).$

Combine with Lemma 3.3 and rearrange gives

$\displaystyle\nu(g(1,2))$
$\displaystyle=(h(1,2)-1)\cdot\frac{A}{N(1-\beta^{2}A)}\cdot\nu(g_{1,2}(h(1,2)-2))+O(H+1),$
(3.6) $\displaystyle=(h(1,2)-1)\cdot
A_{2}^{2}\cdot\nu(g_{1,2}(h(1,2)-2))+O(H+1).$ (3.7)

Now we are ready to perform induction. If $h(1,2)=1$ holds since $a(1)=0$. For
higher moments, we apply the inductive hypothesis on $\nu(g_{1,2}(h(1,2)-2))$.
Plug this back in 3.6 and denote $h^{\prime}(k,l)$ as the moments of $T_{k,l}$
in $g_{1,2}(h(1,2)-2)$,

$\displaystyle\nu(g(1,2))$ $\displaystyle=(h(1,2)-1)\cdot
A_{2}^{2}\cdot\nu(g_{1,2}(h(1,2)-2))+O(H+1),$ $\displaystyle=(h(1,2)-1)\cdot
A_{2}^{2}\cdot{}_{(k,l)}a(h^{\prime}(k,l))A_{2}^{h^{\prime}(k,l)}\nu({}_{k}T_{k}^{h(k)}T^{h}{}_{k}S_{k}^{h^{\prime}(k)}S^{h^{\prime}})+O(H+1),$
$\displaystyle={}_{(k,l)}a(h(k,l))A_{2}^{h(k,l)}\nu({}_{k}T_{k}^{h(k)}T^{h}{}_{k}S_{k}^{h^{\prime}(k)}S^{h^{\prime}})+O(H+1).$

where the last equality follows from $a(h(1,2))=(h(1,2)-1)a(h^{\prime}(1,2))$.
∎

###  3.2 Recursive relation for correlated "basis"

As we mentioned in Section 1.5, our goal is to obtain a recursive relation for
moments of the basis as in [Tal11, Chapter 1.10]. We need to do a little more
work for $T_{1},S_{1}$ and $T,S$ because we expect them to be correlated. We
describe the additional step here before delving into the moment computations.

By the Gaussian integration by part formula (see e.g. [Tal11] A.4), suppose
$[g_{1},g_{2}]\sim\operatorname{\mathcal{N}}(0,\Sigma)$ and some contents
$a,b\geqslant 2$, the two ways of expanding
$\operatorname*{\px@BbbE}[g_{1}^{a}g_{2}^{b}]$ are

$\displaystyle\operatorname*{\px@BbbE}[g_{1}^{a}g_{2}^{b}]$
$\displaystyle=(a-1){}_{1,1}\operatorname*{\px@BbbE}[g_{1}^{a-2}g_{1}^{b}]+b{}_{1,2}\operatorname*{\px@BbbE}[g_{1}^{a-1}g_{1}^{b-1}],$
(3.8)
$\displaystyle=a{}_{1,2}\operatorname*{\px<EMAIL_ADDRESS>(3.9)

As we saw in Section 2.2, the cavity method almost gives the above type of
relations. The cavity method allows us to decouple the last spin at time $0$,
thus if we use the symmetry of spins to rewrite one of the terms using only
the last spin, as in e.g. (2.15), we almost reduce the moment by $1$. The
problem seems to be that even though the last spin is independent of the rest
at time $0$, our function still depends on the corresponding replicas. By
Lemma 2.4, those replicas are treated the same as every other replicas and
could introduce terms that can increase the moment of the RHS of the recursive
relation.

To get some intuition, let’s consider the case $T_{1}^{2}$, recall that we can
rewrite $\nu(T_{1}^{3})$ by applying symmetry of spin on one of the $T_{1}$

$\nu(T_{1}^{3})=\nu_{1}((\varepsilon_{1,3}-\varepsilon_{2,3})T_{1}^{2}).$

Becauase $T_{1}^{2}$ is an order $2$ function, we need to invoke (2.7) and
compute $\nu^{\prime}_{0}((\varepsilon_{1,3}-\varepsilon_{2,3})T_{1}^{2})$ to
get a good enough approximation. By Lemma 2.4, even though $\sigma_{2}$ is
only used by the first term i.e. $(\varepsilon_{1,3}-\varepsilon_{2,3})$, we
still need to consider their contribution in $\nu^{\prime}_{0}(T_{1}^{3})$.
Gathering terms correspond to $(a,b)\in\\{(1,1),(2,2)\\}$ gives

$\nu_{0}((R_{1,1}-R_{2,2})T_{1})\equiv\nu_{0}(T_{1}S_{1}).$

Even if $S_{1}$ does not appear in the initial expression, taking the
derivative at time $0$ would introduce a term where the moment of $S_{1}$ is
increased by $1$.

Still, if we restrict our attention to some fixed replica $k$, we can expand
the mixed moments of $(T_{k},S_{k})$ or $(T,S)$ in two different ways similar
to (3.8). Including moments of terms associated with replicas doesn’t change
the picture here. Intuitively, this follows the pair $(T_{k},S_{k})$ (and
$(S,T)$) being independent from all other basis terms that don’t depend on
replica $k$, as indicated in Lemma 1.4. We prove this formally in Lemma 3.6
and 3.14 below.

To avoid repetition, let’s first characterize the condition under which the
relations given by the cavity method imply the desired recursive relation for
proving CLT.

######  Lemma 3.4.

Consider two sets of constants $\alpha_{2},\alpha_{1},\alpha_{0}$ and
$\beta_{2},\beta_{1},\beta_{0}$. Suppose there exist $H\geqslant 0$ and
$C\in\px@BbbR$. Suppose a function $f:\px@BbbZ\times\px@BbbZ\to\px@BbbR$ with
$f(h,h^{\prime})=0$ if $h<0$ or $h^{\prime}<0$ and $f(0,0)=C$ satisfies the
following relation: For $h,h^{\prime}>0$ and $(h,h^{\prime})\neq(0,0)$,

$\displaystyle f(h,h^{\prime})$
$\displaystyle=\alpha_{2}(h-1)f(h-2,h^{\prime})+\alpha_{1}h^{\prime}f(h-1,h^{\prime}-1)+\alpha_{0}f(h-1,h^{\prime}+1)+O(h+h^{\prime}+H+1),$
(3.10)
$\displaystyle=\beta_{2}(h^{\prime}-1)f(h,h^{\prime}-2)+\beta_{1}hf(h-1,h^{\prime}-1)+\beta_{0}f(h+1,h^{\prime}-1)+O(h+h^{\prime}+H+1).$
(3.11)

If the sets of constants satisfy

$\displaystyle\alpha_{1}+\alpha_{0}\beta_{2}=\beta_{1}+\beta_{0}\alpha_{2}:=\gamma,$
(3.12)

then we can find a set of constants $C(2,0),C(0,2),C(1,1)$ s.t. $f$ satisfies
the following recursive relations

$\displaystyle f(h,h^{\prime})=$
$\displaystyle(h-1)C(2,0)f(h-2,h^{\prime})+h^{\prime}C(1,1)f(h-1,h^{\prime}-1)+O(h+h^{\prime}+H+1),$
(3.13) $\displaystyle=$
$\displaystyle(h^{\prime}-1)C(0,2)f(h,h^{\prime}-2)+hC(1,1)f(h-1,h^{\prime}-1)+O(h+h^{\prime}+H+1).$
(3.14)

with

$C(2,0)=\frac{\alpha_{2}+\alpha_{0}\beta_{1}}{1-\alpha_{0}\beta_{0}}\quad
C(0,2)=\frac{\beta_{2}+\beta_{0}\alpha_{1}}{1-\alpha_{0}\beta_{0}}\quad
C(1,1)=\frac{\gamma}{1-\alpha_{0}\beta_{0}}$

###### Proof.

The idea is to use (3.11) and (3.10) to rewrite $f(h-1,h^{\prime}+1)$. One
thing we need to check is that the resulting constants in front of
$f(h-1,h^{\prime}+1)$ are the same in both equations.

##### Base case:

Note that $f(1,0)=f(0,1)=O(h+h^{\prime}+H+1)$. We will first handle the case
when $(h,h^{\prime})\in\\{(2,0),(0,2),(1,1)\\}$. Plug in the corresponding
values for $h,h^{\prime}$ gives (3.10) and (3.11) gives

$f(2,0)=\alpha_{2}f(0,0)+\alpha_{0}f(1,1)+O(3+H),$
$f(0,2)=\beta_{2}f(0,0)+\beta_{0}f(1,1)+O(3+H),$
$f(1,1)=\alpha_{1}f(0,0)+\alpha_{0}f(0,2)+O(3+H)=\beta_{1}f(0,0)+\beta_{0}f(2,0)+O(3+H).$

Solve the above system of linear equations gives

$\displaystyle(1-\alpha_{0}\beta_{0})f(2,0)=(\alpha_{2}+\alpha_{0}\beta_{1})f(0,0)++O(3+H),$
$\displaystyle(1-\alpha_{0}\beta_{0})f(0,2)=(\beta_{2}+\beta_{0}\alpha_{1})f(0,0)+O(3+H).$

By (3.12) and the expression for $f(2,0)$, $f(0,2)$, we have

$\displaystyle f(1,1)$
$\displaystyle=\left(\alpha_{1}+\alpha_{0}\frac{\beta_{2}+\beta_{0}\alpha_{1}}{1-\alpha_{0}\beta_{0}}\right)f(0,0)+O(3+H)=\left(\frac{\alpha_{1}+\alpha_{0}\beta_{2}}{1-\alpha_{0}\beta_{0}}\right)f(0,0)+O(3+H),$
$\displaystyle=\left(\frac{\beta_{1}+\beta_{0}\alpha_{2}}{1-\alpha_{0}\beta_{0}}\right)f(0,0)+O(3+H)=\left(\beta_{1}+\beta_{0}\frac{\alpha_{2}+\alpha_{0}\beta_{1}}{1-\alpha_{0}\beta_{0}}\right)f(0,0)+O(3+H).$

Rearrange the above equations gives

$f(2,0)=C(2,0)f(0,0)+O(3+H),\ \ f(0,2)=C(0,2)f(0,0)+O(3+H),\ \text{and}\
f(1,1)=C(1,1)f(0,0)+O(3+H).$

For the case when $h^{\prime}=0$ and $h\geqslant 0$, the equation (3.10)
becomes

$\displaystyle f(h,0)$
$\displaystyle=\alpha_{2}(h-1)f(h-2,0)+\alpha_{0}f(h-1,1)+O(h+1+H),$
$\displaystyle=\alpha_{2}(h-1)f(h-2,0)+\alpha_{0}\left[\beta_{1}(h-1)f(h-2,0)+\beta_{0}f(h,0)\right]+O(h+1+H).$

Rearrange and plug in the values of $C(2,0)$ gives

$f(h,0)=(h-1)C(2,0)f(h-2,0)+O(h+1+H).$

For $h^{\prime}\geqslant 0$ and $h=0$, the same arguement applies by starting
from (3.11) with $h=0$.

$f(0,h^{\prime})=(h^{\prime}-1)C(0,2)f(0,h^{\prime}-2)+O(h^{\prime}+1+H).$

##### General case:

Assume $h,h^{\prime}\geqslant 1$. Start from (3.10) and expand
$f(h-1,h^{\prime}+1)$ using (3.11) gives

$\displaystyle f(h,h^{\prime})=$
$\displaystyle\alpha_{2}(h-1)f(h-2,h^{\prime})+\alpha_{1}h^{\prime}f(h-1,h^{\prime}-1)+\alpha_{0}f(h-1,h^{\prime}+1)+O(h+h^{\prime}+H+1),$
$\displaystyle=$
$\displaystyle\alpha_{2}(h-1)f(h-2,h^{\prime})+\alpha_{1}h^{\prime}f(h-1,h^{\prime}-1)+O(h+h^{\prime}+H+1)$
$\displaystyle+\alpha_{0}\left(\beta_{2}h^{\prime}f(h-1,h^{\prime}-1)+\beta_{1}(h-1)f(h-2,h^{\prime})+\beta_{0}f(h,h^{\prime})\right)+O(h+h^{\prime}+H+1).$

Rearrange, we have

$f(h,h^{\prime})=(h-1)C(2,0)f(h-2,h^{\prime})+h^{\prime}C(1,1)f(h-1,h^{\prime}-1)+O(h+h^{\prime}+H+1).$

Similarily, start from (3.11) instead and repeat the above arguement gives

$f(h,h^{\prime})=(h^{\prime}-1)C(0,2)f(h,h^{\prime}-2)+hC(1,1)f(h-1,h^{\prime}-1)+O(h+h^{\prime}+H+1).$

∎

####  3.2.1 Induction on $T_{k}$ and $S_{k}$

In this section, we examine the mixed moments of $T_{k}$ and $S_{k}$. Assume
that there are $n$ replicas in total and the moments of $T_{k,l}$, $h(k,l)=0$,
for all $1\leqslant k<\leqslant n$. Denote the total moments of $T_{k}$ and
$S_{l}$ as

$h_{T}=\sumop\slimits@_{k}h(k),\quad
h_{S}=\sumop\slimits@_{l}h^{\prime}(l),\quad H_{1}=h_{T}+h_{S}+h+h^{\prime}.$

######  Theorem 3.5.

Let $\\{(g_{T_{k}},g_{S_{k}}):k\in[n]\\}$ be i.i.d Gaussian with mean $[0,0]$
and covariance matrix

${}_{1}:=\begin{bmatrix}A_{1}^{2}&C_{1}^{2}\\\
C_{1}^{2}&B_{1}^{2}\end{bmatrix},$

We have

$\nu({}_{k}T_{k}^{h(k)}{}_{l}S_{l}^{h^{\prime}(l)}T^{h}S^{h^{\prime}})=\left({}_{k}\operatorname*{\px@BbbE}[g_{T_{k}}^{h(k)}g_{S_{k}}^{h^{\prime}(k)}]\right)\nu(T^{h}S^{h^{\prime}})+O(H_{1}+1)$

Following the symmetry of replicas and the idea from Lemma 3.4, we will try to
expand higher-order mixed moments by reducing the moment of $T_{k}$ or $S_{k}$
for some fixed replica $k$.

WLOG, suppose $h(1)+h^{\prime}(1)>0$. Let $g_{1}:{}^{2}\to\px@BbbR$ be the
function that tracks the moment of $T_{1}$ and $S_{1}$ only.

$\displaystyle
g_{1}(x,y):=\begin{cases}\nu(T_{1}^{x}S_{1}^{y}{}_{k>1}T_{k}^{h(k)}{}_{l>1}S_{l}^{h^{\prime}(l)}T^{h}S^{h^{\prime}}),&\text{
if }x,y\geqslant 0,\\\ 0,&\text{ otherwise}.\end{cases}$ (3.15)

The lemma below is a generalization of Lemma 2.17 and 2.20.

######  Lemma 3.6.

For $h(1)>1$, $h^{\prime}(1)\geqslant 0$,

$\displaystyle M_{1}\nu(g(h(1),h^{\prime}(1)))$
$\displaystyle=\frac{\beta^{2}}{2}H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)$ (3.16)
$\displaystyle+(h(1)-1)\left(\beta^{2}GA_{2}^{2}+\frac{G}{N}\right)\nu(g_{1}(h(1)-2,h^{\prime}(1)))$
(3.17)
$\displaystyle+h^{\prime}(1)\frac{H}{N}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1)$
(3.18) $\displaystyle+O(H_{1}+1).$ (3.19)

For $h(1)\geqslant 0$, $h^{\prime}(1)>1$,

$\displaystyle M_{2}\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle=-2\beta^{2}E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1)$ (3.20)
$\displaystyle+h(1)\left(\beta^{2}EA_{2}^{2}+\frac{E}{N}\right)\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))$
(3.21)
$\displaystyle+(h^{\prime}(1)-1)\frac{D}{N}\nu(g_{1}(h(1),h^{\prime}(1)-2))$
(3.22) $\displaystyle+O(H_{1}+1).$ (3.23)

######  Remark 3.7.

Observe that (3.16) is again a generalization of the recursive relation in SK
model, see (1.320) (1.323) in [Tal11]. To compare our result to the SK model,
recall that $F\equiv b(2)-b(1)=1-q$, $G\equiv b(1)-b(0)=q-\hat{q}$ and $H=0$,
(3.16) becomes

$\displaystyle(1-\beta^{2}(1-4q+3\hat{q}))\nu(f(h(1),0))=$
$\displaystyle(h(1)-1)(q-\hat{q})(\beta^{2}A^{2}_{1}+\frac{1}{N})\nu(f(h(1)-2,0))+O(H+1).$

The proof of Lemma 3.6 can be found in the following section. Let’s first see
how one can deduce Theorem 3.5 Lemma 3.6. Following the intuition from the
beginning of this section, we apply Lemma 3.4 to get recursive relations that
are of the same form as Gaussian moments.

###### Proof of Theorem 3.5.

We apply Lemma 3.4 to the recursive relations in Lemma 3.6 with the following
constants

$\alpha_{2}=\frac{G}{M_{1}}\left(\beta^{2}A_{2}^{2}+\frac{1}{N}\right),\quad\alpha_{1}=\frac{H}{NM_{1}},\quad\text{and}\
\ \alpha_{0}=\frac{\beta^{2}}{2M_{1}}H.$
$\beta_{2}=\frac{D}{M_{2}N},\quad\beta_{1}=\frac{E}{M_{2}}\left(\beta^{2}A_{2}^{2}+\frac{1}{N}\right),\quad\text{and}\
\ \beta_{0}=-\frac{2\beta^{2}E}{M_{2}}.$

To apply Lemma 3.4, we need to check the consistance condition, then compute
$C(2,0),C(0,2)$ and $C(1,1)$ to get the final result.

The consistency condition is verified by Lemma 2.21. If
$g_{1}(x,y)=T_{1}^{x}S_{1}^{y}$, then we recover results from Section 2.2. We
include the computation for general cases for completeness.

##### To check the consistency condition

Note that by Claim 2.14, we have

$\alpha_{2}=\frac{G}{M_{1}}\frac{1}{NM_{3}},\quad\beta_{1}=\frac{E}{M_{2}}\frac{1}{NM_{3}}.$

To verify (3.12) holds for the current set of constants, check that

$\displaystyle\alpha_{1}+\alpha_{0}\beta_{2}$
$\displaystyle=\frac{H}{NM_{1}}(1+\frac{\beta^{2}}{2}\frac{D}{M_{2}})=\frac{H}{NM_{1}M_{2}},$
$\displaystyle\beta_{1}+\beta_{0}\alpha_{2}$
$\displaystyle=\frac{E}{NM_{2}M_{3}}(1-\frac{2\beta^{4}G}{M_{1}})$
$\displaystyle=\frac{E}{NM_{2}M_{3}}\frac{M_{1}-2\beta^{2}G}{M_{1}},$
$\displaystyle=\frac{E}{NM_{2}M_{3}}\frac{M_{3}}{M_{1}}=\alpha_{1}+\alpha_{0}\beta_{2}.$

The only things left are to compute $C(2,0)$, $C(0,2)$ and $C(1,1)$. First
check that the common denominator for $C(2,0)$, $C(0,2)$ and $C(1,1)$ is

$1-\alpha_{0}\beta_{0}=1+\frac{\beta^{4}E^{2}}{M_{1}M_{2}}=\frac{M}{M_{1}M_{2}}.$

The three constants are then given by

$\displaystyle C(2,0)$
$\displaystyle=\frac{\alpha_{2}+\alpha_{0}\beta_{1}}{1-\alpha_{0}\beta_{0}}=\frac{M_{1}M_{2}}{M}\left(\frac{G}{M_{1}}\frac{1}{NM_{3}}+\frac{\beta^{2}H}{2M_{1}}\frac{E}{M_{2}}\frac{1}{NM_{3}}\right),$
$\displaystyle=\frac{M_{1}M_{2}}{M}\cdot\frac{GM_{2}+\frac{\beta^{2}}{2}E^{2}}{NM_{1}M_{2}M_{3}}=\frac{GM_{2}+\frac{\beta^{2}}{2}E^{2}}{MN},$
$\displaystyle=A_{1}^{2}.$ $\displaystyle C(0,2)$
$\displaystyle=\frac{\beta_{2}+\beta_{0}\alpha_{1}}{1-\alpha_{0}\beta_{0}}=\frac{M_{1}M_{2}}{M}\left(\frac{D}{M_{2}N}-\frac{2\beta^{2}E}{M_{2}}\frac{H}{NM_{1}}\right),$
$\displaystyle=\frac{M_{1}M_{2}}{M}\frac{DM_{1}-2\beta^{2}E^{2}}{M_{1}M_{2}N}=\frac{DM_{1}-2\beta^{2}E^{2}}{NM},$
$\displaystyle=B_{1}^{2}.$ $\displaystyle
C(1,1)=\frac{\gamma}{1-\alpha_{0}\beta_{0}}=\frac{M_{1}M_{2}}{M}\cdot\frac{H}{NM_{1}M_{2}}=\frac{H}{MN}=C_{1}^{2}.$

By Lemma 3.4, we have

$\displaystyle\nu(g_{1}(h(1),h^{\prime}(1)))=$
$\displaystyle(h(1)-1)A_{1}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1)))+h^{\prime}(1)C_{1}^{2}\nu(g(h(1)-1,h^{\prime}(1)-1))+O(H_{1}+1),$
(3.24) $\displaystyle=$
$\displaystyle(h^{\prime}(1)-1)B_{1}^{2}\nu(g(h(1),h^{\prime}(1)-2))+h(1)C_{1}^{2}\nu(g(h(1)-1,h^{\prime}(1)-1))+O(H_{1}+1).$
(3.25)

The proof then is completed by induction on $H_{1}$. The statement holds if
$H_{1}=1$, since
$\operatorname*{\px@BbbE}[g_{T_{k}}]=\operatorname*{\px@BbbE}[g_{S_{k}}]=0$.
For $H_{1}\geqslant 2$: suppose $h(1)+h(2)\geqslant 2$. The terms on the
right-hand side of (3.24) have total moment $H_{1}-2$. We can apply the
inductive hypothesis on the right-hand side gives

$\displaystyle\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle=\left[(h(1)-1)A_{1}^{2}\operatorname*{\px@BbbE}[g_{T_{1}}^{h(1)-2}g_{S_{1}}^{h^{\prime}(1)}]+h^{\prime}(1)C_{1}^{2}\operatorname*{\px@BbbE}[g_{T_{1}}^{h(1)-1}g_{S_{1}}^{h^{\prime}(1)-1}]\right]\operatorname{\mathcal{C}}+O(H_{1}+1).$

where
$\operatorname{\mathcal{C}}=\left({}_{k>1}\operatorname*{\px@BbbE}[g_{T_{k}}^{h(k)}g_{S_{k}}^{h^{\prime}(k)}]\right)\nu(T^{h}S^{h^{\prime}})$.

Similarly, we get

$\displaystyle\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle=\left[(h^{\prime}(1)-1)B_{1}^{2}\operatorname*{\px@BbbE}[g_{T_{1}}^{h(1)}g_{S_{1}}^{h^{\prime}(1)-2}]+h(1)C_{1}^{2}\operatorname*{\px@BbbE}[g_{T_{1}}^{h(1)-1}g_{S_{1}}^{h^{\prime}(1)-1}]\right]\operatorname{\mathcal{C}}+O(H_{1}+1).$

from the second recursive relation from (3.24). Note that mixed moments of
$g_{T_{1}},g_{S_{1}}$ satisfies (3.8) with $a=h(1)$ and $b=h^{\prime}(1)$.

This completes the induction. ∎

######  Remark 3.8.

Note that if $g_{1}(x,y)=T_{1}^{x}S_{1}^{y}$, then we have
$\nu(g_{1}(2,0))=\nu(T_{1}^{2})$, $\nu(g_{1}(0,2))=\nu(S_{1}^{2})$ and
$\nu(g_{1}(1,1))=\nu(T_{1}S_{1})$. In this case, $\nu(g(0,0))=1$. Lemma 3.4
says that the same relation holds for a more general initial expression
$g(0,0)$. In the proof above, we recovered the same set of constants from
(3.16) and (3.20) as from the variance calculation in Section 2.2.

####  3.2.2 Proof of Lemma 3.6

Recall the definition of $U_{v},\varepsilon(v),U^{-}_{v}$ from the beginning
of this section and that we denote $V_{v}=\\{v_{1},v_{2},\cdots\\}$ as the set
of replicas appear in term $U_{v}$. The first step is to approximate
$g(h(1),h^{\prime}(1))$ by (3.2)

$\displaystyle\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle=\nu({}_{1\leqslant v\leqslant H_{1}}U_{v}),$ (3.26)
$\displaystyle=\nu(\varepsilon(1){}_{1<v\leqslant
H_{1}}U^{-}_{v})+\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq 1,v}U^{-}_{u}).$ (3.27)

The idea is to apply the cavity method when $U_{1}$ corresponds to $T_{1}$ and
$S_{1}$.

##### To reduce the moment of $T_{1}$

Suppose $U_{1}$ corresponds to $T_{1}$, then

$\varepsilon(1)=\varepsilon_{1_{1},1_{3}}-\varepsilon_{1_{2},1_{3}}.$

As usual, the first term in (3.27) is an order $H_{1}-1$ function, thus needs
to be approximated using (2.7) as shown in Lemma 3.9. The proof is deferred to
Appendix.

######  Lemma 3.9 (First order derivative structure for $T_{k}$).

For $h(1)\geqslant 1$ and $h^{\prime}(1)\geqslant 0$, suppose $U_{1}$
corresponds to a copy of $T_{1}$

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})=$
$\displaystyle\beta^{2}(F-3G)\nu(g_{1}(h(1),h^{\prime}(1))$
$\displaystyle+\frac{\beta^{2}}{2}H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)$
$\displaystyle+\beta^{2}(h(1)-1)GA_{2}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1)))$
$\displaystyle+O(H_{1}+1).$

The second term is approximated using (2.6).

$\displaystyle\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq 1,v}U^{-}_{u})$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu_{0}(\varepsilon(1)\varepsilon(v))\nu_{0}({}_{u\neq
1,v}U^{-}_{u})+O(H_{1}+1).$

By Lemma 1.4, $\nu_{0}(\varepsilon(1)\varepsilon(v))=0$ if $U_{v}$ doesn’t
correspond to $T_{1}$ or $S_{1}$. Moreover,

$\nu_{0}(\varepsilon(1)\varepsilon(v))=\begin{cases}G,&\text{ if }U_{v}\text{
corresponds to }T_{1},\\\ H,&\text{ if }U_{v}\text{ corresponds to
}S_{1}.\end{cases}$

There are $(h(1)-1)$ terms $T_{1}$ and $h^{\prime}(1)$ terms $S_{1}$ in
${}_{v\geqslant 1}U^{-}_{v}$. Summing up all terms of the same type and
applying Corollary 2.10 on all terms,

$\displaystyle\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq 1,v}U^{-}_{u})=$
$\displaystyle(h(1)-1)\frac{G}{N}\nu(g_{1}(h(1)-2,h^{\prime}(1)),$
$\displaystyle+h^{\prime}(1)\frac{H}{N}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1)+O(H_{1}+1).$

Combine results for both first and second term of (3.27) and rearrange gives
3.16

$\displaystyle(1-\beta^{2}(F-3G))\nu(g(h(1),h^{\prime}(1)))$
$\displaystyle=\frac{\beta^{2}}{2}H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)$
$\displaystyle+(h(1)-1)\left(\beta^{2}GA_{2}^{2}+\frac{G}{N}\right)\nu(g_{1}(h(1)-2,h^{\prime}(1)))$
$\displaystyle+h^{\prime}(1)\frac{H}{N}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1)$
$\displaystyle+O(H_{1}+1).$

##### To reduce the moment of $S_{1}$

Suppose, in this case, $U_{1}$ corresponds to $S_{1}$ term.

$\varepsilon(1)=\varepsilon_{1_{1},1_{1}}-\varepsilon_{1_{2},1_{2}}.$

The first term in (3.27) is characterized by the following lemma.

######  Lemma 3.10 (First order derivative structure for $S_{k}$).

If $h^{\prime}(1)\geqslant 1$ and $h(1)\geqslant 0$, suppose $U_{1}$
corresponds to a copy of $S_{1}$

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})=$
$\displaystyle\frac{\beta^{2}}{2}D\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle+\beta^{2}h(1)EA_{2}^{2}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))$
$\displaystyle-2\beta^{2}E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1)$
$\displaystyle+O(H_{1}+1)).$

For the second term, again, we have

$\displaystyle\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq 1,v}U^{-}_{u})$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu_{0}(\varepsilon(1)\varepsilon(v))\nu_{0}({}_{u\neq
1,v}U^{-}_{u})+O(H_{1}+1).$

Check that

$\nu_{0}(\varepsilon(1)\varepsilon(v))=\begin{cases}D,&\text{ if }U_{v}\text{
corresponds to a copy of }S_{1},\\\ E,&\text{ if }U_{v}\text{ corresponds to a
copy of }T_{1},\\\ 0,&\text{ otherwise. }\end{cases}$

Plug in the above equation gives

$\displaystyle\frac{1}{N}\sumop\slimits@_{1<v\leqslant
H_{1}}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq 1,v}U^{-}_{u})$
$\displaystyle=(h^{\prime}(1)-1)\frac{D}{N}\nu(g_{1}(h(1),h^{\prime}(1)-2))$
$\displaystyle+h(1)\frac{E}{N}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))+O(H_{1}+1).$

Combine the estimations of the two terms gives (3.20)

$\displaystyle(1-\frac{\beta^{2}}{2}D)\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle=-2\beta^{2}E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1)$
$\displaystyle+h(1)\left(\beta^{2}EA_{2}^{2}+\frac{E}{N}\right)\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))$
$\displaystyle+(h^{\prime}(1)-1)\frac{D}{N}\nu(g_{1}(h(1),h^{\prime}(1)-2))$
$\displaystyle+O(H_{1}+1).$

####  3.2.3 Induction on $T$ and $S$

In this section, we consider functions in the form of $T^{h}S^{h^{\prime}}$
for $h,h^{\prime}\in{}_{\geqslant 0}$. As in previous sections, the idea is to
write $T^{h}S^{h^{\prime}}$ as a formula of $T^{h-1}S^{h^{\prime}-1}$ and
$\\{T^{h-2}S^{h^{\prime}},T^{h}S^{h^{\prime}-2}\\}$. To this end, let’s define

$g(h,h^{\prime})=\begin{cases}T^{h}S^{h^{\prime}},&\text{ if
}h,h^{\prime}\geqslant 0,\\\ 0,&\text{ otherwise. }\end{cases}$

######  Theorem 3.11.

Let $\\{(g_{T},g_{S})\\}$ be a Gaussian vector with mean $[0,0]$ and
covariance matrix

$\begin{bmatrix}A_{0}^{2}&C_{0}^{2}\\\ C_{0}^{2}&B_{0}^{2}\end{bmatrix},$

where $A_{0}^{2},B_{0}^{2},C_{0}^{2}$ are given in Theorem 3.17. Then we have

$\nu(g(h,h^{\prime}))=\operatorname*{\px@BbbE}[g_{T}^{h}g_{S}^{h^{\prime}}]+O(h+h^{\prime}+1).$

The proof of Theorem 3.11 uses the same idea as Theorem 3.5: we first use
cavity method to obtain a recursive relation, then apply Lemma 3.4 to see that
moment of $S,T$ is the moments of a correlated Gaussian. The only difference
lies in the structure of overlaps in cavity computation. Because of this
difference, we will first introduce a more refined set of constants that will
appear in the cavity computation, thus also the recursive relations of
moments.

##### Constants

To motivate the set of constants we need to compute the moment of $T,S$,
recall that for variance computation, we started from (2.9).

$\displaystyle\nu(X^{2})$
$\displaystyle=\frac{1}{N}\nu(\varepsilon_{X}^{2})+\nu(\varepsilon_{X}X^{-}),$
(3.28)
$\displaystyle=\frac{1}{N}\nu(\varepsilon_{X}^{2})+\nu^{\prime}_{0}(\varepsilon_{X}X^{-})+O(3).$
(3.29)

By setting $X=T$ or $X=S$, we record the following constants corresponding to
the expectation of the last spin. They mainly appears in
$\nu^{\prime}_{0}(\varepsilon_{X}X^{-})$ as a result of formula from 2.4.

######  Definition 3.12.

From $\nu(\varepsilon_{T}\varepsilon_{X})$:

  * •

$I_{1}=\nu_{0}((\varepsilon_{12}-q)\varepsilon_{12})=\nu_{0}(\varepsilon_{12}\varepsilon_{12}-q^{2})$,

  * •

$I_{2}=\nu_{0}((\varepsilon_{12}-q)\varepsilon_{13})=\nu_{0}(\varepsilon_{12}\varepsilon_{13}-q^{2})$,

  * •

$I_{3}=\nu_{0}((\varepsilon_{12}-q)\varepsilon_{34})=\nu_{0}(\varepsilon_{12}\varepsilon_{34}-q^{2})$,

  * •

$I_{4}=\nu_{0}((\varepsilon_{12}-q)\varepsilon_{11})=\nu_{0}(\varepsilon_{12}\varepsilon_{11}-pq)$,

  * •

$I_{5}=\nu_{0}((\varepsilon_{12}-q)\varepsilon_{33})=\nu_{0}(\varepsilon_{12}\varepsilon_{33}-pq)$.

From $\nu(\varepsilon_{S}\varepsilon_{X})$:

  * •

$K_{1}=\nu_{0}((\varepsilon_{1,1}-p)\varepsilon_{12})=\nu_{0}(\varepsilon_{11}\varepsilon_{12})-pq=I_{4}$,

  * •

$K_{2}=\nu_{0}((\varepsilon_{1,1}-p)\varepsilon_{11})=\nu_{0}(\varepsilon_{11}\varepsilon_{11})-p^{2}$,

  * •

$K_{3}=\nu_{0}((\varepsilon_{1,1}-p)\varepsilon_{23})=\nu_{0}(\varepsilon_{11}\varepsilon_{23})-pq=I_{5}$,

  * •

$K_{4}=\nu_{0}((\varepsilon_{1,1}-p)\varepsilon_{22})=\nu_{0}(\varepsilon_{11}\varepsilon_{22})-p^{2}$.

The constants defined in 2.11 are linear combinations of the ones defined
above.

######  Claim 3.13.

$I_{1}-I_{2}=F,\quad I_{2}-I_{3}=G,\quad I_{4}-I_{5}=E,$ $K_{1}-K_{3}=E,\quad
K_{2}-K_{4}=D.$

###### Proof.

The claim follows by checking the definition of in 2.11 and comparing with the
ones in Definition 3.12 ∎

##### Proof of Theorem 3.11

In this section, we prove Theorem 3.11 and record the variance of $T,S$ as a
special case. As in the proof of Theorem 3.5, we first give two recursive
formulas for mixed moments of $T,S$ using the cavity method. The proof of
Theorem 3.11 follows from rewriting the relations using Lemma 3.4. The proof
of the Lemma 3.14 will be shown in the next subsection.

######  Lemma 3.14.

For $h\geqslant 1;h^{\prime}\geqslant 0$, we have

$\displaystyle M_{1}\nu(g(h,h^{\prime}))=$
$\displaystyle\beta^{2}E\nu(g(h-1,h^{\prime}+1))$ (3.30)
$\displaystyle+\beta^{2}(h-1)\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}I_{3}\right]\nu(g(h-2,h^{\prime}))$
(3.31)
$\displaystyle+\beta^{2}h^{\prime}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}I_{5}\right]\nu(g(h-1,h^{\prime}-1))$
(3.32) $\displaystyle+O(h+h^{\prime}+1).$ (3.33)

For $h\geqslant 0;h^{\prime}\geqslant 1$

$\displaystyle M_{2}\nu(g(h,h^{\prime}))=$
$\displaystyle-\beta^{2}E\nu(g(h+1,h^{\prime}-1))$ (3.34)
$\displaystyle+\beta^{2}h\left[K_{4}C_{1}^{2}+2(E-K_{3})A_{1}^{2}+K_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}K_{3}\right]\nu(g(h-1,h^{\prime}-1))$
(3.35)
$\displaystyle+\beta^{2}(h^{\prime}-1)\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}K_{4}\right]\nu(g(h,h^{\prime}-2))$
(3.36) $\displaystyle+O(h+h^{\prime}+1).$ (3.37)

######  Remark 3.15.

First check that (3.32) reduces to the equation (1.262) of [Tal11] in SK
model: For SK mode, there’s no self-overlap terms and $C_{1}^{2}=0$. To check
other constants, $I_{3}=\hat{q}-q^{2}$,

$2G-I_{3}=2I_{2}-3I_{3}=2(q-q^{2})-3(\hat{q}-q^{2})=2q+q^{2}-3\hat{q}.$

Combined with Remark 2.18, we have

$\beta^{2}\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}I_{3}\right]\equiv\frac{\hat{q}-q^{2}}{N}+\beta^{2}(\hat{q}-q^{2})A^{2}+2\beta^{2}(2q+q^{2}-3\hat{q})B^{2}+O(3).$

where $A,B,C$ are defined in [Tal11, Chapter 1.8].

To apply Lemma 3.4 on $f(h,h^{\prime})=\nu(g(h,h^{\prime}))$, we first need to
check (3.32) and (3.36) satisfies the consistency condition. That is, the goal
is to verify (3.12) with

$\alpha_{2}=\frac{\beta^{2}}{M_{1}}\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}I_{3}\right],$
$\alpha_{1}=\frac{\beta^{2}}{M_{1}}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}I_{5}\right],$
$\alpha_{0}=\frac{\beta^{2}E}{M_{1}}.$

and

$\beta_{2}=\frac{\beta^{2}}{M_{2}}\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}K_{4}\right],$
$\beta_{1}=\frac{\beta^{2}}{M_{2}}\left[K_{4}C_{1}^{2}+2(E-K_{3})A_{1}^{2}+K_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}K_{3}\right],$
$\beta_{0}=-\frac{\beta^{2}E}{M_{2}}.$

We begin by recording some useful expressions of the constants that will
simplify the proof.

######  Claim 3.16.

$2A_{1}^{2}-(A_{2}^{2}+\frac{1}{\beta^{2}N})=-\frac{M_{2}}{\beta^{2}NM},$
$\frac{B_{1}^{2}}{2}+\frac{1}{\beta^{2}N}=\frac{M_{1}}{\beta^{2}MN},$
$M_{1}-M_{3}=2\beta^{2}G.$

###### Proof.

The third equation follows from the definition of $M_{1}$ and $M_{3}$.

For the first and second equation: By Claim 2.14,

$\beta^{2}A_{2}^{2}+\frac{1}{N}=\frac{1}{NM_{3}}.$

Recall the definition of $A_{1}^{2},B_{1}^{2}$

$A_{1}^{2}:=\frac{GM_{2}+\frac{\beta^{2}}{2}EH}{M}\frac{1}{NM_{3}}=\frac{1}{2\beta^{2}N}\left(\frac{1}{M_{3}}-\frac{M_{2}}{M}\right),$
$B_{1}^{2}=\frac{DM_{1}-2\beta^{2}E^{2}}{NM}=\frac{2}{N\beta^{2}}(\frac{M_{1}}{M}-1).$

Combine and rearrange to give the desired result. ∎

To simplify notations, denote LHS and RHS as

$\text{LHS}:=\alpha_{1}+\alpha_{0}\beta_{2},\quad\text{and}\ \
\beta_{1}+\beta_{0}\alpha_{2}=:\text{RHS}.$

We now begin to verify (3.12) by comparing the coefficients in front of
$K_{4},I_{5},I_{3}$ in LHS and RHS.

##### For $K_{4}$

$\displaystyle\text{LHS}=\frac{\beta^{4}E}{M_{1}M_{2}}\left(\frac{1}{2}B_{1}^{2}+\frac{1}{\beta^{2}N}\right)\stackrel{{\scriptstyle\ref{claim:rel1}}}{{=}}\frac{\beta^{4}E}{M_{1}M_{2}}\frac{M_{1}}{\beta^{2}MN}=\frac{\beta^{2}}{M_{2}}\frac{E}{MN}\stackrel{{\scriptstyle\ref{lem:
var s1t1}}}{{=}}\frac{\beta^{2}C_{1}^{2}}{M_{2}}=\text{RHS}.$

##### For $I_{5}$

Recall that by definition, $I_{5}=K_{3}$. On one hand,

LHS
$\displaystyle=\frac{\beta^{2}}{M_{1}}\left(\frac{1}{2}B_{1}^{2}+\frac{1}{\beta^{2}N}\right)-\frac{\beta^{2}E}{M_{1}}\frac{\beta^{2}}{M_{2}}C_{1}^{2}\stackrel{{\scriptstyle\ref{claim:rel1}}}{{=}}\frac{\beta^{2}}{M_{1}}\frac{M_{1}}{\beta^{2}MN}-\frac{\beta^{4}E}{M_{1}M_{2}}\frac{E}{MN},$
$\displaystyle=\frac{1}{MN}\left(1-\frac{\beta^{4}E^{2}}{M_{1}M_{2}}\right).$
RHS
$\displaystyle=\frac{\beta^{2}}{M_{2}}(-2A_{1}^{2}+A_{2}^{2}+\frac{1}{\beta^{2}N})-\frac{\beta^{2}E}{M_{2}}\frac{\beta^{2}}{M_{1}}C_{1}^{2}\stackrel{{\scriptstyle\ref{claim:rel1}}}{{=}}\frac{\beta^{2}}{M_{2}}\frac{M_{2}}{\beta^{2}MN}-\frac{\beta^{4}E^{2}}{M_{2}M_{1}MN},$
$\displaystyle=\frac{1}{MN}(1-\frac{\beta^{4}E^{2}}{M_{1}M_{2}})=\text{LHS}.$

##### For $I_{3}$

RHS
$\displaystyle=-\frac{\beta^{2}E}{M_{2}}\frac{\beta^{2}}{M_{1}}(-2A_{1}^{2}+A_{2}^{2}+\frac{1}{\beta^{2}N})\stackrel{{\scriptstyle\ref{claim:rel1}}}{{=}}-\frac{\beta^{4}E}{M_{2}M_{1}}\frac{M_{2}}{\beta^{2}MN},$
$\displaystyle=-\frac{\beta^{2}E}{M_{1}MN}=-\frac{\beta^{2}}{M_{1}}C_{1}^{2}=\text{LHS}.$

##### The remaining terms

With some abuse of notations, check that

LHS
$\displaystyle=\frac{\beta^{2}}{M_{1}}2GC_{1}^{2}+\frac{\beta^{2}E}{M_{1}}\frac{\beta^{2}}{M_{2}}EC_{1}^{2}=\frac{C_{1}^{2}\left(2\beta^{2}GM_{2}+\beta^{4}E^{2}\right)}{M_{1}M_{2}}\stackrel{{\scriptstyle\ref{T1
var}}}{{=}}\frac{2E\beta^{2}A_{1}^{2}M_{3}}{M_{1}M_{2}},$ RHS
$\displaystyle=\frac{\beta^{2}}{M_{2}}2EA_{1}^{2}-\frac{\beta^{2}E}{M_{2}}\frac{\beta^{2}}{M_{1}}4GA_{1}^{2}=2\beta^{2}EA_{1}^{2}\frac{(M_{1}-\beta^{2}2G)}{M_{1}M_{2}},$
$\displaystyle\stackrel{{\scriptstyle\ref{claim:rel1}}}{{=}}2\beta^{2}EA_{1}^{2}\frac{M_{3}}{M_{1}M_{2}}=\text{LHS}.$

This allows us to apply Lemma 3.4 to obtain a recursive relation for
$\nu(T^{h}S^{h^{\prime}})$. We start by computing the variance of $T$ and $S$.

######  Theorem 3.17.

For $\beta<\beta^{\prime}$, we have

$\nu(T^{2})=A_{0}^{2}+O(3),$

where

$\displaystyle A_{0}^{2}$
$\displaystyle=\frac{\beta^{2}}{M}\left(\beta^{2}EK_{4}+M_{2}I_{5}\right)C_{1}^{2}+2\frac{\beta^{2}}{M}\left(\beta^{2}E(E-K_{3})+M_{2}(2G-I_{3})\right)A_{1}^{2}$
$\displaystyle+\frac{\beta^{2}}{M}\left(\beta^{2}EK_{3}+M_{2}I_{3}\right)A_{2}^{2}+\frac{1}{MN}\left(\beta^{2}EK_{3}+M_{2}I_{3}\right).$

And we further have

$\nu(S^{2})=B_{0}^{2}+O(3),$

where

$B_{0}^{2}=\frac{\beta^{2}}{2M}\left(M_{1}K_{4}-\beta^{2}EI_{5}\right)B_{1}^{2}+\frac{\beta^{2}}{M}\left(M_{1}(E-K_{3})-E\beta^{2}(2G-I_{3})\right)C_{1}^{2}+\frac{1}{MN}\left(M_{1}K_{4}-\beta^{2}EI_{5}\right).$

Finally we also have

$\nu(ST)=C_{0}+O(3),$

where

$C_{0}=\frac{\beta^{2}}{M}\left(\frac{1}{2}M_{2}I_{5}+\beta^{2}EK_{4}\right)B_{1}^{2}+\frac{\beta^{2}}{M}\left(M_{2}(2G-I_{3})+\beta^{2}E(E-K_{3})\right)C_{1}^{2}+\frac{1}{MN}\left(M_{2}I_{5}+\beta^{2}EK_{4}\right).$

###### Proof.

Since the coefficients in (3.32) and (3.36) satisfy the condition (3.12), we
can apply Lemma 3.4 with $h,h^{\prime}\in\\{0,2\\}$ to obtain the desired
result. First, the common denominator of $C(2,0),C(0,2),C(1,1)$ is

$1-\alpha_{0}\beta_{0}=1+\frac{\beta^{4}E^{2}}{M_{1}M_{2}}=\frac{M}{M_{1}M_{2}}.$

For variance of $T$:

$\displaystyle(1-\alpha_{0}\beta_{0})\nu(T^{2})$
$\displaystyle=(1-\alpha_{0}\beta_{0})\nu(g(2,0))=\alpha_{2}+\alpha_{0}\beta_{1}$
$\displaystyle=\frac{\beta^{2}}{M_{1}}\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}I_{3}\right]$
$\displaystyle+\frac{\beta^{4}E}{M_{1}M_{2}}\left[K_{4}C_{1}^{2}+2(E-K_{3})A_{1}^{2}+K_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}K_{3}\right]+O(3).$

Rearrange gives

$\displaystyle\nu(T^{2})$
$\displaystyle=\frac{\beta^{2}}{M}\left(\beta^{2}EK_{4}+M_{2}I_{5}\right)C_{1}^{2}+2\frac{\beta^{2}}{M}\left(\beta^{2}E(E-K_{3})+M_{2}(2G-I_{3})\right)A_{1}^{2}$
$\displaystyle+\frac{\beta^{2}}{M}\left(\beta^{2}EK_{3}+M_{2}I_{3}\right)A_{2}^{2}+\frac{1}{MN}\left(\beta^{2}EK_{3}+M_{2}I_{3}\right)+O(3).$

Next, we compute the variance of $S$:

$\displaystyle(1-\alpha_{0}\beta_{0})\nu(S^{2})$
$\displaystyle=(1-\alpha_{0}\beta_{0})\nu(g(0,2))=\beta_{2}+\beta_{0}\alpha_{1}$
$\displaystyle=\frac{\beta^{2}}{M_{2}}\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}K_{4}\right]-\frac{\beta^{4}E}{M_{1}M_{2}}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}I_{5}\right]+O(3).$

Rearrange gives

$\displaystyle\nu(S^{2})$
$\displaystyle=\frac{\beta^{2}}{2M}\left(M_{1}K_{4}-\beta^{2}EI_{5}\right)B_{1}^{2}+\frac{\beta^{2}}{M}\left(M_{1}(E-K_{3})-E\beta^{2}(2G-I_{3})\right)C_{1}^{2}+\frac{1}{MN}\left(M_{1}K_{4}-\beta^{2}EI_{5}\right)+O(3).$

For the covariance $\nu(TS)$,

$\displaystyle(1-\alpha_{0}\beta_{0})\nu(TS)$
$\displaystyle=(1-\alpha_{0}\beta_{0})\nu(g(1,1))=\alpha_{1}+\alpha_{0}\beta_{2},$
$\displaystyle=\frac{\beta^{2}}{M_{1}}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}I_{5}\right]+\frac{\beta^{4}E}{M_{1}M_{2}}\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}K_{4}\right].$

Rearrange gives

$\displaystyle\nu(TS)=\frac{\beta^{2}}{M}\left(\frac{1}{2}M_{2}I_{5}+\beta^{2}EK_{4}\right)B_{1}^{2}+\frac{\beta^{2}}{M}\left(M_{2}(2G-I_{3})+\beta^{2}E(E-K_{3})\right)C_{1}^{2}+\frac{1}{MN}\left(M_{2}I_{5}+\beta^{2}EK_{4}\right)+O(3).$

∎

Now we turn to the proof of the general moments $T^{h}S^{h^{\prime}}$.

###### Proof of Theorem 3.11.

By Lemma 3.4, we have the following recursive relation for moments of $S,T$.

$\displaystyle\nu(T^{h}S^{h^{\prime}})$
$\displaystyle=(h-1)A_{0}^{2}\nu(T^{h-2}S^{h^{\prime}})+h^{\prime}B_{0}^{2}\nu(T^{h-1}S^{h^{\prime}-1})+O(h+h^{\prime}+1),$
(3.38)
$\displaystyle=hA_{0}^{2}\nu(T^{h-1}S^{h^{\prime}-1})+(h^{\prime}-1)B_{0}^{2}\nu(T^{h}S^{h^{\prime}-2})+O(h+h^{\prime}+1).$
(3.39)

The proof then proceeds with induction on $h^{\prime}+h$. If $h+h^{\prime}=1$,
the expression holds as odd moments of Gaussian is $0$. For
$h+h^{\prime}\geqslant 2$, applying the inductive hypothesis on two terms on
the right-hand side gives

$\displaystyle\nu(T^{h}S^{h^{\prime}})$
$\displaystyle=(h-1)A_{0}^{2}\operatorname*{\px@BbbE}[g_{T}^{h-2}g_{S}^{h^{\prime}}]+h^{\prime}B_{0}^{2}\operatorname*{\px@BbbE}[g_{T}^{h-1}g_{S}^{h^{\prime}-1}]+O(h+h^{\prime}+1),$
(3.40)
$\displaystyle=hA_{0}^{2}\operatorname*{\px@BbbE}[g_{T}^{h-1}g_{S}^{h^{\prime}-1}]+(h^{\prime}-1)B_{0}^{2}\operatorname*{\px@BbbE}[g_{T}^{h}g_{S}^{h^{\prime}-2}]+O(h+h^{\prime}+1).$
(3.41)

Using Gaussian integration by parts (3.8) to rewrite RHS completes the proof.
∎

####  3.2.4 Proof of Lemma 3.14

In this section, we derive Lemma 3.14 using the cavity method. Recall the
definition of $U_{v},\varepsilon(v),U^{-}_{v}$ from the beginning of this
section and that we denote $V_{v}=\\{v_{1},v_{2},\cdots\\}$ as the set of
replicas appears in term $U_{v}$. Here $|V_{v}|=2$ if $U_{v}$ corresponds to
$T$ and $|V_{v}|=1$ if $U_{v}$ corresponds to $S$.

##### To reduce the moment of $T$

We start by proving (3.32). As usual, the first term in (3.2) is approximated
using (2.7). We record the result in Lemma 3.18 The proof is technical but
straight forward, thus pushed to the appendix (Lemma 4.7).

######  Lemma 3.18 (First order derivative structure for $T$).

If $|V_{1}|=2$, then

$\displaystyle\nu(g(h,h^{\prime}))=$
$\displaystyle\beta^{2}(F-3G)\nu(g(h,h^{\prime})+\beta^{2}E\nu(g(h-1,h^{\prime}+1))$
$\displaystyle+\beta^{2}(h-1)\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}\right]\nu(g(h-2,h^{\prime}))$
$\displaystyle+\beta^{2}h^{\prime}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}\right]\nu(g(h-1,h^{\prime}-1))$
$\displaystyle+O(h+h^{\prime}+1).$

##### For the second term in (3.2)

$\displaystyle\frac{1}{N}\sumop\slimits@_{v=2}^{h^{\prime}+h}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq
v}U^{-}_{u})$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{v=2}^{h^{\prime}+h}\nu((\varepsilon_{1_{1},1_{2}})-Q_{1_{1},1_{2}})\varepsilon((\varepsilon_{v_{1},v_{2}})-Q_{v_{1},v_{2}})){}_{u\neq
v}U^{-}_{u}),$ (3.42) $\displaystyle\stackrel{{\scriptstyle(\ref{lemma:last
spin})}}{{=}}\frac{h-1}{N}I_{3}\nu(T^{h-2}S^{h^{\prime}})+\frac{h^{\prime}}{N}I_{5}\nu(T^{h-1}S^{h^{\prime}-1})+O(h^{\prime}+h+1),$
(3.43)
$\displaystyle=\frac{h-1}{N}I_{3}\nu(g(h-2,h^{\prime}))+\frac{h^{\prime}}{N}I_{5}\nu(g(h-1,h^{\prime}-1))+O(h^{\prime}+h+1).$
(3.44)

Combine Lemma 3.18 and (3.42) gives (3.32)

$\displaystyle\left(1-\beta^{2}(F-3G)\right)\nu(g(h,h^{\prime}))=$
$\displaystyle\beta^{2}E\nu(g(h-1,h^{\prime}+1))$
$\displaystyle+\beta^{2}(h-1)\left[I_{5}C_{1}^{2}+2(2G-I_{3})A_{1}^{2}+I_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}I_{3}\right]\nu(g(h-2,h^{\prime}))$
$\displaystyle+\beta^{2}h^{\prime}\left[\frac{1}{2}I_{5}B_{1}^{2}+(2G-I_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}I_{5}\right]\nu(g(h-1,h^{\prime}-1))$
$\displaystyle+O(h+h^{\prime}+1).$

##### To reduce the moment of $S$

Similarily, we approximate the first term in (3.2) using (2.7) to get Lemma
3.19 The proof can be found in appendix (Lemma 4.8).

######  Lemma 3.19 (First order derivative structure for $S$).

Suppose $|V_{1}|=1$,

$\displaystyle\nu(g(h,h^{\prime}))=$
$\displaystyle\frac{\beta^{2}}{2}D\nu(g(h,h^{\prime}))$
$\displaystyle+\beta^{2}h\left[K_{4}C_{1}^{2}+2(E-K_{3})A_{1}^{2}+K_{3}A_{2}^{2}\right]\nu(g(h-1,h^{\prime}-1))$
$\displaystyle+\beta^{2}(h^{\prime}-1)\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}\right]\nu(g(h,h^{\prime}-2))$
$\displaystyle-\beta^{2}E\nu(g(h+1,h^{\prime}-1))$
$\displaystyle+O(h+h^{\prime}+1).$

##### For the second term in (3.2)

$\displaystyle\frac{1}{N}\sumop\slimits@_{v=2}^{h^{\prime}+h}\nu(\varepsilon(1)\varepsilon(v){}_{u\neq
v}U^{-}_{u})$
$\displaystyle=\frac{1}{N}\sumop\slimits@_{v=2}^{h^{\prime}+h}\nu((\varepsilon_{1_{1},1_{2}})-Q_{1_{1},1_{2}})\varepsilon((\varepsilon_{v_{1},v_{2}})-Q_{v_{1},v_{2}})){}_{u\neq
v}U^{-}_{u}),$ (3.45) $\displaystyle\stackrel{{\scriptstyle(\ref{lemma:last
spin})}}{{=}}\frac{h}{N}K_{3}\nu(g(h-1,h^{\prime}-1))+\frac{h^{\prime}-1}{N}K_{4}\nu(g(h,h^{\prime}-2))+O(h^{\prime}+h+1).$
(3.46)

Combining results from Lemma 3.19 and (3.45) gives the desired result.

$\displaystyle\left(1-\frac{\beta^{2}}{2}D\right)\nu(g(h,h^{\prime}))=$
$\displaystyle\beta^{2}h\left[K_{4}C_{1}^{2}+2(E-K_{3})A_{1}^{2}+K_{3}A_{2}^{2}+\frac{1}{\beta^{2}N}K_{3}\right]\nu(g(h-1,h^{\prime}-1))$
$\displaystyle+\beta^{2}(h^{\prime}-1)\left[\frac{1}{2}K_{4}B_{1}^{2}+(E-K_{3})C_{1}^{2}+\frac{1}{\beta^{2}N}K_{4}\right]\nu(g(h,h^{\prime}-2))$
$\displaystyle-\beta^{2}E\nu(g(h+1,h^{\prime}-1))$
$\displaystyle+O(h+h^{\prime}+1).$

###  3.3 Proof of Lemma 3.1

In this section, we put all the pieces together to compute the general mixed
moments of $T_{k,l},T_{K},S_{k},S,T$.

###### Proof.

For $k,l\in[n]$ $k\neq l$, let $\\{g_{T_{k,l}}\\}$ be the family of
independent centered Gaussian random varaible with
$\operatorname*{\px@BbbE}[g^{2}_{T_{k,l}}]=A^{2}_{2}$ as in Theorem 3.2. For
$k\in[n]$, let $\\{(g_{T_{k}},g_{S_{k}})\\}$ be the family of independent
centered Gaussian with covariance matrix 1 as in Theorem 3.5 and independent
from $\\{g_{T_{k,l}}\\}$. Similarily, let $(g_{S},g_{T})$ be the Gaussian
random vector with covariance matrix 0 as in Theorem 3.11 and independent from
$\\{g_{T_{k,l}}\\}$ and $\\{(g_{T_{k}},g_{S_{k}})\\}$. Apply Theorem 3.2, then
3.5, and finally 3.11 gives the desired result. ∎

## References

  * [AC21] Antonio Auffinger and Cathy Xi Chen.  Thouless-Anderson-Palmer equations for the Ghatak-Sherrington mean field spin glass model.  J. Stat. Phys., 184(2):Paper No. 22, 25, 2021.
  * [AG23] Ahmed El Alaoui and Jason Gaitonde.  Bounds on the covariance matrix of the sherrington-kirkpatrick model, 2023\.
  * [CCM23] Francesco Camilli, Pierluigi Contucci, and Emanuele Mingione.  Central limit theorem for the overlaps on the nishimori line, 2023.
  * [Che22] Xi Chen.  Thouless-Anderson-Palmer Equations for the Ghatak-Sherrington Mean Field Spin Glass Model.  PhD thesis, 2022.  Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated - 2023-03-08.
  * [CMP+23] Patrick Charbonneau, Enzo Marinari, Giorgio Parisi, Federico Ricci-tersenghi, Gabriele Sicuro, Francesco Zamponi, and Marc Mezard.  Spin Glass Theory and Far Beyond: Replica Symmetry Breaking after 40 Years.  World Scientific, 2023.
  * [CN95] F. Comets and J. Neveu.  The Sherrington-Kirkpatrick model of spin glasses and stochastic calculus: the high temperature case.  Comm. Math. Phys., 166(3):549–564, 1995.
  * [DCdA00] FA Da Costa and JM de Araújo.  Zero-temperature tap equations for the ghatak-sherrington model.  The European Physical Journal B-Condensed Matter and Complex Systems, 15:313–316, 2000.
  * [dCYS94] Francisco A da Costa, Carlos SO Yokoi, and Silvio RA Salinas.  First-order transition in a spin-glass model.  Journal of Physics A: Mathematical and General, 27(10):3365, 1994\.
  * [DW21] Partha S. Dey and Qiang Wu.  Fluctuation results for multi-species Sherrington-Kirkpatrick model in the replica symmetric regime.  J. Stat. Phys., 185(3):Paper No. 22, 40, 2021.
  * [Gen96a] Barbara Gentz.  An almost sure central limit theorem for the overlap parameters in the Hopfield model.  Stochastic Process. Appl., 62(2):243–262, 1996.
  * [Gen96b] Barbara Gentz.  A central limit theorem for the overlap in the Hopfield model.  Ann. Probab., 24(4):1809–1841, 1996.
  * [GL99] Barbara Gentz and Matthias Löwe.  The fluctuations of the overlap in the Hopfield model with finitely many patterns at the critical temperature.  Probab. Theory Related Fields, 115(3):357–381, 1999.
  * [GS77] SK Ghatak and D Sherrington.  Crystal field effects in a general s ising spin glass.  Journal of Physics C: Solid State Physics, 10(16):3149, 1977.
  * [GT02] Francesco Guerra and Fabio Lucio Toninelli.  Central limit theorem for fluctuations in the high temperature region of the Sherrington-Kirkpatrick spin glass model.  J. Math. Phys., 43(12):6224–6237, 2002.
  * [Han07] Albert Hanen.  Un théorème limite pour les covariances des spins dans le modèle de Sherrington-Kirkpatrick avec champ externe.  Ann. Probab., 35(1):141–179, 2007.
  * [Hop82] J. J. Hopfield.  Neural networks and physical systems with emergent collective computational abilities.  Proc. Nat. Acad. Sci. U.S.A., 79(8):2554–2558, 1982.
  * [KH99] Katsuki Katayama and Tsuyoshi Horiguchi.  Ghatak-sherrington model with spin s.  Journal of the Physical Society of Japan, 68(12):3901–3910, 1999\.
  * [LDA82] EJS Lage and JRL De Almeida.  Stability conditions of generalised ising spin glass models.  Journal of Physics C: Solid State Physics, 15(33):L1187, 1982.
  * [Leu07] Luca Leuzzi.  Spin-glass model for inverse freezing.  Philosophical Magazine, 87(3-5):543–551, 2007.
  * [LS22] Benjamin Landon and Philippe Sosoe.  Fluctuations of the overlap at low temperature in the 2-spin spherical SK model.  Ann. Inst. Henri Poincaré Probab. Stat., 58(3):1426–1459, 2022\.
  * [MS85] PJ Mottishaw and D Sherrington.  Stability of a crystal-field split spin glass.  Journal of Physics C: Solid State Physics, 18(26):5201, 1985.
  * [NS19] Vu Lan Nguyen and Philippe Sosoe.  Central limit theorem near the critical temperature for the overlap in the 2-spin spherical SK model.  J. Math. Phys., 60(10):103302, 13, 2019.
  * [Pan05] Dmitry Panchenko.  Free energy in the generalized Sherrington-Kirkpatrick mean field model.  Rev. Math. Phys., 17(7):793–857, 2005.
  * [Pan14] Dmitry Panchenko.  The Parisi formula for mixed $p$-spin models.  Ann. Probab., 42(3):946–958, 2014.
  * [Pan18] Dmitry Panchenko.  Free energy in the mixed $p$-spin models with vector spins.  Ann. Probab., 46(2):865–896, 2018.
  * [Tal98] Michel Talagrand.  Rigorous results for the Hopfield model with many patterns.  Probab. Theory Related Fields, 110(2):177–276, 1998.
  * [Tal06] Michel Talagrand.  The Parisi formula.  Ann. of Math. (2), 163(1):221–263, 2006.
  * [Tal11] Michel Talagrand.  Mean field models for spin glasses. Volume I, volume 54 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics].  Springer-Verlag, Berlin, 2011.  Basic examples.

##  4 Appendix

In this section, we prove the technical lemmas (Lemma 3.3, 3.6, 3.14) that
characterize the recursive relations of general moments using cavity method.
Recall the decomposition of general moments given in (3.2)

$\displaystyle\nu\left({}_{k,l}T_{k,l}^{h(k,l)}{}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right)$
$\displaystyle=\nu({}_{v\geqslant 1}U_{v})$ (4.1)
$\displaystyle=\nu(\varepsilon(1){}_{v>1}U^{-}_{v})+\frac{1}{N}\sumop\slimits@_{u\geqslant
2}\nu(\varepsilon(1){}_{v\neq 1,u}U^{-}_{v})+O(H+1).$ (4.2)

Note that the first term is of order $H-1$, we should apply the second order
approximation, (2.7) and compute its first order derivative at time $0$. With
some abuse of notation, we will always assume the first term $U_{1}$
corresponds to the type of basis, $T_{1},S_{1},T,S$, that we wish to "peel
off" from the expression. Note that regardless of the type of $U_{1}$,
$\nu_{0}(\varepsilon(1))=0$ by symmetry.

$\displaystyle\nu(\varepsilon(1){}_{v>1}U^{-}_{v})$
$\displaystyle=\nu^{\prime}_{0}(\varepsilon(1){}_{v>1}U^{-}_{v})+O(H+1).$
(4.3)

This section is dedicated to characterizing the structure of such terms.

###  4.1 Proof of Lemma 3.3

######  Lemma 4.1.

Suppose $h(1,2)\geqslant 1$ and $U_{1}$ corresponds to a copy of $T_{1,2}$

$\nu^{\prime}_{0}(\varepsilon(1){}_{v>1}U^{-}_{v})=\beta^{2}A\nu(g_{1,2}(h(1,2)))+O(H+1).$

###### Proof of lemma 3.3.

Let $U_{1}$ be of type $T_{1,2}$, by Claim 2.1,

$\varepsilon(1)=\varepsilon_{1_{1},1_{2}}-\varepsilon_{1_{1},1_{4}}-\varepsilon_{1_{3},1_{2}}+\varepsilon_{1_{3},1_{4}},$

By (2.4), denote $m$ as the total number of replicas used by
$\varepsilon(1){}_{v>1}U^{-}_{v}$.

$\displaystyle\nu^{\prime}_{0}(\varepsilon(1){}_{v>1}U^{-}_{v})$
$\displaystyle=\frac{\beta^{2}}{2}\sumop\slimits@_{1\leqslant a,b\leqslant
2m}\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-\mu_{a,b}){}_{v>1}U^{-}_{v})-\mathcal{R}_{m,\varepsilon(1){}_{v>1}U^{-}_{v}}.$

As we noted in (3.5), $\nu_{0}(\varepsilon(1)\varepsilon_{a,b})=0$ unless
$\varepsilon_{a,b}$ is a monomial in $\varepsilon(1)$. Since $1_{3},1_{4}$ can
not appear in any other terms $\\{U_{v}:v>1\\}$,
$\nu_{0}(\varepsilon(1)\varepsilon_{a,b})=A>0$ only when $a,b\subset V_{1}$
and $a\neq b$. Summing over all such pairs of replicas gives the desired
result.

$\displaystyle\nu^{\prime}_{0}(\varepsilon(1){}_{v>1}U^{-}_{v})$
$\displaystyle=A\beta^{2}\nu(g_{1,2}(h(1,2)))+O(H+1).$ (4.4)

∎

###  4.2 Proof of Lemma 3.9, 3.10

In this section, we derive the structure of (4.3) with

$\nu({}_{v\geqslant
1}U_{v})=\nu\left({}_{k}T_{k}^{h(k)}T^{h}{}_{l}S_{l}^{h^{\prime}(l)}S^{h^{\prime}}\right).$

Recall that the total moments of each type are

$h_{T}=\sumop\slimits@_{k}h(k),\quad
h_{S}=\sumop\slimits@_{l}h^{\prime}(l),\quad H_{1}=h_{T}+h_{S}+h+h^{\prime}.$

and $g_{1}$ is the function indexed by moments of $T_{1},S_{1}$ s.t.

$\nu(g_{1}(h(1),h^{\prime}(1)))=\nu({}_{1\leqslant v\leqslant H_{1}}U_{v}).$

We begin by introducing the following notations for referencing different
terms in $\nu^{\prime}_{t}(\cdot)$ given in (2.4). Denote $m$ as as the number
of total replicas used by $g(h(1),h^{\prime}(1))$

$m:=n+2h_{T}+h_{S}$

For $a\in[2m]$, denote $a^{{}^{\prime\prime}}$ as the new replicas that first
appear in (2.4). For $a,b\in[2m]$, let
$\text{sgn}(a,b):=-1^{|\\{a,b\\}\cap[m]|}$ .

Our goal is to compute the following derivative with $U_{1}$ corresponding to
a copy of $T_{1}$ or $S_{1}$

$\displaystyle\nu_{0}^{\prime}(\varepsilon(1){}_{v>1}U^{-}_{v})$
$\displaystyle=\frac{\beta^{2}}{2}\sumop\slimits@_{1\leqslant a,b\leqslant
2m}\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-Q_{a,b}){}_{v>1}U^{-}_{v})-\mathcal{R}_{m,\varepsilon(1){}_{v>1}U^{-}_{v}}.$
(4.5)

In both cases, we need to consider contributions from terms

$\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-\mu_{a,b}){}_{v>1}U^{-}_{v})-\mathcal{R}_{m,\varepsilon(1){}_{v>1}U^{-}_{v}}.$

Before proceeding, let’s exploit the symmetry of $T_{1}$ and $S_{1}$ to rule
out certain types of replica pair $(a,b)$.

######  Lemma 4.2.

Suppose $\varepsilon(v)=\varepsilon_{v_{1},v_{3}}-\varepsilon_{v_{2},v_{3}}$
or $\varepsilon(v)=\varepsilon_{v_{1},v_{1}}-\varepsilon_{v_{2},v_{2}}$. If
$|\\{a,b\\}\cap\\{v_{1},v_{2}\\}|\neq 1$, then

$\nu_{0}(\varepsilon(v)\varepsilon_{a,b})=0.$

Moreover, for any replica $k\in[2m]\backslash\\{v_{1},v_{2}\\}$, and
$(a,b)\in\\{(v_{1},k),(v_{2},k)\\}$, we have

$\displaystyle\nu_{0}(\varepsilon(v)\varepsilon_{v_{1},k})=-\nu_{0}(\varepsilon(v)\varepsilon_{v_{2},k}).$
(4.6)

###### Proof.

The value of $\nu(\varepsilon_{a,b}\varepsilon_{c,d})$ depends only on the
size of union and intersection of $\\{a,b\\}$ and $\\{c,d\\}$. Check that if
$|\\{a,b\\}\cap\\{v_{1},v_{2}\\}|\neq 1$, the two terms in
$\varepsilon(v)\varepsilon_{a,b}$ are equvilent:

If $|\\{a,b\\}\cap\\{v_{1},v_{2}\\}|=0$, we have

$\nu_{0}(\varepsilon_{v_{1},v_{3}}\varepsilon_{a,b})=\nu_{0}(\varepsilon_{1,2}\varepsilon_{3,4})=\nu_{0}(\varepsilon_{v_{2},v_{3}}\varepsilon_{a,b}),$

and

$\nu_{0}(\varepsilon_{v_{1},v_{1}}\varepsilon_{a,b})=\nu_{0}(\varepsilon_{1,1}\varepsilon_{2,3})=\nu_{0}(\varepsilon_{v_{2},v_{2}}\varepsilon_{a,b}).$

If $|\\{a,b\\}\cap\\{v_{1},v_{2}\\}|=2$, we have

$\nu_{0}(\varepsilon_{v_{1},v_{3}}\varepsilon_{v_{1},v_{2}})=\nu_{0}(\varepsilon_{1,2}\varepsilon_{1,3})=\nu_{0}(\varepsilon_{v_{2},v_{3}}\varepsilon_{v_{1},v_{2}}),$

and

$\nu_{0}(\varepsilon_{v_{1},v_{1}}\varepsilon_{v_{1},v_{2}})=\nu_{0}(\varepsilon_{1,1}\varepsilon_{1,2})=\nu_{0}(\varepsilon_{v_{2},v_{2}}\varepsilon_{v_{1},v_{2}}).$

Suppose $(a,b)\in\\{\\{v_{1},k\\},\\{v_{2},k\\}\\}$. To check (4.6):

If $U_{v}$ corresponds to $T_{1}$,

$\displaystyle\nu_{0}(\left(\varepsilon_{v_{1},v_{3}}-\varepsilon_{v_{2},v_{3}}\right)\varepsilon_{v_{1},k})$
$\displaystyle=\nu_{0}(\varepsilon_{v_{1},v_{1}}\varepsilon_{v_{3},k}-\varepsilon_{v_{1},v_{2}}\varepsilon_{v_{3},k}),$
$\displaystyle=\nu_{0}(\varepsilon_{v_{2},v_{2}}\varepsilon_{v_{3},k}-\varepsilon_{v_{1},v_{2}}\varepsilon_{v_{3},k}),$
$\displaystyle=-\nu_{0}(\left(\varepsilon_{v_{1},v_{3}}-\varepsilon_{v_{2},v_{3}}\right)\varepsilon_{v_{2},k}).$

If $U_{v}$ corresponds to $S_{1}$,

$\displaystyle\nu_{0}(\left(\varepsilon_{v_{1},v_{1}}-\varepsilon_{v_{2},v_{2}}\right)\varepsilon_{v_{1},k})$
$\displaystyle=\nu_{0}(\varepsilon_{v_{1},v_{1}}\varepsilon_{v_{1},k}-\varepsilon_{v_{2},v_{2}}\varepsilon_{v_{1},k}),$
$\displaystyle=\nu_{0}(\varepsilon_{v_{2},v_{2}}\varepsilon_{v_{2},k}-\varepsilon_{v_{1},v_{1}}\varepsilon_{v_{2},k}),$
$\displaystyle=-\nu_{0}(\left(\varepsilon_{v_{1},v_{1}}-\varepsilon_{v_{2},v_{2}}\right)\varepsilon_{v_{2},k}).$

where the second equality follows from $k\notin\\{v_{1},v_{2}\\}$ and by the
linearity of expectation, exchanging the index of replica doesn’t affect the
expectation under $\nu_{0}$. ∎

By Lemma 4.2, the set of replica pairs $(a,b)$ s.t.
$\nu_{0}(\varepsilon(1)(\varepsilon_{a,b}-q))\neq 0$ is given by

$\operatorname{\mathcal{P}}_{1}=\\{(a,b):|\\{a,b\\}\cap\\{1_{1},1_{2}\\}|=1;1\leqslant
a,b\leqslant[2m]\\}.$

Summing up all non-trivial terms in (4.5), the goal can be simplified to

$\nu_{0}^{\prime}(\varepsilon(1){}_{v>1}U^{-}_{v})=\frac{\beta^{2}}{2}\sumop\slimits@_{(a,b)\in\operatorname{\mathcal{P}}_{1}}sgn(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-\mu_{a,b}){}_{v>1}U^{-}_{v}).$

Now we proceed to study the case when $U_{1}$ corresponds to a copy of
$T_{1}$.

######  Lemma 4.3 (restatment of Lemma 3.9).

For $h(1)\geqslant 1$ and $h^{\prime}(1)\geqslant 0$, suppose $U_{1}$
corresponds to a copy of $T_{1}$

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})=$
$\displaystyle\beta^{2}(F-3G)\nu(g_{1}(h(1),h^{\prime}(1))$
$\displaystyle+\frac{\beta^{2}}{2}H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)$
$\displaystyle+\beta^{2}(h(1)-1)GA_{2}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1)))$
$\displaystyle+O(H_{1}+1).$

###### Proof of Lemma 3.9.

The proof follows the same idea as in Section 2.2.

Let’s assume that $h(1)\geqslant 1$, and $U_{1}$ corresponds to a copy of
$T_{1}$ (By definition, this corresponds to $1_{1}=1$ but we will use the
notation $1_{1}$ for the sake of consistency.).

$\varepsilon(1)=\varepsilon_{1_{1},1_{3}}-\varepsilon_{1_{2},1_{3}}.$

To compute $\nu_{0}^{\prime}(\varepsilon(1){}_{v>1}U^{-}_{v})$, we will count
the contribution of terms from $(a,b)\in\operatorname{\mathcal{P}}_{1}$. By
Lemma 4.2, it make sense to group $(a,b)$ based on
$\\{a,b\\}\cap[2m]\backslash\\{v_{1},v_{2}\\}$. For each of those subset, we
will first apply (4.6) to compute the
$\nu_{0}(\varepsilon(1)\varepsilon_{a,b})$ part, then characterize the
structure of $\nu_{0}((R^{-}_{a,b}-\mu_{a,b}){}_{v>1}U^{-}_{v})$.

  * •

If $a=b$: In this case, we have $(a,b)\in\\{(v_{1},v_{1}),(v_{2},v_{2})\\}$.
By a similar argument as the proof of (4.6),

$\nu_{0}(\varepsilon(1)\varepsilon_{1_{1},1_{1}})=-\nu_{0}(\varepsilon(1)\varepsilon_{1_{2},1_{2}})=H$

The sum of the two terms are

$H\nu_{0}((R^{-}_{1_{1},1_{1}}-R^{-}_{1_{2},1_{2}}){}_{v>1}U^{-}_{v})=H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)+O(H_{1}+1).$

  * •

If $\\{a,b\\}\in\\{\\{1_{1},1_{3}\\},\\{1_{2},1_{3}\\}\\}$. For those terms,

$\nu_{0}(\varepsilon(1)\varepsilon_{1_{1},1_{3}})=F.$

Summing over the two terms gives

$2F\nu_{0}((R^{-}_{1_{1},1_{3}}-R^{-}_{1_{2},1_{3}}){}_{v>1}U^{-}_{v})=2F\nu(g_{1}(h(1),h^{\prime}(1)))+O(H_{1}+1).$

  * •

To sum over remaining $(a,b)\in\operatorname{\mathcal{P}}_{1}$ corresponds to
iterating over all replicas $k\in[2m]\backslash V_{1}$ and sum over pairs
$\\{1_{1},k\\},\\{1_{2},k\\}$. It is easier if we account for
$k\in[m]\backslash V_{1}$ and the corresponding new replica
$k^{\prime}\in[2m]\backslash[m]$ introduced by Lemma 2.4.

    * –

For $k\in[m]\backslash V_{1}$, let $k^{\prime}:=m+k$ be the corresponding new
replica from Lemma 2.4. Summing over all four terms gives

$\displaystyle
G\nu_{0}((R^{-}_{1_{1},k}-R^{-}_{1_{2},k}-R^{-}_{1_{1},k^{\prime}}+R^{-}_{1_{2},k^{\prime}}){}_{v>1}U^{-}_{v})$
$\displaystyle=G\nu((T_{1_{1},k}-T_{1_{2},k}-T_{1_{1},k^{\prime}}+T_{1_{2},k^{\prime}}){}_{v>1}U_{v})+O(H_{1}+1).$

We now will explore the structure of this term by viewing it as some general
moment of $T_{1,2},T_{1},S_{1},T,S$. By Theorem 3.2, only even moments of
$T_{k,l}$ give a non-trivial contribution to the sum. By construction, the
replica $k^{\prime}$ and $1_{2}$ do not appear in any other term $U_{v}$. Thus
the non-trivial portion of
$(T_{1_{1},k}-T_{1_{2},k}-T_{1_{1},k^{\prime}}+T_{1_{2},k^{\prime}}){}_{v>1}U_{v}$
can only come from

$\nu(T_{1_{1},k}{}_{v>1}U_{v})$

Now the goal becomes checking if $T_{1_{1},k}$ occurs in ${}_{v>1}U_{v}$.
Becuase $g_{1}(h(1),h^{\prime}(1))$ contains terms of the form
$T_{k},S_{k},T,S$, the only terms where $\\{1_{1},k\\}$ appear together are
terms correspond to $T_{1}$ or $S_{1}$. We will show that only $T_{1}$ terms
are non-trivial. Let’s first rewrite $U_{v}$ using the "basis". For $U_{v}$
corresponds to a copy of $T_{1_{1}}$

$U_{v}=T_{v_{1},v_{3}}-T_{v_{2},v_{3}}+T_{v_{1}}-T_{v_{2}}.$

Thus if $k=u_{3}$ for some $u>1$, then

$\nu(T_{1_{1},k}{}_{v>1}U_{v})=\nu(T_{1_{1},k}^{2}{}_{v\neq
1,u}U_{v})=A_{2}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1)))+O(H_{1}+1).$

If $k$ appears in a copy of $S_{1_{1}}$, $U_{u}$, then the corresponding term
becomes

$\nu(T_{1_{1},k}(S_{1_{1}}-S_{k}){}_{v\neq 1,u}U_{v}))=O(H_{1}+1).$

Thus we only need to count countribution of $\\{1_{1},k\\}$ where
$\\{1_{1},k\\}$ appear together in some term $U_{v}$ where
$V_{v}=\\{v_{1},v_{2},v_{3}\\}$ and by definition, $v_{1}=1_{1}$ $v_{2}=k$.

$\displaystyle 2\times(h(1)-1)GA_{2}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1))).$

    * –

The only $k$ that are not counted now are those that correspond to replicas in
$V_{1}$. For each such $k$, since such $k$ as well as $1_{2}$ do not appear in
any other terms in ${}_{v>1}U^{-}_{v}$

$-G\nu_{0}((R^{-}_{1_{1},k}-R^{-}_{1_{2},k}){}_{v>1}U^{-}_{v})=-G\nu(T_{1_{1}}{}_{v>1}U^{-}_{v})+O(H_{1}+1).$

Summing over terms from all $6$ pairs gives

$-2\times 3G\nu_{0}(g_{1}(h(1),h^{\prime}(1))).$

Combine all the terms gives

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})$
$\displaystyle=\frac{\beta^{2}}{2}H\nu(g_{1}(h(1)-1,h^{\prime}(1)+1)$
$\displaystyle+\beta^{2}F\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle+\beta^{2}(h(1)-1)GA_{2}^{2}\nu(g_{1}(h(1)-2,h^{\prime}(1)))$
$\displaystyle-\beta^{2}3G\nu_{0}(g_{1}(h(1),h^{\prime}(1)))$

Rearrange gives the desired result. ∎

######  Lemma 4.4 (restatement of Lemma 3.10).

If $h^{\prime}(1)\geqslant 1$ and $h(1)\geqslant 0$, suppose $U_{1}$
corresponds to a copy of $S_{1}$, then

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})=$
$\displaystyle\frac{\beta^{2}}{2}D\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle+\beta^{2}h(1)EA_{2}^{2}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))$
$\displaystyle-2\beta^{2}E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1)$
$\displaystyle+O(H_{1}+1)).$

###### Proof of Lemma 3.10.

The proof is similar to that of Lemma 3.9, but with

$\varepsilon(1)=\varepsilon_{1_{1},1_{1}}-\varepsilon_{1_{2},1_{2}}.$

We included it here for completeness. Let’s count the contribution from each
pair $(a,b)$ in $\operatorname{\mathcal{P}}_{1}$.

  * •

$a=b\in V_{1}$: In this case, we have
$(a,b)\in\\{(v_{1},v_{1}),(v_{2},v_{2})\\}$. By a similar argument as the
proof of (4.6),

$\nu_{0}(\varepsilon(1)\varepsilon_{1_{1},1_{1}})=-\nu_{0}(\varepsilon(1)\varepsilon_{1_{2},1_{2}})=D.$

Combining the two terms gives

$D\nu_{0}((R^{-}_{1_{1},1_{1}}-R^{-}_{1_{2},1_{2}}){}_{v>1}U^{-}_{v})=D\nu(g_{1}(h(1),h^{\prime}(1)))+O(H_{1}+1).$

  * •

$a\neq b$.

    * –

For each replica $k\in[m]\backslash V_{1}$, let $k^{\prime}=m+k$ be the
corresponding new replica introduced by the derivative formula. WLOG, first
fix $a\in V_{1}$. Combine terms corresponds to $b\in\\{k,k^{\prime}\\}$, we
have

$\displaystyle
E\nu_{0}((R^{-}_{1_{1},k}-R^{-}_{1_{2},k}-R^{-}_{1_{1},k^{\prime}}+R^{-}_{1_{2},k^{\prime}}){}_{v>1}U^{-}_{v})$
$\displaystyle=E\nu_{0}((T_{1_{1},k}-T_{1_{2},k}-T_{1_{1},k^{\prime}}+T_{1_{2},k^{\prime}}){}_{v>1}U_{v}).$

Following the same argument as in the corresponding type of pair in Lemma 3.9,
the only non-trival contributions come from when $\\{1_{1},k\\}$ appears in
some $V_{u}$ where $U_{u}$ is a copy of $T_{1_{1}}$ and $1_{1}=v_{1}$,
$k=v_{2}$. The contribution from such terms are

$E\nu_{0}(T_{1_{1},k}^{2}{}_{v\neq
1,u}U_{v})=EA_{2}^{2}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))+O(H_{1}+1)$

Summing up all contributions from such terms gives

$2\times h(1)EA_{2}^{2}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))+O(H_{1}+1).$

    * –

The only $k\in[2m]\backslash V_{1}$ that are not counted are the two new
replicas corresponding to $V_{1}$. For each such replica, the contribution is

$-E\nu_{0}((R^{-}_{1_{1},k}-R^{-}_{1_{2},k}){}_{v>1}U^{-}_{v})=-E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1))+O(H_{1}+1).$

The total contribution is

$2\times(-2E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1))+O(H_{1}+1)).$

Summing over contribution from all pairs in $\operatorname{\mathcal{P}}_{1}$,

$\displaystyle\nu(\varepsilon(1){}_{v>1}U_{v})=$
$\displaystyle\frac{\beta^{2}}{2}D\nu(g_{1}(h(1),h^{\prime}(1)))$
$\displaystyle+\beta^{2}h(1)EA_{2}^{2}\nu(g_{1}(h(1)-1,h^{\prime}(1)-1))$
$\displaystyle-2\beta^{2}E\nu(g_{1}(h(1)+1,h^{\prime}(1)-1)$
$\displaystyle+O(H_{1}+1)).$

∎

###  4.3 Proof of Lemma 3.18, 3.19

Recall that

$T^{h}S^{h^{\prime}}=g(h,h^{\prime})={}_{v\geqslant
1}U_{v},\quad\text{and}\quad\varepsilon(v)=\varepsilon_{v_{1},v_{2}}-Q_{v_{1},v_{2}}.$

For $T^{h}S^{h^{\prime}}$, we have the additional property that if $v\neq
v^{\prime}$, then

$\displaystyle V_{v}\cap V_{v^{\prime}}=\emptyset.$ (4.7)

As in the previous section, we first introduce some notation to characterize
the formula in (2.4). Denote the number of total replicas appear
$g(h,h^{\prime})$ as

$m:=2h+h^{\prime}.$

Denote $a^{{}^{\prime\prime}}$ as the new replicas in
$\nu^{\prime}_{0}(g(h,h^{\prime}))$ for each $a\in[2m]$, and
$\text{sgn}(a,b):=-1^{|\\{a,b\\}\cap[m]|}$.

Our goal is to compute the following derivative of $\nu(g(h,h^{\prime}))$,

$\displaystyle\nu_{0}^{\prime}(\varepsilon(1){}_{v>1}U^{-}_{v})$
$\displaystyle=\frac{\beta^{2}}{2}\sumop\slimits@_{1\leqslant a,b\leqslant
2m}\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-\mu_{a,b}){}_{v>1}U^{-}_{v})-R_{n,g},$
(4.8) $\displaystyle=\frac{\beta^{2}}{2}\sumop\slimits@_{1\leqslant
a,b\leqslant
2m}\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-Q_{a,b}){}_{v>1}U^{-}_{v})$
(4.9) $\displaystyle-\frac{\beta^{2}}{2}\sumop\slimits@_{1\leqslant
a,b\leqslant
2m}\text{sgn}(a,b)\nu_{0}(\varepsilon(1)\varepsilon_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}})\nu_{0}((R^{-}_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}}-Q_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}}){}_{v>1}U^{-}_{v}).$
(4.10)

Unlike in the case of $T_{1},S_{1}$,
$\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\neq 0$ for all pairs of replica
$(a,b)$. To simplify the computation, for each $(a,b)\in[2m]\times[2m]$,
consider the corresponding term

$\displaystyle
D_{a,b}:=\nu_{0}(\varepsilon(1)\varepsilon_{a,b})\nu_{0}((R^{-}_{a,b}-Q_{a,b}){}_{v>1}U^{-}_{v})-\nu_{0}(\varepsilon(1)\varepsilon_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}})\nu_{0}((R^{-}_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}}-Q_{a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}}){}_{v>1}U^{-}_{v}).$
(4.11)

Then we can rewrite 2.4 as a sum of all such pairs $(a,b)$.

$\nu_{0}^{\prime}(\varepsilon(1){}_{v>1}U^{-}_{v})=\frac{\beta^{2}}{2}\sumop\slimits@_{(a,b)\in[2m]\times[2m]}D_{a,b}.$

We first characterize paris of $(a,b)$ s.t. $D_{a,b}=0$. Observe that if
$\\{a,b\\}\cap[m]=\emptyset$, neither $(a,b)$ nor
$a^{{}^{\prime\prime}},b^{{}^{\prime\prime}}$ appear in any terms $U^{-}_{v}$.