# Hypergraph counting and Mixed $p$-spin Glass Models under Replica Symmetry

Partha S. Dey⋆ and  Qiang Wu† Department of Mathematics, University of
Illinois at Urbana-Champaign, 1409 W Green Street, Urbana, Illinois 61801
<EMAIL_ADDRESS>
(Date: August 28, 2024)

###### Abstract.

Fluctuation problems in mean-field spin glass theory enjoy a rich history. In
this paper, we address this question in the high-temperature regime by
studying the fluctuation of the partition function in the general mixed
$p$-spin glass models under the weak external field assumption: $h=\rho
N^{-\alpha},\rho>0,\alpha\in[\nicefrac{{1}}{{4}},\infty]$. By extending the
cluster expansion approach to this generic setting, we convert the fluctuation
problem as a hypergraph counting problem and thus obtain a new multiple-
transition phenomenon in terms of the fluctuation of the partition function. A
by-product of our results is an explicit characterization of the critical
inverse temperature for general spin glass models. In particular, all our
fluctuation results hold up to the critical inverse temperature. Combining
with multivariate Stein’s method for exchangeable pairs, we also obtain an
explicit convergence rate under appropriate moment assumptions on the general
symmetric disorder coupling. Our results have several further implications.

– First, our approach works for both even and odd pure $p$-spin models. The
leading cluster structures in the odd $p$ case are different and more involved
than in the even $p$ case. This gives a combinatorial explanation for the
folklore that odd $p$-spin is usually more complicated than even $p$.

– Second, in the mixed $p$-spin setting, the cluster structures differ
depending on the relation between the minimum effective even and odd
$p$-spins: $p_{e}$ and $p_{o}$, respectively. As an example, at $h=0$, there
are three sub-regimes: $p_{e}<p_{o},p_{o}<p_{e}<2p_{o},p_{e}\geqslant 2p_{o}$,
wherein the first and third ones, the mixed $p$-spin model behaves essentially
like a pure $p$-spin model, and only in the second regime, it is more like a
mixture. This presents another criterion for classifying mean-field spin glass
models compared to the work of Auffinger and Ben Arous (Ann. Probab. 41
(2013), no. 6, 4214–4247), where the idea is based on complexity computations
for spherical models.

– Third, our framework naturally implies a multi-scale fluctuation phenomenon
conjectured in the work of Bovier and Schertzer (arXiv:2205.15080), at $h=0$
case for pure $p$-spin model. Our results suggest that this phenomenon holds
for general mixed $p$-spin models under weak external fields, including $h=0$.
We also extend the approach in this paper to general multi-species mixed
$p$-spin models.

###### Key words and phrases:

Spin glass, Phase transition, Central limit theorem, Cluster expansion.

###### 2020 Mathematics Subject Classification:

Primary: 82B26, 82B44, 60F05.

###### Contents

  1. 1 Introduction and Main Results
  2. 2 Results for Mixed $p$-Spin Models
  3. 3 Results for Multi-Species Mixed $p$-Spin Models
  4. 4 Large Hypergraph Decay Regime
  5. 5 Dominant Clusters for $p$-Spin Models
  6. 6 CLT using Stein’s Method
  7. 7 Stein’s Method for Mixed $p$-Spin Models
  8. 8 Proof of Main Results
  9. 9 Open Questions

##  1\. Introduction and Main Results

Cluster expansion, a powerful tool rooted in mathematical physics, has been
used to rigorously study many problems in a wide range of statistical
mechanics models. For example, cluster expansion was used in developing the
Pigorov-Sinai theory to prove phase transitions for the Ising model. The rough
idea is to decompose the configuration space into clusters of simple
structures, then deal with the simple clusters instead to understand the log-
partition function. For more details on this classical treatment, we invite
interested readers to [FV18].

However, there are scarce results on cluster expansion to derive rigorous
solutions in the disordered spin system, such as spin glass models. In the
seminal paper [ALR87], Aizenman, Lebowitz, and Ruelle first utilized the idea
of cluster expansion in the celebrated Sherrington-Kirkpatrick (SK) spin glass
model and obtained a rigorous solution of the model by proving a Gaussian
central limit theorem of free energy. The results hold at high temperatures in
the zero external field. The clusters they obtained are a collection of
finite-sized simple loops in a complete graph. There are similar results for
the diluted SK model [Ko06] and Hopfield model [CKT04, ST92]. However, all
these results are essentially restricted to the two-spin and zero external
field cases. It was believed that the cluster expansion would not work in the
presence of an external field [Tal11a, Tal11b]. Recently, in [DW21], the
authors made some progress by showing that under weak external fields, the
cluster expansion still works, but there is a new cluster structure, _i.e.,_ a
collection of simple paths with finite length. Based on this, they established
the transition behavior of the fluctuation of free energy from zero to a weak
external field.

In this paper, we extend this technique to a much more general setting,
_i.e.,_ the mixed $p$-spin glass models. Compared to the SK model, where the
interactions are present only among two sites, the pure $p$-spin model has
interactions among any finite fixed $p\geqslant 3$ sites. Based on the
combinatorial interpretation obtained in the SK model, one needs to identify
the cluster structures on complete $p$-uniform hypergraphs. However, even in
the zero external field case, the first challenge is what should be the
appropriate notions of “loops”. On the other hand, even if one can figure out
these “loop” structures in a hypergraph, counting the number of such
structures is more complicated than doing it on the usual complete graph.
Besides, a more challenging task is dealing with general mixed $p$-spin
models, where all possible hyperedges can appear. Similar challenges are
present when adding an external field into the system.

In the following part, we will answer all the above questions by
characterizing the cluster structures in all different cases. As a result, we
obtain new fluctuation results for the partition function in general mixed
$p$-spin glass models and new multiple-transition behavior from zero to the
weak external field. This naturally implies the fluctuation results for the
free energy. The idea is robust enough to extend to other variants, such as
the multi-species mixed $p$-spin models. We further emphasize that in many
classical results of $p$-spin glass models, it has been a mystery why the pure
odd and even $p$-spin model behave quite differently. For example, many
important results [MTal06, CHL18, Chen14, Chen14b, CDP17, GJ21] can only be
proven in even $p$ case rigorously. Our results give a combinatorial
explanation of this folklore belief. Finally, a byproduct of our analysis is a
simple characterization of the critical inverse temperature $\beta_{c}$ for
general spin glass models, which is of independent interest. This enables us
to establish all the fluctuation results up to the critical inverse
temperature.

Before presenting the main results, let us introduce the definition of
$p$-spin glass models.

###  1.1. Setting

We start with the pure $p$-spin glass model. Consider the $N$ dimensional
hypercube $\Sigma_{N}:=\\{-1,+1\\}^{N}$. For a configuration
$\boldsymbol{\sigma}:=(\sigma_{1},\sigma_{2},\ldots,\sigma_{N})\in\Sigma_{N}$,
the pure $p$-spin _Hamiltonian_ is given by

$\displaystyle
H_{N,p}(\boldsymbol{\sigma}):=\frac{1}{N^{(p-1)/2}}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}J_{\boldsymbol{i}}\cdot\prod_{j=1}^{p}\sigma_{i_{j}},$
(1)

where the random variables
$(J_{\boldsymbol{i}})_{\boldsymbol{i}\in\mathscr{E}_{N,p}}$, indexed by the
set of $p$-tuples
$\mathscr{E}_{N,p}:=\\{\boldsymbol{i}=(i_{1},i_{2},\ldots,i_{p})\mid
1\leqslant i_{1}<i_{2}<\cdots<i_{p}\leqslant N\\})$, are i.i.d. symmetrically
distributed with variance one. For convenience, we define
$\sigma_{\boldsymbol{i}}:=\prod_{j=1}^{p}\sigma_{i_{j}}$.

One can also consider a general mixed $p$-spin model, where the Hamiltonian is
a mixture of $H_{N,p}$ over $p\geqslant 2$.

######  Assumption 1.1 (Mixed $p$-spin Hamiltonian).

The mixed $p$-spin Hamiltonian is given by

$\displaystyle H_{N}(\boldsymbol{\sigma}):=\sum_{p\geqslant 2}\theta_{p}\cdot
H_{N,p}(\boldsymbol{\sigma}),$ (2)

where $\boldsymbol{\theta}=(\theta_{p})_{p\geqslant 2}$ satisfies
$\sum_{p\geqslant 2}\theta_{p}^{2}/(p-2)!<\infty$.

It is easy to calculate the covariance of the Hamiltonian, for
$\boldsymbol{\sigma},\boldsymbol{\tau}\in\Sigma_{N}$, as

$\displaystyle\operatorname{\mathds{E}}H_{N}(\boldsymbol{\sigma})H_{N}(\boldsymbol{\tau})=N\cdot\xi_{N}(R_{\boldsymbol{\sigma},\boldsymbol{\tau}}),\text{
where
}R_{\boldsymbol{\sigma},\boldsymbol{\tau}}:=\frac{1}{N}\sum_{i}\sigma_{i}\tau_{i}$

and the function $\xi_{N}$111One can explicitly write down $\xi_{N}(x)$ as
$\sum_{p\geqslant 2}\frac{\theta_{p}^{2}}{p!}\cdot(-N)^{-p}B_{p}(-Nx,-1!\cdot
N,-2!\cdot Nx,\ldots,-(p-1)!\cdot Nx^{\mathds{1}\\{p\text{ odd}\\}})$ where
$B_{p}$ is the $p^{\text{th}}$ complete exponential Bell polynomial [andr84].
It is easy to check that
$\left\|\xi_{N}-\xi\right\|_{\infty}\leqslant\sum_{p\geqslant
2}\frac{\theta_{p}^{2}}{p!}(N^{p}-(N)_{p})/N^{p}.$ Moreover, it is easy to
check that for any $p\geqslant 2$, $(-\sqrt{N})^{-p}B_{p}(-\sqrt{N}x,-1!\cdot
N,-2!\sqrt{N}\cdot x,\ldots)$ converges to the Hermite polynomial
$H_{p}(x)=B_{p}(x,-1,0,\ldots)$ as $N\to\infty$. satisfies
$\left\|\xi_{N}-\xi\right\|_{\infty}\leqslant\xi^{\prime\prime}(1)/2N$ where

$\displaystyle\xi(x):=\sum_{p\geqslant 2}\frac{\theta_{p}^{2}}{p!}\cdot
x^{p}\text{ for }x\in[-1,1]$

and $\xi^{\prime\prime}(1)=\sum_{p\geqslant 2}\theta_{p}^{2}/(p-2)!.$ The
function $\xi_{N}$ is known as the structure function and can encode mean-
field spin glass models. The values $\theta_{k}=\mathds{1}_{\\{k=p\\}}$
corresponds to the pure $p$-spin model.

For all these models, adding an external magnetic field
$h\sum_{i=1}^{N}\sigma_{i}$ could create drastic changes to the system. Using
the cluster expansion technique, we obtain a detailed transition picture of
those changes in the mixed $p$-spin and other generalized models.

We recall the standard definitions first. The partition function at inverse
temperature $\beta>0$ and external field $h\geqslant 0$ is defined as

$\displaystyle Z_{N}(\beta,h):=\sum_{\boldsymbol{\sigma}\in\Sigma_{N}}e^{\beta
H_{N}(\boldsymbol{\sigma})+h\sum_{i=1}^{N}\sigma_{i}}$

and the free energy as

$\displaystyle F_{N}(\beta,h):=\frac{1}{N}\log Z_{N}(\beta,h).$

For notational convenience, we may drop the dependence on terms such as
$N,\beta,h$ unless it causes any confusion. It is easy to prove the following
decomposition of the partition function

$\displaystyle Z_{N}(\beta,h)=(2\cosh
h)^{N}\cdot\bar{Z}_{N}(\beta)\cdot\hat{Z}_{N}(\beta,h)$ (3)

where

$\displaystyle\bar{Z}_{N}(\beta)$ $\displaystyle:=\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\cosh(\beta\theta_{p}\cdot
J_{\boldsymbol{i}}/N^{(p-1)/2})\text{ and }$ (4)
$\displaystyle\hat{Z}_{N}(\beta,h)$
$\displaystyle:=\operatorname{\mathds{E}}_{h}\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}(1+\sigma_{\boldsymbol{i}}\tanh(\beta\theta_{p}\cdot
J_{\boldsymbol{i}}/N^{(p-1)/2})).$ (5)

Here the expectation $\operatorname{\mathds{E}}_{h}$ is w.r.t.
$(\sigma_{i})_{1\leqslant i\leqslant N}$ being i.i.d. with
$\operatorname{\mathds{P}}_{h}(\sigma_{1}=\pm 1)=e^{\pm h}/(e^{h}+e^{-h})$ and
mean $\hat{h}:=\tanh h$. Note that,

$\displaystyle\frac{1}{N}\log\bar{Z}_{N}(\beta)=\frac{1}{N}\sum_{p\geqslant
2}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\log\cosh(\beta\theta_{p}\cdot
J_{\boldsymbol{i}}/N^{(p-1)/2})\to\frac{1}{2}\beta^{2}\sum_{p\geqslant
2}\theta_{p}^{2}/p!=\frac{1}{2}\beta^{2}\xi(1)$

as $N\to\infty$ almost surely and in expectation; and

$\displaystyle
N^{p_{m}-2}\cdot\operatorname{Var}(\log\bar{Z}_{N}(\beta))\to\frac{\beta^{4}\theta_{p_{m}}^{4}}{4p_{m}!}\operatorname{Var}(J^{2})$

where $p_{m}:=\min\\{p\geqslant 2\mid\theta_{p}\neq 0\\}$. We will mainly
focus on analyzing the behavior of $\hat{Z}_{N}(\beta,h)$ as $N\to\infty$.

######  Assumption 1.2 (Moment assumptions on Disorder).

We will assume that the random variables
$(J_{\boldsymbol{i}})_{\boldsymbol{i}\in\cup_{p\geqslant 2}\mathscr{E}_{N,p}}$
are i.i.d. symmetrically distributed with mean zero, variance one and finite
fourth moment.

######  Assumption 1.3 (Weak external field).

In the rest of the article, we will assume the external field is of the form

$\displaystyle h=h_{N}=\rho N^{-\alpha}$

for some $\rho>0$, $\alpha\in[\nicefrac{{1}}{{4}},\infty]$. In particular, we
have $hN^{\nicefrac{{1}}{{4}}}\leqslant\rho$ for all $N\geqslant 1$.

Note that $\alpha=\infty$ corresponds to the zero external field case. In this
paper, we focus on the fluctuation and distributional behavior of the free
energy for the mixed $p$-spin model under a weak or no external field
condition. Our approach is built upon the cluster expansion and Stein’s method
framework in [DW21], which uses analytical and combinatorial techniques to
deal with the SK model. This approach does not extend to the regime
$\alpha\in[0,\nicefrac{{1}}{{4}})$ for the same reason as in [DW21]. We will
briefly discuss the details later. Along with the proof, we also obtain a
characterization of the critical inverse temperature for a large class of spin
glass models. We state the main results as follows.

###  1.2. Main results

For the fluctuation results in Ising $p$-spin glass models, a common hurdle is
establishing the results up to the critical inverse temperature. However, the
first fundamental question is how to obtain an explicit expression for
$\beta_{c}$. Arguably, using the celebrated Parisi formula [MTal06, Pan14],
one can characterize $\beta_{c}$ for a given model, but this is usually
implicit via the property of overlap as an order parameter of the system. In
the following part, we will present an explicit characterization of
$\beta_{c}$ for a given model $\xi$.

####  1.2.1. Critical temperature

In the SK case, $\beta_{c}$ is explicitly known. However, computing
$\beta_{c}(\xi)$ for a given structure function $\xi$ is a challenging task in
general. In both works [ALR87, DW21], a key observation is that the
information about $\beta_{c}$ is hidden inside $\hat{Z}_{N}$ (see (3)). A
precise control of $\hat{Z}_{N}$ can lead to a characterization of
$\beta_{c}$. Our first main result is the following characterization of
$\beta_{c}(\xi)$ for general spin glass models $\xi$, including mixed $p$-spin
models and multi-species mixed $p$-spin models. Let

$\displaystyle\beta_{c}=\beta_{c}(\xi):=\min_{x\in(0,1)}\sqrt{\frac{I(x)}{\xi(x)}},$
(6)

where $\xi(x)$ is the structure-function used to define the spin glass model
in (2) and

$I(x):=\frac{1}{2}(1+x)\log(1+x)+\frac{1}{2}(1-x)\log(1-x)$

is the binary entropy function. Note that,

$\displaystyle\beta_{c}^{2}\cdot\xi(x)\leqslant I(x)\text{ for }x\in[-1,1].$

Subag [Sub21] obtained a similar, but different, characterization in the
spherical setting for multi-species mixed $p$-spin glass models. We present a
version for the multi-species SK model in Section 3. These two
characterizations seem to be equivalent after some transformation. Formally we
have the following result.

######  Theorem 1.4.

Assume 1.1, 1.2 and 1.3. For $\beta<\beta_{c}(\xi)$, we have for all $N$ large

$\displaystyle\operatorname{\mathds{E}}(\hat{Z}_{N}(\beta,h))^{2}\leqslant\frac{C}{\sqrt{\beta_{c}^{2}-\beta^{2}}}\exp\left(\frac{\rho^{4}\beta^{2}}{\beta_{c}^{2}-\beta^{2}}\right)$

where $C>0$ is a universal constant. In particular, we have

$\displaystyle\frac{1}{N}\log Z_{N}(\beta)\to\frac{1}{2}\beta^{2}\xi(1)\text{
in probability as }N\to\infty.$ (7)

######  Remark 1.5.

The above convergence result is based on the fact that
$\hat{Z}_{N}(\beta)=1+o_{\operatorname{\mathds{P}}}(1)$. The convergence in
probability can be strengthened to almost sure convergence under stronger
moment assumption than Assumption 1.2.

######  Remark 1.6.

To rigorously prove $\beta_{c}$ is indeed the critical inverse temperature,
one needs to prove for $\beta>\beta_{c}$,

$\displaystyle\lim_{N\to\infty}\frac{1}{N}\log
Z_{N}(\beta)<\frac{1}{2}\beta^{2}\xi(1),$

we leave this as an open question in Section 9.

The proof of the above theorem and further discussions are deferred to Section
4 with the proof of the large hypergraphs decay regime, where the
characterization (6) was derived. Recall that in the pure $p$-spin case,
$\xi_{p}(x)=x^{p}/p!$ and we have the following result.

######  Lemma 1.7.

The function $\phi(x):=xI^{\prime}(x)/I(x)$ for $x\in(0,1)$ and $\phi(0)=2$ is
continuous and strictly increasing on $[0,1)$ with range $[2,\infty)$.
Moreover, for $p\geqslant 2$ and $\xi_{p}(x)=x^{p}/p!$ we have

$\displaystyle\beta_{c}(p)=\sqrt{I(x_{\star})/\xi_{p}(x_{\star})},\text{ where
}x_{\star}:=\phi^{-1}(p).$

######  Remark 1.8.

Talagrand [tal00] obtained a lower bound
$\beta_{p}(p):=\min_{x\in[0,1]}\sqrt{(1+x^{-p})\cdot I(x)}$. Note that the
difference between $\beta_{c}(p)$ and $\beta_{p}(p)$ is a bit subtle. Both of
them converge to $\sqrt{2\log 2}$, which is the critical temperature for the
random energy model. We summarize this in Figure 1. There are also some
numerical simulations for computing the critical inverse temperature for pure
$p$-spin models in [Chen19, PWB20].

Figure 1. Plot of $\beta_{c}(\xi_{p})/\sqrt{p!}$ vs. $p$. The curve below is
the lower bound from Talagrand [tal00]. As $p\to\infty$,
$\beta_{c}(\xi_{p})/\sqrt{p!}\to\sqrt{2\ln 2}$.

Next, we present the fluctuation results up to $\beta_{c}$ for pure $p$-spin
models. The results for mixed $p$-spin glass, and multi-species mixed $p$-spin
glass are deferred to the following sections.

####  1.2.2. Pure $p$-spin glass models at $h=0$

At the log-partition function level, the fluctuation results at $h=0$ were
known about twenty years ago in [BKL02, KL05], which were proved at a high
enough temperature instead of up to $\beta_{c}$. The proof ideas were based on
several different approaches, and in particular, $\hat{Z}_{N}(\beta,0)$ has
never been well understood (see [KL05] for more details). We first restate the
results in [KL05] as follows.

######  Theorem 1.9 (cf. [KL05]*Theorem 1.2).

Assume that $h=0$, $J_{\boldsymbol{i}}$’s are i.i.d. symmetrically distributed
with variance one and finite exponential moment. There exists $\tilde{\beta}$
depending on $p$ such that for $\beta<\tilde{\beta}$, we have

$\displaystyle N^{\frac{p-2}{2}}(\log
Z_{N}(\beta,0)-\log\operatorname{\mathds{E}}Z_{N}(\beta,0))\to\operatorname{N}\left(0,\frac{\beta^{4}}{4p!}\operatorname{Var}(J^{2})\right)$
(8)

$\text{in distribution as }N\to\infty.$

The above theorem is based on the decomposition of the partition function
given in (3) for $h=0$. The key point is that the variance order of
$\log\hat{Z}_{N}$ is much smaller than $\log\bar{Z}_{N}$. Under the scaling
$N^{(p-2)/2}$ of $\log\bar{Z}_{N}$, the fluctuation of $\log\hat{Z}_{N}$ is
negligible. However, it is not even clear what is the correct variance order
of $\log\hat{Z}_{N}$ for $h=0$. The authors used concentration bounds to prove
that the variance order of $\log\hat{Z}_{N}$ is dominated by
$\log\bar{Z}_{N}$. Besides, the result holds at very high temperatures due to
the lack of understanding about $\hat{Z}_{N}$. In the following part, we can
see that $\hat{Z}_{N}$ contains fruitful information about the model. Once
$\hat{Z}_{N}$ can be fully understood, we can prove the fluctuation results
and extract the information about the critical inverse temperature
$\beta_{c}(\xi)$. We state our first main result on the fluctuation of
$\hat{Z}_{N}$ up to $\beta_{c}$ in the zero external field case, _i.e.,_
$\alpha=\infty$.

######  Theorem 1.10 (Pure $p$-spin at $h=0$).

Assume that $h=0$ and assumption 1.2 holds. For $\beta<\beta_{c}$, we have

$\displaystyle
N^{\frac{1}{4}{\ell_{p}(p-2)}}\cdot\log\hat{Z}_{N}(\beta,0)\to\operatorname{N}(0,v_{p}(\beta)^{2})\text{
in distribution as }N\to\infty,$ (9)

where

$\ell_{p}:=\begin{cases}3&\text{if }p\text{ is even,}\\\ 4&\text{if}\ p\text{
is odd,}\end{cases}$

and

$\displaystyle
v_{p}(\beta)^{2}:=\begin{cases}\frac{\beta^{6}}{3!p!^{3/2}}\operatorname{\mathds{E}}H_{3}(H_{p}(\eta)/\sqrt{p!})=\frac{\beta^{6}}{3!p!^{3}}\operatorname{\mathds{E}}(H_{p}(\eta)^{3}),&\text{
for $p$ even},\\\
\frac{\beta^{8}}{4!p!^{2}}\operatorname{\mathds{E}}H_{4}(H_{p}(\eta)/\sqrt{p!}))=\frac{\beta^{8}}{4!p!^{4}}\operatorname{\mathds{E}}(H_{p}(\eta)^{4}-3p!^{2}),&\text{
for $p$ odd}.\end{cases}$ (10)

where $H_{k}(\cdot)$ is the Hermite polynomial of degree $k$ and
$\eta\sim\operatorname{N}(0,1)$.

######  Remark 1.11.

We have the following remarks:

  1. a)

This result is stronger than what was obtained in [KL05], where the author
only showed that the variance order of $\log\hat{Z}_{N}$ is smaller than
$\log\bar{Z}_{N}$. Besides that, Theorem 1.10 holds up to the critical inverse
temperature $\beta_{c}$ with general weights having a finite fourth moment.

  2. b)

Bovier and Schertzer [BS22] recently obtained similar results to Theorem 1.10.
Their proof is based on truncated moment estimates with a different
decomposition of the partition function. They split $\log Z_{N}$ into two
parts:
$J_{N}:=\frac{1}{2}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\beta^{2}J_{\boldsymbol{i}}^{2}/N^{p-1}$
and $\log Z_{N}-J_{N}$, and obtained similar limit theorem for $\log
Z_{N}-J_{N}$.

The above result suggests that pure even and odd $p$-spin models behave
differently at $h=0$. Later in the proof, it will be clear why there is such a
discrepancy. This discrepancy persists until the weak external field becomes
strong enough. This will be touched in Theorem 1.13.

Let us briefly discuss the strategy of the proof for Theorem 1.10. Starting
with the decomposition in (3), we will focus on the term $\hat{Z}_{N}$.
Applying $\operatorname{\mathds{E}}_{\boldsymbol{\sigma}}$, we have

$\displaystyle\hat{Z}_{N}(\beta,0)$
$\displaystyle=\sum_{\Gamma\subseteq\mathscr{E}_{N,p}}\hat{\omega}(\Gamma),$
(11) $\displaystyle\text{ where
}\hat{\omega}(\Gamma)=\prod_{\boldsymbol{i}\in\Gamma}\hat{\omega}_{\boldsymbol{i}},\quad$
$\displaystyle\hat{\omega}_{\boldsymbol{i}}=\tanh(\beta
J_{\boldsymbol{i}}/N^{(p-1)/2}),\text{ for
}\boldsymbol{i}\in\mathscr{E}_{N,p},$

and $\Gamma$ is a sub-hypergraph of the simple complete $p$-uniform
hypergraph. We further decompose the sum depending on the cluster structure,
_i.e.,_ the number of hyperedges used and the number of odd-degree vertices in
$\Gamma$. The main idea is that the contribution of large cluster structures
to $\hat{Z}_{N}$ decays fast under the high-temperature condition
$\beta<\beta_{c}$. It allows us to reduce the sum in $\hat{Z}_{N}$ to smaller-
sized clusters. This will be proved and explained in Section 4. In contrast to
SK model [DW21], the first reduction can only reduce the size of clusters to
size $\mathcal{O}(\log N)$, which is still not amenable to analysis using
Stein’s method and counting argument in the later steps.

To get tractable finite-size dominant clusters, we use a form of chaos
expansion. Built on this, we can make a further reduction to identify the
leading clusters. It turns out that even and odd $p$-spin cases have different
leading cluster structures. We will use the notation $((k,p),\ell)$ to denote
the cluster structure with $k$ many $p$-hyperedges and $\ell$ many odd-degree
vertices. We may also use $(k,\ell)$ notation instead of $((k,p),\ell)$ if
there is no confusion.

We define the sub-hypergraphs corresponding to cluster structures for even and
odd $p$, respectively, as follows:

$\displaystyle\begin{split}\mathcal{S}_{(3,p),0}&:=\\{\Gamma\subseteq\mathscr{E}_{N,p}:(\left|\Gamma\right|,\left|\partial\Gamma\right|)=(3,0)\\}\text{
and }\\\
\mathcal{S}_{(4,p),0}&:=\\{\Gamma\subseteq\mathscr{E}_{N,p}:(\left|\Gamma\right|,\left|\partial\Gamma\right|)=(4,0)\\},\end{split}$
(12)

where $\left|\Gamma\right|,\left|\partial\Gamma\right|$ are, respectively, the
number of hyperedges and odd-degree vertices in $\Gamma$. Example of those
structures is given in Figure 2 and 3. Note that, $\mathcal{S}_{(3,p),0}$ is
non-empty only when $p$ is even. The results of determining the leading
cluster structures are presented in Section 5.

######  Remark 1.12 (On the multi-scale fluctuation phenomenon).

Following the Remark 1.11 part (b), Bovier and Schertzer [BS22] posed a
conjecture that once the leading terms are subtracted from $\log Z_{N}-J_{N}$,
on a smaller scale, there appears yet another limit theorem. This type of
result is particularly clear in our framework. In our setting,
$\log\hat{Z}_{N}$ plays the role of $\log Z_{N}-J_{N}$. Once the leading
clusters’ contribution in (12) are subtracted from $\log\hat{Z}_{N}$, the
dominant contribution will be from larger size clusters (with smaller
variance). Take even $p$-spin as an example; the next leading cluster after
subtracting $(3,0)$ cluster will be cluster $(4,0)$. The variance order will
be in a smaller scale $\Theta(N^{p-2})$. One can obtain a similar limit
theorem under this scale, which can be repeated on a smaller scale. We further
remark that this type of result can also be obtained if some weak external
field is present, where different dominant cluster structures appear with
different variance orders. This will be addressed in Theorem 1.13. Similar
things should happen in general mixed $p$-spin models, where the dominant
clusters become quite complicated; details are presented in Section 2.

To prove the central limit theorem, we use multivariate Stein’s method for
exchangeable pairs [RR09] to analyze the obtained leading clusters. This is in
a similar spirit to [DW21] but is more involved than the SK case since the
cluster structures are now hypergraphs. The complete details are presented in
Section 6. Note that an explicit convergence rate is also obtained while
applying Stein’s method. We further remark that from Figure 2, it is easy to
see that odd $p$-spin cluster structures are more complicated, which in some
sense explains why in many classical results, the odd $p$-spin case is usually
more difficult to deal with. We defer the further discussion of using Stein’s
method to prove joint distributional convergence after Theorem 1.13 in Section
1.4.

(a) Variants of $((4,5),0)$ cluster, _i.e.,_ the leading cluster for
$p=5,h=0$.

(b) Left one corresponds to $((3,5),1)$ cluster, middle one $((2,5),2)$
cluster and right one $((1,5),5)$ cluster.

Figure 2. Example of leading clusters for $p=5$ appearing when
$\alpha\geqslant\nicefrac{{1}}{{4}}$.

Figure 3. Example of leading clusters for $p=4$ appeared when
$\alpha\geqslant\nicefrac{{1}}{{4}}$. The left one is the $((3,4),0)$ cluster
when $\alpha=\infty$. The middle one is the $((2,4),2)$ cluster. The right one
is the $((1,4),4)$ single hyperedge.

####  1.2.3. Pure $p$-spin glass models under weak external field

Next, we state the results for the model under a weak external field. Compared
to the results in SK model [DW21], the picture becomes more complicated and
exhibits a new multiple-transition phenomenon. Here we point out that we can
only obtain the fluctuation results for $\alpha\geqslant\nicefrac{{1}}{{4}}$
due to a fundamental difficulty similar to the SK case. Roughly speaking, we
do not enjoy the large hypergraph decay results for
$\alpha<\nicefrac{{1}}{{4}}$, _i.e.,_ when the external field is strong
enough. The large hypergraph decay results under weak external field are
presented in Section 4.

Let us first introduce some necessary notations. Depending on whether $p$ is
even or odd, we define

$\displaystyle\alpha_{c}=\alpha_{c}(p):=\begin{cases}\quad{p}/{8},&p\text{
even},\\\ (p-1)/4,&p\text{ odd}.\end{cases}$ (13)

We also introduce the following three regimes:

$\displaystyle\begin{split}\mathcal{R}_{1}=\mathcal{R}_{1}(\alpha_{c}):=[\alpha_{c},\infty),\quad\mathcal{R}_{2}=\mathcal{R}_{2}(\alpha_{c}):=[\nicefrac{{1}}{{2}},\alpha_{c}),\text{
and
}\quad\mathcal{R}_{3}:=[\nicefrac{{1}}{{4}},\nicefrac{{1}}{{2}}).\end{split}$
(14)

The cases $\alpha=\nicefrac{{1}}{{2}}$ and $\alpha_{c}$ correspond to critical
transition points, where some new leading structures emerge. For each of the
regimes, we define the corresponding variance exponents

$\displaystyle\gamma_{1}:=\frac{1}{2}\ell_{p}(p-2),\quad\gamma_{2}(\alpha):=\gamma_{1}-4(\alpha_{c}(p)-\alpha)\text{
and }\gamma_{3}(\alpha):=2\alpha p-1.$ (15)

We state the main results when $\alpha\in[\nicefrac{{1}}{{4}},\infty)$ for
pure $p$-spin models. We use the following notation for representing the
limiting variance

$\displaystyle
v_{(k,p),\ell}(\beta,\rho)^{2}:=\frac{\beta^{2k}\rho^{2\ell}}{k!\cdot\ell!\cdot
p!^{k/2}}\operatorname{\mathds{E}}(H_{k}(H_{p}(\eta)/\sqrt{p!})\cdot
H_{\ell}(\eta)),\text{ where }\eta\sim\operatorname{N}(0,1).$ (16)

Note that this is the variance associated with the cluster structure
$((k,p),\ell)$, which is summarized in Table 1. The above limiting variance
result is proved in Lemma 6.1. In the rest, we will drop the dependence on $p$
and write it as $v_{k,\ell}(\beta,\rho)^{2}$ for convenience.

In the $h=0$ case, it can be checked that the limiting variances in Theorem
1.10 are $v_{3,0}(\beta,0)^{2}$ and $v_{4,0}(\beta,0)^{2}$ for even and odd
$p$ respectively. In this case, we will write them as
$v_{3,0}(\beta)^{2},v_{4,0}(\beta)^{2}$ for simplicity.

######  Theorem 1.13 (Pure $p$-spin under weak external field).

Assume that $\alpha\geqslant\nicefrac{{1}}{{4}}$, and assumption 1.2 holds.
For $\beta<\beta_{c}$ with the notations in (15), we have the following
results.

  1. a)

Sub-critical regime: If $\alpha\in\mathcal{R}_{1}=[\alpha_{c},\infty)$, we
have

$\displaystyle
N^{\gamma_{1}/2}\cdot\log\hat{Z}_{N}(\beta,h)\to\operatorname{N}(0,v_{1}^{2})$
(17)

in distribution as $N\to\infty$ where

$\displaystyle v_{1}^{2}:=\begin{cases}v_{p}(\beta)^{2},&\alpha<\alpha_{c},\\\
v_{p}^{c}(\beta)^{2},&\alpha=\alpha_{c},.\end{cases}$

with $v_{p}(\beta)$ as defined in (10) and

$\displaystyle
v_{p}^{c}(\beta)^{2}:=\begin{cases}v_{3,0}(\beta)^{2}+v_{2,2}(\beta,\rho)^{2},&\text{$p$
even},\\\
v_{4,0}(\beta)^{2}+v_{3,1}(\beta,\rho)^{2}+v_{2,2}(\beta,\rho)^{2},&\text{$p$
odd}.\end{cases}$

  2. b)

Unified regime I: If
$\alpha\in\mathcal{R}_{2}=[\nicefrac{{1}}{{2}},\alpha_{c})$, we have

$\displaystyle
N^{\gamma_{2}(\alpha)/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v_{2}^{2})$
(18)

in distribution as $N\to\infty$ where

$\displaystyle
v_{2}^{2}:=\begin{cases}v_{2,2}(\beta,\rho)^{2}+v_{1,p}(\beta,\rho)^{2},&\alpha=\frac{1}{2},\\\
v_{2,2}(\beta,\rho)^{2},&\frac{1}{2}<\alpha<\alpha_{c}.\end{cases}$

  3. c)

Unified regime II: If
$\alpha\in\mathcal{R}_{3}=[\nicefrac{{1}}{{4}},\nicefrac{{1}}{{2}})$, we have

$\displaystyle
N^{\gamma_{3}(\alpha)/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v_{3}^{2})$
(19)

in distribution as $N\to\infty$ where $v_{3}^{2}=v_{1,p}(\beta,\rho)^{2}$.

######  Remark 1.14.

The even and odd $p$-spin models behave consistently once the external field
is in the unified regimes $\mathcal{R}_{2}$ and $\mathcal{R}_{3}$.

The above theorem presents a new multiple transition phenomenon, accordingly
summarized in Figure 4. Note that the picture is more complicated than the SK
($p=2$) model, where only the regime $\mathcal{R}_{1}$ appears, see [DW21] for
further details. Besides that, $p=3,4$ are also special, we can check that
$\alpha_{c}=\nicefrac{{1}}{{2}}$ in these cases. It means that the regime
$\mathcal{R}_{2}$ disappears. All three regimes exist for $p\geqslant 5$,
exhibiting multiple transitions. We summarize this discussion in Figure 4 and
also in Table 1.

Figure 4. Multiple transitions in the pure $p$-spin models. The horizontal
axis is the component of the external field $h$. The vertical axis is the
exponent $\gamma$ in $\operatorname{Var}(\hat{Z}_{N})\approx N^{-\gamma}$.

The proof of Theorem 1.13 gets through in a similar manner as Theorem 1.10
once the following two key ingredients are established.

  1. a)

First, we need to prove that even if the external field, of the scale
$N^{-\alpha},\alpha\geqslant\nicefrac{{1}}{{4}}$, is present, the large
hypergraph decay property still holds.

  2. b)

Second, once we reduce the hypergraph clusters to a finite size, it needs to
identify the dominant clusters depending on the strength of the external
field. This needs careful analysis of the variance order of many possible
cluster structures due to external fields. We rephrase it as a discrete
optimization problem over the cluster size $\left|\Gamma\right|$ and the
number of odd-degree vertices $\left|\partial\Gamma\right|$. The goal is to
find the clusters with the largest variance.

Let us first define those new dominant clusters that emerged after adding the
weak external fields:

$\displaystyle\begin{split}\mathcal{S}_{(3,p),1}:=\\{\Gamma\subseteq\mathscr{E}_{N,p}:(\left|\Gamma\right|,\left|\partial\Gamma\right|)=(3,1)\\},\\\
\mathcal{S}_{(2,p),2}:=\\{\Gamma\subseteq\mathscr{E}_{N,p}:(\left|\Gamma\right|,\left|\partial\Gamma\right|)=(2,2)\\},\\\
\mathcal{S}_{(1,p),p}:=\\{\Gamma\subseteq\mathscr{E}_{N,p}:(\left|\Gamma\right|,\left|\partial\Gamma\right|)=(1,p)\\}.\end{split}$
(20)

There is still a discrepancy between even and odd $p$ in some regimes.

As addressed in Table 1, for even $p$, in the sub-critical regime, the
contributions from clusters $(2,2),(1,p)$ are dominated by the $(3,0)$
cluster. Thus, the fluctuation order and variance are essentially the same as
zero external field cases. Only when $\alpha=\alpha_{c}$, the contributions of
$(2,2)$ and $(3,0)$ are of the same order, this changes the asymptotic
variance in the CLT. While in the unified regime I, the cluster $(2,2)$
dominates, as a result, the fluctuation order and variance both change. When
the external field gets even stronger, _i.e.,_ in the unified regime II, the
cluster $(1,p)$ dominates. Note that $\alpha=1/2$ is the transition point
where the contribution of clusters $(2,2)$ and $(1,p)$ are in the same scale.
For the odd $p$-spin case, the transition picture is essentially the same as
the even $p$ case, except there is a new companion cluster $(3,1)$ for
$(2,2)$. The example of cluster structures and the associated transition
behavior can be found in Figure 2 and Table 1.

Since the behavior of $\hat{Z}_{N}$ is now well understood, it is not hard to
deduce the transitional results for the log-partition function $\log Z_{N}$,
because $\log\bar{Z}_{N}$ as an i.i.d. sum is always easy to analyze. In
particular, we have

$\displaystyle\log\bar{Z}_{N}(\beta,h)$
$\displaystyle=\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\log\cosh(\beta
J_{\boldsymbol{i}}/N^{(p-1)/2})$ (21)
$\displaystyle\approx\frac{\beta^{2}\binom{N}{p}}{2N^{p-1}}-\frac{\beta^{4}\binom{N}{p}}{12N^{2p-2}}\cdot\operatorname{\mathds{E}}J^{4}+\operatorname{N}\left(0,\frac{\beta^{4}\binom{N}{p}}{4N^{2p-2}}\cdot\operatorname{Var}(J^{2})\right).$
(22)

Here we use $\approx$ for distributional approximation after appropriate
centering and scaling. Due to the competitive behavior of $\hat{Z}_{N}$ and
$\bar{Z}_{N}$, we obtained the following results at the free energy level. The
threshold of the external field, in this case, changed. The variance order
$\log Z_{N}$ in the supercritical regime aligns with $\log\hat{Z}_{N}$. While
in the sub-critical case, $\bar{Z}_{N}$ dominates the contribution in $\log
Z_{N}$.

######  Corollary 1.15 (Fluctuations for the log-partition function).

Assume that $\alpha\geqslant\nicefrac{{1}}{{4}}$, and
$J_{\boldsymbol{i}},\boldsymbol{i}\in\mathscr{E}_{N,p}$ satisfy Assumption
1.2. Then for $\beta<\beta_{c}$, we have

  1. a)

If $\alpha>\frac{p-1}{2p}$, we have

$\displaystyle N^{(p-2)/2}\cdot\left(\log
Z_{N}-\binom{N}{p}\operatorname{\mathds{E}}\log\cosh(\beta
J/N^{(p-1)/2})\right)\to\operatorname{N}(0,v_{1}^{2}),$ (23)

in distribution as $N\to\infty$, where
$v_{1}^{2}:=\beta^{4}\operatorname{Var}(J^{2})/4p!$.

  2. b)

If $\alpha=\frac{p-1}{2p}$, we have

$\displaystyle N^{(p-2)/2}\cdot\left(\log
Z_{N}-\binom{N}{p}\operatorname{\mathds{E}}\log\cosh(\beta
J/N^{(p-1)/2}\right)\to\operatorname{N}(0,v_{2}^{2}),$ (24)

in distribution as $N\to\infty$, where
$v_{2}^{2}:=v_{1,p}(\beta,\rho)^{2}+v_{1}^{2}.$

  3. c)

If $\frac{1}{4}\leqslant\alpha<\frac{p-1}{2p}$,we have

$\displaystyle N^{(2\alpha p-1)/2}\cdot\left(\log
Z_{N}-\binom{N}{p}\operatorname{\mathds{E}}\log\cosh(\beta
J/N^{(p-1)/2}\right)\to\operatorname{N}(0,v_{3}^{2})$ (25)

in distribution as $N\to\infty$, where $v_{3}^{2}:=v_{1,p}(\beta,\rho)^{2}$.

######  Remark 1.16.

We remark on the following facts.

  1. a)

One can check that in the $p=2$ case, the critical transition threshold
becomes $\frac{1}{4}$, which was established in [DW21] for SK model. In the SK
case, there is no third regime shown in part c).

  2. b)

In general, for $p\geqslant 3$, it can be checked that
$\frac{1}{4}<\frac{p-1}{2p}<\frac{1}{2}$. Clearly
$\lim_{p\to\infty}\frac{p-1}{2p}=\frac{1}{2}$. It was known [Derr80] that the
random energy model (REM) is the limit of pure $p$-spin model as $p\to\infty$.
Thus it is natural to conjecture that the critical threshold for the
fluctuation in the REM with weak external field is $\alpha_{c}=\frac{1}{2}$ at
high temperature. One can heuristically check this is indeed the case. In the
REM case, the energy of each configuration is i.i.d. distributed; it needs
$\Theta(N^{1/2})$ external field to beat the i.i.d. sum of energies by
classical CLT.

Corollary 1.15 naturally follows from Theorem 1.13. Recall that in [KL05],
Knöpfel and Löwe obtained results stated in Theorem 1.9, which basically says
that at $h=0$ case, the fluctuation of free energy in pure $p$-spin model is
dominated by $\log\bar{Z}_{N}$. Corollary 1.15 generalizes this result in two
ways. One is by establishing the transitional behavior under a weak external
field. The other is by extending the temperature range up to $\beta_{c}$.

In summary, the results for the pure $p$-spin case already exhibit complex
transition phenomena, as illustrated in the above theorems. It is natural to
ask what happens in the general mixed $p$-spin models. Following a similar
idea, the main difficulty lies in identifying the leading cluster structures
and controlling the large hypergraph decay. Unlike the pure $p$-spin model,
now there is enormous freedom to form all kinds of clusters in the mixed
$p$-spin case since $p$ is not fixed anymore. Even if the cluster size is
fixed, there still could be infinitely many possible different structures. Due
to this further complication of the mixed $p$-spin model, we defer the main
results and discussions into Section 2. We further remark that our approach
can be generalized to establish similar results for multi-species mixed
$p$-spin models. Again, the main results and discussions will be postponed to
Section 3.

Table 1. Summary of transition behavior for pure $p$-spin models External field $h$ | Cond. | $\gamma:\operatorname{Var}(\hat{Z}_{N})\approx N^{-\gamma}$ | Cluster structures $(\left|\Gamma\right|,\left|\partial\Gamma\right|)$
---|---|---|---
$\alpha\in(\alpha_{c},\infty]$ |  $p$ even | $3(p-2)/2$ | $(3,0)$
$p$ odd | $2(p-2)$ | $(4,0)$
$\alpha=\alpha_{c}$ |  $p$ even | $3(p-2)/2$ | $(3,0),(2,2)$
$p$ odd | $2(p-2)$ | $(4,0),(2,2),(3,1)$
$\alpha\in(1/2,\alpha_{c})$ |  | $p+4\alpha-3$ | (2,2)
$\alpha=1/2$ |  | $p-1$ | (2,2),(1,$p$)
$\alpha\in[1/4,1/2)$ |  | $2\alpha p-1$ | (1,$p$)

###  1.3. Prior results

This section briefly summarizes the previous fluctuation results in the
$p$-spin models and the study of transitional behavior while changing the
external field. The first fluctuation result for pure $p$-spin models can be
traced back to 2002, in which Bovier, Kurkova, and Löwe [BKL02] used
martingale approach to prove a central limit theorem for the free energy up to
a $\beta_{p}<\beta_{c}$. The Gaussian disorder played a key role in their
proof. Later Knöpfel and Löwe [KL05] adapted the idea of Aizenman et. al.
[ALR87] to obtain a decomposition in (3), where the term $\log\bar{Z}_{N}$ is
essentially a sum of i.i.d. random variables, and they proved that the
variance order of $\log\hat{Z}_{N}$ is smaller than that of $\log\bar{Z}_{N}$
using concentration results. Therefore, the fluctuation of limiting free
energy is dominated by $\log\bar{Z}_{N}$.

The proof worked for general symmetric disorders with finite exponential
moments. Again, this result also holds up to some
$\tilde{\beta}_{p}<\beta_{c}$. In short, those two results were both
established in the zero external field setting for pure $p$-spin models and a
strictly sub-region of the whole high-temperature regime. More importantly, it
was unclear what happens in the term $\hat{Z}_{N}$ in both works. On the other
hand, in 2017, Chen, Dey, and Panchenko [CDP17] extended the fluctuation
results to the mixed even $p$-spin models in the presence of an external
field, and they proved that in this case, the Gaussian fluctuation of free
energy persists for all $\beta<\infty$. The key idea is based on Stein’s
method and superconcentration results using the Parisi formula. However, one
weakness is the even $p$ restriction, which seems a common hurdle in many
other problems, see [Chen14, CHL18, MTal06, Chen14b]. Recently, Banerjee and
Belius [BB21] used the weighted cycle counting technique to obtain the
fluctuation results for mixed $p$-spin models at zero external fields with a
non-vanishing 2-spin term. Note that this technique is different from the loop
counting arising in cluster expansion. One notable weakness is that it can not
prove the results up to the critical inverse temperature $\beta_{c}$ and is
also restricted to zero field cases.

Although there has been much progress in understanding the fluctuation of free
energy at zero and positive external field, the transition behavior between
them has long been elusive, especially in the Ising spin case. In the SK model
($p=2$) at a high-temperature regime, the authors [DW21] utilized various
approaches to prove that in the high-temperature regime, there are three sub-
regimes concerning the strength of the weak external field $h=\rho
N^{-\alpha}$ for some $\rho>0,\alpha\in[0,\infty]$. In the sub-critical case,
$\alpha>\nicefrac{{1}}{{4}}$, the fluctuation is nearly the same as $h=0$. In
the critical case $\alpha=\nicefrac{{1}}{{4}}$, the fluctuation order does not
change, but the asymptotic mean and variance now have some extra corrections.
In the supercritical case, the fluctuation order becomes linear. One of their
approaches is based on cluster expansion. They also established similar
results in some other generalized models, such as the multi-species and
diluted SK models. In the spherical SK model, Baik et.al. [BCLW21] studied the
transition behavior using the steepest descent method originating in random
matrix theory, and some results are computer-aided. The results in the high-
temperature regime are consistent with [DW21] for the Ising spin case. A
concurrent work by Landon and Sosoe [LS20] obtained similar results by
conducting a fully rigorous analysis, and the idea is essentially the same.
All those results are restricted to the $p=2$ spin case. In particular,
extending the random matrix technique used in the spherical SK model to the
spherical $p$-spin setting is challenging, and nothing is known. In summary,
the transition behavior from zero to positive field for the general $p$-spin
model is a complete mystery. Besides that, establishing results up to the
critical inverse temperature is another challenge.

###  1.4. Multivariate Stein’s method for exchangeable pairs

In the previous part, we briefly sketch how to decompose
$\hat{Z}_{N}(\beta,h)$ into a collection of different clusters in different
regimes. To establish the limit theorems described in Theorem 1.10, 1.13 and
Corollary 1.15, we apply the following version of multivariate Stein’s method
for exchangeable pairs. Stein’s method is a modern way to prove normal
approximation; we refer interested readers for the background to [CGS11].

######  Theorem 1.17 ([RR09]*Theorem 2.1).

Assume that $(\boldsymbol{W},\boldsymbol{W}^{\prime})$ is an exchangeable pair
of $\mathds{R}^{d}$ valued random vectors such that
$\operatorname{\mathds{E}}\boldsymbol{W}=0,\operatorname{\mathds{E}}\boldsymbol{W}\boldsymbol{W}^{T}=\Sigma$
where $\Sigma\in\mathds{R}^{d\times d}$ is a symmetric positive definite
matrix and $\boldsymbol{W}$ has finite third moment in each coordinate.
Suppose further that

$\displaystyle\operatorname{\mathds{E}}(\boldsymbol{W}^{\prime}-\boldsymbol{W}\mid\boldsymbol{W})=-\Lambda\boldsymbol{W}\quad\text{a.s.},$
(26)

for an invertible matrix $\Lambda$. If $\boldsymbol{Z}$ has $d$-dimensional
standard normal distribution, we have for every three times differentiable
function $f$,

$\displaystyle\left|\operatorname{\mathds{E}}f(\boldsymbol{W})-\operatorname{\mathds{E}}f(\Sigma^{1/2}\boldsymbol{Z})\right|\leqslant\frac{1}{4}\left|f\right|_{2}A+\frac{1}{12}\left|f\right|_{3}B$

where
$\left|f\right|_{2}:=\sup_{i,j}\left\|\partial_{x_{i}x_{j}}f\right\|_{\infty},\left|f\right|_{3}:=\sup_{i,j,k}\left\|\partial_{x_{i}x_{j}x_{k}}f\right\|_{\infty}$,
$\lambda^{(i)}:=\sum_{m=1}^{d}\left|(\Lambda^{-1})_{m,i}\right|$,

$\displaystyle A$
$\displaystyle:=\sum_{i,j=1}^{d}\lambda^{(i)}\sqrt{\operatorname{Var}\operatorname{\mathds{E}}((W_{i}^{\prime}-W_{i})(W_{j}^{\prime}-W_{j})\mid\boldsymbol{W})}$
$\displaystyle\text{ and }B$
$\displaystyle:=\sum_{i,j,k=1}^{d}\lambda^{(i)}\operatorname{\mathds{E}}\left|(W_{i}^{\prime}-W_{i})(W_{j}^{\prime}-W_{j})(W_{k}^{\prime}-W_{k})\right|.$

In our setting, the random vector we will work on is a collection of random
variables indexed by different cluster structures. For example, in the regime
$\alpha=\alpha_{c}$ for pure even $p$-spin model,

$\displaystyle\boldsymbol{W}=\left(\sum_{\Gamma\subseteq
S_{(3,p),0}}\omega(\Gamma),\sum_{\Gamma\subseteq
S_{(2,p),2}}\omega(\Gamma)\right).$

Moreover, the linearity condition (26) is satisfied with $\Lambda$ of the form
$\lambda D$, where $\lambda=o(1)$ is a scalar, and $D$ is a deterministic
matrix. The details for checking the associated conditions are left to Section
6. We remark that Stein’s method proof presents a more straightforward and
systematic picture than the moment-based method and comes with a convergence
rate under minimal moment assumptions.

###  1.5. Notations

  1. i.

$\gamma$ denotes the exponent in
$\operatorname{Var}(\hat{Z}_{N})=\Theta(N^{-\gamma})$.

  2. ii.

$\alpha_{c}$ is the first critical transition exponent of the weak external
field $h=\rho N^{-\alpha}$.

  3. iii.

$p_{e},p_{o}$ denote the minimum even and odd $p\geqslant 3$ interactions
appearing the mixed $p$-spin model, and $p_{m}=\min\\{p_{o},p_{e}\\}$.

  4. iv.

$H_{k}(\cdot)$ is the Hermite polynomial of degree $k$.

  5. v.

$\hat{h}=\tanh(h)$ and $\tilde{h}^{2}=\tanh^{-1}(\tanh(h)^{2})\approx h^{2}$
for $h$ small.

  6. vi.

For a hyperedge $\boldsymbol{i}\in\mathscr{E}_{N,p}$, we define
$\hat{\omega}_{\boldsymbol{i}}=\tanh(\beta\theta_{p}J_{\boldsymbol{i}}/N^{(p-1)/2})$
and its normalized version
$\omega_{\boldsymbol{i}}=N^{(p-1)/2}\cdot\hat{\omega}_{\boldsymbol{i}}\approx\beta
J_{\boldsymbol{i}}.$

  7. vii.

For a finite sub-hypergraph $\Gamma$, we define
$\hat{\omega}(\Gamma)=\prod_{\boldsymbol{i}\in\Gamma}\hat{\omega}_{\boldsymbol{i}}$
and $\omega(\Gamma)=\prod_{\boldsymbol{i}\in\Gamma}\omega_{\boldsymbol{i}}$.

  8. viii.

$\mathcal{R}_{i}$ for $i=1,2,3$ denotes the sub-critical regime:
$[\alpha_{c},\infty]$, unified regime I: $[\nicefrac{{1}}{{2}},\alpha_{c})$
and unified regime II: $[\nicefrac{{1}}{{4}},\nicefrac{{1}}{{2}})$
respectively. Note $\alpha_{c}$ has different forms in pure and mixed $p$-spin
models.

  9. ix.

We use $c:=((a_{1},p_{1}),\ldots,(a_{t},p_{t}),\ell)$ to denote the type of
cluster structures. Specifically, $\mathcal{S}_{c}$ denotes the set of sub-
hypergraph corresponding to the cluster structure $c$, _i.e.,_ the set of
hypergraphs with $a_{s}$ many $p_{s}$-hyperedges for $s=1,\ldots,t$ and $\ell$
many odd-degree vertices.

  10. x.

For mixed $p$-spin model, $\mathcal{S}_{a_{e},a_{o},b}$ denote the set of
hypergraphs with cluster structure $c=((a_{e},p_{e}),(a_{o},p_{o}),b)$ with
$a$ different $p_{e}$-hyperedges and $b$ different $p_{o}$-hyperedges and $c$
odd-degree vertices. For pure $p$-spin, we use $\mathcal{S}_{a,b}$ to present
the clusters with structure $c=((a,p),b)$.

  11. xi.

Following the last notation, we use $v_{a,b,p}(\beta,\rho)^{2}$ to denote the
limiting variance for clusters in $\mathcal{S}_{(a,p),b}$. Similarly for mixed
$p$-spin, let $v_{a_{e},a_{o},b}(\beta,\rho)^{2}$ be the limiting variance for
clusters in $\mathcal{S}_{a_{e},a_{o},b}$.

  12. xii.

The weight for the cluster $c$ is
$V_{c}:=\hat{h}^{\ell}\cdot\sum_{\Gamma\in\mathcal{S}_{c}}\hat{\omega}(\Gamma)$.
The variance for the cluster $c$ is
$\operatorname{Var}(V_{c})=\Theta(N^{\mathcal{X}(c)})$, where $\mathcal{X}(c)$
denotes the variance exponent. A similar notation is used in the random vector
case for multivariate Stein’s method computations.

  13. xiii.

We define the random variable
$W_{c}:=\sum_{\Gamma\in\mathcal{S}_{c}}\omega(\Gamma)$ for a given cluster
structure $c$.

###  1.6. Organization of the paper

This paper is structured as follows. The main results for pure $p$-spin models
under zero and weak external field are stated in Section 1. For clarity, we
state the corresponding results for general mixed $p$-spin models in Section
2. Besides that, we also discuss and state the results for multi-species mixed
$p$-spin models in Section 3. The proof details are distributed into three
different sections. In Section 4 the large hypergraph decay results for
general mixed $p$-spin (including pure $p$ case) are presented, where we also
include the proof of the simple characterization of critical inverse
temperature for general spin glass models in the replica symmetric regime. We
also include the large $p$-hyperedge truncation results there, which is a core
challenge in the mixed $p$-spin models. In Section 5, we present the general
strategy and the proof details for identifying the dominant cluster structures
in various regimes for different models. In Section 6, we show the details of
using multivariate Stein’s method for exchangeable pairs to analyze the
dominant clusters obtained in Section 5. Finally, in Section 8, we collect
everything together to prove the main theorems present in Section 1 and 2 for
the pure $p$-spin and mixed $p$-spin models respectively. In Section 7, we
devote ourselves to illustrating how to carry out the computations for
applying multivariate Stein’s method for mixed $p$-spin model. Finally, in
Section 9, we present future questions and further discussions about the
results of this paper.

##  2\. Results for Mixed $p$-Spin Models

In Section 1, Theorem 1.10 and 1.13 gives the leading cluster structures and
the fluctuation results for $\log\hat{Z}_{N}$ in the pure $p$-spin model. This
naturally gives rise to the fluctuation results for the free energy in
Corollary 1.15. In this section, we state similar results in a more general
setting: the mixed $p$-spin models. Compared to the pure $p$-spin case, which
corresponds to the spin glass models on $p$-uniform hypergraphs, the mixed
$p$-spin model is a mixture of all possible even and odd $p$-uniform
hypergraphs. This creates several further challenges in the analysis, and
interestingly a two-parameter multiple transition appears. Specifically, the
mixed $p$-spin model also exhibits transition with respect to $p$. It turns
out that the dominant clusters in the mixed $p$-spin case only depend on the
minimum effective even and odd $p$-spins. We define

$\displaystyle p_{e}:=\min\\{\text{even }p\geqslant 4:\theta_{p}>0\\}\quad
p_{o}:=\min\\{\text{odd }p\geqslant 3:\theta_{p}>0\\}.$

Now we state the main theorems for mixed $p$-spin glass models. Similarly, as
in the pure $p$-spin case, we introduce the following notation for the
limiting variance in mixed $p$-spin models.

$\displaystyle v_{(a_{e},p_{e}),(a_{o},p_{o}),b}(\beta,\rho)^{2}$
$\displaystyle:=\frac{\beta^{2(a_{e}+a_{o})}\cdot\theta_{p_{e}}^{2a_{e}}\cdot\theta_{p_{o}}^{2a_{0}}\cdot\rho^{2b}}{a_{e}!\cdot
a_{o}!\cdot
b!\cdot(p_{e}!)^{a_{e}/2}\cdot(p_{o}!)^{a_{o}/2}}\operatorname{\mathds{E}}(H_{a_{e}}(H_{p_{e}}(\eta)/\sqrt{p_{e}!})\cdot
H_{a_{o}}(H_{p_{o}}(\eta)/\sqrt{p_{o}!})\cdot H_{b}(\eta)),$

where $a_{e},a_{o},b\in\mathbb{N}$. This corresponds to the scaled variance
for the cluster structure with $a_{e}$ many $p_{e}$-hyperedges, $a_{o}$ many
$p_{o}$-hyperedges, and $b$ many odd-degree vertices; these structures are
summarized in Table 2. Again we will drop the dependence on $p_{e},p_{o}$ for
convenience by using $v_{a_{e},a_{o},b}(\beta,\rho)^{2}$. In the case $c=0$
corresponding to $h=0$ case, we further drop the dependence on $\rho$ as in
pure $p$-spin case, _i.e.,_ we will use $v_{a_{e},a_{o},0}(\beta)^{2}$.

######  Theorem 2.1 (Mixed $p$-spin at $h=0$).

Under the same assumptions as in Theorem 1.10,

  * •

If $p_{e}<p_{o}$, then we have

$\displaystyle
N^{3(p_{e}-2)/4}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v_{\textup{mix},1}^{2})\text{
in distribution as $N\to\infty$,}$

where $v_{\textup{mix},1}^{2}:=v_{3,0,0}(\beta)^{2}$ as defined in (10).

  * •

If $p_{o}<p_{e}<2p_{o}$, then we have

$\displaystyle
N^{(2p_{o}+p_{e}-6)/4}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v_{\textup{mix},2}^{2})\text{
in distribution as $N\to\infty$,}$

where

$\displaystyle
v_{\textup{mix},2}^{2}:=\begin{cases}v_{1,2,0}(\beta)^{2},&p_{o}<p_{e}<2(p_{o}-1),\\\
v_{1,2,0}(\beta)^{2}+v_{0,4,0}(\beta)^{2},&p_{e}=2(p_{o}-1).\end{cases}$

  * •

If $p_{e}\geqslant 2p_{o}$, then we have

$\displaystyle
N^{(p_{o}-2)}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v_{\textup{mix},3}^{2})\text{
in distribution as $N\to\infty$,}$

where $v_{\textup{mix},3}^{2}:=v_{0,4,0}(\beta)^{2}$.

The above theorem implies that if $p_{e}<p_{o}$, the mixed $p$-spin model
behaves like a pure $p_{e}$-spin model, similar in the case $p_{e}\geqslant
2p_{o}$. Only in the regime $p_{o}<p_{e}<2p_{o}$, the mixed model behaves like
a mixture. Furthermore, from the variance part and associated cluster
structures summarized in Table 2, one can see that $p_{e}=2(p_{o}-1)$ is a
transition point from mixture to pure $p_{o}$-spin case.

In order to state the results under a weak external field, we need to first
introduce some necessary notations. Define

$\displaystyle\alpha_{c}(p_{e},p_{o}):=\begin{cases}p_{e}/8,&\text{ if
}2<p_{e}<p_{o}-1,\\\ (p_{e}-2)/4,&\text{ if }2<p_{e}=p_{o}-1,\\\
p_{e}/8,&\text{ if }p_{o}<p_{e}<2(p_{o}-1),\\\ (p_{o}-1)/4,&\text{ if
}p_{e}\geqslant 2(p_{o}-1).\end{cases}$ (27)

A simple remark is that for $p_{e}=2$, the transition threshold is
$\alpha_{c}=\frac{1}{4}$, which was obtained in [DW21]. Similarly we next
define the associated 3 regimes
$\mathcal{R}_{1}(\alpha_{c}(p_{o},p_{e})),\mathcal{R}_{2}(\alpha_{c}(p_{o},p_{e})),\mathcal{R}_{3}$
as in (14). The corresponding scaling exponents are defined as

$\displaystyle\gamma_{1}:=\begin{cases}\frac{3}{2}(p_{e}-2),&\text{ if
}p_{e}<p_{o},\\\ p_{o}+\frac{1}{2}p_{e}-3,&\text{ if }p_{o}<p_{e}<2p_{o},\\\
2(p_{o}-2),&\text{ if }p_{e}\geqslant 2p_{o}.\end{cases}$ (28)

and

$\displaystyle\gamma_{2}:=\gamma_{2}(\alpha)=\begin{cases}(p_{e}-3+4\alpha),&\text{
if }p_{e}<p_{o}-1,\\\ (p_{e}-2+2\alpha),&\text{ if }p_{e}=p_{o}-1,\\\
(p_{o}-3+4\alpha),&\text{ if }p_{e}=p_{o}+1,\\\ (p_{o}-3+4\alpha),&\text{ if
}p_{e}>p_{o}+1.\end{cases}$ (29)

and

$\displaystyle\gamma_{3}:=\gamma_{3}(\alpha)=\begin{cases}(2\alpha
p_{e}-1),&\text{ if }p_{e}<p_{o},\\\ (2\alpha p_{o}-1),&\text{ if
}p_{e}>p_{o}.\end{cases}$ (30)

The heuristic picture for the above components is as follows. In the zero
external field, the leading clusters are
$(3_{p_{e}},0),(1_{p_{e}},2_{p_{o}},0)$, and $(4_{p_{o}},0)$. To find
$\alpha_{c}$ after adding a weak external field, we have to compute the
contribution from the new clusters with odd-degree vertices and find when
their contribution is in the same order as the clusters at $h=0$. The
specialty is due to the $(1_{p_{e}},1_{p_{o}},1)$ cluster shown in Figure 5,
and it seems to be a discontinuity for the transition w.r.t $p$. The details
of those cluster structures and variance components are summarized in Table 2.
In terms of external field strength, there are three regimes for
$\log\hat{Z}_{N}(\beta,h)$ similarly as in pure $p$ case.

Figure 5. New leading clusters in the mixed $p$-spin models. For example, with
$p_{o}=5$, the left one corresponds to the $(1_{p_{o}+1},2_{p_{o}},0)$
cluster. The right one is the $(1_{p_{o}-1},1_{p_{o}},1)$ cluster.

We state the main theorem for mixed $p$-spin models under weak external field
in Theorem 2.2. We will focus on the case $\theta_{2}=0$. When $\theta_{2}>0$,
the $p=2$ clusters dominate, and one can replicate the proof for the SK model
(see [DW21]) in this case.

######  Theorem 2.2 (Mixed $p$-spin model weak external field).

Under the same assumptions in Theorem 1.13, we have

  1. a)

Sub-critical regime: If $\alpha\in[\alpha_{c}(p_{e},p_{o}),\infty)$, we have

$\displaystyle
N^{\gamma_{1}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,u_{1}^{2})$ (31)

in distribution as $N\to\infty$ where

$\displaystyle
u_{1}^{2}:=\begin{cases}\hat{u}_{1}^{2},&\alpha<\alpha_{c}(p_{e},p_{o}),\\\
\hat{u}_{1,c}^{2},&\alpha=\alpha_{c}(p_{e},p_{o}).\end{cases}$

with

$\displaystyle\hat{u}_{1}^{2}:=\begin{cases}v_{\textup{mix},1}^{2},&p_{e}<p_{o},\\\
v_{\textup{mix},2}^{2},&p_{o}<p_{e}<2p_{o},\\\
v_{\textup{mix},3}^{2},&p_{e}\geqslant 2p_{o}.\end{cases}$

and

$\displaystyle\hat{u}_{1,c}^{2}:=\begin{cases}v_{3,0,0}(\beta)^{2}+v_{2,0,2}(\beta,\rho)^{2},&p_{e}<p_{o}-1,\\\
v_{3,0,0}(\beta)^{2}+v_{1,1,1}(\beta,\rho)^{2},&p_{e}=p_{o}-1,\\\
v_{1,2,0}(\beta)^{2}+v_{1,1,1}(\beta,\rho)^{2},&p_{e}=p_{o}+1,\\\
v_{1,2,0}(\beta)^{2}+v_{0,2,2}(\beta,\rho)^{2},&p_{o}+1<p_{e}<2(p_{o}-1),\\\
v_{1,2,0}(\beta)^{2}+v_{0,4,0}(\beta)^{2}+v_{0,2,2}(\beta,\rho)^{2}+v_{0,3,1}(\beta,\rho)^{2},&p_{e}=2(p_{o}-1),\\\
v_{0,4,0}(\beta)^{2}+v_{0,2,2}(\beta,\rho)^{2}+v_{0,3,1}(\beta,\rho)^{2},&p_{e}\geqslant
2p_{o}.\end{cases}$

  2. b)

Unified regime I: If $\alpha\in[\nicefrac{{1}}{{2}},\alpha_{c}(p_{e},p_{o}))$,
we have

$\displaystyle
N^{\gamma_{2}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,u_{2}^{2})$

in distribution as $N\to\infty$ where

$\displaystyle
u_{2}^{2}:=\begin{cases}\hat{u}_{2}^{2},&\alpha\in(1/2,\alpha_{c}(p_{e},p_{o})),\\\
\hat{u}_{2,c}^{2},&\alpha=1/2.\end{cases}$

with

$\displaystyle\hat{u}_{2}^{2}:=\begin{cases}v_{2,0,2}(\beta,\rho)^{2},&p_{e}<p_{o}-1,\\\
v_{1,1,1}(\beta,\rho)^{2},&p_{e}=p_{o}-1,\\\
v_{1,1,1}(\beta,\rho)^{2},&p_{e}=p_{o}+1,\\\
v_{0,2,2}(\beta,\rho)^{2},&p_{e}>p_{o}+1.\end{cases}$

and

$\displaystyle\hat{u}_{2,c}^{2}:=\begin{cases}v_{2,0,2}(\beta,\rho)^{2}+v_{1,0,p_{e}}(\beta,\rho)^{2},&p_{e}<p_{o}-1,\\\
v_{1,1,1}(\beta,\rho)^{2}+v_{1,0,p_{e}}(\beta,\rho)^{2},&p_{e}=p_{o}-1,\\\
v_{1,1,1}(\beta,\rho)^{2},+v_{0,1,p_{o}}(\beta,\rho)^{2}&p_{e}=p_{o}+1,\\\
v_{0,2,2}(\beta,\rho)^{2}+v_{0,1,p_{o}}(\beta,\rho)^{2},&p_{e}>p_{o}+1.\end{cases}$

  3. c)

Unified regime II: If $\alpha\in[\nicefrac{{1}}{{4}},\nicefrac{{1}}{{2}})$, we
have

$\displaystyle
N^{\gamma_{3}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,u_{3}^{2})$

in distribution as $N\to\infty$ where
$u_{3}^{2}:=\begin{cases}v_{1,0,p_{e}}(\beta,\rho)^{2},&p_{e}<p_{o},\\\
v_{0,1,p_{o}}(\beta,\rho)^{2},&p_{e}>p_{o}.\end{cases}$

We will reuse the notation $\mathcal{R}_{1},\mathcal{R}_{2},\mathcal{R}_{3}$
as in Section 1 to denote the sub-critical regime, unified regime I, and
unified regime II, respectively. In the $h=0$ case, we discussed the criterion
of classifying the mixed $p$-spin models in terms of the transition results
w.r.t $p$. For the model under a weak external field, it is interesting that
the mixed model behaves more like a pure model if the weak external field is
strong enough. In particular, for $\alpha\in\mathcal{R}_{3}$, if
$p_{e}<p_{o}$, the mixed model behaves like a pure $p_{e}$-spin glass and pure
$p_{o}$-spin otherwise.

######  Remark 2.3 (Connection to the triviality of geometry).

Notice from Table 2, as the external field gets strong enough, for example,
for $\alpha\in\mathcal{R}_{3}$, the dominant cluster becomes trivial, which is
a single hyperedge. On the other hand, in the study of geometry for spherical
mixed $p$-spin glass models, it is known [DJSM21] that in the zero external
field case, the Hamiltonian, as a random function, has nontrivial geometry in
the sense that it has exponentially many critical points. However, if the
external field is strong enough, then the Hamiltonian only has two critical
points and thus has a trivial geometry. It will be interesting to explore the
connections between those two triviality behaviors. We leave this to future
work. There is a long line of works on the triviality of the geometry in
spherical spin glass models, one can check [FL14, Fyo15, RBBC19] and
references therein.

With a good understanding of $\log\hat{Z}_{N}(\beta,h)$, we can similarly
deduce the transitional results for free energy. The expansion of
$\log\bar{Z}_{N}$ is similar to in pure $p$-spin case, as we know that as $p$
gets larger, the variance order gets smaller. Therefore, the dominant term
comes from the smallest one $p_{m}:=\min\\{p_{e},p_{o}\\}$ and we have

$\displaystyle\log\bar{Z}_{N}(\beta,h)$ $\displaystyle=\sum_{p\geqslant
3}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\log\cosh(\beta\theta_{p}\cdot
J_{\boldsymbol{i}}/N^{(p-1)/2})$ (32)
$\displaystyle\approx\frac{\beta^{2}\theta_{p_{m}}^{2}\binom{N}{p_{m}}}{2N^{p_{m}-1}}-\frac{\beta^{4}\theta_{p_{m}}^{4}\binom{N}{p_{m}}}{12N^{2p_{m}-2}}\cdot\operatorname{\mathds{E}}J^{4}+\operatorname{N}\left(0,\frac{\beta^{4}\theta_{p_{m}}^{4}\binom{N}{p_{m}}}{4N^{2p_{m}-2}}\cdot\operatorname{Var}(J^{2})\right).$
(33)

######  Corollary 2.4.

Recall $p_{m}=\min\\{p_{e},p_{o}\\}$. Assume that
$\alpha\geqslant\nicefrac{{1}}{{4}}$, and
$J_{\boldsymbol{i}},\boldsymbol{i}\in\cup_{p\geqslant 3}\mathscr{E}_{N,p}$
satisfy Assumption 1.2. Then for $\beta<\beta_{c}$,

  1. a)

If $\alpha>\frac{p_{m}-1}{2p_{m}}$, we have

$\displaystyle N^{(p_{m}-2)/2}\cdot\left(\log
Z_{N}-\binom{N}{p_{m}}\operatorname{\mathds{E}}\log\cosh(\beta\theta_{p_{m}}J/N^{(p_{m}-1)/2})\right)\to\operatorname{N}(0,v_{1}^{2}),$

where
$v_{1}^{2}=\beta^{4}\theta_{p_{m}}^{4}\operatorname{Var}(J^{2})/4p_{m}!$.

  2. b)

If $\alpha=\frac{p_{m}-1}{2p_{m}}$, we have

$\displaystyle N^{(p_{m}-2)/2}\cdot\left(\log
Z_{N}-\binom{N}{p_{m}}\operatorname{\mathds{E}}\log\cosh(\beta\theta_{p_{m}}J/N^{(p_{m}-1)/2})\right)\to\operatorname{N}(0,v_{2}^{2}),$

where $v_{2}^{2}:=v_{1}^{2}+v_{1,p_{m}}(\beta,\rho)^{2}$ with
$v_{1,p_{m}}(\beta,\rho)^{2}:=\begin{cases}v_{1,0,p_{e}}(\beta,\rho)^{2},&\text{if}\
p_{m}=p_{e},\\\ v_{0,1,p_{o}}(\beta,\rho)^{2},&\text{if}\
p_{m}=p_{o}.\end{cases}$

  3. c)

If $\frac{1}{4}\leqslant\alpha<\frac{p_{m}-1}{2p_{m}}$,we have

$\displaystyle N^{(2\alpha p_{m}-1)/2}\cdot\left(\log
Z_{N}-\binom{N}{p_{m}}\operatorname{\mathds{E}}\log\cosh(\beta\theta_{p_{m}}J/N^{(p_{m}-1)/2})\right)\to\operatorname{N}(0,v_{3}^{2})$

where $v_{3}^{2}:=v_{1,p_{m}}(\beta,\rho)^{2}$.

######  Remark 2.5 (multi-scale fluctuation for the free energy).

As we pointed out in Remark 1.12, the multi-scale fluctuation phenomenon holds
for $\log\hat{Z}_{N}$ in the pure $p$-spin model. However, this is not the
case on the free energy level due to the dominance of $\log\bar{Z}_{N}$. In
the mixed $p$-spin setting, even on the free energy level, there still exists
a similar multi-scale fluctuation phenomenon due to the two-parameter multiple
transition picture. From the above corollary, if we subtract the
$\Theta(N^{-(p_{m}-2)})$ terms, there should be another limit theorem in a
smaller scale $\Theta(N^{-(q_{m}-2)})$, where $q_{m}:=\min\\{p\neq
p_{m}:\theta_{p}>0\\}$ is the second smallest effective $p$-spins.

Table 2. Summary of the transition behavior for mixed $p$-spin models with $\alpha_{c}=\alpha_{c}(p_{o},p_{e})$. External field $h$ | Condition | $\gamma$ | Cluster Structure
---|---|---|---
$\alpha\in(\alpha_{c},\infty]$ | $p_{e}<p_{o}$ | $3(p_{e}-2)/2$ | $(3_{p_{e}},0)$
$p_{o}<p_{e}<2(p_{o}-1)$ | $p_{o}-3+p_{e}/2$ | $(1_{p_{e}},2_{p_{o}},0)$
$p_{e}=2(p_{o}-1)$ | $p_{o}-3+p_{e}/2$ | $(1_{p_{e}},2_{p_{o}},0),(4_{p_{o}},0)$
| $p_{e}\geqslant 2p_{o}$ | $2(p_{o}-2)$ | $(4_{p_{o}},0)$
$\alpha=\alpha_{c}(p_{o},p_{e})$ | $p_{e}<p_{o}-1$ | $3(p_{e}-2)/2$ | $(3_{p_{e}},0),(2_{p_{e}},2)$
$p_{e}=p_{o}-1$ | $3(p_{e}-2)/2$ | $(3_{p_{e}},0),(1_{p_{e}},1_{p_{o}},1)$
$p_{e}=p_{o}+1$ | $p_{o}-3+p_{e}/2$ | $(1_{p_{e}},2_{p_{o}},0),(1_{p_{e}},1_{p_{o}},1)$
$p_{o}+1<p_{e}<2(p_{o}-1)$ | $p_{o}-3+p_{e}/2$ | $(1_{p_{e}},2_{p_{o}},0),(2_{p_{o}},2)$
$p_{e}=2(p_{o}-1)$ | $p_{o}-3+p_{e}/2$ |  $(1_{p_{e}},2_{p_{o}},0),(4_{p_{o}},0)$,
|  |  | $(2_{p_{o}},2),(3_{p_{o}},1)$
| $p_{e}\geqslant 2p_{o}$ | $2(p_{o}-2)$ | $(4_{p_{o}},0),(2_{p_{o}},2),(3_{p_{o}},1)$
$\alpha\in(1/2,\alpha_{c})$ | $p_{e}<p_{o}-1$ | $p_{e}-3+4\alpha$ | $(2_{p_{e}},2)$
$p_{e}=p_{o}-1$ | $p_{e}-2+2\alpha$ | $(1_{p_{e}},1_{p_{o}},1)$
$p_{e}=p_{o}+1$ | $p_{o}-3+4\alpha$ | $(1_{p_{e}},1_{p_{o}},1)$
$p_{e}>p_{o}+1$ | $p_{o}-3+4\alpha$ | $(2_{p_{o}},2)$
$\alpha=1/2$ | $p_{e}<p_{o}-1$ | $p_{e}-1$ | $(2_{p_{e}},2),(1_{p_{e}},p_{e})$
$p_{e}=p_{o}-1$ | $p_{e}-1$ | $(1_{p_{e}},1_{p_{o}},1),(1,p_{e})$
$p_{e}=p_{o}+1$ | $p_{o}-1$ | $(1_{p_{e}},1_{p_{o}},1),(1,p_{o})$
$p_{e}>p_{o}+1$ | $p_{o}-1$ | $(2_{p_{o}},2),(1_{p_{o}},p_{o})$
$\alpha\in[1/4,1/2)$ | $p_{e}<p_{o}$ | $2\alpha p_{e}-1$ | $(1_{p_{e}},p_{e})$
$p_{e}>p_{o}$ | $2\alpha p_{o}-1$ | $(1_{p_{o}},p_{o})$

###  2.1. A word on the proof

We briefly discuss the extra challenges for the proof of the main results in
mixed $p$-spin setting and our strategy to deal with it. The first and core
challenge is caused by the flexibility of $p$-hyperedges. We must show that
the dominant clusters can not include large $p$-hyperedges. This is similar to
the large hypergraph decay results in Proposition 4.3. The high-level idea for
establishing this result is to decompose the large $p$-hyperedge into several
smaller hyperedges, this will increase the cluster size, and then we use the
large hypergraph decaying results to conclude. Once we know that the dominant
clusters will have finitely many hyperedges and all the hyperedges are finite
tuples, the next challenge will be identifying the leading clusters. Although
the cluster enjoys the above nice property, there still can be many
possibilities due to various combinations of different $p$-hyperedges. This
part needs very careful analysis. In particular, the leading clusters change
w.r.t. two parameters, external field strength, and the mixture of hyperedges.
Finally, after obtaining all the dominant clusters, we use multivariate
Stein’s method in Theorem 1.17 to establish the joint convergence of different
clusters to the normal limit. The computations there become more involved in
the pure $p$-spin case. We highlight the ideas in Section 7.

##  3\. Results for Multi-Species Mixed $p$-Spin Models

The multi-species model was originally introduced in [BCMT15]. It can be
regarded as a natural inhomogeneous extension of the classical mean field spin
glass models. In recent years, this model has received much attention due to
its connections to the neural network [ACM21, ABCM20, ACCM21] and its inherent
fundamental challenges in the non-convex case [Mou21]. Let us first define
multi-species mixed $p$-spin models. Consider a set $\mathscr{S}$ of species
with $\left|\mathscr{S}\right|=k<\infty$, for the number of spins $N>0$, let

$\displaystyle\\{1,2,\ldots,N\\}=\bigcup_{s\in\mathscr{S}}I_{s},\quad\text{and}\
I_{s}\cap I_{t}=\emptyset\ \text{for}\ s\neq t.$

Assume the density ratio of species $s\in\mathscr{S}$ is
$\lambda_{s}:=\lim_{N\to\infty}\frac{\left|I_{s}\right|}{N}$. For each $p>3$,
let

$\Delta_{p}^{2}:=(\Delta_{s_{1},\ldots,s_{p}}^{2})_{s_{1},\ldots,s_{p}\in\mathscr{S}}\in\mathds{R}_{+}^{\left|\mathscr{S}\right|^{p}}$

be a symmetric $p$-dimensional tensor. For
$\boldsymbol{\sigma}\in\\{-1,+1\\}^{N}$, the Hamiltonian of multi-species pure
$p$-spin model can be defined as follows:

$\displaystyle
H_{N,\Delta_{p}}(\boldsymbol{\sigma})=\frac{1}{N^{(p-1)/2}}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\Delta_{s(\boldsymbol{i})}J_{\boldsymbol{i}}\sigma_{\boldsymbol{i}},$
(34)

where $s(\boldsymbol{i})$ maps each coordinate of the $p$-tuple
$\boldsymbol{i}$ to a species in $\mathscr{S}$. The Hamiltonian of multi-
species mixed $p$-spin model is

$\displaystyle H_{N,\Delta}(\boldsymbol{\sigma}):=\sum_{p\geqslant
3}H_{N,\Delta_{p}}(\boldsymbol{\sigma}).$ (35)

To make sure the model is well-defined, it needs the following fast decaying
assumption,

$\displaystyle\sum_{p\geqslant 3}\left\|\Lambda^{\otimes
p}\Delta_{p}^{2}\right\|_{\infty}/(p-2)!<\infty.$ (36)

Here the limiting structure function is

$\displaystyle\xi(\boldsymbol{x}):=\sum_{p\geqslant
3}\frac{1}{p!}\langle\Lambda^{\otimes p}\Delta_{p}^{2},\boldsymbol{x}^{\otimes
p}\rangle\text{ for }\boldsymbol{x}\in[-1,1]^{k}.$

Parisi formula for the multi-species SK model was first established in [Pan15]
under the positive semi-definite assumption on $\Delta$. For general
indefinite $\Delta$, computing the limiting free energy still remains a
challenging open question. The fluctuation results for the general multi-
species SK model at a high-temperature regime were obtained in [DW20]. In the
spherical setting, some progress has been made in evaluating the limiting free
energy. Recently, for multi-species mixed $p$-spin spherical models, Bates and
Sohn established the Crisanti-Sommers formula in [BateS22a, BateS22b] under
the positive semi-definite assumption on $\Delta$. Shortly after these works,
Subag [Sub21b, sub21c] utilized the Thouless-Anderson-Palmer (TAP) approach
instead to compute the limiting free energy and ground state energy for multi-
species pure $p$-spin case. Notably, this approach works for general
indefinite $\Delta$. The fluctuation results for both Ising and spherical
multi-species mixed $p$-spin models are still missing. Using the intuition
from the last section, we can easily obtain the fluctuation results for
general indefinite multi-species Ising mixed $p$-spin models in the entire
high-temperature regime.

First, let us introduce the characterization of the critical inverse
temperature for multi-species models.

$\displaystyle\beta_{c}(\xi):=\inf_{\boldsymbol{x}\in(0,1]^{k}}\sqrt{\frac{\sum_{i=1}^{k}\lambda_{i}I(x_{i})}{\xi(\boldsymbol{x})}}.$

Analogue of Theorem 1.4 can be easily established in the multi-species
setting. One needs to carefully trace the structure-function.

######  Theorem 3.1.

Under similar assumptions as 1.1, 1.2 and 1.3 for multi-species model. For
$\beta<\beta_{c}(\xi)$, we have for all $N$ large

$\displaystyle\operatorname{\mathds{E}}(\hat{Z}_{N}(\beta,h))^{2}\leqslant\frac{C}{(\beta_{c}^{2}-\beta^{2})^{k/2}}\exp\left(\frac{C\rho^{4}\beta^{2}}{\beta_{c}^{2}-\beta^{2}}\right)$

where $C>0$ is a universal constant. In particular, we have

$\displaystyle\frac{1}{N}\log Z_{N}(\beta)\to\frac{1}{2}\beta^{2}\xi(1)\text{
in probability as }N\to\infty..$ (37)

######  Remark 3.2.

The proof of Theorem 3.1 is morally the same as the single species case in
Theorem 1.4, whose proof is given in Section 4. For multi-species case, one
needs to work on the following vector of magnetization
$S_{N}^{\mathscr{S}}:=\left(\sum_{i\in
I_{s}}\sigma_{i}\right)_{s\in\mathscr{S}}$ and also carefully trace the
structure-function $\xi(\boldsymbol{x})=\sum_{p\geqslant
3}\frac{1}{p!}\langle\Lambda^{\otimes p}\Delta_{p}^{2},\boldsymbol{x}^{\otimes
p}\rangle$.

With the critical inverse temperature, we now state the multi-species
fluctuation results up to $\beta_{c}$. Before that, some necessary notations
are needed. Let

$\displaystyle p_{e}$ $\displaystyle:=\min\\{\text{even }p\geqslant
4:\text{$\Delta_{p}$ is not the zero $p$-tensor}\\},$ $\displaystyle p_{o}$
$\displaystyle:=\min\\{\text{odd }p\geqslant 3:\text{$\Delta_{p}$ is not the
zero $p$-tensor}\\}.$

By abusing the notation a bit, we continue to use the notation for the
partition function $Z_{N}(\beta,h)$ and its similar decomposition in the
multi-species setting. We state the analog of Theorem 2.1. For convenience, we
assume that
$\operatorname{Tr}(\Delta_{p_{e}}),\operatorname{Tr}(\Delta_{p_{o}})>0$, to
make sure that the leading cluster structure remains the same as in the
single-species case. In the general $\Delta_{p}$ case, a similar result holds
after identifying the leading cluster structure.

######  Theorem 3.3 (multi-species mixed $p$-spin at $h=0$).

Under the same assumptions as in Theorem 1.10,

  * •

If $p_{e}<p_{o}$, then we have

$\displaystyle
N^{3(p_{e}-2)/4}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{ in
distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

  * •

If $p_{o}<p_{e}\leqslant 2p_{o}$, then we have

$\displaystyle
N^{(2p_{o}+p_{e}-6)/4}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{
in distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

  * •

If $p_{e}>2p_{o}$, then we have

$\displaystyle
N^{(p_{o}-2)}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{ in
distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

The variances $v^{2}$ in each regime can be explicitly computed once the
leading clusters are identified. We do not pursue the most general form here.
The transitional results are also essentially same as in the single species
case. The thresholds and corresponding scaling components can be defined
similarly. We state the results as follows without repeating the definitions.
The only difference lies in the explicit asymptotic variance.

######  Theorem 3.4 (multi-species mixed $p$-spin under weak external field).

Under the same assumptions in Theorem 1.13, we have

  1. a)

Sub-critical regime: If $\alpha\in[\alpha_{c}(p_{o},p_{e}),\infty)$, we have

$\displaystyle
N^{\gamma_{1}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{ in
distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

  2. b)

Unified regime I: If $\alpha\in[\nicefrac{{1}}{{2}},\alpha_{c}(p_{o},p_{e}))$,
we have

$\displaystyle
N^{\gamma_{2}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{ in
distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

  3. c)

Unified regime II: If $\alpha\in[\nicefrac{{1}}{{4}},\nicefrac{{1}}{{2}}]$, we
have

$\displaystyle
N^{\gamma_{3}/2}\cdot\log\hat{Z}_{N}\to\operatorname{N}(0,v^{2})\text{ in
distribution as $N\to\infty$,}$

for some $v^{2}\in(0,\infty)$.

After this result, one can similarly obtain the analog of Corollary 2.4. We do
not state it formally here. The proof of the main results for the multi-
species model is basically same as the single-species case. The steps for
obtaining finite leading clusters are nearly same since the inhomogeneous
variance profile $\Delta$ has all finite entries and there are finitely many
different species, one can always use a uniform bound to prove the analog of
Proposition 4.3, 4.4 and 4.6. Identifying the leading clusters needs only the
variance order. Therefore, the only thing changing is the asymptotic variance
forms. After this, one can repeat the multivariate Stein’s method to establish
the Gaussian limit as discussed in Section 1.4.

##  4\. Large Hypergraph Decay Regime

###  4.1. Large hypergraph decay

We begin with the proof of the property that large hypergraph contributions
decay fast in the whole high-temperature regime. Compared to the SK ($p=2$)
model, the Gaussian trick (see [DW21]*Lemma 4.1) for controlling the second
moments of $\hat{Z}_{N}$ is not available any more due to the loss of
quadratic features. Instead, we seek an integral approximation of the targeted
discrete sum, then directly control the integral upper bound. We also obtain
an interesting characterization of the critical temperature for the given spin
glass model.

We first look at the second moment of $\hat{Z}_{N}(\beta,h)$. We define
$\left|\boldsymbol{i}\right|=p$ for $\boldsymbol{i}\in\mathscr{E}_{N,p}$.
Recall that

$\displaystyle\hat{\omega}_{\boldsymbol{i}}=\tanh(\beta\theta_{p}\cdot
J_{\boldsymbol{i}}/N^{(p-1)/2})\text{ for }\boldsymbol{i}\in\cup_{p\geqslant
2}\mathscr{E}_{N,p}$

and

$\displaystyle\hat{Z}_{N}(\beta,h)=\operatorname{\mathds{E}}_{h}\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}(1+\sigma_{\boldsymbol{i}}\cdot\hat{\omega}_{\boldsymbol{i}}).$

######  Lemma 4.1.

Assume 1.1, 1.2 and 1.3. For $\beta<\beta_{c}(\xi)$, we have for all $N$
large,

$\displaystyle\operatorname{\mathds{E}}(\hat{Z}_{N}(\beta,h))^{2}\leqslant\frac{c}{\sqrt{\beta_{c}^{2}-\beta^{2}}}\exp\left(\frac{\rho^{4}\beta^{2}}{\beta_{c}^{2}-\beta^{2}}\right)$

where $c>0$ is a universal constant.

######  Remark 4.2.

In Lemma 4.1 we do not need $\theta_{2}$ to be $0$. However, when
$\theta_{2}=0$ one can get a better $N$-dependent upper bound. Moreover, using
$\operatorname{\mathds{E}}\tanh^{2}(\beta
J/N^{(p-1)/2})\leqslant\beta^{2}/N^{p-1}$, one can replace
$\operatorname{\mathds{E}}(\hat{Z}_{N}(\beta,h))^{2}$ by
$\sum_{\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p}}h^{2\left|\partial\Gamma\right|}\prod_{p\geqslant
3}\left(\theta_{p}^{2}\beta^{2}/N^{p-1}\right)^{\left|\Gamma\right|_{p}}$ in
Lemma 4.1, where $|\Gamma|_{p}$ is the number of $p$-hyperedges in $\Gamma$.

###### Proof.

We notice that

$\displaystyle\hat{Z}_{N}(\beta,h)^{2}=\operatorname{\mathds{E}}_{h}\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}(1+(\sigma_{\boldsymbol{i}}+\tau_{\boldsymbol{i}})\cdot\hat{\omega}_{\boldsymbol{i}}+\sigma_{\boldsymbol{i}}\tau_{\boldsymbol{i}}\cdot\hat{\omega}_{\boldsymbol{i}}^{2}).$

After taking the expectation with respect to the disorder, we have

$\displaystyle\phi_{N}(\beta,h):=\operatorname{\mathds{E}}\hat{Z}_{N}(\beta,h)^{2}$
$\displaystyle=\operatorname{\mathds{E}}_{h}\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}(1+\sigma_{\boldsymbol{i}}\tau_{\boldsymbol{i}}\operatorname{\mathds{E}}\hat{\omega}_{\boldsymbol{i}}^{2})$
$\displaystyle=\operatorname{\mathds{E}}_{\tilde{h}^{2}}\prod_{p\geqslant
2}\prod_{\boldsymbol{i}\in\mathscr{E}_{N,p}}(1+\sigma_{\boldsymbol{i}}\cdot\operatorname{\mathds{E}}\hat{\omega}_{\boldsymbol{i}}^{2})\leqslant\operatorname{\mathds{E}}_{\tilde{h}^{2}}\exp\left(\sum_{p\geqslant
2}\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\operatorname{\mathds{E}}\hat{\omega}_{\boldsymbol{i}}^{2}\cdot\sigma_{\boldsymbol{i}}\right)$

where $\tilde{h}^{2}:=\tanh^{-1}(\tanh^{2}(h))\leqslant h^{2}$. Note that,

$\displaystyle\operatorname{\mathds{E}}\hat{\omega}_{\boldsymbol{i}}^{2}\leqslant\beta^{2}\theta_{p}^{2}/N^{p-1}\text{
for }\boldsymbol{i}\in\mathscr{E}_{N,p}$

and

$\displaystyle\left|\left(\sum_{i=1}^{N}\sigma_{i}\right)^{p}-p!\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\sigma_{\boldsymbol{i}}\right|\leqslant
N^{p}-(N)_{p}=N^{p}\left(1-\prod_{k=1}^{p-1}(1-k/N)\right)\leqslant\binom{p}{2}N^{p-1}.$

Thus

$\displaystyle\phi_{N}(\beta,h)$
$\displaystyle\leqslant\operatorname{\mathds{E}}_{\tilde{h}^{2}}\prod_{p\geqslant
2}\exp\biggl{(}N\beta^{2}\theta_{p}^{2}/p!\cdot|S_{N}/N|^{p}+\beta^{2}\theta_{p}^{2}/2(p-2)!\biggr{)}$
(38) $\displaystyle=\exp\biggl{(}\frac{1}{2}\beta^{2}\sum_{p\geqslant
2}\theta_{p}^{2}/(p-2)!\biggr{)}\cdot\operatorname{\mathds{E}}_{\tilde{h}^{2}}\exp\biggl{(}N\beta^{2}\cdot\xi(|S_{N}/N|)\biggr{)}$
(39)

where $S_{N}:=\sum_{i=1}^{N}\sigma_{i}$ and $\xi(x)=\sum_{p\geqslant
2}\theta_{p}^{2}x^{p}/p!$. Continuing from the r.h.s, we have

$\displaystyle\operatorname{\mathds{E}}_{\tilde{h}^{2}}\exp\biggl{(}N\beta^{2}\cdot\xi(|S_{N}/N|)\biggr{)}$
$\displaystyle=e^{-N\log\cosh\tilde{h}^{2}}\cdot\operatorname{\mathds{E}}_{0}\exp\biggl{(}N\beta^{2}\cdot\xi(|S_{N}/N|)+\tilde{h}^{2}\cdot
S_{N}\biggr{)}$
$\displaystyle\leqslant\operatorname{\mathds{E}}_{0}\exp\biggl{(}N\beta^{2}\cdot\xi(|S_{N}/N|)+\tilde{h}^{2}\cdot
S_{N}\biggr{)}.$

Here the expectation $\operatorname{\mathds{E}}_{0}$ is with respect to the
i.i.d. $\\{-1,+1\\}$ valued $\sigma_{i}$ with mean $0$. Our goal is to derive
a uniform bound on the r.h.s. of the above expression.

Now with $t=\beta/\beta_{c}\in(0,1)$, we have

$\displaystyle\beta^{2}\cdot\xi(x)\leqslant t^{2}\cdot I(x)\text{ for all
}x\in[0,1]$

where

$\displaystyle I(x)=\frac{1}{2}(1+x)\ln(1+x)+\frac{1}{2}(1-x)\ln(1-x).$

Moreover, we will use the fact that

$\displaystyle\operatorname{\mathds{P}}(|S_{N}|\geqslant Nx)\leqslant
2\exp(-NI(x)),$

and

$\displaystyle x^{2}/2\leqslant I(x)\leqslant x^{2}/2\cdot(1+x^{2}/2)$

for all $x\in[0,1]$. With

$\displaystyle f_{N}(x)=N\beta^{2}\xi(x/N)+\log\cosh(\hat{h}\cdot x)\leqslant
Nt^{2}\cdot I(x/N)+\log\cosh(\hat{h}\cdot x),$

which is convex and increasing in $[0,N]$ we can write

$\displaystyle\operatorname{\mathds{E}}_{0}\exp\biggl{(}N\beta^{2}\cdot\xi(|S_{N}/N|)+\hat{h}\cdot
S_{N}\biggr{)}$
$\displaystyle=\operatorname{\mathds{E}}_{0}\exp(f_{N}(|S_{N}|))$
$\displaystyle=\operatorname{\mathds{E}}_{0}\exp(f_{N}(|S_{N}|\wedge
k_{N}))+\operatorname{\mathds{E}}_{0}(\exp(f_{N}(|S_{N}|))-\operatorname{\mathds{E}}_{0}\exp(f_{N}(k_{N})))\mathds{1}_{|S_{N}|>k_{N}}$

for some $k_{N}$, to be chosen later so that

$\displaystyle
t_{N}:=t\cdot\sqrt{1+k_{N}^{2}/2N^{2}}\leqslant\sqrt{(1+t^{2})/2}$

or

$\displaystyle 1-t_{N}^{2}\geqslant(1-t^{2})/2>0.$

Thus, we have with $\eta\sim\operatorname{N}(0,1)$,

$\displaystyle\operatorname{\mathds{E}}_{0}\exp(f_{N}(|S_{N}|\wedge k_{N}))$
$\displaystyle\leqslant\operatorname{\mathds{E}}_{0}\exp(t_{N}^{2}S_{N}^{2}/2N+\hat{h}\cdot
S_{N})$ $\displaystyle=\operatorname{\mathds{E}}\exp(N\log\cosh(\eta\cdot
t_{N}+\sqrt{N\hat{h}^{2}})/\sqrt{N})\leqslant\operatorname{\mathds{E}}\exp(\nicefrac{{1}}{{2}}(\eta\cdot
t_{N}+\rho^{2})^{2}).$

Here, we used the Gaussian trick in the equality. Simplifying we get

$\displaystyle\operatorname{\mathds{E}}_{0}\exp(f_{N}(|S_{N}|\wedge k_{N}))$
$\displaystyle\leqslant(1-t_{N}^{2})^{-\nicefrac{{1}}{{2}}}\exp(\rho^{4}/2(1-t_{N}^{2})).$

Thus, we have

$\displaystyle\operatorname{\mathds{E}}_{0}(\exp(f_{N}(|S_{N}|))$
$\displaystyle-\operatorname{\mathds{E}}_{0}\exp(f_{N}(k_{N})))\mathds{1}_{|S_{N}|>k_{N}}$
$\displaystyle=\sum_{k=k_{N}+1}^{N}(\exp(f_{N}(k))-\exp(f_{N}(k-1)))\cdot\operatorname{\mathds{P}}(|S_{N}|\geqslant
k)$
$\displaystyle\leqslant\sum_{k=k_{N}+1}^{N}f_{N}^{\prime}(k)\exp(f_{N}(k))\cdot
2\exp(-NI(k/N)).$

Simplifying, we get

$\displaystyle\leqslant
2(\beta^{2}\xi^{\prime}(1)+\hat{h})\cdot\sum_{k=k_{N}+1}^{N}\exp(-(1-t^{2})NI(k/N)+\hat{h}k)$
$\displaystyle\leqslant
2(\beta^{2}\xi^{\prime}(1)+\hat{h})\cdot\sum_{k=k_{N}+1}^{N}\exp(-(1-t^{2})k^{2}/2N+\rho^{2}k/\sqrt{N})$
$\displaystyle\leqslant
c\sqrt{N}\int_{k_{N}/\sqrt{N}}^{\infty}\exp(-(1-t^{2})x^{2}/2+\rho^{2}x)\ dx$
$\displaystyle\leqslant
c\sqrt{N}\frac{1}{\sqrt{1-t^{2}}}\exp(\rho^{4}/2(1-t^{2}))\cdot\operatorname{\mathds{P}}(\eta\geqslant\sqrt{1-t^{2}}\cdot(k_{N}/\sqrt{N}-\rho^{2}/(1-t^{2}))).$

By choosing

$\displaystyle 4\rho^{4}/(1-t^{2})^{2}+8\log N/(1-t^{2})\leqslant
k_{N}^{2}/N\leqslant(1-t^{2})N$

we get the stated result. $\blacksquare$

From the analysis, it is relatively easy to derive the decay results for large
graph contributions in $p$-spin models. We formulate this into the following
Proposition. First denote that

$\displaystyle\hat{Z}_{N,m}:=\sum_{\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p},\left|\Gamma\right|\leqslant
m}\hat{h}^{\mid\partial\Gamma|}\cdot\hat{\omega}(\Gamma).$ (40)

######  Proposition 4.3 (Large hypergraph error decay).

If $\beta<\beta_{c}$, for $m\geqslant O_{\beta,\rho}(\log N)$, we have

$\displaystyle N^{p_{m}-2}\cdot\left\|\hat{Z}_{N}-\hat{Z}_{N,m}\right\|_{2}\to
0\text{ as }N\to\infty.$ (41)

###### Proof of Proposition 4.3.

Recall that $\beta_{N,p}^{2}=N^{(p-1)}\operatorname{\mathds{E}}\tanh^{2}(\beta
J/N^{(p-1)/2})$ and $h=\rho N^{-\alpha}$,

$\displaystyle\phi_{N}(\beta,h)=\operatorname{\mathds{E}}\hat{Z}_{N}(\beta,h)^{2}=\sum_{\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p}}\hat{h}^{2\left|\partial\Gamma\right|}\prod_{p\geqslant
3}\left(\frac{\theta_{p}^{2}\beta_{N,p}^{2}}{N^{p-1}}\right)^{\left|\Gamma\right|_{p}},$
(42)

where $\left|\Gamma\right|_{p}$ denote the number of $p$-hyperedges in the
hypergraph $\Gamma$, and $\left|\Gamma\right|=\sum_{p\geqslant
3}\left|\Gamma\right|_{p}$. On the other hand, we have

$\displaystyle\sum_{\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p},\left|\Gamma\right|\geqslant
m}\hat{h}^{2\left|\Gamma\right|}\prod_{p\geqslant
3}\left(\frac{\theta_{p}^{2}\beta_{N,p}^{2}}{N^{p-1}}\right)^{\left|\Gamma\right|_{p}}\leqslant
e^{-2my}\phi_{N}(e^{y}\beta,h).$

Following the bound for $\phi_{N}(\beta,h)$ derived before in Lemma 4.1 and
the Remark 4.2, we have

$\displaystyle\phi_{N}(\beta,h)\leqslant\frac{c}{\sqrt{\beta_{c}^{2}-\beta_{N,p}^{2}}}\exp\left(\frac{\rho^{4}\beta_{N,p}^{2}}{\beta_{c}^{2}-\beta_{N,p}^{2}}\right).$
(43)

Then

$\displaystyle N^{2(p_{m}-2)}\cdot\sum_{\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p},\left|\Gamma\right|\geqslant
m.}\hat{h}^{2\left|\Gamma\right|}\cdot\prod_{p\geqslant
3}\left(\frac{\theta_{p}^{2}\beta^{2}}{N^{p-1}}\right)^{\left|\Gamma\right|_{p}}$
$\displaystyle\leqslant cN^{2(p_{m}-2)}\cdot\exp(-2my),$

as long as $e^{y}\beta<\beta_{c}$ or $y<\log(\beta_{c}/\beta)$. From the last
step, as long as $my-(p_{m}-2)\log N\to\infty$ we have the required result.
$\blacksquare$

In the last proposition, we proved that the very large size clusters’
contribution to $\hat{Z}_{N}$ decays exponentially fast. This allows us to
deal with much smaller hypergraphs and simplify the problem significantly.
From the proof, it can be seen that this large graph decaying regime
intrinsically characterizes the critical inverse temperature $\beta_{c}$. In
the SK model ($p=2$ case) [DW21], this type of result is strong enough to
obtain a reduction of the clusters to finite size since the structures in the
ordinary graph are relatively simple. However, in the $p$-spin case, this
reduction can at most reduce the cluster size to $\log N$, which is due to the
complicated structures in the hypergraph. In the following part, we are
devoted to establishing a further reduction, namely reducing the $\log N$ size
clusters to finite size. The key technique we utilized is a form of chaos
expansion and second-moment computations. The strategy is as follows.

######  Proposition 4.4.

For given mixed $p$-spin models, and $\alpha\geqslant\frac{1}{4}$, let

$\displaystyle\hat{Z}_{N,4}:=\sum_{\left|\Gamma\right|\leqslant
4,\left|\partial\Gamma\right|\geqslant
0}\hat{h}^{\left|\partial\Gamma\right|}\cdot\hat{\omega}(\Gamma),$ (44)

we have

$\displaystyle
N^{p_{m}-2}\cdot\left\|\hat{Z}_{N,m}-\hat{Z}_{N,4}\right\|_{2}\to 0$ (45)

as $N\to\infty$.

######  Remark 4.5.

The rescaling $N^{p_{m}-2}$ corresponds to the smallest variance cluster (see
Table 2). The goal is to show that even under this strongest scaling, any
clusters of size greater than 4 are in subleading order. Thus, in all regimes,
it suffices to consider clusters of size less than or equal to 4.

###### Proof.

The proof is divided into two steps. First, we establish the results in the
setting of pure $p$-spin models, then we prove a helpful lemma that suffices
to establish the results for mixed $p$-spin models. For the pure $p$-spin
model, the second moment of the LHS is the sum over $\ell=5,6,\ldots,m$ of

$\displaystyle
N^{2(p-2)}\cdot\sum_{\left|\Gamma\right|=\ell,\left|\partial\Gamma\right|\geqslant
0}\hat{h}^{2\left|\partial\Gamma\right|}\operatorname{\mathds{E}}\prod_{i\in\Gamma}\tanh^{2}\left(\frac{\beta
J_{i}}{N^{(p-1)/2}}\right)$ $\displaystyle\leqslant
N^{2(p-2)}\cdot\sum_{\left|\Gamma\right|=\ell,\left|\partial\Gamma\right|\geqslant
0}\hat{h}^{2\left|\partial\Gamma\right|}\cdot\frac{\beta^{2\ell}}{N^{(p-1)\ell}}.$

Now the RHS can be bounded by

$\displaystyle
N^{2(p-2)}\cdot\frac{\beta^{2\ell}}{N^{(p-1)\ell}}\frac{1}{\ell!}\operatorname{\mathds{E}}_{\hat{h}}\left(\sum_{\boldsymbol{i}\in\mathscr{E}_{N,p}}\sigma_{\boldsymbol{i}}\right)^{\ell}$
$\displaystyle\leqslant
N^{2(p-2)}\cdot\frac{\beta^{2\ell}}{N^{(p-1)\ell}}\cdot\frac{1}{\ell!(p!)^{\ell}}\operatorname{\mathds{E}}_{\hat{h}}\left(\sum_{i=1}^{N}\sigma_{i}\right)^{p\ell}$

Recall that, $\boldsymbol{\sigma}=(\sigma_{1},\sigma_{2},\ldots,\sigma_{N})$
are i.i.d. with mean $\hat{h}=\tanh(h)$. Let
$\bar{\boldsymbol{\sigma}}=(\bar{\sigma}_{1},\bar{\sigma}_{2},\ldots,\bar{\sigma}_{N})$
be the centered version of $\boldsymbol{\sigma}$. Thus, we can further bound
the RHS by

$\displaystyle=N^{2(p-2)}\cdot\frac{\beta^{2\ell}}{N^{(p-1)\ell}}\cdot\frac{1}{\ell!(p!)^{\ell}}\cdot
N^{p\ell/2}\cdot\operatorname{\mathds{E}}_{\bar{\boldsymbol{\sigma}}}\left(\frac{1}{\sqrt{N}}\sum_{i=1}^{N}\bar{\sigma}_{i}+\sqrt{N\hat{h}^{4}}\right)^{p\ell}$
$\displaystyle\leqslant
N^{2(p-2)}\cdot\frac{\beta^{2\ell}}{N^{(p-1)\ell}}\cdot\frac{1}{\ell!(p!)^{\ell}}\cdot
N^{p\ell/2}\cdot\left(\sqrt{cp\ell}+\sqrt{\rho}\right)^{p\ell}\approx
N^{(p-2)(2-\ell/2)}\cdot(cp\ell)^{p\ell/2},$

In the above derivation,
$\operatorname{\mathds{E}}_{\bar{\boldsymbol{\sigma}}}$ is the expectation
with respect to $\bar{\boldsymbol{\sigma}}$, which is basically same as
$\operatorname{\mathds{E}}_{0}$ used in the proof of Lemma 4.1. Since $h$ is
in the regime $\alpha\geqslant\frac{1}{4}$, that is,
$\sqrt{N\hat{h}^{4}}=O(1)$, the constant $c$ in the last two steps depends on
$\rho$. The inequality follows from the classical results about moment
estimates for i.i.d sum of bounded random variables [Pin15, Lata97]. Notice
that $p\geqslant 3$, if $4<\ell<m$, where $m$ is some finite constant
independent of $N$, then the polynomial term $N^{(p-2)(1-\ell/2)}$ decays much
faster. Note that even if $m$ can be as large as $O(\log N)$, the polynomial
decay still dominates.

Next, we argue why the above results also suffice for mixed $p$-spin models.
For mixed $p$-spin models, we need to bound

$\displaystyle
N^{2(p_{m}-2)}\cdot\sum_{\begin{subarray}{c}\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p};\\\
\left|\Gamma\right|=k,\left|\partial\Gamma\right|\geqslant
0\end{subarray}}\hat{h}^{2\left|\partial\Gamma\right|}\prod_{p\geqslant
3}\operatorname{\mathds{E}}\left(\tanh^{2}(\beta\theta_{p}J/N^{(p-1)/2})\right)^{\left|\Gamma\right|_{p}},$
(46)

where $\left|\Gamma\right|_{p}$ is the number of $p$-hyperedges in $\Gamma$,
and $\left|\Gamma\right|=\sum_{p\geqslant 3}\left|\Gamma\right|_{p}$.
Continuing the above expression

$\displaystyle\beta^{2k}N^{2(p_{m}-2)}\sum_{\begin{subarray}{c}\Gamma\subseteq\cup_{p\geqslant
3}\mathscr{E}_{N,p};\\\
\left|\Gamma\right|=k,\left|\partial\Gamma\right|\geqslant
0\end{subarray}}\hat{h}^{2\left|\partial\Gamma\right|}\prod_{p\geqslant
3}\frac{\theta_{p}^{2\left|\Gamma\right|_{p}}}{N^{(p-1)\left|\Gamma\right|_{p}}}$
$\displaystyle\leqslant\beta^{2k}N^{2(p_{m}-2)+k}\cdot\operatorname{\mathds{E}}_{\boldsymbol{\sigma}}\left(\sum_{p\geqslant
3}\theta_{p}^{2}\cdot\frac{(\sum_{i=1}^{N}\sigma_{i})^{p}}{N^{p}p!}\right)^{k}$
$\displaystyle=N^{2(p_{m}-2)+k}\cdot\operatorname{\mathds{E}}_{\boldsymbol{\sigma}}\left(\beta^{2}\xi\left(\frac{1}{N}\sum_{i=1}^{N}\bar{\sigma}_{i}+\hat{h}^{2}\right)\right)^{k}.$

Notice that for mixed $p$-spin models,
$\xi\left(\frac{1}{N}\sum_{i=1}^{N}\bar{\sigma}_{i}+\hat{h}^{2}\right)\leqslant
C\cdot\left(\frac{1}{N}\sum_{i=1}^{N}\bar{\sigma}_{i}+\hat{h}^{2}\right)^{p_{m}}$
for some constant $C<\infty$, that is, mixed $p$-spin is dominated by the
leading pure $p$-spin model up to a constant factor. Thus the mixed $p$-spin
results can be deduced from the pure $p$ case established before.
$\blacksquare$

In the mixed $p$-spin case, there is a further roadblock that even for a fixed
small-size cluster, one could have infinitely many different clusters. This is
due to the flexibility over $p$ and any $p$-hyperedge can appear. In the next
proposition, we will prove that for a fixed-size cluster, the total
contribution of clusters with large $p$-hyperedge is still small.

######  Proposition 4.6.

For a mixed $p$-spin model with structure function $\xi$ and for
$\beta<\beta_{c}(\xi)$, there exists $k<\infty$ such that

$\displaystyle N^{p_{m}-2}\cdot\sum_{\Gamma:e_{\max}\geqslant
k,|\Gamma|\leqslant 4}\hat{h}^{\left|\partial\Gamma\right|}\omega(\Gamma)\to
0$ (47)

in probability as $N\to\infty$, where $e_{\max}$ is the size of the hyperedge
in $\Gamma$ with maximum number of vertices.

The above results basically say that for the small-sized clusters obtained in
the mixed $p$-spin setting, the leading contributions can not contain any
large $p$-hyperedges. This enables us to focus on analyzing the clusters of
small size and small hyperedges. This proof of this proposition basically
follows from the idea that any large $p$-hyperedges can be decomposed as
multiple small $p$-hyperegdes. In this way, the cluster size gets increased
and we know from previous results that the large size cluster contribution is
small.

###### Proof of Proposition 4.6.

Without loss of generality, we focus on the case $\left|\Gamma\right|=2$,
since $\left|\Gamma\right|$ is uniformly bounded by 4, one can always carry
out the proof to the other cases. Assuming $\Gamma$ is composed of
$p_{1}$-hyperedge and $p_{2}$-hyperedge, where $p_{2}>p_{1}$. By computing the
second moment of the l.h.s, we get the bound

$\displaystyle N^{2(p_{m}-2)}\cdot\left(\sum_{\Gamma:p_{2}\geqslant
k}\hat{h}^{2\left|\partial\Gamma\right|}\prod_{i=1}^{2}\frac{\theta_{p_{i}}^{2}\beta^{2}}{N^{p_{i}-1}}\right).$

Notice that for the larger hyperedge, one can always decompose it into smaller
hyperedges as follows. Formally, for some $t<\infty$, we have

$\displaystyle p_{2}=\sum_{j=1}^{t}a_{j}q_{j},\quad\text{where}\
a_{i}\in\mathbb{N},$ (48)

and $\\{q_{j}\\}_{j=1}^{t}$ satisfies $q_{j}\geqslant 3$ are distinct and
$\text{g.c.d}\\{q_{1},q_{2},\cdots,q_{t}\\}=\text{g.c.d}\\{p\geqslant
3:\theta_{p}>0\\}$. Then one can rewrite the edge weights for
$p_{2}$-hyperedge,

$\displaystyle\frac{\theta_{p_{2}}^{2}\beta^{2}}{N^{p_{2}-1}}=\prod_{j=1}^{t}\left(\frac{\beta^{2}\theta_{q_{j}}^{2}}{N^{q_{j}-1}}\right)^{a_{j}}\cdot\frac{1}{\beta^{2\sum_{j}a_{j}-2}}\cdot\frac{\theta_{p_{2}}^{2}}{\prod_{j=1}^{t}\theta_{q_{j}}^{2a_{j}}}\cdot\frac{1}{N^{\sum_{j}a_{j}-1}},$
(49)

Notice that under the condition $\beta<\beta_{c}(\xi)$ implies that the term
$\beta^{2-2\sum_{j}a_{j}}\prod_{j=1}^{t}\theta_{q_{j}}^{-2a_{j}}$ is of
constant order, we denote it as $c$. Then we have

$\displaystyle N^{2(p_{m}-2)}\cdot\left(\sum_{\Gamma:p_{2}\geqslant
k}\hat{h}^{2\left|\partial\Gamma\right|}\prod_{i=1}^{2}\frac{\theta_{p_{i}}^{2}\beta^{2}}{N^{p_{i}-1}}\right)$
(50) $\displaystyle\leqslant$ $\displaystyle
N^{2(p_{m}-2)}\cdot\left(\sum_{\tilde{\Gamma}:\sum_{j=1}^{t}a_{j}q_{j}\geqslant
k}\hat{h}^{2\left|\partial\Gamma\right|}\frac{\theta_{p_{1}}^{2}\beta^{2}}{N^{p_{1}-1}}\cdot\prod_{j=1}^{t}\left(\frac{\beta^{2}\theta_{q_{j}}^{2}}{N^{q_{j}-1}}\right)^{a_{j}}\cdot\frac{c}{N^{\sum_{j}a_{j}-1}}\sum_{p_{2}\geqslant
k}\frac{\theta_{p_{2}}^{2}}{p_{2}!}\right),$ (51)

where $\tilde{\Gamma}$ are the hypergraphs obtained by decomposing the
$p_{2}$-hyperedge in $\Gamma$ as a collection of $a_{j}$ number of
$q_{j}$-hyperedges. Let

$\displaystyle q_{\max}:=\max_{1\leqslant j\leqslant t}q_{j},$

then it is clear that $k\leqslant p_{2}\leqslant
q_{\max}\cdot\sum_{j=1}^{t}a_{j}$, if $k/q_{\max}-1\geqslant 2$, then
$\frac{c}{N^{\sum_{j}a_{j}-1}}\sum_{p_{2}\geqslant
k}\frac{\theta_{p_{2}}^{2}}{p_{2}!}<\infty$ combining with the mixed $p$-spin
assumption 1.1. On the other hand, if $4\leqslant
k/q_{\max}\leqslant\sum_{j}a_{j}$, then the r.h.s. of (51) will converge to 0
as $N\to\infty$ by using the large graph decaying results established before.
Since there are more than 4 hyperedges after decomposing the $p_{2}$-hyperedge
as 4 smaller hyperedges. In summary, one needs to choose $k=4q_{\max}$.
$\blacksquare$

###  4.2. Proof of Lemma 1.7

First, we show that the function $\phi(e^{t})$ is strictly increasing in
$t\in(-\infty,0]$. Note that

$\displaystyle\phi(e^{t})=\left(\log(I(e^{t})/I(1))\right)^{\prime}\text{ and
}I(e^{t})/I(1)=\operatorname{\mathds{E}}(e^{tX}),t\leqslant 0$

where $X$ is an even-integer values random variable with

$\operatorname{\mathds{P}}(X=2k)=1/(\log 2\cdot 2k(2k-1)),k\geqslant 1.$

Thus,

$\displaystyle\frac{d}{dt}\phi(e^{t})=\left(\log(I(e^{t})/I(1))\right)^{\prime\prime}>0\text{
for }t<0.$

In particular, for $p>2$ fixed, $\phi^{-1}(p)\in(0,1)$ is well defined.

Now, for $\xi_{p}(x)=x^{p}/p!$, note that at $\beta=\beta_{c}(\xi_{p})$, the
function $\frac{1}{2}\beta^{2}x^{p}/p!-I(x)$ has two global maxima in $[0,1]$,
one at $0$ and some point $u\in(0,1)$. In particular, we have

$\displaystyle\frac{1}{2}\beta^{2}u^{p}/p!=I(u)\text{ and
}\frac{1}{2}\beta^{2}u^{p-1}/(p-1)!=I^{\prime}(u),$

which implies that

$\displaystyle\phi(u)=\frac{uI^{\prime}(u)}{I(u)}=p\text{ and
}u=\phi^{-1}(p).$

Thus, we have $\beta_{c}^{2}/p!={2I(u)}/{u^{p}}$ and we are done.
$\blacksquare$

###  4.3. Proof of Theorem 1.4

Notice that

$\displaystyle\frac{1}{N}\log\bar{Z}_{N}(\beta,0)\to\frac{1}{2}\beta^{2}\xi(1)\quad\text{in
probability}\quad\text{as}\ N\to\infty.$

For the replica symmetric solution, we need to prove that
$\frac{1}{N}\log\hat{Z}_{N}(\beta,0)\to 0$ in probability. This is clear from
the above control of $\hat{Z}_{N}(\beta,0)$, where we established that the
variance converges to 0 for $\beta<\beta_{c}(\xi)$ as $N\to\infty$. This
implies that $\hat{Z}_{N}(\beta,0)\to 1$ in probability as $N\to\infty$.
$\blacksquare$

##  5\. Dominant Clusters for $p$-Spin Models

In the previous section, we reduced the size of clusters contributing to
$\hat{Z}_{N}(\beta,h)$ to 4. In this section, we will identify the leading
clusters in various regimes among those finite size cluster structures. The
competing influences of those leading clusters naturally give rise to the
multiple transition picture described in Section 1 and 2.

In the pure $p$-spin models, the underlying hypergraph is uniform and the
leading clusters can change depending on strength of the weak external field.
However, in the mixed $p$-spin case, the underlying graph is a general non-
uniform hypergraph, where all possible hyperedges can appear. The leading
clusters now can also change with respect to $p$. This is the fundamental
reason of the two parameter multiple transition summarized in Table 2. This
extra freedom on selecting any $p$-hyperedges creates further roadblocks for
identifying the leading clusters. Even for a fixed-size cluster, there still
can be many different cluster structures since one can always choose any
$p$-hyperedges in the cluster. Although the mixed $p$-spin model looks
complicated, it turns out the ideas obtained while analyzing the pure $p$-spin
case are still helpful in the mixed case. We start with pure $p$-spin model
results. The general strategy of those analyses can be reformulated as a
discrete optimization problem over the set of possible leading clusters.

We will use $c=((a_{1},p_{1}),\ldots,(a_{t},p_{t}),\ell)$ to denote a cluster
structure with $a_{i}$ many $p_{i}$ hyperedges for $i=1,2,\ldots,t$ and $\ell$
many odd-degree vertices. Let

$\displaystyle
V_{c}:=\sum_{\Gamma\subseteq\mathcal{S}_{c}}\hat{h}^{\ell}\cdot\hat{\omega}(\Gamma)=\hat{h}^{\ell}\cdot
N^{-\sum a_{i}(p_{i}-1)/2}\cdot\sum_{\Gamma}\omega(\Gamma)=\hat{h}^{\ell}\cdot
N^{-\sum a_{i}(p_{i}-1)/2}\cdot W_{c}.$

We have,

$\displaystyle\hat{Z}_{N}-1=\sum_{c\in\mathscr{C}}V_{c}$

where the sum is over all cluster structures $\mathscr{C}$. Recall that
${W}_{c}$ is the sum of $\hat{\omega}(\Gamma)$ over all hypergraphs
$\Gamma\in\cup_{p\geqslant 3}\mathscr{E}_{N,p}$ with a given cluster structure
$c$. We define

$\displaystyle\mathcal{X}(c):=-2\alpha\ell-\sum_{i}a_{i}(p_{i}-1)+\frac{1}{2}\left(\sum_{i}a_{i}p_{i}+\ell\right)$

as the real number such that
$\operatorname{Var}(V_{c})=\Theta(N^{\mathcal{X}(c)})$. Here is the general
abstract framework, we need to find

$\mathscr{C}_{\star}:=\operatorname{argmax}_{c\in\mathscr{C}}{\mathcal{X}(c)}\text{
and }\gamma_{\star}=\max_{c\in\mathscr{C}}{\mathcal{X}(c)}$

where $\mathscr{C}$ is the collection of all possible cluster structures
contributing to the variance of the partition function. In particular, we have

$\displaystyle N^{\gamma_{\star}/2}\cdot(\hat{Z}_{N}-1)\approx
N^{\gamma_{\star}/2}\cdot\sum_{c\in\mathscr{C}_{\star}}V_{c}.$

Thus the goal is to find the cluster structure(s) having the highest order of
variance in each regime. We will instantiate this framework while analyzing
different cases in the following subsections.

###  5.1. Clusters for pure $p$-spin model at $h=0$

We first deal with the case of $h=0$ for pure $p$-spin models. The key tool is
the second-moment method. In particular, we calculate the variance order of
all possible clusters and identify the dominant ones. If $h=0$, by the
decomposition of $\hat{Z}_{N}$, we know that the effective clusters should
have zero odd-degree vertices. Since the cluster size has been reduced to 4 in
Section 4, there are only a few cases left. Specifically, we find that the
leading clusters for odd and even $p$ are $((4,p),0)$ and $((3,p),0)$,
respectively. This discrepancy explains why even and odd $p$-spin models
behave differently, as suggested in the past literature. Besides that, the
extra hyperedge in odd $p$-spin case creates quite delicate structures. This
can be seen from the edge intersection mechanism in Figure 2. Later, this
subtlety will become even more apparent when we analyze the case with a weak
external field and apply Stein’s method to establish the CLT. Accordingly, the
analysis in the odd $p$ case is more involved.

Recall that the leading structures are already given in (12). Examples of the
cluster structures are presented in Figures 2 and 3. Also, recall that
$\ell_{p}$ is defined as

$\displaystyle\ell_{p}=\begin{cases}4&\text{ if $p$ is odd, }\\\ 3&\text{ if
$p$ is even.}\end{cases}$ (52)

We state this result formally in the following proposition.

######  Proposition 5.1.

For pure $p$-spin models with $p\geqslant 3$ fixed and $h=0$, we have

$\displaystyle
N^{\frac{\ell_{p}}{4}(p-2)}\left(\hat{Z}_{N,4}-1-\sum_{\Gamma\in\mathcal{S}_{(\ell_{p},p),0}}\hat{\omega}(\Gamma)\right)\to
0$ (53)

in probability as $N\to\infty$.

Note that $N^{-\ell_{p}(p-2)/2}$ is the variance order of clusters
$(\ell_{p},0)$. This proposition essentially says that the cluster
$(\ell_{p},0)$ is the leading contribution to $\hat{Z}_{N}(\beta)$. Recall
that the notation $\mathcal{S}_{(\ell_{p},0)}$ for the set of cluster
structures were defined in Section 1.5.

###### Proof of Proposition 5.1.

Notice that $\ell_{p}(p-2)/4\leqslant(p-2)$, by Proposition 4.4, we know that
the contribution of all clusters with $\left|\Gamma\right|>4$ will converge to
0 in probability under the given rescaling. All we need to consider is the
contributions from cluster sizes 3 and 4. Since given the even degree
restriction, single hyperedge ($\left|\Gamma\right|=1$) can not form a cluster
with zero odd-degree vertices ($\left|\partial\Gamma\right|=0)$. Neither do
two hyperedges without forming multi-edges. In order to prove (53), one needs
to identify the dominant clusters in $\hat{Z}_{N,4}$. Therefore it naturally
leads to the following optimization problem,

$\displaystyle\operatorname{argmax}_{a=3,4}\mathcal{X}(((a,p),0))\text{ such
that }2\text{ divides }ap.$

In general, the zero external field condition requires each cluster to have no
odd-degree vertices. Besides that, for a fixed-size cluster, it is clear that
the leading contribution would be the case where each vertex’s degree is
exactly 2, otherwise, the number of possible structures is much smaller thus
the contribution is in subleading terms.

Now let us compute the contribution of leading clusters. For even $p$, by the
structural restriction, we know that $a=3,4$. In general, for
$\left|\Gamma\right|=a$, since the total weights on the hyperedges are fixed,
one needs to count how many structures of size $a$ are. In this case, there
are $N^{ap/2}$ different ways to form the $(a,0)$ cluster; equivalently, we
need to choose $ap/2$ vertices among $N$. Each of such structures has a
variance of order $N^{-a(p-1)}$. Therefore, the total contribution is of order
$N^{-a(p-2)/2}$. It is clear that the dominant cluster is $a=3$. Therefore,
under the scaling $N^{3(p-2)/4}$, the second moment of all other structures
converges to 0 as $N\to\infty$. The odd $p$ case is much more simpler, since
the restriction $\frac{ap}{2}\in\mathbb{N}$ needs $a$ even, thus $a=4$. The
variance order, in this case, is $\Theta(N^{-2(p-2)})$. The desired results
follow similarly as in the even case. $\blacksquare$

###  5.2. Clusters for pure $p$-spin model with $h\neq 0$

Once an external field is present, the even degree restriction on clusters is
no longer needed. Although the large hypergraph decay results in Proposition
4.3 and 4.4 reduce the cluster size to 4, there are still a lot of possible
structures by allowing odd-degree vertices. It is observed that the new
clusters with odd-degree vertices must have smaller cluster sizes. This
observation seems natural but actually has a bit of subtlety. Take $(3,0)$ and
$(3,1)$ cluster as an example for any particular structure of those two kinds,
allowing odd-degree vertices clearly decrease the variance order. However, the
number of cluster structures for $(3,1)$ also increases, making it less
obvious what is the concluding effect and needs more precise quantitative
control. By explicit computation, we observed that the new dominant cluster
has to be of a smaller size to boost the variance in order to compete with the
cluster contribution at $h=0$. A similar but more quantitative issue is
whether the system prefers more or fewer odd-degree vertices if there are any.
More odd-degree vertices will diminish the contribution of each particular
structure more significantly but also increases the number of possible
structures. A similar trade-off makes the analysis quite delicate. Regarding
this, it is natural to see that the strength of the external field essentially
controls how many odd-degree vertices should appear in the new leading
cluster. This fundamentally characterizes the multiple transition picture in
Table 1.

Guided by the above observations, when $\alpha$ decreases from $\infty$
(corresponds to $h=0$ case), some new clusters beyond $(3,0),(4,0)$ in (12)
will appear, an immediate question is when those new clusters can have same
variance order as the $h=0$ clusters. Based on the above observations, we
found that the new dominant clusters are $(3,1),(2,2),(1,p)$ in (20). To find
the threshold $\alpha_{c}$, we need to compute the variance order of those new
clusters. One can see in Table 1, the clusters at $\alpha=\alpha_{c}$ for even
and odd $p$ are still different. However, if the external field gets stronger
for $\alpha\in\mathcal{R}_{2}$, specifically at $\alpha=\frac{1}{2}$,_i.e.,_
the second threshold, the dominant clusters become unified for both even and
odd $p$. Finally, in the second unified regime $\mathcal{R}_{3}$, the leading
cluster becomes even simpler and just a single hyperedge. As it indicates, the
external field gets stronger and stronger, the cluster size becomes smaller
and smaller.

To explain the details, we will use the notation $\mathcal{C}(\alpha)$ to
represent the set of leading cluster structures for a given $\alpha$, as
listed in Table 1:

$\displaystyle\mathcal{C}(\alpha):=\\{\text{cluster structures in the last
column of Table\leavevmode\nobreak\ \ref{tab:trans} for given $\alpha$}\\}.$
(54)

To compute the first threshold $\alpha_{c}$, we follow the insights obtained
in the SK ($p=2$) case [DW21], where the transition is very simple and only
two clusters appear and compete with each other. The threshold $\alpha_{c}$ is
the point where the contributions from two clusters achieve a balance. In the
$p$-spin case, since the cluster structures are more involved, we need to
first identify all possible dominant clusters and then compute their
contributions. Depending on the strength of external fields, the final step is
to find when those leading clusters’ contributions are in the same order.

We summarize the main results in the following Proposition. The proof details
will give a formal explanation for the above heuristic discussions.

######  Proposition 5.2.

Let $p\geqslant 3$ and $\alpha\geqslant\nicefrac{{1}}{{4}}$ be fixed. We have

$\displaystyle
N^{\gamma}\left(\hat{Z}_{N,4}-1-\sum_{c\in\mathcal{C}(\alpha)}\sum_{\Gamma\in\mathcal{S}_{c}}\hat{\omega}(\Gamma)\hat{h}^{b}\right)\to
0$ (55)

in probability as $N\to\infty$, where $\gamma$ depending on $\alpha$ is as
given in the third column of Table 1.

###### Proof of Proposition 5.2.

Let us start with instantiating the optimization framework first. Recall
$\gamma_{i}\leqslant(p-2)$ uniformly for $i=1,2,3$, for same reason, it
suffices to consider the clusters of size $a\leqslant 4$. As we did in the
$h=0$ case, now consider the following optimization problem

$\displaystyle\operatorname{argmax}_{\begin{subarray}{c}1\leqslant a\leqslant
4,\\\ a+b>2,\\\ 0\leqslant b\leqslant
ap\end{subarray}}\mathcal{X}\left(((a,p),b)\right)\text{ s.t.
}2\mid(ap+b),a>0,b\geqslant 0.$

We briefly explain all those constraints. By allowing an external field, the
degree of vertices can be both even and odd. Thus the cluster size $a$ can be
any integer from 1 to 4 inclusive without creating multi-edges. The number of
odd-degree vertices $b$ can be ranging from 0 to $ap$. As in the $h=0$ case,
notice that the possible leading clusters must have the following property.
Whenever the degree is even, it must be exactly 2. Whenever the degree is odd,
it must be exactly 1. Because for each fixed $(a,b)$, in this way the number
of possible structures is maximal and gives dominant contributions. Thus there
are $N^{(ap-b)/2}$ different ways to choose degree 2 vertices, and $N^{b}$
different ways to choose the degree one vertices. This explains the
restriction $2\mid ap+b$.

For $p$ even, it is clear that $b$ needs be even by the restriction
$\frac{ap+b}{2}\in\mathbb{N}^{+}$. Similarly, for $p$ odd, it needs $a,b$ to
be even or odd in the meantime. In general, for fixed $(a,b)$ in a given
regime $\alpha\in\mathcal{R}_{i}$, the variance order of the corresponding
structure is $N^{(ap+b)/2}\cdot
N^{-a(p-1)-2b\alpha}=N^{-\frac{a}{2}(p-2)-\frac{b}{2}(4\alpha-1)}$.

First, let us consider the case $\alpha=\alpha_{c}(p)$. Recall the definition
of $\alpha_{c}$ in (13), for even $p$, the variance is of order
$N^{-(p-2)(\frac{a}{2}+\frac{b}{4})}$. In this case, the restriction is
$a=1,2,3,4$ and $b$ is a non-negative integer. It is not hard to see that
clusters $(2,2),(3,0)$ have the largest variance. For odd $p$ case, the
variance is of order $N^{-(p-2)(\frac{a}{2}+\frac{b}{2})}$. Now the
restriction becomes $a=1,2,3,4$ and $b\in\mathbb{N}$ are even or odd at the
same time. The leading clusters, in this case, are $(4,0),(2,2),(3,1)$.
Furthermore, if $\alpha>\alpha_{c}$, the new leading clusters’ contribution
will be in subleading order. The dominant clusters are the same as the
$\alpha=\infty$ case. The variance order in this regime is
$\Theta(N^{-\ell_{p}(p-2)/2})$.

Now we analyze the case $\alpha=\frac{1}{2}$ similarly. For even $p$, the
variance is of order $N^{-a(p-2)/2-b/2}$. The even $b$ restriction gives the
leading clusters $(2,2),(1,p)$. For odd $p$, the restriction becomes $a,b$ are
both even or odd. The leading clusters are also $(2,2),(1,p)$. If
$\alpha\in(\frac{1}{2},\alpha_{c})$, since the variance order in general is
$N^{-\frac{a}{2}(p-2)-\frac{b}{2}(4\alpha-1)}$, it is clear that

$\displaystyle-(p-2)-(4\alpha-1)>-\frac{1}{2}(p-2)-\frac{p}{2}(4\alpha-1)$

for $\alpha\in(\frac{1}{2},\alpha_{c})$. The l.h.s and r.h.s. are the exponent
of the variance of $(2,2),(1,p)$ clusters respectively. One can also perform
similar computations for the other cases. In summary, the leading cluster for
$\alpha\in(\frac{1}{2},\alpha_{c})$ is $(2,2)$ for both even and odd $p$. The
variance order, in this case, is $N^{3-4\alpha-p}$.

The analysis in the regime $\alpha\in[\frac{1}{4},\frac{1}{2})$ is very
similar to the previous case. The leading cluster is $(1,p)$, and the variance
order is $N^{1-2\alpha p}$. The computation details are very similar to the
previous cases and are left to interested readers. $\blacksquare$

###  5.3. Clusters for mixed $p$-spin model with $h=0$

In the pure $p$-spin case, one needs to identify the leading cluster
structures on complete $p$-uniform hypergraphs, where all hyperedges are of
$p$-tuples for fixed $p\geqslant 3$. In the mixed $p$-spin case, we need to
deal with the general hypergraph where all possible $p$-hyperedges can appear.
This additional flexibility over $p$ creates a new transition with respect to
$p$ and further challenges in the analysis. As we discussed, although the
results in Proposition 4.3 and 4.4 reduce the size of possible leading
clusters to 4 or less, there still can be infinitely many different structures
even for a fixed-size cluster. For example, in $h=0$ case, consider the
clusters of size $\left|\Gamma\right|=4$, the 4 hyperedges technically can be
any $p_{1},p_{2},p_{3},p_{4}\geqslant 3$ tuples with total degree
$\sum_{i=1}^{4}p_{i}$ even. To get over this issue, it basically needs a
similar type of decaying result with respect to $p$ as dealing with the
external field in Proposition 4.3 and 4.4.

We introduce the following strategy to deal with it. Notice that for a fixed
finite size cluster, if there is any large $p$-hyperedge, one can always
decompose it into several smaller hyperedges. For example, a $9$-hyperedge can
be decomposed into a $4$-hyperedge and a $5$-hyperedge. In this way, the
cluster size would increase. By the results in Proposition 4.4, any clusters
with a size greater than 4 are in subleading order. Thus we need to prove that
any finite-size cluster with a large $p$-hyperedge must be in subleading
order. The formal statement of this result is presented in Proposition 4.6.

With the above result, it suffices to consider clusters that have finitely
many hyperedges, and those hyperedges are finite $p$-tuples. Notice that the
variance order of cluster structures has negative dependence on $p$ in the
exponent, thus it suffices to consider to minimum effective even and odd
$p$-spins $p_{e},p_{o}$ for identifying the leading clusters in the mixed
setting. The rest of the analysis will be similar to the pure $p$-spin case,
except that the cluster structures are now more complicated due to the extra
variation on $p$. The analysis has to be very careful. We start analyzing the
zero external field case in this part. Before stating the main results, we
recall some necessary notations,

$\displaystyle p_{e}=\min\\{p\geqslant 3,p\ \text{even and}\ \theta_{p}\neq
0\\},\ p_{o}=\min\\{p\geqslant 3,p\ \text{odd and}\ \theta_{p}\neq 0\\},\ \
p_{m}=\min\\{p_{e},p_{o}\\}.$

The associated cluster structure for $\alpha=\infty$ is

$\displaystyle\mathcal{C}_{\textup{mix},0}:=\\{\text{the cluster structures in
the last column of Table\leavevmode\nobreak\ \ref{tab:transm} for
$\alpha=\infty$}\\}.$

In the mixed setting, we use the 3-tuples $(a_{e},a_{o},b)$ to characterize
the cluster structure (check the notation in Section 1.5). Note that
$a_{e},a_{o},b$ represent the number of $p_{e}$-hyperedges,
$p_{o}$-hyperedges, and the number of odd-degree vertices in the associated
cluster. The scaling exponent $\gamma_{1}$ is

$\displaystyle\gamma_{1}=\begin{cases}\frac{3}{4}(p_{e}-2),&\text{ if
}p_{e}<p_{o},\\\ \frac{1}{2}p_{o}+\frac{1}{4}p_{e}-\frac{3}{2},&\text{ if
}p_{o}<p_{e}<2p_{o},\\\ (p_{o}-2),&\text{ if }p_{e}\geqslant
2p_{o}.\end{cases}$

Here is the formal statement of the results. The proof strategy is similar to
the pure $p$-spin case by instantiating the optimization framework.

######  Proposition 5.3.

Let $p_{m}\geqslant 3$ be fixed and $h=0$. We have

$\displaystyle
N^{\gamma_{1}}\cdot\left(\hat{Z}_{N,4}-1-\sum_{c\in\mathcal{C}_{\textup{mix},0}}\sum_{\Gamma\in\mathcal{S}_{c}}\hat{\omega}(\Gamma)\right)\to
0$ (56)

in probability as $N\to\infty$.

###### Proof.

Following a similar strategy as before, first, by the definition of
$\gamma_{1}$, it is easy to see that $\gamma_{1}\leqslant p_{m}-2$ for
$p_{m}\geqslant 3$. Thus by Proposition 4.4, it suffices to consider cluster
sizes of 3 and 4. By Proposition 4.6, one needs to consider the clusters with
$p$-hyperedges with finite $p$. Notice that the variance component has
negative dependence on $p$, thus it suffices to focus on the clusters with
$p_{e},p_{o}$-hyperedges. In particular, we consider the following
optimization problem,

$\displaystyle\operatorname{argmax}_{\begin{subarray}{c}a_{e}+a_{o}=3,4;\\\
a_{e},a_{o}\geqslant
0.\end{subarray}}\mathcal{X}\left(((a_{e},p_{e}),(a_{o},p_{o}),0)\right)\text{
s.t. }\frac{a_{e}p_{e}+a_{o}p_{o}}{2}\in\mathbb{N}^{+}.$

Let us first explain the constrains above. The size $3,4$ restriction is due
to simple sub-hypergraphs condition with even degree vertices. The constraints
$\frac{a_{e}p_{e}+a_{o}p_{o}}{2}\in\mathbb{N}^{+}$ is by a similar reason as
before, _i.e.,_ the possible dominant clusters enjoy the property that every
vertex has degree 2, otherwise, it’s in subleading order. From this condition,
it is easy to notice that $a_{o}$ can not be odd.

We first consider $a_{o}=0$, then $a_{o}+a_{e}=3,4$ implies $a_{e}=3,4$, a
simple calculation of the variance order indicates that $a_{e}=3,a_{o}=0$ is
one possible dominant cluster, its variance is of order $N^{3p_{e}/2}\cdot
N^{-3(p_{e}-1)}=N^{-3(p_{e}-2)/2}$. On the other hand, for $a_{o}=2$,
similarly we get $a_{e}=1,2$, and clearly $a_{e}=1$ dominants, thus
$a_{e}=1,a_{o}=2$ is another possible dominant cluster, the variance is
$N^{(p_{e}+2p_{o})/2}\cdot N^{-(p_{e}-1)-2(p_{o}-1)}=N^{-p_{o}-p_{e}/2+3}$.
The last possible dominant cluster is $a_{e}=0,a_{o}=4$, whose variance is of
order $N^{2p_{o}}\cdot N^{-4(p_{o}-1)}=N^{-2(p_{o}-2)}$. For all those 3
possibilities, we identify the dominant one among various regimes depending on
the relation of $p_{e},p_{o}$.

If $p_{e}<p_{o}$, it is clear that

$\displaystyle\frac{-3(p_{e}-2)}{2}+p_{o}+\frac{p_{e}}{2}-3=-\frac{p_{e}}{2}+p_{o}>0,$

and similarly

$\displaystyle\frac{-3(p_{e}-2)}{2}+2(p_{o}-2)=2p_{o}-\frac{3p_{e}}{2}-1\geqslant
2p_{e}+2-\frac{3p_{e}}{2}-1>0.$

Therefore, the dominant cluster in this regime is the $(3,0,0)$ cluster.
Similarly one can identify the leading clusters in the other three cases:
$p_{o}<p_{e}<2(p_{o}-1),p_{e}=2(p_{o}-1),p_{e}\geqslant 2p_{o}$. In the case
$p_{o}<p_{e}<2(p_{o}-1)$, the dominant cluster is $(1,2,0)$. While for
$p_{e}\geqslant 2p_{o}$, the dominant cluster is $(0,4,0)$. One can regard
$p_{e}=2(p_{o}-1)$ as the transition point with respect to $p$, where clusters
$(1,2,0),(0,4,0)$ coexist. $\blacksquare$

###  5.4. Clusters for mixed $p$-spin model with $h\neq 0$

The analysis in the $h=0$ case for the mixed $p$-spin model is already
delicate, we now turn to a more challenging task, which is analyzing the
transitional behavior by adding some weak external field. As indicated in
Table 2, the transitional behavior for mixed $p$-spin case becomes quite
complicated. Thanks to the Proposition 4.6, which allows us to focus on the
clusters with finite size hyperedges. Despite this, there are still many
possible combinations to form different leading clusters when an external
field is present.

However, with the important observations in the pure $p$-spin case and the
results for mixed $p$-spin at $h=0$, we are still able to derive all the
leading clusters by careful analysis. To make life easier, we introduce the
following set to denote the dominant clusters in various regimes. We will use
the notation $\mathcal{C}_{\text{mix}}(\alpha)$ to represent the set of
leading cluster structures for a given $\alpha$, as listed in Table 2:

$\displaystyle\mathcal{C}_{\text{mix}}(\alpha):=\\{\text{the cluster
structures in the last column of Table\leavevmode\nobreak\ \ref{tab:transm}
for a given $\alpha$.}\\}$ (57)

One can always check the cluster structures listed in Table 2. Now we state
the main results.

######  Proposition 5.4.

Let $p_{m}\geqslant 3$ be fixed and $\alpha\in[\nicefrac{{1}}{{4}},\infty)$.
We have

$\displaystyle
N^{\gamma}\left(\hat{Z}_{N,4}-1-\sum_{c\in\mathcal{C}_{\text{mix}}(\alpha)}\sum_{\Gamma\in\mathcal{S}_{c}}\hat{h}^{b}\hat{\omega}(\Gamma)\right)\to
0$ (58)

in probability as $N\to\infty$, where $\alpha$ depending on
$\alpha,p_{e},p_{o}$ is as given in the third column of Table 2.

###### Proof.

One can easily verify the fact that $\gamma_{i}\leqslant p_{m}-2$ for
$i=1,2,3$, thus it suffices to focus on the clusters of size $a_{e}+a_{o}$ no
greater than 4. Due to the effect of external field, now the even degree
restriction no longer holds. In particular, we have the following instance of
the optimization problem to identify dominant clusters in different regimes.

$\displaystyle\operatorname{argmax}_{\begin{subarray}{c}a_{e}+a_{o}=1,2,3,4;\\\
a_{e},a_{o}\geqslant 0,b\geqslant
0.\end{subarray}}\mathcal{X}\left(((a_{e},p_{e}),(a_{o},p_{o}),b)\right)\quad\text{s.t.}\quad\frac{a_{e}p_{e}+a_{o}p_{o}+b}{2}\in\mathbb{N}^{+}.$

Let us first briefly explain the constraints. Notice the size of clusters is
$\left|\Gamma\right|=a_{e}+a_{o}$, _i.e.,_ the sum of the number of
$p_{e}$-hyperedges and $p_{o}$-hyperedges. For the same reason as in the proof
of Proposition 4.6, we focus on the clusters with only $p_{e},p_{o}$
hyperedges. Due to the possible presence of a weak external field, the size of
the cluster can be $\\{1,2,3,4\\}$. On the other hand, the possible dominant
clusters must have the property that every even-degree vertex has degree
exactly 2, and odd-degree vertices must have degree 1. This was argued in the
previous proofs. Therefore, it needs
$\frac{a_{e}p_{e}+a_{o}p_{o}+b}{2}\in\mathbb{N}^{+}$. From this restriction,
one can easily see that $b$, the number of odd-degree vertices, must be even
or odd as $a_{o}$ in the meantime.

Note that in general the variance order of $(a_{e},a_{o},b)$ cluster is

$\displaystyle N^{\frac{a_{e}p_{e}+a_{o}p_{o}+b}{2}}\cdot
N^{-a_{e}(p_{e}-1)-a_{o}(p_{o}-1)-2b\alpha}=N^{-\frac{a_{e}(p_{e}-2)}{2}-\frac{a_{o}(p_{o}-2)}{2}-\frac{b(4\alpha-1)}{2}}.$
(59)

Let us first analyze the regime $\mathcal{R}_{1}$. In particular, for
$\alpha=\alpha_{c}(p_{e},p_{o})$, recall the definition of $\alpha_{c}$ in
(27),

$\displaystyle\alpha_{c}=\begin{cases}p_{e}/8,&\text{ if }2<p_{e}<p_{o}-1,\\\
(p_{e}-2)/4,&\text{ if }2<p_{e}=p_{o}-1,\\\ p_{e}/8,&\text{ if
}p_{o}<p_{e}<2(p_{o}-1),\\\ (p_{o}-1)/4,&\text{ if }p_{e}\geqslant
2(p_{o}-1).\end{cases}$

We drop the dependence on $p_{e},p_{o}$ for convenience if not causing any
confusion. If $p_{e}<p_{o}-1$, then $\alpha_{c}=p_{e}/8$. Plugging into (59),
the variance order for general $(a_{e},a_{o},b)$ cluster is
$N^{-(\frac{a_{e}}{2}+\frac{b}{4})(p_{e}-2)-\frac{a_{o}}{2}(p_{o}-2)}$. Recall
the observation $a_{o}$ and $b$ must be even or odd in the meantime. It’s easy
to see that for fixed $b$, by the condition $p_{e}<p_{o}-1$, we know that the
dominant cluster prefers $p_{e}$-hyperedges in order to maximize variance.
This implies that the dominant cluster should have $a_{o}=0$. This reduces the
problem to the pure $p_{e}$-spin model case, as we did, the dominant clusters
at $\alpha=\alpha_{c}$ should be $a_{e}=2,b=2$ and $a_{e}=3,b=0$.

Now if $p_{e}=p_{o}-1$, in this case $\alpha_{c}=(p_{e}-2)/4$. The variance in
(59) becomes
$N^{-\frac{a_{e}}{2}(p_{e}-2)-\frac{a_{o}}{2}(p_{o}-2)-\frac{b}{2}(p_{e}-3)}$.
Since $p_{e}<p_{o}$, clearly $a_{e}=3,a_{o}=0,b=0$ is one dominant cluster.
The variance order is $N^{-\frac{3(p_{e}-2)}{2}}$. One can also check that
$a_{e}=1,a_{o}=1,b=1$ is another cluster with variance order
$N^{-\frac{3(p_{e}-2)}{2}}$. This implies that all clusters with
$a_{e}+a_{o}>2,b>1$ are in subleading order. To find other possible dominant
clusters, it needs to decrease either $a_{e}+a_{o}$ or $b$. However, by the
structural property, we know $a_{e}=1,a_{o}=1,b=0$, and $a_{e}+a_{o}=1,b=1$
can not happen.

If $p_{e}=p_{o}+1$, then $\alpha_{c}=p_{e}/8$. The variance order is
$N^{-(\frac{a_{e}}{2}+\frac{b}{4})(p_{e}-2)-\frac{a_{o}}{2}(p_{o}-2)}$.
Similarly we know one dominant cluster without odd-degree vertices ($b=0$) is
$a_{o}=2,a_{e}=1,b=0$ with variance order $N^{-p_{o}+3-p_{e}/2}$. It is easy
to find that the variance order of the cluster $a_{e}=1,a_{o}=1,b=1$ is also
$N^{-p_{o}+3-p_{e}/2}$. Similarly as in previous case, we know that other
possible dominant clusters are $a_{e}=1,a_{o}=1,b=0$ and $a_{e}+a_{o}=1,b=1$,
which can not appear due to the hypergraph structural restrictions.

If $p_{o}+1<p_{e}<2(p_{o}-1)$, $\alpha_{c}$ is still $p_{e}/8$. This case is
very similar to the last case. The zero odd vertices dominant cluster is still
$a_{o}=2,a_{e}=1,b=0$. While the cluster $a_{e}=1,a_{o}=1,b=1$ can not appear,
since now the difference $p_{e}-p_{o}\geqslant 2$ implies $b\geqslant 2$.
Still by $p_{e}>p_{o}+1$, it’s natural to check $a_{e}=0,a_{o}=2,b=2$ cluster.
The variance is indeed dominant one $N^{-p_{o}-p_{e}/2+3}$.

If $p_{e}=2(p_{o}-1)$, then $\alpha_{c}=(p_{o}-1)/4$. In general the variance
is of order $N^{-\frac{a_{e}}{2}(p_{e}-2)-\frac{a_{o}+b}{2}(p_{o}-2)}$.
Clearly since $p_{e}>p_{o}$, one dominant cluster is $a_{o}=4,a_{e}=0,b=0$.
The variance order is $N^{-p_{o}+3-p_{e}/2}=N^{-2(p_{o}-2)}$. Note that any
clusters of size 4 with odd-degree vertices are in subleading order. To find
other possible dominant clusters, it needs to decrease the size of cluster.
For size 3 cluster with no odd-degree vertices, the only choice is
$a_{e}=1,a_{o}=2,b=0$. It can be seen that the variance is
$N^{-p_{o}+3-p_{e}/2}=N^{-2(p_{o}-2)}$. By allowing odd-degree vertices,
consider $b=1$, recall $b$ and $a_{o}$ must be even together, thus
$a_{o}=3,a_{e}=0,b=1$ is another dominant cluster with the same variance
order. Similarly one can find the last dominant cluster in this case
$a_{e}=0,a_{o}=2,b=2$.

Finally for $p_{e}\geqslant 2p_{o}$, $\alpha_{c}$ stays the same as the last
case. The only thing changing is that $p_{e}\geqslant 2p_{o}$ make the cluster
$a_{e}=1,a_{o}=2,b=0$ impossible. The rest of analysis are exactly same as the
previous case.

Eventually, when $\alpha>\alpha_{c}$, it’s clear that all the new clusters
with odd-degree vertices are in subleading order. Thus the dominant clusters
will be same as $\alpha=\infty$ case.

In a similar fashion, one can analyze the other regimes, such as
$\alpha=\frac{1}{2}$ and $\frac{1}{4}$. We will not replicate all the details
here. The final dominant clusters in each regime are as shown in Table 2.
$\blacksquare$

##  6\. CLT using Stein’s Method

After reducing $\hat{Z}_{N}$ to contribution from finite sized clusters, we
apply the multivariate Stein’s method to establish the joint central limit
theorem. The computation details for using Stein’s method are presented below.

In this section, we analyze the case of pure $p$-spin glass models. Fix
$p\geqslant 3$. The case $p=2$ has already been analyzed in [DW21]. For
simplicity, we suppress the dependence on $p,N$ in the computations below,
unless needed to avoid confusion.