# Fl RDT based ultimate lowering of the negative spherical perceptron capacity

Mihailo Stojnic 111e-mail<EMAIL_ADDRESS>
Abstract

We consider the classical _spherical_ perceptrons and study their capacities.
The famous zero-threshold case was solved in the sixties of the last century
(see, [45, 46, 8]) through the high-dimensional combinatorial considerations.
The general threshold, $\kappa$, case though turned out to be much harder and
stayed out of reach for the following several decades. A substantial progress
was then made in [22] and [29] where the _positive_ threshold ($\kappa\geq 0$)
scenario was finally fully settled. While the negative counterpart
($\kappa\leq 0$) remained out of reach, [29] did show that the random duality
theory (RDT) is still powerful enough to provide excellent upper bounds.
Moreover, in [33], a _partially lifted_ RDT variant was considered and it was
shown that the upper bounds of [29] can be lowered. After recent breakthroughs
in studying bilinearly indexed (bli) random processes in [36, 38], _fully
lifted_ random duality theory (fl RDT) was developed in [39]. We here first
show that the _negative spherical perceptrons_ can be fitted into the frame of
the fl RDT and then employ the whole fl RDT machinery to characterize the
capacity. To be fully practically operational, the fl RDT requires a
substantial numerical work. We, however, uncover remarkable closed form
analytical relations among key lifting parameters. Such a discovery enables
both shedding a new light on the parametric interconnections within the
lifting structure and performing the needed numerical calculations to obtain
concrete capacity values. After doing all of that, we also observe that an
excellent convergence (with the relative improvement $\sim 0.1\%$) is achieved
already on the third (second non-trivial) level of the _stationarized_ full
lifting.

Index Terms: Negative spherical perceptrons; Fully lifted random duality
theory.

##  1 Introduction

The last two decades have seen a remarkable progress in studying various
aspects of neural networks (NN) and machine learning (ML). Development of
powerful algorithmic techniques and corresponding performance characterizing
analytical tools together with persistent widening of the range of potential
applications are only a couple of the most important ones. We, here, follow
into similar footsteps and continue the analytical progress through a
theoretical studying of _perceptrons_ as key NN/ML building blocks.

We are particularly interested in the so-called _spherical_ perceptrons which
are easily the most popular and quite likely the simplest of all perceptron
variants. Despite the simplicity, their full analytical characterizations in
many important scenarios are not easy to obtain. For example, one of their
most relevant features, the storage or classifying _capacity_ , is, in
general, very difficult to compute. Moreover, designing practical algorithms
that can confirm the capacity achievability is often even harder. Some special
cases are a bit easier though and relevant results can be found throughout the
literature. For example, the so-called _zero-threshold_ capacity was
determined through a combinatorial, high-dimensional geometry based, approach
in seminal works [45, 46, 8] (for relevant geometric followup extensions
related to polytopal neighborliness see, e.g., [24, 25]).

While the results of [45, 46, 8] established a monumental breakthrough at the
time of their appearance, they remained an isolated example of extraordinary
success for the better part of the following several decades. For example,
even the simplest possible extension to general positive thresholds turned out
to be a formidable challenge. As it became apparent that the mathematically
rigorous treatments might be a bit further away than initially predicated, the
emergence of the statistical physics _replica tools_ in the seventies of the
last century provided a glimmer of hope that at least some (not necessarily
mathematically rigorous) analytical characterizations can be obtained. Not
long after, in the second half of the eighties, the Gardner’s seminal work,
[15] appeared and paved the way for many of the very best perceptrons’
analytical results. Namely, [15] and a follow-up [16], utilized the replica
theory and established a generic framework that can be used for the analytical
characterizations of, basically, all relevant features of interest in various
perceptrons models. Among others, these certainly included the storage
capacities in a host of different scenarios: positive/negative thresholds,
correlated/uncorrelated patterns, patterns stored incorrectly and many others.
The predictions obtained in [15, 16] were later on (in identical or similar
statistical contexts) established as mathematically fully rigorous (see, e.g.,
[22, 23, 42, 44, 43, 29, 33, 35]). In particular, [22, 23] proved the
predictions of [15] related to the storage capacity and the volume of the bond
strengths that satisfies the dynamics of the _positive_ spherical perceptrons
(i.e., the perceptrons with spherical constraints and positive thresholds
$\kappa\geq 0$). Talagrand, in [42, 44, 43], reconfirmed these predictions
through a related but somewhat different approach. On the other hand, [29]
designed a completely different, random duality theory (RDT) based, framework
and again confirmed almost all of the predictions from [15], including many
previously not considered in [22, 23, 42, 44, 43]. A substantial help in all
of these, mathematically rigorous, treatments, was provided by the underlying
_convexity_.

###  1.1 _Negative_ spherical perceptron (NSP) — no convexity help

As recognized in [29, 33], the above mentioned convexity help disappears when
the spherical perceptrons have a negative threshold (i.e., when $\kappa<0$).
The underlying deterministic strong duality is not present anymore and
obtaining accurate capacity characterizations becomes notoriously hard. Still,
the power of the RDT remains useful. In particular, relying on the fundamental
principles of the RDT, [33] proved the Talagrand’s conjecture from [42, 44,
43] that the capacity predictions of [15] are, at the very least, rigorous
upper bounds even when $\kappa<0$. [33] went a step further, utilized a
_partially lifted_ RDT variant and established that, these rigorous bounds can
in fact be lowered. This effectively confirmed that the replica symmetry
(assumed in [15, 16]) must be broken.

A series of works based on statistical physics replica approaches then
followed (see, e.g., [11, 10, 12, 13, 3]). [10] was the first one where the
NSP was connected to the recently studied jamming phenomena and hard spheres
packing problems. It established a preliminary version of the phase diagram
and emphasized the relevance of the distribution laws of “ _gaps_ ” and “
_forces_ ” and computed their critical exponents. [11] then provided a more
complete phase diagram characterization with all predicated types of
transitioning in both the so-called SAT and UNSAT phases. Moreover, it
hypothesized a potential universality in gaps and forces distribution laws
exponents. [12] studied similar features in the linear cost NSP variant.
Again, the critical exponents of the distribution laws were found to match the
ones associated with the jamming of the hard spheres. The corresponding
algorithmic confirmations were obtained in [13]. Algorithmic considerations of
a different type were discussed in [3]. Relying on the (access to the) Parisi
replica symmetry breaking (rsb) variational functional, an iterative message-
passing type of procedure is suggested as an algorithmic way of achieving the
capacity.

On the rigorous front though, the results of [33] remained untouchable until
now. Moreover, [19] showed that the upper bounds of [33] are actually (up to
the leading order terms) tight in $\kappa\rightarrow-\infty$ regime. As
mentioned above, [15] also studied many other perceptron properties. It, for
example, gave the replica symmetry prediction for the capacities of spherical
perceptrons when functioning as erroneous storage memories. [35] showed that
these predictions of [15] are again rigorous upper bounds which in certain
range of system parameters can be lowered. This proved that, in the erroneous
scenarios, the replica symmetry (assumed in [15, 16]) must again be broken.

While our primary interest here is in the simplest and possibly most famous
spherical perceptrons, various other perceptron variants are of interest.
Moreover, many of them that belong to the class of analytically “ _hard_ ”
perceptrons have been intensively studied over the last several decades as
well. We here single out probably the most well known _discrete_ $\pm 1$
perceptrons. Their _symmetric_ realizations are analytically a bit easier than
other variants and the corresponding full capacity characterizations are known
to have very particular relatively simple formulations (see, e.g., [2, 21] as
well as [1, 4, 5, 14]). On the other hand, an initial replica symmetry based
treatment of the original _nonsymmetric_ ones was already given in the
foundational papers [15, 16], where the underlying hardness is properly
recognized. After the rsb based results were obtained in [17], a strong
mathematical progress followed first in [9, 20, 6, 41, 30] and then ultimately
in [37] as well.

We here follow the path of [37] and utilize the connection between the so-
called _random feasibility problems_ (rfps) (and the spherical perceptrons as
their particular instances) on the one side and the _random duality theory_
(RDT) (see, e.g., [26, 28, 34, 29, 27]) concepts on the other side. We first
recognize the connection between the rfps and _bilinearly indexed_ (bli)
random processes and then utilize a strong recent progress in studying these
processes in [36, 38]. Namely, relying on [36, 38], in [39] a _fully lifted_
random duality theory (fl RDT) was established. Utilizing further the fl RDT
and its a particular _stationarized_ fl RDT variant (called sfl RDT), we then
obtain desired capacity characterizations. As is usually the case, to have the
fl RDT become practically operational, underlying numerical evaluations need
to be conducted. Doing so is a problem on its own and often requires a rather
strong effort. Here, however, we discover remarkable closed form relations
between key lifting parameters. These provide a direct view into a rather
beautiful structuring of the intrinsic parametric interconnections and
ultimately substantially facilitate the underlying numerical work. Moreover,
they eventually enable us to uncover that the obtained capacity
characterizations, already on the _third_ level of the _full_ lifting (3-sfl
RDT), exhibit an extraordinarily rapid convergence with a relative improvement
$\sim 0.1\%$ for all considered thresholds $\kappa$.

##  2 Connecting NSPs to rfps and free energies

As suggested above, we will rely on the fact that studying the NSP properties
is tightly connected to studying the properties of feasibility problems.
Moreover, studying feasibility problems is then tightly connected to studying
statistical physics objects called _free energies_. Both of these connections
were recognized and utilized in a long line of work [29, 35, 33, 30, 37]. To
capitalize on the existing results and to make the exposition of the main
ideas needed here as smooth as possible, we find it convenient to carefully
parallel the presentations from these papers. Along the same lines, to avoid
an unnecessary repetition of the already introduced concepts, we adopt the
practice to briefly recall on them and then focus on highlighting the main
differences, novelties, and other particularities related to the problems of
out interest here.

###  2.1 NSP $\longleftrightarrow$ rfps connection

As is well known, the _feasibility_ problems with linear inequalities have the
following mathematical form

Feasibility problem $\mathbf{\mathcal{F}}(G,{\bf
b},{\mathcal{X}},\alpha)$:find $\displaystyle{\bf x}$ subject to
$\displaystyle G{\bf x}\geq{\bf b}$ (1) $\displaystyle{\bf
x}\in{\mathcal{X}}.$

In (1), $G\in{\mathbb{R}}^{n\times n}$, ${\bf b}\in{\mathbb{R}}^{m\times 1}$,
${\mathcal{X}}\in{\mathbb{R}}^{n}$, and $\alpha=\frac{m}{n}$. [29, 35, 33, 30,
37] recognized that the above formulation is directly related to both the main
principles of the random duality theory (RDT) and the mathematical description
of perceptrons. The perceptron’s types, however, can be different and are
determined based on matrix $G$, vector ${\bf b}$, and set ${\mathcal{X}}$. For
example, for ${\mathcal{X}}=\\{-\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}}\\}^{n}$
(i.e., for ${\mathcal{X}}$ being the corners of the $n$-dimensional unit norm
hypercube), one has the so-called binary $\pm 1$ perceptrons, whereas for
${\mathcal{X}}={\mathbb{S}}^{n}$ (i.e., for ${\mathcal{X}}$ being the
$n$-dimensional unit sphere ${\mathbb{S}}^{n}$), one has the so-called
spherical perceptrons. Both of these perceptron variants allow for generic
(variable) values of the components of the threshold vector ${\bf b}$. When
${\bf b}$ is a multiple of ${\bf 1}$ (column vector of all ones of appropriate
dimension), i.e., when ${\bf b}=\kappa{\bf 1}$ (where
$\kappa\in{\mathbb{R}}$), one further obtains perceptrons with _fixed_
thresholds $\kappa$. In particular, for ${\mathcal{X}}={\mathbb{S}}^{n}$ and
$\kappa<0$ one obtains that (1) effectively emulates the so-called _negative
spherical_ perceptron (NSP). Moreover, if $G$ is generic and deterministic, we
have a deterministic perceptron. Correspondingly, if $G$ is random, we have a
statistical one. Our main objects of interest in this paper are the random
NSPs and, in particular, the Gaussian NSPs, where the components of $G$ are
iid standard normal random variables. Taking $G$ to be comprised of the iid
standard normal components makes the presentation neater. However, all the key
results that we obtain are adaptable so that they relate to other random NSP
variants where the randomness can come from basically any other distribution
that can be pushed through the Lindenberg variant of the central limit
theorem.

It is easy to see that the feasibility problem from (1) can be rewritten as
the following optimization problem

$\displaystyle\min_{{\bf x}}$ $\displaystyle f({\bf x})$ subject to
$\displaystyle G{\bf x}\geq{\bf b}$ (2) $\displaystyle{\bf
x}\in{\mathcal{X}},$

where an artificial function $f({\bf
x}):{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$ is introduced. As is also well
known, for any optimization problem to be solvable, the necessary precondition
is that it is actually feasible. Assuming the feasibility, (2) can then be
rewritten as

$\displaystyle\xi_{feas}^{(0)}(f,{\mathcal{X}})=\min_{{\bf
x}\in{\mathcal{X}}}\max_{{\bf y}\in{\mathcal{Y}}_{+}}\left(f({\bf x})-{\bf
y}^{T}G{\bf x}+{\bf y}^{T}{\bf b}\right),$ (3)

where ${\mathcal{Y}}_{+}$ is basically a set that collects all ${\bf y}$ such
that ${\bf y}_{i}\geq 0,1\leq i\leq m$. Since $f({\bf x})=0$ is clearly an
artificial object, one can also specialize back to $f({\bf x})=0$ and find

$\displaystyle\xi_{feas}^{(0)}(0,{\mathcal{X}})=\min_{{\bf
x}\in{\mathcal{X}}}\max_{{\bf y}\in{\mathcal{Y}}_{+}}\left(-{\bf y}^{T}G{\bf
x}+{\bf y}^{T}{\bf b}\right).$ (4)

The main point behind perceptron’s functioning and its connection to rfps is
contained precisely in (4). To see this, one starts by observing that
existence of an ${\bf x}$ such that $G{\bf x}\geq{\bf b}$, i.e., such that (1)
is feasible, ensures that the inner maximization in (4) can do no better than
make $\xi_{feas}^{(0)}(0,{\mathcal{X}})=0$. On the other hand, if such an
${\bf x}$ does not exist, then at least one of the inequalities in $G{\bf
x}\geq{\bf b}$ is not satisfied and the inner maximization trivially makes
$\xi_{feas}^{(0)}(0,{\mathcal{X}})=\infty$. It is also easy to see that, from
the feasibility point of view, $\xi_{feas}^{(0)}(0,{\mathcal{X}})=\infty$ and
$\xi_{feas}^{(0)}(0,{\mathcal{X}})>0$ are equivalent which implies that, for
all practical feasibility purposes, the underlying optimization problem in (4)
is structurally insensitive with respect to ${\bf y}$ scaling. One can then
restrict to $\|{\bf y}\|_{2}=1$ and basically ensure that
$\xi_{feas}^{(0)}(0,{\mathcal{X}})$ remains bounded. It is then
straightforward to see from (4), that determining

$\displaystyle\xi_{feas}(0,{\mathcal{X}})$ $\displaystyle=$
$\displaystyle\min_{{\bf x}\in{\mathcal{X}}}\max_{{\bf
y}\in{\mathcal{Y}}_{+},\|{\bf y}\|_{2}=1}\left(-{\bf y}^{T}G{\bf x}+{\bf
y}^{T}{\bf b}\right)=\min_{{\bf x}\in{\mathbb{S}}^{n}}\max_{{\bf
y}\in{\mathbb{S}}_{+}^{m}}\left(-{\bf y}^{T}G{\bf x}+\kappa{\bf y}^{T}{\bf
1}\right),$ (5)

with ${\mathbb{S}}_{+}^{m}$ being the positive orthant part of the
$m$-dimensional unit sphere, is critically important for the analytical
characterization of the rfps from (1). One then has that the sign of the
objective value in (5) (i.e., of $\xi_{feas}(f,{\mathcal{X}})$) determines the
feasibility of (1). In more concrete terms, (1) is infeasible if
$\xi_{feas}(f,{\mathcal{X}})>0$ and feasible if
$\xi_{feas}(f,{\mathcal{X}})\leq 0$.

The above reasoning holds generically, i.e., for any $G$ and ${\bf b}$. It
then automatically applies to the Gaussian NSPs as particular instances of the
above formalism obtained for Gaussian $G$ and ${\bf b}=\kappa{\bf
1},\kappa<0$. Given that the connection between the rfps from (1) and the
corresponding random _optimization_ problem counterpart from (5) is rather
evident, one clearly observes the critically important role of (5) in
characterizing various perceptrons’ features. The feature of our particular
interest here is the storage/classifying _capacity_. In a large dimensional
statistical context, it is defined as follows

$\displaystyle\alpha$ $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\frac{m}{n}$
$\displaystyle\alpha_{c}(\kappa)$ $\displaystyle\triangleq$
$\displaystyle\max\\{\alpha|\hskip
5.78172pt\lim_{n\rightarrow\infty}{\mathbb{P}}_{G}\left(\xi_{perc}(0,{\mathcal{X}})\triangleq\xi_{feas}(0,{\mathcal{X}})>0\right)\longrightarrow
1\\}$ (6) $\displaystyle=$ $\displaystyle\max\\{\alpha|\hskip
5.78172pt\lim_{n\rightarrow\infty}{\mathbb{P}}_{G}\left({\mathcal{F}}(G,{\bf
b},{\mathcal{X}},\alpha)\hskip 5.05942pt\mbox{is
feasible}\right)\longrightarrow 1\\}.$

The above is the so-called _statistical_ capacity. The corresponding
deterministic variant is defined in exactly the same way with
${\mathbb{P}}_{G}$ being removed. Throughout the paper, the subscripts next to
${\mathbb{P}}$ and ${\mathbb{E}}$ denote the randomness with respect to which
the statistical evaluation is taken. On occasion, when this is clear from the
contexts, these subscripts are left unspecified. Moreover, to shorten writing,
we regularly use the term capacity instead of _statistical_ capacity.

###  2.2 Rfps $\longleftrightarrow$ (partially reciprocal) _free energy_
connection

In the previous section, we have established that studying the random
feasibility problems (rfps) is critically important for the NSP’s capacity
analytical characterization. In this section we extend this connection to
studying _free energies_. These object are well known and almost unavoidable
in many statistical physics consideration. To introduce them in a
mathematically proper way that would be of use here, we start by defining the
following, so-called, _bilinear Hamiltonian_

${\cal H}_{sq}(G)={\bf y}^{T}G{\bf x},$ (7)

and its corresponding (so to say, _partially reciprocal_) partition function

$Z_{sq}(\beta,G)=\sum_{{\bf x}\in{\mathcal{X}}}\left(\sum_{{\bf
y}\in{\mathcal{Y}}}e^{\beta{\cal H}_{sq}(G)}\right)^{-1}.$ (8)

To ensure an overall generality of the exposition, we, in (8), take
${\mathcal{X}}$ and ${\mathcal{Y}}$ as general sets (fairly soon, we make
specializations, ${\mathcal{X}}={\mathbb{S}}^{n}$ and
${\mathcal{Y}}={\mathbb{S}}_{+}^{m}$, necessary for perceptrons’ consideration
of our interest here). One quickly notes, the reciprocal nature of the inner
summation, which makes the partition function given in (8) somewhat different
from the counterparts typically seen in statistical physics literature. The
corresponding thermodynamic limit of the average “ _partially reciprocal_ ”
free energy is then given as

$\displaystyle f_{sq}(\beta)$ $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\log{(Z_{sq}(\beta,G)})}{\beta\sqrt{n}}=\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\log\left(\sum_{{\bf
x}\in{\mathcal{X}}}\left(\sum_{{\bf y}\in{\mathcal{Y}}}e^{\beta{\cal
H}_{sq}(G)}\right)^{-1}\right)}{\beta\sqrt{n}}$ (9) $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\log\left(\sum_{{\bf
x}\in{\mathcal{X}}}\left(\sum_{{\bf y}\in{\mathcal{Y}}}e^{\beta{\bf
y}^{T}G{\bf x})}\right)^{-1}\right)}{\beta\sqrt{n}}.$

The ground state special case is obtained by considering the so-called “zero-
temperature” ($T\rightarrow 0$ or $\beta=\frac{1}{T}\rightarrow\infty$) regime

$\displaystyle
f_{sq}(\infty)\triangleq\lim_{\beta\rightarrow\infty}f_{sq}(\beta)$
$\displaystyle=$
$\displaystyle\lim_{\beta,n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\log{(Z_{sq}(\beta,G)})}{\beta\sqrt{n}}=\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\max_{{\bf
x}\in{\mathcal{X}}}-\max_{{\bf y}\in{\mathcal{Y}}}{\bf y}^{T}G{\bf
x}}{\sqrt{n}}$ (10) $\displaystyle=$
$\displaystyle-\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\min_{{\bf
x}\in{\mathcal{X}}}\max_{{\bf y}\in{\mathcal{Y}}}{\bf y}^{T}G{\bf
x}}{\sqrt{n}}.$

Restricting to $G$’s comprised of iid standard normals allows to utilize their
sign symmetry and rewrite the above as

$\displaystyle-f_{sq}(\infty)$ $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\min_{{\bf
x}\in{\mathcal{X}}}\max_{{\bf y}\in{\mathcal{Y}}}{\bf y}^{T}G{\bf
x}}{\sqrt{n}}=\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\min_{{\bf
x}\in{\mathcal{X}}}\max_{{\bf y}\in{\mathcal{Y}}}-{\bf y}^{T}G{\bf
x}}{\sqrt{n}}.$ (11)

It is not that difficult to see that (11) is directly related to (5). This, on
the other hand, also implies that $f_{sq}(\infty)$ is very tightly connected
to $\xi_{feas}(0,{\mathcal{X}})$ , which hints that understanding
$f_{sq}(\infty)$ is likely to play critically important role in understanding
and ultimately characterizing both $\xi_{feas}(0,{\mathcal{X}})$ and the NSPs
capacity. This is, in fact, exactly what happens in the sections that follow
below. Namely, since studying $f_{sq}(\infty)$ directly is not very easy, we
rely on studying $f_{sq}(\beta)$. In other words, we study the above
introduced partially reciprocal variant of the free energy for a general
$\beta$ and then specialize the obtained results to the ground state,
$\beta\rightarrow\infty$, regime. In the interest of easing the exposition,
we, however, on occasion neglect some terms which paly no significant role in
the ground state considerations.

##  3 Negative spherical perceptrons through the prism of sfl RDT

We start with one of the key observations that enables pretty much everything
that follows. It is precisely the recognition that the free energy from (9),

$\displaystyle f_{sq}(\beta)$ $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\log\left(\sum_{{\bf
x}\in{\mathcal{X}}}\left(\sum_{{\bf y}\in{\mathcal{Y}}}e^{\beta{\bf
y}^{T}G{\bf x})}\right)^{-1}\right)}{\beta\sqrt{n}},$ (12)

is a function of _bilinearly indexed_ (bli) random process ${\bf y}^{T}G{\bf
x}$. Such a recognition then puts us in position to establish a connection
between $f_{sq}(\beta)$ and the bli related results of [36, 38, 39]. To do so,
we closely follow [40, 37] and start with a collection of needed technical
definitions. For $r\in{\mathbb{N}}$, $k\in\\{1,2,\dots,r+1\\}$, real scalars
$s$, $x$, and $y$ such that $s^{2}=1$, $x>0$, and $y>0$, sets
${\mathcal{X}}\subseteq{\mathbb{R}}^{n}$ and
${\mathcal{Y}}\subseteq{\mathbb{R}}^{m}$, function
$f_{S}(\cdot):{\mathbb{R}}^{n}\rightarrow R$, vectors ${\bf p}=[{\bf
p}_{0},{\bf p}_{1},\dots,{\bf p}_{r+1}]$, ${\bf q}=[{\bf q}_{0},{\bf
q}_{1},\dots,{\bf q}_{r+1}]$, and ${\bf c}=[{\bf c}_{0},{\bf c}_{1},\dots,{\bf
c}_{r+1}]$ such that

$\displaystyle 1={\bf p}_{0}\geq{\bf p}_{1}\geq{\bf p}_{2}\geq\dots\geq{\bf
p}_{r}\geq{\bf p}_{r+1}$ $\displaystyle=$ $\displaystyle 0$ $\displaystyle
1={\bf q}_{0}\geq{\bf q}_{1}\geq{\bf q}_{2}\geq\dots\geq{\bf q}_{r}\geq{\bf
q}_{r+1}$ $\displaystyle=$ $\displaystyle 0,$ (13)

${\bf c}_{0}=1$, ${\bf c}_{r+1}=0$, and
${\mathcal{U}}_{k}\triangleq[u^{(4,k)},{\bf u}^{(2,k)},{\bf h}^{(k)}]$ such
that the components of $u^{(4,k)}\in{\mathbb{R}}$, ${\bf
u}^{(2,k)}\in{\mathbb{R}}^{m}$, and ${\bf h}^{(k)}\in{\mathbb{R}}^{n}$ are
i.i.d. standard normals, we set

$\displaystyle\psi_{S,\infty}(f_{S},{\cal X},{\cal Y},{\bf p},{\bf q},{\bf
c},x,y,s)={\mathbb{E}}_{G,{\mathcal{U}}_{r+1}}\frac{1}{n{\bf
c}_{r}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{r}}\left(\dots\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\left({\mathbb{E}}_{{\mathcal{U}}_{2}}\left(\left(Z_{S,\infty}\right)^{{\bf
c}_{2}}\right)\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right)\right)^{\frac{{\bf c}_{4}}{{\bf
c}_{3}}}\dots\right)^{\frac{{\bf c}_{r}}{{\bf c}_{r-1}}}\right),$

where

$\displaystyle Z_{S,\infty}$ $\displaystyle\triangleq$ $\displaystyle
e^{D_{0,S,\infty}}$ $\displaystyle D_{0,S,\infty}$ $\displaystyle\triangleq$
$\displaystyle\max_{{\bf x}\in{\mathcal{X}},\|{\bf x}\|_{2}=x}s\max_{{\bf
y}\in{\mathcal{Y}},\|{\bf
y}\|_{2}=y}\left(\sqrt{n}f_{S}+\sqrt{n}y\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right)^{T}{\bf x}+\sqrt{n}x{\bf y}^{T}\left(\sum_{k=2}^{r+1}b_{k}{\bf
u}^{(2,k)}\right)\right)$ $\displaystyle b_{k}$ $\displaystyle\triangleq$
$\displaystyle b_{k}({\bf p},{\bf q})=\sqrt{{\bf p}_{k-1}-{\bf p}_{k}}$
$\displaystyle c_{k}$ $\displaystyle\triangleq$ $\displaystyle c_{k}({\bf
p},{\bf q})=\sqrt{{\bf q}_{k-1}-{\bf q}_{k}}.$ (15)

Having all the above definitions set, we are in position to recall on the
following theorem – unquestionably, one of key fundamental components of sfl
RDT.

######  Theorem 1.

[39] Consider large $n$ context with
$\alpha=\lim_{n\rightarrow\infty}\frac{m}{n}$, remaining constant as $n$
grows. Let the elements of $G\in{\mathbb{R}}^{m\times n}$ be i.i.d. standard
normals and let ${\mathcal{X}}\subseteq{\mathbb{R}}^{n}$ and
${\mathcal{Y}}\subseteq{\mathbb{R}}^{m}$ be two given sets. Assume the
complete sfl RDT frame from [36] and consider a given function $f({\bf
y}):R^{m}\rightarrow R$. Set

$\displaystyle\psi_{rp}$ $\displaystyle\triangleq-\max_{{\bf
x}\in{\mathcal{X}}}s\max_{{\bf y}\in{\mathcal{Y}}}\left(f({\bf y})+{\bf
y}^{T}G{\bf
x}\right)\qquad\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{random
primal}})}$ $\displaystyle\psi_{rd}({\bf p},{\bf q},{\bf c},x,y,s)$
$\displaystyle\triangleq\frac{x^{2}y^{2}}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}{\bf
p}_{k-1}{\bf q}_{k-1}-{\bf p}_{k}{\bf q}_{k}\Bigg{.}\Bigg{)}{\bf
c}_{k}-\psi_{S,\infty}(f({\bf y}),{\cal X},{\cal Y},{\bf p},{\bf q},{\bf
c},x,y,s)\hskip
2.168pt\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{fl
random dual}})}.$

Let $\hat{{\bf p}_{0}}\rightarrow 1$, $\hat{{\bf q}_{0}}\rightarrow 1$, and
$\hat{{\bf c}_{0}}\rightarrow 1$, $\hat{{\bf p}}_{r+1}=\hat{{\bf
q}}_{r+1}=\hat{{\bf c}}_{r+1}=0$, and let the non-fixed parts of $\hat{{\bf
p}}\triangleq\hat{{\bf p}}(x,y)$, $\hat{{\bf q}}\triangleq\hat{{\bf q}}(x,y)$,
and $\hat{{\bf c}}\triangleq\hat{{\bf c}}(x,y)$ be the solutions of the
following system

$\displaystyle\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},x,y,s)}{d{\bf
p}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},x,y,s)}{d{\bf
q}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},x,y,s)}{d{\bf c}}=0.$ (17)

Then,

$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\psi_{rp}}{\sqrt{n}}$
$\displaystyle=$
$\displaystyle\min_{x>0}\max_{y>0}\lim_{n\rightarrow\infty}\psi_{rd}(\hat{{\bf
p}}(x,y),\hat{{\bf q}}(x,y),\hat{{\bf
c}}(x,y),x,y,s)\qquad\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{strong
sfl random duality}})},$

where $\psi_{S,\infty}(\cdot)$ is as in (3)-(3).

###### Proof.

The $s=-1$ scenario follows directly from the corresponding one proven in [39]
after a cosmetic change $f({\bf x})\rightarrow f({\bf y})$. On the other hand,
the $s=1$ scenario, follows after trivial adjustments and a line-by-line
repetition of the arguments of Section 3 of [39] with $s=-1$ replaced by $s=1$
and $f({\bf x})$ replaced by $f({\bf y})$. ∎

Clearly, the above theorem is very generic and holds for any given sets
${\mathcal{X}}$ and ${\mathcal{Y}}$. The corollary that follows below makes it
fully operational for the case of spherical perceptrons which are of our
interest here.

######  Corollary 1.

Assume the setup of Theorem 1 with ${\mathcal{X}}$ and ${\mathcal{Y}}$ having
the unit norm elements. Set

$\displaystyle\psi_{rp}$ $\displaystyle\triangleq-\max_{{\bf
x}\in{\mathcal{X}}}s\max_{{\bf y}\in{\mathcal{Y}}}\left({\bf y}^{T}G{\bf
x}+\kappa{\bf y}^{T}{\bf
1}\right)\qquad\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{random
primal}})}$ $\displaystyle\psi_{rd}({\bf p},{\bf q},{\bf c},x,y,s)$
$\displaystyle\triangleq\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}{\bf
p}_{k-1}{\bf q}_{k-1}-{\bf p}_{k}{\bf q}_{k}\Bigg{.}\Bigg{)}{\bf
c}_{k}-\psi_{S,\infty}(\kappa{\bf y}^{T}{\bf 1},{\cal X},{\cal Y},{\bf p},{\bf
q},{\bf
c},1,1,s)\quad\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{fl
random dual}})}.$

Let the non-fixed parts of $\hat{{\bf p}}$, $\hat{{\bf q}}$, and $\hat{{\bf
c}}$ be the solutions of the following system

$\displaystyle\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,s)}{d{\bf
p}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,s)}{d{\bf
q}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,s)}{d{\bf c}}=0.$ (20)

Then,

$\displaystyle\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\psi_{rp}}{\sqrt{n}}$
$\displaystyle=$ $\displaystyle\lim_{n\rightarrow\infty}\psi_{rd}(\hat{{\bf
p}},\hat{{\bf q}},\hat{{\bf
c}},1,1,s)\qquad\mbox{({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{strong
sfl random duality}})},$

where $\psi_{S,\infty}(\cdot)$ is as in (3)-(3).

###### Proof.

Follows trivially as a direct consequence of Theorem 1, after choosing $f({\bf
y})=\kappa{\bf y}^{T}{\bf 1}$ and recognizing that all elements in
${\mathcal{X}}$ and ${\mathcal{Y}}$ are of unit norm. ∎

As [39, 40] noted, the above random primal problems’ trivial concentrations
enable various corresponding probabilistic variants of (1) and (1) as well.
We, however, skip stating such trivialities.

##  4 Practical realization

To have the results of Theorem 1 and Corollary 1 become practically useful,
one needs to ensure that all the underlying quantities can be valuated. Two
key obstacles might pose a problem in that regard: (i) It is a priori not
clear what should be the correct value for $r$; and (ii) Sets ${\mathcal{X}}$
and ${\mathcal{Y}}$ do not have a component-wise structure characterization
which does not provide any guarantee that the decoupling over both ${\bf x}$
and ${\bf y}$ is very straightforward. It turns out, however, that neither of
these potential obstacles is unsurpassable.

After specialization to ${\mathcal{X}}={\mathbb{S}}^{n}$ and
${\mathcal{Y}}={\mathbb{S}}_{+}^{m}$, we rely on results of Corollary 1 and
start by observing that the key object of practical interest is the following
_random dual_

$\displaystyle\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,s)$
$\displaystyle\triangleq\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}{\bf
p}_{k-1}{\bf q}_{k-1}-{\bf p}_{k}{\bf q}_{k}\Bigg{.}\Bigg{)}{\bf
c}_{k}-\psi_{S,\infty}(0,{\cal X},{\cal Y},{\bf p},{\bf q},{\bf c},1,1,s).$
$\displaystyle=\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}{\bf p}_{k-1}{\bf
q}_{k-1}-{\bf p}_{k}{\bf q}_{k}\Bigg{.}\Bigg{)}{\bf
c}_{k}-\frac{1}{n}\varphi(D^{(per)}(s))-\frac{1}{n}\varphi(D^{(sph)}(s)),$

where analogously to (3)-(3)

$\displaystyle\varphi(D,{\bf c})$ $\displaystyle=$
$\displaystyle{\mathbb{E}}_{G,{\mathcal{U}}_{r+1}}\frac{1}{{\bf
c}_{r}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{r}}\left(\dots\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\left({\mathbb{E}}_{{\mathcal{U}}_{2}}\left(\left(e^{D}\right)^{{\bf
c}_{2}}\right)\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right)\right)^{\frac{{\bf c}_{4}}{{\bf
c}_{3}}}\dots\right)^{\frac{{\bf c}_{r}}{{\bf c}_{r-1}}}\right),$

and

$\displaystyle D^{(per)}(s)$ $\displaystyle=$ $\displaystyle\max_{{\bf
x}\in{\mathcal{X}}}\left(s\sqrt{n}\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right)^{T}{\bf x}\right)$ $\displaystyle D^{(sph)}(s)$
$\displaystyle\triangleq$ $\displaystyle s\max_{{\bf
y}\in{\mathcal{Y}}}\left(\sqrt{n}\kappa{\bf y}^{T}{\bf 1}+\sqrt{n}{\bf
y}^{T}\left(\sum_{k=2}^{r+1}b_{k}{\bf u}^{(2,k)}\right)\right).$ (24)

After a simple evaluation, we find

$\displaystyle D^{(per)}(s)$ $\displaystyle=$ $\displaystyle\max_{{\bf
x}\in{\mathcal{X}}}\left(s\sqrt{n}\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right)^{T}{\bf x}\right)=\sqrt{n}\max_{{\bf
x}\in{\mathbb{S}}^{n}}\left(s\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right)^{T}{\bf x}\right)=\sqrt{n}\left\|\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right\|_{2}.$

We now utilize the _square root trick_ introduced on numerous occasions in
[32, 31, 35, 33]

$\displaystyle D^{(per)}(s)$ $\displaystyle=$
$\displaystyle\sqrt{n}\left\|\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right\|_{2}=\sqrt{n}\min_{\gamma^{(p)}}\left(\frac{\left\|\sum_{k=2}^{r+1}c_{k}{\bf
h}^{(k)}\right\|_{2}^{2}}{4\gamma^{(p)}}+\gamma^{(p)}\right)$ (26)
$\displaystyle=$
$\displaystyle\sqrt{n}\min_{\gamma^{(p)}}\left(\frac{\sum_{i=1}^{n}\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}_{i}^{(k)}\right)^{2}}{4\gamma^{(p)}}+\gamma^{(p)}\right).$

After introducing scaling $\gamma^{(p)}=\gamma^{(p)}_{sq}\sqrt{n}$, we rewrite
(26) as

$\displaystyle D^{(per)}(s)$ $\displaystyle=$
$\displaystyle\sqrt{n}\min_{\gamma_{sq}^{(p)}}\left(\frac{\sum_{i=1}^{n}\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}_{i}^{(k)}\right)^{2}}{4\gamma_{sq}^{(p)}\sqrt{n}}+\gamma_{sq}^{(p)}\sqrt{n}\right)=\min_{\gamma_{sq}^{(p)}}\left(\frac{\sum_{i=1}^{n}\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}_{i}^{(k)}\right)^{2}}{4\gamma_{sq}^{(p)}}+\gamma_{sq}^{(p)}n\right)$
$\displaystyle=$
$\displaystyle\min_{\gamma_{sq}^{(p)}}\left(\sum_{i=1}^{n}D^{(per)}_{i}(c_{k})+\gamma_{sq}^{(p)}n\right)$
. (27)

where

$\displaystyle D^{(per)}_{i}(c_{k})=\frac{\left(\sum_{k=2}^{r+1}c_{k}{\bf
h}_{i}^{(k)}\right)^{2}}{4\gamma_{sq}^{(p)}}.$ (28)

In a similar fashion (and following [37]), we also have

$\displaystyle D^{(sph)}(s)$ $\displaystyle\triangleq$ $\displaystyle
s\sqrt{n}\max_{{\bf y}\in{\mathcal{Y}}}\left(\kappa{\bf y}^{T}{\bf 1}+{\bf
y}^{T}\left(\sum_{k=2}^{r+1}b_{k}{\bf
u}^{(2,k)}\right)\right)=s\sqrt{n}\left\|\max\left(\kappa{\bf
1}+\sum_{k=2}^{r+1}b_{k}{\bf u}^{(2,k)},0\right)\right\|_{2}.$ (29)

Utilizing again the _square root trick_ , we obtain

$\displaystyle D^{(sph)}(s)$
$\displaystyle=\sqrt{n}s\left\|\max\left(\kappa{\bf
1}+\sqrt{n}\sum_{k=2}^{r+1}b_{k}{\bf
u}^{(2,k)},0\right)\right\|_{2}=s\sqrt{n}\min_{\gamma}\left(\frac{\left\|\max\left(\kappa{\bf
1}+\sum_{k=2}^{r+1}b_{k}{\bf
u}^{(2,k)},0\right)\right\|_{2}^{2}}{4\gamma}+\gamma\right)$
$\displaystyle=s\sqrt{n}\min_{\gamma}\left(\frac{\sum_{i=1}^{m}\max\left(\kappa+\sum_{k=2}^{r+1}b_{k}{\bf
u}_{i}^{(2,k)},0\right)^{2}}{4\gamma}+\gamma\right).$ (30)

After introducing scaling $\gamma=\gamma_{sq}\sqrt{n}$, (4) can be rewritten
as

$\displaystyle D^{(sph)}(s)$ $\displaystyle=$ $\displaystyle
s\sqrt{n}\min_{\gamma_{sq}}\left(\frac{\sum_{i=1}^{m}\max\left(\kappa+\sum_{k=2}^{r+1}b_{k}{\bf
u}_{i}^{(2,k)},0\right)^{2}}{4\gamma_{sq}\sqrt{n}}+\gamma_{sq}\sqrt{n}\right)$
$\displaystyle=$ $\displaystyle
s\min_{\gamma_{sq}}\left(\frac{\sum_{i=1}^{m}\max\left(\kappa+\sum_{k=2}^{r+1}b_{k}{\bf
u}_{i}^{(2,k)},0\right)^{2}}{4\gamma_{sq}}+\gamma_{sq}n\right)$
$\displaystyle=$ $\displaystyle
s\min_{\gamma_{sq}}\left(\sum_{i=1}^{m}D_{i}^{(sph)}(b_{k})+\gamma_{sq}n\right),$

with

$\displaystyle
D_{i}^{(sph)}(b_{k})=\frac{\max\left(\kappa+\sum_{k=2}^{r+1}b_{k}{\bf
u}_{i}^{(2,k)},0\right)^{2}}{4\gamma_{sq}}.$ (32)

###  4.1 $s=-1$ particularization

Taking $s=-1$ gives us the opportunity to establish a direct connection
between the ground state energy, $f_{sq}(\infty)$ given in (11), and the
random primal of the above machinery, $\psi_{rp}(\cdot)$, given in Corollary
1. In concrete terms, this basically means the following

$\displaystyle-f_{sq}(\infty)$ $\displaystyle=$
$\displaystyle-\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\max_{{\bf
x}\in{\mathcal{X}}}-\max_{{\bf y}\in{\mathcal{Y}}}{\bf y}^{T}G{\bf
x}}{\sqrt{n}}=\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\psi_{rp}}{\sqrt{n}}=\lim_{n\rightarrow\infty}\psi_{rd}(\hat{{\bf
p}},\hat{{\bf q}},\hat{{\bf c}},1,1,-1),$ (33)

where the non-fixed parts of $\hat{{\bf p}}$, $\hat{{\bf q}}$, and $\hat{{\bf
c}}$ are the solutions of the following system

$\displaystyle\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,-1)}{d{\bf
p}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,-1)}{d{\bf
q}}=0,\quad\frac{d\psi_{rd}({\bf p},{\bf q},{\bf c},1,1,-1)}{d{\bf c}}=0.$
(34)

Relying on (4)-(32), we further have

$\displaystyle\lim_{n\rightarrow\infty}\psi_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},1,1,-1)=\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},\hat{\gamma}_{sq},\hat{\gamma}_{sq}^{(p)},1,1,-1),$ (35)

with

$\displaystyle\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}{\bf p}_{k-1}{\bf
q}_{k-1}-{\bf p}_{k}{\bf q}_{k}\Bigg{.}\Bigg{)}{\bf c}_{k}$
$\displaystyle-\gamma_{sq}^{(p)}-\varphi(D_{1}^{(per)}(c_{k}({\bf p},{\bf
q})),{\bf c})+\gamma_{sq}-\alpha\varphi(-D_{1}^{(sph)}(b_{k}({\bf p},{\bf
q})),{\bf c}).$

Connecting (33), (35), and (4.1), we further find

$\displaystyle-f_{sq}(\infty)$ $\displaystyle=$
$\displaystyle-\lim_{n\rightarrow\infty}\frac{{\mathbb{E}}_{G}\max_{{\bf
x}\in{\mathcal{X}}}-\max_{{\bf y}\in{\mathcal{Y}}}{\bf y}^{T}G{\bf
x}}{\sqrt{n}}$ $\displaystyle=$
$\displaystyle\lim_{n\rightarrow\infty}\psi_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},1,1,-1)=\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},\hat{\gamma}_{sq},\hat{\gamma}_{sq}^{(p)},1,1,-1)$
$\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}\hat{{\bf
p}}_{k-1}\hat{{\bf q}}_{k-1}-\hat{{\bf p}}_{k}\hat{{\bf
q}}_{k}\Bigg{.}\Bigg{)}\hat{{\bf c}}_{k}$
$\displaystyle-\hat{\gamma}_{sq}^{(p)}-\varphi(D_{1}^{(per)}(c_{k}(\hat{{\bf
p}},\hat{{\bf q}})),{\bf
c})+\hat{\gamma}_{sq}-\alpha\varphi(-D_{1}^{(sph)}(b_{k}(\hat{{\bf
p}},\hat{{\bf q}})),{\bf c}).$

The following theorem summarizes the above mechanism.

######  Theorem 2.

Assume the complete sfl RDT setup of [36]. Consider large $n$ linear regime
with $\alpha=\lim_{n\rightarrow\infty}\frac{m}{n}$ and $\varphi(\cdot)$ and
$\bar{\psi}(\cdot)$ from (4) and (4.1). Let the “fixed” parts of $\hat{{\bf
p}}$, $\hat{{\bf q}}$, and $\hat{{\bf c}}$ satisfy $\hat{{\bf
p}}_{1}\rightarrow 1$, $\hat{{\bf q}}_{1}\rightarrow 1$, $\hat{{\bf
c}}_{1}\rightarrow 1$, $\hat{{\bf p}}_{r+1}=\hat{{\bf q}}_{r+1}=\hat{{\bf
c}}_{r+1}=0$, and let the “non-fixed” parts of $\hat{{\bf p}}_{k}$, $\hat{{\bf
q}}_{k}$, and $\hat{{\bf c}}_{k}$ ($k\in\\{2,3,\dots,r\\}$) be the solutions
of the following system of equations

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}=0,$ (38)

and, consequently, let

$\displaystyle c_{k}(\hat{{\bf p}},\hat{{\bf q}})$ $\displaystyle=$
$\displaystyle\sqrt{\hat{{\bf q}}_{k-1}-\hat{{\bf q}}_{k}}$ $\displaystyle
b_{k}(\hat{{\bf p}},\hat{{\bf q}})$ $\displaystyle=$
$\displaystyle\sqrt{\hat{{\bf p}}_{k-1}-\hat{{\bf p}}_{k}}.$ (39)

Then

$-f_{sq}(\infty)=\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}\hat{{\bf
p}}_{k-1}\hat{{\bf q}}_{k-1}-\hat{{\bf p}}_{k}\hat{{\bf
q}}_{k}\Bigg{.}\Bigg{)}\hat{{\bf
c}}_{k}-\hat{\gamma}_{sq}^{(p)}-\varphi(D_{1}^{(per)}(c_{k}(\hat{{\bf
p}},\hat{{\bf q}})),\hat{{\bf
c}})+\hat{\gamma}_{sq}-\alpha\varphi(-D_{1}^{(sph)}(b_{k}(\hat{{\bf
p}},\hat{{\bf q}})),\hat{{\bf c}}).$ (40)

###### Proof.

Follows from the previous discussion, Theorem 1, Corollary 1, and the sfl RDT
machinery presented in [38, 36, 39, 40]. ∎

###  4.2 Numerical evaluations

As stated earlier, the results of Theorem 2 become operational if one can
conduct the underlying numerical evaluations. All technical ingredients for
such evaluations are present in the theorem itself. We start the evaluations
with $r=1$ and proceed by incrementally increasing $r$. Proceeding in such a
way enables one to systematically follow progressing of the entire lifting
machinery. Moreover, it allows us to connect to some to the known results and
show how they can be deduced as special cases of the generic mechanism
presented here. To enable concrete numerical values, the evaluations are, on
occasion, specialized to particular values of $\kappa$. Also, several
analytical closed form results can be obtained along the way that make the
overall evaluation process somewhat easier. We state those below as well.

####  4.2.1 $r=1$ – first level of lifting

For the first level, we have $r=1$ and $\hat{{\bf p}}_{1}\rightarrow 1$ and
$\hat{{\bf q}}_{1}\rightarrow 1$ which, together with $\hat{{\bf
p}}_{r+1}=\hat{{\bf p}}_{2}=\hat{{\bf q}}_{r+1}=\hat{{\bf q}}_{2}=0$, and
$\hat{{\bf c}}_{2}\rightarrow 0$, gives

$\displaystyle\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf q}},\hat{{\bf
c}},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=\frac{1}{2}{\bf
c}_{2}-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{2}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{{\bf
c}_{2}\frac{\left(\sqrt{1-0}{\bf
h}_{1}^{(2)}\right)^{2}}{4\gamma_{sq}^{(p)}}}\right)$
$\displaystyle+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}{4\gamma_{sq}}}\right)$
$\displaystyle\rightarrow-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{2}}\log\left(1+{\mathbb{E}}_{{\mathcal{U}}_{2}}{\bf
c}_{2}\frac{\left(\sqrt{1-0}{\bf
h}_{1}^{(2)}\right)^{2}}{4\gamma_{sq}^{(p)}}\right)$
$\displaystyle\qquad+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}\log\left(1-{\mathbb{E}}_{{\mathcal{U}}_{2}}{\bf
c}_{2}\frac{\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}{4\gamma_{sq}}\right)$
$\displaystyle\rightarrow-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{2}}\log\left(1+{\bf c}_{2}\frac{1}{4\gamma_{sq}^{(p)}}\right)$
$\displaystyle\qquad+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}\log\left(1-\frac{{\bf
c}_{2}}{4\gamma_{sq}}{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}\right)$
$\displaystyle\rightarrow-\gamma_{sq}^{(p)}-\frac{1}{4\gamma_{sq}^{(p)}}+\gamma_{sq}+\frac{\alpha}{4\gamma_{sq}}{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}.$ (41)

One then easily finds $\gamma_{sq}^{(p)}=\frac{1}{2}$ and
$\hat{\gamma}_{sq}=\frac{\sqrt{\alpha}}{2}\sqrt{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}$ and

$\displaystyle-f_{sq}^{(1)}(\infty)=\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},\hat{\gamma}_{sq},\hat{\gamma}_{sq}^{(p)},1,1,-1)$
$\displaystyle=-1+\sqrt{\alpha}\sqrt{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+{\bf
u}_{1}^{(2,2)},0)^{2}}.$ (42)

To obtain the critical $\alpha_{c}^{(1)}$ as a function of $\kappa$, we rely
on condition $f_{sq}^{(1)}(\infty)=0$, which gives

$a_{c}^{(1)}(\kappa)=\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+{\bf
u}_{1}^{(2,2)},0)^{2}}=\frac{1}{\left(\frac{\kappa
e^{-\frac{\kappa^{2}}{2}}}{\sqrt{2\pi}}+\frac{(\kappa^{2}+1)\mbox{erfc}\left(-\frac{\kappa}{\sqrt{2}}\right)}{2}\right)}.$
(43)

To get concrete values we specialize to $\kappa=-1.5$ and find

$(\mbox{{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{first
level:}}})\qquad\qquad
a_{c}^{(1)}(-1.5)=\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(-1.5+{\bf
u}_{1}^{(2,2)},0)^{2}}\rightarrow{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{43.77}}.$
(44)

####  4.2.2 $r=2$ – second level of lifting

We split the second level of lifting into two separate parts: (i) _partial_
second level of lifting; and (ii) _full_ second level of lifting.

#####  4.2.2.1 Partial second level of lifting

When $r=2$ and the partial lifting is considered, then (similarly to the first
level) $\hat{{\bf p}}_{1}\rightarrow 1$ and $\hat{{\bf q}}_{1}\rightarrow 1$,
$\hat{{\bf p}}_{2}=\hat{{\bf q}}_{2}=0$, and $\hat{{\bf p}}_{r+1}=\hat{{\bf
p}}_{3}=\hat{{\bf q}}_{r+1}=\hat{{\bf q}}_{3}=0$ but in general $\hat{{\bf
c}}_{2}\neq 0$. As above, one again has

$\displaystyle\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf q}},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=\frac{1}{2}{\bf
c}_{2}-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{2}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{{\bf
c}_{2}\frac{\left(\sqrt{1-0}{\bf
h}_{1}^{(2)}\right)^{2}}{4\gamma_{sq}^{(p)}}}\right)$
$\displaystyle\quad+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}{4\gamma_{sq}}}\right)$ $\displaystyle=\frac{1}{2}{\bf
c}_{2}-\gamma_{sq}^{(p)}+\frac{1}{2{\bf
c}_{2}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf
c}_{2}}{2\gamma_{sq}^{(p)}}\right)$
$\displaystyle\quad+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}{4\gamma_{sq}}}\right).$

Solving the integrals gives

$\displaystyle\bar{h}$ $\displaystyle=$ $\displaystyle-\kappa$
$\displaystyle\bar{B}$ $\displaystyle=$ $\displaystyle\frac{{\bf
c}_{2}}{4\gamma_{sq}}$ $\displaystyle\bar{C}$ $\displaystyle=$
$\displaystyle\kappa$ $\displaystyle f_{(zd)}$ $\displaystyle=$
$\displaystyle\frac{e^{-\frac{\bar{B}\bar{C}^{2}}{2\bar{B}+1}}}{2\sqrt{2\bar{B}+1}}\mbox{erfc}\left(\frac{\bar{h}}{\sqrt{4\bar{B}+2}}\right)$
$\displaystyle f_{(zu)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\mbox{erfc}\left(-\frac{\bar{h}}{\sqrt{2}}\right),$
(46)

and

${\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}{4\gamma_{sq}}}=f_{(zd)}+f_{(zu)}.$ (47)

Differentiation (optimization) with respect to $\gamma_{sq}^{(p)}$,
$\gamma_{sq}$, and ${\bf c}_{2}$ brings two different scenarios for concrete
optimal parameter values that are distinguished based on the value of
$\kappa$.

(_i_) For $\kappa\geq\kappa_{c}\approx-0.622$, we find $\hat{{\bf
c}}_{2}\rightarrow 0$, $\hat{\gamma}_{sq}^{(p)}=\frac{1}{2}$, and
$\hat{\gamma}_{sq}=\frac{\sqrt{\alpha}}{2}\sqrt{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+\sqrt{1-0}{\bf
u}_{1}^{(2,2)},0)^{2}}$. In other words, when
$\kappa\geq\kappa_{c}\approx-0.622$, one uncovers the first level of lifting
with $a_{c}^{(2,p)}$ as in (44), i.e., with

$a_{c}^{(2,p)}=a_{c}^{(1)}=\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{2}}\max(\kappa+{\bf
u}_{1}^{(2,2)},0)^{2}}=\frac{1}{\left(\frac{\kappa
e^{-\frac{\kappa^{2}}{2}}}{\sqrt{2\pi}}+\frac{(\kappa^{2}+1)\mbox{erfc}\left(-\frac{\kappa}{\sqrt{2}}\right)}{2}\right)}.$
(48)

(_ii_) For $\kappa\leq\kappa_{c}$, after computing the derivatives with
respect to $\gamma_{sq}^{(p)}$, $\gamma_{sq}$, and ${\bf c}_{2}$ and equalling
them to zero, we obtain for, say, $\kappa=-1.5$

$(\mbox{{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{\emph{partial}
second level:}}})\qquad\qquad
a_{c}^{(2,p)}(-1.5)\approx{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{37.36}}.$
(49)

#####  4.2.2.2 Full second level of lifting

The setup presented above can also be utilized for the full lifting on the
second level. However, one has to be careful since now (in addition to
$\hat{{\bf c}}_{2}\neq 0$) one, in general, also has ${\bf p}_{2}\neq 0$ and
${\bf q}_{2}\neq 0$. Analogously to (4.2.2.1), we now write

$\displaystyle\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=$
$\displaystyle\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2}){\bf
c}_{2}-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{{\bf
c}_{2}\frac{\left(\sqrt{1-{\bf q}_{2}}{\bf h}_{1}^{(2)}+\sqrt{{\bf q}_{2}}{\bf
h}_{1}^{(3)}\right)^{2}}{4\gamma_{sq}^{(p)}}}\right)$ (50)
$\displaystyle+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}}{\bf u}_{1}^{(2,3)},0)^{2}}{4\gamma_{sq}}}\right)$ $\displaystyle=$
$\displaystyle\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2}){\bf
c}_{2}-\gamma_{sq}^{(p)}-\Bigg{(}\Bigg{.}-\frac{1}{2{\bf
c}_{2}}\log\left(\frac{2\gamma_{sq}-{\bf c}_{2}(1-{\bf
q}_{2})}{2\gamma_{sq}}\right)$ $\displaystyle+\frac{{\bf
q}_{2}}{2(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2}))}\Bigg{.}\Bigg{)}$
$\displaystyle+\gamma_{sq}-\alpha\frac{1}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}}{\bf u}_{1}^{(2,3)},0)^{2}}{4\gamma_{sq}}}\right).$

After solving the remaining integrals, we also have

$\displaystyle\hat{h}$ $\displaystyle=$ $\displaystyle-\frac{\sqrt{{\bf
p}_{2}}{\bf u}_{1}^{(2,3)}+\kappa}{\sqrt{1-{\bf p}_{2}}}$
$\displaystyle\hat{B}$ $\displaystyle=$ $\displaystyle\frac{{\bf
c}_{2}}{4\gamma_{sq}}$ $\displaystyle\hat{C}$ $\displaystyle=$
$\displaystyle\sqrt{{\bf p}_{2}}{\bf u}_{1}^{(2,3)}+\kappa$ $\displaystyle
f_{(zd)}^{(2,f)}$ $\displaystyle=$
$\displaystyle\frac{e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}}\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)$ $\displaystyle f_{(zu)}^{(2,f)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\mbox{erfc}\left(-\frac{\hat{h}}{\sqrt{2}}\right),$
$\displaystyle f_{(zt)}^{(2,f)}$ $\displaystyle=$ $\displaystyle
f_{(zd)}^{(2,f)}+f_{(zu)}^{(2,f)}.$ (51)

and

$\displaystyle{\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}}{\bf
u}_{1}^{(2,3)},0)^{2}}{4\gamma_{sq}}}\right)={\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left(f_{(zt)}^{(2,f)}\right).$
(52)

One now needs to compute _five_ derivatives with respect to ${\bf q}_{2}$,
${\bf p}_{2}$, ${\bf c}_{2}$, $\gamma_{sq}$,, and $\gamma_{sq}^{(p)}$. We
systematically compute each of them.

(i) ${\bf q}_{2}$ – derivative: We start by writing

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}{\bf
c}_{2}-\Bigg{(}\Bigg{.}-\frac{1}{2((2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})))}+\frac{1}{2((2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})))}$ (53)
$\displaystyle-\frac{{\bf q}_{2}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))^{2}}{\bf c}_{2}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}{\bf c}_{2}+\frac{{\bf
q}_{2}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))^{2}}{\bf c}_{2}$
$\displaystyle=$ $\displaystyle{\bf c}_{2}\left(-\frac{1}{2}{\bf
p}_{2}+\frac{{\bf q}_{2}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))^{2}}\right).$

(ii) ${\bf p}_{2}$ – derivative: As above, we start by writing

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{2}{\bf c}_{2}-\frac{\alpha}{{\bf
c}_{2}}\frac{d\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left(f_{(zt)}^{(2,f)}\right)\right)}{d{\bf
p}_{2}}$ (54) $\displaystyle=$ $\displaystyle-\frac{1}{2}{\bf q}_{2}{\bf
c}_{2}-\frac{\alpha}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{1}{f_{(zt)}^{(2,f)}}\frac{d\left(f_{(zt)}^{(2,f)}\right)}{d{\bf
p}_{2}}\right).$

From (4.2.2.2), we then have

$\displaystyle\frac{df_{(zt)}^{(2,f)}}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d{\bf
p}_{2}}+\frac{df_{(zu)}^{(2,f)}}{d{\bf p}_{2}}.$ (55)

Moreover, we also have

$\displaystyle\frac{df_{(zu)}^{(2,f)}}{d{\bf
p}_{2}}=\frac{e^{-\frac{\hat{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\hat{h}}{d{\bf
p}_{2}},$ (56)

and

$\displaystyle\frac{d\hat{h}}{d{\bf p}_{2}}=-\frac{{\bf
u}_{1}^{(2,3)}}{2\sqrt{{\bf p}_{2}}\sqrt{1-{\bf p}_{2}}}-\frac{(\sqrt{{\bf
p}_{2}}{\bf u}_{1}^{(2,3)}+\kappa)}{2\sqrt{1-{\bf p}_{2}}^{3}}.$ (57)

A combination of (56) and (57) gives

$\displaystyle\frac{df_{(zu)}^{(2,f)}}{d{\bf
p}_{2}}=\frac{e^{-\frac{\hat{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\hat{h}}{d{\bf
p}_{2}}=\frac{e^{-\frac{\hat{h}^{2}}{2}}}{\sqrt{2\pi}}\left(-\frac{{\bf
u}_{1}^{(2,3)}}{2\sqrt{{\bf p}_{2}}\sqrt{1-{\bf p}_{2}}}-\frac{(\sqrt{{\bf
p}_{2}}{\bf u}_{1}^{(2,3)}+\kappa)}{2\sqrt{1-{\bf p}_{2}}^{3}}\right).$ (58)

After observing

$\displaystyle\frac{d\hat{C}}{d{\bf p}_{2}}=\frac{{\bf
u}_{1}^{(2,3)}}{2\sqrt{{\bf p}_{2}}},$ (59)

we can further write

$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d{\bf p}_{2}}=f_{(d{\bf
p})}^{(1)}+f_{(d{\bf p})}^{(2)}+f_{(d{\bf p})}^{(3)},$ (60)

where

$\displaystyle f_{(d{\bf p})}^{(1)}=\left(-\frac{\hat{B}\hat{C}{\bf
u}_{1}^{(2,3)}}{\sqrt{{\bf p}_{2}}(2(1-{\bf
p}_{2})\hat{B}+1)}-\frac{2\hat{B}^{2}\hat{C}^{2}}{(2(1-{\bf
p}_{2})\hat{B}+1).^{2}}\right)e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}\frac{\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\hat{B}+1}},$ (61)

and

$f_{(d{\bf p})}^{(2)}=\frac{e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(\frac{1}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\frac{d\hat{h}}{d{\bf
p}_{2}}+\frac{2\hat{B}\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}^{3}}\right)e^{-\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)^{2}}\right),$ (62)

and

$\displaystyle f_{(d{\bf
p})}^{(3)}=\frac{\hat{B}e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}^{3}}\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right).$ (63)

A combination of (4.2.2.2), (54), (55), (58), and (60)-(63) is then sufficient
to determine ${\bf p}_{2}$–derivative.

(iii) ${\bf c}_{2}$ – derivative: We again start by writing

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2})-\frac{1}{2{\bf
c}_{2}^{2}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})}{2\gamma_{sq}^{(p)}}\right)$ (64) $\displaystyle-\frac{1-{\bf
q}_{2}}{2{\bf c}_{2}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}-\frac{{\bf q}_{2}(1-{\bf q}_{2})}{2(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))^{2}}$ $\displaystyle+\frac{\alpha}{{\bf
c}_{2}^{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\log\left(f_{(zt)}^{(2,f)}\right)-\frac{\alpha}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{1}{f_{(zt)}^{(2,f)}}\frac{d\left(f_{(zt)}^{(2,f)}\right)}{d{\bf
c}_{2}}\right).$

From (4.2.2.2), we then have

$\displaystyle\frac{df_{(zt)}^{(2,f)}}{d{\bf c}_{2}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d{\bf
c}_{2}}+\frac{df_{(zu)}^{(2,f)}}{d{\bf c}_{2}}=\frac{df_{(zd)}^{(2,f)}}{d{\bf
c}_{2}},$ (65)

where we utilized the fact that

$\displaystyle\frac{df_{(zu)}^{(2,f)}}{d{\bf
c}_{2}}=\frac{e^{-\frac{\hat{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\hat{h}}{d{\bf
c}_{2}}=0.$ (66)

After observing

$\displaystyle\frac{d\hat{B}}{d{\bf
c}_{2}}=\frac{1}{4\gamma_{sq}}\quad\mbox{and}\quad\frac{d\hat{C}}{d{\bf
c}_{2}}=0,$ (67)

we can further write

$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d{\bf c}_{2}}=f_{(d{\bf
c})}^{(1)}+f_{(d{\bf c})}^{(2)}+f_{(d{\bf c})}^{(3)},$ (68)

where

$\displaystyle f_{(d{\bf
c})}^{(1)}=\left(-\frac{\hat{C}^{2}}{4\gamma_{sq}(2(1-{\bf
p}_{2})\hat{B}+1)}+\frac{(1-{\bf
p}_{2})\hat{B}\hat{C}^{2}}{2\gamma_{sq}(2(1-{\bf
p}_{2})\hat{B}+1).^{2}}\right)e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}\frac{\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\hat{B}+1}},$ (69)

and

$f_{(d{\bf c})}^{(2)}=\frac{e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(-\frac{(1-{\bf
p}_{2})\hat{h}}{2\gamma_{sq}\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}^{3}}\right)e^{-\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)^{2}}\right),$ (70)

and

$\displaystyle f_{(d{\bf c})}^{(3)}=-\frac{(1-{\bf
p}_{2})e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{8\gamma_{sq}\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}^{3}}\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right),$ (71)

A combination of (4.2.2.2), (64), (65), and (68)-(71) is then sufficient to
determine ${\bf c}_{2}$–derivative.

(iv) $\gamma_{sq}^{(p)}$ – derivative: We easily find

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}$
$\displaystyle=$ $\displaystyle-1-\left(-\frac{1}{{\bf
c}_{2}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))}+\frac{1}{2{\bf
c}_{2}\gamma_{sq}^{(p)}}-\frac{{\bf q}_{2}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))^{2}}\right)$ $\displaystyle=$
$\displaystyle-1-\left(-\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}-\frac{{\bf q}_{2}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))^{2}}\right).$

(v) $\gamma_{sq}$ – derivative: We first write

$\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}=1-\frac{\alpha}{{\bf
c}_{2}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{1}{f_{(zt)}^{(2,f)}}\frac{d\left(f_{(zt)}^{(2,f)}\right)}{d\gamma_{sq}}\right).$
(73)

Relying on (4.2.2.2), we also have

$\displaystyle\frac{df_{(zt)}^{(2,f)}}{d\gamma_{sq}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d\gamma_{sq}}+\frac{df_{(zu)}^{(2,f)}}{d\gamma_{sq}}=\frac{df_{(zd)}^{(2,f)}}{d\gamma_{sq}},$
(74)

where we utilized

$\displaystyle\frac{df_{(zu)}^{(2,f)}}{d\gamma_{sq}}=\frac{e^{-\frac{\hat{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\hat{h}}{d\gamma_{sq}}=0.$
(75)

After observing

$\displaystyle\frac{d\hat{h}}{d\gamma_{sq}}=\frac{d\hat{C}}{d\gamma_{sq}}=0\quad\mbox{and}\quad\frac{d\hat{B}}{d\gamma_{sq}}=-\frac{{\bf
c}_{2}}{4\gamma_{sq}^{2}},$ (76)

we can further write

$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d\gamma_{sq}}=f_{(d\gamma)}^{(1)}+f_{(d\gamma)}^{(2)}+f_{(d\gamma)}^{(3)},$
(77)

where

$\displaystyle f_{(d\gamma)}^{(1)}=\left(\frac{{\bf
c}_{2}\hat{C}^{2}}{4\gamma_{sq}^{2}(2(1-{\bf p}_{2})\hat{B}+1)}-\frac{{\bf
c}_{2}(1-{\bf p}_{2})\hat{B}\hat{C}^{2}}{2\gamma_{sq}^{2}(2(1-{\bf
p}_{2})\hat{B}+1).^{2}}\right)e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}\frac{\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\hat{B}+1}},$ (78)

and

$f_{(d\gamma)}^{(2)}=\frac{e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(\frac{{\bf c}_{2}(1-{\bf
p}_{2})\hat{h}}{2\gamma_{sq}^{2}\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}^{3}}\right)e^{-\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right)^{2}}\right),$ (79)

and

$\displaystyle f_{(d\gamma)}^{(3)}=\frac{{\bf c}_{2}(1-{\bf
p}_{2})e^{-\frac{\hat{B}\hat{C}^{2}}{2(1-{\bf
p}_{2})\hat{B}+1}}}{8\gamma_{sq}^{2}\sqrt{2(1-{\bf
p}_{2})\hat{B}+1}^{3}}\mbox{erfc}\left(\frac{\hat{h}}{\sqrt{4(1-{\bf
p}_{2})\hat{B}+2}}\right),$ (80)

An easy combination of (4.2.2.2), (73), (74), and (77)-(80) ensures that all
the ingredients needed to determine $\gamma_{sq}$–derivative are obtained.
After solving the following system

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{2}}$ $\displaystyle=$
$\displaystyle 0$ $\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle 0$ $\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{2}}$ $\displaystyle=$
$\displaystyle 0$ $\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}$
$\displaystyle=$ $\displaystyle 0$ $\displaystyle\frac{d\bar{\psi}_{rd}({\bf
p},{\bf q},{\bf c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}$
$\displaystyle=$ $\displaystyle 0,$ (81)

and denoting by $\hat{{\bf q}}_{2},\hat{{\bf p}}_{2},\hat{{\bf
c}}_{2},\hat{\gamma}_{sq}^{(p)},\hat{\gamma}_{sq}$ the obtained solution, we
utilize

$\displaystyle-f_{sq}^{(2)}(\infty)=\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},\hat{\gamma}_{sq},\hat{\gamma}_{sq}^{(p)},1,1,-1)$
$\displaystyle=0,$ (82)

to determine the critical $\alpha_{c}(\kappa)$, for any given $\kappa$. For
example, taking $\kappa=-1.5$, we find

$(\mbox{{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{\emph{full}
second level:}}})\qquad\qquad
a_{c}^{(2,f)}(-1.5)\approx{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.57}}.$
(83)

_Closed form relations:_ To handle the above system we found as useful to
utilize the following helpful, closed form, relations. First from (53), we
find

$\displaystyle{\bf p}_{2}=\frac{{\bf q}_{2}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))^{2}}.$ (84)

From (4.2.2.2), we further have

$\displaystyle 1=\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}+\frac{{\bf q}_{2}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))^{2}}.$ (85)

Combining (84) and (85), we obtain

$\displaystyle 1=\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}+\frac{{\bf q}_{2}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))^{2}}=\frac{1-{\bf q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))}+{\bf p}_{2}.$ (86)

A further combination of (84) and (86) gives

$\displaystyle\gamma_{sq}^{(p)}=\frac{1-{\bf q}_{2}}{2(1-{\bf
p}_{2})(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}=\frac{1}{2}\frac{1-{\bf q}_{2}}{1-{\bf p}_{2}}\sqrt{\frac{{\bf
p}_{2}}{{\bf q}_{2}}}.$ (87)

Also, from (84) we have

$\displaystyle{\bf c}_{2}(1-{\bf q}_{2})=2\gamma_{sq}^{(p)}-\sqrt{\frac{{\bf
q}_{2}}{{\bf p}_{2}}}.$ (88)

A combination of (87) and (88) gives

$\displaystyle{\bf c}_{2}(1-{\bf q}_{2})=2\gamma_{sq}^{(p)}-\sqrt{\frac{{\bf
q}_{2}}{{\bf p}_{2}}}=\frac{1-{\bf q}_{2}}{1-{\bf p}_{2}}\sqrt{\frac{{\bf
p}_{2}}{{\bf q}_{2}}}-\sqrt{\frac{{\bf q}_{2}}{{\bf p}_{2}}},$ (89)

and

$\displaystyle{\bf c}_{2}=\frac{1}{1-{\bf p}_{2}}\sqrt{\frac{{\bf p}_{2}}{{\bf
q}_{2}}}-\frac{1}{1-{\bf q}_{2}}\sqrt{\frac{{\bf q}_{2}}{{\bf p}_{2}}}.$ (90)

_Concrete numerical values:_ In Table 1, we complement $a_{c}^{(2,f)}(-1.5)$
with the concrete values of all the relevant quantities related to the second
_full_ (2-sfl RDT) level of lifting. To enable a systematic view of the
lifting progress, we, in parallel, show the same quantities for the first
_full_ (1-sfl RDT) and the second _partial_ (2-spf RDT) level.

Table 1: $r$-sfl RDT parameters; _negative_ spherical perceptron capacity; $\hat{{\bf c}}_{1}\rightarrow 1$; $\kappa=-1.5$; $n,\beta\rightarrow\infty$ $r$-sfl RDT | $\hat{\gamma}_{sq}$ | $\hat{\gamma}_{sq}^{(p)}$ | $\hat{{\bf p}}_{2}$ | $\hat{{\bf p}}_{1}`$ | $\hat{{\bf q}}_{2}$ | $\hat{{\bf q}}_{1}$ | $\hat{{\bf c}}_{2}$ | $\alpha_{c}^{(r)}(-1.5)$
---|---|---|---|---|---|---|---|---
$1$-sfl RDT | $0.5$ | $0.5$ | $0$ | $\rightarrow 1$ | $0$ | $\rightarrow 1$ | $\rightarrow 0$ | $\mathbf{43.77}$
$2$-spl RDT | $0.1737$ | $1.4397$ | $0$ | $\rightarrow 1$ | $0$ | $\rightarrow 1$ | $2.5320$ | $\mathbf{37.36}$
$2$-sfl RDT | $0.1324$ | $1.8884$ | $0.4747$ | $\rightarrow 1$ | $0.0981$ | $\rightarrow 1$ | $3.6835$ | $\mathbf{36.57}$

In Table 2, we show the key, second level of lifting, parameters over a range
of $\kappa$.

Table 2: $2$-sfl RDT parameters; _negative_ spherical perceptron capacity $\alpha=\alpha_{c}^{(2,f)}(\kappa)$ $\kappa$ | $\mathbf{-2.7}$ | $\mathbf{-2.3}$ | $\mathbf{-2.0}$ | $\mathbf{-1.7}$ | $\mathbf{-1.5}$ | $\mathbf{-1.3}$ | $\mathbf{-1}$ | $\mathbf{-0.7}$ | $\mathbf{-0.5}$ | $\mathbf{-0.3}$
---|---|---|---|---|---|---|---|---|---|---
$\hat{\gamma}_{sq}$ | $0.0739$ | $0.0925$ | $0.1086$ | $0.1238$ | $0.1324$ | $0.1403$ | $0.1500$ | $0.1575$ | $0.1609$ | $0.1631$
$\hat{\gamma}_{sq}^{(p)}$ | $3.3827$ | $2.7015$ | $2.3033$ | $2.0219$ | $1.8884$ | $1.7823$ | $1.6660$ | $1.5869$ | $1.5542$ | $1.5339$
$\hat{{\bf p}}_{2}$ | $0.0560$ | $0.1503$ | $0.2664$ | $0.3934$ | $0.4747$ | $0.5520$ | $0.6649$ | $0.7781$ | $0.8550$ | $0.9293$
$\hat{{\bf q}}_{2}$ | $0.0014$ | $0.0070$ | $0.0223$ | $0.0580$ | $0.0981$ | $0.1547$ | $0.2780$ | $0.4591$ | $0.6170$ | $0.7990$
$\hat{{\bf c}}_{2}$ | $6.6181$ | $5.2234$ | $4.4157$ | $3.8852$ | $3.6835$ | $3.5906$ | $3.7194$ | $4.4476$ | $5.8985$ | $10.650$
$\alpha$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{942.9}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{284.5}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{125.43}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{58.80}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.57}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{23.29}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{12.32}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{6.817}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{4.701}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{3.298}}$

The progression of the capacity as the level of lifting increases is shown in
Table 3.

Table 3: _Negative_ spherical perceptron capacity $\alpha_{c}(\kappa)$ — early progression of $r$-sfl RDT mechanism $\kappa$ | $\mathbf{-2.0}$ | $\mathbf{-1.5}$ | $\mathbf{-1}$ | $\mathbf{-0.5}$
---|---|---|---|---
$\alpha_{c}^{(1,f)}(\kappa)$ | $173.4$ | $43.77$ | $13.27$ | $4.770$
$\alpha_{c}^{(2,p)}(\kappa)$ | $126.2$ | $37.36$ | $12.78$ | $4.770$
$\alpha_{c}^{(2,f)}(\kappa)$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{125.4}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.57}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{12.32}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{4.701}}$

####  4.2.3 $r=3$ – third level of lifting

Since we have already seen the main idea behind the partial lifting, we here
immediately consider _full_ third level of lifting. For $r=3$, we have that
$\hat{{\bf p}}_{1}\rightarrow 1$ and $\hat{{\bf q}}_{1}\rightarrow 1$ as well
as $\hat{{\bf p}}_{r+1}=\hat{{\bf p}}_{4}=\hat{{\bf q}}_{r+1}=\hat{{\bf
q}}_{4}=0$. Analogously to (4.2.2.1), we now write

$\displaystyle\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=\frac{1}{2}(1-{\bf
p}_{2}{\bf q}_{2}){\bf c}_{2}+\frac{1}{2}({\bf p}_{2}{\bf q}_{2}-{\bf
p}_{3}{\bf q}_{3}){\bf c}_{3}$
$\displaystyle\quad-\gamma_{sq}^{(p)}-\frac{1}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{{\bf
c}_{2}\frac{\left(\sqrt{1-{\bf q}_{2}}{\bf h}_{1}^{(2)}+\sqrt{{\bf q}_{2}-{\bf
q}_{3}}{\bf h}_{1}^{(3)}+\sqrt{{\bf q}_{3}}{\bf
h}_{1}^{(4)}\right)^{2}}{4\gamma_{sq}^{(p)}}}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right)$ $\displaystyle\quad+\gamma_{sq}-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf p}_{3}}{\bf
u}_{1}^{(2,4)},0)^{2}}{4\gamma_{sq}}}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right)$ $\displaystyle=\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2}){\bf
c}_{2}+\frac{1}{2}({\bf p}_{2}{\bf q}_{2}-{\bf p}_{3}{\bf q}_{3}){\bf
c}_{3}-\gamma_{sq}^{(p)}$ $\displaystyle\quad-\Bigg{(}\Bigg{.}-\frac{1}{2{\bf
c}_{2}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})}{2\gamma_{sq}^{(p)}}\right)-\frac{1}{2{\bf
c}_{3}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3})}{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})}\right)$ $\displaystyle\quad+\frac{{\bf
q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}\Bigg{.}\Bigg{)}$
$\displaystyle\quad+\gamma_{sq}-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf p}_{3}}{\bf
u}_{1}^{(2,4)},0)^{2}}{4\gamma_{sq}}}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right),$ (91)

where we handled the first sequence of integrals utilizing the closed form
solutions obtained in [40]. After solving the remaining integrals, we also
have

$\displaystyle\tilde{h}$ $\displaystyle=$ $\displaystyle-\frac{\sqrt{{\bf
p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf p}_{3}}{\bf
u}_{1}^{(2,4)}+\kappa}{\sqrt{1-{\bf p}_{2}}}$ $\displaystyle\tilde{B}$
$\displaystyle=$ $\displaystyle\frac{{\bf c}_{2}}{4\gamma_{sq}}$
$\displaystyle\tilde{C}$ $\displaystyle=$ $\displaystyle\sqrt{{\bf p}_{2}-{\bf
p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf p}_{3}}{\bf u}_{1}^{(2,4)}+\kappa$
$\displaystyle f_{(zd)}^{(3,f)}$ $\displaystyle=$
$\displaystyle\frac{e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}}\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)$ $\displaystyle f_{(zu)}^{(3,f)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\mbox{erfc}\left(-\frac{\tilde{h}}{\sqrt{2}}\right),$
$\displaystyle f_{(zt)}^{(3,f)}$ $\displaystyle=$ $\displaystyle
f_{(zd)}^{(3,f)}+f_{(zu)}^{(3,f)}.$ (92)

and

$\displaystyle{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left({\mathbb{E}}_{{\mathcal{U}}_{2}}e^{-{\bf
c}_{2}\frac{\max(\sqrt{1-{\bf p}_{2}}{\bf u}_{1}^{(2,2)}+\sqrt{{\bf
p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf p}_{3}}{\bf
u}_{1}^{(2,4)},0)^{2}}{4\gamma_{sq}}}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}}\right)={\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right).$ (93)

Combining (4.2.3) and (93), we obtain

$\displaystyle\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)$ $\displaystyle=$
$\displaystyle\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2}){\bf
c}_{2}+\frac{1}{2}({\bf p}_{2}{\bf q}_{2}-{\bf p}_{3}{\bf q}_{3}){\bf
c}_{3}-\gamma_{sq}^{(p)}$ (94) $\displaystyle-\Bigg{(}\Bigg{.}-\frac{1}{2{\bf
c}_{2}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})}{2\gamma_{sq}^{(p)}}\right)$ $\displaystyle-\frac{1}{2{\bf
c}_{3}}\log\left(\frac{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3})}{2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})}\right)$ $\displaystyle+\frac{{\bf q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))}\Bigg{.}\Bigg{)}$
$\displaystyle+\gamma_{sq}-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right).$

####  4.2.4 Third level derivatives

One now needs to compute _eight_ derivatives with respect to ${\bf q}_{3}$,
${\bf q}_{2}$, ${\bf p}_{3}$, ${\bf p}_{2}$,, ${\bf c}_{3}$, ${\bf c}_{2}$,,
$\gamma_{sq}$,, and $\gamma_{sq}^{(p)}$. We again systematically compute each
of them.

(i) ${\bf q}_{3}$ – derivative: Utilizing (4.2.3) and (94), we have

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{3}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}{\bf
c}_{3}-\Bigg{(}\Bigg{.}-\frac{1}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))}$ (95)
$\displaystyle+\frac{1}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3}))}$ $\displaystyle-\frac{{\bf c}_{3}{\bf
q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{3}{\bf c}_{3}+\frac{{\bf c}_{3}{\bf
q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}.$

(ii) ${\bf q}_{2}$ – derivative: Relying further on (4.2.3) and (94), we also
have

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}({\bf c}_{2}-{\bf
c}_{3})-\Bigg{(}\Bigg{.}-\frac{1}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}$ (96) $\displaystyle-\frac{{\bf c}_{2}-{\bf c}_{3}}{2{\bf
c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}+\frac{{\bf c}_{2}}{2{\bf c}_{3}(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))}$ $\displaystyle-\frac{{\bf q}_{3}({\bf c}_{2}-{\bf
c}_{3})}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}({\bf c}_{2}-{\bf
c}_{3})-\Bigg{(}\Bigg{.}\frac{{\bf c}_{2}-{\bf c}_{3}}{2{\bf
c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))}$
$\displaystyle-\frac{{\bf c}_{2}-{\bf c}_{3}}{2{\bf
c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}$ $\displaystyle-\frac{{\bf q}_{3}({\bf c}_{2}-{\bf
c}_{3})}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf p}_{2}({\bf c}_{2}-{\bf c}_{3})-({\bf
c}_{2}-{\bf c}_{3})\Bigg{(}\Bigg{.}\frac{1}{2{\bf
c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))}$
$\displaystyle-\frac{1}{2{\bf c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))}$ $\displaystyle-\frac{{\bf
q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle({\bf c}_{2}-{\bf c}_{3})\Bigg{(}\Bigg{.}-\frac{{\bf p}_{2}}{2}$
$\displaystyle+\frac{{\bf q}_{2}-{\bf q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3}))}$ $\displaystyle+\frac{{\bf
q}_{3}}{2(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}.$

(iii) ${\bf p}_{3}$ – derivative: As above, we utilize (4.2.3) and (94) and
start by writing

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{3}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{3}{\bf c}_{3}-\frac{\alpha}{{\bf
c}_{3}}\frac{d\left({\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right)\right)}{d{\bf p}_{3}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{3}{\bf c}_{3}-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\left(\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf
c}_{2}}}}\frac{d\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right)}{d{\bf p}_{3}}\right)$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{3}{\bf c}_{3}-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\left(\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{{\bf
c}_{3}}{{\bf c}_{2}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}-1}\frac{df_{(zt)}^{(3,f)}}{d{\bf p}_{3}}\right)\right).$

From (4.2.3), we then have

$\displaystyle\frac{df_{(zt)}^{(3,f)}}{d{\bf p}_{3}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf
p}_{3}}+\frac{df_{(zu)}^{(3,f)}}{d{\bf p}_{3}}.$ (98)

Moreover, utilizing (4.2.3) further, we can also write

$\displaystyle\frac{df_{(zu)}^{(3,f)}}{d{\bf
p}_{3}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d{\bf
p}_{3}},$ (99)

and

$\displaystyle\frac{d\tilde{h}}{d{\bf p}_{3}}=-\frac{-\frac{1}{2\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}+\frac{1}{2\sqrt{{\bf p}_{3}}}{\bf
u}_{1}^{(2,4)}+\kappa}{\sqrt{1-{\bf p}_{2}}}.$ (100)

A combination of (99) and (100) gives

$\displaystyle\frac{df_{(zu)}^{(3,f)}}{d{\bf
p}_{3}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d{\bf
p}_{3}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\left(-\frac{-\frac{1}{2\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}+\frac{1}{2\sqrt{{\bf p}_{3}}}{\bf
u}_{1}^{(2,4)}+\kappa}{\sqrt{1-{\bf p}_{2}}}\right).$ (101)

After observing

$\displaystyle\frac{d\tilde{C}}{d{\bf p}_{3}}=-\frac{1}{2\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}+\frac{1}{2\sqrt{{\bf p}_{3}}}{\bf
u}_{1}^{(2,4)},$ (102)

we can further write

$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf p}_{3}}=f_{(d{\bf
p}_{3})}^{(1)}+f_{(d{\bf p}_{3})}^{(2)},$ (103)

where

$\displaystyle f_{(d{\bf
p}_{3})}^{(1)}=\left(-\frac{\tilde{B}\tilde{C}\left(-\frac{1}{\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}+\frac{1}{\sqrt{{\bf p}_{3}}}{\bf
u}_{1}^{(2,4)}\right)}{(2(1-{\bf
p}_{2})\tilde{B}+1)}\right)e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}\frac{\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\tilde{B}+1}},$ (104)

and

$f_{(d{\bf p}_{3})}^{(2)}=\frac{e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(\frac{1}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\frac{d\tilde{h}}{d{\bf
p}_{3}}\right)e^{-\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)^{2}}\right).$ (105)

A combination of (4.2.3), (4.2.4), (98), (101), and (103)-(105) is then
sufficient to determine ${\bf p}_{3}$–derivative.

(iv) ${\bf p}_{2}$ – derivative: Relying again on (4.2.3) and (94), we have

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{2}({\bf c}_{2}-{\bf
c}_{3})-\frac{\alpha}{{\bf
c}_{3}}\frac{d\left({\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right)\right)}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{2}({\bf c}_{2}-{\bf
c}_{3})-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\left(\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf
c}_{2}}}}\frac{d\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right)}{d{\bf p}_{2}}\right)$ $\displaystyle=$
$\displaystyle-\frac{1}{2}{\bf q}_{2}({\bf c}_{2}-{\bf c}_{3})$
$\displaystyle-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\left(\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{{\bf
c}_{3}}{{\bf c}_{2}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}-1}\frac{df_{(zt)}^{(3,f)}}{d{\bf p}_{2}}\right)\right).$

From (4.2.3), we then find

$\displaystyle\frac{df_{(zt)}^{(3,f)}}{d{\bf p}_{2}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf
p}_{2}}+\frac{df_{(zu)}^{(3,f)}}{d{\bf p}_{2}}.$ (107)

Utilizing (4.2.3) further, we also have

$\displaystyle\frac{df_{(zu)}^{(3,f)}}{d{\bf
p}_{2}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d{\bf
p}_{2}},$ (108)

and

$\displaystyle\frac{d\tilde{h}}{d{\bf p}_{2}}=-\frac{\frac{1}{2\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}}{\sqrt{1-{\bf
p}_{2}}}-\frac{\sqrt{{\bf p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf
p}_{3}}{\bf u}_{1}^{(2,4)}+\kappa}{2\sqrt{1-{\bf p}_{2}}^{3}}.$ (109)

Combining (108) and (109), we obtain

$\displaystyle\frac{df_{(zu)}^{(3,f)}}{d{\bf
p}_{2}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d{\bf
p}_{3}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\left(-\frac{\frac{1}{2\sqrt{{\bf
p}_{2}-{\bf p}_{3}}}{\bf u}_{1}^{(2,3)}}{\sqrt{1-{\bf
p}_{2}}}-\frac{\sqrt{{\bf p}_{2}-{\bf p}_{3}}{\bf u}_{1}^{(2,3)}+\sqrt{{\bf
p}_{3}}{\bf u}_{1}^{(2,4)}+\kappa}{2\sqrt{1-{\bf p}_{2}}^{3}}\right).$ (110)

After observing

$\displaystyle\frac{d\tilde{C}}{d{\bf p}_{2}}=\frac{1}{2\sqrt{{\bf p}_{2}-{\bf
p}_{3}}}{\bf u}_{1}^{(2,3)},$ (111)

we can further write

$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf p}_{2}}=f_{(d{\bf
p}_{2})}^{(1)}+f_{(d{\bf p}_{2})}^{(2)}+f_{(d{\bf p}_{2})}^{(3)},$ (112)

where

$\displaystyle f_{(d{\bf p}_{2})}^{(1)}=\left(-\frac{\tilde{B}\tilde{C}{\bf
u}_{1}^{(2,3)}}{\sqrt{{\bf p}_{2}-{\bf p}_{3}}(2(1-{\bf
p}_{2})\tilde{B}+1)}-\frac{2\tilde{B}^{2}\tilde{C}^{2}}{(2(1-{\bf
p}_{2})\tilde{B}+1).^{2}}\right)e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}\frac{\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\tilde{B}+1}},$ (113)

and

$f_{(d{\bf p}_{2})}^{(2)}=\frac{e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(\frac{1}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\frac{d\tilde{h}}{d{\bf
p}_{2}}+\frac{2\tilde{B}\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}^{3}}\right)e^{-\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)^{2}}\right),$ (114)

and

$\displaystyle f_{(d{\bf
p}_{2})}^{(3)}=\frac{\tilde{B}e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}^{3}}\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right).$ (115)

A combination of (4.2.3), (4.2.4), (107), (110), and (112)-(115) is then
sufficient to determine ${\bf p}_{2}$–derivative.

(v) ${\bf c}_{3}$ – derivative: As usual, we start by utilizing (4.2.3) and
(94) and write

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{3}}$ $\displaystyle=$
$\displaystyle\frac{1}{2}({\bf p}_{2}{\bf q}_{2}-{\bf p}_{3}{\bf q}_{3})$
(116) $\displaystyle-\Bigg{(}\Bigg{.}\frac{1}{2{\bf
c}_{3}^{2}}\log\left(\frac{2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3})}{2\gamma_{sq}-{\bf c}_{2}(1-{\bf
q}_{2})}\right)$ $\displaystyle\frac{{\bf q}_{2}-{\bf q}_{3}}{2{\bf
c}_{3}(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf
q}_{3}))}$ $\displaystyle+\frac{{\bf q}_{3}({\bf q}_{2}-{\bf
q}_{3})}{2(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle+\frac{\alpha}{{\bf
c}_{3}^{2}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\log\left({\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\right)$ $\displaystyle-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf
c}_{2}}}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\frac{1}{{\bf
c}_{2}}\log\left(f_{(zt)}^{(3,f)}\right)\right).$

(116) is then sufficient to determine ${\bf c}_{3}$–derivative.

(vi) ${\bf c}_{2}$ – derivative: We once again start by utilizing (4.2.3) and
(94) and write

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{2}(1-{\bf p}_{2}{\bf q}_{2})$ (117)
$\displaystyle-\Bigg{(}\Bigg{.}\frac{1}{2{\bf
c}_{2}^{2}}\log\left(\frac{2\gamma_{sq}-{\bf c}_{2}(1-{\bf
q}_{2})}{2\gamma_{sq}}\right)+\frac{1-{\bf q}_{2}}{2{\bf
c}_{2}(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2}))}$ $\displaystyle+\frac{1-{\bf
q}_{2}}{2{\bf c}_{3}(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}$ $\displaystyle-\frac{1-{\bf q}_{2}}{2{\bf
c}_{3}(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2}))}+\frac{{\bf q}_{3}(1-{\bf
q}_{2})}{2(2\gamma_{sq}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}}$
$\displaystyle\times{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}\left(-\frac{{\bf c}_{3}}{{\bf
c}_{2}^{2}}\log\left(f_{(zt)}^{(3,f)}\right)+\frac{{\bf c}_{3}}{{\bf
c}_{2}f_{(zt)}^{(3,f)}}\frac{df_{(zt)}^{(3,f)}}{d{\bf c}_{2}}\right)\right).$

From (4.2.3), we find

$\displaystyle\frac{df_{(zt)}^{(3,f)}}{d{\bf c}_{2}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf
c}_{2}}+\frac{df_{(zu)}^{(3,f)}}{d{\bf c}_{2}}=\frac{df_{(zd)}^{(3,f)}}{d{\bf
c}_{2}},$ (118)

where we utilized the fact that

$\displaystyle\frac{df_{(zu)}^{(3,f)}}{d{\bf
c}_{2}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d{\bf
c}_{2}}=0.$ (119)

After observing

$\displaystyle\frac{d\tilde{B}}{d{\bf
c}_{2}}=\frac{1}{4\gamma_{sq}}\quad\mbox{and}\quad\frac{d\tilde{C}}{d{\bf
c}_{2}}=0,$ (120)

we further write

$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d{\bf c}_{2}}=f_{(d{\bf
c}_{2})}^{(1)}+f_{(d{\bf c}_{2})}^{(2)}+f_{(d{\bf c}_{2})}^{(3)},$ (121)

where

$\displaystyle f_{(d{\bf
c}_{2})}^{(1)}=\left(-\frac{\tilde{C}^{2}}{4\gamma_{sq}(2(1-{\bf
p}_{2})\tilde{B}+1)}+\frac{(1-{\bf
p}_{2})\tilde{B}\tilde{C}^{2}}{2\gamma_{sq}(2(1-{\bf
p}_{2})\tilde{B}+1).^{2}}\right)e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}\frac{\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\tilde{B}+1}},$ (122)

and

$f_{(d{\bf c}_{2})}^{(2)}=\frac{e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(-\frac{(1-{\bf
p}_{2})\tilde{h}}{2\gamma_{sq}\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}^{3}}\right)e^{-\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)^{2}}\right),$ (123)

and

$\displaystyle f_{(d{\bf c}_{2})}^{(3)}=-\frac{(1-{\bf
p}_{2})e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{8\gamma_{sq}\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}^{3}}\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right),$ (124)

A combination of (4.2.3), (117), (118), and (121)-(124) is then sufficient to
determine ${\bf c}_{2}$–derivative.

(vii) $\gamma_{sq}^{(p)}$ – derivative: From (4.2.3) and (94) , we easily also
find

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}$
$\displaystyle=$ $\displaystyle-1-\Bigg{(}\Bigg{.}-\frac{1}{{\bf
c}_{2}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))}+\frac{1}{2{\bf
c}_{2}\gamma_{sq}^{(p)}}$ (125) $\displaystyle-\frac{1}{{\bf
c}_{3}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}+\frac{1}{{\bf c}_{3}(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))}$ $\displaystyle-\frac{{\bf
q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}$ $\displaystyle=$
$\displaystyle-1-\Bigg{(}\Bigg{.}-\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2}))}$
$\displaystyle-\frac{{\bf q}_{2}-{\bf q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3}))}$ $\displaystyle-\frac{{\bf
q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))^{2}}\Bigg{.}\Bigg{)}.$

(viii) $\gamma_{sq}$ – derivative: Relying again on (4.2.3) and (94), we write

$\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}=1-\frac{\alpha}{{\bf
c}_{3}}{\mathbb{E}}_{{\mathcal{U}}_{4}}\left(\frac{1}{{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf
c}_{3}}{{\bf c}_{2}}}}{\mathbb{E}}_{{\mathcal{U}}_{3}}\left(\frac{{\bf
c}_{3}}{{\bf c}_{2}}\left(f_{(zt)}^{(3,f)}\right)^{\frac{{\bf c}_{3}}{{\bf
c}_{2}}-1}\frac{df_{(zt)}^{(3,f)}}{d\gamma_{sq}}\right)\right).$ (126)

From (4.2.3), we also have

$\displaystyle\frac{df_{(zt)}^{(3,f)}}{d\gamma_{sq}}$ $\displaystyle=$
$\displaystyle\frac{df_{(zd)}^{(3,f)}}{d\gamma_{sq}}+\frac{df_{(zu)}^{(3,f)}}{d\gamma_{sq}}=\frac{df_{(zd)}^{(3,f)}}{d\gamma_{sq}},$
(127)

where we utilized

$\displaystyle\frac{df_{(zu)}^{(2,f)}}{d\gamma_{sq}}=\frac{e^{-\frac{\tilde{h}^{2}}{2}}}{\sqrt{2\pi}}\frac{d\tilde{h}}{d\gamma_{sq}}=0.$
(128)

After observing

$\displaystyle\frac{d\tilde{h}}{d\gamma_{sq}}=\frac{d\tilde{C}}{d\gamma_{sq}}=0\quad\mbox{and}\quad\frac{d\tilde{B}}{d\gamma_{sq}}=-\frac{{\bf
c}_{2}}{4\gamma_{sq}^{2}},$ (129)

we can further write

$\displaystyle\frac{df_{(zd)}^{(2,f)}}{d\gamma_{sq}}=f_{(d\gamma_{sq})}^{(1)}+f_{(d\gamma_{sq})}^{(2)}+f_{(d\gamma_{sq})}^{(3)},$
(130)

where

$\displaystyle f_{(d\gamma_{sq})}^{(1)}=\left(\frac{{\bf
c}_{2}\tilde{C}^{2}}{4\gamma_{sq}^{2}(2(1-{\bf p}_{2})\tilde{B}+1)}-\frac{{\bf
c}_{2}(1-{\bf p}_{2})\tilde{B}\tilde{C}^{2}}{2\gamma_{sq}^{2}(2(1-{\bf
p}_{2})\tilde{B}+1).^{2}}\right)e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}\frac{\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)}{2\sqrt{2(1-{\bf p}_{2})\tilde{B}+1}},$ (131)

and

$f_{(d\gamma_{sq})}^{(2)}=\frac{e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{2\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}}\left(-\frac{2}{\sqrt{\pi}}\left(\frac{{\bf c}_{2}(1-{\bf
p}_{2})\tilde{h}}{2\gamma_{sq}^{2}\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}^{3}}\right)e^{-\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right)^{2}}\right),$ (132)

and

$\displaystyle f_{(d\gamma_{sq})}^{(3)}=\frac{{\bf c}_{2}(1-{\bf
p}_{2})e^{-\frac{\tilde{B}\tilde{C}^{2}}{2(1-{\bf
p}_{2})\tilde{B}+1}}}{8\gamma_{sq}^{2}\sqrt{2(1-{\bf
p}_{2})\tilde{B}+1}^{3}}\mbox{erfc}\left(\frac{\tilde{h}}{\sqrt{4(1-{\bf
p}_{2})\tilde{B}+2}}\right),$ (133)

Together, (4.2.3), (126), (127), and (130)-(133) provide all necessary
ingredients to determine $\gamma_{sq}$–derivative. One then proceeds by
solving the following system

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{3}}$
$\displaystyle=\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}_{2}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{3}}$
$\displaystyle=\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}_{2}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{3}}$
$\displaystyle=\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf c}_{2}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}$
$\displaystyle=\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}=0.$ (134)

After denoting by $\hat{{\bf q}}_{2},\hat{{\bf p}}_{2},\hat{{\bf
c}}_{2},\hat{\gamma}_{sq}^{(p)},\hat{\gamma}_{sq}$ the obtained solution, one
further utilizes

$\displaystyle-f_{sq}^{(3)}(\infty)=\bar{\psi}_{rd}(\hat{{\bf p}},\hat{{\bf
q}},\hat{{\bf c}},\hat{\gamma}_{sq},\hat{\gamma}_{sq}^{(p)},1,1,-1)$
$\displaystyle=0,$ (135)

to determine the critical $\alpha_{c}(\kappa)$, for any given $\kappa$. For
example, specializing to $\kappa=-1.5$, we find

$(\mbox{{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}{\emph{full}
third level:}}})\qquad\qquad
a_{c}^{(3,f)}(-1.5)\approx{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.40}}.$
(136)

####  4.2.5 Explicit generic closed form parametric relations

Solving the above system is doable in principle. However, in general it is not
an easy task. It often requires a substantial effort to conduct all the
required numerical work. Rather surprisingly and despite heavy analytical
machinery, it turns out that the key lifting parameters are generically
connected to each other. Moreover, we below uncover that the parametric
interconnections can be described via _remarkably simple and elegant_ closed
form expressions. Besides their analytical importance, the relations that we
provide below are practically extremely useful and make the underlying
numerical work immeasurably simpler and smoother.

We first observe that from (95) one can obtain

$\displaystyle{\bf p}_{3}=\frac{{\bf q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))^{2}}.$ (137)

In a similar fashion, from (96), we find

$\displaystyle{\bf p}_{2}=\frac{{\bf q}_{2}-{\bf
q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf
q}_{2}-{\bf q}_{3}))}+\frac{{\bf q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))^{2}}.$ (138)

A combination of (137) and (138) then gives

$\displaystyle 2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})=\frac{{\bf
q}_{2}-{\bf q}_{3}}{({\bf p}_{2}-{\bf p}_{3})(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2})-{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3}))}=\frac{{\bf
q}_{2}-{\bf q}_{3}}{{\bf p}_{2}-{\bf p}_{3}}\sqrt{\frac{{\bf p}_{3}}{{\bf
q}_{3}}}.$ (139)

One then also observes

$\displaystyle{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3})=2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2})-(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3})).$ (140)

A combination of (137), (139), and (140) then gives

$\displaystyle{\bf c}_{3}({\bf q}_{2}-{\bf q}_{3})=\frac{{\bf q}_{2}-{\bf
q}_{3}}{{\bf p}_{2}-{\bf p}_{3}}\sqrt{\frac{{\bf p}_{3}}{{\bf
q}_{3}}}-\sqrt{\frac{{\bf q}_{3}}{{\bf p}_{3}}},$ (141)

and

$\displaystyle{\bf c}_{3}=\frac{1}{{\bf p}_{2}-{\bf p}_{3}}\sqrt{\frac{{\bf
p}_{3}}{{\bf q}_{3}}}-\frac{1}{{\bf q}_{2}-{\bf q}_{3}}\sqrt{\frac{{\bf
q}_{3}}{{\bf p}_{3}}}.$ (142)

From (125), we also have

$\displaystyle 1=\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}+\frac{{\bf q}_{2}-{\bf q}_{3}}{(2\gamma_{sq}^{(p)}-{\bf
c}_{2}(1-{\bf q}_{2}))(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf q}_{2})-{\bf
c}_{3}({\bf q}_{2}-{\bf q}_{3}))}+{\bf p}_{3}.$ (143)

A combination of (138) and (143) further gives

$\displaystyle 1=\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}+({\bf p}_{2}-{\bf p}_{3})+{\bf p}_{3}=\frac{1-{\bf
q}_{2}}{2\gamma_{sq}^{(p)}(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}+{\bf p}_{2}.$ (144)

From (139) and (144), we then find

$\displaystyle\gamma_{sq}^{(p)}=\frac{1-{\bf q}_{2}}{2(1-{\bf
p}_{2})(2\gamma_{sq}^{(p)}-{\bf c}_{2}(1-{\bf
q}_{2}))}=\frac{1}{2}\frac{1-{\bf q}_{2}}{1-{\bf p}_{2}}\frac{{\bf p}_{2}-{\bf
p}_{3}}{{\bf q}_{2}-{\bf q}_{3}}\sqrt{\frac{{\bf q}_{3}}{{\bf p}_{3}}}.$ (145)

Moreover, from (139) and (145), we also have

$\displaystyle{\bf c}_{2}(1-{\bf q}_{2})=2\gamma_{sq}^{(p)}-\frac{{\bf
q}_{2}-{\bf q}_{3}}{{\bf p}_{2}-{\bf p}_{3}}\sqrt{\frac{{\bf p}_{3}}{{\bf
q}_{3}}}.$ (146)

Combining (145) and (146), one then easily also has

$\displaystyle{\bf c}_{2}=\frac{2\gamma_{sq}^{(p)}}{1-{\bf
q}_{2}}-\frac{1}{1-{\bf q}_{2}}\frac{{\bf q}_{2}-{\bf q}_{3}}{{\bf p}_{2}-{\bf
p}_{3}}\sqrt{\frac{{\bf p}_{3}}{{\bf q}_{3}}}=\frac{1}{1-{\bf
p}_{2}}\frac{{\bf p}_{2}-{\bf p}_{3}}{{\bf q}_{2}-{\bf q}_{3}}\sqrt{\frac{{\bf
q}_{3}}{{\bf p}_{3}}}-\frac{1}{1-{\bf q}_{2}}\frac{{\bf q}_{2}-{\bf
q}_{3}}{{\bf p}_{2}-{\bf p}_{3}}\sqrt{\frac{{\bf p}_{3}}{{\bf q}_{3}}}.$ (147)

We found all the above relations (and in particular those given in (142),
(145), and (147)) as extremely useful for the numerical work. Moreover,
following the above procedure one also obtains for any $r$ the analogue
versions of (142), (145), and (147)

$\displaystyle\gamma_{sq}^{(p)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\frac{{\bf q}_{1}-{\bf q}_{2}}{{\bf p}_{1}-{\bf
p}_{2}}\prod_{k=2:2:r-1}\frac{{\bf p}_{k}-{\bf p}_{k+1}}{{\bf q}_{k}-{\bf
q}_{k+1}}\prod_{k=2:2:r-2}\frac{{\bf q}_{k+1}-{\bf q}_{k+2}}{{\bf
p}_{k+1}-{\bf p}_{k+2}}\sqrt{\left(\frac{{\bf q}_{r}}{{\bf
p}_{r}}\right)^{(-1)^{r+1}}}.$ (148)

and for $i\in\\{2,3,\dots,r\\}$

$\displaystyle{\bf c}_{i}$ $\displaystyle=$ $\displaystyle\frac{1}{{\bf
p}_{i-1}-{\bf p}_{i}}\prod_{k=i:2:r-1}\frac{{\bf p}_{k}-{\bf p}_{k+1}}{{\bf
q}_{k}-{\bf q}_{k+1}}\prod_{k=i:2:r-2}\frac{{\bf q}_{k+1}-{\bf q}_{k+2}}{{\bf
p}_{k+1}-{\bf p}_{k+2}}\sqrt{\left(\frac{{\bf q}_{r}}{{\bf
p}_{r}}\right)^{(-1)^{i}}}$ (149) $\displaystyle-\frac{1}{{\bf q}_{i-1}-{\bf
q}_{i}}\prod_{k=i:2:r-1}\frac{{\bf q}_{k}-{\bf q}_{k+1}}{{\bf p}_{k}-{\bf
p}_{k+1}}\prod_{k=i:2:r-2}\frac{{\bf p}_{k+1}-{\bf p}_{k+2}}{{\bf
q}_{k+1}-{\bf q}_{k+2}}\sqrt{\left(\frac{{\bf p}_{r}}{{\bf
q}_{r}}\right)^{(-1)^{i}}}.$

We summarize the above in the following lemma.

######  Theorem 3.

Assume the setup of Theorem 2. Let the “non-fixed” parts of $\hat{{\bf
p}}_{k}$, $\hat{{\bf q}}_{k}$, and $\hat{{\bf c}}_{k}$
($k\in\\{2,3,\dots,r\\}$) be the solutions of the system in (2). The following
holds.

For $r=1$:

$\displaystyle\hat{\gamma}_{sq}^{(p)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}.$ (150)

For $r=2$:

$\displaystyle\hat{\gamma}_{sq}^{(p)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\frac{1-\hat{{\bf q}}_{2}}{1-\hat{{\bf
p}}_{2}}\sqrt{\frac{\hat{{\bf p}}_{2}}{\hat{{\bf q}}_{2}}}$
$\displaystyle\hat{{\bf c}}_{2}$ $\displaystyle=$
$\displaystyle\frac{1}{1-\hat{{\bf p}}_{2}}\sqrt{\frac{\hat{{\bf
p}}_{2}}{\hat{{\bf q}}_{2}}}-\frac{1}{1-\hat{{\bf
q}}_{2}}\sqrt{\frac{\hat{{\bf q}}_{2}}{\hat{{\bf p}}_{2}}}.$ (151)

For $r=3$:

$\displaystyle\hat{\gamma}_{sq}^{(p)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\frac{1-\hat{{\bf q}}_{2}}{1-\hat{{\bf
p}}_{2}}\frac{\hat{{\bf p}}_{2}-\hat{{\bf p}}_{3}}{\hat{{\bf q}}_{2}-\hat{{\bf
q}}_{3}}\sqrt{\frac{\hat{{\bf q}}_{3}}{\hat{{\bf p}}_{3}}}$
$\displaystyle\hat{{\bf c}}_{3}$ $\displaystyle=$
$\displaystyle\frac{1}{\hat{{\bf p}}_{2}-\hat{{\bf
p}}_{3}}\sqrt{\frac{\hat{{\bf p}}_{3}}{\hat{{\bf q}}_{3}}}-\frac{1}{\hat{{\bf
q}}_{2}-\hat{{\bf q}}_{3}}\sqrt{\frac{\hat{{\bf q}}_{3}}{\hat{{\bf p}}_{3}}}$
$\displaystyle\hat{{\bf c}}_{2}$ $\displaystyle=$
$\displaystyle\frac{1}{1-\hat{{\bf p}}_{2}}\frac{\hat{{\bf p}}_{2}-\hat{{\bf
p}}_{3}}{\hat{{\bf q}}_{2}-\hat{{\bf q}}_{3}}\sqrt{\frac{\hat{{\bf
q}}_{3}}{\hat{{\bf p}}_{3}}}-\frac{1}{1-\hat{{\bf q}}_{2}}\frac{\hat{{\bf
q}}_{2}-\hat{{\bf q}}_{3}}{\hat{{\bf p}}_{2}-\hat{{\bf
p}}_{3}}\sqrt{\frac{\hat{{\bf p}}_{3}}{\hat{{\bf q}}_{3}}}.$ (152)

For general $r$:

$\displaystyle\hat{\gamma}_{sq}^{(p)}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\frac{\hat{{\bf q}}_{1}-\hat{{\bf q}}_{2}}{\hat{{\bf
p}}_{1}-\hat{{\bf p}}_{2}}\prod_{k=2:2:r-1}\frac{\hat{{\bf p}}_{k}-\hat{{\bf
p}}_{k+1}}{\hat{{\bf q}}_{k}-\hat{{\bf
q}}_{k+1}}\prod_{k=2:2:r-2}\frac{\hat{{\bf q}}_{k+1}-\hat{{\bf
q}}_{k+2}}{\hat{{\bf p}}_{k+1}-\hat{{\bf p}}_{k+2}}\sqrt{\left(\frac{\hat{{\bf
q}}_{r}}{\hat{{\bf p}}_{r}}\right)^{(-1)^{r+1}}}$ $\displaystyle\hat{{\bf
c}}_{i}$ $\displaystyle=$ $\displaystyle\frac{1}{\hat{{\bf p}}_{i-1}-\hat{{\bf
p}}_{i}}\prod_{k=i:2:r-1}\frac{\hat{{\bf p}}_{k}-\hat{{\bf
p}}_{k+1}}{\hat{{\bf q}}_{k}-\hat{{\bf
q}}_{k+1}}\prod_{k=i:2:r-2}\frac{\hat{{\bf q}}_{k+1}-\hat{{\bf
q}}_{k+2}}{\hat{{\bf p}}_{k+1}-\hat{{\bf p}}_{k+2}}\sqrt{\left(\frac{\hat{{\bf
q}}_{r}}{\hat{{\bf p}}_{r}}\right)^{(-1)^{i}}}$
$\displaystyle-\frac{1}{\hat{{\bf q}}_{i-1}-\hat{{\bf
q}}_{i}}\prod_{k=i:2:r-1}\frac{\hat{{\bf q}}_{k}-\hat{{\bf
q}}_{k+1}}{\hat{{\bf p}}_{k}-\hat{{\bf
p}}_{k+1}}\prod_{k=i:2:r-2}\frac{\hat{{\bf p}}_{k+1}-\hat{{\bf
p}}_{k+2}}{\hat{{\bf q}}_{k+1}-\hat{{\bf q}}_{k+2}}\sqrt{\left(\frac{\hat{{\bf
p}}_{r}}{\hat{{\bf q}}_{r}}\right)^{(-1)^{i}}},\quad\mbox{with}\quad
i\in\\{2,3,\dots,r\\}.$ (153)

###### Proof.

The $r=1$ case follows immediately from (4.2.1), the $r=2$ from (87) and (90),
the $r=3$ from (142), (145), and (147), whereas the general case follows by
repeating the above procedure for an arbitrary $r$. ∎

####  4.2.6 Concrete parameter values

In Table 4, $a_{c}^{(3,f)}(-1.5)$, obtained in (136), is complemented with the
concrete values of all the relevant quantities related to the third _full_
(3-sfl RDT) level of lifting. To enable a systematic view of the lifting
progress, we, in parallel, show the results from Table 1 that contain the same
quantities for the first _full_ (1-sfl RDT), the second _partial_ (2-spf RDT),
and the second _full_ (2-sff RDT) level.

Table 4: $r$-sfl RDT parameters; _negative_ spherical perceptron capacity; $\hat{{\bf c}}_{1}\rightarrow 1$; $\kappa=-1.5$; $n,\beta\rightarrow\infty$ $r$-sfl RDT | $\hat{\gamma}_{sq}$ | $\hat{\gamma}_{sq}^{(p)}$ | $\hat{{\bf p}}_{3}$ | $\hat{{\bf p}}_{2}$ | $\hat{{\bf p}}_{1}$ | $\hat{{\bf q}}_{3}$ | $\hat{{\bf q}}_{2}$ | $\hat{{\bf q}}_{1}$ | $\hat{{\bf c}}_{3}$ | $\hat{{\bf c}}_{2}$ | $\alpha_{c}^{(r)}(\kappa)$
---|---|---|---|---|---|---|---|---|---|---|---
$1$-sfl RDT | $0.5$ | $0.5$ | $0$ | $0$ | $\rightarrow 1$ | $0$ | $0$ | $\rightarrow 1$ | $0$ | $\rightarrow 0$ | $\mathbf{43.77}$
$2$-spl RDT | $0.1737$ | $1.4397$ | $0$ | $0$ | $\rightarrow 1$ | $0$ | $0$ | $\rightarrow 1$ | $0$ | $2.5320$ | $\mathbf{37.36}$
$2$-sfl RDT | $0.1324$ | $1.8884$ | $0$ | $0.4747$ | $\rightarrow 1$ | $0$ | $0.0981$ | $\rightarrow 1$ | $0$ | $3.6835$ | $\mathbf{36.57}$
$3$-sfl RDT | $0.0647$ | $3.8759$ | $0.4075$ | $0.9693$ | $\rightarrow 1$ | $0.0743$ | $0.5384$ | $\rightarrow 1$ | $3.25$ | $12.6$ | $\mathbf{36.40}$

In Table 5, we show the key second level of lifting parameters over a range of
$\kappa$. The progression of the capacity as the level of lifting increases is
shown in Table 6. The systematic showing of the progression in Table 6 (as
well as in in Table 4) allows one to also note, that the first rows in these
tables relate to the results that can be obtained through the _plain_ RDT
(see, e.g., [29]), whereas their second rows relate to the results that can be
obtained through the _partially_ lifted RDT of [33].

Table 5: $3$-sfl RDT parameters; _negative_ spherical perceptron capacity $\alpha=\alpha_{c}^{(3,f)}(\kappa)$ $\kappa$ | $\mathbf{-2.0}$ | $\mathbf{-1.5}$ | $\mathbf{-1.0}$ | $\mathbf{-0.5}$
---|---|---|---|---
$\hat{\gamma}_{sq}$ | $0.0493$ | $0.0647$ | $0.0835$ | $0.0886$
$\hat{\gamma}_{sq}^{(p)}$ | $5.0735$ | $3.8759$ | $3.0237$ | $2.8159$
$\hat{{\bf p}}_{3}$ | $0.2304$ | $0.4075$ | $0.6252$ | $0.8165$
$\hat{{\bf p}}_{2}$ | $0.9821$ | $0.9693$ | $0.9691$ | $0.9821$
$\hat{{\bf q}}_{3}$ | $0.0172$ | $0.0743$ | $0.2500$ | $0.5681$
$\hat{{\bf q}}_{2}$ | $0.5392$ | $0.5384$ | $0.6536$ | $0.8179$
$\hat{{\bf c}}_{3}$ | $4.35$ | $3.25$ | $3.03$ | $3.90$
$\hat{{\bf c}}_{2}$ | $16.4$ | $12.6$ | $12.1$ | $21.0$
$\alpha$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{124.8}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.40}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{12.29}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{4.698}}$
Table 6: _Negative_ spherical perceptron capacity $\alpha_{c}(\kappa)$ — progression of $r$-sfl RDT mechanism $\kappa$ | $\mathbf{-2.0}$ | $\mathbf{-1.5}$ | $\mathbf{-1}$ | $\mathbf{-0.5}$
---|---|---|---|---
$\alpha_{c}^{(1,f)}(\kappa)$ | $173.4$ | $43.77$ | $13.27$ | $4.770$
$\alpha_{c}^{(2,p)}(\kappa)$ | $126.2$ | $37.36$ | $12.78$ | $4.770$
$\alpha_{c}^{(2,f)}(\kappa)$ | $125.4$ | $36.57$ | $12.32$ | $4.701$
$\alpha_{c}^{(3,f)}(\kappa)$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{124.8}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{36.40}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{12.29}}$ | ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{4.698}}$

The obtained results are also visualized in Figures 1 and 2. In Figure 1 a
small $\kappa$ range is shown resulting in not so large scaled capacities (of
order of a few tens). In these regimes the differences between various levels
of lifting are more pronounced. However, as the figure clearly shows, the
convergence is rather remarkably fast. When capacities get larger the relative
differences become even smaller. This is clear from Figure 2, where one can
not make much of a difference between say 2-spl RDT on the one side and 2-sfl
and 3-sfl RDT on the other side. In other words, in the large
$\alpha_{c}(\kappa)$ regimes, the lifted curves are visually indistinguishable
which reconfirms the fact that 2-spl RDT results of [33] are up to the leading
order terms optimal (this was also shown in [19]).

Figure 1: _Negative_ spherical perceptron capacity as a function of $\kappa$

Figure 2: _Negative_ spherical perceptron capacity as a function of $\kappa$;
larger $\alpha_{c}(\kappa)$

####  4.2.7 Modulo-${\bf m}$ sfl RDT

Everything presented above can be repeated relying on the so-called modulo-$m$
sfl RDT frame of [36]. Instead of Theorem 2, one then basically has the
following theorem.

######  Theorem 4.

Assume the setup of Theorem 2 and instead of the complete, assume the
modulo-${\bf m}$ sfl RDT setup of [36]. Let the “fixed” parts of $\hat{{\bf
p}}$, $\hat{{\bf q}}$, and $\hat{{\bf c}}$ satisfy $\hat{{\bf
p}}_{1}\rightarrow 1$, $\hat{{\bf q}}_{1}\rightarrow 1$, $\hat{{\bf
c}}_{1}\rightarrow 1$, $\hat{{\bf p}}_{r+1}=\hat{{\bf q}}_{r+1}=\hat{{\bf
c}}_{r+1}=0$, and let the “non-fixed” parts of $\hat{{\bf p}}_{k}$, and
$\hat{{\bf q}}_{k}$ ($k\in\\{2,3,\dots,r\\}$) be the solutions of the
following system of equations

$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf p}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d{\bf q}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}}=0$
$\displaystyle\frac{d\bar{\psi}_{rd}({\bf p},{\bf q},{\bf
c},\gamma_{sq},\gamma_{sq}^{(p)},1,1,-1)}{d\gamma_{sq}^{(p)}}=0.$ (154)

Consequently, let

$\displaystyle c_{k}(\hat{{\bf p}},\hat{{\bf q}})$ $\displaystyle=$
$\displaystyle\sqrt{\hat{{\bf q}}_{k-1}-\hat{{\bf q}}_{k}}$ $\displaystyle
b_{k}(\hat{{\bf p}},\hat{{\bf q}})$ $\displaystyle=$
$\displaystyle\sqrt{\hat{{\bf p}}_{k-1}-\hat{{\bf p}}_{k}}.$ (155)

Then

$\displaystyle-f_{sq}(\infty)$ $\displaystyle\leq$ $\displaystyle\max_{{\bf
c}}\frac{1}{2}\sum_{k=2}^{r+1}\Bigg{(}\Bigg{.}\hat{{\bf p}}_{k-1}\hat{{\bf
q}}_{k-1}-\hat{{\bf p}}_{k}\hat{{\bf q}}_{k}\Bigg{.}\Bigg{)}{\bf c}_{k}$ (156)
$\displaystyle-\hat{\gamma}_{sq}^{(p)}-\varphi(D_{1}^{(per)}(c_{k}(\hat{{\bf
p}},\hat{{\bf q}})),{\bf
c})+\hat{\gamma}_{sq}-\alpha\varphi(-D_{1}^{(sph)}(b_{k}(\hat{{\bf
p}},\hat{{\bf q}})),{\bf c})=-f_{sq,{\bf m}}(\infty).$

###### Proof.

Follows from the previous discussion, Theorems 1 and 2, Corollary 1, and the
sfl RDT machinery presented in [38, 36, 39]. ∎

We conducted the numerical evaluations using the modulo-${\bf m}$ results of
the above theorem without finding any scenario where the inequality in (156)
is not tight. In other words, we have found that
$f_{sq}^{(r)}(\infty)=f_{sq,{\bf m}}^{(r)}(\infty)$. This indicates that the
_stationarity_ over ${\bf c}$ is actually of the _maximization_ type.

##  5 Conclusion

We studied the statistical capacity of the negative spherical perceptrons
(i.e., the classical spherical perceptron with _negative_ thresholds
$\kappa$). Differently from their positive thresholds counterparts, these
problems belong to the class of hard random structures where standard
analytical approaches are powerless when it comes to approaching the _exact_
capacity characterizations. The random duality (RDT) based results [29]
provided solid generic upper bounds that were substantially improved via the
_partially_ lifted RDT in [33]. A recent breakthroughs in studying bilinearly
indexed random processes [36, 38], enabled [39] to create a _fully lifted_
random duality theory (fl RDT) counterpart to the RDT from [26, 28, 34, 29,
27].

After recognizing the connection between the statistical perceptrons, general
_random feasibility problems_ (rfps), and the bilinearly indexed (bli), we
utilized the fl RDT and its a particular _stationarized_ variant (called sfl
RDT) to establish a general framework for studying the negative spherical
perceptrons. The practical usability of the entire fl RDT machinery relies on
a successful conducting of heavy underlying numerical evaluations. We first
presented a large amount of analytical simplifications that resulted in
uncovering remarkable closed form interconnections among the key lifting
parameters. In addition to providing a direct view into the structure of the
parametric relations, they also greatly helped with the numerical work. In
particular, we obtained concrete numerical results and uncovered a remarkably
rapid convergence of the whole fl RDT mechanism. Over a wide range of
thresholds $\kappa$ (allowing _scaled_ capacities of a few thousands), we
observed that the third (second non-trivial) level of stationarized full
lifting suffices to achieve relative improvements no better than $\sim 0.1\%$.
To ensure that the lifting progress is systematically presented and that the
rapid convergence is clearly visible, we started with the very first level and
then incrementally increased the level of lifting. Such a systematic procedure
also allowed us to deduce as special cases the earlier results obtained
through the _plain_ RDT in [29] and the _partial_ RDT in [33].

The methodology is very generic and various extensions and generalizations are
possible. These include many related to both general _random feasibility
problems_ (rfps) and particular random perceptrons. A lengthy list of random
structures discussed in [38, 36, 39, 39, 40] is an example of a collection of
such problems that can be handled through the methods presented here. As the
technical details are problem specific, we discuss them in separate papers.

As [39, 40] emphasized, the sfl RDT considerations do not require the standard
Gaussianity assumption of the random primals. The Lindeberg variant of the
central limit theorem (see, e.g., [18]) can be utilized to quickly extend the
sfl RDT results to a wide range of different statistics. [7]’s utilization of
the Lindenberg approach is, for example, particularly elegant.

## References

  * [1] E. Abbe, S. Li, and A. Sly.  Binary perceptron: efficient algorithms can find solutions in a rare well-connected cluster.  2021\.  available online at http://arxiv.org/abs/2111.03084.
  * [2] E. Abbe, S. Li, and A. Sly.  Proof of the contiguity conjecture and lognormal limit for the symmetric perceptron.  2021\.  available online at http://arxiv.org/abs/2102.13069.
  * [3] A. E. Alaoui and M. Sellke.  Algorithmic pure states for the negative spherical perceptron.  2020\.  available online at http://arxiv.org/abs/2010.15811.
  * [4] R. Alweiss, Y. P. Liu, and M. Sawhney.  Discrepancy minimization via a self-balancing walk.  In Proc. 53rd STOC, ACM, pages 14–20, 2021.
  * [5] B. Aubin, W. Perkins, and L. Zdeborova.  Storage capacity in symmetric binary perceptrons.  J. Phys. A, 52(29):294003, 2019.
  * [6] E. Bolthausen, S. Nakajima, N. Sun, and C. Xu.  Gardner formula for Ising perceptron models at small densities.  Proceedings of Thirty Fifth Conference on Learning Theory, PMLR, 178:1787–1911, 2022.
  * [7] S. Chatterjee.  A generalization of the Lindenberg principle.  The Annals of Probability, 34(6):2061–2076.
  * [8] T. Cover.  Geomretrical and statistical properties of systems of linear inequalities with applications in pattern recognition.  IEEE Transactions on Electronic Computers, (EC-14):326–334, 1965\.
  * [9] J. Ding and N. Sun.  Capacity lower bound for the Ising perceptron.  STOC 2019: Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 816–827, 2019.
  * [10] S. Franz and G. Parisi.  The simplest model of jamming.  Journal of Physics A: Mathematical and Theoretical, 49(14):145001, 2016.
  * [11] S. Franz, G. Parisi, M. Sevelev, P. Urbani, and F. Zamponi.  Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems.  SciPost Physics, 2:019, 2017.
  * [12] S. Franz, A. Sclocchi, and P. Urbani.  Critical jammed phase of the linear perceptron.  Phys. Rev. Lett., 123(11):115702, 2019.
  * [13] S. Franz, A. Sclocchi, and P. Urbani.  Surfing on minima of isostatic landscapes: avalanches and unjamming transition.  SciPost Physics, 9:12, 2020.
  * [14] D. Gamarnik, E. C. Kizildag, W. Perkins, and C. Xu.  Algorithms and barriers in the symmetric binary perceptron model.  2022\.  available online at http://arxiv.org/abs/2203.15667.
  * [15] E. Gardner.  The space of interactions in neural networks models.  J. Phys. A: Math. Gen., 21:257–270, 1988.
  * [16] E. Gardner and B. Derrida.  Optimal storage properties of neural networks models.  J. Phys. A: Math. Gen., 21:271–284, 1988.
  * [17] W. Krauth and M. Mezard.  Storage capacity of memory networks with binary couplings.  J. Phys. France, 50:3057–3066, 1989.
  * [18] J. W. Lindeberg.  Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung.  Math. Z., 15:211–225, 1922.
  * [19] A. Montanari, Y. Zhong, and K. Zhou.  Tractability from overparametrization: The example of the negative perceptron.  2021\.  available online at http://arxiv.org/abs/2110.15824.
  * [20] S. Nakajima and N. Sun.  Sharp threshold sequence and universality for Ising perceptron models.  Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 638–674, 2023.
  * [21] W. Perkins and C. Xu.  Frozen 1-RSB structure of the symmetric Ising perceptron.  STOC 2021: Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 1579–1588, 2021.
  * [22] M. Shcherbina and B. Tirozzi.  On the volume of the intrersection of a sphere with random half spaces.  C. R. Acad. Sci. Paris. Ser I, (334):803–806, 2002.
  * [23] M. Shcherbina and B. Tirozzi.  Rigorous solution of the Gardner problem.  Comm. on Math. Physics, (234):383–422, 2003.
  * [24] M. Stojnic.  Box constrained $\ell_{1}$ optimization in random linear systems – asymptotics.  available online at http://arxiv.org/abs/1612.06835.
  * [25] M. Stojnic.  Box constrained $\ell_{1}$ optimization in random linear systems – finite dimensions.  available online at http://arxiv.org/abs/1612.06839.
  * [26] M. Stojnic.  Various thresholds for $\ell_{1}$-optimization in compressed sensing.  available online at http://arxiv.org/abs/0907.3666.
  * [27] M. Stojnic.  A simple performance analysis of $\ell_{1}$-optimization in compressed sensing.  ICASSP, International Conference on Acoustics, Signal and Speech Processing, April 2009.
  * [28] M. Stojnic.  $\ell_{1}$ optimization and its various thresholds in compressed sensing.  ICASSP, IEEE International Conference on Acoustics, Signal and Speech Processing, pages 3910–3913, 14-19 March 2010.  Dallas, TX.
  * [29] M. Stojnic.  Another look at the Gardner problem.  2013\.  available online at http://arxiv.org/abs/1306.3979.
  * [30] M. Stojnic.  Discrete perceptrons.  2013\.  available online at http://arxiv.org/abs/1303.4375.
  * [31] M. Stojnic.  Lifting $\ell_{1}$-optimization strong and sectional thresholds.  2013\.  available online at http://arxiv.org/abs/1306.3770.
  * [32] M. Stojnic.  Lifting/lowering Hopfield models ground state energies.  2013\.  available online at http://arxiv.org/abs/1306.3975.
  * [33] M. Stojnic.  Negative spherical perceptron.  2013\.  available online at http://arxiv.org/abs/1306.3980.
  * [34] M. Stojnic.  Regularly random duality.  2013\.  available online at http://arxiv.org/abs/1303.7295.
  * [35] M. Stojnic.  Spherical perceptron as a storage memory with limited errors.  2013\.  available online at http://arxiv.org/abs/1306.3809.
  * [36] M. Stojnic.  Bilinearly indexed random processes – _stationarization_ of fully lifted interpolation.  2023\.  available online at arxiv.
  * [37] M. Stojnic.  Binary perceptrons capacity via fully lifted random duality theory.  2023\.  available online at arxiv.
  * [38] M. Stojnic.  Fully lifted interpolating comparisons of bilinearly indexed random processes.  2023\.  available online at arxiv.
  * [39] M. Stojnic.  Fully lifted random duality theory.  2023\.  available online at arxiv.
  * [40] M. Stojnic.  Studying Hopfield models via fully lifted random duality theory.  2023\.  available online at arxiv.
  * [41] M. Talagrand.  Intersecting random half cubes.  Random Structures Algorithms, 15(3-4):436–449, 1999.
  * [42] M. Talagrand.  The Generic Chaining.  Springer-Verlag, 2005.
  * [43] M. Talagrand.  Mean field models and spin glasse: Volume II.  A series of modern surveys in mathematics 55, Springer-Verlag, Berlin Heidelberg, 2011.
  * [44] M. Talagrand.  Mean field models and spin glasses: Volume I.  A series of modern surveys in mathematics 54, Springer-Verlag, Berlin Heidelberg, 2011.
  * [45] J. G. Wendel.  A problem in geometric probability.  Mathematica Scandinavica, 1:109–111, 1962.
  * [46] R. O. Winder.  Threshold logic.  Ph. D. dissertation, Princetoin University, 1962.