††thanks<EMAIL_ADDRESS>0000-0002-3082-0929

#  Searching for a dark matter induced galactic axion gradient

Edward Hardy<EMAIL_ADDRESS>Rudolf Peierls Centre for
Theoretical Physics, University of Oxford, Parks Road, Oxford OX1 3PU, UK
Mario Reig<EMAIL_ADDRESS>Rudolf Peierls Centre for
Theoretical Physics, University of Oxford, Parks Road, Oxford OX1 3PU, UK
Department of Physics, Royal Holloway University of London, Egham, Surrey,
TW20 0EX, UK.  Juri Smirnov  Department of Mathematical Sciences, University
of Liverpool, Liverpool, L69 7ZL, United Kingdom

###### Abstract

An ultra-light axion with CP violating interactions with a dark sector and CP
preserving interactions with the visible sector can act as a novel portal
between dark matter and the Standard Model. In such theories, dark matter
sources an axion field extending over the entire galaxy, the gradient of which
can be searched for with precise spin precession experiments. A
reinterpretation of existing co-magnetometer data already constrains theories
that are consistent with astrophysical bounds, and near-future experiments
will begin probing well-motivated models. The required interactions can arise
from a confining hidden sector without necessitating fine-tuning of the
axion’s mass.

††preprint: LTH-1373

## Introduction

Despite compelling evidence for its existence, the nature of dark matter (DM)
and its interactions with the Standard Model (SM) remain unknown.
Investigation of the possible portal interactions between DM and the SM is
therefore worthwhile. In this letter we propose a new such portal, in which
the DM and the SM are connected via an ultra-light axion that has CP violating
couplings to the DM. We argue that this possibility is theoretically well-
motivated and can lead to observable signals in table-top experiments.

Axions (by which we mean any light pseudo-scalars) are generic in UV
completions of the SM involving gauge fields in extra space-time dimensions,
with string theory as the foremost example Svrcek and Witten (2006);
Arvanitaki  _et al._ (2010); Conlon (2006), see Reece (2024) for a recent
review. In field-theoretic realisations, an axion can couple to both the SM
and a dark sector as a result of heavy matter charged under gauge groups in
both sectors, while in string theory the couplings of an axion to fermions
depend on the compactification Cicoli  _et al._ (2012).

It is plausible that the interactions of an axion with DM might be CP
violating; the axion itself is not the DM in the scenario we consider. As a
guiding analogy, recall that the QCD axion has minuscule CP violating
interactions with SM particles only because the minimum of its potential is
extremely close to the point where the strong CP angle $\theta_{\rm QCD}=0$.
That $\theta_{\rm QCD}\simeq 0$ is both a quirk of the SM (due to the weak
interactions inducing a non-zero $\theta_{\rm QCD}$ at high loop order
suppressed by powers of $G_{\rm F}\Lambda_{\rm QCD}^{2}\ll 1$ Georgi and
Randall (1986)) and a theoretical problem in that contributions to the axion
potential from the UV must be tiny (the quality problem Georgi  _et al._
(1981); Kamionkowski and March-Russell (1992)). One can imagine hidden sectors
for which similar conditions are not fulfilled and sizable CP violating
interactions arise.

Meanwhile, we require that the axion does not have an anomalous coupling to
QCD. We further assume that the SM is decoupled from the dominant sources of
CP violation Di Luzio  _et al._ (2021), in which case is a reasonable
expectation that an axion only has CP conserving interactions with the SM.

We therefore consider an effective Lagrangian

$\mathcal{L}_{\text{eff}}\supset
g^{\chi}_{s}\phi\bar{\chi}\chi+c_{\psi}\frac{\partial_{\mu}\phi}{f_{\phi}}\bar{\psi}\gamma^{\mu}\gamma^{5}\psi-\frac{1}{2}m_{\phi}^{2}\phi^{2}\,,$
(1)

where $\phi$ is the axion, $f_{\phi}$ is its decay constant (defined such that
the period of the axion field is $2\pi f_{\phi}$), and $m_{\phi}$ is the axion
mass. The field $\psi$ is a generic SM fermion, and $c_{\psi}$ is expected not
to be much bigger than $\mathcal{O}(1)$ in minimal theories. Meanwhile, $\chi$
is the DM, which we assume to be a fermion. In typical UV completions
involving a confining hidden sector with strong coupling scale $\Lambda$, the
axion’s CP violating interactions have strength
$g_{s}^{\chi}\sim\theta\Lambda/f_{\phi}$, where $\theta\in[-\pi,\pi]$. (This
is the form of the QCD axion’s CP violating couplings, up to a mild dependence
on the masses of the light quarks Moody and Wilczek (1984).)

## A galactic axion gradient

As a result of first term in Eq. (1), DM sources an axion field. We consider
axions with masses $m_{\phi}\lesssim 1/({\rm pc})\simeq 10^{-23}~{}{\rm eV}$
such that this induced axion field extends over galactic scales. The equation
of motion of $\phi$ in the background of the DM in our galaxy has a time-
independent solution

$\displaystyle\phi(\vec{r})=g_{s}^{\chi}\int_{V}d^{3}{\vec{r}^{\prime}}\,\frac{n_{\chi}(\vec{r}^{\prime})\,e^{-m_{\phi}|\vec{r}-\vec{r}^{\prime}|}}{4\pi|\vec{r}-\vec{r^{\prime}}|}\,,$
(2)

where $\vec{r}$ is relative to the galactic centre and $n_{\chi}(\vec{r})$ is
the DM number density, which we assume to be time-independent and spherically
symmetric; corrections to this are relatively small. We consider theories for
which effective mass of $\phi$ in the galactic DM background $m_{\rm eff}\geq
m_{\phi}\gtrsim 10^{-30}~{}{\rm eV}$, such that the period of $\phi$
oscillations, $2\pi/m_{\rm eff}$, is fast compared to the galaxy’s formation
timescale and its age. Consequently, we assume that $\phi$ is close to the
solution in Eq. (2) today, as is expected if $\phi$ tracks the minimum of its
changing potential adiabatically in the early universe. We expect that
deviations from this solution, e.g. remnant oscillations, will only strengthen
the signals we consider.

The expression in Eq. (2) simplifies in two limits: First, if the Compton
wavelength of the axion, $1/m_{\phi}$, is shorter than length scale over which
the DM density varies, $\lambda_{\chi}\equiv
n_{\chi}/\left|\boldsymbol{\nabla}n_{\chi}\right|$, the axion field at any
point can be approximated as that sourced from the surrounding
$1/m_{\phi}^{3}$ volume, which contains $N_{\rm eff}\equiv
n_{\chi}/m_{\phi}^{3}$ DM particles Acevedo  _et al._ (2024). As a result,

$\displaystyle\phi(r)\simeq\frac{g_{s}^{\chi}\,N_{\rm
eff}}{\lambda_{\phi}}\simeq\frac{g_{s}^{\chi}\,n_{\chi}(r)}{m_{\phi}^{2}}\,.$
(3)

Second, if the axion’s Compton wavelength is comparable to the scale of the
galaxy or larger, the axion field is well-approximated by the standard
Coulombic potential.

At the Earth’s distance from the galactic centre, $R$, we therefore have an
axion field gradient of

$\displaystyle\left|\boldsymbol{\nabla}\phi(R)\right|\simeq\begin{cases}g_{s}^{\chi}\,\left|\boldsymbol{\nabla}n_{\chi}(R)\right|/m_{\phi}^{2}&\text{if
}1/m_{\phi}<\lambda_{\chi},\\\ g_{s}^{\chi}\,N_{\chi}(R)/R^{2}&\text{if
}1/m_{\phi}>\lambda_{\chi}\,,\end{cases}$ (4)

where $N_{\chi}(R)=4\pi\int_{0}^{R}n_{\chi}(r)r^{2}dr$ is the total number of
DM particles within the Earth’s galactic orbit.

Such an axion gradient affects the SM particles through the second term in Eq.
(1). In the limit that the SM particles are non-relativistic, the
corresponding interaction in the Hamiltonian is (see e.g. Graham  _et al._
(2018); Alonso  _et al._ (2019) for recent reviews)

$H_{\phi}=-\frac{c_{\psi}}{f_{\phi}}\boldsymbol{\nabla}\phi\cdot\mathbf{S}\,,$
(5)

where $\mathbf{S}$ is the spin operator of the SM particle. Consequently, an
axion gradient leads to a spin-dependent energy shift $\Delta E$ in SM states,
analogous to the Zeeman effect. Using Eq. (4), for a particle with spin
aligned with the axion field gradient

$\displaystyle\Delta E$
$\displaystyle\simeq\begin{cases}c_{\psi}g_{s}^{\chi}\left|\mathbf{S}\right|\left|\boldsymbol{\nabla}n_{\chi}(R)\right|/(f_{\phi}m_{\phi}^{2})&\text{if
}1/m_{\phi}<\lambda_{\chi}\,,\\\
c_{\psi}g_{s}^{\chi}\left|\mathbf{S}\right|N_{\chi}(R)/(f_{\phi}R^{2})&\text{if
}1/m_{\phi}>\lambda_{\chi}\,.\end{cases}$ (6)

In other words, there is a monopole-dipole axion force Moody and Wilczek
(1984) with the monopole side from the coupling to DM.

Eq. (6) can easily be evaluated assuming, e.g., an NFW DM profile Navarro  _et
al._ (1996). Fixing the local DM energy density $\rho_{0}\simeq 0.4~{}{\rm
GeV}/{\rm cm}^{3}$ and the scale radius to $17.6\,\rm kpc$, gives

$\displaystyle\Delta E\simeq 3\times 10^{-24}~{}{\rm
eV}\,\kappa\begin{cases}1/\left(\lambda_{\chi}m_{\phi}\right)^{2}&\text{if
}m_{\phi}>1/\lambda_{\chi}\,,\\\ 1&\text{if
}m_{\phi}<1/\lambda_{\chi}\,,\end{cases}$ (7)

with $1/\lambda_{\chi}\simeq 4\times 10^{-28}~{}\text{ eV}$ and

$\displaystyle\kappa=\left|\mathbf{S}\right|\left(\frac{g^{\chi}_{s}}{m_{\chi}/M_{\rm
P}}\right)\left(\frac{10^{9}~{}\text{ GeV}}{f_{\phi}/c_{\psi}}\right)\,,$ (8)

where $M_{\rm P}$ is the (non-reduced) Planck mass. Not surprisingly the
energy shift is largest when $m_{\phi}\lesssim
1/\lambda_{\chi}\sim\left(10\,\rm kpc\right)^{-1}$, such that the induced
axion field is sensitive to the DM distribution throughout the entire galaxy.

## Constraints and Experimental Reach

There are existing, independent, constraints on the interactions of an ultra-
light axion with DM and the SM. CP violating interactions with DM are bounded
because they lead to long-range self-interactions that affect the DM’s
dynamics Kesden and Kamionkowski (2006); Desmond  _et al._ (2019); Bottaro
_et al._ (2023); Bogorad  _et al._ (2023). Parameterising such forces as

$\displaystyle V(r)=-\frac{\alpha\,G_{\rm
N}m_{\chi}^{2}}{r}\,\exp{\left(-r/l\right)}\,,$ (9)

the constraints are on $\alpha$ as a function of $l$, independent of
$m_{\chi}$. In our theories, $l=1/m_{\phi}$ and
$g^{\chi}_{s}=\sqrt{\alpha}\,m_{\chi}/M_{\rm P}$, so the combination
$g_{s}^{\chi}/m_{\chi}$ is bounded. For axion masses of order $10^{-30}~{}{\rm
eV}$, corresponding to super-galactic lengths, these forces must be weaker
than gravity, i.e. $g_{s}^{\chi}/m_{\chi}\lesssim 1/M_{\rm P}$. For larger
axion masses, $m_{\phi}\gtrsim 1/{\rm kpc}$, the dominant constraints are from
observations of the Bullet Cluster and values of $g_{s}^{\chi}/m_{\chi}$ that
are a few order of magnitude larger than $1/M_{\rm P}$ are allowed Bogorad
_et al._ (2023). Meanwhile, the strongest constraints on CP preserving
couplings of light axions to SM fermions come from the evolution of stars; the
limits on couplings to electrons, $c_{e}/f_{\phi}$, and nucleons,
$c_{N}/f_{\phi}$, are both of order $10^{-9}~{}\text{GeV}^{-1}$ Capozzi and
Raffelt (2020); Buschmann  _et al._ (2022), see Caputo and Raffelt (2024) for
a recent review. We also note that CP violating couplings of the axion to the
visible sector (not included in Eq. (1)) must be such that SM monopole-
monopole forces are at least a factor of $10^{10}$ weaker than gravity Bergé
_et al._ (2018).

Remarkably, existing experiments sensitive to spin-dependent energy shifts
from a galactic axion gradient explore parts of parameter space that are not
excluded by the preceding constraints. Unlike measurements of e.g. the muon
magnetic dipole moment, the energy shift due to an axion gradient cannot be
boosted, so what matters is the absolute sensitivity of an experiment, which
makes extremely precise table-top approaches well-suited. Crucially, because
the axion gradient couples only to spin, the relative energy shift $\Delta
E^{(a)}/\Delta E^{(b)}$ of two different atomic or molecular states ($(a)$,
$(b)$) can differ from that due to a magnetic field. This is exploited by so-
called co-magnetometers, which use two or more types of fermionic spins to
distinguish new-physics energy shifts from uncontrolled magnetic field
backgrounds (the precise conditions for general atomic and molecular states
are given in Agrawal  _et al._ (2023a)). Co-magnetometry has been used in the
past to search for electric dipole moments, axion forces, and more exotic
signals, see Safronova  _et al._ (2018); Terrano and Romalis (2022) for
reviews.

The radial orientation of the axion gradient relative to the galactic centre
offers additional experimental opportunities. For a fixed experiment, it makes
the induced energy shift a pseudo-DC signal with daily modulation due to the
Earth’s rotation. This feature of the signal is different to an axion gradient
generated by the Earth Moody and Wilczek (1984); Venema  _et al._ (1992);
Heckel  _et al._ (2008), see Fedderke  _et al._ (2024); Agrawal  _et al._
(2023b, a); Davoudiasl and Szafron (2022) for recent discussions and Crescini
_et al._ (2017, 2022); Arvanitaki and Geraci (2014) for related studies of
forces from test masses. Moreover, by rotating an experiment additional
modulation can be induced. In case of a signal, the sharp prediction of the
orientation of the axion gradient would allow discrimination from possible
neglected backgrounds or other new physics sources.

Out of the current experiments, those of Refs. Brown  _et al._ (2010) and
Smiciklas  _et al._ (2011) are among the most sensitive to a galactic axion
gradient. These use co-magnetometers based on K-3He and 21Ne-Rb-K to reach
frequency resolutions at the level of $0.7~{}{\rm nHz}$ and $0.5~{}{\rm nHz}$
respectively, corresponding to energies of order $10^{-24}~{}{\rm eV}$. Such
experiments have been used to search for Lorentz violation of extra-solar
origin (e.g. as in the Kostelecky SM extension Colladay and Kostelecký
(1998)), making use of a rotating platform to help remove backgrounds and
systematics. The resulting limits can immediately be reinterpreted as
constraints on our theories. Moreover, the experiment based on 21Ne-Rb-K is
particularly promising for future improvements, having obtained a sensitivity
similar to the K-3He experiment with a factor of $8$ less integration time.
Indeed, in Ref. Brown  _et al._ (2010), it is mentioned that with future
upgrades, resolution at the level of $10^{-3}~{}{\rm nHz}$ might be achieved.
There are also several other experiments and proposals with sensitivities to
energy shifts at the level of $\mathcal{O}({\rm nHz})$ Alexander  _et al._
(2022); Suleiman  _et al._ (2023); Brandenstein  _et al._ (2023).

## Underlying theories

So far we have taken Eq. (1) at face value, however this can miss important
effects that arise in a complete theory. In particular, such a Lagrangian is
expected to appear as the first terms in an expansion in an axion field
$\phi\ll f_{\phi}$, and higher order terms will be relevant if values
$\phi\simeq f_{\phi}$ are induced in the galaxy. At this point, given that
axions have a compact field range, the interaction
$g_{s}^{\chi}\phi\bar{\chi}\chi$ should be replaced by
$h(\phi/f_{\phi})\bar{\chi}\chi$ where $h$ is a function with period $2\pi$
(similarly, the axion mass will be replaced by a periodic potential).

It can immediately be seen that the compactness of the axion field is going to
be borderline relevant for values of $g_{s}^{\chi}$ that lead to detectable
energy shifts. Regardless of the axion mass, the maximum galactic axion field
gradient possible without the compactness of the axion field being an issue is
$|\boldsymbol{\nabla}\phi|\lesssim f_{\phi}/R$ such that

$\Delta E\lesssim\frac{c_{\psi}}{f_{\phi}}\frac{f_{\phi}}{R}\simeq
c_{\psi}10^{-27}~{}{\rm eV}\,.$ (10)

For $c_{\psi}\sim\mathcal{O}(1)$ this is a couple of orders of magnitude below
current sensitivities and might be achievable in the future. Assuming
$m_{\phi}\lesssim 1/\lambda_{\chi}$ for simplicity, using Eq. (2), in the
Milky Way $\phi\sim f_{\phi}$ is reached for

$g_{s}^{\chi}=10^{-26}\left(\frac{m_{\chi}}{{\rm
MeV}}\right)\left(\frac{f_{\phi}}{10^{9}~{}{\rm GeV}}\right)~{},$ (11)

which is consistent with DM-DM self-interaction constraints for
$f_{\phi}\simeq 10^{9}~{}{\rm GeV}$. With the parameterization
$g_{s}^{\chi}=\theta\Lambda/f_{\phi}$, Eq. (11) corresponds to

$\theta=10^{-4}\left(\frac{10^{-4}~{}{\rm
eV}}{\Lambda}\right)\left(\frac{m_{\chi}}{{\rm
MeV}}\right)\left(\frac{f_{\phi}}{10^{9}~{}{\rm GeV}}\right)^{2}~{},$ (12)

i.e. only a small CP violating angle in the hidden sector if $m_{\chi}$ is not
too large. We explain the choice of normalisation of $\Lambda$ shortly.

In the case of larger $g_{s}^{\chi}$, so additional operators are relevant,
the induced axion field typically saturates at values of order $f_{\phi}$
rather than continuing to grow. Fields $\phi\gg f_{\phi}$ are not forbidden by
the axion field range’s compact nature; this would simply correspond to the
axion winding its fundamental domain. Instead, saturation occurs because
$\phi$ reaches a value such that $\partial_{\phi}h(\phi/f_{\phi})=0$ at which
point the coupling that sources a $\phi$ field turns off, effectively self-
screening the DM. For instance, a typical form of the full axion potential
might be (further details are given in Appendix A)

$V\supset-\Lambda^{4}\cos\left(\phi/f_{\phi}\right)+\bar{\chi}\chi\Lambda\cos(\phi/f_{\phi}+\theta)~{}.$
(13)

Expanded at small $\phi/f_{\phi}$ the second term leads to a CP violating
coupling $\sin(\theta)\frac{\Lambda}{f_{\phi}}\phi\,\bar{\chi}\chi$. As
$\phi/f_{\phi}\simeq\pi-\theta$ is approached in the galaxy, a non-zero
$\bar{\chi}\chi$ background contributes less and less to the equation of
motion of $\phi$. Eventually, the induced galactic axion field saturates once
$\phi/f_{\phi}\simeq\pi-\theta$ is reached. We also note that, if $\theta\ll
1$, as the background $\phi/f_{\phi}$ increases the effective coupling of
$\phi$ to $\chi$ is first enhanced, reaching a maximum strength at
$\phi/f_{\phi}\simeq\pi/2-\theta$, before being screened.

We therefore identify $\Delta E\lesssim 10^{-27}~{}{\rm eV}$, arising from
minimal theories, as a well-motivated target for future experiments. However,
we also note there are various ways that larger $\Delta E$ can arise from
consistent UV theories. Most simply, values $c_{\psi}\sim 100$ are possible in
normal models. Meanwhile, more unusual UV completions such as clockwork
theories Kaplan and Rattazzi (2016); Farina  _et al._ (2017); Agrawal  _et
al._ (2018) can lead to effective values of $c_{\psi}$ that are exponentially
large. Finally, if the axion potential has a monodromy Silverstein and
Westphal (2008); McAllister  _et al._ (2010); Kaloper and Sorbo (2009) the
induced axion field might not saturate at values of order $f_{\phi}$.

Another question that depends on the particular theory underlying Eq. (1) is
whether an extremely small axion mass, $m_{\phi}\sim 10^{-27}~{}{\rm eV}$, is
possible without fine tuning. If $g_{s}^{\chi}$ indeed arises from a hidden
sector gauge group running into strong coupling at scale $\Lambda$, a typical
expectation is $m_{\phi}\sim\Lambda^{2}/f_{\phi}$ (a smaller $m_{\phi}$ is
possible if there are light hidden sector chiral fermions). To obtain
$m_{\phi}\sim 10^{-26}~{}{\rm eV}$ for $f_{\phi}\sim 10^{9}~{}{\rm GeV}$ then
requires $\Lambda\lesssim 10^{-4}~{}{\rm eV}$.

Small values of $\Lambda$ are not necessarily problematic, although in the
early universe the hidden sector must be colder than the SM to avoid bounds on
new relativistic degrees of freedom Aghanim  _et al._ (2018). However,
observations require that fermionic DM must have a mass much larger than such
$\Lambda$ Tremaine and Gunn (1979a); Boyarsky  _et al._ (2009). This is also
needed so that the galactic DM number density is smaller than $\Lambda^{3}$
and the hidden sector is not locally deconfined. Moreover, $m_{\chi}>\Lambda$
guarantees that the DM mass induced by a background $\phi\lesssim f_{\phi}$ is
smaller than the bare DM mass, both for $g_{s}=\theta\Lambda/f_{\phi}$ (with
$\theta\lesssim 1$) and the example completion in Eq. (13).

Whether a DM candidate with a mass larger than $\Lambda$ gets substantial CP
violating interactions with the axion is model-dependent. In Appendix B, we
describe an example theory in which DM is a particle in the adjoint of a
hidden sector gauge group that forms bound states with dark gluons, as studied
in Ref. Falkowski  _et al._ (2009); Boddy  _et al._ (2014a, b); Contino  _et
al._ (2019). Provided the mass of the adjoint field is greater than roughly a
${\rm keV}$ such a state can be a viable DM candidate, with the required CP
violating interactions arising from the gluon part of the bound state.

## Results

In Figure 1, we show the parameter space of theories with a galactic axion
gradient in the plane of $(c_{\psi}g_{s}^{\chi})/(f_{\phi}m_{\chi})$ against
$m_{\phi}$. In this, we plot the existing constraints from astrophysics by
saturating the allowed values of $c_{\phi}/f_{\phi}$ and
$g_{s}^{\chi}/m_{\chi}$ separately. Meanwhile, as can be seen from Eqs.(7) and
(8), the induced spin-dependent energy shift of SM fields depends on the
couplings via the combination plotted. We show our new limit, obtained by
reinterpreting the results of Ref. Brown  _et al._ (2010), and also the reach
of possible future experiments with plausible improvements in sensitivity to
$\Delta E$. These curves are obtained from the analytic results in Eq. (7); in
reality the transition around $m_{\phi}\simeq 1/\lambda_{\chi}$ will be
smooth.

We also indicate in the same Figure the part of parameter space for which the
galactic axion field induced by Eq. (1) is self-consistently such that the
compactness of the axion field can be neglected. In particular, we fix
$g_{s}^{\chi}/m_{\chi}$ such that the $\phi=f_{\phi}$ at the galactic centre
(assuming the underlying theory is such that a $\phi$ background does not
self-enhance the $\phi$–$\chi$ coupling, as is e.g. the case for $\theta\simeq
1$ in Eq. (13)). Given that the induced $\phi$ is proportional to
$g_{s}^{\chi}/m_{\chi}$, the maximum allowed $g_{s}^{\chi}/m_{\chi}$ is
proportional to $f_{\phi}$, such that the combination
$g_{s}^{\chi}c_{\psi}/(f_{\phi}m_{\chi})$ is independent of $f_{\phi}$, but
depends on $m_{\phi}$ and $c_{\psi}$. The value of $c_{\psi}$ is model
dependent and the maximum typical value not sharply determined, so we blur the
edge of this region over the range corresponding to $1<c_{\psi}<100$. We
stress that values of $g_{s}^{\chi}$ that do not saturate $\phi\simeq
f_{\phi}$ (and smaller $c_{\psi}$) are equally plausible and lead to theories
within the blue “Expected in Minimal Models” region.

For $m_{\phi}\lesssim 10^{-24}~{}{\rm eV}$, existing and future experiments
can probe sizable parts of parameter space of the effective Lagrangian in Eq.
(1) that are not otherwise excluded. As expected, for fixed
$(c_{\psi}g_{s}^{\chi})/(f_{\phi}m_{\chi})$ the signal is weaker at larger
axion masses, but the observational constraints also weaken. Although beyond
current experimental sensitivity, future experiments will begin to explore the
parameter space for which the galactic axion field is smaller than $f_{\phi}$
and the effective Lagrangian Eq. (1) is valid even for the simplest underlying
theories. We also reiterate that more exotic theories (with effective
$c_{\psi}\gg 1$) can lie in the part of parameter space above this that is
already experimentally tested. Finally, we note that for all $m_{\phi}$ and
$f_{\phi}$ considered, the DM relic abundance of $\phi$ itself is indeed
negligible assuming misalignment production.

Figure 1: Existing constraints (labelled “Astrophysics”) and our new limit
(labelled “Limit K-3He”; obtained by reinterpreting the results of Ref. Brown
_et al._ (2010)) on theories that lead to a galactic axion gradient. Results
are shown as a function of the parameters in the effective Lagrangian in Eq.
(1). We also plot the reach of plausible future experiments with sensitivity
to spin-dependent energy shifts of $\Delta E=4\times 10^{-26}~{}{\rm eV}$ and
$4\times 10^{-27}~{}{\rm eV}$. Finally, we indicate the region of parameter
space in which Eq. (1) is self-consistent in minimal UV completions (labelled
“Expected in Minimal Models”).

## Discussion

The theories we have studied have an elegant UV interpretation: It is
plausible that a typical string theory compactification leads to multiple
light axions and several dark sectors. Some of the axions might have CP
violating couplings to the dark sectors. One of these dark sectors could host
the DM and would communicate with the visible sector through a long-range
axion force in the form of a galactic axion gradient. Ultra-precise table top
experiments already have impressive sensitivity to this scenario and orders-
of-magnitude improvement is possible in the future. Such experimental searches
are “looking under a lamppost”, in that a theory need not be in the upper part
of the “minimal theories” parameter space shaded in Figure 1 that can be
explored in the near-future. Nevertheless, given that such technologies are
being developed for independent purposes it is valuable to know that
experimental results can be reinterpreted as a search for DM. Moreover, the
prediction that the energy shift is maximised when the SM particle’s spin is
aligned with the galactic centre (typically not expected for other new-physics
signals) might allow for simple experimental adaptions to increase
sensitivity, e.g. by reorientating existing apparatus.

Although the DM CP violating/ SM CP preserving axion portal that we have
considered is novel, we note that related ideas have been analysed in the
past. Experiments searching for monopole-dipole forces where both parts of the
interaction correspond to SM fermions have been performed Lee  _et al._
(2018); Crescini  _et al._ (2022) and there are promising prospects for the
near future Arvanitaki and Geraci (2014); Alexander  _et al._ (2022); Agrawal
_et al._ (2023b, a); Suleiman  _et al._ (2023); Brandenstein  _et al._ (2023);
Fan and Reig (2023). There has also recently been work analysing the effect of
scalar-scalar interactions between the SM and DM on trapping of dark matter in
celestial bodies Acevedo  _et al._ (2024).

We have given an example, simple, UV completion in which the DM is a bound
state of a dark fermion and a dark gluon. However, there are likely to be
other models that also reduce to Eq. (1) at low energies, and it would be
interesting to explore whether these can lead to new phenomenology or signals.
For example, there might be suitable theories in which the DM is a weakly
coupled bound state of the type $\chi=\mathcal{QQ}$ (where $\mathcal{Q}$ is a
heavy hidden sector quark) similar to bottomonium in QCD. There might also be
UV completions in which the induced axion field is not screened if $\phi\sim
f_{\phi}$ is reached in the galaxy.

We have focused on the time-independent solution for the sourced $\phi$ field,
Eq. (2), which we expect to be, approximately, reached in the Milky Way after
a full cosmological history. However, it would be interesting if $\phi$ was
not fully relaxed to this form yet, e.g. due to oscillations left over from
galactic formation, or had time-dependence, e.g. due to the dynamics of DM
substructure within the galaxy. As mentioned, such effects could lead to
stronger signals, and it would be interesting to investigate them in the
future. More generally, we have conservatively applied constraints on long
range DM-DM forces ignoring the periodicity of the axion potential, and it
would be interesting to reanalyse these. Additionally, as mentioned in Ref.
Bogorad  _et al._ (2023), it would be interesting to investigate whether the
cosmological effects on ${\rm kpc}$ range DM-DM self-interactions, similarly
to the analysis carried out in Refs.Gradwohl and Frieman (1992); Nusser  _et
al._ (2005); Hellwing and Juszkiewicz (2009); Bottaro  _et al._ (2023) for
forces with longer range. These might lead to stronger constraints than the
existing constraints we have made use of as well as possible complementary
signals.

###### Acknowledgements.

We thank Prateek Agrawal for collaboration at early stages of this project as
well as for very useful discussions. We also thank Junwu Huang and Surjeet
Rajendran for useful discussions, and Javier Fernandez Acevedo and Junwu Huang
for useful comments on a draft. EH acknowledges the UK Science and Technology
Facilities Council for support through the Quantum Sensors for the Hidden
Sector collaboration under the grant ST/T006145/1 and UK Research and
Innovation Future Leader Fellowship MR/V024566/1. MRs work is supported by the
STFC grant ST/T006242/1. MR thanks the CERN Theory Department, where this work
was finished, for hospitality.

##  Appendix A Compactness of the axion field

As described in the main text, the fact that the axion couplings are periodic
means that the DM is automatically screened and the induced galactic axion
field saturates once values $\phi\sim f_{\phi}$ are reached (rather than
winding the fundamental domain and growing indefinitely). To see this
explicitly, we consider an axion potential motivated by that of the QCD axion,
including its CP violating couplings to quarks.

Suppose that the axion potential is generated by a hidden sector gauge group
running into strong coupling in the IR. In the UV, we assume the axion is
coupled to the dark gluons as:

$\frac{\alpha_{D}}{f_{\phi}}\phi G_{D}\tilde{G}_{D}\,.$ (14)

Under quite general conditions, the minimum of the contribution to the
potential resulting from strong coupling is CP conserving Vafa and Witten
(1984). In the absence of light chiral fermions in the hidden sector (which
could lead to a chiral suppression of the axion mass, analogous to the QCD
axion mass’ dependence on the light quark masses), the resulting axion mass is
of order $m_{\phi}\sim\Lambda^{2}/f_{\phi}$.

Let us moreover assume that there is an additional CP violating contribution
to the axion’s potential, associated with a scale $\Lambda^{\prime}$, that
shifts the minimum slightly. Temporarily changing conventions such that
$\phi=0$ is the CP conserving point, the resulting axion potential
(considering a simple analytic functional form; more complicated potentials
lead to the same effects) is Moody and Wilczek (1984); Bertolini  _et al._
(2021)

$V=-\Lambda^{4}\cos\left(\frac{\phi}{f_{\phi}}\right)-\Lambda^{\prime
4}\cos\left(\frac{\phi}{f_{\phi}}+\delta\right)~{}.$ (15)

Analogous to the QCD axion, if the DM is charged under the dark gauge group we
expect a CP violating coupling of the axion to dark matter of the form

$V\supset\Lambda\bar{\chi}\chi\cos\left(\frac{\phi}{f_{\phi}}\right)~{}.$ (16)

In more detail, a coupling of simply $\Lambda$ in Eq. (16) is expected for DM
with a mass of order $\Lambda$ and for the quark-gluon bound state described
in the main text. For a $\mathcal{QQ}$ bound state with mass larger than
$\Lambda$ the coupling is expected to take a more complicated form. If there
is a chiral suppression of the axion mass, the strength of the interaction in
Eq. (16) is typically also suppressed (an analogous dependence on $m_{u}m_{d}$
appears in the QCD axion’s CP violating interactions).

Redefining $\phi\mapsto\phi-\theta$ with $\theta$ such that (with
$\left<\bar{\chi}\chi\right>=0$) the minimum of the potential is at $\phi=0$,
the matter part of the potential takes the form

$V\supset\Lambda\bar{\chi}\chi\cos(\phi/f_{\phi}+\theta)~{}.$ (17)

Expanded at small $\phi/f_{\phi}$ this leads to the required CP violating
coupling:

$\mathcal{L}\supset\sin(\theta)\Lambda/f_{\phi}\phi\,\bar{\chi}\chi\,.$ (18)

From Eq. (17) it can be seen that as $\phi/f_{\phi}\simeq\pi-\theta$ is
approached, a non-zero $\bar{\chi}\chi$ background contributes less and less
to the equation of motion of $\phi$ due to the non-linearities. Numerical,
time-independent, solutions of $\phi$’s equation of motion in the background
of a static galactic DM distribution (with boundary condition $\phi=0$ at
spatial infinity) indeed shown that for $\Lambda/f_{\phi}$ sufficiently large
that $\phi\sim f_{\phi}$ is reached, the axion field indeed saturates.

This is consistent with the results in Hook and Huang (2018) (see also Balkin
_et al._ (2022, 2023)), where it is shown that for a light QCD axion – that
is, an axion coupled to QCD with a potential that is suppressed with respect
to the standard prediction – the finite density effects inside neutron stars
can make the axion field to sit near $a/f_{a}=\pi$. In this case, the shift in
axion field value is only due to the finite density potential, which has a
relative $\pi$ phase with respect to the vacuum potential, and is much more
relevant than the CPV contributions for the QCD axion.

##  Appendix B The axion mass and a model of dark matter

Give the extremely light axions that we have considered, it is interesting to
see that this can be achieved without fine-tuning, although of course one can
simply disregard fine-tuning arguments, which opens up a wide range of
possible DM models. Assuming, as before, that the axion potential is
dominantly generated by a hidden sector running into strong coupling, this
requires $\Lambda\lesssim 10^{-3}~{}{\rm eV}$ assuming $f_{\phi}\sim
10^{9}~{}{\rm GeV}$. It also requires that there are no large contributions to
the axion’s potential from the high energy scale; this is a UV dependent issue
analogous to the QCD axion quality problem, which we do not consider further.

Insisting on such small values of $\Lambda$, restricts the range of
phenomenologically viable DM candidates that can also have sizable CP
violating interactions with the axion. In particular, for fermionic DM the
Tremaine-Gunn bound Tremaine and Gunn (1979b) requires $m_{\chi}\gtrsim{\rm
keV}\gg\Lambda$. Additionally, to avoid the hidden sector gauge group being in
the deconfined phase in the galaxy requires the typical inter-DM spacing
$n_{\chi}^{-1/3}\gg\Lambda^{-1}$. Given that the DM energy density in the
galaxy is of order $10^{-6}~{}{\rm eV}^{4}\gg\Lambda^{4}$ this again requires
a DM mass much larger than $\Lambda$.

DM candidates with a mass much larger $\Lambda$ might naively be expected to
have CP violating axion couplings that are strongly suppressed, and therefore
do not lead to a large enough galactic axion gradient to obtain a detectable
signal, but this is not necessarily the case. As an example, suppose that the
hidden sector contains a dark fermion with a mass much larger than the dark
confinement scale, $m_{\mathcal{Q}}\gg\Lambda$, in the adjoint representation,
$\mathcal{Q}\sim\text{Adj}$. Then the DM $\chi$ can consist of bound states of
the fermion, $\mathcal{Q}$, and a dark gluon, $g$. This kind of dark matter
candidate has been proposed in Ref. Falkowski  _et al._ (2009); Boddy  _et
al._ (2014a, b); Contino  _et al._ (2019), see also De Luca  _et al._ (2018)
for a related discussion of a similar DM candidate charged under the SM
$SU(3)$.

The mass of such DM bound states, $\chi\sim\mathcal{Q}g$, is dominated by the
constituent quark $m_{\chi}=m_{\mathcal{Q}}$, while its size is set by the
confinement scale, $r\sim\Lambda^{-1}\gg m_{\mathcal{Q}}^{-1}$. This implies
that despite $\rho_{DM}\gg\Lambda^{4}$, the typical occupancy number is low
and, for sufficiently large $m_{\chi}$, we will be in the confined phase of
the theory provided $m_{\chi}\gg{\rm keV}$ for $\Lambda=10^{-3}~{}{\rm eV}$.
CP violating interactions between the DM and the axion are dominantly induced
by the mixing between the axion and the lightest CP-even glueball. First, note
that the the mixing angle between the axion and the lightest CP-odd glueball
can be estimated in analogy with the axion-pion mixing and is of order
$\Lambda/f_{\phi}$. Then, in the presence of a non-zero $\theta$, the CP-odd
and CP-even glueballs will mix, which leads to an effective CPV coupling of
the axion to the DM bound state of order
$g_{s}^{\chi}\sim\theta\Lambda/f_{\phi}$, which is the parametric form assumed
in the main text.

Finally, we note that even assuming thermal production, the calculation of the
relic abundance in this scenario is complicated, involving many different
regimes and effects as well as non-perturbative dynamics Contino  _et al._
(2019). Moreover, it is possible that the relic abundance is set by an initial
DM asymmetry. Consequently, for our present work, we do not attempt to analyse
the full cosmological history or specify a complete theory that gives the
correct DM relic abundance consistent with all observational constraints. We
do however note that for small $\Lambda$ it is likely to be required that the
dark sector is colder than the visible sector in the early universe. This is
needed both to satisfy observational constraints on additional relativistic
degrees of freedom, and such that the dark sector is in a confined phase when
the cosmic microwave background forms so that the dark matter does not have
strong, long-range, self-interactions at these times.

## References

  * Svrcek and Witten (2006) Peter Svrcek and Edward Witten, “Axions In String Theory,” JHEP 06, 051 (2006), arXiv:hep-th/0605206 [hep-th] .
  * Arvanitaki  _et al._ (2010) Asimina Arvanitaki, Savas Dimopoulos, Sergei Dubovsky, Nemanja Kaloper, and John March-Russell, “String Axiverse,” Phys. Rev. D81, 123530 (2010), arXiv:0905.4720 [hep-th] .
  * Conlon (2006) Joseph P. Conlon, “The QCD axion and moduli stabilisation,” JHEP 05, 078 (2006), arXiv:hep-th/0602233 .
  * Reece (2024) Matthew Reece, “Extra-Dimensional Axion Expectations,” (2024), arXiv:2406.08543 [hep-ph] .
  * Cicoli  _et al._ (2012) Michele Cicoli, Mark Goodsell, and Andreas Ringwald, “The type IIB string axiverse and its low-energy phenomenology,” JHEP 10, 146 (2012), arXiv:1206.0819 [hep-th] .
  * Georgi and Randall (1986) Howard Georgi and Lisa Randall, “Flavor Conserving CP Violation in Invisible Axion Models,” Nucl. Phys. B 276, 241–252 (1986).
  * Georgi  _et al._ (1981) Howard M. Georgi, Lawrence J. Hall, and Mark B. Wise, “Grand Unified Models With an Automatic Peccei-Quinn Symmetry,” Nucl. Phys. B 192, 409–416 (1981).
  * Kamionkowski and March-Russell (1992) Marc Kamionkowski and John March-Russell, “Planck scale physics and the Peccei-Quinn mechanism,” Phys.Lett. B282, 137–141 (1992), arXiv:hep-th/9202003 [hep-th] .
  * Di Luzio  _et al._ (2021) Luca Di Luzio, Ramona Gröber, and Paride Paradisi, “Hunting for $CP$-violating axionlike particle interactions,” Phys. Rev. D 104, 095027 (2021), arXiv:2010.13760 [hep-ph] .
  * Moody and Wilczek (1984) J. E. Moody and Frank Wilczek, “New macroscopic forces?” Phys. Rev. D30, 130 (1984).
  * Acevedo  _et al._ (2024) Javier F. Acevedo, Rebecca K. Leane, and Juri Smirnov, “Evaporation barrier for dark matter in celestial bodies,” JCAP 04, 038 (2024), arXiv:2303.01516 [hep-ph] .
  * Graham  _et al._ (2018) Peter W. Graham, David E. Kaplan, Jeremy Mardon, Surjeet Rajendran, William A. Terrano, Lutz Trahms, and Thomas Wilkason, “Spin Precession Experiments for Light Axionic Dark Matter,” Phys. Rev. D 97, 055006 (2018), arXiv:1709.07852 [hep-ph] .
  * Alonso  _et al._ (2019) Rodrigo Alonso, Diego Blas, and Peter Wolf, “Exploring the ultra-light to sub-MeV dark matter window with atomic clocks and co-magnetometers,” JHEP 07, 069 (2019), arXiv:1810.00889 [hep-ph] .
  * Navarro  _et al._ (1996) Julio F. Navarro, Carlos S. Frenk, and Simon D. M. White, “The Structure of cold dark matter halos,” Astrophys. J. 462, 563–575 (1996), arXiv:astro-ph/9508025 .
  * Kesden and Kamionkowski (2006) Michael Kesden and Marc Kamionkowski, “Galilean Equivalence for Galactic Dark Matter,” Phys. Rev. Lett. 97, 131303 (2006), arXiv:astro-ph/0606566 .
  * Desmond  _et al._ (2019) Harry Desmond, Pedro G. Ferreira, Guilhem Lavaux, and Jens Jasche, “The Fifth Force in the Local Cosmic Web,” Mon. Not. Roy. Astron. Soc. 483, L64–L68 (2019), arXiv:1802.07206 [astro-ph.CO] .
  * Bottaro  _et al._ (2023) Salvatore Bottaro, Emanuele Castorina, Marco Costa, Diego Redigolo, and Ennio Salvioni, “Unveiling dark forces with the Large Scale Structure of the Universe,” (2023), arXiv:2309.11496 [astro-ph.CO] .
  * Bogorad  _et al._ (2023) Zachary Bogorad, Peter W. Graham, and Harikrishnan Ramani, “Coherent Self-Interactions of Dark Matter in the Bullet Cluster,” (2023), arXiv:2311.07648 [hep-ph] .
  * Capozzi and Raffelt (2020) Francesco Capozzi and Georg Raffelt, “Axion and neutrino bounds improved with new calibrations of the tip of the red-giant branch using geometric distance determinations,” Phys. Rev. D 102, 083007 (2020), arXiv:2007.03694 [astro-ph.SR] .
  * Buschmann  _et al._ (2022) Malte Buschmann, Christopher Dessert, Joshua W. Foster, Andrew J. Long, and Benjamin R. Safdi, “Upper Limit on the QCD Axion Mass from Isolated Neutron Star Cooling,” Phys. Rev. Lett. 128, 091102 (2022), arXiv:2111.09892 [hep-ph] .
  * Caputo and Raffelt (2024) Andrea Caputo and Georg Raffelt, “Astrophysical Axion Bounds: The 2024 Edition,” PoS COSMICWISPers, 041 (2024), arXiv:2401.13728 [hep-ph] .
  * Bergé  _et al._ (2018) Joel Bergé, Philippe Brax, Gilles Métris, Martin Pernot-Borràs, Pierre Touboul, and Jean-Philippe Uzan, “MICROSCOPE Mission: First Constraints on the Violation of the Weak Equivalence Principle by a Light Scalar Dilaton,” Phys. Rev. Lett. 120, 141101 (2018), arXiv:1712.00483 [gr-qc] .
  * Agrawal  _et al._ (2023a) Prateek Agrawal, Nicholas R. Hutzler, David E. Kaplan, Surjeet Rajendran, and Mario Reig, “Searching for axion forces with spin precession in atoms and molecules,” (2023a), arXiv:2309.10023 [hep-ph] .
  * Safronova  _et al._ (2018) M. S. Safronova, D. Budker, D. DeMille, Derek F. Jackson Kimball, A. Derevianko, and C. W. Clark, “Search for New Physics with Atoms and Molecules,” Rev. Mod. Phys. 90, 025008 (2018), arXiv:1710.01833 [physics.atom-ph] .
  * Terrano and Romalis (2022) W. A. Terrano and M. V. Romalis, “Comagnetometer probes of dark matter and new physics,” Quantum Sci. Technol. 7, 014001 (2022), arXiv:2106.09210 [hep-ph] .
  * Venema  _et al._ (1992) B. J. Venema, P. K. Majumder, S. K. Lamoreaux, B. R. Heckel, and E. N. Fortson, “Search for a coupling of the earth’s gravitational field to nuclear spins in atomic mercury,” Phys. Rev. Lett. 68, 135–138 (1992).
  * Heckel  _et al._ (2008) B. R. Heckel, E. G. Adelberger, C. E. Cramer, T. S. Cook, S. Schlamminger, and U. Schmidt, “Preferred-frame and $cp$-violation tests with polarized electrons,” Phys. Rev. D 78, 092006 (2008).
  * Fedderke  _et al._ (2024) Michael A. Fedderke, Jedidiah O. Thompson, Raphael Cervantes, Bianca Giaccone, Roni Harnik, David E. Kaplan, Sam Posen, and Surjeet Rajendran, “Measuring axion gradients with photon interferometry,” Phys. Rev. D 109, 015025 (2024), arXiv:2304.11261 [hep-ph] .
  * Agrawal  _et al._ (2023b) Prateek Agrawal, David E. Kaplan, On Kim, Surjeet Rajendran, and Mario Reig, “Searching for axion forces with precision precession in storage rings,” Phys. Rev. D 108, 015017 (2023b), arXiv:2210.17547 [hep-ph] .
  * Davoudiasl and Szafron (2022) Hooman Davoudiasl and Robert Szafron, “Muon g-2 and a Geocentric New Field,” (2022), arXiv:2210.14959 [hep-ph] .
  * Crescini  _et al._ (2017) N. Crescini, C. Braggio, G. Carugno, P. Falferi, A. Ortolan, and G. Ruoso, “Improved constraints on monopole-dipole interaction mediated by pseudo-scalar bosons,” Phys. Lett. B 773, 677–680 (2017), arXiv:1705.06044 [hep-ex] .
  * Crescini  _et al._ (2022) N. Crescini, G. Carugno, P. Falferi, A. Ortolan, G. Ruoso, and C. C. Speake, “Search of spin-dependent fifth forces with precision magnetometry,” Phys. Rev. D 105, 022007 (2022), arXiv:2011.07100 [hep-ex] .
  * Arvanitaki and Geraci (2014) Asimina Arvanitaki and Andrew A. Geraci, “Resonantly Detecting Axion-Mediated Forces with Nuclear Magnetic Resonance,” Phys. Rev. Lett. 113, 161801 (2014), arXiv:1403.1290 [hep-ph] .
  * Brown  _et al._ (2010) J. M. Brown, S. J. Smullin, T. W. Kornack, and M. V. Romalis, “New limit on lorentz- and $cpt$-violating neutron spin interactions,” Phys. Rev. Lett. 105, 151604 (2010).
  * Smiciklas  _et al._ (2011) M. Smiciklas, J. M. Brown, L. W. Cheuk, and M. V. Romalis, “A new test of local Lorentz invariance using 21Ne-Rb-K comagnetometer,” Phys. Rev. Lett. 107, 171604 (2011), arXiv:1106.0738 [physics.atom-ph] .
  * Colladay and Kostelecký (1998) D. Colladay and V. Alan Kostelecký, “Lorentz-violating extension of the standard model,” Phys. Rev. D 58, 116002 (1998).
  * Alexander  _et al._ (2022) Jim Alexander _et al._ , “The storage ring proton EDM experiment,” (2022), arXiv:2205.00830 [hep-ph] .
  * Suleiman  _et al._ (2023) Riad Suleiman, Vasiliy S. Morozov, and Ya. S. Derbenev, “High precision fundamental physics experiments using compact spin-transparent storage rings of low energy polarized electron beams,” Phys. Lett. B 843, 138058 (2023), arXiv:2105.11575 [physics.acc-ph] .
  * Brandenstein  _et al._ (2023) Chiara Brandenstein, Stefan Stelzl, Erwin Gutsmiedl, Wolfgang Schott, Andreas Weiler, and Peter Fierlinger, “Towards an electrostatic storage ring for fundamental physics measurements,” EPJ Web Conf. 282, 01017 (2023), arXiv:2211.08439 [hep-ex] .
  * Kaplan and Rattazzi (2016) David E. Kaplan and Riccardo Rattazzi, “Large field excursions and approximate discrete symmetries from a clockwork axion,” Phys. Rev. D 93, 085007 (2016), arXiv:1511.01827 [hep-ph] .
  * Farina  _et al._ (2017) Marco Farina, Duccio Pappadopulo, Fabrizio Rompineve, and Andrea Tesi, “The photo-philic QCD axion,” JHEP 01, 095 (2017), arXiv:1611.09855 [hep-ph] .
  * Agrawal  _et al._ (2018) Prateek Agrawal, Jiji Fan, and Matthew Reece, “Clockwork Axions in Cosmology: Is Chromonatural Inflation Chrononatural?” JHEP 10, 193 (2018), arXiv:1806.09621 [hep-th] .
  * Silverstein and Westphal (2008) Eva Silverstein and Alexander Westphal, “Monodromy in the CMB: Gravity Waves and String Inflation,” Phys. Rev. D 78, 106003 (2008), arXiv:0803.3085 [hep-th] .
  * McAllister  _et al._ (2010) Liam McAllister, Eva Silverstein, and Alexander Westphal, “Gravity Waves and Linear Inflation from Axion Monodromy,” Phys. Rev. D 82, 046003 (2010), arXiv:0808.0706 [hep-th] .
  * Kaloper and Sorbo (2009) Nemanja Kaloper and Lorenzo Sorbo, “A Natural Framework for Chaotic Inflation,” Phys. Rev. Lett. 102, 121301 (2009), arXiv:0811.1989 [hep-th] .
  * Aghanim  _et al._ (2018) N. Aghanim _et al._ (Planck), “Planck 2018 results. VI. Cosmological parameters,” (2018), arXiv:1807.06209 [astro-ph.CO] .
  * Tremaine and Gunn (1979a) S. Tremaine and J. E. Gunn, “Dynamical Role of Light Neutral Leptons in Cosmology,” Phys. Rev. Lett. 42, 407–410 (1979a).
  * Boyarsky  _et al._ (2009) Alexey Boyarsky, Oleg Ruchayskiy, and Dmytro Iakubovskyi, “A Lower bound on the mass of Dark Matter particles,” JCAP 03, 005 (2009), arXiv:0808.3902 [hep-ph] .
  * Falkowski  _et al._ (2009) Adam Falkowski, Jose Juknevich, and Jessie Shelton, “Dark Matter Through the Neutrino Portal,” (2009), arXiv:0908.1790 [hep-ph] .
  * Boddy  _et al._ (2014a) Kimberly K. Boddy, Jonathan L. Feng, Manoj Kaplinghat, Yael Shadmi, and Timothy M. P. Tait, “Strongly interacting dark matter: Self-interactions and keV lines,” Phys. Rev. D 90, 095016 (2014a), arXiv:1408.6532 [hep-ph] .
  * Boddy  _et al._ (2014b) Kimberly K. Boddy, Jonathan L. Feng, Manoj Kaplinghat, and Tim M. P. Tait, “Self-Interacting Dark Matter from a Non-Abelian Hidden Sector,” Phys. Rev. D 89, 115017 (2014b), arXiv:1402.3629 [hep-ph] .
  * Contino  _et al._ (2019) Roberto Contino, Andrea Mitridate, Alessandro Podo, and Michele Redi, “Gluequark Dark Matter,” JHEP 02, 187 (2019), arXiv:1811.06975 [hep-ph] .
  * Lee  _et al._ (2018) Junyi Lee, Attaallah Almasi, and Michael Romalis, “Improved Limits on Spin-Mass Interactions,” Phys. Rev. Lett. 120, 161801 (2018), arXiv:1801.02757 [hep-ex] .
  * Fan and Reig (2023) Xing Fan and Mario Reig, “New bounds and future prospects for axion force searches at Penning trap experiments,” (2023), arXiv:2310.18797 [hep-ph] .
  * Gradwohl and Frieman (1992) Ben-Ami Gradwohl and Joshua A. Frieman, “Dark matter, long range forces, and large scale structure,” Astrophys. J. 398, 407–424 (1992).
  * Nusser  _et al._ (2005) Adi Nusser, Steven S. Gubser, and P. J. E. Peebles, “Structure formation with a long-range scalar dark matter interaction,” Phys. Rev. D 71, 083505 (2005), arXiv:astro-ph/0412586 .
  * Hellwing and Juszkiewicz (2009) Wojciech A. Hellwing and Roman Juszkiewicz, “Dark Matter Gravitational Clustering With a Long-Range Scalar Interaction,” Phys. Rev. D 80, 083522 (2009), arXiv:0809.1976 [astro-ph] .
  * Vafa and Witten (1984) C. Vafa and Edward Witten, “Restrictions on Symmetry Breaking in Vector-Like Gauge Theories,” Nucl. Phys. B 234, 173–188 (1984).
  * Bertolini  _et al._ (2021) Stefano Bertolini, Luca Di Luzio, and Fabrizio Nesti, “Axion-mediated forces, CP violation and left-right interactions,” Phys. Rev. Lett. 126, 081801 (2021), arXiv:2006.12508 [hep-ph] .
  * Hook and Huang (2018) Anson Hook and Junwu Huang, “Probing axions with neutron star inspirals and other stellar processes,” JHEP 06, 036 (2018), arXiv:1708.08464 [hep-ph] .
  * Balkin  _et al._ (2022) Reuven Balkin, Javi Serra, Konstantin Springmann, Stefan Stelzl, and Andreas Weiler, “White dwarfs as a probe of light QCD axions,” (2022), arXiv:2211.02661 [hep-ph] .
  * Balkin  _et al._ (2023) Reuven Balkin, Javi Serra, Konstantin Springmann, Stefan Stelzl, and Andreas Weiler, “Heavy neutron stars from light scalars,” (2023), arXiv:2307.14418 [hep-ph] .
  * Tremaine and Gunn (1979b) Scott Tremaine and James E. Gunn, “Dynamical role of light neutral leptons in cosmology,” Phys. Rev. Lett. 42, 407–410 (1979b).
  * De Luca  _et al._ (2018) Valerio De Luca, Andrea Mitridate, Michele Redi, Juri Smirnov, and Alessandro Strumia, “Colored Dark Matter,” Phys. Rev. D 97, 115024 (2018), arXiv:1801.01135 [hep-ph] .


$\displaystyle=\left(\frac{1}{{2}}\exp(\lVert\bm{z}\rVert_{2}t)-\frac{1}{{2}}\exp(-\lVert\bm{z}\rVert_{2}t)\right)a^{0}\hat{\bm{z}}$
$\displaystyle~{}~{}+\left(\frac{1}{{2}}\exp(\lVert\bm{z}\rVert_{2}t)+\frac{1}{{2}}\exp(-\lVert\bm{z}\rVert_{2}t)\right)\left<\bm{w}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}$
$\displaystyle~{}~{}-\left<\bm{w}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}+\bm{w}^{0}.$

∎

######  Remark 9.

It is noteworthy that $\bm{w}$ has two components, one is the projection of
$\bm{w}^{0}$ into the direction of $\bm{z}$:

$\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)a^{0}\hat{\bm{z}}+\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$

which evolves with respect to $t$. The other is the projection of $\bm{w}^{0}$
onto $\bm{z}^{\perp}$:

$\bm{w}^{0}-\left<\bm{w}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$

which remains frozen as $t$ evolves.

###  C.2 Difference between Real and Linear Dynamics

We observe that the real dynamics can be written into

$\displaystyle\frac{\nu}{\varepsilon}\frac{\mathrm{d}{a}_{k}}{\mathrm{d}{t}}$
$\displaystyle=\left(-\frac{1}{n}\sum_{i=1}^{n}e_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}-\frac{1}{n}\sum_{i=1}^{n}y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}+\bm{w}_{k}^{\intercal}\bm{z}\right),$
(C.7) $\displaystyle\frac{\mathrm{d}{\bm{w}}_{k}}{\mathrm{d}{t}}$
$\displaystyle=\frac{\nu}{\varepsilon}\left(-\frac{1}{n}\sum_{i=1}^{n}e_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}-\frac{1}{n}\sum_{i=1}^{n}y_{i}{a}_{k}\bm{x}_{i}+a_{k}\bm{z}\right),$

hence the difference between the real and linear dynamics is characterized by
$\\{f_{k},\bm{g}_{k}\\}_{k=1}^{m}$, where

$\displaystyle f_{k}$
$\displaystyle:=\frac{1}{n}\sum_{i=1}^{n}\left(e_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}+y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}\right),$
$\displaystyle\bm{g}_{k}$
$\displaystyle:=\frac{1}{n}\sum_{i=1}^{n}\left(e_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}+y_{i}a_{k}\bm{x}_{i}\right).$

In other words, the real dynamics can be written into

$\displaystyle\frac{\mathrm{d}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}}{\mathrm{d}t}=\bm{A}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}+\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix},\quad\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}(0)\\\
\bm{w}_{k}(0)\end{pmatrix}=\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix},$ (C.8)

and its solution reads

$\displaystyle\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}=\exp\left(t\bm{A}\right)\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix}+\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}(s)\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}(s)\end{pmatrix}\mathrm{d}s.$ (C.9)

###  C.3 $\bm{w}$-lag regime

######  Definition 3 (Neuron $2$-energy, $\bm{w}$-lag regime).

In real dynamics, we define the $2$-energy at time $t$ for each single neuron,
i.e., for each $k\in[m]$,

$q_{k}(t):=\left(\frac{\nu^{2}}{\varepsilon^{2}}\left\lvert
a_{k}(t)\right\rvert^{2}+\left\lVert\bm{w}_{k}(t)\right\rVert_{2}^{2}\right)^{\frac{1}{2}}.$
(C.10)

We denote

$q_{\max}(t):=\max_{k\in[m]}q_{k}(t).$ (C.11)

For simplicity, we hereafter drop the $(t)$s for all $q_{k}(t)$ and
$q_{\max}(t)$. Then the estimates on $\\{f_{k},\bm{g}_{k}\\}_{k=1}^{m}$ read

######  Proposition 8.

For any time $t>0$,

$\displaystyle\left\lvert f_{k}\right\rvert$ $\displaystyle\leq
m\varepsilon^{2}q_{\max}^{2}\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}^{2},$
(C.12) $\displaystyle\left\lVert\bm{g}_{k}\right\rVert_{2}$ $\displaystyle\leq
m\varepsilon^{2}q_{\max}^{2}\left\lvert
a_{k}\right\rvert+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}\left\lvert
a_{k}\right\rvert.$

###### Proof.

We obtain that

$\displaystyle\left\lvert f_{k}\right\rvert$
$\displaystyle=\left\lvert\frac{1}{n}\sum_{i=1}^{n}\left((e_{i}+y_{i})\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}+y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}-y_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}\right)\right\rvert$
$\displaystyle=\left\lvert\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu
a_{k}\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\right)\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}+y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}-y_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}\right)\right\rvert$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\left\lVert{\bm{w}}_{k}\right\rVert_{2}\right)\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon({\bm{w}}_{k}^{\intercal}\bm{x}_{i})^{2}\right)$
$\displaystyle\leq\frac{1}{2n}\sum_{i=1}^{n}\left(\nu\varepsilon\left(\sum_{k=1}^{m}\left(\frac{\nu}{\varepsilon}\left\lvert
a_{k}\right\rvert^{2}+\frac{\varepsilon}{\nu}\left\lVert{\bm{w}}_{k}\right\rVert_{2}^{2}\right)\right)\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon({\bm{w}}_{k}^{\intercal}\bm{x}_{i})^{2}\right)$
$\displaystyle\leq\frac{1}{2n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\varepsilon^{2}q_{k}^{2}\right)\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon({\bm{w}}_{k}^{\intercal}\bm{x}_{i})^{2}\right)$
$\displaystyle\leq\frac{1}{2n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\varepsilon^{2}q_{k}^{2}\right)\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}^{2}\right)$
$\displaystyle\leq
m\varepsilon^{2}q_{\max}^{2}\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}^{2},$

and

$\displaystyle\left\lVert\bm{g}_{k}\right\rVert_{2}$
$\displaystyle=\left\lVert\frac{1}{n}\sum_{i=1}^{n}\left((e_{i}+y_{i}){a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}+y_{i}{a}_{k}\bm{x}_{i}-y_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}\right)\right\rVert_{2}$
$\displaystyle=\left\lVert\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu
a_{k}\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\right){a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}+y_{i}{a}_{k}\bm{x}_{i}-y_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}\right)\right\rVert_{2}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\left\lVert{\bm{w}}_{k}\right\rVert_{2}\right)\left\lvert
a_{k}\right\rvert+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}\left\lvert
a_{k}\right\rvert\right)$
$\displaystyle\leq\frac{1}{2n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\varepsilon^{2}q_{k}^{2}\right)\left\lvert
a_{k}\right\rvert+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}\left\lvert
a_{k}\right\rvert\right)$ $\displaystyle\leq
m\varepsilon^{2}q_{\max}^{2}\left\lvert
a_{k}\right\rvert+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}\left\lvert
a_{k}\right\rvert.$

∎

We denote a useful quantity

$\phi(t):=\sup_{0\leq s\leq t}{q}_{\max}(s).$ (C.13)

Then directly from Lemma 1, we have with probability at least $1-\delta$ over
the choice of $\bm{\theta}^{0}$,

$\max\limits_{k\in[m]}\left\\{\lvert
a_{k}^{0}\rvert,\;\lVert\bm{w}_{k}^{0}\rVert_{\infty}\right\\}\leq\sqrt{2\log\frac{2m(d+1)}{\delta}},$
(C.14)

hence

$\phi(0)\leq\sqrt{2(d+1)\log\frac{2m(d+1)}{\delta}}.$ (C.15)

We define

$T_{\mathrm{eff}}:=\inf\left\\{t>0\mid
m\varepsilon^{2}\phi^{3}(t)>m^{-\tau},\quad\tau=\frac{\gamma-\gamma^{\prime}-1}{4}\right\\},$
(C.16)

then for $m$ large enough, as $\gamma-\gamma^{\prime}>1$, based on (C.15),

$m\varepsilon^{2}\phi^{3}(0)\leq
m\varepsilon^{2}\left(2(d+1)\log\frac{2m(d+1)}{\delta}\right)^{\frac{3}{2}}\leq
m^{-\frac{\gamma-\gamma^{\prime}-1}{2}},$ (C.17)

hence $T_{\mathrm{eff}}\geq 0$.

We observe further that by taking the $2$-norm on both sides of (C.9), the
following holds

$\displaystyle q_{k}(t)$
$\displaystyle\leq\exp\left(t\left\lVert\bm{A}\right\rVert_{2\to
2}\right)q_{k}(0)+\int_{0}^{t}\exp\left((t-s)\left\lVert\bm{A}\right\rVert_{2\to
2}\right)\left(m\varepsilon^{2}q_{\max}^{2}(s)+\varepsilon
q_{\max}(s)\right)q_{k}(s)\mathrm{d}s,$

and by taking supreme over the index $k$ and time $0\leq t\leq
T_{\mathrm{eff}}$ on both sides, and for large enough $m$, the following holds

$\displaystyle\phi(t)$
$\displaystyle\leq\phi(0)\exp(t)+2m^{-\min\left\\{\frac{1}{2},\tau\right\\}}\int_{0}^{t}\exp(t-s)\mathrm{d}s$
(C.18)
$\displaystyle\leq\phi(0)\exp(t)+2m^{-\min\left\\{\frac{1}{2},\tau\right\\}}\left(\exp(t)-1\right)$
$\displaystyle\leq\phi(0)\exp(t)+2m^{-\min\left\\{\frac{1}{2},\tau\right\\}}\exp(t),$

then based on (C.15), we have with probability $1-\delta$ over the choice of
$\bm{\theta}^{0}$, for sufficiently large $m$,

$\displaystyle\phi(t)$
$\displaystyle\leq\phi(0)\exp(t)+2m^{-\min\left\\{\frac{1}{2},\tau\right\\}}\exp(t)$
(C.19) $\displaystyle\leq 2\phi(0)\exp(t)\leq
2\sqrt{2(d+1)\log\frac{2m(d+1)}{\delta}}\exp(t),$

we set $t_{0}$ as the time satisfying

$2\sqrt{2(d+1)\log\frac{2m(d+1)}{\delta}}\exp(t_{0})=\frac{1}{2}m^{\frac{\gamma-\gamma^{\prime}-1}{4}},$
(C.20)

then we obtain that, for any
$\eta_{0}>\frac{\gamma-\gamma^{\prime}-1}{100}>0$,

$T_{\mathrm{eff}}\geq
t_{0}>\log\left(\frac{1}{4}\right)+\left({\frac{\gamma-\gamma^{\prime}-1}{4}}-\eta_{0}\right)\log(m).$
(C.21)

######  Theorem 7 (Condensed regime, $\bm{w}$-lag regime).

Given any $\delta\in(0,1)$, under 1, 3 and 5, if $\gamma>1$ and
$0\leq\gamma^{\prime}<\gamma-1$, then with probability at least $1-\delta$
over the choice of $\bm{\theta}^{0}$, we have

$\lim_{m\to+\infty}\sup\limits_{t\in[0,T_{\mathrm{eff}}]}\mathrm{RD}(\bm{\theta}_{\bm{w}}(t))=+\infty,$
(C.22)

and

$\lim_{m\to+\infty}\sup\limits_{t\in[0,T_{\mathrm{eff}}]}\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert\bm{\theta}_{\bm{w}}(t)\right\rVert_{2}}=1.$
(C.23)

###### Proof.

Since we have

$\displaystyle\frac{\mathrm{d}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}}{\mathrm{d}t}=\bm{A}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}+\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix},\quad\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}(0)\\\
\bm{w}_{k}(0)\end{pmatrix}=\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix},$

and its solution reads

$\displaystyle\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}$
$\displaystyle=\exp\left(t\bm{A}\right)\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix}+\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix}\mathrm{d}s.$

As we notice that for any $k\in[m]$,
$\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\ \bm{w}_{k}\end{pmatrix}$ can
be written into two parts, the first one is the linear part, the second one is
the residual part. For simplicity of proof, we need to introduce some further
notations.

As we already identify the parameters
$\bm{\theta}_{a}=\mathrm{vec}(\\{a_{k}\\}_{k=1}^{m})$ and
$\bm{\theta}_{\bm{w}}=\mathrm{vec}(\\{\bm{w}_{k}\\}_{k=1}^{m})$, with some
slight misuse of notations, we denote
$\bm{\theta}_{a}:=\mathrm{vec}(\left\\{\frac{\nu}{\varepsilon}a_{k}\right\\}_{k=1}^{m})$.
From the observations above,

$\bm{\theta}:=\mathrm{vec}\left(\bm{\theta}_{a},\bm{\theta}_{\bm{w}}\right)=\mathrm{vec}\left(\left\\{\frac{\nu}{\varepsilon}a_{k}\right\\}_{k=1}^{m},\\{\bm{w}_{k}\\}_{k=1}^{m}\right),$

and $\bm{\theta}_{a}$ and $\bm{\theta}_{\bm{w}}$ can be written into

$\displaystyle\bm{\theta}_{a}(t)$
$\displaystyle=\bar{\bm{\theta}}_{a}(t)+\widetilde{\bm{\theta}}_{a}(t),$
$\displaystyle\bm{\theta}_{\bm{w}}(t)$
$\displaystyle=\bar{\bm{\theta}}_{\bm{w}}(t)+\widetilde{\bm{\theta}}_{\bm{w}}(t),$

where the $k$-th component of $\bar{\bm{\theta}}_{a}$ and
$\bar{\bm{\theta}}_{\bm{w}}$ respectively reads

$\displaystyle\left(\bar{\bm{\theta}}_{a}\right)_{k}$
$\displaystyle:=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)a_{k}^{0}+\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>,$
$\displaystyle\left(\bar{\bm{\theta}}_{\bm{w}}\right)_{k}$
$\displaystyle:=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)a_{k}^{0}\hat{\bm{z}}+\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}$
$\displaystyle~{}~{}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}+\bm{w}_{k}^{0},$

and the $k$-th component of $\widetilde{\bm{\theta}}_{a}$ and
$\widetilde{\bm{\theta}}_{\bm{w}}$ altogether reads

$\displaystyle\begin{pmatrix}\left(\widetilde{\bm{\theta}}_{a}(t)\right)_{k}\\\
\left(\widetilde{\bm{\theta}}_{\bm{w}}(t)\right)_{k}\end{pmatrix}$
$\displaystyle:=\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix}\mathrm{d}s.$

Moreover, we observe that ${\bm{\theta}}_{\bm{w}}$ can be decomposed into two
parts, one is the projection of $\bm{w}^{0}$ into the direction of $\bm{z}$,
i.e., ${\bm{\theta}}_{\bm{w},\bm{z}}$ , and the other is the projection of
$\bm{w}^{0}$ onto $\bm{z}^{\perp},$ i.e.,
${\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$. As $\bar{\bm{\theta}}_{\bm{w}}$ and
$\widetilde{\bm{\theta}}_{\bm{w}}$ inherits the same structure as
${\bm{\theta}}_{\bm{w}}$, we may apply the same decomposition to
$\bar{\bm{\theta}}_{\bm{w}}$ and $\widetilde{\bm{\theta}}_{\bm{w}}$. Hence, we
obtain that

$\displaystyle{\bm{\theta}}_{\bm{w},\bm{z}}(t)$
$\displaystyle=\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t),$
$\displaystyle{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)$
$\displaystyle=\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t),$

and these relations concerning $2$-norm hold simultaneously for any $t\geq 0$,

$\displaystyle\left\lVert{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}^{2}$
$\displaystyle=\left\lVert{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}^{2}+\left\lVert{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}^{2},$
$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}^{2}$
$\displaystyle=\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}^{2}+\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}^{2},$
$\displaystyle\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}^{2}$
$\displaystyle=\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}^{2}+\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}^{2},$

and the $k$-th component of $\bar{\bm{\theta}}_{\bm{w},\bm{z}}$ and
$\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$ altogether reads

$\displaystyle\left(\bar{\bm{\theta}}_{\bm{w},\bm{z}}\right)_{k}$
$\displaystyle:=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)a_{k}^{0}\hat{\bm{z}}+\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$
$\displaystyle\left(\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}\right)_{k}$
$\displaystyle:=\bm{w}_{k}^{0}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$

and finally, based on the relations concerning $2$-norm above, for any $t\geq
0$,

$\displaystyle\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}$
$\displaystyle\leq\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2},$
$\displaystyle\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}$
$\displaystyle\leq\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}.$

We are hereby to prove (C.22). Firstly, we observe that

${\bm{\theta}}_{\bm{w}}(0)=\bar{\bm{\theta}}_{\bm{w}}(0),$

hence

$\displaystyle\left\lVert\bm{\theta}_{\bm{w}}(t)-\bm{\theta}_{\bm{w}}(0)\right\rVert_{2}^{2}$
$\displaystyle=$
$\displaystyle\left\lVert\bm{\theta}_{\bm{w}}(t)-\bar{\bm{\theta}}_{\bm{w}}(0)\right\rVert^{2}_{2}$
$\displaystyle=$
$\displaystyle\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\bm{\theta}_{\bm{w},\bm{z}^{\perp}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0)\right\rVert_{2}$
$\displaystyle=$
$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}+\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2},$

by choosing $\eta_{0}=\frac{\gamma-\gamma^{\prime}-1}{8}$, then for time
$0\leq
t\leq\bar{t}_{0}:=\left({\frac{\gamma-\gamma^{\prime}-1}{8}}\right)\log(m)-\log(2)$,

$\displaystyle\left\lVert\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix}\mathrm{d}s\right\rVert_{2}$
$\displaystyle\leq\left(m\varepsilon^{2}\phi^{3}(t)+\varepsilon\phi^{2}(t)\right)\int_{0}^{t}\exp((t-s)\left\lVert\bm{z}\right\rVert_{2})\mathrm{d}s$
$\displaystyle\leq
2m^{-\min\left\\{\tau,\frac{1}{2}\right\\}}\int_{0}^{t}\exp((t-s))\mathrm{d}s$
$\displaystyle\leq 2m^{-\frac{\gamma-\gamma^{\prime}-1}{4}}\exp(t)\leq
2m^{-\frac{\gamma-\gamma^{\prime}-1}{4}}\exp(\bar{t}_{0})=m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}.$

We conclude that for $t\leq\bar{t}_{0}$, the following holds

$\displaystyle\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}$
$\displaystyle\leq\sqrt{m}\left\lVert\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix}\mathrm{d}s\right\rVert_{2}\leq\sqrt{m}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}},$

since the $k$-th component of $\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$
reads

$\displaystyle\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)$
$\displaystyle=\bm{w}_{k}^{0}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$

we observe that since the RHS is independent of time $t$, hence we have

$\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)=\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0),$

so we obtain that

$\displaystyle\left\lVert\bm{\theta}_{\bm{w}}(t)-\bm{\theta}_{\bm{w}}(0)\right\rVert^{2}_{2}$
$\displaystyle=$
$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}+\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}$
$\displaystyle=$
$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}+\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}.$

thus the ratio reads

$\displaystyle\left(\frac{\left\lVert{\bm{\theta}}_{\bm{w}}(t)-{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\right)^{2}$
$\displaystyle=\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}+\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}}$
$\displaystyle=\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}^{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}}+\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}}.$

As

$\left\lVert{{\bm{\theta}}}_{\bm{w}}(0)\right\rVert_{2}^{2}=\left\lVert{{\bm{\theta}}}_{\bm{w}}^{0}\right\rVert_{2}^{2}=\sum_{k=1}^{m}\left(\bm{w}_{k}^{0}\right)^{2},$

and we observe that since $\bm{w}^{0}_{k}\sim\mathcal{N}(\bm{0},\bm{I}_{d})$,
then $\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\sim\mathcal{N}(0,1)$. Moreover,
$\left\\{a_{k}^{0},\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\right\\}_{k=1}^{m}\sim\mathcal{N}(0,1)$
are i.i.d. Gaussian variables. Hence, by application of Theorem 3, with
probability $1-\frac{\delta}{4}$ over the choice of $\bm{\theta}^{0}$, for $m$
large enough,

$\frac{d}{2}\leq\frac{1}{m}\left\lVert{\bm{\theta}}_{\bm{w}}^{0}\right\rVert_{2}^{2}\leq\frac{3d}{2},$

and with probability $1-\frac{\delta}{4}$ over the choice of
$\bm{\theta}^{0}$, for $m$ large enough,

$\frac{1}{2}\leq\frac{1}{m}\sum_{k=1}^{m}\left(a_{k}^{0}\right)^{2}\leq\frac{3}{2},$

and with probability $1-\frac{\delta}{4}$ over the choice of
$\bm{\theta}^{0}$, for $m$ large enough,

$\frac{1}{2}\leq\frac{1}{m}\sum_{k=1}^{m}\left(\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\right)^{2}\leq\frac{3}{2},$

and with probability $1-\frac{\delta}{4}$ over the choice of
$\bm{\theta}^{0}$, for $m$ large enough,

$-\frac{1}{4}\leq\frac{1}{m}a_{k}^{0}\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\leq\frac{1}{4}.$

To sum up, the ratio

$\displaystyle\left(\frac{\left\lVert{\bm{\theta}}_{\bm{w}}(t)-{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\right)^{2}$
$\displaystyle=\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}^{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}}+\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}},$

for the first part of the RHS:

$\displaystyle\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}$
$\displaystyle\geq\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)\right\rVert_{2}-\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}$
$\displaystyle=\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)\right\rVert_{2}}{{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}}-\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}},$

and

$\displaystyle~{}~{}\frac{1}{m}\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)\right\rVert_{2}^{2}$
$\displaystyle=\sum_{k=1}^{m}\left[\frac{1}{2}\left(r^{2}(t)-{r^{-2}(t)}\right)\frac{\nu}{\varepsilon}a_{k}^{0}+\frac{1}{2}\left(r^{2}(t)+{r^{-2}(t)}-2\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\right]^{2}$
$\displaystyle=\frac{1}{4}\left(r(t)-{r^{-1}(t)}\right)^{2}\sum_{k=1}^{m}\left[\left(r(t)+{r^{-1}(t)}\right)\frac{\nu}{\varepsilon}a_{k}^{0}+\left(r(t)-{r^{-1}(t)}\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\right]^{2}$
$\displaystyle=\frac{1}{4}\left(r(t)-{r^{-1}(t)}\right)^{2}\sum_{k=1}^{m}\Bigg{\\{}\left[\left(r(t)+{r^{-1}(t)}\right)^{2}\left(\frac{\nu}{\varepsilon}a_{k}^{0}\right)^{2}+\left(r(t)-{r^{-1}(t)}\right)^{2}\left(\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\right)^{2}\right]$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+2\left(r^{2}(t)-{r^{-2}(t)}\right)\frac{\nu}{\varepsilon}a_{k}^{0}\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\Bigg{\\}}$
$\displaystyle\geq\frac{1}{4}\left(r(t)-{r^{-1}(t)}\right)^{2}\Bigg{[}\left(r(t)+{r^{-1}(t)}\right)^{2}\left(\frac{\nu}{\varepsilon}\right)^{2}\frac{1}{2}+\left(r(t)-{r^{-1}(t)}\right)^{2}\frac{1}{2}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}-\left(r^{2}(t)-{r^{-2}(t)}\right)\frac{\nu}{\varepsilon}\frac{1}{2}\Bigg{]}$
$\displaystyle\geq\frac{3}{32}\left(r(t)-{r^{-1}(t)}\right)^{2}\left(r(t)-{r^{-1}(t)}\right)^{2}=\frac{3}{32}\left(r(t)-{r^{-1}(t)}\right)^{4}.$

Then with probability at least $1-\delta$ over the choice of $\bm{\theta}^{0}$
and large enough $m$, for any $0\leq
t\leq\bar{t}_{0}=\left({\frac{\gamma-\gamma^{\prime}-1}{8}}\right)\log(m)-\log(2)$,
the following holds:

$\displaystyle\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}$
$\displaystyle\geq\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)\right\rVert_{2}}{{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}}-\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}$
$\displaystyle\geq\sqrt{\frac{2}{3md}}\sqrt{\frac{3m}{32}}\left(r(t)-{r^{-1}(t)}\right)^{2}-\sqrt{\frac{2}{d}}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}.$

Specifically, if we choose $t=\bar{t}_{0}$,

$\displaystyle\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t_{0})-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t_{0})\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}$
$\displaystyle\geq\sqrt{\frac{1}{16d}}\left(r^{2}(t_{0})+r^{-2}(t_{0})-2\right)-\sqrt{\frac{2}{d}}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}$
$\displaystyle\gtrsim
m^{\frac{\gamma-\gamma^{\prime}-1}{8}}-m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}.$

By taking limit, we obtain that

$\lim_{m\to\infty}\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t_{0})-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t_{0})\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}=+\infty.$

For the second part of the RHS:

$\displaystyle\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\leq\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}},$

then with probability at least $1-\delta$ over the choice of $\bm{\theta}^{0}$
and large enough $m$, for any $0\leq
t\leq\bar{t}_{0}=\left({\frac{\gamma-\gamma^{\prime}-1}{8}}\right)\log(m)-\log(2)$,
the following holds:

$\displaystyle\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\leq\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\leq\sqrt{\frac{2}{d}}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}.$

By taking limit, we obtain that

$\lim_{m\to\infty}\frac{\left\lVert\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}=0.$

To sum up, since $t_{0}\leq T_{\mathrm{eff}}$, we have that

$\lim_{m\to+\infty}\sup\limits_{t\in[0,T_{\mathrm{eff}}]}\mathrm{RD}(\bm{\theta}_{\bm{w}}(t))=+\infty+0=+\infty,$
(C.24)

which finishes the proof of (C.22).

In order to prove (C.23), firstly we have

$\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert\bm{\theta}_{\bm{w}}(t)\right\rVert_{2}}\leq
1,$

moreover, we observe that

$\displaystyle\left(\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert\bm{\theta}_{\bm{w}}(t)\right\rVert_{2}}\right)^{2}$
$\displaystyle=\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}}{\left\lVert\bm{\theta}_{\bm{w}}(t)\right\rVert^{2}_{2}}=\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}}{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}+\left\lVert\bm{\theta}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}}$
$\displaystyle=\frac{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}}{{\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}}+\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}}.$

Then with probability at least $1-\delta$ over the choice of $\bm{\theta}^{0}$
and large enough $m$, for any $0\leq
t\leq\bar{t}_{0}=\left({\frac{\gamma-\gamma^{\prime}-1}{8}}\right)\log(m)-\log(2)$,
the following holds:

$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}$
$\displaystyle\geq\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}-\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(0)\right\rVert_{2}$
$\displaystyle\geq\sqrt{\frac{3m}{32}}\left(r(t)-{r^{-1}(t)}\right)^{2}-\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}-\left\lVert\bar{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}$
$\displaystyle\geq\sqrt{\frac{3m}{32}}\left(r(t)-{r^{-1}(t)}\right)^{2}-\sqrt{m}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}-\sqrt{\frac{3md}{2}},$
$\displaystyle\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}$
$\displaystyle\leq\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)-\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}+\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(0)\right\rVert_{2}$
$\displaystyle\leq\left\lVert\widetilde{\bm{\theta}}_{\bm{w}}(t)\right\rVert_{2}+\left\lVert\bar{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}$
$\displaystyle\leq\sqrt{m}m^{-\frac{\gamma-\gamma^{\prime}-1}{8}}+\sqrt{\frac{3md}{2}}.$

By taking $t=\bar{t}_{0}$, we observe that
$\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert_{2}$
is of order at least $\sqrt{m}m^{\frac{\gamma-\gamma^{\prime}-1}{8}}$, while
$\left\lVert\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert_{2}$
is of order at most $\sqrt{m}$, which finishes the proof of (C.23). ∎

###  C.4 $a$-lag regime

Recall the real dynamics,

$\displaystyle\frac{\mathrm{d}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}}{\mathrm{d}t}=\bm{A}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}+\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}\end{pmatrix},\quad\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}(0)\\\
\bm{w}_{k}(0)\end{pmatrix}=\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix},$

and its solution (C.8) reads

$\displaystyle\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\bm{w}_{k}\end{pmatrix}=\exp\left(t\bm{A}\right)\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\bm{w}_{k}^{0}\end{pmatrix}+\int_{0}^{t}\exp\left((t-s)\bm{A}\right)\begin{pmatrix}f_{k}(s)\\\
\frac{\nu}{\varepsilon}\bm{g}_{k}(s)\end{pmatrix}\mathrm{d}s.$

Then, for any $\bm{u}\in\bm{z}^{\perp}$, and
$\left\lVert\bm{u}\right\rVert_{2}=1$, we obtain that

$\displaystyle\frac{\mathrm{d}\left<\bm{w}_{k},\bm{u}\right>}{\mathrm{d}t}$
$\displaystyle=\frac{\nu}{\varepsilon}\left<\bm{z},\bm{u}\right>+\frac{\nu}{\varepsilon}\left<\bm{g}_{k},\bm{u}\right>$
$\displaystyle=\frac{\nu}{\varepsilon}\left<\bm{g}_{k},\bm{u}\right>,$

and as

$\bm{\theta}_{\bm{w},\bm{z}^{\perp}}=\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}+\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$

hence the $k$-th component of $\bm{\theta}_{\bm{w},\bm{z}^{\perp}}$,
$\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$ and
$\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}$ altogether reads

$\displaystyle\left(\bar{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}\right)_{k}$
$\displaystyle=\bm{w}_{k}^{0}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}},$
$\displaystyle\left(\widetilde{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}\right)_{k}$
$\displaystyle=\frac{\nu}{\varepsilon}\int_{0}^{t}\left[\bm{g}_{k}(s)-\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\hat{\bm{z}}\right]\mathrm{d}s,$
$\displaystyle\left({\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}\right)_{k}$
$\displaystyle=\bm{w}_{k}^{0}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}+\frac{\nu}{\varepsilon}\int_{0}^{t}\left[\bm{g}_{k}(s)-\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\hat{\bm{z}}\right]\mathrm{d}s.$

Moreover, the real dynamics can also be written as

$\displaystyle\frac{\mathrm{d}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\left<\bm{w}_{k},\hat{\bm{z}}\right>\end{pmatrix}}{\mathrm{d}t}=\bm{B}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\left<\bm{w}_{k},\hat{\bm{z}}\right>\end{pmatrix}+\begin{pmatrix}f_{k}\\\
\frac{\nu}{\varepsilon}\left<\bm{g}_{k},\hat{\bm{z}}\right>\end{pmatrix},\quad\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}(0)\\\
\left<\bm{w}_{k},\hat{\bm{z}}\right>(0)\end{pmatrix}=\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\end{pmatrix},$

where

$\bm{B}:=\begin{pmatrix}0&\left\lVert\bm{z}\right\rVert_{2}\\\
\left\lVert\bm{z}\right\rVert_{2}&0\end{pmatrix}=\left\lVert\bm{z}\right\rVert_{2}\begin{pmatrix}0&1\\\
1&0\end{pmatrix},$

since $\bm{B}$ is a full rank matrix, the solution to the above dyanmics can
be explicitly written out as

$\displaystyle\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}\\\
\left<\bm{w}_{k},\hat{\bm{z}}\right>\end{pmatrix}$
$\displaystyle=\exp\left(t\bm{B}\right)\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\end{pmatrix}+\int_{0}^{t}\exp\left((t-s)\bm{B}\right)\begin{pmatrix}f_{k}(s)\\\
\frac{\nu}{\varepsilon}\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\end{pmatrix}\mathrm{d}s$
$\displaystyle=\begin{pmatrix}\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)&\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\\\
\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)&\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\end{pmatrix}\begin{pmatrix}\frac{\nu}{\varepsilon}a_{k}^{0}\\\
\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\end{pmatrix}$
$\displaystyle~{}~{}+\int_{0}^{t}\begin{pmatrix}\frac{1}{{2}}r^{2}(t-s)+\frac{1}{{2}}r^{-2}(t-s)&\frac{1}{{2}}r^{2}(t-s)-\frac{1}{{2}}r^{-2}(t-s)\\\
\frac{1}{{2}}r^{2}(t-s)-\frac{1}{{2}}r^{-2}(t-s)&\frac{1}{{2}}r^{2}(t-s)+\frac{1}{{2}}r^{-2}(t-s)\end{pmatrix}\begin{pmatrix}f_{k}(s)\\\
\frac{\nu}{\varepsilon}\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\end{pmatrix}\mathrm{d}s,$

hence we obtain that

$\displaystyle\left(\bm{\theta}_{a}\right)_{k}$
$\displaystyle=\bm{a}_{k}=\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)a_{k}^{0}+\frac{\varepsilon}{\nu}\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>$
$\displaystyle~{}~{}+\frac{\varepsilon}{\nu}\int_{0}^{t}\left(\frac{1}{{2}}r^{2}(t-s)+\frac{1}{{2}}r^{-2}(t-s)\right)f_{k}(s)\mathrm{d}s$
$\displaystyle~{}~{}+\int_{0}^{t}\left(\frac{1}{{2}}r^{2}(t-s)-\frac{1}{{2}}r^{-2}(t-s)\right)\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\mathrm{d}s,$
$\displaystyle\left(\bm{\theta}_{\bm{w}}\right)_{k}$
$\displaystyle=\bm{w}_{k}=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t)-\frac{1}{{2}}r^{-2}(t)\right)a_{k}^{0}\hat{\bm{z}}+\left(\frac{1}{{2}}r^{2}(t)+\frac{1}{{2}}r^{-2}(t)\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}$
$\displaystyle~{}~{}+\int_{0}^{t}\left(\frac{1}{{2}}r^{2}(t-s)-\frac{1}{{2}}r^{-2}(t-s)\right)f_{k}(s)\mathrm{d}s\hat{\bm{z}}$
$\displaystyle~{}~{}+\frac{\nu}{\varepsilon}\int_{0}^{t}\left(\frac{1}{{2}}r^{2}(t-s)+\frac{1}{{2}}r^{-2}(t-s)\right)\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\mathrm{d}s\hat{\bm{z}}$
$\displaystyle~{}~{}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}+\bm{w}_{k}^{0}+\frac{\nu}{\varepsilon}\int_{0}^{t}\left[\bm{g}_{k}(s)-\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\hat{\bm{z}}\right]\mathrm{d}s.$

######  Definition 4 (Neuron $\infty$-energy, $a$-lag regime).

In real dynamics, we define the $\infty$-energy at time $t$ for each single
neuron, i.e., for each $k\in[m]$,

$p_{k}(t):=\max_{k\in[m]}\left\\{\left\lvert
a_{k}(t)\right\rvert,\left\lVert\bm{w}_{k}(t)\right\rVert_{\infty}\right\\}.$
(C.25)

We denote

$p_{\max}(t):=\max_{k\in[m]}p_{k}(t).$ (C.26)

For simplicity, we hereafter drop the $(t)$s for all $p_{k}(t)$ and
$p_{\max}(t)$. Then the estimates on $\\{f_{k},\bm{g}_{k}\\}_{k=1}^{m}$ read

######  Proposition 9.

For any time $t>0$,

$\displaystyle\left\lvert f_{k}\right\rvert$ $\displaystyle\leq
dm\nu\varepsilon
p_{\max}^{2}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}+\varepsilon
d\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}^{2},$ (C.27)
$\displaystyle\left\lVert\bm{g}_{k}\right\rVert_{\infty}$
$\displaystyle\leq\sqrt{d}m\nu\varepsilon p_{\max}^{2}\left\lvert
a_{k}\right\rvert+\varepsilon\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}\left\lvert
a_{k}\right\rvert.$

###### Proof.

We obtain that

$\displaystyle\left\lvert f_{k}\right\rvert$
$\displaystyle=\left\lvert\frac{1}{n}\sum_{i=1}^{n}\left((e_{i}+y_{i})\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}+y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}-y_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}\right)\right\rvert$
$\displaystyle=\left\lvert\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu
a_{k}\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\right)\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}+y_{i}{\bm{w}}_{k}^{\intercal}\bm{x}_{i}-y_{i}\frac{\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})}{\varepsilon}\right)\right\rvert$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\left\lVert{\bm{w}}_{k}\right\rVert_{2}\right)\left\lVert{\bm{w}}_{k}\right\rVert_{2}+\varepsilon({\bm{w}}_{k}^{\intercal}\bm{x}_{i})^{2}\right)$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}\right)\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}+\varepsilon
d\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}^{2}\right)$ $\displaystyle\leq
dm\nu\varepsilon
r_{\max}^{2}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}+\varepsilon
d\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}^{2},$

and

$\displaystyle\left\lVert\bm{g}_{k}\right\rVert_{\infty}$
$\displaystyle=\left\lVert\frac{1}{n}\sum_{i=1}^{n}\left((e_{i}+y_{i}){a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}+y_{i}{a}_{k}\bm{x}_{i}-y_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}\right)\right\rVert_{\infty}$
$\displaystyle=\left\lVert\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu
a_{k}\sigma(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\right){a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}+y_{i}{a}_{k}\bm{x}_{i}-y_{i}{a}_{k}\sigma^{(1)}(\varepsilon{\bm{w}}_{k}^{\intercal}\bm{x}_{i})\bm{x}_{i}\right)\right\rVert_{\infty}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\left\lVert{\bm{w}}_{k}\right\rVert_{2}\right)\left\lvert
a_{k}\right\rvert+\varepsilon\left\lVert{\bm{w}}_{k}\right\rVert_{2}\left\lvert
a_{k}\right\rvert\right)$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left(\left(\sum_{k=1}^{m}\nu\varepsilon\left\lvert
a_{k}\right\rvert\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}\right)\left\lvert
a_{k}\right\rvert+\varepsilon\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}\left\lvert
a_{k}\right\rvert\right)$ $\displaystyle\leq\sqrt{d}m\nu\varepsilon
r_{\max}^{2}\left\lvert
a_{k}\right\rvert+\varepsilon\sqrt{d}\left\lVert{\bm{w}}_{k}\right\rVert_{\infty}\left\lvert
a_{k}\right\rvert.$

∎

We denote a useful quantity

$\psi(t):=\sup_{0\leq s\leq t}{p}_{\max}(s).$ (C.28)

Then directly from Lemma 1, we have with probability at least $1-\delta$ over
the choice of $\bm{\theta}^{0}$,

$\max\limits_{k\in[m]}\left\\{\lvert
a_{k}^{0}\rvert,\;\lVert\bm{w}_{k}^{0}\rVert_{\infty}\right\\}\leq\sqrt{2\log\frac{2m(d+1)}{\delta}},$
(C.29)

hence

$\psi(0)\leq\sqrt{2(d+1)\log\frac{2m(d+1)}{\delta}},$ (C.30)

and for all $k\in[m]$ and any time $t>0$,

$\displaystyle\left\lvert f_{k}\right\rvert$ $\displaystyle\leq
dm\nu\varepsilon\psi^{3}(t)+\varepsilon d\psi^{2}(t),$
$\displaystyle\left\lVert\bm{g}_{k}\right\rVert_{\infty}$
$\displaystyle\leq\sqrt{d}m\nu\varepsilon\psi^{3}(t)+\varepsilon\sqrt{d}\psi^{2}(t).$

We are hereby to conduct some simple calculations. Let $\alpha>0$, and there
exists $t_{\alpha}>0$, such that

$r^{2}(t_{\alpha})-r^{-2}(t_{\alpha})=2m^{-\alpha},$

then we obtain immediately that

$\displaystyle r^{2}(t_{\alpha})$
$\displaystyle=\exp(\left\lVert\bm{z}\right\rVert_{2}t_{\alpha})=\sqrt{1+m^{-\alpha}}+m^{-\alpha},$

and

$t_{\alpha}\lesssim\log(\sqrt{1+m^{-\alpha}}+m^{-\alpha})\sim\frac{3}{2}m^{-\alpha}\sim\mathcal{O}(1),$

with

$t_{\alpha}\gtrsim\log(\sqrt{1+m^{-\alpha}}+m^{-\alpha})\sim\frac{3}{2}m^{-\alpha}\sim\Omega(1).$

Moreover,

$\displaystyle\frac{1}{{2}}r^{2}(t_{\alpha})+\frac{1}{{2}}r^{-2}(t_{\alpha})$
$\displaystyle=\sqrt{1+m^{-\alpha}},$
$\displaystyle\frac{1}{{2}}r^{2}(t_{\alpha})-\frac{1}{{2}}r^{-2}(t_{\alpha})$
$\displaystyle=m^{-\alpha},$
$\displaystyle\int_{0}^{t_{\alpha}}\frac{1}{{2}}r^{2}(t_{\alpha}-s)+\frac{1}{{2}}r^{-2}(t_{\alpha}-s)\mathrm{d}s$
$\displaystyle=\frac{m^{-\alpha}}{\left\lVert\bm{z}\right\rVert_{2}},$
$\displaystyle\int_{0}^{t_{\alpha}}\frac{1}{{2}}r^{2}(t_{\alpha}-s)-\frac{1}{{2}}r^{-2}(t_{\alpha}-s)\mathrm{d}s$
$\displaystyle=\frac{\sqrt{1+m^{-\alpha}}-1}{\left\lVert\bm{z}\right\rVert_{2}},$

hence

$\displaystyle\left\lvert a_{k}\right\rvert$
$\displaystyle\leq\psi(0)+m^{-\alpha}\left(m\nu\varepsilon\psi^{3}(t_{\alpha})+\varepsilon\psi^{2}(t_{\alpha})\right),$
(C.31) $\displaystyle\left\lVert\bm{w}_{k}\right\rVert_{\infty}$
$\displaystyle\leq\frac{\nu}{\varepsilon}m^{-\alpha}\psi(0)+\frac{\nu}{\varepsilon}m^{-\alpha}\left(m\nu\varepsilon\psi^{3}(t_{\alpha})+\varepsilon\psi^{2}(t_{\alpha})\right),$

For some $\beta>1$, we define

$\widetilde{T}_{\mathrm{eff},\beta}:=\inf\left\\{t>0\mid
m\nu\varepsilon\psi^{3}(t)+\varepsilon\psi^{2}(t)>m^{-\widetilde{\tau}},\quad\widetilde{\tau}=-\frac{\gamma^{\prime}}{\beta}\right\\},$
(C.32)

then for $m$ large enough, as $\gamma>1$, based on (C.30), we choose $\beta$
satisfying

$\min\left\\{\gamma-1,\frac{1}{2}\right\\}>-\frac{\gamma^{\prime}}{\beta},$

then, we obtain that

$m\nu\varepsilon\psi^{3}(0)+\varepsilon\psi^{2}(0)\leq
m\nu\varepsilon\left(2(d+1)\log\frac{2m(d+1)}{\delta}\right)^{\frac{3}{2}}+m^{-\frac{1}{2}}\left(2(d+1)\log\frac{2m(d+1)}{\delta}\right)\leq
m^{-\frac{\gamma-1}{2\beta}},$

hence $\widetilde{T}_{\mathrm{eff},\beta}\geq 0$.

We observe further that by taking the $\infty$-norm on both sides of (C.31),
and by taking supreme over the index $k$ and time $0\leq
t\leq\min\left\\{t_{\alpha},\widetilde{T}_{\mathrm{eff},\beta}\right\\}$ on
both sides, and for large enough $m$, the following holds

$\displaystyle\psi(t)\leq\frac{\nu}{\varepsilon}m^{-\alpha}\psi(0)+\frac{\nu}{\varepsilon}m^{-\alpha}m^{-\frac{\gamma^{\prime}}{\beta}}\leq
m^{-\gamma^{\prime}-\alpha}\sqrt{2(d+1)\log\frac{2m(d+1)}{\delta}},$

now we shall choose $\alpha>0$ and $\beta>1$, such that

$t_{\alpha}\leq\widetilde{T}_{\mathrm{eff},\beta},$

which is equivalent to solve out the relation

$m^{1-\gamma}m^{-3\gamma^{\prime}-3\alpha}+m^{-\frac{1}{2}}m^{-2\gamma^{\prime}-2\alpha}\leq
m^{\frac{\gamma^{\prime}}{\beta}}.$

Then, we choose $\alpha$ satisfying

$0<\gamma^{\prime}-\alpha\leq\min\left\\{\frac{1}{3}\left(\gamma-1+\frac{\gamma^{\prime}}{\beta}\right),\frac{1}{2}\left(\frac{1}{2}+\frac{\gamma^{\prime}}{\beta}\right)\right\\},$

the existence of $\alpha$ can be guaranteed since $\gamma^{\prime}>0,$ and

$\min\left\\{\frac{1}{3}\left(\gamma-1+\frac{\gamma^{\prime}}{\beta}\right),\frac{1}{2}\left(\frac{1}{2}+\frac{\gamma^{\prime}}{\beta}\right)\right\\}\geq\frac{\gamma^{\prime}}{6\beta}>0.$

We observe that

$\widetilde{T}_{\mathrm{eff},\beta}\geq
t_{\alpha},\quad\widetilde{T}_{\mathrm{eff},\beta}\sim\Omega(1).$ (C.33)

WLOG, we choose
$\bar{\beta}:=\max\left\\{-1024\frac{\gamma^{\prime}}{\gamma-1},-512\gamma^{\prime}\right\\}$,
and $\bar{\alpha}$ accordingly, and finally we have

######  Theorem 8 (Condensed regime, $a$-lag regime).

Given any $\delta\in(0,1)$, under 1, 3 and 5, if $\gamma>1$ and
$\gamma^{\prime}<0$, then with probability at least $1-\delta$ over the choice
of $\bm{\theta}^{0}$, we have

$\lim_{m\to+\infty}\sup\limits_{t\in[0,\widetilde{T}_{\mathrm{eff},\bar{\beta}}]}\mathrm{RD}(\bm{\theta}_{\bm{w}}(t))=+\infty,$
(C.34)

and

$\lim_{m\to+\infty}\sup\limits_{t\in[0,\widetilde{T}_{\mathrm{eff},\bar{\beta}}]}\frac{\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert_{2}}{\left\lVert\bm{\theta}_{\bm{w}}(t)\right\rVert_{2}}=1.$
(C.35)

As the details are almost the same as the ones in Theorem 7, the proof of
Theorem 8 is written in a slightly sketchy way.

###### Proof.

We observe that

$\displaystyle~{}~{}\left(\bm{\theta}_{\bm{w}}(t_{\bar{\alpha}})-\bm{\theta}_{\bm{w}}(0)\right)_{k}$
$\displaystyle=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}})-\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}})\right)a_{k}^{0}\hat{\bm{z}}+\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}})+\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}})-1\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}$
$\displaystyle~{}~{}+\int_{0}^{t_{\bar{\alpha}}}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}}-s)-\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}}-s)\right)f_{k}(s)\mathrm{d}s\hat{\bm{z}}$
$\displaystyle~{}~{}+\frac{\nu}{\varepsilon}\int_{0}^{t_{\bar{\alpha}}}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}}-s)+\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}}-s)\right)\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\mathrm{d}s\hat{\bm{z}}$
$\displaystyle~{}~{}+\frac{\nu}{\varepsilon}\int_{0}^{t_{\bar{\alpha}}}\left[\bm{g}_{k}(s)-\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\hat{\bm{z}}\right]\mathrm{d}s,$

so we obtain that with probability at least $1-\delta$ over the choice of
$\bm{\theta}^{0}$ and large enough $m$, for any $0\leq
t\leq{t}_{\bar{\alpha}}$, the following holds:

$\displaystyle~{}~{}\frac{1}{m}\left\lVert\bm{\theta}_{\bm{w}}(t)-\bm{\theta}_{\bm{w}}(0)\right\rVert^{2}_{2}$
$\displaystyle\geq
m^{-\gamma^{\prime}-\bar{\alpha}}\left(\frac{1}{2}-\sqrt{d}\max_{k\in[m]}\left\lVert\bm{g}_{k}\right\rVert_{\infty}\right)-\left(\sqrt{1+m^{-\bar{\alpha}}}-1\right)\left(\frac{3d}{2}+\sqrt{d}\max_{k\in[m]}\left\lvert
f_{k}\right\rvert\right)$ $\displaystyle\geq
m^{-\gamma^{\prime}-\bar{\alpha}}\left(\frac{1}{2}-\sqrt{d}m^{-\widetilde{\tau}}\right)-\frac{1}{2}m^{-\bar{\alpha}}\left(\frac{3d}{2}+\sqrt{d}m^{-\widetilde{\tau}}\right),$

hence the ratio

$\displaystyle~{}~{}\left(\frac{\left\lVert{\bm{\theta}}_{\bm{w}}(t)-{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}\right)^{2}=\frac{\frac{1}{m}\left\lVert{\bm{\theta}}_{\bm{w}}(t)-{\bm{\theta}}_{\bm{w}}(0)\right\rVert^{2}_{2}}{\frac{1}{m}\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}^{2}}$
$\displaystyle\geq\frac{2}{3d}\left[{m^{-\gamma^{\prime}-\bar{\alpha}}\left(\frac{1}{2}-\sqrt{d}m^{-\widetilde{\tau}}\right)-\frac{1}{2}m^{-\bar{\alpha}}\left(\frac{3d}{2}+\sqrt{d}m^{-\widetilde{\tau}}\right)}\right],$

by taking limit, we obtain that for any $0\leq t\leq{t}_{\bar{\alpha}}$

$\lim_{m\to\infty}\frac{\left\lVert{\bm{\theta}}_{\bm{w}}(t)-{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}{\left\lVert{\bm{\theta}}_{\bm{w}}(0)\right\rVert_{2}}=+\infty.$

Moreover, since

$\displaystyle\left(\bm{\theta}_{\bm{w},\bm{z}}(t_{\bar{\alpha}})\right)_{k}$
$\displaystyle=\frac{\nu}{\varepsilon}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}})-\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}})\right)a_{k}^{0}+\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}})+\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}})\right)\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>$
$\displaystyle~{}~{}+\int_{0}^{t_{\bar{\alpha}}}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}}-s)-\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}}-s)\right)f_{k}(s)\mathrm{d}s$
$\displaystyle~{}~{}+\frac{\nu}{\varepsilon}\int_{0}^{t_{\bar{\alpha}}}\left(\frac{1}{{2}}r^{2}(t_{\bar{\alpha}}-s)+\frac{1}{{2}}r^{-2}(t_{\bar{\alpha}}-s)\right)\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\mathrm{d}s,$
$\displaystyle\left(\bm{\theta}_{\bm{w},\bm{z}^{\perp}}(t_{\bar{\alpha}})\right)_{k}$
$\displaystyle=\bm{w}_{k}^{0}-\left<\bm{w}_{k}^{0},\hat{\bm{z}}\right>\hat{\bm{z}}+\frac{\nu}{\varepsilon}\int_{0}^{t}\left[\bm{g}_{k}(s)-\left<\bm{g}_{k}(s),\hat{\bm{z}}\right>\hat{\bm{z}}\right]\mathrm{d}s,$

so we obtain that with probability at least $1-\delta$ over the choice of
$\bm{\theta}^{0}$ and large enough $m$, for any $0\leq
t\leq{t}_{\bar{\alpha}}$, the following holds:

$\displaystyle\frac{1}{m}\left\lVert\bm{\theta}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}$
$\displaystyle\geq
m^{-\gamma^{\prime}-\bar{\alpha}}\left(\frac{1}{2}-\sqrt{d}\max_{k\in[m]}\left\lVert\bm{g}_{k}\right\rVert_{\infty}\right)-\sqrt{1+m^{-\bar{\alpha}}}\left(\frac{3d}{2}+\sqrt{d}\max_{k\in[m]}\left\lvert
f_{k}\right\rvert\right)$ $\displaystyle\geq
m^{-\gamma^{\prime}-\bar{\alpha}}\left(\frac{1}{2}-\sqrt{d}m^{-\widetilde{\tau}}\right)-2\left(\frac{3d}{2}+\sqrt{d}m^{-\widetilde{\tau}}\right),$
$\displaystyle\frac{1}{m}\left\lVert\bm{\theta}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}$
$\displaystyle\leq\frac{3d}{2}+m^{-\gamma^{\prime}-{\bar{\alpha}}}\sqrt{d}\max_{k\in[m]}\left\lVert\bm{g}_{k}\right\rVert_{\infty}\leq\frac{3d}{2}+m^{-\gamma^{\prime}-{\bar{\alpha}}}\sqrt{d}m^{-\widetilde{\tau}}.$

By taking $t={t}_{\bar{\alpha}}$, we observe that
$\frac{1}{m}\left\lVert{\bm{\theta}}_{\bm{w},\bm{z}}(t)\right\rVert^{2}_{2}$
is of order at least $m^{{-\gamma^{\prime}-\bar{\alpha}}}$, while
$\frac{1}{m}\left\lVert{\bm{\theta}}_{\bm{w},\bm{z}^{\perp}}(t)\right\rVert^{2}_{2}$
is of order at most one, which finishes the proof. ∎