# The chiral phase transition and the axial anomaly

Robert D. Pisarski  Department of Physics, Brookhaven National Laboratory,
Upton, NY 11973  Fabian Rennecke  Institute for Theoretical Physics, Justus
Liebig University Giessen, Heinrich-Buff-Ring 16, 35392 Giessen, Germany
Helmholtz Research Academy Hesse for FAIR (HFHF), Campus Giessen, Giessen,
Germany

###### Abstract

To date numerical simulations of lattice QCD have not found a chiral phase
transition of first order which is expected to occur for sufficiently light
pions. We show how the restoration of an exact global chiral symmetry can
strongly decrease the breaking of the approximate, anomalous $U_{A}(1)$
symmetry. This is testable on the lattice through simulations for one through
four flavors. In QCD a small breaking of the $U_{A}(1)$ symmetry in the
chirally symmetric phase generates novel experimental signals.

One of the most beautiful phenomena in quantum field theory is the axial
anomaly of Adler, Bell, and Jackiw [1, 2, 3, 4]. In four spacetime dimensions
massless fermions are chiral, whose spin is either opposite or along the
direction of motion, and so respectively left or right handed. For chirally
symmetric interactions, as with a gauge field, the current for the total
number of fermions, left plus right, is always conserved. In contrast, the
axial current, equal to the difference of the left and right handed currents,
is conserved classically but not quantum mechanically. Instead, the divergence
of the axial current is proportional to the density of the topological charge
for the gauge field.

In the vacuum of Quantum ChromoDynamics (QCD), large fluctuations in the
topological charge explain why the flavor singlet meson, the $\eta^{\prime}$,
is not a Goldstone boson [5, 6, 7]. It also affects other phenomena, albeit
more indirectly [8, 9, 10, 11, 12, 13, 14, 15, 16]. The axial anomaly also
appears in condensed matter systems [17, 18].

The relationship between the divergence of the axial vector current and
topological charge density, computed at one loop order, is exact [1, 3, 4].
Even so, this does not tell one how large the topologically nontrivial
fluctuations are [19, 20]. At zero temperature they must be large in order to
make the $\eta^{\prime}$ heavy. In contrast, at high temperature instantons
are the dominant topologically nontrivial fluctuations [21, 22]. In this limit
the density of instantons can be computed semiclassically, which implies that
the magnitude of the topological charge susceptibility vanishes as a high
power of the temperature $T$, as $T\rightarrow\infty$. Even though it vanishes
at high $T$, it is natural to expect that the density of topologically
nontrivial fluctuations is nonzero for any finite $T$.

This leaves the relationship between the restoration of the exact chiral
symmetry, and the approximate, anomalous $U_{A}(1)$ symmetry, obscure. Based
upon extensive results from numerical simulations in lattice QCD, in this
Letter we outline how the restoration of an exact chiral symmetry strongly
affects the approximate restoration of the anomalous $U_{A}(1)$ symmetry. This
can be tested in lattice QCD with different numbers of flavors, especially for
a single flavor. The approximate restoration of the anomalous $U_{A}(1)$
symmetry has dramatic implications for the collisions of heavy ions, and
surely implications for condensed matter systems as well.

##  I Effective Lagrangians

We consider QCD-like theories, with a $SU(N_{c})$ gauge field coupled to
$N_{f}$ flavors of massless quarks in the fundamental representation. As
massless fields, the Lagrangian is invariant under the global chiral rotations
$q_{L,R}\rightarrow{\rm e}^{i(\theta_{V}\mp\theta_{A})/2}\;U_{L,R}\;q_{L,R}$,
where $q_{L}$ and $q_{R}$ are left and right handed quarks, and $U_{L}$ and
$U_{R}$ elements of the global symmetry groups $SU_{L}(N_{f})$ and
$SU_{R}(N_{f})$, respectively. There are two $U(1)$ groups, one for quark
number, $\theta_{V}$, and one for axial quark number, $\theta_{A}$.

We assume that in vacuum, the exact global chiral symmetry is is characterized
by an expectation value for a color singlet, spin-zero field $\Phi$,
$\Phi=\overline{q}_{L}q_{R}$, where $\Phi$ transforms under the fundamental
representation of the global symmetry group of ${\cal G}_{\rm
cl}=SU_{L}(N_{f})\times SU_{R}(N_{f})\times U_{A}(1)$ as $\Phi\rightarrow{\rm
e}^{i\theta_{A}}\,U_{L}^{\dagger}\,\Phi\,U_{R}.$ As $\Phi$ is invariant under
$U_{V}(1)$, this symmetry can be ignored.

Through the axial anomaly, the $U_{A}(1)$ symmetry is violated quantum
mechnically by topologically non-trivial fluctuations such as instantons. The
exact chiral symmetry which remains is just ${\cal G}_{\rm
qu}=SU_{L}(N_{f})\times SU_{R}(N_{f})$.

In vacuum, it is expected that chiral symmetry breaks to the maximal diagonal
subgroup of $SU_{V}(N_{f})$, $\langle\Phi^{ab}\rangle=\phi_{0}\,\delta^{ab}$,
where $a,b=1\ldots N_{f}$ are the indices for the $SU_{L}(N_{f})$ and
$SU_{R}(N_{f})$ groups. Phenomenologically, this pattern certainly occurs in
QCD, where $N_{c}=3$ and $N_{f}=2$ or $3$. Coleman and Witten proved that it
arises in the limit of large $N_{c}$ and small $N_{f}$ [23].

The appropriate effective Lagrangian for chiral symmetry breaking is well
known [5, 6, 24, 7, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
38, 39, 11, 40, 41]. There are two types of terms which enter. The first type
are terms invariant under ${\cal G}_{\rm cl}$. Up to terms of sixth order in
$\Phi$, these are

$\displaystyle\mathcal{L}_{\rm cl}$ $\displaystyle=$ $\displaystyle{\rm
tr}(\left|\partial_{\mu}\Phi\right|^{2})+m^{2}\,{\rm tr}(\Phi^{\dagger}\Phi)$
(1) $\displaystyle+$ $\displaystyle\lambda_{1}\,\big{(}{\rm
tr}\,(\Phi^{\dagger}\Phi)\big{)}^{2}\,+\lambda_{2}\,{\rm
tr}\big{(}\Phi^{\dagger}\Phi\big{)}^{2}$ $\displaystyle+$
$\displaystyle\kappa_{1}\,\big{(}{\rm
tr}\,(\Phi^{\dagger}\Phi)\big{)}^{3}+\kappa_{2}\,{\rm
tr}\,(\Phi^{\dagger}\Phi)\;{\rm tr}\big{(}\Phi^{\dagger}\Phi\big{)}^{2}$
$\displaystyle\;\;\;\;\;\;\;\;\;\;+\kappa_{3}\,{\rm
tr}(\Phi^{\dagger}\Phi)^{3}\;.$

Our trace is normalized, so ${\rm tr}\,{\bf 1}=1$. For a gauge theory in $3+1$
dimensions, a phase transition at a nonzero temperature $T$ is characterized
by an effective theory in three dimensions. Couplings to sixth order then
represent the relevant operators: the mass squared, $m^{2}$; two quartic
coupling constants, $\lambda_{1}$ and $\lambda_{2}$, with dimensions of mass;
and the six point couplings: $\kappa_{1}$, $\kappa_{2}$, and $\kappa_{3}$,
with dimensionless coupling constants. Terms of eighth and higher order are
irrelevant operators, whose coupling constants have negative mass dimension.

The second class of terms are invariant under ${\cal G}_{\rm qu}$ but not
$U_{A}(1)$, and so are generated by topologically non-trivial fluctuations [5,
6, 7, 11, 12]:

$\displaystyle\mathcal{L}_{\rm qu}$ $\displaystyle=$
$\displaystyle-\xi_{1}\,\big{(}\det\Phi+\det\Phi^{\dagger}\big{)}/2$ (2)
$\displaystyle-$ $\displaystyle\xi_{1}^{(1,1)}\,\left({\rm
tr}\,\Phi^{\dagger}\Phi\right)\big{(}\det\Phi+\det\Phi^{\dagger}\big{)}/2$
$\displaystyle-$
$\displaystyle\xi_{2}\big{[}\big{(}\det\Phi\big{)}^{2}+\big{(}\det\Phi^{\dagger}\big{)}^{2}\big{]}/2\;.$

For three flavors, these are terms to third, fifth, and sixth order in $\Phi$.

The Atiyah-Singer index theorem relates the change in the axial fermion number
to the topological charge as $n_{L}-n_{R}=N_{f}\,Q$. An instanton with
topological charge $Q>0$ has $N_{f}\,Q$ left handed zero modes, $q_{L}$, while
for an anti-instanton with $Q<0$, the quark zero modes are right-handed [42].
Thus the first two terms, $\sim\det\Phi$, arise from instantons with charge
one, [5, 6, 24, 7], while the last term, $\sim(\det\Phi)^{2}$, is due to
instantons with charge two [11, 12].

We comment that instead of the term $\sim\xi_{1}$, for three flavors Refs.
[30, 43, 44, 45] use $\xi_{1}=\xi_{1}^{(1,1)}=0$, and just a single anomalous
coupling, $\sim\xi_{2}^{\prime}\,[(\det\Phi)^{2}-(\det\Phi^{\dagger})^{2}]$.
The operators $\sim\xi_{2}$ and $\sim\xi_{2}^{\prime}$ differ by a term
$\sim\det\Phi^{\dagger}\det\Phi=\det(\Phi^{\dagger}\Phi)$. This operator is
invariant under $U_{A}(1)$, and so for three flavors, the coupling
$\sim\xi_{2}^{\prime}$ is equivalent to that $\sim\xi_{2}$, plus a
modification of the $U_{A}(1)$ invariant couplings of sixth order in Eq. (1).
A similar relation applies for any number of flavors $\geq 2$. We prefer to
use the coupling $\sim\xi_{2}$, as that is uniquely generated by instantons
with charge two.

The anomalous couplings in Eq. (2) are the first terms in an infinite series,

${\cal V}_{\rm qu}(\Phi)=\sum_{i,j,k}\xi^{(j,k)}_{i}\;({\rm
tr}(\Phi^{\dagger}\Phi)^{j})^{k}\left((\det\Phi)^{i}+(\det\Phi^{\dagger})^{i}\right)/2\;.$
(3)

Terms with couplings $\sim\xi^{(j,k)}_{i}$ are generated by fluctuations with
topological charge $|Q|=i$. For ease of notation, we have denoted
$\xi_{i}^{(0,0)}\equiv\xi_{i}$.

##  II A conjecture about anomalous couplings

At the outset we recognize that especially in vacuum, the topologically
nontrivial configurations are surely truly quantum objects, and far from any
semiclassical approximation [20]. For ease of discussion, we refer to the
dominant configurations in vacuum as quantum instantons, and those which
dominate when $T\rightarrow\infty$ as semiclassical instantons. The
contribution of a single semiclassical instanton to the partition function is
$\sim\exp(-8\pi^{2}/g^{2}(T))$, so by asymptotic freedom this falls off as a
high power of the temperature 111For a $SU(N_{c})$ gauge theory coupled to
$N_{f}$ massless flavors of quarks, the topological susceptibility with
massive quarks of mass $m$ falls off as $\sim m^{N_{f}}/T^{(c-4)}$,
$c=(11N_{c}-4N_{f}C_{f})/3$, $C_{f}=(N^{2}-1)/(2N)$. The factor of $c$ is from
the leading order coefficient of the $\beta$-function, and the factor of $-4$
from integration over the instanton scale size. Note that we work in the
chiral limit, where $m=0$ and the topological susceptibility vanishes as
$m^{N_{f}}$.. Numerical simulations of lattice QCD indicate that the
topological susceptibility falls off close to this power down to temperatures
of $T_{\rm qu}\sim 300$ MeV 222The overall magnitude of the topological
susceptibility from a computation of semiclassical instantons to one loop
order is too small by about an order of magnitude, but surely the two loop
computation is called for.. While astonishingly low, this is still about twice
the temperature for the chiral crossover in QCD, at $T_{\chi}\sim 156$ MeV
[48, 49, 50, 51, 52]. Thus we can take $T_{\rm qu}$ as an estimate of the
change from quantum to semiclassical instantons. 333 It is notable that
lattice QCD finds an intermediate region of $T_{\chi}<T<T_{\rm qu}$ [52],
where the quantum instantons have a topological susceptibility different from
$T>T_{\rm qu}$, but the presence of this intermediate regime does not
significantly affect our qualitative analysis. .

The essential question is what is the relative magnitude of the anomalous
coupling constants in vacuum, and as the temperature increases? The standard
assumption with effective Lagrangians is that the couplings with the highest
mass dimension dominate. For the $U_{A}(1)$ symmetric Lagrangian of Eq. (1),
that is the mass squared, followed by the quartic couplings, etc. In the
standard Wilsonian paradigm, this is inescapable, because the only way of
differentiating these different operators is through their mass dimension.

Of course some operators have a larger symmetry than others: $m^{2}{\rm
tr}(\Phi^{\dagger}\Phi)$ and $\lambda_{1}({\rm tr}(\Phi^{\dagger}\Phi))^{2}$
are invariant under $O(2N_{f}^{2})$, while the coupling $\lambda_{2}{\rm
tr}(\Phi^{\dagger}\Phi)^{2}$ is only invariant under $\mathcal{G}_{\rm cl}$.
But this is standard, and doesn’t affect the renormalization group flow
444This includes the possibility that the symmetry is enlarged at a critical
point, as when $\lambda_{1}^{\rm crit}\neq 0$ and $\lambda_{2}^{\rm crit}=0$.
But in computing in the renormalization group flow, both couplings need to be
included.. The only time that couplings of sixth order need to be included is
at isolated points where both $\lambda_{1}$ and $\lambda_{2}$ vanish; then
there is a tricritcal point, controlled by the evolution of the six-point
coupling constants, $\kappa_{1}$, $\kappa_{2}$, and $\kappa_{3}$.

For the anomalous coupling constants, the operator with the lowest mass
dimension is $\xi_{1}\det\Phi$. Thus naively one expects that this operator
dominates the infrared behavior near the chiral phase transition [19].

However, there is something special about the anomalous couplings, which is
not true in standard effective theories. Terms $\sim\det\Phi$ are due,
uniquely, to the zero modes of an instanton with charge one; those
$\sim(\det\Phi)^{2}$, to the zero modes of an instanton with charge two, etc.
[5, 6, 7, 11, 12].

In vacuum, when chiral symmetry breaking occurs the effective coupling for the
first anomalous coupling, $\sim\det\Phi$, is a sum of an infinite number of
terms:

$\xi_{1}^{\rm
eff}(T)=\sum_{i=1}^{\infty}\sum_{j,k=0}^{\infty}i\,\big{(}\phi_{0}(T)\big{)}^{(i-1)N_{f}+2jk}\;\xi_{i}^{(j,k)}(T)\;.$
(4)

As indicated, all of the anomalous coupling constants, the $\xi_{i}^{(j,k)}$,
and the expectation value of the scalar field, $\phi_{0}$, are functions of
temperature. At very high temperature, the anomalous coupling constants
$\xi_{i}^{(j,k)}$ can be computed semiclassically, and are all nonzero [5, 6,
7, 11, 12].

We conjecture the following. In vacuum, the contribution of $\xi_{1}(0)$ to
the total coupling, $\xi_{1}^{\rm eff}(0)$, is small. Instead, terms nominally
of higher order in $\Phi$ in the the effective action are enhanced by
corresponding powers of the chiral condensate, such as
$\sim\phi_{0}(0)^{N_{f}}\xi_{2}(0)$, $\sim\phi_{0}(0)^{2N_{f}}\xi_{3}(0)$,
etc. Our conjecture is that these terms dominate $\xi_{1}(0)$ numerically.

In contrast, in the chirally symmetric phase for $T\geq T_{\chi}$, the chiral
condensate vanishes, $\phi_{0}(T)=0$. For $T>T_{\rm qu}$, the
$\xi_{i}(T)\sim(\xi_{1}(T))^{i}$, and then $\xi_{1}(T)$ certainly dominates
over $\xi_{n\geq 2}(T)$. This is just because in weak coupling, semiclassical
instantons necessarily form a dilute gas [11, 12].

Why should our conjecture be valid? Consider forming an effective Lagrangian
for chiral symmetry breaking from the underlying gauge theory. We integrate
out quarks and gluons to form an effective theory for $\Phi$, over some volume
$\cal{V}_{\chi}$. The essential question is then, what is the distribution of
quantum instantons which contribute in ${\cal V}_{\chi}$?

If in ${\cal V}_{\chi}$ quantum instantons with net charge one dominate, then
so will the operator $\sim\xi_{1}\det\Phi$. If instead ${\cal V}_{\chi}$
predominately contains quantum instantons with net charge two, then the
operator $\sim\xi_{2}(\det\Phi)^{2}$ will be more important. We suggest, then,
that in vacuum quantum instantons with charge two and greater dominate
$\cal{V}_{\chi}$. Of course in all, the topological charge of the vacuum
vanishes. But it need not within a finite volume $\cal{V}_{\chi}$ 555 Lattice
QCD finds that in measuring quantities related to topologically nontrivial
fluctuations, that the autocorrelation times are much larger than in measuring
mass spectra [52]. Our analysis suggests that relative to standard hadronic
mass scales, that quantum instantons are small and/or overlap strongly.
Presumably much finer lattices are required to measure topological
fluctuations..

We now discuss the implications of our conjecture, beginning with the case of
three flavors, which motivated it.

##  III Three flavors

In QCD there is no true phase transition, only a crossover (albeit with a
large increase in the pressure). If $\xi_{1}(T_{\chi})\neq 0$, however, for
three flavors the operator $\sim\det\Phi$ is a cubic operator. The presence of
a cubic operator implies that the standard effective Lagrangian for a second-
order phase transition, with only terms quartic and quadratic in the fields,
cannot be reached, and so the transition is of first order. Hence a chiral
phase transition of first order must emerge for sufficiently light pions,
$m_{\pi}<m_{\pi}^{\rm crit}$ [19]. For simplicity we discuss the case of three
degenerate quark flavors 666In Ref. [112] the Dyson-Schwinger equations for
three flavors are analyzed and it is found that the chiral transition is not
of first order in the chiral limit. This is because this study is limited to
mesonic fluctuations at quadratic order, and so is insensitive to an
instability induced by an operator of cubic order..

How large $m_{\pi}^{\rm crit}$ is depends upon the magnitude of
$\xi_{1}(T_{\chi})$. We suggest that in vacuum the $\eta^{\prime}$ is heavy
not because $\xi_{1}$ is large, but because the higher order terms, such as
$\xi_{2}$, $\xi_{3}$, etc., contribute and overwhelm $\xi_{1}$. At the chiral
phase transition, however, $\phi_{0}=0$, and one is left with just
$\xi_{1}^{\rm eff}(T_{\chi})=\xi_{1}(T_{\chi})$. If $\xi_{1}(T_{\chi})$ is
small, then so is $m_{\pi}^{\rm crit}$.

In mean field theory, it is customary to assume that $\xi_{1}(T)$ is
independent of temperature. Since the $\eta^{\prime}$ is so heavy at zero
temperature, in vacuum $\xi_{1}(0)$ must be large, and $m_{\pi}^{\rm crit}$
should also be large. In a quark meson model, one finds $m_{\pi}^{\rm
crit}\approx 150$ MeV if the vacuum fluctuations of quarks are ignored [57],
and $m_{\pi}^{\rm crit}\approx 86$ MeV if they are included [58]. Similarly,
using mean field theory in a chiral matrix model yields $m_{\pi}^{\rm
crit}\approx 110$ MeV [59].

Going beyond mean field theory, mesonic fluctuations can be included by using
the functional renormalization group. This gives rise to a critical mass which
is dramatically smaller but still nonzero, $m_{\pi}^{\rm crit}\approx 17$ MeV
[58]. Presumably this occurs because the functional renormalization group is
including, at least in part, higher-order anomalous contributions as in Eq.
(4) 777We note that the approximation used in Ref. [58] is known to over-
estimate mesonic fluctuations which tend to soften the phase transition [113].
But this effect alone cannot explain why $m_{\pi}^{\rm crit}$ is so small in
Ref. [58]..

In contrast, no simulation of lattice QCD has ever found evidence of a first
order transition. Instead, they only place upper bounds on $m_{\pi}^{\rm
crit}$, which are much smaller than the values in mean field theory. This
includes: $m_{\pi}^{\rm crit}<50$ MeV in Ref. [61]; $m_{\pi}^{\rm crit}<100$
MeV in Ref. [62]; $m_{\pi}^{\rm crit}<90$ MeV in Ref. [63]. By considering the
position of the tricritical point as a function of $N_{f}$, it has been
asserted in Ref. [64] that even for three flavors, the chiral transition is of
second order in the chiral limit.

We note that a small value of $\xi_{1}(0)$ is perfectly consistent with
hadronic phenomenology at zero temperature, for both hadronic masses and decay
widths. In fact, these quantities can be reproduced successfully in low-energy
models even with $\xi_{2}$ as the only anomalous coupling [30, 31, 43, 44,
45]. This will be analyzed in greater detail in future work [65].

While we assume that $\xi_{1}(T_{\chi})\neq 0$, we stress that we can not
exclude the possibility that $\xi_{1}(T_{\chi})=0$. From the viewpoint of
effective Lagrangians, this is most unnatural, as then two parameters —
$m^{2}(T)$ and $\xi_{1}(T)$ — vanish as one thermodynamic parameter, the
temperature, is varied.

If the result of Ref. [64] holds and the chiral transition is of second order,
then we speculate that not just $\xi_{1}(T_{\chi}),$ but all of the anomalous
couplings vanish at the critical temperature:

$\xi_{i}^{(j,k)}(T_{\chi})=0\;.$ (5)

This implies that the anomalous $U_{A}(1)$ symmetry is restored at $T_{\chi}$.
This can only happen precisely at $T_{\chi}$, since as $T\rightarrow\infty$
semiclassical instantons are present and give $\xi_{i}^{(j,k)}(T)\neq 0$.
Perhaps Eq. (5) is a consequence of the ’t Hooft anomaly condition [66, 67].

If $\xi_{1}(T_{\chi})=0$, the universality class of a second order chiral
phase transition is that of $\mathcal{G}_{\rm cl}$ symmetry. There has been
extensive work on this possibility, including using the $\epsilon$-expansion
[68, 69, 19], perturbation theory in three dimensions [70], Monte Carlo
simulations in three dimensions [71, 72], the functional renormalization group
[41], and the conformal bootstrap [73, 74, 75]. As opposed to earlier results
[68, 69, 19, 70, 71, 72], recent studies with the functional renormalization
group [41] and conformal bootstrap [73, 74, 75] find infrared stable fixed
points for $\mathcal{G}_{\rm cl}$ in three dimensions. Thus if
$\xi_{1}(T_{\chi})=0$, we assume that three massless flavors could have a
chiral transition of second order.

##  IV Two and four flavors

For two flavors, the term $\sim\xi_{1}^{\rm eff}$ is a mass term which splits
the $\eta$ meson from the pions. The couplings $\sim\xi_{1}^{(1,1)}$ and
$\sim\xi_{2}$ are of quartic order. Thus in the chiral limit, $\xi_{1}\neq 0$
implies that the $\eta$ meson is massive at $T_{\chi}$, and the universality
class is that of $\mathcal{G}_{\rm qu}=SU_{L}(2)\times SU_{R}(2)\equiv O(4)$.
Numerical simulations using Wilson fermions by Brandt et al. [76] find that
the mass of the $\eta$ meson is much smaller near $T_{\chi}$ than at $T=0$, in
accord with our conjecture. If the speculation of Eq. (5) holds, then the
$\eta$ meson is massless at $T_{\chi}$, and the universality class is then
$O(4)\times O(2)$.

For four flavors, the coupling $\sim\xi_{1}\det\Phi$ is of quartic order, and
a relevant quartic coupling, of the same mass dimension as the couplings
$\sim\lambda_{1}$ and $\sim\lambda_{2}$. The critical behavior of
$\mathcal{G}_{\rm qu}$ for $N_{f}=4$ is unknown.

##  V One flavor

An interesting test of our conjecture is for a single, massless flavor [25,
28, 29]. Taking $\Phi=\phi+i\eta$, where
$\Phi^{\dagger}\Phi=\phi^{2}+\eta^{2}$, and
$(\det\Phi+\det\Phi^{\dagger})=2\phi$. Including all couplings to quartic
order, the effective Lagrangian is

$\displaystyle\mathcal{L}_{\rm qu}$ $\displaystyle=$
$\displaystyle(\partial_{i}\phi)^{2}+(\partial_{i}\eta)^{2}+\xi_{1}\,\phi$
$\displaystyle+$ $\displaystyle
m^{2}(\phi^{2}+\eta^{2})+\xi_{2}\,(\phi^{2}-\eta^{2})$ $\displaystyle+$
$\displaystyle\xi_{3}\,\phi(\phi^{2}-\eta^{2})+\xi_{1}^{(1,1)}\,\phi(\phi^{2}+\eta^{2})$
$\displaystyle+$ $\displaystyle\lambda(\phi^{2}+\eta^{2})^{2}$
$\displaystyle+$
$\displaystyle\xi_{4}\,(\phi^{4}-6\phi^{2}\eta^{2}+\eta^{4})+\xi_{2}^{(1,1)}\,(\phi^{4}-\eta^{4})\;.$

If $\xi_{1}\neq 0$ , instantons directly induce a vacuum expectation value for
$\phi$.

If our conjecture is correct, then while there may be no true chiral phase
transition, there could well be a sharp crossover from a low temperature
phase, dominated by quantum instantons with large $\xi_{1}(T)$ and
$\phi_{0}(T)$, to a phase dominated by semiclassical instantons, with small
$\xi_{1}(T)$ and $\phi_{0}(T)$. As $T\rightarrow\infty$, $\xi_{1}(T)$ and
$\phi_{0}(T)\rightarrow 0$.

If the speculation of Eq. (5) is true, only $\lambda(T_{\chi})\neq 0$, with
$m^{2}(T_{\chi})$ and all $\xi_{i}^{(j,k)}(T_{\chi})=0$. There is then a
chiral phase transition of second order for an emergent $U_{A}(1)$ symmetry at
$T_{\chi}$. This would be most dramatic.

##  VI Implications for QCD

We have worked exclusively in the chiral limit. What are the implications for
QCD, where numerical simulations on the lattice find no true phase transition,
but crossover [77, 78, 79]?

If QCD is close to the chiral limit for three massless flavors, then the
restoration of the axial $U_{A}(1)$ symmetry at $T_{\chi}$ surely implies that
the approximate restoration of the axial $U_{A}(1)$ symmetry is closely tied
to the crossover temperature.

In numerical simulations of lattice QCD, it is common to measure the violation
of the anomalous $U_{A}(1)$ symmetry by computing the difference in the two
point functions of pions and $a_{0}$ mesons [80, 81, 82, 83, 84, 85]. This is
useful for two light flavors, but since for three flavors a term
$\sim\det\Phi$ is cubic in $\Phi$, when $\phi_{0}=0$ anomalous terms do not
affect mesonic two point functions [86, 87, 88, 89]. In lattice QCD, at
present the situation is unsettled [48, 90]: Refs. [91, 92, 93, 80, 82, 94,
85, 95] find that the anomalous symmetry is not even approximately restored by
$T_{\chi}$, while Refs. [76, 81, 96, 83, 84, 64, 63] find that it is.

Our analysis also applies to nonzero quark chemical potential, $\mu$. For a
theory at $T\neq 0$, the effective theory is three dimensional. If $T\ll\mu$,
though, the relevant effective theory is then in four dimensions. Assuming
that confinement gaps the quarks and gluons, the effective theory is again
that of Eqs. (1) and (2). While the mass dimensions of the coupling constants
change, the conclusion remains that if $\xi_{1}(T_{\chi})\neq 0$, the chiral
phase transition is of first order in the chiral limit.

Our analysis predicts that the breaking of the anomalous $U_{A}(1)$ symmetry
is uniformly small in a chirally symmetric regime. The $\eta^{\prime}$ meson,
which is heavy is vacuum, must become light.

There is an interesting possibility which arises. Like the $U_{A}(1)$
invariant coupling constants, the anomalous coupling constants are all
functions of both temperature and chemical potential,
$\xi_{i}^{(j,k)}(T,\mu)$. Analogous to the critical endpoint, where for two
light flavors the $O(4)$ invariant quartic coupling constant vanishes,
$\lambda(T^{\rm cr},\mu^{\rm cr})=0$ [97, 98, 99, 100], since we have two
thermodynamic parameters to vary, it is possible that there is a single point
in the phase diagram where $\xi_{1}(T^{A},\mu^{A})=0$. About this point,
instead of $SU_{V}(3)$ flavor eigenstates, the $\pi^{0}$, $\eta$, and
$\eta^{\prime}$ are eigenstates of flavor, and there is a large violation of
isospin [19]. It is very intriguing that such a violation has been reported by
the NA61/SHINE collaboration recently [101, 102].

If $\xi_{1}(T,\mu)$ vanishes at a point in the plane of $T$ and $\mu$, then
perhaps there is a region where $\xi_{1}(T,\mu)$ if of opposite sign to that
in the vacuum. If chiral symmetry is broken, then instead of the $\sigma$
meson condensing, the $\eta^{\prime}$ does. This implies that $CP$ symmetry is
spontaneously broken by an $\eta^{\prime}$ condensate.

Other signals which have been suggested include: Hanbury-Brown-Twiss
correlations [103, 104, 105], possibly confirmed by the PHENIX experiment
[106], and an excess of soft dileptons [107]. The HADES experiment finds that
the $\eta$ meson is about twice as abundant as expected from a statistical
distribution [108, 109]. Certainly when the $\eta^{\prime}$ meson becomes
light, so does the $\eta$ meson 888We note that other effects can also
contribute to the $\eta$ meson abundance, cf., e.g., Ref. [114]..

In conclusion, besides the other implications of our results, it is also
natural to wonder how the suppression of topologically nontrivial fluctuations
in a chirally symmetric phase affects baryeogenesis in the early universe
[111].

###### Acknowledgements.

R.D.P. is supported by the U.S. Department of Energy under contract DE-
SC0012704, and thanks the Alexander V. Humboldt Foundation for their support.
F.R. is supported by the Deutsche Forschungsgemeinschaft (DFG, German Research
Foundation) through the Collaborative Research Center TransRegio CRC-TR 211
“Strong- interaction matter under extreme conditions” – project number
315477589 – TRR 211. We thank M. Creutz, T. Csorgo, H. Davoudiasl, T.
Galatyuk, F. Giacosa, K. Intriligator, D. Kaplan, S. Kousvos, G. Kovacs, P.
Kovacs, S. Mukherjee, V. P. Nair, P. Petreczky, O. Philipsen, A. Sorokin, R.
Szafron, and A. Stergiou for discussions.

## References

  * Adler [1969] S. L. Adler, Axial vector vertex in spinor electrodynamics, Phys. Rev. 177, 2426 (1969).
  * Bell and Jackiw [1969] J. S. Bell and R. Jackiw, A PCAC puzzle: $\pi^{0}\to\gamma\gamma$ in the $\sigma$ model, Nuovo Cim. A 60, 47 (1969).
  * Adler and Bardeen [1969] S. L. Adler and W. A. Bardeen, Absence of higher order corrections in the anomalous axial vector divergence equation, Phys. Rev. 182, 1517 (1969).
  * Bardeen [1969] W. A. Bardeen, Anomalous Ward identities in spinor field theories, Phys. Rev. 184, 1848 (1969).
  * ’t Hooft [1976a] G. ’t Hooft, Symmetry Breaking Through Bell-Jackiw Anomalies, Phys. Rev. Lett. 37, 8 (1976a).
  * ’t Hooft [1976b] G. ’t Hooft, Computation of the Quantum Effects Due to a Four-Dimensional Pseudoparticle, Phys. Rev. D 14, 3432 (1976b), [Erratum: Phys.Rev.D 18, 2199 (1978)].
  * ’t Hooft [1986] G. ’t Hooft, How Instantons Solve the U(1) Problem, Phys. Rept. 142, 357 (1986).
  * Veneziano [1989] G. Veneziano, Is There a QCD Spin Crisis?, Mod. Phys. Lett. A 4, 1605 (1989).
  * Shore [2008] G. M. Shore, The $U_{A}(1)$ Anomaly and QCD Phenomenology, Lect. Notes Phys. 737, 235 (2008), arXiv:hep-ph/0701171 .
  * Kharzeev  _et al._ [2016] D. E. Kharzeev, J. Liao, S. A. Voloshin, and G. Wang, Chiral magnetic and vortical effects in high-energy nuclear collisions—A status report, Prog. Part. Nucl. Phys. 88, 1 (2016), arXiv:1511.04050 [hep-ph] .
  * Pisarski and Rennecke [2020] R. D. Pisarski and F. Rennecke, Multi-instanton contributions to anomalous quark interactions, Phys. Rev. D 101, 114019 (2020), arXiv:1910.14052 [hep-ph] .
  * Rennecke [2020] F. Rennecke, Higher topological charge and the QCD vacuum, Phys. Rev. Res. 2, 033359 (2020), arXiv:2003.13876 [hep-th] .
  * Tarasov and Venugopalan [2020] A. Tarasov and R. Venugopalan, Role of the chiral anomaly in polarized deeply inelastic scattering: Finding the triangle graph inside the box diagram in Bjorken and Regge asymptotics, Phys. Rev. D 102, 114022 (2020), arXiv:2008.08104 [hep-ph] .
  * Tarasov and Venugopalan [2022] A. Tarasov and R. Venugopalan, Role of the chiral anomaly in polarized deeply inelastic scattering. II. Topological screening and transitions from emergent axionlike dynamics, Phys. Rev. D 105, 014020 (2022), arXiv:2109.10370 [hep-ph] .
  * Giacosa  _et al._ [2018] F. Giacosa, A. Koenigstein, and R. D. Pisarski, How the axial anomaly controls flavor mixing among mesons, Phys. Rev. D 97, 091901 (2018), arXiv:1709.07454 [hep-ph] .
  * Giacosa  _et al._ [2023] F. Giacosa, S. Jafarzade, and R. Pisarski, Anomalous interactions for heterochiral mesons with $J^{PC}=1^{+-}$ and $2^{-+}$, (2023), arXiv:2309.00086 [hep-ph] .
  * Rylands  _et al._ [2021] C. Rylands, A. Parhizkar, A. A. Burkov, and V. Galitski, Chiral Anomaly in Interacting Condensed Matter Systems, Phys. Rev. Lett. 126, 185303 (2021), arXiv:2102.04371 [cond-mat.str-el] .
  * Fradkin [2023] E. Fradkin, Field Theoretic Aspects of Condensed Matter Physics: An Overview, (2023), arXiv:2301.13234 [cond-mat.str-el] .
  * Pisarski and Wilczek [1984] R. D. Pisarski and F. Wilczek, Remarks on the Chiral Phase Transition in Chromodynamics, Phys. Rev. D 29, 338 (1984).
  * Nair and Pisarski [2023] V. P. Nair and R. D. Pisarski, Fractional topological charge in SU(N) gauge theories without dynamical quarks, Phys. Rev. D 108, 074007 (2023), arXiv:2206.11284 [hep-th] .
  * Gross  _et al._ [1981] D. J. Gross, R. D. Pisarski, and L. G. Yaffe, QCD and Instantons at Finite Temperature, Rev. Mod. Phys. 53, 43 (1981).
  * Boccaletti and Nogradi [2020] A. Boccaletti and D. Nogradi, The semi-classical approximation at high temperature revisited, JHEP 03, 045, arXiv:2001.03383 [hep-ph] .
  * Coleman and Witten [1980] S. R. Coleman and E. Witten, Chiral Symmetry Breakdown in Large N Chromodynamics, Phys. Rev. Lett. 45, 100 (1980).
  * ’t Hooft [1980a] G. ’t Hooft, Confinement and Topology in Nonabelian Gauge Theories, Acta Phys. Austriaca Suppl. 22, 531 (1980a).
  * Leutwyler and Smilga [1992] H. Leutwyler and A. V. Smilga, Spectrum of Dirac operator and role of winding number in QCD, Phys. Rev. D 46, 5607 (1992).
  * Klevansky [1992] S. P. Klevansky, The Nambu-Jona-Lasinio model of quantum chromodynamics, Rev. Mod. Phys. 64, 649 (1992).
  * Jungnickel and Wetterich [1998] D. U. Jungnickel and C. Wetterich, Effective linear meson model, Eur. Phys. J. C 1, 669 (1998), arXiv:hep-ph/9606483 .
  * Creutz [2007] M. Creutz, One flavor QCD, Annals Phys. 322, 1518 (2007), arXiv:hep-th/0609187 .
  * Creutz [2009] M. Creutz, Anomalies and chiral symmetry in QCD, Annals Phys. 324, 1573 (2009), arXiv:0901.0150 [hep-ph] .
  * Parganlija  _et al._ [2013] D. Parganlija, P. Kovacs, G. Wolf, F. Giacosa, and D. H. Rischke, Meson vacuum phenomenology in a three-flavor linear sigma model with (axial-)vector mesons, Phys. Rev. D 87, 014011 (2013), arXiv:1208.0585 [hep-ph] .
  * Kovács and Wolf [2013] P. Kovács and G. Wolf, Meson Vacuum Phenomenology in a Three-flavor Linear Sigma Model with (Axial-)Vector Mesons: Investigation of the U$(1)_{A}$ Anomaly Term, Acta Phys. Polon. Supp. 6, 853 (2013), arXiv:1304.5362 [hep-ph] .
  * Grahl and Rischke [2013] M. Grahl and D. H. Rischke, Functional renormalization group study of the two-flavor linear sigma model in the presence of the axial anomaly, Phys. Rev. D 88, 056014 (2013), arXiv:1307.2184 [hep-th] .
  * Mitter and Schaefer [2014] M. Mitter and B.-J. Schaefer, Fluctuations and the axial anomaly with three quark flavors, Phys. Rev. D 89, 054027 (2014), arXiv:1308.3176 [hep-ph] .
  * Grahl [2014] M. Grahl, $U(2)_{A}\times U(2)_{V}$-symmetric fixed point from the functional renormalization group, Phys. Rev. D 90, 117904 (2014), arXiv:1410.0985 [hep-th] .
  * Rennecke and Schaefer [2017] F. Rennecke and B.-J. Schaefer, Fluctuation-induced modifications of the phase structure in (2+1)-flavor QCD, Phys. Rev. D 96, 016009 (2017), arXiv:1610.08748 [hep-ph] .
  * Fejos [2015] G. Fejos, Functional dependence of axial anomaly via mesonic fluctuations in the three flavor linear sigma model, Phys. Rev. D 92, 036011 (2015), arXiv:1506.07399 [hep-ph] .
  * Bass and Moskal [2019] S. D. Bass and P. Moskal, $\eta$’ and $\eta$ mesons with connection to anomalous glue, Rev. Mod. Phys. 91, 015003 (2019), arXiv:1810.12290 [hep-ph] .
  * Horvatić  _et al._ [2019] D. Horvatić, D. Kekez, and D. Klabučar, $\eta$’ and $\eta$ mesons at high T when the $U_{A}$(1) and chiral symmetry breaking are tied, Phys. Rev. D 99, 014007 (2019), arXiv:1809.00379 [hep-ph] .
  * Gómez Nicola  _et al._ [2019] A. Gómez Nicola, J. Ruiz De Elvira, and A. Vioque-Rodríguez, The QCD topological charge and its thermal dependence: the role of the $\eta$’, JHEP 11, 086, arXiv:1907.11734 [hep-ph] .
  * Braun  _et al._ [2020] J. Braun, M. Leonhardt, J. M. Pawlowski, and D. Rosenblüh (QCD), Chiral and effective $U(1)_{\rm A}$ symmetry restoration in QCD, (2020), arXiv:2012.06231 [hep-ph] .
  * Fejos [2022] G. Fejos, Second-order chiral phase transition in three-flavor quantum chromodynamics?, Phys. Rev. D 105, L071506 (2022), arXiv:2201.07909 [hep-ph] .
  * Brown  _et al._ [1977] L. S. Brown, R. D. Carlitz, and C.-k. Lee, Massless Excitations in Instanton Fields, Phys. Rev. D 16, 417 (1977).
  * Olbrich  _et al._ [2016] L. Olbrich, M. Zétényi, F. Giacosa, and D. H. Rischke, Three-flavor chiral effective model with four baryonic multiplets within the mirror assignment, Phys. Rev. D 93, 034021 (2016), arXiv:1511.05035 [hep-ph] .
  * Divotgey  _et al._ [2018] F. Divotgey, P. Kovacs, F. Giacosa, and D. H. Rischke, Low-energy limit of the extended Linear Sigma Model, Eur. Phys. J. A 54, 5 (2018), arXiv:1605.05154 [hep-ph] .
  * Parganlija and Giacosa [2017] D. Parganlija and F. Giacosa, Excited Scalar and Pseudoscalar Mesons in the Extended Linear Sigma Model, Eur. Phys. J. C 77, 450 (2017), arXiv:1612.09218 [hep-ph] .
  * Note [1] For a $SU(N_{c})$ gauge theory coupled to $N_{f}$ massless flavors of quarks, the topological susceptibility with massive quarks of mass $m$ falls off as $\sim m^{N_{f}}/T^{(c-4)}$, $c=(11N_{c}-4N_{f}C_{f})/3$, $C_{f}=(N^{2}-1)/(2N)$. The factor of $c$ is from the leading order coefficient of the $\beta$-function, and the factor of $-4$ from integration over the instanton scale size. Note that we work in the chiral limit, where $m=0$ and the topological susceptibility vanishes as $m^{N_{f}}$.
  * Note [2] The overall magnitude of the topological susceptibility from a computation of semiclassical instantons to one loop order is too small by about an order of magnitude, but surely the two loop computation is called for.
  * Kotov  _et al._ [2019] A. Y. Kotov, M. P. Lombardo, and A. M. Trunin, Fate of the $\eta$’ in the quark gluon plasma, Phys. Lett. B 794, 83 (2019), arXiv:1903.05633 [hep-lat] .
  * Jahn  _et al._ [2021] P. T. Jahn, P. M. Junnarkar, G. D. Moore, and D. Robaina, Multicanonical reweighting for the QCD topological susceptibility, Phys. Rev. D 104, 014502 (2021), arXiv:2103.01069 [hep-lat] .
  * Borsanyi and Sexty [2021] S. Borsanyi and D. Sexty, Topological susceptibility of pure gauge theory using Density of States, Phys. Lett. B 815, 136148 (2021), arXiv:2101.03383 [hep-lat] .
  * Chen  _et al._ [2022] Y.-C. Chen, T.-W. Chiu, and T.-H. Hsieh (TWQCD), Topological susceptibility in finite temperature QCD with physical (u/d,s,c) domain-wall quarks, Phys. Rev. D 106, 074501 (2022), arXiv:2204.01556 [hep-lat] .
  * Aarts  _et al._ [2023] G. Aarts _et al._ , Phase Transitions in Particle Physics: Results and Perspectives from Lattice Quantum Chromo-Dynamics, Prog. Part. Nucl. Phys. 133, 104070 (2023), arXiv:2301.04382 [hep-lat] .
  * Note [3] It is notable that lattice QCD finds an intermediate region of $T_{\chi}<T<T_{\rm qu}$ [52], where the quantum instantons have a topological susceptibility different from $T>T_{\rm qu}$, but the presence of this intermediate regime does not significantly affect our qualitative analysis.
  * Note [4] This includes the possibility that the symmetry is enlarged at a critical point, as when $\lambda_{1}^{\rm crit}\neq 0$ and $\lambda_{2}^{\rm crit}=0$. But in computing in the renormalization group flow, both couplings need to be included.
  * Note [5] Lattice QCD finds that in measuring quantities related to topologically nontrivial fluctuations, that the autocorrelation times are much larger than in measuring mass spectra [52]. Our analysis suggests that relative to standard hadronic mass scales, that quantum instantons are small and/or overlap strongly. Presumably much finer lattices are required to measure topological fluctuations.
  * Note [6] In Ref. [112] the Dyson-Schwinger equations for three flavors are analyzed and it is found that the chiral transition is not of first order in the chiral limit. This is because this study is limited to mesonic fluctuations at quadratic order, and so is insensitive to an instability induced by an operator of cubic order.
  * Schaefer and Wagner [2009] B.-J. Schaefer and M. Wagner, The Three-flavor chiral phase structure in hot and dense QCD matter, Phys. Rev. D 79, 014018 (2009), arXiv:0808.1491 [hep-ph] .
  * Resch  _et al._ [2019] S. Resch, F. Rennecke, and B.-J. Schaefer, Mass sensitivity of the three-flavor chiral phase transition, Phys. Rev. D 99, 076005 (2019), arXiv:1712.07961 [hep-ph] .
  * [59] V. V. Skokov, Mean field analysis of the chiral phase transition in a chiral matrix model, unpublished.
  * Note [7] We note that the approximation used in Ref. [58] is known to over-estimate mesonic fluctuations which tend to soften the phase transition [113]. But this effect alone cannot explain why $m_{\pi}^{\rm crit}$ is so small in Ref. [58].
  * Bazavov  _et al._ [2017] A. Bazavov, H. T. Ding, P. Hegde, F. Karsch, E. Laermann, S. Mukherjee, P. Petreczky, and C. Schmidt, Chiral phase structure of three flavor QCD at vanishing baryon number density, Phys. Rev. D 95, 074505 (2017), arXiv:1701.03548 [hep-lat] .
  * Kuramashi  _et al._ [2020] Y. Kuramashi, Y. Nakamura, H. Ohno, and S. Takeda, Nature of the phase transition for finite temperature $N_{\rm f}=3$ QCD with nonperturbatively O($a$) improved Wilson fermions at $N_{\rm t}=12$, Phys. Rev. D 101, 054509 (2020), arXiv:2001.04398 [hep-lat] .
  * Dini  _et al._ [2022] L. Dini, P. Hegde, F. Karsch, A. Lahiri, C. Schmidt, and S. Sharma, Chiral phase transition in three-flavor QCD from lattice QCD, Phys. Rev. D 105, 034510 (2022), arXiv:2111.12599 [hep-lat] .
  * Cuteri  _et al._ [2021] F. Cuteri, O. Philipsen, and A. Sciarra, On the order of the QCD chiral phase transition for different numbers of quark flavours, JHEP 11, 141, arXiv:2107.12739 [hep-lat] .
  * [65] F. Giacosa, G. Kovacs, P. Kovacs, R. D. Pisarski, and F. Rennecke, The order of the chiral phase transition and anomalous correlations in a low-energy model.
  * ’t Hooft [1980b] G. ’t Hooft, Naturalness, chiral symmetry, and spontaneous chiral symmetry breaking, NATO Sci. Ser. B 59, 135 (1980b).
  * Fujikawa and Umetsu [2023] K. Fujikawa and K. Umetsu, Berry’s phase and chiral anomalies, Prog. Part. Nucl. Phys. 128, 103992 (2023), arXiv:2209.08334 [nucl-th] .
  * Pisarski and Stein [1981a] R. D. Pisarski and D. L. Stein, Critical Behavior of Linear $\phi^{4}$ Models With $G\times G^{\prime}$ Symmetry, Phys. Rev. B 23, 3549 (1981a).
  * Pisarski and Stein [1981b] R. D. Pisarski and D. L. Stein, The renormalisation group and global G×G’ theories about four dimensions, J. Phys. A 14, 3341 (1981b).
  * Calabrese and Parruccini [2004] P. Calabrese and P. Parruccini, Five loop epsilon expansion for U(n) x U(m) models: Finite temperature phase transition in light QCD, JHEP 05, 018, arXiv:hep-ph/0403140 .
  * Sorokin [2021] A. O. Sorokin, First-order and pseudo-first-order transition in the high dimensional $O(N)\otimes O(M)$ model, (2021), arXiv:2105.00072 [cond-mat.str-el] .
  * Sorokin [2022] A. O. Sorokin, Phase transition in the three-dimensional $O(N)\otimes O(M)$ model: a Monte Carlo study, (2022), arXiv:2205.07199 [hep-lat] .
  * Nakayama and Ohtsuki [2015] Y. Nakayama and T. Ohtsuki, Bootstrapping phase transitions in QCD and frustrated spin systems, Phys. Rev. D 91, 021901 (2015), arXiv:1407.6195 [hep-th] .
  * Henriksson  _et al._ [2020] J. Henriksson, S. R. Kousvos, and A. Stergiou, Analytic and Numerical Bootstrap of CFTs with $O(m)\times O(n)$ Global Symmetry in 3D, SciPost Phys. 9, 035 (2020), arXiv:2004.14388 [hep-th] .
  * Kousvos and Stergiou [2023] S. R. Kousvos and A. Stergiou, CFTs with $\mathbf{U(m)\times U(n)}$ global symmetry in 3D and the chiral phase transition of QCD, SciPost Phys. 15, 075 (2023), arXiv:2209.02837 [hep-th] .
  * Brandt  _et al._ [2016] B. B. Brandt, A. Francis, H. B. Meyer, O. Philipsen, D. Robaina, and H. Wittig, On the strength of the $U_{A}(1)$ anomaly at the chiral phase transition in $N_{f}=2$ QCD, JHEP 12, 158, arXiv:1608.06882 [hep-lat] .
  * Aoki  _et al._ [2006] Y. Aoki, G. Endrodi, Z. Fodor, S. D. Katz, and K. K. Szabo, The Order of the quantum chromodynamics transition predicted by the standard model of particle physics, Nature 443, 675 (2006), arXiv:hep-lat/0611014 .
  * Bazavov  _et al._ [2019] A. Bazavov _et al._ (HotQCD), Chiral crossover in QCD at zero and non-zero chemical potentials, Phys. Lett. B 795, 15 (2019), arXiv:1812.08235 [hep-lat] .
  * Borsanyi  _et al._ [2020] S. Borsanyi, Z. Fodor, J. N. Guenther, R. Kara, S. D. Katz, P. Parotto, A. Pasztor, C. Ratti, and K. K. Szabo, QCD Crossover at Finite Chemical Potential from Lattice Simulations, Phys. Rev. Lett. 125, 052001 (2020), arXiv:2002.02821 [hep-lat] .
  * Dick  _et al._ [2015] V. Dick, F. Karsch, E. Laermann, S. Mukherjee, and S. Sharma, Microscopic origin of $U_{A}(1)$ symmetry violation in the high temperature phase of QCD, Phys. Rev. D 91, 094504 (2015), arXiv:1502.06190 [hep-lat] .
  * Tomiya  _et al._ [2017] A. Tomiya, G. Cossu, S. Aoki, H. Fukaya, S. Hashimoto, T. Kaneko, and J. Noaki, Evidence of effective axial U(1) symmetry restoration at high temperature QCD, Phys. Rev. D 96, 034509 (2017), [Addendum: Phys.Rev.D 96, 079902 (2017)], arXiv:1612.01908 [hep-lat] .
  * Ding  _et al._ [2021] H. T. Ding, S. T. Li, S. Mukherjee, A. Tomiya, X. D. Wang, and Y. Zhang, Correlated Dirac Eigenvalues and Axial Anomaly in Chiral Symmetric QCD, Phys. Rev. Lett. 126, 082001 (2021), arXiv:2010.14836 [hep-lat] .
  * Aoki  _et al._ [2021] S. Aoki, Y. Aoki, G. Cossu, H. Fukaya, S. Hashimoto, T. Kaneko, C. Rohrhofer, and K. Suzuki (JLQCD), Study of the axial $U(1)$ anomaly at high temperature with lattice chiral fermions, Phys. Rev. D 103, 074506 (2021), arXiv:2011.01499 [hep-lat] .
  * Aoki  _et al._ [2022] S. Aoki, Y. Aoki, H. Fukaya, S. Hashimoto, C. Rohrhofer, and K. Suzuki (JLQCD), Role of the axial U(1) anomaly in the chiral susceptibility of QCD at high temperature, PTEP 2022, 023B05 (2022), arXiv:2103.05954 [hep-lat] .
  * Kaczmarek  _et al._ [2021] O. Kaczmarek, L. Mazur, and S. Sharma, Eigenvalue spectra of QCD and the fate of UA(1) breaking towards the chiral limit, Phys. Rev. D 104, 094518 (2021), arXiv:2102.06136 [hep-lat] .
  * Shuryak [1994] E. V. Shuryak, Which chiral symmetry is restored in hot QCD?, Comments Nucl. Part. Phys. 21, 235 (1994), arXiv:hep-ph/9310253 .
  * Cohen [1996] T. D. Cohen, The High temperature phase of QCD and U(1)-A symmetry, Phys. Rev. D 54, R1867 (1996), arXiv:hep-ph/9601216 .
  * Lee and Hatsuda [1996] S. H. Lee and T. Hatsuda, U-a(1) symmetry restoration in QCD with N(f) flavors, Phys. Rev. D 54, R1871 (1996), arXiv:hep-ph/9601373 .
  * Evans  _et al._ [1996] N. J. Evans, S. D. H. Hsu, and M. Schwetz, Topological charge and U(1)-A symmetry in the high temperature phase of QCD, Phys. Lett. B 375, 262 (1996), arXiv:hep-ph/9601361 .
  * Lahiri [2022] A. Lahiri, Aspects of finite temperature QCD towards the chiral limit, PoS LATTICE2021, 003 (2022), arXiv:2112.08164 [hep-lat] .
  * Bazavov  _et al._ [2012] A. Bazavov _et al._ (HotQCD), The chiral transition and $U(1)_{A}$ symmetry restoration from lattice QCD using Domain Wall Fermions, Phys. Rev. D 86, 094503 (2012), arXiv:1205.3535 [hep-lat] .
  * Buchoff  _et al._ [2014] M. I. Buchoff _et al._ , QCD chiral transition, U(1)A symmetry and the dirac spectrum using domain wall fermions, Phys. Rev. D 89, 054514 (2014), arXiv:1309.4149 [hep-lat] .
  * Bhattacharya  _et al._ [2014] T. Bhattacharya _et al._ , QCD Phase Transition with Chiral Quarks and Physical Quark Masses, Phys. Rev. Lett. 113, 082001 (2014), arXiv:1402.5175 [hep-lat] .
  * Kaczmarek  _et al._ [2020] O. Kaczmarek, F. Karsch, A. Lahiri, L. Mazur, and C. Schmidt, QCD phase transition in the chiral limit (2020) arXiv:2003.07920 [hep-lat] .
  * Kaczmarek  _et al._ [2023] O. Kaczmarek, R. Shanker, and S. Sharma, Eigenvalues of QCD Dirac matrix with improved staggered quarks in the continuum limit, (2023), arXiv:2301.11610 [hep-lat] .
  * Ishikawa  _et al._ [2017] K. I. Ishikawa, Y. Iwasaki, Y. Nakayama, and T. Yoshie, Nature of chiral phase transition in two-flavor QCD, (2017), arXiv:1706.08872 [hep-lat] .
  * Asakawa and Yazaki [1989] M. Asakawa and K. Yazaki, Chiral Restoration at Finite Density and Temperature, Nucl. Phys. A 504, 668 (1989).
  * Stephanov  _et al._ [1998] M. A. Stephanov, K. Rajagopal, and E. V. Shuryak, Signatures of the tricritical point in QCD, Phys. Rev. Lett. 81, 4816 (1998), arXiv:hep-ph/9806219 .
  * Stephanov  _et al._ [1999] M. A. Stephanov, K. Rajagopal, and E. V. Shuryak, Event-by-event fluctuations in heavy ion collisions and the QCD critical point, Phys. Rev. D 60, 114028 (1999), arXiv:hep-ph/9903292 .
  * Bzdak  _et al._ [2020] A. Bzdak, S. Esumi, V. Koch, J. Liao, M. Stephanov, and N. Xu, Mapping the Phases of Quantum Chromodynamics with Beam Energy Scan, Phys. Rept. 853, 1 (2020), arXiv:1906.00936 [nucl-th] .
  * Adhikary  _et al._ [2023] H. Adhikary _et al._ (NA61/SHINE), Excess of charged over neutral $K$ meson production in high-energy collisions of atomic nuclei, (2023), arXiv:2312.06572 [nucl-ex] .
  * Brylinski  _et al._ [2023] W. Brylinski, M. Gazdzicki, F. Giacosa, M. Gorenstein, R. Poberezhnyuk, S. Samanta, and H. Stroebele, Large isospin symmetry breaking in kaon production at high energies, (2023), arXiv:2312.07176 [nucl-th] .
  * Vance  _et al._ [1998] S. E. Vance, T. Csorgo, and D. Kharzeev, Partial UA1 restoration from Bose-Einstein correlations, Phys. Rev. Lett. 81, 2205 (1998), arXiv:nucl-th/9802074 .
  * Csorgo  _et al._ [2010] T. Csorgo, R. Vertesi, and J. Sziklai, Indirect observation of an in-medium $\eta$ ’ mass reduction in $\sqrt{s_{NN}}=200$ GeV Au+Au collisions, Phys. Rev. Lett. 105, 182301 (2010), arXiv:0912.5526 [nucl-ex] .
  * Vertesi  _et al._ [2011] R. Vertesi, T. Csorgo, and J. Sziklai, Significant in-medium $\eta$ ’ mass reduction in $\sqrt{s_{NN}}=200$ GeV Au+Au collisions at the BNL Relativistic Heavy Ion Collider, Phys. Rev. C 83, 054903 (2011), arXiv:0912.0258 [nucl-ex] .
  * Adare  _et al._ [2018] A. Adare _et al._ (PHENIX), Lévy-stable two-pion Bose-Einstein correlations in $\sqrt{s_{NN}}=200$ GeV Au$+$Au collisions, Phys. Rev. C 97, 064911 (2018), arXiv:1709.05649 [nucl-ex] .
  * Vargyas  _et al._ [2013] M. Vargyas, T. Csörgő, and R. Vertesi, Effects of chain decays, radial flow and $U_{A}(1)$ restoration on the low-mass dilepton enhancement in $\sqrt{s_{NN}}=200$ GeV Au+Au reactions, Central Eur. J. Phys. 11, 553 (2013), arXiv:1211.1166 [nucl-th] .
  * Agakishiev  _et al._ [2016] G. Agakishiev _et al._ (HADES), Statistical hadronization model analysis of hadron yields in p + Nb and Ar + KCl at SIS18 energies, Eur. Phys. J. A 52, 178 (2016), arXiv:1512.07070 [nucl-ex] .
  * Adamczewski-Musch  _et al._ [2019] J. Adamczewski-Musch _et al._ (HADES), Probing dense baryon-rich matter with virtual photons, Nature Phys. 15, 1040 (2019).
  * Note [8] We note that other effects can also contribute to the $\eta$ meson abundance, cf., e.g., Ref. [114].
  * Bodeker and Buchmuller [2021] D. Bodeker and W. Buchmuller, Baryogenesis from the weak scale to the grand unification scale, Rev. Mod. Phys. 93, 035004 (2021), arXiv:2009.07294 [hep-ph] .
  * Bernhardt and Fischer [2023] J. Bernhardt and C. S. Fischer, QCD phase transitions in the light quark chiral limit, (2023), arXiv:2309.06737 [hep-ph] .
  * Pawlowski and Rennecke [2014] J. M. Pawlowski and F. Rennecke, Higher order quark-mesonic scattering processes and the phase structure of QCD, Phys. Rev. D 90, 076002 (2014), arXiv:1403.1179 [hep-ph] .
  * Larionov and von Smekal [2022] A. B. Larionov and L. von Smekal, Effects of chiral symmetry restoration on meson and dilepton production in relativistic heavy-ion collisions, Phys. Rev. C 105, 034914 (2022), arXiv:2109.03556 [nucl-th] .


# Grassmannian Packings in Neural Networks:
Learning with Maximal Subspace Packings
for Diversity and Anti-Sparsity

Dian Ang Yap
Stanford University
Stanford, CA 94305
<EMAIL_ADDRESS>
&Nicholas Roberts
Carnegie Mellon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>
&Vinay Uday Prabhu
UnifyID AI Labs
Redwood City, CA 94063
<EMAIL_ADDRESS>

###### Abstract

Kernel sparsity (“dying ReLUs”) and lack of diversity are commonly observed in
CNN kernels, which decreases model capacity. Drawing inspiration from
information theory and wireless communications, we demonstrate the
intersection of coding theory and deep learning through the Grassmannian
subspace packing problem in CNNs. We propose Grassmannian packings for initial
kernel layers to be initialized maximally far apart based on chordal or
Fubini-Study distance. Convolutional kernels initialized with Grassmannian
packings exhibit diverse features and obtain diverse representations. We show
that Grassmannian packings, especially in the initial layers, address kernel
sparsity and encourage diversity, while improving classification accuracy
across shallow and deep CNNs with better convergence rates.

##  1 Introduction

Filter level sparsity, commonly known as “dying kernels” or “dying ReLUs” are
common phenomena in Convolutional Neural Networks (CNNs). This phenomenon has
been attributed to adaptive optimizers (Adam, Adagrad, Adadelta), high L2 or
weight decay, and the usage of the ReLU activation function [15, 12, 19].
Attempts to mitigate the dying ReLU issue with Leaky ReLUs have not been
entirely effective, with minor overall impact on the emergent sparsity [15].

Figure 1: Left: sparse filters in CNNs; Right: diverse, non-sparse
Grassmannian kernels.

While kernel sparsity or pruning could benefit CNNs for faster inference and
smaller models [6, 16, 10, 4], sparsity in the lower layers could be
detrimental as the first few layers, especially the first layer, are
“sensitive to initialization” are critical to forming good predictions [21].
Since the initial layers learn simple patterns and subsequent layers build
upon initial layers to develop complex features, sparsity in initial layers
would decrease model capacity.

Besides sparsity, kernel diversity has garnered much attention: Zeiler et. al.
proposed a set of best practices to improve upon Alexnet with different kernel
sizes, stride-lengths and feature-scale clipping [20] to prevent specific
filters from “dominating”. Capturing distinctive features, especially in lower
level layers, ensures that each filter learns something different and improves
computational capacity that the CNNs dispense at the classification
problem[5].

Here, we seek to address two problems, namely sparsity and lack of diversity
of kernels, especially for the most important first few layers. We propose
Grassmannian subspace packings as an initialization to project channel
information onto a space where the basis vectors are maximally distant to
capture as much diversity as possible and increase model capacity. We refer to
the latter as “channel expander” layers. We demonstrate that models with
first-layer Grassmannian kernels achieve better accuracy and convergence rate
with diverse, non-sparse kernels, which improves model capacity.

##  2 Grassmannian line and subspace packings

Grassmannian subspace packing [2] in experimental mathematics and information
theory has been used in beamforming techniques for physical layer wireless
communications, limited feedback codebook-based downlink beamforming schemes
and sparse signal reconstruction [13, 11, 17].

Set theoretic definition. The real Grassmann manifold $G(m,k)$ is define as
the set of all $k$-dimensional subspaces in $\mathbb{R}^{m}$, where
$G(m,k)=\\{W\subset\mathbb{R}^{m}\;|\;\text{dim}(W)=k\\}$. The Grassmannian
N-subspace packing problem is the problem of finding a set of $N$
$k$-dimensional subspaces in $G(m,k)$ that maximize the minimum pairwise
distance between subspaces in the set under some metric [3]. We define the
pairwise distance between any two subspaces in $\mathbb{R}^{m}$ in terms of
principal angles.

Principle angles. Suppose that $\mathcal{S}$ and $\mathcal{T}$ are two
subspaces in $G(m,k)$. These subspaces are inclined against one another by $k$
different principle angles, with the smallest principal angle $\theta_{1}$
formed by a pair of unit-length vectors $(s_{1},t_{1})$ drawn from
$\mathcal{S}\times\mathcal{T}$, where

$\theta_{1}=\inf_{(s_{1},t_{1})\in\mathcal{S}\times\mathcal{T}}\arccos\langle
s_{1},t_{1}\rangle\quad\quad\text{ where $||s_{1}||_{2}=1$ and
$||t_{1}||_{2}=1$}$ (1)

with $\langle x,y\rangle$ denoting the inner product of vectors $x$ and $y$,
and $||x||_{2}$ denoting the $\ell_{2}$ norm metric such that
$||x||_{2}=\langle x,x\rangle$.

The second principle angle $\theta_{2}$ is defined as the smallest angle
obtained by a pair of unit length vectors $(s_{2},t_{2})$ from the same
subspace $\mathcal{S}\times\mathcal{T}$ that is orthogonal to the first pair,
where

$\theta_{2}=\inf_{(s_{2},t21)\in\mathcal{S}\times\mathcal{T}}\arccos\langle
s_{2},t_{2}\rangle$

$\begin{split}\text{where }&||s_{2}||_{2}=1\;,\;||t_{2}||_{2}=1\\\ \text{and
}&\langle s_{1},s_{2}\rangle=0\;,\;\langle t_{1},t_{2}\rangle=0.\end{split}$
(2)

The remaining principle angles $\theta_{3},\dots,\theta_{k}$ are defined
similarly. The sequence of principle angles
$\\{\theta_{1},\dots,\theta_{k}\\}$ is non-decreasing and bounded in
$[0,\pi/2]$.

Optimizing Grassmannians using principle angles. For an $N$ packing problem of
$G(m,k)$, let $\Omega_{m\times k}$ denote the set of subspaces in
$\mathbb{R}^{m\times k}$. We find $\Omega_{m\times k}$ where the minimum
distance between any $\mathbf{w}_{i},\mathbf{w}_{j}\in\Omega_{m\times k}(i\neq
j)$ is maximized. The two popular distance metrics being chordal distance and
Fubini-Study distance, which are defined below. With the restriction that
$k\leq m$, for two $k$-dimensional subspaces $\mathcal{S}$ and $\mathcal{T}$,

$\begin{split}d_{\text{chord}}(\bm{\mathcal{S}},\bm{\mathcal{T}})&\coloneqq\sqrt{\sin^{2}\theta_{1}+\dots+\sin^{2}\theta_{k}}\\\
&=\left[k-||\bm{S}*\bm{T}||_{F}^{2}\right]^{\frac{1}{2}}.\end{split}$ (3)

$\begin{split}d_{\text{FS}}(\bm{\mathcal{S}},\bm{\mathcal{T}})&\coloneqq\arccos\left(\prod_{k}\cos\theta_{k}^{\prime}\right)\\\
&=\arccos|\det\bm{S}*\bm{T}\;|.\end{split}$ (4)

For our experiments, we use generate a codebook matrix for subspace packing
based on Fubini-Study distance which has better experimental
results111https://www.mathworks.com/matlabcentral/fileexchange/41652-grassmannian-
design-package.[14]. We first generate a codebook matrix,
$\mathbf{W}=\\{\mathbf{w}_{1}|\mathbf{w}_{2}|...|\mathbf{w}_{N}\\};\mathbf{w}_{i}\in\Omega_{m\times
k}$. $\mathbf{W}$ is a rank-$k$ codebook characterized by the minimum distance
of packing $\delta(\mathbf{W})$,

$\delta(\mathbf{W})\coloneqq\min_{i\neq j}d(\mathbf{w}_{i},\mathbf{w}_{j})$
(5)

We seek to find $\mathbf{W}=\\{\mathbf{w}_{i}\\}_{i=1}^{N}$ that maximizes
$\delta(\mathbf{W})$,

$\max_{\\{\mathbf{w}_{i}\\},\mathbf{w}_{i}\in G(m,k)}\min_{i\neq
j}\;d(\mathbf{w}_{i},\mathbf{w}_{j})$ (6)

The Rankin bound [1] provides the theoretical upper bound of the maximum of
the minimum pairwise distances between subspaces and is given by,

$\delta({\mathbf{W}})\leq\sqrt{\frac{{(m-k)N}}{{m(N-1)}}}$ (7)

Grassmannian packing seeks to answer "what is the best way to optimally pack N
k-dim subspaces in an M-dim space?". For the first convolutional kernel with
width = 3, height = 3 (9 parameters per channel) with input channels = 3,
output channels = 32, we ask:

"How can we optimally pack 9 3-dim subspace in a 32-dim space,
such that any two subspaces (kernels) are maximally separated?"

##  3 Experiments

We first generate Grassmannian codebooks for subspace packings, and load the
codebook as CNN kernels. We compare accuracy and rate of convergence of
shallow CNNs and deep CNNs. For a shallow 4-conv CNN, we trained 3 models:
Kaiming-initialized CNNs, the same CNN but with Grassmannian as first layer
(frozen and trainable). These models had their first layer initialized as 32
packings of $G(9,3)$ for CIFAR10 and CIFAR100, and 32 packings of $G(9,1)$ for
single-channel MNIST and KMNIST222The first conv layer has parameters (width
3, height 3, 1 or 3 input channels and 32 output channels)..

For deep CNNs, we used ResNet18 architecture and initialized the first layer
with 64 packings of $G(49,3)$333The first layer of ResNets are conv kernels
(width 7, height 7, 3 input channels and 64 output channels).. We trained
ResNet with batch size of 64, SGD optimizer with learning rate of 0.01,
momentum of 0.9, with a learning rate decay by a factor of $\sqrt{10}$ every
10 epochs [8]. Besides having Kaiming-initialized baseline, first-layer
Grassmann and first-layer frozen Grassmann, we also scaled the Grassmann as
per Kaiming init, such that the activations are correctly scaled that
facilitates learning without vanishing gradients [7]. We scale the
Grassmannian basis such that the magnitude is of factor
$\sqrt{2/d_{\text{in}}}$. Since the scaling factor is constant, the directions
of Grassmannian subspace basis vectors are preserved.

We check for convergence rate by examining first-epoch accuracy of shallow
CNNs on MNIST, KMNIST, CIFAR10 and CIFAR100, as well as accuracy, kernel
sparsity and diversity. As adaptive optimizers and weight decay could affect
sparsity, we test on different optimizers such as SGD, Adadelta and Adam, and
with/without weight decay of 0.0001.

##  4 Results and Discussions

Grassmannian models achieve higher accuracy on the 10 class Imagenette [9]
problem with ResNets, even when Grassmannian kernels are frozen upon
initialization. With weight decay, Grassmannian kernels achieve the highest
accuracy out of the four, whereas Kaiming-scaled Grassmannian packings
outperforms the rest without weight decay, as per Table 1.

Table 1: ResNet18 on 10-class ImageNette with SGD optimizer, with and without weight decay.  | Validation Accuracy (%)
---|---
Model | Without Decay | With Decay
Baseline | 90.7 | 90.4
Grassmannian | 92.4 | 91.8
Grassmannian, Frozen | 90.8 | 91.6
Grassmannian, Kaiming-Scaled | 92.8 | 91.5

Grassmannian initialization also improves convergence for both shallow (Figure
6) and Deep CNNs (Figure 3), with both frozen and trainable Grassmannian
models achieving higher first-epoch accuracy (Table 2). Grassmannian subspace
initialization, where each convolutional filter was assigned the subspace
basis vectors for their corresponding Grassmannian packing scheme,
outperformed the baseline on every dataset.

Table 2: Faster Convergence of Shallow CNN, first epoch accuracy. | Validation Accuracy (%)
---|---
Model | MNIST | KMNIST | CIFAR10 | CIFAR100
Baseline | 90.9 | 59.7 | 33.6 | 2.57
Grassmannian, Frozen | 91.9 | 66.2 | 40.5 | 4.73
Grassmannian | 92.6 | 66.4 | 41.1 | 4.70

Interestingly, we see a marginal improvement over Grassmannian initialization
by simply rendering the Grassmannian initialization untrainable. We postulate
that in certain cases, simply projecting the input feature maps to be as far
apart as possible as opposed to using another trainable convolutional layer is
enough for later layers to make better use of their own capacity, at least in
early training.

Qualitative inspection of the kernels (Figure 7) reveal that Grassmannian
kernels learn diverse features with a mixture of edge, color and pattern
detectors in each kernel, without any kernel dominating that shows improved
diversity and fewer sparsity. While baseline and Grassmannian packings gave
mean close to 0, intra-kernel variance and norms are higher for Grassmannian
packings, indicating diversity and fewer sparsity with the distribution across
kernels shown in Figure 2.

---
Figure 2: Distribution of mean, mariance and norm over the 64 kernels of first
layer of ResNet.

Interestingly, trainable Grassmannian packings outperform baseline which
outperforms frozen Grassmannian packings when adaptive optimizers are used
(especially Adam, see Figure 5). Since Adam contributes to sparsity [15, 19],
it is at odds with Grassmannian packings that are against sparsity and for
diversity. Nonetheless, most models such as ResNet, RandWire, NASNet and
DenseNet achieve SOTA results through SGD with a careful learning rate
scheduler, which could further benefit from Grassmannian packings due to their
demonstrated performance using SGD [18].

##  5 Conclusion

In this work, we showed that the use of Grassmannian packings for first layer
initialization improved performance in the contexts of shallow networks and
deep networks, even with untrainable Grassmannian kernels, across multiple
datasets. Grassmannian kernels are best used with SGD optimization and works
with weight decay or otherwise. Our results suggest improvement of the model’s
capacity to learn by simply initializing the first layer filters to be
maximally distant and capture as many diverse features as possible. In the
future, we hope to extend this analysis to explore the several other options
for metric used in the Grassmannian packing, and search for ways to inform the
choice in metric.

### Acknowledgments

We would like to acknowledge Matthew McAteer for helpful discussions and
contrbution of designs and plots for Figures 5 and 6.

## References

  * Barg & Nogin [2002] Barg, A. and Nogin, D. Y.  Bounds on packings of spheres in the grassmann manifold.  _IEEE Transactions on Information Theory_ , 48(9):2450–2454, 2002.
  * Conway et al. [1996] Conway, J. H., Hardin, R. H., and Sloane, N. J.  Packing lines, planes, etc.: Packings in grassmannian spaces.  _Experimental mathematics_ , 5(2):139–159, 1996\.
  * Dhillon et al. [2008] Dhillon, I. S., Heath, J. R., Strohmer, T., and Tropp, J. A.  Constructing packings in grassmannian manifolds via alternating projection.  _Experimental mathematics_ , 17(1):9–35, 2008\.
  * Gale et al. [2019] Gale, T., Elsen, E., and Hooker, S.  The state of sparsity in deep neural networks.  _arXiv preprint arXiv:1902.09574_ , 2019.
  * Graff & Ellen [2016] Graff, C. A. and Ellen, J.  Correlating filter diversity with convolutional neural network accuracy.  In _2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA)_ , pp. 75–80. IEEE, 2016.
  * Han et al. [2015] Han, S., Mao, H., and Dally, W. J.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.  _arXiv preprint arXiv:1510.00149_ , 2015.
  * He et al. [2015] He, K., Zhang, X., Ren, S., and Sun, J.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 1026–1034, 2015.
  * He et al. [2016] He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * [9] Howard, J.  A smaller subset of 10 easily classified classes from imagenet, and a little more french.  _Github repository with links to dataset. https://github.com/fastai/imagenette_.
  * Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H. P.  Pruning filters for efficient convnets.  _arXiv preprint arXiv:1608.08710_ , 2016.
  * Love & Heath [2005] Love, D. J. and Heath, R. W.  Limited feedback unitary precoding for spatial multiplexing systems.  _IEEE Transactions on Information theory_ , 51(8):2967–2976, 2005.
  * Lu et al. [2019] Lu, L., Shin, Y., Su, Y., and Karniadakis, G. E.  Dying relu and initialization: Theory and numerical examples.  _arXiv preprint arXiv:1903.06733_ , 2019.
  * Malioutov et al. [2005] Malioutov, D., Cetin, M., and Willsky, A. S.  A sparse signal reconstruction perspective for source localization with sensor arrays.  _IEEE transactions on signal processing_ , 53(8):3010–3022, 2005.
  * Medra & Davidson [2014] Medra, A. and Davidson, T. N.  Flexible codebook design for limited feedback systems via sequential smooth optimization on the grassmannian manifold.  _IEEE Transactions on Signal Processing_ , 62(5):1305–1318, 2014.
  * Mehta et al. [2019] Mehta, D., Kim, K. I., and Theobalt, C.  On implicit filter level sparsity in convolutional neural networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 520–528, 2019.
  * Molchanov et al. [2016] Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J.  Pruning convolutional neural networks for resource efficient inference.  _arXiv preprint arXiv:1611.06440_ , 2016.
  * Prabhu et al. [2009] Prabhu, V. U., Karachontzitis, S., and Toumpakaris, D.  Performance comparison of limited feedback codebook-based downlink beamforming schemes for distributed antenna systems.  In _2009 1st International Conference on Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology_, pp. 171–176. IEEE, 2009.
  * Wilson et al. [2017] Wilson, A. C., Roelofs, R., Stern, M., Srebro, N., and Recht, B.  The marginal value of adaptive gradient methods in machine learning.  In _Advances in Neural Information Processing Systems_ , pp. 4148–4158, 2017.
  * Yaguchi et al. [2018] Yaguchi, A., Suzuki, T., Asano, W., Nitta, S., Sakata, Y., and Tanizawa, A.  Adam induces implicit weight sparsity in rectifier neural networks.  In _2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)_ , pp. 318–325. IEEE, 2018.
  * Zeiler & Fergus [2014] Zeiler, M. D. and Fergus, R.  Visualizing and understanding convolutional networks.  In _European conference on computer vision_ , pp. 818–833. Springer, 2014.
  * Zhang et al. [2019] Zhang, C., Bengio, S., and Singer, Y.  Are all layers created equal?  _arXiv preprint arXiv:1902.01996_ , 2019.

## Appendix

---
Figure 3: ResNet with and without weight decay, with Grassmannian achieving
better convergence and better final accuracy with everything else kept
constant. Last image: zooming into last few epochs for ResNet with weight
decay. Figure 4: Illustration of how the first conv layer is initialized with
Grassmannians, with visualizations of Grassmannian packings. Left: 5-packing
of $G(3,1)$, which answers “how should 5 laser beams passing through a single
point in $\mathbb{R}^{3}$ be arranged so as to make the minimum principle
angle between any two of the beams as large as possible". Right: 2-packing of
$G(3,2)$. These concepts can be extended to higher dimensions.

(a) SGD

(b) Adadelta

(c) Adam

Figure 5: Distributions of 30 runs of first-epoch test accuracy at the first
epoch with different optimizers on the same shallow CNN architecture,
comparing initialization of first layer using standard Xavier initialization,
frozen Grassmannian packings, and trainable Grassmannian packings.

(a) MNIST

(b) KMNIST

(c) CIFAR10

(d) CIFAR100

Figure 6: Distributions of 30 runs of first-epoch test accuracy at the first
epoch with SGD with different datasets, comparing initialization of first
layer using standard Xavier initialization, frozen Grassmannian packings, and
trainable Grassmannian packings.

(a) Grassmannian, no weight decay.

(b) Kaiming-scaled Grassmannian, no weight decay.

(c) Grassmannian, weight decay.

(d) Kaiming-scaled Grassmannian, weight decay.

Figure 7: Visualizations of Trainable Grassmannian Kernels of ResNet trained using SGD, with and without weight decay after training for 80 epochs on ImageNette. Table 3: Final Accuracy of ResNet-56 on CIFAR-10 with first layer Xavier, Grassmannian trainable, and Grassmannian Frozen. Initialization | Test Acc
---|---
Standard, Xavier | 91.85
Grassmannian, Fixed | 91.98
Grassmannian, Trainable | 92.31
Figure 8: Comparison of standard initialization and Grassmannian
initialization of first layer as both trainable and untrainable parameters on
ResNet56 trained on CIFAR-10. Grassmannian approaches have a faster
convergence with marginally better test accuracy with Adam optimizer used in
all 3 cases.


* (38) M. Drewes et al., A White Paper on keV Sterile Neutrino Dark Matter, JCAP 01 (2017) 025, [arXiv:1602.04816].
  * (39) K. Perez, K. C. Y. Ng, J. F. Beacom, C. Hersh, S. Horiuchi, and R. Krivonos, Almost closing the $\nu$MSM sterile neutrino dark matter window with NuSTAR, Phys. Rev. D 95 (2017), no. 12 123002, [arXiv:1609.00667].
  * (40) B. M. Roach, K. C. Y. Ng, K. Perez, J. F. Beacom, S. Horiuchi, R. Krivonos, and D. R. Wik, NuSTAR Tests of Sterile-Neutrino Dark Matter: New Galactic Bulge Observations and Combined Impact, Phys. Rev. D 101 (2020), no. 10 103011, [arXiv:1908.09037].
  * (41) J. W. Foster, M. Kongsore, C. Dessert, Y. Park, N. L. Rodd, K. Cranmer, and B. R. Safdi, Deep Search for Decaying Dark Matter with XMM-Newton Blank-Sky Observations, Phys. Rev. Lett. 127 (2021), no. 5 051101, [arXiv:2102.02207].
  * (42) A. G. Cohen and D. B. Kaplan, Thermodynamic Generation of the Baryon Asymmetry, Phys. Lett. B 199 (1987) 251–258.
  * (43) S. Weinberg, Baryon and Lepton Nonconserving Processes, Phys. Rev. Lett. 43 (1979) 1566–1570.
  * (44) A. Kusenko, K. Schmitz, and T. T. Yanagida, Leptogenesis via Axion Oscillations after Inflation, Phys. Rev. Lett. 115 (2015), no. 1 011302, [arXiv:1412.2043].
  * (45) R. T. Co, N. Fernandez, A. Ghalsasi, L. J. Hall, and K. Harigaya, Lepto-Axiogenesis, JHEP 03 (2021) 017, [arXiv:2006.05687].
  * (46) D. J. Gross, J. A. Harvey, E. J. Martinec, and R. Rohm, The Heterotic String, Phys. Rev. Lett. 54 (1985) 502–505.
  * (47) L. J. Dixon, J. A. Harvey, C. Vafa, and E. Witten, Strings on Orbifolds, Nucl. Phys. B 261 (1985) 678–686.
  * (48) L. J. Dixon, J. A. Harvey, C. Vafa, and E. Witten, Strings on Orbifolds. 2., Nucl. Phys. B 274 (1986) 285–314.
  * (49) L. E. Ibanez, H. P. Nilles, and F. Quevedo, Orbifolds and Wilson Lines, Phys. Lett. B 187 (1987) 25–32.
  * (50) O. Lebedev, H. P. Nilles, S. Raby, S. Ramos-Sanchez, M. Ratz, P. K. S. Vaudrevange, and A. Wingerter, A Mini-landscape of exact MSSM spectra in heterotic orbifolds, Phys. Lett. B 645 (2007) 88–94, [hep-th/0611095].
  * (51) M. Blaszczyk, S. Groot Nibbelink, M. Ratz, F. Ruehle, M. Trapletti, and P. K. S. Vaudrevange, A Z2xZ2 standard model, Phys. Lett. B 683 (2010) 340–348, [arXiv:0911.4905].
  * (52) V. Braun, Y.-H. He, B. A. Ovrut, and T. Pantev, A Heterotic standard model, Phys. Lett. B 618 (2005) 252–258, [hep-th/0501070].
  * (53) V. Bouchard and R. Donagi, An SU(5) heterotic standard model, Phys. Lett. B 633 (2006) 783–791, [hep-th/0512149].
  * (54) L. B. Anderson, J. Gray, A. Lukas, and E. Palti, Heterotic Line Bundle Standard Models, JHEP 06 (2012) 113, [arXiv:1202.1757].
  * (55) M. Cvetic, T. Li, and T. Liu, Supersymmetric patiSalam models from intersecting D6-branes: A Road to the standard model, Nucl. Phys. B 698 (2004) 163–201, [hep-th/0403061].
  * (56) F. Gmeiner, R. Blumenhagen, G. Honecker, D. Lust, and T. Weigand, One in a billion: MSSM-like D-brane statistics, JHEP 01 (2006) 004, [hep-th/0510170].
  * (57) R. Blumenhagen, V. Braun, T. W. Grimm, and T. Weigand, GUTs in Type IIB Orientifold Compactifications, Nucl. Phys. B 815 (2009) 1–94, [arXiv:0811.2936].
  * (58) B. S. Acharya, M theory, Joyce orbifolds and superYang-Mills, Adv. Theor. Math. Phys. 3 (1999) 227–248, [hep-th/9812205].
  * (59) J. Halverson and D. R. Morrison, On gauge enhancement and singular limits in G2 compactifications of M-theory, JHEP 04 (2016) 100, [arXiv:1507.05965].
  * (60) A. Grassi, J. Halverson, J. Shaneson, and W. Taylor, Non-Higgsable QCD and the Standard Model Spectrum in F-theory, JHEP 01 (2015) 086, [arXiv:1409.8295].
  * (61) J. Halverson and W. Taylor, ${\mathrm{\mathbb{P}}}^{1}$-bundle bases and the prevalence of non-Higgsable structure in 4D F-theory models, JHEP 09 (2015) 086, [arXiv:1506.03204].
  * (62) W. Taylor and Y.-N. Wang, A Monte Carlo exploration of threefold base geometries for 4d F-theory vacua, JHEP 01 (2016) 137, [arXiv:1510.04978].
  * (63) B. S. Acharya, M. Fairbairn, and E. Hardy, Glueball dark matter in non-standard cosmologies, JHEP 07 (2017) 100, [arXiv:1704.01804].
  * (64) E. D. Carlson, M. E. Machacek, and L. J. Hall, Self-interacting dark matter, Astrophys. J. 398 (1992) 43–52.
  * (65) J. Halverson, B. D. Nelson, and F. Ruehle, String Theory and the Dark Glueball Problem, Phys. Rev. D 95 (2017), no. 4 043527, [arXiv:1609.02151].
  * (66) A. E. Faraggi and M. Pospelov, Selfinteracting dark matter from the hidden heterotic string sector, Astropart. Phys. 16 (2002) 451–461, [hep-ph/0008223].
  * (67) J. L. Feng and Y. Shadmi, WIMPless Dark Matter from Non-Abelian Hidden Sectors with Anomaly-Mediated Supersymmetry Breaking, Phys. Rev. D 83 (2011) 095011, [arXiv:1102.0282].
  * (68) K. K. Boddy, J. L. Feng, M. Kaplinghat, and T. M. P. Tait, Self-Interacting Dark Matter from a Non-Abelian Hidden Sector, Phys. Rev. D 89 (2014), no. 11 115017, [arXiv:1402.3629].
  * (69) A. Soni and Y. Zhang, Hidden SU(N) Glueball Dark Matter, Phys. Rev. D 93 (2016), no. 11 115025, [arXiv:1602.00714].
  * (70) G. D. Kribs and E. T. Neil, Review of strongly-coupled composite dark matter models and lattice simulations, Int. J. Mod. Phys. A 31 (2016), no. 22 1643004, [arXiv:1604.04627].
  * (71) N. Yamanaka, H. Iida, A. Nakamura, and M. Wakayama, Dark matter scattering cross section and dynamics in dark Yang-Mills theory, Phys. Lett. B 813 (2021) 136056, [arXiv:1910.01440].
  * (72) J. Halverson, B. D. Nelson, F. Ruehle, and G. Salinas, Dark Glueballs and their Ultralight Axions, Phys. Rev. D 98 (2018), no. 4 043502, [arXiv:1805.06011].
  * (73) J. Halverson and P. Langacker, TASI Lectures on Remnants from the String Landscape, PoS TASI2017 (2018) 019, [arXiv:1801.03503].
  * (74) B. S. Acharya, S. A. R. Ellis, G. L. Kane, B. D. Nelson, and M. Perry, Categorisation and Detection of Dark Matter Candidates from String/M-theory Hidden Sectors, JHEP 09 (2018) 130, [arXiv:1707.04530].
  * (75) A. Soni, H. Xiao, and Y. Zhang, Cosmic selection rule for the glueball dark matter relic density, Phys. Rev. D 96 (2017), no. 8 083514, [arXiv:1704.02347].
  * (76) T. Cohen, K. Murase, N. L. Rodd, B. R. Safdi, and Y. Soreq, $\gamma$ -ray Constraints on Decaying Dark Matter and Implications for IceCube, Phys. Rev. Lett. 119 (2017), no. 2 021102, [arXiv:1612.05638].
  * (77) P. Asadi, E. D. Kramer, E. Kuflik, G. W. Ridgway, T. R. Slatyer, and J. Smirnov, Thermal squeezeout of dark matter, Phys. Rev. D 104 (2021), no. 9 095013, [arXiv:2103.09827].
  * (78) A. Kusenko, M. Loewenstein, and T. T. Yanagida, Moduli dark matter and the search for its decay line using Suzaku X-ray telescope, Phys. Rev. D 87 (2013), no. 4 043508, [arXiv:1209.6403].
  * (79) T. Gherghetta, N. Nagata, and M. Shifman, A Visible QCD Axion from an Enlarged Color Group, Phys. Rev. D 93 (2016), no. 11 115010, [arXiv:1604.01127].
  * (80) M. K. Gaillard, M. B. Gavela, R. Houtz, P. Quilez, and R. Del Rey, Color unified dynamical axion, Eur. Phys. J. C 78 (2018), no. 11 972, [arXiv:1805.06465].
  * (81) C. Murgui and K. M. Zurek, Dark unification: A UV-complete theory of asymmetric dark matter, Phys. Rev. D 105 (2022), no. 9 095002, [arXiv:2112.08374].
  * (82) R. N. Mohapatra, Supersymmetric grand unification, in Theoretical Advanced Study Institute in Elementary Particle Physics (TASI 97): Supersymmetry, Supergravity and Supercolliders, pp. 601–657, 6, 1997.  hep-ph/9801235.
  * (83) J. Halverson, C. Long, B. Nelson, and G. Salinas, Axion reheating in the string landscape, Phys. Rev. D 99 (2019), no. 8 086014, [arXiv:1903.04495].
  * (84) G. Grilli di Cortona, E. Hardy, J. Pardo Vega, and G. Villadoro, The QCD axion, precisely, JHEP 01 (2016) 034, [arXiv:1511.02867].
  * (85) J. E. Kim, Weak Interaction Singlet and Strong CP Invariance, Phys. Rev. Lett. 43 (1979) 103.
  * (86) M. A. Shifman, A. I. Vainshtein, and V. I. Zakharov, Can Confinement Ensure Natural CP Invariance of Strong Interactions?, Nucl. Phys. B 166 (1980) 493–506.
  * (87) L. Di Luzio, M. Giannotti, E. Nardi, and L. Visinelli, The landscape of QCD axion models, Phys. Rept. 870 (2020) 1–117, [arXiv:2003.01100].
  * (88) M. Dine, W. Fischler, and M. Srednicki, A Simple Solution to the Strong CP Problem with a Harmless Axion, Phys. Lett. B 104 (1981) 199–202.
  * (89) A. R. Zhitnitsky, On Possible Suppression of the Axion Hadron Interactions. (In Russian), Sov. J. Nucl. Phys. 31 (1980) 260.
  * (90) P. Agrawal, M. Nee, and M. Reig, Axion Couplings in Grand Unified Theories, arXiv:2206.07053.
  * (91) J. L. Ouellet et al., First Results from ABRACADABRA-10 cm: A Search for Sub-$\mu$eV Axion Dark Matter, Phys. Rev. Lett. 122 (2019), no. 12 121802, [arXiv:1810.12257].
  * (92) C. P. Salemi et al., The search for low-mass axion dark matter with ABRACADABRA-10cm, arXiv:2102.06722.
  * (93) L. Brouwer et al., Introducing DMRadio-GUT, a search for GUT-scale QCD axions, arXiv:2203.11246.
  * (94) DMRadio Collaboration, L. Brouwer et al., DMRadio-m3: A Search for the QCD Axion Below $1\,\mu$eV, arXiv:2204.13781.
  * (95) M. Bauer, M. Neubert, S. Renner, M. Schnubel, and A. Thamm, Axionlike Particles, Lepton-Flavor Violation, and a New Explanation of $a_{\mu}$ and $a_{e}$, Phys. Rev. Lett. 124 (2020), no. 21 211803, [arXiv:1908.00008].
  * (96) M. Srednicki, Axion Couplings to Matter. 1. CP Conserving Parts, Nucl. Phys. B 260 (1985) 689–700.
  * (97) S. Chang and K. Choi, Hadronic axion window and the big bang nucleosynthesis, Phys. Lett. B 316 (1993) 51–56, [hep-ph/9306216].
  * (98) C. Dessert, A. J. Long, and B. R. Safdi, No evidence for axions from Chandra observation of magnetic white dwarf, arXiv:2104.12772.
  * (99) H. Liu, W. Qin, G. W. Ridgway, and T. R. Slatyer, Lyman-$\alpha$ constraints on cosmic heating from dark matter annihilation and decay, Phys. Rev. D 104 (2021), no. 4 043514, [arXiv:2008.01084].
  * (100) N. Blinov, M. J. Dolan, P. Draper, and J. Kozaczuk, Dark matter targets for axionlike particle searches, Phys. Rev. D 100 (2019), no. 1 015049, [arXiv:1905.06952].
  * (101) Planck Collaboration, N. Aghanim et al., Planck 2018 results. VI. Cosmological parameters, Astron. Astrophys. 641 (2020) A6, [arXiv:1807.06209]. [Erratum: Astron.Astrophys. 652, C4 (2021)].
  * (102) M. Kawasaki, K. Kohri, and N. Sugiyama, MeV scale reheating temperature and thermalization of neutrino background, Phys. Rev. D 62 (2000) 023506, [astro-ph/0002127].
  * (103) S. Hannestad, What is the lowest possible reheating temperature?, Phys. Rev. D 70 (2004) 043506, [astro-ph/0403291].
  * (104) L. Forestell, D. E. Morrissey, and K. Sigurdson, Non-Abelian Dark Forces and the Relic Densities of Dark Glueballs, Phys. Rev. D 95 (2017), no. 1 015032, [arXiv:1605.08048].
  * (105) Y. Chen et al., Glueball spectrum and matrix elements on anisotropic lattices, Phys. Rev. D 73 (2006) 014516, [hep-lat/0510074].
  * (106) J. E. Juknevich, Pure-glue hidden valleys through the Higgs portal, JHEP 08 (2010) 121, [arXiv:0911.5616].
  * (107) A. Athenodorou and M. Teper, SU(N) gauge theories in 3+1 dimensions: glueball spectrum, string tensions and topology, JHEP 12 (2021) 082, [arXiv:2106.00364].
  * (108) C. Bonanno, M. D’Elia, B. Lucini, and D. Vadacchino, Towards glueball masses of large-$N$$\mathrm{SU}(N)$ pure-gauge theories without topological freezing, arXiv:2205.06190.
  * (109) J. Halverson, C. Long, A. Maiti, B. Nelson, and G. Salinas, Gravitational waves from dark Yang-Mills sectors, JHEP 05 (2021) 154, [arXiv:2012.04071].
  * (110) B. Lucini, A. Rago, and E. Rinaldi, SU($N_{c}$) gauge theories at deconfinement, Phys. Lett. B 712 (2012) 279–283, [arXiv:1202.6684].
  * (111) C. G. Callan, Jr., R. F. Dashen, and D. J. Gross, Toward a Theory of the Strong Interactions, Phys. Rev. D 17 (1978) 2717.
  * (112) A. Boyarsky, A. Neronov, O. Ruchayskiy, M. Shaposhnikov, and I. Tkachev, Where to find a dark matter sterile neutrino?, Phys. Rev. Lett. 97 (2006) 261302, [astro-ph/0603660].
  * (113) K. C. Y. Ng, B. M. Roach, K. Perez, J. F. Beacom, S. Horiuchi, R. Krivonos, and D. R. Wik, New Constraints on Sterile Neutrino Dark Matter from $NuSTAR$ M31 Observations, Phys. Rev. D 99 (2019) 083005, [arXiv:1901.01262].
  * (114) R. Laha, J. B. Muñoz, and T. R. Slatyer, INTEGRAL constraints on primordial black holes and particle dark matter, Phys. Rev. D 101 (2020), no. 12 123514, [arXiv:2004.00627].
  * (115) R. Essig, E. Kuflik, S. D. McDermott, T. Volansky, and K. M. Zurek, Constraining Light Dark Matter with Diffuse X-Ray and Gamma-Ray Observations, JHEP 11 (2013) 193, [arXiv:1309.4091].
  * (116) R. Bartels, D. Gaggero, and C. Weniger, Prospects for indirect dark matter searches with MeV photons, JCAP 05 (2017) 001, [arXiv:1703.02546].
  * (117) K. Nandra et al., The Hot and Energetic Universe: A White Paper presenting the science theme motivating the Athena+ mission, arXiv:1306.2307.
  * (118) C. Thorpe-Morgan, D. Malyshev, A. Santangelo, J. Jochum, B. Jäger, M. Sasaki, and S. Saeedi, THESEUS insights into axionlike particles, dark photon, and sterile neutrino dark matter, Phys. Rev. D 102 (2020), no. 12 123003, [arXiv:2008.08306].
  * (119) Particle Data Group Collaboration, P. A. Zyla et al., Review of Particle Physics, PTEP 2020 (2020), no. 8 083C01.
  * (120) V. Domcke, Y. Ema, K. Mukaida, and M. Yamada, Spontaneous Baryogenesis from Axions with Generic Couplings, JHEP 08 (2020) 096, [arXiv:2006.03148].
  * (121) D. Baumann, D. Green, and B. Wallisch, New Target for Cosmic Axion Searches, Phys. Rev. Lett. 117 (2016), no. 17 171301, [arXiv:1604.08614].
  * (122) L. D. McLerran, E. Mottola, and M. E. Shaposhnikov, Sphalerons and Axion Dynamics in High Temperature QCD, Phys. Rev. D 43 (1991) 2027–2035.
  * (123) J. A. Harvey and M. S. Turner, Cosmological baryon and lepton number in the presence of electroweak fermion number violation, Phys. Rev. D 42 (1990) 3344–3349.
  * (124) Planck Collaboration, Y. Akrami et al., Planck 2018 results. X. Constraints on inflation, Astron. Astrophys. 641 (2020) A10, [arXiv:1807.06211].
  * (125) S. Davidson, E. Nardi, and Y. Nir, Leptogenesis, Phys. Rept. 466 (2008) 105–177, [arXiv:0802.2962].
  * (126) V. Domcke, K. Kamada, K. Mukaida, K. Schmitz, and M. Yamada, A new constraint on primordial lepton flavour asymmetries, arXiv:2208.03237.
  * (127) L. Visinelli and P. Gondolo, Dark Matter Axions Revisited, Phys. Rev. D 80 (2009) 035024, [arXiv:0903.4377].
  * (128) Y. Kawamura, Triplet doublet splitting, proton stability and extra dimension, Prog. Theor. Phys. 105 (2001) 999–1006, [hep-ph/0012125].
  * (129) L. J. Hall and Y. Nomura, Gauge unification in higher dimensions, Phys. Rev. D 64 (2001) 055003, [hep-ph/0103125].
  * (130) A. Hebecker and J. March-Russell, A Minimal S**1 / (Z(2) x Z-prime (2)) orbifold GUT, Nucl. Phys. B 613 (2001) 3–16, [hep-ph/0106166].
  * (131) Y. Nomura, D. Tucker-Smith, and N. Weiner, GUT breaking on the brane, Nucl. Phys. B 613 (2001) 147–166, [hep-ph/0104041].
  * (132) H. Georgi and S. L. Glashow, Unity of All Elementary Particle Forces, Phys. Rev. Lett. 32 (1974) 438–441.
  * (133) K.-w. Choi, A QCD axion from higher dimensional gauge field, Phys. Rev. Lett. 92 (2004) 101602, [hep-ph/0308024].
  * (134) A. L. Erickcek and K. Sigurdson, Reheating Effects in the Matter Power Spectrum and Implications for Substructure, Phys. Rev. D 84 (2011) 083503, [arXiv:1106.0536].
  * (135) G. Barenboim and J. Rasero, Structure Formation during an early period of matter domination, JHEP 04 (2014) 138, [arXiv:1311.4034].
  * (136) J. Fan, O. Özsoy, and S. Watson, Nonthermal histories and implications for structure formation, Phys. Rev. D 90 (2014), no. 4 043536, [arXiv:1405.7373].
  * (137) A. E. Nelson and H. Xiao, Axion Cosmology with Early Matter Domination, Phys. Rev. D 98 (2018), no. 6 063516, [arXiv:1807.07176].
  * (138) L. Visinelli and J. Redondo, Axion Miniclusters in Modified Cosmological Histories, Phys. Rev. D 101 (2020), no. 2 023008, [arXiv:1808.01879].
  * (139) J. A. Dror, H. Ramani, T. Trickle, and K. M. Zurek, Pulsar Timing Probes of Primordial Black Holes and Subhalos, Phys. Rev. D 100 (2019), no. 2 023003, [arXiv:1901.04490].
  * (140) V. S. H. Lee, A. Mitridate, T. Trickle, and K. M. Zurek, Probing Small-Scale Power Spectra with Pulsar Timing Arrays, JHEP 06 (2021) 028, [arXiv:2012.09857].
  * (141) L. Dai and J. Miralda-Escudé, Gravitational Lensing Signatures of Axion Dark Matter Minihalos in Highly Magnified Stars, Astron. J. 159 (2020), no. 2 49, [arXiv:1908.01773].
  * (142) P. Schwaller, Gravitational Waves from a Dark Phase Transition, Phys. Rev. Lett. 115 (2015), no. 18 181101, [arXiv:1504.07263].
  * (143) W.-C. Huang, M. Reichert, F. Sannino, and Z.-W. Wang, Testing the dark SU(N) Yang-Mills theory confined landscape: From the lattice to gravitational waves, Phys. Rev. D 104 (2021), no. 3 035005, [arXiv:2012.11614].
  * (144) J. F. Navarro, C. S. Frenk, and S. D. M. White, The Structure of cold dark matter halos, Astrophys. J. 462 (1996) 563–575, [astro-ph/9508025].
  * (145) J. F. Navarro, C. S. Frenk, and S. D. M. White, A Universal density profile from hierarchical clustering, Astrophys. J. 490 (1997) 493–508, [astro-ph/9611107].
  * (146) P. F. de Salas and A. Widmark, Dark matter local density determination: recent observations and future prospects, Rept. Prog. Phys. 84 (2021), no. 10 104901, [arXiv:2012.11477].
  * (147) H. W. Leung, J. Bovy, J. T. Mackereth, J. A. S. Hunt, R. R. Lane, and J. C. Wilson, A direct measurement of the distance to the Galactic center using the kinematics of bar stars, arXiv e-prints (Apr., 2022) arXiv:2204.12551, [arXiv:2204.12551].
  * (148) A. W. Strong and I. V. Moskalenko, Propagation of cosmic-ray nucleons in the galaxy, Astrophys. J. 509 (1998) 212–228, [astro-ph/9807150].
  * (149) G. Cowan, K. Cranmer, E. Gross, and O. Vitells, Asymptotic formulae for likelihood-based tests of new physics, Eur. Phys. J. C 71 (2011) 1554, [arXiv:1007.1727]. [Erratum: Eur.Phys.J.C 73, 2501 (2013)].
  * (150) G. D. Moore and M. Tassler, The Sphaleron Rate in SU(N) Gauge Theory, JHEP 02 (2011) 105, [arXiv:1011.1167].
  * (151) B. Garbrecht and P. Schwaller, Spectator Effects during Leptogenesis in the Strong Washout Regime, JCAP 10 (2014) 012, [arXiv:1404.2915].