aainstitutetext: Department of Physics, Sinop University,
Korucuk, 57000 Sinop, Turkeybbinstitutetext: Department of Mechanical and
Aerospace Eng., West Virginia University, Morgantown, 26506-6101 West
Virginia, USAccinstitutetext: Department of Chemistry, Sinop University,
Korucuk, 57000 Sinop, Turkey

# Emergent entropic nature of fundamental interactions

Emre Dil111Corresponding author. b,c and Tugrul Yumak<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we generalize Verlinde’s entropic gravity proposal on other
fundamental interactions of nature. We begin by introducing the entropic
origin of Coulomb’s electrostatic force and then the magnetic force, by
assuming the holographic principle holds for a charged particle approaching a
screen enclosing the emerged part of the spacetime due to this source, whose
entropy changes through the approaching charge. Thereafter, we obtain the
entropic Maxwell equations in both classical and covariant form by means of
the holograpic principles for the source. Considering the gauge covariant
structure of the fundamental forces, we implicitly generalize the entropic
origination of electromagnetic force into the strong and weak forces. The
possibility of being entropic origination of the all fundamental forces of
nature makes also a sensible contribution to the unification scheme of the
fundamental forces.

##  1 Introduction

Gravity has begun to be considered as an entropic force rather than a
fundamental force with the pioneering work of Verlinde and others a ; b ; c ;
d . According to the Verlinde’s proposal, gravity and spacetime can emerge
from a microscopic description whose specific details are independent from the
general principles of the theory. The main cause deriving the gravity is the
amount of information measured as entropy due to the displacement of matter.
Reaction force to the change of entropy during the displacement of the matter
leads to the gravitational force.

The information stored in a spacetime emerged by a source matter obeys the
holographic principle that a finite number of degrees of freedom (bits) are
associated with the information in the volume. The number of bits on the
holographic screen enclosing the emergent part of the spacetime, is
proportional to the surface area of the holographic screen e ; f . The total
energy due to the source mass is evenly distributed over the bits on the
holographic boundary with that the source leads a certain temperature on it.
On the other hand, the main motivation of Verlinde’s proposal comes from the
Bekenstein’s assumption g  for his entropy formula. Similar to way Bekenstein
argued a Planck mass approaches the horizon of a black hole by one Planck
length leads to an increase of one bit of information, as $2\pi k$, on the
surface area of the horizon, Verlinde proposed a mass approaching a
holographic screen leads to the same amount of entropy change on the screen.
Consequently, the product of the change of entropy and the temperature on the
screen gives the work done by the emergent gravitational force, in Verlinde’s
proposal.

Verlinde obtained the Newton’s law of motion and the gravitational field
equations in the framework of entropic origination and it could motivate
Freund to ask whether the electromagnetic force and the other fundamental
forces can be described as a force having an entropic origin h . He could
qualitatively extend the entropic origin of gravity into abelian and non-
abelian gauge fields. In order to bring a quantitative description for Freund
arguments, we therefore first begin with electrostatic force by applying
Verlinde’s holographic argument on a charge based entropy change and energy
occurrence on the boundary screen of the emergent spacetime due to another
source charge. After obtaining the electrostatic force and Coulomb’s force
equation, we proceed to get the magnetic field equation for a charge current.
We then generalize the entropic origin of electromagnetic field equations to
the classical and covariant form of Maxwell equations as the abelian gauge
field equations in order to construct a bridge for obtaining the entropic
origination of the non-abelian strong and weak field interactions in gauge
field theories structure. The success of the improving the entropic
origination of all the fundamental forces of nature is also though to make a
significant contribution to the unification studies of the four fundamental
forces, in the physics community.

##  2 Entropic origin of electric and magnetic forces

Entropic origin of gravity leads to investigate whether the other forces of
nature can be originated as an entropic force in the literature h ; i ; j .
Although its importance, few leading studies attempt to deduce the entropic
origin of electromagnetic forces. The most significant one in j  tries to
generalize electromagnetic interaction with the steps taken by Verlinde, and
there resulted a bizarre impairing in the physical significance of the formal
thermodynamical interpretation by occurrence of negative electromagnetic
temperature.

In this study, we overcome this inconsistency by repairing the description of
the energy on the holographic screen due to the source charge. We begin with
the assumption that a test particle of Planck charge $q_{p}$ approaches to the
holographic screen from a direction at which spacetime has already emerged due
to a source charge $q_{2}$ which can even be written in terms of the Planck
units. Similar to Bekenstein-Verlinde argument, the change of entropy on the
holographic boundary of the emerged spacetime is $2\pi k$, as the test charge
approaches to the screen by one Compton wavelength of Planck length $l_{p}$:

$\Delta S=2\pi k\,,\qquad q_{p}=\sqrt{4\pi\epsilon_{0}\hbar c}\,.$ (1)

Accordingly, the thermodynamical work done on the test charge due to the
entropy change during the displacement is

$\begin{split}Fl_{P}&=T\Delta S\,\\\ &=T2\pi
k\frac{q_{p}}{\sqrt{4\pi\epsilon_{0}\hbar c}}\,.\end{split}$ (2)

Stated that the source charge leads to the emerging of spacetime which is
bounded by the holographic screen and a temperature on it, this temperature
makes the test particle gain an acceleration with the Unruh law:

$kT=\frac{1}{2\pi}\frac{\hbar}{c}a\,.$ (3)

Substituting the Unruh temperature (3) into (2) gives

$\begin{split}F&=\frac{T2\pi k}{l_{p}}\frac{q_{p}}{\sqrt{4\pi\epsilon_{0}\hbar
c}}\,\\\ &=\frac{\hbar a}{cl_{p}}\frac{q_{p}}{\sqrt{4\pi\epsilon_{0}\hbar
c}}\,\\\ &=\frac{\sqrt{\hbar
c}}{{l_{p}}^{2}}\frac{q_{p}}{\sqrt{4\pi\epsilon_{0}}}\,.\end{split}$ (4)

where we use $a=c/t_{p}$ and $l_{p}=c/t_{p}$ in the second line in terms of
the Planck units, and the first factor in the last line results from the
temperature of the holographic screen due to the source charge $q_{2}$;
therefore, it turns out to be

$F=\frac{q_{2}}{4\pi\epsilon_{0}l^{2}}q_{p}=q_{p}E\,.$ (5)

It is the electrostatic force on a test charge in an electric field $E$
produced by the change of information on the emergent holographic spacetime
boundary, due to a source charge leading to a temperature on it.

###  2.1 Coulomb’s force

We now proceed to repair the inconsistency in j  while describing the total
energy on the emergent spherically symmetric spacetime boundary. Since
Verlinde considers the gravity, he takes the $Mc^{2}$ energy into account as
the reasoning effect for the temperature $T$ on the holographic boundary.
However, Wang makes a direct analogy by replacing the charge $Q$ with the mass
$M$, as the energy source, and takes $Qc^{2}$ into account. Whereas, this
quantity does not define an energy on the holographic screen. We, therefore,
repair this inconsistency by taking the Planck energy $\hbar/t_{p}$ as the
source energy due to the charge in Planck units and this energy results a
temperature $T$ on the screen as

$E_{p}=\frac{\hbar}{t_{p}}=\frac{1}{2}NkT\,,$ (6)

where $N$ is the total number of bits on the spherically symmetric holographic
screen, and it is again proportional to the surface area of the screen, such
that

$N=\frac{A}{c^{2}{t_{p}}^{2}}\,.$ (7)

Since power in Planck units is $\hbar/{t_{p}}^{2}=c^{5}/G$ for the source
charge $q_{2}$, (6) can be rewritten as

$q_{2}\sqrt{\frac{c^{4}}{4\pi\epsilon_{0}G}}=\frac{1}{2}NkT\,,$ (8)

and

$q_{2}\sqrt{\frac{\hbar}{4\pi\epsilon_{0}c{t_{p}}^{2}}}=\frac{1}{2}kT\frac{A}{c^{2}{t_{p}}^{2}}\,,$
(9)

which yields the temperature on the screen, then

$T=\frac{2q_{2}}{kA}\sqrt{\frac{\hbar c^{3}{t_{p}}^{2}}{4\pi\epsilon_{0}}}\,.$
(10)

Now we can obtain the force on a test particle with charge $q_{p}$ approaching
this spherically symmetric spacetime boundary emerged due to the source charge
$q_{2}$ by a distance of one Compton wavelength as Planck length $l_{p}$.
Then, the entropy change on the screen is again $2\pi k$ by Bekenstein-
Verlinde argument, and the thermal work done on the test particle in terms of
the entropy change is

$Fl_{p}=T\Delta S=\frac{2q_{2}}{kA}\sqrt{\frac{\hbar
c^{3}{t_{p}}^{2}}{4\pi\epsilon_{0}}}2\pi
k\frac{q_{p}}{\sqrt{4\pi\epsilon_{0}\hbar c}}\,,$ (11)

where we have used the screen temperature (10). Simplifying (11) gives

$F=\frac{1}{4\pi\epsilon_{0}}\frac{q_{p}q_{2}}{{l_{p}}^{2}}\,,$ (12)

which is obviously the Coulomb’s force originating from the entropy change on
a holographic screen due to the displacement of a test charge under the
influence of the temperature led by the source charge.

###  2.2 Entropic origin of Biot-Savart law

For this case, we assume that the source producing the energy and temperature
on the emergent spacetime boundary is a charge current, and we attempt to
obtain the force on another charge current close to the boundary of the
spacetime. Therefore, the temperature of the screen due to the source current
is obtained by reinterpreting (8) with the fact that
$\epsilon_{0}\mu_{0}=c^{-2}$, such as

$q_{2}\sqrt{\frac{\mu_{0}}{4\pi}\frac{\hbar
c}{{t_{p}}^{2}}}=\frac{1}{2}kT\frac{A}{c^{2}{t_{p}}^{2}}\,,$ (13)

which gives

$T=\frac{2I_{2}}{kA}\sqrt{\frac{\mu_{0}}{4\pi}\hbar c^{5}{t_{p}}^{4}}\,.$ (14)

where we have used $I_{2}=q_{2}/t_{p}$ for the source current. Then the
entropy change due to the approaching test current $I_{1}$ leads to the work,

$Fl=T2\pi kI_{1}\sqrt{\frac{\mu_{0}}{4\pi}\frac{c{t_{p}}^{2}}{\hbar}}\,.$ (15)

Using (13) here, we obtain

$F=\frac{\mu_{0}}{4\pi}\frac{I_{2}l}{r^{2}}I_{1}l=BI_{1}l\,.$ (16)

and

$B=\frac{\mu_{0}}{4\pi}\frac{I_{2}l}{r^{2}}\,.$ (17)

Here, (17) is the Biott-Savart law and $B$ is apparently the magnitude of the
magnetic field due to the current $I_{2}$ and inferred from the factor
standing in the middle of (16).

As it is clear that the Planck scale is only valid for the quantum world, but
here we only assume that the quantities in classical Biott-Savart law are
converted into Planc units as conventional unit conversions. This approach has
also been followed by Verlinde in his entropic gravity study a  as offering to
assume the classical quantities in gravitational law equations to be converted
into Planck units as in the conventional unit conversion method. This does not
mean to treat the macroscobic objects as quantum objects. This does just mean
to mimic the conventional unit conversion method.

##  3 Derivation of Maxwell’s equations through entropic origin

We firstly begin with the derivation of Gauss’s law, by assuming the source
charge causes the thermal energy which is randomly divided over the $N$ bits
on the spherical holographic screen enclosing the emergent part of the
spacetime:

$E=\frac{1}{2}\int{kTdN}$ (18)

Using Planck power in order to convert $E$ in terms of the electrostatic
energy and Unruh law gives

$q_{2}\sqrt{\frac{\hbar}{4\pi\epsilon_{0}c{t_{p}}^{2}}}=\int{\frac{\hbar}{4\pi
c^{2}{t_{p}}^{3}}dA}\,,$ (19)

where $dN=dA/c^{2}{t_{p}}^{2}$ is used. We then write the charge in terms of
the charge density, such as

$\int{\frac{\rho}{\epsilon_{0}}dV}=\int{\sqrt{\frac{\hbar
c}{4\pi\epsilon_{0}c^{4}{t_{p}}^{4}}}dA}\,.$ (20)

Applying the divergence theorem on the right hand side of (20) leads to

$\begin{split}\int{\frac{\rho}{\epsilon_{0}}dV}&=\int{\nabla\bullet\left(\sqrt{\frac{\hbar
c}{4\pi\epsilon_{0}l^{4}}}\hat{l}\right)dV}\,\\\
&=\int{\nabla\bullet\left(\frac{q_{2}}{4\pi\epsilon_{0}l^{2}}\hat{l}\right)dV}\,.\end{split}$
(21)

where $\bullet$ represents the scalar dot product. This is obviously
equivalent to the Gauss’s law as

$\nabla\bullet E=\frac{\rho}{\epsilon_{0}}\,.$ (22)

Similarly, one can obtain the emergent Ampere’s law for a source of current
producing a temperature on the holographic screen due to the magnetic energy
which is again divided over the $N$ bits, as follows

$E=q_{2}\frac{\mu_{0}}{4\pi}Ic=\frac{1}{2}\int{kTdN}$ (23)

where $\mu_{0}Ic/4\pi$ is the impedance due to the source current $I=q_{2}/t$.
Rewriting the Unruh temperature on the left hand side and the current in terms
of the surface current density gives

$\int{\frac{q_{2}c}{4\pi}\mu_{0}J\bullet dA}=\int{\frac{\hbar}{4\pi
c^{2}{t_{p}}^{3}}dA}\,,$ (24)

and rearranging (24) yields

$\int{\mu_{0}J\bullet
dA}=\int{\left(\frac{\mu_{0}}{4\pi}\frac{Il^{\prime}}{l^{3}}\hat{n}\right)\bullet
dA}\,,$ (25)

where $\hat{n}$ is parallel to $dA$. The term in the paranthesis can be
assumed as the rotational of a vector $1/l^{2}$ perpendicular to the direction
of $\hat{n}$, such as

$\int{\mu_{0}J\bullet
dA}=\int{\nabla\times\left(\frac{\mu_{0}}{4\pi}\frac{Il^{\prime}}{l^{2}}\hat{n}_{\bot}\right)\bullet
dA}\,.$ (26)

We obviously reach the emergent Ampere’s law, since the term in parenthesis is
the magnetic field due to the source current, so

$\nabla\times B=\mu_{0}J\,.$ (27)

Combining the processes to obtain the results in (22) and (27) leads to the
Maxwell equations. To proceed, we express the energy on the holographic screen
enclosing the emergent part of the spacetime due to the source charge $q_{2}$
inspired by (19), as follows

$q_{2}=\int{\frac{\hbar c}{4\pi l^{3}\phi}dA}\,,$ (28)

where $\phi$ is the potential due to the source charge. In order to obtaina
Lorentz covariant relativistic form of the emergent field equations, we need
to combine equation (28) both for a rest charge and a current, such that

$\frac{q_{2}}{\epsilon_{0}}=\int{\frac{1}{4\pi\epsilon_{0}}\frac{\hbar c}{\phi
l^{3}}dA}\,,$ (29) $\mu_{0}I=\int{\frac{\mu_{0}}{4\pi}\frac{\hbar c}{\phi
l^{3}t_{p}}dA}\,.$ (30)

Combining these separate equations into one single covariant form means that
there occurs one time-like and one space-like components of the same equation.
For a 4-current $I^{\mu}=(cq_{2},I)$, we can combine them as

$\mu_{0}I^{0}=\int{F_{\alpha}^{0}dA^{\alpha}}\,,$ (31)
$\mu_{0}I^{i}=\int{\partial_{\alpha}{F^{\alpha i}}dA^{i}}\,,$ (32)

where $F^{\alpha\beta}$ is the electromagnetic field strength tensor which
corresponds to the emergent field, as follows

$\mu_{0}I^{0}=\int{\mu_{0}c\rho dV}=\int{\partial_{\alpha}{F^{\alpha
0}}dV}\,,$ (33)
$\mu_{0}I^{i}=\int{\mu_{0}J^{i}dA^{i}}=\int{\partial_{\alpha}{F^{\alpha
i}}dA^{i}}\,.$ (34)

Bringing them into a single covariant form leads to the Maxwell equations,
such that

$\left(\nabla\bullet\left[\frac{1}{4\pi\epsilon_{0}}\frac{\hbar c}{\phi
l^{2}}\hat{l}\right],\nabla\times\left[\frac{\mu_{0}}{4\pi}\frac{\hbar c}{\phi
l^{2}t_{p}}\hat{n}_{\bot}\right]\right)=\left(\partial_{\alpha}{F^{\alpha
0}},\partial_{\alpha}{F^{\alpha i}}\right)$
$\left(\frac{1}{4\pi\epsilon_{0}}\frac{\hbar c}{\phi
l^{3}},\frac{\mu_{0}}{4\pi}\frac{\hbar c}{\phi
l^{3}t_{p}}\hat{n}\right)=\partial_{\alpha}{F^{\alpha\beta}}\,,$ (35)

then substituting this into (33) and (34), we obtain

$\mu_{0}J^{\beta}=\partial_{\alpha}{F^{\alpha\beta}}\,,$ (36)

which is the inhomogeneous Maxwell equations. Since the electromagnetic
interaction is a gauge field theory like the strong and weak interactions as
the fundamental interactions of nature, we can infer the emergent nature of
these two forces by following the similar steps taken for the electromagnetic
interaction.

##  4 Entropic origin of the strong and weak forces

The main difference between electromagnetic, and nuclear strong and weak
forces are the non-abelian nature of the gauge fields of the nuclear forces,
while it is abelian for the electromagnetic force. Despite the gauge symmetry
of strong and weak forces differ as $SU(3)$ and $SU(2)$, respectively, this
does not change the fundamental entropic structure of the interactions. The
similarity of the gauge structure of these three fields yields to propose the
existence of entropic origin for two nuclear forces, once the existence of
entropic origin is clearly expressed for the electromagnetic one.

We need to define the strong and weak charges in Planck units in order to
proceed. It is known that the electric charge is defined in terms of the
coupling strength (fine structure constant) $\alpha_{e}$ in Planck units as
$q_{p}=\sqrt{4\pi\epsilon_{0}\hbar c}=e/\sqrt{\alpha_{e}}$. From this fact the
strong and weak charges are

$q_{s}=\frac{g_{s}}{\sqrt{\alpha_{s}}}\,,\qquad
q_{w}=\frac{g_{w}}{\sqrt{\alpha_{w}}}\,,$ (37)

Here, the coupling strength values for the strong and weak interactions are
$\alpha_{s}=g_{s}^{2}/4\pi$ and $\alpha_{w}=g_{w}^{2}/4\pi$, respectively,
which makes (37) turn out to be

$q_{s}=q_{s}=\sqrt{4\pi}\,,$ (38)

We then assume that a source charge $q_{sw}$ leads to a thermal energy
distributed over the $N$ bits on the holographic screen enclosing the emergent
part of the spacetime due to the existence of the source. Accordingly, the
thermal energy can be expressed in terms of a nuclear potential $A_{sw}$, as

$\begin{split}E=q_{sw}A_{sw}&=\frac{1}{2}\int{kTdN}\,\\\
&=\int{\frac{1}{4\pi}\frac{\hbar c}{l^{3}}dA}\,,\end{split}$ (39)

and

$q_{sw}=\int{\frac{1}{4\pi}\frac{\hbar c}{A_{sw}l^{3}}dA}\,.$ (40)

Since there is no permittivity or permeability terms for strong and weak
interactions, from the symmetry of the structures of the gauge field theories
we postulate that (40) must directly lead to the field equations of nuclear
forces, such that

$D_{\mu}G^{\mu\nu}=gJ^{\nu}\,,$ (41)

where $G^{\mu\nu}$ is the field-strength tensor for the corresponding nuclear
force and should satisfy

$D_{\mu}G^{\mu\nu}=\frac{1}{4\pi}\frac{\hbar c}{A_{sw}l^{3}}\,.$ (42)

Here, we postulate that the $A_{\nu}$ is the 4-potential for the strong and
weak nuclear forces obeying

$G_{\mu\nu}=\partial_{\mu}{A_{\nu}}-\partial_{\nu}{A_{\mu}}+g_{sw}A_{\mu}\times
A_{\nu}\,.$ (43)

These are the non-abelian version of the emergent covariant Maxwell equations
with the covariant derivative $D_{\mu}$.

##  5 Conclusions

In this study, we attempt to describe all the fundamental interactions of
nature as originated from an entropic behavior of the source and test items,
under the influence of Freund’s proposal c  based on the Verlinde’s entropic
gravity approach a . For this purpose, we first start to investigate the
entropic origin of the electromagnetic interaction and compare the results
with the Wang’s results j  in which he obtains a weird inconsistency in the
physical significance of the thermodynamical interpretation by occurrence of
negative temperature. We repair this inconsistency by taking the Planck energy
$\hbar/t$ instead of the Wang’s $Qc^{2}$ value that is in fact not a quantity
defining an energy, for the energy value on the holographic screen causing the
temperature $T$ on the screen. This energy and temperature on the screen due
to the source charge are seen to induce an entropic electrostatic force (11)
on a test charge approaching the screen from the emerged part of the spacetime
due to the source charge. Furthermore, by generalizing the situation for a
moving charge we obtain the emergent magnetic force (15) on a test current due
to the source current inducing an energy and temperature on the screen
enclosing the spacetime emerged by itself.

After obtaining the emergent forms of the Coulomb’s and Biot-Savart laws, we
generalize their entropic origin on electrostatic and magnetic fields by
Gauss’s and Ampere’s law, respectively. As a direct result, we get the
emergent nature of the classical Maxwell equations. Since the covariant
relativistic form of the Maxwell equations lead to the abelian gauge field
theories, we obtain emergent form of the covariant Maxwell equations which
will provide a connection for the emergent nature of the non-abelian gauge
field theories. From this connection, we postulate the emergent nature of the
strong and weak forces with the color and isospin charge-based screen energy
and temperature on a test charge, which leads to the entropic strong and weak
interactions’ field equations (41).

Consequently, the idea that the emergence of gravity and spacetime is entropic
gives us some clues on all the fundamental forces of nature can be originated
due to the entropy changes by certain type of entities, in certain type of
limiting conditions after the big bang. This possibility makes us think that
the universe, as a whole, can be considered as a gas of particles both in
galactic scale and in quarks scale, since the observer’s reference frame is
possible to be taken to so far away from the galactic sample, so that the
observer can infer as if they are Planck sized entities, relatively.
Accordingly, considering the equivalence of the size scales of the entities of
the universe through the observers relative reference position, it sounds
quite sensible the entropic origination of the fundamental forces of nature
which are believed to be the descendant of a grand unified force.

##  6 Conflict of Interest

The authors declare that there is no conflict of interest regarding the
publication of this paper.

## References

  * (1) E. Verlinde, _On the origin of gravity and the laws of Newton_ , _JHEP_ 04 (2011) 029.
  * (2) E. Dil, _q-Deformed Einstein equations_ , _Can. J. Phys._ 93 (2015) 1274.
  * (3) E. Dil and E. Kolay, _Solution of deformed Einstein equations and quantum black holes_ , _Adv. High Energy Phys._ 2016 (2016) 3973706.
  * (4) E. Dil, _Space creation mechanism during the expansion of universe_ , _Adv. Astron._ 2016 (2016) 4695065.
  * (5) G. ’t Hooft, _Dimensional reduction in quantum gravity_ , arxiv: gr-qc/9310026.
  * (6) L. Susskind, _The World as a hologram_ , _J. Math. Phys._ 36 (1995) 6377.
  * (7) J.D. Bekenstein, _Black holes and entropy_ , _Phys. Rev. D_ 7 (1973) 2333.
  * (8) P. Freund, _Emergent gauge fields_ , arxiv: 1008.4147 [hep-th].
  * (9) O. Domènech, M. Montull, A. Pomarol, et al., _Emergent gauge fields in holographic superconductors_ , _JHEP_ 08 (2010) 033.
  * (10) T. Wang, _Coulomb force as an entropic force_ , _Phys. Rev. D_ 81 (2010) 104045.


Metric Flows with Neural Networks

James<EMAIL_ADDRESS>Fabian
<EMAIL_ADDRESS>
${}^{\text{a}}$ Department of Physics, Northeastern University, Boston, MA
02115, USA
${}^{\text{b}}$ Department of Mathematics, Northeastern University, Boston, MA
02115, USA
${}^{\text{c}}$ NSF Institute for Artificial Intelligence and Fundamental
Interactions

###### Abstract

We develop a theory of flows in the space of Riemannian metrics induced by
neural network gradient descent. This is motivated in part by recent advances
in approximating Calabi-Yau metrics with neural networks and is enabled by
recent advances in understanding flows in the space of neural networks. We
derive the corresponding metric flow equations, which are governed by a metric
neural tangent kernel, a complicated, non-local object that evolves in time.
However, many architectures admit an infinite-width limit in which the kernel
becomes fixed and the dynamics simplify. Additional assumptions can induce
locality in the flow, which allows for the realization of Perelman’s
formulation of Ricci flow that was used to resolve the 3d Poincaré conjecture.
We apply these ideas to numerical Calabi-Yau metrics, including a discussion
on the importance of feature learning.

###### Contents

  1. 1 Introduction
  2. 2 Metric Flows with Neural Networks
    1. 2.1 General Theory and Metric-NTK
    2. 2.2 Infinite Neural Network Metric Flows
    3. 2.3 Local Metric Flows
      1. 2.3.1 Architecture Design for Local Metric Flow
      2. 2.3.2 Concrete Architectures for Local and Non-local Metric Flows
    4. 2.4 Ricci Flow and More General Flows as Neural Network Metric Flows
  3. 3 Numerical Implementation
    1. 3.1 Clustering and Preprocessing
    2. 3.2 Metrics from Kernels
  4. 4 Conclusions
  5. A Primer on Neural Networks
  6. B Kähler-Ricci flows for String Theory

##  1 Introduction

There are no known nontrivial compact Calabi-Yau metrics, objects of central
importance in string theory and algebraic geometry, despite decades of study.

The essence of the problem is that theorems by Calabi [1] and Yau [2, 3]
guarantee the existence of a Ricci-flat Kähler metric (Calabi-Yau metric) when
certain criteria are satisfied, but Yau’s proof is non-constructive. It is not
for lack of examples satisfying the criteria, since topological constructions
ensure the existence of an exponentially large number of examples [4, 5, 6].
The problem also does not prevent certain types of progress in string theory,
since aspects of Calabi-Yau manifolds can be studied without knowing the
metric. For instance, much is known about volumes of calibrated submanifolds
[7], an artifact of supersymmetry and the existence of BPS objects, as well a
metric deformations the preserve Ricci-flatness, the (in)famous moduli spaces
[8]. Nevertheless, the central problem persists: general geometric properties
of manifolds, and associated physics arising from compactification, requires
knowing the metric.

For this reason, it is interesting to study approximations of Calabi-Yau
metrics. Efforts to do so generally require a sequence of approximations that
converge toward the desired metric, which can be thought of as a flow in the
space of metrics if the sequence is continuous. A classical example with a
discrete sequence is Donaldson’s algorithm, which uses a balanced metric to
converge to the Calabi-Yau metric [9]. More recently, there has been progress
in approximating Calabi-Yau metrics with neural networks [10, 11, 12, 13, 14,
15], which is the current state-of-the-art and is a continuous metric flow in
the limit of infinitesimal update step size. Thirty minutes on a modern laptop
gives an approximate Calabi-Yau metric on par with those that would take a
decade to compute with Donaldson’s algorithm.

The success of these neural network techniques prompts a number of
mathematical questions. What is the mathematical theory underlying flows in
the space of metrics induced by continuous neural network gradient descent?
Since empirical results suggest that the flows are converging to the Calabi-
Yau metric, how does the flow relate to other flows that have Calabi-Yau
metrics as fixed points, such as the Ricci flow?

The main result of this paper is to develop a theory of metric flows induced
by neural network gradient descent, utilizing a recent result from ML theory
known as the neural tangent kernel (NTK) [16, 17]. We will derive the relevant
flow equations and demonstrate that a metric-NTK governs the flow, which in
general is non-local and evolves in time. However, many architectures [18]
admit a certain large-parameter limit [16] in which the dynamics simplify and
the metric-NTK becomes constant. Additional choices may be made to induce
locality in the flow, and an appropriate choice of loss function reproduces
Perelman’s formulation of Ricci flow. In total, we show that neural network
metric flow provides a rich generalization of certain flows in the
differential geometry literature.

##  2 Metric Flows with Neural Networks

In this section we will develop a theory of flows in the space of metrics
under the assumption that it is represented by a neural network $g_{\theta}$,
where $\theta$ are the parameters of the neural network and a flow in
$g_{\theta}$ is induced by a flow in $\theta$.

For clarity, we summarize the results of this section. We will focus on the
case that the parameters $\theta$ are updated by gradient descent with respect
to a scalar loss functional $\mathcal{L}$, and derive the associated flow
equations in a “time” parameter $t$. Without making further assumptions, the
metric flow is non-local and is given by an integral differential equation
that involves a $t$-dependent kernel. However, many neural network
architectures have a hyperparameter $N$ such that the associated kernel
becomes deterministic and $t$-independent in the limit $N\to\infty$. This is a
dramatic simplification of the dynamics, and we call such a metric flow an
_infinite neural network metric flow_ ; it is related to more traditional
kernel methods, with the fixed kernel determined by the neural network
architecture. An infinite NN metric flow is still non-local and exhibits a
certain type of mixing, but under additional assumptions about the
architecture locality is achieved and mixing is eliminated, which we call a
_local neural network metric flow_. Such a flow is a local gradient flow,
which allows metric flows defined as gradient flows to be realized in a neural
network context. In particular, we show that Perelman’s formulation of Ricci
flow as a gradient flow [19] is a local neural network metric flow. See Figure
1 for a graphical summary of these various flows. We collect other types of
gradient flows tailored towards Kähler metrics in Appendix B. Being gradient
flows of scalar loss functionals, these flows are amendable to an analysis
that parallels our discussion of Perelman’s Ricci flow; indeed, Perelman’s
Ricci flow can be written in terms of energy functionals of Kähler-Ricci flow
[20], see Appendix B.

NN Metric Flows$\infty$-NN Metric FlowsLocal Metric FlowsPerelman’sRicci Flow
Figure 1: Graphical depiction of the different types of NN metric flows
developed in this paper. Perelman’s formulation of Ricci flow is realized as
an infinite neural network metric flow under architectural assumptions that
induce locality.

###  2.1 General Theory and Metric-NTK

Consider a Riemannian manifold $X$ with metric $g$, given by $g_{ij}$ in some
local coordinates. In this work we consider the case that $g_{ij}$ is
represented by a deep neural network, i.e. it is a function composed of
simpler functions, with a set of parameters $\theta_{I}$, where $I=1,\dots,N$;
see Appendix A. Generally $N$ is large for modern neural networks, and in this
work we will exploit a different description that emerges in the $N\to\infty$
limit. Throughout, we use lower case Latin indices for the coordinates of $X$,
and capital Latin indices for the network parameters. Written this way, the
metric evolves as

$\frac{dg_{ij}(x)}{dt}=\frac{dg_{ij}(x)}{d\theta_{I}}\frac{d\theta_{I}}{dt},$
(2.1)

with Einstein summation implied for repeated indices, unless stated otherwise.
Training the network means updating the parameters by some non-zero
$d\theta_{I}/dt$, and therefore this equation says that the metric flow may be
thought of as a flow induced by training the neural network.

There are many mechanisms for training a neural network in the deep learning
literature, but one of the most common is gradient descent, in which case the
parameter update is

$\frac{d\theta_{I}}{dt}=-\frac{\partial\mathcal{L}[g]}{\partial\theta_{I}},$
(2.2)

where $\mathcal{L}[g]$ is a scalar loss functional that depends on the metric.
The loss sometimes depends on a finite set of points $B$ on $X$ known as the
batch, in which case

$\mathcal{L}[g]=\sum_{x^{\prime}\in B}l[g](x^{\prime}),$ (2.3)

where $l[g](x^{\prime})$ is a pointwise loss that still depends on the metric.
Alternatively, the loss could be over all of $X$

$\mathcal{L}[g]=\int_{X}d\mu(x^{\prime})\,l[g](x^{\prime}).$ (2.4)

where $d\mu(x^{\prime})$ is a chosen measure on $X$. In typical machine
learning applications the batch $B$ is fixed, which in the continuous case
corresponds to utilizing a fixed measure; in particular, below we will use a
volume form with respect to a fixed reference metric $\tilde{g}$. Henceforth,
we drop the $[g]$ on $\mathcal{L}$ and $l(x^{\prime})$; it is to be understood
that they depend on $g$.

Either type of loss yields additional expressions for the metric flow induced
by gradient descent. For the discrete and continuous case, we obtain

$\frac{dg_{ij}(x)}{dt}=-\frac{\partial
g_{ij}(x)}{\partial\theta_{I}}\sum_{x^{\prime}\in B}\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta_{I}}\frac{\delta l(x^{\prime})}{\delta
g_{kl}(x^{\prime})}$ (2.5)

and

$\frac{dg_{ij}(x)}{dt}=-\frac{\partial
g_{ij}(x)}{\partial\theta_{I}}\int_{X}d\mu(x^{\prime})\,\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta_{I}}\frac{\delta l(x^{\prime})}{\delta
g_{kl}(x^{\prime})}\,,$ (2.6)

respectively. These expressions are written in a way to emphasize that one of
the $\partial g/\partial\theta$ factors is outside of the sum or integral.
Pulling them inside the sum or integral, we see the appearance of a
distinguished object,

$\Theta_{ijkl}(x,x^{\prime}):=\frac{\partial
g_{ij}(x)}{\partial\theta_{I}}\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta_{I}},$ (2.7)

which is a type of Neural Tangent Kernel (NTK) that we call the metric-NTK,
which is symmetric in the first and last pair of indices. In terms of the
metric-NTK, the neural network metric flows are

$\displaystyle\begin{split}\frac{dg_{ij}(x)}{dt}&=-\sum_{x^{\prime}\in
B}\Theta_{ijkl}(x,x^{\prime})\frac{\delta l(x^{\prime})}{\delta
g_{kl}(x^{\prime})}\\\
\frac{dg_{ij}(x)}{dt}&=-\int_{X}d\mu(x^{\prime})\,\Theta_{ijkl}(x,x^{\prime})\frac{\delta
l(x^{\prime})}{\delta g_{kl}(x^{\prime})}\end{split}$ (2.8)

for the discrete batch and continuous batch, respectively. We emphasize that
the metric-NTK $\Theta_{ijkl}$ is not a 4-tensor, since it depends on both $x$
and $x^{\prime}$ : the $(ij)$ indices transform with respect to
diffeomorphisms of the external point $x$, and the $(kl)$ indices transform
with respect to diffeomorphisms of the integrated point $x^{\prime}$, with
invariance of metric updates under $x^{\prime}$-diffeomorphisms requiring an
invariant measure.

In general, this is a complicated metric flow. Some of the reasons include:

  * •

Evolving Kernel. The metric flow is induced by gradient descent, which updates
the parameters and therefore the metric-NTK; $\Theta$ and $\delta l/\delta g$
in (2.8) are time-dependent.

  * •

Non-locality. In general, $\Theta$ induces non-local dynamics: it is a
smearing function that updates the metric at $x$ according to properties at
other points $y$, which in principle could be far away from $x$.

  * •

Component Mixing. We see that any $(k,l)$ components of the metric may update
fixed $(i,j)$ components of the metric via mixing of $\Theta_{ijkl}$ and
$\delta l/\delta g_{kl}$.

In the continuous case we have an integral differential equation that is
difficult to solve and analyze. We will identify cases in which the situation
simplifies.

###  2.2 Infinite Neural Network Metric Flows

In certain limits the metric-NTK enjoys properties that simplify the neural
network metric flow. We utilize the two central observations from the NTK
literature [16, 17] and refer the reader to Section 2.3.2 for details in an
example; we review only the essentials here.

First, for appropriately chosen architecture (including normalization), the
metric-NTK in the $N\to\infty$ limit may be interpreted as an expectation
value by the law of large numbers, in which case the associated integral over
parameters renders it parameter-independent. Schematically, we have

$\lim_{N\to\infty}\Theta_{ijkl}(x,x^{\prime})=\mathbb{E}_{\theta}[\alpha_{ijkl}(x,x^{\prime})]=:\bar{\Theta}_{ijkl}(x,x^{\prime}),$
(2.9)

where the bar over $\bar{\Theta}$ reminds us that the metric-NTK in this limit
is an average (over parameters) of a tensor $\alpha_{ijkl}(x,x^{\prime})$ that
may be computed in examples. While $\Theta$ is stochastic, due to its
dependence on the parameters associated to some initial neural network draw,
$\bar{\Theta}$ is deterministic, depending only on the network architecture
and parameter distribution.

The second simplification occurs for linearized models (linearized in the
parameters, not the input $x$), defined to be

$g^{L}_{ij}(x):=g_{ij}(x)\Big{|}_{\theta=\theta_{0}}+\left(\theta_{I}-\theta_{0,I}\right)\frac{\partial
g_{ij}(x)}{\partial\theta_{I}}\Big{|}_{\theta=\theta_{0}},$ (2.10)

i.e., it is just the Taylor expansion in parameters, truncated at linear
order. The metric-NTK associated to the linear model is

$\Theta^{L}_{ijkl}(x,x^{\prime})=\frac{\partial
g_{ij}(x)}{\partial\theta_{I}}\Big{|}_{\theta=\theta_{0}}\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta_{I}}\Big{|}_{\theta=\theta_{0}}=\Theta_{ijkl}(x,x^{\prime})\Big{|}_{\theta=\theta_{0}}.$
(2.11)

It is the metric-NTK $\Theta$ associated to $g$, evaluated at initialization.
That is, though $\Theta$ evolves in $t$, $\Theta^{L}$ does not. Taking the
$N\to\infty$ limit of the linearized model, its metric-NTK $\bar{\Theta}^{L}$
is both $t$-independent and $\theta$-independent. $\bar{\Theta}^{L}$ is the
so-called “frozen” NTK.

Though linearization may seem like a violent truncation, it has been shown
that deep neural networks evolve as linear models [17] in the $N\to\infty$
limit, i.e., $\bar{\Theta}^{L}$ governs its gradient descent dynamics to a
controllable approximation. This regime is known as “lazy learning.” For this
reason, we henceforth drop the superscript $L$ and write the frozen NTK as
$\bar{\Theta}$. In general, any quantity with a bar is frozen, i.e.,
deterministic and $t$-independent.

In summary, in the frozen-NTK limit the general NN metric flow (2.8) becomes

$\displaystyle\begin{split}\frac{dg_{ij}(x)}{dt}&=-\sum_{x^{\prime}\in
B}\bar{\Theta}_{ijkl}(x,x^{\prime})\frac{\delta l(x^{\prime})}{\delta
g_{kl}(x^{\prime})}\,,\\\
\frac{dg_{ij}(x)}{dt}&=-\int_{X}d\mu(x^{\prime})\,\bar{\Theta}_{ijkl}(x,x^{\prime})\frac{\delta
l(x^{\prime})}{\delta g_{kl}(x^{\prime})}\,,\end{split}$ (2.12)

in the discrete and continuous case, with the only difference being that the
dynamics are governed by the deterministic $t$-independent kernel
$\bar{\Theta}$ rather than the generally stochastic $t$-dependent kernel
$\Theta$. The dynamics still exhibits non-locality and component mixing, but
no longer has an evolving kernel. We refer to such a flow as an _infinite
neural network metric flow_.

Though the equations have only changed by the introduction of the bar, this is
actually a dramatic simplification of the dynamics! Naively, one might think
that although the kernel is better behaved, one cannot train an infinite
neural network because one cannot put it on a computer. This is in fact not
true: while initializing an infinite NN and running parameter space gradient
descent is indeed impossible, the large-$N$ limit gives a new description of
the system — a different duality frame — that allows to analyze the same
dynamics in a different way that does not require explicitly updating an
infinite number of a parameters. For instance, for mean-squared-error loss the
dynamics may be solved _exactly_ , with computable mean and covariance across
different initializations [17].

###  2.3 Local Metric Flows

Let us discuss how to simplify the flows even further by getting rid of non-
locality and component mixing. While such flows are presumably less general,
they are of interest because they are more analytically tractable, and we will
also see that they recover some famous metric flows.

To simplify matters, we will consider local metric flows that do not exhibit
component mixing, though the latter could be relaxed. Specifically, if the
discrete and continuous frozen metric-NTK satisfy

$\displaystyle\begin{split}\bar{\Theta}_{ijkl}(x,x^{\prime})&=\delta_{x,x^{\prime}}\,\,\delta_{ik}\delta_{jl}\,\,\bar{\Omega}(x)\,,\\\
\bar{\Theta}_{ijkl}(x,x^{\prime})&=\delta(x-x^{\prime})\,\,\delta_{ik}\delta_{jl}\,\,\bar{\Omega}(x)\,,\end{split}$
(2.13)

respectively, for some deterministic function $\bar{\Omega}(x)$, the dynamics
becomes

$\displaystyle\frac{dg_{ij}(x)}{dt}=-\bar{\Omega}(x)\frac{\delta l(x)}{\delta
g_{ij}(x)}\,.$ (2.14)

Here $\delta_{x,x^{\prime}}$ is the discrete Kronecker delta and
$\delta(x-x^{\prime})$ is the Dirac delta function. We call such a flow a
_local metric flow_.

However, there is a pathology that must be discussed. In deep learning, one
typically splits the input into disjoint sets containing train inputs $x$ and
test inputs $x^{\prime}$, respectively. NN updates are trained on train inputs
$x$ only and their generalization properties to unseen points is tested with
the test set inputs $x^{\prime}$. In the continuous case, there simply is no
disjoint test set since we integrate over the entire manifold $X$. In the
discrete case, enforcing delta-functions in the frozen metric NTK in (2.13)
means that only train points are updated and there would be no learning on any
point outside the train set. Hence, a non-local metric-NTK is essential to
ensuring that the metric gets updated at all manifold points. Thus, the local
case (2.14) only makes sense for continuous flows.

Finally, there is a simple trick for defining a new architecture that gets rid
of $\bar{\Omega}(x)$ or reshapes it, if desired. We will phrase the trick in
terms of a general neural network, but apply it in the context of metrics.

Let $\phi(x)$ be any network function with associated NTK
$\Theta_{\phi}(x,x^{\prime})$. Now multiply $\phi(x)$ by a _deterministic_
function $h(x)$ to obtain a new network

$\widetilde{\phi}(x)=h(x)\phi(x).$ (2.15)

Since $h(x)$ is deterministic, $\phi$ and $\widetilde{\phi}$ have the same
parameters and

$\Theta_{\widetilde{\phi}}(x,x^{\prime})=h(x)h(x^{\prime})\Theta_{\phi}(x,x^{\prime}).$
(2.16)

This also gives the same result for frozen NTKs,
$\bar{\Theta}_{\widetilde{\phi}}(x,x^{\prime})=h(x)h(x^{\prime})\bar{\Theta}_{\widetilde{\phi}}(x,x^{\prime})$.
In the case of a local flow, where we have

$\Theta_{\phi}(x,x^{\prime})=\delta(x-x^{\prime})\,\Omega(x),$ (2.17)

the $\delta$-function imposes that the two $h$’s are evaluated at the same
point, yielding

$\Theta_{\phi}(x,x^{\prime})=\delta(x-x^{\prime})\,h^{2}(x)\,\Omega(x).$
(2.18)

This trick is useful because we may use the freedom of $h(x)$ to define a new
architecture $\widetilde{\phi}$ whose NTK we can shape, and in particular
cancel out potentially unwanted local factors such as $\Omega(x)$ in the NTK
or $\sqrt{g(x^{\prime})}$ in the volume form.

####  2.3.1 Architecture Design for Local Metric Flow

We have seen that a metric-NTK satisfying (2.13) induces a local metric flow
(2.14) that evolves with a deterministic kernel, a local evolution equation,
and without mixing induced by $\Theta_{ijkl}$ and $\delta l/\delta g_{kl}$. In
this section, we study whether NN architectures that satisfy (2.13) actually
exist. Specifically, we investigate how the different delta functions may
arise in the metric-NTK.

There is a simple sufficient condition for obtaining the Kronecker deltas
$\delta_{ik}\delta_{jl}$.111These Kronecker deltas amount to a choice of gauge
that is not preserved under diffeomorphisms in $x$ and $x^{\prime}$ on the
metric-NTK. Choosing each independent component of the metric $g_{ij}(x)$ to
be a separate neural network with parameters $\theta^{ij}$, the set of
parameters $\theta$ for the entire metric is partitioned as

$\theta=\bigcup_{i,j}\theta^{ij},$ (2.19)

and the metric-NTK is

$\bar{\Theta}_{ijkl}(x,x^{\prime})=\sum_{m,n}\frac{\partial
g_{ij}(x)}{\partial\theta^{mn}_{I}}\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta^{mn}_{I}}=\delta_{ik}\delta_{jl}\,\frac{\partial
g_{ij}(x)}{\partial\theta^{ij}_{I}}\frac{\partial
g_{kl}(x^{\prime})}{\partial\theta^{ij}_{I}},$ (2.20)

with no Einstein summation over lower-case Latin indices on the RHS. The last
pair of derivatives is the NTK $\bar{\Theta}^{(ij)}(x,x^{\prime})$ of the
$(i,j)$ component of the metric, giving

$\bar{\Theta}_{ijkl}(x,x^{\prime})=\delta_{ik}\delta_{jl}\,\bar{\Theta}^{ij}(x,x^{\prime}),$
(2.21)

Thus, the metric-NTK for a metric with components given by independent neural
networks evolves according to the NTK of the individual components, as
expected. By symmetry, we should choose the independent neural networks
associated to each component to have identical architecture. Since the
architectures for the components are the same, they have the same frozen NTK,

$\bar{\Theta}(x,x^{\prime}):=\bar{\Theta}^{ij}(x,x^{\prime}),$ (2.22)

and we have that the frozen metric-NTK is

$\bar{\Theta}_{ijkl}(x,x^{\prime})=\delta_{ik}\delta_{jl}\,\bar{\Theta}(x,x^{\prime}).$
(2.23)

This gets us part of the way to (2.13); we have the Kronecker deltas that
prevent component-mixing.

We still need the Dirac delta function that induces locality, however. This is
a non-trivial step. Given (2.23), we must find an architecture such that

$\bar{\Theta}(x,x^{\prime})=\delta(x-x^{\prime})\,\bar{\Omega}(x)$ (2.24)

for some $\bar{\Omega}(x)$. The $\delta$-function is defined with respect to
the measure $d\mu(x^{\prime})$, which for simplicity we take to be the volume
form with respect to a fixed reference metric $\tilde{g}$ on $X$, although the
same argument applies for a more general probability density
$d\mu(x^{\prime})=d^{d}x^{\prime}\,P(x^{\prime})$. Since the $\delta$-function
has support on a local patch $\mathbb{R}^{d}$, we have

$\int_{X}dV_{\tilde{g}}\,\delta(x-x^{\prime})f(x^{\prime})=\int_{\mathbb{R}^{d}}d^{d}x^{\prime}\sqrt{|\tilde{g}(x^{\prime})|}\,\delta(x-x^{\prime})\,f(x^{\prime})=f(x).$
(2.25)

This $\delta$-function on $X$ is related to the $\delta$-function on
$\mathbb{R}^{d}$ by

$\delta_{\mathbb{R}^{d}}(x-x^{\prime})=\sqrt{|\tilde{g}(x^{\prime})|}\,\delta(x-x^{\prime}),$
(2.26)

which gives the usual identity $\int
d^{d}x^{\prime}\delta_{\mathbb{R}^{d}}(x-x^{\prime})f(x^{\prime})=f(x)$ when
inserted into (2.25).

We take a two-step process to obtain (2.24): we must figure out how to obtain
$\delta_{\mathbb{R}^{d}}$ inside an NTK, and then how to account for the
factor $\sqrt{|g|}$ in the volume measure. Let $\phi_{\sigma}(x)$ with
$\sigma\in\mathbbm{R}^{+}$ be a network with frozen NTK
$\bar{\Theta}_{\sigma}(x,x^{\prime})$ given by

$\bar{\Theta}_{\sigma}(x,x^{\prime})=\frac{1}{(2\pi\sigma^{2})^{d/2}}\exp^{-\frac{1}{2}\frac{|x-x^{\prime}|^{2}}{\sigma^{2}}}\,\,\bar{\alpha}(x,x^{\prime})$
(2.27)

for some deterministic function $\bar{\alpha}(x,x^{\prime})$. Clearly, we get
the $\delta_{\mathbb{R}^{d}}$ inside the NTK for $\sigma\to 0$. Using the
rescaling trick introduced in the previous section, we may define a new
network function that is $\phi_{\sigma}(x)$ multiplied by
$\tilde{g}(x)^{-1/4}$, in which case the new NTK is

$\bar{\Theta}_{\sigma}(x,x^{\prime})=\frac{1}{(2\pi\sigma^{2})^{d/2}}(\tilde{g}(x)\tilde{g}(x^{\prime}))^{-1/4}\exp^{-\frac{1}{2}\frac{|x-x^{\prime}|^{2}}{\sigma^{2}}}\,\,\bar{\alpha}(x,x^{\prime}).$
(2.28)

This new NTK satisfies

$\lim_{\sigma\to
0}\,\bar{\Theta}_{\sigma}(x,x^{\prime})=\delta(x-x^{\prime})\,\,\bar{\Omega}(x),$
(2.29)

i.e., it satisfies (2.24) with $\bar{\Omega}(x)=\bar{\alpha}(x,x)$. The
parameter $\sigma\in\mathbb{R}^{+}$ clearly sets a scale of non-locality in
the network evolution, which is an interesting object to study. The family has
an NTK given by (2.28), which induces a metric flow

$\frac{dg_{ij}(x)}{dt}=-\int\,d^{d}x\left(\frac{\tilde{g}(x^{\prime})}{\tilde{g}(x)}\right)^{1/4}\,\,\frac{1}{(2\pi\sigma^{2})^{d/2}}\exp^{-\frac{1}{2}\frac{|x-x^{\prime}|^{2}}{\sigma^{2}}}\bar{\alpha}(x,x^{\prime})\,\,\frac{\delta
l(x^{\prime})}{\delta g_{ij}(x^{\prime})}.$ (2.30)

Here we see that the variance of the Gaussian $\sigma$ is a parameter
controlling the amount of non-locality in the dynamics, since it affects how
strongly updates at $x^{\prime}$ affects the evolution of the metric at $x$.
In the limit $\sigma\to 0$, the normalized Gaussian becomes the
$\delta$-function and the dynamics is

$\frac{dg_{ij}(x)}{dt}=-\bar{\Omega}(x)\,\,\frac{\delta l(x^{\prime})}{\delta
g_{ij}(x^{\prime})},$ (2.31)

where $\bar{\Omega}(x)=\bar{\alpha}(x,x)$, and we have recovered a local
metric flow.

Summarizing, we obtain a network with the desired properties as follows:

  * •

No component mixing between $\Theta_{ijkl}$ and $\delta l/\delta g_{kl}$ is
obtained by taking each component to be a neural network of the same
architecture, but with independent parameters.

  * •

Locality arises from a $\delta$-function as in (2.24), which we achieve by
first obtaining a $\delta_{\mathbb{R}^{d}}$ in the NTK as the limit of a
normalized Gaussian, and then passing to the $\delta$-function on the
Riemannian manifold by using the rescaling trick to obtain the geometric
factor in the volume form.

####  2.3.2 Concrete Architectures for Local and Non-local Metric Flows

We have just shown that any family of architectures with parameter $\sigma$
satisfying (2.27) may be rescaled to give a local metric flow as $\sigma\to
0$. Since we are considering cases where each metric component is independent,
it suffices to consider scalar network functions.

Consider a network architecture based on cosine activation, with

$\displaystyle\begin{split}\phi(x)=\frac{A}{\sqrt{N}}\sum_{i=1}^{N}\sum_{j=1}^{d}a_{i}\cos(w_{ij}x_{j}+b_{i}),\\\
a\sim\mathcal{N}(0,\sigma_{a}^{2})\,,\qquad
w\sim\mathcal{N}(0,\sigma_{w}^{2}/d)\,,\qquad
b\sim\mathcal{U}(-\pi,\pi)\,,\end{split}$ (2.32)

where $N$ is the width of the network and $A$ is a normalization factor that
will be fixed later. To simplify the picture, we freeze all weights to their
initialization values except the $a_{i}$, in which case the NTK is

$\Theta(x,x^{\prime})=\sum_{k=1}^{N}\frac{\partial\phi(x)}{\partial
a_{k}}\frac{\partial\phi(x^{\prime})}{\partial
a_{k}}=\frac{A^{2}}{N}\sum_{k=1}^{N}\sum_{j=1}^{d}\cos(w_{kj}x_{j}+b_{k})\cos(w_{kj}x^{\prime}_{j}+b_{k})\,.$
(2.33)

As we take the width $N\to\infty$, the NTK $\Theta$ becomes an expectation
value, by the law of large numbers,

$\bar{\Theta}(x,x^{\prime})=A^{2}\,\mathbb{E}[\cos(w_{kj}x_{j}+b_{k})\cos(w_{kj}x^{\prime}_{j}+b_{k})].$
(2.34)

Evaluating the expectation gives

$\bar{\Theta}(x,x^{\prime})=\frac{A^{2}}{2}\exp\left(-\frac{1}{2}\frac{\sigma_{w}^{2}}{d}|x-x^{\prime}|^{2}\right).$
(2.35)

We see that our NTK is a Gaussian, as desired. Normalizing it, so that in an
appropriate limit it is a $\delta$-function, we have

$A=2^{\frac{1}{4}}\left(\frac{\sigma_{w}}{\sqrt{2\pi d}}\right)^{d/4}.$ (2.36)

With this normalization factor fixed, the NTK is a normalized Gaussian of
width

$\sigma=\frac{\sqrt{d}}{\sigma_{w}}.$ (2.37)

We obtain a local metric flow by taking the limit $\sigma_{w}\to\infty$, which
sends $\sigma\to 0$.

The trick of freezing all weights but those of the last (linear) layer is
common in the literature. In such a case the NTK is determined (up to a
constant) by the so-called NNGP kernel $\mathbb{E}[\phi(x)\phi(x^{\prime})]$,
also known as the two-point correlation function. The architecture that we
have presented is a simple modification (via so-called NTK parameterization,
which gives the $1/\sqrt{N}$ factor) of an architecture in [21] that is known
to have a Gaussian NNGP kernel. Similarly, the architecture called Gauss-net
in [22] has a Gaussian NNGP kernel, and can be appropriately modified to give
a different architecture (based on exponentials in the network function,
rather than cosines) that yields a local metric flow.

###  2.4 Ricci Flow and More General Flows as Neural Network Metric Flows

Having developed a theory of neural network metric flows, we ask: are any
canonical metric flows from differential geometry realized as neural network
metric flows?

Many well-studied metric flows are local: they are not an integral PDE and
take the form

$\frac{dg_{ij}(x)}{dt}=-u_{ij}(x),$ (2.38)

with metric updates depending on the local properties of some rank two
symmetric tensor $u_{ij}(x)$. A particularly well-studied example is Ricci
flow

$\frac{dg_{ij}(x)}{dt}=-2R_{ij}(x).$ (2.39)

In general, flows of the form (2.38) are not necessarily gradient flows. To
make a direct connection, one could study neural network metric flows that are
not gradient flows either, but following a standard assumption in deep
learning we studied those neural network metric flows that are induced by the
gradient of a scalar loss functional.

Famously, Perelman showed [19] that Ricci flow is a gradient flow after
applying a $t$-dependent diffeomorphism (see [23] for a review). Specifically,

$\frac{dg_{ij}}{dt}=\frac{\delta\mathcal{F}[\phi,g]}{\delta
g_{ij}(x)}=-2[R_{ij}+\nabla_{i}\nabla_{j}\phi]\,,$ (2.40)

where

$\mathcal{F}[\phi,g]=\int_{X}\left(R+|\nabla\phi|^{2}\right)\,e^{-\phi}\,dV$
(2.41)

is the Perelman functional and (2.40) is equivalent to (2.39) by a
$t$-dependent diffeomorphism. Using our formulation, local metric flow induced
by neural network gradient descent, e.g. via the concrete architecture of
Section (2.3.2), we obtain Perelman’s formulation of Ricci flow by simply
choosing the loss function appropriately,

$\displaystyle l(x)=-\frac{\mathcal{F}[\phi,g]}{\bar{\Omega}(x)}\,.$ (2.42)

Similarly, we obtain a parametrically non-local generalization of Ricci flow
by backing off of the $\sigma\to 0$ limit. Doing so replaces the
$\delta$-function in the NTK by a Gaussian, and allows nearby points $y$ to
influence metric updates at $x$, in a way controllable by tuning $\sigma$.

Similarly, any other local metric flow that is a gradient flow may be
achieved, together its non-local generalization for finite $\sigma$, by the
choice of loss function and a tuning of $\sigma$.

##  3 Numerical Implementation

In this section we describe the use of these (kernel) methods to approximate
CY metrics numerically. We will use the standard test case of the Fermat
Quintic to benchmark the methods. The Fermat Quintic is a Calabi-Yau threefold
that is given as the anti-canonical hypersurface inside a complex projective
space $\mathbbm{P}^{4}$ with homogeneous coordinates $z_{i}$ by the equation

$\displaystyle z_{0}^{5}+z_{1}^{5}+z_{2}^{5}+z_{3}^{5}+z_{4}^{5}=0\,.$ (3.1)

A simple NN with three layers, 64 nodes, and ReLU activation function can pick
up a factor of 10-100 in the overall loss when trained for around 50 epochs on
100k points on the Fermat Quintic [13, 14].

The main takeaway is that kernel methods work excellently on the train set,
but generalization to test points is challenging. One reason is that there are
multiple numerical challenges associated with the implementation. We will
describe how we overcome these by implementing feature clustering. We then
evaluate performance using ReLU, Gaussian, and semi-local kernels. In all
cases, test set performance lacks behind that of simple (finite width) NNs. We
attribute this lack of performance of frozen metric NTK methods to importance
of feature learning, which is impossible for frozen NTKs. Indeed, feature
learning necessitates that the kernel corresponding to the NN adapts to
features, i.e., evolves during training.

###  3.1 Clustering and Preprocessing

We study the discrete NTK $\Theta_{ijkl}(x_{a},x^{\prime}_{b})$, where
$i,j,k,l\in\\{1,2,3\\}$ labeling the 3 complex CY coordinates and
$a\in\\{1,\ldots,N_{\text{pts}}\\}$,
$b\in\\{1,\ldots,N_{\text{pts}}^{\prime}\\}$ label the $N_{\text{pts}}$ and
$N_{\text{pts}}^{\prime}$ points in the train and test set, respectively.
During training, when we use all $N_{\text{pts}}$ points, this is a $3\times
3\times 3\times 3\times N_{\text{pts}}\times N_{\text{pts}}$ tensor, which has
almost a trillion entries for $N_{\text{pts}}=100,000$. This is a problem
generic to NTK implementations [24, 25, 26] and means that hardware far beyond
academic grade is required for the full computation.

Instead of computing the full trillion-parameter tensor, we cluster the input
features. For the problem at hand, it might be reasonable to assume that the
metric at some test point $x^{\prime}$ is most strongly correlated with the
metric at nearby points $x$. Typically, there is no canonical metric on
feature space, so the notion of distance is somewhat arbitrary. In our case,
however, the features are actual points on a CY manifold, so a good notion of
distance would be the shortest geodesic distance between these points with
respect to the canonical (unique, Ricci-flat) metric. In practice, this is
unfeasible for multiple reasons: First, we do not yet have the Ricci-flat CY
metric – we are computing it with the NTK. Second, even if we had the metric,
computing the geodesic distance between two points requires solving an initial
value PDE numerically (which can be done using e.g. shooting methods), but is
very computationally costly.

Hence, we resort to a much simpler proxy which is less accurate but cheap to
compute: the shortest geodesic distance between two points in the ambient
space Fubini-Study metric. For two points $z$ and $z^{\prime}$ in
$\mathbbm{P}^{n}$ given in terms of homogeneous coordinates, the geodesic
distance with respect to the Fubini-Study metric is

$\displaystyle d(z,z^{\prime})=\arccos\sqrt{\frac{(z^{\dagger}\cdot
z^{\prime})(z^{\prime\dagger}\cdot z)}{|z|^{2}|z^{\prime}|^{2}}}\,,$ (3.2)

where the dot product is with respect to the flat metric.

In order to perform clustering based on distances for $N_{\text{pts}}=100,000$
points, we would need to fit a $100,000\times 100,000$ distance matrix into
memory, which is also unfeasible. Instead, we implement the following
procedure:

  1. 1.

Divide the points into mini batches. The size of these should be adapted to
the available hardware. We used a size of $M=10,000$.

  2. 2.

Cluster each mini batch independently based on the FS geodesic distance. We
use agglomerative clustering and average to compute the linkage of clusters,
as implemented by sklearn. Again the total cluster size depends on the
available hardware. We aimed for $\mathcal{O}(5k)$ points in each cluster,
which allowed us to compute the $3\times 3\times 3\times 3\times 5000\times
5000$ NTK tensor, so we aimed for $C=N_{\text{pts}}/5000$ clusters.

At this point, we have $B=N_{\text{pts}}/M$ batches (which we label by
$I=1,2,\ldots B$), each of which has $C$ clusters $c^{I}_{\alpha}$,
$\alpha=1,2,\ldots C$. Next, we need to merge the clusters across the
different mini batches. To do so, we proceed as follows:

  1. 3.

Use the clusters of the first mini batch as a starting point

  2. 4.

For each cluster in mini batch 2, compute the geodesic distance between all
points in this mini batch and the first cluster. The cluster from mini batch
2, whose maximum distance over all points in cluster 1 of mini batch 1 is
smallest, gets merged into cluster 1 of mini batch 1.

  3. 5.

Continue for all clusters in all mini batches.

This means that the final clusters $c_{\alpha}$ are given by

$\displaystyle c_{\alpha}=\bigcup_{\text{mini batches
$I$}}\min\left[\max_{\text{clusters
$\beta$}}(d_{\text{FS}}(c_{\alpha}^{1},c_{\beta}^{I}))\right]\,.$ (3.3)

Based on the clusters, we can compute the NTK updates for each cluster
separately, assuming that the contribution from other clusters can be
neglected in the updates. While this algorithm is not perfect and depends on
the random order of mini batches and clusters, it produces decent results as
can be checked by comparing the result of this algorithm to the result when we
just cluster without batching, which is feasible if the number of points is
small enough. To illustrate the efficiency of the algorithm we present a
heatmap of the distance matrix for 1000 points in Figure 2. On the left, we
see the distance matrix without any clustering. In the middle, we give the
distance matrix when ordering the points when clustered according to (3.3)
with 5 batches. On the right, we give the distance matrix when running the
clustering algorithm on all 1000 points (i.e., without batching).

Figure 2: Comparison of distance between points with no clustering, batch
clustering and full clustering for 1000 points and 10 clusters, using 5
batches for batch clustering.

There are further numerical subtleties that need to be taken into account.
First, the input features are the real and imaginary part of the 5 homogeneous
coordinates $z_{i}\in\mathbbm{P}^{4}$, $i=0,\ldots,4$. In contrast, the CY
metric is naturally expressed as a Hermitian $3\times 3$ matrix,
$g_{a\bar{b}}\;dx^{a}(z)\otimes dx^{b}(z)$, $a,b=1,2,3$. Here, the three
complex coordinates $x_{a}$ are functions of the five homogeneous ambient
space coordinates $z_{i}$. Typically, one chooses a coordinate system on the
CY by going to affine coordinates in a patch of the ambient space (this
removes one of the five $z_{i}$ coordinates through scaling) and eliminating
one of the affine coordinates via the hypersurface equation that defines the
CY in that patch. While physics does of course not depend on the choice of
coordinate system, the metric $g_{a\bar{b}}$ will get modified by the Jacobian
of the transformation from one coordinate system to another.

Since computing the relevant data like the holomorphic $3$-form $\Omega$, the
pullbacks, and the integration weights is prone to numerical errors, we use
this freedom to choose patches and scalings that minimize the numerical error
sources. In practice, this amounts to transforming to the patch where the
coordinate $z_{i}$ with the largest absolute value is scaled to one, and then
using the CY equation to eliminate the coordinate which results in the
smallest value for $|\Omega|^{2}$. However, this means that there are 20
possible coordinate choices and the metric at different points will be in
different coordinate systems. Hence, updating the matrix $g_{a\bar{b}}(y)$
using the matrix entries of $g_{i\bar{j}}(x)$ does not make any sense. One way
around this issue is to transform the metric at each points into the same
coordinate system. instead of doing this, we simply sample about 20 times more
points than needed and then use rejection sampling to obtain a set of points
whose metrics are in the same coordinate system under the above procedure.

Another problem is a numerical instability related to the geodesic distance.
Note that

$\displaystyle\arccos(1-\epsilon)=\sqrt{\epsilon}.$ (3.4)

This means that points $x$ and $y$ which are numerically identical (meaning
$\epsilon\sim 10^{-7}$) would still have a sizable distance (.0003). Since our
kernel exponentially suppresses updates at $x$ by $d(x,y)$, this leads to
unwanted behavior during training. To ameliorate this, we set the geodesic
distance to zero once it falls below a certain threshold, since we would only
be amplifying numerical noise otherwise.

###  3.2 Metrics from Kernels

In this subsection we summarize a number of techniques that we used to
approximate metrics on test points using kernels. To determine the kernel
hyperparameters, we use a python implementation [27] of a Bayesian
optimization scheme [28]. We choose this hyperparameter optimizer since the
clustering steps are quite expensive and we hence want to minimize the number
of hyperparameter tuning runs. When we used the same data set for all
optimization steps, we observe that the optimizer tunes the hyperparameters to
the data set and shows poor performance on new data.

#### Metrics from a ReLU NTK

A simple single-layer network with ReLU activation may pickle up a factor of
$10$ to $100$ in the total loss. We use this as a baseline of comparison for
kernel methods, and specifically study the NTK associated to this ReLU network
as computed by the python package neural-tangents [24], which has the ability
to compute the infinite-width NTK of many architectures.

#### Metrics from a Gaussian kernel

We also use a simple Gaussian kernel, i.e., we set

$\displaystyle\Theta_{ijkl}(x,x^{\prime})=\delta_{ik}\delta_{jl}e^{-\frac{d(x,x^{\prime})}{2\sigma}}\,,$
(3.5)

where the variance $\sigma$ is a hyperparameter and $d(x,x^{\prime})$ is the
shortest geodesic distance w.r.t. the ambient space FS metric. We recover the
local update limit for $\sigma\to 0$. In the finite $\sigma$ regime, the
kernel updates at a point $x^{\prime}$ are just the sums of all contributions
from all training points $x$, weighted by a Gaussian proportional to their
distance to $x^{\prime}$. This suppression means that if the distance $d$ is
larger than $\sigma$, the updates are tiny. For this reason, we include the
possibility of normalizing the updates such that they are sizable at least at
some points. Overall, we found that the optimizer prefers a narrow Gaussians.
However, the performance is very sensitive to the hyperparameters and can
easily lead to exploding gradients or metric updates that lead to non-positive
definite matrices. When restricting the updates akin to gradient clipping, the
performance suffers substantially.

#### Metrics from a Delta kernel

Given that the Bayesian optimizer prefers narrow Gaussians, we finally study a
kernel which simply updates the metric at a test point $x^{\prime}$ with the
metric correction computed for the closest point $x$ in the training set. We
call this the Delta kernel.

#### Results and Discussion

With the techniques that we have described, we performed many experiments with
different values of the various hyperparameters, using the $\sigma$-loss
defined in (B.11) for gradient descent. Though we were able to obtain
significant gains in the train loss, our experiments failed to achieve a gain
of more than a factor of $\sim 5$ in the test loss. We would like to
streamline the presentation of our results to help understand this failure to
generalize to test points.

To do so, we need to compute the drop in the loss as a function of the
geodesic distance between the test and train points. In practice, it requires
sophisticated sampling techniques to produce a sample of test points at some
fixed geodesic distance from the training set. For our purposes, instead of
implementing such sampling, we approximate the same effect by adding Gaussian
noise with varying variance to each training point. Hence the test points will
only be “approximately” on the CY. The analysis effectively allows us to
interpolate between testing on train points and testing on test points as a
function of the variance of the Gaussian noise. The factor gained in the
$\sigma$-loss versus average shortest geodesic distance between noised and
sampled points is presented in the left plot of Figure 3; in the right plot,
we show how the maximum, mean, and minimum geodesic distance changes with the
total number of sampled points. The experiments were run with hyperparameters
given in Table 1.

| ReLU | Gaussian | Delta
---|---|---|---
# Points | 2500 | 2500 | 1000
Learning Rate | 0.1 | 0.1583 | .3162
$\sigma$ | - | 30.0 | -
$\sigma_{b}$ | 80.52 | - | -
$\sigma_{w}$ | 5.716 | - | -
Table 1: Hyperparameters for CY metric learning with various kernels. An empty
entry indicates that the hyperparameter is not applicable to a given kernel.

Figure 3: _Left:_ Factor gained in the $\sigma$-loss as a function of average
shortest geodesic distance between noised and sampled points. _Right:_
Average, maximum, and minimum geodesic distance as a function of the number of
sampled points.

From Figure 3 we see a number of results. First, as the geodesic distance
approaches zero, i.e., when using the train set as the test set, we see that
the Gaussian and Delta kernels achieve a factor of $\sim 10^{6}$ in the
$\sigma$-loss, even though there are only $\sim 1000$ train points; the ReLU
kernel achieves a factor of $\sim 5$. However, as noise is added the test
point become further away from the train points, corresponding to a growing
geodesic distance. As the geodesic distance increases, the gain factor falls
off, until the gain factor is only $\sim 1$ for distance $1$. This indicates
that when the test points are too far from the train points, the kernel
methods fail to generalize. From the right plot in Figure 3, we see that the
mean geodesic distance between points (and therefore train and test points,
since they are sampled fairly from the Shiffman-Zelditch measure) is about
$.3$ and $.1$ for $10^{3}$ and $10^{5}$ points, respectively. Comparing these
distances to the plot on the left in Figure 3, we see that with geodesic
distances in this range we expect the gain factor to be about $5$, regardless
of whether you train at $10^{3}$ points or $10^{5}$ points. We confirmed this
with direct experimentation: at $10^{3}$ and $10^{5}$ points, the best our
experiments did on test loss was about a factor of $5$.

The advantage of the noise variance analysis is that it helps us understand
what gains we might hope for in the test loss as we scale up the number of
points. From the right plot in Figure 3 we see a linear dependence in the mean
geodesic distance on a log-log scale. From the best linear fit to the data we
obtain

$\displaystyle\mu=.968\times N_{\text{pts}}^{-0.184}.$ (3.6)

Note that on theoretical grounds, one could expect $\mu$ to be proportional to
$(N_{\text{pts}})^{-1/d}$ for a point sample on a (real) $d$-dimensional
manifold. In our case, this would lead to an exponent of $-0.167$, which is
close to the fitted value. The prefactor is more complicated and determining
it analytically would require solving a sphere-packing type problem on the CY
with respect to the geodesic distance using the FS metric and the Shiffman-
Zelditch distribution. We just note that the mean line segment length between
points on a 6d unit hypercube is $0.969$, which is numerically very close to
the prefactor from our fit.

Extrapolating, to obtain the gain factor $\sim 100$ achieved with finite
neural networks requires a mean geodesic distance of $5\times 10^{-3}$ for the
Gaussian kernel, which in turn requires $10^{12}$ points. For the Delta
kernel, a gain factor of $\sim 100$ requires a mean geodesic distance of
$5\times 10^{-2}$, which in turn requires $10^{8}$ points. Since the finite
neural network achieves this accuracy by learning from $10^{5}$ points, it is
significantly more data efficient than the kernel methods. To obtain the gain
factor of $\sim 10^{6}$ we observe for the training set also for the test
points, requires $\sim 10^{35}$ points.

_Discussion of the Importance of Feature Learning._ The fact that the kernel
methods fail to generalize well for larger distances is perhaps not too
surprising. The NTK is fixed purely by the architecture and the parameter
initialization. It is set in stone before training even starts and does not
change ever. This means that, unlike a finite NN that can dynamically adapt to
its inputs and learn useful embeddings for regression, the fixed kernel
dictates how results at training points $x$ are combined to form a prediction
at test points $x^{\prime}$.

The fixed kernel methods seem at odds with the fact that the Calabi-Yau metric
is unique in a fixed Kähler class. If a kernel method is to be used to predict
the metric at $x^{\prime}$ given nearby metric information, it probably has to
be a quite special kernel. Its analytic expression is likely a very
complicated function and not a simple Gaussian or the NTK of a ReLU. In the
only case where we know the CY metric, which is the trivial case of a two-
torus, we can compute this function. For the analog of the Fermat Cubic, which
is the 1d analog of the Fermat Quintic in (3.1), this function is given in
terms of inverse Weierstrass $\wp$-functions [29], which is certainly not
reproduced by adding metrics at nearby points weighted by a Gaussian or any
natural guess for a kernel function. This makes the fact that the finite NN
performs so well even more impressive.

##  4 Conclusions

We develop a theory for metric flows induced by neural network gradient
descent. In general, the evolution of a NN that describes a metric is a
complicated function of its hyperparameters, its parameters at initialization,
and it evolves over the course of the training process. In the infinite width
limit, this complicated training process becomes tractable and can even be
described analytically in some cases.

We illustrated how NNs can be engineered to reproduce flows with specific
properties, such as Perelman’s Ricci flow. The concept generalizes to other
kinds of flows that are gradient flows of some energy functional, as reviewed
in Appendix (B). This allows to engineer metric flows that are well-studied in
the mathematics literature in terms of neural networks. Conversely, one can
ask which type of flow a NN with a specific choice of architecture induces.
Beyond that, one can study flows from kernel methods that are not the Neural
Tangent Kernel of any known NN, leading to more general kernel methods.

Numerically, we encounter multiple challenges. The most important one is that
kernel methods require huge matrices that exceed the available computational
resources of most users. To ameliorate this, we develop a batched feature
clustering algorithm based on the geodesic distance in feature space. In
general, there is no canonical metric for feature space, or even a natural
choice of coordinates for the input data manifold. However, for the case at
hand, the input data manifold is just the manifold for which we want to study
the metric flow, such that there is a canonical choice of coordinates and
reference metric in this coordinate system, with respect to which geodesic
distances can be computed.

For the test case of the Fermat Quintic, we observe that the metric flow on
the training set leads to almost Ricci-flat metrics. However, these results
fail to generalize well for the kernels we tested. These included both Neural
Tangent Kernels as well as other kernels that are not necessarily derived from
a Neural Network. We attribute the fact that the results are worse than those
obtained from a simple finite NN to the importance of feature learning. Kernel
methods where the kernel is fixed during training cannot learn features.

The failures of infinite-NN kernel methods as compared to the performance of
their finite counterparts is likely a consequence of the Calabi-Yau theorem:
since the Calabi-Yau metric is unique, kernels that make correct predictions
for the metric at $x^{\prime}$ given nearby metric information must be very
special. A priori such kernels would have nothing to do with the kernels
associated to randomly initialized infinitely wide neural networks. Instead,
for neural network kernel methods to work, the kernel should be learned so
that it falls into this distinguished subclass. This is precisely what the
finite neural networks achieve. The extent to which feature learning is
important could be quantified by studying how the empirical NTK evolves as
compared to the frozen NTK. Such analyses are, however, beyond the scope of
this work.

#### Acknowledgements

We thank Robert Bryant, Michael Douglas, Mike Freedman, Sergei Gukov, Mark
Haskins, Matt Headrick, Jonathan Heckman, Cumrun Vafa, and Toby Wiseman for
useful conversations. We are grateful to The Simons Center for Geometry and
Physics for an enlightening program on computational differential geometry.
J.H. is supported by NSF CAREER grant PHY-1848089. F.R. is supported by NSF
grant PHY-2210333 and startup funding from Northeastern University. Both are
supported by the National Science Foundation under Cooperative Agreement
PHY-2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental
Interactions).

##  Appendix A Primer on Neural Networks

Deep neural networks (NNs) are the backbone of recent progress in machine
learning and artificial intelligence, including image classification [30, 31]
and generation [32], gameplay such as in Chess [33] and Go [34], and natural
language processing [35].

Neural networks, at their core, are functions. In most cases they are
comprised out of sets of smaller functions, which are then composed with one
another to form the so-called topology of the network, also known as the
_architecture_. This process typically involves the introduction of a set of
parameters $\theta$ that are drawn from a distribution $P(\theta)$, written
$\theta\sim P(\theta)$, at initialization, i.e., when the program begins. For
fixed values of $\theta$ the network function

$f_{\theta}:D\to R$ (A.1)

is a fixed function from some domain $D$ to some codomain $R$, typically
$\mathbb{R}^{n}$ and $\mathbb{R}^{m}$; we will often suppress the subscript
$\theta$, keeping in mind that neural networks always have parameters.

Part of the progress in ML in the last decade is the development of
architectures that are well-chosen for certain tasks. A particularly famous
architecture is the _single-layer fully connected network_ , also known as the
_perceptron_ , which takes the form

$f_{i}(x)=w^{(1)}_{ij}\sigma\left(w^{(0)}_{jk}x_{k}+b_{j}^{(0)}\right)+b_{i}^{(1)},$
(A.2)

where the matrices $w$ and vectors $b$ are parameters known as the weights and
biases, superscripts denote the weights and biases of the $0^{\text{th}}$ and
$1^{\text{st}}$ layer, and $\sigma:\mathbb{R}\to\mathbb{R}$ is an element-wise
nonlinearity that is chosen as part of the architecture choice, e.g.
$\sigma(z)=\tanh(z)$ or $\sigma(z)=\text{ReLU}(z):=\text{max}(0,z)$. Einstein
summation is implied, and the full set of parameters is
$\theta=\\{w^{(1)},b^{(1)},w^{(0)},b^{(0)}\\}$. Our general sketch is that a
neural network is a complicated function composed out of simpler functions
according to the choice of architecture, and we see that sketch borne out
here. Namely, we have

$f:\mathbb{R}^{n}\xrightarrow{w^{(0)}_{jk}(\cdot)+b^{(0)}_{j}(\cdot)}\mathbb{R}^{N}\xrightarrow{\sigma(\cdot)}\mathbb{R}^{N}\xrightarrow{w^{(1)}_{ij}(\cdot)+b^{(1)}_{i}(\cdot)}\mathbb{R}^{m},$
(A.3)

showing that the network is a parameter-dependent affine transformation
composed with an elementwise non-linearity $\sigma$ and then another
parameter-dependent affine transformation. The hyperparameter $N$ is known as
the _width_ of the fully-connected network, but _depth_ can be added by
sticking in more compositions of affine transformations and nonlinearities.
More generally, a neural network is called a _deep neural network_ as the
number of compositions is increased.

At initialization, a fixed network with fixed parameters $\theta$ is drawn
from some distribution of functions. It is a random function that has not yet
been tuned to any particular task. However, after initialization, the network
is _trained_ to achieve some goal, such as those mentioned above, by updating
the parameters $\theta$. The most famous update algorithm is _gradient
descent_ (GD), which updates the parameters according to

$\frac{d\theta_{I}}{dt}=-\frac{\partial{\cal L}[f]}{\partial{\theta_{I}}},$
(A.4)

where $\mathcal{L}$ is a scalar loss functional that depends on $f$, and
therefore on the parameters $\theta$. We have written the continuum time
equation, but in practice discrete time steps are taken on a computer.
Typically $\mathcal{L}$ is a sum of an element-wise loss $\ell$

$\mathcal{L}=\sum_{x^{\prime}\in B}\ell[f](x^{\prime}),$ (A.5)

where $B$ is a set of data points known as the _batch_. A simple variant of
gradient descent is _stochastic gradient descent_ (SGD), which at a fixed time
step $t$ during training applies gradient descent to a random (hence,
stochastic) proper subset $B_{t}\subset B$ known as the _mini-batch_. While GD
gets stuck when $\theta$ is at a critical point of the loss function, SGD
often escapes the critical point due to the stochastic nature of the mini-
batch.

Clearly this brief introduction only scratches the surface, but it should aid
in reading this paper. For the reader interested in a thorough introduction to
neural networks, we recommend [36, 37].

##  Appendix B Kähler-Ricci flows for String Theory

It has been recognized early on that a Ricci-flat manifold with external
Minkowski space and a constant dilaton provide a solution to the supergravity
equations of motion from string theory [38]. Hence, Perelman’s or Hamilton’s
Ricci flow can be used to find consistent background metrics for string
compactifications. If there is no additional structure to the compactification
space, and we are just interested in finding a Ricci-flat metric in a given
topological class (e.g. for the case of M-theory compactifications on $G_{2}$
manifolds), this seems like a promising avenue.

In the case of compactifications on Calabi-Yau manifolds, we can make use of
the fact that the metric is Kähler, which means it can be obtained from taking
a holomorphic and an anti-holomorphic derivative of a real quantity $K$ called
the Kähler potential,222The Kähler potential is only determined up to Kähler
transformations, which means that $K$ is the section of a line bundle.

$\displaystyle g_{a\bar{b}}=\partial_{a}\bar{\partial}_{\bar{b}}K\,,$ (B.1)

subject to the constraint that the metric is positive definite. We can use
this metric to define a closed (1,1)-form, the so-called Kähler form $J$, via

$\displaystyle J=\frac{i}{2}\,g_{a\bar{b}}\,dz^{a}\wedge d\bar{z}^{\bar{b}}$
(B.2)

Due to this special structure, many quantities simplify. For example, the
Christoffel symbols can only have 3 holomorphic or 3 anti-holomorphic indices.
For the Ricci tensor, one finds the simple expression

$\displaystyle R_{a\bar{b}}=-\partial_{a}\bar{\partial}_{\bar{b}}\ln g\,$
(B.3)

where $g=\det(g_{a\bar{b}})$. Inserting this in Hamilton’s formulation of the
Ricci-Flow, one finds

$\displaystyle\frac{dg_{a\bar{b}}}{dt}=\partial_{a}\bar{\partial}_{\bar{b}}\ln
g\,.$ (B.4)

We immediately see that the metric update for Kähler-Ricci flows is
$\partial$-exact and $\bar{\partial}$-exact, and therefore the Kähler class is
fixed under Kähler-Ricci flow.

We can further exploit the fact that the metric and all quantities that are
derived from it are controlled by the $K$ and formulate the metric flow on the
level of the Kähler potential, i.e., on the space333We want to point out two
potential points of confusion that are owed to using standard symbols for
quantities. The quantity $\varphi$ is not the same as the dilaton $\phi$.
Likewise, the holomorphic top form $\Omega$ that will appear later is not
related to the $\Omega$ appearing in the NTK kernel, and the loss function
$\sigma$ is not related to any Gaussian width or noise variance.

$\displaystyle\mathcal{H}=\\{\varphi\in
C^{\infty}(X)~{}|~{}J_{0}+\frac{i}{2\pi}\partial\bar{\partial}\varphi>0\\}\,.$
(B.5)

Here $J_{0}$ is a reference Kähler form (derived from a reference Kähler
metric) which specifies the Kähler class of the metric we are interested in
and is assumed to be constant throughout the flow. This means that the metric
flow of

$\displaystyle
g_{a\bar{b}}(t)=g_{0,a\bar{b}}+\frac{i}{2\pi}\partial\bar{\partial}\varphi(t)=\partial_{a}\bar{\partial}_{\bar{b}}(K_{0}+\frac{i}{2\pi}\varphi(t))$
(B.6)

is induced by the flow of the Kähler potential correction $\varphi(t)$. This
allows us to recast the problem of finding a Ricci-flat metric into the
problem of solving a partial differential equation of Monge-Ampere type, which
is what Yau used to prove Calabi’s conjecture [2, 3]. On the level of the
flow, this leads to a parabolic flow equation in $\varphi$.

Just as in the case of the Perelman functional, whose functional variation
gives rise to Hamilton flow (after a time-dependent diffeomorphism), there are
several related functionals for flows in $\varphi$: the Mabuchi energy
functional [39], the $J$-functional [40, 41], and the Calabi functional [42].
The $J$-functional is the second term of the Mabuchi energy functional and the
Calabi functional is the square of the derivative of the $J$-functional [43].
We refer the reader to [44, 45, 46] for more in-depth discussions of these
quantities, and to [20] for a nice overview and application to CY metrics.

The Calabi functional reads

$\displaystyle E_{\text{Calabi}}=\int_{X}R^{2}J^{n}\,,$ (B.7)

where $R$ is the scalar curvature and $J^{n}$ is the integral measure. The
flow associated with Calabi’s functional (B.7) is

$\displaystyle\frac{dg_{a\bar{b}}}{dt}=\partial_{a}\bar{\partial}_{\bar{b}}R\,,\qquad\qquad
R=g^{a\bar{b}}R_{a\bar{b}}=g^{a\bar{b}}\partial_{a}\bar{\partial}_{\bar{b}}\ln
g\,.$ (B.8)

Note that the flow ends on metrics with constant curvature, $R=\text{const}$.
For Calabi-Yau manifolds, this condition is equivalent to the seemingly
stronger condition of Ricci flatness $R_{a\bar{b}}=0$ [45]: a trivial anti-
canoncial bundle means that the first Chern class of the Calabi-Yau manifold
is zero, $c_{1}(X)=0$, which implies that the Ricci tensor is exact,
$R_{a\bar{b}}=\frac{i}{2\pi}\partial_{a}\bar{\partial}_{\bar{b}}h$, for some
function $h$. Taking the trace on both sides gives $\Delta h=0$, which means
$h$ is constant and hence $R_{a\bar{b}}=0$. Since
$g_{a\bar{b}}=g_{0,a\bar{b}}+\frac{i}{2\pi}\partial_{a}\bar{\partial}_{\bar{b}}\varphi$,
and $g_{0,a\bar{b}}$ is constant along the flow, the Calabi flow (B.8) can be
written (up to an integration constant which can be fixed from the initial
conditions) as

$\displaystyle\frac{d\varphi}{dt}=g^{a\bar{b}}\partial_{a}\bar{\partial}_{\bar{b}}\ln
g=\Delta\ln[\det(g_{0,c\bar{d}}+\partial_{c}\bar{\partial}_{\bar{d}}\varphi)]\,.$
(B.9)

The $\varphi$-flows relevant for us has already been discussed in [20], so we
merely review it here following their exposition. As relevant background for
utilizing their results, recall that a trivial anti-canonical bundle means
that there exists a nowhere-vanishing holomorphic top form $\Omega$. In
contrast to the Ricci-flat metric, analytic expressions for this $(n,0)$-form
are known and can be computed straight-forwardly for Calabi-Yau manifolds that
are given as complete intersections in toric ambient spaces [47, 48]. From
$\Omega$, one can construct a volume form
$\mu_{\text{CY}}=(-i)^{n}\Omega\wedge\bar{\Omega}$. Since $h^{n,n}=1$ on a
Calabi-Yau, this volume form is uniquely defined up to a constant. This means
that $\mu_{J}=J^{n}/n!$, which is also an $(n,n)$-form, must be proportional
to $|\Omega|^{2}$,

$\displaystyle J^{n}=\kappa|\Omega|^{2}\,.$ (B.10)

In the literature, it is customary to define the sigma loss

$\displaystyle\sigma:=|1-\eta|\,,\qquad\qquad\eta=\frac{J^{n}}{\kappa|\Omega|^{2}}\,.$
(B.11)

Hence, the Calabi-Yau metric has $\eta=1$ and $\sigma=0$.

The authors propose two classes of energy functionals: Ultra-local (meaning no
derivatives) and two-derivative functionals. The ultra-local energy
functionals are of the form

$\displaystyle E_{\text{ultra}}=\int_{X}F(\eta)\mu_{\text{CY}}\,,$ (B.12)

for any differentiable, convex function $F:\mathbbm{R}^{+}\to\mathbbm{R}$ that
is bounded from below. This has the variation

$\displaystyle\delta
E_{\text{ultra}}=\frac{1}{2}\int_{X}\mu_{J}\nabla^{2}F^{\prime}(\eta)\varphi\,,$
(B.13)

which implies that $F^{\prime}(\eta)$ is constant. The authors propose using

$\displaystyle F(\eta)=(\eta-1)^{2}\,,$ (B.14)

since it is simple, contains no derivatives, and is convex. Choosing
$F=\eta\ln\eta$, i.e., the (negative) von Neumann entropy of the probability
distribution $\eta$ on $X$, leads to a gradient flow that is the Calabi flow
(B.8).

For the two-derivative case, the authors consider

$\displaystyle
E_{\text{2deriv}}=\int_{X}\mu_{\text{CY}}g^{a\bar{b}}\partial_{a}\eta\bar{\partial}_{\bar{b}}\eta\,,$
(B.15)

which has the variation

$\displaystyle\delta
E_{\text{2deriv}}=4\int_{X}\mu_{\text{CY}}\eta^{1/2}\partial^{a}\bar{\partial}^{b}\varphi\partial_{a}\bar{\partial}_{\bar{b}}\eta^{-1/2}\,.$
(B.16)

As before, $E_{\text{2deriv}}$ is stationary iff $\eta$ is constant.
Furthermore, since $J^{n}\propto\det(g_{a\bar{b}})$, we have
$R_{a\bar{b}}=-\partial_{a}\bar{\partial}_{\bar{b}}\ln\eta$ and the functional
(B.15) becomes proportional to the Einstein-Hilbert action

$\displaystyle E_{\text{EH}}=-\frac{1}{2}\int_{X}\mu_{\text{CY}}R\,.$ (B.17)

This is also the variation of the $J$-functional and the “square root” of the
Calabi functional. As explained before, the scalar curvature of a Calabi–Yau
metric is zero and hence this functional is identically zero.

Let us finally come back to Ricci flow and the Perelman functional. Our goal
is to realize Ricci flow as a $\varphi$-flow, and then realize $\varphi$-flow
as a neural network metric flow. Setting $\phi=\ln\eta$ in (2.41) (note that
$\eta=1$ is constant for the Calabi-Yau, as is the dilaton $\phi$), the
Perelman functional reads

$\displaystyle\mathcal{F}=-\int_{X}\mu_{\text{CY}}(R+g^{ab}\partial_{a}\ln\eta\;\partial_{b}\ln\eta)\,.$
(B.18)

As we have just noted,
$R+g^{a\bar{b}}\partial_{a}\ln\eta\;\bar{\partial}_{\bar{b}}\ln\eta=0$ for the
Calabi-Yau case. This suggests that instead of the Ricci flow (2.39) obtained
from the Perelman functional $\mathcal{F}$, one can take the flow
corresponding to $E_{\text{2deriv}}$ in (B.15), which gives

$\displaystyle\frac{dg_{a\bar{b}}}{dt}=-\partial_{a}\bar{\partial}_{\bar{b}}\eta^{-1/2}\,,$
(B.19)

or, using
$g_{a\bar{b}}=g_{0,a\bar{b}}+\partial_{a}\bar{\partial}_{\bar{b}}\varphi$,

$\displaystyle\frac{d\varphi}{dt}=-\left(\frac{\det(g_{0,a\bar{b}}+\partial_{a}\bar{\partial}_{\bar{b}}\varphi)}{\mu_{\text{CY}}}\right)^{-1/2}\,,$
(B.20)

Finally, the gradient flow associated with the ultra local energy functional
(B.14) is

$\displaystyle\frac{d\varphi}{dt}=-2(\eta-1)=-2\left(\frac{\det(g_{0,a\bar{b}}+\partial_{a}\bar{\partial}_{\bar{b}}\varphi)}{\mu_{\text{CY}}}\right)\,.$
(B.21)

This flow ends on the CY metric, for which $\eta=1$.

## References

  * [1] E. Calabi “On Kähler manifolds with vanishing canonical class,” Algebraic geometry and topology. A symposium in honor of S. Lefschetz (1957) 78–89.
  * [2] S.-T. Yau “Calabi’s Conjecture and Some New Results in Algebraic Geometry,” Proceedings of the National Academy of Science 74 (May, 1977) 1798–1799.
  * [3] S.-T. Yau “On the ricci curvature of a compact kähler manifold and the complex monge-ampére equation, i,” Communications on Pure and Applied Mathematics 31 (1978) no. 3, 339–411 [https://onlinelibrary.wiley.com/doi/pdf/10.1002/cpa.3160310304].
  * [4] P. Candelas, A. M. Dale, C. A. Lutken, and R. Schimmrigk “Complete Intersection Calabi-Yau Manifolds,” Nucl. Phys. B 298 (1988) 493\.
  * [5] M. Kreuzer and H. Skarke “Complete classification of reflexive polyhedra in four-dimensions,” Adv. Theor. Math. Phys. 4 (2002) 1209–1230 [hep-th/0002240].
  * [6] J. Halverson, C. Long, and B. Sung “Algorithmic universality in F-theory compactifications,” Phys. Rev. D 96 (2017) no. 12, 126006 [1706.02299].
  * [7] R. Harvey and H. B. Lawson “Calibrated geometries,” Acta Mathematica 148 (1982) 47 – 157.
  * [8] P. Candelas and X. C. de la Ossa “Moduli space of Calabi-Yau manifolds,” in XIII International School of Theoretical Physics: The Standard Model and Beyond.  9, 1989.
  * [9] S. K. Donaldson “Some numerical results in complex differential geometry,” [math/0512625].
  * [10] L. B. Anderson, M. Gerdes, J. Gray, S. Krippendorf, N. Raghuram, and F. Ruehle “Moduli-dependent Calabi-Yau and SU(3)-structure metrics from Machine Learning,” JHEP 05 (2021) 013 [2012.04656].
  * [11] M. R. Douglas, S. Lakshminarasimhan, and Y. Qi “Numerical Calabi-Yau metrics from holomorphic networks,” [[arxiv:2012.04797]].
  * [12] V. Jejjala, D. K. Mayorga Pena, and C. Mishra “Neural Network Approximations for Calabi-Yau Metrics,” [[arxiv:2012.15821]].
  * [13] M. Larfors, A. Lukas, F. Ruehle, and R. Schneider “Learning Size and Shape of Calabi-Yau Spaces,” in Accepted to Proceedings of Advances in Neural Information Processing Systems 34 - Machine Learning and the Physical Sciences.  11, 2021.  [2111.01436].
  * [14] M. Larfors, A. Lukas, F. Ruehle, and R. Schneider “Numerical metrics for complete intersection and Kreuzer-Skarke Calabi-Yau manifolds,” Mach. Learn. Sci. Tech. 3 (2022) no. 3, 035014 [2205.13408].
  * [15] M. Gerdes and S. Krippendorf “CYJAX: A package for Calabi-Yau metrics with JAX,” Mach. Learn. Sci. Tech. 4 (2023) no. 2, 025031 [2211.12520].
  * [16] A. Jacot, F. Gabriel, and C. Hongler “Neural tangent kernel: Convergence and generalization in neural networks,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, eds. vol. 31.  Curran Associates, Inc. 2018.
  * [17] J. Lee, L. Xiao, S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, and J. Pennington “Wide neural networks of any depth evolve as linear models under gradient descent,” in Advances in Neural Information Processing Systems vol. 32 p. 8572.  2019\.
  * [18] G. Yang “Tensor programs ii: Neural tangent kernel for any architecture,” arXiv preprint arXiv:2006.14548 (2020).
  * [19] G. Perelman “The entropy formula for the ricci flow and its geometric applications,” [math/0211159].
  * [20] M. Headrick and A. Nassar “Energy functionals for Calabi-Yau metrics,” Adv. Theor. Math. Phys. 17 (2013) no. 5, 867–902 [0908.2635].
  * [21] J. Halverson “Building Quantum Field Theories Out of Neurons,” [2112.04527].
  * [22] J. Halverson, A. Maiti, and K. Stoner “Neural Networks and Quantum Field Theory,” Mach. Learn. Sci. Tech. 2 (2021) no. 3, 035002 [2008.08601].
  * [23] B. Kleiner and J. Lott “Notes on perelman’s papers,” Geometry and Topology 12 (nov, 2008) 2587–2855.
  * [24] R. Novak, L. Xiao, J. Hron, J. Lee, A. A. Alemi, J. Sohl-Dickstein, and S. S. Schoenholz “Neural tangents: Fast and easy infinite neural networks in python,” [1912.02803].
  * [25] J. Lee, S. Schoenholz, J. Pennington, B. Adlam, L. Xiao, R. Novak, and J. Sohl-Dickstein “Finite versus infinite neural networks: an empirical study,” Advances in Neural Information Processing Systems 33 (2020) 15156–15172.
  * [26] R. Novak, J. Sohl-Dickstein, and S. S. Schoenholz “Fast finite width neural tangent kernel,” [2206.08720].
  * [27] F. Nogueira “Bayesian Optimization: Open source constrained global optimization tool for Python.” https://github.com/fmfn/BayesianOptimization 2014–.
  * [28] J. Snoek, H. Larochelle, and R. P. Adams “Practical bayesian optimization of machine learning algorithms,” in Advances in Neural Information Processing Systems, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, eds. vol. 25.  Curran Associates, Inc. 2012.
  * [29] H. Ahmed and F. Ruehle “Level crossings, attractor points and complex multiplication,” JHEP 06 (2023) 164 [2304.00027].
  * [30] A. Krizhevsky, I. Sutskever, and G. E. Hinton “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, eds. vol. 25.  Curran Associates, Inc. 2012.
  * [31] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich “Going deeper with convolutions,” [1409.4842].
  * [32] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen “Hierarchical text-conditional image generation with clip latents,” [2204.06125].
  * [33] D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis “Mastering chess and shogi by self-play with a general reinforcement learning algorithm,” [1712.01815].
  * [34] D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis “Mastering the game of go without human knowledge,” Nature 550 (Oct, 2017) 354–359.
  * [35] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin “Attention is all you need,” Advances in neural information processing systems 30 (2017).
  * [36] F. Ruehle “Data science applications to string theory,” Phys. Rept. 839 (2020) 1–117.
  * [37] M. M. Bronstein, J. Bruna, T. Cohen, and P. “Geometric deep learning: Grids, groups, graphs, geodesics, and gauges,” arXiv preprint arXiv:2104.13478 (2021).
  * [38] P. Candelas, G. T. Horowitz, A. Strominger, and E. Witten “Vacuum Configurations for Superstrings,” Nucl. Phys. B 258 (1985) 46–74.
  * [39] T. Mabuchi “K-energy maps integrating futaki inyariants,” Tohoku Mathematical Journal 38 (1986) 575–593.
  * [40] X. Chen “On the lower bound of the Mabuchi energy and its application,” International Mathematics Research Notices 2000 (01, 2000) 607–623.
  * [41] S. K. Donaldson “Moment maps and diffeomorphisms,” Surveys in differential geometry 3 (2002) 107–127.
  * [42] E. Calabi “Extremal kähler metrics,” in Seminar on Differential Geometry vol. 102 of Ann. of Math. Stud. pp. 259–290.  Princeton University Press (PUP) Princeton, N.J. 1982.
  * [43] K. Zheng “I-properness of mabuchi’s k-energy,” Calculus of Variations and Partial Differential Equations 54 (2015) no. 3, 2807–2830.
  * [44] G. Tian Canonical Metrics in Kähler Geometry.  Canonical Metrics in Kähler Geometry. Birkhäuser 2000.
  * [45] J. Song and B. Weinkove “Lecture notes on the Kähler-Ricci flow,” [1212.3653].
  * [46] G. Székelyhidi An Introduction to Extremal Kahler Metrics.  Graduate Studies in Mathematics. American Mathematical Society 2014.
  * [47] E. Witten “Symmetry Breaking Patterns in Superstring Models,” Nucl. Phys. B 258 (1985) 75.
  * [48] A. Strominger and E. Witten “New Manifolds for Superstring Compactification,” Commun. Math. Phys. 101 (1985) 341.