#  Learning Group Invariant Calabi–Yau Metrics by Fundamental Domain
Projections

Yacoub<EMAIL_ADDRESS>Magdalena
<EMAIL_ADDRESS>Moritz Walden333
<EMAIL_ADDRESS>
###### Abstract

We present new invariant machine learning models that approximate the Ricci-
flat metric on Calabi–Yau (CY) manifolds with discrete symmetries. We
accomplish this by combining the $\phi$-model of the cymetric package with
non-trainable, $G$-invariant, canonicalization layers that project the
$\phi$-model’s input data (i.e. points sampled from the CY geometry) to the
fundamental domain of a given symmetry group $G$. These $G$-invariant layers
are easy to concatenate, provided one compatibility condition is fulfilled,
and combine well with spectral $\phi$-models. Through experiments on different
CY geometries, we find that, for fixed point sample size and training time,
canonicalized models give slightly more accurate metric approximations than
the standard $\phi$-model. The method may also be used to compute Ricci-flat
metric on smooth CY quotients. We demonstrate this aspect by experiments on a
smooth $\mathbb{Z}^{2}_{5}$ quotient of a 5-parameter quintic CY manifold.

Department of Physics and Astronomy & Department of Mathematics,
Uppsala University,
Lägerhyddsv. 1, SE-751 20 Uppsala, Sweden.

###### Contents

  1. 1 Introduction
  2. 2 Related work
  3. 3 Machine learning Ricci-flat CY metrics
    1. 3.1 CY construction
    2. 3.2 The $\phi$-model of the cymetric package
    3. 3.3 Spectral layers
  4. 4 Invariant metric by fundamental domain projections
    1. 4.1 Examples
      1. 4.1.1 Homogeneous rescalings
      2. 4.1.2 Permutations
      3. 4.1.3 Scaling by roots of unity
    2. 4.2 Combining invariant layers
  5. 5 Experiments
    1. 5.1 Fermat quintic
    2. 5.2 Tetraquadric
    3. 5.3 A 2-polynomial CICY
  6. 6 Ricci-flat metrics on non simply connected CY manifolds
  7. 7 Conclusion
  8. A CY geometry
  9. B Computing volume predictions
  10. C Computing error measures
  11. D Computational performance

##  1 Introduction

String compactifications lie at the heart of many formal and phenomenological
studies of string theory. By compactifying some of the spatial dimensions of
the space-time probed by string theory one can construct models that match
observations from particle physics and cosmology, or discover how different
limits of string theory are related by duality. The topology and geometry
formed by the compactified dimensions are key to these constructions, and
there is an interesting interplay between physics and geometric constraints.
Among the compact spaces, Calabi–Yau manifolds provide a sweet spot; their
mathematics is highly non-trivial, yet examples can be constructed in the
thousands.

Calabi–Yau (CY) manifolds are complex, Kähler manifolds, with vanishing first
Chern class; by the Calabi–Yau theorem these spaces admit a Ricci-flat metric,
which is unique in a given Kähler class. Yau [1] has proven that the Kähler
form $J$ associated to the Ricci-flat CY metric $g_{\text{CY}}$ satisfies a
complex Monge-Ampère (MA) equation, which it reads, in $n$ complex dimensions,

$J\wedge J\wedge...\wedge J=\kappa\,\Omega\wedge\overline{\Omega}\;,$ (1)

where we wedge $n$ copies of $J$ on the left hand side, $\Omega$ denotes the
unique holomorphic top form on the CY manifold, and $\kappa$ is a complex
constant. There is no known analytic solution to this equation for $n>1$.444In
one complex dimension, the CY is a torus, which admits a flat metric. In 2D,
i.e. for a K3 surface, a duality argument has been proposed to determine the
Ricci-flat metric [2, 3].

Lacking an analytic solution to a partial differential equation (PDE), one may
try to find numerical solutions. This is indeed possible, owing to the fact
that, since $J$ is related to any reference Kähler for $J^{\prime}$ via

$J=J^{\prime}+i\partial\bar{\partial}\phi\;,$ (2)

the MA equation can be rewritten as a second order PDE for a real function
$\phi$. One approach to solve the PDE for $\phi$, is to expand $\phi$ in
degree-$k$ polynomials on the CY, an idea that goes back to Tian [4], and then
solve for the expansion coefficients. This can be done using a fixed-point
iteration scheme, known as the Donaldson algorithm, [5, 6, 7, 8], or via
functional minimization [9, 10, 11]; collectively, the metrics obtained in
this way are called algebraic metrics. Since the polynomial, or spectral,
basis is complete in the limit $k\to\infty$ [5, 9], this method will indeed
give the unique Ricci-flat metric in this limit. However, for most CY spaces,
the spectral basis dimension grows rapidly with $k$, implying that practical
applications are limited to rather small $k$.555For a CY hypersurface in
$\mathbb{CP}^{N}$, the dimension of the polynomial basis scales like $k^{2N}$
for large $k$ [10]. For symmetric CY geometries, using an invariant spectral
basis, of smaller dimension, allows us to compute the metric at larger $k$ (in
fact, the most accurate approximations of CY metrics are obtained on highly
symmetric Fermat quintics [9]).666However, constructing the invariant basis
sometimes comes at a non-trivial computational cost, as we will discuss in
section 6.

With the advent of machine learning (ML), novel opportunities, based on neural
networks, have arisen to approximate CY metrics. Since neural networks are
universal approximators [12, 13, 14], they can learn any function; thus
$\phi$, or why not the Ricci-flat CY metric directly, may be learned by an ML
model. Developing machine learning methods for CY geometry has been an active
field of research in later years [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27]. A major practical gain of ML methods is that they are less dependent
on expansions in the spectral basis. While ML methods may be used in
combination with the spectral expansion of $\phi$, they can also make away
completely with this crutch. As such, the ML methods remain computationally
feasible on generic CY manifolds, as evidenced in Refs. [21, 22, 27].

Admittedly, the accuracy of ML algorithms has been observed to improve on more
symmetric spaces [18, 19, 24, 27], and the training time (for similar
performance) is slightly longer on non-symmetric spaces [27]. These
differences, which are small compared to the effects seen in pre-ML methods,
indicate that the ML models learn the symmetries of the data they are trained
on. Symmetries have also been used to test the accuracy of ML predictions of
CY metrics [17]. However, discrete symmetries have, so far, not played any
significant role in the design of ML models for CY metrics. In this paper, we
will perform a first exploration of this topic, using novel methods to
construct group invariant ML models.777In methods based on spectral basis,
such as [16, 23] and [18, 19], one can restrict to group invariant elements of
the basis; to the knowledge of the authors this has not been explored to any
greater degree.

The motivation for our study is two-fold. First, as we will review below, in
the ML literature there is ample evidence that for symmetric data, equivariant
and invariant models outperform networks that are agnostic about symmetries in
the data. The Geometric Deep Learning program [28, 29] shows how symmetries
explain the success of state-of-the-art ML models such as such as CNNs, GNNs,
and Transformers; ref. [30] analyse the CNNs using the concept of covariance
in theoretical physics. Formally, it has been proven that group invariant
neural networks are universal approximators for group invariant functions
[31], thus symmetry-enforcing ML models are as expressive as the symmetry-
agnostic ones.

Secondly, from the perspective of string theory, there is phenomenological
motivation to study non-simply connected CY geometries, obtained by
quotienting a CY manifold by a freely acting discrete symmetry $G$.
$G$-invariant ML models, provides a new way to compute the Ricci-flat metrics
of such quotients. This problem has been explored using Donaldson’s algorithm
in the past [8], with positive effects on the accuracy of the metric
approximation. Ref. [8] also explains the technical subtleties associated with
the creation of a group invariant spectral basis; this provides further
motivation to explore invariant ML methods, as do recent application of ML
methods to the computations of heterotic Yukawa couplings on CY manifolds with
discrete symmetries [32, 33].

##### Present work

In this work, we will design group invariant ML models that provide
approximations of the Ricci-flat metric of CY manifolds. Our work is motivated
by the importance of symmetries in the context of spectral methods for CY
metrics, and the role symmetries play in leading ML models.

For our studies, we will explore CY geometries created as complete
intersections in complex projective spaces, so-called CICYs [34]. We will use
the cymetric package, introduced in [21, 22], which allows us to sample points
on such spaces, and provide several ML model architectures that either predict
the function $\phi$, or the metric directly. In this work, we will limit our
attention to the $\phi$-model. To create group invariant $\phi$-models, we
will include linear, non-trainable layers into the neural network
architecture. These layers pre-process the the input data of the $\phi$-model,
and ensure that it is $G$-invariant by projecting to a fundamental domain of
the group $G$. This idea, which is often referred to as canonicalization, has
recently been proposed in [35, 36]. Our main contribution is a comparison of
the performance and accuracy of the canonicalized ML models with the standard
$\phi$-model. We also discuss the similarities and differences of the
canonicalization approach with the spectral networks suggested by [24], and
how canonicalization can be combined with spectral $\phi$-models. Finally, we
study how canonicalized ML models can be used to compute the metric on a non-
simply connected CY manifold.

This paper is organised as follows. In section 2, we briefly review related
work on machine learned, Ricci-flat CY metrics, and outline the key approaches
to group invariant ML models. Section 3 describes CY geometry and the ML model
we will use for our experiments. We also recall how the spectral layer is
constructed. In section 4, we present our main contribution: how to construct
canonicalization layers for the symmetry groups relevant for CY point samples;
experiments using these techniques are presented and discussed in section 5.
Some technical aspects of our analysis are described in appendices. In section
6 we apply canonicalization layers to the computation of Ricci-flat CY metrics
on smooth quotients of a 5-parameter quintic CY. Conclusions and discussions
are given in section 7.

##  2 Related work

The past five years have witnessed a flurry of research activities on ML
methods for Ricci-flat CY metrics [15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25], and there are several recent examples in the literature on how these ML
methods may be used to address open questions in string theory [25, 32, 33,
37, 38]. Since the Ricci-flat CY metric is unknown, all ML models for CY
metrics rely on some sort of semi-supervised learning, where a custom loss
function is used to ensure that the learned metric is Ricci-flat. This is in
reminiscent of Physics Informed Neural Networks (PINNs) [39]; in the parlance
of PINNs, the Ricci-flatness is encoded as a soft constraint.

To date, three open-source ML libraries have been developed. While we will
only explore one of these libraries in this work, the key features of the
other packages may also be of interest to the reader, and are therefor given
here

  * •

The MLgeometry package was developed for refs. [18, 19, 38]. This package
introduces the bihomogeneous (holomorphic) neural networks where the layers
are bihomogeneous (holomorphic) functions of the ambient space coordinates for
the CY manifold, with quadratic non-linear activation functions. These layers
encode the spectral basis of the algebraic CY metrics, and the learned metric
is automatically globally defined and Kähler (i.e. these are hard
constraints). The Ricci-flatness condition is imposed softly via a loss
function.

  * •

The cymetric package, presented in refs. [21, 22] and further tested in ref.
[40], instead encodes the CY metric, or alternatively the Kähler function,
directly in a dense, shallow, fully connected neural network (typically with
ReLU or GELU activation functions). Custom loss functions are used to enforce
that the learned metric is globally defined, Kähler, Ricci-flat, and
restricted to the Kähler class selected when starting the training.888The
motivation for the soft encoding of all mathematical constraints on the
metric, was to make the package generalize, at little cost, to other geometric
constructions, beyond the CY regime.

  * •

the cyjax package [23] is a spectral method. The ML model learns algebraic CY
metrics; the neural net encodes an Hermitian coefficient matrix, which, when
combined with the spectral basis, provides an approximation to the Ricci-flat
metric. This automatically provides a globally defined, Kähler metric; the
Ricci-flatness condition is imposed via a loss function.

In ref. [24], it was proposed that the $\phi$-model of the cymetric package
can be augmented by a non-linear, non-trainable spectral layer, where the
input data is passed to the $k=1$ spectral basis, and then fed to the dense,
trainable layers of the model. The benefit of this layer is that the predicted
metric is automatically globally defined and Kähler. Thus, some of the soft
constraints of the $\phi$-model become hard constraints with the addition of a
spectral layer. In ref. [24], this was shown to be of essence in learning the
Ricci-flat metric on singular, or nearly singular, CY manifolds. The spectral
layer is, from a machine learning perspective, a data pre-processing step; as
we will discuss below it shares some features with the ideas explored in this
paper.

In this paper, we will be interested in predicting a function
$\phi:X\to\mathbb{R}$, where $X$ is a CY manifold. Working locally, we may
decompose $X$ into patches which are isomorphic to $\mathbb{R}^{n}$. The
prediction of a function

$f:\mathbb{R}^{n}\,\to\,\mathbb{R}$

is a standard task in machine learning. When the function domain has
symmetries, invariant (and equivariant) machine learning models are of
interest. This topic is extensively studied. An excellent starting point for
studies of the mathematical foundations of invariant and equivariant ML models
can be found in the book [29], which also provides a comprehensive list of
references to this field of data science.

There are different approaches to the construction of invariant machine
learning models. Following Yarotsky [31], see also [35], one may distinguish
between symmetrization-based and intrinsic approaches. Following the first
approach, one may, for example, construct $G$-invariant models by augmenting
the training data by $G$-transformed copies with the same label as the
original data point. The network will then learn that data points related by
$G$-transformations give the same output, as required by invariance. When
applied to unseen data, a network trained on augmented data will perform
better than a network trained on non-augmented data. As discussed in ref.
[41], from a mathematical point of view, and under exact invariance, data
augmentation is equivalent to averaging over orbits of the group $G$; it can
thus be proven that this reduces the variance of the prediction of a model.

Intrinsic approaches instead produce invariant networks by imposing
restrictions on the network design. CNNs, which are equivariant under
translations, are early incarnations of this idea [42, 43]. Other network
designs that are invariant under other group transformations include [44, 45,
46, 47, 48]; these and related ideas are further discussed in [29] and the
review [49]. Common to these designs is that in- or equivariance is
implemented by weight-sharing in some layer of the network.

Restricting the trainable weights of some layer in the network is not the only
intrinsic way to achieve invariant/equivariant layers. Yarotsky has shown,
using classical polynomial invariant theories, that an extra polynomial layer
wich is invariant/equivariant and complete, results in an
invariant/equivariant network [31]. This is particularly useful for finite
groups, where complete invariant layers can be constructed from a finite
number of polynomial invariants. For continuous groups, invariant layers can
also be complete, but, lacking completeness, one cannot prove that they are
universal invariant approximators [31]. The spectral layer [24] (described
briefly above and in detail below) is, up to rescaling by the norm, an
incarnation of such an invariant layer for the continuous group
$\mathbb{C}^{*}$.

Of key interest to us is the alternative intrinsic approach proposed in [35,
36]. Here, a pre-processing step is used to put the data into a canonical
form, by projection to a fundamental domain of the symmetry group. The
canonicalization can be achieved in different ways, and we will describe this
method in more detail below. Importantly, the method works for both continuous
and discrete groups. It is also good to note that, in contrast to weight-
sharing schemes, that methods based on polynomial invariants or
canonicalization are not restricted to models using neural networks. Since the
concatenation of an invariant map with any other map preserves invariance,
such data pre-processing steps can be used to impose invariance on virtually
any type of ML model, for example the packages developed for machine learning
of CY metrics.

##  3 Machine learning Ricci-flat CY metrics

In this section, we review some aspects of the construction of examples of CY
manifolds, and present some facts about their symmetries. A more detailed
discussion of the symmetries is relegated to appendix A. We then describe the
key points of the cymetric package, and the ML methods we employ to learn CY
metrics. For more details on the cymetric package, we refer to [21, 22].

###  3.1 CY construction

For our experiments, we need explicit realizations of CY geometries. In this
paper, we will restrict our studies to the Complete Intersection Calabi Yau
(CICY) manifolds [34, 50].999It would be interesting to extend our studies to
CY manifolds in toric ambient spaces. The cymetric package has point
generators also for such spaces, and the methods we develop here can be
adapted to this setting. We leave this generalisation for future work. These
are constructed as the zero loci of a set of homogeneous polynomials $p_{i}$
in an ambient space $\mathbb{CP}^{n}_{1}\times...\times\mathbb{CP}^{n}_{2}$. A
prime example is the Fermat quintic, defined by the solutions of

$\sum_{i=0}^{4}z_{i}^{5}=0$ (3)

in $\mathbb{CP}^{4}$. CICY manifolds are often described in terms of
configuration matrices, which in the case of the quintic is $[4|5]$. This
class of CY manifolds is fully classified, and a detailed list of their
topological properties can be found in [51].

CICY manifolds may, at certain subloci in their complex structure moduli
space, admit discrete symmetries.101010We define CY $3$-folds to have strict
$SU(3)$ holonomy; consequently, the spaces we study do not admit continuous
isometries. These are manifest as invariances of the defining polynomials of
the CICY. For example, the defining equation of the Fermat quintic (3) is
invariant under permutations of the coordinates, multiplications of the
coordinates by 5th roots of unity (up to a global multiplication by the same
root, which is removed by the scaling invariance of the homogeneous
coordinates), and complex conjugation; the full symmetry group is
$\mathbb{Z}_{5}^{4}\rtimes(S^{5}\times\mathbb{Z}_{2})$ [8]. The freely acting
symmetries of CICY manifolds have been classified in [52], non-freely acting
symmetries are classified in [53], and symmetries of CICY quotients are
classified in [54]. The invariance of the defining polynomials translates into
an isometry of the Ricci-flat CY metric (see appendix A for a proof).

We want to learn the Ricci-flat metric on some CICY manifold $X$ with
dimension $n$. Starting from some reference Kähler form $J^{\prime}$, we know,
by the CY theorem, that there exists a unique Kähler form $J$, in the same
cohomology class as $J^{\prime}$, which gives a Ricci-flat metric.

For simplicity, let us discuss the case where $X$ is realized as one
hypersurface in one projective space. All our statements generalize to the
case of complete intersections of hypersurfaces, so this is just for ease of
presentation. Let $\iota:X\hookrightarrow\mathbb{CP}^{n+1}$ be the inclusion
of $X$ in its ambient space, and $J^{\prime}$ be the pullback of the Kähler
form $\omega$ associated to the Fubini-Study metric on $\mathbb{CP}^{n+1}$ to
$X$, i.e. $J^{\prime}=\iota^{\star}\omega$. Then we want to learn a real
function $\phi:X\to\mathbb{R}$ such that

$J=J^{\prime}+i\,\partial\overline{\partial}\phi$ (4)

is Ricci-flat. We use the `cymetric`
library111111https://github.com/pythoncymetric/cymetric [21, 22] for learning
said metric, including the $\phi$-model, which encodes eq. (4).

###  3.2 The $\phi$-model of the cymetric package

The cymetric package consists of point generators, that create samples of
points on the CY manifold, and a number of machine learning models that take
the coordinates of the sampled points as input, and gives some data related to
the Ricci-flat CY metric as output. In the $\phi$-model, the output is a
numerical estimate of the real function $\phi$, defined in eq. (4).

Since data pre-processing is a theme of this paper, we need to carefully
describe the input data of the $\phi$-model. As mentioned, this data
represents points on the CY manifold, as constructed by the point generators
of the cymetric package; these routines are discussed in depth in ref. [22].
For simplicity, we continue to focus on the case where $X$ is a hypersurface
in $\mathbb{CP}^{n+1}$. Points in the CY $n$-fold $X$ are represented by
homogeneous coordinates, $z$, of the ambient space $\mathbb{CP}^{n+1}$, and
are repackaged, by the point generators, as a real tuple

$\left(\Re(z_{0})\quad\dots\quad\Re(z_{n+1})\quad\Im(z_{0})\quad\dots\quad\Im(z_{n+1})\right)\in\mathbb{R}^{2(n+2)}\,,$

where the homogeneous scaling invariance of $\mathbb{CP}^{n+1}$ has been used
to ensure that the input data take values in the range $[0,1]$. In accordance
with the philosophy of this paper, we will still refer to the network as a map
$f:X\to\mathbb{R}$.

In order to train the neural network, we rely on loss functions encoding the
relevant mathematical constraints. In the cymetric package, these constraints
are encoded in five custom loss functions. First, we want to ensure Ricci-
flatness. As mentioned in the introduction, this can be phrased as requiring
$J$ to obey the Monge–Ampère equation, which in 3 dimensions reads

$J\wedge J\wedge J=\kappa\,\Omega\wedge\overline{\Omega}\,,$ (5)

where $\kappa$ is some complex constant and $\Omega$ is the no-where vanishing
holomorphic $(n,0)$-from.121212Here $J$ is predicted by the network and
$\kappa$ and $\Omega$ are computed by the point generator. With this at hand,
the first loss function is defined as

$\mathcal{L}_{\textrm{MA}}=\left\lVert 1-\frac{1}{\kappa}\frac{J\wedge J\wedge
J}{\Omega\wedge\overline{\Omega}}\right\rVert_{1}\,,$ (6)

where $\lVert\cdot\rVert_{1}$ denotes the $L_{1}$ norm.131313The cymetric
package can use any $L_{n}$ norm for the loss functions, we use the default
setting of the package.

Furthermore, we want to ensure that $\phi$ is a global function. Hence we
define the transition loss

$\mathcal{L}_{\textrm{transition}}=\frac{1}{d}\sum_{(s,t)}\left\lVert\phi^{(t)}-\phi^{(s)}\right\rVert_{1}\,,$
(7)

where $s,t$ denote different patches and $d$ denotes the number of patch
transitions.

In the cymetric package, three more constraints are encoded as custom loss
functions $\mathcal{L}_{i}$: the Kähler constraint $\mathrm{d}J=0$ gives
$\mathcal{L}_{\text{dJ}}$, preserving the Kähler class is enforced by
$\mathcal{L}_{\text{Kclass}}$,141414The Kähler class loss function is
necessary when learning the metric on any CY manifold with $h^{(1,1)}>1$. It
reduces to a volume-preserving constraint when $h^{(1,1)}=1$. The cymetric
routines require such a loss since they can be applied to any CICY manifold,
and CY manifolds defined as hypersurface in the toric ambient spaces contained
in the Kreuzer–Skarke list [55].

$\displaystyle\mathcal{L}$
$\displaystyle=\alpha_{1}\mathcal{L}_{\text{MA}}+\alpha_{2}\mathcal{L}_{\text{dJ}}+\alpha_{3}\mathcal{L}_{\text{transition}}+\alpha_{4}\mathcal{L}_{\text{Ricci}}+\alpha_{5}\mathcal{L}_{\text{Kclass}}\;.$
(8)

Here $\alpha_{i}$ are tuneable hyperparameters of the model that are set by
the user. In our experiments, we use the default values for the cymetric
losses in the $\phi$-model; thus the Kähler and Ricci losses are disabled and
$\alpha_{i}=1$. We refer the reader to [21, 22] for more discussions of the
loss functions.

###  3.3 Spectral layers

The spectral network, introduced in [24], repackages the input data of the
$\phi$-model of the cymetric package in a form that is manifestly invariant
under homogeneous rescalings of the ambient space coordinates. The motivation
for this feature engineering step is to ensure that the learned function
$\phi$ is globally well-defined. Recall that this is only enforced ”softly”,
by a loss function, in the cymetric package. As a consequence, the network may
predict line bundle sections rather than globally defined functions. An
observed ill consequence of this is the discrepancy of topological quantities
computed with the predicted Ricci-flat metric [24], which becomes particularly
noticeable for singular, or nearly singular, CY spaces.

It may nevertheless appear surprising that such feature engineering is needed.
The homogeneous rescaling is not a symmetry of the CY manifold; this is a
complex, $n$-dimensional space with $n$ holomorphic coordinates. However, as
we have described in the previous section, the input data of the cymetric
models is not an $n$-dimensional complex tuple. Rather, for a CICY with
ambient space $\mathbb{CP}^{n_{1}}\times..\times\mathbb{CP}^{n_{k}}$, the
point generators of cymetric produces $\sum_{i}2(n_{i}+1)$-dimensional real
input data tuples (with entries in the range $[0,1]$). In this pre-processing
the points on the CY space are separated into coordinate patches (that overlap
but for zero measure sets). The training is then done locally, using the
transition loss function to impose that the function matches on patch
overlaps. It is an empirical fact that the transition loss function is around
$10^{-3}$ for standard $\phi$-models (trained for 10 epochs on the Fermat
quintic). As we will see in section 5, with the addition of a spectral layer
in the network, the transition loss function is of the order $10^{-9}$, which
indicates the prediction of $\phi$ is indeed a function.

The effect of the spectral layer is thus to impose invariance under
homogeneous rescalings. Following ref. [24] (see also [37]), we have encoded
the layer as follows.151515This encoding shows the relation between the
spectral layers and the bihomogeneous layers of [18]; they are essentially
related by a division by the norm. However, the bihomogeneous layers have
trainable weights, whereas the spectral layer is non-trainable. For a CY with
ambient space $\mathbb{CP}^{n}$, the spectral layer is a map

$(z_{0},\ldots,z_{n})\mapsto\begin{pmatrix}\frac{z_{0}\bar{z}_{0}}{|z|^{2}}&\frac{z_{0}\bar{z}_{1}}{|z|^{2}}&\ldots&\frac{z_{0}\bar{z}_{n}}{|z|^{2}}\\\
\frac{z_{1}\bar{z}_{0}}{|z|^{2}}&\frac{z_{1}\bar{z}_{1}}{|z|^{2}}&\ldots&\frac{z_{1}\bar{z}_{n}}{|z|^{2}}\\\
\vdots&\vdots&\ddots&\vdots\\\
\frac{z_{n}\bar{z}_{0}}{|z|^{2}}&\frac{z_{n}\bar{z}_{1}}{|z|^{2}}&\ldots&\frac{z_{n}\bar{z}_{n}}{|z|^{2}}\\\
\end{pmatrix}$

where $|z|^{2}=\sum_{i=0}^{n}z_{i}\bar{z}_{i}$. This is clearly invariant
under

$z\to\lambda z,\lambda\in\mathbb{C}^{\star}\;.$

The entries in the spectral layer matrix corresponds to the degree-1 basis of
eigenfunctions of the Laplacian on $\mathbb{CP}^{4}$ introduced in [37]; this
is the rationale for the name of the layer. Comparing to the machine learning
literature, this is almost, up to the division by the norm, an instance of the
polynomial invariant layers studied by [31].

For CICY manifolds with product ambient spaces
$\mathbb{CP}^{n_{1}}\times\mathbb{CP}^{n_{2}}\times...\mathbb{CP}^{n_{k}}$ the
spectral layer is implemented in a block diagonal fashion,161616Note, that
these matrices are never explicitly constructed in our implementation.
Instead, the input is directly computed as a flattened vector, where the zero
entries are discarded, that can be fed into the neural network. with separate
maps for the homogeneous coordinates associated to each $\mathbb{CP}^{n_{j}}$.

As it is evident from the formulae above, our spectral layer uses complex
inputs. To match the real input of the cymetric package, we use a conversion
between the real inputs to complex and then back to real. Whenever we refer to
the spectral layer below, we refer to this composition.

Note that there are no trainable parameters in spectral layers. Their only
effect is to map the input data to a convenient, canonical form, that is
invariant under homogeneous rescalings. Since the composition of an invariant
map with any other map, results in a map with the same invariance, the
inclusion of a spectral layer results in an invariant machine learning model,
as illustrated in figure 1.

In the next section, we will explore other types of invariant layers, that
serve the purpose of making the network prediction $\phi$ invariant under
isometries of the CY manifold.

Figure 1: A dense neural network with an invariant layer. The invariant layer
does not have trainable parameters. It’s sole purpose is to put the input
data, $z_{i}$, in a form, $\alpha_{i}(z)$ that is invariant under the relevant
symmetry group.

##  4 Invariant metric by fundamental domain projections

In the following, we will discuss a method to construct a group invariant
neural network predicting the Ricci-flat metric on some CY manifold. This will
be done by projecting the input data, i.e. the tuple containing the
homogeneous coordinates of the sampled points on the CY space, onto the
fundamental domain of said group, following the strategy of [35, 36]. We
recall that the rationale of these methods is based on the fact that the
composition of a group invariant map with any other map, results in a map with
the same invariance. Thus, by including a non-trainable, $G$-canonicalization
layer, as introduced below, before the dense layers of the $\phi$-model, we
get a group invariant network. An illustration of this model architecture is
given in figure 1.

We consider a neural network $f:X\to\mathbb{R}$ for some CY manifold $X$.
Given some function $h:X\to X$, satisfying

$h\left(g\cdot x\right)=h(x)\,,\qquad\forall x\in X\,,\;\forall g\in G\,,$ (9)

for some group $G$ acting on $X$, one can construct a $G$-invariant neural
network by

$f\left(h\left(x\right)\right)=y\,,\qquad x\in X,\,y\in\mathbb{R}\,.$ (10)

This can be achieved by defining a map $\pi:X\to G$, obeying

$\pi(g\cdot x)=\pi(x)g^{-1}\,,\qquad\forall x\in X\,,\;\forall g\in G\,,$ (11)

such that

$h(x)=\pi(x)\cdot x\,.$ (12)

We call the map $h$ a _ $G$-canonicalization_ and it is easy to see that $h$
is a $G$-invariant map

$h(g\cdot x)=\pi(g\cdot x)\cdot(g\cdot x)=\pi(x)(g^{-1}g)\cdot x=\pi(x)\cdot
x=h(x)\,.$ (13)

The remaining task is to define an appropriate map that obeys eq. (11). We
will do so by defining maps to a fundamental domain of $G$.

######  Definition 4.1 (Fundamental domain).

Let $G$ be a group acting on a topological $G$-set $X$. Then a closed subset
$\mathcal{F}$ of $X$ is called a fundamental domain of $G$ in $X$ if $X$ is
the union of all conjugates of $\mathcal{F}$, i.e.,

$X=\bigcup_{g\in G}g\,\mathcal{F}\,,$ (14)

and the intersection of any two conjugates has no interior.

With this definition at hand, we define $h:X\to\mathcal{F}$, such that
$h(g\cdot x)=h(x)$ which defines $\pi$ with respect to a choice of
$\mathcal{F}$.

It is important to note that the projection into a fundamental domain can give
rise to discontinuities. For a given fundamental domain $\mathcal{F}$ of some
group $G$ acting on some manifold $X$, there exist points on $X$ which will
get mapped to the boundary of $\mathcal{F}$. Moving infinitesimally within $X$
can then result into discontinuous jumps in $\mathcal{F}$. This can lead to
problems computing global properties like the volume which require integration
over the entire manifold. We will talk about this further in section 5 and the
conclusion.

###  4.1 Examples

In the following examples we will consider symmetries on the Fermat quintic,
whose defining equation was given in eq. (3). Let $f\circ h:X\to\mathbb{R}$ be
a neural network learning the function $\phi(z)$ in the Ricci-flat metric
$g_{\textrm{CY}}(z)=g_{\textrm{FS}}(z)+\partial\overline{\partial}\phi(z)$.
Let $\mathbf{z}\in\mathbb{C}^{5}$ encode $z$ as a vector and let
$\rho:G\to\textrm{GL}(5,\mathbb{C})$ be a representation of some symmetry
group $G$. A $G$-invariant network is then constructed by defining
$h(\mathbf{z})=\rho(\pi(z))\mathbf{z}$ together with an appropriate choice of
$\pi(z)$. In the following, we will construct examples considering several
different symmetry groups $G$. In each example, we specify explicitly the
action of $G$ on the ambient space, the fundamental domain $\mathcal{F}$ and
the projection map $h$ on $\mathcal{F}$.

####  4.1.1 Homogeneous rescalings

Consider the set of homogeneous rescalings of the homogeneous coordinates of
the embedding space of the quintic, $\mathbb{CP}^{4}$:

$[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\sim\lambda[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\mbox{,
for }\lambda\in\mathbb{C}^{\star}\;.$

The group $G=\mathbb{C}^{\star}$ is a symmetry of the input data

$\left(\Re(z_{0})\quad\dots\quad\Re(z_{4})\quad\Im(z_{0})\quad\dots\quad\Im(z_{4})\right)\in\mathbb{R}^{10}$

of the $\phi$-model.

We define the fundamental domain

$\mathcal{F}=\Big{\\{}[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\in\mathbb{CP}^{4}\,\Big{|}\,\forall
i:\,|z_{i}|\leq 1,\exists i:z_{i}=1\Big{\\}}\,,$ (15)

and the projection map $h_{\rm hom}$ to be

$h_{\rm
hom}:\;[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\mapsto\left[\frac{z_{0}}{z_{i}}:\frac{z_{1}}{z_{i}}:\frac{z_{2}}{z_{i}}:\frac{z_{3}}{z_{i}}:\frac{z_{4}}{z_{i}}\right],\forall
j\neq i:\ |z_{i}|\geq|z_{j}|\,.$

In case more than one coordinate has the biggest norm, then we choose to
divide by the coordinate with smallest index.171717This is straightforwardly
implemented in ``TensorFlow— by finding the largest coordinate with
``tf.math.argmax(tf.abs(complex_input))— and then dividing the input by said
coordinate.

Eq. (15) thus uniquely determines the canonicalization maps $h$ and $\pi$
defined in the beginning of this section. The action transforms any input into
the canonical form of $\max(|z_{0}|,\dots,|z_{4}|)=1$. Note that the
`cymetric` input of the point generator is already of this canonical form,
hence the layer has no effect. However, upon training, points are re-scaled
within the transition loss to compare the prediction of $\phi$ in different
patches. This re-scaling is inverted by $h_{\rm hom}(\mathbf{z})$, leading to
an almost vanishing transition loss.

As an example of the action of this map, consider the point

$z=[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\mbox{, where }\forall j\neq
3:|z_{3}|>|z_{j}|\,.$

Then $\pi(z)=1/z_{3}$ and in terms $\mathbf{z}\in\mathbb{C}^{5}$ we have

$h_{\rm
hom}(\mathbf{z})=\rho(\pi(z))\cdot\mathbf{z}=\begin{pmatrix}1/z_{3}&0&0&0&0\\\
0&1/z_{3}&0&0&0\\\ 0&0&1/z_{3}&0&0\\\ 0&0&0&1/z_{3}&0\\\
0&0&0&0&1/z_{3}\end{pmatrix}\begin{pmatrix}z_{0}\\\ z_{1}\\\ z_{2}\\\ z_{3}\\\
z_{4}\end{pmatrix}=\begin{pmatrix}z_{0}/z_{3}\\\ z_{1}/z_{3}\\\ z_{2}/z_{3}\\\
1\\\ z_{4}/z_{3}\end{pmatrix}\,.$ (16)

In case of a product of complex projective spaces, we apply homogeneous
rescaling as described above for the coordinates of each factor, separately.

####  4.1.2 Permutations

Let $G=S_{5}$, which acts by permutation of variables. Clearly, the action of
this group allows to reorder the entries of a tuple $(z_{0},...z_{n})$ in any
way we desire, and we may select a fundamental domain by any consistent order.
Thus, given a set of points $[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]$, we define the
fundamental domain

$\mathcal{F}=\Big{\\{}[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\in\mathbb{CP}^{4}\,\Big{|}\,|z_{0}|\geq|z_{1}|\geq|z_{2}|\geq|z_{3}|\geq|z_{4}|\Big{\\}}\,.$
(17)

The projection map $h_{\rm perm}$ thus performs the required
sorting.181818Again, this is straightforwardly implemented in ``TensorFlow— by
obtaining the absolute values of the input with
``norms=tf.math.abs(complex_inputs)— and then sorting by ``tf.argsort(norms,
axis=-1, direction=’DESCENDING’)—.  By default, if the norms are equal,
$h_{\rm perm}$ keeps the original order.

As an example, consider the point $z=[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]$ where
$|z_{1}|>|z_{0}|>|z_{2}|>|z_{3}|>|z_{4}|$, then

$\pi(z)=\begin{pmatrix}0&1&2&3&4\\\ 1&0&2&3&4\end{pmatrix}\,,$ (18)

where we have used Cauchy’s two line notation for $\pi(z)\in S_{5}$. The first
line corresponds to indices $i$ and the second line to their permutations
$\sigma(i)$. In terms of $\mathbf{z}\in\mathbb{C}^{5}$, we have

$h_{\rm
perm}(\mathbf{z})=\rho(\pi(z))\cdot\mathbf{z}=\begin{pmatrix}0&1&0&0&0\\\
1&0&0&0&0\\\ 0&0&1&0&0\\\ 0&0&0&1&0\\\
0&0&0&0&1\end{pmatrix}\begin{pmatrix}z_{0}\\\ z_{1}\\\ z_{2}\\\ z_{3}\\\
z_{4}\end{pmatrix}=\begin{pmatrix}z_{1}\\\ z_{0}\\\ z_{2}\\\ z_{3}\\\
z_{4}\end{pmatrix}\,.$ (19)

In the case of product ambient spaces, there may be multiple permutation
invariances. Below, we will explore CY manifolds whose ambient space is
$(\mathbb{CP}^{n})^{m}$, with defining polynomials that are invariant under an
$S_{n}$ that acts identically on each factor. Concretely, the fundamental
domain of such a diagonal $S_{n}$ acting on
$\mathbb{CP}^{n}\times\mathbb{CP}^{n}$ is defined to be

$\mathcal{F}=\Big{\\{}([z_{0}:z_{1}:\ldots:z_{n}],[w_{0}:w_{1}:\ldots:w_{n}])\in\mathbb{CP}^{n}\times\mathbb{CP}^{n}\,\Big{|}\,|z_{0}|\geq|z_{1}|\geq\ldots\geq|z_{n}|\Big{\\}}\,$
(20)

where the restriction is only enforced on the coordinates of the first
component. To project on $\mathcal{F}$, we order the coordinates in the first
factor by norms and then use the same ordering to order the rest coordinates
of the remaining factors. For example, the point
$([z_{0}:z_{1}:\ldots:z_{n}],[w_{0}:w_{1}:\ldots:w_{n}])\in\mathbb{CP}^{n}\times\mathbb{CP}^{n}$,
such that $|z_{1}|\geq|z_{0}|\geq|z_{2}|\geq|z_{3}|\geq\ldots\geq|z_{n}|$ gets
mapped under the projection to
$([z_{1}:z_{0}:z_{2}:z_{3}:\ldots:z_{n}],[w_{1}:w_{0}:w_{2}:w_{3}:\ldots:w_{n}])$.
While we do not foresee any large complications, we leave the adaptation of
the fundamental domain projections to ambient spaces of the type
$\mathbb{CP}^{m}\times\mathbb{CP}^{n}$ with $m\neq n$, and similar
generalizations, to future work.

####  4.1.3 Scaling by roots of unity

Re($z_{0}$)Im($z_{0}$)

Figure 2: Under scaling by a fifth root of unity, $z_{0}$ maps into the
fundamental domain $\mathcal{F}$ (first quintant) by an appropriate rotation.

Let $G=(\mathbb{Z}_{5})^{5}$ act by re-scaling each coordinate by a fifth root
of unity, e.g. $z_{i}\to\exp(2\pi ik/5)z_{i}$ for some $k\in\\{0,\dots,4\\}$.
Note, that different coordinates can be re-scaled by different roots
simultaneously. We define the fundamental domain

$\mathcal{F}:=\\{[z_{0}:\ldots:z_{4}]\in\mathbb{CP}\ |\ \forall i:0\leq
arg(z_{i})\leq 2\pi/5\\}\,,$

and the projection map $h_{\rm rs}:X\to\mathcal{F}$ is given by191919We
implement this projection in ``TensorFlow— by finding the angles with
``angles=tf.math.angle(complex_input)— and then applying the modulo operation
with
``tf.math.floormod(angles, 2*np.pi/5.)—.

$h_{\rm
rs}:\;[r_{0}e^{i\theta_{0}}:\ldots:r_{4}e^{i\theta_{4}}]\to[r_{0}\exp[i(\theta_{0}\mod
2\pi/5)],\ldots,r_{4}\exp[i(\theta_{4}\mod 2\pi/5)]]\,.$

As an example, consider the point $z=[r_{0}e^{2\pi i\varphi_{0}}:r_{1}e^{2\pi
i\varphi_{1}}:r_{2}e^{2\pi i\varphi_{2}}:r_{3}e^{2\pi
i\varphi_{3}}:r_{4}e^{2\pi i\varphi_{4}}]$ where $\varphi_{i}<1/5\;\forall
i\neq 0$ and $\varphi_{0}=3/10$. We have for $\mathbf{z}\in\mathbb{C}^{5}$

$h_{\rm rs}(\mathbf{z})=\rho(\pi(z))\mathbf{z}=\begin{pmatrix}\exp(2\pi
i4/5)&0&0&0&0\\\ 0&1&0&0&0\\\ 0&0&1&0&0\\\ 0&0&0&1&0\\\
0&0&0&0&1\end{pmatrix}\begin{pmatrix}r_{0}e^{2\pi i3/10}\\\ r_{1}e^{2\pi
i\varphi_{1}}\\\ r_{2}e^{2\pi i\varphi_{2}}\\\ r_{3}e^{2\pi i\varphi_{3}}\\\
r_{4}e^{2\pi i\varphi_{4}}\end{pmatrix}=\begin{pmatrix}r_{0}e^{2\pi i1/10}\\\
r_{1}e^{2\pi i\varphi_{1}}\\\ r_{2}e^{2\pi i\varphi_{2}}\\\ r_{3}e^{2\pi
i\varphi_{3}}\\\ r_{4}e^{2\pi i\varphi_{4}}\end{pmatrix}\,.$ (21)

We illustrate the projection in figure 2.

###  4.2 Combining invariant layers

To achieve metric predictions that are invariant under a (semi)-direct product
of groups, as is the case for the Fermat quintic, we would like to construct
ML models that are invariant under several groups. This can be attained by
adding the $G$-canonicalization layers in sequence to the network. However, we
must then make sure that the canonicalizations satisfy a non-trivial
condition: Let $G=G_{1}\ltimes G_{2}$ act on a space $X$ and let $h_{i}$ be a
$G_{i}$-canonicalization for $i=1,2$. Then $h:=h_{2}\circ h_{1}$ defines a
$G$-canonicalization if and only if $h_{1}$ descends to a well defined map on
the orbits of $G$, namely $h_{1}:X/G_{2}\to X/G_{2}$ is well defined. Note
that this condition on $h_{1}$ is weaker than $h_{1}$ being equivariant with
respect to $G_{2}$.

To make this concrete, let’s return to the metric on the Fermat quintic $X$
defined by (3), which is invariant under action of the group
$G=S_{5}\ltimes\mathbb{Z}_{5}^{5}$. For later reference, we note that the
action of this group is not free. Let $h_{\rm perm}$ be the
$S_{5}$-canonicalization defined ordering according norm, and let $h_{\rm rs}$
be a $\mathbb{Z}_{5}^{5}$-canonicalization as defined in (4.1.3). Then $h_{\rm
rs}\circ h_{\rm perm}$ is a $G$-canonicalization, since for any
$r\in\mathbb{Z}_{5}^{5}$ and $z\in X$ we have

$\begin{split}h_{\rm perm}(r\cdot z)&=\pi_{\rm perm}(r\cdot z)\cdot(r\cdot
z)=\pi_{\rm perm}(z)\cdot(r\cdot z)\\\ &=\pi_{\rm perm}(z)r\pi_{\rm
perm}(z)^{-1}\pi_{\rm perm}(z)\cdot z=\underbrace{\pi_{\rm perm}(z)r\pi_{\rm
perm}(z)^{-1}}_{\in\mathbb{Z}_{5}^{5}}\cdot h_{\rm perm}(z).\end{split}$

Notice that $\pi_{\rm perm}(r\cdot z)=\pi_{\rm perm}(z)$ because $\pi_{\rm
perm}$ orders the entries in the $z$-tuple according to their norms and $r$
acting on $z$ only changes their arguments.

Performing a similar analysis, we find that $h_{\rm rs}\circ h_{\rm perm}\circ
h_{\rm hom}$ is a $[(\mathbb{C}^{\star}\times
S_{5})\ltimes\mathbb{Z}_{5}^{5}]$-canonicalization, where $h_{\rm hom}$ is
given by (4.1.1), and $\text{Spectral}\circ h_{\rm perm}$ is invariant under
$S_{5}\times\mathbb{C}^{\star}$.

##  5 Experiments

We have performed a number of experiments, on different CICY geometries, to
compare the accuracy and performance of invariant $\phi$-models, obtained
through the canonicalization procedure described in section 4, with the
standard and spectral $\phi$-models. In this section, we present the results
of these experiments. We use the pointgenerator classes of the cymetric
package to generate samples of 100 000 points on the CY spaces.202020For CY
hypersurfaces, we use PointGenerator, for CICYs with product ambient space we
use CICYPointGenerator. The data sets are split (1:9) into validation and
training data sets. The experiments consist of runs over five different point
sets, and we report the average and the standard deviation of these runs. We
test the models on an unseen test set of 50 000 points.

In more detail, we set up the standard, dense $\phi$-model with 3 hidden
layers with 64 nodes each. The network is fully connected, has GELU activation
function, Adam optimizer with 0.001 learning rate, and the weights of the
custom losses of the cymetric package take default values. We use the default
(glorot_uniform) initialization of TensorFlow for the trainable weights of the
neural networks. These hyperparameters were chosen after an initial
optimization study on the standard $\phi$-model, and were kept constant across
the invariant networks. The rationale for this choice was to isolate the
effects of the invariant layers from effects of hyperparameter tuning. We
leave a more extensive optimization study of the hyperparameters of invariant
architectures for the future.

###  5.1 Fermat quintic

Working on the Fermat quintic, we compare the standard $\phi$-model with
networks where one, or several, non-trainable layers are added. The added non-
trainable layers correspond to the spectral layer which, as described in
section 3.3, maps the data to a form which is invariant under homogeneous
rescalings. We also explore $G$-canonicalization layers that enforce
invariance under homogeneous rescalings, permutations and scaling by roots of
unity, as described in 4.1.1-4.1.3.

Figure 3:  Performance of different ML models on the Fermat quintic. From left
to right: Monge–Ampère, Ricci, and transition loss evolution on validation
data; volume computed with trained metric on all data (ground truth is 5). The
upper (lower) row shows dense (spectral) networks with and without
G-canonicalization layers. Solid line is average over 5 experiments, lighter
color bands are the standard deviation.

We have performed a number of experiments on this geometry, and report the
results from training the models on 90 000 points, over 50 epochs in figure 3
(we have checked that the conclusions are reproduced in 300-epochs
experiments). Since training and validation plots look very similar, we only
report the latter.212121In more detail: the MA and transition losses on
training data look very similar to the shown validation plots, showing that
there is no problem with overfitting. The Kclass loss evolution is consistent
with the Volume plot on validation data. Kähler and Ricci loss is disabled
during training. Thus, figure 3 shows the evolution, over the training epochs,
of Monge-Ampère, Ricci and transition loss on a validation set consisting of
10 000 points (the Kähler loss evaluates to $\mathcal{O}(10^{-16})$, and is
omitted from the plot). We also track the evolution of the volume, as computed
by the integral of the volume measure of the predicted metric (cf. appendix
B). Since the computation of the volume requires an integral, it is performed
over the full data set, i.e. 100 0000 points.

As is clear from the MA loss (leftmost in figure 3), the spectral layer and
canonicalization layers all have a positive impact on the loss. This is
clearly visible at initialization of the network. After training, the effect
of the invariant layers on the MA loss is still positive, albeit less
pronounced. This is in accord with previous observations in the literature
indicating that the standard $\phi$-model is fairly good at learning
symmetries.

When examining the Ricci loss, we notice a number of interesting features.
First, the spectral layer has a larger positive impact on performance, both at
initialization and during training. Second, while the canonicalization layers
may have a positive impact on the Ricci loss at initialization of standard
$\phi$-models, the effect becomes negative after 10 epochs of training. While
this discrepancy decreases with training, the standard $\phi$-model has a
slightly lower Ricci loss on validation than its canonicalized counterparts.
For spectral models, adding canonicalized layers has an even larger negative
effect on the Ricci loss. We interpret this as an effect of a complicated loss
landscape where optimizing with respect to the MA loss does not necessarily
improve the Ricci loss (despite the fact that both of these losses are zero
for the true Ricci-flat metric). Similar effects have been observed when
learning CY metrics on more complicated CICY manifolds [40], and may here be
related to the fact that the symmetry group has fixed points, that may affect
the computation of Ricci loss. As suggested in ref. [40], by making a
different hyperparameter choice, so that both MA and Ricci losses are used to
guide the training of the ML model, performance may be improved; we leave such
studies for the future.

The third subplot in figure 3 shows the transition loss: this is
$\mathcal{O}(10^{-3})$ for the dense network, and 6 orders of magnitude lower
for the spectral network, and any network that includes a homogeneous
canonicalization layer. This is in accordance with expectations, and indicates
that $\mathbb{C}^{*}$-invariance is achieved.

Finally, the volumes computed with the trained models all converge to the
ground truth upon training (within numerical errors). The spectral layer gives
a constant volume under training, which shows that this model indeed predicts
a globally defined $\phi$ at any time during training. For the other
architectures, there is more variance, so the same claim cannot be made. The
standard, dense $\phi$-model exhibits most variation, and imposing
$G$-canonicalization has a positive effect on the constancy of the volume.

In table 1, we list the $\sigma$ and $\mathcal{R}$ measures, which correspond
to the integrated MA and Ricci loss respectively (see appendix C), for the
fully trained models as well as the measures for the Fubini-Study metric as a
benchmark. We can see that adding the canonicalization layers has a positive
effect on the $\sigma$ measure. Concatenating the homogeneous, permutation and
rootscaling canonicalization layers improves the $\sigma$ measure by two
orders of magnitude compared to the Fubini-Study metric, and by a factor of 2
compared to the standard, dense $\phi$-model. Similar improvement is observed
for the spectral layer. For comparison, the $\sigma$ measure of the best
performing experiments in table 1 is $0.0027$, which would correspond to
$k\approx 33$ with Donaldson’s algorithm, and $k\approx 5$ with Headrick-
Nassar’s functional minimization.222222This comparison uses the observations
of $\sigma$ measures for $k\in[1,12]$, which shows that the $\sigma$ measure
decreases polynomially, like $\sigma\sim 3.1k^{-2}-4.2k^{-3}$, for Donaldson’s
balanced metric [7], and exponentially, like $\sigma\sim 0.17\times 2.2^{-k}$,
for Headrick-Nassar’s optimal metric [10].  On the $\mathcal{R}$ measure, the
spectral network performs the best. Any addition of canonicalization layers
worsens the accuracy of the prediction, in particular when combined with the
spectral layer. This is in accordance with the observed Ricci loss values on
validation data, as discussed above.

Model |  $\sigma$ measure |  $\mathcal{R}$ measure
---|---|---
FS metric | 0.3720 | 0.6919
Dense | 0.0106 $\pm$ 0.0011 | 0.0502 $\pm$ 0.0040
Hom. | 0.0100 $\pm$ 0.0006 | 0.0645 $\pm$ 0.0052
Hom. + Perm. | 0.0058 $\pm$ 0.0007 | 0.0754 $\pm$ 0.0066
Hom. + Perm. + Rootsc. | 0.0040 $\pm$ 0.0012 | 0.0891 $\pm$ 0.0084
Spectral | 0.0047 $\pm$ 0.0004 | 0.0149 $\pm$ 0.0011
Perm. + Spectral | 0.0040 $\pm$ 0.0004 | 0.0600 $\pm$ 0.0049
Hom. + Perm. + Rootsc. + Spectral | 0.0027 $\pm$ 0.0005 | 0.0956 $\pm$ 0.0091
Table 1: $\sigma$ and $\mathcal{R}$ measure for the Fermat quintic evaluated
over a test set of 50 000 points. The values are rounded to four decimals and
the error is given by the standard deviation of five experiments.

###  5.2 Tetraquadric

In this example, we study the tetraquadric defined by the configuration matrix

$\displaystyle\left[\begin{array}[]{c|c}1&2\\\ 1&2\\\ 1&2\\\
1&2\end{array}\right]$ (26)

This CY 3-fold has $h^{(1,1)}=4$ and $h^{(2,1)}=68$. We pick a smooth member
of this CICY family, defined by a degree (2,2,2,2)-polynomial $p$ of
$\mathcal{M}$ which is invariant under the action of $S_{2}$, as defined in
§4.1.2. Concretely, we set

$\displaystyle p(x_{0},x_{1},y_{0},y_{1},z_{0},z_{1},w_{0},w_{1}):=$
$\displaystyle
2x_{0}^{2}y_{0}^{2}z_{0}^{2}w_{0}^{2}-2x_{0}^{2}y_{0}^{2}z_{0}^{2}w_{0}w_{1}+2x_{0}x_{1}y_{0}^{2}z_{0}^{2}w_{1}^{2}+x_{1}^{2}y_{0}^{2}z_{0}^{2}w_{0}^{2}+2x_{1}^{2}y_{0}^{2}z_{0}^{2}w_{0}w_{1}$
$\displaystyle+x_{0}^{2}y_{0}y_{1}z_{0}^{2}w_{0}^{2}+x_{0}^{2}y_{0}y_{1}z_{0}^{2}w_{1}^{2}-2x_{0}x_{1}y_{0}y_{1}z_{0}^{2}w_{0}w_{1}-x_{0}x_{1}y_{0}y_{1}z_{0}^{2}w_{1}^{2}$
$\displaystyle+x_{1}^{2}y_{0}y_{1}z_{0}^{2}w_{0}^{2}-x_{1}^{2}y_{0}y_{1}z_{0}^{2}w_{1}^{2}-2x_{0}^{2}y_{1}^{2}z_{0}^{2}w_{0}^{2}-x_{0}^{2}y_{1}^{2}z_{0}^{2}w_{0}w_{1}-2x_{0}^{2}y_{1}^{2}z_{0}^{2}w_{1}^{2}$
$\displaystyle-2x_{0}x_{1}y_{1}^{2}z_{0}^{2}w_{0}^{2}-2x_{0}x_{1}y_{1}^{2}z_{0}^{2}w_{0}w_{1}+x_{1}^{2}y_{1}^{2}z_{0}^{2}w_{0}^{2}+2x_{1}^{2}y_{1}^{2}z_{0}^{2}w_{0}w_{1}+2x_{1}^{2}y_{1}^{2}z_{0}^{2}w_{1}^{2}$
$\displaystyle+x_{0}^{2}y_{0}^{2}z_{0}z_{1}w_{0}^{2}-2x_{0}^{2}y_{0}^{2}z_{0}z_{1}w_{1}^{2}-x_{0}x_{1}y_{0}^{2}z_{0}z_{1}w_{0}^{2}-2x_{0}x_{1}y_{0}^{2}z_{0}z_{1}w_{0}w_{1}$
$\displaystyle+x_{0}x_{1}y_{0}^{2}z_{0}z_{1}w_{1}^{2}-x_{1}^{2}y_{0}^{2}z_{0}z_{1}w_{0}^{2}+x_{1}^{2}y_{0}^{2}z_{0}z_{1}w_{0}w_{1}-2x_{1}^{2}y_{0}^{2}z_{0}z_{1}w_{1}^{2}$
$\displaystyle-
x_{0}^{2}y_{0}y_{1}z_{0}z_{1}w_{0}w_{1}+2x_{0}^{2}y_{0}y_{1}z_{0}z_{1}w_{1}^{2}-x_{0}x_{1}y_{0}y_{1}z_{0}z_{1}w_{0}^{2}-2x_{0}x_{1}y_{0}y_{1}z_{0}z_{1}w_{0}w_{1}$
$\displaystyle-
x_{0}x_{1}y_{0}y_{1}z_{0}z_{1}w_{1}^{2}+2x_{1}^{2}y_{0}y_{1}z_{0}z_{1}w_{0}^{2}-x_{1}^{2}y_{0}y_{1}z_{0}z_{1}w_{0}w_{1}-2x_{0}^{2}y_{1}^{2}z_{0}z_{1}w_{0}^{2}$
$\displaystyle+x_{0}^{2}y_{1}^{2}z_{0}z_{1}w_{0}w_{1}-x_{0}^{2}y_{1}^{2}z_{0}z_{1}w_{1}^{2}+x_{0}x_{1}y_{1}^{2}z_{0}z_{1}w_{0}^{2}-2x_{0}x_{1}y_{1}^{2}z_{0}z_{1}w_{0}w_{1}$
$\displaystyle-
x_{0}x_{1}y_{1}^{2}z_{0}z_{1}w_{1}^{2}-2x_{1}^{2}y_{1}^{2}z_{0}z_{1}w_{0}^{2}+x_{1}^{2}y_{1}^{2}z_{0}z_{1}w_{1}^{2}+2x_{0}^{2}y_{0}^{2}z_{1}^{2}w_{0}^{2}+2x_{0}^{2}y_{0}^{2}z_{1}^{2}w_{0}w_{1}$
$\displaystyle+x_{0}^{2}y_{0}^{2}z_{1}^{2}w_{1}^{2}-2x_{0}x_{1}y_{0}^{2}z_{1}^{2}w_{0}w_{1}-2x_{0}x_{1}y_{0}^{2}z_{1}^{2}w_{1}^{2}-2x_{1}^{2}y_{0}^{2}z_{1}^{2}w_{0}^{2}$
$\displaystyle-
x_{1}^{2}y_{0}^{2}z_{1}^{2}w_{0}w_{1}-2x_{1}^{2}y_{0}^{2}z_{1}^{2}w_{1}^{2}-x_{0}^{2}y_{0}y_{1}z_{1}^{2}w_{0}^{2}+x_{0}^{2}y_{0}y_{1}z_{1}^{2}w_{1}^{2}$
$\displaystyle-
x_{0}x_{1}y_{0}y_{1}z_{1}^{2}w_{0}^{2}-2x_{0}x_{1}y_{0}y_{1}z_{1}^{2}w_{0}w_{1}+x_{1}^{2}y_{0}y_{1}z_{1}^{2}w_{0}^{2}+x_{1}^{2}y_{0}y_{1}z_{1}^{2}w_{1}^{2}$
$\displaystyle+2x_{0}^{2}y_{1}^{2}z_{1}^{2}w_{0}w_{1}+x_{0}^{2}y_{1}^{2}z_{1}^{2}w_{1}^{2}+2x_{0}x_{1}y_{1}^{2}z_{1}^{2}w_{0}^{2}-2x_{1}^{2}y_{1}^{2}z_{1}^{2}w_{0}w_{1}+2x_{1}^{2}y_{1}^{2}z_{1}^{2}w_{1}^{2}\;.$

Note that this instance of the tetraquadric has no root scaling invariance. We
project to the fundamental domain associated to $S_{2}$ as defined in eq.
(20). For the homogenous rescalings, we consider each ambient space sperately
and project into the fundamental domain as in eq. (4.1.1). The spectral layer
is realized as describe in section 3.3 for a product of ambient spaces.

Figure 4: Performance of different ML models on a $S_{2}$ invariant
tetraquadric CY 3-fold. From left to right: Monge–Ampère, Ricci, and
transition loss evolution on validation data; volume computed with trained
metric on all data (ground truth is 48). The upper (lower) row shows dense
(spectral) networks with and without G-canonicalization layers.

We perform analogous experiments as in the previous section, and report the
validation results for five experiments trained over 90 000 points for 50
epochs in figure 4. As expected from the previous experiment, adding the
homogenous canonicalization layer leads to an almost vanishing transition loss
while it does not improve the performance on the Monge–Ampère and Ricci loss.
However, adding the permutation canonicalization layer to the architecture
improves performance both on Monge–Ampère and Ricci loss. The spectral layer
performs slightly better than the previous configuration, while the
combination of permutation canonicalization and spectral layer performs best.

Model |  $\sigma$ measure |  $\mathcal{R}$ measure
---|---|---
FS metric | 0.4393 | 7.0902
Dense | 0.1168 $\pm$ 0.0020 | 4.5680 $\pm$ 0.0823
Hom. | 0.1183 $\pm$ 0.0017 | 4.5906 $\pm$ 0.0481
Hom. + Perm. | 0.0960 $\pm$ 0.0018 | 4.1339 $\pm$ 0.0882
Spectral | 0.0665 $\pm$ 0.0002 | 2.9522 $\pm$ 0.0373
Perm. + Spectral | 0.0582 $\pm$ 0.0007 | 2.6980 $\pm$ 0.0326
Table 2: $\sigma$ and $\mathcal{R}$ measure for the tetraquadric evaluated
over a test set of 50 000 points. The values are rounded to four decimals and
the error is given by the standard deviation of five experiments.

For the volume, we make the following observations. The spectral layer is most
stable over all epochs and follows, within a numerical error, the ground truth
of $48$ while all other combinations show higher variance and seem to converge
to a slightly shifted value. As discussed in section 4, the variance could be
explained due to the fact that the fundamental domain projections introduce
discontinuities which might skew calculations, like the volume, which require
integration over the entire manifold. However, the shifted volume prediction
on the tetraquadric is not observed in the other experiments we have
performed. The explanation behind this difference is likely that $h^{(1,1)}>1$
for the tetraquadric, while it is 1 for the other CY manifolds, and that using
a loss function to fix the Kähler class during training is more challenging on
manifolds with higher $h^{(1,1)}$.

In table 2, we compute the $\sigma$ and $\mathcal{R}$ measures for the
different models. For the $\sigma$ measure, we observe a similar trend as for
the Fermat quintic. Namely, adding any combination of canonicalization
layers232323Adding only the homogeneous canicalization layer does not improve
the $\sigma$ measure. The main purpose of this layer is to reduce the
transition loss. has a positive effect on the measure. The spectral layer
shows the most significant improvement, while the combination of permutation
canonicalization layer and spectral layer perform the best. The $\mathcal{R}$
measure shows a different trend as for the Fermat quintic. Adding the
permutation canonicalization layer improves the measure slightly, while again
the spectral layer shows the most significant improvement. Again, the
combination of permutation canonicalization layer and spectral layer performs
the best.

###  5.3 A 2-polynomial CICY

Figure 5: Performance of different ML models on a $[5|3\ 3]$ CY 3-fold. From
left to right: Monge–Ampère, Ricci, and transition loss evolution on
validation data; volume computed with trained metric on all data (ground truth
is 9). The upper (lower) row shows dense (spectral) networks with and without
G-canonicalization layers.

We look at the CICY $[5|3\ 3]$, where the two defining polynomials are given
by (cf. [32])

$\begin{split}p(x_{0},x_{1},x_{2},x_{3},x_{4},x_{5})&:=x_{0}^{3}+x_{1}^{3}+x_{2}^{3}-3x_{3}x_{4}x_{5},\\\
q(x_{0},x_{1},x_{2},x_{3},x_{4},x_{5})&:=x_{3}^{3}+x_{4}^{3}+x_{5}^{3}-3x_{0}x_{1}x_{2}.\end{split}$

This CICY has a discrete group $\mathbb{Z}_{2}(S_{3}\times S_{3})$ of
permutation symmetries, where the non-trivial element in $\mathbb{Z}_{2}$ acts
by sending $[x_{0}:x_{1}:x_{2}:x_{3}:x_{4}:x_{5}]$ to
$[x_{3}:x_{4}:x_{5}:x_{0}:x_{1}:x_{2}]$. while the first $S_{3}$ factor
permutes the coordinates $(x_{0},x_{1},x_{2})$ and the second $S_{3}$ factor
permutes the coordinates $(x_{3},x_{4},x_{5})$. The group does not act freely.

We implement a canonicalization for the action of $\mathbb{Z}_{2}(S_{3}\times
S_{3})$ in two steps. In the first step, we reorder the coordinates in the
first triple and the second triple in descending order with respect to norm,
separately.

$\begin{split}&[x_{0}:x_{1}:x_{2}:x_{3}:x_{4}:x_{5}]\mapsto[x_{\sigma(0)}:x_{\sigma(1)}:x_{\sigma(2)}:x_{\tau(3)}:x_{\tau(4)}:x_{\tau(5)}]\\\
&\text{where}\ (\sigma,\tau)\in S_{3}\times S_{3},\
|x_{\sigma(0)}|\geq|x_{\sigma(1)}|\geq|x_{\sigma(2)}|,\
|x_{\tau(3)}|\geq|x_{\tau(4)}|\geq|x_{\tau(5)}|.\end{split}$

In the second step, we shift such that the triple with highest sum of norms is
first. We combine these two steps in one permutation canonicalization layer.

We perform experiments similar to the previous sections where we combine
homogeneous and permutation canonicalization layers, and add them to the dense
and spectral $\phi$-models. In figure 5, we see that the homogeneous
canonicalization layer leads to an almost vanishing transition loss when
combined with the dense network. Regarding volume computation, we can see that
adding the permutation canonicalization layer helps the dense network to
converge faster to the correct volume, but it worsens the computation for the
spectral network. With respect to the Monge–Ampère loss, the permutation
canonicalization layer improves the performance of both networks by a factor
of 2. In contrast to the previous experiments, the Ricci loss shows different
trends from the Monge–Ampère loss. Only the spectral network reaches a clear
plateau value after 10 epochs, while the other networks seems to oscillate.
The permutation canonicalized models has an initial rise followed by a steep
descent; since the model has not settled, further training might improve the
Ricci loss, and training against the Ricci loss, in addition to the MA loss,
might also improve performance, as discussed for the Fermat quintic
experiments.

In table 3, we compute the $\sigma$ and $\mathcal{R}$ measures for the
different models. For the $\sigma$ measure we observe almost identical
performance for all architectures. The $\mathcal{R}$ measure does not improve
for any architecture except the spectral model, which is in accordance with
the Ricci loss validation plots. The explanation is likely that the loss
landscape for this geometry is hard to navigate, and that models get stuck in
a local minimum. This is supported by the observation that even the standard,
dense $\phi$-model shows poorer performance on the $\mathcal{R}$ measure than
on the other geometries.

Model |  $\sigma$ measure |  $\mathcal{R}$ measure
---|---|---
FS metric | 0.2620 | 1.0907
Dense | 0.1971 $\pm$ 0.0015 | 0.9796 $\pm$ 0.1174
Hom. | 0.1974 $\pm$ 0.0020 | 1.0003 $\pm$ 0.0463
Hom. + Perm. | 0.1983 $\pm$ 0.0006 | 1.2831 $\pm$ 0.1380
Spectral | 0.1944 $\pm$ 0.0002 | 0.7203 $\pm$ 0.0291
Perm. + Spectral | 0.1981 $\pm$ 0.0007 | 2.2978 $\pm$ 2.5123
Table 3: $\sigma$ and $\mathcal{R}$ measure for the $[5|3\ 3]$ CY 3-fold
evaluated over a test set of 50 000 points. The values are rounded to four
decimals and the error is given by the standard deviation of five experiments.

##  6 Ricci-flat metrics on non simply connected CY manifolds

In heterotic string compactifications, phenomenologically interesting particle
physics models are often based on CY manifolds that are not simply connected.
Such manifolds admit Wilson lines, which allow us to break the heterotic
$E_{8}$ group to the gauge group of the Standard Model, and achieve the
correct number of particle generations; this idea is pedagogically explained
in [56], and dates back to [57, 58, 59]. There is a large literature on the
use of discrete symmetries in heterotic model building, and the reader is
referred to the papers cited above for more discussion on this topic. In this
section, we will follow [8] and study the Ricci-flat metric on a quotient $Q$
of a smooth, 5-parameter quintic $X$ by a freely acting $\mathbb{Z}_{5}^{2}$
group. As explained e.g. in [56, 8], $Q$ has Euler characteristic -8, and
leads to a particle physics model with 4 generations, so we will refer to this
geometry as a 4-generation quotient of the quintic.

In more detail, we will study the 5-parameter quintic CY specified, in
$\mathbb{CP}^{4}$, by the zero locus of

$\begin{split}p=&\,(z_{0}^{5}+z_{1}^{5}+z_{2}^{5}+z_{3}^{5}+z_{4}^{5})\\\
&+\,\psi_{1}z_{0}z_{1}z_{2}z_{3}z_{4}\\\
&+\,\psi_{2}(z_{0}^{3}z_{1}z_{4}+z_{0}z_{1}^{3}z_{2}+z_{0}z_{3}z_{4}^{3}+z_{1}z_{2}^{3}z_{3}+z_{2}z_{3}^{3}z_{4})\\\
&+\,\psi_{3}(z_{0}^{2}z_{1}z_{2}^{2}+z_{1}^{2}z_{2}z_{3}^{2}+z_{2}^{2}z_{3}z_{4}^{2}+z_{3}^{2}z_{4}z_{0}^{2}+z_{4}^{2}z_{0}z_{1}^{2})\\\
&+\,\psi_{4}(z_{0}^{2}z_{1}^{2}z_{2}+z_{1}^{2}z_{2}^{2}z_{3}+z_{2}^{2}z_{3}^{2}z_{4}+z_{3}^{2}z_{4}^{2}z_{0}+z_{4}^{2}z_{0}^{2}z_{1})\\\
&+\,\psi_{5}(z_{0}^{3}z_{2}z_{3}+z_{1}^{3}z_{3}z_{4}+z_{2}^{3}z_{4}z_{0}+z_{3}^{3}z_{0}z_{1}+z_{4}^{3}z_{1}z_{2})\,.\end{split}$
(27)

We pick a generic instance of this family of CY manifolds by drawing
$\psi_{i}$ from $\mathcal{N}(0,1)$.

As is shown in ref. [8], this 5-parameter quintic $X$ is invariant under a
group $G=\mathbb{Z}_{5}^{2}$ generated by

$g_{1}=\begin{pmatrix}0&0&0&0&1\\\ 1&0&0&0&0\\\ 0&1&0&0&0\\\ 0&0&1&0&0\\\
0&0&0&1&0\\\ \end{pmatrix}\;,\;g_{2}=\begin{pmatrix}1&0&0&0&0\\\
0&e^{\frac{2\pi i}{5}}&0&0&0\\\ 0&0&e^{\frac{4\pi i}{5}}&0&0\\\
0&0&0&e^{\frac{6\pi i}{5}}&0\\\ 0&0&0&0&e^{\frac{8\pi i}{5}}\\\ \end{pmatrix}$
(28)

Moreover, the group $G$ is shown to act freely on $X$, so that the quotient
$Q=X/G$ is smooth.

In ref. [8], Braun and collaborators obtain an approximation for the Ricci-
flat CY metric on $Q$ using the Donaldson algorithm. As was mentioned in the
introduction of this paper, this algorithm computes the CY metric via an
expansion in a spectral basis, consisting of degree-$k$ homogeneous
polynomials, using a fixed point iteration scheme to determine the
coefficients of the expansion. The accuracy of metric improves as
$k\to\infty$. The discrete symmetry is imposed as a restriction on the
spectral basis, and results in a slower growth of the basis dimension. The
invariant spectral basis can thus be constructed for larger $k$ than the non-
invariant one; consequently more accurate approximations for the metric can be
achieved.

However, it is somewhat non-trivial to determine the basis of sections on $Q$,
i.e. identify the linearly indepentdent $G$-invariant sections on $X$ that
restrict to $Q$, already for moderate values of $k$. There are two subtleties:
first, polynomials of the form $q_{i}p$, where $p=0$ defines the CY
hypersurface, evaluate to zero on the CY, and must be removed from the
invariant basis. Second, the generators $g_{1},g_{2}$ only commute up to
homogeneous rescaling of the coordinates, which makes the construction of
invariant polynomials non-trivial. Braun and collaborators rely on algorithms
of invariant theory, implemented using SINGULAR [60], to identify the basis,
and explore these methods up to $k=25$. After running the Donaldson algorithm
on a a sample with $10^{6}$ points, they find that the $\sigma$ measure
evaluates to $0.015$ for $k=25$ on an unseen test set of 20 000 points.242424
When comparing the performance against number of basis elements, they find
that the $G$-invariant metric approximation has smaller $\sigma$ error than a
metric computed using the unrestricted spectral basis.

In this section, we revisit the computation of the Ricci-flat CY metric on $Q$
using the methods introduced in section 4. We thus encode the $G$-invariance
as non-trainable canonicalization layers in the $\phi$-model of cymetric , and
compare the performance of these invariant ML models with the standard dense
counterpart, as well as spectral networks. Notice that the predicted metric
using the above canonicalization layers is on the covering space $X$, rather
than being on the quotient $Q$: the metric is invariant under the action of
$G$, and so constant along the fibers, but it is expressed in coordinate
patches on $X$. Subsequently, in order to obtain the metric on the quotient,
one needs to pushforward the learned metric via the projection from $X$ to
$Q$. In this work, we do not execute this last step.

Figure 6: Performance of different ML models on a 4-generation quintic. From
left to right: Monge–Ampère, Ricci, and transition loss evolution on
validation data; volume computed with trained metric on all data (ground truth
is 5). The upper (lower) row shows dense (spectral) networks with and without
G-canonicalization layers.

We implement the canonicalization layers as follows. Let $G_{i}\subset G$ be
the subgroup generated by the generator $g_{i}$ for $i=1,2$. Then, by
projecting on the fundamental domains of $G_{1}$ and $G_{2}$ consecutively, we
construct an invariant function on $X$ w.r.t $G$ or, in other words, a
function on the smooth quotient $Q=X/G$. A technical complication, compared to
the Fermat quintic canonicalization layers of Sec. 5.1 is that we need to
address the non-commutativity of $G_{1}$ and $G_{2}$. This can be resolved by
inserting the homogeneous canonicalization layer before the other
$G$-canonicalization layers; $g_{1}$ and $g_{2}$ commute up to homogeneous
rescalings.

The fundamental domains of $G_{1}$ and $G_{2}$ are given by

$\displaystyle\mathcal{F}_{1}$
$\displaystyle:=\\{[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\in\mathbb{CP}^{4}|\ \forall
i:|z_{0}|\geq|z_{i}|\\},$ (29) $\displaystyle\mathcal{F}_{2}$
$\displaystyle:=\\{[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]\in\mathbb{CP}^{4}|\
0\leq\arg(z_{0})\leq 2\pi/5\\}.$ (30)

To project on the fundamental domains $\mathcal{F}_{1}$ and $\mathcal{F}_{2}$,
we implement two canonicalization layers that act onn the homogeneous
coordinates $[z_{0}:z_{1}:z_{2}:z_{3}:z_{4}]$ of a point on $X$. The first
layer returns the coordinate tuple shifted to the left such that the highest
norm entry takes the first spot,

$\displaystyle h_{\rm shift}:\,(z_{0},z_{1},z_{2},z_{3},z_{4})\mapsto$
$\displaystyle(z_{i},z_{(i+1\mod 5)},z_{(i+2\mod 5)},z_{(i+3\mod
5)},z_{(i+4\mod 5)}),$ $\displaystyle\ \forall j\ |z_{i}|\geq|z_{j}|.$

For the second canonicalization layer, we find the fifth root of unity $a$
such that $0\leq\arg(az_{0})\leq 2\pi/5$ and then multiply the rest of the
coordinates with successive powers of $a$ as follows,

$h_{\rm
FreeRootsc.}:\,(z_{0},z_{1},z_{2},z_{3},z_{4})\mapsto(az_{0},a^{2}z_{1},a^{3}z_{2},a^{4}z_{3},z_{4}).$

Our experiments run over 50 epochs with the same settings as in section 5. The
results on validation data are shown in figure 6 (we again omit the training
plots which are more or less identical to the validation plots). Just as
observed in other experiments, including a homogeneous canonicalization layer,
or a spectral layer, significantly reduces the transition loss. The validation
plots of Monge-Ampère and Ricci loss shows that canonicalization with respect
to $G_{1},G_{2}$ reduces the losses, albeit not by a large factor.252525The
spikes in the Ricci loss in the second plot of the top row in figure 6 are
transient numerical fluctuations. The reduction of the loss is more pronounced
when $G$-canonicalization is added to the dense model, than when similar
additions are made in the spectral network. The addition of canonicalization
layers also has a small positive effect on the volume for the dense network.
For the spectral network, the effect on the volume prediction is instead
negative at initialization. While this effect is subdued by training, it is an
indication that the canonicalization layers introduce discontinuities, which
affect global predictions of the model in a negative way.

In table 4, we compute the $\sigma$ and $\mathcal{R}$ measure for all the
models. We can observe an improvement on the $\sigma$ measure by adding any
combination of canonicalization layers to the standard, dense
$\phi$-model.262626Again, except for the case in which only the homogeneous
canonicalization layer is added, which performs on par with the dense network.
The spectral network performs better than any model based on the standard
model, and combining the spectral layer with canonicalization layers gives the
lowest $\sigma$ measure. For comparison, the error measure values obtained by
the fully canonicalized dense (spectral) networks correspond to the value
obtained with $k$ about $15$ ($20$) using Donaldson’s algorithm [8]. For the
$\mathcal{R}$ measure, we observe the same trend as for the $\sigma$ measure.

Summing up the results of this experiment, we note that the canonicalization
approach is easily implemented on this quintic, and that the layers combine
nicely with both the dense and spectral $\phi$-models. Combining this with the
pushforward, via the projection from $X$ to $Q$, we can thus learn the metric
on quotient CY manifolds using these machine learning techniques. Furthermore,
the accuracy of the prediction, as measured by the $\sigma$ and $\mathcal{R}$
measures, improves, and compares well to the results of ref. [8]. We leave a
more rigorous study of the performance gains that can be attained with these
models, compared to previous work, for future hyperparameter optimization
studies. What we can conclude already now is that, since there is no need to
construct the $G$-invariant spectral basis, the $G$-canonicalization layers
seem to provide a technical simplification compared to spectral methods used
in ref. [8]

Model |  $\sigma$ measure |  $\mathcal{R}$ measure
---|---|---
FS metric | 0.4006 | 0.7876
Dense | 0.0802 $\pm$ 0.0007 | 0.3070 $\pm$ 0.0048
Hom. | 0.0818 $\pm$ 0.0011 | 0.3115 $\pm$ 0.0053
Hom. + Shift | 0.0512 $\pm$ 0.0009 | 0.2374 $\pm$ 0.0039
Hom. + Shift + FreeRootsc. | 0.0523 $\pm$ 0.0012 | 0.2396 $\pm$ 0.0062
Spectral | 0.0419 $\pm$ 0.0004 | 0.1571 $\pm$ 0.0026
Shift + Spectral | 0.0404 $\pm$ 0.0014 | 0.1570 $\pm$ 0.0032
Hom. + Shift + FreeRootsc. + Spectral | 0.0399 $\pm$ 0.0011 | 0.1549 $\pm$ 0.0017
Table 4: $\sigma$ and $\mathcal{R}$ measure for the 4-generation quotient of
the quintic evaluated over a test set of 50 000 points. The values are rounded
to four decimals and the error is given by the standard deviation of five
experiments.

##  7 Conclusion

In this paper, we have addressed the question: how do we encode symmetries in
a machine learned Ricci-flat CY metric? From a geometric point of view, CY
manifolds may have discrete symmetries, with interesting consequences in
mathematics and physics. From a machine learning perspective, these symmetries
are manifest as invariances, under group transformations, of the input data of
machine learning models designed to predict CY metrics. By constructing
invariant ML models, one expects better performance and accuracy. A clear
application of these invariant models are in string phenomenology, for example
the computation of heterotic Yukawa couplings, which was recently explored
using machine learning tools in [32, 33].

Our work is inspired by recent developments in the ML literature, that
advocate to achieve model invariance by a pre-processing step, where the input
data is projected into a fundamental domain of the relevant group [35, 36]. We
have used an approach where non-trainable layers performs these projections;
collectively, we refer to these layers as canonicalization layers. This is a
modular implementation, and the layers can easily be exchanged to match
different setting. While we have limited our studies to the cymetric package
[21, 22], we expect similar canonicalization layers can be added with ease in
other ML packages for CY geometry.

We have performed a number of experiments on different CICY manifolds,
comparing the performance of the standard $\phi$-model of the cymetric package
with and without canonicalization layers, but otherwise identical
hyperparameter settings. We have also compared with the spectral network,
originally proposed by Berglund and collaborators [24], which may also be
combined with canonicalization layers. We find that adding canonicalization
layers increases performance on the losses that are used to train the
networks. In particular, the increase in performance on the transition loss is
several orders of magnitudes for all architectures. The volume, controlled by
the K-class loss, converges faster if canonicalization layers are added to the
standard $\phi$-model. However, the volume seems to be captured best by the
spectral networks without canonicalization. Being a global property, this
could be explained by discontinuities, related to the boundaries of
fundamental domains, that the canonicalization layers may introduce. These are
not present in the spectral layer.

The improvement on the Monge–Ampère loss is noticeable but not very large.
Combining the spectral layer with the canonicalization layers seems to
additionally increase performance while it introduces variance in the global
prediction of the volume. However, adding these custom layers can still be of
use regarding performance, as in many cases the losses converge faster,
allowing one to reduce the number of training epochs. In addition, it has not
yet been tested how the canonicalized layers perform in a systematic
hyperparameter optimization study.

The Ricci loss yields different results depending on which geometry we look
at. For the tetraquartic, as well as the 4-generation quotient of the quintic,
we are quotienting by a freely acting symmetry group and we observe that
canonicalization improves the Ricci loss on validation data, under training
governed by the MA loss. The $\mathcal{R}$ error measures also decrease for
canonicalized networks, compared to the standard dense and spectral
$\phi$-models. For the Fermat quintic and the $[5|3\ 3]$ CY 3-fold, adding the
canonicalization layers instead increases the Ricci loss on validation data,
and leads to a higher Ricci measure on test data. This could be explained by a
more complicated loss landscape on these geometries, where training against
the MA loss does not directly decrease the Ricci loss. The fact that the
canonicalization layers encode a symmetry group that does not act freely on
these geometries may contribute to this complicated loss landscape.

Furthermore, we would like to emphasize the advantage of the canonicalization
layers following the philosophy of the Geometric Deep Learning program. The
unrestricted models, consisting solely of dense layers, seem to already learn
the symmetry of the data. This is in accord with previous observations in the
literature, see e.g. [27], showing that the performance of the same dense
network improves slightly on symmetric CY manifolds, compared to generic ones.
However, adding the canonicalization layers creates an architecture that knows
about the symmetries a priori, allowing for better interpretability of the
model.

Moreover, using the canonicalization approach, we have taken steps towards
learning metrics of non-simply connected CY manifolds. Such manifolds, $Q$,
always have a simply connected CY manifold $X$ as a covering space; the
relation between the two spaces is $Q=X/\Gamma$, where $\Gamma$ is a freely
acting discrete symmetry group. The canonicalization idea provides a simple
way of imposing $\Gamma$-invariance on the learned Ricci-flat metric of $X$,
and thus, by pushforward, we can obtain the Ricci-flat metric of $Q$. While
the approximation of CY quotient metrics is already well-studied, using other
techniques [8, 10], the simplicity of the canonicalization layers, and the
fact that they can be combined with relative ease, makes for a practical
method, that can easily be adapted to different CY spaces and groups.

Looking ahead, there are many other directions yet to be explored. We have
already mentioned that a thorough hyperparameter study might improve the
performance of canonicalized architectures. Once this is done, it would be
interesting to establish if scaling laws, as first observed in ref. [61], hold
for these ML models, and use these laws to compare the efficiency of different
architectures. One could also explore the alternative invariant architectures
suggested in ref. [31], i.e. construct invariant layers using polynomial
invariants. While we have performed some investigations using this approach,
they indicated that such layers quickly become too wide to be practically
useful, even for the discrete symmetries prevalent among CY manifolds. In
addition, this approach lacks the modular aspect of the canonicalization
layers. A related avenue is to construct invariant models based on the
invariant spectral basis to higher orders in $k$. This is similar to what has
been attempted for algebraic CY metrics in the past, e.g. [7, 10, 8], so one
would expect performance gains, even if we expect the construction of such
invariant layers face similar computational subtleties as reported in [8].

Finally, there are many interesting phenomenological applications of
$G$-invariant ML models, such as the numerical computation of Hermitian Yang
Mills connections [62] and Yukawa couplings [32, 33]. We expect that the
canonicalization approach will be particularly relevant for heterotic
compactifications with non-standard embedding of the gauge group; as shown in
the examples explored in [33], several neural nets must be trained to learn
the Ricci-flat CY metric, Hermitian-Yang-Mills connection and harmonic
representatives of certain vector bundle cohomologies. The $G$-invariance-
imposing canonicalization layers can be applied to all of these networks; any
gain in the computation cost achieved by the canonicalization layer will be
multiplied by the number of networks to which it is applied. We expect this to
be of importance for phenomenological studies.

## Acknowledgement

This work was in part supported by the Wallenberg AI, Autonomous Systems and
Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
Support was also obtained from Vetenskapsrådet, grant no. 2020-03230, and from
Uppsala University’s AI4Research center. The computations were partly enabled
by resources provided by the National Academic Infrastructure for
Supercomputing in Sweden (NAISS), partially funded by Vetenskapsrådet through
grant agreement no. 2022-06725. ML thanks the Erwin Schrödinger Institute in
Vienna, and the organisers of the program The Landscape vs. the Swampland, for
hospitality during the completion of this work. MW thanks Nordita for
hospitality during the completion of this work.

We thank Lara Anderson, Georgios Dimitroglou Rizell, Cristofero Fraser-
Taliente, James Halverson, Thomas Hardy, Carlos Rodriguez, Fabian Ruehle,
Robin Schneider, and Ayca Özcelikkale for valuable discussions and
suggestions. We also thank Carlos Rodriguez and Robin Schneider for comments
on a draft of this paper; Fabian Ruehle for his continuous work in maintaining
the cymetric package; Muyang Liu and Martin Svärdsjö for collaborations on
related topics; and, finally, Daniel Persson and Jan Gerken for organizing the
Mathy-AI Journal Club, as well as its attendants, for providing a lot of
inspiration for this work.

##  Appendix A CY geometry

In this section, we prove that symmetries of the defining polynomial of a CICY
manifold are isometries of the Ricci-flat metric. This should be well-known,
but we have not found the proof in the standard references on the subject,
which motivates including this section.

Note that the notation in this section differs from other sections, where $J$
is used to define the Kähler form. Here, we will follow the prevalent notation
of the mathematical literature and use $\omega$ for the Kähler form, and $J$
for a complex structure.

We start with some preliminary definitions from §8.4 in [63] . Let $X$ be a
complex manifold with a complex structure $J$. Then a hermitian metric $g$ on
$X$ is a Riemann metric that preserves the complex structure i.e
$g(J\cdot,J\cdot)=g(\cdot,\cdot)$. Given a hermitian metric $g$ on $X$, we can
define a 2-form $\omega(\cdot,\cdot):=g(J\cdot,\cdot)$ on $X$ which is called
the _Kähler_ form, and $X$ is called _Kähler_ if its Kähler form is closed,
i.e. $d\omega=0$. Hence a Kähler manifold is equipped with three structure
compatible structures $J,g$ and $\omega$. A fact from Kähler geometry tells us
that given any two of the three structures, the compatibility conditions
between them completely determine the third one. This immediately implies that
if a given map between Kähler manifolds preserve any two of the structures
then it preserves the third one. Finally, a map $f:X\to X$ is holomorphic if
and only if it satisfies the Riemann-Cauchy equation given by

$J\circ df=df\circ J.$

######  Definition A.1.

A Calabi-Yau manifold $X$ is a compact Kähler manifold with a vanishing first
Chern class, or equivalently with a nowhere-vanishing holomorphic top form.

Let $X$ be a Calabi-Yau manifold embedded in a product of complex projective
spaces. The Fubini-Study form restricts to a Kähler form $\omega_{FS}$ on $X$.
The Calabi-Yau conjecture, or more precisely theorem, states that there exists
a unique smooth function $\phi:X\to\mathbb{R}$, up to addition of a constant,
such that the hermitian metric $g_{CY}$ associated with the form
$\omega_{CY}:=\omega_{FS}+i\partial\bar{\partial}\phi$ is Ricci-flat. In the
following propositions, we fix $X$ to be a Calabi-Yau manifold embedded in a
product of projective spaces.

######  Proposition 1.

Let $f:X\to X$ be a biholomorphic isometry of $g_{FS}$ then $f$ is an isometry
of the Ricci-flat metric $g_{CY}$ on $X$ i.e $f^{*}g_{CY}=g_{CY}$.

###### Proof.

The map $f$ preserves the Fubini-Study form $\omega_{FS}$ on $X$. Using this
and the fact that the pullback via a holomorphic map commutes with the
Dolbeault operators, we get

$f^{*}\omega_{CY}=f^{*}\omega_{FS}+if^{*}\partial\bar{\partial}\phi=\omega_{FS}+i\partial\bar{\partial}(\phi\circ
f).$

Notice that $f^{*}g_{CY}$ is the associated hermitian metric with
$f^{*}\omega_{CY}$, and

$Ric(f^{*}g_{CY})=f^{*}Ric(g_{CY})=0\,.$

Therefore, by the uniqueness part of the Calabi-Yau theorem, we have
$f^{*}\omega_{CY}=\omega_{CY}$ and thus $f^{*}g_{CY}=g_{CY}$. ∎

######  Proposition 2.

Let $\phi:X\to\mathbb{R}$ be a smooth function that satisfies the equation
$\omega_{CY}=\omega_{FS}+i\partial\bar{\partial}\phi$. Assume that
$\mathcal{S}$ is a finite group of biholomorphic symmetries of $\omega_{FS}$.
Then $\phi$ is invariant under the action $\mathcal{S}$, i.e. for all
$f\in\mathcal{S}$, we have $f^{*}\phi=\phi\circ f=\phi$.

###### Proof.

By Proposition 1, we have $\mathcal{S}$ is a group of isometries of
$\omega_{CY}$. Consequently, for all $f\in\mathcal{S}$, we have

$\omega_{FS}+i\partial\bar{\partial}\phi=\omega_{CY}=\omega_{FS}+i\partial\bar{\partial}f^{*}\phi.$

Hence, by the Calabi-Yau theorem we have $f^{*}\phi$ and $\phi$ differs by a
constant, which we denote by $c_{f}$. Note that $c_{f^{-1}}=-c_{f}$, and
therefore

$\frac{1}{|\mathcal{S}|}\sum_{f\in\mathcal{S}}f^{*}\phi=\frac{1}{|\mathcal{S}|}\sum_{f\in\mathcal{S}}\phi+c_{f}=\phi+\frac{1}{|\mathcal{S}|}\sum_{f\in\mathcal{S}}c_{f}=\phi.$

For an arbitrary element $h\in\mathcal{S}$, then

$h^{*}\phi=\frac{1}{|\mathcal{S}|}\sum_{f\in\mathcal{S}}h^{*}(f^{*}\phi)=\frac{1}{|\mathcal{S}|}\sum_{f\in\mathcal{S}}(f\circ
h)^{*}\phi=\frac{1}{|\mathcal{S}|}\sum_{\tilde{f}\in
h\cdot\mathcal{S}}\tilde{f}^{*}\phi=\phi,$

where the last equality is due to $\mathcal{S}\simeq h\cdot\mathcal{S}$.

∎

##  Appendix B Computing volume predictions

For each of the experiments, we check the global consistency of the prediction
by computing the volume. This requires numeric integration, which can be
performed using Monte-Carlo methods. We follow the procedure of the cymetric
package, which is described in [22]. Thus, over a sample of points, the
integral of a function $f$ is computed as the weighted sum

$\displaystyle\int_{X}\text{d}\text{Vol}_{\text{CY}}\;f=\int_{X}dA\,\frac{\text{d}\text{Vol}_{\text{CY}}}{dA}\;f\simeq\frac{1}{N}\sum_{i=1}^{N}w_{i}f(p_{i})\;,$
(31)

where $dA$ is the Fubini–Study measure, and the weights of each point are
defined as

$\displaystyle
w_{i}=\left.\frac{\text{d}\text{Vol}_{\text{CY}}}{dA}\right|_{p_{i}}=\left.\frac{\Omega\wedge\overline{\Omega}}{dA}\right|_{p_{i}}\;.$
(32)

These weights are computed by the point generator routines of the cymetric
package.

In order to check the global accuracy of the learned metric, we will instead
compute

$\displaystyle\int_{X}\text{d}\text{Vol}_{\text{CY}}\;f\simeq\frac{1}{N}\sum_{i=1}^{N}\tilde{w}_{i}\,{\rm
det}(g_{\text{CY}}(p_{i}))\,f(p_{i})\;,$ (33)

using the auxiliary weights $\tilde{w}_{i}$, defined as
$\tilde{w}_{i}=w_{i}/(\Omega\wedge\overline{\Omega})|_{p_{i}}$.

##  Appendix C Computing error measures

For each experiment, we check the accuracy of the prediction by computing the
$\sigma$ and Ricci measure which are given by

$\displaystyle\sigma$
$\displaystyle=\frac{1}{\text{Vol}_{\text{CY}}}\int_{X}\left\lVert
1-\frac{1}{\kappa}\frac{J\wedge J\wedge
J}{\Omega\wedge\overline{\Omega}}\right\rVert_{1}\,,$ (34)

where $\text{Vol}_{\text{CY}}=\int_{X}\mathrm{d}\text{Vol}_{\text{CY}}$ and
$\kappa=(\int_{X}J\wedge J\wedge J)/\text{Vol}_{\text{CY}}$, as well as

$\mathcal{R}=\frac{(\int_{X}J\wedge J\wedge
J)^{1/3}}{\text{Vol}_{\text{CY}}}\int_{X}\left\lVert\partial\bar{\partial}\ln{\rm
det}(g_{\text{CY}})\right\rVert_{1}\,.$ (35)

Following the default settings of the cymetric package, the integrals are
numerically evaluated as in Eq. (31), except for the integral over the Ricci-
scalar which is evaluated as in Eq. (33).

##  Appendix D Computational performance

Some of the experiments were run on an Intel\tinyR⃝ CoreTM i7-10700K CPU with
8 cores. 64 GB of RAM were available during all experiments. Other experiments
were run on clusters provided by NAISS, using either different CPU
configurations or a GPU.

The homogeneous layer combined with the permutation and root scaling layer
trained on 90 000 points for $50$ epochs used a maximum of 11 GB of RAM and
took approximately $45$ minutes to train. This RAM usage is higher than for
the dense layer, and running experiments on a standard laptop with 16 GB of
RAM would often crash due to lack of memory. For comparison, the runtimes
reported for the dense cymetric models in Refs. [21, 22, 27] were about one
hour on a laptop.

## References

  * [1] S.-T. Yau “On the ricci curvature of a compact kähler manifold and the complex monge-ampére equation, I,” Commun. Pure Appl. Math. 31 (1978) no. 3, 339–411.
  * [2] S. Kachru, A. Tripathy, and M. Zimet “K3 metrics from little string theory,” [1810.10540].
  * [3] S. Kachru, A. Tripathy, and M. Zimet “K3 metrics,” [2006.02435].
  * [4] G. Tian “On a set of polarized Kähler metrics on algebraic manifolds,” Journal of Differential Geometry 32 (1990) no. 1, 99 – 130.
  * [5] S. K. Donaldson “Some numerical results in complex differential geometry,” 2005\.
  * [6] M. R. Douglas, R. L. Karp, S. Lukic, and R. Reinbacher “Numerical solution to the hermitian Yang-Mills equation on the Fermat quintic,” JHEP 12 (2007) 083 [hep-th/0606261].
  * [7] M. R. Douglas, R. L. Karp, S. Lukic, and R. Reinbacher “Numerical Calabi-Yau metrics,”
J. Math. Phys. 49 (2008) 032302 [hep-th/0612075].

  * [8] V. Braun, T. Brelidze, M. R. Douglas, and B. A. Ovrut “Calabi-Yau Metrics for Quotients and Complete Intersections,” JHEP 05 (2008) 080 [0712.3563].
  * [9] M. Headrick and T. Wiseman “Numerical Ricci-flat metrics on K3,” Class. Quant. Grav. 22 (2005) 4931–4960 [hep-th/0506129].
  * [10] M. Headrick and A. Nassar “Energy functionals for Calabi-Yau metrics,” Adv. Theor. Math. Phys. 17 (2013) no. 5, 867–902 [0908.2635].
  * [11] W. Cui and J. Gray “Numerical Metrics, Curvature Expansions and Calabi-Yau Manifolds,” JHEP 05 (2020) 044 [1912.11068].
  * [12] G. Cybenko “Approximations by superpositions of sigmoidal functions,” Mathematics of Control, Signals, and Systems 2 (1989) 303.
  * [13] M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function,” Neural Networks 6 (1993) no. 6, 861 – 867.
  * [14] V. Maiorov “Approximation by neural networks and learning theory,” J. Complexity 22 (02, 2006) 102–117.
  * [15] A. Ashmore, Y.-H. He, and B. A. Ovrut “Machine Learning Calabi–Yau Metrics,” Fortsch. Phys. 68 (2020) no. 9, 2000068 [1910.08605].
  * [16] L. B. Anderson, M. Gerdes, J. Gray, S. Krippendorf, N. Raghuram, and F. Ruehle “Moduli-dependent Calabi-Yau and SU(3)-structure metrics from Machine Learning,” JHEP 05 (2021) 013 [2012.04656].
  * [17] V. Jejjala, D. K. Mayorga Pena, and C. Mishra “Neural Network Approximations for Calabi-Yau Metrics,” [2012.15821].
  * [18] M. R. Douglas, S. Lakshminarasimhan, and Y. Qi “Numerical Calabi-Yau metrics from holomorphic networks,” [2012.04797].
  * [19] M. R. Douglas “Holomorphic feedforward networks,” Pure Appl. Math. Quart. 18 (2022) no. 1, 251–268 [2105.03991].
  * [20] A. Ashmore, L. Calmon, Y.-H. He, and B. A. Ovrut “Calabi-Yau Metrics, Energy Functionals and Machine-Learning,” [2112.10872].
  * [21] M. Larfors, A. Lukas, F. Ruehle, and R. Schneider “Learning Size and Shape of Calabi-Yau Spaces,” Machine Learning and the Physical Sciences, Workshop at 35th NeurIPS (2021) [2111.01436].
  * [22] M. Larfors, A. Lukas, F. Ruehle, and R. Schneider “Numerical metrics for complete intersection and Kreuzer–Skarke Calabi–Yau manifolds,” Mach. Learn. Sci. Tech. 3 (2022) no. 3, 035014 [2205.13408].
  * [23] M. Gerdes and S. Krippendorf “CYJAX: A package for Calabi-Yau metrics with JAX,” [2211.12520].
  * [24] P. Berglund, G. Butbaia, T. Hübsch, V. Jejjala, D. Mayorga Peña, C. Mishra, and J. Tan “Machine Learned Calabi-Yau Metrics and Curvature,” Adv. Theor. Math. Phys. 27, 1107-1158 (2023) [2211.09801].
  * [25] H. Ahmed and F. Ruehle “Level Crossings, Attractor Points and Complex Multiplication,” JHEP 06, 164 (2023) [2304.00027].
  * [26] J. Halverson and F. Ruehle “Metric Flows with Neural Networks,” [2310.19870].
  * [27] L. B. Anderson, J. Gray, and M. Larfors “Lectures on Numerical and Machine Learning Methods for Approximating Ricci-flat Calabi-Yau Metrics,”  12, 2023.  [2312.17125].
  * [28] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst “Geometric deep learning: Going beyond euclidean data,” IEEE Signal Processing Magazine 34 (2017) no. 4, 18–42.
  * [29] M. M. Bronstein, J. Bruna, T. Cohen, and P. Velickovic “Geometric deep learning: Grids, groups, graphs, geodesics, and gauges,” 2021.
  * [30] M. C. N. Cheng, V. Anagiannis, M. Weiler, P. de Haan, T. S. Cohen, and M. Welling “Covariance in Physics and Convolutional Neural Networks,” [1906.02481].
  * [31] D. Yarotsky “Universal approximations of invariant maps by neural networks,” Constructive Approximation 55 (2022) no. 1, 407–474.
  * [32] G. Butbaia, D. Mayorga Peña, J. Tan, P. Berglund, T. Hübsch, V. Jejjala, and C. Mishra “Physical Yukawa Couplings in Heterotic String Compactifications,” [2401.15078].
  * [33] A. Constantin, C. S. Fraser-Taliente, T. R. Harvey, A. Lukas, and B. Ovrut “Computation of Quark Masses from String Theory,” [2402.01615].
  * [34] P. Candelas, A. M. Dale, C. A. Lutken, and R. Schimmrigk “Complete Intersection Calabi-Yau Manifolds,” Nucl. Phys. B298 (1988) 493\.
  * [35] B. Aslan, D. Platt, and D. Sheard “Group invariant machine learning by fundamental domain projections,” 2022.
  * [36] S.-O. Kaba, A. K. Mondal, Y. Zhang, Y. Bengio, and S. Ravanbakhsh “Equivariance with learned canonicalization functions,” in International Conference on Machine Learning pp. 15546–15566 PMLR.  2023\.
  * [37] A. Ashmore and F. Ruehle “Moduli-dependent KK towers and the swampland distance conjecture on the quintic Calabi-Yau manifold,” Phys. Rev. D 103 (2021) no. 10, 106028 [2103.07472].
  * [38] M. R. Douglas, D. Platt, and Y. Qi “Harmonic $1$-forms on real loci of Calabi-Yau manifolds,” [2405.19402].
  * [39] M. Raissi, P. Perdikaris, and G. Karniadakis “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational Physics 378 (2019) 686–707.
  * [40] R. Schneider Heterotic Compactifications in the Era of Data Science.
PhD thesis Uppsala University, 2022.

  * [41] S. Chen, E. Dobriban, and J. H. Lee “A group-theoretic framework for data augmentation,” Journal of Machine Learning Research 21 (2020) no. 245, 1–71.
  * [42] K. Fukushima and S. Miyake “Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position,” Pattern Recognition 15 (1982) no. 6, 455–469.
  * [43] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner “Gradient-based learning applied to document recognition,” Proceedings of the IEEE 86 (12, 1998) 2278 – 2324.
  * [44] T. S. Cohen and M. Welling “Group Equivariant Convolutional Networks,” [1602.07576].
  * [45] T. S. Cohen, M. Geiger, J. Koehler, and M. Welling “Spherical CNNs,” [1801.10130].
  * [46] R. Gens and P. M. Domingos “Deep symmetry networks,” in Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, eds. vol. 27.  Curran Associates, Inc. 2014.
  * [47] S. Ravanbakhsh, J. Schneider, and B. Poczos “Equivariance Through Parameter-Sharing,” [1702.08389].
  * [48] H. Maron, O. Litany, G. Chechik, and E. Fetaya “On Learning Sets of Symmetric Elements,” [2002.08599].
  * [49] J. E. Gerken, J. Aronsson, O. Carlsson, H. Linander, F. Ohlsson, C. Petersson, and D. Persson “Geometric Deep Learning and Equivariant Neural Networks,” [2105.13926].
  * [50] P. S. Green, T. Hubsch, and C. A. Lutken “All Hodge Numbers of All Complete Intersection Calabi-Yau Manifolds,” Class. Quant. Grav. 6 (1989) 105–124.
  * [51] A. Lukas “The list of complete intersection calabi-yau three-folds.” [cicylist].
  * [52] V. Braun “On Free Quotients of Complete Intersection Calabi-Yau Manifolds,” JHEP 04 (2011) 005 [1003.3235].
  * [53] A. Lukas and C. Mishra “Discrete Symmetries of Complete Intersection Calabi-Yau Manifolds,” Commun. Math. Phys. 379 (2020) no. 3, 847–865 [1708.08943].
  * [54] P. Candelas and C. Mishra “Highly Symmetric Quintic Quotients,” Fortsch. Phys. 66 (2018) no. 4, 1800017 [1709.01081].
  * [55] M. Kreuzer and H. Skarke “Complete classification of reflexive polyhedra in four-dimensions,” Adv. Theor. Math. Phys. 4 (2002) 1209–1230 [hep-th/0002240].
  * [56] M. B. Green, J. H. Schwarz, and E. Witten Superstring theory: volume 2, loop amplitudes, anomalies and phenomenology.  Cambridge university press 2012.
  * [57] P. Candelas, G. T. Horowitz, A. Strominger, and E. Witten “Vacuum Configurations for Superstrings,” Nucl. Phys. B 258 (1985) 46–74.
  * [58] E. Witten “Symmetry breaking patterns in superstring models,” Nuclear Physics B 258 (1985) 75–100.
  * [59] J. D. Breit, B. A. Ovrut, and G. C. Segré “E 6 symmetry breaking in the superstring theory,” Physics Letters B 158 (Aug., 1985) 33–39.
  * [60] W. Decker, G.-M. Greuel, G. Pfister, and H. Schönemann “Singular 4-3-0 — A computer algebra system for polynomial computations.” http://www.singular.uni-kl.de 2022\.
  * [61] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei “Scaling Laws for Neural Language Models,” arXiv e-prints (Jan., 2020) arXiv:2001.08361 [2001.08361].
  * [62] A. Ashmore, R. Deen, Y.-H. He, and B. A. Ovrut “Machine Learning Line Bundle Connections,” [2110.12483].
  * [63] M. Nakahara Geometry, topology and physics.  2003\.  Bristol, UK: Hilger (1990) 505 p. (Graduate student series in physics).