aainstitutetext:  Department of Mathematics, City, University of London, EC1V
0HB, UKbbinstitutetext:  London Institute for Mathematical Sciences, Royal
Institution, London W1S 4BS, UKccinstitutetext:  Merton College, University of
Oxford, OX1 4JD, UKddinstitutetext:  School of Physics, NanKai University,
Tianjin, 300071, P.R. Chinaeeinstitutetext:  School of Mathematical Sciences,
University of Nottingham, Nottingham, NG7 2RD, UK

# Polytopes and Machine Learning

Jiakang Bao a,b,c,d Yang-Hui He a,b Edward Hirst e Johannes Hofscheier e
Alexander Kasprzyk a Suvajit Majumder<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We introduce machine learning methodology to the study of lattice polytopes.
With supervised learning techniques, we predict standard properties such as
volume, dual volume, reflexivity, etc, with accuracies up to 100%. We focus on
2d polygons and 3d polytopes with Plücker coordinates as input, which out-
perform the usual vertex representation.

††preprint:  LIMS-2021-011

##  1 Introduction

Lattice polytopes – convex bodies with integral vertices – lie in the
intersection of combinatorics, optimisation theory, number theory, geometry,
and theoretical physics. In combinatorics, problems include finding lattice
points inside a lattice polytope (for example, ehrhart1967probleme ;
Betke1985LatticePI ). In number theory, lattice polytopes constitute the
cornerstone of the geometry of numbers (for example, minkowski1910geometrie ;
poonen2000lattice ). In geometry, lattice polytopes play a key role in the
study of toric varieties, whose geometry can be described combinatorially in
terms of cones and fans (for example, cox2011toric ; kasprzyk2011reflexive ;
batyrev2010generalization ; batyrev2013lattice ). In string theory,
Batyrev–Borisov mirror symmetry BatyrevBorisov+2011+39+66  – describing how
Calabi–Yau manifolds can be naturally constructed from a reflexive polytope –
initiated a large-scale collaboration between fundamental physics and
computational algebraic geometry, culminating in the classification of
reflexive polytopes in dimensions three and four Kreuzer:1995cd ;
Kreuzer:1998vb ; Kreuzer:2000xy . Details of these, and many other,
applications of lattice polytopes can be found in rebhan2013strings ;
hibi2019algebraic .

There has been considerable recent interest in applying techniques from data
science and machine learning (ML) to the study of pure mathematical data.
Whilst initially this arose in large part from the investigation of the string
theory/algebraic geometry landscape He:2017aed ; Krefl:2017yox ;
Ruehle:2017mzq ; Carifio:2017bov ; He:2017set ; Bao:2020sqg , it is natural to
ask whether this paradigm can be applied to different disciplines within pure
mathematics He:2018jtw ; He:2021oav ; Bao:2021auj ; He:2020eva ; Bao:2020nbi ;
Bao:2021olg .

Combinatorial geometry lends itself to ML. The central objects – lattice
polytopes – can be cast into a matrix of integers and fed to a neural network
(NN) classifier or regressor. We will discuss different representations of
this input data in this paper. Meanwhile, much of the work within the geometry
of polytopes involves the extraction of numerical invariants such as volume,
integral point enumeration, identifying topological invariants of the toric
variety, etc. Can NN techniques “learn” and extrapolate from this underlying
structure? We shall see that this is indeed possible.

We should point out that our point d’appui is _not_ ML but is how to use ML to
study the mathematics. Polytope ML is an interesting field in itself, where
learning algorithms are developed to find the hyperplanes that define the
convex body, q.v., gottlieb2018learning ; polytopeMLtalk . What we shall hope
for, instead, is whether standard ML algorithms can find patterns in lattice
polytopes and related algebraic geometry that are difficult or unknown.

The paper is organized as follows. In §2, we give a brief review on polytopes.
In §3 the features of the data for polytopes used in machine learning are
described. Then, in §4, we present our main results on machine learning for
polytopes in different dimensions. In particular, we predict the
volumes111Note that unlike the nice paper Krefl:2017yox  where the authors
tried to learn the associated Calabi-Yau volume, we are learning the polytope
volume here. and dual volumes from their Plücker coordinates. Other
invariants, such as the Gorenstein index and codimension, are also examined.

One is referred to https://github.com/edhirst/PolytopeML.git for the data and
Python code scripts used in this paper.

##  2 Polytope Preliminaries

We begin with a brief reminder of the relevant concepts, which also serves to
lay out notation and conventions. See, for example,
sommerville2020introduction  for more details.

Polytopes are geometric objects formed from $n$ points in $d$-dimensional
Euclidean space. Specifically,

######  Definition 2.1.

A convex polytope $P$ has two equivalent definitions:

Vertex Representation:

the convex hull of set of $n$ points (called vertices)
$p_{i}\in\mathbb{R}^{d}$

${\rm
Conv}(\\{p_{i}\\})=\left\\{\sum\limits_{i=1}^{n}\alpha_{i}p_{i}\,\Big{|}\,\alpha_{i}\geq
0,\ \sum\limits_{i=1}^{n}\alpha_{i}=1\right\\};$ (2.1)

Half-hyperplane Representation:

the intersection of linear inequalities (hyperplanes called facets)

$H\cdot\underline{x}\geq\underline{b}\ ,$ (2.2)

where $\underline{b}$ and $\underline{x}$ are real $n,d$-vectors respectively,
and $H$ is some $n\times d$ matrix.

The vertices are considered as 0-faces, the extremal edges are called 1-faces,
and so on, until codimension 1, which are the $(n-1)$-faces that are also
called facets. The simplest polytope in $\mathbb{R}^{d}$, consisting of $d+1$
vertices, is called a simplex.

In this paper, we will exclusively deal with lattice polytopes where the
vertices are integral points (lattice vectors), i.e., all
$p_{i}\in\mathbb{Z}^{d}$. Henceforth, by polytope, we will mean a (convex)
lattice polytope, denoted by $P$.

Given polytope $P$, a key concept is _duality_ :

######  Definition 2.2.

The (polar) dual of $P$ is the polyhedron

$P^{\circ}\vcentcolon=\\{\textbf{v}\in\mathbb{R}^{d}\;|\;\textbf{u}\cdot\textbf{v}\geq-1\;\forall\;\textbf{u}\in
P\\}.$ (2.3)

When $\textbf{0}\in\textup{int}(P)$ we have that $P^{\circ}$ is a polytope,
albeit with rational-valued vertices. Another important action on $P$ is:

######  Definition 2.3.

The $r$-dilation of $P$ for $r\in\mathbb{Z}_{\geq 0}$ is the polytope

$rP:=\\{r\textbf{x}\;|\;\textbf{x}\in P\\}.$ (2.4)

Finally, we need a notion of primitivity:

######  Definition 2.4.

A vector with integer components is called primitive if the GCD of the
components is equal to $\pm 1$.

In this paper, we are concerned with the following types of polytopes.

######  Definition 2.5.

For a lattice polytope $P$, we have the following:

Fano:

A convex polytope $P$ is called _Fano_ if the origin is strictly in its
interior $\textup{int}(P)$ and all the vertices are primitive lattice points
(q.v.Kasprzyk_2010 ; Kasprzyk2012FANOP ; balletti2016maximum ; batyrev2019fine
). These polytopes are called _canonical Fano_ if further
$\textup{int}(P)\cap\mathbb{Z}^{d}=\\{\mathbf{0}\\}$.

Reflexive:

A Fano polytope $P$ is called _reflexive_ if its polar dual $P^{\circ}$ is
also Fano. Indeed, for a generic lattice polytope the dual will have vertices
in $\mathbb{Q}^{d}$, for reflexives, the vertices of the dual are also in
$\mathbb{Z}^{d}$.

The terminology clearly comes from the study of toric geometry and these terms
also apply to their corresponding toric varieties. Importantly, we have that
borisov1992singular ; borisov2000convex

######  Theorem 2.1.

In any dimension $d$, there is a finite number of canonical Fano polytopes, as
well as reflexive polytopes, up to $\textup{GL}(d,\mathbb{Z})$, which acts on
the vertices.

######  Example 1.

It is illustrative to give a concrete running example. Consider the lattice
polygon in Figure 2.1(a).

(a)(b) Figure 2.1: The lattice polygons which are dual of each other.

The vertices are
$v_{1}=(1,0),v_{2}=(0,-1),v_{3}=(-1,-1),v_{4}=(-1,0),v_{5}=(0,1)$. One can
check that its dual is the polygon shown in Figure 2.1(b) whose vertices are
$v^{\prime}_{1}=(1,0),v^{\prime}_{2}=(1,-1),v^{\prime}_{3}=(0,-1),v^{\prime}_{4}=(-1,0),v^{\prime}_{5}=(-1,2)$.
Indeed, they are both lattice polygons, having vertices in $\mathbb{Z}^{2}$.
Because $P$ and $P^{\circ}$ here are both lattice, this is a pair of reflexive
polytopes in dimension 2. Furthermore, they are clearly both canonical Fano.

We will also investigate the efficiency of ML in predicting the codimension of
the embedding of the toric variety into the weighted projective space induced
by the given polytope. For instance, let $P\subset\mathbb{R}^{d}$ be a Fano
polytope and let $P^{\circ}$ its polar dual. Lattice points in the cone over
the polytope $P^{\circ}\times\\{1\\}\subset\mathbb{R}^{d+1}$ yield a semigroup
which accepts a unique Hilbert basis. Let $n$ be the cardinality of this
Hilbert basis. The _codimension_ of $P$ is $n-\dim(P)-1$.

An important property of $P$ is its volume (which is clearly a
$\textup{GL}(d,\mathbb{Z})$-invariant quantity). For a simplex, there is a
standard formula (q.v. sommerville2020introduction )

######  Theorem 2.2 (Cayley-Menger).

The volume of simplex in $\mathbb{R}^{d}$ is
$V_{d}=\frac{1}{d!}\prod\limits_{i=1}^{d}h_{i}$, where $h_{1}$ is the distance
between the first two vertices (the final answer is independent of the
ordering of the vertices), $h_{2}$ is the height of the third vertex above the
line containing the first two, $h_{3}$ is the height of the fourth vertex
above the plane containing the first three vertices, etc.

For general polytope, we first have that:

######  Definition 2.6.

A triangulation of a polytope $P$ is a partition of $P$ into _simplices_ such
that (i) the union of all them equals $P$ and (ii) the intersection of any
pair of them is a (possibly empty) common face.

In other words, a triangulation is a covering of $P$ by geometrical simplices.
Thereby, we have

######  Definition 2.7.

The volume of a polytope $P$ is the sum of volumes of the simplices which
constitutes any triangulation.

The _normalized volume_ of a $d$-dimensional polytope is given by $d!$ times
the volume. We will also be interested in the volume of the dual polytope
$P^{\circ}$, to which we will refer as dual volume.

Finally we introduce the Gorenstein index of $P$:

######  Definition 2.8.

Let $P$ be a Fano polytope. The dual polytope $P^{\circ}$ is, in general, a
rational polytope. Let $r\in\mathbb{Z}_{>0}$ be the smallest positive integer
such that $rP^{\circ}$ is a lattice polytope (and hence Fano). We call $r$ the
_Gorenstein index_ of $P$.

Clearly a Fano polytope is reflexive if and only if the Gorenstein index is
$1$.

######  Example 2.

For our running example in Figure 2.1(a), the polygon has (normalized) volume
5. Its dual volume, which is the volume of the polygon in Figure 2.1(b), is
equal to 7. Since in general the dual polygon can have rational vertices
(instead of only integer ones), the dual volume can be a rational number. The
Gorenstein index is 1 in both cases, since both polygons are reflexive. The
codimension of $P$ is $8-2-1=5$, and the codimension of $P^{\circ}$ is
$6-2-1=3$.

###  2.1 Plücker Coordinates

In Definition 2.1, we gave two standard representations of $P$. However, there
is another important representation which we will use in this paper: the
Plücker coordinate representation.

Many problems in mathematics and physics rely on the structure of polytopes,
without regard for the specific embedding in the lattice they are drawn on. It
is with this motivation that Plücker coordinates are a natural consideration
for polytope representation in our subsequent work.

######  Definition 2.9.

Let $P$ be an $d$-dimensional polytope with $n$ vertices. This structure can
be reformatted into an $d\times n$ matrix $V$ where each column corresponds to
a vertex $v_{i}$. Consider the integer kernel of $V$ (aka grading) and take
the maximal minors of $\ker(V)$; this gives a list of integers known as the
Plücker coordinate, which we regard as a point in projective space.

######  Example 3.

For the pentagon in Figure 2.1(a), the vertices matrix and its kernel are

$V=\begin{pmatrix}v_{1}&v_{2}&v_{3}&v_{4}&v_{5}\\\ \hline\cr 1&0&-1&-1&0\\\
0&-1&-1&0&1\\\
\end{pmatrix}\Rightarrow\ker(V)=\begin{pmatrix}v_{1}&v_{2}&v_{3}&v_{4}&v_{5}\\\
\hline\cr 1&0&1&0&1\\\ 1&0&0&1&0\\\ 0&1&0&0&1\\\ \end{pmatrix},$ (2.5)

which in fact reveals the linear relations among the vertices:
$v_{1}+v_{3}+v_{5}=0$, $v_{1}+v_{4}=0$ and $v_{2}+v_{5}=0$. By taking all of
the possible $3\times 3$ minors, of which there are ${5\choose 3}=10$, we get
the Plücker coordinate as
$(1:-1:1:0:-1:1:1:0:-1:1)\in\mathbb{C}\mathbb{P}^{9}$. Likewise, the polygon
in Figure 2.1(b) has Plücker coordinate
$(2:-1:1:-1:-1:1:2:0:-1:1)\in\mathbb{C}\mathbb{P}^{9}$.

##### Remark:

Note that due to the GL$(d,\mathbb{Z})$ invariance, it is possible that two
inequivalent lattice polytopes can have the same Plücker coordinates. For
example, the polygon with vertices $\\{(-1,-1),(1,0),(0,1)\\}$ and that with
$\\{(-1,-1),(2,-1),(-1,2)\\}$ both have Plücker coordinates
$(1:1:1)\in\mathbb{C}\mathbb{P}^{2}$. We can distinguish such polytopes by
using an additional piece of combinatorial data called the quotient gradings.
However, we will not take this into consideration in this paper. Instead, _we
require from here onwardwards that the vertices of $P$ generate the lattice
$\mathbb{Z}^{d}$_. With this additional assumption, the Plücker coordinates
uniquely determine $P$ (up to the action of GL$(d,\mathbb{Z})$).

There is some trivial redundancy in the Plücker coordinates, based on the
initial ordering of the polytope’s vertices (columns of the vertex matrix).
This leads to at most $n!$ equivalent ways of writing the Plücker coordinates
for a polytope with $n$ vertices, coming from the permutation group
$\mathfrak{S}_{n}$ shuffling the vertices. Such ordering is irrelevant for the
Plücker coordinate object, but for tensor (vector) representation for ML input
it allows some redundancy for data augmentation, as we will see.

It is not hard to check that the Plücker coordinates are GL$(d,\mathbb{Z})$
invariant. As aforementioned, Plücker coordinates reveal the linear relations
among the vertices and are hence endowed with geometric meanings. These
properties make the Plücker coordinates favourable as input vectors for our
machine learning problem.

We summarize the key properties of Plücker coordinates as follows.

######  Proposition 2.3.

Given a $d$-dimensional polytope $P$ with $n$ vertices, the number of Plücker
coordinates is $N={n\choose n-d}$ as the $n$ vertices of a $d$-dimensional
polytope satisfy $(n-d)$ linear relations, with (at most) $n!$ choices of
Plücker coordinates in $\mathbb{CP}^{N-1}$. The Plücker coordinates are
GL$(d,\mathbb{Z})$ invariant.

##  3 Polytope Datasets

Having introduced the preliminaries of polytopes which will be used in the
paper, we now move on to describe the data we considered.

###  3.1 Polygons in dimension 2

Here we consider $P$ a Fano polygon whose vertices generate the lattice
$\mathbb{Z}^{2}$. The dataset of such polygons used is available from the link
in §1. Each polygon has Gorenstein index at most $30$. The dataset consists of
polygons with vertices $n\in\\{3,4,5,6\\}$ with frequencies
$\\{277,7041,16637,3003\\}$, respectively, giving a total of $26958$ polygons.

For each polygon we compute the series of invariants:

$(\mbox{volume, dual volume, Gorenstein index, codimension})$ (3.1)

These properties take values within the ranges:
$[3,514],(0.21,15.37),[1,30],[2,42]$ respectively, where all properties are
integer-valued except dual volume which are rational and hence computationally
represented with floating-point reals.

Plots of the frequencies of each of these properties across the dataset for
each number of vertices are shown in Figure 3.1. Furthermore, Figure 1(a)
shows the distribution of the number of vertices across the dataset.

(a)

(b)

(c)

(d)

(e)

Figure 3.1: Distributions of the polygon dataset’s properties, split by the
number of polygon’s vertices, $n$. (a) shows the frequencies of polygons with
each number of vertices in the dataset. (b-e) show the distributions over:
volume, dual volume, Gorenstein index, and codimension respectively.

Now, we need to represent the polygons as appropriate tensors for the ML
input. The naive initial choice of representation was a flattened list of the
vertices, however as will be shown in the first investigation of §4.2 this is
outperformed by the Plücker coordinate representation.

Since for $n$ vertices the Plücker coordinate has length ${n\choose n-2}$, the
input vectors vary in length. Furthermore, there is redundancy in the ordering
of the Plücker coordinate (or, more precisely, the ordering of the vertices
when computing the Plücker coordinate), as discussed in Proposition 2.3. The
dataset was thus augmented with (at most) 3 randomly chosen distinct Plücker
representations for each polygon. This serves two roles: it prevents the
machine from inadvertently learning in a way that depends on our choice of
vertex ordering; and it provides a larger dataset on which to train. This made
the final dataset a list of 80874 Plücker representations with their
respective polygon invariants. For reference, the Plücker coordinates over the
polygon dataset take value in the range [-450,450], whilst the coordinates of
the vertices take value in the range [-290,180] (although this, of course,
depends on the choice of basis).

###  3.2 Polytopes in dimension 3

Next, we move onto dimension 3. Here our focus is the 674688 canonical Fano
polytopes, as classified in Kas08 222The data can be found at
http://www.grdb.co.uk/forms/toricf3c in the GRDB database.. The full data
contains many invariants for each polytope, including the volume, the dual
volume, whether it is reflexive, the Gorenstein index, and the codimension. As
before, we enhance our dataset using the $n!$ choices of Plücker coordinate
for a polytope with $n$ vertices. We randomly choose (at most) 10 distinct
Plücker coordinates for each polytope. This augmentation of the data removes
bias due to our choice of ordering of the vertices (and hence Plücker
coordinate). The enhanced dataset has 6744246 data points.

Since this is a huge dataset, and since we would only need much fewer data
points to train the model, we will consider 800000 random samples here. As the
choice is completely random and this is still a large amount of data, we
should lose no generality here.

The distribution of the volumes is plotted in Figure 2(a).

(a)

(b)

Figure 3.2: (a) The distribution of the volumes for 800000 random samples. (b)
The distribution of the dual volumes for 800000 random samples.

Similarly, we plot the distribution of the dual volumes in Figure 2(b). Unlike
the volumes which have a symmetric distribution, the dual volumes have a
cluster of data between 10 and 20 while there are much fewer samples for large
dual volumes.

A famous subset of the canonical Fano polytopes are the 4319 reflexive
polytopes Kreuzer:1998vb , whose geometry was explored in He:2017gam . In our
enhanced dataset, we have 42943 data points that correspond to these reflexive
polytopes. For our ML task in §4.3.3, we need to check whether the reflexive
and non-reflexive ones distribute differently. Therefore, we randomly choose
42943 data points corresponding to non-reflexive polytopes and plot the
distributions in terms of the numbers of vertices for the two cases in Figure
3.3.

(a)

(b)

Figure 3.3: (a) The distribution of the data points corresponding to reflexive
polytopes, in terms of the numbers of their vertices. (b) The distribution of
the 42943 randomly chosen data points corresponding to non-reflexive
polytopes, in terms of the numbers of their vertices.

As we can see, these two plots have similar distributions.

##  4 Machine Learning

In this section we initiate the study of applying ML to the data introduced
above. In particular, we consider the labelled data, each entry being of the
form

$\mbox{Pl\"{u}cker}(P)\longrightarrow\mbox{Property}(P)$ (4.1)

where “property” is one of: volume, dual volume, reflexivity, codimension, and
Gorenstein index. This is a supervised ML problem, where we will train a NN on
some percentage of data and validate on the remaining. In this paper, we shall
be examining the two lowest dimensional cases, subdividing investigations into
2d polygons333 Since there are only 16 reflexive polygons, reflexivity
considerations only apply for 3d and above. , and 3d polytopes.

###  4.1 Methodology

##### MLP for 2d polygons

In each ML investigation the respective datasets are split into 5 subsets for
a 5-fold cross-validation analysis, training 5 independent NNs each on a
different complement of a fifth of the data set aside for independent
validation. The NNs trained are standard dense feed-forward, each with 4 dense
layers of 64 neurons, and a leaky-ReLU activation (with $\alpha=0.01$). The
final layer is a dense layer with a single neuron outputting the predicted
property for each input vector.

They are trained in batches of 32 for 20 epochs, using an Adam optimiser
kingma2017adam  to minimise a log(cosh) loss function. In addition to the mean
absolute error (MAE) learning measure for the supervised regression problem
style, accuracy bins were also used to determine the ratio of test data that
was within a specified range of the true value. Note that although simple MAE
was used for assessing learning performance, log(cosh) was used as the loss
function in training as a smoother equivalent to improve training.

##### MLP for 3d polytopes

For 3d polytopes, we used a slightly different structure of MLP since the
Plücker coordinates can be much longer. Here, we use one hidden layer with 100
perceptrons. The leaky ReLU activation function now has $\alpha=10^{-5}$. The
data is trained in batches of 16 with an Adam optimizer to minimize the mean
squared error loss.

##### CNN for reflexivity

Besides MLP, we also use a CNN for the classification problem of reflexivity
for 3d polytopes, where we have one convolution layer followed by a Leaky ReLU
layer ($\alpha=0.1$) and a MaxPooling layer. We add a SoftMax layer at the end
of this structure. The data is trained in batches of 16 for 100 epochs with
Adam as the optimizer to minimize the mean squared error loss.

##### Random forest for reflexivity

We can further perform the same tests using random forest, where we have 70
estimators for the classifier.

###  4.2 Polygons

To motivate the choice of Plücker coordinates, over the naive list of vertex
coordinates, as the polygons’ representation method, an initial investigation
is carried out. In this ML investigation the full dataset of polygons is
considered to learn the respective polygon’s volume from each input form, for
each number of vertices considered.

The NNs are trained as regressors, then assessed using varying measures. The
first measure calculates the MAE between predicted and true volumes on the
test data, where a value of 0 indicates perfect learning. The binned accuracy
measures determine the ratio of test polygons whose predicted volume is within
some bin centred on the true value. These bins are chosen to have three sizes:
width 1, width $0.025\times\text{range}$, and $0.05\times\text{range}$; where
“range” is the difference between the maximum and minimum volumes of polygons
in the full dataset. Accuracies of 1 indicated perfect prediction within the
bin size error. For volume, the range takes the value of 511, but is lower for
other properties.

These bin styles are used across all subsequent investigations in this
subsection. The bin width of 1 identifies where properties can be predicted to
the nearest integer, particularly important for those properties restricted to
take integer values. The second two bin styles based on the ranges of the
properties across the dataset give a more representative and comparable
measure of learning performance.

####  4.2.1 Predicting Polygon Properties

Results for these investigations, comparing inputs of vertices and Plücker
coordinates, are given in Table 4.1. For all investigations carried out, at
each number of vertices, for MAE and all accuracy bin sizes considered,
learning is better using Plücker coordinates than vertices. Interestingly in
the investigations using polygons with less vertices the test set predictions
are always within the range-based bins, showing that volume can be learnt very
well from the polygon Plücker coordinates. One reason we believe is that the
Plücker representation preserves the rotational invariance of the polytope, so
does the polytope volume. Further to this, volume is a determinant calculation
from vertices which is a long computation to machine learn, whereas Plücker
coordinates already encode part of this information in the computation of the
minors. Therefore in subsequent investigations we adhere to the Plücker
representation for the polygons.

Number of Vertices | Representation | MAE | Accuracy
---|---|---|---
$\qquad\pm 0.5\qquad$ | $\pm 0.025\times\text{range}$ | $\pm 0.05\times\text{range}$
3 | Vertices | 4.941 | 0.302 | 0.891 | 0.945
Plücker | 0.209 | 0.827 | 1.000 | 1.000
4 | Vertices | 10.012 | 0.072 | 0.891 | 0.945
Plücker | 0.625 | 1.000 | 1.000 | 1.000
5 | Vertices | 8.640 | 0.060 | 0.777 | 0.926
Plücker | 1.051 | 0.451 | 1.000 | 1.000
6 | Vertices | 14.947 | 0.036 | 0.603 | 0.826
Plücker | 3.359 | 0.139 | 0.969 | 0.997
Table 4.1: Machine learning polygon volume from polygon representations:
flattened list of vertices, or Plücker coordinates. ML runs on Dense NN
Regressors, and investigations are carried out for each subset of polygons
with each number of vertices. Learning is measured using MAEs, and accuracies
(to 3d.p.) of test set predicted volumes being within some bin centred on the
true value. The first bin has width 1, the second and third have widths of
$5\%\text{ or }10\%\times\text{range}$ respectively, where “range” is the
difference between the maximum and minimum volumes in the full dataset (511).
Plücker coordinates consistently performs better as the input representation
method.

Results showing the learning accuracies (with the bin structure as described
previously) for the four properties considered are given in Table 4.2. The
ranges used in calculating the latter bin widths are: 511, 15.16, 29, 40; for
volume, dual volume, Gorenstein index, and codimension respectively. Note that
the volume learning results from Plücker coordinates are repeated in both
tables 4.1 and 4.2, for ease of comparison. The results show that volume can
be learnt especially well for all number of vertices. In addition dual volume
can be learnt well, and codimension with some success. Conversely Gorenstein
index could not be learnt with this NN architecture.

Whereas volume is a determinant and likely a simpler function of Plücker
coordinates to learn, hence the good performance. Dual volume is less simple
to compute and surprisingly still is learnt well, hinting at some simpler
function that connects these inputs and outputs that shortcuts computation of
the dual polytope. The lesser performance for codimension and Gorenstein index
indicate they are likely incredibly complex, if not even non-existent,
functions of the Plücker coordinates (and all performed even worse using input
as vertices). In general learning is more successful for lower numbers of
vertices, perhaps due to the smaller Plücker vector inputs, and the larger
datasets available for these polygons. Presumably this also explains the
occasional lower performances learning with triangles compared to
quadrilaterals and pentagons.

Property |  Number of Vertices | MAE | Accuracy
---|---|---|---
$\qquad\pm 0.5\qquad$ | $\pm 0.025\times\text{range}$ | $\pm 0.05\times\text{range}$
Volume | 3 | 0.209 | 0.826 | 1.000 | 1.000
4 | 0.615 | 0.625 | 1.000 | 1.000
5 | 1.051 | 0.452 | 1.000 | 1.000
6 | 3.359 | 0.139 | 0.969 | 0.997
Dual Volume | 3 | 1.181 | 0.370 | 0.370 | 0.501
4 | 0.642 | 0.634 | 0.634 | 0.754
5 | 0.818 | 0.496 | 0.496 | 0.638
6 | 0.941 | 0.405 | 0.405 | 0.557
Gorenstein index | 3 | 5.710 | 0.039 | 0.064 | 0.132
4 | 5.002 | 0.069 | 0.101 | 0.196
5 | 4.632 | 0.071 | 0.102 | 0.202
6 | 5.343 | 0.056 | 0.080 | 0.150
Codimension | 3 | 1.897 | 0.192 | 0.361 | 0.615
4 | 2.726 | 0.140 | 0.268 | 0.496
5 | 3.182 | 0.103 | 0.210 | 0.404
6 | 2.884 | 0.126 | 0.251 | 0.468
Table 4.2: ML results for each of the properties considered, using input
Plücker coordinates representing the polygons in each subset of the full
dataset based on the number of polygon vertices. Learning is measured with MAE
and accuracies based on test set predictions being within some bin centred on
the true value. The bin widths are based on the range of values each property
can take in the full dataset. Results show volume and dual volume learnt well,
codimension learnt to some extent, and Gorenstein index could not be learnt.
For reference the respective ranges are: 511, 15, 29, 40 for volume, dual
volume, Gorenstein index, and codimension respectively.

To further represent these learning results, the NN regressor predictions for
the more successful learning of volume and dual volume are plotted against the
true values over the range of numbers of vertices considered. These plots are
shown in Figure 4.1 (plotted with the $y=x$ line for comparison), and show how
a NN with the structure as before trained on 80% of the data performed on the
complement 20% test dataset. In addition to the plotted true vs predicted
values over the respective test datasets, the respective MAEs are repeated in
the respective captions of Figure 4.1 also.

(a) Volume $n=3$, MAE: 0.209

(b) Volume $n=4$, MAE: 0.615

(c) Volume $n=5$, MAE: 1.051

(d) Volume $n=6$, MAE: 3.359

(e) Dual Volume $n=3$, MAE: 1.181

(f) Dual Volume $n=4$, MAE: 0.642

(g) Dual Volume $n=5$, MAE: 0.818

(h) Dual Volume $n=6$, MAE: 0.941

Figure 4.1: The true values vs NN predictions for ML volume (a-d), and dual
volume (e-h), over the range of number of vertices, $n$, considered. Plots
show the $y=x$ line for ease of comparison; and the MAE values are also given
for each plot in the respective subcaptions to 3 decimal places.

These plots reiterate the strong learning results of particularly volume, but
also dual volume, from the Plücker coordinates.

####  4.2.2 Plücker Augmentation

It is anticipated the function that the NNs are trying to approximate, perhaps
relies on computation of greatest common divisors (gcds). Therefore to test
this idea, the input vectors of Plücker coordinates for the pentagons (the
largest dataset) are augmented with gcds in two different ways to see if the
NNs could improve their learning by using these values directly. The two gcd
schemes considered used either: all $\binom{l}{2}$ pairwise gcds (denoted
2-gcds), or all gcds of the coordinate subsets formed from omitting one
coordinate (denoted $(l-1)$-gcds); note $l$ is the length of the Plücker
vector.

The learning results are shown in Table 4.3, learning each of the properties
with either: no augmentation, 2-gcds augmentation, or $(l-1)$-gcds
augmentation. Any learning improvement for either scheme is negligibly
improved if at all, indicating that gcds were not especially important in the
underlying property calculation from the input Plücker coordinates. Note the
no augmentation cases are repeated from previous tables for ease of
comparison.

Property |  GCD Scheme | MAE | Accuracy
---|---|---|---
$\qquad\pm 0.5\qquad$ | $\pm 0.025\times\text{range}$ | $\pm 0.05\times\text{range}$
Volume | none | 1.051 | 0.452 | 1.000 | 1.000
$2$-gcds | 1.246 | 0.356 | 0.999 | 1.000
$(l-1)$-gcds | 0.949 | 0.455 | 1.000 | 1.000
Dual Volume | none | 0.818 | 0.496 | 0.496 | 0.638
$2$-gcds | 0.865 | 0.465 | 0.465 | 0.608
$(l-1)$-gcds | 0.832 | 0.487 | 0.487 | 0.629
Gorenstein Index | none | 4.632 | 0.071 | 0.102 | 0.202
$2$-gcds | 4.685 | 0.080 | 0.114 | 0.219
$(l-1)$-gcds | 4.489 | 0.082 | 0.119 | 0.234
Codimension | none | 3.182 | 0.103 | 0.210 | 0.404
$2$-gcds | 2.569 | 0.135 | 0.263 | 0.493
$(l-1)$-gcds | 3.215 | 0.103 | 0.205 | 0.399
Table 4.3: ML Results for learning the pentagon properties from Plücker
coordinate inputs with varying augmentations schemes based on GCDs. The
learning shows the results for no augmentation, marked ‘none’, then for
augmentation with the Plücker pairwise gcds ($2$-gcds), then for augmentation
with the Plücker gcds of all coordinates excluding one coordinate each time
($(l-1)$-gcds), where $l$ is the length of the Plücker coordinate vector.
Learning is measured with MAE and accuracies based on the test set predictions
being within some bin centred on the true value.

Further data input reformulation through one-hot encoding and principle
component analysis (PCA) were also both attempted, neither leading to non-
negligible improvements in the aforementioned results for learning the
considered properties.

####  4.2.3 Restricting Training

To further examine the learning, the ML of volume from Plücker coordinates is
examined at different train:test ratios. The results for pentagons, are shown
in Figure 4.2, the performance at each ratio is assessed with a variety of
learning measures, including: MAE, Log(cosh), Mean Absolute Percentage Error
(MAPE), and Mean Squared Error (MSE). The NNs are still trained on the
Log(cosh) loss function, which is well approximated by the more easily
interpretable MAE used to assess learning in previous investigations. In
addition the MAPE and MSE measures provide further assessments of learning
performance and are plotted with those aforementioned.

Learning is still very strong when smaller datasets are used for training,
since the MAE regressor measure remains low consistently as training ratio
increases. The MSE measure does however take a significantly larger value for
lower accuracies, highlighting that more training data does lead to improved
learning performance, as perhaps expected. The good performance from smaller
sized training datasets reinforces the idea of there being a simple function
from Plücker coordinates to volume that can quickly be approximated by the NNs
without much input data requires.

Figure 4.2: NN regressor measures for learning pentagon volume from Plücker
coordinates, at varying proportions of the dataset used for the training data.
The results show strong learning for the standard MAE measure used such that
good predictions can be achieved quickly; whilst the improvement for MSE shows
that more training data does lead to better learning.

####  4.2.4 The Inverse Problem

In vain of this success, let us consider the inverse problem of learning one
of the Plücker coordinates of the polygon, given the remaining coordinates and
the volume of the polygon as input data. The NN input is now thus the vector
of Plücker coordinates with final coordinated omitted and the polygon volume
augmented onto the end.

Motivated by previous investigations, due to the apparent existence of a
closed form expression for the volume that can be inverted, we hence expect
this problem to be machine-learnable. Indeed, this expectation is realised by
the NN with the same architecture as before. For the polygons, we achieved
MAEs $>3.5$ consistently, noting that the range in the final Plücker entry is
173 this is an impressive result. Also, accuracies are $>90\%$ for an error
margin of 0.05 times the range of the respective Plücker coordinate predicted.
These results are summarised in table 4.4.

Coordinates | Number of vertices | MAE | Accuracy
---|---|---|---
$\pm 0.025\times\text{range}$ | $\pm 0.05\times\text{range}$
Plücker coordinates | 3 | 0.295 | 1.000 | 1.000
4 | 2.599 | 0.950 | 0.983
5 | 2.061 | 0.981 | 0.994
6 | 3.169 | 0.900 | 0.971
Table 4.4: ML Results for learning one of the Plücker coordinates from the
rest of the Plücker coordinates and the volumes of the polygons. Results show
MAE (note the predicted final Plücker coordinate has a range of 173), and the
binned accuracies.

###  4.3 Polytopes

Following success learning polygon properties in 2d, we can turn our attention
to performing similar tests on polytopes in 3d as well. The input vectors are
still the Plücker coordinates, and we pad 0’s to the end of the vectors to
cope with varying input length. We start with learning their volumes.

####  4.3.1 Predicting Polytope Volumes

We use MLP regressors to predict the polytope volumes $V$ as $4\leq V\leq 72$,
now across all values for the number of vertices. We randomly choose 800000
samples from the dataset for training (whose distribution was shown in Figure
2(a)). At $90\%$ training percentage, the results are listed in Table 4.5.

MAE | 1.680
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.196 | 0.380 | 0.667 | 0.846 | 0.936
Table 4.5: The MAE and accuracies for predicting the volumes with 800000
samples at 90% training percentage. The errors indicate the
accuracies/percentages of the predictions
$V_{\text{pred}}\in[V_{\text{actual}}-\text{err},V_{\text{actual}}+\text{err}]$.

As we can see, both MLP and CNN can give pretty high accuracies with $\pm
5.797\%(=\pm 4/69)$ error tolerance. In particular, MLP easily gives over
$90\%$ accuracy. Moreover, given such a large range of volumes, the MAE result
shows that the mean difference between a predicted volume and a true volume is
even less than 2. One may wonder how these trained models would perform for
the remaining samples among the 6.7 million data points. In fact, it is
natural to expect that the models would keep their efficacy and have the same
performance as in Table 4.5. This is because random samples are chosen from a
fixed dataset and hence there is no extrapolation for the machine. More
importantly, the results would not change since we are dealing with
mathematical objects without noise in the data. If the machine finds any
patterns in the training data (which is chosen randomly), all such data should
also follow the patterns. Indeed, this is verified in Table 4.6 with 100000
more samples from the remaining data444Henceforth, we will not repeat this for
future tests. Unless specified, we will only list the validation results for
part of the data (due to the huge amount of the data) and the efficacy of the
models on _any_ polytope should be guaranteed following the argument here..

MAE | 1.644
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.202 | 0.382 | 0.678 | 0.852 | 0.942
Table 4.6: The MAE and accuracies for predicting the volumes with 100000 more
samples.

We also plot the true values versus predicted values for the 80000 test
samples as 10% of the dataset in Figure 4.3.

Figure 4.3: The true volumes vs predicted volumes. The PMCC is 0.967.

This shows that by training only $720000/6744246=10.676\%$ of the data, we can
already reach rather high accuracy for the polytope volumes.

Similar to the polygon tests before, we may also augment the input Plücker
coordinates with for instance their gcds to see if the results would be
improved. It turns out that including all the “($l-1$)-gcds” (viz, gcds of
$(l-1)$ elements in an input vetor of length $l$) would give roughly the same
performance as in Table 4.7.

MAE | 1.696
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.190 | 0.371 | 0.661 | 0.844 | 0.936
Table 4.7: The MAE and accuracies for predicting the volumes with 720000
training samples. The input Plücker coordinates are augmented with their
“($l-1$)-gcds”.

One may also try other manipulations of the input (such as augmentation with
pairwise gcds or using one-hot encoding or PCAs), but they would all lead to
worse results. Therefore, taking the computation cost into account, bare
Plücker coordinates (without further modifications) is the optimal choice for
predicting polytope volumes.

##### Plücker coordinates with fixed length

Recall that we pad 0’s to the input vectors due to their different lengths.
Although the Plücker coordinates may already have 0 as an entry indicating
certain relations among the vertices, 0 is yet the most natural choice for
padding555One may also try some other values for padding such as sufficiently
large positive or negative numbers. However, all would make the predictions
much worse.. On the other hand, we may also restrict the input Plücker
coordinates to some fixed length as in §4.2 so as to see whether such padding
would bring noise to the input and whether fixed length in lieu of varying
lengths would help the machine to learn the volumes.

We first choose Plücker coordinates of length 10. Now there are only 36487
samples of length 10 in the dataset and $6\leq V\leq 70$. We train 4500
samples (i.e., $12\%$ of 36487 samples) as in Table 4.8.

MAE | 0.305
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.820 | 0.974 | 1.000 | 1.000 | 1.000
Table 4.8: The MAE and accuracies for predicting the volumes with 4500
training samples. The input Plücker coordinates have length fixed to 10.

We also test this with augmenting ($l-1$)-gcds in Table 4.9.

MAE | 0.311
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.786 | 0.978 | 0.996 | 0.998 | 1.000
Table 4.9: The accuracies for predicting the volumes with 4500 training
samples. The input Plücker coordinates have length fixed to 10 and are further
augmented with ($l-1$)-gcds (i.e., 9-gcds).

As a result, we see that the performance is greatly improved for Plücker
coordinates of just length 10 compared to those for various lengths.

We may also try Plücker coordinates with longer fixed lengths. For instance,
the results for those of lengths 35 and 56 are listed in Table 4.10.

Length 35 | MAE | 0.908
---|---|---
Accuracies | 0.351 | 0.636 | 0.916 | 0.986 | 0.997
Length 35 with ($l-1$)-gcds | MAE | 1.090
MLP | 0.290 | 0.546 | 0.859 | 0.967 | 0.993
Length 56 | MAE | 1.408
MLP | 0.226 | 0.434 | 0.749 | 0.910 | 0.973
Length 56 with ($l-1$)-gcds | MAE | 1.543
MLP | 0.209 | 0.404 | 0.706 | 0.877 | 0.956
Table 4.10: The MAE and accuracies for predicting the volumes with input
vectors of fixed lengths. The accuracies are still for error tolerance $(\pm
0.5,\pm 1,\pm 2,\pm 3,\pm 4)$. For length 35, there are 899564 samples and
$10\leq V\leq 69$. We train 90000 samples (i.e., $10\%$ of all the samples)
for the models. For length 56, there are 1728583 samples and $12\leq V\leq
70$. We train 90000 samples (i.e., $5\%$ of all the samples) for the models.

We find that the performance also has a significant improvement though shorter
fixed lengths would have greater improvements. It is also worth noting that
the training percentages of all the samples are similar to the percentage for
the case with varying lengths. Remarkably, for the tests with fixed length 56,
performance is good with the training percentage only $5\%$.

####  4.3.2 Predicting Dual Volumes

In addition to examining volume, the Plücker coordinates are used in this
investigation to predict the dual volumes $V_{\text{dual}}$ of the polytopes.
Again, we use MLP and CNN regressors as $4\leq V_{\text{dual}}\leq 72$. We
still train 720000 samples, with results are listed in Table 4.11.

MAE | 2.590
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$ | $\pm 5$
Accuracy | 0.129 | 0.253 | 0.478 | 0.669 | 0.807 | 0.890
Table 4.11: The MAE and accuracies for predicting the dual volumes with 720000
samples.

As we can see, the performance is not as good as for volumes. Nevertheless,
the mean difference between predicted values and true values is only slightly
above 2.5. The accuracy could still approach almost $90\%$ if we allow an
error tolerance of $\pm 7.2\%(=\pm 5/69)$.

We also plot the true values versus predicted values for 80000 random test
samples in Figure 4.4.

Figure 4.4: The true dual volumes vs predicted dual volumes. The PMCC is
0.770.

We can also augment the input Plücker coordinates with their $(l-1)$-gcds as
before to see how this would affect the performance. The results can be found
in Table 4.12.

MAE | 2.414
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$ | $\pm 5$
Accuracy | 0.145 | 0.280 | 0.529 | 0.713 | 0.833 | 0.902
Table 4.12: The accuracies for predicting the dual volumes with 720000
samples. The input Plücker coordinates are augmented with their $(l-1)$-gcds.

One may also modify the input with other methods such as one-hot encoding or
PCA, but they would not improve the performance as seen in previous
investigations.

To check whether the data distribution has affected the above machine-learning
results, recall the uneven distribution for dual volumes in Figure 2(b). It is
possible that the machine is just predicting values in the range $[10,20]$ all
the time to get a near 0.9 accuracy (for $\text{err}=\pm 5$). To avoid
potential “bias” caused by this distribution, we further choose 10000 random
samples whose dual volumes are greater than or equal to 25 and use the above
trained models to predict their dual volumes. The results are listed in Table
4.13.

Plücker coordinates | MAE | 2.647
---|---|---
Accuracy | 0.129 | 0.253 | 0.478 | 0.663 | 0.801 | 0.881
Augmentation with ($l-1$)-gcds | MAE | 2.447
Accuracy | 0.140 | 0.274 | 0.519 | 0.708 | 0.824 | 0.900
Table 4.13: The MAEs and accuracies for predicting the dual volumes in the
range $[25,72]$. The accuracies are still for error tolerance $(\pm 0.5,\pm
1,\pm 2,\pm 3,\pm 4,\pm 5)$.

Compared to the results in Table 4.11 and 4.12, we find that the models still
have quite similar performance.

##### Plücker coordinates with fixed length

As before, let us also check whether fixing the input length would improve the
predictions for dual volumes. Again, we choose Plücker coordinates of length
10 as an example, where the dual volume ranges from 4.8 to 66.5. Altogether
there are 36487 samples. It turns out that even at $90\%$ training as in Table
4.14, the results are not improved (even worse) compared to varying input
lengths.

Plücker coordinates | MAE | 3.675
---|---|---
Accuracy | 0.086 | 0.171 | 0.347 | 0.501 | 0.650 | 0.752
Augmentation with ($l-1$)-gcds | MAE | 4.051
Accuracy | 0.073 | 0.146 | 0.295 | 0.443 | 0.583 | 0.710
Table 4.14: The MAEs and accuracies for predicting the dual volumes whose
Plücker coordinates are all of length 10 using different types of input
vectors. The accuracies are still for error tolerance $(\pm 0.5,\pm 1,\pm
2,\pm 3,\pm 4,\pm 5)$.

Even though this is already $90\%$ of the whole data, the size of the training
set is still not comparable to 72000. Therefore, it is natural to expect the
worse predictions here. However, even if we had sufficient data, there would
still be a failure of possible improvements. To see this, we also train the
same amount of data (i.e., 32838 samples) whose Plücker coordinates have
different lengths. The results are listed in Table 4.15. As we can see, the
volumes and dual volumes behave very differently in terms of Plücker
coordinates.

Plücker coordinates | MAE | 3.331
---|---|---
Accuracy | 0.106 | 0.218 | 0.400 | 0.559 | 0.696 | 0.787
Augmentation with ($l-1$)-gcds | MAE | 3.190
Accuracy | 0.106 | 0.208 | 0.413 | 0.582 | 0.703 | 0.803
Table 4.15: The MAEs and accuracies for predicting the dual volumes whose
Plücker coordinates are of different lengths using different types of input
vectors. The accuracies are still for error tolerance $(\pm 0.5,\pm 1,\pm
2,\pm 3,\pm 4,\pm 5)$. The training set has $32838\,(=36487\times 90\%)$ data
points.

####  4.3.3 Reflexivity of the Polytopes

Unlike 2d lattice polygons among which only 16 are reflexive, there are 4319
reflexive polytopes in our dataset of 3d canonical polytopes. Therefore, we
may also use the Plücker coordinates as input to tell whether a polytope is
reflexive or not. Now this becomes a binary classification where we use 0 to
denote non-reflexive polytopes and 1 for reflexive ones.

In our dataset with 6744246 polytopes, 42943 of them are reflexive. Therefore,
we further choose 42943 non-reflexive ones to create a balanced dataset. We
use $90\%$ of them to train the models666Recall that this is essentially
training $\sim 77000$ samples out of the 6.7 million data points and the
results should be valid for the whole dataset as we are machine-learning
mathematical objects.. Balanced datasets prevent the machine easily reaching
over 0.9 accuracy by simply predicting 0 all the time, which would happen if
the full unbalanced (therefore biased) dataset is used.

Here, we compare different types of inputs including the bare Plücker
coordinates, augmentation with $(l-1)$-gcds and one-hot encoding. Besides MLP
and CNN classifiers, we also employ a random forest classifier here.

Classifier | Random Forest | MLP | CNN
---|---|---|---
Plücker coordinates | 0.764 | 0.706 | 0.746
Augmentation with ($l-1$)-gcds | 0.767 | 0.734 | 0.735
One-hot encoding | 0.770 | 0.790 | 0.813
Table 4.16: The accuracies for classifying reflexivity using different types
of input vectors.

As we can see from Table 4.18, one-hot encoding can give the best performance
to classify reflexivity, especially when using CNN.

##### Plücker coordinates with fixed length

Again let us restrict the lengths of the Plücker coordinates to be 10 and
check whether the performance of the classification can be improved. There are
2410 reflexive samples and we randomly choose same number non-reflexive ones
to make the dataset balanced. The performance of the models at $90\%$ training
can be found in Table 4.17.

Classifier | Random Forest | MLP | CNN
---|---|---|---
Plücker coordinates | 0.712 | 0.724 | 0.718
Augmentation with ($l-1$)-gcds | 0.761 | 0.761 | 0.747
One-hot encoding | 0.707 | 0.713 | 0.724
Table 4.17: The accuracies for classifying reflexivity using different types
of input vectors. The length of Plücker coordinates is fixed to be 10.

Again, let us also check the results with (2410+2410) samples of varying input
lengths for comparison. The accuracies are shown in Table 4.18.

Classifier | Random Forest | MLP | CNN
---|---|---|---
Plücker coordinates | 0.763 | 0.601 | 0.645
Augmentation with ($l-1$)-gcds | 0.746 | 0.674 | 0.668
One-hot encoding | 0.728 | 0.693 | 0.753
Table 4.18: The accuracies for classifying reflexivity using different input
vectors whose corresponding Plücker coordinates have varying lengths. There
are $4338\,(=4820\times 90\%)$ training samples.

As we can see, although MLP and CNN have better performance for fixed input
length, the best results amongst all the models, which are from random forest,
remain roughly the same. Therefore padding 0’s to the input vectors with
varying length does not seem to affect the learning results for reflexivity.

####  4.3.4 From Polygons to Polytopes

So far, we have conducted various tests for both 2d and 3d polytopes. It would
be an interesting question to ask if a trained model for 2d polygons can learn
the same quantities for 3d polytopes with less training data compared to a
model trained with exclusively 3d data.

Let us start with the volumes. We first consider 2d and 3d polytopes having
Plücker coordinates of same length. A good choice would be length equal to 10
because both 2d and 3d polytopes happen to have 5 vertices in this situation
as $\binom{5}{3}=\binom{5}{2}=10$ respectively. We first train our model only
with 12000 2d data points, whose good performance is expected from the
previous results for polygons. Then we further train this model on 4800 3d
data points. It turns out that the model originally trained for 2d data has
successfully learn the 3d data. In fact, this result is just as good as
training 3d data only with fixed length 10 as in Table 4.8. See Table 4.19.

MAE | 0.301
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.825 | 0.977 | 1.000 | 1.000 | 1.000
Table 4.19: The MAE and accuracies for predicting polytope volumes with input
of fixed length 10 using a model pre-trained for 2d data.

As a sanity check, we now train the model only with 2d data and extrapolate it
to 3d data without further fitting any 3d samples. We find that even at
$\text{err}=\pm 5$, the accuracy can only reach 0.007 as one may expect.

Since the results are always good for Plücker coordinates of length 10 no
matter whether we have pre-trained 2d data or not, one may wonder whether pre-
training 2d data would make a difference for varying input lengths. Since the
longest input vector would have length 560 (from 3d data), we pad all the 2d
and 3d data with 0’s to length 560. As padding many 0’s would introduce noise,
we now include more data for training for both 2d and 3d polytopes. Same as
the steps above, we first train the model with 76000 2d samples and then
further fit the model with 45000 3d samples. The results are reported in Table
4.20.

MAE | 2.149
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.152 | 0.302 | 0.570 | 0.751 | 0.860
Table 4.20: The MAE and accuracies for predicting the polytope volumes with input of varying lengths (30000 samples) using the model pre-trained with 2d data (15000 samples). MAE | 2.193
---|---
Error | $\pm 0.5$ | $\pm 1$ | $\pm 2$ | $\pm 3$ | $\pm 4$
Accuracy | 0.148 | 0.295 | 0.550 | 0.748 | 0.861
Table 4.21: The MAE and accuracies for predicting the polytope volumes with
input of varying lengths (30000 samples) without pre-training any 2d data.

Again, the model can still learn the 3d data pretty well. However, if we only
train same amount of 3d data (45000 samples) without pre-training 2d data, the
accuracy would not make a big difference as shown in Table 4.21.

We can also perform the similar test for dual volumes. Again, we find that a
model pre-trained with 2d data gives similar performance as a model without
pre-training does. Although the polytope properties for 2d polygons and 3d
polytopes in terms of Plücker coordinates might share some common ground, the
above results show that different dimensions would still have certain
different features.

###  4.4 The MDS Embedding

We have used different models to predict various properties of the polytopes
with Plücker coordinates. In particular, the performance of predicting volumes
is most impressive amongst them. In fact, the volume $V$ is the sum of a
subset of Plücker coordinates in the input vector, that is,

$V=\sum_{\alpha\in\mathcal{V}}p^{\alpha}$ (4.2)

where $(p^{0},p^{1},\dots,p^{l-1})$ is the Plücker coordinates (of length $l$)
and $\mathcal{V}\subset\\{0,1,\dots,l-1\\}$.

Now let us apply manifold embedding with the help of scikit-learn scikit-learn
and Yellowbrick bengfort_yellowbrick_2019  to understand how the machine
learning models make the predictions and what results we can extract from such
analysis.

It turns out that multi-dimensional scaling (MDS) borg2005modern  leads to
nice distributions of the data777We have also tried PCA and other different
manifold embeddings. However, only MDS could give scattering plots with such
distributions.. See Figure 4.5.

(a)

(b)

Figure 4.5: The (2-component) MDS projections of Plücker coordinates for (a)
polygons and (b) polytopes. Different colours of data points indicate
different volumes. Here, we use 1000 random samples for each plot as an
illustration.

In general, the MDS applied here projects a Plücker vector
$\bm{p}=(p^{0},\dots,p^{l-1})$ to a $k$-dimensional vector
$\bm{x}=(x^{0},\dots,x^{k-1})$ ($k<l$) by minimizing the cost function known
as the stress:

$S=\left(\sum_{i<j\leq N}(D_{ij}-d_{ij})^{2}\right)^{1/2},$ (4.3)

where $N$ is the number of samples and $D_{ij}$ ($d_{ij}$) denotes the
Euclidean distance between $\bm{p}_{i}$ ($\bm{x}_{i}$) and $\bm{p}_{j}$
($\bm{x}_{j}$).

From Figure 4.5, we can see that the vectors $\bm{x}$ are distributed around a
centre point. The closer a vector is to the centre, the smaller volume the
corresponding polytope (either 2d or 3d) would have. Since this is a
mathematical fact, the number of samples should not affect the results and
hence minimizing $S$ is equivalent to the goal of finding $\bm{x}$ such that
$D_{ij}-d_{ij}\approx 0$ (for any $i,j$). Since the positions of the data
points with different colours/volumes mainly depend on the distances/radii
from the origin in Figure 4.5, we may also plot a 1-component MDS projection,
i.e., $\bm{x}=(x^{0})\equiv x$. As shown in Figure 4.6, the points with
different colours/volumes are still determined by their distances to the
origin.

(a)

(b)

Figure 4.6: The _$1$ -component_ MDS projections of Plücker coordinates for
(a) polygons and (b) polytopes. Different colours of data points indicate
different volumes. Here, we use 1000 random samples for each plot as an
illustration.

This seems to hint some possible relation between Plücker coordinates and
polytope volumes.

######  Conjecture 4.1.

There exists a map
$f:\bigcup\limits_{n}G(n-d,n)\rightarrow\mathbb{R},\,\bm{p}\mapsto x$ such
that

$V\sim|x|+1,$ (4.4)

where $V$ is the corresponding polytope volume of the Plücker coordinates
$\bm{p}$, and $1$ indicates the smallest (normalized) volume among the
polytopes.

Recall that $G(n-d,n)$ is the Grassmannian which parametrizes the Plücker
coordinates of $n$-topes in $d$ dimensions. Since the Plücker vectors would
have different lengths, we pad 0’s to make $\bm{p}$ have the same length $l$.

In terms of minimizing $S$, for any two Plücker vectors $\bm{p}$ and $\bm{q}$
with $x=f(\bm{p})$ and $y=f(\bm{q})$,

$D(\bm{p},\bm{q})-d(x,y)=0.$ (4.5)

Hence,

$(|x|-|y|)^{2}=x^{2}+y^{2}-2|xy|=\sum_{\alpha=1}^{l}((p^{\alpha})^{2}+(q^{\alpha})^{2}-2p^{\alpha}q^{\alpha}).$
(4.6)

Since $x=f(\bm{p})$ and $y=f(\bm{q})$, the crossing terms
$p^{\alpha}q^{\alpha}$ on the right hand side should only come from the
crossing term $|xy|$ on the left hand side. Therefore, by (4.2), we have

$\begin{split}&x^{2}=\left(\sum_{\alpha\in\mathcal{V}_{\bm{p}}}p^{\alpha}\right)^{2}=\sum_{\alpha\in\mathcal{V}_{\bm{p}}}(p^{\alpha})^{2}+\sum_{\alpha\neq\beta\in\mathcal{V}_{\bm{p}}}p^{\alpha}p^{\beta},\\\
&|xy|=\sum_{\alpha=1}^{l}p^{\alpha}q^{\alpha}+\sum_{\alpha\neq\beta\in\mathcal{V}_{\bm{p}}}p^{\alpha}p^{\beta}+\sum_{\alpha\neq\beta\in\mathcal{V}_{\bm{q}}}q^{\alpha}q^{\beta}-\sum_{\alpha\neq\beta\not\in\mathcal{V}_{\bm{p}}}p^{\alpha}p^{\beta}-\sum_{\alpha\neq\beta\not\in\mathcal{V}_{\bm{q}}}q^{\alpha}q^{\beta}.\end{split}$
(4.7)

Overall, the MDS embedding hints some non-trivial relation between Plücker
coordinates and volumes as in Figure 4.5 and Figure 4.6. It would also be
interesting to study its connection to the Plücker relations.

##### A comment on dual volumes

We may also apply MDS to dual volumes. As shown in Figure 4.7, although data
points with larger dual volumes tend to lie closer to the centres in the plots
(as opposed to volumes), the ascending of dual volumes from centres to
peripheries is not comparable to the case for volumes.

(a)

(b)

Figure 4.7: The MDS projections of Plücker coordinates for (a) polygons and
(b) polytopes. Different colours of data points indicate different dual
volumes. Here, we use 1000 random samples for each plot as an illustration.

##  5 Summary & Outlook

In this paper, we explored the machine learning for polytopes in terms of
their Plücker coordinates. Here, we summarize the main results in Table 5.1.

These results indicate that it could be possible to find certain relations,
albeit very non-trivial, between Plücker coordinates and various properties of
the polytopes. In particular, some of the results show much better performance
using Plücker coordinates than using the vertices. From Figure 4.6, we also
found that the volumes have nice distributions under MDS embedding for Plücker
coordinates. This indicates possible relations between volumes and Plücker
coordinates. It is then natural to ask what the exact relation is and whether
this could be connected to our current knowledge on Plücker coordinates such
as Plücker relations etc. Moreover, it would be important to see how neural
network would perform when predicting the MDS-projected data from Plücker
coordinates as well as predicting volumes from such data MLKS .

It is also natural to expect that the above study could be extended to
polytopes in other dimensions. For instance, in 4d, the Kreuzer-Skarke (KS)
programme of studying Calabi-Yau manifolds as hypersurfaces (and, more
generally, complete intersections) in toric varieties from reflexive polytopes
is still a very active area of research Braun:2012vh ; Altman:2014bfa ;
He:2015fif ; He:2017gam ; Altman:2018zlc ; Demirtas:2018akl ; Demirtas:2020dbm
. The important role played by the KS database in both mathematics and physics
makes it a particularly interesting future direction for this research.

4  | Polygons
---
(frequency)
| Triangles
---
(277)
| Quadrilaterals
---
(7041)
| Pentagons
---
(16637)
| Hexagons
---
(3003)
4 Input | Vertices
Output | Volume
MAE | 4.941 | 10.012 | 8.640 | 14.947
Accuracy | 0.945 | 0.945 | 0.926 | 0.826
4 Input | Plücker coordinates
Output | Volume
MAE | 0.209 | 0.625 | 1.051 | 3.359
Accuracy | 1.000 | 1.000 | 1.000 | 0.997
4 Input | Plücker coordinates
Output | Dual Volume
MAE | 1.181 | 0.642 | 0.818 | 0.941
Accuracy | 0.501 | 0.754 | 0.638 | 0.557
4 Input | ($\ell$-1)-Plücker coordinates & Volume
Output | The missing Plücker coordinate
MAE | 0.295 | 2.599 | 2.061 | 3.169
Accuracy | 1.000 | 0.983 | 0.994 | 0.971
4  | Polytopes
---
(frequency)
| Toric canonical Fano polytopes
---
(780000)
Input | Plücker coordinates
Output | Volume | Dual Volume | Reflexivity
MAE | 1.680 | 2.590 | N/A
Accuracy | 0.936 | 0.890 | 0.813
4 |  |  |  |
Table 5.1: Summary of the key machine learning results. Notice that for
polygons with Plücker coordinates, the “Accuracy” in the table is the accuracy
($\pm 0.05\times\text{range}$). Likewise, for those involving polytopes with
Plücker coordinates, the “Accuracy” in the table is the accuracy ($\pm 4$).
For reflexivity the result is simply the accuracy itself since this is a
binary classification. Moreover, the input used for classifying reflexivity is
one-hot encoding of Plücker coordinates for the result listed here.

## Acknowledgement

JB is supported by a CSC scholarship. YHH would like to thank STFC for grant
ST/J00037X/1. EH would like to thank STFC for a PhD studentship. JH is
supported by a Nottingham Research Fellowship. AK is supported by EPSRC
Fellowship EP/N022513/1. SM is funded by a SMCSE Doctoral Studentship. This
collaboration was made possible by a Focused Research Workshop grant from the
Heilbronn Institute for Mathematical Research.

## References

  * (1) E. Ehrhart, “Sur un probleme de géométrie diophantienne linéaire ii,” J. reine angew. Math 227 no. 25, (1967) C49.
  * (2) U. Betke and P. McMullen, “Lattice points in lattice polytopes,” Monatshefte für Mathematik 99 (1985) 253–265.
  * (3) H. Minkowski, Geometrie der Zahlen.  Geometrie der Zahlen. B.G. Teubner, 1910.  https://books.google.co.uk/books?id=MusGAAAAYAAJ.
  * (4) B. Poonen and F. Rodriguez-Villegas, “Lattice polygons and the number 12,” The American Mathematical Monthly 107 no. 3, (2000) 238–250.
  * (5) D. Cox, J. Little, and H. Schenck, Toric Varieties.  Graduate studies in mathematics. American Mathematical Soc., 2011.  https://books.google.co.uk/books?id=eXLGwYD4pmAC.
  * (6) A. M. Kasprzyk and B. Nill, “Reflexive polytopes of higher index and the number 12,” Electron. J. Combin. 19 no. 3, (2012) Paper 9, 18\.
  * (7) V. Batyrev and J. Hofscheier, “A generalization of a theorem of white,” arXiv:1004.3411 [math.CO]. To appear in Mosc. J. Comb. Number Theory.
  * (8) V. Batyrev and J. Hofscheier, “Lattice polytopes, finite abelian subgroups in $\mathrm{SL}(n,\mathbb{C})$ and coding theory,” arXiv:1309.5312 [math.CO].
  * (9) V. V. Batyrev and L. A. Borisov, On Calabi-Yau complete intersections in toric varieties, pp. 39–66.  De Gruyter, 2011.  https://doi.org/10.1515/9783110814736.39.
  * (10) M. Kreuzer and H. Skarke, “On the classification of reflexive polyhedra,” Commun. Math. Phys. 185 (1997) 495–508, arXiv:hep-th/9512204.
  * (11) M. Kreuzer and H. Skarke, “Classification of reflexive polyhedra in three-dimensions,” Adv. Theor. Math. Phys. 2 (1998) 853–871, arXiv:hep-th/9805190.
  * (12) M. Kreuzer and H. Skarke, “Complete classification of reflexive polyhedra in four-dimensions,” Adv. Theor. Math. Phys. 4 (2002) 1209–1230, arXiv:hep-th/0002240.
  * (13) A. Rebhan, L. Katzarkov, and J. Knapp, Strings, Gauge Fields, and the Geometry Behind: The Legacy of Maximilian Kreuzer.  World Scientific, 2013.
  * (14) T. Hibi and A. Tsuchiya, Algebraic And Geometric Combinatorics On Lattice Polytopes - Proceedings Of The Summer Workshop On Lattice Polytopes.  World Scientific, 2019.
  * (15) Y.-H. He, “Deep-Learning the Landscape,” Phys. Lett. B 774 (6, 2017) 564–568, arXiv:1706.02714 [hep-th].
  * (16) D. Krefl and R.-K. Seong, “Machine Learning of Calabi-Yau Volumes,” Phys. Rev. D 96 no. 6, (2017) 066014, arXiv:1706.03346 [hep-th].
  * (17) F. Ruehle, “Evolving neural networks with genetic algorithms to study the String Landscape,” JHEP 08 (2017) 038, arXiv:1706.07024 [hep-th].
  * (18) J. Carifio, J. Halverson, D. Krioukov, and B. D. Nelson, “Machine Learning in the String Landscape,” JHEP 09 (2017) 157, arXiv:1707.00655 [hep-th].
  * (19) Y.-H. He, “Machine-learning the string landscape,” Phys. Lett. B 774 (2017) 564–568.
  * (20) J. Bao, Y.-H. He, E. Hirst, and S. Pietromonaco, “Lectures on the Calabi-Yau Landscape,” arXiv:2001.01212 [hep-th].
  * (21) Y.-H. He, “The Calabi-Yau Landscape: from Geometry, to Physics, to Machine-Learning; to appear, Springer LNM,” arXiv:1812.02893 [hep-th].
  * (22) Y.-H. He, “Machine-Learning Mathematical Structures,” arXiv:2101.06317 [cs.LG].
  * (23) J. Bao, Y.-H. He, E. Hirst, J. Hofscheier, A. Kasprzyk, and S. Majumder, “Hilbert Series, Machine Learning, and Applications to Physics,” arXiv:2103.13436 [hep-th].
  * (24) Y.-H. He, E. Hirst, and T. Peterken, “Machine-learning dessins d’enfants: explorations via modular and Seiberg–Witten curves,” J. Phys. A 54 no. 7, (2021) 075401, arXiv:2004.05218 [hep-th].
  * (25) J. Bao, S. Franco, Y.-H. He, E. Hirst, G. Musiker, and Y. Xiao, “Quiver Mutations, Seiberg Duality and Machine Learning,” Phys. Rev. D 102 no. 8, (2020) 086013, arXiv:2006.10783 [hep-th].
  * (26) J. Bao, Y.-H. He, and E. Hirst, “Neurons on Amoebae,” arXiv:2106.03695 [math.AG].
  * (27) L.-A. Gottlieb, E. Kaufman, A. Kontorovich, and G. Nivasch, “Learning convex polytopes with margin,” in NeurIPS.  2018\.  arXiv:1805.09719 [cs.LG].
  * (28) N. Goyal, “The polytope learning problem, talk @ simons workshop on “computational challenges in machine learning”,” May 1 – 5, 2017.  https://simons.berkeley.edu/workshops/machinelearning2017-3.
  * (29) D. Sommerville, Introduction to the Geometry of N Dimensions.  Dover Books on Mathematics. Dover Publications, 2020.  https://books.google.co.uk/books?id=4vXDDwAAQBAJ.
  * (30) A. M. Kasprzyk, “Canonical toric fano threefolds,” Canadian Journal of Mathematics 62 no. 6, (Dec, 2010) 1293–1309. http://dx.doi.org/10.4153/CJM-2010-070-3.
  * (31) A. M. Kasprzyk and B. Nill, “Fano polytopes,” in Strings, gauge fields, and the geometry behind, pp. 349–364.  World Sci. Publ., Hackensack, NJ, 2013.
  * (32) G. Balletti, A. M. Kasprzyk, and B. Nill, “On the maximum dual volume of a canonical fano polytope,” arXiv:1611.02455 [math.CO].
  * (33) V. Batyrev, A. Kasprzyk, and K. Schaller, “On the Fine Interior of Three-dimensional Canonical Fano Polytopes,” arXiv:1911.12048 [math.AG].
  * (34) A. A. Borisov and L. A. Borisov, “Singular toric fano varieties,” Matematicheskii Sbornik 183 no. 2, (1992) 134–141.
  * (35) A. Borisov, “Convex lattice polytopes and cones with few lattice points inside, from a birational geometry viewpoint,” arXiv:math/0001109.
  * (36) A. M. Kasprzyk, “Canonical toric fano threefolds,” Canadian Journal of Mathematics 62 no. 6, (2010) 1293–1309.
  * (37) Y.-H. He, R.-K. Seong, and S.-T. Yau, “Calabi–Yau Volumes and Reflexive Polytopes,” Commun. Math. Phys. 361 no. 1, (2018) 155–204, arXiv:1704.03462 [hep-th].
  * (38) D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412.6980 [cs.LG].
  * (39) F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research 12 (2011) 2825–2830.
  * (40) B. Bengfort and R. Bilbro, “Yellowbrick: Visualizing the Scikit-Learn Model Selection Process,”. http://joss.theoj.org/papers/10.21105/joss.01075.
  * (41) I. Borg and P. J. Groenen, Modern multidimensional scaling: Theory and applications.  Springer Science & Business Media, 2005.
  * (42) J. Bao, Y.-H. He, E. Hirst, J. Hofscheier, A. Kasprzyk, and S. Majumder, “Machine Learning the Kreuzer-Skarke Landscape of Reflexive $4$-Polytopes.”. To appear.
  * (43) A. P. Braun, J. Knapp, E. Scheidegger, H. Skarke, and N.-O. Walliser, PALP - a User Manual.  5, 2012.  arXiv:1205.4147 [math.AG].
  * (44) R. Altman, J. Gray, Y.-H. He, V. Jejjala, and B. D. Nelson, “A Calabi-Yau Database: Threefolds Constructed from the Kreuzer-Skarke List,” JHEP 02 (2015) 158, arXiv:1411.1418 [hep-th].
  * (45) Y.-H. He, V. Jejjala, and L. Pontiggia, “Patterns in Calabi–Yau Distributions,” Commun. Math. Phys. 354 no. 2, (2017) 477–524, arXiv:1512.01579 [hep-th].
  * (46) R. Altman, J. Carifio, J. Halverson, and B. D. Nelson, “Estimating Calabi-Yau Hypersurface and Triangulation Counts with Equation Learners,” JHEP 03 (2019) 186, arXiv:1811.06490 [hep-th].
  * (47) M. Demirtas, C. Long, L. McAllister, and M. Stillman, “The Kreuzer-Skarke Axiverse,” JHEP 04 (2020) 138, arXiv:1808.01282 [hep-th].
  * (48) M. Demirtas, L. McAllister, and A. Rios-Tascon, “Bounding the Kreuzer-Skarke Landscape,” arXiv:2008.01730 [hep-th].