# Accelerated Discovery of Machine-Learned Symmetries:
Deriving the Exceptional Lie Groups $G_{2}$, $F_{4}$ and $E_{6}$

Roy T. Forestano111All authors share equal contributions to this paper.
Konstantin T. Matchev222All authors share equal contributions to this paper.
Katia Matcheva333All authors share equal contributions to this paper.
Alexander Roman444All authors share equal contributions to this paper. Eyup B.
Unlu555All authors share equal contributions to this paper. Sarunas
Verner666All authors share equal contributions to this paper.

###### Abstract

Recent work has applied supervised deep learning to derive continuous symmetry
transformations that preserve the data labels and to obtain the corresponding
algebras of symmetry generators. This letter introduces two improved
algorithms that significantly speed up the discovery of these symmetry
transformations. The new methods are demonstrated by deriving the complete set
of generators for the unitary groups $U(n)$ and the exceptional Lie groups
$G_{2}$, $F_{4}$, and $E_{6}$. A third post-processing algorithm renders the
found generators in sparse form. We benchmark the performance improvement of
the new algorithms relative to the standard approach. Given the significant
complexity of the exceptional Lie groups, our results demonstrate that this
machine-learning method for discovering symmetries is completely general and
can be applied to a wide variety of labeled datasets.

††journal: Physics Letters B

[UF]organization=Institute for Fundamental Theory, Physics Department,
University of Florida,city=Gainesville, state=FL, postcode=32611, country=USA

##  1 Introduction

The beginning of the 20th century significantly changed theoretical physics.
It became evident that the laws of nature are closely tied to principles of
symmetry [1]. Emmy Noether taught us that a continuous symmetry within any
physical system inherently implies a conservation law, providing profound
insights about the system [2]. This concept has deepened our understanding of
fundamental physics, from the simplest principles of classical mechanics to
the intricacies of the Universe at large. In particle physics, these
symmetries serve as the foundation that organizes the particles and their
associated interactions. They are instrumental in guiding theoretical
physicists in exploring possible extensions of the Standard Model [3, 4].

Group theory has traditionally served as the framework for studying symmetries
[5]. Among the numerous types of classical Lie groups, the special orthogonal
groups $SO(n)$ and the special unitary groups $SU(n)$ are most commonly used
in particle physics. While some classical Lie groups describe symmetries
observed in nature, the exceptional Lie groups [6] open up new possibilities
for theoretical physics, potentially enabling us to describe new kinds of
gauge theories, including grand unified theories (GUTs) [7].

There exist five exceptional Lie groups — $G_{2}$, $F_{4}$, $E_{6}$, $E_{7}$,
and $E_{8}$, all of which have found various applications in theoretical
physics [8]. For example, the smallest among them, $G_{2}$, has been widely
used in string theory, e.g., in the studies of seven-dimensional manifolds
known as $G_{2}$-manifolds, used in M-theory compactifications [9, 10, 11,
12]. The next exceptional group, $F_{4}$, plays an important role in Jordan
algebra theory [13] and has also been used in the context of gauge theories
[14, 15]. The exceptional Lie group $E_{6}$ has been broadly employed in GUTs
[16, 17]. The two remaining exceptional groups, $E_{7}$ and $E_{8}$, have also
been considered as potential GUT candidates [18, 19].

The use of machine learning (ML) to uncover and recognize symmetries in
datasets has recently sparked considerable interest [20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31]. Specific applications to group theory include
calculating tensor products and branching rules of irreducible representations
of Lie groups [32] or testing for the presence of a conjectured Lie group
symmetry in the data [25, 27]. More recent work has focused on discovering
from first principles the Lie group generators reflecting symmetries in the
data [28, 29, 30, 31]. In this Letter, we introduce new algorithms that
significantly enhance the symmetry discovery process outlined in [28, 29, 30,
31]. After introducing the problem in Section 2, in Section 3 we describe the
new methods and demonstrate their advantages on the unitary groups $U(n)$.
Then in Sections 4, 5 and 6 we consecutively consider three of the five
exceptional Lie groups: $G_{2}$, $F_{4}$, and $E_{6}$. We derive the complete
set of generators, in sparse form, which preserve the respective polynomial
invariants. Our approach is completely general, and could be easily extended
to the remaining two exceptional Lie groups $E_{7}$ and $E_{8}$. Section 7 is
reserved for our summary.

##  2 Problem Description

The classical groups are the linear groups of transformations over the real
numbers $\mathbb{R}$, the complex numbers $\mathbb{C}$, and quaternions
$\mathbb{H}$. Consequently, a symmetry transformation operates on a feature
vector ${\mathbf{x}}\equiv\\{x^{(1)},x^{(2)},\ldots,x^{(n)}\\}$, where
${\mathbf{x}}\in\mathbb{R}^{n}$ (${\mathbf{x}}\in\mathbb{C}^{n}$) for real
(complex) representations. To encapsulate the effect of a group transformation
on ${\mathbb{R}}^{n}$ or ${\mathbb{C}}^{n}$, we examine a representative set
of $m$ points
$\left\\{\mathbf{x}\right\\}\equiv\left\\{\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{m}\right\\}$
sampled from a finite domain. The selection of a sampling distribution, as
well as the size and location of the domain, are inconsequential. We adopt a
standard normal distribution and pick $m$ on the order of several hundred,
depending on the complexity of the problem.

The classical groups can be defined in terms of polynomial invariants over
their respective fields. For example, the orthogonal group $O(n)$ preserves
the values of the polynomial oracle,

$\varphi_{O}(\mathbf{x})\;\equiv\;|\mathbf{x}|^{2}=\sum_{j=1}^{n}\bigl{(}x^{(j)}\bigr{)}^{2},\quad
x^{(j)}\in\mathbb{R}\,,$ (1)

the Lorentz group in $n=4$ dimensions preserves

$\varphi_{L}(\mathbf{x})\equiv\bigl{(}x^{(1)}\bigr{)}^{2}-\bigl{(}x^{(2)}\bigr{)}^{2}-\bigl{(}x^{(3)}\bigr{)}^{2}-\bigl{(}x^{(4)}\bigr{)}^{2},~{}~{}x^{(j)}\in\mathbb{R}\,,$
(2)

and the unitary group $U(n)$ preserves

$\varphi_{U}(\mathbf{x})\;\equiv\;\sum_{j=1}^{n}\bigl{(}x^{(j)}\bigl{)}^{\ast}x^{(j)},\quad
x^{(j)}\in\mathbb{C}\,.$ (3)

A symmetry transformation $\mathbf{f}$ is a map
$\mathbf{x}^{\prime}=\mathbf{f}(\mathbf{x})$ that preserves the respective
oracle (1-3) everywhere, or in our case, for each of the sampled $m$ points:

$\varphi(\mathbf{x}^{\prime}_{i})\;\equiv\;\varphi(\mathbf{f}(\mathbf{x}_{i}))=\varphi(\mathbf{x}_{i}),\quad\forall
i\,=\,1,2,\ldots,m\,.$ (4)

The basic task is to find such a symmetry map $\mathbf{f}$ in parametric form.
To focus on the generators of the symmetry transformation group, $\mathbf{f}$
is linearized by considering infinitesimal transformations
$\delta{\mathbf{f}}$ in the vicinity of the identity transformation
$\mathbb{I}$:

$\delta{\mathbf{f}}\;\equiv\;\mathbb{I}+\varepsilon\,{\mathbb{G}}\,,$ (5)

where $\varepsilon$ is an infinitesimal parameter and ${\mathbb{G}}$ is an
$n\times n$ matrix. After training with the invariance loss function
$L_{\text{inv}}({\mathbb{G}},\\{\mathbf{x}\\})$ defined in (20), the
components of ${\mathbb{G}}$ evolve towards their trained values, thereby
producing a valid symmetry generator [29]

$\mathbb{J}\;\equiv\;\operatorname*{arg\,min}_{\mathbb{G}}\Bigl{(}L({\mathbb{G}},\\{\mathbf{x}\\})\Bigr{)}.$
(6)

Figure 1: Flowchart illustrating the algorithms discussed in Section 3 (left
side) and the algorithms from Refs. [29] and [31] (right side).

As shown in the flowchart of Figure 1, this idea was used in [29] to
simultaneously train $N_{g}$ generators, which were designed to be normalized,
orthogonal to each other, and to form a closed algebra, by adding suitable
terms to the loss function. However, as indicated in Figure 1, the resulting
set of $N_{g}$ symmetry generators $\\{\mathbb{J}_{\alpha}\\}$ was not sparse.
An improvement was implemented in [31], by adding an extra term to the loss
function, Eq. (23), which encourages the learning of sparse generators in the
canonical textbook form.

##  3 Iterative Constructions of Symmetry Generators

The advantage of learning $N_{g}$ symmetry generators in one go is that one
can impose the closure condition in the loss function and thus guarantee that
the learned set $\\{\mathbb{J}_{\alpha}\\}$ is closed. However, there are
drawbacks to this approach as well. First, due to the large number of
trainable parameters ($\sim n^{2}\times N_{g}$), the training is challenging
and becomes prohibitively slow for large groups (large $N_{g}$) and/or high
dimensions (large $n$). For this reason, in Section 3.1 we explore an
alternative “greedy” approach, which trains one generator at a time. Secondly,
a closed algebra of generators only exists for certain values of $N_{g}$,
which are a priori unknown, and with the previous algorithms, would have to be
guessed by trial and error. In contrast, the greedy approach automatically
finds the largest possible closed set of generators. Furthermore, we can use
the closure condition to find additional generators directly, without any
training. We refer to this last method as the “Lie bracket trick” (LBT), which
is described in Section 3.2. In principle, both of the two new methods could
accommodate the sparsity condition (23). However, we find that the training is
more efficient when we look for nonsparse generators, therefore we postpone
their sparsification to a postprocessing step which we describe in Section
3.3.

1 Parameters: $\lambda,L_{min},N_{epochs}$;

2 $\\{\mathbb{J}\\}$ $\leftarrow$ $[]$;

3 $\mathcal{W}\leftarrow\mathcal{W}_{initial}\sim{\mathcal{N}}$;

4 for  _i from 1 to $N_{epochs}$_ do

5

6 L $\leftarrow
L_{\text{greedy}}(\mathbb{G}({\mathcal{W}}),\\{\mathbb{J}\\},\mathbf{x})$ ;

7 if  _$L <L_{min}$_ then

8 append $\mathbb{G}(\mathcal{W})$ to $\\{\mathbb{J}\\}$;

9 goto 3;

10 end if

11 $\mathcal{W}$
$\leftarrow\mathcal{W}-\lambda\nabla_{\mathcal{W}}L_{\text{greedy}}$;

12

13 end for

stop

Algorithm 1 The greedy algorithm.

###  3.1 Greedy Algorithm

The basic idea of the greedy algorithm is illustrated in Figure 1 and the
corresponding pseudocode is shown in Algorithm 1. We use Eq. (6) to train one
generator $\mathbb{G}(\mathcal{W})$ at a time, where $\mathcal{W}$ denotes the
trainable parameters of the matrix $\mathbb{G}$. The generator $\mathbb{G}$ is
required to i) preserve the oracle, ii) be normalized, and iii) be orthogonal
to the set of generators $\\{\mathbb{J}_{\alpha}\\}$ already found so far.
Therefore, the loss function is

$L_{\text{greedy}}(\mathbb{G}(\mathcal{W}),\\{\mathbb{J}\\},\\{\mathbf{x}\\})=L_{\text{inv}}+L_{\text{norm}}+L_{\text{ortho}},$
(7)

where $L_{\text{inv}}$, $L_{\text{norm}}$ and $L_{\text{ortho}}$ are defined
in (20), (21) and (22), respectively.

As shown in Figure 1, at the start of the algorithm the learned set
$\\{\mathbb{J}\\}$ is empty ($i=0$). At this point, the orthogonality loss
$L_{\text{ortho}}$ is not applicable and is turned off. The minimization in
Eq. (6) will yield a single new symmetry generator
$\mathbb{J}_{\text{new}}=\mathbb{J}_{1}$, the first to be added to the learned
set $\\{\mathbb{J}\\}$. Now the orthogonality loss $L_{\text{ortho}}$ is
turned on and the process continues until the algorithm fails to find a new
valid symmetry generator $\mathbb{J}_{\text{new}}$. The resulting set
$\\{\mathbb{J}_{1},\ldots,\mathbb{J}_{N_{g}}\\}$ is the largest orthogonal set
of nontrivial symmetry generators. For the orthogonal groups $O(n)$ with the
oracle (1), this procedure terminates when $N_{g}=n(n-1)/2$, while for the
unitary groups $U(n)$ with the oracle (3), it stops at $N_{g}=n^{2}$.

###  3.2 Lie Bracket Trick

Algorithm 2 shows the pseudocode for the Lie bracket trick algorithm, which
leverages the existing group structure among the symmetry generators. As shown
with the left side branch in Figure 1, the LBT can be (optionally) applied in
conjunction with the greedy algorithm, once two orthogonal generators,
$\mathbb{J}_{1}$ and $\mathbb{J}_{2}$, have been found. At that point, one can
compute their commutator (Lie bracket)
$\left[\mathbb{J}_{1},\mathbb{J}_{2}\right]$, with three possible outcomes.
First, if the commutator is zero, we gain nothing from the LBT and must return
to the greedy algorithm. If, however, the chosen pair does not commute, the
result may include a component outside the span of the generators discovered
so far, which would lead to a new valid symmetry generator that we can extract
for free. To isolate this component, we employ Gram-Schmidt orthogonalization
(line 11 in Algorithm 2). If the result is zero, we are again out of luck and
must return to the greedy method. However, if the out-of-span component is
nonzero, it becomes (after the normalization in line 13) a new generator which
can be added to the set $\\{J\\}$. This process can be repeated for every pair
of known generators, including the newly found ones via the LBT method. The
LBT algorithm terminates when all possible Lie brackets close in the current
set $\\{\mathbb{J}\\}$. We note that when using the two algorithms 1 and 2
together, one should choose judiciously the respective values of the loss
thresholds $L_{min}$.

1 Input: $\\{J_{1},\ldots,J_{i}\\}$: known algebra; $J_{i+1}$: new generator;

2 append $J_{i+1}$ to $G$;

3 repeat

4 $k\leftarrow|G|$;

5 $i\leftarrow|J|$;

6 append G to J;

7 clear G;

8 for  _p from 1 to i_ do

9 for  _q from i+1 to i+k_ do

10 C $\leftarrow J_{p}J_{q}-J_{q}J_{p}$;

11 C $\leftarrow\text{C}-\sum_{g\in{J}}\frac{g}{||g||}\times(\text{C}\cdot
g)$;

12 if  _$||C||\neq 0$_ then

13 C $\leftarrow\frac{\text{C}}{||\text{C}||}$;

14 if  _$L_{inv}(C,{\bf{x}}) <L_{min}$_ then

15 append C to G;

16

17 end if

18

19 end if

20

21 end for

22

23 end for

24

25until  _$|G|=0$_ ;

Algorithm 2 The Lie bracket trick (LBT) algorithm.

###  3.3 Sparsification

Suppose we have already found a set of $N_{g}$ generators $J_{\alpha}$,
$\alpha=1,2,\ldots,N_{g}$. We can transform them to a new sparse basis,
$\tilde{J}_{\alpha}$, by rotating with an orthogonal $N_{g}\times N_{g}$
matrix $O$,

$\tilde{J}_{\alpha}(O)=O_{\alpha\beta}J_{\beta}.$ (8)

In analogy to (23), the loss function for the sparsification of the generators
can be defined as

$\displaystyle
L_{\text{sp}}(O)=\sum_{\alpha=1}^{N_{g}}\sum_{\begin{subarray}{c}j,j^{\prime}=1\\\
k,k^{\prime}=1\end{subarray}}^{n}\left|\tilde{J}_{\alpha}^{(jk)}(O)\tilde{J}_{\alpha}^{(j^{\prime}k^{\prime})}(O)\right|\left(1-\delta_{jj^{\prime}}\delta_{kk^{\prime}}\right).$
(9)

It takes into account all possible pairs of entries in each transformed
generator $\tilde{J}_{\alpha}$. We minimize over this loss to obtain the
desired orthogonal transformation $O$, and subsequently, apply this
transformation to the original generators $J_{\alpha}$ to find the sparse
generators $\tilde{J}_{\alpha}$ as in (8).

###  3.4 Timing tests

Figure 2: Comparison of the running times of the different algorithms in
finding the full algebras of the $U(n)$ family: the standard algorithm [31]
(blue squares), the greedy algorithm 1 (orange diamonds) and the Lie bracket
trick algorithm 2 (green circles).

The main advantage of the two new algorithms is that they significantly speed
up the training procedure. To quantify this improvement, in Figure 2 we
present the results from timing tests on a personal laptop for different
$U(n)$ groups, using the three approaches discussed earlier: the standard
algorithm [31] (blue squares), the greedy algorithm 1 (orange diamonds) and
the LBT algorithm 2 (green circles). The plot shows the time in seconds that
it took to learn all $n^{2}$ generators for the $U(n)$ group, as a function of
the dimension $n$, at a learning rate $2.5\times 10^{-2}$. The generators were
found in sparse form by applying the post-processing step from Section 3.3,
whose duration was included in the total time shown in Figure 2. We observe
that for small $n$ the performance of all three methods is comparable, but for
large $n$ the new methods offer significant improvement. In particular, for
$n\sim 10$ the standard method would require training for days, while the new
methods reduce the training time to less than a minute.

##  4 The Exceptional Group $G_{2}$

The ML approach described in the previous two sections can be used to discover
the orthogonal groups $O(n)$ [29] and the unitary groups $U(n)$ [31]. The
method can also be generalized to the case of vector (i.e., multicomponent)
oracles [30]. We shall now apply it to exceptional (non-classical) Lie
algebras, which have relatively large number of generators, and would benefit
from the speed-up offered by the new algorithms. We consider three of the five
exceptional Lie groups: $G_{2}$ in this section, $F_{4}$ in Section 5, and
$E_{6}$ in Section 6. These exceptional Lie groups have found various
applications in high energy physics in the context of gauge theories and model
building [16, 33, 14, 8, 34, 35, 36, 37, 38, 39]. Given the significant
mathematical complexity of these groups, our ability to successfully derive
their algebras attests to the robustness and generality of our ML techniques.

The smallest exceptional Lie group is the $G_{2}$ group, which has rank $2$
and dimension $14$. This group emerges as the automorphism group of the
octonion algebra [40]. An octonion $\mathbf{o}$ is a linear combination

$\mathbf{o}=\sum_{i=0}^{7}x^{(i)}e_{i}$ (10)

of the unit octonions $e_{i}$ with real coefficients $x^{(i)}$. Here $e_{0}$
is the real element which obeys $e_{0}^{2}=+1$ and can be identified with the
real number 1. The remaining $e_{1},\ldots,e_{7}$ are the seven imaginary unit
octonions which obey $e_{i}^{2}=-1$. Their multiplication rules can be
visualized in the Fano plane of Figure 3. The figure shows only one of 480
possible definitions for octonion multiplication with $e_{0}=1$; the other
definitions are isomorphic and can be obtained by permuting and/or changing
the signs of the imaginary basis elements.

Figure 3: Fano plane illustrating the multiplication rules for the imaginary
unit octonions $e_{1},\ldots,e_{7}$ in our conventions. For each triple
$e_{i}$, $e_{j}$ and $e_{k}$ connected by a solid line, the result of the
multiplication $e_{i}e_{j}$ is equal to $+e_{k}$ ($-e_{k}$) when going along
(against) the arrows. The dashed lines have been added to guide the eye in
following each triplet cycle.

The group $G_{2}$ has a fundamental representation of dimension 7, hence in
this section $\mathbf{x}\in\mathbb{R}^{7}$. The components of the feature
vector $\mathbf{x}$ will be identified with the coefficients $x^{(i)}$,
$i=1,\ldots,7$, of the imaginary unit octonions in (10). Correspondingly, the
generators $\\{J\\}$ will be real $7\times 7$ matrices.

In general, the exceptional groups preserve $K$ vector oracles
$\varphi^{(1)},\ldots,\varphi^{(K)}$, where each component $\varphi^{(i)}$
represents an invariant polynomial characteristic of the group. The group
$G_{2}$ preserves $K=2$ such polynomials. The first one is the norm of a
purely imaginary octonion,

$\varphi_{G_{2}}^{(1)}(\mathbf{x})=\sum_{i=1}^{7}\bigl{(}x^{(i)}\bigr{)}^{2}.$
(11)

To define the second oracle, we need to introduce the real part of the product
of three octonions $\mathbf{o}_{1}$, $\mathbf{o}_{2}$ and $\mathbf{o}_{3}$

$\real(\mathbf{o}_{1}\mathbf{o}_{2}\mathbf{o}_{3}\bigr{)}=\sum_{i,j,k=0}^{7}\mathcal{D}_{ijk}\,x_{1}^{(i)}\,x_{2}^{(j)}\,x_{3}^{(k)}.$
(12)

Figure 4: A pictorial visualization of the rank three tensor
$\mathcal{D}_{ijk}$ defined in Eq. (12). Each plane represents a slice at a
fixed $i=0,1,\ldots,7$ (from left to right). The blue, red and grey boxes
indicate coefficient values of $+1$, $-1$ and $0$, respectively.

Figure 5: The fourteen $G_{2}$ generators learned with the greedy method (top
panels) and the result from their sparsification (bottom panels). In this and
all subsequent such figures, each panel represents a learned generator
${\mathbb{J}}_{\alpha}$ in matrix form, where the values of the individual
elements of the matrix are indicated by the color bar.

Using the multiplication rules from Figure 3, the components of the rank three
tensor $\mathcal{D}_{ijk}$ can be easily derived and are shown in Figure 4.
The group $G_{2}$ preserves the real component of the product of three purely
imaginary octonions, hence the second $G_{2}$ oracle is (note that the sums
start from 1 instead of 0)

$\varphi^{(2)}_{G_{2}}({\bf x}_{1},{\bf x}_{2},{\bf
x}_{3})=\sum_{i,j,k=1}^{7}\mathcal{D}_{ijk}\,x_{1}^{(i)}\,x_{2}^{(j)}\,x_{3}^{(k)}.$
(13)

For the training of the $G_{2}$ generators, we use $m=900$ samples
(7-dimensional vectors in $\mathbb{R}^{7}$) and fix $\varepsilon=10^{-3}$. For
the computation of the second oracle (13) we split the sample into three
equally sized groups of $300$, from which we draw the three vectors
$\mathbf{x}_{1}$, $\mathbf{x}_{2}$ and $\mathbf{x}_{3}$. The training was done
with the Adam optimizer [41] for 1,000 epochs and with learning rate of
$2.5\times 10^{-2}$.

Our results for the learned $G_{2}$ generators are shown in Figure 5. The
panels in the top two rows depict the 14 generators found by the greedy
algorithm (as expected, the algorithm failed to find a valid 15th generator,
where even after 10,000 epochs, the loss stayed of order 1). The panels in the
bottom two rows show the corresponding results after applying the
sparsification procedure of Section 3.3. Note that all found generators are
antisymmetric ($G_{2}$ is a subgroup of $SO(7)$), and that they can be
organised into 7 pairs which share common matrix elements: $\mathbb{J}_{1}$
and $\mathbb{J}_{9}$, $\mathbb{J}_{2}$ and $\mathbb{J}_{7}$, $\mathbb{J}_{3}$
and $\mathbb{J}_{4}$, $\mathbb{J}_{5}$ and $\mathbb{J}_{10}$, $\mathbb{J}_{6}$
and $\mathbb{J}_{12}$, $\mathbb{J}_{8}$ and $\mathbb{J}_{11}$,
$\mathbb{J}_{13}$ and $\mathbb{J}_{14}$. Note that in each pair, one generator
has six non-zero elements, while the other has only four. Therefore our
version is sparser than the conventional textbook representation, in which
each generator has six nonvanishing elements.

##  5 The Exceptional Group $F_{4}$

In this and the next section we follow the notation of refs. [42] and [43],
where the generators for $F_{4}$ and $E_{6}$ have been explicitly derived. The
exceptional group $F_{4}$ has rank $4$ and dimension 52. It is the
automorphism group of the Jordan algebra

${\mathfrak{h}}_{3}=\left(\begin{matrix}r_{1}&{\mathbf{o}}_{1}&{\mathbf{o}}_{2}\\\
{\mathbf{o}}^{\ast}_{1}&r_{2}&{\mathbf{o}}_{3}\\\
{\mathbf{o}}^{\ast}_{2}&{\mathbf{o}}^{\ast}_{3}&r_{3}\end{matrix}\right),$
(14)

where $r_{a}$ are three real numbers ($a=1,2,3$), and $\mathbf{o}_{a}$ are
three octonions [42]. The asterisk notation in (14) stands for octonion
conjugation

$\mathbf{o}^{\ast}=x^{(0)}e_{0}-\sum_{i=1}^{7}x^{(i)}e_{i}.$ (15)

The fundamental representation is of dimension 26. Following [42, 43], we find
it convenient to work in $\mathbb{R}^{27}$ instead and map the components of
the feature vector $\mathbf{x}$ onto $r_{a}$ and $\mathbf{o}_{a}$ as follows

$\displaystyle r_{1}$ $\displaystyle=x^{(1)},\qquad$
$\displaystyle\mathbf{o}_{1}$ $\displaystyle=\sum_{i=0}^{7}x^{(2+i)}\,e_{i},$
(16a) $\displaystyle r_{2}$ $\displaystyle=x^{(18)},$
$\displaystyle\mathbf{o}_{2}$ $\displaystyle=\sum_{i=0}^{7}x^{(10+i)}\,e_{i},$
(16b) $\displaystyle r_{3}$ $\displaystyle=x^{(27)},$
$\displaystyle\mathbf{o}_{3}$ $\displaystyle=\sum_{i=0}^{7}x^{(19+i)}\,e_{i}.$
(16c)

The group $F_{4}$ preserves $K=3$ different oracles (invariant polynomials),
which can be expressed in terms of the variables (16) as follows:

$\displaystyle\varphi^{(1)}_{F_{4}}(\mathbf{x})$
$\displaystyle=\text{Tr}\,\mathfrak{h}_{3}=\sum_{a=1}^{3}r_{a}=x^{(1)}+x^{(18)}+x^{(27)},$
(17) $\displaystyle\varphi^{(2)}_{F_{4}}(\mathbf{x})$
$\displaystyle=\text{Tr}\,\mathfrak{h}^{2}_{3}=\sum_{a=1}^{3}\left(\,r_{a}^{2}+2\,|{\mathbf{o}}_{a}|^{2}\,\right)$
$\displaystyle=2\sum_{i=1}^{27}\left(x^{(i)}\right)^{2}-\sum_{i\in\\{1,18,27\\}}\left(x^{(i)}\right)^{2},$
(18) $\displaystyle\varphi^{(3)}_{F_{4}}(\mathbf{x})$
$\displaystyle=\det\mathfrak{h}_{3}=r_{1}r_{2}r_{3}-\sum_{a=1}^{3}r_{a}\,|{\mathbf{o}}_{4-a}|^{2}+2\real\left({\mathbf{o}}_{3}{\mathbf{o}}_{2}^{\ast}{\mathbf{o}}_{1}\right)$
$\displaystyle=x^{(1)}x^{(18)}x^{(27)}-x^{(1)}\sum_{i=19}^{26}\left(x^{(i)}\right)^{2}$
$\displaystyle-x^{(18)}\sum_{i=10}^{17}\left(x^{(i)}\right)^{2}-x^{(27)}\sum_{i=2}^{9}\left(x^{(i)}\right)^{2}$
$\displaystyle+2\,\sum_{i,j,k=0}^{7}\mathcal{D}_{ijk}\,x^{(19+i)}\,x^{(10+j)}\,x^{(2+k)}\,(2\delta_{j,10}-1),$
(19)

where the tensor $\mathcal{D}_{ijk}$ was defined in Eq. (12) and pictorially
illustrated in Figure 4. The additional factor of $(2\delta_{j,10}-1)$ in the
last line flips the sign of the $x^{(11)},x^{(12)},\ldots,x^{(17)}$ factors in
the sum and thus accounts for the conjugation of ${\mathbf{o}}_{2}$ in the
triple product ${\mathbf{o}}_{1}{\mathbf{o}}_{2}^{\ast}{\mathbf{o}}_{3}$.

Figure 6: The learned $27\times 27$ sparse generators $J_{\alpha}$,
$\alpha=1,\ldots,52$, for the case of $F_{4}$.

The results from the training of 52 $F_{4}$ generators with the greedy
algorithm and with the oracles (17-19) are shown in Figure 6. We used $m=900$
samples of the $n=27$ input features $x^{(i)}$, organized as in (16). The
training ran over up to 26,000 epochs with the Adam optimizer,
$\varepsilon=10^{-4}$ and learning rate of $5\times 10^{-4}$. A 53rd generator
was not found after more than 250,000 epochs.

The numerically derived generators in Figure 6 can be compared against
explicit analytical constructions of the $F_{4}$ generators in the literature.
For example, we have verified that the set of generators in Figure 6 is
isomorphic (in the sense that the matrix $O$ relating the two sets as in (8)
is orthogonal) to the set of fifty two $27\times 27$ matrices
$\mathbb{C}_{\alpha}$ listed in Appendix C of [42]. This comparison is rather
nontrivial in light of the following differences between the two studies:

  * 1.

Octonion multiplication table. The multiplication rules satisfied by the
imaginary unit octonions $e^{\prime}_{i}$ in [42] are different from those of
Figure 3. The two bases are related as $e^{\prime}_{1}=e_{1}$,
$e^{\prime}_{2}=e_{2}$, $e^{\prime}_{3}=e_{4}$, $e^{\prime}_{4}=e_{3}$,
$e^{\prime}_{5}=e_{6}$, $e^{\prime}_{6}=-e_{7}$, $e^{\prime}_{7}=e_{5}$. As a
result, the two sets of generators appear visually different, as the non-
vanishing components of the respective matrices are located in different rows
and/or columns. As it turns out, the conventions of Figure 3 happen to produce
an aesthetically more pleasing result — note how all non-vanishing entries in
Figure 6 are nicely lined up diagonally, while the corresponding patterns in
Appendix C of [42] appear more scattered and chaotic.

  * 2.

Normalization. The $F_{4}$ generators $\mathbb{C}_{\alpha}$ in [42] are
normalized as
$\text{Tr}(\mathbb{C}_{\alpha}\mathbb{C}_{\beta})=-\,6\,\delta_{\alpha\beta}$,
while those in Figure 6 are normalized as
$\text{Tr}(\mathbb{J}_{\alpha}\mathbb{J}^{T}_{\beta})=\delta_{\alpha\beta}$ as
per (21) and (22).

  * 3.

Basis of $\mathbb{R}^{27}$. Since the first oracle (17) is linear in
$\mathbf{x}$, one can apply an orthogonal rotation to $r_{1}$, $r_{2}$ and
$r_{3}$, so that one of the coordinates in the new basis, say the last one, is
directly proportional to $\varphi_{F_{4}}^{(1)}$ [42]. This choice has the
advantage that the last row and the last column of each generator matrix are
filled with zeros, as in [42], confirming that the fundamental representation
of $F_{4}$ is 26-dimensional. If we apply the same rotation to the learned
generators in Figure 6, the resulting matrices all have zeros in their last
rows and columns as well.

##  6 The Exceptional Group $E_{6}$

Figure 7: The additional learned $27\times 27$ sparse generators $J_{\alpha}$,
$\alpha=53,\ldots,78$, for the case of $E_{6}$. The hyperparameters used in
the training of the single non-sparse generator $\mathbb{J}_{53}$ used to seed
the algorithm were as in Section 5.

The exceptional group $E_{6}$ is of rank $6$ and dimension $78$. Following
[43], we work in the real case, which results in the split form of the $E_{6}$
algebra, with signature $(52,26)$. (By multiplying the 26 added generators by
$i$, the algebra remains real, and the Killing form becomes the compact one.
For details, we refer the interested reader to [43].) The relevant $27$
variables are those in (16). In this case, we require invariance with respect
to only the last oracle, (19), but not the first two, (17) and (18). This
allows for the presence of $78-52=26$ additional generators beyond those of
Fig. 6. Our goal in this section will be to derive those additional 26
generators which are not contained in the $F_{4}$ subalgebra of $E_{6}$.

The straightforward approach to deriving all $E_{6}$ generators would be to
apply the greedy algorithm and learn from scratch 78 generators preserving the
oracle (19). However, we can save a significant amount of work by leveraging
the learned $F_{4}$ generators from the previous section which already satisfy
(19) by definition. In other words, given the 52 already discovered $F_{4}$
generators, we are looking to find the additional 26 which complete $F_{4}$ to
$E_{6}$. This is the perfect setup for applying the Lie bracket trick — treat
the $F_{4}$ generators as the starter set
$\\{\mathbb{J}_{1},\ldots,\mathbb{J}_{52}\\}$, learn a single new non-sparse
generator $\mathbb{J}_{53}$ with the greedy algorithm, and then hand those
$52+1$ generators over to the LBT algorithm and let it do its job. The result
from this procedure (after the corresponding sparsification of the 26 new
generators among themselves) is shown in Figure 7. Notably, the LBT algorithm
was able to find all of the missing 25 generators from a single non-sparse
seed $\mathbb{J}_{53}$. The key to this was that the seed $\mathbb{J}_{53}$
was non-sparse, and therefore a linear combination involving a large number of
the canonical sparse $E_{6}$ generators.

The meachine-learned generators shown in Figure 7 can be verified against
analytically derived results in the literature. We have compared our results
to the additional 26 $E_{6}$ generators
$\mathbb{C}_{53},\ldots,\mathbb{C}_{78}$ listed in Appendix C of [43] and
found perfect agreement, once we account for the differences in our
conventions (see discussion at the end of Section 5). For example, the LBT
algorithm correctly finds exactly two diagonal generators, namely
$\mathbb{J}_{66}$ and $\mathbb{J}_{72}$ (respectively proportional to
$\mathbb{C}_{53}$ and $\mathbb{C}_{70}$ in [43]). This is expected, since the
rank of $E_{6}$ is larger than the rank of $F_{4}$ by 2. Also note that all
generators are antisymmetric in the octonionic indices $2-17,19-26$. Once
again, we find regular linear patterns in the locations of the nonzero matrix
elements in the generators in Figure 7.

##  7 Summary

Discovering symmetries in data is crucial for both fundamental theory and data
science. Recent advancements in ML algorithms, coupled with the growth in
computational resources, have facilitated progress in this area. However, when
dealing with highly complex and multidimensional problems, training machine-
learning models remains a significant bottleneck. This paper introduces two
novel algorithms that greatly accelerate the symmetry discovery process. The
new methods were rigorously tested and showcased using examples of symmetries
from the exceptional groups $G_{2}$, $F_{4}$, and $E_{6}$. Remarkably, the
symmetry generators were learned accurately and in a nicer form than the
conventional representations found in textbooks. With these groundbreaking
techniques at our disposal, we can now confidently tackle even more intricate
and challenging problems in mathematical physics, particle phenomenology, and
data analysis.

Acknowledgements. We are greatly indebted to P. Ramond for setting us on this
exceptional path by suggesting this project. We thank S. Gleyzer, R. Houtz, K.
Kong, S. Mrenna, H. Prosper and P. Shyamsundar for useful discussions. This
work is supported in part by US Department of Energy award DE-SC0022148.

##  Appendix A Loss function

In this appendix we adapt the loss functions from [29, 31] to the case where
we train a single generator $\mathbb{G}$. The loss function is chosen to
ensure that $\mathbb{G}$ has the following properties:

Invariance: preserving the values of a vector oracle
$\vec{\varphi}(\mathbf{x})$ for all sampled datapoints
$\left\\{\mathbf{x}\right\\}$ under a given candidate transformation
$\mathbb{G}$:

$L_{\text{inv}}\bigl{(}\mathbb{G},\\{\mathbf{x}\\}\bigl{)}=\frac{1}{m\varepsilon^{2}}\sum_{i=1}^{m}\Bigl{[}\vec{\varphi}\bigl{(}\mathbf{x}_{i}+\varepsilon{\mathbb{G}}\cdot\mathbf{x}_{i}\bigr{)}-\vec{\varphi}(\mathbf{x}_{i})\Bigr{]}^{2}\,,$
(20)

where “$\cdot$” denotes ordinary tensor multiplication and the arrow vector
notation implies summation over the $K$ oracle components.

Normalization: ensuring that the transformation $\mathbb{G}$ is not trivial
and normalized to 1:

$L_{\text{norm}}\bigl{(}\mathbb{G}\bigr{)}=\Bigl{[}\text{Tr}\Bigl{(}{\mathbb{G}}\cdot{\mathbb{G}}^{T}\Bigr{)}-1\Bigr{]}^{2}.$
(21)

Orthogonality. This condition is used if we already have some existing
orthonormalized generators $\\{J\\}$ and want to find an additional generator
$\mathbb{G}$. It guarantees that the transformation $\mathbb{G}$ is not in the
set $\\{J\\}$:

$L_{\text{ortho}}\bigl{(}\mathbb{G},\\{\mathbb{J}\\}\bigr{)}=\sum_{\begin{subarray}{c}\alpha\end{subarray}}\Bigl{[}\text{Tr}\bigl{(}{\mathbb{G}}\cdot{\mathbb{J}}_{\alpha}^{T}\bigr{)}\Bigr{]}^{2}.$
(22)

Sparsity: an additional loss term designed to encourage sparsity was
introduced in Ref. [31] :

$L_{\text{sp}}\bigl{(}\mathbb{G}\bigr{)}=\sum_{j,k=1}^{n}\sum_{\begin{subarray}{c}j^{\prime},k^{\prime}=1\end{subarray}}^{n}\left|{\mathbb{G}}^{(jk)}{\mathbb{G}}^{(j^{\prime}k^{\prime})}\right|\Bigl{(}1-\delta_{jj^{\prime}}\delta_{kk^{\prime}}\Bigr{)},$
(23)

where ${\mathbb{G}}^{(jk)}$ denotes the $jk$-component of $\mathbb{G}$.

In this study, the total loss function was formed as a plain sum of the
relevant individual terms (20-23).

## References

  * [1] D. J. Gross, The role of symmetry in fundamental physics, PNAS 93 (25) (1996) 14256–14259.  doi:10.1073/pnas.93.25.14256.
URL https://doi.org/10.1073/pnas.93.25.14256

  * [2] E. Noether, Invariante variationsprobleme, Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse 1918 (1918) 235–257.
  * [3] M. E. Peskin, Beyond the standard model, in: The 1996 European School of High-Energy Physics (formerly CERN / JINR School of Physics), 1997, pp. 49–142.  arXiv:hep-ph/9705479.
  * [4] C. Csáki, S. Lombardo, O. Telem, TASI Lectures on Non-supersymmetric BSM Models, WSP, 2018, pp. 501–570.  arXiv:1811.04279, doi:10.1142/9789813233348_0007.
  * [5] E. Wigner, J. Griffin, J. Griffin, Group Theory and Its Application to the Quantum Mechanics of Atomic Spectra, Pure and Applied Physics : a series of monographs and textbooks, Academic Press, 1959.
  * [6] P. Ramond, Introduction to Exceptional Lie Groups and Algebras, Tech. Rep. CALT-68-577, California Institute of Technology, Pasadena, California 91125 (12 1976).
  * [7] R. Slansky, Group Theory for Unified Model Building, Phys. Rept. 79 (1981) 1–128.  doi:10.1016/0370-1573(81)90092-2.
  * [8] P. Ramond, Exceptional groups and physics (2003).  arXiv:hep-th/0301050.
  * [9] B. S. Acharya, M theory, Joyce orbifolds and superYang-Mills, Adv. Theor. Math. Phys. 3 (1999) 227–248.  arXiv:hep-th/9812205, doi:10.4310/ATMP.1999.v3.n2.a3.
  * [10] B. S. Acharya, S. Gukov, M theory and singularities of exceptional holonomy manifolds, Phys. Rept. 392 (2004) 121–189.  arXiv:hep-th/0409191, doi:10.1016/j.physrep.2003.10.017.
  * [11] M. Atiyah, E. Witten, M theory dynamics on a manifold of G(2) holonomy, Adv. Theor. Math. Phys. 6 (2003) 1–106.  arXiv:hep-th/0107177, doi:10.4310/ATMP.2002.v6.n1.a1.
  * [12] J. Halverson, D. R. Morrison, On gauge enhancement and singular limits in G2 compactifications of M-theory, JHEP 04 (2016) 100.  arXiv:1507.05965, doi:10.1007/JHEP04(2016)100.
  * [13] S. Catto, Y. S. Choun, L. Kurt, Invariance Properties of the Exceptional Quantum Mechanics $(F_{4})$ and Its Generalization to Complex Jordan Algebras $(E_{6})$, Springer Proc. Math. Stat. 36 (2013) 469–475.  doi:10.1007/978-4-431-54270-4_34.
  * [14] A. Shahlaei, S. Rafibakhsh, $F_{4}$ , $E_{6}$ and $G_{2}$ exceptional gauge groups in the vacuum domain structure model, Phys. Rev. D 97 (5) (2018) 056015.  arXiv:1802.02905, doi:10.1103/PhysRevD.97.056015.
  * [15] S. Rafibakhsh, A. Shahlaei, Confinement in $F_{4}$ Exceptional Gauge Group Using Domain Structures, EPJ Web Conf. 137 (2017) 13013.  doi:10.1051/epjconf/201713713013.
  * [16] F. Gursey, P. Ramond, P. Sikivie, A Universal Gauge Theory Model Based on E6, Phys. Lett. B 60 (1976) 177–180.  doi:10.1016/0370-2693(76)90417-2.
  * [17] D. Croon, T. E. Gonzalo, L. Graf, N. Košnik, G. White, GUT Physics in the era of the LHC, Front. in Phys. 7 (2019) 76.  arXiv:1903.04977, doi:10.3389/fphy.2019.00076.
  * [18] F. Gursey, P. Sikivie, E(7) as a Universal Gauge Group, Phys. Rev. Lett. 36 (1976) 775.  doi:10.1103/PhysRevLett.36.775.
  * [19] I. Bars, M. Gunaydin, Grand Unification With the Exceptional Group E8, Phys. Rev. Lett. 45 (1980) 859.  doi:10.1103/PhysRevLett.45.859.
  * [20] R. Iten, T. Metger, H. Wilming, L. del Rio, R. Renner, Discovering physical concepts with neural networks, Phys. Rev. Lett. 124 (2020) 010508.  doi:10.1103/PhysRevLett.124.010508.
URL https://link.aps.org/doi/10.1103/PhysRevLett.124.010508

  * [21] S. Krippendorf, M. Syvaeri, Detecting Symmetries with Neural Networks (3 2020).  arXiv:2003.13679.
  * [22] Z. Liu, M. Tegmark, Machine Learning Conservation Laws from Trajectories, Phys. Rev. Lett. 126 (18) (2021) 180604.  arXiv:2011.04698, doi:10.1103/PhysRevLett.126.180604.
  * [23] G. Barenboim, J. Hirn, V. Sanz, Symmetry meets AI, SciPost Phys. 11 (2021) 014\.  arXiv:2103.06115, doi:10.21468/SciPostPhys.11.1.014.
  * [24] B. M. Dillon, G. Kasieczka, H. Olischlager, T. Plehn, P. Sorrenson, L. Vogel, Symmetries, safety, and self-supervision, SciPost Phys. 12 (6) (2022) 188.  arXiv:2108.04253, doi:10.21468/SciPostPhys.12.6.188.
  * [25] Z. Liu, M. Tegmark, Machine Learning Hidden Symmetries, Phys. Rev. Lett. 128 (18) (2022) 180201.  arXiv:2109.09721, doi:10.1103/PhysRevLett.128.180201.
  * [26] K. Desai, B. Nachman, J. Thaler, Symmetry discovery with deep learning, Phys. Rev. D 105 (9) (2022) 096031.  arXiv:2112.05722, doi:10.1103/PhysRevD.105.096031.
  * [27] S. Craven, D. Croon, D. Cutting, R. Houtz, Machine learning a manifold, Phys. Rev. D 105 (9) (2022) 096030.  arXiv:2112.07673, doi:10.1103/PhysRevD.105.096030.
  * [28] A. Moskalev, A. Sepliarskaia, I. Sosnovik, A. Smeulders, Liegg: Studying learned lie group generators (2022).  arXiv:2210.04345, doi:10.48550/ARXIV.2210.04345.
URL https://arxiv.org/abs/2210.04345

  * [29] R. T. Forestano, K. T. Matchev, K. Matcheva, A. Roman, E. B. Unlu, S. Verner, Deep learning symmetries and their Lie groups, algebras, and subalgebras from first principles, Mach. Learn. Sci. Tech. 4 (2) (2023) 025027.  arXiv:2301.05638, doi:10.1088/2632-2153/acd989.
  * [30] A. Roman, R. T. Forestano, K. T. Matchev, K. Matcheva, E. B. Unlu, Oracle-preserving latent flows, Symmetry 15 (7) (2023).  doi:10.3390/sym15071352.
URL https://www.mdpi.com/2073-8994/15/7/1352

  * [31] R. T. Forestano, K. T. Matchev, K. Matcheva, A. Roman, E. B. Unlu, S. Verner, Discovering Sparse Representations of Lie Groups with Machine Learning, Physics Letters B (2 2023).  arXiv:2302.05383.
  * [32] H.-Y. Chen, Y.-H. He, S. Lal, S. Majumder, Machine learning Lie structures & applications to physics, Phys. Lett. B 817 (2021) 136297.  arXiv:2011.00871, doi:10.1016/j.physletb.2021.136297.
  * [33] K. Holland, P. Minkowski, M. Pepe, U. J. Wiese, Exceptional confinement in G(2) gauge theory, Nucl. Phys. B 668 (2003) 207–236.  arXiv:hep-lat/0302023, doi:10.1016/S0550-3213(03)00571-6.
  * [34] S. Deldar, H. Lookzadeh, S. M. Hosseini Nejad, Confinement in G(2) Gauge Theories Using Thick Center Vortex Model and domain structures, Phys. Rev. D 85 (2012) 054501.  arXiv:1112.4963, doi:10.1103/PhysRevD.85.054501.
  * [35] S. M. Hosseini Nejad, S. Deldar, Role of the SU($2$) and SU($3$) subgroups in observing confinement in the G($2$) gauge group, Phys. Rev. D 89 (1) (2014) 014510\.  arXiv:1401.3968, doi:10.1103/PhysRevD.89.014510.
  * [36] T. Deppisch, E6Tensors: A Mathematica Package for E6 Tensors, Comput. Phys. Commun. 213 (2017) 130–135.  arXiv:1605.05920, doi:10.1016/j.cpc.2016.09.010.
  * [37] I. Todorov, S. Drenska, Octonions, exceptional Jordan algebra and the role of the group $F_{4}$ in particle physics, Adv. Appl. Clifford Algebras 28 (4) (2018) 82.  arXiv:1805.06739, doi:10.1007/s00006-018-0899-y.
  * [38] I. Todorov, M. Dubois-Violette, Deducing the symmetry of the standard model from the automorphism and structure groups of the exceptional Jordan algebra, Int. J. Mod. Phys. A 33 (20) (2018) 1850118.  arXiv:1806.09450, doi:10.1142/S0217751X1850118X.
  * [39] D. Corradetti, Complexification of the Exceptional Jordan Algebra and its Application to Particle Physics, J. Geom. Symmetry Phys. 61 (2021) 1–16.  doi:10.7546/jgsp-61-2021-1-16.
  * [40] J. C. Baez, The Octonions, Bull. Am. Math. Soc. 39 (2002) 145–205, [Erratum: Bull.Am.Math.Soc. 42, 213 (2005)].  arXiv:math/0105155, doi:10.1090/S0273-0979-01-00934-X.
  * [41] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization (2017).  arXiv:1412.6980.
  * [42] F. Bernardoni, S. L. Cacciatori, B. L. Cerchiai, A. Scotti, Mapping the geometry of the F(4) group, Adv. Theor. Math. Phys. 12 (4) (2008) 889–994.  arXiv:0705.3978, doi:10.4310/ATMP.2008.v12.n4.a6.
  * [43] F. Bernardoni, S. L. Cacciatori, B. L. Cerchiai, A. Scotti, Mapping the geometry of the E(6) group, J. Math. Phys. 49 (2008) 012107.  arXiv:0710.0356, doi:10.1063/1.2830522.