# Interpretable correlator Transformer for image-like quantum matter data

Abhinav Suresh  Department of Physics, Indian Institute of Science Education
and Research, Pune  Institut für Theoretische Physik, Universität Regensburg,
D-93035 Regensburg, Germany  Henning Schlömer  Fakultät für Physik, Munich
Quantum Center, Ludwig-Maximilians-Universität München  Munich Center for
Quantum Science and Technology (MCQST), Schellingstr. 4, D-80799 München,
Germany  Baran Hashemi  ORIGINS Data Science Lab  Technical University of
Munich, München, Germany  Annabelle Bohrdt  Institut für Theoretische Physik,
Universität Regensburg, D-93035 Regensburg, Germany  Munich Center for Quantum
Science and Technology (MCQST), Schellingstr. 4, D-80799 München, Germany

###### Abstract

Due to their inherent capabilities of capturing non-local dependencies,
Transformer neural networks have quickly been established as the paradigmatic
architecture for large language models and image processing. Next to these
traditional applications, machine learning methods have also been demonstrated
to be versatile tools in the analysis of image-like data of quantum phases of
matter, e.g. given snapshots of many-body wave functions obtained in ultracold
atom experiments. While local correlation structures in image-like data of
physical systems can reliably be detected, identifying phases of matter
characterized by global, non-local structures with interpretable machine
learning methods remains a challenge. Here, we introduce the correlator
Transformer (CoTra), which classifies different phases of matter while at the
same time yielding full interpretability in terms of physical correlation
functions. The network’s underlying structure is a tailored attention
mechanism, which learns efficient ways to weigh local and non-local
correlations for a successful classification. We demonstrate the versatility
of the CoTra by detecting local order in the Heisenberg antiferromagnet, and
show that local gauge constraints in one- and two-dimensional lattice gauge
theories can be identified. Furthermore, we establish that the CoTra reliably
detects non-local structures in images of correlated fermions in momentum
space (Cooper pairs) and that it can distinguish percolating from non-
percolating images.

##  I Introduction

Machine learning (ML) techniques provide powerful tools for image analysis
Song  _et al._ (1996); Ripley and Hjort (1996). Since the dawn of deep neural
networks, considerable progress has been made in image classification
(computer vision) Krizhevsky  _et al._ (2012); Donahue  _et al._ (2014);
Rowley  _et al._ (1998); Dosovitskiy  _et al._ (2021). In recent years, this
has led to an increasing interest in applying machine learning based analysis
tools to a wide range of physically relevant input data Carleo  _et al._
(2019). For instance, it has been demonstrated that neural networks can
efficiently classify and differentiate between various phases of matter Wang
(2016); Carrasquilla and Melko (2017); van Nieuwenburg  _et al._ (2017),
including topologically ordered states Zhang  _et al._ (2018); Sun  _et al._
(2018); Huembeli  _et al._ (2018); Yoshioka  _et al._ (2018); Carvalho  _et
al._ (2018); Rodriguez-Nieva and Scheurer (2019); Holanda and Griffith (2020)
and systems with strong correlations Ch’ng  _et al._ (2017); Broecker  _et
al._ (2017a). Furthermore, progress has been made towards discovering novel
quantum phases Hu  _et al._ (2017); Miles  _et al._ (2023); Cassella  _et al._
(2023); further unsupervised and supervised learning of quantum phases and
their transitions have been studied in Refs. Ch’ng  _et al._ (2018); Wetzel
and Scherzer (2017); Tanaka and Tomiya (2017); Zhang  _et al._ (2017); Beach
_et al._ (2018); Wang and Zhai (2017); Broecker  _et al._ (2017b); Suchsland
and Wessel (2018); Iakovlev  _et al._ (2018); Schäfer and Lörch (2019);
Lozano-Gómez  _et al._ (2022); Kottmann  _et al._ (2020); Salcedo-Gallo  _et
al._ (2020).

Figure 1: The correlator Transformer. (a) The overall flow starts from the 2D
input data, here illustrated by a snapshot with two channels. After cutting
the input into patches, they are linearly embedded to dimension $d$. The
collection of $d$-dimensional vector tokens is denoted by $X^{1}$ (green box).
Higher-order correlations are then generated in the Transformer encoder. To
analyze $n^{\text{th}}$ order correlations, $n-1$ encoding steps are carried
out. While the query remains identical throughout all layers, keys ($K_{j}$)
and values ($V_{j}$) are learned and computed in each layer, shown in panel
(b). In the $j^{\text{th}}$ layer, the attention then includes correlations up
to order $j+1$. By multiplying with the value $V_{j}$, $X^{j+1}$ is computed.
PE denotes the absolute positional encoding. Averaging the outputs results in
the vector $[\bar{X}^{1},\bar{X}^{2},\dots,\bar{X}^{n}]$, which is then used
for classification. Through a regularization path analysis, weights
$\beta^{\ell}$, $\ell=1,\dots,n$ of the final layer give information about
which correlations are important for successful classification.

With the development of highly controllable analog quantum simulation
platforms, e.g. ultracold atoms in optical lattices Bloch  _et al._ (2008);
Esslinger (2010); Bohrdt  _et al._ (2021), the application of neural networks
to projective measurements of the many-body wave function constitutes one
particular field of interest Bohrdt  _et al._ (2019). These snapshots of the
wave function contain a plethora of information that goes beyond local
observables Islam  _et al._ (2015); Hilker  _et al._ (2017); Sompet  _et al._
(2022), making them a promising platform for machine learning tools Käming
_et al._ (2021); Schlömer  _et al._ (2023); Miles  _et al._ (2023).

In order to gain physical insights, interpretable machine learning techniques
that are able to detect underlying order in a given set of measurements are
desirable. Examples for physically interpretable algorithms include finding
order parameters using support vector machines (SVM) Ponte and Melko (2017);
Liu  _et al._ (2019); Greitemann  _et al._ (2019), interpretability in terms
of principle component analysis (PCA) Wetzel (2017); Costa  _et al._ (2017);
Hu  _et al._ (2017) and analyzing the Hessian matrix of the loss function
Dawid  _et al._ (2021). Using network theory, wave function networks have been
developed to study such quantum simulation snapshots Mendes-Santos  _et al._
(2024). In relatively shallow neural networks, insights can potentially be
gained by looking into learned weights for a given classification task
Suchsland and Wessel (2018); Blücher  _et al._ (2020); Khatami  _et al._
(2020).

Typically, however, despite their effectiveness, neural networks tend to
operate as black boxes. _Interpretable_ neural network architectures ideally
combine a high performance with the possibility to uncover patterns and
correlations in physical systems with complex dependencies. For this purpose,
the correlation convolutional neural networks (CCNN) Miles  _et al._ (2020);
Schlömer and Bohrdt (2023); Miles  _et al._ (2023) has been developed, which
allows to extract the most defining correlations in the Fock basis of a given
snapshot. However, the convolutional structure of the CCNN architecture
prevents an efficient detection of global structures and hidden or non-local
order.

In order to access long-range dependencies in a snapshot, a different choice
of architecture is more natural: The introduction of the Transformer
architecture Vaswani  _et al._ (2023) marked a groundbreaking moment in the
development of large language models. Although conceived as a sequence-to-
sequence model, this was later adapted to the vision Transformer Dosovitskiy
_et al._ (2021) to be compatible with image data. The attention mechanism,
which aims to mimic human-like attention, provides a map of pertinent portions
and correlations of the image, which may help to gain a physical understanding
of (global) underlying structures. It has been shown that the Transformer
architecture can be used to learn for example entanglement growth and state
complexities from experimental data Kim  _et al._ (2024). In Schlömer and
Bohrdt (2023), the Transformer architecture was applied to quantum snapshots
of the Heisenberg model, however, without giving a physical interpretation of
the resulting attention mechanism. Here, we demonstrate that the vision
Transformer can reliably classify images of correlated systems, with full
interpretability in terms of local and non-local correlation functions. In
particular, we introduce a tailored attention mechanism that enables us to
isolate and capture specific orders of correlations from different Transformer
layers.

This paper is organized as follows. In Section II, we introduce the correlator
Transformer architecture. We discuss how higher-order correlators are
constructed in each subsequent layer and demonstrate that by adding tailored
$L_{1}$ regularization, one can obtain the most relevant order of correlation.
After obtaining the latter, an analysis of the learned weights of that
particular Transformer layer (attention maps) enables further insights into
the specific correlator. In Section III, we show our results for local
correlations upon applying the correlator Transformer to different physical
models: we demonstrate how the Transformer learns local correlations in the
two-dimensional (2D) antiferromagnetic (AFM) Heisenberg model, the one-
dimensional (1D) $\mathbb{Z}_{2}$ lattice gauge theory (LGT) as well as the 2D
$\mathbb{Z}_{2}$ Ising LGT, using local correlations up to fourth order. In
Section IV, we apply the Transformer to problems involving non-local
correlations, in particular in the analysis of momentum space images of Cooper
pairs and in the classification of percolating and non-percolating snapshots.
Finally, we summarize our results in Section V.

##  II Correlator Transformer

The correlator Transformer (CoTra) architecture is inspired by the vision
Transformer, a state-of-the-art image classification model. Like the vision
Transformer, the CoTra is a versatile model that can handle different types of
images. In this work, the input to our model is image-like quantum matter
data, including, for instance, projective measurements in real or momentum
space of quantum gas microscopes Bloch  _et al._ (2008); Esslinger (2010) or
tunneling microscope images Fujita  _et al._ (2014).

For a given depth $n$ of the CoTra (corresponding to the number of layers in
the Transformer architecture), the model has access to $n+1$ order weighted
correlations (from now on denoted by $X^{n+1}$) with respect to these input
measurements. These weights then give insights into the specific correlations
that the model uses for its classification.

###  II.1 Architecture

Consider a snapshot in $\mathbb{R}^{(h,h,c)}$ where $h,c$ are the height
(width) and number of channels. As done in standard vision Transformer
workflows, we first divide this snapshot into equal patches of dimension
$p\times p$, where $p$ is the patch size; hence, the transformed snapshot is
of dimension $(h/p)^{2}\times cp^{2}$, where $M=(h/p)^{2}$ denotes the number
of patches. Afterwards, a linear projection is performed that transforms the
patches to vectors of hidden dimension $d$. We label this projected snapshot
as $X^{1}$ $\in$ $\mathbb{R}^{((h/p)^{2},d)}$; it can be directly used as
input of the Transformer encoder or it can be supplemented with additional
learnable positional encoding depending on the input data, as further
commented on below. As the above only consists of reordering and linear
transforming the original data, the resulting matrix corresponds to first
order terms $X^{1}$, see Fig. 1 (a). By default, we do not use any
$Layer~{}Norm$, and it is used only if it is crucial for the model’s
performance. This choice is made to keep the model as simple as possible for
better interpretability.

The general idea of the CoTra is as follows (see Fig. 1): In the first layer,
both the query and key projections are computed from $X^{1}$, which, together
with a positional embedding, returns the second-order terms $X^{2}$. In a
second Transformer layer, the query projection remains the same, while the key
projection is now computed from the output of the first layer, i.e. $X^{2}$.
This second layer now computes the third-order terms $X^{3}$ and so on.
Therefore, using $n$ Transformer layers enables us to compute correlations up
to order $(n+1)$.

In particular, the query and key for the first Transformer layer can be
written as

$\displaystyle Q_{1}$ $\displaystyle=X^{1}W^{1Q}$ (1) $\displaystyle K_{1}$
$\displaystyle=X^{1}W^{1K},$

Where $W^{1Q},W^{1K}$ are the query and key projection matrices of the first
layer. Meanwhile, the value is a linear projection of the sine-cosine
encoding, defined by Vaswani  _et al._ (2023)

$\displaystyle\text{PE}_{(i,2j)}$
$\displaystyle=\sin\left(\frac{i}{100000^{2j/d}}\right)$ (2)
$\displaystyle\text{PE}_{(i,2j+1)}$
$\displaystyle=\cos\left(\frac{i}{100000^{2j/d}}\right).$

Here, $i\in\\{0,1,...N-1\\}$ is the position of $N$ patches and
$j\in\\{0,1,....,d-1\\}$ where $d$ is the hidden dimension. Since these are
not learnable parameters, they do not increase the complexity of the model.
The value, then, is

$\displaystyle V_{1}=(\text{PE})W^{1V}.$ (3)

The Transformer output of the first layer ($X^{2}$) can now be written as [see
Fig. 1 (b)]

$\displaystyle X^{2}$ $\displaystyle=\frac{1}{\sqrt{d}}Q_{1}K^{T}_{1}V_{1}$
(4)
$\displaystyle=\frac{1}{\sqrt{d}}X^{1}W^{1Q}W^{1K^{T}}X^{1^{T}}(\text{PE})W^{1V}.$

In particular, $X^{2}$ and the corresponding attention map
$a^{2}=\frac{1}{\sqrt{d}}Q^{1}K^{1^{T}}$ both contain only $2^{\text{nd}}$
order terms. This can be directly seen by writing out the matrix elements [see
Eq. (24) in Apendix A.1] of the $2^{\text{nd}}$ order attention map (with
patch labels $i,j$),

$\displaystyle
a^{2}_{ij}=\sum_{rs=1}^{cp^{2}}\sum_{klsm=1}^{d}x_{ir}x^{T}_{sj}P_{ik}P^{T}_{lj}$
$\displaystyle W^{P}_{rk}$ (5) $\displaystyle
W^{P^{T}}_{ls}W^{1Q}_{km}W^{1K^{T}}_{ml}.$

Here, $x$ denotes the pixel values (projective spin measurements, phase space
measurements etc.) of a given snapshot, $W^{P}$ is the linear projection
matrix to the hidden dimension, $W^{1Q}$, $W^{1K}$ are the query and key
projection matrices of the first layer and $P$ is an additional learnable
positional encoding. Depending on the dataset, this learnable positional
encoding is included at $X^{1}$ along with the sine-cosine encoding to improve
accuracy. As the indices in Eq. (5) $i,j$ go through all the patches and $r,s$
through all the pixels inside the patches, we can obtain all the
$2^{\text{nd}}$ order local and non-local correlators and their corresponding
attention weights.

Similarly, the $n^{\text{th}}$ layer output [see Fig. 1 (b)] is defined as

$\displaystyle X^{n+1}=\frac{1}{\sqrt{d}}Q^{1}_{n}K^{T}_{n}V_{n},$ (6)

where $Q^{1}_{n}$ denotes the query projection of $X^{1}$ by $W^{nQ}$, $K_{n}$
is the key projection of $X^{n}$ and $V_{n}$ is the value projection of the
positional embedding at each layer $n$. This attention can be thought of as
weighing the different correlations in $QK^{T}$ with their positional
information. In this manner, we can build up further orders of weighted
correlations using the Transformer, and by inspecting the weights of the
corresponding attention maps, the most relevant correlators can be obtained.

For each order $X^{n}$, we compute the mean $\bar{X}^{n}$ over all dimensions,
which is used as input for the final classification,
$\mathbf{\tilde{x}_{n}}=[\bar{X}^{1},\bar{X}^{2},\dots,\bar{X}^{n}]$. A linear
transformation on $\mathbf{\tilde{x}_{n}}$ is then performed to form the
logits for the cross-entropy loss function coupled with a regularization loss
of strength $\alpha$. In particular, the loss function reads

$\displaystyle\mathcal{L}\equiv-\sum_{i=1}^{N}y_{i}\log(\hat{y}_{i})+\alpha\|\mathbf{\tilde{x}_{n}}\|,$
(7)

where $y$ is the true distribution, $\hat{y}$ is the predicted distribution
(logits), and $||.||$ is the norm.

This modified architecture contains a similar number of parameters as a
traditional vision Transformer and is able to provide direct interpretability
in terms of local as well as non-local correlations. After training the CoTra
with a particular hidden dimension ($d$), learning rate ($lr$), regularization
strength ($\alpha$) and other hyperparameters, we collect the learned
correlators and perform a regularization path analysis to understand which
order terms influence the classification the most, as described next.

###  II.2 Regularization Path Analysis

Regularization path analysis is a common technique used for feature selection
Tibshirani (1996). Here, as done in Miles  _et al._ (2020), we use the $L_{1}$
regularization. The idea is to collect the learned mean correlators
$\bar{X}^{n}$ and perform a lasso regression to see which of the $n$-point
correlators are the most important for phase classification. We only re-train
the architecture’s logistic classifier part (final linear projection) rather
than the whole model.

Figure 2: 2D Heisenberg model. (a) Exemplary Heisenberg AFM (left) and random
(right) configuration. The inset illustrates the labeling of the spins inside
a $2\times 2$ patch. (b) Regularization path analysis showing that
$2^{\text{nd}}$ order correlations are used for classification. (c) Mean of
the 2nd order attention map for the two data sets. The two distributions are
almost entirely separate, underlining that the network uses second-order
correlations for classification. (d) Intra-patch correlation weights
$C_{rs}^{2}$, whereby nearest neighbor correlations show the most prominent
signals.

The loss function for the regularization path analysis is given by

$\displaystyle\mathcal{L}_{\text{path}}\equiv-\sum_{i}y_{i}\log(\hat{y}_{i})+\lambda\|\mathbf{\beta}\|,$
(8)

where $\beta\equiv[\beta^{1},\beta^{2},\dots,\beta^{n}]$ are the logistic
weights of the corresponding (mean) $n$-point correlator. We use the Scikit-
learn Pedregosa  _et al._ (2011) logistic regression library to optimize the
classifier over a range of $\lambda$ starting from a very high value, where
the classification fails, to gradually smaller strengths where the
classification is successful (see also Miles  _et al._ (2020)). By looking at
the strengths of $\beta^{n}$ over the range of $\lambda$ one can see which
particular $\beta^{n}$ (and hence which order of correlation) is responsible
for a successful classification. Afterward, a closer look into the attention
weights inside that particular Transformer layer tells us which specific
$n$-point correlators are used by the network for classification. For local
correlations, we can fix a patch and look at the attention weights inside that
patch to see how each correlator is weighed. On the other hand, the attention
map itself gives information about the dependencies between patches for long-
range correlations.

##  III Learning Local correlations and gauge constraints

In this section, we discuss the results of our model in learning local
correlations. In the following, we consider three exemplary datasets: the 2D
Heisenberg model, a 1D $\mathbb{Z}_{2}$ lattice gauge theory (LGT) and the 2D
$\mathbb{Z}_{2}$ Ising LGT.

###  III.1 2D Heisenberg model

We start by analyzing snapshots of a prototypical spin model, namely the
Heisenberg model. The Hamiltonian of the system is given by

$\displaystyle\hat{H}=J\sum_{\langle
i,j\rangle}\left(\hat{S}_{i}^{x}\hat{S}_{j}^{x}+\hat{S}_{i}^{y}\hat{S}_{j}^{y}+\hat{S}_{i}^{z}\hat{S}_{j}^{z}\right),$
(9)

where $J$ is the nearest neighbour coupling constant and $S^{x,y,z}$ are
spin-$1/2$ operators. We train the Transformer on snapshots from an
equilibrium finite temperature state at $T/J=0.6$ with AFM couplings ($J>0$)
against random snapshots with vanishing magnetization. This ensures that the
model does not compare magnetization values directly, but instead looks for
higher-order correlations when classifying the snapshots. We use a lattice
size of $16\times 16$ and sample Heisenberg snapshots using stochastic series
expansion quantum Monte Carlo Miles (2024); Sandvik and Kurkijärvi (1991);
Sandvik (1999). Fig. 2 (a) depicts one of the Heisenberg snapshots as well as
a random configuration. The chosen hyperparameters for this data are
$d=16,\text{patch size}=2,\lambda=10^{-3}$, and $lr=5\times 10^{-3}$. The
inset in the right panel of Fig. 2 (a) shows how the spins inside a single
$2\times 2$ patch are numbered, which is important when evaluating the
correlations at a later stage. We obtain an accuracy of 96.4%.

Figure 3: 1D lattice gauge theory. (a) 1D chain indicating the matter sites
$n_{j}$ and link sites $\tau^{x}_{j,j+1}$. A patch size of 2 is chosen (which
contains two matter and two link sites), indicated by the yellow box. The
numbering of elements inside the patch is shown in the inset. (b)
Regularization path analysis showing $3^{rd}$ order correlations to be crucial
for classification. (c) Mean of the $3^{rd}$ order attention map for the two
states. While random snapshots show a broad distribution, the data that
fulfils the gauge constraint features a sharp peak. (d) Third-order intra-
patch correlation weights. Large weights attributed to correlators between the
elements $0,2,3$ indicate that the CoTra is indeed checking if the gauge
constraint is fulfilled.

Fig. 2 (b) shows the regularization path, which indicates that the model is
capable of classifying the two datasets by evaluating $2$-point correlations.
We take the mean over the $2^{\text{nd}}$ order attention map
$a^{2}=\frac{1}{\sqrt{d}}Q^{1}K^{1^{T}}$ for each snapshot in the dataset and
plot a histogram. Indeed, Fig. 2 (c) shows that the distributions of the mean
$2^{\text{nd}}$ order attention for the AFM and random snapshots separate to a
large degree, corroborating that the network uses $2^{\text{nd}}$ order
correlations for its classification. Now, to see which particular
$2^{\text{nd}}$ order correlators are used by the network, we take a more
detailed look into the attention weights. The learned weights of the
$2^{\text{nd}}$ order correlator (labels inside a patch are given by $r,s$)
can be written as

$\displaystyle
C^{{}^{(2)}}_{rs}=\Bigg{|}\frac{1}{M}\sum_{i=1}^{M}\sum_{klm=1}^{d}P_{ik}$
$\displaystyle P_{li}^{T}W^{P}_{rk}$ (10) $\displaystyle
W^{P^{T}}_{ls}W^{1Q}_{km}W^{1K^{T}}_{ml}\Bigg{|},$

where we have taken the average over all patches $i$. $P$ is the learnable
positional encoding, $W^{P}$ is the linear projection matrix to the hidden
dimension and $W^{nQ},W^{nK}$ denotes the query, key projections in the
$n^{\text{th}}$ Transformer layer, see also Appendix A.1 for an explicit
derivation of Eq. (10).

Fig. 2 (d) depicts this $2$-point correlation matrix where $r,s$ runs through
all four spins inside the patch. We can see that the model puts its emphasis
on nearest neighbour correlations, cf. the labeling of spins within the patch
in the inset of Fig. 2 (a). With the combined information of the
regularization path and the learned weights, we conclude that the correlator
Transformer is calculating second-order (intra-patch) correlations to make its
classification decision.

###  III.2 1D $\mathbb{Z}_{2}$ LGT

Originally proposed in the context of non-perturbative theories in particle
physics, lattice gauge theories have become a paradigmatic class of systems
also in the field of strongly correlated phases of matter Levin and Wen
(2005); Lee  _et al._ (2006). Their defining feature is the presence of a
local gauge symmetry. Here, we consider a (1+1)D $\mathbb{Z}_{2}$ LGT where
hard-core bosonic matter is coupled to $\mathbb{Z}_{2}$ gauge fields. The
Hamiltonian of the system is described as Zohar  _et al._ (2017); Borla  _et
al._ (2020); Kebrič  _et al._ (2021); Halimeh  _et al._ (2022)

$\displaystyle\hat{H}_{0}=J\sum_{j=1}^{L-1}\left(\hat{a}^{\dagger}_{j}\hat{\tau}^{z}_{j,j+1}\hat{a}_{j+1}+\text{H.c.}\right)-h\sum_{j=1}^{L}\hat{\tau}^{x}_{j,j+1},$
(11)

where $\hat{a}^{\dagger}_{j},\hat{a}_{j}$ are the bosonic ladder operators
acting on matter site $j$, $\tau^{z}_{j,j+1}$ denotes the gauge field between
the matter site $j$ and $j+1$ and $\tau^{x}_{j,j+1}$ is the electric field
between the matter site $j$ and $j+1$. The $\mathbb{Z}_{2}$ electric field is
subject to Gauss’ law (GL), defined by the local generator

$\displaystyle\hat{G}_{j}=(-1)^{\hat{n}_{j}}\hat{\tau}^{x}_{j-1,j}\hat{\tau}^{x}_{j,j+1},$
(12)

which has eigenvalues $g_{j}=\pm 1$. In the following, we test whether the
CoTra can identify the local gauge structure. In order to do so, we generate
two classes of snapshots: One set consists of states with $g_{j}=+1$ $\forall
j$ in the lattice. The network is then given the task to distinguish this set
from snapshots with random gauge sectors $g_{j}=\pm 1$ $\forall j$.

We consider a chain of length $16$ with open boundaries. Fig. 3 (a) depicts
the left end of the 1D chain with matter sites $n_{j}$ and link variables
$\tau^{x}_{j,j-1}$. In the following, a single patch consists of two matter
and link variables, with labelling as depicted in the inset of Fig. 3 (a). For
better interpretability, we describe both matter and gauge fields on the same
footing, i.e., we convert the occupancy of the matter site (which is measured
in the Fock basis as 0 or 1) to $\pm 1$. We note that this transformation
changes the local generator of gauge invariance to

$\displaystyle\hat{G}_{j}=\hat{n}_{j}\hat{\tau}^{x}_{j-1,j}\hat{\tau}^{x}_{j,j+1}.$
(13)

For the matter site corresponding to $n_{1}$ within the unit cell depicted in
Fig. 3 (a), only the elements $0,2,3$ contribute to the calculation of Eq.
(13).

Figure 4: 2D Ising lattice gauge theory. (a) After training, a regularization
path analysis shows that $4^{\text{th}}$ order correlations are used to
classify snapshots that fulfil Gauss’ law from ones that do not. Indeed, the
mean ($4^{\text{th}}$ order) attention values fully separate the two data
sets, shown in (b). (c) Inset: Upper left part of snapshot fulfilling (not
fulfilling) Gauss’ law Eq. (16) on the left-hand (right-hand) side. The
labeling within each unit cell (purple box) is shown. The fourth order term on
the plaquette that checks Gauss’ law in each unit cell is given by
$\hat{\sigma}^{z}_{2}\hat{\sigma}^{z}_{5}\hat{\sigma}^{z}_{6}\hat{\sigma}^{z}_{7}$,
illustrated by the black circles. Indeed, weights of $4^{\text{th}}$ order
intra-patch correlations show significant weight for indices 2567, underlining
that the CoTra learns to check Gauss’ law. For illustrative reasons, only
weights above a threshold of 0.1 are shown in the main panel of (c).

The hyperparameters when training the CoTra for this data are $d=12,\text{
patch size}=2~{}(\text{contains 2 matter and 2 links}),\lambda=2\times
10^{-4},lr=10^{-2}$ and we obtain an accuracy of 97%. After training, we again
perform a regularization path analysis, see Fig. 3 (b). Indeed, third-order
correlations are used to distinguish the two different classes as expected
from Eq. (13). When evaluating the mean third-order attention maps for each
snapshot, we see that states that fulfil the constraint $g_{j}=1$ form a sharp
peak, while random snapshots have a broad distribution [Fig. 3 (c)]. Indeed,
the two histograms only have marginal overlap, underlining that the network is
making its decision based on three-point correlations.

In order to analyze which particular three-point correlator within a patch is
used for classification, we construct the third-order attention weights
$a^{3}=\frac{1}{\sqrt{d}}Q^{1}K^{2^{K}}$. The weights for the different
possible three-point correlations are given by (see Appendix A.2)

$\displaystyle C^{{}^{(3)}}_{ers}=$
$\displaystyle\bigg{|}\frac{1}{M}\sum_{i=1}^{M}\sum_{kln=1}^{d}\sum_{abcm=1}^{d}P_{ia}P^{T}_{ki}P_{il}W^{P}_{ea}W^{P^{T}}_{kr}$
(14) $\displaystyle
W^{P}_{sl}W^{1Q^{T}}_{mk}W^{2Q}_{ac}W^{1K}_{lm}W^{2K^{T}}_{cb}W^{1V^{T}}_{bn}(\text{PE})_{ni}\bigg{|},$

where we again average over all patch labels $i$ and the indices $e,r,s$ go
through the $4$ elements inside each patch. We note that in contrast to Eq.
(10), the third-order correlator weight matrix explicitly includes the
absolute positional embedding, as it involves terms ($X^{2}$) that have passed
through the first Transformer layer.

Fig. 3 (d) shows that significant weight is only on the correlator with
indices $(e,r,s)=(0,2,3)$, which exactly corresponds to evaluating Eq. (13).
We note that choosing a larger patch size leads to identical conclusions. For
instance, for a patch size of four, there are three possible combinations that
contribute to the Gauss law; correspondingly, this leads to the appearance of
three peaks in the correlation weights, i.e., the network calculates identical
correlations to perform its classification. We were also able to classify
states with 1-2 Gauss’s law violations along the chain against the completely
fulfilling snapshots with the addition of $Layer~{}Norm$ for the query and key
projections.

###  III.3 Classical 2D $\mathbb{Z}_{2}$ Ising LGT

Lastly, we focus on the classical $\mathbb{Z}_{2}$ Ising lattice gauge theory
in 2D, where Gauss law is given by a $4^{\text{th}}$ order term. The
Hamiltonian is defined as Greplova  _et al._ (2020)

$\displaystyle\hat{H}=-J\sum_{p}\prod_{l\in p}\hat{\sigma}^{z}_{l},$ (15)

where $J$ denotes the magnetic coupling constant, $\hat{\sigma}^{z}$ is the
$z$ Pauli matrix, and the summation covers all plaquettes $p$ in the lattice.
In the ground state, each plaquette $p$ of the lattice satisfies the Gauss’
law:

$\displaystyle G_{p}=\prod_{l\in p}\hat{\sigma}^{z}_{l}=1$ (16)

The system has a highly degenerate ground state, comprising all possible
configurations where $G_{p}=1$ is obeyed. At finite temperature, the gauge
constraint is violated locally. Excited states are thus characterized by

$\displaystyle G_{p}=\prod_{l\in p}\hat{\sigma}^{z}_{l}=\pm 1.$ (17)

As in the 1D LGT, we generate two classes of snapshots: One where Gauss’s law
($G_{p}=+1$) is obeyed globally and one where it is violated randomly.

In particular, we consider a $16\times 16$ lattice, with each lattice point
having one horizontal and vertical link. The left-hand side of the inset in
Fig. 4 (c) depicts an exemplary configuration of the ground state, where on
each plaquette, the product of Pauli $z$ operators is positive. On the other
hand, the products are both positive and negative in a finite temperature
snapshot; see the right-hand side of the inset in Fig. 4 (c). It is important
to note that a single patch of patch size $2$ contains $8$ links. We arrange
the input snapshots into $2$ channels, one for the horizontal and one for the
vertical links. By construction, out of the $8$ links in a patch, only the
links $2,5,6,7$ contribute to the central plaquette of the patch; see the
explicit labeling in the inset of Fig. 4 (c).

The hyperparameters for this data are $d=6,\text{patch size}=2,\lambda=2\times
10^{-4},lr=5\times 10^{-3}$. For these parameters, the network is able to
fully classify the snapshots with an accuracy of $100\%$. A regularization
path analysis for the given choice of hyperparameters shows that the model is
focusing on fourth-order correlators, Fig. 4 (a). Note that we do not employ
any additional learnable positional encoding here because the model is able to
achieve perfect accuracy with absolute positional encoding. This is
corroborated by the distributions of mean attention scores, as shown in Fig. 4
(b). Indeed, both distributions fully separate, making a full distinction
between the two data sets possible.

Figure 5: Detecting Cooper pairs in momentum space. (a) Left: a simulated
snapshot of a Fermi gas in momentum space with two Cooper pairs on the Fermi
surface, mimicking the data obtained in Holten  _et al._ (2022). Blue (red)
pixels denote electrons with spin up (down). The Fermi surface is highlighted
by the black dashed line. Right: Attention map with respect to the black
square (whose patch label is given by $\delta$). The CoTra is seen to attend
to the spin on the other side of the Fermi surface, with the opposite
attention weight sign. (b) The same for an unpaired snapshot. Here, though the
attention mostly focuses on the Fermi surface as in the paired state, all
attention weights are negative. (c) Regularization path analysis indicating
$2^{\text{nd}}$ order correlations to be crucial for classification. (d) Mean
of the $2^{\text{nd}}$ order attention map for the two data sets; while in the
paired state the average attention is sharply peaked around slightly positive
values, in the unpaired state average values are broadly distributed and
negative. A vanishing mean separates the two data sets, underlined by the
black dashed line.

In order to check which particular four-point correlator the model learns, we
construct the fourth-order attention weights
$a^{4}=\frac{1}{\sqrt{d}}Q^{1}K^{3^{T}}$, given by (see Appendix A.3)

$\displaystyle C^{{}^{(4)}}_{hers}=\bigg{|}\frac{1}{M}\sum_{i=1}^{M}$
$\displaystyle\sum_{kln=1}^{d}\sum_{abcm=1}^{d}\sum_{pqto=1}^{d}W^{P}_{hp}W^{P^{T}}_{ae}W^{P}_{rk}$
(18) $\displaystyle
W^{P^{T}}_{ls}W^{1Q}_{km}W^{2Q^{T}}_{ca}W^{3Q}_{pt}W^{1K^{T}}_{ml}W^{2K}_{bc}$
$\displaystyle
W^{3K^{T}}_{qt}W^{1V}_{nb}W^{2V^{T}}_{qo}(\text{PE})_{in}(\text{PE})^{T}_{oi}\bigg{|},$

where the indices $h,e,r,s$ run through the $8$ elements inside a single
patch, and we take an average over all the patches to compute
$C^{{}^{(4)}}_{hers}$. Note that the only dependency on patches comes from
absolute positional embedding in this case. We show the intra-patch weights as
given in Eq. (18) in Fig. 4 (c), and find that the model is giving the most
significant weight to the correlator corresponding to the term
$(h,e,r,s)=(2,5,6,7)$, which matches Gauss’ law [cf. the labeling in each unit
cell as shown in the inset of Fig. 4 (c)].

We note that for the case of the $\mathbb{Z}_{2}$ Ising LGT, we additionaly
use a $Layer~{}Norm$ after the query and key projections, which we identify as
crucial for a successful classification (without it, the network fails to
identify the gauge structure, leading to an accuracy of $50\%$). We further
find that the network’s output depends on the choice of hyperparameters: runs
with various choices of parameters filter out $2^{\text{nd}}$, $3^{\text{rd}}$
as well as $4^{\text{th}}$ as the most important correlations. Nevertheless,
in Appendix B, we show that for all cases, corresponding results are
interpretable and that, indeed, classification can be achieved in various ways
by looking at different orders of correlations for the classical
$\mathbb{Z}_{2}$ LGT.

##  IV Learning Non-Local Structures

So far, we have seen that the CoTra can detect local correlations as well as
local gauge structures through the evaluation of local $n$-point correlations.
In this section, we consider datasets containing snapshots that can be
distinguished only by detecting non-local structures. This demonstrates the
ability of the Transformer to capture long-range dependencies, which
constitutes a challenge for many architectures and thus establishes the CoTra
as a versatile analysis tool for quantum matter data.

###  IV.1 Cooper Pairs

Our first testing case is the detection of non-local structures of Fermi
surfaces in BCS-type states that feature bound Cooper pairs Bardeen  _et al._
(1957). This is in particular motivated by recent experimental observations of
Fermi surface correlations in a mesoscopic 2D Fermi gas Holten  _et al._
(2022). There, using a 2D optical dipole trap-optical tweezer setup, momentum-
resolved snapshots have been obtained through time of flight (TOF) expansion
measurements. In Ref. Holten  _et al._ (2022), single particle momentum
correlations are analyzed in the experimental data. At weak attractions, the
particles show no correlations; however, at sufficiently strong interaction
strengths, they form opposite momentum opposite spin pairs across the Fermi
surface. Here, we generate synthetic TOF data akin to the experimental results
in Ref. Holten  _et al._ (2022) and train the CoTra to analyze whether it can
successfully capture these long-range correlations in continuous image-like
data.

To this end, we again generate two classes of data: In both cases, the
momentum space snapshots are generated by distributing $+1$ for spin-up and
$-1$ for spin-down fermions on a $30\times 30$ grid with a Fermi momentum of
$10$ units. The paired state features two Cooper pairs on the Fermi surface
and four particles randomly distributed inside the Fermi surface. Images in
the second class also have two spin-up and two spin-down fermions on the Fermi
surface, but none of these form a pair. Exemplary snapshots for paired and
unpaired states are shown in the left-hand side of Fig. 5 (a) and (b),
respectively.

The hyperparameters for this data are $d=16,\text{patch
size}=2,\lambda=10^{-4},\text{ and }lr=10^{-2}$; we obtain an accuracy of
$98.1\%$. The regularization path analysis shown in Fig. 5 (c) indicates that
the Transformer is using second-order correlations to make its decision. The
distributions of mean second order attention maps (mean over the whole
attention map matrix) are shown in Fig. 5 (d). Similar to earlier examples,
the paired state has a sharp distribution of attention values, which is
largely separated from a broad distribution of the unpaired state.

We obtain the attention map with respect to a given patch $\mathbf{\delta}$
through

$\displaystyle a^{2}_{i\mathbf{\delta}}=\sum_{rs=1}^{cp^{2}}\sum_{klsm=1}^{d}$
$\displaystyle x_{ir}x^{T}_{s\mathbf{\delta}}P_{ik}P^{T}_{l\mathbf{\delta}}$
(19) $\displaystyle W^{p}_{rk}W^{p^{T}}_{ls}W^{1q}_{km}W^{1k^{T}}_{ml}.$

The index $i$ runs through all the patches, thereby giving us correlations
between patch $\mathbf{\delta}$ and all others. Thus, by focusing on
quantities like Eq. (19), long-range correlations can be analyzed. The right-
hand side of Fig. 5 (a) and (b) show the resulting second-order attention
maps.

Figure 6: Detecting percolation. (a) Left: percolated structure of a binary
snapshot. Right: Attention map $a_{\delta i}$ with respect to the black square
(whose patch label is given by $\delta$). The CoTra is seen to attend to the
percolating structure, with uniform attention weight sign. (b) The same for a
non-percolated snapshot. Here, the attention is focused mostly around the
reference patch $\delta$, with a seemingly random sign structure. (c)
Regularization path analysis indicating $2^{\text{nd}}$ order correlations to
be crucial for classification. (d) Mean of the $2^{\text{nd}}$ order attention
map for the two data sets; while in the percolated phase the average attention
is sharply peaked around slightly negative values, in the non-percolated state
average values are broadly distributed and positive. A vanishing mean
separates the two data sets, underlined by the black dashed line.

For the paired snapshot, the attention is clearly given to the Cooper pair;
see Fig. 5 (a). In contrast, no such structure exists in the unpaired state,
see Fig. 5 (b). A particular feature of the attention maps is their sign
structure: In the paired state, attention weights of the Cooper pair are of
opposite sign, while in the random state, all weights are negative. This, in
turn, leads to a separation of the mean attention values, and hence the
separation of the corresponding histograms, cf. Fig. 5 (d). We note that the
above holds independent of the reference patch $\delta$. In Appendix C, we
show attention maps when using a reference patch containing a particle inside
the Fermi surface. In this case, the attention is on both Cooper pairs, while
no particular structures exist for the unpaired state.

###  IV.2 Percolation

Percolation theory, first introduced by Flory and Stockmayer in the 1940s
Flory (1941), considers the formation of clusters that eventually span an
extended system in a percolating cluster Hu (1984). Naturally, the distinction
of whether a particular snapshot is percolating or not constitutes a problem
that requires the evaluation of non-local correlations. In this section, we
train the CoTra with synthetic data of percolating and non-percolating binary
images on a 2D lattice.

In particular, we focus on a simplified percolating system with only one
percolating path (directed percolation). We generate a $16\times 16$ lattice
of random spins directed from one edge to the opposite edge (i.e. we generate
random percolating structures). Half of the generated percolating images are
then randomly shuffled such that they become non-percolating. The left-hand
sides of Fig. 6 (a) and (b) show exemplary percolating (non-percolating)
snapshots. We note that due to our construction of the snapshots, the total
number of grey pixels is the same; this prevents the network from simply
counting the number of active pixels.

We train the CoTra with hyperparameters $d=16,\text{patch
size}=2,\lambda=10^{-3},\text{ and }lr=10^{-2}$. We obtain an accuracy of
$99.3\%$. After training, a regularization path analysis shown in Fig. 6 (a)
indicates second-order correlations to be crucial for the classification. Mean
second-order attention values are seen to separate, whereby percolating (non-
percolating) images have a sharp (broad) distribution, see Fig. 6 (b). We
emphasize the generality of this observation and suspect that a global
underlying structure typically leads to distributions of the CoTra mean
attention map values that have small widths.

We plot second-order attention maps given by Eq. (19) on the corresponding
right-hand sides of Fig. 6 (a). The reference patch $\delta$ is given by the
black square in the snapshot images. In the percolating case, the attention
mechanism indeed puts emphasis on the percolating structure with uniform sign.
In case of the non-percolating image, the attention seems to be localized
relatively close to the reference patch, with a seemingly random sign
structure. We note that, by utilizing the capability of analyzing the
dependencies between all patches, second-order inter-patch (i.e., long-range)
correlations allow the network to make a classification. We note that when
choosing a reference patch that does not contain any pixel of the percolating
cluster, the attention maps is zero.

For both Cooper pairs and percolation, the sign of the attention map is seen
to be an important factor contributing to the classification. In both cases,
we see that the classification boundary lies around zero [see the black dashed
lines in Fig. 5 (d) and Fig. 6 (d)]. By assigning a certain sign structure to
the inter-patch attention, the CoTra is able to reliably detect non-local
structures through inter-patch correlations.

##  V Summary and Outlook

We have introduced a Transformer-inspired machine learning framework to
analyze image-like quantum matter data with a tailored attention mechanism
that yields full interpretability in terms of correlation functions. Combined
with a regularization path analysis, we have shown that the network is able to
identify both local and non-local structures in a variety of testing cases. We
have demonstrated that the network can learn local correlations, such as
present in the AFM Heisenberg model, and that it can identify gauge
constraints, e.g. in 1D and 2D lattice gauge theories. We further demonstrated
the network’s capabilities of detecting non-local dependencies and
correlations, by training on images of correlated Fermi surfaces featuring BCS
pairs as well as percolating and non-percolating images.

Our proposed architecture can readily be used for experimental image-like
data, for instance, real- and momentum-space snapshots of ultracold atoms Fu
_et al._ (2020); Lunt  _et al._ (2024) or scanning tunneling microscopes. This
may help to reveal hidden order, topological phases, and other global, non-
local structures such as symmetries of e.g. charge density wave phases Fujita
_et al._ (2014). The CoTra may further be used for learning and interpreting
disentangling gates, akin to the reinforcement learning protocol demonstrated
in Tashev  _et al._ (2024). Recently, it has been proposed that percolation
can be used as an effective order parameter for confinement in
$\mathbb{Z}_{2}$ LGTs Linsel  _et al._ (2024). It would be interesting to see
how the CoTra performs when given finite temperature snapshots of confined and
deconfined phases, and if any connection to percolation can be made from the
resulting attention maps. It would further be interesting to combine the CoTra
with a confusion learning scheme akin to Schlömer and Bohrdt (2023), which
allows for a fully unsupervised phase detection and classification. This, in
particular, can be useful to analyze doping and temperature scans of the
strongly correlated Fermi-Hubbard model, where non-local, hidden structures
are believed to play an essential role in the low-doping regime Hilker  _et
al._ (2017); Schlömer  _et al._ (2023).

Acknowledgements.– We thank Simon M. Linsel for valuable comments. This
research was funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germany’s Excellence Strategy – EXC2111 –
390814868, as well as the European Research Council (ERC) under the European
Union’s Horizon 2020 research and innovation programme (grant agreement number
948141 — ERC Starting Grant SimUcQuam). AS acknowledges support by the
Graduiertenkolleg of the University of Regensburg (IRTG SFB 1277), as well as
the KVPY program of DST (Department of Science and Technology) Govt of India.
BH acknowledges support by Excellence Cluster ORIGINS, funded by the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s
Excellence Strategy - EXC-2094-390783311.

##  Appendix A Correlator Weights

We here derive the expressions for the attention maps as well as correlator
weights presented in the main text, see i.p. Eqs. (10), (14), (18).

###  A.1 $2^{\text{nd}}$ Order Correlators

Let $x$ be the input image in $\mathbb{R}^{(h,w,c)}$, where $h,w,c$ represents
the height, width and number of channels. First, we patch this image with a
patch size of $p$; the flattened image will then be of size
$(\frac{hw}{p^{2}},cp^{2})$ for a 2D snapshot and $(h/p,cp)$ for a 1D
snapshot. We denote the number of patches ($\frac{hw}{p^{2}}$) with $M$.

Now we linearly project these flattened patches to a hidden dimension $d$ and
multiply with a learnable position encoding $P$ in an element-wise fashion,

$\displaystyle X^{p}$ $\displaystyle=P*W^{p}X$ (20) $\displaystyle X^{p}_{ij}$
$\displaystyle=\sum_{r}^{cp^{2}}x_{ir}W^{p}_{rj}P_{ij}.$

Here, $W^{p}$ is the projection matrix in $\mathbb{R}^{(cp^{2},d)}$ and $X$
denotes the flattened patches. The learnable position encoding helps in more
accurate classification since it encodes each patch with a certain weight.

After this, we introduce three independent linear projections by $W^{\alpha
Q},W^{\alpha K},W^{\alpha V}$ to form the query, key and value, as in standard
Transformer architectures Vaswani  _et al._ (2023). $\alpha$ denotes the
particular Transformer layer that the projection is applied to.

$\displaystyle Q^{1}_{ij}$
$\displaystyle=\sum_{k}^{d}\sum_{r}^{cp^{2}}x_{ir}W^{p}_{rk}P_{ik}W^{1Q}_{kj}$
(21) $\displaystyle K^{1}_{ij}$
$\displaystyle=\sum_{l}^{d}\sum_{s}^{cp^{2}}x_{is}W^{p}_{sl}P_{il}W^{1Q}_{lj}$
(22) $\displaystyle V^{1}_{ij}$ $\displaystyle=\sum_{n}^{d}PE_{in}W^{1V}_{nj}$
(23)

Where the value ($V^{1}$) is derived from the sine-cosine positional encoding,
see Eq. (2). Now, we calculate the $2^{\text{nd}}$ order attention map $a^{2}$
(setting the constants ($d$) to 1 for simplicity) by

$\displaystyle a^{2}=$ $\displaystyle Q^{1}K^{1^{T}}$ $\displaystyle
a^{2}_{ij}=$
$\displaystyle\sum_{rs=1}^{cp^{2}}\sum_{klsm=1}^{d}x_{ir}x^{T}_{sj}P_{ik}P^{T}_{lj}W^{p}_{rk}$
$\displaystyle W^{p^{T}}_{ls}W^{1q}_{km}W^{1k^{T}}_{ml}$ (24)

The $2^{nd}$ order weight matrix for a given patch can be constructed from the
coefficients of $a^{2}$ by fixing $i=j$ and then taking an average over all
$M$ patches $i$. In the analysis in the main text, we focus on the absolute
value of all the $n^{th}$ order correlator weights $C^{{}^{n}}$.

$\displaystyle
C^{{}^{(2)}}_{rs}=\left|\frac{1}{M}\sum_{i=1}^{M}\sum_{klm=1}^{d}P_{ik}P_{li}^{T}W^{P}_{rk}W^{P^{T}}_{ls}W^{1Q}_{km}W^{1K^{T}}_{ml}\right|,$
(25)

The output of the first layer $X^{2}$ can be written as

$\displaystyle X^{2}=$ $\displaystyle a^{2}V^{1}$ $\displaystyle X^{2}_{ij}=$
$\displaystyle\sum_{g=1}^{M}\sum_{rs=1}^{cp^{2}}\sum_{klsmn=1}^{d}x_{ir}x^{T}_{sg}P_{ik}P^{T}_{lg}W^{p}_{rk}$
$\displaystyle W^{p^{T}}_{ls}W^{1q}_{km}W^{1k^{T}}_{ml}(PE)_{gn}W^{1v}_{nj}$
(26)

###  A.2 $3^{rd}$ Order Correlators

The key projection in the $2^{\text{nd}}$ layer is derived from $X^{2}$ while
the query projection remains the same, ie from $X^{1}$ itself but with a
different projection matrix $W^{2Q}$,

$\displaystyle K^{2}=$ $\displaystyle X^{2}W^{2k}$ $\displaystyle K^{2}_{ij}=$
$\displaystyle\sum_{g=1}^{M}\sum_{rs=1}^{cp^{2}}\sum_{klsm=1}^{d}\sum_{nb=1}^{d}x_{ir}x^{T}_{sg}P_{ik}P^{T}_{lg}W^{p}_{rk}W^{p^{T}}_{sl}$
$\displaystyle W^{{1q}}_{km}W^{1k^{T}}_{ml}(PE)_{gn}W^{{1v}}_{nb}W^{2k}_{bj}.$
(27)

The $3^{rd}$ order attention map can be constructed by

$\displaystyle a^{3}=$ $\displaystyle Q^{2}K^{2^{T}}$ $\displaystyle
a^{3}_{ij}=$
$\displaystyle\sum_{g=1}^{M}\sum_{ers=1}^{cp^{2}}\sum_{klsn=1}^{d}\sum_{abcm=1}^{d}x_{ie}x^{T}_{rj}x_{ng}P_{ia}P^{T}_{kj}P_{gl}W^{p}_{ea}$
$\displaystyle
W^{p^{T}}_{kr}W^{p}_{sl}W^{1q^{T}}_{mk}W^{2q}_{ac}W^{1k}_{lm}W^{2k^{T}}_{cb}W^{1v^{T}}_{bn}(PE)^{T}_{ng},$
(28)

where $Q^{2}$ is the $2^{nd}$ layer query projection of $X^{1}$. The $3^{rd}$
order correlator weights for a given patch can be constructed from learnable
coefficients of the $3rd$ order attention map by fixing $i=j=g$ and then
taking a mean over all the $M$ patches.

$\displaystyle C^{{}^{(3)}}_{ers}=$
$\displaystyle\bigg{|}\frac{1}{M}\sum_{i=1}^{M}\sum_{kln=1}^{d}\sum_{abcm=1}^{d}P_{ia}P^{T}_{ki}P_{il}W^{P}_{ea}W^{P^{T}}_{kr}$
(29) $\displaystyle
W^{P}_{sl}W^{1Q^{T}}_{mk}W^{2Q}_{ac}W^{1K}_{lm}W^{2K^{T}}_{cb}W^{1V^{T}}_{bn}(\text{PE})_{ni}\bigg{|},$

The output of the $2^{\text{nd}}$ layer $X^{3}$ can be written as

$\displaystyle X^{3}_{ij}=$
$\displaystyle\sum_{gf=1}^{M}\sum_{ers=1}^{cp^{2}}\sum_{klsn=1}^{d}\sum_{abcmo=1}^{d}x_{ie}x^{T}_{rf}x_{gs}P_{ia}P^{T}_{kf}$
(30) $\displaystyle
P_{gl}W^{p}_{ea}W^{p^{T}}_{kr}W^{p}_{sl}W^{1q^{T}}_{mk}W^{2q}_{ac}W^{1k}_{lm}W^{2k^{T}}_{cb}W^{1v^{T}}_{bn}$
$\displaystyle(PE)^{T}_{ng}W^{2v}_{oj}(PE)_{fo}.$

###  A.3 $4^{\text{th}}$ Order Correlators

Similar to constructing the previous order correlator weights, we can obtain
the key projection from $X^{3}$ while keeping the query projection the same.
Note that we do not use any learnable positional encoding for the Ising LGT.

$\displaystyle K^{3}=$ $\displaystyle X^{3}W^{3k}$ (31) $\displaystyle
K^{3}_{ij}=$
$\displaystyle\sum_{gf=1}^{M}\sum_{ers=1}^{cp^{2}}\sum_{klsn=1}^{d}\sum_{abcm=1}^{d}\sum_{oq=1}^{d}x_{ie}x^{T}_{rf}x_{ns}$
$\displaystyle
W^{p}_{ea}W^{p^{T}}_{kr}W^{p}_{sl}W^{1q^{T}}_{mk}W^{2q}_{ac}W^{1k}_{lm}W^{2k^{T}}_{cb}W^{1v^{T}}_{bn}$
$\displaystyle(PE)^{T}_{nf}W^{2v}_{oq}(PE)_{fo}W^{3k}_{qj}$

The $4^{\text{th}}$ order attention map is written as

$\displaystyle a^{4}=$ $\displaystyle Q^{3}K^{3^{T}}$ (32) $\displaystyle
a^{4}_{ij}=$
$\displaystyle\sum_{gf=1}^{M}\sum_{ers=1}^{cp^{2}}\sum_{klsn=1}^{d}\sum_{abcm=1}^{d}\sum_{pqto=1}^{d}x_{ih}x^{T}_{ej}x_{qr}x^{T}_{sn}$
$\displaystyle
W^{p}_{hg}W^{p^{T}}_{ae}W^{p}_{rk}W^{p^{T}}_{ls}W^{1q}_{km}W^{2q^{T}}_{ca}W^{3q}_{gp}W^{1Kk^{T}}_{ml}$
$\displaystyle
W^{2k}_{bc}W^{3k^{T}}_{pq}W^{1v}_{nb}W^{2v^{T}}_{qo}(PE)_{gn}(PE)^{T}_{of}.$

Where $Q^{3}$ is the $3^{rd}$ layer query projection of $X^{1}$. The
$4^{\text{th}}$ order correlator weights can then be written as

$\displaystyle C^{{}^{(4)}}_{hers}=\bigg{|}\frac{1}{M}\sum_{i=1}^{M}$
$\displaystyle\sum_{kln=1}^{d}\sum_{abcm=1}^{d}\sum_{pqto=1}^{d}W^{p}_{hp}W^{p^{T}}_{ae}W^{p}_{rk}$
(33) $\displaystyle
W^{p^{T}}_{ls}W^{1q}_{km}W^{2q^{T}}_{ca}W^{3q}_{pt}W^{1k^{T}}_{ml}W^{2k}_{bc}$
$\displaystyle
W^{3k^{T}}_{qt}W^{1v}_{nb}W^{2v^{T}}_{qo}(\text{PE})_{in}(\text{PE})^{T}_{oi}\bigg{|}.$

##  Appendix B Discussion on hyperparameter dependence

As noted in the main text, we find that especially for the Ising
$\mathbb{Z}_{2}$ LGT, the results depend on the choice of hyperparameters. One
possible explanation for this is our use of the $LayerNorm$, which is crucial
for a successful classification in the case of the Ising LGT. The use of
$LayerNorm$ complicates the interpretability of the network because we do not
account for the weights and bias of the normalization in our analysis. By
varying the hyperparameters, we indeed observe that in different runs the
Transformer focuses on $2^{\text{nd}}$, $3^{\text{rd}}$ and $4^{\text{th}}$
order correlators to classify the Ising LGT. In the following, we look in more
detail into the output of the Transformer for all of the above cases and show
that the network’s decisions remain interpretable.

Figure 7: a) Regularization path analysis indicating $2^{nd}$ order terms to
be important b) $2^{nd}$ order weights from the attention map Figure 8: a)
Schematic showing possible spin alignments on a plaquette (blue indicates spin
up and red indicates spin down) for the ground state; the resulting sums of
the 2-point terms are shown in the centres. b) Additional possible
configurations in an excited state, which does not fulfil the gauge constraint
on each plaquette.

Fig. 7 shows the results for the hyperparameters: $d=4,\text{patch
size}=2,\lambda=1e^{-3},lr=0.005$. For this set of hyperparameters, the
network uses $2^{\text{nd}}$ order correlations to discriminate the phases.
Though this may seem surprising, it is indeed possible to discriminate the two
phases by adding certain $2$-point correlations: If we look at the sum of the
(horizontal) product of two links - call it $g^{2}$ \- on the plaquette, we
get the following criteria (see Fig. 8):

$\displaystyle g^{2}$ $\displaystyle=\pm 2\text{~{}~{}for states that globally
fulfil GL}$ (34) $\displaystyle g^{2}$ $\displaystyle=0,\pm 2\text{~{}~{}for
excited states}.$

Figure 9: (a) Regularization path analysis indicating $3^{\text{rd}}$ order
term to be important. (b) $3^{rd}$ order weights from the attention map (only
values above a threshold of $0.005$ are plotted). The network calculates
3-point correlators on the plaquette, from which it can judge whether the
gauge constraint is fulfilled or not.

Indeed, this is similar to Gauss’ law Eq. (17). In the weight matrix
$C^{2}_{rs}$, shown in Fig. 8 (b), we see that 2-point correlations involving
the plaquette sites $2,5,6,7$ are used. By adding these correlations, the
Transformer is able to reliably detect the local gauge constraints.

Fig. 9 shows the results for the hyperparameters $d=28,\text{patch
size}=2,\lambda=1e^{-4},lr=0.01$; here, the network utilizes 3-point
correlations for classification.

The reasoning for the $2^{\text{nd}}$ correlations can also be extended to the
$3^{\text{rd}}$ order. The sum of $3$-point terms $g^{3}$ on a plaquette can
be categorized as

$\displaystyle g^{3}$ $\displaystyle=0,\pm 4\text{~{}~{}for ground states}$
(35) $\displaystyle g^{3}$ $\displaystyle=\pm 2\text{~{}~{}for excited
states}.$

As we can see from the correlation weights for the $3^{rd}$ order terms, Fig.
9 (b), the larger weights are associated with the terms $256$, $257$, $267$
and $567$. Thus, by summing over these 3-point correlations, perfect accuracy
can be achieved.

##  Appendix C Varying reference patch for Cooper pair data

Figure 10: (a) Attention map for Cooper paired state w.r.t the patch denoted
by the black square. Attention is seen to be put on the two Cooper pairs on
the Fermi surface, regardless of the reference patch. (b) Attention map for
the unpaired state w.r.t the patch denoted by the black square. No structures
as observed in (a) can be seen. Figure 11: a) Attention map w.r.t the
highlighted patch for the percolating state, b) attention map w.r.t the
highlighted patch for the non-percolating state, c) regularization path
indicating $2^{nd}$ order correlators to be important, d) Mean of the $2^{nd}$
order attention map for the two data sets

Fig. 10 shows the attention map w.r.t a particle inside the Fermi surface. For
a snapshot that features Cooper pairs, the attention is - regardless of the
reference patch - seen to focus on the paired electrons on the Fermi surface,
see Fig. 10 (a). On the other hand, for an unpaired snapshot, Fig. 10 (b), no
such structures are observable.

##  Appendix D Additional Results

###  D.1 Percolation

In this section, we train the model with slightly modified percolation data in
which the percolating path is broken at very few points. We get similar
results to the main text data with $2^{nd}$ order correlations to be the most
important.

The hyperparameters used are $d=16,\text{patch size}=2,\lambda=2\times
10^{-3},\text{ and }lr=5\times 10^{-3}$; we obtain an accuracy of $95.78\%$.
This indicates that the CoTra is able to precisely classify percolation in
this dataset. Fig. 11 (a,b) shows $2^{nd}$ attention maps for the percolating
and non-percolating states. While Fig. 11 (c,d) shows the regularization path
and histogram of mean attention.

###  D.2 Staggered Magnetization

In this section, we train the model with snapshots of $16\times 16$ lattice
having staggered magnetization equal to zero against snapshots with staggered
magnetization equal to $0.015625$.

The hyperparameters used are $d=4,\text{patch size}=2,\lambda=10^{-4},\text{
and }lr=10^{-1}$. The model is able to successfully classify the dataset with
$99.38\%$ accuracy.

Figure 12: a) Detecting staggered magnetization. Regularization path analysis
b) Corresponding $2^{nd}$ order correlation weights.

Fig. 12 (a) shows the $2^{nd}$ order regularization path, and Fig. 12 (b)
shows the corresponding order weights. Though we get more than $99\%$
accuracy, the regularization paths and the weights depend on the choice of
hyperparameters (we show the results for the lowest hidden dimension).

## References

##

  * Song  _et al._ (1996) H.-H. Song, S.-M. Kang, and S.-W. Lee, in  _Proceedings of 13th International Conference on Pattern Recognition_, Vol. 4 (1996) pp. 718–722 vol.4.
  * Ripley and Hjort (1996) B. D. Ripley and N. L. Hjort (1996).
  * Krizhevsky  _et al._ (2012) A. Krizhevsky, I. Sutskever, and G. E. Hinton, in  _Advances in Neural Information Processing Systems_, Vol. 25, edited by F. Pereira, C. Burges, L. Bottou, and K. Weinberger (Curran Associates, Inc., 2012).
  * Donahue  _et al._ (2014) J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, in  _Proceedings of the 31st International Conference on Machine Learning_, Proceedings of Machine Learning Research, Vol. 32, edited by E. P. Xing and T. Jebara (PMLR, Bejing, China, 2014) pp. 647–655.
  * Rowley  _et al._ (1998) H. Rowley, S. Baluja, and T. Kanade, IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 23 (1998).
  * Dosovitskiy  _et al._ (2021) A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” (2021), arXiv:2010.11929 [cs.CV] .
  * Carleo  _et al._ (2019) G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová, Rev. Mod. Phys. 91, 045002 (2019).
  * Wang (2016) L. Wang, Phys. Rev. B 94, 195105 (2016).
  * Carrasquilla and Melko (2017) J. Carrasquilla and R. G. Melko, Nature Physics 13, 431–434 (2017).
  * van Nieuwenburg  _et al._ (2017) E. van Nieuwenburg, Y.-H. Liu, and S. Huber, Nature Physics 13, 435–439 (2017).
  * Zhang  _et al._ (2018) P. Zhang, H. Shen, and H. Zhai, Physical Review Letters 120 (2018), 10.1103/physrevlett.120.066401.
  * Sun  _et al._ (2018) N. Sun, J. Yi, P. Zhang, H. Shen, and H. Zhai, Phys. Rev. B 98, 085402 (2018).
  * Huembeli  _et al._ (2018) P. Huembeli, A. Dauphin, and P. Wittek, Phys. Rev. B 97, 134109 (2018).
  * Yoshioka  _et al._ (2018) N. Yoshioka, Y. Akagi, and H. Katsura, Phys. Rev. B 97, 205110 (2018).
  * Carvalho  _et al._ (2018) D. Carvalho, N. A. García-Martínez, J. L. Lado, and J. Fernández-Rossier, Phys. Rev. B 97, 115453 (2018).
  * Rodriguez-Nieva and Scheurer (2019) J. F. Rodriguez-Nieva and M. S. Scheurer, Nature Physics 15, 790–795 (2019).
  * Holanda and Griffith (2020) N. L. Holanda and M. A. R. Griffith, Phys. Rev. B 102, 054107 (2020).
  * Ch’ng  _et al._ (2017) K. Ch’ng, J. Carrasquilla, R. G. Melko, and E. Khatami, Phys. Rev. X 7, 031038 (2017).
  * Broecker  _et al._ (2017a) P. Broecker, J. Carrasquilla, R. G. Melko, and S. Trebst, Scientific Reports 7 (2017a), 10.1038/s41598-017-09098-0.
  * Hu  _et al._ (2017) W. Hu, R. R. P. Singh, and R. T. Scalettar, Phys. Rev. E 95, 062122 (2017).
  * Miles  _et al._ (2023) C. Miles, R. Samajdar, S. Ebadi, T. T. Wang, H. Pichler, S. Sachdev, M. D. Lukin, M. Greiner, K. Q. Weinberger, and E.-A. Kim, Phys. Rev. Res. 5, 013026 (2023).
  * Cassella  _et al._ (2023) G. Cassella, H. Sutterud, S. Azadi, N. D. Drummond, D. Pfau, J. S. Spencer, and W. M. C. Foulkes, Phys. Rev. Lett. 130, 036401 (2023).
  * Ch’ng  _et al._ (2018) K. Ch’ng, N. Vazquez, and E. Khatami, Phys. Rev. E 97, 013306 (2018).
  * Wetzel and Scherzer (2017) S. J. Wetzel and M. Scherzer, Phys. Rev. B 96, 184410 (2017).
  * Tanaka and Tomiya (2017) A. Tanaka and A. Tomiya, Journal of the Physical Society of Japan 86, 063001 (2017).
  * Zhang  _et al._ (2017) Y. Zhang, R. G. Melko, and E.-A. Kim, Phys. Rev. B 96, 245119 (2017).
  * Beach  _et al._ (2018) M. J. S. Beach, A. Golubeva, and R. G. Melko, Phys. Rev. B 97, 045207 (2018).
  * Wang and Zhai (2017) C. Wang and H. Zhai, Phys. Rev. B 96, 144432 (2017).
  * Broecker  _et al._ (2017b) P. Broecker, F. F. Assaad, and S. Trebst, “Quantum phase recognition via unsupervised machine learning,” (2017b), arXiv:1707.00663 [cond-mat.str-el] .
  * Suchsland and Wessel (2018) P. Suchsland and S. Wessel, Phys. Rev. B 97, 174435 (2018).
  * Iakovlev  _et al._ (2018) I. A. Iakovlev, O. M. Sotnikov, and V. V. Mazurenko, Phys. Rev. B 98, 174411 (2018).
  * Schäfer and Lörch (2019) F. Schäfer and N. Lörch, Phys. Rev. E 99, 062107 (2019).
  * Lozano-Gómez  _et al._ (2022) D. Lozano-Gómez, D. Pereira, and M. J. P. Gingras, Phys. Rev. Res. 4, 043118 (2022).
  * Kottmann  _et al._ (2020) K. Kottmann, P. Huembeli, M. Lewenstein, and A. Acín, Physical Review Letters 125 (2020), 10.1103/physrevlett.125.170603.
  * Salcedo-Gallo  _et al._ (2020) J. Salcedo-Gallo, C. Galindo-González, and E. Restrepo-Parra, Journal of Magnetism and Magnetic Materials 501, 166482 (2020).
  * Bloch  _et al._ (2008) I. Bloch, J. Dalibard, and W. Zwerger, Rev. Mod. Phys. 80, 885 (2008).
  * Esslinger (2010) T. Esslinger, Annual Review of Condensed Matter Physics 1, 129 (2010).
  * Bohrdt  _et al._ (2021) A. Bohrdt, L. Homeier, C. Reinmoser, E. Demler, and F. Grusdt, Annals of Physics 435, 168651 (2021), special issue on Philip W. Anderson.
  * Bohrdt  _et al._ (2019) A. Bohrdt, C. S. Chiu, G. Ji, M. Xu, D. Greif, M. Greiner, E. Demler, F. Grusdt, and M. Knap, Nature Physics 15, 921 (2019).
  * Islam  _et al._ (2015) R. Islam, R. Ma, P. M. Preiss, M. Eric Tai, A. Lukin, M. Rispoli, and M. Greiner, Nature 528, 77 (2015).
  * Hilker  _et al._ (2017) T. A. Hilker, G. Salomon, F. Grusdt, A. Omran, M. Boll, E. Demler, I. Bloch, and C. Gross, Science 357, 484 (2017), https://www.science.org/doi/pdf/10.1126/science.aam8990 .
  * Sompet  _et al._ (2022) P. Sompet, S. Hirthe, D. Bourgund, T. Chalopin, J. Bibo, J. Koepsell, P. Bojović, R. Verresen, F. Pollmann, G. Salomon, C. Gross, T. A. Hilker, and I. Bloch, Nature 606, 484 (2022).
  * Käming  _et al._ (2021) N. Käming, A. Dawid, K. Kottmann, M. Lewenstein, K. Sengstock, A. Dauphin, and C. Weitenberg, Machine Learning: Science and Technology 2, 035037 (2021).
  * Schlömer  _et al._ (2023) H. Schlömer, T. A. Hilker, I. Bloch, U. Schollwöck, F. Grusdt, and A. Bohrdt, Communications Materials 4, 64 (2023).
  * Ponte and Melko (2017) P. Ponte and R. G. Melko, Phys. Rev. B 96, 205146 (2017).
  * Liu  _et al._ (2019) K. Liu, J. Greitemann, and L. Pollet, Phys. Rev. B 99, 104410 (2019).
  * Greitemann  _et al._ (2019) J. Greitemann, K. Liu, and L. Pollet, Phys. Rev. B 99, 060404 (2019).
  * Wetzel (2017) S. J. Wetzel, Physical Review E 96 (2017), 10.1103/physreve.96.022140.
  * Costa  _et al._ (2017) N. C. Costa, W. Hu, Z. J. Bai, R. T. Scalettar, and R. R. P. Singh, Phys. Rev. B 96, 195138 (2017).
  * Dawid  _et al._ (2021) A. Dawid, P. Huembeli, M. Tomza, M. Lewenstein, and A. Dauphin, Machine Learning: Science and Technology 3, 015002 (2021).
  * Mendes-Santos  _et al._ (2024) T. Mendes-Santos, M. Schmitt, A. Angelone, A. Rodriguez, P. Scholl, H. Williams, D. Barredo, T. Lahaye, A. Browaeys, M. Heyl, and M. Dalmonte, Physical Review X 14 (2024), 10.1103/physrevx.14.021029.
  * Blücher  _et al._ (2020) S. Blücher, L. Kades, J. M. Pawlowski, N. Strodthoff, and J. M. Urban, Physical Review D 101 (2020), 10.1103/physrevd.101.094507.
  * Khatami  _et al._ (2020) E. Khatami, E. Guardado-Sanchez, B. M. Spar, J. F. Carrasquilla, W. S. Bakr, and R. T. Scalettar, Physical Review A 102 (2020), 10.1103/physreva.102.033326.
  * Miles  _et al._ (2020) C. Miles, A. Bohrdt, R. Wu, C. S. Chiu, M. Xu, G. Ji, M. Greiner, K. Q. Weinberger, E. A. Demler, and E.-A. Kim, Nature Communications 12 (2020).
  * Schlömer and Bohrdt (2023) H. Schlömer and A. Bohrdt, SciPost Phys. 15, 099 (2023).
  * Vaswani  _et al._ (2023) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” (2023), arXiv:1706.03762 [cs.CL] .
  * Kim  _et al._ (2024) H. Kim, Y. Zhou, Y. Xu, K. Varma, A. H. Karamlou, I. T. Rosen, J. C. Hoke, C. Wan, J. P. Zhou, W. D. Oliver, Y. D. Lensky, K. Q. Weinberger, and E.-A. Kim, “Attention to quantum complexity,”  (2024), arXiv:2405.11632 [quant-ph] .
  * Fujita  _et al._ (2014) K. Fujita, M. H. Hamidian, S. D. Edkins, C. K. Kim, Y. Kohsaka, M. Azuma, M. Takano, H. Takagi, H. Eisaki, S. ichi Uchida, A. Allais, M. J. Lawler, E.-A. Kim, S. Sachdev, and J. C. S. Davis, Proceedings of the National Academy of Sciences 111, E3026 (2014), https://www.pnas.org/doi/pdf/10.1073/pnas.1406297111 .
  * Tibshirani (1996) R. Tibshirani, Journal of the royal statistical society series b-methodological 58, 267 (1996).
  * Pedregosa  _et al._ (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Journal of Machine Learning Research 12, 2825 (2011).
  * Miles (2024) C. Miles, “Qgasml,” https://github.com/ColeMiles/QGasML/tree/main/notebook (2024).
  * Sandvik and Kurkijärvi (1991) A. W. Sandvik and J. Kurkijärvi, Phys. Rev. B 43, 5950 (1991).
  * Sandvik (1999) A. W. Sandvik, Phys. Rev. B 59, R14157 (1999).
  * Levin and Wen (2005) M. Levin and X.-G. Wen, Rev. Mod. Phys. 77, 871 (2005).
  * Lee  _et al._ (2006) P. A. Lee, N. Nagaosa, and X.-G. Wen, Rev. Mod. Phys. 78, 17 (2006).
  * Zohar  _et al._ (2017) E. Zohar, A. Farace, B. Reznik, and J. I. Cirac, Phys. Rev. Lett. 118, 070501 (2017).
  * Borla  _et al._ (2020) U. Borla, R. Verresen, F. Grusdt, and S. Moroz, Phys. Rev. Lett. 124, 120503 (2020).
  * Kebrič  _et al._ (2021) M. c. v. Kebrič, L. Barbiero, C. Reinmoser, U. Schollwöck, and F. Grusdt, Phys. Rev. Lett. 127, 167203 (2021).
  * Halimeh  _et al._ (2022) J. C. Halimeh, L. Homeier, C. Schweizer, M. Aidelsburger, P. Hauke, and F. Grusdt, Phys. Rev. Res. 4, 033120 (2022).
  * Greplova  _et al._ (2020) E. Greplova, A. Valenti, G. Boschung, F. Schäfer, N. Lörch, and S. D. Huber, New Journal of Physics 22, 045003 (2020).
  * Holten  _et al._ (2022) M. Holten, L. Bayha, K. Subramanian, S. Brandstetter, C. Heintze, P. Lunt, P. M. Preiss, and S. Jochim, Nature 606, 287–291 (2022).
  * Bardeen  _et al._ (1957) J. Bardeen, L. N. Cooper, and J. R. Schrieffer, Phys. Rev. 108, 1175 (1957).
  * Flory (1941) P. J. Flory, Journal of the American Chemical Society 63, 3083 (1941).
  * Hu (1984) C.-K. Hu, Phys. Rev. B 29, 5103 (1984).
  * Fu  _et al._ (2020) H. Fu, Z. Zhang, K.-X. Yao, L. Feng, J. Yoo, L. W. Clark, K. Levin, and C. Chin, Physical Review Letters 125 (2020), 10.1103/physrevlett.125.183003.
  * Lunt  _et al._ (2024) P. Lunt, P. Hill, J. Reiter, P. M. Preiss, M. Galka, and S. Jochim, “Realization of a laughlin state of two rapidly rotating fermions,”  (2024), arXiv:2402.14814 [cond-mat.quant-gas] .
  * Tashev  _et al._ (2024) P. Tashev, S. Petrov, F. Metz, and M. Bukov, “Reinforcement learning to disentangle multiqubit quantum states from partial observations,”  (2024), arXiv:2406.07884 [quant-ph] .
  * Linsel  _et al._ (2024) S. M. Linsel, A. Bohrdt, L. Homeier, L. Pollet, and F. Grusdt, “Percolation as a confinement order parameter in $\mathbb{Z}_{2}$ lattice gauge theories,”  (2024), arXiv:2401.08770 [quant-ph] .