††thanks:  We acknowledge Krid Jinklub for useful comments. AM acknowledges
support by the National Science Foundation of China, through the grant No.
11875113, the Shanghai Municipality, through the grant No. KBH1512299, and by
Fudan University, through the grant No. JJH1512105. ML acknowledges the
support from National Science Foundation of China grant No. 12050410244.
(Corresponding author: Filippo Fabrocini.)

# Deep Neural Networks as the Semi-classical Limit of Topological Quantum
Neural Networks:
The problem of generalisation

Antonino Marcianò  Center for Field Theory and Particle Physics & Department
of Physics, Fudan University, Jingwan campus, Jingsan Rd, 200433 Shanghai,
China and Laboratori Nazionali di Frascati INFN, Via Enrico Fermi, 54, 00044
Frascati (Rome), Italy, EU and INFN sezione Roma “Tor Vergata”, 00133 Rome,
Italy, EU<EMAIL_ADDRESS>Deen Chen  College of Design and Innovation,
Tongji University, 281 Fuxin Rd, 200092 Shanghai, China
<EMAIL_ADDRESS>Filippo Fabrocini  College of Design and Innovation,
Tongji University, 281 Fuxin Rd, 200092 Shanghai, China and Institute for
Computing Applications "Mario Picone", Italy National Research Council, Via
dei Taurini, 19, 00185 Rome, Italy, EU<EMAIL_ADDRESS>Chris Fields
Allen Discovery Center, Tufts University, 200 College Avenue, 02155 Medford,
MA, US<EMAIL_ADDRESS>Matteo Lulli  Department of Mechanics and
Aerospace Engineering, Southern University of Science and Technology, 1088
Xueyuan Avenue, 518055 Shenzhen, China<EMAIL_ADDRESS>Emanuele Zappala
Yale School of Medicine, Yale University, 300 George Street, New Haven, CT,
06511, USA<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Deep Neural Networks miss a principled model of their operation. A novel
framework for supervised learning based on Topological Quantum Field Theory
that looks particularly well suited for implementation on quantum processors
has been recently explored. We propose the use of this framework for
understanding the problem of generalization in Deep Neural Networks. More
specifically, in this approach Deep Neural Networks are viewed as the semi-
classical limit of Topological Quantum Neural Networks. A framework of this
kind explains easily the overfitting behavior of Deep Neural Networks during
the training step and the corresponding generalization capabilities.

##  I Introduction

Deep neural networks (DNNs), i.e. neural networks with several hidden layers,
have become popular due to their success in a variety of learning task ranging
from molecular design Segler et al. (2018) and socioeconomic predictions
Abitbol and Karsai (2020), to machine translation Vaswani et al. (2017) and
approximating partial differential operators Li et al. (2021). However, our
fundamental understanding of this technology has lagged far behind. DNNs are
largely considered to be "black box" systems to the extent that, on one side,
they miss a principled model of their operation while, on the other side, the
internal operation of DNNs cannot be easily read out by human beings. The
first issue is mainly related to our limited understanding of how DNNs achieve
those generalization capabilities that let us state that they are able to
learn (whatever “learning” means here) while the second has to do with the
difficulty of interpreting the operational behavior of a non-linear weighted
model trained over hundreds of thousands of inputs that contribute
microscopically to the final configuration of the network. These issues
together constitute the main technical challenge for achieving a fair,
accountable, and transparent Artificial Intelligence (AI), with the first
generally considered under the rubric of verifiability (see Wing (2021);
Seshia et al. (2022) for recent reviews) and the second generally considered
under the rubric of XAI (see Wing (2021); Seshia et al. (2022); Samek et al.
(2021) for recent reviews).

The objective of this paper is to deal with the first issue by developing and
validating a principled model of generalization in DNNs. Our approach thus
differs from post-hoc or example-based approaches (see e.g. discussion in
Samek et al. (2021)) by seeking a tractable mathematical model of the function
$f$ computed by some DNN system following training with a training set $S$. In
this sense, our study falls in the field of generalization theory. The
generalization ability of DNNs is not yet understood despite the rich
literature concerning the topic and it is still a mystery Keskar et al.
(2017); Arpit et al. (2017); Dinh et al. (2017); Dziugaite and Roy (2017);
Hoffer et al. (2017); Krueger et al. (2017); Neyshabur et al. (2017, 2018); Wu
et al. (2017); Shwartz-Ziv and Tishby (2017); Lin et al. (2017); Wang et al.
(2017); Li et al. (2020). A principled approach to a trustable AI clearly
requires understanding how DNNs generalize, and hence how they fail to
generalize correctly in particular situations. A principled model of
generalization will enable performance validation at the theoretical level, as
opposed to merely the empirical one. In particular, a principled model of DNN
generalization will hopefully enable the designer of Machine Learning (ML)
applications to meet specific, pre-established generalization and performance
objectives. As Kevin Hartnett Hartnett (2019) has written in Quanta Magazine:

> When we design a skyscraper, we expect it will perform to specification:
> that the tower will support so much weight and be able to withstand an
> earthquake of a certain strength. But with one of the most important
> technologies of the modern world, we are effectively building blind. We play
> with different designs, tinker with different setups, but until we take it
> out for a test run, we do not really know what it can do or where it will
> fail.

We do not really know, in fact, after one or even several test runs: it always
remains the case that the “next” generalization problem will reveal failure. A
principled approach for studying generalisation in DNNs is to prove a
generalisation bound which is typically an upper bound on the test error based
on some features of the training set. Unfortunately, finding tight bounds has
proven to be a difficult challenge so far.

To further motivate our goal of a principled approach to generalisation, it is
useful to contrast it with state-of-the-art example-based, post-hoc
explanations Samek et al. (2021). As emphasized in Taylor and Taylor (2020),
post-hoc methods are, effectively, applications of experimental cognitive
psychology techniques to ML systems. Such techniques have an obvious weakness:
a successful ML system may be detecting patterns in both training and test
data that human observers cannot detect, as AlphaFold Senior et al. (2020)
appears to do. ML systems work with uninterpreted inputs; what we regard as
“image” data, for example, are just bit patterns to a ML system. In
particular, they have no access to the object categorizations – the semantics
– imposed by humans. Experiments such as in Brady et al. (2008) show that
humans are very poor at processing images as layouts of pixels, independently
of object identification and categorization. It is not clear whether
explanation in the usual sense is possible when ML systems detect patterns
that humans cannot detect, and hence have no semantics for, particularly
patterns that conflict with typical human object categories. Our approach is
independent of categories or other imposed semantics; we model the process of
generalisation in the abstract, not its outcomes in particular cases. Hence
our approach – like any verification approach – can underlie and give
confidence to, but does not replace, example-based explanation techniques that
are aimed at humans, and thus must employ shared human semantics. A two-part
solution of this kind, if successful, can be expected to have a beneficial
social impact since it will improve the trustworthiness of AI systems in the
interest of an Artificial Intelligence (AI) “for social good”, as a number of
high-profile initiatives in the Ethical AI field have emphasized (e.g.
Asilomar AI Principles, Montreal Declaration for Responsible AI, EU Ethics
Guidelines for Trustworthy Artificial Intelligence, IEEE Global Initiative for
Ethical Considerations in Artificial Intelligence and Autonomous Systems,
etc.; see Shin (2021); Choung et al. (2022); Hasija and Esper (2022) for
discussion).

We address the generalisation problem by leveraging results coming from
Topological Quantum Field Theory (TQFT). TQFT is used to construct a
topological version of Quantum Neural Networks (QNNs) named Topological
Quantum Neural Networks (TQNNs) Marcianò et al. (2022). TQNNs differ from
conventional QNNs Farhi and Neven (2018); Beer et al. (2020) by allowing the
number of “layers” and their connection topology to vary arbitrarily, provided
only that the input and output boundary conditions are preserved. The full
generality of the TQNN framework as a representation of computational
processes acting on classical data to yield classical outputs has been
recently proven Fields et al. (2022a). We use this framework to explain the
generalisation issues in DNNs by adopting a quantum physics approach according
to which the generalisation capabilities of DNNs are the analogue of the semi-
classical manifestation of quantum effects or, more specifically, of the
topological encoding achieved by TQNN. This allows us, in particular, to “read
off” detected topological invariants independently of their semantics, and
hence independently of their detectability by human observers.

The plan of the paper is the following. In Sec. II we remind the reader about
the state of the art of neural networks and provide motivations for our study.
In Sec. III we summarize the novel strategy rooted in the framework provided
by TQFT, namely TQNN. In Sec. IV we discuss generalisation within the TQNN
framework. In Sec. V we provide a dictionary correlating DNN and TQNN notions.
In Sec. VI we outline possible perspectives associated with the new strategy
developed. Finally, in Sec. VII we offer some preliminary conclusions based on
the TQNN models.

##  II Motivations and theoretical background

We will recall here a few essential notions of ML with the aim of introducing
the problem of generalisation in DNNs. A learning algorithm receives as input
a training set $S$, sampled from an unknown distribution $D$ and labelled by a
function $f^{\prime}$, the restriction to $S$ of some unknown target function
$f$ defined on $D$. Hence we represent the learning algorithm as an operation:

$\mathcal{L}:(r,f^{\prime})\mapsto f,$ (1)

where $r$ is the (presumably random) function implemented by the network prior
to training. The target function $f$ is unknown because we do not, in fact,
know how humans classify data sets, e.g. how humans identify dogs in images.
The goal of the algorithm $\mathcal{L}$ (implementing e.g. some form of error
backpropagation) is to construct, using $S$ as data, a classifier or predictor
$h_{S}$ belonging to a class of hypotheses $H$ that minimizes the error
$L_{D}$ (the true error) with respect to the unknown $D$ and $f$. The error of
a classifier $L_{D}(h)$ is the probability that it does not predict the
correct label for an unseen instance $x$, i.e. not present in the training set
S, drawn from $D$. As $f$ is not known, what counts as a “correct” label is
ill-defined; in practice, it is defined ostensibly by a human choice. Since
both $D$ and $f$ are unknown, we will calculate the training error rather than
the true error based on the dataset $S$. The problem will thus correspond to
the minimization of the empirical risk function $R_{\rm emp}(h)$, namely the
estimate of the Empirical Risk Minimization (ERM). The empirical risk
function,

$R_{\rm emp}(h)=\frac{1}{n}\sum_{i=1}^{n}L(h(x_{i}),y_{i})\,,$ (2)

is defined as the average over a training set of the loss function $L$, which
is such that $L\neq 0$ if $h(x_{i})\neq y_{i}$ and $L=1$ otherwise. For a
given class $H$ and a training sample $S$, the $ERM_{H}$ learner uses the ERM
rule to choose a predictor $h\in H$, with the lowest possible error over $S$.
Formally, $ERM_{H}(S)\in argmin_{h\in H}L_{S}(h)$. The empirical error of $h$
is then its average error over the training set $S$, while the generalisation
error is its expected error based on the distribution $D$. The accuracy of
this expectation clearly depends on the representativeness of $S$ as a sample
of $D$, and hence the representativeness of the training labels $f^{\prime}$
as a sample of the unknown target function $f$.

If $S$ does not adequately represent $D$, the ERM rule might easily lead to
overfitting, with the consequence that the model will generalize poorly.
Therefore, we need to search for conditions that allow the learning algorithm
to show good performance on the training set $S$ without underperforming on
the underlying data distribution $D$. In other words, we will need to find the
right balance between underfitting and overfitting. Typically the balance will
be achieved by selecting the most appropriate capacity111The capacity of a
model, defined in Sec. V, specifies the hypothesis class from which a learning
algorithm $\mathcal{L}$ can choose a specific function _$\mathfrak{h}$_. of
the class $H$ and/or resorting to regularization techniques (e.g. early
stopping, weight decay, dropout, etc.). For instance, a traditional approach,
rather than fixing in advance the complexity of the class, consists in
building a hierarchy of classes $H_{c}$ splitting the complexity of a richer
class $H$ in which $c$ is the complexity parameter. A method that follows this
approach is given by Support Vector Machines (SVMs) where the complexity of
the class is chosen based on the data by making use of Reproducing Kernel
Hilbert spaces (RKHS) Bartlett et al. (2021). A methodology of this kind is
looking for the best trade-off between the minimization of the empirical risk
and the complexity of the class $H_{c}$.

The goal of generalisation theory is to explain and justify why and how
minimizing the computable empirical error (the training error) is a reasonable
approach to minimize the non-computable error (the test error) by analyzing
the generalisation gap test error – training error. This challenge stems from
the dependence of the predictor $h_{S}$ on the same dataset $S$ in the case of
both the training error and the test error. However, as noted above, the
training set $S$ might not be sufficient to direct the learner towards a good
classifier if the training sample is insufficiently representative of the
underlying distribution $D$ that the learner intends to capture. In a ML model
one is by definition more interested in the generalisation error rather than
in the empirical error to the extent that our main goal is evaluating the
performance of the model on the unseen cases, i.e. on $D\setminus S$. If we
look at the problem from this point of view, then the challenge comes from the
mismatch between the optimization task consisting of minimizing the empirical
or training risk and the machine learning task consisting of minimizing the
true or test risk insofar as:

$L_{D,f}(h)=P_{x\sim D}[h(x)\neq f(x)]=D[\\{x:h(x)\neq f(x)\\}],$

where the error of such $h$ is the probability of randomly drawing an example
$x$, according to the distribution $D$, for which $h(x)\neq f(x)$ and the
error is measured with respect to the probability distribution $D$ and the
correct labeling function $f$. Here $P$ is the probability of a random
variable, and $x\sim D$ represents sampling $x$ according to $D$. This
probability can also be expressed in measure theory terminology. Formally,
given a member, $A$, of some $\sigma$-algebra of subsets of the domain
$\mathcal{X}$, the probability distribution $D$, assigns a number, $D(A)$,
which determines how likely it is to observe a point $x\in A$.

The dependence of the predictor $h$ on the training data set $S$ has led to
the definition of several examples of complexity measures providing bounds on
the test error or on the sample-complexity. An instance of these measures are
the VC-dimension or the Rademacher complexity. In particular, if the VC-
dimension provides an upper bound on the test error, the Rademacher complexity
measures the richness of a class of predictors with respect to a probability
distribution. These results provide bounds on the test error that depend on
the complexity or capacity of the class of functions $H$. In this sense,
generalisation theory and capacity control are strictly related. Capacity
control consists in using models that are rich enough to get good fits without
using those which are so rich that they overfit.

Yet, the empirical success of DNNs, which are typically overparametrised
models Frankle and Carbin (2019), challenges the traditional complexity
measures insofar as, according for instance to the VC-dimension, the
discrepancy between training error and generalisation error is bounded from
above by a quantity that grows at least linearly as the number of adjustable
parameters grows but shrinks as the number of training examples increases,
with the consequence that the traditional complexity measures tend to control
the capacity by minimizing the number of parameters Vapnik and Chervonenkis
(1982); Blumer et al. (1989); Haussler (2018). Moreover, experimental results
prove that DNNs that have been trained to interpolate the training data
achieve a near-optimal test result even when the training data have been
corrupted by a massive amount of noise Belkin et al. (2018). As Poggio et al.
Poggio et al. (2017) write, the main puzzle of DNN revolves around the absence
of overfitting despite large overparametrization and despite the large
capacity demonstrated by zero training error on randomly labeled data. It is,
in other words, the fact that DNNs can implement an operation
$\mathcal{L^{\prime}}:(r,g)\mapsto f$ that (at least approximately) correctly
generalizes to the desired $f$ from a function $g$ that is not the restriction
of $f$ to the training set $S$, but rather a noise function, namely a function
of randomized data, on $S$.

Historically the puzzle has been raised by a seminal paper from Zhang et al.
Zhang et al. (2017); see also Neyshabur et al. (2015a). This paper has shown
that successful deep model classes have sufficient capacity to memorize
randomised data sets while having the ability to produce zero training error
for particular natural datasets (e.g. CIFAR-10). The authors also empirically
observed that explicit regularization on the norm of weights seemed to be
unnecessary to obtain small test errors, in contradiction to “traditional
wisdom”. In the case of DNNs, the large capacity of the model looks sufficient
to memorize the training data by brute force. However, this behavior conflicts
with the traditional idea of generalisation since learning by explicit
memorisation of training examples should not imply generalisation
capabilities. Generalising is traditionally understood as learning some
underlying rule associated with the data generation process, i.e. learning
some compact representation of the training function $f^{\prime}$, and
therefore being able to extrapolate that rule from the training data to new
unseen data. Moreover, as we have already mentioned, a result of this kind is
a challenge to traditional complexity measures and, in general, to
computational learning theory since none of the existing bounds produces non-
trivial results for interpolating solutions of the sort generated by such DNN
models.

Several efforts have recently focused on achieving non-vacuous generalisation
bounds for DNN models Kawaguchi et al. (2017); Neyshabur et al. (2017); Arora
et al. (2017). In the light of this situation, Belkin et al. Belkin et al.
(2019) propose to subsume the traditional U-shaped risk curve describing the
trade-off between underfitting and overfitting with a double-descent risk
curve. In their view, when the number of parameters $N$ is much smaller than
the sample size $n$, i.e. $N<\\!\\!<n$, the traditional complexity measures
assume that the training risk is close to the test risk. However, according to
the double descent risk curve that the authors propose, by increasing
progressively $N$ and thus by increasing the class capacity, the model will
also increase the function classes until they are rich enough to achieve zero
training error, with the consequence of constructing progressively near-
perfect fit functions that typically show a smaller norm and, consequently,
are simpler in the sense of the Occam’s razor. A view of this kind contradicts
the traditional framework according to which, by increasing the class
capacity, the predictors will perfectly interpolate the data with the
consequence of achieving an overfitting regime and, therefore, of having a
very high generalisation risk. Yet, as the authors state, increasing further
the function class capacity beyond this point, leads to decreasing test risk,
typically below the risk achieved when balancing underfitting and overfitting
under the traditional approach (see Figure 1).

Figure 1: Risk curves in classical and modern regime.

This view matches recent results from Poggio et al. Poggio et al. (2017) and
Bartlett et al. Bartlett et al. (2021) according to which overparametrisation
leads to “benign overfitting” in which an accurate level of generalisation is
achieved despite a near-fit to training data. In particular, according to the
authors, overparametrisation enables gradient techniques to impose
regularization implicitly while leading to accurate predictions despite their
overfitting behaviour. A behavior of this kind will prefer matrices showing a
smaller norm, a result that, according to a number of a scholars, matches the
thesis that the generalisation performance depends on the size of the weights
rather than on the number of weights Neyshabur et al. (2015b); Bartlett et al.
(2017); Neyshabur et al. (2018); Shawe-Taylor et al. (1998); Bartlett and
Mendelson (2002). A thesis of this kind will guide our approach as well, even
though it will be embedded into a theoretical framework totally different from
complexity theory. In particular, we wish to unveil the source of
generalisation in DNNs by deploying techniques borrowed from Topological
Quantum Physics (TQP). Our proposal is based on two hypotheses, both grounded
on results coming from TQP and, in particular, from Topological Quantum Field
Theory (TQFT): a) TQNNs generalise by encoding topological, not just metric,
invariants that characterize their training sets; b) DNNs can be represented
as the semi-classical limit of a class of TQNNs. As we will see, in the TQNN
framework, generalisation is achieved as the selection of topological
invariants that are captured in the TQP approach by the linking and knotting
quantum numbers assigned to graphs and to the 2-complexes of TQNNs.

##  III TQNN in TQFT

Before tackling the issue of generalisation, elaborating on the results
attained by Zhang et al., Zhang et al. (2017, 2021), we summarize in this
section the novel strategy rooted in the framework provided by TQFT Marcianò
et al. (2022). This is an effective quantum theoretic approach that offers the
pathway to address the problem of generalisation. We will show that the origin
of this problem can be related to the topological encoding within the network
structure, achieved through path selection and parameter optimisation, which
coincide, in the classical limit, with the training of classical neural
networks. In addition, we argue that training TQNNs naturally implements,
through the selection of topological features, an optimisation over the
architecture of the corresponding classical neural networks in the semi-
classical limit. This is in stark contrast with the classical case, where the
architecture of the network is fixed beforehand, and the weights are learned
during training. Consequently, optimal architecture selection and
generalisation become, in the semi-classical limit, different interpretations
of the same underlying TQNN procedure.

Within the framework of TQNN introduced in Marcianò et al. (2022), both the
elements of the training and test samples are associated to quantum states —
the boundary states of the underlying TQFT — that are colored with irreducible
representations of Lie groups. In practice, these boundary states are
represented by spin-networks that are dual to triangulations of a manifold.
They encode the discretisation of space-time in that they combinatorially
represent the continuous underlying manifold. In this context, the $G$-bundle
structure of the manifold is encoded in terms of group elements on the edges
of the spin-network. Quantum states are represented by cylindrical functionals
of the boundary group elements that are associated to the input data, and are
supported on boundary graphs (1-complexes). Output boundary states represent
instead the TQNN ability to react to solicitations provided by the input
(training/test samples). The functorial evolution from input to output
boundary states is captured by 2-complexes, which realise the sum over
histories proper of path integral in quantum mechanics. The physical scalar
product between boundary states is then used to produce a numerical output
that later determines the train and test errors.

On the one hand, the “input to output” functorial evolution realises a sum
over all the geometries of the system, similarly to the Misner-Hawking
integral (see Rovelli (2011)). Certain geometries resonate giving rise to
dominating terms in the output of the TQNN. In other words, the structure of
the data induces TQNNs to select certain geometries. On the other hand, the
learning procedure here consists of an optimisation on the parameters of the
heat kernel corresponding to coherent spin-network states (see e.g. Bianchi et
al. (2010)). Therefore, the resonances are determined by the integral over the
histories learned during training. This duplicity reflects the double role of
learning in TQNNs, where both the architecture and the weights are learned, as
previously mentioned.

Therefore our working framework, developed within an extended effective
framework that follows the very same axioms of quantum field theories Marcianò
et al. (2022), hinges on the following steps:

  1. 1.

To data we associate (possibly a superposition of) spin-networks through some
encoding procedure (see Marcianò et al. (2022)). The spin-networks are
determined by the irreducible representations that label their edges and live
in the boundary Hilbert space of the TQFT theory.

  2. 2.

The generic boundary states are characterized by two classes of parameters,
which we dub as topological and metric parameters. The former ones are
determined by the topology of the graph supporting the spin-networks and the
topology of the manifolds, whose simplicial decompositions are dual to the
spin-networks. These are quantum invariants obtained through state sums of
admissible states. The metric parameters relate to the differential structure
of the manifold and appear in the spin-network formalism from the fact that
spin-network edges support the connection of the underlying manifold. These
parameters are captured by the spin of the representation itself.

  3. 3.

Information provided by the training samples, together with the analogical
definition of training and test error, in terms of the internal product of
boundary quantum states, allows to fix the functorial structure of the bulk of
the TQNN, namely the topological structure of the TQNN 2-complex. This
procedure determines the topological parameters, since it gives the quantum
groups in terms of which the topological invariants are expressed. We also
observe that the weights of the boundary coherent states are learned during
the training process.

  4. 4.

The topological parameters are enough to capture the pattern underlying the
training set; whenever the “information” about the topology specified by the
training data is not sufficient and/or noise exceeds the possibility to
individuate topological structures, an interpolating pattern which does not
select specific topological features is then deployed.

  5. 5.

The metric parameters are individuated by the Gaussian weights associated to
the coherent group elements assigned to the TQNN states. In the semi-classical
limit these weights correspond to the matrix weights of DNNs. For instance, in
Marcianò et al. (2022) we recover the perceptron as the semi-classical limit
of a specific TQNN.

  6. 6.

The measure of the Hilbert spaces associated to the links, that are deemed to
be equivalent to the training sample set, characterises the minimal amount of
information flow required to achieve pattern identification: if not enough
information is provided by the group elements, which is determined through the
association of spin-network states to input data, not enough information is
provided to reconstruct the topological invariants.

  7. 7.

The “richness” or “energy” of the irreducible representation sets allows to
“switch on” the links, and thus the nodes and the topological linking and
knotting invariants, only for non-trivial (non-zero) values of the spin
representations. Thus, the analogue of the minimal length in theories of
quantum gravity, or Planck length, is the minimal spin, or information bit, in
the framework of Marcianò et al. (2022).

Relying on this framework, we propose in the next section that generalisation,
within the context of classical DNNs, emerges from the optimisation in the
semi-classical limit of the TQFT path-integral (states’ sum) deployed to
estimate the classifier. Moreover, topological features, including topological
invariants (states’ sum) of the underlying TQFT and graph connectivity of
spin-network boundary states, are essential elements to achieve
generalisation. In fact, as argued in more detail in Section IV, we have that
learning corresponds to singling out preferred topologies in the form of
resonances in the transition amplitudes. These are dominant terms in the
partition function induced by the path-integral of the TQFT.

Change of the graphs’ topology is achieved at the (infinite number of quantum)
hidden layers by vertices structures implementing the TQNN evolution. Through
the topological features of the 1- and 2- complexes, the TQNN can capture the
topological invariants from the training sets. On the other hand, parameter
optimisation in the states’ sum corresponds to the individuation of the semi-
classical limit of the quantum theory: the “classical” path is individuated by
the extremisation of the action of the theory in the sum over histories;
fluctuations around that path still enable to achieve generalisation, as a
minimisation of the training error.

Within this novel picture, topological and metric data captured by the TQNNs’
structure increase the class capacity and thus ultimately the function classes
of the quantum ML models under scrutiny. Indeed, refining the
triangulation/tessellation of geometric manifolds underlying the description
of data ensembles results into an increase — either in the simplicial
skeletons extracted by manifolds triangulation/tessellation or in their dual
1-complexes — of the topological connectivity of the graphs on which TQNNs are
supported. While increasing the ensemble of metric data saturates the risk
curve of Figure 1 to provide a standard U-shaped form, the complexity of the
TQNNs graph structure that encodes the topological data provide an asymptotic
improvement of generalisation. For TQNNs graph structures that are not
sufficiently enough rich, the model, only characterised by the flow of metric
data along the links of the 1-complexes, undergoes overfitting as the ensemble
of metric informations increases. Nonetheless, increasing the graph complexity
through enlarging the topological data ensembles enables metric data to
increase function capacity. This suggests that the one-dimensional curves that
appear in Figure 1 should be replaced by a curve that takes into account two
different independent classes for the parameters characterising the models
under scrutiny, to be represented respectively on the axes for the metric and
the topological complexity, as depicted in Figure 2.

Figure 2: Topological and metric complexity.

##  IV Generalisation in the framework of TQNN

Our perspective relies on the conjecture that generalisation happens as the
analogue of the macroscopic manifestation of quantum mechanical effects.
Specifically, generalisation can be addressed as the manifestation of
topological (quantum) encoding achieved in TQNN and classical path-selection.

The texture of the webs of vertices and edges, which is determined by the data
and captures the topology of the (triangulated) manifold as its dual
simplicial complex, concretely implements a higher level topological pattern
Marcianò et al. (2022), unveiling which path coincides with the procedure of
learning. Generalisation is consequently achieved in this framework as the
selection of topological invariants, the most adequate to the achievement of a
specific task. These topological invariants are captured in the TQP framework
by the linking and knotting quantum numbers assigned to graphs and to the
2-complexes of TQNN.

###  IV.1 The notion of generalisation in DNNs as semi-classical limit of
TQNNs

Let us now consider in detail the issue of generalisation in TQNNs, and
consequently attempt to answer the problem raised in Zhang et al. (2021) for
DNN. When a coherent states representation of the Hilbert space is chosen,
this corresponds to the selection of a family of states, in the training
sample dataset, which is characterized by a random assignment of the
irreducible group representation labels. The representation labels are in turn
peaked around the labels of the coherent states that capture the mean values
of the observable quantities and are determined through the quantum annealing
procedure. This represents therefore a natural definition of labels’
randomisation in the training set. Label randomisation can be compared to
randomly defining the elements of the Hilbert space, as this corrupts the
correspondence between underlying data and correct label.

A classical DNN has only to learn the function $f^{\prime}$ in Eq. (1), which
specifies example input-output pairs; it has no access to the “intrinsic”
structure of the training examples. TQNNs, however, are sensitive to such
intrinsic structure in the form of topological features. The structure of
TQNNs naturally encode topological charges through the functorial quantum
dynamics ensured by the 2-complexes, which create either vertices and then
functions of $j$-representation and intertwiner quantum numbers, i.e.
amplitudes, or other topological charges encoded in the knotting and linking
of the edges in the bulk of the 2-complex.

For clarity, let us consider the case of the TQNN analogue of the classical
architecture of perceptron. As shown in Marcianò et al. (2022) Section 5, the
classical perceptron is recovered as the semi-classical limit of a TQNN. In
fact, considering the composability properties of TQNN, inherited from the
functoriality of TQFTs (the Atiyah axioms), it follows that this procedure
describes also the multilayer perceptron (MLP) in the semi-classical limit. In
particular, we argue that the semiclassical limit, which singles out the
classical trajectories from the infinite amount of quantum trajectories,
corresponds to the minimisation of the weight vector. For perceptrons, the
components of the weight vector are determined by the minimisation of the loss
functional or the learning task at hand. In the framework of TQNN, the weight
vector is substituted by the set encoding the values of the action at the
nodes and links of the boundary states — see Marcianò et al. (2022) Section 5
— and at the infinitely many quantum hidden layers interpolating among the
boundary states. The minimisation process that constitutes learning,
therefore, in this case consists in the minimisation of a Feynman path-
integral (associated to the TQFT), and it therefore amounts to selecting the
dominant contributors to the infinitely many paths.

When the action is minimised, the weight amplitudes of the path integral are
maximised. On the other hand, minimisation of the action ensures selection of
the classical paths. In turn, tiny fluctuations around these minima correspond
to the semi-classical paths nearest to the classical ones.

$x$$t$$(x_{1},t_{1})$$(x_{0},t_{0})$$t$$t^{\prime}$$t^{\prime\prime}$$t^{\prime\prime\prime}$
Figure 3: Possible paths in $xt$-plane.

The case of DNNs like MLPs consists of composing (via functoriality of the
given TQFT) single units as in the case of the perceptron. The paradigm
described above is substantially unchanged.

###  IV.2 Generalisation for the special case of one-node states

The simplest example we can provide is the one that concerns the quantum
mechanics of a one-particle system. This would correspond within the DNN/TQNN
correspondence to an architecture where the boundary states consist of a
single node. Hidden (DNN) and quantum (TQNN) layers differ in this picture by
their cardinality, i.e. the number of copies interpolating among the boundary
(one-node) states, and their size, i.e. the number of nodes at each
intermediate layer. This difference is substantially prompted by the fact that
classical DNNs have a fixed architecture, which corresponds to a single path
in the Feynman formulation, while TQNNs superpose all the possible paths.
Hidden layers in DNNs are indeed finite in number of copies and finite in
size, while quantum layers can be also infinite in number of copies and in
size. DNNs would then account for a finite sum over a finite set of
intermediate steps, while quantum mechanics realizes a sum over infinite
possible quantum states. Furthermore, DNNs assign probability weights among
the edges connecting one-node states to nodes of the hidden layers, while
TQNNs assign to the internal edges probability amplitudes, i.e. complex
numbers the absolute value square of which are probabilities. The infinite
number of quantum layers between input and output layers arises from the fact
that any TQFT used to define a TQNN is invariant under triangulation changes,
i.e. it is invariant under Pachner moves, and therefore arbitrary refinements
of the simplicial decomposition between input and output are all taken into
account by the TQFT. A related perspective concerns the Matveev-Piergallini
moves, as described in Kauffman et al. (1994) for the quantum group
$\mathcal{U}_{q}(\mathfrak{sl}_{2})$.

Representing then boundary states with one-particle quantum states of quantum
mechanics, we can associate

$\displaystyle|{\rm in}\rangle\rightarrow|x_{\rm in},t_{\rm in}\rangle\,,$
$\displaystyle|{\rm out}\rangle\rightarrow|x_{\rm out},t_{\rm out}\rangle\,,$
(3)

where the Cartesian coordinate $x$ labels the particle position (in a one-
dimensional linear space) and $t$ denotes the time parameter deployed to
follow its dynamical evolution.

Suppose now to partition $t\in[t_{\rm in},t_{\rm out}]$ in $N-1$ steps of
interval $\Delta t=(t_{\rm out}-t_{\rm in})/(N-1)$. While in TQNNs we take the
limit $N\rightarrow\infty$, when considering the semi-classical limit for DNNs
$N$ is fixed to be a finite number. Now, from the perspective of TQNNs, we
estimate the classifier $\mathcal{Z}_{\ell}(x_{\rm in},x_{\rm out})$ as the
partition function of the one-particle system, with $x_{\rm in}$ and $x_{\rm
out}$ labelling the boundary states, and $\ell$ labelling the infinite set of
all the possible paths connecting $x_{\rm in}$ to $x_{\rm out}$, as determined
by the partition function of the underlying TQFT. This means that

$\mathcal{Z}_{\ell}(x_{\rm in},x_{\rm out})=\langle x_{\rm out},t_{\rm
out}|x_{\rm in},t_{\rm in}\rangle=\langle x_{\rm N},t_{\rm N}|x_{\rm 1},t_{\rm
1}\rangle\,.$ (4)

Now we can decompose $\mathcal{Z}_{\ell}(x_{\rm in},x_{\rm out})$ into the
product of the corresponding intermediate steps (using the functoriality of
the theory) and get

$\displaystyle\langle x_{\rm N},t_{\rm N}|x_{\rm 1},t_{\rm 1}\rangle\\!$
$\displaystyle=$ $\displaystyle\\!\int dx_{N-1}\int dx_{N-2}\dots\int
dx_{2}\,\times$ $\displaystyle\langle x_{\rm N},t_{\rm N}|x_{\rm N-1},t_{\rm
N-1}\rangle\times$ $\displaystyle\langle x_{\rm N-1},t_{\rm N-1}|x_{\rm
N-2},t_{\rm N-2}\rangle\dots\langle x_{\rm 2},t_{\rm 2}|x_{\rm 1},t_{\rm
1}\rangle\,.$

Within each interval of time $\Delta t$, we consider the Feynman amplitude

$\langle x_{k},t_{k}|x_{k-1},t_{k-1}\rangle=\frac{1}{w(\Delta t)}\,e^{\imath
S(k,k-1)}\,,$ (6)

where

$S(k,k-1)\equiv\int_{t_{k-1}}^{t_{k}}dt\,L_{\rm classical}(x,\dot{x})\,,$ (7)

is the classical action evaluated in the $k$-th time interval $\Delta t$, with
$k=1,\dots N-1$.

Notice that the generic $k$-th amplitude has the meaning of a propagator from
the point $\\{x_{\rm k},t_{\rm k}\\}$ to the point $\\{x_{\rm k-1},t_{\rm
k-1}\\}$, since the generic intermediate states $|x_{\rm k},t_{\rm k}\rangle$
and $|x_{\rm k-1},t_{\rm k-1}\rangle$ evolve according to the Schrödinger
picture, i.e.

$\displaystyle\langle x_{\rm k},t_{\rm k}|x_{\rm k-1},t_{\rm k-1}\rangle$
$\displaystyle=$ $\displaystyle\langle x_{\rm
k}|e^{-\frac{\imath}{\hbar}H\Delta t}|x_{\rm k-1}\rangle$ (8) $\displaystyle=$
$\displaystyle K(x_{\rm k},t_{\rm k};x_{\rm k-1},t_{\rm k-1})\,.$

By construction, the propagator $K(x_{\rm k},t_{\rm k};x_{\rm k-1},t_{\rm
k-1})$ satisfies the Schrödinger time-dependent wave equation, and the
property

$\lim_{t_{\rm k}\rightarrow t_{\rm k-1}}K(x_{\rm k},t_{\rm k};x_{\rm
k-1},t_{\rm k-1})=\delta(x_{\rm k}-x_{\rm k-1})\,.$ (9)

The propagator hence determined is nothing but the Green’s function of the
time-dependent Schrödinger wave equation, i.e.

$\displaystyle\left[H(x_{\rm k})-\imath\hbar\frac{\partial}{\partial t_{\rm
k}}\right]K(x_{\rm k},t_{\rm k};x_{\rm k-1},t_{\rm k-1})=$
$\displaystyle\delta(x_{\rm k}-x_{\rm k-1})\delta(t_{\rm k}-t_{\rm k-1})\,,$
(10)

where $H(x_{\rm k})$ is the differential representation of the Hamiltonian
operator in $x_{\rm k}$, and with boundary condition

$\displaystyle K(x_{\rm k},t_{\rm k};x_{\rm k-1},t_{\rm
k-1})=0\,,\qquad\forall t\notin[t_{\rm k-1},t_{\rm k}]\,.$ (11)

In (6), the factor in front of the exponential can only depend on the time
interval $\Delta t$. Being independent from the potential to which the
particle is subjected, it can be estimated by calculating from (IV.2) the
propagator of a non-relativistic free particle, with Hamiltonian
$H=\frac{1}{2}m\dot{x}^{2}=-\frac{\hbar^{2}}{2m}\nabla^{2}$, hence finding the
expression

$\displaystyle\frac{1}{w(\Delta t)}=\sqrt{\frac{m}{2\pi\imath\hbar\Delta
t}}\,.$ (12)

We can now consider the limit for which the partition acquires an infinite
amount of “filters”, and hence the time interval $\Delta t$ shrinks to zero.
Within this limit, each $k$-th amplitude will contribute according to

$\displaystyle\langle x_{\rm k},t_{\rm k}|x_{\rm k-1},t_{\rm
k-1}\rangle=\sqrt{\frac{m}{2\pi\imath\hbar\Delta
t}}\,e^{\frac{\imath}{\hbar}S(k,k-1)}\,,$ (13)

providing as a final expression for the transition from $|{\rm in}\rangle$ to
$|{\rm out}\rangle$ the relation

$\displaystyle\langle x_{\rm N},t_{\rm N}|x_{\rm 1},t_{\rm 1}\rangle$
$\displaystyle=$
$\displaystyle\lim_{N\rightarrow{\infty}}\left(\frac{m}{2\pi\imath\hbar\Delta
t}\right)^{\frac{N-1}{2}}\,\int dx_{N-1}\int dx_{N-2}$ (14)
$\displaystyle\dots\int
dx_{2}\prod_{k=2}^{N}\,e^{\frac{\imath}{\hbar}S(k,k-1)}\,.$

By the definition of $S(k,k-1)$ in (7), and denoting the sum over the paths as
$\mathcal{D}[x(t)]$, namely

$\displaystyle\int_{x_{1}}^{x_{\rm
N}}\mathcal{D}[x(t)]=\lim_{N\rightarrow{\infty}}\left(\frac{m}{2\pi\imath\hbar\Delta
t}\right)^{\frac{N-1}{2}}\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!$
$\displaystyle\,\int dx_{N-1}\times$ $\displaystyle\int dx_{N-2}\dots\int
dx_{2}$

we find the celebrated expression for the Feynman path-integral

$\displaystyle\langle x_{\rm N},t_{\rm N}|x_{\rm 1},t_{\rm
1}\rangle=\int_{x_{1}}^{x_{\rm
N}}\mathcal{D}[x(t)]\,e^{\frac{\imath}{\hbar}\int_{t_{1}}^{t_{N}}L_{\rm
classical}(x,\dot{x})}\,.$ (16)

When a free particle is considered, one can observe that the semi-classical
limit corresponds to the minimization of the action, hence to imposing the
stationarity of the norm of $X=p/\sqrt{2m}$. Taking into account relativistic
invariance, in a $d+1$-dimensional space-time manifold, the vector
$\vec{X}=\vec{p}/\sqrt{2m}$ will turn out to be $d$-dimensional. Thus the
dimensionality of the vector $\vec{X}$ equals the space dimension of the
ambient space-time manifold.

The interpretation we provide here is straightforwardly preserved every time
we consider a Lagrangian that is quadratic in the configuration variables and
their momenta. More in general, we can resort to a symplectic geometry
analysis to identify in full generality the norm of the vector $\vec{X}$ in
terms of an Hermitian inner product. This latter is generated by the
symplectic structure associated to the manifold — see e.g. Refs. Crnkovic and
Witten (1988); Crnković (1987); Crnkovic (1988) — and corresponds to the norm
of the action of the space-time translation generator applied to the
configuration fields of the system. In fact, when the Lagrangian presents
higher order terms, one can proceed by perturbing the path integral as shown
for example in Sawon (2006) for Chern-Simons theory. This allows to compute
higher order terms in the perturbation as described above. Moreover, this
paves the way to further parametrizations that can be learned during training
in the form of topological charges. This situation, albeit very interesting,
will not be considered explicitly in the present article.

Notice furthermore that once manifolds with Lorentzian signature are taken
into account, the extremization of the action would not correspond to a
minimization of $\vec{X}$, being the system hyperbolic. In this latter case,
the extremization of the action on a hyperbolic manifold provides the
classical trajectories/geodesics of the systems, namely the classical paths
that instantiate generalisation within the DNNs framework.

Therefore the estremisation of the weights, whose role in generalisation has
been commented in Sec. II referring to Neyshabur et al. (2015b); Bartlett et
al. (2017); Neyshabur et al. (2018); Shawe-Taylor et al. (1998); Bartlett and
Mendelson (2002), is replaced in this picture by the minimisation of the
action. This latter in turns corresponds to maximise the probability
amplitudes in the path integral formulation of a classifier.

$(x_{1},t_{1})$$(x_{0},t_{0})$ Figure 4: Predominant paths in the
$h\rightarrow 0$ limit.

We are now in the position to express our thesis on the generalisation
process. The partition function that determines the TQFT used by the TQNN
utilises a superposition of all colorings of the input/output pairs, and
intermediate states amplitudes, as shown above, give the single probabilities
(upon taking moduli squared) of the single transitions of color configurations
that constitute the summands of the partition function. Optimisation here
shows the dominant terms in the superposition function, according to the given
underlying ground truth. Thus generalisation emerges as an artifact of the
semiclassical limit: DNNs structures are only able to represents fixed
2-skeletons within the 2-complex evolution of TQNNs. These are nevertheless
the most dominant contributions to the path integral for the TQNNs, which is
in general realised by the topological quantum neural 2-complexes (TQN2Cs),
summing over all the quantum histories. This simple case with boundary states
with single nodes is generalised directly to the case of spin-networks with
higher number of nodes, where the reasoning still holds true. In fact, this
formulation was used in Marcianò et al. (2022).

###  IV.3 Generalisation for TQNNs

We have so far addressed the process of generalisation for DNNs as the
semiclassical limit of TQNNs, with particular emphasis on the case of single-
node boundary states. However, a more general question naturally arises,
regarding the notion of generalisation for TQNNs. In other words, we ask (and
provide an answer to) the question of what a TQNN learns, and therefore what
the generalisation procedure looks like in the quantum case.

Although the notion of TQNN seems to share similarities with graph neural
networks (GNNs), we observe that the two methods are substantially different.
In fact, GNNs are determined by a fixed geometry and, in addition, such
geometry determines the way information is processed through the learning
procedure (message passing). On the contrary, TQNNs have no fixed geometric
structure, but they are applied through their defining functorial rules on
graphs that correspond to spin-networks, and therefore elements of the
boundary Hilbert space, via some pre-determined assignment (see Marcianò et
al. (2022) Section 4 for such a concrete correspondence). Information from the
boundary states is then processed according to the defining rules of the given
choice of TQFT — quantum $\mathfrak{sl}_{2}$ in the case discussed in detail
in Marcianò et al. (2022). Since a TQFT is supported on the bulk manifold and
it is independent on the triangulation used, it follows that the information
contained in the boundary is processed according to methods that are not
determined by the geometric structure of the supporting boundary graph, as it
happens in the case of message passing for GNNs. This is a fundamental
perspective that TQNNs leverage to generalise.

TQNNs evolve according to the dynamics dictated by topological quantum field
theories (TQFTs). Transition amplitudes calculated according to TQFTs
instantiate classifier rules that interconnect TQNN states. As in Marcianò et
al. (2022), we focus on $BF$-extended theories, which is a theoretical
framework general enough to encode Yang-Mills gauge theories as well as
effective field theories. Specifically, TQNNs are states of the kinematical
Hilbert space of TQFTs. Their topological features are easily captured once
their expansion on the spin-networks basis is taken into account. Nonetheless,
an equivalent expansion in the multi-loop basis is also possible Rovelli and
Smolin (1995), which renders less evident the connectivity of graphs and thus
(some of) their topological features. Such expansion is obtained by unraveling
the Jones-Wenzl symmetrizer (projector) placed on each edge of the input
boundary, following the definition of spin-network state. For our purposes, it
suffices to consider the expansion of TQNNs on the spin-networks basis, found
e.g. in Baez (1996).

We consider a manifold $\mathcal{M}$, and a submanifold $\mathcal{S}$ such
that $\mathcal{M}=\mathcal{M}_{0}\cup_{\mathcal{S}}\mathcal{M}_{1}$, for
submanifolds $\mathcal{M}_{i}$, $i=0,1,$ with
$\partial\mathcal{M}_{i}=\mathcal{S}$. The one-complexes (graphs) $\gamma$ are
then considered as embedded in $\mathcal{S}$. A generic TQNN state can be
expanded on the elements of the spin-network basis, each one being supported
on a generic graph $\gamma\in S$. In turn, each spin-network state is defined
as a triple $\Psi=(\gamma_{\Psi},\rho,\iota)$ consisting of a graph
$\gamma_{\Psi}\in S$, an irreducible representation $\rho_{l}$ of $G$ for each
link $\gamma_{i}\in\gamma_{\Psi}$, an intertwiner $\iota_{n}$ for each node
$n$ such that

$\iota_{n}:\rho_{l_{1}}\otimes\dots\otimes\rho_{l_{n}}\rightarrow\rho_{l_{1}^{\prime}}\otimes\dots\otimes\rho_{l_{n}^{\prime}}\,,$
(17)

where the links incoming into the node $n$ have been denoted as $l_{1},\dots
l_{n}$, and the links outgoing from the node $n$ have been denoted with
$l_{1}^{\prime},\dots l_{n}^{\prime}$.

Without loss of generality, we may directly focus on a generic spin-network
state $\Psi$. Suppose now that $\Psi$ is the spin-network state associated to
some boundary initial state derived from a data-point, namely

$|{\rm in}\rangle\rightarrow|\Psi\rangle\,,$ (18)

where the correspondence is determined according to some given rule as in
Section 4 of Marcianò et al. (2022), for example. Similarly, introduce the
boundary final state $\Phi$, i.e.

$|{\rm out}\rangle\rightarrow|\Phi\rangle\,.$ (19)

Generic TQNN states $\Psi$ and $\Phi$ encode two types of data: topological
data, namely the connectivity of the graphs $\gamma_{\Psi}$ and
$\gamma_{\Phi}$; metric data, corresponding to the assignments of quantum
numbers to the links and nodes of the graph $\gamma_{\Psi}$ and
$\gamma_{\Phi}$, namely the irreducible representations $\rho_{l}$ for each
link $l\in\gamma_{\Psi}$ or $l\in\gamma_{\Phi}$, and the intertwiner quantum
number $\iota_{n}$ for each node $n\in\gamma_{\Psi}$ or $n\in\gamma_{\Phi}$.
Both the topological and metric data that are encoded in $\Psi$ and $\Phi$ are
processed by the classifier $\mathcal{Z}$. This latter is nothing but the
matrix element of the quantum evolution operator — in the path integral
representation — that associates the initial state $|{\rm
in}\rangle=|\Psi\rangle$ to the final state $|{\rm out}\rangle=|\Phi\rangle$.
The action of the evolution operator/classifier on the boundary states amounts
to the assignment of the transition amplitude $\langle\Phi|\Psi\rangle_{\rm
phys}$. Here, the subscript “phys” is reminding us that the amplitude is
calculated in terms of the dynamics of the specific TQFT taken into account,
thus it differs from the scalar product $\langle\Phi|\Psi\rangle$ calculated
in the kinematical Hilbert spaces to which the TQNN states $\Psi$ and $\Phi$
belong. In practice, the transition amplitude is one of the summands appearing
in the partition function that defines the chosen TQFT.

Once a specific TQFT is selected, and a gauge group $G$ is fixed, the states
of the TQNN can be represented as cylindrical functionals

$\Psi(H_{l}):=\langle H_{l}|\Psi\rangle$ (20)

that depend on the holonomies $H_{l}\in G$ along the links
$l\in\gamma_{\Psi}$. Holonomies are group elements of $G$ that realize the
parallel transport along a path $\gamma\subset S\subset\mathcal{M}$, with
respect to the connection $A$ over the principle $G$-bundle. The graph
$\gamma_{\Psi}$ here represents a discretization of the underlying manifold,
and its links represent small paths in the manifold — cf. the notion of edge
in lattice gauge theory. Denoting path-ordering with $P$, the parallel
transport of a vector in a representation $\rho$ of $G$ reads

$H_{l}=Pe^{\int_{l}A_{a}\tau^{a}}\,,$ (21)

with $a$ an index in the adjoint representation of $G$ and $\tau^{a}$ a
representation of the generators of $G$.

The classifier quantum amplitudes can be derived for an extended $BF$-theory
over a $G$-bundle. We consider a local trivialization on $\mathcal{M}_{d}$,
and denote the curvature of the $G$-connection $A$, which is a
$\mathfrak{g}$-valued 1-form, with a $\mathfrak{g}$-valued 2-form $F$,
$\mathfrak{g}$ standing for the Lie algebra of $G$. A frame field $B$ can be
introduced, as the field conjugated to $A$, and such that the symplectic
structure is fulfilled

$\omega((\delta A,\delta B),(\delta A^{\prime},\delta
B^{\prime}))=\int_{S}\langle\delta A\wedge\delta B^{\prime}-\delta
A^{\prime}\wedge\delta B\rangle\,,$ (22)

where the conjugated fields have been restricted on an initial (in time, for
the Lorentzian case) slice $\\{0\\}\times S$ of $\mathcal{M}_{4}$, and we have
denoted the trace over the internal indices with $\langle\dots\rangle$. Notice
that the frame field $B$ is a $\mathfrak{g}$-valued $2$-form. This allows to
write consistently the action of the $BF$-extended theory over either a
Riemannian or a Lorentzian $4$-dimensional manifold $\mathcal{M}_{4}$ as

$\displaystyle\mathcal{S}_{\rm BF}^{\rm
ext.}[A,B]\\!=\\!\\!\int_{\mathcal{M}_{4}}\\!\\!\\!\langle B\wedge
F+\lambda_{1}B\wedge B+\lambda_{2}B\wedge\star B\rangle\,,\ \ \ \ $ (23)

with $\lambda_{1},\lambda_{2}\in\mathbb{R}$ bare coupling-constants. For
$\lambda_{1}\neq 0$ and $\lambda_{2}=0$, the theory is still topological, and
corresponds to the Crane-Yetter model, whose quantization involves recoupling
theory of quantum groups — see e.g. the notable example that corresponds to
$G=SU(2)$. For $\lambda_{1}=0$ and $\lambda_{2}\neq 0$, the theory is non-
topological and corresponds on-shell to a Yang-Mills action with internal
gauge group $G$. Furthermore, when the Lorentzian case with
$G=SL(2,\mathbb{C})$ is taken into account, and $\lambda_{1}$ is promoted to a
multiplet of scalar fields, with a pair of symmetric indices that are in the
adjoint representation of $SL(2,\mathbb{C})$, one can recover the Einstein-
Hilbert-Holst action of gravity for $4$-dimensional space-time Fields et al.
(2022a).

For the sake of simplicity, we focus on the topological realization of the
$BF$-theory that corresponds to selecting $\lambda_{1}=\lambda_{2}=0$. The
equations of motions then read

$F=0\,,\qquad\qquad d_{A}B=0\,,\,$ (24)

where $d_{A}$ denotes the covariant derivative with respect to the
$G$-connection $A$.

The classifier evolution is easily recovered in terms of the instantiation of
the curvature constraint $F=0$ in a physical projector $\mathcal{P}$. In turn,
the implementation of the physical projector $\mathcal{P}$ was discussed in
Noui and Perez (2005), where it was shown how to make sense of the formal
expression

$\mathcal{P}=\int\mathcal{D}N\exp(\imath\int_{S}\langle
N\,\widehat{F}\rangle)\,,$ (25)

with $N$ a $\mathfrak{g}$-valued Lagrangian multiplier and $\imath=\sqrt{-1}$
imaginary unit.

In particular, the curvature constraint $F=0$ can be implemented in the
physical amplitudes among spin-network states according to

$\langle\Psi,\Phi\rangle_{\rm phys}=\langle\mathcal{P}\Psi,\Phi\rangle\,.$
(26)

A local patch $\Sigma\in S$ can be considered that is provided with cellular
decomposition composed of squares of infinitesimal coordinates length
$\delta$. The regularized curvature constraint then reads

$F[N]=\int_{\Sigma}\langle NF(A)\rangle=\lim_{\delta\rightarrow
0}\sum_{p^{j}}\delta^{2}\langle N_{p^{j}}F_{p^{j}}\rangle\,,$ (27)

in which ${p^{j}}$ labels the $j$-th plaquette and $N_{p^{j}}$ is the
discretization of the $\mathfrak{g}$-valued Lagrangian multiplier $N$,
evaluated at an interior point of the plaquette ${p^{j}}$.

The holonomy $H_{p^{j}}[A]$ around the plaquette ${p^{j}}$ is a $G$ group-
element that casts

$H_{p^{j}}[A]=1\\!\\!1+\delta^{2}F_{p^{j}}(A)+\mathcal{O}(\delta^{2})\,,$ (28)

which implies

$F[N]=\int_{\Sigma}\langle NF(A)\rangle=\lim_{\delta\rightarrow
0}\sum_{p^{j}}\delta^{2}\langle N_{p^{j}}H_{p^{j}}[A]\rangle\,.$ (29)

The regularized expression for the action of the physical projection operator
immediately follows. This enters the physical scalar product of spin-network
states:

$\displaystyle\langle\Psi,\Phi\rangle_{\rm phys}$ $\displaystyle=$
$\displaystyle\lim_{\delta\rightarrow
0}\langle\prod_{p^{j}}\int\mathcal{D}N_{p^{j}}\exp(\iota\langle
N_{p^{j}}\widehat{H}_{p^{j}}\rangle)\Psi,\Phi\rangle$ (30) $\displaystyle=$
$\displaystyle\lim_{\delta\rightarrow
0}\langle\prod_{p^{j}}\delta(H_{p^{j}})\Psi,\Phi\rangle\,.$

Having reminded the structures of TQNNs and their TQFT evolution in terms of
TQN2C classifier, we can now address the problem of generalisation in this
extended theoretical framework. We may assume that the size of the training
data is sufficient to select or, better, to learn specific paths in the
boundary graphs and bulk 2-complexes within the most general available TQNN
architecture. These paths are characterised by different types of associated
non-perturbative topological charges, which in turn provide the sub-structures
involved in the generalisation process, as a subset supported on general
2-complexes. More concretely, these correspond to summands in the partition
function of the TQFT that is used to define the TQNN, and determine the
dominating contributions to the topological invariants. In other words,
certain intrinsic algebro-geometric features emerge that characterize the
learning of the TQNN.

The BF-extended formulation provided by (23) encode Yang-Mills theories over a
$G$-bundle for the choice $\lambda_{1}=0$ and $\lambda_{2}=2g^{2}_{\rm YM}$.
Thus the optimization of the classifier will correspond to the minimization of
the classical Yang-Mills action, namely

$\displaystyle\mathcal{S}_{\rm YM}=\frac{1}{2g^{2}_{\rm
YM}}\int_{\mathcal{M}_{4}}\langle F\wedge\star F\rangle\,.$ (31)

Cast in terms of the spacetime components, with $a$ denoting the indices of
the adjoint-representation of the algebra $\mathfrak{g}$, the Lagrangian
density $F_{\ \mu\nu}^{a}F^{a\ \mu\nu}$ turns out to be proportional to the
norm of a tensor on a hyperbolic manifold with Lorentzian signature.

Path selection is attained by minimisation of the classical action and
extremisation of the path-integral formulation of the classifier, which is
naturally realised in the semi-classical limit, in a way that is reminiscent
of the free energy principle formalism Friston (2010a, b, 2019); Fields et al.
(2022b).

The topological charges that are switched on over the learning process and
correspond to the configurations that extremise the action, together with the
corresponding metric properties, implement effectively the generalisation
process. In this sense, our approach is expected to provide a solution to the
problem as raised by Zhang et al, 2016. In particular:

  * •

The randomisation of the labels of a TQNN state will not induce overfitting as
a consequence of the encoding of information achieved by the TQNN through the
topological features and topological invariants. The quantum nature of the
TQNN will induce fluctuations around values of the parameters to be estimated.
Nonetheless, these fluctuations are small in the semi-classical limit.

  * •

However, a DNN architecture will be trapped into an overfitting regime until
memorising the training examples by brute force, since by definition of DNNs
the training error vanishes — the variance for the $j$ scale as
$1/\sqrt{\bar{j}}$. In other words, associating a DNN to a set of spin-
networks evaluated into coherent group elements, the corresponding training
error is zero.

Thus, brute-force learning is not possible for a TQNN, since the topological
information of the input/output states would automatically make the transition
amplitude vanish, most of the time, with random labelling.

##  V A dictionary for TQNNs

Along the lines specified in Marcianò et al. (2022), we propose once again in
this section a dictionary between TQNNs and the most relevant notions in
standard machine learning, including DNN theory.

It is useful to restrict our focus to supervised learning. This latter task
implements learning of a usually unknown function $f:D\rightarrow Y$ that maps
an input set $D$ to an output set $Y$, and is based on a training set
$S\subset D$ and a function $f^{\prime}:S\rightarrow Y$ that specifies example
input-output pairs. Considering $f:X\rightarrow Y$ as the (presumably random)
function $r$ implemented by the network before the training, the learning
algorithm can be specified, as in Eq. (1), as an operation
$\mathcal{L}:(r,f^{\prime})\mapsto f$. A statistical learning framework for
supervised learning can be then introduced, along the lines of Shalev-Shwartz
and Ben-David (2014), as well as some standard definitions for DNN that we
list below.

  * •

Sample complexity:
It represents the number of training-samples (i.e. $Card(S)$) that a learning
algorithm needs in order to learn successfully a family of target functions.

  * •

Model capacity:
It is the ability of the model to fit a wide variety of functions; in
particular, it specifies the class of functions $\mathfrak{H}$ (the hypothesis
class) from which the learning algorithm $\mathcal{L}$ can choose the specific
function _$\mathfrak{h}$_.

  * •

Overfitting:
A model is overfitting when the gap between training error and test error is
too large; this phenomenon occurs when the model learns the training function
$f^{\prime}$ but $\mathcal{L}$ incorrectly maps $(r,f^{\prime})\mapsto h\neq
f$, i.e. the trained network generalises to the wrong function $h$ and fails
to predict future observations (i.e. additional samples from $D$) reliably.
The training function $f^{\prime}$ has been merely “memorised” to the extent
that $h$ is incorrect (e.g. random) on $D$ outside of the training sample $S$.

  * •

Underfitting:
A model is underfitting when it is not able to achieve a sufficiently low
error on the training function $f^{\prime}$; this phenomenon occurs when the
model does not adequately capture the underlying structure of the training
data set and, therefore, may also fail to predict future observations
reliably.

  * •

Bias:
It is the restriction of the learning system towards choosing a classifier or
predictor _$\mathfrak{h}$_ from a specific class of functions $\mathfrak{H}$
(the hypothesis class).

  * •

Empirical Risk Minimisation (ERM):
It consists in minimising the error on the set of training data (the
“empirical” risk), with the hope that the training data is enough
representative of the real distribution (the “true” risk).

  * •

Generalisation:
It is conceived as the ability of the learner to find a predictor, i.e. an
embedding $S\rightarrow D$, which is able to enlarge successfully its own
predictions from the training samples to the test or unseen samples.

These notions can be reformulated into the dictionary of TQNNs.

  * •

Sample complexity:
It is a measure of the Hilbert-space of the entire spin-network state that is
supported on a specific graph $\Gamma$. It is then dependent on the
connectivity of the graph (nodes and links of each graph, i.e. the
multiplicity of connectivity that characterizes the graph $\Gamma$) and on the
dimensionality of the Hilbert spaces connected to each link and node. In this
sense complexity, once extended to the different classes of graphs
corresponding to the training set, provides a measure of the entropy of the
set. Therefore, in the TQNN framework, the notion of “complexity” has a wider
meaning than its counterpart in DNN, for which the sample complexity is
nothing but the size of the training set. This is summarised in the expression
for the dimension of the Hilbert space $\mathcal{H}_{\Gamma}$ of the (whole)
spin-network supported on $\Gamma$, namely

${\rm
dim}[\mathcal{H}_{\Gamma}]=\oplus_{j_{l}}\otimes_{n}\otimes_{l\in\partial
n}\,{\rm dim}[\mathcal{H}_{j_{l}}].$

This directly encodes both the size of the maximal graph where the
input/output states live, as well as the algebro/analytical structure used in
the TQFT from which the corresponding TQNN arises, as encoded by the
dimensionality of the Hilbert spaces $\mathcal{H}_{j}$, for instance;

  * •

Model capacity:
It is now distinguished in topological model capacity and metric model
capacity, the latter being the extension of the definition provided for DNNs
to the context of TQNNs.
i) Topological model capacity: It is quantified in terms of the
interconnectivity of the graph $\Gamma$. It depends on the topological
structure of the graphic support $\Gamma$ of the spin-network states, and
neither on the dimensionality of the Hilbert space of the irreducible
representations nor on the intertwiner quantum numbers, respectively assigned
to each link and node of $\Gamma$; in other words, it depends on the total
valence $V$ of $\Gamma$, defined in terms of the valences $v_{n}$ of each node
of $\Gamma$ through the expression

$V=\sum_{n}v_{n}\,;$

ii) Metric model capacity: At fixed graph $\Gamma$, it depends on the
dimensionality of the Hilbert space of the irreducible representations and
intertwiner quantum numbers assigned to $\Gamma$;
iii) Combined model capacity: It combines the topological and metric capacity,
so as to mimic the standard DNN notion of model capacity. It provides a a
representation of the double Belkin curve as in 2, which fits Figure 1.

  * •

Overfitting:
As pointed out in Section III, in the semi-classical limit, the integrals that
allow us to compute the transition amplitudes that characterise a TQFT are
interpreted as a “sum over all the geometries” of the ground topological
manifold, where the integrand is some approximation of the Einstein-Hilbert
action. During the learning process, then a TQNN learns how to select certain
geometries with respect to certain others in order to maximise certain
transition amplitudes corresponding to “a more suitable” classification. The
information available to make this selection during the learning process is
that given by the metric data, namely the irreducible representations and
intertwiner quantum numbers assigned to the TQNNs graphs, and by the
connectivity of the input graphs/spin-networks and their given correlation
$f^{\prime}$ with the label set $Y$. Keeping metric data fixed, if
$f^{\prime}$ is insufficiently representative of the target function $f$, the
TQNN may only partially capture the topological structure of the full input
set $D$ and therefore be unlikely to classify correctly spin-network states
that are not part of, or are significantly dissimilar from those contained in,
the training set $S$. Conversely, when connectivity is kept fixed, overfitting
follows the standard behaviour of DNNs. Then a U-shaped risk curve as in
Figure 1 describes the trade-off between underfitting and overfitting;

  * •

Underfitting:
It represents the converse of the overfitting scenario. The geometries that
have been selected in the learning process do not correspond to the graphs
$\Gamma$ at the starting point. Less information channels (links) are present,
and lower dimensionality of the information channels (dimensions of the
Hilbert space associated to each holonomy) as well. As a consequence, the TQNN
cannot fit the training set and may therefore also fail to predict future
observations reliably;

  * •

Bias:
It amounts to the predisposition of the spin-network to account for a specific
set of data; it depends on the topological structure of the spin-network
states, encoded in the connectivity properties of input $\Gamma$’s and on the
specific realisation of the TQNN quantum state, i.e. on the weight of the
quantum state on the spin-networks basis elements of the Hilbert space.

  * •

Empirical Risk Minimisation (ERM):
It is the variance of the Gaussian distribution of the irreducible
representations assigned to the holonomies on the links in the semi-classical
limit, i.e.

${\rm ERM}:=\sum_{l}\frac{(j_{l}-\bar{j}_{l})^{2}}{2L}\,,$

with $L$ equal to the total number of links.

  * •

Generalisation:
It is the behavior of the system in response to test or unseen data analogous
to a functor (amplitude) either from a boundary spin-network to another
boundary spin-network, or from a boundary spin-network to a complex number.
This is determined by the geometries that have been selected as the most
representative of a certain training sample during the learning process. This
is in practice captured by the parameters that give higher relevance, in the
integral computing the transition amplitudes in a TQNN, to certain boundary
transitions, while suppress others. These parameters are determined by (i)
connectivity of 1- and 2-complexes (nodes and links, vertices and edges
respectively), (ii) linking and knotting (e.g. for loops in a different
Hilbert space representation), and (iii) states’ sum (as a global topological
charge, invariant under refinement of the triangulation, i.e. invariant under
refinement of the data/group elements/intertwiners assigned to the links and
the nodes). In Marcianò et al. (2022), parameters enter the expression for the
amplitudes thanks to the coherent states formalism. For $U_{l}$ elements of a
group $G$ we may resort to the formula for the partition function of the
model:

$\displaystyle\mathcal{Z}_{\mathcal{C}}(U_{l})=\int_{{\rm
SU}(2)^{2(E-L)-V}}dU_{v(e)}\,\int_{{\rm SU}(2)^{\mathcal{V}-L}}dU_{f}\,$
$\displaystyle\times\prod_{f}\,\mathcal{K}_{f*}(U_{e*},U_{f})\,,$ (32)

where the “face amplitude” casts

$\displaystyle\mathcal{K}_{f*}(U_{e*},U_{f})\equiv\sum_{j_{f*}}\,\Delta_{j_{f*}}\,\chi^{\scriptscriptstyle
j_{\\!f*}}\\!\Big{(}\\!\prod_{e*\in\partial f}U_{e*}\\!\Big{)}\,$
$\displaystyle\times\prod_{e*\in\partial f}\\!\chi^{\scriptscriptstyle
j_{\\!f*}}(U_{f})\,.$ (33)

Finally, from the definitions of the present article, we can provide the
meaning of Learner’s input and output in the context of TQNN.

  * •

Learner’s input:
i) The domain set $D$: It corresponds to links $l$ and nodes $n$, and attached
holonomies $U_{l}$ and invariant tensors $\iota_{n}$ respectively along the
links and at the nodes: it is concisely denoted as a state of the Hilbert
space of the theory:

$\Psi_{\Gamma;\\{j_{l}\\},\\{\iota_{n}\\}}[A]\equiv\Psi_{\Gamma}(U_{l},\iota_{n}):=|\Gamma;\\{j_{l}\\},\\{\iota_{n}\\}\rangle;$

ii) The label set $Y$: It is a set of topological charges and quantum numbers,
with which the 2-complex is endowed; for instance, recalling the group-
isomorphism $\pi_{3}(S_{3})$, for the mapping individuated by the homotopy
group $\pi_{3}(S_{3})=\mathbb{Z}$ the winding number $w$ is defined as the
integral over the SU$(2)$ group element

$w=\frac{1}{24\pi^{2}}\int_{\rm SU(2)}dU;$

iii) The training data $S$: it is the union of the (initial) boundary colored
graphs together with the topological invariants associated to them through the
TQN2C functorial action.

  * •

Learner’s output:
It is a prediction rule, i.e. the TQN2C that identifies the topological
charges of the boundary states (training/test samples) and thus implements the
classifier; for $\gamma$ a $1$-complex supporting a disjoint boundary state,
and $\mathcal{C}$ a $2$-complex with boundaries $\partial\mathcal{C}=\gamma$,
the classifier is captured by the probability amplitude that results from the
internal product

$\mathcal{A}=\langle\gamma;\\{j_{l}\\},\\{\iota_{n}\\}|\,|\mathcal{Z}_{\mathcal{C},\partial\mathcal{C}=\gamma};\\{j_{l}\\},\\{\iota_{n}\\}\rangle\,,$

or, once the physical projector $\mathcal{P}$ has been determined,
$\mathcal{A}=\langle\psi_{\rm out}|\mathcal{P}\psi_{\rm in}\rangle$, for
generic boundary states $\psi_{\rm in}$ and $\psi_{\rm out}$.

These definitions illustrate, in a very explicit way, the difference between
how a TQNN “sees” the input and label sets – and hence the semantics that it
assigns to these sets – and how we, as human engineers, see them. From an XAI
perspective, the semantics assigned by the TQNN is effectively
uninterpretable. Hence as noted above, what our current approach provides is
not an explanation of how generalisation has worked in any particular case,
but rather an assurance, up to relevant conditions, that it has worked.

##  VI Perspective

The main contributions in dealing with the problem outlined by Refs. Zhang et
al. (2021, 2017) can be subdivided into two classes : (i) theoretical
approaches that try to understand the generalisation issue by proving a
generalisation bound on the test error; (ii) phenomenological approaches
motivated by experimentation (as concerns a review of these approaches, see
e.g. Jiang et al. (2021)).

Further elaborating on the consequences of the results attained by Zhang et
al. (2021) a novel strategy has been developed in Marcianò et al. (2022) and
here that is rooted on the analogue framework provided by TQFT. This is an
effective quantum theoretic approach that offers the pathway to address the
problem of generalisation. The origin of this latter has been here related to
the topological encoding of the input degrees of freedom by the network
structure, achieved through pattern selection and parameter optimisation, in
the semiclassical limit. The perspective we have then pushed forward here
relies on the conjecture that generalisation happens as the analogue of the
macroscopic manifestation of quantum effects.

A canonical experiment shedding light on our proposed outlook was carried out
by Philipp Lenard, who unveiled the existence of the photoelectric effect. The
occurrence of this effect, which only happens at frequencies of the impinging
electromagnetic radiation (photons) that are above a certain threshold,
provides a macroscopic manifestation of the existence of a quantised energy
gap between electronic bounded and valence states, occurring in electric
conductive materials.

This was indeed the framework adopted by Albert Einstein, who achieved a
theoretical understanding of such a semiclassical effect grounding it on
features of the newly developed theory of quanta. Analogously, generalisation
can be addressed as the manifestation of a topological (quantum) encoding
achieved by TQNNs. The texture of the webs of vertices and edges, which
capture the topological structures of the graphs dual to the input data,
implements the pattern Marcianò et al. (2022). For instance, the skeleton
graph dual to the letter ’L’ differs topologically from the one dual to the
number ’8’, but metric properties alone distinguish among the number ’8’ and
the symbol ‘$\infty$’.

Generalisation is achieved in the TQNNs framework as a selection, induced by
the quantum algorithm, of topological features that are the most adequate to
the achievement of a specific task. These topological features are captured by
the connectivity of graphs in the boundary states and of the vertices of the
$2$-complexes. Thus in our proposed picture, the analog of the quantum states
of the photoelectric effect – the electrons that appear either in bounded or
in valence energy levels, and the photons impinging the condenser’s plates of
the conductor metal in the Lenard experiment – are the quantum states
represented by the cylindrical functionals of the boundary group elements
(labelling the input data). These functionals are supported on the boundary
graphs ($1$-complexes), and it is their functorial evolution that is captured
by the TQN2C classifiers. Output boundary states represent instead the TQNN’s
ability to react to perturbations imposed by — effectively, the queries posed
by — the input training sets and test samples. The measure of success, for
both training and test measures, is provided by the internal product among the
boundary states, which is instantiated through the TQN2C functor, accounting
for the evolution of the TQNN states Marcianò et al. (2022).

The novelty of our theoretical approach, in particular with respect to recent
inspiring studies on the TQNN framework Marcianò et al. (2022); Fields et al.
(2022a), allows to consider a richer architecture that enables to associate
machine learning concepts entailing complexity to the topological features the
are coded therein. These properties include not only the inter-connectivity of
the edges belonging to the graphs, but possibly also associated linking and
knotting numbers, and the topological invariant properties of the 2-complexes
spanned by the graphs’ evolution Marcianò et al. (2022); Fields et al.
(2022a).

Within this framework, generalisation emerges from the optimisation of the
topological structures, topological invariants (states’ sum) and quantum
numbers (topological parameters), while the other parameters, which we may
call metric, eventually instantiate effective macroscopic thresholds, such as
in the photoelectric effect. Indeed, switching metric parameters on, does
trigger the emergence of the topological features too — see Marcianò et al.
(2022) for the details of the dual graphs selection out of the input data.
Change of the graphs’ topology is then achieved at the hidden layers by
vertices structures implementing the TQNN evolution through the TQN2C.
Furthermore, the volume of the input data set, increasing with the number of
links and nodes, to which holonomies and intertwiner tensors respectively are
associated, will play the analogue of the intensity of the radiation in the
photoelectric effect (number of photons). While the dimensions of the spin-
representations, assigned to the boundary TQNN states, namely the dimension of
the Hilbert spaces associated to each link and node, will play an analogous
role to the frequencies of the electromagnetic radiation.

Through the definition of this architecture, TQNNs can capture the topological
invariants from the training sets, which enables the identification of the
correct output, once test samples are deployed. This happens through
minimisation and optimisation of the classical action, namely by
stationarising the path integral representation of the TQN2C classifiers. This
procedure is reminiscent of the free energy principle Friston (2010a, b,
2019); Fields et al. (2022b): in the semiclassical limit the most important
contributions to the path-integral evaluation of the classifier are the paths
that are closest to the classical ones.

Asymptotically, in the semiclassical limit, a finite sample of labels suffices
to the success of the generalisation process Marcianò et al. (2022). This
observation suggests a different novel comprehension of the problem of
generalisation. Indeed, we have also to remind that, along Zhang et others’
work, the generalisation process is independent on the regularisation of the
dimensionality of the label sets that are involved. The proposed resolution of
the problem may be then naturally achieved through the architectures of the
TQNN states, as pointed out in Marcianò et al. (2022), while accounting for a
class of TQNN states that are solely supported on graphs and $2$-complexes of
reduced connectivity.

##  VII Conclusions

Moving from the innovative framework of Marcianò et al. (2022), we tackled the
most relevant theoretical issue related to DNNs: how is it possible that DNNs
are able to generalise and, therefore, learn? Understanding how generalisation
works may allow to build a principled model of the operation of Deep Learning
architectures. On the other hand, delving into the DNNs generalisation process
from the perspective of topological quantum physics can provide the key to
unprecedented technological implementations.

Considering first the heuristic case of one-node states, which can be treated
in full analogy with standard quantum mechanics, and then extending the
analysis to multi-node states of a quantum version of graph neural networks,
i.e. topological quantum neural networks, we have shown that the origin of the
problem of generalisation can be related to the topological encoding within
the quantum graph neural network structure, achieved through path selection
and parameter optimisation that corresponds to the semi-classical limit on
quantum theories.

Our proposed explanation may have a social and economic impact as well to the
extent that it improves the trustworthiness of AI systems, and their practical
and industrial applications. We further emphasize that the innovative flavour
of our analysis comes from its own interdisciplinary features, which achieve
constructing a bridge between traditional ML, with particular regard to DNNs,
topological quantum physics and quantum field theory, and materials science.

## References

  * Segler et al. (2018) M. H. Segler, M. Preuss, and M. P. Waller, Nature 555, 604 (2018).
  * Abitbol and Karsai (2020) J. L. Abitbol and M. Karsai, Nature Machine Intelligence 2, 684 (2020).
  * Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, in _Proc. NIPS_ (2017), vol. 30.
  * Li et al. (2021) Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, in _Proc. ICLR_ (2021).
  * Wing (2021) J. M. Wing, Communications of the ACM 64, 64 (2021).
  * Seshia et al. (2022) S. Seshia, D. Sadigh, and S. S. Sastry, Communications of the ACM 65, 56 (2022).
  * Samek et al. (2021) W. Samek, G. Montavon, S. Lapuschkin, C. J. Anders, and K.-R. Müller, Proceedings of the IEEE 109, 247 (2021).
  * Keskar et al. (2017) N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, in _Proc. ICLR_ (2017), eprint 1609.04836.
  * Arpit et al. (2017) D. Arpit, S. Jastrzkebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. C. Courville, Y. Bengio, et al., in _Proc. ICML_ (PMLR, 2017), pp. 233–242, eprint arXiv:1706.05394.
  * Dinh et al. (2017) L. Dinh, R. Pascanu, S. Bengio, and Y. Bengio, in _Proc. ICML_ (PMLR, 2017), pp. 1019–1028.
  * Dziugaite and Roy (2017) G. K. Dziugaite and D. M. Roy, in _Proc. UAI_ (2017).
  * Hoffer et al. (2017) E. Hoffer, I. Hubara, and D. Soudry, in _Proc. NIPS_ (2017), pp. 1729–1739.
  * Krueger et al. (2017) D. Krueger, N. Ballas, S. Jastrzebski, D. Arpit, M. S. Kanwal, T. Maharaj, E. Bengio, A. Fischer, and A. Courville, in _Proc. ICLR Workshop_ (2017), pp. 1–4.
  * Neyshabur et al. (2017) B. Neyshabur, S. Bhojanapalli, D. McAllester, and N. Srebro, in _Proc. NIPS_ (2017), pp. 5947–5956, eprint 1706.08947.
  * Neyshabur et al. (2018) B. Neyshabur, S. Bhojanapalli, and N. Srebro, in _Proc. ICLR_ (2018).
  * Wu et al. (2017) L. Wu, Z. Zhu, and W. E, in _Proc. ICML Workshop_ (2017).
  * Shwartz-Ziv and Tishby (2017) R. Shwartz-Ziv and N. Tishby, arXiv 1703.00810 (2017), URL https://arxiv.org/abs/1703.00810.
  * Lin et al. (2017) H. W. Lin, M. Tegmark, and D. Rolnick, Journal of Statistical Physics 168, 1223 (2017).
  * Wang et al. (2017) A. Wang, H. Zhou, W. Xu, and X. Chen, arXiv, 1708.05029 (2017), URL https://arxiv.org/abs/1708.05029.
  * Li et al. (2020) J. Li, Y. Sun, J. Su, T. Suzuki, and F. Huang, in _International Conference on Artificial Intelligence and Statistics_ (PMLR, 2020), pp. 504–515, eprint 2001.05070.
  * Hartnett (2019) K. Hartnett, _Foundations built for a general theory of neural networks_ (2019), https://www.quantamagazine.org/foundations-built-for-a-general-theory-of-neural-networks-20190131/. Accessed 8 July 2022.
  * Taylor and Taylor (2020) J. E. T. Taylor and G. W. Taylor, Psychonomic Bulletin & Review 28, 454 (2020).
  * Senior et al. (2020) A. W. Senior, R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. Green, C. Qin, A. Žídek, A. W. Nelson, A. Bridgland, et al., Nature 577, 706 (2020).
  * Brady et al. (2008) T. F. Brady, T. Konkle, G. A. Alvarez, and A. Oliva, Proceedings of the National Academy of Sciences 105, 14325 (2008).
  * Shin (2021) D. Shin, International Journal of Human-Computer Studies 146, 102551 (2021).
  * Choung et al. (2022) H. Choung, P. David, and A. Ross, International Journal of Human–Computer Interaction pp. 1–13 (2022).
  * Hasija and Esper (2022) A. Hasija and T. L. Esper, Journal of Business Logistics (2022).
  * Marcianò et al. (2022) A. Marcianò, D. Chen, F. Fabrocini, C. Fields, E. Greco, N. Gresnigt, K. Jinklub, M. Lulli, K. Terzidis, and E. Zappala, Neural Networks 153, 164 (2022).
  * Farhi and Neven (2018) E. Farhi and H. Neven, arXiv:1802.06002 (2018), URL https://arxiv.org/abs/1802.06002.
  * Beer et al. (2020) K. Beer, D. Bondarenko, T. Farrelly, T. J. Osborne, R. Salzmann, D. Scheiermann, and R. Wolf, Nature communications 11, 1 (2020).
  * Fields et al. (2022a) C. Fields, J. F. Glazebrook, and A. Marcianò, Fortschritte der Physik 70, in press (2022a).
  * Bartlett et al. (2021) P. L. Bartlett, A. Montanari, and A. Rakhlin, Acta numerica 30, 87 (2021).
  * Frankle and Carbin (2019) J. Frankle and M. Carbin, in _Proc. ICLR_ (2019).
  * Vapnik and Chervonenkis (1982) V. N. Vapnik and A. Y. Chervonenkis, Theory of Probability & Its Applications 26, 532 (1982).
  * Blumer et al. (1989) A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, Journal of the ACM (JACM) 36, 929 (1989).
  * Haussler (2018) D. Haussler, in _The Mathematics of Generalization_ (CRC Press, 2018), pp. 37–116.
  * Belkin et al. (2018) M. Belkin, S. Ma, and S. Mandal, in _Proc. ICML_ (PMLR, 2018), pp. 541–549.
  * Poggio et al. (2017) T. Poggio, K. Kawaguchi, Q. Liao, B. Miranda, L. Rosasco, X. Boix, J. Hidary, and H. Mhaskar, arXiv:1801.00173 (2017), URL https://arxiv.org/abs/1801.00173.
  * Zhang et al. (2017) C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, in _Proc. ICLR_ (Toulon, CS, France, 2017).
  * Neyshabur et al. (2015a) B. Neyshabur, R. Tomioka, and N. Srebro, in _Proc. ICLR Workshop_ (2015a).
  * Kawaguchi et al. (2017) K. Kawaguchi, L. P. Kaelbling, and Y. Bengio, arXiv 1710.05468 (2017), URL https://arxiv.org/abs/1710.05468.
  * Arora et al. (2017) S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang, in _Proc. ICML_ (PMLR, 2017), pp. 224–232.
  * Belkin et al. (2019) M. Belkin, D. Hsu, S. Ma, and S. Mandal, Proceedings of the National Academy of Sciences 116, 15849 (2019).
  * Neyshabur et al. (2015b) B. Neyshabur, R. Tomioka, and N. Srebro, in _Conference on Learning Theory_ (PMLR, 2015b), pp. 1376–1401.
  * Bartlett et al. (2017) P. L. Bartlett, D. J. Foster, and M. J. Telgarsky, in _Proc. NIPS_ (2017), vol. 30.
  * Shawe-Taylor et al. (1998) J. Shawe-Taylor, P. L. Bartlett, R. C. Williamson, and M. Anthony, IEEE transactions on Information Theory 44, 1926 (1998).
  * Bartlett and Mendelson (2002) P. L. Bartlett and S. Mendelson, Journal of Machine Learning Research 3, 463 (2002).
  * Zhang et al. (2021) C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, Communications of the ACM 64, 107 (2021), eprint 1611.03530.
  * Rovelli (2011) C. Rovelli, in _Journal of Physics: Conference Series_ (IOP Publishing, 2011), vol. 314, p. 012006.
  * Bianchi et al. (2010) E. Bianchi, E. Magliaro, and C. Perini, Physical Review D 82, 024012 (2010).
  * Kauffman et al. (1994) L. Kauffman, S. Lins, and S. Lins, _Temperley-Lieb Recoupling Theory and Invariants of 3-manifolds_ , Annals of Mathematics Studies (Princeton University Press, 1994), ISBN 9780691036403.
  * Crnkovic and Witten (1988) C. Crnkovic and E. Witten, Class. Quant. Grav 5, 1557 (1988).
  * Crnković (1987) Č. Crnković, Nuclear Physics B 288, 419 (1987).
  * Crnkovic (1988) C. Crnkovic, Classical and Quantum Gravity 5, 1557 (1988).
  * Sawon (2006) J. Sawon, Geom. Topol. Monogr 8, 145 (2006).
  * Rovelli and Smolin (1995) C. Rovelli and L. Smolin, Physical Review D 52, 5743 (1995).
  * Baez (1996) J. C. Baez, Advances in Mathematics 117, 253 (1996).
  * Noui and Perez (2005) K. Noui and A. Perez, Classical and Quantum Gravity 22, 1739 (2005).
  * Friston (2010a) K. J. Friston, Nature Reviews Neuroscience 11, 127 (2010a).
  * Friston (2010b) K. J. Friston, Journal of The 1299 quantum mechanics, and then extending the analysis to Royal Society Interface 11, 20130475 (2010b).
  * Friston (2019) K. J. Friston, arXiv:1906.10184 (2019), URL https://arxiv.org/abs/1906.10184.
  * Fields et al. (2022b) C. Fields, K. J. Friston, J. F. Glazebrook, and M. Levin, Progress in Biophysics and Molecular Biology 173, 36 (2022b).
  * Shalev-Shwartz and Ben-David (2014) S. Shalev-Shwartz and S. Ben-David, _Understanding machine learning: From theory to algorithms_ (Cambridge university press, 2014).
  * Jiang et al. (2021) Y. Jiang, P. Natekar, M. Sharma, S. K. Aithal, D. Kashyap, N. Subramanyam, C. Lassance, D. M. Roy, G. K. Dziugaite, S. Gunasekar, et al., in _Proc. NeurIPS_ (PMLR, 2021), pp. 170–190.