# Noncommutative Geometry of computational models and Uniformization for
framed Quiver Varieties

George Jeffreys  Department of Mathematics and Statistics, Boston University,
111 Cummington Mall, Boston MA 02215, USA<EMAIL_ADDRESS>and  Siu-Cheong Lau
Department of Mathematics and Statistics, Boston University, 111 Cummington
Mall, Boston MA 02215, USA<EMAIL_ADDRESS>
###### Abstract.

We formulate a mathematical setup for computational neural networks using
noncommutative algebras and near-rings, in motivation of quantum automata. We
study the moduli space of the corresponding framed quiver representations, and
find moduli of Euclidean and non-compact types in light of uniformization.

##  1\. Introduction

The connections between computer science and algebra are profound. In the
early 1900’s, both were deeply tied to practical and philosophical
developments towards understanding what it truly means to calculate something.
For example, there was Turing’s Halting problem and Gödel’s Incompleteness
theorem.

As modern abstract algebra was developed in the 50’s and 60’s, it was
fruitfully turned towards this path with the creation of the theory of finite
automata. The first fundamental result in this development was Kleene’s
Theorem demonstrating that the class of recognizable languages is the class of
rational languages [Kle16]. In 1956, Schützenberger defined the syntactic
monoid, a canonical monoid attached to each language [Sch56]. Later, he proved
that a language is star-free exactly when its syntactic monoid is finite and
aperiodic [Sch65]. At this point mathematicians started to consider the
algebraic geometry of these monoids as Birkhoff [Bir35] and latter Eilenberg
[ET76] and Reiterman [Rei82] developed wrote about varieties of these monoids
(infinite and finite respectively). Thus we have a well established and
important connection between theoretical computer science and pure algebraic
geometry.

The theory of finite automata arose from an extremely widespread
interdisciplinary effort to understand calculation. Modern science suggests
that the brain operates as a so-called neural network, the structure of which
has inspired the computational tool known as the artificial neural network.
Neural network models heavily use graphs and their linear representations.
This gives rise to further deep relations between mathematics and computer
science.

In this paper, we build an algebraic abstraction that models a neural network
and quantum automata. We are motivated as follows. A finite automata consists
of a set of states of a machine, a set of transitions between the states, and
an alphabet set that will form a machine language, whose elements label the
transitions of states.

A quantum version of this replaces the set of states by a collection of vector
spaces whose elements are known as state vectors. The set of transitions is
replaced by a set of linear maps between the vector spaces. This forms a so-
called quiver representation, which is a linear representation of the directed
graph $Q$ (called a quiver) whose vertices label the collection of vector
spaces, and whose arrows label the set of linear maps.

${\mathrm{In}}$${\mathbb{C}^{3}}$${\mathrm{Out}}$$\scriptstyle{w^{[1]}}$$\scriptstyle{w^{[2]}}$

${\mathbb{C}}$${\mathrm{In}}$${\mathbb{C}}$${\mathrm{Out}}$${\mathbb{C}}$$\scriptstyle{w^{[2]}_{1}}$$\scriptstyle{w^{[1]}_{1}}$$\scriptstyle{w^{[1]}_{2}}$$\scriptstyle{w^{[1]}_{3}}$$\scriptstyle{w^{[1]}_{2}}$$\scriptstyle{w^{[2]}_{3}}$

Figure 1. Two artificial neural networks with similar graphs.

Paths in the quiver play the role of words of a machine language. The path
algebra

$\mathcal{A}=\mathbb{C}Q$

consists of complex linear combinations of paths, with concatenation of paths
serving as the product. Taking linear combinations can be interpreted as
forming superpositions of quantum states.

In summary, _a quiver algebra and its modules provide a nice model of a
quantum automata_.

One crucial component that one cannot miss is _taking observation of the
quantum particles_. Most mathematical physics literature concentrate on the
quantum propagation process, and have left away the mysterious observation
step, perhaps due to its probabilistic and singular nature. However, this step
is crucial in true understanding of quantum physics, and also in practical
applications. For modeling quantum propagations, operator algebras serve as a
very successful mathematical tool. However, to include the observation
process, we find that a _near-ring_ , which is much less studied than an
algebra, is necessary.

To model the observation process in a quantum world, we need two more
ingredients: Hermitian metrics $h$ of the state spaces $V$, and a framing
linear map $e:F\to V$ where $F=\mathbb{C}^{n}$ is called a framing vector
space. Then we take

$e^{*_{h}}(v)=\sum_{j=1}^{n}h(e(\epsilon_{j}),v)\epsilon_{j}^{*},$

where $\epsilon_{j}$ denotes the standard basis of $\mathbb{C}^{n}$ (and
$\epsilon_{j}^{*}$ denotes the dual basis). The coefficients
$h(e(\epsilon_{j}),v)$ are interpreted as the quantum amplitudes of a state
$v$ being $e(\epsilon_{j})$. Then the quantum collapsing after observation is
modeled by composing this with a fixed non-linear activation function
$\sigma:F\to F$ (for instance a certain step function, or a smoothing of it).
In the quantum world, $\sigma$ is indeed an $F$-valued probability
distribution on $F$.

Thus, a quantum machine consists of not just linear transitions of states, but
also the framings and non-linear activation functions that correspond to
taking observations. We will make the following definition. See also Figure 2.

######  Definition 1.1 (Definition 2.4).

An _activation module_ consists of:

  1. (1)

a (noncommutative) algebra $\mathcal{A}$ and vector spaces
$V,\,F=F_{\mathrm{in}}\oplus F_{\mathrm{out}}\oplus F_{\mathrm{m}}$; (‘m’
stands for ‘memory’ or ‘middle’.)

  2. (2)

A family of metrics $h_{\left(w,e\right)}$ on $V$ over the space of framed
$\mathcal{A}$-modules

$R=\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(F,V\right)$

which is $\mathrm{GL}\left(V\right)$-equivariant;

  3. (3)

a collection of possibly non-linear functions

$\sigma_{j}^{F}\colon F_{\mathrm{m}}\to F_{\mathrm{m}}.$

In above, $R$ parametrizes computing machines that have the same underlying
framed quiver, and hence is governed by the same language. Moreover, framed
$\mathcal{A}$-modules that differ by a $\mathrm{GL}(V)$-action have the same
computational effect and hence should be identified. $[R/\mathrm{GL}(V)]$
forms a moduli stack of computing machines.

In this formulation, a machine language is composed of not just linear
transitions of state spaces, but also non-linear (or probabilistic) operations
$\sigma$ that models quantum observations. The set of operations generated by
these is no longer an algebra, since

$\sigma\circ(\gamma_{1}+\gamma_{2})\not=\sigma\circ\gamma_{1}+\sigma\circ\gamma_{2}$

where $\gamma_{1},\gamma_{2}$ are composed of linear operations in
$\mathcal{A}$ and the dual framing map $e^{*_{h}}$. Rather, it generates a
near-ring $\widetilde{A}$, which is almost a ring except that the
multiplication (which is realized by composition of maps in the current setup)
fails to be distributive on one side.

Motivated by this, we extend the theory of noncommutative differential forms
by Connes [Con85], Cuntz-Quillen [CQ95], Ginzburg [Gin05] to the context of
near-rings. The main idea is that, every element in the near-ring
$\widetilde{A}$, which is interpreted as a program written in the language of
$\widetilde{A}$, produces a family of maps on the framing space $F$ over the
moduli of machines $[R/G]$, that is, each machine in $[R/G]$ performs a
computation $F\to F$ specified by the program. This statement naturally
extends to differential forms.

######  Theorem 1.2 (Theorem 2.40).

There exists a degree-preserving map

$DR^{\bullet}(\widetilde{\mathcal{A}})\to(\Omega^{\bullet}(R,\mathbf{Map}\left(F,F\right)))^{G}$

which commutes with $d$ on the two sides. In above,
$\mathbf{Map}\left(F,F\right)$ denotes the trivial bundle
$\mathrm{Map}\left(F,F\right)\times R$, and the action of $G=\mathrm{GL}(V)$
on fiber direction is trivial.

$0$-forms and $1$-forms are particularly important for machine learning.
Namely, for a fixed algorithm $\tilde{\gamma}\in\widetilde{\mathcal{A}}$, a
learning process attempts to find a machine $p\in[R/G]$ that produces the best
fit computation $\phi^{\tilde{\gamma}}_{p}:F\to F$ by minimizing a certain
$0$-form (for instance
$\int_{K}\left|\varphi_{p}^{\tilde{\gamma}}(x)-f\left(x\right)\right|^{2}dx$
for a given $f:K\to\mathbb{R}$ and $K\subset F$ in supervised learning). Its
differential, which is a $1$-form in $DR^{1}(\widetilde{\mathcal{A}})$,
governs the gradient flow on $[R/G]$ with the help of a metric.

In general, $[R/G]$ is a singular stack. Fortunately, for quiver algebras, a
fine moduli of framed quiver representations was constructed by taking a GIT
quotient (with respect to a suitably chosen stability condition) [Kin94,
Nak01]. Such moduli spaces $\mathcal{M}$ can be used in place of $[R/G]$ and
their topologies are well studied by [Rei08].

In [JL21], we formulated learning of neural networks over the moduli spaces
$\mathcal{M}$. Namely, the state space $V_{i}$ over each vertex $i\in Q_{0}$
patches up as a universal bundle $\mathcal{V}_{i}$ over $\mathcal{M}$. The
transition arrows $a\in Q_{1}$ correspond to bundle maps over $\mathcal{M}$.
The framing linear maps $e_{i}:F_{i}\to V_{i}$ correspond to bundle maps from
the trivial bundle $\mathbf{F_{i}}$ to $\mathcal{V}_{i}$. Then data and states
of the family of machines are naturally modeled by sections over
$\mathcal{M}$; propagation of signals is modeled by bundle maps. In this
formulation, learning is a stochastic gradient descent over the moduli
$\mathcal{M}$.

It is tempting to ask how this formulation relates to the most common method
of machine learning over an Euclidean space, rather than a moduli space
$\mathcal{M}$. In this paper, we will answer this question in light of
uniformization of metrics.

The main observation is that, $\mathcal{M}$ in effect is a compactification of
the most commonly used Euclidean space, now denoted as $\mathcal{M}^{0}$.
Moreover, the Euclidean space $\mathcal{M}^{0}$ can be interpreted as a moduli
space of positive-definite quiver representations with respect to a certain
Hermitian form $H_{i}^{0}$ for the universal bundles $\mathcal{V}_{i}$. Thus,
the most popular approach using Euclidean space indeed also falls into our
formulation of learning in the moduli space of computing machines.

This uniformization picture naturally includes a hyperbolic version of the
moduli space. Namely, by changing the signature of the quadratic form (see
(22)), we obtain another type of moduli space $\mathcal{M}^{-}$ of positive-
definite quiver representations with respect to $H_{i}^{-}$. We show that
$\mathcal{M}^{-}$ comes with a natural metric.

######  Theorem 1.3 (Theorem 3.15).

Define $H_{T}^{-}$ to be
$H_{T}^{-}:=-i\sum\limits_{i}\partial\overline{\partial}\log\det H_{i}^{-}$ on
$\mathcal{M}^{-}$. Then $H_{T}^{-}$ is a Kähler metric on $\mathcal{M}^{-}$.

In classical applications, one can also restrict to real coefficients.
Correspondingly, the formulae provided by this paper give bundle metrics for
$\mathcal{V}_{i}^{\mathbb{R}}|_{\mathcal{M}_{\mathbb{R}}}$ and Riemannian
metrics on $\mathcal{M}_{\mathbb{R}}$.

As a result, we can run machine learning over
$\mathcal{M},\mathcal{M}^{0},\mathcal{M}^{-}$, or an interpolation of them. We
can also set learnable parameters that interpolate these spaces, and let the
machine learn which metric serves the best for a given task.

### Some related works

Recently, there is a rising interest in the connections between neural
networks and quiver representations. The paper [AJ20] found an interesting way
of encoding the data flow as a quiver representation, which makes a crucial
use of the assumption of thin representations (where dimensions of
representing vector spaces over vertices are all $1$). On the other hand, the
learning that they take is not directly carried over the quiver moduli, and
hence is different from our approach in [JL21] and this paper. [GW21] studied
the symmetries coming from the quiver approach to neural networks.

There are also newly invented approaches to apply higher mathematics to
machine learning. Most literature concerns about the input data set and endows
it with more interesting mathematical structures, for instance, Lie group
symmetry [CW16, CGW19, CGKW18, CWKW19, CAW+19, dHCW20], or categorical
structures [SY21]. On the other hand, in our current approach, we focus on the
computing machine itself, and formulate its algebro-geometric structure and
makes use of its internal symmetry.

For learning using hyperbolic spaces, there are several beautiful works, see
for instance [NK17], [GBH18a], [SSGR18], [GBH18b]. The non-compact dual of the
moduli space $\mathcal{M}^{-}$ that we introduce in this paper can be
understood as a higher rank generalization of hyperbolic spaces in the sense
of Hermitian symmetric spaces. See more in Section 3.5.

### Organization of this paper

In Section 2, we will define computing machines in the context of
noncommutative geometry. In Section 3, we will apply the idea of
uniformization of metrics to construct non-compact duals to neural network
quiver moduli spaces.

##  2\. An AG formulation of computing machine

In this section, we give a mathematical formulation of a computing machine
based on algebra and geometry. First, we formulate a machine as a framed
module over an algebra, together with a metric on the module and a collection
of non-linear functions. Second, we take into account of isomorphisms of
framed modules and make sure the construction is equivariant under the
automorphism group, and hence descends to the moduli stack of framed modules.
Finally, we extend the noncommutative geometry developed by [Con85, CQ95,
Gin05] to the context of near-rings, and show how it fits into this framework.

###  2.1. Intuitive construction

Let $\mathcal{A}$ be an associative algebra with unit $1_{\mathcal{A}}$. This
algebra encodes all possible linear operations of the machine. Later, in the
context of neural network, we will take $\mathcal{A}$ to be the path algebra
of a directed graph (which is also called a quiver).

Let $V$ be a vector space. $V$ is understood as the space of abstract states
of the machine prior to any physical observation. It is basis-free, namely, we
do not pick any preferred choice of basis.

We consider $\mathcal{A}$-module structures, that is algebra homomorphisms
$w\colon\mathcal{A}\rightarrow\mathfrak{gl}\left(V\right)$. Each module
structure $w$ realizes $a\in A$ as a linear operation on the state space.

In reality, data are observed and recorded in fixed basis. For this, we define
a framing vector space $F=F_{\mathrm{in}}\oplus F_{\mathrm{out}}\oplus
F_{\mathrm{m}}$. Each component is a vector space with a fixed basis. We may
simply write $F=\mathbb{C}^{n}$ with the standard basis. Moreover, we consider
linear maps $e\colon F\rightarrow V$, $e=e_{\mathrm{in}}\oplus
e_{\mathrm{out}}\oplus e_{\mathrm{m}}$ which are called the framing maps.
$F_{\mathrm{in}}\oplus F_{\mathrm{out}}$ are vector spaces of all possible
inputs and outputs. $F_{\mathrm{m}}$ can be understood as a space for memory
of the machine. The framing maps $e$ are used to observe and record the
abstract states.

A triple $\left(V,w,e\right)$ is called a framed $\mathcal{A}$-module. We
denote by

$R\coloneqq\left\\{\left(w,e\right)\colon
w\colon\mathcal{A}\rightarrow\mathfrak{gl}\left(V\right)\text{ alg.
homo.};\leavevmode\nobreak\ e\colon F\rightarrow V\right\\}$

the _set of framed modules_. It serves as the parameter space of the machine.
$R$ is a subvariety in
$\mathrm{Lin}(\mathcal{A},\mathfrak{gl}(V))\times\mathrm{Lin}(F,V)$.

Let $\mathcal{A}_{\mathrm{m}}$ be the augmented algebra

(1) $\mathcal{A}_{\mathrm{m}}=\mathcal{A}\langle
1_{\mathrm{m}},\mathfrak{e}_{\mathrm{m}},\mathfrak{e}^{*}_{\mathrm{m}}\rangle/I$

where $I$ is the two-sided ideal generated by the relations

$\displaystyle
1_{\mathrm{m}}\cdot\mathfrak{e}_{\mathrm{m}},\,\mathfrak{e}_{\mathrm{m}}\cdot
1_{\mathrm{m}}-\mathfrak{e}_{\mathrm{m}},\,1_{\mathcal{A}}\cdot\mathfrak{e}_{\mathrm{m}}-\mathfrak{e}_{\mathrm{m}},$
$\displaystyle\mathfrak{e}_{\mathrm{m}}^{*}\cdot
1_{\mathrm{m}},\,1_{\mathrm{m}}\cdot\mathfrak{e}_{\mathrm{m}}^{*}-\mathfrak{e}_{\mathrm{m}}^{*},\,\mathfrak{e}_{\mathrm{m}}^{*}\cdot
1_{\mathcal{A}}-\mathfrak{e}_{\mathrm{m}}^{*},$
$\displaystyle\mathfrak{e}_{\mathrm{m}}^{2},\,(\mathfrak{e}^{*}_{\mathrm{m}})^{2},\,a\cdot\mathfrak{e}^{*}_{\mathrm{m}},\,\mathfrak{e}_{\mathrm{m}}\cdot
a,\,a\cdot 1_{\mathrm{m}},\,1_{\mathrm{m}}\cdot a$

for all $a\in\mathcal{A}$. (This means, for instance,
$1_{\mathrm{m}}\cdot\mathfrak{e}_{\mathrm{m}}=0$ and
$\mathfrak{e}_{\mathrm{m}}\cdot 1_{\mathrm{m}}=\mathfrak{e}_{\mathrm{m}}$ in
the algebra $\mathcal{A}_{\mathrm{m}}$.) The unit of
$\mathcal{A}_{\mathrm{m}}$ is $1_{\mathcal{A}}+1_{\mathrm{m}}$.

Let’s equip $V$ with a Hermitian metric $h$. Then for each framing map
$e=e_{\mathrm{in}}\oplus e_{\mathrm{out}}\oplus e_{\mathrm{m}}$, the element
$\mathfrak{e}_{\mathrm{m}}\in\mathcal{A}_{\mathrm{m}}$ is realized as the map
$e_{\mathrm{m}}:F_{\mathrm{m}}\to V$, and $\mathfrak{e}_{\mathrm{m}}^{*}$ is
realized as the metric adjoint
$(h(e_{\mathrm{m},l},\cdot))_{l=1}^{n_{\mathrm{m}}}:V\to
F_{\mathrm{m}}=\mathbb{C}^{n_{\mathrm{m}}}$.

To consider linear maps that have domain and target being $V$, we can form the
subalgebra

$\mathcal{A}_{{\mathrm{m}},0}:=\mathcal{A}\cdot\mathcal{A}_{\mathrm{m}}\cdot\mathcal{A}.$

An element $a\in\mathcal{A}_{{\mathrm{m}},0}$ is understood as a linear
algorithm. Fixing $(w,e)\in R$, each linear algorithm
$a\in\mathcal{A}_{{\mathrm{m}},0}$ is associated with $f^{a}\colon
F_{\mathrm{in}}\rightarrow F_{\mathrm{out}}$,

$f^{a}\left(v\right)\coloneqq e_{\mathrm{out}}^{\mathrm{*}}\left(a\cdot
e_{\mathrm{in}}\left(v\right)\right)$

which is called a machine function. ($e_{\mathrm{out}}^{*}:V\to
F_{\mathrm{out}}$ is the metric adjoint
$(h(e_{\mathrm{out},l},\cdot))_{l=1}^{n_{\mathrm{out}}}$.) In other words, we
have the map

$R\times\mathcal{A}_{{\mathrm{m}},0}\to\mathrm{Hom}(F_{\mathrm{in}},F_{\mathrm{out}})$

which is linear in the second component.

So far, this is just a linear model. In order to capture non-linearity, we
also need to incorporate with _non-linear operations_
$\sigma_{1},\ldots,\sigma_{N}$. Let’s define these as functions $V\rightarrow
V$ for the moment. (In the next subsection, we shall see that defining in this
way is not good from the moduli point of view and it will be modified.)

Consider the $\mathbb{C}$-near-ring
$\tilde{\mathcal{A}}=\mathcal{A}\left\\{\varsigma_{1},\ldots,\varsigma_{N}\right\\}$.
The elements $\varsigma_{j}$ are algebraic symbols for recording the non-
linear operations $\sigma_{j}$. See Definition 2.11 for the notion of a near-
ring. Essentially it is recording the compositions of module maps and the non-
linear operations. Similar to above, we take the augmented near-ring

(2) $\tilde{\mathcal{A}}_{\mathrm{m}}=\tilde{\mathcal{A}}\langle
1_{\mathrm{m}},\mathfrak{e}_{\mathrm{m}},\mathfrak{e}^{*}_{\mathrm{m}}\rangle/\tilde{I}$

where $\tilde{I}$ is generated by the relations in $I$ as in (1), together
with the relations

$\varsigma_{l}\cdot 1_{\mathrm{m}},\,\,\,1_{\mathrm{m}}\cdot\varsigma_{l}.$

(This means $\sigma_{l}$ and $1_{\mathrm{m}}$ compose to be zero. We want this
since $\sigma_{l}$ is acting on $V$ and $1_{\mathrm{m}}$ is acting on
$F_{\mathrm{m}}$.) An element
$\tilde{\gamma}\in\tilde{\mathcal{A}}_{\mathrm{m},0}:=\mathcal{A}\cdot\tilde{\mathcal{A}}_{\mathrm{m}}\cdot\mathcal{A}$
is understood as a non-linear algorithm.

Fixing $\left(w,e\right)\in R$, each algorithm
$\tilde{\gamma}\in\tilde{\mathcal{A}}_{\mathrm{m},0}$ is associated with a
non-linear machine function $f_{\left(w,e\right)}^{\tilde{\gamma}}\colon
F_{\mathrm{in}}\rightarrow F_{\mathrm{out}}$ ,

(3)
$f_{\left(w,e\right)}^{\tilde{\gamma}}\left(v\right)=e_{\mathrm{out}}^{\mathrm{*}}\left(\tilde{\gamma}\circ_{\left(w,e\right)}e_{\mathrm{in}}\left(v\right)\right).$

That is, we have the map

$R\times\tilde{\mathcal{A}}_{\mathrm{m},0}\to\mathrm{Map}(F_{\mathrm{in}},F_{\mathrm{out}}).$

###  2.2. Construction over moduli spaces

An important principle in mathematics and physics is that isomorphic objects
should produce the same result. In other words, we want to have
$f_{\left(w,e\right)}^{\tilde{\gamma}}$ well-defined over the moduli stack of
framed $\mathcal{A}$-modules $\mathcal{M}=[R/G]$ for $G=\mathrm{GL}(V)$. Let’s
recall the following definition.

######  Definition 2.1.

For two framed $\mathcal{A}$-modules $(V,w,e)$ and
$(V^{\prime},w^{\prime},e^{\prime})$, where both $e$ and $e^{\prime}$ have the
same domain $F$, a morphism (or an isomorphism) from $(V,w,e)$ to
$(V^{\prime},w^{\prime},e^{\prime})$ is a linear map (or a linear isomorphism)
$g:V\to V^{\prime}$ such that $w^{\prime}(a)\circ g=g\circ w(a)$ for all
$a\in\mathcal{A}$ and $e^{\prime}=g\circ e$.

Unfortunately, this is not the case in the above formulation due to the
presence of non-linear functions $\sigma:V\to V$. Any useful non-linear
function $\sigma:V\to V$ cannot satisfy
$\mathrm{GL}\left(V\right)$-equivariance:

(4) $g\cdot\left(\sigma\left(v\right)\right)=\sigma\left(g\cdot
v\right)\textrm{ for all }g\in\mathrm{GL}(V).$

It produces a crucial gap between the subject of machine learning and
representation theory.

Here is a simple solution to this problem. Let $\mathcal{V}$ be the universal
bundle over the moduli stack $\mathcal{M}$, which is descended from the
trivial bundle $V\times R$, where $G=\mathrm{GL}(V)$ acts diagonally.

Rather than defining $\sigma$ as a single linear map $V\rightarrow V$, let’s
take $\sigma$ to be a fiber-bundle map $V\times R\to V\times R$ over $R$. Then
$\sigma$ descends as a fiber-bundle map $\mathcal{V}\to\mathcal{V}$ over
$\mathcal{M}$ if it satisfies the equivariance equation

(5)
$g\cdot\left(\sigma_{\left(w,e\right)}\left(v\right)\right)=\sigma_{\left(g\cdot
w,g\cdot e\right)}\left(g\cdot v\right)\textrm{ for all }g\in\mathrm{GL}(V).$

The difference between Equation (5) and (4) is that $\sigma$ is now allowed to
also depend on $(w,e)\in R$.

Now suppose we have $\mathrm{GL}(V)$-equivariant fiber-bundle maps
$\sigma_{1},\ldots,\sigma_{N}:V\times R\to V\times R$. As in the last
subsection, we have the map
$R\times\tilde{\mathcal{A}}_{\mathrm{m},0}\to\mathrm{Map}(F_{\mathrm{in}},F_{\mathrm{out}})$
by realizing $\varsigma_{i}\in\tilde{\mathcal{A}}$ as
$(\sigma_{i})_{(w,e)}:\mathcal{V}\to\mathcal{V}$.

Recall that we have used a Hermitian metric on $V$ for taking the adjoint of
framing $e^{*}$. To make sure $e^{*}$ is also equivariant, we need to equip
$V$ with a family of Hermitian metrics $h_{\left(w,e\right)}$ for
$\left(w,e\right)\in R$, in a $\mathrm{GL}(V)$-equivariant way:

(6) $h_{\left(g\cdot w,g\cdot e\right)}\left(g\cdot u,g\cdot
v\right)=h_{\left(w,e\right)}\left(u,v\right)\textrm{ for all
}g\in\mathrm{GL}(V).$

That is, $h$ descends to be a Hermitian metric on the universal bundle
$\mathcal{V}$ over $\mathcal{M}$. Note that we are NOT asking for
$\mathrm{GL}(V)$-invariance $h\left(g\cdot u,g\cdot
v\right)=h\left(u,v\right)$ for a single metric $h$, which is impossible.

######  Proposition 2.2.

In the above setting, the non-linear machine function defined by Equation (3)
satisfies the equivariance
$f_{\left(w,e\right)}^{\tilde{\gamma}}=f_{g\cdot\left(w,e\right)}^{\tilde{\gamma}}$
for all $g\in\mathrm{GL}(V)$.

###### Proof.

The fiber-bundle map $f_{\left(w,e\right)}^{\tilde{\gamma}}:V\times R\to
V\times R$ defined by (3) is a composition of
$e_{\mathrm{out}}^{\mathrm{*}}=(h_{(w,e)}(e_{\mathrm{out},l},\cdot))_{l=1}^{n_{\mathrm{out}}}$,
$w_{a}$ for $a\in\mathcal{A}$, the fiber-bundle maps
$(\sigma_{i})_{(w,e)}:V\times R\to V\times R$, and $e_{\mathrm{in}}$. Under
the action of $g\in\mathrm{GL}(V)$, They change to

$e_{\mathrm{out}}^{\mathrm{*}}=(h_{g\cdot(w,e)}(g\cdot
e_{\mathrm{out},l},\cdot))_{l=1}^{n_{\mathrm{out}}}=(h_{(w,e)}(e_{\mathrm{out},l},g^{-1}(\cdot)))_{l=1}^{n_{\mathrm{out}}}=e_{\mathrm{out}}^{\mathrm{*}}\cdot
g^{-1},$

$g\cdot w_{a}\cdot g^{-1}$,

$\sigma_{\left(g\cdot w,g\cdot e\right)}=g\cdot\sigma_{(w,e)}(g^{-1}(\cdot))$

and $g\cdot e_{\mathrm{in}}$ respectively, using Equation (5) and (6). The
composition remains the same. ∎

In this way, we obtain the map
$\mathcal{M}\times\tilde{\mathcal{A}}_{\mathrm{m},0}\to\mathrm{Map}(F_{\mathrm{in}},F_{\mathrm{out}})$.

In applications, we need concrete fiber bundle maps
$\sigma:\mathcal{V}\to\mathcal{V}$. They can be cooked up using the Hermitian
metric $h$ on $\mathcal{V}$ as follows. Given any function $\sigma^{F}\colon
F_{\textrm{m}}\rightarrow F_{\textrm{m}}$, define $\sigma_{\left(w,e\right)}$
as

$\sigma_{\left(w,e\right)}\left(v\right)\coloneqq
e^{(\textrm{m})}\cdot\sigma^{F}\left(h_{\left(w,e\right)}\left(e^{(\textrm{m})}_{1},v\right),\ldots,h_{\left(w,e\right)}\left(e^{(\textrm{m})}_{n_{\textrm{m}}},v\right)\right).$

In other words, we observe and record the state $v$ to memory using
$e^{(\textrm{m})}$ and $h$; then we perform the non-linear operation
$\sigma^{F}$ on the memory $F_{\textrm{m}}$; finally we send it back as a
state in $V$. Unlike the setting in the last subsection, the non-linear
operation $\sigma^{F}$ is now defined on the framing space $F_{\textrm{m}}$
instead of on the basis-free state space $V$.

######  Proposition 2.3.

The above $\sigma_{\left(w,e\right)}\colon V\times R\rightarrow V\times R$ is
$\mathrm{GL}\left(V\right)$-equivariant.

###### Proof.

$\displaystyle\sigma_{\left(g\cdot w,g\cdot e\right)}\left(g\cdot v\right)$
$\displaystyle=g\cdot e^{(\textrm{m})}\cdot\sigma^{F}\left(h_{\left(g\cdot
w,g\cdot e\right)}\left(g\cdot e^{(\textrm{m})}_{1},g\cdot
v\right),\ldots,h_{\left(g\cdot w,g\cdot e\right)}\left(g\cdot
e^{(\textrm{m})}_{n_{\textrm{m}}},g\cdot v\right)\right)$
$\displaystyle=g\cdot
e^{(\textrm{m})}\cdot\sigma^{F}\left(h_{\left(w,e\right)}\left(e^{(\textrm{m})}_{1},v\right),\ldots,h_{\left(w,e\right)}\left(e^{(\textrm{m})}_{n_{\textrm{m}}},v\right)\right)=g\cdot\sigma_{(w,e)}(v)$

using Equation (6). ∎

The non-linear operations are called activation functions in machine learning.
We conclude the current setting by the following definition.

######  Definition 2.4.

An _activation module_ consists of:

  1. (1)

a (noncommutative) algebra $\mathcal{A}$ and vector spaces
$V,\,F=F_{\mathrm{in}}\oplus F_{\mathrm{out}}\oplus F_{\mathrm{m}}$;

  2. (2)

A family of metrics $h_{\left(w,e\right)}$ on $V$ over the space of framed
$\mathcal{A}$-modules

$R=\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(F,V\right)$

which is $\mathrm{GL}\left(V\right)$-equivariant;

  3. (3)

a collection of possibly non-linear functions

$\sigma_{j}^{F}\colon F_{\mathrm{m}}\to F_{\mathrm{m}}.$

The data of (1) and (2) (without (3)) is called a Hermitian family of framed
modules.

Figure 2 shows a schematic picture of an activation module.

Figure 2.

In this setting, $\sigma_{j}^{F}$ is a function on $F_{\mathrm{m}}$. We take
the subalgebra

(7)
$\mathcal{L}(\mathcal{A}_{\mathrm{m}}):=\mathfrak{e}_{\mathrm{m}}^{*}\cdot\mathcal{A}_{\mathrm{m}}\cdot\mathfrak{e}_{\mathrm{m}}$

consisting of loops at $F_{\mathrm{m}}$, the near-ring

(8)
$\widetilde{\mathcal{A}_{\mathrm{m}}}:=(\mathcal{L}(\mathcal{A}_{\mathrm{m}}))\\{\varsigma_{1},\ldots,\varsigma_{N}\\},$

and

$(\widetilde{\mathcal{A}_{\mathrm{m}}})_{0}:=\mathcal{A}\cdot\mathfrak{e}_{\mathrm{m}}\cdot\widetilde{\mathcal{A}_{\mathrm{m}}}\cdot\mathfrak{e}_{\mathrm{m}}^{*}\cdot\mathcal{A}.$

Note that $\widetilde{\mathcal{A}_{\mathrm{m}}}$ is different from
$\tilde{\mathcal{A}}_{\mathrm{m}}$ in Equation (2), since we now have non-
linear functions defined on $F$ instead of $V$.

Using Proposition 2.2 and 2.3, each algorithm
$\tilde{\gamma}\in(\widetilde{\mathcal{A}_{\mathrm{m}}})_{0}$ and
$[w,e]\in\mathcal{M}$ gives a machine function $f^{\tilde{\gamma}}_{[w,e]}$.
This gives a map

$(\widetilde{\mathcal{A}_{\mathrm{m}}})_{0}\to\Gamma\left(\mathcal{M},\mathbf{Map}\left(F_{\mathrm{in}},F_{\mathrm{out}}\right)\right).$

In applications, an activation module may consist of several linear
submodules, which are connected by possibly-nonlinear transitions
$\sigma^{F}_{i}$. This means the algebra $\mathcal{A}$ is a direct sum
$\bigoplus_{k\in K}\mathcal{A}^{(k)}$ where each $\mathcal{A}^{(k)}$ is
understood as a linear component of the activation module (and $K$ is an index
set). Similarly we have $V=\bigoplus_{k\in K}V^{(k)}$ and $F=\bigoplus_{k\in
K}F^{(k)}$. We take the moduli stack $\prod_{k\in
K}[R^{(k)}/\mathrm{GL}(V^{(k)})]$ (where
$R^{(k)}=\mathrm{Hom}_{\mathrm{alg}}\left(A^{\left(k\right)},\mathrm{End}\left(V^{\left(k\right)}\right)\right)\times\mathrm{Hom}\left(F^{\left(k\right)},V^{\left(k\right)}\right)$)
instead of $[R/\mathrm{GL}(V)]$. Each $F^{(k)}$ has three components
$F^{(k)}=F^{(k)}_{\mathrm{in}}\oplus F^{(k)}_{\mathrm{out}}\oplus
F^{(k)}_{\mathrm{m}}$ (where some of the components can simply be $\\{0\\}$).
Furthermore, the non-linear functions $\sigma^{F}_{j}:F_{\mathrm{m}}\to
F_{\mathrm{m}}$ is a composition $\iota\circ s_{j}^{F}\circ\pi$, where
$s_{j}^{F}:F_{\mathrm{m}}^{\left(p_{j,1}\right)}\times\ldots\times
F_{\mathrm{m}}^{\left(p_{j,{m_{j}}}\right)}\rightarrow
F_{\mathrm{m}}^{\left(q_{j,1}\right)}\times\ldots\times
F_{\mathrm{m}}^{\left(q_{j,{n_{j}}}\right)}$ for some fixed
$\\{p_{j,1},\ldots,p_{j,{m_{j}}}\\}$ and $\\{q_{j,1},\ldots,q_{j,{n_{j}}}\\}$;
$\pi$ is the projection $F_{\mathrm{m}}\to
F_{\mathrm{m}}^{\left(p_{j,1}\right)}\oplus\ldots\oplus
F_{\mathrm{m}}^{\left(p_{j,{m_{j}}}\right)}$ and $\iota$ is the inclusion (or
extension by zero) $F_{\mathrm{m}}^{\left(q_{j,1}\right)}\oplus\ldots\oplus
F_{\mathrm{m}}^{\left(q_{j,{n_{j}}}\right)}\to F_{\mathrm{m}}$. Finally, $h$
is a direct sum $h_{\left(w,e\right)}=\bigoplus_{k\in
K}h_{\left(w^{\left(k\right)},e^{\left(k\right)}\right)}$ where each
$h_{\left(w^{\left(k\right)},e^{\left(k\right)}\right)}$ is a family of
metrics $h_{\left(w^{\left(k\right)},e^{\left(k\right)}\right)}$ on
$V^{\left(k\right)}$ over the space of framed $A^{\left(k\right)}$-modules
$R^{(k)}$ which is $\mathrm{GL}\left(V^{\left(k\right)}\right)$-equivariant.

We can also define a closely related setting that uses unitary framed modules,
which takes the unitary group $U(V,h)$ in place of $\mathrm{GL}(V)$, and takes
a single Hermitian metric $h$ in place of a family of Hermitian metrics.

######  Definition 2.5.

A _unitary activation module_ consists of:

  1. (1)

A Hermitian vector space $(V,h)$, a framing vector space
$F=F_{\mathrm{in}}\oplus F_{\mathrm{out}}\oplus F_{\mathrm{m}}=\mathbb{C}^{n}$
(equipped with the standard metric), and unitary framing maps
$e_{\bullet}:F_{\bullet}\to V$, where
$\bullet=\mathrm{in},\mathrm{out},\mathrm{m}$.

  2. (2)

A group ring $\mathcal{A}=\mathbb{C}[G]$ where $G$ is a subgroup of the
unitary group $U(V,h)$. $\mathbb{C}[G]$ consists of linear combinations
$\sum_{g\in G}c_{g}g$ for $c_{g}\in\mathbb{C}$.

  3. (3)

a collection of possibly non-linear functions

$\sigma_{j}^{F}\colon F_{\mathrm{m}}\to F_{\mathrm{m}}.$

Such a setting suits well for quantum computing. Namely, $(V,h)$ can be taken
to be the state space of a quantum system of particles. $G$ is a subgroup of
unitary operators on $(V,h)$. $F_{\textrm{m}}$ can be taken to have the same
dimension as $V$, and $e_{\textrm{m}}:F_{\textrm{m}}\to V$ maps the standard
basis of $F_{\textrm{m}}$ to an assigned unitary basis of $V$. (For instance,
the assigned basis can be $\\{|00\rangle,|01\rangle,|10\rangle,|11\rangle\\}$
for a 2-qubit system). There is a probabilistic projection
$\sigma_{0}:F_{\textrm{m}}\to F_{\textrm{m}}$ that corresponds to wave-
function collapse following each observation. We also have other non-linear
classical operations $\sigma_{j}^{F}$ on $F_{\textrm{m}}$.

In application, we are given input data $v\in F_{\textrm{in}}$. $v$
(normalized to have length $1$) is sent to the Hermitian state space $V$ by
$e_{\textrm{in}}$, and operated under a prescribed linear algorithm
$a\in\mathbb{C}[G]$. Then the system is observed and recorded using the basis
$e_{\textrm{m}}$. This gives $\sigma_{0}\cdot\sum_{l}h(e_{\textrm{m},l},a\cdot
e_{\textrm{in}}\cdot v)e_{\textrm{m},l}$. The recorded memory can be operated
by a non-linear algorithm consisting of $\sigma_{j}^{F}$. The process can be
iterated and give a function $F_{\textrm{in}}\to F_{\textrm{out}}$.

In this paper, we focus on Definition 2.4, for the purpose of neural network
and deep learning which works with $\mathrm{GL}(n)$ rather than $U(n)$.

###  2.3. Noncommutative geometry and machine learning

We have formulated a computing machine by a Hermitian family of framed
$\mathcal{A}$-modules and a collection of non-linear functions. If we ignore
the non-linear functions for the moment, and merely consider the augmented
algebra $\mathcal{A}_{\mathrm{m}}$, it fits well to the framework of
noncommutative geometry developed by Connes [Con85], Cuntz-Quillen [CQ95],
Ginzburg [Gin05]. Below we give a quick review and apply to our situation.
[Tac17] gives a beautiful survey on this theory. We will extend it to near-
ring in the next subsection.

####  2.3.1. A quick review

The theory develops an analog of the de Rham complex of differential forms for
an associative algebra $A$ over a field $\mathbb{K}$ (that we take to be
$\mathbb{C}$ in this paper). This is a crucial step to develop the notions of
cohomology, connection and curvature for the noncommutative space associated
to $A$ and its associated vector bundles.

The noncommutative differential forms can be described as follows. Consider
the quotient vector space $\overline{A}=A/\mathbb{K}$ (which is no longer an
algebra). We think of elements in $\overline{A}$ as differentials. Define

$D(A):=\bigoplus_{n\in\mathbb{Z}_{\geq
0}}D(A)_{n},\,\,D(A)_{n}:=A\otimes\overline{A}\otimes\ldots\otimes\overline{A}$

where $n$ copies of $\overline{A}$ appear in $D(A)_{n}$, and the tensor
product is over the ground field $\mathbb{K}$. We should think of elements in
$\overline{A}$ as _matrix-valued_ differential one-forms. Note that $X\wedge
X$ may not be zero, and $X\wedge Y\not=-Y\wedge X$ in general for matrix-
valued differential forms $X,Y$.

The differential $d_{n}:D(A)_{n}\to D(A)_{n+1}$ is defined as

$d_{n}(a_{0}\otimes\overline{a_{1}}\otimes\ldots\otimes\overline{a_{n}}):=1\otimes\overline{a_{0}}\otimes\ldots\otimes\overline{a_{n}}.$

The product $D(A)_{n}\otimes D(A)_{m-1-n}\to D(A)_{m-1}$ is more tricky:

$\displaystyle(a_{0}\otimes\overline{a_{1}}\otimes\ldots\otimes\overline{a_{n}})\cdot(a_{n+1}\otimes\overline{a_{n+2}}\otimes\ldots\otimes\overline{a_{m}})$
(9) $\displaystyle:=$
$\displaystyle(-1)^{n}a_{0}a_{1}\otimes\overline{a_{2}}\otimes\ldots\otimes\overline{a_{m}}+\sum_{i=1}^{n}(-1)^{n-i}a_{0}\otimes\overline{a_{1}}\otimes\ldots\otimes\overline{a_{i}a_{i+1}}\otimes\ldots\otimes\overline{a_{m}}$

which can be understood by applying the Leibniz rule on the terms
$\overline{a_{i}a_{i+1}}$. Note that we have chosen representatives $a_{i}\in
A$ for $i=1,\ldots,n+1$ on the RHS, but the sum is independent of choice of
representatives (while the product $\overline{a_{i}a_{i+1}}$ itself depends on
representatives).

The above product in particular gives a bimodule structure on $D(A)$ over
$A=D(A)_{0}$. For instance, $D(A)_{1}$ has the bimodule structure

$a\cdot(a_{0}\otimes\overline{a_{1}})=aa_{0}\otimes\overline{a_{1}},\,\,(a_{0}\otimes\overline{a_{1}})\cdot
a=-a_{0}a_{1}\otimes\overline{a}+a_{0}\otimes\overline{a_{1}a}.$

(If $a_{1}$ is replaced by $a_{1}+k$ for $k\in\mathbb{K}$, then RHS
$=-a_{0}a_{1}\otimes\overline{a}-ka_{0}\otimes\overline{a}+a_{0}\otimes\overline{a_{1}a}+ka_{0}\otimes\overline{a}=-a_{0}a_{1}\otimes\overline{a}+a_{0}\otimes\overline{a_{1}a}$
remains unchanged.)

By [CQ95],

$d^{2}=0.$

The above differential $d$ and product defines a dg-algebra structure on
$D(A)$; indeed this is the unique one that satisfies $a_{0}\cdot
da_{1}\cdot\ldots\cdot
da_{n}=a_{0}\otimes\overline{a_{1}}\otimes\ldots\otimes\overline{a_{n}}$.
Moreover, $(D(A),i)$, where $i:A\to D(A)_{0}=A$ is the identity map, has the
following universal property: for every $(\Gamma,\psi)$ where $\Gamma$ is a dg
algebra and $\psi:A\to\Gamma_{0}$ is an algebra homomorphism, there exists an
extension as a dg-algebra map $u_{\psi}:D(A)\to\Gamma$ such that the degree-
zero part satisfies $(u_{\psi})_{0}\circ i=\psi$.

Here is another realization of differential forms for $A$. First, define the
$A$-bimodule $\Omega^{1}(A):=\mathrm{Ker}\,(\mu)$ where $\mu:A\otimes A\to A$
is the multiplication map for $A$. Moreover, define $d:A\to\Omega^{1}(A)$ by
$da:=1\otimes a-a\otimes 1$. Thus $\sum_{i}a_{i}da_{i}^{\prime}$ for
$a_{i},a_{i}^{\prime}\in A$ is an element in $\Omega^{1}(A)$. Conversely, any
element in $\Omega^{1}(A)$ is of the form $\sum_{i}a_{i}\otimes
a_{i}^{\prime}$ with $\sum_{i}a_{i}\cdot a_{i}^{\prime}=0$, and this equals to

$\sum_{i}a_{i}da_{i}^{\prime}=-\sum_{i}(da_{i})a_{i}^{\prime}.$

Then we take the tensor algebra

$\Omega^{\bullet}(A):=T_{A}(\Omega^{1}(A))=\bigoplus_{i\in\mathbb{Z}_{\geq
0}}\Omega^{1}(A)\otimes_{A}\ldots\otimes_{A}\Omega^{1}(A)$

where there are $i$ copies of $\Omega^{1}(A)$ for the summands on the right.
An element in $\Omega^{\bullet}(A)$ takes the form
$a_{1}db_{1}\otimes_{A}a_{2}db_{2}\otimes_{A}\ldots\otimes_{A}a_{k}db_{k}\cdot
a_{k+1}$. Recall that tensoring over $A$ means the identification $db_{1}\cdot
a\otimes_{A}db_{2}=db_{1}\otimes_{A}adb_{2}$.

The two defined graded algebras $\Omega^{\bullet}(A)$ and $D(A)$ are
isomorphic. For one forms, we have the $A$-bimodule map $\psi:\Omega^{1}(A)\to
D(A)_{1}$ defined by $da\mapsto 1\otimes\overline{a}$. It has the inverse
$a_{0}\otimes\overline{a_{1}}\mapsto a_{0}\otimes a_{1}-a_{0}a_{1}\otimes 1$
(which is again independent of choice of representative $a_{1}$). For higher
forms, $\Omega^{n}\to D(A)_{n}$ is given by
$\alpha_{1}\otimes_{A}\ldots\otimes_{A}\alpha_{n}\mapsto\psi(\alpha_{1})\cdot\ldots\cdot\psi(\alpha_{n})$
(where the non-trivial product on $D(A)$ is given in Equation (9)), whose
inverse is
$a_{0}\otimes\overline{a_{1}}\otimes\ldots\otimes\overline{a_{n}}=(a_{0}\otimes\overline{a_{1}})\cdot(1\otimes\overline{a_{2}})\ldots(1\otimes\overline{a_{n}})\mapsto\psi^{-1}(a_{0}\otimes\overline{a_{1}})\otimes_{A}\psi^{-1}(1\otimes\overline{a_{2}})\otimes_{A}\ldots\otimes_{A}\psi^{-1}(1\otimes\overline{a_{n}})$.

The Karoubi-de Rham complex is defined as

(10)
$DR^{\bullet}(A):=\Omega^{\bullet}(A)/[\Omega^{\bullet}(A),\Omega^{\bullet}(A)]$

where $[a,b]:=ab-(-1)^{ij}ba$ is the graded commutator for a graded algebra.
$d$ descends to be a well-defined differential on $DR^{\bullet}(A)$. Note that
$DR^{\bullet}(A)$ is not an algebra since
$[\Omega^{\bullet}(A),\Omega^{\bullet}(A)]$ is not an ideal. $DR^{\bullet}(A)$
is the non-commutative analog for the space of de Rham forms. Moreover, there
is a natural map by taking trace to the space of $G$-invariant differential
forms on the space of representations $R(A)$:

(11)
$DR^{\bullet}\left(A\right)\rightarrow\Omega^{\bullet}\left(R\left(A\right)\right)^{G}.$

$DR^{0}(A)$ and $DR^{1}(A)$ will be the most relevant to us. We have
$DR^{0}(A)=A/[A,A]$ and $DR^{1}(A)=\Omega^{1}(A)/[A,\Omega^{1}(A)]$.

Dually, derivations $\theta\in\mathrm{Der}(A)$ play the role of vector fields.
A derivation $\delta:A\to A$ is a linear map satisfying
$\delta(ab)=\delta(a)\cdot b+a\cdot\delta(b)$. $\mathrm{Der}(A)$ is the vector
space of all derivations. We have the $A$-bimodule map
$\iota_{\theta}:\Omega^{1}(A)\to A$, $\iota_{\theta}(da):=\theta(a)$ called
contraction. $\iota_{\theta}$ extends to
$\Omega^{\bullet}(A)\to\Omega^{\bullet-1}(A)$ by using graded Leibniz rule,
and descends to $DR^{\bullet}(A)\to DR^{\bullet-1}(A)$.

The following version of differential forms relative to a subalgebra [CQ95]
will be useful for framings and quivers. Let $B\subset A$ be a commutative
subalgebra. We take

$D(A/B)_{n}:=A\otimes_{B}\bar{A}\otimes_{B}\ldots\otimes_{B}\bar{A}$

where $\bar{A}$ is the vector space

$\bar{A}:=A/B.$

Then we repeat the same definitions as above for $DR^{\bullet}(A/B)$. Note
that zero-th forms are the same as before:
$DR^{0}\bullet(A/B)=DR^{0}\bullet(A)$. There is a natural map [Gin05]

$DR^{\bullet}(A/B)\to\Omega^{\bullet}\left(R_{B}\left(A\right)\right)^{G_{B}}$

where $R_{B}(A)$ is the set of $A$-modules whose restriction to $B$ equals to
a prescribed $B$-module, and $G_{B}$ is the subgroup in $\mathrm{GL}(V)$ that
preserves the prescribed $B$-bimodule structure.

In the context of $A$ being the path algebra of a quiver, we shall take $B$ to
be the subalgebra generated by the trivial paths $1_{i}$ at all vertices $i\in
Q_{0}$. Then a differential form

$a_{0}(da_{1})a_{2}\ldots(da_{k})\in DR^{\bullet}(A/B)$

is non-zero only if the paths $a_{i}$ can be concatenated:
$t(a_{j})=h(a_{j+1})$ for all $j\in\mathbb{Z}/(k+1)$. In this case a
prescribed $B$-module structure on $V$ is given by a decomposition
$V=\bigoplus_{i\in Q_{0}}V_{i}$ and $1_{i}$ acts as the projection $V\to
V_{i}$. Then $G_{B}=\prod_{i\in Q_{0}}\mathrm{GL}(V_{i})$.

####  2.3.2. Application to linear machine learning

Now we come back to the context of the last subsection. The additional
ingredient we need to take care of is the equivariant family of Hermitian
metrics $h$ on the $A$-modules.

To precisely match the language, first let’s modify the definition for
$\mathcal{A}_{\mathrm{m}}$ (Equation (1)) as follows. Recall that the framing
vector space $F=F_{\mathrm{in}}\oplus F_{\mathrm{out}}\oplus
F_{\mathrm{m}}=\mathbb{C}^{n_{\mathrm{in}}}\oplus\mathbb{C}^{n_{\mathrm{out}}}\oplus\mathbb{C}^{n_{\mathrm{m}}}$,
where $\dim F=n$. Then a framing $e$ can be written as $(e_{1}\ldots e_{n})$
where $e_{j}\in V$, and $e^{*}$ is the column vector
$(e^{*}_{1},\ldots,e^{*}_{n})$ where $e^{*}_{j}\in V^{*}$.

First we take the augmentation

$\mathcal{A}^{\mathfrak{e}}:=\mathcal{A}\langle
1_{F},\mathfrak{e}_{j}:j=1,\ldots,n\rangle/I$

where $I$ is the two-sided ideal generated by
$1_{F}\cdot\mathfrak{e}_{j},\,\mathfrak{e}_{j}\cdot
1_{F}-\mathfrak{e}_{j},\,1_{\mathcal{A}}\cdot\mathfrak{e}_{j}-\mathfrak{e}_{j},\,\mathfrak{e}_{j}\mathfrak{e}_{k},\,\,\mathfrak{e}_{j}\cdot
a,a\cdot 1_{F},\,1_{F}\cdot a$ for all $a\in\mathcal{A},j,k=1,\ldots,n$.

Then we take its _doubling_ $\hat{\mathcal{A}}$, which is generated by two
copies of $\mathcal{A}^{\mathfrak{e}}$ (whose generators are denoted by
$a,1_{F},\mathfrak{e}_{j}$ and $a^{*},1_{F}^{*},\mathfrak{e}_{j}^{*}$
respectively), quotient out the ideal of relations
$1_{\mathcal{A}}-1_{\mathcal{A}}^{*},1_{F}-1_{F}^{*}$. The unit of
$\hat{\mathcal{A}}$ is

$1_{\hat{\mathcal{A}}}=1_{F}+1_{\mathcal{A}}.$

We also use the rule $(ab)^{*}:=b^{*}a^{*}$ to define the formal adjoint of a
general element in $\hat{\mathcal{A}}$.

######  Remark 2.6.

This doubling procedure is standard in the construction of Nakajima quiver
varieties, which is an algebraic analog of taking the cotangent bundle (or
complexification) of a variety. We will restrict to a section to go back to
$[R/G]$.

In the notation of the last subsection, we take $A=\hat{\mathcal{A}}$ and the
commutative subalgebra

$B=\mathrm{Span}_{\mathbb{C}}\\{1_{F},1_{\mathcal{A}}\\}\subset\hat{\mathcal{A}}.$

Consider $V\oplus\mathbb{C}$. We fix its $B$-module structure in the way that
$1_{\mathcal{A}}$ and $1_{F}$ act as $(\mathrm{Id}_{V},0)$ and
$(0,\mathrm{Id}_{\mathbb{C}})$ respectively. $V\oplus\mathbb{C}$ can be
equipped with $\hat{\mathcal{A}}$-module structure that restricts to be this
fixed $B$-module structure.

######  Lemma 2.7.

Given a Hermitian family of framed modules $(\mathcal{A},V,F,h)$, there is a
one-to-one correspondence between elements in
$R=\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(F,V\right)$
and $\hat{\mathcal{A}}$-modules of the form $V\oplus\mathbb{C}$ that respect
the $B$-module structure and have $\mathfrak{e}_{j}^{*},a^{*}$ acting as the
adjoints of $e_{j}$ and $w(a)$ respectively with respect to $h$.

###### Proof.

Given $(w,e)\in R$, the $\hat{\mathcal{A}}$-module structure on
$V\oplus\mathbb{C}$ is defined as follows. $w$ gives the action of
$\mathcal{A}$ on $V$, and $\mathcal{A}$ acts on the component $\mathbb{C}$ by
zero. $\mathfrak{e}_{j}$ acts as the linear map $e_{j}:\mathbb{C}\to V$ where
$e_{j}$ is the $j$-th column of $e$, and acts on $V$ trivially.
$\mathfrak{e}_{j}^{*}$ and $a^{*}$ act on the component $\mathbb{C}$ by zero,
and act as the adjoint maps of $\mathfrak{e}_{j}$ and $w(a)$ with respect to
$h$. The adjoint maps are

$e_{j}^{*_{h}}:V\to\mathbb{C},\,e_{j}^{*_{h}}(v)=h(e_{j},v)$

and

$w(a)^{*_{h}}=h_{(w,e)}^{-1}w(a)^{*}h_{(w,e)}$

in matrix form.

Conversely, since the $\hat{A}$-module is required to restrict as the given
$B$-module structure, we must have $\mathcal{A}$ acting trivially on the
component $\mathbb{C}$, $\mathfrak{e}_{j}$ acting trivially on $V$, and
$\mathfrak{e}_{j}^{*}$ acting trivially on $\mathbb{C}$. (For instance,
$a=a\cdot 1_{\mathcal{A}}$ acts as $(a,0)$ on $V\oplus\mathbb{C}$.) Then the
action of $\mathcal{A}$ and $(\mathfrak{e}_{j}:j=1,\ldots,n)$ gives an element
in $R$. ∎

Similar to (11), we have the following map for $\hat{\mathcal{A}}$. The only
difference is that for the forms $d\mathfrak{e}_{j}^{*}$ and $da^{*}$, the
corresponding forms on

$R=\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(F,V\right)$

are defined using the metrics $h$.

######  Proposition 2.8.

Given a Hermitian family of framed modules $(\mathcal{A},V,F,h)$, there is a
(degree-preserving) map

$DR^{\bullet}(\hat{\mathcal{A}}/B)\to\Omega^{\bullet}\left(R\right)^{G_{B}}$

that commutes with differential, and equals to the trace of the corresponding
representations given in Lemma 2.7 when restricted to
$DR^{0}(\hat{\mathcal{A}}/B)\to\Omega^{0}\left(R\right)^{G_{B}}$.

###### Proof.

$DR^{\bullet}(\hat{\mathcal{A}}/B)$ is generated by the one forms $da$,
$da^{*}$, $d\mathfrak{e}_{j}$ and $d\mathfrak{e}_{j}^{*}$ over
$\hat{\mathcal{A}}$. For $da$ and $d\mathfrak{e}_{j}$, the corresponding
matrix-valued one-forms on $R$ are obvious (by substituting $a$ and
$\mathfrak{e}_{j}$ by the corresponding representing matrices $w(a)$ and
$e_{j}$). For $d\mathfrak{e}_{j}^{*}$ and $da^{*}$, the corresponding matrix-
valued one-form over $R$ are

$(\bar{\partial}e_{j}^{*})\cdot h+e_{j}^{*}\cdot
dh=(\bar{\partial}e_{j}^{*})\cdot h+e_{j}^{*}\cdot(\bar{\partial}h+\partial
h)$

and

(12) $-h^{-1}\cdot dh\cdot
h^{-1}w_{a}^{*}h+h^{-1}(\bar{\partial}w_{a}^{*})h+h^{-1}w_{a}^{*}dh$

respectively, where $h$ is now represented by a square matrix in a basis of
$V$, $e_{j}^{*}$ (a row vector) and $w_{a}^{*}$ are the conjugate transpose of
$e_{j}$ and $w_{a}$ respectively. Note that $h_{(w,e)}$ is a function on
$(w,e)\in R$ and so it has a non-trivial differential $dh$. More
intrinsically, $d\mathfrak{e}_{j}^{*}$ corresponds to $h(\nabla
e_{j},\cdot)+h(e_{j},\nabla\cdot)$, where $\nabla$ is the Chern connection of
$h$ on the trivial vector bundle $V\times R$ (and $e_{j}$ is a section).

Note that non-zero elements in $DR^{\bullet}(\hat{\mathcal{A}}/B)$ are
represented by loops (meaning that the source and target are the same), due to
the defining equation (10). The corresponding forms on $R$ are obtained by
composing the above matrices and taking trace. In particular, it is the trace
of the corresponding representing matrix when restricted to
$DR^{0}(\hat{A}/B)$. Since trace is independent of cyclic permutations of the
composition, the map $DR^{\bullet}(\hat{A}/B)\to\Omega^{\bullet}(R)$ is well-
defined. Moreover, it is obvious that it commutes with differential by
definition.

Under the action of $g\in\mathrm{GL}(V)$, $d(w(a))\mapsto g\cdot d(w(a))\cdot
g^{-1}$, $de_{j}\mapsto g\cdot de_{j}$,

$(\bar{\partial}e_{j}^{*})\cdot h+e_{j}^{*}\cdot
dh\mapsto(\bar{\partial}e_{j}^{*})g^{*}\cdot(g^{*})^{-1}hg^{-1}+e_{j}^{*}g^{*}\cdot(g^{*})^{-1}dh\,g^{-1}=((\bar{\partial}e_{j}^{*})\cdot
h+e_{j}^{*}\cdot dh)\cdot g^{-1}$

and (12) transforms by $g\,(\cdot)\,g^{-1}$, using the
$\mathrm{GL}(V)$-equivariance of the family of metrics $h$. Since trace is
invariant under conjugation, the corresponding forms on $R$ are
$G_{B}$-invariant. Here $G_{B}=\mathbb{C}^{\times}\times\mathrm{GL}(V)$, where
$\mathbb{C}^{\times}$ is Abelian and acts trivially on $R$. ∎

######  Remark 2.9.

Since the above uses the family of Hermitian metrics $h$, the resulting forms
in $\Omega^{\bullet}(R)^{G_{B}}$ are no longer holomorphic. In the usual
algebraic construction, we have a map $\rho$ from
$DR^{p}(\hat{\mathcal{A}}/B)$ to $\mathrm{GL}(V)$-invariant holomorphic
$(p,0)$-forms on

$(\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right))^{2}\times\mathrm{Hom}\left(F,V\right)\times\mathrm{Hom}\left(V,F\right).$

The above can be understood as a composition of the usual map

$\rho:DR^{\bullet}(\hat{\mathcal{A}}/B)\to\Omega^{\bullet}(R\times(\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(V,F\right)))$

together with pulling back by the smooth section of
$R\times(\mathrm{Hom}_{\mathrm{alg}}\left(\mathcal{A},\mathrm{End}\left(V\right)\right)\times\mathrm{Hom}\left(V,F\right))\to
R$ defined by

$e_{j}^{\prime}=h_{(w,e)}(e_{j},\cdot)=e_{j}^{*}\cdot
h_{(w,e)},\,\,w_{a}^{\prime}=h_{(w,e)}^{-1}w_{a}^{*}h_{(w,e)}.$

On the LHS, $(e_{j}^{\prime}:j=1,\ldots,n)\in\mathrm{Hom}\left(V,F\right)$ and
$w_{a}^{\prime}\in\mathrm{End}(V)$ denotes fiber coordinates; on the RHS,
$e_{j}^{*}$ is the conjugate transpose of the column vector in $(e_{1}\ldots
e_{n})\in\mathrm{Hom}(F,V)$. Note that the action of $\mathrm{GL}(V)$ on both
sides of the first and second equations are right multiplication by $g^{-1}$
and conjugation $g\,(\cdot)\,g^{-1}$ respectively.

Now define the subalgebra

$\mathcal{L}(\hat{\mathcal{A}}):=\bigoplus_{j,k=1}^{n}\mathfrak{e}_{j}^{*}\cdot\hat{\mathcal{A}}\cdot\mathfrak{e}_{k}.$

Recall that elements in $\mathcal{L}(\hat{\mathcal{A}})$ are understood as
linear algorithms.

In $DR^{0}(A/B)=A/(B+[A,A])$ (vector-space quotient), note that elements that
do not form loop (for instance, $a\cdot\mathfrak{e}_{j}$ and
$\mathfrak{e}_{j}^{*}\cdot a$) are in the zero class. Moreover, loops that are
cyclic permutation of each other are identified as the same class.

In our context, elements in $\mathcal{L}(\hat{\mathcal{A}})$ are loops, and
descend to non-trivial elements in $DR^{0}(A/B)$. As a consequence:

######  Corollary 2.10.

An element in $\mathcal{L}(\hat{\mathcal{A}})$ induces a $G$-invariant
function $f$ on $R$ where $G=\mathrm{GL}(V)$. Its differential lies in
$DR^{1}(A/B)$ and induces the corresponding differential
$df\in\Omega^{1}(R)^{G}$.

Note that the target of $\mathfrak{e}_{j}^{*}$ and the domain of
$\mathfrak{e}_{j}$ are the one-dimensional vector space $\mathbb{C}$. Thus the
matrix corresponding to $\mathfrak{e}_{j}^{*}\cdot
a\cdot\mathfrak{e}_{k}\in\mathcal{L}(\hat{\mathcal{A}})$ is one-by-one whose
trace just equals to itself.

An $(n\times n)$-matrix whose entries lie in $\mathcal{L}(\hat{\mathcal{A}})$
gives a linear function $F\to F$ over each point in $[R/G]$. We can also
restrict it to

$f_{[w,e]}:F_{\mathrm{in}}\to F_{\mathrm{out}}$

by taking an $(n_{\mathrm{out}}\times n_{\mathrm{in}})$-matrix whose entries
$\gamma_{jk}$ belong to
$\mathfrak{e}_{\mathrm{out},k}^{*}\cdot\hat{\mathcal{A}}\cdot\mathfrak{e}_{\mathrm{in},j}$
where $(\mathfrak{e}_{\mathrm{in},j}:j=1,\ldots,n_{\mathrm{in}})$ denotes the
part of $(\mathfrak{e}_{j}:j=1,\ldots,n)$ that has source in $F_{\mathrm{in}}$
(and similar for $\mathfrak{e}_{\mathrm{out},k}$). This produces a linear
machine function $f^{\gamma}_{[w,e]}$ corresponding to a linear algorithm
$\gamma$.

The cost function can also be defined algebraically as an element in
$DR^{0}(A/B)$. Namely, given a function $f:F_{\mathrm{in}}\to
F_{\mathrm{out}}$ and fixing $v\in
F_{\mathrm{in}}=\mathbb{C}^{n_{\mathrm{in}}}$, the expression

$\displaystyle E$
$\displaystyle=\int_{K}\left|\left(\sum_{j}\gamma_{jk}\,v_{j}:k=1,\ldots,n_{\mathrm{out}}\right)-f\left(v\right)\right|_{F_{\mathrm{out}}}^{2}dv$
$\displaystyle=\int_{K}\sum_{k}\left(\sum_{j}\gamma_{jk}\,v_{j}-f_{k}(v)\right)\left(\sum_{j}\gamma_{jk}^{*}\,\overline{v_{j}}-\overline{f_{k}(v)}\right)dv$

lies in $DR^{0}(A/B)$. Its differential in $DR^{1}(A/B)$ induces a one-form on
$[R/G]$, which plays a central role in machine learning.

Suppose $\mathcal{A}$ is finitely generated, and so does $\hat{\mathcal{A}}$.
Let $\\{x_{j}:j=1,\ldots,M\\}$ be the generators of $\hat{\mathcal{A}}$. Then
the algebraic Jacobian ring

$DR^{0}(\hat{\mathcal{A}}/B)/\langle\partial_{x_{j}}E:j=1,\ldots,M\rangle,$

where $\partial_{x_{j}}E$ is the cyclic differential, is useful in capturing
the critical locus of $E$.

###  2.4. Differential forms for near-ring

The associative algebra $A$ in the last subsection captures linear operations
of a computing machine, and has interesting noncommutative geometries. In this
subsection, we incorporate non-linear operations and extend the geometric
construction to a near-ring.

####  2.4.1. Near-rings and their representations

######  Definition 2.11.

A near-ring is a set $\tilde{A}$ with two binary operations $+,\circ$ called
addition and multiplication such that

  1. (1)

$\tilde{A}$ is a group under addition.

  2. (2)

Multiplication is associative.

  3. (3)

Right multiplication is distributive over addition:

$(x+y)\circ z=x\circ z+y\circ z$

for all $x,y,z\in\tilde{A}$.

In this paper, the near-ring we use will be required to satisfy that:

  1. (4)

$(\tilde{A},+)$ is a vector space over $\mathbb{F}=\mathbb{C}$, with
$c\cdot(x\circ y)=(c\cdot x)\circ y$ for all $c\in\mathbb{C}$ and
$x,y\in\tilde{A}$.

  2. (5)

There exists $1\in\tilde{A}$ such that $1\circ x=x=x\circ 1$.

We call it a near-ring over $\mathbb{C}$ with identity, or a
$\mathbb{C}$-near-ring with identity.

Note that $x\circ(c\cdot y)\not=c\cdot x\circ y$ in general. The following
gives a prototype example.

######  Example 2.12.

The set $\mathrm{Map}(V,V)$ of $\mathbb{C}$-valued smooth functions $f:V\to V$
on a vector space $V$ forms a near-ring over $\mathbb{C}$ with identity, with
$+$ being the addition on the vector space, $\circ$ being the composition of
functions, and $1$ being the identity function on $V$.

######  Definition 2.13.

Given a $\mathbb{C}$-near-ring with identity $\tilde{A}$, a $\mathbb{C}$-sub-
near-ring is a $\mathbb{C}$-subspace $\tilde{A}^{\prime}\subset\tilde{A}$
which is closed under the multiplication $\circ$. $\tilde{A}^{\prime}$ is
called a $\mathbb{C}$-sub-near-ring with identity if in addition,
$1\in\tilde{A}^{\prime}$.

Given an algebra $A$ and a set $S$, we have the $\mathbb{C}$-near-ring
$A\\{S\\}$ defined as follows.

######  Definition 2.14.

Let $A$ be a $\mathbb{C}$-algebra with identity and $S$ be a set. we define
the $\mathbb{C}$-near-ring with identity $A\\{S\\}$ as follows. As a vector
space,

$A\\{S\\}:=\bigoplus_{p=0}^{\infty}A\\{S\\}_{p}$

where:

  1. (1)

$A\\{S\\}_{0}=A$;

  2. (2)

Given $A\\{S\\}_{p}$ defined, $A\\{S\\}_{p+1}$ is spanned by the elements
$a\varsigma\circ\alpha$, where $a\in A$, $\varsigma\in S$, and $\alpha\in
A\\{S\\}_{p}$, subject to the relation
$(a_{1}\varsigma_{1}+ca_{2}\varsigma_{2})\circ\alpha=a_{1}\varsigma_{1}\circ\alpha+ca_{2}\varsigma_{2}\circ\alpha$
for all $c\in\mathbb{C}$, $a_{1},a_{2}\in A$.

Moreover, we define $1_{A}\circ\varsigma=\varsigma\circ 1_{A}=\varsigma$. Thus
$1_{A}$ is also the identity for $A\\{S\\}$.

In the application to neural network, the elements $\varsigma\in S$ are
symbols for the activation functions. Each element of $A\\{S\\}$ can be
recorded by a rooted tree (oriented towards the root) defined as follows.

######  Definition 2.15.

Given $\tilde{A}=A\\{S\\}$, an activation tree is a rooted tree with the
following labels.

  1. (1)

Leaves and the root are labeled by $1_{\tilde{A}}$;

  2. (2)

Edges are labeled by $a\in A$ ;

  3. (3)

Nodes that are neither leaves nor the root are labeled by $\varsigma\in S$.

Each node gives the output

(13) $\sum_{k}a_{k}\varsigma_{k}\circ\alpha_{k},$

where $a_{k}$ are the labels of the incoming edges, $\varsigma_{k}$ and
$\alpha_{k}$ are the labels of the tails of the incoming edges and their
outputs respectively. (At a leaf, the label is $1_{\tilde{A}}$ and the output
is $1_{\tilde{A}}$.) The element in $A\\{S\\}$ corresponding to the tree is
the output of its root.

######  Remark 2.16.

The expression (13) takes the pre-activation value as output of a node. One
can also slightly modify the definition of an activation tree and use the
other convention that takes the activation value as output.

######  Example 2.17.

Figure 3 shows examples of activation trees that represent elements in
$A\\{S\\}$. The expression corresponding to the rightmost tree is

$a_{0}+a_{1}\varsigma_{1}\circ(a_{1,0}+a_{1,1}\varsigma_{1,1}\circ
a_{1,1,0})+a_{2}\varsigma_{2}\circ a_{2,0}$

for some $a_{0},a_{1},a_{1,0},a_{1,1},a_{1,1,0},a_{2,0}\in A$,
$\varsigma_{1},\varsigma_{1,1},\varsigma_{2}\in S$.

Note that the tree here is not the digraph (quiver) that we will consider in
the later part of this paper. The labels $a$ for the edges will be taken to be
elements in the double of a quiver algebra $\hat{\mathcal{A}}$ later, and
required to be loops from the framing of the quiver back to itself.

Figure 3.

The above definition goes from a $\mathbb{C}$-algebra to a $\mathbb{C}$-near-
ring. In the reverse direction, we can define the following.

######  Definition 2.18.

The canonical subalgebra of a $\mathbb{C}$-near-ring $\tilde{A}$ with identity
is defined as

$A:=\\{x\in\tilde{A}:x\circ(cy+z)=cx\circ y+x\circ z\textrm{ for all
}y,z\in\tilde{A}\textrm{ and }c\in\mathbb{C}\\}.$

It is easy to check that

######  Lemma 2.19.

$A$ is a $\mathbb{C}$-algebra with identity.

######  Example 2.20.

For the above example that $\tilde{A}=\mathrm{Map}(V,V)$, the canonical
subalgebra is the subset $\mathrm{Lin}(V)$ of linear endomorphisms of $V$.
This can be seen by taking $y,z\in\mathrm{Map}(V,V)$ to be constant maps in
the above definition of $A$.

Given a subset $S$ of $\tilde{A}$, we have the sub-near-ring generated by $S$
defined as follows.

######  Definition 2.21.

The sub-near-ring of $\tilde{A}$ generated by $S$, which is denoted as
$\langle S\rangle_{\tilde{A}}$, is defined inductively as follows. As a vector
space,

$\langle S\rangle_{\tilde{A}}:=\sum_{p=0}^{\infty}\langle
S\rangle_{\tilde{A},p}\subset\tilde{A}$

where:

  1. (1)

$\langle S\rangle_{\tilde{A},0}=A$;

  2. (2)

Given $\langle S\rangle_{\tilde{A},p}$ defined, $\langle
S\rangle_{\tilde{A},p+1}$ is spanned by the elements
$a\circ\varsigma\circ\alpha$, where $a\in A$, $\varsigma\in S$, and
$\alpha\in\langle S\rangle_{\tilde{A},p}$.

$\tilde{A}$ is said to be finitely generated if $\tilde{A}=\langle
S\rangle_{\tilde{A}}$ for a finite subset $S\subset\tilde{A}$.
$S\subset\tilde{A}$ is said to be a free generating subset if $\langle
S\rangle_{\tilde{A}}=A\\{S\\}$.

It is easy to check that:

######  Proposition 2.22.

$\langle S\rangle_{\tilde{A}}$ defined above is a sub-near-ring.

######  Example 2.23.

Let’s continue the example of the set of functions $\mathrm{Map}(V,V)$. Fix a
collection of non-linear functions $\sigma_{1},\ldots,\sigma_{N}:V\to V$. This
corresponds to a finitely generated sub-near-ring
$A\\{\sigma_{1},\ldots,\sigma_{N}\\}\subset\tilde{A}$.
$\sigma_{1},\ldots,\sigma_{N}$ can be chosen such that they are not related by
iterated compositions and linear combinations. Then they form a free
generating subset.

######  Definition 2.24.

A morphism of $\mathbb{C}$-near-rings with identities is a map
$\Psi:\tilde{A}_{1}\to\tilde{A}_{2}$ that satisfies:

  1. (1)

$\Psi(x+y)=\Psi(x)+\Psi(y)$;

  2. (2)

$\Psi(x\circ y)=\Psi(x)\circ\Psi(y)$;

  3. (3)

$\Psi(1_{\tilde{A}_{1}})=1_{\tilde{A}_{2}}$.

$\Psi$ is said to be a strong morphism if in addition, it satisfies:

  1. (4)

$\Psi$ maps the canonical subalgebra of $\tilde{A}_{1}$ to that of
$\tilde{A}_{2}$.

It easily follows from the definition that a surjective morphism of
$\mathbb{C}$-near-rings is automatically strong.

Now we consider modules of a $\mathbb{C}$-near ring.

######  Definition 2.25.

For a $\mathbb{C}$-near ring $\tilde{A}$ with identity, an $\tilde{A}$-module
is a $\mathbb{C}$-vector space $V$ together with a strong $\mathbb{C}$-near-
ring morphism $\tilde{A}\to\mathrm{Map}(V,V)$.

For two $\tilde{A}$-modules $V,W$, a morphism from $V$ to $W$ is a map
$\phi\in\mathrm{Map}(V,W)$ that commutes with the actions of $\tilde{A}$:

$\phi\circ V(\alpha)(v)=W(\alpha)\circ\phi(v)$

for all $\alpha\in\tilde{A}$.

It follows from the above definition that an $\tilde{A}$-module is
automatically an $A$-module (where $A$ denotes the canonical subalgebra).

Essentially, the method of deep learning is performing a (stochastic) gradient
descent on a certain subvariety of the space of $\tilde{A}$-modules for a
fixed near-ring $\tilde{A}$. However, such a space of $\tilde{A}$-modules is
typically infinite-dimensional (since the choice of non-linear maps is
infinite-dimensional). It is important to systematically construct explicit
$\tilde{A}$-modules. A useful construction for
$\tilde{A}=A\\{\varsigma_{1},\ldots,\varsigma_{N}\\}$ is the following. Given
an algebra and an $A$-module $V$, a choice of
$\sigma_{1},\ldots,\sigma_{N}\in\mathrm{Map}(V,V)$ enhances $V$ to be an
$A\\{\varsigma_{1},\ldots,\varsigma_{N}\\}$-module. (Here, $\varsigma_{l}$ are
the formal symbols corresponding to $\sigma_{l}$.)

Unfortunately, such a correspondence between $A$-modules and
$\tilde{A}$-modules does not behave well in the morphism level. Namely, an
$A$-module endomorphism $\phi\in\mathrm{Lin}(V,V)$ typically does not satisfy
$\phi\circ\sigma_{l}=\sigma_{l}\circ\phi$ for non-linear functions
$\sigma_{l}\in\mathrm{Map}(V,V)$, and hence cannot be lifted as an
$\tilde{A}$-module morphism. So we do not have a map from the space of
$A$-modules to the space of $\tilde{A}$-modules that descend to isomorphism
classes.

Below, we use our setting of an activation module to remedy this
correspondence between $A$ and $\tilde{A}$. See Proposition 2.27.

####  2.4.2. Forms over near-ring

Let $\mathcal{A}$ be an algebra, and fix a framing vector space
$F=\mathbb{C}^{n}$. In Section 2.3, we have taken the doubled augmented
algebra $\hat{\mathcal{A}}$. Now, we consider the set
$\mathrm{Mat}_{F}(\hat{\mathcal{A}})$ of $n\times n$ matrices whose $(k,j)$-th
entries lie in
$\mathfrak{e}_{k}^{*}\cdot\hat{\mathcal{A}}\cdot\mathfrak{e}_{j}$.

It is easy to check that:

######  Lemma 2.26.

$A:=\mathrm{Mat}_{F}(\hat{\mathcal{A}})$ forms an algebra under matrix
addition and multiplication (where multiplication between entries is given by
$\hat{\mathcal{A}}$).

This is essentially the algebra $\mathcal{L}(\mathcal{A})$ defined in Equation
(7), adapted to the current setting by identifying
$\mathfrak{e}=(\mathfrak{e}_{j}:j=1,\ldots,n)$. As explained previously right
after Corollary 2.10, each element of $\mathrm{Mat}_{F}(\hat{\mathcal{A}})$
induces a section of the trivial bundle $\mathrm{End}(F)$ over $[R/G]$, where
$R$ is the space of framed representations of $\mathcal{A}$.

Similar to (8), we take the $\mathbb{C}$-near-ring

$\widetilde{\mathcal{A}}:=\mathrm{Mat}_{F}(\hat{\mathcal{A}})\\{\varsigma_{1},\ldots,\varsigma_{N}\\}$

where each $\varsigma_{l}$ represents a non-linear function $\sigma_{l}:F\to
F$.

As in Definition 2.14, we have a natural grading on $\widetilde{\mathcal{A}}$.
Recall that the elements of $\widetilde{\mathcal{A}}$ can be recorded by
rooted trees. The generation of rooted trees gives a grading on
$\widetilde{\mathcal{A}}$:

$\widetilde{\mathcal{A}}=\bigoplus_{k}\widetilde{\mathcal{A}}_{k}.$

$\widetilde{\mathcal{A}}_{0}=\mathrm{Mat}_{F}(\hat{\mathcal{A}})$;
$\widetilde{\mathcal{A}}_{p}$ consists of linear combinations of
$a\cdot\varsigma_{j}\circ\alpha$ for
$a\in\mathrm{Mat}_{F}(\hat{\mathcal{A}})$,
$\alpha\in\widetilde{\mathcal{A}}_{p-1}$, and $j=1,\ldots,N$.

In the last subsection, we have explained a correspondence between $A$-modules
and $A\\{\varsigma_{1},\ldots,\varsigma_{N}\\}$-modules, by choosing maps
$\sigma_{1},\ldots,\sigma_{N}\in\textrm{Map}(V,V)$. However, such a
correspondence does not descend to isomorphism classes. The advantage of the
construction here (after fixing a framing vector space $F$) is that the
correspondence is well-defined on the moduli space.

######  Proposition 2.27.

Fix $\sigma^{F}_{l}\in\mathrm{Map}(F,F)$ for $l=1,\ldots,N$. A framed
$\mathcal{A}$-module $(V,F)$ with a Hermitian metric $h$ on $V$ induces an
$\widetilde{\mathcal{A}}$-module structure on $F$. Moreover, if two such
modules with metrics are isomorphic
$(V,F,h)\cong(V^{\prime},F^{\prime},h^{\prime})$, then the induced
$\widetilde{\mathcal{A}}$-module structures on $F$ are the same. Thus, fixing
an equivariant family of metrics on $V$, we have the map

$[R(A)/G]\to R_{F}\left(\widetilde{\mathcal{A}}\right)$

where $R_{F}(\widetilde{\mathcal{A}})$ denotes the space of
$\widetilde{\mathcal{A}}$-module structures on $F$.

###### Proof.

As explained below Corollary 2.10, by using the framed $\mathcal{A}$-module
structure and metric, each element in $\mathrm{Mat}_{F}(\hat{\mathcal{A}})$
induces a linear endomorphism of $F$, which is invariant under
$\mathrm{GL}(V)$. Thus two isomorphic framed modules with metrics produce the
same linear endomorphism of $F$. Moreover, $\sigma^{F}_{l}$ are maps on $F$
which receive no action by $\mathrm{GL}(V)$. As a result, this gives an
$\tilde{A}$-module structure on $F$ which remains the same for isomorphic
$(V,F,h)$. ∎

The above proposition explains why we want Definition 2.4 for an activation
module.

######  Remark 2.28.

In Definition 2.4, we have a splitting $F=F_{\mathrm{m}}\oplus
F_{\mathrm{in}}\oplus F_{\mathrm{out}}$. It is easy to restrict to the
component $F_{\mathrm{m}}$ (or other components). We have the projection
$p:F\to F_{\mathrm{m}}$ and inclusion $\iota:F_{\mathrm{m}}\to F$. The
functions $\sigma_{j}^{F}:F_{\mathrm{m}}\to F_{\mathrm{m}}$ can also be
understood as functions on $F$. From now on, we will simply work with the
whole framing vector space $F$, keeping in mind that we can restrict to the
components if we want.

We are going to define differential forms on $\widetilde{\mathcal{A}}$. Under
the setting of Definition 2.4, they will induce
$\mathrm{Map}\left(F,F\right)$-valued forms on $[R/G]$ (Theorem 2.40).

First, recall that we have the Karoubi-de Rham complex
$DR^{\bullet}(\hat{\mathcal{A}}/B)$. It contains the subspace of forms over
loops at the framing vertex. These forms are linear combinations of elements
$\mathfrak{e}_{k}^{*}\ldots\mathfrak{e}_{j},(d\mathfrak{e}_{k}^{*})\ldots\mathfrak{e}_{j},\mathfrak{e}_{k}^{*}\ldots(d\mathfrak{e}_{j}),(d\mathfrak{e}_{k}^{*})\ldots(d\mathfrak{e}_{j})$
for some $j,k=1,\ldots,n$. In other words, the subspace is
$\sum_{j,k=1}^{n}DR^{\bullet}(\hat{\mathcal{A}}/B)_{j,k}$, where
$DR^{\bullet}(\hat{\mathcal{A}}/B)_{j,k}$ is defined as

$\mathfrak{e}_{k}^{*}\cdot
DR^{\bullet}(\hat{\mathcal{A}}/B)\cdot\mathfrak{e}_{j}+d\mathfrak{e}_{k}^{*}\cdot
DR^{\bullet}(\hat{\mathcal{A}}/B)\cdot\mathfrak{e}_{j}+\mathfrak{e}_{k}^{*}\cdot
DR^{\bullet}(\hat{\mathcal{A}}/B)\cdot
d\mathfrak{e}_{j}+d\mathfrak{e}_{k}^{*}\cdot
DR^{\bullet}(\hat{\mathcal{A}}/B)\cdot d\mathfrak{e}_{j}.$

We define the linear part as follows.

######  Definition 2.29.

$DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$ is defined to be the space
of $n\times n$ matrices whose $(k,j)$-th entries lie in
$DR^{\bullet}(\hat{\mathcal{A}}/B)_{j,k}$.

Like $DR^{\bullet}(\hat{\mathcal{A}}/B)$, this space is graded by the degree
of forms.

From Proposition 2.8, we have the map

(14)
$DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))\to(\Omega^{\bullet}(R,\mathrm{End}\left(F\right)))^{G}.$

($F$, and hence $\mathrm{End}(F)$, are treated as a trivial bundle over
$[R/G]$.)

To define differential forms on $\widetilde{\mathcal{A}}$, we need to use the
symbols $\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(a_{1},\ldots,a_{p})$,
which represent the $p$-th order _symmetric_ differentials of the non-linear
functions $\sigma_{l}$. For instance, $D^{(1)}\varsigma_{l}$ represents the
usual differential $d\sigma_{l}$; $D^{(2)}\varsigma_{l}$ represents the
Hessian of $\sigma_{l}$, which is a symmetric bilinear two-form.
$D^{(p)}\varsigma_{l}$ is supersymmetric about its $p$ inputs:

(15)
$\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(a_{1},\ldots,a_{k},a_{k+1},\ldots,a_{p})=(-1)^{\deg
a_{k}\cdot\deg
a_{k+1}}\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(a_{1},\ldots,a_{k+1},a_{k},\ldots,a_{p})$

where $\deg a$ denotes the degree of $a$. The inputs $a_{i}$ are again
differential forms on $\widetilde{\mathcal{A}}$. The point of evaluation
$\alpha$ is an element of $\widetilde{\mathcal{A}}$.

######  Definition 2.30.

A form-valued tree is a rooted tree (oriented towards the root) whose edges
are labeled by $\phi\in DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$;
leaves are labeled by $\alpha\in\widetilde{\mathcal{A}}$; the root (if not
being a leaf) is labeled by $1$; nodes which are neither leaves nor the root
are labeled by $\left.D^{(p)}\varsigma_{l}\right|_{\alpha}$ for some
$l=1,\ldots,N$, $\alpha\in\widetilde{\mathcal{A}}$, and $p>0$ is the number of
incoming edges.

The trivial rooted tree, which has a single node with no edge, corresponds to
zero-form. The node is attached with an element
$\alpha\in\widetilde{\mathcal{A}}$.

For a non-trivial rooted tree, the output of each node which are neither
leaves nor the root is

$\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(\phi_{1}\cdot\eta_{1},\ldots,\phi_{p}\cdot\eta_{p})$

where $\phi_{k}\in DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$ are
attached to the incoming edges, and $\eta_{k}$ are the outputs of the nodes
adjacent to the incoming edges. The input edges to the node are read
clockwisely. Its degree is defined as the sum of
$\deg(\phi_{k}\cdot\eta_{k})=\deg\phi_{k}+\deg\eta_{k}$. The output of each
leaf is simply its label $\alpha\in\widetilde{\mathcal{A}}$ which has degree
0. The output of the root, which is the sum of $\phi_{k}\cdot\eta_{k}$ for the
incoming edges $\phi_{k}$ and outputs of incoming nodes $\eta_{k}$, is taken
to be the differential form associated to the form-valued tree.

######  Remark 2.31.

Now we have introduced two different kinds of rooted trees. The activation
tree represents an element in $\widetilde{\mathcal{A}}$ (which is identified
as a zero-form); the form-valued tree represents a $p$-form. For $p=0$, the
form-valued tree is trivial consisting of a single root, which is labeled by
$\alpha\in\widetilde{\mathcal{A}}$. $\alpha$ is represented by an activation
tree, which is more useful in this situation.

######  Definition 2.32.

A differential zero-form over $\widetilde{\mathcal{A}}$ is simply an element
in $\widetilde{\mathcal{A}}$. Denote

$DR^{0}(\widetilde{\mathcal{A}}):=\widetilde{\mathcal{A}}.$

A differential $p$-form (for $p\geq 1$) is a sum of forms associated to form-
valued trees with at most $p$ leaves, with total of degrees of forms attached
to edges being $p$. The space of $p$-forms is denoted by
$DR^{p}(\widetilde{\mathcal{A}})$.

######  Remark 2.33.

Since we require the trees contributing to a $p$-form to have at most $p$
leaves, $D^{(k)}\varsigma_{l}$ that appear at the nodes must have $k\leq p$.

######  Example 2.34.

Figure 4 shows examples of one-form and two-form. The corresponding
expressions are $a_{1}da_{2}\cdot
D^{(1)}\left.\varsigma_{l}\right|_{\alpha_{1}}(a_{3}\cdot\alpha_{2})$,
$a_{1}da_{2}\cdot
D^{(1)}\left.\varsigma_{l}\right|_{\alpha_{1}}((a_{3}da_{4})\cdot\alpha_{2})$
and $a_{1}da_{2}\cdot
D^{(2)}\left.\varsigma_{l}\right|_{\alpha_{1}}(a_{3}\cdot\alpha_{2},(a_{4}da_{5})\cdot\alpha_{3})$
respectively.

Figure 4.

######  Definition 2.35.

The differential of a form over $\widetilde{\mathcal{A}}$ is defined as
follows.

A zero-form in the $0$-th graded piece $\alpha\in\widetilde{\mathcal{A}}_{0}$
is simply an element in $\mathrm{Mat}_{F}(\hat{\mathcal{A}})$, and its
differential is given by the entriwise differential in
$DR^{\bullet}(\hat{\mathcal{A}}/B)$. A zero-form in the $p$-th graded piece
$\alpha\in\widetilde{\mathcal{A}}_{p}$ can be written as

$\alpha=a_{0}+\sum_{k=1}^{m}a_{k}\circ\varsigma_{l(k)}\circ\alpha_{k}\in
DR^{0}(\widetilde{\mathcal{A}})$

where $a_{k}\in\mathrm{Mat}_{F}(\hat{\mathcal{A}})$ for $k=0,\ldots,m$,
$\alpha_{k}\in\widetilde{\mathcal{A}}_{p-1}$, and $l(k)=1,\ldots,N$. Then

$d\alpha:=da_{0}+\sum_{k}da_{k}\cdot(\varsigma_{l(k)}\circ\alpha_{k})+\sum_{k}a_{k}\cdot\left.D^{(1)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k})\in
DR^{1}(\widetilde{\mathcal{A}})$

where $d\alpha_{k}$ has already been defined by the inductive assumption since
$\alpha_{k}\in\widetilde{\mathcal{A}}_{p-1}$.

For $p$-forms with $p>0$, it suffices to define differential of a $p$-form
attached to a form-valued tree. For a leaf, the output is simply its label
$\alpha\in\widetilde{\mathcal{A}}$, whose differential has been defined above.
For a node which is neither a leaf nor the root, its output is of the form
$\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(\phi_{1}\cdot\eta_{1},\ldots,\phi_{p}\cdot\eta_{p})$,
where $\phi_{k}\in DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$ are
attached to the incoming edges, and $\eta_{k}$ are the outputs of the nodes
adjacent to the incoming edges. Its differential is defined as

$\displaystyle
d\left(\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(\phi_{1}\cdot\eta_{1},\ldots,\phi_{p}\cdot\eta_{p})\right)$
$\displaystyle:=$
$\displaystyle\left.D^{(p+1)}\varsigma_{l}\right|_{\alpha}(d\alpha,\phi_{1}\cdot\eta_{1},\ldots,\phi_{p}\cdot\eta_{p})$
$\displaystyle+\sum_{k=1}^{p}(-1)^{\deg(\phi_{1}\eta_{1})+\ldots+\deg(\phi_{k-1}\eta_{k-1})}\left.D^{(p)}\varsigma_{l}\right|_{\alpha}(\phi_{1}\cdot\eta_{1},\ldots,(d\phi_{k})\cdot\eta_{k}$
$\displaystyle+(-1)^{\deg\phi_{k}}\phi_{k}\cdot
d\eta_{k},\ldots,\phi_{p}\cdot\eta_{p})$

where the differential $d\eta_{k}$ is already known by induction assumption on
the generation of the tree. The $p$-form attached to the tree is the output of
the root, which is of the form $\sum_{k}\phi_{k}\cdot\eta_{k}$. Its
differential is defined as
$\sum_{k}\left(d\phi_{k}\cdot\eta_{k}+(-1)^{\deg\phi_{k}}\phi_{k}\cdot
d\eta_{k}\right)$, where $d\eta_{k}$ has been defined by inductive assumption.

The differential of a zero-form has a nice expression in terms of a sum over
sub-trees of the activation tree as follows.

######  Proposition 2.36.

Consider $\alpha\in\widetilde{\mathcal{A}}$ represented by an activation tree
$T$. Then $d\alpha\in DR^{1}(\widetilde{\mathcal{A}})$ is a sum over all the
nodes of $T$, and the terms are given as follows. For each node, there is a
unique path $\gamma_{1}\ldots\gamma_{r}$ in $T$ connecting from that node to
the root, where $\gamma_{k}$ denotes the (oriented) edges. (When the node is
the root, the path is trivial and the corresponding term is simply $0$.) The
corresponding term equals to

(16)
$a_{\gamma_{1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{1}))}\right|_{\alpha_{t(\gamma_{1})}}\ldots
a_{\gamma_{r-1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{r-1}))}\right|_{\alpha_{t(\gamma_{r-1})}}\,da_{\gamma_{r}}\cdot(\varsigma_{l(t(\gamma_{r}))}\circ\alpha_{t(\gamma_{r})})$

where $\alpha_{i}$ for a node $i$ of $T$ denotes the output at the node $i$.

###### Proof.

The statement easily holds for the zeroth generation: the tree only has the
root and leaves as nodes, and the zeroth form has an expression
$\sum_{i}a_{i}$ for $a_{i}\in
DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$, whose differential is
simply $\sum_{i}da_{i}$, which is a sum over the leaves.

Suppose the statement holds for all elements in the $p$-th generation. For
$\alpha=a_{0}+\sum_{k=1}^{m}a_{k}\circ\varsigma_{l(k)}\circ\alpha_{k}$ in the
$(p+1)$-th generation,
$d\alpha=da_{0}+\sum_{k}da_{k}\cdot(\varsigma_{l(k)}\circ\alpha_{k})+\sum_{k}a_{k}\cdot\left.D^{(1)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k})\in
DR^{1}(\widetilde{\mathcal{A}})$, where $d\alpha_{k}$ is a sum over the nodes
of the activation tree of $\alpha_{k}$ as given in Equation (16). The first
term $da_{0}$ and second term $da_{k}\cdot(\varsigma_{l(k)}\circ\alpha_{k})$
correspond to the tail nodes of the edges of $a_{k}$ for $k=0,\ldots,m$. Thus
$d\alpha$ is a sum over all the nodes with the summands given by (16). ∎

######  Example 2.37.

Consider the $0$-form

$\alpha=a_{0}+a_{1}\varsigma_{1}\circ(a_{1,0}+a_{1,1}\varsigma_{1,1}\circ
a_{1,1,0}).$

Its differential equals to

$\displaystyle d\alpha=$ $\displaystyle da_{0}+da_{1}\cdot\alpha_{1}$
$\displaystyle+a_{1}\left.D^{(1)}\varsigma_{1}\right|_{a_{1,0}+a_{1,1}\varsigma_{1,1}\circ
a_{1,1,0}}(da_{1,0}+da_{1,1}\cdot\alpha_{1,1}+a_{1,1}\left.D^{(1)}\varsigma_{1,1}\right|_{a_{1,1,0}}da_{1,1,0})$

where $\alpha_{1}=\varsigma_{1}\circ(a_{1,0}+a_{1,1}\varsigma_{1,1}\circ
a_{1,1,0})$ and $\alpha_{1,1}=\varsigma_{1,1}\circ a_{1,1,0}$. It equals to
the sum over the nodes of the activation tree of $\alpha$ as shown in Figure
5.

Figure 5.

######  Remark 2.38.

The output at a node of an activation tree representing
$\alpha\in\widetilde{\mathcal{A}}$ can be computed by the algorithm called
_forward propagation_. Namely, the previous results $\alpha_{k}$ (pre-
activation values) have been stored in memory, and the current output is
computed as $\sum_{k}a_{k}\cdot\varsigma_{l(k)}\circ\alpha_{k}$ (where
$\varsigma_{l(k)}$ are the activation functions at previous nodes and $a_{k}$
are labeling the incoming edges) and stored to memory for later steps.

For the differential $d\alpha$, the computation (16) uses the stored outputs
$\alpha_{i}$ in the forward propagation. Moreover, the expression
$a_{\gamma_{1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{1}))}\right|_{\alpha_{t(\gamma_{1})}}\ldots
a_{\gamma_{r-1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{r-1}))}\right|_{\alpha_{t(\gamma_{r-1})}}$
appears in every term of $d\alpha$ corresponding to a path in $T$ that
contains $\gamma_{r-1}\ldots\gamma_{1}$. Thus it is good to start with the
root to compute and store the values of
$a_{\gamma_{1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{1}))}\right|_{\alpha_{t(\gamma_{1})}}\ldots$
$a_{\gamma_{r-1}}\left.D^{(1)}\varsigma_{l(t(\gamma_{r-1}))}\right|_{\alpha_{t(\gamma_{r-1})}}$,
and move backward with respect to the orientation of the tree $T$. This is
well known as the _backward propagation algorithm_.

######  Proposition 2.39.

$d^{2}=0$.

###### Proof.

First consider a zero-form, that is, $\alpha\in\widetilde{\mathcal{A}}$.
$\alpha$ is represented by an activation tree. Recall that $d^{2}a=0$ for
$a\in A=\mathrm{Mat}_{F}(\hat{\mathcal{A}})$ is already known for differential
of forms on an algebra $A$.

We can write

$\alpha=a_{0}+\sum_{k=1}^{m}a_{k}\circ\varsigma_{l(k)}\circ\alpha_{k}\in
DR^{0}(\widetilde{\mathcal{A}})$

where $a_{k}\in\mathrm{Mat}_{F}(\hat{\mathcal{A}})$ for $k=0,\ldots,m$,
$\alpha_{k}\in\widetilde{\mathcal{A}}$ has one less generation than $\alpha$,
and $l(k)=1,\ldots,N$. Then

$\displaystyle d^{2}\alpha=$ $\displaystyle
d\left(da_{0}+da_{k}\cdot(\varsigma_{l(k)}\circ\alpha_{k})+a_{k}\cdot\left.D^{(1)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k})\right)$
$\displaystyle=$ $\displaystyle-da_{k}\cdot
d(\varsigma_{l(k)}\circ\alpha_{k})+da_{k}\cdot\left.D^{(1)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k})+a_{k}\cdot\left.D^{(2)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k},d\alpha_{k}).$

The first two terms cancel since
$d(\varsigma_{l(k)}\circ\alpha_{k})=\left.D^{(1)}\varsigma_{l(k)}\right|_{\alpha_{k}}(d\alpha_{k})$.
The third term vanishes since $D^{(2)}\varsigma_{l(k)}$ is supersymmetric
about its input (Equation (15)).

For a general $p$-form, it suffices to prove $d\psi=0$ for $\psi$ represented
by a form-valued tree. We will do induction on the generation of the tree. We
already know the statement when the tree is trivial (which is the case of a
zero-form). The $p$-form $\psi$ is given as
$\psi=\sum_{k}\phi_{k}\cdot\eta_{k}$ for some $\phi_{k}\in
DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$ and $\eta_{k}$ has a
smaller generation than $\psi$. Then

$d^{2}\psi=\sum_{k}\left((-1)^{\deg\phi_{k}}d\phi_{k}\cdot
d\eta_{k}+(-1)^{\deg\phi_{k}+1}d\phi_{k}\cdot
d\eta_{k}+(-1)^{2\deg\phi_{k}}\phi_{k}\cdot d^{2}\eta_{k}\right).$

The first two terms cancel. The last term vanishes by inductive assumption. ∎

Finally, we show that differential forms on the near-ring
$\widetilde{\mathcal{A}}$ induce $G$-invariant
$\mathrm{Map}\left(F,F\right)$-valued differential forms over the space of
framed $\mathcal{A}$-modules $R$.

######  Theorem 2.40.

There exists a degree-preserving map

$DR^{\bullet}(\widetilde{\mathcal{A}})\to(\Omega^{\bullet}(R,\mathbf{Map}\left(F,F\right)))^{G}$

which commutes with $d$ on the two sides, and equals to the map (14):
$DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))\to(\Omega^{\bullet}(R,\mathrm{End}\left(F\right)))^{G}$
when restricted to $DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$. Here,
$\mathbf{Map}\left(F,F\right)$ denotes the trivial bundle
$\mathrm{Map}\left(F,F\right)\times R$, and the action of $G=\mathrm{GL}(V)$
on fiber direction is trivial.

###### Proof.

First consider the case of a zero-form. We associate $\alpha\in
DR^{0}(\widetilde{\mathcal{A}})$ to a $G$-invariant $\mathrm{Map}(F,F)$-valued
function over $R$ inductively on its generation as an element in
$\widetilde{\mathcal{A}}$. In the zeroth generation, it is just an element in
$\mathrm{Mat}_{F}(\hat{\mathcal{A}})$, which induces a matrix whose entries
lie in $\Omega^{0}(R)^{G}$ by Proposition 2.8. This gives a self-map $F\to F$
over $[R/G]$. If $\alpha$ is in the $p$-th generation, then it is written as
$\alpha=a_{0}+\sum_{k=1}^{m}a_{k}\circ\varsigma_{l(k)}\circ\alpha_{k}\in
DR^{0}(\widetilde{\mathcal{A}})$, where $\alpha_{k}$ is in the $(p-1)$-th
generation and induces a self-map $F\to F$ over $[R/G]$. By composing with the
corresponding functions $\sigma_{l(k)}:F\to F$ and the induced functions of
$a_{k}\in\mathrm{Mat}_{F}(\hat{\mathcal{A}})$, we obtain a self-map $F\to F$
over $[R/G]$ corresponding to $\alpha$.

For a $k$-form $\psi\in DR^{\bullet}(\widetilde{\mathcal{A}})$, we do an
induction on the generation of its corresponding form-valued tree to associate
it with a $G$-invariant $\mathrm{Map}\left(F,F\right)$-valued $k$-form over
$R$. In th zeroth generation it must be a zero-form (where the associated
form-valued tree is simply a single node), which is done by the previous
paragraph. In general $\psi=\sum_{k}\phi_{k}\cdot\eta_{k}$ for some
$\phi_{k}\in DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$ and $\eta_{k}$
has a smaller generation than $\psi$. Both $\phi_{k}$ and $\eta_{k}$ have been
associated with $G$-invariant $\mathrm{Map}\left(F,F\right)$-valued $k$-forms.
Then their matrix products (and by wedge product entriwise) give the required
$k$-form associated to $\psi$.

It follows from the chain rule that the differential for
$DR^{\bullet}(\widetilde{\mathcal{A}})$ given in Definition 2.35 agrees with
that for $\mathrm{Map}\left(F,F\right)$-valued forms over $R$. Moreover, for
$\phi\in DR^{\bullet}(\mathrm{Mat}_{F}(\hat{\mathcal{A}}))$, it is in the
first generation written as $\phi\cdot 1$. By the above definition, the
association is given by the map (14). ∎

So far, this gives matrix-valued differential forms on $[R/G]$. To produce
$\mathbb{C}$-valued forms, that is, to remove the component
$\mathrm{Map}\left(F,F\right)$ in the above theorem, we proceed as follows.
The near-ring $\widetilde{\mathcal{A}}$ can be augmented with the inclusion
and projection symbols $\iota_{i}$ and $p_{j}$, where $\iota_{i}$ represents
the inclusion $\mathbb{C}\to F$ of the $i$-th coordinate axis, and $p_{j}$
represents the projection $F\to\mathbb{C}$ in the $i$-th direction. This forms
an augmented near-ring

$\bigoplus_{k=1}^{\infty}\left(\\{p_{1},\ldots,p_{n}\\}\circ\widetilde{\mathcal{A}}\circ\left(\mathbb{C}\cdot\\{\iota_{1},\ldots,\iota_{n}\\}\right)\right)^{k}$

consisting of linear combinations of elements
$\left(p_{i}\circ\alpha\circ\left(\sum_{j}x_{j}\iota_{j}\right)\right)^{k}$
for $\alpha\in\widetilde{\mathcal{A}}$, with the relations
$p_{i}\circ\iota_{j}=\delta_{ij}\cdot 1$ and $\iota_{j}\circ
1_{\widetilde{\mathcal{A}}}\circ p_{i}=\delta_{ij}\cdot 1$. Then differential
forms in this augmented near-ring induces $G$-invariant differential forms in
$(\Omega^{\bullet}(R))^{G}$. The proof is similar and we shall not repeat.

In application, we fix an algorithm $\tilde{\gamma}\in\widetilde{\mathcal{A}}$
and consider

$\varphi^{\tilde{\gamma}}(x)=\left(p_{i}\circ\tilde{\gamma}\circ\left(\sum_{j}x_{j}\iota_{j}\right)\right)_{i=1}^{n}$

for each element $x=(x_{1},\ldots,x_{n})\in F$. $\varphi^{\tilde{\gamma}}(x)$
is a vector whose entries are elements inside the above augmented near-ring.
Given $f:K\to F$, we have

$\int_{K}\left|\varphi^{\tilde{\gamma}}(x)-f\left(x\right)\right|^{2}dx$

which is a 0-form on the augmented near-ring. This 0-form and its differential
induces the cost function and its differential on $[R/G]$ respectively, which
are the central objects in machine learning.

##  3\. Uniformization

In this section, we apply the idea of uniformization of metrics on framed
quiver moduli spaces, which are interpreted as moduli of computing machines
from the previous section.

The uniformization theorem for Riemann surfaces was a big discovery of Klein,
Poincaré and Koebe in the 19th century. It asserts that every simply connected
Riemann surface is conformally equivalent to either the complex plane, the
Riemann sphere, or the hyperbolic disc.

Such a classification also holds for Riemannian symmetric spaces. Namely, any
irreducible simply connected symmetric space is either of Euclidean type,
compact type, and non-compact type, depending on whether its sectional
curvature is identically zero, non-negative, or non-positive.

As a key example, $\textrm{Gr}(n,d)$ is a compact Hermitian symmetric space.
It has a non-compact dual which embeds as an open subset of
$\textrm{Gr}(n,d)$. This is the celebrated Borel embedding, and was uniformly
studied for symmetric R-spaces and generalized Grassmannians in [CHL]. The
non-compact dual to $\textrm{Gr}(n,d)$ is the "space-like Grassmannian" which
can be thought of as a generalization of hyperbolic space.

We generalize this to framed quiver varieties. The key idea is that different
types of quiver varieties will arise by considering space-like representations
with respect to different choices of quadratic forms on the framing. As
explained in the Introduction, our motivation is to find a relation between
our formulation of neural network and the original Euclidean formulation.
Using this construction, we not only get an interpolation between these two
different formulations, but also find a non-compact type quiver varieties
which can also be used in machine learning. Such a family of quiver varieties
of different types is what we refer as uniformization of framed quiver
varieties in the title.

###  3.1. A quick review

Let $Q$ be a directed graph. Denote by $Q_{0},Q_{1}$ the set of vertices and
arrows respectively. A quiver representation $w$ with dimension vector
$d\in\mathbb{Z}_{\geq 0}^{Q_{0}}$ associates each arrow $a$ with a matrix
$w_{a}$ of size $d_{h(a)}\times d_{t(a)}$ (where $h(a),t(a)$ denote the head
and tail vertices of $a$ respectively). The set of complex quiver
representations with dimension $\vec{d}$ form a vector space denoted by
$R_{\vec{d}}(Q)$. $\mathrm{GL}(d):=\prod_{i\in
Q_{0}}\mathrm{GL}(d_{i},\mathbb{C})$ acts on $R_{\vec{d}}(Q)$ via

(17) $g\cdot(w_{a}:a\in Q_{1})=(g_{h(a)}\cdot w_{a}\cdot g_{t(a)}^{-1}:a\in
Q_{1}).$

Let $d,n\in\mathbb{Z}_{\geq 0}^{Q_{0}}$. $n$ will be the dimension vector for
the framing, which is a linear map $e^{(i)}:\mathbb{C}^{n_{i}}\to V_{i}$ at
each $i\in Q_{0}$ (where $V_{i}=\mathbb{C}^{d_{i}}$).

######  Theorem 3.1 ([Nak96]).

The vector space of framed representations is given by

$R_{n,d}=R_{d}\times\bigoplus_{i\in
Q_{0}}\mathrm{Hom}(\mathbb{C}^{n_{i}},\mathbb{C}^{d_{i}}).$

It carries a natural action of $\mathrm{GL}(d)$ given by $g\cdot(w,e)=(g\cdot
w,(ge^{(i)}:i\in Q_{0}))$, where $g\cdot V$ is given by Equation (17).
$(w,e)\in R_{n,d}$ is called stable if there is no proper subrepresentation
$U$ of $w$ which contains $\mathrm{Im}\,e$. The set of all stable points of
$R_{n,d}$ is denoted by $R_{n,d}^{s}$. Then the quotient
$\mathcal{M}_{n,d}:=R_{n,d}^{s}/\mathrm{GL}(d)$ is a smooth variety, which is
called to be a framed quiver moduli.

The topology of $\mathcal{M}_{n,d}$ is well-understood. Let’s make an ordering
of the vertices. Namely the vertices are labeled by $\\{1,\ldots,N\\}$, such
that $i<j$ implies there is no arrow going from $j$ to $i$. Such a labeling
exists if $Q$ has no oriented cycle.

######  Theorem 3.2 (Reineke [Rei08]).

Assume $Q$ has no oriented cycle. Consider the chain of iterated Grassmannian
bundles $M^{(N)}\stackrel{{\scriptstyle
p_{N}}}{{\to}}M^{(N-1)}\stackrel{{\scriptstyle
p_{N-1}}}{{\to}}\ldots\stackrel{{\scriptstyle
p_{2}}}{{\to}}M^{(1)}\stackrel{{\scriptstyle p_{1}}}{{\to}}\mathrm{pt}$ (where
$\mathrm{pt}$ denotes a singleton) defined by induction:

$M^{(i)}=\textrm{Gr}_{M^{(i-1)}}\left(\underline{\mathbb{C}^{n_{i}}}\oplus\bigoplus_{j\to
i}p_{i-1}^{*}\dots p_{j+1}^{*}(S_{j}),d_{i}\right)\to M^{(i-1)},$

where $S_{i}$ denotes the tautological bundle on $M_{i}$ (as a Grassmannian
bundle over $M_{i-1}$). (The direct sum is over each arrow $j\to i$.) Then
$\mathcal{M}_{\vec{n},\vec{d}}\cong M^{(N)}$, with universal bundles
$\mathcal{V}_{i}\cong p_{N}^{*}\dots p_{i+1}^{*}S_{i}$ for all $i\in Q_{0}$.

In the previous paper [JL21] we introduced a Hermitian metric $H_{i}$ for each
of these $\mathcal{V}_{i}$ and showed that its Ricci curvature induces a
Kähler metric on $\mathcal{M}$. Let’s quickly review this construction.

######  Theorem 3.3 ([JL21]).

Let $Q$ be a finite quiver. Let $R_{n,d}$ be the space of framed quiver
representations of $Q$ with representing dimension $d$ and framing dimension
$n$. For any path $\gamma$ in $Q$, let $e^{t(\gamma)}$ be the framing map
associated to the vertex $t(\gamma)$ and let $w_{\gamma}$ be the matrix
representation of $\gamma$.

For a fixed vertex $(i)$, let $\rho_{i}$ be the row vector whose entries are
all the elements of the form
$w_{\gamma}e^{t(\gamma)}:R_{n,d}\to\mathrm{Hom}(\mathbb{C}^{n_{t(\gamma)}},\mathbb{C}^{d_{i}})$
such that $h(\gamma)=i$. Consider

(18)
$\rho_{i}\rho_{i}^{*}=\sum\limits_{h(\gamma)=i}\left(w_{\gamma}e^{t(\gamma)}\right)\left(w_{\gamma}e^{t(\gamma)}\right)^{*}$

as a map $\rho_{i}\rho_{i}^{*}:R_{n,d}\to\text{End}(\mathbb{C}^{d_{i}})$.

Then $(\rho_{i}\rho_{i}^{*})^{-1}$ is $\mathrm{GL}(d)$-equivariant and
descends to a Hermitian metric on $\mathcal{V}_{i}$ over $\mathcal{M}$. We
denote this resulting metric as $H_{i}$.

Suppose $Q$ has no oriented cycle. Then

(19) $H_{T}:=\sum_{i}\partial\bar{\partial}\log\det
H_{i}=\sum_{i}\left(tr(\partial\rho_{i})^{*}H_{i}\partial\rho_{i}-tr\left(H_{i}\rho_{i}(\partial\rho_{i})^{*}H_{i}(\partial\rho_{i})\rho_{i}^{*}\right)\right)$

defines a Kähler metric on $\mathcal{M}$.

###  3.2. Illustration by examples

First, consider the simplest possible example, namely the quiver with a single
vertex.

######  Example 3.4.

Let $Q$ consist of a single vertex $(1)$ with no arrows. Let the representing
dimension and the framing dimension be $d$ and $n$ respectively where $d<n$.
The framed quiver moduli is simply $\textrm{Gr}(n,d)$, the Grassmannian of
surjective linear maps $\mathbb{C}^{n}\to\mathbb{C}^{d}$. Equation 18 becomes
$H=ee^{*}$ on the universal bundle over the dual Grassmannian. If we take the
chart where the first $d$-many components of $e$ form an invertible map, we
can rewrite $e$ as $e=(Id_{d},b)$ due to the $G_{d}$-equivalence. Then $H$
becomes $(Id_{d}+b)^{-1}$, the standard metric on the universal bundle over
$\textrm{Gr}(n,d)$. In particular, for $d=1$, $\textrm{Gr}(n,1)$ is the
projective space $\mathbb{P}^{n-1}$, and the Ricci curvature of $H$ is the
Fubini-Study metric.

${(3)}$${(1)}$${(4)}$${(2)}$$\scriptstyle{a_{4}}$$\scriptstyle{a_{1}}$$\scriptstyle{a_{2}}$$\scriptstyle{a_{3}}$
Figure 6.

Now consider a slightly more involved example. Let $Q$ be the quiver depicted
in Figure 6. Thus the vertex set is $\\{(1),(2),(3),(4)\\}$ and the arrow set
is $\\{a_{1}:(1)\to(2),a_{2}:(1)\to(3),a_{3}:(2)\to(4),a_{4}:(3)\to(4)\\}$. We
define $\mathcal{A}$ to be the path algebra of $Q$ over $\mathbb{C}$. That is,
an element of $\mathcal{A}$ is a formal sum over $\mathbb{C}$ generated by the
paths on the underlying directed graph of $Q$. In addition, we associate to
$Q$ a vector space $V=\mathbb{C}^{d}$ and a framing space $F=\mathbb{C}^{n}$
with some kind of decomposition as $F=F_{\mathrm{in}}\oplus F_{2}\oplus
F_{3}\oplus F_{\mathrm{out}}$. In this setting, we take $(1)$ as our input
vertex, $(4)$ as our output vertex, and vertices $(2)$ and $(3)$ as "middle"
or "memory" vertices, so we think of $F_{in}$ as being associated to $(1)$,
$F_{\mathrm{out}}$ to $(4)$, and $F_{m}\cong F_{2}\oplus F_{3}$ with $(2)$ and
$(3)$ respectively. In addition, we want a decomposition of $V$ as
$V=V_{1}\oplus V_{2}\oplus V_{3}\oplus V_{4}$ with $V_{i}$ associated to
vertex $(i)$ for each $i$. We endow this space with a $GL(V)$-equivariant
family of Hermitian metrics $\bigoplus_{i}H_{i}$: (Theorem 3.3):

$\rho_{i}\rho_{i}^{*}=\sum\limits_{h(\gamma)=i}\left(w_{\gamma}e^{t(\gamma)}\right)\left(w_{\gamma}e^{t(\gamma)}\right)^{*}.$

Take activation functions $\sigma_{j}^{F}:F_{m}\to F_{m}$. We have an
activation module (Definition 2.4). The framing maps are of the form
$(e_{\mathrm{in}},e_{2},e_{3},e_{\mathrm{out}})$ where
$(e_{\mathrm{in}})_{j}:F_{\mathrm{in}}\to V_{1}$, $(e_{2})_{j}:F_{2}\to
V_{2}$, $(e_{3})_{j}:F_{3}\to V_{3}$, and
$(e_{\mathrm{out}})_{j}:F_{\mathrm{out}}\to V_{4}$. We can be extend by zero
and encode all of them as $e:F\to V$. Furthermore, we can require the maps
$\sigma_{j}^{F}$ to decompose as
$(\sigma_{j}^{F})_{2}\oplus(\sigma_{j}^{F})_{3}$ where $(\sigma_{j}^{F})_{i}$
is a map from $F_{i}\to F_{i}$.

Now, we want to choose an algorithm, that is, an element
$\tilde{\gamma}\in\tilde{\mathcal{A}}$ that starts at $F_{\mathrm{in}}$ and
ends at $F_{\mathrm{out}}$. This takes the form

$\tilde{\gamma}=e_{\mathrm{out}}^{*}(a_{4}e_{m}\sigma_{2}e_{m}^{*}a_{2}e_{\mathrm{in}}+a_{3}e_{m}\sigma_{3}e_{m}^{*}a_{1})e_{\mathrm{in}}.$

Figure 7.

The activation tree for this algorithm is shown in Figure 7 on the left. The
rest of Figure 7 is the form-valued tree derived from the activation tree as
in Figure 5. This activation tree gives the forward propagation used in deep
learning for actual computation. Just as vital, the back propagation attained
from the form-valued tree is used in order to train the network.

###  3.3. The non-compact dual of framed quiver moduli

Assume that $n_{i}\geq d_{i}$ $\forall i$. We write the framing map as
$e^{(i)}=(\epsilon_{i}$ $b_{i})$ where $\epsilon_{i}$ and $b_{i}$ are
respectively the "basis part" and "bias part of our framing map $e^{(i)}$.
Then Equation 18 can be modified to be:

(20) $H_{i}^{\alpha}=\left(\epsilon_{i}\epsilon_{i}^{*}+\alpha
b_{i}b_{i}^{*}+\sum_{\gamma:h(\gamma)=i,\gamma\neq\emptyset}\alpha_{\gamma}w_{\gamma}e^{t(\gamma)}\left(w_{\gamma}e^{t(\gamma)}\right)^{*}\right)^{-1}.$

It is this generalization of the metric which we use for the uniformization.
By varying $\alpha$ and $\alpha_{\gamma}$, we get different quadratic forms.
For example, in Equation 18, $\alpha$ and all $\alpha_{\gamma}$ are simply
$1$. The zero curvature case will elaborated on later in Section 3.4.

######  Remark 3.5.

The application of hyperbolic geometry has mostly focused on fiber direction
in existing literature, namely the representation spaces (and their
corresponding universal bundles over the moduli). Here, we are concerned about
metrics on the moduli space (playing the role of the weight space). It is
general for all quiver moduli, not just restricted to specific models. Thus,
in this moduli approach, the method of varying metrics (with positive, zero or
negative curvatures) can be applied to any model of machine learning.

For now we will set the $\alpha$ and $\alpha_{\gamma}$ to -1 to consider the
negative curvature case. Namely,

(21)
$H_{i}^{-}:=\left(\epsilon_{i}\epsilon_{i}^{*}-b_{i}b_{i}^{*}-\sum_{\gamma:h(\gamma)=i,\gamma\neq\emptyset}w_{\gamma}e^{t(\gamma)}\left(w_{\gamma}e^{t(\gamma)}\right)^{*}\right)^{-1}.$

It must be emphasized that this quadratic form is not positive-definite on
$\mathcal{V}_{i}$ and thus cannot serve as a metric.

The brilliant idea here is that we restrict to the subset of the moduli space
where this quadratic form is positive-definite and thus gives a metric. This
restriction gives the non-compact dual of the framed quiver moduli. As before,
let’s consider the $A_{1}$-quiver and what this metric looks like on that
quiver in particular.

######  Example 3.6.

Let $Q$ consist of a single vertex with no arrows. Let the framing dimension
of the representation space be $n$, and suppose the representing dimension at
the single vertex be 1. Equation 21 becomes

$H^{-}=(|\epsilon|^{2}-|b|^{2})^{-1}.$

Since we restrict to the subset where $H^{-}$ is positive-definite, $\epsilon$
needs to be nonzero. By applying the quiver automorphism, $\epsilon$ can be
rescaled to be 1. Thus $H^{-}=(1-|b|^{2})^{-1}$, and $|b|^{2}<1$. This gives
the hyperbolic moduli, which is the open unit ball in $\mathbb{C}^{n-1}$. The
Ricci curvature of $H^{-}$ gives the Poincaré metric.

Thus from Examples 3.4 and 3.6 we can see the motivating duality mentioned at
the start of the section.

######  Definition 3.7.

Assume $Q$ has no oriented cycle. Let $\rho_{i}$ be as in Definition 3.3 so
that $\rho_{i}$ is a row vector with entries of the form
$w_{\gamma}e^{t(\gamma)}$ where $\gamma$ is some path in $Q$ ending at vertex
$(i)$, $w_{\gamma}$ is the representing matrix of this path, and
$e^{t(\gamma)}$ is the framing map at $t(\gamma)$, the starting vertex of
$\gamma$. Arrange the entries of $\rho_{i}$ so that the first $n_{i}$-many
entries correspond to the framing arrows at vertex $(i)$. Then let $H_{i}^{-}$
be the quadratic form defined by:

(22) $H_{i}^{-}=\left(\rho_{i}\left(\begin{matrix}I_{d_{i}}&0\\\
0&-I_{N_{i}-d_{i}}\end{matrix}\right)\rho_{i}^{*}\right)^{-1}$

Here, $N_{i}=\sum\limits_{(j)\rightsquigarrow(i)}n_{j}$. We define
$R_{n,d}^{-}$ to be the subset of $R_{n,d}$ where $H_{i}^{-}$ is positive-
definite for all $i$.

In above, the notation $(j)\rightsquigarrow(i)$ means all paths starting from
$(j)$ and ending at $(i)$, not just single arrows. In particular, $n_{j}$ gets
counted once for each distinct path from $(j)$ to $(i)$. Note that $N_{i}\geq
d_{i}$ $\forall i$ when $\mathcal{M}\neq\emptyset$, which we always assume to
be the case.

######  Proposition 3.8.

$R_{n,d}^{-}\subset R_{n,d}^{s}.$

###### Proof.

Consider a point in $R_{n,d}^{-}$. Write $\rho_{i}=(\epsilon_{i}\,\,R)$
evaluated at this point as a $(d_{i}\times N_{i})$-matrix, where
$\epsilon_{i}$ is a $(d_{i}\times d_{i})$-matrix and $R$ is the remaining
part. Then $H_{i}^{-}=(\epsilon_{i}\epsilon_{i}^{*}-RR^{*})^{-1}$. We claim
that $\epsilon_{i}$ must be invertible, and hence $\rho_{i}$ is surjective.
This is true for all $i$, and hence the point is stable.

Suppose $\epsilon_{i}$ is not invertible. Then there exists $v$ such that
$\epsilon_{i}^{*}\cdot v=0$. Then $v^{*}H_{i}^{-}v=-v^{*}RR^{*}v\leq 0$,
contradicting that $H_{i}^{-}$ evaluated at each point in $R_{n,d}^{-}$ is
positive-definite. ∎

######  Lemma 3.9.

$H_{i}^{-}$ is $G_{d}$-equivariant and $R_{n,d}^{-}$ is $G_{d}$-invariant.

###### Proof.

$H_{i}^{-}$ is $GL(d)$-equivariant because

$\left((g\cdot w_{\gamma}e^{t(\gamma)})(g\cdot
w_{\gamma}e^{t(\gamma)})^{*}\right)^{-1}=g^{-1}\left((w_{\gamma}e^{t(\gamma)})(w_{\gamma}e^{t(\gamma)})^{*}\right)^{-1}(g^{*})^{-1}.$

The reason that $R_{n,d}^{-}$ is $GL(d)$-invariant is because if
$x^{*}\left((w_{\gamma}e^{t(\gamma)})(w_{\gamma}e^{t(\gamma)})^{*}\right)^{-1}x>0$,
then

$(g\cdot x)^{*}\left((g\cdot w_{\gamma}e^{t(\gamma)})(g\cdot
w_{\gamma}e^{t(\gamma)})^{*}\right)^{-1}(g\cdot
x)=x^{*}\left((w_{\gamma}e^{t(\gamma)})(w_{\gamma}e^{t(\gamma)})^{*}\right)^{-1}x>0$

by the $GL(d)$-equivariance of $H_{i}^{-}$. Thus, action by $GL(d)$ sends
$R_{n,d}^{-}$ to itself.

∎

######  Definition 3.10.

If $n_{i}\geq d_{i}$, then $e^{(i)}$ can be written as
$e^{(i)}=(\epsilon_{i},b_{i})$ where $\epsilon_{i}$ is the $d_{i}$-many
components of $e^{(i)}$ and $b_{i}$ is the remaining $(n_{i}-d_{i})$-many
components. We call $\epsilon_{i}$ to be the basis part of $e^{(i)}$ and
$b_{i}$ to be the bias part of $e^{(i)}$.

We call $\epsilon_{i}$ the basis part because we think of it as imposing a
basis on $V_{i}$.

From now on, we will assume $n_{i}\geq d_{i}$, which is the case in
applications. This assumption also ensures that the choices of negative signs
in defining $H_{i}^{-}$ for different vertices $i$ are compatible, so that
$R_{n,d}^{-}\not=\emptyset$.

######  Proposition 3.11.

Assume that $n_{i}\geq d_{i}$ for all $i$.

$\emptyset\not=R_{n,d}^{-}\subset\\{\epsilon_{i}\text{ is invertible for all
}i\\}\subset R_{n,d}^{s}.$

###### Proof.

From the proof of Proposition 3.8, it is clear that $\epsilon_{i}$ is
invertible over $R_{n,d}^{-}$ and these points belong to $R_{n,d}^{s}$. To see
that $R_{n,d}^{-}\not=\emptyset$, we can take $\epsilon_{i}=\mathrm{Id}$ and
$b_{i}=0$ for all $i\in Q_{0}$, and all the representing matrices for the
arrows of $Q$ to be $0$. This gives a point in $R_{n,d}$ at which
$H_{i}^{-}=\mathrm{Id}$ is positive-definite. ∎

Suppose a Lie group $G$ acts on a vector bundle $V\overset{\pi}{\to}M$
equivariantly fiberwise linearly, and the action of $G$ on $M$ is free and
proper. A metric $H$ on $V$ is $G$-equivariant if

$H_{x}(v,w)=H_{g\cdot x}(c\cdot v,g\cdot w).$

It is possible that $V$ may not descend to a vector bundle over $M/G$ if
$G_{p}\subset G$ acts on $V$ non-trivially at a point $p\in M$. In the case
that the corresponding bundle does exist, $H$ will descend to that bundle if
and only if $H$ is $G$-equivariant.

Since we know that $R_{n,d}^{-}$ is a $\mathrm{GL}(d)$-invariant non-compact
open subset, we can quotient by $\mathrm{GL}(d)$ in the same way we do for
$R_{n,d}^{s}$.

######  Definition 3.12.

We define the dual of $\mathcal{M}$ as the quotient
$\mathcal{M}^{-}=R_{n,d}^{-}/\mathrm{GL}(d)$ with universal bundles
$\mathcal{V}_{i}^{-}:=(R_{n,d}^{-}\times\mathbb{C}^{d_{i}})/\mathrm{GL}(d_{i})$.
Since $H_{i}^{-}$ is Hermitian and $G_{d}$-equivariant, it descends to a
metric on $\mathcal{V}_{i}^{-}$ over $\mathcal{M}^{-}$.

######  Remark 3.13.

As a result of Proposition 3.11 and the fact that $\mathrm{GL}(d)$ acts only
on the left on the framing space,
$e^{(i)}=(\epsilon_{i},b_{i})=(I_{d_{i}},\tilde{b}_{i})$ where
$\tilde{b}_{i}=\epsilon_{i}^{-1}b_{i}$ and is itself a generic bias vector for
each $i$. Thus, from this point forward we will be assuming both that
$n_{i}\geq d_{i}$ for all $i$ and that all framing maps are of the form
$e^{(i)}=(\mathrm{Id},b_{i})$. Thus $\mathcal{M}^{-}\subset R_{n-d,d}$.

######  Example 3.14.

Consider the framed $A_{1}$ quiver (the quiver with one vertex and zero
arrows). Let $d\leq n=N$. Then $\mathcal{M}$ is $\textrm{Gr}(n,d)$. As a
Hermitian symmetric space, this is dual to the space-like Grassmannian
$\textrm{Gr}^{-}(n,d)$. Here, we define $\textrm{Gr}^{-}(n,d)$ to be the open
subset of $\textrm{Gr}(n,d)$ consisting of $d$-planes in $\mathbb{C}^{n}$
where the quadratic form

$Q(x,y)=\sum_{i=1}^{d}\overline{x}_{i}y_{i}-\sum_{j=d+1}^{n}\overline{x}_{j}y_{j}$

is positive-definite.

Similar to Remark 3.13, we can take elements of $\textrm{Gr}^{-}(n,d)$ to be
of the form $(\mathrm{Id},b)$ where the first $d$-many columns are the
$d\times d$ identity matrix $b$ is the remaining $d\times(n-d)$ columns. Then
we can say that $\textrm{Gr}^{-}(n,d)$ is the set
$\\{b\in\mathbb{C}^{d\times(n-d)}:\mathrm{Id}-bb^{*}\geq 0\\}$.

Going back to the quiver, since there is no other arrow, we see that
$H_{1}^{-}=(\mathrm{Id}-bb^{*})^{-1}$. Thus, $\mathcal{M}^{-}$ is going to be
the set $\\{b:\mathrm{Id}-bb^{*}\geq 0\\}$.

In particular, when $d=1$, $\textrm{Gr}(n,1)^{-}$ is complex hyperbolic space
and the Ricci curvature of $H_{1}^{-}=\frac{1}{1-|b|^{2}}$ is the standard
metric for the Poincare disk model of complex hyperbolic space.

Now we define an explicit metric on $\mathcal{M}^{-}$, using (22) written in
terms of paths in $Q$, in an analogous way as the one given in Theorem 3.3.

######  Theorem 3.15.

Assume $Q$ is acyclic. Define
$H_{T}^{-}:=-i\sum\limits_{i}\partial\overline{\partial}\log\det H_{i}^{-}$ on
$\mathcal{M}^{-}$. Then $H_{T}^{-}$ is a Kähler metric on $\mathcal{M}^{-}$.

###### Proof.

This proof is similar to that of Theorem 3.15 in [JL21]. We include the
details for the reader’s convenience.

Let’s denote
$\rho=\rho^{(i)}=\left(w_{\gamma}e^{(t(\gamma)}\right)_{\gamma:h(\gamma)=i}$
which is a matrix-valued function on $R_{n,d}^{-}$. At each point of
$R_{n,d}^{-}$, we have that $\rho$ is a linear map from
$W_{i}:=\bigoplus\limits_{j\leftsquigarrow i}\mathbb{C}^{n_{i}}$ to $V_{i}$.
The Ricci curvature of the metric $H_{i}^{-}$ is given by
$i\partial\overline{\partial}\log\det\rho A\rho^{*}$ where $A$ is the matrix
$\text{diag}(1,-1,-1,\dots,-1)$. Let $B$ be the matrix
$\text{diag}(1,\sqrt{-1},\dots,\sqrt{-1})$ and define $\hat{\rho}:=\rho B$ so
that $\hat{\rho}\hat{\rho}^{*}=\rho A\rho^{*}$. Thus we have that
$H_{i}^{-}=(\hat{\rho}\hat{\rho}^{*})^{-1}$.

We can take the singular valued decomposition of $\hat{\rho}$ to write it as

$\hat{\rho}=U\cdot(\begin{matrix}\text{diag}(\lambda_{1},\dots,\lambda_{d_{i}})&0\\\
\end{matrix})\cdot V^{*}$

where $U\in U(d_{i})$, $V\in U(\dim W_{i})$, and the $\lambda_{i}$ are all
positive real numbers. We know that none of the $\lambda_{i}$ are zero since
that would make corresponding quiver representations non-surjective and thus
unstable. Then

$\hat{\rho}=U\cdot(\begin{matrix}\text{diag}(\lambda_{1},\lambda_{2},\dots,\lambda_{d_{i}})&0\\\
\end{matrix})\cdot V^{*},$
$\hat{\rho}\hat{\rho}^{*}=U\left(\text{diag}(\lambda_{1}^{2},\lambda_{2}^{2},\dots,\lambda_{d_{i}}^{2})\right)U^{*},$
$\hat{\rho}^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}=V\left(\begin{matrix}\text{diag}(\lambda_{1},\dots,\lambda_{d_{i}})\\\
0\\\
\end{matrix}\right)(\text{diag}(\lambda_{1}^{-1},\dots,\lambda_{d_{i}}^{-1})U^{*}=V\left(\begin{matrix}I_{d_{i}}\\\
0\\\ \end{matrix}\right)U^{*}.$

Let us consider the decomposition
$W_{i}=(\mathrm{Im}\,\hat{\rho}^{*})\oplus(\mathrm{Im}\,\hat{\rho}^{*})^{\perp}$.
In particular, this shows that
$\hat{\rho}^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}$ is the orthogonal
embedding of $V_{i}$ to $\mathrm{Im}\,\hat{\rho}^{*}\subset W_{i}$.

Then

$\partial\overline{\partial}\log\det\hat{\rho}\hat{\rho}^{*}=\partial\left(\mathrm{tr}\,\left((\hat{\rho}\hat{\rho}^{*})^{-1}\overline{\partial}(\hat{\rho}\hat{\rho}^{*})\right)\right)$
$=\mathrm{tr}\,\left(\partial\left((\hat{\rho}\hat{\rho}^{*})^{-1}(\hat{\rho})(\partial\hat{\rho})^{*}\right)\right)$
$=\mathrm{tr}\,\left((\hat{\rho}\hat{\rho}^{*})^{-1}(\partial\hat{\rho})(\partial\hat{\rho})^{*}+\left(\partial(\hat{\rho}\hat{\rho}^{*})^{-1}\right)\hat{\rho}(\partial\hat{\rho})^{*}\right)$
$=\mathrm{tr}\,\left((\partial\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}(\partial\hat{\rho})\right)-\mathrm{tr}\,\left((\hat{\rho}\hat{\rho}^{*})^{-1}(\partial(\hat{\rho}\hat{\rho}^{*}))(\hat{\rho}\hat{\rho}^{*})^{-1}\hat{\rho}(\partial\hat{\rho})^{*}\right)$
$=\mathrm{tr}\,\left((\partial\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}(\partial\hat{\rho})\right)-\mathrm{tr}\,\left((\hat{\rho}\hat{\rho}^{*})^{-1}\hat{\rho}(\partial\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}(\partial\hat{\rho})\hat{\rho}^{*}\right)$
$=\mathrm{tr}\,\left((\partial\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}(\partial\hat{\rho})\right)-\mathrm{tr}\,\left(\left((\partial\hat{\rho})\cdot\left(\hat{\rho}^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}\right)\right)^{*}(\rho\rho^{*})^{-1}\left((\partial\hat{\rho})\cdot\left(\hat{\rho}^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}\right)\right)\right).$

Consider a vector $v\in T^{1,0}R^{-}_{n,d}\cong TR^{-}_{n,d}$. We can see that
the term
$\mathrm{tr}\,\left((\partial_{v}\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}(\partial_{v}\hat{\rho})\right)$
is in fact the square norm of the linear map $\partial_{v}\hat{\rho}$ with
respect to the metric $H_{i}^{-}$. Using the decomposition of $W_{i}$ above,
let’s write $\partial_{v}\hat{\rho}$ as the decomposition
$\partial_{v}\hat{\rho}=((\partial_{v}\hat{\rho})_{1},(\partial_{v}\hat{\rho})_{2})$
where $(\partial_{v}\hat{\rho})_{1}:\mathrm{Im}\,\hat{\rho}^{*}\to V_{i}$ and
$(\partial_{v}\hat{\rho})_{2}:(\mathrm{Im}\,\hat{\rho}^{*})^{\perp}\to V_{i}$.
In particular, given the previous discussion, we see that
$(\partial_{v}\hat{\rho})_{1}$ is actually $\partial_{v}\hat{\rho}$ composed
with $\hat{\rho}^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}$. Thus, we can
see that the other term

$\mathrm{tr}\,\left(\left((\partial_{v}\hat{\rho})\cdot\left((\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}\right)\right)^{*}(\hat{\rho}\hat{\rho}^{*})^{-1}\left((\partial_{v}\hat{\rho})\cdot\left((\hat{\rho})^{*}(\hat{\rho}\hat{\rho}^{*})^{-\frac{1}{2}}\right)\right)\right)$

is actually the square norm of $\partial\hat{\rho}_{1}$ (with respect to the
$H_{i}^{-}$ metric). Then we have

$i\partial\overline{\partial}\log\det
H_{i}^{-}=|\partial\hat{\rho}|_{H_{i}^{-}}-|\partial\hat{\rho}_{1}|_{H_{i}^{-}}=|\partial\hat{\rho}_{2}|_{H_{i}^{-}}.$

Thus, the Ricci curvature is semi-positive definite.

Now suppose $\left(\partial_{v}\hat{\rho}^{(i)}\right)_{2}=0$ for all $i$.
Then the image of
$\left(\partial_{v}\hat{\rho}^{(i)}\right)^{*}=\partial_{v}(\hat{\rho}^{(i)})^{*}$
is in the image of $(\hat{\rho}^{(i)})^{*}$. Thus $\partial_{v}$ does not
alter the subspaces given by $(\hat{\rho}^{(i)})^{*}:V_{i}\to W_{i}$.
$((\hat{\rho}^{(i)})^{*})_{i\in I}$ gives an embedding of $\mathcal{M}^{-}$ to
the product of Grassmannians of subspaces in $W_{i}$. Since $\partial_{v}$
does not change the subspaces, it must be the zero tangent vector. As a
result, the curvature is positive definite and defines a Kähler metric. ∎

######  Example 3.16.

Consider the framed $A_{2}$ quiver. This quiver has vertices (1) and (2) and
has one arrow $a$ going from $(1)$ to $(2)$. Then

$H_{1}^{-}=\left(Id_{d_{1}}-b_{1}b_{1}^{*}\right)^{-1}$

and

$H_{2}^{-}=\left(Id_{d_{2}}-b_{2}b_{2}^{*}-w_{a}w_{a}^{*}-w_{a}b_{1}b_{1}^{*}w_{a}^{*}\right)^{-1}=(Id_{d_{2}}-b_{2}b_{2}^{*}-w_{a}H^{-1}_{1}w_{a}^{*})^{-1}$

where $H_{1}=(\mathrm{Id}_{d_{1}}+b_{1}b_{1}^{*})^{-1}$ is the Hermitian
metric on $\mathcal{V}_{1}$ in Definition 3.3.

Using Gram-Schmidt orthonormalization, we can write
$H_{1}^{-1}=g(b_{1})g(b_{1})^{*}$ for some $g(b_{1})\in\mathrm{GL}(d_{1})$.
Thus

$H_{2}^{-}=(Id_{d_{2}}-(w_{a}g(b_{1}))(w_{a}g(b_{1}))^{*}-b_{2}b_{2}^{*})^{-1}.$

$\mathcal{M}^{-}=\\{(b_{1},w_{a},b_{2}):H_{1}^{-}\textrm{ and
}H_{2}^{-}\textrm{ are positive definite}\\}$. Then we have the map

$\mathcal{M}^{-}\to\textrm{Gr}(n_{1},d_{1})^{-}\times\textrm{Gr}(n_{2}+d_{1},d_{2})^{-}$

by $(b_{1},w_{a},b_{2})\mapsto(b_{1},(w_{a}g(b_{1}),b_{2}))$, which is
invertible. We have identifications of the universal bundles
$(\mathcal{V}_{i},H_{i}^{-})$ with the pullback of tautological bundles over
$\textrm{Gr}(n_{1},d_{1})^{-}$ and $\textrm{Gr}(n_{2}+d_{1},d_{2})^{-}$
respectively, which are compatible with this diffeomorphism.

We can go much further than this. In fact, for general acyclic quivers there
exists an identification between $(\mathcal{V}_{i}^{-},H_{i}^{-})$ over
$\mathcal{M}^{-}$ and the tautological bundles over space-like Grassmannians
as in the above example, if we ignore complex structures.

######  Theorem 3.17.

Assume that the underlying quiver $Q$ is acyclic. Then there exists a
symplectomorphism

$\phi:(\mathcal{M}^{-},H_{T\mathcal{M}^{-}}^{-})\stackrel{{\scriptstyle\cong}}{{\to}}\prod\limits_{i}(\textrm{Gr}^{-}(m_{i},d_{i}),H_{(m_{i},d_{i})}^{-})$

that restricts to a diffeomorphism between the real loci, and a bundle
isomorphism

$(\mathcal{V}_{i}^{-},H_{i}^{-})\overset{\cong}{\to}(\phi^{*}U_{i},H_{(m_{i},d_{i})}^{-})$

that restricts to a bundle isomorphism between the corresponding real vector
bundles over the real loci. Here $U_{i}$ is the tautological bundle over
$\textrm{Gr}^{-}(m_{i},d_{i})$, $m_{i}=n_{i}+\sum_{a:h(a)=i}d_{t(a)}$, and
$H_{(m_{i},d_{i})}^{-}$ is the standard metric of
$\textrm{Gr}^{-}(m_{i},d_{i})$.

First, we make the following lemma.

######  Lemma 3.18.

$H_{i}^{-}=\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum\limits_{a:h(a)=i}w_{a}H_{t(a)}^{-1}w_{a}^{*}\right)^{-1}$.

###### Proof.

Consider vertex $(i)$ in quiver $Q$. Let’s denote
$\Gamma_{i}:=\\{\gamma:h(\gamma)=i\\}$, the paths ending at $(i)$.

Aside from the trivial path, every $\gamma$ in $\Gamma_{i}$ must be of the
form $a\cdot\gamma$ for some arrow $a$ with $h(a)=i$. Thus, we can decompose
$\Gamma_{i}=\\{(i)\\}\cup\bigcup\limits_{a:h(a)=i}a\cdot\Gamma_{t(a)}$ where
$(i)$ denotes the trivial path. Because of this, we can write

$H_{i}^{-}=\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}\sum_{\gamma\in\Gamma_{t(a)}}w_{a\cdot\gamma}e^{t(a\cdot\gamma)}\left(w_{a\cdot\gamma}e^{t(a\cdot\gamma)}\right)^{*}\right)^{-1}.$

We have $w_{a\cdot\gamma}=w_{a}\cdot w_{\gamma}$ where $w_{a}$ is the linear
map associated to the arrow $a$. Moreover, $t(a\gamma)=t(\gamma)$. Thus

$\displaystyle H_{i}^{-}=$
$\displaystyle\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}\sum_{\gamma\in\Gamma_{t(a)}}w_{a}w_{\gamma}e^{t(\gamma)}\left(w_{a}w_{\gamma}e^{t(\gamma)}\right)^{*}\right)^{-1}$
$\displaystyle=$
$\displaystyle\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}w_{a}\left(\sum_{\gamma\in\Gamma_{t(a)}}w_{\gamma}e^{t(\gamma)}\left(w_{\gamma}e^{t(\gamma)}\right)^{*}\right)w_{a}^{*}\right)^{-1}$
$\displaystyle=$
$\displaystyle\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}w_{a}H_{j}^{-1}w_{a}^{*}\right)^{-1}.$

∎

###### Proof of Theorem 3.17.

Let $(i)$ be a vertex. By Lemma 3.18, we can write $H_{i}^{-}$ as

$\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}w_{a}H_{t(a)}^{-1}w_{a}^{*}.$

By Gram-Schmidt normalization, we can write
$H_{t(a)}^{-1}=g_{t(a)}g_{t(a)}^{*}$ for some
$g_{t(a)}\in\mathrm{GL}(d_{t(a)})$. Then

$H_{i}^{-}=\left(\mathrm{Id}-b_{i}b_{i}^{*}-\sum_{a:h(a)=i}w_{a}g_{t(a)}g_{t(a)}^{*}w_{a}^{*}\right)^{-1}=\left(\mathrm{Id}-ww^{*}\right)^{-1}$

where

$w=b_{i}\oplus\bigoplus\limits_{a:h(a)=i}w_{a}g_{t(a)}.$

Thus, we define
$\phi:(\mathcal{M}^{-},H_{T\mathcal{M}^{-}}^{-})\to\prod\limits_{i}(\textrm{Gr}^{-}(m_{i},d_{i}),H_{(m_{i},d_{i})}^{-})$
by sending $(b_{i},w_{a})_{i\in Q_{0},a\in Q_{1}}$ to
$(b_{i},(w_{a}g_{t(a)})_{a:h(a)=i})_{i\in Q_{0}}$. $\phi$ is invertible: for
each $i\in Q_{0}$, $g_{i}$ only depends on $b_{j}$ for $j\in Q^{(i)}_{0}$ and
$w_{a}$ for $a\in Q^{(i)}_{1}$, where $Q^{(i)}$ is the sub-quiver containing
those arrows that can be a part of a path heading to $i$. Then we can solve
back $w_{a}$ inductively from $b_{i}$ and $w_{a}g_{t(a)}$ (where $g_{t(a)}$ is
invertible). Since $\phi$ identifies $H_{i}^{-}$ with the standard metric on
the tautological bundle of $\textrm{Gr}^{-}(m_{i},d_{i})$, and the symplectic
form is $H_{T}^{-}=-i\sum\limits_{i}\partial\overline{\partial}\log\det
H_{i}^{-}$, $\phi$ is a symplectomorphism. Written in these coordinates,
$(\mathcal{V}_{i}^{-},H_{i}^{-})\overset{\cong}{\to}(\phi^{*}U_{i},H_{(m_{i},d_{i})})$
is simply given by identity.

Restricting to $b_{i}$ and $w_{a}$ having real coordinates, $g_{i}$ produced
from the Gram-Schmidt process is a real matrix. Thus $\phi$ restricts as a
diffeomorphism between the real loci. ∎

######  Remark 3.19.

This correspondence between $\mathcal{M}^{-}$ and
$\textrm{Gr}^{-}(m_{i},d_{i}),H_{(m_{i},d_{i})}^{-}$ is only a
symplectomorphism, since the Gram-Schmidt process is not holomorphic.

###  3.4. Euclidean Signature

In addition to the non-compact dual, we can use Equation 20 to get other
interesting moduli spaces in the same vein. The most straightforward variant
is achieved by setting $\alpha$ and all of the $\alpha_{\gamma}$ to zero. This
means throwing out the contribution coming from anything other than the first
$d_{i}$-many framing arrows.

######  Definition 3.20.

Assume $Q$ has no oriented cycle. Let $\rho_{i}$ be as in definition 3.3 so
that $\rho_{i}$ is a row vector with entries of the form
$w_{\gamma}e^{t(\gamma)}$ where $\gamma$ is some path in $Q$ ending at vertex
$(i)$, $w_{\gamma}$ is the representing matrix of this path, and
$e^{t(\gamma)}$ is the framing map at $t(\gamma)$, the starting vertex of
$\gamma$. Arrange the entries of $\rho_{i}$ so that the first $n_{i}$-many
entries correspond to the framing arrows at vertex $(i)$. Then let $H_{i}^{0}$
be the quadratic form defined by:

(23) $H_{i}^{0}=\left(\rho_{i}\left(\begin{matrix}I_{d_{i}}&0\\\
0&0\end{matrix}\right)\rho_{i}^{*}\right)^{-1}$

Here, $N_{i}=\sum\limits_{(j)\rightsquigarrow(i)}n_{j}$. We define
$R_{n,d}^{0}$ to be the subset of $R_{n,d}$ where $H_{i}^{0}$ is positive-
definite for all $i$.

Note that we still need $N_{i}\geq d_{i}\forall i$ to have
$\mathcal{M}\neq\emptyset$, thus we will still be assuming that to be the
case. Indeed, most of the following statements are copied or follow from
analogous statements in Section 3.3.

######  Proposition 3.21.

$R_{n,d}^{0}\subset R_{n,d}^{s}$.

###### Proof.

As in Proposition 3.8, consider a point in $R_{n,d}^{0}$. Write
$\rho_{i}=(\epsilon_{i}\,\,R)$ evaluated at this point as a $(d_{i}\times
N_{i})$-matrix, where $\epsilon_{i}$ is a $(d_{i}\times d_{i})$-matrix and $R$
is the remaining part. Then $H_{i}^{0}=(\epsilon_{i}\epsilon_{i}^{*})^{-1}$.
If $\epsilon_{i}$ is not invertible, then $\epsilon_{i}\epsilon_{i}^{*}$ is
not positive-definite. Thus, for a point in $R_{n,d}^{0}$, we have that
$\epsilon_{i}$ is invertible for all $i$ which means that $\rho_{i}$ is
surjective for all $i$. Thus the point is stable. ∎

######  Lemma 3.22.

$H_{i}^{0}$ is $G_{d}$-equivariant and $R_{n,d}^{0}$ is $G_{d}$-invariant.

###### Proof.

This follows directly from Lemma 3.9.

∎

Similar to Section 3.3, we will assume $n_{i}\geq d_{i}$ from this point
forward. Thus, we can talk about the framing part $\epsilon_{i}$ of $e^{(i)}$
corresponding to the first $d_{i}$-many components, and the bias part $b_{i}$
of $e^{(i)}$ corresponding to the remaining $(n_{i}-d_{i})$-many components.
With this, $H_{i}^{0}$ can be written simply as

$H_{i}^{0}=(\epsilon_{i}\epsilon_{i}^{*})^{-1}$

######  Proposition 3.23.

Assume that $n_{i}\geq d_{i}$ for all $i$.

$\emptyset\not=R_{n,d}^{0}=\\{\epsilon_{i}\text{ is invertible for all
}i\\}\subset R_{n,d}^{s}.$

###### Proof.

From the proof of Proposition 3.21, it is clear that $\epsilon_{i}$ is
invertible over $R_{n,d}^{0}$ and these points belong to $R_{n,d}^{s}$.
Moreover, let $w$ be any point of $R_{n,d}^{s}$ such that the framing parts
$\epsilon_{i}$ of the framing maps $e^{(i)}$ are all invertible. Since
$H_{i}^{0}$ is only defined using $\epsilon_{i}$, we can see that $w\in
R_{n,d}^{0}$. Thus, $R_{n,d}^{0}$ is the subset of $R_{n,d}^{s}$ of points
where the framing part is invertible. To see that $R_{n,d}^{0}\not=\emptyset$,
we can take $\epsilon_{i}=\mathrm{Id}$ for all $i\in Q_{0}$ and set the
remaining arrows to be zero. This gives a point in $R_{n,d}$ at which
$H_{i}^{0}=\mathrm{Id}$ is positive-definite. ∎

Similar to $R_{n,d}^{-}$, since we know that $R_{n,d}^{0}$ is a
$\mathrm{GL}(d)$-invariant non-compact open subset of $R_{n,d}^{s}$, we can
directly quotient by $\mathrm{GL}(d)$.

######  Definition 3.24.

We define the Euclidean restriction of $\mathcal{M}$ as the quotient
$\mathcal{M}^{0}=R_{n,d}^{0}/\mathrm{GL}(d)$ with universal bundles
$\mathcal{V}_{i}^{0}:=(R_{n,d}^{0}\times\mathbb{C}^{d_{i}})/\mathrm{GL}(d_{i})$.
Since $H_{i}^{0}$ is Hermitian and $G_{d}$-equivariant, it descends to a
metric on $\mathcal{V}_{i}^{0}$ over $\mathcal{M}^{0}$.

As a result of Proposition 3.23 and the fact that $\mathrm{GL}(d)$ acts only
on the left on the framing space,
$e^{(i)}=(\epsilon_{i},b_{i})=(I_{d_{i}},\tilde{b}_{i})$ where
$\tilde{b}_{i}=\epsilon_{i}^{-1}b_{i}$ and is itself a generic bias vector for
each $i$. Thus, from this point forward we will be assuming both that
$n_{i}\geq d_{i}$ for all $i$ and that all framing maps are of the form
$e^{(i)}=(\mathrm{Id},b_{i})$. Thus $\mathcal{M}^{0}\cong R_{n-d,d}$ and
$H_{i}^{0}$ can be taken to be the trivial metric on $\mathbb{C}^{d_{i}}$ for
each $i$.

Over $\mathcal{M}^{0}$, activation functions have the simplest possible
definition: smooth (or piece-wise smooth) maps from $\mathbb{C}^{d_{i}}$ to
itself. Any of the standard activation functions used in machine learning
(sigmoid, ReLu, softmax, etc.) directly fit in this Euclidean restriction
setting without any further modification.

######  Corollary 3.25.

$H_{i}^{0}$ is the trivial metric on $\mathbb{C}^{d_{i}}$. Thus,
$H_{T}^{0}:=\sum\limits_{i}\partial\overline{\partial}\log\det H_{i}^{0}$ is a
Ricci-flat Kähler-Einstein metric and $R_{n,d}^{-}\subset R_{n,d}^{0}$.

######  Remark 3.26.

Consider an acyclic quiver $Q$ with dimension vector $(n,d)$ such that
$n_{i}=d_{i}$ for all source and sink vertices $i$, and $n_{i}=d_{i}+1$ for
all others. If $Q$ with dimension vector $(n,d)$ gives the underlying neuron
structure for a neural network, then $\mathcal{M}^{0}$ is the training space
for this network. In particular, the standard backward propagation algorithm
for a feed-forward neural network is standard gradient descent in the relevant
vector space, matching up exactly with the gradient descent on
$\mathcal{M}^{0}$ induced by $H_{i}^{0}$.

###  3.5. Hyperbolic Activation Functions

This point of view of uniformization provides a learning model over hyperbolic
moduli, or more generally, interpolations of spherical, Euclidean and
hyperbolic moduli. (One can add learnable parameters in the Hermitian metrics
$H_{i}$, interpolating the metrics of different types.) This is hyperbolic
learning in the base (that is the parameter space). There is another direction
that we can consider hyperbolic learning, namely the fiber bundle direction.

Recall that we have the universal vector bundles $\mathcal{V}_{i}$. In [JL21],
we cooked up activation function (as a fiber bundle map of $\mathcal{V}_{i}$)
by composing the following:

$\mathcal{V}_{i}\stackrel{{\scriptstyle
H_{i}}}{{\cong}}\mathcal{V}_{i}^{*}\stackrel{{\scriptstyle(e^{(i)})^{*}}}{{\to}}\underline{\mathbb{C}^{n_{i}}}\stackrel{{\scriptstyle\sigma}}{{\to}}\underline{\mathbb{C}^{n_{i}}}\stackrel{{\scriptstyle
e^{(i)}}}{{\to}}\mathcal{V}_{i}$

where $\sigma:\mathbb{C}^{n_{i}}\to\mathbb{C}^{n_{i}}$ is a continuous
function. We can do the same thing uniformly for $\mathcal{M},\mathcal{M}^{0}$
and $\mathcal{M}^{-}$.

Additionally, in [JL21], we constructed a specific activation function as a
symplectomorphism
$(\mathbb{C}^{n},\omega_{\mathbb{P}^{n}}|_{\mathbb{C}^{n}})\cong(B,\omega_{\textrm{std}})$,
where $B\subset\mathbb{C}^{n}$ is the ball $\\{\|\vec{z}\|^{2}<1\\}$,
$\omega_{\mathbb{P}^{n}}$ is the Fubini-Study metric on $\mathbb{P}^{n}$, and
$\omega_{\textrm{std}}$ is the standard symplectic form of $\mathbb{C}^{n}$.
$\sigma$ has the expression

$(z_{1},\dots,z_{n})\to\left(\frac{z_{1}}{\sqrt{1+\sum_{i=1}^{n}|z_{i}|^{2}}},\dots,\frac{z_{n}}{\sqrt{1+\sum_{i=1}^{n}|z_{i}|^{2}}}\right).$

In view of hyperbolic metrics, we provide an alternative interpretation of the
same function here.

######  Proposition 3.27.

$\sigma$ gives a symplectomorphism
$(\mathbb{C}^{n},\omega_{\textrm{std}})\to(\mathbb{C}\mathbb{H}^{n},\omega_{\mathbb{C}\mathbb{H}^{n}})$
where $\mathbb{C}\mathbb{H}^{n}$ denotes the hyperbolic ball.

###### Proof.

By definition, $\omega_{\mathbb{C}\mathbb{H}^{n}}$ equals to
$-\partial\overline{\partial}\log(1-|w|^{2})$ up to a simple scaling. Here, we
will be thinking of $w$ as the row vector $(w_{1},\dots,w_{n})$. Then

$-\partial\overline{\partial}\log(1-|w|^{2})=\partial\frac{wdw^{*}}{1-|w|^{2}}$
$=\frac{(1-|w|^{2})dw\wedge dw^{*}+(dw\cdot
w^{*})wdw^{*}}{(1-|w|^{2})^{2}}=\frac{(1-|w|^{2})dw\wedge
dw^{*}+\overline{w}dw^{t}\overline{dw}w^{t}}{(1-|w|^{2})^{2}}$

Now, let’s similarly write $z$ as the row vector $(z_{1},\dots,z_{n})$. We
compute the pullback as

$\sigma^{*}(dz\wedge dz^{*})=d\frac{z}{\sqrt{1-|z|^{2}}}\wedge
d\frac{z^{*}}{\sqrt{1-|z|^{2}}}$
$=\frac{(1-zz^{*})dz+\frac{1}{2}(\overline{zdz^{*}}+zdz^{*})z}{(1-zz^{*})^{3/2}}\wedge\frac{(1-zz^{*})dz^{*}+\frac{1}{2}(\overline{zdz^{*}}+zdz^{*})z^{*}}{(1-zz^{*})^{3/2}}$
$=\frac{1}{(1-zz^{*})^{3}}\left((1-zz^{*})^{2}dz\wedge
dz^{*}+\frac{1}{4}(\overline{zdz^{*}}+zdz^{*})^{2}zz^{*}\right)$
$+\frac{1}{(1-zz^{*})^{3}}\left(\frac{1}{2}(1-zz^{*})\left((\overline{zdz^{*}}+zdz^{*})z\wedge
dz^{*}+dz\wedge(\overline{zdz^{*}}+zdz^{*})z^{*}\right)\right)$

At this point, $\overline{zdz^{*}}\wedge zdz^{*}+dzz^{*}\wedge zdz^{*}$ can be
rewritten as $2dz^{t}\overline{dz}z^{t}$. This gives us

$=\frac{(1-zz^{*})dz\wedge
dz^{*}+\overline{z}dz^{t}\overline{d}zz^{t}}{(1-zz^{*})^{2}}$

which equals to above. ∎

In other words, $\sigma$ gives an identification between $\mathbb{C}^{n}$ and
$\mathbb{C}\mathbb{H}^{n}$. Then signal propagation between hyperbolic spaces
can be modeled simply as linear maps between $\mathbb{C}^{n}$, and the
composition $\iota\circ\sigma:\mathbb{C}^{n}\to\mathbb{C}^{n}$, where
$\iota:\mathbb{C}\mathbb{H}^{n}\to\mathbb{C}^{n}$ is the inclusion of a ball
in the space, gives an activation function.

## References

  * [AJ20] M. A. Armenta and P.-M. Jodoin, _The representation theory of neural networks_ , preprint (2020), arXiv:2007.12213.
  * [Bir35] G. Birkhoff, _On the structure of abstract algebras_ , Mathematical Proceedings of the Cambridge Philosophical Society 31 (1935), no. 4, 433–454.
  * [CAW+19] M.C.N. Cheng, V. Anagiannis, M. Weiler, P. de Haan, T.S. Cohen, and M. Welling, _Covariance in physics and convolutional neural networks_ , preprint (2019), arXiv:1906.02481.
  * [CGKW18] T.S. Cohen, M. Geiger, J. Koehler, and M. Welling, _Spherical CNNs_ , ICLR (2018).
  * [CGW19] T.S. Cohen, M. Geiger, and M. Weiler, _A general theory of equivariant cnns on homogeneous spaces_ , NeurlPS (2019), arXiv:1811.02017.
  * [CHL] Y. Chen, Y. Huang, and N. Leung, _Embeddings from noncompact symmetric spaces to their compact duals_.
  * [Con85] A. Connes, _Noncommutative differential geometry_ , Inst. Hautes Études Sci. Publ. Math. (1985), no. 62, 257–360.
  * [CQ95] J. Cuntz and D. Quillen, _Algebra extensions and nonsingularity_ , J. Amer. Math. Soc. 8 (1995), no. 2, 251–289.
  * [CW16] T.S. Cohen and M. Welling, _Group equivariant convolutional networks_ , Proceedings of The 33rd International Conference on Machine Learning, vol. 48, 2016, pp. 2990–2999.