# Continual Lie algebras determined by chain complexes

A. Zuevsky  Institute of Mathematics
Czech Academy of Sciences
Zitna 25, Prague
Czech Republic<EMAIL_ADDRESS>
###### Abstract.

Continual Lie algebras are infinite dimensional generalizations of ordinary
Lie with continuous set of roots. In this paper we establish a general
relation between chain complexes and continual Lie algebras. First, by
defining and appropriate product among a chain complex spaces we endow it with
the structure of an exterior differential graded algebra $\mathcal{C}$. The
natural orthogonality condition with respect to a product bring about to
$\mathcal{C}$ the structure of a graded algebra with differential relations
and compatibility conditions for elements of chain complex spaces. By
considering the union of all relations for independent elements in the tree
graph structure of the system, we prove the main result of this paper: a chain
complex brings about the structure of a continual Lie algebra with the root
space determined by parameters for the complex. This provides a new source of
examples of continual Lie algebras. Finally, as an example, we consider the
case of Čech-de Rham complex associated to a foliation on a smooth manifold.
In a particular case of this chain complex, we derive explicitly the
commutation relations for corresponding continual Lie algebra.

AMS Classification: 53C12, 57R20, 17B69

###### Key words and phrases:

Continual Lie algebras, graded differential algebras, cohomology

##  1\. Introduction

Continual Lie algebras introduced in [12] are generalizations of ordinary Lie
algebras. Then the notion was generalized and developed in [13]–[17]. These
algebras are formulated in such a way that the space of roots is defined by
continual sets of vectors. In commutation relations, the generators depend on
kernels which are functionals of continual roots. Jacobi identities for
continual Lie algebras result in non-trivial functional relations for kernels.

Though the general theory of continual Lie algebras and their representations
is missing, there exist many applications, especially in the theory of
completely integrable and exactly solvable models [10, 10]. Various
applications can be also found in other fields of mathematics [1, 3, 13, 17].

In original papers [12]–[17], various approaches in order to find new classes
of examples of continual Lie algebras were studied. In a series of papers [15,
16] they have generalized first known examples and have found new non-trivial
and fundamental ones arrising from various branches of mathematics, in
particular, functional analysis [17], differential geometry [13], algebraic
geometry, non-commutative geometry, and mathematical physics [1, 5]. Some of
them were have come from physical applications, e.g., equations related to the
Ricci flow and cosmology [1, 5]. Though they appear in various corners of
modern mathematical research, at some stage, the arsenal of new ideas of
generation of new examples of continual Lie algebras was exhausted. An
intriguing question is how construct a general way which would allow to create
continual Lie algebras. In many cases, this problem reduces to the question of
finding new appropriate solutions for relations (1.3) (which is a very
interesting problem by itself) following from Jacobi identities.

The idea of this work is to use properties of general chain complexes, (with
spaces possessing sets of parameters) in order to derive the structure of
continual Lie algebras. The plan of the paper is the following. In the next
subsection we recall the notion of a continual Lie algebra [12]. In Section 2,
starting from a general (infinite) chain complex, we first define a product
acting among spaces of a complex. We require Leibniz rule to be fulfilled for
(co)boundary operators with respect to this product. In addition to that we
assume its skew-symmetry properties. Thus we endow the complex with the
structure of an exterior differential graded algebra $\mathcal{C}$.

We then apply the natural condition of orthogonality with respect to a product
among the chain complex spaces. For instance, in differential geometry, the
orthogonality condition reduces to the integrability condition for
differential forms. Further actions of (co)boundary operators, as well as
consequences from the definition of the multiplication, generates a system of
differential relations for elements of various chain complex spaces with extra
compatibility conditions for indexes. This determines the structure of a
graded algebra with differential relations for $\mathcal{C}$. By choosing
independent elements satisfying the above relations, we then show the main
result of this paper: for an exterior graded differential algebra associated
to a chain complex, the independent generators, the system of differential
relations, and Jacobi identities define the structure of a continual Lie
algebra with the root space provided by the spaces parameters of a chain
complex.

In Section 3 we consider the example of the Čech-de Rham cochain bicomplex [2]
(c.f. Appendix) associated to a smooth manifold foliation [4]. This bicomplex
has a deep geometric meaning [6] and is defined for the spaces of differential
forms and holonomy embeddings acting between sections of the transversal basis
for a foliation. In Appendix we recall the notions of holonomy mappings,
transversal sections, and transversal basis for a foliation. The space of
differential forms possesses already a structure of bigraded differential
algebra with respect to the natural product [2]. For the bicomplex under
consideration, we define the second product, satisfying properties required
for the construction of a system of differential relations. According to the
exposition we give in Section 2, by involving the orthogonality conditions for
an arbitrary pairs of the Čech-de Rham bicomplex, we apply the general scheme
of construction of a bigraded differential algebra. Then we single out
generators and commutation relations for corresponding continual Lie algebra
with the space of roots and kernels given by the sets of holonomy embedding.
In a particular geometric case associated to a codimension one foliation over
a three-dimensional smooth manifold, we start from the integrability condition
applied to elements of the same bicomplex space which leads to the appearance
of a continual Lie algebra with generators, kernels, and commutation relations
explicitly described.

###  1.1. Continual Lie algebras

In this subsection we recall the notion of a continual Lie algebra introduced
in [14] which is a generalization of an ordinary Lie algebra. It was then
studied in [15, 16]. Suppose $E$ is an associative algebra over $\mathbb{R}$
or $\mathbb{C}$, and

$K_{0},\;K_{\pm 1},\;K_{0,0}:E\times E\to E,$

are bilinear mappings. The local Lie part of a continual Lie algebra can be
defined as

${\widehat{\mathcal{G}}}=\mathcal{G}_{-1}\oplus\mathcal{G}_{0}\oplus\mathcal{G}_{+1},$

where $\mathcal{G}_{i}$, $i=0,\pm 1$, are isomorphic to $E$ and parametrized
by its elements. The subspaces $\mathcal{G}_{i}$ consist of the elements

$\left\\{X_{i}(\phi),\phi\in E\right\\},$

$i=0,\pm 1$. The generators $X_{i}(\phi)$ are subject to commutation
relations. For instance, in the principle grading they can have the form

$\begin{array}[]{c}\vskip 12.0pt plus 4.0pt minus
4.0pt\left[X_{0}(\phi),X_{0}(\psi)\right]=X_{0}(K_{0,0}(\phi,\psi)),\\\ \vskip
12.0pt plus 4.0pt minus 4.0pt\left[X_{0}(\phi),X_{\pm 1}(\psi)\right]=X_{\pm
1}(K_{\pm 1}(\phi,\psi)),\\\
\left[X_{+1}(\phi),X_{-1}(\psi)\right]=X_{0}(K_{0}(\phi,\psi)),\end{array}$
(1.1)

for all $\phi$, $\psi\in E$. It is also assumed that Jacobi identities are
satisfied. Then the conditions on mappings
$\widehat{K}=\left(K_{0,0},K_{0},K_{\pm 1}\right)$, follow:

$\begin{array}[]{c}\vskip 12.0pt plus 4.0pt minus 4.0ptK_{\pm
1}(K_{0,0}(\phi,\psi),\chi)=K_{\pm 1}(\phi,K_{\pm 1}(\psi,\chi))-K_{\pm
1}(\psi,K_{\pm 1}(\phi,\chi)),\\\
K_{0,0}(\psi,K_{0}(\phi,\chi))=K_{0}(K_{+1}(\psi,\phi),\chi))+K_{0}(\phi,K_{-1}(\psi,\chi)),\end{array}$
(1.2)

for all $\phi$, $\psi$, $\chi\in E$. An infinite dimensional algebra

$\mathcal{G}\left(E;\widehat{K}\right)=\mathcal{G}^{\prime}\left(E;\widehat{K}\right)/J,$

is called a continual contragredient Lie algebra, where
$\mathcal{G}^{\prime}(E;\widehat{K})$ is a Lie algebra freely generated by
${\widehat{\mathcal{G}}}$, and $J$ is the largest homogeneous ideal with
trivial intersection with $\mathcal{G}_{0}$ (consideration of the quotient is
equivalent to imposing the Serre relations in an ordinary Lie algebra case)
[15, 16]. By introducing a grading operator (as in ordinary Lie algebra case),
one can endow $\mathcal{G}$ with a $\mathbb{Z}$-grading

$\mathcal{G}=\bigoplus_{i\in\mathbb{Z}}\mathcal{G}_{n},$

where elements of subspaces $\mathcal{G}_{n}$ satisfy the standard grading
condition

$\left[\mathcal{G}_{n},\mathcal{G}_{m}\right]\subset\mathcal{G}_{n+m},$

where $X_{n}(\phi)\in\mathcal{G}_{n}$, and higher mapping $K_{n,m}(\phi,\psi)$
are present in commutation relations among $X_{n}(\phi)$ and $X_{n}(\psi)$. In
general, Jacobi identity for generators that belong to all grading spaces has
the form

$\left[X_{i}(\phi),\left[X_{j}(\psi),X_{k}(\theta)\right]\right]+\left[X_{j}(\psi),\left[X_{k}(\theta),X_{i}(\phi)\right]\right]+\left[X_{k}(\theta),\left[X_{i}(\phi),X_{j}(\psi)\right]\right]=0,$
$K_{i,j+k}\left(\phi,K_{j,k}\left(\psi,\theta\right)\right)+K_{j,k+i}\left(\psi,K_{k,i}\left(\theta,\phi\right)\right)+K_{k,i+j}\left(\theta,K_{i,j}\left(\phi,\psi\right)\right)=0.$
(1.3)

##  2\. The algebra of differential relations from a chain complex

###  2.1. The exterior differential graded algebra

In this section we formulate the construction which allow to generate an
algebra of graded differential relations starting from a chain complex.

Consider a complex of spaces depending on sets of parameters $\Theta_{i}$,
$i\in Z$, and given by

$\ldots\stackrel{{\scriptstyle\delta(i-1)}}{{\longrightarrow}}C(i,\Theta_{i})\stackrel{{\scriptstyle\delta(i)}}{{\longrightarrow}}C(i+1,\Theta_{i+1})\stackrel{{\scriptstyle\delta(i+1)}}{{\longrightarrow}}\ldots.$
(2.1)

First, we require it to be a a differential graded algbebra, that is, it
admits a differential $\delta(i)$ satisfying the chain property

$\delta(i+1)\circ\delta(i)=0,$

for $\in\mathbb{Z}$. Note that usually the index $i$ also plays a role of a
parameter counting some objects in in $C(i,\Theta_{i})$.

We assume also that there exists a (not necessary associative) product among
spaces $C(i,\Theta_{i})$,

$\cdot:C(i,\Theta_{i})\cdot C(j,\Theta_{j})\to C(f(i,j),\Theta_{g(i,j)}),$
(2.2)

so for elements $\Phi_{i}\in C(i,\Theta_{i})$ and $\Phi_{j}\in
C(j,\Theta_{j})$,

$\Phi_{i}\cdot\Phi_{j}=\Phi_{f(i,j)},$

where $f(i,j)$ and $g(i,j)$ are some functions of indeces $i$ and $j$.

Let us assume that for all $i\in\mathbb{Z}$, Leibniz rule formula for the
operator $\delta(i)$ take place, i.e.,

$\delta(k+l)\left(\Phi_{k}\cdot\Phi_{l}\right)=\left(\delta(k)\Phi_{k}\right)\cdot\Phi_{l}+(-1)^{\deg\left(\Phi_{k}\right)}\Phi_{k}\cdot\delta(l)\Phi_{l},$
(2.3)

where $|\Phi_{k}|$ is the degree of $\Phi_{k}$.

Secondly, we assume that the chain complex above possesses some properties of
an exterior algebra with respect to the product (2.2). Namely, consider the
elements $\Phi_{k}\in C(k,\Theta_{k})$, and $\Phi_{l}\in C(l,\Theta_{l})$. Let
us assume that the product (2.2) satisfies the condition

$\Phi_{k}\cdot\Phi_{l}=-\Phi_{l}\cdot\Phi_{k}.$ (2.4)

for all $k$, $l\in\mathbb{Z}$, $\Phi_{k}\in C(k,\Theta_{k})$ and $\Phi_{l}\in
C(l,\Theta_{l})$, and for any $p\in\mathbb{Z}$,

$\displaystyle\Phi_{p}\cdot\Phi_{p}=0.$ (2.5)

Altogether, the complex (2.1) form an exterior differential graded algebra
$\mathcal{C}$. In what follows, we skip the sets of parameters $\Theta_{i}$ in
notations for spaces of complexes, i.e., we denote $C(i)=C(i,\Theta_{i})$.

###  2.2. The algebra of graded differential relations

In this subsection we show how to make an exterior differential graded algebra
$\mathcal{C}$ into an graded algebra with differential relations.

For some $i$ and $j\in\mathbb{Z}$, let us introduce the orthogonality
condition for a pair $C(i,\Theta_{i})$ and $C(j,\Theta_{j})$, with respect to
the product (2.2). In particular, let us require that for a pair
$C(i,\Theta_{i})$, $C(j,\Theta_{j})$, there exist subspaces
$C^{\prime}(i)\subset C(i,\Theta_{i})$ and $C^{\prime}(j)\subset
C(j,\Theta_{j})$, such that for any $\Phi_{i}\in C^{\prime}(i)$ and
$\Phi_{j}\in C^{\prime}(j)$,

$\Phi_{i}\cdot\delta(j)\Phi_{j}=0,$ (2.6)

namely, $\Phi_{i}$ is supposed to be orthogonal to $\delta(j)\Phi_{j}$ with
respect to (2.2).

By applying further differentials to (2.6) (and further consequences of such
action), and using properties of a function $f(i,j)$ we obtain relations among
elements of spaces $C_{i}(\Theta_{i})$, $i\in\mathbb{Z}$. In particular,
taking into account that both sides of such relations belong to the same
space, we obtain limitations (depending on the function $f$) on indexes. In
differential geometry, the orthogonality condition (2.6) provides the
definition of integrability conditions for differential forms, and leads to
the Frobenius theorem (see, e.g., [7]).

Let $\chi\in C(n_{0})$, $\Phi\in C(n)$, for any $n_{0}$, $n\in\mathbb{Z}$. Due
to the property (2.5) of the multiplication (2.2), the orthogonality condition
(2.6) applied to $\Phi$ and $\chi$, i.e.,

$\Phi\cdot\delta(n_{0})\chi=0,$ (2.7)

implies that there exists $\alpha_{1}^{(R)}\in C\left({n_{1}^{(R)}}\right)$,
such that

$\delta_{n_{0}}\chi=\Phi\cdot\alpha^{(R)}_{1}.$ (2.8)

Let $r_{1}^{(R)}$ be the number of common parameters among $k$ and
$n_{1}^{(R)}$ sets of parameters for $\Phi$ and $\alpha^{(R)}_{1}$. Since both
sides of the last relation have to belong to the same bicomplex space, the
compatibility condition

$n_{0}+1=n+n_{1}^{(R)}-r_{1}^{(R)},$ (2.9)

occurs. Acting by $\delta(n_{0}+1)$ on (2.8) we obtain

$0=\left(\delta(n)\Phi\right)\cdot\alpha^{(R)}_{1}+(-1)^{n_{0}+1}\Phi\cdot\delta\left(n^{(R)}_{1}\right)\alpha^{(R)}_{1}.$
(2.10)

On the other hand, (2.7) implies that there exists $\alpha_{1}^{(L)}\in
C\left(n_{1}^{(L)}\right)$, such that

$\displaystyle\Phi=\alpha_{1}^{(L)}\cdot\delta_{n_{0}}\chi,$ (2.11)

with the condition

$n=n^{(L)}_{1}+n_{0}+1-r_{1}^{(L)},$ (2.12)

where $r_{1}^{(l)}$ is the number of common parameters among $n_{1}^{(L)}$ and
$(n_{0}+1)$ for $\alpha_{1}^{(L)}$ and $\delta_{n_{0}}\chi$.

Consequently applying corresponding $\delta$ operators to (2.7), (2.8) and
(2.11) we obtain the system of relations:

$\displaystyle 0$ $\displaystyle=$
$\displaystyle\Phi\cdot\delta(n_{0})\chi,\qquad(1)$
$\displaystyle{\rm\;(1)}\;\Rightarrow\;\Phi$ $\displaystyle=$
$\displaystyle\alpha_{1}^{(L)}\cdot\delta(n_{0})\chi,\qquad(2)$
$\displaystyle{\delta.\rm(2)\;}\Rightarrow\;\delta(n)\Phi$ $\displaystyle=$
$\displaystyle\left(\delta\left(n^{(L)}_{1}\right)\alpha_{1}^{(L)}\right)\cdot\delta(n_{0})\chi,\;\;(2^{\prime})\;\Rightarrow
0,$ $\displaystyle{\delta.\rm(1)}\;\Rightarrow\;0$ $\displaystyle=$
$\displaystyle\left(\delta(n)\Phi\right)\cdot\delta(n_{0})\chi,\qquad(3)$
$\displaystyle{\rm(1)}\;\Rightarrow\;\delta(n_{0})\chi$ $\displaystyle=$
$\displaystyle\Phi\cdot\alpha^{(R)}_{1}.$ $\displaystyle 0$ $\displaystyle=$
$\displaystyle\delta(n)\Phi\cdot\alpha^{(R)}_{1}+(-1)^{n}\Phi\cdot\delta\left(n^{(R)}_{1}\right)\alpha^{(R)}_{1}\;\Rightarrow
0,$ $\displaystyle{\rm(3)}\;\;\Rightarrow\;\delta(n_{0})\chi$ $\displaystyle=$
$\displaystyle\left(\delta(n)\Phi\right)\cdot\alpha^{(RR)}_{2},$
$\displaystyle{\rm(4):}\;\;0$ $\displaystyle=$
$\displaystyle\left(\delta(n)\Phi\right)\cdot\delta\left(n_{2}^{(RR)}\right)\alpha^{(RR)}_{2}\Rightarrow\left(\alpha^{(RRR)}_{3}\right)\Rightarrow...$
$\displaystyle\Downarrow\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad(3^{\prime})$
$\displaystyle\left(\alpha^{(RRL)}_{3}\right)\Rightarrow...$
$\displaystyle\Downarrow$ $\displaystyle...$
$\displaystyle{\rm(3)}\;\Rightarrow\;\delta(n)\Phi$ $\displaystyle=$
$\displaystyle\alpha^{(LL)}_{2}\cdot\delta(n_{0})\chi,$
$\displaystyle{\rm(5):}\;\;0$ $\displaystyle=$
$\displaystyle\delta\alpha^{(LL)}_{2}\cdot\delta(n_{0})\chi\Rightarrow\left(\alpha^{(LLR)}_{3}\right)\Rightarrow...$
(2.15)
$\displaystyle\Downarrow\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad(3^{\prime\prime})$
$\displaystyle\left(\alpha^{(LLL)}_{3}\right)\Rightarrow...$
$\displaystyle\Downarrow$ $\displaystyle...$

where we obtain (infinite) sequences (2.2) and (2.2) of pairs of relations for
$\alpha_{i}^{(K_{i})}\in C\left({n_{i}^{(K_{i})}}\right)$, $i\geq 1$. Recall
that we denote such relations as $\left(\alpha_{i}^{(K_{i})}\right)$.
Corresponding indexes $n_{i}^{(K_{i})}$ satisfy (in addition to (2.9) and
(2.12)) relations for the sequence starting from $(4)$:

$n_{0}=n+n_{i}^{(RRK_{i+2})}-r_{i}^{(RRK_{i+2})},$ (2.16)

for the sequence starting from $(5)$:

$n=n_{0}+n_{j}^{(LLK_{j+2})}-r_{j}^{(LLK_{j+2})},$ (2.17)

$i$, $j\geq 2$.

Let us explain the notations. In (2.2) we obtain (infinite) sequences of pairs
of differential relations the form (2.8)–(2.10). The (infinite) sequence of
pair of relations has a tree graph structure with two sequences outgoing from
one point. At each point we call one branch ”left” and another branch as
”right” marking corresponding pair by $L$ or $R$. We denote such pairs by
$\left(\alpha_{i}^{(K_{i})}\right)$, where $\alpha_{i}^{(K_{i})}$ is an
element of $C\left({n_{i}^{(K_{i})}}\right)$ involved in differential
relations, and $\left(K_{i}\right)$ is a sequence of $i$ entries each is
either $L$ or $R$ for $i\geq 1$.

One can easily see that, not all elements in (2.2) are independent. For
instance, from (2.9) and (2.12) we obtain

$\left(n^{(L)}_{1}-r^{(L)}_{1}\right)=-\left(n_{1}^{(R)}-r^{(R)}_{1}\right)=n-n_{0}-1.$

From (2.8)-(2.11), and from $(3^{\prime})$-$(3^{\prime\prime})$ of (2.2) we
infer that $\alpha_{1}^{(L)}$, $\alpha_{1}^{(R)}$, and
$\alpha_{i}^{(LLK_{i})}$, $\alpha_{i}^{(RRK_{i})}$ are related by a
conjugation with respect to the product (2.2):

$\Phi=\alpha_{1}^{(L)}\cdot\left(\Phi\cdot\alpha_{1}^{(R)}\right),$
$\delta(n_{0})\chi=\left(\alpha_{i}^{(LLK_{i})}\cdot\delta(n_{0})\;\chi\right)\cdot\alpha_{i}^{(RRK_{i})}.$

Similar relations applies among other elements $\alpha_{i}^{(K_{i})}$. The
sequence of relations (2.2) cancels when one of relations (2.9)–(2.12) or
(2.16)–(2.17) for a sequence of pairs of equations is not fulfilled. The
natural grading for $\mathcal{G}(\Theta_{(n(i))})$ is given by the condition
that both sides of differential relations in (2.2) belong to the same chain
complex space.

######  Remark 1.

In this paper we consider the simplest form (2.1) of a complex. In general,
for more complicated actions of $\delta$, such that

$\delta(i):C(i)\to C(i+k(i)),$

where $k(i)$ depends on $i\in\mathbb{Z}$ (see, e.g., [8] for non-trivial
actions of certain $\delta$ operators among chain complex spaces for vertex
algebras. Consideration of such complexes will be given by the author
elsewhere). corresponding differential relations as well as compatibility
relations could be different from (2.2)

As an upshot, the orthogonality condition (2.6) for all choices of $n_{0}$,
$n\in\mathbb{Z}$, and the conditions (2.16)–(2.17), applied to the chain
complex (2.1) bring about the structure of a graded algebra with differential
relations (2.2) with respect to the multiplication (2.4).

As we can see, the system of relations (2.2) has a tree structure. ”Left” and
”right” directions have a mixture of dependent elements. Let us denote
$n(i)=n_{i}^{(K_{i})}$. For $n_{0}$, $n\in\mathbb{Z}$, let $I(n_{0},n)$ be the
set of sequences of indexes $(n(i))$, $i\geq 0$, marking all paths $(K_{i})$
in the tree structure of (2.2), describing ”left” or ”right” choice at each
point. Let

$I=\bigcup\limits_{n_{0},n\in\mathbb{Z}}\;I(n_{0},n),$

be the space of all sequences over the tree graph for a complex (2.1). Denote
by $I_{0}\subset I$ the subset of such paths that include pairs of independent
elements only. Then we are able to single out generators and commutation
relations of a a generalization of an ordinary Lie algebra with a continual
space of roots, (see subsection (1.1) [14]).

###  2.3. Construction of a continual Lie algebra from a chain complex

Let us further assume the spaces $C(i)$, $i\in\mathbb{Z}$ admit also an an
ordinary (not necessary commutative) product $\Phi\Psi$ among elements for
$\Phi_{k}\in C(k)$, and $\Phi_{l}\in C(l)$, for all $k$, $l\in\mathbb{Z}$.

Then, as a product (2.2), satisfying conditions (2.4) and (2.3), one can take

$\displaystyle\Phi_{k}\cdot\Phi_{l}=\left[\Phi_{k},\Phi_{l}\right]=\Phi_{k}\Phi_{l}-\Phi_{l}\Phi_{k},$
(2.18)

where brackets mean the ordinary commutator. It is known that the introduction
of the commutator with respect to the original multiplication of an algebra
transfers it into a Lie algebra when Jacobi conditions are satisfied. In our
case we show that the exterior differential algebra $\mathcal{C}$ defined
above being supplied with the orthogonality conditions deliver the structure
of a continual Lie algebra,

To complete this section, we formulate the main result of this paper:

######  Proposition 1.

For the set $I_{0}$ of all pairs of independent elements
$\alpha_{i}^{(K_{i})}$, the orthogonality condition (2.6), the generators

$\left\\{\chi,\;\delta(n_{0})\chi,\;\Phi,\;\delta(n)\Phi,\;\alpha_{i}^{(K_{i})},\;\delta(n(i))\alpha_{i}^{(K_{i})}\right\\},$

$n(i)=n_{i}^{(K_{i})}$, the relations (2.2), and Jacobi identities (1.3) form
a continual Lie algebra $\mathcal{G}(\Theta_{(n(i))})$ with the root space
depending on a set of parameters $\Theta_{n(i)}$, of spaces
$C\left(n(i),\Theta_{n(i)}\right)$.

Though the structure of the system (2.2) may seem to be not very complicated,
actual properties of corresponding continual Lie algebra depends on properties
of the spaces of a specific the bicomplex (2.1) and the nature of parameters
$\Theta_{i}$. For a fixed choice of a path of independent
functions/differential equation in the system (2.2), there exists a variety of
choices how to identify generators of a continual Lie algebra with generators
of the differential algebra. Therefore, the actual form of commutation
relations for corresponding continual Lie algebra varies accordingly. One can
also chose various ways how to define a grading for each specific
$\mathcal{G}(\Theta_{(n(i))})$, for generators of a continual Lie algebra (see
subsection 1.1) resulting from (2.2). The structure of the product (2.2)
together with the condition (2.3), and the action of the differentials
$\delta$ provide Jacobi identities for generators on the continual Lie algebra
$\mathcal{G}(\Theta_{(n(i))})$, and, simultaneously, apply conditions of the
form (1.3) to elements of the parameter spaces $\Theta_{(n(i))}$, $i\geq 0$.
In the next Section we specify the above construction and Proposition (1) for
the case of double cochain complex (5.1)–(5.2) associated to differential
forms [2]. We derive explicitly the generators and commutation relations for
corresponding continual Lie algebras.

##  3\. An example: double complex associated with foliations

Recall the formulation of the Čech-de Rham cohomology given in [2] for
foliations on smooth manifolds (see Appendix). In this case the spaces in
(2.1) are differential forms $C^{n,m}(\mathcal{F})$ defined on a foliation,
and the coboundary operators is given by

$\delta^{p,q}=(-1)^{p}d+\delta^{q},$

where $d$ and $\delta$ operators are defined in of Appendix. The ordinary
product for differential forms $\omega^{n,m}(h_{1},\ldots,h_{m})\in
C^{n,m}\left(\mathcal{F}\right)$ is given by (5.3). The product (2.2) required
for the formulation of Section 2 is provided by the commutator (2.18). As it
was mentioned in [2], the bicomplex (5.1)–(5.2) possess the structure of a
differential bigraded algebra with respect to the ordinary product (5.3).
According to explanations of Section 2, for the bicomplex (5.1)–(5.2) generate
the exterior differential graded algebra with relations (2.2) with respect to
the product (2.18).

###  3.1. General case

Let us recall, that we assume in this case non-negative indexes for all
bicomples spaces $C^{n_{i},m_{i}}$. Let $\chi\in C^{n_{0},m_{0}}$, $\Phi\in
C^{n,m}$. The orthogonality condition (2.6) with respect ot the product (2.18)
leads to systems of the form (2.2) when applied to the double complex (2.1).
In particular, for $\chi\in C^{n_{0},m_{0}}$ and $\Phi\in C^{n_{0},m_{0}}$, we
obtain the system (2.2) of relations for elements $\chi$,
$\delta^{n_{0},m_{0}}\chi$, $\Phi$, $\delta^{n,m}\Phi$,
$\alpha_{i}^{(K_{i})}\in C^{n_{i}^{(K_{i})},m_{i}^{(K_{i})}}$,
$\delta^{n(i),m(i)}\alpha_{i}^{(K_{i})}$, $i\geq 1$, for $n\geq 0$, $m\geq 0$.
Let $r^{(K_{i})}_{i}$ and $t^{(K_{i})}_{i}$ be numbers of common degrees and
transversal sections of $C^{n_{i},m_{i}}$ for the forms $\chi\in
C^{n_{0},m_{0}}$ and $\Phi\in C^{n_{0},m_{0}}$. Then the compatibility
conditions (2.9)–(2.12) and (2.16)–(2.17) for indexes $n$, $m$, $n_{0}$,
$m_{0}$, $n_{i}^{(K_{i})}$, $m_{i}^{(K_{i})}$, satisfy the relations in vector
form:

$\displaystyle(n_{0}+1,m_{0}+1)$ $\displaystyle=$
$\displaystyle(n,m)+\left(n_{1}^{(R)},m_{1}^{(R)}\right)-\left(r_{1}^{(R)},t_{1}^{(R)}\right),$
(3.1)

for $(2)$ in (2.2):

$\displaystyle(n,m)$ $\displaystyle=$
$\displaystyle\left(n^{(L)}_{1},m^{(L)}_{1}\right)+(n_{0}+1,m_{0}+1)-\left(r^{(L)}_{1},t^{(L)}_{1}\right),$
(3.2)

For the sequence starting from $(4)$ in (2.2) we have:

$\displaystyle(n_{0},m_{0})=(n,m)+\left(n_{i}^{(RRK_{i})},m_{j}^{(RRK_{j})}\right)-\left(r_{i}^{(RRK_{i})},t_{j}^{(RRK_{j})}\right),$
(3.3)

for the sequence starting from $(5)$ in (2.2):

$\displaystyle(n,m)=(n_{0},m_{0})+\left(n_{i}^{(LLK_{i})},m_{j}^{(LLK_{j})}\right)-\left(r_{i}^{(LLK_{i})},t_{j}^{(LLK_{j})}\right).$
(3.4)

$i$, $j\geq 2$. Note that it is assumed that all resulting indexes
compatibility conditions are non-negative. For the general complex (2.1),
$i\in\mathbb{Z}$. For the bicomplex (5.1)–(5.2) we have $p$, $q$ are non-
negative. Since we assume that $0\leq r^{(LLK_{i})}_{i}\leq
n^{(LLK_{i})}_{i}$, and $0\leq t^{(LLK_{i})}\leq m^{(LLK_{i})}_{i}$, from the
compatibility conditiions (3.1)–(3.2) and (3.3)–(3.4) we see that, depending
on the signs of $n-n_{0}-\delta_{i,1}$, $m-m_{0}-\delta_{i,1}$, and only one
branch of systems of the form $(2)$–$(2^{\prime})$ exists at each vertex of
the three graph associated to the double complex (5.1)–(5.2).

Recall that, according to Theorem 1 of [2], the definition of the spaces
$C^{n,m}$ do not depend on the choice of the transversal basis $\mathcal{U}$,
still it depends on parameters of foliation $\mathcal{F}$. Nevetheless,
differential forms $\omega^{n,m}$ do depend on holonomy mappings (see
Appendix) $h_{j}$, $j\geq 0$, and play the role of extra parameters in the
consideration.

As explained at the end of Section 2, due to Proposition (1), a path
$(K_{i})$, $i\geq 1$, defining the generators
$\left\\{\chi,\;\delta^{n_{0},m_{0}}\chi,\;\Phi,\;\delta^{n,m}\Phi,\;\alpha_{i}^{(K_{i})},\;\delta^{n(i),m(i)}\alpha_{i}^{(K_{i})}\right\\}$,
$n(i)=n_{i}^{(K_{i})}$, $m(i)=m_{i}^{(K_{j})}$, and relations for independent
elements of $C^{n,m}$ in (2.2), form a continual Lie algebra
$\mathcal{G}(\mathcal{F})$ with the space of roots provided by the holonomy
mappings and of parameters of the foliation $\mathcal{F}$.

###  3.2. Double cochain complex: Godbillon-Vey type example

In this subsection we provide the explicit example for Proposition (1) in the
case of the orthogonality condition (2.6) applied to the particular case when,
in the consideration of the precious subsection, $\Phi=\chi$. In particular,
in differential geometry, the case of a foliation $\mathcal{F}$ of codimension
one defined by a one-form on a three-dimensional manifold, and the formulation
of the Gadbillon-Vey cohomology class, are included in this consideration.

We require the orthogonality for $\chi\in C^{n,m}(\mathcal{F})$ within its own
bicomplex space, i.e., to satisfy the condition

$\displaystyle\chi\cdot\delta^{n,m}\chi=0.$ (3.5)

Thus, for $\alpha_{1}^{(R)}\in C^{n^{\prime},m^{\prime}}(\mathcal{F})$ one has

$\displaystyle\delta^{n,m}\chi=\chi\cdot\alpha_{1}^{(R)},$ (3.6)

and $n+1=n+n^{\prime}-r$, $m+1=m+m^{\prime}-t$, and (3.6) is possible only
when $n^{\prime}=r+1$, $m^{\prime}=t+1$, $0\leq r\leq n$, $0\leq t\leq n$, and
$\alpha_{1}^{(R)}\in C^{r+1,t+1}(\mathcal{F})$. If we requite from (3.5) that
for $\alpha_{1}^{(L)}\in C^{\alpha,\beta}(\mathcal{F})$,

$\chi=\alpha_{1}^{(L)}\cdot\delta^{n,m}\chi,$

then $n=\alpha+n+1-r^{\prime}$, $\alpha=r^{\prime}-1$, i.e., $\alpha$ is
smaller than the common degree which is not possible and thus such
$\alpha_{1}^{(L)}$ does not exists.

Then, as a result of (2.2), we obtain the system of differential relations:

$\displaystyle 0$ $\displaystyle=$
$\displaystyle\chi\cdot\delta^{n,m}\chi,\qquad(1),$ $\displaystyle 0$
$\displaystyle=$
$\displaystyle\left(\delta^{n,m}\chi\right)\cdot\delta^{n,m}\chi,\qquad(3)$
$\displaystyle\delta^{n,m}\chi$ $\displaystyle=$
$\displaystyle\chi\cdot\alpha^{(R)}_{1}.$ $\displaystyle 0$ $\displaystyle=$
$\displaystyle\delta^{n,m}\chi\cdot\alpha^{(R)}_{1}+(-1)^{n}\chi\cdot\delta^{r+1,t+1}\alpha^{(R)}_{1},$
(3.7)

and the rest of the system (2.2) collapses since further its branches follows
from $(3)$ which is trivial.

Let us denote ${\bf h}_{n}=\left(h_{1},\ldots,h_{n}\right)$, an $n$-tuple of
holonomy mappings (see Appendix). In this case, for forms

$\left\\{\chi,\;\delta^{n,m}\chi,\;\alpha^{(R)}_{1},\;\delta^{r+1,t+1}\alpha_{1}^{(L)}\right\\},$

we obtain the following continual Lie algebra by identifying the differential
forms with generators of $\mathcal{G}(\mathcal{F})$ as

$\displaystyle X_{+}$ $\displaystyle=$ $\displaystyle\chi,$ $\displaystyle
X_{-}$ $\displaystyle=$ $\displaystyle\delta^{n,m}\chi,$ $\displaystyle H$
$\displaystyle=$ $\displaystyle\alpha^{(R)}_{1},$ $\displaystyle H^{*}$
$\displaystyle=$ $\displaystyle\delta^{r+1,t+1}\alpha^{(R)}_{1},$

and the commutation relations (in a non-principal grading):

$\displaystyle\left[X_{+}({\bf h}_{n}),X_{-}({\bf h^{\prime}}_{n})\right]$
$\displaystyle=$ $\displaystyle 0,$ $\displaystyle\left[X_{+}({\bf
h}_{n}),H({\bf h}_{r+1})\right]$ $\displaystyle=$ $\displaystyle
X_{-}\left(K_{+1,0}\left({\bf h}_{n},{\bf h}_{r+1}\right)\right),$
$\displaystyle\left[X_{-}({\bf h}_{n+1}),H({\bf h}_{r+1})\right]$
$\displaystyle+$ $\displaystyle(-1)^{n}\left[X_{+}({\bf h}_{n}),H^{*}({\bf
h}_{r+2})\right]=0.$ (3.8)

Note that an element of $C^{n,m}(\mathcal{F})$ is an $n$-form
$\omega(h_{1},\ldots,h_{n})$ depending on $n$ holonomy maps. Thus the space of
continual roots is provided by the space of holonomy embeddings (see
Appendix). Taking into account (3.2), we find the kernels

$\displaystyle K_{+1,-1}\left({\bf h}_{n},{\bf h}_{r+1}\right)$
$\displaystyle=$ $\displaystyle 0,$ $\displaystyle K_{+1,0}\left({\bf
h}_{n},{\bf h}_{r+1}\right)$ $\displaystyle=$ $\displaystyle{\bf h}_{n+1},$
$\displaystyle K_{0,-1}\left({\bf h}_{n},{\bf h}_{r+1}\right)$
$\displaystyle=$ $\displaystyle{\bf h}_{n+2},$ $\displaystyle
K_{0,1}\left({\bf h}_{n},{\bf h}_{r+1}\right)$ $\displaystyle=$
$\displaystyle{\bf h}_{n+2}.$

It is easy to check that Jacobi identities for generators (3.2) are fulfilled.

In the case of codimension one foliation defines by a one-form, $\chi$,
$\alpha^{(R)}_{1}$, $\chi\in C^{1,m}(\mathcal{F})$. Recall, [7], that the
Godbillon-Vey cohomology class is given by
$\left[\alpha^{(R)}_{1}\wedge\delta^{1,m}\alpha^{(R)}_{1}\right]$. The
construction above clarifies the Lie-algebraic meaning of this cohomology
class.

##  4\. Conclusion

In conclusion, we would like to mention a few directions of development and
further applications of the material presented in this paper. We propose a way
how to associate a continual Lie algebra to a chain complex. Thus the
properties, in particular, Jacobi identities, kernels, and relations of
resulting continual Lie algebras depend on the kind of chain complex spaces as
well as on set of their parameters. One can think of introducing various types
of products suitable for the construction of more complicated than (2.2)
systems of differential relations. In our particular case (Section 2), in
order to make connection with continual Lie algebras, we have chosen the
commutator (2.18) (with respect to the original product defined in bicomplex
spaces) as the simplest natural product. One could think of other
possibilities which would be coherent with the orthogonality condition (2.6).

In our exposition, the standard form of chain complexes was involved.
Nevertheless, one can consider more complicated setups, in particular,
complexes where differentials act in non-trivial ways with respect to indexes
of spaces (c.f. examples in [8]). That would lead to alternative forms of
systems of differential relations as well as compatibility conditions. What
could be especially interesting, is to treat multiple chain complexes
containing combinations of few chain-cochains.

The example of the Čhech-de Rham cohomology that we study in Section 3 comes
from the differential geometry of foliations. In classics, the orthogonality
condition applied to elements and their differentials of one particular
bicomplex space, boiles down to the integrability condition and leads to the
Frobenius theorem. Then it delivers the Godbillon-Vey cohomological class
whose geometric meaning is not studied completely yet [6]. As for further
applications in differential geometry, starting from the orthogonality
condition, it would be interesting to find other cohomological invariants
related to the Čech-de Rham bicomplex for foliations, and to understand their
geometric meaning. The constructions considered in this paper can be also used
for the cohomology theory of smooth manifolds, in particular, in various
approaches to the construction of cohomological classes (cf., in partiucar,
[9]).

Wide applications are awating new examples of continual Lie algebras in the
field of integrable models [11, 10]. The cases of non-commutative fields used
to define continual Lie algebras would also be useful for in non-commutative
geometry.

## Acknowledgments

The author would like to thank A. Galaev, Y.-Zh. Huang, H. V. Lê, P. Somberg,
and P. Zusmanovich for related discussions. Research of the author was
supported by the GACR project 18-00496S and RVO: 67985840.

##  5\. Appendix: Čech-de Rham complex for foliations

In this Appendix we recall [2] the notion of the basis of transversal
sections, and Čech-de Rham complex for a foliation.

Let $M$ be a manifold of dimension $n$, equipped with a foliation
$\mathcal{F}$ of co-dimension $l$ [2]. A transversal section of $\mathcal{F}$
is an embedded $l$-dimensional submanifold $U\subset M$ which is everywhere
transverse to the leaves. If $\alpha$ is a path between two points $x$ and $y$
on the same leaf, and if $U$ and $V$ are transversal sections through $x$ and
$y$, then $\alpha$ defines a transport along the leaves from a neighborhood of
$x$ in $U$ to a neighborhood of $y$ in $V$, hence a germ of a diffeomorphism
$hol(\alpha):(U,x)\longrightarrow(V,y)$, called the holonomy of the path
$\alpha$. Two homotopic paths always define the same holonomy.

If such a transport is defined in all of $U$ and embeds $U$ into $V$, this
embedding $h:U\hookrightarrow V$ is sometimes also denoted by
$hol(\alpha):U\hookrightarrow V$. Embeddings of this form will be called
holonomy embeddings.

Transversal sections $U$ through a point $x$ are neighborhoods of the leaf
through $x$ in the leaf space. One defines a transversal basis for
$(M,\mathcal{F})$ as a family $\mathcal{U}$ of transversal sections $U\subset
M$ with the property that, if $V$ is any transversal section through a given
point $y\in M$, there exists a holonomy embedding $h:U\hookrightarrow V$ with
$U\in\mathcal{U}$ and $y\in h(U)$. A transversal section is a $l$-disk given
by a chart for the foliation. One then constructs a transversal basis
$\mathcal{U}$ out of a basis $\tilde{\mathcal{U}}$ of $M$ by domains of
foliation charts
$\phi_{U}:\tilde{U}\tilde{\longrightarrow}\mathbb{R}^{n-l}\times U$,
$\tilde{U}\in\tilde{\mathcal{U}}$, with $U=\mathbb{R}^{l}$. Note that each
inclusion $\tilde{U}\hookrightarrow\tilde{V}$ between opens of
$\tilde{\mathcal{U}}$ induces a holonomy embedding $h_{U,V}:U\longrightarrow
V$ defined by the condition that the plaque in $\tilde{U}$ through $x$ is
contained in the plaque in $\tilde{V}$ through $h_{U,V}(x)$.

Let us recall the construction of the Čech-de Rham cohomology in [2]. Let
$\mathcal{U}$ be a family of transversal sections of $\mathcal{F}$. Such a
family is called a transversal basis if for each transversal section $V$ of
$\mathcal{F}$ and each point $y\in V$, there exists a section
$U\in\mathcal{U}$ and a holonomy embedding $h:U\to V$ such that $y\in h(U)$.

Consider the double complex

$C^{p,q}(\mathcal{F})=\prod_{U_{0}\stackrel{{\scriptstyle
h_{1}}}{{\longrightarrow}}\cdots\stackrel{{\scriptstyle
h_{p}}}{{\longrightarrow}}U_{p}}\Omega^{q}(U_{0}),$ (5.1)

where the product ranges over all $p$-tuples of holonomy embeddings $h_{i}$,
$0\leq i\leq p$, between transversal sections from a fixed transversal basis
$\mathcal{U}$, and $\Omega^{q}$ is the space of differential forms of order
$q$. The vertical differential is defined as

$(-1)^{p}d:C^{p,q}(\mathcal{F})\to C^{p,q+1}(\mathcal{F}),$

where $d$ is the usual de Rham differential. The horizontal differential

$\delta:C^{p,q}(\mathcal{F})\to C^{p+1,q}(\mathcal{F}),$

is given by

$\delta=\sum\limits_{i=0}^{k+1}(-1)^{i}\delta_{i},$

where

$\delta_{i}\omega(h_{1},\ldots,h_{k+1})=\left\\{\begin{array}[]{lll}h_{1}^{*}\omega(h_{2},\ldots,h_{k+1})\
\ \mbox{if $i=0,$}\\\ \omega(h_{1},\ldots,h_{i+1}h_{i},\ldots,h_{k+1})\ \
\mbox{if $0<i<k+1,$}\\\ \omega(h_{1},\ldots,h_{k})\ \ \mbox{if
$i=k+1.$}\end{array}\right.$ (5.2)

The usual product

$(\omega\eta)\left(h_{1},\ldots,h_{n+n^{\prime}}\right)=(-1)^{nn^{\prime}}\omega(h_{1},\ldots,h_{n})\;\left(h_{1}^{*}\ldots
h_{n}^{*}\right).\eta\left(h_{n+1},\ldots h_{n+n^{\prime}}\right),$ (5.3)

for $\omega\in C^{n,m}$ and $\eta\in C^{n^{\prime},m^{\prime}}$ and
$h^{*}_{i}$ being the dual to $h_{i}$ Thus
$(\omega\;\eta)(h_{1},\ldots,h_{n+n^{\prime}})\in
C^{n+n^{\prime},m+m^{\prime}}$, and the product (5.3) delivers the structure
of a bigraded differential algebra.

The cohomology of this complex is called the Čech-de Rham cohomology
${\check{H}}^{*}_{\mathcal{U}}(M/\mathcal{F})$ of the leaf space
$M/\mathcal{F}$ with respect to the transversal basis $\mathcal{U}$.

## References

  * [1] Bakas, I., Ricci flows and infinite dimensional algebras. Proceedings of the 36th International Symposium Ahrenshoop on the Theory of Elementary Particles. Fortschr. Phys. 52 (2004), no. 6-7, 464?471.
  * [2] M. Crainic and I. Moerdijk, Čech-De Rham theory for leaf spaces of foliations. Math. Ann. 328 (2004), no. 1–2, 59–85.
  * [3] Devchand, Ch., Saveliev, M. V. Comultiplication for quantum deformations of the centreless Virasoro algebra in the continuum formulation. Phys. Lett. B 258 (1991), no. 3-4, 364–368.
  * [4] Dubrovin, B. A.; Novikov, S. P.; Fomenko, Modern geometry. Methods and applications. ”Nauka”, Moscow, 1979. 760 pp.
  * [5] Finley, D., McIver, J. K., Solutions of the sDiff(2) Toda equation with SU(2) symmetry. Classical Quantum Gravity 27 (2010), no. 14, 145001, 21 pp.
  * [6] Galaev A. S. Comparison of approaches to characteristic classes of foliations, arXiv:1709.05888
  * [7] Ghys E. L’invariant de Godbillon-Vey. Seminaire Bourbaki, 41–eme annee, n 706, S. M. F. Asterisque 177–178 (1989)
  * [8] Y.-Zh. Huang A cohomology theory of grading-restricted vertex algebras. Comm. Math. Phys. 327 (2014), no. 1, 279–307.
  * [9] Losik, M. V.: Orbit spaces and leaf spaces of foliations as generalized manifolds, arXiv:1501.04993.
  * [10] Razumov, A. V.; Saveliev, M. V. Lie groups, differential geometry, and nonlinear integrable systems. Nonassociative algebra and its applications (San Paulo, 1998), 321–336, Lecture Notes in Pure and Appl. Math., 211, Dekker, New York, 2000.
  * [11] Leznov, A. N.; Saveliev, M. V. Group methods for the integration of nonlinear dynamical systems, (Russian), ”Nauka”, Moscow, 1985. 280 pp.
  * [12] M. V. Saveliev, Integro-differential nonlinear equations and continual Lie algebras. Comm. Math. Phys. 121 (1989), no. 2, 283–290.
  * [13] M. V. Saveliev, On an Equation Associated with the Contact Lie Algebras, LPTENS–93/43, arXiv:hep-th/9311035.
  * [14] M. V. Saveliev, A. M. Vershik. Continuum analogues of contragredient Lie algebras. Commun. Math. Phys. 126, 367, 1989;
  * [15] M. V. Saveliev, A. M. Vershik. New examples of continuum graded Lie algebras. Phys. Lett. A, 143, 121, 1990.
  * [16] M. V. Saveliev, A. M. Vershik. New examples of continuum graded Lie algebras. Proceedings of the International Conference on Algebra, Part 2 (Novosibirsk, 1989), 123–133, Contemp. Math., 131, Part 2, Amer. Math. Soc., Providence, RI, 1992.
  * [17] Vershik, A. Lie algebras generated by dynamical systems. Algebra i Analiz 4 (1992), no. 6, 103?113; reprinted in St. Petersburg Math. J. 4 (1993), no. 6, 1143–1151.


# Weak Alignment Supervision from Hybrid Model
Improves End-to-End ASR

###### Abstract

In this paper, we aim to create weak alignment supervision from an existing
hybrid system to aid the end-to-end modeling of automatic speech recognition.
Towards this end, we use the existing hybrid ASR system to produce triphone
alignments of the training audios. We then create a cross-entropy loss at a
certain layer of the encoder using the derived alignments. In contrast to the
general one-hot cross-entropy losses, here we use a cross-entropy loss with a
label smoothing parameter to regularize the supervision. As a comparison, we
also conduct the experiments with one-hot cross-entropy losses and CTC losses
with loss weighting. The results show that placing the weak alignment
supervision with the label smoothing parameter of 0.5 at the third encoder
layer outperforms the other two approaches and leads to about 5% relative WER
reduction on the TED-LIUM 2 dataset over the baseline. We see similar
improvements when applying the method out-of-the-box on a Tagalog end-to-end
ASR system.

Index Terms—  weak alignment supervision, end-to-end, hybrid ASR, label
smoothing, loss weighting

##  1 Introduction

In recent years, end-to-end (E2E) automatic speech recognition (ASR) modeling
has gained tremendous successes [1, 2, 3, 4]. Compared to hybrid modeling, E2E
modeling does not need expert language knowledge and the cumbersome HMM
training [5]. The major E2E architectures are Attention-based Encoder Decoder
(AED) [6, 7], Connectionist Temporal Classification (CTC) [8, 9, 10, 11], and
Recurrent Neural Networks-Transducer (RNN-T) [12, 13, 14, 15, 16]. E2E
modeling can easily be combined with various algorithms/models/processes, such
as attention [17], self-supervised learning (SSL) [18, 19], pretraining [7],
labeling smoothing [20], multi-task learning [11, 21, 22, 23, 24, 25, 26], and
data augmentation [19, 27, 28, 29].

Due to the simplicity of E2E modeling, researchers have been moving away from
hybrid ASR modeling. However, in cases where a hybrid system is available, why
do we throw away the hybrid ASR model instead of making use of it for the E2E
modeling? In this paper, we aim at utilizing the existing hybrid ASR models to
aid the E2E modeling [15, 25, 30]. Towards this end, we use an existing hybrid
ASR system to align the training audios to derive the frame-by-frame
triphone/senones alignment. Then we use the alignments to create an auxiliary
task with frame-wise cross-entropy (CE) loss function. This basic idea has
been explored in the past [15, 25, 30]. However, in this work, we propose a
novel approach to combine the multi-task learning and label smoothing [20,
31]. We think the alignment supervision should not overwhelm the primary ASR
objective and a weak supervision should be beneficial. In multi-task learning,
usually CTC or one-hot CE losses with loss weighting are used. Here, we
propose to use a weak CE loss through adjusting the label smoothing parameter.
That is, we use a label smoothing parameter of 0.5 to create a weak
supervision that produces robust and consistent Word Error Rate (WER)
improvements.

##  2 Related Work

Multi-task learning has been widely used in E2E modeling [11, 21, 22, 23, 24,
25, 32], and some researchers have tried using the state/triphone/senones
alignments from hybrid ASR models for the auxiliary tasks [15, 25, 30].

In [30], The authors use lower-level tasks, such as state or phoneme
recognition, in a multi-task training approach with an AED model for direct
character transcription. The AED encoder has four Bidirectional Long Short-
Term Memory (BLSTM) layers with time-domain sub-sampling [33, 34]. They
experiment with different auxiliary tasks: a phoneme decoder loss, a phoneme
CTC loss, and a frame-wise state-level CE loss. For the phoneme decoder and
phoneme CTC losses, the phoneme sequences are produced by composing the
transcripts with the pronunciation dictionary. This approach has a limitation
in the sense that there are usually pronunciation variants for many of the
English words. For the frame-wise state-level CE loss, the authors obtain
state-level alignments from a Kaldi HMM/DNN ASR system. Their results on
Switchboard and CallHome show consistent improvements over baseline E2E
models, obtaining the best performance with a combination of a phoneme decoder
at Layer 3 and a frame-level state loss at Layer 2.

In [15], the authors attempt using two auxiliary tasks to improve their
streaming RNN-T models. For one auxiliary task, they use an auxiliary RNN-T
task at an intermediate encoder layer and exploit a symmetric Kullback–Leibler
divergence loss between the output posterior distributions of primary and
auxiliary tasks. Similar to [30], the authors also use state/triphone/senones
alignments derived from existing hybrid ASR models. The RNN-T encoder has five
latency-controlled BLSTM layers with time-domain sub-sampling after the first
and second layers. The primary task uses wordpieces as targets. The main
difference from [30] is that [15] has a linear+rectified layer before the CE
loss, tunes the weights of the auxiliary tasks, and applies them at the third
and fifth layers. [15] shows that the best results are obtained with loss
weighting of 0.6.

The paper [7] is of great interest here as its authors share a full recipe for
replicating their AED modeling experiments and their TensorFlow based deep
learning platform RETURNN [35] at GitHub. Besides, the paper describes all the
components needed for an E2E ASR modeling: AED, RNN-based decoder, dropout
[36], time-domain sub-sampling, Byte Pair Encoding (BPE) [37], CTC-based
multi-task learning, label smoothing, pretraining, feedback/fertility
modeling, learning rate control, BLSTM/Transformer based encoder layers,
SpecAugment, Scheduled Sampling [38], etc.

##  3 Methodology

Auxiliary supervision with loss weighting has been widely used in the
literature [10, 11, 15, 23, 30] to regularize the strength of the auxiliary
tasks.

$L_{\text{all}}=L_{\text{ASR}}+\lambda L_{\text{AUX}}$ (1)

Here, we propose to use the label smoothing to modulate the strength of the
auxiliary tasks.

$L_{\text{all}}=L_{\text{ASR}}+L_{\text{AUX}}(m_{\text{labelsmoothing}})$ (2)

For each frame, the auxiliary loss with label smoothing can be further
expressed as [17, 20]:

$\begin{split}&L_{\text{AUX}}(m)=\\\
&-\sum_{v=1}^{V}{\left((1-m)p_{v}+\frac{m}{V-1}(1-p_{v})\right)\log{q_{v}}}\\\
&=(1-m)L_{\text{CE}}+mL_{\text{smooth}}\end{split}$ (3)

where v is a running index in the target vocabulary V, $m$ is the hyper-
parameter (between 0 and 1) that controls the amount of probability mass to
discount, $p_{v}$ is the one-hot true target distribution and $q_{v}$ is the
output distribution of the model.

That is, instead of explicitly weighting the auxiliary losses using $\lambda$
in Equation 1, we implicitly adjust the auxiliary loss strengths through label
smoothing using $m$ in Equations 2 and 3, with the hypothesis that higher
label smoothing parameters produce weaker supervision [20]. Interestingly, the
label smoothing loss can be considered as a weighted combination of the one-
hot CE loss ($L_{\text{CE}}$) and the uniform CE loss ($L_{\text{smooth}}$).
Here, the latter refers to that the target token’s true probability is set to
0, while non-target tokens equally share the probability of 1. Therefore, we
can use the label smoothing parameter $m$ to control the balance between the
discrimination loss ($L_{\text{CE}}$) and the smoothing loss
($L_{\text{smooth}}$).

We use the recipe from [7] with a BLSTM encoder as our baseline AED ASR
system. Figure 1 shows the diagram of the original networks and the proposed
frame-wise CE loss with the phoneme alignments.

Fig. 1: Diagram of the network.

MFCC features extracted from audio files are sent to SpecAugment [28] and then
to the six BLSTM layers. At the first and second layers, time-domain sub-
sampling is applied. The output from the sixth BLSTM goes to a linear encoder
that connects to an existing auxiliary CTC loss with BPEs as targets, and the
encoder output also connects to the RNN decoder. Note that the CTC loss output
is not used in decoding. Inside the decoder, there are multi-head attention
computation, feedback/fertility modeling, decoder LSTM, and a CE loss with the
BPEs as targets. The dotted lines and boxes indicate the proposed CE loss with
triphone alignments and label smoothing. The proposed auxiliary task can be
put after BLSTM Layers 1, 2, 3, 4, 5, and the encoder output layer. For the
label smoothing parameter, we test different values of 0.0, 0.1, 0.3, 0.5,
0.7, 0.9, and 1.0. As in [7], pretraining is used to help convergence. As a
comparison, the CE loss with triphone alignments and loss weighting is applied
at different layers and with different weights of 0.05, 0.1, 0.2, 0.5, and
1.0. To examine whether there is any benefit of introducing triphone
alignments, we also experiment with the usual auxiliary CTC loss task with
loss weighting.

##  4 Experimental Results

###  4.1 Setup

The experiments are performed on the TED-LIUM release 2 (200 hours) [39] using
RETURNN [35]. We follow a recipe from [7]
111https://github.com/rwth-i6/returnn-
experiments/tree/master/2019-asr-e2e-trafo-vs-lstm/tedlium2 with a specific
config file 222base2.specaug.bs18k.curric3.pfixedlr.lrd07.eos.config. In the
repository, a full setup is provided. For this dataset, we use the same
train/dev/test division as in [7]. The RETURNN platform builds on TensorFlow
[40]. Note that in this study, language models are not applied. We acknowledge
that stronger baselines are possible, e.g. with Conformer/E-Branchformer [41,
42] and wave2vec 2.0 [18], however, we think this baseline is sufficient for
the purpose of this work.

As in [7], we limit the training time by a fixed number of epochs (37.5) over
the training dataset in all experiments. (There are 150 sub-epochs in total.)
The training takes about 32 hours on a single NVIDIA 1080 Ti GPU.

MFCC features are extracted with a 25-ms window, a 10-ms step size, and 40
Mel-filters. Before training, the mean and standard deviation statistics for
training data are estimated and thus used in pre-processing for feature
normalization.

Triphone alignments are obtained using an existing hybrid BLSTM ASR model that
uses 5000 triphones, 80-dimension MFCC as acoustic features, and four
1024-dimension hidden BLSTM layers. The hybrid model was trained using about
11,000 hours of training data. The TED-LIUM release 2 data were included in
the training. Traditional Gaussian Mixture Models (GMM) was trained to obtain
the triphone CART tree and then the BLSTM model was trained with SpecAugment
using RETURNN [35]. The RASR toolkit [43] is used to perform the forward-
backward alignments.

As shown in Figure 1, the BLSTM layers have a hidden dimension of 1024. We use
dropout and Adam optimizer [44] with a variant of Newbob learning rate
scheduling. In the RNN decoder, the CE loss is applied with a label smoothing
of 0.1. During decoding, a beam size of 12 is used. WER is computed for the
decoder output (after merging BPEs into words) using NIST SCLITE Scoring
Toolkit [45].

###  4.2 Triphone CE losses with loss weighting

We first apply the triphone CE losses with loss weighting (0.05, 0.1, 0.2,
0.5, and 1.0) at different encoder BLSTM layers. Figure 2 shows WERs for the
dev dataset. When the loss weighting is 1.0, the best result is obtained when
placing the auxiliary task at the second BLSTM layer that is consistent with
results from [30]. When the loss weighting is 0.5, the best result is obtained
when placing the auxiliary task at the third BLSTM layer. Subsequently, when
the loss weights are 0.2, 0.1, and 0.05, the auxiliary task at the fourth
layer produces the best results. It is interesting to notice the trend here
that when the loss weighting is higher, the auxiliary task should be placed at
the lower encoder layers (e.g., the second), and when the loss weighting is
lower, the auxiliary task should be placed at the higher encoder layers (e.g.,
the fourth). Overall, a loss weighting of 0.5 at the third encoder BLSTM layer
gives the best results, and this is again consistent with [15] that uses a CE
loss task with loss weighting of 0.6 at the third and fifth layers. Therefore,
given that triphone target in the auxiliary task is different from the BPE
target in the primary task, the auxiliary task requires a weak supervision
that can be achieved either through loss weighting, being far away from the
primary task, or using the proposed label smoothing.

Fig. 2: WERs for the dev dataset with different loss weights and at different
encoder layers. L1, L2, L3, L3, L4, L5, and L6 refer to the BLSTM layers 1, 2,
3, 4, 5, and 6, respectively. E refers to the encoder output layer. The green
horizontal line indicates the baseline.

We run the same tests on the test dataset and the results are shown in Figure
3. When the loss weighting is 1.0, the best result is obtained when placing
the triphone CE loss at the second layer that is consistent with results for
the dev dataset. However, for the test dataset, the triphone CE loss with a
loss weighting of 1.0 at the second layer does not improve over the baseline.
Similarly, the loss weighting of 0.5 CE loss at the third layer does not
improve over the baseline for the test dataset. However, with the loss weights
of 0.05 and 0.2, the auxiliary tasks at the fourth layer produce the best
results (better than the baseline) for the test data. For these specific cases
(loss weights of 0.05 and 0.2), the results for the dev dataset are consistent
with those for the dev dataset. The results are summarized in Table 1.
Overall, the performance patterns from the loss weighting and triphone
auxiliary task placement interaction differ across the dev and test datasets,
indicating that the loss weighting is not a robust approach to regularize the
triphone CE losses.

Fig. 3: WERs for the test dataset. Note that this figure is shown for information completeness only and the loss weighting and layer selection are not based on it. Loss Weighting |  Best CE layer (based on dev) | WER
---|---|---
dev | test
Baseline | - | 13.7 | 10.7
0.05 | 4 | 12.6 | 10.5
0.1 | 4 | 12.5 | 10.5
0.2 | 4 | 12.3 | 10.6
0.5 | 3 | 12.0 | 10.7
1.0 | 2 | 12.5 | 10.7
Table 1: CE loss weighting results

###  4.3 CTC losses with loss weighting

We use the BPEs that are already prepared for the primary task to create the
proposed auxiliary CTC task. Note that in Figure 1, an auxiliary CTC loss
without loss weighting at the encoder output layer exists already. Therefore,
when testing auxiliary CTC tasks, we only place it at the encoder BLSTM layers
1-5, and not on top of the whole encoder stack. The decoding results are shown
in Figures 4 and 5.

Figure 4 shows that overall CTC losses should be placed at the higher encoder
BLSTM layers, compared to those in Figure 2. Specifically, when the loss
weights are 0.05, 0.1, 0.2, 0.5, and 1.0, the corresponding best layer
placement are 3, 5, 4, 5, and 5, respectively.

Fig. 4: WERs for the dev dataset with different CTC loss weights and at different encoder layers. Fig. 5: WERs for the test dataset. Loss Weighting |  Best CTC layer (based on dev) | WER
---|---|---
dev | test
Baseline | - | 13.7 | 10.7
0.05 | 3 | 12.4 | 10.7
0.1 | 5 | 12.1 | 10.4
0.2 | 4 | 12.2 | 10.4
0.5 | 5 | 11.9 | 10.5
1.0 | 5 | 12.5 | 10.7
Table 2: CTC loss weighting results

Similarly to Section 4.2, we create a table to summarize the results from
Figures 4 and 5. Generally, the auxiliary CTC loss should be placed at higher
layers. Again, this is consistent with the observations in [30]. For lower
loss weights, the CTC loss should be placed at the lower BLSTM layers. This
pattern is different from that of CE loss in Section 4.2. Overall, the
performance patterns from the loss weighting and the CTC auxiliary task
placement interaction differ across the dev and test datasets, indicating that
the loss weighting is not a robust approach to regularize the BPE CTC losses.

Fig. 6: WERs (dev) with different $m$’s and at different layers. Fig. 7: WERs
(test) with different $m$’s and at different layers.

###  4.4 Triphone CE losses with label smoothing

In this experiment, we focus on the auxiliary CE loss at different encoder
layers with label smoothing. Here, we set the loss weighting at 1.0 given that
the label smoothing can be considered as another way to regularize the
strength of the auxiliary tasks [20]. We run the experiments with label
smoothing values of 0.0, 0.1, 0.3, 0.5, 0.7, 0.9, and 1.0. Results are shown
in Figures 6 and 7.

Figure 6 shows the WERs obtained for the dev dataset with different label
smoothing parameters at different encoder layers. Comparing to Figures 2 and
4, we have the following observations: (1) Label smoothing performs better
than loss weighting in terms of regularizing the auxiliary task losses; (2)
Label smoothing tends to perform the best in the middle layers (i.e., Layer 3
here); (3) There exists an optimal value of the label smoothing parameter,
which is 0.5; and (4) Label smoothing produces similar results across the dev
and test datasets, indicating that label smoothing is a robust and effective
way of regularizing auxiliary task losses. This supports our thinking that a
relatively weak and robust supervision should be preferred when augmenting the
training with auxiliary tasks. At the same time, the proposed approach is
different from others [30, 15] in the sense that it directly adjusts the
targets’ true probability to 0.5, a value that is optimal in balancing the
discriminative loss ($L_{\text{CE}}$) and the smoothing loss
($L_{\text{smooth}}$) (see Equation 3).

Label Smoothing |  Best CE layer (based on dev) | WER
---|---|---
dev | test
Baseline | - | 13.7 | 10.7
0.0 | 2 | 12.5 | 10.7
0.1 | 2 | 12.7 | 10.6
0.3 | 3 | 12.0 | 10.3
0.5 | 3 | 11.6 | 10.1
0.7 | 2 | 12.1 | 10.4
0.9 | 5 | 12.5 | 10.4
1.0 | 3 | 12.3 | 10.6
Table 3: Label smoothing results Loss Regularization |  Best paramemeters (based on dev) | WER
---|---|---
dev | test
Label Smoothing |  $m$=0.5, layer=3 | 11.6 | 10.1
CE LW | weight=0.5, layer=3 | 12.0 | 10.7
CTC LW | weight=0.5, layer=5 | 11.9 | 10.5
Table 4: System comparison (LW = Loss Weighting)

We summarize the results from Figures 6 and 7 into Table 3. Note the label
smoothing parameter of 0.0 is equal to the CE loss with a loss weight of 1.0
which serves as a sanity check. That is, the yellow lines in Figures 6 and 7
correspond to those in Figures 2 and 3, respectively. For the label smoothing
parameter of 1.0, according to Equation 3, it produces a minimally
discriminative loss objective where the smoothing loss ($L_{smooth}$)
overtakes the CE loss ($L_{CE}$). Nevertheless, such non-discriminative loss
objective still produces small improvement over baseline. Table 3 clearly
shows that the label smoothing parameter of 0.5 produces the overall best
results at Layer 3 with a relative 5.6% in WER reduction on the test dataset.
Interestingly, the label smoothing parameters of 0.3 and 0.7, the left and
right sides of neighbors of the optimal parameter of 0.5, produce the second-
to-the-best results at Layer 2.

Based on the results on the dev dataset, we select the best setups from Tables
1, 2, and 3 and put the corresponding results in Table 4. Obviously, the label
smoothing approach of loss regularization outperforms the other two.

###  4.5 Applying to the Tagalog ASR

Systems | WER
---|---
dev | test
Baseline | 20.6 | 25.1
Auxiliary CE loss (smooth=0.5) | 19.9 | 23.9
Relative improvement | 3.4% | 4.8%
Table 5: WERs for Tagalog ASR

We take the best configuration from Section 4.4, that is, an auxiliary CE loss
with label smoothing parameter of 0.5 at Layer 3, to our E2E Tagalog ASR
modeling. The data are divided into three subsets: train (232 hours), dev (22
hours), and test (11 hours). Our existing hybrid Tagalog ASR model was trained
with the same datasets. The hybrid model was trained with the same network
architecture as in Section 4.1 except that 4500 triphones were used here.
Then, we use the existing hybrid Tagalog ASR to align the training audios to
produce triphone alignments. We compute 1K BPE from the training transcripts.
After the E2E training, the recognition results are presented in Table 5. For
the test dataset, the proposed approach leads to 4.8% relatively in WER
reduction, confirming that the proposed weak triphone alignment supervision is
robust and effective in improving E2E ASR modeling.

##  5 Conclusion and Discussion

In this work, we propose to use label smoothing to construct a weak alignment
supervision to aid the E2E ASR modeling. We compare the proposed approach with
the CE loss with loss weighting [30, 15] and CTC loss with loss weighting
[15]. Our results show that the proposed weak alignment supervision with a
label smoothing of 0.5 produces consistent and robust results and outperforms
the other two approaches. When the proposed approach with the optimal
configuration is applied to model a Tagalog E2E ASR system, we observe similar
improvements in WER reduction. Thus, the contribution from this study is to
provide a different and novel perspective in designing weak alignment
supervision. Future work includes improving the baseline, applying the
approach to large datasets, and exploring with Conformer/Ebranchformer network
architectures [46, 41, 47, 42].

## References

  * [1] R. Prabhavalkar, T. Hori _et al._ , “End-to-end speech recognition: A survey,” _arXiv preprint arXiv:2303.03329_ , 2023.
  * [2] J. Li _et al._ , “Recent advances in end-to-end automatic speech recognition,” _APSIPA Transactions on Signal and Information Processing_ , vol. 11, no. 1, 2022.
  * [3] D. Bahdanau, J. Chorowski _et al._ , “End-to-end attention-based large vocabulary speech recognition,” in _ICASSP_. IEEE, 2016.
  * [4] L. Lu, X. Zhang, and S. Renais, “On training the recurrent neural network encoder-decoder for large vocabulary end-to-end speech recognition,” in _ICASSP_. IEEE, 2016.
  * [5] M. Gales, S. Young _et al._ , “The application of hidden markov models in speech recognition,” _Foundations and Trends in Signal Processing_ , vol. 1, no. 3, pp. 195–304, 2008.
  * [6] W. Chan, N. Jaitly _et al._ , “Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,” in _ICASSP_. IEEE, 2016.
  * [7] A. Zeyer, P. Bahar _et al._ , “A comparison of transformer and lstm encoder decoder models for asr,” in _ASRU_. IEEE, 2019.
  * [8] A. Graves, S. Fernández _et al._ , “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in _ICML_ , 2006.
  * [9] A. Graves and N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks,” in _ICML_. PMLR, 2014.
  * [10] J. Lee and S. Watanabe, “Intermediate loss regularization for ctc-based speech recognition,” in _ICASSP_. IEEE, 2021\.
  * [11] S. Kim, T. Hori, and S. Watanabe, “Joint ctc-attention based end-to-end speech recognition using multi-task learning,” in _ICASSP_. IEEE, 2017.
  * [12] A. Graves, “Sequence transduction with recurrent neural networks,” _arXiv preprint arXiv:1211.3711_ , 2012.
  * [13] K. Rao, H. Sak, and R. Prabhavalkar, “Exploring architectures, data and units for streaming end-to-end speech recognition with RNN-transducer,” in _ASRU_. IEEE, 2017.
  * [14] Y. He, T. N. Sainath _et al._ , “Streaming end-to-end speech recognition for mobile devices,” in _ICASSP_. IEEE, 2019.
  * [15] C. Liu, F. Zhang _et al._ , “Improving rnn transducer based asr with auxiliary tasks,” in _SLT_. IEEE, 2021.
  * [16] M. Jain, K. Schubert _et al._ , “RNN-T for latency controlled asr with improved beam search,” _arXiv preprint arXiv:1911.01629_ , 2019.
  * [17] A. Vaswani, N. Shazeer _et al._ , “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [18] A. Baevski, Y. Zhou _et al._ , “wav2vec 2.0: A framework for self-supervised learning of speech representations,” _Advances in neural information processing systems_ , vol. 33, pp. 12 449–12 460, 2020.
  * [19] M. Sang, H. Li _et al._ , “Self-supervised speaker verification with simple siamese network and self-supervised regularization,” in _ICASSP_. IEEE, 2022.
  * [20] Y. Gao, W. Wang _et al._ , “Towards a better understanding of label smoothing in neural machine translation,” in _1st Conference of the Asia-Pacific Chapter of the ACL and the 10th International Joint Conference on NLP_ , 2020.
  * [21] F. Boyer, Y. Shinohara _et al._ , “A study of transducer based end-to-end asr with espnet: Architecture, auxiliary loss and decoding strategies,” in _ASRU_. IEEE, 2021.
  * [22] M.-T. Luong, Q. V. Le _et al._ , “Multi-task sequence to sequence learning,” _arXiv preprint arXiv:1511.06114_ , 2015.
  * [23] M. Crawshaw, “Multi-task learning with deep neural networks: A survey,” _arXiv preprint arXiv:2009.09796_ , 2020.
  * [24] Y. Zhang and Q. Yang, “A survey on multi-task learning,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 34, no. 12, pp. 5586–5609, 2021.
  * [25] A. Søgaard and Y. Goldberg, “Deep multi-task learning with low level tasks supervised at lower layers,” in _ACL_ , 2016.
  * [26] Z. Yang, Y. Gao _et al._ , “Predicting and using target length in neural machine translation,” in _1st Conference of the Asia-Pacific Chapter of the ACL and the 10th International Joint Conference on NLP_ , 2020.
  * [27] X. Cui, V. Goel, and B. Kingsbury, “Data augmentation for deep neural network acoustic modeling,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 23, no. 9, pp. 1469–1477, 2015.
  * [28] D. S. Park, W. Chan _et al._ , “Specaugment: A simple data augmentation method for automatic speech recognition,” in _Interspeech_ , 2019.
  * [29] S. Ruder, “An overview of multi-task learning in deep neural networks,” _arXiv preprint arXiv:1706.05098_ , 2017.
  * [30] S. Toshniwal, H. Tang _et al._ , “Multitask learning with low-level auxiliary tasks for encoder-decoder based speech recognition,” _Interspeech_ , pp. 3532–3536, 2017.
  * [31] C. Szegedy, V. Vanhoucke _et al._ , “Rethinking the inception architecture for computer vision,” in _CVPR_ , 2016.
  * [32] W. Chen, B. Yan _et al._ , “Improving massively multilingual asr with auxiliary ctc objectives,” in _ICASSP_. IEEE, 2023.
  * [33] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [34] A. Graves, S. Fernández, and J. Schmidhuber, “Bidirectional LSTM networks for improved phoneme classification and recognition,” in _ICANN_ , 2005.
  * [35] A. Zeyer, T. Alkhouli, and H. Ney, “Returnn as a generic flexible neural toolkit with application to translation and speech recognition,” in _ACL_ , 2018.
  * [36] N. Srivastava, G. Hinton _et al._ , “Dropout: a simple way to prevent neural networks from overfitting,” _The journal of Machine Learning Research_ , vol. 15, no. 1, pp. 1929–1958, 2014.
  * [37] R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” _arXiv preprint arXiv:1508.07909_ , 2015.
  * [38] S. Bengio, O. Vinyals _et al._ , “Scheduled sampling for sequence prediction with recurrent neural networks,” _Advances in neural information processing systems_ , vol. 28, 2015.
  * [39] A. Rousseau, P. Deléglise _et al._ , “Enhancing the TED-LIUM corpus with selected data for language modeling and more TED talks,” in _LREC_ , 2014.
  * [40] M. Abadi, P. Barham _et al._ , “Tensorflow: A system for large-scale machine learning,” in _12th USENIX Symposium on Operating Systems Design and Implementation_ , 2016.
  * [41] K. Kim, F. Wu _et al._ , “E-branchformer: Branchformer with enhanced merging for speech recognition,” in _SLT_. IEEE, 2022.
  * [42] M. Zeineldeen, A. Zeyer _et al._ , “Chunked attention-based encoder-decoder model for streaming speech recognition,” _arXiv preprint arXiv:2309.08436_ , 2023.
  * [43] D. Rybach, S. Hahn _et al._ , “RASR-The RWTH Aachen University open source speech recognition toolkit,” in _ASRU_ , 2011.
  * [44] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [45] J. Fiscus, “SCTK, the NIST scoring toolkit,” _https://github.com/usnistgov/SCTK_ , 2021.
  * [46] A. Gulati, J. Qin _et al._ , “Conformer: Convolution-augmented transformer for speech recognition,” in _Interspeech_ , 2020.
  * [47] S. Majumdar, J. Balam _et al._ , “Citrinet: Closing the gap between non-autoregressive and autoregressive end-to-end models for automatic speech recognition,” _arXiv preprint arXiv:2104.01721_ , 2021.