Bijective and Automated Approaches to Abel Sums

Gil KALAI and Doron ZEILBERGER

Dedicated to Dominique Foata (b. Oct. 12, 1934), on his forthcoming 90th
birthday

Abstract: In this tribute to our guru, Dominique Foata, we return to one of
our (and Foata’s) first loves, and revisit Abel sums and their identities,
from two different viewpoints.

Preface

In the very first issue of Crelle’s journal (the first mathematical periodical
solely dedicated to mathematics), Niels Henrik Abel published a two-page paper
[A], stating and proving his eponymous identity. This lead to an intensive
study of general Abel Sums by many people, (see [R] [C] and their numerous
references), and to beautiful bijective approaches pioneered by Dominique
Foata and Aimé Fuchs [Fo][FoFu] that led to Francon’s elegant proof [Fr] (see
[C], p. 129). This tribute consists of two independent parts. The first part
is bijective, while the second part is automated, elaborating and extending
John Majewicz’s 1997 PhD thesis [M1][M2], and more important, fully
implementing it (and its extension) in a Maple package
https://sites.math.rutgers.edu/~zeilberg/tokhniot/AbelCeline.txt .

Part I: Bijective Proofs (à la Foata) of an Abel-Type identity and a
generalization

In [Ka] (see also [Ka’]), the first author proved (as a special case of more
general results) the following Abel-type identity. Let $n,p$ by nonnegative
integers, then

$\sum_{k=0}^{n}{{n}\choose{k}}k^{k}(n-k)^{n-k+p}=\sum_{k=0}^{n}{{n}\choose{k}}n^{k}(n-k)!S(p+n-k,n-k)\quad.$
$None$

Here, $S(n,k)$ are the Stirling numbers of the second kind, and $k!S(n,k)$ is
the number of maps from a set of size $k$ onto a set of size $n$. (This
property can serve as the definition of these numbers.)

A special case of this formula is attributed to Cauchy (Equation (24) in
Chapter 1 of [R]):

$\sum_{k=0}^{n}{{n}\choose{k}}k^{k}(n-k)^{n-k}=\sum_{k=0}^{n}{{n}\choose{k}}n^{k}(n-k)!\quad.$
$None$

Here we present a combinatorial proof, in the style of Foata [Fo][FoFu] for
Formulas $(2)$ and $(1)$. We use the notation $[n]=\\{1,2,\dots,n\\}$, and to
make the argument clearer we will present both proofs.

The Proofs

Proof of Equation $(2)$: The left hand side counts triples $f,A,B$ where $f$
is a function from [n] to [n] and the following conditions hold:

$A\cup B=[n],A\cap B=\emptyset,f(A)\subset A{\rm~{}and~{}}f(B)\subset B.$
$None$

Indeed, if $|A|=k$, there are ${{n}\choose{k}}$ ways to choose $A$ ($B$ is now
determined, $B=[n]\backslash A$), $k^{k}$ ways to choose the restriction of
$f$ to $A$ subject to the condition $f(A)\subset A$ and $(n-k)^{n-k}$ ways to
choose the restriction of $f$ to $B$ subject to the condition $f(B)\subset B$.

The right hand side counts triples $f,C,D$ where $f$ is a function from $[n]$
to $[n]$ and following conditions hold:

$C\cup D=[n],C\cap D=\emptyset,f(D)=D.$ $None$

(Here, $f(D)=D$ means that $f$ is a bijection from $D$ to $D$; Note that we
relaxed the condition for $C$ compared to $A$ and strengthened the condition
for $D$ compared to $B$.) Indeed, if $|C|=k$, there are ${{n}\choose{k}}$ ways
to choose $C$, $n^{k}$ ways to choose the restriction of $f$ to $C$ (no
conditions here) and $(n-k)!$ ways to choose the restriction of $f$ to $D$
subject to the condition $f(D)=D$.

The crucial observation is:

$\bullet$ For every function $f$ from $[n]$ to $[n]$ the number of pairs
$(A,B)$ that satisfies Equation $(3)$ equals the number of pairs $(C,D)$ that
satisfies equation $(4)$.

Indeed if $(A,B)$ is a pair that satisfies Equation $(3)$, we can take

$D=f(f(\cdots(f(B))\cdots){\rm~{}and~{}}C=f^{-1}(f^{-1}(\cdots(f^{-1}(A))\cdot).$
$None$

In other words $D=\\{d\in[n]:d=f^{k}(a){\rm~{}for~{}some~{}}a\in
B,{\rm~{}and~{}}k\geq 0\\}$ and $C=\\{c\in[n]:f^{k}(c)\in
B{\rm~{}for~{}some~{}}k\geq 0\\}$. Taking the inverse operation brings you
from a pair $(C,D)$ to a pair $(A,B)$

Proof of Equation $(1)$:

Let $X=[n]=\\{1,2,\dots.n\\}$, and $Y=[n+p]$ Consider all triples $(f,A,B)$
where, this time, $f$ maps $[n+p]$ to $[n]$, and $A,B$ satisfy

$A\cup B=[n],A\cap B=\emptyset,f(A)\subset
A{\rm~{}and~{}}f(B\cup[n+1,n+p])\subset B.$

We also consider triples $(f,C,D)$ where, $f$ maps $[n+p]$ to $[n]$, and $C,D$
satisfy

$C\cup D=[n],C\cap D=\emptyset,f(D\cup[n+1,n+p])=D.$ $None$

Also in this case a stronger statement holds: For every function
$f:[n+p]\to[n]$ there is a bijection between between pairs $(A,B)$ satisfying
equation $(6)$ and pairs $(C,D)$ satisfying equation $(7)$ and the bijection
is given again by $(5)$.

This bijection is implemented in the Maple package AbelBijection.txt,
available from

https://sites.math.rutgers.edu/~zeilberg/tokhniot/AbelBijection.txt .

See the front of this article
https://sites.math.rutgers.edu/~zeilberg/mamarim/mamarimhtml/abelKZ.html for a
sample input file and its corresponding output file.

Remark: If we give each element $k\in[n]$ a weight $w_{k}$ and a function $f$
from $[n+p]\to[n]$ the weight $w(f):=\prod\\{w_{f(k)}\,:\,k\in[n]\\}$ then our
bijective proof gives a Hurwitz-type generalization of Abel’s formula, see
Hurwitz [H] and Excercise 30 in Section 2.3.4.4. of Knuth [Kn]. (For Formula
$(1)$ we obtain a Hurwitz-type generalization of the Stirling numbers of the
second kind.)

Part II: Automating Abel Sums

Abel’s original identity had many proofs, but the one by Shalosh B. Ekhad and
John Majewicz [EM] (written $30$ years ago, and dedicated to Dominique Foata
on his $60^{th}$ birthday) has the distinction that it was computer-generated,
yet with a bit of patience, humanly readable and verifiable. In order to
motivate the sequel, let’s reproduce it in its entirety.

Abel’s identity: For any non-negative integer, $n$

$\sum_{k=0}^{n}{{n}\choose{k}}\,(r+k)^{k-1}(s-k)^{n-k}={(r+s)^{n}\over
r}\quad.$ $None$

Proof Let $F_{n,k}(r,s)$ and $a_{n}(r,s)$ denote, respectively, the summand
and sum on the LHS of (8), and let
$G_{n,k}:=(s-n){{n-1}\choose{k-1}}\,(k+r)^{k-1}(s-k)^{n-k-1}$. Since

$F_{n,k}(r,s)-sF_{n-1,k}(r,s)-(n+r)F_{n-1,k}(r+1,s-1)+(n-1)(r+s)F_{n-2,k}(r+1,s-1)=G_{n,k}-G_{n,k+1}\,,$

(check!), we have by summing from $k=0$ to $k=n$, thanks to the telescoping on
the right:

$a_{n}(r,s)-sa_{n-1}(r,s)-(n+r)a_{n-1}(r+1,s-1)+(n-1)(r+s)a_{n-2}(r+1,s-1)=0.$
$None$

Since $(r+s)^{n}\cdot r^{-1}$ also satisfies this recurrence (check!) with the
same initial conditions $a_{0}(r,s)=r^{-1}$ and $a_{1}(r,s)=(r+s)\cdot
r^{-1}$, $(8)$ follows.

This proof was derived using John Majewicz’s brilliant adaptation of Sister
Celine’s technique [Fa1][Fa2][Z] (see also [PWZ], chapter 4). Recall that
Sister Celine was interested in finding pure recurrence relations of the form

$c_{0}(n)a_{n}+c_{1}(n)a_{n+1}+\dots+c_{L}(n)a_{n+L}=0\quad,$ $None$

where $c_{0}(n),\dots,c_{L}(n)$ are polynomials in $n$ for sequences $a_{n}$,
that are defined by expressions of the form

$a_{n}:=\sum_{k=-\infty}^{\infty}F_{n,k}\quad,$

where $F_{n,k}$ is proper hypergeometric (see [PWZ] for the definition, in
particular, $F_{n+1,k}/F_{n,k}$ and $F_{n,k+1}/F_{n,k}$ are both rational
functions of $n$ and $k$) . The way she did it was to (in her case by hand,
but nowadays it has all been fully automated) to search for a recurrence of
the form:

$\sum_{i=0}^{L}\sum_{j=0}^{M}b_{ij}(n)F_{n+i,k+j}\,=\,0\quad,$ $None$

for some positive integers $L$ and $M$.

Dividing by $F_{n,k}$, and clearing denominators, looking at the numerator,
and then setting all the coefficients of powers of $k$ to $0$, yields a system
of linear equations (with coefficients that are polynomials in $n$) for the
undetermined $b_{ij}(n)$.

Having found the $b_{ij}(n)$, summing Eq. $(11)$ from $k=-\infty$ to
$k=\infty$, yields Eq. $(10)$ with

$c_{i}(n)=\sum_{j=0}^{M}b_{ij}(n)\quad,\quad 0\leq i\leq L\quad.$

In his PhD thesis [M1][M2] (written under the direction of the second author),
John Majewicz adapted Sister Celine’s method to Abel Type sums, of the form

$a_{n}(r,s)=\sum_{k=0}^{n}\,F_{n,k}\,(r+k)^{k-1+p}(s-k)^{n-k+q}x^{k}\quad,$
$None$

where $F_{n,k}$ is hypergeometric in $n$ and $k$. Here $p$ and $q$ are
arbitrary integers, and $x$ is any number (or symbol). It is no longer the
case that $a_{n}(r,s)$ satisfies a pure recurrence in $n$, with $r$ and $s$
fixed, but it does satisfy a functional recurrence, similar (but often much
more complicated) to Eq. $(10)$. Denoting the summand of $(12)$ by
$\overline{F}_{n,k}(r,s)$

$\overline{F}_{n,k}(r,s)\,:=\,F_{n,k}(r+k)^{k-1+p}(s-k)^{n-k+q}x^{k}\quad,$

one looks for polynomials $b_{ij}(n)$ (that also depend on $r,s,p,q$ and $x$,
but are free of $k$), such that

$\sum_{i=0}^{L}\sum_{j=0}^{M}b_{ij}(n)\overline{F}_{n+i,k+j}(r-j,s+j)\,=\,0\quad.$
$None$

Dividing by $\overline{F}_{n,k}(r,s)$ (since
$\overline{F}_{n+i,k+j}(r-j,s+j)/\overline{F}_{n,k}(r,s)$ is still a rational
function of $n$ and $k$), clearing denominators, looking at the numerator, and
setting all the coefficients of powers of $k$ to $0$, we get again a system of
linear equations for the undetermined quantities $b_{ij}(n)$. We then ask
Maple to kindly solve them, and if in luck we get a non-zero solution. It can
be shown ([M1][M2]) that one can always find orders $L$ and $M$ for which such
a system is solvable (for sufficiently large $L$ and $M$, there are more
unknowns than equations). Having found such a recurrence for
$\overline{F}_{n,k}$, summing over $k$ yields a functional recurrence for
$a_{n}(r,s)$.

$\sum_{i=0}^{L}\sum_{j=0}^{M}b_{ij}(n)a_{n+i}(r-j,s+j)\,=\,0\quad.$ $None$

Implementation

We fully implemented the Celine-Majewicz algorithm in a Maple packge
AbelCeline.txt available from

https://sites.math.rutgers.edu/~zeilberg/tokhniot/AbelCeline.txt . The
function call is

FindOpe(F,n,k,r,s,R,S,N,K,MaxOrd);  ,

where MaxOrd is the maximal order of the recurrence you are willing to
tolerate.

It inputs a hypergeometric term $F_{n,k}$ and outputs the recurrence in the
form of the corresponding operator, where $N$, $R$, and $S$ are the forward
shift operators in $n$, $r$, and $s$, respectively. To get a computer-
generated paper, in humanese, the function call is:

Paper(F,n,k,r,s,R,S,N,K,MaxOrd):  .

Sample Output

Typing

Paper(binomial(n,k)*x**k,n,k,r,s,2):,

yields in $0.12$ seconds the following deep fact.

Let, for any integers $p$ and $q$ and number (or symbol) $x$

$a_{n}(r,s):=\sum_{k=0}^{n}{{n}\choose{k}}(r+k)^{k-1+p}(s-k)^{n-k+q}\,x^{k}\quad,$

then

$a_{n}\\!\left(r,s\right)=\left(nx+rx\right)a_{n-1}\\!\left(r+1,s-1\right)+sa_{n-1}\\!\left(r,s\right)+\left(-nrx-
nsx+rx+sx\right)a_{n-2}\\!\left(r+1,s-1\right)\quad.$

Typing

Paper(1/(k!**2*(n-k)!)*x**k,n,k,r,s,3):

yields that the sequence of polynomials in $(r,s)$ defined by

$a_{n}(r,s):=\sum_{k=0}^{n}{1\over
k!^{2}(n-k)!}\,(r+k)^{k-1+p}(s-k)^{n-k+q}\,x^{k}\quad,$

satisfies the functional recurrence

$a_{n}\\!\left(r,s\right)={x\left(n+r\right)a_{n-1}\\!\left(r+1,s-1\right)\over
n^{2}}+{\left(2n^{2}-2ns-2n+s\right)sa_{n-1}\\!\left(r,s\right)\over\left(n-1-s\right)n^{2}}$
$-{\left(n^{2}+2nr-2rs-s^{2}-n-r\right)xa_{n-2}\\!\left(r+1,s-1\right)\over\left(n-1-s\right)n^{2}}$
$-{\left(n-s\right)s^{2}a_{n-2}\\!\left(r,s\right)\over\left(n-1-s\right)n^{2}}+{\left(nr+ns-
rs-s^{2}\right)xa_{n-3}\\!\left(r+1,s-1\right)\over\left(n-1-s\right)n^{2}}\quad.$

To see numerous other examples, read the output files in the front of this
article:

https://sites.math.rutgers.edu/~zeilberg/mamarim/mamarimhtml/abelKZ.html .

In particular, to see the functional recurrence satisfied by the ‘innocent’
sum:

$a_{n}(r,s):=\sum_{k=0}^{n}{{n}\choose{k}}{{n+k}\choose{k}}(r+k)^{k-1+p}(s-k)^{n-k+q}\,x^{k}\quad,$

see the output file

https://sites.math.rutgers.edu/~zeilberg/tokhniot/oAbelCeline4.txt .

Notice how complicated things get!

Getting differential recurrences

Since $r$ and $s$, as opposed to $n$ and $k$, are ‘continuous variables’, and
since the functional recurrences gotten by Majewicz’ Abel-Celine technique get
so complicated even with very simple summands, it occurred to us to look for
differential recurrences. For any bi-variate (proper) hypergeometric term
$F_{n,k}$, defining (as above)

$a_{n}(r,s):=\sum_{k=0}^{n}F_{n,k}(r+k)^{k-1+p}(s-k)^{n-k+q}\,x^{k}\quad,$

where $p$ and $q$ are integers (but may be left symbolic) and $x$ is a number
(again, it can be left symbolic), one looks for differential-recurrence
equations of the form

$\sum_{i=0}^{L}\sum_{j=0}^{M}b_{i,j}(n,r,s){d^{i}\over
dr^{i}}a_{n+j}(r,s)\,=\,0\quad,$

and

$\sum_{i=0}^{L}\sum_{j=0}^{M}c_{i,j}(n,r,s){d^{i}\over
ds^{i}}a_{n+j}(r,s)\,=\,0\quad,$

Together these pairs of differential recurrence equations, combined with
initial conditions, uniquely determine $a_{n}(r,s)$.

In fact, one can consider far more general ‘kernels’. We can find a pair of
differential recurrence equations for sums of the from

$a_{n}(r,s):=\sum_{k=0}^{n}F_{n,k}K(r,s,n,k)\quad,$

for any kernel $K(r,s,n,k)$ such that $({\partial K\over\partial r})/K$, and
$({\partial K\over\partial s})/K$ are rational functions of $(r,s)$ (and $k$).

We proceed analogously. One applies the generic operator to the summand
$\overline{F}(n,k)$, expands, clears denominators, looks at the numerator,
then equates all the powers of $k$ to $0$, getting a system of linear
equations that Maple can solve for you.

To get these pairs of equations, in verbose, human-readable from, type:

PaperD(F,n,k,r,s,MaxOrd,KER):

where MaxOrd is the maximum order you are willing to tolerate, and KER is the
kernel. For example, typing

PaperD(binomial(n,k),n,k,r,s,2,(r+k)**(k-1+p)*(s-k)**(n-k+q)*x**k): ,

yields in a fraction of a second the facts that the sequence of polynomials in
$r$ and $s$, $a_{n}(r,s)$ (for any $p$ and $q$ and $x$)

$a_{n}(r,s):=\sum_{k=0}^{n}{{n}\choose{k}}(r+k)^{k-1+p}(s-k)^{n-k+q}\,x^{k}\quad,$

satisfy the pair of recurrence-differential equations (the first in $r$, the
second in $s$)

$-\left(pn+ns-n+p+s-1\right)a_{n}\\!\left(r,s\right)+\left(nr+ns+r+s\right)\left({\partial\over\partial
r}a_{n}\\!\left(r,s\right)\right)$
$+(n+p)a_{n+1}\\!\left(r,s\right)-\left(n+r+1\right)\left({\partial\over\partial
r}a_{n+1}\\!\left(r,s\right)\right)=0\quad,$

and

$-\left(n+1\right)\left(q+n-s+1\right)a_{n}\\!\left(r,s\right)+qa_{n+1}\\!\left(r,s\right)+\left(n-s+1\right)\left({\partial\over\partial
s}a_{n+1}\\!\left(r,s\right)\right)\,=0\quad.$

Note that setting $x=1$ and $p=0,q=0$ yields yet-another (automatic) proof of
the original identity $(8)$. In fact it is closer in spirit to Niels Abel’s
original proof, that also used differentiation (or rather integration).

Remark: The proof technique of this part can prove (2) and can be adapted
(with some pre-processing) to prove (1).

To see many other examples, look at the output files

https://sites.math.rutgers.edu/~zeilberg/tokhniot/oAbelCeline5.txt ,

and

https://sites.math.rutgers.edu/~zeilberg/tokhniot/oAbelCeline6.txt .

To see examples with more complicated kernels, see

https://sites.math.rutgers.edu/~zeilberg/tokhniot/oAbelCeline7.txt .

Readers can easily find many other such deep facts by playing with
AbelCeline.txt. Enjoy!

References

[A] Niels H. Abel, Beweis eines Ausdruckes, von welchem die Binomial-Formel
ein einzelner Fall ist, Crelle’s J. Mathematik 1 (1826), 159-160.
https://sites.math.rutgers.edu/~zeilberg/akherim/Abel1826.pdf .

[C] Louis Comtet, “Advanced Combinatorics”, D. Reidel Publ. Co.,
Dordrecht/Boston, 1974 .

[EM] Shalosh B. Ekhad and John Majewicz, A short WZ-style proof of Abel’s
identity, Electronic Journal of Combinatorics 3(2) (the Foata Festschrift)
#R16. https://sites.math.rutgers.edu/~zeilberg/mamarim/mamarimPDF/abel.pdf .

[Fa1] Sister Mary Celine Fasenmyer, Some generalized hypergeometric
polynomials, Bull. Amer. Moth. Soc. 53 (1947), 806-812.

[Fa2] Sister Mary Celine Fasenmyer, A note on pure recurrence relations, Amer.
Math. Monthly 56 (1949), 14-17.

[Fo] Dominique Foata, Enumerating $k$-Trees, Discrete Mathematics 2 (1971),
181-186. https://irma.math.unistra.fr/~foata/paper/pub16.pdf .

[FoFu] Dominique Foata and Aimé Fuchs, Réarrangements de fonctions et
dénombrement, Discrete Mathematics 2 (1971), 181-186.
https://irma.math.unistra.fr/~foata/paper/pub12.pdf .

[Fr] Jean Francon, Preuves combinatoires des identités d’Abel, Discrete
Mathematics 8 (1974), 331-343.

[H] Adolf Hurwitz, Über Abel’s Verallgemeinerung der binomischen Formel, Acta
Math. 26 (1902), 199-203.

[Ka] Gil Kalai, A note on an evaluation of Abel sums, J. Combin, Theory (Ser.
A), 27 (1979), 213-217.
https://sites.math.rutgers.edu/~zeilberg/akherim/kalai1979.pdf .

[Ka’] Gil Kalai, Nostalgia corner: John Riordan’s referee report of my first
paper, blog post on Combinatorics and More, 2021.
https://gilkalai.wordpress.com/2021/02/19/nostalgia-corner-john-riordans-
referee-report-of-my-first-paper/

[Kn] Donald E. Knuth, The Art of Computer Programming, Vol. I: Fundamental
Algorithms (Addison-Wesley, Reading, MA, 1969).

[M1] John Majewicz, WZ-type certification procedures and Sister Celine’s
technique for Abel-type sums, J. of Difference Equations and Applications 2
(1996), 55-65.

[M2] John Majewicz, WZ certification of Abel-type identities and Askey’s
positivity conjecture, Ph.D. thesis, Temple University 1997.
https://sites.math.rutgers.edu/~zeilberg/Theses/JohnMajewiczThesis.pdf .

[PWZ] Marko Petkovsek, Herbert S. Wilf, and Doron Zeilberger, “A=B”, A.K.
Peters, 1996. https://sites.math.rutgers.edu/~zeilberg/AeqB.pdf .

[R] John Riordan, “Combinatorial identities”, Wiley, New York, 1968.

[Z] Doron Zeilberger, Sister Celine’s technique and its generalizations, J.
Math. Anal. Appl. 85 (1982), 114-145.
https://sites.math.rutgers.edu/~zeilberg/mamarimY/celine1982.pdf .

Gil Kalai, Hebrew University of Jerusalem, Einstein Institute of Mathematics,
and Reichman University, Efi Arazi School of Computer Science. Email: gil dot
kalai at gmail dot com .

Doron Zeilberger, Department of Mathematics, Rutgers University (New
Brunswick), Hill Center-Busch Campus, 110 Frelinghuysen Rd., Piscataway, NJ
08854-8019, USA. Email: DoronZeil at gmail dot com .

May 1, 2024 .


# Continual learning using lattice-free MMI for speech recognition

###### Abstract

Continual learning (CL), or domain expansion, recently became a popular topic
for automatic speech recognition (ASR) acoustic modeling because practical
systems have to be updated frequently in order to work robustly on types of
speech not observed during initial training. While sequential adaptation
allows tuning a system to a new domain, it may result in performance
degradation on the old domains due to catastrophic forgetting. In this work we
explore regularization-based CL for neural network acoustic models trained
with the lattice-free maximum mutual information (LF-MMI) criterion. We
simulate domain expansion by incrementally adapting the acoustic model on
different public datasets that include several accents and speaking styles. We
investigate two well-known CL techniques, _elastic weight consolidation_ (EWC)
and _learning without forgetting_ (LWF), which aim to reduce forgetting by
preserving model weights or network outputs. We additionally introduce a
sequence-level LWF regularization, which exploits posteriors from the
denominator graph of LF-MMI to further reduce forgetting. Empirical results
show that the proposed sequence-level LWF can improve the best average word
error rate across all domains by up to 9.4% relative compared with using
regular LWF.

Index Terms—  Automatic Speech Recognition, Continual Learning, Domain
Expansion, Lattice-free MMI, Learning Without Forgetting

##  1 Introduction

Commercial automatic speech recognition (ASR) systems must frequently work
robustly with a variety of accents, recording devices, background noises, and
speech styles. While training the acoustic models on thousands [1, 2] and
hundreds of thousands [3] of hours of diverse speech helps the model
generalize better, it still needs to be updated from time to time to cover
domains that have not been sufficiently represented in the training data.
Similar to [4] we define domain as a group of utterances that share some
common characteristics. Retraining the model with new data from scratch is
expensive and difficult if privacy limitations do not allow keeping target
domain data and the initial training data in one place. In practice,
adaptation of the acoustic models using target domain speech appears to be a
more favorable approach [5]. One problem is that the adapted model’s
performance on previously learned data may significantly degrade due to
catastrophic forgetting [6, 7], which limits the model’s constant improvement
across multiple domains.

Continual learning (CL) or life-long learning [8], which was intensively
studied especially in computer vision, is a variety of machine learning
methods that aim to learn new tasks without forgetting previously acquired
knowledge. Broadly, CL methods are based on regularization techniques [9, 10,
11], dynamic neural network architectures [12, 13, 14], and rehearsal (also
called memory replay) [12, 15, 16].

Overcoming catastrophic forgetting when adapting the ASR acoustic models to
new domains can be seen as a special case of CL, as the model usually keeps
doing the same task (predicting the same set of classes). Authors of [4]
defined this task as domain expansion and explored several CL regularization
techniques to optimize the ASR performance on initial and adapted speech
accents. Later, [17] explored similar regularization techniques to
sequentially train a multi-dialect ASR system. Alternatively, [18] proposed a
model expansion–based approach, which demonstrated good results but at a price
of growing the model size with the number of training tasks, where tasks are
defined as public datasets used by the authors. Similarly, [19] explored a
combination of model parameters and network outputs as a method for
longitudinal training – a scenario in which the model learns a single domain
but with incrementally appearing data and without access to historical data.
[20] compared regularization- and rehearsal-based techniques for incrementally
learning an end-to-end CTC-based ASR system. They demonstrated better
performance of rehearsal technique that, however, requires storing some
portion of the training data from each domain.

Similar to [17], we focus on two popular regularization-based techniques,
namely, elastic weight consolidation (EWC) [11] and learning without
forgetting (LWF) [9]. We do not consider rehearsal-based techniques as in
general they require access to the source data during adaptation and therefore
are hard or impossible to use in privacy-preserving settings. Dynamic
architectures are also not considered as these methods generally result in
growing the model size over time, eventually slowing down the inference.

In this work111The source codes for this work will be released soon. we apply
EWC and LWF on acoustic models trained using the LF-MMI [21] criterion and
assess their efficacy. Furthermore, as the main contribution of this work, we
propose a sequence-level version of LWF regularization that is based on LF-MMI
and we demonstrate its effectiveness through multiple experiments. The
experiments are conducted on several public databases that involve several
English accents (US, Indian, Singapore), speech styles (read speech, public
talks, and conversations) and channels (studio and telephone).

##  2 Lattice-free MMI

Lattice-free MMI is a method of training an acoustic model using the MMI
objective function without generating utterance-specific denominator lattices
[21]. More specifically, there is only one denominator graph that is used for
all utterances during training. The objective function is as follows222Note
that in this and the rest of the equations in this paper, for brevity we show
the objective functions for one utterance only. The full objective function
would be the sum over all utterances. Also note that an objective function is
maximized during training.:

$\displaystyle\mathcal{F}_{MMI}$
$\displaystyle=\log\frac{p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{\boldsymbol{w}})}{p_{\lambda}(\boldsymbol{x})}$
(1)

The denominator can be approximated as:

$\displaystyle p_{\lambda}(\boldsymbol{x})$
$\displaystyle=\sum_{\boldsymbol{l}}p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{\boldsymbol{l}})\approx
p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{den})$ (2)

where $\mathbb{M}_{den}$ is a finite-state transducer (FST) graph that
includes all possible sequences of words. This is called the denominator
graph, as opposed to $\mathbb{M}_{\boldsymbol{w}}$, which is called the
numerator graph and is utterance specific. Expanding the log gives us:

$\displaystyle\mathcal{F}_{MMI}$ $\displaystyle=\log
p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{\boldsymbol{w}})-\log
p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{den})$ (3)

Note that the first term is maximizing the likelihood of the paths in the
numerator graph which represents the true label and the second term is
minimizing the likelihood of all other paths in the denominator graph.

##  3 Continual learning

The two most popular regularization-based approaches for CL (LWF and EWC)
address the forgetting problem by forcing the network to preserve either its
outputs or its parameters, respectively. The following subsections briefly
describe these approaches in more detail.

###  3.1 Learning Without Forgetting

The LWF objective function is a cross-entropy term that is added as a
regularization term to the main objective function for training the neural
network:

$\mathcal{F}_{LWF}=\alpha\sum_{i}y^{src}_{i}\log(y_{i})$ (4)

where $y_{i}$ are the outputs of the network while training and $y^{src}_{i}$
are the pre-computed outputs of the network using the source model (i.e., the
previous step of CL) for all of the target domain training examples. We may
simply call the $y^{src}_{i}$ reference posteriors in this context. The scale
$\alpha$ controls the effect of the LWF regularization.

###  3.2 Elastic Weight Consolidation

While LWF concerns remembering the output distribution, EWC is focused on the
parameters of the network. Specifically, the EWC objective function is as
follows:

$\mathcal{F}_{EWC}=-\alpha\sum_{d}\sum_{j}F^{d}_{j}(\theta^{d}_{j}-\theta_{j})^{2}$
(5)

where $d$ loops over previous domains, and $F^{d}_{j}$ is the Fisher diagonal
value for the $j_{th}$ parameter of the network estimated using the data and
model at CL step $d$. Also $\theta_{j}$ is the $j_{th}$ parameter of the
current network and $\theta^{d}_{j}$ is the corresponding parameter from the
model at CL step $d$. Note that similar to [17] we use the online computation
approach for the Fisher matrix estimation. Also note that while LWF relies
only on the output of CL at the previous domain, EWC needs statistics (i.e.,
Fisher matrices) from all previous steps.

##  4 Proposed sequence level objective

As mentioned earlier, the LWF loss is cross-entropy, which is frame-level.
However, LF-MMI is a sequence-level loss function [21]. It appears sub-optimal
to train the acoustic model with a sequence-level objective and apply LWF
using a frame-level objective. Therefore, instead of using the neural network
outputs as reference posteriors for LWF, we propose to use the denominator
posteriors (i.e., occupancies) as the reference posteriors. This leads to the
following term for the LWF objective function:

$\mathcal{F}=\alpha\sum_{i}\gamma^{src}_{i}\log(y_{i})$ (6)

where $\gamma^{src}_{i}$ are the denominator posteriors from the previous
domain. These posteriors are computed by running the forward-backward
algorithm on the denominator graph given the neural network outputs and are
therefore sequence-level. However, this cross-entropy term poses a problem
because the target distribution (i.e., $\gamma^{src}$) is sequence-level
whereas the predicted distribution (i.e., the neural network outputs while
training: $y_{i}$) is frame-level. This results in excessive boosting of the
denominator posteriors as we are treating them as targets.333Note that if we
try to train the network with this term, catastrophic forgetting is not
prevented (results are not shown). To offset this effect, we subtract the
denominator log-prob term scaled by the same LWF scale $\alpha$. This gives us
the proposed sequence-level LWF objective function:

$\mathcal{F}_{DenLWF}=\alpha\sum_{i}\gamma^{src}_{i}\log(y_{i})-\alpha\log
p_{\lambda}(\boldsymbol{x}|\mathbb{M}_{den})$ (7)

Taking the derivative of this term with respect to log outputs leads to the
following term:

$\frac{\partial\mathcal{F}_{DenLWF}}{\partial\log(y_{i})}=\alpha(\gamma^{src}_{i}-\gamma_{i})$
(8)

where the $\gamma_{i}$ are the denominator occupancies during training. We can
see immediately from this equation that the gradients are $0$ at the beginning
of training since the training denominator posteriors match the source ones.
This is a desired characteristic, as in the beginning there is no forgetting
so we do not expect any gradients from the LWF regularization term. The same
cannot be said about Equation 6, which leads to an unbalanced derivative.

##  5 Experimental setup

We choose 5 different English datasets (summarized in Table 1) for sequential
training.

Name |  Dataset |  Speech type |  English accent |  Size, hours
---|---|---|---|---
FSH |  Fisher English [22] subset |  CTS444Conversational Telephony speech |  US |  200
LIB |  Librispeech [23] subset |  Read |  Mostly US |  50
TED |  TED-LIUM [24] subset |  Talks |  Mostly US |  50
NSC |  National Speech Corpus [25] (IRV subset) |  CTS |  Singaporean |  50
CVI |  Mozilla Common Voice [26] v6.1 (Indian speakers) |  Read |  Indian |  40
Table 1: Summary of datasets used in the experiments.

A 50-hour subset from each domain is used for domain expansion, except for
CVI, for which there is only 40 hours of speech available. The 200-hour FSH
dataset is used to train the seed model in all experiments. For each dataset,
evaluation is done on their official dev/test subsets. For LIB, a combination
of dev_clean and test_clean subsets is used and for TED, a combination of dev
and test subsets. A 3-gram language model is estimated using the training text
of all domains and is used in all experiments. We do not use i-vectors or
other kinds of speaker adaptation in training the acoustic models. In the
following experiments, $FT$ refers to a model that has been trained using
naive fine-tuning and therefore is considered as the worst-case baseline of
catastrophic forgetting. On the other hand, $Comb$ refers to a model trained
on all the available data from all domains combined 555Note that unlike [17],
we do not train separate $comb$ models at each step as there is negligible
difference in WER.. Table 2 summarizes the WERs for the seed and $comb$ models
on all domains. It is clear that the most challenging domains for this seed
model are CVI and NSC.

Kaldi ASR toolkit is used for all experiments [27]. The network is made up of
14 TDNN-F layers with a total of 14M parameters [28] and is trained using LF-
MMI with mini-batch gradient descent. The seed model’s decision tree is used
during all steps of domain expansion. The alignments at each step are
generated using a Gaussian mixture model (GMM) trained on the target domain at
that step.

Model | FSH | TED | LIB | NSC | CVI | Avg
---|---|---|---|---|---|---
Seed (FSH) | 18.2 | 15.6 | 11.8 | 56.3 | 58.4 | 32.1
Comb | 18.1 | 11.8 | 8.3 | 25.3 | 25.0 | 17.7
Table 2: Results for the seed model and the combined-data model. | FSH |  |  $\rightarrow$ TED |  |  $\rightarrow$ LIB |  |  $\rightarrow$ NSC |  |  $\rightarrow$ CVI
---|---|---|---|---|---|---|---|---|---
Method | FSH |  | FSH | TED | Avg |  | FSH | TED | LIB | Avg |  | FSH | TED | LIB | NSC | Avg |  | FSH | TED | LIB | NSC | CVI | Avg
FT | 18.2 |  | 30.6 | 13.3 | 21.9 |  | 29.0 | 16.7 | 9.0 | 18.2 |  | 48.4 | 42.3 | 33.9 | 27.6 | 38.0 |  | 59.5 | 42.2 | 23.5 | 58.0 | 27.9 | 42.2
EWC | 18.2 |  | 21.7 | 13.5 | 17.6 |  | 22.1 | 14.4 | 9.7 | 15.4 |  | 29.9 | 20.6 | 15.5 | 34.0 | 25.0 |  | 38.4 | 25.3 | 15.4 | 47.0 | 31.7 | 31.6
LWF | 18.2 |  | 23.0 | 13.5 | 18.2 |  | 22.8 | 14.0 | 9.2 | 15.3 |  | 30.5 | 18.3 | 12.5 | 30.5 | 23.0 |  | 39.7 | 23.7 | 14.0 | 41.7 | 30.0 | 29.8
DenLWF | 18.2 |  | 20.7 | 13.4 | 17.1 |  | 22.5 | 13.9 | 9.1 | 15.2 |  | 27.5 | 16.2 | 10.9 | 34.5 | 22.3 |  | 33.0 | 19.7 | 12.0 | 40.0 | 30.3 | 27.0
Table 3: WERs,% of 4 CL steps using different baseline and proposed methods.
The first column shows the seed model. The next 3 columns under “$\rightarrow
TED$” are for the first step FSH $\rightarrow$ TED, the columns under
“$\rightarrow LIB$” are for the second step TED $\rightarrow$ LIB, and so
forth.

Similar to [17], we also compute gap recovery rates for the CL steps as
follows:

$\text{Gap
Recovery}=1.0-\frac{\text{WER}_{CL}-\text{WER}_{Comb}}{\text{WER}_{FT}-\text{WER}_{Comb}}$
(9)

where $\text{WER}_{CL}$, $\text{WER}_{Comb}$, and $\text{WER}_{FT}$ refer to
WERs of the model trained using the CL method, the $comb$ model, and the $FT$
model, respectively. In addition to gap recovery rates, at each step we also
define _Relative Forgetting_ , which measures the degradation of WER on the
past domains as a result of that step, and _Relative Learning_ , which
measures the improvement of WER on the target domain at the end of that step.
Formally, they are defined as follows:

Relative Learning $\displaystyle=1.0-\frac{\text{CWER}}{\text{CWER}^{src}}$
(10) Relative Forgetting
$\displaystyle=\frac{\text{PWER}}{\text{PWER}^{src}}-1.0$ (11)

where CWER is the WER on the current domain at each step (e.g., at step TED
$\rightarrow$ LIB, it would be the WER on the LIB test set at the end of the
step). $\text{CWER}^{src}$ is the WER on the same dataset, but using the model
at the end of the previous step. PWER and $\text{PWER}^{src}$ are similar to
CWER and $\text{CWER}^{src}$ except they are measured on past domains
(averaged). For example, at step TED $\rightarrow$ LIB of the sequence FSH
$\rightarrow$ TED $\rightarrow$ LIB, the past domains are FSH and TED.

##  6 Experiments and Results

This section describes a series of experiments to assess the efficacy of LWF,
EWC, and the proposed method on LF-MMI trained acoustic models. Specifically,
FSH is used to create the seed model, after which we do 4 steps of CL in the
following order: FSH $\rightarrow$ TED $\rightarrow$ LIB $\rightarrow$ NSC
$\rightarrow$ CVI. The detailed results are listed in Table 3. Note that after
each step, only the domains involved in training up to that step are
evaluated. We can see that naive FT leads to the best performance on target
data but severely degrades performance on all previous domains. After the
first step, EWC outperforms LWF. However, in the subsequent steps LWF performs
considerably better, especially in the last 2 steps. This is consistent with
observations of [17]. The results also show that DenLWF achieves a WER similar
to that of LWF in the second step and remarkably better WERs in the other
steps. Specifically, for the last two steps, which are the most challenging,
DenLWF relatively improves LWF’s average WER by 3% and 9.4%, respectively. If
we look at the individual domain WERs, the improvements are even more visible.
For example, in the last step, while DenLWF achieves a slightly worse (but
close) WER on CVI, it achieves a much better WER on previous domains. In
particular, DenLWF improves LWF’s WER on FSH from 39.7% to 33.0% and on TED
from 23.7% to 19.7%.

It should be noted that all methods were tuned across multiple combinations of
source-target transfers. For LWF, we use a scale of $\alpha=1.0$ while for
DenLWF a lower scale of $\alpha=0.6$ works best. For EWC, after normalizing
the Fisher diagonal values to have a median of $1.0$, we use $\alpha=300$.

Fig. 1: Overall performance of different methods. The upper chart shows
average gap recovery percentages for different steps of the CL training
pipeline. The middle and lower charts show relative learning and forgetting
percentages, respectively.

Figure 1 shows the gap recovery and relative learning/forgetting percentages
(as defined in Equations 9-11) for the steps in Table 3. In the top plot we
can see that DenLWF achieves higher gap recovery rates in all steps,
especially in the first and last steps. In the second step, all methods have a
similar performance, which might be attributable to the fact that LIB is an
easy domain for our seed model. Specifically, EWC and LWF achieve a gap
recovery rate of 40-60% and 50-68%, respectively, in different steps of CL,
while DenLWF achieves a gap recovery rate of 55-71% for these steps. In the
middle and lower charts we see the relative learning and forgetting rates,
respectively. DenLWF and LWF show a similar capacity to learn new domains,
with LWF learning slightly more in the third step. However, as can be seen in
the lower chart, this comes at the cost of much more forgetting for LWF.
Specifically, LWF has a relative forgetting rate of around 30% in the last two
steps whereas DenLWF only forgets around 20% from the past domains.

Fig. 2: Comparing LWF cross-entropy objective during training for regular
(frame-level) LWF and proposed DenLWF.

###  6.1 Analysis and Discussion

In order to look more closely into how the proposed DenLWF functions and how
it compares with regular LWF, we do two experiments. In the first experiment,
we measure and plot the regular LWF objective function (i.e., Equation 4) when
training a model using regular LWF as well as DenLWF. Note that LWF cross-
entropy objective for DenLWF is computed here only for visualization purposes
and is not used in back-propagation. The resulting plot is shown in Figure 2.
It can be observed that for regular LWF, the objective function degrades right
after the first iteration of training and then starts to improve as training
progresses. This effectively restricts the output distribution from diverging
too far from the source domain posteriors. However, in the case of DenLWF, we
can see that the output posteriors keep diverging from the reference
posteriors until halfway through training, at which point they plateau. This
seems to allow the network to learn better, as the network output is under
weaker frame-level restriction compared to regular LWF. In other words,
regular LWF forces the network to remember the source domain on a frame-level
basis, whereas the proposed DenLWF objective forces the network to remember
the source domain on a sequence-level basis – which is more consistent with
the speech signal – leading to better overall performance (i.e., average WER).

In the second experiment, we investigate the impact of the LWF scale $\alpha$
on the average WER as well as on individual WERs. Figure 3 compares regular
LWF and DenLWF when trained using different scales’ $\alpha$ (all other
hyperparameters are the same). The training step used in this experiment is
FSH $\rightarrow$ CVI. The upper plot shows the average WER (on FSH and CVI)
while the lower plots show the WERs on FSH and CVI separately. It can be seen
that regular LWF achieves its best performance when the scale is close to 1.0;
whereas DenLWF achieves its best performance when we use a smaller scale –
closer to 0.6. This suggests that DenLWF is a more powerful regularizing term
that needs a smaller scale. We can also see that, generally, DenLWF achieves
much less forgetting (i.e., lower WER on FSH), which comes at the cost of less
learning (i.e., higher WER on CVI). However, at the scales where DenLWF works
best (i.e., $0.5$ and $0.6$), the difference in CVI WER between DenLWF and LWF
is quite small. Note that this plot also shows that for all the scales tested,
the worst-case performance of DenLWF is better than the best-case performance
of LWF on the FSH $\rightarrow$ CVI step. We observed similar plots for other
steps (not shown due to lack of space).

Fig. 3: Average and individual WERs% for regular LWF and DenLWF on FSH
$\rightarrow$ CVI transfer.

##  7 Conclusions

In this study, we applied two well-known regularization-based methods for
continual learning, namely, EWC and LWF, to the state-of-the-art lattice-free
MMI acoustic models and investigated their efficacy through a series of
experiments. We observed gap recovery rates of 40-60% using EWC and 50-68%
using LWF, for different steps of CL. Compared to naive fine-tuning, EWC
improved the final average WER (after all the steps of CL) from 42.2% to
31.6%, while LWF improved it to 29.8%. In addition, given the fact that
lattice-free MMI is a sequence-level objective function, we proposed a new LWF
objective function called DenLWF where we use MMI’s denominator posteriors as
reference posteriors in LWF while adding an offsetting term. Through
experiments we showed that DenLWF improved the gap recovery rates to 55-71%
for different steps of CL. Also, DenLWF improved the final average WER from
29.8% down to 27.0%.

#### Acknowledgements

The authors would like to thank Dr. Daniel Povey for his valuable suggestions,
and Stephen Gaudet for proofreading the paper.

## References

  * [1] Tatiana Likhomanenko et al.,  “Rethinking evaluation in ASR: Are our models robust enough?,”  in INTERSPEECH. ISCA, 2021.
  * [2] William Chan et al.,  “Speechstew: Simply mix all available speech recognition data to train one large neural network,”  arXiv preprint arXiv:2104.02133, 2021.
  * [3] Arun Narayanan et al.,  “Toward domain-invariant speech recognition via large scale training,”  in SLT. IEEE, 2018.
  * [4] Shahram Ghorbani, Soheil Khorram, and John HL Hansen,  “Domain expansion in DNN-based acoustic models for robust speech recognition,”  in ASRU. IEEE, 2019.
  * [5] Peter Bell et al.,  “Adaptation algorithms for neural network-based speech recognition: An overview,”  IEEE Open Journal of Signal Processing, vol. 2, pp. 33–66, 2020\.
  * [6] Michael McCloskey and Neal J Cohen,  “Catastrophic interference in connectionist networks: The sequential learning problem,”  in Psychology of learning and motivation, vol. 24, pp. 109–165. Elsevier, 1989.
  * [7] Robert M French,  “Catastrophic forgetting in connectionist networks,”  Trends in cognitive sciences, vol. 3, no. 4, pp. 128–135, 1999\.
  * [8] German I Parisi et al.,  “Continual lifelong learning with neural networks: A review,”  Neural Networks, vol. 113, pp. 54–71, 2019.
  * [9] Zhizhong Li and Derek Hoiem,  “Learning without forgetting,”  IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935–2947, 2017.
  * [10] Friedemann Zenke, Ben Poole, and Surya Ganguli,  “Continual learning through synaptic intelligence,”  in International Conference on Machine Learning. PMLR, 2017, pp. 3987–3995.
  * [11] James Kirkpatrick et al.,  “Overcoming catastrophic forgetting in neural networks,”  Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017.
  * [12] Tyler L Hayes, Nathan D Cahill, and Christopher Kanan,  “Memory efficient experience replay for streaming learning,”  in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 9769–9776.
  * [13] Andrei A Rusu et al.,  “Progressive neural networks,”  arXiv preprint arXiv:1606.04671, 2016.
  * [14] Jaehong Yoon et al.,  “Lifelong learning with dynamically expandable networks,”  in ICLR, 2018.
  * [15] David Lopez-Paz and Marc’Aurelio Ranzato,  “Gradient episodic memory for continual learning,”  Advances in neural information processing systems, vol. 30, pp. 6467–6476, 2017.
  * [16] Geoffrey E Hinton and David C Plaut,  “Using fast weights to deblur old memories,”  in Proceedings of the 9th Annual Conference of the Cognitive Science Society, 1987, pp. 177–186.
  * [17] Brady Houston and Katrin Kirchhoff,  “Continual learning for multi-dialect acoustic models.,”  in INTERSPEECH. ISCA, 2020.
  * [18] Samik Sadhu and Hynek Hermansky,  “Continual learning in automatic speech recognition.,”  in INTERSPEECH. ISCA, 2020.
  * [19] Joachim Fainberg,  Robust learning of acoustic representations from diverse speech data,  Ph.D. thesis, The University of Edinburgh, 2021.
  * [20] Heng-Jui Chang, Hung-yi Lee, and Lin-shan Lee,  “Towards Lifelong Learning of End-to-end ASR,”  in INTERSPEECH. ISCA, 2021.
  * [21] Daniel Povey et al.,  “Purely sequence-trained neural networks for ASR based on lattice-free MMI.,”  in INTERSPEECH. ISCA, 2016.
  * [22] Christopher Cieri, David Miller, and Kevin Walker,  “The Fisher corpus: A resource for the next generations of speech-to-text.,”  in LREC, 2004.
  * [23] Vassil Panayotov et al.,  “Librispeech: an ASR corpus based on public domain audio books,”  in ICASSP. IEEE, 2015.
  * [24] Anthony Rousseau, Paul Deléglise, Yannick Esteve, et al.,  “Enhancing the TED-LIUM corpus with selected data for language modeling and more TED talks.,”  in LREC, 2014.
  * [25] Jia Xin Koh et al.,  “Building the Singapore English national speech corpus,”  in INTERSPEECH, 2019.
  * [26] R. Ardila et al.,  “Common voice: A massively-multilingual speech corpus,”  in LREC, 2020.
  * [27] Daniel Povey et al.,  “The kaldi speech recognition toolkit,”  in ASRU. IEEE, 2011.
  * [28] Daniel Povey et al.,  “Semi-orthogonal low-rank matrix factorization for deep neural networks.,”  in INTERSPEECH. ISCA, 2018.


# LIPSFUS: A neuromorphic dataset for audio-visual sensory fusion of lip
reading
††thanks: This research was partially supported by the Spanish grants MINDROB
(PID2019-105556GB-C33/AEI/10.13039/501100011033) and SMALL
(PCI2019-111841-2/AEI/10.1309/501100011033) projects. E. P.-F. was supported
by a ”Formación de Personal Universitario” Scholarship from the Spanish
Ministry of Education, Culture and Sport. Authors thank the Ulster University
people and their HPC. A. L-B thanks the Salvador de Madariaga mobility
program.

A. Rios-Navarro1, E. Piñero-Fuentes1, S. Canas-Moreno1, A. Javed2, J. Harkin2
and A. Linares-Barranco1 1Robotics and Technology of Computers Lab. I3US.
SCORE. University of Seville. Seville, Spain  2School of Computing, Eng &
Intel. Sys. Magee campus. Derry, UK<EMAIL_ADDRESS>
###### Abstract

This paper presents a sensory fusion neuromorphic dataset collected with
precise temporal synchronization using a set of Address-Event-Representation
sensors and tools. The target application is the lip reading of several
keywords for different machine learning applications, such as digits, robotic
commands, and auxiliary rich phonetic short words. The dataset is enlarged
with a spiking version of an audio-visual lip reading dataset collected with
frame-based cameras. LIPSFUS is publicly available and it has been validated
with a deep learning architecture for audio and visual classification. It is
intended for sensory fusion architectures based on both artificial and spiking
neural network algorithms.

###### Index Terms:

Neuromorphic dataset, sensory fusion, dynamic vision sensor, neuromorphic
auditory sensor

##  I Introduction

Sensor fusion is known as the process of combining sensor data derived from
several sources of the same reality such that the fused information has less
uncertainty than when these sources are used individually. In [1] authors
demonstrated that combining radar and visual information improves the accuracy
in vehicle detection systems, where radar information is used to focus on the
important part of the visual information. In the healthcare field sensory
fusion improves decision-making when combining data from different sensors,
like in [2], where up to eight data sources are combined for the detection of
diabetes. In urban search and rescue robots the sensory fusion goes further
and combines proprioceptive (inertial measurement unit and tracks odometry)
and exteroceptive sensors (omnidirectional camera and rotating laser
rangefinder) to improve the accuracy [3]. In [4] a review of sensory fusion
for quality control in manufacturing is presented, where different kinds of
sensors are combined including visual, acoustic, laser, vibration, thermal,
etc. In general, data fusion is a challenging task because of a several
difficulties. The majority of these difficulties arise from the data being
fused, imperfection and diversity of the sensor technologies, and the nature
of the application environment as data imperfection, outliers and spurious,
conflicting, modality, correlation, alignment, association data, operational
timing, etc. [5].

Living organisms usually have several sensory mechanism to interact with the
real world. Those with neural architectures, learn from the experience taken
from sensory systems and their combination or fusion. Neuromorphic engineering
is devoted to study these and others neural systems in biology by the
implementation of engineering systems that mimics those present in biology
[6]. One of those neural architectures is in charge of audio-visual sensory
fusion, where the temporal synchrony is one of the strongest binding cues in
multi-sensory perception [7][8], where the consideration of a temporal window
is of utmost importance for this type of sensory fusion [9]. This window
temporal length decreases in humans with the age [10].

Audio-visual sensory fusion developments with neuromorphic engineering open
the range of applications i.e, Mobile robotics, Internet of Things (IoT), Edge
Computing, etc where low latency and low power consumption are important
factors. The use of spiking sensors and spiking neural networks will
considerably improve these characteristics over conventional sensors and
artificial neural networks.

This paper is focused on the collection of a neuromorphic audio-visual dataset
for machine learning applications around lip reading. It takes into account
the temporal synchronization of the measured data using specific neuromorphic
sensors and hardware tools. The dataset is collected from people of different
nationalities and age, speaking the number of Natural Language Processing
(NLP) words and one English-language pangram ”The quick brown fox jumps over
the lazy dog” for training of learning architectures. The dataset is publicly
available and it has been tested, for validation, with convolutional neural
networks. To the best of our knowledge, there is no previous neuromorphic
dataset recorded with spiking sensors and specific logic to ensure temporal
synchronization.

##  II Materials and methods

This section explains the setup for the dataset collection that has been
recorded directly from neuromorphic sensors while maintaining the time
synchronization of the visual and audio parts. It also explains how the BBC
dataset has been converted to the spikes domain.

###  II-A LIPSFUS recording setup

The Neuromorphic Auditory Sensor (NAS) [11] and the Neuromorphic Dynamic
Vision Sensor (DVS) [12] has been used to record the LIPSFUS dataset. Both
sensors generate spike information at their outputs and both are encoded in
AER (Address-Event-Representation) format at their outputs using a digital
parallel bus (16-bit maximum in our setup) with a 2-bit asynchronous
handshake. The digital words encode the identifier of the emitter neuron, in
the corresponding sensor, called the address. The length of an address is
fixed by the number of neurons in the sensor: pixels for the DVS or channels
for the NAS. In our setup, it is used the DVS developed at IMSE-CNM in
Seville, called cnmDV. This DVS has 128x128 pixels, so the output parallel AER
bus requires 15-bit for the addresses plus the polarity bit. For the audition,
the selected NAS has been obtained from the openNAS tool [13] and synthesized
for the Spartan6 FPGA of the AER-Node board [14]. A binaural sensor composed
of two identical banks of 64 band-pass filter per ear has been selected, with
the cut-off frequencies shown in figure 2 in the range from 18,91Hz to
20,81KHz, with an average error of 0,005% and a standard deviation of 0,001
along the 64 channels. For this audio sensor, the AER bus is sending addresses
of 8-bit length (6-bit to identify an active channel, 1-bit for polarity and
1-bit for left or right filter bank). The input of the NAS comes from a stereo
set of ear-shaped microphones that mimics a human head. This is the 3DIO Free
Space XLR Binaural setup [15]. These two neuromorphic sensors produce spikes
with their own temporal distributions, that intrinsically depend on the sensed
activity. An AER-tool [16] is used for merging the two sensors’ activity in
such a way that the temporal distribution is maintained for both sensors. This
requires the AER-merger to add 1-bit into the AER bus to distinguish between a
visual spike or an audio spike. The most significant bit is set to ’0’ for a
DVS event, and to ’1’ for a NAS event. Therefore, the full range of 16-bit
allowed by the hardware is used in this setup. The output of the AER-merger is
then connected to an AER-monitor board [17], that assigns a timestamp mark to
each received event, regardless of which sensor it belongs to, before queuing
the data (address and timestamp) for sending USB packets to a computer running
jAER, which stores .AEDAT files for each sample of the dataset.

Figure 1: LIPSFUS recording dataset setup. Left figure shows the sensors and
boards distribution and the right figure shows the setup sensor distances.
Figure 2: Cut-off frequencies of the binaural NAS 64-channels. Ideals versus
real implemented ones on FPGA with error in %.

Figure 1 on the left shows the distribution of the neuromorphic sensors (cnmDV
and NAS), the AER-merger platform and the AER-monitor. In addition, another
neuromorphic sensor, DV-Xplorer-Lite-346-USB, can be seen in the figure, but
it has not been included in the dataset presented in this paper. It is
incorporated in the setup for future use in applications that requires higher
visual resolutions. The right side of the figure shows the distances of each
neuromorphic sensor to the person to be recorded. The cnmDV is placed at 45cm
from the person face, while the 3DIO microphones are placed at 75cm and they
can be oriented at different angles (0º, -45º, -90º, 45º and 90º in our
dataset). The microphones are placed in the holes of two silicone ears that
simulate the human ear, and both are separated at the same distance as in an
average human head. The dataset is recorded in two different environments:
noisy and quiet. The noisy environment consisted of a glass meeting room,
where the air-conditioning system was running, and behind the window was a car
park and a main street in the city. In contrast, the quiet environment
consisted of a small room with acoustically insulated walls and a door. The
lighting condition in both environments were kept similar. The dataset, for
each of the environments, consisted of 22 persons of 5 different nationalities
(Indian, Iranian, Irish, Pakistani, Spanish), of both genders and aged between
6 and 61 years.

The dataset consists of a series of words that have been selected from
different challenges or are considered to be of interest in the area of
language processing. The words in the dataset are:

  * •

Spoken digits: One, two, three, four, five, six, seven, eight, nine, zero and
o [18].

  * •

Robotic commands: Yes, no, up, down, left, right, on, off, stop and go [19].

  * •

Bed, bird, cat, dog, happy, house, Marvin, Sheila, tree, wow [20].

  * •

About, border, forward, missing, press, short, threat, young [21].

  * •

The quick brown fox jumps over the lazy dog [22].

Each participant was reading each word (or sentence) when it is shown in a
presentation with 2 seconds delay between words. An AEDAT file was recorded
for the whole presentation from the word ”one” to the ”fox” sentence. This is
repeated five times per participant placing the ears at the orientations
commented before. Therefore, a total of 5 different AEDAT recordings are
stored for the each person. A repository with access to these files is
available on GitHub[23].

###  II-B BBC-LIPSFUS dataset

The forth set of words recorded with the previous setup are also part of the
”Lips reading in the wild” dataset from BBC [21, 24, 25]. In this work, this
set of words has been converted to a neuromorphic format and made available
for training spiking learning architectures. This dataset includes 1k
sentences with 500 different words taken from news and interviews (MP4 videos)
from BBC channels with 1k different speakers[26]. Each MP4 video has 29 frames
(1.16 seconds) and selected word is in its temporal center. An important
difference from our recorded dataset is that our speakers pronounced each word
/ sentence in an isolated way. Nevertheless, for the BBC dataset, the words
are part of a sentence and they are pronounced in a natural conversation. In
order to properly extract the required word from each MP4, we have used the
IBM Watson Studio engine [27], as it was used in [21] for validation, to
extract the precise timestamps where our chosen words start and end at each
MP4. Using these timestamps, each MP4 was cut to contain only the right word,
in an isolated way, as it is in our recorded dataset. These cut MP4s were then
used for the neuromorphic conversion. The same neuromorphic auditory sensor
(NAS) used in the previous recordings was connected to the audio output of a
computer and used to capture .AEDAT files in jAER while the computer was
playing each MP4 file of the dataset for the 8 selected words and 1k speakers
of each word. For the visual neuromorphic representation of the MP4 videos,
the ESIM [28] has been used, which also produces AEDAT files for each of the
8k given inputs. ESIM behaves as the DVS for the presented input video.

###  II-C Dataset validation

This dataset has been validated by performing a classification task on a
subset of the words in the dataset, the spoken digits. This paper proposes an
Artificial Neural Network (ANN) based learning model to classify extracted
neuromorphic words dataset. Currently, the authors are also working on the
design of a Spiking Neural Network (SNN) to perform the classification task
using the spiking information from the sensors.

####  II-C1 Data conversion and augmentation

This work has considered Convolutional Neural Network (CNN), to perform the
classification of the spoken digits, the first step is to convert the spiking
information into a data type that the CNN can process i.e, an histogram based
image. As we have information from two different sensors that generate
different nature of temporal information. Therefore a separate conversion is
designed for each sensory data.

The NAS used for the recording of this dataset consists of 64 stereo channels.
The number of channels corresponds to the number of spike-based filters used
to decompose the signal into different frequencies. As this sensor is stereo,
it has 128 channels in total. At the output of each of these spike-based
filters there are two neurons, one encoding the positive part of the signal
and one encoding the negative part of the same signal. Therefore, this NAS
contains 256 neurons that emit information at its outputs. For the conversion
of the pulsed information of the NAS, we proceeded to generate the sonogram of
each spoken digit sample. The sonogram shows the activity of each sensor
channel over time. To calculate the activity of each channel, a time window is
used in which the number of spikes emitted by each channel is accumulated and
grouped and the channel value is set to that accumulation for that time
window. Thus, the information of each channel is encoded in the intensity of
spikes produced in each time window. The figure 3 (left) shows the sonogram of
a sample where on the x-axis is the time, on the y-axis are the neurons of the
channels and the colour represents the intensity of the channel.

Figure 3: Stereo sonogram of a spoken digit sample (left). The spoken word
information is delimited by the red box. Mono sonogram for right (top-right)
and left (bottom right) channels cut from the red box.

As can be seen in figure 3, the spoken word information is at the beginning of
the recording (delimited by the red rectangle). It can be seen that there is a
large part of the recording with some activity in the channels, which
corresponds to some noise. Part of this noise is from the environment as well
as noise generated by the sensor at the output of the filters. This work
focused on the part of the recording that contains the most activity, the part
delimited by the red rectangle, and we will also divide the stereo recording
to obtain two mono samples as shown in the figure 3 (right).

In order to obtain more samples from the recorded dataset, a data augmentation
process has been carried out. This is done by starting with a temporal window
that is adjusted to the region where the spoken word activity is located and
enlarging that region by allowing some noise prior to the spoken word
activity. Then, to generate new samples, this region of noise prior to the
activity is reduced a little and enlarged by the same amount at the end of the
spoken word activity allowing for some noise. This process was repeated 10
times per sample. The Number of Samples (NS) in the audio dataset is 26,620
(see equation 1, for 22 People (P), 11 Words (W), 2 channel audio Sensor (S),
10 Data Augmentation (DA) techniques and 5 different microphones Orientation
(O).

$\displaystyle NS=P\times(W\times S+W\times S\times DA)\times O$ (1)
$\displaystyle=22\times(11\times 2+11\times 2\times 10)\times 5$

The DVS used to record this dataset has a resolution of 128x128 neurons that
emit positive and negative spikes depending on the change in brightness that
the scene represents. For the conversion of the spiking information from the
DVS sensor, only the lip area of the original sample was trimmed, discarding
the rest of the information from the scene. The entire recording is then
divided into equal temporal regions and the spikes are accumulated to generate
a histogram equivalent to an image. The activity of each pixel or neuron in
the sensor is encoded in the colour intensity of the pixel in the generated
image. This approach provides a sequence of images that capture the movement
of the lips when saying the corresponding word. Figure 4 shows how this
conversion is performed.

Figure 4: Conversion process from spiking information to histograms.

A similar data augmentation process has been repeated for the audio dataset to
generate more samples. A temporal window is adjusted to the region where the
lip movement activity is found when the word is spoken. This region is
enlarged thus allowing some noise before the activity of the spoken word. To
generate new samples, the selected region of noise prior to the activity is
reduced marginally and enlarged by the same amount at the end of the speech
activity allowing for some noise. This process was repeated 10 times per
sample. The number of samples in the visual dataset is 13,310 (see equation
2):

$\displaystyle NS=P\times(W+W\times DA)\times O$ (2)
$\displaystyle=22\times(11+11\times 10)\times 5$

####  II-C2 Visual and Audio networks

As mentioned at the end of the introduction section, to validate the dataset,
two CNNs have been used to classify the spoken digits, one for the audio part
and one for the visual part. The aim is to demonstrate with a simple
application that the data collected in this dataset are valid for use by the
community. The CNN architecture that will be trained to classify the spoken
digits using the audio information from the sonograms (via NAS sensor) has the
following layout:

  * •

Input(64,32,1) $\,\to\,$ Conv2D(3,3,100) $\,\to\,$ MaxPool(2, 2) $\,\to\,$
Conv2D(2,2,200) $\,\to\,$ MaxPool(2,2) $\,\to\,$ Dropout(0.5) $\,\to\,$
Dense(10).

In contrast, the CNN architecture that will be trained to classify the spoken
digits using the visual information from the sequence of histograms (via the
DVS sensor) has the following configuration:

  * •

Input(33,49,6) $\,\to\,$ Conv2D(3,3,16) $\,\to\,$ MaxPool(2,2) $\,\to\,$
Conv2D(2,2,32) $\,\to\,$ MaxPool(2,2,2) $\,\to\,$ Conv2D(2,2,64) $\,\to\,$
MaxPool(2,2,2) $\,\to\,$ Dropout(0.5) $\,\to\,$ Dense(10).

Both networks have been trained using the Keras+Tensorflow framework.

##  III Results

The Keras+Tensorflow framework was used to train the CNNs described in the
previous section. For the different training sessions performed, the datasets
have been partitioned into 70% for training, 20% for validation and the
remaining 10% has been used to test the network once it has been trained. The
samples of the 10% used for testing belong to a subject whose samples have not
been used in the training phase, thus ensuring that during the test all the
samples are totally unknown to the trained network.

Table I shows the results obtained when testing the audio (A_) and the visual
(V_) CNNs across different training scenarios. Each row of the table
represents the result of testing the CNN using the specified number of
training epochs (Tr_ep) and batch size (B_size). The learning ratio has been
set to the same value for all training epochs, in this case to 0.001.

Tr_ep; B_size | A_Loss | A_Acc | V_Loss | V_Acc
---|---|---|---|---
100; 32 | 0.661 | 0.853 | 1.266 | 0.685
200; 32 | 0.704 | 0.882 | 1.191 | 0.694
100; 64 | 0.589 | 0.850 | 1.030 | 0.709
200; 64 | 0.475 | 0.896 | 0.997 | 0.727
100; 128 | 0.605 | 0.832 | 1.099 | 0.701
200; 128 | 0.591 | 0.824 | 1.145 | 0.698
TABLE I: Test audio and visual results using different training parameters.
The learning rate value is 0.001 for all.

In the table the best test loss and accuracy values are highlighted for both
cases. These results do not seem promising initially when compared to other
applications that can be found in the literature, however the aim of this work
is not to design a network model that is optimal in the classification of the
spoken digits, but rather to validate the dataset in order to make it
available to the community.

##  IV Conclusion

This paper presents a dataset in which there is visual and auditory
information when speaking a set of words. The visual information consists of
the lip movement when the subject articulates a word, while the auditory
information refers to the sound that the subject makes when pronouncing a
word. This information has been captured by the NAS sensor for the audio and
the DVS sensor for the visual part, synchronising the information from both
sensors with the same timing source, therefore, achieving a perfect temporal
sequencing of the spikes generated by both sensors. To validate the dataset, a
classification task has been performed on a subset of words from the dataset
using Deep Learning algorithms. Although the reported classification results
are not the same as those obtained with Deep Learning techniques in the
literature, they are satisfactory to validate that the dataset samples are
useful and suitable for the research community to implement sensory fusion
algorithms.

## References

  * [1] G. Alessandretti, A. Broggi, and P. Cerri, “Vehicle and guard rail detection using radar and vision data fusion,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 8, no. 1, pp. 95–105, 2007.
  * [2] A. A. Abd El-Latif, B. Ihnaini, M. A. Khan, T. A. Khan, S. Abbas, M. S. Daoud, M. Ahmad, and M. A. Khan, “A smart healthcare recommendation system for multidisciplinary diabetes patients with data fusion based on deep ensemble learning,” _Computational Intelligence and Neuroscience. Hindawi_ , vol. 2021, no. 4243700, pp. 95–105, 2021.
  * [3] V. Kubelka, L. Oswald, F. Pomerleau, F. Colas, T. Svoboda, and M. Reinstein, “Robust data fusion of multimodal sensory information for mobile robots,” _Journal of Field Robotics_ , vol. 32, no. 4, pp. 447–473, 2015. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21535
  * [4] J. Akhavan and S. Manoochehri, “Sensory data fusion using machine learning methods for in-situ defect registration in additive manufacturing: A review,” in _2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS)_ , 2022, pp. 1–10.
  * [5] B. Khaleghi, A. Khamis, F. O. Karray, and S. N. Razavi, “Multisensor data fusion: A review of the state-of-the-art,” _Information Fusion_ , vol. 14, no. 1, pp. 28–44, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1566253511000558
  * [6] C. Mead, “Neuromorphic electronic systems,” _Proceedings of the IEEE_ , vol. 78, no. 10, pp. 1629–1636, 1990.
  * [7] A. J. King, “Multisensory integration: Strategies for synchronization,” _Current Biology_ , vol. 15, no. 9, pp. R339–R341, 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0960982205004227
  * [8] C. Spence and S. Squire, “Multisensory integration: Maintaining the perception of synchrony,” _Current Biology_ , vol. 13, no. 13, pp. R519–R521, 2003\. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0960982203004457
  * [9] V. van Wassenhove, K. W. Grant, and D. Poeppel, “Temporal window of integration in auditory-visual speech perception,” _Neuropsychologia_ , vol. 45, no. 3, pp. 598–607, 2007, advances in Multisensory Processes. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S002839320600011X
  * [10] F. Kopp, “Audiovisual temporal fusion in 6-month-old infants,” _Developmental Cognitive Neuroscience_ , vol. 9, pp. 56–67, 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1878929314000036
  * [11] A. Jiménez-Fernández, E. Cerezuela-Escudero, L. Miró-Amarante, M. J. Domínguez-Morales, F. de Asís Gómez-Rodríguez, A. Linares-Barranco, and G. Jiménez-Moreno, “A binaural neuromorphic auditory sensor for fpga: A spike signal processing approach,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 28, no. 4, pp. 804–818, 2017.
  * [12] T. Serrano-Gotarredona and B. Linares-Barranco, “A 128$\,\times$ 128 1.5contrast sensitivity 0.9dynamic vision sensor using transimpedance preamplifiers,” _IEEE Journal of Solid-State Circuits_ , vol. 48, no. 3, pp. 827–838, 2013.
  * [13] D. Gutierrez-Galan, J. Dominguez-Morales, A. Jimenez-Fernandez, A. Linares-Barranco, and G. Jimenez-Moreno, “Opennas: Open source neuromorphic auditory sensor hdl code generator for fpga implementations,” _Neurocomputing_ , 2021.
  * [14] A. Yousefzadeh, M. Jabłoński, T. Iakymchuk, A. Linares-Barranco, A. Rosado, L. A. Plana, S. Temple, T. Serrano-Gotarredona, S. B. Furber, and B. Linares-Barranco, “On multiple aer handshaking channels over high-speed bit-serial bidirectional lvds links with flow-control and clock-correction on commercial fpgas for scalable neuromorphic systems,” _IEEE Transactions on Biomedical Circuits and Systems_ , vol. 11, no. 5, pp. 1133–1147, 2017.
  * [15] Seesense, “Microphone, 3dio free space xlr binaural.” [Online]. Available: https://seesense.eu/shop/3dio-free-space-xlr-binaural-microphone/
  * [16] F. Gomez-Rodriguez, R. Paz, A. Linares-Barranco, M. Rivas, L. Miro, S. Vicente, G. Jimenez, and A. Civit, “Aer tools for communications and debugging,” in _2006 IEEE International Symposium on Circuits and Systems_ , 2006, p. 4 pp.
  * [17] R. Berner, T. Delbruck, A. Civit-Balcells, and A. Linares-Barranco, “A 5 meps $100 usb2.0 address-event monitor-sequencer interface,” in _2007 IEEE International Symposium on Circuits and Systems_ , 2007, pp. 2451–2454.
  * [18] P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” _arXiv preprint arXiv:1804.03209_ , 2018.
  * [19] [Online]. Available: https://developer.ibm.com/exchanges/data/all/speech-commands/
  * [20] [Online]. Available: https://www.kaggle.com/c/tensorflow-speech-recognition-challenge
  * [21] J. S. Chung and A. Zisserman, “Lip reading in the wild,” in _Computer Vision – ACCV 2016_ , S.-H. Lai, V. Lepetit, K. Nishino, and Y. Sato, Eds. Cham: Springer International Publishing, 2017, pp. 87–103.
  * [22] G. Yang, “Wide feedforward or recurrent neural networks of any architecture are gaussian processes,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [23] R. research group, “Lipsfus-event-driven-dataset.” [Online]. Available: https://github.com/RTC-research-group/LIPSFUS-Event-driven-dataset/
  * [24] J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, “Lip reading sentences in the wild,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2017.
  * [25] J. S. Chung and A. Zisserman, “Lip reading in profile,” in _British Machine Vision Conference_ , 2017.
  * [26] Oxford-BBC, “The oxford-bbc lip reading in the wild (lrw) datasett.” [Online]. Available: https://www.robots.ox.ac.uk/~vgg/data/lip_reading/lrw1.html
  * [27] “Ibm watson studio,” IBM cloud product, Nov. 2016. [Online]. Available: https://www.ibm.com/cloud/watson-studio
  * [28] H. Rebecq, D. Gehrig, and D. Scaramuzza, “Esim: an open event camera simulator,” in _Proceedings of The 2nd Conference on Robot Learning_ , ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. Morimoto, Eds., vol. 87. PMLR, 29–31 Oct 2018, pp. 969–982. [Online]. Available: https://proceedings.mlr.press/v87/rebecq18a.html