# A general result on the approximation of local conservation laws by nonlocal
conservation laws: The singular limit problem for exponential kernels

Giuseppe Maria Coclite  Polytechnic University of Bari, Department of
Mechanics, Mathematics, and Management, Via E. Orabona 4, 70125 Bari, Italy.
<EMAIL_ADDRESS>Jean-Michel Coron  Université Pierre et Marie
Curie, Laboratoire Jacques-Louis Lions, Place Jussieu 4, 75252 Paris, France.
<EMAIL_ADDRESS>Nicola De Nitti  Friedrich-Alexander-Universität
Erlangen-Nürnberg, Department of Mathematics, Chair of Applied Analysis
(Alexander von Humboldt Professorship), Cauerstr. 11, 91058 Erlangen, Germany.
<EMAIL_ADDRESS>Alexander Keimer  Institute for Transportation Studies
(ITS), University of California, Berkeley, 94720 Berkeley, CA, USA.
<EMAIL_ADDRESS>Lukas Pflug  Friedrich-Alexander-Universität Erlangen-
Nürnberg, Central Institute for Scientific Computing, Martensstr. 5a, 91058
Erlangen, Germany.  Friedrich-Alexander-Universität Erlangen-Nürnberg,
Department of Mathematics, Chair of Applied Mathematics (Continuous
Optimization), Cauerstr. 11, 91058 Erlangen, Germany<EMAIL_ADDRESS>
###### Abstract

We deal with the problem of approximating a scalar conservation law by a
conservation law with nonlocal flux. As convolution kernel in the nonlocal
flux, we consider an exponential-type approximation of the Dirac distribution.
This enables us to obtain a total variation bound on the nonlocal term. By
using this, we prove that the (unique) weak solution of the nonlocal problem
converges strongly in $C(L^{1}_{\text{loc}})$ to the entropy solution of the
local conservation law. We conclude with several numerical illustrations which
underline the main results and, in particular, the difference between the
solution and the nonlocal term.

###### keywords:

Nonlocal conservation laws, nonlocal flux, balance laws, singular limits,
approximation of local conservation laws, entropy solution.

###### MSC:

[2010]35L65

##  1 Introduction

Nonlocal conservation laws have been studied and analyzed quite intensively
over the last decade from an application point of view with a particular focus
on traffic flow [5, 20, 30, 45, 53, 37], supply chains [39, 55, 32],
pedestrian flow/crowd dynamics [21], opinion formation [2, 51], chemical
engineering [50, 57], sedimentation [6], conveyor belts [54] and more. For the
underlying dynamics existence and uniqueness [35, 40, 45, 44, 46, 12, 25],
(optimal) control problems [33, 4, 14, 19, 38, 24], and suitable numerical
schemes [1, 11, 13, 28, 52] have been analyzed.

In this work, “nonlocal” refers to the fact that the velocity $V$ of the
corresponding flux, i.e., $f(s)=V(s)s,\ s\in\mathbb{R}$, does not depend on
the solution locally at a given space point but on an integral of the solution
on a neighborhood.

First in [3] it has been observed that, at least numerically, there is some
hope that the solution of the nonlocal conservation law converges to the local
entropy solution when the nonlocal term approaches a Dirac delta. Positive
results in this direction were obtained in [59], provided that the limit
entropy solution is smooth and the convolution kernel is even, and in [42] for
a large class of nonlocal conservation laws under the assumption of having
monotone initial data. Under the assumption that the initial datum has bounded
total variation, is bounded away from zero and satisfies a one-sided Lipschitz
condition, a positive result was obtained in [17]. In [9], for an exponential
weight in the nonlocal term, is was shown – provided that the initial datum is
bounded away from zero and has bounded total variation (but without
monotonicity assumptions) – that, the nonlocal solutions converge (up to
subsequences) to weak solutions of the corresponding local conservation law;
they also showed that the limit is the unique entropy solution under the
additional assumption that $V$ is an affine function. More recently, in [8],
the result was extended to more general fluxes.

A viscous nonlocal conservation law with kernel of exponential type was
considered in [15]: as the nonlocal term together with the viscosity
approaches zero, the sequence of solutions converges to the local entropy
solution. The positive effect of viscosity in the nonlocal-to-local
approximation process was previously studied in [18, 16] for more general
compactly supported kernels (see also [10] in the case of more regular initial
data and linear velocity). In contrast to the proof in [18], which was based
on a priori estimates obtained by extensively using energy estimates for the
heat kernel and the Duhamel representation formula, in [15], the authors
established an energy estimate on the nonlocal term by relying just on the
special structure provided by the exponential kernel (as in Remark 3.1 below)
and used it to apply Tatar’s compensated compactness theory (see [58]).

In conclusion, although some progress has been made under quite restrictive
assumptions, a general theory is missing. Even more, [17] demonstrates via a
counterexample that a total variation blow-up of the solution of the nonlocal
conservation law can occur if the data is not bounded away from zero so that
the standard methods via compactness in $L^{1}$ seemed to be out of reach.

This is why, in this work, we focus on the nonlocal term: it turns out that
the nonlocal term itself satisfies a local transport equation with nonlocal
source (see Lemma 3.1) and we can use this to show a uniform total variation
bound (see Theorem 3.2). Thanks to the specific structure of the nonlocal term
this directly implies that also the solution of the conservation law converges
strongly in $L^{1}$ (see Theorem 4.1 and Corollary 4.1).

More precisely, in the present paper, we consider the following setting. For a
nonlocal parameter $\eta\in\mathbb{R}_{>0}$, let $q_{\eta}$ be the unique weak
solution (weak solutions are unique in the nonlocal setup) of the nonlocal
conservation law on $\mathbb{R}$

$\displaystyle\partial_{t}q_{\eta}(t,x)+\partial_{x}\big{(}V\big{(}W_{\eta}[q_{\eta}](t,x))q_{\eta}(t,x)\big{)}$
$\displaystyle=0$ $\displaystyle(t,x)\in\Omega_{T}$ $\displaystyle
q_{\eta}(0,x)$ $\displaystyle=q_{0}(x)$ $\displaystyle x\in\mathbb{R}$ with
${\Omega_{T}}\mathrel{\coloneqq}(0,T)\times\mathbb{R}$, supplemented by the
nonlocal term $W_{\\!\eta}$ with exponential weight $\displaystyle
W_{\eta}[q](t,x)$
$\displaystyle\coloneqq\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q(t,y)\,\mathrm{d}y$
$\displaystyle(t,x)\in\Omega_{T}$

and let $q$ be the entropy solution of the corresponding local conservation
law on $\mathbb{R}$

$\displaystyle\partial_{t}q(t,x)+\partial_{x}\big{(}V\big{(}q(t,x)\big{)}q(t,x)\big{)}$
$\displaystyle=0$ $\displaystyle(t,x)\in\Omega_{T}$ (1) $\displaystyle q(0,x)$
$\displaystyle=q_{0}(x)$ $\displaystyle x\in\mathbb{R}.$ (2)

For the “local theory” and corresponding Entropy solutions, we refer to [7,
31, 26, 36]. Finally, in Theorem 4.2, we prove the convergence of the nonlocal
solution to the local entropy solution when $\eta$ approaches zero, i.e., the
nonlocal term approaches a Dirac distribution

$q_{\eta}\stackrel{{\scriptstyle\eta\rightarrow 0}}{{\longrightarrow}}q\
\text{ in }C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}.$

We do this by first analyzing the nonlocal term $W_{\eta}[q_{\eta}]$ which –
as can be shown – satisfies its own transport equation with a nonlocal source
and possess a uniform total variation bound. Thanks to the relation
$\eta\partial_{2}W_{\eta}[q_{\eta}]\equiv W_{\eta}[q_{\eta}]-q_{\eta}$ it
immediately follows also the strong convergence of $q$ to a weak solution of
the local conservation law. Then, we can use [8] to obtain that the solution
is indeed also entropic. Even more, we also obtain that the nonlocal term
$W_{\eta}[q_{\eta}]$ also converges to the local entropy solution.

Our “nonlocal-to-local convergence” result closes the gap between local and
nonlocal modelling of phenomena governed by conservation laws; moreover, it
provides a way of defining the entropy admissible solutions of local
conservation laws as limits of weak solutions to nonlocal conservation laws,
which usually do not require an entropy condition for uniqueness (see [23, 41,
44, 45]). This kind of singular limit would be an alternative to the classical
vanishing viscosity approach (see [36, 7, 26] and references therein). In the
case of a nonlocal approximation without artificial viscosity, no smoothing
phenomena happen and the character of the approximating equation remains
somewhat “hyperbolic” (finite propagation of mass, but infinite propagation of
information).

Such a convergence result would also give additional insights into questions
related to control theory (see [4]), in the spirit of [22, 34, 29, 47].
Showing control results for nonlocal conservation laws might be easier due to
the fact that these equations are invertible in time, so that one can actually
go back from a current state to the initial datum. Optimal control problems
might also become mathematically more approachable as the problem with adjoint
equations and shocks of the local equations prohibiting differentiability in a
certain local framework might be resolvable in the nonlocal theory and one
might then just consider the limit controls when the nonlocal term approaches
a Dirac.

##  2 Preliminary results on nonlocal conservation laws

######  Definition 2.1 (The nonlocal conservation law and the weak solution).

Let $T\in\mathbb{R}_{>0}$ be given. We consider for $\eta\in\mathbb{R}_{>0}$
the following nonlocal conservation law in the “density”
$q_{\eta}:{\Omega_{T}}\rightarrow\mathbb{R},\
{\Omega_{T}}\mathrel{\coloneqq}(0,T)\times\mathbb{R}$

$\displaystyle\partial_{t}q_{\eta}(t,x)+\partial_{x}\big{(}V\big{(}W_{\eta}[q_{\eta}](t,x))q_{\eta}(t,x)\big{)}$
$\displaystyle=0$ $\displaystyle(t,x)\in\Omega_{T}$ (3) $\displaystyle
q_{\eta}(0,x)$ $\displaystyle=q_{0}(x)$ $\displaystyle x\in\mathbb{R}$ (4)
supplemented by the nonlocal term $W_{\\!\eta}$ $\displaystyle
W_{\eta}[q_{\eta}](t,x)$
$\displaystyle\coloneqq\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{\eta}(t,y)\,\mathrm{d}y$
$\displaystyle(t,x)\in\Omega_{T}.$ (5)

We call $q_{0}:\mathbb{R}\rightarrow\mathbb{R}$ initial datum and
$W_{\eta}[q_{\eta}]:{\Omega_{T}}\rightarrow\mathbb{R}$ the nonlocal impact
affecting the velocity function $V:\mathbb{R}\rightarrow\mathbb{R}$ of the
nonlocal conservation law. We say that $q_{\eta}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ is a weak solution
for $q_{0}\in L^{1}_{\textnormal{loc}}(\mathbb{R})$ and
$\eta\in\mathbb{R}_{>0}$ iff $\forall\varphi\in
C^{1}_{\text{c}}((-42,T)\times\mathbb{R})$ it holds that

$\iint_{{\Omega_{T}}}\partial_{t}\varphi(t,x)q_{\eta}(t,x)+\partial_{x}\varphi(t,x)V(W_{\\!\eta}[q_{\eta}](t,x))q_{\eta}(t,x)\,\mathrm{d}x\,\mathrm{d}t+\int_{\mathbb{R}}\varphi(0,x)q_{0}(x)\,\mathrm{d}x=0.$
(6)

For the analysis and well-posedness, we require the following not restrictive
assumptions:

######  Assumption 2.1 (Assumptions on input data).

The involved functions in Definition 2.1 satisfy

$\bullet\ q_{0}\in L^{\infty}(\mathbb{R};\mathbb{R}_{\geq 0})\cap
TV(\mathbb{R})\qquad\bullet\ V\in
W^{1,\infty}_{\textnormal{loc}}(\mathbb{R}):\ V^{\prime}(s)\leq 0\ \forall
s\in\big{(}\operatorname*{ess-\inf}_{x\in\mathbb{R}}q_{0}(x),\|q_{0}\|_{L^{\infty}(\mathbb{R})}\big{)}.$

######  Theorem 2.1 (Existence and uniqueness of weak solutions and maximum
principle).

Given Assumption 2.1 there exists a unique weak solution $q\in
C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}\cap
L^{\infty}((0,T);L^{\infty}(\mathbb{R}))\cap L^{\infty}((0,T);TV(\mathbb{R}))$
of the nonlocal conservation law in Definition 2.1 and the following maximum
principle is satisfied

$\displaystyle\operatorname*{ess-\inf}_{x\in\mathbb{R}}q_{0}(x)\leq
q(t,x)\leq\|q_{0}\|_{L^{\infty}(\mathbb{R})}\text{ a.e.
}(t,x)\in{\Omega_{T}}.$ (7)

###### Proof.

See [40, Theorem 2.20 & Theorem 3.2 & Corollary 4.3]. ∎

######  Remark 2.1 (Generalization of the assumptions on the velocity function
$V$).

The assumption on $V$ being monotonically decreasing (see Assumption 2.1) can
be changed to $V$ monotonically increasing as long as one also changes the
nonlocal range for $q\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ as

$W_{\eta}[q](t,x)\mathrel{\coloneqq}\int_{-\infty}^{x}\exp(\tfrac{y-x}{\eta})q(t,y)\,\mathrm{d}y,\quad(t,x)\in{\Omega_{T}}.$

Analogously, the results can be extended to hold also for non-positive initial
datum when changing the nonlocal term accordingly. We do not go into details.

Even more when assuming that $V^{\prime}(s)s$ has a sign for all
$s\in\mathbb{R}$, one does not need even a maximum principle to be satisfied
and thus the initial datum can be chosen arbitrarily in
$L^{\infty}(\mathbb{R})\cap TV(\mathbb{R})$ (no sign restrictions). However,
then one does not obtain convergence of $q_{\eta}$ but of $W_{\eta}$ which
remains essentially bounded and for which the total variation bound derived in
Theorem 3.2 still holds. However, Theorem 4.2 is not directly applicable and
we are left with that the limit is a weak solution. Compare also Remark 3.2.

##  3 Total variation bound on the nonlocal term

As we will tackle the convergence first in the nonlocal term,
$W_{\eta}[q_{\eta}]$, we deduce a transport equation with a nonlocal source
which will enable us to study $W_{\eta}[q_{\eta}]$ without $q_{\eta}$ itself.

######  Lemma 3.1 (The transport equation with nonlocal source satisfied by
the nonlocal term).

Given the dynamics in Definition 2.1, the nonlocal term $W_{\eta}[q_{\eta}]$
as in Eq. 8 is Lipschitz-continuous and satisfies the following transport
equation with nonlocal source in the strong sense

$\displaystyle\partial_{t}W_{\eta}(t,x)+V(W_{\eta}(t,x))\partial_{x}W_{\eta}(t,x)$
$\displaystyle=-\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})V^{\prime}(W_{\eta}(t,y))\partial_{y}W_{\eta}(t,y)W_{\eta}(t,y)\,\mathrm{d}y$
$\displaystyle(t,x)\in{\Omega_{T}}$ (8) $\displaystyle W_{\eta}(0,x)$
$\displaystyle=\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{0}(y)\,\mathrm{d}y$
$\displaystyle x\in\mathbb{R}.$ (9)

In particular, for $\eta\in\mathbb{R}_{>0}$, we have $W_{\eta}\in
W^{1,\infty}({\Omega_{T}})$.

###### Proof.

We first show that $W_{\eta}[q_{\eta}]$ is Lipschitz-continuous. To this end,
recall the definition Eq. 8 and compute for $(t,x)\in{\Omega_{T}}$

$\displaystyle\partial_{x}W_{\eta}[q_{\eta}](t,x)$
$\displaystyle=\partial_{x}\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{\eta}(t,y)\,\mathrm{d}y=\tfrac{1}{\eta}W_{\eta}[q_{\eta}](t,x)-\tfrac{1}{\eta}q_{\eta}(t,x).$
(10)

However, as $\eta\in\mathbb{R}_{>0}$, $W_{\eta}[q_{\eta}]\in
L^{\infty}(({\Omega_{T}}))$ and $q_{\eta}\in L^{\infty}(({\Omega_{T}}))$
thanks to Theorem 2.1, we obtain the uniform boundedness on the spatial
derivative. The time derivative is slightly more tricky. Due to the lack of
regularity we use the method of characteristics analyzed in [40, Lemma 2.6] to
write down the solution $q_{\eta}$ and have on $(t,x)\in{\Omega_{T}}$

$\displaystyle\partial_{t}W_{\eta}[q_{\eta}](t,x)$
$\displaystyle=\partial_{t}\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{\eta}(t,y)\,\mathrm{d}y$
(11)
$\displaystyle=\partial_{t}\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{0}(\xi(t,y;0))\partial_{2}\xi(t,y;0)\,\mathrm{d}y$
(12)
$\displaystyle=\partial_{t}\tfrac{1}{\eta}\int_{\xi(t,x;0)}^{\infty}\exp\Big{(}\tfrac{x-\xi(0,z;t)}{\eta}\Big{)}q_{0}(z)\,\mathrm{d}z$
(13)
$\displaystyle=-\tfrac{1}{\eta^{2}}\int_{\xi(t,x;0)}^{\infty}\exp\Big{(}\tfrac{x-\xi(0,z;t)}{\eta}\Big{)}q_{0}(z)\partial_{3}\xi(0,z;t)\,\mathrm{d}z-\tfrac{1}{\eta}q_{0}(\xi(t,x;0))\partial_{1}\xi(t,x;0).$
(14)

Recalling some nice properties of the characteristics [40, Lemma 2.6] and in
particular

$\displaystyle\partial_{3}\xi(0,\xi(t,y;0);t)$
$\displaystyle=V(W_{\eta}[q_{\eta}](t,y))$
$\displaystyle\forall(t,y)\in{\Omega_{T}}$
$\displaystyle\partial_{1}\xi(t,y;0)$
$\displaystyle=-\partial_{2}\xi(t,y;0)V(W_{\eta}[q_{\eta}](t,y))$
$\displaystyle\forall(t,y)\in{\Omega_{T}}$

we obtain by continuing Eq. 14

$\displaystyle\partial_{t}W[q_{\eta}](t,x)$
$\displaystyle=-\tfrac{1}{\eta^{2}}\int_{\xi(t,x;0)}^{\infty}\exp\Big{(}\tfrac{x-\xi(0,z;t)}{\eta}\Big{)}q_{0}(z)\partial_{3}\xi(0,z;t)\,\mathrm{d}z-\tfrac{1}{\eta}q_{0}(\xi(t,x;0))\partial_{1}\xi(t,x;0)$
$\displaystyle=-\tfrac{1}{\eta^{2}}\int_{x}^{\infty}\exp\Big{(}\tfrac{x-y}{\eta}\Big{)}q_{0}(\xi(t,y;0))\partial_{3}\xi(0,\xi(t,y;0)\partial_{2}\xi(t,y;0)\,\mathrm{d}y$
$\displaystyle\qquad+\tfrac{1}{\eta}q_{0}(\xi(t,x;0))\partial_{2}\xi(t,x;0)V(W_{\eta}[q_{\eta}](t,x))$
$\displaystyle=-\tfrac{1}{\eta^{2}}\int_{x}^{\infty}\exp\Big{(}\tfrac{x-y}{\eta}\Big{)}q_{\eta}(t,y)V(W_{\eta}[q_{\eta}](t,y))\,\mathrm{d}y+\tfrac{1}{\eta}q_{\eta}(t,x)V(W_{\eta}[q_{\eta}](t,x)).$

This expression is essentially bounded for $\eta\in\mathbb{R}_{>0}$ so that we
obtain the differentiability. Next, we show that the nonlocal operator indeed
satisfies the Cauchy problem in Eqs. 8 to 9. Using the identity compute for
$W_{t}$ above we have for the left hand side of Eq. 8 and
$(t,x)\in{\Omega_{T}}$

$\displaystyle\partial_{t}W_{\eta}[q_{\eta}](t,x)+V(W_{\eta}[q_{\eta}](t,x))\partial_{x}W_{\eta}[q_{\eta}](t,x)$
$\displaystyle=\tfrac{1}{\eta}q_{\eta}(t,x)V(W_{\eta}[q_{\eta}](t,x))-\tfrac{1}{\eta^{2}}\int_{x}^{\infty}\exp\big{(}\tfrac{x-y}{\eta}\big{)}q_{\eta}(t,y)V(W_{\eta}[q_{\eta}](t,y))\,\mathrm{d}y$
$\displaystyle\qquad+V(W_{\eta}[q_{\eta}](t,x))\big{(}\tfrac{1}{\eta}W_{\eta}[q_{\eta}](t,x)-\tfrac{1}{\eta}q_{\eta}(t,x)\big{)}$
$\displaystyle=V(W_{\eta}[q_{\eta}](t,x))\tfrac{1}{\eta}W_{\eta}[q_{\eta}](t,x)$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{x}^{\infty}\exp\big{(}\tfrac{x-y}{\eta}\big{)}\big{(}W_{\eta}[q_{\eta}]t,y)-\eta\partial_{y}W_{\eta}[q_{\eta}](t,y))V(W_{\eta}[q_{\eta}](t,y)\big{)}\,\mathrm{d}y$
$\displaystyle=V(W_{\eta}[q_{\eta}](t,x))\tfrac{1}{\eta}W_{\eta}[q_{\eta}](t,x)$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{x}^{\infty}\exp\big{(}\tfrac{x-y}{\eta}\big{)}W_{\eta}[q_{\eta}]t,y)V(W_{\eta}[q_{\eta}](t,y)\big{)}\,\mathrm{d}y$
$\displaystyle\qquad+\tfrac{1}{\eta}\int_{x}^{\infty}\exp\big{(}\tfrac{x-y}{\eta}\big{)}\partial_{y}W_{\eta}[q_{\eta}](t,y)V(W_{\eta}[q_{\eta}](t,y)\big{)}\,\mathrm{d}y$
$\displaystyle=-\tfrac{1}{\eta}\int_{x}^{\infty}\exp\big{(}\tfrac{x-y}{\eta}\big{)}V^{\prime}(W_{\eta}[q_{\eta}](t,x))\partial_{y}W_{\eta}[q_{\eta}](t,y)W_{\eta}[q_{\eta}](t,y)\,\mathrm{d}y$

where we have used two times the identity in Eq. 10 and integration by parts.
However, the last term is indeed the right hand side of Eq. 8. The nonlocal
term $W_{\eta}$ also satisfies the initial datum in Eq. 9 which is a direct
consequence of the definition of $W_{\eta}$ in Eq. 5 when plugging in $t=0$
(this is possible as the solution is regular enough, i.e., $q_{\eta}\in
C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}$. ∎

######  Remark 3.1 (Fully local equation in $W_{\eta}$).

The transport equation in $W_{\eta}$ in Eq. 8 with nonlocal source can also be
transformed into a fully local equation (as in [15]):

$\displaystyle\partial_{t}W_{\eta}(t,x)+\partial_{x}\big{(}V(W_{\eta}(t,x))W_{\eta}(t,x)\big{)}$
$\displaystyle=\eta\partial_{tx}^{2}W_{\eta}(t,x)+\partial_{x}\big{(}V(W_{\eta}(t,x))\partial_{x}W_{\eta}(t,x)\big{)}$
$\displaystyle(t,x)\in{\Omega_{T}}$ $\displaystyle W_{\eta}(0,x)$
$\displaystyle=\tfrac{1}{\eta}\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})q_{0}(y)\,\mathrm{d}y$
$\displaystyle x\in\mathbb{R}.$

For our main theorem Theorem 3.2 where we prove a total variation bound on
$W_{\eta}$ uniform in $\eta$ we require a density or stability result which
enables us to smooth the solution. This result, stated below, is borrowed from
[42, Theorem 4.17].

######  Theorem 3.1 (Stability of the nonlocal conservation law w.r.t. the
initial datum).

Let Assumption 2.1 hold, and let
$\mathcal{C}_{1},\mathcal{C}_{2}\in\mathbb{R}_{\geq 0}$ be given such that

$Q(\mathcal{C}_{1},\mathcal{C}_{2})\mathrel{\coloneqq}\big{\\{}u\in
TV_{\text{loc}}(\mathbb{R})\,:\,\|u\|_{L^{\infty}(\mathbb{R})}\leq\mathcal{C}_{1}\wedge|u|_{TV(\mathbb{R})}\leq\mathcal{C}_{2}\big{\\}}.$

Let $q_{0}\in Q(\mathcal{C}_{1},\mathcal{C}_{2})$ be given and denote by $q$
the solutions to the corresponding nonlocal conservation law.

Then, the solutions to the corresponding nonlocal conservation laws (denoted
by $q$) satisfy the following $C([0,T];L^{1}(\mathbb{R}))$ stability estimate,
i.e.

$\displaystyle\forall\epsilon\in\mathbb{R}_{>0}\
\exists\delta\in\mathbb{R}_{>0}:\forall\tilde{q}_{0}\in
Q(\mathcal{C}_{1},\mathcal{C}_{2})\text{ with }\
\|q_{0}-\tilde{q}_{0}\|_{L^{1}(\mathbb{R})}\leq\delta\ \implies\
\|q-\tilde{q}\|_{C([0,T];L^{1}(\mathbb{R}))}\leq\epsilon$

where $\tilde{q}$ is the solution to the corresponding nonlocal conservation
law with initial datum $\tilde{q}_{0}$.

###### Proof.

Almost the required result can be found in [42, Theorem 4.17] with the
difference that the kernel of the nonlocal operator is supposed to be finite
while here we have the exponential kernel (Eq. 5). However, the changes for
this result are minor, we do not go into details. ∎

The next theorem shows that the nonlocal term has a total variation which
cannot increase over time.

######  Theorem 3.2 (Total variation bound in the spatial component of $W$ –
uniformly in $\eta$).

The nonlocal term $W_{\eta}$ defined in Eq. 5 but which also satisfies the
identity demonstrated in Lemma 3.1 satisfies – uniformly in $\eta$ a total
variation bound, i.e.

$|W_{\eta}(t,\cdot)|_{TV(\mathbb{R})}\leq|W_{\eta}(0,\cdot)|_{TV(\mathbb{R})}\leq|q_{0}|_{TV(\mathbb{R})}\
\forall\eta\in\mathbb{R}_{>0}.$

###### Proof.

We take advantage of the stability result in Theorem 3.1 which tells us that
when smoothing $q_{0}$ by $q_{0}^{\varepsilon}\vcentcolon\equiv
q_{0}\ast\varphi_{\varepsilon}$ with $\varphi_{\varepsilon}$ being a standard
mollifier [48, C.4 Mollifiers] with smoothing parameter
$\varepsilon\in\mathbb{R}_{>0}$, the corresponding solution
$q_{\eta}^{\epsilon}$ will be close in the $C([0,T];L^{1}(\mathbb{R}))$
topology. Additionally, as the initial datum is smooth, so is the
corresponding solution (see [40, Corollary 5.3]) which we will denote by
$q_{\eta}^{\varepsilon}$. We now prove the total variation bound. As the
solution is smooth, the total variation coincides with the $L^{1}$-norm of the
derivative and we obtain for $t\in[0,T]$, which can be estimated as follows.

$\displaystyle\tfrac{\mathrm{d}}{\mathrm{d}t}\int_{\mathbb{R}}|\partial_{x}W^{\varepsilon}_{\eta}(t,x)|\,\mathrm{d}x$
$\displaystyle=\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)\partial_{tx}^{2}W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle=-\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V(W^{\varepsilon}_{\eta}(t,x))\partial^{2}_{xx}W^{\varepsilon}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V^{\prime}(W^{\varepsilon}_{\eta}(t,x))\big{(}\partial_{x}W^{\varepsilon}_{\eta}(t,x)\big{)}^{2}\,\mathrm{d}x$
$\displaystyle\qquad+\tfrac{1}{\eta}\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V^{\prime}(W^{\varepsilon}_{\eta}(t,x))W^{\varepsilon}_{\eta}(t,x)\partial_{x}W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})V^{\prime}(W^{\varepsilon}_{\eta}(t,y))\partial_{y}W^{\varepsilon}_{\eta}(t,y)W^{\varepsilon}_{\eta}(t,y)\,\mathrm{d}y\,\mathrm{d}x$
$\displaystyle=\int_{\mathbb{R}}2\delta_{0}(\partial_{x}W^{\varepsilon}_{\eta}(t,x))V(W^{\varepsilon}_{\eta}(t,x))\partial_{x}W^{\varepsilon}_{\eta}(t,x)\partial^{2}_{xx}W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad+\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V^{\prime}(W^{\varepsilon})\big{(}\partial_{x}W^{\varepsilon}_{\eta}(t,x)\big{)}^{2}\,\mathrm{d}x$
$\displaystyle\qquad-\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V^{\prime}(W^{\varepsilon}_{\eta}(t,x))\big{(}\partial_{x}W^{\varepsilon}_{\eta}(t,x)\big{)}^{2}\,\mathrm{d}x$
$\displaystyle\qquad+\tfrac{1}{\eta}\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)V^{\prime}(W^{\varepsilon}_{\eta}(t,x))W^{\varepsilon}_{\eta}(t,x)\partial_{x}W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{\mathbb{R}}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)\int_{x}^{\infty}\exp(\tfrac{x-y}{\eta})V^{\prime}(W^{\varepsilon}_{\eta}(t,y))\partial_{y}W^{\varepsilon}_{\eta}(t,y)W^{\varepsilon}_{\eta}(t,y)\,\mathrm{d}y\,\mathrm{d}x$
$\displaystyle\qquad+\tfrac{1}{\eta}\int_{\mathbb{R}}|\partial_{x}W^{\varepsilon}_{\eta}(t,x)|V^{\prime}(W^{\varepsilon}_{\eta}(t,x))W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{\mathbb{R}}V^{\prime}(W^{\varepsilon}_{\eta}(t,y)\partial_{y}W^{\varepsilon}_{\eta}(t,y)W^{\varepsilon}_{\eta}(t,y)\int_{-\infty}^{y}\operatorname{sgn}\left(\partial_{x}W^{\varepsilon}_{\eta}(t,x)\right)\exp(\tfrac{x-y}{\eta})\,\mathrm{d}x\,\mathrm{d}y$
$\displaystyle\leq\tfrac{1}{\eta}\int_{\mathbb{R}}|\partial_{x}W^{\varepsilon}_{\eta}(t,x)|V^{\prime}(W^{\varepsilon}_{\eta}(t,x))W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\tfrac{1}{\eta^{2}}\int_{\mathbb{R}}V^{\prime}(W^{\varepsilon}_{\eta}(t,y))|\partial_{y}W^{\varepsilon}_{\eta}(t,y)|W^{\varepsilon}_{\eta}(t,y)\int_{-\infty}^{y}\exp(\tfrac{x-y}{\eta})\,\mathrm{d}x\,\mathrm{d}y$
(15)
$\displaystyle=\tfrac{1}{\eta}\int_{\mathbb{R}}|\partial_{x}W^{\varepsilon}_{\eta}(t,x)|V^{\prime}(W^{\varepsilon}_{\eta}(t,x))W^{\varepsilon}_{\eta}(t,x)\,\mathrm{d}x$
$\displaystyle\qquad-\tfrac{1}{\eta}\int_{\mathbb{R}}V^{\prime}(W^{\varepsilon}_{\eta}(t,y))|\partial_{y}W^{\varepsilon}_{\eta}(t,y)|W^{\varepsilon}_{\eta}(t,y)\exp(\tfrac{y-y}{\eta})\,\mathrm{d}y$
$\displaystyle=0.$

We thus obtain

$\displaystyle|W_{\eta}^{\epsilon}(t,\cdot)|_{TV(\mathbb{R})}\leq|W_{\eta}^{\epsilon}(0,\cdot)|_{TV(\mathbb{R})}\leq|q_{0}|_{TV(\mathbb{R})},$

where the last inequality follows from the assumption on $q_{0}\in
TV(\mathbb{R})$ as stated in Assumption 2.1 and the definition of the initial
value for $W_{\eta}$ as in Eq. 9:

$\displaystyle\|W_{\eta}^{\varepsilon}(0,\cdot)\|_{TV(\mathbb{R})}$
$\displaystyle=\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\\!\\!\\!\int_{\mathbb{R}}\psi^{\prime}(x)W_{\eta}^{\varepsilon}[q_{0}^{\varepsilon}](x)\,\mathrm{d}x=\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\\!\\!\\!\int_{\mathbb{R}}\psi^{\prime}(x)\tfrac{1}{\eta}\int_{\mathbb{R}_{>0}}\exp(\tfrac{x-y}{\eta})q_{0}^{\varepsilon}(y)\,\mathrm{d}y\,\mathrm{d}x$
$\displaystyle=\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\\!\\!\\!\int_{\mathbb{R}}\psi^{\prime}(x)\tfrac{1}{\eta}\int_{\mathbb{R}_{<0}}\exp(\tfrac{z}{\eta})q_{0}^{\varepsilon}(x-z)\,\mathrm{d}y\,\mathrm{d}x=\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\sup_{z\in\mathbb{R}_{<0}}\int_{\mathbb{R}}\psi^{\prime}(x+z)q_{0}^{\varepsilon}(x)\,\mathrm{d}y$
$\displaystyle=\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\\!\\!\\!\int_{\mathbb{R}}\psi^{\prime}(x)\int_{\mathbb{R}}\varphi_{\varepsilon}(x-y)q_{0}(x)\,\mathrm{d}x\,\mathrm{d}y\leq\sup_{y\in\mathbb{R}}\int_{\mathbb{R}}\varphi_{\varepsilon}(x-y)\\!\\!\\!\\!\\!\sup_{\begin{subarray}{c}\psi\in
C^{1}_{\text{c}}(\mathbb{R}):\\\ \|\psi\|_{L^{\infty}(\mathbb{R})}\leq
1\end{subarray}}\int_{\mathbb{R}}\psi^{\prime}(z)q_{0}(z)\,\mathrm{d}z\,\mathrm{d}x$
$\displaystyle\leq|q_{0}|_{TV(\mathbb{R})}.$

∎

######  Remark 3.2 (Total variation bound and the required assumptions on the
velocity $V$).

The key step in the proof of the total variation bound stated in Theorem 3.2
can be located in the estimate around Eq. 15. Reconnecting to Remark 2.1 it is
enough to assume the velocity to satisfy $V^{\prime}(s)s\leq 0\ \forall
s\in\mathbb{R}$ to obtain the uniform total variation bound without any sign
restriction on the initial datum.

##  4 Compactness argument and proof of the convergence result

######  Theorem 4.1 (Compactness of $W_{\eta}$ in
$C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}$).

The set $\big{(}W_{\eta}\big{)}_{\eta\in\mathbb{R}_{>0}}\subseteq
C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}$ of solutions to Eqs. 8 to
9 is compactly embedded into
$C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)},$ i.e.

$\displaystyle\Big{\\{}W_{\eta}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}:\ W_{\eta}\text{
satisfies \lx@cref{creftypeplural~{}refnum}{eq:W} to~\ref{eq:initial_W}},\
\eta\in\mathbb{R}_{>0}\Big{\\}}\overset{\text{c}}{\hookrightarrow}C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}.$

###### Proof.

The proof consists basically of applying the Ascoli theorem in [56, Lemma 1].
We state the details in the following. Let $B$ be a Banach space.

Then, a set $F\subset C([0,T];B)$ is relatively compact in $C([0,T];B)$ iff

  * 1.

$F(t)\mathrel{\coloneqq}\big{\\{}f(t)\in B:f\in F\\}$ is relatively compact in
$B\ \forall t\in[0,T]$.

  * 2.

$F$ is uniformly equi-continuous, i.e.

$\forall\sigma\in\mathbb{R}_{>0}\ \exists\delta\in\mathbb{R}_{>0}\ \forall
f\in F\ \forall(t_{1},t_{2})\in[0,T]^{2}\text{ with }|t_{1}-t_{2}|\leq\delta:\
\|f(t_{1})-f(t_{2})\|_{B}\leq\sigma.$

We start with setting $B=L^{1}_{\textnormal{loc}}(\mathbb{R})$ and
$F(t)\mathrel{\coloneqq}\\{W_{\eta}(t,\cdot)\in
L^{1}_{\textnormal{loc}}(\mathbb{R}):\eta\in\mathbb{R}_{>0}\\}$. Thanks to
Theorem 3.2 we know that $W_{\eta}(t,\cdot)$ has a uniform total variation
bound and by [48, Theorem 13.35], the set $F(t)$ is compact in
$L^{1}_{\textnormal{loc}}(\mathbb{R})$, i.e.

$F(t)\overset{\text{c}}{\subseteq}L^{1}_{\textnormal{loc}}(\mathbb{R}),\quad\forall
t\in[0,T].$

It remains to show the second point, the uniformly equi-continuity. To this
end, we again smooth the initial datum $q_{0}$ by a $q_{0}^{\varepsilon}$ for
$\varepsilon\in\mathbb{R}_{>0}$ as in the proof of Theorem 3.2 and call the
corresponding smooth nonlocal term $W_{\eta}^{\varepsilon}$ for an
$\eta\in\mathbb{R}_{>0}$. Then, we can estimate

$\displaystyle\big{\|}W_{\eta}^{\varepsilon}(t_{1},\cdot)-W_{\eta}^{\varepsilon}(t_{2},\cdot)\big{\|}_{L^{1}(\mathbb{R})}$
$\displaystyle=\left\|\int_{t_{2}}^{t_{1}}\partial_{t}W_{\eta}^{\varepsilon}(s,\cdot)\,\mathrm{d}s\right\|_{L^{1}(\mathbb{R})}$
plugging Eq. 8 in
$\displaystyle\leq\left\|\int_{t_{2}}^{t_{1}}V(W_{\eta}^{\varepsilon}(s,\cdot))\partial_{2}W^{\varepsilon}_{\eta}(s,\cdot)\,\mathrm{d}s\right\|_{L^{1}(\mathbb{R})}$
$\displaystyle\quad+\left\|\int_{t_{2}}^{t_{1}}\tfrac{1}{\eta}\int_{\ast}^{\infty}\exp(\tfrac{\ast-y}{\eta})V^{\prime}(W_{\eta}^{\varepsilon}(s,y))\partial_{y}W_{\eta}^{\varepsilon}(s,y)W^{\varepsilon}_{\eta}(s,y)\,\mathrm{d}y\,\mathrm{d}s\right\|_{L^{1}(\mathbb{R})}$
integrating by parts
$\displaystyle\leq\|V\|_{L^{\infty}((0,\|q_{0}\|_{L^{\infty}(\mathbb{R})}))}|W_{\eta}^{\epsilon}|_{L^{\infty}((0,T);TV(\mathbb{R}))}|t_{1}-t_{2}|$
$\displaystyle\quad+\|V^{\prime}\|_{L^{\infty}((0,\|q_{0}\|_{L^{\infty}(\mathbb{R})}))}\|W_{\eta}^{\epsilon}\|_{L^{\infty}((0,T);L^{\infty}(\mathbb{R}))}|W_{\eta}^{\epsilon}|_{L^{\infty}((0,T);TV(\mathbb{R}))}|t_{1}-t_{2}|$
applying Theorem 3.2 and Eq. 7
$\displaystyle\leq\big{(}\|V\|_{L^{\infty}((0,\|q_{0}\|_{L^{\infty}(\mathbb{R})}))}+\|V^{\prime}\|_{L^{\infty}((0,\|q_{0}\|_{L^{\infty}(\mathbb{R})}))}\|q_{0}\|_{L^{\infty}(\mathbb{R})}\big{)}|q_{0}|_{TV(\mathbb{R})}|t_{1}-t_{2}|.$

As this is a uniform bound in $\eta\in\mathbb{R}_{>0}$ and
$\varepsilon\in\mathbb{R}_{>0}$ we have the uniform equi-continuity so that we
obtain by applying Ascoli’s theorem indeed the claimed compactness. ∎

As a direct result, from the strong convergence of $W_{\eta}$ we have also the
strong convergence of $q_{\eta}$ to a weak solution of the local conservation
law as the following corollary states:

######  Corollary 4.1 (Limit of $q_{\eta}$ and $W_{\eta}$ are weak solution to
the local equation).

For every sequence $(\eta_{k})_{k\in\mathbb{N}_{\geq
1}}\subset\mathbb{R}_{>0}$ with $\lim_{k\rightarrow\infty}\eta_{k}=0$ there
exists a subsequence (for reasons of convenience again denoted by $\eta_{k}$)
and a function $q^{*}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ so that the
solution $q_{\eta_{k}}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ of the nonlocal
conservation law as given in Definition 2.1 converges in
$C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}$ to the limit point
$q^{*}$ and so does the nonlocal term $W_{\eta_{k}}$ as given in Eq. 5.
Additionally, $q^{*}$ is a weak solution of the local conservation law Eqs. 1
to 2. In equations,

$\displaystyle\lim_{\eta\rightarrow
0}\|q_{\eta}-q^{*}\|_{C([0,T];L^{1}_{\text{loc}}(\mathbb{R}))}=0\ \wedge\
\lim_{\eta\rightarrow
0}\|W_{\eta}-q^{*}\|_{C([0,T];L^{1}_{\text{loc}}(\mathbb{R}))}=0,$

when $q^{*}$ satisfies $\forall\varphi\in
C^{1}_{\text{c}}((-42,T)\times\mathbb{R})$

$\iint_{{\Omega_{T}}}\partial_{t}\varphi(t,x)q^{*}(t,x)+\partial_{x}\varphi(t,x)V(q^{*}(t,x))q^{*}(t,x)\,\mathrm{d}x\,\mathrm{d}t+\int_{\mathbb{R}}\varphi(0,x)q_{0}(x)\,\mathrm{d}x=0.$
(16)

###### Proof.

Thanks to Theorem 4.1,
$\mathcal{W}\mathrel{\coloneqq}\\{W_{\eta_{k}};k\in\mathbb{N}_{\geq
1}\\}\subset C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)},$ i.e.,
the set $\mathcal{W}$ is compact in
$C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ and there exists a
limit point $q^{*}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ so that we obtain

$\lim_{k\rightarrow\infty}\|W_{\eta_{k}}-q^{*}\|_{C([0,T];L^{1}_{\textnormal{loc}}(\mathbb{R}))}=0.$

The identity in Eq. 10 directly implies

$\displaystyle\|W_{\eta_{k}}(t,\cdot)-q_{\eta_{k}}(t,\cdot)\|_{L^{1}(\mathbb{R})}$
$\displaystyle=\eta_{k}|W_{\eta_{k}}(t,\cdot)|_{TV(\mathbb{R})}\leq\eta_{k}|q_{0}|_{TV(\mathbb{R})}$
and thus we also obtain
$\displaystyle\lim_{k\rightarrow\infty}\|q_{\eta_{k}}-q^{*}\|_{C([0,T];L^{1}_{\textnormal{loc}}(\mathbb{R}))}$
$\displaystyle=0.$

It remains to be shown that $q^{*}$ is indeed a weak solution. This directly
follows from the strong convergence of $q_{\eta_{k}}$ to $q^{*}$ in
$C\big{(}[0,T];L^{1}_{\text{loc}}(\mathbb{R})\big{)}$ and due to the essential
and uniform bound on $q_{\eta}$ as given in Theorem 2.1 in Eq. 7. ∎

However, the previous result can actually be strengthened and indeed we obtain
that the limit $q^{*}$ is unique (in particular, every subsequence converges)
and that this limit is the weak entropy solution of the corresponding local
conservation law.

######  Theorem 4.2 (Convergence to the Entropy solution).

Given Assumption 2.1 the nonlocal term $W_{\eta}[q_{\eta}]$ and the
corresponding nonlocal solution $q_{\eta}\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ of the nonlocal
conservation law Definition 2.1 converges in
$C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$ to the Entropy
solution of the corresponding local conservation law (see Eqs. 1 to 2).

###### Proof.

This is a direct consequence of the convergence of $W_{\eta},q_{\eta}$ to a
weak solution of the local conservation laws in
$C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}$, Corollary 4.1 and
of [8]. Therein, by taking advantage of the minimal entropy condition in [27,
49], it is shown that a solution $q_{\eta}$ of the nonlocal conservation law
in Definition 2.1 with uniform $TV$ bound converges to the entropy solution of
the local problem. However, when checking the proof carefully, it turns out
that it suffices to assume that the solution $q_{\eta}$ converges strongly to
a weak solution $q^{*}$. ∎

##  5 Numerical illustrations

Figure 1: Solution of the nonlocal balance law with exponential kernel (top,
Eq. 5) and constant kernel (bottom, Eq. 17) supplemented by the piecewise
constant initial datum stated in Eq. 18 plotted in the space-time domain. From
left to right $\eta$ is decreasing, $\eta\in\big{\\{}10^{-1},\
10^{-2},10^{-3}\big{\\}}$. The rightmost figure is “by eye” not
distinguishable from the corresponding local solution. Colorbar: $0\ $$\ 1$

Figure 2: Left: Solution of the nonlocal balance law with exponential kernel
(top, Eq. 5) and constant kernel (bottom, Eq. 17) supplemented by the
piecewise constant initial datum stated in Eq. 18 and its corresponding
nonlocal term plotted for $t=0.5$ and $\eta\in\\{10^{-1},10^{-2},10^{-3}\\}$.
Right: Evolution of the corresponding total variations showing a monotone
decreasing nature in terms of the nonlocal term (dotted lines) which is also
the case for the local counterpart. In terms of the total variation of the
solution itself (dashed dotted lines), the total variation approaches $3$.
This is due to that the zero in the initial datum
($x\in(\tfrac{1}{3},\tfrac{2}{3})$) moves and shrinks but does not vanish for
all $\eta\in\mathbb{R}_{>0}$ and $t\in(0,T],$ resulting in an additional total
variation of $2$ compared to the total variation of the solution to the local
equation being $1$ for all $t\in(1,T)$.

Some numerical results concerning the convergence can already be found in
[42]. We rely on a solver based on characteristics [43] which is non
dissipative. On the basis of a simple example we want to shed more light on
the difference between the total variation of $q_{\eta}$ and the nonlocal
counterpart $W_{\eta}[q_{\eta}]$. We further demonstrate that the result
should still hold for general nonlocal kernels by using as “worst case” a
constant kernel, i.e. for $q\in
C\big{(}[0,T];L^{1}_{\textnormal{loc}}(\mathbb{R})\big{)}\cap
L^{\infty}((0,T);L^{\infty}(\mathbb{R}))$

$\displaystyle
W_{\eta}[q](t,x)\mathrel{\coloneqq}\tfrac{1}{\eta}\int_{x}^{x+\eta}q(t,y)\,\mathrm{d}y,\qquad
x\in\mathbb{R}.$ (17)

It seems to be true that a total variation bound on the nonlocal term holds
and that also the solution still converges to the local entropy solution. The
following examples rely on the following initial datum:

$\displaystyle q_{0}$
$\displaystyle\equiv\tfrac{1}{2}\chi_{(0,\frac{1}{3})}+\chi_{\mathbb{R}_{>\frac{2}{3}}}$
(18)

The crucial point of the chosen initial datum are the roots for
$x\in(\tfrac{1}{3},\tfrac{2}{3})$. These roots are moving but kept in the
nonlocal solution $q_{\eta}$ for all times. This results in an increase of the
total variation. In the nonlocal term $W$ there are by construction of the
initial datum as well as the exponential kernel no roots and the solution is
smoothed resulting in a – as proven – non-increasing total variation.

##  6 Future work

What remains an open question is whether it is possible to obtain the same
results for different kernels still satisfying the required monotonicity
assumption for the solution to satisfy a maximum principle (see for this
particularly Section 5 and Fig. 1 bottom). The considered exponential kernel
clearly provides a nice structure which seems to be crucial in our analysis
for showing the stated results.

Another interesting problem consists of what happens in the case of a fully
symmetric nonlocal kernel which is sensitive to both propagating directions.
However, such a kernel immediately implies that the solutions cannot satisfy a
maximum principle (for an illustration see for instance [42, Example 7.3, Fig.
9]). Then, recalling [17] it is also apparent that one cannot expect the
solution to converge in a strong or weak sense to the entropy solution, but
there is hope – compare particularly the numerics in [42, Example 7.3] – for
convergence in a measure valued sense.

## Acknowledgments

G. M. Coclite is a member of the Gruppo Nazionale per l’Analisi Matematica, la
Probabilità e le loro Applicazioni (GNAMPA) of the Istituto Nazionale di Alta
Matematica (INdAM). He has been partially supported by the Research Project of
National Relevance “Multiscale Innovative Materials and Structures” granted by
the Italian Ministry of Education, University and Research (MIUR Prin 2017,
project code 2017J4EAYB and the Italian Ministry of Education, University and
Research under the Programme Department of Excellence Legge 232/2016 (Grant
No. CUP - D94I18000260001).

J.-M. Coron acknowledges funding from the Miller Institute and from the Agence
Nationale de La Recherche (ANR), grant ANR Finite4SoS (ANR-15-CE23-0007). He
also thanks the Miller Institute and UC Berkeley for their hospitality.

N. De Nitti has been partially supported by the Alexander von Humboldt
Foundation and by the TRR-154 project of the DFG.

L. Pflug has been supported by the Deutsche Forschungsgemeinschaft (DFG,
German Research Foundation) – Project-ID 416229255 – SFB 1411.

## References

  * [1] A. Aggarwal, R. Colombo, and P. Goatin.  Nonlocal systems of conservation laws in several space dimensions.  SIAM Journal on Numerical Analysis, 53(2):963–983, 2015.
  * [2] G. Aletti, G. Naldi, and G. Toscani.  First-order continuous models of opinion formation.  SIAM Journal on Applied Mathematics, 67(3):837–853, 2007.
  * [3] P. Amorim, R. M. Colombo, and A. Teixeira.  On the numerical integration of scalar nonlocal conservation laws.  ESAIM Math. Model. Numer. Anal., 49(1):19–37, 2015.
  * [4] A. Bayen, J.-M. Coron, N. De Nitti, A. Keimer, and L. Pflug.  Boundary controllability and asymptotic stabilization of a nonlocal traffic flow model.  Preprint, 2020.
  * [5] A. Bayen, A. Keimer, L. Pflug, and T. Veeravalli.  Modeling multi-lane traffic with moving obstacles by nonlocal balance laws.  Preprint (09 2020). doi, 10, 2020.
  * [6] F. Betancourt, R. Bürger, K. H. Karlsen, and E. M. Tory.  On nonlocal conservation laws modelling sedimentation.  Nonlinearity, 24(3):855–885, 2011.
  * [7] A. Bressan.  Hyperbolic systems of conservation laws, volume 20 of Oxford Lecture Series in Mathematics and its Applications.  Oxford University Press, Oxford, 2000.  The one-dimensional Cauchy problem.
  * [8] A. Bressan and W. Shen.  Entropy admissibility of the limit solution for a nonlocal model of traffic flow.  preprint, 2020.
  * [9] A. Bressan and W. Shen.  On traffic flow with nonlocal flux: a relaxation representation.  Arch. Ration. Mech. Anal., 237(3):1213–1236, 2020.
  * [10] P. Calderoni and M. Pulvirenti.  Propagation of chaos for Burgers’ equation.  Ann. Inst. H. Poincaré Sect. A (N.S.), 39(1):85–97, 1983.
  * [11] C. Chalons, P. Goatin, and L. M. Villada.  High-order numerical schemes for one-dimensional nonlocal conservation laws.  SIAM Journal on Scientific Computing, 40(1):A288–A305, 2018.
  * [12] F. A. Chiarello, P. Goatin, and E. Rossi.  Stability estimates for non-local scalar conservation laws.  Nonlinear Analysis: Real World Applications, 45:668 – 687, 2019\.
  * [13] F. A. Chiarello, P. Goatin, and L. M. Villada.  Lagrangian-antidiffusive remap schemes for non-local multi-class traffic flow models.  Computational and Applied Mathematics, 39(2):1–22, 2020.
  * [14] J. Chu, P. Shang, and Z. Wang.  Controllability and stabilization of a conservation law modeling a highly re-entrant manufacturing system.  Nonlinear Anal., 189:111577, 19, 2019.
  * [15] G. M. Coclite, N. De Nitti, A. Keimer, and L. Pflug.  Singular limits with vanishing viscosity for nonlocal conservation laws.  Preprint, 2020.
  * [16] M. Colombo, G. Crippa, M. Graff, and L. V. Spinolo.  Recent results on the singular local limit for nonlocal conservation laws, 2019.
  * [17] M. Colombo, G. Crippa, E. Marconi, and L. V. Spinolo.  Local limit of nonlocal traffic models: convergence results and total variation blow-up, 2018.
  * [18] M. Colombo, G. Crippa, and L. V. Spinolo.  On the singular local limit for conservation laws with nonlocal fluxes.  Arch. Ration. Mech. Anal., 233(3):1131–1167, 2019.
  * [19] R. Colombo, M. Herty, and M. Mercier.  Control of the continuity equation with a non local flow.  ESAIM Control Optim. Calc. Var., 17(2):353–379, 2011.
  * [20] R. M. Colombo, M. Garavello, and M. Lécureux-Mercier.  A class of nonlocal models for pedestrian traffic.  Math. Models Methods Appl. Sci., 22(4):1150023, 34, 2012.
  * [21] R. M. Colombo, M. Lecureux-Mercier, and M. Garavello.  Crowd dynamics through conservation laws.  In Crowd Dynamics, Volume 2, pages 83–110. Springer, 2020.
  * [22] J.-M. Coron and S. Guerrero.  Singular optimal control: a linear 1-D parabolic-hyperbolic example.  Asymptot. Anal., 44(3-4):237–257, 2005.
  * [23] J.-M. Coron, M. Kawski, and Z. Wang.  Analysis of a conservation law modeling a highly re-entrant manufacturing system.  Discrete Contin. Dyn. Syst. Ser. B, 14(4):1337–1359, 2010.
  * [24] J.-M. Coron and Z. Wang.  Controllability for a scalar conservation law with nonlocal velocity.  Journal of Differential Equations, 252(1):181–201, 2012.
  * [25] G. Crippa and M. Lécureux-Mercier.  Existence and uniqueness of measure solutions for a system of continuity equations with non-local flow.  Nonlinear Differential Equations and Applications NoDEA, 20(3):523–537, 2013.
  * [26] C. M. Dafermos.  Hyperbolic conservation laws in continuum physics, volume 325 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences].  Springer-Verlag, Berlin, fourth edition, 2016.
  * [27] C. De Lellis, F. Otto, and M. Westdickenberg.  Minimal entropy conditions for burgers equation.  Quarterly of applied mathematics, 62(4):687–700, 2004.
  * [28] J. Friedrich, O. Kolb, and S. Göttlich.  A godunov type scheme for a class of lwr traffic flow models with non-local flux.  Networks & Heterogeneous Media, 13:531, 2018.
  * [29] O. Glass and S. Guerrero.  On the uniform controllability of the Burgers equation.  SIAM J. Control Optim., 46(4):1211–1238, 2007.
  * [30] P. Goatin and S. Scialanga.  Well-posedness and finite volume approximations of the lwr traffic flow model with non-local velocity.  Networks and Hetereogeneous Media, 11(1):107–121, 2016.
  * [31] E. Godlewski and P.-A. Raviart.  Hyperbolic systems of conservation laws.  Ellipses, 1991.
  * [32] X. Gong and M. Kawski.  Weak measure-valued solutions to a nonlinear hyperbolic conservation law modeling a highly re-entrant manufacturing system.  arXiv preprint arXiv:1903.00797, 2019.
  * [33] M. Gröschel, A. Keimer, G. Leugering, and Z. Wang.  Regularity Theory and Adjoint Based Optimality Conditions for a Nonlinear Transport Equation with Nonlocal Velocity.  SIAM Journal on Control and Optimization, 52(4):2141–2163, 2014\.
  * [34] S. Guerrero and G. Lebeau.  Singular optimal control for a transport-diffusion equation.  Comm. Partial Differential Equations, 32(10-12):1813–1836, 2007\.
  * [35] M. Gugat, A. Keimer, G. Leugering, and Z. Wang.  Analysis of a system of nonlocal conservation laws for multi-commodity flow on networks.  Networks & Het. Media, 10(4):749–785, 2015.
  * [36] H. Holden and N. H. Risebro.  Front tracking for hyperbolic conservation laws, volume 152 of Applied Mathematical Sciences.  Springer, Heidelberg, second edition, 2015.
  * [37] K. Huang and Q. Du.  Stability of a nonlocal traffic flow model for connected vehicles, 2020\.
  * [38] I. Karafyllis, D. Theodosis, and M. Papageorgiou.  Analysis and control of a non-local pde traffic flow model, 2020.
  * [39] A. Keimer, G. Leugering, and T. Sarkar.  Analysis of a system of nonlocal balance laws with weighted work in progress.  Journal of Hyperbolic Diff. Equations, 15(03):375–406, 2018.
  * [40] A. Keimer and L. Pflug.  Existence, uniqueness and regularity results on nonlocal balance laws.  J. Differential Equations, 263(7):4023–4069, 2017.
  * [41] A. Keimer and L. Pflug.  Existence, uniqueness and regularity results on nonlocal balance laws.  Journal of Differential Equations, 263:4023–4069, 2017.
  * [42] A. Keimer and L. Pflug.  On approximation of local conservation laws by nonlocal conservation laws.  J. Math. Anal. Appl., 475(2):1927–1955, 2019.
  * [43] A. Keimer, L. Pflug, and M. Spinola.  Nonlocal balance laws: Theory of convergence for nondissipative numerical schemes.  submitted.
  * [44] A. Keimer, L. Pflug, and M. Spinola.  Existence, uniqueness and regularity of multi-dimensional nonlocal balance laws with damping.  J. Math. Anal. Appl., 466(1):18–55, 2018.
  * [45] A. Keimer, L. Pflug, and M. Spinola.  Nonlocal scalar conservation laws on bounded domains and applications in traffic flow.  SIAM J. Math. Anal., 50(6):6271–6306, 2018.
  * [46] A. Keimer, M. Singh, and T. Veeravalli.  Existence and uniqueness results for a class of nonlocal conservation laws by means of a lax-hopt-tye solution formula.  Journal of Hyperbolic Differential Equations, 17(4):1–28, 2020\.
  * [47] M. Léautaud.  Uniform controllability of scalar conservation laws in the vanishing viscosity limit.  SIAM J. Control Optim., 50(3):1661–1699, 2012.
  * [48] G. Leoni.  A first course in Sobolev spaces, volume 105 of Graduate Studies in Mathematics.  American Mathematical Society, Providence, RI, 2009.
  * [49] E. Y. Panov.  Uniqueness of the solution of the cauchy problem for a first order quasilinear equation with one admissible strictly convex entropy.  Mathematical Notes, 55(5):517–525, 1994.
  * [50] L. Pflug, T. Schikarski, A. Keimer, W. Peukert, and M. Stingl.  emom: Exact method of moments—nucleation and size dependent growth of nanoparticles.  Computers & Chemical Engineering, 136:106775, 2020.
  * [51] B. Piccoli, N. Duteil, and E. Trélat.  Sparse control of hegselmann–krause models: Black hole and declustering.  SIAM Journal on Control and Optimization, 57(4):2628–2659, 2019\.
  * [52] B. Piccoli and F. Rossi.  Transport equation with nonlocal velocity in wasserstein spaces: Convergence of numerical schemes.  Acta Applicandae Mathematicae, 124(1):73–105, Apr 2013.
  * [53] J. Ridder and W. Shen.  Traveling waves for nonlocal models of traffic flow.  Discrete & Continuous Dynamical Systems - A, 39:4001, 2019.
  * [54] E. Rossi, J. Weißen, P. Goatin, and S. Göttlich.  Well-posedness of a non-local model for material flow on conveyor belts.  ESAIM: Mathematical Modelling and Numerical Analysis, 54(2):679–704, 2020.
  * [55] P. Shang and Z. Wang.  Analysis and control of a scalar conservation law modeling a highly re-entrant manufacturing system.  J. Differential Equations, 250(2):949–982, 2011.
  * [56] J. Simon.  Compact sets in the space $L^{p}(0,T;B)$.  Ann. Mat. Pura Appl. (4), 146:65–96, 1987.
  * [57] M. Spinola, A. Keimer, D. Segets, G. Leugering, and L. Pflug.  Model-based optimization of ripening processes with feedback modules.  Chemical Engineering & Technology, 43(5):896–903, 2020.
  * [58] L. Tartar.  Compensated compactness and applications to partial differential equations.  In Nonlinear analysis and mechanics: Heriot-Watt Symposium, Vol. IV, volume 39 of Res. Notes in Math., pages 136–212. Pitman, Boston, Mass.-London, 1979.
  * [59] K. Zumbrun.  On a nonlocal dispersive equation modeling particle suspensions.  Quart. Appl. Math., 57(3):573–600, 1999.


Copyright for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).

IberLEF 2024, September 2024, Valladolid, Spain

[1]

[orcid=0009-0003-9099-3826<EMAIL_ADDRESS>url=https://github.com/marccasals98, ] [1]

[orcid=0000-0002-1389-3595<EMAIL_ADDRESS>]

[orcid=0000-0002-3107-3662<EMAIL_ADDRESS>]

[orcid=0000-0002-1730-8154<EMAIL_ADDRESS>]

[1]Corresponding author.

# BSC-UPC at EmoSPeech-IberLEF2024: Attention Pooling for Emotion Recognition

Marc Casals-Salvador  Barcelona Supercomputing Center, Eusebi Güell Square
1-3, 08034 Barcelona, Spain  Federico Costa  Miquel India  Universitat
Politècnica de Catalunya, Jordi Girona 31, 08034 Barcelona, Spain  Javier
Hernando

(2024)

###### Abstract

The domain of speech emotion recognition (SER) has persistently been a
frontier within the landscape of machine learning. It is an active field that
has been revolutionized in the last few decades and whose implementations are
remarkable in multiple applications that could affect daily life.
Consequently, the Iberian Languages Evaluation Forum (IberLEF) of 2024 held a
competitive challenge to leverage the SER results with a Spanish corpus. This
paper presents the approach followed with the goal of participating in this
competition. The main architecture consists of different pre-trained speech
and text models to extract features from both modalities, utilizing an
attention pooling mechanism. The proposed system has achieved the first
position in the challenge with an 86.69% in Macro F1-Score.

###### keywords:

Speech Emotion Recognition Deep Learning Attention Transformers

##  1 Introduction

Emotions are undoubtedly fundamental parts of our idiosyncrasy. They play an
important role in interpersonal relationships and decision-making and
generally take part in the evolution and consciousness of any mental process
[1]. Moreover, there is empirical evidence that emotions immensely influence
human health [2], which creates the necessity to monitor them. Affective
computing is of great interest in medical health fields [3]. Consequently,
developing a system capable of discerning various emotions is highly valuable.
Researchers have attempted to predict emotions using machine learning
approaches, but their effectiveness depends on the quality and quantity of
available data.

In the field of Natural Language Processing (NLP), numerous emotion
recognition models rely exclusively on text-based features. Since the creation
of Transformers [4], multiple approaches [5, 6, 7] seek to use pre-trained
Transformer encoders, such as BERT [8] as text feature extractors. These
encoders are pre-trained models using self-supervised learning techniques on
extensive datasets. This pre-training enables the models to project words in a
latent space with a rich semantic representation, from which classifiers can
then be employed to predict the corresponding emotion classes. On the other
hand, speech is crucial for expressing emotions. Elements such as pitch,
prominence, and phrasing contribute generously to providing emotion
information. As explained in [9], the human brain is capable of recognizing
emotions pan-culturally and independently of the language they are expressed
with. Following this principle, the researchers have proposed different
architectures to process and extract information from speech signals. In the
realm of Speech Emotion Recognition (SER), Machine Learning (ML) and Deep
Learning (DL) models often utilize hand-crafted features such as Mel-Frequency
Cepstral Coefficients (MFCC) [10, 11, 12, 13]. Nevertheless, recent advances
in Deep Learning allow cutting-edge architectures to combine text and speech
to provide better results. Multimodal emotion Recognition (MER) is a complex
task since it requires models to be able to learn complex patterns of the
data. For this reason, the usage of text and audio pre-trained models
significantly improves the embedding representation of their features,
allowing them to be combined in their latent space [14, 15, 16].

In summary, the exceptional progress that has been made in this field is
appreciable, yet there also exists a disparity in the amount of work carried
out in Spanish. Developing these models using Spanish data is crucial for
several reasons. On the one hand, Spanish is one of the languages with the
most native speakers worldwide. On the other hand, it is necessary to leverage
the engineering opportunities of Spanish-speaking countries, opening the door
to developing new technologies that could satisfy their population needs.
However, one of the difficulties this presents is the lack of labelled data in
Spanish needed to train any supervised learning approach.

In order to encourage the creation of an emotion recognition model trained
with Spanish data, the Iberian Languages Evaluation Forum (IberLEF) of 2024
created the challenge EmoSPeech 2024. This competition evaluates the Macro
F1-Score of the participants in two tasks: Emotion Recognition with text and
with speech and text. The training corpus is Spanish MEACorpus 2023 [17]. This
dataset contains 13.16 h of speech, and its transparent methodology
distinguishes it from other datasets of the same task. The speech samples are
collected from YouTube videos and are labelled using the categorical taxonomy
proposed by P. Ekman [18], which include emotions such as surprise, disgust,
anger, joy, sadness, fear, and neutral expressions. Nevertheless, it was
impossible for the annotators to find any sample that contained speech
expressing surprise emotion, though this class is not represented in the
dataset.

This paper endeavours to leverage the research of Spanish models with cutting-
edge technologies in the current state of the art by making use of the
MEACorpus 2023. The current state of the art is the usage of Transformers-
based pre-trained models with high capacity that, leveraging the enormous
datasets they are trained with, are capable of describing complex patterns in
both text and speech. Following this lead, the system combines a speech pre-
trained model, the XLSR-wav2vec 2.0 [19], that is trained with 436,000h and a
RoBERTA text model fine-tuned in Spanish [20]. Both models are used as feature
extractors for speech and text respectively, and they output a vector with the
relevant information of the utterances. The two vectors are then concatenated
into a single vector that contains information about text and speech.
Subsequently, this vector is reduced to lower-dimension representation by
making use of an attention pooling mechanism. Finally, dense layers are
utilized to project this reduced vector, determining the classification of the
utterance. This approach has reached an F1-Score of 86.69%, achieving the
first position in the multimodal task of the EmoSPeech 2024 challenge.

##  2 Challenge

The EmoSPeech 2024 [21] is a Challenge proposed by the Iberian Languages
Evaluation Forum (IberLEF) of 2024 [22], which is a Spanish workshop hosted by
the Sociedad Española para el Procesamiento del Lenguaje Natural (SEPLN). This
event is dedicated to producing models in the frame of the Iberian peninsula,
encompassing different national languages such as Spanish, Portuguese,
Catalan, Basque, or Galician.

To develop the competition, the organisers proposed a dataset named Spanish
MEACorpus 2023 [17]. The dataset, comprising 13.16h of speech divided into
5,129 audios, was meticulously labelled by the research team of the article.
As explained in their paper, the procedure to extract the audio files is as
follows: The authors of the dataset selected YouTube videos according to their
topic and extracted audio segments considering the noise of the audio files
and the silence gaps. Once this part was done, the audio files were classified
using the emotion taxonomy developed by P. Ekman [18]. It comprehended the
basic emotions of disgust, anger, joy, sadness, fear, surprise, and neutral
emotion. Nevertheless, despite the efforts, finding any speech audio that
contained the surprise emotion was impossible.

Figure 1: Some of the characteristics of the MEACorpus 2023. On the left is
the distribution of the number of speech fragments over emotions. On the
right, a histogram of the durations of the audio fragments.

As is common in the field, the dataset exhibits an unequal representation of
the emotion classes. Figure 1 (left)shows that neutrality and disgust are the
most prevalent emotions, while fear is notably scarce. Another important
aspect is the length of the audio files, which can directly affect the
performance of the network by providing more contextual information about the
speech. Therefore, audio duration is an essential characteristic of any speech
dataset and must be considered a quality metric. A histogram of the audio file
duration is shown in Figure 1(right). The mean of the duration is 9.24 s.
Lastly, another fundamental consideration is the variability of the
recordings. The fact that this competition uses third-party audio files makes
it more difficult to control other parameters, such as noise or the magnitude
of the audio. Some videos are recorded outdoors and are more likely to have
background noise or exhibit poorer recording quality, while others are studio-
recorded and of higher quality. Furthermore, the dataset’s paper detailed that
46% of the speech segments are attributed to female voices, with the remainder
belonging to males. The paper affirmed that the text transcriptions were
extracted directly from the raw audio file using Whisper [23] followed by a
manual revision by the researchers.

##  3 Architecture

The system created is a multimodality model that combines text and speech and,
trained with the Spanish MEACorpus 2023 dataset [17], effectuates a
classification of the emotion. In Figure 2, it is possible to see a global
representation of the system. As can be seen, both text and speech are fed to
the network and processed with a self-supervised learning (SSL) model that
works as a feature extractor. Then, the model concatenates these features and
pools them into one single vector. Finally, a classification is performed by
processing this vector with multilayer perceptrons that serve as a classifier.

Figure 2: Diagram Attention Pooling for the Multimodal Emotion Recognition
System. The speech utterances are represented in red and the text is
represented in blue.

###  3.1 Speech

It is a standard practice to apply regularisation techniques to preprocess the
data before its utilisation in a machine learning model. In the case of this
system, the overall mean and standard deviation of all the audio files were
calculated and used to normalise the values in the dataset. This technique is
widespread in Deep Learning literature for speech due to the fact that it
makes backpropagation more efficient and reduces the impact of the outliers.
The normalisation process was extended to the validation and test sets,
whereby the mean and standard deviation values obtained from the training set
were utilised. It is essential to mention that, during the training stage, the
samples were randomly cropped with a window of 5.5 seconds. The shorter
utterances were enlarged using repetition padding.

Given the limited size of the dataset, it became necessary to use data
augmentation to mitigate overfitting to the training set. The specific
techniques utilized included speed perturbation, which alters the speed of the
audio; reverberation, which simulates a reverberant environment; and
background noise, which adds ambient sounds to the audio. PyTorch implements
data augmentation using the following scheme: first of all, defining the
transformations used to create synthetic data. Then, for each sample, a
probability decides whether the model will see the original data or the one
created after applying these transformations. This process occurs in every
epoch, so data augmentation is applied to different samples in each epoch.
This streaming data augmentation guarantees the model is exposed to a wide
diversity of data without physically increasing the dataset size. As is common
in the field, when doing inference is not desired to perturb the data, so in
the validation and test sets, the probability of applying these
transformations is zero.

Recent advancements in the field of Deep Learning have highlighted the
importance of using Transformer-based pre-trained models. It has been proven
that their ability to adapt to changes in the domain makes them suitable for
extracting features from audio files in any dataset. In this work, the
following models were experimented with:

  * •

WavLM [24]: This cutting-edge model is trained with 80,000 hours and
encompasses datasets such as Libri-Light [25], GigaSpeech [26] and VoxPopuli
[27]. Two versions of this model were tried, the Large version and the Base.
The output vector is 768 in the case of the base version and 1,024 in the case
of the large.

  * •

XLSR-wav2vec 2.0 [19]: This model is based on wav2vec 2.0 [28] and it is
trained with the datasets Common Voice [29], BABEL [30] and Multilingual
LibriSpeech [31], which makes a total of 436,000 hours of audio in 128
languages. This model outputs a vector of dimension 1,024.

  * •

HuBERT [32]: This model is trained with 60,000 hours of Libri-Light [25]. The
output vector is of dimension 1,024.

###  3.2 Text

The text domain was the first to employ pre-trained large language models
(LLMs) for different tasks. Since the creation of the BERT model [33],
different approaches have emerged in the state of the art, following the same
idea with some variations. In the approach of this work, the following SSL
models were experimented with:

  * •

BERT [33]: The large uncased version was used, with an an output dimension of
1,024.

  * •

XLM-RoBERTa Spanish [34]: It is a pre-trained model based on XLM-RoBERTa [35]
and trained with Spanish Unannotated Corpora [36]. This model outputs a vector
with 1,024 dimensions.

  * •

BETO [37]: BETO is one of the first pre-trained models produced with Spanish
data. It follows the same structure as the BERT base. Consequently, it outputs
a hidden vector of 768 dimensions. It is trained using Wikipedia data and all
of the sources of the OPUS Project [38]. Additionally, a fine-tuned version of
BETO for emotions was used in the system.

###  3.3 Classifier

Following the attention pooling process, the resultant vector undergoes
further processing via a classifier module. This module is designed to include
a layer that adjusts the vector’s dimension to fit the desired hidden layer
width. It also consists of a stack of hidden layers and an output layer that
has a dimension equivalent to the number of classes being considered.

The model architecture consists of several linear layers, each followed by a
dropout, layer normalization, and a Gaussian Error Linear Unit (GELU)
activation function [39]. The Softmax activation function is used in the
output layer to select the predicted class.

##  4 Attention Pooling

Different approaches have emerged for integrating information extracted from
pre-trained models in the field of multimodal learning. Attention mechanisms,
particularly Multi-Head Attention (MHA), have been popular in recent years for
combining text and speech utterances. This study used an alternative Attention
Pooling mechanism used in works such as [40, 41, 42] to reduce the
dimensionality of the hidden state vector created by concatenating the outputs
of the two pre-trained models.

Figure 3: Diagram of the Attention Pooling operation. The hidden vector
$h=h_{1},...,h_{T}$ is pooled into a lower dimension $c$, using a learnable
parameter $u$.

Considering the embedding dimension $E$ and a batch size of one, we define the
hidden states as the sequences of the extracted features
$\\{h_{t}\in\mathbb{R}^{E}|t=1,...,T\\}$. Then, for each hidden state $h_{t}$
we calculate its weight as described in Equation (1):

$w_{t}=\frac{\exp\left(\cfrac{h_{t}^{\top}u}{\sqrt{E}}\right)}{\sum_{i=1}^{T}\exp\left(\cfrac{h_{i}^{\top}u}{\sqrt{E}}\right)}$
(1)

where $u\in\mathbb{R}^{E}$ is a trainable parameter initialized with the
Xavier initialization [43] and $w_{t}$ is the weight associated at the hidden
vector $h_{t}$. Then, the pooled representation of the hidden vector is
calculated using Equation (2).

$c=\sum_{t=1}^{T}w_{t}h_{t}$ (2)

The vector $c$ encapsulates the relevant information of the features extracted
in the text and speech systems. This approach is computationally more
efficient compared to the general Attention mechanism, where the key, query,
and values are calculated. This characteristic is especially convenient for
this system due to the scarce data provided. Figure 3 demonstrates graphically
the functioning of the attention pooling mechanism.

##  5 Experimental Setup

PyTorch requires all samples in the batch to have the same dimensions.
Therefore, during training, the audio files were cropped using a window of 5.5
seconds, which was the optimal value found. In inference, the whole audio
waveform was used. As detailed in Section 3, the audio waveforms were
normalized using the mean and standard deviation of the training set. The
values extracted were -33.62 and 56.15, respectively. These same values are
applied to the other sets when doing inference. Data augmentation techniques
were applied by varying the probability based on the capacity of the model.
The optimal value was found to be 0.3.

A batch size of 16 samples was selected, as it provided an optimal trade-off
between minimizing the duration of each epoch and avoiding GPU memory
exhaustion. To further accelerate computation, data parallelization across two
GPUs was utilized. In particular, the GPUs employed were two NVIDIA GeForce
RTX 2080Ti. The optimizer selected was the AdamW [44], with a learning rate of
0.00005, which decayed by 10% after five epochs without improvements in the
validation F1-score. The dropout rate, set at 0.1, was adjusted according to
the network’s complexity. The number of epochs utilized also depended on the
model’s capacity. Although model parameters were only stored when the F1-score
improved, early stopping was necessary due to the noisy and variable learning
curves. This variability could lead to an overfitting model being saved based
on local improvements in the F1-Score during validation. Each experiment
lasted one to two days, depending on the configurations.

To improve our position on the leaderboard, we made Macro F1-Score our top
priority since it was the metric used to evaluate the participants’
submissions. This metric combines the Precision and the Recall into one single
number by applying their harmonic mean. Specifically, the Macro F1-Score is
the average of the F1-Score of each class. This metric treats all classes
equally, regardless of their amount of data, making it a fair measure of
overall performance. Given that the metric of interest is the macro F1-Score,
it is imperative to mitigate the class imbalance present in this dataset. A
wide variety of losses try to palliate this disparity. Beyond these
possibilities, the loss criterion finally chosen is the weighted cross-entropy
loss.

After doing the hyperparameter search, two classical machine learning
techniques were employed with the aim of improving the results. The first
approach involved applying thresholds to modify the final decision over the
logits. However, this did not enhance the results. The second strategy
leveraged the variability of different models by using a 3-model ensemble.
Hard voting was chosen as the ensemble technique, where the most voted
prediction among the three models was selected. In the event of a tie, the
prediction from the model with the highest F1-Score on the validation set was
chosen. The code of the project is available here:
https://github.com/marccasals98/BSC-UPC_EmoSPeech

##  6 Results

In the initial stages of the competition, it was necessary to evaluate various
pre-trained self-supervised models to determine the most suitable one for the
data. At this moment, only the training corpus was available; therefore, it
was necessary to make a validation partition to evaluate the performances of
the different self-supervised models. Table 1 presents the best results
obtained with the diverse text and feature extractors. The best configuration
used RoBERTa for text and XLSR-wav2vec 2.0 for audio, achieving an F1-score of
89.73% on the validation set. This superior performance is likely because
RoBERTa was trained on Spanish data, making it more effective for this domain
than other models trained in English. In addition, XLS-wav2vec 2.0 was trained
by using 436,000 hours of audio in 128 languages, including Spanish, which
possibly contributed to the improvement of this metric score. It is worth
noting that, despite BETO being a Spanish version of BERT, the results with
this encoder were poor. This fact could be attributed to WavLM not being
trained with as much Spanish data as XLSR-wav2vec 2.0 or that BETO’s vector
dimension is 768 instead of 1,024, resulting in fewer features captured by the
model.

Text Model | Audio Model | Output Dimensions | Validation F1-Score
---|---|---|---
RoBERTa | WavLM LARGE | 1,024 | 80.04%
RoBERTa | XLSR-wav2vec 2.0 | 1,024 | 89.73%
RoBERTa | HuBERT LARGE | 1,024 | 76.033%
BERT Large Uncased | WavLM LARGE | 1,024 | 83.27%
BERT Large Uncased | XLSR-wav2vec 2.0 | 1,024 | 86.59%
BETO | WavLM BASE PLUS | 768 | 74.79%
BETO-EMO | WavLM BASE PLUS | 768 | 73.19%
Table 1: Different results were obtained during the validation test with the
different feature extractor models. All of these configurations had their
corresponding hyperparameter tuning, and the best of each one was selected.

It is remarkable that, in this initial stage, some experiments were conducted
with other architectures that involved more parameters. For instance, attempts
were made to combine features extracted from the encoder models using multi-
head attention (MHA) with one and two heads. These experiments yielded
unsatisfactory results, with F1-scores of 84.4% and 82.2%, and the models
exhibited significant overfitting. Consequently, it was decided to discontinue
these lines of experimentation and concentrate all efforts on the
hyperparameter tuning using RoBERTa and XLSR-wav2vec 2.0 and the attention
pooling.

The top three different models that obtained the best score achieved 86.20%,
85.96%, and 82.43%. Their confusion matrices over the test set are displayed
in Figure 4. As can be seen, anger is the most difficult emotion to classify,
normally getting confused with disgust. It is remarkable that, despite being
very scarce in the dataset, fear is very separable from the rest of the
emotional spectrum.

(a) Hidden dense layers = 3
Weight decay = 0.1

(b) Hidden dense layers = 2
Weight decay = 0.01

(c) Hidden dense layers = 2
Weight decay = 0.1

Figure 4: Confusion matrices in the test set. The drop-out was set to 0.1 and
the data augmentation probability to 0.3.

Section 5 remarked that thresholding techniques failed to outperform the
models and were, therefore, discarded. Consequently, the only non-trainable
approach used to improve the model’s F1-Score was model ensembling. Initially,
models with different feature extractors were employed to leverage the
diversity of features and create a robust system. However, this approach
proved to not be effective. Instead, the three best-performing models on the
validation set were ensembled, improving the F1-score to 86.69%. Table 2
compares the results of these approaches with the challenge baseline.

Model Name | Hidden dense layers | Weight Decay | Test F1-Score
---|---|---|---
Top 1 Model | 2 | 0.01 | 86.20%
Top 2 Model | 2 | 0.1 | 85.96%
Top 3 Model | 3 | 0.1 | 82.43%
Model Ensemble | - | - | 86.69%
Baseline | - | - | 53.08%
Table 2: Results of the top 3 models, the model ensemble (hard voting), and
the baseline of the challenge over the test set.

##  7 Conclusions and future work

This study presented a multimodal model for the emotion recognition challenge
EmoSPeech 2024 within the IberLEF 2024 framework, aimed at recognizing
emotions from speech and text inputs. The architecture comprised two pre-
trained models, one dedicated to speech and the other to text. They extracted
feature vectors that the model concatenates into a unified hidden
representation vector. On the one hand, for the audio side, different
experiments were conducted with WavLM, XLSR-wav2vec 2.0, and HuBERT. On the
other hand, the optimization of the textual component of the architecture
involved exploration with the models BERT, XLM-RoBERTa for Spanish, BETO, and
its finetuned version for emotion. The best performance is achieved by jointly
combining RoBERTa and XLSR-wav2vec 2.0.

After the model concatenates the text and speech feature vectors, it employs a
dimensionality reduction via reduced attention pooling. This mechanism, with
fewer parameters than its standard counterpart, facilitates the seamless
integration of text and audio while mitigating the risk of overfitting to the
training set. Subsequently, a stack of dense layers processed the output
vector, using its compressed information to extract the class prediction.
Additionally, to optimize performance and maximize the F1-Score in the
competition, model ensembling techniques were adopted, employing hard voting
on the top three models. In summary, the system was capable of achieving an
F1-Score of 86.69%, an absolute increment of 33.61% compared to the baseline,
and securing the first position in the challenge.

After the conclusion of this competition, continuing the research of new
paradigms for Speech Emotion Recognition (SER) could be a captivating line of
research. One of the lines is improving the efficiency of speech feature
extractors. In the paper of WavLM, the authors claim that, in general, most
self-supervised learning models (SSL) models focus primarily on Automatic
Speech Recognition (ASR) tasks. However, by training an SSL model to jointly
learn masked speech prediction and denoising in the pretraining stage, the
model’s capabilities extend beyond ASR, outperforming other SSL in fields such
as SER.

This statement could seem contradictory because, in Section 6, it is proven
that XLSR-wav2vec 2.0 outperforms WavLM Large. Nevertheless, this outcome is
likely due to XLSR-wav2vec2.0 being trained with a very extensive multilingual
dataset. If WavLM was trained with a comparable volume of data, it could
potentially outperform XLSR-wav2vec 2.0, leveraging its joint learning of
masked speech prediction and denoising to achieve superior performance in
various tasks, including Speech Emotion Recognition.

###### Acknowledgements.

This work has been promoted and financed by the Government of Catalonia
through the Aina project, as well as by the Spanish Ministerio de Ciencia e
Innovación through the AdaVoice project (PID2019-107579RB-I00).

## References

  * Izard [2009] C. E. Izard,  Emotion Theory and Research: Highlights, Unanswered Questions, and Emerging Issues,  Annual review of psychology 60 (2009) 1–25. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2723854/. doi:10.1146/annurev.psych.60.110707.163539.
  * Consedine and Moskowitz [2007] N. S. Consedine, J. T. Moskowitz,  The role of discrete emotions in health outcomes: A critical review,  Applied and Preventive Psychology 12 (2007) 59–75. URL: https://www.sciencedirect.com/science/article/pii/S0962184907000315. doi:10.1016/j.appsy.2007.09.001.
  * Jaques et al. [2017] N. Jaques, O. O. Rudovic, S. Taylor, A. Sano, R. Picard,  Predicting Tomorrowâ€™s Mood, Health, and Stress Level using Personalized Multitask Learning and Domain Adaptation,  in: Proceedings of IJCAI 2017 Workshop on Artificial Intelligence in Affective Computing, PMLR, 2017, pp. 17–33. URL: https://proceedings.mlr.press/v66/jaques17a.html, iSSN: 2640-3498.
  * Vaswani et al. [2023] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention Is All You Need, 2023. URL: http://arxiv.org/abs/1706.03762, arXiv:1706.03762 [cs].
  * Adoma et al. [2020] A. F. Adoma, N.-M. Henry, W. Chen,  Comparative Analyses of Bert, Roberta, Distilbert, and Xlnet for Text-Based Emotion Recognition,  in: 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 2020, pp. 117–121. URL: https://ieeexplore.ieee.org/document/9317379. doi:10.1109/ICCWAMTIP51612.2020.9317379, iSSN: 2576-8964.
  * Kumar and Raman [2022] P. Kumar, B. Raman,  A BERT based dual-channel explainable text emotion recognition system,  Neural Networks 150 (2022) 392–407. URL: https://www.sciencedirect.com/science/article/pii/S0893608022000958. doi:10.1016/j.neunet.2022.03.017.
  * Qin et al. [2023] X. Qin, Z. Wu, T. Zhang, Y. Li, J. Luan, B. Wang, L. Wang, J. Cui,  BERT-ERC: Fine-Tuning BERT Is Enough for Emotion Recognition in Conversation,  Proceedings of the AAAI Conference on Artificial Intelligence 37 (2023) 13492–13500. URL: https://ojs.aaai.org/index.php/AAAI/article/view/26582. doi:10.1609/aaai.v37i11.26582.
  * Devlin et al. [2019] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019. URL: http://arxiv.org/abs/1810.04805, arXiv:1810.04805 [cs].
  * Pell et al. [2009] M. D. Pell, L. Monetta, S. Paulmann, S. A. Kotz,  Recognizing Emotions in a Foreign Language,  Journal of Nonverbal Behavior 33 (2009) 107–120. URL: https://doi.org/10.1007/s10919-008-0065-7. doi:10.1007/s10919-008-0065-7.
  * Ingale and Chaudhari [2012] A. B. Ingale, D. S. Chaudhari,  Speech Emotion Recognition 2 (2012).
  * Xu et al. [2021] M. Xu, F. Zhang, W. Zhang,  Head Fusion: Improving the Accuracy and Robustness of Speech Emotion Recognition on the IEMOCAP and RAVDESS Dataset,  IEEE Access 9 (2021) 74539–74549. URL: https://ieeexplore.ieee.org/document/9381872/. doi:10.1109/ACCESS.2021.3067460.
  * Singh and Prasad [2023] V. Singh, S. Prasad,  Speech emotion recognition system using gender dependent convolution neural network,  Procedia Computer Science 218 (2023) 2533–2540. URL: https://www.sciencedirect.com/science/article/pii/S1877050923002272. doi:10.1016/j.procs.2023.01.227.
  * Mohan et al. [2023] M. Mohan, P. Dhanalakshmi, R. S. Kumar,  Speech Emotion Classification using Ensemble Models with MFCC,  Procedia Computer Science 218 (2023) 1857–1868. URL: https://www.sciencedirect.com/science/article/pii/S1877050923001631. doi:10.1016/j.procs.2023.01.163.
  * Macary et al. [2021] M. Macary, M. Tahon, Y. EstÃ¨ve, A. Rousseau,  On the Use of Self-Supervised Pre-Trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition,  in: 2021 IEEE Spoken Language Technology Workshop (SLT), 2021, pp. 373–380. URL: https://ieeexplore.ieee.org/abstract/document/9383456. doi:10.1109/SLT48900.2021.9383456.
  * Pepino et al. [2021] L. Pepino, P. Riera, L. Ferrer, Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings, 2021. URL: http://arxiv.org/abs/2104.03502. doi:10.48550/arXiv.2104.03502, arXiv:2104.03502 [cs, eess].
  * Zhao et al. [2022] Z. Zhao, Y. Wang, Y. Wang, Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognition, 2022. URL: http://arxiv.org/abs/2207.04697. doi:10.48550/arXiv.2207.04697, arXiv:2207.04697 [cs, eess].
  * Pan et al. [2024] R. Pan, J. A. García-Díaz, M. Á. Rodríguez-García, R. Valencia-García,  Spanish MEACorpus 2023: A multimodal speech-text corpus for emotion analysis in spanish from natural environments,  Computer Standards & Interfaces (2024) 103856.
  * Ekman and Oster [1979] P. Ekman, H. Oster,  Facial Expressions of Emotion,  Annual Review of Psychology 30 (1979) 527–554. URL: https://www.annualreviews.org/content/journals/10.1146/annurev.ps.30.020179.002523. doi:10.1146/annurev.ps.30.020179.002523, publisher: Annual Reviews.
  * Conneau et al. [2020] A. Conneau, A. Baevski, R. Collobert, A. Mohamed, M. Auli, Unsupervised Cross-lingual Representation Learning for Speech Recognition, 2020. URL: http://arxiv.org/abs/2006.13979, arXiv:2006.13979 [cs, eess].
  * Lange et al. [2021] L. Lange, H. Adel, J. Strötgen,  Boosting transformers for job expression extraction and classification in a low-resource setting,  CoRR abs/2109.08597 (2021). URL: https://arxiv.org/abs/2109.08597. arXiv:2109.08597.
  * Pan et al. [2024] R. Pan, J. A. García-Díaz, M. Á. Rodríguez-García, F. García-Sánchez, R. Valencia-García,  Overview of EmoSPeech 2024@IberLEF: Multimodal Speech-text Emotion Recognition in Spanish,  Procesamiento del Lenguaje Natural 73 (2024).
  * Chiruzzo et al. [2024] L. Chiruzzo, S. M. Jiménez-Zafra, F. Rangel,  Overview of IberLEF 2024: Natural Language Processing Challenges for Spanish and other Iberian Languages,  in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org, 2024.
  * Radford et al. [2022] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision, 2022. arXiv:2212.04356.
  * Chen et al. [2022] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian, Y. Qian, J. Wu, M. Zeng, X. Yu, F. Wei,  WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,  IEEE Journal of Selected Topics in Signal Processing 16 (2022) 1505–1518. URL: http://arxiv.org/abs/2110.13900. doi:10.1109/JSTSP.2022.3188113, arXiv:2110.13900 [cs, eess].
  * Kahn et al. [2020] J. Kahn, M. Riviere, W. Zheng, E. Kharitonov, Q. Xu, P. Mazare, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, E. Dupoux,  Libri-light: A benchmark for asr with limited or no supervision,  in: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020. URL: http://dx.doi.org/10.1109/ICASSP40776.2020.9052942. doi:10.1109/icassp40776.2020.9052942.
  * Chen et al. [2021] G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhang, M. Jin, S. Khudanpur, S. Watanabe, S. Zhao, W. Zou, X. Li, X. Yao, Y. Wang, Y. Wang, Z. You, Z. Yan, Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio, 2021. arXiv:2106.06909.
  * Wang et al. [2021] C. Wang, M. Rivière, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, E. Dupoux, Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation, 2021. arXiv:2101.00390.
  * Baevski et al. [2020] A. Baevski, H. Zhou, A. Mohamed, M. Auli, wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations, 2020. URL: http://arxiv.org/abs/2006.11477, arXiv:2006.11477 [cs, eess].
  * Ardila et al. [2020] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, G. Weber, Common voice: A massively-multilingual speech corpus, 2020. arXiv:1912.06670.
  * Gales et al. [2014] M. J. F. Gales, K. Knill, A. Ragni, S. P. Rath,  Speech recognition and keyword spotting for low-resource languages: Babel project research at cued,  in: Workshop on Spoken Language Technologies for Under-resourced Languages, 2014. URL: https://api.semanticscholar.org/CorpusID:7439227.
  * Pratap et al. [2020] V. Pratap, Q. Xu, A. Sriram, G. Synnaeve, R. Collobert,  MLS: A Large-Scale Multilingual Dataset for Speech Research,  in: Interspeech 2020, 2020, pp. 2757–2761. URL: http://arxiv.org/abs/2012.03411. doi:10.21437/Interspeech.2020-2826, arXiv:2012.03411 [cs, eess].
  * Hsu et al. [2021] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, A. Mohamed, HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units, 2021. URL: http://arxiv.org/abs/2106.07447, arXiv:2106.07447 [cs, eess].
  * Devlin et al. [2019] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. arXiv:1810.04805.
  * Lange et al. [2021] L. Lange, H. Adel, J. Strötgen,  Boosting transformers for job expression extraction and classification in a low-resource setting,  in: Proceedings of The Iberian Languages Evaluation Forum (IberLEF 2021), CEUR Workshop Proceedings, 2021. URL: http://ceur-ws.org/Vol-2943/meddoprof_paper1.pdf.
  * Conneau et al. [2020] A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov, Unsupervised cross-lingual representation learning at scale, 2020. arXiv:1911.02116.
  * Cañete [2019] J. Cañete, Compilation of large spanish unannotated corpora, 2019. URL: https://doi.org/10.5281/zenodo.3247731. doi:10.5281/zenodo.3247731.
  * Cañete et al. [2020] J. Cañete, G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, J. Pérez,  Spanish pre-trained bert model and evaluation data,  in: PML4DC at ICLR 2020, 2020.
  * Tiedemann [2012] J. Tiedemann,  Parallel data, tools and interfaces in OPUS,  in: N. Calzolari, K. Choukri, T. Declerck, M. U. Doğan, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, S. Piperidis (Eds.), Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), European Language Resources Association (ELRA), Istanbul, Turkey, 2012, pp. 2214–2218. URL: http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf.
  * Hendrycks and Gimpel [2016] D. Hendrycks, K. Gimpel,  Bridging nonlinearities and stochastic regularizers with gaussian error linear units,  CoRR abs/1606.08415 (2016). URL: http://arxiv.org/abs/1606.08415. arXiv:1606.08415.
  * India et al. [2019] M. India, P. Safari, J. Hernando,  Self multi-head attention for speaker recognition,  CoRR abs/1906.09890 (2019). URL: http://arxiv.org/abs/1906.09890. arXiv:1906.09890.
  * Costa et al. [2024] F. Costa, M. India, J. Hernando, Double multi-head attention multimodal system for odyssey 2024 speech emotion recognition challenge, 2024. arXiv:2406.10598.
  * India et al. [2021] M. India, P. Safari, J. Hernando, Double multi-head attention for speaker verification, 2021. arXiv:2007.13199.
  * Glorot and Bengio [2010] X. Glorot, Y. Bengio,  Understanding the difficulty of training deep feedforward neural networks,  in: Y. W. Teh, M. Titterington (Eds.), Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, PMLR, Chia Laguna Resort, Sardinia, Italy, 2010, pp. 249–256. URL: https://proceedings.mlr.press/v9/glorot10a.html.
  * Loshchilov and Hutter [2019] I. Loshchilov, F. Hutter, Decoupled weight decay regularization, 2019. arXiv:1711.05101.