# Specification and $\omega$-chaos in non-compact systems

Cordell Hammon  Department of Mathematics, Baylor University, Waco, TX
76798–7328,USA<EMAIL_ADDRESS>,  Jonathan Meddaugh  Department of
Mathematics, Baylor University, Waco, TX 76798–7328,USA
<EMAIL_ADDRESS>,  Jasmin Mohn  Department of Mathematics, Baylor
University, Waco, TX 76798–7328,USA<EMAIL_ADDRESS>and  Brian E.
Raines  Department of Mathematics, Baylor University, Waco, TX 76798–7328,USA
<EMAIL_ADDRESS>
###### Abstract.

In this paper, we demonstrate conditions under which a Lindelöf dynamical
system exhibits $\omega$-chaos. In particular, if a system exhibits a
generalized version of the specification property and has at least three
points with mutually separated orbit closures, then the system exhibits dense
$\omega$-chaos.

##  1\. Introduction

Chaotic behavior in dynamical systems is an area of considerable mathematical
interest. Fittingly, there are a large number of non-equivalent notions of
chaos including Li-Yorke chaos [6], distributional chaos [13], Devaney chaos
[2], and $\omega$-chaos [5].

In this paper, we focus on $\omega$-chaos, which was introduced by Li in 1993
and shown to be equivalent to positive topological entropy in the context of
continuous interval maps [5]. Briefly, a system exhibits $\omega$-chaos
provided there exists an uncountable $\omega$-scrambled set, i.e., an
uncountable set of points, each of which has $\omega$-limit set containing
non-periodic points, and each pair of which have nonempty intersection but
uncountable relative complements.

In 2009, Lampart and Oprocha demonstrated that a large class of shift spaces
exhibit $\omega$-chaos [4]. In particular they demonstrate that (weak)
specification and the existence of a non-transitive orbit is enough to
guarantee that a non-degenerate shift space exhibits $\omega$-chaos. Briefly,
a system $(X,f)$ exhibits _(weak) specification_ provided that for a desired
tolerance $\delta$ there exists a “relaxation time” $N_{\delta}$ such that for
$a_{1}\leq b_{1}<b_{1}+N_{\delta}\leq a_{2}\leq
b_{2}<\cdots<b_{n-1}+N_{\delta}\leq a_{n}\leq b_{n}$ ($n=2$ for weak
specification) and $x_{1},\ldots x_{n}\in X$, there exists a point $z\in X$
such that the orbits of $z$ and $x_{i}$ agree for iterates of $f$ between
$a_{i}$ and $b_{i}$.

The specification property was first studied by Bowen [1] and has implications
in a wide variety of settings, including the existence of certain invariant
measures [14]. In addition, the relationship between the specification
property and chaos has been well-studied, see e.g., [11, 12, 15, 16].

In their 2009 paper, Lampart and Oprocha asked whether every system with the
specification property exhibits $\omega$-chaos. Meddaugh and Raines
demonstrated that a weak form of specification was sufficient to ensure
$\omega$-chaos for shift spaces [7]. Hunter and Raines tendered a partial
answer to this question for more general systems when they showed that compact
metric systems with the specification property and uniform expansion near a
fixed point exhibit $\omega$-chaos [3].

The main result of this paper is a natural extension of this result to
Lindelöf systems.

######  Main Theorem.

Let $(X,f)$ be an expansive dynamical system such that $X$ is Lindelöf, every
open set of $X$ is uncountable, $(X,f)$ has ISP, and there exist $t_{0}$,
$t_{1},s\in X$ such that
$\overline{Orb(t_{0})},\overline{Orb(t_{1})},\overline{Orb(s)}$ are each a
non-zero distance away from each other. Then $(X,f)$ exhibits dense
$\omega$-chaos.

##  2\. Preliminaries

For the purposes of this paper, we adopt the following notational conventions.
We use $\omega$ to denote $\mathbb{N}\cup\\{0\\}$. Given a metric space $X$
with metric $d$ and $A,B\subseteq X$, we take $d(A,B)=\inf\\{d(a,b):a\in
A,b\in B\\}$. A space $X$ is Lindelöf provided that every open cover of $X$
has a countable subcover.

A dynamical system is a pair $(X,f)$ consisting of a metric space $X$ and
continuous surjection $f:X\to X$. For a given system $(X,f)$, $f^{n}$ denotes
the $n$-fold composition of $f$ and the orbit of a point $x\in X$ is the set
$Orb_{f}(x)=\\{f^{n}(x):n\in\omega\\}$. A point $x$ is periodic with prime
period $p$ provided that $f^{p}(x)=x$ and $p$ is the minimal iterate for which
that holds. The $\omega$-limit set of $f$ in the system $(X,f)$ is the set of
accumulation points of the orbit of $x$, i.e.,
$\omega_{f}(x)=\bigcap_{n\in\omega}\overline{\\{f^{i}(x):i\geq n\\}}$. When
the function $f$ is clear from context, the subscripts on $\omega_{f}$ and
$Orb_{f}$ will be supressed.

The primary object of investigation in this paper is the notion of
$\omega$-chaos, as defined by Li [5].

######  Definition 1.

Let $(X,f)$ be a dynamical system.

A subset $A$ of $X$ is $\omega$-scrambled provided that, for $x\neq y\in A$,
the following hold

  1. (1)

$\omega(x)\cap\omega(y)\neq\varnothing$,

  2. (2)

$\omega(x)\setminus\omega(y)$ is uncountable, and

  3. (3)

$\omega(x)$ contains non-periodic points

The system exhibits $\omega$-chaos if it contains an uncountable
$\omega$-scrambled set.

As mentioned in the introduction, the specification property, as defined by
Bowen, will be important in what follows.

######  Definition 2.

We say a dynamical system $(X,f)$ has the specification property (SP) if for
every $\delta>0$ there exists some $N_{\delta}\in\mathbb{N}$ such that for any
finite collection of points $x_{1},x_{2},x_{3}\dots x_{n}\in X$ and any
sequence $0\leq a_{1}\leq b_{1}<a_{2}\leq b_{2}<\dots<a_{n}\leq b_{n}$ with
$a_{i+1}-b_{i}\geq N_{\delta}$ there is a point $p\in X$ such that
$d(f^{j}(p),f^{j}(x_{i}))<\delta$ for $a_{i}\leq j\leq b_{i}$.

In the context of non-compact dynamical systems, we will make use of a
generalized notion of the specification property. It is not surprising that
such a generalization is appropriate—indeed similar generalizations naturally
appear in the study of $\omega$-limits in non-compact settings [8].

######  Definition 3.

We say a dynamical system $(X,f)$ has the infinite specification property
(ISP) if for every $\delta>0$ there exists some $N_{\delta}\in\mathbb{N}$ such
that for any countable collection of points $x_{1},x_{2},x_{3},\ldots\in X$
and any sequence $0\leq a_{1}\leq b_{1}<a_{2}\leq b_{2}<\dots$ with
$a_{i+1}-b_{i}\geq N_{\delta}$ there is a point $p\in X$ such that
$d(f^{j}(p),f^{j}(x_{i}))<\delta$ for $a_{i}\leq j\leq b_{i}$.

This definition may seem too specific to be applicable. But as we will see,
there are many systems with ISP. It is worth nothing that, in compact systems,
the two forms of specification are equivalent.

######  Lemma 4.

If $(X,f)$ has ISP then it has SP. If $X$ is compact, then the converse holds.

###### Proof.

Suppose $(X,f)$ is a system with ISP. Then for any finite collection of points
$x_{1},x_{2},\ldots,x_{n}$ there is a countable collection of points
$y_{1},y_{2},\ldots$ such that $y_{i}=x_{i}$ for all $i\leq n$. There exists a
point $p\in X$ that witnesses ISP with respect to $y_{1},y_{2},\ldots$ and
thus witnesses SP with respect to $x_{1},x_{2},\ldots,x_{n}$.

Suppose now that $(X,f)$ is a compact dynamical system with SP. Fix $\delta>0$
and let $N_{\delta/2}$ be given by SP. Let $x_{1},x_{2},\dots$ be a countable
collection of points in $X$, and let $a_{1}\leq b_{1}<a_{2}\leq b_{2}<\dots$
with $a_{i+1}-b_{i}\geq N_{\delta/2}$ be a sequence of natural numbers. For
each $n\in\mathbb{N}$ we can find $p_{n}$ so that
$d(f^{j}(p_{n}),f^{j}(x_{i}))<\delta/2$ for $a_{i}\leq j\leq b_{i},i\leq n$.
The set $\\{p_{n}\\}_{n\in\mathbb{N}}$ is an infinite subset of a compact
space and thus has an accumulation point, $q$. Since $f^{i}$ is continuous for
all $i\in\mathbb{N}$ we have $\lim_{n\to\infty}d(f^{i}(p_{n}),f^{i}(q))\to 0$
and so $d(f^{j}(q),f^{j}(x_{i}))\leq\delta/2<\delta$ for $a_{i}\leq j\leq
b_{i}$. ∎

For the purposes of this paper, we will make use of the fact that for a
countable (finite or infinite) alphabet $\Sigma$, the shift space on $\Sigma$
exhibits ISP.

######  Definition 5.

Let $\Sigma$ be a nonempty collection of symbols. The space $\Sigma^{\omega}$
is the space of all sequences $(x_{0},x_{1},x_{2},\ldots)$ where
$x_{i}\in\Sigma$ for all $i\in\omega$. The space is equipped with a map
$\sigma$ called the shift map where
$\sigma(x_{0},x_{1},x_{2},\ldots)=(x_{1},x_{2},x_{3},\ldots)$.

In the case that $\Sigma$ is a collection of numbers, $\Sigma^{\omega}$ is
equipped with a metric.

$d((x_{0},x_{1},\ldots),(y_{0},y_{1},\ldots))=\sum_{i=0}^{\infty}\frac{\min\\{|x_{i}-y_{i}|,1\\}}{2^{i}}$

Of particular import going forward is the shift map with $\Sigma=2=\\{0,1\\}$,
i.e. $2^{\omega}=\\{(x_{0},x_{1},\ldots):x_{i}\in\\{0,1\\}\text{ for all
}i\in\omega\\}$ due to the fact that Lampart and Oprocha demonstrated that it
exhibits $\omega$-chaos [4].

######  Lemma 6.

The shift space $\Sigma^{\omega}$ with the shift map has ISP if $|\Sigma|\geq
2$.

###### Proof.

Let $\delta>0$ and let $N$ be such that $2^{-N}<\delta/2$. Let
$x_{1},x_{2},\dots$ be a countable collection of points in
$\Sigma^{\mathbb{N}}$, and let $a_{1}\leq b_{1}<a_{2}\leq b_{2}<a_{3}\leq
b_{3}<\dots$ be a sequence of natural numbers such that $a_{i}-b_{i-1}>N$.

Finally, let $y$ be a point in $\Sigma^{\mathbb{N}}$ such that
$y_{[a_{i},b_{i}+N]}=(x_{i})_{[a_{i},b_{i}+N]}$. Note that such points exist
since $a_{i}-b_{i-1}>N$. Then $\sigma^{j}(y)$ and $\sigma^{j}(x_{i})$ agree
for at least $N$ symbols when $a_{i}\leq j\leq b_{i}$, so
$d(\sigma^{j}(y),\sigma^{j}(x_{i}))<2^{-N}<\delta$ when $a_{i}\leq j\leq
b_{i}$ as required for ISP. ∎

In order to make discussing the patterns of the orbit of a point given by
specification easier, we introduce the following lemma.

######  Lemma 7.

Let $(X,f)$ be a dynamical system with infinite specification and fix
$\delta>0$. Let $N_{\delta}>0$ witness specification. For
$z_{0},z_{1},\ldots\in X$, $c_{0},c_{1},\ldots\in\omega$, and $M\geq
N_{\delta}$, there exists a $p\in X$ such that the $f^{i}(p)$ is less than
$\delta$ away from the $i$-th term in the following sequence whenever that
term is not $*$. (We use the convention that $(*)^{M}$ denotes $M$-many
consecutive entries which are $*$.)

$\displaystyle z_{0},f(z_{0}),\dots,f^{c_{0}}(z_{0}),$
$\displaystyle(*)^{M},z_{1},f(z_{1}),\dots,f^{c_{1}}(z_{1}),(*)^{M},\ldots$
$\displaystyle\dots,$
$\displaystyle(*)^{M},z_{n},f(z_{n}),\dots,f^{c_{n}}(z_{n}),(*)^{M},\ldots$

###### Proof.

Fix $z_{0},z_{1},\ldots\in X$, $c_{0},c_{0},\ldots\in\omega$, and $M\geq
N_{\delta}$. We will apply the infinite specification property to the sequence
$x_{0},x_{1},\ldots$ and $0\leq a_{0}\leq b_{0}<a_{1}\leq b_{1}\dots$ defined
as follows. Fix $a_{0}=0$ and $b_{0}=c_{0}$. If $b_{i}$ is defined, choose
$a_{i+1}=b_{i}+M\geq b_{i}+N_{\delta}$. If $a_{i}$ is defined, choose
$b_{i}=a_{i}+c_{i}$. By surjectivity, choose $x_{i}\in f^{-a_{i}}(z_{i})$. By
infinite specification property, choose a point $p\in X$ such that
$d(f^{j}(p),f^{j}(x_{p}))<\delta$ when $a_{i}\leq j\leq b_{i}$. It is
immediately clear that $p$ satisfies the conclusion of the lemma. ∎

If $(X,f)$ has only the standard specification property, a similar result
holds for any finite lists $x_{0},\ldots x_{n}$ and $c_{0},\ldots c_{n}$.

Our main result requires a form of local expansion.

######  Definition 8.

We say $f$ is expansive if there is some $\eta>0,\lambda>1$ such that if
$0<d(x,y)<\eta$ then $d(f(x),f(y))>\lambda d(x,y)$ for all $x,y\in X$.

The following lemma will be used frequently in the following section.

######  Lemma 9.

Let $(X,f)$ be expansive and let $\eta>0,\lambda>1$ witness expansivity. If
$p,q\in X,j\in\mathbb{N}$ are such that $d(f^{i}(p),f^{i}(q))<\eta$ for all
$i\leq j$, then $d(p,q)<\eta\lambda^{-j}$. Moreover, if $p\in X$ and
$\\{q_{n}\\}_{n\in\mathbb{N}}$ is a sequence of points such that
$d(f^{i}(p),f^{i}(q_{n}))<\eta$ for all $i\leq j_{n}$ where $j_{n}\to\infty$.
Then $q_{n}\to p$.

###### Proof.

Since $d(f^{i}(p),f^{i}(q))<\eta$ for all $i\leq j$ we have
$\eta>d(f^{j}(p),f^{j}(q))>\lambda
d(f^{j-1}(p),f^{j-1}(q))>\lambda^{2}d(f^{j-2}(p),f^{j-2}(q))>\dots>\lambda^{j}d(p,q).$
Dividing both sides by $\lambda^{j}$ yields the result.

From here, $d(f^{j_{n}}(p),f^{j_{n}}(q_{n}))<\eta$ implies
$d(p,q_{n})<\eta\lambda^{-j_{n}}$. Since $j_{n}\to\infty$ we have $q_{n}\to
p$. ∎

It is worth noting that $\omega$-chaos is a rather local property. There are
cases where a dynamical system $(X,f)$ exhibits $\omega$-chaos even though the
only “chaotic” part of $X$ is a measure theoretically small portion of the
space. Consider a finite measure system $(X,f)$ with $\omega$-chaos and affix
to it a much larger space $X^{\prime}$ and extend $f$ to $X^{\prime}$ in such
a way that $f|X^{\prime}$ is the identity. The new system $(X\cup
X^{\prime},f)$ still has $\omega$-chaos, but the chaos is relegated to the
comparatively small portion of the system, $X$. To consider a more global form
of $\omega$-chaos we introduce the following.

######  Definition 10.

We say a system exhibits dense $\omega$-chaos if any open set contains an
uncountable $\omega$-scrambled set.

Systems with dense $\omega$-chaos do exist. In fact, $2^{\omega}$ has dense
$\omega$-chaos To see this, let $\alpha\in 2^{\omega}$. Note that prepending
any finite word $w$ to an infinite word $\alpha$ does not alter the
$\omega$-limit set of $\alpha$. That is to say
$\omega(w\alpha)=\omega(\alpha)$. Note, also, that basic open sets in
$2^{\omega}$ take the form of cylinder sets. So pick any finite word $w\in
2^{\omega}$. Prepend every point in $2^{\omega}$ with $w$ to get the open set
$[w]\subseteq 2^{\omega}$. This open set exhibits $\omega$-chaos.

##  3\. Existence of $\omega$-chaos in certain Lindelöf systems

In this section, we will prove the following theorem.

######  Main Theorem.

Let $(X,f)$ be an expansive dynamical system such that $X$ is Lindelöf, every
open set of $X$ is uncountable, $(X,f)$ has ISP, and there exist $t_{0}$,
$t_{1},s\in X$ such that
$\overline{Orb(t_{0})},\overline{Orb(t_{1})},\overline{Orb(s)}$ are each a
non-zero distance away from each other. Then $(X,f)$ exhibits dense
$\omega$-chaos.

We begin with the following constructions. Let $(X,f)$ be given as in the main
theorem. Let $\eta,\lambda$ witness the expansivity of $f$. Let
$t_{0},t_{1},s$ be three points such that the orbit closures of these points
have non-zero distance between each other. Fix $\xi$ in $X$ and $U$ a
neighborhood thereof. Let $D>0$ be small enough so that the ball of radius $D$
around $\xi$ is contained entirely within $U$. Fix $\epsilon>0$ such that

$\min\\{d(\overline{Orb(t_{o})},\overline{Orb(t_{1})}),d(\overline{Orb(t_{o})},\overline{Orb(s)}),d(\overline{Orb(t_{1})},\overline{Orb(s)}),D,\eta\\}>2\epsilon.$

Lastly, let $N$ be given by ISP with respect to $\epsilon/2$ and let
$P\in\mathbb{N}$ such that $P>N$. We construct two sequences of numbers as
follows: $a_{0}=0$, if $a_{i}$ is defined, let $b_{i}=a_{i}+P$, and if $b_{i}$
is defined, let $a_{i+1}=b_{i}+N$.

With that construction done, we are now ready to leverage ISP.

######  Definition 11.

For $\beta\in 2^{\omega}$, let $E_{\beta}=\\{x\in X:a_{i}\leq j\leq
b_{i},d(f^{j}(x),f^{j-a_{i}}(t_{\beta_{i}}))\leq\frac{\epsilon}{2}\text{ for
all }i\in\omega\\}$.

Note that $E_{\beta}\neq\varnothing$ due to ISP.

######  Lemma 12.

$E_{\beta}$ is closed, and $E_{\beta}\cap E_{\gamma}\neq\varnothing$ if and
only if $\beta=\gamma$.

###### Proof.

Let $\\{p_{k}\\}_{k\in\omega}$ be a sequence of points in $E_{\beta}$ which
limits to $p$. Choose $i\in\omega$ and choose some
$j\in\\{a_{i},\dots,b_{i}\\}$. Since $f^{j}$ is a continuous function,
$f^{j}(p_{k})\to f^{j}(p)$ and thus
$d(f^{j}(p),f^{j-a_{i}}(t_{\beta_{i}}))\leq\epsilon/2$. So $p\in E_{\beta}$.

To verify the other claim, let $p\in E_{\beta}\cap E_{\gamma}$. Then for any
$j\in\\{a_{i},\dots,b_{i}\\}$ we have that
$d(f^{j}(p),f^{j-a_{i}}(t_{\beta_{i}}))\leq\epsilon/2$ and
$d(f^{j}(p),f^{j-a_{i}}(t_{\gamma_{i}}))\leq\epsilon/2$. But, since
$d(Orb(t_{0}),Orb(t_{1}))>2\epsilon$, we must have
$t_{\beta_{i}}=t_{\gamma_{i}}$ for all $i$ and thus $\beta=\gamma$. ∎

######  Lemma 13.

If $\beta\in 2^{\omega}$ is not periodic, then $E_{\beta}$ does not contain
periodic points.

###### Proof.

Suppose $q\in E_{\beta}$ is periodic with period $K$. Then
$d(f^{j}(q),f^{j-a_{i}}(t_{\beta_{i}}))\leq\epsilon/2$ for all
$i\in\omega,a_{i}\leq j\leq b_{i}$. Note that $a_{i}=i(N+P)$ so
$a_{i}+a_{h}=i(N+P)+h(N+P)=(i+h)(N+P)=a_{i+h}$. Let $L=lcm(K,N+P)$ and choose
$m$ such that $L=m(N+P)$ and note that $L=a_{m}$. Moreover,
$f^{L+j}(q)=f^{j}(q)$, and thus we have the following.

$\displaystyle d(t_{\beta_{m+i}},t_{\beta_{i}})$ $\displaystyle\leq
d(t_{\beta_{m+i}},f^{a_{i}}(q))+d(f^{a_{i}}(q),t_{\beta_{i}})$
$\displaystyle=d(t_{\beta_{m+i}},f^{L+a_{i}}(q))+d(f^{a_{i}}(q),t_{\beta_{i}})$
$\displaystyle=d(t_{\beta_{m+i}},f^{a_{m}+a_{i}}(q))+d(f^{a_{i}}(q),t_{\beta_{i}})$
$\displaystyle=d(t_{\beta_{m+i}},f^{a_{m+i}}(q))+d(f^{a_{i}}(q),t_{\beta_{i}})$
$\displaystyle\leq\epsilon$

Since $Orb(t_{\beta_{0}})$ and $Orb(t_{\beta_{1}})$ are $2\epsilon$ apart this
means that $\beta_{i+m}=\beta_{i}$ for all $i\in\omega$, i.e. $\beta$ is
periodic. ∎

######  Lemma 14.

Let $\beta\in 2^{\omega}$ and let $q\in E_{\beta}$. Then $f^{a_{k}}(q)\in
E_{\sigma^{k}(\beta)}$.

###### Proof.

Fix $k\in\omega$. By definition of $E_{\beta}$,
$d(f^{j}(q),f^{j-a_{i}}(t_{\beta_{i}}))\leq\frac{\epsilon}{2},a_{i}\leq j\leq
b_{i}\text{ for all }i\in\omega$ and so it certainly holds for all $i>k$. So
$d(f^{j}(f^{a_{k}}(q)),f^{j-a_{i}}(t_{\sigma^{k}(\beta)_{i}})\leq\epsilon/2$.
∎

Going forward, the following sets will be useful

$G_{\beta}={\bigcup_{i\in\omega}f^{i}(E_{\beta})},\quad
H_{\beta}^{\prime}={\bigcup_{\alpha\in\omega_{\sigma}(\beta)}G_{\alpha}},\quad
H_{\beta}=\overline{\bigcup_{\alpha\in\omega_{\sigma}(\beta)}G_{\alpha}}$

The definition of $H_{\beta}$ leads to the following useful lemmas which will
be used to establish that certain $\omega$-limit sets have uncountable set
differences. The first lemma is immediate from the definitions.

######  Lemma 15.

If $\beta\in 2^{\omega}$ is such that $\omega(\beta)$ is uncountable, then
$H_{\beta}^{\prime}$ and $H_{\beta}$ are uncountable.

Less immediately, we have the following.

######  Lemma 16.

If $\beta,\chi\in 2^{\omega}$ are such that $\beta\notin\overline{Orb(\chi)}$,
then $E_{\beta}\cap H_{\chi}=\varnothing$.

###### Proof.

Let $\beta,\chi\in 2^{\omega}$ such that $\beta\notin\overline{Orb(\chi)}$.
Let $r$ be such that $\beta_{[0,r]}$ does not occur in $\chi$. Suppose, by way
of contradiction, that $\\{z_{k}\\}_{k\in\omega}$ is a sequence of points in
$H_{\chi}^{\prime}$ which limits to $z\in E_{\beta}$. Then for all but
finitely many $z_{k}$ we have, for $j\leq b_{r}$,

(1) $d(f^{j}(z_{k}),f^{j}(z))\leq\epsilon/2$

Since each $z_{k}$ belongs to $H_{\chi}^{\prime}$, for each $k$ there exists
$v_{k}\in\omega$ minimal such that $z_{k}\in f^{v_{k}}(E_{\alpha_{k}})$ for
some $\alpha_{k}$. As such, by Lemma 14, for each $i\in\omega$ with
$a_{i}>v_{k}$ we have $f^{a_{i}-v_{k}}(z_{k})\in E_{\sigma^{i}(\alpha_{k})}$.
For each $k$, let $i_{k}$ be minimal so that $a_{i_{k}}\geq v_{k}$ and let
$q_{k}=a_{i_{k}}-v_{k}.$ Then $f^{q_{k}}(z_{k})\in E_{\theta^{k}}$ for some
$\theta^{k}\in\omega(\chi)$.

Notice that by construction of the $E_{\alpha}$ sets, any point in
$E_{\alpha}$ is exactly $P+N$ many iterates away from being in
$E_{\sigma(\alpha)}.$ So any point in $H_{\chi}^{\prime}$ is fewer than $P+N$
iterates away from being in some $E_{\alpha}$ (if it take $N+P$ many iterates,
then we must have started out in some $E_{\sigma^{-1}(\alpha)}$ set and thus
needed 0 iterates to be in an $E$ set). Moreover, for any point in
$E_{\alpha}$, the first $P$ many iterates are controlled by specification, and
iterates number $P+1,P+2,\dots,P+N-1$ are all not controlled by specification.
So, if $q_{k}<N$, then $z_{k}$ is in a part of the orbit of $E_{\alpha^{k}}$
that is not controlled by specification. Similarly, if $q_{k}\geq N$, then
$z_{k}$ is in a part of the orbit of $E_{\alpha^{k}}$ that _is_ controlled by
specification. As such we have two cases.

If $q_{k}<N$, let $y_{k}=q_{k}$ and let $\delta^{k}=\theta^{k}$. So
$f^{y_{k}}(z_{k})\in E_{\delta^{k}}$. If, on the other hand, $q_{k}\geq N$,
then since $z_{k}$ is in a part of the orbit of $E_{\alpha^{k}}$ that is
controlled by specification, we let $y_{k}=0$ and let $\delta^{k}=\alpha^{k}$.

Thus, in either case we have
$f^{a_{i}}(f^{y_{k}}(z_{k}))=f^{a_{i}+y_{k}}(z_{k})\in
B_{\epsilon/2}(Orb(t_{\delta^{k}_{i}}))$ for all $i\in\omega$.

But, $\beta_{[0,r]}$ does not occur in $\chi$, so for each $k$ there is some
index $i_{k}\leq r$ such that $\beta_{i_{k}}\neq\delta^{k}_{i_{k}}$ and so
$f^{a_{i_{k}}}(f^{y_{k}}(z_{k}))\in B_{\epsilon/2}(t_{1-\beta_{i_{k}}})$.

Since $z\in E_{\beta}$, we have $f^{a_{i_{k}}}(z)\in
B_{\epsilon/2}(t_{\beta_{i_{k}}})$ for each $k\in\omega$, but more importantly
$f^{a_{i_{k}}+y_{k}}(z)\in B_{\epsilon/2}(f^{y_{k}}(t_{\beta_{i_{k}}}))$ since
$y_{k}\leq N$ and so $a_{i_{k}}+y_{k}<b_{i_{k}}.$

But this, combined with (1), means $f^{a_{i_{k}}+y_{k}}(z)\in
B_{\epsilon}(Orb(t_{\beta_{i_{k}}}))\cap
B_{\epsilon}(Orb(t_{1-\beta_{i_{k}}}))$, which is empty, a contradiction. ∎

We now demonstrate how, given $\beta$ and $\chi$ in $2^{\omega}$, to construct
a pair of points whose $\omega$-limit sets meet and will each contain one of
$H_{\beta}$ or $H_{\chi}$ and miss the other.

For $n\in\omega,z\in H_{\beta}$ let
$U_{z,n}=\\{y:d(f^{i}(y),f^{i}(z))<\epsilon/4\text{ for }i\leq b_{n}\\}$. Then
$\mathcal{U}_{n}=\\{U_{z,n}:z\in H_{\beta}\\}$ is an open cover of
$H_{\beta}$. $H_{\beta}$ is closed subset of the Lindelöf space $X$ and is
therefore Lindelöf [10]. Thus, we can choose a countable set
$\\{U_{z_{1},n},U_{z_{2},n},\dots\\}$ which is an open cover of $H_{\beta}$.
Finally, let $H_{\beta,n}=\\{z_{1},z_{2},z_{3},\dots\\}$. Since each
$H_{\beta,n}$ is countable, we can enumerate the sets:

$\displaystyle H_{\beta,1}$
$\displaystyle=\\{\gamma_{1,0},\gamma_{1,1},\gamma_{1,2},\dots\\}$
$\displaystyle H_{\beta,2}$
$\displaystyle=\\{\gamma_{2,0},\gamma_{2,1},\gamma_{2,2},\dots\\}$
$\displaystyle H_{\beta,3}$
$\displaystyle=\\{\gamma_{3,0},\gamma_{3,1},\gamma_{3,2},\dots\\}$

We can list the elements of the union of the $H_{\beta,i}$ sets via the
following diagonalization:

$(\gamma_{1,0},\gamma_{1,1},\gamma_{2,0},\gamma_{1,2},\gamma_{2,1},\gamma_{3,0},\dots)$

Since the collection of sets $H_{\beta,n}$ are not necessarily disjoint, the
previous list may have repetition, but that does not affect the proof.

Let $M$ be given by ISP with respect to $\epsilon/8$. Now, apply Lemma 7 to
find a point $p_{\beta}$ such that $f^{i}(p_{\beta})$ is less than
$\epsilon/8$ from the $i$-th term the sequence

$\displaystyle\xi,(*)^{M},$
$\displaystyle\gamma_{1,0},\dots,f^{b_{1}}(\gamma_{1,0}),(*)^{M},s,(*)^{M},$
$\displaystyle\gamma_{1,1},\dots,f^{b_{1}}(\gamma_{1,1}),(*)^{M},s,f(s),(*)^{M},$
$\displaystyle\gamma_{2,0},\dots,f^{b_{2}}(\gamma_{2,0}),(*)^{M},s,f(s),f^{2}(s),(*)^{M},$
$\displaystyle\dots.$

whenever the $i$-th term is not a $*$. Note that the sequence follows the
orbit of the point $\gamma_{j,w}$ for $b_{j}$-many iterates, and we follow the
orbit of $s$ for $(n-1)$-many iterates $n-$th time we arrive at $s$ in the
sequence.

We now turn our attention to the $\omega$-limit set of $p_{\beta}$.

######  Lemma 17.

$H_{\beta}\cup\\{s\\}\subseteq\omega(p_{\beta})$.

###### Proof.

To see that $s\in\omega(p_{\beta})$, use the specification pattern of
$p_{\beta}$ to define a sequence of integers $i_{k}$ such that
$f^{i_{k}}(p_{\beta})\in B_{\epsilon/8}(s)$ and $f^{i_{k}+j}(p_{\beta})\in
B_{\epsilon/8}(f^{j}(s))$ for all $j\leq k,k\in\omega$. Lemma 9 then gives us
that $\lim_{k\to\infty}f^{i_{k}}(p_{\beta})=s$.

Now to see that $H_{\beta}$ is contained in $\omega(p_{\beta})$, let
$\upsilon\in H_{\beta}$. For each $n\in\mathbb{N}$ there exists an $l_{n}$
such that $\upsilon\in U_{\gamma_{n,l_{n}},n}$. Notice that since $\upsilon\in
U_{\gamma_{n,l_{n}},n}$ we have that
$d(f^{i}(\upsilon),f^{i}(\gamma_{n,l_{n}}))<\epsilon/4$ for all $i\leq b_{n}$.
Thus, by the expansivity of $f$, together with Lemma 9 we have that
$\lim_{n\to\infty}\gamma_{n,l_{n}}=\upsilon$.

Now define a sequence $i_{n}$ such that $f^{i_{n}}(p_{\beta})\in
B_{\epsilon/8}(\gamma_{n,l_{n}})$ and for all $j\in\\{0,1,\dots,b_{n}\\}$ we
have $f^{i_{n}+j}(p_{\beta})\in B_{\epsilon/8}(f^{j}(\gamma_{n,l_{n}}))$. Such
a sequence exists by the construction of $p_{\beta}$ via ISP. Then,

$d(f^{i_{n}}(p_{\beta}),\upsilon)\leq
d(f^{i_{n}}(p_{\beta}),\gamma_{n,l_{n}})+d(\gamma_{n,l_{n}},\upsilon).$

But, as we have already seen $d(\gamma_{n,l_{n}},\upsilon)\to 0$ as
$n\to\infty$. Moreover, by Lemma 9, since

$d(f^{i_{n}+j}(p_{\beta}),f^{j}(\gamma_{n,l_{n}}))<\eta$

for all $j<b_{n}$, we have
$d(f^{i_{n}}(p_{\beta}),\gamma_{n,l_{n}})<\eta\lambda^{-b_{n}}$. So as
$n\to\infty,d(f^{i_{n}}(p_{\beta}),\gamma_{n,l_{n}})\to 0$. ∎

######  Lemma 18.

If $\beta,\gamma\in 2^{\omega}$ are such that
$\overline{Orb(\beta)}\cap\overline{Orb(\gamma)}=\varnothing$, then
$\omega(p_{\gamma})\cap H_{\beta}^{\prime}=\varnothing$.

###### Proof.

Let $q\in H_{\beta}^{\prime}$. Then
$q\in\bigcup_{\alpha\in\omega(\beta)}G_{\alpha}$. As such, be Lemma 14 there
is some $m\in\omega$ such that $f^{m}(q)\in E_{\alpha}$ for some
$\alpha\in\omega(\beta)$. By definition of $E_{\alpha}$ we have
$d(f^{j}(f^{m}(q)),f^{j-a_{i}}(t_{\alpha_{i}}))\leq\epsilon/2$ for all
$j\in\\{a_{i},\dots,b_{i}\\},i\in\omega$.

Now suppose, by way of contradiction, $q\in\omega(p_{\gamma})$. This implies
that $f^{m}(q)\in\omega(p_{\gamma})$ as well since $\omega$-limit sets are
forward invariant. Let $\\{k_{n}\\}_{n\in\omega}$ be a sequence of natural
numbers such that $\lim_{n\to\infty}f^{k_{n}}(p_{\gamma})=f^{m}(q)$.

By the construction of $p_{\gamma}$, for every $k_{n}$,
$f^{k_{n}}(p_{\gamma})$ is either

  1. (1)

in a part of $Orb(p_{\gamma})$ that is controlled by specification and within
$\epsilon/8$ of $\xi$ (but this only occurs once at the start of the spec
pattern of $p_{\gamma}$),

  2. (2)

in a part of $Orb(p_{\gamma})$ that is controlled by specification and within
$\epsilon/8$ of the orbit of $s$,

  3. (3)

in a part of $Orb(p_{\gamma})$ that is controlled by specification and within
$\epsilon/8$ of the orbit of some $\gamma_{i,j}$, or

  4. (4)

not in a part of $Orb(p_{\gamma})$ that is controlled by specification, in
which case $f^{k_{n}}(p_{\gamma})$ is fewer than $M+1$ iterates away from
being within $\epsilon/8$ of some $\gamma_{i,j}$ or $s$.

For the sake conciseness, let $p_{n}=f^{k_{n}}(p_{\gamma})$. Observe that at
least one of conditions (1)-(4) must be met infinitely often, and since
condition (1) is only satisfied once, at least one of conditions (2)-(4) must
be met infinitely often.

Suppose condition (2) is satisfied by infinitely many $p_{n}$. Then
$f^{m}(q)=\lim_{n\to\infty}p_{n}$ would be within $\epsilon/4$ of $Orb(s)$.
But this cannot be since, by choice of $\epsilon$, we have
$d(\overline{Orb(s)},\overline{Orb(t_{\alpha_{0}})})>2\epsilon$, and
$f^{m}(q)$ is within $\epsilon/2$ of $t_{\alpha_{0}}$ by virtue of being in
$E_{\alpha}$.

Suppose condition (3) is satisfied by infinitely many $p_{n}$. Then for
infinitely many $n$, $p_{n}$ is within $\epsilon/8$ of some point
$\gamma^{\prime}_{i_{n},j_{n}}$ in the orbit of some $\gamma_{i_{n},j_{n}}$.
Let $L_{n}$ be minimal so that $L_{n}+k_{n}=b_{i}+1$ for some $i\in\omega$.
Then, $p_{n}$ is guaranteed to be within $\epsilon/8$ of the orbit of
$\gamma_{i_{n},j_{n}}$ for at least $L_{n}$ many iterates.

We have two possibilities. Either $\\{L_{n}\\}_{n\in\omega}$ is unbounded or
not. If it is unbounded, then we can pass to $\\{p_{n_{r}}\\}_{r\in\omega}$, a
subseqeunce of $p_{n}$ on which $L_{n_{r}}$ is monotone increasing. Then by
Lemma 9, we have that $d(p_{n_{r}},\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}})\to
0$. Moreover, since $p_{n_{r}}\to f^{m}(q)$, we have $d(p_{n_{r}},f^{m}(q))\to
0$.

So, by the triangle inequality

$d(f^{m}(q),\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}})\leq
d(f^{m}(q),p_{n_{r}})+d(p_{n_{r}},\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}})$

The latter two terms go to 0 as $n_{r}\to\infty$ and so
$d(f^{m}(q),\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}})\to 0$. This means that
$\lim_{r\to\infty}\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}}=f^{m}(q)$. But each
$\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}}\in H_{\gamma}$ since each
$\gamma^{\prime}_{i_{n_{r}},j_{n_{r}}}$ is in the orbit of some
$\gamma_{i,j}\in H_{\gamma}$ and $H_{\gamma}$ is forward invariant. Moreover,
$H_{\gamma}$ is closed, which implies that $f^{m}(q)\in H_{\gamma}$. But,
$E_{\alpha}\cap H_{\gamma}=\varnothing$ by Lemma 16, a contradiction.

If, on the other hand, $\\{L_{n}\\}_{n\in\omega}$, is bounded, then we can
pass to $\\{p_{n_{r}}\\}_{r\in\omega}$, a subseqeunce of $p_{n}$, on which
$L_{n_{r}}$ is constant. Let $L$ be this constant. Then $f^{M+L}(p_{n})$ is
within $\epsilon/8$ of $s$ for all $r$. Additionally, every time an iterate of
$p_{\gamma}$ in the specification pattern is within $\epsilon/8$ of $s$, it
will stay within $\epsilon/8$ of $Orb(s)$ for increasingly more iterates.
Hence as $n_{r}\to\infty$ increasingly more iterates of $f^{M+L}(p_{n_{r}})$
will stay within $\epsilon/8$ of $Orb(s)$. This fact, in conjunction with
Lemma 9, implies that the sequence $\\{p_{n_{r}}\\}_{r\in\omega}$ is actually
converging to $s$. But, the forward orbit of $f^{m}(q)$ will be within
$\epsilon/2$ of $\\{t_{0},t_{1}\\}$ infinitely often. So, since
$d(\overline{Orb(s)},\overline{Orb(t_{i})})>2\epsilon$ for $i\in\\{0,1\\}$, we
cannot have $\\{f^{M+L}(p_{n})\\}_{n\in\omega}$ converging to $s$ and to
$f^{M+L}(f^{m}(q))$.

Lastly, suppose that for infinitely many $p_{n}$ condition (4) is satisfied.
If infinitely many $p_{n}$ are not in a part of $Orb(p_{\gamma})$ controlled
by specification and are fewer than $M+1$ iterates away from being within
$\epsilon/8$ of $s$, then there is some $w\leq M+1$ such that infinitely many
$p_{n}$ are exactly $w$ iterates away from being within $\epsilon/8$ of $s$.
By construction of $p_{\gamma}$, as $n\to\infty$, increasingly more iterates
of $f^{w}(p_{n})$ will be within $\epsilon/8$ of $s$ which implies
$f^{w}(p_{n})\to s$ by Lemma 9. This would mean $f^{m}(q)=s$, a contradiction.

On the other hand, if infinitely many $p_{n}$ are fewer than $M+1$ iterates
away from being within $\epsilon/8$ of some $\gamma_{i_{n},j_{n}}$, then there
is some $w\leq M+1$ such that infinitely many $p_{n}$ are exactly $w$ iterates
away from being within $\epsilon/8$ of $\gamma_{i_{n},j_{n}}$. By construction
of the orbit of $p_{\gamma}$, this means that
$d(f^{j}(f^{w}(p_{n})),f^{j}(\gamma_{i_{n},j_{n}}))<\epsilon/8$ for $j\leq
b_{i_{n}}$.

Again, we have two possibilities. Either the collection
$\\{i_{n}\\}_{n\in\omega}$ is unbounded or bounded. If the collection is
unbounded, then we can pass to $\\{p_{n_{r}}\\}_{r\in\omega}$, a subseqeunce
of $\\{p_{n}\\}_{n\in\omega}$ on which $\\{i_{n_{r}}\\}_{r\in\omega}$ is
monotone increasing and $f^{w}(p_{n_{r}})$ is within $\epsilon/8$ of
$\gamma_{i_{n_{r}},j_{n_{r}}}$. Since $i_{n_{r}}$ is monotone increasing,
$f^{w}(p_{n_{r}})$ will be within $\epsilon/8$ of
$Orb(\gamma_{i_{n_{r}},j_{n_{r}}})$ for increasingly many iterates, which by
Lemma 9 means $d(f^{w}(p_{n_{r}}),\gamma_{i_{n_{r}},j_{n_{r}}})\to 0$. As
before, this means $d(f^{w}(f^{m}(q)),\gamma_{i_{n},j_{n}})\to 0$. Now, let
$g\in\omega$ be such that $f^{g}(f^{w}(f^{m}(q)))\in E_{\delta}$ for some
$\delta\in\omega(\beta)$. Such a $g$ exists by Lemma 14.

Since $f^{g}$ is continuous, we have
$\lim_{r\to\infty}f^{g}(\gamma_{i_{n_{r}},j_{n_{r}}})=f^{g}(f^{w}(f^{m}(q)))$.
But, $H_{\gamma}$ is forward invariant, so $f^{g}(\gamma_{i_{n},j_{n}})\in
H_{\gamma}$, and $H_{\gamma}$ is closed, so $f^{g}(f^{w}(f^{m}(q)))\in
H_{\gamma}$. But, $\delta\notin\overline{Orb(\gamma)}$ so $E_{\delta}\cap
H_{\gamma}=\varnothing$, a contradiction.

If, on the other hand $\\{i_{n}\\}_{n\in\omega}$ is bounded, we can pass to
$\\{p_{n_{r}}\\}_{r\in\omega}$, a subsequence of $\\{p_{n}\\}_{n\in\omega}$,
such that $\\{i_{n_{r}}\\}_{r\in\omega}$ is constant. Let $L$ be this
constant. Then $f^{M+b_{L}+w}(p_{n_{r}})$ is within $\epsilon/8$ of $s$. And
again, since we stay within $\epsilon/8$ of the orbit of $s$ for increasingly
many iterates each time we visit in the specification pattern of $p_{\gamma}$,
this means $f^{M+b_{L}+w}(p_{n_{r}})\to s$. But as before, since inifnitely
many iterates of $f^{m}(q)$ are within $\epsilon/2$ of $\\{t_{0},t_{1}\\}$ and
$d(\overline{Orb(s)},\overline{Orb(t_{\alpha_{i}})})>2\epsilon$ for
$i\in\\{0,1\\}$, we cannot have $f^{M+b_{L}+w}(p_{n_{r}})$ converging to $s$
and $f^{M+b_{L}+w}(f^{m}(q))$.

Thus, $f^{m}(q)\notin\omega(p_{\gamma})$, implying $q\notin\omega(p_{\gamma})$
which is the desired contradiction. ∎

We are now ready to prove our main theorem.

######  Main Theorem.

Let $(X,f)$ be an expansive dynamical system such that $X$ is Lindelöf, every
open set of $X$ is uncountable, $(X,f)$ has ISP, and there exist $t_{0}$,
$t_{1},s\in X$ such that
$\overline{Orb(t_{0})},\overline{Orb(t_{1})},\overline{Orb(s)}$ are each a
non-zero distance away from each other. Then $(X,f)$ exhibits dense
$\omega$-chaos.

###### Proof.

Fix $\xi$ in $X$ and $U$ a neighborhood thereof. Let $W\subset 2^{\omega}$ be
an uncountable set of non-periodic points with pairwise disjoint orbits and
uncountable $\omega$-limit sets. Such a set does exist [9]. Let
$\beta,\gamma\in W$. Find $p_{\beta}$ and $p_{\gamma}$ in $X$ following the
method outlined above. By Lemma 13, since $\beta,\gamma$ are not periodic,
$E_{\beta}$ and $E_{\gamma}$ do not contain periodic points implying that
$H_{\beta}$ and $H_{\gamma}$ do not contain _only_ periodic points. Moreover,
$\omega(p_{\beta})\cap\omega(p_{\gamma})\ni s$, and
$\omega(p_{\beta})\setminus\omega(p_{\gamma})\supset H_{\beta}^{\prime}$ by
Lemma 18 which is uncountable as shown in Lemma 15. Similarly,
$\omega(p_{\gamma})\setminus\omega(p_{\beta})\supset H_{\gamma}^{\prime}$
which is also uncountable. There are uncountably many such $\beta$ and
$\gamma$ in $W$. From this, it follows that $(X,f)$ exhibits $\omega$-chaos.

Moreover, $p_{\beta},p_{\gamma}\in B_{\epsilon/8}(\xi)\subset U$. Thus $(X,f)$
exhibits dense $\omega$-chaos. ∎

Note that if $X$ contains some open sets which are not uncountable, then $X$
will still exhibit $\omega$-chaos, though not densely.

## References

  * [1] Rufus Bowen.  Periodic points and measures for Axiom $A$ diffeomorphisms.  Trans. Amer. Math. Soc., 154:377–397, 1971.
  * [2] Robert L. Devaney.  An introduction to chaotic dynamical systems.  Addison-Wesley Studies in Nonlinearity. Addison-Wesley Publishing Company, Advanced Book Program, Redwood City, CA, second edition, 1989.
  * [3] Reeve Hunter and Brian E. Raines.  Omega chaos and the specification property.  J. Math. Anal. Appl., 448(2):908–913, 2017.
  * [4] M. Lampart and P. Oprocha.  Shift spaces, $\omega$-chaos and specification property.  Topology Appl., 156(18):2979–2985, 2009.
  * [5] Shi Hai Li.  $\omega$-chaos and topological entropy.  Trans. Amer. Math. Soc., 339(1):243–249, 1993.
  * [6] T. Y. Li and James A. Yorke.  Period three implies chaos.  Amer. Math. Monthly, 82(10):985–992, 1975.
  * [7] Jonathan Meddaugh and Brian E. Raines.  Weak specification and Baire space.  J. Math. Anal. Appl., 479(1):1355–1363, 2019.
  * [8] Jonathan Meddaugh and Brian E. Raines.  A characterization of $\omega$-limit sets in subshifts of Baire space.  J. Math. Anal. Appl., 500(1):Paper No. 125097, 11, 2021.
  * [9] Marston Morse and Gustav A. Hedlund.  Symbolic dynamics II. Sturmian trajectories.  Amer. J. Math., 62:1–42, 1940.
  * [10] James R. Munkres.  Topology.  Prentice Hall, Inc., Upper Saddle River, NJ, 2000.  Second edition of [ MR0464128].
  * [11] Piotr Oprocha.  Specification properties and dense distributional chaos.  Discrete Contin. Dyn. Syst., 17(4):821–833, 2007.
  * [12] Piotr Oprocha and Marta Štefánková.  Specification property and distributional chaos almost everywhere.  Proc. Amer. Math. Soc., 136(11):3931–3940, 2008.
  * [13] B. Schweizer and J. Smítal.  Measures of chaos and a spectral decomposition of dynamical systems on the interval.  Trans. Amer. Math. Soc., 344(2):737–754, 1994.
  * [14] Karl Sigmund.  On dynamical systems with the specification property.  Trans. Amer. Math. Soc., 190:285–299, 1974.
  * [15] A. Sklar and J. Smítal.  Distributional chaos on compact metric spaces via specification properties.  J. Math. Anal. Appl., 241(2):181–188, 2000.
  * [16] Hui Wang and Lidong Wang.  The weak specification property and distributional chaos.  Nonlinear Anal., 91:46–50, 2013.


The critical exponents are derived by inserting the scaling of the Green functions eq:Green.function.scaling into the expression for the observables appendixC:scaling.of.P–appendixC:scaling.of.R. The critical exponent $\delta$ can be easily obtained from the $G^{(0,1)}_R$ scaling. The scaling form of $\langle N  \rangle (t,\tau)$ is obtained by changing the integration variable in the last expression of appendixC:scaling.of.N to $\bi{r}/ \xi$ and relating the length scale $\xi$ to a time scale, i.e., $\xi \propto t^{1/ z}$. At last, the critical exponent $\nu_{\parallel}$ is obtained by inserting the correlation length $\xi \propto (\tau)^{\nu} $ in the second argument of the scaling function of the Green function eq:Green.function.scaling and using that $\tau$ encodes how far we are from criticality, i.e.,  $\tau \propto (\sigma-\sigma_c) $.
The final values for the critical exponents read:

\begin{eqnarray}
z-2=W_D-\frac{\sigma+\tilde{\sigma}}{2}=-\frac{7}{30}\epsilon \\
\delta=\frac{d-2+\tilde{\sigma}}{2z}= 1-\frac{1}{4}\epsilon \\
\theta=-\frac{\sigma+\tilde{\sigma}}{2z}=\frac{3}{20}\epsilon \qquad \qquad \\ \nu_{\parallel}=z \nu=1-\frac{1}{20}\epsilon
\end{eqnarray}

§ CHOICE OF THE TIME INTERVALS

This section provides additional details regarding the choice of the time interval ${\Delta t = 10}$ in the averaged local slope equation eq:local.slope.theta [73, 70].
The averaged local slope technique explained in Sec. <ref> of the main text, aims to approximate the derivative in every point via the notion of a difference quotient.
However, the underlying data is inherently noisy as it results from averaging over a finite number of stochastic realizations.
Therefore, one has to strike a balance between a more accurate approximation of a derivative (i.e., smallest possible $\Delta t$) and not disproportionately considering noise-induced slope changes (i.e., large $\Delta t$).
By testing various values of $\Delta t$, we find that the choice of ${\Delta t=10}$ is a sensible middle ground.
This value is comparable to values used in previous studies [70, 18, 34, 77, 40].
Most importantly, our estimates of the critical exponents are stable against varying the value of $\Delta t$ by up to $40 \%$.
While previous work [41] on the DGEP model chose a substantially smaller $\Delta t$, they additionally employ a moving average to smooth out stochastic fluctuations.
This is similar to choosing a larger time interval in the sense that both methods mitigate what are assumed to be noise-induced slope changes. Increasing the time interval achieves this by a coarser difference quotient, while the moving average per definition results in fewer deviations between neighboring points.
Furthermore, a direct comparison of $\Delta t$ between the Gillespie algorithm [53] used in this paper with a clear notion of time and one where time is defined through the number of updating steps [41] should be done with caution.
One additional constraint for the time interval is that it has to be compatible with the recording interval of the simulation algorithm; otherwise, the local slope cannot be calculated. Our recording interval is chosen to be $1$ in terms of our base units.
Hence, $\Delta t$ has to be chosen as an integer in our case.

§ ERROR ESTIMATION

This section summarizes our systematic approaches to estimating the errors of the critical exponents obtained from the numerical data of stochastic simulations.
We present several complementary approaches for error estimation and use the one that gives the largest value.

§.§ Error estimation for the averaged local slope method

Error estimation methods. List of methods used for the error estimation for critical exponents. The parameters used for each of the methods are listed as well. In particular, for the sampling methods, we report the fitting interval and the size of the random samples, while for the fitting range method, the range of fitting intervals is listed.

Model     Exponent     Error estimation method     Parameters

GEP     $\theta, \delta, z$     Non-averaged min-max     1/$t_{\mathrm{min}}=2 \times 10^{-5}$

DGEP     $\theta$     Sampling     1/$t_{\mathrm{min}}=2 \times 10^{-5}$, 5% of the data per repetition

$\delta$    Sampling    1/$t_{\mathrm{min}}=2 \times 10^{-5}$, 5% of the data per repetition

$z$     Sampling     1/$t_{\mathrm{min}}=2 \times 10^{-5}$, 5% of the data per repetition

CGEP     $\theta$     Sampling    1/$t_{\mathrm{min}}=2 \times 10^{-5}$, 5% of the data per repetition

$\delta$    Sampling     1/$t_{\mathrm{min}}=2 \times 10^{-5}$, 5% of the data per repetition

$z$     Fitting Range    1/ $t_{\mathrm{min}} \in [5\times 10^{-6}, 4\times 10^{-5}] $

In summary, the estimation of critical exponents and their errors from numerical data using the averaged local slope method is susceptible to the following partially subjective factors: The choice of data points used to fit to condition eq:asymptotic.local.slope and the method used to estimate the impact of deviations from the resulting linear fit.
Here, we can confidently present results and assert strong statements about universality classes by choosing relatively large errors derived from a range of error estimation methods.

Error estimations.
Visualization of three different methods of error estimations on the example of the critical exponent $\theta$.
The filled dots represent artificial numerical data of the local slope $\theta_{\rm data}(t)$, the gray straight line the corresponding fit $\theta_{\rm fit}(t)$ and $\Delta$ the resulting error estimates.
(a) Fitting range error obtained by varying the fitting range. For the next smaller fitting range, only the data points colored in light blue and dark blue are taken into account, and for the smallest fitting range, only the data points colored in dark blue are taken into account. The respective linear extrapolations are shown as light blue and dark blue straight lines.
(b) Example of the extrapolated min-max error. The data point with the largest deviation $\delta$ from $\theta_{\rm fit}(t)$ is shown in dark blue. The new linear extrapolations $\tilde\theta_{\rm fit}(t)$ obtained by shifting $\theta_{\rm fit}(t_{\rm max})$ are shown in light blue.
(c) Example of the sampling error obtained from selecting a random sample (shown in dark blue) and its corresponding linear fit is shown in light blue.

As we discussed in the main text, a common method to extract critical exponents from numerical data—here described in the example of $\theta$—is to first obtain its time-dependent approximation $\theta(t)$.
This can be achieved by calculating the time-dependent  averaged local slope of the corresponding observable—$\langle N\rangle (t)$ in the case of $\theta$—which amounts to locally approximating the slope of $\langle N\rangle (t)$ in a double logarithmic plot by a discretized derivative (see eq:local.slope.theta).
At criticality, $\theta(t)$ is expected to asymptotically approach the actual value of the critical exponent $\theta$ as [70]

\begin{eqnarray}
\lim_{t\rightarrow\infty}\theta(t)=\theta+\frac{a}{t}, \label{eq:asymptotic.local.slope}
\end{eqnarray}

with some value $a$ for the local correction to the actual critical exponent.
Hence, the critical exponent $\theta$ can be obtained by plotting $\theta(1/t)$, estimating $a$ by approximating the slope of $\theta(t)$ in the limit $1/t\rightarrow 0$ and calculating the intercept of the ensuing straight line with the $1/t=0$ axis.
For this procedure to work, one additionally has to find the critical value of the tuning parameter (in our case, the critical infection rate $\sigma_c$), which is obtained by requiring that the condition eq:asymptotic.local.slope holds, i.e, by requiring that $\theta(1/t)$ approaches a straight line close to $1/t=0$.
In summary, we take the following steps to estimate critical exponents from numerical data: Estimate the critical infection rate as described above and determine the best fit of the corresponding dataset of local slopes $\theta_{\rm data}(t)$ (shown as bullets in fig:figureE1) to the expected asymptotic behavior $\theta_{\rm fit}(t) = \theta_{\rm fit} + a_{\rm fit}/t$ (shown as a gray straight line in fig:figureE1).
Importantly, only data points inside a certain fitting range $I=[1/t_{\rm max}, 1/t_{\rm min}]$—where $t_{\rm min}$ is approximately the smallest time where we see the asymptotic behavior eq:asymptotic.local.slope and $t_{\max}$ is given by the maximal simulation time—are taken into account for the above fit.
The resulting $\theta_{\rm fit}$, which is equal to the intercept of $\theta_{\rm fit}(t)$ with the $1/t=0$ axis, is then our best estimate for the critical exponent. The same procedure is subsequently applied to the remaining critical exponents.
In the following paragraphs, all mentioned linear fits and straight lines refer to the estimated value of $a_{\rm fit}$ and the resulting straight lines in the vicinity of $1/t=0$.

Even though this method is widely used, there is no single gold standard for estimating the errors of critical exponents obtained via the above procedure [23, 24].
Nevertheless, two guidelines for the error estimation of critical exponents exist:
Firstly, slopes of straight lines should not be estimated via fitting methods like $\chi^2$–error estimates [23].
Secondly, the error of critical exponent estimates is usually one order of magnitude larger than the error estimate of the critical value of the tuning parameter [23].
In particular, there is no established approach to determine the interval over which to fit the data—thereby obtaining the extrapolated linear fit, its intercept with the $1/t=0$ axis and thus the estimated exponent.
This choice, however, can have a significant influence on the results.
Due to stochastic fluctuations in the data and corrections to the described asymptotic behavior at finite times, the estimated value of a critical exponents is susceptible to this choice of the fitting range $I$.
Given these challenges, we explored three distinct methods to assess the exponents and their associated errors.
Opting for the approach that produces the widest range of possible values for the exponents, we derive a highly conservative estimate of errors.

We are interested in the critical dynamics on longer time scales, where asymptotic scaling behavior is to be expected.
The procedure detailed above, used to extract this asymptotic behavior from numerical data, however, depends on the choice of $t_{\rm min}$.
This choice corresponds to the point where the system is close to fulfilling eq:asymptotic.local.slope and is, at least partially, subjective.
A possible error estimation, therefore, involves varying $t_{\rm min}$ within a reasonable range and, in accordance with the asymptotic behavior eq:asymptotic.local.slope, fitting a straight line to the corresponding data points, obtained by calculating the averaged local slope as described in eq:local.slope.theta.
The error is then estimated as the difference between the largest and smallest resulting intercept (see fig:figureE1(a)).
We term this the fitting range error.

Another error estimation can be obtained by a method we term the extrapolated min-max error, explained on the example of the critical exponent $\theta$. To calculate this error, we proceed as follows: We first pick a sensible value for $t_{\rm min}$, resulting in the fitting range ${I=[1/t_{\rm max}, 1/t_{\rm min}]}$.
Subsequently, the local correction $a_{\rm fit}$ and the critical exponent $\theta_{\rm fit}$ are again estimated by fitting the averaged local slope to condition eq:asymptotic.local.slope.
We then calculated the maximal deviation from the resulting straight line, i.e.,

\begin{eqnarray}
\delta = \max_{1/t\in I}|\theta_{\rm fit}(t) - \theta_{\rm data}(t)| = \max_{1/t\in I} |\theta_{\rm fit}(t_{\rm min}) + \frac{a}{t} - \theta_{\rm data}(t)|.
\end{eqnarray}

Subsequently, a new straight line $\tilde\theta_{\rm fit}(t)$ is obtained by adding (or subtracting) $\delta$ to $\theta_{\rm fit}(t_{\rm max})$ and connecting this to $\theta_{\rm fit}(t_{\rm min})$.
Note that we add $\delta$ to $\theta_{\rm fit}(t_{\rm max})$ instead of directly using the data point of the largest deviation to draw a new straight line to avoid error overestimations in cases where the largest deviation is observed close to $1/t_{\rm min}$.
The extrapolated min-max error is then the distance between the original estimate $\theta_{\rm}$ and the intercept of $\tilde\theta_{\rm fit}(t)$ with the $1/t=0$ axis; it is thus given by

\begin{eqnarray}
\Delta_{\rm min-max} = \frac{\delta}{1-{t_{\rm min}}/{t_{\rm max}}}.
\end{eqnarray}

A visualization of the above procedure is shown in fig:figureE1(b).

Yet another approach is to test how stable the fit obtained through condition eq:asymptotic.local.slope is against removing data points.
To achieve this, we randomly sampled between 1–5$\%$ of the data points within a given fitting interval and fitted a straight line to each sample.
This process was repeated $5,000$ times. The ensuing sampling error was then determined as the maximum difference between the intercept of the full fit and the intercepts of the sampled fits (see fig:figureE1(c)).

We calculated all three types of errors for each exponent and chose the largest one for the final estimate. This approach led to conservative error estimates in comparison to other studies [70, 18, 34, 77, 40].
Table <ref> provides a summary of the chosen error estimates and fitting ranges for the respective exponents across all studied epidemic models.
For the GEP model, however, the situation is more involved.
Despite investing significant computational resources, two possible candidates for the critical infection rate—${\sigma_c=1.376}$ and ${\sigma_c=1.377}$—remained.
Resolving this ambiguity would require longer simulation times, which are currently impractical with the employed algorithm.
Lacking such studies, we calculated a point-wise mean; for example, ${\bar\theta(t)=(\theta_-(t) + \theta_+(t))/2}$, where $\theta_\pm$ correspond to data obtained from simulations with ${\sigma=1.376}$ and ${\sigma=1.377}$, respectively.
The resulting averaged data set was subsequently used to obtain an estimate for the critical exponents.
This approach, however, results in an underestimation of the error when applying all of the above error estimation methods to the averaged data.
Thus, we did not use the averaged data ($\bar\theta(t)$) but rather calculated the extrapolated min-max error for both candidate data sets ($\theta_-(t)$ and $\theta_+(t)$) and chose the largest deviation from the critical exponents obtained from the averaged data set as the non-averaged min-max error.
The non-averaged min-max error is a more conservative approach to estimating the error and, therefore a reasonable substitute.

§.§ Error estimation for data collapses

Data collapse error estimation for DGEP. Data collapses for the average number of infected individuals for the DGEP model at different values of $\nu_{\parallel}$ close to our final estimate of $\nu_{\parallel}=1.4$, as indicated by the graph titles. Apart from $\nu_\parallel$, the parameters and data used in these figures are identical. The individual curves are rescaled as explained in the main text in section <ref>. They are obtained from simulations for different values of the infection rate $\sigma$ as indicated below the figures; the critical value $\sigma_c=0.93$ and the exponent $\theta=0.54$ were chosen for the rescaling.

Data collapse error estimation for CGEP. Data collapses for the average number of infected individuals for the CGEP model at different values of $\nu_{\parallel}$ close to our final estimate of $\nu_{\parallel}=1.4$, as indicated by the graph titles. Apart from $\nu_\parallel$, the parameters and data used in these figures are identical. The individual curves are rescaled as explained in the main text in section <ref>. They are obtained from simulations for different values of the infection rate $\sigma$ as indicated below the figures; the critical value $\sigma_c=0.43537$ and the exponent $\theta=0.55$ were chosen for the rescaling.

There are two equally valid approaches to estimating the error of critical exponents obtained through a data collapse.
One way is to define a loss function for the quality of the fit and then let an optimization algorithm determine the best collapse and the critical exponents with their errors [96, 97, 98, 99, 100]. Alternatively, one can employ a manual approach and determine the best data collapse visual inspection [23, 53].

However, employing a loss-function approach introduces some inherent bias.
First of all, initial values of the critical exponents for the algorithm are required, and the results change depending on this choice.
The choice of loss function also plays a significant role in the final estimates.
Hence, we decided to adhere to a manual approach to mitigate these potential biases.
The best fit is determined by visual inspection, and the error margins are then estimated from the breakdown of a reasonable data collapse [23, 53].
In particular, we determine the value of $\nu_\parallel$ for which the appropriate rescaling (which can be determined from eq:observable1) leads to only two distinct curves: one for all data obtained from simulations with infection rates larger than the critical infection rate $\sigma_c$ and one for data obtained for infection rates smaller than $\sigma_c$.
Figures <ref> and <ref> show the data collapses for different values of $\nu_\parallel$ of the DGEP and CGEP data, respectively.
In both cases, we determine that the best collapse is achieved for ${\nu_\parallel=1.4}$.
To estimate the error, we further determine for which values the collapse is still “reasonably good” and for which this is no longer the case. It should, however, be noted that this choice is at least partially subjective.
In figure <ref>, one can nevertheless clearly observe that the curves for larger infection rates—${\sigma=0.935,\ 0.940}$ and $0.945$—
diverge from each other for ${\nu_\parallel=1.2}$ and ${\nu_\parallel=1.6}$.
Similarly, in figure <ref>, both the curves for larger and smaller infection rates diverge from each other for ${\nu_\parallel=1.4}$ and the curves for ${\sigma=0.432}$–$0.435$ also clearly separate for ${\nu_\parallel=1.6}$.
We thus conclude that in both cases, the value of $\nu_\parallel$ is given by ${\nu_\parallel=1.4\pm 0.1}$.

§ ADDITIONAL NUMERICAL DATA

This section presents additional data we omitted in the main text, as well as the exact definitions of the observables from a numerical point of view.
We define the survival probability as the number of realizations with at least one infected individual divided by the total number of realizations. The average number of infected individuals is calculated as an average over all realizations, including those where no infected individuals remain. The mean squared cluster radius is obtained by squaring the difference between the right- and leftmost infected individual in the x- or y-direction, whichever is larger. The average is taken over all realizations with at least one infected individual.
Figure <ref> shows the survival probability and the mean-squared cluster radius of the DGEP models. It is evident from the indicated resulting critical exponents that we cannot distinguish them from the GEP ones with statistical significance. The same is true for figure <ref>, where the corresponding observables are shown for the CGEP model. In table <ref>, we summarize the simulation parameters used for each model under study.

Simulation parameters. Parameters used for the simulations of GEP, DGEP and CGEP. We list the number of realizations of the stochastic process taken into account for computing the observables' average; the length of the 2D square-shaped domain measured in units of the lattice spacing; the maximal time measured in units of the diffusion constants (see discussion in section <ref>); the number of infected individuals in the initial conditions $N_0$; the diffusion constants of all species; the reaction rates $\zeta$ for recovery and $\gamma$. In addition, we provide the figures to which the respective parameters apply for easy reference.

Model     Realizations      Length     Time     $N_0$     $D_A$     $D_B$     $D_C$     $\zeta$     $\gamma$     Figures

GEP     24,000     10,000     50,000     10     0     0.1     0     0.1     0     <ref>

DGEP     20,000     2,600     10,000     10     0.1     0.1     0     0.1     0     <ref>(a), <ref>(a), <ref>, <ref>

critical     38,000     8,800     40,000     10     0     0.1     0.1     0.05     0.1     <ref>(b), <ref>(b), <ref>, <ref>

off-critical     34,000     4,400     20,000     10     0     0.1     0.1     0.05     0.1     <ref>(b), <ref>(b), <ref>, <ref>

Critical behavior of the DGEP model's other observables.

(a) Averaged local slope for the survival probability of the infected individuals for the DGEP model, for a set of values for the infection rate $\sigma$ indicated below the graph.
Based on these results and the critical value from figure <ref>(a), we estimate the value of ${\delta=0.091 \pm 0.009}$ as indicated by the circular marker with error bars. The triangle shows our estimate for GEP for comparison, where we omitted the error bars because they are smaller than the marker.
(b) Averaged local slope for the mean-squared radius of the infected cluster for the DGEP model for a set of values for the infection rate $\sigma$ indicated below the graph and identical to the values of $\sigma$ used for (a).
Based on these results and the critical value from figure <ref>(a), we estimate the value of ${z=1.75 \pm 0.02}$ as indicated by the circular marker with error bars. The triangle shows our estimate for GEP for comparison, where we omitted the error bars because they are smaller than the marker.
The parameters for both figures are identical to those used for figure <ref>(a) and are summarized in table <ref>.

Critical behavior of the CGEP model's other observables.

(a) Averaged local slope for the survival probability of the infected individuals for the CGEP model, for a set of values for the infection rate $\sigma$ indicated below the graph.
Based on these results and the critical value from figure <ref>(b), we estimate the value of ${\delta=0.093 \pm 0.003}$ as indicated by the circular marker with error bars. The triangle shows our estimate for GEP for comparison, where we omitted the error bars because they are smaller than the marker.
(b) Averaged local slope for the mean-squared radius of the infected cluster for the CGEP model for a set of values for the infection rate $\sigma$ indicated below the graph and identical to the values of $\sigma$ used for (a).
Based on these results and the critical value from figure <ref>(a), we estimate the value of ${z=1.772 \pm 0.006}$ as indicated by the circular marker with error bars. The triangle shows our estimate for GEP for comparison, where we omitted the error bars because they are smaller than the marker.
The parameters for both figures are identical to those used for figure <ref>(b) and are summarized in table <ref>.

§ REFERENCES

[1]
Chinazzi M, Davis J T, Ajelli M, Gioannini C, Litvinova M, Merler S, y Piontti A P, Mu K, Rossi L, Sun K, Viboud C, Xiong X, Yu H, Halloran M E, Longini I M and Vespignani A 2020 Science 368 395–400 ISSN 0036-8075

[2]
Kissler S M, Tedijanto C, Goldstein E, Grad Y H and Lipsitch M 2020 Science 368 860–868 ISSN 0036-8075

[3]
Arenas A, Cota W, Gómez-Gardeñes J, Gómez S, Granell C, Matamalas J T, Soriano-Paños D and Steinegger B 2020 Physical Review X 10 041055

[4]
Bauer S, Contreras S, Dehning J, Linden M, Iftekhar E, Mohr S B, Olivera-Nappa A and Priesemann V 2021 PLoS Computational Biology 17 e1009288 ISSN 1553-734X

[5]
Dehning J, Zierenberg J, Spitzner F P, Wibral M, Neto J P, Wilczek M and Priesemann V 2020 Science 369 eabb9789 ISSN 0036-8075

[6]
Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, Parker M, Bonsall D and Fraser C 2020 Science 368 eabb6936 ISSN 0036-8075

[7]
Gros C, Valenti R, Schneider L, Valenti K and Gros D 2021 Scientific Reports 11 6848

[8]
Muley D, Shahin M, Dias C and Abdullah M 2020 Sustainability 12 7367

[9]
Epstein J M, Parker J, Cummings D and Hammond R S 2008 PLoS ONE 3 e3955

[10]
Weitz J S, Park S W, Eksin C and Dushoff J 2020 Proceedings of the National Academy of Sciences 117 32764–32771 ISSN 0027-8424

[11]
Lloyd A L and May R M 1996 Journal of Theoretical Biology 179 1–11 ISSN 0022-5193

[12]
Pastor-Satorras R, Castellano C, Mieghem P V and Vespignani A 2015 Reviews of Modern Physics 87 925–979 ISSN 0034-6861

[13]
Holley R A and Liggett T M 1975 The Annals of Probability 3 ISSN 0091-1798

[14]
Castellano C, Fortunato S and Loreto V 2009 Reviews of Modern Physics 81 591–646 ISSN 0034-6861

[15]
Drossel B and Schwabl F 1992 Physical Review Letters 69 1629–1632 ISSN 0031-9007

[16]
Albano E V 1995 Physica A 216 213–226 ISSN 0378-4371

[17]
Ziff R M, Gulari E and Barshad Y 1986 Physical Review Letters 56 2553–2556 ISSN 0031-9007

[18]
Jensen I, Fogedby H C and Dickman R 1990 Physical Review A 41 3411–3414 ISSN 1050-2947

[19]
Marro J and Dickman R 1999 Nonequilibrium Phase Transitions in Lattice Models (Cambridge University Press) ISBN 9780511524288

[20]
Ódor G 2004 Reviews of Modern Physics 76 663–724 ISSN 0034-6861

[21]
Tauber U C 2014 Critical Dynamics (Cambridge, United Kingdom: Press, Cambridge University)

[22]
Täuber U C, Howard M and Vollmayr-Lee B P 2005 Journal of Physics A 38 R79 ISSN 0305-4470

[23]
Henkel M, Hinrichsen H and Lübeck S 2008 Absorbing Phase Transitions vol 1 Non-Equilibrium Phase Transitions (Heidelberg, Germany: Springer Netherlands)

[24]
Hinrichsen H 2000 Advances in Physics 49 815–958 ISSN 0001-8732

[25]
Janssen H K 1981 Zeitschrift für Physik B 42 151

[26]
Janssen H K and Tauber U C 2005 Annals of Physics 315 147–192 ISSN 0003-4916

[27]
Grassberger P 1983 Mathematical Biosciences 63 157–172 ISSN 0025-5564

[28]
Janssen H K 1985 Zeitschrift für Physik B 58 311–317 ISSN 0722-3277

[29]
Harris T E 1974 The Annals of Probability 2 ISSN 0091-1798

[30]
Kermack W O and McKendrick A G 1927 Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 115 700–721 ISSN 0950-1207

[31]
Grassberger P, Chaté H and Rousseau G 1997 Physical Review E 55 2488–2495 ISSN 1539-3755

[32]
Janssen H K and Stenull O 2008 Physical Review E 78 061117 ISSN 1539-3755

[33]
van der Kolk J, Raßhofer F, Swiderski R, Haldar A, Basu A and Frey E 2023 Physical Review Letters 131 ISSN 0031-9007

[34]
Jensen I 1993 Physical Review Letters 70 1465–1468 ISSN 0031-9007

[35]
Muñoz M A, Grinstein G and Dickman R 1998 Journal of Statistical Physics 91 541–569 ISSN 0022-4715

[36]
da Silva J K L and Dickman R 1999 Physical Review E 60 5126–5129 ISSN 1539-3755

[37]
Grassberger P 1982 Zeitschrift für Physik B 47 365–374 ISSN 0722-3277

[38]
Howard M J and Täuber U C 1997 Journal of Physics A 30 7721 ISSN 0305-4470

[39]
Henkel M and Hinrichsen H 2004 Journal of Physics A 37 R117 ISSN 0305-4470

[40]
Deng S, Li W and Täuber U C 2020 Physical Review E 102 042126 ISSN 2470-0045

[41]
Deng S and Ódor G 2023 Physical Review E 107 014303 ISSN 2470-0045

[42]
Jensen I and Dickman R 1999 Journal of Physics A 26 L151 ISSN 0305-4470

[43]
van Wijland F, Oerding K and Hilhorst H J 1998 Physica A 251 179–201

[44]
Doussal P L and Wiese K J 2015 Physical Review Letters 114 110601 ISSN 0031-9007

[45]
Janssen H K and Stenull O 2016 Physical Review E 94 042138 ISSN 2470-0045

[46]
Kree R, Schaub B and Schmittmann B 1989 Physical Review A  2214–2221 ISSN 1050-2947

[47]
Janssen H K 2001 Physical Review E 64 058101 ISSN 1539-3755

[48]
Tarpin M, Benitez F, Canet L and Wschebor N 2017 Physical Review E 96 022137 ISSN 2470-0045

[49]
Oerding K, Wijland F, Leroy J P and Hilhorst H J 2000 Journal of Statistical Physics 99 1365–1395 ISSN 0022-4715

[50]
Maia D S and Dickman R 2007 Journal of Physics: Condensed Matter 19 065143 ISSN 0953-8984

[51]
Dickman R and Maia D S 2008 Journal of Physics A: Mathematical and Theoretical 41 405002 ISSN 1751-8121

[52]
Argolo C, Tenório V and Lyra M L 2019 Physica A: Statistical Mechanics and its Applications 517 422–430 ISSN 0378-4371

[53]
Polovnikov B, Wilke P and Frey E 2022 Physical Review Letters 128 078302 ISSN 0031-9007

[54]
Kermack W O and McKendrick A G 1927 Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 115 700–721 ISSN 0950-1207

[55]
Cardy J L 1983 Journal of Physics A 16 L709 ISSN 0305-4470

[56]
Cardy J L and Grassberger P 1985 Journal of Physics A 18 L267 ISSN 0305-4470

[57]
Tomé T and Ziff R M 2010 Physical Review E 82 051921 ISSN 1539-3755

[58]
Frey E and Brauns F 2018 Self-organization of Protein Patterns Active Matter and Nonequilibrium Statistical Physics (Lecture Notes of the Les Houches Summer School vol 112) (Oxford, United Kingdom: Oxford University Press) ISBN 9780192858313

[59]
COMSOL M 1998 COMSOL Multiphysics, Burlington, MA, accessed Feb 9 2018

[60]
Weber M F and Frey E 2017 Reports on Progress in Physics 80 046601 ISSN 0034-4885

[61]
Wilson K G 1975 Reviews of Modern Physics 47 773–840 ISSN 0034-6861

[62]
Wilson K G and Fisher M E 1972 Physical Review Letters 28 240–243 ISSN 0031-9007

[63]
Fisher M E 1974 Reviews of Modern Physics 46 597–616 ISSN 0034-6861

[64]
Frey E and Täuber U C 1994 Physical Reviews E 50(2) 1024–1045

[65]
Weinberg S 1995 The Quantum Theory of Fields (Cambridge, United Kingdom: Cambridge University Press)

[66]
da Costa N V, Fulco U L, Lyra M L and Gléria I M 2007 Physical Review E 75 031112 ISSN 1539-3755

[67]
Muñoz M A, Dickman R, Vespignani A and Zapperi S 1999 Physical Review E 59 6175–6179 ISSN 1539-3755

[68]
Pastor-Satorras R and Vespignani A 2000 Physical Review E 62 R5875–R5878 ISSN 1539-3755

[69]
Janssen H K 1997 Physical Review E 55 6253–6256 ISSN 1539-3755

[70]
Grassberger P 1989 Journal of Physics A 22 3673–3679

[71]
Argolo C, Quintino Y, Gleria I and Lyra M L 2011 Physica A 390 1433–1439 ISSN 0378-4371

[72]
de Souza D R, Tomé T and Ziff R M 2011 Journal of Statistical Mechanics: Theory and Experiment 2011 P03006

[73]
Grassberger P and de la Torre A 1979 Annals of Physics 122 373–396

[74]
Gillespie D T 1976 Journal of Computational Physics 22 403–434

[75]
Gillespie D T 2007 Annual Review of Physical Chemistry 58 35–55

[76]
Gibson M A and Bruck J 2000 The Journal of Physical Chemistry A 104 1876–1889

[77]
Voigt C A and Ziff R M 1997 Physical Review E 56 R6241–R6244

[78]
et al G M G 2021 Nature Communications 12

[79]
Schwarzl M, Godec A and Metzler R 2017 Scientific Reports 7 ISSN 2045-2322

[80]
Reichenbach T, Mobilia M and Frey E 2007 Nature 448 1046–1049 ISSN 1476-4687

[81]
Knebel J, Geiger P M and Frey E 2020 Physical Review Letters 125 ISSN 1079-7114

[82]
Mukhamadiarov R I, Deng S, Serrao S R, Priyanka, Nandi R, Yao L H and Täuber U C 2021 Scientific Reports 11 ISSN 2045-2322

[83]
Cao X, Doussal P L and Rosso A 2022 Physical Review Letters 129 ISSN 1079-7114

[84]
Grassberger P 2013 Journal of Statistical Physics 153 289–311 ISSN 1572-9613

[85]
Homrighausen I, Winkler A and Frey E 2013 Physical Review E 88 012111 ISSN 1539-3755

[86]
Shapoval D, Blavatska V and Dudka M 2022 Journal of Physics A: Mathematical and Theoretical 55 455002 ISSN 1751-8121

[87]
Hallatschek O and Fisher D S 2014 Proceedings of the National Academy of Sciences 111 ISSN 1091-6490

[88]
Villiger N and Paulose J 2023 G3 Genes|Genomes|Genetics ISSN 2160-1836

[89]
Martin P C, Siggia E and Rose H A 1973 Phys. Rev. A 8(1) 423–437

[90]
Janssen H K 1976 Zeitschrift für Physik B Condensed Matter and Quanta 23 377–380 ISSN 1434-6036

[91]
de Dominicis C 1976 Techniques de renormalisation de la théorie des champs et dynamique des phénomenes critiques J. Phys., Colloq vol 37 p 247

[92]
Bausch R, Janssen H K and Wagner H 1976 Zeitschrift für Physik B Condensed Matter 24 113–127 ISSN 0722-3277

[93]
Benitez F, Duclut C, Chaté H, Delamotte B, Dornic I and Muñoz M A 2016 Physical Review Letters 117 100601 ISSN 0031-9007

[94]
Zinn-Justin J 2002 Quantum Field Theory and Critical Phenomena (Oxford, United Kingdom: Oxford University Press) ISBN 9780198509233

[95]
Janssen H K 2005 Journal of Physics: Condensed Matter 17 S1973 ISSN 0953-8984

[96]
Sorge A 2015 Pyfssa 0.7.6

[97]
Melchert O 2009 autoscale.py - a program for automatic finite-size scaling analyses: A user's guide

[98]
Houdayer J and Hartmann A 2004 Physical Review B 70 014418

[99]
Nelder J A and Mead R 1965 The Computer Journal 7 308–313

[100]
Bhattacharjee S M and Seno F 2001 Journal of Physics A 34 6375–6380


# Ensemble Fine-tuned mBERT for Translation Quality Estimation

Shaika Chowdhury
University of Illinois at Chicago, US
<EMAIL_ADDRESS>
Naouel Baili
IQVIA, US
<EMAIL_ADDRESS>
Brian Vannah
IQVIA, US
<EMAIL_ADDRESS>
work done during internship at IQVIA

###### Abstract

Quality Estimation (QE) is an important component of the machine translation
workflow as it assesses the quality of the translated output without
consulting reference translations. In this paper, we discuss our submission to
the WMT 2021 QE Shared Task. We participate in Task 2 sentence-level sub-task
that challenge participants to predict the HTER score for sentence-level post-
editing effort. Our proposed system is an ensemble of multilingual BERT
(mBERT)-based regression models, which are generated by fine-tuning on
different input settings. It demonstrates comparable performance with respect
to the Pearson’s correlation and beats the baseline system in MAE/ RMSE for
several language pairs. In addition, we adapt our system for the zero-shot
setting by exploiting target language-relevant language pairs and pseudo-
reference translations.

##  1 Introduction

Progress in machine translation (MT) has accelerated due to the introduction
of deep learning based approaches, dubbed as neural machine translation (NMT)
Cho et al. (2014); Sutskever et al. (2014); Bahdanau et al. (2014). Several
metrics (e.g., BLEU Papineni et al. (2002), METEOR Agarwal and Lavie (2008))
are used to automatically evaluate the quality of the translations outputted
by the NMT systems. However, these evaluation metrics require comparing the
NMT outputs against human-prepared reference translations, which cannot be
readily obtained. To tackle this predicament, recently quality estimation (QE)
Blatz et al. (2004); Specia et al. (2018) has emerged as an alternative
evaluation approach for NMT systems. QE obviates the need for human judgements
and hence can be efficiently integrated into the dynamic translation pipeline
in the industry setting.

QE is performed at different granularity (e.g., word, sentence, document)
Kepler et al. (2019a); in this work we focus on the sentence-level post-
editing effort task, which predicts the quality of the translated sentence as
a whole in terms of the number of edit operations that need to be made to
yield a post-edited translation, termed as HTER Snover et al. (2006).

Sentence-level QE using neural approaches is generally treated as a supervised
regression problem involving mainly two steps. In the first step, an encoder
is used to learn vector representation/s of the source and translation
sentences. While in the second step, the learned representations are passed
through a sigmoid output layer to estimate the HTER score. These two steps can
be performed either with a single model in an end-to-end fashion (e.g., Bi-RNN
Ive et al. (2018)), or using two separate models (e.g., POSTECH Kim et al.
(2017)). The different QE systems vary in their choice of the encoder, which
range from RNN-based to Transformer-based models.

In this work, we leverage the fine-tuning capability of a Transformer-based
encoder, namely the mBERT Devlin et al. (2018) pre-trained model. Alongside
the standard practice of feeding both the source and target (i.e.,
translation) sentences as the input sequence Kepler et al. (2019a); Kim et al.
(2019), we also explore other input settings based on only the target-side
sentences (i.e., monolingual context). To this end, our final QE system is an
ensemble of several mBERT models 111we also experimented with XLM-RoBERTa
Conneau et al. (2019) as the component model in our preliminary run; however,
the results were worse compared to mBERT, each generated by fine-tuning on a
different input combination comprising the source and/or target sentences. We
experiment with the following three input settings: (1) both source and
target, (2) just target and (3) both target and a randomly-sampled target
sentence in the data forming the input sequence. Empirical analysis on 6
language pairs shows that the ensemble model is able to perform better than
the individual fine-tuned models. Moreover, we provide experimental results
for zero-shot QE, where training data for the test language pair is not
available. This we tackle by improvising on the available training/dev data
that match the target language of the test language pair and also by
generating the pseudo-reference translations in that language.

##  2 Data

We use the WMT21 QE Shared Task 2 sentence-level data Fomicheva et al. (2020a,
b) for the following 7 language pairs: English-German (En-De), Romanian-
English (Ro-En), Estonian-English (Et-En), Nepalese-English (Ne-En), Sinhala-
English (Si-En), Russian-English (Ru-En) and Khmer-English (Km-En). Source-
side data for each language pair includes sentences from Wikipedia articles,
with part of the data gathered from Reddit articles for Ru-En. To obtain the
translations, state-of-the-art MT models Vaswani et al. (2017) built using
fairseq toolkit Ott et al. (2019) were used. The label for this task is the
HTER score for the source-translation pair. Annotation was performed first at
the word-level with the help of TER 222http://www.cs.umd.edu/ snover/tercom/
tool. The word-level tags were then aggregated deterministically to obtain the
sentence-level HTER score. The training, development, test and blind test data
sizes for each language pair (except Km-En) are 7K, 1K, 1K and 1K instances
respectively. As Km-En language pair was introduced for zero-shot prediction,
only the test data containing 990 source and translation sentences was
provided.

##  3 Our Approach

A key innovation in recent neural models lies in learning the contextualized
representations by pre-training on a language modeling task. One such model,
the multilingual BERT (mBERT) 333https://github.com/google-
research/bert/blob/master/multilingual.md, is a transformer-based masked
language model that is pre-trained on monolingual Wikipedia corpora of 104
languages with a shared word-piece vocabulary. Training the pre-trained mBERT
model for a supervised downstream task, aka finetuning, has dominated
performance across a wide spectrum of NLP tasks Devlin et al. (2018). Our
proposed approach leverages this fine-tuning capability of mBERT so as to form
the component models in the ensemble QE system (Section 3.3). That is, each
component model is a re-purposed mBERT that is fine-tuned for the sentence-
level HTER score prediction task on one of the three input settings discussed
in Section 3.2.

###  3.1 Fine-tuning mBERT for Regression

mBERT’s model architecture is similar to BERT 444https://huggingface.co/bert-
base-uncased and contains the following parameter settings: 12 layers, 12
attention heads and 768 hidden dimension per token. However, the only
difference is that mBERT is trained on corpora of multiple languages instead
of just on English. This enables mBERT to share representations across the
different languages and hence can be conveniently used for all language pairs
in the WMT21 data.

We first load the pre-trained mBERT model 555https://huggingface.co/bert-base-
multilingual-uncased and use its weights as the starting point of fine-tuning.
The pre-trained mBERT is then trained on QE-specific input sequences (Section
3.2) for a few epochs such that the constructed sequence $X$ is consumed by
mBERT to output the contextualized representation
$\textbf{h}=(h_{CLS},h_{x_{1}},h_{x_{2}},..h_{x_{T}},h_{SEP})$. Here, $[CLS]$
is a special symbol that denotes downstream classification and $[SEP]$ is for
separating non-consecutive token sequences. Considering the final hidden
vector of the $[CLS]$ token as the aggregate representation, it is then passed
into the output layer with sigmoid activation to predict the HTER score:

$y=sigmoid(\mathbf{W}\cdot\mathbf{h}_{CLS}+\mathbf{b})$ (1)

W is a weight matrix for sentence-level QE fine-tuning that is trained along
with all the parameters of mBERT end-to-end.

###  3.2 Input Settings

We construct the input sequence for each language pair in the following three
ways:

  1. SRC-MT: Given a source sentence $\textbf{s}=(s_{1},s_{2},...s_{N})$ from a source language (e.g., English) and its translation $\textbf{t}=(t_{1},t_{2},...t_{M})$ from a target language (e.g., German), we concatenate them together as $X=([CLS],t_{1},t_{2},...t_{M},[SEP],s_{1},s_{2},...s_{N},\\\ \ [SEP])$ to form the input sequence.

  2. MT: The target sentence is only used to form the input sequence, $X=([CLS],t_{1},t_{2},...t_{M},[SEP])$.

  3. MT-MT’:

Given the translation t for a source sentence s, we randomly sample another
translation $\textbf{t'}=(t^{\prime}_{1},t^{\prime}_{2},...t^{\prime}_{K})$
from the training data having HTER label close to t 666to ensure that t’ is
similar to t, we check that the difference between their HTER scores is within
0.1. Although the source sentences for t and t’ are different, we assume the
additional monolingual context would help mBERT learn the correlating QE-
specific features between t and t’ for the target-side language. The resultant
input sequence is
$X=([CLS],t_{1},t_{2},...t_{M},[SEP],t^{\prime}_{1},t^{\prime}_{2},...t^{\prime}_{K},\\\
\ [SEP])$.

We fine-tune each of these mBERT models using AdamW optimizer Kingma and Ba
(2014); Loshchilov and Hutter (2017) for two epochs with a batch size of 32
and a learning rate of $2e^{-5}$.

###  3.3 Ensemble Model

To take advantage of the individual strengths of the three mBERT component
models fine-tuned on the aforementioned input settings, we combine their HTER
score predictions by training an ensemble model. In particular, we experiment
with three different ensemble models - Gradient Boosting Friedman (2001),
AdaBoost Freund and Schapire (1997) and Average. For Gradient Boosting and
AdaBoost we use the implementation in scikit-learn 777https://scikit-learn.org
with 10-fold cross validation. The settings for Gradient Boosting are: number
of estimators 600, learning rate 0.01, minimum number of samples 3 and other
default settings. We use the default settings for AdaBoost. In Average
ensembling, we average the HTER score predictions by the three mBERT models.
Our system submission to WMT21 is based on Gradient Boosting as it gave the
best performance on the test data, as shown in Table 1.

Table 1: Performance of ENSBRT with different ensemble methods on the En-De test set. | Avg | AdaBoost | GradBoost
---|---|---|---
Pearson’s | 0.266 | 0.458 | 0.473
Spearman’s | 0.249 | 0.436 | 0.443
Table 2: Performance in Pearson’s correlation of mBERT fine-tuned with different input settings on the test set. ENSBRT is the proposed ensemble mBERT QE system. | En-De | Ro-En | Ru-En | Si-En | Et-En | Ne-En
---|---|---|---|---|---|---
SRC-MT | 0.389 | 0.793 | 0.400 | 0.526 | 0.601 | 0.489
MT | 0.469 | 0.762 | 0.374 | 0.552 | 0.580 | 0.491
MT-MT’ | 0.431 | 0.761 | 0.350 | 0.492 | 0.556 | 0.454
ENSBRT | 0.473 | 0.802 | 0.418 | 0.576 | 0.632 | 0.525
Table 3: Performance of BASELINE and ENSBRT on the WMT21 blind test set for different language pairs. Bold indicates ENSBRT beats BASELINE in that metric. | En-De | Ro-En | Ru-En | Si-En | Et-En | Ne-En | Km-En
---|---|---|---|---|---|---|---
BASELINE Pearson’s $\uparrow$ | 0.529 | 0.831 | 0.448 | 0.607 | 0.714 | 0.626 | 0.576
MAE $\downarrow$ | 0.183 | 0.142 | 0.255 | 0.204 | 0.195 | 0.205 | 0.241
RMSE $\downarrow$ | 0.129 | 0.115 | 0.188 | 0.159 | 0.149 | 0.160 | 0.196
ENSBRT Pearson’s $\uparrow$ | 0.519 | 0.795 | 0.376 | 0.522 | 0.666 | 0.572 | 0.529
MAE $\downarrow$ | 0.171 | 0.171 | 0.251 | 0.206 | 0.171 | 0.176 | 0.262
RMSE $\downarrow$ | 0.129 | 0.141 | 0.189 | 0.162 | 0.132 | 0.139 | 0.197

###  3.4 Zero-Shot QE

Performing sentence-level QE in the zero-shot setting presents a unique
challenge as the QE system is expected to predict HTER scores for sentences in
a test language pair (e.g., Km-En) without having been trained on any
instances from that test language pair. We address this by training on
language pairs in the WMT21 QE data that match the target-side language (i.e.,
En) in the test language pair. The reason we focus on the target-side language
is because the component mBERT models in the proposed ensemble QE system are
fine-tuned on monolingual input sequences in the target-side language, which
could potentially help the QE system generalize on the unseen test language
pair. We consider the training and development data for the following language
pairs in WMT21 QE data: Ro-En, Si-En, Et-En. Additionally, we augment this
data by generating pseudo-references in the target language. A pseudo-
reference Scarton and Specia (2014) is a translation for a source sentence
that is outputted by a different NMT system than the one that produced the
actual translations (e.g., transformer-based translation system proposed in
Vaswani et al. (2017)) and has shown to improve sentence-level QE performance
Soricut and Narsale (2012). We use Google Translate
888https://github.com/lushan88a/google_trans_new to get the pseudo-references
in En for the Ro, Si and Et source sentences. The HTER scores for the
translation and pseudo-reference pairs are then obtained using the TER tool.
We train the ensemble QE system on the combined WMT21 QE data and the pseudo-
reference parallel data, and test on the unseen test language pair.

##  4 Baseline

The baseline QE system (BASELINE) set by the WMT21 organizers this year is the
Transformer-based Predictor-Estimator model Kepler et al. (2019b); Moura et
al. (2020). XLM-RoBERTa is used as the Predictor for feature generation. The
baseline system is fine-tuned on the HTER scores and word-level tags jointly.

##  5 Results

Table 2 presents the experimental results of mBERT fine-tuned on the
$SRC$-$MT$, $MT$ and $MT$-$MT^{\prime}$ input settings, as well as the
performance of the ensemble of the three mBERT models, which we call ENSBRT.
First, comparing among the three input settings, it seems that mBERT exhibits
competitive results even when it does not have knowledge of the source-side
text in the $MT$ and $MT$-$MT^{\prime}$ settings, in particular for the
following language pairs - En-DE, Si-En, Ne-En. While the ensemble mBERT
model, ENSBRT, outperforms the independent counterparts for all the language
pairs. This shows that the ensemble method can help to balance out the
weakness of any component model, thereby benefiting the sentence-level QE task
overall. We also visualize ENSBRT’s predictions against the ground truth HTER
scores in Figure 1.

Table 3 compares the QE performance between the BASELINE and ENSBRT in terms
of Pearson’s correlation, RMSE and MAE on the WMT21 blind test set, for which
the ground truth HTER scores were not available at the time. We submitted
results for 6 language pairs (En-De, Ro-En, Ru-En, Si-En, Et-En, Ne-En) in the
normal QE setting and one language pair (Km-En) for zero-shot prediction.
ENSBRT demonstrates comparable performance to the BASELINE for Pearson’s and
outperforms it in either MAE or RMSE for the following language pairs: En-De,
Ru-En, Et-En and Ne-En.

##  6 Conclusion

In this work, we describe the ENSBRT system submission to the WMT21 QE Shared
Task. ENSBRT is based on fine-tuning the multilingual BERT pre-trained model
for sentence-level translation quality score prediction. We explore three
different input settings for fine-tuning which include either bilingual or
monolingual context, and combine the predictions of the three models using
ensemble methods as our final system. Furthermore, zero-shot QE is facilitated
by using labeled data for existing language pairs and pseudo-references that
align with the target language of the unseen test data.

Figure 1: Visualization comparing HTER score predictions by ENSBRT (i.e.,
predicted (red)) against the gold labels (i.e., original (blue)) for 6
language pairs on the test set. X-axis represents each data point and Y-axis
is the HTER score. The closer the corresponding red line and blue dot are to
each other the better, as we expect the HTER prediction to be closer to the
ground truth.

## References

  * Agarwal and Lavie (2008) Abhaya Agarwal and Alon Lavie. 2008.  Meteor, m-bleu and m-ter: Evaluation metrics for high-correlation with human rankings of machine translation output.  In _Proceedings of the Third Workshop on Statistical Machine Translation_ , pages 115–118.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_.
  * Blatz et al. (2004) John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004.  Confidence estimation for machine translation.  In _Coling 2004: Proceedings of the 20th international conference on computational linguistics_ , pages 315–321.
  * Cho et al. (2014) Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  _arXiv preprint arXiv:1406.1078_.
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  _arXiv preprint arXiv:1911.02116_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Fomicheva et al. (2020a) Marina Fomicheva, Shuo Sun, Erick Fonseca, Frédéric Blain, Vishrav Chaudhary, Francisco Guzmán, Nina Lopatina, Lucia Specia, and André F. T. Martins. 2020a.  MLQE-PE: A multilingual quality estimation and post-editing dataset.  _arXiv preprint arXiv:2010.04480_.
  * Fomicheva et al. (2020b) Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. 2020b.  Unsupervised quality estimation for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:539–555.
  * Freund and Schapire (1997) Yoav Freund and Robert E Schapire. 1997.  A decision-theoretic generalization of on-line learning and an application to boosting.  _Journal of computer and system sciences_ , 55(1):119–139.
  * Friedman (2001) Jerome H Friedman. 2001.  Greedy function approximation: a gradient boosting machine.  _Annals of statistics_ , pages 1189–1232.
  * Ive et al. (2018) Julia Ive, Frédéric Blain, and Lucia Specia. 2018.  Deepquest: a framework for neural-based quality estimation.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 3146–3157. Association for Computational Linguistics.
  * Kepler et al. (2019a) Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, António Góis, M Amin Farajian, António V Lopes, and André FT Martins. 2019a.  Unbabel’s participation in the wmt19 translation quality estimation shared task.  _arXiv preprint arXiv:1907.10352_.
  * Kepler et al. (2019b) Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, and André FT Martins. 2019b.  Openkiwi: An open source framework for quality estimation.  _arXiv preprint arXiv:1902.08646_.
  * Kim et al. (2017) Hyun Kim, Hun-Young Jung, Hongseok Kwon, Jong-Hyeok Lee, and Seung-Hoon Na. 2017\.  Predictor-estimator: Neural quality estimation based on target word prediction for machine translation.  _ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)_ , 17(1):1–22.
  * Kim et al. (2019) Hyun Kim, Joon-Ho Lim, Hyun-Ki Kim, and Seung-Hoon Na. 2019.  Qe bert: bilingual bert using multi-task learning for neural quality estimation.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2)_ , pages 85–89.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_.
  * Moura et al. (2020) Joao Moura, Miguel Vera, Daan van Stigt, Fabio Kepler, and André FT Martins. 2020.  Ist-unbabel participation in the wmt20 quality estimation shared task.  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 1029–1036.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  _arXiv preprint arXiv:1904.01038_.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , pages 311–318.
  * Scarton and Specia (2014) Carolina Scarton and Lucia Specia. 2014.  Document-level translation quality estimation: exploring discourse and pseudo-references.  In _Proceedings of the 17th Annual conference of the European Association for Machine Translation_ , pages 101–108.
  * Snover et al. (2006) Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006.  A study of translation edit rate with targeted human annotation.  In _Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers_ , pages 223–231.
  * Soricut and Narsale (2012) Radu Soricut and Sushant Narsale. 2012.  Combining quality prediction and system selection for improved automatic translation output.  In _Proceedings of the Seventh Workshop on Statistical Machine Translation_ , pages 163–170.
  * Specia et al. (2018) Lucia Specia, Carolina Scarton, and Gustavo Henrique Paetzold. 2018.  Quality estimation for machine translation.  _Synthesis Lectures on Human Language Technologies_ , 11(1):1–162.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  In _Advances in neural information processing systems_ , pages 3104–3112.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.