# Intrinsic and tunable quantum anomalous Hall effect and magnetic topological
phases in XYBi2Te5

Xin-Yi Tang  State Key Laboratory of Low Dimensional Quantum Physics,
Department of Physics, Tsinghua University, Beijing 100084, China  Zhe Li
<EMAIL_ADDRESS>Beijing National Research Center for Condensed Matter
Physics, and Institute of Physics, Chinese Academy of Sciences, Beijing
100190, China  Feng Xue  State Key Laboratory of Low Dimensional Quantum
Physics, Department of Physics, Tsinghua University, Beijing 100084, China
Beijing Academy of Quantum Information Sciences, Beijing 100193, China
Pengfei Ji  State Key Laboratory of Low Dimensional Quantum Physics,
Department of Physics, Tsinghua University, Beijing 100084, China  Zetao Zhang
State Key Laboratory of Low Dimensional Quantum Physics, Department of
Physics, Tsinghua University, Beijing 100084, China  Xiao Feng  State Key
Laboratory of Low Dimensional Quantum Physics, Department of Physics, Tsinghua
University, Beijing 100084, China  Beijing Academy of Quantum Information
Sciences, Beijing 100193, China  Frontier Science Center for Quantum
Information, Beijing 100084, China  Yong Xu  State Key Laboratory of Low
Dimensional Quantum Physics, Department of Physics, Tsinghua University,
Beijing 100084, China  Frontier Science Center for Quantum Information,
Beijing 100084, China  Tencent Quantum Laboratory, Tencent, Shenzhen,
Guangdong 518057, China  RIKEN Center for Emergent Matter Science (CEMS),
Wako, Saitama 351-0198, Japan  Quansheng Wu  Beijing National Research Center
for Condensed Matter Physics, and Institute of Physics, Chinese Academy of
Sciences, Beijing 100190, China  University of Chinese Academy of Sciences,
Beijing 100049, China  Ke He  State Key Laboratory of Low Dimensional Quantum
Physics, Department of Physics, Tsinghua University, Beijing 100084, China
Beijing Academy of Quantum Information Sciences, Beijing 100193, China
Frontier Science Center for Quantum Information, Beijing 100084, China  Hefei
National Laboratory, Hefei 230088, China

###### Abstract

By first-principles calculations, we study the magnetic and topological
properties of XYBi2Te5-family (X, Y = Mn, Ni, V, Eu) compounds. The strongly
coupled double magnetic atom-layers can significantly enhance the magnetic
ordering temperature while keeping the topologically non-trivial properties.
Particularly, NiVBi2Te5 is found to be a magnetic Weyl semimetal in bulk and a
Chern insulator in thin film with both the Curie temperature ($\sim$150 K) and
full gap well above 77 K. Ni2Bi2Te5, MnNiBi2Te5, NiVBi2Te5 and NiEuBi2Te5
exhibits intrinsic dynamic axion state. Among them, MnNiBi2Te5 has a Néel
temperature over 200 K and Ni2Bi2Te5 even demonstrates antiferromagnetic order
above room temperature. These results indicate an approach to realize high
temperature quantum anomalous Hall effect and other topological quantum
effects for practical applications.

##  I introduction

In the past two decades, topological states of materials have grown into a
large branch of modern condensed matter physics [1, 2, 3, 4, 5, 6]. Emergent
topological phases, especially the ones associated with magnetism, like the
quantum anomalous Hall effect (QAHE) [7, 8, 9, 10, 11], axion insulators
(AxIs) [12, 13, 14, 15, 16, 17] and magnetic Weyl semimetals (WSMs) [18, 19,
20], have attracted lots of attention. The QAHE has been experimentally
achieved in Cr-doped topological insulator thin films initially in 2013 [9].
However, the reliance on extremely low temperature hindered its applications.
Many candidate materials that may realize high temperature QAH insulators,
magnetic WSMs or AxIs have been proposed, such as Co3Sn2S2 [19, 20], EuIn2As2
[21], LiFeSe [22], PdBr3-family materials [23], NiAsO3 and PdSbO3 [24].
Although some of them have been prepared and well-studied, quantized transport
properties—the crucial mile stone—have not been achieved. Thin films of the
intrinsic magnetic topological insulator MnBi2Te4 [25, 26] are a high
temperature QAH system that has shown quantized transport properties in
experiment. The magnetically induced surface state gap (briefly, magnetic gap)
of MnBi2Te4 thin films can reach $\sim$50 meV [27, 28], more than enough to
support QAHE above 77 K. Zero-field quantization of the anomalous Hall
resistance have been observed in odd septuple-layer (SL) MnBi2Te4 thin flakes
at 1.4 K [29], higher than that in magnetically doped (Bi,Sb)2Te3. Amazingly,
under high magnetic field, nearly quantized Hall resistance has been observed
in some MnBi2Te4 thin flake samples at a temperature as high as $\sim$40 K
[30]. Other magnetic topological phases such as magnetic WSM and AxI can be
realized in MnBi2Te4 with certain magnetic configurations and thicknesses [27,
28, 31, 32, 33, 30]. Obviously, MnBi2Te4 provides a solid base for the
exploration of high-temperature QAHE and other related topological quantum
effects.

The temperature to achieve the QAHE is determined by both the magnetic
ordering temperature and the magnetic gap size. Since the magnetic gap size of
MnBi2Te4 thin films is as large as $\sim$50 meV, the bottleneck is its
magnetic ordering temperature which is only $\sim$25 K [26]. Such a low
temperature originates not only from the weak magnetic coupling strength but
also from magnetic fluctuations, largely due to the two-dimensional (2D)
nature of its magnetism [34]. A straightforward way to solve the two problems
is to make the strongly coupled magnetic atomic layer a little thicker while
not thick enough to destroy the 2D topological properties. Actually, the Curie
or Néel temperature of magnetic thin films is significantly elevated as the
thickness increases from monolayer to bilayer [35].

In fact, there exists such a material: Mn2Bi2Te5, which can be considered as
MnBi2Te4 with an additional MnTe bilayer inserted in each septuple layer (SL).
The material was predicted to be a possible host of QAHE, topological
magnetoelectric effect (TME) or AFM topological insulator (TI) phase depending
on its layer magnetization [36, 37, 38]. Mn2Bi2Te5 single crystals have
recently been successfully prepared [39]. However, as discussed below, the
inherent magnetic couplings of Mn2Bi2Te5 are not strong and thus its magnetic
ordering temperature is just 20 K. We can replace one of Mn atom layer in
Mn2Bi2Te5 with other magnetic atoms (partially discussed in Ref. [40]), or
even both, namely XYBi2Te5 where X and Y represent magnetic atoms, to
implement the above idea.

In this paper, via systematic first-principles calculations, we demonstrate
our findings on high-temperature topological phases in some compounds of
XYBi2Te5-family materials, where X, Y = Mn, Ni, V, Eu. We predict that
NiVBi2Te5 can be a FM WSM in the bulk phase with a Curie temperature of
$\sim$150 K and a Chern insulator in thin films that can possibily show high-
temperature QAHE. Furthermore, the bulk phases of Ni2Bi2Te5 and MnNiBi2Te5 are
predicted to be high-temperature dynamic AxIs, in which the first one even
shows magnetic ordering temperature above room temperature.

##  II methods

The density functional theory (DFT) calculations were carried out via the
Vienna Ab initio Simulation Package (VASP) [41]. The projector-augmented wave
(PAW) method and the plane-wave basis with an energy cutoff of 350 eV were
utilized. The exchange-correlation energy is described by Perdew-Burke-
Ernzerhof (PBE) functional under the generalized gradient approximation (GGA)
[42]. To treat the localized $d$\- and $f$-orbitals, the GGA+$U$ approach was
adopted. $U$ parameters were selected to be 4 eV, 4 eV, 3 eV and 5 eV for
Mn-$3d$, Ni-$3d$, V-$3d$ and Eu-$4f$ orbitals, respectively. In addition, the
modified Becke-Johnson (mBJ) functional was used in bulk band computations
[43]. In order to correctly describe the inherent van der Waal (vdW)
interactions, the DFT-D3 method [44] was considered. Moreover, VASPKIT [45]
and PHONOPY [46] codes were used for data post-processing. Plus, structural
visualization was achieved with the help of VESTA [47].

The $\Gamma$-centered Monkhorst-Pack $k$-point meshes of 9$\times$9$\times$3
and 13$\times$13$\times$1 were adopted for bulk and thin-film structures
respectively. Denser grids of 11$\times$11$\times$3 and 23$\times$23$\times$5
were used to calculate magnetocrystalline anisotropy energies (MAEs). For
energy and band calculations, the geometry optimizations were performed until
the Hellmann-Feynman force on each atom is smaller than 0.01 eV/Å, while the
energy convergence criterion was chosen to be 1.0$\times$10-6 eV. As for
phonon spectra, structural relaxations were done with threshold of
1.0$\times$10-4 eV/Å and 1.0$\times$10-7 eV for higher accuracy and dispersion
relations were calculated by density functional perturbation theory (DFPT).

The interlayer and interatomic-layer exchange energies ($E_{ex}$) are defined
as the energy subtraction between antiferromagnetic (AFM) and ferromagnetic
(FM) configurations, i.e. $E_{ex}=E_{\text{AFM}}-E_{\text{FM}}$. Notice that
when we calculated the interlayer (interatomic-layer) $E_{ex}$, the
interatomic-layer (interlayer) couplings were kept unchanged. The MAEs are
represented by the energy subtraction between out-of-plane and in-plane
configurations, i.e. $E_{\text{MAE}}=E_{\text{in-plane}}-E_{\text{out-of-
plane}}$. The in-plane magnetic moments are set along $x$-axis, since there is
neglecting energy difference between $x$-axis and $y$-axis polarizations in
all materials within the scope of this paper. Collinear spin-polarized
calculations without spin-orbital couplings (SOCs) and non-collinear ones with
SOCs were respectively conducted for the calculations of exchange energies and
MAEs.

We also obtained tight-binding Hamiltonians of bulk materials based on the
maximally localized Wannier functions (MLWFs) by the Wannier90 package [48,
49, 50]. Thin-film Hamiltonians were extracted from corresponding bulk ones,
originated from mBJ-functional calculations. Next, all the topological
properties, including Fermi surface visualizations, and Weyl points (WPs)
characterizations of bulk materials, as well as edge states and Chern-number
calculations of thin films, were accomplished using the tight-binding
Hamiltonian method as implemented in the WannierTools package [51] based on
MLWFs.

##  III results

###  III.1 Lattice structures

Figure 1: Four typical structures of bulk XYBi2Te5. Dashed lines stands for
unit cells. If X=Y, type-I and type-II (also type-III and type-IV) become the
same. $\theta_{1}$ and $\theta_{2}$ in panels (a) and (c) are
$90^{\circ}$-like and $180^{\circ}$-like angles, which are determining factors
of magnetic coupling between X and Y atomic layers inside one NL. Panel (d)
illustrates the three kinds of magnetic couplings discussed in Sec. III.2 by
using type-IV lattice structure as an example.

The unit cell of XYBi2Te5 consists of Te-Bi-Te-X-Te-Y-Te-Bi-Te nonuple layers
(NLs), with a vdW vacuum gap between neighboring NLs. There have been
published theoretical works [36, 37] focusing on Mn2Bi2Te5, one special case
of XYBi2Te5 where X=Y=Mn. They assumed an ABC-stacking structure of the
internal Mn-Te bilayer structures, which is similar with the well-known
MnBi2Te4. There are also different viewpoints on what the most stable
structure should be like. As the number of Mn-Te bilayers increases, these
atomic layers may tend to form an ABAC-stacking structure [52, 53] which
resembles the bulk phase of MnTe, and this seems to be the case in
Mn2Bi2Te5-family materials [38].

Beyond MnTe, bulk phases of NiTe and VTe also appear as ABAC-stacking, while
EuTe remains ABC-stacking [54]. This suggests that ABAC-stacking and ABC-
stacking lattice structures (NiAs-type and NaCl-type respectively [38]) may
coexist in XYBi2Te5 when different magnetic elements are included.

Therefore, we took both types of lattice structures into consideration.
Another important question is the ordering of magnetic atomic layers when
X$\neq$Y. To answer it, we studied two simple bulk models with the sequence of
magnetic atomic layers being like ”$\cdots$-XY-XY-XY-XY-$\cdots$” (XY-XYBT) or
”$\cdots$-XY-YX-XY-YX-$\cdots$” (XY-YXBT). Thus a total of four lattice
structures, named as type-I, type-II, type-III and type-IV, were investigated,
as depicted in Fig. 1. These four typical structures are NiAs-type XY-XYBT
(type-I), NiAs-type XY-YXBT (type-II), NaCl-type XY-XYBT (type-III) and NaCl-
type XY-YXBT (type-IV). Their corresponding space group is shown in Table 1.
Monolayer XYBi2Te5, however, can just reflect the stacking order inside one
single NL, so only two types of structures should be considered (type-I and
type-III).

Table 1:  Space group of different lattice structures. Magnetic moments have been ignored. When X=Y, type-I and type-II (also type-III and type-IV) become the same, so we omit the space group of type-II (and type-IV) in this table. | type-I | type-II | type-III | type-IV
---|---|---|---|---
X=Y | No. 194 |  | No. 164 |
X$\neq$Y | No. 186 | No. 164 | No. 156 | No. 164

The relaxed lattice constants of both monolayer and bulk XYBi2Te5 (X, Y=Mn,
Ni, V, Eu) compounds are demonstrated in Appendix A. To confirm the structural
stability, we performed calculations on phonon dispersions. Due to the
layered-stacking nature of XYBi2Te5, we only carried out phonon-dispersion
computations on monolayers. Monolayers with type-I and type-III lattice
structures can be seen as basic building blocks of all the structures
discussed here. Therefore, we analyzed the twenty monolayer structures as a
whole. Their phonon spectra are shown in Fig. 2. For the majority of them, no
virtual frequency exists, indicating that either NiAs-type or NaCl-type
XYBi2Te5 should be stable. The only exception appears in NaCl-type Ni2Bi2Te5
since there is slight virtual frequency of the acoustic phonons around the
$\Gamma$ point [see Fig. 2(b)]. This is not a rare case in first-principles
calculations and might be a result of insufficient accuracy [27, 55, 56]. For
this reason, the stability of NaCl-type Ni2Bi2Te5 may be possibly acceptable
(perhaps metastable) and we still include it in the following sections also
for the completeness of this work.

We also examined the influence of X-Y mixing (for $X\neq Y$ only) on
structural stability. Here X-Y mixing refers to situations where the two
distinguishable kinds of magnetic atoms are mixed instead of locating at
separate atomic layers [see Fig. 1]. Our systematic investigation suggests
that X-Y mixing will result in significant virtual frequencies on phonon
spectra, which is direct evidence for structural instability. Therefore, we
think it is reasonable to study XYBi2Te5 structures without X-Y mixing
problem. Computational details can be found in Appendix B.

Figure 2: Phonon dispersions of (a) type-I (NiAs-type) and (b) type-III (NaCl-
type) monolayer XYBi2Te5-family materials.

###  III.2 Magnetic properties

We then come to discuss the magnetic properties of XYBi2Te5. Both of the
ground state magnetic configurations and the coupling strength are determined
by superexchange rules [57]. In XYBi2Te5, three types of magnetic couplings
determine the stable magnetic configurations, including couplings between X
and Y atoms in neighboring NLs (interlayer couplings), between X or Y atoms in
the same atomic layer plane (intralayer couplings), and between X and Y atoms
in the same NL (interatomic-layer couplings). All of these couplings are
illustrated in Fig. 1(d), where type-IV lattice structure is used as an
example. The interlayer and intralayer couplings resemble those in
MnBi2Te4-family materials, originated from Goodenough-Kanamori 180∘ and 90∘
rules separately [27, 31, 32, 58], so we mainly need to understand the extra
interatomic-layer couplings.

Table 2: Summary of interlayer/interatomic-layer $E_{ex}$ (in unit of meV) and MAE (meV/NL) of bulk XYBi2Te5 materials. Positive (negative) values stand for FM (AFM) coupling in $E_{ex}$, as well as out-of-plane (in-plane) magnetization direction in MAE. X-Y | type-I | type-II111In Type-II and Type-IV structures, two kinds of interlayer couplings exist because of the XY-YX order. Hence, the interlayer $E_{ex}$ here can be seen as the average values of them. | type-III | type-IV11footnotemark: 1
---|---|---|---|---
Mn-Mn | Interlayer | -1.48 |  | -1.74 |
Interatomic | -58.45 |  | 1.36 |
MAE | 0.34 |  | 0.18 |
Ni-Ni | Interlayer | -9.77 |  | -10.82 |
Interatomic | -283.38 |  | -266.89 |
MAE | 2.59 |  | 5.22 |
V-V | Interlayer | -0.56 |  | -0.68 |
Interatomic | -156.41 |  | -96.10 |
MAE | -0.46 |  | -0.17 |
Eu-Eu | Interlayer | -0.14 |  | -0.11 |
Interatomic | -0.74 |  | -3.17 |
MAE | -0.08 |  | -0.06 |
Mn-Ni | Interlayer | -3.62 | -8.06 | -4.14 | -8.23
Interatomic | -132.65 | -135.32 | -104.69 | -106.89
MAE | 1.90 | 1.33 | 2.18 | 2.53
Mn-V | Interlayer | 1.66 | -1.44 | 1.33 | -0.85
Interatomic | -33.07 | -32.42 | 34.40 | 34.90
MAE | -0.14 | -0.13 | -0.03 | -0.07
Ni-V | Interlayer | 1.41 | -11.55 | 1.59 | -8.89
Interatomic | 56.69 | 58.10 | 84.46 | 85.25
MAE | 0.94 | 1.08 | 0.85 | 1.00
Mn-Eu | Interlayer | 0.25 | -0.57 | 0.38 | -0.56
Interatomic | 15.25 | 15.38 | 11.68 | 11.38
MAE | 0.05 | 0.06 | 0.10 | 0.11
Ni-Eu | Interlayer | 0.27 | -9.19 | 0.80 | -12.48
Interatomic | 28.24 | 26.81 | 24.52 | 23.76
MAE | 0.82 | 0.96 | 1.02 | 1.21
V-Eu | Interlayer | -0.32 | -0.44 | -0.30 | -0.52
Interatomic | -1.01 | -2.91 | -13.43 | -13.71
MAE | -0.13 | -0.11 | -0.09 | -0.06

Table 2 shows values of interlayer/interatomic-layer $E_{ex}$, and also MAE of
XYBi2Te5-family materials. Similar to the MnBi2Te4-family materials, the sign
and relative strength of the magnetic couplings can also be understood with
the Goodenough-Kanamori rules [58]. Here, we focus on the interatomic-layer
couplings which are expected to be much stronger than the interlayer ones.
Firstly, we analyze the type-III structure, whose inherent ABC-stacking
structure resembles that of well-known MnBi2Te4. Its interatomic-layer
couplings are contributed by two kinds of hopping channels of X-Te-Y bonds,
one with the bond angle near $90^{\circ}$ [$\theta_{1}$ in Fig. 1(c)], and the
other with the bond angle near $180^{\circ}$ [$\theta_{2}$ in Fig. 1(c)]. The
signs of the couplings contributed by the two channels (referred as
$\theta_{1}$ and $\theta_{2}$ channels below, respectively) are opposite [see
Fig. 3(a)]. $\theta_{1}$ and $\theta_{2}$ values of type-III XYBi2Te5 are
summarized in Fig. 3(b). For Ni2Bi2Te5 and V2Bi2Te5, $\theta_{2}$ is very
close to $180^{\circ}$, meanwhile $\theta_{1}$ deviates several degrees from
$90^{\circ}$. Therefore, the coupling is dominated by the $\theta_{2}$ channel
which is AFM. The interatomic-layer coupling of NiVBi2Te5 is also dominated by
the $\theta_{2}$ channel, which however gives a FM ground state since the
number of $3d$ electron is above five in Ni and below five in V [58]. In
Mn2Bi2Te5, the interatomic-layer coupling is even smaller than the interlayer
one. It is because the couplings via the $\theta_{1}$ and $\theta_{2}$
channels are largely compensated with each other. Note that by varying the on-
site Coulomb repulsion $U$ or the exchange-correlation functionals [see Fig.
3(c)], the interatomic-layer coupling of Mn2Bi2Te5 can be changed from FM to
A-type AFM. This does not contradict previous work on Mn2Bi2Te5 [36, 37, 38].
Nonetheless, whether the interatomic-layer coupling in Mn2Bi2Te5 is FM or AFM
has not been checked by experiment [39].

The values of $\theta_{1}$ in XEuBi2Te5 also approach 90∘, but the reverse
(caused by 90∘-rule) of interatomic-layer coupling ground state which is
mainly determined by 180∘-rule does not appear. It can be understood in the
following way. According to typical superexchange mechanism, the coupling
strength of the 180∘ rule and the 90∘ rule can be expressed as [57]

$J_{180}=\frac{4t_{pd}^{4}}{\left(U_{d}+\Delta_{pd}\right)^{2}}\left(\frac{1}{U_{d}}+\frac{1}{U_{d}+\Delta_{pd}}\right),$
(1)
$J_{90}=-\frac{4t_{pd}^{4}}{\left(U_{d}+\Delta_{pd}\right)^{2}}\frac{2J_{xy}}{4\left(U_{d}+\Delta_{pd}\right)^{2}-J_{xy}^{2}},$
(2)
$J_{90}/J_{180}\approx-\frac{-U_{d}J_{xy}}{2\left(U_{d}+\Delta_{pd}\right)\left(2U_{d}+\Delta_{pd}\right)}.$
(3)

Here $t_{pd}$ represents hopping strength between $p$ and $d$ orbitals.
$U_{d}$ stands for Hubbard $U$ term of magnetic atoms, while $\Delta_{pd}$ can
be obtained by subtracting the energy of occupied $p$ orbitals from that of
occupied $d$ orbitals. $J_{xy}$ is the Coulomb exchange term between two
orthogonal $p$ orbitals. Eq. (3) is extracted by considering $J_{xy}\ll
U_{d}+\Delta_{pd}$. The $5d$ orbitals of Eu, compared with $3d$ orbitals of
Mn, V and Ni, locate quite far away from the Fermi level [58]. So,
$\Delta_{pd}$ of materials containing Eu is obviously larger than that of
others, leading to weaker contributions from 90∘ rule. Therefore, there is no
anomaly in the interatomic-layer couplings of XEuBi2Te5.

Figure 3: Interatomic-layer couplings of type-III XYBi2Te5. (a) Schematic
illustration of the Goodenough-Kanamori 180∘ and 90∘ superexchange mechanisms.
Here we adopt $d$ orbitals of Mn as an example. (b) Bond angle values of type-
III XYBi2Te5. (c) Energy difference between FM and AFM interatomic-layer
couplings of type-III Mn2Bi2Te5 obtained with different $U$ and exchange-
correlation functionals.

The above analysis on type-III structure is also suitable for type-IV, since
they bare resembling X-Te-Y bond structures and also interatomic-layer
$E_{ex}$. However, due to the $ABAC$-stacking order in type-I and type-II
structures, the $\theta_{1}$ and $\theta_{2}$ stay far away from either 180∘
or 90∘ [see Fig. 1(a)] and it becomes hard to qualitatively settle the
interatomic-layer couplings via Goodenough-Kanamori 180∘ and 90∘ rules. Also,
the shortest distance between magnetic atoms in the two separate layers
becomes smaller in NiAs-type structure ($\lesssim$4 Å) than that in NaCl-type
structure and even the distance between the nearest magnetic atoms within one
atomic layer ($\textgreater$4 Å), which means the direct exchange interaction
should not be ignored, making the interatomic-layer couplings more
complicated. Based on DFT calculations, we found that, besides Mn2Bi2Te5
mentioned in Ref. [38], the interatomic-layer coupling of MnVBi2Te5 also
changes (both from FM to AFM) when the lattice structures evolve from NaCl-
Type to NiAs-type, while others remain unchanged.

After figuring out the magnetic couplings and especially the ground state
configurations of all XYBi2Te5, we tried to make it clear which of the four
lattice structures proposed in this paper, should be energetically the most
favorable. As can be seen in Fig. 4, we made comparisons of ground state
energies within every kind of XYBi2Te5. A NiAs-type lattice structure is
preferred when Eu is excluded from the component magnetic elements, while a
NaCl-type lattice structure is preferred once Eu is included. This may be a
result of the lackness of ABAC-stacking structure in bulk phase EuTe, as
mentioned before. With the increasing number of magnetic-cation and Te-anion
bilayers, the local chemical environment around magnetic atoms will approach
that in their corresponding bulk phases, leading to similar stacking orders
eventually.

Figure 4: Energy comparisons of magnetic ground states based on four types of
bulk phase XYBi2Te5 lattice structures (two types indeed when X=Y). The lowest
energies are equally set to zero and marked in red in every compound of
XYBi2Te5. Type-II MnVBi2Te5 and type-III MnEuBi2Te5 may be quasi-ground states
since they show slight difference compared to type-I MnVBi2Te5 and type-IV
MnEuBi2Te5 respectively, which is beyond the precision of computations.

Another finding is about the sequence of magnetic atomic layers. It is
influenced by the strength of interlayer couplings. For example, the
energetically favorite structures in MnNiBi2Te5 (type-II), NiVBi2Te5 (type-II)
and NiEuBi2Te5 (type-IV) are all XY-YXBT. This is because the interlayer
coupling between Ni atomic layers is much stronger than others [see Table 2],
which is beneficial for lowering the total energy. This rule is not apparent
or applicable to cases where the strength of interlayer couplings between X-X,
Y-Y and X-Y are comparable, since their relative values may be sensitive to
even slight change in lattice structures. MnVBi2Te5 and MnEuBi2Te5 are two
typical examples. Negligible energy difference is found between their
type-I(III) and type-II(IV) structures.

Figure 5: Néel/Curie temperatures of six kinds of XYBi2Te5 materials with out-
of-plane magnetic ground states. (a) $C_{V}$-$T$ curves of those XYBi2Te5
combinations obtained by Monte Carlo simulations. (b) Their corresponding
Néel/Curie temperatures are extracted and shown in a bar chart for clarity.
The boiling temperature of liquid nitrogen (77 K) is marked with a red dashed
line.

We estimated the Néel/Curie temperatures of selected bulk XYBi2Te5 materials
with Monte Carlo (MC) simulations, mainly including the six XYBi2Te5 with out-
of-plane magnetic ground states [see Table 2]. A Heisenberg model was employed
to describe their magnetic behaviors, which involves the aforementioned three
kinds of magnetic couplings up to next-nearest neighbors and single-ion
anisotropy energy.

$\displaystyle H=\sum_{i\in X,Y}A_{i}(S_{i}^{z})^{2}+\sum_{i,j\in
X}J_{ij}^{X}\bm{S}_{i}\cdot\bm{S}_{j}+\sum_{i,j\in
Y}J_{ij}^{Y}\bm{S}_{i}\cdot\bm{S}_{j}$ (4)
$\displaystyle+\sum_{i,j\in\text{interlayer}}J_{ij}^{I}\bm{S}_{i}\cdot\bm{S}_{j}+\sum_{i,j\in\text{interatomic-
layer}}J_{ij}^{S}\bm{S}_{i}\cdot\bm{S}_{j}.$

$A_{i}$, $J_{ij}^{X}$ and $J_{ij}^{Y}$ are single ion anisotropy energy and
the intralayer couplings in X and Y atomic layers respectively. $J_{ij}^{I}$
and $J_{ij}^{S}$ refer to interlayer and interatomic-layer interaction
parameters. Note that in type-II and type-IV structures, $J_{ij}^{I}$ can be
further decomposed into $J_{ij}^{IX}$ and $J_{ij}^{IY}$ in order to describe
the two kinds of interlayer couplings.

During simulations, we firstly adopted 2$\times$105 MC steps for equilibrating
the system at each temperature, and next 1$\times$106 steps for computing
physical quantity such as heat capacity. A supercell of 12$\times$12$\times$3
unit cells as well as periodic boundary conditions were used. Larger supercell
were also tested, and we find in general a supercell of 12$\times$12$\times$3
is enough.

As shown in Fig. 5, type-I Ni2Bi2Te5, type-II NiVBi2Te5 and type-II MnNiBi2Te5
have Néel/Curie temperatures above 77 K, in which type-I Ni2Bi2Te5 even has a
Néel temperature above room temperature. The result coincides with their
strong interatomic-layer couplings (all above 50 meV per unit cell) shown in
Tab. 2.

Our calculation for type-I Mn2Bi2Te5 shows a moderate value of its Néel
temperature (61 K), which is much higher than what was obtained in experiment
[39] and another theoretical work [38]. However, it’s worth mentioning that
our calculations for type-III Mn2Bi2Te5 (27 K) is close to the experimental
result (20 K) [39] (see Appendix C). This might imply that the Mn2Bi2Te5
samples synthesized in laboratories possessed a stable but not energetically
favorite structure (type-III, not type-I), which is also likely to happen in
other XYBi2Te5 combinations. We think more experimental information, like
images of atomically resolved high-angle annular dark field STEM, is necessary
before the end of this structural puzzle.

One can see from Fig. 5 and Tab. 2 that the compounds including Ni atoms have
higher Néel/Curie temperature. The projected band structures and partial
density of states (DOS) of Ni, V, Mn and Eu are plotted in Fig. 6
respectively. Appreciably, the distributions of Ni $3d$ orbitals locate near
the Fermi level, which benefit the virtual hopping processes and greatly
enhance the magnetic coupling strength [57]. With unoccupied $e_{g}$ orbitals
situating even nearer the Fermi level, Ni behaves much better than V. In
contrast, the Mn $3d$ and Eu $5d$ orbitals locate quite far from the Fermi
level, generating much weaker coupling strength.

Figure 6: Band structures along high symmetry lines (with SOC) and partial DOS
distributions (without SOC) using monolayer type-I NiVBi2Te5 (a, c) and type-
III MnEuBi2Te5 (b, d) as examples. We have highlighted the $d$-orbital
components related to magnetic couplings.

###  III.3 Band topologies

Now we are ready to talk about both the intrinsic and tunable magnetic
topological properties of XYBi2Te5-family materials. Up to now, experimental
results [39] are not sufficient to decide which type of lattice structures can
be achieved in laboratory. Besides, due to the quite weak vdW interactions, it
may be possible to artificially tune the interlayer stacking orders of X and Y
atomic layers by methods like exfoliation. Therefore, although it is
theoretically reasonable to exclusively discuss the structures with the lowest
energies, we extend our discussions beyond this range and consider other
dynamically stable lattice structures if necessary.

Firstly, we discuss about a kind of symmetry-protected topological phase. It
was reported that in a system whose low-energy physics can be described by a
Bi2Te3-like effective Hamiltonian [59, 60], a $\mathcal{P}$-breaking,
$\mathcal{T}$-breaking and $\mathcal{PT}$-conserving perturbation brings
chances on achieving dynamic axion states [13]. This has been theoretically
shown in type-III Mn2Bi2Te5 with A-type AFM magnetic configuration [36, 37].

Figure 7: Intrinsic dynamic axion states in ground state XYBi2Te5. (a) DOS of
the five candidates. SOC is considered. (b) The evolution of band gap under
increasing SOC strength. (c, d) Band structures of type-I Ni2Bi2Te5 with SOC
(w SOC) and without SOC (w/o SOC), respectively. (e) Calculated surface states
of type-I Ni2Bi2Te5 on its (001) and (1$\bar{1}$0) planes. (f) Schematic
illustration of these two surfaces. Panels (g) to (i) are similar with panels
(c) to (e), but demonstrate situations in type-II MnNiBi2Te5.

Beyond this, we also found intrinsic dynamic axion states in type-I Ni2Bi2Te5,
type-II MnNiBi2Te5, type-II NiVBi2Te5 and type-IV NiEuBi2Te5, but not in
type-I Mn2Bi2Te5 by mBJ+$U$ calculations. They are all stable and
energetically favored lattice structures. Besides, their ground state magnetic
couplings [see Table 2 and Fig. 4], together with the lattice structures,
satisfied the aforementioned symmetry constraints. Although they belong to
different space groups from Bi2Te3 (No. 166), the threefold rotation along the
$c$ direction, twofold rotation along the $a$ direction and also inversion
symmetry (when magnetic moments are ignored) remain, which means a perturbed
Bi2Te3-like effective Hamiltonian is still applicable.

As reported previously, a band inversion process without gap-closing point and
gapped surface states on all surfaces are supposed to appear in materials
preserving $\mathcal{PT}$ symmetry but breaking individual $\mathcal{P}$ and
$\mathcal{T}$ symmetries, serving as the signatures on the emergence of
dynamic axion fields [61]. Figure 7(a) clearly illustrates the insulating
behavior of type-I Mn2Bi2Te5, type-I Ni2Bi2Te5, type-II MnNiBi2Te5, type-II
NiVBi2Te5 and type-IV NiEuBi2Te5. Ground state magnetic configurations and SOC
are considered. Then we artificially tuned the SOC strength and studied their
band gap evolution [see Fig. 7(b)]. By increasing the SOC strength from 60% to
100%, in which 100% represents the realistic value, the band gaps in four of
them firstly decreases, reaching their minimum and then reincreases. The
turning point of type-II MnNiBi2Te5 (99%) and type-IV NiEuBi2Te5 (97%) is very
close to the realistic value, thus implying large inherent dynamic axion
field. However, type-I Mn2Bi2Te5 can only demonstrate this topological phase
transition under a bit larger SOC strength (107%), so we claim it to be
trivial.

Figures 7(c) and 7(d) show the orbital-projected band structures of type-I
Ni2Bi2Te5 with and without SOC, respectively. Band inversion between Bi
$6p_{z}$ and Te $5p_{z}$ indicates its topologically non-trivial character.
Surface states on the (001) and (1$\bar{1}$0) planes were calculated and the
minimum gap of surface states reaches approximately 6 meV ($\sim$74 K) [see
Figs. 7(e) and 7(f)]. Figures 7(g), 7(h) and 7(i) shows the dynamic AxI
features in type-II MnNiBi2Te5. Specially, the minimum gap of surface states
in type-II MnNiBi2Te5 is larger than 17 meV ($\sim$210 K), which is comparable
with its high enough Néel temperature ($\sim$214 K). The band inversion
characters and surface state visualizations of the rest two candidates can be
found in Appendix D.

Figure 8: Topological features of type-I NiVBi2Te5 in bulk and thin films. (a)
Band structure of WSM type-I NiVBi2Te5. Details of projected band near one of
the WPs are shown in the inset. (b) Surface states on the (100) plane, showing
a Fermi arc connecting the two WPs. (c) The motions of the sum of WCCs on
spheres surrounding each WP in the momentum space. (d) The evolution of band
gap and Chern number with thickness of type-I NiVBi2Te5 slab. Chiral edge
states of (e) 6-NL ($\mathcal{C}=1$) and (f) 20-NL ($\mathcal{C}=2$) slabs.

In addition, we identified intrinsic magnetic topological phases beyond the
aforementioned dynamic AxIs. Figure 8 shows the topological properties of
type-I NiVBi2Te5, a XYBi2Te5 compound with FM ground state and a high Curie
temperature of $\sim$152 K [see Appendix C]. The band structure is
demonstrated in Fig. 8(a), and one pair of Weyl points (WPs) emerge along the
-$A-\Gamma-A$ line, with one of them shown in the inset. Our surface state
calculations confirm that one pair of WPs exist due to $\mathcal{T}$-symmetry
breaking. Figure 8(b) clearly shows that a Fermi arc connects two WPs that are
symmetrically located along the -$A-\Gamma-A$ on the (100) surface plane. Two
WPs demonstrate opposite chirality by checking the motions of the sum of
Wannier charge centers (WCCs) [see Fig. 8(c)], indicating that type-I
NiVBi2Te5 exhibits a intrinsic WSM state in bulk under zero external magnetic
field. The strong magnetic coupling makes NiVBi2Te5 a charming magnetic WSM
with high enough Curie temperature (up to 152 K). Beyond type-I NiVBi2Te5,
intrinsic high-temperature WSM phase can also appear in type-III NiVBi2Te5.
See Appendix E for more discussions.

The intrinsic bulk WSM phase in type-I (also type-III) NiVBi2Te5 indicates
that its thin films can host Chern insulator phases with Chern number growing
with increasing thickness [62]. Figure. 8(d) shows the evolution of the gap
size and the Chern number with thickness, which were calculated using
maximally localized Wannier function (MLWF) based tight-binding models. A
type-I NiVBi2Te5 film is a normal insulator below 6-NL, Chern insulator with
$\mathcal{C}=1$ between 6-NL and 16-NL, and high Chern insulator
($\mathcal{C}\geqslant 2$) above 16-NL. The relatively short distance between
the two WPs leads to slow growth of Chern number, compared to that of FM state
MnBi2Te4 [30]. Naturally, the slab band gaps experience a close and reopen
process every time the Chern number changes. In the Chern insulator region,
which supports QAHE, the full gap reaches its maximum value of 19.6 meV in
10-NL slabs, corresponding to 227 K. Figures 8(e) and 8(f) demonstrate the
calculated chiral edge states of 6-NL and 20-NL respectively, validating the
Chern insulator phases with Chern number $\mathcal{C}=1$ and $\mathcal{C}=2$
respectively.

In order to further understand the origin of WSM in bulk phase NiVBi2Te5, and
also the tunability of different topological phases, we investigated the
topological properties of NiVBi2Te5 under hydrostatic pressures from -1.0 GPa
to +1.0 GPa, where ambient pressure is labeled as 0.0 GPa and experimentally
unavailable negative pressure conditions were adopted to complete the study of
phase transition. We studied the electronic structures of bulk NiVBi2Te5 with
out-of-plane FM magnetization. This magnetic configuration is exactly the
magnetic ground state of type-I and type-III NiVBi2Te5. It can also be
achieved in type-II and type-IV NiVBi2Te5 if their AFM interlayer couplings
are flipped under strong enough external magnetic field.

Figure 9: Phase diagrams of out-of-plane FM NiVBi2Te5 obtained by tuning
external pressures. The orange, green and blue zones stand for NI, WSM and MTI
phase, respectively.

Three topologically distinguishable types of pressure-induced topological
phases are identified in Fig. 9, including normal insulator (NI), WSM and
magnetic topological insulator (MTI). Different phases of bulk NiVBi2Te5
corresponds to different 2D phases in thin films. As the thin films grow
thicker, the Chern number of NIs fixes at zero and that of WSMs grows as
analyzed in Fig. 8(d), while that of MTI jumps from zero to one at certain
thickness and remains unchanged afterwards. In the pressure range discussed
here, all types of NiVBi2Te5 behaves like NI at low pressure, evolves into WSM
near the ambient pressure region and steps into MTI eventually at high
pressure.

Theoretically there have been well-established phase diagrams of NI-
WSM(QAH)-MTI transition based on magnetically doped NI-TI superlattices [63,
64]. In these models, intralayer coupling strength $\Delta_{S}$ between top
and bottom surface states (SSs), interlayer coupling strength $\Delta_{D}$
between bottom and top SSs of neighboring layers, and Zeeman splitting denoted
as $m$ decide the final topological character. Increasing (Decreasing) the
external pressure can shrink (enlarge) the ratio between $\Delta_{S}$ and
$\Delta_{D}$, leading to phase transitions. Besides, symmetries play a crucial
rule, and the basic model in Ref. [63] requires the presence of inversion
symmetry $\mathcal{P}$. Once $\mathcal{P}$ is broken, two extra parameters
including the electrostatic potential difference $V$ between bottom and top
SSs, and Dresselhaus-like SOC interaction $\lambda$, should be considered to
handle the symmetry change [64].

For NiVBi2Te5, $\mathcal{P}$ is conserved in type-II and type-IV structures,
while broken in type-I and type-III structures. However, Ni and V atoms share
very similar size and thus SOC strength, leading to ignorable $\lambda$. Based
on Ref. [64], the phase diagram remains almost unchanged as long as we replace
$m$ with an effective Zeeman splitting parameter $\sqrt{m^{2}-V^{2}}$.
Therefore, it’s reasonable that all these four compounds can experience a
phase transition from NI to WSM and MTI under different external pressure.

##  IV conclusions

In conclusion, we systematically study the structural, magnetic and
topological properties of XYBi2Te5-family materials. Interatomic-layer
exchange couplings play a crucial role in keeping magnetic order of some
compounds above 77 K (several compounds even above 150 K), while interlayer
couplings and hybridization between top and bottom surfaces determine the band
topology. We find type-I(III) NiVBi2Te5 to be an emergent material with high
Chern number and possible high-temperature ($\sim$152 K) QAHE state and type-I
Ni2Bi2Te5 as another candidate demonstrating above-room-temperature AFM order.
Ni2Bi2Te5, as well as some other kinds of XYBi2Te5 like type-II MnNiBi2Te5,
type-II NiVBi2Te5 and type-IV NiEuBi2Te5, also demonstrates nontrivial dynamic
axion states. The surface state gaps of, for example, type-II MnNiBi2Te5
($\sim$17 meV) can be large enough for high-temperature explorations. Under
external pressure or magnetic field, possible high-temperature WSM and QAH
phases can also be tunable.

###### Acknowledgements.

We thank Boxuan Li for helpful discussions and Jiaheng Li for technical
support. This work was supported by the National Natural Science Foundation of
China (92065206). X.-Y.T. and Z.L. contributed equally to this work.

Figure 10: Illustration of X-Y mixing using monolayer type-I MnNiBi2Te5 as an
example. (a) Pristine lattice structure with only Ni (AtomicLayer-1) and Mn
(AtomicLayer-2) atomic layers plotted. (b-f) Five considered lattice
structures with X-Y mixing are denoted as M1-M5. Only the two magnetic atomic
layers are plotted for clarity. We marked the unit cells inside every layer as
thin dashed lines. (g-i) Schematic first Brillouin zones of M1-M2, M3 and
M4-M5. Figure 11: Phonon dispersions of lattice structures with X-Y mixing.
Figure 12: $C_{V}$-$T$ curves of (a) type-I and type-III Mn2Bi2Te5 and (b)
type-I and type-III NiVBi2Te5.

##  Appendix A LATTICE CONSTANTS OF MONOLAYER AND BULK XYBi2Te5

The lattice constants are almost the same under different magnetic
configurations. Thus we only demonstrate the data calculated at magnetic
ground states. Detailed discussions on intrinsic magnetism of XYBi2Te5 can be
found in Sec. III.2. For Mn2Bi2Te5, all presented data in this paper are based
on theoretical lattice constants. The difference between theoretical and
reported experimental lattice constants [39] of bulk Mn2Bi2Te5 is less than
1%.

Table 3: Lattice constants of all ten kinds of XYBi2Te5 combinations. Here $a$ and $c$ refer to in-plane and out-of-plane lattice constants individually. Both are in unit of Å. X-Y | type-I | type-II | type-III | type-IV
---|---|---|---|---
Mn-Mn | monolayer | $a$ | 4.338 |  | 4.333 |
| bulk | $a$ | 4.310 |  | 4.307 |
|  | $c$ | 33.951 |  | 17.045 |
Ni-Ni | monolayer | $a$ | 4.222 |  | 4.189 |
| bulk | $a$ | 4.289 |  | 4.243 |
|  | $c$ | 32.393 |  | 16.423 |
V-V | monolayer | $a$ | 4.337 |  | 4.325 |
| bulk | $a$ | 4.315 |  | 4.307 |
|  | $c$ | 33.377 |  | 16.819 |
Eu-Eu | monolayer | $a$ | 4.537 |  | 4.549 |
| bulk | $a$ | 4.519 |  | 4.532 |
|  | $c$ | 35.864 |  | 17.734 |
Mn-Ni | monolayer | $a$ | 4.276 |  | 4.257 |
| bulk | $a$ | 4.285 | 4.285 | 4.261 | 4.259
|  | $c$ | 33.196 | 33.256 | 16.718 | 33.452
Mn-V | monolayer | $a$ | 4.336 |  | 4.323 |
| bulk | $a$ | 4.311 | 4.311 | 4.301 | 4.302
|  | $c$ | 33.771 | 33.786 | 16.994 | 33.994
Ni-V | monolayer | $a$ | 4.283 |  | 4.258 |
| bulk | $a$ | 4.263 | 4.262 | 4.265 | 4.266
|  | $c$ | 32.925 | 33.113 | 16.697 | 33.393
Mn-Eu | monolayer | $a$ | 4.405 |  | 4.438 |
| bulk | $a$ | 4.400 | 4.405 | 4.419 | 4.420
|  | $c$ | 35.049 | 35.071 | 17.404 | 34.830
Ni-Eu | monolayer | $a$ | 4.363 |  | 4.362 |
| bulk | $a$ | 4.333 | 4.347 | 4.355 | 4.356
|  | $c$ | 34.646 | 34.602 | 17.175 | 34.340
V-Eu | monolayer | $a$ | 4.499 |  | 4.427 |
| bulk | $a$ | 4.398 | 4.398 | 4.409 | 4.355
|  | $c$ | 34.938 | 34.907 | 17.326 | 34.472

##  Appendix B DISCUSSIONS ON XYBi2Te5 WITH X-Y MIXING

We considered the situations where X and Y atoms (X$\neq$Y) are mixed instead
of locating at separate atomic layers. We showed that for some combinations of
X and Y, X-Y mixing is neither energetically favored nor structurally stable.
For the others, despite X-Y mixing leads to energy gain, the structural
instability remains.

Taking monolayer XYBi2Te5 as an example, we constructed five representative
structures with X-Y mixing, denoted as M1-M5 [see Figs. 10(b-f)]. Each
magnetic atomic layer in M1(M2) shows stripy structures. The only difference
between M1 and M2 is the interatomic-layer stacking order. M3 simulates the
mixing problem using a $\sqrt{3}\times\sqrt{3}$ supercell. M4 possesses a
zigzag structure while M5 is derived from M4 but with a different mixing ratio
of 3:1 in each single layer. Note that the mixed structures are all based on
monolayer ground-state lattice structures, which means we selected type-I
MnNiBi2Te5, MnVBi2Te5 and NiVBi2Te5 but type-III MnEuBi2Te5, NiEuBi2Te5 and
VEuBi2Te5 instead.

Table 4: Energy comparisons between mixed (M1-M5) and pristine structures of monolayer XYBi2Te5 (X$\neq$Y). The ground state energies of pristine structures (without mixing) are set as zero points and all values are in unit of meV. Negative (positive) values refer to occasions where mixing leads to energy gain (cost). All these values have been averaged to cells as large as original primitive cells (including one X and one Y) for better comparison. X-Y | M1 | M2 | M3 | M4 | M5
---|---|---|---|---|---
Mn-Ni | 55.08 | 60.07 | 64.47 | 60.68 | 48.71
Mn-V | -5.01 | -18.57 | -9.46 | -11.19 | -5.98
Ni-V | -71.15 | -73.78 | -59.87 | -68.99 | -52.66
Mn-Eu | 134.80 | 4.89 | 64.08 | 74.12 | 37.05
Ni-Eu | 114.92 | 82.43 | 109.42 | 55.33 | 102.38
V-Eu | 230.44 | 27.26 | 111.52 | 147.97 | 78.15

We calculated the magnetic ground state energies of different mixed
structures. Similar to calculations in Sec. III.2, we also used collinear
calculations and ignored SOC. The data are summarized in Table 4. As we can
see, the ground state energy may be further reduced in MnVBi2Te5 and NiVBi2Te5
once Mn(Ni) and V atoms are mixed, while in other XYBi2Te5 considered here,
separate X and Y atomic layers are energetically favored.

Then we studied the phonon dispersions of mixed structures [see Fig. 11]. We
found that no matter how the X and Y atoms are mixed, significant virtual
frequency across the first Brillouin zone (FBZ) emerges, indicating highly
unstable structures. Specifically, the appearance of virtual frequency is not
sensitive to the interatomic-layer stacking order, but highly related to the
in-plane direction along which mixing happens [see M1 and M2 in Fig. 11]. The
latter conclusion can be made when the phonon dispersions along $\Gamma-X$ and
$\Gamma-Y$ are compared with each other in cases of M1 and M2. Therefore, we
think X-Y mixing should not be a severe problem theoretically and XYBi2Te5
materials are supposed to inherently establish separate magnetic atomic
layers.

##  Appendix C SUPPLEMENTARY $C_{V}$-$T$ CURVES of Mn2Bi2Te5 and NiVBi2Te5

Extra $C_{V}$-$T$ curves of Mn2Bi2Te5 and NiVBi2Te5 are demonstrated in Fig.
12. The Curie temperatures of type-I and type-III NiVBi2Te5 (152 K and 134 K
respectively) are also well above 77 K, firmly supporting the description of
high-temperature WSM in the main text. We also tested the influence of SOC on
the results using type-III NiVBi2Te5, type-I and type-III Mn2Bi2Te5 as
examples. Typically, the introduction of SOC leads to a slight increase in
magnetic critical temperature. As for type-III Mn2Bi2Te5, since the estimated
Néel temperature is not high and close to experimental results, we decided to
consider an extra next-next-nearest intralayer coupling term in our simulation
and adopted experimental lattice in the calculation of exchange constants. As
can be seen in Fig 12(a), its weak interatomic-layer $E_{ex}$ leads to a Néel
temperature of 10 K when SOC is ignored, while 27 K when SOC is considered,
which is much closer to the experimental result than type-I Mn2Bi2Te5.

##  Appendix D BAND INVERSION FEATURES AND SURFACE STATES OF DYNAMIC AXION
INSULATORS

Figure 13: Band structures and surface states of (a-c) type-II NiVBi2Te5 and
(d-f) type-IV NiEuBi2Te5. (a,d) Band structures with SOC. (b,e) Band
structures without SOC. (c,f) Surface states on (001) and (1$\bar{1}$0)
planes.

The bulk band structures and also surface states of type-II NiVBi2Te5 and
type-IV NiEuBi2Te5 are shown in Fig. 13. Clearly, the band inversion features
and gapped surface states can be found out. Combining these characters with
information provided in Fig. 7, we can conclude that these two kinds of
XYBi2Te5 are also dynamic AxIs.

##  Appendix E WEYL SEMIMETAL AND CHERN INSULATOR PHASES OF TYPE-III NiVBi2Te5

Figure 14: Topological features of type-III NiVBi2Te5 in bulk and thin films.
(a) Band structure of WSM type-III NiVBi2Te5. Details of projected band near
one of the WPs are shown in the inset. (b) Surface states on the (100) plane,
showing a Fermi arc connecting the two WPs. (c) The motions of the sum of WCCs
on spheres surrounding each WP in the momentum space. (d) The evolution of
band gap and Chern number with thickness of type-III NiVBi2Te5 slab. Chiral
edge states of (e) 6-NL ($\mathcal{C}=1$) and (f) 20-NL ($\mathcal{C}=2$)
slabs.

Similar to type-I NiVBi2Te5, type-III NiVBi2Te5 also has a FM ground state and
behaves as intrinsic magnetic WSM. Figure 14 summarizes the overall
topological properties of type-III NiVBi2Te5 both in its 3D bulk phase and 2D
thin films.

## References

  * Hasan and Kane [2010] M. Z. Hasan and C. L. Kane, Colloquium: Topological insulators, Rev. Mod. Phys. 82, 3045 (2010).
  * Qi and Zhang [2011] X.-L. Qi and S.-C. Zhang, Topological insulators and superconductors, Rev. Mod. Phys. 83, 1057 (2011).
  * Armitage  _et al._ [2018] N. P. Armitage, E. J. Mele, and A. Vishwanath, Weyl and Dirac semimetals in three-dimensional solids, Rev. Mod. Phys. 90, 015001 (2018).
  * Tokura  _et al._ [2019] Y. Tokura, K. Yasuda, and A. Tsukazaki, Magnetic topological insulators, Nat. Rev. Phys. 1, 126 (2019).
  * Sekine and Nomura [2021] A. Sekine and K. Nomura, Axion electrodynamics in topological materials, J. Appl. Phys. 129, 141101 (2021).
  * Chang  _et al._ [2023] C.-Z. Chang, C.-X. Liu, and A. H. MacDonald, Colloquium: Quantum anomalous Hall effect, Rev. Mod. Phys. 95, 011002 (2023).
  * Haldane [1988] F. D. M. Haldane, Model for a Quantum Hall Effect without Landau Levels: Condensed-Matter realization of the ”Parity Anomaly”, Phys. Rev. Lett. 61, 2015 (1988).
  * Yu  _et al._ [2010] R. Yu, W. Zhang, H.-J. Zhang, S.-C. Zhang, X. Dai, and Z. Fang, Quantized Anomalous Hall Effect in Magnetic Topological Insulators, Science 329, 61 (2010).
  * Chang  _et al._ [2013] C.-Z. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, Z.-Q. Ji, Y. Feng, S. Ji, X. Chen, J. Jia, X. Dai, Z. Fang, S.-C. Zhang, K. He, Y. Wang, L. Lu, X.-C. Ma, and Q.-K. Xue, Experimental Observation of the Quantum Anomalous Hall Effect in a Magnetic Topological Insulator, Science 340, 167 (2013).
  * Mogi  _et al._ [2015] M. Mogi, R. Yoshimi, A. Tsukazaki, K. Yasuda, Y. Kozuka, K. S. Takahashi, M. Kawasaki, and Y. Tokura, Magnetic modulation doping in topological insulators toward higher-temperature quantum anomalous hall effect, Appl. Phys. Lett. 107, 182401 (2015).
  * Chang  _et al._ [2015] C.-Z. Chang, W. Zhao, D. Y. Kim, H. Zhang, B. A. Assaf, D. Heiman, S.-C. Zhang, C. Liu, M. H. W. Chan, and J. S. Moodera, High-precision realization of robust quantum anomalous Hall state in a hard ferromagnetic topological insulator, Nat. Mater. 14, 473 (2015).
  * Qi  _et al._ [2008] X.-L. Qi, T. L. Hughes, and S.-C. Zhang, Topological field theory of time-reversal invariant insulators, Phys. Rev. B 78, 195424 (2008).
  * Li  _et al._ [2010] R. Li, J. Wang, X.-L. Qi, and S.-C. Zhang, Dynamical axion field in topological magnetic insulators, Nat. Phys. 6, 284 (2010).
  * Wang  _et al._ [2015] J. Wang, B. Lian, X.-L. Qi, and S.-C. Zhang, Quantized topological magnetoelectric effect of the zero-plateau quantum anomalous Hall state, Phys. Rev. B 92, 081107(R) (2015).
  * Morimoto  _et al._ [2015] T. Morimoto, A. Furusaki, and N. Nagaosa, Topological magnetoelectric effects in thin films of topological insulators, Phys. Rev. B 92, 085113 (2015).
  * Mogi  _et al._ [2017] M. Mogi, M. Kawamura, R. Yoshimi, A. Tsukazaki, Y. Kozuka, N. Shirakawa, K. S. Takahashi, M. Kawasaki, and Y. Tokura, A magnetic heterostructure of topological insulators as a candidate for an axion insulator, Nat. Mater. 16, 516 (2017).
  * Xiao  _et al._ [2018] D. Xiao, J. Jiang, J.-H. Shin, W. Wang, F. Wang, Y.-F. Zhao, C. Liu, W. Wu, M. H. W. Chan, N. Samarth, and C.-Z. Chang, Realization of the Axion Insulator State in Quantum Anomalous Hall Sandwich Heterostructures, Phys. Rev. Lett. 120, 056801 (2018).
  * Wan  _et al._ [2011] X. Wan, A. M. Turner, A. Vishwanath, and S. Y. Savrasov, Topological semimetal and fermi-arc surface states in the electronic structure of pyrochlore iridates, Phys. Rev. B 83, 205101 (2011).
  * Liu  _et al._ [2019] D. F. Liu, A. J. Liang, E. K. Liu, Q. N. Xu, Y. W. Li, C. Chen, D. Pei, W. J. Shi, S. K. Mo, P. Dudin, T. Kim, C. Cacho, G. Li, Y. Sun, L. X. Yang, Z. K. Liu, S. S. P. Parkin, C. Felser, and Y. L. Chen, Magnetic Weyl semimetal phase in a Kagomé crystal, Science 365, 1282 (2019).
  * Liu  _et al._ [2018] E. Liu, Y. Sun, N. Kumar, L. Muechler, A. Sun, L. Jiao, S.-Y. Yang, D. Liu, A. Liang, Q. Xu, J. Kroder, V. Süß, H. Borrmann, C. Shekhar, Z. Wang, C. Xi, W. Wang, W. Schnelle, S. Wirth, Y. Chen, S. T. B. Goennenwein, and C. Felser, Giant anomalous Hall effect in a ferromagnetic kagome-lattice semimetal, Nat. Phys. 14, 1125 (2018).
  * Xu  _et al._ [2019] Y. Xu, Z. Song, Z. Wang, H. Weng, and X. Dai, Higher-Order Topology of the Axion Insulator EuIn2As2, Phys. Rev. Lett. 122, 256402 (2019).
  * Li  _et al._ [2020a] Y. Li, J. Li, Y. Li, M. Ye, F. Zheng, Z. Zhang, J. Fu, W. Duan, and Y. Xu, High-Temperature Quantum Anomalous Hall Insulators in Lithium-Decorated Iron-Based Superconductor Materials, Phys. Rev. Lett. 125, 086401 (2020a).
  * You  _et al._ [2019] J.-Y. You, Z. Zhang, B. Gu, and G. Su, Two-Dimensional Room-Temperature Ferromagnetic Semiconductors with Quantum Anomalous Hall Effect, Phys. Rev. Applied 12, 024063 (2019).
  * Li  _et al._ [2022] Z. Li, Y. Han, and Z. Qiao, Chern Number Tunable Quantum Anomalous Hall Effect in Monolayer Transitional Metal Oxides via Manipulating Magnetization Orientation, Phys. Rev. Lett. 129, 036801 (2022).
  * Gong  _et al._ [2019] Y. Gong, J. Guo, J. Li, K. Zhu, M. Liao, X. Liu, Q. Zhang, L. Gu, L. Tang, X. Feng, D. Zhang, W. Li, C. Song, L. Wang, P. Yu, X. Chen, Y. Wang, H. Yao, W. Duan, Y. Xu, S.-C. Zhang, X. Ma, Q.-K. Xue, and K. He, Experimental Realization of an Intrinsic Magnetic Topological Insulator, Chin. Phys. Lett. 36, 076801 (2019).
  * Otrokov  _et al._ [2019a] M. M. Otrokov, I. I. Klimovskikh, H. Bentmann, D. Estyunin, A. Zeugner, Z. S. Aliev, S. Gaß, A. U. B. Wolter, A. V. Koroleva, A. M. Shikin, M. Blanco-Rey, M. Hoffmann, I. P. Rusinov, A. Y. Vyazovskaya, S. V. Eremeev, Y. M. Koroteev, V. M. Kuznetsov, F. Freyse, J. Sánchez-Barriga, I. R. Amiraslanov, M. B. Babanly, N. T. Mamedov, N. A. Abdullayev, V. N. Zverev, A. Alfonsov, V. Kataev, B. Büchner, E. F. Schwier, S. Kumar, A. Kimura, L. Petaccia, G. Di Santo, R. C. Vidal, S. Schatz, K. Kißner, M. Ünzelmann, C. H. Min, S. Moser, T. R. F. Peixoto, F. Reinert, A. Ernst, P. M. Echenique, A. Isaeva, and E. V. Chulkov, Prediction and observation of an antiferromagnetic topological insulator, Nature 576, 416 (2019a).
  * Li  _et al._ [2019a] J. Li, Y. Li, S. Du, Z. Wang, B.-L. Gu, S.-C. Zhang, K. He, W. Duan, and Y. Xu, Intrinsic magnetic topological insulators in van der Waals layered MnBi2Te4-family materials, Sci. Adv. 5, eaaw5685 (2019a).
  * Li  _et al._ [2019b] J. Li, C. Wang, Z. Zhang, B.-L. Gu, W. Duan, and Y. Xu, Magnetically controllable topological quantum phase transitions in the antiferromagnetic topological insulator ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$, Phys. Rev. B 100, 121103(R) (2019b).
  * Deng  _et al._ [2020] Y. Deng, Y. Yu, M. Z. Shi, Z. Guo, Z. Xu, J. Wang, X. H. Chen, and Y. Zhang, Quantum anomalous Hall effect in intrinsic magnetic topological insulator MnBi2Te4, Science 367, 895 (2020).
  * Ge  _et al._ [2020] J. Ge, Y. Liu, J. Li, H. Li, T. Luo, Y. Wu, Y. Xu, and J. Wang, High-Chern-number and high-temperature quantum Hall effect without Landau levels, Natl. Sci. Rev. 7, 1280 (2020).
  * Otrokov  _et al._ [2019b] M. M. Otrokov, I. P. Rusinov, M. Blanco-Rey, M. Hoffmann, A. Y. Vyazovskaya, S. V. Eremeev, A. Ernst, P. M. Echenique, A. Arnau, and E. V. Chulkov, Unique Thickness-Dependent Properties of the van der Waals Interlayer Antiferromagnet MnBi2Te4 Films, Phys. Rev. Lett. 122, 107202 (2019b).
  * Zhang  _et al._ [2019] D. Zhang, M. Shi, T. Zhu, D. Xing, H. Zhang, and J. Wang, Topological Axion States in the Magnetic Insulator MnBi2Te4 with the Quantized Magnetoelectric Effect, Phys. Rev. Lett. 122, 206401 (2019).
  * Liu  _et al._ [2020] C. Liu, Y. Wang, H. Li, Y. Wu, Y. Li, J. Li, K. He, Y. Xu, J. Zhang, and Y. Wang, Robust axion insulator and Chern insulator phases in a two-dimensional antiferromagnetic topological insulator, Nat. Mater. 19, 522 (2020).
  * Mermin and Wagner [1966] N. D. Mermin and H. Wagner, Absence of Ferromagnetism or Antiferromagnetism in One- or Two-Dimensional Isotropic Heisenberg Models, Phys. Rev. Lett. 17, 1133 (1966).
  * Huang  _et al._ [1994] F. Huang, M. T. Kief, G. J. Mankey, and R. F. Willis, Magnetism in the few-monolayers limit: A surface magneto-optic Kerr-effect study of the magnetic behavior of ultrathin films of Co, Ni, and Co-Ni alloys on Cu(100) and Cu(111), Phys. Rev. B 49, 3962 (1994).
  * Zhang  _et al._ [2020] J. Zhang, D. Wang, M. Shi, T. Zhu, H. Zhang, and J. Wang, Large Dynamical Axion Field in Topological Antiferromagnetic Insulator Mn2Bi2Te5, Chin. Phys. Lett. 37, 077304 (2020).
  * Li  _et al._ [2020b] Y. Li, Y. Jiang, J. Zhang, Z. Liu, Z. Yang, and J. Wang, Intrinsic topological phases in ${\mathrm{Mn}}_{2}{\mathrm{Bi}}_{2}{\mathrm{Te}}_{5}$ tuned by the layer magnetization, Phys. Rev. B 102, 121107(R) (2020b).
  * Eremeev  _et al._ [2022] S. V. Eremeev, M. M. Otrokov, A. Ernst, and E. V. Chulkov, Magnetic ordering and topology in ${\mathrm{Mn}}_{2}{\mathrm{Bi}}_{2}{\mathrm{Te}}_{5}$ and ${\mathrm{Mn}}_{2}{\mathrm{Sb}}_{2}{\mathrm{Te}}_{5}$ van der Waals materials, Phys. Rev. B 105, 195105 (2022).
  * Cao  _et al._ [2021] L. Cao, S. Han, Y.-Y. Lv, D. Wang, Y.-C. Luo, Y.-Y. Zhang, S.-H. Yao, J. Zhou, Y. B. Chen, H. Zhang, and Y.-F. Chen, Growth and characterization of the dynamical axion insulator candidate ${\mathrm{Mn}}_{2}{\mathrm{Bi}}_{2}{\mathrm{Te}}_{5}$ with intrinsic antiferromagnetism, Phys. Rev. B 104, 054421 (2021).
  * Zhang  _et al._ [2022] H. Zhang, J. Zhang, Y. Zhang, W. Yang, Y. Wang, X. Xu, and F. Liu, A generic dual d-band model for interlayer ferromagnetic coupling in a transition-metal doped MnBi2Te4 family of materials, Nanoscale 14, 13689 (2022).
  * Kresse and Furthmüller [1996] G. Kresse and J. Furthmüller, Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set, Phys. Rev. B 54, 11169 (1996).
  * Perdew  _et al._ [1996] J. P. Perdew, K. Burke, and M. Ernzerhof, Generalized Gradient Approximation Made Simple, Phys. Rev. Lett. 77, 3865 (1996).
  * Becke and Johnson [2006] A. D. Becke and E. R. Johnson, A simple effective potential for exchange, J. Chem. Phys. 124, 221101 (2006).
  * Grimme  _et al._ [2010] S. Grimme, J. Antony, S. Ehrlich, and H. Krieg, A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu, J. Chem. Phys. 132, 154104 (2010).
  * Wang  _et al._ [2021] V. Wang, N. Xu, J.-C. Liu, G. Tang, and W.-T. Geng, VASPKIT: A user-friendly interface facilitating high-throughput computing and analysis using VASP code, Comput. Phys. Commun. 267, 108033 (2021).
  * Togo and Tanaka [2015] A. Togo and I. Tanaka, First principles phonon calculations in materials science, Scr. Mater. 108, 1 (2015).
  * Momma and Izumi [2011] K. Momma and F. Izumi, VESTA3 for three-dimensional visualization of crystal, volumetric and morphology data, J. Appl. Crystallogr. 44, 1272 (2011).
  * Mostofi  _et al._ [2014] A. A. Mostofi, J. R. Yates, G. Pizzi, Y.-S. Lee, I. Souza, D. Vanderbilt, and N. Marzari, An updated version of wannier90: A tool for obtaining maximally-localised Wannier functions, Comput. Phys. Commun. 185, 2309 (2014).
  * Marzari and Vanderbilt [1997] N. Marzari and D. Vanderbilt, Maximally localized generalized Wannier functions for composite energy bands, Phys. Rev. B 56, 12847 (1997).
  * Souza  _et al._ [2001] I. Souza, N. Marzari, and D. Vanderbilt, Maximally localized Wannier functions for entangled energy bands, Phys. Rev. B 65, 035109 (2001).
  * Wu  _et al._ [2018] Q. Wu, S. Zhang, H.-F. Song, M. Troyer, and A. A. Soluyanov, Wanniertools: An open-source software package for novel topological materials, Comput. Phys. Commun. 224, 405 (2018).
  * Eremeev  _et al._ [2018] S. V. Eremeev, M. M. Otrokov, and E. V. Chulkov, New Universal Type of Interface in the Magnetic Insulator/Topological Insulator Heterostructures, Nano Lett. 18, 6521 (2018).
  * Hirahara  _et al._ [2020] T. Hirahara, M. M. Otrokov, T. T. Sasaki, K. Sumida, Y. Tomohiro, S. Kusaka, Y. Okuyama, S. Ichinokura, M. Kobayashi, Y. Takeda, K. Amemiya, T. Shirasawa, S. Ideta, K. Miyamoto, K. Tanaka, S. Kuroda, T. Okuda, K. Hono, S. V. Eremeev, and E. V. Chulkov, Fabrication of a novel magnetic topological heterostructure and temperature evolution of its massive Dirac cone, Nat. Commun. 11, 4821 (2020).
  * Jain  _et al._ [2013] A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, and K. A. Persson, Commentary: The Materials Project: A materials genome approach to accelerating materials innovation, APL Mater. 1, 011002 (2013).
  * Han  _et al._ [2021] Y. Han, S. Sun, S. Qi, X. Xu, and Z. Qiao, Interlayer ferromagnetism and high-temperature quantum anomalous Hall effect in $p$-doped $\mathrm{Mn}{\mathrm{Bi}}_{2}{\mathrm{Te}}_{4}$ multilayers, Phys. Rev. B 103, 245403 (2021).
  * Tan  _et al._ [2021] H. Tan, Y. Liu, Z. Wang, and B. Yan, Charge density waves and electronic properties of superconducting kagome metals, Phys. Rev. Lett. 127, 046401 (2021).
  * Pavarini  _et al._ [2012] E. Pavarini, E. Koch, F. Anders, and M. Jarrell,  _Correlated Electrons: From Models to Materials_ (2012).
  * Li  _et al._ [2020c] Z. Li, J. Li, K. He, X. Wan, W. Duan, and Y. Xu, Tunable interlayer magnetism and band topology in van der Waals heterostructures of MnBi2Te4-family materials, Phys. Rev. B 102, 081107(R) (2020c).
  * Zhang  _et al._ [2009] H. Zhang, C.-X. Liu, X.-L. Qi, X. Dai, Z. Fang, and S.-C. Zhang, Topological insulators in Bi2Se3, Bi2Te3 and Sb2Te3 with a single Dirac cone on the surface, Nat. Phys. 5, 438 (2009).
  * Liu  _et al._ [2010] C.-X. Liu, X.-L. Qi, H.-J. Zhang, X. Dai, Z. Fang, and S.-C. Zhang, Model hamiltonian for topological insulators, Phys. Rev. B 82, 045122 (2010).
  * Wang  _et al._ [2020] H. Wang, D. Wang, Z. Yang, M. Shi, J. Ruan, D. Xing, J. Wang, and H. Zhang, Dynamical axion state with hidden pseudospin Chern numbers in ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$-based heterostructures, Phys. Rev. B 101, 081109(R) (2020).
  * Xu  _et al._ [2011] G. Xu, H. Weng, Z. Wang, X. Dai, and Z. Fang, Chern Semimetal and the Quantized Anomalous Hall Effect in ${\mathrm{HgCr}}_{2}{\mathrm{Se}}_{4}$, Phys. Rev. Lett. 107, 186806 (2011).
  * Burkov and Balents [2011] A. A. Burkov and L. Balents, Weyl Semimetal in a Topological Insulator Multilayer, Phys. Rev. Lett. 107, 127205 (2011).
  * Zyuzin  _et al._ [2012] A. A. Zyuzin, S. Wu, and A. A. Burkov, Weyl semimetal with broken time reversal and inversion symmetries, Phys. Rev. B 85, 165110 (2012).


# CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

Peter Polák1 Ngoc-Quan Ngoc2 Tuan-Nam Nguyen2 Danni Liu3 Carlos Mullov2 Jan
Niehues2 Ondřej Bojar1 Alexander Waibel2,4
<EMAIL_ADDRESS>
1 Charles University
2 Karlsruhe Institute of Technology
3 Maastricht University
4 Carnegie Mellon University

###### Abstract

In this paper, we describe our submission to the Simultaneous Speech
Translation at IWSLT 2022. We explore strategies to utilize an offline model
in a simultaneous setting without the need to modify the original model. In
our experiments, we show that our onlinization algorithm is almost on par with
the offline setting while being $3\times$ faster than offline in terms of
latency on the test set. We also show that the onlinized offline model
outperforms the best IWSLT2021 simultaneous system in medium and high latency
regimes and is almost on par in the low latency regime. We make our system
publicly available.111https://hub.docker.com/repository/docker/polape7/cuni-
kit-simultaneous

##  1 Introduction

This paper describes the CUNI-KIT submission to the Simultaneous Speech
Translation task at IWSLT 2022 (Anastasopoulos et al., 2022) by Charles
University (CUNI) and Karlsruhe Institute of Technology (KIT).

Recent work on end-to-end (E2E) simultaneous speech-to-text translation (ST)
is focused on training specialized models specifically for this task. The
disadvantage is the need of storing an extra model, usually a more difficult
training and inference setup, increased computational complexity (Han et al.,
2020; Liu et al., 2021) and risk of performance degradation if used in offline
setting (Liu et al., 2020a).

In this work, we base our system on a robust multilingual offline ST model
that leverages pretrained wav2vec 2.0 (Baevski et al., 2020) and mBART (Liu et
al., 2020b). We revise the onlinization approach by Liu et al. (2020a) and
propose an improved technique with a fully controllable quality-latency trade-
off. We demonstrate that without any change to the offline model, our
simultaneous system in the mid- and high-latency regimes is on par with the
offline performance. At the same time, the model outperforms previous IWSLT
systems in medium and high latency regimes and is almost on par in the low
latency regime. Finally, we observe a problematic behavior of the average
lagging metric for speech translation (Ma et al., 2020) when dealing with long
hypotheses, resulting in negative values. We propose a minor change to the
metric formula to prevent this behavior.

Our contribution is as follows:

  * •

We revise and generalize onlinization proposed by Liu et al. (2020a); Nguyen
et al. (2021) and discover parameter enabling quality-latency trade-off,

  * •

We demonstrate that one multilingual offline model can serve as simultaneous
ST for three language pairs,

  * •

We demonstrate that an improvement in the offline model leads also to an
improvement in the online regime,

  * •

We propose a change to the average lagging metric that avoids negative values.

##  2 Related Work

Simultaneous speech translation can be implemented either as a (hybrid)
cascaded system (Kolss et al., 2008; Niehues et al., 2016; Elbayad et al.,
2020; Liu et al., 2020a; Bahar et al., 2021) or an end-to-end model (Han et
al., 2020; Liu et al., 2021). Unlike for the offline speech translation where
cascade seems to have the best quality, the end-to-end speech translation
offers a better quality-latency trade-off (Ansari et al., 2020; Liu et al.,
2021; Anastasopoulos et al., 2021).

End-to-end systems use different techniques to perform simultaneous speech
translation. Han et al. (2020) uses wait-$k$ (Ma et al., 2019) model and
metalearning (Indurthi et al., 2020) to alleviate the data scarcity. Liu et
al. (2020a) uses a unidirectional encoder with monotonic cross-attention to
limit the dependence on future context. Other work (Liu et al., 2021) proposes
Cross Attention augmented Transducer (CAAT) as an extension of RNN-T (Graves,
2012).

Nguyen et al. (2021) proposed a hypothesis stability detection for automatic
speech recognition (ASR). The _shared prefix_ strategy finds the longest
common prefix in all beams. Liu et al. (2020a) explore such strategies in the
context of speech recognition and translation. The most promising is the
longest common prefix of two consecutive chunks. The downside of this approach
is the inability to parametrize the quality-latency trade-off. We directly
address this in our work.

##  3 Onlinization

In this section, we describe the onlinization of the offline model and propose
two ways to control the quality-latency trade-off.

###  3.1 Incremental Decoding

Depending on the language pair, translation tasks may require reordering or a
piece of information that might not be apparent until the source utterance
ends. In the offline setting, the model processes the whole utterance at once,
rendering the strategy most optimal in terms of quality. If applied in online
mode, this ultimately leads to a large latency. One approach to reducing the
latency is to break the source utterance into chunks and perform the
translation on each chunk.

In this paper, we follow the incremental decoding framework described by Liu
et al. (2020a). We break the input utterance into small fixed-size chunks and
decode each time after we receive a new chunk. After each decoding step, we
identify a stable part of the hypothesis using _stable hypothesis detection_.
The stable part is sent to the user (“committed” in the following) and is no
longer changed afterward (i.e., no retranslation).222This is a requirement for
the evaluation in the Simultaneous Speech Translation task at IWSLT 2022. Our
current implementation assumes that the whole speech input fits into memory,
in other words, we are only adding new chunks as they are arriving. This
simplification is possible because the evaluation of the shared task is
performed on segmented input, on individual utterances. With each newly
arrived input chunk, the decoding starts with forced decoding of the already
committed tokens and continues with beam search decoding.

###  3.2 Chunk Size

Speech recognition and translation use chunking for simultaneous inference
with various chunk sizes ranging from 300 ms to 2 seconds (Liu, 2020; Nguyen
et al., 2021) although the literature suggests that the turn-taking in
conversational speech is shorter, around 200 ms (Levinson and Torreira, 2015).
We investigate different chunk sizes in combination with various stable
hypothesis detection strategies. As we document later, the chunk size is the
principal factor that controls the quality-latency trade-off.

###  3.3 Stable Hypothesis Detection

Committing hypotheses from incomplete input presents a possible risk of
introducing errors. To reduce the instability and trade time for quality, we
employ a _stable hypothesis detection_. Formally, we define a function
$prefix(W)$ that, given a set of hypotheses (i.e., $W_{all}^{c}$ if we want to
consider the whole beam or $W_{best}^{c}$ for the single best hypothesis
obtained during the beam search decoding of the $c$-th chunk), outputs a
stable prefix. We investigate several functions:

##### Hold-$n$

(Liu et al., 2020a) Hold-$n$ strategy selects the best hypothesis in the beam
and deletes the last $n$ tokens from it:

$\operatorname{prefix}(W_{best}^{c})=W_{0:\max(0,|W|-n)},$ (1)

where $W_{best}^{c}$ is the best hypothesis obtained in the beam search of
$c$-th chunk. If the hypothesis has only $n$ or fewer tokens, we return an
empty string.

##### LA-$n$

Local agreement (Liu et al., 2020a) displays the agreeing prefixes of the two
consecutive chunks. Unlike the hold-$n$ strategy, the local agreement does not
offer any explicit quality-latency trade-off. We generalize the strategy to
take the agreeing prefixes of $n$ consecutive chunks.

During the first $n-1$ chunks, we do not output any tokens. From the $n$-th
chunk on, we identify the longest common prefix of the best hypothesis of the
$n$ consecutive chunks:

$\operatorname{prefix}(W_{best}^{c})=\\\ \begin{cases}\varnothing,&\text{if
$c<n$},\\\
\operatorname{LCP}(W_{best}^{c-n+1},...,W_{best}^{c}),&\text{otherwise},\end{cases}$
(2)

where $LCP(\cdot)$ is longest common prefix of the arguments.

##### SP-$n$

Shared prefix (Nguyen et al., 2021) strategy displays the longest common
prefix of all the items in the beam of a chunk. Similarly to the LA-$n$
strategy, we propose a generalization to the longest common prefix of all
items in the beams of the $n$ consecutive chunks:

$\operatorname{prefix}(W_{all}^{c})=\\\ \begin{cases}\varnothing,&\text{if
$c<n$},\\\ \operatorname{LCP}(W_{\text{beam
$1...B$}}^{c-n+1},...,W_{\text{beam
$1...B$}}^{c}),&\text{otherwise},\end{cases}$ (3)

i.e., all beam hypotheses $1,...,B$ (where $B$ is the beam size) of all chunks
$c-n+1,...,c$.

###  3.4 Initial Wait

The limited context of the early chunks might result in an unstable hypothesis
and an emission of erroneous tokens. The autoregressive nature of the model
might cause further performance degradation in later chunks. One possible
solution is to use longer chunks, but it inevitably leads to a higher latency
throughout the whole utterance. To mitigate this issue, we explore a
lengthening of the first chunk. We call this strategy an initial wait.

##  4 Experiments Setup

In this section, we describe the onlinization experiments.

###  4.1 Evaluation Setup

We use the SimulEval toolkit (Ma et al., 2020). The toolkit provides a simple
interface for evaluation of simultaneous (speech) translation. It reports the
quality metric BLEU (Papineni et al., 2002; Post, 2018) and latency metrics
Average Proportion (AP, Cho and Esipova 2016), Average Lagging (AL, Ma et al.
2019), and Differentiable Average Lagging (DAL, Cherry and Foster 2019)
modified for speech source.

Specifically, we implement an Agent class. We have to implement two important
functions: policy(state) and predict(state), where state is the state of the
agent (e.g., read processed input, emitted tokens, …). The policy function
returns the action of the agent: (1) READ to request more input, (2) WRITE to
emit new hypothesis tokens.

We implement the policy as specified in Algorithm 1. The default action is
READ. If there is a new chunk, we perform the inference and use the
$prefix(W^{c})$ function to find the stable prefix. If there are new tokens to
display (i.e., $|prefix(W^{c})|>|prefix(W^{c-1})|$), we return the WRITE
action. As soon as our agent emits an end-of-sequence (EOS) token, the
inference of the utterance is finished by the SimulEval. We noticed that our
model was emitting the EOS token quite often, especially in the early chunks.
Hence, we ignore the EOS if returned by our model and continue the inference
until the end of the source.333This might cause an unnecessary increase in
latency, but it might be partially prevented by voice activity detection.

Algorithm 1 Policy function

$state$

if $state$.$new\\_input>chunk\\_size$ then

$hypothesis\leftarrow predict(state)$

if $|hypothesis|>0$ then

return $WRITE$

end if

end if

return $READ$

###  4.2 Speech Translation Models

In our experiments, we use two different models. First, we do experiments with
a monolingual _Model A_ , then for the submission, we use a multilingual and
more robust _Model B_.444We also did experiments with a dedicated English-
German model similar to Model B (i.e., based on wav2vec and mBART), but it
performed worse both in offline and online setting compared to the
multilingual version.

Model A is the KIT IWSLT 2020 model for the Offline Speech Translation task.
Specifically, it is an end-to-end English to German Transformer model with
relative attention. For more described description, refer to Pham et al.
(2020b).

####  4.2.1 Multilingual Model

For the submission, we use a multilingual Model B. We construct the SLT
architecture with the encoder based on the wav2vec 2.0 Baevski et al. (2020)
and the decoder based on the autoregressive language model pretrained with
mBART50 Tang et al. (2020).

##### wav2vec 2.0

is a Transformer encoder model which receives raw waveforms as input and
generates high-level representations. The architecture consists of two main
components: first, a convolution-based feature extractor downsamples long
audio waveforms into features that have similar lengths with
spectrograms.After that, a deep Transformer encoder uses self-attention and
feed-forward neural network blocks to transform the features without further
downsampling.

During the self-supervised training process, the network is trained with a
contrastive learning strategy Baevski et al. (2020), in which the already
downsampled features are randomly masked and the model learns to predict the
quantized latent representation of the masked time step.

During the supervised learning step, we freeze the feature extraction weights
to save memory since the first layers are among the largest ones. We fine-tune
all of the weights in the Transformer encoder. Moreover, to make the model
more robust to the fluctuation in absolute positions and durations when it
comes to audio signals,we added the relative position encodings Dai et al.
(2019); Pham et al. (2020a) to alleviate this problem.555This has the added
advantage of better generalization in situations where training and testing
data are segmented differently.

Here we used the same pretrained model with the speech recognizer, with the
large architecture pretrained with $53k$ hours of unlabeled data.

##### mBART50

is an encoder-decoder Transformer-based language model. During training,
instead of the typical language modeling setting of predicting the next word
in the sequence, this model is trained to reconstruct a sequence from its
noisy version Lewis et al. (2019) and later extended to a multilingual version
Liu et al. (2020b); Tang et al. (2020) in which the corpora from multiple
languages are combined during training. mBART50 is the version that is
pretrained on $50$ languages.

The mBART50 model follows the Transformer encoder and decoder Vaswani et al.
(2017). During fine-tuning, we combine the mBART50 decoder with the wav2vec
2.0 encoder, where both encoder and decoder know one modality. The cross-
attention layers connecting the decoder with the encoder are the parts that
require extensive fine-tuning in this case, due to the modality mismatch
between pretraining and fine-tuning.

Finally, we use the model in a multilingual setting, i.e., for English to
Chinese, German, and Japanese language pairs by training on the combination of
the datasets. The mBART50 vocabulary contains language tokens for all three
languages and can be used to control the language output Ha et al. (2016).

For more details on the model refer to Pham et al. (2022).

###  4.3 Test Data

For the onlinization experiments, we use MuST-C (Cattoni et al., 2021) tst-
COMMON from the v2.0 release. We conduct all the experiments on the English-
German language pair.

##  5 Experiments and Results

In this section, we describe the experiments and discuss the results.

###  5.1 Chunks Size

We experiment with chunk sizes of 250 ms, 500 ms, 1s, and 2 s. We combine the
sizes of the chunks with different partial hypothesis selection strategies.
The results are shown in Figure 1.

The results document that the chunk size parameter has a stronger influence on
the trade-off than different prefix strategies. Additionally, this enables
constant trade-off strategies (e.g., LA-2) to become flexible.

00.511.522.53202530250500100020002505001000200050010002000Average Lagging
(seconds)BLEULA-2SP-2Hold-6Offline Figure 1: Quality-latency trade-off of
different chunk sizes combined with different stable hypothesis detection
strategies. The number next to the marks indicates chunk size in milliseconds.

###  5.2 Stable Hypothesis Detection Strategies

We experiment with three strategies: hold-$n$ (withholds last $n$ tokens),
shared prefix (SP-$n$; finds the longest common prefix of all beams in $n$
consecutive chunks) and local agreement (LA-$n$; finds the longest common
prefix of the best hypothesis in $n$ consecutive chunks). For hold-$n$, we
select $n=3,6,12$; for SP-$n$, we select $n=1,2$ ($n=1$ corresponds to the
strategy by Nguyen et al. (2021)); for LA-$n$ we select $n=2,3,4$ ($n=2$
corresponds to the strategy by Liu et al. (2020a)). The results are in Figures
2 and 3.

00.511.522.5315202530361236123612Average Lagging (seconds)BLEU500 ms1000
ms2000 msOffline Figure 2: Quality-latency trade-off of hold-$n$ strategy with
different values of $n$. The number next to the marks indicates $n$. Colored
lines connect results with equal chunk size.
00.511.522.531520253025050010002000250500100020002505001000200025050010002505003612Average
Lagging (seconds)BLEUSP-1SP-2LA-2LA-3LA-4Hold-$n$ 500Offline Figure 3:
Quality-latency trade-off of shared prefix (SP-$n$) and local agreement
(LA-$n$) with different $n$ and chunk size.

##### Hold-$n$

The results suggest (see Figure 2) that the hold-$n$ strategy can use either
$n$ or chunk size to control the quality-latency trade-off with equal effect.
The only exception seems to be too low $n<=3$, which slightly underperforms
the options with higher $n$ and shorter chunk size.

##### Local agreement (LA-$n$)

The local agreement seems to outperform all other strategies (see Figure 3).
LA-$n$ for all $n$ follows the same quality-latency trade-off line. The
advantage of LA-2 is in reduced computational complexity compared to the other
LA-$n$ strategies with $n>2$.

##### Shared prefix (SP-$n$)

SP-1 strongly underperforms other strategies in quality (see Figure 3). While
the SP-1 strategy performs well in the ASR task (Nguyen et al., 2021), it is
probably too lax for the speech translation task. The generalized and more
conservative SP-2 performs much better. Although, the more relaxed LA-2, which
considers only the best item in the beam, has a better quality-latency trade-
off curve than the more conservative SP-2.

###  5.3 Initial Wait

As we could see in Section 5.1, the shorter chunk sizes tend to perform worse.
One of the reasons might be the limited context of the early chunks.666If we
translated a non-pre-segmented input, this problem would be limited only
onetime to the beginning of the input. To increase the early context, we
prolong the first chunk to 2 seconds.

Initial wait | Chunk size | BLEU | AL | AP | DAL
---|---|---|---|---|---
0 | 250 | 16.34 | -35.97 | 0.66 | 1435.06
500 | 25.40 | 727.55 | 0.73 | 1791.21
1000 | 30.29 | 1660.59 | 0.83 | 2662.18
2000 | 250 | 16.60 | 358.35 | 0.74 | 2121.54
500 | 25.42 | 952.15 | 0.77 | 2142.53
1000 | 30.29 | 1654.77 | 0.83 | 2657.48

Table 1: Quality-latency trade-off of the LA-2 strategy with and without the
initial wait.

The results are in Table 1. We see a slight (0.3 BLEU) increase in quality for
a chunk size of 250 ms, though the initial wait does not improve the BLEU and
a considerable increase in the latency.

The performance seems to be influenced mainly by the chunk size. The reason
for smaller chunks’ under-performance might be caused by (1) acoustic
uncertainty towards the end of a chunk (e.g., words often get cut in the
middle), or (2) insufficient information difference between two consecutive
chunks.

This is supported by the observation in Figure 3. Increasing the number of
consecutive chunks (i.e., increasing the context for the decision) considered
in the local agreement strategy (LA-$2,3,4$), improves the quality, while it
adds latency.

###  5.4 Negative Average Lagging

Interestingly, we noticed that some of the strategies achieved negative
average lagging (e.g., LA-2 in Section 5.1) with a chunk size of 250 ms has AL
of -36 ms). After a closer examination of the outputs, we found that the
negative AL is in utterances where the hypothesis is significantly longer than
the reference. Recall the average latency for speech input defined by Ma et
al. (2020):

$\text{AL}_{\text{speech}}=\frac{1}{\tau^{\prime}(|\mathbf{X}|)}\sum_{i=1}^{\tau^{\prime}(|\mathbf{X}|)}d_{i}-d^{*}_{i},$
(4)

where $d_{i}=\sum_{k=1}^{j}T_{k}$, $j$ is the index of raw audio segment that
has been read when generating $y_{i}$, $T_{k}$ is duration of raw audio
segment,
$\tau^{\prime}(|\mathbf{X}|)=\text{min}\\{i|d_{i}=\sum_{j=1}^{|\mathbf{X}|}T_{j}\\}$
and $d^{*}_{i}$ are the delays of an ideal policy:

$d^{*}_{i}=(i-1)\times\sum_{j=1}^{|\mathbf{X}|}T_{j}\ /|\mathbf{Y^{*}}|,$ (5)

where $\mathbf{Y^{*}}$ is reference translation.

If the hypothesis is longer than the reference, then $d^{*}_{i}>d_{i}$, making
the sum argument in Equation 4 negative. On the other hand, if we use the
length of the hypothesis instead, then a shorter hypothesis would
benefit.777Ma et al. (2019) originally used the hypothesis length in the
Equation 5 and then Ma et al. (2020) suggested to use the reference length
instead. We, therefore, suggest using the maximum of both to prevent the
advantage of either a shorter or a longer hypothesis:

$d^{*}_{i}=(i-1)\times\sum_{j=1}^{|\mathbf{X}|}T_{j}\
/max(|\mathbf{Y}|,|\mathbf{Y^{*}}|).$ (6)

##  6 Submitted System

In this section, we describe the submitted system. We follow the allowed
training data and pretrained models and therefore our submission is
_constrained_ (see Section 4.2.1 for model description).

For stable hypothesis detection, we decided to use the local agreement
strategy with $n=2$. As shown in Section 5.2, the LA-2 has the best latency-
quality trade-off along with other LA-$n$ strategies. To achieve the different
latency regimes, we use various chunk sizes, depending on the language pair.
We decided not to use larger $n>2$ to control the latency, as it increases the
computation complexity while having the same effect as using a different chunk
size. The results on MuST-C tst-COMMON are in Table 2. The quality-latency
trade-off is in Figure 4.

Model | Language pair | Latency regime | Chunk size | BLEU | AL | AP | DAL
---|---|---|---|---|---|---|---
Best IWSLT21 system | En-De | Low | - | 27.40 | 920 | 0.68 | 1420
Medium | - | 29.68 | 1860 | 0.82 | 2650
High | - | 30.75 | 2740 | 0.90 | 3630
Model A | En-De | Low | 600 | 27.05 | 947 | 0.76 | 1993
Medium | 1000 | 30.30 | 1660 | 0.84 | 2662
High | 2000 | 31.41 | 2966 | 0.93 | 3853
Offline | - | 31.36 | 5794 | 1.00 | 5794
Model B | En-De | Low | 500 | 26.93 | 945 | 0.77 | 2052
Medium | 1000 | 31.60 | 1906 | 0.86 | 2945
High | 2500 | 32.98 | 3663 | 0.96 | 4452
Offline | - | 33.14 | 5794 | 1.00 | 5794
En-Ja | Low | 1000 | 16.84 | 2452 | 0.90 | 3212
Medium | 2400 | 16.99 | 3791 | 0.97 | 4296
High | 3000 | 16.97 | 4140 | 0.98 | 4536
Offline | - | 16.88 | 5119 | 1.00 | 5119
En-Zh | Low | 800 | 23.69 | 1761 | 0.85 | 2561
Medium | 1500 | 24.29 | 2788 | 0.93 | 3500
High | 2500 | 24.56 | 3669 | 0.97 | 4212
Offline | - | 24.54 | 5119 | 1.00 | 5119

Table 2: Results of the older model used for the experiments (Model A) and the
submitted system (Model B) on the MuST-C v2 tst-COMMON. We also include the
best IWSLT 2021 system (USTC-NELSLIP (Liu et al., 2021)).
11.522.533.5283032Average Lagging (seconds)BLEUModel AModel BBest IWSLT21
Figure 4: Quality-latency trade-off on English-German tst-COMMON of our two
models: a dedicated English-German model trained from scratch (Model A) and a
multilingual model based on wav2vec and mBART (Model B). We also include the
best IWSLT 2021 system (USTC-NELSLIP (Liu et al., 2021)).

From Tables 2 and 4, we can see that the proposed method works well on two
different models and various language pairs. We see that an improvement in the
offline model (offline BLEU of 31.36 and 33.14 for Model A and B,
respectively) leads to improvement in the online regime.

Finally, we see that our method beats the best IWSLT 2021 system (USTC-NELSLIP
(Liu et al., 2021)) in medium and high latency regimes using both models
(i.e., a model trained from scratch and a model based on pretrained wav2vec
and mBART), and is almost on par in the low latency regime (Model A is losing
0.35 BLEU and Model B is losing 0.47 BLEU).

###  6.1 Computationally Aware Latency

In this paper, we do not report any computationally aware metrics, as our
implementation of Transformers is slow. Later, we implemented the same
onlinization approach using wav2vec 2.0 and mBART from Huggingface
Transformers Wolf et al. (2020). The new implementation reaches faster than
real-time inference speed.

##  7 Conclusion

In this paper, we reviewed onlinization strategies for end-to-end speech
translation models. We identified the optimal stable hypothesis detection
strategy and proposed two separate ways of the quality-latency trade-off
parametrization. We showed that the onlinization of the offline models is easy
and performs almost on par with the offline run. We demonstrated that an
improvement in the offline model leads to improved online performance. We also
showed that our method outperforms a dedicated simultaneous system. Finally,
we proposed an improvement in the average latency metric.

## Acknowledgments

This work has received support from the project “Grant Schemes at CU” (reg.
no. CZ.02.2.69/0.0/0.0/19_073/0016935), the grant 19-26934X (NEUREM3) of the
Czech Science Foundation, the European Union’s Horizon 2020 Research and
Innovation Programme under Grant Agreement No 825460 (ELITR), and partly
supported by a Facebook Sponsored Research Agreement “Language Similarity in
Machine Translation”.

## References

  * Anastasopoulos et al. (2022) Antonios Anastasopoulos, Luisa Bentivogli, Marcely Z. Boito, Ondřej Bojar, Roldano Cattoni, Anna Currey, Georgiana Dinu, Kevin Duh, Maha Elbayad, Marcello Federico, Christian Federmann, Hongyu Gong, Roman Grundkiewicz, Barry Haddow, Benjamin Hsu, Dávid Javorský, Věra Kloudová, Surafel M. Lakew, Xutai Ma, Prashant Mathur, Paul McNamee, Kenton Murray, Maria Nădejde, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, Juan Pino, Elizabeth Salesky, Jiatong Shi, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Yogesh Virkar, Alex Waibel, Changhan Wang, and Shinji Watanabe. 2022\.  FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN.  In _Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022)_ , Dublin, Ireland. Association for Computational Linguistics.
  * Anastasopoulos et al. (2021) Antonios Anastasopoulos, Ondřej Bojar, Jacob Bremerman, Roldano Cattoni, Maha Elbayad, Marcello Federico, Xutai Ma, Satoshi Nakamura, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Alexander Waibel, Changhan Wang, and Matthew Wiesner. 2021\.  FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN.  In _Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)_ , pages 1–29, Bangkok, Thailand (online). Association for Computational Linguistics.
  * Ansari et al. (2020) Ebrahim Ansari, Amittai Axelrod, Nguyen Bach, Ondřej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir Durrani, Marcello Federico, Christian Federmann, Jiatao Gu, Fei Huang, Kevin Knight, Xutai Ma, Ajay Nagesh, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Xing Shi, Sebastian Stüker, Marco Turchi, Alexander Waibel, and Changhan Wang. 2020.  FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN.  In _Proceedings of the 17th International Conference on Spoken Language Translation_ , pages 1–34, Online. Association for Computational Linguistics.
  * Baevski et al. (2020) Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020.  wav2vec 2.0: A framework for self-supervised learning of speech representations.  _Advances in Neural Information Processing Systems_ , 33:12449–12460.
  * Bahar et al. (2021) Parnia Bahar, Patrick Wilken, Mattia A. Di Gangi, and Evgeny Matusov. 2021.  Without further ado: Direct and simultaneous speech translation by AppTek in 2021.  In _Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)_ , pages 52–63, Bangkok, Thailand (online). Association for Computational Linguistics.
  * Cattoni et al. (2021) Roldano Cattoni, Mattia Antonino Di Gangi, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021.  Must-c: A multilingual corpus for end-to-end speech translation.  _Computer Speech & Language_, 66:101155.
  * Cherry and Foster (2019) Colin Cherry and George Foster. 2019.  Thinking slow about latency evaluation for simultaneous machine translation.  _arXiv preprint arXiv:1906.00048_.
  * Cho and Esipova (2016) Kyunghyun Cho and Masha Esipova. 2016.  Can neural machine translation do simultaneous translation?  _arXiv preprint arXiv:1606.02012_.
  * Dai et al. (2019) Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019.  Transformer-XL: Attentive language models beyond a fixed-length context.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL)_.
  * Elbayad et al. (2020) Maha Elbayad, Ha Nguyen, Fethi Bougares, Natalia Tomashenko, Antoine Caubrière, Benjamin Lecouteux, Yannick Estève, and Laurent Besacier. 2020\.  ON-TRAC consortium for end-to-end and simultaneous speech translation challenge tasks at IWSLT 2020.  In _Proceedings of the 17th International Conference on Spoken Language Translation_ , pages 35–43, Online. Association for Computational Linguistics.
  * Graves (2012) Alex Graves. 2012.  Sequence transduction with recurrent neural networks.  _arXiv preprint arXiv:1211.3711_.
  * Ha et al. (2016) Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016.  Toward multilingual neural machine translation with universal encoder and decoder.  In _Proceedings of the 13th International Workshop on Spoken Language Translation (IWSLT 2016)_ , Seattle, USA.
  * Han et al. (2020) Hou Jeung Han, Mohd Abbas Zaidi, Sathish Reddy Indurthi, Nikhil Kumar Lakumarapu, Beomseok Lee, and Sangha Kim. 2020.  End-to-end simultaneous translation system for IWSLT2020 using modality agnostic meta-learning.  In _Proceedings of the 17th International Conference on Spoken Language Translation_ , pages 62–68, Online. Association for Computational Linguistics.
  * Indurthi et al. (2020) Sathish Indurthi, Houjeung Han, Nikhil Kumar Lakumarapu, Beomseok Lee, Insoo Chung, Sangha Kim, and Chanwoo Kim. 2020.  End-end speech-to-text translation with modality agnostic meta-learning.  In _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 7904–7908.
  * Kolss et al. (2008) Muntsin Kolss, Stephan Vogel, and Alex Waibel. 2008.  Stream decoding for simultaneous spoken language translation.  In _Ninth Annual Conference of the International Speech Communication Association_.
  * Levinson and Torreira (2015) Stephen C Levinson and Francisco Torreira. 2015.  Timing in turn-taking and its implications for processing models of language.  _Frontiers in psychology_ , 6:731.
  * Lewis et al. (2019) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_.
  * Liu et al. (2021) Dan Liu, Mengge Du, Xiaoxi Li, Yuchen Hu, and Lirong Dai. 2021.  The USTC-NELSLIP systems for simultaneous speech translation task at IWSLT 2021.  In _Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)_ , pages 30–38, Bangkok, Thailand (online). Association for Computational Linguistics.
  * Liu (2020) Danni Liu. 2020.  Low-latency end-to-end speech recognition with enhanced readability.  Master’s thesis, Maastricht University.
  * Liu et al. (2020a) Danni Liu, Gerasimos Spanakis, and Jan Niehues. 2020a.  Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection.  In _Proc. Interspeech 2020_ , pages 3620–3624.
  * Liu et al. (2020b) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020b.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Ma et al. (2019) Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, et al. 2019.  Stacl: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3025–3036.
  * Ma et al. (2020) Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, and Juan Pino. 2020.  Simuleval: An evaluation toolkit for simultaneous translation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 144–150.
  * Nguyen et al. (2021) Thai-Son Nguyen, Sebastian Stüker, and Alex Waibel. 2021.  Super-Human Performance in Online Low-Latency Recognition of Conversational Speech.  In _Proc. Interspeech 2021_ , pages 1762–1766.
  * Niehues et al. (2016) Jan Niehues, Thai Son Nguyen, Eunah Cho, Thanh-Le Ha, Kevin Kilgour, Markus Müller, Matthias Sperber, Sebastian Stüker, and Alex Waibel. 2016.  Dynamic Transcription for Low-Latency Speech Translation.  In _Proc. Interspeech 2016_ , pages 2513–2517.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , pages 311–318.
  * Pham et al. (2020a) Ngoc-Quan Pham, Thanh-Le Ha, Tuan-Nam Nguyen, Thai-Son Nguyen, Elizabeth Salesky, Sebastian Stüker, Jan Niehues, and Alex Waibel. 2020a.  Relative Positional Encoding for Speech Recognition and Direct Translation.  In _Proc. Interspeech 2020_ , pages 31–35.
  * Pham et al. (2022) Ngoc-Quan Pham, Tuan-Nam Nguyen, Thai-Binh Nguyen, Danni Liu, Carlos Mullov, Jan Niehues, and Alexander Waibel. 2022.  Effective combination of pretrained models - KIT@IWSLT2022.  In _Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022)_ , Dublin, Ireland. Association for Computational Linguistics.
  * Pham et al. (2020b) Ngoc-Quan Pham, Felix Schneider, Tuan Nam Nguyen, Thanh-Le Ha, Thai-Son Nguyen, Maximilian Awiszus, Sebastian Stüker, and Alex Waibel. 2020b.  Kit’s iwslt 2020 slt translation system.  In _Proceedings of the 17th International Conference on Spoken Language Translation_ , pages 55–61.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting bleu scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.  _arXiv preprint arXiv:2008.00401_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , pages 5998–6008.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.