# Chiral valley phonons and flat phonon bands in moiré materials

Indrajit Maity, Arash A. Mostofi  Johannes Lischner<EMAIL_ADDRESS>Departments of Materials and Physics and the Thomas Young Centre for Theory
and Simulation of Materials, Imperial College London, South Kensington Campus,
London SW7 2AZ, UK

###### Abstract

We investigate the chirality of phonon modes in twisted bilayer
$\mathrm{WSe_{2}}$ and demonstrate distinct chiral behavior of the
$K/K^{\prime}$ valley phonons for twist angles close to $0^{\circ}$ and close
to $60^{\circ}$. In particular, multiple chiral non-degenerate $K/K^{\prime}$
valley phonons are found for twist angles near $60^{\circ}$ whereas no non-
degenerate chiral modes are found for twist angles close to $0^{\circ}$.
Moreover, we discover two sets of emergent chiral valley modes that originate
from an inversion symmetry breaking at the moiré scale and find similar modes
in moiré patterns of strain-engineered bilayers $\mathrm{WSe_{2}}$ and
$\mathrm{MoSe_{2}/WSe_{2}}$ heterostructures. At the energy gap between
acoustic and optical modes, the formation of flat phonon bands for a broad
range of twist angles is observed in twisted bilayer $\mathrm{WSe_{2}}$. Our
findings are relevant for understanding electron-phonon and exciton-phonon
scattering in moiré materials and also for the design of phononic analogues of
flat band electrons.

Introduction. Chirality - the characteristic of an object that can be
distinguished from its mirror image - plays a fundamental role in physics,
chemistry, and biology. For example, different enantiomers of chiral molecules
absorb different amounts of right-handed and left-handed polarized light,
which enables their spectroscopic characterization through measurement of the
circular dichroism [1]. In condensed matter physics, the chirality of
electrons gives rise to many exotic effects, such as Klein tunneling [2] in
graphene, and the chiral magnetic effect in three-dimensional semi-metals [3,
4].

Recently, the study of chirality in phonons has attracted significant
interest. For example, Zhang and Niu [5] predicted the existence of chiral
phonons at the $K$ and $K^{\prime}$ valleys of monolayer $\mathrm{WSe_{2}}$,
which was subsequently verified by experiments that measured the circular
dichroism of phonon-assisted intervalley transitions of holes [6]. The
interaction of such chiral valley phonons with other quasiparticles is
relevant for understanding and controlling many electronic and optical
phenomena [7, 8, 9, 10, 11, 12, 13]. For instance, Li et al. demonstrated that
the coupling between a chiral valley phonon and an intervalley exciton in
monolayer $\mathrm{WSe_{2}}$ can lead to a long exciton lifetime maintaining
the valley polarization, which is important for valley-excitonics [12]. Chen
et al. observed the entanglement of chiral phonon modes in monolayer
$\mathrm{WSe_{2}}$ and single photons emitted from an embedded quantum dot in
the material, which promises to reveal new avenues for phonon-driven
entanglement of quantum dots [14].

Besides monolayer $\mathrm{WSe_{2}}$, chiral phonons have been found in other
materials [15, 16, 17, 18, 19]. A particularly promising platform for
observing and manipulating chiral phonons are twisted bilayers of two-
dimensional (2D) materials. Since the discovery of flat electronic bands in
twisted bilayer graphene [20], such moiré materials have emerged as rich
systems to investigate the properties of correlated electrons, excitons and
phonons [21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
38, 39, 40, 41, 42]. For example, at small twist angles phonon properties are
significantly modified as a consequence of large atomic reconstructions [43,
38, 44] which result in the localization of incipient phason modes [45] and
optical phonon modes [45, 46, 47, 48, 49, 50, 51]. To date, however, the
chirality of phonons in twisted bilayer systems has not been studied.

In this Letter, we study the chirality of phonon modes in moiré materials.
Focusing on twisted bilayer $\mathrm{WSe_{2}}$, we first demonstrate
qualitatively different phonon chiralities for twist angles close to
$0^{\circ}$ and close to $60^{\circ}$ and explain these differences by
analyzing the folding of the monolayer $K$ and $K^{\prime}$-valleys into the
Brillouin zone of the twisted bilayer. For twist angles near $60^{\circ}$, we
find two sets of emergent chiral phonons modes which originate from symmetry
breaking at the moiré scale. Finally, we discover flat chiral phonon bands in
the energy gap between acoustic and optical modes without the requirement of a
magic angle. Such flat phonon bands have recently been reported in
metamaterials [52, 53, 54, 55], but not yet in moiré systems.

Methods. For twist angles near $0\degree$ ($\theta$) or $60\degree$
($60^{\circ}-\theta$), the size of the moiré unit cell becomes very large
making standard first-principles approaches for calculating phonon properties
unfeasible. We therefore use simpler models for interatomic interactions.
Specifically, intralayer interactions in $\mathrm{WSe_{2}}$ and
$\mathrm{MoSe_{2}}$ are described using a Stillinger-Weber potential, which
has been demonstrated to give accurate results for monolayers [56, 57]. For
interlayer interactions, we use the Kolmogorov-Crespi potential that can
correctly reproduce the interlayer binding energy landscape obtained using
first-principles calculations [58]. The relaxed atomic positions of our
twisted bilayer systems are determined using the implementation of these
potentials within the LAMMPS code [59]. Phonon frequencies and polarization
vectors are obtained by diagonalizing the dynamical matrix using a modified
version of the PHONOPY code [60] (see Supplementary Information (SI), Sec. I,
for additional details). For monolayer graphene, we use the REBO potential
[61].

Following Zhang and Niu [5], we define the chirality of a phonon with crystal
momentum $\mathbf{q}$ and band index $n$ along the $z$-direction (assuming
that the 2D material lies in the $x$-$y$ plane) as

$\begin{split}S^{z}_{n{\bf q}}=&\bra{\epsilon_{n{\bf
q}}}\widehat{S}^{z}\ket{\epsilon_{n{\bf q}}}\\\ =&\bra{\epsilon_{n{\bf
q}}}\sum_{j=1}^{N}\Big{[}\ket{R_{j}}\bra{R_{j}}-\ket{L_{j}}\bra{L_{j}}\Big{]}\ket{\epsilon_{n{\bf
q}}},\end{split}$ (1)

where $\ket{\epsilon_{n{\bf q}}}$ is the normalized polarization vector,
$\widehat{S}^{z}$ denotes the phonon circular polarization operator and $N$ is
the number of atoms in the moiré unit cell. $\ket{R_{j}}$ and $\ket{L_{j}}$
denote the right and left circularly polarized basis vectors for atom $j$. For
example, for the first two atoms these basis vectors are given by
$|R_{1}\rangle=1/\sqrt{2}\times{(1,i,0,0,\ldots)^{\mathrm{T}}}$,
$|L_{1}\rangle=1/\sqrt{2}\times{(1,-i,0,0,\ldots)^{\mathrm{T}}}$,
$|R_{2}\rangle=1/\sqrt{2}\times{(0,0,1,i,0,0,\ldots)^{\mathrm{T}}}$, and
$|L_{2}\rangle=1/\sqrt{2}\times{(0,0,1,-i,0,0,\ldots)^{\mathrm{T}}}$. The
phonon mode is linearly polarized when $S_{n\bf{q}}^{z}=0$, circularly
polarized when $S_{n\bf{q}}^{z}=\pm 1$ and elliptically polarized when
$0<|S_{n\bf{q}}^{z}|<1$. We refer to both the circularly and elliptically
polarized modes as chiral phonons. Note that for systems with time-reversal
symmetry, we have $S^{z}_{n{\bf q}}=-S^{z}_{n-{\bf q}}$, whereas systems with
inversion-symmetry obey $S^{z}_{n{\bf q}}=S^{z}_{n-{\bf q}}$. Therefore, the
existence of non-degenerate chiral phonon modes requires the breaking of one
of these symmetries [5, 6].

As a test, we first study the chirality of phonons in monolayer graphene
(which has both time-reversal and inversion symmetry) and monolayer
$\mathrm{WSe_{2}}$ (which is not inversion symmetric). The results are shown
in Sec. II of the SI [62]. As expected, all graphene phonons are achiral. For
$\mathrm{WSe_{2}}$, we find multiple non-degenerate chiral phonon modes near
the $K$ and $K^{\prime}$ valleys, consistent with previous first-principles
calculations [6], and the signs of the chiralities in the two valleys are
opposite.

Figure 1: (a),(b): Chiral phonons in twisted bilayer $\mathrm{WSe_{2}}$ at
twist angles of $21.78^{\circ}$ and $38.22^{\circ}$, respectively. Dark solid
lines represent phonon bands and blue and red color indicates the associated
chirality. (c),(d): Red circles denote the unfolded $K_{\mathrm{M}}$ point of
the moiré BZ into the monolayer BZ for $21.78^{\circ}$ and $38.22^{\circ}$,
respectively. The green and blue solid lines represent the unrotated and
rotated monolayer BZs, whereas the black solid line represents the moiré BZ.

Chiral valley phonons in twisted bilayer ${WSe_{2}}$. The naturally occuring
bilayer $\mathrm{WSe_{2}}$ has 2H stacking and is inversion symmetric [63].
The untwisted bilayer, therefore, does not exhibit chiral valley phonons.
However, the introduction of a twist between the layers results in the
emergence of chiral phonons, as demonstrated in Figs. 1(a) and (b), which show
the phonon chirality $S^{z}$ of each phonon mode at twist angles of
$21.78^{\circ}$ and $38.22^{\circ}$, respectively. At $21.78^{\circ}$, all
phonons near the $K_{\mathrm{M}}$ and $K^{\prime}_{\mathrm{M}}$ valleys are
achiral (with the subscript $\mathrm{M}$ denoting that these k-points belong
to the moiré Brillouin zone), but there are some chiral phonons between
$\Gamma_{\mathrm{M}}$ and $M_{\mathrm{M}}$. In contrast, we find several
chiral phonon modes in the $K_{\mathrm{M}}$/$K^{\prime}_{\mathrm{M}}$ valleys
for the $38.22^{\circ}$ twist angle (but none between $\Gamma_{\mathrm{M}}$
and $M_{\mathrm{M}}$). Qualitatively similar results are found for other twist
angles near $0\degree$ and $60\degree$, respectively.

The different behaviour of the $K_{\mathrm{M}}$/$K^{\prime}_{\mathrm{M}}$
valley phonons for twist angles near $0^{\circ}$ and near $60^{\circ}$ can be
understood by analyzing the folding of the monolayer phonon modes into the
smaller moiré Brillouin zone. For the untwisted system (i.e.,
$\theta=0^{\circ}$), the $K$ point of the top layer (denoted $K_{\mathrm{t}}$)
coincides with the $K$ point of the bottom layer (denoted $K_{\mathrm{b}}$)
and similarly $K^{\prime}_{\mathrm{t}}$ and $K^{\prime}_{\mathrm{b}}$
coincide. When instead a small, but finite twist angle is considered, the BZs
of the two layers are rotated relative to each other and a smaller moiré BZ is
created, see Fig. 1(c). To understand which crystal momenta $\mathbf{q}_{i}$
of the monolayer BZs fold onto a specific point $\mathbf{Q}_{\mathrm{M}}$ in
the moiré BZ, we “unfold” $\mathbf{Q}_{\mathrm{M}}$ by adding all possible
reciprocal lattice vectors ${\bf G}_{i}$ corresponding to the moiré crystal
according to [64]

${\bf q}_{i}={\bf Q}_{\mathrm{M}}+{\bf G}_{i}.$ (2)

Figure 1(c) shows the set of points in the monolayer BZs that result from
unfolding $K_{\mathrm{M}}$ for a twist angle of $21.78^{\circ}$. It can be
seen that both $K^{\prime}_{\mathrm{t}}$ and $K_{\mathrm{b}}$ fold onto
$K_{\mathrm{M}}$. Since the chirality of the phonons at
$K^{\prime}_{\mathrm{t}}$ and $K_{\mathrm{b}}$ have equal magnitudes, but
opposite signs, the phonons of the twisted system have a vanishing chirality.

Instead, for the system with $60^{\circ}$ twist angle, $K_{\mathrm{t}}$
coincides with $K^{\prime}_{\mathrm{b}}$ and similarly
$K^{\prime}_{\mathrm{t}}$ and $K_{\mathrm{b}}$ coincide. Fig. 1(d) shows that
for a twist angle near $60^{\circ}$, $K_{\mathrm{M}}$ unfolds onto
$K^{\prime}_{\mathrm{t}}$ and $K^{\prime}_{\mathrm{b}}$ which have phonons
with the same chirality sign. As a consequence, the phonons of the twisted
bilayer exhibit a non-vanishing chirality at the $K_{\mathrm{M}}$ valley (and
also for the $K^{\prime}_{\mathrm{M}}$ valley).

Figure 2: (a) Chiral $K$-valley phonon modes of monolayer $\mathrm{WSe_{2}}$
(first column) and twisted bilayer $\mathrm{WSe_{2}}$ at different twist
angles (columns labelled $\beta$). For comparison, results are also shown for
twisted bilayers without interlayer interactions (columns labelled $\alpha$).
The colour of the symbols indicates the chirality of the phonon modes. The
emergent chiral modes are highlighted by curly brackets. (b)-(e) Absolute
value of the in-plane components of the normalized polarization vector for
some of the chiral modes for a twist angle of $58^{\circ}$. In (b) and (c),
the polarization vector of W atoms in the top layer is used, In (d) and (e),
the polarization vector of Se atoms in the top layer is shown. Different
stacking locations and associated chiralities within the moiré unit cell,
which is denoted by dashed lines, are indicated.

Figure 2(a) shows the chiralities of phonons in the $K_{\mathrm{M}}$-valley as
a function of twist angle near $60\degree$ (columns labelled $\beta$) and
compares them to the $\mathrm{WSe_{2}}$ monolayer result (the first column)
and also to the results of _decoupled_ twisted bilayers at the same twist
angles (columns labelled $\alpha$), in which no interlayer interactions
between the layers are considered. Note that no atomic relaxations occur in
decoupled twisted bilauers. For a twist angle of $38.22\degree$, the phonon
chiralities of the twisted bilayer are very similar to those of the monolayer
and also to those of the decoupled system. As the twist angle approaches
$60\degree$, two novel sets of chiral valley phonons emerge [indicated by the
curly brackets in Fig. 2(a)] in the twisted system which are absent in the
monolayer and in the decoupled bilayer: one set with frequencies near 10
$\mathrm{cm^{-1}}$ and the other with frequencies near 160 $\mathrm{cm^{-1}}$.
The frequencies of the first set soften as the twist angle approaches
$60\degree$.

To understand the origin of these emergent chiral valley phonons, we plot the
magnitude of the polarization vector in the moiré unit cell for 58∘, see Figs.
2(b)-(e). Figs. 2(b)-(c) show results for the two emergent valley phonons with
the largest chiralities from the low-energy set (near
$10~{}\mathrm{cm^{-1}}$). In particular, the mode with
$6.2~{}\mathrm{cm^{-1}}$ has a chirality of $-0.56$ and is localized in the
$\mathrm{B^{Se/Se}}$ stacking regions while the one with
$9.6~{}\mathrm{cm^{-1}}$ has a chirality of 0.35 and is localized in the
$\mathrm{B^{W/W}}$ stacking regions. Analyzing the contributions to the total
chirality of these modes, we find that both layers and both W and Se atoms
contribute. Figs. 2(d)-(e) show the polarization vectors of the two valley
phonons belonging to the second emergent set (with frequencies near
$160~{}\mathrm{cm^{-1}}$) that have the largest chiralities. The mode with
frequency $159.2~{}\mathrm{cm^{-1}}$ is localized in the $\mathrm{B^{Se/Se}}$
region while the one with frequency $159.4~{}\mathrm{cm^{-1}}$ is
approximately localized around the $\mathrm{B^{W/W}}$ region. Interestingly,
only the Se atoms contribute to the total chiralities of these modes.

The emergence of these novel chiral valley phonons can be explained by an
inversion symmetry breaking on the moiré scale that occurs as the twist angle
approaches $60\degree$. Near this twist angle, the moiré unit cell contains
three high-symmetry stackings: 2H, $\mathrm{B^{W/W}}$, and
$\mathrm{B^{Se/Se}}$. Among these stackings 2H is most stable and for twist
angles larger than $\sim~{}55\degree$ atomic relaxations result in a
significant increase in the regions with 2H stacking as well as the formation
of domain walls [43] (see SI, Sec. III). It can be shown [43] that this system
can be mapped onto an equivalent system of two particles (with one particle
representing $\mathrm{B^{W/W}}$ regions and the other $\mathrm{B^{Se/Se}}$
regions) that occupy different sublattices of a triangular lattice and are
connected by springs. Since the energies of $\mathrm{B^{W/W}}$ and
$\mathrm{B^{Se/Se}}$ regions are different, the two particles have different
masses and therefore the effective system lacks inversion symmetry. This in
turn results in the emergence of novel chiral phonon modes. In contrast, the
twisted bilayer near $0\degree$ features three high-symmetry stackings: AA,
$\mathrm{B^{Se/W}}$, and $\mathrm{B^{W/Se}}$. Among these stackings
$\mathrm{B^{Se/W}}$ and $\mathrm{B^{W/Se}}$ are the most stable stackings.
This system can be mapped onto a set of particles (representing $\mathrm{AA}$
regions) which occupy the sites of a triangular lattice and are connected by
springs. This effective system exhibits inversion symmetry and therefore no
emergent chiral phonon modes are observed.

Our analysis shows that the polarization vectors of chiral phonons near
$0^{\circ}$ and $60^{\circ}$ are qualitatively different from the those of the
monolayer phonons. This suggests that commonly used approaches for calculating
phonon properties in twisted bilayers that are based on a zone folding
procedure of monolayer results are unreliable [65, 66].

Figure 3: (a) The phonon band structure of twisted bilayer $\mathrm{WSe_{2}}$
at $58^{\circ}$ in the vicinity of the energy gap between acoustic and optical
modes. (b) Chirality $S^{z}$ (b) and band width (c) of flat acoustic
($\mathrm{A1^{\prime}}$) and optical ($\mathrm{O1^{\prime}}$) phonon bands as
function of twist angle near $60^{\circ}$. (d) and (e): Polarization vectors
of $\mathrm{A1^{\prime}}$ and $\mathrm{O1^{\prime}}$ modes at a twist angle of
$58^{\circ}$. Different stackings are indicated by colored symbols and the
moiré unit cell is outlined using dashed gray lines. In (d), the displacements
of W atoms in the top layer is shown, while in (e) that of Se atoms in the top
layer was used.

Phononic band flattening in twisted bilayer $\mathrm{WSe_{2}}$. Figure 3(a)
demonstrates the emergence of flat phonon bands in the vicinity of the energy
gap between acoustic and optical modes for a $58\degree$ twisted bilayer. This
phononic band gap is inherited from the monolayer which has an energy gap of
17.9 $\mathrm{cm^{-1}}$ between the acoustic and optical phonon modes with the
highest-energy acoustic mode being located at $K$ and the lowest-energy
optical mode at the $\Gamma$ point of the BZ. The highest-energy acoustic mode
at $K$ is non-degenerate and chiral with $S^{z}=-0.5$, while the lowest-energy
optical modes at $\Gamma$ are degenerate and achiral. Fig. 3(b) shows that the
chirality associated with the highest-energy acoustic mode at $K_{\mathrm{M}}$
(denoted as $\mathrm{A1^{\prime}}$) changes dramatically and becomes achiral
as the twist angle approaches $60^{\circ}$. This change in chirality is caused
by the localization of the phonon mode in regions with inversion-symmetric 2H
stacking (see discussion below). In contrast, the lowest-energy optical mode
at $K_{\mathrm{M}}$ (denoted as $\mathrm{O1^{\prime}}$) becomes chiral as the
twist angle approaches $60^{\circ}$. Fig. 3(c) shows the bandwidths of the A1′
and O1′ modes and demonstrates that these bands become extremely flat as the
twist angle approaches $60^{\circ}$ and the size of the moiré cell increases.
Similar to the electronic bandwidths in twisted transition metal
dichalcogenide bilayers, the flattening does not occur at a specific magic
angle, unlike the case of twisted bilayer graphene [20]. Interestingly, the
widths of the flat phonon bands are significantly smaller than the width of
the flat electron bands at the same twist angle. For example, at a twist angle
of 56.5∘ of the twisted bilayer of TMDs, the width of the electronic valence
band is 5 meV [27] and the width of the largest acoustic phonon band is 0.1
meV.

The flattening of the phonon bands is a consequence of both zone folding and
the localization of phonons in real space (see SI, Sec. IV for details). Figs.
3(d) and (e) show the polarization vectors (at $K_{\mathrm{M}}$) of the A1′
and O1′ modes for a twist angle of $58^{\circ}$. The A1′ mode is localized in
regions of 2H stacking, while O1′ is localized in $\mathrm{B^{Se/Se}}$ regions
as well as the domain walls. It is interesting to note that the localization
of the highest acoustic phonons is strikingly similar to that of the highest
electronic valence states in these systems [67]. Similar band flattening is
also observed for twist angles near $0^{\circ}$ (see SI[62], Sec. IV), where
the highest-energy acoustic mode is localized in the
$\mathrm{B^{W/Se}/B^{Se/W}}$ stacking region and the lowest-energy optical
mode is localized in regions of $\mathrm{AA}$ stacking and domain walls.

The distinct localization of the acoustic and optical modes can be explained
by analyzing the frequencies of these modes in the various untwisted bilayers
with high-symmetry stacking arrangements. For example, the A1′ mode originates
from the $K$ point of the monolayer BZ. For the untwisted bilayers, the phonon
frequencies at $K$ are 141.08 cm-1 for $\mathrm{B^{Se/Se}}$ stacking, 141.1
cm-1 for $\mathrm{B^{W/W}}$ stacking and 141.8 cm-1 for $\mathrm{2H}$
stacking. Therefore, the highest-energy acoustic mode localizes on the
$\mathrm{2H}$ regions as the moiré unit cell increases in size.

Chiral valley phonons in strain-engineered moiré materials and
heterostructures. Besides twisting, there are two approaches to create a moiré
pattern in bilayer systems: (i) by applying strain to only one of two
identical layers and (ii) by stacking two different 2D materials on top of
each other. We investigate the existence of chiral valley phonons in both
cases. In contrast to the twisted bilayer at small twist angles, we find that
the strain-engineered bilayer exhibits chiral phonons in the $K_{\mathrm{M}}$
valley (see Sec. V of the SI for a summary of these modes and more details
[62]). In addition, emergent chiral phonon modes with frequencies of
$\sim~{}10~{}\mathrm{cm^{-1}}$ and $\sim~{}156~{}\mathrm{cm^{-1}}$ are found.
Finally, we also observe emergent chiral phonon modes in a
$\mathrm{WSe_{2}/MoSe_{2}}$ heterostructure at a twist angle of $3.14^{\circ}$
(see SI [62], Sec. V).

Summary. In this paper, we have demonstrated the presence of chiral phonons in
twisted bilayer WSe2 as well as other moiré materials and studied their
properties. We have found that the phonon chirality depends on the twist angle
with systems near $0^{\circ}$ exhibiting qualitatively different chiralities
than systems near $60^{\circ}$. Very close to $60^{\circ}$, we observe
emergent chiral modes as well as flat phonon bands. The predicted chiral
properties of phonons in twisted bilayer materials can be measured with
helicity-resolved Raman spectroscopy [15, 68]. While the resolution of such
techniques is typically not high enough to access individual modes in systems
near 0∘ and 60∘, they can potentially measure the k-point and helicity-
resolved phonon density of states (see SI, Sec. VII). Future work should
investigate the scattering of chiral phonons with other quasiparticles, such
as electrons, excitons, and photons and the effects of such scattering
processes on the electronic and optical properties of moiré materials.

###### Acknowledgements.

Acknowledgments This project has received funding from the European Union’s
Horizon 2020 research and innovation programme under the Marie Skłodowska-
Curie grant agreement No 101028468. The authors acknowledge support from the
Thomas Young Centre under grant TYC-101, and discussions with Rup Chowdhury,
Nikita Tepliakov and Saurabh Srivastav.

Supplementary Information (SI) :
Chiral valley phonons and flat phonon bands in moiré materials

##  I I: Simulation details

###  I.1 Generation of structures

#### Moiré pattern due to twist:

The commensurate twist angles that give rise to the least-area moiré unit-cell
for twisted bilayer of $\mathrm{WSe_{2}}$ are given by,

$\cos(\theta_{i})=\frac{3i^{2}+3i+1/2}{3i^{2}+3i+1}$ (3)

for an integer $i$ [69]. One of the set of moiré lattice vectors are- ${\bf
A_{1}}=i{\bf a_{1}}+(i+1){\bf a_{2}}$ and ${\bf A_{2}}=-(i+1){\bf
a_{1}}+(2i+1){\bf a_{2}}$ with ${\bf a_{1}}=(1/2,\sqrt{3}/2)a_{0}$ and ${\bf
a_{2}}=(-1/2,\sqrt{3}/2)a_{0}$, where $a_{0}$ is the lattice constant of singe
layer $\mathrm{WSe_{2}}$. For twist angles ($\theta_{i}$) close to
$0^{\circ}$, we construct the moiré unit-cell starting from AA stacking using
TWISTER [70]. The high-symmetry stackings present in the moiré patterns are-
$\mathrm{AA}$ (with W and Se of the top layer are directly above W and Se of
bottom layer), $\mathrm{B^{W/Se}}$ (Bernal stacking with W of the top layer
directly above Se of bottom layer), and $\mathrm{B^{Se/W}}$ (Bernal stacking
with Se of the top layer directly above W of bottom layer). These stackings
are sometimes referred to as AA, AB, and BA, as well. For twist angles
($60^{\circ}-\theta_{i}$) close to $60^{\circ}$, we construct the moiré unit-
cell starting from 2H stacking. The high-symmetry stackings present in the
moiré patterns are- $\mathrm{2H}$ (with W and Se of the top layer are directly
above Se and W of bottom layer), $\mathrm{B^{Se/Se}}$ (Bernal stacking with Se
of the top layer directly above Se of bottom layer), and $\mathrm{B^{W/W}}$
(Bernal stacking with W of the top layer directly above W of bottom layer).
These stackings are sometimes referred to as AA′, A′B, and AB′, respectively.

Twist angles |  Number of atoms |  Moiré length (in Å)
---|---|---
21.78∘ |  42 |  8.8
9.4∘ |  222 |  20.4
7.3∘ |  366 |  26.1
6∘ |  546 |  31.9
4.41∘ |  1626 |  54.6
3.14∘ |  1986 |  60.4
2∘ |  4902 |  94.9
38.22∘ |  42 |  8.8
50.6∘ |  222 |  20.4
52.7∘ |  366 |  26.1
54∘ |  546 |  31.9
55.59∘ |  1626 |  54.6
56.86∘ |  1986 |  60.4
58∘ |  4902 |  94.9
58.31∘ |  6846 |  113.3
Table 1: Moiré patterns of twisted bilayer $\mathrm{WSe_{2}}$ studied in this
work.

#### Moiré pattern due to strain

We use the unit-cell lattice vectors of single layer of $\mathrm{WSe_{2}}$ as
${\bf a_{1}}=a_{0}(1,0,0)$, ${\bf a_{2}}=a_{0}(1/2,\sqrt{3}/2,0)$, and ${\bf
a_{3}}=a_{0}(0,0,30)$, where $a_{0}$ is the lattice constant. We can create
the moiré patterns in two ways: by straining only one of the layer of
$\mathrm{WSe_{2}}$, and by straining both the layers. We use the first
approach in this work. We apply a tensile strain to the bottom layer. We
discuss the details of this approach below. The periodicity of the strained
bilayer is computed in the following manner:

$\begin{bmatrix}m&0\\\ 0&m\end{bmatrix}\begin{pmatrix}{\bf a_{1}}\\\ {\bf
a_{2}}\end{pmatrix}=\begin{bmatrix}n&0\\\ 0&n\end{bmatrix}\begin{pmatrix}{\bf
a_{1}^{b}}\\\ {\bf a_{2}^{b}}\end{pmatrix}$ (4)

In the case of 3.3 % biaxial strain, we use $m=30$, and $n=29$ starting from
AA stacking configuration. The moiré lattice constant of the strained bilayer
is 98.9 Å and contains 5223 atoms.

#### Moiré heterostructure

We replace the top layer of bilayer of $\mathrm{WSe_{2}}$ by
$\mathrm{MoSe_{2}}$ and construct moiré lattice by rotating the layer. Since
the experimental lattice constants of these two materials are similar, we
simply replace one of the $\mathrm{WSe_{2}}$ layer by $\mathrm{MoSe_{2}}$ in
the twisted bilayer $\mathrm{WSe_{2}}$ and create the moiré heterostructure.

###  I.2 Structural relaxations

The moiré patterns are relaxed using the LAMMPS package with the Stillinger-
Weber [56], and Kolmogorov-Crespi [58] potentials to capture the intralayer
and interlayer interactions of the twisted bilayer of $\mathrm{WSe_{2}}$,
respectively. The Kolmogorov-Crespi parameters used in this work can correctly
reproduce the interlayer binding energy landscape, obtained using density
functional theory. The atomic relaxations produced using these parameters are
in excellent agreement with relaxations performed using density functional
theory. The intralayer Stillinger-Weber potential quite accurately captures
the full phonon dispersion, in particular the acoustic modes. The accurate
potentials used in our simulations are expected to capture the phonon
renormalization of the moiré materials. At first, we relax the simulation box
to ensure the in-plane pressure is as small as possible. Next, we relax the
atoms within a fixed simulation box with the force tolerance of $10^{-5}$ eV/Å
for any atom along any direction.

###  I.3 Force constants calculations

For monolayer $\mathrm{WSe_{2}}$, we use a $6\times 6\times 1$ supercell to
compute the force constants. For large twist angles of twisted bilayer
$\mathrm{WSe_{2}}$, we use a $2\times 2\times 1$ supercell to compute the
force constants. The moiré patterns contain thousands of atoms for small twist
angles. Therefore, we use $1\times 1\times 1$ supercell to compute the force-
constants.

##  II II : Chiral phonons in 2D materials

###  II.1 Absence of chiral valley-phonons in graphene

Figure 4: Chirality of phonons in single layer of graphene. The solid lines represent the phonon frequencies, and hexagons (absent in the figure as $S^{z}=0$) represent the computed $S_{z}$ values after summing over all the basis atoms. There are no non-degenerate chiral modes in single layer of graphene due to inversion symmetry. $\omega$ |  $S^{z}$ |  $S^{z}_{C1}$ |  $S^{z}_{C{2}}$
---|---|---|---
(in $\mathrm{cm^{-1}}$) |  |  |
505.2 |  0.0 |  0.0 |  0.0
505.2 |  0.0 |  0.0 |  0.0
821.1 |  0.0 |  0.5 |  -0.5
1201.5 |  0.0 |  -0.5 |  0.5
1201.5 |  0.0 |  -0.5 |  0.5
1724.5 |  0.0 |  0.5 |  -0.5
Table 2: Phonon modes, total chirality, and chirality of the each types of
atoms in the case of single-layer of graphene. Within the unit-cell, two atoms
can have opposite chirality, but the total chirality is 0.

$\omega=505.2$ cm-1 $\omega=505.2$ cm-1 $\omega=821.1$ cm-1 $\omega=1201.5$
cm-1 $\omega=1201.5$ cm-1 $\omega=1724.5$ cm-1 Eigenvectors associated with
phonon modes in grapheneThe Blue circles denote the Carbon atoms

###  II.2 Presence of chiral non-degenerate valley phonons in monolayer
$\mathrm{WSe_{2}}$

Figure 5: Chirality of phonons in monolayer of $\mathrm{WSe_{2}}$. The solid lines represent the phonon frequencies, and hexagons represent the computed $S_{z}$ values after summing over all the basis atoms. There are multiple non-degenerate chiral modes in monolayer of $\mathrm{WSe_{2}}$ due to inversion symmetry breaking. $\omega$ |  $S^{z}$ |  $S^{z}_{W1}$ |  $S^{z}_{Se{2}}$ |  $S^{z}_{Se{3}}$
---|---|---|---|---
(in $\mathrm{cm^{-1}}$) |  |  |  |
118.9 |  -0.38 |  0.0 |  -0.19 |  -0.19
121.2 |  0.76 |  0.76 |  0.0 |  0.0
141.4 |  -0.49 |  -0.75 |  0.13 |  0.13
200.66 |  0.0 |  0.0 |  0.0 |  0.0
212.06 |  -1.0 |  0.0 |  -0.5 |  -0.5
216.52 |  1.0 |  0.0 |  0.5 |  0.5
240.28 |  0.49 |  -0.25 |  0.37 |  0.37
249.2 |  -0.62 |  0.0 |  -0.31 |  -0.31
255.38 |  0.24 |  0.24 |  0.0 |  0.0
Table 3: Chirality associated with all the valley phonon modes (at $K$) in
single layer of $\mathrm{WSe_{2}}$.

$\omega=118.9$ cm-1 $\omega=121.2$ cm-1 $\omega=141.4$ cm-1 $\omega=200.6$
cm-1 $\omega=212$ cm-1 $\omega=216.5$ cm-1 $\omega=240.3$ cm-1 $\omega=249.2$
cm-1 $\omega=255.4$ cm-1 Eigenvectors associated with phonon modes in single
layer $\mathrm{WSe_{2}}$The Blue and red circles denote the Se and W atoms,
respectively.

##  III III : Interlayer separation for twist angles close to $60^{\circ}$ and
$0^{\circ}$

Figure 6: (a),(b): Significant structural relaxations and inversion symmetry
breaking at the moiré scale illustrated using interlayer separation for twist
angles of $58^{\circ}$ and $2^{\circ}$, respectively. The associated color bar
denotes the magnitude of interlayer separation and is in Å. The moiré unit
cell and stackings are marked.

##  IV IV : Flat phonon bands

Figure 7: (a), (b) Evidence of band-flattening for a twist angle of 3.14∘ and $56.86^{\circ}$ twisted bilayer $\mathrm{WSe_{2}}$ near the acoustic-optic phonon band gap. The blue lines denote the calculations with twisted bilayers and the red dashed lines denote calculations with decoupled twisted bilayers. Importantly, the band flattens and degeneracies in the band-structure of the decoupled systems are significantly reduced. (c),(d) ((e),(f)) Absolute value of the polarization vectors at $\Gamma_{\mathrm{M}}$ of $\mathrm{A1}$ and $\mathrm{O1}$ modes at a twist angle of $3.14^{\circ}$ ($56.86^{\circ}$). Different stackings are indicated by colored symbols and the moiré unit cell is outlined using dashed gray lines. In (c),(e) the displacements of W atoms in the top layer is shown, while in (d),(f) that of Se atoms in the top layer was used. (g) The band-flattening is indicated by the reduction of band-width with twist angles.  | Bandwidth in cm-1
---|---
| Twisted bilayer | Decoupled twisted bilayer
Twist angle | A1 | O1 | A1 | O1
55.59∘ | 1.029 | 0.185 | 1.321 | 0.327
56.86∘ | 0.427 | 0.069 | 0.672 | 0.110
Table 4: Comparison of bandwidths in the twisted bilayers that include all the
relaxation effects and in the decoupled twisted bilayers due to zone-folding
for $55.59^{\circ}$ and $56.86^{\circ}$ twisted bilayers. Note that no atomic
relaxations occur in decoupled twisted bilayers.

##  V V : Chiral valley phonons in strain engineered moiré and
$\mathrm{MoSe_{2}/WSe_{2}}$ heterostructure

Figure 8: (a),(b)-Chiralities at K point for all the phonon modes in 3.3$\%$
biaxially strained moiré of $\mathrm{WSe_{2}}$ and the
$\mathrm{MoSe_{2}/WSe_{2}}$ heterostructure. The left column of (a) denote the
non-interacting and the right column of (a) interacting bilayer calculations.
Similarly, we show the results for (b). The emergent chiral modes are
highlighted by curly brackets.

We apply a bi-axial tensile strain to the bottom layer of an AA stacked
bilayer $\mathrm{WSe_{2}}$. The resulting strained structure contains three
unique high-symmetry stackings: $\mathrm{A\widetilde{A}}$,
$\mathrm{B^{W/\widetilde{Se}}}$ and $\mathrm{B^{Se/\widetilde{W}}}$, where the
tilde indicates a sublattice of the strained layer. These stackings are
distinct from those of the twisted bilayer near $0^{\circ}$ as they break
layer symmetry.

##  VI VI: Achiral phonon modes at $K/K^{\prime}$ for twist angles close to
$0^{\circ}$

Figure 9: Chiral phonons in twisted bilayer $\mathrm{WSe_{2}}$ at twist angle
of (a) $4.41^{\circ}$, (b) $55.59^{\circ}$, (c) $3.14^{\circ}$ and (d)
$56.86^{\circ}$). Black solid circles represent phonon bands (computed at a
few discrete points) and blue and red color indicates the associated chirality
(computed at the same set of discrete points). Moreover, the size of the
colored circles are proportional to absolute value of the chirality. As this
figure is somewhat difficult to interpret, we have also directly inspected the
computed chirality values at $\mathrm{K_{M}}$ and find that there are non-
degenerate chiral modes for twist angles close to 60∘ but none for twist
abgles close to 0∘.

##  VII VII: Chirality resolved and total density of states

Figure 10: (a),(b),(c): Total and chirality resolved density of states (DOS).
We have used $36\times 36\times 1$, $12\times 12\times 1$, and $3\times
3\times 1$ grids to compute the phonon DOS for monolayer $\mathrm{WSe_{2}}$,
$38.22^{\circ}$ twisted bilayer $\mathrm{WSe_{2}}$, and $55.59^{\circ}$
twisted bilayer $\mathrm{WSe_{2}}$, respectively. The delta function in DOS
calculation has been replaced by a Gaussian with a standard deviation of 4.8
cm-1. Figure 11: (a),(b),(c): $k$-point resolved density of states (DOS). We
have used $36\times 36\times 1$, $12\times 12\times 1$, and $3\times 3\times
1$ k-point grids to compute the phonon DOS for monolayer $\mathrm{WSe_{2}}$,
$38.22^{\circ}$ twisted bilayer $\mathrm{WSe_{2}}$, and $55.59^{\circ}$
twisted bilayer $\mathrm{WSe_{2}}$, respectively. A Gaussian broadening with a
standard deviation of 4.8 cm-1 has been used.

## References

  * Kneer  _et al._ [2018] L. M. Kneer, E.-M. Roller, L. V. Besteiro, R. Schreiber, A. O. Govorov, and T. Liedl, ACS Nano 12, 9110 (2018).
  * Katsnelson  _et al._ [2006] M. I. Katsnelson, K. S. Novoselov, and A. K. Geim, Nature Physics 2, 620 (2006).
  * Li and Kharzeev [2016] Q. Li and D. E. Kharzeev, Nuclear Physics A 956, 107 (2016), the XXV International Conference on Ultrarelativistic Nucleus-Nucleus Collisions: Quark Matter 2015.
  * Li  _et al._ [2016] Q. Li, D. E. Kharzeev, C. Zhang, Y. Huang, I. Pletikosifá, A. V. Fedorov, R. D. Zhong, J. A. Schneeloch, G. D. Gu, and T. Valla, Nature Physics 12, 550 (2016).
  * Zhang and Niu [2015] L. Zhang and Q. Niu, Phys. Rev. Lett. 115, 115502 (2015).
  * Zhu  _et al._ [2018] H. Zhu, J. Yi, M.-Y. Li, J. Xiao, L. Zhang, C.-W. Yang, R. A. Kaindl, L.-J. Li, Y. Wang, and X. Zhang, Science 359, 579 (2018).
  * Carvalho  _et al._ [2017] B. R. Carvalho, Y. Wang, S. Mignuzzi, D. Roy, M. Terrones, C. Fantini, V. H. Crespi, L. M. Malard, and M. A. Pimenta, Nature Communications 8, 14670 (2017).
  * Chen  _et al._ [2015] S.-Y. Chen, C. Zheng, M. S. Fuhrer, and J. Yan, Nano Letters 15, 2526 (2015).
  * Kang  _et al._ [2018] M. Kang, S. W. Jung, W. J. Shin, Y. Sohn, S. H. Ryu, T. K. Kim, M. Hoesch, and K. S. Kim, Nature Materials 17, 676 (2018).
  * Li  _et al._ [2019a] Z. Li, T. Wang, C. Jin, Z. Lu, Z. Lian, Y. Meng, M. Blei, S. Gao, T. Taniguchi, K. Watanabe, T. Ren, S. Tongay, L. Yang, D. Smirnov, T. Cao, and S.-F. Shi, Nature Communications 10, 2469 (2019a).
  * Liu  _et al._ [2019] E. Liu, J. van Baren, T. Taniguchi, K. Watanabe, Y.-C. Chang, and C. H. Lui, Phys. Rev. Research 1, 032007 (2019).
  * Li  _et al._ [2019b] Z. Li, T. Wang, C. Jin, Z. Lu, Z. Lian, Y. Meng, M. Blei, M. Gao, T. Taniguchi, K. Watanabe, T. Ren, T. Cao, S. Tongay, D. Smirnov, L. Zhang, and S.-F. Shi, ACS Nano 13, 14107 (2019b).
  * Delhomme  _et al._ [2020] A. Delhomme, D. Vaclavkova, A. Slobodeniuk, M. Orlita, M. Potemski, D. M. Basko, K. Watanabe, T. Taniguchi, D. Mauro, C. Barreteau, E. Giannini, A. F. Morpurgo, N. Ubrig, and C. Faugeras, 2D Materials 7, 041002 (2020).
  * Chen  _et al._ [2019] X. Chen, X. Lu, S. Dubey, Q. Yao, S. Liu, X. Wang, Q. Xiong, L. Zhang, and A. Srivastava, Nature Physics 15, 221 (2019).
  * Chen  _et al._ [2018] H. Chen, W. Zhang, Q. Niu, and L. Zhang, 2D Materials 6, 012002 (2018).
  * Juraschek and Spaldin [2019] D. M. Juraschek and N. A. Spaldin, Phys. Rev. Materials 3, 064405 (2019).
  * Du  _et al._ [2019] L. Du, J. Tang, Y. Zhao, X. Li, R. Yang, X. Hu, X. Bai, X. Wang, K. Watanabe, T. Taniguchi, D. Shi, G. Yu, X. Bai, T. Hasan, G. Zhang, and Z. Sun, Advanced Functional Materials 29, 1904734 (2019).
  * Zhang  _et al._ [2020a] W. Zhang, A. Srivastava, X. Li, and L. Zhang, Phys. Rev. B 102, 174301 (2020a).
  * Grissonnanche  _et al._ [2020] G. Grissonnanche, S. Thériault, A. Gourgout, M.-E. Boulanger, E. Lefrancois, A. Ataei, F. Laliberté, M. Dion, J.-S. Zhou, S. Pyon, T. Takayama, H. Takagi, N. Doiron-Leyraud, and L. Taillefer, Nature Physics 16, 1108 (2020).
  * Bistritzer and MacDonald [2011] R. Bistritzer and A. H. MacDonald, Proceedings of the National Academy of Sciences 108, 12233 (2011).
  * Cao  _et al._ [2018a] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Nature 556, 43 (2018a).
  * Cao  _et al._ [2018b] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, R. C. Ashoori, and P. Jarillo-Herrero, Nature 556, 80 (2018b).
  * Tran  _et al._ [2020] K. Tran, J. Choi, and A. Singh, 2D Materials 8, 022002 (2020).
  * Wang  _et al._ [2020] L. Wang, E.-M. Shih, A. Ghiotto, L. Xian, D. A. Rhodes, C. Tan, M. Claassen, D. M. Kennes, Y. Bai, B. Kim, K. Watanabe, T. Taniguchi, X. Zhu, J. Hone, A. Rubio, A. N. Pasupathy, and C. R. Dean, Nature Materials 19, 861 (2020).
  * Huang  _et al._ [2020] M. Huang, Z. Wu, J. Hu, X. Cai, E. Li, L. An, X. Feng, Z. Ye, N. Lin, K. T. Law, _et al._ , arXiv preprint arXiv:2006.05615 (2020).
  * Scuri  _et al._ [2020] G. Scuri, T. I. Andersen, Y. Zhou, D. S. Wild, J. Sung, R. J. Gelly, D. Bérubé, H. Heo, L. Shao, A. Y. Joe, A. M. Mier Valdivia, T. Taniguchi, K. Watanabe, M. Lončar, P. Kim, M. D. Lukin, and H. Park, Phys. Rev. Lett. 124, 217403 (2020).
  * Naik and Jain [2018] M. H. Naik and M. Jain, Phys. Rev. Lett. 121, 266401 (2018).
  * Zhang  _et al._ [2020b] Z. Zhang, Y. Wang, K. Watanabe, T. Taniguchi, K. Ueno, E. Tutuc, and B. J. LeRoy, Nature Physics 16, 1093 (2020b).
  * An  _et al._ [2020] L. An, X. Cai, D. Pei, M. Huang, Z. Wu, Z. Zhou, J. Lin, Z. Ying, Z. Ye, X. Feng, R. Gao, C. Cacho, M. Watson, Y. Chen, and N. Wang, Nanoscale Horiz. 5, 1309 (2020).
  * Xian  _et al._ [2020] L. Xian, M. Claassen, D. Kiese, M. M. Scherer, S. Trebst, D. M. Kennes, and A. Rubio, arXiv preprint arXiv:2004.02964 (2020).
  * Devakul  _et al._ [2021] T. Devakul, V. Crépel, Y. Zhang, and L. Fu, arXiv preprint arXiv:2106.11954 (2021).
  * Vitale  _et al._ [2021] V. Vitale, K. Atalar, A. A. Mostofi, and J. Lischner, arXiv preprint arXiv:2102.03259 (2021).
  * Venkateswarlu  _et al._ [2020] S. Venkateswarlu, A. Honecker, and G. Trambly de Laissardière, Phys. Rev. B 102, 081103 (2020).
  * Li  _et al._ [2021] E. Li, J.-X. Hu, X. Feng, Z. Zhou, L. An, K. T. Law, N. Wang, and N. Lin, arXiv preprint arXiv:2103.06479 (2021).
  * Soriano and Lado [2021] D. Soriano and J. Lado, arXiv preprint arXiv:2105.04502 (2021).
  * Shabani  _et al._ [2021] S. Shabani, D. Halbertal, W. Wu, M. Chen, S. Liu, J. Hone, W. Yao, D. N. Basov, X. Zhu, and A. N. Pasupathy, Nature Physics 17, 720 (2021).
  * Brem  _et al._ [2020] S. Brem, K.-Q. Lin, R. Gillen, J. M. Bauer, J. Maultzsch, J. M. Lupton, and E. Malic, Nanoscale 12, 11088 (2020).
  * Weston  _et al._ [2020] A. Weston, Y. Zou, V. Enaldiev, A. Summerfield, N. Clark, V. Zólyomi, A. Graham, C. Yelgel, S. Magorrian, M. Zhou, J. Zultak, D. Hopkinson, A. Barinov, T. H. Bointon, A. Kretinin, N. R. Wilson, P. H. Beton, V. I. Fal’ko, S. J. Haigh, and R. Gorbachev, Nature Nanotechnology 15, 592 (2020).
  * Sinha  _et al._ [2020] S. Sinha, P. C. Adak, R. S. Surya Kanthi, B. L. Chittari, L. D. V. Sangani, K. Watanabe, T. Taniguchi, J. Jung, and M. M. Deshmukh, Nature Communications 11, 5548 (2020).
  * Goodwin  _et al._ [2021] Z. A. Goodwin, L. Klebl, V. Vitale, X. Liang, V. Gogtay, X. van Gorp, D. M. Kennes, A. A. Mostofi, and J. Lischner, arXiv preprint arXiv:2105.12641 (2021).
  * Graham  _et al._ [2020] A. J. Graham, J. Zultak, M. J. Hamer, V. Zolyomi, S. Magorrian, A. Barinov, V. Kandyba, A. Giampietri, A. Locatelli, F. Genuzio, N. C. Teutsch, C. Salazar, N. D. M. Hine, V. I. Falḱo, R. V. Gorbachev, and N. R. Wilson, 2D Materials 8, 015016 (2020).
  * Pizarro  _et al._ [2019] J. M. Pizarro, M. J. Calderón, and E. Bascones, Journal of Physics Communications 3, 035024 (2019).
  * Maity  _et al._ [2021] I. Maity, P. K. Maiti, H. R. Krishnamurthy, and M. Jain, Phys. Rev. B 103, L121102 (2021).
  * Carr  _et al._ [2018] S. Carr, D. Massatt, S. B. Torrisi, P. Cazeaux, M. Luskin, and E. Kaxiras, Phys. Rev. B 98, 224102 (2018).
  * Maity  _et al._ [2020] I. Maity, M. H. Naik, P. K. Maiti, H. R. Krishnamurthy, and M. Jain, Phys. Rev. Research 2, 013335 (2020).
  * Gadelha  _et al._ [2021] A. C. Gadelha, D. A. A. Ohlberg, C. Rabelo, E. G. S. Neto, T. L. Vasconcelos, J. L. Campos, J. S. Lemos, V. Ornelas, D. Miranda, R. Nadas, F. C. Santana, K. Watanabe, T. Taniguchi, B. van Troeye, M. Lamparski, V. Meunier, V.-H. Nguyen, D. Paszko, J.-C. Charlier, L. C. Campos, L. G. Can√ßado, G. Medeiros-Ribeiro, and A. Jorio, Nature 590, 405 (2021).
  * Koshino and Son [2019] M. Koshino and Y.-W. Son, Phys. Rev. B 100, 075416 (2019).
  * Quan  _et al._ [2021] J. Quan, L. Linhart, M.-L. Lin, D. Lee, J. Zhu, C.-Y. Wang, W.-T. Hsu, J. Choi, J. Embley, C. Young, T. Taniguchi, K. Watanabe, C.-K. Shih, K. Lai, A. H. MacDonald, P.-H. Tan, F. Libisch, and X. Li, Nature Materials (2021), 10.1038/s41563-021-00960-1.
  * Lamparski  _et al._ [2020] M. Lamparski, B. V. Troeye, and V. Meunier, 2D Materials 7, 025050 (2020).
  * [50] K.-Q. Lin, J. Holler, J. M. Bauer, P. Parzefall, M. Scheuck, B. Peng, T. Korn, S. Bange, J. M. Lupton, and C. Schüller, Advanced Materials n/a, 2008333.
  * Debnath  _et al._ [2020] R. Debnath, I. Maity, R. Biswas, V. Raghunathan, M. Jain, and A. Ghosh, Nanoscale 12, 17272 (2020).
  * Deng  _et al._ [2020] Y. Deng, M. Oudich, N. J. Gerard, J. Ji, M. Lu, and Y. Jing, Phys. Rev. B 102, 180304 (2020).
  * Rosendo López  _et al._ [2020] M. Rosendo López, F. Peñaranda, J. Christensen, and P. San-Jose, Phys. Rev. Lett. 125, 214301 (2020).
  * Gardezi  _et al._ [2021] S. M. Gardezi, H. Pirie, S. Carr, W. Dorrell, and J. E. Hoffman, 2D Materials 8, 031002 (2021).
  * Martí-Sabaté and Torrent [2021] M. Martí-Sabaté and D. Torrent, Phys. Rev. Applied 15, L011001 (2021).
  * Mobaraki  _et al._ [2018] A. Mobaraki, A. Kandemir, H. Yapicioglu, O. Gülseren, and C. Sevik, Computational Materials Science 144, 92 (2018).
  * Kandemir  _et al._ [2016] A. Kandemir, H. Yapicioglu, A. Kinaci, T. Çağın, and C. Sevik, Nanotechnology 27, 055703 (2016).
  * Naik  _et al._ [2019] M. H. Naik, I. Maity, P. K. Maiti, and M. Jain, The Journal of Physical Chemistry C 123, 9770 (2019).
  * Plimpton [1995] S. Plimpton, Journal of Computational Physics 117, 1 (1995).
  * Togo and Tanaka [2015] A. Togo and I. Tanaka, Scripta Materialia 108, 1 (2015).
  * Brenner  _et al._ [2002] D. W. Brenner, O. A. Shenderova, J. A. Harrison, S. J. Stuart, B. Ni, and S. B. Sinnott, Journal of Physics: Condensed Matter 14, 783 (2002).
  * [62] See Supplementary Information (SI) for simulation details, chiral phonon in monolayer graphene, $\mathrm{WSe_{2}}$ and polarization vectors, interlayer separation for twist angles close to $0^{\circ}$ and close to $60^{\circ}$, flat phonon bands for twist angles close to 0∘, chiral valley phonons in strain engineered moiré and $\mathrm{MoSe_{2}/WSe_{2}}$ heterostructure, achiral phonons for twist angles close to $0^{\circ}$, chirality and k-point resolved density of states, and movies to illustrate distinct chiral valley phonons twist angles close to $0^{\circ}$ and close to $60^{\circ}$ (attached as mp4 files) .
  * Jones  _et al._ [2014] A. M. Jones, H. Yu, J. S. Ross, P. Klement, N. J. Ghimire, J. Yan, D. G. Mandrus, W. Yao, and X. Xu, Nature Physics 10, 130 (2014).
  * Popescu and Zunger [2012] V. Popescu and A. Zunger, Phys. Rev. B 85, 085201 (2012).
  * Lin  _et al._ [2018] M.-L. Lin, Q.-H. Tan, J.-B. Wu, X.-S. Chen, J.-H. Wang, Y.-H. Pan, X. Zhang, X. Cong, J. Zhang, W. Ji, P.-A. Hu, K.-H. Liu, and P.-H. Tan, ACS Nano 12, 8770 (2018).
  * Parzefall  _et al._ [2021] P. Parzefall, J. Holler, M. Scheuck, A. Beer, K.-Q. Lin, B. Peng, B. Monserrat, P. Nagler, M. Kempf, T. Korn, and C. Schüller, 2D Materials 8, 035030 (2021).
  * Kundu  _et al._ [2021] S. Kundu, M. H. Naik, H. Krishnamurthy, and M. Jain, arXiv preprint arXiv:2103.07447 (2021).
  * Tatsumi  _et al._ [2018] Y. Tatsumi, T. Kaneko, and R. Saito, Phys. Rev. B 97, 195444 (2018).
  * Lopes dos Santos  _et al._ [2007] J. M. B. Lopes dos Santos, N. M. R. Peres, and A. H. Castro Neto, Phys. Rev. Lett. 99, 256802 (2007).
  * Naik  _et al._ [2022] S. Naik, M. H. Naik, I. Maity, and M. Jain, Computer Physics Communications 271, 108184 (2022).


# SYGMA: System for Generalizable Modular Question Answering Over Knowledge
Bases

Sumit Neelam, Udit Sharma, Hima Karanam, Shajith Ikbal, Pavan Kapanipathi
Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Young-Suk Lee, Santosh
Srivastava
Cezar Pendus, Saswati Dana, Dinesh Garg, Achille Fokoue, G P Shrivatsa Bhargav
Dinesh Khandelwal, Srinivas Ravishankar, Sairam Gurajada, Maria Chang,
Rosario Uceda-Sosa, Salim Roukos, Alexander Gray, Guilherme Lima
Ryan Riegel, Francois Luus, L Venkata Subramaniam
IBM Research

###### Abstract

Knowledge Base Question Answering (KBQA) tasks that involve complex reasoning
are emerging as an important research direction. However, most KBQA systems
struggle with generalizability, particularly on two dimensions: (a) across
multiple reasoning types where both datasets and systems have primarily
focused on multi-hop reasoning, and (b) across multiple knowledge bases, where
KBQA approaches are specifically tuned to a single knowledge base. In this
paper, we present SYGMA, a modular approach facilitating generalizability
across multiple knowledge bases and multiple reasoning types. Specifically,
SYGMA contains three high level modules: 1) KB-agnostic question understanding
module that is common across KBs 2) Rules to support additional reasoning
types and 3) KB-specific question mapping and answering module to address the
KB-specific aspects of the answer extraction. We demonstrate effectiveness of
our system by evaluating on datasets belonging to two distinct knowledge
bases, DBpedia and Wikidata. In addition, to demonstrate extensibility to
additional reasoning types we evaluate on multi-hop reasoning datasets and a
new Temporal KBQA benchmark dataset on Wikidata, named TempQA-
WD111https://github.com/IBM/tempqa-wd, introduced in this paper. We show that
our generalizable approach has better competetive performance on multiple
datasets on DBpedia and Wikidata that requires both multi-hop and temporal
reasoning.

##  1 Introduction

The goal of Knowledge Base Question Answering (KBQA) systems is to answer
natural language questions by retrieving and reasoning over facts in a
Knowledge Base (KB). KBQA has gained significant popularity both due to its
practical real-world applications and challenging research problems associated
with it (Fu et al. 2020). However, most existing datasets and research in this
area are primarily focused on single/multi-hop reasoning on a single knowledge
base (Trivedi et al. 2017; Usbeck et al. 2017; Yih et al. 2016). Consequently,
this has encouraged research in methodologies that are tuned to a restricted
set of reasoning types on a single knowledge base, in turn lacking
generalizability (Kapanipathi et al. 2021; Zou et al. 2014a; Sorokin and
Gurevych 2018a). In this work, we propose a modular approach that is
generalizable across knowledge bases and reasoning types.

Figure 1: A description of how information about Nero’s predecessor is
represented in DBpedia vs. Wikidata

Open Knowledge Bases such as DBpedia (Auer et al. 2007), Wikidata (Vrandečić
and Krötzsch 2014), and Freebase (Bollacker et al. 2008) form the basis of
many KBQA datasets. Each of these knowledge Bases, however, have different
representations. For instance: information associated to the question “Who was
roman emperor before Nero?" from DBpedia and Wikidata are shown in Figure 1.
We can see that Wikidata represents properties of facts such as replaces,
temporal and spatial with reification.222Information about facts are
explicitly mapped by making facts as primitive nodes. On the other hand,
DBpedia manages to represent it as a simple fact with the relationship to
existing entity nodes. Handling such different representations is an
unexplored challenge and requires to be addressed for a generalizable KBQA
approach.The second aspect where KBQA approaches fail to generalize is in
handling complex reasoning types such as temporal and spatial reasoning. Table
1 shows examples of questions that require different types of reasoning to
answer them. Most research have focused on multi-hop reasoning questions
(Bordes et al. 2015a; Dubey et al. 2019; Berant et al. 2013), lacking both
approaches and datasets that are adaptable to complex reasoning types.
Particularly a single approach that can answer questions with different
reasoning types.

In this paper, we present a modular approach called SYGMA (System for
Generalizable and Modular question Answering over knowledge bases), that is
built on a framework adaptable to different KB representations and reasoning
types. SYGMA introduces intermediate representations based on lambda calculus
to adapt to new reasoning types and representations. SYGMA , following the
recent advances in KBQA such as NSQA (Kapanipathi et al. 2021) and ReTraCk
(Chen et al. 2021), is a modular approach with independent Abstract Meaning
Representation, Entity Linking, and Relation Linking modules. In contrast to
NSQA which is tuned specifically to DBpedia-based KBQA datasets, SYGMA is
generalizable to multiple knowledge bases and multiple reasoning types. In
order to evaluate our generalizable approach, we consider datasets that have:
(a) DBpedia and Wikidata as knowledge bases, and (b) multi-hop and temporal as
the reasoning types.

In pursuit of this goal, we also address the lack of temporal reasoning
datasets on knowledge bases that we are considering in this work, specifically
Wikidata. We annotate and create TempQA-WD; a Wikidata-based dataset focused
on temporal reasoning. TempQA-WD is a parallel dataset to TempQuestions
dataset (Jia et al. 2018) which is a Freebase based temporal reasoning dataset
derived from three other datasets Free917 (Cai and Yates 2013), WebQuestions
(Berant et al. 2013) and ComplexQuestions (Bao et al. 2016). While the
Freebase dataset consists of question answer pairs along with temporal signal
classifications, our dataset is annotated with intermediate SPARQL queries
enabling evaluation of different modules in a KBQA pipeline.

| Category
---
Example
| Single-Hop
---
| _Who directed Titanic Movie?_
---
SPARQL: select distinct ?a where {
wd:Q44578 wdt:P57 ?a}
| Multi-hop
---
| _Which movie is directed by James Cameron_
---
_starring Leonardo DiCaprio?_
SPARQL: select distinct ?a where
{?a wdt:P57 wd:Q42574.
?a wdt:P161 wd:Q38111. }
| Temporal
---
| _Who was the US President_
---
_during cold war?_ SPARQL: in Figure 3
Table 1: Examples of Single-hop, Multi-hop and Temporal reasoning questions on
Wikidata.

In summary, the main contributions of this paper are:

  * •

A modular, generalizable approach, called SYGMA, for KBQA with lambda calculus
based intermediate representations that enable adaptation to: (a) multiple
knowledge bases, specifically DBpedia and Wikidata, and (b) multiple reasoning
types such as multi-hop and temporal reasoning.

  * •

A benchmark dataset, called TempQA-WD, for building temporal-aware KBQA
systems on Wikidata with parallel annotations on Freebase.

  * •

Experimental results show SYGMA achieves state-of-the-art numbers on LC-QuAD
1.0, WebQSP-WD, and comparable numbers on QALD-9 dataset. We also present
baseline numbers for Simple WebQuestions and new dataset introduced in this
paper; TempQA-WD.

##  2 Related Work

Datasets | Knowledge Base | Emphasis | SPARQL | Intermediate | Natural questions
---|---|---|---|---|---
|  |  |  | Evaluation |
QALD-9 | DBpedia | Multi-hop | ✓ | ✓ | ✓
LC-QuAD 1.0 | DBpedia | Multi-hop | ✓ | ✓ | ✗
LC-QuAD 2 | DBpedia, Wikidata | Multi-hop | ✓ | ✓ | ✗
Simple Questions | Freebase, DBpedia, Wikidata | Single-hop | ✓ | ✓ | ✓
Web Questions | Freebase | Multi-hop | ✗ | ✗ | ✓
WebQSP | Freebase, Wikidata | Multi-hop | ✓ | ✓ | ✓
Complex Web Questions | Freebase | Multi-hop | ✗ | ✗ | ✓
TempQuestions | Freebase | Temporal | ✗ | ✗ | ✓
TempQA-WD | Freebase, Wikidata | Temporal | ✓ | ✓ | ✓

Table 2: This table compares most of the KBQA datasets based on features
relevant to the work presented in this paper.

KBQA Systems: KBQA approaches can be primarily categorized into two groups:
(a) Question-to-entities: where techniques output the answer entities from the
knowledge graph ignoring the SPARQL query (Saxena, Tripathi, and Talukdar
2020; Sun et al. 2018; Vakulenko et al. 2019), and (b) semantic parsing based:
where approaches output intermediate SPARQL queries (or logic forms) that can
retrieve the answer from the knowledge base (Singh et al. 2018; Kapanipathi et
al. 2021; Zou et al. 2014b). First, in question-to-entities category,
(Vakulenko et al. 2019) leverages a message passing technique on a two hop
graph from the entities mentioned in the question to retrieve answers from the
knowledge graph. (Saxena, Tripathi, and Talukdar 2020) encodes both text and
entities, text based on language models and entities based on knowledge graph
embeddings (Trouillon et al. 2016) and shows that text can help KBQA in an
incomplete setting. In contrast to question-to-entities approach, the semantic
parsing based approaches improves interpretability and facilitates evaluation
of different modules as shown by Frankestein (Singh et al. 2018), and NSQA
(Kapanipathi et al. 2021). Both, Frankestein (Singh et al. 2018), and NSQA
(Kapanipathi et al. 2021) follow a pipeline based approach with the
differentiating factor being the use of Abstract Meaning Representation as a
starting point by NSQA. We build on top of these representational efforts and
introduce $\lambda$-expressions with the required additional functions for
adapting to new knowledge graphs and new reasoning types.

KBQA Datasets: Over the years, many question answering datasets have been
developed for KBQA, such as Free917 (Cai and Yates 2013), SimpleQuestions
(Bordes et al. 2015b), WebQuestions (Berant et al. 2013), QALD-9 (Usbeck et
al. 2017), LC-QuAD 1.0 (Trivedi et al. 2017), and LC-QuAD 2.0 (Dubey et al.
2019). In Table 2, we compare each of these datasets across the following
features: (a) underlying KB, including subsequent extensions, e.g. Wikidata
(Diefenbach et al. 2017) and DBpedia (Azmy et al. 2018) based versions of
SimpleQuestions, as well as the Wikidata subset of WebQSP (Sorokin and
Gurevych 2018a); (b) reasoning types that are emphasized in the dataset; (c)
availability of SPARQL queries, entities, and relationships for intermediate
evaluation; and (d) the use of question templates, which can often generate
noisy, unnatural questions. As Table 2 shows, our dataset is distinguished
from prior work in its emphasis on temporal reasoning, its application to both
Freebase and Wikidata, and its annotation of intermediate representations and
SPARQL queries. The most relevant KBQA dataset to our work is TempQuestions
(Jia et al. 2018), upon which we base TempQA-WD, as described in dataset
section. CRONQUESTIONS (Saxena, Chakrabarti, and Talukdar 2021) is another
temporal QA dataset proposed recently over a small subset of Wikidata. It has
template based questions targeted at temporal KG embeddings and QA.

##  3  SYGMA: System Description

Motivated by Kapanipathi et al. (2021), SYGMA is designed as a neuro-symbolic
system, avoiding the need for end-to-end training. Figure 2 shows the overall
KBQA pipeline with independent modules and their intermediate representations.
The approach is designed with the goal of generalization across multiple KBs
and reasoning types. Supporting generalization is accomplished in two stages
as shown in Figure 2, driven by an example in Figure 3. The first is a KB-
agnostic question understanding stage, which takes in a natural language
question as input and outputs an intermediate representation that can be
common across different KBs. The second is question mapping and reasoning
stage where, first, the necessary heuristics to introduce templates for new
reasoning types are applied. Next, the KB-agnostic representation is mapped to
the vocabulary of the KG to output KB-specific representation that has a
deterministic mapping to SPARQL. We use Wikidata and DBpedia to show
generalization across KBs, and use multi-hop and temporal reasoning to
evaluate across reasoning types. Before we get into the details of these
modules and intermediate representations, we give a brief description of the
Lambda calculus and the temporal functions used by SYGMA to generate its
intermediate logic representation.

Figure 2: Architecture of SYGMA that shows the pipeline with independent
modules and intermediate representations. The intermediate representations are
supported by appropriate heuristics to facilitate generalizability. Figure 3:
An illustration of the outputs at the intermediate stages of the pipeline in
SYGMA

Lambda Calculus $\lambda$-calculus, by definition, is considered the smallest
universal programming language that expresses any computable function. In
particular, we have adopted Typed $\lambda$-Calculus presented in (Zettlemoyer
and Collins 2012) which support addition of new higher order functions
necessary for handling various reasoning types. We use constants and logical
connectives like AND, OR, NEGATION and functions like argmin, argmax, count,
etc., presented in this work. Apart from this, we also added new temporal
functions to demonstrate the system’s adaptability to support new reasoning
types. For example, consider the following question and its corresponding
logical form:

Question: | When was Barack Obama born?
---|---
Logical Form: |  |  $\lambda$t. born(b,“Barack Obama") $\wedge$
---
interval(t, b)

Here, b is instance variable for event born(b, “Barack Obama") and interval(t,
b) finds time for the event denoted by b. Variable t is unknown which is
marked as $\lambda$ variable.

Temporal Functions: We introduce interval, overlap, before, after, teenager,
year; where interval gets time interval associated with event and overlap,
before, after are used to compare temporal events. teenager gets teenager age
interval for a person, and year return year of a date.333Although there are
many other possible temporal functions for specific English words, such as
adult and tween, we defined teenager because it was featured in the datasets
explored here.

###  3.1 KB-Agnostic Question Understanding

The modules in this stage aim at deriving logical expressions of the natural
language question that is common for all the knowledge bases. In particular,
we use the formalism of $\lambda$-calculus for logical representation, i.e.,
to map questions to their corresponding $\lambda$-expressions representing the
semantics.

#### AMR

In order to have a generic parse (logical expression) that is common across
KBs, SYGMA performs initial language understanding using Abstract Meaning
Representation (AMR) (Banarescu et al. 2013), a semantic representation
language based on PropBank. AMR encodes the meaning of a sentence into a
rooted directed acyclic graph where nodes represent concepts and edges
represent relations. We adopt an action pointer transformer architecture of
Zhou et al. (2021) for transition-based AMR parsing and self-training
technique of Lee et al. (2020) for domain adaptation to KBQA.

AMR provides generic representation that can be used for multi-hop reasoning
(Kapanipathi et al. 2021). However for temporal expressions with :time
relation, we have to augment the AMR annotations with implicit predicates that
cannot be captured by ellipsis and/or re-entrancies. An example of AMR
annotation for the question Who was US president during cold war? is given in
Figure 3. The representation encodes time edge with cold war event as sub-
event of time. Also in case of before/after constraints it explicitly captures
the constraint as part of the time edge. If there are no explicit constraints
like before/after or ordinal in the time edge, we treat them as overlap
constraints.

#### KB-Agnostic Lambda Expression

AMR is a fairly granular representation. However, for a KBQA system,
specifically on KGs such as DBpedia, Wikidata, and Freebase, such granularity
can add noise. Therefore to construct KB-agnostic $\lambda$-expressions of the
questions from their corresponding AMRs and their identified entity and
relation mentions, we use the transformational heuristics described in Table
3. The table shows the transformation from AMR frame (high level) to the
corresponding Lambda expression. The rule type describes where the rule is in
general applied to all reasoning (base) vs temporal. We handle these as
conjunction of multiple triples in the KB with some projection variable and/or
numerical operation to get the final answer. We use AMR unknown/imperative
constructs to identify the projection variables and AMR polarity/interrogative
frames for queries which fall into boolean answers or ASK questions in SPARQL.

The next set of rules present in Table 3 are few sample template rules used to
capture temporal events and constraints to help in temporal reasoning. Figure
3 gives an example question that falls into the temporal overlap rule in the
table and how the AMR constructs are used to split the events is highlighted
with nodes being bold in the table. Complete set of rules used can be found in
the supplementary material. Note that we covered temporal reasoning as an
additional form of reasoning in the current system. However, more reasoning
templates like spatial reasoning with additional operators like coordinates(),
south() …etc can be added. Table 3 shows an example template for spatial
questions such as "which states are to the south of California?".

| Rule
---
Type
AMR A = (v/frame …) | Lambda Expression L = $\psi$(v)
| Base
---
| (v/frame :arg0(v0/frame0) … :argn(vn/framen))
---
| frame(v, v0, … vn) $\land$ $\psi$(v0) $\land$ … $\psi$(vn)
---
| Base
---
| (v/frame :arg1(a/amr-unknown) … :argn(vn/framen))
---
|  $\lambda$a. $\psi$(v)
---
| Numerical
---
| (v/frame :arg0(v0/frame0 :quant(a/amr-unknown)) …
---
:argn(vn/framen))
| count($\lambda$v0. $\psi$(v))
---
| Temporal
---
| (v/frame :arg0(v0/frame0) … :argn(vn/framen)
---
:time(a/amr-unknown))
|  $\lambda$ev. $\psi$(v) $\land$ interval(ev, v)
---
| Temporal
---
| (v/frame :arg0(a/amr-unknown) … :argn(vn/framen)
---
:time(b/before :op1(n/nested-frame)))
| argmax($\lambda$a. $\psi$(v) , $\lambda$a. $\lambda$ev. $\psi$(n) $\land$
interval(ev, v)
---
$\land$ interval(en, n) $\land$ before(ev, en), 0, 1)
| Temporal
---
| (v/frame :arg0(a/amr-unknown) … :argn(vn/framen)
---
:time(n/nested-frame))
|  $\lambda$a. $\psi$(v) $\land$ $\psi$(n) $\land$ interval(ev, v) $\land$
interval(en, n)
---
$\land$ overlap(ev, en)
| Spatial
---
| (b/be-located-at-91 :arg0(a/amr-unknown),
---
:mod(s/south) :op1(n/nested-frame))
|  $\lambda$a. $\psi$(b)$\land$ $\psi$(n) $\land$ coordinate(cb, b) $\land$
coordinate(cn, n)
---
$\land$ south(cb, cn)

Table 3: AMR to Lambda Translation

###  3.2 Question Mapping and Reasoning

This is the second stage which comprises of modules that transform the KB-
agnostic $\lambda$-expression of the questions into KB-specific
$\lambda$-expression. These modules are entity linking and relationship
linking that are specific to the underlying KB along with any KB specific
rules to handle special forms of transformation or reasoning. The modules are
interchangeable to different knowledge graphs. Currently, we demonstrate the
adaptability of these modules to DBpedia and Wikidata.

#### KB-Specific Lambda Expression:

Structurally, it is similar to $\lambda$-expression with all entities and
relations mapped to KB entities and relations respectively.

_Entity Linking:_ The goal of Entity Linking is to map entity mentions as
captured in the $\lambda$-expression of the question to their corresponding
KB-specific entities. We use a recently proposed zero-shot entity linking
approach called BLINK (Wu et al. 2020). For a question where entity mentions
are already identified, bi-encoder piece of the BLINK is used to predict top-K
entities. For this prediction, we use an entity dictionary of 5.9M English
Wikipedia entities, mapped to their corresponding Wikidata and DBpedia
entities.

_Relation Linking:_ SYGMA’s Relation linking component takes in the question
text and an AMR graph as input and returns a ranked list of KG relationships.
For instance, given a question such as "Who was the US President during cold
war?" (see Fig 3) and the corresponding AMR, the goal of the relation linking
component is to find the corresponding Wikidata KB relations position held
(P39), start time (P580), end time (P582). For this task, we use state-of-the-
art AMR-based relation linking approach with models built for both DBpedia and
Wikidata (Naseem et al. 2021).

SPARQL Query: This module maps KB-specific $\lambda$-expressions onto SPARQL
queries through a deterministic approach. Each KB-specific
$\lambda$-expression construct is mapped to an equivalent SPARQL construct, as
rules given in Table 4. Each of lambda expressions contains one or more terms
such that each term $T_{i}$ is comprised of one or more predicates connected
via $\land$ or $\lor$. Translation of different predicates is present in Table
4. Some of the KB-spe cific aspects like handling reification in Wikidata are
shown in the Table. To get the time interval in case of reified events, Start
time(wdt:P580), end time(wdt:P582), or point in time(wdt:P585) connected to
intermediate statement node are used.

Type | Expression/Predicate E | SPARQL S = $\phi$(E)
---|---|---
|  $\lambda$ abstraction
---
|  $\lambda$x.T
---
| SELECT DISTINCT ?x WHERE { $\phi$(T) }
---
| Count expression
---
| count($\lambda$x.T)
---
| SELECT (COUNT(?x) AS ?c) WHERE { $\phi$(T) }
---
| Argmax expression
---
| argmax($\lambda$x.T1, $\lambda$x.$\lambda$y. T2, O, L)
---
| SELECT DISTINCT ?x WHERE { $\phi$(T1) $\phi$(T2)
---
} ORDER BY DESC(?yStart) LIMIT L OFFSET O
| KB Predicate
---
| IRIp(r, s$|$IRIs, o$|$IRIo)
---
| ?s$|$IRIs IRIp ?o$|$IRIo.
---
| Interval predicate
---
for reified facts
| wdt:PID(e, s$|$wd:QID1, o$|$wd:QID2)
---
$\land$ interval(ei, e)
| ?s$|$wd:QID1 p:PID ?e. ?e ps:PID ?o$|$wd:QID2.
---
?e pq:P580 ?eiStart. ?e pq:P582 ?eiEnd.
| Overlap predicate
---
| overlap(e1i e2i)
---
| FILTER(?e1iStart<=?e2iEnd && ?e2iStart<=?e1iEnd)
---
| Before predicate
---
| before(e1i, e2i)
---
| FILTER(?e1iEnd<=?e2iStart)
---
| After predicate
---
| after(e1i, e2i)
---
| FILTER(?e1iStart>=?e2iEnd)
---

Table 4: Translation of KB Specific Lambda Expression

##  4 TempQA-WD Dataset

In order to evaluate our approach on temporal reasoning, specifically on
DBpedia and Wikidata as KGs, we require a dataset to be based on one of these
two KGs. TempQuestions (Jia et al. 2018) is the first KBQA dataset intended to
focus specifically on temporal reasoning. However, TempQuestions is based on
Freebase.We adapt TempQuestions to Wikidata444Creating a parallel dataset on
two KGs (DBpedia and Wikidata) is non trivial and labor intensive. Therefore,
we opted to create a dataset based on Wikidata alone since it is the most up-
to-date KG. to create a temporal QA dataset that has three desirable
properties. First, we create a generalizable benchmark that has parallel
answer annotations on two KBs. Second, we take advantage of Wikidata’s
evolving, up-to-date knowledge. Lastly, we enhance TempQuestions with SPARQL
queries which was missing in original dataset. We also add entity, relation,
intermediate lambda expression annotations for a subset of the dataset that
are used by the SYGMA.

There has been two previous attempts at transferring Freebase-QA questions to
Wikidata; namely WebQSP-WD (Sorokin and Gurevych 2018a) and SimpleQuestions-
WD(SWQ-WD) (Diefenbach, Singh, and Maret 2017). SWQ-WD contains only single
triple questions whereas WebQSP-WD have only the final question answers that
map directly to corresponding entities in Wikidata. However, as stated
(Sorokin and Gurevych 2018a), one challenge is that not all Freebase answers
can be directly mapped to entities in Wikidata. For example, the Freebase
answer annotation for the question “When did Moscow burn?” is “1812 Fire of
Moscow”, despite the year being entangled with the event itself. In contrast,
Wikidata explicitly represents the year of this event, with an entity for
“Fire in Moscow” and an associated year of “1812”. Thus, a direct mapping
between the two answers is not possible, as it would amount to a false
equivalence between “1812 Fire of Moscow” and “1812”.

In order to address such issues, we enlisted a team of annotators to manually
create and verify SPARQL queries, ensuring not only that the SPARQL
formulation was correct, but that the answers accurately reflected the
required answer type (as in the “Fire in Moscow” example above) and the
evolving knowledge in Wikidata. Having SPARQL queries also facilitates
intermediate evaluation of the approaches that use semantic parsing to
directly generate the query or the query graph, increasing interpretability
and performance in some cases (Sorokin and Gurevych 2018a).

Dataset Details

Dataset | Size | Answer | Add’l Details
---|---|---|---
TempQuestions(Freebase) | 1271 | Freebase | A
TempQA-WD (Wikidata) | 839 | Wikidata | A + B
A Subset of TempQA-WD | 175 | Wikidata | A + B + C
Table 5: Benchmark dataset details. TempQuestions denote original dataset with
Freebase answers (Jia et al. 2018). Second row is the subset adapted to
Wikidata and the third row is the devset taken out of it. Set-A denote
TempQuestions with- {temporal signal, question type, data source}. Set-B
-{Wikidata SPARQL , answer, category}. Set-C-{AMR, $\lambda$-expression,
entities, relations, KB-specific $\lambda$-expression}

Table 5 gives details of our new benchmark dataset. We took all the questions
from TempQuestions dataset (of size $1271$) and chose a subset for which we
could find Wikidata answers. This subset has 839 questions that constitute our
new dataset, TempQA-WD. We annotated this set with their corresponding
Wikidata SPARQL queries and Wikidata answers. We also retained the Freebase
answers from the TempQuestions dataset effectively creating parallel answers
from two KBs. Additionally, we added a question complexity label to each
question, according to the reasoning complexity required to answer the
question. Details of categorization are present in supplementary material.
Within this dataset, we chose a subset of $175$ questions for detailed
annotations as described in Set-C in the table description leaving $664$
question as test set.

##  5 Evaluation

KB $\rightarrow$ | DBpedia | Wikidata
---|---|---
DataSet $\rightarrow$ | LC-QuAD 1.0 | QALD-9 | WebQSP-WD | SWQ-WD | TempQA-WD
Reasoning $\rightarrow$ | multi-hop | multi-hop | multi-hop | single-hop | Temporal
System $\downarrow$ | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1
WDAqua | 0.22 | 0.38 | 0.28 | 0.26 | 0.26 | 0.25 | - | - | - | - | - | - | - | - | -
gAnswer | - | - | - | 0.29 | 0.32 | 0.29 | - | - | - | - | - | - | - | - | -
QAMP | 0.25 | 0.50 | 0.33 | - | - | - | - | - | - | - | - | - | - | - | -
NSQA | 0.44 | 0.45 | 0.44 | 0.31 | 0.32 | 0.31 | - | - | - | - | - | - | - | - | -
GGNN | - | - | - | - | - | - | 0.27 | 0.32 | 0.26 | - | - | - | - | - | -
SYGMA | 0.47 | 0.48 | 0.47 | 0.29 | 0.30 | 0.29 | 0.32 | 0.36 | 0.31 | 0.42 | 0.55 | 0.44 | 0.32 | 0.34 | 0.32
Table 6:  SYGMA’s Performance across knowledge bases and reasoning types.
Baseline numbers are taken from Kapanipathi et al. (2021) and Sorokin and
Gurevych (2018b). P-Precision, R-Recall

#### Implementation Details:

We implemented our pipeline using a Flow Compiler (Chakravarti et al. 2019)
stitching individual modules exposed as gRPC services. We defined the ANTLR
grammar to define $\lambda$-expressions. KB-Specific $\lambda$-expression to
SPARQL module is implemented in Java using Apache Jena SPARQL modules, to
create SPARQL objects and generate the final SPARQL query that is run on
target KB end point. The rest of the modules are implemented in Python and are
exposed as gRPC services.

#### Datasets:

We considered two different KBs namely DBpedia and Wikidata to evaluate our
system. We did not consider Freebase as its no longer actively maintained and
is not up-to-date. Along with the new benchmark dataset TempQA-WD introduced
in this paper, we also evaluate our baseline system on two other benchmark
datasets adopted to Wikidata, to evaluate its effectiveness beyond temporal
questions. They are SWQ-WD (Diefenbach et al. 2017), which consists of 14894
train and 5622 test set questions, and WebQSP-WD (Sorokin and Gurevych 2018b),
which consists of 2880 train and 1033 test set questions. For DBpedia, we used
QALD-9 (Usbeck et al. 2017), that has 408 training and 150 test questions, and
LC-QuAD 1.0 (Trivedi et al. 2017) data set, that has 4k training set and 1k
test set. Each of these datasets operate on their own version of the DBpedia
and SPARQLs provided and we used the same instance of DBpedia for evaluation.
The baseline system is tuned with dev sets of 175 from TempQA-WD and 200 from
SWQ-WD train and 200 from LC-QuAD 1.0 and evaluated on all the the five test
sets.

#### Baselines

We Compare SYGMA with various KBQA baseline systems as given in Table 6. To
our knowledge there is no other system that works across both DBpedia and
Wikidata. We took the numbers reported from NSQA (Kapanipathi et al. 2021)
work, which is current state-of-the-art on both LC-QuAD 1.0 and QALD-9
datasets. We also compare our work with GGNN (Sorokin and Gurevych 2018a)
which is the only known benchmark for WebQSP-WD dataset. We also compare our
work with WDAqua (Diefenbach, Singh, and Maret 2017), gAnswer (Zou et al.
2014b) and QAMP (Vakulenko et al. 2019) systems.

###  5.1 Results & Discussion

Table 6 shows performance of SYGMA on two different KBs with five different
datasets described in Section 5. We also show the type of reasoning required
for each dataset along with the precision/recall and F1 measure. We use GERBIL
(Usbeck et al. 2019) to compute performance metrics from the pairs of gold
answers and system generated answers from the pipeline. To our knowledge, ours
is the first system reporting KBQA numbers across two different KBs and
varying reasoning types. We get state of the art numbers beating NSQA
(Kapanipathi et al. 2021) on LC-QuAD 1.0 and also achieve comparable numbers
for QALD. For WebQSP-WD dataset we get state of the art numbers nearly $20\%$
gain over GGNN (Sorokin and Gurevych 2018a). The accuracy numbers for Wikidata
datasets show that there is ample scope for improvement for different modules.
This gets evident when we look at the module performances on a small set of
development sets across datasets. For WebQSP-WD we did not have any ground
truth for evaluating entity linking and relation linking.

AMR: Table 7 show the performance of the AMR parser on the 5 development sets
in Smatch (standard measure used to measure AMRs) and Exact Match ($\%$ of
questions that match fully with ground truth AMR).

Dataset | KB | Smatch | Exact Match
---|---|---|---
LC-QuAD 1.0 | DBpedia | 87.6 | 30.0
QALD-9 | DBpedia | 89.3 | 41.8
WebQSP-WD | Wikidata | 88.0 | 43.8
SWQ-WD | Wikidata | 83.0 | 37.8
TempQA-WD | Wikidata | 89.6 | 39.8
Table 7: AMR parser performances on the development sets

Entity Linking: Table 8 below captures the performance of entity linking
module on different datasets. The question level accuracy is computed by
considering all the entities present in each question. However, mention level
accuracy is computed by considering single mention at a time. We can see that
entity linking performance for Wikidata datasets is low compared to DBpedia.
One of the reasons for is all these datasets are adopted from Freebase dataset
and the way entities represented in these two KBs is different. Also, very
little research is done on entity linking for Wikidata and KBQA in general.

Dataset | KB | Accuracy (%)
---|---|---
Mention Level | Question Level
LC-QuAD 1.0 | DBpedia | $86.81\;(91.94)$ | $84.00\;(90.50)$
QALD-9 | DBpedia | $89.52\;(94.28)$ | $89.79\;(93.87)$
TempQA-WD | Wikidata | $74.01\;(82.47)$ | $57.14\;(69.71)$
SWQ | Wikidata | $72.27\;(83.18)$ | $70.14\;(81.59)$
Table 8: Entity linking performance on development sets when gold mentions are
provided. The numbers inside parentheses denote the Hits@5 scores.

Relation Linking: Table 9 captures the performance of the relation linking
module. It demonstrates that relation linking is still a challenging tasks
specially with multi-hop reasoning and temporal reasoning, where the query
graph is disconnected across events, there is room for improvement.

Dataset | KB | Precision | Recall | F1
---|---|---|---|---
LC-QuAD 1.0 | DBpedia | 0.52 | 0.50 | 0.50
QALD-9 | DBpedia | 0.55 | 0.53 | 0.53
TempQA-WD | Wikidata | 0.43 | 0.43 | 0.42
SWQ | Wikidata | 0.67 | 0.68 | 0.67
Table 9: Relation linking performance on development sets.

#### Ablation Study:

To gain more insights on the performance, we also did an ablation study of
SYGMA using TempQA-WD dev set for Wikidata and 100 question dev set from LC-
QuAD 1.0 that we manually annotated with ground truth for all the modules.
This shows the impact of individual module on overall performance is
evaluated. Table 10 shows the results. For example GT-$AMR$ refers to the case
where ground truth AMR is fed directly into $\lambda$-module. The table shows
large jump in accuracy (in both the datasets) when fed with ground truth
entities (GT-EL) and ground truth relations (GT-RL). This points to the need
for improved entity linking and relation linking on both datasets across KBs.

| TempQA-WD | LC-QuAD 1.0
---|---|---
| P | R | F1 | P | R | F1
NO GT | 0.47 | 0.50 | 0.47 | 0.50 | 0.52 | 0.50
GT-AMR | 0.50 | 0.51 | 0.50 | 0.51 | 0.53 | 0.51
GT-$\lambda$ | 0.52 | 0.53 | 0.52 | 0.52 | 0.53 | 0.52
GT-EL | 0.60 | 0.62 | 0.60 | 0.63 | 0.66 | 0.64
GT-RL | 0.92 | 0.93 | 0.92 | 0.99 | 0.98 | 0.98
GT-KB-$\lambda$ | 0.93 | 0.93 | 0.93 | 1.0 | 0.99 | 0.99
GT-SPARQL | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0
Table 10: Ablation Study on TempQA-WD and LC-QuAD 1.0 dev sets. P-Precision,
R-Recall

##  6 Conclusion

In this paper, we present SYGMA system that is generalizable across knowledge
bases and reasoning types. We introduce KB-agnostic question understanding
component that is common across KBs with AMR based intermediate
$\lambda$-Calculus representation. Question Mapping and reasoning module on
the specific KB is customized per KB. we also presented rule modules that can
aid in adding new reasoning type. We introduced a new benchmark dataset
TempQA-WD for temporal KBQA on Wikidata. Experimental results show that SYGMA
indeed achieves its generalization goals with state of the art results on LC-
QuAD 1.0 and WebQSP-WD.

## References

  * Auer et al. (2007) Auer, S.; Bizer, C.; Kobilarov, G.; Lehmann, J.; Cyganiak, R.; and Ives, Z. 2007\.  Dbpedia: A nucleus for a web of open data.  In _The semantic web_ , 722–735. Springer.
  * Azmy et al. (2018) Azmy, M.; Shi, P.; Lin, J.; and Ilyas, I. 2018.  Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , 2093–2103. Santa Fe, New Mexico, USA: Association for Computational Linguistics.
  * Banarescu et al. (2013) Banarescu, L.; Bonial, C.; Cai, S.; Georgescu, M.; Griffitt, K.; Hermjakob, U.; Knight, K.; Koehn, P.; Palmer, M.; and Schneider, N. 2013.  Abstract Meaning Representation for Sembanking.  In _Proceedings of Linguistic Annotation Workshop 2013_.
  * Bao et al. (2016) Bao, J.; Duan, N.; Yan, Z.; Zhou, M.; and Zhao, T. 2016.  Constraint-Based Question Answering with Knowledge Graph.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , 2503–2514. Osaka, Japan: The COLING 2016 Organizing Committee.
  * Berant et al. (2013) Berant, J.; Chou, A.; Frostig, R.; and Liang, P. 2013.  Semantic Parsing on Freebase from Question-Answer Pairs.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , 1533–1544. Seattle, Washington, USA: Association for Computational Linguistics.
  * Bollacker et al. (2008) Bollacker, K.; Evans, C.; Paritosh, P.; Sturge, T.; and Taylor, J. 2008.  Freebase: a collaboratively created graph database for structuring human knowledge.  In _Proceedings of the 2008 ACM SIGMOD international conference on Management of data_ , 1247–1250.
  * Bordes et al. (2015a) Bordes, A.; Usunier, N.; Chopra, S.; and Weston, J. 2015a.  Large-scale simple question answering with memory networks.  _arXiv preprint arXiv:1506.02075_.
  * Bordes et al. (2015b) Bordes, A.; Usunier, N.; Chopra, S.; and Weston, J. 2015b.  Large-scale Simple Question Answering with Memory Networks.  _CoRR_ , abs/1506.02075.
  * Cai and Yates (2013) Cai, Q.; and Yates, A. 2013.  Large-scale Semantic Parsing via Schema Matching and Lexicon Extension.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 423–433. Sofia, Bulgaria: Association for Computational Linguistics.
  * Chakravarti et al. (2019) Chakravarti, R.; Pendus, C.; Sakrajda, A.; Ferritto, A.; Pan, L.; Glass, M.; Castelli, V.; Murdock, J. W.; Florian, R.; Roukos, S.; and Sil, A. 2019.  CFO: A Framework for Building Production NLP Systems.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations_ , 31–36. Hong Kong, China: Association for Computational Linguistics.
  * Chen et al. (2021) Chen, S.; Liu, Q.; Yu, Z.; Lin, C.-Y.; Lou, J.-G.; and Jiang, F. 2021.  ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations_ , 325–336. Online: Association for Computational Linguistics.
  * Diefenbach, Singh, and Maret (2017) Diefenbach, D.; Singh, K.; and Maret, P. 2017.  Wdaqua-core0: A question answering component for the research community.  In _Semantic Web Evaluation Challenge_ , 84–89. Springer.
  * Diefenbach et al. (2017) Diefenbach, D.; Tanon, T. P.; Singh, K. D.; and Maret, P. 2017.  Question Answering Benchmarks for Wikidata.  In _Proceedings of the ISWC 2017 Posters & Demonstrations and Industry Tracks co-located with 16th International Semantic Web Conference (ISWC 2017), Vienna, Austria, October 23rd - to - 25th, 2017._
  * Dubey et al. (2019) Dubey, M.; Banerjee, D.; Abdelkawi, A.; and Lehmann, J. 2019.  _LC-QuAD 2.0: A Large Dataset for Complex Question Answering over Wikidata and DBpedia_ , 69–78.  ISBN 978-3-030-30795-0.
  * Fu et al. (2020) Fu, B.; Qiu, Y.; Tang, C.; Li, Y.; Yu, H.; and Sun, J. 2020.  A survey on complex question answering over knowledge base: Recent advances and challenges.  _arXiv preprint arXiv:2007.13069_.
  * Jia et al. (2018) Jia, Z.; Abujabal, A.; Saha Roy, R.; Strötgen, J.; and Weikum, G. 2018.  TempQuestions: A Benchmark for Temporal Question Answering.  In _Companion Proceedings of the The Web Conference 2018_ , WWW ’18, 1057–1062. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee.  ISBN 9781450356404.
  * Kapanipathi et al. (2021) Kapanipathi, P.; Abdelaziz, I.; Ravishankar, S.; Roukos, S.; Gray, A.; Astudillo, R.; Chang, M.; Cornelio, C.; Dana, S.; Fokoue, A.; et al. 2021.  Leveraging Abstract Meaning Representation for Knowledge Base Question Answering.  _Findings of the Association for Computational Linguistics: ACL_.
  * Lee et al. (2020) Lee, Y.-S.; Astudillo, R. F.; Naseem, T.; Reddy, R. G.; Florian, R.; and Roukos, S. 2020.  Pushing the Limits of AMR Parsing with Self-Learning.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_.
  * Naseem et al. (2021) Naseem, T.; Ravishankar, S.; Mihindukulasooriya, N.; Abdelaziz, I.; Lee, Y.-S.; Kapanipathi, P.; Roukos, S.; Gliozzo, A.; and Gray, A. 2021.  A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , 256–262. Online: Association for Computational Linguistics.
  * Saxena, Chakrabarti, and Talukdar (2021) Saxena, A.; Chakrabarti, S.; and Talukdar, P. P. 2021.  Question Answering Over Temporal Knowledge Graphs.  In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , 6663–6676. Association for Computational Linguistics.
  * Saxena, Tripathi, and Talukdar (2020) Saxena, A.; Tripathi, A.; and Talukdar, P. 2020.  Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 4498–4507. Online: Association for Computational Linguistics.
  * Singh et al. (2018) Singh, K.; Both, A.; Sethupat, A.; and Shekarpour, S. 2018.  Frankenstein: a platform enabling reuse of question answering components.  In _European Semantic Web Conference_ , 624–638. Springer.
  * Sorokin and Gurevych (2018a) Sorokin, D.; and Gurevych, I. 2018a.  Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , 3306–3317. Association for Computational Linguistics.
  * Sorokin and Gurevych (2018b) Sorokin, D.; and Gurevych, I. 2018b.  Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , 3306–3317. Association for Computational Linguistics.
  * Sun et al. (2018) Sun, H.; Dhingra, B.; Zaheer, M.; Mazaitis, K.; Salakhutdinov, R.; and Cohen, W. 2018.  Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , 4231–4242. Brussels, Belgium: Association for Computational Linguistics.
  * Trivedi et al. (2017) Trivedi, P.; Maheshwari, G.; Dubey, M.; and Lehmann, J. 2017.  Lc-quad: A corpus for complex question answering over knowledge graphs.  In _Proceedings of the 16th International Semantic Web Conference (ISWC)_ , 210–218. Springer.
  * Trouillon et al. (2016) Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, E.; and Bouchard, G. 2016.  Complex Embeddings for Simple Link Prediction.  In _Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48_ , ICML’16, 2071–2080. JMLR.org.
  * Usbeck et al. (2017) Usbeck, R.; Ngonga Ngomo, A.-C.; Haarmann, B.; Krithara, A.; Röder, M.; and Napolitano, G. 2017.  7th Open Challenge on Question Answering over Linked Data (QALD-7).  In _Semantic Web Evaluation Challenge_ , 59–69. Springer International Publishing.
  * Usbeck et al. (2019) Usbeck, R.; Röder, M.; Hoffmann, M.; Conrads, F.; Huthmann, J.; Ngomo, A. N.; Demmler, C.; and Unger, C. 2019.  Benchmarking question answering systems.  _Semantic Web_ , 10(2): 293–304.
  * Vakulenko et al. (2019) Vakulenko, S.; Fernández, J.; Polleres, A.; de Rijke, M.; and Cochez, M. 2019\.  Message Passing for Complex Question Answering over Knowledge Graphs.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM2019_ , 1431–1440. Beijing, China: ACM.
  * Vrandečić and Krötzsch (2014) Vrandečić, D.; and Krötzsch, M. 2014.  Wikidata: a free collaborative knowledgebase.  _Communications of the ACM_ , 57(10): 78–85.
  * Wu et al. (2020) Wu, L.; Petroni, F.; Josifoski, M.; Riedel, S.; and Zettlemoyer, L. 2020.  Scalable Zero-shot Entity Linking with Dense Entity Retrieval.  In _EMNLP_ , 6397–6407.
  * Yih et al. (2016) Yih, W.-t.; Richardson, M.; Meek, C.; Chang, M.-W.; and Suh, J. 2016.  The Value of Semantic Parse Labeling for Knowledge Base Question Answering.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , 201–206. Berlin, Germany: Association for Computational Linguistics.
  * Zettlemoyer and Collins (2012) Zettlemoyer, L. S.; and Collins, M. 2012.  Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars.  arXiv:1207.1420.
  * Zhou et al. (2021) Zhou, J.; Naseem, T.; Astudillo, R. F.; and Florian, R. 2021.  AMR Parsing with Action-Pointer Transformer.  _https://arxiv.org/abs/2104.14674_.
  * Zou et al. (2014a) Zou, L.; Huang, R.; Wang, H.; Yu, J. X.; He, W.; and Zhao, D. 2014a.  Natural language question answering over RDF: a graph data driven approach.  In _Proceedings of the 2014 ACM SIGMOD international conference on Management of data_ , 313–324.
  * Zou et al. (2014b) Zou, L.; Huang, R.; Wang, H.; Yu, J. X.; He, W.; and Zhao, D. 2014b.  Natural Language Question Answering over RDF: A Graph Data Driven Approach.  In _Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data_ , SIGMOD ’14, 313–324. New York, NY, USA: Association for Computing Machinery.  ISBN 9781450323765.