#  Reservoir Memory Effect in the Establishment of Collective Emission: Non-
Markovianity beyond Retardation

Yu-Xiang Zhang<EMAIL_ADDRESS>Beijing National Laboratory for Condensed
Matter Physics and Institute of Physics, Chinese Academy of Sciences, Beijing
100190, China  School of Physical Sciences, University of Chinese Academy of
Sciences, Beijing 100049, China

(August 27, 2024)

###### Abstract

To establish a collective emission, the atoms in an ensemble must coordinate
their behavior by exchanging virtual photons. We study this non-Markovian
process in a subwavelength atom chain coupled to a one-dimensional (1D)
waveguide and find that retardation is not the only cause of non-Markovianity.
The other factor is the memory of the photonic environment, for which a single
excited atom needs a finite time to cross from quadratic decay, the Zeno
regime, to exponential decay. In waveguide setup, this crossover has a time
scale longer than the retardation, thus impacts on the building up of
collective behavior. By comparing a full quantum treatment with an
approximation incorporating only the retardation effect, we find that field
memory effect, characterized by the population of atomic excitation, is much
more pronounced in collective emissions than that in the decay of a single
atom.

It is well known since the 1960s Winter (1961); Newton (1961) that for an
excited atom, the exponential decay derived from the Weisskopf-Wigner
formalism Weisskopf and Wigner (1930) does not apply to the short-time
dynamics: Due to the finite memory of the photonic reservoir, the decay rate
is built up from zero quadratically at the first beginning Wilkinson  _et al._
(1997); Crespi  _et al._ (2019). It has inspired the attempts to prohibit
decaying by quickly repeating measurements Chiu  _et al._ (1977); Misra and
Sudarshan (1977); Itano  _et al._ (1990), i.e., the Zeno effect. Actually, the
duration of the non-exponential decay, characterized by the Zeno time
$\tau_{Z}$, is typically many orders of magnitude shorter than the lifetime,
rendering the field memory effect insignificant. For example, an optimal
estimation of the 2P-1S transition of the hydrogen atom reads $\tau_{Z}\sim
10^{-13}\mathrm{s}$ Zheng  _et al._ (2008). Instead of the decay of a single
atom, in this Letter, we shall show prominent reservoir memory effect in the
formation of collective emissions from a subwavelength atom array, where the
separation between two adjacent atoms, $d$, is shorter than the resonant
wavelength $\lambda$.

Emission from an ensemble shows collective behavior because the atoms are
interacting with a common radiation field Dicke (1954). For compact ensembles
and weak atom-field couplings, collective behaviors are well described by
Lehmberg’s formalism based on the Markov approximation Lehmberg (1970a, b).
For example, it predicts subradiant states with decay rates $\gamma\propto
N^{-\alpha}$ with $N$ the number of atoms in subwavelength atom arrays Asenjo-
Garcia  _et al._ (2017); Zhang and Mølmer (2019, 2020); Kornovan  _et al._
(2019). However, Markovian theories are not able to describe the process of
how the individual atoms are organized to eventually present a collective
behavior. It is necessarily non-Markovian because of the retarded nature of
photon-mediated atom-atom interaction. Then, a physical picture arises that
the Markovian description is upgraded minimally by including the delayed
feedback: Every atom starts from exponential decay independently and adjusts
the decay rate in response to the signal from another atom. This picture has
been studied for two atoms with the photonic reservoir being the free space
Milonni and Knight (1974) or 1D waveguide Gonzalez-Ballestero  _et al._
(2013); Sinha  _et al._ (2020a, b). Retardation effects of waveguide quantum
electrodynamics (QED) are also studied in Refs. Dinc and Brańczyk (2019); Shen
_et al._ (2019); Carmele  _et al._ (2020); Arranz Regidor  _et al._ (2021);
Trivedi  _et al._ (2021); Barkemeyer  _et al._ (2022).

But how does the Zeno regime come into effect? We notice that the 2P-1S
transition of the hydrogen atom has $\tau_{Z}\gg 2\pi/\omega_{0}\sim
10^{-15}\mathrm{s}$. In a subwavelength atom array, the minimal retardation
time $t_{\text{retard}}=d/c$, with $c$ the speed of light, satisfies the
inequality that $t_{\text{retard}}<\lambda/c=2\pi/\omega_{0}\ll\tau_{Z}$. It
means that the virtual photons sent from an atom have passed many other atoms,
building up cooperativeness to a certain extent before the onset of
exponential decay. Hence the establishment of collective emission and the
reduction to Markovian behavior are simultaneous processes highly intertwined,
as illustrated in Fig. 1(a). The scenario can be rephrased from the
perspective of one atom in the array: The surrounding atoms play the role of
observers that repeatedly alter its evolution every $t_{\text{retard}}$. Thus,
the memory of the photonic reservoir must have nontrivial impacts on the
establishment of collective emission.

To reveal it, we shall study the discrepancy between a full quantum treatment
and the retardation-only picture mentioned above. This discrepancy is found to
be much larger than the gap between the full quantum treatment of the decay of
a single atom and the memory-less exponential decay, making subwavelength atom
array a better system to detect the nontrivial properties of the photonic
field. In this work we will focus on the system of waveguide QED. Possible
experimental proposals and extensions to the memory effect of the free space
radiation field will also be discussed.

#### Zeno Time of the Waveguide QED.

The coupling between a 1D waveguide and two-level atoms with the ground state
$\ket{g}$ and excited state $\ket{e}$ is conventionally described by a
Hamiltonian in analogy with the multipolar gauge Hamiltonian of quantum
optics, colloquially the “$\bm{d}\cdot\bm{E}$” Hamiltonian Roy  _et al._
(2017); Chang  _et al._ (2018); Sheremet  _et al._ (2021)

$\displaystyle H_{M}=$
$\displaystyle\sum_{i=1}^{N}\omega_{0}\sigma_{i}^{\dagger}\sigma_{i}+\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\omega_{k}a_{k}^{\dagger}a_{k}$
(1)
$\displaystyle-i\sum_{i=1}^{N}\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}g_{k}\sigma_{i,X}(a_{k}e^{ikx_{i}}-a_{k}^{\dagger}e^{-ikx_{i}})$

where $\sigma_{i}=\ket{g}_{i}\bra{e}$,
$\sigma_{i,X}=\sigma_{i}+\sigma_{i}^{\dagger}$,
$\omega_{k}=v_{g}\mathinner{\\!\left\lvert k\right\rvert}$, $v_{g}$ is the
speed of light in the waveguide, $g_{k}$ is the coupling strength to be
specified, $x_{i}$ denotes the coordinate of an atom, and $v_{g}\Lambda$ is
the spectrum cutoff (here the atomic transition is not near the photonic band
edge Vats and John (1998)), $a_{k}$ ($a_{k}^{\dagger}$) is the bosonic
annihilation (generation) operator satisfying the commutation relation
$[a_{k},a_{k^{\prime}}^{\dagger}]=2\pi\delta(k-k^{\prime})$ (all others
vanish). We note that the bosons can also be surface acoustic waves Gustafsson
_et al._ (2014); Guo  _et al._ (2017).

Since the rotating-wave approximation is not proper for short-time dynamics
with a temporal resolution finer than $t_{\text{prop}}<2\pi/\omega_{0}$, the
counter-rotating terms of (1) cannot be ignored Zheng  _et al._ (2008). But
the difficulties for theoretical treatment brought by them can be folded
approximately by transforming the Hamiltonian to another gauge introduced by
Drummond Drummond (1987), which is also called the Jaynes-Cummings (JC) gauge
Stokes and Nazir (2019, 2022). The unitary transformation from Hamiltonian (1)
to the JC gauge is realized as $H_{\text{JC}}=e^{-iS}H_{M}e^{iS}$ where

$S=\sum_{i=1}^{N}\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\frac{g_{k}}{\omega_{0}+\omega_{k}}\sigma_{i,X}(b_{k}e^{ikx_{i}}+b_{k}^{\dagger}e^{-ikx_{i}}).$
(2)

At the first order of $g_{k}$, we obtain

$\displaystyle H_{\text{JC}}=$
$\displaystyle\sum_{i=1}^{N}\omega_{0}\sigma_{i}^{\dagger}\sigma_{i}+\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\omega_{k}a_{k}^{\dagger}a_{k}$
(3)
$\displaystyle-i\sum_{i=1}^{N}\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}g_{k}^{\text{JC}}(\sigma_{i}^{\dagger}a_{k}e^{ikx_{i}}-\sigma_{i}a_{k}^{\dagger}e^{-ikx_{i}})$

where $g_{k}^{\text{JC}}=2g_{k}\omega_{0}/(\omega_{k}+\omega_{0})$. One can
further introduce the corrections to $\omega_{0}$ at the order of $g_{k}^{2}$
to obtain a better approximation Wang  _et al._ (2010); Li  _et al._ (2012).

An important property of $H_{\text{JC}}$ is that the ground state of the full
Hamiltonian $\ket{G}=\ket{g,\emptyset}$ is identical to the ground state of
the free Hamiltonian, where $\emptyset$ denotes the field vacuum. In contrast,
the same physical state reads $e^{iS}\ket{G}$ for $H_{M}$, an entangled state
between the atoms and the field. To coordinate the theoretical
convention–dividing the Hilbert space into the atomic sector and the field
sector and using an initial state in the product of them–with the laboratory
practice of exciting atoms from the ground state of the full Hamiltonian (one
can never turn off the atom-field interaction), we choose to work with
$H_{\text{JC}}$ in this work.

Figure 1: (a) Sketch of the idea: For closely separated atoms, the process of
building up collective behaviors by exchanging virtual photons (dotted blue
arrows) and the process to build up the Markovian decay rate $\Gamma_{0}$ (due
to the memory effect of the field represented by the red back-flow arrows) are
highly intertwined. (b) The instantaneous emission rate of a single atom
coupled to the waveguide, $\Gamma_{\text{inst}}(t)$ (in the unit of
$\Gamma_{0}$) for $t\leq 10/\omega_{0}$. (c) Zoom-out view of (b) for $t\leq
50/\omega_{0}$.

Next, we specify the coupling strength $g_{k}$. In the majority of literature,
a local atom-field interaction is assumed Shen and Fan (2005) so that $g_{k}$
is a $k$-independent constant Roy  _et al._ (2017); Chang  _et al._ (2018);
Sheremet  _et al._ (2021), which can be expressed through the Markovian decay
rate $\Gamma_{0}$ as $g^{2}_{k}=\Gamma_{0}v_{g}/2$. Another option is
$g_{k}\propto\mathinner{\\!\left\lvert k\right\rvert}^{1/2}$, the same as the
multipolar Hamiltonian of atoms in the free space Sheremet  _et al._ (2021)
where the atoms couple to the transverse component of the displacement field
Cohen-Tannoudji  _et al._ (1997). Explicitly, we have
$g^{2}_{k}=\Gamma_{0}v_{g}\mathinner{\\!\left\lvert k\right\rvert}/(2k_{0})$
with $k_{0}=\omega_{0}/v_{g}$. Waveguide QED with the above two kinds of
$g_{k}$ has a constant and a linear spectral density, hence will be denoted by
“const-wQED” and “lin-wQED”, respectively.

Now it is ready to calculate the Zeno time. Given an initial state
$\ket{\Psi_{0}}$ and a general unitary $\exp(-iHt)$, the Zeno time $\tau_{Z}$
is defined from the short-time expansion of the non-decay probability

$\mathinner{\\!\left\lvert\braket{\Psi_{0}}{e^{-iHt}}{\Psi_{0}}\right\rvert}^{2}=1-t^{2}/\tau_{Z}^{2}+\cdots.$
(4)

The Zeno time is just a characterization but not exactly the duration of non-
exponential decay. Nevertheless, we substitute $H_{\text{JC}}$ and
$\ket{e,\emptyset}$ into Eq. (4) and obtain

$\tau_{Z}^{-2}=\begin{cases}2\Gamma_{0}\omega_{0}/\pi&\text{const.}\\\
2\Gamma_{0}\omega_{0}\ln(\Lambda/k_{0})/\pi&\text{lin.}\end{cases}$ (5)

Remarkably, $\tau_{Z}$ of const-wQED is $\Lambda$-independent. The above
results imply that a subwavelength atom array has $\tau_{Z}\gg 1/\omega_{0}$
provided the weak couplings $\omega_{0}/\Gamma_{0}\gg 1$ and
$\ln(\Lambda/k_{0})$, respectively.

#### Equation of Motion.

Since $H_{\text{JC}}$ preserves the number of excitations, we can use the
singly-excited ansatz

$\ket{\Psi(t)}=\sum_{i=1}^{N}\alpha_{i}(t)\sigma_{i}^{\dagger}\ket{G}+\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\beta_{k}(t)a_{k}^{\dagger}\ket{G}.$
(6)

Then the Schrödinger equation in the interaction picture implies the following
integro-differential equation for the atomic amplitudes,

$\displaystyle\frac{d}{dt}\alpha_{i}(t)=-\sum_{j=1}^{N}\int_{-\Lambda}^{\Lambda}$
$\displaystyle\frac{dk}{2\pi}\mathinner{\\!\left\lvert
g_{k}^{\text{JC}}\right\rvert}^{2}\int_{0}^{t}d\tau\alpha_{j}(\tau)$ (7)
$\displaystyle\times e^{ik(x_{i}-x_{j})+i(\omega_{k}-\omega_{0})(t-\tau)}.$

See further manipulation of this equation in the Supplemental Material sp .
Both the amplitudes $\alpha_{i}(t)$ and the photonic population
$\int\mathinner{\\!\left\lvert\beta_{k}\right\rvert}^{2}dk/(2\pi)$ are
calculated so that the normalization condition $\braket{\Psi(t)}{\Psi(t)}=1$
can be used to reduce numerical error.

To have an idea of the field memory effect, we consider the decay of a single
atom and plot in Figs. 1(b,c) the instantaneous decay rate
$\Gamma_{\text{inst}}(t)\equiv-\frac{d}{dt}\ln P_{e}(t)$, where $P_{e}(t)$
denotes the population in the atomic excited state. Therein, we apply three
values of $\Gamma_{0}/\omega_{0}$, a slightly strong coupling $10^{-2}$ and
weak couplings $10^{-3}$ and $10^{-6}$. The cutoff is set at
$\Lambda/k_{0}=10^{4}$. We find that curves belonging to the three
$\Gamma_{0}/\omega_{0}$ almost overlap. For each value of
$\Gamma_{0}/\omega_{0}$, $\Gamma_{\text{inst}}(t)$ of lin-wQED (dashed curve)
increases faster at first ($t\lesssim 1/\omega_{0}$) but soon becomes more
gradual than $\Gamma_{\text{inst}}(t)$ of const-wQED (solid curve). The later
increases to roughly $1.4\Gamma_{0}$ and starts to oscillate around
$\Gamma_{0}$ with a waning amplitude. As shown in Fig. 1(c), the oscillation
is still visible at $t=50/\omega_{0}$, equivalent to a propagation length of
$8\lambda$.

Figure 2: Decay of the superradiant timed-Dicke state $\ket{\Psi_{k_{0}}}$.
(a) The instantaneous total decay rate $\Gamma_{\text{inst}}(t)$ for
$\omega_{0}t\in[0,10]$ of a chain of 20 atoms with separation $d=0.1\pi/k_{0}$
and $\Gamma_{0}/\omega_{0}=10^{-4}$. (b) The instantaneous decay rate of five
selected atoms [the legend is same as (a)]. (c) $\Delta P_{e}(t)$ (in the unit
of $\Gamma_{0}/\omega_{0}$) of atom No. 1. The parameters are
$d=0.1\pi/k_{0}$, $N=10$, and $\Gamma_{0}/\omega_{0}=10^{-2}$, $10^{-3}$, and
$10^{-6}$. Predictions of Eq. (7) of the constant case, the linear case, and
Eq. (8) are plotted by concrete, dashed, and dotted curves, respectively. (d)
Same as (c) but for atom No. 10. (e) Left panel: $\Delta P_{e}(t)$ of the last
atom (No. 10) for $\omega_{0}t\in[45,\,50]$; Right panels: $\Delta P_{e}(t)$
of a single atom ($N=1$) coupled to the waveguide with
$\Gamma_{0}/\omega_{0}=10^{-2}$, $10^{-3}$ and $10^{-6}$, respectively.

In experiments, measuring $\Gamma_{\text{inst}}(t)$ will be extremely
demanding in the detector temporal resolution. If evidence of the memory
effect can be obtained by measuring the change of population $\Delta
P_{e}(t)=P_{e}(t)-P_{e}(0)$ at some relatively large $t$, the requirement of
temporal resolution can be significantly relaxed. However, due to the
oscillations of $\Gamma_{\text{inst}}(t)$, it is perceivable, and will be
shown in Fig. 2(e), that $\Delta P_{e}(t)$ is quickly averaged out to the
memory-less Markovian result.

Fortunately, we will see that it is not the case for collective emissions. To
study the field memory effect while excluding the non-Markovianity attributed
to retardation, Eq. (7) will be compared with the following one reproduced
from Refs. Sinha  _et al._ (2020a, b)

$\frac{d}{dt}\alpha_{i}(t)=-\frac{\Gamma_{0}}{2}\bigg{[}\alpha_{i}(t)+\sum_{j\neq
i}e^{ik_{0}r_{ij}}\alpha_{j}(t-\frac{r_{ij}}{v_{g}})\Theta(t-\frac{r_{ij}}{v_{g}})\bigg{]}$
(8)

where $r_{ij}=\mathinner{\\!\left\lvert x_{i}-x_{j}\right\rvert}$ and
$\Theta(t)=1$ for $t>0$ and vanishes otherwise. One can derive this equation
from Eq. (7) of either const-wQED or lin-wQED at the limit
$\Lambda\rightarrow\infty$ by an approximation presented in Ref. Milonni and
Knight (1974); sp . Predictions of Eq. (8) is labeled by “retard.”, remarking
that it details the physical picture which goes beyond the Markovian formalism
minimally by incorporating only the retardation effect.

#### Superradiance.

We start from a chain of atoms initialized in the timed-Dicke state

$\ket{\Psi_{k}}=\frac{1}{\sqrt{N}}\sum_{j=1}^{N}e^{ikx_{j}}\sigma_{j}^{\dagger}\ket{G}.$
(9)

The preparation of such states is experimentally accessible Röhlsberger  _et
al._ (2010); Corzo  _et al._ (2019) and $\ket{\Psi_{k_{0}}}$ is the single-
photon superradiant state Scully  _et al._ (2006). We assume $N=20$,
$\Gamma_{0}/\omega_{0}=10^{-4}$, and a short atom-atom separation
$d=0.1\pi/k_{0}$. Larger values of $d$ weaken the field memory effect. See
results of $d=0.5\pi/k_{0}$ in the Supplemental Material sp . The
instantaneous decay rate of the whole chain predicted by Eqs. (7) and (8) are
plotted in Fig. 2(a) for $t\leq 10/\omega_{0}$. The former shows continuous
growth while the later gives a step-like increase. They agree well after
$t\approx 6/\omega_{0}$.

Then we plot in Fig. 2(b) the instantaneous decay rate of five atoms in the
chain, No. 1, 2, 5, 10, 20 ($x_{i}<x_{j}$ if $i<j$). It can be seen that they
decay in different paces. Atom No. 1 decays slowly while Atom No. 20
accelerates to $20\Gamma_{0}$. The plots show agreement between the three
curves for atoms in the middle, i.e., atom No. 10, but not those near the
ends. For Atom No. 1, the grey curve given by Eq. (8) shows two cycles of
emission ($\Gamma_{\text{inst}}>0$) and absorption ($\Gamma_{\text{inst}}<0$)
followed by an almost suppressed decay rate; the red curve of const-wQED shows
only emission; the blue curve of lin-wQED exhibits emission at first but
quickly turns to absorption.

To further illuminate the discrepancy, we plot $\Delta P_{e}(t)$ of Atom No. 1
for a longer time window $t\leq 50/\omega_{0}$ in Fig. 2(c). Note that an atom
in state (9) has only $1/N$ initial excitation. To avoid a too small $\Delta
P_{e}(t)$, we assume a smaller chain of $N=10$ atoms. Meanwhile, to compare
with the curves in Fig. 1, we consider the same three values of
$\Gamma_{0}/\omega_{0}$. Figure 2(c) shows that for relatively weak couplings
($\Gamma_{0}/\omega_{0}=10^{-3}$ and $10^{-6}$) there is a gap of $\lesssim
10\Gamma_{0}/\omega_{0}$ between the predictions of Eq. (7) of const-wQED and
lin-wQED, while the predictions of Eq. (8) is roughly in the middle of them.
Moreover, for const-wQED with $\Gamma_{0}/\omega_{0}=10^{-2}$, we can see the
approaching to the prediction of Eq. (8) in the plotted regime. Similarly, we
plot $\Delta P_{e}(t)$ of the last atom (No. 10) in Fig. (2)(d). It shows gaps
of roughly the same scale (except for the case of
$\Gamma_{0}/\omega_{0}=10^{-2}$ where the atom-field coupling is stronger). A
zoom-in view for $\omega_{0}t\in[45,50]$ is given in the left-most panel of
Fig. 2(e).

As promised above, we plot $\Delta P_{e}(t)$ of the decay of a single atom for
$\omega_{0}t\in[45,50]$ in the right three panels of Fig. 2(e), in parallel to
the curves of $\Gamma_{\text{inst}}(t)$ in Fig. 1(b,c). We see that $\Delta
P_{e}(t)$ of const-wQED almost overlaps with the Markovian exponential decay,
and $\Delta P_{e}(t)$ of lin-wQED differs from them by much less than
$1\Gamma_{0}/\omega_{0}$. Thus, in terms of $\Delta P_{e}(t)$, we conclude
that the field memory effect is much more prominent in collective emissions
than the decay of a single atom. Moreover, although our plots terminate at
$t=50/\omega_{0}$, a much larger $t$ can be expected only after which the
predictions of Eq. (7) tend toward the predictions of Eq. (8). Consequently,
the temporal resolution needed to observe the field memory effect is
significantly relaxed.

Figure 3: Decay of a subradiant eigenstate. (a) The instantaneous decay rate
$\Gamma_{\text{inst}}(t)$ (in the unit of $\Gamma_{0}$) of 20 atoms
initialized in $\ket{\Psi_{\text{sub}}}$. The other parameters are same as
Fig. 2(a). (b) The total change of population on the excited state $\Delta
P_{e}(t)$ (in the unit of $\Gamma_{0}/\omega_{0}$).

#### Subradiance.

Now we study the decay of a subradiant state
$\ket{\Psi_{\text{sub}}}=(\ket{\Psi_{k}}-\ket{\Psi_{-k}})/\sqrt{2}$ with
$kd=\pi N/(N+1)$. It is an eigenstate of the effective non-Hermitian
Hamiltonian obtained by the Markovian approximation Asenjo-Garcia  _et al._
(2017); Zhang and Mølmer (2019, 2020). We suppose the same parameters as in
Fig. 2(a,b) and plot $\Gamma_{\text{inst}}(t)$ of the whole chain in Fig.
3(a). In all cases, the subradiance is built through quick oscillations
between emissions and absorptions. But the amplitudes of oscillation are
different: the histogram-like curve of Eq. (8) has the largest amplitudes,
while the amplitudes of const-wQED is the modest. In Fig. 3(b), we plot
$\Delta P_{e}(t)$ of the whole chain. It also shows apparent relative
discrepancies between the three predictions. But because the state is
subradiant, the values of $\Delta P_{e}(t)$ are comparable to the decay of a
single atom.

#### Discussions.

It is of fundamental interest to extend from 1D waveguide to the 3D free
space. But a controversial issue is on which Hamiltonian should all
calculations be based. Most works chose the Coulomb gauge ($\bm{A}\cdot\bm{p}$
interaction) disregarding the $\bm{A}^{2}$ term, see, e.g., Refs. Moses
(1973); Wódkiewicz and Eberly (1976); Seke and Herfort (1988); Facchi and
Pascazio (1998); Berman and Ford (2010). Other gauges are seldom used. And
recently Hamiltonians of quantum optics, spontaneous emission rate and Lamb
shift is revisited by causal perturbation theory Marzlin and Fitzgerald
(2018). Among them, the Coulomb-gauge Hamiltonian without both the
$\bm{A}^{2}$ term and the rotating-wave approximation gives
$\tau_{Z}^{-2}\sim\ln\Lambda$ Zheng  _et al._ (2008), the same as lin-wQED.
Thus, we expect the same field memory effect, or perhaps even more pronounced,
because the resonant dipole-dipole interaction in the 3D free space diverges
as $1/r^{3}$ for short distances, giving rise to strong photon blockade Cidrim
_et al._ (2020). In this sense, non-Markovianity beyond retardation of
collective emissions from a small subwavelength atom array might be viewed as
a probe to resolve which theory describes the real physics.

For experimental feasibility, the crux is to have a temporal resolution at the
level of $\omega_{0}^{-1}$ and subwavelength atom-atom separation. Among the
various platforms of waveguide QED Sheremet  _et al._ (2021), the
superconducting circuit works in the microwave regime with a centimeter-long
wavelength. Thus, the requirement on a temporal resolution is not too harsh,
and fabricating the transmon qubits into a subwavelength chain is
straightforward Mirhosseini  _et al._ (2019).

#### Conclusions.

We have studied the early stage of the decay of a collective excitation in
subwavelength atom arrays coupled to a 1D waveguide. It is shown that
retardation is not the only cause of non-Markovian effects, the memory of the
photonic reservoir also strongly impact on the way how collective emissions
are shaped. This is verified by calculations of the instantaneous decay rate
and the change of population in the atomic excited state. In particular, the
later (in the single-photon superradiant state) provides signals of memory
effect more prominent than the decay of a single atom, and significantly
relaxes the requirement on temporal resolution. For future works, one may
inspect the non-Markovianity in collective emissions quantitatively using the
tools of non-Markovian open systems Rivas  _et al._ (2014); Breuer  _et al._
(2016); de Vega and Alonso (2017). It is also important to study the possible
experimental signatures of the memory effect of the 3D free space.

###### Acknowledgements.

The author thanks Klaus Mølmer for valuable comments and acknowledges the
financial support from the startup grant of IOP-CAS.

## References

  * Winter (1961) R. G. Winter, Phys. Rev. 123, 1503 (1961).
  * Newton (1961) R. G. Newton, Annals of Physics 14, 333 (1961).
  * Weisskopf and Wigner (1930) V. Weisskopf and E. Wigner, Zeitschrift für Physik 63, 54 (1930).
  * Wilkinson  _et al._ (1997) S. R. Wilkinson, C. F. Bharucha, M. C. Fischer, K. W. Madison, P. R. Morrow, Q. Niu, B. Sundaram, and M. G. Raizen, Nature 387, 575 (1997).
  * Crespi  _et al._ (2019) A. Crespi, F. V. Pepe, P. Facchi, F. Sciarrino, P. Mataloni, H. Nakazato, S. Pascazio, and R. Osellame, Phys. Rev. Lett. 122, 130401 (2019).
  * Chiu  _et al._ (1977) C. B. Chiu, E. C. G. Sudarshan, and B. Misra, Phys. Rev. D 16, 520 (1977).
  * Misra and Sudarshan (1977) B. Misra and E. C. G. Sudarshan, Journal of Mathematical Physics 18, 756 (1977).
  * Itano  _et al._ (1990) W. M. Itano, D. J. Heinzen, J. J. Bollinger, and D. J. Wineland, Phys. Rev. A 41, 2295 (1990).
  * Zheng  _et al._ (2008) H. Zheng, S. Y. Zhu, and M. S. Zubairy, Phys. Rev. Lett. 101, 200404 (2008).
  * Dicke (1954) R. H. Dicke, Phys. Rev. 93, 99 (1954).
  * Lehmberg (1970a) R. H. Lehmberg, Phys. Rev. A 2, 883 (1970a).
  * Lehmberg (1970b) R. H. Lehmberg, Phys. Rev. A 2, 889 (1970b).
  * Asenjo-Garcia  _et al._ (2017) A. Asenjo-Garcia, M. Moreno-Cardoner, A. Albrecht, H. J. Kimble, and D. E. Chang, Phys. Rev. X 7, 031024 (2017).
  * Zhang and Mølmer (2019) Y.-X. Zhang and K. Mølmer, Phys. Rev. Lett. 122, 203605 (2019).
  * Zhang and Mølmer (2020) Y.-X. Zhang and K. Mølmer, Phys. Rev. Lett. 125, 253601 (2020).
  * Kornovan  _et al._ (2019) D. F. Kornovan, N. V. Corzo, J. Laurat, and A. S. Sheremet, Phys. Rev. A 100, 063832 (2019).
  * Milonni and Knight (1974) P. W. Milonni and P. L. Knight, Phys. Rev. A 10, 1096 (1974).
  * Gonzalez-Ballestero  _et al._ (2013) C. Gonzalez-Ballestero, F. J. García-Vidal, and E. Moreno, New Journal of Physics 15, 073015 (2013).
  * Sinha  _et al._ (2020a) K. Sinha, P. Meystre, E. A. Goldschmidt, F. K. Fatemi, S. L. Rolston, and P. Solano, Phys. Rev. Lett. 124, 043603 (2020a).
  * Sinha  _et al._ (2020b) K. Sinha, A. González-Tudela, Y. Lu, and P. Solano, Phys. Rev. A 102, 043718 (2020b).
  * Dinc and Brańczyk (2019) F. Dinc and A. M. Brańczyk, Phys. Rev. Res. 1, 032042 (2019).
  * Shen  _et al._ (2019) H. Z. Shen, S. Xu, H. T. Cui, and X. X. Yi, Phys. Rev. A 99, 032101 (2019).
  * Carmele  _et al._ (2020) A. Carmele, N. Nemet, V. Canela, and S. Parkins, Phys. Rev. Res. 2, 013238 (2020).
  * Arranz Regidor  _et al._ (2021) S. Arranz Regidor, G. Crowder, H. Carmichael, and S. Hughes, Phys. Rev. Res. 3, 023030 (2021).
  * Trivedi  _et al._ (2021) R. Trivedi, D. Malz, S. Sun, S. Fan, and J. Vučković, Phys. Rev. A 104, 013705 (2021).
  * Barkemeyer  _et al._ (2022) K. Barkemeyer, A. Knorr, and A. Carmele, Phys. Rev. A 106, 023708 (2022).
  * Roy  _et al._ (2017) D. Roy, C. M. Wilson, and O. Firstenberg, Rev. Mod. Phys. 89, 021001 (2017).
  * Chang  _et al._ (2018) D. E. Chang, J. S. Douglas, A. González-Tudela, C.-L. Hung, and H. J. Kimble, Rev. Mod. Phys. 90, 031002 (2018).
  * Sheremet  _et al._ (2021) A. S. Sheremet, M. I. Petrov, I. V. Iorsh, A. V. Poshakinskiy, and A. N. Poddubny, “Waveguide quantum electrodynamics: collective radiance and photon-photon correlations,” (2021), arXiv:2103.06824 [quant-ph] .
  * Vats and John (1998) N. Vats and S. John, Phys. Rev. A 58, 4168 (1998).
  * Gustafsson  _et al._ (2014) M. V. Gustafsson, T. Aref, A. F. Kockum, M. K. Ekström, G. Johansson, and P. Delsing, Science 346, 207 (2014).
  * Guo  _et al._ (2017) L. Guo, A. Grimsmo, A. F. Kockum, M. Pletyukhov, and G. Johansson, Phys. Rev. A 95, 053821 (2017).
  * Drummond (1987) P. D. Drummond, Phys. Rev. A 35, 4253 (1987).
  * Stokes and Nazir (2019) A. Stokes and A. Nazir, Nature Communications 10 (2019), 10.1038/s41467-018-08101-0.
  * Stokes and Nazir (2022) A. Stokes and A. Nazir, Rev. Mod. Phys. 94, 045003 (2022).
  * Wang  _et al._ (2010) D.-w. Wang, Z.-h. Li, H. Zheng, and S.-y. Zhu, Phys. Rev. A 81, 043819 (2010).
  * Li  _et al._ (2012) Y. Li, J. Evers, H. Zheng, and S.-Y. Zhu, Phys. Rev. A 85, 053830 (2012).
  * Shen and Fan (2005) J.-T. Shen and S. Fan, Phys. Rev. Lett. 95, 213001 (2005).
  * Cohen-Tannoudji  _et al._ (1997) C. Cohen-Tannoudji, J. Dupont-Roc, and G. Grynberg,  _Photons and Atoms: Introduction to Quantum Electrodynamics_ , 1st ed. (Wiley-VCH, 1997).
  * (40) _Supplemental Material_.
  * Röhlsberger  _et al._ (2010) R. Röhlsberger, K. Schlage, B. Sahoo, S. Couet, and R. Rüffer, Science 328, 1248 (2010).
  * Corzo  _et al._ (2019) N. V. Corzo, J. Raskop, A. Chandra, A. S. Sheremet, B. Gouraud, and J. Laurat, Nature 566, 359 (2019).
  * Scully  _et al._ (2006) M. O. Scully, E. S. Fry, C. H. R. Ooi, and K. Wódkiewicz, Phys. Rev. Lett. 96, 010501 (2006).
  * Moses (1973) H. E. Moses, Phys. Rev. A 8, 1710 (1973).
  * Wódkiewicz and Eberly (1976) K. Wódkiewicz and J. Eberly, Annals of Physics 101, 574 (1976).
  * Seke and Herfort (1988) J. Seke and W. N. Herfort, Phys. Rev. A 38, 833 (1988).
  * Facchi and Pascazio (1998) P. Facchi and S. Pascazio, Physics Letters A 241, 139 (1998).
  * Berman and Ford (2010) P. R. Berman and G. W. Ford, Phys. Rev. A 82, 023818 (2010).
  * Marzlin and Fitzgerald (2018) K.-P. Marzlin and B. Fitzgerald, Journal of Mathematical Physics 59, 042103 (2018).
  * Cidrim  _et al._ (2020) A. Cidrim, T. S. do Espirito Santo, J. Schachenmayer, R. Kaiser, and R. Bachelard, Phys. Rev. Lett. 125, 073601 (2020).
  * Mirhosseini  _et al._ (2019) M. Mirhosseini, E. Kim, X. Zhang, A. Sipahigil, P. B. Dieterle, A. J. Keller, A. Asenjo-Garcia, D. E. Chang, and O. Painter, Nature 569, 692 (2019).
  * Rivas  _et al._ (2014) Á . Rivas, S. F. Huelga, and M. B. Plenio, Reports on Progress in Physics 77, 094001 (2014).
  * Breuer  _et al._ (2016) H.-P. Breuer, E.-M. Laine, J. Piilo, and B. Vacchini, Rev. Mod. Phys. 88, 021002 (2016).
  * de Vega and Alonso (2017) I. de Vega and D. Alonso, Rev. Mod. Phys. 89, 015001 (2017).

##  Appendix A Supplemental Material

###  A.1 A. Kernel of the Integro-Differential Equation

In our numerical calculations, the integro-differential equation (7) is
transformed to an integral equation

$\alpha_{i}(t)-\alpha_{i}(0)=i\frac{2\Gamma_{0}}{\pi}\int_{0}^{t}d\tau[A_{ij}(t-\tau)]\alpha_{j}(\tau),$
(10)

where the kernel is specified in the below. To determine the kernels in the
above integral equations, we introduce the unit less notations
$\phi=\omega_{0}(t-\tau)$, $r_{ij}=k_{0}\mathinner{\\!\left\lvert
x_{i}-x_{j}\right\rvert}$ and a short hand replacement
$\Lambda/k_{0}\rightarrow\Lambda$. Then we have that in the constant case,

$A^{\text{const.}}_{ij}(\phi)=\int_{0}^{\Lambda}dx\frac{\cos(xr_{ij})}{(1+x)^{2}}\frac{1-e^{i(1-x)\phi}}{x-1}=\frac{1}{4}(I_{1}-I_{2}-2I_{3}),$
(11)

and in the linear case,

$A^{\text{lin.}}_{ij}(\phi)=\int_{0}^{\Lambda}dx\frac{x\cos(xr_{ij})}{(1+x)^{2}}\frac{1-e^{i(1-x)\phi}}{x-1}=\frac{1}{4}(I_{1}-I_{2}+2I_{3});$
(12)

where the abbreviated terms are

$(I1)\;\int_{0}^{\Lambda}dx\frac{\cos(xr_{ij})}{x-1}(1-e^{i(1-x)\phi})=\bigg{[}\cos(r_{ij})\mathrm{ci}(xr_{ij})-\sin(r_{ij})\mathrm{si}(xr_{ij})-\frac{e^{ir_{ij}}}{2}\mathrm{csi}(xr^{-}_{ij})-\frac{e^{-ir_{ij}}}{2}\mathrm{csi}^{*}(xr^{+}_{ij})\bigg{]}\bigg{|}_{-1}^{\Lambda-1}$
(13a) $\displaystyle(I2)$
$\displaystyle\int_{0}^{\Lambda}dx\frac{\cos(xr_{ij})}{x+1}(1-e^{i(1-x)\phi})$
(13b) $\displaystyle=$
$\displaystyle\bigg{[}\cos(r_{ij})\mathrm{ci}(xr_{ij})+\sin(r_{ij})\mathrm{si}(xr_{ij})-\frac{e^{i(\phi-r^{-}_{ij})}}{2}\mathrm{csi}(xr^{-}_{ij})-\frac{e^{i(\phi+r^{+}_{ij})}}{2}\mathrm{csi}^{*}(xr^{+}_{ij})\bigg{]}\bigg{|}_{1}^{\Lambda+1}$
$\displaystyle(I3)$
$\displaystyle\int_{0}^{\Lambda}dx\frac{\cos(xr)}{(1+x)^{2}}[1-e^{i(1-x)\phi}]=\frac{\cos(\Lambda
r)}{\Lambda+1}(e^{-i(\Lambda+1)\phi}-1)+1-e^{i\phi}$ (13c)
$\displaystyle\qquad-\bigg{\\{}r\big{[}\cos(r)\mathrm{si}(xr)-\sin(r)\mathrm{ci}(xr)\big{]}+\frac{i}{2}r_{-}e^{i(2\phi-r)}\mathrm{csi}(xr_{-})-\frac{i}{2}r_{+}e^{i(2\phi+r)}\mathrm{csi}^{*}(xr_{+})\bigg{\\}}\bigg{|}_{1}^{\Lambda+1}$

where $r_{ij}^{\pm}=r_{ij}\pm\phi$, and we have introduced

$\mathrm{csi}(x)\equiv\mathrm{ci}(x)+i\mathrm{si}(x)\equiv-\int_{x}^{\infty}dt\frac{e^{it}}{t}.$

The population of the bosonic field excitation

$N_{b}(t)=\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\mathinner{\\!\left\lvert\beta_{k}(t)\right\rvert}^{2}$

is obtained from the atomic amplitudes by

$N_{b}(t)=\sum_{i,j=1}^{N}\int_{0}^{t}d\tau\int_{0}^{t}d\tau^{\prime}B_{ij}(\tau-\tau^{\prime})\alpha_{i}(\tau)\alpha_{j}(\tau^{\prime}),$
(14)

where the kernel is defined by

$B_{ij}(t-\tau)=\int_{-\Lambda}^{\Lambda}\frac{dk}{2\pi}\mathinner{\\!\left\lvert
g_{k}^{\text{JC}}\right\rvert}^{2}e^{-ik(x_{i}-x_{j})-i(\omega_{0}-\omega_{k})(t-\tau)}$
(15)

For the constant case, the kernel of photon population is evaluated as

$\displaystyle\;B_{ij}^{\text{const.}}(\phi)$
$\displaystyle=\frac{\Gamma_{0}\omega_{0}}{\pi}\int_{0}^{\Lambda}dx\frac{1}{(1+x)^{2}}2\cos(xr_{ij})e^{i(x-1)\phi}$
(16)
$\displaystyle=\frac{\Gamma_{0}\omega_{0}e^{-i\phi}}{\pi}\bigg{[}-\frac{2\cos(xr_{ij})}{1+x}e^{ix\phi}\big{|}_{0}^{\Lambda}$
$\displaystyle\quad+\sum_{s=\pm}i\phi_{s}e^{-i\phi_{s}}\mathrm{csi}(x\phi_{s})\big{|}_{1}^{\Lambda+1}\bigg{]}$

where $\phi_{\pm}=\phi\pm r_{ij}$; for the linear case, we have

$\displaystyle B_{ij}^{\text{lin.}}(\phi)$
$\displaystyle=\frac{\Gamma_{0}\omega_{0}}{\pi}\int_{0}^{\Lambda}dx\frac{x}{(1+x)^{2}}2\cos(xr_{ij})e^{i(x-1)\phi}$
(17)
$\displaystyle=\sum_{s=\pm}e^{-i\phi_{s}}\mathrm{csi}(x\phi_{s})\big{|}_{1}^{\Lambda+1}-B_{ij}^{\text{const}}(\phi).$

###  A.2 B. Derivation of Eq. (8) of the main text

In the derivation of Eq. (8), the trick is to apply the approximation

$\int\frac{dk}{2\pi}f(k)e^{i(k_{0}-k)x}\approx f(k_{0})\delta(x)$ (18)

for a function $f(k)$ that changes slowly with $k$. By this approximation,
only the coupling strength at the resonant momentum, $g_{k_{0}}$, shows up in
the end. Thus, there is no difference to start the derivation from either
const-wQED or lin-wQED.

###  A.3 C. Results of $d=0.5\pi/d$

Results presented in the main text belong to subwavelength arrays with the
atom-atom separation $d=0.1\pi/k_{0}$. The field memory effect will be
diminished if $d$ becomes larger. To see it we consider a chain of 10 atoms
with $d=0.5\pi/d$ and $\Gamma_{0}/\omega_{0}=10^{-4}$, and suppose the chain
is initialized in the single-photon superradiant state $\ket{\Psi_{k_{0}}}$.
We show in Fig. 4 the instantaneous decay rate of the whole chain and of five
atoms, No. 1, 2, 5, 6, 10. It can be seen that the differences between const-
wQED, lin-wQED and Eq. (8) of the main text are less prominent than the case
of $d=0.1\pi/k_{0}$.

Figure 4:  The instantaneous decay rate $\Gamma_{\text{inst}}(t)$ (in the unit
of $\Gamma_{0}$) of a chain of 10 atoms initialized in $\ket{\Psi_{k_{0}}}$
separated by $d=0.5\pi/k_{0}$.


11institutetext: Innopolis University, Universitetskaya, 1, 420500, Innopolis,
Russia
11email: {a.khusainova<EMAIL_ADDRESS>22institutetext: University of
Campinas (UNICAMP), 13083-970, Campinas, Brazil
22email<EMAIL_ADDRESS>
# SART - Similarity, Analogies, and Relatedness for Tatar Language: New
Benchmark Datasets for Word Embeddings Evaluation

Albina Khusainova 11 Adil Khan 11 Adín Ramírez Rivera 22

###### Abstract

There is a huge imbalance between languages currently spoken and corresponding
resources to study them. Most of the attention naturally goes to the “big”
languages—those which have the largest presence in terms of media and number
of speakers. Other less represented languages sometimes do not even have a
good quality corpus to study them. In this paper, we tackle this imbalance by
presenting a new set of evaluation resources for Tatar, a language of the
Turkic language family which is mainly spoken in Tatarstan Republic, Russia.

We present three datasets: Similarity and Relatedness datasets that consist of
human scored word pairs and can be used to evaluate semantic models; and
Analogies dataset that comprises analogy questions and allows to explore
semantic, syntactic, and morphological aspects of language modeling. All three
datasets build upon existing datasets for the English language and follow the
same structure. However, they are not mere translations. They take into
account specifics of the Tatar language and expand beyond the original
datasets. We evaluate state-of-the-art word embedding models for two languages
using our proposed datasets for Tatar and the original datasets for English
and report our findings on performance comparison.

The datasets are available at https://github.com/tat-nlp/SART

###### Keywords:

word embeddings, evaluation, analogies, similarity, relatedness, low-resourced
languages, Turkic languages, Tatar language

##  1 Introduction

Word embeddings have become almost an intrinsic component of NLP systems based
on deep learning. Therefore, there is a need for their evaluation and
comparison tools. It is not always computationally feasible to evaluate
embeddings directly on the task they were built for; that is why there is a
need for inexpensive preliminary evaluations. For example, there are
similarity/relatedness tests, where human judgements are obtained for pairs of
words, such that each pair is rated based on the degree of
similarity/relatedness between the words, and then model scores are compared
to humans judgements. Another type is analogies test—questions of the form
A:B::C:D, meaning A to B is as C to D, and D is to be predicted. While such
tests exist for widespread languages, e.g., SimLex-999 [8] was translated to
four major languages [9], less represented languages suffer from the absence
of such resources. In this work, we attempt to close this gap for Tatar, an
agglutinative language with rich morphology, by proposing three evaluation
datasets. In general, their use is not limited to embeddings evaluation; they
can benefit any system which models semantic/morphosyntactic relationships.
For example, they can be used for automated thesauri, dictionary building,
machine translation [7], or semantic parsing [2].

##  2 Related Works

The most well-known similarity/relatedness datasets for English are RG [13],
WordSim-353 [6], MEN [5], and SimLex-999 [8]. The problem with WordSim-353 and
MEN is that there’s no distinction between similarity and relatedness
concepts, and we tried to address it in our work. The analogies task was first
introduced by Mikolov et al. [10] and then adapted for a number of languages.
The adaptation process is not trivial since analogies should examine specifics
of a given language while too much customization would make datasets
incomparable. We aimed at finding a compromise between these two extremes.

##  3 Proposed Datasets

###  3.1 Similarity Dataset

For constructing the Similarity dataset we used the WordSim-353 dataset;
namely, it’s version by Agirre et al. [1], in which they split the original
dataset into two subsets, one for similarity, and the other for relatedness
evaluation. We took the first subset (for similarity), consisting of a total
of 202 words, and manually translated it to Tatar. Then we removed or replaced
with analogies when possible pairs containing rare words, and those which
needed to be adapted to account for cultural differences, e.g., the Harvard-
Yale pair was replaced with KFU-KAI, which are acronyms of two largest
Tatarstan university names. We also filtered out most of the pairs with
loanwords from Russian.

We defined the distribution of synonymy classes we want to be present in the
dataset as follows:

  * •

Strong synonyms, $22\%$;

  * •

Weak synonyms, $30\%$;

  * •

Co-hyponyms, $17\%$;

  * •

Hypernym-hyponyms, $15\%$;

  * •

Antonyms, $5\%$; and

  * •

Unrelated, $11\%$.

These categories and percentages were chosen to represent the diversity in
synonymy relations and to focus on not-so-obvious pairs, which constitute the
majority ($67\%$) of the dataset—everything apart from strong synonyms and
unrelated words. After the described preprocessing, we split the remaining
pairs between these categories and added our own pairs such that the total is
still 202 and all categories are full. We used SimLex-999 to find some of the
new word pairs. From the part-of-speech point of view, the dataset is mostly
build up from nouns, $87\%$, a small fraction of adjectives, $12\%$, and $1\%$
of mixed pairs.

###  3.2 Relatedness Dataset

The Relatedness dataset was constructed using a similar procedure - we took
the second relatedness subset of WordSim-353, and translated it
adapting/replacing pairs with rare or irrelevant words. For this dataset we
kept more loanwords from Russian to keep it close to the original dataset for
comparability. The dataset contains 252 words, $98\%$ of which are nouns, and
$2\%$ are mixed ones.

#### Annotation

Here we explain the process of obtaining human scores for the datasets
described above. We constructed a survey for each dataset and provided a set
of instructions to respondents. For Similarity dataset it was motivated by
SimLex-999: we showed the examples of synonyms (life-existence), nearly
synonyms (hair-fur), and clearly explained the difference between similarity
and relatedness concepts using such examples as car-road. For Relatedness
dataset instructions explained different association types (by contrast, by
causation, etc.). Then annotators were asked to rate each pair by assigning it
to one of the four categories. Depending on the survey, the options were
different, below are the versions for similarity and (relatedness):

  1. 1.

Words are absolutely dissimilar (unrelated)

  2. 2.

Weak similarity (relatedness)

  3. 3.

Moderate similarity (relatedness)

  4. 4.

Words are very similar or identical (strongly related)

Later this scale was converted to 0–10 to match the existing datasets (1 $\to$
0; 2 $\to$ $\frac{10}{3}$; 3 $\to$ $\frac{20}{3}$; 4 $\to$ 10). A total of 13
respondents rated each dataset, all native Tatar speakers. Inter-annotator
agreement measured as average Spearman’s $\rho$ between pairwise ratings
equals $0.68$ for Similarity and $0.61$ for Relatedness dataset.

###  3.3 Analogies Dataset

We used several existing analogies datasets to identify common categories to
include to our new dataset, namely, the original one [10], and ones for Czech
[14] and Italian [3]. We identified 8 such categories (marked below as †). We
also included new categories, most of which explore the morphological richness
of the Tatar language, and some account for cultural/geographic
characteristics. We applied a frequency threshold: we did not include pairs
where any of the words was not in the top 100 000 of most frequent words. The
final list of the categories is as follows:

#### Semantic Categories

Capital-country†: capitals of countries worldwide, mostly taken from the
original dataset, plus four additional countries, e.g., “Prague”-“Czech
Republic”, the latter expressible as a single word in Tatar. 51 pairs.
Country-currency†: national currencies worldwide, e.g., “Turkey”-“lira”. 11
pairs.
Capital-republic inside Russia: capitals and names of republics, which are
federal subjects of Russia, e.g., “Kazan”-“Tatarstan”. 14 pairs.
Man-woman†: family relations, like brother-sister, but also masculine/feminine
forms of professions and honorifics. E.g., afande-khanym, which can be
translated as “mister”-“missis”. 27 pairs.
Antonyms (adjectives): e.g. “clean”-“dirty”. Differs from ‘Opposite’ category
in the original dataset: words here do not share roots. 50 pairs.
Antonyms (nouns): e.g., “birth”-“death”, roots also differ. Both antonym
categories were built using the dictionary of Tatar antonyms111Safiullina,
ISBN 5-94113-178-X.. 50 pairs.
Name-occupation: e.g., “Tolstoi”-“writer”, for people famous in / associated
with Tatarstan Republic. 40 pairs.

#### Syntactic Categories

Comparative†: positive and comparative forms of adjectives, e.g.
“big”-“bigger”. 50 pairs.
Superlative†: positive and superlative forms of adjectives. Superlatives in
Tatar are usually formed by adding separate in’ “most” word before the
adjective, but sometimes the first part of the word is repeated twice, e.g.,
yashel-yam-yashel, “green”-“greenest”. We include 30 such pairs.
Opposite†: basically antonyms sharing the root, e.g., “tasty”-“tasteless”. 45
pairs.
Plural†: singular and plural word forms, e.g., “school”-“schools”. Contains
two subcategories: for nouns, 50 pairs, and pronouns, 10 pairs.
Cases: specific for Tatar, which has 6 grammatical cases: nominative,
possessive, dative, accusative, ablative, and locative. We pair 30 words in
nominative case with their forms in each of other cases, resulting in 5
subcategories. The example of nominative-possessive subcategory can be
“he”-“his”. Words are a mix of nouns and pronouns.
Derivation (profession): nouns and derived profession names, e.g.,
“history”-“historian”. 25 pairs.
Derivation (adjectives): nouns and derived adjectives, e.g., “salt”-“salty”.
30 pairs.

The following 12 categories explore different verb forms. Tatar, an
agglutinative language, is extremely rich w.r.t. word forms. When it comes to
verbs, the form depends on negation, mood, person, number, and tense, among
others. We explore only some combinations of these aspects, which we think are
most important. To make it easier to comprehend, we only provide examples (for
the verb “go”), and the above-mentioned details can be implied from them,
while main aspect is described by category name. We picked 21 verbs and put
them in different forms as required by the following categories:
Negation: “he goes”-“he doesn’t go”.
Mood (imperative): “to go”-“go!”.
Mood (conditional): “go!”-“if he goes”.
Person1-2: “I go”-“you go”.
Person1-3: “I go”-“he goes”.
Plural, 1 person: “I go”-“we go”.
Plural, 2 person:“you go (alone)”-“you go (as a group)”.
Plural, 3 person: “he goes”-“they go”.
Tense, past (definite)†: “he goes”-“he went”.
Tense, past (indefinite): “he goes”-“he probably went”.
Tense, future (definite): “he goes”-“he will go”.
Tense, future (indefinite): “he goes”-“he will probably go”.
Verbal adverbs, type 1: imperative and adverb, e.g. “go!”-“while going”. For
the same 21 verbs.
Verbal adverbs, type 2: imperative and adverb, e.g. “go!”-“on arrival”. For
the same 21 verbs.
Passive voice: two verbs, second being a passive voice derivation from the
first, e.g., “he writes”-“it is being written”. 25 pairs.

So, in total we constructed 34 categories: 7 semantic and 27 syntactic ones.
For each category/subcategory we generated all possible combinations of pairs
belonging to it, e.g., the first category (capital-country) contains 51 unique
pairs, hence, $50\cdot 51=2550$ combinations. So, we have 10004 semantic and
20140 syntactic questions, in total 30144.

##  4 Experiments and Evaluation

In this section we evaluate three word embedding models, Skip-gram with
negative sampling SG [11], FastText [4], and GloVe [12], with the proposed
datasets. These models were chosen for evaluation for their popularity and
ease of training. As for Fasttext, it was chosen also because it works with
$n$-grams, hence, was expected to handle complex morphology of Tatar better.
We trained SG and FastText using gensim222https://radimrehurek.com/gensim
library and GloVe using the original
code333https://github.com/stanfordnlp/GloVe. Private 126 M tokens corpus,
kindly provided by Corpus of Written Tatar444http://www.corpus.tatar/en, was
used to train these models. The corpus was obtained primarily from web-
resources and is made up of texts in different genres, such as news,
literature, official.

All models were trained with 300 dimensions and window size 5. We used
different versions of SG and FastText in our experiments, all trained for 10
epochs using batch size 128. For other parameters, we will refer to minimum
word count as $\mathit{mc}$, subsampling threshold as $\mathit{sub}$, negative
samples number as $\mathit{neg}$, minimum/maximum $n$-gram length as
$\mathit{gram\\_l}$. When tuning these parameters we were focused on Analogies
task, and for Similarity and Relatedness tasks we chose best results among all
trained models.

As for GloVe, there is only one version for all experiments—trained for 100
epochs with x_max parameter set to 100, and other parameters set to default.

###  4.1 Similarity and Relatedness Results

For evaluation we calculate Spearman’s $\rho$ correlation between average
human score and cosine similarity between embeddings of words in pairs. We
report Spearman’s $\rho$ for SG, FastText, and GloVe for both tasks and
parameter configuration which led to the best performance in Table 1.

Table 1: Spearman’s $\rho$ for Similarity and Relatedness and model parameters. Model |  Similarity |  Relatedness
---|---|---
SG |  0.52 |  0.60
|  $\mathit{mc}$=5, $\mathit{sub}$=0, $\mathit{neg}$=64 |  $\mathit{mc}$=5, $\mathit{sub}$=0, $\mathit{neg}$=20
FastText |  0.54 |  0.62
|  $\mathit{mc}$=2, $\mathit{sub}$=$1e^{-4}$, $\mathit{neg}$=64, $\mathit{gram\\_l}$=3/6 |  $\mathit{mc}$=2, $\mathit{sub}$=$1e^{-4}$, $\mathit{neg}$=64, $\mathit{gram\\_l}$=3/6
GloVe |  0.48 |  0.53

As we see, FastText does better on both tasks and GloVe performs substantially
worse, while for all models the similarity task appears to be trickier, which
does not positively correlate with the inter-annotator agreement.

###  4.2 Analogies Results

We follow the same evaluation procedure as in the original work [10], so, we
report accuracy, where true prediction means exact match (1st nearest
neighbor). We used same parameters for both SG and FastText: $\mathit{mc}=5$,
$\mathit{sub}=0$, $\mathit{neg}=20$, initial/minimal learning rates were set
to $0.05$ and $1e^{-3}$ respectively. For FastText $\mathit{gram\\_l}$ was set
to $3/8$. The results are presented in Table 2, where we compare performance
of SG, FastText, and GloVe for each category of analogies. For summaries (bold
lines), we calculate average over categories instead of overall accuracy, to
cope with imbalance in categories’ sizes: this way all categories have the
same impact on summary score.

Table 2: Accuracy (%) per analogy category. Category | SG | FastText | GloVe
---|---|---|---
Semantic categories
capital-country | $40.51$ | $32.31$ | $15.53$
country-currency | $4.55$ | $5.45$ | $5.45$
capital-republic-rf | $33.52$ | $23.08$ | $30.22$
man-woman | $40.46$ | $38.32$ | $41.03$
adj-antonym | $8.61$ | $7.43$ | $6.73$
noun-antonym | $8.78$ | $7.39$ | $7.67$
name-occupation | $27.95$ | $12.76$ | $15.71$
Semantic average | $23.48$ | $18.11$ | $17.48$
Syntactic categories
comparative | $76.04$ | $76.78$ | $44.65$
superlative | $26.90$ | $33.33$ | $5.29$
opposite | $15.15$ | $13.08$ | $5.30$
plural-nouns | $38.04$ | $41.59$ | $16.94$
plural-pronouns | $23.33$ | $27.78$ | $16.67$
cases-possessive | $32.07$ | $39.66$ | $5.40$
cases-dative | $14.83$ | $12.53$ | $3.33$
cases-accusative | $30.80$ | $36.09$ | $7.70$
cases-ablative | $5.98$ | $12.64$ | $1.15$
cases-locative | $12.99$ | $15.17$ | $4.94$
profession | $20.33$ | $17.33$ | $8.50$
noun-adj | $7.01$ | $4.14$ | $6.09$
negation | $52.14$ | $57.62$ | $19.76$
imperative | $40.24$ | $44.52$ | $8.33$
conditional | $40.00$ | $52.14$ | $7.14$
person1-2 | $65.71$ | $81.90$ | $28.57$
person1-3 | $74.29$ | $71.67$ | $64.52$
plural-1person | $91.43$ | $93.57$ | $65.71$
plural-2person | $63.57$ | $95.24$ | $35.24$
plural-3person | $91.90$ | $86.43$ | $75.24$
present-past-def | $82.62$ | $88.33$ | $65.48$
present-past-indef | $87.86$ | $86.43$ | $84.29$
present-future-def | $45.24$ | $70.24$ | $26.90$
present-future-indef | $55.95$ | $55.71$ | $28.81$
verbal-adv-1 | $23.81$ | $29.05$ | $9.76$
verbal-adv-2 | $40.24$ | $50.71$ | $5.48$
passive-voice | $34.50$ | $35.33$ | $14.67$
Syntactic average | $44.18$ | $49.22$ | $24.66$
All average | $39.92$ | $42.82$ | $23.18$

We observe that syntactic questions are substantially easier to answer for all
models, which indicates they better capture morphological relationships than
semantic ones. If we examine syntactic categories, we see that analogy
questions with nouns and adjectives (from the top to noun-adj) are on average
more challenging for models to answer than questions with verbs (from negation
till the end). Let’s take, for example, plural categories for nouns and
verbs—the difference is drastic, accuracy for plural verb categories is 2-4
times higher than that for nouns. This suggests that more emphasis should be
made on modeling grammatical aspects of nouns and adjectives in the future.

Overall, categories’ complexity varies a lot, e.g., all models show more than
$80\%$ accuracy in present-past-indefinite category, whereas country-currency
and noun-adj are especially complex. Among other categories which challenge
our models we see some types of noun cases, as well as antonyms: whether
sharing the root (opposite) or not (adj-antonym).

As expected, SG and FastText exhibit more similar behaviour, when compared to
GloVe, due to their architectural commonalities. GloVe performs much worse in
syntactic categories; for most of them the results are incomparably lower. We
see that SG beats FastText in semantic questions, but is inferior to it in
syntactic ones, which is as expected because FastText by construction should
learn more about morphology, and Tatar is morphologically rich language.

###  4.3 Comparison with English

To make a comparison between languages, we took a 126 M tokens snippet of
English News Crawl 2017 corpus555http://www.statmt.org/wmt18/translation-
task.html, trained models with same parameters as described in section 4.2,
and evaluated them against the original English datasets.

#### Similarity and Relatedness

First, we measured Spearman’s $\rho$ for similarity and relatedness splits of
WordSim-353 by Agirre et al. [1] for trained English models, and compared with
corresponding results for Tatar in Table 3.

Table 3: Spearman’s $\rho$ for Similarity and Relatedness for Tatar and English. Model |  Task |  Tatar |  English
---|---|---|---
SG |  Similarity |  0.52 |  0.68
Relatedness |  0.60 |  0.55
FastText |  Similarity |  0.54 |  0.68
Relatedness |  0.62 |  0.55
GloVe |  Similarity |  0.48 |  0.46
Relatedness |  0.53 |  0.39

For the English language, performance of SG and FastText on similarity and
relatedness tasks is the same, and for GloVe the numbers are lower. If we
compare with Tatar, we see that the values lie in the same range which is good
because this indicates that the datasets are probably comparable, as it was
designed. Interestingly, though, in contrast to Tatar, relatedness task
appears to be more difficult for English models.

#### Analogies

Second, we evaluated trained English models on the original analogies dataset.
We compare average accuracies in semantic and syntactic categories for two
languages in Table 4.

Table 4: Comparison of average accuracies (%) for Tatar and English. Model |  |  Tatar |  English
---|---|---|---
SG |  Semantic |  23.48 |  53.11
Syntactic |  44.18 |  56.39
FastText |  Semantic |  18.11 |  41.27
Syntactic |  49.22 |  65.48
GloVe |  Semantic |  17.48 |  47.47
Syntactic |  24.66 |  36.41

English FastText falls behind English SG in semantic questions ($41\%$ vs.
$53\%$) but outperforms in syntactic questions ($65\%$ vs. $56\%$), the same
tendency as it was observed for the Tatar models and in the original work [4].
Notice that regardless of language GloVe shows poor results in syntactic
categories.

What’s most interesting, however, is that English models perform much better:
they are from $12\%$ to $30\%$ more accurate than their Tatar counterparts. We
think that the overall better performance of English models can be partially
explained by easier questions: our dataset includes such nontrivial categories
as antonyms, cases, and derivations.

Table 5: Accuracy (%) of FastText for common categories. Category | Tatar | English
---|---|---
Semantic categories
country-capital | $32.31$ | $62.04$
country-currency | $5.45$ | $4.02$
man-woman | $38.32$ | $52.37$
Syntactic categories
comparative | $76.78$ | $86.94$
superlative | $33.33$ | $75.58$
opposite | $13.08$ | $35.71$
plural-nouns | $41.59$ | $76.43$
present-past-def | $88.33$ | $56.60$

To analyze it further, we can compare results for Tatar and English category-
wise. We mentioned in section 3.3 that 8 common categories were selected to be
included in Analogies dataset to make it comparable with existing ones. Now in
Table 5 we report accuracy values for these common categories for Tatar and
English languages tested on FastText model, as it showed better overall
results for both languages.

We see from the results that there’s some correlation between Tatar and
English models. Both models perform well in comparatives, worse in opposites,
and very poorly in country-currency category. Surprisingly, Tatar model
performs better in past tense. For other categories gaps are too huge to
compare. For country-capital the better performance may be explained by the
fact that the news dataset used for training English models is probably an
especially good resource for such data compared to Tatar corpus. For
superlatives the probable reason is the earlier explained
characteristic—superlatives in Tatar are usually formed by adding a separate
word, and adjectives included in this category represent a very small subset
of adjectives, which have one-word superlative form.

Overall, the difference may also be due to dissimilarities of training
corpora. There are no large parallel corpora for English and Tatar,
unfortunately; if this was the case, the comparison would be more accurate.

##  5 Conclusion

In this paper, we introduced three new datasets for evaluating and exploring
word embeddings for the Tatar language. The datasets were constructed
accounting for cultural specificities and allow in-depth analysis of embedding
performance w.r.t. various language characteristics. We examined the
performance of SG, FastText and GloVe word embedding models on introduced
datasets, showing that for all three tests there is much room for improvement.
Cross-language comparison demonstrated that models’ performance varies greatly
with language; nonetheless, similar trends were observed across languages.

##  6 Acknowledgments

We thank Mansur Saykhunov, the main author and maintainer of Corpus of Written
Tatar666http://www.corpus.tatar/en, for providing us data; and all respondents
of the surveys for constructing Similarity/Relatedness datasets. This research
was partially supported by the Brazilian National Council for Scientific and
Technological Development (CNPq grant # 307425/2017-7).

## References

  * [1] Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Pasca, M., Soroa, A.: A study on similarity and relatedness using distributional and wordnet-based approaches. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. pp. 19–27. Association for Computational Linguistics (2009), http://aclweb.org/anthology/N09-1003
  * [2] Beltagy, I., Erk, K., Mooney, R.: Semantic parsing using distributional semantics and probabilistic logic. In: Proceedings of the ACL 2014 Workshop on Semantic Parsing. pp. 7–11. Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/W14-2402, http://aclweb.org/anthology/W14-2402
  * [3] Berardi, G., Esuli, A., Marcheggiani, D.: Word embeddings go to italy: A comparison of models and training datasets. In: IIR (2015)
  * [4] Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606 (2016)
  * [5] Bruni, E., Boleda, G., Baroni, M., Tran, N.K.: Distributional semantics in technicolor. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 136–145. Association for Computational Linguistics (2012), http://aclweb.org/anthology/P12-1015
  * [6] Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., Ruppin, E.: Placing search in context: The concept revisited. In: Proceedings of the 10th International Conference on World Wide Web. pp. 406–414. WWW ’01, ACM, New York, NY, USA (2001). https://doi.org/10.1145/371920.372094
  * [7] He, X., Yang, M., Gao, J., Nguyen, P., Moore, R.: Indirect-hmm-based hypothesis alignment for combining outputs from machine translation systems. In: Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. pp. 98–107. Association for Computational Linguistics (2008), http://aclweb.org/anthology/D08-1011
  * [8] Hill, F., Reichart, R., Korhonen, A.: Simlex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics 41(4), 665–695 (2015). https://doi.org/10.1162/COLI_a_00237, http://aclweb.org/anthology/J15-4004
  * [9] Leviant, I., Reichart, R.: Judgment language matters: Multilingual vector space models for judgment language aware lexical semantics. CoRR, abs/1508.00106 (2015)
  * [10] Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013a)
  * [11] Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. pp. 3111–3119. NIPS’13, Curran Associates Inc., USA (2013b), http://dl.acm.org/citation.cfm?id=2999792.2999959
  * [12] Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP). pp. 1532–1543 (2014), http://www.aclweb.org/anthology/D14-1162
  * [13] Rubenstein, H., Goodenough, J.B.: Contextual correlates of synonymy. Communications of the ACM 8(10), 627–633 (1965)
  * [14] Svoboda, L., Brychcin, T.: New word analogy corpus for exploring embeddings of czech words. In: International Conference on Intelligent Text Processing and Computational Linguistics. pp. 103–114. Springer (2016)