# Novel correlated 5$f$ electronic states in cubic $An$Sn3 ($An$=U, Np, Pu)
intermetallics

Haiyan Lu<EMAIL_ADDRESS>Science and Technology on Surface Physics and
Chemistry Laboratory, P.O. Box 9-35, Jiangyou 621908, China  Li Huang  Science
and Technology on Surface Physics and Chemistry Laboratory, P.O. Box 9-35,
Jiangyou 621908, China

###### Abstract

The intricate interplay between itinerant-localized 5$f$ states and strongly
correlated electronic states have been systematically investigated in
isostructural actinide compounds $An$Sn3 ($An$=U, Np, Pu) by using a
combination of the density functional theory and the embedded dynamical mean-
field approach. The obvious narrow flat 5$f$ electronic band with remarkable
spectral weight emerges in the vicinity of Fermi level for three compounds.
Subsequently the significant hybridization between 5$f$ states and conduction
bands opens evident gaps together with conspicuous valence state fluctuations
jointly indicating the partially itinerant 5$f$ electrons. Especially,
prominent quasiparticle multiplets only appear in PuSn3 due to the sizable
valence state fluctuations and multiple competing atomic eigenstates.
Therefore itinerant 5$f$ states tend to involve in active chemical bonding,
restraining the formation of local magnetic moment of actinide atoms, which
partly elucidates the underlying mechanism of paramagnetic USn3 and PuSn3, as
well as itinerant-electron antiferromagnetic NpSn3. Correspondingly, the 5$f$
electronic correlation strength expressed in band renormalization and electron
effective masse intertwines with itinerant-localized 5$f$ states.
Consequently, detail electronic structure of 5$f$ states dependence on
actinide series shall gain deep insight into our understanding of $An$Sn3
($An$=U, Np, Pu) intermetallics and promote ongoing research.

##  I Introduction

The bewildering 5$f$ electron and its hybridization with ligand states ($f-c$
hybridization) interrelate the advent of unprecedented quantum phenomena
including complex magnetic ordering, heavy-fermion behavior and unconventional
superconductivity Das  _et al._ (2012); Palstra  _et al._ (1985); Trainor  _et
al._ (1976); Sarrao  _et al._ (2002); Moore and van der Laan (2009); Shim  _et
al._ (2007), to name a few. Unlike the mostly localized 4$f$ states of
lanthanides, the extended 5$f$ electronic wave function produces partially
itinerant 5$f$ states. The itinerant-localized dual nature of 5$f$ electrons
ubiquitously exists in actinides which enables the tunability of itinerant
degree of freedom with enlarging atomic series. Generally, 5$f$ electrons are
believed to be mainly itinerant until plutonium (Pu) which locates on the
boundary between itinerant and localized 5$f$ electrons Moore and van der Laan
(2009). On account of the narrow energy band of 5$f$ electrons, the 5$f$
states is easily affected by external temperature, pressure, chemical doping
and magnetic field. To unveil the long-standing issue of itinerant-localized
5$f$ electrons and their relationship with $f-c$ hybridization, binary $An$Sn3
($An$=U, Np, Pu) compounds are established as an ideal archetype system to
explore the subtle electronic structure and related exotic properties.

Figure 1: (Color online). (a) Crystal structure of $An$Sn3 ($An$=U, Np, Pu).
(b) Schematic picture of the first Brillouin zone of $An$Sn3 ($An$=U, Np, Pu).
Some high-symmetry $k$ points $X$ [0.5, 0.0, 0.0], $\Gamma$ [0.0, 0.0, 0.0],
$M$ [0.5, 0.5, 0.0], and $R$ [0.5, 0.5, 0.5] are marked.

$An$Sn3 ($An$=U, Np, Pu) intermetallic compounds crystalize in cubic AuCu3
structure (space group $Pm$-3$m$) [see figure 1(a)] with $An$ atom locating at
cubic corners and Sn atoms occupying the center surface. As is listed in table
1, the experimental lattice constants are 4.603 Å Koelling  _et al._ (1985),
4.627 Å Gal  _et al._ (1973) and 4.630 Å Sari  _et al._ (1983) for USn3, NpSn3
and PuSn3, respectively. Obviously, lattice constants of three isostructural
compounds are approximately 4.6 Å and slightly stretch from USn3 to PuSn3.
Since the interatomic distance $d_{An-An}$ is much larger than the Hill limit
($\sim$3.5 Å) Miner (1970), these intermetallics with weak 5$f$ electrons
overlap are expected to develop certain magnetic ordering at low temperature.
However, some exceptions Loewenhaupt and Loong (1990); Eriksson  _et al._
(1989) with paramagnetic ground state implicate the crucial role of
hybridization between 5$f$ electron and ligand states, which is also tightly
correlated with the heavy-fermion behavior Norman  _et al._ (1986); Trainor
_et al._ (1976); Lu and Huang (2021a) of $An$Sn3 ($An$=U, Np, Pu). .

USn3 is a heavy-fermion compound with an electronic specific heat coefficient
171 mJ/(mol$\times$K2) Norman  _et al._ (1986), which has been verified by the
temperature dependence of electronic specific heat coefficient Norman  _et
al._ (1986) and electronic resistivity experiments Brodsky (1974). Peculiarly,
USn3 remains paramagnetism at very low temperature 1 K with a single weak de
Haas–van Alphen (dHvA) frequency at 0.5 K Cornelius  _et al._ (1999). The
paramagnetic ground state has been elucidated by the 5$f$ orbitals
hybridization with ligand states which induces strong spin fluctuations
observed by inelastic neutron scattering study Loewenhaupt and Loong (1990).
Since USn3 approaches the verge of magnetic instability, a small doping with
Pb may initiate antiferromagnetic ordering Loewenhaupt and Loong (1990). The
physical picture is consistent with a spin density wave magnetic unstability
at the quantum critical point detected by using 119Sn nuclear spin-echo decay
rate measurement Kambe  _et al._ (2009). Since the heavy-fermion behavior and
paramagnetism are closely associated with the subtle electronic structure,
x-ray photoemission spectroscopy of USn3 detects a significant peak around the
Fermi level which is contributed by 5$f$ states, as well as a small shoulder
consisted of 6$d$-5$p$ hybridization, accompanied by a hump at about 7 eV from
Sn-5$s$ states Schneider  _et al._ (1987). Nevertheless, the extremal orbits
predicted by previous first-principles caculation Kadowaki and Woods (1986)
have not been experimentally corroborated. The reason my arise from the heavy
electrons which are substantially renormalized by many-body interactions could
not be correctly described with single-particle band theory. Moreover, angle-
resolved photoemission spectroscopy experiment which unravel the fine 5$f$
electronic states are still deficient.

NpSn3 is the first itinerant-electron antiferromagnetic compound with colinear
antiferromagnetic ordering and a small magnetic moment 0.28 $\mu_{B}$ Trainor
_et al._ (1976); Fedders and Martin (1966). Similar to USn3, the atomic
distance between two nearest Np atoms exceeds Hill criterion Miner (1970)
indicating the basically localized 5$f$ electrons. Abundant experiments have
envinced the localized behavior of 5$f$ electrons comprising the deduction of
electronic specific heat coefficient from paramagnetic state 242
mJ/(mol$\times$K2) to 88 mJ/(mol$\times$K2) below Néel temperature 9.5 K
Norman and Koelling (1986), temperature dependent electronic properties
Kalvius  _et al._ (1990), electrical resistivity, magnetic susceptbility,
Mössbaurer spectroscopy and single crystal neutron scattering experiments
Charvolin  _et al._ (1994). In spite of this, the small magnetic moments
detected by Mössbaurer spectroscopy and single crystal neutron scattering
experiments still manifest partially itinerant 5$f$ states. That is to say,
the partially itinerant or localized 5$f$ electrons of NpSn3 is still
perplexing. In analogy with USn3, the hybridzation of 5$f$ states with
conduction bands greatly affects the localization degree of freedom and heavy-
fermion behavior. On the other hand, limited theoretical studies could not
elucidate the itinerant antiferromagnetic ordering using crystal field
splitting model Kalvius  _et al._ (1990) and fail to accurately delineate the
strongly correlated 5$f$ electrons Pourghazi  _et al._ (2007); Khan  _et al._
(2018). It is reported that PuSn3 shows temperature-independent paramagnetism
Wohlfarth and Buschow (1988); Eriksson  _et al._ (1989) and pseudogap around
the Fermi level in low-temperature electrical resistivity Brodsky (1978),
still lacking electronic structure characterization experiments such as
photoemission spectroscopy, angle-resolved photoemission spectroscopy, and
x-ray adsorption spectroscopy. Meanwhile, theoretical investigations have made
great efforts to systematically obtain the crystal structure, bulk modulus,
electronic structure, quantum oscillation Eriksson  _et al._ (1989); Baizaee
and Pourghazi (2005, 2007); Joseph Wang  _et al._ (2013), leaving the
underlying physics of paramagnetic ground state being untouched.

Actually, plentiful theoretical efforts have been devoted to study uranium
pnictides and chalcogenides Arko  _et al._ (1975); Arko and Koelling (1978).
The band structure, density of states, fermi surface and electron effective
mass of USn3 have been previously investigated by employing first-principles
calculations Cornelius  _et al._ (1999); Strange (1986). Subsequently, the
strong correlation among 5$f$ electrons and large spin-orbit coupling
complicate the calculation of $An$Sn3 ($An$=U, Np, Pu) compounds, blurring a
comprehensive physical picture of electronic structure and ground state
properties. Evidently, the missing narrow flat 5$f$ bands within the
traditional density functional theory is attributed to the underestimation of
strongly correlated 5$f$ electrons. Furthermore, the spin-orbit coupling of
actinides should be carefully treated. Especially, the intrinsic mechanism for
the paramagnetic ground state of USn3 and NpSn3 remains elusive even though
the spin-fluctuation theory has been proposed for USn3. Additionally, the
bewildering heavy-fermion behavior of $An$Sn3 ($An$=U, Np, Pu) is rarely
scrutinized. Now that the hybridization strength between 5$f$ states and
conduction bands greatly influences the low temperature magnetic ordering and
itinerant-localized dual nature of 5$f$ electrons, it is illuminating to
investigate the 5$f$ series dependence of 5$f$ electronic structure and 5$f$
electron hydridization with ligand states to uncover the profound bonding
behavior and strongly correlated 5$f$ states, which shall gain deep insight on
the diverse ground state ordering as well as alluring heavy-fermion behavior.

Table 1: Crystal structure parameters, electronic specific heat coefficient and ground state ordering of $An$Sn3 ($An$=U, Np, Pu).  cases |  $d_{An-An}$ (Å) |  $d_{An-Sn}$ (Å) |  $\gamma$ (mJ/(mol$\times$K2)) | ground state
---|---|---|---|---
USn3 | 4.603 | 3.255 | 171111See Ref. [Norman  _et al._ , 1986]. | paramagnetism444See Ref. [Loewenhaupt and Loong, 1990].
NpSn3 | 4.627 | 3.272 | 242222See Ref. [Trainor  _et al._ , 1976]. | itinerant-electron antiferromagnetism222See Ref. [Trainor  _et al._ , 1976]. (TN = 9.5 K)
PuSn3 | 4.630 | 3.274 | 96333See Ref. [Lu and Huang, 2021a]. | paramagnetism555See Ref. [Eriksson  _et al._ , 1989].

##  II Methods

In order to capture the essence in the strongly correlated 5$f$ electrons for
uranium, neptonium and plutonium, 5$f$ correlated electronic states dependence
on actinide elements are investigated by means of a combination of the density
functional theory and the embedded dynamical mean-field theory (DFT + DMFT).
The DFT + DMFT method integrates the DFT realistic band structure calculation
and a non-perturbative way to tackle the many-body local interaction effects
in DMFT Georges  _et al._ (1996); Kotliar  _et al._ (2006). Here the strong
electronic correlation and large spin-orbit coupling are treated on an equal
footing in the calculation. Then the DFT + DMFT approach is implemented into
DFT and DMFT parts, which are solved separately by using the WIEN2K code Blaha
_et al._ (2001) and the EDMFTF software package Haule  _et al._ (2010).

In the DFT calculation, the experimental crystal structures of $An$Sn3
($An$=U, Np, Pu) were utilized. Since the calculated temperature 116 K is
above the antiferromagnetic transition temperature of NpSn3, the system was
assumed to be nonmagnetic. The generalized gradient approximation was tested
and used to formulate the exchange-correlation functional Perdew  _et al._
(1996). Since the actinides belong to heavy element, the considerable spin-
orbit coupling was treated in a second-order variational manner. The
$k$-points’ mesh was $15\times 15\times 15$ and
$R_{\text{MT}}K_{\text{MAX}}=8.0$.

In the DMFT part, 5$f$ electrons of $An$ atom were treated as correlated. The
four-fermions’ interaction matrix was parameterized using the Coulomb
interaction $U=5.0$ eV and the Hund’s exchange $J_{H}=0.6$ eV via the Slater
integrals Fujiwara and Korotin (1999) for $An$Sn3 ($An$=U, Np, Pu). It should
be noted that the band structure and density of states for $U=$ 4.0 eV, 5.0
eV, 6.0 eV reveal subtle nuance for $An$Sn3 ($An$=U, Np, Pu). The fully
localized limit scheme was used to compute the double-counting term for
impurity self-energy function Anisimov  _et al._ (1997). The vertex-corrected
one-crossing approximation (OCA) impurity solver Stahrenberg  _et al._ (2001)
was employed to solve the resulting multi-orbital Anderson impurity models. In
order to improve the computation burden, some good quantum numbers such as $N$
and $J$ were chosen to divide the whole Hilbert space Haule (2007) into
subblocks. Moreover, the truncation approximation for the atomic eigenstates
was adopted to gain further acceleration. Explicitly, only those atomic
eigenstates whose occupancy $N$ satisfy $N$ $\in$ $[N_{low},N_{high}]$ will be
taken into account in the local trace evaluation. Finally, the convergence
criteria for energy and charge were $10^{-5}$ Ry and $10^{-5}$ e,
respectively. It is worth mentioning that the direct output of OCA impurity
solver is real axis self-energy $\Sigma(\omega)$ which was used to calculate
the momentum-resolved spectral functions $A(\mathbf{k},\omega)$, density of
states $A(\omega)$ and other physical observables.

##  III Results

Figure 2: (Color online). Momentum-resolved spectral functions
$A(\mathbf{k},\omega)$ of $An$Sn3 ($An$=U, Np, Pu) at 116 K under ambient
pressure calculated by the DFT + DMFT method. The horizontal lines denote the
Fermi level. (a) USn3. (b) NpSn3. (c) PuSn3.  Figure 3: (Color online).
Electronic density of states of $An$Sn3 ($An$=U, Np, Pu) at 116 K obtained by
the DFT + DMFT method. Total density of states (thick solid lines) and partial
5$f$ density of states (color-filled regions) of (a) USn3, (b) NpSn3, (c)
PuSn3. These peaks in figure (c) resulting from the quasiparticle multiplets
are denoted with “P1”, “P2”, “P3”, and “P4”. The $j$-resolved 5$f$ partial
density of states with $5f_{5/2}$ and $5f_{7/2}$ components represented by red
and green lines, respectively. (d) USn3, (e) NpSn3, (f) PuSn3.  Figure 4:
(Color online). Three-dimensional Fermi surface of $An$Sn3 ($An$=U, Np, Pu) at
116 K calculated by the DFT + DMFT method. (a1)-(a2) USn3. (b1)-(b2) NpSn3.
(c1)-(c2) PuSn3.  Figure 5: (Color online). Two-dimensional Fermi surface on
the $k_{x}-k_{y}$ plane (with $k_{z}=\pi/2$) of $An$Sn3 ($An$=U, Np, Pu) at
116 K calculated by the DFT + DMFT method. (a) USn3. (b) NpSn3. (c) PuSn3.
They are visualized with different colors.

###  III.1 Electronic band structures

Here we concentrate on the momentum-resolved spectral functions
$A(\mathbf{k},\omega)$ of $An$Sn3 ($An$=U, Np, Pu) to reveal the inherent
nature of 5$f$ electrons. To verify the reliability of our calculations, we
examine the electronic band structure of USn3 along high-symmetry line
$X-\Gamma-M-R$ in the irreducible Brillouin zone [see Fig. 2(a)]. Compared
with the available theoretical results by A. L. Cornelius et al. employing a
localized spin density approximation Cornelius  _et al._ (1999), the essential
feature of band structures [see Fig. 2(a)] generally coincide with each other.
It is noticed that the hole-like orbit around the Fermi level at $\Gamma$
point and similar hole-like orbits at $X$ and $M$ points are almost alike.
However, the major difference is that flat narrow 5$f$ electronic bands
intersect the conduction bands in the vicinity of the Fermi level and open
evident hybridization gaps. It is proposed that the absence of quasiparticle
5$f$ bands in literature Cornelius  _et al._ (1999); Strange (1986) is
probably concerned with the underestimation of strongly correlated 5$f$
electrons without sophisticated treating the many-body effects.

Then we scrutinize the salient traits encoded in the band structure of $An$Sn3
($An$=U, Np, Pu). It is observed that the overall profile of the momentum-
resolved spectral functions for USn3 and NpSn3 [see Fig. 2(a) and 2(b)] at 116
K share abundant similarities with analogous conduction orbits and remarkable
dispersionless 5$f$ electronic bands around the Fermi level. Even so, a slight
discrepancy gives a higher 5$f$ band intensity of NpSn3 than that of USn3,
concretising the itinerant instinct of 5$f$ electrons. In comparison with the
band structure of PuSn3 [see Fig. 2(c)], it is found out that a mild
difference in the conduction bands above the Fermi level and almost the same
conduction bands below the Fermi level contributed by 5$p$ states of Sn atom.
In particular, the apparent narrow 5$f$ bands emerge at -0.9 eV, -0.47 eV and
the Fermi level, which hybridize with ligand states to form obvious
hybridization gaps. Correspondingly, prominent spectral weight of 5$f$
electrons, dramatic quasiparticle bands and striking hybridization entirely
hint the itinerant and coherent 5$f$ states of PuSn3. Then it is speculated
that the distinct character of 5$f$ electrons roots from various actinide
series and various hybridization strength bewteen 5$f$-ligand states.

###  III.2 Density of states

To unveil the 5$f$ electronic structure of $An$Sn3 ($An$=U, Np, Pu), it is
enlightening to explore the density of states and quasiparticle peaks depicted
in Figs. 3(a)-(g). Firstly, the entire outline of total density of states for
USn3 and NpSn3 bears striking resemblances including a sharp narrow
quasiparticle peak at the Fermi level and a broad “hump” in the energy region
of 0.5 eV $\sim$ 2 eV. The prominent quasiparticle peaks in the Fermi level
delineate the itinerancy and coherence of 5$f$ electrons. Owing to the spin-
orbit coupling, the fourteen-fold degenerated 5$f$ orbitals are split into
six-fold degenerated 5$f_{5/2}$ and eight-fold degenerated 5$f_{7/2}$ subbands
[see Figs. 3(d)-(f)]. The archetypical quasiparticle peak is mostly
contributed by 5$f_{5/2}$ orbital of USn3 and NpSn3. Meanwhile, a slight
increase in the spectral weight of 5$f$ orbital of NpSn3 with respect to USn3
may attributed a visible spectral weight of 5$f_{7/2}$ orbital in the Fermi
level of NpSn3. As is listed in table 3, the quasi-particle weight $Z$ of
5$f_{7/2}$ orbital in NpSn3 is maximal which demonstrates the substantial
valence state fluctuations and mixed-valence behavior.

Interestingly, the most distinctive feature of the density of states of PuSn3
[see Fig. 3(c)] is the typical atomic multiplets with one keen-edged
quasiparticle peak in the Fermi level and four satellite peaks above and below
the Fermi level. In combination with the partial 5$f$ density of states [see
Fig. 3(f)], the quasiparticle peak is contributed by 5$f_{5/2}$ orbital.
Meanwhile, the two satellite peaks “P1” and “P2” at –0.9 eV and –0.47 eV with
energy gap about 0.43 eV are attributed to 5$f_{5/2}$ and 5$f_{7/2}$ orbitals,
respectively. Above the Fermi level, the reflected peaks “P3” and “P4”
distribute in 0.47 eV and 0.9 eV relative to the central quasiparticle. Thus
the five representative peaks are called “quasiparticle multiplets” which
commonly exist in plutonium-based compounds such as Pu3Ga, PuIn3 and PuB6 Lu
and Huang (2021b, 2022). To trace the origin of these quasiparticle
multiplets, it is deduced that “P3” and “P4” peaks are formed by a mix of
5$f_{5/2}$ and 5$f_{7/2}$ orbitals. It is mentioned that the quasiparticle
multiplets are induced by abundant competing atomic eigenstates of 5$f$
electrons synergy with 5$f$ valence state fluctuations discussed in subsection
III.4. In other words, ample competitive atomic eigenstates act as
prerequisites for the onset of quasiparticle multiplets. For instance, nominal
occupancy of 5$f$ electrons of Pu atom is 5.0, which is larger than those of
uranium and neptunium 2.0 and 3.0, respectively. Then the number of occupied
5$f$ states of Pu atom is larger than those of uranium and neptunium. In
contrast, the enhancing localization of 5$f$ electrons for curium and
californium with relatively weak valence state fluctuations fail to produce
quasiparticle multiplets even though their nominal occupancy of 5$f$ electrons
is larger than 5.0. That is why only PuSn3 develops observable quasiparticle
multiplets even if valence state fluctuations are remarkable in $An$Sn3
($An$=U, Np, Pu).

###  III.3 Fermi surface topology

Fermi surface topology is a pivotal physical quantity to character the detail
electronic structure of $An$Sn3 ($An$=U, Np, Pu). Figure 4 visualizes the
three-dimensional Fermi surface topology at 116 K. There exist two doubly
degenerated bands crossing the Fermi level (No. of bands: 16 and 17, 18 and
19) for USn3 and (No. of bands: 18 and 19, 20 and 21) for both NpSn3 and
PuSn3, which are labeled by $\alpha$ and $\beta$, respectively. For USn3 [see
Fig. 4(a1) and (a2)], $\alpha$ band occupies eight corners of the first
Brillouin zone and $\beta$ band resembles an anisotropic huge form.
Especially, $\alpha$ band develops a distorted hexahedral structure for NpSn3
[see Fig. 4(b1) and (b2)], while $\beta$ band displays twelve small scattered
fragments in the edge centers of the first Brillouin zone. Note that for PuSn3
[see Fig. 4(c1) and (c2)] the Fermi surface topology of $\alpha$ band takes an
ellipsoid shape, which agrees with those in previous DFT calculations Joseph
Wang  _et al._ (2013). Nevertheless, the Fermi surface topology of $\beta$
band forms an anisotropic shape, which acts distinctively from the previous
results Joseph Wang  _et al._ (2013). The discrepancy of $\beta$ band might
originate from the temperature effect and the strongly correlated 5$f$
electronic states on Fermi surface. The DFT calculations Joseph Wang  _et al._
(2013) are performed at zero temperature which also underestimates the 5$f$
electronic correlation. Besides, it is pointed out that the Fermi surface
could be detected by following de Haas–van Alphen quantum oscillation, then
the experimental results may clarify possible physical causes behind the
paramagnetic ground state of USn3 and PuSn3 provided that no Fermi surface
nesting is observed.

Since the interior structure of three-dimensional Fermi surface is hard to
discern, the two-dimensional Fermi surface of $\alpha$ and $\beta$ bands are
plotted in Fig. 5. Corresponding to the three-dimensional Fermi surface of
USn3 [see Fig. 5(a1) and (a2)], both $\alpha$ and $\beta$ bands intersect the
$\Gamma$ \- $M$ line and $M$ \- $X$ line, while no bands cross the $\Gamma$ \-
$X$ line. Meanwhile, only $\alpha$ band cuts through the $\Gamma$ \- $X$ line
for NpSn3 [see Fig. 5(b1) and (b2)] with $\alpha$ and $\beta$ bands passing
through the $\Gamma$ \- $M$ line and $M$ \- $X$ line. Finally, both $\alpha$
and $\beta$ bands cross the $\Gamma$ \- $X$ line and $\Gamma$ \- $M$ line for
PuSn3 [see Fig. 5(c1) and (c2)], along with $\beta$ band intersecting $M$ \-
$X$ line, which is in accordance with the momentum-resolved spectral functions
[see Fig. 2].

###  III.4 Valence state fluctuations

Table 2: Probabilities of 5$f^{n}$ electronic configuration at 116 K for USn3 ($n\in$ [0, 4]), NpSn3 ($n\in$ [1, 5]) and PuSn3 ($n\in$ [3, 7]), 5$f$ occupancy $n_{5f}$, x-ray absorption branching ratio $\mathcal{B}$.  USn3 | $5f^{0}$ | $5f^{1}$ | $5f^{2}$ | $5f^{3}$ | $5f^{4}$ | $n_{5f}$ | $\mathcal{B}$
---|---|---|---|---|---|---|---
| 9.048$\times 10^{-5}$ | 0.026 | 0.671 | 0.297 | 0.012 | 2.283 | 0.6855
NpSn3 | $5f^{1}$ | $5f^{2}$ | $5f^{3}$ | $5f^{4}$ | $5f^{5}$ | $n_{5f}$ | $\mathcal{B}$
| 9.234$\times 10^{-5}$ | 0.018 | 0.472 | 0.496 | 0.014 | 3.506 | 0.7291
PuSn3 | $5f^{3}$ | $5f^{4}$ | $5f^{5}$ | $5f^{6}$ | $5f^{7}$ | $n_{5f}$ | $\mathcal{B}$
| 9.976$\times 10^{-4}$ | 0.075 | 0.772 | 0.150 | 1.343$\times 10^{-3}$ | 5.076 | 0.7806

Now that alluring quasiparticle multiplets are closely interrelated with
valence state fluctuations, the 5$f$ electron atomic eigenstates obtained from
the output of DMFT many-body states contain the valence state fluctuations.
$p_{\Gamma}$ is used to quantify the probability of 5$f$ electrons which
remains in each atomic eigenstate $\Gamma$. Then the average 5$f$ valence
electron is defined as $n_{5f}=\sum_{\Gamma}p_{\Gamma}n_{\Gamma}$, where
$n_{\Gamma}$ denotes the number of electrons in each atomic eigenstate
$\Gamma$. Finally, the probability of 5$f^{n}$ electronic configuration can be
written as $P_{n}=w(5f^{n})=\sum_{\Gamma}p_{\Gamma}\delta(n-n_{\Gamma})$.

The calculated probabilities of 5$f^{n}$ electronic configuration for $An$Sn3
($An$=U, Np, Pu) are listed in table 2. Evidently, for USn3, the probability
of 5$f^{2}$ electronic configuration accounts for 67.1%, followed by the
probability of 5$f^{3}$ about 29.7%, giving the probabilities of 5$f^{1}$ and
5$f^{4}$ electronic configurations about 9.1% and 1.2%, respectively. The
conspicuous valence state fluctuations give rise to the 5$f$ electron
occupancy $n_{5f}$ = 2.283, which deviates from its nominal value 2.0.
Accordingly, 5$f$ electrons tend to spend more time on 5$f^{2}$ and 5$f^{3}$
electronic configurations, promoting mixed-valence behavior. Referring to
NpSn3, the competitive probabilities of 5$f^{3}$ and 5$f^{4}$ electronic
configurations approximate 47.2% and 49.6%, respectively, which render
substantial valence state fluctuations. Together with a small portion of the
probabilities of 5$f^{2}$ and 5$f^{5}$ electronic configurations around 1.8%
and 1.4%, the 5$f$ electron occupancy $n_{5f}$ = 3.506 diverges from its
nominal value 3.0 to a large extent. Therefore NpSn3 exhibits noteworthy
valence state fluctuations and related mixed-valence states. Eventually, we
focus on the probability of electronic configuration of PuSn3. The leading
probability of 5$f^{5}$ electronic configuration reaches as high as 77.2%.
Meanwhile, the probability of 5$f^{6}$ and 5$f^{4}$ attain 15.0% and 7.5%,
respectively, neglecting other electronic configurations. In this context, a
moderate valence state fluctuations regulate the 5$f$ electron occupancy
$n_{5f}$ = 5.076, which approaches its nominal value 5.0. In order to
quantitatively describe the strength of valence state fluctuations, we define
the quantity ${\cal V}=\sum_{n}P_{n}(P_{n}-1)$, where $n$ denotes the
occupancy number of 5$f$ electrons. Two extreme cases are listed for
illustration. The case for $P_{n}=1$, ${\cal V}=0$ signifies the absence of
valence state fluctuations with only one electronic configuration. In this
sense, electrons are extremely localized where they prefer to stay in one
electronic configuration. On the contrary, dispersive distribution of
electronic configurations generates arresting valence state fluctuations with
${\cal V}=1$. Actually, such ideal systems hardly exist in real materials and
${\cal V}$ evaluates 0.689, 0.623, 0.449 for USn3, NpSn3, PuSn3, respectively.
Consequently, the strength of valence state fluctuations obeys the following
sequence, ${\cal V}$(USn${}_{3})$ $\textgreater$ ${\cal V}$(NpSn${}_{3})$
$\textgreater$ ${\cal V}$(PuSn${}_{3})$. The order conforms with the
qualitative picture that increscent atomic number with growing localization of
5$f$ electrons suppresses valence state fluctuations. It is expected that
ongoing research on a wide range of $f$ electronic states for lanthanide and
actinide compounds shall shed light on the comprehensive results about precise
description of valence state fluctuations.

###  III.5 Self-energy functions

Table 3: The effective electron mass $m^{\star}$ and quasi-particle weight $Z$ of $5f_{5/2}$ and $5f_{7/2}$ states for $An$Sn3 ($An$=U, Np, Pu) at 116 K.  | $5f_{5/2}$ | $5f_{7/2}$
---|---|---
compounds | $m^{\star}/m_{e}$ | $Z$ | $m^{\star}/m_{e}$ | $Z$
USn3 | 18.717 | 0.053 | 17.233 | 0.058
NpSn3 | 18.673 | 0.054 | 1.890 | 0.529
PuSn3 | 29.111 | 0.034 | 13.873 | 0.072

Inspiringly, the electronic correlations are encoded in the electron self-
energy functions Georges  _et al._ (1996); Kotliar  _et al._ (2006). The
renormalization factor $Z$ which embodies the electronic correlation strength
can be deduced from the real part of self-energy functions via the following
equation Georges  _et al._ (1996):

$Z^{-1}=\frac{m^{\star}}{m_{e}}=1-\frac{\partial\text{Re}\Sigma(\omega)}{\partial\omega}\Big{|}_{\omega=0}.$
(1)

Here renormalization factor $Z$ quantitatively depicts the quasiparticle
weight and band compression degree of correlated electronic bands which is
inversely proportional to the electron effective mass $m^{\star}$. Table 3
lists the computed effective electron mass $m^{\star}$ and renormalization
factor $Z$ of $5f_{5/2}$ and $5f_{7/2}$ states for $An$Sn3 ($An$=U, Np, Pu) at
116 K. For USn3, sizable electron effective mass for both $5f_{5/2}$ and
$5f_{7/2}$ states implicates the strongly correlated 5$f$ electronic states,
with suppressed quasiparticle weight $Z$. Similar to USn3, PuSn3 reveals
strong correlation strength among 5$f$ states where the electron effective
mass of $5f_{5/2}$ state is much larger than $5f_{7/2}$ state. As mentioned in
subsection III.2, the finite distribution of spectral weight of $5f_{7/2}$
state of NpSn3 arises from the smooth variation across the Fermi level, which
indicates the small electron effective mass of $5f_{7/2}$ state with
significant quasi-particle weight $Z$. The comparatively weak electron
correlation of $5f_{7/2}$ state compared to $5f_{5/2}$ state, evincing the
orbital dependent character. It is speculated that the orbital selective
correlation of $5f_{5/2}$ and $5f_{7/2}$ states for NpSn3 holds underlying
connection with itinerant antiferromagnetic ordering.

##  IV Discussion

In this section, we discuss the hybridization between 5$f$ states and
conduction bands to reveal the paramagnetic ground state of USn3 and PuSn3, as
well as the antiferromagnetic ordering of NpSn3. Moreover, we would like to
interpret the angular momentum coupling scheme through x-ray absorption
spectroscopy.

5$f$ electron dependence of electronic structure. As atomic number increases
from uranium to plutonium, the itinerant-localized 5$f$ states interrelates
with 5$f$ electron occupancy. As mentioned above, the spectral weight of 5$f$
states near the Fermi level demonstrates that 5$f$ electrons are inclined to
hybridize with Sn-5$p$ bands to form partially itinerant 5$f$ states. The
bonding behavior is also evidenced in the significant valence state
fluctuations and apparent mixed-valence behavior for $An$Sn3 ($An$=U, Np, Pu).
It is speculated that larger atomic actinide distance may result in weaker
$f-c$ hybridization so as to suppress itinerant degree of freedom and restrain
valence state fluctuations. Unexpectedly, quasiparticle multiplets in PuSn3 is
prominent which is intimately associated with valence state fluctuations and
multiple competing atomic eigenstates. The distinctive feature of PuSn3
restates that Pu sits on the edge of itinerant-localized 5$f$ states. In this
sense, USn3 and NpSn3 tend to develop paramagnetic ground state because of
substantial $f-c$ hybridization. Even though NpSn3 exhibits antiferromagnetic
ordering, the magnetic moment of Np atom is small. This is mainly attributed
to the two competitive electronic configurations 5$f^{3}$ and 5$f^{4}$ which
renders noteworthy valence state fluctuations and mixed-valence nature.

Angular momentum coupling scheme. For the actinides with intricate spin-orbit
coupling, 5$f$ electrons occupy the $5f_{5/2}$ and $5f_{7/2}$ subbands. Since
the evolution trend of 5$f$ electron in the $5f_{5/2}$ and $5f_{7/2}$ levels
with respect to incremental atomic radius is a vital problem, the angular
momentum coupling scheme serves as the primary determinant factor. For the
multielectronic systems, there exist three ways of angular momentum coupling
in light of the relative strength of spin-orbit coupling and electronic
interaction, including Russell-Saunders (LS) coupling, $jj$ coupling, and
intermediate coupling (IC) Moore and van der Laan (2009). For the ground state
of heavy actinides, intermediate coupling scheme is usually the favorite. A
natural question rises with the 5$f$ orbital occupancy and angular momentum
coupling scheme for $An$Sn3 ($An$=U, Np, Pu).

Since x-ray absorption spectroscopy is a powerful technique to detect the
electronic transitions between core 4$d$ and valence 5$f$ states, it is
employed to observe the occupancy of 5$f$ electrons of actinides. The strong
spin-orbit coupling for the 4$d$ states leads to two absorption lines,
representing the $4d_{5/2}\rightarrow 5f$ and $4d_{3/2}\rightarrow 5f$
transitions, respectively. The x-ray absorption branching ratio $\mathcal{B}$
is defined as the relative strength of the $4d_{5/2}$ absorption line Thole
and van der Laan (1988). It calibrates the spin-orbit coupling interaction
strength in 5$f$ shell. Under the approximation that the electrostatic
interaction between core and valence electrons is ignored, the expression for
$\mathcal{B}$ is written as Shim  _et al._ (2007):

$\mathcal{B}=\frac{3}{5}-\frac{4}{15}\frac{1}{14-n_{5/2}-n_{7/2}}\left(\frac{3}{2}n_{7/2}-2n_{5/2}\right),$
(2)

where $n_{7/2}$ and $n_{5/2}$ are the 5$f$ occupation numbers for the
$5f_{7/2}$ and $5f_{5/2}$ states, respectively. The computed results are
listed in table 2, obeying $\mathcal{B}$(PuSn3) $\textgreater$
$\mathcal{B}$(NpSn3) $\textgreater$ $\mathcal{B}$(USn3). The sequence evinces
that the angular momentum coupling scheme of three compounds belongs to the
intermediate coupling scheme, which is analogous to that in heavy actinides Pu
Huang and Lu (2020) and Cm Huang  _et al._ (2020).

##  V conclusion

In summary, we employ the density functional theory in combination with
embedded dynamical mean-field theory to study extensively the 5$f$ strongly
correlated electronic states of isostructural compounds $An$Sn3 ($An$=U, Np,
Pu). It is found that apparent quasiparticle weight of 5$f$ states around the
Fermi level is accompanied by dispersionless 5$f$ electronic band. Even though
quasiparticle multiplets only develop in PuSn3, striking valence state
fluctuations imply the itinerancy of 5$f$ electrons and the tendency to
hybridize with ligand states to facilitate paramagnetic ground state. Usually,
localized 5$f$ states interrelate with strongly correlated 5$f$ electrons with
renormalized bands and large electron effective mass. Then 5$f$ electrons are
strongly correlated in three compounds with orbital selective correlation of
$5f_{7/2}$ and $5f_{5/2}$ states for NpSn3. Additionally, the angular momentum
coupling scheme is found to be intermediate which could be utilized to detect
the 5$f$ electron occupancy and related physical properties. Briefly, the
evolution pattern of strongly correlated 5$f$ electron and itinerant-localized
5$f$ states dependence on actinide series gain valuable implications into the
actinide compounds and pave the way for future research.

###### Acknowledgements.

This work was supported by National Natural Science Foundation of China
2022YFA140220, CAEP Project (No. TCGH0710) and the National Natural Science
Foundation of China (No. 11874329, No. 11934020).

## References

  * Das  _et al._ (2012) T. Das, J.-X. Zhu, and M. J. Graf, Phys. Rev. Lett. 108, 017001 (2012).
  * Palstra  _et al._ (1985) T. T. M. Palstra, A. A. Menovsky, J. v. d. Berg, A. J. Dirkmaat, P. H. Kes, G. J. Nieuwenhuys, and J. A. Mydosh, Phys. Rev. Lett. 55, 2727 (1985).
  * Trainor  _et al._ (1976) R. J. Trainor, M. B. Brodsky, B. D. Dunlap, and G. K. Shenoy, Phys. Rev. Lett. 37, 1511 (1976).
  * Sarrao  _et al._ (2002) J. L. Sarrao, L. A. Morales, J. D. Thompson, B. L. Scott, G. R. Stewart, F. Wastin, J. Rebizant, P. Boulet, E. Colineau, and G. H. Lander, Nature 420, 297 (2002).
  * Moore and van der Laan (2009) K. T. Moore and G. van der Laan, Rev. Mod. Phys. 81, 235 (2009).
  * Shim  _et al._ (2007) J. H. Shim, K. Haule, and G. Kotliar, Nature 446, 513 (2007).
  * Koelling  _et al._ (1985) D. D. Koelling, B. D. Dunlap, and G. W. Crabtree, Phys. Rev. B 31, 4966 (1985).
  * Gal  _et al._ (1973) J. Gal, Z. Hadari, E. R. Bauminger, I. Nowik, and S. Ofer, Solid State Commun. 13, 647 (1973).
  * Sari  _et al._ (1983) C. Sari, F. Vernazza, and W. Müller, J. Less-Common Met. 92, 301 (1983).
  * Miner (1970) W. N. Miner, ed.,  _Plutonium 1970 and other Actinides_ (Metallurgical Society of the AIME, New York, 1970).
  * Loewenhaupt and Loong (1990) M. Loewenhaupt and C.-K. Loong, Phys. Rev. B 41, 9294 (1990).
  * Eriksson  _et al._ (1989) O. Eriksson, M. S. S. Brooks, and B. Johansson, Phys. Rev. B 39, 13115 (1989).
  * Norman  _et al._ (1986) M. R. Norman, S. D. Bader, and H. A. Kierstead, Phys. Rev. B 33, 8035 (1986).
  * Lu and Huang (2021a) H.-Y. Lu and L. Huang, J. Phys.: Condens. Matter 33, 435603 (2021a).
  * Brodsky (1974) M. B. Brodsky, Phys. Rev. B 9, 1381 (1974).
  * Cornelius  _et al._ (1999) A. L. Cornelius, A. J. Arko, J. L. Sarrao, J. D. Thompson, M. F. Hundley, C. H. Booth, N. Harrison, and P. M. Oppeneer, Phys. Rev. B 59, 14473 (1999).
  * Kambe  _et al._ (2009) S. Kambe, H. Sakai, Y. Tokunaga, T. D. Matsuda, Y. Haga, H. Chudo, and R. E. Walstedt, Phys. Rev. Lett. 102, 037208 (2009).
  * Schneider  _et al._ (1987) W. D. Schneider, C. Laubschat, and B. Reihl, Phys. Rev. B 35, 7922 (1987).
  * Kadowaki and Woods (1986) K. Kadowaki and S. B. Woods, Solid State Commun. 58, 507 (1986).
  * Arko  _et al._ (1975) A. J. Arko, M. B. Brodsky, G. W. Crabtree, D. Karim, D. D. Koelling, L. R. Windmiller, and J. B. Ketterson, Phys. Rev. B 12, 4102 (1975).
  * Arko and Koelling (1978) A. J. Arko and D. D. Koelling, Phys. Rev. B 17, 3104 (1978).
  * Strange (1986) P. Strange, J. Phys. F: Met. Phys. 16, 1515 (1986).
  * Fedders and Martin (1966) P. A. Fedders and P. C. Martin, Phys. Rev. 143, 245 (1966).
  * Norman and Koelling (1986) M. R. Norman and D. D. Koelling, Phys. Rev. B 33, 3803 (1986).
  * Kalvius  _et al._ (1990) G. M. Kalvius, S. Zwirner, U. Potzel, J. Moser, W. Potzel, F. J. Litterst, J. Gal, S. Fredo, I. Yaar, and J. C. Spirlet, Phys. Rev. Lett. 65, 2290 (1990).
  * Charvolin  _et al._ (1994) T. Charvolin, A. Blaise, M. N. Bouillet, P. Burlet, J. M. Fournier, J. Larroque, J. Rossat-Mignod, and J. P. Sanchez, J. Magn. Magn. Mater. 132, 46 (1994).
  * Pourghazi  _et al._ (2007) A. Pourghazi, Z. Nourbakhsh, and M. T. Fallahi, J. Magn. Magn. Mater. 320, 503 (2007).
  * Khan  _et al._ (2018) S. Khan, M. Yazdani-Kachoei, S. Jalali-Asadabadi, and I. Ahmad, Intermetallics 93, 77 (2018).
  * Wohlfarth and Buschow (1988) E. P. Wohlfarth and K. H. J. Buschow, eds.,  _The Handbook of Ferromagnetic Materials_ , North-Holland, Vol. 4 (North-Holland, Amsterdam, 1988).
  * Brodsky (1978) M. B. Brodsky, Rep. Prog. Phys. 41, 1547 (1978).
  * Baizaee and Pourghazi (2005) S. M. Baizaee and A. Pourghazi, Phys. B 358, 247 (2005).
  * Baizaee and Pourghazi (2007) S. M. Baizaee and A. Pourghazi, Phys. B 387, 287 (2007).
  * Joseph Wang  _et al._ (2013) C.-C. Joseph Wang, M. D. Jones, and J.-X. Zhu, Phys. Rev. B 88, 125106 (2013).
  * Georges  _et al._ (1996) A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Rev. Mod. Phys. 68, 13 (1996).
  * Kotliar  _et al._ (2006) G. Kotliar, S. Y. Savrasov, K. Haule, V. S. Oudovenko, O. Parcollet, and C. A. Marianetti, Rev. Mod. Phys. 78, 865 (2006).
  * Blaha  _et al._ (2001) P. Blaha, K. Schwarz, G. Madsen, D. Kvasnicka, and J. Luitz,  _WIEN2k, An Augmented Plane Wave + Local Orbitals Program for Calculating Crystal Properties_ (Karlheinz Schwarz, Techn. Universität Wien, Austria, 2001).
  * Haule  _et al._ (2010) K. Haule, C.-H. Yee, and K. Kim, Phys. Rev. B 81, 195107 (2010).
  * Perdew  _et al._ (1996) J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * Fujiwara and Korotin (1999) T. Fujiwara and M. Korotin, Phys. Rev. B 59, 9903 (1999).
  * Anisimov  _et al._ (1997) V. I. Anisimov, F. Aryasetiawan, and A. I. Lichtenstein, J. Phys.: Condens. Matter 9, 767 (1997).
  * Stahrenberg  _et al._ (2001) K. Stahrenberg, T. Herrmann, K. Wilmers, N. Esser, W. Richter, and M. J. G. Lee, Phys. Rev. B 64, 115111 (2001).
  * Haule (2007) K. Haule, Phys. Rev. B 75, 155113 (2007).
  * Lu and Huang (2021b) H.-Y. Lu and L. Huang, Phys. Rev. B 103, 205134 (2021b).
  * Lu and Huang (2022) H.-Y. Lu and L. Huang, J. Phys.: Condens. Matter 34, 215601 (2022).
  * Thole and van der Laan (1988) B. T. Thole and G. van der Laan, Phys. Rev. A 38, 1943 (1988).
  * Huang and Lu (2020) L. Huang and H.-Y. Lu, Phys. Rev. B 101, 125123 (2020).
  * Huang  _et al._ (2020) L. Huang, R.-F. Chen, and H.-Y. Lu, Phys. Rev. B 101, 195123 (2020).


#  Neural Machine Translation for Low-Resourced Indian Languages

###### Abstract

A large number of significant assets are available online in English, which is
frequently translated into native languages to ease the information sharing
among local people who are not much familiar with English. However, manual
translation is a very tedious, costly, and time-taking process. To this end,
machine translation is an effective approach to convert text to a different
language without any human involvement. Neural machine translation (NMT) is
one of the most proficient translation techniques amongst all existing machine
translation systems. In this paper, we have applied NMT on two of the most
morphological rich Indian languages, i.e. English-Tamil and English-Malayalam.
We proposed a novel NMT model using Multihead self-attention along with pre-
trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an
efficient translation system that overcomes the OOV (Out Of Vocabulary)
problem for low resourced morphological rich Indian languages which do not
have much translation available online. We also collected corpus from
different sources, addressed the issues with these publicly available data and
refined them for further uses. We used the BLEU score for evaluating our
system performance. Experimental results and survey confirmed that our
proposed translator (24.34 and 9.78 BLEU score) outperforms Google translator
(9.40 and 5.94 BLEU score) respectively.

Keywords: Multihead self-attention, Byte-Pair-Encodding, MultiBPE, low-
resourced, Morphology, Indian Languages

languageresourceLanguage Resources

Neural Machine Translation for Low-Resourced Indian Languages

Himanshu Choudhary, Shivansh Rao, Rajesh Rohilla
---
Delhi Technological University (Formerly Delhi college of Engineering)
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

##  1\. Introduction

Many populated countries such as India and China have several languages which
change region by region. for example, India has 23 constitutionally recognized
official languages (_e.g._ , Hindi, Malayalam, Telugu, Tamil, and Punjabi) and
numerous unofficial local languages. Not only big countries, even small
countries also rich in language diversity. There are 851 languages spoken in
Papua New Guinea, which is one of the smallest populated regions. In India,
the population is about three billion, but only about 10% of them can speak
English111https://www.bbc.com/news/magazine-20500312. Some studies say that
out of those 10% English speakers only 2% can talk, write, and examine English
well, and rest 8% can merely recognize simple English and talk with a variety
of accents. Thinking about a large number of valuable sources is available on
the web in English and most people in India can not understand it well, it
becomes important to translate such content into neighborhood languages to
facilitate people. Sharing pieces of information between human beings is
important not only for business purposes but also for sharing their emotions,
reviews, and acts. For this, translation plays an essential role in minimizing
the communication hole between different peoples. considering the vast amount
of text, it is not viable to translate them manually. Hence, it becomes
crucial to translate text from one language (say, English) to other languages
(say, Tamil, Malayalam) automatically. This technique is also referred to as
_machine translation_.

English to Indian language translation poses the challenge of morphological
and structural divergence. For instance, (i) the number of parallel corpora
and (ii) differences between languages, mainly the morphological richness and
variation in word order due to syntactical divergence. Indian languages (IL)
suffers from both of these problems, especially when they are being translated
from English. Moreover, Indian languages such as Malayalam and Tamil differ
not only in word order but are also more agglutinative as compared to English
which is fusional. For instance, English has Subject-Verb-Object (SVO) whereas
Tamil and Malayalam have Subject-Object-Verb (SOV). While syntactic
differences contribute to difficulties of translation models, morphological
differences contribute to data sparsity. We attempt to overcome both issues in
this paper.

There are various papers on machine translation, but apart from foreign
languages most of the works on Indian languages are limited to Hindi and on
conventional machine translation techniques such as [Patel et al., 2018] and
[Raju and Raju, 2016]. Most of the previous work is focused on separating the
words in suffix and prefix based on some rules and then applying translation
techniques. We addressed this issue with BPE to make this whole process more
efficient and reliable. Moreover, We observed that very less work is being
done on low resourced Indian languages and techniques such as Byte-pair-
encoding (BPEmb), MultiBPEmb, word-embedding, and self-attention are still
unexplored which have shown a significant improvement in Natural Language
Processing. Though unsupervised machine translation [Artetxe et al., 2017] is
also in the focus of many researchers, still it is not as precise as
supervised learning. We, also addressed that there is no trustworthy Public
data available for the translation of such languages. Thus, in this paper, we
have applied a neural machine translation technique with Multihead-self
attention along with word embeddings and Pre-Trained Byte-Pair-Encoding. We
worked on English-Tamil and English-Malayalam language pairs as it is one of
the most difficult languages pair [ZdenekŽabokrtskỳ, 2012] to translate due to
morphological richness of Tamil and Malayalam language. A similar approach can
be applied to other languages as well. We obtained the data from EnTamv2.0,
Opus and UMC005, preprocessed them and evaluated our result using the
evaluation matric BLEU. We used OpenNMT-py for the implementation of our
models 222http://opennmt.net/OpenNMT-py/. Experimental results, as well as the
survey by native peoples, confirms that our result is far better than
conventional translation techniques on Indian languages.

The Main contributions of our work are as follows:

  * •

This is the first work to apply pre-trained BPE and MultiBPE embeddings on
Indian language pairs (English-Tamil, English-Malayalam) along with Multihead
self-attention technique.

  * •

We achieved good accuracy with a relatively simpler model and in less training
time rather than training on a complex neural network which requires much
resources and time to train.

  * •

We have addressed the issues with data preprocessing of Indian languages and
shown why it is a crucial step in neural machine translation.

  * •

We made our preprocessed data publicaly available, which by our knowledge
contains the largest number of a parallel corpus for the languages (English-
Tamil, English-Malayalam, English-Telugu, English-Bengali, English-Urdu)

  * •

Our model outperforms Google translator with a margin of 3.36 and an 18.07
BLEU score.

The paper is organized as follows. Sections Background and Approach describe
the related work and the method that we used for our translator, respectively.
Section experiments and Results show data preprocessing and results and
analysis of our model. Finally, Section 5. concludes the paper and future
work.

##  2\. Background

A large amount of work has been reported on machine translation (MT) in the
last few decades, the first one in the 1950s [Booth, 1955]. Various approaches
is used by researchers, such as rule-based [Ghosh et al., 2014], corpus-based
[Wong et al., 2006], and hybrid-based approach [Salunkhe et al., 2016]. Each
approach has its own flaws and strength. Rule-based machine translation (RBMT)
is MT systems based on the linguistic information about the source and target
languages which is retrieved from ( multilingual, bilingual or monolingual)
dictionaries and grammars covering the main syntactic, semantic and
morphological regularities. It is further divided into transfer-based approach
(TBA)[Shilon, 2011] and inter-lingual based approach (IBA). In the Corpus-
based approach, we use a large-sized parallel corpus as raw data. This raw
data contains ground truth translation for the desired languages. These
corpora are used to train the model for translation. A corpus-based approach
further classified in (i) statistical machine translation (SMT) [Patel et al.,
2018] and (ii) example-based machine translation (EBMT) [Somers, 2003]. SMT is
the combination of decoding algorithms and basic statistical language
models.EBMT, on the other hand, uses the translation examples and generates
the new translation accordingly. It is done by finding the examples which are
matching with the input. The alignment has to be performed after that to find
out the parts of translation that can be reused. Hybrid-base machine
translation combines any corpus-based approach and transfer approach in order
to overcome their limitations. According to the recent research [Khan et al.,
2017] the machine translation performance of Indian languages such as (_e.g._
, Hindi, Bengali, Tamil, Punjabi, Gujarati, and Urdu) is of an average of 10%
accuracy. This demands the necessity of building better translation systems
for Indian languages.
Unsupervised machine translation is further a new way of translation without
using the parallel corpus, but the results are still not remarkable. On the
other hand, NMT is an emerging technique and shown significant improvement in
the translation results. In this paper [Hans and Milton, 2016] phrase-based
hierarchical model is used and trained after morphological preprocessing.
[Patel et al., 2017] trained their model after compound splitting and suffix
separation. Many researchers also tried the same way and achieved a decent
result on their respective datasets [Pathak and Pakray,]. We observed that
morphological pre-processing, compound splitting and suffix or prefix
separation can be overcome by using Byte-Pair-Encoding and produce similar or
even better translation results without making the model complex.

##  3\. Approach

In this paper, we present a neural machine translation technique using
Multihead self-attention and word-embedding along with pre-trained Byte-Pair-
Encoding (BPE) on our preprocessed dataset of Indian languages. We developed
an efficient translation system, that overcomes the OOV (Out Of Vocabulary)
and morphological analysis problem for Indian languages which do not have many
translations available on the web. first, we provide an overview of NMT,
Multi-head self-attention, word embedding, and Byte Pair Encoding. Next, we
describe the framework of our translation model.

###  3.1. Neural Machine Translation Overview

Neural Machine translation is a powerful algorithm based on neural networks
and uses the conditional probability of translated sentences to predict the
target sentences of given source language [Revanuru et al., 2017a]. When
coupled with the power of attention mechanisms, this architecture can achieve
impressive results with different variations. The following sub-sections
provide an overview of basic sequence to sequence architecture, self-attention
and other techniques that are used in our proposed translator.

####  3.1.1. Sequence to sequence architecture

Sequence to sequence architecture is used for response generation whereas in
Machine Translation systems it is used to find the relations between two
language pairs. It consists of two important parts, an encoder, and a decoder.
The encoder takes the input from the source language and the decoder leads to
the output based on hidden layers and previously generated vectors. Let $A$ be
the source and $B$ be a target sentence. The encoding part converts the source
sentence $a_{1},a_{2},a_{3}...,a_{n}$ into the vector of fixed dimensions and
the decoder part gives the word by word output using conditional probability.
Here, $A_{1},A_{2},...,A_{M}$ in the equation are the fixed size encoding
vectors. Using chain rule, the Eq. 1 is transformed to the Eq. 2.

$\displaystyle P(B/A)=P(B|A_{1},A_{2},A_{3},...,A_{M})$ (1)

$\displaystyle P(B|A)=P(b_{i}|b_{0},b_{1},b_{2},...,b_{i-1};$ (2)
$\displaystyle a_{1},a_{2},a_{3},...,a_{m}$

The decoder generates output using previously predicted word vectors and
source sentence vectors in Eq. 1.

Figure 1: Seq2Seq architecture for English-Tamil

####  3.1.2. Attention Model

In a basic encoder-decoder architecture, encoder memorizes the whole sentence
in terms of vector, and store it in the final activation layer, then the
decoder uses that vector to generates the target sentence. This architecture
works quite well for small sentences, but for larger sentences, maybe longer
than 30 or 40 words, the performance degrades. To overcome this problem
attention mechanisms play an important role. The basic idea behind this is
that each time, when the model predicts an output word, it only uses the parts
of input where the most relevant information is concentrated instead of the
whole sentence. In other words, it only pays attention to some weighted words.
Many types of attention mechanisms are used in order to improvise the
translation accuracy, but the multi-head self-attention overcomes most of the
problems.

##### Self-attention

In self-attention architecture [Vaswani et al., 2017] at every time step of an
RNN, a weighted average of all the previous states will be used as an extra
input to the function that computes the next state. With the self-attentive
mechanism, the network can decide to attend to a state produced many time
steps earlier. This means that the latest state does not need to store all the
information. The mechanism also makes it easier for the gradient to flow more
easily to all previous states, which can help against the vanishing gradient
problem.

Figure 2: Attention model

##### Multi-Head Attention

When we have multiple queries q, we can combine them in a matrix Q. If we
compute alignment using dot-product attention, the set of equations that are
used to calculate context vectors can be reduced as shown in figure 3. Q, K,
and V are mapped into lower-dimensional vector spaces using weight matrices
and the results are used to compute attention (which we call a Head). In Muti-
Head Attention we have h such sets of weight matrices which give us h Heads.

Figure 3: Multi-Head Attention

####  3.1.3. Word Embedding

Word embedding is a unique way of representing the word in a vector space such
that we can capture the semantic similarity of each word. Each word is
represented in hundreds of dimensions. Generally, pre-trained embeddings are
used trained on the larger data sets, and with the help of transfer learning,
we convert the words from vocabulary to vector. [Cho et al., 2014].

####  3.1.4. Byte Pair Encoding

BPE [Gage, 1994] is a data compression technique that replaces the most
frequent pair of bytes in a sequence. We use this algorithm for word
segmentation, and by merging frequent pairs of charters or character sequences
we can get the vocabulary of desired size [Sennrich et al., 2015]. BPE helps
in the suffix, prefix separation, and compound splitting which in our case
used for creating new and complex words of Malayalam and Tamil language by
interpreting them as sub-words units. We used BPE along with pre-trained fast-
text word embeddings 333https://github.com/bheinzerling/bpemb [Heinzerling and
Strube, 2018] for both the languages with the variation in the vocabulary
size. In our model, we got the best results with vocabulary size 25000 and
dimension 300.

##### MultiBPEmb

MultiBPEmb is a collection of multiple languages subword segmentation models
and pre-trained subword embeddings trained on Wikipedia data similar to
monolingual BPE. On the contrary, instead of training one segmentation model
for each language, here we train a single model and a single embedding for all
the languages. We can also create a vocabulary of only two languages, source,
and target. It deals with the mixed language sentences (Native language along
with English) which are being popular nowadays on social media. Since our
sentences were clean it almost produced similar results, with variation in the
BLEU score by 0.60 in Tamil and 1.15 in Malayalam.

##  4\. Experimentation and Results

###  4.1. Evaluation Metric

BLEU score is a method to measure the difference between machine translation
and human translation [Papineni et al., 2002]. The approach works by matching
n-grams in result translation to n-grams in the reference text, where unigram
is a unique token, bigram is a word pair and so on. A perfect match results in
a score of 1.0 or 100%.

###  4.2. Dataset

We obtained the data from different resources such as EnTamV2.0 [Ramasamy et
al., 2012], Opus [Tiedemann, 2012] and UMC005[Jawaid and Zeman, 2011] .The
sentences are of domain news, cinema, bible and movie subtitles. We combined
and preprocessed the data of Tamil, Malayalam, Telugu, Bengali, and Urdu.
After preprocessing (as described below) and cleaning, the dataset is split
into train, test, and validation. Our final dataset is described in table 1.
In our knowledge this is the largest, clean and preprocessed public dataset
444https://github.com/himanshudce/Indian-Language-Dataset available on the web
for general purpose uses. As there is no publicly available dataset to compare
various approaches on Indian languages, our datasets can be used to set
baseline results to compare with.

ID | Language | Train | Test | Dev
---|---|---|---|---
1 | Tamil | 183451 | 2000 | 1000
2 | Malayalam | 548000 | 3660 | 3000
3 | Telugu | 75000 | 3897 | 3000
4 | Bengali | 658000 | 3255 | 3500
5 | Urdu | 36000 | 2454 | 2000
Table 1: Dataset for Indian Languages

###  4.3. Data Pre-processing

In the Research works [Hans and Milton, 2016] [Ramesh and Sankaranarayanan,
2018] EnTamV2.0 dataset is used. Also, the Opus dataset is a much widely used
parallel corpus resource in various researcher’s works. However, we observed
that in both of these well-known parallel resources there are many repeated
sentences, which may results into the wrong results (can be higher or lower)
after dividing into train, validation, and test sets, as many of the
sentences, occur both in train and test sets. In most of the work, the focus
relies on the models without interpreting the data which performs much better
on our own test set rather than on general translated sentences. Thus, it is
essential to analyses, correct and cleans the data before using it for the
experiments. Researchers should also provide a detailed source of the corpus
otherwise results can be misleading such as in paper [Revanuru et al., 2017b].
We observed the following four important issues in the online available
corpus.

  * •

Sentence repetition with the same source and target.

  * •

Different translations by the same source.

  * •

Same translated sentences by different source sentences.

  * •

Indian language tokenization.

To overcome the first issue, we took unique pairs from all the parallel
sentences and removed the repeating ones. To tackle the second and third case
we removed sentence pairs which were repeated more than twice and the
difference between their length are in the window of 5 words. It is because
for both of these cases we cannot identify that which source is correct for
the same translation and which translated sentence is comes from the same
source. We observed that there were some sentences, which were repeating even
more than 20 times in the Opus dataset. This confuses the model to learn,
identify and capture different features and overfits the model. Though data-
augmentation [Fadaee et al., 2017] can improve the translation results, but in
that case, the original data should be pre-processed, otherwise many augmented
sentences may appear in both train and test data which leads to higher but
wrong BLEU score as it will not work efficiently on new sentences.

For the tokenization of the English language, there are many libraries and
frameworks such as (_e.g._ , Perl tokenizer) but these do not work well on the
Indian languages, due to the difference between morphological symbols. The
word-formation of Indian languages is quite different which we believed can
only be handled by either special library for that particular language or by
Byte-Pair-Encoding. In the case of BPE, we don’t need to tokenize the words
which generally leads to better translation results.

After working on all these minor, but effective pre-processing we got our
final dataset. While extracting the data from the web, we also removed
sentences with a length greater than 50, known translated words in target
sentences, noisy translations, and unwanted punctuations. For the reliability
of data, we also took the help of native speakers of these languages.

###  4.4. Translator

We tried various new techniques as described above to get a better intuition
of the effects on these two Indian language pairs. Our first model consists of
4 layer Bi-directional LSTM encoder and a decoder with 500 dimensions each
along with a vocabulary size of 50,004 words for both source and target.
First, we used Bahdanau’s attention and Adam optimizer with the dropout
(regularization) of 0.3 and the learning rate 0.001. Here we used the 300
dimensional Pre-trained fast text 555https://fasttext.cc/docs/en/crawl-
vectors.html word embeddings for both the languages. Secondly, we used Pre-
trained fast text Byte-Pair-Encoding 666https://github.com/bheinzerling/bpemb
with the same attention. In the third model, we changed the attention to
multi-head with 8 heads and 6 encoding and decoding layers. It shows an
improvement of 1.2 and 6.18 BLEU scores for Tamil and Malayalam respectively.
For the final model we used Multilingual fast text pre-trained Byte-pair-
Encoddings 777https://nlp.h-its.org/bpemb/multi/ and got our final best
results of 9.67 and 25.36 respectively as shown in table 2 and table 3.

###  4.5. Result

Our results is shown in table 2 and table 3. For Google translate we used
Python API to translate the English sentences and compared the results with
our various models. From the test results, It is observed that our model
overcomes the OOV (Out of Vocabulary) problem in some cases, and is handy
enough to be used in day to day life and official work.

English-Tamil translation models

Figure 4: English-Tamil model comparison with Google Translator Table2

English-Malayalam translation models

Figure 5: English-Malayalam model comparison with Google Translator Table 2 ID | Model |  | BLEU
---
Score
| A
---
Google Translator | 5.71
1 |  | Bi-LSTM(4-Layers)+ A+
---
Bahdanau Attention + WE
5.96
2 |  | Bi-LSTM(4-Layers)+ A +
---
Bahdanau Attention + Pre-BPE(25000)
7.87
3 |  | Bi-LSTM(6-Layer)+ A +
---
Multi-Head Attention+ Pre-BPE(25000)
9.07
4 |  | Bi-LSTM(6-Layer)+ A +
---
Multi-Head self Attention
\+ Pre-MultiBPE(100000)
9.67
Table 2: English-Tamil model comparison with Google Translator ( A=Adam, WE=Word Embeddings) ID | Model |  | BLEU
---
Score
| A
---
Google Translator | 9.40
1 |  | Bi-LSTM(4-Layers)+ A+
---
Bahdanau Attention + WE
11.76
2 |  | Bi-LSTM(4-Layers)+ A +
---
Bahdanau Attention + Pre-BPE(25000)
18.03
3 |  | Bi-LSTM(6-Layer)+ A +
---
Multi-Head Attention+ Pre-BPE(25000)
24.21
4 |  | Bi-LSTM(6-Layer)+ A +
---
Multi-Head self Attention
\+ Pre-MultiBPE(100000)
25.36
Table 3: BLEU Score of English-Malayalam translated system. (A=Adam, B=
Bahdanau, WE=Word Embedding)

Attention Visualization

Figure 6: Attention visualization of English-Tamil sentence pair from our test
data

###  4.6. Analysis

We conducted a survey with ten random sentences from our test data and
accumulated the reviews of native Tamil speaking peoples. On comparing the
reviews of Google translator and our translator, it was found, that our
translation results were better in 60% cases than the Google translator. The
visualization of an Attention can be seen in 6 of one of the sample sentences
from our test data.

##  5\. Conclusion

In this paper, we applied Neural Machine Translation (NMT) on two of the most
difficult Indian language pairs (English-Tamil, English-Malayalam). We
addressed the issues of data pre-processing and tokenization. To handle
morphology and word complexities of Indian languages we applied pre-trained
fast text BPEmb, MultiBPEmb embeddings along with multi-head self-attention
which outperformed Google translator with a margin of 3.96 and 15.96 BLEU
points respectively. The same approach can be applied to other Indian
languages as well. Since the accuracy of our model was fairly good, it can be
used for creating English-Malayalam and English-Tamil translation applications
that will be very useful in domains like tourism, education and corporate. In
the future, we can also explore the possibility to improve the translation
results for code-switched languages using MultiBPE and other variations.

##  6\. Bibliographical References

## References

  * Artetxe et al., 2017 Artetxe, M., Labaka, G., Agirre, E., and Cho, K.  (2017).  Unsupervised neural machine translation.  CoRR, abs/1710.11041.
  * Booth, 1955 Booth, A. D.  (1955).  Machine translation of languages, fourteen essays.
  * Cho et al., 2014 Cho, K., Van Merriënboer, B., Bahdanau, D., and Bengio, Y.  (2014).  On the properties of neural machine translation: Encoder-decoder approaches.  arXiv preprint arXiv:1409.1259.
  * Fadaee et al., 2017 Fadaee, M., Bisazza, A., and Monz, C.  (2017).  Data augmentation for low-resource neural machine translation.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567–573, Vancouver, Canada, July. Association for Computational Linguistics.
  * Gage, 1994 Gage, P.  (1994).  A new algorithm for data compression.  The C Users Journal, 12(2):23–38.
  * Ghosh et al., 2014 Ghosh, S., Thamke, S., et al.  (2014).  Translation of telugu-marathi and vice-versa using rule based machine translation.  arXiv preprint arXiv:1406.3969.
  * Hans and Milton, 2016 Hans, K. and Milton, R.  (2016).  Improving the performance of neural machine translation involving morphologically rich languages.  arXiv preprint arXiv:1612.02482.
  * Heinzerling and Strube, 2018 Heinzerling, B. and Strube, M.  (2018).  BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages.  In Nicoletta Calzolari (Conference chair), et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 7-12, 2018. European Language Resources Association (ELRA).
  * Jawaid and Zeman, 2011 Jawaid, B. and Zeman, D.  (2011).  Word-order issues in English-to-Urdu statistical machine translation.
  * Khan et al., 2017 Khan, N. J., Anwar, W., and Durrani, N.  (2017).  Machine translation approaches and survey for indian languages.  arXiv preprint arXiv:1701.04290.
  * Papineni et al., 2002 Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J.  (2002).  Bleu: a method for automatic evaluation of machine translation.  In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
  * Patel et al., 2017 Patel, R. N., Pimpale, P. B., et al.  (2017).  Mtil17: English to indian langauge statistical machine translation.  arXiv preprint arXiv:1708.07950.
  * Patel et al., 2018 Patel, R., Pimpale, P., and Mukundan, S.  (2018).  Machine translation in indian languages: Challenges and resolution.  Journal of Intelligent Systems, 09.
  * Pathak and Pakray,  Pathak, A. and Pakray, P.  ).  Neural machine translation for indian languages.  Journal of Intelligent Systems.
  * Raju and Raju, 2016 Raju, B. N. V. N. and Raju, M. S. V. S. B.  (2016).  Statistical machine translation system for indian languages.  In 2016 IEEE 6th International Conference on Advanced Computing (IACC), pages 174–177, Feb.
  * Ramasamy et al., 2012 Ramasamy, L., Bojar, O., and Žabokrtský, Z.  (2012).  Morphological processing for english-tamil statistical machine translation.  In Proceedings of the Workshop on Machine Translation and Parsing in Indian Languages (MTPIL-2012), pages 113–122.
  * Ramesh and Sankaranarayanan, 2018 Ramesh, S. H. and Sankaranarayanan, K. P.  (2018).  Neural machine translation for low resource languages using bilingual lexicon induced from comparable corpora.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 112–119.
  * Revanuru et al., 2017a Revanuru, K., Turlapaty, K., and Rao, S.  (2017a).  Neural machine translation of indian languages.  In Proceedings of the 10th Annual ACM India Compute Conference on ZZZ, pages 11–20. ACM.
  * Revanuru et al., 2017b Revanuru, K., Turlapaty, K., and Rao, S.  (2017b).  Neural machine translation of indian languages.  11\.
  * Salunkhe et al., 2016 Salunkhe, P., Kadam, A. D., Joshi, S., Patil, S., Thakore, D., and Jadhav, S.  (2016).  Hybrid machine translation for english to marathi: A research evaluation in machine translation:(hybrid translator).  In Electrical, Electronics, and Optimization Techniques (ICEEOT), International Conference on, pages 924–931. IEEE.
  * Sennrich et al., 2015 Sennrich, R., Haddow, B., and Birch, A.  (2015).  Neural machine translation of rare words with subword units.  arXiv preprint arXiv:1508.07909.
  * Shilon, 2011 Shilon, R.  (2011).  Transfer-based Machine Translation between morphologically-rich and resource-poor languages: The case of Hebrew and Arabic.  Ph.D. thesis, Citeseer.
  * Somers, 2003 Somers, H.  (2003).  An overview of ebmt.  In Recent advances in example-bas ed machine translation, pages 3–57. Springer.
  * Tiedemann, 2012 Tiedemann, J.  (2012).  Parallel data, tools and interfaces in opus.  In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey, may. European Language Resources Association (ELRA).
  * Vaswani et al., 2017 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I.  (2017).  Attention is all you need.  In I. Guyon, et al., editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc.
  * Wong et al., 2006 Wong, F., Dong, M., and Hu, D.  (2006).  Machine translation using constraint-based synchronous grammar.  Tsinghua Science and Technology, 11(3):295–306.
  * ZdenekŽabokrtskỳ, 2012 ZdenekŽabokrtskỳ, L. O.  (2012).  Morphological processing for english-tamil statistical machine translation.  In 24th International Conference on Computational Linguistics, page 113.