# Design of spin-orbital-textures in ferromagnetic/topological insulator
interfaces

A. L. Araújo Ilum School of Science, Brazilian Center for Research in Energy
and Materials (CNPEM), Campinas, SP, Brazil  F. Crasto de Lima
<EMAIL_ADDRESS>Ilum School of Science, Brazilian Center for
Research in Energy and Materials (CNPEM), Campinas, SP, Brazil  C. H.
Lewenkopf Instituto de Física, Universidade Federal Fluminense, Niterói, RJ,
Brazil  A. Fazzio<EMAIL_ADDRESS>Ilum School of Science,
Brazilian Center for Research in Energy and Materials (CNPEM), Campinas, SP,
Brazil

###### Abstract

Spin-orbital textures in topological insulators due to the spin locking with
the electron momentum, play an important role in spintronic phenomena that
arise from the interplay between charge and spin degrees of freedom. We have
explored interfaces between a ferromagnetic system (CrI3) and a topological
insulator (Bi2Se3) that allow the manipulation of spin-orbital textures.
Within an ab initio approach we have extracted the spin-orbital-textures
dependence of experimentally achievable interface designs. The presence of the
ferromagnetic system introduces anisotropic transport of the electronic spin
and charge. From a parameterized Hamiltonian model we capture the anisotropic
backscattering behavior, showing its extension to other
ferromagnetic/topological insulator interfaces. We verified that the van der
Waals TI/MI interface is an excellent platform for controlling the spin degree
of freedom arising from topological states, providing a rich family of
unconventional spin texture configurations.

##  I Introduction

Recent years have witnessed an enormous growth in the interest in the role
played by symmetry, dimensionality, and topology in quantum properties of
condensed matter and material science systems. The study and manipulation of
the spin-orbit coupling (SOC) and the spin degree of freedom is key for
advancing this line of research and revealed a plethora of new fascinating
phenomena, such as quantum Hall effects [1], charge/spin interplay effects [2;
3], that lead to many spintronic applications [4].

Since SOC occurs through the response of the electron’s spin to the nuclear
electric field or in response to a polarization field resulting from symmetry
breaking, efficient control of the spin configuration of electronic states has
been achieved by manipulation of their inversion and/or time-reversal (TRS)
symmetries. For instance, in materials without inversion symmetry, spin
polarization is accompanied by a spin splitting of electronic states and the
appearance of a chiral spin texture due to a spin-momentum locking, thought
Rashba [5; 6] and Dresselhaus [7] effects.

The SOC also plays a central role in the topological states of matter, where
the nontrivial phase of the bulk is usually due to a band inversion at the
Fermi level. Topological insulators (TI) are characterized by the presence of
insulating bulk states but metallic states at the interfaces with
topologically trivial systems [8; 9], which also exhibit a spin-momentum
locking and a helical spin texture with opposite chiralities of spin. These
states are robust against external perturbations due to the presence of TRS,
which suppresses backscattering processes.

One of the main characteristics of chiral spin textures is that they can be
exploited to generate out-of-equilibrium spin polarizations, a process
directly linked to one of the primary objectives of spintronics: at room
temperature achieving an efficient mechanism for interconversion between pure
spin excitation and an electrical signal [10; 11], with the Rashba-Edelstein
(REE) [2] and the inverse Rashba-Edelstein (IREE) [3] effects being among the
most studied. Additionally, another challenge faced by spintronics is to
increase the carrier lifetime by suppressing the spin relaxation interaction.
One path to achieve this goal is to obtain a material where the spin
configuration is uniform in space and independent of the moment, configuring
the persistent spin helix (PSH), where electron motion is accompanied by spin
precession [12]. The advancement of spintronics is not only closely tied to
the ability to obtain spin-textured surface states but primarily to the
capacity to manipulate the configuration of such states. Substantial effort is
currently dedicated to this task, particularly concerning the investigation of
spin textures originating from topological materials (such as topological
insulators [8; 9; 13; 14], Weyl semimetals [15; 16]) or materials with
inversion asymmetry.

In this context, in recent years research on Topological Insulator/Magnetic
Insulator (TI/MI) interfaces for spintronic applications has been driven by
the quest of efficient charge-spin interconversion mechanisms [17] and spin
torque transfer [18; 19], the latter being a promising effect for the
development of non-volatile and low-power magnetic memories [20].
Additionally, for practical applications, TI/MI heterostructures with van der
Waals (vdW) coupling are excellent candidates for spintronics, as they
preserve the topological properties and spin texture behavior of TI surface
states, suppressing the occurrence of interfacial charges while protecting the
material lattice avoiding the formation of structural defects [21].

In this paper, we propose using a Topological Insulator/Magnetic Insulator van
der Waals interface to enable the creation and design of a large variety of
topological spin-textures. Our proposal is based on a systematic study of the
electronic states of a CrI3/Bi2Se3 interface using realistic ab initio
calculations. We choose the CrI3/Bi2Se3 interface as a study platform due to
its amenable synthesis and recent experimental interest [22; 23; 24; 25]. In
addition, Bi2Se3 is one of the most theoretically and experimentally studied
three-dimensional topological insulators [26; 27; 28; 29; 30], whereas CrI3
stands as a robust ferromagnetic (FM) insulator [31; 32; 23], being a
promising platform for two-dimensional magnetism due to its magnetic
anisotropy [31; 32], and all-optical control of magnetization [33]. We provide
an interpretation of the results employing a simple effective model that
captures the main features of coupling between the TI and FM surfaces.
Further, we discuss some features of the disorder scattering of electrons in
TI states depending on the FM magnetization direction.

This paper is organized as follows: In Sec. II we present the geometry of the
CrI3/Bi2Se3 interface, the methodology used in first-principles calculations,
as well as the effective Hamiltonian model that describes the interface states
and the behavior of the scattering processes. In Sec. III, we analyze the
behavior and properties of spin-orbital textures and scattering rates as a
function of different interface degrees of freedom. We present our conclusions
in Sec. IV.

##  II Methods

###  II.1 CrI3/Bi2Se3 Interface

The CrI3/Bi2Se3 interface is constructed by considering the CrI3 layer on the
surface of the Bi2Se3 with quintuple layers (QLs) stacking, as illustrated in
Fig. 1(a). To achieve a lattice match between the two materials, we adopt a
supercell of $(\sqrt{3}\times\sqrt{3})$-Bi2Se3, with the CrI3 layer slightly
strained ($\sim 2.17\%$). It is worth pointing out that for such strain value,
the CrI3 system retains its ferromagnetic and insulating character. Among the
possible alignments between the Bi2Se3 and CrI3 layers, we use the lowest
energy configuration, namely, that with the Cr atom positioned atop the Se
atom [34]. The optimized separation distance between the two materials is
$3.07$ Å.

Figure 1:  (a) Side view of the CrI3/Bi2Se3 interface. CrI3 layer, indicating
the in-plane (b) and out-of-plane (c) magnetization directions adopted in our
calculations. (d) Electronic structure emphasizing the absence of overlap
between CrI3 and Bi2Se3 states in low-energy states (close to the Fermi energy
$E_{F}$). The mixing of the color pattern indicates states with contributions
coming from different regions of the material.

###  II.2 Density functional theory

The first principles calculations are performed using the density functional
theory (DFT) [35; 36] within the generalized gradient approximation (GGA) for
the exchange and correlation functional, employing the Perdew-Burke-Ernzerhof
(PBE) parametrization [37]. A fully relativistic $j$-dependent
pseudopotential, within the projector augmented wave method (PAW) [38; 39],
was used in the noncollinear spin-DFT formalism self-consistently. We use the
Vienna ab initio Simulation Package (VASP) [40; 41], with plane wave basis set
with a cut-off energy of 400 eV. The Brillouin zone is sampled using a number
of $k$-points such that the total energy converges within the meV scale. The
vacuum space is set to $15$ Å and atomic structures are optimized requiring
that the force on each atom to be less than $0.01$ eV/Å.

To accurately describe the strong relativistic effect produced by the Bi and I
atoms, as well as the weak van der Waals interaction between the Bi2Se3 and
CrI3 layers, we include the SOC term [42] and the nonlocal vdW functional
(optB86b-vdW) [43]. Finally, the strong exchange effect due to $d$-orbitals of
the Cr atoms is accounted for by introducing an on-site Coulomb $U_{\rm
eff}=2.1$ eV interaction within the L(S)DA+U approximation [44].

The 2D spin-orbital textures are computed on a $21\times 21$ $k$-points grid
over the $k_{x}k_{y}$ plane in the BZ. More specifically, we study the 2D
spin-orbital textures over wide and small reciprocal space regions around the
$\Gamma$-point, namely, $-0.1(2\pi/a)\leq k_{x},k_{y}\leq 0.1(2\pi/a)$ and
$-0.01(2\pi/a)\leq k_{x},k_{y}\leq 0.01(2\pi/a)$. The analysis of the DFT
results, including band structure plots, surface/layer contribution
projections, and spin textures along 2D Fermi surfaces, has been performed
using the VASProcar post-processing code [45].

###  II.3 Effective Hamiltonian

Bi2Se3 is a 3D topological insulator with electronic surface states
characterized by a Dirac-like dispersion and chiral spin textures, as shown by
our DFT results (see, for instance, Figs. 1 and 5). The surface states are
nicely described by the effective Hamiltonian model

$\hat{H}_{0}=\hbar v_{\rm F}\left({\bf k}\times\bm{\sigma}\right)\cdot\hat{\bf
z}\otimes\tau_{z}\,$ (1)

where ${\bm{\sigma}}=(\sigma_{x},\sigma_{y},\sigma_{z})$ are Pauli matrices
operating on the electron spin degree of freedom (up/down), whereas
${\bm{\tau}}=(\tau_{x},\tau_{y},\tau_{z})$ act on the surface degree of
freedom (top/bottom). For clarity, the $2\times 2$ identity matrix is
represented by $\sigma_{0}$ and $\tau_{0}$.

For thin slabs (3QLs) an inter-surface coupling opens an energy gap at the
vertex of the Dirac cone. Such effect is captured by adding an inter-layer
coupling term to the effective model, Eq. (1), namely

$\hat{H}_{D}=\delta\,\sigma_{0}\otimes\tau_{x}\,,$ (2)

where $\delta$ stands for the inter-layer hybridization matrix element.

At the interface, the magnetic moments of the CrI3 couple to electronic spin
of the topological surface states, changing both their energy dispersion as
well as their spin texture. Figure 1(d) shows the energy bands of a system
formed by CrI3 on the surface of Bi2Se3 5QL, with out-of-plane magnetization.
Despite the magnetic moment been located at the Cr atoms, the Bi2Se3 surface
states effectively feel an almost uniform magnetic field. Therefore, we can
add an uniform magnetic field term $H_{M}$ to the effective model Hamiltonian

$\hat{H}_{M}=\sum_{i=t,b}\bm{m}_{i}\cdot\bm{\sigma}\otimes\tau_{i}$ (3)

here $\bm{m}_{i}=\hbar\mu_{b}\bm{B}_{i}/2$, where $\bm{B}_{i}$ is the
effective external magnetic field of the CrI3 in the top ($i=t$) and botton
($i=b$) interfaces, and $\mu_{b}$ the Bohr magneton.. Note also that
$\tau_{t/b}=\left(\tau_{z}\pm\tau_{0}\right)/2$ project the surface subspace,
and therefore $\hat{H}_{M}$ can describe a single or both surfaces interacting
with a uniform magnetic field. Given the possible asymmetric configuration
between top/botton surfaces, we also add an inversion symmetry breaking term

$\hat{V}=U\sigma_{0}\otimes\tau_{z},$ (4)

with $U=eV$, $e$ the electron charge and $V$ the potential difference between
surfaces. Here, the inversion symmetry breaking term represents a dipole
moment originated by the presence of CrI3. Similar effect can also be induced
by an external electric field due to a gate.

We can write the low energy Hamiltonian in the basis
$\\{|t,\uparrow\rangle,|t,\downarrow\rangle,|b,\uparrow\rangle,|b,\downarrow\rangle\\}$,
where $t\;(b)$ stand for the top (bottom) surface and $\uparrow(\downarrow)$
for spin up (down) in the $\sigma_{z}$ basis, as

$H=\begin{pmatrix}U+m_{t,z}&m_{t,x}+\hbar v_{F}\tilde{k}^{*}&\delta&0\\\
m_{t,x}+\hbar v_{F}\tilde{k}&U-m_{t,z}&0&\delta\\\
\delta&0&-U+m_{b,z}&m_{b,x}-\hbar v_{F}\tilde{k}^{*}\\\ 0&\delta&m_{b,x}-\hbar
v_{F}\tilde{k}&-U-m_{b,z}\end{pmatrix},$

where we define $\tilde{k}=k_{y}+ik_{x}=ke^{i\theta}$ with $\theta$ with
respect to the $k_{y}$ direction.

Figure 2 shows a plethora of spin textures in the 2D Fermi surfaces obtained
from the study of the effective model, for different possible parameters. For
the CrI3/Bi2Se3 interfaces, several of the effective Hamiltonian parameters
are extracted from DFT calculations and are, from now on, fixed. Particularly
for the CrI3/Bi2Se3 interfaces we have extracted from the density functional
theory calculations each parameter for the effective Hamiltonian. For
instance, $\hbar v_{F}=1.5$ eV for all cases; $\delta=7.5$ meV and $0.1$ meV
for the 3QL and 5QL slabs, respectively; $U=19.5$ meV and $4.9$ meV for single
surface interfaces 3QL and 5QL slabs, respectively, and zero for both surfaces
interface, i.e. CrI3/Bi2Se3/CrI3; the magnetization term $m_{\beta,i}=6.4$ meV
for $\beta=t,\,b$ and $i=x,\,z$.

Figure 2:  Family of spin textures and 2D Fermi surfaces generated by the
effective Hamiltonian, by modulating thedifferent degrees of freedom of the
TI/MI interface.

###  II.4 Scattering Probability

In order to estimate the robustness of the different spin texture
configurations against disorder, we study the scattering probability from a
given initial state $|n_{1},{\bf k}_{1}\rangle$ to a final state $|n_{2},{\bf
k}_{2}\rangle$, where $n_{1},n_{2}$ stand for band indices and ${\bf
k}_{1},\,{\bf k}_{2}$ for the crystal momenta. For a given impurity potential
$\hat{H}^{\prime}$, we estimate for the transition probability between initial
and final states using the Fermi golden rule

$\Gamma_{i\rightarrow f}=\frac{2\pi}{\hbar}|\langle n_{2},{\bf
k}_{2}|\hat{H}^{\prime}|n_{1},{\bf k}_{1}\rangle|^{2}\rho(E_{n_{2},{\bf
k}_{2}}),$ (5)

where $\rho(E)$ is the density of states at the energy $E$.

Since we are interested in general properties of elastic scattering processes
further simplifications is in order. To avoid addressing specifics of the
electron-impurity cross sections, we consider disorder processes that does not
hybridize different surface states and non-magnetic impurities, that do not
flip spin. Further, we address the long range disorder limit, for which the
scattering lengths $a_{s}$ are much larger than the electron wave length
$\lambda$, namely, $a_{s}\gg\lambda$. Hence, we can approximate the transition
matrix squared by a product of a function that depends on some average
impurity potential times a 4-dimensional spinor (spin+surface) projection and
write

$|\langle n_{2},{\bf k}_{2}|\hat{H}^{\prime}|n_{1},{\bf
k}_{1}\rangle|^{2}\propto|\langle n_{2},{\bf k}_{2}|n_{1},{\bf
k}_{1}\rangle|^{2}.$ (6)

Consequently, $\Gamma_{i\rightarrow f}\propto|\langle n_{2},{\bf
k}_{2}|n_{1},{\bf k}_{1}\rangle|^{2}$. These approximations allow us to study
the impurity independent term and to infer some anisotropic features of
disorder scattering processes.

##  III Results and discussion

Here we present a comprehensive ab initio characterization of the electronic
structure and spin-orbital texture behavior of CrI3/Bi2Se3 interfaces. In
addition, we interpret the underlying physics in term of the effective
Hamiltonian and scattering probabilities introduced above.

Our calculations explore different system settings and configurations, namely,
($i$) finite-size effects caused by a TI slab width, ($ii$) the magnetization
direction of the FM, and ($iii$) the presence of CrI3 in the slab interface
(single or both surfaces). The interplay of the effects corresponding to the
later settings give rise to a rich variety of emergent spin-orbital texture
configurations.

###  III.1 Low-Energy Dispersion and 2D Fermi Surfaces

Figure 3:  Out-of-plane magnetization. Projection of the top and bottom
surfaces contribution to the low energy dispersion. Spin textures over the 2D
Fermi surfaces, where the size of the arrows and the colors represent the
contribution surfaces to the spin vector $(S_{x},\,S_{y})$ in the
$(k_{x},\,k_{y})$ plane. Scattering probability, for different Fermi energies
(in eV).

The control of finite-size effects is achieved by varying the thickness of the
Bi2Se3 stacking. We show that, while there is a significant surface states
hybridization in 3 QLs systems, the coupling is practically absent in 5 QLs.
This is inferred by observing that for thinner slabs (3 QLs), the topological
surface states from opposing surfaces are coupled, opening an energy gap at
the crossing of states with the same spin. This overlap is suppressed in 5QLs
slabs ($\delta\sim 0$). For instance, comparing Fig. 3(a) with Fig. 3(b),
corresponding to 5QL and 3QL systems, respectively, (keeping other degrees of
freedom fix) we see a gap opening and a superposition between top (red) and
bottom (blue) surface states. In those systems, there is a CrI3/Bi2Se3
interface only at one of the Bi2Se3 slab surfaces, with the CrI3 magnetization
normal to the interface, namely, $n$QL+1CrI3($m_{z}$) with $n=$ 3 and 5.

Regarding the magnetization direction of CrI3, we analyzed the magnetizations
in-plane ($m_{x}$) and out-of-plane ($m_{z}$). The main effect of $m_{z}$
magnetization is to induce an energy gap in the surface states adjacent to the
CrI3 layer, due to the introduction of coupling between states of the same
surface and opposite spins (terms $m_{t/b,z}$ in the Hamiltonian).
Additionally, there is a Zeeman-like energy shift of the Dirac states, lifting
the degeneracy of the surface states, as shown in Fig. 3.

Figure 4:  In-plane magnetization. Projection of the top and bottom surfaces
contribution to the low energy dispersion. Spin textures over the 2D Fermi
surfaces, where the size of the arrows and the colors represent the
contribution surfaces to the spin vector $(S_{x},\,S_{y})$ in the
$(k_{x},\,k_{y})$ plane. Scattering probability, for different Fermi energies
(in eV).

In turn, the in-plane $m_{x}$ magnetization preserves the crossing of the
Dirac cones. However, an efficient switching occurs between the magnetic
moment direction and the adjacent surface states, resulting in a momentum
shift of the states perpendicular to the magnetization direction, whose
direction is associated with helicity of the spin texture, leading to the
breaking of Fermi contours concentricity, as shown in Fig. 4. Hence, by
modulating the intensity and direction of magnetization, allow for the control
of the direction and shape of both the energy dispersion and the spin
component orientation of the topological states of Bi2Se3, a feature which is
potentially advantageous for spintronic applications. Conversely, the
reciprocal effect of this coupling can enable electrical control of the
magnetization direction of CrI3 when a charge current is introduced in Bi2Se3,
given the well stablished presence of the spin-orbit torque (SOT) effect
mediated by REE in TI/FM interfaces with in-plane magnetization [46; 18], with
potential application for the development of magnetic memories.

###  III.2 Spin-Orbital Textures

The surface states of Bi2Se3 slabs show a very strong contribution from the
$p_{z}$ orbital, which is responsible for their helical spin texture, with a
spin-momentum locking characteristic of topological states. Meanwhile, the in-
plane $p_{x}$ and $p_{y}$ orbitals induce more complex and opposing spin
textures, which cancel each other out, as seen in Fig. 5 for the upper Dirac
cone of a 5QL stacking of Bi2Se3. Interestingly, several studies have reported
[47; 48; 49; 50; 51] that in Bi2Se3, in addition to the usual spin-momentum
locking, the spin texture is tied to the orbital texture, giving rise to the
so-called spin-orbital texture, with the $p_{z}$ orbital perpendicular to the
layer’s plane, while the $p_{x}$ and $p_{y}$ orbitals respectively form a
radial and tangential pattern for the upper Dirac cone, and vice-versa for the
lower cone.

For each of the analyzed CrI3/Bi2Se3 interface configurations, we have
examined the behavior of the spin-orbital textures arising from the $p_{x}$,
$p_{y}$, and $p_{z}$ orbitals. By comparison with the texture pattern
originating from the stacking of pristine Bi2Se3 (see Fig. 5), we have
verified that the main effects consist of the relative shifts in energy and
momentum of the textures derived from opposite surfaces, as well as a blending
of the textures patterns due to overlapping states. However, the predominant
features of the spin texture from the $p_{z}$ orbital, and the mutual
cancellation of textures related to the $p_{x}$ and $p_{y}$ orbitals, remain
unchanged, indicating that the magnetization originating from the Cr atom does
not substantially modify the orbital distribution of the topological states of
Bi2Se3. Due to the fact that the spin-orbital texture associated with the
$p_{z}$ full is equivalent to the complete spin texture of the topological
state, and therefore relevant to scattering processes.

Figure 5:  Spin-orbital texture (near the $\Gamma$ point) corresponding to the
$p_{x}$, $p_{y}$, and $p_{z}$ orbitals of the surface states of a 5 QL
stacking of Bi2Se3.

Regarding the $p_{z}$ spin-orbital texture, all the configurations we analyzed
preserve the spin-momentum locking at the 2D Fermi surface, except for the
3QL+2CrI3($m_{x}$) configuration [see the Fig.4(d)]. For this specific case,
we observe that the spin configuration near the $\Gamma$ point becomes uniform
in space and independent of momentum, similar to a persistent spin helix
(PSH). In addition to obtaining the PSH state, we verify that the resulting
spin texture aligns perpendicularly to the magnetization direction. Thus, it
can be controlled through the tuning of the in-plane magnetization direction.
This result aligns with one of the challenges in spintronics, which is
obtaining a material where PSH is an intrinsic, controllable, and robust
property of the system.

Another unusual spin texture occurs for a similar configuration, now with out-
of-plane magnetization [see the Fig. 3(d)]. In the 3QL+2CrI3($m_{z}$)
configuration, the combination of $m_{z}$ magnetization on both surfaces with
the hybridization effect leads each state to be distributed throughout the
Bi2Se3 stacking and exhibit a null net spin texture. However, when projecting
the spin components onto each surface, we observe spin textures with opposite
chiralities, similar to those of the non-hybridized surface state, but here,
each state has equal contributions from both surfaces.

Figure 6:  Out-of-plane magnetization: Scattering probabilities as a function
of $E_{F}$ and propagation direction of the state along the BZ (each figure
within the same frame refers to a specific position at moment $\vec{k}_{1}$ of
the initial state, respectively, forming angles of 0∘, 45∘ and 90∘ with
respect to the $k_{x}$ axis). Figure 7:  In-plane magnetization: Scattering
probabilities as a function of $E_{F}$ and deflection angle of the state along
the BZ. Each figure within the same frame refers to a different orientation of
the initial state ${\bf k}_{1}$ namely, 0∘, 45∘ and 90∘ with respect to the
$k_{x}$ axis).

###  III.3 Anisotropic scattering

Here, we analyze the scattering profile by setting the incomming state
$|n_{1},{\bf k}_{1}\rangle$ as a top surface state (colored in red in Figs. 3
and 4). We summarized our results in Figs. 6 and 7, depicting the scattering
profile for out-of-plane and in-plane CrI3 magnetizations, respectively. The
most notable features are an isotropic behavior of the scattering for the out-
of-plane magnetization ($m_{z}$) in Fig. 6 and an anisotropic behavior for the
in-plane magnetization ($m_{x}$) in Fig. 7.

For interfaces with in-plane magnetization, we find that the combination of
the $k$-moment shift of the Dirac cones on the surface covered by CrI3, with
the overlapping of the wave functions between opposite surface states,
promotes a strong anisotropy of the scattering profile as a function of the
propagation direction of the initial state, which intensifies in the low
energy limit (see the Figs. 4 and 7). However, the 5QL+1CrI3($m_{x}$)
configuration [Fig. 4(a)] does not present anisotropy, given the non displaced
Dirac cone at $k$-moment (CrI3 free surface). For this specific case, the
inter-surface overlap term ($\delta$) does not affect the scattering, as (i)
the intersection between states (above the Fermi level) only occurs at a
single $k$-point along the entire BZ and (ii) is smaller for 5QL system.

Back scattering is expected to be favored in 3QL stacking due to inter-surface
hybridization caused by finite size effects. This is in line with the results
shown in Figs. 6(b) and 7(b) for $E_{F}=0.012$ eV. At larger values of
$E_{F}$, backscattering is suppressed. Interestingly, the 3QL+2CrI3($m_{z}$)
configuration [Fig. 3(d)], deviates from tis behavior and show an unusual and
robust protection against backscattering for any analyzed energy value. Here a
resonance occurs between the interlayer coupling term $\delta$ and the
magnetization terms $m_{t/b,z}$, from DFT calculations we verify
$\delta\approx m_{t/b,z}$. This resonance introduces an additional symmetry in
the system: although time-reversal (TR) is broken a combination of TR and
inversion is still preserved ([$\mathcal{I}\mathcal{T}$]) leading to the
backscattering protection. On the other hand, in systems with 5QL
backscattering protection is due to the suppression of overlap between surface
states with opposite spin chiralities, as seen in the Figs. 3 and 4. However,
in the 5QL+2CrI3($m_{z}$) configuration [Fig. 3(c)] in the region near the
$\Gamma$-point, the states acquire a finite $S_{z}$ component with the same
direction, which in addition to the small (but finite) overlap between the
wave functions, results in the breakdown of protection against backscattering.

Finally, we emphasize that the effects mentioned above are more prominent in
the low energy limit, as the state is shifted to higher energies, the effects
of the coupling terms cease to be significant, with the behavior of the
pristine topological state (mainly protection against backscattering), being
gradually re-established.

##  IV Conclusions

In conclusion, we have demonstrate that the surface states of a 3D topological
insulator can give origin of a rich family of unconventional spin-orbital
texture configurations on the 2D Fermi surfaces when in contact with a 2D
ferromagnetic insulator. Further, we show that such van der Waals interface
provides simple handler to control over the spin degree of freedom. By mapping
each experimentally achievable texture configuration and estimating the
corresponding scattering rates, we observe that the scattering of states due
to the breaking of time-reversal symmetry is significant only at low energy
scales ($\sim 10$ meV). The expected robustness of topological states is
reestablished at higher energy values, even in the presence of magnetization.

Additionally, we note that magnetization in the plane of the ferromagnetic
layers induces anisotropy in transport phenomena and efficient switching
between the directions of the magnetic moment and spin orientation.
Furthermore, we find that properties of interest in spintronics can be
obtained for specific interface configurations when finite-sized effects are
present. These include robust protection against backscattering in the
presence of magnetization [3QL+2CrI3 ($m_{z}$)] and a unidirectional spin
configuration independent of momentum near the $\Gamma$ point [3QL+2Cr3
($m_{x}$)].

We believe that these results can provide an efficient guide for manipulating
spin-polarized states in topological systems, aiming to enhance control over
emerging surface phenomena. This includes improving spintronic mechanisms
directly related to spin texture configuration, such as charge/spin
interconversion, spin relaxation time suppression, and spin torque transfer,
as well as aiding in the modulation of transport properties and the
development of devices with lower energy loss.

###### Acknowledgements.

The authors acknowledge financial support from the Brazilian agencies FAPESP
(grants 22/08478-6, 19/20857-0, and 17/02317-2), CNPq (INCT - Materials
Informatics and INCT - Nanocarbono), FAPERJ, and LNCC - Laboratório Nacional
de Computação Científica for computer time (projects ScafMat2 and EMT2D).

## References

  * Sinova  _et al._ [2015] J. Sinova, S. O. Valenzuela, J. Wunderlich, C. H. Back, and T. Jungwirth, Rev. Mod. Phys. 87, 1213 (2015).
  * Edelstein [1990] V. Edelstein, Solid State Commun. 73, 233 (1990).
  * Ganichev  _et al._ [2002] S. D. Ganichev, E. L. Ivchenko, V. V. Bel’kov, S. A. Tarasenko, M. Sollinger, D. Weiss, W. Wegscheider, and W. Prettl, Nature 417, 153 (2002).
  * Žutić  _et al._ [2004] I. Žutić, J. Fabian, and S. Sarma, Rev. Mod. Phys. 76, 323 (2004).
  * Rashba [1960] E. I. Rashba, Sov. Phys.-Solid State 2, 1109 (1960).
  * Bychkov and Rashba [1984] Y. A. Bychkov and E. I. Rashba, JETP Lett. 39, 78 (1984).
  * Dresselhaus [1955] G. Dresselhaus, Phys. Rev. 100, 580 (1955).
  * Hasan and Kane [2010] M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010).
  * Qi and Zhang [2011] X.-L. Qi and S.-C. Zhang, Rev. Mod. Phys. 83, 1057 (2011).
  * Wolf  _et al._ [2001] S. A. Wolf, D. D. Awschalom, R. A. Buhrman, J. M. Daughton, S. von Molnár, M. L. Roukes, A. Y. Chtchelkanova, and D. M. Treger, Science 294, 1488 (2001).
  * Hirohata  _et al._ [2020] A. Hirohata, K. Yamada, Y. Nakatani, I.-L. Prejbeanu, B. Diény, P. Pirro, and B. Hillebrands, J. Magn. Magn. Mater. 509, 166711 (2020).
  * Tao and Tsymbal [2018] L. L. Tao and E. Y. Tsymbal, Nat. Commun. 9, 2763 (2018).
  * Fu [2011] L. Fu, Phys. Rev. Lett. 106, 106802 (2011).
  * Ando and Fu [2015] Y. Ando and L. Fu, Annu. Rev. Condens. Matter Phys. 6, 361 (2015).
  * Wan  _et al._ [2011] X. Wan, A. M. Turner, A. Vishwanath, and S. Y. Savrasov, Phys. Rev. B 83, 205101 (2011).
  * Xu  _et al._ [2015] S.-Y. Xu, I. Belopolski, N. Alidoust, M. Neupane, G. Bian, C. Zhang, R. Sankar, G. Chang, Z. Yuan, C.-C. Lee, S.-M. Huang, H. Zheng, J. Ma, D. S. Sanchez, B. Wang, A. Bansil, F. Chou, P. P. Shibayev, H. Lin, S. Jia, and M. Z. Hasan, Science 349, 613 (2015).
  * Sun  _et al._ [2019] R. Sun, S. Yang, X. Yang, E. Vetter, D. Sun, N. Li, L. Su, Y. Li, Y. Li, Z.-z. Gong, _et al._ , Nano Lett. 19, 4420 (2019).
  * Mellnik  _et al._ [2014a] A. Mellnik, J. Lee, A. Richardella, J. Grab, P. Mintun, M. H. Fischer, A. Vaezi, A. Manchon, E.-A. Kim, N. Samarth, _et al._ , Nature 511, 449 (2014a).
  * Ralph and Stiles [2008] D. Ralph and M. Stiles, J. Magn. Magn. Mater. 320, 1190 (2008).
  * Mellnik  _et al._ [2014b] A. Mellnik, J. Lee, A. Richardella, J. Grab, P. Mintun, M. H. Fischer, A. Vaezi, A. Manchon, E.-A. Kim, N. Samarth, _et al._ , Nature 511, 449 (2014b).
  * Li  _et al._ [2019] P. Li, J. Yu, J. Xu, L. Zhang, and K. Huang, Physica B: Condensed Matter 573, 77 (2019).
  * Zhang  _et al._ [2015] W.-B. Zhang, Q. Qu, P. Zhu, and C.-H. Lam, J. Mater. Chem. C 3, 12457 (2015).
  * Mounet  _et al._ [2018] N. Mounet, M. Gibertini, P. Schwaller, D. Campi, A. Merkys, A. Marrazzo, T. Sohier, I. E. Castelli, A. Cepellotti, G. Pizzi, _et al._ , Nat. Nanotechnol. 13, 246 (2018).
  * Wang and Law [2021] Z. Wang and S. Law, Cryst. Growth Des. 21, 6752 (2021).
  * Nasir  _et al._ [2022] S. Nasir, W. J. Smith, T. E. Beechem, and S. Law, J. Vac. Sci. Technol. A 41, 012202 (2022).
  * Hsieh  _et al._ [2008] D. Hsieh, D. Qian, L. Wray, Y. Xia, Y. S. Hor, R. J. Cava, and M. Z. Hasan, Nature 452, 970 (2008).
  * Hsieh  _et al._ [2009] D. Hsieh, Y. Xia, D. Qian, L. Wray, F. Meier, J. H. Dil, J. Osterwalder, L. Patthey, A. V. Fedorov, H. Lin, A. Bansil, D. Grauer, Y. S. Hor, R. J. Cava, and M. Z. Hasan, Phys. Rev. Lett. 103, 146401 (2009).
  * Zhang  _et al._ [2009] H. Zhang, C.-X. Liu, X.-L. Qi, X. Dai, Z. Fang, and S.-C. Zhang, Nat. Phys. 5, 438 (2009).
  * Zhang  _et al._ [2010] W. Zhang, R. Yu, H.-J. Zhang, X. Dai, and Z. Fang, New J. Phys. 12, 065013 (2010).
  * Yazyev  _et al._ [2010] O. V. Yazyev, J. E. Moore, and S. G. Louie, Phys. Rev. Lett. 105, 266806 (2010).
  * Huang  _et al._ [2017] B. Huang, G. Clark, E. Navarro-Moratalla, D. R. Klein, R. Cheng, K. L. Seyler, D. Zhong, E. Schmidgall, M. A. McGuire, D. H. Cobden, _et al._ , Nature 546, 270 (2017).
  * Lado and Fernández-Rossier [2017] J. L. Lado and J. Fernández-Rossier, 2D Mater. 4, 035002 (2017).
  * Zhang  _et al._ [2022] P. Zhang, T.-F. Chung, Q. Li, S. Wang, Q. Wang, W. L. Huey, S. Yang, J. E. Goldberger, J. Yao, and X. Zhang, Nature materials 21, 1373 (2022).
  * Hou  _et al._ [2019] Y. Hou, J. Kim, and R. Wu, Sci. Adv. 5, eaaw1874 (2019).
  * Hohenberg and Kohn [1964] P. Hohenberg and W. Kohn, Phys. Rev. 136, B864 (1964).
  * Kohn and Sham [1965] W. Kohn and L. J. Sham, Phys. Rev. 140, A1133 (1965).
  * Perdew  _et al._ [1996] J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * Blöchl [1994] P. E. Blöchl, Phys. Rev. B 50, 17953 (1994).
  * Corso [2010] A. D. Corso, Phys Rev B 82, 075116 (2010).
  * Kresse and Furthmüller [1996a] G. Kresse and J. Furthmüller, Comput. Mater. Sci. 6, 15 (1996a).
  * Kresse and Furthmüller [1996b] G. Kresse and J. Furthmüller, Phys. Rev. B 54, 11169 (1996b).
  * Steiner  _et al._ [2016] S. Steiner, S. Khmelevskyi, M. Marsmann, and G. Kresse, Phys. Rev. B 93, 224425 (2016).
  * Klimeš  _et al._ [2009] J. Klimeš, D. R. Bowler, and A. Michaelides, J. Phys. Condens. Matter 22, 022201 (2009).
  * Andriotis  _et al._ [2010] A. N. Andriotis, R. M. Sheetz, and M. Menon, Phys. Rev. B 81, 245103 (2010).
  * Araújo and Maciel [2023] A. L. Araújo and R. P. Maciel, “Vasprocar: Python tools for dft calculations,”  (2023).
  * Moghaddam  _et al._ [2020] A. G. Moghaddam, A. Qaiumzadeh, A. Dyrdał, and J. Berakdar, Phys. Rev. Lett. 125, 196801 (2020).
  * Zhang  _et al._ [2013] H. Zhang, C.-X. Liu, and S.-C. Zhang, Phys. Rev. Lett. 111, 066801 (2013).
  * Zhu  _et al._ [2013] Z.-H. Zhu, C. Veenstra, G. Levy, A. Ubaldini, P. Syers, N. Butch, J. Paglione, M. Haverkort, I. Elfimov, and A. Damascelli, Phys. Rev. Lett. 110, 216401 (2013).
  * Cao  _et al._ [2013] Y. Cao, J. Waugh, X. Zhang, J.-W. Luo, Q. Wang, T. Reber, S. Mo, Z. Xu, A. Yang, J. Schneeloch, G. D. Gu, M. Brahlek, N. Bansal, S. Oh, A. Zunger, and D. S. Dessau, Nat. Phys. 9, 499 (2013).
  * Xie  _et al._ [2014] Z. Xie, S. He, C. Chen, Y. Feng, H. Yi, A. Liang, L. Zhao, D. Mou, J. He, Y. Peng, _et al._ , Nat. Commun. 5, 3382 (2014).
  * Waugh  _et al._ [2016] J. A. Waugh, T. Nummy, S. Parham, Q. Liu, X. Zhang, A. Zunger, and D. S. Dessau, npj Quantum Mater, 1, 16025 (2016).


# Applying a Generic Sequence-to-Sequence Model for Simple and Effective
Keyphrase Generation

Md Faisal Mahbub Chowdhury, Gaetano Rossiello, Michael Glass,
Nandana Mihindukulasooriya  Alfio Gliozzo
IBM Research AI, Yorktown Heights, NY, USA
{mchowdh, mrglass<EMAIL_ADDRESS>
{gaetano.rossiello<EMAIL_ADDRESS>
###### Abstract

In recent years, a number of keyphrase generation (KPG) approaches were
proposed consisting of complex model architectures, dedicated training
paradigms and decoding strategies. In this work, we opt for simplicity and
show how a commonly used seq2seq language model, BART, can be easily adapted
to generate keyphrases from the text in a single batch computation using a
simple training procedure. Empirical results on five benchmarks show that our
approach is as good as the existing state-of-the-art KPG systems, but using a
much simpler and easy to deploy framework.

##  1 Introduction

Keyphrases are a collection of salient terms that summarize a document.
Keyphrase generation (KPG) is unique from other NLG tasks since the target
model prediction is to generate multiple sequences (i.e. a set of multi-word
phrases) rather than a single sequence (for other tasks such as summarization
and translation) Meng et al. (2021).

As pointed out by Meng et al. (2021), the ever-growing collection of KPG
studies, albeit novel, are overwhelming from their model design to hyper-
parameter selection. It is worth to mention that, as of now, _none of the
existing KPG approaches_ achieves outright state-of-the-art results on all of
the five widely used keyphrase benchmarks.

Çano and Bojar (2019) experimented with popular text summarization neural
architectures on several datasets. They found, despite using large quantities
of data and many days of computation, the advanced deep learning based
summarization models could not produce better keyphrases than the specifically
designed keyphrase generation and extraction models. In this work, we revisit
the above finding regarding the effectiveness of summarization models for
keyphrases.

Document (only abstract shown below)
---
In order to monitor a region for traffic traversal, sensors can be deployed to
perform collaborative target detection. Such a sensor network achieves a
certain level of detection performance with an associated cost of deployment.
This paper addresses this problem by proposing path exposure as a measure of
the goodness of a deployment and presents an approach for sequential
deployment in steps. It illustrates that the cost of deployment can be
minimized to achieve the desired detection performance by appropriately
choosing the number of sensors deployed in each step.
Generated present keyphrases: target detection, path exposure, sensor network,
collaborative target detection, sequential deployment
Generated absent keyphrases: traffic monitoring, sensor networks, sensor
deployment, deployment strategy
Table 1: An example of keyphrases generated using our fine-tuned model of
BART-large Lewis et al. (2020a). The document is selected from the SemEval
dataset.

Recently, some proposed works Lewis et al. (2020c); Petroni et al. (2021) show
that a multitude of downstream NLP tasks (e.g. question answering, fact
checking, dialog) can be effectively addressed using a unified generative
framework without designing ad-hoc architectures for individual task.
Moreover, standard seq2seq (aka encoder-decoder) models Lewis et al. (2020a);
Raffel et al. (2020) have been successfully applied to extract structured
information from text, such as relation linking Rossiello et al. (2021), text-
to-table Wu et al. (2021), text-to-RDF Agarwal et al. (2020); Guo et al.
(2020), intent classification Ahmad et al. (2021), and slot filling Glass et
al. (2021). So far, there is no empirical evidence showing effectiveness of
such a simple generative paradigm for KPG.

Inspired by the aforementioned literature, this study examines whether a
standard text summarization neural architecture can be used for KPG. We
demonstrate that, even with default hyper-parameters and limited fine tuning,
our system can yield results on par (in some cases even better) with the
state-of-the-art systems that have specifically designed architectures for
KPG. We hope this study will have a substantial impact on implementing real-
world KPG systems, therefore facilitating simplification of the development
and deployment process.

##  2 Related Work

KPG maps a sequence (the text) to a set of sequences (the keyphrases). This
task is typically transformed to a seq2seq setting either through One2One or
One2Seq. In One2One, during training each keyphrase is paired with the input
text as the target output. Then at test time the different keyphrases are
taken from the different beams of the generation process. In contrast, One2Seq
uses the list of keyphrases as the target text during training. At test time
the keyphrases are taken from the top-scoring beam or merged from the top-k
beams.

Meng et al. (2017) was the first to propose a KPG approach, called CopyRNN.
They analyzed a number of benchmarks and found a considerable number (ranges
between 44–58%) of keyphrases do not appear in the corresponding documents.
Such keyphrases are labelled as _absent keyphrases_ as opposed to the _present
keyphrases_ that appear in the documents. CopyRNN is based on One2One training
paradigm. Several subsequent studies Chen et al. (2018); Zhao and Zhang (2019)
are extensions of CopyRNN framework. Some Ye and Wang (2018); Chen et al.
(2019b); Kim et al. (2021) exploited document structure information to improve
KPG performance.

Yuan et al. (2020) implemented an approach based on One2Seq paradigm to
generate diverse keyphrases and control number of outputs by manipulating
decoder hidden states.

Unlike the aforementioned studies, Chen et al. (2020) and Chan et al. (2019)
proposed reinforcement learning (RL) based KPG approaches. Chen et al. (2019a)
proposed multi-task learning framework to jointly learn extractive and
generative model for keyphrases and exploited information retrieval. They did
not report separated results for present and absent keyphrases.

Ye et al. (2021) proposed a graph neural network based approach that capture
explicit knowledge from a predefined index (e.g., the training set) that is
similar to the input document. This idea is intuitively similar to what was
proposed by Chen et al. (2019a). Since we do not use explicit knowledge from
other documents, these results are not comparable to ours.

##  3 Proposed Approach

We exploit BART Lewis et al. (2020b), a denoising self-supervised autoencoder,
for pretraining seq2seq models. It is a standard seq2seq framework that has
been used to produce state-of-the-art results on a variety of NLG tasks
including dialogue, question answering, abstractive summarization and
translation. It has a bidirectional encoder (like BERT) and a left-to-right
decoder (like GPT). BART uses a noise-added source text (e.g. by corrupting
some tokens in source text) as input. It later uses language model for
reconstructing the original text by predicting the true replacement of
corrupted tokens.

There are two pretrained generic BART models, dubbed as bart-base and bart-
large. They are made publicly available by Wolf et al. (2020).

###  3.1 Proposed training procedure

Let us define $D={\\{w_{1},\dots,w_{n}\\}}$ as a document containing $n$
words, and $K={\\{k_{1},\dots,k_{m}\\}}$ as a set consisting of $m$
keyphrases111Keyphrases can be represented as one or more words.. We frame the
KPG task as a sequence-to-sequence problem by fine-tuning BART using its
conditional generation method typically used, e.g., for text summarization.

For this purpose, we need to define the representations for both source ($S$)
and target ($T$) sequences. $S$ is represented as a sequence of WordPieces
obtained by applying the BART tokenizer on the text in $D$. In order to create
the sequence $T$, we organize all the keyphrases in $K$ in the following
format: [k1, k2, ..., km]. We organize the keyphrases in the same order as
given in the ground truth. $T$ is obtained by applying the BART tokenizer on
this sequence. This representation allows us to fine-tune the model by
generating all the keyphrases jointly in a single decoding computation.
Moreover, the characters ([],) are used as special tokens allowing us to
easily parse the target sequence in order to extract the single keyphrases
later during the inference phase. Then, BART is fine-tuned by maximizing the
probability $P(T|S)=\prod_{i=1}^{m}P(t_{i}|t_{<m},S)$ across all the instances
in the given training set. We deliberately avoided adding new special tokens,
e.g. to separate the keyphrases in the target sequence, in order to keep the
model as simple as possible, thus promoting the usage of off-the-shelf
training procedures provided in the Huggingface library.

It is worth noting that BART cannot handle documents having more than 1024
tokens. To address this issue, we split the input document into multiple
paragraphs. We split222Using NLTK - https://www.nltk.org/ the document into
sentences and construct paragraphs by merging consecutive sentences until the
maximum length accepted by the encoder is reached. Then, we consider the
keyphrases provided as ground truth for the original document as the ground
truth for each of these paragraphs. With this strategy, we treat each
paragraph as an independent training instance.

###  3.2 Proposed inference procedure

Let’s assume given an input test document, the system is asked to generate up
to $N$ keyphrases. First, like in the training procedure, the document is
split into paragraphs. Second, the keyphrase model is used to generate ranked
list of $N$ keyphrases per paragraph.Let $\mathcal{K}(p)$ be the ordered list
of keyphrases generated for the paragraph $p$. The rank of each keyphrase
$\kappa$ in $\mathcal{K}(p)$ is given by $rank(\kappa,\mathcal{K}(p))$. Then
the $score$ for a keyphrase is the sum of its inverse ranks over all the
paragraphs in the document. We add $\epsilon=1/(N+1)$ to break score ties by
the number of times a keyphrase is generated.

$\displaystyle score(\kappa)=\sum_{p_{i}\in
D|\kappa\in\mathcal{K}(p_{i})}\frac{1}{rank(\kappa,\mathcal{K}(p_{i}))}+\epsilon$

Finally the top scoring $N$ keyphrases are selected as the final keyphrases
for the document.

##  4 Experiments

###  4.1 Datasets

We use the following 5 keyphrase benchmarks consisting of scientific
publications for evaluation that have been widely used in existing literature
- Inspec Hulth (2003), NUS Nguyen and Kan (2007), SemEval Kim et al. (2010),
Kp20K Meng et al. (2017) and Krapivin Krapivin et al. (2009). We downloaded
them from https://github.com/memray/OpenNMT-kpg-release shared by Meng et al.
(2021). Due to space limitation, we refer the readers to Meng et al. (2021) or
Yuan et al. (2020) for various details and statistics about these datasets.
All of these datasets have a test and validation data split. Kp20K also has a
train split containing 500K+ abstracts. SemEval, Krapivin and NUS also contain
full texts in addition to abstracts.

Model | Inspec | NUS | SemEval | Kp20K | Krapivin
---|---|---|---|---|---
F@5 |  F@10 |  F@5 |  F@10 |  F@5 |  F@10 |  F@5 |  F@10 |  F@5 |  F@10
ExHiRD-h Chen et al. (2020) |  25.3 |  - |  - |  - |  28.4 |  - |  31.1 |  - |  28.6 |
Transformer Chen et al. (2020) |  21.0 |  - |  - |  - |  25.7 |  - |  28.2 |  - |  25.2 |
CopyRNN Meng et al. (2017) |  29.2 |  33.6 |  34.2 |  31.7 |  29.1 |  29.6 |  32.8 |  25.5 |  30.2 |  25.2
CorrRNN Chen et al. (2018) |  - |  - |  35.8 |  33.0 |  32.0 |  32.0 |  - |  - |  31.8 |  27.8
ParaNetT+CoAtt Zhao and Zhang (2019) |  29.6 |  35.7 |  36.0 |  35.0 |  31.1 |  31.2 |  36.0 |  28.9 |  32.9 |  28.2
catSeqTG-2RF Chan et al. (2019) |  25.3 |  - |  37.5 |  - |  28.7 |  - |  32.1 |  - |  30.0 |  -
CatSeq Yuan et al. (2020) |  29.0 |  30.0 |  35.9 |  34.9 |  30.2 |  30.6 |  31.4 |  27.3 |  30.7 |  27.4
CatSeqD Yuan et al. (2020) |  27.6 |  33.3 |  37.4 |  36.6 |  32.7 |  35.2 |  34.8 |  29.8 |  32.5 |  28.5
CatSeq+2RL(FB) Luo et al. (2021) |  26.7 |  - |  - |  - |  - |  - |  33.0 |  - |  30.5 |  -
GSEnc Kim et al. (2021) |  - |  - |  - |  - |  - |  - |  32.9 |  - |  - |
bart-base-kp (our) |  33.1 |  35.6 |  33.4 |  35.3 |  28.5 |  31.1 |  32.8 |  30.9 |  27.8 |  25.8
bart-large-kp (our) |  35.2 |  38.7 |  34.6 |  38.0 |  29.3 |  32.3 |  33.1 |  31.1 |  26.3 |  25.0
Table 2: Evaluation for present keyphrases.

###  4.2 Keyphrase model training

We follow a simple recipe for training to avoid over-optimizing the models
simply for the sake of obtaining the best possible results. For this reason,
we fine-tune the conditional generation task in BART using a set of default
hyper-parameters.333 _(i)_ Epochs: 3, _(ii)_ learning rate: 5e-05, _(iii)_
train batch size: 128, _(iv)_ Adam beta: 0.9, _(v)_ Adam epsilon: 1e-08.

With a small number of epochs (3), we use the train and validation splits in
Kp20K to train a single keyphrase model using BART-base and another single
model using BART-large. Henceforth, we will refer to them as _bart-base-kp_
and _bart-large-kp_. Unlike some previous approaches (e.g. Yuan et al.
(2020)), we do not do any pre-processing (e.g. lowercasing, digits
normalization, etc) of either the document texts or ground-truth.

Also, we do not fine tune the _bart-base-kp_ and _bart-large-kp_ models for
the other 4 non-Kp20K benchmarks using their corresponding validation split.

We used a machine with the following configuration for the training –
Processor: Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, CPU cores: 16, RAM:
118G, GPU: Tesla V100 PCIe 16G, GPU count: 2\. It took 21.82 hours to train
the _bart-base-kp_ model and 51.68 hours for _bart-large-kp_.

Model | Inspec | NUS | SemEval | Kp20K | Krapivin
---|---|---|---|---|---
R@10 | R@10 | R@10 | R@10 | R@10
CopyRNN Meng et al. (2017) | 5.1 | 7.8 | 4.9 | 11.5 | 11.6
CorrRNN Chen et al. (2018) | - | 5.9 | 4.1 | - | -
CatSeq Yuan et al. (2020) | 2.8 | 3.7 | 2.5 | 6.0 | 7.0
CatSeqD Yuan et al. (2020) | 5.2 | 8.4 | 4.6 | 11.7 | 12.0
bart-base-kp (our) | 4.8 | 5.6 | 3.0 | 6.1 | 11.2
bart-large-kp (our) | 5.4 | 4.4 | 2.8 | 6.1 | 13.2
Table 3: Evaluation for absent keyphrases.

###  4.3 Keyphrase prediction

As noted by Meng et al. (2021), most of the the existing approaches rely on
beam search (with large beam size, e.g. 50, 150 and 200 in Yuan et al. (2020);
Chen et al. (2019b); Meng et al. (2017) respectively) to over-generate lots of
keyphrases and then select the top N. This is not suitable in real-world
situations that demand faster response with less memory footprint. In our
experiments, we report results based on a default beam size _20_.

We generate 10 keyphrases per source text, and hence report macro-average
F-score@5 and F-score@10 for present keyphrases and macro-average Recall@10
for absent keyphrases over all the test documents in each benchmark.

We use the evaluation script of Meng et al. (2021). It replaces punctuation
marks with whitespace and tokenize texts/phrases using Python string method
split() to reduce phrase matching errors. We used the concatenation of title
and abstract as the source text and ignored full texts for the 3 benchmarks
where they were available.

##  5 Results and Analysis

#### Results:

As shown in Table 2 and 3, our model (bart-large-kp) achieves state-of-the-art
_F@10_ in 3/5 and _R@10_ in 2/5 benchmarks for present and absent keyphrases
respectively. Interestingly, our bart-base-kp model also produces very
competitive results.

#### Qualitative Analysis:

Given the comparatively low performance of our system for absent keyphrases in
SemEval, we examined 25% randomly selected SemEval test documents and manually
analyzed the absent keyphrases generated by the bart-large-kp model.

On average, each of these selected documents contains 8.36 ground-truth absent
keyphrases. But bart-large-kp generated only 2.05 absent keyphrases. So,
recall was the main issue here. It should be noted that 36% of the ground-
truth in Kp20K (our training data) are absent keyphrases, but that number is
55% in SemEval Meng et al. (2021). Also, SemEval has 3 times more ground-truth
per document than Kp20K.

We noticed 64% of the total generated absent keyphrases in the above selected
documents were found correct by the manual human analysis. But, 58% of them
were not in ground-truth, and could be considered as false negatives.

Regarding the generated absent keyphrases that we deemed incorrect (aka true
negatives), there was a common pattern of errors where a number of absent
keyphrases have spurious repetition of some tokens. For example, malmalware,
datadata transfer, pairpairwise key establishment, etc.

assessment but were not part of the gold standard). It highlights the
challenge of formulating gold standards for absent/generative keyphrases as
the number of possibilities are quite large.

##  6 Conclusions

Boudin et al. (2020) showed that KPG can significantly boost effectiveness of
information retrieval. But through extrinsic evaluation, they found existing
KPG models have limited generalization ability. In this work, we proposed an
effective KPG system implemented using the same standard procedures used to
fine-tune seq2seq language models on sequence generation tasks. This strategy
helps generalization since pre-trained transformer models, such as BART,
promote transferability across domains. In our experiments, we showed that the
same model fine-tuned on a single dataset can be competitive also on other
benchmarks compared with the existing state-of-the-art KPG models. Moreover,
its design simplicity is beneficial for deployment in real-world scenarios.

## References

  * Agarwal et al. (2020) Oshin Agarwal, Mihir Kale, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2020.  Machine translation aided bilingual data-to-text generation and semantic parsing.  In _Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+)_ , pages 125–130, Dublin, Ireland (Virtual). Association for Computational Linguistics.
  * Ahmad et al. (2021) Wasi Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, and Kai-Wei Chang. 2021\.  Intent classification and slot filling for privacy policies.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4402–4417, Online. Association for Computational Linguistics.
  * Boudin et al. (2020) Florian Boudin, Ygor Gallina, and Akiko Aizawa. 2020.  Keyphrase generation for scientific document retrieval.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1118–1126, Online. Association for Computational Linguistics.
  * Çano and Bojar (2019) Erion Çano and Ondřej Bojar. 2019.  Keyphrase generation: A text summarization struggle.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 666–672, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Chan et al. (2019) Hou Pong Chan, Wang Chen, Lu Wang, and Irwin King. 2019.  Neural keyphrase generation via reinforcement learning with adaptive rewards.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2163–2174, Florence, Italy. Association for Computational Linguistics.
  * Chen et al. (2018) Jun Chen, Xiaoming Zhang, Yu Wu, Zhao Yan, and Zhoujun Li. 2018.  Keyphrase generation with correlation constraints.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4057–4066, Brussels, Belgium. Association for Computational Linguistics.
  * Chen et al. (2019a) Wang Chen, Hou Pong Chan, Piji Li, Lidong Bing, and Irwin King. 2019a.  An integrated approach for keyphrase generation via exploring the power of retrieval and extraction.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2846–2856, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Chen et al. (2020) Wang Chen, Hou Pong Chan, Piji Li, and Irwin King. 2020.  Exclusive hierarchical decoding for deep keyphrase generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1095–1105, Online. Association for Computational Linguistics.
  * Chen et al. (2019b) Wang Chen, Yifan Gao, Jiani Zhang, Irwin King, and Michael R. Lyu. 2019b.  Title-guided encoding for keyphrase generation.  In _Proceedings of the The 33rd AAAI Conference on Artificial Intelligence (AAAI-19)_.
  * Glass et al. (2021) Michael R. Glass, Gaetano Rossiello, Md. Faisal Mahbub Chowdhury, and Alfio Gliozzo. 2021.  Robust retrieval augmented generation for zero-shot slot filling.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021_ , pages 1939–1949. Association for Computational Linguistics.
  * Guo et al. (2020) Qipeng Guo, Zhijing Jin, Ning Dai, Xipeng Qiu, Xiangyang Xue, David Wipf, and Zheng Zhang. 2020.  ${\mathcal{p}^{2}}$: A plan-and-pretrain approach for knowledge graph-to-text generation.  In _Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+)_ , pages 100–106, Dublin, Ireland (Virtual). Association for Computational Linguistics.
  * Hulth (2003) Anette Hulth. 2003.  Improved automatic keyword extraction given more linguistic knowledge.  In _Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing_ , pages 216–223.
  * Kim et al. (2021) Jihyuk Kim, Myeongho Jeong, Seungtaek Choi, and Seung-won Hwang. 2021.  Structure-augmented keyphrase generation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2657–2667, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Kim et al. (2010) Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2010.  SemEval-2010 task 5 : Automatic keyphrase extraction from scientific articles.  In _Proceedings of the 5th International Workshop on Semantic Evaluation_ , pages 21–26, Uppsala, Sweden. Association for Computational Linguistics.
  * Krapivin et al. (2009) Mikalai Krapivin, R Autaeu, Maurizio Marchese, Mikalai Krapivin, Aliaksandr Autaeu, and Maurizio Marchese. 2009.  Large dataset for keyphrases extraction.  Technical report.
  * Lewis et al. (2020a) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lewis et al. (2020b) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020b.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lewis et al. (2020c) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020c.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Luo et al. (2021) Yichao Luo, Yige Xu, Jiacheng Ye, Xipeng Qiu, and Qi Zhang. 2021.  Keyphrase generation with fine-grained evaluation-guided reinforcement learning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
  * Meng et al. (2021) Rui Meng, Xingdi Yuan, Tong Wang, Sanqiang Zhao, Adam Trischler, and Daqing He. 2021\.  An empirical study on neural keyphrase generation.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4985–5007, Online. Association for Computational Linguistics.
  * Meng et al. (2017) Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017.  Deep keyphrase generation.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_ , pages 582–592, Vancouver, Canada. Association for Computational Linguistics.
  * Nguyen and Kan (2007) Thuy Dung Nguyen and Min-Yen Kan. 2007.  Keyphrase extraction in scientific publications.  In _Asian Digital Libraries. Looking Back 10 Years and Forging New Frontiers_ , pages 317–326, Berlin, Heidelberg. Springer Berlin Heidelberg.
  * Petroni et al. (2021) Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021\.  KILT: a benchmark for knowledge intensive language tasks.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2523–2544, Online. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Rossiello et al. (2021) Gaetano Rossiello, Nandana Mihindukulasooriya, Ibrahim Abdelaziz, Mihaela A. Bornea, Alfio Gliozzo, Tahira Naseem, and Pavan Kapanipathi. 2021.  Generative relation linking for question answering over knowledge bases.  In _The Semantic Web - ISWC 2021 - 20th International Semantic Web Conference, ISWC 2021, Virtual Event, October 24-28, 2021, Proceedings_ , volume 12922 of _Lecture Notes in Computer Science_ , pages 321–337. Springer.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Wu et al. (2021) Xueqing Wu, Jiacheng Zhang, and Hang Li. 2021.  Text-to-table: A new way of information extraction.  _CoRR_ , abs/2109.02707.
  * Ye and Wang (2018) Hai Ye and Lu Wang. 2018.  Semi-supervised learning for neural keyphrase generation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4142–4153, Brussels, Belgium. Association for Computational Linguistics.
  * Ye et al. (2021) Jiacheng Ye, Ruijian Cai, Tao Gui, and Qi Zhang. 2021.  Heterogeneous graph neural networks for keyphrase generation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
  * Yuan et al. (2020) Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, and Adam Trischler. 2020.  One size does not fit all: Generating and evaluating variable number of keyphrases.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7961–7975, Online. Association for Computational Linguistics.
  * Zhao and Zhang (2019) Jing Zhao and Yuxiang Zhang. 2019.  Incorporating linguistic constraints into keyphrase generation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5224–5233, Florence, Italy. Association for Computational Linguistics.