aainstitutetext: School of Physics and Astronomy, University of Nottingham,
Nottingham NG7 2RD, UKbbinstitutetext: Nottingham Centre of Gravity,
University of Nottingham, Nottingham NG7 2RD, UK

# The effect of quantum correction on Hawking radiation for Schwarzschild
black holes

Yang Liu<EMAIL_ADDRESS>
###### Abstract

We investigate the effect of quantum correction on Hawking radiation for
Schwarzschild black holes. We consider Hawking temperature and entropy to
order $G^{2}$ and find that the area law of black holes should be modified. We
think of Hawking radiation as tunneling and find that at certain frequency
$\omega$ the radiation spectrum of black holes can be pure blackbody spectrum.
Therefore, it is possible to break down information conservation. In order to
ensure information conservation at any energy scales, we suggest that black
holes cannot release all information they have. We briefly discuss the bound
on greybody factors for Schwarzschild black holes as well. Since the
modification of horizon is very tiny, the bound on greybody factors has a very
small difference between classical metric and quantum corrected metric.

##  1 Introduction

In 1974, since Hawking’s original discovery [1,2] has showed that black holes
can spontaneously emit particles at a temperature inversely proportional to
their mass. Black holes, as a fascinating and elegant object, have been
increasingly popular in classical and quantum gravity theories [3]. Meanwhile,
a plenty of works [4–8] devoted to calculating the Hawking temperature and
radiation in order to explore a possible quantum gravity theory [3]. For
example, in refs.[4] and [5], Page has calculated the particle emission rates
from a nonrotating and rotating black hole, respectively. In 1999, Wilczek and
Parikh [6] showed that Hawking radiation can be viewed as a tunnelling
process, in which particles pass through the contracting horizon of the black
hole, lending formal justification to the most intuitive picture we have of
black hole evaporation [9].
Other studies have generalized these results to other cases, such as charged
black holes [3,10], Einstein-Gauss-Bonnet de Sitter black hole [11], etc. Here
we only list some recent studies. Avijit Chowdhury and Narayan Banerjee [10]
considered Hawking radiation from a charged spherical black hole with scalar
hair and showed that the scalar and electric charges contribute oppositely to
the greybody factor and the sparsity of the Hawking radiation cascade. Yan-
Gang Miao and Zhen-Ming Xu [3] investigated the Hawking radiation cascade from
the five-dimensional charged black hole with a scalar field coupled to higher-
order Euler densities in a conformally invariant manner. The authors found
that the Hawking radiation cascade from this five-dimensional black hole is
extremely sparse and the charge enhances the sparsity of the Hawking
radiation, while the conformally coupled scalar field reduces this sparsity.
Cheng-Yong Zhang, Peng-Cheng Li and Bin Chen [11] studied the greybody factors
of the scalar fields in spherically symmetric Einstein-Gauss-Bonnet–de Sitter
black holes in higher dimensions. They derived the greybody factors
analytically for both minimally and nonminimally coupled scalar fields and
found that the nonminimal coupling may suppress the greybody factor and the
Gauss-Bonnet coupling could enhance it, but they both suppress the energy
emission rate of Hawking radiation.
All articles, however, assume the metrics of black holes are classical, i.e.,
the quantum effect on metrics have been neglected. Due to effective field
theory (EFT) methods, the quantum corrected metric of Schwarzschild black
holes has been obtained recently [12]. Since the seminal work of Weinberg in
1979 [13], much progress has been made in quantum gravity using effective
field theory methods [14–20]. Although finding a consistent quantum gravity at
all energy scales is still an extremely difficult problem at the moment, EFT
methods can be applied at energies below the Planck mass and enables
calculations in quantum gravity which are model independent [12,21]. The only
two requirements are that Lorentz invariance is a fundamental symmetry and
general coordinate invariance is the correct symmetry at low energy scale
[12]. The quantum gravitational effective action contains two parts: local and
nonlocal operators. The Wilson coefficients of the local operators cannot be
calculated if we have no knowledge of the ultraviolet complete theory of
quantum gravity, however, the Wilson coefficients of the non-local operator
can be obtained from first principles and only depend on the infrared physics
which we have understood very well [21]. More related details can be found in
refs.[12,21].
The main aim of the original paper [12] is to describe the metric inside and
outside a star, taking into account quantum effects. However, the authors also
proposed that the exterior metric may provide a good model of quantum black
holes [12]. Therefore, one direct following research of ref.[12] is to
consider the quantum corrections to Hawking temperature and radiation power,
which is the research topic of this article. In section 2, we briefly review
the quantum corrected metric. In section 3, the quantum correction of Hawking
temperature and entropy for Schwarzschild black holes to order $G^{2}$ have
been obtained. In section 4, we think of Hawking radiation as tunneling. In
section 5, we determine the bound on greybody factors for Schwarzschild black
holes to the lowest order. In section 6, the results of this article have been
discussed. We take $c=k_{B}=\hbar=1$ throughout. $G$ is the Newton
gravitational constant.

##  2 Brief review of the quantum corrected metric

In this section, we will review the quantum corrected metric to order $G^{2}$
obtained from Effective Field Theory (EFT). The quantum effective action is
given by [12]:

$\Gamma[g]=\Gamma_{L}[g]+\Gamma_{NL}[g],$ (1)

where the local part of the action is given by [12]

$\Gamma_{L}[g]=\int d^{4}x\sqrt{g}[\frac{R}{16\pi
G}+c_{1}(\mu)R^{2}+c_{2}(\mu)R_{\mu\nu}R^{\mu\nu}+c_{3}(\mu)R_{\mu\nu\alpha\beta}R^{\mu\nu\alpha\beta}]$
(2)

and the non-local part of the action by [12]

$\Gamma_{NL}[g]=-\int d^{4}x\sqrt{g}[\alpha R\ln(\frac{\Box}{\mu^{2}})R+\beta
R_{\mu\nu}\ln(\frac{\Box}{\mu^{2}})R^{\mu\nu}+\gamma
R_{\mu\nu\alpha\beta}\ln(\frac{\Box}{\mu^{2}})R^{\mu\nu\alpha\beta}]$ (3)

Here, the parameters $\alpha$, $\beta$ and $\gamma$ are non-local Wilson
coefficients for fields. The values of the parameters for different fields
have been listed in ref.[12].
In ref.[12], the quantum corrected metric for a static and spherical star and
black hole has been obtained, which can be written as:

$ds^{2}=-f(r)dt^{2}+g(r)dr^{2}+r^{2}d\Omega^{2}$ (4)

where $d\Omega^{2}=d\theta^{2}+(\sin\theta)^{2}d\phi^{2}$. Outside the radius
of star $R_{s}$ (i.e., $r>R_{s}$), the metric functions are given by

$f(r)=1-\frac{2GM}{r}+\alpha_{e}(r)$ (5)
$g(r)=\left(1-\frac{2GM}{r}\right)^{-1}+\beta_{e}(r)$ (6)

where

$\alpha_{e}(r)=\tilde{\alpha}\frac{2G^{2}M}{R^{3}_{s}}[2\frac{R_{s}}{r}+\ln(\frac{r-R_{s}}{r+R_{s}})]+O(G^{3})$
(7) $\beta_{e}(r)=\tilde{\beta}\frac{2G^{2}M}{r(r^{2}-R^{2}_{s})}+O(G^{3})$
(8)

with

$\tilde{\alpha}=96\pi(\alpha+\beta+3\gamma)$ (9)
$\tilde{\beta}=192\pi(\gamma-\alpha)$ (10)

and $M$ is the mass of the black hole. We should emphasize that the
corrections to the metric should include two parts: one is from the local part
of $(2.1)$ (i.e., eq.$(2.2)$), while the other part is from the non-local part
of $(2.1)$ (i.e., eq.$(2.3)$) [12]. According to ref.[12], the local part has
no contribute. In particular, the corrections outside the star are trivially
$0$ while the corrections inside the star turn out to be order $O(G^{3})$, and
thus subleading [12]. The two terms $\tilde{\alpha}$ and $\tilde{\beta}$ are
the contributions of non-local part of $(2.1)$, i.e., eq.$(2.3)$. Therefore,
we cannot expand the first term of $(2.6)$ in powers of $G$ to obtain the term
$\tilde{\beta}$. Moreover, we should point out that the true perturbation
parameters are the inverse of the radius of the curvature in units of Planck
length and the compactness of the star, which are dimensionless [21].
By introducing the tortoise coordinate [21]

$r_{\ast}=\int^{r}\sqrt{\frac{g(r^{\prime})}{f(r^{\prime})}}dr^{\prime}$ (11)

The metric can be rewritten as [21]

$ds^{2}=-f(r)(dt^{2}+dr_{\ast}^{2})+r^{2}d\Omega^{2}$ (12)

In the exterior region, the tortoise coordinate is [21]

$r_{\ast}(r)=r+2GM\ln(\frac{r}{2GM}-1)+\frac{1}{2}\int_{r}^{\infty}[\alpha_{e}(r^{\prime})-\beta_{e}(r^{\prime})]dr^{\prime}+C,$
(13)

where $C$ is an integration constant.

##  3 Quantum correction of Hawking temperature and entropy for Schwarzschild
black holes

In this section, we study the semiclassical tunnelling of particles through
the horizon of Schwarzschild black holes, in line with the analysis of
ref.[9]. Further discussion can be seen in refs.[22-25]. On the one hand, the
emitted rate of particles $\Gamma$ is given by

$\Gamma\sim\exp\left(-2ImS\right)$ (14)

where $S$ is the tunnelling action of particles and $ImS$ is the imaginary
part of the tunnelling action $S$. On the other hand, based on the Planck
radiation law, the emitted rate $\Gamma$ of particles with frequency $\omega$
can be written as

$\Gamma\sim\exp\left(-\omega/T_{BH}\right)$ (15)

Therefore, the temperature at which the black hole radiates can be read off:

$T_{BH}=\frac{\omega}{2ImS}$ (16)

In section 3, we will study the process in Painlevé-Gullstrand coordinates.
Firstly, we need to rewrite the corrected metric in Painlevé-Gullstrand
coordinates to order $G^{2}$. We define

$t_{r}=t-a(r)$ (17)

where $a(r)$ is a function of radial coordinate $r$. Then we have

$dt^{2}=dt^{2}_{r}+a^{\prime 2}(r)dr^{2}+2a^{\prime}dt_{r}dr$ (18)

where $a^{\prime}=da(r)/dr$. The time coordinate $t_{r}$, just as for
Schwarzschild time $t$, corresponds to the time measured by a stationary
observer at infinity [9]. Inserting eq.$(3.5)$ into eq.$(2.4)$, then the
metric can be rewritten as

$ds^{2}=-f(r)dt^{2}_{r}-2f(r)a^{\prime}dt_{r}dr+\left(g(r)-f(r)a^{\prime
2}\right)dr^{2}+r^{2}d\Omega^{2}$ (19)

We require

$g(r)-f(r)a^{\prime 2}=1$ (20)

then we have $a^{\prime 2}=\frac{g(r)-1}{f(r)}$. We choose
$a^{\prime}=-\sqrt{\frac{g(r)-1}{f(r)}}$. Then

$-2f(r)a^{\prime}=2\sqrt{f(r)\left(g(r)-1\right)}$ (21)

For classical Schwarzschild metric, $f(r)=1-\frac{2GM}{r}$ and
$g(r)=\left(1-\frac{2GM}{r}\right)^{-1}$, then we have

$ds^{2}=-\left(1-\frac{2GM}{r}\right)dt^{2}_{r}+2\sqrt{\frac{2GM}{r}}dt_{r}dr+dr^{2}+r^{2}d\Omega^{2}$
(22)

For the quantum corrected Schwarzschild metric, $f(r)$ and $g(r)$ have been
given in eqs.$(2.5)$ and $(2.6)$, then we have

$f(r)\left(g(r)-1\right)=\frac{2GM}{r}+\frac{2\tilde{\beta}G^{2}M}{r(r^{2}-R^{2}_{s})}+O(G^{3})$
(23)

and

$-2f(r)a^{\prime}=2\sqrt{f(r)\left(g(r)-1\right)}=2\sqrt{\frac{2GM}{r}+\frac{2\tilde{\beta}G^{2}M}{r(r^{2}-R^{2}_{s})}}$
(24)

Therefore, to order $G^{2}$, the quantum corrected Schwarzschild metric can be
written as

$ds^{2}=-f(r)dt^{2}_{r}+2\sqrt{\frac{2GM}{r}+\frac{2\tilde{\beta}G^{2}M}{r(r^{2}-R^{2}_{s})}}dt_{r}dr+dr^{2}+r^{2}d\Omega^{2}$
(25)

In the following part of section 3, we will take $dt^{2}\equiv dt^{2}_{r}$ for
convenience, i.e.,

$ds^{2}=-f(r)dt^{2}+2\sqrt{\frac{2GM}{r}+\frac{2\tilde{\beta}G^{2}M}{r(r^{2}-R^{2}_{s})}}dtdr+dr^{2}+r^{2}d\Omega^{2}$
(26)

where $f(r)$ has been given in eq.$(2.5)$.
Furthermore, we can expand eq.$(3.11)$ in powers of $G$. In order to compare
with the Schwarzschild metric easily, then the metric $(3.13)$ is approximate
to

$ds^{2}=-f(r)dt^{2}+2\sqrt{\frac{2GM}{r}}\left(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}\right)dtdr+dr^{2}+r^{2}d\Omega^{2}$
(27)

The action for a particle moving freely in a curved background can be written
as

$S=\int p_{\mu}dx^{\mu}$ (28)

with

$p_{\mu}=mg_{\mu\nu}\frac{dx^{\nu}}{d\sigma}$ (29)

where $\sigma$ is an affine parameter along the worldline of the particle,
chosen so that $p_{\mu}$ coincides with the physical 4-momentum of the
particle. For a massive particle, this requires that $d\sigma=d\tau/m$, with
$\tau$ the proper time [9].
For simplicity, we only discuss the excitations for a massless scalar field
$\Phi$ in this article and ignore angular directions in section 3. Following
the spirit of ref.[9], we will present a calculation of the imaginary part of
the action for a massless point particle tunnelling through the horizon of
Schwarzschild spacetime in section 3.

###  3.1 Quantum correction to Hawking temperature

The radial dynamics of massless particles in Schwarzschild spacetime are
determined by the equations

$f(r)\dot{t}^{2}-2\sqrt{\frac{2GM}{r}}\left(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}\right)\dot{t}\dot{r}-\dot{r}^{2}=0$
(30)
$f(r)\dot{t}-\sqrt{\frac{2GM}{r}}\left(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}\right)\dot{r}=\omega$
(31)

The second equation is the geodesic equation corresponding to the time-
independence of the metric; in terms of the momentum defined in eq. $(3.16)$,
it can be written $p_{t}=-\omega$, and so $\omega$ has the interpretation of
the energy of the particle as measured at infinity [9].
We now consider the the trajectory of an outgoing particle. Classical outgoing
trajectory which crosses the horizon is forbidden. However, the analytic
continuation of an outgoing trajectory with $r>2M$ backwards across the
horizon will give rise to an imaginary term in our action, and this term
represents the tunnelling amplitude [9]. Eq.$(3.17)$ can be factorised to the
equation:

$\frac{dr}{dt}\approx\pm(1-\sqrt{\frac{2GM}{r}})-\sqrt{\frac{2GM}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}$
(32)

with the upper (lower) sign in eq.$(3.19)$ corresponding to outgoing (ingoing)
geodesics, under the implicit assumption that $t$ increases towards the future
[6]. As we have pointed out, in section 3, we only consider the case of upper
sign, i.e., outgoing particles. Considering eq.$(3.19)$, we then have

$ImS=Im\int
p_{r}dr=Im\int[\sqrt{\frac{2GM}{r}}\left(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}\right)\dot{t}+\dot{r}]dr=Im\int\dot{t}dr$
(33)

Combining eqs.$(3.18)$ and $(3.19)$, we have

$\dot{t}=\frac{\omega}{1-\sqrt{\frac{2GM}{r}}-\sqrt{\frac{2GM}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}+\frac{1}{4}K(r)G^{2}}$
(34)

where $K(r)=\frac{4}{G^{2}}\left(\alpha_{e}(r)+\beta_{e}(r)\right)$. To
calculate eq.$(3.20)$, we can define $\tilde{M}$ which satisfies the following
relation

$1-\sqrt{\frac{2G\tilde{M}}{r}}=1-\sqrt{\frac{2GM}{r}}-\sqrt{\frac{2GM}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}+\frac{1}{4}K(r)G^{2}$
(35)

namely,

$\sqrt{\frac{2G\tilde{M}}{r}}=\sqrt{\frac{2GM}{r}}+\sqrt{\frac{2GM}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}-\frac{1}{4}K(r)G^{2}$
(36)

Then eq.$(3.20)$ can be written as

$ImS=Im\int\dot{t}dr=\int\frac{\omega}{1-\sqrt{\frac{2G\tilde{M}}{r}}}$ (37)

The integrand has a pole at $r_{h}=2G\tilde{M}$, the horizon. Choosing the
prescription to integrate clockwise around this pole (into the upper-half
complex-$r$ plane) [9], we find

$ImS=4\pi G\tilde{M}\omega$ (38)

Then according eq.$(3.3)$, we have the temperature at which the black hole
radiates

$T_{BH}=\frac{1}{8\pi G\tilde{M}}$ (39)

From $(3.23)$, we can obtain the expression of $\tilde{M}$:

$\tilde{M}=M+\frac{\tilde{\beta}G}{r^{2}-R^{2}_{s}}M+\frac{\tilde{\beta}^{2}G^{2}}{4(r^{2}-R^{2}_{s})}M-\frac{1}{4}K(r)r\sqrt{\frac{2GM}{r}}G$
(40)

Furthermore, if the last term of $(3.27)$ can be neglected, which is the case
we mainly consider, then we have

$\tilde{M}=M+\frac{\tilde{\beta}G}{r^{2}-R^{2}_{s}}M+\frac{\tilde{\beta}^{2}G^{2}}{4(r^{2}-R^{2}_{s})}M=\left(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}\right)^{2}M$
(41)

Considering eqs.$(3.25)$ and $(3.26)$, the imaginary part of the action $ImS$
and the Hawking temperature $T_{BH}$ are given by

$ImS\approx 4\pi
GM\omega+\frac{4\pi\tilde{\beta}G^{2}M\omega}{r^{2}-R^{2}_{s}}+O(G^{3})$ (42)
$T_{BH}\approx\frac{1}{8\pi
GM}\left(1-\frac{\tilde{\beta}G}{r^{2}-R^{2}_{s}}-\frac{\tilde{\beta}^{2}G^{2}}{4(r^{2}-R^{2}_{s})}\right)$
(43)

These results can be compared with that in section 4.

###  3.2 Quantum correction to Hawking entropy

On the one hand, according to thermodynamics, we have
$dS_{BH}=\frac{dM}{T_{BH}}=8\pi G\tilde{M}dM$, where $S_{BH}$ represents the
Hawking entropy of black holes. We notice that
$M=\frac{4\pi}{3}R^{3}_{s}\rho_{0}$, where $\rho_{0}$ is the average density
of black holes [12,21]. On the one hand, if we fix the radius $R_{s}$ and
allow $\rho_{0}$ to vary, then we can obtain the quantum corrected Hawking
entropy to order $G^{2}$

$S_{BH}=4\pi GM^{2}+\frac{4\pi\tilde{\beta}G^{2}M\omega}{r^{2}-R^{2}_{s}}$
(44)

On the other hand, if we fix the radius $\rho_{0}$ and allow $R_{s}$ to vary,
then to order $G^{2}$, the quantum corrected Hawking entropy should be

$S_{BH}=4\pi
GM^{2}+\frac{64}{3}\pi^{3}\tilde{\beta}G^{2}\rho^{2}_{0}[-r^{4}\ln(\frac{r^{2}-R^{2}_{s}}{r^{2}})+\frac{1}{2}r^{4}-\frac{1}{2}(r^{2}-R^{2}_{s})^{2}-2R^{2}_{s}r^{2}]$
(45)

Moreover, we can also obtain the area of horizon at $r_{h}=2G\tilde{M}$

$A=4\pi r^{2}_{h}=16\pi^{2}G^{2}M^{2}+O(G^{3})$ (46)

Comparing eqs.$(3.28)$, $(3.29)$ and $(3.30)$, we can find that the classical
area law of black holes $S=\frac{1}{4G}A$ should be modified.
Furthermore, the heat capacity of black holes is given by

$C=T_{BH}\frac{\partial S}{\partial T_{BH}}=-8\pi
G(1+\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})})^{2}M^{2}<0$ (47)

Therefore, thermal equilibrium cannot always exist between black holes and
external environment.
We can find that the results in eqs.$(3.28)-(3.34)$ all depend on $R_{s}$.
However, in order to ensure that the quantum modifications are tiny, then the
term $\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}$ in $(3.28)$ should be very
small, namely, $r\gg R_{s}$. If $R_{s}$ can be neglected, then the results in
eqs.$(3.28)-(3.34)$ do not depend on $R_{s}$. However, it is easy to check
that the main conclusions will not change, namely, the classical area law of
black holes $S=\frac{1}{4G}A$ should be modified and the heat capacity of
black holes is always negative.

##  4 Hawking radiation as tunneling for Schwarzschild black holes

In this section, we will think of Hawking radiation as tunneling along another
method. Maulik K. Parikh and Frank Wilczek [6] have written a famous paper on
this topic. The authors found that the exact spectrum of Hawking radiation is
not precisely thermal [6]. Therefore, information can be conserved. Following
the spirit of ref.[6], We present a short and direct derivation of Hawking
radiation as a tunneling process, based on particles in the quantum corrected
metric $(2.4)$.
The radial null geodesics have been given by eq.$(3.19)$, namely,

$\frac{dr}{dt}\approx\pm(1-\sqrt{\frac{2GM}{r}})-\sqrt{\frac{2GM}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}$
(48)

However, these equations should be modified if we take into account the effect
of the particle’s self-gravitation. Self-gravitating shells in Hamiltonian
gravity were studied by Kraus and Wilczek [22]. The authors found that when
the black hole mass is held fixed and the total ADM mass allowed to vary, a
shell of energy $\omega$ moves in the geodesics of a spacetime with $M$
replaced by $M+\omega$ [6,22]. Therefore, if we fix the total mass and allow
the black hole mass to vary, then the shell of energy $\omega$ travels on the
geodesics given by the line element

$ds^{2}=-f(r,M-\omega)dt^{2}+2\sqrt{\frac{2G(M-\omega)}{r}+\beta_{e}(r,M-\omega)}dtdr+dr^{2},$
(49)

where

$f(r,M-\omega)=1-\frac{2G(M-\omega)}{r}+\alpha_{e}(r,M-\omega)$ (50)
$\alpha_{e}(r,M-\omega)=\tilde{\alpha}\frac{2G^{2}(M-\omega)}{R^{3}_{s}}[2\frac{R_{s}}{r}+\ln(\frac{r-R_{s}}{r+R_{s}})]+O(G^{3})$
(51)
$\beta_{e}(r,M-\omega)=\tilde{\beta}\frac{2G^{2}(M-\omega)}{r(r^{2}-R^{2}_{s})}+O(G^{3})$
(52)

and we have ignored the angular directions. So we should use eq.$(4.1)$ with
$M\rightarrow M-\omega$.
The imaginary part of the action for an $s$-wave outgoing positive energy
particle which crosses the horizon outwards from $r_{in}$ to $r_{out}$ can be
expressed as [6]

$ImS=Im\int^{r_{out}}_{r_{in}}p_{r}dr=Im\int^{r_{out}}_{r_{in}}\int^{p_{r}}_{0}dp^{\prime}_{r}dr$
(53)

Then we use Hamilton’s equation $\dot{r}=+\frac{dH}{dp_{r}}|_{r}$, change
variable from momentum to energy, and switch the order of integration to
obtain [6]

$\displaystyle
ImS=Im\int^{M-\omega}_{M}\int^{r_{out}}_{r_{in}}\frac{dr}{\dot{r}}dH=-Im\int^{\omega}_{0}d\omega^{\prime}\int^{r_{out}}_{r_{in}}\frac{dr}{\left(1-\sqrt{\frac{2G(M-\omega^{\prime})}{r}}\right)-\sqrt{\frac{2G(M-\omega^{\prime})}{r}}\frac{\tilde{\beta}G}{2(r^{2}-R^{2}_{s})}}$
(54)

where we have used the modified eq.$(4.1)$, and the minus sign appears from
$H=M-\omega^{\prime}$.
We can define the same $\tilde{M}$ as in section 3.1, namely,

$ImS=Im\int^{M-\omega}_{M}\int^{r_{out}}_{r_{in}}\frac{dr}{\dot{r}}dH=-Im\int^{\omega}_{0}d\omega^{\prime}\int^{r_{out}}_{r_{in}}\frac{dr}{1-\sqrt{\frac{2G(\tilde{M}-\omega^{\prime})}{r}}}$
(55)

The integral can be done by deforming the contour, so as to ensure that
positive energy solutions decay in time, i.e., into the lower half
$\omega^{\prime}$ plane [6]. Then we obtain

$ImS=4\pi GM\omega+\frac{4\pi\tilde{\beta}G^{2}M\omega}{r^{2}-R^{2}_{s}}-2\pi
G\omega^{2}+O(G^{3})$ (56)

It is obvious that if $\omega^{2}$ is very small, the result eq.$(3.29)$ would
be recovered. In other words, the result which we have obtained in section 3.1
is reasonable.
Here we should know that the derivation in section 3 and section 4 are
slightly different. In section 3, we have neglected the back-reaction of the
radiation on the black hole geometry, since we are only interested in the
nature of the spectrum to leading order [9]. However, in section 4 the
particle’s self-gravitation has been taken into account [22]. The authors
found that when the black hole mass is held fixed and the total ADM mass
allowed to vary, a shell of energy $\omega$ moves in the geodesics of a
spacetime with $M$ replaced by $M+\omega$ [6,22]. Therefore, the mass $M$ in
eqs. $(4.1)-(4.5)$ should be replaced by $M-\omega$. These are the main
differences of derivation between section 3 and section 4.
For classical Schwarzschild black holes, Maulik K. Parikh and Frank Wilczek
[6] have found that

$ImS=4\pi GM\omega(1-\frac{\omega}{2M})$ (57)

Only when the energy of emitted particles $\omega$ is very low, namely,
$\omega^{2}$ can be neglected, pure blackbody spectrum can be recovered. If
the energy of emitted particles $\omega$ is very high, blackbody spectrum
would be deviated. This result satisfies unitarity and supports information
conservation.
However, when

$\omega=\frac{2\tilde{\beta}GM}{r^{2}-R^{2}_{s}},$ (58)

eq.$(4.9)$ will become pure blackbody spectrum. In particular, for $r\gg
R_{s}$, $\omega=2\tilde{\beta}GM/r^{2}$. Therefore, for the quantum corrected
metric, it is possible to break down information conservation. Therefore, we
suggest that, in order to ensure conservation of information at any energy
scales, black hole needs to retain some “remnant”, namely, a black hole cannot
release all information it has as Hawking suggested [26].
Of course, all the above discussions are only true for quasi-static reversible
processes. However, since the heat capacity of black holes is negative (i.e.,
eq.$(3.31)$), thermal equilibruim cannot always exist between black holes and
external environment. For irreversible processes, further studies are needed.
Careful readers can find that since $ImS$ can be obtained by integrating along
the trajectory, such as eqs.$(3.20)$ and $(4.7)$, the results should not
depend on the generic radial coordinate $r$. However, the results in
eqs.$(3.28)-(3.30)$ and $(4.9)$ all depend on radial coordinate $r$. So, what
are the differences between these two arguements? The explanation consists of
three parts.
Firstly, the integrations in eqs.$(3.20)$ and $(4.7)$ are hard to obtain the
exact expressions (although can be obtained by Mathematica). So, in this
article, we do not hope to obtain the exact formulae. However, in section 3
and 4 we have used slightly different methods respectively to obtain the same
results for $ImS$ when $\omega^{2}$ can be neglected. Therefore, we can
guarantee that the methods and results in section 3 and 4 are reasonable and
consistent.
Secondly, even if we cannot obtain the exact formulae, the main results in
section 3 and 4 are remained when quantum correction has been considered. The
first one is that the famous area law of black holes $S=\frac{1}{4G}A$ should
be corrected. The second one is that the heat capacity of black holes is
negative, which means that thermal equilibrium cannot always exist between
black holes and external environment. The third one is that at certain
frequency $\omega$, the radiation spectrum will become pure blackbody spectrum
and in order to ensure information conservation a black hole needs to retain
some “remnant”.
Thirdly, the results in eqs.$(3.28)-(3.30)$ and $(4.9)$ can be easily compared
with the results for Schwarzschild solution.

##  5 Bound on greybody factors for Schwarzschild black holes

In section 3 and 4, we have found that Hawking radiation spectrum of black
holes deviates from blackbody spectrum in most cases for the quantum corrected
metric. Therefore, it is natural to discuss greybody factors
$\sigma_{l}(\omega)$ for Schwarzschild black holes. The total radiation power
is given by

$P(\omega)=\sum_{l}\int^{\infty}_{0}P_{l}(\omega)d\omega,$ (59)

where

$P_{l}(\omega)=\frac{A}{8\pi^{2}}\sigma_{l}(\omega)\frac{\omega^{3}}{\exp(\omega/T_{BH})-1}$
(60)

is the power emitted per unit frequency in the $l^{th}$ mode, $A$ is a a
multiple of the horizon area, $l$ is the angular momentum quantum number and
$\sigma_{l}(\omega)$ is the frequency dependent greybody factor [10]. One
finds that if the black hole is to be in thermal equilibrium with a bath of
particles surrounding it, the rate at which it emits these particles must
equal the rate at which particles in the bath fall into the black hole [9].
There are various methods in literature [27–31] to estimate the greybody
factors, however, derivation of exact analytical expression of greybody factor
is limited only to very few cases [10]. Following ref.[32], we will briefly
consider the bounds on the greybody factor for the quantum corrected black
hole metric. The general bounds on the greybody factor, as proposed by Petarpa
Boonserm and Matt Visser [32] is given by

$\sigma_{l}(\omega)\geq sech^{2}\left(\int^{\infty}_{-\infty}\Theta
dr_{\ast}\right)$ (61)

where

$\Theta=\frac{\sqrt{[h^{\prime}(r)^{2}]+[\omega^{2}-V_{eff}-h(r)^{2}]^{2}}}{2h(r)}$
(62)

The arbitrary function $h(r)$ has to be positive definite everywhere and
satisfy the boundary condition, $h(\infty)=h(-\infty)=\omega$ for the bound
eq.$(5.2)$ to hold [3] and $V_{eff}$ is the effective potential which will be
given later.
The equation of motion for a free massless scalar field $\Phi$ is given by

$\Box\Phi=0$ (63)

Since metric $(2.12)$ has spherical symmetry, the angular variables can be
seperated from the other coordinates, i.e.[21],

$\Phi(t,r,\theta,\phi)=\Phi(t,r)S(\theta,\phi)$ (64)

where $S(\theta,\phi)$ can be decomposed in the usual spherical harmonics
satisfying [21]

$\left(\partial^{2}_{\theta}+\frac{\cos\theta}{\sin\theta}\partial_{\theta}+\frac{1}{\sin^{2}\theta}\partial^{2}_{\phi}\right)Y(\theta,\phi)=-l(l+1)Y(\theta,\phi)$
(65)

Moreover, time $t$ can be seperated from the radial coordinate, i.e.,

$\Phi(t,r)=\Psi(t)\Phi(r)$ (66)

where $\Psi(t)\sim e^{i\omega t}$ and satisfies
$\ddot{\Psi}(t)=-\omega^{2}\Psi(t)$ [21].
Furthermore, cosidering the metric $(2.12)$, the radial equation for a
massless scalar field $\Phi$ with the tortoise like coordinate $r_{\ast}$,
which has been given by eq.$(2.11)$, is [21]

$[\partial^{2}_{r_{\ast}}+\omega^{2}]u(r)=V_{eff}u(r)$ (67)

where

$V_{eff}=f(r)[\frac{l(l+1)}{r^{2}}+\frac{2GM}{r^{3}}]$ (68)

and $l$ is the angular momentum of the specific wave mode under consideration
[10].
A particularly simple choice of $h(r)$ for the present case is [3,10,32]

$h(r)=\omega$ (69)

Combining eqs.$(2.11)$, $(2.13)$, $(5.3)$ and $(5.4)$, if we only consider the
case of lowest order, then

$\sigma_{l}(\omega)\geq
sech^{2}\left(\frac{1}{2\omega}\int^{\infty}_{r_{+}}[\frac{l(l+1)}{r^{2}}+\frac{2GM}{r^{3}}]dr\right)$
(70)

where $r_{+}$ is the modified horizon at $r=2G\tilde{M}$. Therefore,

$\sigma_{l}(\omega)\geq
sech^{2}\left(\frac{l(l+1)}{4G\tilde{M}\omega}+\frac{M}{8G\tilde{M}^{2}\omega}\right)$
(71)

For classical Schwarzschild black holes, the horizon is at $r=2GM$, then the
bound of greybody factors is [32]

$\sigma_{l}(\omega)\geq
sech^{2}\left(\frac{1}{2\omega}\int^{\infty}_{2GM}[\frac{l(l+1)}{r^{2}}+\frac{2GM}{r^{3}}]dr\right)$
(72)

namely,

$\sigma_{l}(\omega)\geq
sech^{2}\left(\frac{l(l+1)}{4GM\omega}+\frac{1}{8GM\omega}\right)$ (73)

Considering eq.$(3.24)$ and comparing eqs.$(5.13)$ and $(5.15)$, it is easy to
find out the bound on greybody factors has a very small difference between
classical metric and quantum corrected metric. It can be expected that the
modification of total radiation power will be very small as well.

##  6 Conclusions and Discussions

Since 1974, Hawking radiation [1,2] has become a fascinating research field
since it relates classical general relativity and quantum field theory and is
a deep insight into quantum gravity. Many researchers [4-8] devoted to
calculating the Hawking temperature and radiation power for exploring a
possible quantum gravity theory [3]. Furthermore, these methods have been
generalized to other cases, such as charged black holes [3,10], Einstein-
Gauss-Bonnet de Sitter black hole [11], etc. However, the quantum effect on
metrics have been neglected in all mentioned studies.
Many researches have been conducted in quantum gravity using effective field
theory (EFT) [13-20]. Though finding a consistent quantum gravity theory is
still an extremely difficult challenge at the moment, one can still apply EFT
methods to do calculations in quantum gravity which are model independent at
energies below the Planck mass [12,21]. Xavier Calmet, Roberto Casadio and
Folkert Kuipers [12] have used EFT methods [12,21] to obtain the metric inside
and outside a star, taking into account quantum effects. Moreover, in ref.[12]
the authors also proposed that the exterior metric may provide a good model of
quantum black holes. Therefore, one direct research step to consider the
effect of quantum correction on Hawking radiation. For simplicity, we only
discuss the excitations for a massless scalar field $\Phi$ in this article.
In section 3, we have studied quantum correction of Hawking temperature and
entropy for Schwarzschild black holes. Firstly, we studied the radial dynamic
equations of massless particles in quantum corrected Schwarzschild spacetime.
Then, following the spirit of ref.[9], we obtained the modified Hawking
temperature. Moreover, by calculating the modifed Hawking entropy and the area
of horizon, we found that the famous area law of black holes $S=\frac{1}{4G}A$
need to be corrected. Some researches have been conducted to study the
possibility to generalize area law of black holes to more general forms
[33,34]. For example, if we consider non-extensive statistical mechanics, the
area law should be written as $S=\gamma A^{\beta}$, where $A$ is the black
hole horizon area, $\gamma$ is an unknown constant and $\beta$ known as non-
extensive parameter which is a constant [34]. However, for the case we have
studied in this article, $\gamma=\frac{1}{4G}$ is a constant while the
parameter $\beta$ is dependent on $A$ or $M$. More future studies should focus
on this difference and relationship. Furthermore, we have determined the heat
capacity of Schwarzschild black holes as well. Even if the quantum corrections
have been considered, the heat capacity of black holes is negative, which
means that thermal equilibruim cannot always exist between black holes and
external environment.
In section 4, we considered Hawking radiation as tunneling following the
methods in ref.[6]. To leading order, we obtained the same results in section
3.1, therefore, the results are reasonable. For most cases, when the energy of
emitted particles $\omega$ is very low, pure blackbody spectrum can be
recovered. While if the energy of emitted particles $\omega$ is very high,
blackbody spectrum would be deviated. This result is as the same as the result
of ref.[6] for classical Schwarzschild black holes. However, there exits a
special case for the quantum corrected metric. When the frequency of emitted
particles takes certain value (in this article
$\omega=\frac{2\tilde{\beta}GM}{r^{2}-R^{2}_{s}}$), the radiation spectrum
will become pure blackbody spectrum. Therefore, it has the possibility to
break down information conservation. In order to ensure the conservation of
information at any energy scales, we suggested that a black hole needs to keep
some information it has. We should point out that all the results in section 4
are only true for quasi-static reversible processes. Since the heat capacity
of black holes is negative, thermal equilibrium cannot always exist between
black holes and external environment. Further studies for irreversible
processes are needed.
In section 5, we briefly discussed the bound on greybody factors for
Schwarzschild black holes to lowest order and found that the bound has a very
small difference between classical metric and quantum corrected metric.
Future work can be directed along at least three lines of further research.
Firstly, the quantum corrected metric for other classical solutions of black
holes should be obtained using EFT methods. Secondly, we can generalize the
methods and results in this article to other quantum corrected black hole
metrics. Thirdly, we can apply the methods in refs.[6,9] to study the
properties of Hawking radiation in stringy gravity and test if information is
conserved or not. There is therefore great potential for development of this
work in the future.

## References

  * (1) S.W. Hawking, Black hole explosions?. _Nature_ 248 (1974) 30.
  * (2) S.W. Hawking, Particle creation by black holes. _Commun. Math. Phys._ 43 (1975) 199.
  * (3) Yan-Gang Miao, Zhen-Ming Xu, Hawking Radiation of Five-dimensional Charged Black Holes with Scalar Fields. _Phys. Lett. B_ 772 (2017) 542-546.
  * (4) D.N.Page, Particle emission rates from a black hole: Massless particles from an uncharged, nonrotating hole. _Phys. Rev. D_ 13 (1975) 198.
  * (5) D.N.Page, Particle emission rates from a black hole. II. Massless particles from a rotating hole. _Phys. Rev. D_ 14 (1976) 3260.
  * (6) M.K. Parikh, F. Wilczek, Hawking Radiation As Tunneling. _Phys. Rev. Lett._ 85 (2000) 5042.
  * (7) R. Emparan, G.T. Horowitz, R.C. Myers, Black Holes Radiate Mainly on the Brane. _Phys. Rev. Lett._ 85 (2000) 499.
  * (8) S. Hemming, E. Keski-Vakkuri, Hawking radiation from AdS black holes. _Phys. Rev. D_ 64 (2001) 044006.
  * (9) George Johnson and John March-Russell, Hawking radiation of extended objects._JHEP_ 04 (2020) 205.
  * (10) Avijit Chowdhury, Narayan Banerjee, Greybody factor and sparsity of Hawking radiation from a charged spherical black hole with scalar hair. _Phys. Lett. B_ 805 (2020) 135417.
  * (11) Cheng-Yong Zhang, Peng-Cheng Li and Bin Chen, Greybody factors for a spherically symmetric Einstein-Gauss-Bonnet–de Sitter black hole. _Phys. Rev. D_ 97 (2018) 044013.
  * (12) Xavier Calmet, Roberto Casadio, and Folkert Kuipers, Quantum Gravitational Corrections to a Star Metric and the Black Hole Limit. _Phys. Rev. D_ 100 (2019) 086010.
  * (13) S. Weinberg, in General Relativity. An Einstein Centenary Survey, edited by S.W. Hawking and W. Israel (Cambridge University Press, Cambridge, England, 1979), p. 790.
  * (14) A. O. Barvinsky and G. A. Vilkovisky, The generalized Schwinger-DeWitt technique and the unique effective action in quantum gravity. _Phys. Lett._ 131B (1983) 313.
  * (15) A. O. Barvinsky and G. A. Vilkovisky, The generalized Schwinger-Dewitt technique in gauge theories and quantum gravity. _Phys. Rep._ 119 (1985) 1.
  * (16) A. O. Barvinsky and G. A. Vilkovisky, Beyond the Schwinger-DeWitt technique: Converting loops into trees and in-in currents. _Nucl. Phys. B_ 282 (1987) 163.
  * (17) A. O. Barvinsky and G. A. Vilkovisky, Covariant perturbation theory (II). Second order in the curvature. General algorithms. _Nucl. Phys. B_ 333 (1990) 471.
  * (18) I. L. Buchbinder, S. D. Odintsov, and I. L. Shapiro, Effective Action in Quantum Gravity (CRC Press, Bristol, 1992).
  * (19) J. F. Donoghue, General relativity as an effective field theory: The leading quantum corrections. _Phys. Rev. D_ 50 (1994)3874.
  * (20) X. Calmet, Vanishing of quantum gravitational corrections to vacuum solutions of general relativity at second order in curvature. _Phys. Lett. B_ 787 (2018)36-38.
  * (21) Xavier Calmet, Roberto Casadio and Folkert Kuipers, Quantum corrected equations of motion in the interior and exterior Schwarzschild spacetimes. _Phys. Rev. D_ 102 (2020)026018.
  * (22) P. Kraus and F. Wilczek, Self-Interaction Correction to Black Hole Radiance. _Nucl. Phys. B_ 433 (1995)403.
  * (23) M.K. Parikh, A Secret Tunnel Through The Horizon. _Int. J. Mod. Phys. D_ 13 (2004)2351.
  * (24) M.K. Parikh, Energy conservation and Hawking radiation, in the proceedings of the 10th Marcel Grossmann Meeting (MG10, July 2026, Rio de Janeiro, Brazil (2003).
  * (25) L. Vanzo, G. Acquaviva and R. Di Criscienzo, Tunnelling Methods and Hawking’s radiation: achievements and prospects. _Class. Quant. Grav._ 28 (2011)183001.
  * (26) Zheng Zhao, Wenbiao Liu, Basics of General Relativity, Tsinghua University Press, (2010).
  * (27) V. Cardoso, M. Cavaglià, L. Gualtieri, Black hole particle emission in higher-dimensional spacetimes. _Phys. Rev. Lett._ 96 (2006)071301.
  * (28) V. Cardoso, M. Cavaglià, L. Gualtieri, Hawking emission of gravitons in higher dimensions: non-rotating black holes. _JHEP_ 2006 (2006)021.
  * (29) T. Harmark, J. Natário, R. Schiappa, Greybody Factors for d–Dimensional Black Holes. _Adv. Theor. Math. Phys._ 14 (2010)727-794.
  * (30) J. Grain, A. Barrau, P. Kanti, Exact Results for Evaporating Black Holes in Curvature-Squared Lovelock Gravity : Gauss-Bonnet Greybody Factors. _Phys. Rev. D_ 72 (2005)104016.
  * (31) S. Fernando, Greybody Factors of Charged Dilaton Black Holes in 2 + 1 Dimensions. _Gen. Relativ. Gravit._ 37 (2005)461.
  * (32) Petarpa Boonserm and Matt Visser, Bounding the greybody factors for Schwarzschild black holes. _Phys. Rev. D_ 78 (2008)101502.
  * (33) Tamás S. Biró, Viktor G. Czinner, A q-parameter bound for particle spectra based on black hole thermodynamics with Rényi-entropy. arXiv:1309.4261v1.
  * (34) Ahmad Sheykhi, Modified Friedmann equations from Tsallis entropy. _Phys. Lett. B_ 785 (2018)118-126.


# Neural Named Entity Recognition from Subword Units

###### Abstract

Named entity recognition (NER) is a vital task in spoken language
understanding, which aims to identify mentions of named entities in text e.g.,
from transcribed speech. Existing neural models for NER rely mostly on
dedicated word-level representations, which suffer from two main shortcomings.
First, the vocabulary size is large, yielding large memory requirements and
training time. Second, these models are not able to learn morphological or
phonological representations. To remedy the above shortcomings, we adopt a
neural solution based on bidirectional LSTMs and conditional random fields,
where we rely on subword units, namely _characters_ , _phonemes_ , and
_bytes_. For each word in an utterance, our model learns a representation from
each of the subword units. We conducted experiments in a real-world large-
scale setting for the use case of a voice-controlled device covering four
languages with up to $5.5$M utterances per language. Our experiments show that
(1) with increasing training data, performance of models trained solely on
subword units becomes closer to that of models with dedicated word-level
embeddings ($91.35$ vs $93.92$ F1 for English), while using a much smaller
vocabulary size ($332$ vs $74$K), (2) subword units enhance models with
dedicated word-level embeddings, and (3) combining different subword units
improves performance.

11footnotetext: Work done while at Amazon

##  1 Introduction

Named Entity Recognition (NER) is an important task in spoken language
technology applications, such as voice-controlled smart assistants like the
Amazon Echo or Google Home. For example, if a user requests an assistant to
“play we are the champions by queen”, an automatic speech recognizer (ASR) can
be applied to transcribe the utterance, and subsequently a named entity
recogniser can be applied to the ASR output to determine that ‘we are the
champions’ refers to a song and ‘queen’ to an artist. As new utterances are
collected from the device’s users over time, which are annotated with named
entities, regular retraining of NER models with increasing data amounts is
needed.

Recently, several neural models for named entity recognition have been
proposed (e.g., [1, 2]), indicating promising performance on rather small and
artificially generated datasets [3, 4]. However, these models either rely on
word-level representations or combine them with character-level
representations. Such models suffer from the following shortcomings:

  * •

The vocabulary size is large, yielding a large number of parameters, and
hence, large memory requirements and training time. This is problematic if
large amounts of data are available, in particular if there are memory
constraints for the application.

  * •

They ignore the combination with other subword units e.g., phonemes or bytes,
which can potentially improve performance by contributing to the modelling of
morphology and phonology in case of phonemes. The latter one appears to be
particularly useful if a named entity recognizer is applied to transcribed
speech.

  * •

Out-of-vocabulary (OOV) words can be problematic.

In this paper, we adopt a neural solution relying solely on subword units,
namely _characters_ , _phonemes_ and _bytes_. For each word in an utterance,
we learn representations from each of the three subword units. The character-
level unit looks at the characters of each word, while the phoneme-level unit
treats a word as a sequence of phonemes, using ASR lexica that map a given
word into its corresponding phoneme sequence. The byte-level unit reads a word
as bytes, where we use the variable length UTF-8 encoding.

A major advantage of subword-based models is the small vocabulary size which
can positively affect memory requirements and training time of models. This is
particularly relevant for large-scale applications and for systems that
operate under certain constraints like memory constraints; e.g., handheld or
voice-controlled devices. In addition, subword-units could improve modelling
of out-of-vocabulary words and support learning of morphological and
phonological information. Specifically, character-level networks have already
proven to boost the performance of many sequence tagging tasks, including
part-of-speech tagging and NER, in particular for morphologically rich
languages [1, 5, 2]. However, while characters have been successfully used to
boost NER performance, combining different types of subword units have not yet
been explored, leaving open the question if there are additive gains, which we
address in this paper.

We present experiments in a large-scale setting for the use case of a voice-
controlled device covering four languages, including English, German, and
French, with up to 5.5M utterances per language. Our experiments show that:

  * •

With increasing training data size, performance of models trained solely on
subword units becomes closer to that of models with dedicated word-level
embeddings ($91.35$ vs $93.92$ F1 for English), however, with much smaller
vocabulary size (e.g. $332$ vs $74$K for English).

  * •

Subword units enhance models with dedicated word-level embeddings, in
particular, for languages with smaller training data sets.

  * •

Combining the three subword units (byte-, phoneme- and character-level) yields
better results than using only one or two of them.

The remainder of this paper is organised as follows. Next, we present our
neural network model using subword units. Subsequently, we present
experimental results. Before concluding, we discuss related work.

##  2 Model

We follow recent work on neural named entity recognition and base our solution
on bidirectional LSTMs and conditional random fields (CRF) [1, 6, 2, 7]. For
each word in an utterance, our model learns a low-dimensional representation
from each subword unit (character-, phoneme- and byte-level), which are then
concatenated and fed into a bidirectional LSTM-CRF model [6, 2]. Our model is
depicted in Figure 1. Bidirectional LSTMs capture long-range dependencies
among input tokens [8]. In this work we use the LSTM implementation that was
adopted by Lample et al. [2]:

$\begin{split}i_{t}=&\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+b_{i})\\\
c_{t}=&(1-i_{t})\odot c_{t-1}+\\\ &i_{t}\odot
tanh(W_{xc}x_{t}+W_{hc}h_{t-1}+b_{c})\\\
o_{t}=&\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+W_{co}c_{t}+b_{o})\\\
h_{t}=&o_{t}\odot tanh(c_{t}),\\\ \end{split}$

where W’s are shared weight matrices, b’s are the biases, $\sigma$ is the
element-wise sigmoid function, $x_{t}$ represents the token at position $t$,
$h_{t}$ is hidden state at time $t$, and $\odot$ is the element-wise product.
For sequence tagging problems, a softmax layer is used on top of the output of
the bidirectional LSTM network to calculate a probability distribution of
output tags for a given token. However, the model assumes independence among
output tags, which is not practical for named entity recognition. As a remedy,
a CRF layer is incorporated for decoding. For details, see [2].

Figure 1: Our model with a word-level bidirectional LSTM layer and CRF layer
for decoding. For each word in an utterance, our model learns embeddings from
the three subword units. Dedicated word embeddings are optional. Figure 2: The
outputs of the three subword units are concatenated to learn embeddings for
the whole word (‘dark’). Optionally, we can concatenate dedicated word
embeddings. Learned embeddings are then fed into a word-level BiLSTM.

Subword units.  We rely on subword units to learn embeddings that represent
the full word. As shown in Figure 2, each subword unit is a bidirectional LSTM
network, where the last hidden state of the forward and the backward networks
are concatenated, which constructs $V_{c},V_{ph}$ and $V_{by}$ from the
character-, phoneme- and byte-level networks, respectively. The vectors
$V_{c},V_{ph}$ and $V_{by}$ are, in turn, concatenated to represent the final
embeddings of a word. Optionally, we can concatenate dedicated word embeddings
that are either randomly initialized or pre-trained. Subword units enable the
model to mitigate the out-of-vocabulary problem, contribute to modelling of
morphology and phonology and to have smaller vocabulary size compared to
models that rely on word-level representations.

For the phoneme-level unit, we use lexica that map a given word into its
corresponding phoneme sequence. Phonemes are represented using the X-SAMPA
phoneme set with some additional symbols. For example, the word ‘dark’ is
mapped to $\\{d\\_B,A,r\setminus,k\\_E\\}$, where $\\_B$ marks the first
phoneme in a word, while $\\_E$ marks the last one. In addition, we add a
special symbol $UNK$ to which we map words which are not comprised in our
lexica. Including the additional symbols, the final phoneme vocabulary
contains $265$ entries with $60$ unique phonemes. In general, for the setting
explored in our paper, i.e. voice-controlled devices, phoneme lexica with good
coverage are developed for the agent’ s text to speech (TTS) and automated
speech recognition components which can be re-used. In case lexica with good
coverage are not available, tools for grapheme-to-phoneme conversion can be
used.

For the byte-level unit, we use the variable length UTF-$8$ encodings to keep
the vocabulary small. For example, ‘Schön’ is represented as {0x53 0x63 0x68
0xc3 0xb6 0x6e}. Note that the character ö corresponds to two bytes, {0xc3
0xb6}. This distinguishes this unit from the character-level one.

##  3 Experiments

In the following, we first describe our experimental setup and subsequently
present our results. In our experiments, we explored 1) performance of
individual subword units and different combinations of subword units, both
with and without using word-level embeddings, 2) performance of subword only
models versus models with dedicated word level embeddings versus models
combining both, and 3) whether subword units help in the presence of out-of-
vocabulary words.

Table 1: Number of utterances of per language. | EN | DE | FR | ES
---|---|---|---|---
Size of train set |  $3.3$M |  $0.6$M |  $12$K |  $8$K
Size of dev set |  $1.1$M |  $0.2$M |  $4$K |  $2.6$K
Size of test set |  $1.1$M |  $0.2$M |  $4$K |  $2.6$K

###  3.1 Experimental Setup

Datasets. We use a large real-world dataset covering four languages, namely
American English (EN), German (DE), French (FR) and Spanish (ES). The data is
representative of user requests to voice-controlled devices, which were
manually transcribed and annotated with named entities. Overall, the data
covers several domains, comprising different intents and types of named
entities. On average, we have $36$ types of named entities per language. Table
1 shows data statistics for each language.

Metric. To evaluate our models, we use the CoNLL script [3] to compute
precision, recall and F1 scores on a per-token basis. We report the average F1
score.

Table 2: Vocabulary size of subword-based versus word-based models. Lang | Subwords | Word-level
---|---|---
EN | $332$ |  $74$K
DE | $225$ |  $46$K
FR | $148$ |  $18$K
ES | $120$ |  $3.7$K

Training.  We used a mini-batch Adam optimizer [9] with a learning rate of
$0.0007$ for all the models presented in this paper. We tried different
optimizers with different learning rates (e.g., stochastic gradient descent),
however, they performed worse than Adam. The batch size was set to 1024, 256,
4 and 4 utterances for EN, DE, FR and ES, respectively. The embedding
dimension of the subword units is set to 35, while its counterpart of the
word-level network is set to 64 (in case dedicated word-level representations
are used). Both subword and word-level networks have a single layer for the
forward and the backward LSTMs whose dimensions are set to 35 and 128,
respectively. We tried several different values, however, the performance was
inferior to the one reported with the above values. When a given number of
epochs is reached (40 epochs), training is terminated. The model with the best
F1 score on the development set is used to make predictions 111We extended the
model at https://github.com/glample/tagger.

We used dropout training [10], applying a dropout mask to the final embedding
layer just before the input to the word-level bidirectional LSTM, with dropout
rate set to $0.5$.

Table 2 shows the vocabulary sizes of different languages for both subword-
based and word-level models, highlighting the large differences between
subword-based models and models with dedicated word embeddings. In addition,
in terms of model complexity, subword-based models have a much smaller number
of parameters e.g., for EN, $74$K $*$ $64=4.7$M fewer parameters to fine-tune
during training.

Table 3: F1 scores of the subword-only models with different units being used.
The model with the three subword units combined achieved best performance
across languages, except for FR.

| Char | Phoneme | Byte | Char + Phoneme | Char + Byte | Phoneme + Byte | All
---|---|---|---|---|---|---|---
EN | $89.63$ | $90.3$ | $89.7$ | $91.15$ | $90.58$ | $91.1$ | $\bm{91.35}$
DE | $84.94$ | $84.21$ | $84.95$ | $86.81$ | $86.32$ | $86.76$ | $\bm{87.37}$
FR | $80.57$ | $73.65$ | $80.4$ | $80.1$ | $\bm{82.44}$ | $82.15$ | $81.05$
ES | $67.64$ | $62.4$ | $67.07$ | $69.6$ | $70.33$ | $68.94$ | $\bm{71.07}$

Table 4: F1 scores of the combined models with different units being used.

| Char | Phoneme | Byte | Char + Phoneme | Char + Byte | Phoneme + Byte | All
---|---|---|---|---|---|---|---
EN | $93.96$ | $93.99$ | $93.99$ | $\bm{94.02}$ | $93.89$ | $93.97$ | $93.88$
DE | $90.17$ | $90.02$ | $90.17$ | $\bm{90.25}$ | $90$ | $90.19$ | $90.1$
FR | $86.37$ | $86.38$ | $86.49$ | $\bm{87.45}$ | $85.98$ | $85.86$ | $86.38$
ES | $79$ | $80.03$ | $79.08$ | $79.57$ | $79.1$ | $\bm{80.23}$ | $78.72$

###  3.2 Results

Subwords only models.  Table 3 shows the performance of models that rely
solely on subword units. When used individually, different subword units yield
the best results for different languages. For example, for English, the best
individual subword unit is phoneme (with $0.67$ points in F1 more than
character), while character-level unit achieved best results for French. Here
it must be noted that the phoneme lexicon for French had much lower coverage
than the one for English, which explains the low F1 score.

When several subword units are combined, results improve for all languages,
and except for French, the best results are achieved when using all of the
subword units combined. For French, the best combination is characters and
bytes, i.e. without using phonemes, which we again attribute to the low
coverage of the French phoneme lexicon. To explore further whether these
improvements are indeed due to using several subword units rather than the
increased dimensionality of the hidden embedding representation, we trained
models for the different languages using a single subword unit, however, with
higher embedding and LSTM hidden dimensions. The performance was inferior to
that reported in Table 3 (last column), indicating that there are indeed
additive gains from combining different subword units.

Combined models.  Table 4 shows results for combining word-level embeddings
and subword units. As can be seen, in this setting there are again additive
gains by using several subword units compared to using only one. Depending on
the language, phonemes yield the best results in combination with either
characters or bytes, indicating that phonemes are useful for named entity
recognition, which, to the best of out knowledge, is first explored in this
work. The reason might be that they contribute to modelling phonology and/or
morphology, thus improving performance in particular for out-of-vocabulary
words. For three out of the four languages, the combination of characters,
phonemes and word-level embeddings achieved the best results.

Table 5: Comparison of subword only models versus word-level models and models combining word-level and subword units. For combined and subword models, the best combination is given. Numbers correspond to F1 values.  Lang | Subwords | Word-level | Combined
---|---|---|---
EN | $91.35$ |  $93.92$($+2.57$) |  $94.02$($+0.1$)
DE | $87.37$ |  $90.12$($+2.75$) |  $90.25$($+0.13$)
FR | $82.44$ |  $86.87$($+4.43$) |  $87.45$($+0.58$)
ES | $71.07$ |  $79.43$($+8.36$) |  $80.23$($+0.8$)

Comparison.  Table 5 compares the performance of models using only subword
units, models using only word-level representations and models that combine
both. We observe that with increasing training data size, performance of
models trained solely on subword units becomes closer to that of models with
dedicated word-level embeddings ($91.35$ vs $93.92$ F1 for EN), however, with
smaller vocabulary size ($332$ vs $74$K). The gap in performance increases as
the size of train data decreases ($71.07$ vs $79.43$ F1 for ES). That is, with
sufficient training data, subword-based models achieve rather similar results
to word-level ones. Models that use both word-level embeddings and subword
units achieve the best results (Table 5, last column), showing that subword
units can enhance word-level models. As train data decreases, the positive
effect of subword units increases ($+0.1$ F1 point for EN and $+0.8$ F1 point
for ES). Notably, besides training size, the languages vary in our
experiments. In order to draw concrete conclusions on performance in relation
to the size of training data, we also trained different models (subword-only,
word-level and combined) using different splits of DE train data ($20\%$,
$50\%$ and $70\%$). The results of this experiment confirm our observation
that with increasing train data, performance of subword-based models
approaches that of word-level models. Notably, while performance of subwords-
only models for EN is still below that of models making use of dedicated word
embeddings, the difference may be small enough to opt for the subwords-only
model in case there are system requirements e.g., on memory.

Out-of-vocabulary words.  To explore whether our subword units contribute to
improved modelling of out-of-vocabulary words, we ran an experiment with the
ES data. Out of the ES test set $625$ utterances contain at least one out-of-
vocabulary word, with $703$ words in total. F1 values on these utterances are
$44.6$, $50.1$ and $51$ for subwords only, word-level and combined models,
respectively, and are thus following the trends observed in Table 5. We also
computed F1 scores on the out-of-vocabulary words, where, interestingly, the
subword-based model outperformed the corresponding word-level model ($34.9$ vs
$34.8$), while combined model achieved $37.1$ F1, indicating that subword
units are useful in the presence of out-of-vocabulary words.

##  4 Related Work

Named entity recognition is a widely studied problem, where methods have been
characterized by the use of CRFs with heavy feature engineering, gazetteers
and external knowledge resources [11, 12, 13, 5, 14, 15, 16, 17, 18]. Ratinov
and Roth [16] use non-local features and gazetteers extracted from Wikipedia,
while Kazama and Torisawa [13] harness type information of candidate entities.
In our work, we opt for a neural solution without hand-crafted features or
external resources.

Recently, the focus has shifted towards adopting neural architectures for NER
[1, 19, 20, 6, 2, 7, 21, 22]. Huang et al. [6] use a word-level bidirectional
LSTM-CRF for several sequence tagging problems including POS tagging and named
entity recognition. They made use of heavy feature engineering to extract
character-level features. Lample et al. [2] extend the previous model by using
a character-level BiLSTM-based unit, where a word is represented by
concatenating word-level embeddings and embeddings learned from its
characters. Chiu and Nichols [1] use a convolutional neural network to learn
character-level embeddings and LSTM units on the word level. Santos and
Guimaraẽs [21] propose the CharWNN network, a similar model to that of Chiu
and Nichols [1]. Gillick et al. [20] employ a sequence-to-sequence model with
a novel tagging scheme. The model relies on bytes, allowing the joint training
on different languages for NER, and eliminating the need for tokenization.
Bharadwaj et al. [23] represent words as sequences of phonemes, which serve as
universal representation across languages to facilitate cross-lingual transfer
learning. Finally, Yang et al. [22] adopt a similar model to that of Lample et
al. [2], however, they replaced LSTMs with Gated Recurrent Units (GRUs).
Furthermore, they studied the multi-lingual and multi-task joint training,
which we plan to address in the future.
Overall, existing neural methods for named entity recognition rely mostly on
dedicated word embeddings rather than learning such representations from
subword units. While some work has also addressed characters or bytes,
combining different types of subword units has not been explored, which we
address in this work. For a comprehensive survey on NER, see [24].

##  5 Conclusion and Future Work

We presented a neural model for named entity recognition using three subword
units: characters, phonemes and bytes. For each word in an utterance, the
model learns a representation from each of the three subword units, which are
then concatenated and fed into a word-level bidirectional LSTM and CRF layer
for decoding. Our experiments show that i) with increasing training data,
performance of models trained solely on subword units becomes closer to that
of models with dedicated word embeddings, while using a much smaller
vocabulary and fewer trainable model parameters, ii) subword units enhance
models with dedicated word embeddings, and iii) combining subword units
improves performance.
In this paper, we used ASR lexica to map words into sequences of phonemes. An
interesting direction for future work is the application together with an
automatic speech recogniser, where phonemes are transcribed from the speech
utterances and subsequently used in the NER model. This could improve NER
performance further by taking additionally information directly from the
speech signal into account, which could be in particular useful for modelling
homonyms.

## References

  * [1] J. P. C. Chiu and E. Nichols, “Named entity recognition with bidirectional lstm-cnns,” _TACL_ , vol. 4, pp. 357–370, 2016. [Online]. Available: https://transacl.org/ojs/index.php/tacl/article/view/792
  * [2] G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, “Neural architectures for named entity recognition,” in _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_ , 2016, pp. 260–270. [Online]. Available: http://aclweb.org/anthology/N/N16/N16-1030.pdf
  * [3] E. F. T. K. Sang, “Introduction to the conll-2002 shared task: Language-independent named entity recognition,” in _Proceedings of the 6th Conference on Natural Language Learning, CoNLL 2002, Held in cooperation with COLING 2002, Taipei, Taiwan, 2002_ , 2002. [Online]. Available: http://aclweb.org/anthology/W/W02/W02-2024.pdf
  * [4] E. F. T. K. Sang and F. D. Meulder, “Introduction to the conll-2003 shared task: Language-independent named entity recognition,” in _Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003_ , 2003, pp. 142–147. [Online]. Available: http://aclweb.org/anthology/W/W03/W03-0419.pdf
  * [5] D. Klein, J. Smarr, H. Nguyen, and C. D. Manning, “Named entity recognition with character-level models,” in _Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003_ , 2003, pp. 180–183. [Online]. Available: http://aclweb.org/anthology/W/W03/W03-0428.pdf
  * [6] Z. Huang, W. Xu, and K. Yu, “Bidirectional LSTM-CRF models for sequence tagging,” _CoRR_ , vol. abs/1508.01991, 2015. [Online]. Available: http://arxiv.org/abs/1508.01991
  * [7] X. Ma and E. H. Hovy, “End-to-end sequence labeling via bi-directional lstm-cnns-crf,” in _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_ , 2016. [Online]. Available: http://aclweb.org/anthology/P/P16/P16-1101.pdf
  * [8] A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” _Neural Networks_ , vol. 18, no. 5-6, pp. 602–610, 2005. [Online]. Available: https://doi.org/10.1016/j.neunet.2005.06.042
  * [9] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _CoRR_ , vol. abs/1412.6980, 2014. [Online]. Available: http://arxiv.org/abs/1412.6980
  * [10] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Improving neural networks by preventing co-adaptation of feature detectors,” _CoRR_ , vol. abs/1207.0580, 2012. [Online]. Available: http://arxiv.org/abs/1207.0580
  * [11] J. R. Finkel, T. Grenager, and C. D. Manning, “Incorporating non-local information into information extraction systems by gibbs sampling,” in _ACL 2005, 43rd Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 25-30 June 2005, University of Michigan, USA_ , 2005, pp. 363–370. [Online]. Available: http://aclweb.org/anthology/P/P05/P05-1045.pdf
  * [12] R. Florian, A. Ittycheriah, H. Jing, and T. Zhang, “Named entity recognition through classifier combination,” in _Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003_ , 2003, pp. 168–171. [Online]. Available: http://aclweb.org/anthology/W/W03/W03-0425.pdf
  * [13] J. Kazama and K. Torisawa, “Exploiting wikipedia as external knowledge for named entity recognition,” in _EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic_ , 2007, pp. 698–707. [Online]. Available: http://www.aclweb.org/anthology/D07-1073
  * [14] D. Lin and X. Wu, “Phrase clustering for discriminative learning,” in _ACL 2009, Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2-7 August 2009, Singapore_ , 2009, pp. 1030–1038. [Online]. Available: http://www.aclweb.org/anthology/P09-1116
  * [15] W. Radford, X. Carreras, and J. Henderson, “Named entity recognition with document-specific KB tag gazetteers,” in _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015_ , 2015, pp. 512–517. [Online]. Available: http://aclweb.org/anthology/D/D15/D15-1058.pdf
  * [16] L. Ratinov and D. Roth, “Design challenges and misconceptions in named entity recognition,” in _Proceedings of the Thirteenth Conference on Computational Natural Language Learning, CoNLL 2009, Boulder, Colorado, USA, June 4-5, 2009_ , 2009, pp. 147–155. [Online]. Available: http://aclweb.org/anthology/W/W09/W09-1119.pdf
  * [17] T. Zhang and D. Johnson, “A robust risk minimization based named entity recognition system,” in _Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003_ , 2003, pp. 204–207. [Online]. Available: http://aclweb.org/anthology/W/W03/W03-0434.pdf
  * [18] A. Zidouni, S. Rosset, and H. Glotin, “Efficient combined approach for named entity recognition in spoken language,” in _INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010_ , 2010, pp. 1293–1296.
  * [19] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. P. Kuksa, “Natural language processing (almost) from scratch,” _Journal of Machine Learning Research_ , vol. 12, pp. 2493–2537, 2011. [Online]. Available: http://dl.acm.org/citation.cfm?id=2078186
  * [20] D. Gillick, C. Brunk, O. Vinyals, and A. Subramanya, “Multilingual language processing from bytes,” in _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_ , 2016, pp. 1296–1306. [Online]. Available: http://aclweb.org/anthology/N/N16/N16-1155.pdf
  * [21] C. N. dos Santos and V. Guimarães, “Boosting named entity recognition with neural character embeddings,” in _Proceedings of the Fifth Named Entity Workshop, NEWS@ACL 2015, Beijing, China, July 31, 2015_ , 2015, pp. 25–33. [Online]. Available: https://doi.org/10.18653/v1/W15-3904
  * [22] Z. Yang, R. Salakhutdinov, and W. W. Cohen, “Multi-task cross-lingual sequence tagging from scratch,” _CoRR_ , vol. abs/1603.06270, 2016. [Online]. Available: http://arxiv.org/abs/1603.06270
  * [23] A. Bharadwaj, D. R. Mortensen, C. Dyer, and J. G. Carbonell, “Phonologically aware neural model for named entity recognition in low resource transfer settings,” in _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_ , 2016, pp. 1462–1472.
  * [24] V. Yadav and S. Bethard, “A survey on recent advances in named entity recognition from deep learning models,” in _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_ , 2018, pp. 2145–2158.