# Time-resolved XUV Opacity Measurements of Warm-Dense Aluminium

S.M. Vinko<EMAIL_ADDRESS>Department of Physics, Clarendon
Laboratory, University of Oxford, Parks Road, Oxford OX1 3PU, UK  V. Vozda
Charles University, Faculty of Mathematics and Physics, Institute of Physics,
Ke Karlovu 5, CZ-121 16, Prague 2, Czech Republic  Institute of Physics, Czech
Academy of Sciences, Na Slovance 2, Prague 8, Czech Republic  J. Andreasson
ELI Beamlines, Institute of Physics, Czech Academy of Sciences, Na Slovance 2,
CZ-182 21, Prague 8, Czech Republic  Chalmers University of Technology,
Department of Physics, Göteborg, Sweden  S. Bajt  Deutsches Elektronen-
Synchrotron DESY, Notkestrasse 85, Hamburg, Germany  J. Bielecki  European
XFEL GmbH, Holzkoppel 4, 22869 Schenefeld, Germany  T. Burian  Institute of
Physics, Czech Academy of Sciences, Na Slovance 2, Prague 8, Czech Republic
J. Chalupsky  Institute of Physics, Czech Academy of Sciences, Na Slovance 2,
Prague 8, Czech Republic  O. Ciricosta  Department of Physics, Clarendon
Laboratory, University of Oxford, Parks Road, Oxford OX1 3PU, UK  M. P.
Desjarlais  Pulsed Power Sciences Center, Sandia National Laboratories,
Albuquerque, NM 87185, USA  H. Fleckenstein  Center for Free-Electron Laser
Science, DESY, Notkestrasse 85, 22607 Hamburg, Germany  J. Hajdu  Laboratory
of Molecular Biophysics, Department of Cell and Molecular Biology, Uppsala
University, Husargatan 3, Box 596, SE-75124 Uppsala, Sweden  ELI Beamlines,
Institute of Physics, Czech Academy of Sciences, Na Slovance 2, CZ-182 21,
Prague 8, Czech Republic  V. Hajkova  Institute of Physics, Czech Academy of
Sciences, Na Slovance 2, Prague 8, Czech Republic  P. Hollebon  Department of
Physics, Clarendon Laboratory, University of Oxford, Parks Road, Oxford OX1
3PU, UK  L. Juha  Institute of Physics, Czech Academy of Sciences, Na Slovance
2, Prague 8, Czech Republic  M.F. Kasim  Department of Physics, Clarendon
Laboratory, University of Oxford, Parks Road, Oxford OX1 3PU, UK  E. E.
McBride  SLAC National Accelerator Laboratory, Menlo Park, CA 94025, USA  K.
Muehlig  Laboratory of Molecular Biophysics, Department of Cell and Molecular
Biology, Uppsala University, Husargatan 3Box 596SE-751 24 Uppsala, Sweden
T.R. Preston  European XFEL GmbH, Holzkoppel 4, 22869 Schenefeld, Germany  D.
S. Rackstraw  Department of Physics, Clarendon Laboratory, University of
Oxford, Parks Road, Oxford OX1 3PU, UK  S. Roling  Universität Münster, Busso-
Peus-Strasse 10, 48149 Münster, Germany  S. Toleikis  Deutsches Elektronen-
Synchrotron DESY, Notkestrasse 85, Hamburg, Germany  J. S. Wark  Department of
Physics, Clarendon Laboratory, University of Oxford, Parks Road, Oxford OX1
3PU, UK  H. Zacharias  Universität Münster, Busso-Peus-Strasse 10, 48149
Münster, Germany

###### Abstract

The free-free opacity in plasmas is fundamental to our understanding of energy
transport in stellar interiors and for inertial confinement fusion research.
However, theoretical predictions in the challenging dense plasma regime are
conflicting and there is a dearth of accurate experimental data to allow for
direct model validation. Here we present time-resolved transmission
measurements in solid-density Al heated by an XUV free-electron laser. We use
a novel functional optimization approach to extract the temperature-dependent
absorption coefficient directly from an oversampled pool of single-shot
measurements, and find a pronounced enhancement of the opacity as the plasma
is heated to temperatures of order the Fermi energy. Plasma heating and
opacity-enhancement is observed on ultrafast time scales, within the duration
of the femtosecond XUV pulse. We attribute further rises in the opacity on ps
timescales to melt and the formation of warm-dense matter.

The free-free opacity of a dense plasma at finite temperatures is a
fundamental manybody problem on the boundary between plasma Ron and Tzoar
(1963) and condensed matter physics Sturm and Pajanne (1973); Sturm  _et al._
(1990), with further practical applications across astrophysics, laser-plasma
interactions and inertial confinement fusion research Hu  _et al._ (2014). At
solid density, particle correlations, degeneracy and manybody effects all play
an important role in determining how materials interact with light Hopfield
(1965); Sturm (1982), invalidating the classical Coulomb-logarithm-based
inverse bremsstrahlung picture widely applied in plasma physics modelling
Dawson and Oberman (1962); Cauble and Rozmus (1985); Pfalzner and Gibbon
(1998). At the same time, the need to treat finite temperatures makes detailed
approaches from condensed matter theory challenging to implement in practice,
and increasingly unfeasible for temperatures exceeding of order 10 eV Hollebon
_et al._ (2019).

Because of its convenient electronic structure and the near-free behaviour of
its valence electrons, aluminium irradiated at extreme ultra-violet (XUV)
wavelengths is an ideal testbed to study free-free light-matter interactions
at high electron densities. In this context, recent attempts have been made
aimed at understanding historical discrepancies in the predicted and measured
absorption coefficients in ground state Al at XUV wavelengths Hollebon  _et
al._ (2019), with further investigations pushing well into the warm-dense
matter regime Kettle  _et al._ (2016); Williams  _et al._ (2018). Current
theoretical predictions generally agree that the free-free absorption cross
section should initially increase as the temperature of the system is raised
to the Fermi temperature Vinko  _et al._ (2009); Iglesias (2010); Shaffer  _et
al._ (2017); Hollebon  _et al._ (2019), before starting to fall off at higher
temperatures according to the $T^{-3/2}$ temperature dependence of inverse
bremsstrahlung (IB) theory Decker  _et al._ (1994). There are, however,
considerable discrepancies in the predicted size, shape, extent and
explanation of this effect.

Few experimental results of sufficient quality for opacity model benchmarking
in regimes beyond the ground state are available in the literature. Kettle et
al. conducted measurements of the XUV free-free opacity in laser-heated Al
Kettle  _et al._ (2016), but observed no significant change in the overall
absorption at the 1 eV estimated temperature of the plasma. More recently,
Williams et al. used 3 keV x-rays of the LCLS free-electron laser (FEL) to
isochorically heat an Al foil which was subsequently probed by high-order
laser harmonics Williams  _et al._ (2018). Average temperatures were estimated
to have reached around 6 eV. The authors observed an increase in the
absorption, an effect they attributed to a change in the ion structure, i.e.,
to melt and the formation of a warm-dense matter state. However, the measured
transmission was averaged over a range of different temperatures and the
authors did not report an absorption coefficient for the heated sample.

A common problem in opacity experiments is that it is challenging to measure
accurately the plasma temperature, especially in the presence of strong
gradients created by the heating source. If the opacity is a non-linear
function of temperature it will also be increasingly difficult to interpret
average measurements where the absorption takes place across a range of
different plasma conditions. This makes measuring the temperature dependence
of the absorption coefficient a formidable challenge, albeit an important one
if theoretical approaches are to be quantitatively validated. Here we describe
an alternative approach to tackling this issue. We combine a forward model for
the self-heating of an FEL-irradiated sample with functional optimization and
exploration algorithms to extract the temperature dependence of the absorption
coefficient directly from an oversampled transmission dataset. This approach
allows us to access the temperature dependence of the opacity without ever
having to measure or average the temperature distribution. We then use these
results to interpret time-resolved pump-probe measurements to infer the effect
of electron and ion heating in the dense plasma regime.

Figure 1: Experimental setup: the XUV pulse is split and delayed in time
before both pulses are focused on the sample by a multilayer coated off-axis
parabolic mirror. The transmitted signal is measured by a downstream CCD
camera.

The experimental setup is illustrated schematically in Fig. 1. A 32 nm XUV
pulse from the Hamburg FEL FLASH Ackermann  _et al._ (2007) is split via an
autocorrelator into a pump and probe pulse Wöstmann  _et al._ (2013), with
time-delays of up to 5 picoseconds. The pulses co-propagate toward a
multilayer-coated off-axis parabolic mirror (OAP, reflectivity of 31% at 32
nm) which focuses them onto thin Al foil targets. The pulses overlap spatially
in focus only. Foil thicknesses of 200 nm and 300 nm were chosen to optimize
changes of the transmitted signal as a function of heating. The FEL pulse
duration was 100–150 fs, optimized to maximize the total energy in the beam.
This pulse duration determines the overall time resolution of the experiment.
Alignment of the OAP and microfocus characterization was done using ablative
imprints on poly(methyl methacrylate) Chalupský  _et al._ (2010). The spot
sizes in focus were $(4.0\pm 0.5)$ and $(5.4\pm 0.7)$ $\mu$m2 for the pump and
probe, respectively. This is consistent with previous microfocusing efforts
Nelson  _et al._ (2009). The focal spot and beam overlap are monitored via an
on-axis, in-vacuum microscope, with a hole drilled through its optic to allow
the FEL beam to pass through, while still providing micron-scale imaging
resolution. The transmitted beam expands behind the target and illuminates a
filtered CCD detector. The pulse energy was measured upstream by a gas monitor
detector (GMD) Tiedtke  _et al._ (2008) and was correlated with the observed
signal intensity on the CCD in the absence of a target over a range of signal
levels. This calibration is used to infer the energy of the pulse when a
target is placed in focus and the transmission measured.

We irradiate samples at a photon energy of 38.8 eV (32 nm), significantly
above the Al plasma frequency (15 eV) but below the first bound-edge of inner
shell $2p$ states at 73 eV. The photons thus interact only with the near-free
valence electrons, in the non-collective regime, making this an ideal
prototypical system to study free-free absorption at electron densities
exceeding $10^{23}$ cm-3. The XUV pulse photoexcites the electrons which then
collisionally redistribute their energy, and create a warm electron gas within
a crystal ion lattice on time scales $<$100 fs Vinko  _et al._ (2010);
Medvedev  _et al._ (2011); Dharma-wardana (2015). On picosecond time scales
energy is transferred to the ions which heat and the system melts, forming a
warm dense plasma Cho  _et al._ (2011).

While the absorption and electron heating processes are isochoric, significant
gradients in temperature and electron density can be formed by the spatial
distribution of the XUV pulse both on the surface of the sample and
volumetrically. The sample is therefore not homogeneously heated, and a
single, average temperature is a poor descriptor of a plasma for determining
free-free opacities.

The presence of gradients in FEL-irradiated systems thus seems to be a major
disadvantage to plasma investigations. However, because gradients are related
to the imprint of the XUV intensity distribution in focus on target, they are
both predictable and measurable. Unlike in optical laser-plasma experiments,
here there is no mechanism for MeV hot-electron generation, the Keldysh
parameter is negligible, and the mean free path of the electrons created by
the XUV pulse is of order of 10 nm, small compared with the FEL spot size of a
few micron. The intensity distribution can be complex, determined by source
profile effects, beam quality and beamline and focusing optics, but it remains
fixed over the course of the experiment. Crucially, it can be measured via
ablative imprints, and the 3D structure of the pulse can be reconstructed
Chalupský  _et al._ (2013). This provides a fascinating opportunity not only
to dispose of the problems generated by gradients in the first place, but to
use them to our advantage in understanding the absorption process itself.

Figure 2: Schematic representation of the functional optimization approach
used to extract the opacity as a function of electron temperature and density
by constraining it to best-match the dataset of integrated transmission
measurements.

We proceed as follows. Firstly, we note that the XUV intensities here are
relatively low, far below the non-linear regime threshold of $\sim 10^{16}$
Wcm-2 Nagler  _et al._ (2009), so photoabsorption is a linear process and
follows the Beer-Lambert law:

$\frac{\mathrm{d}I}{\mathrm{d}x}=-\kappa(T_{e},n_{e})x,$ (1)

with $I$ the pulse intensity, $x$ the depth into the target and
$\kappa(\cdot)$ the absorption coefficient, a function of the free-electron
temperature $T_{e}$ and density $n_{e}$. As the energy is absorbed by the
electrons, knowledge of the electron equation of state (EoS) and of the
functional form of the absorption coefficient thus fully determines the
heating dynamics of the irradiated sample.

Figure 3: Total experimental and simulated transmission measured through 200
nm (a) and 300 nm (b) foils for a range of different total FEL energies on
target, alongside the simulated transmissions within a 2$\sigma$ band. c)
Absorption coefficient extracted by our functional optimization approach. The
curve is constrained by the data up to the peak temperatures of 26 eV. Bands
are determined by MCMC calculations. Also plotted are the theoretical
predictions based on ref. Hollebon  _et al._ (2019) assuming either cold ions
or an equilibrated system, the IB-based theory of Iglesias Iglesias (2010),
and the cold absorption from the CXRO database CXRO Database .

A simple EoS can be constructed by assuming the electronic structure of Al is
well-described by finite-temperature density functional theory (DFT) Mazevet
and Zérah (2008); Vinko  _et al._ (2010); Williams  _et al._ (2018). The
energy density of the electrons at a temperature $T_{e}$ is given by

$E/V=\int_{0}^{\infty}\varepsilon\;\mathcal{D}(\varepsilon)f_{\rm
FD}(\varepsilon,T_{e};\mu)\;\mathrm{d}\varepsilon,$ (2)

where we set the energy of the bottom of the valence band to 0,
$\mathcal{D}(\varepsilon)$ denotes the density of states, $f_{\rm FD}(\cdot)$
the Fermi-Dirac distribution and $\mu$ the chemical potential. To calculate
the density of states across the temperatures of interest we used the ABINIT
code Torrent  _et al._ (2008); Gonze  _et al._ (2009, 2016). We observe that
the valence electrons remain at constant (solid) density up to temperatures
around 10 eV, but above that thermal ionization of the $2p$ and $2s$ states
produces a temperature-dependent valence electron density $n_{e}(T_{e})$. This
process is fully accounted for in the DFT modelling. We will assume that
electrons excited by 30-40 eV above the Fermi energy can thermalise most of
their energy on timescales short compared with the duration of the XUV pulse.
This assumption will be validated later by our experimental results. Self-
consistently solving Eqs.(1) and (2), given some form for
$\kappa(T_{e},n_{e})$, and using the XUV energy distribution on target,
constitutes a forward model for XUV self-heating. According to this model we
can calculate the energy absorption and target heating as the XUV pulse
propagates through our sample. By the end of the pulse we obtain a 3D map of
the conditions present, and a measure of the total transmitted pulse energy.

Given that we measure the energy distribution on target, one can hope to
extract the absorption coefficient as a function of temperature and density
purely from measurements of the total energy in the beam before and after
interacting with the sample. Clearly, each such datapoint systematically
encodes a range of different plasma conditions within the absorbing system,
and so the gradients contain all the information on the energy-density
dependence of the absorption coefficient. Thus, by sampling the transmission
of many pulses with different total energy content, we can reconstruct the
functional form of the absorption coefficient. What is perhaps surprising in
this approach, illustrated in Fig. 2, is that it allows us to extract the
absorption coefficient as a function of temperature without ever having to
explicitly measure the temperature.

The experimental transmission measured through 200 and 300 nm foils for a
range of different FEL energies on target is displayed in Figs. 3a and 3b, for
a total of over 5000 single shots split into 13 equally-spaced energy bins.
The uncertainties represent the 1-$\sigma$ scatter within each bin. Our
forward model is run for each point shown, and the optimization objective is
to find a form for $\kappa(T_{e},n_{e})$ that produces the best match to every
point in the dataset. The absorption function is sampled every 2 eV on a
temperature grid up to 50 eV. The modelling suggests peak temperatures of 26
eV are reached at the highest experimental pulse energies.

Uncertainties on the data remain significant as it is challenging to measure
the transmission to better than 10-20%. As such, a calculation of how these
uncertainties impact the final form of $\kappa(T_{e},n_{e})$ is needed. For
this we use Bayesian inference to explore the space of functions able to
represent the absorption coefficient, via a combination of optimization and
Markov Chain Monte Carlo (MCMC) algorithms Kasim  _et al._ (2019). We start by
finding the best-fit solution using the stochastic CMA-ES optimization
algorithm Hansen (2016), and use it as a starting point for the ensemble MCMC
Goodman and Weare (2010) using only the stretch move. For the sampling we
deployed 32 walkers in parallel collecting a total of over 300,000 samples
after 100,000 were discarded for burn-in. The resulting absorption coefficient
is shown in Fig. 3c, with 1$\sigma$ (68%) and 2$\sigma$ (95%) confidence
intervals. We also show the theoretical predictions from time-dependent DFT
calculations based on the work of Hollebon et al. for both an equilibrated
system Hollebon  _et al._ (2019) and for a system with ions at 300 K, and from
Iglesias Iglesias (2010).

Having understood how a single XUV pulse heats the sample, we now turn our
attention to the pump-probe measurements. The probe pulses typically have an
energy of around 1–1.5 $\mu J$, while the pump varies between 2–4 $\mu J$. We
see from Fig. 3 that a 1 $\mu J$ pulse already gives rise to heating and a
change in the transmission from the cold value. The heating of the probe is
thus not negligible, but experimentally is was not possible to further reduce
its energy and still acquire reliable transmission values. The measured pump-
probe transmission is shown in Fig. 4 for both 200 and 300 nm foils. Negative
times indicate the prior arrival of the probe pulse. We observe a marked
decrease in the transmission of the probe pulse around $t=0$, the size of
which is consistent with the absorption coefficient extracted from single-shot
measurements. At negative times the probe pulse arrives first and mildly heats
the system, but as we move to the zero-delay region, where both pump and probe
pulses hit the target at the same time, more heating occurs leading to higher
absorption. At later times the system probed by the second pulse remains
heated on picosecond timescales so the transmission remains suppressed. We fit
the probe data at $\pm 1$ ps with a sigmoid, illustrating that the change in
the absorption takes place on femtosecond timescales, within the duration of
the pulse. This supports our initial assumption of thermalization within the
XUV pulse in our heating model. The pump pulse shows a similar behaviour to
the probe, but because it contains more energy the change in transmission at
$t=0$ is smaller. In contrast to the probe pulse, the pump arrives earlier in
time for positive delays, so here the transmission is higher as the total
heating is lower. For negative times the pump comes after the probe, and so
the transmission is further suppressed. The width of the transition here is
harder to estimate but is consistent with the timescales observed for the
probe pulse.

We ran our forward model on the pump-probe data using the absorption
coefficient extracted from the self-heating results. We show these results
with the diamond and cross symbols in Fig. 4: the diamonds indicate the
simulation of a single pulse only, while the crosses indicate the transmission
of a pulse through an already heated sample. We see broad agreement with the
experimental data for all cases. The probe pulse is slightly larger than the
pump, and the outer wings of the pulses do not overlap. However, this does not
seem to matter much from the modelling, due to the small values of the
absorption coefficient at low temperatures. We note that the values of the
simulated transmissions assume a single well-defined energy on target, but
experimentally data was collected over many shots with a considerable spread
in energy, in part determined by the FWHM of the reflectivity peak of the OAP.
Here we show the mean of the data as an indicative value, with the error bar
determined by the variation in the data.

Figure 4: Pump-probe transmission measurements in Al samples 200 and 300 nm
thick. Negative times correspond to the probe pulse arriving first and
positive times to the pump arriving first. The simulated transmissions for a
single (diamond) or two pulses (cross) are also shown, where the forward model
was used with the absorption coefficient extracted from the single-short self-
heating data.

Theoretically there are two processes that give rise to an enhanced absorption
at finite temperatures. The first is electronic, due to thermal broadening of
the plasmon peak Vinko  _et al._ (2009), increases in the many-body screening
length and a reduction in the electron degeneracy as the electrons heat
Iglesias (2010). The second is due to the change in the ion structure factor
as the ions heat and the system melts Vinko  _et al._ (2009); Hollebon  _et
al._ (2019). The electron contribution is readily observed on femtosecond
timescales in our data. In contrast, the effects of melt are expected to take
place over longer, picosecond timescales. From the data in Fig. 4 we observe
an indication of an additional systematic decrease in the probe transmission
for time delays beyond 1 ps. Our results contain only four points in this
region so the effect is challenging to quantify accurately, but the change in
the transmission implies a further 10-20% enhancement of the absorption
coefficient due to the formation of warm dense matter.

In summary, we presented time-resolved measurements of the free-free opacity
in XUV-heated Al. By using a functional optimization approach to the
interpretation of the experimental transmission data we were able to use the
gradients created by the FEL isochoric heating process to extract the
absorption coefficient as a function of electron temperature and density,
without needing to measure the electron temperature, density or ionization
explicitly. We find that the absorption increases initially with heating and
peaks at temperatures around the Fermi temperature. While the heating of the
electron subsystem dominates the overall change in the opacity, both
electronic and ionic effects lead to an increased absorption at finite
temperatures. We find hot opacities that are significantly larger than
predicted by theoretical calculations.

###### Acknowledgements.

Portions of this research were carried out at the FLASH facility. We are
grateful to the scientific and technical staff of FLASH for their outstanding
facility and support. This work was supported by the European Community’s
Seventh Framework Programme (FP7/2007-2013) under the grant agreement CALIPSO
312284 (EU Support of Access to Synchrotrons/FELs in Europe ). S.M.V. is a
University Research Fellow of the Royal Society. S.M.V., M.F.K, and J.S.W.
acknowledge support from the U.K. EPSRC grant EP/P015794/1 and from the Royal
Society. T.B., J.Ch., V. H., L.J. and V.V. appreciate support by CMEYS (grants
LTT17015 and CZ.02.1.01/0.0/0.0/16_013/0001552) and CSF (grants 17-05167S a
19-03314S). J.A. and J.H. are supported by projects
CZ.02.1.01/0.0/0.0/16_019/0000789 (ADONIS) and
CZ.02.1.01/0.0/0.0/15_003/0000447 (ELIBIO) from European Regional Development
Fund. J.A. acknowledges support from the Röntgen Ångström Cluster, Chalmers
Area of Advance; Material Science and the Ministry of Education, Youth and
Sports as part of targeted support from the National Programme of
Sustainability II. S.R and H.Z. acknowledge support from the BMBF under grant
05K13PM2.

## References

  * Ron and Tzoar (1963) A. Ron and N. Tzoar, Phys. Rev. 131, 12 (1963).
  * Sturm and Pajanne (1973) K. Sturm and E. Pajanne, J. Phys. F 3, 199 (1973).
  * Sturm  _et al._ (1990) K. Sturm, E. Zaremba, and K. Nuroh, Phys. Rev. B 42, 6973 (1990).
  * Hu  _et al._ (2014) S. X. Hu, L. A. Collins, V. N. Goncharov, T. R. Boehly, R. Epstein, R. L. McCrory, and S. Skupsky, Phys. Rev. E 90, 033111 (2014).
  * Hopfield (1965) J. J. Hopfield, Phys. Rev. 139, A419 (1965).
  * Sturm (1982) K. Sturm, Adv. Phys. 31, 1 (1982).
  * Dawson and Oberman (1962) J. Dawson and C. Oberman, Phys. Fluids 5, 517 (1962).
  * Cauble and Rozmus (1985) R. Cauble and W. Rozmus, Phys. Fluids 28, 3387 (1985).
  * Pfalzner and Gibbon (1998) S. Pfalzner and P. Gibbon, Phys. Rev. E 57, 4698 (1998).
  * Hollebon  _et al._ (2019) P. Hollebon, O. Ciricosta, M. P. Desjarlais, C. Cacho, C. Spindloe, E. Springate, I. C. E. Turcu, J. S. Wark, and S. M. Vinko, Phys. Rev. E 100, 043207 (2019).
  * Kettle  _et al._ (2016) B. Kettle _et al._ , Phys. Rev. E 94, 023203 (2016).
  * Williams  _et al._ (2018) G. O. Williams _et al._ , Phys. Rev. A 97, 023414 (2018).
  * Vinko  _et al._ (2009) S. M. Vinko, G. Gregori, M. P. Desjarlais, B. Nagler, T. J. Whitcher, R. W. Lee, P. Audebert, and J. S. Wark, HEDP 5, 124 (2009).
  * Iglesias (2010) C. A. Iglesias, HEDP 6, 311 (2010).
  * Shaffer  _et al._ (2017) N. Shaffer, N. Ferris, J. Colgan, D. Kilcrease, and C. Starrett, HEDP 23, 31 (2017).
  * Decker  _et al._ (1994) C. D. Decker, W. B. Mori, and J. M. Dawson, Phys. Plasmas 1, 4043 (1994).
  * Ackermann  _et al._ (2007) W. Ackermann _et al._ , Nat. Photon. 1, 336 (2007).
  * Wöstmann  _et al._ (2013) M. Wöstmann, R. Mitzner, T. Noll, S. Roling, B. Siemer, F. Siewert, S. Eppenhoff, F. Wahlert, and H. Zacharias, Journal of Physics B: Atomic, Molecular and Optical Physics 46, 164005 (2013).
  * Chalupský  _et al._ (2010) J. Chalupský, J. Krzywinski, L. Juha, V. Hájková, J. Cihelka, T. Burian, L. Vyšín, J. Gaudin, A. Gleeson, M. Jurek, _et al._ , Opt. Express 18, 27836 (2010).
  * Nelson  _et al._ (2009) A. Nelson _et al._ , Optics Express 17, 18271 (2009).
  * Tiedtke  _et al._ (2008) K. Tiedtke _et al._ , J. Appl. Phys. 103, 094511 (2008).
  * Vinko  _et al._ (2010) S. M. Vinko _et al._ , Phys. Rev. Lett. 104, 225001 (2010).
  * Medvedev  _et al._ (2011) N. Medvedev, U. Zastrau, E. Förster, D. O. Gericke, and B. Rethfeld, Phys. Rev. Lett. 107, 165003 (2011).
  * Dharma-wardana (2015) M. W. C. Dharma-wardana, Contributions to Plasma Physics 55, 85 (2015).
  * Cho  _et al._ (2011) B. I. Cho, K. Engelhorn, A. A. Correa, T. Ogitsu, C. P. Weber, H. J. Lee, J. Feng, P. A. Ni, Y. Ping, A. J. Nelson, D. Prendergast, R. W. Lee, R. W. Falcone, and P. A. Heimann, Phys. Rev. Lett. 106, 167601 (2011).
  * Chalupský  _et al._ (2013) J. Chalupský, T. Burian, V. Hájková, L. Juha, T. Polcar, J. Gaudin, M. Nagasono, R. Sobierajski, M. Yabashi, and J. Krzywinski, Opt. Express 21, 26363 (2013).
  * Nagler  _et al._ (2009) B. Nagler _et al._ , Nat. Phys. 5, 693 (2009).
  * (28) CXRO Database, http://henke.lbl.gov/optical_constants/ .
  * Mazevet and Zérah (2008) S. Mazevet and G. Zérah, Phys. Rev. Lett. 101, 155001 (2008).
  * Torrent  _et al._ (2008) M. Torrent, F. Jollet, F. Bottin, G. Zérah, and X. Gonze, Comput. Mater. Sci. 42, 337 (2008).
  * Gonze  _et al._ (2009) X. Gonze _et al._ , Computer Physics Communications 180, 2582 (2009).
  * Gonze  _et al._ (2016) X. Gonze _et al._ , Comput. Phys. Commun. 205, 106 (2016).
  * Kasim  _et al._ (2019) M.F. Kasim, T. Galligan, J. Topp-Mugglestone, G. Gregori, and S.M. Vinko, Phys. Plasmas 26, 112706 (2019).
  * Hansen (2016) N. Hansen, ArXiv e-prints: 1604.00772 (2016).
  * Goodman and Weare (2010) J. Goodman and J. Weare, Comm. App. Math and Comp. Sci. 5, 65 (2010).


# News Meets Microblog: Hashtag Annotation via Retriever-Generator

Xiuwen Zheng${}^{1,2\dagger}\;\;\;\;\;$ Dheeraj
Mekala${}^{1\dagger}\;\;\;\;\;$ Amarnath Gupta${}^{2}\;\;\;\;\;$ Jingbo
Shang1,3
1 Department of Computer Science and Engineering, University of California San
Diego, CA, USA
2 San Diego Supercomputer Center, University of California San Diego, CA, USA
3 Halıcıoğlu Data Science Institute, University of California San Diego, CA,
USA
{xiz675, dmekala, a1gupta<EMAIL_ADDRESS>
###### Abstract

††$\dagger$ Represents equal contribution

Hashtag annotation for microblog posts has been recently formulated as a
sequence generation problem to handle emerging hashtags that are unseen in the
training set. The state-of-the-art method leverages conversations initiated by
posts to enrich contextual information for the short posts. However, it is
unrealistic to assume the existence of conversations before the hashtag
annotation itself. Therefore, we propose to leverage news articles published
before the microblog post to generate hashtags following a Retriever-Generator
framework. Extensive experiments on English Twitter datasets demonstrate
superior performance and significant advantages of leveraging news articles to
generate hashtags.

##  1 Introduction

Hashtag annotation, which aims to generate hashtags for microblog posts, is a
fundamental task in microblog platforms (e.g., Twitter) because hashtags play
a crucial role in user engagement and a wide range of downstream tasks Zhang
et al. (2019). This problem was recently formulated as sequence generation, so
it can handle emerging hashtags that never or rarely appeared in training. Our
analysis on 36K tweets shows that more than $80\%$ hashtags appear at most 5
times (Appendix A), verifying the necessity of this formulation.

Microblog posts are usually short, which makes it hard to generate hashtags
merely from the posts. State-of-the-art method Wang et al. (2019) introduces
conversations initiated by the post for more context. However, such
conversations are not available when composing the post and hashtags.

Table 1: A microblog post and its related news articles. Entities based on
which news are retrieved are in blue. Words indicative for the hashtag are in
red. Figure 1: Overview of our Retriever-Generator framework.

In fact, microblog posts are typically reflections of recent events happening
in the real world and the information of such events would be widely available
in news articles. Therefore, news articles could serve as a powerful source
for hashtag annotation in addition to the post itself. In our experiments, we
observe that around 28% of named entities in hashtags are present in retrieved
news articles. Table 1 presents an example where a microblog post $T1$ and
news $N1$ talk about the same event (i.e., choice of supreme court judge). The
hashtag #BrettKavanaugh is not present in the post, however, it appears in
$N1$. We also observe that recent news articles are more relevant to the post
content than older news articles. For example, the news $N2$ talks about the
supreme court, however, it is outdated to predict the hashtags.

Inspired by these observations, we propose HashNews following a Retriever-
Generator framework as illustrated in Figure 1. Our retriever adapts the
traditional information retrieval model to emphasize emerging entities while
retrieving relevant news articles. Our generator jointly models posts and
relevant news with a novel hybrid bi-attention.

Our contributions are summarized as follows:

  * •

We develop a time-aware entity-focused ranking function that ranks news
articles by taking temporal popularity of entities into consideration.

  * •

We design a novel hybrid bi-attention that captures effects of news and posts
on each other with emphasis on important entities from news.

  * •

Experimental results on two real-world datasets demonstrate superiority of
leveraging news articles to generate hashtags.

To the best of our knowledge, this is the first work on leveraging news
articles for hashtag annotation with a sequence generation architecture.

Reproducibility. Our code is made public
here111https://github.com/dheeraj7596/HashNews and datasets will be provided
upon request.

##  2 Our Retriever-Generator Framework

Given a microblog post $p$ and a set of news articles published before $p$,
our goal is to generate hashtag $y$ represented by a word sequence $\langle
y_{1},y_{2},\cdots y_{|y|}\rangle$. Here, $|y|$ is the number of the words.

We propose a Retriever-Generator framework shown in Figure 1. It has two
modules: (1) a time-aware entity-focused news retriever and (2) a hybrid bi-
attention hashtag generator.

###  2.1 Time-Aware Entity-Focused Retriever

Table 2: Statistics of our datasets. % of entity hashtags: the number of
hashtags with at least one entity, % of all-entity hashtags: the number of
hashtags consisting of only entities, $\bm{P^{p}}$: percentage of words in
hashtags that are also present in their corresponding posts.

Datasets |  | # of
---
posts
| # of distinct
---
hashtags
| Avg # of hashtags
---
per post
| Avg # of entities
---
per post
| % of entity
---
hashtags
| % of all-entity
---
hashtags
| $\bm{P^{p}}$
---
Tweets2018 | 35966 | 19635 | 2.70 | 1.78 | 40.48 | 21.43 | 9.49
Tweets2020 | 27418 | 14687 | 2.24 | 1.02 | 47.47 | 18.81 | 11.79

Entities, especially emerging ones, shall be more important than other words
to analyze the focus of a post. As shown in Table 2, we observe that more than
$40\%$ hashtags in our datasets contain at least one entity, and about $20\%$
hashtags consist of only entities. Thus, we extract entities in a post as
query keywords to retrieve relevant news. During retrieval, we emphasize
important entities by exploiting their temporal popularities.

Temporal Popularity of Entities. We propose an adaptive accumulation method to
collect candidate news articles based on timestamps. Specifically, for a post
$p$ created at day $t$, we construct a series of $k$ candidate corpora ($k=5$
in experiments) $\mathcal{D}$ = $\\{D_{1},D_{2},\ldots,D_{k-1},D_{k}\\}$.
$D_{i}$ contains all the news articles published in the time window from day
$t-i$ to day $t$. From each corpus $D_{i}\in\mathcal{D}$, we retrieve the most
relevant article (excluding the ones have been retrieved from
$D_{1},\ldots,D_{i-1}$), leading to a total of $k$ retrieved news articles.
$D_{i}$ is a subset of $D_{i+1}$, so, all $k$ news articles could be from
$D_{1}$.

Given an entity $e$ and a candidate corpus $D_{i}$, we compute the temporal
popularity of $e$ through a comparative analysis. Intuitively, if an entity
becomes more popular in the recent news than usual, it can be referred as an
emerging entity, which should be emphasised more when ranking news articles.
Therefore, we randomly sample a time-independent news corpus $R$ and compare
the inverse document frequency (IDF) Han et al. (2011) values of entity $e$ in
the two corpora, $D_{i}$ and $R$. Formally, we define the temporal popularity
(TP) of $e$ in corpus $D_{i}$ as

$\displaystyle\text{TP}(e,D_{i})=\frac{IDF(e,R)}{IDF(e,D_{i})}.$

Ranking Function. Our ranking function is inspired by the Okapi BM25 function
Robertson et al. (1995) and further incorporates temporal popularity to
emphasize emerging entities. Given a microblog post $p$ and a news $d$ from
the corpus $D_{i}$, the score is calculated as follows:

$\displaystyle s(p,d|D_{i})=\sum_{e\in
p}\text{TP}(e,D_{i}){\frac{f_{e}^{d}\cdot(a+1)}{f_{e}^{d}+a\left(1-b+b\cdot{\frac{|d|}{\mathbb{E}_{D_{i}}[|d|]}}\right)}}$

where $f_{e}^{d}$ is the frequency of entity $e$ in the document $d$.
$\mathbb{E}_{D_{i}}[|d|]$ is the average document length of corpus $D_{i}$.
$a$ and $b$ are parameters inherent from Okapi BM25 typically set to 1.2 and
0.75 Jones et al. (2000).

Microblogs are usually written in an informal style, so two different phrases
in a post and a news article may refer to the same entity in the real world.
Therefore, the entities in posts and news have to be carefully matched to
better estimate $f_{e}^{d}$ and $IDF(e,\cdot)$. We develop a soft entity
matching mechanism described in detail in Appendix B.

For each microblog $p$, we rank news articles in $\mathcal{D}$ using the above
ranking function and retrieve top-$k$ articles
$H=\\{h_{1},h_{2},\ldots,h_{k}\\}$. Since news articles are usually long, to
reduce computational cost, we construct context words $c^{p}$ and use them as
input to generator. The weight of a word $t$ in selected news is computed by,

$\displaystyle w^{p}_{t}=\sum_{i=1}^{k}s(p,h_{i}|D_{i})\cdot freq(t,h_{i})$
(1)

where $freq(t,h_{i})$ denotes the frequency of $t$ in news $h_{i}$. Words with
the highest weights are selected as context words $c^{p}$ for post $p$.

###  2.2 Hybrid Bi-Attention Generator

Our generator extends the dual encoder network Wang et al. (2019) which
consists of two encoders, one for the microblog post and the other for the
context constructed from news as illustrated in Figure 1. To effectively
capture their joint effects on each other, we employ a novel hybrid bi-
attention over the encoders’ outputs, incorporating local weights of context
tokens from Eq. (1).

Encoders. The inputs for the two encoders are post tokens
$\mathbf{x}^{p}_{1},\cdots,\mathbf{x}^{p}_{|p|}$ and context tokens
$\mathbf{x}^{c}_{1},\cdots,\mathbf{x}^{c}_{|c^{p}|}$. We adopt the
bidirectional gated recurrent unit (Bi-GRU) Cho et al. (2014) for both
encoders to encode a post token $\mathbf{x}^{p}_{i}$ into $\mathbf{h}_{i}^{p}$
and context token $\mathbf{x}^{c}_{i}$ into $\mathbf{h}_{i}^{c}$.
Specifically,
$\mathbf{h}_{i}^{p}=[\overrightarrow{\mathbf{h}_{i}^{p}};\overleftarrow{\mathbf{h}_{i}^{p}}]$
is the concatenation of forward and backward hidden states for the $i$-th post
token $\mathbf{x}^{p}_{i}$. Likewise, $\mathbf{h}_{j}^{c}$ is the
concatenation of forward and backward hidden states for the $j$-th context
token $\mathbf{x}^{c}_{j}$ via another Bi-GRU. $\mathbf{h}_{i}^{p}$ and
$\mathbf{h}_{j}^{c}$ are $d$-dimensional vectors.

Our Hybrid Bi-attention. Building upon traditional bi-attention Seo et al.
(2016), we introduce local attentions to emphasize temporally popular entities
while obtaining news-aware post representation. Our intuition is that a news
article containing trending entities would get large ranking scores, and
frequent entity tokens in these news will be selected out and assigned higher
weights by the retriever (Eq.(1)). Their weights should be taken into
consideration while obtaining news-aware post representation. Specifically,
the post-aware news representation $\mathbf{r}_{j}^{p}$ and news-aware post
representation $\mathbf{r}_{i}^{c}$ are computed as follows,

$\displaystyle\mathbf{r}_{j}^{p}=\sum_{i=1}^{|p|}a_{ij}^{p}\cdot\mathbf{h}_{i}^{p}\quad\text{and}\quad\mathbf{r}_{i}^{c}=\sum_{j=1}^{|c^{p}|}a_{ij}^{c}\cdot\mathbf{h}_{j}^{c}$

where

$\displaystyle a_{ij}^{p}$
$\displaystyle=\frac{\exp(\mathbf{h}_{i}^{p}\cdot\mathbf{W}_{b}\cdot\mathbf{h}_{j}^{c})}{\sum_{i^{\prime}}\exp(\mathbf{h}_{i^{\prime}}^{p}\cdot\mathbf{W}_{b}\cdot\mathbf{h}_{j}^{c})},$
$\displaystyle a_{ij}^{c}$
$\displaystyle=\frac{\exp(w^{p}_{j}\cdot\mathbf{h}_{i}^{p}\cdot\mathbf{W}_{b}\cdot\mathbf{h}_{j}^{c})}{\sum_{j^{\prime}}\exp(w^{p}_{j^{\prime}}\cdot\mathbf{h}_{i}^{p}\cdot\mathbf{W}_{b}\cdot\mathbf{h}_{j^{\prime}}^{c})},$
$\displaystyle w_{j}^{p}$ $\displaystyle=\text{weight of }j^{th}\text{ context
token $\mathbf{x}^{c}_{j}$ by Eq.~{}\eqref{eq:weight}}.$

Note that, $\mathbf{W}_{b}\in\mathbb{R}^{d\times d}$ is a learnable weight
matrix measuring the proximity between $\mathbf{h}_{i}^{p}$ and
$\mathbf{h}_{j}^{c}$.

Table 3: Evaluation results of compared methods and our Retriever-Generator
framework (HashNews).

Models | Tweets2018 | Tweets2020
---|---|---
$F_{1}@1$ | $F_{1}@5$ | $F_{1}@10$ | ACC | MAP | RG-1 | $F_{1}@1$ | $F_{1}@5$ | $F_{1}@10$ | ACC | MAP | RG-1
Seq2Seq | 7.17 | 9.95 | 8.14 | 13.13 | 6.41 | 8.33 | 6.18 | 6.82 | 5.23 | 9.92 | 5.25 | 9.33
Seq2Seq+Copy | 7.97 | 11.13 | 8.92 | 14.51 | 7.21 | 9.18 | 6.81 | 7.45 | 5.69 | 10.73 | 5.64 | 9.67
LSTM-TOP | 10.18 | 14.03 | 11.36 | 18.65 | 8.36 | 8.92 | 9.65 | 10.95 | 8.39 | 15.51 | 8.99 | 10.51
HashNews | 11.51 | 15.16 | 12.08 | 21.11 | 10.96 | 13.36 | 10.59 | 10.90 | 8.08 | 17.00 | 9.84 | 14.77
HashNews-NoRank | 10.93 | 14.50 | 11.65 | 20.02 | 10.20 | 12.69 | 9.26 | 9.82 | 7.49 | 14.92 | 8.48 | 13.05
HashNews-NoRankNoLocal | 10.50 | 14.21 | 11.68 | 19.24 | 9.52 | 12.21 | 9.20 | 9.47 | 7.27 | 14.78 | 8.40 | 12.98

Merge Layer, Decoder, & Generation. As shown in Fig. 1, the representations
returned by hybrid bi-attention on each encoder is fused by passing them
through a multi-layer perceptron layer. Then the decoder concatenates the
output and uses attention-based GRU to generate hashtag as a word sequence
$\mathbf{y}$. These parts of model follow Seo et al. (2016).

We follow the previous work Wang et al. (2019) and use negative log-likelihood
loss while training. Stochastic Gradient descent algorithm is used to minimize
the loss function and learn parameters. During the inference and generation of
hashtags, beam search is applied on the word distribution to select word at
each timestamp. We generate a ranking list of hashtags during inference.

##  3 Experiments

###  3.1 Datasets

Tweets. Two large-scale datasets from English Twitter are collected using
Twitter developer API: (1) _Tweets2018_ , contains posts published from
October 1 to October 13, 2018; (2) _Tweets2020_ , contains posts published
from May 25 to June 8, 2020.

News. We crawled news from over 170 newspapers, which were published during
September 27 to October 13, 2018 for _Tweets2018_ and during May 20 to June 8,
2020 for _Tweets2020_. Nearly 5,000 news articles are collected for each day.

We filter out the posts without any non-inline hashtags222An inline hashtag is
part of a sentence and when segmented into sequence of words, it adds meaning.
A non-inline hashtag is usually at the end of the post.. To preserve
information in tweets, we replace inline hashtags and mentions in the posts
with their respective segmented word sequences using ekphrasis Baziotis et al.
(2017). We refer non-inline hashtags as hashtags. Hashtags are segmented into
sequences of words and are used as the targets to be generated. We detect
named entities in the processed posts and news using Stanford CoreNLP toolkit
Manning et al. (2014). Table 2 summarizes the statistics of the datasets.

###  3.2 Compared Methods

We compare with the following methods and we use the same hyperparameters for
all generator baselines and our model for fair comparison, and for LSTM-TOP
model Li et al. (2016), we use the hyperparameters stated in their paper.

  * •

Seq2Seq Bahdanau et al. (2014) uses attention based encoder-decoder
architecture to generate hashtags from the processed post.

  * •

Seq2Seq+Copy Gu et al. (2016) uses Seq2Seq architecture with copy mechanism.

  * •

LSTM-TOP Li et al. (2016) is a topic-modeling based approach that uses
attention-based LSTM model for learning representations.

HashNews-NoRank is an ablation of our HashNews that applies BM25 ranking
function for retriever instead of our ranking function. HashNews-NoRankNoLocal
further replaces our hybrid attention by bi-attention, which is used in Wang
et al. (2019). The implementation details can be found in Appendix C.

###  3.3 Evaluation Metrics and Results

We adopt some of the evaluation metrics used in Wang et al. (2019) including
$F_{1}$ scores at top $K$ predictions ($F_{1}@K$) where $k=1,5,10$, mean
average precision (MAP) Manning et al. (2008) computed on the top 5
predictions, and ROUGE metric (RG-1) Lin (2004) for the highest-ranked
hashtags. We also report the prediction accuracy($ACC$) for the highest
ranking hashtags.

Table 3 summarizes the evaluation results of all methods. Our proposed
framework achieves the best performance among all the compared sequence
generation methods. HashNews and its variants outperform Seq2Seq and
Seq2Seq+Copy with a significant margin. This observation indicates that news
articles are helpful in generating hashtags. The comparison between HashNews
and HashNews-NoRank demonstrates that the time-aware ranking function which
emphasizes emerging entities retrieves more relevant news, resulting in
superior performance. HashNews performs better than LSTM-TOP on all metrics on
Tweets2018 dataset and on $F_{1}@1$, accuracy, MAP, RG-1 on Tweets2020
dataset. The superior performance is because the topic modeling approaches are
extractive in nature and are unable to produce phrasal hashtags. LSTM-TOP
slightly outperforms HashNews on $F1@5$ and $F1@10$ on Tweets2020 dataset.
Their model is a classification-based model which selects hashtags from a fix
set. For tweets2020 dataset, in the training set, the top 300 frequent
hashtags take up 50%, leading to the classification-based model to perform
well. HashNews outperforms the dual encoder with bi-attention model (HashNews-
NoRankNoLocal/ Wang et al. (2019)) demonstrating that the weights of context
words reflect their importance and our hybrid attention effectively
incorporates it while generating hashtags.

We also present two case studies to demonstrate the effectiveness of HashNews
in Appendix D.

##  4 Related Work

Hashtag Annotation has been modeled as different kinds of problems in the
previous works. Many methods view this task as a classification problem and
predict hashtags from a pre-defined list Zhang et al. (2017); Huang et al.
(2016); Gong and Zhang (2016), however, this formulation makes it impossible
to generate emerging hashtags. Another line of approach employs topic models
to generate hashtags Wu et al. (2016); Li et al. (2016). However, these models
are usually unable to produce phrasal hashtags, which can be achieved by our
framework by generating hashtag word sequences. Since a hashtag summarizes the
key ideas of the microblog, some approaches model this problem as
keyphrase/sequence generation, for example, using deep RNN model Zhang et al.
(2016) and leveraging conversations as a supplementary data source Zhang et
al. (2018); Wang et al. (2019). However, it is unrealistic to assume the
existence of conversations before the annotation. Sedhai and Sun (2014)
studies the hashtag recommendation problem for hyperlinked tweets(i.e., tweets
containing links to web pages) by mining the context from linked webpages. In
contrast to this, our framework is applicable to any tweet and we leverage
widely available, up-to-date news articles to generate hashtags. Note that,
although hashtag generation and text summarization tasks look similar,
summarization methods can’t be applied to the posts as they are too short and
lack enough context to summarize. Moreover, summarization methods Nallapati et
al. (2016); Liu et al. (2018) usually aim to generate a summary of few
sentences, whereas, in the case of hashtags, they have to be short (1-2 words)
and carry maximum information to represent the post.

##  5 Conclusions and Future Work

In this paper, we propose HashNews, a Retriever-Generator framework that
leverages news articles published before a microblog post created, to enrich
the contextual information and jointly models the context with post to
generate hashtags. Experimental results and case studies demonstrate that our
model outperforms previous methods, thereby signifying the advantages of
leveraging news articles for generating hashtag. In the future, we are
interested in user-aware hashtag generation by taking the properties of users
into consideration.

## References

  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_.
  * Baziotis et al. (2017) Christos Baziotis, Nikos Pelekis, and Christos Doulkeridis. 2017.  Datastories at semeval-2017 task 4: Deep lstm with attention for message-level and topic-based sentiment analysis.  In _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_ , pages 747–754, Vancouver, Canada. Association for Computational Linguistics.
  * Cho et al. (2014) Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  _arXiv preprint arXiv:1406.1078_.
  * Gong and Zhang (2016) Yuyun Gong and Qi Zhang. 2016.  Hashtag recommendation using attention-based convolutional neural network.  In _IJCAI_ , pages 2782–2788.
  * Gu et al. (2016) Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016.  Incorporating copying mechanism in sequence-to-sequence learning.  _arXiv preprint arXiv:1603.06393_.
  * Han et al. (2011) Jiawei Han, Jian Pei, and Micheline Kamber. 2011.  _Data mining: concepts and techniques_.  Elsevier.
  * Huang et al. (2016) Haoran Huang, Qi Zhang, Yeyun Gong, and Xuan-Jing Huang. 2016.  Hashtag recommendation using end-to-end memory networks with hierarchical attention.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 943–952.
  * Jones et al. (2000) K Sparck Jones, Steve Walker, and Stephen E. Robertson. 2000.  A probabilistic model of information retrieval: development and comparative experiments: Part 2.  _Information processing & management_, 36(6):809–840.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Klein et al. (2017) Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017\.  Opennmt: Open-source toolkit for neural machine translation.  _arXiv preprint arXiv:1701.02810_.
  * Li et al. (2016) Yang Li, Ting Liu, Jing Jiang, and Liang Zhang. 2016.  Hashtag recommendation with topical attention-based lstm.  Coling.
  * Lin (2004) Chin-Yew Lin. 2004.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pages 74–81.
  * Liu et al. (2018) Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018.  Generative adversarial network for abstractive text summarization.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32.
  * Manning et al. (2008) Christopher D Manning, Hinrich Schütze, and Prabhakar Raghavan. 2008.  _Introduction to information retrieval_.  Cambridge university press.
  * Manning et al. (2014) Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014.  The Stanford CoreNLP natural language processing toolkit.  In _Association for Computational Linguistics (ACL) System Demonstrations_ , pages 55–60.
  * Nallapati et al. (2016) Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016.  Abstractive text summarization using sequence-to-sequence rnns and beyond.  _arXiv preprint arXiv:1602.06023_.
  * Robertson et al. (1995) Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995.  Okapi at trec-3.  _Nist Special Publication Sp_ , 109:109.
  * Sedhai and Sun (2014) Surendra Sedhai and Aixin Sun. 2014.  Hashtag recommendation for hyperlinked tweets.  In _Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval_, pages 831–834.
  * Seo et al. (2016) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016.  Bidirectional attention flow for machine comprehension.  _arXiv preprint arXiv:1611.01603_.
  * Smith et al. (1981) Temple F Smith, Michael S Waterman, et al. 1981.  Identification of common molecular subsequences.  _Journal of molecular biology_ , 147(1):195–197.
  * Wang et al. (2019) Yue Wang, Jing Li, Irwin King, and Michael R Lyu1 Shuming Shi. 2019.  Microblog hashtag generation via encoding conversation contexts.  In _Proceedings of NAACL-HLT_ , pages 1624–1633.
  * Wu et al. (2016) Yong Wu, Yuan Yao, Feng Xu, Hanghang Tong, and Jian Lu. 2016.  Tag2word: Using tags to generate words for content based tag recommendation.  In _Proceedings of the 25th ACM international on conference on information and knowledge management_ , pages 2287–2292.
  * Zhang et al. (2017) Qi Zhang, Jiawen Wang, Haoran Huang, Xuanjing Huang, and Yeyun Gong. 2017.  Hashtag recommendation for multimodal microblog using co-attention network.  In _IJCAI_ , pages 3420–3426.
  * Zhang et al. (2016) Qi Zhang, Yang Wang, Yeyun Gong, and Xuan-Jing Huang. 2016.  Keyphrase extraction using deep recurrent neural networks on twitter.  In _Proceedings of the 2016 conference on empirical methods in natural language processing_ , pages 836–845.
  * Zhang et al. (2019) Suwei Zhang, Yuan Yao, Feng Xu, Hanghang Tong, Xiaohui Yan, and Jian Lu. 2019.  Hashtag recommendation for photo sharing services.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 5805–5812.
  * Zhang et al. (2018) Yingyi Zhang, Jing Li, Yan Song, and Chengzhi Zhang. 2018.  Encoding conversation context for neural keyphrase extraction from microblog posts.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1676–1686.

Appendix

##  Appendix A Hashtag Frequency Distribution

The frequency distribution of hashtags in both datasets is shown in Figure 2.
This figure shows that the distribution is skewed with more than $80\%$ of
hashtags appearing at most $5$ times.

Figure 2: Frequency distribution of hashtags.

##  Appendix B Entity Matching between Microblog and News

Same entity might be expressed in different ways in microblog posts and news
articles. Therefore, the entities in posts and news have to be carefully
matched to better estimate the frequency and IDF of entities in our ranking
function. To address this problem, we develop a soft entity matching
mechanism.

We observe that entities in microblogs are usually written in a concise way,
e.g., Donald Trump as Trump; whereas in news, the formal and complete version
of the entity usually occurs at least once in the whole content, and the
shorter versions may occur multiple times after it. Based on these
observations, we propose two intuitive and powerful matching criteria named
_Strict Match_ and _Conditional Match_.

For a microblog post $p$ and a news article $d\in D_{i}$, let the entity
sequences extracted from them be $E^{p}$ and $E^{d}$ respectively. The strict
match aims at finding the formal and longer version of entities in $E^{d}$
that match to an entity in $E^{p}$.

######  Definition 1 (Strict Match).

An entity pair $(e^{p},e^{d})$ is called a strict match if

$\frac{|\\{\mathbf{t}^{p}|\mathbf{t}^{p}\in e^{p}\text{and
}\exists\mathbf{t}^{d}\in e^{d}\text{ s.t.
}align(\mathbf{t}^{p},\mathbf{t}^{d})\geq t\\}|}{|e^{p}|}\geq q,$ (2)

where $\mathbf{t}^{p},\mathbf{t}^{d}$ are tokens of entities $e^{p},e^{d}$,
$align(\cdot,\cdot)$ is the character-level alignment score function between
two tokens, $|e^{p}|$ is the number of tokens in $e^{p}$ and $t,q\in(0,1]$ are
two pre-specified thresholds.

According to Definition 1, $(e^{p},e^{d})$ is a strict match when more than
$q|e^{p}|$ tokens in $e^{p}$ are matched with tokens in $e^{d}$ at character
level, based on alignment score calculated using Smith–Waterman algorithm
Smith et al. (1981).

We expand the strict match results set by adding the matching results of
shortened versions of strictly matched entities from the same news using our
matching rule called Conditional Match, defined as follows:

######  Definition 2 (Conditional Match).

For an entity $e^{d}\in E^{d}$, entity pair $(e^{p},e^{d})$ is called a
conditional match given $e^{\prime d}$ if $\exists$ $e^{\prime d}\in E^{d}$
such that both $(e^{p},e^{\prime d})$ and $(e^{d},e^{\prime d})$ are strict
matches.

The following example illustrates the strict match and conditional match.

Example 1.  Suppose there is a microblog post containing an entity
$e_{1}=\text{``Donald Trump''}$, and a news article containing entities
$e_{2}=\text{``Donald Trump''}$ and $e_{3}=\text{``Trump''}$. In the first
step, $(e_{1},e_{2})$ is recognized as a strict match since all tokens in
$e_{1}$ are found in $e_{2}$. $(e_{1},e_{3})$ is then detected as a
conditional match given $e_{2}$ because (“Trump”, “Donald Trump”) is also a
strict match and $e_{2},e_{3}$ are from the same news article.

Based on the strict match and conditional match, we revise the TF and IDF
formulations as follows:

$\displaystyle f_{e}^{d}$ $\displaystyle=|M^{soft}(e,d)|,$ (3) $\displaystyle
IDF(e,D_{i})$ $\displaystyle=\log\frac{|D_{i}|}{|\\{d|d\in D_{i}\text{ and
}M^{strict}(e,d)\neq\phi\\}|}.$

where

$M^{strict}(e,d)=\\{e^{d}|e^{d}\in E^{d}\text{ and }(e,e^{d})\text{ is a
strict match}\\}\\\ $ $\displaystyle M^{soft}(e,d)=\\{e^{d}|e^{d}\in
E^{d},\exists e^{\prime d}\in M^{strict}(e,d)\text{ and }$
$\displaystyle(e^{d},e^{\prime d})\text{ is a strict match}\\}$

raw tweet | sentences in relevant news | context words with weights rank | our model results | seq2seq results
---|---|---|---|---
A huge #BlackLivesMatter banner has been hung on the fence separating protesters from Lafayette Park in front of the White House at DC’s #JusticeforGeorgeFloyd #DCProtests #WashingtonDCProtests’ | Wednesday got started with law enforcement officers cutting off the main protest area, near Lafayette Park and the White House. A large group of protesters laid down on a Washington D.C. street to repeat the words George Floyd said as … | dc: 6th;
floyd: 7th;
washington:10th;
george:13th;
protests: 64th | dc protests;
dc protest;
black lives matter dc;
black lives matter;
protests 2020 | trump resign now;
trumpout2020;
trumpresign;
defundthepolice;
whitehouseprotests
Don Lemon @donlemon Is really shining bright right now! #BlackLivesMatter #JusticeForGeorgeFloyd #GeorgeFloydWasMurdered #Trump #TrumpIsAnIdiot #TrumpIsALoser #TrumpIsRacist’ | “… Not one of them tried to do anything to help him,” Tera Brown, Floyd’s cousin, told CNN’s Don Lemon. Protesters marched in Minneapolis Tuesday night in response to the death of George Floyd. … called for the four officers to face murder charges. | floyd: 1st
george: 10th
murder: 125th
racist: 131st | george floyd;
police brutality;
black lives matter;
george floyd protests;
trump | colin powell;
message;
r4 today;
tucker;
arizona
Table 4: Case study 2. Two real examples of tweets, news and generated
hashtags.

##  Appendix C Implementation Detail of Models

Our generator is implemented on the OpenNMT framework Klein et al. (2017). We
choose the embedding size as 300; We use two-layered Bi-GRU cells for encoder
and one layer of GRU cells for decoder and the hidden size of GRU is set to
400. Adam optimizer Kingma and Ba (2014) is used to learn parameters with
learning rate initialized to $0.001$. The learning rate decreases with a decay
rate of 0.5 and we adopt the early stopping strategy. The batch size is $64$
and dropout rate is $0.1$. While generating hashtags, we set a maximum
sequence length as 10 and the beam size as 20.

##  Appendix D Case Study

We present two case studies demonstrating the effectiveness of HashNews.

Effectiveness of time-aware ranking function. In BM25, the weight of a query
keyword is measured by IDF whereas in our ranking score formulation, it is
measured by Temporal Popularity (TP). We present the normalized IDF and TP
values for a few words from Tweets2018 dataset in Figure 3. After going
through the news, we learnt about two major events that happened during the
time window of the dataset: (1) the 73rd session of the United Nations General
Assembly was opened and plenary sessions were going on; (2) the Senate voted
50–48 to confirm Brett Kavanaugh’s nomination to the Supreme Court. From
Figure 3, we observe that TP formulation assigns the emerging and trending
entities higher weights compared to BM25 formulation.

Figure 3: Case study 1. Comparison between IDF score and Temporal Popularity
(TP) used in our time-aware entity-focused ranking function. Note that all the
values are normalized to range $[0,1]$ for a fair comparison.

Effectiveness of HashNews on short posts. Table 4 presents two examples of
real-world tweets, a few sentences from their respective retrieved news,
context tokens generated by the retriever, and top hashtags generated by
HashNews and Seq2Seq. Named entities detected from tweets are colored blue and
words in hashtag sequences that appear in news are colored red. In the first
example, based on the two entities detected in the tweet, news about the
Washington DC protests and George Floyd protests are retrieved which reflect
the topic of this tweet. We observe that many hashtag words can be found in
the context and they have relatively high weights, demonstrating the
effectiveness of our time-aware entity-focused ranking function. The hashtags
generated are quite relevant to the tweet and this shows the effectiveness of
our generator in leveraging the news articles during generation. In the second
example, since the tweet is really short, without much contextual information,
the Seq2Seq generates irrelevant hashtags. However, our framework selects news
related to George Floyd’s death using the entity “Don Lemon” and generates
hashtags that are informative and relevant to the tweet.