# Intergalactic medium dispersion measures of fast radio bursts estimated from
IllustrisTNG simulation and their cosmological applications

Z. J. Zhang School of Astronomy and Space Science, Nanjing University, Nanjing
210093, China  K. Yan School of Astronomy and Space Science, Nanjing
University, Nanjing 210093, China  C. M. Li School of Astronomy and Space
Science, Nanjing University, Nanjing 210093, China  G. Q. Zhang School of
Astronomy and Space Science, Nanjing University, Nanjing 210093, China  F. Y.
Wang School of Astronomy and Space Science, Nanjing University, Nanjing
210093, China  Key Laboratory of Modern Astronomy and Astrophysics (Nanjing
University), Ministry of Education, Nanjing 210093, China

###### Abstract

Fast radio bursts (FRBs) are millisecond-duration radio transients and can be
used as a cosmological probe. However, the dispersion measure (DM) contributed
by intergalactic medium (IGM) is hard to be distinguished from other
components. In this paper, we use the IllustrisTNG simulation to realistically
estimate the $DM_{\rm IGM}$ up to $z\sim 9$. We find $DM_{\rm
IGM}=892^{+721}_{-270}$ pc cm-3 at $z=1$. The probability distribution of
$DM_{\rm IGM}$ can be well fitted by a quasi-Gaussian function with a long
tail. The tail is caused by the structures along the line of sight in IGM.
Subtracting DM contributions from the Milky Way and host galaxy for localized
FRBs, the $DM_{\rm IGM}$ value is close to the derived $DM_{\rm IGM}-z$
relation. We also show the capability to constrain the cosmic reionization
history with the $DM_{\rm IGM}$ of high-redshift FRBs in the IllustrisTNG
universe. The derived $DM_{\rm IGM}-z$ relation at high redshifts can be well
fitted by a $tanh$ reionization model with the reionization redshift $z=5.95$,
which is compatible with the reionization model used by the IllustrisTNG
simulation. The $DM_{\rm IGM}$ of high-redshift FRBs also provides an
independent way to measure the optical depth of cosmic microwave background
(CMB). Our result can be used to derive the pseudo-redshifts of non-localized
FRBs for $DM_{\rm IGM}<4000$ pc cm-3.

radio bursts; intergalactic medium; reionization

##  1 Introduction

Fast radio bursts (FRB) are millisecond luminous radio pulses with large
dispersion measures (DMs), well in excess of the Milky Way contribution. There
have been more than one hundred FRBs reported since Lorimer et al. (2007)
found the first one from archival data. Among them, only thirteen FRBs have
been localized (Chatterjee et al., 2017; Bannister et al., 2019; Ravi et al.,
2019; Prochaska et al., 2019; Marcote et al., 2020; Macquart et al., 2020).
Apparently, FRBs can be divided into to two types, repeating and non-repeating
ones and their host galaxy properties may be different.

The DM is defined as the column density of free electron density along a given
line of sight (LoS). The observed DM is usually divided into several parts

$DM=DM_{\rm{MW}}+DM_{\rm{halo}}+DM_{\rm{IGM}}+\frac{DM_{\rm{host}}+DM_{\rm{source}}}{1+z}.$
(1)

In the above equation, $DM_{\rm MW}$ is the contribution of the interstellar
medium in the Milky Way, which can be derived from the NE2001 Galactic free
electron density model (Cordes & Lazio, 2002) or the YMW model (Yao et al.,
2017). $DM_{\mathrm{halo}}$ is contributed by the free electrons in the
Galactic halo. Prochaska & Zheng (2019) found that the $DM_{\mathrm{halo}}$ is
50 pc cm-3 < $DM_{\mathrm{halo}}$ < 80 pc cm-3. Recently, Yamasaki & Totani
(2020) estimated that the mean $DM_{\rm halo}$ is 43 pc cm-3, with a full
range of 30-245 pc cm-3. Zhang et al. (2020) derived host contribution
$DM_{\rm host}$ distributions of repeating and non-repeating FRBs with the
IllustrisTNG simulation. They found that the distributions of
$DM_{\mathrm{host}}$ can be well fitted by a log-normal function. For non-
repeating FRBs, the median of $DM_{\mathrm{host}}$ is about $30-70$ pc cm-3 in
the redshift range $z=0.1-1.5$. The $DM_{\mathrm{source}}$ depends on the
central engine of FRBs. If FRB is generated by mergers of binary neutron stars
(Wang et al., 2016; Zhang, 2020), the value of $DM_{\mathrm{source}}$ is small
(Wang et al., 2020; Zhao et al., 2020).

The DM contributed by intergalactic medium (IGM) is an important cosmological
probe (Zhou et al., 2014; Gao et al., 2014; Muñoz et al., 2016; Yu & Wang,
2017; Wang & Wang, 2018; Walters et al., 2018; Jaroszynski, 2019; Wei et al.,
2019; Li et al., 2019; Wu et al., 2020; Zhao et al., 2020). By assuming the
cosmic reionization history, the value of $DM_{\mathrm{IGM}}$ can be derived
theoretically from (Ioka, 2003; Inoue, 2004; Deng & Zhang, 2014)

$DM_{\rm IGM}(z)=\frac{3c\Omega_{\rm b}H_{0}}{8\pi Gm_{\rm
p}}\int^{z}_{0}\frac{(1+z^{\prime})f_{\rm IGM}(z^{\prime})f_{\rm
e}(z^{\prime})}{E(z^{\prime})}{\rm d}z^{\prime},$ (2)

where $E(z)=H(z)/H_{0}$, $H(z)$ is Hubble parameter, $H_{0}$ is the Hubble
constant, $m_{p}$ is the mass of proton, $\Omega_{\rm b}=0.0486$ is the
density of baryons, and $f_{\rm IGM}$ is the fraction of baryon mass in the
IGM. $f_{\rm e}=Y_{\rm H}X_{\rm e,H}(z)+\frac{1}{2}Y_{\rm He}X_{\rm e,He}(z)$.
$Y_{\rm H}=3/4$ and $Y_{\rm He}=1/4$ are the mass fractions of hydrogen and
helium, respectively. $X_{\rm e,H}$ and $X_{\rm e,He}$ are the ionization
fractions of intergalactic hydrogen and helium.

Different from the previous theoretical investigations which use extragalactic
DM with a homogeneous universe (Ioka, 2003; Inoue, 2004), McQuinn (2014)
considered the effect of inhomogeneity with three models for halo gas profile
of the ionized baryons. Dolag et al. (2015), Pol et al. (2019) and Zhu & Feng
(2020) studied $DM_{\rm IGM}$ with different cosmological simulations at low-
redshift (z < 2) universe. Jaroszynski (2019) used the Illustris simulation to
estimate the $DM_{\rm IGM}$ and its scatters in the z < 5 universe. However,
the simulation accuracy can be improved with the latest IllustrisTNG
simulation. Another advantage of IllustrisTNG is that it can provide the
electron density directly instead of converting from the dark matter particle
number density to baryonic matter density like Pol et al. (2019). The
complicated conversion may bring in extra uncertainties. Besides, the
IllustrisTNG has a wide redshift range.

Considering these advantages, we choose the IllustrisTNG (the successor of
Illustris) simulation which possesses both the high accuracy and large
structures in the range 0 < z < 20\. Such a broad range of redshifts enables
us to examine the prospect of constraining the cosmic reionization history
with high-redshift FRBs. It also gives a chance to check whether FRBs can be a
new type of standard candles besides supernovae, which is crucial for distance
measurement.

In this paper, we use the IllustrisTNG simulation to study $DM_{\rm IGM}$ and
their cosmological applications, especially at the high-redshift universe. The
outline is as follows. An introduction to the IllustrisTNG simulation and our
method to derive $DM_{\rm IGM}$ from the simulation are given in Section 2. We
show the result in Section 3. The scenario of constraining the cosmic
reionization history with FRBs is discussed in Section 4. We estimate the
redshifts of non-localized FRBs with $DM_{\rm IGM}-z$ relation in Section 5.
Conclusions are given in Section 6.

##  2 Methods

###  2.1 Data access to IllustrisTNG

Figure 1: A schematic diagram of choosing LoS. There are 5125 pipes at each
redshift as shown in this figure.

The IllustrisTNG project, a successor of Illustris, consists of three large
volume, cosmological and gravo-magnetohydrodynamical simulations (Pillepich et
al., 2018; Springel et al., 2018; Nelson et al., 2018; Marinacci et al., 2018;
Naiman et al., 2018), such as TNG50, TNG100, and TNG300. The number represents
the scale of simulations in the unit of cMpc (c for comoving similarly
hereinafter). Each simulation contains several runs with different
resolutions. The final results are stored in 600 HDF5 files. According to the
cosmological principle, TNG300 is the best choice for our research and
TNG300-1 is chosen among the three runs for its best resolution. There are 100
snapshots at different redshifts stored in each run and each snapshot has
15,625,000,000 Voronoi gas cells in total. Each cell corresponds to a particle
and its physical parameters given by IllustrisTNG represent the whole cell. In
the 100 snapshots, 20 snapshots are ‘full’ and 80 snapshots are ‘mini’ which
only lack some particle fields. We use all the full snapshots at z < 10 and
several ‘mini’ snapshots for better accuracy. The web-based JupyterLab
Workspace and high-performance computing resources provided by IllustrisTNG
are used in this work.

###  2.2 Dispersion of IGM

For an FRB at zs, the $DM_{\rm IGM}$ can be written as

$DM_{\rm IGM}(z_{s})=\int_{0}^{z_{s}}\frac{n_{\rm e}(z)}{1+z}{\rm d}l_{\rm
prop},$ (3)

where $n_{\rm e}(z)$ is electron density in the comoving frame and ${\rm
d}l_{\rm prop}$ is the differential proper distance. Then we use the redshift
differential ${\rm d}z$ to express ${\rm d}l_{\rm prop}$ as

${\rm d}l_{\rm prop}=\frac{c}{H_{\rm 0}(1+z)E(z)}{\rm d}z,$ (4)

where

$E(z)=\sqrt{\Omega_{\rm m}(1+z)^{3}+\Omega_{\rm
k}(1+z)^{2}+\Omega_{\rm\Lambda}}.$ (5)

So the $DM_{\rm IGM}$ can be rewritten as

$DM_{\rm IGM}(z_{\rm s})=\frac{c}{H_{0}}\int_{0}^{z_{s}}\frac{n_{\rm
e}(z)}{(1+z)^{2}E(z)}{\rm d}z.$ (6)

The cosmological parameters are taken as $\Omega_{\rm m}=0.3089$,
$\Omega_{\Lambda}=0.6911$, and $H_{0}=67.74\ \rm{km\ s^{-1}\ Mpc^{-1}}$, which
are the same as those used by the IllustrisTNG simulation (Pillepich et al.,
2018).

None of cosmological simulations provides continuous universe evolution,
that’s why snapshots exist. Therefore, equation (6) cannot be applied
directly. In practice, Jaroszynski (2019) and Pol et al. (2019) clipped and
stacked the snapshots to construct the LoS. Here we solve the problem from
another aspect. The basic idea is to convert the integral into summation.

If $n_{\rm e}(z_{\rm i})$ is available, the ${\rm d}DM_{\rm IGM}/{\rm d}z$ at
a specific redshift ${z_{\rm i}}$ (${z_{\rm i}}$ = 0, 0.1, 0.2, 0.3, 0.4, 0.5,
0.7, 1, $\cdots$) can be derived by

$\frac{{\rm d}DM_{\rm IGM}}{{\rm d}z}\bigg{|}_{z={z_{\rm
i}}}=\frac{c}{H_{0}}\frac{n_{\rm e}(z_{\rm i})}{(1+z_{\rm i})^{2}E(z_{\rm
i})},$ (7)

Then we use

$\displaystyle DM_{\rm IGM}(z_{\rm i+1})=DM_{\rm IGM}(z_{\rm i})+$
$\displaystyle\frac{1}{2}(\frac{{\rm d}DM_{\rm IGM}}{{\rm
d}z}\bigg{|}_{{z_{\rm i}}}+\frac{{\rm d}DM_{\rm IGM}}{{\rm
d}z}\bigg{|}_{{z_{\rm i+1}}})(z_{\rm i+1}-z_{\rm i})$ (8)

to calculate the $DM_{\rm IGM}$ with the initial condition $DM_{\rm
IGM}(z)\bigg{|}_{z=0}=0$. Next subsection will introduce how to obtain the
electron density.

###  2.3 Calculations of $DM_{\rm IGM}$

IllustrisTNG snapshot data is not organized according to spatial position. In
order to obtain the average electron density along a given LoS, we use a
traversal method on all the 15,625,000,000 particles and find the particles
belonging to the given LoS. For computational simplicity, the LoS is chosen to
parallel to the x axis, which is similar to Jaroszynski (2019). Then we make
5125 square pipes with 200 ckpc/h ($\textit{H}_{\rm 0}\rm=100\ \textit{h}\ \rm
km\ s^{-1}\ Mpc^{-1}$) side in each snapshot and find the particles as well as
necessary parameters (including Coordinates, Density, ElectronAbundance,
GFM_Metals and StarFormationRate) in the pipes (see Figure 1). The 5125 pipes
are chosen from different locations at 24 snapshots randomly and K-S test
shows the sample size is representative. The electron density can be
calculated by

$n_{\rm e}(z)_{\rm prop}=\eta_{\rm e}X_{\rm H}\frac{\rho}{m_{\rm
p}}(1+z)^{3},$ (9)

where $\eta_{\rm e}$ is the electron abundance, $X_{\rm H}$ is the hydrogen
mass abundance, $\rho$ is the density and $m_{\rm p}$ is mass of a hydrogen
atom. The fourth parameter is the particle coordinate, which is used to select
particles. The factor $(1+z)^{3}$ converts the density in the simulation
comoving units into the proper units.

Figure 2: $DM_{\rm IGM}$ distributions from $z=0.1$ to $9$. Dashed lines are
$DM_{\rm IGM}$ distributions derived from IllustrisTNG simulations and solid
lines are the fitting results using equation (10). The best fitting parameters
are shown in Table 1. At $z>6$, the distributions overlap as shown in panel
(e), which indicates the universe do not reionize at such high redshifts.

The equation (9) cannot be used in star-forming gas because the calculation is
based on the ‘effective’ temperature of the equation of state, which is not a
physical temperature111https://www.tng-project.org/data/docs/specifications/.
Therefore, we use the parameter StarFormationRate to exclude the star-
formation gas. For comparison, star-forming particles are also excluded in
Dolag et al. (2015). Jaroszynski (2019) excluded gas cells belonging to haloes
where dark matter density is 15 times larger than the cosmic critical density.

It is hard to get the electron density field along the LoS analytically, which
requires to calculate the boundary of all the cells. While we know the cell to
which any point on a LoS belongs. We divide the pipe into 10,000 bins along
the x axis and take the geometric center coordinates as the representation of
bins. The distances between each bin and each particle in the pipe are
calculated. We choose the nearest particle of each bin and assume the bin
belongs to the cell of the chosen particle. We take an average of the electron
density of 10,000 bins and put it into equation (7). As a result, 5125 ${\rm
d}DM_{\rm IGM}/{\rm d}z$ are obtained at each redshift. Ten million $DM_{\rm
IGM}-z$ relations are built by randomly selected ${\rm d}DM_{\rm IGM}/{\rm
d}z$ from $z=0.1$ to 9.

##  3 Result

The redshifts of snapshots besides $z=0$ are shown in Table 1. The
distributions of $DM_{\rm IGM}$ (from 0 to $z$ similarly hereafter) at
different redshifts are shown in Figure 2. The DM distributions are fitted
with (McQuinn, 2014; Prochaska & Zheng, 2019; Macquart et al., 2020):

$p_{\rm
IGM}(\Delta)=A\Delta^{-\beta}\exp[-\frac{(\Delta^{-\alpha}-C_{0})^{2}}{2\alpha^{2}\sigma_{\rm
DM}^{2}}],\Delta\textgreater 0,$ (10)

where $\Delta=DM_{\rm IGM}/<DM_{\rm IGM}>$, and $\beta$ is related to the
inner density profile of gas in halos. We take $\alpha=\beta=3$, which is the
same as Macquart et al. (2020). $\sigma_{\rm DM}$ is an effective standard
deviation. $C_{0}$, which can affect the horizontal position, is the remaining
parameter to be fitted. The fitting results are shown in Table 1.

$z$ | $A$ | $C_{0}$ | $\sigma_{\rm DM}$
---|---|---|---
0.1 | 0.04721 | -13.17 | 2.554
0.2 | 0.005693 | -1.008 | 1.118
0.3 | 0.003584 | 0.596 | 0.7043
0.4 | 0.002876 | 1.010 | 0.5158
0.5 | 0.002423 | 1.127 | 0.4306
0.7 | 0.001880 | 1.170 | 0.3595
1 | 0.001456 | 1.189 | 0.3044
1.5 | 0.001098 | 1.163 | 0.2609
2 | 0.0009672 | 1.162 | 0.2160
2.4 | 0.0009220 | 1.142 | 0.1857
3 | 0.0008968 | 1.119 | 0.1566
3.5 | 0.0008862 | 1.104 | 0.1385
4 | 0.0008826 | 1.092 | 0.1233
4.4 | 0.0008827 | 1.084 | 0.1134
5 | 0.0008834 | 1.076 | 0.1029
5.2 | 0.0008846 | 1.073 | 0.09918
5.5 | 0.0008863 | 1.070 | 0.09481
5.8 | 0.0008878 | 1.067 | 0.09072
6 | 0.0008881 | 1.066 | 0.08971
6.5 | 0.0008881 | 1.066 | 0.08960
7 | 0.0008881 | 1.066 | 0.08952
8 | 0.0008881 | 1.066 | 0.08944
9 | 0.0008881 | 1.066 | 0.08941
Table 1: Redshifts of the snapshots and fitting parameters of the $DM_{\rm
IGM}$ distributions in Figure 2.

It’s obvious that the asymmetrical distributions have long tails at high
$DM_{\rm IGM}$, so we choose the most probable value for analysis, which is
also used by Dolag et al. (2015) and Pol et al. (2019). We find $DM_{\rm
IGM}(z=1)=892^{+721}_{-270}$ pc cm-3 (errors represent 95% confidence level).
Ioka (2003) and Inoue (2004) predicted $DM_{\rm IGM}(z=1)\sim 1200$ pc cm-3.
Zhang (2018) predicted $DM_{\rm IGM}(z=1)\sim 855\pm 345$ pc cm-3. Jaroszynski
(2019) found $DM_{\rm IGM}(z=1)\sim 905\pm 115$ pc cm-3 (errors represent
$1\sigma$ standard deviation). Pol et al. (2019) derived $DM_{\rm
IGM}(z=1)\sim 800^{+7000}_{-170}$ pc cm-3 with uniform weighting and $DM_{\rm
IGM}(z=1)\sim 960^{+350}_{-160}$ pc cm-3 with weighting by matter distribution
(errors represent 95% confidence level). In Figure 3, we show these above
$DM_{\rm IGM}-z$ relations. Our result is shown as blue solid line with 95%
confidence region (blue region). The $DM_{\rm IGM}$ estimated in our work is
consistent with others within 95% confidence level including the one derived
by Pol et al. (2019). The difference between Pol et al. (2019) and our result
may be caused by the conversion from the dark matter number density to the
free electron density in the MareNostrum Instituto de Ciencias del Espacio
Onion Universe simulation. A non-negligible systematic error may be from the
different cosmological parameters used by these simulations. For example,
Illustris uses the cosmological parameters from WMAP-9 measurements
(Vogelsberger et al., 2014), while IllustrisTNG uses those from Planck
(Pillepich et al., 2018).

Figure 3: $DM_{\rm IGM}-z$ relation at low redshift ($z<1.5$). The blue line
is our result from IllustrisTNG simulation and the blue shaded region is the
95% confidence region. Other results are shown for comparison. The pink line
with inverted triangle markers is taken from Ioka (2003) and Inoue (2004). The
dotted yellow line is taken from Dolag et al. (2015), the orange line with
star marker is taken from Zhang (2018) and the green line with cross marks is
Pol et al. (2019) uniform weighting result. Our result is consistent with
other works except for the result of Pol et al. (2019). However, if
considering the large confidence level of Pol et al. (2019) (not shown in this
figure), our result is consistent with theirs.

We also compare our result with several localized FRBs (Bannister et al.,
2019; Ravi et al., 2019; Prochaska et al., 2019; Macquart et al., 2020) in
Figure 4. The blue line is our result derived from TNG300 and the blue shade
region is the 95% confidence region. The theoretical one from equation (2) is
shown as dash line for the cosmological parameters from Planck (Planck
Collaboration et al., 2016). The $DM_{\rm host}$ value is adopted from Zhang
et al. (2020). Based on the host galaxy observations, Zhang et al. (2020)
calculated the $DM_{\rm host}$ of repeating and non-repeating FRBs from the
IllustrisTNG simulation. They found $DM_{\rm host}=32.97(1+z)^{0.84}$ pc cm-3
for non-repeating FRBs. For FRB 190608 we adopt its $DM_{\rm host}=137\pm 43$
pc cm-3 from observations (Chittidi et al., 2020). The $DM_{\rm MW}$ value is
derived with the NE2001 model (Cordes & Lazio, 2002). We take $DM_{\rm
halo}=50$ pc cm-3 for all FRBs. The contributions from FRB sources are
ignored. All the derived $DM_{\rm IGM}$ are compatible with our result.

Figure 4: $DM_{\rm IGM}-z$ (Macquart) relation from IllustrisTNG simulation
and several localized FRBs. The orange dashed line is the model of equation
(2). We take $f_{IGM}=0.82+0.08z/0.9$, $f_{\rm e}=7/8$, and $X_{\rm
e,H}=X_{\rm e,He}=1$. The blue line is our result derived from TNG300 and the
blue shaded region is the 95% confidence level. The two lines differ because
the ionized fraction and IGM baryon fraction are different between the model
and the simulation. We also show the $DM_{\rm IGM}$ value of several localized
FRBs. In calculation, the $DM_{\rm MW}$ value is derived with the NE2001 model
and $DM_{\rm host}$ is taken from Zhang et al. (2020) except FRB 190608 (taken
from Chittidi et al. (2020)).

##  4 The scenario of constraining the cosmic reionization history

High-redshift FRBs can be used to probe the cosmic reionization history and we
test the expectation with TNG300 simulation. The universe is mostly neutral
and non-transparent after the recombination (Peebles, 1968; Zel’dovich et al.,
1969; Seager et al., 2000). The transition from the neutral to ionized
universe called reionization, which is the first chance to detect the universe
evolution after the end of cosmic dark ages. Reionization history is a
frontier and challenging area in cosmology (Loeb & Furlanetto, 2013), because
it is hard to detect now. Usually, it is believed that the reionization of
hydrogen occurs between $z\sim 12$ and $z\sim 6$. For helium the range is
between $z\sim 6$ and $z\sim 2$ (Planck Collaboration et al., 2016). Some
works have proposed to use the FRBs to constrain the reionization history
(Zheng et al., 2014; Fialkov & Loeb, 2016; Linder, 2020; Dai & Xia, 2020;
Bhattacharya et al., 2020).

It is still difficult to localize non-repeating FRBs even at low redshifts.
The capability of localization is improving. Leung et al. (2020) demonstrated
that the blind interferometric detection and localization of non-repeating
FRBs by the CHIME and the CHIME Pathfinder can be down to about milliarcsecond
precision. In future, they will observe thousands of single FRB events to this
precision. However, considering the offsets which have been observed in FRBs,
spectroscopy should be applied for high-redshift FRBs besides radio
interferometer. To confirm the redshift, optical/infrared observations should
also be applied. Finding high-redshift quasars is a challenging task at
present, let alone the high-redshift FRB hosts which may be faint.

Combining the high burst rate and the construction of next generation
telescopes, it is hopeful to constrain the reionization history with FRBs. The
estimated $\gtrsim 10^{3}$ day-1 rate of FRBs (Petroff et al., 2019) would
provide abundant chances for future detections. Although the redshift
distribution of FRBs is unknown, it is optimistic to gain sufficient high-
redshift FRBs with the Five-hundred-meter Aperture Spherical radio Telescope
(FAST) (Zhang, 2018) and future Square Kilometer Array (SKA) (Fialkov & Loeb,
2016). If high-redshift FRBs are observed with accurate redshift information
from optical/infrared band, we can extend the $DM_{\rm IGM}-z$ relation to the
epoch of reionization (EoR). The relation can constrain the parameters in
reionization model and the ionized fraction precisely.

The wide redshift range of IllustrisTNG gives us a chance to constrain the
cosmic reionzation history. We use the tanh model given by Lewis (2008) (also
applied by Planck Collaboration et al. (2016)):

$x_{\rm e}(z)=\frac{f}{2}[1+\tanh(\frac{y_{\rm re}-y}{\Delta y})],$ (11)

where $x_{\rm e}$ is ionized fraction. f is a normalized parameter for
considering both hydrogen and helium and expressed as $f=1+f_{\rm He}=1+n_{\rm
He}/n_{\rm H}$. Typically $f\sim 1.08$, $y=(1+z)^{3/2}$, $y_{\rm re}=(1+z_{\rm
re})^{3/2}$ and $z_{\rm re}$ is defined as the redshift at which $x_{\rm
e}=f/2$. $\Delta y=1.5\sqrt{1+z_{\rm re}}\Delta z$ and $\Delta z$ reflects the
duration of reionization. In the model of Lewis (2008), $\Delta z$ is a fixed
value. It was estimated with an upper limit of 1.3 at 95% confidence level in
Planck Collaboration et al. (2016) redshift-symmetric case. Then we calculate
$n_{\rm e}(z)$ from

$n_{\rm e}(z)=\overline{n}_{\rm b}(z)X_{\rm H}x_{\rm e}(z)$ (12)

and

$\overline{n}_{\rm b}(z)=\frac{\Omega_{\rm b}\rho_{\rm cr,0}(1+z)^{3}}{m_{\rm
p}},$ (13)

where $\rho_{\rm cr,0}$ is the critical density of universe. After taking
$n_{\rm e}(z)$ into equation (6), the theoretical value of $DM_{\rm IGM}$ can
be obtained in this cosmic reionization model. Figure 5 gives the value of
$DM_{\rm IGM}$ as a function of redshift up to $z\sim 9$. The blue line shows
the derived $DM_{\rm IGM}$ from TNG300 simulation with 95% confidence region
(blue region). Cosmic reionization affects both CMB power spectrum and
kinematic Sunyaev-Zeldovich (kSZ) effect. We display the results from Planck
data combined with the kSZ effect (Planck Collaboration et al., 2016) with the
tanh model (see equation (11)) for comparison as well. The green line takes
$z_{\rm re}=7.2$, which is the result from a uniform prior on the redshift at
which the reionization ends ($z_{\rm end}$). The orange line is the result
from the prior $z_{\rm end}\ \textgreater\ 6$ and it gives $z_{\rm re}=7.8$.
The least-square-method fit (red line) indicates a very fast reionization
process ($\Delta z=0.05$ ) at $z_{\rm re}=5.95$ with the tanh model, which is
also compatible with the model used by IllustrisTNG (Faucher-Giguère et al.,
2009)222Dec 2011 version, https://galaxies.northwestern.edu/uvb-fg09/.
Therefore, high-redshift FRBs are promising probes of the cosmic reionization
history.

Figure 5: The blue line is our result and the red line is the best fit with
the $tanh$ model. For comparison, the value derived from Planck data is also
shown (Planck Collaboration et al., 2016). The green line takes $z_{\rm
re}=7.2$, which is the result from a uniform prior on the redshift at which
the reionization ends. The orange line is the result from the prior $z_{\rm
end}\ \textgreater\ 6$ and it gives $z_{\rm re}=7.8$.

Meanwhile, we calculate the optical depth $\tau(z)$ of CMB contributed by IGM
from

$\tau(z)=\int_{0}^{z}\sigma_{\rm T}n_{\rm e}(z^{\prime}){{\rm d}l_{\rm
prop}},$ (14)

where $\sigma_{\rm T}=6.25\times 10^{-25}$ cm2 is the Thompson scattering
cross section. The measurement of $\tau$ is changing for different instruments
and we are curious whether FRBs may help. Using the electron number density
derived from TNG300 simulation, we find that the total optical depth
$\tau(z>6)=0.037^{+0.006}_{-0.004}$ with 95% confidence level. Figure 6 shows
the value of $\tau(z)$ as a function of redshfit from TNG300 simulation (blue
line). The ionized electron fraction drops to zero before reionization so the
optical depth saturates at high redshifts (Fialkov & Loeb, 2016). The
saturation value can be detected by high redshift FRBs which have no
connection with CMB. The optical depth $\tau=0.058\ \pm\ 0.023$ (95%
confidence level) from Planck result is shown as the orange region (Planck
Collaboration et al., 2016). Considering the uncertainties, we find these two
results are consistent with each other. Therefore, the DM of high-redshift
FRBs provides an independent way to measure the optical depth of CMB, which
can tightly constrain the cosmic reionization history. However, we must state
that the resolution of Planck is 12 degrees which is much larger than that of
FRBs. Comparing the two results with different resolution directly may be kind
of inappropriate, however there is no need to decrease the resolution of FRBs.

Figure 6: The blue line and shade region is our result with 95% confidence
region. The orange line and region is $Planck$ result with 95% confidence
region (Planck Collaboration et al., 2016), which gives $\tau=0.058\ \pm\
0.023$.

##  5 Estimating the redshifts of non-localized FRBs

As mentioned before, there are only ten localized FRBs (Chatterjee et al.,
2017; Bannister et al., 2019; Ravi et al., 2019; Prochaska et al., 2019;
Marcote et al., 2020; Macquart et al., 2020), which means the redshifts for
most FRBs are unknown. Assuming that the accurate $DM_{\rm IGM}$ is known, we
check whether the redshift of FRB can be precisely derived from the $z-DM_{\rm
IGM}$ relation.

As mentioned in Section 2 and Section 3, we get $23\times 10,000,000$ $DM_{\rm
IGM}-z$ data points (FRBs) in total (24 snapshots and 10,000,000
combinations). We divide the data points whose $DM_{\rm IGM}$ is between 0 and
6000 pc cm-3 into 150 bins (40 pc cm-3 in each bin) and calculate the mean
redshift of these FRBs in each bin. The $z-DM_{\rm IGM}$ relation is shown in
Figure 7. The standard deviation of derived redshift is 0.74 at the pseudo-
redshfit 4.61 ($DM_{\rm IGM}=4000$ pc cm-3), which means the relative error is
16.1%. The small redshift standard deviations of FRBs whose $DM_{\rm IGM}$ are
less than 4000 pc cm-3 show a good prospect for calculating pseudo-redshfits
of non-localized FRBs.

We only take full snapshots at low redshifts, which means the redshifts of our
simulated FRBs are restricted to some fixed redshifts such as 0.1, 0.2, 0.3,
0.4, 0.5, 0.7, 1 and 1.5. However, the standard deviation should not be larger
when including mini snapshots because the standard deviation does not change
for different sampling methods. What’s more, the 40 pc cm-3 bin width can be
taken as the systematic error during the estimation of $DM_{\rm halo}$,
$DM_{\rm host}$ and $DM_{\rm source}$, which means it is even optimistic to
derive the pseudo-redshift with a ‘not very accurate’ $DM_{\rm IGM}$.

Figure 7: The $z-DM_{\rm IGM}$ relation. The short blue lines show one
standard deviation of each bin.

##  6 Conclusions

In this work, we derive $DM_{\rm IGM}$ of FRBs at the redshift range of $0\
\textless\ z\ \textless\ 9$ from IllustrisTNG simulation. We obtain $DM_{\rm
IGM}=892^{+721}_{-270}$ pc cm-3 at $z=1$. The $DM_{\rm IGM}$ value of
localized FRBs are consistent with the derived $DM_{\rm IGM}-z$ relation. At
high redshifts $z>5$, we show the scenario of probing the cosmic reionization
history with FRBs. The $tanh$ reionization model is used to fit the derived
$DM_{\rm IGM}-z$ relation at high redshifts. We find the reionization of
IllustrisTNG universe occurs quickly at $z=5.95$. This reionization model is
compatible with the theoretical model used by the IllustrisTNG simulation
(Faucher-Giguère et al., 2009). The optical depth of CMB is also derived from
the IllustrisTNG simulation, which is consistent with that from Planck
Collaboration et al. (2016). We try to limit the redshifts of non-localized
FRBs with their $DM_{\rm IGM}$. The standard deviation of pseudo-redshifts for
non-localized FRBs is 16.1% for $DM_{\rm IGM}=4000$ pc cm-3.

The highest DM of observed FRB is $2596.1\pm 0.3$ pc cm-3 and its nominal
redshift is 2.1 (Bhandari et al., 2018; Caleb et al., 2018). It is predictable
that there will be two orders of magnitude more FRBs in the next few years
(Keane, 2018). According to Fialkov & Loeb (2016) and Zhang (2018), FAST and
SKA will have enough capability to detect FRBs out to $z=14\sim 15$.
Therefore, FRBs will be a powerful and independent probe of the universe
during epoch of reionization besides hydrogen 21-cm line. The future large
sample of FRBs can be used to test our result about estimating redshifts for
non-localized FRBs. What’s more, most of previous works only considered the
mean value of $DM_{\rm IGM}$ (equation 2) for cosmological constraints, while
the scatter of it (Figure 2) which can degrade the cosmological constraints
was not handled properly. Since IllustrisTNG shows a certain shape of the
scatter, more authentic distributions of $DM_{\rm IGM}$ and $DM_{\rm host}$
can be used to test the FRB’s capability of constraining cosmological
parameters, such as $\Omega_{\rm m}$ and $\Omega_{\rm\Lambda}$.

We thank the anonymous referee for valuable comments. We thank Zhimei Tan,
Lingrui Lin and Yichen Sun for helpful discussions and thank Dylan Nelson for
his theoretical and technical help. This work is supported by the National
Natural Science Foundation of China (grant U1831207).

## References

  * Bannister et al. (2019) Bannister, K. W., Deller, A. T., Phillips, C., et al. 2019, Science, 365, 565
  * Bhandari et al. (2018) Bhandari, S., Keane, E. F., Barr, E. D., et al. 2018, MNRAS, 475, 1427
  * Bhattacharya et al. (2020) Bhattacharya, M., Kumar, P. & Linder, E. V., 2020, arXiv: 2010.14530
  * Caleb et al. (2018) Caleb, M., Keane, E. F., van Straten, W., et al. 2018, MNRAS, 478, 2046
  * Chatterjee et al. (2017) Chatterjee, S., Law, C. J., Wharton, R. S., et al. 2017, Nature, 541, 58
  * Chittidi et al. (2020) Chittidi, J. S., Simha, S., Mannings, A., et al. 2020, arXiv:2005.13158
  * Cordes & Lazio (2002) Cordes, J. M., & Lazio, T. J. W. 2002, arXiv e-prints, astro-ph/0207156
  * Dai & Xia (2020) Dai, J. P. & Xia, J. Q., 2020, arXiv: 2004.11276
  * Deng & Zhang (2014) Deng, W., & Zhang, B. 2014, ApJ, 783, L35
  * Dolag et al. (2015) Dolag, K., Gaensler, B. M., Beck, A. M., et al. 2015, MNRAS, 451, 4277
  * Faucher-Giguère et al. (2009) Faucher-Giguère, C.-A., Lidz, A., Zaldarriaga, M., et al. 2009, ApJ, 703, 1416
  * Fialkov & Loeb (2016) Fialkov, A., & Loeb, A. 2016, J. Cosmology Astropart. Phys, 2016, 004
  * Gao et al. (2014) Gao, H., Li, Z., & Zhang, B. 2014, ApJ, 788, 189
  * Inoue (2004) Inoue, S. 2004, MNRAS, 348, 999
  * Ioka (2003) Ioka, K. 2003, ApJ, 598, L79
  * Jaroszynski (2019) Jaroszynski, M. 2019, MNRAS, 484, 1637
  * Keane (2018) Keane, E. F. 2018, Nature Astronomy, 2, 865
  * Leung et al. (2020) Leung, C., Mena-Parra, J., Masui, K., et al. 2020, arXiv:2008.11738
  * Lewis (2008) Lewis, A. 2008, Phys. Rev. D, 78, 023002
  * Li et al. (2019) Li, Z., Gao, H., Wei, J.-J., et al. 2019, ApJ, 876, 146
  * Linder (2020) Linder, E. V. 2020, Phys. Rev. D, 101, 103019
  * Loeb & Furlanetto (2013) Loeb, A., & Furlanetto, S. R. 2013, The First Galaxies in the Universe
  * Lorimer et al. (2007) Lorimer, D. R., Bailes, M., McLaughlin, M. A., et al. 2007, Science, 318, 777
  * Macquart et al. (2020) Macquart, J.-P., Prochaska, J. X., McQuinn, M., et al. 2020, Nature, 581, 391
  * Marcote et al. (2020) Marcote, B., Nimmo, K., Hessels, J. W. T., et al. 2020, Nature, 577, 190
  * Marinacci et al. (2018) Marinacci, F., Vogelsberger, M., Pakmor, R., et al. 2018, MNRAS, 480, 5113
  * McQuinn (2014) McQuinn, M. 2014, ApJ, 780, L33
  * Muñoz et al. (2016) Muñoz, J. B., Kovetz, E. D., Dai, L., et al. 2016, Phys. Rev. Lett., 117, 091301
  * Naiman et al. (2018) Naiman, J. P., Pillepich, A., Springel, V., et al. 2018, MNRAS, 477, 1206
  * Nelson et al. (2018) Nelson, D., Pillepich, A., Springel, V., et al. 2018, MNRAS, 475, 624
  * Peebles (1968) Peebles, P. J. E. 1968, ApJ, 153, 1
  * Petroff et al. (2019) Petroff, E., Hessels, J. W. T., & Lorimer, D. R. 2019, A&A Rev., 27, 4
  * Pillepich et al. (2018) Pillepich, A., Nelson, D., Hernquist, L., et al. 2018, MNRAS, 475, 648
  * Planck Collaboration et al. (2016) Planck Collaboration, Adam, R., Aghanim, N., et al. 2016, A&A, 596, A108
  * Planck Collaboration et al. (2016) Planck Collaboration, Ade, P. A. R., Aghanim, N., et al. 2016, A&A, 594, A13
  * Pol et al. (2019) Pol, N., Lam, M. T., McLaughlin, M. A., et al. 2019, ApJ, 886, 135
  * Prochaska et al. (2019) Prochaska, J. X., Macquart, J.-P., McQuinn, M., et al. 2019, Science, 366, 231
  * Prochaska & Zheng (2019) Prochaska, J. X., & Zheng, Y. 2019, MNRAS, 485, 648
  * Ravi et al. (2019) Ravi, V., Catha, M., D’Addario, L., et al. 2019, Nature, 572, 352
  * Seager et al. (2000) Seager, S., Sasselov, D. D., & Scott, D. 2000, ApJS, 128, 407
  * Springel et al. (2018) Springel, V., Pakmor, R., Pillepich, A., et al. 2018, MNRAS, 475, 676
  * Vogelsberger et al. (2014) Vogelsberger, M., Genel, S., Springel, V., et al. 2014, Nature, 509, 177
  * Walters et al. (2018) Walters, A., Weltman, A., Gaensler, B. M., et al. 2018, ApJ, 856, 65
  * Wang et al. (2020) Wang, F. Y., Wang, Y. Y., Yang, Y.-P., et al. 2020, ApJ, 891, 72
  * Wang et al. (2016) Wang, J.-S., Yang, Y.-P., Wu, X.-F., et al. 2016, ApJ, 822, L7
  * Wang & Wang (2018) Wang, Y. K., & Wang, F. Y. 2018, A&A, 614, A50
  * Wei et al. (2019) Wei, J.-J., Li, Z., Gao, H., et al. 2019, J. Cosmology Astropart. Phys, 2019, 039
  * Wu et al. (2020) Wu, Q., Yu, H., & Wang, F.-Y. 2020, ApJ, 895, 33
  * Yamasaki & Totani (2020) Yamasaki, S., & Totani, T. 2020, ApJ, 888, 105
  * Yao et al. (2017) Yao, J. M., Manchester, R. N., & Wang, N. 2017, ApJ, 835, 29
  * Yu & Wang (2017) Yu, H., & Wang, F. Y. 2017, A&A, 606, A3
  * Zel’dovich et al. (1969) Zel’dovich, Y. B., Kurt, V. G., & Syunyaev, R. A. 1969, Soviet Journal of Experimental and Theoretical Physics, 28, 146
  * Zhang (2018) Zhang, B. 2018, ApJ, 867, L21
  * Zhang (2020) Zhang, B. 2020, ApJ, 890, L24
  * Zhang et al. (2020) Zhang, G.-Q., Yu, H.,He, J. H. & Wang, F.-Y. 2020, ApJ, 900, 170
  * Zhao et al. (2020) Zhao, Z. et al., 2020, arXiv: 2006.01450
  * Zhao et al. (2020) Zhao, Z. Y., Zhang, G. Q., Wang, Y. Y. & Wang, F. Y., 2020, arXiv: 2010.10702
  * Zheng et al. (2014) Zheng, Z., et al., 2014, ApJ, 797, 71
  * Zhou et al. (2014) Zhou, B., Li, X., Wang, T., et al. 2014, Phys. Rev. D, 89, 107303
  * Zhu & Feng (2020) Zhu, W. S. & Feng, L. L., arXiv: 2011.08519


# Bringing Structure into Summaries:
a Faceted Summarization Dataset for Long Scientific Documents

Rui Meng, Khushboo Thaker, Lei Zhang, Yue Dong, Xingdi Yuan, Tong Wang and
Daqing He  Rui Meng♣ Khushboo Thaker♣ Lei Zhang♣ Yue Dong♢
Xingdi Yuan♠ Tong Wang♠ Daqing He♣
♣School of Computing and Information, University of Pittsburgh
♢ Mila / McGill University
♠Microsoft Research, Montréal
{rui.meng, k.thaker, lez39<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
{eric.yuan<EMAIL_ADDRESS>
###### Abstract

Faceted summarization provides briefings of a document from different
perspectives. Readers can quickly comprehend the main points of a long
document with the help of a structured outline. However, little research has
been conducted on this subject, partially due to the lack of large-scale
faceted summarization datasets. In this study, we present FacetSum, a faceted
summarization benchmark built on Emerald journal articles, covering a diverse
range of domains. Different from traditional document-summary pairs, FacetSum
provides multiple summaries, each targeted at specific sections of a long
document, including the purpose, method, findings, and value. Analyses and
empirical results on our dataset reveal the importance of bringing structure
into summaries. We believe FacetSum will spur further advances in
summarization research and foster the development of NLP systems that can
leverage the structured information in both long texts and summaries.

##  1 Introduction

Text summarization is the task of condensing a long piece of text into a short
summary without losing salient information. Research has shown that a well-
structured summary can effectively facilitate comprehension Hartley et al.
(1996); Hartley and Sydes (1997). A case in point is the structured abstract,
which consists of multiple segments, each focusing on a specific facet of a
scientific publication Hartley (2014), such as background, method,
conclusions, etc. The structure therein can provide much additional clarity
for improved comprehension and has long been adopted by databases and
publishers such as MEDLINE and Emerald.

Figure 1: An example of the proposed FacetSum dataset. Each facet of the
structured abstract summarizes different sections of the paper.

Despite these evident benefits of structure, summaries are often framed as a
linear, structure-less sequence of sentences in the flourishing array of
summarization studies (Nallapati et al., 2017; See et al., 2017; Paulus et
al., 2018; Grusky et al., 2018; Narayan et al., 2018; Sharma et al., 2019; Lu
et al., 2020; Cachola et al., 2020). We postulate that a primary reason for
this absence of structure lies in the lack of a high-quality, large-scale
dataset with structured summaries. In fact, existing studies in faceted
summarization Huang et al. (2020); Tauchmann et al. (2018); Jaidka et al.
(2016); Contractor et al. (2012); Kim et al. (2011); Jaidka et al. (2018);
Stead et al. (2019) are often conducted with rather limited amount of data
that are grossly insufficient to meet today’s ever-growing model capacity.

We aim to address this issue by proposing the FacetSum dataset. It consists of
60,024 scientific articles collected from Emerald journals, each associated
with a _structured_ abstract that summarizes the article from distinct aspects
including purpose, method, findings, and value. Scale-wise, we empirically
show that the dataset is sufficient for training large-scale neural generation
models such as BART Lewis et al. (2020) for adequate generalization. In terms
of quality, each structured abstract in FacetSum is provided by the original
author(s) of the article, who are arguably in the best position to summarize
their own work. We also provide quantitative analyses and baseline
performances on the dataset with mainstream models in Sections 2 and 3.

##  2 FacetSum for Faceted Summarization

# documents
---
Train: 46,289 / Dev: 6,000 / Test: 6,000 / OA-Test: 2,243
# words in abstracts
| Full | Purpose | Method | Findings | Value
mean | 290.4 | 54.1 | 52.0 | 68.6 | 47.3
std |  $\pm$82.8 |  $\pm$28.4 |  $\pm$27.8 |  $\pm$32.4 |  $\pm$24.2
# words in paper sections
| Full | Intro. | Method | Result | Conc.
recall% | - | 84.3% | 67.0% | 72.4% | 79.0%
mean | 6,827 | 885 | 1,194 | 2,371 | 747
std |  $\pm$2,704 |  $\pm$557 |  $\pm$861 |  $\pm$1,466 |  $\pm$567

Table 1: Statistics of the FacetSum dataset.

The FacetSum dataset is sourced from journal articles published by Emerald
Publishing111The data has been licensed to researchers at subscribing
institutions to use (including data mining) for non-commercial purposes. See
detailed policies at https://www.emerald.com/ (Figure 1). Unlike many
publishers, Emerald imposes explicit requirements that authors summarize their
work from multiple aspects Emerald (2021): Purpose describes the motivation,
objective, and relevance of the research; Method enumerates specific measures
taken to reach the objective, such as experiment design, tools, methods,
protocols, and datasets used in the study; Findings present major results such
as answers to the research questions and confirmation of hypotheses; and Value
highlights the work’s value and originality222There are three optional facets
(about research, practical and social implications) that are missing from a
large number of articles and hence omitted in this study.. Together, these
facets give rise to a comprehensive and informative structure in the abstracts
of the Emerald articles, and by extension, to FacetSum’s unique ability to
support faceted summarization.

###  2.1 General Statistics

We collect 60,532 publications from Emerald Publishing spanning 25 domains.
Table 1 lists some descriptive statistics of the dataset. Since FacetSum is
sourced from journal articles, texts therein are naturally expected to be
longer compared to other formats of scientific publications. In addition,
although each facet is more succinct than the traditional, structure-less
abstracts, a full length abstract containing all facets can be considerably
longer. Empirically, we compare the source and the target lengths with some
existing summarization datasets in similar domains including CLPubSum (Collins
et al., 2017), PubMed (Cohan et al., 2018), ArXiv (Cohan et al., 2018),
SciSummNet (Yasunaga et al., 2019), and SciTldr (Cachola et al., 2020). On
average, the source length in FacetSum is 58.9% longer (6,827 vs 4,297), and
the target length is 37.0% longer (290.4 vs 212.0).

From a summarization perspective, these differences imply that FacetSum may
pose significantly increased modeling and computation challenges due to the
increased lengths in both the source and the target. Moreover, the wide range
of research domains (Figure 3, Appendix D) may also introduce much linguistic
diversity w.r.t. vocabulary, style, and discourse. Therefore, compared to
existing scientific publication datasets that only focus on specific academic
disciplines Cohan et al. (2018); Cachola et al. (2020), FacetSum can also be
used to assess a model’s robustness in domain shift and systematic
generalization.

To facilitate assessment of generalization, we reserve a dev and a test set
each consisting of 6,000 randomly sampled data points; the remaining data are
intended as the training set. We ensure that the domain distribution is
consistent across all three subsets. Besides, we intentionally leave out Open-
Access papers as another test set, to facilitate researchers who do not have
full Emerald access333 Both the split information of FacetSum and the code for
scraping and parsing the data are available at
https://github.com/hfthair/emerald_crawler.

Figure 2: Oracle sentence distribution over a paper. X-axis: 10,000 papers
sampled from FacetSum, sorted by full text length from long to short; y-axis:
normalized position in a paper. We provide each sub-figure’s density histogram
on their right.

###  2.2 Structural Alignment

In this section, we focus our analysis on one of the defining features of
FacetSum — its potential to support faceted summarization. Specifically, we
investigate how the abstract structure (i.e., facets) aligns with the article
structure. Given an abstract facet $A$ and its corresponding article $S$, we
quantify this alignment by:

$S_{A}=\\{\operatorname*{arg\,max}_{s_{i}\in
S}(\texttt{Rouge-1}(s_{i},a_{j})):a_{j}\in A\\}$ (1)

Semantically, $S_{A}$ consists of sentence indices in $S$ that best align with
each sentence in $A$.

Sentence-level Alignment We first plot the tuples $\\{(s_{i},i/|S|):i\in
S_{A}\\}$, where $s_{i}$ is the $i$-th sentence in $S$, and $|S|$ is the
number of sentences in $S$. Intuitively, the plot density around position
$i/|S|$ entails the degree of alignment between the facet $A$ and the article
$S$ at that position444We use the relative position $i/|S|$ so that all
positions are commensurate across multiple documents.. With 10,000 articles
randomly sampled from FacetSum, Figure 2 exhibits distinct differences in the
density distribution among the facets in FacetSum. For example, with $A=$
Purpose, resemblance is clearly skewed towards the beginning of the articles,
while Findings are mostly positioned towards the end; the Method distribution
is noticeably more uniform than the others. These patterns align well with
intuition, and are further exemplified by the accompanying density histograms.

Table 2: Scores of sentence aligning in Rouge-L.

Section-level Alignment We now demonstrate how different abstract facets align
with different sections in an article. Following conventional structure of
scientific publications Suppe (1998); Rosenfeldt et al. (2000), we first
classify sections into Introduction, Method, Result and Conclusion using
keyword matching in the section titles.555 To ensure close-to-perfect
precision, we choose keywords that are as specific and prototypical to each
section as possible (listed in Appendix A). The resulting recall is around
$0.7$, i.e. about 70% of sections can be correctly retrieved with the title-
keyword matching method. And we find 2,751 (out of 6,000) test samples that
all four sections are matched successfully. Though far from perfect, we
believe this size is sufficient for the significance of subsequent analyses.

Given a section $S^{i}\subseteq S$ and an abstract $A_{j}\subseteq A$, we
define the section-level alignment $g(S^{i},A_{j})$ as
$\texttt{Rouge-1}(\texttt{cat}(S^{i}_{A_{j}}),\texttt{cat}(A_{j}))$, where
$\texttt{cat}(\cdot)$ denotes sentences concatenation, and $S^{i}_{A_{j}}$ is
defined by Equation (1). Table 2 is populated by varying $A_{j}$ and $S^{i}$
across the rows and columns, respectively. Full denotes the full paper or
abstract (concatenation of all facets). We also include the concatenation of
introduction and conclusion (denoted I+C) as a possible value for $S^{i}$, due
to its demonstrated effectiveness as summaries in prior work Cachola et al.
(2020).

The larger numbers on the diagonal (in red) empirically confirm a strong
alignment between FacetSum facets and their sectional counterparts in
articles. We also observe a significant performance gap between using I+C and
the full paper as $S^{i}$. One possible reason is that the summaries in
FacetSum (particularly Method and Findings) may contain more detailed
information beyond introduction and conclusion. This suggests that for some
facets in FacetSum, simple tricks to condense full articles do not always
work; models need to instead comprehend and retrieve relevant texts from full
articles in a more sophisticated manner.

| Model | Source Text | Full | Purpose | Method | Findings | Value
---|---|---|---|---|---|---|---
FacetSum Test
Oracle | Greedy Extractive Nallapati et al. (2017) | corresponding | 60.39 | 44.66 | 41.00 | 46.44 | 38.10
Heuristic | Lead-K | corresponding | 36.78 | 17.83 | 15.29 | 15.92 | 16.08
Models | Tail-K | sections | 33.31 | 21.67 | 12.62 | 16.66 | 17.43
| SumBasic (Vanderwende et al., 2007) |  | 38.71 | 18.17 | 15.41 | 16.31 | 16.57
Unsupervised | LexRank (Erkan and Radev, 2004) | corresponding | 42.18 | 18.72 | 16.23 | 18.11 | 17.75
Models | LSA (Gong and Liu, 2001) | sections | 35.98 | 18.29 | 15.86 | 16.92 | 16.62
| TextRank (Mihalcea and Tarau, 2004) |  | 41.87 | 21.67 | 13.62 | 18.63 | 19.23
| HipoRank (Dong et al., 2020) |  | 42.89 | 22.73 | 15.20 | 18.38 | 19.68
| BART (Lewis et al., 2020) | I+C | 44.36 | 41.14 | 20.75 | 14.72 | 5.85
Supervised | BART-Facet | I+C | 47.09 | 43.47 | 29.07 | 30.97 | 28.90
Models | BART | full paper | 42.74 | 41.21 | 20.53 | 14.33 | 5.07
| BART-Facet | full paper | 45.76 | 42.55 | 28.07 | 28.98 | 28.70
FacetSum OA-Test
| BART | I+C | 44.97 | 43.51 | 26.73 | 11.79 | 0.31
| BART-Facet | I+C | 51.32 | 43.66 | 30.16 | 32.22 | 29.68

Table 3: Model performance on FacetSum (Rouge-L). See Table 6 and 7 in
Appendix E for full results. Bold text indicates the best scores on FacetSum
test split in each column.

##  3 Experiments and Results

We use FacetSum to benchmark a variety of summarization models from state-of-
the-art supervised models to unsupervised and heuristics-based models. We also
provide the scores of a sentence-level extractive oracle system Nallapati et
al. (2017). We report Rouge-L in this section and include Rouge-1/2 results in
Appendix E.

Unsupervised Models vs Heuristics We report performances of unsupervised and
heuristics summarization methods (see Table 3). Tailoring to the unique task
of generating summaries for a specific facet, we only use the section (defined
in Section 2.2) corresponding to a facet as model input. Evaluation is also
performed on the concatenation of all facets (column Full), which resembles
the traditional research abstract. Lead-K/Tail-K are two heuristic-based
models that extract the first/last $k$ sentences from the source text.

We observe that heuristic models do not perform well on Full, where the
unsupervised models can achieve decent performance. Nevertheless, all models
perform poorly on summarizing individual facets, and unsupervised models fail
to perform better than simple heuristics consistently. The inductive biases of
those models may not be good indicators of summary sentences on specific
facets. A possible reason is that they are good at locating overall important
sentences of a document, but they cannot differentiate sentences of each
facet, even we try to alleviate this by using the corresponding section as
input.

Supervised Models As for the supervised baseline, we adopt the BART model
(Lewis et al., 2020), which has recently achieved SOTA performance on
abstractive summarization tasks with scientific articles (Cachola et al.,
2020). We propose two training strategies for the BART model, adapting it to
handle the unique challenge of faceted summarization in FacetSum. In BART, we
train the model to generate the concatenation of all facets, joined by special
tokens that indicate the start of a specific facet (e.g., |PURPOSE| to
indicate the start of Purpose summary). During evaluation, the generated text
is split into multiple facets based on the special tokens, and each facet is
compared against the corresponding ground-truth summary. In BART-Facet, we
train the model to generate one specific facet given the source text and an
indicator specifies which facet to generate. Inspired by CATTS (Cachola et
al., 2020), we prepend section tags at the beginning of each training input to
generate summaries for a particular facet (see implementation details in
Appendix C).

Empirically, supervised models outperform unsupervised baselines by a large
margin (Table 3). Comparing between the two training strategies, BART-Facet
outperforms BART significantly. While BART performs comparably on Purpose,
performance decreases drastically for subsequent facets, possibly due to
current models’ inadequacy with long targets. Thus it can perform decently at
the beginning of generation ($\approx$40 on Purpose), where the dependency is
relatively easy-to-handle. However, the output quality degrades quickly
towards the end ($\approx$5 on Value).

With I+C as source text, both training strategies exhibit much better results
than using full paper. This is opposite to the observation in Table 2,
potentially due to the limitation of the current NLG systems, i.e., the length
of source text has crucial impacts to the model performance. With the much
extended positional embeddings in our models (10,000 tokens), we suspect some
other issues such as long term dependencies may lead to this discrepancy,
which warrants further investigation.

##  4 Related Work

We acknowledge several previous efforts towards faceted summarization. Prior
to our study, generating structured abstracts for scientific publications was
also discussed in Gidiotis and Tsoumakas (2019, 2020). The authors built a
structured abstract dataset PMC-SA, consisting of 712,911 biomedical articles
from PubMed Central, and they proposed to summarize a paper from a specific
facet by taking corresponding sections as inputs. Compared with their works,
FacetSum covers a wider range of academic fields and we provide in-depth
discussions on the structured abstract to justify its value as a novel NLP
challenge. Our research shares some resemblance to studies on abstract
sentence classification, whose goal is to classify abstract sentences into
several facets, instead of summarizing the full text. MEDLINE is commonly used
for this task Kim et al. (2011), so as the Emerald data Stead et al. (2019). A
recent study Huang et al. (2020) introduced a new dataset CODA, in which
10,966 abstracts are split into subsentences and labelled into five categories
by third-party annotators. However, we think scientific documents are
generally difficult to comprehend for people without specific training, thus
original authors are in the best position to summarize their own work. Faceted
summarization was also involved in CL-SciSumm 2016 Shared Task Jaidka et al.
(2018), where the faceted summary of a paper is defined as the citation
sentences in its successor studies, since new studies typically describe
previous work from different perspectives. However, this idea may not easily
scale up in the real world since many papers do not have enough number of
citations, especially for newly published ones.

##  5 Conclusion & Future Work

We introduce FacetSum to support the research of faceted summarization, which
targets summarizing scientific documents from multiple facets. We provide
extensive analyses and results to investigate the characteristics of FacetSum.
Our observations call for the development of models capable of handling very
long documents and outputting controlled text. Specifically, we will consider
exploring the following topics in future work: (1) incorporating methods for
long-document processing, such as reducing input length by extracting key
sentences (Pilault et al., 2020) or segments Zhao et al. (2020); (2) examining
the possibility of building a benchmark for systematic generalization Bahdanau
et al. (2018) with the domain categories; (3) automatically structuring
traditional abstracts Huang et al. (2020) with FacetSum.

## References

  * Bahdanau et al. (2018) Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. 2018.  Systematic generalization: What is required and can it be learned?  In _International Conference on Learning Representations_.
  * Belica (2021) Mišo Belica. 2021.  sumy: Automatic text summarizer.  https://github.com/miso-belica/sumy.
  * Cachola et al. (2020) Isabel Cachola, Kyle Lo, Arman Cohan, and Daniel S Weld. 2020.  Tldr: Extreme summarization of scientific documents.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings_ , pages 4766–4777.
  * Cohan et al. (2018) Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018.  A discourse-aware attention model for abstractive summarization of long documents.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 615–621.
  * Collins et al. (2017) Ed Collins, Isabelle Augenstein, and Sebastian Riedel. 2017.  A supervised approach to extractive summarisation of scientific papers.  In _Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)_ , pages 195–205, Vancouver, Canada. Association for Computational Linguistics.
  * Contractor et al. (2012) Danish Contractor, Yufan Guo, and Anna Korhonen. 2012.  Using argumentative zones for extractive summarization of scientific articles.  In _Proceedings of COLING 2012_ , pages 663–678.
  * Dong et al. (2020) Yue Dong, Andrei Romascanu, and Jackie CK Cheung. 2020.  Hiporank: Incorporating hierarchical and positional information into graph-based unsupervised long document extractive summarization.  _arXiv preprint arXiv:2005.00513_.
  * Emerald (2021) Emerald. 2021.  Writing an article abstract.  https://www.emeraldgrouppublishing.com/how-to/authoring-editing-reviewing/write-article-abstract.  [Online; accessed 26-January-2021].
  * Erkan and Radev (2004) Günes Erkan and Dragomir R Radev. 2004.  Lexrank: Graph-based lexical centrality as salience in text summarization.  _Journal of artificial intelligence research_ , 22:457–479.
  * Gidiotis and Tsoumakas (2019) Alexios Gidiotis and Grigorios Tsoumakas. 2019.  Structured summarization of academic publications.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , pages 636–645. Springer.
  * Gidiotis and Tsoumakas (2020) Alexios Gidiotis and Grigorios Tsoumakas. 2020.  A divide-and-conquer approach to the summarization of long documents.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 28:3029–3040.
  * Gong and Liu (2001) Yihong Gong and Xin Liu. 2001.  Generic text summarization using relevance measure and latent semantic analysis.  In _Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval_ , pages 19–25.
  * Grusky et al. (2018) Max Grusky, Mor Naaman, and Yoav Artzi. 2018.  Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 708–719.
  * Hartley (2014) James Hartley. 2014.  Current findings from research on structured abstracts: an update.  _Journal of the Medical Library Association: JMLA_ , 102(3):146.
  * Hartley and Sydes (1997) James Hartley and Matthew Sydes. 1997.  Are structured abstracts easier to read than traditional ones?  _Journal of Research in Reading_ , 20(2):122–136.
  * Hartley et al. (1996) James Hartley, Matthew Sydes, and Anthony Blurton. 1996.  Obtaining information accurately and quickly: are structured abstracts more efficient?  _Journal of information science_ , 22(5):349–356.
  * Honnibal et al. (2020) Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020.  spaCy: Industrial-strength Natural Language Processing in Python.
  * Huang et al. (2020) Ting-Hao Huang, Chieh-Yang Huang, Chien-Kuang Cornelia Ding, Yen-Chia Hsu, and C Lee Giles. 2020.  Coda-19: Using a non-expert crowd to annotate research aspects on 10,000+ abstracts in the covid-19 open research dataset.  In _Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020_.
  * Jaidka et al. (2016) Kokil Jaidka, Muthu Kumar Chandrasekaran, Sajal Rustagi, and Min-Yen Kan. 2016.  Overview of the cl-scisumm 2016 shared task.  In _Proceedings of the joint workshop on bibliometric-enhanced information retrieval and natural language processing for digital libraries (BIRNDL)_ , pages 93–102.
  * Jaidka et al. (2018) Kokil Jaidka, Muthu Kumar Chandrasekaran, Sajal Rustagi, and Min-Yen Kan. 2018.  Insights from cl-scisumm 2016: the faceted scientific document summarization shared task.  _International Journal on Digital Libraries_ , 19(2-3):163–171.
  * Kim et al. (2011) Su Nam Kim, David Martinez, Lawrence Cavedon, and Lars Yencken. 2011.  Automatic classification of sentences to support evidence based medicine.  In _BMC bioinformatics_ , volume 12, pages 1–10. BioMed Central.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880.
  * Lu et al. (2020) Yao Lu, Yue Dong, and Laurent Charlin. 2020.  Multi-xscience: A large-scale dataset for extreme multi-document summarization of scientific articles.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8068–8074.
  * Mihalcea and Tarau (2004) Rada Mihalcea and Paul Tarau. 2004.  TextRank: Bringing order into text.  In _Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing_ , pages 404–411, Barcelona, Spain. Association for Computational Linguistics.
  * Nallapati et al. (2017) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017.  Summarunner: a recurrent neural network based sequence model for extractive summarization of documents.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ , pages 3075–3081.
  * Narayan et al. (2018) Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  In _Proceedings of NAACL-HLT 2019: Demonstrations_.
  * Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018.  A deep reinforced model for abstractive summarization.  In _International Conference on Learning Representations_.
  * Pilault et al. (2020) Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020.  On extractive and abstractive neural document summarization with transformer language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9308–9319, Online. Association for Computational Linguistics.
  * Rosenfeldt et al. (2000) Franklin L Rosenfeldt, John T Dowling, Salvatore Pepe, and Meryl J Fullerton. 2000\.  How to write a paper for publication.  _Heart, Lung and Circulation_ , 9(2):82–87.
  * See et al. (2017) Abigail See, Peter J Liu, and Christopher D Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1073–1083.
  * Sharma et al. (2019) Eva Sharma, Chen Li, and Lu Wang. 2019.  Bigpatent: A large-scale dataset for abstractive and coherent summarization.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2204–2213.
  * Stead et al. (2019) Connor Stead, Stephen Smith, Peter Busch, and Savanid Vatanasakdakul. 2019.  Emerald 110k: a multidisciplinary dataset for abstract sentence classification.  In _Proceedings of the The 17th Annual Workshop of the Australasian Language Technology Association_ , pages 120–125.
  * Suppe (1998) Frederick Suppe. 1998.  The structure of a scientific paper.  _Philosophy of Science_ , 65(3):381–405.
  * Tauchmann et al. (2018) Christopher Tauchmann, Thomas Arnold, Andreas Hanselowski, Christian M Meyer, and Margot Mieskes. 2018.  Beyond generic summarization: A multi-faceted hierarchical summarization corpus of large heterogeneous data.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_.
  * Vanderwende et al. (2007) Lucy Vanderwende, Hisami Suzuki, Chris Brockett, and Ani Nenkova. 2007.  Beyond sumbasic: Task-focused summarization with sentence simplification and lexical expansion.  _Information Processing & Management_, 43(6):1606–1618.
  * Yasunaga et al. (2019) Michihiro Yasunaga, Jungo Kasai, Rui Zhang, Alexander R. Fabbri, Irene Li, Dan Friedman, and Dragomir R. Radev. 2019.  Scisummnet: A large annotated corpus and content-impact models for scientific paper summarization with citation networks.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):7386–7393.
  * Zhao et al. (2020) Yao Zhao, Mohammad Saleh, and Peter J Liu. 2020.  Seal: Segment-wise extractive-abstractive long-form text summarization.  _arXiv preprint arXiv:2006.10213_.

##  Appendix A Keyword List for Identifying Paper Sections

Category | Keyword
---|---
Introduction | intro, purpose
Method | design, method, approach
Result | result, find, discuss, analy
Conclusion | conclu, future
Table 4: Keywords for identifying paper sections used in Section 2.2.

##  Appendix B Most Frequent Words in Each Abstract Facet

Facet |  Verb |  Noun |  Adjective
---|---|---|---
Purpose |  aim |  paper |  social
|  examin |  purpos |  new
|  investig |  studi |  organiz
|  explor |  manag |  differ
|  develop |  research |  public
Method |  base |  studi |  structur
|  conduct |  data |  qualit
|  collect |  analysi |  differ
|  test |  model |  empir
|  develop |  paper |  social
Findings |  found |  result |  signific
|  indic |  studi |  posit
|  suggest |  manag |  social
|  provid |  effect |  differ
|  identifi |  relationship |  higher
Value |  provid |  studi |  new
|  contribut |  paper |  social
|  develop |  research |  differ
|  base |  manag |  empir
|  examin |  literatur |  import
Table 5: Top five frequent verbs/nouns/adjectives in each facet of structured
abstract. We preprocess the text with lowercasing, stemming and stopword
removal and extract part-of-speech tags using Spacy Honnibal et al. (2020).

##  Appendix C Implementation Details

To make BART take full text as input, we extend the positional embedding to
10,000 tokens. This was required to leverage long text of papers in FacetSum
with average length of 6000 words.

Experiments of unsupervised baselines are implemented with Sumy Belica (2021)
and official code of HipoRank. We tune the hyperparameters of HipoRank with
the validation set. The BART experiments are finetuned using Fairseq Ott et
al. (2019), with learning rate of $3e^{-5}$, batch size of 1, max tokens per
batch of 10,000 and update frequency of 4. We finetune all models for 20,000
steps with single NVIDIA Tesla V100 16GB and we report the results of the last
checkpoint. The small batch size is the consequence of the large input size.
For inference, we use beam size of 4 and maximum length of 500/200 tokens for
BART/BART-Facet respectively.

##  Appendix D Domains Covered by FacetSum

In Figure 3, we show the distribution of domain categories in FacetSum.

Figure 3: Data distribution of domain categories, sorted in descending order.

##  Appendix E Full Results

In this section, we provide additional experiment results. In Table 6, we show
the full results of the extractive oracle system (first row in Table 3). In
Table 7, we provide full results of all other models (heuristic models,
unsupervised models, and supervised models in Table 3).

R1/R2/RL | Full | Purpose | Method | Findings | Value
---|---|---|---|---|---
Fullbody | 64.92/33.75/60.39 | 57.35/30.24/49.42 | 53.30/26.40/45.58 | 59.30/33.25/52.42 | 53.39/26.84/45.55
ICbody | 58.82/28.42/54.17 | 53.60/27.13/45.73 | 43.13/17.08/35.64 | 52.03/25.90/44.86 | 48.97/22.84/41.09
Introbody | 53.32/22.96/48.59 | 52.51/26.48/44.66 | 41.27/16.05/34.03 | 44.67/17.49/37.10 | 44.65/17.80/36.47
Methodbody | 52.05/20.52/47.35 | 45.16/16.61/36.84 | 48.60/21.67/41.00 | 44.77/17.69/37.67 | 40.94/13.55/32.94
Resultbody | 56.85/23.79/51.97 | 47.90/18.07/38.96 | 42.31/14.46/34.41 | 53.71/26.32/46.44 | 44.93/16.91/36.66
Conclubody | 55.26/25.26/50.58 | 47.76/18.88/38.94 | 40.53/13.84/32.83 | 51.81/25.81/44.73 | 46.14/19.66/38.10

Table 6: Full results (Rouge-1/2/L) of the extractive oracle system Nallapati et al. (2017) on FacetSum. Bold text indicates the best scores in the lower four rows in each column. R1/R2/RL | Full | Purpose | Method | Findings | Value
---|---|---|---|---|---
FacetSum Test
Lead-K | 39.65/11.01/36.78 | 21.95/4.89/17.83 | 18.69/5.94/15.29 | 18.84/4.31/15.92 | 20.14/3.05/16.08
Tail-K | 35.90/10.96/33.31 | 25.48/7.23/21.67 | 14.88/2.64/12.62 | 19.25/4.41/16.66 | 20.90/4.71/17.43
SumBasic | 42.11/10.01/38.71 | 22.23/4.68/18.17 | 18.40/5.02/15.41 | 19.15/3.93/16.31 | 20.64/3.08/16.57
LexRank | 46.35/15.12/42.18 | 22.97/5.28/18.72 | 19.44/5.84/16.23 | 21.66/5.66/18.11 | 22.39/4.05/17.75
LSA | 39.84/9.59/35.98 | 22.47/4.91/18.29 | 19.10/5.58/15.86 | 20.29/4.59/16.92 | 20.96/3.31/16.62
TextRank | 46.90/16.04/41.87 | 28.29/9.39/21.67 | 17.55/4.32/13.62 | 23.90/7.17/18.63 | 25.99/7.07/19.23
HipoRank | 46.48/15.42/42.89 | 27.71/8.29/22.73 | 18.27/4.65/15.20 | 21.75/5.31/18.38 | 24.54/5.26/19.68
BART I+C | 47.21/19.59/44.36 | 46.61/27.10/41.14 | 23.85/7.98/20.75 | 16.84/5.34/14.72 | 7.21/1.93/5.85
BART-Facet I+C | 50.62/20.97/47.09 | 49.59/28.70/43.47 | 34.61/11.82/29.07 | 36.42/12.63/30.97 | 35.37/11.75/28.90
BART full body | 45.49/18.10/42.74 | 46.74/27.09/41.21 | 23.66/7.92/20.53 | 16.39/4.63/14.33 | 6.30/1.62/5.07
BART-Facet full body | 49.29/19.60/45.76 | 48.65/27.72/42.55 | 33.49/11.01/28.07 | 34.46/10.49/28.98 | 35.27/11.44/28.70
FacetSum OA-Test
BART I+C | 48.85/20.84/44.97 | 49.43/29.44/43.51 | 31.1/10.16/26.73 | 13.78/4.45/11.79 | 0.4/0.1/0.31
BART-Facet I+C | 48.31/22.63/51.32 | 49.59/28.69/43.66 | 35.82/12.84/30.16 | 37.46/14.02/32.22 | 35.9/12.75/29.68
Table 7: Full results (Rouge-1/2/L) of different models on FacetSum. Bold text
indicates the best scores on FacetSum test split in each column.

##  Appendix F Example of Outputs by BART and BART-Facet

In Table 8, we show an example of the generated faceted summaries by BART and
BART-Facet of the same paper, compared against the ground-truth faceted
abstract.

Facet |  Ground Truth |  BART |  BART-Facet
---|---|---|---
Purpose |  The purpose of this paper is to advance the information society research by examining and better understanding the impact of the adoption information and communication technologies (ICT) within households on improving sustainability. |  The purpose of this paper is to answer the question: Does the adoption of information and communication technologies (ICT) within households contribute to sustainability? For this purpose, the study focuses on addressing the following specific questions: What criteria clearly and deeply describe the ICT adoption within households? What are the criteria that clearly describe sustainability within households, and are there statistically significant correlations between ICT and sustainability? |  The purpose of this paper is to answer the question: Does the adoption of information and communication technologies (ICT) within households contribute to sustainability?
Method |  The survey questionnaire was used and data collected from 679 Polish households were statistically analyzed to understand the phenomena of ICT adoption and sustainability as well as identify correlations between them. |  A survey was carried out in Poland. The data were collected through a structured questionnaire and analyzed using structural equation modeling (SEM). |  The study is based on a quantitative research design. The data were collected through a questionnaire survey among the households in Poland. The survey was carried out in two phases. In the first phase, the ICT adoption within households was measured using a structured questionnaire, and in the second phase the data were analyzed using structural equation modeling (SEM) to test the hypothesized relationships.
Findings |  The research findings reveal that the ICT adoption is well described by the ICT outlay, information culture, ICT management and ICT quality, whereas sustainability is composed of ecological, economic, socio-cultural and political sustainability. Furthermore, the ICT quality, ICT management and information culture have a significant impact on sustainability, whereas the ICT outlay does not have such an impact. |  Results show that ICT is a key enabler of sustainability in households. The results also show that there are statistically significant correlation between the IIT adoption within the households and sustainability. |  The results show that the adoption of ICT within households is positively related to sustainability. The results also show that there are statistically significant correlations between the ICT adoption within households and sustainability.
Value |  The paper provides and verifies a new theoretical model of sustainable information society to depict various dimensions shaping the ICT adoption and their impact on different types of sustainability in the context of households. |  This study is the first to empirically investigate the impact of ICT on sustainability. The findings of this study will be complementary with findings concerning the contribution of IIT to sustainability in enterprises and allow for the advancement in the sustainable information society (SIS) research. |  This study contributes to the literature by providing a deeper understanding of the ICT adoption within households and the contribution of ICT to sustainability in transition economies, i.e. the former European Eastern Bloc countries.

Table 8: Outputs by BART and BART-Facet on different facets. Both models are
able to generate reasonable summaries given the specified facet. BART-Facet
provides more information of Method and less errors than BART (e.g. “IIT” is a
typo of “ICT”). However both models tend to directly copy text from the
source, for example both outputs of Purpose can be found in the introduction
of the paper.